DevOps&SRE Library

What to do when Pods are Partying TOO Hard For a Node https://medium.com/@jeremycastle/what-to-do-when-pods-are-partying-too-hard-for-a-node-953eb5de9f21

YAML templating was a mistake

Modern Kubernetes deployment methodologies have grown increasingly complex, layering abstraction upon abstraction in pursuit of flexibility. This article challenges that trajectory by examining how fundamental Unix tools combined with Makefiles can provide a more transparent and maintainable alternative to popular solutions like Helm and Kustomize.

https://dev.to/avkr/replace-helm-with-kiss-456a

ChatOps fatigue: how to create alerts that matter

In today's workplace, communication tools like Slack or Microsoft Teams are essential for staying connected at work. However, as orchestration and automation needs increase, so does the volume of notifications flooding these channels. What’s meant to streamline work can quickly become overwhelming. We call it "ChatOps fatigue" - when teams get so many alerts, they start tuning them out.

https://www.tines.com/blog/chatops-fatigue-how-to-create-alerts-that-matter

💀 Как работают процессы в Linux и что такое процессы-зомби? 👉 На открытом уроке «Процессы в Linux: от демонов до зомби» 29 июля в 20:00 МСК мы разберём жизненный цикл процессов, отличия демонов от обычных процессов, а также расскажем о зомби и сиротах. Вы узнаете, как управлять процессами и диагностировать их с помощью мощных утилит. 💪 Понимание того, как работают процессы в Linux, — это ключевая компетенция для системного администратора. Научитесь отслеживать состояние процессов, корректно завершать или перезапускать их, а также избегать накопления зомби в системе. 🎁 Присоединяйтесь к вебинару и получите специальное предложение на курс «Administrator Linux. Professional». 👉 Для участия в вебинаре зарегистрируйтесь https://vk.cc/cNVIAE Реклама. ООО «Отус онлайн-образование», ОГРН 1177746618576, erid: 2VtzqxAyhix

Speeding Up My ZSH Shell

Super quick one I want to document here! I got myself on a side quest, again! No biggie, my ZSH shell was taking ages to load. When I say ages, more like 5+ seconds every time I opened a new terminal, that sort of thing can add up. This is just something I’ve lived with over the years, nothing has prompted this other than me wondering why it’s slow, then searching for how to profile it.

https://scottspence.com/posts/speeding-up-my-zsh-shell

Canary Deployments Using Argo Rollouts and Istio Service-mesh https://dev.to/ezejioforog/mastering-canary-deployments-zero-downtime-integration-with-argo-rollouts-and-istio-44mn

⚡️Хаос с зависимостями может стать угрозой безопасности вашего проекта. Присоединяйтесь к открытому уроку «Страх и ненависть при работе с зависимостями (SCA)» 29 июля в 20:00 МСК и научитесь контролировать зависимости без уязвимостей. Вы узнаете: - Какие зависимости проходят проверки и как это влияет на инфраструктуру. - Как правильно хранить, обновлять и обогащать список компонентов. - Методы контроля целостности и версионности зависимостей. Урок станет отличной подготовкой к курсу «Внедрение и работа в DevSecOps», все участники получат скидку на обучение. Научитесь управлять зависимостями эффективно и безопасно. 👉Записывайтесь на вебинар, получайте скидку и готовьтесь к углубленному изучению DevSecOps с OTUS: https://vk.cc/cNVFoo Реклама. ООО «Отус онлайн-образование», ОГРН 1177746618576, www.otus.ru, erid: 2VtzqxJBYoA

Cloud-Native Secret Management: OIDC in K8s Explained

External Secrets is the de-facto choice for secrets management in Kubernetes clusters. It simplifies the task of the administrator(s) of the cluster, ensuring only the secrets that are explicitly defined are present and accessible. It comes with many great features but most important than all is its integration with major cloud providers. In this blog post you will learn how to deploy it without hard-coded credentials and using only the power of OpenID Connect for trust relationship between services.

https://developer-friendly.blog/blog/2025/03/24/cloud-native-secret-management-oidc-in-k8s-explained/

Upgrading Stateful Kubernetes Clusters with near-zero downtime

At Freshworks, we regularly perform blue-green migrations to upgrade our EKS clusters and implement Redis-related changes with minimal disruption. In this article, we’ll walk through how we migrate approximately 900 Redis endpoints — spanning one staging region and five production regions, each with 4–5 EKS clusters — while ensuring high availability for our stateful Redis workloads. Our mission was clear: complete the migration with minimal disruption to our services while ensuring data consistency. Here’s how we tackled this complex engineering challenge and achieved near-zero downtime migrations at scale.

https://medium.com/freshworks-engineering-blog/fast-k8s-upgrades-9cb60be7f93e

🚀💪 Как администратору Linux выйти на уровень Middle+? 👉 Приобрести необходимые навыки под руководством топовых экспертов из ведущих российских и международных компаний на онлайн-курсе «Administrator Linux. Professional» от OTUS. ⚠️ Программа идеально подойдет для системных администраторов Linux и Windows, DevOps-инженеров и SRE, Fullstack и Backend-разработчиков, сетевых и инженеров по нагрузочному тестированию, а также для специалистов по ИБ. 💪 Вы на профессиональном уровне изучите подбор конфигураций, управление процессами, обеспечение безопасности, развертывание, настройку и обслуживание сетей, что позволит вам претендовать на вакантные должности в крупных компаниях. 🎁 За успешное прохождение вступительного тестирования на странице курса вам откроется доступ к записям вебинаров от экспертов курса. 👉 Пройти вступительный тест https://vk.cc/cNUO7f Реклама. ООО «Отус онлайн-образование», ОГРН 1177746618576, erid: 2Vtzquu3UE4

Securing Kubernetes API Server Health Checks Without Anonymous Access https://dev.to/azalio/securing-kubernetes-api-server-health-checks-without-anonymous-access-31f9

Graceful External Termination: Handling Pod Deletions in Kubernetes Data Ingestion and Streaming Jobs https://medium.com/ibm-data-ai/graceful-external-termination-handling-pod-deletions-in-kubernetes-data-ingestion-and-streaming-df1b2cd8d727

⚙️GitlabCI + ArgoCD — сборка и доставка приложений, не покидая кластер В этом вебинаре мы покажем, как выстраивать современный процесс CI/CD, полностью внутри Kubernetes-кластера — от сборки кода до доставки в продакшн, не выходя за его пределы. Используя связку GitlabCI и ArgoCD, вы научитесь создавать автономную и управляемую платформу, где всё — от пайплайнов до деплоя — происходит под полным контролем и в полной безопасности. На вебинаре вы узнаете: - Как развернуть Gitlab Runners и ArgoCD внутри кластера и настроить их взаимодействие. - Настройка пайплайнов в GitlabCI: сборка, тестирование, упаковка образов. - Как задействовать ArgoCD для GitOps-доставки: автообновления из Git-репозитория. - Как добиться полной автономии CI/CD без выхода за границы инфраструктуры. Вебинар проходит в рамках курса "Инфраструктурная платформа на основе Kubernetes" 👉 Регистрация и подробности о курсе "Инфраструктурная платформа на основе Kubernetes" https://vk.cc/cNSFPx Реклама. ООО «Отус онлайн-образование», ОГРН 1177746618576, erid: 2Vtzqufnc9n

Lessons from a Rollback Gameday

Insights and best practices from a real-world rollback gameday

https://medium.com/expedia-group-tech/lessons-from-a-rollback-gameday-4d05cf1c9524

Automating Tooling Upgrades with Updatecli: A Scalable Solution for Platform Teams https://medium.com/sequra-tech/automating-tooling-upgrades-with-updatecli-a-scalable-solution-for-platform-teams-c9f599e301dc

Yoke is really cool

With Yoke, you write your infrastructure definitions in Go or Rust, compile it to WebAssembly, and then you take input and output Kubernetes manifests that get applied to the cluster.

https://xeiaso.net/blog/2025/yoke-k8s

Our Journey to GitOps: Migrating to ArgoCD with Zero Downtime https://medium.com/safetycultureengineering/our-journey-to-gitops-migrating-to-argocd-with-zero-downtime-932d0eefbe0d

rustfs

RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. Along with MinIO, it shares a range of advantages such as simplicity, S3 compatibility, open-source nature, support for data lakes, AI, and big data. Furthermore, it has a better and more user-friendly open-source license in comparison to other storage systems, being constructed under the Apache license. As Rust serves as its foundation, RustFS provides faster speed and safer distributed features for high-performance object storage.

https://github.com/rustfs/rustfs

Three Mighty Alerts Supporting Hugging Face’s Production Infrastructure

The Infrastructure team at Hugging Face is excited to share a behind-the-scenes look at the inner workings of Hugging Face's production infrastructure, which we’ve had the privilege of helping to build and maintain. Our team's dedication to designing and implementing a robust monitoring and alerting system has been instrumental in ensuring the stability and scalability of our platforms. We’re constantly reminded of the impact that our alerts have on our ability to identify and respond to potential issues before they become major incidents. In this blog post, we’ll dive into the details of three mighty alerts that play their unique role in supporting our production infrastructure, and explore how they've helped us maintain the high level of performance and uptime that our community relies on.

https://huggingface.co/blog/infrastructure-alerting

Seventh-generation server hardware at Dropbox: our most efficient and capable architecture yet

Fourteen years ago, Dropbox took its first steps toward building its own hardware infrastructure—and as our product and user base has grown, so has our infrastructure. What started with just a handful of servers has evolved into one of the largest custom-built storage systems in the world. We've scaled from a few dozen machines to tens of thousands of servers with millions of drives. That evolution didn’t happen by accident. It took years of iteration, close collaboration with suppliers, and a product-first mindset that treated infrastructure as a strategic advantage. Now we’re excited to share what’s next: the launch of our seventh-generation hardware platform, now featuring Crush, Dexter, and Sonic for our traditional compute, database, and storage workloads, and our newest GPU tiers, Gumby and Godzilla. To make this leap possible, we dramatically increased storage bandwidth, effectively doubled our available rack power, and introduced a next-gen storage chassis designed to even further minimize vibration and heat. This generation represents our most efficient, capable, and scalable architecture yet—and it’ll help us as we continue to build and scale helpful AI products like Dropbox Dash. Below, we’ll walk you through how we designed the latest version of our server hardware as well as key lessons we’ll carry into generations to come.

https://dropbox.tech/infrastructure/seventh-generation-server-hardware