DevOps&SRE Library

Lessons from a Rollback Gameday

Insights and best practices from a real-world rollback gameday

https://medium.com/expedia-group-tech/lessons-from-a-rollback-gameday-4d05cf1c9524

Automating Tooling Upgrades with Updatecli: A Scalable Solution for Platform Teams https://medium.com/sequra-tech/automating-tooling-upgrades-with-updatecli-a-scalable-solution-for-platform-teams-c9f599e301dc

Yoke is really cool

With Yoke, you write your infrastructure definitions in Go or Rust, compile it to WebAssembly, and then you take input and output Kubernetes manifests that get applied to the cluster.

https://xeiaso.net/blog/2025/yoke-k8s

Our Journey to GitOps: Migrating to ArgoCD with Zero Downtime https://medium.com/safetycultureengineering/our-journey-to-gitops-migrating-to-argocd-with-zero-downtime-932d0eefbe0d

rustfs

RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. Along with MinIO, it shares a range of advantages such as simplicity, S3 compatibility, open-source nature, support for data lakes, AI, and big data. Furthermore, it has a better and more user-friendly open-source license in comparison to other storage systems, being constructed under the Apache license. As Rust serves as its foundation, RustFS provides faster speed and safer distributed features for high-performance object storage.

https://github.com/rustfs/rustfs

Three Mighty Alerts Supporting Hugging Face’s Production Infrastructure

The Infrastructure team at Hugging Face is excited to share a behind-the-scenes look at the inner workings of Hugging Face's production infrastructure, which we’ve had the privilege of helping to build and maintain. Our team's dedication to designing and implementing a robust monitoring and alerting system has been instrumental in ensuring the stability and scalability of our platforms. We’re constantly reminded of the impact that our alerts have on our ability to identify and respond to potential issues before they become major incidents. In this blog post, we’ll dive into the details of three mighty alerts that play their unique role in supporting our production infrastructure, and explore how they've helped us maintain the high level of performance and uptime that our community relies on.

https://huggingface.co/blog/infrastructure-alerting

Seventh-generation server hardware at Dropbox: our most efficient and capable architecture yet

Fourteen years ago, Dropbox took its first steps toward building its own hardware infrastructure—and as our product and user base has grown, so has our infrastructure. What started with just a handful of servers has evolved into one of the largest custom-built storage systems in the world. We've scaled from a few dozen machines to tens of thousands of servers with millions of drives. That evolution didn’t happen by accident. It took years of iteration, close collaboration with suppliers, and a product-first mindset that treated infrastructure as a strategic advantage. Now we’re excited to share what’s next: the launch of our seventh-generation hardware platform, now featuring Crush, Dexter, and Sonic for our traditional compute, database, and storage workloads, and our newest GPU tiers, Gumby and Godzilla. To make this leap possible, we dramatically increased storage bandwidth, effectively doubled our available rack power, and introduced a next-gen storage chassis designed to even further minimize vibration and heat. This generation represents our most efficient, capable, and scalable architecture yet—and it’ll help us as we continue to build and scale helpful AI products like Dropbox Dash. Below, we’ll walk you through how we designed the latest version of our server hardware as well as key lessons we’ll carry into generations to come.

https://dropbox.tech/infrastructure/seventh-generation-server-hardware

OpenAI: Scaling PostgreSQL to the Next Level

At the PGConf.dev 2025 Global Developer Conference, Bohan Zhang from OpenAI shared OpenAI’s best practices with PostgreSQL, offering a glimpse into the database usage of one of the most prominent unicorn company.

https://www.pixelstech.net/article/1747708863-openai%3a-scaling-postgresql-to-the-next-level

📈Ваши ORM-запросы на SQLAlchemy тормозят и создают N+1-проблемы? Хотите понимать, как управлять подгрузкой данных, чтобы ускорить приложение на Python? 22 июля в 20:00 МСК OTUS проведёт открытый вебинар «Подгрузка связей в SQLAlchemy». Мы подробно разберём методы joinedload и selectinload, сравним их с lazy-подгрузкой и покажем, как выбор стратегии влияет на производительность запросов и нагрузку на базу данных. Вы узнаете, когда использовать каждую технику подгрузки, получите готовые рецепты для реальных задач и сможете избежать «тормозящих» запросов в продакшене. 🧑🏻‍💻Присоединяйтесь к уроку и получите скидку на полный курс «Python Developer. Professional»: https://vk.cc/cNqxCk Реклама. ООО «Отус онлайн-образование», ОГРН 1177746618576, www.otus.ru, erid: 2VtzquxMGtP

Postgres query plan visualization tools https://www.pgmustard.com/blog/postgres-query-plan-visualization-tools

Fix a top cause of slow queries in PostgreSQL (no slow query log needed) https://render.com/blog/postgresql-top-cause-slow-queries

Реклама. ООО «Аксель Про», ОГРН 1237700125750, erid: 2Vtzqw3xT8T 🚀 Шерлок прокачивает управление ИБ-дефектами! 24 июля в 11:00 мск приглашаем на вебинар по новому релизу «Шерлока» — AppSec-платформы, которая оптимизирует процессы безопасной разработки. На вебинаре расскажем: 🐾 О возможностях Шерлока для автоматизации 🐾 О новом функционале платформы 🐾 Проведем живое демо и ответим на все ваши вопросы 📌 Успейте зарегистрироваться! 👇 👉 РЕГИСТРАЦИЯ НА ВЕБИНАР #AppSec #DevSecOps #Кибербезопасность #AxelPRO

Don't Do This

A short list of common mistakes.

https://wiki.postgresql.org/wiki/Don%27t_Do_This

Life Altering Postgresql Patterns

There is a set of things that you can do when working with a Postgres database which I have found made my and my coworker's lives much more pleasant. Each one is by itself small, but in aggregate have a noticeable effect.

https://mccue.dev/pages/3-11-25-life-altering-postgresql-patterns

😱⚙️ Вас когда-нибудь пугала рутина развёртывания и тестирования приложений вручную? Курс «DevOps практики и инструменты» от OTUS — ваш путь от теории к практике: знакомство с IaC, CI/CD, системами конфигурации, хранением артефактов и observability (monitoring, logging, tracing). Программа создана практикующими экспертами, материалы регулярно обновляются, а все лекции проходят вживую. 🎁🚀 Пройдите короткое тестирование, чтобы получить доступ к записи вебинара «Организуем CD с помощью Ansible и GitLab CI». Вы увидите, как автоматизировать доставку и развёртывание, структурировать pipeline и защитить чувствительные данные. 👉 Готовы повысить свою ценность на рынке и сэкономить время на рутинных задачах? Пройдите тест прямо сейчас: https://vk.cc/cNIW1y Реклама. ООО «Отус онлайн-образование», ОГРН 1177746618576, www.otus.ru, erid: 2Vtzqv9wpEs

Mastering Postgres Replication Slots: Preventing WAL Bloat and Other Production Issues https://www.morling.dev/blog/mastering-postgres-replication-slots

When SIGTERM Does Nothing: A Postgres Mystery

The ClickPipes team had encountered a bug with logical replication slot creation on Postgres read replicas—specifically, an issue where a query that was already taking hours rather than the few seconds it usually took couldn’t be terminated by any of the usual methods in Postgres, causing customer frustration and risking the stability of production databases. In this blog post, I’ll walk through how I investigated the problem and ultimately discovered it was due to a Postgres bug. We’ll also share how we fixed it and our experience working with the Postgres community.

https://clickhouse.com/blog/sigterm-postgres-mystery

tfautomv

Generate Terraform moved blocks automatically for painless refactoring

https://github.com/busser/tfautomv

Все, что вы хотели знать про Kubernetes on Bare Metal Selectel проведет вебинар, где расскажут: - как перенести production-нагрузку в Managed Kubernetes на выделенных серверах, - как сократить расходы на 35% и повысить производительность сервисов. 🧑‍💻 Будет интересно DevOps-инженерам, системным администраторам и техлидам. Вебинар бесплатный. Подробности и регистрация по ссылке: https://slc.tl/dctv6   Чтобы не пропустить вебинар и узнавать о других мероприятиях, воркшопах и бесплатных курсах Selectel, подписывайтесь на @selectel_events Реклама, АО «Селектел», ИНН: 7810962785, ERID: 2VtzqwUvRSF

SLA vs SLO

Demystifying the most common misconception in Service Level jargon

https://blog.alexewerlof.com/p/sla-vs-slo