DevOps&SRE Library

How do you find the root cause of a configuration management failure when you have a peak of hundreds of changes in 15 minutes on thousands of servers? That was the challenge we faced as we built the infrastructure to reduce release delays due to failures of Salt, a configuration management tool. (We eventually reduced such failures on the edge by over 5%, as we’ll explain below.) We’ll explore the fundamentals of Salt, and how it is used at Cloudflare. We then describe the common failure modes and how they delay our ability to release valuable changes to serve our customers. By first solving an architectural problem, we provided the foundation for self-service mechanisms to find the root cause of Salt failures on servers, datacenters and groups of datacenters. This system is able to correlate failures with git commits, external service failures and ad hoc releases. The result of this has been a reduction in the duration of software release delays, and an overall reduction in toilsome, repetitive triage for SRE. To start, we will go into the basics of the Cloudflare network and how Salt operates within it. And then we’ll get to how we solved the challenge akin to finding a grain of sand in a heap of Salt.

PHP 8.5 has now been officially released, and developers naturally want to know what kind of performance improvements they can expect across popular CMSs and frameworks. To find out, we benchmarked 13 widely used CMSs and frameworks, including WordPress, WooCommerce, Drupal, Joomla, Laravel, Symfony and CodeIgniter, on PHP 8.2, 8.3, 8.4, and 8.5 under identical conditions. WordPress was also tested on PHP 7.4, since a notable share of sites still run on that version. Our intention is to provide a clear, practical look at how performance shifts across recent PHP releases and what you can expect when upgrading.

Enroll inspects a Debian-like or RedHat-like system, harvests the state that matters, and generates Ansible roles/playbooks so you can bring snowflakes under management fast.

ch-vmm is a Kubernetes add-on for running Cloud Hypervisor virtual machines. By using Cloud Hypervisor as the underlying hypervisor, ch-vmm enables a lightweight and secure way to run fully virtualized workloads in a canonical Kubernetes cluster.

The Altinity Kubernetes Operator for ClickHouse creates, configures and manages ClickHouse clusters running on Kubernetes.

Kexa is an open-source compliance management tool that simplifies security and compliance across multiple cloud platforms including Azure, Google Cloud, AWS, and more.

Shipwright is an extensible framework for building container images on Kubernetes.

Real-Time, Client-Side Load Balancing for Internal and Ingress Traffic in Kubernetes

At the beginning of this year, we (Beekeeper by LumApps Engineering) decided to adopt Karpenter for our EKS (Kubernetes/K8s) workloads, replacing our previous node autoscaling setup that used cluster-autoscaler with a managed autoscaling group (ASG). We made this decision before the release and hype of EKS Auto Mode, which is why we chose to implement a self-managed Karpenter solution.

Take a deep technical dive into GitLab.com's deployment pipeline, including progressive rollouts, Canary strategies, database migrations, and multiversion compatibility.

Pulse is a modern, unified dashboard for monitoring your infrastructure across Proxmox, Docker, and Kubernetes. It consolidates metrics, alerts, and AI-powered insights from all your systems into a single, beautiful interface.

The open-source alternative to Mailchimp, Brevo, Mailjet, Listmonk, Mailerlite, and Klaviyo, Loop.so, etc. Notifuse is a modern, self-hosted emailing platform that allows you to send newsletters and transactional emails at a fraction of the cost. Built with Go and React, it provides enterprise-grade features with the flexibility of open-source software.

Databasus is a free, open source and self-hosted tool to backup databases. Make backups with different storages (S3, Google Drive, FTP, etc.) and notifications about progress (Slack, Discord, Telegram, etc.).

Ente is a service that provides a fully open source, end-to-end encrypted platform for you to store your data in the cloud without needing to trust the service provider. On top of this platform, we have built two apps so far: Ente Photos (an alternative to Apple and Google Photos) and Ente Auth (a 2FA alternative to the deprecated Authy).

Modern Docker Management, Designed for Everyone

The pgEdge Postgres Model Context Protocol (MCP) server enables SQL queries against PostgreSQL databases through MCP-compatible clients like Claude Desktop. The Natural Language Agent provides supporting functionality that allows you to use natural language to form SQL queries.