DevOps & SRE notes

⚡️ LocalStack archived its GitHub repo — what happened and what it means On March 23, 2026, LocalStack archived localstack/localstack on GitHub (read-only) and consolidated everything into a single Docker image that requires an auth token — including in CI. What changed: - docker pull localstack/localstack:latest without LOCALSTACK_AUTH_TOKEN → your pipeline breaks - Free "Hobby" plan exists but requires account creation and is non-commercial only - Paid plans start at $39/mo - CI needs a dedicated CI Auth Token stored in secrets Your options: - Pin to an older tag (e.g. 4.12) — works short-term, but you accumulate parity drift and unpatched CVEs - Create a free account — enough for individual non-commercial dev - Pay — if LocalStack is embedded in team CI For open-source projects: LocalStack launched a separate program offering free Ultimate tier licenses (100+ AWS services, Cloud Pods, IAM enforcement) to eligible OSS projects with OSI-approved licenses. https://blog.localstack.cloud/introducing-localstack-for-open-source/

Delivers efficient, stable, and secure data distribution and acceleration powered by P2P technology, with an optional content‑addressable filesystem that accelerates OCI container launch. https://github.com/dragonflyoss/dragonfly

Good introduction for k8s Gateway API https://www.youtube.com/watch?v=HLXyrQT8zV0

Trivy has been hacked, again. The malicious payload is designed to execute within GitHub Actions runners, targeting sensitive data in CI/CD environments. Observed behavior includes dumping runner process memory to extract secrets, harvesting SSH keys, and exfiltrating credentials for AWS, GCP, and Azure, as well as Kubernetes service account tokens. https://socket.dev/blog/trivy-under-attack-again-github-actions-compromise

Container and file artifact promotion tooling for the Kubernetes project https://github.com/kubernetes-sigs/promo-tools

Short-lived public TLS certificates are our future, with a 46-day maximum validity by 2029. https://knowledge.digicert.com/alerts/public-tls-certificates-199-day-validity

Looking for a hosting platform to practice with Linux, Kubernetes, etc.? Register using my referral link on DigitalOcean and get $200 in credit for 60 days. By registering through my referral link, you also support this Telegram channel. 👉 Register

🔥 You can now create Amazon S3 general purpose buckets in your own reserved namespace, eliminating the need to find globally unique bucket names! The announce - https://aws.amazon.com/about-aws/whats-new/2026/03/amazon-s3-account-regional-namespaces/

AI Group now in K8s https://kubernetes.io/blog/2026/03/09/announcing-ai-gateway-wg/

Last week, I switched the default search in zsh (Ctrl+R) to fzf, and it’s working out pretty well. https://github.com/junegunn/fzf

A utility for generating Mermaid diagrams from Terraform configurations https://github.com/RoseSecurity/Terramaid

As announced November 2025, Kubernetes will retire Ingress-NGINX in March 2026. Despite its widespread usage, Ingress-NGINX is full of surprising defaults and side effects that are probably present in your cluster today. This blog highlights these behaviors so that you can migrate away safely and make a conscious decision about which behaviors to keep. This post also compares Ingress-NGINX with Gateway API and shows you how to preserve Ingress-NGINX behavior in Gateway API. The recurring risk pattern in every section is the same: a seemingly correct translation can still cause outages if it does not consider Ingress-NGINX's quirks. https://kubernetes.io/blog/2026/02/27/ingress-nginx-before-you-migrate/

A good starting point for finding a Helm chart that is not officially provided by the vendor is the Community Helm Chart Repository. https://github.com/trueforge-org/truecharts

K8sQuest — A local, hands-on Kubernetes learning game with real-world troubleshooting challenges. Practice Pods, Deployments, Services, networking, storage, and debugging using kubectl on a local cluster (kind/k3d). No cloud required. https://github.com/Manoj-engineer/k8squest

Understanding how many pods your infrastructure can actually support is crucial for reliability. This overview breaks down the nuances of Kubernetes cluster capacity and resource allocation. https://dnastacio.medium.com/kubernetes-cluster-capacity-d96d0d82b380

AWS Cost Optimization Game Day — a hands-on, interactive session focused on improving cloud efficiency and reducing costs in real-world scenarios. You’ll collaborate, analyze architectures, uncover cost-saving opportunities, and compete in a fun, gamified environment. Ready to optimize and win? Let’s play smart with AWS! When: Wednesday, Mar 11 · 4:30 PM to 7:30 PM GMT+2 Language: English Registration link is here

The second part of bench https://github.com/howardjohn/gateway-api-bench/blob/main/README-v2.md

Amazon's response https://www.aboutamazon.com/news/aws/aws-service-outage-ai-bot-kiro

The original article under the pay wall TL;DR Amazon service was taken down by AI coding bot Amazon’s cloud unit has suffered at least two outages due to errors involving its own AI tools, leading some employees to raise doubts about the US tech giant’s push to roll out these coding assistants. Amazon Web Services experienced a 13-hour interruption to one system used by its customers in mid-December after engineers allowed its Kiro AI coding tool to make certain changes, according to four people familiar with the matter. The people said the agentic tool, which can take autonomous actions on behalf of users, determined that the best course of action was to “delete and recreate the environment”. Amazon posted an internal postmortem about the “outage” of the AWS system, which lets customers explore the costs of its services. Multiple Amazon employees told the FT that this was the second occasion in recent months in which one of the group’s AI tools had been at the centre of a service disruption. “We’ve already seen at least two production outages [in the past few months],” said one senior AWS employee. “The engineers let the AI [agent] resolve an issue without intervention. The outages were small but entirely foreseeable.” AWS, which accounts for 60 per cent of Amazon’s operating profits, is seeking to build and deploy AI tools including “agents” capable of taking actions independently based on human instructions. Like many Big Tech companies, it is seeking to sell this technology to outside customers. The incidents highlight the risk that these nascent AI tools can misbehave and cause disruptions. Amazon said it was a “coincidence that AI tools were involved” and that “the same issue could occur with any developer tool or manual action”. “In both instances, this was user error, not AI error,” Amazon said, adding that it had not seen evidence that mistakes were more common with AI tools. The company said the incident in December was an “extremely limited event” affecting only a single service in parts of mainland China. Amazon added that the second incident did not have an impact on a “customer facing AWS service”. Neither disruption was anywhere near as severe as a 15-hour AWS outage in October 2025 that forced multiple customers’ apps and websites offline — including OpenAI’s ChatGPT. Employees said the group’s AI tools were treated as an extension of an operator and given the same permissions. In these two cases, the engineers involved did not require a second person’s approval before making changes, as would normally be the case. Amazon said that by default its Kiro tool “requests authorisation before taking any action” but said the engineer involved in the December incident had “broader permissions than expected — a user access control issue, not an AI autonomy issue”. AWS launched Kiro in July. It said the coding assistant would advance beyond “vibe coding” — which allows users to quickly build applications — to instead write code based on a set of specifications. The group had earlier relied on its Amazon Q Developer product, an AI-enabled chatbot, to help engineers write code. This was involved in the earlier outage, three of the employees said. Some Amazon employees said they were still sceptical of AI tools’ utility for the bulk of their work given the risk of error. They added that the company had set a target for 80 per cent of developers to use AI for coding tasks at least once a week and was closely tracking adoption. Amazon said it was experiencing strong customer growth for Kiro and that it wanted customers and employees to benefit from efficiency gains. “Following the December incident, AWS implemented numerous safeguards”, including mandatory peer review and staff training, Amazon added. src: https://www.ft.com/content/00c282de-ed14-4acd-a948-bc8d6bdb339d

What is the difference between a good and a bad commit message? A good commit message tells you the reason why the change was made, while a bad one tells you nothing or repeats the same changes in the code. This commit doesn't explain anything and provides zero motivation; it is just a fact not attached to any reasoning. https://github.com/kubernetes/kubernetes/commit/94f7f922054d0aa4aa07d572a940ec0dda842646#diff-b2ad44e189798d2d03c3b05e0334899474353de68e03d71653b69ea5fd807c87L287-L387