Adventures in DevOps

Share Episode

This week's adventure tackles the absolute absurdity of modern enterprise infrastructure, where a single company can easily find itself running multiple different CI/CD platforms due to unchecked mergers and acquisitions. We've brought in Chris Farris, AWS Security Hero and consults with companies via Securosis. And dig deep to find the security cracks and philosophize about the real world impacts of tech debt in the AI age.

Management rarely prioritizes standardization, leaving security teams to defend a chaotic swamp of mixed cloud providers, GitHub repositories, and nostalgic on-prem Bitbucket instances. We define this accumulated technical debt not as some abstract concept, but as literal potholes on the infrastructure Autobahn—annoying speed bumps that permanently damage velocity and set organizations up for an inevitable disaster. We contrast this with the evolution from old-school sysadmins cutting their fingers on rack screws to modern engineers spinning up entire architectures with a few lines of code, noting that the ease of deployment has far outpaced our willingness to clean up our own mess.

The crisis is only accelerating now that the cost of writing code (but not having to maintain it) is rapidly approaching zero. While letting an AI agent autonomously build a website or manipulate an AWS sandbox over a single Saturday afternoon sounds magical, it creates a terrifying volume of unreviewed, context-devoid software. Compounding this systemic frailty, massive cloud provider layoffs mean the crucial institutional memory and human operational experience required to survive are walking right out the door. We expose the fundamental flaw of modern agentic tooling: they completely lack fine-grained access control, operating on a dangerous all-or-nothing identity model. Until autonomous agents are engineered with actual conscience, consequence, and common sense, security teams will continue fighting a losing battle against a digital supply chain.

💡 Notable Links:
Chris' Article on AI Tech Debt
Breaking Open Source: Malus - Article
Vercel Security Incident
✨ Episode:
🎯 Picks:
Warren - Rick & Morty S02 + S03
Chris - Risky Business: The latest actually good cybersecurity news

Share Episode

We grabbed Donald Nguyen, co-founder and CTO at Corvic, to discuss the absurd complexities of enterprise data and multimodal inference. We explore how organizations habitually hoard mountains of useless, "dead" data just out of the sheer fantascy that someone might ask for it later. We highlight the fundamental disconnect where data collectors using tools like Airbyte and Kafka speak a completely different language than the business consumers analyzing it in Excel.

True scale isn't just about managing petabytes; it's the absolute nightmare of extracting subjective business meaning from flat PDFs and invoices. In the deep-end of vector embeddings, we're challenging translating data into a different semantic universe requires imposing a heavy business bias. Auditors and artists will view the exact same invoice completely differently, meaning your embedding model selection is incredibly subjective to the business context.

The industry's desperate search for actual AI success stories beyond basic workflow automation is still ongoing as we laugh—and cry—at the reality that companies are likely budgeting 50% of an engineer's salary for LLM token usage, effectively enabling product managers to burn cash on infinite loops to generate prototype code. Reasonable or unreasonable?

And lastly, we tackle the existential dread of securing autonomous AI agents. Because fine-grained access control for agent actions is basically an unsolved fantasy, we must treat their execution environments as entirely untrusted, relying on rigid sandboxes like AWS Firecracker VMs. Prompt injection attacks are an inevitable flaw of the transformer architecture, and the industry's best defense mechanism seems to be wrapping models inside of other models to validate the outputs. It is quite literally turtles all the way down, and the winner of enterprise security is simply the organization that manages to put one more turtle ahead of the attackers.

💡 Notable Links:
Kuuk Thaayorre Aboriginal Tribe - Cardinal Directions
✨ Episode: Generating automatic integrations at scale
🎯 Picks:
Warren - Dr. NEMO: Clockwise circle pit
Donald - Book: InvestiGators

Share Episode

Down to business with GitHub's Cassidy Williams, Senior Director of Developer Advocacy at GitHub, where we try to untangle the existential dread of modern software development. It includes the sheer absurdity of managing a platform that officially crossed the one billion commit mark in 2025. Currently absorbing a completely unreasonable 275 million commits per week, GitHub's technical debt is naturally showing its age under the weight of AI agents aggressively creating pull requests. And with company's own copilot advocating for more, we explore the daily reality of being the internet's punching bag during an outage, and how the "Tiny Wins" buy back developer affection by still shipping the critical features.

Which of course is a small signal in the sea of the industry's collective identity crisis: vibe coding and the valley of AI-generated garbage. Discussed is one suggested solution of strongly typed languages which are skyrocketing in popularity because we desperately need rigid guardrails to babysit the hallucinated code our non-human agents are frantically pushing to production. Things have gotten so dire that we commiserate on missing the good old days of Stack Overflow, where instead of a chatbot agreeably telling you your terrible idea is great, a grumpy human engineer would just ruthlessly roast your architecture honestly.

💡 Notable Links:
Cassidy's post on Typed Language
Fermat's Last Theorem
Cassidy's newsletter
Book: 4-Hour Work Week
✨ Episode: Typed Languages
✨ Episode: Vibecoding
✨ Episode: Productivity Isn't Real
🎯 Picks:
Warren - Book: The Light Eaters
Cassidy - Obsidian Offline Wiki

Share Episode

Founder of Bespinian and long-time cloud solutions architect, Lena Fuhrimann, sits down with us to clarify the widespread confusion around serverless architecture. We discuss how serverless is often incorrectly equated solely with Function as a Service (FaaS), when it actually represents a broader spectrum on the abstraction ladder—including managed AI inference, container platforms, and databases.

Lena shares her early career traps of building a fragmented landscape of sixty "nano-services" and explains why starting with a well-architected monolith and progressively breaking out microservices based on distinct resource or lifecycle requirements is a much saner approach. Then we shift to drivers behind cloud migrations, emphasizing that the primary financial benefit of serverless isn't necessarily shrinking the monthly cloud provider bill, but rather optimizing your most expensive resource: engineering time. By offloading mundane infrastructure patching to the cloud provider, teams can focus entirely on delivering tangible business value to customers. But cost is still there too.

We also explore the psychological challenges of adopting new paradigms, sharing a fascinating story of bridging the gap for a VM-loving engineer by introducing immutable infrastructure concepts through Packer and Ansible before fully transitioning them to containers. And of course we tackle the dreaded topic of "cold starts" and why complex workarounds—like building custom Lambda warmers to periodically call APIs—often defeat the core benefits of reduced total cost of ownership.

💡 Notable Links:
Bespinian
Book: Drive — Motivation 3.0
✨ Episode: Typed Languages, Haskell, and building monoliths
🎯 Picks:
Warren - Better thank coffee: Himmelstau tea
Lena - Home Assistant open source project and Awtrix Clocks

Share Episode

We sit down with Ian Duncan, senior staff engineer on the stability team at Mercury, to discuss the delicate balance of choosing your tech stack and the implications. That means explore the concept of the novelty budget or frequently known as "Choose Boring Technology". It emphasizes why companies should carefully spend their innovation tokens on things that actually move the needle, rather than reinventing the wheel.

Mercury leverages simple technology like Postgres and EC2 instances alongside high-innovation bets like Haskell and Nix to maintain stability. The conversation unpacks the hidden complexities of over-relying on standard tools, sharing a cautionary tale about using a Postgres table as a massive queuing system until it consumed all the database resources and caused login failures. To solve architectural scaling without descending into nanoservice madness, we jump to discussing monolithic build systems. By leveraging hermetically sealed, modular build targets, teams can achieve massive parallelism and avoid endless local rebuilds while maintaining a single coherent view of the codebase.

We also advocate for separating management tools from primary systems by utilizing dedicated control planes, and touch on the rising popularity of durable execution frameworks like Temporal to handle resilient workflows. And it turns out Ian might be a bigger advocate of microservices that he thought!

💡 Notable Links:
Ian's blog
Book: Blah Blah Blah
Using Innovation Tokens
Novelty budget
Buck2
🎯 Picks:
Warren - Why Archers Didn’t Volley Fire
Ian - Band - Gloryhammer