https://www.linkedin.com/embed/feed/update/urn:li:share:7291032976509779968

Stop guessing at Kubernetes security. Get actionable advice and real-world fixes with the Kubernetes Security Playbook. Real problems, trusted frameworks, exam prep too! Subscribe now.

Hey cloud-native friends! 👋

Kubernetes is booming, powering modern infrastructure for millions. But with massive adoption comes massive security risk. Incidents are rising, costing companies and delaying deployments. We can't afford to guess at security anymore. To navigate this complex landscape and validate your skills, the KCSA certification is becoming essential.

That's why I built the Kubernetes Security Playbook. to help you master real-world Kubernetes security. Plus, it helps you prep for your exam.

What to Expect: Real-World best practices and actionable advice & fixes:

This Week: Pod Security Standards (PSS)

This article dives into Pod Security Standards: Baseline vs. Restricted Policies. Learn how PSS provides your essential first line of defense!

[Link to article: https://cloud-native.nikkei.one/kubernetes-security-fundamentals/kubernetes-security-fundamentals/pod-security-standards-pss-baseline-vs.-restricted-policies]

Quick Security Updates:

• Microsoft Defender for Cloud: New RBAC integration boosts Kubernetes security visibility and attack path detection! [Link: https://techcommunity.microsoft.com/blog/microsoftdefendercloudblog/unveiling-kubernetes-lateral-movement-and-attack-paths-with-microsoft-defender-f/4374958]

• Kubernetes Security Report (Preview): Good news! Severe pod vulnerabilities are down 50% and high-privilege pods are decreasing. [Link: https://www.wiz.io/blog/kubernetes-report-preview-2025]

Want more bite-sized cloud-native goodness?

Join me and other cloud enthusiasts at Down Under Cloud Nights: Bi-Weekly Bytes, hosted by Cloud Native Melbourne! It's a virtual, 40-minute speed run into cloud-native topics happening every other Monday.

Next Event: eBPF Revolution - Feb 10th!

Learn how eBPF is transforming cloud-native computing.

[Link to Cloud Native Melbourne Page: https://community.cncf.io/cloud-native-melbourne/]

Secure your Kubernetes journey with the Kubernetes Security Playbook and stay connected with the community!

Your Cloud-Native Friend,

Thiago, Kubestronaut.

Are you ready to level up your Kubernetes security game in 2025? With open source software expanding and supply chain threats becoming ever more critical, securing your software pipeline is no longer optional – it's mandatory. Regulations are coming, attacks are evolving, and proactive defense is the only way forward.

This week's Kubernetes Security Playbook deep dive is your guide to building a robust defense: "Supply Chain Security: SBOMs, SLSA, and Sigstore." Learn how to implement layered security using Software Bill of Materials (SBOMs), Supply-chain Levels for Software Artifacts (SLSA), and Sigstore to ensure the integrity and trustworthiness of your Kubernetes deployments. Stop vulnerabilities at the source and build confidence in your cloud-native stack!

Read the full post: https://cloud-native.nikkei.one/platform-security/platform-security/supply-chain-security-sboms-slsa-and-sigstore

Quick Security Bytes:

• 🔐Microsoft Enhances AKS Security with Ratify: Learn how Azure Kubernetes Service is bolstering container image security by validating signatures using Ratify and Azure Policy. A practical step towards a more secure deployment pipeline! "Securing AKS workloads: Validating container image signatures with Ratify and Azure Policy"

• ✅Google Cloud's Transparency Boost for GKE: Google Kubernetes Engine is raising the bar on transparency by implementing SLSA Verification Summary Attestations for GKE components. Gain deeper insights into your software origins and ensure component integrity. "How we’re making GKE more transparent with supply-chain attestation and SLSA"

Don't Miss Out! 🇦🇺 Cloud Native Down Under: Sustainability Speed Run!🍃♻️🏞️💚🌱

Join Cloud Native Melbourne for a lightning-fast virtual session on Sustainability with Kepler! Discover how Kepler leverages eBPF to monitor energy consumption in Kubernetes and drive eco-friendly cloud-native practices. Happening TODAY!

Cloud Native Melbourne Event: https://community.cncf.io/events/details/cncf-cloud-native-melbourne-presents-down-under-cloud-nights-bi-weekly-bytes-2025-02-24/

Secure your supply chain, keep shipping trusted software! Dive into the Kubernetes Security Playbook and stay ahead of the curve.

Your Cloud-Native Friend,

Thiago, Kubestronaut.

Are your Kubernetes trust boundaries clearly defined? They should be! Recent breaches prove misconfigurations are kubernetes' weakest link. Our latest post in the Kubernetes Security Playbook dives deep into mapping data flows and attack surfaces within your clusters.

🛡️ Learn to identify critical trust boundaries (you might be surprised where they are!)

🔑 Get actionable steps to enforce least privilege, zero trust networking, and more.

💥 Real-world incidents highlight the RIsks - are you protected?

Read the full post: Kubernetes Trust Boundaries: Mapping Data Flows and Attack Surfaces

Quick Security Bytes:

"Learn Azure Defender for Cloud Containers support matrix in Defender for Cloud": This doc clearly defines the boundaries of protection offered by Microsoft Defender for Cloud for container workloads. It specifies what features are supported for different environments (Azure, AWS, GCP, external registries), operating systems, and Kubernetes distributions.

"Amazon EKS now envelope encrypts all Kubernetes API data by default": This announcement highlights a strengthening of the security boundary within Amazon EKS. By default encrypting Kubernetes API data, Amazon is reinforcing the data protection boundary for EKS clusters, making it harder for unauthorized access even if the control plane is compromised.

"Integrating Security into DevOps Workflows with Microsoft Defender CSPM": This blog post directly addresses the concept of shifting security boundaries left in the development lifecycle. This is about defining and enforcing security boundaries not just at runtime, but also during development and deployment.

Don't miss tonight's Down Under Cloud Nights! 🚀🌏

Join Cloud Native Melbourne for Lap #3: DevEx with Inspektor Gadget & mirrord!

📅 When: TODAY, Mar 10, 7:00–7:40 PM (GMT+11)

📍 Where: Virtual (Limited Zoom seats available!)

Seats are filling fast—secure your spot now!

🔗 Register Here

Discover how Inspektor Gadget leverages eBPF to provide real-time Kubernetes debugging, monitoring network traffic, file I/O, and system calls. Plus, see how mirrord mirrors production traffic to local environments, solving the dreaded "works on my machine" syndrome.

Don't wait for the next breach to learn about trust boundaries. Arm yourself with knowledge from the Kubernetes Security Playbook and community insights from Down Under Cloud Nights. Secure your Kubernetes environment today, for peace of mind tomorrow.

Your Cloud-Native Friend,

Thiago, Kubestronaut.

But I am a Kubestronaut. My background is built on the bedrock of orchestration. And as I looked closer at the "magic" of AI, the mist cleared. I realized that under the hood, the ghost in the machine is just a workload. AI is just containers. AI is just Kubernetes with heavier math.

I decided to stop renting. I decided to build.

To do that, I needed iron. I needed sovereignty. My modest homelab was fine for running Plex and a few microservices, but the game has changed. You cannot build the future on yesterday’s silicon.

The Object of Desire

Then, the email arrived.

Subject: Your DGX Spark is Ready for Purchase Reservation #26525

The specs read like science fiction transcribed into a receipt:

• Two DGX Spark units

• NVIDIA GB10 Grace Blackwell Superchip

• NVIDIA GB10 Grace Blackwell Superchip

• 128GB of coherent, unified system memory

• 1 PFLOPS of FP4 AI performance

• ConnectX-7 Smart NIC

• 4TB Self-Encrypting NVMe

Total: $8,049.00.

It is a salty price tag. But for 1 Petaflop of compute that fits in a carry-on? It wasn’t a purchase; it was a transfer of power.

The Weekend Run

There was a catch. They shipped to the US. I am in Australia.

A rational person would use a freight forwarder. A rational person would wait. I am not a rational person; I am building a sovereign AI lab.

I booked a flight.

It was the most paranoid trip of my life. I left Australia on Friday. I landed in the US, retrieved the package—my precious cargo, the silicon brain of my future infrastructure—and turned right back around.

Melbourne → Los Angeles → New York → Los Angeles → Sydney → Melbourne.

A weekend round-trip for hardware.

Returning to Sydney, I felt like a character in a bad episode of Border Control. I walked the green line—no, the red line. I declared it. I paid the taxes. I stood there, exhausted, clutching a box containing two Grace Blackwell superchips, waiting for a customs officer to ask me why a man needs a Petaflop in his luggage.

I walked out into the Australian sun, lighter in the wallet but heavy with potential.

The Nest

You cannot just plug a DGX Spark into the wall and hope for the best. You do not treat a superchip like a toaster.

I looked at my home office. It wasn't ready.

I went into procurement mode. If I was going to run enterprise-grade AI, I needed enterprise-grade stability.

• Power: A new UPS 1600VA. If the grid flickers, the AI must not blink.

• Networking: New switches, new NICs. I had to build a Bastion to protect the crown jewels.

And then, the humble reality of high-tech engineering: I was two cables short.

There I was, holding the cutting edge of NVIDIA’s Blackwell architecture, forced to make a frantic, mundane run to Officeworks on a Sunday afternoon for generic patch cables. The contrast was perfect.

Everything is connected. The UPS is humming. The Bastion is racked. The Sparks are energized.

The holiday is over. The tourist phase is done.

Next: We turn on the lights. In Part 1, we design the network architecture that keeps this beast fed, secure, and silent.

You want the silence.

You want the Sovereign AI Lab, but sovereignty is a paradox—to rule, you must first build the walls, yet to govern, you must maintain the lines of supply. We are not building a network; we are constructing a filter against the entropy of the open web. And because true security demands obscurity, we do not publish our maps. We publish only the architecture of the maze.

The Double-Blind Shell Game

Consider the topology not as a tree, but as a series of airlocks. The traffic arrives from the ISP, wet and chaotic, hitting the Primary Router. It is a pedestrian device, a decoy. The traffic thinks it has found home. It hasn’t.

It flows downstream to the Secondary Router, the second layer of the onion, the bureaucratic obfuscation where the real work begins. This is the Double NAT—a geometric frustration for the uninvited. But sitting there, humming in the rack, is the Bastion. It is the gatekeeper. It is the three-faced Janus.

The Bastion: Maxwell’s Demon in Rocky Linux

The Bastion is not merely a server; it is a checkpoint Charlie for packets. It possesses three distinct nervous systems, three interfaces wired into the kernel’s cortex:

• The External Face: Staring into the WAN, taking the brunt of the noise from the upstream network.

• The Internal Face: The gateway to the Sanctum, the Compute Zone.

• The Ghost Face: The WireGuard interface. The wormhole.

This Bastion runs , a piece of software acting as the adjudicator of truth. It implements a ruthless logic: nothing enters the Compute Zone without a visa.

The Compute Zone: Where the GPUs Dream

Deep inside a private, unrouted subnet lie the heavy irons—the DGX Spark nodes and the Kubernetes Control Plane. This is the Secure Compute Zone.

Here, the air is still. The silence is absolute. These machines cannot see the internet. If a node tries to ping Google, the packet hits the Bastion and dies, rejected by a rich rule in the firewall configuration. It is a digital Faraday cage.

But the machines must eat. They need updates, Python wheels, Docker layers. Enter Squid, the transparent proxy.

The Bastion runs Squid on a local port. It intercepts HTTP/HTTPS traffic from the Compute Zone. It inspects the destination. Is it ? Is it ? If the destination is on the whitelist, the packet is blessed and sent forth. If it is some unknown telemetry endpoint trying to phone home, it is dissolved into nothingness.

The WireGuard Wormhole

How do you, the architect, enter this hermetically sealed tomb? You do not open a door. You fold space.

We use WireGuard on a specific UDP port, a protocol of elegant, cryptographic silence. Unlike the chatty VPNs of old that announce their presence with elaborate handshakes, WireGuard stays silent until it sees a key it recognizes.

The path is circuitous, a Rube Goldberg mechanism of routing:

Your client, drifting somewhere on the 4G cellular ether, sends a packet to the public endpoint. The Primary Router catches it, sees the UDP tag, and flings it to the Bastion’s dedicated backchannel interface, separate from the primary wired WAN.

The key turns. The tunnel opens. Suddenly, you are bilocated. Your physical body is in a coffee shop; your digital soul is inside the Compute Zone, SSH-ing into the DGX Spark as if you were sitting on the rack.

We have built a fortress. The GPUs are safe. The data is sovereign. The silence is ours.

Next: Now that the walls are up, we must fill the void. In Part 2, we descend into the Setup, where we will attempt to birth a Kubernetes cluster inside this vacuum—and discover the agony of the whitelist.

The theory is sovereignty. The reality is that we have locked ourselves in a room with no windows and are now surprised that we cannot see the sky.

The Invocation of the Void

You sit at the terminal of the control plane, a Proxmox VM humming with the naive potential of a blank slate. The cursor blinks, a rhythmic taunt. You type the invocation, the command that has worked a thousand times in the promiscuous, open-air markets of AWS and Azure: kubeadm init.

You wait for the cascade of ASCII green, the dopamine rush of successful initialization.

Instead, the console hangs. A timeout. The machine is screaming into the void, trying to phone home to registry.k8s.io, but the line is dead. There is no route. There is only the Bastion, sitting at the edge of the world, an unsmiling bureaucrat enforcing a policy of total embargo.

This is the price of paranoia. You cannot simply install software in a sovereign lab; you must petition for it. You open a second terminal, tunneling through WireGuard to the Bastion, and begin the forensic audit of the Squid proxy logs.

The screen fills with the carnage of rejection. TCP_DENIED. TCP_DENIED. The modern software stack reveals itself not as a monolith, but as a desperate, sprawling organism, firing tendrils out to every corner of the globe, begging for dependencies. You see the requests piling up against the firewall: cdn.dl.k8s.io, quay.io, production.cloudflare.docker.com.

The Customs Officer

You are no longer an engineer; you are a customs officer in a Kafka novel. You open the configuration file /etc/squid/squid.conf, the ledger of allowed thoughts. You type the domains in, one by one, granting visas to these digital travelers. You restart the proxy. You run kubeadm again.

It advances. It pulls the core images. And then it chokes again.

The logs reveal a new, more insidious truth. You expected to whitelist a few project domains, perhaps a GitHub repository. But there, buried in the CONNECT requests, is the hallucination:

You stare at it. Your sovereign, on-premise, strictly controlled AI lab—built on local NVMe and owned hardware—is secretly trying to hold hands with a random storage bucket in Oregon. It turns out that Cloud Native is often just a euphemism for Hidden S3 Dependencies. The infrastructure of the independent web is a facade; scratch the surface of a CNCF project, and you bleed Amazon object storage.

If you had left the firewall open, you would be blissfully ignorant, pulling blobs from the ether, unaware of the invisible tethers binding your sovereign AI to a datacenter three thousand miles away. But here, in the silence of the vacuum, the dependencies are loud.

The Compromise

You add the wildcard to the whitelist. You feel the compromise in your bones, but kubeadm finally completes its ritual. The control plane initializes.

The pain of the whitelist, the agony of tracking down every single redirect and CNAME, is the only proof that the cage is working. You have mapped the veins of the system, and now, finally, the brain is alive.

Next: The infrastructure is breathing. The power is stable. The network is secure. Now, we stop building walls and start teaching the machine to think. In Part 3, we finally start making use of the NVIDIA stack and begin our journey into Deep Learning with the Game of Go.

I have built the network (Log 1) and initialized the cluster (Log 2), but without a proper design for managing state, the system is brittle. It invites human error.

To solve this, I must change the affordances of the system. I must make the invisible visible

The System Image: GitOps as a Conceptual Model

I refuse to configure servers by hand (I use Kiro, btw). To do so is to create a system that cannot be understood or reproduced. Instead, I adopt GitOps.

GitOps is not just a tool; it is a design philosophy. It ensures that the "system image"—the code in the repository—always matches the physical state of the cluster.

I deployed Gitea on the management network to serve as the single source of truth. This is where the intent is stored. Then, I introduced ArgoCD into the cluster. Think of ArgoCD as a feedback mechanism. It constantly compares my intent (the Git repo) with the reality (the Kubernetes cluster). If they drift apart, it signals an error. It forces the system back into alignment. This provides the visibility and feedback that are essential for good control.

Designing for Persistence (NFS)

Computers are ephemeral; data must be permanent. The design challenge here is mapping the abstract concept of "storage" to the physical constraints of the hardware.

I designated the Proxmox host and a future NAS to act as the NFS backend. But good design imposes constraints, so I carved the storage into distinct categories based on their function:

• /srv/k8s/datasets: This is for raw inputs. The policy is "Retain" because the cost of re-acquiring data is high.

• /srv/k8s/models: This is for the output—the trained intelligence. It is also "Retained".

• /srv/k8s/artifacts: This is for temporary workspace. The policy is "Delete," signaling its transient nature.

By mapping these directories to Kubernetes StorageClasses, I created a natural mapping. When a pod requests "datasets," the system intuitively knows where to look.

The Affordance of Recovery

A well-designed system must handle errors gracefully. It must assume that things will go wrong. In the physical world, we have safety rails. In the digital world, we have backups.

However, the traditional backup process is flawed. It usually involves copying data to a local disk before moving it off-site. This introduces a point of failure—what if the local disk is full? What if it is corrupted?

I designed a "stream-based" backup workflow that eliminates this intermediate step. I wrote a script that pipes the data directly from the virtualization layer (vzdump) to the cloud storage (az blob upload).

The data flows from the memory of the Bastion, encrypts in transit, and rests securely in an Azure Blob Storage container in Australia. The design is elegant because it removes unnecessary steps. It reduces the cognitive load. I do not have to worry about local disk space. I only have to trust the stream.

Next: The system now has memory and a reliable way to manage its own state. In Log 4, I must address the interaction between the user and the machine. I must wake the brain.

In Log 3, I talked about the Gulf of Evaluation—the difficulty of knowing the state of the system. But now I face its twin, the Gulf of Execution. This is the gap between what I want the system to do and the means the system provides to do it.

Right now, that gulf is massive.

Bridging the Gulf with AI

Let’s be real: I didn’t bridge this gap by writing boilerplate code by hand. That is the old way. I leveraged the very thing I am trying to build.

I used Kiro, Gemini, ChatGPT, and Perplexity. I treated them as my team of engineers. I used them to brainstorm, to generate the Kubernetes manifests, debug the Python syntax, and explain the obscure error messages. I stopped thinking about the syntax of the bridge and focused entirely on the architecture.

By leaning on these tools, I condensed weeks of frustration into few days

We have spent the last few logs building the temple. We racked the DGX Sparks, wrestled the Kubernetes networking into submission, and locked down the security. The infrastructure is beautiful. It’s a Ferrari with the engine idling. But a Ferrari sitting in a garage isn’t a race car; it’s just expensive furniture.

Now, I turn the key.

This marks the start of a new phase. I’m moving from Hardware & Orchestration to Applied Machine Learning. I’m following the roadmap from Deep Learning and the Game of Go by Kevin Ferguson and Max Pumperla (https://amzn.to/49t64uQ), but I’m doing it my way: inside a sovereign, containerized, GPU-accelerated cluster.

Defining the Universe

To cross the gulf, I first had to define the laws of physics for this new universe. Therefore before you can train an AI to dominate a world, you first have to create that world.

In design terms, the rules of Go are constraints. They tell the agent what is possible. I had to code the concept of "Liberties" (the empty points that allow a stone to breathe) and "Capture" (what happens when those liberties vanish). I had to strictly define "Ko"—the rule that prevents the game from entering an infinite loop.

If I get this wrong, the "System Image" fails. The bot will try to do things that physics doesn't allow, and the gulf will widen again.

This isn't AI. This is just the plumbing. It’s the digital equivalent of gravity.

Meet the "Agent" (The Idiot Savant)

Once the physics were working, I wrote my first bot.

In the industry, we use fancy terms like "Agent" or "Actor." It implies intelligence. It implies intent.

My agent is... well, it’s an idiot.

I implemented what is effectively a Random Bot. It looks at the board, asks the physics engine, "What moves are legal?" and then rolls a virtual die.

It has no strategy. It has no fear. It has no desire to win. It just does.

I set it loose on the 9x9 board, running on the most advanced consumer grade silicon NVIDIA has ever produced. It placed stones with the confidence of a grandmaster and the logic of a goldfish.

The Problem with Random

Watching two Random Bots play against each other is a surreal experience. Because they don't understand the concept of "territory" or "winning," they don't know when to stop.

Humans stop playing when the boundaries are clear. We look at the board, nod to our opponent, and say, "Okay, that's your territory."

This bot? It’s stubborn. It keeps placing stones until literally every legal spot on the board is filled. It will fill in its own eyes. It will kill its own groups. It plays until the game collapses under the weight of its own absurdity.

Why This Matters (The "Sovereign" Flex)

You might ask: "Why did you buy two DGX Spark nodes and build a Kubernetes cluster just to run a random number generator?"

That’s a fair question. I could run this Random Bot on a Raspberry Pi. I could have just installed Python directly on the metal and been done with it.

But here is the difference: Architecture.

I didn't just write a script. I deployed a System.

I’m not running this locally on my laptop; I’m running it as a defined job in a namespace. My code is decoupled from the hardware. Today, the "Brain" block of my system is a simple random function. Tomorrow, I can swap that block out for a Deep Neural Network, and the rest of the pipeline—the board state, the game rules, the scoring engine—doesn't need to change.

This is what sets this design apart from just using the Spark out of the box. I have laid the foundation for a Multi-Agent System. I can spin up fifty of these bots in parallel containers, have them play a thousand games in seconds, and aggregate the results. The DGX isn't sweating yet, but the plumbing is ready for the flood.

The Baseline

This Random Bot is terrible, but it is crucial. It gives me a Baseline.

Right now, my win rate is 50/50 (because it’s random vs. random). This is the floor. Everything I do from this point on—every neural network I design, every reinforcement learning loop I run—has to beat this idiot.

The engine is started. The physics work. The bot is alive, even if it is completely mindless.

Next Log: We stop rolling dice. We give the bot some lessons. We start feeding it human games and teaching it to mimic the masters. We enter the world of Machine Learning, Tree Search algorithms, which are essential tools in game AI and all kinds of optimization problems. Deep Learning, Neural Networks, Reinforcement Learning! Are you excited?!