After 18 months of running GPU workloads on Kubernetes, we made a decision that surprised even us: we migrated our entire inference infrastructure to HashiCorp Nomad. This wasn't a whim — it was the result of hitting fundamental scaling limits with how K8s handles heterogeneous hardware.

The Breaking Point

Kubernetes was designed for stateless web services. It's brilliant at that. But when you're scheduling workloads that need specific GPU models, fractional GPU allocation, NUMA-aware memory placement, and topology-aware networking, you're fighting the scheduler instead of working with it.

Our pain points: the K8s device plugin framework treats GPUs as opaque integer resources — you can't express "2 A100s on the same NUMA node with NVLink interconnect." Scheduling latency averaged 4 minutes due to device plugin, topology manager, and pod admission overhead. Bin-packing was terrible — 35-40% GPU waste.

The moment we realized we'd written more custom K8s schedulers than actual application code, we knew something had to change.

Why Nomad

Nomad's scheduler maintains a rich understanding of node topology. GPU types, interconnects, memory layout — all first-class scheduling constraints. Results: GPU bin-packing improved 23% (~$2.1M annual savings), deploy latency dropped from ~4min to ~45s, operational overhead cut roughly in half.

The Trade-offs

Nomad's ecosystem is a fraction of Kubernetes'. We lost hundreds of operators, CRDs, and integrations. Service mesh needed custom work. Hiring is harder — every infra engineer knows K8s, not Nomad.

Our Recommendation

For GPU-heavy inference: Nomad is better. For general microservices: keep K8s. For many orgs, a hybrid approach makes sense. We've open-sourced our Nomad job specs and GPU bin-packing plugin. Links in thread below.

Having SOC 2 Type II certification tells me almost nothing about whether your organization is actually secure. It tells me you're auditable. Those are very different things.

The Compliance-Security Gap

Behind pristine SOC 2 reports I've found: unpatched CVEs in production for months, overly broad IAM policies never reviewed since setup, logging infra that exists but nobody monitors, and incident response plans never tested. All passed audit. Because SOC 2 checks whether controls exist, not whether they're effective.

What Actually Makes You Secure

Three things no compliance framework adequately measures: threat modeling specific to your architecture, continuous validation via real attack simulations (not annual pentests), and security culture where engineers think about security daily.

Beyond Checkboxes

SOC 2 is a baseline. Treating compliance as the ceiling rather than the floor is how breaches happen.

For a decade, workflow automation followed "when X happens, do Y, then Z." Powerful for simple flows, breaks spectacularly when reality is messy.

The Problem with Trigger-Action Chains

"When new customer signs up → create CRM record → send welcome email → provision account → notify sales." Sounds simple. In practice: a dozen edge cases. CRM API down? Email bounces? Duplicate signup? Each adds branches until it's unmaintainable.

Enter Agentic Middleware

Instead of defining every step, define the intent: "onboard this customer." The AI agent understands your context and figures out the right sequence dynamically. CRM API down? It queues, retries, or escalates. Duplicate? It merges. Error rates on complex automations dropping 40-60%.

What This Means

Simple deterministic flows keep trigger-action. Complex cross-system orchestration? Agentic middleware will be default within two years.