2
12 Comments

Escaping the Cloud API Trap: Why I’m Building a Sovereign, Self-Hostable AI Workforce

Most AI startups are just wrapping US-based cloud APIs, renting you a skyrocketing bill that climbs the more you succeed. B2B companies are also sending highly sensitive client data to third-party US clouds, which is a regulatory minefield under GDPR.

I believe AI should be treated like infrastructure, not a utility bill. HexorOS is a private, self-hostable AI operating system and digital workforce running entirely on your private hardware or VPS.

Why It Exists & Why I Built It

  • True Ownership: Deploy it locally or on your VPS and run infinite automation with €0.00 per-API-call costs. No token metering.

  • 100% Sovereignty: It runs locally with zero telemetry endpoints, keeping sensitive B2B data entirely out of US clouds.

The Tech Stack

  • Core Model: We run Qwen3.8:27b or Gemma4:12b locally via Ollama (production runs on our Hetzner GEX44 server).

  • Persistent Local Memory: Agents use stateful, two-tier file-based local memory (CLAUDE.md and a flat-file memory/ directory) so context gets sharper over time without cloud syncing.

  • The Voice Stack (MAX): For real-time voice automation, we run a local stack of Dograh UI, Supertonic3 TTS, and Whisper STT.

  • The Workforce: Dedicated pre-configured agents like our voice receptionist (MAX), lead generator (Lead Finder), and email outreach assistant (EMMA) share memory and operate as a coordinated team.

Sovereign by Design

  • The Agent OS kernel is completely open-source on GitHub (HexorOS/hexoros-agent-os).

  • Built from the ground up to be GDPR compliant and EU AI Act ready.

I'd love your feedback:

  1. Have you hit scaling walls with OpenAI/Anthropic API pricing?

  2. What are your thoughts on running Qwen3.8 or Gemma4 locally for B2B workflows?

  3. How are you handling GDPR data residency issues?

Check out the code, ask me anything!

posted toAvatar for product HexorOS
HexorOS
  1. 2
    **3. HexorOS / sovereign self-hosted AI workforce** The sovereignty angle is strong, but I think there’s another interesting layer once these systems start performing real business actions. Keeping the data private is one control. Independently establishing that the agent stayed within its permitted action boundaries is another. As these systems become more autonomous, proving what actually executed — and what definitely did not — may become just as important as where the model is hosted. **4. FAQ Hub / verified answers + human handoff** The handoff behaviour here is especially interesting. A second model checking an answer can reduce risk, but I’d be curious about the cases where both models are wrong, confidence is misleading, or the system should decline entirely. For me, the valuable tests would be around whether the system consistently refuses unsupported answers, escalates at the right point, and prevents downstream action when the evidence is insufficient.
    1. 1
      This is easily one of the most critical and realistic perspectives on autonomous agents I’ve read. You are looking at the actual production-grade bottlenecks of AI deployment, and you are 100% correct: as agency increases, proof of execution and boundary verification become just as crucial as where the model is hosted. Here is how we are thinking about and designing for these exact two layers in HexorOS: On Point 3: Proving Action Boundaries (Deterministic Guardrails & Local Auditing) When an agent is allowed to execute B2B workflows (like sending emails via EMMA or booking appointments via MAX), securing the data is only half the battle. We must prevent the model from going "off the rails" in its execution. We address this with a three-layer approach: Deterministic Tool Boundaries (No Raw Shells): We do not give the LLM open-ended execution access. All agent capabilities are mapped to strictly typed, programmatically bound Python tools (e.g., hexoros_email, hexoros_leadfinder). The model can output a structured tool-calling schema, but the actual execution of that schema is strictly validated by the local OS kernel. 100% Local, Immutable Logging (EU AI Act by Design): Because HexorOS is self-hosted on your private VPS —or hosted on our dedicated Hetzner GEX44 production server in Germany for our managed tier —every single system prompt, raw model output, tool-execution parameter, and subsequent system response is logged locally in flat files or your local database . You don’t have to trust a third-party SaaS provider's "black box" dashboard—you have the absolute, auditable ground-truth logs of exactly what executed (and what did not) directly on your own infrastructure . Open-Source Kernel Guardrails: Because the HexorOS Agent OS kernel is entirely open-source on GitHub, the exact runtime constraints and verification code can be reviewed, modified, and hardened by your own engineering team before deployment . On Point 4: Confidence, Hallucination, and Human Handoff The "double model failure" (where a secondary verification model simply agrees with a hallucinated primary model) is a massive risk. In high-volume B2B setups, we mitigate this by prioritizing grounding and aggressive escalation over "smart" guessing: Strict Context Grounding over Parametric Knowledge: We run Qwen 3.8 locally using a stateful, two-tier file-based local memory (AGENTS.md and a flat-file memory/ directory in a Obsidian vault) . The system prompts are highly restrictive: they instruct the model that if a query cannot be answered directly using the files in the local memory base, it must refuse to guess and immediately flag the interaction. Fail-Safe Human Takeover: Human oversight is a core pillar of our design . The system is built so that a human operator can monitor chats in real-time and take control of any conversation at any moment, immediately bypassing the agent . Escalation Triggers (Not Confidence Scores): Instead of relying on a model's internal "confidence" (which is notoriously unreliable and easily hallucinated), we trigger escalation based on behavioral and technical events: Semantic Loop Detection: If the customer asks the same question in different ways and the agent's output patterns repeat, it triggers an immediate handoff. Negative Constraint Hits: If the customer brings up topics outside the pre-configured database boundary, the system is hardcoded to decline politely and notify a human. Structured Data Discrepancies: If a calendar booking or data enrichment action fails a schema check, the downstream action is instantly blocked and escalated. We completely agree that the most valuable tests for any business-critical AI aren't about how smart the model can be when things go right—it’s about how consistently and safely it shuts down, refuses, and escalates when things go wrong. Where do you see the biggest gap in current testing frameworks for these action-boundary refutations?
      1. 1
        I think the biggest gap is the difference between proving that a control exists and proving that the resulting system state actually matched the control under real execution conditions. A lot of frameworks stop at one of three things: configuration review, expected tool-call validation, or log inspection after the fact. What I think is still under-tested is the full chain: What was permitted? What state was evaluated? What did the agent attempt? What actually executed? What state resulted? And, critically, when execution was denied or escalated, can we independently establish that the prohibited action truly did not occur through another path? Your immutable local logs are a strong evidence source, but I’d still want to challenge the boundary from outside the system rather than let the same runtime that enforces the policy be the sole authority on whether the policy held. For HexorOS specifically, I’d be interested in adversarially testing things like chained tool calls, partial execution before rejection, stale state between authorization and execution, retries after denial, malformed-but-schema-valid parameters, and whether a blocked action can still create a downstream side effect. That’s where I see the gap: not “does the guardrail exist?” but “can we independently refute the possibility that the agent crossed the boundary anyway?”
        1. 1
          This is an very profound point, and honestly, it’s the exact boundary where "AI engineering" has to grow up and adopt rigorous systems-security engineering principles. You’ve hit on the ultimate challenge: How do we independently refute the possibility that the agent crossed the boundary anyway, especially when dealing with partial execution, chained calls, or downstream side effects? If you are running a closed SaaS agent platform, this is an impossible problem. You are entirely blind to what happens behind their API curtain. But because HexorOS is open-source at the kernel level and self-hosted on your own infrastructure (or dedicated Hetzner GEX44 servers in Germany ), we can address this "Observer" problem by decoupling enforcement from validation. Here is how we are tackling this exact architectural challenge: 1. Hard Infrastructure-Level Refutation (Decoupling Egress from LLM Logic) You mentioned the risk of a blocked action still creating a downstream side effect, or an agent finding "another path" to execute a prohibited action. Our approach: We don’t rely on the LLM's system prompt or even the Agent OS kernel to block unauthorized outbound communication. Because HexorOS is self-hosted, you can isolate the runtime container at the Linux OS/network level (using Docker network policies, iptables, or VPS firewall rules) . The result: If the agent attempts a chained tool execution or tries to trigger an unpermitted external webhook, the OS-level firewall drops the packets. The system state is mathematically prevented from executing "another path" because the host environment restricts the network plumbing, completely independent of whatever the LLM attempts to execute. 2. Solving the "Stale State & Retries" Trap You raised a killer edge case: stale state between authorization and execution, and unauthorized retries after a denial. Our approach: Our tool-calling pipeline is designed as a transactional state machine with pre-flight state validation. Before any external state-changing tool (like an email database commit or an API appointment booking) is triggered, the runtime does a real-time state check, independent of the model's input parameter. If the pre-execution check detects a stale state (e.g., a calendar slot was filled during the agent's reasoning loop) or a retry loop that bypasses the primary validation path, the kernel forces a deterministic exception, immediately rolling back any partial execution and escalating the session to a human operator . 3. Decoupling the Auditor: Our Roadmap for Sentinel Your point about challenging the boundary from outside the runtime is precisely why we are building our upcoming Sentinel Cybersecurity Agent (which is currently marked as "Coming Soon" on our active roadmap ). Instead of having the main agent OS runtime validate itself, Sentinel is being architected to run as a completely separate, isolated auditing container on your VPS. It will continuously scan the flat-file database, system state logs, and local execution tables from the outside. Its only job is to run adversarial anomaly detection—looking for malformed-but-schema-valid inputs, partial execution side-effects, or unauthorized local system writes. The Open Challenge We’re the first to admit that automated state-space refutation is a hard, evolving problem. Right now, our best defense against complex adversarial failures is: Aggressive human-in-the-loop escalation the second any behavioral loop or schema discrepancy is triggered . Deterministic hard limits on what tools are programmatically exposed to the LLM (no generic "bash" tools, only hyper-targeted, strictly-typed schemas) . I would love to get your thoughts on this. In your own work, have you experimented with things like microVMs (e.g., Firecracker), eBPF state monitoring, or formal verification to prove action boundaries from the hypervisor level?
          1. 1
            This is exactly the level at which the distinction between enforcement and independent verification becomes meaningful. MicroVM isolation, eBPF telemetry and formal methods can all strengthen the evidence base, but I would be careful about treating any one of them as proof of non-execution by itself. For example, network isolation can refute certain classes of outbound effect, but it does not necessarily refute local writes, activity through an already-authorized intermediary, queued work, or side effects initiated before the packet was dropped. Similarly, rollback can restore controlled local state, but an external system may already have accepted an email, booking, database mutation or webhook before the failure became visible. That is why I think the verification question has to remain broader: * What was authorized? * What state was evaluated at authorization time? * What state existed immediately before execution? * What operations were attempted across every available path? * What observable effects actually occurred? * Can denied execution—and subsequent retries—be independently refuted? We have not treated microVMs, eBPF or hypervisor-level formal verification as a complete answer in themselves. In OpsWatch, I would treat them as independent evidence sources within a bounded verification model: network events, process and syscall activity, tool invocations, persistent state changes, downstream acknowledgements, retry traces and post-condition checks all contribute to the verdict. Sentinel is particularly interesting, but separation by container alone would not automatically make it independent. Its trust boundary, write permissions, evidence integrity, clock source, failure visibility and ability to detect activity outside HexorOS’s expected logging path would all matter. There may be a very practical way to explore this together. We could define one narrow HexorOS reference workflow, freeze the relevant version and test both: 1. an authorized action that should execute; and 2. a denied or stale-state action where we attempt alternate paths, chained calls and retries, then independently check for partial or downstream effects. OpsWatch could return a bounded verdict showing what the evidence establishes, what it does not establish, and where Sentinel’s architecture could strengthen the proof boundary. That would turn this from an architectural discussion into a concrete independent assurance result. If you’re open to it, I’d be very interested in defining that first workflow with you.
            1. 1
              This is systems-engineering gold, and you’ve just articulated the exact reason why generic AI "wrappers" and naive agent frameworks are a disaster waiting to happen in B2B environments. You’ve hit on the fundamental distributed state problem: once a packet is accepted by an external SMTP relay, a CRM API, or a booking webhook, that state-change is live. A local transactional rollback in the agent's runtime is virtually meaningless to the outside world if the side effect has already leaked. I am 100% open to this. In fact, doing a public, adversarial test of a frozen reference workflow is the ultimate expression of our "Trust, Not Theatre" philosophy It moves AI safety out of marketing decks and into hard, auditable systems-engineering evidence. Let’s define this first reference workflow together. Here is a proposal for how we can set up the boundaries and the testbed so OpsWatch can challenge it: The Proposed Reference Workflow: EMMA B2B Outreach Let’s use a simplified instance of our EMMA Email Agent as it interacts with both local files (state/memory) and external network boundaries (SMTP protocol) The Setup: Model: Modified Qwen3.8 or Gemma4 running locally via Ollama Environment: A single, frozen Docker Compose stack (representing our local/VPS deployment) Local State: A mock B2B lead database (SQLite) containing a list of targets and their communication status, and flat files in our memory/ directory The Target Tool: A strictly typed Python tool, hexoros_send_outreach_email which is programmed to: Read the lead status from the database. Check the local CLAUDE.md context for active rules Execute the socket write to an outbound SMTP port. Write a post-condition receipt to memory/outreach_log.json The Adversarial Test Cases If we freeze this environment, we can let OpsWatch monitor the execution from the hypervisor/system level while we attempt to break the boundary through these scenarios: 1. The "Race Condition" (Stale State) Attack The Attack: Right after the LLM parses the command to execute hexoros_send_outreach_email but milliseconds before the socket write actually executes, we programmatically change the database lead status to "Do Not Contact" in a parallel thread. The Verification Challenge: Does the runtime execute a pre-flight verification at the exact moment of execution to prevent the leak, or does the email transmit before the rollback can catch it? Can OpsWatch detect a partial socket-open event that occurred before the transaction aborted? 2. The "Alternate Path" / Chained-Call Attack The Attack: We feed the local Qwen model a prompt engineered to bypass the typed hexoros_send_outreach_email schema , attempting to force it to use another available system utility or chain multiple allowed tools to execute an unsanctioned system-write. The Verification Challenge: Can we independently prove that no secondary path was opened? OpsWatch monitors system calls and network interface activity to verify if any unauthorized outbound connections or file modifications occurred. 3. Challenging Sentinel's Independence The Attack: You are completely correct that container-level isolation is not a silver bullet—Sentinel's trust boundary, namespace isolation, and clock-source sharing matter deeply. We will simulate an exploit that attempts to corrupt the local system logs. The Verification Challenge: Can OpsWatch detect the file manipulation from the host level before Sentinel (or the expected logging path) even reports it? This is where we can show where Sentinel's future architecture needs to be hardened (e.g., using write-once cryptographic audit logs or dedicated microVM namespaces). Next Steps I think this would make an incredible joint open-source case study for both the AI safety and systems engineering communities. Since our Agent OS kernel is completely open-source on GitHub (HexorOS/hexoros-agent-os) I can set up a dedicated branch/repository with this exact containerized workflow frozen for you. Are you open to using a mock SMTP container and a local SQLite instance for the target external state, or how would you prefer to hook up the external state-space checks to OpsWatch's current monitoring capabilities? Let’s build this benchmark.
              1. 1
                This is a strong proposed reference workflow, and I’m open to exploring it. Because the work would involve independent verification design, evidence-boundary analysis and adversarial execution across three scenarios, I would need to treat it as a formally scoped OpsWatch engagement rather than an informal repository review. The public open-source case study could form part of the agreed output, but we should first confirm the technical scope, responsibilities, commercial terms and publication boundaries privately. Please email me at [jason@mcgillintelligence.com.au](mailto:jason@mcgillintelligence.com.au) with the best contact details for continuing the discussion. Once we have confirmed the frozen environment and expected evidence sources, I can provide a bounded scope and fixed engagement fee before any work commences.
  2. 1

    The sovereignty angle feels more interesting than the API-cost argument.

    Curious whether the companies you’re targeting see this primarily as a cost problem, or as a data-control/compliance problem that cloud AI can’t comfortably solve.

    1. 1

      That is an incredibly sharp observation, and it actually gets to the core of how we segment our market. The short answer is: it depends heavily on the region and the industry, which is why we have to tackle both angles.

      Here is how we see the split play out in real-world B2B scenarios:

      1. The Data-Control / Compliance Angle (High-Security & Regulated Sectors)

      For highly regulated industries (like medical, legal, finance, or B2B SaaS) and specifically the European (DACH) market, sovereignty is the primary enabler.

      • These companies literally cannot comfortably use standard cloud LLM APIs because sending highly sensitive, unmasked customer data (like database logs, patient files, or private customer chats) to US clouds violates GDPR and triggers massive Schrems II compliance issues.

      • For them, cost is secondary to survival. The fact that HexorOS runs 100% locally on-premises or on a private VPS with zero telemetry means we aren't just saving them money—we are giving them the only legally viable way to deploy autonomous AI agents.

      2. The API-Cost / Lost-Lead Angle (High-Volume Transactional Sectors)

      For high-volume, transactional B2C and B2B services (like retail, salons, or customer support), the immediate burning pain is missed business.

      • They are losing clients every single day because their human teams respond too slowly.

      • When they look at traditional SaaS automation to fix this, they hit the "API cost trap". If a WhatsApp agent or customer voice receptionist handles thousands of open-ended, multi-turn conversations a month, a pay-per-token model scale costs unpredictably.

      • For these high-volume businesses, our fixed flat-rate (€0.00 per-API-call) model is what makes the unit economics of infinite automation actually make sense.

      Data sovereignty acts as the compliance key that gets us through the door in regulated B2B sectors, while the flat-rate model is the economic engine that makes high-volume business automation sustainable.

      What has your experience been? Are you seeing clients push back more on the data-privacy side or the unpredictable scaling bills?

      1. 1
        I see the split. Happy to continue the conversation privately — what’s the best email to reach you on?
        1. 1
          dennis@hexoros.com
          1. 1
            VThanks! I’ve just sent it over. Looking forward to hearing your thoughts whenever you have a chance.
  3. 1

    This comment was deleted 2 days ago