5
32 Comments

We're debating the next feature for StateGuard. Would you trust this?

Over the past few weeks, we've been building StateGuard, an open-source runtime reliability SDK for AI applications.

The goal isn't to improve model quality. It's to keep AI workflows running when something around the model changes unexpectedly - schema drift, provider response changes, tool output mismatches, validation errors, missing fields, and similar runtime failures.

Instead of the application crashing because one component changed, StateGuard attempts to repair the mismatch and continue the workflow safely.

One capability we're considering next is Semantic Repair.

For example:

A provider returns "done" instead of "completed"
A tool renames "assistant_message" to "assistant"
Two APIs use different field names for the same concept
Equivalent values are represented differently across providers

Instead of failing, StateGuard would recognize that the intent is equivalent, normalize the payload, and continue execution.

The question is:

Would you actually trust something like this in production?

Or would you rather these cases fail loudly so you can fix them yourself?

We're trying to build the roadmap around real developer feedback, so I'd genuinely love to know:

Where would you draw the line between "safe automatic repair" and "developer intervention required"?
Have you run into failures like these yourself?
Is there another runtime failure you'd solve before semantic repair?

GitHub: https://github.com/dood1ebyte/stateguard

on August 5, 2026
  1. 1

    Where I'd draw it is on what you persist rather than on whether the repair runs. If the trace keeps the normalized object, then three weeks later when a customer disputes an outcome the raw provider response that caused it is gone and the incident can't be reconstructed. Does the SDK keep the original payload plus the repair diff, or does the repaired version replace it in the trace?

    1. 2

      That's a really good point. We definitely don't want the repaired payload to replace the original in the trace - the original response and the repair diff need to remain available so you can reconstruct exactly what happened later.

      I'm going to make that explicit in the design. Appreciate you catching that.

  2. 1

    I'd trust it more if the repair still logged a warning even when it succeeds, so you get the safety net without losing visibility into drift. Nice problem to tackle.

    1. 1

      Yeah, I agree. A successful repair shouldn't make the drift invisible. We'd want the repair to still show up in the logs/trace so you know StateGuard had to intervene, even if the workflow continued successfully.

      That's something we're going to keep in the design.

  3. 1

    Great question. In production, I'd trust semantic repair only for low-risk normalizations — like "done" vs "completed" — where the intent is unambiguous. But for field renames across providers, I'd want a confidence threshold with a fallback to loud failure, not silent repair.

    We ran into something similar building Tasbazi (real-time multiplayer game platform) — when syncing game state between Socket IO events, small payload mismatches could silently corrupt board state. We chose loud failure + client rollback over auto-repair, because silent errors in games are worse than visible ones.

    My line: if the repair is reversible and logged, automate it. If it touches business logic or data integrity, fail loudly.

    1. 1

      If you're open to it, I'd also be happy to stay in touch as we build this out. Your experience with those failures would be really useful as we test where the boundaries of automatic repair should be.

    2. 1

      This is a really useful example, especially the Socket.IO state mismatch point. I agree that the line probably comes down to whether the repair can be proven to be safe and reversible.

      The confidence + fallback approach is something we're definitely thinking about. We're trying to make StateGuard conservative by default rather than have it guess and silently change application behavior.

      Thanks for sharing the game-state example - this is exactly the kind of real failure mode we want to learn from.

  4. 1

    Before enabling Semantic Repair automatically, I’d consider shipping it in shadow mode first.

    For each candidate repair, StateGuard could record:

    original payload
    proposed normalization
    rule or similarity that produced it
    validation result after the proposed repair
    whether the developer later made the same effective fix

    Nothing changes in production during this phase. You are collecting evidence about which repair classes developers consistently accept.

    After enough examples, the policy could graduate per contract:

    observe → require approval → auto-repair

    I’d also make automatic repair opt-in by repair class rather than one global switch. Renaming a known enum value may earn automation quickly, while anything touching identifiers, money, permissions, or side-effecting tool calls could permanently remain approval-only.

    That gives you an empirical answer to “would developers trust this?” instead of asking them to predict their own trust.

    Would you consider using shadow-mode acceptance rate as the gate for deciding which semantic repairs become automatic?

    1. 1

      This is actually very close to how I'm thinking about it now. Shadow mode would let us collect real evidence about which repairs are consistently safe before giving them automatic authority.

      I especially like the idea of graduating individual repair classes rather than having one global "semantic repair" switch. That feels much safer.

      And yes, using acceptance rate / observed outcomes as the gate is much more useful than simply asking developers whether they'd trust it. I'm going to think this one through more.

      1. 1

        Glad it was useful.

        I think the strongest part of the approach is that trust becomes something you can earn per repair class with evidence, rather than asking developers to grant broad authority upfront.

        Shadow mode should give you a much cleaner signal for where that boundary belongs.

        Good luck with the next iteration.

        1. 1

          Yeah, I think that's the direction we're leaning toward. The interesting part is using Shadow Mode to build evidence around which repair classes are actually safe before giving them automatic authority.

          We're going to keep that principle as we develop the next iteration. Thanks again for the suggestion - it helped shape how we're thinking about the trust boundary.

  5. 1

    I'd trust it more if the repair still logged a warning even when it succeeds, so you get the safety net without losing visibility into drift. Nice problem to tackle.

    1. 1

      Yeah, absolutely. I think that's an important part of making automatic repair useful without making the system invisible.

      Even if the workflow succeeds, the fact that a repair happened is still valuable because it tells you the underlying contract has started drifting.

      Definitely something I'd want StateGuard to surface.

  6. 1

    Trust is such a hard sell when you're asking users to put confidence in your tool — what's the specific problem the feature is solving?

    1. 1

      Reading it back, I realize I jumped straight into "would you trust it?" without first making the pain concrete.

      The problem we're actually trying to solve is when otherwise healthy workflows break because two components no longer agree on a contract - a provider changes a field, a tool returns a slightly different shape, or an SDK update changes an assumption.

  7. 1

    I am sure I would trust it but only with limits.

    When it comes to mapping equivalent field names or handling predictable format differences, that seems reasonable. As soon as it begins inferring intent, or making decisions that would change the business logic, then let it fail and tell me why.

    The outlined repair that others can also see will instil confidence over time.

    1. 1

      I think we're on exactly the same page. I'd never want StateGuard making business decisions on behalf of an application. The boundary in my mind is deterministic compatibility rather than inferred intent.

      Your point about making every repair visible is a great one too. The more I think about it, the more I feel trust comes from being able to understand why something was repaired, not just the fact that it was.

  8. 1

    I read the M9 audit, and I'd solve one diagnosability gap before adding Semantic Repair: make ambiguity a first-class result.

    The current cross-branch fuzzy collision behavior is safely conservative, but the audit says a withheld repair appears as FAILED/PARTIAL with a generic remaining violation. Operationally, "no mapping found" and "two mappings were plausible, so StateGuard refused to guess" need different handling.

    I'd expose an AMBIGUOUS_REPAIR reason with the candidate paths, scores, strategy, and contract ID. Applications could route that case to review and alert on it separately, while ordinary validation failures keep their current path.

    Would you prioritize that before Semantic Repair? It seems like a small change that makes the existing safety boundary observable and testable.

    AI-assisted draft, reviewed and verified against the public repository.

    1. 1

      appreciate all the thoughtful feedback. We're trying to shape StateGuard around conversations like this rather than assumptions. If you're open to it, I'd love to connect on LinkedIn or GitHub and occasionally bounce design ideas off you.

    2. 1

      Not going to lie, one of the most useful pieces of feedback we've gotten.

      I hadn't thought about treating ambiguity as its own first-class outcome rather than just another failed repair. Exposing the candidate mappings and why StateGuard refused to choose one feels much more useful than returning a generic failure.

      I'm definitely going to spend some time thinking about that before expanding semantic repair further.

      Really appreciate you taking the time to read through the repo and the audit. If you spot anything else that feels off, I'd genuinely love to hear it.

  9. 1

    I’d trust automatic repair only when the transformation is closed, enumerable, and lossless—for example, a versioned alias map from done to completed, or a provider field rename with the original payload retained. I’d want every repair to emit the rule used, before/after values, and a metric or trace so drift is visible. I would fail closed when the repair changes permissions, payment state, identity, persistence, or tool side effects, and I’d also stop when multiple mappings are plausible. A useful middle ground could be shadow mode first: propose and log the repair, but don’t apply it until the same rule has been reviewed. That would make semantic repair feel like controlled compatibility handling rather than an LLM guessing past validation.

    1. 1

      I really like the phrase "closed, enumerable, and lossless." That captures the boundary much better than I'd been describing it.

      The shadow mode suggestion is interesting too. We've talked about dry-running repairs before enabling them, but framing it as "show me what would've happened before doing it" makes a lot of sense from a trust perspective.

      I think you've probably helped define where the line should be between automatic compatibility repair and developer intervention.

  10. 1

    Interesting question because trust is usually the hardest feature to build — and the easiest thing to lose.

    One thing I’ve noticed with products in trust/security-related spaces is that users often don’t just evaluate what the product does, they evaluate whether the product feels reliable enough to depend on.

    Before adding another feature, it can be useful to ask:
    “Does this reduce the user’s biggest fear or uncertainty?”

    The features that usually win are the ones that make users feel more confident taking action.

    1. 1

      That's a perspective I probably needed to hear,
      it's easy to focus on what the next feature should be, but if users don't feel confident depending on the product, another capability won't really matter.

      You've definitely made me think more about making StateGuard's decisions explainable and predictable before making them more powerful. Thanks for that.

      1. 1

        This is exactly the kind of decision where I'd look beyond the feature list and into the user's confidence journey.

        For a product like StateGuard, I'd be curious about three things:

        • What fear makes someone hesitate before trusting it?
        • What proof or explanation removes that hesitation?
        • Does the landing page communicate that confidence before asking for commitment?

        A lot of products in trust-heavy categories don't lose because the technology isn't strong — they lose because users can't quickly understand why they should rely on it.

        This is actually the kind of messaging problem I spend time analyzing with SaaS products.

        1. 1

          That's a good point. I think with StateGuard the trust problem is actually just as important as the technical problem - especially when you're asking developers to let something automatically modify a failing workflow.

          That's also why I'm leaning toward making the repairs observable and explainable rather than just having them happen silently. Developers should be able to understand what changed, why it changed, and why StateGuard considered it safe.

          The landing page point is useful too. We're still early on the messaging, so I'll keep that in mind.

          1. 1

            That distinction between “automatic” and “silent” feels really important here.

            For something like StateGuard, I don't think the trust story has to be “trust us to make the right repair.” That's a pretty big leap for a developer.

            It may be easier to communicate:

            “We show you what changed, why we changed it, and why we considered the repair safe.”

            That turns explainability from a technical implementation detail into part of the product's value proposition.

            I'd also be careful about leading with “semantic repair” on the landing page. It's accurate, but a developer encountering StateGuard for the first time may understand the risk/problem faster than the terminology.

            Something closer to “Keep AI workflows running when APIs and schemas change—without hiding what was repaired” communicates the outcome and the trust mechanism together.

            That's the kind of distinction I'd test with early users before settling the final positioning.

  11. 1

    The trust question seems more important than whether Semantic Repair is technically possible.

    From the developers using or evaluating StateGuard so far, have you seen anything in their actual behavior that tells you how much repair authority they're comfortable giving the system automatically?

    1. 1

      That's actually one of the biggest questions we're trying to answer right now.
      So far, the pattern seems to be that developers are comfortable giving the runtime authority over deterministic compatibility issues, but they want control the moment intent or business logic comes into play.

      That's exactly why I made this post - to figure out where people naturally draw that boundary before we build further in that direction.

      Have you run into any situations where you wanted the runtime to recover automatically instead of failing?

      1. 1

        That’s helpful context. The distinction between deterministic fixes and decisions involving intent is an interesting boundary to be exploring.

        I’d like to continue the conversation outside the thread. What’s the best email to reach you on?

        1. 1

          Absolutely. You can reach me at:
          adivaishnav.personal@gmail.com

          Or feel free to connect with me on LinkedIn as well: https://www.linkedin.com/in/adivaishnav

          Looking forward to continuing the conversation. We're still in the phase of talking to developers and shaping the roadmap around real-world feedback.

          1. 1

            Thanks! I’ve just sent it over.

            Looking forward to hearing your thoughts whenever you have a chance.