Logical Validation

One thing I've been thinking about while building my AI Reliability Engine:

Schema validation is not enough.

Most validation systems can tell you:

• Is the output valid JSON?

• Are all required fields present?

• Are the data types correct?

But production failures often happen after that.

Example:

{
  "name": "John",
  "age": 900
}

This passes:

• JSON validation
• Required field checks
• Type validation

Yet it's obviously problematic.

The challenge I'm exploring now is logical validation.

Questions like:

• Does this value make sense?
• Does it violate business rules?
• Does it create downstream risk?
• Should the response be allowed, warned, or regenerated?

The tricky part is building this in a generic way.

For some use cases, age > 120 may be invalid.

For others, the field might not even be an age.

Unlike schema validation, logical validation depends heavily on context.

I'm curious how teams currently handle this when putting LLM outputs into production.

Do you rely on custom business rules, manual checks, secondary LLM validation, or something else?