Issues / #2893
#2893 [Feature Request] Opt-in LaTeX-safe JSON recovery for json_object responses (LightRAG VLM)
open · @BoysGameStudio · 0 comments · View on GitHub
Description
## Summary
When a vision-language model generates JSON containing an unescaped LaTeX
backslash, Strata's `response_format: {"type":"json_object"}` validation
rejects the completion with HTTP 502 (`structured_output_failed`).
This breaks LightRAG's multimodal PDF ingestion because LightRAG already
implements tolerant JSON parsing and LaTeX escape recovery, but Strata
rejects the response before LightRAG can access the generated text.
The issue has been reproduced using the same image and prompt.
## Environment
- Strata v0.1.44.1 (Strata-NVFP4 fork)
- Qwen3.8-Flash-Next NVFP4
- GPU vision enabled
- LightRAG v1.5.7
- MinerU PDF pipeline
- OpenAI-compatible Chat Completions API
## Minimal Reproduction
Model-generated content:
```text
{"name":"alpha coefficient","type":"diagram","description":"Coefficient \alpha_0"}
```
The sequence `\a` is not a valid JSON escape.
This can also be reproduced directly against Strata's validator:
```python
from serve.structured import _ObjectOnly, validated_json
raw = r'{"name":"alpha coefficient","type":"diagram","description":"Coefficient \alpha_0"}'
print(validated_json(raw, _ObjectOnly(), "stop"))
```
### Actual Behavior
- Strict JSON parsing fails.
- Strata returns HTTP 502 (`structured_output_failed`).
- The original generated response is unavailable to LightRAG.
- LightRAG retries the failed provider request.
- The document eventually enters the FAILED state.
### Expected Behavior
An optional recovery mode could repair unambiguous LaTeX escaping errors
before strict validation.
For example:
```json
{"name":"alpha coefficient","type":"diagram","description":"Coefficient \\alpha_0"}
```
The repaired output should still pass strict JSON validation before
being returned to the client.
## Proposed Solution
Introduce an opt-in, conservative JSON recovery mechanism for
`response_format=json_object`.
Suggested behavior:
1. Preserve existing strict validation by default.
2. Attempt limited LaTeX-aware repair when explicitly enabled.
3. Validate the repaired JSON using the existing validator.
4. Return canonical JSON only after successful validation.
5. Continue returning HTTP 502 for unrecoverable output.
6. Preserve existing `json_schema` behavior.
The implementation must also account for LaTeX commands such as:
- `\alpha`
- `\beta`
- `\frac`
- `\tau`
- `\nabla`
Some commands conflict with valid JSON escape sequences, so simply
doubling every backslash is unsafe.
Regression tests should cover legitimate JSON escapes, already-correct
LaTeX escaping, duplicate keys, invalid constants, and truncated output.
## Related Issues and PRs
- Strata #2368 — Grammar-Constrained Sampling
- Strata #1328 — Token Masks for JSON Formats with MTP
- LightRAG PR #3238 — Repair LaTeX Escape Damage in LLM JSON Responses
https://github.com/HKUDS/LightRAG/pull/3238
- LightRAG PR #3242 — Retry Malformed Multimodal Analysis JSON
https://github.com/HKUDS/LightRAG/pull/3242
Constrained decoding may prevent syntactically invalid JSON, but it does
not necessarily preserve LaTeX semantics when a model emits JSON-valid
escape sequences.
This feature would improve compatibility with LightRAG and other
applications processing mathematical or technical documents.Related on this site
Quick links to related pages.