Skip to content

fix(accumulator): sanitize invalid JSON in Input before marshaling#307

Open
jkakar wants to merge 1 commit intoanthropics:mainfrom
jkakar:jkakar/fix-accumulate-invalid-json
Open

fix(accumulator): sanitize invalid JSON in Input before marshaling#307
jkakar wants to merge 1 commit intoanthropics:mainfrom
jkakar:jkakar/fix-accumulate-invalid-json

Conversation

@jkakar
Copy link
Copy Markdown

@jkakar jkakar commented Mar 30, 2026

Problem

Message.Accumulate() fails with JSON marshaling errors when processing streaming responses where InputJSONDelta events produce truncated or malformed JSON. This happens because:

  1. InputJSONDelta events accumulate partial JSON strings into cb.Input (a json.RawMessage) by byte concatenation
  2. At ContentBlockStopEvent and MessageStopEvent, json.Marshal is called to populate JSON.raw for AsAny()
  3. Go's json.Marshal validates json.RawMessage contents via json.Compact, so if the accumulated bytes form invalid JSON, the marshal fails

This produces errors like:

  • json: error calling MarshalJSON for type json.RawMessage: unexpected end of JSON input
  • json: error calling MarshalJSON for type json.RawMessage: invalid character 's' after object key

The error is intermittent and depends on specific API response patterns where partial JSON chunks don't assemble into valid JSON (stream interruptions, network issues, etc.).

Solution

Add a sanitizeInputJSON helper that checks json.Valid() on the accumulated Input before marshaling. If the JSON is invalid, it replaces the bytes with null. This allows Accumulate() to succeed gracefully — callers can detect the issue by checking for a null Input on tool_use blocks.

The fix is applied at both ContentBlockStopEvent (per-block) and MessageStopEvent (all blocks) in both messageutil.go and betamessageutil.go.

Test plan

  • Added test case: tool use block with truncated input JSON ({"argument": without closing)
  • Added test case: tool use block with malformed input JSON ({"key s)
  • Both cases verify that Accumulate() succeeds and the resulting Input is null
  • All 12 existing TestAccumulate cases continue to pass

Fixes #255

When InputJSONDelta events are accumulated into a content block's
Input field (a json.RawMessage), the result may contain truncated
or malformed JSON if the stream is interrupted or the API sends
incomplete chunks. The subsequent json.Marshal call at
ContentBlockStopEvent and MessageStopEvent fails because
json.Compact validates the raw bytes.

Add a sanitizeInputJSON helper that checks json.Valid before
marshaling and replaces invalid Input with null. This allows
Accumulate to succeed gracefully -- callers can detect the issue
by checking for a null Input on tool_use blocks.

Fixes anthropics#255

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jkakar jkakar requested a review from a team as a code owner March 30, 2026 22:38
@jkakar
Copy link
Copy Markdown
Author

jkakar commented Mar 30, 2026

I'm not convinced this change is actually useful, because it doesn't address the core problem of why JSON is malformed, but I run into this periodically and have had to introduce defensive code in my application to deal with this problem so I decided to push this fix. This code was generated by Claude, and I've read and reviewed it but my overall understanding of the SDK is somewhat shallow so please apply skepticism as you read and contemplate whether incorporating this is useful or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming Accumulate() fails with 'invalid character' JSON error on malformed API responses

1 participant