fix(accumulator): sanitize invalid JSON in Input before marshaling#307
Open
jkakar wants to merge 1 commit intoanthropics:mainfrom
Open
fix(accumulator): sanitize invalid JSON in Input before marshaling#307jkakar wants to merge 1 commit intoanthropics:mainfrom
jkakar wants to merge 1 commit intoanthropics:mainfrom
Conversation
When InputJSONDelta events are accumulated into a content block's Input field (a json.RawMessage), the result may contain truncated or malformed JSON if the stream is interrupted or the API sends incomplete chunks. The subsequent json.Marshal call at ContentBlockStopEvent and MessageStopEvent fails because json.Compact validates the raw bytes. Add a sanitizeInputJSON helper that checks json.Valid before marshaling and replaces invalid Input with null. This allows Accumulate to succeed gracefully -- callers can detect the issue by checking for a null Input on tool_use blocks. Fixes anthropics#255 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Author
|
I'm not convinced this change is actually useful, because it doesn't address the core problem of why JSON is malformed, but I run into this periodically and have had to introduce defensive code in my application to deal with this problem so I decided to push this fix. This code was generated by Claude, and I've read and reviewed it but my overall understanding of the SDK is somewhat shallow so please apply skepticism as you read and contemplate whether incorporating this is useful or not. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Message.Accumulate()fails with JSON marshaling errors when processing streaming responses whereInputJSONDeltaevents produce truncated or malformed JSON. This happens because:InputJSONDeltaevents accumulate partial JSON strings intocb.Input(ajson.RawMessage) by byte concatenationContentBlockStopEventandMessageStopEvent,json.Marshalis called to populateJSON.rawforAsAny()json.Marshalvalidatesjson.RawMessagecontents viajson.Compact, so if the accumulated bytes form invalid JSON, the marshal failsThis produces errors like:
json: error calling MarshalJSON for type json.RawMessage: unexpected end of JSON inputjson: error calling MarshalJSON for type json.RawMessage: invalid character 's' after object keyThe error is intermittent and depends on specific API response patterns where partial JSON chunks don't assemble into valid JSON (stream interruptions, network issues, etc.).
Solution
Add a
sanitizeInputJSONhelper that checksjson.Valid()on the accumulatedInputbefore marshaling. If the JSON is invalid, it replaces the bytes withnull. This allowsAccumulate()to succeed gracefully — callers can detect the issue by checking for a nullInputon tool_use blocks.The fix is applied at both
ContentBlockStopEvent(per-block) andMessageStopEvent(all blocks) in bothmessageutil.goandbetamessageutil.go.Test plan
{"argument":without closing){"key s)Accumulate()succeeds and the resultingInputisnullTestAccumulatecases continue to passFixes #255