ValueError
builtins.ValueError
Stack trace
Traceback (most recent call last):
File "app.py", line 42, in <module>
response = client.chat.completions.create(model="together-gpt", messages=messages, stream=True)
File "/usr/local/lib/python3.9/site-packages/togetherai/client.py", line 120, in create
parsed = json.loads(chunk)
ValueError: Expecting value: line 1 column 1 (char 0) Why it happens
Together AI streams partial JSON chunks that may be incomplete or split across multiple events. Attempting to parse these partial chunks as full JSON causes ValueError. This happens because the streaming data is not buffered or concatenated before parsing.
Detection
Monitor your streaming response handler for JSON parsing exceptions and log raw chunks received to identify incomplete or malformed JSON before the error crashes your app.
Causes & fixes
Parsing each streamed chunk as a complete JSON object without buffering.
Accumulate streamed chunks until a full JSON object is received before calling json.loads().
Together AI streaming response includes control characters or empty chunks.
Filter out empty or whitespace-only chunks before parsing, and handle control characters gracefully.
Using a synchronous JSON parser on asynchronous streaming data.
Use an incremental JSON parser or a streaming JSON parser library that supports partial data.
Code: broken vs fixed
from togetherai import TogetherAI
import os
client = TogetherAI(api_key=os.environ["TOGETHER_API_KEY"])
response = client.chat.completions.create(
model="together-gpt",
messages=[{"role": "user", "content": "Hello"}],
stream=True
)
for chunk in response:
data = json.loads(chunk) # This line causes ValueError on partial chunks
print(data)from togetherai import TogetherAI
import os
import json
client = TogetherAI(api_key=os.environ["TOGETHER_API_KEY"])
response = client.chat.completions.create(
model="together-gpt",
messages=[{"role": "user", "content": "Hello"}],
stream=True
)
buffer = ""
for chunk in response:
if not chunk.strip():
continue
buffer += chunk
try:
data = json.loads(buffer)
print(data)
buffer = "" # Reset buffer after successful parse
except json.JSONDecodeError:
# Wait for more chunks to complete JSON
continue # Keep bufferingWorkaround
Wrap the json.loads call in try/except JSONDecodeError, accumulate chunks in a buffer, and parse only when the buffer contains a complete JSON string.
Prevention
Use a streaming JSON parser or Together AI SDK utilities designed for streaming responses that handle chunk assembly and parsing internally to avoid manual parsing errors.