Quality Degradation with Complex Prompts Under Concurrent Load

[rcoenen]

Quality Degradation with Complex Prompts Under Concurrent Load

Context

Processing PDF forms to markdown via the Z.AI API (https://api.z.ai/api/paas/v4/chat/completions). PDFs are converted to 150 DPI WEBP bitmaps (one per page) and sent to GLM-4.6V-Flash (9B) for concurrent extraction to achieve cost-effective high-volume processing.

Per API documentation, GLM-4.6V-Flash supports 3 concurrent requests - that's exactly what we're testing with.

Problem

Complex prompts cause ~50% output degradation only when processing multiple pages concurrently. Single-page and sequential processing work perfectly with identical prompts.

Reproduction

Test Setup:

• Model: GLM-4.6V-Flash (9B) via Z.AI API
• Input: 3 pages as 150 DPI WEBP images
• Concurrency: 3 parallel API requests (documented limit)
• Parameters: temperature=0.0, max_tokens=4000

Simple Prompt (works):

Extract all content as Github-flavored markdown.
FORMAT: headings, lists, field values in backticks

Complex Prompt (fails concurrently):

Extract all content as Github-flavored markdown.

FIRST: Output page type in 3-6 words as: **PAGE_TYPE**: `type`

FORMAT:
- Headings: ## main, ### sub
- Radio: (x) selected, ( ) unselected
- Checkboxes: [x] checked, [ ] unchecked
- Fields: `value` or `` if empty
- Dates: DD-MM-YYYY format
[...detailed radio button, checkbox, date formatting rules]

Results:

Prompt	Processing Mode	Output Tokens	Quality
Simple	Concurrent (3 pages)	4,455	✅ 100%
Complex	Concurrent (3 pages)	2,510	❌ 56%
Complex	Sequential (1 at a time)	4,472	✅ 100%
Complex	Single page	1,664	✅ 100%

Validation with GLM-4.6V paid (106B) via the same Z.AI API: Same degradation (2,215 tokens, 50%), confirming this affects the entire GLM-4.6V family on their API.

Characteristics

• Trigger: Complex multi-task prompts (classification + extraction + detailed formatting) + concurrent API requests
• Operating within limits: Using exactly 3 concurrent requests as documented
• Repeatable: Consistent ~50% degradation across multiple test runs
• No errors: Requests succeed with normal finish_reason, output is just truncated
• Not rate limiting: No 429 errors or rate limit responses

Impact

Forces choosing between:

1. Sequential processing (3x slower, defeats Flash cost advantage)
2. Simplified prompts (loses classification and formatting capabilities)

Questions

1. Is this expected behavior for concurrent API requests with complex prompts?
2. Does the backend apply adaptive truncation under concurrent load?
3. Are there documented prompt complexity limits for concurrent usage?

Workaround

Removing the PAGE_TYPE classification task and simplifying formatting rules restores full quality in concurrent mode.

---

Note: GLM-V docs mention "overthinking with complex prompts" but that doesn't explain this issue since sequential/single-page processing with the same complex prompts produces perfect output.