Skip to content

ForEach and Get disagree on duplicate object keys (parser smuggling vector) #393

Description

I'm Thiago (@TryZeroDay). I found
this in my first fuzzing campaign against gjson. I'm transparent
about my workflow: I directed the methodology, ran the campaign,
reproduced the bug, and minimised it manually — but I used an LLM
(Claude) to write the fuzzing harnesses and to help me draft this
report. I originally wrote it in Portuguese and translated it to
English.

Summary

I found that gjson.Result.ForEach and gjson.Get resolve a duplicated
object key to different values. ForEach yields every occurrence
in source order; Get returns the first occurrence only. The two APIs
are not internally consistent.

This opens a JSON parser smuggling vector when gjson is mixed with
other JSON parsers — notably the Go standard library, which keeps the
last duplicate instead.

Affected version

I tested against:

  • github.com/tidwall/gjson v1.19.0
    (commit 0fac2c9aa6eb5d5564bfaaaad513ce0d5d2314de)
  • Go 1.23 / linux-amd64

Minimal reproducer

package main

import (
    "fmt"
    "github.com/tidwall/gjson"
)

func main() {
    const json = `{"a":1,"a":2}`

    gjson.Parse(json).ForEach(func(k, v gjson.Result) bool {
        fmt.Printf("ForEach  key=%q  value=%s\n", k.String(), v.Raw)
        return true
    })
    fmt.Printf("Get(\"a\")  value=%s\n", gjson.Get(json, "a").Raw)
}

Output

ForEach  key="a"  value=1
ForEach  key="a"  value=2
Get("a")  value=1

ForEach exposes both 1 and 2; Get resolves only to 1. The two
APIs cannot both be correct simultaneously for any single
key-resolution policy.

Why I think this matters

RFC 8259 §4 leaves the meaning of duplicate object members undefined
and warns implementations that interoperability is at risk if they
diverge. gjson itself diverges internally, and across the
ecosystem:

Parser Resolution
gjson.Get First occurrence
gjson.ForEach All, in order
Go encoding/json Last
JavaScript JSON.parse Last
Python json.loads Last

This is the same shape as published vulnerabilities:

  • CVE-2017-12635 — Apache CouchDB (admin role injection via
    duplicate roles key; CVSS 9.8 critical)
  • CVE-2022-23541 — Auth0 jsonwebtoken (JWT verification bypass
    via parser disagreement; CVSS 7.6 high)
  • CVE-2020-27619 — Python (multiple parser-disagreement issues)

Concrete attack scenario I built to demonstrate it

package main

import (
    "encoding/json"
    "fmt"
    "github.com/tidwall/gjson"
)

// authorize uses gjson on the hot path.
func authorize(payload string) string {
    return gjson.Get(payload, "role").String()
}

// auditLog uses encoding/json (already wired into the audit pipeline).
func auditLog(payload string) string {
    var m map[string]interface{}
    _ = json.Unmarshal([]byte(payload), &m)
    if r, ok := m["role"].(string); ok {
        return r
    }
    return ""
}

func main() {
    payload := `{"role":"admin","role":"user"}`
    fmt.Println("authorize() sees:", authorize(payload))  // admin
    fmt.Println("auditLog()  sees:", auditLog(payload))   // user
}

For payload {"role":"admin","role":"user"}:

Function Sees
authorize() "admin" (gjson returns the first occurrence)
auditLog() "user" (encoding/json keeps the last)

The attacker is granted admin access while the audit log records a
user-level action.

How I found it

I ran a libFuzzer + AddressSanitizer campaign with 13 harnesses spread
across 3 Docker containers (~103k execs/s combined), built on top of
the Google OSS-Fuzz tooling. One of the harnesses I wrote checks
ForEach/Get consistency on randomly-generated valid JSON. It
panicked on a 733-byte input after ~750k executions; I minimised the
crashing input manually to the 13-byte case above.

By the time this bug surfaced the campaign had tested roughly 38
million JSON documents.

Suggested resolutions

Any of these would close the issue from my point of view:

  1. Align Get with encoding/json — return the last occurrence
    of a duplicated key. This brings gjson into alignment with the
    wider ecosystem (stdlib encoding/json, browser JSON.parse,
    Python json.loads) and removes the cross-library smuggling vector.
  2. Document the divergence prominently in the README, in a
    "Duplicate keys" section, with a clear warning about parser
    smuggling when gjson is mixed with other JSON parsers.
  3. Add a strict mode — for example gjson.ParseStrict(json) that
    returns an error on duplicate keys, plus a gjson.ValidStrict
    predicate. This gives security-sensitive callers a way to reject
    ambiguous input.

A combination of (1) and (3) would be ideal in my opinion.


Thiago — @TryZeroDay.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions