Skip to content

JSON encoder: guard the protocolData parse and drop the parse/re-serialize round trip #48

Description

@Thecave3

The JSON path re-parses and re-serializes every SM payload on its way into the E3AP indication
envelope. That costs ~7 µs per indication for no benefit, and the parse is unguarded so a malformed
payload throws out of the encode path. Both live in src/encoder/json_encoder.cpp.

1. Unguarded parse (robustness)

encode_indication_message() (src/encoder/json_encoder.cpp:154):

nlohmann::json JsonE3Encoder::encode_indication_message(const IndicationMessage& msg) const {
    nlohmann::json j;
    j["dAppIdentifier"] = msg.dapp_identifier;
    j["ranFunctionIdentifier"] = msg.ran_function_identifier;
    j["protocolData"] = nlohmann::json::parse(msg.protocol_data);   // throws on bad input
    return j;
}

nlohmann::json::parse throws nlohmann::json::parse_error by default. A service model that emits
malformed JSON (a truncated buffer, an unescaped string, a field added without updating the
serializer) therefore throws from inside the encoder rather than getting a diagnosable error. Other
functions in this file do handle exceptions (:382, :466, :470), so this looks like an
oversight rather than a decision.

Suggested: parse with nlohmann::json::parse(..., nullptr, false) (non-throwing) or wrap in
try/catch, log the offending RAN function id and payload size, and fail the encode cleanly.

2. The round trip (latency)

protocolData is the only field that is parsed purely to be serialized again. The payload arrives
as a finished JSON byte string from the SM, gets turned into a DOM, and is then dumped straight back
out as part of the envelope. The decode side mirrors it — decode_indication_message() at
src/encoder/json_encoder.cpp:269 does j["protocolData"].dump() to hand the payload up.

Measured with nlohmann/json 3.11.3, -O2, 200k iterations, on a 184-byte payload representative of
an IQ-metadata indication (shm_name, buffer/write index, timestamp, sfn, slot, cell id, antenna
count, symbol mask):

Send-side approach Cost per indication
today: json::parse(payload) + envelope dump() 7.3 µs
…of which the payload parse alone 4.8 µs
verbatim splice of the payload into the envelope string 0.08 µs

So ~90× cheaper, and byte-for-byte identical output (verified by comparing both envelopes for the
same input). The envelope is trivially shaped:

{"dAppIdentifier":<n>,"ranFunctionIdentifier":<n>,"protocolData":<payload verbatim>}

Suggested: build the indication envelope by concatenation on encode, and extract the raw
protocolData substring on decode.

Caveat worth planning for: nlohmann keeps no source offsets on a parsed DOM, so the decode side
cannot recover the original substring from j — it needs either a small scan for the value bounds
or a SAX handler. If the encode side splices but the decode side still calls .dump(), the payload
is re-serialized on ingress and the saving is only half realised.

Validation trade-off: today's parse incidentally validates the SM payload. Splicing without
validating moves a malformed payload's failure to the receiver, where it is harder to attribute. If
that matters, parse once purely to validate and still splice the original bytes — that keeps ~4.8 µs
but drops the dump and preserves the payload bytes exactly. Given this is the hot indication path,
splicing unvalidated (with item 1 fixed for the paths that do parse) seems the better default.

Not proposing: nlohmann::ordered_json

A side effect of the current round trip is that payload keys come out alphabetically sorted,
because nlohmann::json's object type is std::map. That is not a bug — JSON objects are unordered
per RFC 8259 and every consumer parses into a dict — so it is not a reason to change anything on its
own. For the record, switching the encoder to nlohmann::ordered_json would preserve insertion
order but measured slower (8.1 µs/indication vs 7.3), since key lookup becomes a linear scan.
The splice above preserves order anyway, as a free side effect.

Scope and impact

  • Confined to src/encoder/json_encoder.{hpp,cpp}; nlohmann does not appear in any public header,
    so there is no API/ABI change and the C API and SWIG bindings are unaffected.
  • The ASN.1 and protobuf paths already keep the payload opaque (e.g. asn1_encoder.cpp:594 just
    assigns the octet-string bytes), so they are not affected by either item.
  • No wire-format change: the emitted envelope is byte-identical for well-formed payloads.

Acceptance

  • Malformed SM payload produces a logged error and a failed encode, not an exception escaping
    the encoder.
  • Encode and decode preserve the SM payload bytes end to end.
  • Benchmark before/after recorded in the PR.
  • ctest green; C↔Python interop unchanged for asn1, json and protobuf.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions