Add YAML CST as a pipeline source and harden the CST surface (kyo-schema)#1655
Merged
fwbrasil merged 56 commits intoJun 14, 2026
Merged
Conversation
c295f4d to
fd4d863
Compare
9325897 to
d535bac
Compare
Detect duplicate mapping keys along an edit path and fail instead of silently editing the first match. Make insert fail on an existing key rather than overwriting it. Render complex collection keys as flow YAML instead of emitting a node toString in the trivia path.
Process the same document bodies the source-backed stream parser produces by dropping a leading marker-only segment, so processor and no-processor cstAll return matching document counts. Replace the hand-rolled Result sequencing and its synthetic exception with a plain Result match, and document that cstAll returns every document.
Document that source-backed render ignores WriterConfig until edited, comment the unreachable event-emission failure boundaries and the known collection size hint, convert appendIndent to a tail-recursive loop, and skip the clearSource rebuild when a node carries no source.
Replace multiple per-test asserts with grouped assertResult blocks over named tuples, matching the existing YAML test style.
Collapse tests with multiple per-test asserts into single grouped assertResult blocks over named tuples, matching the existing YAML test style. Stateful reader checks hoist intermediate reads into ordered vals to preserve evaluation order. No assertions dropped or weakened.
Add emitDocumentBody and emitStream to YamlCstBuilder. emitDocument now delegates its body emission to emitDocumentBody. emitStream emits one stream boundary around all document bodies via a tail-recursive helper.
Add visit/render/parse overloads to Pipeline for Cst.Document and Cst.Stream inputs, and a parseAll overload for Cst.Stream. The stream render path joins documents with --- separators, mirroring YamlCstRenderer.stream, and short-circuits to the original source when no processors are attached.
Add the stream-end `...` marker to render(stream) when documentMarkers is StartAndEnd, fix the scaladoc to explain the real reason for the per-document render-and-join approach, avoid allocating a child Pipeline for single-document streams, expand scaladocs on the CST visit overloads, and strengthen the parseAll test with concrete Mapping type assertions.
Add `throughCst` and `throughCstStream` builder methods on `Pipeline[Err]`. When set, String read terminals (decode/render/parse) build a source-backed CST, apply the stream transform then the per-document transform, and continue through event processors. The fast path (no CST transform) is unchanged.
Add a test exercising throughCstStream + throughCst together, proving stream transform runs before per-document transform on survivors. Also consolidate the duplicated widening closures in throughCst/throughCstStream to use the shared documentTransformWiden/streamTransformWiden helpers, and add a clarifying comment on the per-document render path.
Rebasing onto the kyo-native test framework (getkyo#1658) removed ScalaTest's assertResult. Convert the 182 assertResult calls across the YAML test suites to kyo-test assert(actual == expected), splitting named-tuple comparisons into one assertion per field. Helper methods that call fail or assert outside a leaf body now take a using kyo.test.AssertScope. Fix two failures the migration surfaced: restore the SOH control byte in the writer control-character test, and compare the pipeline single-document error by message substring instead of full equality, since ParseException messages embed the call-site frame.
Bind the per-document parse result to a typed val so the exhaustivity checker can resolve the Success/Failure/Panic match on the opaque Result type. The rebased compiler emitted an E029 warning on the inferred flatMap result type at this site; the sibling mapDocuments loop already matches on a cleanly typed scrutinee and does not warn.
f4ff51c to
9b15e3c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
kyo-schema gained YAML support, including an opt-in concrete syntax tree (CST) that preserves comments, trivia, scalar styles, and source spans. Two gaps remained after the initial CST work:
toString, and processor-backedcstAllcould diverge in document count from the source-backed path.Yaml.Cst.Document/Yaml.Cst.Streamas a decode/render/visit source, so a parsed-and-edited CST had to be rendered back to text and re-parsed to decode it.Solution
Two themes.
CST hardening:
|/>) on the trivia path instead of double-quoting them, reusing the writer'sappendBlock(with aheaderSuffixso trailing comments survive on the header line).insertfails on an existing key rather than overwriting.toString.cstAllwith the source-backed path (drop a leading marker-only segment), and document thatcstAllreturns every document.assertResultblocks over named tuples.CST as a source (new capability):
YamlCstBuilderemits events from a CST (emitDocumentBody/emitStream).Yaml.Pipelinegains CST-source overloads:visit,render,parse,parseAll,decode,decodeAlloverCst.Document/Cst.Stream. Stream decoding honorsReaderConfig.documentIndex/documentModeexactly as the String path (same precedence, same error messages), including a CST-level top-level-mapping merge.throughCst(per document) andthroughCstStream(whole stream), composed CST-first: the String read terminals build a source-backed CST, apply the stream transform then the per-document transform, emit events, then run any event processors. This lets a pipeline edit structure with comments intact and decode/render the edited result in one pass.Yaml.decode/decodeAll/render/parse/parseAllover a CST, delegating to the default pipeline.Notes
Yaml.decode(String)andYaml.pipeline.decode/render/parse(String)are byte-identical to before and construct no CST. This is pinned by a regression test.throughCst-stagedrenderuses the trivia-aware CST renderer (not the event renderer) so comments on unedited nodes survive;render(stream)re-derivesYamlCstRenderer.streamframing (document markers,---/...separators). A stream transform clears the stream'soriginalSourceso a structural edit (e.g. dropping a document) cannot render stale source.Yaml.render(doc)always emits canonically through the writer;Cst.Document.renderreturns the original source verbatim until edited. Both coexist and are documented.MergeTopLevelMappingsis structural (skips non-mapping-rooted documents), which differs from the byte-level String merge; this is documented and tested.kyo-schema/doctest(66 blocks), andkyo-bench/compileall green;git diff --checkclean.