fix(docx): break Hangul lines at eojeol boundaries, not syllables

[developer0hye]

Summary

Word breaks Korean at eojeol (space-delimited word) boundaries on an ordinary line, falling back to syllable breaking only to keep a justified line from going loose. We broke every syllable, so the last word of a line was split and one syllable stranded above: the contract's 제1조 ended …사항을 정 where Word ends …제반 사항을, with 20pt of room still on the line. The same file's justified paragraphs split 보관|한다 on both sides, which is what identifies syllable breaking as Word's fallback rather than its default.

Typst 0.14 offers no lever for this — text(lang: "ko"), par(linebreaks:) and text(costs:) were each measured to leave the breakpoints untouched, because typst-layout builds its ICU4X segmenters with default options and never consults Lang::KOREAN. A frame is the one thing UAX #14 cannot break through, so each qualifying eojeol is emitted inside an inline #box — the mechanism the repo already uses in the opposite direction for PowerPoint kinsoku. A no-break marker between syllables also works and is rejected: U+2060 lands in the text layer and makes the text unsearchable (#664, reproduced on a probe before choosing).

Supporting work the measurements forced:

• Baselines. Under the fixed line box a bare frame seats its baseline at the box's bottom edge, so the frame restores the paragraph's own text edges and shifts its baseline by the descent, resolved against the token's largest run. Not one baseline moves on any Korean fixture (1151 shared baselines on the 37-page technical brief, zero added or dropped).
• Markup leaks. Splitting runs at eojeol boundaries makes each inter-word fragment its own escaping unit, where a leading + or = became Typst markup — the + silently replaced by an enum marker, the = deleted from page and text layer. escape_typst now neutralises the whole line-leading marker set through one leading space; the character census over the brief is identical to main.
• Width guard. Framing applies only to tokens measuring narrower than their container, with the measure taken from the page or the cell's own column, so a token too wide for a narrow column keeps today's breaking instead of taking a line of its own.

Gated to DOCX flow pages and non-justified paragraphs. Lists follow their item's alignment; floating text boxes are deliberately excluded and documented as such.

Related issue

Fixes #626

Testing

• cargo test --workspace — green (2211 passed). 13 red-first tests: eojeol framing in paragraphs/cells/lists, the baseline-restoration arithmetic (triangulated at two font sizes), every line-leading marker through one leading space, the width guard (same token framed at 400pt, unframed at 150pt), and negative guards for justified / slide / sheet / Latin.
• cargo +1.97 clippy --workspace --all-targets — clean; cargo fmt --check — clean; wasm check clean.
• Corpus sweep, 377 files: only the five Korean DOCX change. Every PPTX (92) and XLSX (124) is byte-identical.
• Text layer: pdftotext character census over the 37-page Korean brief is identical to main — zero Cf characters, no U+2060/U+200B.
• Perf: 2.44s → 2.47s on the brief (~1.3%).

Visual impact

• No rendered PDF change
• Rendered PDF change or visual evidence added
• Reason:

Visual audit

• Issue: DOCX: Hangul lines break at syllable boundaries, splitting words Word keeps whole #626
• Fixture: tests/golden_mocks/business/sources/docx/02_contract_ko.docx
• Page(s): 1
• Renderer and DPI: pdftoppm, 150 DPI
• Evidence mode: fix
• New follow-up issues found in this audit: None
• GT: assets/bugfixes/issue-626/gt.jpg
• Before: assets/bugfixes/issue-626/before.jpg
• After: assets/bugfixes/issue-626/after.jpg

Visual comparison

GT	Before	After

Required inspection

• Rendered all evidence at 150 DPI or higher
• Stored progressive JPEG quality 86 assets with metadata stripped
• Inspected matched region crops at full resolution
• Ran the 5% fuzz pixel-difference sweep
• Inventoried hairlines and border dash styles
• Inventoried font weight, italic, and underline emphasis

Deviation audit

Check	Result
Page count/order	Matches GT — one page each, no reordering; media box within 0.1pt (A4 rounding)
Element presence	Matches GT — 21 matched lines, 0 missing, 0 extra, 0 re-wrapped; all rules, shading and footer present (GT's higher rect census is per-segment op-splitting, confirmed visually)
Position/size	Remaining: #724 — table borders 0.48–0.96pt left/up (column proportions exact: divider span identical at 225.60pt); Remaining: #649 — cell text baselines 0.50–0.59pt high; Remaining: #646 — footer 0.96pt right of centre. Every other line is within one 150-DPI pixel
Rotation/flip	No deviation observed — every transform axis-aligned in both traces
Fill	Matches GT — header shading identical to 8 decimals; extent within 0.48pt, tracking the #724 border offset; colour shift 0.0001
Stroke/border	Matches GT — five solid rules in GT, five present, no dash deviations; the apparent width difference is GT's two-pixel antialiasing vs our one-pixel snap at equal integrated ink
Text content	Fixed — 제1조 ends '…필요한 제반 사항을' and 제4조 ends '…VAT 별도)으로', both matching GT (main: '…사항을 정' / '…으로 하'); page-1 text is line-for-line identical to GT; AE 5% 50,388 → 47,825
Font family/weight/style	Remaining: #628 — family and weight match GT exactly (bold on precisely the two w:b runs, per-line ink mass within 2%), but pair kerning tightens V→A/A→T by 0.66/0.81pt, leaving −1.41pt across the rest of the line
Text color	Matches GT — identical colour census
Alignment	Remaining: #728 — the centred date takes a 0.25em auto-space at each digit→Hangul boundary Word leaves alone (+2.62pt × 3); Remaining: #646 — centred footer 0.96pt right. Left-aligned and both justified paragraphs land on GT within 0.48pt
Line/paragraph spacing	Remaining: #630 — the footer baseline sits 1.92–2.10pt low, making the table→footer gap +2.69pt; all 19 other pitches are within ±0.16pt of GT
Clipping/overflow	No deviation observed — no glyph crosses the text edge, no cell truncated

Note for reviewers: 10_research_report_ko page 2 goes 37/37 → 35/37 matched lines. That is entirely one note paragraph which Word itself breaks mid-eojeol; the fixture defines no Normal style, so its unstyled paragraphs inherit the Word application's wordWrap=0 default, which we cannot honour until w:wordWrap is parseable — tracked in #730 together with the native-Word one-factor probes that identified the discriminator.

Checklist

• Commits include a Signed-off-by line
• PR scope contains one root cause
• Remaining visual deviations each reference an open issue

🤖 Generated with Claude Code