fix: correct CID TrueType text extraction for Identity-H fonts#75
Merged
Conversation
…oUnicode CMap Five root causes of raw glyph IDs being returned instead of Unicode text: 1. parseBfRange array format [<dst0> <dst1>...] was skipped with 'continue'. Now fully parsed: each source code maps to its explicit destination rune. 2. No-ToUnicode early-return paths hardcoded use2ByteGlyphs=false even when encoding was Identity-H/V. Now detects Identity encoding and Type0 subtype and sets use2ByteGlyphs=true on all early-return paths. 3. begincodespacerange was not parsed. CMapTable gains CodeBytes field (default 1). Parser reads the hex-string length from the range to determine 1-byte vs 2-byte codes and stores it in CodeBytes. loadFontDecoder promotes use2ByteGlyphs when CodeBytes==2, even without an Identity encoding name. 4. UTF-16BE surrogate pairs in bfchar/bfrange destinations were silently truncated to a single uint16. New decodeUTF16BEHex helper detects 4-byte (8-hex-char) sequences, checks for high-surrogate (0xD800-0xDBFF), and decodes via unicode/utf16.DecodeRune. Both parseBfChar and parseBfRange now use this helper for all destination parsing. 5. Garbage-detection fallback in DecodeString downgrades 2-byte composite font decoding to 1-byte, silently corrupting all CID glyph IDs above 0xFF. FontDecoder gains isCompositeFont bool field. When true the fallback is suppressed entirely. loadFontDecoder sets the flag for Type0 fonts. Tests: 33 new tests covering all five bugs plus integration test with a synthetic Type0/CIDFontType2 PDF using bfrange array format.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix CID TrueType font text extraction that returned raw glyph IDs (
\x01 \x02 \x03) or empty strings instead of Unicode text for PDFs with Identity-H encoding and ToUnicode CMap.Five bugs fixed in the CMap/font decoder pipeline:
beginbfrangearray format — previously silently skipped, now fully parsedloadFontDecoderfallback paths now honor 2-byte modebegincodespacerangeparsing — byte width determined from PDF spec signallooksLikeGarbageheuristic no longer downgrades Type0 fontsFixes #74
Test plan
go fmt ./...cleango vet ./...cleango test ./...— 23 packages, all passinggolangci-lint run— no new issues