Skip to content

Preserve code spaces before punctuation#347

Open
mathitme wants to merge 1 commit into
basketikun:mainfrom
mathitme:fix-preserve-code-spaces
Open

Preserve code spaces before punctuation#347
mathitme wants to merge 1 commit into
basketikun:mainfrom
mathitme:fix-preserve-code-spaces

Conversation

@mathitme

@mathitme mathitme commented Jul 7, 2026

Copy link
Copy Markdown

Summary

  • Preserve ordinary spaces before punctuation in sanitized assistant output
  • Keep cleanup for spaces created when internal ChatGPT annotations are removed before punctuation
  • Add regression coverage for shell code such as find . and if !

Tests

  • UV_LINK_MODE=copy uv run python -m unittest test.test_chat_completion_cache

Copilot AI review requested due to automatic review settings July 7, 2026 12:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts sanitize_output_text so it no longer removes ordinary spaces before punctuation (important for preserving shell/code snippets), while still removing the extra space that can be left behind when an internal ChatGPT annotation is stripped immediately before punctuation.

Changes:

  • Remove the global “strip spaces before punctuation” cleanup to preserve code like find . and if ! ....
  • Add a targeted annotation-removal pass that drops only the whitespace that becomes orphaned when an annotation before punctuation is removed.
  • Add regression tests covering common shell/code spacing cases.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
services/protocol/conversation.py Refines output sanitization by removing the broad space-stripping rule and adding a punctuation-adjacent annotation cleanup.
test/test_chat_completion_cache.py Adds regression tests to ensure code-like spacing before punctuation is preserved.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

# ChatGPT web sometimes returns rich annotation markers using private-use
# characters. API clients cannot render those. Preserve readable labels
# from entity/link annotations, while removing internal citation pointers.
text = re.sub(r"(\s*)\ue200([^\ue201]*)\ue201(?=[.,;:!?])", replace_annotation_before_punctuation, text)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants