Skip to content

Fix mysql reconnection test: proxy must close both sides on planned disconnect#49

Draft
Copilot wants to merge 2 commits intomasterfrom
copilot/fix-mysql-test-failure
Draft

Fix mysql reconnection test: proxy must close both sides on planned disconnect#49
Copilot wants to merge 2 commits intomasterfrom
copilot/fix-mysql-test-failure

Conversation

Copy link
Copy Markdown

Copilot AI commented Mar 2, 2026

pitr_binlog_server_reconnection_test.sh consistently failed with ~286/301 rows replicated because binlog_proxy.py was reconnecting only the server-side socket while keeping the MySQL client connection open.

Root cause

When the proxy triggered a planned disconnect, it closed and reopened its connection to the wal-g binlog server while holding the MySQL connection. The binlog server immediately issued a fresh MySQL protocol handshake on the new TCP connection; the proxy dutifully forwarded that handshake to MySQL—which was mid-stream expecting binlog events. MySQL's IO thread flagged a protocol violation, entered Connecting state, and never recovered through the same session.

Fix

On a planned disconnect, close both the client and server sockets and return from handle_client_connection. The outer start() accept loop then picks up MySQL's standard reconnect (governed by MASTER_CONNECT_RETRY/MASTER_RETRY_COUNT), which carries the correct GTID set and triggers a proper COM_BINLOG_DUMP_GTID against the binlog server's existing streamer.

# Before: only server socket closed, MySQL kept connected → protocol corruption
self.server_socket.close()
time.sleep(2)
if not self.connect_to_server():
    break
self.bytes_transferred = 0

# After: close both, return; outer loop accepts MySQL's clean reconnect
print(f"[Proxy] Closing both connections to force client reconnect")
return  # finally block closes both sockets

Because the 256-byte threshold fires during the TCP auth phase (before COM_BINLOG_DUMP_GTID), the two planned disconnects occur before replication actually starts. The third connection operates in stable mode and replicates all 301 rows cleanly.

Original prompt

This section details on the original issue you should resolve

<issue_title>[FEATURE] addressing mysql test failure</issue_title>
<issue_description>### What feature would you like?

we need to address the following failure https://github.com/wal-g/wal-g/actions/runs/22561524850/job/65349492528?pr=2187

Why is this feature useful?

No response</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

…nection testing

Co-authored-by: chipitsine <2217296+chipitsine@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix MySQL test failure in CI pipeline Fix mysql reconnection test: proxy must close both sides on planned disconnect Mar 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] addressing mysql test failure

2 participants