-
Notifications
You must be signed in to change notification settings - Fork 963
Description
Hello,
I’m looking for information on the correct way to re-IP an Apache Pulsar cluster using ZooKeeper and BookKeeper.
Before performing the re-IP, I wanted to make some configuration changes so that the BookieID uses <FQDN>:<port> instead of the default <IP>:<port>. To do this, I set useHostNameAsBookieID=true. However, after restarting the BookKeeper systemd service, the service fails to start and the following error appears in the logs:
2025-12-23T10:22:39,564 - ERROR [main:Main@218] - Failed to build bookie server
org.apache.bookkeeper.bookie.BookieException$InvalidCookieException: Cookie [5
bookieHost: “<hostname>.dummy.com:3181"
journalDir: “/data/bookkeeper/journal01"
ledgerDirs: "1\t/data/bookkeeper/ledger01"
instanceId: "9a36eaf5-564e-4b5d-9636-24b082125688"
] is not matching with [5
bookieHost: “10.0.0.1:3181"
journalDir: "/data/bookkeeper/journal01"
ledgerDirs: "1\t/data/bookkeeper/ledger01"
instanceId: "9a36eaf5-564e-4b5d-9636-24b082125688"
]
I tried the following steps to resolve the issue:
- Stopping the Bookkeeper systemd service
- Moving the Cookie/VERSION files from the journal01 and ledger01 directories to /tmp
- Deleting the old cookie using:
bin/bookkeeper shell cookie_delete 10.0.0.1:3181
- Running updateBookieInLedger with the old srcBookie and new destBookie
bin/bookkeeper shell updateBookieInLedger --srcBookie ’10.0.0.1:3181' --destBookie ‘<hostname>.dummy.com:3181' --updatespersec 5
- Generating a new cookie and coping the created file to the VERSION files for the journal01 and ledger01 directories.
bin/bookkeeper shell cookie_generate -j /data/bookkeeper/journal01 -l /data/bookkeeper/ledger01 -o /tmp/test-cookie <hostname>.dummy.com:3181
None of these steps unfortunately seemed to work. The only way I was able to recover was by clearing the entire journal and ledger directories and letting the node re-sync from the other bookies.
So my main questions are:
- What is the correct/recommended approach to re-IP Bookkeeper nodes while preserving existing ledger data?
- Are there any additional steps required when changing
useHostNameAsBookieIDon an existing cluster like clearing or updating the Cookie?
Some additional context about the environment:
Pulsar version: 3.0.5
Bookkeeper version: 4.16.5
Zookeeper version: 3.8.4
openJDK version: 17.0.17
Running on RHEL8 VM's
Thanks in advance!