feat: support java bindings & cross-platform fat jar compilation#711
feat: support java bindings & cross-platform fat jar compilation#711PongPong wants to merge 12 commits into
Conversation
Updated test/test_string_utils.c to include the renamed libpostal_features.h instead of features.h
Updated test/Makefile.am to reference the renamed libpostal_features.c instead of features.c in test_libpostal_SOURCES
|
this seems like an A.I. pull request (fine for small changes, tests, and bugfixes but not major refactors, etc.). This is the C repo and should not contain any binding-specific code or build steps. The Java repo is at: https://github.com/openvenues/jpostal so anything related to JNI bindings should be raised there. Adding a new build system in a different language when autotools is more standard and works fine seems weird to me although in a future version I'm experimenting with doing most/all of the config/cross-platform stuff with the preprocessor |
|
@albarrentine, may I submit an MR to resolve the src/features.[c|h] conflicts and the missing drand48() issues on Windows? I wasn’t aware of the jpostal repository before, but I’m planning to submit an MR to pull in the C source code, build it with Zig, and publish it to the Maven repository. What do you think? |
|
There's some discussion on Maven packaging in jpostal and it looks like there's a working implementation though we're just waiting on a pull request (looking over it briefly I wasn't sure why the compiled shared objects needed to be checked in to the fork but not a Java/Maven user): openvenues/jpostal#7 (comment) jpostal currently uses Gradle for the build and I think it's already working on Windows. Introducing Zig just as a build system in a non-Zig project seems like a solution looking for a problem IMHO. It's adding several hundred lines of code that need to be maintained alongside autotools and it's not clear what the performance implications are i.e. if Zig's C compiler (which is not even 1.0 yet) will produce the level of optimizations that battle-tested compilers like gcc or clang do and am not going to make that choice for everyone by default. There's a listing of C projects built with Zig at https://github.com/allyourcodebase, and there's not currently a Zig interface to libpostal, so you're welcome to make one and we'll add to the README. For libpostal and other C projects the change I'm looking to introduce to the build system is removing steps from the build so it can be run with just drand48 for Windows issue is addressed by the existing autotools build. Since Windows support was initially added, the C lib now includes an implementation of drand48 and checks for it during the configure step. For Windows need to make sure to use these build instructions: https://github.com/openvenues/libpostal?tab=readme-ov-file#installation-windows. In a future version will likely rip out the klib shuffle implementation and use the xorshift implementation for the random number generator (https://github.com/goodcleanfun/random - this is tested on Linux/Mac/Windows even MSVC). Haven't seen features.{hc} come up as a bug. The other issue that needs to be resolved on the Java/jpostal side if interested is to consolidate the two PRs re: Java's "unique" take on UTF-8. There's one here: openvenues/jpostal#38 and another by the Overture Maps folks here: OvertureMaps/jpostal#1 so looking to consolidate those ideas into one PR and will merge that. |
📋 What's New
1. Zig Build System (
build.zig) - New FileFeatures:
zig build cross) for all 6 platformslibpostal) and JNI wrapper (libpostal_jni)-Os) with dead code eliminationArchitecture:
buildLibrary(),buildJNILibrary(),configureLibrary()Source List:
Includes 43 source files from the existing autotools build:
Core library: libpostal, expand, address_dictionary, etc.
Parsers: address_parser, language_classifier, etc.
Utilities: string_utils, file_utils, utf8proc, etc.
Scanner: klib/drand48.c, scanner.c
2. Header Name Conflict Resolution
Problem Discovered: Cross-compiling revealed that
src/features.hconflicts with system glibc<features.h>Root Cause: System headers like
stdlib.hinclude<features.h>, but with-I srcin the include path (needed for JNI), the compiler found our local file instead, breaking critical system macros (__GLIBC_USE,__GNUC_PREREQ)Solution:
Renamed:
src/features.c→src/libpostal_features.cRenamed:
src/features.h→src/libpostal_features.hUpdated all includes in 11 source files (
.cand.h)Updated
src/Makefile.am(6 source list references)Updated
test/test_string_utils.cImpact: This benefits both Zig and autotools builds - the conflict could occur in any build system that includes
src/in the include path3. Windows Cross-Compilation Support
Challenge: Windows (MinGW) doesn't have POSIX
drand48()but needs the custom implementationSolution:
For Windows targets: Skip
config.h(which incorrectly definesHAVE_DRAND48=1from host system)Manually define
HAVE_DIRENT_H(MinGW provides this)Let existing
src/klib/drand48.cprovide the implementation whenHAVE_DRAND48is not definedimplementation
Fixed
-Wunused-parameterwarning for Windows buildsAdded
(void)old_size;to explicitly mark unused parameter.gitignore:Added Zig build artifacts:
build/,/build(compiled build.zig binary)Added editor backup files:
*~Added test artifacts:
test_arraylist,test_arraylist2,configure~Organized with clear section comments
🏗️ Build Capabilitie
Using Zig Build System:
All platforms built successfully:
Build time: ~7 minutes for all 6 platforms + native
🔄 Compatibility
Existing Build System:
Requirements:
🧪 Testing
Zig Build:
MinGW Build:
Autotools Build:
💡 Benefits
📝 Notes
🎯 Use Cases