Integration: BerlinMOD benchmark branch — 907/907 green on the b183b12 MEOS + th3index/h3 surface

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,14 @@
+<!--
+👀 Reviewers: this project's tier ranking, dependency chains and the
+standards checklist live in:
+  doc/contributing/reviewer-guide.md
+PR authors: update that file in the same commit as any PR queue change.
+-->
+
+## Summary
+
+<!-- What this PR does, in 1–3 bullets. Focus on the "why". -->
+
+## Test plan
+
+<!-- - [ ] Bullet checklist of TODOs for verifying this PR -->

.github/workflows/maven.yml

@@ -0,0 +1,220 @@
+name: Maven CI
+
+on:
+  push:
+    branches: ["main", "develop", "feat/**", "fix/**", "ci/**"]
+    paths-ignore:
+      - "**/*.md"
+      - "doc/**"
+  pull_request:
+    branches: ["main", "develop", "feat/**", "fix/**"]
+    paths-ignore:
+      - "**/*.md"
+      - "doc/**"
+  workflow_dispatch:
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  # ── Linux ────────────────────────────────────────────────────────────────────
+  linux:
+    name: Build and test — Linux (Java 21 / Spark 3.5)
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Java 21
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: "21"
+          cache: maven
+
+      - name: Install MEOS build dependencies (Linux)
+        run: |
+          sudo apt-get update -qq
+          sudo apt-get install -y \
+            cmake ninja-build \
+            libjson-c-dev libgeos-dev libproj-dev libgsl-dev
+
+      - name: Checkout MobilityDB source (for MEOS build)
+        uses: actions/checkout@v4
+        with:
+          repository: MobilityDB/MobilityDB
+          ref: v1.3.0
+          path: MobilityDB-src
+
+      - name: Build and install libmeos.so
+        run: |
+          cmake -S MobilityDB-src -B meos-build \
+            -G Ninja \
+            -DCMAKE_BUILD_TYPE=Release \
+            -DMEOS=ON \
+            -DCBUFFER=ON -DNPOINT=ON -DPOSE=OFF -DRGEO=OFF
+          cmake --build meos-build -j
+          sudo cmake --install meos-build
+          echo "LD_LIBRARY_PATH=/usr/local/lib" >> "$GITHUB_ENV"
+
+      - name: License header check
+        run: bash tools/scripts/check_license.sh
+
+      - name: Compile
+        run: mvn -B compile
+
+      - name: Unit tests
+        run: mvn -B test
+
+      - name: Package (fat jar)
+        run: mvn -B package -DskipTests
+
+      - name: Upload fat jar
+        uses: actions/upload-artifact@v4
+        with:
+          name: mobilityspark-spark.jar
+          path: target/*-spark.jar
+
+  # ── macOS ────────────────────────────────────────────────────────────────────
+  macos:
+    name: Build and test — macOS (Java 21 / Spark 3.5)
+    runs-on: macos-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Java 21
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: "21"
+          cache: maven
+
+      - name: Install MEOS dependencies (Homebrew)
+        run: |
+          brew install json-c geos proj gsl cmake ninja
+
+      - name: Checkout MobilityDB source (for MEOS build)
+        uses: actions/checkout@v4
+        with:
+          repository: MobilityDB/MobilityDB
+          ref: v1.3.0
+          path: MobilityDB-src
+
+      - name: Build and install libmeos.dylib
+        run: |
+          # Homebrew on macOS-ARM installs to /opt/homebrew; on Intel to /usr/local.
+          BREW_PREFIX="$(brew --prefix)"
+          cmake -S MobilityDB-src -B meos-build \
+            -G Ninja \
+            -DCMAKE_BUILD_TYPE=Release \
+            -DCMAKE_PREFIX_PATH="$BREW_PREFIX" \
+            -DCMAKE_INSTALL_RPATH="$BREW_PREFIX/lib" \
+            -DMEOS=ON \
+            -DCBUFFER=ON -DNPOINT=ON -DPOSE=OFF -DRGEO=OFF
+          cmake --build meos-build -j
+          sudo cmake --install meos-build
+          # Ad-hoc codesign so that the JVM's hardened-runtime library validation
+          # accepts the unsigned CMake-built dylib.
+          sudo codesign --force --sign - /usr/local/lib/libmeos.dylib
+          # JarLibraryLoader reads LD_LIBRARY_PATH first in CI mode (GITHUB_WORKFLOW
+          # set), then falls back to DYLD_LIBRARY_PATH. Set both env vars so the
+          # correct /usr/local/lib path reaches JNR-FFI regardless of which one
+          # the JVM process sees after the hardened runtime strips DYLD_* vars.
+          echo "LD_LIBRARY_PATH=/usr/local/lib" >> "$GITHUB_ENV"
+          echo "DYLD_LIBRARY_PATH=/usr/local/lib:$BREW_PREFIX/lib" >> "$GITHUB_ENV"
+
+      - name: Compile
+        run: mvn -B compile
+
+      - name: Unit tests
+        run: mvn -B test
+
+      - name: Package (fat jar)
+        run: mvn -B package -DskipTests
+
+  # ── Windows ──────────────────────────────────────────────────────────────────
+  windows:
+    name: Build and test — Windows (Java 21 / Spark 3.5)
+    runs-on: windows-latest
+    defaults:
+      run:
+        shell: msys2 {0}
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup MSYS2 (UCRT64)
+        uses: msys2/setup-msys2@v2
+        with:
+          msystem: UCRT64
+          update: true
+          install: >-
+            mingw-w64-ucrt-x86_64-gcc
+            mingw-w64-ucrt-x86_64-cmake
+            mingw-w64-ucrt-x86_64-ninja
+            mingw-w64-ucrt-x86_64-json-c
+            mingw-w64-ucrt-x86_64-geos
+            mingw-w64-ucrt-x86_64-proj
+            mingw-w64-ucrt-x86_64-gsl
+            mingw-w64-ucrt-x86_64-tzdata
+
+      - name: Set up Java 21
+        uses: actions/setup-java@v4
+        with:
+          distribution: temurin
+          java-version: "21"
+          cache: maven
+
+      - name: Checkout MobilityDB source (for MEOS build)
+        # meos-windows-bootstrap = v1.3.0 + minimal -DMEOS_TZDATA_DIR cmake
+        # patch (MobilityDB issue #513).  Switch to ref: v1.3.0 once the
+        # tzdata cmake option merges into MobilityDB upstream.
+        uses: actions/checkout@v4
+        with:
+          repository: estebanzimanyi/MobilityDB
+          ref: meos-windows-bootstrap
+          path: MobilityDB-src
+
+      - name: Resolve native timezone data path
+        run: |
+          # cygpath -m converts the MSYS2 POSIX prefix to a Windows-native path
+          # (e.g. C:/msys64/ucrt64/share/zoneinfo) that fopen() inside the DLL
+          # can open at runtime.
+          TZDATA_WIN=$(cygpath -m "$MSYSTEM_PREFIX/share/zoneinfo")
+          echo "MEOS_TZDATA_WIN=$TZDATA_WIN" >> "$GITHUB_ENV"
+
+      - name: Build and install libmeos.dll
+        run: |
+          cmake -S MobilityDB-src -B meos-build \
+            -G Ninja \
+            -DCMAKE_BUILD_TYPE=Release \
+            -DMEOS=ON \
+            -DCBUFFER=ON -DNPOINT=ON -DPOSE=OFF -DRGEO=OFF \
+            -DMEOS_TZDATA_DIR="$MEOS_TZDATA_WIN"
+          cmake --build meos-build -j
+          cmake --install meos-build --prefix "$PWD/meos-install"
+          # Save the DLL directory as a separate env var rather than stomping
+          # PATH: the MSYS2 PATH is POSIX-style and would overwrite the Windows
+          # PATH (losing Maven) in subsequent PowerShell steps.
+          echo "MEOS_DLL_DIR=$(cygpath -w "$PWD/meos-install/bin")" >> "$GITHUB_ENV"
+          # JarLibraryLoader (JMEOS) in CI mode reads LD_LIBRARY_PATH and passes
+          # it to jnr.ffi.LibraryLoader.search() to locate libmeos.dll; PATH is
+          # not checked.  Use the Windows-native path so JNR-FFI's File lookup works.
+          echo "LD_LIBRARY_PATH=$(cygpath -w "$PWD/meos-install/bin")" >> "$GITHUB_ENV"
+          # MSYS2 runtime DLLs (libgeos, libproj, libjson-c, libgsl) are needed
+          # when the JVM loads libmeos.dll's transitive dependencies at runtime.
+          echo "UCRT64_BIN=$(cygpath -w "$MSYSTEM_PREFIX/bin")" >> "$GITHUB_ENV"
+
+      - name: Compile
+        shell: pwsh
+        run: |
+          $env:PATH = "$env:MEOS_DLL_DIR;$env:PATH"
+          mvn -B compile
+
+      - name: Unit tests
+        shell: pwsh
+        run: |
+          $env:PATH = "$env:MEOS_DLL_DIR;$env:UCRT64_BIN;$env:PATH"
+          mvn -B test

.gitignore

@@ -3,14 +3,21 @@
 .project
 .settings/
 
-# Intellij
+# IntelliJ IDEA
 .idea/
 *.iml
 *.iws
+*.ipr
 
-# Mac
+# macOS
 .DS_Store
+**/.DS_Store
 
 # Maven
 log/
 target/
+
+# Large BerlinMOD benchmark data (generated locally — too large for GitHub)
+berlinmod/data/trips.csv
+dependency-reduced-pom.xml
+hs_err_pid*.log

INTEGRATION_NOTES.md

@@ -0,0 +1,20 @@
+# INTEGRATION BRANCH NOTE — JMEOS regen needed for Th3IndexUDFs
+
+The accumulated integration branch (integration/berlinmod-bench) carries
+PR #11 (Th3IndexUDFs full API, 86 UDFs) which calls JMEOS methods like
+`always_eq_th3index_h3index` / `always_ne_th3index_h3index`.  Neither the
+bundled JMEOS-1.4.jar nor JMEOS-1.5.jar exposes these — they require a
+JMEOS regeneration against MEOS-with-th3index.
+
+For the parallel benchmark task: either
+  (a) replace libs/JMEOS-1.5.jar with a regen built against the MEOS pin
+      that includes the th3index module (estebanzimanyi/MobilityDB:
+      fix/meos-pg-symbol-collision-plus-h3 @ b183b12ee692, OR
+      split/th3index-stacked, OR integration/meos-1.4-bump merged with #1045),
+  (b) comment-out the th3index method call-sites in Th3IndexUDFs.java
+      (lines around 498, 540, 580, etc.) — disables only the h3-cell
+      h3-cell join queries (Tier-1 Spark column-store prefilter), the
+      Trips × Trips queries Q5/Q6/Q10/Q16 still run via portable SQL +
+      BROADCAST hints.
+
+Option (a) is the right fix and is the JMEOS-session's standing task.

LICENSE

@@ -0,0 +1,22 @@
+-------------------------------------------------------------------------------
+This MobilityDB code is provided under The PostgreSQL License.
+
+Copyright (c) 2020-2025, Université libre de Bruxelles and MobilityDB
+contributors
+
+Permission to use, copy, modify, and distribute this software and its
+documentation for any purpose, without fee, and without a written agreement is
+hereby granted, provided that the above copyright notice and this paragraph and
+the following two paragraphs appear in all copies.
+
+IN NO EVENT SHALL UNIVERSITE LIBRE DE BRUXELLES BE LIABLE TO ANY PARTY FOR
+DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
+LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION,
+EVEN IF UNIVERSITE LIBRE DE BRUXELLES HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGE.
+
+UNIVERSITE LIBRE DE BRUXELLES SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
+BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND
+UNIVERSITE LIBRE DE BRUXELLES HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE,
+SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

README.md

@@ -16,6 +16,14 @@ The MobilityDB project is developed by the Computer & Decision Engineering Depar
 
 More information about MobilityDB, including publications, presentations, etc., can be found in the MobilityDB [website](https://mobilitydb.com).
 
+### For contributors and reviewers
+
+- Reviewing a pull request?  See the
+  [PR Reviewer Guide](doc/contributing/reviewer-guide.md) — tier ranking,
+  dependency chains and the standards checklist.  Reviewers landing in
+  any of the three platform repos (MobilityDB / MobilityDuck /
+  MobilitySpark) find the same canonical structure at the same path.
+
 
 ## Table of Contents
 
@@ -40,7 +48,7 @@ More information about MobilityDB, including publications, presentations, etc.,
 
 - 🚀 MobilityDB installed with MEOS
 - 🔧 JMEOS working version
-- ⚡ Spark 3.4.0
+- ⚡ Apache Spark 3.5.x (LTS); see [`doc/spark-version.md`](doc/spark-version.md) for the Spark-version target and the rationale for not yet supporting Spark 4
 - 📝 Maven 4
 - ☕ Java 17 (recommended)