From 790b98ef68421d90cd35d5829a6a83323c72b6f2 Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 15:27:35 +0200 Subject: [PATCH 01/10] feat(DBI): column_info via PRAGMA, AutoCommit tracking, per-dbh prepare_cached - Implement column_info() using SQLite PRAGMA table_info() to preserve original type case (JDBC uppercases type names, breaking DBIC tests) - Bless metadata result sets (table_info, column_info, etc.) into DBI class with proper Type/Executed/FetchHashKeyName attributes - Intercept literal BEGIN/COMMIT/ROLLBACK SQL in execute() and use JDBC API (setAutoCommit/commit/rollback) instead of executing SQL directly, fixing SQLite JDBC autocommit conflicts - Track AutoCommit state changes when literal transaction SQL is detected - Fix prepare_cached to use per-dbh CachedKids hash instead of global cache, preventing cross-connection cache pollution with :memory: DBs - Handle metadata sth execute() gracefully (no PreparedStatement) - Support PRAGMA pre-fetched rows in fetchrow_hashref DBIx::Class test results: 60/68 active tests pass (88%, was 78%) - t/64db.t: 4/4 real tests pass (was 2/4) - t/752sqlite.t: 34/34 real tests pass (was ~11/34) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbix_class.md | 59 ++++-- .../org/perlonjava/core/Configuration.java | 2 +- .../perlonjava/runtime/perlmodule/DBI.java | 195 +++++++++++++++++- src/main/perl/lib/DBI.pm | 29 ++- 4 files changed, 244 insertions(+), 41 deletions(-) diff --git a/dev/modules/dbix_class.md b/dev/modules/dbix_class.md index 8bea570a7..9865af5e1 100644 --- a/dev/modules/dbix_class.md +++ b/dev/modules/dbix_class.md @@ -173,20 +173,25 @@ a module whose `.pod`/`.pm` files were previously installed as read-only (0444), | 5.10 | Fix DBI `get_info()` to accept numeric constants per DBI spec | `DBI.java` | DONE | | 5.11 | Add DBI SQL type constants (`SQL_BIGINT`, `SQL_INTEGER`, etc.) | `DBI.pm` | DONE | | 5.12 | Fix `bind_columns` + `fetch` to update bound scalar references | `DBI.java` | DONE | +| 5.13 | Implement `column_info()` via SQLite PRAGMA; bless metadata sth | `DBI.java` | DONE | +| 5.14 | Add `AutoCommit` state tracking for literal transaction SQL | `DBI.java` | DONE | +| 5.15 | Intercept BEGIN/COMMIT/ROLLBACK via JDBC API instead of executing SQL | `DBI.java` | DONE | +| 5.16 | Fix `prepare_cached` to use per-dbh `CachedKids` cache | `DBI.pm` | DONE | **t/60core.t results** (17 tests emitted): - **ok 1–12**: Basic CRUD, update, dirty columns — all pass - **not ok 13–17**: Garbage collection tests — expected failures (JVM has no reference counting / `weaken`) - RowParser.pm line 260 crash still occurs in END block cleanup (non-blocking — all real tests pass first) -**Full test suite results** (92 test files): -- **15 fully passing** (no failures at all) -- **36 GC-only failures** (all real tests pass, only `weaken`-based GC leak tests fail) -- **12 tests with real failures** (see Blocking Issues below) -- **27 skipped** (DB-specific: Pg, Oracle, MSSQL, etc.; threads; fork) -- **2 zero-test** (MySQL-specific, result_set_column) +**Full test suite results** (92 test files, updated 2026-04-01): +- **21 fully passing** (no failures at all) +- **39 GC-only failures** (all real tests pass, only `weaken`-based GC leak tests fail) +- **8 tests with real failures** (see Blocking Issues below) +- **22 skipped** (DB-specific: Pg, Oracle, MSSQL, etc.; threads; fork) +- **2 compile-error** (t/52leaks.t — `wait` operator; t/88result_set_column.t — zero tests) -**Effective pass rate**: 51/65 active test files have all real tests passing (78%). +**Effective pass rate**: 60/68 active test files have all real tests passing (88%). +Previous: 51/65 (78%) → Current: 60/68 (88%) — **+10 percentage points**. --- @@ -238,21 +243,22 @@ a module whose `.pod`/`.pm` files were previously installed as read-only (0444), --- -## Remaining Real Failures (12 tests) +## Remaining Real Failures (8 tests) -### Tests needing DBI/Storage fixes +### Tests needing DBI/Storage fixes — RESOLVED -| Test | Failing | Root cause | Fix needed | -|------|---------|------------|------------| -| `t/64db.t` | tests 3-4 | `column_info()` not implemented in DBI shim | Implement `$dbh->column_info()` using JDBC `DatabaseMetaData.getColumns()` | -| `t/752sqlite.t` | test 6 | Transaction state tracking incomplete — `$dbh->{AutoCommit}` not updated on BEGIN/COMMIT | Track txn state in DBI `do()` or implement `begin_work`/`commit`/`rollback` that update `AutoCommit` | +| Test | Status | What was fixed | +|------|--------|----------------| +| `t/64db.t` | **FIXED** (4/4 real pass) | `column_info()` implemented via SQLite PRAGMA (step 5.13) | +| `t/752sqlite.t` | **FIXED** (34/34 real pass) | AutoCommit tracking + BEGIN/COMMIT/ROLLBACK interception (steps 5.14-5.15); `prepare_cached` per-dbh cache (step 5.16) | ### Tests needing caller/carp fixes | Test | Failing | Root cause | Fix needed | |------|---------|------------|------------| -| `t/106dbic_carp.t` | tests 2-3 | DBIx::Class::Carp callsite detection — `caller()` returns wrong package/line | Fix `caller()` to return correct info through `namespace::clean`'d frames | -| `t/100populate.t` | test 2 | Exception message doesn't include expected callsite info | Same caller/carp issue as above | +| `t/106dbic_carp.t` | tests 2-3 | DBIx::Class::Carp callsite detection — `caller()` returns wrong package/line; also `__LINE__` inside `qr//` differs from Perl 5 | Fix `caller()` to return correct info through `namespace::clean`'d frames | +| `t/100populate.t` | test 2 | `execute_for_fetch()` doesn't throw on duplicate key constraint violation | DBI needs unique constraint error propagation | +| `t/101populate_rs.t` | test(s) | Similar to t/100populate.t — `execute_for_fetch` exception handling | Same as above | ### Tests needing serialization/Storable fixes @@ -272,10 +278,16 @@ a module whose `.pod`/`.pm` files were previously installed as read-only (0444), | Test | Failing | Root cause | Fix needed | |------|---------|------------|------------| -| `t/40compose_connection.t` | (GC only) | Actually all real tests pass — has 7 GC failures instead of 5 | No fix needed (mis-categorized by test harness due to extra GC tests) | -| `t/52leaks.t` | test 4 | Dedicated leak testing — "how did we get so far?!" means previous leak tests should have aborted | `weaken` absence; same systemic issue as GC tests | | `t/85utf8.t` | test 7 | Warning about incorrect `use utf8` ordering not issued | May need to implement `utf8` pragma ordering detection | -| `t/93single_accessor_object.t` | (GC only) | Actually all real tests pass — has 8 GC failures | No fix needed | + +### GC-only failures (not real failures) + +| Test | GC failures | Notes | +|------|-------------|-------| +| `t/40compose_connection.t` | 7 | All real tests pass | +| `t/93single_accessor_object.t` | 15 | All real tests pass | +| `t/752sqlite.t` | 25 | All 34 real tests pass | +| 36 other files | 5 each | Standard GC leak detection tests | --- @@ -349,10 +361,17 @@ a module whose `.pod`/`.pm` files were previously installed as read-only (0444), - 5.11: Added DBI SQL type constants (`SQL_BIGINT`, `SQL_INTEGER`, etc.) - 5.12: Fixed `bind_columns` + `fetch` to update bound scalar references — unblocks ALL join/prefetch queries - Result: 51/65 active tests now pass all real tests (was ~15/65 before) +- [x] Phase 5 steps 5.13–5.16 (2026-04-01) + - 5.13: Implemented `column_info()` via SQLite `PRAGMA table_info()` — preserves original type case (JDBC uppercases), returns pre-fetched rows; also blessed metadata sth into `DBI` class with proper attributes + - 5.14: Added `AutoCommit` state tracking — `execute()` now detects literal BEGIN/COMMIT/ROLLBACK SQL and updates `$dbh->{AutoCommit}` accordingly + - 5.15: Intercepted literal transaction SQL via JDBC API — `conn.setAutoCommit(false)`, `conn.commit()`, `conn.rollback()` instead of executing SQL directly; fixes SQLite JDBC autocommit conflicts + - 5.16: Fixed `prepare_cached` to use per-dbh `CachedKids` cache instead of global hash — prevents cross-connection cache pollution when multiple `:memory:` SQLite connections share the same DSN name; added `if_active` parameter handling + - Also: `execute()` now handles metadata sth (no PreparedStatement) gracefully; `fetchrow_hashref` supports PRAGMA pre-fetched rows + - Result: 60/68 active tests now pass all real tests (was 51/65 = 78%, now 88%) ### Next Steps -1. **Quick wins**: Implement `column_info()` in DBI (fixes t/64db.t) and `AutoCommit` txn tracking (fixes t/752sqlite.t) -2. **Medium**: Fix caller/carp callsite detection (fixes t/106dbic_carp.t, t/100populate.t) +1. **Medium**: Fix caller/carp callsite detection (fixes t/106dbic_carp.t) +2. **Medium**: Fix `execute_for_fetch` exception propagation on constraint violations (fixes t/100populate.t, t/101populate_rs.t) 3. **Long-term**: Investigate VerifyError bytecode compiler bug (HIGH PRIORITY for broader CPAN compat) 4. **Pragmatic**: Accept GC-only failures as known JVM limitation; consider adding skip-leak-tests env var diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index cc3f6b755..a3c2a4574 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "509cc4f94"; + public static final String gitCommitId = "6a272a1cd"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). diff --git a/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java b/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java index 0eea451d3..3bb5b7a17 100644 --- a/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java +++ b/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java @@ -279,8 +279,62 @@ public static RuntimeList execute(RuntimeArray args, int ctx) { throw new IllegalStateException("Bad number of arguments for DBI->execute"); } + // Metadata statement handles (from column_info, table_info, etc.) + // don't have a PreparedStatement — result set is already available + RuntimeScalar stmtScalar = sth.get("statement"); + if (stmtScalar == null || stmtScalar.type == RuntimeScalarType.UNDEF) { + // Already "executed" by the metadata method — just return success + sth.put("Executed", scalarTrue); + dbh.put("Executed", scalarTrue); + return new RuntimeScalar(-1).getList(); + } + // Get prepared statement from statement handle - PreparedStatement stmt = (PreparedStatement) sth.get("statement").value; + PreparedStatement stmt = (PreparedStatement) stmtScalar.value; + + // Detect literal transaction SQL before execution + // Strip leading comments (-- comment\n) for detection + String sql = sth.get("sql").toString().trim(); + String strippedSql = sql; + while (strippedSql.startsWith("--")) { + int nlIdx = strippedSql.indexOf('\n'); + if (nlIdx >= 0) { + strippedSql = strippedSql.substring(nlIdx + 1).trim(); + } else { + break; + } + } + String sqlUpper = strippedSql.toUpperCase(); + boolean isBegin = sqlUpper.startsWith("BEGIN"); + boolean isCommit = sqlUpper.startsWith("COMMIT") || sqlUpper.startsWith("END"); + boolean isRollback = sqlUpper.startsWith("ROLLBACK"); + + Connection conn = (Connection) dbh.get("connection").value; + + // Intercept transaction control SQL — use JDBC API instead of executing + // as SQL, because JDBC drivers (especially SQLite) don't handle literal + // BEGIN/COMMIT/ROLLBACK properly when autocommit is enabled + if (isBegin || isCommit || isRollback) { + if (isBegin) { + conn.setAutoCommit(false); + dbh.put("AutoCommit", scalarFalse); + } else if (isCommit) { + conn.commit(); + conn.setAutoCommit(true); + dbh.put("AutoCommit", scalarTrue); + } else { + conn.rollback(); + conn.setAutoCommit(true); + dbh.put("AutoCommit", scalarTrue); + } + sth.put("Executed", scalarTrue); + dbh.put("Executed", scalarTrue); + RuntimeHash result = new RuntimeHash(); + result.put("success", scalarTrue); + result.put("has_resultset", scalarFalse); + sth.put("execute_result", result.createReference()); + return new RuntimeScalar("0E0").getList(); + } // Bind parameters to prepared statement if provided for (int i = 1; i < args.size(); i++) { @@ -404,6 +458,19 @@ public static RuntimeList fetchrow_hashref(RuntimeArray args, int ctx) { RuntimeHash dbh = sth.get("Database").hashDeref(); return executeWithErrorHandling(() -> { + // Handle pre-fetched rows (from PRAGMA-based column_info, etc.) + RuntimeScalar pragmaRows = sth.get("_pragma_rows"); + if (pragmaRows != null && pragmaRows.type != RuntimeScalarType.UNDEF) { + RuntimeArray rows = pragmaRows.arrayDeref(); + int idx = sth.get("_pragma_idx").getInt(); + if (idx < rows.size()) { + sth.put("_pragma_idx", new RuntimeScalar(idx + 1)); + RuntimeHash row = rows.get(idx).hashDeref(); + return row.createReference().getList(); + } + return scalarUndef.getList(); + } + RuntimeHash executeResult = sth.get("execute_result").hashDeref(); ResultSet rs = (ResultSet) executeResult.get("resultset").value; @@ -645,7 +712,8 @@ public static RuntimeList table_info(RuntimeArray args, int ctx) { // Create statement handle for results RuntimeHash sth = createMetadataResultSet(dbh, rs); - return sth.createReference().getList(); + RuntimeScalar sthRef = ReferenceOperators.bless(sth.createReference(), new RuntimeScalar("DBI")); + return sthRef.getList(); }, dbh, "table_info"); } @@ -658,20 +726,117 @@ public static RuntimeList column_info(RuntimeArray args, int ctx) { } Connection conn = (Connection) dbh.get("connection").value; + String table = args.get(3).toString(); + + // For SQLite, use PRAGMA table_info() to preserve original type case + // (JDBC getColumns() uppercases type names like varchar -> VARCHAR) + String jdbcUrl = dbh.get("Name").toString(); + if (jdbcUrl.contains("sqlite")) { + return columnInfoViaPragma(dbh, conn, table); + } + DatabaseMetaData metaData = conn.getMetaData(); - String catalog = args.get(1).toString(); - String schema = args.get(2).toString(); - String table = args.get(3).toString(); + // Handle undef args: pass null to JDBC (means "any") instead of "" + String catalog = args.get(1).type == RuntimeScalarType.UNDEF ? null : args.get(1).toString(); + String schema = args.get(2).type == RuntimeScalarType.UNDEF ? null : args.get(2).toString(); String column = args.size() > 4 ? args.get(4).toString() : "%"; ResultSet rs = metaData.getColumns(catalog, schema, table, column); RuntimeHash sth = createMetadataResultSet(dbh, rs); - return sth.createReference().getList(); + RuntimeScalar sthRef = ReferenceOperators.bless(sth.createReference(), new RuntimeScalar("DBI")); + return sthRef.getList(); }, dbh, "column_info"); } + /** + * SQLite-specific column_info using PRAGMA table_info(). + * Preserves original type case from CREATE TABLE (e.g., "varchar" not "VARCHAR"). + * Returns a pre-populated sth with DBI-standard column names. + */ + private static RuntimeList columnInfoViaPragma(RuntimeHash dbh, Connection conn, String table) throws SQLException { + Statement stmt = conn.createStatement(); + ResultSet rs = stmt.executeQuery("PRAGMA table_info(" + table + ")"); + + // Build arrays of rows matching DBI column_info format + RuntimeArray rows = new RuntimeArray(); + while (rs.next()) { + RuntimeHash row = new RuntimeHash(); + String colName = rs.getString("name"); + String declType = rs.getString("type"); + int notNull = rs.getInt("notnull"); + String dfltValue = rs.getString("dflt_value"); + int pk = rs.getInt("pk"); + + // Parse declared type: "varchar(100)" -> type="varchar", size=100 + String typeName = declType; + String columnSize = null; + if (declType != null) { + int parenIdx = declType.indexOf('('); + if (parenIdx >= 0) { + typeName = declType.substring(0, parenIdx); + int closeIdx = declType.indexOf(')', parenIdx); + if (closeIdx > parenIdx) { + columnSize = declType.substring(parenIdx + 1, closeIdx); + } + } + } + + row.put("COLUMN_NAME", new RuntimeScalar(colName)); + row.put("TYPE_NAME", new RuntimeScalar(typeName != null ? typeName : "")); + row.put("COLUMN_SIZE", columnSize != null ? new RuntimeScalar(columnSize) : scalarUndef); + // DBI NULLABLE: 0=not nullable, 1=nullable, 2=unknown + row.put("NULLABLE", new RuntimeScalar(notNull != 0 ? 0 : 1)); + row.put("COLUMN_DEF", dfltValue != null ? new RuntimeScalar(dfltValue) : scalarUndef); + row.put("ORDINAL_POSITION", new RuntimeScalar(rs.getInt("cid") + 1)); + + RuntimeArray.push(rows, row.createReference()); + } + rs.close(); + stmt.close(); + + // Create a synthetic sth that yields these rows via fetchrow_hashref + RuntimeHash sth = new RuntimeHash(); + sth.put("Database", dbh.createReference()); + sth.put("Type", new RuntimeScalar("st")); + sth.put("Executed", scalarTrue); + + // Inherit error handling and fetch settings from dbh + RuntimeScalar raiseError = dbh.get("RaiseError"); + if (raiseError != null) sth.put("RaiseError", raiseError); + RuntimeScalar printError = dbh.get("PrintError"); + if (printError != null) sth.put("PrintError", printError); + + // Store pre-fetched rows for iteration + sth.put("_pragma_rows", rows.createReference()); + sth.put("_pragma_idx", new RuntimeScalar(0)); + + // Set up column names for the result set + String[] colNames = {"COLUMN_NAME", "TYPE_NAME", "COLUMN_SIZE", "NULLABLE", "COLUMN_DEF", "ORDINAL_POSITION"}; + RuntimeArray names = new RuntimeArray(); + RuntimeArray namesLc = new RuntimeArray(); + RuntimeArray namesUc = new RuntimeArray(); + for (String n : colNames) { + RuntimeArray.push(names, new RuntimeScalar(n)); + RuntimeArray.push(namesLc, new RuntimeScalar(n.toLowerCase())); + RuntimeArray.push(namesUc, new RuntimeScalar(n.toUpperCase())); + } + sth.put("NAME", names.createReference()); + sth.put("NAME_lc", namesLc.createReference()); + sth.put("NAME_uc", namesUc.createReference()); + sth.put("NUM_OF_FIELDS", new RuntimeScalar(colNames.length)); + + // Create a dummy execute_result with no JDBC resultset + RuntimeHash result = new RuntimeHash(); + result.put("success", scalarTrue); + result.put("has_resultset", scalarTrue); + sth.put("execute_result", result.createReference()); + + RuntimeScalar sthRef = ReferenceOperators.bless(sth.createReference(), new RuntimeScalar("DBI")); + return sthRef.getList(); + } + public static RuntimeList primary_key_info(RuntimeArray args, int ctx) { RuntimeHash dbh = args.get(0).hashDeref(); @@ -690,7 +855,8 @@ public static RuntimeList primary_key_info(RuntimeArray args, int ctx) { ResultSet rs = metaData.getPrimaryKeys(catalog, schema, table); RuntimeHash sth = createMetadataResultSet(dbh, rs); - return sth.createReference().getList(); + RuntimeScalar sthRef = ReferenceOperators.bless(sth.createReference(), new RuntimeScalar("DBI")); + return sthRef.getList(); }, dbh, "primary_key_info"); } @@ -716,7 +882,8 @@ public static RuntimeList foreign_key_info(RuntimeArray args, int ctx) { fkCatalog, fkSchema, fkTable); RuntimeHash sth = createMetadataResultSet(dbh, rs); - return sth.createReference().getList(); + RuntimeScalar sthRef = ReferenceOperators.bless(sth.createReference(), new RuntimeScalar("DBI")); + return sthRef.getList(); }, dbh, "foreign_key_info"); } @@ -729,7 +896,8 @@ public static RuntimeList type_info(RuntimeArray args, int ctx) { ResultSet rs = metaData.getTypeInfo(); RuntimeHash sth = createMetadataResultSet(dbh, rs); - return sth.createReference().getList(); + RuntimeScalar sthRef = ReferenceOperators.bless(sth.createReference(), new RuntimeScalar("DBI")); + return sthRef.getList(); }, dbh, "type_info"); } @@ -737,6 +905,14 @@ private static RuntimeHash createMetadataResultSet(RuntimeHash dbh, ResultSet rs RuntimeHash sth = new RuntimeHash(); sth.put("Database", dbh.createReference()); + // Inherit error handling and fetch settings from dbh + RuntimeScalar raiseError = dbh.get("RaiseError"); + if (raiseError != null) sth.put("RaiseError", raiseError); + RuntimeScalar printError = dbh.get("PrintError"); + if (printError != null) sth.put("PrintError", printError); + RuntimeScalar fetchKeyName = dbh.get("FetchHashKeyName"); + if (fetchKeyName != null) sth.put("FetchHashKeyName", fetchKeyName); + // Create statement handle with result set RuntimeHash result = new RuntimeHash(); result.put("success", scalarTrue); @@ -767,6 +943,7 @@ private static RuntimeHash createMetadataResultSet(RuntimeHash dbh, ResultSet rs sth.put("Executed", scalarTrue); sth.put("execute_result", result.createReference()); + // Bless into DBI so method calls like $sth->fetchrow_hashref() work return sth; } diff --git a/src/main/perl/lib/DBI.pm b/src/main/perl/lib/DBI.pm index 9cd00af4d..a9d63c91b 100644 --- a/src/main/perl/lib/DBI.pm +++ b/src/main/perl/lib/DBI.pm @@ -434,24 +434,31 @@ sub trace_msg { sub prepare_cached { my ($dbh, $sql, $attr, $if_active) = @_; - my $cache_key = $dbh->{Name} . ':' . $sql; + # Use a per-dbh cache (like real DBI's CachedKids) to avoid cross-connection + # cache hits when multiple connections share the same Name (e.g., :memory:) + $dbh->{CachedKids} ||= {}; + my $cache = $dbh->{CachedKids}; - if (exists $CACHED_STATEMENTS{$cache_key}) { - my $sth = $CACHED_STATEMENTS{$cache_key}; + if (exists $cache->{$sql}) { + my $sth = $cache->{$sql}; if ($sth->{Database}{Active}) { + # Handle if_active parameter: + # 1 = warn and finish, 2 = finish silently, 3 = return new sth + if ($if_active && $sth->{Active}) { + if ($if_active == 3) { + # Return a fresh sth instead of the active cached one + my $new_sth = $dbh->prepare($sql, $attr) or return undef; + $cache->{$sql} = $new_sth; + return $new_sth; + } + $sth->finish; + } return $sth; } } my $sth = $dbh->prepare($sql, $attr) or return undef; - - # Implement simple LRU by removing oldest if cache is full - if (keys %CACHED_STATEMENTS >= $MAX_CACHED_STATEMENTS) { - my @keys = keys %CACHED_STATEMENTS; - delete $CACHED_STATEMENTS{$keys[0]}; - } - - $CACHED_STATEMENTS{$cache_key} = $sth; + $cache->{$sql} = $sth; return $sth; } From c8048706160a87cfff88a23573b6141183d590ca Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 15:37:47 +0200 Subject: [PATCH 02/10] fix: $^S correctly reports 1 inside $SIG{__DIE__} for require failures in eval When require fails inside eval {}, the __DIE__ handler now sees $^S=1 (inside eval) instead of incorrectly seeing $^S=0 (top-level). The bug was that evalDepth was decremented by the eval catch block before catchEval() invoked the handler. Fixed by temporarily incrementing evalDepth around the handler call in catchEval(). This unblocks t/00describe_environment.t in DBIx::Class which uses $^S to distinguish eval-guarded optional requires from real crashes. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbix_class.md | 25 ++++++++++++++++++- .../org/perlonjava/core/Configuration.java | 2 +- .../perlonjava/runtime/operators/WarnDie.java | 6 +++++ 3 files changed, 31 insertions(+), 2 deletions(-) diff --git a/dev/modules/dbix_class.md b/dev/modules/dbix_class.md index 9865af5e1..88d50a225 100644 --- a/dev/modules/dbix_class.md +++ b/dev/modules/dbix_class.md @@ -197,11 +197,34 @@ Previous: 51/65 (78%) → Current: 60/68 (88%) — **+10 percentage points**. ## Blocking Issues — Not Quick Fixes +### HIGH PRIORITY: `$^S` wrong inside `$SIG{__DIE__}` when `require` fails in `eval {}` + +**Symptom**: `$^S` is 0 (top-level) instead of 1 (inside eval) when `require` triggers `$SIG{__DIE__}` from within `eval {}`. This causes die handlers that check `$^S` to misidentify eval-guarded require failures as top-level crashes. + +**Affected tests**: `t/00describe_environment.t` — the test installs a `$SIG{__DIE__}` handler that uses `$^S` to distinguish eval-caught exceptions from real crashes. Because `$^S` is wrong, the optional `require File::HomeDir` (inside `eval {}`) triggers the "Something horrible happened" path and `exit 0`, aborting the test. The `Class::Accessor::Grouped->VERSION` check also crashes the same way. + +**Repro**: +```bash +# PerlOnJava (wrong): S=0 +./jperl -e '$SIG{__DIE__} = sub { print "S=", defined($^S) ? $^S : "undef", "\n" }; eval { require No::Such::Module }; print "after eval\n"' + +# Perl 5 (correct): S=1 +perl -e '$SIG{__DIE__} = sub { print "S=", defined($^S) ? $^S : "undef", "\n" }; eval { require No::Such::Module }; print "after eval\n"' +``` + +**Root cause**: The `require` failure path does not propagate the eval depth / `$^S` state when invoking `$SIG{__DIE__}`. A plain `die` inside `eval {}` correctly reports `$^S=1`, but a failed `require` inside `eval {}` reports `$^S=0`. + +**What's needed to fix**: +- Find where `require` failure invokes the `__DIE__` handler (likely in `Require.java` or `WarnDie.java`) +- Ensure `$^S` reflects the enclosing eval context, matching the behavior of `die` inside `eval {}` + +**Impact**: HIGH — blocks `t/00describe_environment.t` and any code that relies on `$^S` in `$SIG{__DIE__}` with `require` inside `eval {}`. Common pattern in CPAN (Test::Exception, DBIx::Class, Moose). + ### HIGH PRIORITY: VerifyError (bytecode compiler bug) **Symptom**: `java.lang.VerifyError: Bad type on operand stack` when compiling complex anonymous subroutines with many local variables. -**Affected tests**: `t/00describe_environment.t` (crashes after already emitting `1..0` skip) +**Affected tests**: `t/00describe_environment.t` (secondary issue — also blocked by `$^S` bug above) **Root cause**: The JVM bytecode emitter generates incorrect stack map frames when a subroutine has many locals and complex control flow (ternary chains, nested `eval`, `for` loops). The JVM verifier rejects the class because `java/lang/Object` on the stack is not assignable to `RuntimeScalar`. diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index a3c2a4574..dc2c79f22 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "6a272a1cd"; + public static final String gitCommitId = "790b98ef6"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). diff --git a/src/main/java/org/perlonjava/runtime/operators/WarnDie.java b/src/main/java/org/perlonjava/runtime/operators/WarnDie.java index d59e1cf34..58db9d501 100644 --- a/src/main/java/org/perlonjava/runtime/operators/WarnDie.java +++ b/src/main/java/org/perlonjava/runtime/operators/WarnDie.java @@ -73,6 +73,11 @@ public static RuntimeScalar catchEval(Throwable e) { int level = DynamicVariableManager.getLocalLevel(); DynamicVariableManager.pushLocalVariable(sig); + // Temporarily restore eval depth so $^S reads 1 inside the handler. + // By the time we reach catchEval(), evalDepth has already been decremented + // by the eval catch block, but the handler should see $^S=1 since we are + // conceptually still inside eval (Perl 5 calls the handler before unwinding). + RuntimeCode.evalDepth++; try { RuntimeCode.apply(sigHandler, args, RuntimeContextType.SCALAR); } catch (Throwable handlerException) { @@ -90,6 +95,7 @@ public static RuntimeScalar catchEval(Throwable e) { err.set(new RuntimeScalar(ErrorMessageUtil.stringifyException(handlerException))); } } finally { + RuntimeCode.evalDepth--; // Restore $SIG{__DIE__} DynamicVariableManager.popToLocalLevel(level); } From abad323cf8db5d36686e68a0a654991eced202fe Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 15:46:41 +0200 Subject: [PATCH 03/10] fix: __LINE__ inside @{[]} string interpolation returns correct line number When __LINE__ appeared inside @{[expr]} in double-quoted strings or qr//, it returned the line number from the start of the enclosing statement instead of the actual line. This was caused by the inner sub-parser sharing the outer errorUtil which uses a forward-only cache. Fixed by: - Adding baseLineNumber field to Parser for string sub-parsers - Setting it from the outer source position in StringDoubleQuoted - Using it in CoreOperatorResolver to compute __LINE__ by counting inner newlines from the base This fixes t/106dbic_carp.t tests 2-3 in DBIx::Class which depend on __LINE__ inside qr// matching the actual source line. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- .../java/org/perlonjava/core/Configuration.java | 2 +- .../frontend/parser/CoreOperatorResolver.java | 16 +++++++++++++++- .../org/perlonjava/frontend/parser/Parser.java | 4 ++++ .../frontend/parser/StringDoubleQuoted.java | 5 +++++ 4 files changed, 25 insertions(+), 2 deletions(-) diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index dc2c79f22..ffaaa8496 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "790b98ef6"; + public static final String gitCommitId = "c80487061"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). diff --git a/src/main/java/org/perlonjava/frontend/parser/CoreOperatorResolver.java b/src/main/java/org/perlonjava/frontend/parser/CoreOperatorResolver.java index 06e14b861..b15ab093a 100644 --- a/src/main/java/org/perlonjava/frontend/parser/CoreOperatorResolver.java +++ b/src/main/java/org/perlonjava/frontend/parser/CoreOperatorResolver.java @@ -35,7 +35,21 @@ public static Node parseCoreOperator(Parser parser, LexerToken token, int startI return switch (operatorName) { case "__LINE__" -> { handleEmptyParentheses(parser); - yield new NumberNode(Integer.toString(parser.ctx.errorUtil.getLineNumber(parser.tokenIndex)), parser.tokenIndex); + int lineNumber; + if (parser.baseLineNumber > 0) { + // Inside string interpolation sub-parser: count newlines in inner tokens + // up to current position and add to base line of the enclosing string + int newlineCount = 0; + for (int i = 0; i < parser.tokenIndex && i < parser.tokens.size(); i++) { + if (parser.tokens.get(i).type == LexerTokenType.NEWLINE) { + newlineCount++; + } + } + lineNumber = parser.baseLineNumber + newlineCount; + } else { + lineNumber = parser.ctx.errorUtil.getLineNumber(parser.tokenIndex); + } + yield new NumberNode(Integer.toString(lineNumber), parser.tokenIndex); } case "__FILE__" -> { handleEmptyParentheses(parser); diff --git a/src/main/java/org/perlonjava/frontend/parser/Parser.java b/src/main/java/org/perlonjava/frontend/parser/Parser.java index dc00cfe9e..a611ae8b5 100644 --- a/src/main/java/org/perlonjava/frontend/parser/Parser.java +++ b/src/main/java/org/perlonjava/frontend/parser/Parser.java @@ -53,6 +53,10 @@ public class Parser { public int heredocSkipToIndex = -1; // The specific NEWLINE token index that should trigger the skip. public int heredocNewlineIndex = -1; + // Base line number for string sub-parsers. When > 0, this parser operates on + // re-tokenized string content and __LINE__ should use this as the base line, + // counting newlines from the inner token list to offset from it. + public int baseLineNumber = 0; /** * Constructs a Parser with the given context and tokens. diff --git a/src/main/java/org/perlonjava/frontend/parser/StringDoubleQuoted.java b/src/main/java/org/perlonjava/frontend/parser/StringDoubleQuoted.java index 9737d7d89..667062231 100644 --- a/src/main/java/org/perlonjava/frontend/parser/StringDoubleQuoted.java +++ b/src/main/java/org/perlonjava/frontend/parser/StringDoubleQuoted.java @@ -157,6 +157,11 @@ static Node parseDoubleQuotedString(EmitterContext ctx, StringParser.ParsedStrin // Copy any other relevant context flags as needed } + // Set base line number so __LINE__ inside @{[...]} interpolation + // returns the correct line from the original source, not the inner token list. + // Use rawStr.index (position of opening delimiter in outer token list). + parser.baseLineNumber = ctx.errorUtil.getLineNumberAccurate(rawStr.index); + // Create and run the double-quoted string parser with original token offset tracking var doubleQuotedParser = new StringDoubleQuoted(ctx, tokens, parser, tokenIndex, isRegex, parseEscapes, interpolateVariable, isRegexReplacement); From d7dbe0f9e1a4f72d52e90f539f1b8444b90b8d6c Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 16:00:35 +0200 Subject: [PATCH 04/10] fix(DBI): execute_for_fetch error propagation matching real DBI behavior - execute_for_fetch now tracks error count and stores [$sth->err, $sth->errstr, $sth->state] in tuple_status on failure (matching DBI 1.647 behavior) - After the loop, dies with error count if RaiseError is on - execute() now sets err/errstr/state on both sth and dbh (was dbh only) - Clears sth error state before each execute call Fixes t/100populate.t test 2 (execute_for_fetch exception detection). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbix_class.md | 60 ++++++++++++++++++- .../org/perlonjava/core/Configuration.java | 2 +- .../perlonjava/runtime/perlmodule/DBI.java | 14 ++++- src/main/perl/lib/DBI.pm | 45 +++++++++----- 4 files changed, 101 insertions(+), 20 deletions(-) diff --git a/dev/modules/dbix_class.md b/dev/modules/dbix_class.md index 88d50a225..a8304157e 100644 --- a/dev/modules/dbix_class.md +++ b/dev/modules/dbix_class.md @@ -303,6 +303,20 @@ perl -e '$SIG{__DIE__} = sub { print "S=", defined($^S) ? $^S : "undef", "\n" }; |------|---------|------------|------------| | `t/85utf8.t` | test 7 | Warning about incorrect `use utf8` ordering not issued | May need to implement `utf8` pragma ordering detection | +### Spurious "Subroutine redefined" warnings + +Multiple DBIx::Class tests emit these warnings at startup: +``` +Subroutine set_todo redefined at jar:PERL5LIB/Test2/Event/Ok.pm line 29. +Subroutine set_subevents redefined at jar:PERL5LIB/Test2/Event/Subtest.pm line 32. +``` + +**Root cause**: Test2::Event::Ok and Test2::Event::Subtest define subroutines that PerlOnJava considers already defined — likely because the modules are loaded twice (once from the bundled jar, once from an installed copy) or because accessor generation re-defines subs without checking first. + +**Impact**: Low — cosmetic noise only, does not cause test failures. But could cause failures in tests that check for unexpected warnings (e.g., `Test::Warn` / `warnings_like`). + +**Fix needed**: Investigate whether the modules are double-loaded (check `%INC` for duplicate entries) or whether the subroutine redefinition warning threshold differs from Perl 5. + ### GC-only failures (not real failures) | Test | GC failures | Notes | @@ -314,7 +328,21 @@ perl -e '$SIG{__DIE__} = sub { print "S=", defined($^S) ? $^S : "undef", "\n" }; --- -## Known Bugs +## Must Fix + +### Ternary-as-lvalue with assignment branches + +Expressions like `($x) ? @$a = () : $b = []` trigger "Modification of a read-only value attempted" at runtime. Perl 5 parses this as `($x) ? (@$a = ()) : ($b = [])`, where each branch is an independent assignment expression. PerlOnJava's compile-time `LValueVisitor` (Phase 4.7) correctly classifies assignment branches as `ASSIGN_SCALAR`, but the **JVM backend code generator** emits incorrect code for the runtime ternary-as-lvalue path when the condition is non-constant. + +**Root cause**: When the ternary condition is non-constant (e.g., `$x`, `wantarray`), the emitter generates bytecode that evaluates both branches and selects the result — but the selected value is not returned as a modifiable lvalue. The assignment to the "winning" branch's target fails because the emitter wraps the result in a read-only temporary. Constant-folded cases (`1 ? @rv = eval $src : $rv[0]`) work correctly because only one branch is emitted. + +**What's needed to fix**: +- In the JVM backend emitter (likely `EmitOperator.java` or `EmitAssignment.java`), fix the ternary-as-lvalue code path to emit proper lvalue references for both branches +- The emitter should generate a conditional jump that executes the selected branch's assignment in place, rather than evaluating both branches and selecting a result +- Reference: Perl 5's `pp_cond_expr` in `pp_ctl.c` — the ternary simply jumps to the selected branch, which then evaluates (including any assignments) and returns its result +- Alternatively, detect ternary expressions where both branches are complete assignments (not lvalue targets) and compile them as `if`/`else` instead of ternary-as-lvalue + +**Impact**: Affects real DBI's `execute_for_fetch` implementation (worked around with if/else in our DBI.pm), Class::Accessor::Grouped patterns, and potentially other CPAN code using this idiom. ### File::stat VerifyError - `use File::stat` triggers `java.lang.VerifyError: Bad type on operand stack` @@ -323,6 +351,36 @@ perl -e '$SIG{__DIE__} = sub { print "S=", defined($^S) ? $^S : "undef", "\n" }; - Impact: Path::Class cannot load; DBIx::Class works without it - Same class of bug as the t/00describe_environment.t VerifyError (see HIGH PRIORITY above) +### JDBC error message format mismatch + +**Symptom**: t/100populate.t test 52 (`bad slice fails PK insert`) — the exception IS thrown correctly by `execute_for_fetch`, but the regex doesn't match because the JDBC SQLite driver wraps error messages differently from native SQLite. + +**Example**: +- Expected: `execute_for_fetch() aborted with 'datatype mismatch` +- Got: `execute_for_fetch() aborted with '[SQLITE_MISMATCH] Data type mismatch (datatype mismatch)'` + +**What's needed to fix**: +- Strip or normalize JDBC error message prefixes (e.g., `[SQLITE_MISMATCH]`) in `setError()` in `DBI.java` so that `$sth->errstr` returns the same text as native SQLite +- Alternatively, extract the parenthesized message at the end: `(datatype mismatch)` → `datatype mismatch` + +**Impact**: Affects t/100populate.t tests 52-53, and potentially any code that pattern-matches on SQLite error strings. + +### SQL expression formatting differences (t/100populate.t tests 37-42) + +**Symptom**: Tests compare generated SQL against expected strings, but PerlOnJava's SQL::Abstract produces slightly different whitespace or column ordering. + +**Example**: +- Got: `INSERT INTO link ( id, title, url) VALUES ( ?, ?, ? )` +- Expected format likely differs in spacing or column order + +**What's needed to fix**: Investigate whether this is a SQL::Abstract::Classic difference or a column ordering issue in populate's internal logic. May need to normalize SQL whitespace in comparisons or fix column ordering. + +### bind parameter attribute handling (t/100populate.t tests 58-59) + +**Symptom**: Test 58 (`literal+bind with differing attrs throws`) expects an exception that isn't thrown. Test 59 (`literal+bind with semantically identical attrs works after normalization`) also fails. + +**What's needed to fix**: Investigate how `bind_param` attributes (data types) are compared during populate's bind parameter deduplication. May need to implement attribute-aware bind parameter comparison in DBI.pm or Storage::DBI. + ## Summary | Phase | Complexity | Description | Status | diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index ffaaa8496..0e4e4315a 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "c80487061"; + public static final String gitCommitId = "abad323cf"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). diff --git a/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java b/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java index 3bb5b7a17..44db1903d 100644 --- a/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java +++ b/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java @@ -75,12 +75,19 @@ public static void initialize() { * @return RuntimeList result from the operation or error result */ private static RuntimeList executeWithErrorHandling(DBIOperation operation, RuntimeHash handle, String methodName) { + return executeWithErrorHandling(operation, handle, null, methodName); + } + + private static RuntimeList executeWithErrorHandling(DBIOperation operation, RuntimeHash handle, RuntimeHash secondHandle, String methodName) { try { return operation.execute(); } catch (SQLException e) { setError(handle, e); + if (secondHandle != null) setError(secondHandle, e); } catch (Exception e) { - setError(handle, new SQLException(e.getMessage(), GENERAL_ERROR_STATE, DBI_ERROR_CODE)); + SQLException sqlEx = new SQLException(e.getMessage(), GENERAL_ERROR_STATE, DBI_ERROR_CODE); + setError(handle, sqlEx); + if (secondHandle != null) setError(secondHandle, sqlEx); } RuntimeScalar msg = new RuntimeScalar("DBI " + methodName + "() failed: " + getGlobalVariable("DBI::errstr")); if (handle.get("RaiseError").getBoolean()) { @@ -274,6 +281,9 @@ public static RuntimeList execute(RuntimeArray args, int ctx) { RuntimeHash sth = args.get(0).hashDeref(); RuntimeHash dbh = sth.get("Database").hashDeref(); + // Clear previous error state on sth before executing + setError(sth, null); + return executeWithErrorHandling(() -> { if (args.isEmpty()) { throw new IllegalStateException("Bad number of arguments for DBI->execute"); @@ -396,7 +406,7 @@ public static RuntimeList execute(RuntimeArray args, int ctx) { } return new RuntimeScalar(updateCount).getList(); } - }, dbh, "execute"); + }, dbh, sth, "execute"); } /** diff --git a/src/main/perl/lib/DBI.pm b/src/main/perl/lib/DBI.pm index a9d63c91b..04ada2933 100644 --- a/src/main/perl/lib/DBI.pm +++ b/src/main/perl/lib/DBI.pm @@ -133,24 +133,37 @@ sub finish { # Batch execution: calls $fetch_tuple->() repeatedly to get parameter arrays, # executes the prepared statement for each, and tracks results in $tuple_status. sub execute_for_fetch { - my ($sth, $fetch_tuple, $tuple_status) = @_; - $tuple_status ||= []; - @$tuple_status = (); - - my $total_rows = 0; - while (my $tuple = $fetch_tuple->()) { - my $rv; - eval { - $rv = $sth->execute(@$tuple); - }; - if ($@) { - push @$tuple_status, [$@]; - next; + my ($sth, $fetch_tuple_sub, $tuple_status) = @_; + # start with empty status array + if ($tuple_status) { + @$tuple_status = (); + } else { + $tuple_status = []; + } + + my $rc_total = 0; + my $err_count; + while ( my $tuple = &$fetch_tuple_sub() ) { + my $rc = eval { $sth->execute(@$tuple) }; + if ($rc) { + push @$tuple_status, $rc; + $rc_total = ($rc >= 0 && $rc_total >= 0) ? $rc_total + $rc : -1; } - push @$tuple_status, $rv; - $total_rows += $rv if defined $rv && $rv >= 0; + else { + $err_count++; + push @$tuple_status, [ $sth->err, $sth->errstr || $@, $sth->state ]; + } + } + my $tuples = @$tuple_status; + if ($err_count) { + my $err_msg = "executing $tuples generated $err_count errors"; + die $err_msg if $sth->{Database}{RaiseError}; + warn $err_msg if $sth->{Database}{PrintError}; + return undef; } - return $total_rows; + $tuples ||= "0E0"; + return $tuples unless wantarray; + return ($tuples, $rc_total); } sub bind_param { From 977e22d72a3caaab44bc14f4782567a88007d267 Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 16:06:36 +0200 Subject: [PATCH 05/10] =?UTF-8?q?docs:=20update=20DBIx::Class=20plan=20?= =?UTF-8?q?=E2=80=94=2062/68=20tests=20passing=20(91%)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updated test results after fixing $^S, __LINE__ interpolation, and execute_for_fetch error propagation. Added new "Must Fix" entries for JDBC error message format, SQL formatting differences, and bind parameter attribute handling. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbix_class.md | 34 ++++++++++++++++++++-------------- 1 file changed, 20 insertions(+), 14 deletions(-) diff --git a/dev/modules/dbix_class.md b/dev/modules/dbix_class.md index a8304157e..217100370 100644 --- a/dev/modules/dbix_class.md +++ b/dev/modules/dbix_class.md @@ -184,14 +184,14 @@ a module whose `.pod`/`.pm` files were previously installed as read-only (0444), - RowParser.pm line 260 crash still occurs in END block cleanup (non-blocking — all real tests pass first) **Full test suite results** (92 test files, updated 2026-04-01): -- **21 fully passing** (no failures at all) -- **39 GC-only failures** (all real tests pass, only `weaken`-based GC leak tests fail) -- **8 tests with real failures** (see Blocking Issues below) +- **22 fully passing** (no failures at all) +- **40 GC-only failures** (all real tests pass, only `weaken`-based GC leak tests fail) +- **6 tests with real failures** (see Remaining Real Failures below) - **22 skipped** (DB-specific: Pg, Oracle, MSSQL, etc.; threads; fork) - **2 compile-error** (t/52leaks.t — `wait` operator; t/88result_set_column.t — zero tests) -**Effective pass rate**: 60/68 active test files have all real tests passing (88%). -Previous: 51/65 (78%) → Current: 60/68 (88%) — **+10 percentage points**. +**Effective pass rate**: 62/68 active test files have all real tests passing (91%). +Previous: 51/65 (78%) → 60/68 (88%) → Current: 62/68 (91%). --- @@ -266,7 +266,7 @@ perl -e '$SIG{__DIE__} = sub { print "S=", defined($^S) ? $^S : "undef", "\n" }; --- -## Remaining Real Failures (8 tests) +## Remaining Real Failures (6 tests) ### Tests needing DBI/Storage fixes — RESOLVED @@ -274,14 +274,15 @@ perl -e '$SIG{__DIE__} = sub { print "S=", defined($^S) ? $^S : "undef", "\n" }; |------|--------|----------------| | `t/64db.t` | **FIXED** (4/4 real pass) | `column_info()` implemented via SQLite PRAGMA (step 5.13) | | `t/752sqlite.t` | **FIXED** (34/34 real pass) | AutoCommit tracking + BEGIN/COMMIT/ROLLBACK interception (steps 5.14-5.15); `prepare_cached` per-dbh cache (step 5.16) | +| `t/00describe_environment.t` | **FIXED** (fully passing) | `$^S` correctly reports 1 inside `$SIG{__DIE__}` for `require` failures in `eval {}` (step 5.17) | +| `t/106dbic_carp.t` | **FIXED** (3/3 real pass) | `__LINE__` inside `@{[]}` string interpolation returns correct line number (step 5.18) | +| `t/100populate.t` | **PARTIAL** (test 2 fixed) | `execute_for_fetch` error propagation now matches real DBI (step 5.19); tests 37-42, 52-53, 58-59 are new failures exposed by progressing further (see Must Fix) | ### Tests needing caller/carp fixes | Test | Failing | Root cause | Fix needed | |------|---------|------------|------------| -| `t/106dbic_carp.t` | tests 2-3 | DBIx::Class::Carp callsite detection — `caller()` returns wrong package/line; also `__LINE__` inside `qr//` differs from Perl 5 | Fix `caller()` to return correct info through `namespace::clean`'d frames | -| `t/100populate.t` | test 2 | `execute_for_fetch()` doesn't throw on duplicate key constraint violation | DBI needs unique constraint error propagation | -| `t/101populate_rs.t` | test(s) | Similar to t/100populate.t — `execute_for_fetch` exception handling | Same as above | +| `t/101populate_rs.t` | test 4 | `warnings_like` doesn't find expected warning | Investigate warning emission during populate | ### Tests needing serialization/Storable fixes @@ -324,7 +325,8 @@ Subroutine set_subevents redefined at jar:PERL5LIB/Test2/Event/Subtest.pm line 3 | `t/40compose_connection.t` | 7 | All real tests pass | | `t/93single_accessor_object.t` | 15 | All real tests pass | | `t/752sqlite.t` | 25 | All 34 real tests pass | -| 36 other files | 5 each | Standard GC leak detection tests | +| `t/106dbic_carp.t` | 5 | All 3 real tests pass (fixed in step 5.18) | +| 37 other files | 5 each | Standard GC leak detection tests | --- @@ -449,12 +451,16 @@ Expressions like `($x) ? @$a = () : $b = []` trigger "Modification of a read-onl - 5.16: Fixed `prepare_cached` to use per-dbh `CachedKids` cache instead of global hash — prevents cross-connection cache pollution when multiple `:memory:` SQLite connections share the same DSN name; added `if_active` parameter handling - Also: `execute()` now handles metadata sth (no PreparedStatement) gracefully; `fetchrow_hashref` supports PRAGMA pre-fetched rows - Result: 60/68 active tests now pass all real tests (was 51/65 = 78%, now 88%) +- [x] Phase 5 steps 5.17–5.19 (2026-04-01) + - 5.17: Fixed `$^S` to correctly report 1 inside `$SIG{__DIE__}` when `require` fails in `eval {}` — temporarily restores `evalDepth` in `catchEval()` before calling handler. Unblocks t/00describe_environment.t + - 5.18: Fixed `__LINE__` inside `@{[expr]}` string interpolation — added `baseLineNumber` to Parser for string sub-parsers, computed from outer source position. Fixes t/106dbic_carp.t tests 2-3 + - 5.19: Fixed `execute_for_fetch` to match real DBI 1.647 behavior — tracks error count, stores `[$sth->err, $sth->errstr, $sth->state]` on failure, dies with error count if `RaiseError` is on. Also fixed `execute()` to set err/errstr/state on both sth and dbh. Fixes t/100populate.t test 2 + - Result: 62/68 active tests now pass all real tests (91%, was 88%) ### Next Steps -1. **Medium**: Fix caller/carp callsite detection (fixes t/106dbic_carp.t) -2. **Medium**: Fix `execute_for_fetch` exception propagation on constraint violations (fixes t/100populate.t, t/101populate_rs.t) -3. **Long-term**: Investigate VerifyError bytecode compiler bug (HIGH PRIORITY for broader CPAN compat) -4. **Pragmatic**: Accept GC-only failures as known JVM limitation; consider adding skip-leak-tests env var +1. **Must fix**: See "Must Fix" section — ternary-as-lvalue, JDBC error message format, SQL formatting, bind param attrs +2. **Long-term**: Investigate VerifyError bytecode compiler bug (HIGH PRIORITY for broader CPAN compat) +3. **Pragmatic**: Accept GC-only failures as known JVM limitation; consider adding skip-leak-tests env var ### Open Questions - `weaken`/`isweak` absence causes GC test noise but no functional impact — Option B (accept) or Option C (skip env var)? From 723eac2e8f8de4a596bebf293239459c7987a1f4 Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 16:15:59 +0200 Subject: [PATCH 06/10] Fix -w flag overriding 'no warnings "redefine"' pragma In Perl 5, lexical 'no warnings "redefine"' takes precedence over the global -w flag ($^W). PerlOnJava was checking $^W with a simple OR, causing subroutine redefinition warnings even inside no-warnings blocks. Now checks isWarningDisabled("redefine") first, matching Perl 5 behavior. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- src/main/java/org/perlonjava/core/Configuration.java | 2 +- .../java/org/perlonjava/frontend/parser/SubroutineParser.java | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index 0e4e4315a..0d2b426c3 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "abad323cf"; + public static final String gitCommitId = "977e22d72"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). diff --git a/src/main/java/org/perlonjava/frontend/parser/SubroutineParser.java b/src/main/java/org/perlonjava/frontend/parser/SubroutineParser.java index 7792fe5b0..edb9b1dc1 100644 --- a/src/main/java/org/perlonjava/frontend/parser/SubroutineParser.java +++ b/src/main/java/org/perlonjava/frontend/parser/SubroutineParser.java @@ -790,7 +790,8 @@ public static ListNode handleNamedSubWithFilter(Parser parser, String subName, S String msg = "Constant subroutine " + subName + " redefined" + location; org.perlonjava.runtime.operators.WarnDie.warn( new RuntimeScalar(msg), new RuntimeScalar("")); - } else if (dollarW || Warnings.warningManager.isWarningEnabled("redefine")) { + } else if (!Warnings.warningManager.isWarningDisabled("redefine") + && (dollarW || Warnings.warningManager.isWarningEnabled("redefine"))) { String msg = "Subroutine " + subName + " redefined" + location; org.perlonjava.runtime.operators.WarnDie.warn( new RuntimeScalar(msg), new RuntimeScalar("")); From 220e597b212727cb4b5e4630bb61da1c686f5dc1 Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 16:48:56 +0200 Subject: [PATCH 07/10] Fix InterpreterFallbackException not caught at top-level script compilation When ASM's Frame.merge() crashes on methods with extremely high fan-in (e.g. Sub::Quote-generated subs with 600+ jumps to a single label), EmitterMethodCreator throws InterpreterFallbackException. However, the top-level compileToExecutable() in PerlLanguageProvider didn't catch this exception type, causing the compilation to fail entirely instead of falling back to the interpreter. Add explicit catch for InterpreterFallbackException in compileToExecutable() that returns the pre-built interpreted code from the exception. This fixes DBIx::Class tests that use Sub::Quote (e.g. t/88result_set_column.t). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- .../app/scriptengine/PerlLanguageProvider.java | 10 ++++++++++ src/main/java/org/perlonjava/core/Configuration.java | 2 +- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/src/main/java/org/perlonjava/app/scriptengine/PerlLanguageProvider.java b/src/main/java/org/perlonjava/app/scriptengine/PerlLanguageProvider.java index 00d307006..2b164bb3d 100644 --- a/src/main/java/org/perlonjava/app/scriptengine/PerlLanguageProvider.java +++ b/src/main/java/org/perlonjava/app/scriptengine/PerlLanguageProvider.java @@ -7,6 +7,7 @@ import org.perlonjava.backend.jvm.CompiledCode; import org.perlonjava.backend.jvm.EmitterContext; import org.perlonjava.backend.jvm.EmitterMethodCreator; +import org.perlonjava.backend.jvm.InterpreterFallbackException; import org.perlonjava.backend.jvm.JavaClassInfo; import org.perlonjava.frontend.analysis.ConstantFoldingVisitor; import org.perlonjava.frontend.astnode.Node; @@ -433,6 +434,15 @@ private static RuntimeCode compileToExecutable(Node ast, EmitterContext ctx) thr ); return compiled; + } catch (InterpreterFallbackException fallback) { + // getBytecode() already compiled interpreter code as fallback + // when ASM frame computation failed (e.g., high fan-in to shared labels). + // Use the pre-compiled interpreter code directly. + boolean showFallback = System.getenv("JPERL_SHOW_FALLBACK") != null; + if (showFallback) { + System.err.println("Note: Using interpreter fallback (ASM frame compute crash)."); + } + return fallback.interpretedCode; } catch (Throwable e) { // Check if this is a recoverable compilation error that can use interpreter fallback // Catch Throwable (not just RuntimeException) because ClassFormatError diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index 0d2b426c3..e836912c4 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "977e22d72"; + public static final String gitCommitId = "723eac2e8"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). From 3368551e705be178c63ba0b472a932775786c935 Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 16:57:00 +0200 Subject: [PATCH 08/10] Implement MODIFY_CODE_ATTRIBUTES and fix ROLLBACK TO SAVEPOINT Two fixes for DBIx::Class compatibility: 1. Implement MODIFY_CODE_ATTRIBUTES call for subroutine attributes. When a sub is declared with attributes (sub foo : Attr { }), Perl calls MODIFY_CODE_ATTRIBUTES($package, \&code, @attrs) at compile time. This was not implemented, causing DBIx::Class ResultSetManager and any module using code attributes to fail. Fixes t/40resultsetmanager.t (5/5 planned tests now pass). 2. Fix ROLLBACK TO SAVEPOINT being intercepted as full ROLLBACK. DBI.java transaction control interceptor matched any SQL starting with ROLLBACK, including ROLLBACK TO SAVEPOINT. This caused savepoint operations to incorrectly reset autocommit. Fixes t/752sqlite.t (171/172 planned tests now pass). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- cpan_smoke_20260331_135137.dat | 39 +++++++++++++ cpan_smoke_20260331_142811.dat | 39 +++++++++++++ .../org/perlonjava/core/Configuration.java | 2 +- .../frontend/parser/SubroutineParser.java | 58 +++++++++++++++++++ .../perlonjava/runtime/perlmodule/DBI.java | 2 +- 5 files changed, 138 insertions(+), 2 deletions(-) create mode 100644 cpan_smoke_20260331_135137.dat create mode 100644 cpan_smoke_20260331_142811.dat diff --git a/cpan_smoke_20260331_135137.dat b/cpan_smoke_20260331_135137.dat new file mode 100644 index 000000000..080c0bb42 --- /dev/null +++ b/cpan_smoke_20260331_135137.dat @@ -0,0 +1,39 @@ +Test::Deep FAIL 1266 1268 pure-perl +Try::Tiny FAIL 91 94 pure-perl +Test::Fatal PASS 19 19 pure-perl +MIME::Base32 PASS 31 31 pure-perl +HTML::Tagset PASS 33 33 pure-perl +Test::Warn FAIL 0 14 pure-perl +Path::Tiny FAIL 1488 1542 pure-perl +namespace::clean CONFIG_FAIL pure-perl +Parse::RecDescent FAIL pure-perl +Spreadsheet::WriteExcel FAIL pure-perl +Image::ExifTool FAIL pure-perl +DateTime FAIL java-xs +Spreadsheet::ParseExcel FAIL java-xs +IO::Stringy FAIL pure-perl +Moo FAIL xs-with-pp-fallback +MIME::Base64 FAIL java-xs +URI FAIL pure-perl +IO::HTML FAIL pure-perl +LWP::MediaTypes FAIL pure-perl +Test::Needs FAIL pure-perl +Test::Warnings FAIL pure-perl +Encode::Locale FAIL pure-perl +Log::Log4perl FAIL 623 624 pure-perl +JSON FAIL 23683 24886 pure-perl +Type::Tiny FAIL 18 20 pure-perl +List::MoreUtils INSTALLED xs-with-pp-fallback +Template FAIL xs-with-pp-fallback +Mojolicious FAIL pure-perl +Devel::Cover FAIL xs-required +HTTP::Message FAIL pure-perl +HTML::Parser FAIL xs-required +IO::Compress::Gzip FAIL xs-required +Moose FAIL xs-required +Plack FAIL pure-perl +LWP::UserAgent FAIL pure-perl +DBIx::Class FAIL pure-perl +DBI FAIL xs-required +Params::Util FAIL xs-with-pp-fallback +Class::Load FAIL xs-with-pp-fallback diff --git a/cpan_smoke_20260331_142811.dat b/cpan_smoke_20260331_142811.dat new file mode 100644 index 000000000..16a79ae5e --- /dev/null +++ b/cpan_smoke_20260331_142811.dat @@ -0,0 +1,39 @@ +Test::Deep FAIL 1266 1268 pure-perl +Try::Tiny FAIL 91 94 pure-perl +Test::Fatal PASS 19 19 pure-perl +MIME::Base32 PASS 31 31 pure-perl +HTML::Tagset PASS 33 33 pure-perl +Test::Warn FAIL 0 14 pure-perl +Path::Tiny FAIL 1488 1542 pure-perl +namespace::clean FAIL 0 44 pure-perl +Parse::RecDescent FAIL 2 64 pure-perl +Spreadsheet::WriteExcel FAIL 1124 1189 pure-perl +Image::ExifTool PASS 600 600 pure-perl +DateTime FAIL 5 8 java-xs +Spreadsheet::ParseExcel PASS 1612 1612 java-xs +IO::Stringy PASS 127 127 pure-perl +Moo FAIL 809 840 xs-with-pp-fallback +MIME::Base64 FAIL 315 348 java-xs +URI FAIL 844 947 pure-perl +IO::HTML FAIL 0 52 pure-perl +LWP::MediaTypes FAIL 41 47 pure-perl +Test::Needs PASS 227 227 pure-perl +Test::Warnings FAIL 86 88 pure-perl +Encode::Locale FAIL 0 11 pure-perl +Log::Log4perl FAIL 715 719 pure-perl +JSON FAIL 23683 24886 pure-perl +Type::Tiny FAIL 18 20 pure-perl +List::MoreUtils INSTALLED xs-with-pp-fallback +Template FAIL 170 2072 xs-with-pp-fallback +Mojolicious TIMEOUT pure-perl +Devel::Cover PASS 1 1 xs-required +HTTP::Message PASS 0 0 pure-perl +HTML::Parser FAIL 190 415 xs-required +IO::Compress::Gzip FAIL 0 847 xs-required +Moose CONFIG_FAIL xs-required +Plack TIMEOUT pure-perl +LWP::UserAgent TIMEOUT pure-perl +DBIx::Class CONFIG_FAIL pure-perl +DBI FAIL 0 490 xs-required +Params::Util INSTALLED xs-with-pp-fallback +Class::Load FAIL 69 86 xs-with-pp-fallback diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index e836912c4..1b647eaf5 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "723eac2e8"; + public static final String gitCommitId = "220e597b2"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). diff --git a/src/main/java/org/perlonjava/frontend/parser/SubroutineParser.java b/src/main/java/org/perlonjava/frontend/parser/SubroutineParser.java index edb9b1dc1..b8c23640a 100644 --- a/src/main/java/org/perlonjava/frontend/parser/SubroutineParser.java +++ b/src/main/java/org/perlonjava/frontend/parser/SubroutineParser.java @@ -14,6 +14,7 @@ import org.perlonjava.frontend.semantic.SymbolTable; import org.perlonjava.runtime.debugger.DebugState; import org.perlonjava.runtime.mro.InheritanceResolver; +import org.perlonjava.runtime.perlmodule.Universal; import org.perlonjava.runtime.perlmodule.Warnings; import org.perlonjava.runtime.runtimetypes.*; @@ -818,6 +819,12 @@ public static ListNode handleNamedSubWithFilter(Parser parser, String subName, S placeholder.subName = subName; placeholder.packageName = parser.ctx.symbolTable.getCurrentPackage(); + // Call MODIFY_CODE_ATTRIBUTES if attributes are present + // In Perl, this is called at compile time after the sub is defined + if (attributes != null && !attributes.isEmpty()) { + callModifyCodeAttributes(packageToUse, codeRef, attributes, parser); + } + // Optimization - https://github.com/fglock/PerlOnJava/issues/8 // Prepare capture variables Map outerVars = parser.ctx.symbolTable.getAllVisibleVariables(); @@ -1029,6 +1036,57 @@ public static ListNode handleNamedSubWithFilter(Parser parser, String subName, S return result; } + /** + * Call MODIFY_CODE_ATTRIBUTES on the package if it exists. + * In Perl, when a subroutine is defined with attributes (sub foo : Attr { }), + * the package's MODIFY_CODE_ATTRIBUTES method is called at compile time with + * ($package, \&code, @attributes). If it returns any values, those are + * unrecognized attributes and an error is thrown. + */ + private static void callModifyCodeAttributes(String packageName, RuntimeScalar codeRef, + List attributes, Parser parser) { + // Check if the package has MODIFY_CODE_ATTRIBUTES + RuntimeArray canArgs = new RuntimeArray(); + RuntimeArray.push(canArgs, new RuntimeScalar(packageName)); + RuntimeArray.push(canArgs, new RuntimeScalar("MODIFY_CODE_ATTRIBUTES")); + + InheritanceResolver.autoloadEnabled = false; + RuntimeList codeList; + try { + codeList = Universal.can(canArgs, RuntimeContextType.SCALAR); + } finally { + InheritanceResolver.autoloadEnabled = true; + } + + if (codeList.size() == 1) { + RuntimeScalar method = codeList.getFirst(); + if (method.getBoolean()) { + // Build args: ($package, \&code, @attributes) + RuntimeArray callArgs = new RuntimeArray(); + RuntimeArray.push(callArgs, new RuntimeScalar(packageName)); + RuntimeArray.push(callArgs, codeRef); + for (String attr : attributes) { + RuntimeArray.push(callArgs, new RuntimeScalar(attr)); + } + + RuntimeList result = RuntimeCode.apply(method, callArgs, RuntimeContextType.LIST); + + // If MODIFY_CODE_ATTRIBUTES returns any values, they are unrecognized attributes + RuntimeArray resultArray = result.getArrayOfAlias(); + if (resultArray.size() > 0) { + StringBuilder sb = new StringBuilder(); + for (int i = 0; i < resultArray.size(); i++) { + if (i > 0) sb.append(", "); + sb.append("\"").append(resultArray.get(i).toString()).append("\""); + } + throw new PerlCompilerException(parser.tokenIndex, + "Invalid CODE attribute" + (resultArray.size() > 1 ? "s" : "") + ": " + sb, + parser.ctx.errorUtil); + } + } + } + } + private static SubroutineNode handleAnonSub(Parser parser, String subName, String prototype, List attributes, BlockNode block, int currentIndex) { // Now we check if the next token is one of the illegal characters that cannot follow a subroutine. // These are '(', '{', and '['. If any of these follow, we throw a syntax error. diff --git a/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java b/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java index 44db1903d..e3866650c 100644 --- a/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java +++ b/src/main/java/org/perlonjava/runtime/perlmodule/DBI.java @@ -317,7 +317,7 @@ public static RuntimeList execute(RuntimeArray args, int ctx) { String sqlUpper = strippedSql.toUpperCase(); boolean isBegin = sqlUpper.startsWith("BEGIN"); boolean isCommit = sqlUpper.startsWith("COMMIT") || sqlUpper.startsWith("END"); - boolean isRollback = sqlUpper.startsWith("ROLLBACK"); + boolean isRollback = sqlUpper.startsWith("ROLLBACK") && !sqlUpper.contains("SAVEPOINT"); Connection conn = (Connection) dbh.get("connection").value; From 2e00e78fcbce4255c58476491c5d3e2114bc48af Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 17:00:40 +0200 Subject: [PATCH 09/10] Support CODE reference returns from @INC hooks (PAR simulation) When an @INC hook returns a CODE reference (line-reader sub), Perl repeatedly calls it - the sub sets $_ to each line and returns true for more data, or false to stop. This pattern is used by PAR and similar module loaders. Previously only filehandle, scalar ref, and array ref returns from @INC hooks were handled. Now CODE ref returns are properly processed by calling the sub in a loop and accumulating lines from $_. Fixes t/90ensure_class_loaded.t tests 14 and 17 (27/28 now pass). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- .../org/perlonjava/core/Configuration.java | 2 +- .../runtime/operators/ModuleOperators.java | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/main/java/org/perlonjava/core/Configuration.java b/src/main/java/org/perlonjava/core/Configuration.java index 1b647eaf5..7b7d910c6 100644 --- a/src/main/java/org/perlonjava/core/Configuration.java +++ b/src/main/java/org/perlonjava/core/Configuration.java @@ -33,7 +33,7 @@ public final class Configuration { * Automatically populated by Gradle/Maven during build. * DO NOT EDIT MANUALLY - this value is replaced at build time. */ - public static final String gitCommitId = "220e597b2"; + public static final String gitCommitId = "3368551e7"; /** * Git commit date of the build (ISO format: YYYY-MM-DD). diff --git a/src/main/java/org/perlonjava/runtime/operators/ModuleOperators.java b/src/main/java/org/perlonjava/runtime/operators/ModuleOperators.java index 0382f2cf5..4b80ac31d 100644 --- a/src/main/java/org/perlonjava/runtime/operators/ModuleOperators.java +++ b/src/main/java/org/perlonjava/runtime/operators/ModuleOperators.java @@ -506,6 +506,25 @@ else if (code == null) { } catch (Exception e) { // Continue to next @INC entry } + } else if (hookResultScalar.type == RuntimeScalarType.CODE) { + // Hook returned a CODE reference (line-reader sub) + // Perl calls this repeatedly; the sub sets $_ to each line + // and returns true for more data, false to stop + RuntimeCode lineReader = (RuntimeCode) hookResultScalar.value; + RuntimeScalar dollarUnderscore = GlobalVariable.getGlobalVariable("main::_"); + StringBuilder codeBuilder = new StringBuilder(); + while (true) { + RuntimeArray readerArgs = new RuntimeArray(); + RuntimeBase result = lineReader.apply(readerArgs, RuntimeContextType.SCALAR); + if (result == null || !result.scalar().getBoolean()) { + break; + } + codeBuilder.append(dollarUnderscore.toString()).append("\n"); + } + code = codeBuilder.toString(); + actualFileName = fileName; + incHookRef = dirScalar; + break; } } // If hook returned undef or we couldn't use the result, continue to next @INC entry From 4f108619e5225215ad0ea0e9957531fececcdcb3 Mon Sep 17 00:00:00 2001 From: "Flavio S. Glock" Date: Wed, 1 Apr 2026 17:03:42 +0200 Subject: [PATCH 10/10] Update DBIx::Class plan with steps 5.20-5.24 progress and next steps - 68/314 test programs fully passing, 93.7% individual test pass rate - Documented fixes: -w/redefine, InterpreterFallbackException, MODIFY_CODE_ATTRIBUTES, ROLLBACK TO SAVEPOINT, @INC CODE hooks - Updated remaining failures and next steps list Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- dev/modules/dbix_class.md | 53 +++++++++++++++++++++++++++------------ 1 file changed, 37 insertions(+), 16 deletions(-) diff --git a/dev/modules/dbix_class.md b/dev/modules/dbix_class.md index 217100370..dee18e427 100644 --- a/dev/modules/dbix_class.md +++ b/dev/modules/dbix_class.md @@ -177,21 +177,25 @@ a module whose `.pod`/`.pm` files were previously installed as read-only (0444), | 5.14 | Add `AutoCommit` state tracking for literal transaction SQL | `DBI.java` | DONE | | 5.15 | Intercept BEGIN/COMMIT/ROLLBACK via JDBC API instead of executing SQL | `DBI.java` | DONE | | 5.16 | Fix `prepare_cached` to use per-dbh `CachedKids` cache | `DBI.pm` | DONE | +| 5.17 | Fix `-w` flag overriding `no warnings 'redefine'` pragma | `SubroutineParser.java` | DONE | +| 5.18 | Fix InterpreterFallbackException not caught at top-level | `PerlLanguageProvider.java` | DONE | +| 5.19 | Implement `MODIFY_CODE_ATTRIBUTES` for subroutine attributes | `SubroutineParser.java` | DONE | +| 5.20 | Fix ROLLBACK TO SAVEPOINT intercepted as full ROLLBACK | `DBI.java` | DONE | +| 5.21 | Support CODE reference returns from @INC hooks (PAR simulation) | `ModuleOperators.java` | DONE | **t/60core.t results** (17 tests emitted): - **ok 1–12**: Basic CRUD, update, dirty columns — all pass - **not ok 13–17**: Garbage collection tests — expected failures (JVM has no reference counting / `weaken`) - RowParser.pm line 260 crash still occurs in END block cleanup (non-blocking — all real tests pass first) -**Full test suite results** (92 test files, updated 2026-04-01): -- **22 fully passing** (no failures at all) -- **40 GC-only failures** (all real tests pass, only `weaken`-based GC leak tests fail) -- **6 tests with real failures** (see Remaining Real Failures below) -- **22 skipped** (DB-specific: Pg, Oracle, MSSQL, etc.; threads; fork) -- **2 compile-error** (t/52leaks.t — `wait` operator; t/88result_set_column.t — zero tests) +**Full test suite results** (314 test files with PERL5LIB, updated 2026-04-01): +- **68 fully passing** (no failures at all) +- **144 with some failures** (mostly GC-only; some have real failures) +- **100 skipped/errors** (DB-specific: Pg, Oracle, MSSQL, etc.; CDBI; threads; fork) +- **2 incomplete** (t/79aliasing.t, t/inflate/file_column.t — HandleError crash) +- **Individual test pass rate: 93.7%** (5579/5953 tests OK) -**Effective pass rate**: 62/68 active test files have all real tests passing (91%). -Previous: 51/65 (78%) → 60/68 (88%) → Current: 62/68 (91%). +Previous: 62/68 active (91%) → Current: 68/314 fully passing, 93.7% individual tests --- @@ -294,10 +298,13 @@ perl -e '$SIG{__DIE__} = sub { print "S=", defined($^S) ? $^S : "undef", "\n" }; | Test | Failing | Root cause | Fix needed | |------|---------|------------|------------| -| `t/90ensure_class_loaded.t` | tests 14,17,28 | PAR (Perl Archive) detection + `$INC{...}` manipulation edge cases | Fix `%INC` handling for modules that set `$INC{file}` without returning true | -| `t/40resultsetmanager.t` | tests 2-4 | Deprecated `ResultSetManager` uses source filtering (`Module::Pluggable` + runtime class creation) | Likely needs `Module::Pluggable` fixes or is acceptable as deprecated-feature failure | +| `t/90ensure_class_loaded.t` | test 28 | Error message has `{UNKNOWN}:` prefix and absolute path instead of relative | Fix error reporting path format in parser/compiler | | `t/53lean_startup.t` | test 5 | Module loading tracking — test checks exact set of loaded modules | PerlOnJava loads extra modules; would need to match exact Perl load footprint | +**Previously fixed:** +- `t/90ensure_class_loaded.t` tests 14,17 — fixed by implementing CODE reference returns from @INC hooks (PAR simulation) +- `t/40resultsetmanager.t` tests 2-4 — fixed by implementing `MODIFY_CODE_ATTRIBUTES` call for subroutine attributes + ### Tests needing misc fixes | Test | Failing | Root cause | Fix needed | @@ -323,10 +330,12 @@ Subroutine set_subevents redefined at jar:PERL5LIB/Test2/Event/Subtest.pm line 3 | Test | GC failures | Notes | |------|-------------|-------| | `t/40compose_connection.t` | 7 | All real tests pass | +| `t/40resultsetmanager.t` | 1 | All 5 real tests pass (fixed in step 5.20) | +| `t/88result_set_column.t` | 5 | 46/47 real tests pass (fixed by InterpreterFallbackException catch) | | `t/93single_accessor_object.t` | 15 | All real tests pass | -| `t/752sqlite.t` | 25 | All 34 real tests pass | +| `t/752sqlite.t` | 30 | 171/172 real tests pass (fixed ROLLBACK TO SAVEPOINT) | | `t/106dbic_carp.t` | 5 | All 3 real tests pass (fixed in step 5.18) | -| 37 other files | 5 each | Standard GC leak detection tests | +| Many other files | 5 each | Standard GC leak detection tests | --- @@ -451,16 +460,28 @@ Expressions like `($x) ? @$a = () : $b = []` trigger "Modification of a read-onl - 5.16: Fixed `prepare_cached` to use per-dbh `CachedKids` cache instead of global hash — prevents cross-connection cache pollution when multiple `:memory:` SQLite connections share the same DSN name; added `if_active` parameter handling - Also: `execute()` now handles metadata sth (no PreparedStatement) gracefully; `fetchrow_hashref` supports PRAGMA pre-fetched rows - Result: 60/68 active tests now pass all real tests (was 51/65 = 78%, now 88%) -- [x] Phase 5 steps 5.17–5.19 (2026-04-01) +- [x] Phase 5 steps 5.17–5.19 (2026-04-01, earlier session) - 5.17: Fixed `$^S` to correctly report 1 inside `$SIG{__DIE__}` when `require` fails in `eval {}` — temporarily restores `evalDepth` in `catchEval()` before calling handler. Unblocks t/00describe_environment.t - 5.18: Fixed `__LINE__` inside `@{[expr]}` string interpolation — added `baseLineNumber` to Parser for string sub-parsers, computed from outer source position. Fixes t/106dbic_carp.t tests 2-3 - 5.19: Fixed `execute_for_fetch` to match real DBI 1.647 behavior — tracks error count, stores `[$sth->err, $sth->errstr, $sth->state]` on failure, dies with error count if `RaiseError` is on. Also fixed `execute()` to set err/errstr/state on both sth and dbh. Fixes t/100populate.t test 2 - Result: 62/68 active tests now pass all real tests (91%, was 88%) +- [x] Phase 5 steps 5.20–5.24 (2026-04-01, current session) + - 5.20: Fixed `-w` flag overriding `no warnings 'redefine'` pragma — changed condition in SubroutineParser.java to check `isWarningDisabled("redefine")` first + - 5.21: Fixed `InterpreterFallbackException` not caught at top-level `compileToExecutable()` — ASM's Frame.merge() crashes on methods with 600+ jumps to single label (Sub::Quote-generated subs); added explicit catch in PerlLanguageProvider.java. Fixes t/88result_set_column.t (46/47 pass) + - 5.22: Implemented `MODIFY_CODE_ATTRIBUTES` call for subroutine attributes — when `sub foo : Attr { }` is parsed, now calls `MODIFY_CODE_ATTRIBUTES($package, \&code, @attrs)` at compile time. Fixes t/40resultsetmanager.t (5/5 pass) + - 5.23: Fixed ROLLBACK TO SAVEPOINT being intercepted as full ROLLBACK — `sqlUpper.startsWith("ROLLBACK")` now excludes SAVEPOINT-related statements. Fixes t/752sqlite.t (171/172 pass) + - 5.24: Added CODE reference returns from @INC hooks — PAR-style module loading where hook returns a line-reader sub that sets `$_` per line. Fixes t/90ensure_class_loaded.t tests 14,17 (27/28 pass) + - Result: 68/314 fully passing, 93.7% individual test pass rate (5579/5953 OK) ### Next Steps -1. **Must fix**: See "Must Fix" section — ternary-as-lvalue, JDBC error message format, SQL formatting, bind param attrs -2. **Long-term**: Investigate VerifyError bytecode compiler bug (HIGH PRIORITY for broader CPAN compat) -3. **Pragmatic**: Accept GC-only failures as known JVM limitation; consider adding skip-leak-tests env var +1. **JDBC error message format**: Strip/normalize JDBC error prefixes like `[SQLITE_MISMATCH]` in DBI.java `setError()` — affects t/100populate.t tests 52-53 +2. **SQL expression formatting**: Investigate column ordering / whitespace differences in SQL::Abstract output — affects t/100populate.t tests 37-42 +3. **Bind parameter attributes**: Implement attribute-aware bind parameter comparison — affects t/100populate.t tests 58-59 +4. **Ternary-as-lvalue**: Fix JVM backend to emit proper lvalue references for ternary branches with non-constant conditions — affects Class::Accessor::Grouped patterns +5. **Error message path format**: Fix `{UNKNOWN}:` prefix and absolute vs relative paths in error messages — affects t/90ensure_class_loaded.t test 28 +6. **File::stat VerifyError**: Debug bytecode generation for `Class::Struct` + `use overload` combination — affects Path::Class +7. **Long-term**: Investigate ASM Frame.merge() crash (the root cause behind step 5.18's fallback) — affects any Sub::Quote-generated sub with high fan-in control flow +8. **Pragmatic**: Accept GC-only failures as known JVM limitation; consider adding `DBIC_SKIP_LEAK_TESTS` env var ### Open Questions - `weaken`/`isweak` absence causes GC test noise but no functional impact — Option B (accept) or Option C (skip env var)?