Skip to content

WIP: Lexical warnings - Perl 5 parity#400

Open
fglock wants to merge 23 commits intomasterfrom
feature/lexical-warnings-phase1
Open

WIP: Lexical warnings - Perl 5 parity#400
fglock wants to merge 23 commits intomasterfrom
feature/lexical-warnings-phase1

Conversation

@fglock
Copy link
Copy Markdown
Owner

@fglock fglock commented Mar 29, 2026 •
edited
Loading

Summary

Implements Perl 5 compatible lexical warnings with zero runtime overhead when warnings are disabled.

Phase 1 (this PR): Infrastructure

  • Create WarningBitsRegistry for class name → warning bits mapping
  • Add Perl 5 compatible category offsets to WarningFlags
  • Add warningFatalStack to ScopedSymbolTable for FATAL warnings
  • Add getWarningBitsString() for caller()[9] support
  • Fix warning bits layout to match Perl 5's exact positions from warnings.h
  • Add qualified name aliases (e.g., io::closed → offset 6)
  • Add $warnings::BYTES = 21 constant

Test results:

  • op/caller.t test 3 passes: ${^WARNING_BITS} produces correct \x55 x 21 pattern
  • op/caller.t test 28 requires per-call-site warning bits (Phase 9)

Remaining phases:

  • Phase 2: Two-variant operator methods (add() vs addWarn() pattern)
  • Phase 3: Per-closure warning bits storage for JVM backend
  • Phase 4: Per-closure warning bits storage for interpreter
  • Phase 5: caller()[9] warning bits implementation
  • Phase 6: warnings:: Perl module functions
  • Phase 7: FATAL warnings implementation
  • Phase 8: $^W interaction with lexical warnings
  • Phase 9: Per-call-site warning bits for caller()

See dev/design/lexical-warnings.md for full design document.

Test plan

  • make passes (all unit tests)
  • Warning bits format matches Perl 5's WARN_ALLstring
  • Add tests for FATAL warnings
  • Add tests for caller()[9] per-call-site bits

Generated with Devin

fglock and others added 23 commits March 29, 2026 10:17
@fglock @devin-ai-integration
Lexical warnings Phase 1: Infrastructure
1eb9d89 
Implements the infrastructure for Perl 5 compatible lexical warnings:

- Create WarningBitsRegistry.java: HashMap registry mapping class name
  to compile-time warning bits, enabling caller()[9] lookups

- Enhance WarningFlags.java:
  - Add PERL5_OFFSETS map with Perl 5 compatible category offsets
  - Add userCategoryOffsets for warnings::register support
  - Add toWarningBitsString() for Perl 5 compatible bits format
  - Add isEnabledInBits() and isFatalInBits() utility methods
  - Add registerUserCategoryOffset() for dynamic category allocation

- Enhance ScopedSymbolTable.java:
  - Add warningFatalStack for FATAL warnings tracking
  - Update enterScope()/exitScope() to handle fatal stack
  - Update snapShot() and copyFlagsFrom() to copy fatal stack
  - Add fatal warning category methods
  - Add getWarningBitsString() for caller()[9] support

This is Phase 1 of the lexical warnings implementation as documented
in dev/design/lexical-warnings.md.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Lexical warnings Phase 2: add/subtract warn variants
abb9da9 
Add warn variants for + and - operators that check for uninitialized
values when 'use warnings "uninitialized"' is in effect.

- MathOperators.java: Added addWarn() and subtractWarn() methods
  - Fixed tied scalar double-fetch issue by calling getNumber() first
    then checking for scalarZero (the cached value returned for UNDEF)
  - This ensures a single FETCH for tied scalars while still detecting
    uninitialized values correctly

- OperatorHandler.java: Added +_warn, -_warn operator entries and
  getWarn() method to get warning variant names

- EmitOperator.java: Modified emitOperator() and emitOperatorWithKey()
  to select warn variants based on isWarningCategoryEnabled("uninitialized")

- EmitBinaryOperatorNode.java: Updated binary operator switch to use
  warn variants for + and - when warnings are enabled

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Refactor: move uninitialized warning to getNumberWarn()
28b048a 
Centralize the uninitialized value warning logic in RuntimeScalar.getNumberWarn()
instead of checking for scalarZero in each operator method.

Benefits:
- Single place for warning logic, easier to maintain
- Correctly handles tied scalars (fetch once, then check the fetched value)
- Reusable by all arithmetic operators (*, /, %, etc.)
- Cleaner operator implementations

RuntimeScalar.java:
- Added getNumberWarn(String operation) method that checks for UNDEF
  before converting to number, emitting warning when needed
- For tied scalars, fetches first then recursively checks the fetched value

MathOperators.java:
- Simplified addWarn() and subtractWarn() methods to use getNumberWarn()
- Removed scalarZero identity checks

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 2: Add warn variants for all arithmetic operators
f19cbc4 
Add two-variant pattern for *, /, %, **, and unary - operators.
When 'use warnings "uninitialized"' is enabled, the warn variants
are called, checking for undefined values via getNumberWarn().

Changes:
- MathOperators.java:
  - Refactored multiply() to remove inline warnings (fast path)
  - Added multiplyWarn() using getNumberWarn()
  - Refactored divide() to remove inline warnings (fast path)
  - Added divideWarn() using getNumberWarn()
  - Added modulusWarn() for % operator
  - Refactored pow() to remove inline warnings (fast path)
  - Added powWarn() for ** operator
  - Added unaryMinusWarn() for unary - operator

- OperatorHandler.java:
  - Added *_warn, /_warn, %_warn, **_warn, unaryMinus_warn entries

The emitter (EmitBinaryOperatorNode, EmitOperator) already uses
OperatorHandler.getWarn() which automatically selects the warn
variant when uninitialized warnings are enabled.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Update lexical-warnings.md: Phase 2 complete
49ce45a 
Document completion of Phase 2 (Two-variant operator methods):
- getNumberWarn() for centralized undef checking
- Warn variants for all arithmetic operators
- OperatorHandler entries for warn variants

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 3: Per-closure warning bits storage for JVM backend
d29b427 
Implement caller()[9] support for the JVM backend by storing
compile-time warning bits in generated classes and registering
them with WarningBitsRegistry.

Changes:
- EmitterMethodCreator: Add WARNING_BITS static field and <clinit>
  static initializer to register bits with WarningBitsRegistry
- RuntimeCode: Add extractJavaClassNames() helper to extract class
  names from stack trace, update callerWithSub() to look up warning
  bits from registry for element 9

Known limitation: Warning bits are per-class, not per-call-site.
All calls from the same class share the same warning bits, but
different closures correctly get their own warning bits.

Refs: dev/design/lexical-warnings.md

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 4: Per-closure warning bits storage for interpreter
9a0090d 
Implement caller()[9] support for the interpreter backend by storing
compile-time warning bits in InterpretedCode and registering them
with WarningBitsRegistry.

Changes:
- InterpretedCode: Add warningBitsString field, register with
  WarningBitsRegistry in constructor using identity hash code key
- BytecodeCompiler: Extract warningBitsString from symbolTable
  and pass to InterpretedCode constructor

Both JVM and interpreter backends now support caller()[9] returning
the compile-time warning bits for the calling frame.

Refs: dev/design/lexical-warnings.md

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 6: warnings:: functions using caller()[9]
8a500ef 
Update warnings:: module functions to use caller()[9] for checking
warning bits from the calling scope. This enables proper lexical
warning control to work across subroutine calls.

Changes to Warnings.java:
- Add getWarningBitsAtLevel() helper to get warning bits from caller()
- enabled() now uses caller()[9] with WarningFlags.isEnabledInBits()
- warnif() checks caller()[9] and handles FATAL warnings (dies if fatal)
- Add fatal_enabled() using WarningFlags.isFatalInBits()
- Add enabled_at_level() for checking at specific stack levels
- Add fatal_enabled_at_level() for FATAL check at specific levels
- Add warnif_at_level() for warning at specific stack levels

New registered methods:
- warnings::enabled_at_level
- warnings::fatal_enabled
- warnings::fatal_enabled_at_level
- warnings::warnif_at_level

Refs: dev/design/lexical-warnings.md

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix FATAL warnings bits propagation to named subroutines
234a420 
Phase 7 of lexical warnings: Fixed warnings::fatal_enabled() to correctly
report FATAL status by:

1. SubroutineParser.java: Copy warningFatalStack and warningDisabledStack
   when creating filteredSnapshot for named subroutines. This was the root
   cause - named subs were not inheriting FATAL flags from their definition
   scope.

2. Warnings.java: Updated getWarningBitsAtLevel() to use level+1 to skip
   the Java implementation frame (Warnings.java) in the caller stack trace.

3. WarningFlags.java: Added isFatalInBits() method to check FATAL bits in
   a Perl 5 format warning bits string.

4. CompilerFlagNode.java: Added warningFatalFlags and warningDisabledFlags
   fields to track FATAL and disabled state separately from enabled state.

5. EmitCompilerFlag.java: Apply fatal and disabled flags from CompilerFlagNode.

6. EmitterMethodCreator.java: Added applyCompilerFlagNodes() to pre-apply
   CompilerFlagNodes so WARNING_BITS captures effective flags including FATAL.

7. BytecodeCompiler.java: Push/pop warningFatalStack and warningDisabledStack
   in enterScope/exitScope for interpreter.

8. StatementParser.java: Pass fatalFlags and disabledFlags to CompilerFlagNode.

Test results:
- warnings::fatal_enabled() now correctly returns true when 'use warnings
  FATAL => "all"' is in effect, including for nested subroutine calls.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Implement FATAL warnings for 'uninitialized' category
72f81b7 
- Add warnWithCategory() method to check if warning category is FATAL
- Use caller() to look up warning bits from Perl code's scope
- Convert warning to die() when category is marked FATAL
- Update StringOperators to use warnWithCategory() for concat warnings

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Add ThreadLocal warning bits context stack for FATAL warnings
97b6a09 
- Add pushCurrent/popCurrent/getCurrent to WarningBitsRegistry for tracking
  current warning context during code execution
- Update RuntimeCode.apply() to push/pop warning bits around code execution
- Update InterpretedCode.apply() to push/pop warning bits
- Modify warnWithCategory() to check both caller() and context stack

This enables FATAL warnings to work correctly for:
- File-scope 'use warnings FATAL => ...'
- Named subroutines inheriting or setting their own warning bits
- Top-level code (no named subroutine)

Note: Block-scoped 'use warnings FATAL' inside a subroutine/program
doesn't work due to per-class warning bits storage (not per-scope).

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Update design doc: Phase 7-8 FATAL warnings progress
49bb1f3 
- Document FATAL warnings implementation status
- Note ThreadLocal context stack approach
- Document known limitation for block-scoped FATAL warnings

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Implement $^W interaction for lexical warnings (Phase 8)
e2556ea 
- Add isWarnFlagSet() helper to check $^W global variable
- Update warnif() to fall back to $^W when category not enabled
- Update warnIfAtLevel() with same $^W fallback logic
- Update design doc with Phase 8 completion details

$^W now works with warnings::warnif() when lexical warnings
are not enabled for the category being checked.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Add Phase 9 plan: per-call-site warning bits
8adc4f1 
Document the future approach for block-scoped use warnings / no warnings
support, including implementation approaches, trade-offs, and files to modify.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
docs: Add lexical warnings to changelog and feature matrix
c39009c 
Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix compound assignment warnings not working with -w flag
688dfc9 
- Add warn variants for compound assignment operators (+=, -=, *=, /=, %=)
- Update EmitBinaryOperator to select warn variant when warnings enabled
- Register warn variants in OperatorHandler

This fixes the regression where `$x *= 1` would not warn about
uninitialized values when using `-w` or `use warnings`.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix spurious uninitialized warnings for regex capture variables
22b4afa 
ScalarSpecialVariable (used for regex captures like $1, $2) was missing
getNumberWarn() override. Since the proxy type field is UNDEF,
getNumberWarn() incorrectly reported uninitialized warnings for
defined capture values.

This fix adds getNumberWarn() override that properly delegates to
getValueAsScalar().getNumberWarn().

Fixes regressions in opbasic/arith.t (183/183) and op/negate.t (48/48).

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix -w flag to set \$^W and fix compound assignment warnings
8cc3c4b 
Key changes:
- -w flag now sets \$^W = 1 (was incorrectly using "use warnings")
- Added Warnings.shouldWarn() to check warnings at runtime
- Updated getNumberWarn() to check shouldWarn() before emitting
- Updated bytecode interpreter to use warn variants for *=, /=, %=, **=
- Only warn-enabled operators (* / % ** << >> x &) emit warnings,
  while + - . | ^ do not (matching Perl behavior)

This fixes op/assignwarn.t (106 -> 116) and run/switches.t (36 -> 38).

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix default warning bits to match Perl semantics
dc503e8 
- Remove default experimental warnings from ScopedSymbolTable constructor
  Code without explicit 'use warnings' now has empty warning bits,
  matching Perl behavior where default warning state is empty.

- Add $BYTES = 21 to warnings.pm for proper warning bits testing
  This allows caller()[9] tests to compare against the expected
  number of warning bytes.

This partially fixes op/caller.t test 27 (warnings match caller).
Test 28 still fails because it requires per-call-site warning bits
(Phase 9 of the lexical warnings design) - a known limitation where
caller()[9] returns bits from subroutine definition site, not call site.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Implement ${^WARNING_BITS} special variable (read/write)
7610f3d 
- Add WARNING_BITS to ScalarSpecialVariable with getter and setter
- Register ${^WARNING_BITS} as a special variable in GlobalContext
- Inherit warning flags from caller scope in executePerlAST
  This fixes BEGIN blocks not seeing warnings from 'use warnings'
- Add setWarningBitsFromString() to parse warning bits and update
  symbol table flags (reverse of toWarningBitsString)
- Add $warnings::BYTES = 21 constant for proper warning bits testing

This fixes Test::More's cmp_ok() which sets ${^WARNING_BITS} in
eval blocks to restore warning state. Without the setter, tests
would fail with "Modification of a read-only value attempted".

op/caller.t: 44/112 (was 46/112 on master, -2)
- Test 3: bit layout differs from Perl 5's exact offsets (known)
- Test 28: requires per-call-site warning bits (Phase 9)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix use integer/strict applying to code before the pragma
3fbed75 
The applyCompilerFlagNodes() function was pre-applying ALL compiler
flags (including feature flags and strict options) before emitting
code. This caused 'use integer' and 'use strict' to incorrectly
affect code that appeared BEFORE them in the source file.

The fix is to only pre-apply warning flags (needed for WARNING_BITS
capture), not feature/strict flags which must be applied in order
during code emission to maintain proper lexical scoping.

This fixes run/fresh_perl.t test 2 which was failing because:
  $cusp = ~0 ^ (~0 >> 1);  # Should NOT use integer semantics
  use integer;              # Should only affect code AFTER this

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix warning bits layout to match Perl 5 exact positions
df6f5aa 
- Add qualified name aliases in PERL5_OFFSETS (e.g., io::closed -> 6)
- Add missing categories: missing_import, experimental::enhanced_xx
- Add placeholder categories __future_81-83 to match WARN_ALLstring
- Fix experimental::signature_named_parameters offset (78 -> 79)
- Update warnings.pm to declare BYTES = 21

This fixes op/caller.t test 3 (default bits on via use warnings).
Test 28 still fails because it requires per-call-site warning bits
(Phase 9 feature) - it now checks real values instead of passing
vacuously with empty strings.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix experimental warning regression: enable warnings when features ar…
7c9516d 
…e enabled

- Sort getWarningList() for stable bit positions across runs
- Enable experimental::X warning when 'use feature X' enables an experimental feature
- In Perl 5, experimental warnings are ON by default unless explicitly disabled

This fixes the op/decl-refs.t regression (322/408 restored from 232/408).

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@fglock