Skip to content

WIP: Attribute system - attributes.pm, MODIFY/FETCH callbacks#420

Merged
fglock merged 23 commits intomasterfrom
feature/attribute-system
Apr 2, 2026
Merged

WIP: Attribute system - attributes.pm, MODIFY/FETCH callbacks#420
fglock merged 23 commits intomasterfrom
feature/attribute-system

Conversation

@fglock
Copy link
Copy Markdown
Owner

@fglock fglock commented Apr 1, 2026

Summary

Implement the Perl attributes pragma for PerlOnJava, enabling package-specific attribute handling.

Current baseline (62/216 = 28.7%)

  • op/attrs.t: 49/130
  • op/attrproto.t: 3/52 (incomplete)
  • op/attrhand.t: 0/0 (crash)
  • uni/attrs.t: 10/34

Planned work

  • Create src/main/perl/lib/attributes.pm with import(), get(), reftype()
  • Wire MODIFY_SCALAR/ARRAY/HASH_ATTRIBUTES dispatch for variable declarations
  • Wire FETCH_*_ATTRIBUTES callbacks in attributes::get()
  • Built-in attribute validation (lvalue, method, const, shared, prototype)
  • Attribute removal with - prefix
  • Error message formatting to match Perl 5

Test plan

  • make passes
  • Measure improvement on op/attrs.t, op/attrproto.t, op/attrhand.t, uni/attrs.t

Generated with Devin

fglock and others added 20 commits April 2, 2026 14:41
@fglock @devin-ai-integration
WIP: attribute system implementation
b9d43f6 
Update test-failures doc: (?{}) non-fatal workaround ruled out,
64-bit int ops noted as expected (32-bit declared), attribute system
marked as next target.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
docs: add attribute system implementation design
f078929 
Design doc covering attributes.pm, MODIFY/FETCH callbacks,
variable attribute dispatch, and implementation phases.

Baseline: 62/216 (28.7%) across attrs.t, attrproto.t, attrhand.t, uni/attrs.t.
Target: ~157/216 (73%).

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
feat: implement Perl attributes system
48be995 
- Create Attributes.java backend with _modify_attrs, _fetch_attrs,
  _guess_stash, reftype XS-equivalent functions
- Create attributes.pm Perl module with import/get/reftype
- Add callModifyVariableAttributes() in OperatorParser for my/our/state
  variable attribute dispatch via MODIFY_*_ATTRIBUTES
- Fix JVM backend anonymous sub attributes in EmitSubroutine
- Support space-separated attributes after single colon (: locked method)
- Fix prototype(...) attribute overriding parenthesized prototype
- Add "Unterminated attribute parameter" error message
- Add "Invalid separator character" error for bad chars in attr list
- Handle -shared variable attribute ("may not be unshared" error)
- Fix NPE when code.attributes is null
- Remove quotes from error message format to match Perl
- Make throwInvalidAttributeError package-accessible

Test improvement: 62/216 (28.7%) -> 179/244 (73.4%)
- attrs.t: 49 -> 131/158
- attrproto.t: 3 -> 25/52
- uni/attrs.t: 10 -> 23/34

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix warnings::warnif location reporting and auto-load attributes.pm
ce178eb 
- warnIf now walks caller stack to find first frame outside internal
  packages (attributes, warnings) for accurate location reporting
- getCallerLocation provides " at file line N" suffix for warning msgs
- Separate bitsLevel tracking: location from external caller, warning
  bits searched further up if null at that level (eval STRING workaround)
- Auto-require attributes.pm when attribute syntax (: attr) is used,
  making attributes::get() available without explicit 'use attributes'
- Also auto-require in callModifyVariableAttributes for variable attrs

attrs.t: 133 -> 138 passing (tests 130, 133, 33, 35, 36 now pass)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix compile-time warning scope for attributes and sync warning catego…
c0a08bc 
…ry aliases

- Fix emitCategoryWarning() to use compile-time scope as authoritative
  source during BEGIN/use processing, preventing incorrect warning bits
  from outer scopes leaking across eval boundaries
- Use Perl5-format bits string (not internal BitSet) for compile-time
  scope checks, correctly handling aliases like "illegalproto" and
  "syntax::illegalproto" that share Perl5 offset 47
- Sync warning hierarchy shorthand categories with their qualified forms:
  "illegalproto" -> "syntax::illegalproto", "prototype" -> "syntax::prototype",
  "digit" -> "syntax::digit", etc. so enable/disable propagates correctly
- attrproto.t: 46/52 -> 48/52 (tests 37-38 now pass)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Update attribute system design doc with Phase 1 completion and next s…
ab64dad 
…teps

- Document Phase 1 completion: Attributes.java, attributes.pm, warning
  scope fixes, category alias sync
- Update test results: 62/216 -> 205/244 (84%)
- Add detailed failure analysis for all 39 remaining test failures
  across attrs.t, attrproto.t, uni/attrs.t, attrhand.t
- Plan Phases 2-8 with estimated test gains per phase
- Categorize failures by root cause: _fetch_attrs, variable dispatch,
  my sub parsing, :const, error messages, closure prototypes

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix ref() for CODE stubs, interpreter backslash-ampersand, and attrib…
4f9bc5a 
…ute persistence

Three related fixes:

1. ReferenceOperators.ref(): Remove incorrect defined() check for CODE
   type. In Perl, ref of a stub always returns CODE even for forward-
   declared subs without a body.

2. BytecodeCompiler: Set isSymbolicReference on RuntimeCode when
   compiling backslash-ampersand-Name in the interpreter backend,
   matching the JVM backend createCodeReference behavior. This fixes
   defined of a stub returning false inside eval strings.

3. SubroutineParser: Preserve attributes through sub redefinition.
   - Forward decl attrs persist when body is added later
   - Re-declaration merges new attrs into existing sub

Also fixes _guess_stash for blessed CODE refs in Attributes.java.

attrs.t: 134/158 -> 140/158 (+6 tests)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix reserved word warning scope, CvSTASH dispatch, and attribute para…
719daaa 
…meter parsing

- emitReservedWordWarning: Use compile-time symbol table to check
  syntax::reserved category instead of runtime WarnDie.warnWithCategory.
  Fixes tests 27-28 in attrs.t where no warnings reserved later in
  the file leaked into earlier eval strings.

- MODIFY_CODE_ATTRIBUTES dispatch: Use existing code ref packageName
  (CvSTASH equivalent) instead of current package. Fixes test 32 where
  *Y::bar = \&X::foo; sub Y::bar : attr should dispatch X::MODIFY_CODE_ATTRIBUTES.

- Set packageName from fully-qualified sub name (e.g., sub X::foo gets
  packageName X not main), matching Perl 5 CvSTASH behavior.

- Attribute parameter parsing: Use raw string parsing instead of q()-style
  parsing. Perl attribute parameters preserve backslashes literally
  (e.g., :Foo(\() gives parameter \( not (). Fixes test 20 in attrs.t
  and 3 tests in uni/attrs.t.

Test results: attrs.t 143/156 (was 140/158), uni/attrs.t 26/32 (was 23/34)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 4: Add attribute parsing after prototype for my/state sub
8ca671c 
Lexical sub declarations (my sub, state sub) now parse attributes
after parenthesized prototypes, matching the behavior of package subs.

For 'my sub lexsub1(bar) : prototype(baz) {}':
- Emit illegal proto warning for (bar) paren prototype
- Parse :prototype(baz) attribute after the paren prototype
- Emit illegal proto warning for (baz) attribute prototype
- Emit 'Prototype overridden' warning
- Create SubroutineNode with the final prototype and attributes

Test results: attrproto.t 51/52 (was 48/52, +3 tests fixed: 49-51)
Test 48 remains due to pre-existing \&lexsub issue in eval context.

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 5: :const attribute support and MODIFY_CODE_ATTRIBUTES dispatch
54a1393 
- Implement const folding: when :const is applied to a sub with a callable
  body, invoke it immediately and store the result as constantValue
- Deep-copy the folded result to prevent aliased mutable variables from
  being modified after const-folding (fixes test 143)
- Clear constantValue when -const removal is applied
- Add runtimeDispatchModifyCodeAttributes() for anonymous subs with
  non-builtin attributes (e.g., sub : Const { ... })
- Emit bytecode in EmitSubroutine to call the dispatch after anonymous
  sub creation
- Add constantValue return path in RuntimeCode.apply() methods

attrs.t: 145/156 (up from 144)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix :const attribute for interpreter backend (eval STRING)
2746c40 
- Add MODIFY_CODE_ATTRIBUTES dispatch in interpreter's closure creation
  (OpcodeHandlerExtended.executeCreateClosure) and non-closure path
  (BytecodeCompiler)
- Fix hasCallableBody check in Attributes.applyAttribute to recognize
  InterpretedCode as callable (it uses bytecode, not subroutine/methodHandle)
- Add constantValue check in InterpretedCode.apply() overrides to return
  the cached const-folded value (the overrides were bypassing RuntimeCode's
  constantValue check

attrs.t: 146/156 (up from 145)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
EOF
)
@fglock @devin-ai-integration
Fix reserved word warning: only warn for lowercase attribute names
8f53644 
Only emit 'may clash with future reserved word' warning for all-lowercase
attribute names, matching Perl 5 behavior. Previously TieLoop (mixed case)
incorrectly triggered the warning.

attrs.t: 147/158 (TieLoop warning fixed, 2 new sub-tests exposed)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 6: Detect scalar dereference in my/our/state declarations
7ae6ab6 
Add checkForDereference() to detect expressions like 'our ${""}' and
'my $$foo' which are dereferences, not simple variables. Perl 5 errors
with 'Can'\''t declare scalar dereference in "our"' etc.

attrs.t: 149/158, uni/attrs.t: 29/34

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 7: Attribute::Handlers support, isDeclared flag, error format fix
86f07e9 
- Add isDeclared flag to RuntimeCode to track explicitly declared subs
  (forward declarations and sub definitions). In Perl 5, declared subs
  are visible via *{glob}{CODE} even before their body is compiled.

- Update getGlobSlot(CODE) in RuntimeGlob to return code refs for
  declared subs, fixing Attribute::Handlers findsym() function which
  iterates the stash looking for CODE slots.

- Fix variable attribute error format for SCALAR/ARRAY/HASH attributes
  to match Perl 5 use attributes style with BEGIN failed suffix.

- Update design doc with closure prototype feature details and progress.

Test improvements:
- attrhand.t: 4/4 (new, all pass)
- attrs.t: 152/158 (was 149/158, +3)
- Total: 236/248 (was 233/248)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 8: Fix NPE in reference operations, add caller info for attribu…
286f3bf 
…te handlers

- Fix NPE in blessedId() when reference-typed scalars have null value
  (e.g., ${^LAST_SUCCESSFUL_PATTERN} before any regex match)
- Fix ${^LAST_SUCCESSFUL_PATTERN} to return undef instead of REGEX(null)
  when no regex match has occurred yet
- Add null-safety checks in ReferenceOperators.ref() for all reference types
- Push CallerStack frames in callModifyCodeAttributes/callModifyVariableAttributes
  so Attribute::Handlers can get source file/line via caller(2)
- Add CallerStack fallback in caller() builtin for frames beyond Java stack

Test improvements:
- multi.t: 0/0 (crash) -> 45/51 passing
- linerep.t: 13/18 -> 15/18 passing (filename/linenum for CODE attrs)
- All other attribute tests unchanged (no regressions)
- Remaining multi.t failures: DESTROY (unimplemented), END handler warning
- Remaining linerep.t failures: eval context file/line, my var ref identity

Also: minor fixes to IO/Socket/INET.pm (safer Errno check) and
overload.pm (remove AddrRef prototype)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Update attributes design doc with Phase 8 progress tracking
9bd0f54 
Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Phase 3: Runtime attribute dispatch for my/state variable declarations
c3af1f8 
Move MODIFY_*_ATTRIBUTES dispatch from compile-time to runtime for
my/state variables, so the reference passed to the handler points to
the actual lexical variable (not a disconnected temporary). This
enables tie operations inside attribute handlers to work correctly.

Changes:
- Attributes.java: Add runtimeDispatchModifyVariableAttributes() method
  with CallerStack support for Attribute::Handlers compatibility
- OperatorParser.java: Skip compile-time dispatch for my/state (keep for
  our), store attributePackage annotation for the emitter
- EmitVariable.java: Emit runtime attribute dispatch bytecode after
  variable creation in handleMyOperator (JVM backend)
- BytecodeCompiler.java: Add DISPATCH_VAR_ATTRS opcode emission and
  emitVarAttrsIfNeeded() helper (interpreter backend)
- CompileAssignment.java: Handle attributes in assignment paths
  (my $x : attr = val) with proper ordering: create scalar, dispatch
  attributes (which may tie), then assign value so STORE fires
- Opcodes.java: Add DISPATCH_VAR_ATTRS opcode (451)
- SlowOpcodeHandler.java: Implement executeDispatchVarAttrs()
- BytecodeInterpreter.java: Wire DISPATCH_VAR_ATTRS to handler

Test results:
- attrs.t: 155/159 (was 152/158, +3 passing)
- uni/attrs.t: 32/35 (was 29/34, +3 passing)
- multi.t: 53/57 (was 45/51, +8 passing)

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Implement closure prototype semantics for MODIFY_CODE_ATTRIBUTES
b4c434a 
When MODIFY_CODE_ATTRIBUTES is called for a closure (a sub that captures
lexical variables), the handler now receives a "closure prototype" - the
original non-callable CV. The expression result is a callable clone.
Calling a captured closure prototype dies with "Closure prototype called",
matching Perl 5 behavior.

Changes:
- RuntimeCode: add isClosurePrototype flag, cloneForClosure() method,
  and prototype checks in all apply() overloads
- Attributes: overloaded runtimeDispatchModifyCodeAttributes with
  isClosure parameter; when true, clones code and marks original as prototype
- EmitSubroutine (JVM): detect closures via captured variable count,
  pass isClosure flag to dispatch method
- OpcodeHandlerExtended (interpreter): pass isClosure=true for closures

Test results: attrs.t 157/159, uni/attrs.t 34/35, multi.t 53/57.
Remaining failures are pre-existing (DESTROY unimplemented, error ordering).

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Implement parse-time strict vars checking (perl #49472)
db38bd0 
Add strict vars checking at parse time so that undeclared variables
inside lazily-compiled named subroutine bodies are caught at compile
time rather than at call time (or never).

Key changes:
- Variable.java: Add checkStrictVarsAtParseTime() with comprehensive
  exemption logic mirroring EmitVariable/BytecodeCompiler checks
- OperatorParser.java: Set parsingDeclaration flag to suppress strict
  check while parsing my/our/state variable declarations
- Parser.java: Add parsingDeclaration flag
- SignatureParser.java: Register signature parameters in the symbol
  table during parsing so default values and sub body can reference them
- SubroutineParser.java: Enter scope for signature variables before
  parsing signature, exit after block body
- StatementParser.java: Register catch variable in scope and suppress
  strict check for catch parameter declaration

Test results:
- attrs.t: 158/159 (was 157/159) - test 88 now passes
- uni/attrs.t: 35/35 (was 34/35) - test 24 now passes
- All unit tests pass

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Update attributes design doc with strict vars fix progress
165f43c 
Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock fglock force-pushed the feature/attribute-system branch from 4bf86e9 to 165f43c Compare April 2, 2026 12:42
fglock and others added 3 commits April 2, 2026 15:17
@fglock @devin-ai-integration
Fix strict vars check for class features and eval STRING contexts
3403e69 
- Scope parse-time strict vars check to named subroutine bodies only,
  since that is the specific case where lazy compilation prevents the
  existing code-generation-time check from firing. This fixes the
  eval STRING regression (re/pat_advanced.t, re/pat.t, op/goto.t).

- Register self in scope before signature parsing for regular methods
  and lexical (my) methods in class bodies.

- Register self in scope for ADJUST blocks in class bodies.

- Register field variables with sigil in symbol table so later field
  default expressions can reference earlier fields under strict vars.

Generated with Devin (https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Document regression analysis for decl-refs.t, lexsub.t, deprecate.t
f98f6f7 
Three tests regressed vs PR #417 baseline:
- op/decl-refs.t: 322->174 (state declared-ref attribute dispatch bugs)
- op/lexsub.t: 105->0 (state sub prototype not visible after scope changes)
- lib/deprecate.t: 4->0 (defined(&foo) wrong for forward declarations)

Generated with Devin (https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock @devin-ai-integration
Fix all 3 PR #417 test regressions in attribute system branch
3a6de9b 
- decl-refs.t: 174→330 (baseline 322) — defer variable attribute
  validation for all declarator types (my/state/our) since the
  pre-existing \K regex bug corrupts handler names, making handlers
  invisible to compile-time checks. Runtime dispatch silently returns
  when no handler is found. Document \K bug in attributes.md.
- lexsub.t: 0→105 (baseline 105) — add prototype annotation for
  state sub full definitions in StatementResolver
- deprecate.t: 0→4 (baseline 4) — already fixed by prior commit

Also: emit runtime attribute dispatch for state variables
(BytecodeCompiler was missing emitVarAttrsIfNeeded in state path).

Generated with [Devin](https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@fglock fglock merged commit a40e7c5 into master Apr 2, 2026
2 checks passed
@fglock fglock deleted the feature/attribute-system branch April 2, 2026 15:22
@fglock fglock mentioned this pull request Apr 2, 2026
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@fglock