fix: defer coordinate parsing for swipe commands with script variables

[vazidmansuri005]

Summary

Fixes #2149 — The output of relativePoint() JavaScript function can be used with tapOn (as the point parameter) but cannot be used with swipe (as start/end parameters). This is because swipe coordinate strings are parsed to Point objects during YAML deserialization, before evaluateScripts() resolves ${...} variables, causing a NumberFormatException.

Root cause: YamlFluentCommand.toCommand() calls start.split(",").map { it.trim().toInt() } at parse time, which throws on ${output.startPoint}.

Fix: Defer coordinate parsing to execution time when script variables are detected:

1. YamlSwipeDeserializer — Detects ${ in start/end values and skips format validation, returning a raw YamlCoordinateSwipe
2. YamlFluentCommand — When ${ is detected, stores raw strings in startPointStr/endPointStr instead of parsing to Point
3. SwipeCommand — New startPointStr/endPointStr fields that pass through evaluateScripts()
4. Orchestra — Parses the evaluated strings at execution time, supporting both absolute (x,y) and relative (x%,y%) coordinates

Before (broken)

- evalScript: ${output.startPoint = relativePoint(0.5, 0.8)}
- swipe:
    start: ${output.startPoint}  # NumberFormatException at parse time
    end: 50%,12%

After (works)

- evalScript: ${output.startPoint = relativePoint(0.5, 0.8)}
- swipe:
    start: ${output.startPoint}  # Deferred until runtime
    end: 50%,12%

Test plan

• Existing swipe tests pass (./gradlew :maestro-orchestra:test)
• Builds cleanly (compileKotlin for both maestro-orchestra and maestro-orchestra-models)
• Unit test: swipe with script variables defers coordinate parsing — verifies YAML with ${...} variables produces SwipeCommand with startPointStr/endPointStr
• Integration test: Case 139 - Swipe with script variables — verifies full execution path through Orchestra with FakeDriver
• Non-variable swipe commands (direction, absolute coords, relative coords) unaffected
• Manual E2E: iPhone 16 simulator (iOS 18.5) — relativePoint() and absolute coordinate variables both execute successfully via installLocally.sh

[Fishbowler]

Should this be draft until your testing is complete?

This also feels like this would be worth both an automated test.

I think your examples are incorrect (or possibly my memory) - I don't think assertVisible sets those values.

[vazidmansuri005]

Thanks for the review @Fishbowler — all three points addressed:

1. Examples fixed — You were right, assertVisible does not set output.startPoint/output.endPoint. The actual use case from issue Output of relativePoint cannot be used with swipe command start and end parameters #2149 is relativePoint() via evalScript. Updated the PR description with the correct examples.

2. Automated test added — New test swipe with script variables defers coordinate parsing in YamlCommandReaderTest verifies that YAML swipe commands containing ${...} variables produce SwipeCommand with deferred startPointStr/endPointStr instead of being immediately parsed to Point objects.

3. Not marking as draft — The automated test is passing and manual testing is the only remaining item. I do not have a Maestro dev environment set up to run E2E flows, but the unit test covers the YAML parsing → command generation path which is where the bug lives.

[Fishbowler]

A Maestro dev environment is incredibly similar to a normal Maestro environment. Just run ./installLocally.sh and use Maestro as you typically would.

You've added a YAML parsing test that proves you can read the string from the YAML, but nothing to say that the point is then honoured - maybe a test elsewhere could go a little further?

[vazidmansuri005]

Good point — added an integration test (Case 139 - Swipe with script variables) in IntegrationTest.kt that verifies the full execution path:

1. evalScript sets output.startPoint and output.endPoint as string variables
2. Swipe command defers parsing (stores startPointStr/endPointStr)
3. evaluateScripts() resolves ${output.startPoint} → "100,500"
4. Orchestra parses the evaluated string and executes the swipe
5. FakeDriver asserts it received Event.Swipe(start = Point(100, 500), End = Point(100, 200), durationMs = 3000)

Thanks for the suggestion on installLocally.sh — will look into that for manual testing as well.

[vazidmansuri005]

Ran full E2E on iPhone 16 simulator (iOS 18.5) using locally built Maestro via installLocally.sh:

Test 1 — relativePoint() variables (exact issue #2149 scenario):

Launch app "com.apple.Preferences"... COMPLETED
Run ${output.startPoint = relativePoint(0.5, 0.8)}... COMPLETED
Run ${output.endPoint = relativePoint(0.5, 0.3)}... COMPLETED
Swipe from (${output.startPoint}) to (${output.endPoint}) in 500 ms... COMPLETED

Test 2 — absolute coordinate variables:

Launch app "com.apple.Preferences"... COMPLETED
Run ${output.startPoint = '200,600'}... COMPLETED
Run ${output.endPoint = '200,200'}... COMPLETED
Swipe from (${output.startPoint}) to (${output.endPoint}) in 500 ms... COMPLETED

Both pass. Marking manual testing as complete in the PR description.

[vazidmansuri005]

Hey @Fishbowler @amanjeetsingh150, could you please review this PR when you get a chance? Thanks!