Solana: 'Malformed Solana transaction (code 3)' when attempting to purchase NFT on OpenSea

[illithics]

Summary

When connected to OpenSea via KeepKey Vault and attempting to purchase a Solana NFT, the transaction fails with:

Transaction failed: Vault sign failed (500): {"error":"Malformed Solana transaction","code":3}

Steps to Reproduce

1. Connect KeepKey Vault to OpenSea via WalletConnect
2. Find a Solana NFT and initiate a purchase
3. Approve the transaction prompt
4. Error: Vault sign failed (500): {"error":"Malformed Solana transaction","code":3}

Root Cause

The /solana/sign-transaction handler in Vault strips the signature prefix from the raw transaction before sending to the device using a hand-rolled compact-array parser:

let pos = 0, sigCount = 0;
if (fullTx[0] < 128) {
  sigCount = fullTx[0];
  pos = 1;
} else if (fullTx.length >= 2 && fullTx[1] < 128) {
  sigCount = fullTx[0] & 127 | fullTx[1] << 7;
  pos = 2;
} else if (fullTx.length >= 3) {
  sigCount = fullTx[0] & 127 | (fullTx[1] & 127) << 7 | fullTx[2] << 14;
  pos = 3;
}
const messageStart = pos + sigCount * 64;
deviceRawTx = Buffer.from(fullTx.subarray(messageStart)).toString("base64");

OpenSea sends versioned transactions (Solana v0 with address lookup tables), which have a 0x80 version prefix byte before the compact-array signature count. This causes the parser to misidentify the transaction boundaries, producing a malformed message payload that the firmware rejects with code: 3.

Expected Behavior

Vault should correctly parse both legacy and versioned (v0) Solana transactions before passing them to the device.

The fix should detect the version prefix byte (0x80 for v0 transactions) and adjust parsing accordingly:

let versionedOffset = 0;
if (fullTx[0] === 0x80) {
  versionedOffset = 1; // skip version prefix
}
// then parse compact-array from fullTx[versionedOffset]

Environment

• Vault version: 1.2.16
• Firmware: 7.14.0
• Experimental policy: enabled
• OS: macOS
• dApp: OpenSea (Solana NFT purchase)

Related

• WalletConnect: Cannot connect to Solana dApps (e.g. jup.ag) — missing Solana namespace in session proposal handler #121 — WalletConnect Solana namespace not supported (jup.ag)

[illithics]

Reproduction Script

Confirmed reproducible without OpenSea — can be triggered directly against the Vault API.

TOKEN="your-vault-api-key"  # obtain via POST /auth/pair

# Test 1: Legacy transaction — works correctly (code 9 = key not signer, device received it fine)
LEGACY_TX=$(python3 -c "
import base64
sig_count = bytes([1])
sigs = bytes(64)
header = bytes([1, 0, 0])
num_accounts = bytes([1])
account = bytes(32)
blockhash = bytes(32)
num_ix = bytes([0])
msg = header + num_accounts + account + blockhash + num_ix
tx = sig_count + sigs + msg
print(base64.b64encode(tx).decode())
")

curl -s -X POST http://localhost:1646/solana/sign-transaction \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"raw_tx\":\"$LEGACY_TX\",\"address_n\":[2147483692,2147484149,2147483648,2147483648]}"
# → {"error":"Derived key is not a signer for this tx","code":9}  ✅ parsed correctly

# Test 2: Versioned v0 transaction — reproduces the bug
VERSIONED_TX=$(python3 -c "
import base64
version_prefix = bytes([0x80])  # v0 versioned tx prefix
sig_count = bytes([1])
sigs = bytes(64)
header = bytes([1, 0, 0])
num_accounts = bytes([1])
account = bytes(32)
blockhash = bytes(32)
num_ix = bytes([0])
alt_count = bytes([0])          # address lookup table count (v0 only)
msg = header + num_accounts + account + blockhash + num_ix + alt_count
tx = version_prefix + sig_count + sigs + msg
print(base64.b64encode(tx).decode())
")

curl -s -X POST http://localhost:1646/solana/sign-transaction \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"raw_tx\":\"$VERSIONED_TX\",\"address_n\":[2147483692,2147484149,2147483648,2147483648]}"
# → {"error":"Malformed Solana transaction","code":3}  ❌ bug reproduced

Result

Transaction type	Response	Notes
Legacy	code: 9 (key not signer)	Parsed correctly by Vault
Versioned v0 (0x80 prefix)	code: 3 (malformed)	Parser fails — bug confirmed

The 0x80 version prefix byte causes the compact-array sig count parser to misread the transaction boundaries, sending a garbage payload to the firmware.

[MaryanBog]

This is caused by incorrect parsing of versioned (v0) transactions.

OpenSea sends v0 transactions with a 0x80 prefix, but your parser treats the first byte as the signature count. This shifts offsets and produces an invalid message, resulting in code: 3.

Fix: detect and skip the version byte before parsing:

let offset = 0;
if (fullTx[0] === 0x80) {
offset = 1;
}

// parse signature count starting from offset

After this adjustment, the transaction should be parsed correctly.

If needed, I can help implement a proper parser for both legacy and v0 transactions.