Flakey Test List

[dozyio]

List of flaky CI tests to review

  1) WebRTC transport interface compliance
       interface-transport
         should handle many small writes:
     StreamResetError: The stream has been reset
      at WebRTCStream.onRemoteReset (file:///Users/runner/work/js-libp2p/js-libp2p/packages/utils/src/abstract-message-stream.ts:351:17)
      at WebRTCStream.processIncomingProtobuf (file:///Users/runner/work/js-libp2p/js-libp2p/packages/transport-webrtc/src/stream.ts:264:14)
      at file:///Users/runner/work/js-libp2p/js-libp2p/packages/transport-webrtc/src/stream.ts:95:14

  1) webrtc private-to-private
       should dial WebRTC with existing relayed connection:
     TimeoutError: The operation was aborted due to timeout
      at new DOMException (node:internal/per_context/domexception:76:18)
      at Timeout._onTimeout (node:internal/abort_controller:154:9)
      at listOnTimeout (node:internal/timers:605:17)
      at processTimers (node:internal/timers:541:7)

[tabcat]

in firefox and firefox-worker

  1) WebRTC transport interface compliance
       interface-transport
         should handle one big write:
     the event {"isTrusted":false,"error":{"name":"StreamResetError"},"local":false} was thrown, throw an Error :)

[dozyio]

+1

[dozyio]

test-interop

ed25519/tls/yamux echo
       "before all" hook for "go sender to js listener":
     Error: Timeout of 20000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/runner/work/js-libp2p/js-libp2p/packages/integration-tests/dist/test/interop.js)
      at listOnTimeout (node:internal/timers:605:17)
      at processTimers (node:internal/timers:541:7)

[dozyio]

test-node (macos-latest, lts/*)

WebRTC transport interface compliance
       interface-transport
         should close all streams when the connection closes:
     TimeoutError: The operation was aborted due to timeout
      at new DOMException (node:internal/per_context/domexception:76:18)
      at Timeout._onTimeout (node:internal/abort_controller:154:9)
      at listOnTimeout (node:internal/timers:605:17)
      at processTimers (node:internal/timers:541:7)

[dozyio]

interop

rsa/noise/mplex echo
       "before all" hook for "go sender to go listener":
     Error: Timeout of 20000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/runner/work/js-libp2p/js-libp2p/packages/integration-tests/dist/test/interop.js)
      at listOnTimeout (node:internal/timers:605:17)
      at processTimers (node:internal/timers:541:7)

[dozyio]

+1

[dozyio]

 1) rsa/noise/mplex echo
       "before all" hook for "js sender to go listener":
     Error: Timeout of 20000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/runner/work/js-libp2p/js-libp2p/packages/integration-tests/dist/test/interop.js)
      at listOnTimeout (node:internal/timers:605:17)
      at processTimers (node:internal/timers:541:7)

[dozyio]

interop

rsa/noise/yamux echo
       "before all" hook for "js sender to go listener":
     Error: Timeout of 20000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/runner/work/js-libp2p/js-libp2p/packages/integration-tests/dist/test/interop.js)
      at listOnTimeout (node:internal/timers:605:17)
      at processTimers (node:internal/timers:541:7)

[dozyio]

interop

rsa/plaintext/yamux echo
       "before all" hook for "js sender to go listener":
     Error: Timeout of 20000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/runner/work/js-libp2p/js-libp2p/packages/integration-tests/dist/test/interop.js)
      at listOnTimeout (node:internal/timers:605:17)
      at processTimers (node:internal/timers:541:7)

[dozyio]

WebRTC-Direct interface-transport compliance
transport events
15:29:05 WARN UDP socket binding failed on port 37783, errno=98
15:29:05 ERROR UDP socket opening failed
15:29:05 FATAL UDP socket creation failed
15:29:05 FATAL Registry initialization failed

[dozyio]

errno=98 is port already in use

[tabcat]

wonder whats happening there.

[dozyio]

firefox

  1) WebRTC transport interface compliance
       interface-transport
         should handle one big write:
     the event {"isTrusted":false,"error":{"name":"StreamResetError"},"local":false} was thrown, throw an Error :)
  thrown2Error@http://127.0.0.1:34629/bundle-out.js:15586:16
  done2@http://127.0.0.1:34629/bundle-out.js:14189:13
  promise callback*callFn@http://127.0.0.1:34629/bundle-out.js:14235:20
  next@http://127.0.0.1:34629/bundle-out.js:15205:20
  next@http://127.0.0.1:34629/bundle-out.js:15121:20
  timeslice@http://127.0.0.1:34629/bundle-out.js:18300:33
  next@http://127.0.0.1:34629/bundle-out.js:15207:17
  next@http://127.0.0.1:34629/bundle-out.js:15121:20
  cbHookRun@http://127.0.0.1:34629/bundle-out.js:15177:17
  done2@http://127.0.0.1:34629/bundle-out.js:14189:13
  promise callback*callFn@http://127.0.0.1:34629/bundle-out.js:14235:20
  next@http://127.0.0.1:34629/bundle-out.js:15139:17
  timeslice@http://127.0.0.1:34629/bundle-out.js:18300:33
  next@http://127.0.0.1:34629/bundle-out.js:15207:17
  done2@http://127.0.0.1:34629/bundle-out.js:14189:13
  promise callback*callFn@http://127.0.0.1:34629/bundle-out.js:14235:20
  next@http://127.0.0.1:34629/bundle-out.js:15205:20
  next@http://127.0.0.1:34629/bundle-out.js:15121:20
  timeslice@http://127.0.0.1:34629/bundle-out.js:18300:33

[tabcat]

this is common one

[dozyio]

ran 150 tests locally with new playwright for this test - passed every time... old playwright failed ~1 in 20

[dozyio]

electron

  instantiate the transport
    ✔ create

  listen
    ip4
      1) listen, check for promise


  1 passing (37ms)
  1 failing

  1) listen
       ip4
         listen, check for promise:
     Error: listen EADDRINUSE: address already in use 127.0.0.1:47382
      at Server.setupListenHandle [as _listen2] (node:net:1940:16)
      at listenInCluster (node:net:1997:12)
      at node:net:2206:7
      at process.processTicksAndRejections (node:internal/process/task_queues:90:21)

[dozyio]

 1) WebRTC transport interface compliance
      interface-transport
        should handle many small writes:
    Timeout of 360000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
 createTimeoutError$1@http://127.0.0.1:39525/bundle-out.js:13998:21
 callFn@http://127.0.0.1:39525/bundle-out.js:14234:19
 next@http://127.0.0.1:39525/bundle-out.js:15205:20
 next@http://127.0.0.1:39525/bundle-out.js:15121:20
 timeslice@http://127.0.0.1:39525/bundle-out.js:18300:33
 next@http://127.0.0.1:39525/bundle-out.js:15207:17
 next@http://127.0.0.1:39525/bundle-out.js:15121:20
 cbHookRun@http://127.0.0.1:39525/bundle-out.js:15177:17
 done2@http://127.0.0.1:39525/bundle-out.js:14189:13
 promise callback*callFn@http://127.0.0.1:39525/bundle-out.js:14235:20
 next@http://127.0.0.1:39525/bundle-out.js:15139:17
 timeslice@http://127.0.0.1:39525/bundle-out.js:18300:33
 next@http://127.0.0.1:39525/bundle-out.js:15207:17
 done2@http://127.0.0.1:39525/bundle-out.js:14189:13
 promise callback*callFn@http://127.0.0.1:39525/bundle-out.js:14235:20
 next@http://127.0.0.1:39525/bundle-out.js:15205:20
 next@http://127.0.0.1:39525/bundle-out.js:15121:20
 timeslice@http://127.0.0.1:39525/bundle-out.js:18300:33
 next@http://127.0.0.1:39525/bundle-out.js:15207:17
 next@http://127.0.0.1:39525/bundle-out.js:15121:20
 cbHookRun@http://127.0.0.1:39525/bundle-out.js:15177:17
 done2@http://127.0.0.1:39525/bundle-out.js:14189:13
 promise callback*callFn@http://127.0.0.1:39525/bundle-out.js:14235:20
 next@http://127.0.0.1:39525/bundle-out.js:15139:17
 timeslice@http://127.0.0.1:39525/bundle-out.js:18300:33
 next@http://127.0.0.1:39525/bundle-out.js:15207:17
 done2@http://127.0.0.1:39525/bundle-out.js:14189:13
 promise callback*callFn@http://127.0.0.1:39525/bundle-out.js:14235:20
 next@http://127.0.0.1:39525/bundle-out.js:15205:20
 next@http://127.0.0.1:39525/bundle-out.js:15121:20
 timeslice@http://127.0.0.1:39525/bundle-out.js:18300:33

[dozyio]

This is mainly on firefox - did some debugging, think its a browser issue. Seems to happen after ~1900 writes

[dozyio]

More debugging - think this is firefox webrtc issue - seems to be fixed in firefox 149. Waiting on playright to release new version with 149 support

[dozyio]

  1) keychain
       rotate keychain passphrase
         can rotate keychain passphrase:
     Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/runner/work/js-libp2p/js-libp2p/packages/keychain/dist/test/keychain.spec.js)
      at listOnTimeout (node:internal/timers:605:17)
      at processTimers (node:internal/timers:541:7)

[dozyio]

#3436

[dozyio]

1) PersistentPeerStore
     should return peerInfo:

    AssertionError: expected { …(2) } to deeply equal { …(2) }
    + expected - actual

     {
       "id": "12D3KooWPiGEUNY4ZqbJufvzYVWKLeqS2ZpYYtDWTgpniBNcM1VY"
    -  "multiaddrs": []
    +  "multiaddrs": [
    +    {}
    +  ]
     }

[dozyio]

#3437

[dozyio]

Electron main

  1) Gossipsub acceptFrom
       should recompute score after 1s:
     TypeError: Cannot read properties of null (reading 'args')
      at Context.<anonymous> (file:///home/runner/work/js-libp2p/js-libp2p/packages/gossipsub/dist/test/accept-from.spec.js:60:35)

[tabcat]

#3440

[tabcat]

1. go-libp2p-pubsub gossipsub tests
   test mixed gossipsub:
   Error: Message never received, sender 21, receiver 0, index 6
   at Timeout._onTimeout (file:///home/runner/work/js-libp2p/js-libp2p/packages/gossipsub/test/e2e/go-gossipsub.spec.ts:45:18)
   at listOnTimeout (node:internal/timers:605:17)
   at processTimers (node:internal/timers:541:7)

In ubuntu-node

[tabcat]

1. go-libp2p-pubsub gossipsub tests
   test mixed gossipsub:
   Error: Message never received, sender 0, receiver 19, index 44
   at Timeout._onTimeout (file:///Users/runner/work/js-libp2p/js-libp2p/packages/gossipsub/test/e2e/go-gossipsub.spec.ts:45:18)
   at listOnTimeout (node:internal/timers:605:17)
   at processTimers (node:internal/timers:541:7)

in macos-node

[tabcat]

1. fetch
   different lookups for different prefixes:
   TimeoutError: signal timed out

in chrome

[tabcat]

1. WebRTC transport interface compliance
   interface-transport
   can close local stream for writing while a remote stream is reading:
   TimeoutError: signal timed out

in chrome

[tabcat]

dht.contentFetching
"before all" hook for "go peer to go peer":
OperationFailedError: failed to dial: failed to dial 12D3KooWRWckKcwZC84mHXHh21JoRg99LZRsXsGTqTg2LhqjM9zn: all dials failed

• [/ip4/127.0.0.1/tcp/40597] failed to negotiate security protocol: error reading handshake message: noise: message is too short
  at Client.connect (file:///home/runner/work/js-libp2p/js-libp2p/packages/libp2p-daemon-client/src/index.ts:106:13)
  at processTicksAndRejections (node:internal/process/task_queues:104:5)
  at Context. (file:///home/runner/work/js-libp2p/js-libp2p/packages/interop/src/dht/content-fetching.ts:56:11)

interop tests

[tabcat]

✔ can close local stream while a remote stream is writing (2078ms)
Error: Process completed with exit code 1.

likely test after this is causing uncaught exception somewhere.
node on mac os

[dozyio]

node

 1) WebRTC transport interface compliance
       interface-transport
         should handle one big write:
     UnexpectedEOFError: Underread: 9269382/10485760 bytes
      at file:///Users/runner/work/js-libp2p/js-libp2p/packages/protocol-echo/src/echo.ts:112:23
      at WebRTCStream.<anonymous> (file:///Users/runner/work/js-libp2p/js-libp2p/node_modules/main-event/src/index.ts:140:9)
      at WebRTCStream.[nodejs.internal.kHybridDispatch] (node:internal/event_target:843:20)
      at WebRTCStream.dispatchEvent (node:internal/event_target:776:26)
      at WebRTCStream.onTransportClosed (file:///Users/runner/work/js-libp2p/js-libp2p/packages/utils/src/abstract-message-stream.ts:396:14)
      at file:///Users/runner/work/js-libp2p/js-libp2p/packages/utils/src/abstract-stream-muxer.ts:130:16
      at Array.forEach (<anonymous>)
      at DataChannelMuxer.onTransportClosed (file:///Users/runner/work/js-libp2p/js-libp2p/packages/utils/src/abstract-stream-muxer.ts:129:25)
      at muxerOnMaConnClose (file:///Users/runner/work/js-libp2p/js-libp2p/packages/utils/src/abstract-stream-muxer.ts:76:12)
      at RTCPeerConnectionMultiaddrConnection.<anonymous> (file:///Users/runner/work/js-libp2p/js-libp2p/node_modules/main-event/src/index.ts:140:9)
      at RTCPeerConnectionMultiaddrConnection.[nodejs.internal.kHybridDispatch] (node:internal/event_target:843:20)
      at RTCPeerConnectionMultiaddrConnection.dispatchEvent (node:internal/event_target:776:26)
      at RTCPeerConnectionMultiaddrConnection.onTransportClosed (file:///Users/runner/work/js-libp2p/js-libp2p/packages/utils/src/abstract-message-stream.ts:396:14)
      at RTCPeerConnectionMultiaddrConnection.peerConnection.onconnectionstatechange (file:///Users/runner/work/js-libp2p/js-libp2p/packages/transport-webrtc/src/rtcpeerconnection-to-conn.ts:25:14)
      at RTCPeerConnection.<anonymous> (file:///Users/runner/work/js-libp2p/js-libp2p/node_modules/node-datachannel/src/polyfill/RTCPeerConnection.ts:194:12)
      at RTCPeerConnection.[nodejs.internal.kHybridDispatch] (node:internal/event_target:843:20)
      at RTCPeerConnection.dispatchEvent (node:internal/event_target:776:26)
      at file:///Users/runner/work/js-libp2p/js-libp2p/node_modules/node-datachannel/src/polyfill/RTCPeerConnection.ts:151:12

[tabcat]

this test is crashing the process now as well. might be a good one to look at.

[tabcat]

1. WebRTC transport interface compliance
   interface-transport
   should listen on multiple addresses:
   TimeoutError: signal timed out

chrome