The build on main went red once on the test “the app of a card on a host is forwarded again after its connection dropped”. The cause was in Obeya itself, not just in the test: right after a dropped connection, opening the app led to a dead address for a short while.
For a card whose workspace is on another machine, Obeya reaches the app there through an ssh forward to a port on this machine.
When the connection drops, the forward's port closes at once, but Obeya only learned of it once it heard that the ssh process had ended, which comes a little later.
On a busy machine that gap is long enough to fall into.
ssh ended.| Green build before it | The red build | |
|---|---|---|
| Test duration | 539 ms | 170 ms |
| Obeya tries to rebuild the forward while the host is down (log line “ssh: connect to host … refused”) | yes | no: it never noticed the forward had gone |
Making Obeya hear of the ssh exit 300 ms late reproduced the failure exactly, every time. With the fix, the same delay no longer matters.
| Run | Before the fix | After |
|---|---|---|
New test step: the forward's port closes while its ssh still seems to run | fails | passes |
The test with the ssh exit heard 300 ms late | fails | passes |
| Remote-app tests in Linux (as on CI), 2 CPUs plus 3 busy processes, 40 runs | — | 40 / 40 |
Whole test suite, bun test --parallel | — | green on an idle Mac; under 6 extra busy processes, the remote-app tests passed in 3 of 3 runs |
Under that extra load, an unrelated test of the headless browser skill timed out in 2 of the 3 runs (its browser took longer than 10 s to close). It is proposed as a card of its own.