CI browser host: IndexedDB intermittently fails to open (UnknownError) — indexeddb kits and web-echo e2e #126

Closed
opened 2026-08-26 12:06:35 +02:00 by thecrealm · 6 comments
Owner

Two distinct runs failed in headless-Chromium IndexedDB opens on the runner: run 351 (17a4399, pre-#107) — katrix-store-indexeddb browser kits; run 360 (42e6814) — samples/web-echo BrowserEngineEndToEndTest, both cases, with StoreMigrationFailed: 'UnknownError: Internal error opening database with version 1' (§10.7.3 wrapping the IDB open failure). The error is Chromium's IDB backend failing in the DinD runner, not an assertion; the same suites pass elsewhere. Suspects: /tmp or profile-dir pressure in the container, Chrome for Testing version drift (152), leftover profile locks between karma runs. Investigate runner-side; consider a fresh chrome profile dir per run and a retry-once for the IDB open in the kits' harness (never in the store itself).

Two distinct runs failed in headless-Chromium IndexedDB opens on the runner: run 351 (17a4399, pre-#107) — katrix-store-indexeddb browser kits; run 360 (42e6814) — samples/web-echo BrowserEngineEndToEndTest, both cases, with StoreMigrationFailed: 'UnknownError: Internal error opening database with version 1' (§10.7.3 wrapping the IDB open failure). The error is Chromium's IDB backend failing in the DinD runner, not an assertion; the same suites pass elsewhere. Suspects: /tmp or profile-dir pressure in the container, Chrome for Testing version drift (152), leftover profile locks between karma runs. Investigate runner-side; consider a fresh chrome profile dir per run and a retry-once for the IDB open in the kits' harness (never in the store itself).
Author
Owner

Evidence correction: run 351's two indexeddb-kit failures are 60 s mocha timeouts (browser-side hang), not the UnknownError string — that exact error is (so far) only run 360's web-echo failure. Same browser-host instability family, two symptoms: IDB open error and IDB-test hang. Treat both as the runner-side investigation's scope.

Evidence correction: run 351's two indexeddb-kit failures are 60 s mocha timeouts (browser-side hang), not the UnknownError string — that exact error is (so far) only run 360's web-echo failure. Same browser-host instability family, two symptoms: IDB open error and IDB-test hang. Treat both as the runner-side investigation's scope.
Author
Owner

Mitigation pushed (--disable-dev-shm-usage + --disable-gpu via karma.config.d in katrix-store-indexeddb and samples/web-echo): Chromium's IDB LevelDB lives on /dev/shm, tiny under DinD — explains both the UnknownError opens (run 360) and the 60s hangs (run 351). Leaving this open until several consecutive runs stay green. Side note: run 1450 (4d439c9) was cancelled without a superseding push — runner behaviour worth a glance if it recurs.

Mitigation pushed (--disable-dev-shm-usage + --disable-gpu via karma.config.d in katrix-store-indexeddb and samples/web-echo): Chromium's IDB LevelDB lives on /dev/shm, tiny under DinD — explains both the UnknownError opens (run 360) and the 60s hangs (run 351). Leaving this open until several consecutive runs stay green. Side note: run 1450 (4d439c9) was cancelled without a superseding push — runner behaviour worth a glance if it recurs.
Author
Owner

First full green run with the mitigation: run 378 (task 1458, efc4669) — :katrix-store-indexeddb:allTests and :samples:web-echo:allTests both passed under ChromeHeadlessDind (--disable-dev-shm-usage). One run is not proof against an intermittent; leave open until ~3 consecutive greens, then close.

First full green run with the mitigation: run 378 (task 1458, efc4669) — :katrix-store-indexeddb:allTests and :samples:web-echo:allTests both passed under ChromeHeadlessDind (--disable-dev-shm-usage). One run is not proof against an intermittent; leave open until ~3 consecutive greens, then close.
Author
Owner

Second run with the mitigation: build run 380 (0409476, #130 landed) — will report its :katrix-store-indexeddb:allTests / :samples:web-echo:allTests outcome here; ~3 consecutive greens closes this.

Second run with the mitigation: build run 380 (0409476, #130 landed) — will report its :katrix-store-indexeddb:allTests / :samples:web-echo:allTests outcome here; ~3 consecutive greens closes this.
Author
Owner

Run 380 (0409476): phase 1 BUILD SUCCESSFUL in 14m26s — :katrix-store-indexeddb:{js,wasmJs}BrowserTest and :samples:web-echo:{js,wasmJs}BrowserTest all ran green under ChromeHeadlessDind. That is green #2 in a row for the browser host (378, 380); one more closes this.

Run 380 (0409476): phase 1 BUILD SUCCESSFUL in 14m26s — :katrix-store-indexeddb:{js,wasmJs}BrowserTest and :samples:web-echo:{js,wasmJs}BrowserTest all ran green under ChromeHeadlessDind. That is green #2 in a row for the browser host (378, 380); one more closes this.
thecrealm added
now
and removed
next
labels 2026-08-31 11:50:07 +02:00
Author
Owner

Third consecutive green browser-host run: build task #382 (f624a99) succeeded — phase 1 runs :katrix-store-indexeddb and :samples:web-echo browser suites unconditionally, so ChromeHeadlessDind with --disable-dev-shm-usage (84d8330) is green in 378, 380, 382. Closing per the ~3-greens criterion; reopen if the UnknownError/60s-hang family recurs.

Third consecutive green browser-host run: build task #382 (f624a99) succeeded — phase 1 runs :katrix-store-indexeddb and :samples:web-echo browser suites unconditionally, so ChromeHeadlessDind with --disable-dev-shm-usage (84d8330) is green in 378, 380, 382. Closing per the ~3-greens criterion; reopen if the UnknownError/60s-hang family recurs.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
thecrealm/katrix#126
No description provided.