ConcurrentWriterTest surfaces SQLITE_BUSY on a loaded CI runner (run 198) #74
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Run 198 (
8d93a62, phase 1): concurrent_writers_never_surface_sqlite_busy failed with raw org.sqlite.SQLiteException SQLITE_BUSY while cargoCheckAndroidX64Debug ran concurrently — the D33 contention family, but in phase 1 where store tests share the runner with cargo. Third BUSY sighting after runs #158/#166 (pre-WAL, D32); first since WAL + imperative IMMEDIATE + busy_timeout=30000 (D25). Local: 10/10 green in isolation, so the shape needs a saturated runner. Open diagnostic questions: gradle console carries only the exception header — the test-results XML with the stack isn't archived, so we can't tell genuine 30s starvation (unfair busy handler; would argue a longer timeout per D25's stall-beats-crash) from an immediate-BUSY hole (would need a driver-seam fix). Consider: archive test-result XMLs as workflow artifacts on failure, and/or move :katrix-store-sqlite jvmTest behind the compile/cargo barrier like the container e2es (D33).Taking the diagnostic half first: the build workflow gains an
if: failure()upload-artifact step archiving**/build/test-results/**/*.xml(both phases — the job-status predicate covers whichever gradle step failed). Next BUSY sighting carries its stack, which settles the open question: a 30s-starvation trace argues a longer busy_timeout per D25's stall-beats-crash; an immediate-BUSY frame argues a driver-seam fix. Deliberately NOT moving :katrix-store-sqlite:jvmTest behind the D33 barrier yet — quieting the runner before we can read the stack would mask exactly the immediate-BUSY hole we need to rule out, and production won't run on a quiet runner. Barrier move (plus its D-entry) stays on the table once a stack is in hand. Push is held until run #203 (the #73 deflake validation) finishes, so the follow-up push can't auto-cancel it.Local repro attempt (2026-08-11): 40 reruns of ConcurrentWriterTest on the dev Mac under full saturation — one busy-spin per core plus two continuous dd writers — all green. Doesn't discriminate the two hypotheses (macOS/APFS on NVMe is a different scheduler and disk from the CI DinD volume, and there was no concurrent cargo link eating the same IO), but it does confirm there's no easily-reachable immediate-BUSY hole in the driver posture under plain oversubscription. Diagnostics are armed on main: run 205 validated the failure-path XML archiving, so the next CI sighting carries the stack that settles starvation-vs-hole. Parking as blocked until then; the D33 barrier move stays deliberately unplayed so the repro conditions survive.
Recurrence: build run #257 (
7288cd9, 2026-08-21 09:43 UTC, phase 1 — the store tests again shared the runner with the cargo/android tasks): concurrent_writers_never_surface_sqlite_busy FAILED with raw SQLITE_BUSY. Still header-only in the console — and the diagnostic half never landed: the archive step's upload-artifact@v4 is rejected by this Forgejo (GHESNotSupportedError in runs #253 and #257), so no XML has ever been captured. Follow-up in the next CI change: print the blocks of failing test XMLs straight into the job log on failure.First stack, from build run #259 (
bd0babf, 2026-08-21 10:02 UTC) via the new log-print step: SQLITE_BUSY is thrown from org.sqlite.SQLiteConnection.setAutoCommit(false) ← app.cash.sqldelight.driver.jdbc.JdbcDriver.beginTransaction ← SqliteSendQueueStore.enqueue (ConcurrentWriterTest.kt:60), with a suppressed identical BUSY under SqliteFactStores.write. The throw site is the JDBC driver's autocommit toggle (plain BEGIN), not the imperative IMMEDIATE path — i.e. the immediate-BUSY-at-the-driver-seam hypothesis, not a 30s starvation. Unblocks the decision between the two fixes discussed above.