on main help sign in
fix(db): prevent SQLITE_NOTADB corruption under concurrent sub-agents (#2690)

Crush could fail to start with "file is not a database (26)" after a session ran many sub-agents in parallel. The shared *sql.DB was opened with Go's default unlimited connection pool, so each concurrent sub-agent (session/message/cost writes via coordinator.runSubAgent) could open its own SQLite handle. Under load, WAL frames from different handles interleaved and auto-checkpoints could race; if the process was cancelled or killed mid-checkpoint, the main DB header and WAL desynced, producing SQLITE_NOTADB (26) on the next open. Two changes address this: - Set db.SetMaxOpenConns(1). SQLite serializes writes at the file level anyway, so extra pool connections add no throughput, only race surface. Forcing a single pooled connection makes database/sql queue all callers through one SQLite handle, eliminating cross-handle WAL/checkpoint races. Sub-agents still function; their writes just serialize (which SQLite was going to do regardless). - Add _txlock=immediate to both driver DSNs (modernc.org/sqlite and ncruces/go-sqlite3). BEGIN IMMEDIATE acquires the reserved-writer lock up front so concurrent writers queue cleanly on busy_timeout instead of racing the deferred-to-writer upgrade (the pattern behind prior SQLITE_BUSY reports, e.g. #2129, #2576). Defense in depth alongside the pool limit. For the ncruces driver the DSN must use the file: URI prefix for query params to be parsed; the plain path form silently ignores _txlock.

commit401084133df3e7b1895de169c4b67aea0f72b572 authorGreg Slepak <contact@taoeffect.com> dateMon 4 May 2026, 07:22 (6 months ago) parent481202e
3 files changed+15−1
internal/db/connect.go+7 internal/db/connect_modernc.go+3 internal/db/connect_ncruces.go+5−1
internal/db/connect.go+7
@@ -50,6 +50,13 @@ func Connect(ctx context.Context, dataDir string) (*sql.DB, error) {
5050 return nil, err
5151 }
5252
53+ // Serialize all access through a single connection. SQLite serializes
54+ // writes at the file level anyway, and allowing multiple pool
55+ // connections to interleave writes/checkpoints (especially under
56+ // concurrent sub-agents) has caused WAL/header desync resulting in
57+ // SQLITE_NOTADB (26) on the next open.
58+ db.SetMaxOpenConns(1)
59+
5360 if err = db.PingContext(ctx); err != nil {
5461 db.Close()
5562 return nil, fmt.Errorf("failed to connect to database: %w", err)
internal/db/connect_modernc.go+3
@@ -17,6 +17,9 @@ func openDB(dbPath string) (*sql.DB, error) {
1717 for name, value := range pragmas {
1818 params.Add("_pragma", fmt.Sprintf("%s(%s)", name, value))
1919 }
20+ // Use BEGIN IMMEDIATE so writers acquire the reserved lock up front,
21+ // preventing deferred-to-writer upgrade deadlocks.
22+ params.Set("_txlock", "immediate")
2023
2124 dsn := fmt.Sprintf("file:%s?%s", dbPath, params.Encode())
2225 db, err := sql.Open("sqlite", dsn)
internal/db/connect_ncruces.go+5−1
@@ -11,7 +11,11 @@ import (
1111)
1212
1313func openDB(dbPath string) (*sql.DB, error) {
14− db, err := driver.Open(dbPath, func(c *sqlite3.Conn) error {
14+ // Use BEGIN IMMEDIATE so writers acquire the reserved lock up front,
15+ // preventing deferred-to-writer upgrade deadlocks. The "file:" prefix
16+ // is required for the ncruces driver to parse query parameters.
17+ dsn := fmt.Sprintf("file:%s?_txlock=immediate", dbPath)
18+ db, err := driver.Open(dsn, func(c *sqlite3.Conn) error {
1519 // Set pragmas for better performance via _pragma query params.
1620 // Format: PRAGMA name = value;
1721 for name, value := range pragmas {