Drop schema migrations
The index is a cache. Everything in it is re-derived from the statements and rules.toml, and has been since the last hand-set tags went, so keeping machinery to nurse an old index through a schema change was paying for a guarantee nothing needs. The ALTER TABLE lists, the dropped-column list and the pragma_table_info reader are gone; Open applies the schema and returns. What replaces it is a line in the release note: delete index.db and import again. The two directions are not symmetric, which is worth knowing before assuming something is broken. Removing a column leaves an older index working, carrying the dead column and its data unread, because every statement here names its columns -- that is why the drop half was never really load-bearing. Adding a column the code reads breaks every command against an older index with `no such column` until the file is deleted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -34,6 +34,23 @@ reproducible and retag stops being safe. Covered by `TestRetagRewritesEveryTag`.
|
|||||||
`index.db` at the root of the data directory and re-importing must reproduce
|
`index.db` at the root of the data directory and re-importing must reproduce
|
||||||
everything, with nothing lost.
|
everything, with nothing lost.
|
||||||
|
|
||||||
|
**The index is a cache, so there are no migrations.** `store.Open` applies
|
||||||
|
`schema` and nothing else — no `ALTER TABLE`, no version column, no repair of
|
||||||
|
an index an older build wrote. Changing the schema costs one line in the
|
||||||
|
release note: delete `index.db` and import again. The two directions are not
|
||||||
|
symmetric, which is worth knowing before you assume something is broken:
|
||||||
|
|
||||||
|
- *Removing* a column leaves an older index still working, carrying the dead
|
||||||
|
column and its data unread.
|
||||||
|
- *Adding* one the code reads breaks every command against an older index with
|
||||||
|
`query transactions: SQL logic error: no such column: …` until it is deleted.
|
||||||
|
|
||||||
|
That asymmetry holds only because every statement names its columns —
|
||||||
|
**no `SELECT *`, and nothing may depend on column order.** Keep it that way.
|
||||||
|
Do not reintroduce migration machinery either; if re-parsing ever becomes too
|
||||||
|
expensive to ask for, that is a decision to revisit deliberately rather than a
|
||||||
|
helper to slip back in.
|
||||||
|
|
||||||
**Dedupe is by fingerprint**: `sha256(date | amount | normalised description |
|
**Dedupe is by fingerprint**: `sha256(date | amount | normalised description |
|
||||||
ordinal)`, where the ordinal distinguishes identical lines *within one
|
ordinal)`, where the ordinal distinguishes identical lines *within one
|
||||||
statement*. Two identical purchases on one day both survive; the same line in an
|
statement*. Two identical purchases on one day both survive; the same line in an
|
||||||
@@ -123,6 +140,11 @@ money --root /tmp/.../demo import # must report 0 new
|
|||||||
money --root /tmp/.../demo ls --wide
|
money --root /tmp/.../demo ls --wide
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If you changed the schema, delete that root's `index.db` first. Nothing
|
||||||
|
migrates it, so a demo root left over from an earlier build either carries dead
|
||||||
|
columns or fails with `no such column`, depending on which way the schema
|
||||||
|
moved.
|
||||||
|
|
||||||
### Driving the TUI in tests
|
### Driving the TUI in tests
|
||||||
|
|
||||||
Prefer feeding `tea.Msg` values to `Model.Update` directly — that is how every
|
Prefer feeding `tea.Msg` values to `Model.Update` directly — that is how every
|
||||||
|
|||||||
@@ -23,6 +23,8 @@ tagged by glob rules you write.
|
|||||||
|
|
||||||
\* Nothing lives only in the index: it is derived entirely from the statements
|
\* Nothing lives only in the index: it is derived entirely from the statements
|
||||||
plus rules.toml, so deleting it and re-importing gets you exactly what you had.
|
plus rules.toml, so deleting it and re-importing gets you exactly what you had.
|
||||||
|
It is a cache and nothing more, which is why it is never migrated — if a new
|
||||||
|
build changes its shape, delete it and run `money import` again.
|
||||||
|
|
||||||
## Where the data root lives
|
## Where the data root lives
|
||||||
|
|
||||||
|
|||||||
+5
-71
@@ -58,74 +58,12 @@ CREATE INDEX IF NOT EXISTS idx_txn_date ON transactions(date);
|
|||||||
CREATE INDEX IF NOT EXISTS idx_txn_account ON transactions(account_id);
|
CREATE INDEX IF NOT EXISTS idx_txn_account ON transactions(account_id);
|
||||||
`
|
`
|
||||||
|
|
||||||
// migrations bring an index created by an older build up to date. SQLite
|
|
||||||
// errors on a duplicate column, which is how we detect "already applied".
|
|
||||||
var migrations = []string{
|
|
||||||
`ALTER TABLE transactions ADD COLUMN type TEXT NOT NULL DEFAULT ''`,
|
|
||||||
`ALTER TABLE transactions ADD COLUMN balance_minor INTEGER`,
|
|
||||||
}
|
|
||||||
|
|
||||||
// dropped are columns an older build created that this one no longer reads.
|
|
||||||
// Nothing indexes or constrains them, so they can simply go; leaving them
|
|
||||||
// would keep a NOT NULL column alive that no INSERT here ever names.
|
|
||||||
var dropped = []string{"counterparty", "rule_transfer", "manual_transfer", "manual_tag"}
|
|
||||||
|
|
||||||
// migrate brings an index created by an older build up to date, adding the
|
|
||||||
// columns it lacks and removing the ones it should no longer have.
|
|
||||||
func migrate(db *sql.DB) error {
|
|
||||||
have, err := columns(db, "transactions")
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
for _, stmt := range migrations {
|
|
||||||
name := addedColumn(stmt)
|
|
||||||
if have[name] {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if _, err := db.Exec(stmt); err != nil {
|
|
||||||
return fmt.Errorf("migrate (%s): %w", stmt, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for _, name := range dropped {
|
|
||||||
if !have[name] {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
stmt := fmt.Sprintf(`ALTER TABLE transactions DROP COLUMN %s`, name)
|
|
||||||
if _, err := db.Exec(stmt); err != nil {
|
|
||||||
return fmt.Errorf("migrate (%s): %w", stmt, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func columns(db *sql.DB, table string) (map[string]bool, error) {
|
|
||||||
rows, err := db.Query(`SELECT name FROM pragma_table_info(?)`, table)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("inspect %s: %w", table, err)
|
|
||||||
}
|
|
||||||
defer rows.Close()
|
|
||||||
out := map[string]bool{}
|
|
||||||
for rows.Next() {
|
|
||||||
var name string
|
|
||||||
if err := rows.Scan(&name); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
out[name] = true
|
|
||||||
}
|
|
||||||
return out, rows.Err()
|
|
||||||
}
|
|
||||||
|
|
||||||
// addedColumn pulls the column name out of an ADD COLUMN statement.
|
|
||||||
func addedColumn(stmt string) string {
|
|
||||||
_, rest, ok := strings.Cut(stmt, "ADD COLUMN ")
|
|
||||||
if !ok {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
name, _, _ := strings.Cut(rest, " ")
|
|
||||||
return name
|
|
||||||
}
|
|
||||||
|
|
||||||
// Open opens (creating if needed) the index at path.
|
// Open opens (creating if needed) the index at path.
|
||||||
|
//
|
||||||
|
// There is no schema migration: the index caches what the statements and
|
||||||
|
// rules.toml already say, so a build that changes the schema is answered by
|
||||||
|
// deleting index.db and importing again, not by patching the old one in place.
|
||||||
|
// Until it is deleted, queries against it fail on the columns it lacks.
|
||||||
func Open(path string) (*DB, error) {
|
func Open(path string) (*DB, error) {
|
||||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||||
return nil, fmt.Errorf("create index dir: %w", err)
|
return nil, fmt.Errorf("create index dir: %w", err)
|
||||||
@@ -138,10 +76,6 @@ func Open(path string) (*DB, error) {
|
|||||||
sqlDB.Close()
|
sqlDB.Close()
|
||||||
return nil, fmt.Errorf("apply schema: %w", err)
|
return nil, fmt.Errorf("apply schema: %w", err)
|
||||||
}
|
}
|
||||||
if err := migrate(sqlDB); err != nil {
|
|
||||||
sqlDB.Close()
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
return &DB{sql: sqlDB}, nil
|
return &DB{sql: sqlDB}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user