Remove manual tagging

A tag could come from two places: rules.toml, or the TUI's t key, which wrote
manual_tag with COALESCE(manual_tag, rule_tag) deciding the winner. That split
paid for itself in the first invariant of the codebase, in ClearOverrides and
the c key, in the * marker on the tag column, and in the one exception to a
disposable index -- a tag set by hand was the only thing in index.db that the
statements could not reproduce.

Now rules.toml decides every tag. The index is derived entirely from the
statements plus that file, so deleting it and re-importing gets back exactly
what was there, and retag has nothing to be careful of. Tagging a one-off means
writing a narrow rule on screen 4, which previews what the glob catches before
it is saved.

A manual tag in an existing index is dropped along with the column the first
time this build opens it, and those rows read as whatever the rules say, or as
untagged. TestManualTagSurvivesRetag guarded the invariant that has just been
removed; TestRetagRewritesEveryTag replaces it with the one that took its
place, and keeps Retag itself covered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-11 23:19:24 +02:00
co-authored by Claude Opus 5
parent 5ca6cff87b
commit 02814d09e5
10 changed files with 85 additions and 247 deletions
+7 -8
View File
@@ -17,23 +17,22 @@ internal/glob the `*` / `?` matcher used by rules (linear time, no backt
internal/parser Parser interface + registry; nlb, revolut, traderepublic
internal/store SQLite index (modernc.org/sqlite, no cgo)
internal/importer directory walk, dedupe, balance checks
internal/rules applies ordered rules to rule_* columns only
internal/rules applies ordered rules, writing rule_tag
internal/report per-tag aggregation
internal/tui Bubble Tea models
```
## Invariants
**Rule verdicts and manual edits never share a column.** `rule_tag` is
rewritten wholesale on every retag; `manual_tag` is only ever written by the
user. The effective tag is `COALESCE(manual_tag, rule_tag)`. This is what makes
`money retag` safe to run at any time, and it is the first thing to preserve
when touching the schema or the rules engine. Covered by
`TestManualTagSurvivesRetag`.
**Every tag comes from rules.toml.** There is no way to tag a transaction by
hand; `rule_tag` is derived state that `Engine.Retag` rewrites wholesale, which
is what makes `money retag` safe to run at any time. Nothing may write a tag
from anywhere else — the moment something does, the index stops being
reproducible and retag stops being safe. Covered by `TestRetagRewritesEveryTag`.
**Statements are the source of truth; the index is disposable.** Deleting
`index.db` at the root of the data directory and re-importing must reproduce
everything except the tags set by hand.
everything, with nothing lost.
**Dedupe is by fingerprint**: `sha256(date | amount | normalised description |
ordinal)`, where the ordinal distinguishes identical lines *within one
+8 -7
View File
@@ -21,8 +21,8 @@ tagged by glob rules you write.
2026-01.csv
```
\* Deleting the index loses the tags you set by hand, which live only there.
Everything else is re-derived from the statements.
\* Nothing lives only in the index: it is derived entirely from the statements
plus rules.toml, so deleting it and re-importing gets you exactly what you had.
## Where the data root lives
@@ -80,8 +80,7 @@ imported — creating an `account.toml` is not enough on its own. Run
| --- | --- |
| `1` `2` `3` `4` `5` / `tab` | accounts · transactions · report · rule builder · rules |
| `enter` | open the selected account (accounts view) |
| `t` | set the tag on the selected transaction |
| `c` | clear a tag set by hand, falling back to the rules |
| `4` | write a rule for what you are looking at — the only way to tag |
| `/` | filter by description |
| `u` | show only untagged transactions |
| `a` | clear the account filter |
@@ -222,9 +221,11 @@ match = "*LIDL*"
tag = "groceries"
```
Editing rules never touches tags you set by hand: rule verdicts and manual
overrides are stored separately, and the manual one always wins. `money retag`
is therefore safe to run whenever you change the file.
Every tag comes from this file, so a tag is never something you have to keep
safe: `money retag` recomputes all of them from the current rules and is safe
to run whenever you change it. A transaction no rule matches simply stays
untagged, and shows up under `money ls --untagged`, the `u` view, and
`(untagged)` in the report.
## account.toml
+4 -4
View File
@@ -27,7 +27,7 @@ const usage = `money - statement-driven personal finance tracker
usage: money [--root DIR] <command> [flags]
commands:
tui browse and tag transactions, build and prune rules (default)
tui browse transactions, build and prune the rules that tag them (default)
import extract transactions from every statement into the index
retag re-apply rules.toml to everything already imported
ls list transactions
@@ -230,7 +230,7 @@ func cmdLs(root string, args []string) error {
account := fs.String("account", "", "only this account slug")
month := fs.String("month", "", "only this month (YYYY-MM)")
search := fs.String("search", "", "only descriptions containing this text")
untagged := fs.Bool("untagged", false, "only transactions with no effective tag")
untagged := fs.Bool("untagged", false, "only transactions no rule tagged")
limit := fs.Int("limit", 0, "maximum rows (0 = no limit)")
wide := fs.Bool("wide", false, "also show type and reported balance")
if err := fs.Parse(args); err != nil {
@@ -268,11 +268,11 @@ func cmdLs(root string, args []string) error {
}
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n",
t.Date, t.AccountSlug, t.FormatAmount(), t.Currency, balance,
t.Tag(), t.Type, t.Description)
t.RuleTag, t.Type, t.Description)
continue
}
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n",
t.Date, t.AccountSlug, t.FormatAmount(), t.Currency, t.Tag(), t.Description)
t.Date, t.AccountSlug, t.FormatAmount(), t.Currency, t.RuleTag, t.Description)
}
w.Flush()
fmt.Printf("\n%d transactions\n", len(txns))
+3 -10
View File
@@ -39,16 +39,9 @@ type Transaction struct {
// transaction, when it reports one. Used to verify the balance chain.
BalanceMinor *int64
RuleTag string
ManualTag string
}
// Tag is the effective category: a manual tag always beats a rule tag.
func (t Transaction) Tag() string {
if t.ManualTag != "" {
return t.ManualTag
}
return t.RuleTag
// RuleTag is the category, decided entirely by rules.toml. It is derived
// state: `money retag` rewrites it wholesale.
RuleTag string
}
// FormatAmount renders the amount using the account's minor-unit scale.
+1 -1
View File
@@ -30,7 +30,7 @@ func ByTag(txns []model.Transaction) []TagTotal {
acc := map[key]*TagTotal{}
for _, t := range txns {
tag := t.Tag()
tag := t.RuleTag
if tag == "" {
tag = Untagged
}
+2 -4
View File
@@ -1,8 +1,6 @@
// Package rules applies the ordered glob rules from rules.toml to
// transactions, deciding their automatic tag.
//
// Only rule_tag is ever written. A tag set by hand in the TUI lives in its own
// column and survives any number of re-runs.
// transactions, deciding their tag. It is the only thing that decides a tag,
// so rule_tag is derived state and Retag can rewrite it wholesale at any time.
package rules
import (
+29 -44
View File
@@ -66,60 +66,45 @@ func byDescription(t *testing.T, db *store.DB, desc string) model.Transaction {
return model.Transaction{}
}
// A manual tag must outlive any number of rule changes and re-runs. This is
// the property that makes retag safe to run at will.
func TestManualTagSurvivesRetag(t *testing.T) {
// A tag is derived state: rules.toml is the only thing that decides it, so
// retagging rewrites every tag from the current rules however often it runs,
// including back to nothing when no rule matches any more.
func TestRetagRewritesEveryTag(t *testing.T) {
db := testDB(t)
seed(t, db, "LIDL SOFIA", "SOME RANDOM SHOP")
engine := New(&config.Rules{Rule: []config.Rule{
{Match: "*LIDL*", Tag: "groceries"},
{Match: "*SHOP*", Tag: "shopping"},
}})
n, err := engine.Retag(db)
if err != nil {
t.Fatal(err)
}
if n != 2 {
t.Errorf("retagged %d rows, want 2", n)
}
if got := byDescription(t, db, "LIDL SOFIA").RuleTag; got != "groceries" {
t.Errorf("tag = %q, want groceries", got)
}
// Re-running with the same rules changes nothing and says so.
if n, err := engine.Retag(db); err != nil || n != 0 {
t.Errorf("second retag: %d rows, %v; want 0 and no error", n, err)
}
// New rules win, and a row no rule matches loses its tag entirely.
engine = New(&config.Rules{Rule: []config.Rule{
{Match: "*LIDL*", Tag: "food"},
}})
if _, err := engine.Retag(db); err != nil {
t.Fatal(err)
}
// The user hand-tags the row the rules did not match, and overrides the
// one they did.
shop := byDescription(t, db, "SOME RANDOM SHOP")
if err := db.SetManualTag(shop.ID, "hobby"); err != nil {
t.Fatal(err)
if got := byDescription(t, db, "LIDL SOFIA").RuleTag; got != "food" {
t.Errorf("tag = %q, want food", got)
}
lidl := byDescription(t, db, "LIDL SOFIA")
if err := db.SetManualTag(lidl.ID, "party supplies"); err != nil {
t.Fatal(err)
}
// Rules change completely, and are re-applied twice for good measure.
engine = New(&config.Rules{Rule: []config.Rule{
{Match: "*LIDL*", Tag: "food"},
{Match: "*SHOP*", Tag: "shopping"},
}})
for range 2 {
if _, err := engine.Retag(db); err != nil {
t.Fatal(err)
}
}
shop = byDescription(t, db, "SOME RANDOM SHOP")
if got := shop.Tag(); got != "hobby" {
t.Errorf("manual tag = %q, want %q (manual tags must win)", got, "hobby")
}
if shop.RuleTag != "shopping" {
t.Errorf("rule_tag = %q, want %q (rules must still be recorded underneath)", shop.RuleTag, "shopping")
}
lidl = byDescription(t, db, "LIDL SOFIA")
if got := lidl.Tag(); got != "party supplies" {
t.Errorf("manual override = %q, want %q", got, "party supplies")
}
// Clearing the override falls back to whatever the rules now say.
if err := db.ClearOverrides(lidl.ID); err != nil {
t.Fatal(err)
}
if got := byDescription(t, db, "LIDL SOFIA").Tag(); got != "food" {
t.Errorf("after clearing override, tag = %q, want %q", got, "food")
if got := byDescription(t, db, "SOME RANDOM SHOP").RuleTag; got != "" {
t.Errorf("tag = %q, want empty once no rule matches", got)
}
}
+9 -24
View File
@@ -51,7 +51,6 @@ CREATE TABLE IF NOT EXISTS transactions (
type TEXT NOT NULL DEFAULT '',
balance_minor INTEGER,
rule_tag TEXT,
manual_tag TEXT,
UNIQUE(account_id, fingerprint)
);
@@ -69,7 +68,7 @@ var migrations = []string{
// dropped are columns an older build created that this one no longer reads.
// Nothing indexes or constrains them, so they can simply go; leaving them
// would keep a NOT NULL column alive that no INSERT here ever names.
var dropped = []string{"counterparty", "rule_transfer", "manual_transfer"}
var dropped = []string{"counterparty", "rule_transfer", "manual_transfer", "manual_tag"}
// migrate brings an index created by an older build up to date, adding the
// columns it lacks and removing the ones it should no longer have.
@@ -225,8 +224,8 @@ func (d *DB) SourceFileSHA(accountID int64, path string) (string, bool, error) {
// InsertTransaction adds a transaction unless its fingerprint already exists
// for that account. It reports whether a new row was created.
//
// Existing rows are deliberately left untouched: re-importing an overlapping
// statement must never clobber tags the user set by hand.
// Existing rows are deliberately left untouched, so re-importing a statement
// that overlaps one already imported adds nothing rather than duplicating it.
func (d *DB) InsertTransaction(t model.Transaction) (bool, error) {
var balance any
if t.BalanceMinor != nil {
@@ -265,7 +264,7 @@ func (d *DB) Transactions(f Filter) ([]model.Transaction, error) {
SELECT t.id, t.account_id, a.slug, a.currency, a.minor_digits,
t.fingerprint, t.date, t.description, t.amount_minor,
COALESCE(s.path, ''), t.type, t.balance_minor,
COALESCE(t.rule_tag, ''), COALESCE(t.manual_tag, '')
COALESCE(t.rule_tag, '')
FROM transactions t
JOIN accounts a ON a.id = t.account_id
LEFT JOIN source_files s ON s.id = t.source_file_id
@@ -276,7 +275,7 @@ func (d *DB) Transactions(f Filter) ([]model.Transaction, error) {
args = append(args, f.AccountSlug)
}
if f.Untagged {
q += ` AND COALESCE(NULLIF(t.manual_tag, ''), NULLIF(t.rule_tag, '')) IS NULL`
q += ` AND NULLIF(t.rule_tag, '') IS NULL`
}
if f.Month != "" {
q += ` AND substr(t.date, 1, 7) = ?`
@@ -301,8 +300,7 @@ func (d *DB) Transactions(f Filter) ([]model.Transaction, error) {
)
if err := rows.Scan(&t.ID, &t.AccountID, &t.AccountSlug, &t.Currency, &t.MinorDigits,
&t.Fingerprint, &t.Date, &t.Description, &t.AmountMinor, &t.SourcePath,
&t.Type, &balance,
&t.RuleTag, &t.ManualTag); err != nil {
&t.Type, &balance, &t.RuleTag); err != nil {
return nil, err
}
if balance.Valid {
@@ -320,19 +318,6 @@ func (d *DB) Transactions(f Filter) ([]model.Transaction, error) {
return out, rows.Err()
}
// SetManualTag sets (or, with an empty tag, clears) the user's tag override.
func (d *DB) SetManualTag(id int64, tag string) error {
_, err := d.sql.Exec(`UPDATE transactions SET manual_tag = NULLIF(?, '') WHERE id = ?`, tag, id)
return err
}
// ClearOverrides drops the manual tag for a transaction, so the rules decide
// it again.
func (d *DB) ClearOverrides(id int64) error {
_, err := d.sql.Exec(`UPDATE transactions SET manual_tag = NULL WHERE id = ?`, id)
return err
}
// RuleAssignment is one row's recomputed rule verdict.
type RuleAssignment struct {
ID int64
@@ -382,11 +367,11 @@ func (d *DB) Count(accountID int64) (int, error) {
return n, err
}
// Tags lists every effective tag in use, for completion in the TUI.
// Tags lists every tag in use, for completion in the TUI.
func (d *DB) Tags() ([]string, error) {
rows, err := d.sql.Query(`
SELECT DISTINCT COALESCE(NULLIF(manual_tag, ''), NULLIF(rule_tag, '')) AS tag
FROM transactions WHERE tag IS NOT NULL ORDER BY tag`)
SELECT DISTINCT rule_tag FROM transactions
WHERE rule_tag IS NOT NULL AND rule_tag != '' ORDER BY rule_tag`)
if err != nil {
return nil, err
}
+7 -59
View File
@@ -1,5 +1,5 @@
// Package tui is the terminal interface: browse accounts, tag transactions,
// and see where the money went.
// Package tui is the terminal interface: browse accounts, write the rules that
// tag transactions, and see where the money went.
package tui
import (
@@ -47,12 +47,11 @@ const (
)
// input is the modal state: the transaction list is read-only until the user
// starts typing a tag or a search.
// starts typing a search.
type input int
const (
inputNone input = iota
inputTag
inputSearch
)
@@ -267,16 +266,12 @@ func (m *Model) reloadTxns() error {
rows := make([]table.Row, 0, len(txns))
for _, t := range txns {
tag := t.Tag()
if tag != "" && t.ManualTag != "" {
tag += "*" // marks a manual override
}
rows = append(rows, table.Row{
t.Date, t.AccountSlug, t.FormatAmount(), tag, t.Description,
t.Date, t.AccountSlug, t.FormatAmount(), t.RuleTag, t.Description,
})
}
// Keep the cursor in range after the row count shrinks (e.g. tagging the
// last untagged row while the untagged filter is on).
// Keep the cursor in range after the row count shrinks (e.g. a new rule
// tagging the last untagged row while the untagged filter is on).
cursor := m.txnTable.Cursor()
m.txnTable.SetRows(rows)
if cursor >= len(rows) {
@@ -989,20 +984,6 @@ func (m *Model) updateInput(msg tea.KeyMsg) (tea.Model, tea.Cmd) {
m.text.SetValue("")
switch mode {
case inputTag:
t, ok := m.selected()
if !ok {
return m, nil
}
if err := m.db.SetManualTag(t.ID, value); err != nil {
m.err = err
return m, nil
}
if value == "" {
m.status = "cleared manual tag"
} else {
m.status = "tagged as " + value
}
case inputSearch:
m.filter.Search = value
m.status = "search: " + value
@@ -1084,20 +1065,6 @@ func (m *Model) updateNormal(msg tea.KeyMsg) (tea.Model, tea.Cmd) {
m.err = m.reloadTxns()
return m, nil
case "t":
if m.view != viewTxns {
return m, nil
}
t, ok := m.selected()
if !ok {
return m, nil
}
m.input = inputTag
m.text.SetValue(t.ManualTag)
m.text.CursorEnd()
m.text.Focus()
return m, nil
case "/":
m.view = viewTxns
m.input = inputSearch
@@ -1106,23 +1073,6 @@ func (m *Model) updateNormal(msg tea.KeyMsg) (tea.Model, tea.Cmd) {
m.text.Focus()
return m, nil
case "c":
// Drop the manual tag and fall back to what the rules say.
if m.view != viewTxns {
return m, nil
}
t, ok := m.selected()
if !ok {
return m, nil
}
if err := m.db.ClearOverrides(t.ID); err != nil {
m.err = err
return m, nil
}
m.status = "manual overrides cleared"
m.err = m.reloadTxns()
return m, nil
case "r":
if m.importing {
m.status = "import in progress…"
@@ -1189,8 +1139,6 @@ func (m *Model) View() string {
switch {
case m.importing:
b.WriteString(statusStyle.Render(m.spinner.View() + m.importingLabel()))
case m.input == inputTag:
b.WriteString(statusStyle.Render("tag: ") + m.text.View())
case m.input == inputSearch:
b.WriteString(statusStyle.Render("search: ") + m.text.View())
case m.err != nil:
@@ -1389,6 +1337,6 @@ func (m *Model) help() string {
case viewReport:
return "1 accounts · 2 transactions · 4 new rule · 5 rules · u untagged · a all accounts · q quit"
default:
return "t tag · c clear · / search · u untagged · a all · i import · r retag · 4 new rule · 5 rules · q quit"
return "4 new rule · / search · u untagged · a all · i import · r retag · 5 rules · q quit"
}
}
+15 -86
View File
@@ -105,87 +105,6 @@ func cursorTo(t *testing.T, m *Model, desc string) {
t.Fatalf("no visible transaction with description %q", desc)
}
func TestTagFlow(t *testing.T) {
m, db := newTestModel(t)
key(t, m, "2") // transactions view
cursorTo(t, m, "SOME RANDOM SHOP")
key(t, m, "t") // open the tag prompt
if m.input != inputTag {
t.Fatal("expected the tag prompt to open")
}
typeText(t, m, "hobby")
key(t, m, "enter")
if m.input != inputNone {
t.Error("expected the prompt to close after enter")
}
txns, err := db.Transactions(store.Filter{})
if err != nil {
t.Fatal(err)
}
var found bool
for _, txn := range txns {
if txn.Description == "SOME RANDOM SHOP" {
found = true
if txn.ManualTag != "hobby" {
t.Errorf("manual_tag = %q, want %q", txn.ManualTag, "hobby")
}
}
}
if !found {
t.Fatal("transaction disappeared after tagging")
}
// The tag column marks manual overrides with a trailing asterisk.
if !strings.Contains(m.View(), "hobby*") {
t.Error("expected the tagged row to render with a manual-override marker")
}
}
func TestTagPromptCancel(t *testing.T) {
m, db := newTestModel(t)
key(t, m, "2")
cursorTo(t, m, "SOME RANDOM SHOP")
key(t, m, "t")
typeText(t, m, "nope")
key(t, m, "esc")
if m.input != inputNone {
t.Error("expected esc to close the prompt")
}
txns, _ := db.Transactions(store.Filter{Untagged: true})
if len(txns) != 1 {
t.Errorf("esc should not have tagged anything; untagged count = %d, want 1", len(txns))
}
}
// c drops a tag set by hand, restoring whatever the rules say.
func TestClearManualTag(t *testing.T) {
m, db := newTestModel(t)
key(t, m, "2")
cursorTo(t, m, "LIDL SOFIA")
key(t, m, "t")
typeText(t, m, "treats")
key(t, m, "enter")
if got := findTxn(t, db, "LIDL SOFIA").ManualTag; got != "treats" {
t.Fatalf("manual tag = %q, want treats", got)
}
key(t, m, "c")
txn := findTxn(t, db, "LIDL SOFIA")
if txn.ManualTag != "" {
t.Errorf("expected c to clear the manual tag, got %q", txn.ManualTag)
}
if txn.Tag() != "groceries" {
t.Errorf("tag after clearing = %q, want groceries from the rule", txn.Tag())
}
}
func TestUntaggedToggle(t *testing.T) {
m, _ := newTestModel(t)
@@ -197,11 +116,21 @@ func TestUntaggedToggle(t *testing.T) {
t.Fatalf("untagged view shows %+v, want only SOME RANDOM SHOP", m.txns)
}
// Tagging the last untagged row empties the view; the cursor must stay valid.
// A rule that claims the last untagged row empties the view; the cursor
// must stay valid. Saving a rule is the only way to tag anything now.
//
// Saving reloads the rules from rules.toml, so the fixture's rule has to
// exist there too or retagging would drop the tag it granted.
if err := config.AppendRule(m.root, config.Rule{Match: "*LIDL*", Tag: "groceries"}); err != nil {
t.Fatal(err)
}
cursorTo(t, m, "SOME RANDOM SHOP")
key(t, m, "t")
typeText(t, m, "hobby")
key(t, m, "4")
m.ruleGlob.SetValue("*RANDOM*")
m.setRuleFocus(2)
m.ruleTag.SetValue("hobby")
key(t, m, "enter")
key(t, m, "esc") // back to the transaction list, where u applies again
if len(m.txns) != 0 {
t.Errorf("expected the untagged view to be empty, got %d rows", len(m.txns))
}
@@ -572,7 +501,7 @@ func TestRuleBuilderSavesRuleAndRetags(t *testing.T) {
}
tagged := 0
for _, txn := range txns {
if txn.Tag() == "groceries" {
if txn.RuleTag == "groceries" {
tagged++
}
}
@@ -1019,7 +948,7 @@ func TestRuleListPruneUnused(t *testing.T) {
}
tagged := 0
for _, txn := range txns {
if txn.Tag() == "groceries" {
if txn.RuleTag == "groceries" {
tagged++
}
}