nikola/money
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Pair transfers from rules.toml

Browse Source 
The boolean transfer flag went two commits ago because a one-sided verdict let
half a movement vanish and left the report unbalanced. This is what replaces
it: a [[transfer]] block names both legs, and only a matched pair is dropped
from the report -- both legs together, never one.

Legs pair within five days, nearest date first, and a transaction belongs to at
most one transfer, so the first definition to claim a leg keeps it, exactly as
the first matching rule keeps a tag. The pairing is derived state like the tags:
Engine.Link rewrites the whole transfers table from rules.toml, which is why
retag re-derives both halves of what that file decides, and why it runs over
the whole index rather than a filtered view -- pairing inside one would let a
movement count as a transfer in one report and not in another. An unmatched leg
is not a transfer and keeps counting, surfaced as a warning instead.

Within one currency the amount is the evidence and must be the exact opposite.
Across currencies it is not checked at all: there are no rates here, so the two
numbers are unrelated and the dates carry the pairing alone.

tolerance_pct is the one exception, per definition, for a route where the bank
takes a fee and the two statements genuinely disagree. It defaults to zero and
belongs on the one definition that charges; a global or default tolerance would
loosen every route that does not. The difference it admits is not forgiven --
the pair leaves the report entirely, so a fee hidden inside one would be
spending that appears nowhere. Pair.Fee is what left less what arrived, and
report.Excluded carries it out per currency alongside the legs. It counts only
pairs whose legs are both in view, for the same reason it counts legs and not
transfers: half a pair cannot say what the other half received.

The screens:

- 6 builds a definition against the index as you type, showing the pairs it
  would form and the legs it would catch but leave unpaired. Six fields need
  more room than the rule builder's four, so the form sheds its spacing, then
  its hints, then the borders on unfocused fields.
- 7 lists every definition with what it pairs. Two counts, because they mean
  different things: an unpaired leg is a definition doing something and not
  finishing it, no pairs at all is dead weight. Tol names the tolerance, blank
  where amounts must agree.
- 3 grows a (transfers) row under TOTAL, and a fees row beneath it, or the
  report silently disagrees with the account balances.

Two things that are not part of transfers but are the same day's work:

- ls --uniq lists each account and description once, normalised the way a glob
  sees them, which is the shape of "what still needs a rule?" -- fifty visits
  to one shop are one pattern to write, not fifty rows to read.
- The rule builder's preview now filters to what the glob matches instead of
  marking matches in a full list. The count carries the context the rows no
  longer can: 2 of 7, measured against everything still in view.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
nikola
2026-08-15 19:19:12 +02:00
co-authored by Claude Opus 5
parent ef6a231d98
commit ebf7770569
16 changed files with 3472 additions and 177 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/config/config.go
+184 -33
View File
@@ -39,9 +39,40 @@ type Rule struct {
Note string `toml:"note"`
}
// Rules is the parsed rules.toml.
// Transfer is one entry in rules.toml describing money moved between two
// accounts the user owns. Both sides are named: a transfer is only ever a pair,
// which is what lets the report drop it without leaving half a movement behind.
//
// All four patterns are required. A one-sided definition would be a rule.
type Transfer struct {
FromAccount string `toml:"from_account"`
FromDesc string `toml:"from_desc"` // glob vs. the leaving leg's description
ToAccount string `toml:"to_account"`
ToDesc string `toml:"to_desc"` // glob vs. the arriving leg's description
// TolerancePct is how far short (or over) the arriving leg may be and still
// count as the same movement, as a percentage of the leg that left. It exists
// for routes where the bank takes a fee on the way, so the two statements
// genuinely disagree about the amount.
//
// Zero — the default, and what every definition written before this key
// existed means — requires the exact opposite amount. Keep it that way unless
// a route actually charges: the wider the tolerance, the more likely two
// unrelated movements in one window pair with each other.
//
// It applies within one currency only. Across currencies the amount is not
// checked at all, so there is nothing for a tolerance to loosen.
TolerancePct float64 `toml:"tolerance_pct"`
// Note is free text for the reader, exactly as on a Rule: it never takes
// part in matching.
Note string `toml:"note"`
}
// Rules is the parsed rules.toml: the tagging rules and the transfer
// definitions, which share the file because they are both hand-maintained
// statements about the same transactions.
type Rules struct {
Rule []Rule `toml:"rule"`
Rule []Rule `toml:"rule"`
Transfer []Transfer `toml:"transfer"`
}
// LoadRules reads rules.toml from the data root. A missing file is not an
@@ -63,9 +94,37 @@ func LoadRules(root string) (*Rules, error) {
return nil, fmt.Errorf("%s: rule %d (%q) sets no tag", path, i+1, rule.Match)
}
}
for i, t := range r.Transfer {
if err := checkTransfer(t); err != nil {
return nil, fmt.Errorf("%s: transfer %d: %w", path, i+1, err)
}
}
return &r, nil
}
// checkTransfer rejects a half-written definition. Both sides are needed to
// pair anything at all, so a missing one is refused on load rather than
// silently matching nothing.
func checkTransfer(t Transfer) error {
switch {
case t.FromAccount == "":
return fmt.Errorf("no from_account")
case t.FromDesc == "":
return fmt.Errorf("no from_desc pattern")
case t.ToAccount == "":
return fmt.Errorf("no to_account")
case t.ToDesc == "":
return fmt.Errorf("no to_desc pattern")
}
// A negative tolerance is a typo, and one at 100% or beyond would let any
// amount pair with any other — at which point the dates decide alone, which
// is the cross-currency rule and not something to arrive at by accident.
if t.TolerancePct < 0 || t.TolerancePct >= 100 {
return fmt.Errorf("tolerance_pct %g is not between 0 and 100", t.TolerancePct)
}
return nil
}
// AppendRule adds a rule to the end of rules.toml, creating the file if it is
// not there yet. Appending rather than inserting means an existing rule always
// keeps precedence, since the first match wins.
@@ -80,6 +139,30 @@ func AppendRule(root string, r Rule) error {
return fmt.Errorf("a rule needs a tag")
}
return appendBlock(root, formatRule(r))
}
// AppendTransfer adds a transfer definition to the end of rules.toml. Order
// matters for transfers as it does for rules: an earlier definition claims a
// transaction first, so appending cannot steal a leg from one already written.
func AppendTransfer(root string, t Transfer) error {
if err := checkTransfer(t); err != nil {
// A missing side reads better as what the form still wants; anything
// else already says what is wrong with what was typed.
if rest, ok := strings.CutPrefix(err.Error(), "no "); ok {
return fmt.Errorf("a transfer needs %s", rest)
}
return err
}
return appendBlock(root, formatTransfer(t))
}
// appendBlock adds a rendered TOML table to the end of rules.toml, creating the
// file if it is not there yet.
//
// The file is rewritten through a temporary file so a failure part-way cannot
// leave the user with a truncated config.
func appendBlock(root, block string) error {
path := filepath.Join(root, RulesFile)
existing, err := os.ReadFile(path)
if err != nil && !os.IsNotExist(err) {
@@ -92,20 +175,52 @@ func AppendRule(root string, r Rule) error {
b.WriteString("\n")
}
b.WriteString("\n")
b.WriteString(formatRule(r))
b.WriteString(block)
return writeFileAtomic(root, path, b.String())
}
// DeleteRules removes the rules at the given positions (0-based, as loaded by
// LoadRules) from rules.toml.
func DeleteRules(root string, positions []int) (int, error) {
return deleteBlocks(root, "rule", positions)
}
// DeleteTransfers removes the transfer definitions at the given positions
// (0-based, as loaded by LoadRules) from rules.toml.
func DeleteTransfers(root string, positions []int) (int, error) {
return deleteBlocks(root, "transfer", positions)
}
// blockStart is where one array-of-tables entry begins in rules.toml.
type blockStart struct {
table string // "rule" or "transfer"
line int
}
// blockStarts finds every [[table]] header in the file. Every kind is
// collected, not just the one being deleted: a block ends where the *next*
// block of any kind begins, so deleting a rule that happens to sit above a
// transfer must not swallow it.
func blockStarts(lines []string) []blockStart {
var out []blockStart
for i, line := range lines {
s := strings.TrimSpace(line)
if strings.HasPrefix(s, "[[") && strings.HasSuffix(s, "]]") {
out = append(out, blockStart{table: strings.TrimSpace(s[2 : len(s)-2]), line: i})
}
}
return out
}
// deleteBlocks removes entries of one table from rules.toml.
//
// The file is edited textually rather than re-serialised from the parsed
// rules, so comments, ordering and formatting the user put there by hand
// survive. A comment block sitting directly above a deleted rule goes with it,
// since it documents that rule; a comment separated by a blank line is treated
// values, so comments, ordering and formatting the user put there by hand
// survive. A comment block sitting directly above a deleted entry goes with it,
// since it documents that entry; a comment separated by a blank line is treated
// as a section heading and left alone.
func DeleteRules(root string, positions []int) (int, error) {
func deleteBlocks(root, table string, positions []int) (int, error) {
if len(positions) == 0 {
return 0, nil
}
@@ -121,23 +236,25 @@ func DeleteRules(root string, positions []int) (int, error) {
}
lines := strings.Split(string(raw), "\n")
// Where each [[rule]] block begins.
var starts []int
for i, line := range lines {
if strings.TrimSpace(line) == "[[rule]]" {
starts = append(starts, i)
starts := blockStarts(lines)
// mine[p] is where the p-th entry of this table sits among all the blocks.
var mine []int
for k, s := range starts {
if s.table == table {
mine = append(mine, k)
}
}
for _, p := range positions {
if p < 0 || p >= len(starts) {
return 0, fmt.Errorf("rule %d is out of range; %s holds %d rules", p+1, path, len(starts))
if p < 0 || p >= len(mine) {
return 0, fmt.Errorf("%s %d is out of range; %s holds %d %ss",
table, p+1, path, len(mine), table)
}
}
// A rule owns the run of comment lines directly above it, with no blank
// An entry owns the run of comment lines directly above it, with no blank
// line in between. Anything further up is a heading for what follows.
prefix := func(k int) int {
i := starts[k]
i := starts[k].line
for i > 0 && strings.HasPrefix(strings.TrimSpace(lines[i-1]), "#") {
i--
}
@@ -145,12 +262,10 @@ func DeleteRules(root string, positions []int) (int, error) {
}
drop := map[int]bool{}
for k := range starts {
if !doomed[k] {
continue
}
// The block runs up to the next rule's comment prefix, so a comment
// introducing the following rule is not swept up with this one.
for p := range doomed {
k := mine[p]
// The block runs up to the next block's comment prefix, so a comment
// introducing the following one is not swept up with this one.
end := len(lines)
if k+1 < len(starts) {
end = prefix(k + 1)
@@ -158,9 +273,9 @@ func DeleteRules(root string, positions []int) (int, error) {
for i := prefix(k); i < end; i++ {
drop[i] = true
}
// Blank lines are the gap between rules, not part of either; leaving
// Blank lines are the gap between blocks, not part of either; leaving
// them avoids gluing the neighbours together.
for i := end - 1; i >= starts[k] && strings.TrimSpace(lines[i]) == ""; i-- {
for i := end - 1; i >= starts[k].line && strings.TrimSpace(lines[i]) == ""; i-- {
delete(drop, i)
}
}
@@ -173,14 +288,27 @@ func DeleteRules(root string, positions []int) (int, error) {
}
out := collapseBlankRuns(kept)
// Never write something that will not load again.
// Never write something that will not load again, and never let deleting
// one kind of block take a different kind with it.
var check Rules
if _, err := toml.Decode(out, &check); err != nil {
return 0, fmt.Errorf("deleting from %s would produce invalid TOML: %w", path, err)
}
if want := len(starts) - len(doomed); len(check.Rule) != want {
return 0, fmt.Errorf("deleting from %s would leave %d rules, expected %d",
path, len(check.Rule), want)
counts := map[string]int{"rule": len(check.Rule), "transfer": len(check.Transfer)}
for _, kind := range []string{"rule", "transfer"} {
want := 0
for _, s := range starts {
if s.table == kind {
want++
}
}
if kind == table {
want -= len(doomed)
}
if counts[kind] != want {
return 0, fmt.Errorf("deleting from %s would leave %d %ss, expected %d",
path, counts[kind], kind, want)
}
}
if err := writeFileAtomic(root, path, out); err != nil {
@@ -238,15 +366,38 @@ func writeFileAtomic(dir, path, content string) error {
return nil
}
// writeKey renders one TOML key, skipping it when empty. The column is wide
// enough for the longest key either block uses, so the values line up.
func writeKey(b *strings.Builder, key, value string) {
if value != "" {
fmt.Fprintf(b, "%-12s = %s\n", key, strconv.Quote(value))
}
}
// formatTransfer renders a transfer as a TOML table, the two sides in the
// order money travels.
func formatTransfer(t Transfer) string {
var b strings.Builder
b.WriteString("[[transfer]]\n")
writeKey(&b, "from_account", t.FromAccount)
writeKey(&b, "from_desc", t.FromDesc)
writeKey(&b, "to_account", t.ToAccount)
writeKey(&b, "to_desc", t.ToDesc)
// Only when set: a zero written out would suggest the key is doing something
// when it is exactly the default every other definition already has.
if t.TolerancePct != 0 {
fmt.Fprintf(&b, "%-12s = %s\n", "tolerance_pct",
strconv.FormatFloat(t.TolerancePct, 'f', -1, 64))
}
writeKey(&b, "note", t.Note)
return b.String()
}
// formatRule renders a rule as a TOML table, omitting empty fields.
func formatRule(r Rule) string {
var b strings.Builder
b.WriteString("[[rule]]\n")
write := func(key, value string) {
if value != "" {
fmt.Fprintf(&b, "%-12s = %s\n", key, strconv.Quote(value))
}
}
write := func(key, value string) { writeKey(&b, key, value) }
write("match", r.Match)
write("type", r.Type)
write("account", r.Account)