Compare commits
12
Commits
8e4df08f43
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a086f9242d | ||
|
|
c271cf2c8f | ||
|
|
3d72a54be1 | ||
|
|
ba7e0e2b42 | ||
|
|
ed52be3f1d | ||
|
|
bd406a110d | ||
|
|
a5f7541980 | ||
|
|
9ad6c05f9a | ||
|
|
7f5b4846ef | ||
|
|
17428a4f51 | ||
|
|
7ec9976b80 | ||
|
|
ec8a844441 |
@@ -2,7 +2,8 @@
|
||||
|
||||
Statement-driven personal finance tracker. A data directory holds one folder per
|
||||
account; statements dropped into those folders are parsed into a rebuildable
|
||||
SQLite index, categorised by ordered glob rules, and browsed in a Bubble Tea TUI.
|
||||
SQLite index, categorised by ordered glob rules, and browsed in a web app
|
||||
(`money serve`, which is also the default command).
|
||||
|
||||
Read `README.md` first — it is the user-facing reference for every config key.
|
||||
This file covers what the code assumes and why.
|
||||
@@ -10,7 +11,7 @@ This file covers what the code assumes and why.
|
||||
## Layout
|
||||
|
||||
```
|
||||
cmd/money/main.go subcommands; the TUI is the default
|
||||
cmd/money/main.go subcommands; serve is the default
|
||||
internal/config rules.toml (rules + transfers), account.toml, XDG config, data-root resolution
|
||||
internal/model Account, Transaction, amount formatting, description normalisation
|
||||
internal/glob the `*` / `?` matcher used by rules (two-pointer, no exponential blowup)
|
||||
@@ -20,7 +21,6 @@ internal/importer directory walk, dedupe, balance checks
|
||||
internal/rules applies ordered rules, writing rule_tag
|
||||
internal/transfers pairs the two legs of a movement, writing the transfers table
|
||||
internal/report per-tag aggregation
|
||||
internal/tui Bubble Tea models
|
||||
internal/web `money serve`: JSON API + embedded single page (static/)
|
||||
```
|
||||
|
||||
@@ -59,7 +59,7 @@ than forgiven.** That is the condition on the tolerance existing at all: the
|
||||
pair leaves the report entirely, so a difference swallowed inside one would be
|
||||
spending that never appears anywhere. `Pair.Fee` is what left less what
|
||||
arrived, and `report.Excluded` carries it out per currency alongside the legs —
|
||||
named on the `money report` line and on its own row under the TUI's report.
|
||||
named on the `money report` line and on its own row under the web report.
|
||||
Nothing may pair on a mismatch without that difference reaching `Excluded`.
|
||||
It is a fee and not a tag: no rule produces it, `report.ByTag` never sees it,
|
||||
and it must not be turned into a synthetic transaction to make the total
|
||||
@@ -143,51 +143,43 @@ Anything new that reports a rule must report its file position too.
|
||||
settles ties: a saved rule cannot displace an equally specific one written by
|
||||
hand, while a narrower one is meant to take precedence and does.
|
||||
|
||||
**The report's period narrows the report and nothing else.** The screen opens
|
||||
on last month, and `←`/`→` step along `report.Periods` — the named windows,
|
||||
then the months the index holds. `reloadReport` therefore runs its *own* query
|
||||
rather than reusing the rows the transaction list is showing: the two share
|
||||
`m.filter` (account, search, untagged) and differ only in the date bounds, so
|
||||
opening on last month must not hide the rest of the index from the list beside
|
||||
it. Nothing may put the period into `m.filter`, which is exactly what would
|
||||
make it leak. Covered by `TestReportPeriodLeavesTheTransactionListAlone`.
|
||||
**The report's period narrows the report and nothing else.** The screen opens on
|
||||
last month, and `←`/`→` step along `report.Periods` — the named windows, then
|
||||
the months the index holds. The report and the transaction list share a scope
|
||||
(account, search, untagged), which `web.filterFrom` reads and `state.filter`
|
||||
holds in the page; the period is read by the report handler alone and kept in
|
||||
`state.report`, so opening on last month cannot hide the rest of the index from
|
||||
the list. Nothing may put the period into that shared scope, which is exactly
|
||||
what would make it leak. Covered by `TestReportPeriodLeavesTransactionsAlone`.
|
||||
|
||||
The axis is built from `store.Months` over the *whole* index, not from the rows
|
||||
in view, or it would grow and shrink as the account or search filter changed and
|
||||
move under the cursor. It is rebuilt on every `reload` (an import can reach
|
||||
further back) but keeps the window the user was on. The windows are relative to
|
||||
today, never to the newest statement: "last month" with nothing in it reports
|
||||
nothing and says so, because silently answering for a month nobody asked for is
|
||||
worse than an empty screen. That is also why an empty period and an empty index
|
||||
give different messages — one asks for another period, the other for an import.
|
||||
move under the cursor. It is rebuilt on every request (an import can reach
|
||||
further back), and the page names the period by its label rather than its
|
||||
position, so a grown axis keeps the window the user was on. The windows are
|
||||
relative to today, never to the newest statement: "last month" with nothing in
|
||||
it reports nothing and says so, because silently answering for a month nobody
|
||||
asked for is worse than an empty screen. That is also why an empty period and
|
||||
an empty index give different messages — one asks for another period, the
|
||||
other for an import.
|
||||
|
||||
**The report's sort rearranges rows; it never changes which rows there are.**
|
||||
`report.Order` is passed to `ByTag` and is deliberately *not* part of
|
||||
`store.Filter` or `m.filter` — the period decides what is counted, the order
|
||||
only how it is listed, and merging the two would make a sort able to hide a
|
||||
tag. Currency stays the outer sort key under every order, because there are no
|
||||
`store.Filter` or the shared scope — the period decides what is counted, the
|
||||
order only how it is listed, and merging the two would make a sort able to hide
|
||||
a tag. Currency stays the outer sort key under every order, because there are no
|
||||
exchange rates to compare two currencies by, and every order falls back to the
|
||||
tag so ties keep a fixed position instead of shuffling between reloads.
|
||||
`Order.Column` is what lets the TUI mark the sorted heading without keeping its
|
||||
`Order.Column` is what lets the page mark the sorted heading without keeping its
|
||||
own copy of that mapping; a new order needs a column of its own, which
|
||||
`TestOrderColumnsAreDistinct` checks.
|
||||
|
||||
**The builders are forms, so the global keymap must not apply there.**
|
||||
`Update` routes to `updateRules` / `updateTransfers` before `updateNormal`
|
||||
whenever the view is `viewRules` or `viewTransfers`, or typing `q` would quit
|
||||
and `i` would start an import. Any new full-screen input needs the same
|
||||
treatment. It cuts the other way too: a command *inside* a builder has to be a
|
||||
chord — `ctrl+s` re-sorts the preview — because every printable key belongs to
|
||||
the field being typed in.
|
||||
|
||||
**In the rule builder, `tab` completes first and moves focus second.** The
|
||||
account and tag fields use `textinput.ShowSuggestions`, whose own `AcceptSuggestion`
|
||||
key is `tab` and whose `NextSuggestion`/`PrevSuggestion` are `up`/`down` — all
|
||||
three already meant something here. So `updateRules` intercepts `tab` and calls
|
||||
`acceptCompletion` before falling back to `setRuleFocus`, keeps `up`/`down` on
|
||||
field navigation, and lets `ctrl+n` / `ctrl+p` through to the input for cycling.
|
||||
`SetValue` does not re-match the suggestion list, so `acceptCompletion` re-sets
|
||||
it afterwards or `ctrl+n` would offer candidates that no longer fit the value.
|
||||
**There are no keyboard shortcuts.** The page used to carry the TUI's keymap —
|
||||
screens on `1`–`8`, `u`, `/`, `i`, `r`, and chords inside the builders — and it
|
||||
was removed on purpose: every action is a link or a button. Do not add a global
|
||||
`keydown` handler back without being asked; a single-letter key fires while
|
||||
typing in a field unless every one of them is guarded, which is what the old one
|
||||
existed to do.
|
||||
|
||||
**A rule's usage count is how many transactions it wins, not how many its glob
|
||||
could match** — `Engine.Usage` counts by `MatchIndex`, so a rule shadowed by a
|
||||
@@ -211,41 +203,76 @@ writer goes through `writeFileAtomic`, and both editors re-parse the result
|
||||
before replacing the file.
|
||||
|
||||
**An edit must not lose what the builder does not show.** The form has four
|
||||
fields and a `Rule` has five, so `saveRule` carries `Type` through from the rule
|
||||
being edited and the form says it is doing so. Nothing may round-trip a rule
|
||||
through those four fields alone — a pattern the user was never shown is not a
|
||||
pattern they chose to remove. A new field on `Rule` needs the same treatment or
|
||||
a field of its own. Covered by `TestRuleEditKeepsTypePattern`.
|
||||
fields and a `Rule` has five, so `editRule` takes `Type` from the rule on disk —
|
||||
never from the request — and the form says it is keeping it. Nothing may
|
||||
round-trip a rule through those four fields alone — a pattern the user was
|
||||
never shown is not a pattern they chose to remove. A new field on `Rule` needs
|
||||
the same treatment or a field of its own. Covered by `TestEditKeepsTypePattern`.
|
||||
|
||||
**The rule builder's preview is what the rule is judged against, which is not
|
||||
always "what is untagged".** For a new rule those are the same thing. For an
|
||||
edit they are not: the rule's own transactions are tagged, so an untagged-only
|
||||
preview would be empty for a rule that works. `reloadPreviewGroups` adds them
|
||||
back, and `refreshRulePreview` keeps the ones the new glob stops catching on
|
||||
screen marked `−` instead of dropping them silently, because giving one up is
|
||||
the decision being made.
|
||||
preview would be empty for a rule that works. `previewGroups` adds them back,
|
||||
and `rulePreview` keeps the ones the new glob stops catching on screen marked
|
||||
`−` instead of dropping them silently, because giving one up is the decision
|
||||
being made. Covered by `TestEditPreviewShowsWhatIsLetGo`.
|
||||
|
||||
**The web app is a second frontend, not a second implementation.**
|
||||
`internal/web` ports the TUI's screens over HTTP, and everything that decides
|
||||
something stays in Go: amounts are formatted server-side (`money` is text plus
|
||||
a sign, never a number the browser could add up), the rule builder's preview is
|
||||
matched by `glob.Match` on the server rather than re-implemented in JS, and the
|
||||
transfer preview runs `transfers.Analyze`. Keep it that way — a JS glob or a
|
||||
JS sum is a second copy of a rule that would drift from the first.
|
||||
**The web app is a frontend, not a second implementation.** Everything that
|
||||
decides something stays in Go: amounts are formatted server-side (`money` is
|
||||
text plus a sign, never a number the browser could add up), the rule builder's
|
||||
preview is matched by `glob.Match` on the server rather than re-implemented in
|
||||
JS, and the transfer preview runs `transfers.Analyze`. Keep it that way — a
|
||||
JS glob or a JS sum is a second copy of a rule that would drift from the
|
||||
first.
|
||||
`static/app.js` only lays out what the API returns.
|
||||
|
||||
The web server outlives edits to rules.toml in a way the TUI does not, so
|
||||
`/api/retag` and `/api/import` re-read it first (import also re-reads the
|
||||
The server outlives edits to rules.toml in a way a one-shot command does not,
|
||||
so `/api/retag` and `/api/import` re-read it first (import also re-reads the
|
||||
account folders), and `/api/overview` reports `stale` when the file on disk no
|
||||
longer equals what the engines hold. Every edit and delete sends the rule or
|
||||
transfer it showed alongside the position, and is refused with 409 unless
|
||||
rules.toml still holds exactly that there — a position from a stale page can
|
||||
otherwise name a different rule. An edit takes `Type` from the rule *on disk*,
|
||||
never from the request, which is the web side of `TestRuleEditKeepsTypePattern`.
|
||||
otherwise name a different rule. Covered by `TestStalePositionIsRefused`.
|
||||
|
||||
There is no auth by design (`--addr` defaults to loopback). Non-GET requests
|
||||
must be `application/json`, which is what keeps a cross-site form from posting
|
||||
to it; do not relax that without putting something else in its place.
|
||||
That is why `/api/upload` takes files base64 inside JSON rather than as
|
||||
multipart: multipart is precisely what a cross-site form can send.
|
||||
|
||||
**An upload only puts a file where `money import` looks.** It writes into an
|
||||
existing account folder and stops there: importing stays the user's call, made
|
||||
with the Import button, so the statements stay the source of truth and nothing
|
||||
reaches the index any other way. It never overwrites a statement (identical
|
||||
contents are a no-op, different ones a 409), refuses any name import would not
|
||||
read back — not a plain file name, a dotfile, `account.toml`, outside the
|
||||
account's `include` — and checks a whole batch before writing any of it. Covered
|
||||
by `TestUploadSavesWithoutImporting`, `TestUploadNeverReplacesAStatement` and
|
||||
`TestUploadRefusesBadNames`.
|
||||
|
||||
**The statements list is read from the folders, not the index.** `/api/files`
|
||||
walks `importer.StatementFiles` — the same list import reads — and only then
|
||||
looks each file up in `store.SourceFiles`, comparing `importer.Checksum`, so a
|
||||
file is listed exactly when import would read it and a file the index remembers
|
||||
but the disk lost shows as `missing` instead of vanishing.
|
||||
`/api/files/{account}/{name}` serves a file only by finding it in that list,
|
||||
never by joining the name onto a path. A statement is served from the app's
|
||||
origin, where a script could drive the API, so nothing is ever rendered as a
|
||||
page: text is `text/plain` under `CSP: sandbox`, anything not text or PDF is a
|
||||
sandboxed download, and PDFs — whose viewers refuse a sandbox — open in the
|
||||
browser's own isolated viewer. Covered by `TestServeFileServesOnlyStatements`.
|
||||
|
||||
**Deleting a statement marks its neighbours to be re-read.** `/api/files/delete`
|
||||
finds the file the same way `serveFile` does and deletes it — permanently, by
|
||||
the user's choice; the page says so before it asks. `store.ForgetSourceFile`
|
||||
then drops its transactions and their transfer rows, and `Link` re-pairs, which
|
||||
reads no statement. It does not import, any more than an upload does. A
|
||||
transaction two overlapping statements share is stored once, under the file
|
||||
imported first, so it goes too; `ForgetSourceFile` therefore blanks the
|
||||
checksums of the account's other statements, which show as `changed` until the
|
||||
next Import re-reads them and restores what they hold. Without that the checksum
|
||||
skip would leave those rows gone for good. Covered by
|
||||
`TestRemoveFileKeepsWhatAnotherStatementHolds`.
|
||||
|
||||
## Adding a bank parser
|
||||
|
||||
@@ -286,6 +313,19 @@ description — at the end, and not in the position it held on the page, so an
|
||||
IBAN wrapped across continuation lines stays contiguous for a glob to match.
|
||||
A new parser must do the same rather than reintroduce a structured field.
|
||||
|
||||
**A parser may name its statements, and only uploads use it.** `parser.Namer` is
|
||||
optional, like `parser.Warner`: `nlb` names an izpisek `izpisek_YYYY-MM-DD`
|
||||
after its *Datum izpiska*, ported from the `rename_izpiski.py` it replaced, and
|
||||
`traderepublic` names a statement `traderepublic_YYYY_MM_DD_YYYY_MM_DD` after
|
||||
the period in its header. A name is derived from what the statement says, never
|
||||
from the upload's own name, and comes back without an extension; the upload's is
|
||||
kept, lowercased. `/api/upload` stages each file as a dotfile — invisible to
|
||||
import — so the parser can read it, then numbers a chosen name that is taken by
|
||||
different contents (`_2`, as the script did) where a name the user gave is
|
||||
refused with 409. Nothing renames a file already in a folder: `source_files`
|
||||
records statements by path, so a rename would orphan its rows as `missing` until
|
||||
the index is rebuilt. Covered by `TestUploadNamesStatementsTheParserCanName`.
|
||||
|
||||
## Verifying
|
||||
|
||||
```
|
||||
@@ -306,20 +346,18 @@ migrates it, so a demo root left over from an earlier build either carries dead
|
||||
columns or fails with `no such column`, depending on which way the schema
|
||||
moved.
|
||||
|
||||
### Driving the TUI in tests
|
||||
### Testing the web app
|
||||
|
||||
Prefer feeding `tea.Msg` values to `Model.Update` directly — that is how every
|
||||
existing TUI test works, and it covers the keymap without a terminal.
|
||||
Drive the API through `Server.Handler()` with `httptest` — `newTestServer` in
|
||||
`internal/web/server_test.go` builds a data root and an index tagged and paired
|
||||
as an import would leave them, with "today" fixed so the report's default
|
||||
window is predictable. That is how every web test works, and since the page
|
||||
only lays out what the API returns, it is where behaviour gets tested.
|
||||
|
||||
If a real terminal is genuinely needed, note that piping into `script` does not
|
||||
deliver keystrokes. Use a pty and answer the two capability queries Bubble Tea
|
||||
sends on startup, or the program blocks before its first render:
|
||||
|
||||
- `ESC]11;?` (background colour) → reply `ESC]11;rgb:1e1e/1e1e/1e1e ESC\`
|
||||
- `ESC[6n` (cursor position) → reply `ESC[1;1R`
|
||||
|
||||
Also delete the index first, or the import you are trying to observe will be
|
||||
skipped by checksum and finish instantly.
|
||||
For a look at the real page, run `money --root <demo> serve --addr
|
||||
127.0.0.1:<port>` against a throwaway root. Delete the index first if you want
|
||||
to watch an import do work, or it will be skipped by checksum and finish
|
||||
instantly.
|
||||
|
||||
## Conventions
|
||||
|
||||
|
||||
@@ -84,8 +84,7 @@ only if you hand the binary to someone else.
|
||||
## Usage
|
||||
|
||||
```
|
||||
money # open the TUI (default)
|
||||
money serve # the same screens as a web app, on 127.0.0.1:8080
|
||||
money # the web app on 127.0.0.1:8080 (same as `money serve`)
|
||||
money serve --addr :8080 # listen on every interface (there is no login)
|
||||
money import # extract new transactions from every statement
|
||||
money import --force # re-parse statements even if unchanged
|
||||
@@ -103,7 +102,7 @@ money config # which data root is in use, and why
|
||||
|
||||
An account folder appears in the app only once its statements have been
|
||||
imported — creating an `account.toml` is not enough on its own. Run
|
||||
`money import` (or press `i` in the TUI) after adding one.
|
||||
`money import` (or press Import in the web app) after adding one.
|
||||
|
||||
`--uniq` turns `ls` into a list of patterns still to write rather than a list of
|
||||
rows to read: one line per distinct description, since fifty visits to the same
|
||||
@@ -124,32 +123,125 @@ LIDL SOFIA 4412
|
||||
|
||||
The account is not part of what makes a row distinct. A rule matches on the
|
||||
description and only optionally narrows to an account, so one payee seen on two
|
||||
accounts is still one pattern to write — and it is one row in the rule builder
|
||||
on `4`, which groups the same way. Pass `--account` to scope the listing
|
||||
instead.
|
||||
accounts is still one pattern to write — and it is one row in the rule builder,
|
||||
which groups the same way. Pass `--account` to scope the listing instead.
|
||||
|
||||
It composes with the other filters (`--account`, `--month`, `--search`), and
|
||||
`--limit` caps the rows printed, saying how many it held back. It cannot be
|
||||
combined with `--wide`, whose columns all belong to a single transaction.
|
||||
|
||||
### TUI keys
|
||||
## The web app
|
||||
|
||||
| Key | Action |
|
||||
`money serve`, which is also what `money` runs with no command, puts eight
|
||||
screens in a browser: accounts, transactions, report, rule builder, rules,
|
||||
transfer builder, transfers and tags. It runs the same code as the CLI: amounts
|
||||
are formatted, globs matched and transfers paired on the server, and the page
|
||||
only shows the answers. It is one binary with the page built in; nothing else to
|
||||
deploy.
|
||||
|
||||
```
|
||||
money # http://127.0.0.1:8080
|
||||
money serve --root /srv/money --addr 0.0.0.0:8080
|
||||
```
|
||||
|
||||
**There is no authentication.** Anyone who can reach the port can read every
|
||||
transaction, rewrite `rules.toml` and start an import, which is why it listens
|
||||
on loopback unless `--addr` says otherwise. To use it from elsewhere, put it
|
||||
behind a reverse proxy that does the logging in, or reach it over SSH / a VPN.
|
||||
Requests that change anything must be sent as JSON, so another website open in
|
||||
the same browser cannot post a form to it.
|
||||
|
||||
The server outlives hand edits to `rules.toml`, so it treats the file the way a
|
||||
fresh command would:
|
||||
|
||||
- **Retag and Import re-read `rules.toml`** (and Import the account folders)
|
||||
before running, exactly as `money retag` / `money import` would. Until then a
|
||||
banner says the file on disk no longer matches what the index was derived
|
||||
from.
|
||||
- **Edits and deletes check the file first.** They go by rule position, and the
|
||||
page also sends the rule it showed you; if `rules.toml` no longer holds that
|
||||
rule there (another tab, a hand edit) the change is refused and you are asked
|
||||
to reload, rather than editing whichever rule moved into its place.
|
||||
|
||||
**Re-read all** is `import --force`: every statement is parsed again, including
|
||||
the ones an Import skips because they have not changed. Use it after updating
|
||||
money, when a parser has learned to read something it used to get wrong; rows
|
||||
already in the index are recognised and not duplicated.
|
||||
|
||||
### Statements
|
||||
|
||||
The Accounts screen also lists every statement file in every account
|
||||
folder, with what the index made of it:
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
| `1` `2` `3` `4` `5` `6` `7` / `tab` | accounts · transactions · report · rule builder · rules · transfer builder · transfers |
|
||||
| `enter` | open the selected account (accounts view) |
|
||||
| `/` | filter by description |
|
||||
| `u` | show only untagged transactions (matched transfer legs are not among them) |
|
||||
| `a` | clear the account filter |
|
||||
| `←` `→` | move the report's period (report view) |
|
||||
| `s` | change how the report is sorted (report view) |
|
||||
| `i` | import · `r` re-apply rules · `q` quit |
|
||||
| `✓` imported | read, and unchanged since |
|
||||
| `⚠` changed | the file differs from what was imported; Import re-reads it |
|
||||
| `•` not imported | new since the last import — or an import tried and failed, in which case the error is in the import report |
|
||||
| `✗` gone from disk | imported once, then removed; its transactions stay in the index only until it is rebuilt |
|
||||
|
||||
There is no key that tags a transaction. Tags come from `rules.toml` and
|
||||
nowhere else, so tagging what you are looking at means writing a rule for it on
|
||||
`4` — which is why that screen shows you what a glob catches before you save.
|
||||
**Transactions** reads like `3 rows · 0 new`: how many transactions the file
|
||||
holds, then how many of them it was the first to bring in. Statements that
|
||||
overlap share rows, and a shared row is stored once, under whichever file was
|
||||
imported first — so a later statement covering the same days shows fewer new
|
||||
rows than it holds, down to `0 new` when an earlier one already had them all.
|
||||
That is deduplication working, not a file that failed to parse.
|
||||
|
||||
### Report (`3`)
|
||||
Click a file name to open it. PDFs and CSVs open in the browser; anything else
|
||||
downloads. Only files import would read are listed or served — not
|
||||
`account.toml`, dotfiles, or anything outside an account's `include` patterns.
|
||||
|
||||
**Delete** removes a statement from disk for good — there is no copy kept
|
||||
and no undo, so keep the bank's original if you might want it back. Its
|
||||
transactions leave the index, and transfers that used them lose their pairing:
|
||||
the other leg shows as unpaired, which is the truth once one side is gone.
|
||||
Deleting does not import. A row that an overlapping statement also contains goes
|
||||
too, for now: the account's other statements are marked *changed*, and the next
|
||||
**Import** re-reads them and puts it back. For a file already gone from disk the
|
||||
button says **Forget** and only clears the index.
|
||||
|
||||
### Adding statements from the browser
|
||||
|
||||
The Accounts screen has an **Add statements** panel: pick the account,
|
||||
drop files on it (or click to choose them) and press **Upload**. The files are
|
||||
saved into that account's folder, exactly where you would have copied them by
|
||||
hand — the folder stays the source of truth, so deleting `index.db` and
|
||||
re-importing still gets everything back. Uploading does not import: the files
|
||||
wait on the statements list as *not imported* until you press **Import**, so you
|
||||
can put a batch together and look it over first.
|
||||
|
||||
An upload never replaces a statement. A file whose name is already in the folder
|
||||
is skipped if its contents are identical and refused if they differ; rename it
|
||||
or remove the old one first. Names import would not read back — dotfiles,
|
||||
`account.toml`, anything outside an account's `include` patterns — are refused
|
||||
too, and a batch with one bad file writes none of them. The folder itself must
|
||||
already exist with an `account.toml`: an upload adds statements to an account,
|
||||
it does not create one.
|
||||
|
||||
Some banks name their downloads unhelpfully, so a parser may name the statement
|
||||
instead — lowercase throughout, extension included — and the status line says
|
||||
what each file became. An NLB izpisek is saved as `izpisek_YYYY-MM-DD.pdf` after
|
||||
the *Datum izpiska* in its header, and a Trade Republic statement, which
|
||||
downloads as `document-N.pdf`, as `traderepublic_YYYY_MM_DD_YYYY_MM_DD.pdf`
|
||||
after the period on its first page (`DATE 01 May 2025 - 31 Jul 2026`). Either
|
||||
way the folder sorts by date. Downloading the same statement twice then lands on
|
||||
the same name and is recognised as already there; a different statement with the
|
||||
same name (a reissue) is numbered `_2`, `_3` rather than refused. A statement
|
||||
with no date it can find keeps its own name. Only uploads are named — files
|
||||
already in a folder are never renamed, since the index records them by path.
|
||||
|
||||
### Navigating
|
||||
|
||||
Every screen is a link in the bar along the top, and every action is a button or
|
||||
a click; there are no keyboard shortcuts. Clicking an account on the Accounts
|
||||
screen filters the transaction list to it.
|
||||
|
||||
There is no control that tags a transaction. Tags come from `rules.toml` and
|
||||
nowhere else, so tagging what you are looking at means writing a rule for it in
|
||||
the rule builder — which is why it shows you what a glob catches before you
|
||||
save.
|
||||
|
||||
### Report
|
||||
|
||||
The report opens on **last month**, not on everything you have ever imported:
|
||||
an all-time total is the one number a spending report is least often asked for,
|
||||
@@ -157,34 +249,22 @@ and the month that has just ended is the last one your statements can be
|
||||
complete for. If you have not downloaded that month yet, the screen says the
|
||||
period is empty rather than quietly showing you a different one.
|
||||
|
||||
`←` and `→` move along the time axis on the left of the totals, which runs from
|
||||
the widest window down to the oldest month in the index:
|
||||
|
||||
```
|
||||
Period Tag Cur Out In Net N
|
||||
all time groceries EUR 10.00 0.00 -10.00 1
|
||||
this year TOTAL EUR 10.00 0.00 -10.00
|
||||
last 12 months
|
||||
last 3 months
|
||||
this month
|
||||
▸ last month
|
||||
2026-05
|
||||
2026-04
|
||||
```
|
||||
The time axis beside the totals runs from the widest window down to the oldest
|
||||
month in the index — `all time`, `this year`, `last 12 months`, `last 3
|
||||
months`, `this month`, `last month`, then one entry per month. Click one to
|
||||
report on it.
|
||||
|
||||
The three rolling windows run to the end of *this* month rather than to the
|
||||
last complete one — you ask for "last 3 months" to see what is happening now,
|
||||
and leaving out the days since the 1st would answer a different question. Each
|
||||
month below `last month` is a month the index actually holds; months that a
|
||||
named window above already covers are not repeated. The axis is hidden on a
|
||||
terminal too narrow for it and the totals both, where the title still names the
|
||||
period.
|
||||
named window above already covers are not repeated.
|
||||
|
||||
`s` cycles how the rows are arranged, and the marked column heading says which
|
||||
one they are arranged by: `Out ▾` largest spend first (the default question a
|
||||
spending report answers), then `In ▾`, `Net ▾` lowest first so the biggest
|
||||
losses lead, `N ▾` most transactions first, and `Tag ▴` A→Z. `money report`
|
||||
takes the same choice as `--sort out|in|net|count|tag`.
|
||||
Click a column heading to change how the rows are arranged; the marked heading
|
||||
says which one they are arranged by: `Out ▾` largest spend first (the default
|
||||
question a spending report answers), then `In ▾`, `Net ▾` lowest first so the
|
||||
biggest losses lead, `N ▾` most transactions first, and `Tag ▴` A→Z. `money
|
||||
report` takes the same choice as `--sort out|in|net|count|tag`.
|
||||
|
||||
Currency is always the outer grouping and no sort changes that — there are no
|
||||
exchange rates here, so two currencies interleaved by amount would invite a
|
||||
@@ -192,20 +272,20 @@ comparison between numbers that cannot be compared. Rows that tie fall back to
|
||||
the tag, so they keep a fixed position rather than shuffling between reloads.
|
||||
The sort rearranges rows; it never changes which rows there are.
|
||||
|
||||
The period narrows the report and only the report. The account filter (`a`,
|
||||
`enter`), the search (`/`) and the untagged toggle (`u`) are shared with the
|
||||
transaction list as before, so opening on last month does not hide the rest of
|
||||
the index from `2`.
|
||||
The period narrows the report and only the report. The account, the search and
|
||||
the untagged toggle are shared with the transaction list, so opening on last
|
||||
month does not hide the rest of the index from the transaction list.
|
||||
|
||||
`money report` takes `--month` instead; there is no command-line equivalent of
|
||||
the wider windows.
|
||||
|
||||
### Rule builder (`4`)
|
||||
### Rule builder
|
||||
|
||||
Writing rules by hand means guessing what a glob will catch. This screen shows
|
||||
the answer as you type: the form is on the left, and on the right the
|
||||
still-untagged descriptions the glob currently matches, marked `▸`, grouped by
|
||||
description, under a running "*n* of *m* descriptions match" count.
|
||||
the answer as you type: the form is on the left — glob, account, tag, note —
|
||||
and on the right the still-untagged descriptions the glob currently matches,
|
||||
marked `▸`, grouped by description, under a running "*n* of *m* descriptions
|
||||
match" count.
|
||||
|
||||
The list narrows as you type, so what is on screen is what the rule would
|
||||
claim — nothing else is left there to read past. The count keeps the context
|
||||
@@ -214,60 +294,26 @@ glob picked two descriptions out of seven, and `0 of 7` says the glob is wrong.
|
||||
With the glob still empty there is nothing to filter by, and the list is every
|
||||
untagged description in the data — which is the other question this screen
|
||||
answers, and where you go looking for the next thing to write a rule for.
|
||||
Clicking a description fills the glob with `*THAT DESCRIPTION*`, as a starting
|
||||
point to narrow down.
|
||||
|
||||
```
|
||||
glob Untagged description ↓ N
|
||||
╭────────────────────────────╮ ▸ LIDL SOFIA 4412 2
|
||||
│ *LIDL* │ ▸ LIDL VARNA 9911 1
|
||||
╰────────────────────────────╯
|
||||
vs. the description
|
||||
`enter` (or **Save rule**) appends the rule to `rules.toml` and retags
|
||||
immediately, so the rows it caught disappear from the list. The account stays
|
||||
filled in, since the next rule is usually for the same one.
|
||||
|
||||
account
|
||||
╭────────────────────────────╮
|
||||
│ blank = every account │
|
||||
╰────────────────────────────╯
|
||||
blank = all accounts
|
||||
Clicking the column headings switches the list between by name and by count,
|
||||
most seen first; the `↓` says which column the order is read from. The two
|
||||
answer different questions: by name finds the payee you are looking at, by count
|
||||
finds the rule worth writing next, since one pattern claiming forty rows is
|
||||
worth more than the first of forty claiming one. Ties keep the alphabetical
|
||||
order, so the list does not reshuffle under you, and the choice lasts until you
|
||||
change it.
|
||||
|
||||
▸ tag
|
||||
╭────────────────────────────╮
|
||||
│ groceries │
|
||||
╰────────────────────────────╯
|
||||
tab completes · ctrl+n: 1 more
|
||||
|
||||
note
|
||||
╭────────────────────────────╮
|
||||
│ optional │
|
||||
╰────────────────────────────╯
|
||||
why this rule exists
|
||||
|
||||
2 of 7 descriptions match
|
||||
```
|
||||
|
||||
Only `gro` was typed in the tag field; `ceries` is the ghosted completion.
|
||||
|
||||
`tab` / `↑↓` move between the glob, account, tag and note fields, `pgup` /
|
||||
`pgdn` scroll the list, and `enter` appends the rule to `rules.toml` and retags
|
||||
immediately, so the rows it caught disappear from the list. `esc` goes back.
|
||||
On a short window the form gives up its spacing and then its hints, so all four
|
||||
fields stay on screen.
|
||||
|
||||
`ctrl+s` switches the list between by name and by count, most seen first — a
|
||||
chord rather than a letter, because every printable key belongs to the field you
|
||||
are typing in. The `↓` in the header says which column the order is read from.
|
||||
The two answer different questions: by name finds the payee you are looking at,
|
||||
by count finds the rule worth writing next, since one pattern claiming forty
|
||||
rows is worth more than the first of forty claiming one. Ties keep the
|
||||
alphabetical order, so the list does not reshuffle under you, and the choice
|
||||
lasts until you change it.
|
||||
|
||||
The account and tag fields complete as you type: the rest of the match is
|
||||
ghosted in grey after the cursor, and `tab` (or `→` at the end of the line)
|
||||
takes it. When several candidates share the prefix, the hint under the box says
|
||||
how many, and `ctrl+n` / `ctrl+p` cycle through them. Nothing is committed until
|
||||
you accept it, so a new tag is still just typed out in full. Accounts come from
|
||||
The account and tag fields offer completions as you type. Accounts come from
|
||||
the folders on disk and the index; tags from every tag in use plus any named in
|
||||
`rules.toml`, so a tag is completable from the moment a rule mentions it —
|
||||
which is what stops `groceries` acquiring a `grocery` twin.
|
||||
which is what stops `groceries` acquiring a `grocery` twin. A new tag is just
|
||||
typed out in full.
|
||||
|
||||
Leaving the account blank applies the rule everywhere; filling it in also
|
||||
narrows the preview to that account. The preview lists only transactions no
|
||||
@@ -281,8 +327,8 @@ report leaves them out anyway.
|
||||
|
||||
#### Editing a rule
|
||||
|
||||
`e` on the rules screen (`5`) opens the selected rule in this same form with the
|
||||
fields filled in, and `enter` rewrites that rule where it sits instead of
|
||||
**Edit** on the rules screen opens the rule in this same form with the
|
||||
fields filled in, and saving rewrites that rule where it sits instead of
|
||||
appending a new one. It keeps its position, since position still breaks ties
|
||||
between equally specific rules; `rules.toml` is edited textually, so the
|
||||
comments and formatting around it survive, exactly as when deleting.
|
||||
@@ -297,17 +343,17 @@ you are about to hand back to whatever rule catches it next.
|
||||
|
||||
The form has no `type` field, so a rule that sets one carries it through
|
||||
unchanged rather than losing it; it is shown under the glob as `+ type:… ·
|
||||
kept`. Saving lands back on the rules screen with the new counts, and `esc`
|
||||
leaves the rule as it was.
|
||||
kept`. Saving lands back on the rules screen with the new counts; **Cancel**, or
|
||||
leaving the screen leaves the rule as it was.
|
||||
|
||||
### Rules (`5`)
|
||||
### Rules
|
||||
|
||||
Every rule in file order — the order they are written in, not the order they
|
||||
are tried in — with the number of transactions it actually claims. Rules that
|
||||
claim none are marked `✗`.
|
||||
|
||||
```
|
||||
money · rules · 5 rules · 2 match nothing
|
||||
Rules · 5 rules · 2 match nothing
|
||||
|
||||
# Pattern Account Tag Txns Note
|
||||
1 *LIDL SOFIA* (all) groceries 3 the weekly shop
|
||||
@@ -326,75 +372,39 @@ deleting. Note that shadowing has nothing to do with the numbering: rule 2 would
|
||||
be just as dead written above rule 1, because precedence is decided by how
|
||||
specific a rule is and not by where it sits.
|
||||
|
||||
`e` opens the selected rule in the builder to edit it, `d` deletes it, and `p`
|
||||
deletes every rule marked `✗` at once; both deletions ask for a `y` first. `r`
|
||||
recounts against what is currently in the index, which is what you want after an
|
||||
import has added rows; `rules.toml` itself is read at startup and whenever you
|
||||
save a rule from the builder, so an edit made in another window needs a restart.
|
||||
Editing and deleting both work on `rules.toml` textually, so your comments,
|
||||
ordering and formatting survive. A comment sitting directly above a deleted rule
|
||||
goes with it, while one separated by a blank line is treated as a section
|
||||
heading and left alone; an edited rule keeps its comments, since they say why it
|
||||
is there and changing its glob rarely changes that.
|
||||
**Edit** opens a rule in the builder, **Delete** deletes it, and **Prune
|
||||
unused** deletes every rule marked `✗` at once; both deletions ask first.
|
||||
**Refresh counts** recounts against what is currently in the index. Editing and
|
||||
deleting both work on `rules.toml` textually, so your comments, ordering and
|
||||
formatting survive. A comment sitting directly above a deleted rule goes with
|
||||
it, while one separated by a blank line is treated as a section heading and
|
||||
left alone; an edited rule keeps its comments, since they say why it is there
|
||||
and changing its glob rarely changes that.
|
||||
|
||||
### Transfer builder (`6`)
|
||||
### Transfer builder
|
||||
|
||||
Same idea as the rule builder, for money moved between your own accounts. Both
|
||||
sides are named, and the preview on the right shows what the pair would be:
|
||||
`▸` for a movement it matches end to end, `⚠` for a leg it catches on one side
|
||||
and cannot pair with anything on the other.
|
||||
Same idea as the rule builder, for money moved between your own accounts. The
|
||||
form names both sides — from account, from desc, to account, to desc — plus an
|
||||
optional tolerance and note, and the preview beside it shows what the pair
|
||||
would be: `▸` for a movement it matches end to end, `⚠` for a leg it catches on
|
||||
one side and cannot pair with anything on the other.
|
||||
|
||||
```
|
||||
from account Date Amount Movement Description
|
||||
╭────────────────────────────╮ ⚠ 2026-04-01 500.00 checking → ? TRANSFER TO SAVINGS
|
||||
│ checking │ ▸ 2026-03-01 500.00 checking → savings TRANSFER TO SAVINGS
|
||||
╰────────────────────────────╯
|
||||
money leaves here
|
||||
|
||||
▸ from desc
|
||||
╭────────────────────────────╮
|
||||
│ *TO SAVINGS* │
|
||||
╰────────────────────────────╯
|
||||
glob vs. the leaving leg
|
||||
|
||||
to account
|
||||
╭────────────────────────────╮
|
||||
│ savings │
|
||||
╰────────────────────────────╯
|
||||
money arrives here
|
||||
|
||||
to desc
|
||||
╭────────────────────────────╮
|
||||
│ *FROM CHECKING* │
|
||||
╰────────────────────────────╯
|
||||
glob vs. the arriving leg
|
||||
|
||||
tolerance %
|
||||
╭────────────────────────────╮
|
||||
│ 0 │
|
||||
╰────────────────────────────╯
|
||||
0 = amounts must match exactly
|
||||
|
||||
note
|
||||
╭────────────────────────────╮
|
||||
│ optional │
|
||||
╰────────────────────────────╯
|
||||
why this transfer exists
|
||||
Date Amount Movement Description
|
||||
⚠ 2026-04-01 500.00 checking → ? TRANSFER TO SAVINGS
|
||||
▸ 2026-03-01 500.00 checking → savings TRANSFER TO SAVINGS
|
||||
|
||||
1 pairs · 1 unpaired
|
||||
```
|
||||
|
||||
The account fields complete exactly as the rule builder's do. `tab` / `↑↓` move
|
||||
between fields, `pgup` / `pgdn` scroll the list, `enter` appends the definition
|
||||
to `rules.toml` and re-pairs immediately, and `esc` goes back. Saving keeps both
|
||||
account names in place and clears the two globs, since the next transfer you
|
||||
write is usually the same route in the other direction. The tolerance is cleared
|
||||
with them: carried over silently it would loosen a route that never asked for
|
||||
one. Six fields need more room than the rule builder's four, so on a short
|
||||
window this form gives up its spacing, then its hints, then the borders on the
|
||||
fields you are not editing — every field stays on screen.
|
||||
The account fields complete exactly as the rule builder's do. `enter` (or
|
||||
**Save transfer**) appends the definition to `rules.toml` and re-pairs
|
||||
immediately. Saving keeps both account names in place and clears the two globs,
|
||||
since the next transfer you write is usually the same route in the other
|
||||
direction. The tolerance is cleared with them: carried over silently it would
|
||||
loosen a route that never asked for one.
|
||||
|
||||
`tolerance %` is the one field worth previewing before you save. Type a
|
||||
`Tolerance %` is the one field worth previewing before you save. Type a
|
||||
percentage and the pairs it buys appear immediately, showing both amounts —
|
||||
`500.00 → 495.00` — with what they cost summarised beside the counts:
|
||||
|
||||
@@ -406,12 +416,12 @@ A half-written definition previews too: fill in one side and its legs show up as
|
||||
unpaired, which is the quickest way to see that a glob is wrong before you have
|
||||
written the other half.
|
||||
|
||||
### Transfers (`7`)
|
||||
### Transfers
|
||||
|
||||
Every definition in file order, with what it currently pairs.
|
||||
|
||||
```
|
||||
money · transfers · 4 definitions · 1 leg(s) unpaired
|
||||
Transfers · 4 definitions · 1 leg(s) unpaired
|
||||
|
||||
# From To Pairs Unpaired Tol Note
|
||||
1 ⚠ checking *TO SAVINGS* savings *FROM CHECKING* 11 1 monthly saving
|
||||
@@ -425,56 +435,53 @@ definition catching legs it cannot complete: a wrong glob on the other side, a
|
||||
statement not imported yet, or a movement that genuinely went missing. `✗` is a
|
||||
definition matching nothing at all, which is just dead weight.
|
||||
|
||||
Click a `⚠` row to see the legs behind it, newest first. Each one says which
|
||||
side is missing and what that side would have had to look like:
|
||||
|
||||
```
|
||||
Date Amount Movement Missing Would pair with
|
||||
2026-04-01 500.00 checking → ? arriving leg +500.00 in savings matching *FROM CHECKING*,
|
||||
dated 2026-03-27 to 2026-04-06
|
||||
```
|
||||
|
||||
That is the search to run on the other statement. If it is not there, the
|
||||
statement probably is not imported yet — when nothing at all from that account
|
||||
is, the line says so. If something close is there, the difference tells you
|
||||
which condition failed: a date outside the window, an amount off by a fee (see
|
||||
[When the bank takes a fee](#when-the-bank-takes-a-fee)), or a description the
|
||||
glob does not match. Across currencies the line says *any amount*, since only
|
||||
the dates are checked there.
|
||||
|
||||
`Tol` is the definition's `tolerance_pct`, and it qualifies the counts beside
|
||||
it: row 3's two pairs were matched on slack rather than on the amount agreeing.
|
||||
It is blank for every definition that requires the exact amount — which is the
|
||||
default, so an empty column means nothing here is pairing on a mismatch.
|
||||
|
||||
`d` deletes the selected definition and `p` deletes every `✗` one at once, both
|
||||
after a `y`. Definitions marked `⚠` are never pruned — they are doing something,
|
||||
just not finishing it, and deleting one would hide the problem rather than fix
|
||||
it. `r` re-pairs against what is currently in the index.
|
||||
**Delete** deletes a definition and **Prune unmatched** deletes every `✗` one at
|
||||
once, both after asking. Definitions marked `⚠` are never pruned — they are
|
||||
doing something, just not finishing it, and deleting one would hide the problem
|
||||
rather than fix it. **Refresh pairing** re-reads what is currently in the index.
|
||||
|
||||
## Web app
|
||||
### Tags
|
||||
|
||||
`money serve` puts the TUI's seven screens in a browser — accounts,
|
||||
transactions, report, rule builder, rules, transfer builder, transfers — with
|
||||
the same behaviour, because it runs the same code: amounts are formatted, globs
|
||||
matched and transfers paired on the server, and the page only shows the
|
||||
answers. It is one binary with the page built in; nothing else to deploy.
|
||||
Every distinct tag, A→Z: the ones your transactions carry and any `rules.toml`
|
||||
names, with how many transactions carry each and the rules that write it.
|
||||
|
||||
```
|
||||
money serve # http://127.0.0.1:8080
|
||||
money --root /srv/money serve --addr 0.0.0.0:8080
|
||||
Tags · 2 tags
|
||||
|
||||
Tag Txns Rules
|
||||
clothes 0 #4 *ZARA* (0)
|
||||
groceries 2 #1 *LIDL* (0) #2 *KAUFLAND* (1) #3 *LIDL SOFIA* (1)
|
||||
```
|
||||
|
||||
**There is no authentication.** Anyone who can reach the port can read every
|
||||
transaction, rewrite `rules.toml` and start an import, which is why it listens
|
||||
on loopback unless `--addr` says otherwise. To use it from elsewhere, put it
|
||||
behind a reverse proxy that does the logging in, or reach it over SSH / a VPN.
|
||||
Requests that change anything must be sent as JSON, so another website open in
|
||||
the same browser cannot post a form to it.
|
||||
|
||||
What differs from the TUI:
|
||||
|
||||
- **Retag and Import re-read `rules.toml`** (and Import the account folders)
|
||||
before running, exactly as a fresh `money retag` / `money import` would — the
|
||||
server outlives hand edits to both. Until then a banner says the file on disk
|
||||
no longer matches what the index was derived from.
|
||||
- **Edits and deletes check the file first.** They go by rule position, as in
|
||||
the TUI, but the page also sends the rule it showed you; if `rules.toml` no
|
||||
longer holds that rule there (another tab, a hand edit) the change is refused
|
||||
and you are asked to reload, rather than editing whichever rule moved into
|
||||
its place.
|
||||
- Clicking a description in the rule builder's preview fills the glob with
|
||||
`*THAT DESCRIPTION*`, as a starting point to narrow down.
|
||||
- Shift-click **Import** for `import --force`.
|
||||
|
||||
The keys still work where they do not fight the browser: `1`–`7` switch
|
||||
screens, `/` searches, `u` toggles untagged, `a` clears the account filter, `i`
|
||||
imports and `r` retags. On the report `←`/`→` move the period and `s` cycles
|
||||
the sort (or click a column heading); in the rule builder `ctrl+s` re-sorts the
|
||||
preview. Inside a form every printable key belongs to the field, as in the TUI.
|
||||
Rules are numbered by their position in `rules.toml`, as on the rules screen,
|
||||
and the count after each is how many transactions it wins — so a tag's rules
|
||||
add up to its transactions, and a rule at `(0)` is matching nothing or being
|
||||
beaten by a more specific one. It is the place to spot near-duplicate tags
|
||||
(`grocery` beside `groceries`) and tags that several rules feed. A tag no rule
|
||||
names any more — `rules.toml` was edited since the last retag — says so; the
|
||||
next Retag clears it. The transfer label is not a tag and never appears here.
|
||||
|
||||
## rules.toml
|
||||
|
||||
@@ -531,7 +538,7 @@ tag = "groceries"
|
||||
Every tag comes from this file, so a tag is never something you have to keep
|
||||
safe: `money retag` recomputes all of them from the current rules and is safe
|
||||
to run whenever you change it. A transaction no rule matches simply stays
|
||||
untagged, and shows up under `money ls --untagged`, the `u` view, and
|
||||
untagged, and shows up under `money ls --untagged`, **Untagged only**, and
|
||||
`(untagged)` in the report — unless it is one leg of a matched transfer, which
|
||||
is already spoken for and is left out of all three.
|
||||
|
||||
@@ -597,7 +604,7 @@ amount wins.
|
||||
|
||||
**The difference is not forgiven, it is reported.** A pair leaves the report
|
||||
entirely, so a fee hidden inside one would be spending that appears nowhere at
|
||||
all. `money report` names it on the excluded line and the TUI's report gives it
|
||||
all. `money report` names it on the excluded line and the web app's report gives it
|
||||
its own row (see below). Across currencies there is no fee to compute — the two
|
||||
numbers are in different units — so a tolerance there means nothing and is
|
||||
ignored.
|
||||
@@ -633,7 +640,7 @@ that is the number your spending is short by:
|
||||
transfers excluded: 2 legs in EUR, 500.00 out, 495.00 in, 5.00 in fees
|
||||
```
|
||||
|
||||
The report screen (`3`) shows the same thing under its `TOTAL`, the fee on its
|
||||
The report screen shows the same thing under its `TOTAL`, the fee on its
|
||||
own row (for whichever period it is on):
|
||||
|
||||
```
|
||||
@@ -659,7 +666,7 @@ An *unpaired* leg is not a transfer and still counts, tagged like anything
|
||||
else. That is deliberate: money that left an account and cannot be shown to
|
||||
have arrived is exactly what you want to see, not something to quietly drop.
|
||||
`money import` and `money retag` report unpaired legs on stderr, and the
|
||||
transfers screen (`7`) shows which definition they belong to.
|
||||
transfers screen shows which definition they belong to.
|
||||
|
||||
The pairing is derived state, like the tags: it lives in the index, is
|
||||
recomputed wholesale by `money retag` and after every import, and disappears
|
||||
@@ -685,8 +692,8 @@ nothing else to configure — `parser` names one of these and that is all.
|
||||
|
||||
| `parser` | Statement | Notes |
|
||||
| --- | --- | --- |
|
||||
| `nlb` | NLB izpisek PDF | Wrapped descriptions are folded in from continuation lines, and the IBAN column is appended to the description. |
|
||||
| `traderepublic` | Trade Republic PDF | Handles both the single-line and the stacked layout by measuring column positions. |
|
||||
| `nlb` | NLB izpisek PDF | Wrapped descriptions are folded in from continuation lines, and the IBAN column is appended to the description. Uploads are saved as `izpisek_YYYY-MM-DD.pdf` after the statement date. |
|
||||
| `traderepublic` | Trade Republic PDF | Handles both the single-line and the stacked layout by measuring column positions. Uploads are saved as `traderepublic_YYYY_MM_DD_YYYY_MM_DD.pdf` after the statement period. |
|
||||
| `revolut` | `account-statement*.csv` | Skips non-COMPLETED rows, folds the fee into the amount. |
|
||||
|
||||
```toml
|
||||
|
||||
+6
-18
@@ -2,7 +2,7 @@
|
||||
//
|
||||
// The data root holds one folder per account. Drop statements into those
|
||||
// folders, run `money import`, and tag what comes out with glob rules in
|
||||
// rules.toml. Running it with no subcommand opens the TUI.
|
||||
// rules.toml. Running it with no subcommand serves the web app.
|
||||
package main
|
||||
|
||||
import (
|
||||
@@ -28,7 +28,6 @@ import (
|
||||
"git.petrovv.com/nikola/money/internal/rules"
|
||||
"git.petrovv.com/nikola/money/internal/store"
|
||||
"git.petrovv.com/nikola/money/internal/transfers"
|
||||
"git.petrovv.com/nikola/money/internal/tui"
|
||||
"git.petrovv.com/nikola/money/internal/web"
|
||||
)
|
||||
|
||||
@@ -37,9 +36,9 @@ const usage = `money - statement-driven personal finance tracker
|
||||
usage: money [--root DIR] <command> [flags] (--root may also follow the command)
|
||||
|
||||
commands:
|
||||
tui browse transactions, build the rules that tag them and the
|
||||
transfers that pair them across accounts (default)
|
||||
serve the same screens as a web app (--addr, default 127.0.0.1:8080)
|
||||
serve browse transactions, build the rules that tag them and the
|
||||
transfers that pair them across accounts, in a browser
|
||||
(--addr, default 127.0.0.1:8080; the default command)
|
||||
import extract transactions from every statement into the index
|
||||
retag re-apply rules.toml: retag everything and re-pair transfers
|
||||
ls list transactions
|
||||
@@ -83,7 +82,7 @@ func run(args []string) error {
|
||||
}
|
||||
}
|
||||
|
||||
cmd := "tui"
|
||||
cmd := "serve"
|
||||
if len(args) > 0 {
|
||||
cmd, args = args[0], args[1:]
|
||||
}
|
||||
@@ -120,8 +119,6 @@ func run(args []string) error {
|
||||
case "parsers":
|
||||
fmt.Println(strings.Join(parser.Names(), "\n"))
|
||||
return nil
|
||||
case "tui":
|
||||
return cmdTUI(abs, args)
|
||||
case "serve":
|
||||
return cmdServe(abs, args)
|
||||
case "help", "-h", "--help":
|
||||
@@ -488,15 +485,6 @@ func cmdAccounts(root string, _ []string) error {
|
||||
return w.Flush()
|
||||
}
|
||||
|
||||
func cmdTUI(root string, _ []string) error {
|
||||
o, err := open(root)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer o.db.Close()
|
||||
return tui.Run(root, o.db, o.accounts, o.engine, o.links)
|
||||
}
|
||||
|
||||
// takeRoot removes --root/-root (as `--root DIR` or `--root=DIR`) from a
|
||||
// subcommand's arguments, returning the last value given. It stops at `--`,
|
||||
// after which everything is positional.
|
||||
@@ -529,7 +517,7 @@ func takeRoot(args []string) (root string, rest []string, err error) {
|
||||
return root, rest, nil
|
||||
}
|
||||
|
||||
// cmdServe serves the TUI's screens over HTTP until interrupted. There is no
|
||||
// cmdServe serves the web app until interrupted. There is no
|
||||
// authentication, so it listens on loopback unless told otherwise: anyone who
|
||||
// can reach the port can rewrite rules.toml and start an import.
|
||||
func cmdServe(root string, args []string) error {
|
||||
|
||||
@@ -4,38 +4,16 @@ go 1.25.6
|
||||
|
||||
require (
|
||||
github.com/BurntSushi/toml v1.6.0
|
||||
github.com/charmbracelet/bubbles v1.0.0
|
||||
github.com/charmbracelet/bubbletea v1.3.10
|
||||
github.com/charmbracelet/lipgloss v1.1.0
|
||||
modernc.org/sqlite v1.56.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/atotto/clipboard v0.1.4 // indirect
|
||||
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
|
||||
github.com/charmbracelet/colorprofile v0.4.1 // indirect
|
||||
github.com/charmbracelet/x/ansi v0.11.6 // indirect
|
||||
github.com/charmbracelet/x/cellbuf v0.0.15 // indirect
|
||||
github.com/charmbracelet/x/term v0.2.2 // indirect
|
||||
github.com/clipperhouse/displaywidth v0.9.0 // indirect
|
||||
github.com/clipperhouse/stringish v0.1.1 // indirect
|
||||
github.com/clipperhouse/uax29/v2 v2.5.0 // indirect
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/lucasb-eyer/go-colorful v1.3.0 // indirect
|
||||
github.com/mattn/go-isatty v0.0.24 // indirect
|
||||
github.com/mattn/go-localereader v0.0.1 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.19 // indirect
|
||||
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
|
||||
github.com/muesli/cancelreader v0.2.2 // indirect
|
||||
github.com/muesli/termenv v0.16.0 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
github.com/rivo/uniseg v0.4.7 // indirect
|
||||
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
|
||||
golang.org/x/sys v0.47.0 // indirect
|
||||
golang.org/x/text v0.3.8 // indirect
|
||||
modernc.org/libc v1.74.4 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
modernc.org/memory v1.11.0 // indirect
|
||||
|
||||
@@ -1,76 +1,25 @@
|
||||
github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk=
|
||||
github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
|
||||
github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4=
|
||||
github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
|
||||
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
|
||||
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
|
||||
github.com/aymanbagabas/go-udiff v0.3.1 h1:LV+qyBQ2pqe0u42ZsUEtPiCaUoqgA9gYRDs3vj1nolY=
|
||||
github.com/aymanbagabas/go-udiff v0.3.1/go.mod h1:G0fsKmG+P6ylD0r6N/KgQD/nWzgfnl8ZBcNLgcbrw8E=
|
||||
github.com/charmbracelet/bubbles v1.0.0 h1:12J8/ak/uCZEMQ6KU7pcfwceyjLlWsDLAxB5fXonfvc=
|
||||
github.com/charmbracelet/bubbles v1.0.0/go.mod h1:9d/Zd5GdnauMI5ivUIVisuEm3ave1XwXtD1ckyV6r3E=
|
||||
github.com/charmbracelet/bubbletea v1.3.10 h1:otUDHWMMzQSB0Pkc87rm691KZ3SWa4KUlvF9nRvCICw=
|
||||
github.com/charmbracelet/bubbletea v1.3.10/go.mod h1:ORQfo0fk8U+po9VaNvnV95UPWA1BitP1E0N6xJPlHr4=
|
||||
github.com/charmbracelet/colorprofile v0.4.1 h1:a1lO03qTrSIRaK8c3JRxJDZOvhvIeSco3ej+ngLk1kk=
|
||||
github.com/charmbracelet/colorprofile v0.4.1/go.mod h1:U1d9Dljmdf9DLegaJ0nGZNJvoXAhayhmidOdcBwAvKk=
|
||||
github.com/charmbracelet/lipgloss v1.1.0 h1:vYXsiLHVkK7fp74RkV7b2kq9+zDLoEU4MZoFqR/noCY=
|
||||
github.com/charmbracelet/lipgloss v1.1.0/go.mod h1:/6Q8FR2o+kj8rz4Dq0zQc3vYf7X+B0binUUBwA0aL30=
|
||||
github.com/charmbracelet/x/ansi v0.11.6 h1:GhV21SiDz/45W9AnV2R61xZMRri5NlLnl6CVF7ihZW8=
|
||||
github.com/charmbracelet/x/ansi v0.11.6/go.mod h1:2JNYLgQUsyqaiLovhU2Rv/pb8r6ydXKS3NIttu3VGZQ=
|
||||
github.com/charmbracelet/x/cellbuf v0.0.15 h1:ur3pZy0o6z/R7EylET877CBxaiE1Sp1GMxoFPAIztPI=
|
||||
github.com/charmbracelet/x/cellbuf v0.0.15/go.mod h1:J1YVbR7MUuEGIFPCaaZ96KDl5NoS0DAWkskup+mOY+Q=
|
||||
github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91 h1:payRxjMjKgx2PaCWLZ4p3ro9y97+TVLZNaRZgJwSVDQ=
|
||||
github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91/go.mod h1:wDlXFlCrmJ8J+swcL/MnGUuYnqgQdW9rhSD61oNMb6U=
|
||||
github.com/charmbracelet/x/term v0.2.2 h1:xVRT/S2ZcKdhhOuSP4t5cLi5o+JxklsoEObBSgfgZRk=
|
||||
github.com/charmbracelet/x/term v0.2.2/go.mod h1:kF8CY5RddLWrsgVwpw4kAa6TESp6EB5y3uxGLeCqzAI=
|
||||
github.com/clipperhouse/displaywidth v0.9.0 h1:Qb4KOhYwRiN3viMv1v/3cTBlz3AcAZX3+y9OLhMtAtA=
|
||||
github.com/clipperhouse/displaywidth v0.9.0/go.mod h1:aCAAqTlh4GIVkhQnJpbL0T/WfcrJXHcj8C0yjYcjOZA=
|
||||
github.com/clipperhouse/stringish v0.1.1 h1:+NSqMOr3GR6k1FdRhhnXrLfztGzuG+VuFDfatpWHKCs=
|
||||
github.com/clipperhouse/stringish v0.1.1/go.mod h1:v/WhFtE1q0ovMta2+m+UbpZ+2/HEXNWYXQgCt4hdOzA=
|
||||
github.com/clipperhouse/uax29/v2 v2.5.0 h1:x7T0T4eTHDONxFJsL94uKNKPHrclyFI0lm7+w94cO8U=
|
||||
github.com/clipperhouse/uax29/v2 v2.5.0/go.mod h1:Wn1g7MK6OoeDT0vL+Q0SQLDz/KpfsVRgg6W7ihQeh4g=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6baUTXGLOoWe4PQhGxaX0KpnayAqC48p4=
|
||||
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM=
|
||||
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
|
||||
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
|
||||
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
|
||||
github.com/lucasb-eyer/go-colorful v1.3.0 h1:2/yBRLdWBZKrf7gB40FoiKfAWYQ0lqNcbuQwVHXptag=
|
||||
github.com/lucasb-eyer/go-colorful v1.3.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
|
||||
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
|
||||
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
|
||||
github.com/mattn/go-localereader v0.0.1 h1:ygSAOl7ZXTx4RdPYinUpg6W99U8jWvWi9Ye2JC/oIi4=
|
||||
github.com/mattn/go-localereader v0.0.1/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88=
|
||||
github.com/mattn/go-runewidth v0.0.19 h1:v++JhqYnZuu5jSKrk9RbgF5v4CGUjqRfBm05byFGLdw=
|
||||
github.com/mattn/go-runewidth v0.0.19/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
|
||||
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 h1:ZK8zHtRHOkbHy6Mmr5D264iyp3TiX5OmNcI5cIARiQI=
|
||||
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6/go.mod h1:CJlz5H+gyd6CUWT45Oy4q24RdLyn7Md9Vj2/ldJBSIo=
|
||||
github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELUXHmA=
|
||||
github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo=
|
||||
github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc=
|
||||
github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
|
||||
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
|
||||
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
|
||||
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
|
||||
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
|
||||
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
|
||||
golang.org/x/exp v0.0.0-20231006140011-7918f672742d h1:jtJma62tbqLibJ5sFQz8bKtEM8rJBtfilJ2qTU199MI=
|
||||
golang.org/x/exp v0.0.0-20231006140011-7918f672742d/go.mod h1:ldy0pHrwJyGW56pPQzzkH36rKxoZW1tw7ZJpeKx+hdo=
|
||||
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
|
||||
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
|
||||
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
|
||||
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
|
||||
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
golang.org/x/text v0.3.8 h1:nAL+RVCQ9uMn3vJZbV+MRnydTJFPf8qqY42YiA6MrqY=
|
||||
golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
|
||||
golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
|
||||
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
|
||||
modernc.org/cc/v4 v4.29.1 h1:MKgdCV3WykTSPqpVrnxdEDS0HEd2FHpKZDzxzU5LyeI=
|
||||
|
||||
@@ -101,7 +101,7 @@ func Run(root string, db *store.DB, accounts []*config.Account, engine *rules.En
|
||||
continue
|
||||
}
|
||||
|
||||
files, err := statementFiles(acc)
|
||||
files, err := StatementFiles(acc)
|
||||
if err != nil {
|
||||
return res, err
|
||||
}
|
||||
@@ -134,7 +134,7 @@ func importFile(root string, db *store.DB, acc *config.Account, accountID int64,
|
||||
}
|
||||
fr := FileResult{Account: acc.Slug, Path: rel}
|
||||
|
||||
sum, err := checksum(path)
|
||||
sum, err := Checksum(path)
|
||||
if err != nil {
|
||||
fr.Err = err
|
||||
return fr
|
||||
@@ -162,7 +162,7 @@ func importFile(root string, db *store.DB, acc *config.Account, accountID int64,
|
||||
}
|
||||
fr.Warnings = append(fr.Warnings, checkBalances(txns, acc.Digits())...)
|
||||
|
||||
sourceID, err := db.SourceFile(accountID, rel, sum, time.Now().UTC().Format(time.RFC3339))
|
||||
sourceID, err := db.SourceFile(accountID, rel, sum, time.Now().UTC().Format(time.RFC3339), len(txns))
|
||||
if err != nil {
|
||||
fr.Err = err
|
||||
return fr
|
||||
@@ -252,10 +252,11 @@ func fingerprint(key string, ordinal int) string {
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// statementFiles lists the files in an account folder that should be parsed:
|
||||
// StatementFiles lists the files in an account folder that should be parsed:
|
||||
// every regular file except account.toml and dotfiles, narrowed by the
|
||||
// account's optional include globs.
|
||||
func statementFiles(acc *config.Account) ([]string, error) {
|
||||
// account's optional include globs. It is also what the web app lists and
|
||||
// serves, so a file is shown exactly when import would read it.
|
||||
func StatementFiles(acc *config.Account) ([]string, error) {
|
||||
entries, err := os.ReadDir(acc.Dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read account dir %s: %w", acc.Dir, err)
|
||||
@@ -284,7 +285,9 @@ func matchAny(patterns []string, name string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
func checksum(path string) (string, error) {
|
||||
// Checksum is the sha256 a statement is recorded under; an import skips a file
|
||||
// whose checksum has not changed.
|
||||
func Checksum(path string) (string, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return "", err
|
||||
|
||||
@@ -43,6 +43,10 @@ var (
|
||||
nlbAccount = regexp.MustCompile(`^SI\d{2}(?:\s?\d{4}){3}\s?\d{3}$`)
|
||||
)
|
||||
|
||||
// nlbStatementDate is the issue date in an izpisek's header. It names the
|
||||
// statement: NLB's downloads are not named for what they hold.
|
||||
var nlbStatementDate = regexp.MustCompile(`Datum izpiska\s+(\d{2})\.(\d{2})\.(\d{4})`)
|
||||
|
||||
// nlbMinContinuationIndent is the shallowest indent a wrapped description line
|
||||
// may have. The real threshold is the description column of the transaction
|
||||
// the line belongs to, measured per line rather than hardcoded: pdftotext
|
||||
@@ -61,6 +65,24 @@ func (p *nlbParser) Parse(path string, acc *config.Account) ([]RawTxn, error) {
|
||||
return parseNLBText(text, p.digits)
|
||||
}
|
||||
|
||||
// StatementName names an izpisek after its statement date, izpisek_YYYY-MM-DD,
|
||||
// so the folder sorts by date and two downloads of one statement collide.
|
||||
func (p *nlbParser) StatementName(path string) (string, error) {
|
||||
text, err := pdfToText(path)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return nlbStatementName(text), nil
|
||||
}
|
||||
|
||||
func nlbStatementName(text string) string {
|
||||
m := nlbStatementDate.FindStringSubmatch(text)
|
||||
if m == nil {
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf("izpisek_%s-%s-%s", m[3], m[2], m[1])
|
||||
}
|
||||
|
||||
// parseNLBText holds the whole parser, separated from PDF extraction so it can
|
||||
// be tested against captured pdftotext output.
|
||||
func parseNLBText(text string, digits int) ([]RawTxn, error) {
|
||||
|
||||
@@ -112,3 +112,15 @@ func TestParseNLBRejectsMalformedLine(t *testing.T) {
|
||||
t.Error("expected an error for a transaction line with no amount")
|
||||
}
|
||||
}
|
||||
|
||||
// An izpisek is named after its statement date, as rename_izpiski.py did.
|
||||
func TestNLBStatementName(t *testing.T) {
|
||||
header := " IZPISEK 001/2026\n" +
|
||||
" Datum izpiska 31.01.2026 Stran 1\n"
|
||||
if got := nlbStatementName(header + nlbPage); got != "izpisek_2026-01-31" {
|
||||
t.Errorf("name = %q, want izpisek_2026-01-31", got)
|
||||
}
|
||||
if got := nlbStatementName(nlbPage); got != "" {
|
||||
t.Errorf("name without a statement date = %q, want none", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -43,6 +43,17 @@ type Warner interface {
|
||||
Warnings() []string
|
||||
}
|
||||
|
||||
// Namer is an optional interface for parsers whose bank names its downloads
|
||||
// unhelpfully. StatementName reads the statement at path and returns the name,
|
||||
// without extension, it should be stored under — the upload's extension is
|
||||
// kept, lowercased — or "" when the statement does not say, in which case it
|
||||
// keeps the name it came with. Only an upload uses
|
||||
// it; files already in a folder are never renamed, since the index records
|
||||
// them by path.
|
||||
type Namer interface {
|
||||
StatementName(path string) (string, error)
|
||||
}
|
||||
|
||||
// Factory builds a parser from an account's config, validating it up front so
|
||||
// a bad account.toml fails before any file is read.
|
||||
type Factory func(acc *config.Account) (Parser, error)
|
||||
|
||||
@@ -46,6 +46,12 @@ const trAmountSlack = 5
|
||||
// trDateLayout is the date Trade Republic prints, e.g. "05 Jan 2026".
|
||||
const trDateLayout = "02 Jan 2006"
|
||||
|
||||
// trPeriod is the statement period in the address block at the top of the
|
||||
// first page, e.g. "DATE 01 May 2025 - 31 Jul 2026". It names the
|
||||
// statement: Trade Republic's downloads are all called document-N.pdf. The
|
||||
// transaction table's own DATE heading never matches, since no dates follow it.
|
||||
var trPeriod = regexp.MustCompile(`\bDATE\s+(\d{2} [A-Z][a-z]{2} \d{4})\s*[-\x{2013}\x{2212}]\s*(\d{2} [A-Z][a-z]{2} \d{4})`)
|
||||
|
||||
func (p *tradeRepublicParser) Parse(path string, acc *config.Account) ([]RawTxn, error) {
|
||||
text, err := pdfToText(path)
|
||||
if err != nil {
|
||||
@@ -54,6 +60,33 @@ func (p *tradeRepublicParser) Parse(path string, acc *config.Account) ([]RawTxn,
|
||||
return parseTradeRepublicText(text, p.digits)
|
||||
}
|
||||
|
||||
// StatementName names a statement after its period,
|
||||
// traderepublic_YYYY_MM_DD_YYYY_MM_DD, so the folder sorts by when it starts
|
||||
// and two downloads of one statement collide.
|
||||
func (p *tradeRepublicParser) StatementName(path string) (string, error) {
|
||||
text, err := pdfToText(path)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return tradeRepublicStatementName(text), nil
|
||||
}
|
||||
|
||||
func tradeRepublicStatementName(text string) string {
|
||||
m := trPeriod.FindStringSubmatch(text)
|
||||
if m == nil {
|
||||
return ""
|
||||
}
|
||||
from, err := time.Parse(trDateLayout, m[1])
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
to, err := time.Parse(trDateLayout, m[2])
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
return "traderepublic_" + from.Format("2006_01_02") + "_" + to.Format("2006_01_02")
|
||||
}
|
||||
|
||||
// parseTradeRepublicText holds the whole parser, separated from PDF extraction
|
||||
// so it can be tested against captured pdftotext output.
|
||||
func parseTradeRepublicText(text string, digits int) ([]RawTxn, error) {
|
||||
|
||||
@@ -162,3 +162,19 @@ func TestParseTradeRepublicIgnoresAmountsInDescription(t *testing.T) {
|
||||
t.Errorf("description = %q, want the inline figure kept", txns[0].Description)
|
||||
}
|
||||
}
|
||||
|
||||
// A statement is named after the period in its header; the transaction
|
||||
// table's DATE heading, with no dates after it, is not mistaken for it.
|
||||
func TestTradeRepublicStatementName(t *testing.T) {
|
||||
header := "NIKOLA PETROV DATE 01 May 2025 - 31 Jul 2026\n" +
|
||||
"Ulica 21 IBAN DE00000000000000000000\n\n" +
|
||||
"DATE TYPE DESCRIPTION MONEY IN MONEY OUT BALANCE\n"
|
||||
if got := tradeRepublicStatementName(header); got != "traderepublic_2025_05_01_2026_07_31" {
|
||||
t.Errorf("name = %q, want traderepublic_2025_05_01_2026_07_31", got)
|
||||
}
|
||||
table := "DATE TYPE DESCRIPTION MONEY IN MONEY OUT BALANCE\n" +
|
||||
"01 Jul 2026 Interest Interest payment €1.17 €671.58\n"
|
||||
if got := tradeRepublicStatementName(table); got != "" {
|
||||
t.Errorf("name without a period = %q, want none", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -28,7 +28,7 @@ func seed(t *testing.T, db *store.DB, descriptions ...string) int64 {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sourceID, err := db.SourceFile(accountID, "checking/st.csv", "sha", "2026-01-01T00:00:00Z")
|
||||
sourceID, err := db.SourceFile(accountID, "checking/st.csv", "sha", "2026-01-01T00:00:00Z", 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
+99
-7
@@ -37,6 +37,9 @@ CREATE TABLE IF NOT EXISTS source_files (
|
||||
path TEXT NOT NULL,
|
||||
sha256 TEXT NOT NULL,
|
||||
imported_at TEXT NOT NULL,
|
||||
-- How many transactions the statement holds, as parsed. Not how many it
|
||||
-- brought in: rows an overlapping statement already had are deduplicated.
|
||||
rows INTEGER NOT NULL DEFAULT 0,
|
||||
UNIQUE(account_id, path)
|
||||
);
|
||||
|
||||
@@ -130,15 +133,17 @@ func (d *DB) Accounts() ([]model.Account, error) {
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// SourceFile records that a statement file was imported, returning its id.
|
||||
func (d *DB) SourceFile(accountID int64, path, sha, importedAt string) (int64, error) {
|
||||
// SourceFile records that a statement file was imported, holding rows
|
||||
// transactions, and returns its id.
|
||||
func (d *DB) SourceFile(accountID int64, path, sha, importedAt string, rows int) (int64, error) {
|
||||
_, err := d.sql.Exec(`
|
||||
INSERT INTO source_files (account_id, path, sha256, imported_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
INSERT INTO source_files (account_id, path, sha256, imported_at, rows)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
ON CONFLICT(account_id, path) DO UPDATE SET
|
||||
sha256 = excluded.sha256,
|
||||
imported_at = excluded.imported_at`,
|
||||
accountID, path, sha, importedAt)
|
||||
imported_at = excluded.imported_at,
|
||||
rows = excluded.rows`,
|
||||
accountID, path, sha, importedAt, rows)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("record source file %s: %w", path, err)
|
||||
}
|
||||
@@ -393,7 +398,94 @@ func (d *DB) Months() ([]string, error) {
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// Tags lists every tag in use, for completion in the TUI.
|
||||
// SourceFileInfo is what the index recorded about one imported statement.
|
||||
type SourceFileInfo struct {
|
||||
AccountSlug string
|
||||
Path string // relative to the data root, as the importer records it
|
||||
SHA256 string
|
||||
ImportedAt string
|
||||
// Rows is how many transactions the statement holds, as parsed.
|
||||
Rows int
|
||||
// Added counts the transactions this file introduced. A row seen again in
|
||||
// an overlapping statement is deduplicated and stays with the file that
|
||||
// brought it first, so this is not how many rows the file holds.
|
||||
Added int
|
||||
}
|
||||
|
||||
// SourceFiles lists every statement the index has imported.
|
||||
func (d *DB) SourceFiles() ([]SourceFileInfo, error) {
|
||||
rows, err := d.sql.Query(`
|
||||
SELECT a.slug, s.path, s.sha256, s.imported_at, s.rows, COUNT(t.id)
|
||||
FROM source_files s
|
||||
JOIN accounts a ON a.id = s.account_id
|
||||
LEFT JOIN transactions t ON t.source_file_id = s.id
|
||||
GROUP BY s.id
|
||||
ORDER BY a.slug, s.path`)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list source files: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []SourceFileInfo
|
||||
for rows.Next() {
|
||||
var f SourceFileInfo
|
||||
if err := rows.Scan(&f.AccountSlug, &f.Path, &f.SHA256, &f.ImportedAt, &f.Rows, &f.Added); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, f)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// ForgetSourceFile removes a statement from the index: its transactions, any
|
||||
// transfer pairing that used them, and the record of it, returning how many
|
||||
// transactions went. Pairing is derived state that the next Link rewrites, so
|
||||
// dropping a pair here loses nothing.
|
||||
//
|
||||
// A transaction two overlapping statements share is stored once, under the
|
||||
// file that brought it first, so removing that file can take rows another
|
||||
// statement still holds. The account's other statements therefore forget
|
||||
// their checksums, and the next import re-reads them and restores those rows.
|
||||
func (d *DB) ForgetSourceFile(accountSlug, path string) (int, error) {
|
||||
tx, err := d.sql.Begin()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
|
||||
var id, accountID int64
|
||||
err = tx.QueryRow(`
|
||||
SELECT s.id, s.account_id FROM source_files s JOIN accounts a ON a.id = s.account_id
|
||||
WHERE a.slug = ? AND s.path = ?`, accountSlug, path).Scan(&id, &accountID)
|
||||
if err == sql.ErrNoRows {
|
||||
return 0, nil
|
||||
}
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("find source file %s: %w", path, err)
|
||||
}
|
||||
if _, err := tx.Exec(`
|
||||
DELETE FROM transfers WHERE
|
||||
out_txn_id IN (SELECT id FROM transactions WHERE source_file_id = ?) OR
|
||||
in_txn_id IN (SELECT id FROM transactions WHERE source_file_id = ?)`, id, id); err != nil {
|
||||
return 0, fmt.Errorf("forget transfers from %s: %w", path, err)
|
||||
}
|
||||
res, err := tx.Exec(`DELETE FROM transactions WHERE source_file_id = ?`, id)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("forget transactions from %s: %w", path, err)
|
||||
}
|
||||
n, err := res.RowsAffected()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if _, err := tx.Exec(`DELETE FROM source_files WHERE id = ?`, id); err != nil {
|
||||
return 0, fmt.Errorf("forget %s: %w", path, err)
|
||||
}
|
||||
if _, err := tx.Exec(`UPDATE source_files SET sha256 = '' WHERE account_id = ?`, accountID); err != nil {
|
||||
return 0, fmt.Errorf("mark %s's statements for re-reading: %w", accountSlug, err)
|
||||
}
|
||||
return int(n), tx.Commit()
|
||||
}
|
||||
|
||||
// Tags lists every tag in use, for completion in the rule builder.
|
||||
func (d *DB) Tags() ([]string, error) {
|
||||
rows, err := d.sql.Query(`
|
||||
SELECT DISTINCT rule_tag FROM transactions
|
||||
|
||||
@@ -214,6 +214,41 @@ func bestCounterpart(out model.Transaction, ins []model.Transaction, used map[in
|
||||
return best
|
||||
}
|
||||
|
||||
// Wanted describes the counterpart an unpaired leg is missing, in the terms
|
||||
// bestCounterpart judges it by, so a person can go and look for it.
|
||||
type Wanted struct {
|
||||
Account string // where the other leg would be
|
||||
Desc string // the glob it would have to match
|
||||
From string // the window it would have to fall in, inclusive
|
||||
To string
|
||||
// Amount is the other leg's signed amount, exact unless TolerancePct
|
||||
// widens it. AnyAmount is set across currencies, where the amount is not
|
||||
// checked at all and the dates carry the pairing alone.
|
||||
Amount int64
|
||||
TolerancePct float64
|
||||
AnyAmount bool
|
||||
}
|
||||
|
||||
// Want says what would have paired l. currencyOf names an account's currency,
|
||||
// or "" for one the index holds nothing for.
|
||||
func (e *Engine) Want(l Leg, currencyOf func(slug string) string) Wanted {
|
||||
def := e.defs[l.Def]
|
||||
w := Wanted{Account: def.ToAccount, Desc: def.ToDesc, Amount: -l.Txn.AmountMinor}
|
||||
if !l.Out {
|
||||
w.Account, w.Desc = def.FromAccount, def.FromDesc
|
||||
}
|
||||
if c := currencyOf(w.Account); c != "" && c != l.Txn.Currency {
|
||||
w.AnyAmount = true
|
||||
} else {
|
||||
w.TolerancePct = def.TolerancePct
|
||||
}
|
||||
if d, ok := day(l.Txn.Date); ok {
|
||||
w.From = d.AddDate(0, 0, -WindowDays).Format("2006-01-02")
|
||||
w.To = d.AddDate(0, 0, WindowDays).Format("2006-01-02")
|
||||
}
|
||||
return w
|
||||
}
|
||||
|
||||
// allowance is how far the arriving leg may miss the leaving one, in minor
|
||||
// units. It is a share of the amount that left, not of the difference, so the
|
||||
// same percentage means the same thing on a large transfer as on a small one.
|
||||
|
||||
@@ -353,3 +353,40 @@ func TestEqualGapPrefersTheNearerAmount(t *testing.T) {
|
||||
t.Errorf("paired with %d, want 3: same gap, exact amount", res.Pairs[0].In.ID)
|
||||
}
|
||||
}
|
||||
|
||||
// Want describes the missing side in the terms the pairing judges it by: the
|
||||
// other definition half, the window, and the amount — exact by default,
|
||||
// widened only by the definition's own tolerance, unchecked across currencies.
|
||||
func TestWantDescribesTheMissingSide(t *testing.T) {
|
||||
eur := func(string) string { return "EUR" }
|
||||
|
||||
out := Leg{Def: 0, Out: true, Txn: txn(1, "nlb", "2026-03-06", "TRANSFER TO REVOLUT", -50000)}
|
||||
got := engine(topUp).Want(out, eur)
|
||||
want := Wanted{Account: "revolut", Desc: "*FROM NLB*", From: "2026-03-01", To: "2026-03-11", Amount: 50000}
|
||||
if got != want {
|
||||
t.Errorf("arriving side = %+v, want %+v", got, want)
|
||||
}
|
||||
|
||||
in := Leg{Def: 0, Txn: txn(2, "revolut", "2026-03-09", "Top-up from NLB", 50000)}
|
||||
got = engine(topUp).Want(in, eur)
|
||||
want = Wanted{Account: "nlb", Desc: "*TO REVOLUT*", From: "2026-03-04", To: "2026-03-14", Amount: -50000}
|
||||
if got != want {
|
||||
t.Errorf("leaving side = %+v, want %+v", got, want)
|
||||
}
|
||||
|
||||
fee := topUp
|
||||
fee.TolerancePct = 1.5
|
||||
if got := engine(fee).Want(out, eur); got.TolerancePct != 1.5 || got.AnyAmount {
|
||||
t.Errorf("tolerant = %+v, want the definition's 1.5%%", got)
|
||||
}
|
||||
|
||||
usd := func(slug string) string {
|
||||
if slug == "revolut" {
|
||||
return "USD"
|
||||
}
|
||||
return "EUR"
|
||||
}
|
||||
if got := engine(fee).Want(out, usd); !got.AnyAmount || got.TolerancePct != 0 {
|
||||
t.Errorf("cross-currency = %+v, want any amount and no tolerance", got)
|
||||
}
|
||||
}
|
||||
|
||||
-2483
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+605
-22
@@ -1,17 +1,21 @@
|
||||
// Package web serves the TUI's screens over HTTP: a JSON API over the index
|
||||
// Package web is money's interactive frontend: a JSON API over the index
|
||||
// and rules.toml, and a single page that drives it. It holds no logic of its
|
||||
// own about money — every number is formatted, every glob matched and every
|
||||
// pair decided by the same packages the TUI and the CLI use, so the browser
|
||||
// never does arithmetic on an amount or re-implements the matcher.
|
||||
// pair decided by the same packages the CLI uses, so the browser never does
|
||||
// arithmetic on an amount or re-implements the matcher.
|
||||
package web
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"embed"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"mime"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"sort"
|
||||
"strconv"
|
||||
@@ -23,6 +27,7 @@ import (
|
||||
"git.petrovv.com/nikola/money/internal/glob"
|
||||
"git.petrovv.com/nikola/money/internal/importer"
|
||||
"git.petrovv.com/nikola/money/internal/model"
|
||||
"git.petrovv.com/nikola/money/internal/parser"
|
||||
"git.petrovv.com/nikola/money/internal/report"
|
||||
"git.petrovv.com/nikola/money/internal/rules"
|
||||
"git.petrovv.com/nikola/money/internal/store"
|
||||
@@ -32,9 +37,9 @@ import (
|
||||
//go:embed static
|
||||
var static embed.FS
|
||||
|
||||
// Server is the long-lived counterpart of the TUI's Model.
|
||||
// Server holds the index and the engines derived from rules.toml.
|
||||
//
|
||||
// Unlike the TUI it outlives any one edit of rules.toml by hand, so retag and
|
||||
// It outlives any one edit of rules.toml by hand, so retag and
|
||||
// import re-read that file (and import the account folders) before running,
|
||||
// exactly as a fresh `money retag` or `money import` would. Between those, the
|
||||
// engines are what the index was last derived from, which is what every screen
|
||||
@@ -73,6 +78,7 @@ func (s *Server) Handler() http.Handler {
|
||||
mux.HandleFunc("GET /api/report", s.read(s.report))
|
||||
|
||||
mux.HandleFunc("GET /api/rules", s.read(s.ruleList))
|
||||
mux.HandleFunc("GET /api/tags", s.read(s.tagList))
|
||||
mux.HandleFunc("POST /api/rules/preview", s.read(s.rulePreview))
|
||||
mux.HandleFunc("POST /api/rules", s.write(s.createRule))
|
||||
mux.HandleFunc("PUT /api/rules/{pos}", s.write(s.editRule))
|
||||
@@ -84,6 +90,10 @@ func (s *Server) Handler() http.Handler {
|
||||
mux.HandleFunc("POST /api/transfers/delete", s.write(s.deleteTransfers))
|
||||
|
||||
mux.HandleFunc("POST /api/import", s.write(s.runImport))
|
||||
mux.HandleFunc("POST /api/upload", s.write(s.upload))
|
||||
mux.HandleFunc("GET /api/files", s.read(s.fileList))
|
||||
mux.HandleFunc("GET /api/files/{account}/{name}", s.serveFile)
|
||||
mux.HandleFunc("POST /api/files/delete", s.write(s.removeFile))
|
||||
mux.HandleFunc("POST /api/retag", s.write(s.retag))
|
||||
return mux
|
||||
}
|
||||
@@ -159,7 +169,11 @@ func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||
}
|
||||
|
||||
func decode(r *http.Request, v any) error {
|
||||
dec := json.NewDecoder(http.MaxBytesReader(nil, r.Body, 1<<20))
|
||||
return decodeLimit(r, v, 1<<20)
|
||||
}
|
||||
|
||||
func decodeLimit(r *http.Request, v any, limit int64) error {
|
||||
dec := json.NewDecoder(http.MaxBytesReader(nil, r.Body, limit))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(v); err != nil {
|
||||
return badRequest("bad request body: %v", err)
|
||||
@@ -167,8 +181,8 @@ func decode(r *http.Request, v any) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// status is the reply to every write: one line saying what happened, as the
|
||||
// TUI's status bar would.
|
||||
// status is the reply to every write: one line saying what happened, shown
|
||||
// in the page's status banner.
|
||||
type status struct {
|
||||
Status string `json:"status"`
|
||||
}
|
||||
@@ -196,13 +210,17 @@ type accountRow struct {
|
||||
|
||||
type overview struct {
|
||||
Root string `json:"root"`
|
||||
// Configured is how many account folders are on disk, which the empty
|
||||
// accounts screen needs: an account.toml is not enough until an import.
|
||||
Configured int `json:"configured"`
|
||||
Accounts []accountRow `json:"accounts"`
|
||||
// AccountSlugs and Tags are what the builders' fields complete against.
|
||||
AccountSlugs []string `json:"accountSlugs"`
|
||||
Tags []string `json:"tags"`
|
||||
// Folders are the account folders on disk now — what an upload can go
|
||||
// into, and what the empty accounts screen counts, since an account.toml
|
||||
// is not enough to appear until an import. Read fresh, since a folder made
|
||||
// after startup is a valid target.
|
||||
// A broken account.toml is reported rather than failing the whole page.
|
||||
Folders []string `json:"folders"`
|
||||
FoldersErr string `json:"foldersError,omitempty"`
|
||||
// Stale reports that rules.toml on disk no longer says what the index was
|
||||
// derived from — it was edited by hand — so the page can offer a retag
|
||||
// instead of quietly describing rules that are not the ones in force.
|
||||
@@ -215,7 +233,7 @@ func (s *Server) overview(*http.Request) (any, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := overview{Root: s.root, Configured: len(s.accounts), Accounts: []accountRow{}}
|
||||
out := overview{Root: s.root, Accounts: []accountRow{}}
|
||||
for _, a := range accounts {
|
||||
bal, err := s.db.Balance(a.ID)
|
||||
if err != nil {
|
||||
@@ -236,6 +254,14 @@ func (s *Server) overview(*http.Request) (any, error) {
|
||||
if out.Tags, err = s.knownTags(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out.Folders = []string{}
|
||||
if folders, err := config.LoadAccounts(s.root); err != nil {
|
||||
out.FoldersErr = err.Error()
|
||||
} else {
|
||||
for _, f := range folders {
|
||||
out.Folders = append(out.Folders, f.Slug)
|
||||
}
|
||||
}
|
||||
onDisk, err := config.LoadRules(s.root)
|
||||
if err != nil {
|
||||
out.Stale, out.RulesErr = true, err.Error()
|
||||
@@ -531,6 +557,55 @@ func (s *Server) ruleList(*http.Request) (any, error) {
|
||||
return map[string]any{"rules": rows}, nil
|
||||
}
|
||||
|
||||
type tagRule struct {
|
||||
Pos int `json:"pos"` // file position, as the rules screen numbers it
|
||||
Pattern string `json:"pattern"`
|
||||
Account string `json:"account"`
|
||||
Usage int `json:"usage"`
|
||||
}
|
||||
|
||||
type tagEntry struct {
|
||||
Tag string `json:"tag"`
|
||||
Transactions int `json:"transactions"`
|
||||
Rules []tagRule `json:"rules"`
|
||||
}
|
||||
|
||||
// tagList is every distinct tag — the ones the index holds and the ones
|
||||
// rules.toml names — with how many transactions carry it and the rules that
|
||||
// write it. A tag is only ever what a rule wrote, so the transfer label is
|
||||
// never among them, and a tag no rule names any more is one a retag will
|
||||
// clear.
|
||||
func (s *Server) tagList(*http.Request) (any, error) {
|
||||
txns, err := s.db.Transactions(store.Filter{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
counts := map[string]int{}
|
||||
for _, t := range txns {
|
||||
if t.RuleTag != "" {
|
||||
counts[t.RuleTag]++
|
||||
}
|
||||
}
|
||||
rules := map[string][]tagRule{}
|
||||
usage := s.engine.Usage(txns)
|
||||
for i, r := range s.engine.Rules() {
|
||||
rules[r.Tag] = append(rules[r.Tag], tagRule{Pos: i, Pattern: rulePattern(r), Account: r.Account, Usage: usage[i]})
|
||||
}
|
||||
tags, err := s.knownTags()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rows := make([]tagEntry, 0, len(tags))
|
||||
for _, tag := range tags {
|
||||
rs := rules[tag]
|
||||
if rs == nil {
|
||||
rs = []tagRule{}
|
||||
}
|
||||
rows = append(rows, tagEntry{Tag: tag, Transactions: counts[tag], Rules: rs})
|
||||
}
|
||||
return map[string]any{"tags": rows}, nil
|
||||
}
|
||||
|
||||
// rulePattern renders whichever patterns a rule sets, labelled so a type rule
|
||||
// is not mistaken for a description one.
|
||||
func rulePattern(r config.Rule) string {
|
||||
@@ -576,7 +651,7 @@ type descGroup struct {
|
||||
Claimed bool
|
||||
}
|
||||
|
||||
// rulePreview is the TUI builder's right-hand panel: what the glob would claim
|
||||
// rulePreview is the rule builder's right-hand panel: what the glob would claim
|
||||
// among everything still waiting for a rule, plus — when editing — what the
|
||||
// rule claims already, since those are tagged and an untagged-only preview
|
||||
// would be empty for a rule that works. Matching runs here rather than in the
|
||||
@@ -872,6 +947,20 @@ type transferRow struct {
|
||||
Tolerance string `json:"tolerance"` // blank for the exact-amount default
|
||||
Paired int `json:"paired"`
|
||||
Orphaned int `json:"orphaned"`
|
||||
// Legs are the transactions behind Orphaned, newest first, so a ⚠ can be
|
||||
// opened to see which movement is missing its other side.
|
||||
Legs []unpairedLeg `json:"legs"`
|
||||
}
|
||||
|
||||
type unpairedLeg struct {
|
||||
Date string `json:"date"`
|
||||
Amount string `json:"amount"`
|
||||
Movement string `json:"movement"` // "checking → ?" or "? → savings"
|
||||
Description string `json:"description"`
|
||||
// Missing names the side not found, "arriving" or "leaving", and Want
|
||||
// what it would have had to look like to pair.
|
||||
Missing string `json:"missing"`
|
||||
Want string `json:"want"`
|
||||
}
|
||||
|
||||
func (s *Server) transferList(*http.Request) (any, error) {
|
||||
@@ -879,18 +968,80 @@ func (s *Server) transferList(*http.Request) (any, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
accounts, err := s.db.Accounts()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
currency := map[string]string{}
|
||||
digits := map[string]int{}
|
||||
for _, a := range accounts {
|
||||
currency[a.Slug], digits[a.Slug] = a.Currency, a.MinorDigits
|
||||
}
|
||||
|
||||
defs := s.links.Transfers()
|
||||
res := s.links.Analyze(txns)
|
||||
rows := make([]transferRow, 0, len(defs))
|
||||
for i, t := range defs {
|
||||
rows = append(rows, transferRow{
|
||||
transferJSON: toTransferJSON(t), Pos: i, Tolerance: formatTolerance(t.TolerancePct),
|
||||
Paired: res.Paired[i], Orphaned: res.Orphaned[i],
|
||||
Paired: res.Paired[i], Orphaned: res.Orphaned[i], Legs: []unpairedLeg{},
|
||||
})
|
||||
}
|
||||
for _, l := range res.Unmatched {
|
||||
row := legRow(l)
|
||||
w := s.links.Want(l, func(slug string) string { return currency[slug] })
|
||||
d, imported := digits[w.Account]
|
||||
if !imported {
|
||||
d = l.Txn.MinorDigits
|
||||
}
|
||||
leg := unpairedLeg{
|
||||
Date: row.Date, Amount: row.Amount, Movement: row.Movement, Description: row.Description,
|
||||
Missing: "arriving", Want: describeWant(w, d),
|
||||
}
|
||||
if !l.Out {
|
||||
leg.Missing = "leaving"
|
||||
}
|
||||
if !imported {
|
||||
// The commonest cause of all, so it is named outright.
|
||||
leg.Want += fmt.Sprintf(" — nothing from %s is imported", w.Account)
|
||||
}
|
||||
rows[l.Def].Legs = append(rows[l.Def].Legs, leg)
|
||||
}
|
||||
for i := range rows {
|
||||
sort.SliceStable(rows[i].Legs, func(a, b int) bool { return rows[i].Legs[a].Date > rows[i].Legs[b].Date })
|
||||
}
|
||||
return map[string]any{"transfers": rows, "unpaired": len(res.Unmatched)}, nil
|
||||
}
|
||||
|
||||
// describeWant puts transfers.Wanted into the words the transfers screen uses.
|
||||
// The amount is signed as the other statement would show it.
|
||||
func describeWant(w transfers.Wanted, digits int) string {
|
||||
amount := model.FormatMinor(w.Amount, digits)
|
||||
if w.Amount > 0 {
|
||||
amount = "+" + amount
|
||||
}
|
||||
switch {
|
||||
case w.AnyAmount:
|
||||
amount = "any amount (another currency)"
|
||||
case w.TolerancePct > 0:
|
||||
amount += " ±" + formatTolerance(w.TolerancePct)
|
||||
}
|
||||
return fmt.Sprintf("%s in %s matching %s, dated %s to %s", amount, w.Account, w.Desc, w.From, w.To)
|
||||
}
|
||||
|
||||
// legRow is an unpaired leg as the builder's preview and the transfers screen
|
||||
// both show it: the amount that moved, and the side that has no partner.
|
||||
func legRow(l transfers.Leg) transferPreviewRow {
|
||||
amount, movement := l.Txn.AmountMinor, "? → "+l.Txn.AccountSlug
|
||||
if l.Out {
|
||||
amount, movement = -amount, l.Txn.AccountSlug+" → ?"
|
||||
}
|
||||
return transferPreviewRow{
|
||||
Marker: "⚠", Date: l.Txn.Date, Amount: model.FormatMinor(amount, l.Txn.MinorDigits),
|
||||
Movement: movement, Description: l.Txn.Description,
|
||||
}
|
||||
}
|
||||
|
||||
// formatTolerance leaves the default blank, so the one or two definitions
|
||||
// actually pairing on slack are what the column shows.
|
||||
func formatTolerance(pct float64) string {
|
||||
@@ -989,14 +1140,7 @@ func (s *Server) transferPreview(r *http.Request) (any, error) {
|
||||
if l.Def != mine {
|
||||
continue
|
||||
}
|
||||
amount, movement := l.Txn.AmountMinor, "? → "+l.Txn.AccountSlug
|
||||
if l.Out {
|
||||
amount, movement = -amount, l.Txn.AccountSlug+" → ?"
|
||||
}
|
||||
out.Rows = append(out.Rows, transferPreviewRow{
|
||||
Marker: "⚠", Date: l.Txn.Date, Amount: model.FormatMinor(amount, l.Txn.MinorDigits),
|
||||
Movement: movement, Description: l.Txn.Description,
|
||||
})
|
||||
out.Rows = append(out.Rows, legRow(l))
|
||||
}
|
||||
sort.SliceStable(out.Rows, func(i, j int) bool { return out.Rows[i].Date > out.Rows[j].Date })
|
||||
if fees != 0 {
|
||||
@@ -1158,6 +1302,445 @@ func (s *Server) runImport(r *http.Request) (any, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// uploadLimit caps an upload request. Statements are small — a year of PDF is
|
||||
// a few hundred kilobytes — and the body is base64, a third larger than the
|
||||
// files it carries.
|
||||
const uploadLimit = 64 << 20
|
||||
|
||||
type uploadFile struct {
|
||||
Name string `json:"name"`
|
||||
Data []byte `json:"data"` // base64 on the wire
|
||||
}
|
||||
|
||||
type uploadReq struct {
|
||||
Account string `json:"account"`
|
||||
Files []uploadFile `json:"files"`
|
||||
}
|
||||
|
||||
// upload saves statements into an account folder. The files on disk are the
|
||||
// source of truth and the index is derived from them, so an upload is nothing
|
||||
// more than putting a file where `money import` looks. It does not import:
|
||||
// that stays the user's call, made with the Import button, so a batch can be
|
||||
// put together and checked on the statements list before it is read.
|
||||
//
|
||||
// Files arrive base64 inside JSON rather than as multipart: a multipart body
|
||||
// is exactly what a cross-site form can send, and the JSON-only rule is the
|
||||
// only thing standing between another page and this server.
|
||||
func (s *Server) upload(r *http.Request) (any, error) {
|
||||
var req uploadReq
|
||||
if err := decodeLimit(r, &req, uploadLimit); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(req.Files) == 0 {
|
||||
return nil, badRequest("no files to upload")
|
||||
}
|
||||
accounts, err := config.LoadAccounts(s.root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var acc *config.Account
|
||||
for _, a := range accounts {
|
||||
if a.Slug == req.Account {
|
||||
acc = a
|
||||
}
|
||||
}
|
||||
if acc == nil {
|
||||
return nil, badRequest("no account folder %q in %s (an account is a folder containing %s)",
|
||||
req.Account, s.root, config.AccountFile)
|
||||
}
|
||||
|
||||
p, err := parser.For(acc)
|
||||
if err != nil {
|
||||
return nil, badRequest("%v", err)
|
||||
}
|
||||
namer, _ := p.(parser.Namer)
|
||||
|
||||
// Each file is staged as a dotfile — invisible to import — so a parser
|
||||
// that names its statements can read it. Everything is then checked
|
||||
// before anything is moved into place, so a bad file in a batch leaves
|
||||
// the folder as it was rather than half uploaded.
|
||||
type staged struct {
|
||||
orig, name, tmp string
|
||||
data []byte
|
||||
named bool // the parser chose name, from the statement
|
||||
}
|
||||
var batch []*staged
|
||||
defer func() {
|
||||
for _, f := range batch {
|
||||
if f.tmp != "" {
|
||||
os.Remove(f.tmp)
|
||||
}
|
||||
}
|
||||
}()
|
||||
for _, f := range req.Files {
|
||||
if err := checkPlainName(f.Name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tmp, err := stageStatement(acc.Dir, f.Data)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
st := &staged{orig: f.Name, name: f.Name, tmp: tmp, data: f.Data}
|
||||
batch = append(batch, st)
|
||||
if namer != nil {
|
||||
stem, err := namer.StatementName(tmp)
|
||||
if err != nil {
|
||||
return nil, badRequest("%s: %v", f.Name, err)
|
||||
}
|
||||
if stem != "" {
|
||||
st.name, st.named = stem+strings.ToLower(filepath.Ext(f.Name)), true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var write []*staged
|
||||
var same, renamed []string
|
||||
claimed := map[string][]byte{}
|
||||
for _, f := range batch {
|
||||
base, n := f.name, 2
|
||||
resolve:
|
||||
for {
|
||||
if err := checkStatementName(acc, f.name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
existing, inBatch := claimed[f.name]
|
||||
var err error
|
||||
if !inBatch {
|
||||
existing, err = os.ReadFile(filepath.Join(acc.Dir, f.name))
|
||||
if err != nil && !errors.Is(err, fs.ErrNotExist) {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case !inBatch && err != nil: // free
|
||||
claimed[f.name] = f.data
|
||||
write = append(write, f)
|
||||
break resolve
|
||||
case bytes.Equal(existing, f.data):
|
||||
same = append(same, f.orig)
|
||||
break resolve
|
||||
case f.named:
|
||||
// A name the parser chose can be taken by a different
|
||||
// statement of the same date — a reissue — so it is
|
||||
// numbered, as rename_izpiski.py did, rather than refused.
|
||||
f.name = fmt.Sprintf("%s_%d%s", strings.TrimSuffix(base, filepath.Ext(base)), n, filepath.Ext(base))
|
||||
n++
|
||||
case inBatch:
|
||||
return nil, badRequest("%s is in the upload twice", f.orig)
|
||||
default:
|
||||
// A statement is the source of truth for what it already
|
||||
// imported; replacing it under the same name is not an
|
||||
// upload's decision to make.
|
||||
return nil, &apiError{http.StatusConflict, fmt.Sprintf(
|
||||
"%s/%s already exists with different contents; rename the file or remove the old one first",
|
||||
acc.Slug, f.name)}
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, f := range write {
|
||||
path := filepath.Join(acc.Dir, f.name)
|
||||
if err := os.Rename(f.tmp, path); err != nil {
|
||||
return nil, fmt.Errorf("write %s: %w", path, err)
|
||||
}
|
||||
f.tmp = ""
|
||||
if f.name != f.orig {
|
||||
renamed = append(renamed, f.orig+" → "+f.name)
|
||||
}
|
||||
}
|
||||
|
||||
msg := fmt.Sprintf("uploaded %d file(s) to %s", len(write), acc.Slug)
|
||||
if len(same) > 0 {
|
||||
msg += fmt.Sprintf(" (%d already there)", len(same))
|
||||
}
|
||||
if len(renamed) > 0 {
|
||||
msg += ", named by statement date: " + strings.Join(renamed, ", ")
|
||||
}
|
||||
if len(write) > 0 {
|
||||
msg += " · press Import to read them"
|
||||
}
|
||||
return status{msg}, nil
|
||||
}
|
||||
|
||||
// checkPlainName refuses anything that is not a plain file name, before the
|
||||
// file is so much as staged.
|
||||
func checkPlainName(name string) error {
|
||||
if name == "" || name != filepath.Base(name) || strings.ContainsAny(name, "/\\\x00") ||
|
||||
name == "." || name == ".." {
|
||||
return badRequest("%q is not a plain file name", name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkStatementName refuses any name the importer would not read back as a
|
||||
// statement of this account, and anything that is not a plain file name.
|
||||
func checkStatementName(acc *config.Account, name string) error {
|
||||
if err := checkPlainName(name); err != nil {
|
||||
return err
|
||||
}
|
||||
if strings.HasPrefix(name, ".") || name == config.AccountFile {
|
||||
return badRequest("%s would be ignored by import (dotfiles and %s are not statements)",
|
||||
name, config.AccountFile)
|
||||
}
|
||||
if len(acc.Include) > 0 && !slices.ContainsFunc(acc.Include, func(p string) bool { return glob.Match(p, name) }) {
|
||||
return badRequest("%s does not match %s's include patterns (%s), so import would ignore it",
|
||||
name, acc.Slug, strings.Join(acc.Include, ", "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// stageStatement writes an upload to a dotfile in the account folder. Import
|
||||
// skips dotfiles, so it is invisible until renamed into place — and a
|
||||
// concurrent `money import` never reads half a statement.
|
||||
func stageStatement(dir string, data []byte) (string, error) {
|
||||
tmp, err := os.CreateTemp(dir, ".upload-*")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("write to %s: %w", dir, err)
|
||||
}
|
||||
if _, err := tmp.Write(data); err != nil {
|
||||
tmp.Close()
|
||||
os.Remove(tmp.Name())
|
||||
return "", fmt.Errorf("write %s: %w", tmp.Name(), err)
|
||||
}
|
||||
if err := tmp.Close(); err != nil {
|
||||
os.Remove(tmp.Name())
|
||||
return "", fmt.Errorf("write %s: %w", tmp.Name(), err)
|
||||
}
|
||||
if err := os.Chmod(tmp.Name(), 0o644); err != nil {
|
||||
os.Remove(tmp.Name())
|
||||
return "", err
|
||||
}
|
||||
return tmp.Name(), nil
|
||||
}
|
||||
|
||||
type fileRow struct {
|
||||
Account string `json:"account"`
|
||||
Name string `json:"name"`
|
||||
Size int64 `json:"size"`
|
||||
// Modified is the file's mtime, YYYY-MM-DD HH:MM in local time; blank for
|
||||
// a file that is gone from disk.
|
||||
Modified string `json:"modified"`
|
||||
// Status is "imported", "changed" (on disk differs from what was
|
||||
// imported), "new" (not imported yet, or its import failed) or "missing"
|
||||
// (imported, then removed from disk; its rows stay in the index until it
|
||||
// is rebuilt).
|
||||
Status string `json:"status"`
|
||||
ImportedAt string `json:"importedAt"`
|
||||
// Rows is how many transactions the statement holds; Added how many of
|
||||
// them it was the first to bring in, which overlap with another statement
|
||||
// can make fewer.
|
||||
Rows int `json:"rows"`
|
||||
Added int `json:"added"`
|
||||
}
|
||||
|
||||
// fileList is every statement on disk next to what the index recorded about
|
||||
// it. The folders are the source of truth, so they decide what is listed, and
|
||||
// a file the index remembers but the disk no longer holds is called out: its
|
||||
// rows would not survive a rebuild.
|
||||
func (s *Server) fileList(*http.Request) (any, error) {
|
||||
recorded, err := s.db.SourceFiles()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
byPath := map[string]store.SourceFileInfo{}
|
||||
for _, f := range recorded {
|
||||
byPath[f.Path] = f
|
||||
}
|
||||
accounts, err := config.LoadAccounts(s.root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
out := []fileRow{}
|
||||
onDisk := map[string]bool{}
|
||||
for _, acc := range accounts {
|
||||
paths, err := importer.StatementFiles(acc)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, path := range paths {
|
||||
rel, err := filepath.Rel(s.root, path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
onDisk[rel] = true
|
||||
row := fileRow{Account: acc.Slug, Name: filepath.Base(path), Status: "new"}
|
||||
if info, err := os.Stat(path); err == nil {
|
||||
row.Size, row.Modified = info.Size(), info.ModTime().Format("2006-01-02 15:04")
|
||||
}
|
||||
if rec, ok := byPath[rel]; ok {
|
||||
row.ImportedAt, row.Rows, row.Added, row.Status = rec.ImportedAt, rec.Rows, rec.Added, "imported"
|
||||
if sum, err := importer.Checksum(path); err != nil {
|
||||
return nil, err
|
||||
} else if sum != rec.SHA256 {
|
||||
row.Status = "changed"
|
||||
}
|
||||
}
|
||||
out = append(out, row)
|
||||
}
|
||||
}
|
||||
for _, rec := range recorded {
|
||||
if !onDisk[rec.Path] {
|
||||
out = append(out, fileRow{
|
||||
Account: rec.AccountSlug, Name: filepath.Base(rec.Path), Status: "missing",
|
||||
ImportedAt: rec.ImportedAt, Rows: rec.Rows, Added: rec.Added,
|
||||
})
|
||||
}
|
||||
}
|
||||
sort.SliceStable(out, func(i, j int) bool {
|
||||
if out[i].Account != out[j].Account {
|
||||
return out[i].Account < out[j].Account
|
||||
}
|
||||
return out[i].Name < out[j].Name
|
||||
})
|
||||
return map[string]any{"files": out}, nil
|
||||
}
|
||||
|
||||
// serveFile opens a statement in the browser. Only a file import would read
|
||||
// is served — it is looked up in the account's statement list, never joined
|
||||
// onto a path — so nothing else under the data root can be fetched.
|
||||
//
|
||||
// A statement is whatever the bank sent, and it is served from this origin,
|
||||
// where a script could drive the API; so nothing is ever rendered as a page.
|
||||
// Text opens inline as text/plain under a sandboxing CSP, and anything that is
|
||||
// neither text nor PDF downloads. A PDF opens inline without the sandbox: the
|
||||
// browsers' viewers refuse to run under one, and they render it in their own
|
||||
// isolated viewer rather than in this origin's DOM.
|
||||
func (s *Server) serveFile(w http.ResponseWriter, r *http.Request) {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
fail := func(code int, msg string) {
|
||||
writeJSON(w, code, map[string]string{"error": msg})
|
||||
}
|
||||
accounts, err := config.LoadAccounts(s.root)
|
||||
if err != nil {
|
||||
fail(http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
account, name := r.PathValue("account"), r.PathValue("name")
|
||||
var path string
|
||||
for _, acc := range accounts {
|
||||
if acc.Slug != account {
|
||||
continue
|
||||
}
|
||||
paths, err := importer.StatementFiles(acc)
|
||||
if err != nil {
|
||||
fail(http.StatusInternalServerError, err.Error())
|
||||
return
|
||||
}
|
||||
for _, p := range paths {
|
||||
if filepath.Base(p) == name {
|
||||
path = p
|
||||
}
|
||||
}
|
||||
}
|
||||
if path == "" {
|
||||
fail(http.StatusNotFound, fmt.Sprintf("no statement %s/%s", account, name))
|
||||
return
|
||||
}
|
||||
|
||||
disposition := "attachment"
|
||||
switch strings.ToLower(filepath.Ext(name)) {
|
||||
case ".pdf":
|
||||
w.Header().Set("Content-Type", "application/pdf")
|
||||
disposition = "inline"
|
||||
case ".csv", ".txt", ".tsv":
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
w.Header().Set("Content-Security-Policy", "sandbox")
|
||||
disposition = "inline"
|
||||
default:
|
||||
w.Header().Set("Content-Type", "application/octet-stream")
|
||||
w.Header().Set("Content-Security-Policy", "sandbox")
|
||||
}
|
||||
w.Header().Set("Content-Disposition", mime.FormatMediaType(disposition, map[string]string{"filename": name}))
|
||||
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
http.ServeFile(w, r, path)
|
||||
}
|
||||
|
||||
type removeFileReq struct {
|
||||
Account string `json:"account"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
// removeFile takes a statement out of the data: the file is deleted and its
|
||||
// transactions leave the index. A file already gone from disk is only
|
||||
// forgotten by the index. The deletion is permanent — the page says so before
|
||||
// asking — so nothing here keeps a copy.
|
||||
//
|
||||
// It does not import. Rows the file shared with an overlapping statement went
|
||||
// with it, and ForgetSourceFile marks the account's other statements so the
|
||||
// next Import re-reads them and restores those rows; until then they show as
|
||||
// changed. Pairing is re-derived here, though: it reads no statement, and a
|
||||
// leg whose partner just went should not stay counted as a transfer.
|
||||
func (s *Server) removeFile(r *http.Request) (any, error) {
|
||||
var req removeFileReq
|
||||
if err := decode(r, &req); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := checkPlainName(req.Name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rel := filepath.Join(req.Account, req.Name)
|
||||
|
||||
// The file is found in the account's statement list, never by joining the
|
||||
// request onto a path, exactly as serveFile does.
|
||||
var path string
|
||||
accounts, err := config.LoadAccounts(s.root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, acc := range accounts {
|
||||
if acc.Slug != req.Account {
|
||||
continue
|
||||
}
|
||||
paths, err := importer.StatementFiles(acc)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, p := range paths {
|
||||
if filepath.Base(p) == req.Name {
|
||||
path = p
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if path != "" {
|
||||
if err := os.Remove(path); err != nil {
|
||||
return nil, fmt.Errorf("remove %s: %w", path, err)
|
||||
}
|
||||
}
|
||||
dropped, err := s.db.ForgetSourceFile(req.Account, rel)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if path == "" && dropped == 0 {
|
||||
known, err := s.db.SourceFiles()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !slices.ContainsFunc(known, func(f store.SourceFileInfo) bool { return f.Path == rel }) {
|
||||
return nil, &apiError{http.StatusNotFound, fmt.Sprintf("no statement %s", rel)}
|
||||
}
|
||||
}
|
||||
|
||||
_, unpaired, err := s.links.Link(s.db)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
verb := "deleted"
|
||||
if path == "" {
|
||||
verb = "forgot"
|
||||
}
|
||||
msg := fmt.Sprintf("%s %s, %d transaction(s) dropped", verb, rel, dropped)
|
||||
if unpaired > 0 {
|
||||
msg += fmt.Sprintf(" · %d transfer leg(s) unpaired", unpaired)
|
||||
}
|
||||
if dropped > 0 {
|
||||
msg += " · press Import to restore any of them another statement also holds"
|
||||
}
|
||||
return status{msg}, nil
|
||||
}
|
||||
|
||||
// retag is `money retag`: both halves of what rules.toml decides, re-derived
|
||||
// from the file as it is on disk now.
|
||||
func (s *Server) retag(*http.Request) (any, error) {
|
||||
|
||||
+129
-1
@@ -44,7 +44,7 @@ func newTestServer(t *testing.T, rulesToml string, txns ...fixtureTxn) (*Server,
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
src, err := db.SourceFile(id, slug+"/st.csv", "sha", "2026-01-01T00:00:00Z")
|
||||
src, err := db.SourceFile(id, slug+"/st.csv", "sha", "2026-01-01T00:00:00Z", 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -386,3 +386,131 @@ func TestPageIsServed(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A paired leg is spoken for, so it is neither listed as untagged nor offered
|
||||
// to the rule builder; an unpaired leg is untagged like anything else, which
|
||||
// is how it gets noticed.
|
||||
func TestPairedLegsAreNotUntagged(t *testing.T) {
|
||||
_, h := newTestServer(t, `
|
||||
[[transfer]]
|
||||
from_account = "checking"
|
||||
from_desc = "*TO SAVINGS*"
|
||||
to_account = "savings"
|
||||
to_desc = "*FROM CHECKING*"
|
||||
`,
|
||||
fixtureTxn{"checking", "2026-02-01", "TRANSFER TO SAVINGS", -1000},
|
||||
fixtureTxn{"savings", "2026-02-01", "TRANSFER FROM CHECKING", 1000},
|
||||
fixtureTxn{"checking", "2026-02-20", "TRANSFER TO SAVINGS", -2000},
|
||||
fixtureTxn{"checking", "2026-02-05", "LIDL SOFIA", -500},
|
||||
)
|
||||
|
||||
var txns struct{ Rows []txnRow }
|
||||
call(t, h, "GET", "/api/transactions?untagged=1", nil, http.StatusOK, &txns)
|
||||
var seen []string
|
||||
for _, r := range txns.Rows {
|
||||
seen = append(seen, r.Date+" "+r.Description)
|
||||
}
|
||||
want := "2026-02-20 TRANSFER TO SAVINGS|2026-02-05 LIDL SOFIA"
|
||||
if strings.Join(seen, "|") != want {
|
||||
t.Errorf("untagged = %v, want only the unpaired leg and the shopping", seen)
|
||||
}
|
||||
|
||||
var p rulePreviewJSON
|
||||
call(t, h, "POST", "/api/rules/preview", rulePreviewReq{Glob: "*"}, http.StatusOK, &p)
|
||||
for _, r := range p.Rows {
|
||||
if r.Description == "TRANSFER FROM CHECKING" {
|
||||
t.Error("the arriving leg of a matched transfer is still offered for tagging")
|
||||
}
|
||||
if r.Description == "TRANSFER TO SAVINGS" && r.Count != 1 {
|
||||
t.Errorf("departing leg offered %d times, want only the unpaired one", r.Count)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A ⚠ on the transfers screen opens onto the legs behind it: which side each
|
||||
// is missing, and what that side would have to be.
|
||||
func TestTransferListNamesTheMissingLegs(t *testing.T) {
|
||||
_, h := newTestServer(t, `
|
||||
[[transfer]]
|
||||
from_account = "checking"
|
||||
from_desc = "*TO SAVINGS*"
|
||||
to_account = "savings"
|
||||
to_desc = "*FROM CHECKING*"
|
||||
`,
|
||||
fixtureTxn{"checking", "2026-02-01", "TO SAVINGS", -1000},
|
||||
fixtureTxn{"savings", "2026-02-02", "FROM CHECKING", 1000},
|
||||
fixtureTxn{"checking", "2026-02-20", "TO SAVINGS", -2500},
|
||||
fixtureTxn{"savings", "2026-01-10", "FROM CHECKING", 700},
|
||||
)
|
||||
var list struct {
|
||||
Transfers []transferRow
|
||||
Unpaired int
|
||||
}
|
||||
call(t, h, "GET", "/api/transfers", nil, http.StatusOK, &list)
|
||||
legs := list.Transfers[0].Legs
|
||||
if list.Unpaired != 2 || len(legs) != 2 {
|
||||
t.Fatalf("unpaired = %d, legs = %+v; want the two lone legs", list.Unpaired, legs)
|
||||
}
|
||||
newest, oldest := legs[0], legs[1]
|
||||
if newest.Movement != "checking → ?" || newest.Missing != "arriving" ||
|
||||
newest.Want != "+25.00 in savings matching *FROM CHECKING*, dated 2026-02-15 to 2026-02-25" {
|
||||
t.Errorf("newest leg = %+v", newest)
|
||||
}
|
||||
if oldest.Movement != "? → savings" || oldest.Missing != "leaving" ||
|
||||
oldest.Want != "-7.00 in checking matching *TO SAVINGS*, dated 2026-01-05 to 2026-01-15" {
|
||||
t.Errorf("oldest leg = %+v", oldest)
|
||||
}
|
||||
}
|
||||
|
||||
// The tags screen lists each tag once, with the transactions carrying it and
|
||||
// every rule that writes it — a shadowed one at zero — and never the
|
||||
// transfer label, which no rule wrote.
|
||||
func TestTagListGroupsRulesByTag(t *testing.T) {
|
||||
_, h := newTestServer(t, `
|
||||
[[rule]]
|
||||
match = "*LIDL*"
|
||||
tag = "groceries"
|
||||
|
||||
[[rule]]
|
||||
match = "*KAUFLAND*"
|
||||
tag = "groceries"
|
||||
|
||||
[[rule]]
|
||||
match = "*LIDL SOFIA*"
|
||||
tag = "groceries"
|
||||
|
||||
[[rule]]
|
||||
match = "*ZARA*"
|
||||
tag = "clothes"
|
||||
|
||||
[[transfer]]
|
||||
from_account = "checking"
|
||||
from_desc = "*TO SAVINGS*"
|
||||
to_account = "savings"
|
||||
to_desc = "*FROM CHECKING*"
|
||||
`,
|
||||
fixtureTxn{"checking", "2026-02-01", "LIDL SOFIA", -1000},
|
||||
fixtureTxn{"checking", "2026-02-02", "KAUFLAND", -500},
|
||||
fixtureTxn{"checking", "2026-02-03", "TO SAVINGS", -100},
|
||||
fixtureTxn{"savings", "2026-02-03", "FROM CHECKING", 100},
|
||||
)
|
||||
var res struct{ Tags []tagEntry }
|
||||
call(t, h, "GET", "/api/tags", nil, http.StatusOK, &res)
|
||||
if len(res.Tags) != 2 || res.Tags[0].Tag != "clothes" || res.Tags[1].Tag != "groceries" {
|
||||
t.Fatalf("tags = %+v, want clothes and groceries only", res.Tags)
|
||||
}
|
||||
clothes, groceries := res.Tags[0], res.Tags[1]
|
||||
if clothes.Transactions != 0 || len(clothes.Rules) != 1 {
|
||||
t.Errorf("clothes = %+v, want its rule and no transactions", clothes)
|
||||
}
|
||||
if groceries.Transactions != 2 || len(groceries.Rules) != 3 {
|
||||
t.Fatalf("groceries = %+v, want 2 transactions from 3 rules", groceries)
|
||||
}
|
||||
usage := map[string]int{}
|
||||
for _, r := range groceries.Rules {
|
||||
usage[r.Pattern] = r.Usage
|
||||
}
|
||||
if usage["*LIDL*"] != 0 || usage["*LIDL SOFIA*"] != 1 || usage["*KAUFLAND*"] != 1 {
|
||||
t.Errorf("usage = %v, want *LIDL* shadowed by *LIDL SOFIA*", usage)
|
||||
}
|
||||
}
|
||||
|
||||
+262
-93
@@ -12,6 +12,7 @@ const VIEWS = [
|
||||
{ id: 'rules', title: 'Rules', mount: mountRules },
|
||||
{ id: 'transfer', title: 'Transfer builder', mount: mountTransferBuilder },
|
||||
{ id: 'transfers', title: 'Transfers', mount: mountTransfers },
|
||||
{ id: 'tags', title: 'Tags', mount: mountTags },
|
||||
];
|
||||
|
||||
const state = {
|
||||
@@ -41,7 +42,9 @@ function blankTransfer() {
|
||||
// --- plumbing -------------------------------------------------------------
|
||||
|
||||
// h builds an element. Text always goes in as a text node, never as HTML,
|
||||
// since descriptions come straight off bank statements.
|
||||
// since descriptions come straight off bank statements. Children flatten all
|
||||
// the way down: a map that yields a group of rows — the report's transfers
|
||||
// and their fee — would otherwise be stringified into the page.
|
||||
function h(tag, attrs, ...children) {
|
||||
const el = document.createElement(tag);
|
||||
for (const [k, v] of Object.entries(attrs || {})) {
|
||||
@@ -51,7 +54,7 @@ function h(tag, attrs, ...children) {
|
||||
else if (k === 'value') el.value = v;
|
||||
else el.setAttribute(k, v === true ? '' : v);
|
||||
}
|
||||
for (const c of children.flat()) {
|
||||
for (const c of children.flat(Infinity)) {
|
||||
if (c === null || c === undefined || c === false) continue;
|
||||
el.append(c instanceof Node ? c : document.createTextNode(String(c)));
|
||||
}
|
||||
@@ -140,8 +143,8 @@ async function loadOverview() {
|
||||
fillDatalists();
|
||||
}
|
||||
|
||||
// The account and tag fields complete natively, from the lists the TUI
|
||||
// completes against. (transfer) is never among the tags: it is a label.
|
||||
// The account and tag fields complete natively, from lists the server
|
||||
// builds. (transfer) is never among the tags: it is a label.
|
||||
function fillDatalists() {
|
||||
const o = state.overview;
|
||||
if (!o) return;
|
||||
@@ -172,7 +175,7 @@ function renderNav() {
|
||||
nav.replaceChildren(...VIEWS.map((v, i) => h('a', {
|
||||
href: '#' + v.id,
|
||||
class: v.id === state.view ? 'active' : null,
|
||||
}, h('kbd', {}, String(i + 1)), v.title)));
|
||||
}, v.title)));
|
||||
}
|
||||
|
||||
function go(id) {
|
||||
@@ -186,8 +189,8 @@ function go(id) {
|
||||
function show() {
|
||||
const id = location.hash.slice(1) || 'accounts';
|
||||
const view = VIEWS.find((v) => v.id === id) || VIEWS[0];
|
||||
// Leaving the builder mid-edit abandons the edit, as esc does in the TUI: a
|
||||
// new rule must not start out as a copy of the one being edited.
|
||||
// Leaving the builder mid-edit abandons the edit, as Cancel does: a new
|
||||
// rule must not start out as a copy of the one being edited.
|
||||
if (view.id !== 'rule' && state.rule.editing) state.rule = blankRule();
|
||||
state.view = view.id;
|
||||
renderNav();
|
||||
@@ -203,12 +206,16 @@ async function refreshAll() {
|
||||
|
||||
// --- import and retag -----------------------------------------------------
|
||||
|
||||
// runImport is `money import`, or with force `money import --force`: every
|
||||
// statement re-read, including the ones the checksum would skip.
|
||||
async function runImport(force) {
|
||||
if (state.busy) return;
|
||||
state.busy = true;
|
||||
const btn = document.getElementById('import');
|
||||
btn.disabled = true;
|
||||
btn.textContent = 'Importing…';
|
||||
const buttons = ['import', 'reread'].map((id) => document.getElementById(id));
|
||||
const btn = buttons[force ? 1 : 0];
|
||||
const label = btn.textContent;
|
||||
for (const b of buttons) b.disabled = true;
|
||||
btn.textContent = force ? 'Re-reading…' : 'Importing…';
|
||||
setStatus('importing… parsing PDF statements can take a while');
|
||||
try {
|
||||
const res = await api('POST', '/api/import', { force: !!force });
|
||||
@@ -221,8 +228,8 @@ async function runImport(force) {
|
||||
setError(e);
|
||||
} finally {
|
||||
state.busy = false;
|
||||
btn.disabled = false;
|
||||
btn.textContent = 'Import';
|
||||
for (const b of buttons) b.disabled = false;
|
||||
btn.textContent = label;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -255,17 +262,21 @@ async function retag() {
|
||||
|
||||
function mountAccounts(main) {
|
||||
const body = h('div');
|
||||
main.append(h('h2', {}, 'Accounts'), body);
|
||||
const files = h('div');
|
||||
const upload = h('div');
|
||||
main.append(h('h2', {}, 'Accounts'), body, files, upload);
|
||||
|
||||
async function refresh() {
|
||||
if (!state.overview) await loadOverview();
|
||||
const o = state.overview;
|
||||
upload.replaceChildren(uploader(o));
|
||||
api('GET', '/api/files').then((res) => files.replaceChildren(fileList(res.files))).catch(setError);
|
||||
if (!o.accounts.length) {
|
||||
// An account.toml is not enough on its own: folders reach the index
|
||||
// only through an import, so say that rather than show nothing.
|
||||
body.replaceChildren(h('div', { class: 'panel empty' }, o.configured
|
||||
? `No accounts imported yet.\n\n${o.configured} account folder(s) configured in ${o.root}.\nPress Import to read their statements.`
|
||||
: `No account folders found in ${o.root}.\n\nAn account is a folder containing an account.toml.\nCreate one, drop statements in, then press Import.`));
|
||||
body.replaceChildren(h('div', { class: 'panel empty' }, o.folders.length
|
||||
? `No accounts imported yet.\n\n${o.folders.length} account folder(s) in ${o.root}.\nAdd statements below, or put them in the folders and press Import.`
|
||||
: `No account folders found in ${o.root}.\n\nAn account is a folder containing an account.toml.\nCreate one, then add statements to it here.`));
|
||||
return;
|
||||
}
|
||||
body.replaceChildren(h('div', { class: 'panel' }, h('table', {},
|
||||
@@ -282,6 +293,164 @@ function mountAccounts(main) {
|
||||
return { refresh };
|
||||
}
|
||||
|
||||
const FILE_STATUS = {
|
||||
imported: ['✓', '', 'imported'],
|
||||
changed: ['⚠', 'warn', 'changed'],
|
||||
new: ['•', 'warn', 'not imported'],
|
||||
missing: ['✗', 'neg', 'gone from disk'],
|
||||
};
|
||||
|
||||
// fileList shows the statements in every account folder beside what the index
|
||||
// recorded about each, and opens one on click. The folders decide what is
|
||||
// listed; the index only says how far each file has got.
|
||||
function fileList(list) {
|
||||
const panel = h('div', { class: 'panel files' });
|
||||
if (!list.length) {
|
||||
panel.append(h('div', { class: 'empty' }, 'No statement files yet.'));
|
||||
return h('div', {}, h('h3', { class: 'section' }, 'Statements'), panel);
|
||||
}
|
||||
const pending = list.filter((f) => f.status !== 'imported').length;
|
||||
panel.append(h('table', {},
|
||||
h('thead', {}, h('tr', {}, h('th', {}, 'Account'), h('th', {}, 'File'), h('th', { class: 'num' }, 'Size'),
|
||||
h('th', {}, 'Modified'), h('th', {}, ''), h('th', {}, 'Status'),
|
||||
h('th', { class: 'num', title: 'Transactions the file holds, and how many of them it was the first to bring in — an overlapping statement imported earlier already holds the rest' }, 'Transactions'),
|
||||
h('th', {}, 'Imported'), h('th', {}, ''))),
|
||||
h('tbody', {}, list.map((f) => {
|
||||
const [mark, cls, text] = FILE_STATUS[f.status] || ['', '', f.status];
|
||||
return h('tr', {},
|
||||
h('td', {}, f.account),
|
||||
h('td', { class: 'desc' }, f.status === 'missing' ? h('span', { class: 'muted' }, f.name) : h('a', {
|
||||
href: `/api/files/${encodeURIComponent(f.account)}/${encodeURIComponent(f.name)}`,
|
||||
target: '_blank', rel: 'noopener', title: 'Open this statement',
|
||||
}, f.name)),
|
||||
h('td', { class: 'num' }, f.status === 'missing' ? '' : formatSize(f.size)),
|
||||
h('td', { class: 'muted' }, f.modified),
|
||||
h('td', { class: 'mark ' + cls }, mark),
|
||||
h('td', { class: cls || 'muted' }, text),
|
||||
h('td', { class: 'num' }, f.status === 'new' ? '' : `${f.rows} ${f.rows === 1 ? 'row' : 'rows'} · ${f.added} new`),
|
||||
h('td', { class: 'muted' }, localTime(f.importedAt)),
|
||||
h('td', { class: 'actions' }, h('button', {
|
||||
type: 'button', class: 'danger', onclick: () => removeFile(f),
|
||||
}, f.status === 'missing' ? 'Forget' : 'Delete')));
|
||||
}))));
|
||||
return h('div', {}, h('h3', { class: 'section' }, 'Statements ',
|
||||
h('span', { class: 'sub' }, `· ${list.length} file(s)` + (pending ? ` · ${pending} need attention` : ''))), panel);
|
||||
}
|
||||
|
||||
// localTime shows the index's UTC import time in the same local
|
||||
// YYYY-MM-DD HH:MM that the server gives the files' modified times.
|
||||
function localTime(iso) {
|
||||
if (!iso) return '';
|
||||
const d = new Date(iso);
|
||||
if (isNaN(d)) return iso;
|
||||
const p = (n) => String(n).padStart(2, '0');
|
||||
return `${d.getFullYear()}-${p(d.getMonth() + 1)}-${p(d.getDate())} ${p(d.getHours())}:${p(d.getMinutes())}`;
|
||||
}
|
||||
|
||||
// removeFile deletes a statement for good. It does not import: rows another
|
||||
// statement also holds come back at the next Import, so the question asked
|
||||
// here is only about what this file alone brought in.
|
||||
function removeFile(f) {
|
||||
const where = `${f.account}/${f.name}`;
|
||||
const rows = f.status === 'new' ? 'It has not been imported, so no transactions change.'
|
||||
: `The ${f.added} transaction(s) it brought in leave the index; any another statement also holds come back at the next Import.`;
|
||||
const question = f.status === 'missing'
|
||||
? `Forget ${where}? The file is already gone from disk. ${rows}`
|
||||
: `Delete ${where} permanently? The file is removed from disk and cannot be restored from here. ${rows}`;
|
||||
if (!confirm(question)) return;
|
||||
api('POST', '/api/files/delete', { account: f.account, name: f.name })
|
||||
.then((res) => { setStatus(res.status); return refreshAll(); })
|
||||
.catch(setError);
|
||||
}
|
||||
|
||||
function formatSize(bytes) {
|
||||
if (bytes < 1024) return `${bytes} B`;
|
||||
if (bytes < 1024 * 1024) return `${Math.round(bytes / 1024)} KB`;
|
||||
return `${(bytes / 1024 / 1024).toFixed(1)} MB`;
|
||||
}
|
||||
|
||||
// uploader puts statement files into an account folder and imports them. The
|
||||
// folder is where they belong: the index is rebuilt from it, so a statement
|
||||
// that lived only in the index would be lost the next time it is deleted.
|
||||
function uploader(o) {
|
||||
const panel = h('div', { class: 'panel upload' }, h('h3', {}, 'Add statements'));
|
||||
if (!o.folders.length) {
|
||||
panel.append(h('div', { class: 'muted' }, o.foldersError
|
||||
? `Account folders do not load: ${o.foldersError}`
|
||||
: `There is no account folder to add statements to. Create one in ${o.root} with an account.toml.`));
|
||||
return panel;
|
||||
}
|
||||
|
||||
let files = [];
|
||||
const account = h('select', { 'aria-label': 'Account to add statements to' },
|
||||
o.folders.map((f) => h('option', { value: f }, f)));
|
||||
// Default to the account being browsed, since that is usually the one.
|
||||
account.value = o.folders.includes(state.filter.account) ? state.filter.account : o.folders[0];
|
||||
|
||||
const picker = h('input', {
|
||||
type: 'file', multiple: true, hidden: true,
|
||||
onchange: (e) => choose(e.target.files),
|
||||
});
|
||||
const chosen = h('div', { class: 'chosen muted' }, 'No files chosen');
|
||||
const send = h('button', { type: 'button', class: 'primary', disabled: true, onclick: submit }, 'Upload');
|
||||
const zone = h('div', {
|
||||
class: 'dropzone',
|
||||
tabindex: 0,
|
||||
role: 'button',
|
||||
onclick: () => picker.click(),
|
||||
onkeydown: (e) => { if (e.key === 'Enter' || e.key === ' ') { e.preventDefault(); picker.click(); } },
|
||||
ondragover: (e) => { e.preventDefault(); zone.classList.add('over'); },
|
||||
ondragleave: () => zone.classList.remove('over'),
|
||||
ondrop: (e) => { e.preventDefault(); zone.classList.remove('over'); choose(e.dataTransfer.files); },
|
||||
}, 'Drop statement files here, or click to choose');
|
||||
|
||||
function choose(list) {
|
||||
files = Array.from(list || []);
|
||||
send.disabled = !files.length;
|
||||
chosen.textContent = files.length
|
||||
? files.map((f) => `${f.name} (${Math.ceil(f.size / 1024)} KB)`).join(' · ')
|
||||
: 'No files chosen';
|
||||
}
|
||||
|
||||
async function submit() {
|
||||
send.disabled = true;
|
||||
try {
|
||||
const payload = {
|
||||
account: account.value,
|
||||
files: await Promise.all(files.map(async (f) => ({ name: f.name, data: await base64(f) }))),
|
||||
};
|
||||
const res = await api('POST', '/api/upload', payload);
|
||||
picker.value = '';
|
||||
choose([]);
|
||||
setStatus(res.status);
|
||||
await refreshAll();
|
||||
} catch (e) {
|
||||
setError(e);
|
||||
} finally {
|
||||
send.disabled = !files.length;
|
||||
}
|
||||
}
|
||||
|
||||
panel.append(
|
||||
h('div', { class: 'toolbar' }, h('label', {}, 'Account ', account), send),
|
||||
zone, picker, chosen,
|
||||
h('div', { class: 'hint muted' },
|
||||
'Files are saved into the account folder, next to the statements already there; press Import to read them. ' +
|
||||
'A file with the same name and different contents is refused rather than replaced.'));
|
||||
return panel;
|
||||
}
|
||||
|
||||
// base64 encodes a file for the JSON body. Chunked, because spreading a whole
|
||||
// statement into String.fromCharCode overflows the call stack.
|
||||
async function base64(file) {
|
||||
const bytes = new Uint8Array(await file.arrayBuffer());
|
||||
let bin = '';
|
||||
for (let i = 0; i < bytes.length; i += 0x8000) {
|
||||
bin += String.fromCharCode.apply(null, bytes.subarray(i, i + 0x8000));
|
||||
}
|
||||
return btoa(bin);
|
||||
}
|
||||
|
||||
// --- shared scope toolbar ---------------------------------------------------
|
||||
|
||||
function scopeToolbar(onChange) {
|
||||
@@ -294,7 +463,7 @@ function scopeToolbar(onChange) {
|
||||
select.value = state.filter.account;
|
||||
|
||||
const search = h('input', {
|
||||
type: 'search', id: 'search', placeholder: 'Filter by description ( / )',
|
||||
type: 'search', id: 'search', placeholder: 'Filter by description',
|
||||
value: state.filter.search,
|
||||
oninput: debounce((e) => { state.filter.search = e.target.value.trim(); onChange(); }, 200),
|
||||
});
|
||||
@@ -303,7 +472,7 @@ function scopeToolbar(onChange) {
|
||||
onchange: (e) => { state.filter.untagged = e.target.checked; onChange(); },
|
||||
});
|
||||
return h('div', { class: 'toolbar' }, select, search,
|
||||
h('label', { class: 'check', title: 'No tag and not a matched transfer leg (u)' }, untagged, 'Untagged only'));
|
||||
h('label', { class: 'check', title: 'No tag and not a matched transfer leg' }, untagged, 'Untagged only'));
|
||||
}
|
||||
|
||||
// --- transactions ---------------------------------------------------------
|
||||
@@ -406,7 +575,7 @@ function mountReport(main) {
|
||||
// An empty period is not an empty index, and the default window is the
|
||||
// one most likely to be empty: last month's statement may not be in yet.
|
||||
content = h('div', { class: 'panel empty' }, period.bounded && !data.indexEmpty
|
||||
? `Nothing in ${period.name}.\n\nChoose another period (← →), or import.`
|
||||
? `Nothing in ${period.name}.\n\nChoose another period, or import.`
|
||||
: 'Nothing to report yet.\n\nImport some statements first.');
|
||||
} else {
|
||||
const order = data.orders.find((o) => o.name === data.order);
|
||||
@@ -446,21 +615,7 @@ function mountReport(main) {
|
||||
} catch (e) { setError(e); }
|
||||
}
|
||||
refresh();
|
||||
return {
|
||||
refresh,
|
||||
key(e, typing) {
|
||||
if (typing || !data || e.ctrlKey || e.metaKey || e.altKey) return false;
|
||||
if (e.key === 'ArrowLeft') { setPeriod(data.period - 1); return true; }
|
||||
if (e.key === 'ArrowRight') { setPeriod(data.period + 1); return true; }
|
||||
if (e.key === 's') {
|
||||
const i = data.orders.findIndex((o) => o.name === data.order);
|
||||
state.report.sort = data.orders[(i + 1) % data.orders.length].name;
|
||||
refresh();
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
},
|
||||
};
|
||||
return { refresh };
|
||||
}
|
||||
|
||||
// --- form helpers ---------------------------------------------------------
|
||||
@@ -481,7 +636,7 @@ function textInput(id, value, placeholder, oninput, list) {
|
||||
function mountRuleBuilder(main) {
|
||||
const r = state.rule;
|
||||
const editing = r.editing;
|
||||
// A new rule starts on the account being browsed, as in the TUI.
|
||||
// A new rule starts on the account being browsed.
|
||||
if (!editing && !r.account && state.filter.account) r.account = state.filter.account;
|
||||
|
||||
const summary = h('div', { class: 'summary' });
|
||||
@@ -547,8 +702,8 @@ function mountRuleBuilder(main) {
|
||||
}
|
||||
preview.replaceChildren(h('table', {},
|
||||
h('thead', {}, h('tr', {}, h('th', {}, ''),
|
||||
h('th', { class: 'sortable', onclick: toggle, title: 'Sort by name / by count (ctrl+s)' }, descTitle),
|
||||
h('th', { class: 'num sortable', onclick: toggle, title: 'Sort by name / by count (ctrl+s)' }, 'N' + (byCount ? ' ↓' : '')))),
|
||||
h('th', { class: 'sortable', onclick: toggle, title: 'Sort by name / by count' }, descTitle),
|
||||
h('th', { class: 'num sortable', onclick: toggle, title: 'Sort by name / by count' }, 'N' + (byCount ? ' ↓' : '')))),
|
||||
h('tbody', {}, res.rows.map((row) => h('tr', {
|
||||
class: 'clickable',
|
||||
title: 'Use this description as the glob',
|
||||
@@ -593,23 +748,7 @@ function mountRuleBuilder(main) {
|
||||
|
||||
inputs.match.focus();
|
||||
refreshNow();
|
||||
return {
|
||||
refresh: refreshNow,
|
||||
form: true,
|
||||
key(e) {
|
||||
if (e.key === 's' && e.ctrlKey) {
|
||||
state.ruleSort = state.ruleSort === 'count' ? 'name' : 'count';
|
||||
refreshNow();
|
||||
return true;
|
||||
}
|
||||
if (e.key === 'Escape' && editing) {
|
||||
state.rule = blankRule();
|
||||
go('rules');
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
},
|
||||
};
|
||||
return { refresh: refreshNow };
|
||||
}
|
||||
|
||||
// --- rules ----------------------------------------------------------------
|
||||
@@ -773,7 +912,7 @@ function mountTransferBuilder(main) {
|
||||
|
||||
inputs.fromAccount.focus();
|
||||
refreshNow();
|
||||
return { refresh: refreshNow, form: true };
|
||||
return { refresh: refreshNow };
|
||||
}
|
||||
|
||||
// --- transfers ------------------------------------------------------------
|
||||
@@ -795,6 +934,9 @@ function mountTransfers(main) {
|
||||
h('button', { type: 'button', onclick: () => refresh().then(() => setStatus('pairing refreshed')) }, 'Refresh pairing')),
|
||||
body);
|
||||
let defs = [];
|
||||
let unpaired = 0;
|
||||
// Which ⚠ rows are open, by position, so a refresh does not fold them away.
|
||||
const open = new Set();
|
||||
|
||||
async function del(positions, question) {
|
||||
if (!confirm(question)) return;
|
||||
@@ -803,6 +945,7 @@ function mountTransfers(main) {
|
||||
positions, expect: positions.map((p) => transferExpect(defs[p])),
|
||||
});
|
||||
setStatus(res.status);
|
||||
open.clear(); // positions shift under a delete
|
||||
await refreshAll();
|
||||
} catch (e) { setError(e); }
|
||||
}
|
||||
@@ -813,8 +956,13 @@ function mountTransfers(main) {
|
||||
res = await api('GET', '/api/transfers');
|
||||
} catch (e) { setError(e); return; }
|
||||
defs = res.transfers;
|
||||
title.replaceChildren('Transfers ', h('span', { class: 'sub' }, res.unpaired
|
||||
? `· ${defs.length} definitions · ${res.unpaired} leg(s) unpaired`
|
||||
unpaired = res.unpaired;
|
||||
draw();
|
||||
}
|
||||
|
||||
function draw() {
|
||||
title.replaceChildren('Transfers ', h('span', { class: 'sub' }, unpaired
|
||||
? `· ${defs.length} definitions · ${unpaired} leg(s) unpaired — open a ⚠ row to see them`
|
||||
: `· ${defs.length} definitions · every leg paired`));
|
||||
// Only definitions catching nothing at all are pruned. One with unpaired
|
||||
// legs is doing something, and deleting it would hide the problem.
|
||||
@@ -834,9 +982,17 @@ function mountTransfers(main) {
|
||||
h('th', {}, 'Note'), h('th', {}, ''))),
|
||||
h('tbody', {}, defs.map((d) => {
|
||||
let mark = '', cls = 'mark', tip = null;
|
||||
if (d.orphaned > 0) [mark, cls, tip] = ['⚠', 'mark warn', 'Catches legs it cannot pair'];
|
||||
if (d.orphaned > 0) [mark, cls, tip] = [open.has(d.pos) ? '▾' : '⚠', 'mark warn', 'Show the legs it cannot pair'];
|
||||
else if (d.paired === 0) [mark, cls, tip] = ['✗', 'mark neg', 'Matches nothing at all'];
|
||||
return h('tr', {},
|
||||
const toggle = () => {
|
||||
if (open.has(d.pos)) open.delete(d.pos); else open.add(d.pos);
|
||||
draw();
|
||||
};
|
||||
return [h('tr', {
|
||||
class: d.orphaned > 0 ? 'clickable' : null,
|
||||
title: d.orphaned > 0 ? 'Show the legs it cannot pair' : null,
|
||||
onclick: d.orphaned > 0 ? (e) => { if (!e.target.closest('button')) toggle(); } : null,
|
||||
},
|
||||
h('td', { class: 'num muted' }, d.pos + 1),
|
||||
h('td', { class: cls, title: tip }, mark),
|
||||
h('td', {}, d.fromAccount, ' ', h('span', { style: 'font-family: var(--mono)' }, d.fromDesc)),
|
||||
@@ -848,51 +1004,64 @@ function mountTransfers(main) {
|
||||
h('td', { class: 'actions' }, h('button', {
|
||||
type: 'button', class: 'danger',
|
||||
onclick: () => del([d.pos], `Delete transfer ${d.pos + 1} (${d.fromAccount} → ${d.toAccount}), ${d.paired} matched?`),
|
||||
}, 'Delete')));
|
||||
}, 'Delete'))),
|
||||
open.has(d.pos) && d.orphaned > 0 ? h('tr', { class: 'legs' }, h('td', { colspan: 9 }, legsTable(d))) : null];
|
||||
})))));
|
||||
}
|
||||
|
||||
// legsTable lists a definition's unpaired legs: which side each is missing,
|
||||
// and what that side would have had to look like to pair.
|
||||
function legsTable(d) {
|
||||
return h('table', { class: 'sub' },
|
||||
h('thead', {}, h('tr', {}, h('th', {}, 'Date'), h('th', { class: 'num' }, 'Amount'), h('th', {}, 'Movement'),
|
||||
h('th', {}, 'Description'), h('th', {}, 'Missing'), h('th', {}, 'Would pair with'))),
|
||||
h('tbody', {}, d.legs.map((l) => h('tr', {},
|
||||
h('td', {}, l.date), h('td', { class: 'num' }, l.amount), h('td', {}, l.movement),
|
||||
h('td', { class: 'desc' }, l.description), h('td', { class: 'warn' }, l.missing + ' leg'),
|
||||
h('td', { class: 'desc muted' }, l.want)))));
|
||||
}
|
||||
refresh();
|
||||
return { refresh };
|
||||
}
|
||||
|
||||
// --- keyboard -------------------------------------------------------------
|
||||
// --- tags -----------------------------------------------------------------
|
||||
|
||||
// The TUI's keys, where they do not fight the browser. A builder is a form,
|
||||
// so outside of chords its keys belong to the field being typed in — exactly
|
||||
// the rule the TUI keeps.
|
||||
document.addEventListener('keydown', (e) => {
|
||||
const t = e.target;
|
||||
const typing = t instanceof HTMLInputElement || t instanceof HTMLSelectElement || t instanceof HTMLTextAreaElement;
|
||||
if (current && current.key && current.key(e, typing)) {
|
||||
e.preventDefault();
|
||||
// mountTags lists every distinct tag with how many transactions carry it and
|
||||
// the rules that write it, numbered by file position as on the rules screen.
|
||||
function mountTags(main) {
|
||||
const title = h('h2');
|
||||
const body = h('div');
|
||||
main.append(title, body);
|
||||
|
||||
async function refresh() {
|
||||
let tags;
|
||||
try {
|
||||
tags = (await api('GET', '/api/tags')).tags;
|
||||
} catch (e) { setError(e); return; }
|
||||
title.replaceChildren('Tags ', h('span', { class: 'sub' }, `· ${tags.length} tags`));
|
||||
if (!tags.length) {
|
||||
body.replaceChildren(h('div', { class: 'panel empty' }, 'No tags yet.\n\nA tag comes from a rule: build one on the rule builder.'));
|
||||
return;
|
||||
}
|
||||
if (typing || e.ctrlKey || e.metaKey || e.altKey) return;
|
||||
|
||||
if (e.key >= '1' && e.key <= String(VIEWS.length)) {
|
||||
go(VIEWS[Number(e.key) - 1].id);
|
||||
} else if (e.key === '/') {
|
||||
if (state.view !== 'txns' && state.view !== 'report') go('txns');
|
||||
setTimeout(() => { const s = document.getElementById('search'); if (s) s.focus(); });
|
||||
} else if (e.key === 'u') {
|
||||
state.filter.untagged = !state.filter.untagged;
|
||||
if (state.view === 'txns' || state.view === 'report') show();
|
||||
else go('txns');
|
||||
} else if (e.key === 'a') {
|
||||
state.filter.account = '';
|
||||
if (state.view === 'txns' || state.view === 'report') show();
|
||||
} else if (e.key === 'i') {
|
||||
runImport(false);
|
||||
} else if (e.key === 'r') {
|
||||
retag();
|
||||
} else {
|
||||
return;
|
||||
body.replaceChildren(h('div', { class: 'panel' }, h('table', {},
|
||||
h('thead', {}, h('tr', {}, h('th', {}, 'Tag'), h('th', { class: 'num' }, 'Txns'), h('th', {}, 'Rules'))),
|
||||
h('tbody', {}, tags.map((t) => h('tr', {},
|
||||
h('td', { class: t.transactions ? null : 'muted' }, t.tag),
|
||||
h('td', { class: 'num' + (t.transactions ? '' : ' muted') }, t.transactions),
|
||||
h('td', { class: 'desc' }, t.rules.length
|
||||
? t.rules.map((r) => h('span', { class: 'tag-rule' + (r.usage ? '' : ' muted') },
|
||||
h('span', { class: 'muted' }, `#${r.pos + 1} `),
|
||||
h('span', { class: 'mono' }, r.pattern),
|
||||
r.account ? h('span', { class: 'muted' }, ` on ${r.account}`) : null,
|
||||
` (${r.usage})`))
|
||||
: h('span', { class: 'warn' }, 'no rule writes it — Retag clears it'))))))));
|
||||
}
|
||||
refresh();
|
||||
return { refresh };
|
||||
}
|
||||
e.preventDefault();
|
||||
});
|
||||
|
||||
document.getElementById('import').addEventListener('click', (e) => runImport(e.shiftKey));
|
||||
document.getElementById('import').title += ' (shift-click re-parses unchanged files too)';
|
||||
document.getElementById('import').addEventListener('click', () => runImport(false));
|
||||
document.getElementById('reread').addEventListener('click', () => runImport(true));
|
||||
document.getElementById('retag').addEventListener('click', retag);
|
||||
window.addEventListener('hashchange', show);
|
||||
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
|
||||
<rect width="32" height="32" rx="7" fill="#4b46c8"/>
|
||||
<g fill="#fff">
|
||||
<rect x="7" y="8" width="18" height="4" rx="2"/>
|
||||
<rect x="7" y="14" width="11" height="4" rx="2"/>
|
||||
<rect x="7" y="20" width="15" height="4" rx="2"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 309 B |
@@ -4,6 +4,7 @@
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>money</title>
|
||||
<link rel="icon" href="icon.svg" type="image/svg+xml">
|
||||
<link rel="stylesheet" href="style.css">
|
||||
</head>
|
||||
<body>
|
||||
@@ -12,6 +13,7 @@
|
||||
<nav id="nav" aria-label="Screens"></nav>
|
||||
<div class="actions">
|
||||
<button type="button" id="retag" title="Re-apply rules.toml: retag everything and re-pair transfers">Retag</button>
|
||||
<button type="button" id="reread" title="Re-read every statement, including ones unchanged since they were imported">Re-read all</button>
|
||||
<button type="button" id="import" class="primary" title="Import new statements from every account folder">Import</button>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
@@ -68,7 +68,6 @@ button, input, select { font: inherit; color: inherit; }
|
||||
}
|
||||
#nav a:hover { color: var(--text); background: var(--row-hover); }
|
||||
#nav a.active { color: var(--accent); background: var(--accent-soft); font-weight: 600; }
|
||||
#nav kbd { font-family: var(--mono); font-size: 11px; opacity: .6; margin-right: 4px; }
|
||||
.actions { display: flex; gap: 8px; }
|
||||
|
||||
button {
|
||||
@@ -145,6 +144,11 @@ tr.total td { font-weight: 600; border-top: 1px solid var(--border); }
|
||||
tr.excluded td { color: var(--muted); }
|
||||
tr.excluded.first td { border-top: 1px dashed var(--border); }
|
||||
td.actions { text-align: right; }
|
||||
.mono { font-family: var(--mono); }
|
||||
.tag-rule { display: inline-block; margin-right: 14px; white-space: nowrap; }
|
||||
tr.legs > td { padding: 0 10px 10px 36px; background: var(--bg); }
|
||||
tr.legs:hover { background: none; }
|
||||
table.sub th { background: var(--bg); font-weight: 500; color: var(--muted); }
|
||||
td.actions button { padding: 1px 8px; font-size: 12px; }
|
||||
|
||||
.empty { padding: 32px 16px; color: var(--muted); white-space: pre-line; text-align: center; }
|
||||
@@ -176,6 +180,31 @@ td.actions button { padding: 1px 8px; font-size: 12px; }
|
||||
.kept { font-size: 12px; color: var(--muted); font-family: var(--mono); }
|
||||
.preview { max-height: calc(100vh - 170px); }
|
||||
|
||||
h3.section { font-size: 14px; font-weight: 600; margin: 20px 0 8px; }
|
||||
h3.section .sub { color: var(--muted); font-weight: 400; }
|
||||
.files a { color: var(--accent); text-decoration: none; }
|
||||
.files a:hover { text-decoration: underline; }
|
||||
|
||||
.upload { margin-top: 16px; padding: 14px; overflow: visible; }
|
||||
.upload h3 { font-size: 14px; margin: 0 0 10px; }
|
||||
.upload .toolbar { margin-bottom: 10px; }
|
||||
.dropzone {
|
||||
border: 2px dashed var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 22px 16px;
|
||||
text-align: center;
|
||||
color: var(--muted);
|
||||
cursor: pointer;
|
||||
}
|
||||
.dropzone:hover, .dropzone:focus, .dropzone.over {
|
||||
border-color: var(--accent);
|
||||
color: var(--accent);
|
||||
background: var(--accent-soft);
|
||||
outline: none;
|
||||
}
|
||||
.upload .chosen { margin-top: 8px; font-family: var(--mono); font-size: 12px; overflow-wrap: anywhere; }
|
||||
.upload .hint { margin-top: 8px; font-size: 12px; }
|
||||
|
||||
.import-files { margin-top: 6px; font-size: 13px; }
|
||||
.import-files li { font-family: var(--mono); }
|
||||
|
||||
|
||||
@@ -0,0 +1,442 @@
|
||||
package web
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.petrovv.com/nikola/money/internal/config"
|
||||
"git.petrovv.com/nikola/money/internal/parser"
|
||||
"git.petrovv.com/nikola/money/internal/rules"
|
||||
"git.petrovv.com/nikola/money/internal/store"
|
||||
"git.petrovv.com/nikola/money/internal/transfers"
|
||||
)
|
||||
|
||||
// The upload tests are about where files land and what is refused, not about
|
||||
// any bank's layout, so they read "date,description,amount" with a header.
|
||||
func init() {
|
||||
parser.Register("webtest", func(acc *config.Account) (parser.Parser, error) {
|
||||
return csvParser{digits: acc.Digits()}, nil
|
||||
})
|
||||
parser.Register("webnamed", func(acc *config.Account) (parser.Parser, error) {
|
||||
return namingParser{csvParser{digits: acc.Digits()}}, nil
|
||||
})
|
||||
}
|
||||
|
||||
type csvParser struct{ digits int }
|
||||
|
||||
// namingParser is csvParser for a bank whose downloads are named unhelpfully:
|
||||
// it names a statement after its first date, as nlb does after the statement
|
||||
// date, and says nothing for a statement with no rows.
|
||||
type namingParser struct{ csvParser }
|
||||
|
||||
func (namingParser) StatementName(path string) (string, error) {
|
||||
body, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
lines := strings.Split(strings.TrimSpace(string(body)), "\n")
|
||||
if len(lines) < 2 {
|
||||
return "", nil
|
||||
}
|
||||
return "stmt_" + strings.Split(lines[1], ",")[0], nil
|
||||
}
|
||||
|
||||
func (p csvParser) Parse(path string, _ *config.Account) ([]parser.RawTxn, error) {
|
||||
body, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var txns []parser.RawTxn
|
||||
for i, line := range strings.Split(strings.TrimSpace(string(body)), "\n")[1:] {
|
||||
f := strings.Split(line, ",")
|
||||
if len(f) != 3 {
|
||||
return nil, fmt.Errorf("row %d: want 3 fields", i+2)
|
||||
}
|
||||
amount, err := parser.ParseAmount(f[2], ".", "", p.digits)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
txns = append(txns, parser.RawTxn{Date: f[0], Description: f[1], AmountMinor: amount})
|
||||
}
|
||||
return txns, nil
|
||||
}
|
||||
|
||||
// newUploadServer builds a server over a data root with one empty account
|
||||
// folder, as `money serve` sees it before the first import.
|
||||
func newUploadServer(t *testing.T, accountTOML string) (string, http.Handler) {
|
||||
t.Helper()
|
||||
root := t.TempDir()
|
||||
dir := filepath.Join(root, "checking")
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, config.AccountFile), []byte(accountTOML), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
db, err := store.Open(config.IndexPath(root))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { db.Close() })
|
||||
accounts, err := config.LoadAccounts(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
loaded, err := config.LoadRules(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s := New(root, db, accounts, rules.New(loaded), transfers.New(loaded))
|
||||
s.now = func() time.Time { return time.Date(2026, 3, 15, 0, 0, 0, 0, time.UTC) }
|
||||
return dir, s.Handler()
|
||||
}
|
||||
|
||||
const checkingTOML = "currency = \"EUR\"\nparser = \"webtest\"\n"
|
||||
|
||||
const statement = "date,description,amount\n2026-02-01,LIDL SOFIA,-12.50\n2026-02-03,SALARY,1000.00\n"
|
||||
|
||||
// An upload only saves: the file waits on the statements list as new until
|
||||
// the user presses Import.
|
||||
func TestUploadSavesWithoutImporting(t *testing.T) {
|
||||
dir, h := newUploadServer(t, checkingTOML)
|
||||
req := uploadReq{Account: "checking", Files: []uploadFile{{Name: "2026-02.csv", Data: []byte(statement)}}}
|
||||
|
||||
var res status
|
||||
call(t, h, "POST", "/api/upload", req, http.StatusOK, &res)
|
||||
if res.Status != "uploaded 1 file(s) to checking · press Import to read them" {
|
||||
t.Errorf("status = %q", res.Status)
|
||||
}
|
||||
if got, err := os.ReadFile(filepath.Join(dir, "2026-02.csv")); err != nil || string(got) != statement {
|
||||
t.Errorf("file on disk = %q, %v", got, err)
|
||||
}
|
||||
if got := descriptions(t, h); got != "" {
|
||||
t.Errorf("upload imported %s; it should only save", got)
|
||||
}
|
||||
var files struct{ Files []fileRow }
|
||||
call(t, h, "GET", "/api/files", nil, http.StatusOK, &files)
|
||||
if len(files.Files) != 1 || files.Files[0].Status != "new" {
|
||||
t.Errorf("files = %+v, want the upload waiting as new", files.Files)
|
||||
}
|
||||
|
||||
call(t, h, "POST", "/api/import", importReq{}, http.StatusOK, nil)
|
||||
if got := descriptions(t, h); got != "SALARY,LIDL SOFIA" {
|
||||
t.Errorf("after Import the index holds %s", got)
|
||||
}
|
||||
|
||||
// The same file again is not an error, and saves nothing.
|
||||
call(t, h, "POST", "/api/upload", req, http.StatusOK, &res)
|
||||
if res.Status != "uploaded 0 file(s) to checking (1 already there)" {
|
||||
t.Errorf("re-upload status = %q", res.Status)
|
||||
}
|
||||
}
|
||||
|
||||
// A statement already in the folder is the source of truth for what it
|
||||
// imported, so an upload never replaces one with different contents.
|
||||
func TestUploadNeverReplacesAStatement(t *testing.T) {
|
||||
dir, h := newUploadServer(t, checkingTOML)
|
||||
path := filepath.Join(dir, "2026-02.csv")
|
||||
if err := os.WriteFile(path, []byte(statement), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
req := uploadReq{Account: "checking", Files: []uploadFile{{Name: "2026-02.csv", Data: []byte("date,description,amount\n")}}}
|
||||
call(t, h, "POST", "/api/upload", req, http.StatusConflict, nil)
|
||||
if got, _ := os.ReadFile(path); string(got) != statement {
|
||||
t.Errorf("statement was overwritten: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A batch is checked as a whole before anything is written, and nothing may
|
||||
// land outside the account folder or where import would not read it back.
|
||||
func TestUploadRefusesBadNames(t *testing.T) {
|
||||
dir, h := newUploadServer(t, checkingTOML+"include = [\"*.csv\"]\n")
|
||||
for _, bad := range []string{"../escape.csv", "sub/x.csv", ".hidden.csv", config.AccountFile, "notes.txt", ""} {
|
||||
req := uploadReq{Account: "checking", Files: []uploadFile{
|
||||
{Name: "good.csv", Data: []byte(statement)},
|
||||
{Name: bad, Data: []byte(statement)},
|
||||
}}
|
||||
call(t, h, "POST", "/api/upload", req, http.StatusBadRequest, nil)
|
||||
}
|
||||
entries, _ := os.ReadDir(dir)
|
||||
if len(entries) != 1 {
|
||||
var names []string
|
||||
for _, e := range entries {
|
||||
names = append(names, e.Name())
|
||||
}
|
||||
t.Errorf("folder holds %v, want only %s", names, config.AccountFile)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(filepath.Dir(dir), "escape.csv")); err == nil {
|
||||
t.Error("a file escaped the account folder")
|
||||
}
|
||||
|
||||
call(t, h, "POST", "/api/upload", uploadReq{Account: "nope", Files: []uploadFile{{Name: "a.csv"}}},
|
||||
http.StatusBadRequest, nil)
|
||||
}
|
||||
|
||||
// Multipart is what a cross-site form can send, so it is refused like any
|
||||
// other non-JSON write.
|
||||
func TestUploadRefusesMultipart(t *testing.T) {
|
||||
_, h := newUploadServer(t, checkingTOML)
|
||||
r := httptest.NewRequest("POST", "/api/upload", strings.NewReader("--x\r\n"))
|
||||
r.Header.Set("Content-Type", "multipart/form-data; boundary=x")
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, r)
|
||||
if w.Code != http.StatusUnsupportedMediaType {
|
||||
t.Errorf("status %d, want %d", w.Code, http.StatusUnsupportedMediaType)
|
||||
}
|
||||
}
|
||||
|
||||
// The statements list is the folders, each beside what the index recorded:
|
||||
// imported, changed since, not imported yet, and gone from disk.
|
||||
func TestFileListStatuses(t *testing.T) {
|
||||
dir, h := newUploadServer(t, checkingTOML)
|
||||
write := func(name, body string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write("a.csv", statement)
|
||||
write("b.csv", "date,description,amount\n2026-03-01,ZARA,-40.00\n")
|
||||
write("c.csv", "date,description,amount\n2026-04-01,KAUFLAND,-9.00\n")
|
||||
call(t, h, "POST", "/api/import", importReq{}, http.StatusOK, nil)
|
||||
|
||||
write("b.csv", "date,description,amount\n2026-03-01,ZARA,-41.00\n")
|
||||
if err := os.Remove(filepath.Join(dir, "c.csv")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
write("d.csv", statement)
|
||||
write(".hidden.csv", statement)
|
||||
|
||||
var res struct{ Files []fileRow }
|
||||
call(t, h, "GET", "/api/files", nil, http.StatusOK, &res)
|
||||
got := map[string]fileRow{}
|
||||
var names []string
|
||||
for _, f := range res.Files {
|
||||
got[f.Name] = f
|
||||
names = append(names, f.Name)
|
||||
}
|
||||
if strings.Join(names, " ") != "a.csv b.csv c.csv d.csv" {
|
||||
t.Fatalf("files = %v, want the four statements and nothing import would skip", names)
|
||||
}
|
||||
for name, want := range map[string]string{"a.csv": "imported", "b.csv": "changed", "c.csv": "missing", "d.csv": "new"} {
|
||||
if got[name].Status != want {
|
||||
t.Errorf("%s status = %q, want %q", name, got[name].Status, want)
|
||||
}
|
||||
}
|
||||
if got["a.csv"].Rows != 2 || got["a.csv"].Added != 2 || got["c.csv"].Rows != 1 || got["c.csv"].Added != 1 ||
|
||||
got["a.csv"].ImportedAt == "" {
|
||||
t.Errorf("a.csv = %+v, c.csv = %+v", got["a.csv"], got["c.csv"])
|
||||
}
|
||||
if got["a.csv"].Size != int64(len(statement)) {
|
||||
t.Errorf("a.csv size = %d, want %d", got["a.csv"].Size, len(statement))
|
||||
}
|
||||
}
|
||||
|
||||
// Only a file import would read is served, and never as a page.
|
||||
func TestServeFileServesOnlyStatements(t *testing.T) {
|
||||
dir, h := newUploadServer(t, checkingTOML)
|
||||
for name, body := range map[string]string{"a.csv": statement, "page.html": "<script>alert(1)</script>", ".secret": "x"} {
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
get := func(path string) *httptest.ResponseRecorder {
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, httptest.NewRequest("GET", path, nil))
|
||||
return w
|
||||
}
|
||||
|
||||
w := get("/api/files/checking/a.csv")
|
||||
if w.Code != http.StatusOK || w.Body.String() != statement {
|
||||
t.Fatalf("a.csv: %d %q", w.Code, w.Body.String())
|
||||
}
|
||||
if ct := w.Header().Get("Content-Type"); ct != "text/plain; charset=utf-8" {
|
||||
t.Errorf("a.csv content type = %q", ct)
|
||||
}
|
||||
|
||||
w = get("/api/files/checking/page.html")
|
||||
if w.Code != http.StatusOK || w.Header().Get("Content-Type") != "application/octet-stream" ||
|
||||
w.Header().Get("Content-Security-Policy") != "sandbox" ||
|
||||
!strings.HasPrefix(w.Header().Get("Content-Disposition"), "attachment") {
|
||||
t.Errorf("page.html served as %d %v, want a sandboxed download", w.Code, w.Header())
|
||||
}
|
||||
|
||||
for _, path := range []string{
|
||||
"/api/files/checking/" + config.AccountFile,
|
||||
"/api/files/checking/.secret",
|
||||
"/api/files/checking/..%2F" + config.RulesFile,
|
||||
"/api/files/nope/a.csv",
|
||||
} {
|
||||
if w := get(path); w.Code != http.StatusNotFound {
|
||||
t.Errorf("%s: status %d, want 404", path, w.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A parser that names its statements decides the stored name, so the bank's
|
||||
// "download (3).pdf" lands as something that sorts by date. A reissue of the
|
||||
// same date is numbered rather than refused, and a statement that does not
|
||||
// say keeps the name it came with.
|
||||
func TestUploadNamesStatementsTheParserCanName(t *testing.T) {
|
||||
dir, h := newUploadServer(t, "currency = \"EUR\"\nparser = \"webnamed\"\n")
|
||||
upload := func(files ...uploadFile) string {
|
||||
t.Helper()
|
||||
var res status
|
||||
call(t, h, "POST", "/api/upload", uploadReq{Account: "checking", Files: files}, http.StatusOK, &res)
|
||||
return res.Status
|
||||
}
|
||||
reissue := "date,description,amount\n2026-02-01,LIDL SOFIA,-12.50\n2026-02-04,ZARA,-30.00\n"
|
||||
other := "date,description,amount\n2026-03-01,KAUFLAND,-9.00\n"
|
||||
|
||||
status := upload(uploadFile{Name: "download (3).csv", Data: []byte(statement)})
|
||||
if !strings.Contains(status, "download (3).csv → stmt_2026-02-01.csv") {
|
||||
t.Errorf("status = %q, want the rename named", status)
|
||||
}
|
||||
// The same statement downloaded again is recognised under its new name.
|
||||
if status := upload(uploadFile{Name: "download (4).csv", Data: []byte(statement)}); !strings.Contains(status, "1 already there") {
|
||||
t.Errorf("second download: status = %q", status)
|
||||
}
|
||||
// A different statement of the same date, and two of one date in a batch.
|
||||
upload(uploadFile{Name: "x.csv", Data: []byte(reissue)},
|
||||
uploadFile{Name: "y.csv", Data: []byte(reissue + "2026-02-05,BOLT,-4.00\n")},
|
||||
uploadFile{Name: "Z.CSV", Data: []byte(other)}, // a chosen name is lowercase
|
||||
uploadFile{Name: "empty.csv", Data: []byte("date,description,amount\n")})
|
||||
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var names []string
|
||||
for _, e := range entries {
|
||||
names = append(names, e.Name())
|
||||
}
|
||||
want := config.AccountFile + " empty.csv stmt_2026-02-01.csv stmt_2026-02-01_2.csv stmt_2026-02-01_3.csv stmt_2026-03-01.csv"
|
||||
if strings.Join(names, " ") != want {
|
||||
t.Errorf("folder = %v, want %s", names, want)
|
||||
}
|
||||
}
|
||||
|
||||
func writeFile(t *testing.T, path, body string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func descriptions(t *testing.T, h http.Handler) string {
|
||||
t.Helper()
|
||||
var txns struct{ Rows []txnRow }
|
||||
call(t, h, "GET", "/api/transactions", nil, http.StatusOK, &txns)
|
||||
var out []string
|
||||
for _, r := range txns.Rows {
|
||||
out = append(out, r.Description)
|
||||
}
|
||||
return strings.Join(out, ",")
|
||||
}
|
||||
|
||||
// Removing a statement deletes the file and takes out what it brought in. A
|
||||
// row an overlapping statement also holds was stored under the file imported
|
||||
// first; the next Import brings it back from the other one.
|
||||
func TestRemoveFileKeepsWhatAnotherStatementHolds(t *testing.T) {
|
||||
dir, h := newUploadServer(t, checkingTOML)
|
||||
writeFile(t, filepath.Join(dir, "a.csv"), "date,description,amount\n2026-01-30,ONLY IN A,-1.00\n2026-02-01,SHARED,-2.00\n")
|
||||
writeFile(t, filepath.Join(dir, "b.csv"), "date,description,amount\n2026-02-01,SHARED,-2.00\n2026-02-03,ONLY IN B,-3.00\n")
|
||||
call(t, h, "POST", "/api/import", importReq{}, http.StatusOK, nil)
|
||||
|
||||
// Each holds two rows; b.csv was imported second, so its shared row
|
||||
// stayed with a.csv and it brought in only one.
|
||||
var before struct{ Files []fileRow }
|
||||
call(t, h, "GET", "/api/files", nil, http.StatusOK, &before)
|
||||
if f := before.Files; len(f) != 2 || f[0].Rows != 2 || f[0].Added != 2 || f[1].Rows != 2 || f[1].Added != 1 {
|
||||
t.Errorf("files = %+v, want a.csv 2 rows · 2 new and b.csv 2 rows · 1 new", f)
|
||||
}
|
||||
|
||||
var res status
|
||||
call(t, h, "POST", "/api/files/delete", removeFileReq{Account: "checking", Name: "a.csv"}, http.StatusOK, &res)
|
||||
if !strings.HasPrefix(res.Status, "deleted checking/a.csv, 2 transaction(s) dropped") {
|
||||
t.Errorf("status = %q", res.Status)
|
||||
}
|
||||
// Deleting does not import, so the shared row is gone for now, and b.csv
|
||||
// says it is due to be read again.
|
||||
if got := descriptions(t, h); got != "ONLY IN B" {
|
||||
t.Errorf("index holds %s right after the delete, want only b's own row", got)
|
||||
}
|
||||
var files struct{ Files []fileRow }
|
||||
call(t, h, "GET", "/api/files", nil, http.StatusOK, &files)
|
||||
if len(files.Files) != 1 || files.Files[0].Status != "changed" {
|
||||
t.Errorf("files = %+v, want b.csv marked to be re-read", files.Files)
|
||||
}
|
||||
|
||||
call(t, h, "POST", "/api/import", importReq{}, http.StatusOK, nil)
|
||||
if got := descriptions(t, h); got != "ONLY IN B,SHARED" {
|
||||
t.Errorf("after Import the index holds %s, want the shared row restored from b", got)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "a.csv")); !errors.Is(err, fs.ErrNotExist) {
|
||||
t.Errorf("a.csv is still on disk: %v", err)
|
||||
}
|
||||
if entries, _ := os.ReadDir(dir); len(entries) != 2 {
|
||||
t.Errorf("folder holds %d entries, want account.toml and b.csv only", len(entries))
|
||||
}
|
||||
|
||||
call(t, h, "GET", "/api/files", nil, http.StatusOK, &files)
|
||||
if len(files.Files) != 1 || files.Files[0].Name != "b.csv" || files.Files[0].Status != "imported" {
|
||||
t.Errorf("files = %+v, want only b.csv, imported", files.Files)
|
||||
}
|
||||
}
|
||||
|
||||
// A removed leg takes its transfer pairing with it, and the other leg is left
|
||||
// unpaired — which is the truth once one side is gone.
|
||||
func TestRemoveFileUnpairsItsTransfers(t *testing.T) {
|
||||
dir, h := newUploadServer(t, checkingTOML)
|
||||
writeFile(t, filepath.Join(filepath.Dir(dir), config.RulesFile), `
|
||||
[[transfer]]
|
||||
from_account = "checking"
|
||||
from_desc = "*OUT*"
|
||||
to_account = "checking"
|
||||
to_desc = "*IN*"
|
||||
`)
|
||||
writeFile(t, filepath.Join(dir, "out.csv"), "date,description,amount\n2026-02-01,MOVE OUT,-5.00\n")
|
||||
writeFile(t, filepath.Join(dir, "in.csv"), "date,description,amount\n2026-02-02,MOVE IN,5.00\n")
|
||||
call(t, h, "POST", "/api/import", importReq{}, http.StatusOK, nil)
|
||||
|
||||
var res status
|
||||
call(t, h, "POST", "/api/files/delete", removeFileReq{Account: "checking", Name: "in.csv"}, http.StatusOK, &res)
|
||||
if !strings.Contains(res.Status, "1 transfer leg(s) unpaired") {
|
||||
t.Errorf("status = %q, want the remaining leg reported unpaired", res.Status)
|
||||
}
|
||||
}
|
||||
|
||||
// A file already gone from disk is forgotten by the index; a name the index
|
||||
// never had is not found.
|
||||
func TestRemoveFileForgetsAMissingOne(t *testing.T) {
|
||||
dir, h := newUploadServer(t, checkingTOML)
|
||||
writeFile(t, filepath.Join(dir, "a.csv"), statement)
|
||||
call(t, h, "POST", "/api/import", importReq{}, http.StatusOK, nil)
|
||||
if err := os.Remove(filepath.Join(dir, "a.csv")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var res status
|
||||
call(t, h, "POST", "/api/files/delete", removeFileReq{Account: "checking", Name: "a.csv"}, http.StatusOK, &res)
|
||||
if !strings.HasPrefix(res.Status, "forgot checking/a.csv, 2 transaction(s) dropped") {
|
||||
t.Errorf("status = %q", res.Status)
|
||||
}
|
||||
if got := descriptions(t, h); got != "" {
|
||||
t.Errorf("index still holds %s", got)
|
||||
}
|
||||
|
||||
call(t, h, "POST", "/api/files/delete", removeFileReq{Account: "checking", Name: "a.csv"}, http.StatusNotFound, nil)
|
||||
call(t, h, "POST", "/api/files/delete", removeFileReq{Account: "checking", Name: "../rules.toml"}, http.StatusBadRequest, nil)
|
||||
if _, err := os.Stat(filepath.Join(dir, config.AccountFile)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
call(t, h, "POST", "/api/files/delete", removeFileReq{Account: "checking", Name: config.AccountFile}, http.StatusNotFound, nil)
|
||||
}
|
||||
Reference in New Issue
Block a user