nikolaandClaude Opus 5.5
0324ff7f2a
Scrapes slo-tech.com (Ljubljana) every 24h, stores jobs in SQLite and tracks which are new vs. seen. New sites can be added as modules in src/scrapers/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
42 lines
1.6 KiB
Markdown
42 lines
1.6 KiB
Markdown
# Job Board
|
|
|
|
Scrapes job sites once a day, stores results in SQLite (`data/jobs.sqlite`), and shows them in a web UI where you can mark jobs as seen.
|
|
|
|
No runtime dependencies. It uses Bun's built-in SQLite, HTTP server and HTMLRewriter.
|
|
|
|
## Run
|
|
|
|
```sh
|
|
bun install
|
|
bun start # http://localhost:3000
|
|
```
|
|
|
|
The server scrapes every 24h. On startup it waits out the rest of the 24h since the last scrape (or scrapes right away if none happened in the last 24h).
|
|
You can also click **Scrape now** in the UI, or run `bun run scrape` once (e.g. from cron).
|
|
|
|
Env vars: `PORT` (3000), `SCRAPE_INTERVAL_HOURS` (24), `DB_PATH` (`data/jobs.sqlite`).
|
|
|
|
## UI
|
|
|
|
- **New / Seen / All** tabs. Opening a job, or clicking *Mark seen*, moves it to Seen.
|
|
- **no longer listed** means the job was missing from that source's latest successful scrape.
|
|
- Search filters on title, company and tags. Recent scrape runs (and errors) are listed at the bottom.
|
|
|
|
## Adding a site
|
|
|
|
1. Create `src/scrapers/<site>.ts` exporting an object that implements `Scraper` (`src/scrapers/types.ts`):
|
|
```ts
|
|
export const mySite: Scraper = {
|
|
id: "my-site", // stable, stored in DB
|
|
name: "My Site",
|
|
async scrape() {
|
|
const html = await fetchHtml("https://..."); // helper in ./http.ts
|
|
return [/* { externalId, title, url, company?, location?, tags?, postedAt? } */];
|
|
},
|
|
};
|
|
```
|
|
2. Add it to the list in `src/scrapers/index.ts`.
|
|
|
|
`externalId` must stay stable for a job, because it is what decides whether a job is new.
|
|
For another slo-tech city, just add `sloTech("Maribor")` to the list.
|