job/scrapper/README.md

Job Board

Scrapes job sites once a day, stores results in SQLite (data/jobs.sqlite), and shows them in a web UI where you can mark jobs as seen.

No runtime dependencies. It uses Bun's built-in SQLite, HTTP server and HTMLRewriter.

Sources

Source	How	Notes
slo-tech.com (Ljubljana)	HTML scrape	~20 newest jobs
startup.si	HTML scrape (no public API or feed)	Site lists every posting since 2019 on one page, so only the last 90 days are kept (startupSi(days) in src/scrapers/index.ts)
adecco.com (Slovenia, IT)	Internal JSON API the site's job search uses (no public API)	Filtered by category; add more with adecco("<CategoryFilter value>", "<label>")
mojedelo.com (IT, Osrednjeslovenska)	Internal JSON API the site's search uses (no public API)	Takes a search URL copied from the site; add more with mojeDelo("<label>", "<search URL>") (don't rename the label later, it's part of the source ID)
optius.com (IT, Osrednjeslovenska)	Official RSS feed	Build a feed URL at https://www.optius.com/rss/ (its IDs differ from the search page's); add more with optius("<label>", "<feed URL>") (same label rule as MojeDelo)

Run

bun install
bun start            # http://localhost:3000

The server scrapes every 24h. On startup it waits out the rest of the 24h since the last scrape (or scrapes right away if none happened in the last 24h). You can also click Scrape now in the UI, or run bun run scrape once (e.g. from cron).

Env vars: PORT (3000), SCRAPE_INTERVAL_HOURS (24), DB_PATH (data/jobs.sqlite).

UI

• New / Seen / All tabs. Opening a job, or clicking Mark seen, moves it to Seen.
• no longer listed means the job was missing from that source's latest successful scrape.
• Search filters on title, company and tags. Recent scrape runs (and errors) are listed at the bottom.

Adding a site

1. Create src/scrapers/<site>.ts exporting an object that implements Scraper (src/scrapers/types.ts):

   export const mySite: Scraper = {
     id: "my-site",          // stable, stored in DB
     name: "My Site",
     async scrape() {
       const html = await fetchHtml("https://...");   // helper in ./http.ts
       return [/* { externalId, title, url, company?, location?, tags?, postedAt? } */];
     },
   };
   

2. Add it to the list in src/scrapers/index.ts.

externalId must stay stable for a job, because it is what decides whether a job is new. For another slo-tech city, just add sloTech("Maribor") to the list.