The problem
Problem: I was triaging about 150 offers by hand, and keyword alerts could not tell a good fit from a near miss.
Objective: a search tailored to me. Each morning, a top 5 of fresh offers (≤14 days), scored 0–100 with evidence, aiming to rank them the way I would. Read-only: it never applies or sends.
How the pipeline works
Why only 30 offers reach the LLM. Cheap steps run first: rules, freshness and similarity to the profile are instant and free. The local judge is slow, so it only sees the 30 most promising offers. The risk is a good offer cut before the judge, which the weekly check of rejected offers is there to catch.
Who does what
| Claude Code | Designs and writes the code, tests and run scripts. In the search pipeline, it never sees my profile data at runtime. |
|---|---|
| Local AI | LM Studio runs the judge, Gemma 4 26B today after starting with Qwen3.5-9B, and nomic for embeddings. ChromaDB and SQLite are local, so profile and scores stay on my PC during a search. |
| Orchestration | LangGraph, with a Dify version using the same tools via a local API. Arize Phoenix traces every step. |
| Scheduling | Windows task at logon and daily at 08:00, guarded to run once a day. |
How I keep it reliable
- Measure: per-run metrics (found per source, scored, shown, in queue, retries, unverified, time). A typical run: about 350 offers found, 30 scored by the LLM, 5 shown.
- Find another route: if a source is down, the run goes on with the others. If the LLM is down, it falls back to rules + similarity, capped at class B.
- Bounded: step limit, shortlist of 30, max 2 LLM retries.
- Check, don't trust: output is validated with Pydantic, and an independent verify step halves the score when evidence isn't found.
- Minimal rights: HTTP GET and localhost only. No destructive actions exist.
How it learns from my decisions
Every decision I take in the dashboard becomes a label: favorite or applied is positive, disqualified is negative, with a short reason. The judge reads these, flags offers that look like a past rejection and lowers their score.
So far: more than 100 reviewed offers, so the first gate below is passed. The first real agreement measurement is the next update on this page.
Now: more users, and offline on my phone
Since November the radar has accounts. Each person imports a CV or a LinkedIn PDF export, and that becomes their own profile. Each account runs its own search and only sees its own data. Other accounts use the local model only, so nothing leaves the PC.
When the PC is off, a copy of the dashboard on my Synology NAS lets me keep sorting offers from my phone. My decisions sync back when the PC starts again. Both stay on the home network, not on the internet.
What unlocks the next step
| Gate 1 · 50 reviews | First measurement: judge agreement and a precision@5 baseline (how many of the daily top 5 I would really keep). Passed in November 2026. |
|---|---|
| Gate 2 · 10–15 days | Daily marking of the top 5, plus a weekly check of rejected offers to catch false negatives. |
| Then · tune | Tune the weights of rules, similarity and LLM score, and the A/B/C thresholds. Keep a change only if precision@5 goes up. |
What I learned
The first judge, Qwen3.5-9B, clustered its scores (78–84), so it barely told offers apart. I benchmarked Gemma 4 26B on the same offers and switched the judge to it, while rules and similarity still carry much of the signal. The bigger lesson: a model doesn't know what I want until I tell it. Written disqualify reasons turned out to be the most useful feedback, because the judge can read them and cite them. Early agreement, below 50%, showed the prompt alone did not capture my judgement, so the next step is measurement, not more prompt tweaks. Tuning without data is guessing.
