The problem
The marketplace competes with six other lead-marketplace platforms. Nobody was systematically watching what any of them changed — pricing, funnel copy, positioning, hiring. Competitive intelligence was whoever happened to notice something.
So I built the noticing.
Every Monday morning, GitHub Actions scrapes all six. An LLM analyst interprets the diffs and maintains a per-competitor profile.yaml "world model". Two cloud routines then deliver a stakeholder Slack digest and per-competitor Confluence deep-dive pages.
How it's built
Stack: Python — Pydantic v2, Playwright, trafilatura, httpx, pytest — with all AI routed through OpenRouter. Storage is files on disk with git history as the changelog. No database. Any week's state, and exactly how it changed, is inspectable with git log.
Three subsystems worth calling out:
- Sitemap-first discovery with relevance scoring, so the crawl targets what matters instead of everything
- A deterministic, PII-safe funnel walker that steps through competitors' multi-step quote wizards and diffs the copy week over week. Changes to a competitor's funnel questions are a leading indicator of a strategy change.
- A vacancies walker with per-ATS parsers (Greenhouse, SmartRecruiters, Teamtailor, Ashby, Homerun), feeding hiring signals into strategic predictions per competitor. What a company is hiring for is what it's about to do.
The architecture split that made it work
The system is deliberately cut along credential model, not along function:
- The dumb, reliable scrape runs in GitHub Actions — which has a secret store (one secret: the OpenRouter key) but no MCP connectors
- The LLM analysis and delivery run as cloud routines — which have Slack and Confluence connectors but no env-var store
The consequence: no delivery credentials live in the repo at all. That constraint initially looked like an obstacle. Designing around it produced a cleaner system than fighting it would have.
Outcomes
- In production since May 4th: weekly digests every Monday, Confluence competitor pages under the company wiki, hiring-signal predictions per competitor
- A living per-competitor world model in git, so any week's state is inspectable and diffable
- Spec committed and first weekly run in the same week — one design day from idea to production
What we learned
Unattended pipelines need health checks for silent failure, not just error alerts. The Confluence delivery routine died quietly for about seven weeks. Nothing errored. It just stopped, and nothing was watching for absence. An alert on "something broke" never fires when the failure mode is "nothing happened."
The follow-up is almost funnier: the first "no digest found" alarm after adding that check was itself a false alarm, caused by date coupling in the check.
Split reliability tiers by credential model. See above — schedulable CI has secrets but no connectors; cloud routines have connectors but no secrets. Let that shape the architecture.
Document open issues honestly. A prompt-cache passthrough bug and unwired checkpoint resumability are both written down in the repo rather than quietly carried. Standardized on a mid-tier model via OpenRouter for cost.
How it was built
Brainstorm → design spec → implementation plan, all committed to the repo before implementation, then designed, built and operated with an AI coding agent. One design day took it from idea to production, then lightweight ops sessions whenever it drifted.
The full arc — design doc, implementation, silent failure, post-mortem, fix — lives in the repo's git history.