Cybersecurity-Projects/PROJECTS/intermediate/security-news-scraper
CarterPerez-dev f65d197c96 feat(nadezhda): M7 watch daemon with optional webhook notify
Add a watch command (--interval, --once, --no-enrich): a long-lived daemon that re-ingests every enabled source on a ticker and, when a webhook is configured, POSTs genuinely new high-signal stories to it (Slack, Discord, or any JSON endpoint via a text/content/items payload).

internal/watch is a pure, stdlib-only scheduler: the pipeline is injected as a Cycle closure, so the daemon is fully unit-testable with a fake ticker and notifier. Graceful shutdown on SIGINT/SIGTERM returns cleanly; a cycle error is logged and the loop continues (fail-soft).

Notify uses a dedicated fetch-time watermark (store.NewlyFetchedClusters) so a freshly ingested but older-dated advisory is surfaced, which a publish-time filter would drop. Notable stories are capped by watch.notify_max_items and filtered by score threshold or KEV status.

Extract the shared ingest and cluster sequence (pipeline.go) so scrape and watch cannot drift, and route scrape and enrich through cmd.Context() so they honor SIGTERM too.
2026-07-06 22:07:31 -04:00
..
cmd/nadezhda feat(nadezhda): M7 watch daemon with optional webhook notify 2026-07-06 22:07:31 -04:00
internal feat(nadezhda): M7 watch daemon with optional webhook notify 2026-07-06 22:07:31 -04:00
testdata feat(nadezhda): keyless CVE enrichment, folded into scrape, news-first ranking 2026-07-06 01:44:53 -04:00
.gitignore feat(nadezhda): M7 watch daemon with optional webhook notify 2026-07-06 22:07:31 -04:00
README.md feat(nadezhda): M0 scaffold for security news + CVE aggregator 2026-07-05 13:22:08 -04:00
go.mod feat(nadezhda): AI setup wizard (nadezhda ai) + credential store + Ollama port fix 2026-07-06 21:26:10 -04:00
go.sum feat(nadezhda): AI setup wizard (nadezhda ai) + credential store + Ollama port fix 2026-07-06 21:26:10 -04:00
justfile feat(nadezhda): AI setup wizard (nadezhda ai) + credential store + Ollama port fix 2026-07-06 21:26:10 -04:00

README.md

Nadezhda

A concurrent security-news and CVE aggregation engine, written in Go.

Nadezhda ingests cybersecurity news from reliable RSS feeds, enriches every referenced CVE with authoritative exploit intelligence (NVD, CISA KEV, FIRST EPSS), clusters the same story across outlets, ranks items by real-world significance, and surfaces content angles. It ships as a single static binary with a local SQLite store, a colorful terminal UI, and Markdown/JSON export.

Status

Early development. The scaffold is in place: configuration, source registry, SQLite store with forward-only migrations, and the command skeleton.

nadezhda version     # print version
nadezhda sources     # list configured feeds and persist them to the store

Ingestion, CVE enrichment, ranking, the TUI, and the AI ideation layer land in subsequent milestones.

Build

just build     # -> ./nadezhda
just test

Requires Go 1.25+.


Full documentation lands in learn/ as the project matures.