Cybersecurity-Projects/PROJECTS/intermediate/security-news-scraper/testdata/epss
CarterPerez-dev 5865bb6149 feat(nadezhda): M3 CVE extraction + NVD/KEV/EPSS enrichment
The intelligence layer. scrape now extracts CVE IDs (cheap, no network) into
article_cves, which activates M2's shared-CVE clustering. A separate 'enrich'
command does the slow, cached API work (Carter's split: fast scrape, deliberate
enrich).

- internal/cve: regex CVE-ID extraction + three clients (stdlib net/http):
  - nvd: API 2.0, CVSS precedence v4.0>v3.1>v3.0>v2 with nullable metrics and v2
    metric-level severity fallback, totalResults==0 => not-found, apiKey header
    (exact case), token-bucket rate limit (5/30s anon, 50/30s keyed) + retry on
    429/5xx, timeouts non-retryable.
  - kev: one catalog download -> membership map; knownRansomwareCampaignUse is the
    STRING 'Known'/'Unknown' mapped explicitly to bool.
  - epss: batched; epss/percentile are QUOTED STRINGS parsed with ParseFloat (a
    parse failure is skipped, a legitimate 0.0 is kept); partial batches survive
    a transient error.
- internal/enrich: enriches only unenriched/stale CVEs (positive + negative TTL),
  KEV once per run, EPSS batched, NVD per-CVE. NVD/EPSS soft per-item (resumable),
  KEV fatal by design (enriching without it would persist wrong is_kev for a TTL).
- store: UpsertCVEStub/LinkArticleCVE, CVEsNeedingEnrichment (TTL), UpdateCVEEnrichment
  (nullable *float64 -> SQL NULL), GetCVE, ArticlesForCVE, ListArticles (parameterized
  --source/--since/--min-cvss/--kev/--keyword via EXISTS subqueries).
- commands: enrich, cve, list (cve/list off the stub list). Trimmed kev-sample.json
  (7KB) fixture for KAT tests; nvd/epss fixtures committed.
- KAT proven offline AND live: CVE-2021-44228 -> CVSS 10.0 CRITICAL (3.1), CWE-20,
  KEV added 2021-12-10 ransomware yes, EPSS 0.99999. 215 articles -> 82 CVEs, and
  shared-CVE clustering lit up (multi-source clusters 2 -> 5). Suite offline + -race.

One read-only audit agent run; 0 Crit/High/Med, Low/Nit fixes applied in-phase.

NVD apiKey header set with exact case to bypass Go header canonicalization.
2026-07-05 15:54:49 -04:00
..
CVE-2021-44228.json feat(nadezhda): M3 CVE extraction + NVD/KEV/EPSS enrichment 2026-07-05 15:54:49 -04:00