looks magnificent
This commit is contained in:
CarterPerez-dev
parent
2d84ef62f7
commit
47cb20d23f
|
|
@ -12,10 +12,18 @@ bin/
|
|||
|
||||
# Test
|
||||
*.test
|
||||
coverage.out
|
||||
*.prof
|
||||
trace.out
|
||||
|
||||
# Build
|
||||
tmp/
|
||||
|
||||
# Cache
|
||||
.ruff_cache/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
|
|
|
|||
|
|
@ -1,23 +1,23 @@
|
|||
# angela
|
||||
|
||||
A fast CLI tool that updates Python dependencies in `pyproject.toml` and scans for known vulnerabilities using [OSV.dev](https://osv.dev).
|
||||
A fast Python dependency updater and vulnerability scanner, written in Go.
|
||||
|
||||
Built in Go for speed — parallel HTTP requests, local caching with ETag support, and a single binary with zero runtime dependencies.
|
||||
angela scans your `pyproject.toml` or `requirements.txt`, updates all dependencies to their latest stable versions, and checks for known CVEs using [OSV.dev](https://osv.dev) — in a single command.
|
||||
|
||||
---
|
||||
## Highlights
|
||||
|
||||
## Why
|
||||
- **One command updates everything** in `pyproject.toml` or `requirements.txt`, like `pnpm update` for Python.
|
||||
- Fast parallel queries against the [PyPI Simple API](https://peps.python.org/pep-0691/) with local ETag caching.
|
||||
- Built-in vulnerability scanning via [OSV.dev](https://osv.dev) batch queries — no API key required.
|
||||
- Full [PEP 440](https://peps.python.org/pep-0440/) version parsing with pre-release filtering and epoch support.
|
||||
- Comment-preserving file updates — inline comments, section headers, and formatting stay intact.
|
||||
- Configurable via `.angela.toml` or `[tool.angela]` in `pyproject.toml`.
|
||||
- Single binary with zero runtime dependencies.
|
||||
|
||||
The Python ecosystem lacks a single tool that updates all dependencies in `pyproject.toml` the way `pnpm update` works for JavaScript. Existing options either show outdated packages without updating them (`pip list --outdated`), upgrade one at a time (`pip install --upgrade`), or require CI infrastructure (Dependabot, Renovate).
|
||||
|
||||
**angela** fills this gap: one command updates everything, with built-in CVE scanning.
|
||||
|
||||
---
|
||||
|
||||
## Install
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
go install github.com/CarterPerez-dev/angela@latest
|
||||
go install github.com/CarterPerez-dev/angela/cmd/angela@latest
|
||||
```
|
||||
|
||||
Or build from source:
|
||||
|
|
@ -25,119 +25,211 @@ Or build from source:
|
|||
```bash
|
||||
git clone https://github.com/CarterPerez-dev/angela.git
|
||||
cd angela
|
||||
go build -o bin/angela ./cmd/angela
|
||||
go build -o angela ./cmd/angela
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
### Update all dependencies
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# Initialize a new pyproject.toml
|
||||
angela init
|
||||
|
||||
# Update all dependencies to latest stable versions
|
||||
angela update
|
||||
```
|
||||
|
||||
Reads `pyproject.toml` in the current directory, queries PyPI for the latest stable versions, and writes the updates back to the file.
|
||||
|
||||
### Dry run (show what would change)
|
||||
|
||||
```bash
|
||||
# Preview updates without writing changes
|
||||
angela check
|
||||
```
|
||||
|
||||
Shows available updates and vulnerabilities without modifying any files.
|
||||
|
||||
### Scan for vulnerabilities only
|
||||
|
||||
```bash
|
||||
# Scan for known vulnerabilities
|
||||
angela scan
|
||||
|
||||
# Update + scan for vulnerabilities in one pass
|
||||
angela update --vulns
|
||||
```
|
||||
|
||||
Checks all pinned dependencies against the OSV.dev vulnerability database.
|
||||
## Commands
|
||||
|
||||
### Update with vulnerability scan
|
||||
### `angela init`
|
||||
|
||||
```bash
|
||||
angela update --scan-vulns
|
||||
Create a new `pyproject.toml` with angela configuration in the current directory.
|
||||
|
||||
```console
|
||||
$ angela init
|
||||
|
||||
Created pyproject.toml
|
||||
```
|
||||
|
||||
Updates dependencies and reports any known CVEs in your current versions.
|
||||
### `angela update`
|
||||
|
||||
### Skip major version bumps
|
||||
Update all dependencies in `pyproject.toml` to their latest stable versions.
|
||||
|
||||
```bash
|
||||
angela update --safe
|
||||
```
|
||||
```console
|
||||
$ angela update
|
||||
|
||||
Only applies minor and patch updates, skipping anything that crosses a major version boundary.
|
||||
|
||||
### Include pre-release versions
|
||||
|
||||
```bash
|
||||
angela update --include-prerelease
|
||||
```
|
||||
|
||||
Considers alpha, beta, release candidate, and dev versions when resolving the latest.
|
||||
|
||||
### Specify a different file
|
||||
|
||||
```bash
|
||||
angela update --file path/to/pyproject.toml
|
||||
```
|
||||
|
||||
### Clear the local cache
|
||||
|
||||
```bash
|
||||
angela cache clear
|
||||
```
|
||||
|
||||
Removes all cached PyPI responses from `~/.angela/cache/`.
|
||||
|
||||
---
|
||||
|
||||
## Example output
|
||||
|
||||
```
|
||||
Scanning 9 dependencies...
|
||||
|
||||
Updates available:
|
||||
django 3.2.0 -> 5.1.5 (major)
|
||||
requests 2.28.0 -> 2.32.3 (minor)
|
||||
click 8.0.0 -> 8.1.8 (patch)
|
||||
pydantic 2.0.0 -> 2.10.6 (minor)
|
||||
django 3.2.0 -> 6.0.1 (major)
|
||||
requests 2.28.0 -> 2.32.5 (minor)
|
||||
flask 2.0.0 -> 3.1.0 (major)
|
||||
pytest 7.0.0 -> 8.3.4 (major)
|
||||
black 23.0.0 -> 25.1.0 (major)
|
||||
ruff 0.1.0 -> 0.9.4 (minor)
|
||||
mypy 1.0.0 -> 1.14.1 (minor)
|
||||
|
||||
Vulnerabilities found:
|
||||
django
|
||||
GHSA-2hrw-hx67-34x6 [CRITICAL] Potential denial-of-service in django.utils.text.Truncator
|
||||
Fixed in: 4.2.16
|
||||
requests
|
||||
GHSA-9wx4-h78v-vm56 [MEDIUM] Requests `Session` object does not verify requests after making first request with verify=False
|
||||
Fixed in: 2.32.0
|
||||
click 8.0.0 -> 8.1.8 (patch)
|
||||
|
||||
Updated pyproject.toml
|
||||
9 packages checked
|
||||
9 updated
|
||||
4 updated
|
||||
Done in 1.8s
|
||||
```
|
||||
|
||||
Flags:
|
||||
|
||||
```
|
||||
-f, --file string Path to dependency file (default "pyproject.toml")
|
||||
--safe Skip major version bumps
|
||||
--vulns Also scan for vulnerabilities
|
||||
--include-prerelease Include pre-release versions (alpha, beta, rc, dev)
|
||||
```
|
||||
|
||||
### `angela check`
|
||||
|
||||
Show available updates without modifying any files.
|
||||
|
||||
```console
|
||||
$ angela check
|
||||
```
|
||||
|
||||
Accepts the same flags as `update`. Useful for CI or previewing changes before applying them.
|
||||
|
||||
### `angela scan`
|
||||
|
||||
Scan all dependencies for known vulnerabilities via OSV.dev.
|
||||
|
||||
```console
|
||||
$ angela scan
|
||||
|
||||
Scanning 9 dependencies...
|
||||
|
||||
Vulnerabilities: 36 across 4 packages
|
||||
|
||||
django (30 vulns: 5 critical, 15 high, 10 moderate)
|
||||
CRITICAL CVE-2022-28346 SQL Injection in Django Fixed: 2.2.28
|
||||
CRITICAL CVE-2022-34265 Trunc()/Extract() SQL Injection Fixed: 3.2.14
|
||||
HIGH CVE-2023-24580 Resource exhaustion Fixed: 3.2.18
|
||||
HIGH CVE-2022-36359 Reflected File Download attack Fixed: 3.2.15
|
||||
HIGH CVE-2023-46695 Potential denial-of-service Fixed: 4.2.8
|
||||
...and 25 more
|
||||
|
||||
requests (3 vulns: 3 moderate)
|
||||
MODERATE CVE-2024-35195 Session verify=False bypass Fixed: 2.32.0
|
||||
MODERATE CVE-2023-32681 Proxy-Authorization header leak Fixed: 2.31.0
|
||||
MODERATE CVE-2024-47081 .netrc credentials leak via URLs Fixed: 2.32.4
|
||||
|
||||
Run angela scan -v for full details.
|
||||
|
||||
9 packages checked
|
||||
36 vulnerabilities found
|
||||
Done in 2.4s
|
||||
```
|
||||
|
||||
---
|
||||
Use `-v` for full vulnerability details including advisory links:
|
||||
|
||||
```console
|
||||
$ angela scan -v
|
||||
```
|
||||
|
||||
Flags:
|
||||
|
||||
```
|
||||
-f, --file string Path to dependency file (default "pyproject.toml")
|
||||
```
|
||||
|
||||
### `angela cache clear`
|
||||
|
||||
Remove all cached PyPI responses from `~/.angela/cache/`.
|
||||
|
||||
### Global flags
|
||||
|
||||
```
|
||||
-v, --verbose Show full vulnerability details
|
||||
--min-severity Minimum severity to report (critical, high, moderate, low)
|
||||
```
|
||||
|
||||
### Subcommand aliases
|
||||
|
||||
| Command | Alias |
|
||||
|---------|-------|
|
||||
| `angela update` | `angela u` |
|
||||
| `angela check` | `angela c` |
|
||||
| `angela scan` | `angela s` |
|
||||
|
||||
## File support
|
||||
|
||||
angela works with both `pyproject.toml` and `requirements.txt`. The file type is detected automatically by extension:
|
||||
|
||||
```bash
|
||||
# pyproject.toml (default)
|
||||
angela update
|
||||
|
||||
# requirements.txt
|
||||
angela update -f requirements.txt
|
||||
```
|
||||
|
||||
For `pyproject.toml`, angela reads `[project.dependencies]` and `[project.optional-dependencies]`. For `requirements.txt`, it parses standard pip-format lines, skipping comments, blank lines, and pip options (`-r`, `-e`, `--index-url`).
|
||||
|
||||
Both parsers handle PEP 508 dependency strings including extras (`[security,socks]`) and environment markers (`; python_version >= "3.8"`).
|
||||
|
||||
## Configuration
|
||||
|
||||
angela supports project-level configuration through either a standalone `.angela.toml` file or a `[tool.angela]` section in `pyproject.toml`.
|
||||
|
||||
### `.angela.toml`
|
||||
|
||||
```toml
|
||||
min-severity = "moderate"
|
||||
|
||||
ignore = [
|
||||
"django",
|
||||
"numpy",
|
||||
]
|
||||
|
||||
ignore-vulns = [
|
||||
"CVE-2024-3772",
|
||||
]
|
||||
```
|
||||
|
||||
### `[tool.angela]` in pyproject.toml
|
||||
|
||||
```toml
|
||||
[tool.angela]
|
||||
min-severity = "moderate"
|
||||
ignore = ["django", "numpy"]
|
||||
ignore-vulns = ["CVE-2024-3772"]
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
| Key | Type | Description |
|
||||
|-----|------|-------------|
|
||||
| `min-severity` | string | Minimum severity to report: `critical`, `high`, `moderate`, `low` |
|
||||
| `ignore` | string array | Package names to skip during updates |
|
||||
| `ignore-vulns` | string array | Vulnerability IDs to suppress (CVE or GHSA) |
|
||||
|
||||
### Resolution order
|
||||
|
||||
1. CLI flags (`--min-severity`)
|
||||
2. `.angela.toml`
|
||||
3. `[tool.angela]` in `pyproject.toml`
|
||||
4. Defaults
|
||||
|
||||
## How it works
|
||||
|
||||
```
|
||||
1. Parse pyproject.toml
|
||||
Extract all [project.dependencies] and [project.optional-dependencies]
|
||||
1. Parse dependency file
|
||||
Read pyproject.toml or requirements.txt
|
||||
Extract package names and version specifiers
|
||||
|
||||
2. Query PyPI Simple API (parallel)
|
||||
Fetch version lists using the lightweight JSON format
|
||||
ETag-based caching avoids redundant downloads
|
||||
2. Query PyPI (parallel)
|
||||
Fetch version lists from the Simple API with JSON format
|
||||
Cache responses locally with ETag support (1-hour TTL)
|
||||
|
||||
3. Resolve updates
|
||||
Parse versions per PEP 440 (epochs, pre-releases, post-releases)
|
||||
|
|
@ -145,40 +237,43 @@ Removes all cached PyPI responses from `~/.angela/cache/`.
|
|||
Classify changes as major, minor, or patch
|
||||
|
||||
4. Scan for vulnerabilities (optional)
|
||||
Batch query OSV.dev for all dependencies
|
||||
Batch query OSV.dev for all dependencies in a single request
|
||||
Hydrate results with full advisory details
|
||||
Deduplicate overlapping CVE/GHSA identifiers
|
||||
|
||||
5. Write updates
|
||||
Regex-based surgery preserves comments and formatting
|
||||
Regex-based surgery preserves all comments and formatting
|
||||
Atomic write via temp file + rename
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
## Project structure
|
||||
|
||||
```
|
||||
cmd/angela/main.go CLI entry point
|
||||
cmd/angela/main.go Entry point
|
||||
internal/
|
||||
cli/
|
||||
update.go Command definitions and orchestration
|
||||
output.go Terminal formatting with color
|
||||
update.go Commands and orchestration
|
||||
output.go Terminal formatting
|
||||
config/
|
||||
config.go Configuration loading
|
||||
pypi/
|
||||
version.go PEP 440 parser and comparator
|
||||
client.go PyPI Simple API client with retry
|
||||
cache.go File-backed ETag cache
|
||||
client.go PyPI Simple API client with retry
|
||||
version.go PEP 440 parser and comparator
|
||||
cache.go File-backed ETag cache
|
||||
osv/
|
||||
client.go OSV.dev batch vulnerability scanner
|
||||
client.go OSV.dev batch vulnerability scanner
|
||||
pyproject/
|
||||
parser.go TOML parsing and PEP 508 splitting
|
||||
writer.go Comment-preserving regex updater
|
||||
parser.go TOML parsing and PEP 508 splitting
|
||||
writer.go Comment-preserving regex updater
|
||||
requirements/
|
||||
parser.go requirements.txt parsing
|
||||
writer.go requirements.txt updater
|
||||
pkg/types/
|
||||
types.go Shared domain types
|
||||
types.go Shared domain types
|
||||
testdata/ Test fixtures
|
||||
learn/ Educational deep dives
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Development
|
||||
|
||||
Requires Go 1.24+ and [just](https://github.com/casey/just).
|
||||
|
|
@ -187,20 +282,16 @@ Requires Go 1.24+ and [just](https://github.com/casey/just).
|
|||
just test # Run all tests with race detector
|
||||
just lint # Run golangci-lint
|
||||
just build # Build binary to bin/angela
|
||||
just cover # Generate HTML coverage report
|
||||
just check # Lint + test in one step
|
||||
just cover # Generate coverage report
|
||||
just ci # Lint + test in one step
|
||||
just run check # Run angela check via go run
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part of Cybersecurity-Projects
|
||||
|
||||
This tool is project #1 in the [Cybersecurity-Projects](https://github.com/CarterPerez-dev/Cybersecurity-Projects) repository — a collection of 60 security-focused projects built for learning and reference. The code is written to be educational: clear structure, proper error handling, and thorough testing.
|
||||
This tool is project #1 in [Cybersecurity-Projects](https://github.com/CarterPerez-dev/Cybersecurity-Projects) — a collection of 60 security-focused projects built for learning and reference.
|
||||
|
||||
See the [`learn/`](learn/) directory for deep dives into the techniques used here.
|
||||
|
||||
---
|
||||
The code is written to be educational: clear structure, proper error handling, and thorough testing. See the [`learn/`](learn/) directory for deep dives into the techniques used.
|
||||
|
||||
## License
|
||||
|
||||
|
|
|
|||
|
|
@ -11,8 +11,8 @@ require (
|
|||
|
||||
require (
|
||||
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
||||
github.com/mattn/go-colorable v0.1.13 // indirect
|
||||
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/spf13/pflag v1.0.9 // indirect
|
||||
golang.org/x/sys v0.25.0 // indirect
|
||||
github.com/spf13/pflag v1.0.10 // indirect
|
||||
golang.org/x/sys v0.40.0 // indirect
|
||||
)
|
||||
|
|
|
|||
|
|
@ -3,9 +3,8 @@ github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
|
|||
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
|
||||
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
|
||||
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
|
||||
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
|
||||
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
|
||||
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
|
||||
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
|
||||
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
|
||||
|
|
@ -13,13 +12,13 @@ github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8
|
|||
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
|
||||
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
|
||||
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
|
||||
github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY=
|
||||
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
|
||||
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
|
||||
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
|
||||
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
|
||||
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.25.0 h1:r+8e+loiHxRqhXVl6ML1nO3l1+oFoWbnlu2Ehimmi34=
|
||||
golang.org/x/sys v0.25.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/sys v0.40.0 h1:DBZZqJ2Rkml6QMQsZywtnjnnGvHza6BTfYFWY9kjEWQ=
|
||||
golang.org/x/sys v0.40.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
|
|
|
|||
|
|
@ -9,31 +9,25 @@ import (
|
|||
"strings"
|
||||
|
||||
"github.com/CarterPerez-dev/angela/internal/pypi"
|
||||
"github.com/CarterPerez-dev/angela/internal/ui"
|
||||
"github.com/CarterPerez-dev/angela/pkg/types"
|
||||
"github.com/fatih/color"
|
||||
)
|
||||
|
||||
var (
|
||||
bold = color.New(color.Bold).SprintFunc()
|
||||
dim = color.New(color.Faint).SprintFunc()
|
||||
cyan = color.New(color.FgCyan, color.Bold).SprintFunc()
|
||||
green = color.New(color.FgGreen).SprintFunc()
|
||||
yellow = color.New(color.FgYellow).SprintFunc()
|
||||
red = color.New(color.FgRed).SprintFunc()
|
||||
redBold = color.New(color.FgRed, color.Bold).SprintFunc()
|
||||
greenBold = color.New(color.FgGreen, color.Bold).SprintFunc()
|
||||
white = color.New(color.FgWhite, color.Bold).SprintFunc()
|
||||
)
|
||||
func printDivider() {
|
||||
fmt.Printf("\n %s\n", ui.HiBlack(ui.HRule(44)))
|
||||
}
|
||||
|
||||
// PrintScanning announces the start of a scan
|
||||
func PrintScanning(count int) {
|
||||
func printSectionHeader(
|
||||
symbol, title string,
|
||||
symbolColor, titleColor func(a ...any) string,
|
||||
) {
|
||||
fmt.Printf(
|
||||
"\n %s %d dependencies...\n\n",
|
||||
cyan("Scanning"), count,
|
||||
"\n %s %s\n",
|
||||
symbolColor(symbol),
|
||||
titleColor(title),
|
||||
)
|
||||
}
|
||||
|
||||
// PrintUpdates displays a formatted table of available dependency updates
|
||||
func PrintUpdates(updates []types.UpdateResult) {
|
||||
var actionable []types.UpdateResult
|
||||
for _, u := range updates {
|
||||
|
|
@ -43,11 +37,22 @@ func PrintUpdates(updates []types.UpdateResult) {
|
|||
}
|
||||
|
||||
if len(actionable) == 0 {
|
||||
fmt.Printf(" %s\n\n", dim("All dependencies are up to date."))
|
||||
printDivider()
|
||||
fmt.Printf(
|
||||
"\n %s %s\n",
|
||||
ui.HiGreen(ui.Check),
|
||||
ui.HiBlackItalic(
|
||||
"All dependencies are up to date.",
|
||||
),
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
fmt.Printf(" %s\n", cyan("Updates available:"))
|
||||
printDivider()
|
||||
printSectionHeader(
|
||||
ui.Diamond, "Updates available",
|
||||
ui.Cyan, ui.Red,
|
||||
)
|
||||
|
||||
nameWidth := 0
|
||||
oldWidth := 0
|
||||
|
|
@ -63,18 +68,19 @@ func PrintUpdates(updates []types.UpdateResult) {
|
|||
for _, u := range actionable {
|
||||
changeColor := changeColorFn(u.Change)
|
||||
fmt.Printf(
|
||||
" %-*s %s %s %s %s\n",
|
||||
nameWidth, white(u.Name),
|
||||
dim(padRight(u.OldVer, oldWidth)),
|
||||
dim("->"),
|
||||
" %-*s %s %s %s %s %s\n",
|
||||
nameWidth, ui.HiBlue(u.Name),
|
||||
ui.HiBlackCrossed(
|
||||
padRight(u.OldVer, oldWidth),
|
||||
),
|
||||
ui.Green(ui.Arrow),
|
||||
changeColor(u.NewVer),
|
||||
dim("("+u.Change+")"),
|
||||
ui.HiBlack(ui.ArrowRight),
|
||||
changeColor(u.Change),
|
||||
)
|
||||
}
|
||||
fmt.Println()
|
||||
}
|
||||
|
||||
// PrintSkipped displays dependencies that were not updated with reasons
|
||||
func PrintSkipped(updates []types.UpdateResult) {
|
||||
var skipped []types.UpdateResult
|
||||
for _, u := range updates {
|
||||
|
|
@ -86,14 +92,21 @@ func PrintSkipped(updates []types.UpdateResult) {
|
|||
return
|
||||
}
|
||||
|
||||
fmt.Printf(" %s\n", dim("Skipped:"))
|
||||
printDivider()
|
||||
printSectionHeader(
|
||||
ui.ArrowRight, "Skipped",
|
||||
ui.Yellow, ui.Red,
|
||||
)
|
||||
|
||||
for _, u := range skipped {
|
||||
fmt.Printf(" %s %s\n", dim(u.Name), dim(u.Reason))
|
||||
fmt.Printf(
|
||||
" %s %s\n",
|
||||
ui.HiBlue(u.Name),
|
||||
ui.HiBlackItalic(u.Reason),
|
||||
)
|
||||
}
|
||||
fmt.Println()
|
||||
}
|
||||
|
||||
// PrintVulnerabilities displays security advisories in compact or verbose mode
|
||||
func PrintVulnerabilities(
|
||||
vulns map[string][]types.Vulnerability,
|
||||
) {
|
||||
|
|
@ -106,12 +119,15 @@ func PrintVulnerabilities(
|
|||
total += len(vl)
|
||||
}
|
||||
|
||||
printDivider()
|
||||
fmt.Printf(
|
||||
" %s %d across %d %s\n",
|
||||
redBold("Vulnerabilities:"),
|
||||
total,
|
||||
len(vulns),
|
||||
pluralize("package", len(vulns)),
|
||||
"\n %s %s %s %s %s %s\n",
|
||||
ui.Red(ui.TriangleUp),
|
||||
ui.Red("Vulnerabilities:"),
|
||||
ui.RedBold(fmt.Sprintf("%d", total)),
|
||||
ui.HiBlack("across"),
|
||||
ui.RedBold(fmt.Sprintf("%d", len(vulns))),
|
||||
ui.HiBlack(pluralize("package", len(vulns))),
|
||||
)
|
||||
|
||||
pkgs := sortedVulnPackages(vulns)
|
||||
|
|
@ -134,11 +150,15 @@ func printVulnsCompact(
|
|||
sortVulnsBySeverity(vlist)
|
||||
|
||||
fmt.Printf(
|
||||
"\n %s (%d %s: %s)\n",
|
||||
white(pkg),
|
||||
len(vlist),
|
||||
pluralize("vuln", len(vlist)),
|
||||
severityBreakdown(vlist),
|
||||
"\n %s %s %s\n",
|
||||
ui.Blue(ui.Gem),
|
||||
ui.HiCyan(pkg),
|
||||
ui.HiBlack(fmt.Sprintf(
|
||||
"(%d %s: %s)",
|
||||
len(vlist),
|
||||
pluralize("vuln", len(vlist)),
|
||||
severityBreakdown(vlist),
|
||||
)),
|
||||
)
|
||||
|
||||
limit := min(maxVulnsPerPackage, len(vlist))
|
||||
|
|
@ -148,8 +168,8 @@ func printVulnsCompact(
|
|||
|
||||
if len(vlist) > maxVulnsPerPackage {
|
||||
fmt.Printf(
|
||||
" %s\n",
|
||||
dim(fmt.Sprintf(
|
||||
" %s\n",
|
||||
ui.HiBlackItalic(fmt.Sprintf(
|
||||
"...and %d more",
|
||||
len(vlist)-maxVulnsPerPackage,
|
||||
)),
|
||||
|
|
@ -158,8 +178,10 @@ func printVulnsCompact(
|
|||
}
|
||||
|
||||
fmt.Printf(
|
||||
"\n %s\n\n",
|
||||
dim("Run angela scan -v for full details."),
|
||||
"\n %s\n",
|
||||
ui.HiBlackItalic(
|
||||
"Run angela scan -v for full details.",
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
|
|
@ -172,11 +194,15 @@ func printVulnsVerbose(
|
|||
sortVulnsBySeverity(vlist)
|
||||
|
||||
fmt.Printf(
|
||||
"\n %s (%d %s: %s)\n",
|
||||
white(pkg),
|
||||
len(vlist),
|
||||
pluralize("vuln", len(vlist)),
|
||||
severityBreakdown(vlist),
|
||||
"\n %s %s %s\n",
|
||||
ui.Magenta(ui.Gem),
|
||||
ui.HiCyan(pkg),
|
||||
ui.HiBlack(fmt.Sprintf(
|
||||
"(%d %s: %s)",
|
||||
len(vlist),
|
||||
pluralize("vuln", len(vlist)),
|
||||
severityBreakdown(vlist),
|
||||
)),
|
||||
)
|
||||
|
||||
for _, v := range vlist {
|
||||
|
|
@ -184,22 +210,28 @@ func printVulnsVerbose(
|
|||
sevColor := severityColorFn(v.Severity)
|
||||
|
||||
fmt.Printf(
|
||||
"\n %s %s\n",
|
||||
"\n %s %s\n",
|
||||
sevColor(
|
||||
padRight(strings.ToUpper(v.Severity), 8),
|
||||
),
|
||||
bold(id),
|
||||
ui.Magenta(id),
|
||||
)
|
||||
fmt.Printf(
|
||||
" %s\n",
|
||||
ui.HiBlack(truncate(v.Summary, 72)),
|
||||
)
|
||||
fmt.Printf(" %s\n", truncate(v.Summary, 72))
|
||||
if v.FixedIn != "" {
|
||||
fmt.Printf(
|
||||
" %s %s\n",
|
||||
dim("Fixed in:"),
|
||||
green(v.FixedIn),
|
||||
" %s %s\n",
|
||||
ui.HiBlack("Fixed in:"),
|
||||
ui.Green(v.FixedIn),
|
||||
)
|
||||
}
|
||||
if v.Link != "" {
|
||||
fmt.Printf(" %s\n", dim(v.Link))
|
||||
fmt.Printf(
|
||||
" %s\n",
|
||||
ui.BlueItalic(v.Link),
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -212,14 +244,17 @@ func printVulnLine(v types.Vulnerability) {
|
|||
|
||||
fixedStr := ""
|
||||
if v.FixedIn != "" {
|
||||
fixedStr = dim("Fixed: ") + green(v.FixedIn)
|
||||
fixedStr = ui.HiBlack("Fixed: ") +
|
||||
ui.Green(v.FixedIn)
|
||||
}
|
||||
|
||||
fmt.Printf(
|
||||
" %s %s %s %s\n",
|
||||
sevColor(padRight(strings.ToUpper(v.Severity), 8)),
|
||||
bold(padRight(id, 16)),
|
||||
truncate(v.Summary, 38),
|
||||
" %s %s %s %s\n",
|
||||
sevColor(
|
||||
padRight(strings.ToUpper(v.Severity), 8),
|
||||
),
|
||||
ui.Blue(padRight(id, 16)),
|
||||
ui.HiBlack(truncate(v.Summary, 38)),
|
||||
fixedStr,
|
||||
)
|
||||
}
|
||||
|
|
@ -242,7 +277,9 @@ func sortedVulnPackages(
|
|||
return pkgs
|
||||
}
|
||||
|
||||
func sortVulnsBySeverity(vulns []types.Vulnerability) {
|
||||
func sortVulnsBySeverity(
|
||||
vulns []types.Vulnerability,
|
||||
) {
|
||||
sort.Slice(vulns, func(i, j int) bool {
|
||||
ri := severityRank(vulns[i].Severity)
|
||||
rj := severityRank(vulns[j].Severity)
|
||||
|
|
@ -283,12 +320,13 @@ func severityBreakdown(
|
|||
var parts []string
|
||||
for _, sev := range order {
|
||||
if n := counts[sev]; n > 0 {
|
||||
parts = append(parts, fmt.Sprintf(
|
||||
sevColor := severityColorFn(sev)
|
||||
parts = append(parts, sevColor(fmt.Sprintf(
|
||||
"%d %s", n, strings.ToLower(sev),
|
||||
))
|
||||
)))
|
||||
}
|
||||
}
|
||||
return strings.Join(parts, ", ")
|
||||
return strings.Join(parts, ui.HiBlack(", "))
|
||||
}
|
||||
|
||||
func preferredID(v types.Vulnerability) string {
|
||||
|
|
@ -336,7 +374,9 @@ func filterIgnoredVulns(
|
|||
return filtered
|
||||
}
|
||||
|
||||
func isVulnIgnored(v types.Vulnerability, ignored map[string]bool) bool {
|
||||
func isVulnIgnored(
|
||||
v types.Vulnerability, ignored map[string]bool,
|
||||
) bool {
|
||||
if ignored[v.ID] {
|
||||
return true
|
||||
}
|
||||
|
|
@ -368,73 +408,106 @@ func filterVulnsBySeverity(
|
|||
return filtered
|
||||
}
|
||||
|
||||
// PrintSummary displays final counts after an update or scan operation
|
||||
func PrintSummary(result types.ScanResult, updated bool) {
|
||||
func PrintSummary(
|
||||
result types.ScanResult, updated bool,
|
||||
) {
|
||||
printDivider()
|
||||
printSectionHeader(
|
||||
ui.Star, "Summary",
|
||||
ui.Green, ui.RedItalic,
|
||||
)
|
||||
|
||||
if updated {
|
||||
fmt.Printf(" %s\n", greenBold("Updated pyproject.toml"))
|
||||
fmt.Printf(
|
||||
" %s %s\n",
|
||||
ui.HiGreen(ui.Check),
|
||||
ui.HiGreen("Updated pyproject.toml"),
|
||||
)
|
||||
}
|
||||
fmt.Printf(
|
||||
" %s packages checked\n",
|
||||
bold(fmt.Sprintf("%d", result.TotalPackages)),
|
||||
" %s %s %s\n",
|
||||
ui.Green(ui.Check),
|
||||
ui.Green(fmt.Sprintf("%d", result.TotalPackages)),
|
||||
ui.GreenItalic("packages checked"),
|
||||
)
|
||||
if result.TotalUpdated > 0 {
|
||||
fmt.Printf(
|
||||
" %s updated\n",
|
||||
bold(fmt.Sprintf("%d", result.TotalUpdated)),
|
||||
" %s %s %s\n",
|
||||
ui.Green(ui.ArrowUp),
|
||||
ui.Green(
|
||||
fmt.Sprintf("%d", result.TotalUpdated),
|
||||
),
|
||||
ui.GreenItalic("updated"),
|
||||
)
|
||||
}
|
||||
if result.VulnsScanned {
|
||||
if result.TotalVulns > 0 {
|
||||
fmt.Printf(
|
||||
" %s\n",
|
||||
red(fmt.Sprintf(
|
||||
"%d %s found",
|
||||
result.TotalVulns,
|
||||
pluralize("vulnerability", result.TotalVulns),
|
||||
)),
|
||||
" %s %s %s\n",
|
||||
ui.HiRed(ui.Cross),
|
||||
ui.RedBold(
|
||||
fmt.Sprintf("%d", result.TotalVulns),
|
||||
),
|
||||
ui.HiRed(
|
||||
pluralize(
|
||||
"vulnerability",
|
||||
result.TotalVulns,
|
||||
)+" found",
|
||||
),
|
||||
)
|
||||
} else {
|
||||
fmt.Printf(
|
||||
" %s\n",
|
||||
green("No vulnerabilities found"),
|
||||
" %s %s\n",
|
||||
ui.HiGreen(ui.Check),
|
||||
ui.HiGreen("No vulnerabilities found"),
|
||||
)
|
||||
}
|
||||
}
|
||||
fmt.Printf(
|
||||
" %s %s\n\n",
|
||||
dim("Done in"),
|
||||
dim(result.Duration.Round(1e6).String()),
|
||||
" %s %s\n\n",
|
||||
ui.HiCyan(ui.Timer),
|
||||
ui.HiBlueItalic(
|
||||
result.Duration.Round(1e6).String(),
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
// PrintError displays a user-friendly error message
|
||||
func PrintError(msg string) {
|
||||
fmt.Printf("\n %s %s\n\n", red("error:"), msg)
|
||||
fmt.Printf(
|
||||
"\n %s %s %s\n\n",
|
||||
ui.RedBold(ui.Cross),
|
||||
ui.RedBold("error:"),
|
||||
ui.HiRed(msg),
|
||||
)
|
||||
}
|
||||
|
||||
func changeColorFn(kind string) func(a ...any) string {
|
||||
func changeColorFn(
|
||||
kind string,
|
||||
) func(a ...any) string {
|
||||
switch kind {
|
||||
case pypi.Major.String():
|
||||
return red
|
||||
return ui.Red
|
||||
case pypi.Minor.String():
|
||||
return yellow
|
||||
return ui.HiYellow
|
||||
default:
|
||||
return green
|
||||
return ui.Green
|
||||
}
|
||||
}
|
||||
|
||||
func severityColorFn(sev string) func(a ...any) string {
|
||||
func severityColorFn(
|
||||
sev string,
|
||||
) func(a ...any) string {
|
||||
switch strings.ToUpper(sev) {
|
||||
case "CRITICAL":
|
||||
return redBold
|
||||
return ui.Red
|
||||
case "HIGH":
|
||||
return red
|
||||
return ui.RedBold
|
||||
case "MODERATE":
|
||||
return yellow
|
||||
return ui.HiYellow
|
||||
case "LOW":
|
||||
return cyan
|
||||
return ui.Cyan
|
||||
default:
|
||||
return dim
|
||||
return ui.HiBlack
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@ import (
|
|||
"github.com/CarterPerez-dev/angela/internal/pypi"
|
||||
"github.com/CarterPerez-dev/angela/internal/pyproject"
|
||||
"github.com/CarterPerez-dev/angela/internal/requirements"
|
||||
"github.com/CarterPerez-dev/angela/internal/ui"
|
||||
"github.com/CarterPerez-dev/angela/pkg/types"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
|
@ -50,8 +51,25 @@ func Execute() {
|
|||
latest stable versions, and checks for known CVEs using OSV.dev.`,
|
||||
SilenceUsage: true,
|
||||
SilenceErrors: true,
|
||||
PersistentPreRun: func(
|
||||
_ *cobra.Command, _ []string,
|
||||
) {
|
||||
ui.PrintBanner()
|
||||
},
|
||||
}
|
||||
|
||||
defaultHelp := root.HelpFunc()
|
||||
root.SetHelpFunc(
|
||||
func(cmd *cobra.Command, args []string) {
|
||||
if cmd.Root() == cmd {
|
||||
ui.PrintBannerWithArt()
|
||||
} else {
|
||||
ui.PrintBanner()
|
||||
}
|
||||
defaultHelp(cmd, args)
|
||||
},
|
||||
)
|
||||
|
||||
root.PersistentFlags().BoolVarP(
|
||||
&verbose, "verbose", "v", false,
|
||||
"show full vulnerability details",
|
||||
|
|
@ -116,11 +134,19 @@ func runInit() error {
|
|||
name := filepath.Base(dir)
|
||||
content := fmt.Sprintf(pyprojectTemplate, name)
|
||||
|
||||
if err := os.WriteFile("pyproject.toml", []byte(content), 0o644); err != nil { //nolint:gosec
|
||||
if err := os.WriteFile(
|
||||
"pyproject.toml",
|
||||
[]byte(content),
|
||||
0o644,
|
||||
); err != nil { //nolint:gosec
|
||||
return fmt.Errorf("write pyproject.toml: %w", err)
|
||||
}
|
||||
|
||||
fmt.Printf("\n %s pyproject.toml\n\n", greenBold("Created"))
|
||||
fmt.Printf(
|
||||
"\n %s %s\n\n",
|
||||
ui.HiGreen(ui.Check),
|
||||
ui.HiGreen("Created pyproject.toml"),
|
||||
)
|
||||
return nil
|
||||
}
|
||||
|
||||
|
|
@ -128,17 +154,19 @@ func newUpdateCmd() *cobra.Command {
|
|||
f := &updateFlags{}
|
||||
|
||||
cmd := &cobra.Command{
|
||||
Use: "update",
|
||||
Use: "update [path]",
|
||||
Aliases: []string{"u"},
|
||||
Short: "Update dependencies to latest stable versions",
|
||||
RunE: func(cmd *cobra.Command, _ []string) error {
|
||||
Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
f.file = resolveFile(f.file, args)
|
||||
return runUpdate(cmd.Context(), f, false)
|
||||
},
|
||||
}
|
||||
|
||||
cmd.Flags().StringVarP(
|
||||
&f.file, "file", "f", "pyproject.toml",
|
||||
"path to pyproject.toml",
|
||||
"path to dependency file",
|
||||
)
|
||||
cmd.Flags().BoolVar(
|
||||
&f.safe, "safe", false,
|
||||
|
|
@ -159,17 +187,19 @@ func newCheckCmd() *cobra.Command {
|
|||
f := &updateFlags{}
|
||||
|
||||
cmd := &cobra.Command{
|
||||
Use: "check",
|
||||
Use: "check [path]",
|
||||
Aliases: []string{"c"},
|
||||
Short: "Show available updates without modifying files",
|
||||
RunE: func(cmd *cobra.Command, _ []string) error {
|
||||
Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
f.file = resolveFile(f.file, args)
|
||||
return runUpdate(cmd.Context(), f, true)
|
||||
},
|
||||
}
|
||||
|
||||
cmd.Flags().StringVarP(
|
||||
&f.file, "file", "f", "pyproject.toml",
|
||||
"path to pyproject.toml",
|
||||
"path to dependency file",
|
||||
)
|
||||
cmd.Flags().BoolVar(
|
||||
&f.safe, "safe", false,
|
||||
|
|
@ -190,17 +220,19 @@ func newScanCmd() *cobra.Command {
|
|||
var file string
|
||||
|
||||
cmd := &cobra.Command{
|
||||
Use: "scan",
|
||||
Use: "scan [path]",
|
||||
Aliases: []string{"s"},
|
||||
Short: "Scan dependencies for known vulnerabilities",
|
||||
RunE: func(cmd *cobra.Command, _ []string) error {
|
||||
Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
file = resolveFile(file, args)
|
||||
return runScan(cmd.Context(), file)
|
||||
},
|
||||
}
|
||||
|
||||
cmd.Flags().StringVarP(
|
||||
&file, "file", "f", "pyproject.toml",
|
||||
"path to pyproject.toml",
|
||||
"path to dependency file",
|
||||
)
|
||||
return cmd
|
||||
}
|
||||
|
|
@ -222,7 +254,11 @@ func newCacheCmd() *cobra.Command {
|
|||
if err := client.ClearCache(); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(" Cache cleared.")
|
||||
fmt.Printf(
|
||||
" %s %s\n",
|
||||
ui.HiGreen(ui.Check),
|
||||
ui.HiGreen("Cache cleared."),
|
||||
)
|
||||
return nil
|
||||
},
|
||||
})
|
||||
|
|
@ -243,10 +279,14 @@ func runUpdate(
|
|||
return err
|
||||
}
|
||||
|
||||
PrintScanning(len(deps))
|
||||
spin := ui.NewSpinner(fmt.Sprintf(
|
||||
"Scanning %d dependencies...", len(deps),
|
||||
))
|
||||
spin.Start()
|
||||
|
||||
client, err := pypi.NewClient(defaultCacheDir())
|
||||
if err != nil {
|
||||
spin.Stop()
|
||||
return err
|
||||
}
|
||||
|
||||
|
|
@ -271,12 +311,21 @@ func runUpdate(
|
|||
sortUpdates(updates)
|
||||
|
||||
var vulns map[string][]types.Vulnerability
|
||||
var scanErr error
|
||||
if f.vulns {
|
||||
vulns = scanForVulns(ctx, deps)
|
||||
vulns, scanErr = scanForVulns(ctx, deps)
|
||||
}
|
||||
|
||||
spin.Stop()
|
||||
|
||||
if scanErr != nil {
|
||||
PrintError(scanErr.Error())
|
||||
}
|
||||
|
||||
if !dryRun && len(updateSpecs) > 0 {
|
||||
if err := updateDepsFile(f.file, updateSpecs); err != nil {
|
||||
if err := updateDepsFile(
|
||||
f.file, updateSpecs,
|
||||
); err != nil {
|
||||
return fmt.Errorf("write updates: %w", err)
|
||||
}
|
||||
}
|
||||
|
|
@ -318,10 +367,21 @@ func runScan(ctx context.Context, file string) error {
|
|||
return err
|
||||
}
|
||||
|
||||
PrintScanning(len(deps))
|
||||
spin := ui.NewSpinner(fmt.Sprintf(
|
||||
"Scanning %d dependencies for vulnerabilities...",
|
||||
len(deps),
|
||||
))
|
||||
spin.Start()
|
||||
|
||||
minSev := resolveMinSeverity(cfg.MinSeverity)
|
||||
vulns := scanForVulns(ctx, deps)
|
||||
vulns, scanErr := scanForVulns(ctx, deps)
|
||||
|
||||
spin.Stop()
|
||||
|
||||
if scanErr != nil {
|
||||
PrintError(scanErr.Error())
|
||||
}
|
||||
|
||||
vulns = filterIgnoredVulns(vulns, cfg.IgnoreVulns)
|
||||
vulns = filterVulnsBySeverity(vulns, minSev)
|
||||
PrintVulnerabilities(vulns)
|
||||
|
|
@ -446,7 +506,7 @@ func resolveUpdates(
|
|||
func scanForVulns(
|
||||
ctx context.Context,
|
||||
deps []types.Dependency,
|
||||
) map[string][]types.Vulnerability {
|
||||
) (map[string][]types.Vulnerability, error) {
|
||||
var queries []osv.PackageQuery
|
||||
for _, dep := range deps {
|
||||
ver := pyproject.ExtractMinVersion(dep.Spec)
|
||||
|
|
@ -460,16 +520,17 @@ func scanForVulns(
|
|||
}
|
||||
|
||||
if len(queries) == 0 {
|
||||
return nil
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
client := osv.NewClient()
|
||||
vulns, err := client.ScanPackages(ctx, queries)
|
||||
if err != nil {
|
||||
PrintError(fmt.Sprintf("vulnerability scan: %v", err))
|
||||
return nil
|
||||
return nil, fmt.Errorf(
|
||||
"vulnerability scan: %w", err,
|
||||
)
|
||||
}
|
||||
return vulns
|
||||
return vulns, nil
|
||||
}
|
||||
|
||||
func latestAny(versions []string) (pypi.Version, error) {
|
||||
|
|
@ -493,6 +554,18 @@ func latestAny(versions []string) (pypi.Version, error) {
|
|||
return latest, nil
|
||||
}
|
||||
|
||||
func resolveFile(flagVal string, args []string) string {
|
||||
if len(args) > 0 {
|
||||
path := args[0]
|
||||
info, err := os.Stat(path)
|
||||
if err == nil && info.IsDir() {
|
||||
return filepath.Join(path, "pyproject.toml")
|
||||
}
|
||||
return path
|
||||
}
|
||||
return flagVal
|
||||
}
|
||||
|
||||
func isRequirementsTxt(path string) bool {
|
||||
return strings.HasSuffix(strings.ToLower(path), ".txt")
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,79 @@
|
|||
// ©AngelaMos | 2026
|
||||
// banner.go
|
||||
|
||||
package ui
|
||||
|
||||
import "fmt"
|
||||
|
||||
var angelaBanner = []string{
|
||||
" ▄▀▄ █▄ █ ▄▀ ██▀ █ ▄▀▄ ",
|
||||
" █▀█ █ ▀█ ▀▄█ █▄▄ █▄▄ █▀█ ",
|
||||
}
|
||||
|
||||
var bannerColors = []func(a ...any) string{
|
||||
Red,
|
||||
Blue,
|
||||
}
|
||||
|
||||
var animeArt = []string{
|
||||
"⣿⣿⣿⡷⠊⡢⡹⣦⡑⢂⢕⢂⢕⢂⢕⢂⠕⠔⠌⠝⠛⠶⠶⢶⣦⣄⢂⢕⢂⢕",
|
||||
"⣿⣿⠏⣠⣾⣦⡐⢌⢿⣷⣦⣅⡑⠕⠡⠐⢿⠿⣛⠟⠛⠛⠛⠛⠡⢷⡈⢂⢕⢂",
|
||||
"⠟⣡⣾⣿⣿⣿⣿⣦⣑⠝⢿⣿⣿⣿⣿⣿⡵⢁⣤⣶⣶⣿⢿⢿⢿⡟⢻⣤⢑⢂",
|
||||
"⣾⣿⣿⡿⢟⣛⣻⣿⣿⣿⣦⣬⣙⣻⣿⣿⣷⣿⣿⢟⢝⢕⢕⢕⢕⢽⣿⣿⣷⣔",
|
||||
"⣿⣿⠵⠚⠉⢀⣀⣀⣈⣿⣿⣿⣿⣿⣿⣿⣿⣿⣗⢕⢕⢕⢕⢕⢕⣽⣿⣿⣿⣿",
|
||||
"⢷⣂⣠⣴⣾⡿⡿⡻⡻⣿⣿⣴⣿⣿⣿⣿⣿⣿⣷⣵⣵⣵⣷⣿⣿⣿⣿⣿⣿⡿",
|
||||
"⢌⠻⣿⡿⡫⡪⡪⡪⡪⣺⣿⣿⣿⣿⣿⠿⠿⢿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠃",
|
||||
"⠣⡁⠹⡪⡪⡪⡪⣪⣾⣿⣿⣿⣿⠋⠐⢉⢍⢄⢌⠻⣿⣿⣿⣿⣿⣿⣿⣿⠏⠈",
|
||||
"⡣⡘⢄⠙⣾⣾⣾⣿⣿⣿⣿⣿⣿⡀⢐⢕⢕⢕⢕⢕⡘⣿⣿⣿⣿⣿⣿⠏⠠⠈",
|
||||
"⠌⢊⢂⢣⠹⣿⣿⣿⣿⣿⣿⣿⣿⣧⢐⢕⢕⢕⢕⢕⢅⣿⣿⣿⣿⡿⢋⢜⠠⠈",
|
||||
}
|
||||
|
||||
var artColors = []func(a ...any) string{
|
||||
Red,
|
||||
Red,
|
||||
Blue,
|
||||
Blue,
|
||||
Red,
|
||||
Red,
|
||||
Blue,
|
||||
Blue,
|
||||
Blue,
|
||||
Red,
|
||||
}
|
||||
|
||||
func PrintBanner() {
|
||||
fmt.Println()
|
||||
fmt.Println()
|
||||
for i, line := range angelaBanner {
|
||||
c := bannerColors[i%len(bannerColors)]
|
||||
fmt.Printf(" %s\n", c(line))
|
||||
}
|
||||
fmt.Printf(" %s\n", White(HRule(52)))
|
||||
fmt.Printf(
|
||||
" %s\n\n",
|
||||
HiBlackItalic(
|
||||
"Python dependency updater & vulnerability scanner",
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
func PrintBannerWithArt() {
|
||||
fmt.Println()
|
||||
for i, line := range angelaBanner {
|
||||
c := bannerColors[i%len(bannerColors)]
|
||||
fmt.Printf(" %s\n", c(line))
|
||||
}
|
||||
fmt.Printf(" %s\n", White(HRule(52)))
|
||||
fmt.Printf(
|
||||
" %s\n",
|
||||
HiBlackItalic(
|
||||
"Python dependency updater & vulnerability scanner",
|
||||
),
|
||||
)
|
||||
fmt.Println()
|
||||
for i, line := range animeArt {
|
||||
c := artColors[i%len(artColors)]
|
||||
fmt.Printf(" %s\n", c(line))
|
||||
}
|
||||
fmt.Println()
|
||||
}
|
||||
|
|
@ -0,0 +1,94 @@
|
|||
// ©AngelaMos | 2026
|
||||
// color.go
|
||||
|
||||
package ui
|
||||
|
||||
import "github.com/fatih/color"
|
||||
|
||||
var (
|
||||
Black = color.New(color.FgBlack).SprintFunc()
|
||||
Red = color.New(color.FgRed).SprintFunc()
|
||||
Green = color.New(color.FgGreen).SprintFunc()
|
||||
Yellow = color.New(color.FgYellow).SprintFunc()
|
||||
Blue = color.New(color.FgBlue).SprintFunc()
|
||||
Magenta = color.New(color.FgMagenta).SprintFunc()
|
||||
Cyan = color.New(color.FgCyan).SprintFunc()
|
||||
White = color.New(color.FgWhite).SprintFunc()
|
||||
|
||||
HiBlack = color.New(color.FgHiBlack).SprintFunc()
|
||||
HiRed = color.New(color.FgHiRed).SprintFunc()
|
||||
HiGreen = color.New(color.FgHiGreen).SprintFunc()
|
||||
HiYellow = color.New(color.FgHiYellow).SprintFunc()
|
||||
HiBlue = color.New(color.FgHiBlue).SprintFunc()
|
||||
HiMagenta = color.New(color.FgHiMagenta).SprintFunc()
|
||||
HiCyan = color.New(color.FgHiCyan).SprintFunc()
|
||||
HiWhite = color.New(color.FgHiWhite).SprintFunc()
|
||||
|
||||
RedBold = color.New(color.FgRed, color.Bold).SprintFunc()
|
||||
GreenBold = color.New(color.FgGreen, color.Bold).SprintFunc()
|
||||
YellowBold = color.New(color.FgYellow, color.Bold).SprintFunc()
|
||||
BlueBold = color.New(color.FgBlue, color.Bold).SprintFunc()
|
||||
MagentaBold = color.New(
|
||||
color.FgMagenta, color.Bold,
|
||||
).SprintFunc()
|
||||
CyanBold = color.New(
|
||||
color.FgCyan, color.Bold,
|
||||
).SprintFunc()
|
||||
HiRedBold = color.New(
|
||||
color.FgHiRed, color.Bold,
|
||||
).SprintFunc()
|
||||
HiMagentaBold = color.New(
|
||||
color.FgHiMagenta, color.Bold,
|
||||
).SprintFunc()
|
||||
HiCyanBold = color.New(
|
||||
color.FgHiCyan, color.Bold,
|
||||
).SprintFunc()
|
||||
|
||||
Dim = color.New(color.Faint).SprintFunc()
|
||||
DimItalic = color.New(
|
||||
color.Faint, color.Italic,
|
||||
).SprintFunc()
|
||||
|
||||
CyanItalic = color.New(
|
||||
color.FgCyan, color.Italic,
|
||||
).SprintFunc()
|
||||
MagentaItalic = color.New(
|
||||
color.FgMagenta, color.Italic,
|
||||
).SprintFunc()
|
||||
RedItalic = color.New(
|
||||
color.FgRed, color.Italic,
|
||||
).SprintFunc()
|
||||
GreenItalic = color.New(
|
||||
color.FgGreen, color.Italic,
|
||||
).SprintFunc()
|
||||
HiBlackItalic = color.New(
|
||||
color.FgHiBlack, color.Italic,
|
||||
).SprintFunc()
|
||||
BlueItalic = color.New(
|
||||
color.FgBlue, color.Italic,
|
||||
).SprintFunc()
|
||||
HiBlueItalic = color.New(
|
||||
color.FgHiBlue, color.Italic,
|
||||
).SprintFunc()
|
||||
|
||||
CyanUnderline = color.New(
|
||||
color.FgCyan, color.Underline,
|
||||
).SprintFunc()
|
||||
MagentaUnderline = color.New(
|
||||
color.FgMagenta, color.Underline,
|
||||
).SprintFunc()
|
||||
|
||||
HiCyanBoldItalic = color.New(
|
||||
color.FgHiCyan, color.Bold, color.Italic,
|
||||
).SprintFunc()
|
||||
HiMagentaBoldItalic = color.New(
|
||||
color.FgHiMagenta, color.Bold, color.Italic,
|
||||
).SprintFunc()
|
||||
HiRedBoldItalic = color.New(
|
||||
color.FgHiRed, color.Bold, color.Italic,
|
||||
).SprintFunc()
|
||||
|
||||
HiBlackCrossed = color.New(
|
||||
color.FgHiBlack, color.CrossedOut,
|
||||
).SprintFunc()
|
||||
)
|
||||
|
|
@ -0,0 +1,83 @@
|
|||
// ©AngelaMos | 2026
|
||||
// spinner.go
|
||||
|
||||
package ui
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
var frames = []string{
|
||||
"⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏",
|
||||
}
|
||||
|
||||
type Spinner struct {
|
||||
msg string
|
||||
done chan struct{}
|
||||
wg sync.WaitGroup
|
||||
mu sync.Mutex
|
||||
active bool
|
||||
}
|
||||
|
||||
func NewSpinner(msg string) *Spinner {
|
||||
return &Spinner{msg: msg}
|
||||
}
|
||||
|
||||
func (s *Spinner) Start() {
|
||||
s.mu.Lock()
|
||||
if s.active {
|
||||
s.mu.Unlock()
|
||||
return
|
||||
}
|
||||
s.active = true
|
||||
s.done = make(chan struct{})
|
||||
s.wg.Add(1)
|
||||
s.mu.Unlock()
|
||||
|
||||
go s.run()
|
||||
}
|
||||
|
||||
func (s *Spinner) Stop() {
|
||||
s.mu.Lock()
|
||||
if !s.active {
|
||||
s.mu.Unlock()
|
||||
return
|
||||
}
|
||||
s.active = false
|
||||
close(s.done)
|
||||
s.mu.Unlock()
|
||||
s.wg.Wait()
|
||||
}
|
||||
|
||||
func (s *Spinner) run() {
|
||||
defer s.wg.Done()
|
||||
fmt.Print("\033[?25l")
|
||||
|
||||
ticker := time.NewTicker(80 * time.Millisecond)
|
||||
defer ticker.Stop()
|
||||
|
||||
idx := 0
|
||||
for {
|
||||
select {
|
||||
case <-s.done:
|
||||
clearLine()
|
||||
fmt.Print("\033[?25h")
|
||||
return
|
||||
case <-ticker.C:
|
||||
frame := frames[idx%len(frames)]
|
||||
fmt.Printf(
|
||||
"\r %s %s",
|
||||
CyanBold(frame),
|
||||
HiMagenta(s.msg),
|
||||
)
|
||||
idx++
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func clearLine() {
|
||||
fmt.Print("\r" + strings.Repeat(" ", 80) + "\r")
|
||||
}
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
// ©AngelaMos | 2026
|
||||
// symbol.go
|
||||
|
||||
package ui
|
||||
|
||||
import "strings"
|
||||
|
||||
const (
|
||||
Arrow = "→"
|
||||
ArrowRight = "▸"
|
||||
ArrowUp = "↑"
|
||||
Diamond = "◆"
|
||||
Gem = "◈"
|
||||
Star = "✦"
|
||||
TriangleUp = "▲"
|
||||
Check = "✓"
|
||||
Cross = "✗"
|
||||
Timer = "⏱"
|
||||
DividerChar = "━"
|
||||
)
|
||||
|
||||
func HRule(width int) string {
|
||||
return strings.Repeat(DividerChar, width)
|
||||
}
|
||||
|
|
@ -1,20 +1,20 @@
|
|||
# Go Concurrency Patterns Used in angela
|
||||
|
||||
This document explains the concurrency patterns angela uses to fetch data from PyPI and OSV.dev in parallel. These are production patterns used at Uber, Google, and Cloudflare — not textbook exercises.
|
||||
angela fetches data from PyPI and OSV.dev in parallel. These patterns show up in production codebases at places like Uber and Cloudflare, but they're not complicated — they just require knowing which tool to reach for.
|
||||
|
||||
---
|
||||
|
||||
## The problem
|
||||
|
||||
angela needs to query PyPI for every dependency in your `pyproject.toml`. A typical project has 20-50 dependencies. Making those requests sequentially would take 10-25 seconds. Making them all at once would hammer PyPI with 50 simultaneous connections.
|
||||
angela needs to query PyPI for every dependency in your `pyproject.toml` or `requirements.txt`. A typical project has 20-50 dependencies. Doing those requests one at a time would take 10-25 seconds. Doing them all at once would hammer PyPI with 50 simultaneous connections.
|
||||
|
||||
The solution: **bounded concurrency** — run up to N requests in parallel, queuing the rest.
|
||||
The answer is **bounded concurrency** — run up to N requests in parallel, queue the rest.
|
||||
|
||||
---
|
||||
|
||||
## Pattern 1: errgroup.SetLimit
|
||||
## errgroup.SetLimit
|
||||
|
||||
The `golang.org/x/sync/errgroup` package provides the cleanest API for bounded concurrent work in Go. Here's how angela uses it in `internal/pypi/client.go`:
|
||||
The `golang.org/x/sync/errgroup` package is the go-to for bounded concurrent work in Go. Here's how angela uses it in `internal/pypi/client.go`:
|
||||
|
||||
```go
|
||||
g, ctx := errgroup.WithContext(ctx)
|
||||
|
|
@ -35,18 +35,18 @@ for _, name := range names {
|
|||
_ = g.Wait()
|
||||
```
|
||||
|
||||
Key decisions:
|
||||
A few things to notice:
|
||||
|
||||
1. **`SetLimit(10)`** — caps concurrent HTTP requests. PyPI recommends 5-10.
|
||||
2. **Always returns nil** — individual package failures are collected in results, not propagated. One failed package shouldn't cancel the others.
|
||||
3. **Mutex protects shared slice** — `results` is appended to from multiple goroutines.
|
||||
- **`SetLimit(10)`** caps concurrent HTTP requests. PyPI recommends 5-10.
|
||||
- **Always returns nil** — individual failures get collected in results, not propagated. One package failing shouldn't kill the others.
|
||||
- **Mutex on the shared slice** — `results` gets appended to from multiple goroutines, so it needs a lock.
|
||||
|
||||
### Why not channels?
|
||||
|
||||
A channel-based worker pool would work, but errgroup.SetLimit does the same thing with less code:
|
||||
Channels would work, but errgroup gets you the same thing with half the code:
|
||||
|
||||
```go
|
||||
// Channel approach: ~30 lines of setup
|
||||
// Channel-based worker pool: ~30 lines
|
||||
jobs := make(chan string, len(names))
|
||||
results := make(chan FetchResult, len(names))
|
||||
for range maxWorkers {
|
||||
|
|
@ -57,9 +57,9 @@ for range maxWorkers {
|
|||
}
|
||||
}()
|
||||
}
|
||||
// ... send jobs, collect results ...
|
||||
// ... send jobs, collect results, close channels ...
|
||||
|
||||
// errgroup approach: ~15 lines
|
||||
// errgroup: ~15 lines
|
||||
g.SetLimit(maxWorkers)
|
||||
for _, name := range names {
|
||||
g.Go(func() error { /* ... */ })
|
||||
|
|
@ -67,13 +67,13 @@ for _, name := range names {
|
|||
g.Wait()
|
||||
```
|
||||
|
||||
The errgroup version is shorter, handles context cancellation automatically, and has no channel lifecycle to manage.
|
||||
errgroup handles context cancellation for free and there's no channel lifecycle to think about.
|
||||
|
||||
---
|
||||
|
||||
## Pattern 2: Panic recovery in goroutines
|
||||
## Panic recovery
|
||||
|
||||
Every goroutine angela launches includes panic recovery:
|
||||
Every goroutine angela launches wraps itself in a recover:
|
||||
|
||||
```go
|
||||
g.Go(func() (err error) {
|
||||
|
|
@ -89,32 +89,33 @@ g.Go(func() (err error) {
|
|||
})
|
||||
```
|
||||
|
||||
**Why this matters**: an unrecovered panic in a goroutine kills the entire process. In a CLI tool, that means the user sees a stack trace instead of a helpful error message. The `defer recover()` converts panics into errors that flow through the normal error path.
|
||||
An unrecovered panic in a goroutine kills the entire process. For a CLI tool, that means the user gets a stack trace dump instead of an actual error message. The `defer recover()` turns panics into errors that flow through the normal path.
|
||||
|
||||
The named return `(err error)` is essential — it lets the deferred function assign the recovered panic as the return value.
|
||||
One subtle thing — the named return `(err error)` is what makes this work. Without it, the deferred function has no way to set the return value.
|
||||
|
||||
---
|
||||
|
||||
## Pattern 3: Context cancellation
|
||||
## Context cancellation
|
||||
|
||||
Every HTTP request in angela uses context:
|
||||
Every HTTP request uses context:
|
||||
|
||||
```go
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
```
|
||||
|
||||
This means:
|
||||
- If the user presses Ctrl+C, pending requests are cancelled
|
||||
- If `errgroup.WithContext` detects an error, remaining work stops
|
||||
This does a lot of work for you:
|
||||
|
||||
- Ctrl+C cancels pending requests
|
||||
- If errgroup's context gets cancelled, remaining work stops
|
||||
- HTTP timeouts are enforced at the transport level
|
||||
|
||||
The context flows from `cobra.Command.Context()` through `runUpdate()` through `FetchAllVersions()` through `FetchVersions()` to the actual HTTP call. This is the standard Go pattern: context as the first parameter, threaded through the entire call chain.
|
||||
The context flows from `cobra.Command.Context()` → `runUpdate()` → `FetchAllVersions()` → `FetchVersions()` → the HTTP call itself. Standard Go pattern — context as the first parameter, threaded through everything.
|
||||
|
||||
---
|
||||
|
||||
## Pattern 4: Retry with exponential backoff
|
||||
## Retry with exponential backoff
|
||||
|
||||
`internal/pypi/client.go` implements retry for transient failures:
|
||||
`internal/pypi/client.go` retries transient failures:
|
||||
|
||||
```go
|
||||
for attempt := range maxRetries {
|
||||
|
|
@ -142,18 +143,13 @@ for attempt := range maxRetries {
|
|||
}
|
||||
```
|
||||
|
||||
Design choices:
|
||||
|
||||
1. **Exponential backoff** — `1<<shift` doubles the delay each attempt (500ms, 1s, 2s)
|
||||
2. **Context-aware wait** — the `select` respects cancellation even during the delay
|
||||
3. **Only retry server errors** — 5xx is retryable, 4xx is not
|
||||
4. **Max 3 attempts** — enough to survive a transient blip, not enough to block the user
|
||||
The `1<<shift` doubles the delay each attempt — 500ms, 1s, 2s. The `select` means cancellation is still respected while waiting between retries. Only server errors (5xx) get retried; client errors (4xx) don't because retrying a 404 won't make the package appear. Three attempts is enough to ride out a transient blip without making the user wait forever.
|
||||
|
||||
---
|
||||
|
||||
## Pattern 5: Mutex vs channel for result collection
|
||||
## Mutex vs channel
|
||||
|
||||
angela uses `sync.Mutex` to protect the results slice, not a channel:
|
||||
angela uses `sync.Mutex` to protect the results slice:
|
||||
|
||||
```go
|
||||
var mu sync.Mutex
|
||||
|
|
@ -165,7 +161,7 @@ results = append(results, result)
|
|||
mu.Unlock()
|
||||
```
|
||||
|
||||
**When to use which:**
|
||||
Quick rule of thumb:
|
||||
|
||||
| Use Mutex | Use Channel |
|
||||
|-----------|-------------|
|
||||
|
|
@ -173,18 +169,12 @@ mu.Unlock()
|
|||
| Simple append/read operations | Producer-consumer pipelines |
|
||||
| Order doesn't matter | Sequential processing needed |
|
||||
|
||||
For angela's case — appending results from concurrent workers — a mutex is simpler and allocates less.
|
||||
For appending results where order doesn't matter, a mutex is simpler and cheaper.
|
||||
|
||||
---
|
||||
|
||||
## What to learn from this
|
||||
## The bigger picture
|
||||
|
||||
1. **errgroup.SetLimit is the standard** — it replaced raw `go func()` + `sync.WaitGroup` + semaphore channels. Use it for any bounded concurrent work.
|
||||
All of these patterns boil down to the same idea: **know when your goroutine exits**. Every goroutine angela spawns has a clear owner (the errgroup), a clear exit condition (the function returns), and panic protection. That's how you avoid leaks.
|
||||
|
||||
2. **Always recover panics in goroutines** — an unrecovered panic kills the process. Production code wraps every `g.Go` callback with `defer recover()`.
|
||||
|
||||
3. **Thread context through everything** — from CLI command to HTTP request, context enables cancellation and timeouts at every level.
|
||||
|
||||
4. **Choose the right concurrency primitive** — mutex for shared state, channels for data flow, errgroup for work coordination. Don't use channels when a mutex is clearer.
|
||||
|
||||
5. **`go test -race` is mandatory** — Go's race detector catches data races that code review misses. Run it in CI, always.
|
||||
`go test -race` catches most of what code review misses. Run it in CI, always.
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
# PEP 440: Python Version Parsing from Scratch
|
||||
|
||||
This document explains how angela parses Python version strings — one of the hardest parts of the project. If you're building tools that interact with the Python ecosystem, understanding PEP 440 is non-negotiable.
|
||||
Parsing Python version strings was one of the harder parts of building angela. If you're working on anything that touches the Python ecosystem, PEP 440 is worth understanding — it's weirder than you'd expect.
|
||||
|
||||
---
|
||||
|
||||
## The problem
|
||||
|
||||
Python versions are **not** semantic versioning. They follow [PEP 440](https://peps.python.org/pep-0440/), which allows components that semver doesn't:
|
||||
Python versions are **not** semantic versioning. They follow [PEP 440](https://peps.python.org/pep-0440/), which has components semver doesn't:
|
||||
|
||||
```
|
||||
[epoch!] release [pre-release] [.postN] [.devN] [+local]
|
||||
|
|
@ -22,26 +22,26 @@ Real examples from PyPI:
|
|||
| `1.0b2` | Beta pre-release |
|
||||
| `1.0rc1` | Release candidate |
|
||||
| `1.0.dev3` | Development snapshot |
|
||||
| `1.0.post1` | Post-release (stable — fixes to docs or metadata) |
|
||||
| `1.0+ubuntu1` | Local version label (never on PyPI) |
|
||||
| `1.0.post1` | Post-release (stable — just a docs or metadata fix) |
|
||||
| `1.0+ubuntu1` | Local version label (never shows up on PyPI) |
|
||||
|
||||
---
|
||||
|
||||
## Why this matters for dependency tools
|
||||
## Why you can't just string-compare
|
||||
|
||||
If you naively compare version strings, `1.0a1` looks "newer" than `1.0` because `a` > `` in ASCII. But in Python's world:
|
||||
If you naively compare version strings, `1.0a1` looks "newer" than `1.0` because `a` > empty string in ASCII. But Python's actual ordering is:
|
||||
|
||||
```
|
||||
1.0.dev1 < 1.0a1 < 1.0b1 < 1.0rc1 < 1.0 < 1.0.post1
|
||||
```
|
||||
|
||||
A tool that doesn't understand this will upgrade users to **unstable pre-releases**. That's why angela implements the full PEP 440 parser.
|
||||
A tool that gets this wrong will upgrade users to unstable pre-releases. That's why angela implements the full spec.
|
||||
|
||||
---
|
||||
|
||||
## How angela's parser works
|
||||
## How the parser works
|
||||
|
||||
The parser lives in `internal/pypi/version.go`. It uses a single compiled regex to extract all components in one pass:
|
||||
The parser lives in `internal/pypi/version.go`. It uses a single compiled regex to pull out all the components in one pass:
|
||||
|
||||
```
|
||||
(?i)^v?
|
||||
|
|
@ -53,7 +53,7 @@ The parser lives in `internal/pypi/version.go`. It uses a single compiled regex
|
|||
(?:\+([a-z0-9]...))?$ # local version
|
||||
```
|
||||
|
||||
Each named section maps to a field in the `Version` struct:
|
||||
Each section maps to a field in the `Version` struct:
|
||||
|
||||
```go
|
||||
type Version struct {
|
||||
|
|
@ -68,13 +68,13 @@ type Version struct {
|
|||
}
|
||||
```
|
||||
|
||||
The sentinel value `-1` for Post and Dev is critical — it distinguishes "not present" from "present with value 0". Both `1.0.post0` and `1.0.post` are valid (implicit zero), but they're different from `1.0` (no post-release at all).
|
||||
The `-1` sentinel for Post and Dev is important — it's how you tell "not present" apart from "present with value 0". Both `1.0.post0` and `1.0.post` are valid PEP 440 (implicit zero), but they're different from `1.0` which has no post-release at all.
|
||||
|
||||
---
|
||||
|
||||
## Normalization edge cases
|
||||
## Normalization
|
||||
|
||||
PEP 440 allows multiple spellings that all mean the same thing:
|
||||
PEP 440 allows a bunch of different spellings that all mean the same thing:
|
||||
|
||||
| Input | Normalized |
|
||||
|-------|-----------|
|
||||
|
|
@ -86,13 +86,13 @@ PEP 440 allows multiple spellings that all mean the same thing:
|
|||
| `1.0a` | `1.0a0` (implicit zero) |
|
||||
| `v1.0` | `1.0` (strip leading v) |
|
||||
|
||||
The parser handles all of these by normalizing during extraction.
|
||||
The parser normalizes all of these during extraction, so the rest of the codebase never has to think about variant spellings.
|
||||
|
||||
---
|
||||
|
||||
## Version comparison: the tuple trick
|
||||
## Version comparison
|
||||
|
||||
Python's `packaging` library compares versions by converting them into tuples that sort naturally. angela uses the same idea:
|
||||
Python's `packaging` library compares versions by converting them to tuples that sort naturally. angela does the same thing:
|
||||
|
||||
```go
|
||||
func (v Version) Compare(other Version) int {
|
||||
|
|
@ -104,7 +104,7 @@ func (v Version) Compare(other Version) int {
|
|||
}
|
||||
```
|
||||
|
||||
The key insight is the sentinel values for sorting:
|
||||
The trick is in the sentinel values:
|
||||
|
||||
| Component state | Sort key |
|
||||
|----------------|----------|
|
||||
|
|
@ -114,13 +114,13 @@ The key insight is the sentinel values for sorting:
|
|||
| Pre-release `rc1` | `2, 1` — rc rank 2 |
|
||||
| Final release (no pre) | `MaxInt, MaxInt` — sorts after all pre-releases |
|
||||
|
||||
This produces the correct PEP 440 ordering without any special-case branching in the comparison loop.
|
||||
Using `math.MinInt` and `math.MaxInt` means the comparison function is just a sequence of `cmp.Compare` calls — no special-case branching needed.
|
||||
|
||||
---
|
||||
|
||||
## Stability detection
|
||||
|
||||
A version is **stable** if it has no pre-release tag AND no dev tag:
|
||||
A version is **stable** if it has no pre-release tag and no dev tag:
|
||||
|
||||
```go
|
||||
func (v Version) IsStable() bool {
|
||||
|
|
@ -128,16 +128,14 @@ func (v Version) IsStable() bool {
|
|||
}
|
||||
```
|
||||
|
||||
Post-releases are stable — `1.0.post1` is a documentation fix or metadata correction, not an unstable build.
|
||||
Post-releases count as stable — `1.0.post1` is a docs fix or metadata correction, not an unstable build.
|
||||
|
||||
---
|
||||
|
||||
## What to learn from this
|
||||
## Takeaways
|
||||
|
||||
1. **Regex can be the right tool** — a single compiled regex handles all PEP 440 forms in one pass. The alternative (hand-written state machine) would be 3x the code for no real benefit.
|
||||
A single compiled regex handles every PEP 440 form in one pass. The alternative would be a hand-written state machine with 3x the code and no real upside.
|
||||
|
||||
2. **Sentinel values simplify comparison** — using `math.MinInt` and `math.MaxInt` lets the comparison function be a clean sequence of `cmp.Compare` calls without branches.
|
||||
Normalizing at parse time (alpha→a, preview→rc) keeps things clean — nothing downstream ever has to handle variant spellings.
|
||||
|
||||
3. **Test-driven development** — the version parser has 67 test cases across 10 functions. When implementing something spec-driven like PEP 440, write the test cases first from the spec, then make them pass.
|
||||
|
||||
4. **Normalization at parse time** — by normalizing spellings during parsing (alpha→a, preview→rc), the rest of the codebase never has to think about variant forms.
|
||||
The version parser currently has 90+ test cases across 10 functions. When you're implementing something spec-driven like this, writing the test cases from the spec first and then making them pass is the way to go.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Preserving TOML Comments: Why Regex Surgery is the Right Call
|
||||
# Preserving Comments in Dependency Files
|
||||
|
||||
When angela updates version specifiers in `pyproject.toml`, it must preserve every comment, blank line, and formatting choice the developer made. This is harder than it sounds. This document explains the approach and why alternatives don't work.
|
||||
When angela updates version specifiers, it can't blow away the developer's comments, blank lines, and formatting. This turns out to be a surprisingly annoying problem. Here's how angela handles it for both `pyproject.toml` and `requirements.txt`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -36,13 +36,13 @@ dependencies = [
|
|||
]
|
||||
```
|
||||
|
||||
Only the version numbers changed. Every comment, every space, every quote style — preserved.
|
||||
Only the version numbers changed. Every comment, every space, every quote style — still there. Same goes for `requirements.txt` — inline comments, section headers, all of it should survive.
|
||||
|
||||
---
|
||||
|
||||
## Why TOML libraries can't do this
|
||||
|
||||
Both major Go TOML libraries (BurntSushi/toml and pelletier/go-toml) work by unmarshaling TOML into Go structs, then marshaling back:
|
||||
Both major Go TOML libraries (BurntSushi/toml and pelletier/go-toml) work by unmarshaling into Go structs, then marshaling back:
|
||||
|
||||
```go
|
||||
var proj PyProject
|
||||
|
|
@ -51,32 +51,32 @@ proj.Dependencies[0] = "requests>=2.32.3"
|
|||
output, _ := toml.Marshal(proj) // comments gone, formatting changed
|
||||
```
|
||||
|
||||
Go's reflection system has no mechanism to store comment metadata on struct fields. The unmarshal/marshal round-trip fundamentally **destroys comments**. This isn't a bug — it's an architectural limitation of how Go TOML libraries work.
|
||||
Go's reflection system has no way to store comment metadata on struct fields. The unmarshal/marshal round-trip just destroys them. This isn't a library bug — it's a fundamental limitation of how Go struct serialization works.
|
||||
|
||||
pelletier/go-toml v2 has an `unstable` package with AST access that preserves comments in the parse tree, but it provides no serialization — you'd have to write your own TOML emitter from the AST.
|
||||
pelletier/go-toml v2 does have an `unstable` package that exposes AST access with comments preserved in the parse tree, but there's no serializer — you'd have to write your own TOML emitter from scratch.
|
||||
|
||||
---
|
||||
|
||||
## The regex surgery approach
|
||||
## Regex surgery
|
||||
|
||||
angela's solution: don't parse and re-serialize. Instead, treat the file as a byte buffer and surgically replace only the version specifier substring.
|
||||
angela's approach: don't parse and re-serialize. Treat the file as a byte buffer and surgically replace only the version specifier.
|
||||
|
||||
The implementation in `internal/pyproject/writer.go`:
|
||||
The implementation for pyproject.toml lives in `internal/pyproject/writer.go`:
|
||||
|
||||
1. **Build a regex** that matches the full dependency string including quotes:
|
||||
1. Build a regex that matches the full dependency string inside quotes:
|
||||
```
|
||||
"requests>=2.28.0"
|
||||
```
|
||||
|
||||
2. **Capture groups** isolate the parts we need:
|
||||
2. Capture groups isolate the pieces:
|
||||
- Group 1: package name (`requests`)
|
||||
- Group 2: extras (`[async]` or empty)
|
||||
- Group 3: version specifier (`>=2.28.0`)
|
||||
- Group 4: markers (`;python_version>='3.8'` or empty)
|
||||
|
||||
3. **Replace only group 3** (the version spec) while preserving everything else.
|
||||
3. Replace only group 3, keep everything else.
|
||||
|
||||
4. **Validate before and after** — feed the result through go-toml v2's unmarshaler to catch syntax errors.
|
||||
4. Validate before and after — feed the result through go-toml v2's unmarshaler to make sure the edit didn't break the TOML syntax.
|
||||
|
||||
```go
|
||||
func (u *Updater) UpdateDependency(pkg, newSpec string) error {
|
||||
|
|
@ -90,7 +90,6 @@ func (u *Updater) UpdateDependency(pkg, newSpec string) error {
|
|||
},
|
||||
)
|
||||
if found {
|
||||
// Re-validate TOML syntax
|
||||
var probe map[string]any
|
||||
if err := toml.Unmarshal(u.content, &probe); err != nil {
|
||||
return fmt.Errorf("update produced invalid TOML: %w", err)
|
||||
|
|
@ -104,6 +103,21 @@ func (u *Updater) UpdateDependency(pkg, newSpec string) error {
|
|||
|
||||
---
|
||||
|
||||
## requirements.txt uses the same idea
|
||||
|
||||
The `requirements.txt` writer in `internal/requirements/writer.go` works the same way — regex-based surgery on raw bytes. The difference is simpler: no quotes to worry about, no TOML validation needed. Lines look like:
|
||||
|
||||
```
|
||||
django>=3.2.0
|
||||
requests==2.28.1 # HTTP client
|
||||
```
|
||||
|
||||
The regex matches the package name (with PEP 503 normalization) and its version specifier, replaces the spec, and leaves everything else alone — including inline comments.
|
||||
|
||||
Both writers share the same atomic write pattern (temp file + rename) and the same PEP 503 name normalization approach. The pyproject writer just has extra complexity for quote styles and TOML validation.
|
||||
|
||||
---
|
||||
|
||||
## The tricky parts
|
||||
|
||||
### PEP 503 name normalization
|
||||
|
|
@ -114,7 +128,7 @@ Package names on PyPI are case-insensitive and treat `-`, `_`, and `.` as equiva
|
|||
Some_Package == some-package == some.package
|
||||
```
|
||||
|
||||
The regex must match all variants. angela splits the normalized name on `-` and joins with `[-_.]?`:
|
||||
The regex has to match all variants. angela splits the normalized name on `-` and joins with `[-_.]?`:
|
||||
|
||||
```go
|
||||
parts := strings.Split(normalized, "-")
|
||||
|
|
@ -124,17 +138,17 @@ for i, p := range parts {
|
|||
namePattern := strings.Join(parts, `[-_.]?`)
|
||||
```
|
||||
|
||||
This means the pattern for `some-package` becomes `some[-_.]?package`, matching `some_package`, `some.package`, and `somepackage`.
|
||||
So the pattern for `some-package` becomes `some[-_.]?package`, which matches `some_package`, `some.package`, and `somepackage`.
|
||||
|
||||
### Go RE2 doesn't support backreferences
|
||||
|
||||
The TOML research suggested using `\1` to match the closing quote with the opening quote. Go's `regexp` package uses RE2, which doesn't support backreferences.
|
||||
The TOML research suggested using `\1` to match closing quotes with the opening quote. Go's `regexp` package uses RE2, which doesn't do backreferences.
|
||||
|
||||
The fix: try each quote style separately. Loop over `{'"', '\''}` and build a pattern specific to that quote character.
|
||||
The workaround: try each quote style separately. Loop over `{'"', '\''}` and build a pattern specific to that quote character.
|
||||
|
||||
### Atomic file writes
|
||||
|
||||
Even the write is careful — angela writes to a `.tmp` file first, then renames over the original. If the process is killed mid-write, the original file is untouched:
|
||||
angela writes to a `.tmp` file first, then renames over the original. If the process dies mid-write, the original is untouched:
|
||||
|
||||
```go
|
||||
func (u *Updater) WriteFile(path string) error {
|
||||
|
|
@ -150,20 +164,12 @@ func (u *Updater) WriteFile(path string) error {
|
|||
}
|
||||
```
|
||||
|
||||
It's two syscalls, but the file is never in a half-written state.
|
||||
|
||||
---
|
||||
|
||||
## This is how production tools do it
|
||||
|
||||
Renovate (GitHub's automated dependency updater) and Dependabot both use regex/string manipulation for updating dependency files. They don't parse and re-serialize — they surgically modify. The reason is the same: no TOML/YAML/JSON library in any language perfectly round-trips formatting and comments.
|
||||
Renovate and Dependabot both use regex/string manipulation for updating dependency files. They don't parse and re-serialize either — they do the same kind of surgical modification. No TOML, YAML, or JSON library in any language perfectly round-trips formatting and comments. Regex surgery sounds hacky, but it's the approach that actually works.
|
||||
|
||||
---
|
||||
|
||||
## What to learn from this
|
||||
|
||||
1. **Sometimes the "crude" approach is correct** — regex surgery sounds hacky, but it's the only way to preserve comments in TOML. The fancier approaches (AST manipulation, custom serializer) are more complex and still lose whitespace.
|
||||
|
||||
2. **Validate at boundaries** — angela validates TOML syntax before and after every edit. The regex does the surgery; go-toml v2 confirms the patient survived.
|
||||
|
||||
3. **Atomic writes prevent corruption** — write to temp + rename is the standard pattern for file updates in Unix. It's two syscalls, but it guarantees the file is never half-written.
|
||||
|
||||
4. **Know your regex engine** — Go uses RE2, which guarantees linear-time matching but doesn't support backreferences or lookaheads. Design patterns accordingly.
|
||||
Go's RE2 engine guarantees linear-time matching, so even on large files with complex patterns there's no risk of catastrophic backtracking. Just keep in mind it doesn't support backreferences or lookaheads — design your patterns around that.
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ name = "sample-app"
|
|||
version = "0.1.0"
|
||||
requires-python = ">=3.10"
|
||||
|
||||
# Core runtime dependencies
|
||||
# Core runtime dependencies (outdated)
|
||||
dependencies = [
|
||||
"requests>=2.28.0",
|
||||
"django>=3.2,<4.0",
|
||||
|
|
@ -14,7 +14,7 @@ dependencies = [
|
|||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
# Development and testing tools
|
||||
# Development and testing tools (outdated)
|
||||
dev = [
|
||||
"pytest>=7.0.0",
|
||||
"black>=23.0.0", # Code formatter
|
||||
|
|
|
|||
Loading…
Reference in New Issue