diff --git a/.github/README.md b/.github/README.md new file mode 100644 index 000000000..007fdff04 --- /dev/null +++ b/.github/README.md @@ -0,0 +1,183 @@ +# .github — Repository Automation & Contributing Guide + +This directory contains all GitHub-specific configuration: CI/CD workflows, issue templates, pull request templates, and tooling documentation. + +--- + +## 📁 Directory Structure + +``` +.github/ +├── workflows/ # GitHub Actions CI/CD pipelines +│ ├── tests-ubuntu.yml # Test matrix on Ubuntu (Python 3.10–3.14, PyPy) +│ ├── tests-macos.yml # Test matrix on macOS +│ ├── tests-windows.yml # Test matrix on Windows +│ ├── checks.yml # Static analysis: pylint, mypy, docs, twine +│ ├── publish.yml # PyPI publish on release +│ └── auto-close-llm-pr.yml # Auto-close unreviewed LLM-generated PRs +├── ISSUE_TEMPLATE/ +│ ├── bug_report.md # Bug report template +│ ├── feature_request.md # Feature request template +│ └── question.md # General question template +├── pull_request_template.md # PR checklist and contributor guidance +└── README.md # This file +``` + +--- + +## ⚙️ CI/CD Workflows + +### Test Workflows + +Tests run automatically on every push to `master` / version branches, and on every pull request. + +| Workflow | OS | Python versions | +| --- | --- | --- | +| `tests-ubuntu.yml` | Ubuntu | 3.10, 3.11, 3.12, 3.13, 3.14, PyPy 3.11 | +| `tests-macos.yml` | macOS | 3.14 | +| `tests-windows.yml` | Windows | 3.14 | + +Each matrix entry runs `tox` with a specific `TOXENV`, covering: + +- **Standard test environments** (`py`, `default-reactor`, `no-reactor`) +- **Minimum dependency environments** (`min`, `min-extra-deps`, `min-botocore`, etc.) +- **Extra dependency environments** (`extra-deps`, `botocore`, `mitmproxy`) +- **PyPy environments** (`pypy3`, `pypy3-extra-deps`) + +Coverage reports are uploaded to [Codecov](https://codecov.io/) at the end of each test run. + +### Checks Workflow (`checks.yml`) + +Runs on every push and PR: + +| Check | Tool | Python | +| --- | --- | --- | +| Type checking | `mypy` | 3.10 | +| Type checking (tests) | `mypy-tests` | 3.10 | +| Linting | `pylint` | 3.14 | +| Documentation build | `docs` | 3.14 | +| Documentation tests | `docs-tests` | 3.13 | +| Package check | `twinecheck` | 3.14 | +| Pre-commit hooks | `pre-commit` | any | + +### Publish Workflow (`publish.yml`) + +Publishes to [PyPI](https://pypi.org/project/Scrapy/) when a new GitHub Release is created. + +--- + +## 🧠 Delta Smart Test Selection (Local Development) + +[**Delta**](https://deltatest.dev) (`deltatest-cli`) is installed in the local `.venv` and enables **running only the tests affected by your code changes** — dramatically reducing feedback time during development. + +### How it works + +1. Delta tracks which tests execute which lines of source code via `coverage.py` context tracking. +2. On each run, it diffs your working branch against a base branch (e.g. `master`). +3. Only the tests that cover the changed lines are selected and executed. + +### Usage + +```bash +# Run only tests affected by your changes vs master +.venv/bin/delta run --local --base-branch master + +# Preview which tests would run (dry run) +.venv/bin/delta run --local --base-branch master --dry-run + +# Explain exactly why each test was selected +.venv/bin/delta run --local --base-branch master --explain + +# Rebuild the coverage mapping database (first time or after large refactors) +.venv/bin/delta build-mapping --local +``` + +### Benchmark Results + +A helper script at [`extras/delta_benchmark.py`](../extras/delta_benchmark.py) measures the actual savings Delta provides per commit. Example result on this project: + +| Commit | No Delta (P/F/S/Total) | Delta (P/F/S/Total) | Test Savings | Time Savings | +| --- | --- | --- | --- | --- | +| Add coverage for `Item.__delitem__()` | 3645 / 0 / 478 / 4123 | 125 / 0 / 0 / 125 | **97.0%** | **87.2%** (395s → 51s) | + +Run the benchmark yourself: +```bash +.venv/bin/python extras/delta_benchmark.py +``` + +--- + +## 💡 Idea: Delta as a GitHub App / Check + +> **Can Delta pre-check every commit and run only relevant tests before any full CI job starts?** + +Yes — and here's how it could work: + +### Architecture Overview + +``` +PR opened / commit pushed + │ + ▼ +┌─────────────────────────┐ +│ GitHub App Webhook │ ← registers a "pending" check on the commit +│ (Node.js / Python) │ +└────────────┬────────────┘ + │ git diff (changed files + lines) + ▼ +┌─────────────────────────┐ +│ Delta Cloud API │ ← queries which tests cover changed lines +│ (test_mapping.db) │ +└────────────┬────────────┘ + │ list of affected test IDs + ▼ +┌─────────────────────────┐ +│ Ephemeral Runner │ ← runs ONLY the selected tests (fast feedback) +│ (GitHub Actions / pod) │ +└────────────┬────────────┘ + │ pass / fail + ▼ +┌─────────────────────────┐ +│ GitHub Check API │ ← posts result as a required status check +│ "Delta Pre-Check ✅" │ +└─────────────────────────┘ + │ + ▼ (if passed, full CI matrix starts in parallel or is skipped) + Full CI (tox matrix) +``` + +### Implementation Steps + +1. **Register a GitHub App** with `checks:write`, `pull_requests:read`, and `contents:read` permissions. +2. **Webhook endpoint** listens for `pull_request` and `push` events. +3. **On event**: create a `check_run` in `in_progress` state via the [Checks API](https://docs.github.com/en/rest/checks). +4. **Query Delta Cloud** (or local DB) with the changed files/lines to get affected test IDs. +5. **Spin up a runner** (GitHub Actions `workflow_dispatch`, a container, or a self-hosted runner) and run only those tests. +6. **Report result** back to the commit via `PATCH /check_runs/{check_run_id}` with `conclusion: success | failure`. +7. **Gate the full CI matrix** behind this check using [branch protection rules](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches) — the full matrix only starts if the Delta pre-check passes, or runs in parallel and can be cancelled early on failure. + +### Key Benefits + +- 🚀 **Instant feedback** (seconds instead of minutes) on whether the changed code breaks any related tests. +- 💰 **CI cost reduction** — full matrix only runs when the fast pre-check passes. +- 🔁 **Incremental**: the mapping database grows more accurate with every run. + +### Existing Integration Point + +The `delta.pytest_plugin` is already installed in this project. Passing `--delta` to pytest activates smart test filtering inline, which is exactly what the runner step above would use: + +```bash +pytest --delta --delta-local --delta-base origin/master +``` + +--- + +## 🤝 Contributing + +Please read the [contributing guide](../CONTRIBUTING.md) and the [pull request template](pull_request_template.md) before submitting changes. + +Key points: +- Run `tox` locally and ensure all existing tests pass. +- Add or update relevant tests — all new code should have complete test coverage. +- Update documentation for any user-facing changes. +- Reference any issues you are resolving (e.g. `Resolves #123`).