mirror of https://github.com/razor-ai/soup.git
80 lines
2.9 KiB
Markdown
80 lines
2.9 KiB
Markdown
# v0.71.14 — Export QA log
|
||
|
||
Patch: **v0.71.14 — "Export QA + serve(transformers) + GPU smoke finale"**
|
||
Box: Windows 11 + RTX 3050 Laptop GPU (4 GB, Ampere, compute capability 8.6)
|
||
Date: 2026-06-05
|
||
|
||
Environment:
|
||
|
||
| lib | version |
|
||
|---|---|
|
||
| torch | 2.5.1+cu121 |
|
||
| transformers | 4.57.6 |
|
||
| optimum | 2.1.0 |
|
||
| onnxruntime | 1.23.2 |
|
||
| CUDA available | yes (RTX 3050 Laptop GPU) |
|
||
|
||
---
|
||
|
||
## #71 — ONNX export pipeline — **PASS**
|
||
|
||
Verified `soup export --format onnx` end-to-end on a locally-built tiny GPT-2
|
||
(2 layers / 2 heads / n_embd=32 / vocab 512, saved with safetensors so it loads
|
||
under torch 2.5.1).
|
||
|
||
```bash
|
||
soup export --model ./model --format onnx --output ./onnx_out
|
||
```
|
||
|
||
Result:
|
||
|
||
- Export exited **0**.
|
||
- `onnx_out/model.onnx` written — **252,445 bytes**.
|
||
- `onnxruntime.InferenceSession(model.onnx, providers=["CPUExecutionProvider"])`
|
||
loaded it successfully.
|
||
- Inputs: `['input_ids', 'attention_mask', 'position_ids']` — `input_ids` present
|
||
as required by the acceptance criteria.
|
||
- One forward `sess.run(...)` on `input_ids=[[1,2,3,4]]` produced output shape
|
||
`(1, 4, 512)` = (batch, seq_len, vocab) — a real, runnable graph.
|
||
|
||
Notes:
|
||
- Tracer warnings (`aten::index` advanced indexing, masking_utils boolean
|
||
conversion) are emitted by `optimum.exporters.onnx.main_export` during the
|
||
trace; the exported graph still loads + runs. These are upstream optimum/torch
|
||
warnings, not a Soup bug.
|
||
- The historical v0.53.2 limitation (TinyLlama-1.1B ONNX export OOM during
|
||
`onnx.load(load_external_data=True)`) is a host-RAM constraint, unchanged;
|
||
the pipeline integrity is what #71 verifies, and that is confirmed here.
|
||
|
||
---
|
||
|
||
## #70 — GGUF export — **DEFERRED (infra-blocked)**
|
||
|
||
Needs a built llama.cpp C++ toolchain (the pipeline shells out to a pinned
|
||
llama.cpp tag). The maintainer's Windows 11 + RTX 3050 box cannot reliably
|
||
build the llama.cpp C++ chain — this is stated in issue #70 itself. Kept OPEN
|
||
with the `infra-blocked` label; recipe + acceptance criteria live in
|
||
`tests/qa/v053_qa.md`.
|
||
|
||
## #72 — AWQ + GPTQ export — **DEFERRED (infra-blocked)**
|
||
|
||
Needs `autoawq` + `auto-gptq` (or `gptqmodel`). None are installed and their
|
||
Windows wheels are unmaintained / build-from-source only (autoawq is archived).
|
||
Kept OPEN with the `infra-blocked` label.
|
||
|
||
## #144 — CUDA + llama.cpp manual QA smoke doc — **DEFERRED**
|
||
|
||
Depends on #70 (GGUF) and #72 (AWQ/GPTQ) actually running, both of which are
|
||
infra-blocked on this box. Kept OPEN.
|
||
|
||
## #74 — HF Hub push + Spaces deploy QA — **DEFERRED (external account)**
|
||
|
||
Needs an HF Write token + account (no credentials available on the QA box).
|
||
Previously deferred in v0.53.4 to "a contributor with private HF credentials".
|
||
Kept OPEN with the `infra-blocked` label.
|
||
|
||
## #79 — [Tracking] Community-QA meta-issue — **STAYS OPEN**
|
||
|
||
Closes only once #70–#75 all land. Since #70 / #72 / #73 / #74 / #75 remain
|
||
blocked on this box, the tracking issue stays open.
|