Jaret Burkett c78b1404e3 Deepen offload prefetch pipeline with per-slot events ... Replace the 2-slot ping-pong + single global "compute-started" event with a depth-N ring buffer where each transfer waits only on the slot it's reusing (D layers back) instead of the most-recent compute. Applies to forward and backward, Linear and Conv. Depth is tunable via AI_TOOLKIT_OFFLOAD_DEPTH (default 4). Bit-exact vs non-offload (output, grad_input, weight grads). No speedup on a bandwidth-bound PCIe link (already saturated at depth 2), but the cleaner per-slot design removes the fragile shared-event serialization and lets deeper prefetch help on faster buses.		2026-06-07 16:07:13 -06:00
..
__init__.py	git status	2025-10-01 14:12:17 -06:00
manager.py	Fix text encoder offload bug when caching embeddings (#868)	2026-06-07 14:09:32 -06:00
manager_modules.py	Deepen offload prefetch pipeline with per-slot events	2026-06-07 16:07:13 -06:00
test_memory_manager.py	Deepen offload prefetch pipeline with per-slot events	2026-06-07 16:07:13 -06:00