Section 01 · Host Runtime & Process Containment
1. Underlying Systems & Compute Architecture
Repetere AI 1.0 is engineered as an out-of-process orchestrator written in native C++ (Dear ImGui, GLFW, OpenGL 3), operating completely free of Electron, Node.js, Chromium, or Python runtimes.
- Process Isolation via Win32 Job Objects: Instead of linking inference libraries into the UI thread or running a shared Python kernel, all backends (
llama-server.exe,audiocpp_cli.exe,sd.exe) are spawned as isolated child processes via Win32CreateProcessW. Every process is bound to a Win32 Job Object withJOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE. If the main executable faults or exits, the OS hardware scheduler guarantees every sidecar process drops instantly, leaving no orphaned server daemons consuming ports or VRAM. - Dynamic Compute Backends: The inference engine hot-swaps between two low-level acceleration runtimes: NVIDIA CUDA 12 (cuBLAS) for Tensor Core pipelines and Vulkan 1.3 (SPIR-V / ggml-vulkan) for hardware-neutral execution across AMD Radeon, Intel Arc, and unified memory architectures (APUs).
-
Dynamic Layer Partitioning (
-nglOffloading):- In GPU Mode, weights and context are pinned strictly to VRAM via
-ngl 99, minimizing latency and maximizing memory bandwidth. - In Hybrid Mode,
HardwareScanner.hleverages DirectX Graphics Infrastructure (DXGI) to poll dedicated video memory against total host RAM. The orchestrator calculates the exact layer budget the GPU can safely hold, streaming the overflow layers into system RAM. This architecture enables 14B and 32B checkpoints to run on mainstream 8 GB–12 GB GPUs without throwing CUDA Out-of-Memory (cudaErrorMemoryAllocation) faults.
- In GPU Mode, weights and context are pinned strictly to VRAM via
- Modality Hot-Swapping & VRAM Garbage Collection: Because consumer GPUs cannot concurrently hold a quantized 8B LLM, a 3B autoregressive audio model, and a 12B parameter rectified flow diffusion pipeline, navigating between workspaces executes a deterministic unload pipeline. The host sends an ejection command to the running sidecar, drops memory-mapped buffers (
UnloadFromMemory), flushes OpenGL display textures, and then bootstraps the next engine cleanly.