GPU & VRAM Management

360 Hextile runs AI diffusion models directly on your NVIDIA GPU. The Loaded Models panel gives you full visibility into what is occupying VRAM and fine-grained control over memory usage.

Loaded Models Panel

The Loaded Models panel, found in Settings under GPU Information, shows every model currently loaded in VRAM with per-model detail:

• Model name and display name — the processor identifier and its human-readable label
• VRAM footprint — tracked in GB where the model's memory allocation is measurable; untracked models show a category tag (offloaded for CPU-offloaded weights, ncnn — untracked for external engines like Real-ESRGAN)
• Active indicator — highlights the currently active diffusion processor (the one that handles renders)
• Unload button — evict an individual model to free its VRAM. The active model uses a render-safe eviction path: it first stops any in-progress render, then unloads, preventing mid-operation crashes

The panel polls VRAM status every second, so changes appear in near real-time.

Total VRAM Bar

A horizontal bar at the top shows device VRAM usage as a percentage of total capacity. Below the bar:

• Load-time model attribution — the sum of per-model footprints measured when each model loaded. It is not live Torch usage.

Allocator pools

A separate Allocator pools section sits between Loaded models and Preview caches. It is not a model list.

• Torch unused — live reserved − allocated: unused blocks in PyTorch’s CUDA cache
• CuPy pool — used vs held in the CuPy allocator
• Release unused — returns unused Torch and CuPy blocks. Loaded models stay loaded. The button is disabled while a render is running or another GPU job holds the card.

After Release, reserved should drop. On Windows, the device-used number on the gauge may stay high (the driver often keeps pages). The next tile will grow the pool again. That is expected.

Bulk Eviction

Use Unload All to free all VRAM at once. The button evicts every loaded model:

1. All inactive (non-rendering) models are unloaded directly
2. If the active diffusion model has a render-safe path, it is handed off for stop-then-unload
3. If no render-safe path is available (or no active model), the legacy unload-all runs directly

Bulk eviction is useful when switching between GPU-intensive workflows outside 360 Hextile, or when you need a clean VRAM slate for loading a different pipeline.

VRAM Tracking

360 Hextile classifies VRAM usage into three categories:

Live Torch allocated/reserved and CuPy pool sizes appear in Allocator pools, not on the model rows. The model list is named holders you can Unload. Pools are unused allocator blocks you can Release.

NVML Reconciliation

360 Hextile cross-references its own VRAM tracking with NVIDIA's Management Library (NVML) for accurate readings that PyTorch alone cannot provide:

• PyTorch (torch.cuda.memory_allocated) only sees its own allocations — it misses CuPy memory pools, NCNN upscaler allocations, and other processes
• NVML (nvidia-smi equivalent) gives the absolute hardware-level view: total, used, and free VRAM across all consumers
• The reconciliation runs on a dedicated single-worker thread pool to avoid driver contention during concurrent queries

This dual-source approach prevents the "missing VRAM" problem where nvidia-smi shows high usage but PyTorch reports little — the gap is often CuPy memory pools or NCNN allocations that NVML captures.

Quantization Policy

360 Hextile supports multiple quantization tiers for compatible pipelines. Quantization trades precision for VRAM: lower precision means less VRAM but potential quality loss.

Supported Quantizations by Pipeline

How the Policy Selects Quantization

The quantization policy runs a two-stage decision:

1. Quantization selection — walks the ladder from highest quality (BF16) to lowest (GGUF Q4 / NF4), stopping at the first rung whose transformer footprint fits in your effective budget AND whose capability gate is satisfied by your GPU
2. Residency decision — checks whether the full model set (transformer + text encoder + VAE) fits in VRAM. If yes, the model stays resident for maximum speed. If no, it loads with CPU offload — same quantization, slower inference

Capability gates: FP8 requires compute capability >= 8.9 (RTX 40-series+). BF16 requires >= 8.0 (RTX 30-series+). No gate means the quant runs on any CUDA-capable card.

Performance Level Slider

The Performance Level slider, available in Settings, sets a global VRAM budget cap (in GB) that limits how much memory the quantization policy can plan against. It operates as a spectrum from Max Quality to Max Speed:

• Max Quality (right) — the slider is at your total VRAM (or "AUTO"). The policy picks the highest-quality quantization that fits. On a 24GB card, Qwen-Image loads as BF16 resident if possible; on a 12GB card, it picks the best-fitting quantized rung
• Max Speed (left, floor at 6GB) — the slider is pulled down to the minimum budget. The policy is forced toward aggressive quantization (GGUF Q4, NF4) regardless of actual VRAM headroom. Less VRAM usage, faster off-the-shelf loads, but typically lower quality
• Mid-range — the slider caps the budget at N GB. If your card has 24GB but you set the slider to 16GB, the policy plans against 16GB — it may select NF4 instead of BF16 even though the card could fit BF16

The slider is a budget cap, not a hard limit. It controls the quantization policy's decision, not an actual VRAM allocation clamp.

Automatic Eviction

360 Hextile automatically manages VRAM to prevent out-of-memory errors during render operations:

• Single diffusion model — only one diffusion processor is loaded at a time. Switching pipelines auto-unloads the previous model before loading the new one
• VRAM admission checks — before loading a model, the system verifies that enough VRAM is free (expected footprint + 1.5GB safety margin). If insufficient, it triggers cleanup (garbage collection, CuPy pool release, PyTorch cache flush) and retries
• Vision model strategy — the optional vision/describe model has three VRAM strategies: auto (smart load/unload based on idle time and VRAM pressure), resident (keep loaded at all times), and on_demand (load per request, unload immediately after)
• Cleanup cycle — after each render completes, 360 Hextile runs a deep cleanup cycle: Python garbage collection, CuPy memory pool release, and torch.cuda.empty_cache() to return cached memory to the OS

If VRAM is full and a new model needs to load, 360 Hextile will not evict models automatically from the Loaded Models panel — you must unload them manually or use Unload All. The system prioritizes safe, explicit memory management over surprise eviction.

VRAM Requirements by Pipeline

See also: - Settings — GPU Information card and configuration - Getting Started — System requirements and GPU detection - Pipelines — Model management and pipeline selection