Self-host MiMo-V2.6-Flash-RL: GPU requirements and deployment

Deploy XiaomiMiMo/MiMo-V2.6-Flash-RL through iwant or serve the checkpoint directly with vLLM or SGLang. The model stores MoE weights in mxfp4 format, so the serving engine must understand that representation. Keep the RL checkpoint distinct from the newer MOPD release when configuring the server and testing agent behavior.

GPU and storage requirements

iwant requests eight H200s, at least 16 vCPUs, at least 64 GB of host RAM, and a 150 GB boot disk. The roughly 173 GB checkpoint needs a separate weight cache. The vLLM and SGLang H200 paths below use four GPUs. A model’s advertised 1M-token context is not evidence that your chosen cache and concurrency settings can accommodate every million-token request.

For the raw-runtime layouts and checkpoint variants, consult the linked vLLM and SGLang recipes below. For iwant, preview the selected cloud resources with iwant up mimo-v2.6-flash --infra gcp --dry-run --yes before spending money.

Deploy with iwant

Complete iwant installation and cloud authentication once. Your GCP project needs billing enabled, GPU quota, and available capacity. Then launch by model name:

iwant up mimo-v2.6-flash

The CLI guides you through the launch choices and selects the serving configuration automatically. Choose a real launch rather than the dry-run preview when prompted. For a scripted on-demand launch, use iwant up mimo-v2.6-flash --infra gcp --yes instead. You do not need to choose an image or assemble engine flags.

Save the printed Server URL and API key after startup. Readiness means the models API responded; use the first request below to check inference. GCP is currently supported. For download authentication or provisioning problems, use the shared setup and troubleshooting guide.

Serve with vLLM

The official vLLM MiMo recipe  documents stable v0.31.0 for NVIDIA mxfp4 loading and a TP4 H200 path. Its default model recommendation is MOPD; the command below deliberately selects the original RL checkpoint. Use four H200s, compatible Docker/NVIDIA setup, and space for the weight cache.

Replace YOUR_SERVER_KEY with your chosen server key before starting the process. Run this on the GPU host, with any required HF_TOKEN set in that shell.

export MODEL_API_KEY="YOUR_SERVER_KEY"
mkdir -p "$PWD/hf-cache"
 
docker run --rm --gpus all --ipc=host --shm-size 32g \
  -p 127.0.0.1:8000:8000 \
  -v "$PWD/hf-cache:/root/.cache/huggingface" \
  -e HF_TOKEN \
  vllm/vllm-openai:v0.31.0 \
  XiaomiMiMo/MiMo-V2.6-Flash-RL \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --max-model-len auto \
  --generation-config vllm \
  --tool-call-parser mimo \
  --enable-auto-tool-choice \
  --reasoning-parser mimo \
  --host 0.0.0.0 --port 8000 \
  --api-key "$MODEL_API_KEY"

The example binds to loopback. Test from another terminal on the same host, or use an SSH tunnel to reach its port. Stop the foreground process with Ctrl+C when finished; removing a rented host is a separate cloud action.

Serve with SGLang

The official SGLang MiMo-V2.6 cookbook  accepts a local Flash checkpoint directory and documents a four-H200 Marlin path. Its architecture references include the RL checkpoint. Install the Hugging Face CLI , download the exact RL weights on the GPU host, and mount that directory into the serving container. The API name below is explicitly set to mimo-v2.6-flash.

Use your chosen server key and set HF_TOKEN if the download requires authentication. This is an alternative server: stop any earlier process using port 8000 before starting it.

export MODEL_API_KEY="YOUR_SERVER_KEY"
export HF_MODEL_DIR="$PWD/MiMo-V2.6-Flash-RL"
hf download XiaomiMiMo/MiMo-V2.6-Flash-RL --local-dir "$HF_MODEL_DIR"
 
docker run --rm --gpus all --ipc=host --shm-size 32g \
  -p 127.0.0.1:8000:8000 \
  -v "$HF_MODEL_DIR:/model/MiMo-V2.6-Flash-RL:ro" \
  -e HF_TOKEN \
  lmsysorg/sglang:dev \
  sglang serve \
  --model-path /model/MiMo-V2.6-Flash-RL \
  --served-model-name mimo-v2.6-flash \
  --tp 4 \
  --moe-runner-backend marlin \
  --trust-remote-code \
  --reasoning-parser mimo \
  --tool-call-parser mimo \
  --host 0.0.0.0 --port 8000 \
  --api-key "$MODEL_API_KEY"

Wait for startup, then run the shared API check from the same host or through an SSH tunnel. Ctrl+C stops this foreground server; it does not remove the underlying VM.

Verify the model API

After any launch path above, confirm the server with OpenAI-compatible HTTP. Set OPENAI_BASE_URL to the base URL including /v1 (iwant prints this; vLLM and SGLang examples use http://127.0.0.1:8000/v1; Ollama uses port 11434; LocalAI uses 8080). Set OPENAI_API_KEY to the key from iwant or the value you chose when starting the server. Run curls on the GPU host or through SSH when the process binds to loopback.

export OPENAI_BASE_URL="YOUR_SERVER_URL_WITH_V1"
export OPENAI_API_KEY="YOUR_SERVER_API_KEY"

curl -fsS "$OPENAI_BASE_URL/models" \
  -H "Authorization: Bearer $OPENAI_API_KEY"

Copy the served id from the response into SERVED_MODEL_ID.

iwant typically returns XiaomiMiMo/MiMo-V2.6-Flash-RL; the SGLang command above may register mimo-v2.6-flash. Model aliases can differ even when the underlying weights are the same.

export SERVED_MODEL_ID="COPY_THE_ID_FROM_MODELS"

curl -fsS "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$SERVED_MODEL_ID\",\"messages\":[{\"role\":\"user\",\"content\":\"What is 2 + 2? Answer briefly.\"}],\"max_tokens\":256}"

Expect a JSON chat completion with choices and token usage. Inspect choices[0].message for the answer and any separate reasoning output. A finish_reason of length means the output cap was reached. After a basic response works, validate the context lengths, tools, and concurrency your application needs.

Troubleshoot the deployment

If loading fails after an engine change, check mxfp4 support for that exact release. If an agent repeatedly emits the same tool calls, consult the official vLLM guide’s RL-versus-MOPD discussion. Selecting MOPD in the client does not replace RL weights already loaded by the server.

If weights do not download, check repository access and token forwarding for the runtime you selected. If startup stalls, read the server logs and check storage and GPU memory before changing parallelism. For iwant provisioning or interrupted launches, follow the shared troubleshooting guide and check existing resources before launching another node.

Stop serving and remove cloud resources

Stopping a foreground server (Ctrl+C, ollama stop, or ending a container) does not delete a cloud VM. For iwant, remove the cluster with billing and teardown (iwant list, iwant down EXACT_CLUSTER_NAME). For vLLM, SGLang, or Docker you started on a rented host, stop the process and remove the host through your cloud workflow. The self-hosting hub summarizes idle-timer behavior and interrupted launches. Log token usage and GPU runtime when comparing self-hosted costs.