Self-host DeepSeek-V4.1-Flash: GPU requirements and deployment

Deploy deepseek-ai/DeepSeek-V4.1-Flash with iwant, vLLM, or SGLang. iwant’s cloud launch serves text and handles the host-memory offload configuration automatically. Direct runtime paths let you select a GPU topology and change text-only versus multimodal serving.

GPU and storage requirements

iwant requests eight H200s, at least 32 vCPUs, and at least 256 GB of host RAM. Its 150 GB boot disk covers the OS and container; the roughly 511 GB checkpoint needs separate local storage. Engram CPU offload makes host RAM part of the capacity requirement. The direct vLLM example uses the upstream H200 TP4 configuration with CPU offload; SGLang uses eight H200s. Provision at least 256 GB of host RAM and enough cache storage before either direct launch.

For the raw-runtime layouts and checkpoint variants, consult the linked vLLM and SGLang recipes below. For iwant, preview the selected cloud resources with iwant up deepseek-v4.1-flash --infra gcp --dry-run --yes before spending money.

Deploy with iwant

Complete iwant installation and cloud authentication once. Your GCP project needs billing enabled, GPU quota, and available capacity. Then launch by model name:

iwant up deepseek-v4.1-flash

The CLI guides you through the launch choices and selects the serving configuration automatically. Choose a real launch rather than the dry-run preview when prompted. For a scripted on-demand launch, use iwant up deepseek-v4.1-flash --infra gcp --yes instead. You do not need to choose an image or assemble engine flags.

Save the printed Server URL and API key after startup. Readiness means the models API responded; use the first request below to check inference. GCP is currently supported. For download authentication or provisioning problems, use the shared setup and troubleshooting guide.

Serve with vLLM

The official vLLM DeepSeek-V4.1 recipe  uses the nightly container for this architecture and documents H200 TP4 with Engram offload. Use a host with four H200s available to the server, Docker Engine, NVIDIA Container Toolkit, and the storage/RAM described above. This example serves text only; omit --language-model-only only after reviewing the upstream vision configuration and memory requirements.

Replace YOUR_SERVER_KEY with your chosen server key before starting the process. Run this on the GPU host, with any required HF_TOKEN set in that shell.

export MODEL_API_KEY="YOUR_SERVER_KEY"
mkdir -p "$PWD/hf-cache"
 
docker run --rm --gpus all --ipc=host --shm-size 32g \
  -p 127.0.0.1:8000:8000 \
  -v "$PWD/hf-cache:/root/.cache/huggingface" \
  -e HF_TOKEN \
  -e VLLM_ENGINE_READY_TIMEOUT_S=3600 \
  vllm/vllm-openai:nightly \
  deepseek-ai/DeepSeek-V4.1-Flash \
  --tensor-parallel-size 4 \
  --tokenizer-mode deepseek_v41 \
  --language-model-only \
  --engram-config '{"cpu_offload":true}' \
  --tool-call-parser deepseek_v41 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v41 \
  --host 0.0.0.0 --port 8000 \
  --api-key "$MODEL_API_KEY"

The example binds to loopback. Test from another terminal on the same host, or use an SSH tunnel to reach its port. Stop the foreground process with Ctrl+C when finished; removing a rented host is a separate cloud action.

Serve with SGLang

The official SGLang DeepSeek-V4.1 cookbook  supplies the eight-H200 configuration below and its dedicated container. It uses the DSV4 attention backend, eight-way expert parallelism, and automatic parsers. This is a separate engine setup from the vLLM CPU-offload command; follow the SGLang cookbook for its memory and multimodal behavior.

Use your chosen server key and set HF_TOKEN if the download requires authentication. This is an alternative server: stop any earlier process using port 8000 before starting it.

export MODEL_API_KEY="YOUR_SERVER_KEY"
mkdir -p "$PWD/hf-cache"
 
docker run --rm --gpus all --ipc=host --shm-size 32g \
  -p 127.0.0.1:8000:8000 \
  -v "$PWD/hf-cache:/root/.cache/huggingface" \
  -e HF_TOKEN \
  lmsysorg/sglang:dev-dsv41 \
  sglang serve \
  --trust-remote-code \
  --model-path deepseek-ai/DeepSeek-V4.1-Flash \
  --tp 8 \
  --ep-size 8 \
  --mem-fraction-static 0.8 \
  --attention-backend dsv4 \
  --moe-runner-backend flashinfer_mxfp4 \
  --enable-decoder-swa-bounded-replay \
  --cuda-graph-max-bs-decode 64 \
  --reasoning-parser auto \
  --tool-call-parser auto \
  --host 0.0.0.0 --port 8000 \
  --api-key "$MODEL_API_KEY"

Wait for startup, then run the shared API check from the same host or through an SSH tunnel. Ctrl+C stops this foreground server; it does not remove the underlying VM.

Verify the model API

After any launch path above, confirm the server with OpenAI-compatible HTTP. Set OPENAI_BASE_URL to the base URL including /v1 (iwant prints this; vLLM and SGLang examples use http://127.0.0.1:8000/v1; Ollama uses port 11434; LocalAI uses 8080). Set OPENAI_API_KEY to the key from iwant or the value you chose when starting the server. Run curls on the GPU host or through SSH when the process binds to loopback.

export OPENAI_BASE_URL="YOUR_SERVER_URL_WITH_V1"
export OPENAI_API_KEY="YOUR_SERVER_API_KEY"

curl -fsS "$OPENAI_BASE_URL/models" \
  -H "Authorization: Bearer $OPENAI_API_KEY"

Copy the served id from the response into SERVED_MODEL_ID.

iwant typically returns deepseek-ai/DeepSeek-V4.1-Flash. Model aliases can differ even when the underlying weights are the same.

export SERVED_MODEL_ID="COPY_THE_ID_FROM_MODELS"

curl -fsS "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$SERVED_MODEL_ID\",\"messages\":[{\"role\":\"user\",\"content\":\"What is 2 + 2? Answer briefly.\"}],\"max_tokens\":256}"

Expect a JSON chat completion with choices and token usage. Inspect choices[0].message for the answer and any separate reasoning output. A finish_reason of length means the output cap was reached. After a basic response works, validate the context lengths, tools, and concurrency your application needs.

Troubleshoot the deployment

An image request cannot enable vision on iwant’s text-only deployment or the text-only vLLM command shown here. Use the runtime’s multimodal configuration instead. If startup cannot complete, check host RAM, the checkpoint-cache location, and download progress separately; the 150 GB boot disk is not a 511 GB weight cache.

If weights do not download, check repository access and token forwarding for the runtime you selected. If startup stalls, read the server logs and check storage and GPU memory before changing parallelism. For iwant provisioning or interrupted launches, follow the shared troubleshooting guide and check existing resources before launching another node.

Stop serving and remove cloud resources

Stopping a foreground server (Ctrl+C, ollama stop, or ending a container) does not delete a cloud VM. For iwant, remove the cluster with billing and teardown (iwant list, iwant down EXACT_CLUSTER_NAME). For vLLM, SGLang, or Docker you started on a rented host, stop the process and remove the host through your cloud workflow. The self-hosting hub summarizes idle-timer behavior and interrupted launches. Log token usage and GPU runtime when comparing self-hosted costs.