Self-host DeepSeek-V4-Flash-0731: GPU requirements and deployment

Self-host the deepseek-ai/DeepSeek-V4-Flash-0731 checkpoint with iwant on GCP, or configure vLLM or SGLang on an existing GPU host. The 0731 release, earlier Flash weights, vision checkpoint, and NVFP4 conversions are distinct deployment choices; this walkthrough keeps the 0731 model ID.

GPU and storage requirements

iwant requests eight H100s, at least 16 vCPUs, at least 64 GB of host RAM, and a 300 GB disk; its configuration estimates about 155 GB of checkpoint data and uses attached local storage when available. The direct vLLM path below also uses eight H100s. The SGLang path uses four H200s, following its documented high-throughput layout. These counts describe particular deployments, rather than a minimum GPU count for every precision and workload.

For the raw-runtime layouts and checkpoint variants, consult the linked vLLM and SGLang recipes below. For iwant, preview the selected cloud resources with iwant up deepseek-v4-flash --infra gcp --dry-run --yes before spending money.

Deploy with iwant

Complete iwant installation and cloud authentication once. Your GCP project needs billing enabled, GPU quota, and available capacity. Then launch by model name:

iwant up deepseek-v4-flash

The CLI guides you through the launch choices and selects the serving configuration automatically. Choose a real launch rather than the dry-run preview when prompted. For a scripted on-demand launch, use iwant up deepseek-v4-flash --infra gcp --yes instead. You do not need to choose an image or assemble engine flags.

Save the printed Server URL and API key after startup. Readiness means the models API responded; use the first request below to check inference. GCP is currently supported. For download authentication or provisioning problems, use the shared setup and troubleshooting guide.

Serve with vLLM

Complete vLLM setup on the eight-H100 host. The official DeepSeek-V4 recipe  covers checkpoint variants, GPU-specific flags, and DSpark. This command uses the 0731 checkpoint with FP8 KV cache, expert parallelism, and DeepSeek V4 parsers. Check the upstream prerequisites for the vLLM/FlashInfer combination you install.

Replace YOUR_SERVER_KEY with your chosen server key before starting the process. Run this on the GPU host, with any required HF_TOKEN set in that shell.

export MODEL_API_KEY="YOUR_SERVER_KEY"
 
vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
      --host 127.0.0.1 \
      --port 8000 \
      --api-key "$MODEL_API_KEY" \
      --trust-remote-code \
      --gpu-memory-utilization 0.85 \
      --kv-cache-dtype fp8 \
      --block-size 256 \
      --enable-expert-parallel \
      --tensor-parallel-size 8 \
      --moe-backend auto \
      --attention_config.indexer_kv_dtype fp8 \
      --tokenizer-mode deepseek_v4 \
      --tool-call-parser deepseek_v4 \
      --enable-auto-tool-choice \
      --reasoning-parser deepseek_v4 \
      --reasoning-config '{"reasoning_parser":"deepseek_v4","reasoning_start_str":"","reasoning_end_str":""}' \
      --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}'

The example binds to loopback. Test from another terminal on the same host, or use an SSH tunnel to reach its port. Stop the foreground process with Ctrl+C when finished; removing a rented host is a separate cloud action.

Serve with SGLang

The official SGLang DeepSeek-V4 recipe  includes a four-H200, Flash Official (0731), high-throughput configuration. It uses the Marlin MoE backend without speculative decoding in this selected path. Use Docker Engine, NVIDIA Container Toolkit, a compatible driver, and enough checkpoint storage on that GPU host.

Use your chosen server key and set HF_TOKEN if the download requires authentication. This is an alternative server: stop any earlier process using port 8000 before starting it.

export MODEL_API_KEY="YOUR_SERVER_KEY"
mkdir -p "$PWD/hf-cache"
 
docker run --rm --gpus all --ipc=host --shm-size 32g \
  -p 127.0.0.1:8000:8000 \
  -v "$PWD/hf-cache:/root/.cache/huggingface" \
  -e HF_TOKEN \
  lmsysorg/sglang:latest \
  sglang serve \
  --trust-remote-code \
  --model-path deepseek-ai/DeepSeek-V4-Flash-0731 \
  --tp 4 \
  --moe-runner-backend marlin \
  --host 0.0.0.0 --port 8000 \
  --api-key "$MODEL_API_KEY"

Wait for startup, then run the shared API check from the same host or through an SSH tunnel. Ctrl+C stops this foreground server; it does not remove the underlying VM.

Verify the model API

After any launch path above, confirm the server with OpenAI-compatible HTTP. Set OPENAI_BASE_URL to the base URL including /v1 (iwant prints this; vLLM and SGLang examples use http://127.0.0.1:8000/v1; Ollama uses port 11434; LocalAI uses 8080). Set OPENAI_API_KEY to the key from iwant or the value you chose when starting the server. Run curls on the GPU host or through SSH when the process binds to loopback.

export OPENAI_BASE_URL="YOUR_SERVER_URL_WITH_V1"
export OPENAI_API_KEY="YOUR_SERVER_API_KEY"

curl -fsS "$OPENAI_BASE_URL/models" \
  -H "Authorization: Bearer $OPENAI_API_KEY"

Copy the served id from the response into SERVED_MODEL_ID.

iwant typically returns deepseek-ai/DeepSeek-V4-Flash-0731. Model aliases can differ even when the underlying weights are the same.

export SERVED_MODEL_ID="COPY_THE_ID_FROM_MODELS"

curl -fsS "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$SERVED_MODEL_ID\",\"messages\":[{\"role\":\"user\",\"content\":\"What is 2 + 2? Answer briefly.\"}],\"max_tokens\":256}"

Expect a JSON chat completion with choices and token usage. Inspect choices[0].message for the answer and any separate reasoning output. A finish_reason of length means the output cap was reached. After a basic response works, validate the context lengths, tools, and concurrency your application needs.

Troubleshoot the deployment

If the endpoint serves another Flash checkpoint, check the launch command and /models before changing your app. The 0731, preview, vision, and converted variants are not interchangeable. If tool calls appear as raw markup, compare the runtime’s parser settings with its own upstream recipe; vLLM and SGLang do not necessarily use identical parser flags.

If weights do not download, check repository access and token forwarding for the runtime you selected. If startup stalls, read the server logs and check storage and GPU memory before changing parallelism. For iwant provisioning or interrupted launches, follow the shared troubleshooting guide and check existing resources before launching another node.

Stop serving and remove cloud resources

Stopping a foreground server (Ctrl+C, ollama stop, or ending a container) does not delete a cloud VM. For iwant, remove the cluster with billing and teardown (iwant list, iwant down EXACT_CLUSTER_NAME). For vLLM, SGLang, or Docker you started on a rented host, stop the process and remove the host through your cloud workflow. The self-hosting hub summarizes idle-timer behavior and interrupted launches. Log token usage and GPU runtime when comparing self-hosted costs.