Self-host Step-3.7-Flash: GPU requirements and deployment

Self-host the BF16 stepfun-ai/Step-3.7-Flash checkpoint with iwant, vLLM, or SGLang. iwant takes the model name and launches an eight-H200 endpoint. Direct engine paths expose the precision, parallelism, and parser choices if you need to configure them yourself.

GPU and storage requirements

iwant requests eight H200s, at least 16 vCPUs, at least 64 GB of host RAM, and a 150 GB boot disk. The BF16 weights are estimated at roughly 376 GB and need separate cache storage. The commands below also select eight-H200 configurations. FP8 and NVFP4 are separate checkpoints with different hardware constraints; they are not client-side switches on the BF16 server.

For the raw-runtime layouts and checkpoint variants, consult the linked vLLM and SGLang recipes below. For iwant, preview the selected cloud resources with iwant up step-3.7-flash --infra gcp --dry-run --yes before spending money.

Deploy with iwant

Complete iwant installation and cloud authentication once. Your GCP project needs billing enabled, GPU quota, and available capacity. Then launch by model name:

iwant up step-3.7-flash

The CLI guides you through the launch choices and selects the serving configuration automatically. Choose a real launch rather than the dry-run preview when prompted. For a scripted on-demand launch, use iwant up step-3.7-flash --infra gcp --yes instead. You do not need to choose an image or assemble engine flags.

Save the printed Server URL and API key after startup. Readiness means the models API responded; use the first request below to check inference. GCP is currently supported. For download authentication or provisioning problems, use the shared setup and troubleshooting guide.

Serve with vLLM

The official vLLM Step recipe  supplies the dedicated image and BF16 TP8 configuration. Use eight H200s and a Docker host with NVIDIA Container Toolkit. The served API alias is step3p7-flash; the step3p5 parser names are intentional. Cascade attention is disabled for this model’s hybrid attention layout.

Replace YOUR_SERVER_KEY with your chosen server key before starting the process. Run this on the GPU host, with any required HF_TOKEN set in that shell.

export MODEL_API_KEY="YOUR_SERVER_KEY"
mkdir -p "$PWD/hf-cache"
 
docker run --rm --gpus all --ipc=host --shm-size 32g \
  -p 127.0.0.1:8000:8000 \
  -v "$PWD/hf-cache:/root/.cache/huggingface" \
  -e HF_TOKEN \
  vllm/vllm-openai:stepfun37 \
  stepfun-ai/Step-3.7-Flash \
  --served-model-name step3p7-flash \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --disable-cascade-attn \
  --reasoning-parser step3p5 \
  --tool-call-parser step3p5 \
  --enable-auto-tool-choice \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --trust-remote-code \
  --host 0.0.0.0 --port 8000 \
  --api-key "$MODEL_API_KEY"

The example binds to loopback. Test from another terminal on the same host, or use an SSH tunnel to reach its port. Stop the foreground process with Ctrl+C when finished; removing a rented host is a separate cloud action.

Serve with SGLang

The official SGLang Step cookbook  documents its Docker installation and a Hopper BF16 selection. The command generator uses TP8 for that selection; the cookbook also discusses smaller and other precision layouts. This walkthrough sets an API alias explicitly so it can share client settings with the vLLM path.

Use your chosen server key and set HF_TOKEN if the download requires authentication. This is an alternative server: stop any earlier process using port 8000 before starting it.

export MODEL_API_KEY="YOUR_SERVER_KEY"
mkdir -p "$PWD/hf-cache"
 
docker run --rm --gpus all --ipc=host --shm-size 32g \
  -p 127.0.0.1:8000:8000 \
  -v "$PWD/hf-cache:/root/.cache/huggingface" \
  -e HF_TOKEN \
  lmsysorg/sglang:latest \
  sglang serve \
  --model-path stepfun-ai/Step-3.7-Flash \
  --served-model-name step3p7-flash \
  --tp 8 \
  --trust-remote-code \
  --reasoning-parser step3p5 \
  --tool-call-parser step3p5 \
  --host 0.0.0.0 --port 8000 \
  --api-key "$MODEL_API_KEY"

Wait for startup, then run the shared API check from the same host or through an SSH tunnel. Ctrl+C stops this foreground server; it does not remove the underlying VM.

Verify the model API

After any launch path above, confirm the server with OpenAI-compatible HTTP. Set OPENAI_BASE_URL to the base URL including /v1 (iwant prints this; vLLM and SGLang examples use http://127.0.0.1:8000/v1; Ollama uses port 11434; LocalAI uses 8080). Set OPENAI_API_KEY to the key from iwant or the value you chose when starting the server. Run curls on the GPU host or through SSH when the process binds to loopback.

export OPENAI_BASE_URL="YOUR_SERVER_URL_WITH_V1"
export OPENAI_API_KEY="YOUR_SERVER_API_KEY"

curl -fsS "$OPENAI_BASE_URL/models" \
  -H "Authorization: Bearer $OPENAI_API_KEY"

Copy the served id from the response into SERVED_MODEL_ID.

iwant typically returns step3p7-flash. Model aliases can differ even when the underlying weights are the same.

export SERVED_MODEL_ID="COPY_THE_ID_FROM_MODELS"

curl -fsS "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$SERVED_MODEL_ID\",\"messages\":[{\"role\":\"user\",\"content\":\"What is 2 + 2? Answer briefly.\"}],\"max_tokens\":256}"

Expect a JSON chat completion with choices and token usage. Inspect choices[0].message for the answer and any separate reasoning output. A finish_reason of length means the output cap was reached. After a basic response works, validate the context lengths, tools, and concurrency your application needs.

Troubleshoot the deployment

If the client cannot find the model, check /models. The iwant and direct examples use step3p7-flash, while the downloaded checkpoint is stepfun-ai/Step-3.7-Flash. For a customized vLLM launch, preserve --disable-cascade-attn and the documented parser names before tuning other flags.

If weights do not download, check repository access and token forwarding for the runtime you selected. If startup stalls, read the server logs and check storage and GPU memory before changing parallelism. For iwant provisioning or interrupted launches, follow the shared troubleshooting guide and check existing resources before launching another node.

Stop serving and remove cloud resources

Stopping a foreground server (Ctrl+C, ollama stop, or ending a container) does not delete a cloud VM. For iwant, remove the cluster with billing and teardown (iwant list, iwant down EXACT_CLUSTER_NAME). For vLLM, SGLang, or Docker you started on a rented host, stop the process and remove the host through your cloud workflow. The self-hosting hub summarizes idle-timer behavior and interrupted launches. Log token usage and GPU runtime when comparing self-hosted costs.