Original hardware-and-sizing review: July 19, 2026. Runtime-support update: August 26, 2026. Historical benchmark and release-state notes remain explicitly dated.
MiniMax M3 can run on your own infrastructure, but it is a data-center-scale model, not a laptop model. MiniMax describes M3 as a native multimodal mixture-of-experts model with approximately 428 billion total parameters, approximately 23 billion activated parameters per token, and a one-million-token context window. The official BF16 repository is about 854 GB before runtime memory, while the official MXFP8 checkpoint is about 440 GB.
The phrase “23B activated” describes how much of the MoE network participates in a token’s computation. It does not mean that only 23 billion parameters need to be downloaded or kept available. Every expert can be selected, so the complete checkpoint still has to reside in accelerator memory, host memory, or an explicitly supported offload tier.
This guide sizes the weights and long-context cache, separates RAM from VRAM, compares BF16 and primary-source quantized checkpoints, and provides conservative vLLM and SGLang deployment paths. It does not claim that we ran an eight-GPU benchmark; the validated hardware statements come from the official MiniMax model card and the deployment recipes maintained by the vLLM and SGLang projects.
MiniMax M3 local requirements: quick answer
| Question | Practical answer |
|---|---|
| Can MiniMax M3 run on a laptop? | No realistic primary-source deployment path treats the full M3 checkpoint as a laptop model. Even a theoretical 4-bit copy exceeds 198 GiB before metadata, unquantized layers, cache, and runtime overhead. |
| How large are the official BF16 weights? | About 854 GB decimal, or about 795 GiB, in the official Hugging Face repository. |
| How much aggregate VRAM does BF16 need? | More than the approximately 795 GiB weight floor. The official SGLang BF16 path uses tensor parallelism across 8×H200 GPUs; the vLLM recipe also characterizes an eight-GPU Hopper-class fit as tight. |
| What is the smaller MiniMax checkpoint? | MiniMaxAI/MiniMax-M3-MXFP8, approximately 440 GB according to SGLang’s M3 recipe. It still targets multi-GPU Blackwell or AMD Instinct servers. |
| Can the full 1M context fit automatically? | No. Context consumes a large KV cache and an additional sparse-index cache. Cap --max-model-len to the longest prompt plus output you actually serve. |
| Which engines have primary-source M3 recipes? | vLLM and SGLang. vLLM added M3 in v0.24.0 and still lists both text and multimodal architectures. SGLang v0.5.18 is the latest tagged release checked August 26 and includes MiniMax-M3 PR #34542, but its current M3 cookbook still contains a stale pre-release installation block pointing to closed reference PR #27944 and development images. Pin and test the exact release or image digest, checkpoint, accelerator backend, and flags. |
If you need M3 without owning a multi-GPU server, use the hosted route described in our MiniMax API guide and compare it with self-hosting through the MiniMax pricing guide.
What you are actually loading
The official MiniMax M3 model card reports approximately 428B total parameters, approximately 23B activated parameters, native text/image/video input, MiniMax Sparse Attention (MSA), and a 1M context window. Hugging Face’s file metadata displays 427B parameters; that one-billion difference is a reporting or rounding distinction, not a smaller downloadable edition.
| Property | Official configuration | Why it matters locally |
|---|---|---|
| Architecture | Multimodal MoE with MiniMax Sparse Attention | Only some experts compute each token, but all expert weights must remain accessible. |
| Total / activated parameters | ~428B total / ~23B activated | Activated parameters affect compute; total parameters dominate storage and weight memory. |
| Language-model layers | 60 | KV-cache use scales with the number of layers. |
| Experts | 128 local experts, 4 routed experts per token, plus a shared expert | MoE lowers per-token compute without shrinking the checkpoint to 23B. |
| Attention configuration | 64 query heads, 4 KV heads, head dimension 128 | Grouped-query attention reduces KV memory relative to 64 KV heads. |
| Configured context | 1,048,576 positions | The configured maximum is not a promise that every server has enough cache for it. |
| Base checkpoint dtype | BF16 with some F32 tensors/metadata | Two bytes per parameter is a useful weight-floor calculation, not the complete runtime requirement. |
These architecture values are visible in the repository’s official config.json. For a capability-focused description rather than deployment sizing, read our MiniMax M3 model guide; for an upgrade decision, use the MiniMax M3 vs M2.7 comparison.
Weight storage and VRAM calculations
Hardware vendors, operating systems, and model hosts do not always display units the same way. Model repositories generally show decimal gigabytes (GB), while Linux tools may report binary gibibytes (GiB). Use both when planning:
decimal GB = bytes / 1,000,000,000
binary GiB = bytes / 1,073,741,824
427,000,000,000 parameters × 2 bytes (BF16)
= 854 GB decimal
≈ 795.35 GiB binaryThe result matches the roughly 854 GB size displayed for the official BF16 repository. This is only the resident weight payload. A serving process also needs memory for KV cache, the MSA sparse-index state, temporary activations, CUDA graphs, communication buffers, the multimodal encoder, allocator fragmentation, and concurrent requests.
| Precision assumption | Theoretical weight floor | What to use for planning |
|---|---|---|
| BF16, 2 bytes/parameter | 854 GB / 795 GiB | The official repository is about 854 GB. Add runtime and context headroom. |
| 8-bit, 1 byte/parameter | 427 GB / 398 GiB | The official MXFP8 checkpoint is approximately 440 GB because real checkpoints include scales, metadata, and components not represented by the one-byte shortcut. |
| 4-bit, 0.5 byte/parameter | 213.5 GB / 199 GiB | Do not treat 199 GiB as an executable requirement. Primary NVFP4/MXFP4 checkpoints retain excluded or higher-precision tensors and target specific accelerators. |
How context changes the VRAM requirement
M3’s sparse attention reduces long-context computation, but “sparse” does not mean the cache is free. A useful transparent estimate starts with the main K/V state shown by the config:
main KV bytes
= tokens × 60 layers × 2 (K and V)
× 4 KV heads × 128 head dimension × bytes per value
At BF16: 122,880 bytes per token = 120 KiB per tokenThe config marks the first three language-model layers as non-sparse and the remaining 57 as sparse. Those sparse layers maintain a K-only index buffer. If that 4-head, 128-dimension index state is held in two-byte values, it contributes another 57 KiB per token. The table below is therefore a derived raw-cache estimate, not a guaranteed allocation reported by the runtime.
| Configured sequence length | Main BF16 K/V | Estimated BF16 sparse index | Combined raw state |
|---|---|---|---|
| 32,768 tokens | 3.75 GiB | 1.78 GiB | 5.53 GiB |
| 131,072 tokens | 15.00 GiB | 7.13 GiB | 22.13 GiB |
| 262,144 tokens | 30.00 GiB | 14.25 GiB | 44.25 GiB |
| 524,288 tokens | 60.00 GiB | 28.50 GiB | 88.50 GiB |
| 1,048,576 tokens | 120.00 GiB | 57.00 GiB | 177.00 GiB |
That state is distributed across tensor-parallel ranks. At TP=8, the simplified one-sequence 1M estimate is roughly 22.1 GiB per GPU before blocks, padding, workspace, activations, and concurrency. Add about 99.4 GiB of BF16 weights per rank and the simplified total is already about 121.5 GiB per GPU. This explains why an eight-H200 BF16 node is described as a tight fit, even though its aggregate capacity exceeds the raw weight size.
vLLM’s official recipe says --kv-cache-dtype fp8 can provide roughly 1.5× the KV pool for M3. It is not advertised as a 2× total-cache gain because the sparse-index state and other allocations do not all shrink with the main K/V tensors. Treat the runtime’s startup memory report and a workload test as the final authority.
RAM is not a substitute for VRAM
- Disk: stores the downloaded checkpoint. Plan for more than 854 GB for BF16 or more than 440 GB for MXFP8, plus container layers and cache. A 2 TB NVMe volume is a sensible BF16 staging target when you need room for temporary files or a second checkpoint.
- System RAM: supports the OS, containers, tokenizer, download process, staging, and any CPU-offloaded tensors. Neither primary framework recipe publishes one universal host-RAM minimum for an all-GPU deployment.
- VRAM / HBM: holds accelerator-resident weights, cache, activations, and runtime buffers. This is the binding resource in the validated serving paths.
If you try to hold the full BF16 checkpoint in CPU memory, the weight floor alone is about 795 GiB. After the OS, framework objects, buffers, and cache, a “1 TB RAM server” is a floor rather than generous headroom. CPU-only or heavy PCIe offload is not an official performance path in the M3 vLLM or SGLang recipes and can turn token generation into a bandwidth-bound workload. It should not be presented as a practical laptop solution.
Hardware paths documented by vLLM and SGLang
| Checkpoint / precision | Primary documented path | Status and constraint |
|---|---|---|
MiniMaxAI/MiniMax-M3 BF16 | 8×NVIDIA H200 with TP=8 | SGLang documents a full eight-GPU Hopper node. Reduce context for usable KV and activation headroom. |
MiniMaxAI/MiniMax-M3-MXFP8 | NVIDIA B200 at TP=8; B300/GB300 at TP=4; AMD Instinct paths at TP=8 | SGLang reports the checkpoint at about 440 GB and validates hardware-specific recipes. The vision tower remains unquantized. |
MiniMaxAI/MiniMax-M3-MXFP8 in vLLM | Blackwell or AMD Instinct, normally multi-GPU | vLLM’s recipe recommends native MX hardware for throughput and provides separate AMD launch settings. |
nvidia/MiniMax-M3-NVFP4 | 8×NVIDIA B200 with vLLM | NVIDIA says weights and activations are quantized to NVFP4 and reports about 2× lower disk/VRAM than its FP8 baseline. Its documented command still uses TP=8. Stable base-model support exists from v0.24.0, but quantized checkpoints remain hardware- and backend-specific; validate this exact NVFP4 checkpoint against the current vLLM recipe before provisioning. |
amd/MiniMax-M3-MXFP4 | 8×AMD MI355X with ROCm/vLLM | AMD documents static OCP MXFP4 weights, dynamic MXFP4 activations, and a patched/runtime-specific vLLM path. |
| CPU-only, one consumer GPU, or laptop | No validated primary-source M3 serving recipe | Do not infer support from the active-parameter count or from a community conversion’s filename. |
The detailed hardware flags and support state are maintained in the vLLM MiniMax M3 recipe and the SGLang MiniMax M3 cookbook. Recheck those pages before provisioning hardware or pinning a container digest.
Choose the checkpoint before choosing the server
BF16: reference weights, highest memory demand
Choose BF16 when you need the reference checkpoint, have an eight-H200-class node, and can accept the weight and context footprint. It is also the clean baseline for checking whether a quantized edition changes output quality for your workload.
MXFP8: official MiniMax quantized checkpoint
The MiniMax M3 MXFP8 repository is the most direct lower-memory checkpoint under MiniMax’s own namespace. SGLang reports approximately 440 GB and documents native paths for Blackwell and AMD CDNA4, plus conversion at load for selected CDNA3 hardware. MXFP8 lowers weight memory; it does not remove KV-cache, vision-encoder, or activation requirements.
NVFP4 and MXFP4: hardware-specific partner checkpoints
NVIDIA’s NVFP4 checkpoint targets Blackwell and uses NVIDIA ModelOpt. AMD’s MXFP4 checkpoint targets MI350/MI355 hardware and uses AMD Quark. These are primary vendor releases, but neither is a generic “4-bit file for any GPU.” Match the checkpoint, engine build, GPU architecture, and documented tensor-parallel degree.
Community GGUF, AWQ, GPTQ, or MLX conversions
Hugging Face may list community quantizations, but a conversion can change multimodal support, tool-call parsing, MSA execution, context limits, and output quality. Verify the converter, source commit, calibration data, tensor coverage, engine compatibility, checksum, and license before use. A community 4-bit checkpoint is not proof that the full M3 experience works on consumer hardware.
Read the MiniMax Community License before deployment
M3 is not released under MIT or Apache 2.0. The MiniMax Community License permits non-commercial use and applies conditions to commercial use. The license text requires commercial users to display “Built with MiniMax M3.” It also says a business whose relevant products or services generate more than US$20 million in yearly revenue must obtain separate prior written authorization; below that threshold, the text requires a one-time notice to MiniMax. Prohibited-use terms also apply.
Self-hosting does not remove these conditions. Review the complete license and obtain legal advice for a commercial deployment; this summary is not legal advice.
Preflight checklist
- Use Linux on hardware covered by the chosen framework recipe.
- Confirm every GPU and its free memory with
nvidia-smiorrocm-smi. - Use high-bandwidth GPU interconnects. TP=8 across slow PCIe-only links may fit but perform poorly.
- Reserve NVMe capacity for weights, container layers, and a second copy during upgrades or conversion.
- Start with a 32K or 128K context cap, one request, and a short output. Increase only after measuring peak memory.
- Keep the local OpenAI-compatible port bound to loopback until authentication, TLS, request limits, and logging policy are configured.
- Record the model revision, engine image digest, driver, CUDA/ROCm version, context cap, and quantization in every benchmark.
# NVIDIA inventory
nvidia-smi --query-gpu=index,name,memory.total,memory.free --format=csv
# Filesystem capacity
df -h /srv/models
# Docker GPU access
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smiDownload MiniMax M3 without duplicating the weights
Install the Hugging Face CLI, choose one checkpoint, and download to a fast local volume. Do not download BF16 and MXFP8 merely to compare filenames; together they require well over one terabyte.
python3 -m pip install -U "huggingface_hub[cli]"
# BF16: about 854 GB
hf download MiniMaxAI/MiniMax-M3 \
--local-dir /srv/models/MiniMax-M3
# OR MiniMax's MXFP8 checkpoint: about 440 GB
hf download MiniMaxAI/MiniMax-M3-MXFP8 \
--local-dir /srv/models/MiniMax-M3-MXFP8For reproducibility, pin a Hugging Face revision with --revision <commit-hash> after validating it. A floating main revision can change configuration or templates between deployments.
Run MiniMax M3 with vLLM
Runtime update — August 15, 2026: vLLM v0.24.0 added MiniMax-M3 in a tagged release. v0.27.1 is the latest tag and its model registry lists MiniMax-M3 and MiniMax-M3-MXFP8 in both the text and multimodal registries. v0.27.0 also added MiniMax-M3 MSA speculative-decode verification and a default video processor. The official vLLM recipe still carries inconsistent pre-stable wording, so verify support against the selected tag and registry. The development-image command below remains a July 19 historical snapshot; use only a pinned release or image digest validated on the exact checkpoint and hardware.
docker pull vllm/vllm-openai:minimax-m3
# Conservative BF16 starting point for an 8-GPU H200-class node.
# The host port is loopback-only.
docker run --rm --gpus all --ipc=host \
-p 127.0.0.1:8000:8000 \
-v /srv/models:/models:ro \
vllm/vllm-openai:minimax-m3 \
--model /models/MiniMax-M3 \
--served-model-name MiniMaxAI/MiniMax-M3 \
--tensor-parallel-size 8 \
--block-size 128 \
--max-model-len 131072 \
--tool-call-parser minimax_m3 \
--reasoning-parser minimax_m3 \
--enable-auto-tool-choice--block-size 128 is mandatory in the vLLM M3 recipe because MSA indexes 128-token blocks. The 128K context cap is intentional: it leaves more room for runtime buffers and concurrency than loading the configured 1M maximum. For text-only serving, the recipe also supports --language-model-only to skip the vision encoder. Add --kv-cache-dtype fp8 only after checking output quality and the exact GPU path.
For MXFP8, change the model ID to MiniMaxAI/MiniMax-M3-MXFP8 and use a Blackwell or AMD command from the recipe. Do not copy NVIDIA backend flags to AMD or mix an NVFP4 checkpoint with an MXFP8 command.
Run MiniMax M3 with SGLang
Runtime status — August 26, 2026: SGLang v0.5.18, released August 22, is the latest tagged release checked for this guide. Its release notes include the MiniMax-M3 shared/routed-expert overlap change from PR #34542.
Documentation conflict: the current SGLang M3 cookbook still says M3 is not in a tagged release and points to closed reference PR
#27944plus development images. That installation block is stale relative to v0.5.18. Do not rungit fetch origin pull/27944/headas a current installation path, and do not assume thatpip install sglang==0.5.18is turnkey for every checkpoint and accelerator.
Choose the command generated for the exact checkpoint and accelerator, then pin the release, container digest, and flags. SGLang documents BF16 on 8×H200, B200 at TP=8, and B300/GB300 at TP=4. GB200 is described as inferred-supported rather than directly benchmarked. The M3 MXFP8, NVIDIA NVFP4, and AMD MXFP4 routes have different kernels and flags; do not mix them.
Validation boundaries: SGLang documents validated AMD text, reasoning, and tool paths, while AMD vision remains unvalidated. It also says video input has not been tested. A model’s native multimodality is not proof that every serving build supports images and video on every hardware path.
The open-source MiniMax Sparse Attention kernel accelerates supported Blackwell paths. A custom environment may need additional build and warm-up steps. Treat vLLM and SGLang recipes as framework-maintainer guidance, not as a MiniMax guarantee for your server.
Test the local endpoint
Both engines expose an OpenAI-compatible local API. Start with model discovery and a short text request before adding images, tools, reasoning controls, long prompts, or concurrency.
curl http://127.0.0.1:8000/v1/models
curl http://127.0.0.1:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "MiniMaxAI/MiniMax-M3",
"messages": [
{"role": "user", "content": "Return exactly: local M3 is responding"}
],
"temperature": 1.0,
"top_p": 0.95,
"max_tokens": 64
}'For SGLang, change the port to 30000. For MXFP8, use the exact served model ID returned by /v1/models.
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:8000/v1",
api_key="local-only",
)
response = client.chat.completions.create(
model="MiniMaxAI/MiniMax-M3",
messages=[
{"role": "user", "content": "Explain tensor parallelism in two sentences."}
],
temperature=1.0,
top_p=0.95,
max_tokens=256,
)
print(response.choices[0].message.content)Increase context and concurrency safely
- Start with
--max-model-len 32768or131072. - Send one short text request and record idle and peak memory on every rank.
- Test the longest real prompt plus output, not only an empty server.
- Add concurrent requests gradually and watch prefill activation peaks.
- If memory fails, reduce context or concurrency before changing allocator limits.
- Move to FP8 KV cache or more GPUs only after a quality and throughput comparison.
A one-million-token model limit is not the same as a recommended default. Long prompts increase prefill time, cache residency, request latency, and the cost of a retry. Measure retrieval at multiple prompt positions, record failures, and compare shorter structured contexts instead of assuming every token remains equally useful.
Multimodal and coding workloads
M3 is a multimodal model, but text success does not prove the vision path is configured. vLLM and SGLang use separate multimodal encoder settings. SGLang marks AMD vision as unvalidated and says video input has not been tested; GB200 support is inferred rather than directly benchmarked. Test local files, URL fetching, image size limits, processor cache, and base64 requests on the exact engine build.
For coding agents, serving the model is only one layer. The client still controls repository access, shell execution, approval gates, tool results, and secrets. Review the MiniMax AI programming guide before connecting a local endpoint to an autonomous IDE or terminal agent.
Secure the self-hosted endpoint
A local vLLM or SGLang API can accept prompts without a real API key. The placeholder value in the Python example is not authentication. Binding to 127.0.0.1 prevents direct access from other hosts; binding to 0.0.0.0 exposes the service to every reachable interface.
- Place remote access behind an authenticated reverse proxy with TLS.
- Use network allowlists, request-size limits, timeouts, and per-user quotas.
- Do not log prompts or images by default; define retention and access controls.
- Run the container without unnecessary host mounts or privileged access.
- Restrict coding-agent tool permissions separately from model API access.
- Scan model and container updates, pin revisions, and retain a rollback image.
Use the controls in our MiniMax deployment security guide before sending private repositories, customer records, credentials, or regulated data.
Troubleshooting MiniMax M3 local deployment
| Symptom | Likely cause | Action |
|---|---|---|
| Out of memory while loading weights | Wrong checkpoint, too few GPUs, unsupported quantization, or uneven rank visibility | Confirm model ID, per-rank free memory, TP size, GPU architecture, and engine recipe. Do not solve weight OOM only by reducing context. |
| Server loads, then OOMs on the first long prompt | KV cache, sparse index, prefill activations, or CUDA graphs exceeded headroom | Reduce --max-model-len, prompt length, output allowance, and concurrency. |
| vLLM reports a block-size or sparse-index error | MSA expects 128-token blocks | Use --block-size 128 exactly as required by the M3 vLLM recipe. |
| Unknown architecture or parser | The engine version or parser configuration does not support the selected M3 checkpoint | Confirm vLLM is v0.24.0 or newer—prefer a currently tested pinned stable build. Use a special image only when the current checkpoint and hardware recipe explicitly require it. |
| Tool calls appear as raw markup | M3 parser was not enabled | Use vLLM’s minimax_m3 tool/reasoning parsers or SGLang’s auto parsers. |
| Blackwell SGLang JIT appears stuck | MSA kernels compile on first import | Allow the initial compile and warm the kernel cache once before starting all TP ranks. |
| Very low throughput across GPUs | Slow interconnect, incorrect parallelism, CPU offload, or backend mismatch | Check NVLink/InfiniBand topology, framework logs, offload, and the hardware-specific recipe. |
| Text works but images fail | Vision backend, URL access, processor, or checkpoint path is not configured | Test a small base64 image and apply the recipe’s multimodal encoder flags for that GPU platform. |
| Model name rejected by the client | Client uses a different ID than the served endpoint | Query GET /v1/models and copy the returned ID. |
When the hosted API is the better engineering choice
Self-host M3 when you can operate multi-GPU infrastructure, need controlled network placement, can comply with the Community License, and have enough traffic or governance requirements to justify a dedicated deployment. Use the hosted API when you need elastic capacity, do not have suitable accelerators, or want to validate the model before buying hardware.
| Choose self-hosting when… | Choose the API when… |
|---|---|
| You already operate supported H200, Blackwell, or AMD Instinct clusters. | Your workload is intermittent or difficult to forecast. |
| Network placement and infrastructure control are required. | You need a working endpoint without engine and driver maintenance. |
| You can benchmark quantization and monitor quality. | You want to compare M3 before committing capital. |
| You can maintain security, patches, observability, and capacity. | Operational simplicity matters more than owning the weights. |
Frequently asked questions
Can I run MiniMax M3 locally?
Yes, on data-center multi-GPU infrastructure. MiniMax publishes BF16 and MXFP8 weights, and both vLLM and SGLang maintain M3 deployment recipes. The complete model is not a realistic laptop workload.
How much VRAM does MiniMax M3 need?
BF16 weights alone are approximately 795 GiB. Runtime VRAM must also hold cache, activations, communication buffers, and framework overhead. SGLang’s documented BF16 path uses 8×H200. MXFP8 is approximately 440 GB on disk and still targets a multi-GPU Blackwell or AMD Instinct server.
Does 23B active parameters mean M3 fits like a 23B model?
No. About 23B parameters participate in a token’s compute, but the router may select experts from the complete approximately 428B model. Storage and resident weight memory follow the total checkpoint, not only the activated subset.
Can 1 TB of system RAM run the BF16 model?
It may hold the roughly 795 GiB weight floor plus limited overhead, but that does not create a practical CPU inference server. The primary vLLM and SGLang recipes use accelerator memory and multi-GPU parallelism. Heavy CPU offload can be severely bandwidth-bound.
Is there an official 4-bit MiniMax M3?
MiniMax publishes the MXFP8 checkpoint. NVIDIA publishes an NVFP4 checkpoint for Blackwell, and AMD publishes an MXFP4 checkpoint for MI350/MI355 hardware under the MiniMax Community License. They are hardware-specific primary releases, not universal consumer-GPU packages.
Should I use vLLM or SGLang?
Use the engine whose current recipe validates your exact accelerator and checkpoint. vLLM has a tagged M3 path. SGLang v0.5.18 contains M3 changes, but its cookbook still carries a stale pre-release installation block and explicit multimodal validation limits. Benchmark only on identical model revisions, context lengths, batch sizes, and outputs.
Why does M3 OOM below one million tokens?
The one-million-token value is an architectural limit, not reserved memory included with the weights. KV cache, sparse-index state, prefill activations, and concurrent sequences consume additional VRAM. Set a smaller context cap and grow from measured workloads.
Is self-hosted MiniMax M3 automatically private?
No. Privacy depends on network exposure, reverse proxies, client tools, logs, telemetry, storage, backups, administrators, and any external image URLs. Self-hosting gives you more control; it does not configure those controls for you.
Bottom line
To run MiniMax M3 locally, size for the complete 428B-class checkpoint, not the 23B activated subset. The BF16 repository is about 854 GB and its weight floor is about 795 GiB; the official MXFP8 checkpoint is about 440 GB. Long context adds tens or hundreds of GiB of cache state, so start at 32K or 128K and measure before expanding.
Use the hardware-specific vLLM or SGLang recipe, pin the model and engine revisions, preserve 128-token sparse blocks where required, and keep the endpoint private until it has real access controls. For most individual developers, the hosted API is more practical than buying and operating the server required by the full M3 weights.