NeuroGenesysCan I run gpt-oss-20b?
Last updated • By Will Schulz
gpt-oss-20b needs about 14.4 GB of memory at 4-bit (Q4_K_M) with an 8K-token context, so it fits on a single 16 GB GPU, or runs on a Mac with 24 GB of unified memory. At 16-bit it needs 44.5 GB, so it fits on a single 48 GB GPU such as the RTX A6000, RTX 6000 Ada or L40S, or runs on a Mac with 64 GB of unified memory.
How much memory does gpt-oss-20b need?
| Precision | Weights | Total, 8K context | Total, 32K context | Total, 128K context |
|---|---|---|---|---|
| 16-bit (BF16 / FP16) | 42.0 GB | 44.5 GB | 45.1 GB | 47.5 GB |
| 8-bit (Q8_0) | 22.3 GB | 24.2 GB | 24.8 GB | 27.2 GB |
| 4-bit (Q4_K_M) | 12.8 GB | 14.4 GB | 15.0 GB | 17.4 GB |
| Native release (4.25 bits) | 11.2 GB | 12.7 GB | 13.3 GB | 15.7 GB |
Total = weights + FP16 KV cache + 3% of the weights + 1 GB of runtime overhead, as in the VRAM calculator. 12 of 24 layers use a 128-token window, so their cache stops growing past that. Longer contexts are only usable up to the model's own context window; check the model card.
Which GPUs can run gpt-oss-20b?
| GPU memory | 16-bit (BF16 / FP16) | 8-bit (Q8_0) | 4-bit (Q4_K_M) |
|---|---|---|---|
| 8 GB | 8 cards | 4 cards | 2 cards |
| 12 GB | 4 cards | 4 cards | 2 cards |
| 16 GB | 4 cards | 2 cards | fits on 1 |
| 24 GBRTX 3090, RTX 4090, L4 | 2 cards | fits on 1 | fits on 1 |
| 32 GBRTX 5090 | 2 cards | fits on 1 | fits on 1 |
| 40 GBA100 40GB | 2 cards | fits on 1 | fits on 1 |
| 48 GBRTX A6000, RTX 6000 Ada, L40S | fits on 1 | fits on 1 | fits on 1 |
| 80 GBA100 80GB, H100 PCIe, H100 SXM | fits on 1 | fits on 1 | fits on 1 |
| 141 GBH200 | fits on 1 | fits on 1 | fits on 1 |
| 180 GBB200 | fits on 1 | fits on 1 | fits on 1 |
| 192 GBAMD MI300X | fits on 1 | fits on 1 | fits on 1 |
Assumes the runtime can allocate 95% of each card and every card pays the fixed overhead. Vendor memory sizes are binary (a 24 GB card holds 25.8 × 10⁹ bytes), and that is accounted for.
Can I run gpt-oss-20b on a Mac?
| Unified memory | 16-bit (BF16 / FP16) | 8-bit (Q8_0) | 4-bit (Q4_K_M) |
|---|---|---|---|
| 24 GBM4 Pro | no | no | fits |
| 32 GBM2 Max | no | fits | fits |
| 48 GBM3 Max (40-core GPU), M4 Pro, M4 Max (40-core GPU) | no | fits | fits |
| 64 GBM2 Max, M2 Ultra, M3 Max (40-core GPU), M4 Pro, M4 Max (40-core GPU) | fits | fits | fits |
| 96 GBM2 Max, M3 Ultra | fits | fits | fits |
| 128 GBM2 Ultra, M3 Max (40-core GPU), M4 Max (40-core GPU) | fits | fits | fits |
| 192 GBM2 Ultra | fits | fits | fits |
| 256 GBM3 Ultra | fits | fits | fits |
| 512 GBM3 Ultra | fits | fits | fits |
macOS lets the GPU use about 75% of unified memory by default; you can raise the limit with sudo sysctl iogpu.wired_limit_mb.
How fast does gpt-oss-20b run at 4-bit?
| Hardware | Memory bandwidth | Tokens / s | |
|---|---|---|---|
| RTX 309024 GB | 936 GB/s | 284 | estimator |
| RTX 409024 GB | 1,008 GB/s | 306 | estimator |
| RTX 509032 GB | 1,792 GB/s | 544 | estimator |
| RTX A600048 GB | 768 GB/s | 233 | estimator |
| RTX 6000 Ada48 GB | 960 GB/s | 292 | estimator |
| L424 GB | 300 GB/s | 91.1 | estimator |
| L40S48 GB | 864 GB/s | 262 | estimator |
| A100 40GB40 GB | 1,555 GB/s | 472 | estimator |
| A100 80GB80 GB | 2,039 GB/s | 619 | estimator |
| H100 PCIe80 GB | 2,000 GB/s | 608 | estimator |
| H100 SXM80 GB | 3,350 GB/s | 1018 | estimator |
| H200141 GB | 4,800 GB/s | 1458 | estimator |
| B200180 GB | 8,000 GB/s | 2430 | estimator |
| AMD MI300X192 GB | 5,300 GB/s | 1610 | estimator |
| Apple M2 Max32 GB config or larger | 400 GB/s | 113 | estimator |
| Apple M2 Ultra64 GB config or larger | 800 GB/s | 226 | estimator |
| Apple M3 Max (40-core GPU)48 GB config or larger | 400 GB/s | 113 | estimator |
| Apple M3 Ultra96 GB config or larger | 819 GB/s | 231 | estimator |
| Apple M4 Pro24 GB config or larger | 273 GB/s | 77.0 | estimator |
| Apple M4 Max (40-core GPU)48 GB config or larger | 546 GB/s | 154 | estimator |
Decoding reads the active expert weights and the KV cache once per token, so speed is roughly memory bandwidth ÷ bytes per token. Assumes 70% of peak bandwidth on GPUs and 65% on Macs, one GPU or Mac holding the whole model. Only hardware that holds it alone is listed.
Questions
How much VRAM does gpt-oss-20b need?
About 14.4 GB at 4-bit (Q4_K_M), 24.2 GB at 8-bit (Q8_0) and 44.5 GB at 16-bit (BF16), for one request with an 8K-token context. At 4-bit the weights are 12.8 GB; the rest is the KV cache and runtime overhead. With 128K tokens of context the 4-bit total rises to 17.4 GB.
Can I run gpt-oss-20b on a 24 GB GPU like the RTX 4090?
Yes, at 8-bit (Q8_0) with an 8K context; 16-bit does not fit on one card. A 24 GB card such as the RTX 3090, RTX 4090 or L4 holds it with the runtime's usual headroom.
Can I run gpt-oss-20b on a Mac?
Yes: at 4-bit it needs a Mac with at least 24 GB of unified memory, and at 16-bit 64 GB. By default macOS lets the GPU use about 75% of unified memory, which these figures allow for.
How fast does gpt-oss-20b run?
At 4-bit, one request generates about 306 tokens per second on a RTX 4090 and 154 tokens per second on an Apple M4 Max (40-core GPU). These are estimates from memory bandwidth (70% of peak on GPUs, 65% on Macs) with 4K tokens in context; real runtimes vary.
Why does gpt-oss-20b need so much memory if only 3.6B parameters are active?
gpt-oss-20b is a mixture-of-experts model: each token uses about 3.6B of its 21B parameters, but any token can be routed to any expert, so all 21B must be in memory. Memory follows the total; speed follows the active count, which is why it runs faster than a dense model of the same size.
Work it out for your setup
- Free toolVRAM calculator: gpt-oss-20b at Q4_K_MChange the quantization, context, KV-cache precision and batch size.
- Free toolLLM inference speed estimatorTokens per second, time to first token and batching on any GPU or Mac.
- GuideCan I run it? All open-weight modelsMemory needed for every model we track, and what each GPU or Mac can run.
Sources
- gpt-oss-20b model card and config.json — Hugging Face
