Before the download
Run these commands in your Linux terminal. Use the available RAM and free disk space when picking a model.
Examplebash
lscpu
free -h
df -h "$HOME"Start with Qwen2 0.5B on a laptop. Its compressed file is about 380 MiB. The running program also needs space for converted weights, temporary work and conversation history. Begin with a 1,024-token context. If Linux kills the process when loading a larger model, try the smaller example first.
Which machines have results?
- Intel Core i7: commodity x86 inference and quantized text runs. The i7-14700T has AVX2 and AVX-VNNI profiling results.
- AMD Ryzen 9 9950X3D: large-model and long-context runs on a 160 GB workstation. Check the exact model before expecting the same capacity on a laptop.
- Intel Xeon: 2nd-, 3rd- and 5th-generation machines have validation runs. BF16 and AMX availability depends on the processor. Xeon 6 is listed as planned.
- ARM: TI TDA4VM has NEON portability work. Check for a result on your board before selecting a model.
- GPU or NPU: the walkthroughs here use the Linux CPU runner. No CUDA, Metal or NPU setup is required or verified by these examples.