Can I run Qwen2.5 Coder 32B Instruct on Apple M3 (10-core GPU)?

Q: Can I run Qwen2.5 Coder 32B Instruct on Apple M3 (10-core GPU)?

Partially. Qwen2.5 Coder 32B Instruct needs CPU offload on Apple M3 (10-core GPU) (24 GB) — expect reduced speed.

Partially — with CPU offload.best quant:

20.7 GB of weights, plus 2.4 GB for the software that runs it and the smallest conversation it can hold, comes to 23.1 GB against the 18 GB this 24 GB device leaves free.

Decode speed

—

Usable context

—

Memory

24 GB

Spills to system RAM

Spills to system RAMest

Too large

What is quantisation? →

Plan B — rent it

Qwen2.5 Coder 32B Instruct is hosted by 2 providers from $1.00 per 1M output tokens (Cloudflare Workers AI).Compare providers →

Qwen2.5 Coder 32B Instruct

32.8B · full specs & benchmarks →

Apple M3 (10-core GPU)

24 GB · everything it runs →