Which MacBook Pro for running AI models locally

For running AI models on the Mac itself, the number that matters most is unified memory, then memory bandwidth, then GPU cores. A model has to fit in memory to run at a useful speed, so the memory you pick when you order sets the biggest model you can use, and you cannot add more later. On Apple's current line-up that ranges from 16GB on the 14-inch M6 to 128GB on the top M5 Max.

Short version: the M6 suits small models and experiments. An M5 Pro with 48GB or 64GB suits most people who use local models daily. An M5 Max with the 40-core GPU is for the largest models, or for work where every wait holds you up. The rest of this post explains why, and where Apple's own claims stop.

Why memory comes first

A language model is a very large file of numbers (its weights). To produce each word, the chip reads through most of those weights. On a Mac, the CPU, the GPU and the Neural Engine all share one pool of unified memory, so the GPU can use almost everything you bought, with no separate graphics memory to run out of.

A ballpark helps, as long as you treat it as one. A model compressed to 4 bits per weight needs very roughly half a gigabyte for every billion parameters, before anything else. So a model with a few billion parameters fits in a few gigabytes, and one with tens of billions needs tens of gigabytes. The real figure depends on how the model was compressed, and the model also needs working space, which grows as a conversation gets longer. macOS and your other apps want their share too. Keep the model well under the memory you have, not right up against it.

That is why the memory ceiling matters more than any speed figure. If a model does not fit, no amount of GPU cores helps. How much memory a MacBook Pro needs covers the general case; this post is the AI version.

Memory and bandwidth on each model

All figures below are from Apple's tech specs pages.

Bandwidth is how fast the chip can read from memory. Producing text one word at a time is mostly limited by that read speed, so a chip with more bandwidth tends to answer faster with the same model. That is a general rule about how these models work, not a figure from Apple. It is also why a Max can feel quicker than a Pro even on a model that fits comfortably in both.

Note the pairing on the Max. The 32-core GPU comes only with 36GB. To get 48GB, 64GB or 128GB you take the 40-core GPU. The M5 Max GPU choice goes through who needs which.

What Apple says about AI speed

Apple's AI claims are specific about what they measure, and it is worth reading them closely.

• M6: Apple says up to 1.3x faster time to first token for LLM prompt processing than the 14-inch MacBook Pro with M5, and a dual 16-core Neural Engine up to 1.9x faster than M5. It also says the 12-core GPU has a Neural Accelerator in each core.
• M5 Pro: Apple says up to 3.9x faster LLM prompt processing than the MacBook Pro with M4 Pro, and up to 3.7x faster AI image generation.
• M5 Max: Apple says up to 4x faster LLM prompt processing than the MacBook Pro with M4 Max, and up to 3.8x faster AI image generation.

Two things to take from that. First, "prompt processing" and "time to first token" describe how quickly the model reads what you give it and starts to answer. That is the part that suffers when you paste in a long document. It is not the same as how fast the words appear afterwards, and Apple's headline claims do not give a figure for that. Second, every number is "up to", measured by Apple on specific configurations against specific older Macs. Treat them as direction, not a promise for your workload.

The Neural Accelerators sit inside the GPU cores, which is why the GPU core count matters for AI on these chips as well as for graphics.

Choosing by what you do

Trying models, summarizing, light coding help

Small and medium models, from a few billion parameters up to a dozen or so, generally fit in 16GB to 32GB. The M6 with 32GB is a reasonable machine for this. The 16GB version leaves little room once a browser and a few apps are open, and it is also the one Apple lists with the lower bandwidth. If AI is a side use, take more memory before you take more of anything else.

Daily use with larger models

This is the M5 Pro with 48GB or 64GB. The 20-core GPU is the one that reaches 64GB, so if you want that ceiling you take the 20-core configuration. It has the bandwidth to feel responsive, and enough memory to hold a large model with your normal apps open.

The biggest models, or long documents all day

The M5 Max with the 40-core GPU and 64GB or 128GB. Apple's March announcement puts the M5 Max's ceiling at 128GB of memory and 614GB/s of bandwidth. It is the only MacBook Pro you can configure with 128GB, so it is the one with room for the biggest models a laptop can realistically hold. It also sits at the top of the line-up, so be sure the work needs it. If you would use a model that size once a month, a cloud service is likely the better fit for those runs.

Image generation

Apple's M5 Pro and M5 Max claims for AI image generation are real gains over M4, but image models are usually smaller than the biggest language models. Memory matters less here, and GPU cores matter more. A 20-core M5 Pro is plenty for most hobby use.

What the MacBook Pro does not solve

• Model availability. Which models run, and how well they are tuned for the Mac, depends on the software you use. Check that the model you want has a Mac-friendly version before you buy memory for it.
• Heat and battery. Apple says these MacBook Pros keep their full speed unplugged, M6 and M5 Pro and M5 Max alike, which helps when you run a model away from a desk. Long runs still warm the Mac and drain the battery far faster than a web page does. Apple's battery hours are measured on video streaming and web browsing, not on AI work.
• Storage. Models are large files and people collect them. Check the storage you configure before you plan to keep several on the Mac.

Where Notchpals fits

A few weeks of trying new models can fill a drive before you notice, and Notchpals keeps half an eye on that. Its reminder characters live in the notch, which every mini-LED MacBook Pro has, M6, M5 Pro and M5 Max alike. Once the startup disk has less than 10 percent of its space left, a Tech care character such as Juice comes down to say so, at most once a week.

JuiceSo many downloads, so little room. A tragedy. Let some of them go.

It reads how much space is left and nothing about the files themselves, so it can't tell which model you haven't opened in a month. Deciding what to delete is still your job.

The short decision

If you are not sure, buy memory first. The chip choice can be small and the memory large, and the memory is the part you cannot change later. For a first local setup, a 14-inch M6 with 32GB is the entry point. For an everyday setup with room to grow, an M5 Pro with 64GB. For the biggest models, an M5 Max with the 40-core GPU. If you are weighing the whole range, which MacBook Pro to buy puts the models side by side.