Running MiniCPM-V 4.7 35B on a 16GB M4 Mac mini: MoE Setup and Benchmarks
Contents
OpenBMB claims that MiniCPM-V 4.7 35B-A3B runs on an iPhone using just 3GB of RAM.
Curious whether it would run on a 16GB M4 Mac mini, we set it up to test firsthand. If it worked, we wanted to evaluate it against prior LLM comparisons on identical coding, reasoning, and table extraction benchmarks.
Complete Package Size vs. MoE Expert Splitting
According to the official OpenBMB README, the model generates over 13 tokens per second (13 tokens/s) on an iPhone 17 Pro Max.
The target model is MiniCPM-V 4.7 35B-A3B. A separate 1B edition exists, but we focused strictly on the 35B-A3B version.
The 35B-A3B architecture is a Mixture of Experts (MoE) model with roughly 35 billion total parameters, activating approximately 3 billion parameters per token.
MoE systems select specialized computational subsets (experts) depending on incoming tokens. Because unselected experts may be called in subsequent tokens, the complete weight collection still needs persistent storage.
Checking the official mobile apps documentation, the mobile bundle splits into a core file and distinct expert files.
The experts reside on the SSD, loading into RAM on demand. This differs from conventional GGUF models where the entire network footprint is held in memory.
| Package Format | File Size | Setup Usage |
|---|---|---|
| Standard Q4_K_M | 21.17 GB (vision weights separate) | Not used |
| LES Core GGUF | 1.66 GB | Used |
| LES 40 Expert Files | 13.84 GB total | Used |
| mmproj Vision Projector | 1.12 GB | Used for vision tasks |
| Complete LES Bundle | 16.62 GB (includes manifest) | Stored on internal SSD |
Sizes were calculated from byte counts in the standard release repo and the split release repo (using 1 GB = 1 billion bytes). We used the tc-mb/MiniCPM-V-Apps-gguf repository referenced by OpenBMB.
Speculative Loading with LES
Reading expert weights from SSD on demand introduces latency that bottlenecks generation speed.
To mitigate this, Lookahead Expert Scheduling (LES) speculatively predicts which experts subsequent layers will need, overlapping SSD read operations with GPU computation.
The prediction governs read order; it does not drop unpredicted experts chosen by the actual router. All routed experts are loaded and calculated. We did not artificially restrict the active expert count.
Official benchmarks also use quantization and DSpark speculative decoding. For our initial baseline, we used mixed quantization with LES while keeping DSpark disabled.
Building and Setup on an M4 Mac mini
The test environment was configured as follows, placing all model files on the internal SSD to maximize transfer rates:
| Component | Specification |
|---|---|
| Machine | Mac mini (Apple M4) |
| Memory | 16 GiB (17,179,869,184 bytes) |
| OS | macOS 26.5.2 |
| Inference Engine | OpenBMB/llama.cpp-omni (commit 0b74404) |
| GPU Backend | Metal |
| Model | MiniCPM-V-4_7-35B-A3B-mix_q3gud_im-les.gguf and associated experts |
| Model Revision | 8716e5209f308fd53f197ccd7d50c6fdcc2dc37d |
| Scripts | Python 3.14.4 within a dedicated uv virtual environment |
The core model file cannot run without the experts/ directory and mmproj-model-f16.gguf.
The 43 files totaled 16,623,585,508 bytes.
The engine was built from source. The specific commit lacked a standard llama-server build target, so we compiled and ran via llama-cli. We did not modify engine source files.
cmake -S llama.cpp-omni -B llama.cpp-omni/build \
-DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON \
-DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_APP=OFF -DLLAMA_BUILD_UI=OFF
cmake --build llama.cpp-omni/build --target llama-cli -j 4A smoke test prompt (“What is the capital of Japan? Answer only the city name in Japanese.”) returned “東京” (Tokyo).
Total run time was approximately 6.6 seconds, with a peak resident set size (RSS) of 2.46 GB and a reported generation speed of 10.0 tokens/s. Because the reply was so brief, this prompt was not used for throughput evaluation.
Running Benchmark Tasks
We reused the test prompts and validation harness from our earlier comparison, omitting an ambiguous edge question from task D.
| Task | Objective | Evaluation Metric |
|---|---|---|
| Python Interval Merging | Merge overlapping intervals; keep touching endpoints separate; handle invalid types | 24 test cases including empty input, NaN, and huge integers |
| Event Aggregation | Deduplicate by highest version, break ties by arrival order, sum paid events by currency | Match against golden JSON |
| Document Fact Verification | Distinguish inputs from outputs, preview from GA, and pricing constraints | Correctness on 3 targeted questions |
All three tasks were combined into a single prompt and submitted across three independent runs per model.
Generated Python scripts were checked for unauthorized network or file access before local execution. Models had no runtime Internet or tool execution access.
Thinking mode (thinking) produces internal reasoning tokens before final output. We disabled it (off) for the initial baseline comparison across MiniCPM-V, Qwen, and DeepSeek.
| Model | Invocation | Parameters |
|---|---|---|
| MiniCPM-V 4.7 35B-A3B | Local llama-cli | thinking off, temperature 0.2, seeds 43/44/45 |
| Qwen3.8 Max | ModelScope (Qwen-Ambassador account) | enable_thinking=false, temperature 0.2 |
| GPT-6.1 Sol | Codex CLI 0.162.1 | medium reasoning effort |
| Claude Opus 5.5 | Claude Code 2.1.285 | medium reasoning effort |
| DeepSeek-V4.1-Flash | ModelScope | thinking.type=disabled, temperature 0.2 |
MiniCPM-V was configured with an 8,192-token context window and a 3,072-token output limit. Qwen and DeepSeek also had a 3,072-token limit. Sol and Opus used CLI defaults with reasoning set to medium.
MiniCPM-V expert caching parameters were set to --moe-slot-bank 640, LLAMA_MOE_DECODE_WINDOW=16, and lookahead depth 8. Batch and micro-batch were both 128. Processes restarted clean between runs.
Text Benchmark Results
Across three test runs, all models scored 100% on event aggregation (3/3) and fact verification (9/9).
The differentiator was the Python interval merging problem: MiniCPM-V and Qwen3.8 Max scored 23/24 across all three runs, while the other three models scored 24/24.
| Model | Code Passing (out of 24) | Response Time (Mean ± Std Dev) |
|---|---|---|
| MiniCPM-V 4.7 35B-A3B | 23, 23, 23 | 93.24 ± 2.17 s |
| Qwen3.8 Max | 23, 23, 23 | 7.76 ± 0.18 s |
| GPT-6.1 Sol | 24, 24, 24 | 19.21 ± 2.90 s |
| Claude Opus 5.5 | 24, 24, 24 | 18.82 ± 0.74 s |
| DeepSeek-V4.1-Flash | 24, 24, 24 | 4.13 ± 0.38 s |
Response times include process startup, model loading, and network latency for cloud endpoints. MiniCPM-V executed locally on the M4 Mac mini, whereas the others ran on cloud APIs.
OverflowError in Float Conversion on Large Integers
Both MiniCPM-V and Qwen3.8 Max failed on an edge case interval with an endpoint of 10**400.
Both implementations called math.isnan() or math.isinf() on integer inputs, triggering an OverflowError in Python 3.14.4.
MiniCPM-V’s generated solution checked values without distinguishing integers from floating-point numbers:
if math.isnan(start) or math.isnan(end):
raise ValueError("NaN values are not allowed")
if math.isinf(start) or math.isinf(end):
raise ValueError("Infinite values are not allowed")Under Python’s math library, isnan and isinf expect floating-point arguments. Passing an arbitrary-precision integer that exceeds float limits triggers an explicit overflow during float conversion.
In contrast, Sol, Opus, and DeepSeek verified finiteness only when the input was already a float:
if isinstance(endpoint, float) and not math.isfinite(endpoint):
raise ValueError('endpoints must be finite')Testing with Japanese Order Table Images
Because MiniCPM-V supports visual inputs, we tested it with an image of a Japanese order table.
The test image was a 1,000×650 pixel PNG containing five order rows.
We instructed models to transcribe all rows into JSON, calculate the total value of paid orders, identify the paid order with the highest value, and return null for date (which is absent from the image). The table contains the following data:
| Order ID | Status | Quantity | Unit Price |
|---|---|---|---|
| A-17 | Paid (支払済) | 3 | 1,200 |
| B-04 | Pending (保留) | 2 | 3,400 |
| C-23 | Paid (支払済) | 7 | 450 |
| D-81 | Cancelled (キャンセル) | 5 | 2,100 |
| E-95 | Paid (支払済) | 4 | 875 |
The correct paid total is 10,250 yen (3×1,200 + 7×450 + 4×875), the highest paid order is A-17 (3,600 yen), and the date is null.
We compared MiniCPM-V with Qwen3.5-35B-A3B on ModelScope and GPT-6.1 Sol via Codex CLI, sending identical prompts across three runs.
All three models achieved perfect OCR transcription (3/3) and correctly returned null for the missing date (3/3).
The divergence came during arithmetic summation and highest-value identification.
| Model | Paid Sum Correct | Highest Order ID Correct | Response Time (Mean ± Std Dev) |
|---|---|---|---|
| MiniCPM-V 4.7 35B-A3B | 0/3 | 3/3 | 68.09 ± 1.21 s |
| Qwen3.5-35B-A3B | 0/3 | 2/3 | 7.57 ± 0.80 s |
| GPT-6.1 Sol | 3/3 | 3/3 | 12.78 ± 1.36 s |
MiniCPM-V returned 7,700 yen on all three runs.
Qwen3.5 returned 8,500 yen, 9,200 yen, and 8,300 yen, misidentifying the highest order as C-23 on its second run. Sol answered accurately on all attempts.
The JSON output from both MiniCPM-V and Qwen captured every row, quantity, and price accurately. The OCR stage succeeded; the arithmetic step failed. Extracting data to JSON and computing sums with code avoids this arithmetic issue.
Comparing Input Formats and Thinking Mode
To isolate whether arithmetic errors stemmed from visual encoding or core reasoning, we fed the same five rows as plain text and JSON.
We toggled reasoning mode (thinking on/off) across three runs per format on MiniCPM-V using identical seeds.
We also added three runs of Qwen3.5-35B-A3B with thinking on on the original image. Every configuration maintained 100% transcription accuracy (3/3).
| Model | Format | Thinking | Paid Sum Correct | Response Time (Mean ± Std Dev) |
|---|---|---|---|---|
| MiniCPM-V | Image | off | 0/3 | 68.09 ± 1.21 s |
| MiniCPM-V | Image | on | 1/3 | 102.14 ± 66.96 s |
| MiniCPM-V | Text | off | 0/3 | 46.82 ± 3.65 s |
| MiniCPM-V | Text | on | 3/3 | 142.53 ± 18.51 s |
| MiniCPM-V | JSON | off | 0/3 | 47.79 ± 3.90 s |
| MiniCPM-V | JSON | on | 3/3 | 119.37 ± 17.18 s |
| Qwen3.5-35B-A3B | Image | off | 0/3 | 7.57 ± 0.80 s |
| Qwen3.5-35B-A3B | Image | on | 3/3 | 10.44 ± 0.87 s |
Without thinking mode (off), MiniCPM-V failed the arithmetic step across text (8,000 / 7,700 / 7,700 yen) and JSON (8,900 yen on all runs). The calculation errors occurred independently of visual processing.
Enabling thinking mode (on) resolved all arithmetic errors for text and JSON inputs (3/3).
On image input, however, MiniCPM-V scored only 1/3 with thinking enabled. On the two failed attempts, generation stopped prematurely while writing raw JSON inside the reasoning block (<think> tags) and left the outputs incomplete.
Enabling thinking mode on Qwen3.5-35B-A3B brought image arithmetic to 3/3 accuracy.
Additional Evaluation Across 10 Generated Tables
To test whether these observations were artifacts of a single table, we evaluated 10 distinct synthetic table images varying in row order, quantities, unit prices, and paid status.
Layout and typography remained identical: 5 rows each, with 1 to 4 paid rows.
All models achieved 10/10 on row transcription and 10/10 on returning null for dates.
| Model | Mode | Paid Sum Correct | Highest ID Correct | Response Time (Mean ± Std Dev) |
|---|---|---|---|---|
| MiniCPM-V | off | 4/10 | 8/10 | 61.95 ± 0.76 s |
| MiniCPM-V | on | 3/10 | 8/10 | 73.95 ± 33.50 s |
| Qwen3.5-35B-A3B | off | 2/10 | 8/10 | 6.96 ± 3.30 s |
| Qwen3.5-35B-A3B | on | 10/10 | 10/10 | 10.36 ± 2.02 s |
| GPT-6.1 Sol | medium | 10/10 | 10/10 | 10.32 ± 1.63 s |
For MiniCPM-V with thinking on, 9 out of 10 runs terminated inside the reasoning log before producing clean final answers. The single run that generated proper closing tags answered the arithmetic correctly.
Recalculating the totals by piping transcribed JSON outputs into Python produced 100% correct answers across all 50 runs.
Separating OCR transcription from arithmetic aggregation bypasses these calculation errors.
Memory Usage Across Workloads (Peak RSS)
We tracked peak resident set size (RSS) using /usr/bin/time -l alongside reported generation throughput:
| Workload | Peak RSS Range | Generation Throughput |
|---|---|---|
| 3 Text Tasks (No Images) | 3.52–3.53 GB | 7.6–8.0 tokens/s |
| 1 Japanese Table Image | 4.60–4.61 GB | 5.8–6.0 tokens/s |
The 16GB M4 Mac mini handled image inference without memory exhaustion.
The 3GB memory footprint advertised for iPhone targets iOS environments and differs from peak RSS accounting on macOS.
Text Generation Speed and Memory with DSpark
DSpark accelerates inference through speculative decoding, using a smaller draft model to generate candidate tokens verified in parallel by the main model.
We downloaded the official DSpark weights and converted them to an F16 GGUF file (approximately 789 MB).
While vision speculative decoding encountered Metal buffer constraints and positional embedding errors, text speculative decoding ran stably. We compared three runs each with DSpark enabled and disabled:
| Setting | Parameter Value |
|---|---|
| Input | 5-row table passed as text |
| Thinking / Temperature | off / 0 |
| Context / Max Output | 8,192 / 1,024 tokens |
| Expert Cache Width / Lookahead | 32 / 8 |
| Slot Bank | 640 |
| DSpark Draft Candidates | Up to 3 tokens per step |
| DSpark | Speed (Mean ± Std Dev) | Response Time (Mean ± Std Dev) | Peak RSS Range |
|---|---|---|---|
| Disabled | 7.36 ± 0.23 tokens/s | 45.17 ± 1.35 s | 4.40–4.46 GB |
| Enabled | 8.05 ± 0.14 tokens/s | 43.24 ± 1.05 s | 5.52–5.53 GB |
On this text query, DSpark increased generation throughput by roughly 9.5% and reduced total response time by 4.3%.
Peak RSS increased by approximately 1.1 GB. Across all runs, 201 out of 216 speculative draft tokens were accepted by the main model.
Test scripts, prompts, generated code, and raw outputs are stored in LiltingChannelLabo at 2026/10/14/minicpm-v-4-7-mac-mini/.