Soverius AI

For everyone who's using CopilotKit and Google's Hashtag#A2UI and is interested in running them with local LLMs.

Following up on our article last week, here's the fully interactive, visual dashboard to showcase our benchmarks, created by Murat Sari:

👉 https://a2ui-bench.web.app/

The three takeaways:

1️⃣ The "Small Model Cliff" 📉
Models under 14B (like Llama 3.2 1B or Qwen 3 8B) scored near 0%. Surprisingly, this wasn't a failure of reasoning. They consistently struggled to output the proper JSON format. Bluntly: They failed at following the prompt instructions.

2️⃣ Inference Engines Shift Accuracy ⚙️
The engine matters. Running the exact same model weights on different engines yielded completely different semantic accuracy scores. For instance, Qwen 3 14B achieved 90.4% accuracy on vLLM, but dropped to 68.2% on llama.cpp.

3️⃣ The Winner 🏆
Gemma 4 26B MoE (NVFP4 via vLLM) is the winner. It achieved 91.8% accuracy and clocked the fastest median inference speed.

Read the full post

Got a question?

Just hit reply to this email and it'll land in our inbox.

— The Soverius team

Soverius AI · Austria

Legal notice (Impressum) · Privacy Policy

hello@soverius.ai

View in browser · Unsubscribe · Email preferences