Hy3 is currently free on Openrouter, and works locally on a single DGX Spark. It's a serious contender for best self-hosted model - get to know it now, test, or just build some apps, while it's free.

Nick AntonaccioAdmin
Jul 13, 2026 at 15:58 (edited, 7 revisions)

Today I downloaded the Hy3 IQ3_XXS quant by Vcruz and got it running on a single Asus GX10 (DGX Spark) machine. It did not run directly in LM Studio, because a particular version of Llama CPP is required, which needs to be compiled from source. Deepseek V4 Flash had no trouble completing that task, and got it running in just a few minutes.

IQ3_XXS is a quality quant which generated 17 tokens per second on the GX10. For longer context, better quality, and faster performance, I'll prefer to run the Q4_KM quant on a cluster of 2 machines. vLLM will likely be the best engine to use for this model, because it enables tensor parallelism, which typically speeds up clustered performance by about 1.8x.

This is an exciting model to add to the local stable. It is very capable, compared to other models that can run on modest local hardware - even beating Deepseek V4 Pro on many key benchmarks (and the local hardware required to run Deepseek V4 Pro is much more expensive).

I've explored creating apps with Hy3 multiple times in the past, while it was in preview on OpenRouter, and have always been impressed with what it can accomplish. Even the preview version was close to current frontier quality - and now that it's out of preview, it seems even stronger.

Hy3's world knowledge and coding skills are absolutely impressive for its size, especially for front end work. This is definitely a top-tier model to get for DGX Spark and Strix Halo class machines. I'm surprised that it performs much faster than Deepseek V4 Flash does on its own DS4 platform, and Hy3 may likely end up being an even smarter model than V4 Flash.

One critical difference between Hy3 and V4 Flash, is that the V4 Flash DS4 engine only currently enables slower pipeline parallelism on clustered machines, and that slow clustered speed actually reduces the performance of DS4 when run across multiple servers. So, I'm going to be really putting Hy3 through the ringer to see if it can be a full and complete improved replacement for V4 Flash.

I'll try using Hy3 for a few weeks and report back about how it compares to V4 Flash in agentic tasks.

History

Nick AntonaccioAdmin
Jul 13, 2026 at 16:01 (edited, 5 revisions)

Today I made a Northwind database demo with Hy3:

http://1y1z.com:4832

It has all the typical bells and whistles:

full CRUD editing of all fields in the database (create, read, update, delete)
complete sort and filter functionality in all datatable displays (click column headers to sort)
auth login to save changes to the database ('admin'/'admin')
input/export CSV files to/from every table
export order invoices to PDF
an analytics page with charts
responsive layout for mobile and desktop screens
toggle UI between dark/light mode

Hy3 produced great output for every software development prompt I gave it, and for all the agentic tasks I tried with it. It built everything quickly without any hitches, and didn't choke for a moment. It's also done amazingly well with the suite of world knowledge tests I run on every new model. For a model in the several hundred billion parameter class, which can run on a single machine with 128GB RAM (DGX Spark, Strix Halo, and PCs with 4x 32GB GPUs), I haven't seen anything better yet.

So, this model is certainly going to be among my top 3 for local hosting.

My next tests will be with clustered DGX Sparks. From what I understand, vLLM already supports Hy3, with fast tensor parallelism. That means, using two Asus GX10s connected by the ConnectX ports, the expectation is for speeds to be about 1.8x that of running on a single machine.

If this holds true, Hy3 should end up running at over 30 tokens per second in a cluster of just 2 GX10s. Performance like that would make Hy3 my go-to for local inference, and likely the first solution I'd recommend to clients who want to build a serious on-premise self-hosted LLM solution.

Add Mimo 2.5 and Stepfun 3.7 Flash models for multimodal needs, Deepseek V4 Flash for additional reliable coding and agentic capability (V4 Flash is still my main daily driver LLM), Qwen 3.6 35a3 MTP for fast sub-agent coding tasks, plus Gemma 4 26a4 QAT for fast multimodal sub-agent tasks (vision, audio, video), and that collection should form an extremely capable & performant group of digital brains to handle any challenge.

I'm very excited by the current state of all these models, which push the limits of what can be accomplished with relatively affordable local GPU hardware. Together with improved harnesses, we're pushing past what was possible with frontier models less than a year ago. Hy3, Deepseek V4 Flash, Mimo 2.5, and Stepfun 3.7 are each near frontier in various respects, and Qwen 3.6 & Gemma 4 turn even small consumer GPUs into useful LLM machines - these are exciting times!

History

Nick AntonaccioAdmin
Jul 13, 2026 at 16:01 (edited, 2 revisions)

Note that Hy3 is free to use on OpenRouter until July 21, 2026, so now is a great time to get to know it well, and to build lots of software without spending a penny.

History

Please login to post a reply.