Today I downloaded the Hy3 IQ3_XXS quant by Vcruz and got it running on a single Asus GX10 (DGX Spark) machine. It did not run directly in LM Studio, because a particular version of Llama CPP is required, which needs to be compiled from source. Deepseek V4 Flash had no trouble completing that task, and got it running in just a few minutes.

IQ3_XXS is a quality quant which generated 17 tokens per second on the GX10. For longer context, better quality, and faster performance, I'll prefer to run the Q4_KM quant on a cluster of 2 machines. vLLM will likely be the best engine to use for this model, because it enables tensor parallelism, which typically speeds up clustered performance by about 1.8x.

This is an exciting model to add to the local stable. It is very capable, compared to other models that can run on modest local hardware - even beating Deepseek V4 Pro on many key benchmarks (and the local hardware required to run Deepseek V4 Pro are huge in comparison). I've explored creating apps with it multiple times in the past, while it was in preview on OpenRouter, and have always been impressed with what it can accomplish. Even the preview version was close to current frontier quality - and now that it's out of preview, it seems even stronger.

This is definitely a top-tier model to get for DGX Spark and Strix Halo class machines. I'm surprised that it performs faster than Deepseek V4 Flash does on the DS4 platform, and it may end up being an even smarter model than V4 Flash - plus DS4 only currently enables slower pipeline parallelism on clustered machines, and that slow clustered speed actually slows down DS4 :( Hy3's world knowledge and coding skills are absolutely impressive for its size, especially for front end work.

I'll try using Hy3 for a few weeks and report back about how it compares to V4 Flash in agentic tasks.

Today I downloaded the Hy3 IQ3_XXS quant by Vcruz and got it running on a single Asus GX10 (DGX Spark) machine. It did not run directly in LM Studio, because it requires a particular version of Llama CPP which needs to be compiled from source. Deepseek V4 Flash had no trouble completing that task, and got it running in just a few minutes.

IQ3_XXS is a quality quant which generated 17 tokens per second on the GX10. For longer context, better quality, and faster performance, I'd prefer the Q4_KM quant on a cluster of 2 machines. vLLM will likely be the best engine to use for this model, because it enables tensor parallelism, which typically speeds up clustered performance by ~1.8x.

This is an exciting model to add to the local stable. It is very capable - even beating Deepseek V4 Pro on many key benchmarks. I've explored creating apps with it many times in the past, while it was in preview on OpenRouter, and have always been impressed with what it accomplished. It's very close to frontier quality - and now that it's out of preview, it seems even stronger.

This is definitely a top-tier model to get for the DGX Spark and Strix Halo class machines. It performs faster than Deepseek V4 Flash does on the DS4 platform, and it may end up being an even smarter model (plus DS4 only currently enables slower pipeline parallelism on clustered machines). Hy3's world knowledge and coding skills are absolutely impressive for its size (especially for front end work).

I'll try using it for a few weeks and report back about how it compares to V4 Flash in agentic tasks.

I downloaded the Hy3 IQ3_XXS quant by Vcruz and got it running today on a single Asus GX10 (DGX Spark) machine. It did not run directly in LM Studio, but Deepseek V4 Flash had no trouble compiling the particular version of Llama CPP from source which is required to run it.

IQ3_XXS is a quality quant, and it ran at 17 tokens per second. For longer context, better quality, and faster performance, I'd run the Q4_KM quant on a cluster of 2 machines.

This is an exciting model to add to the local stable. It is very capable - even beating Deepseek V4 Pro on many key benchmarks. I explored creating apps with it many times, while it was in preview on OpenRouter, and have always been impressed with what it accomplished - very near frontier quality - and now that it's out of preview, it seems even stronger.

This is definitely a top-tier model to get for the DGX Spark and Strix Halo class machines. It performs faster than Deepseek V4 Flash does on the DS4 platform, and it may end up being an even smarter model. It's world knowledge and coding skills are absolutely impressive for its size (especially for front end work). I'll try using it for a few weeks and report back about how it compares to V4 Flash in agentic tasks.

I was happy to get Hy3 running today on a single Asus GX10 (DGX Spark) machine. I downloaded the IQ3_XXS quant by Vcruz. It did not run directly in LM Studio, but Deepseek V4 Flash did not have any trouble compiling the particular version of Llama CPP from source which is required to run it.

The IQ3_XXS is a quality quant, and it ran at 17 tokens per second. For longer context and better quality, I'd run the Q4_KM quant on a cluster of 2 machines.

This is an exciting model to add to the local stable. It is very capable - even beating Deepseek V4 Pro on many key benchmarks. I explored creating apps with it many times while it was in preview on OpenRouter, and have always been impressed with what it accomplished - very near frontier quality - and now that it's out of preview, it seems even stronger.

This is definitely a top-tier model to get for the DGX Spark and Strix Halo class machines. It performs faster than Deepseek V4 Flash on the DS4 platform, and it may end up being an even smarter model. It's world knowledge and coding skills are very impressive - I'll try using it for a few weeks and report back about how it compares to V4 Flash in agentic tasks.