SqueezeBits

Check out highlights from the 2nd vLLM Korea Meetup! open-source use cases and real-world production examples that showcase vLLM's technical maturity!

High signal Matched: korea, open-source

Explore why Physical AI deployment needs synthetic data at scale with Squeezebits' research and discover how to overcome inference bottlenecks to accelerate Roboost Agent.

High signal Matched: inference, research, agent

Scaling Physical AI requires reliable synthetic data. Learn how RoBoost Agent integrates NVIDIA Cosmos to transform world models into trustworthy data engines for robotics and autonomous driving.

High signal Matched: agent

A recap of the Intel® Gaudi® hands-on workshop co-hosted by SqueezeBits and Lablup. AI model compression, fine-tuning, and vLLM serving on Gaudi® hardware with Backend.AI.

High signal Matched: serving, model, fine-tuning

Introducing ATOM™-Max, rebellions’ next-generation NPU designed for high-performance AI inference. Learn how its runtime, profiling tools, and PyTorch-native integrations enable developers to run and serve models efficiently without sacrif...

High signal Matched: inference, generation, serve, performance, npu, introducing, rebellions

Rebellions and SqueezeBits Co-Host a vLLM Hands-on Workshop: Workshop Highlights, PyTorch Best Practices, Performance Optimization, and Developer First-Hand Tips!

High signal Matched: performance, rebellions

Explore how the Yetter Inference Engine overcomes the limitations of step caching and model distillation for diffusion models. We analyze latency, diversity, quality, and negative-prompt handling to reveal what truly matters for scalable,...

High signal Matched: inference, generation, latency, model

Explore how Intel’s new Gaudi-3 compares to Gaudi-2, NVIDIA A100, and H100. We analyze real-world GEMM efficiency, attention performance, and LLM serving results to uncover what truly matters for AI inference and training workloads.

High signal Matched: inference, serving, gemm, performance, h100, training

Meet 'Yetter': the generative AI API service built for speed, efficiency, and scalability. Powered by our optimization inference engine, it delivers reliable image, video, and future LLM services at a fraction of the cost.

High signal Matched: inference, cost, api

The guide to LLM guided decoding! This deep-dive benchmark compares XGrammar and LLGuidance on vLLM and SGLang to help you find the optimal setup for generating structured output based on your use case.

High signal Matched: decoding, benchmark, performance

In this article, we introduce how to run LLMs efficiently on Apple Silicon with disaggregated inference technique.

High signal Matched: inference, prefill, gpu, npu

Efficient AI Study & Meetup recap: SqueezeBits' community study on AI model compression, featuring paper reviews, participant interviews, and networking from the offline meetup.

High signal Matched: model, paper

Trimming large multilingual vocabularies in Small Language Models (SLM) is a simple, low-risk way to boost efficiency to its limit. It accelerates the model inference significantly while keeping accuracy almost unchanged.

High signal Matched: inference, model

LoRA excels at efficient fine-tuning but suffers at higher ranks due to gradient entanglement. We introduce GraLoRA, which addresses these issues through finer-grained, block-wise updates, significantly enhancing performance and expressivi...

High signal Matched: performance, cost, fine-tuning, lora

At SqueezeBits we have been empowering developers to efficiently deploy complex AI models while minimizing performance trade-offs with OwLite toolkit. With OwLite v2.5, we're excited to announce official support for Qualcomm Neural Network...

High signal Matched: performance

SqueezeBits has partnered with Intel to make Gaudi NPUs more usable in practice. We optimized LLMs and diffusion models for Gaudi-2 and created yetter, a generative AI API service.

High signal Matched: api

SqueezeBits at Japan IT Week Spring 2025 in Tokyo: AI model compression demos, OwLite and Fits on Chips introductions, Japan market entry experiences, and team stories from the frontline.

High signal Matched: model

This article describes the experimental results of quantized Vision Transformer model and its variants with OwLite.

High signal Matched: model, quantized

This article describes the experimental results of quantized YOLO models with OwLite.

High signal Matched: quantized

Discover how OwLite simplifies AI model optimization with seamless integration and secure architecture.

High signal Matched: performance, model, quantization

This article discusses inference efficiency when running the FLUX.1 models on Intel Gaudi-2 hardware.

High signal Matched: inference

With TensorRT-LLM now open source, we can finally take a deep dive into the secret sauce behind its impressive performance.

High signal Matched: performance, open source

This article introduces Fits on Chips, an LLMOps toolkit for performance evaluation.

High signal Matched: performance, evaluation

A brief review of the research paper from our team, published at ICML 2024.

High signal Matched: verification, paper, research

This article is about an open-source library for direct conversion of PyTorch models to TensorRT-LLM.

High signal Matched: open-source

This article provides a comparative analysis of serving vision-language models on vLLM and TensorRT-LLM.

High signal Matched: serving

In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability.

High signal Matched: performance, accelerator, fp8, quantization, evaluate

In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability.

High signal Matched: performance, accelerator, evaluation, evaluate

This article provides a comparative analysis of speculative decoding.

High signal Matched: decoding, speculative decoding

This article provides a comparative analysis of multi-LoRA serving capabilities of vLLM and TensorRT-LLM frameworks.

High signal Matched: serving, lora

In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability.

High signal Matched: performance, accelerator, evaluation, evaluate

In this blog series, we thoroughly evaluate Intel's AI accelerator, the Gaudi series, focusing on its performance, features, and usability.

High signal Matched: performance, accelerator, evaluate

This article provides a comparative analysis of the effects of KV cache quantization on vLLM and TensorRT-LLM frameworks.

High signal Matched: kv cache, quantization

This article provides a comparative analysis of the effects of weight-activation quantization on vLLM and TensorRT-LLM frameworks.

High signal Matched: quantization

This article provides a comparative analysis of the effects of weight-only quantization on vLLM and TensorRT-LLM frameworks.

High signal Matched: quantization

This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks, focusing on performance with fixed and dynamic datasets.

High signal Matched: performance

This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks with various sampling methods.

High signal Matched: performance

This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks, focusing on batching configurations and thoroughly examining the effects of maximum batch size and maximum number of tokens.

High signal Matched: serving

This article provides a comparative analysis of vLLM and TensorRT-LLM frameworks for serving LLMs, evaluating their performance based on key metrics like throughput, TTFT, and TPOT to offer insights for practitioners in optimizing LLM depl...

High signal Matched: serving, throughput, performance, ttft, tpot, evaluation, evaluating

Estimating the cost savings from model compression.

High signal Matched: cost, model

Clarifying the misunderstandings in AI model compression

High signal Matched: model

The Blackwell GPU from GTC 2024 was astonishing.Analysis of the Nvidia GPU evolution & what it means for GPU users.

High signal Matched: gpu, blackwell

Do I need to COMPRESS my AI model? : the short answer is “YES” — and here’s why.

High signal Matched: model

AI model compression for acceleration is essential. The question is HOW? Here are 4 key methodologies.

High signal Matched: model

Wrap up 8 weeks of online studies and take a look at how SqueezeBits makes an effort to maintain the AI compression community to expand!

Watchlist Matched: none

Sharing GTC 2026 insights, which is the Largest AI Industry Conference for developers! If you’ve ever wondered what it’s like for an AI startup to run a booth at such a massive event, you won’t want to miss this!

Watchlist Matched: none

From Edge AI to NVIDIA GTC: Squeezebits team members share firsthand stories from global AI events, including networking insights, technical trends, and conference experiences.

Watchlist Matched: none

This article describes when to use Fits on Chips toolkit with specific use cases.

Watchlist Matched: none

This article explores the rise and fall of ONNX, from its early success as a unifying stasndard for AI frameworks to its gradual shift into a niche tool in the era of PyTorch 2.0.

Watchlist Matched: none

This article provides a comparative analysis of automatic prefix caching.

Watchlist Matched: none

This article provides a comparative analysis of different parallelism strategies on vLLM and TensorRT-LLM frameworks.

Watchlist Matched: none

This article provides a comparative analysis of schedulers in vLLM and TensorRT-LLM frameworks.

Watchlist Matched: none

SqueezeBits' IT exhibition recap: from AI model compression demos to hands-on OwLite experiences, booth visitor reactions, and more. Read our on-the-ground event story!

Watchlist Matched: model

An introduction to tokenizers and their implications in language models.

Watchlist Matched: none