CLaRa: How 128x Document Compression Supercharges RAG Without Labels
24 days ago 高效码农# CLaRa: Teaching a Language Model to Compress, Retrieve, and Answer in One Breath How to shrink Wikipedia 128× and still beat full-text baselines—without ever labeling “relevant” documents. ## TL;DR CLaRa (Continuous Latent Reasoning) unifies retrieval and generation inside a single LLM by: Offline-compressing every document into 32–256 “memory tokens”; Learning to retrieve with a differentiable top-k operator; Training everything end-to-end with nothing more than next-token prediction loss. On four open QA data sets the framework matches or outperforms full-text RAG while using 1–2 % of the usual context length. ## Table of Contents The Two Walls Hitting Every RAG …
TiDAR: The Breakthrough Language Model Architecture Merging Diffusion and Autoregression
25 days ago 高效码农TiDAR: The Next-Gen Language Model Architecture Merging Diffusion and Autoregression This article answers the core question: How can language models maintain generation quality while drastically improving efficiency, achieving a balance between high throughput and optimal GPU utilization? Introduction: The Efficiency-Quality Dilemma in Language Models Core question of this section: What inherent trade-offs exist between generation efficiency and quality in current mainstream language models? As artificial intelligence evolves toward general intelligence, the success of large language models (LLMs) relies heavily on leveraging GPU computational resources effectively. However, the two dominant language model architectures—autoregressive (AR) models and diffusion language models (dLMs)—face an …
On-Device Language Models: How MiniCPM4 Achieves 128K Context AI on Mobile Devices
6 months ago 高效码农MiniCPM4: Run Powerful Language Models on Your Phone or Laptop Achieve 128K context processing with 78% less training data using 0.5B/8B parameter models optimized for edge devices Why We Need On-Device Language Models While cloud-based AI models like ChatGPT dominate the landscape, edge devices (smartphones, laptops, IoT systems) have remained largely excluded due to computational constraints. Traditional large language models face three fundamental barriers: Compute Overload: Processing 128K context requires calculating all token relationships Memory Constraints: Loading an 8B parameter model demands ~32GB RAM Training Costs: Standard models require 36 trillion training tokens MiniCPM Team’s breakthrough solution, MiniCPM4, shatters these …
IBM’s Bamba Model: Merging Transformers and SSMs to Break AI Efficiency Barriers
7 months ago 高效码农The rise of large language models (LLMs) like ChatGPT has made the Transformer architecture a household name. Yet, as conversations grow longer, Transformers face a critical roadblock: escalating latency and computational costs. To tackle this, IBM Research partnered with Carnegie Mellon University, Princeton University, and other leading institutions to launch Bamba, an open-source hybrid model that combines the expressive power of Transformers with the runtime efficiency of state-space models (SSMs). This breakthrough promises to redefine AI efficiency. Let’s dive into how Bamba works and why it matters. The Transformer Dilemma: Why Long Conversations Slow Down AI 1.1 The Power of …