Deploying Tencent Hy4 Preview: vLLM vs SGLang Setup Guide + 1M Context Window API Tips
12 days ago 高效码农Deploying Tencent Hy4 Preview: vLLM and SGLang Setup Notes Tencent’s Hy4 preview is a 770B-parameter MoE model with 49B active parameters and a 1M context window. The weights are available on Hugging Face, ModelScope, GitCode, and CNB, including an FP8 quantized version. This guide walks through production deployment using vLLM or SGLang, plus the API calls you’ll need to actually use the model. Where to Find the Real Value in the README The official README is straightforward—architecture specs, benchmark numbers, license—but the most actionable content lives in the Inference and Deployment section. Both vLLM and SGLang are officially supported, and …
Gemma 3: Master Lightweight AI Deployment & Performance Optimization
1 years ago 高效码农Gemma 3: The Complete Guide to Running and Fine-Tuning Google’s Lightweight AI Powerhouse 🧠 Unlocking Next-Generation AI for Every Device Google’s Gemma 3 represents a quantum leap in accessible artificial intelligence. Born from the same groundbreaking research that created the Gemini models, this open-weight family delivers unprecedented capabilities in compact form factors. Unlike traditional bulky AI systems requiring data center infrastructure, Gemma 3 brings sophisticated multimodal understanding to everyday devices – from smartphones to laptops. What makes Gemma 3 revolutionary? 🌐 Multilingual mastery: Processes 140+ languages out-of-the-box 🖼️ Vision-Language fusion: Larger models (4B+) analyze images alongside text ⏱️ Real-time responsiveness: …
How to Run Kimi K2 at Home: A Non-Expert’s 10-Minute Guide
1 years ago 高效码农Running Kimi K2 at Home: A 3,000-Word Practical Guide for Non-Experts What does it actually take to run a one-trillion-parameter model on your own hardware, without hype, without shortcuts, and without a data-center budget? This article walks you through every step—from hardware checklists to copy-paste commands—using only the official facts released by Moonshot AI and Unsloth. 1. What Exactly Is Kimi K2? Kimi K2 is currently the largest open-source dense-or-MoE model available. Parameter count: 1 T (one trillion) Original size: 1.09 TB Quantized size: 245 GB after Unsloth Dynamic 1.8-bit compression—an 80 % reduction Claimed capability: new state-of-the-art on knowledge, …