Building Chinese Reward Models: Mastering CheemsBench & CheemsPreference for AI Alignment
1 months ago 高效码农Building Chinese Reward Models from Scratch: A Practical Guide to CheemsBench and CheemsPreference Why Do We Need Dedicated Chinese Reward Models? In the development of large language models (LLMs), reward models (RMs) act as “value referees” that align AI outputs with human preferences. However, current research faces two critical challenges: Language Bias: 90% of existing studies focus on English, leaving Chinese applications underserved Data Reliability: Synthetic datasets dominate current approaches, failing to capture authentic human preferences The Cheems project – a collaboration between the Institute of Software (Chinese Academy of Sciences) and Xiaohongshu – introduces the first comprehensive framework for …
Decoding WorldPM: How 15 Million Forum Posts Are Revolutionizing AI Alignment Strategies
1 months ago 高效码农Decoding WorldPM: How 15 Million Forum Posts Are Reshaping AI Alignment Visual representation of AI alignment concepts (Credit: Unsplash) The New Science of Preference Modeling: Three Fundamental Laws 1. The Adversarial Detection Principle When analyzing 15 million StackExchange posts, researchers discovered a power law relationship in adversarial task performance: # Power law regression model def power_law(C, α=0.12, C0=1e18): return (C/C0)**(-α) # Empirical validation training_compute = [1e18, 5e18, 2e19] test_loss = [0.85, 0.72, 0.63] Key Findings: 72B parameter models achieve 92.4% accuracy in detecting fabricated technical answers Requires minimum 8.2M training samples for stable pattern recognition False positive rate decreases exponentially: …