T5Gemma Revolutionizes LLM Efficiency: How Encoder-Decoder Adaptation Outperforms Traditional Models

T5Gemma: A New Collection of Encoder-Decoder Gemma Models Introduction In the fast-paced world of large language models (LLMs), encoder-decoder models have often been overshadowed by their decoder-only counterparts. However, encoder-decoder models like T5 still hold significant advantages in many practical applications due to their high inference efficiency, design flexibility, and rich encoder representation for input understanding. Today, we are excited to introduce T5Gemma, a new collection of encoder-decoder LLMs developed by adapting pretrained decoder-only models into the encoder-decoder architecture. From Decoder-Only to Encoder-Decoder T5Gemma explores the potential of building top-tier encoder-decoder models based on pretrained decoder-only models through a technique …