Introducing Qwen3: The Next Generation of Advanced Language Models
The Qwen LLM series has reached a new milestone with the introduction of Qwen3, a groundbreaking advancement in natural language processing and multimodal capabilities. Building on the success of its predecessors, Qwen3 models are equipped with larger datasets, enhanced architectures, and superior fine-tuning, enabling them to handle even more complex reasoning, language understanding, and generation tasks.
Key Features of Qwen3
Expanded Token Limits
Qwen3 models come with expanded token limits, allowing them to generate longer, more coherent responses and manage more intricate conversational flows. This enhancement ensures that the models can handle complex interactions with ease.
Dense and Mixture-of-Experts (MoE) Models
Qwen3 offers a full range of dense and MoE models, each designed to cater to different needs and applications. These models introduce major breakthroughs in reasoning, instruction-following, agent capabilities, and multilingual support.
Multilingual Support
Qwen3 supports over 100 languages and dialects, ensuring robust performance in multilingual instruction following and translation. This extensive linguistic capability makes Qwen3 a versatile tool for global applications.
Hybrid Thinking Modes
One of the standout features of Qwen3 is its ability to seamlessly switch between thinking mode and non-thinking mode. This dual capability optimizes performance across a wide range of tasks:
- Thinking Mode: Ideal for complex logical reasoning, mathematics, and coding.
- Non-Thinking Mode: Suited for efficient, general-purpose conversation.
Enhanced Reasoning Capabilities
Qwen3 outperforms previous models in reasoning tasks, excelling in mathematics, code generation, and commonsense logical reasoning. This improvement is evident in both thinking and non-thinking modes.
Human Preference Alignment
Qwen3 models are designed to excel at creative writing, role-playing, multi-turn conversations, and instruction following. This alignment ensures a more natural, engaging, and immersive dialogue experience.
Advanced Agent Capabilities
Qwen3 models can interact precisely with external tools in both thinking and non-thinking modes, achieving state-of-the-art results in complex agent-driven tasks.
Model Variants
Qwen3-235B-A22B
This large model features 235 billion total parameters with 22 billion activated parameters, delivering competitive performance across various benchmarks in coding, mathematics, general capabilities, and more.
Qwen3-30B-A3B
A smaller MoE model with 30 billion total parameters and 3 billion activated parameters, surpassing QwQ-32B despite having only one-tenth the number of activated parameters.
Dense Models
Six dense models — Qwen3-32B, Qwen3-14B, Qwen3-8B, Qwen3-4B, Qwen3-1.7B, and Qwen3-0.6B — are open-weighted and released under the Apache 2.0 license, making them accessible for a wide range of applications.
Availability and Deployment
Qwen3 models, including post-trained versions like Qwen3-30B-A3B and their pre-trained counterparts, are available on platforms such as Hugging Face, ModelScope, and Kaggle. For deployment, frameworks like SGLang and vLLM are recommended. For local use, tools such as Ollama, LMStudio, MLX, llama.cpp, and KTransformers are highly recommended.
Empowering Innovation
The release and open-sourcing of Qwen3 aim to drive significant progress in the research and development of large foundation models. Our mission is to empower researchers, developers, and organizations worldwide to create innovative solutions with these state-of-the-art models.
Experience Qwen3
You can experience Qwen3 firsthand through Qwen Chat Web (chat.qwen.ai) and the Qwen mobile app!
Pre-training and Post-training
Pre-training
For Qwen3, the pretraining dataset has been significantly expanded compared to Qwen2.5. While Qwen2.5 was trained on 18 trillion tokens, Qwen3 uses nearly double that amount — approximately 36 trillion tokens — spanning 119 languages and dialects. The pretraining process follows three stages:
- Stage 1 (S1): The model was pretrained on over 30 trillion tokens with a context length of 4K tokens.
- Stage 2 (S2): The dataset was further refined by increasing the proportion of knowledge-intensive content.
- Final Stage: High-quality, long-context data was used to extend the model’s context window to 32K tokens.
Post-training
The hybrid model training pipeline for Qwen3 includes four stages:
- Long Chain-of-Thought (CoT) Cold Start: Fine-tuning on a wide variety of long CoT datasets.
- Reasoning-Based Reinforcement Learning (RL): Enhancing the model’s exploration and exploitation abilities.
- Thinking Mode Fusion: Integrating non-thinking capabilities into the reasoning model.
- General Reinforcement Learning (General RL): Further improving the model’s overall capabilities.
Qwen3 represents a significant leap forward in the development of large language models, offering unparalleled performance and versatility for a wide range of applications.
🛠️ Developer Resources
The Qwen3 GitHub repository provides comprehensive tools and documentation, including:
- Fine-Tuning Examples: Guides for adapting Qwen3 to specific tasks. (GitHub)
- Inference Guides: Instructions for deploying Qwen3 models efficiently. (GitHub)
- Community Support: An active discussions forum for collaboration and troubleshooting. (GitHub)
🔗 Explore Qwen3
- GitHub Repository: (GitHub)
- Hugging Face Models: (GitHub)
- ModelScope Access: (GitHub)
- Official Chat Interface: chat.qwen.ai
Qwen3 represents a significant step forward in open-source AI development, offering advanced capabilities for a wide range of applications.
