Qwen3-Next-80B
Qwen3-Next uses a highly sparse MoE design: 80B total parameters, but only ~3B activated per inference step. Experiment…
Description
Qwen3-Next uses a highly sparse MoE design: 80B total parameters, but only ~3B activated per inference step. Experiments show that, with global load balancing, increasing total expert parameters while keeping activated experts fixed steadily reduces training loss.Compared to Qwen3’s MoE (128 total experts, 8 routed), Qwen3-Next expands to 512 total experts, combining 10 routed experts + 1 shared expert — maximizing resource usage without hurting performance. The Qwen3-Next-80B-A3B-Instruct performs comparably to our flagship model Qwen3-235B-A22B-Instruct-2507, and shows clear advantages in tasks requiring ultra-long context (up to 256K tokens). File Support: Text, Markdown and PDF files Context window: 131k tokens
Community Metrics
Author
Novita AI
Platform
web
Pricing model
subscription
Categories
- Productivity
Tags
- poe
- novita-ai
- text
Capabilities
- Text input
- Text generation
- By Novita AI