MoA-100M

Description

MoA-100M is an open-weight text-generation model with roughly 100M parameters published by reaperdoesntknow on the Hugging Face Hub. It has 2,990 downloads. Runs with transformers.

Categories

• Research

Tags

• moa_metric
• mixture-of-attentions
• distance-attention
• metric-attention
• mqa
• hyperffn
• router-gating
• convergentintel

Capabilities

• Text generation
• Library: transformers
• 100M parameters