Mistral
Mistral 7B v0.3
Total params
7.25B
Active params
7.25B
Layers
32
Hidden size
4096
Attention heads
32
KV heads (GQA)
8
Vocab size
32,768
Native context window
32,768
Native precision
BF16
Size this model
Opens the sizing calculator pre-filled with this model at BF16 weights / FP16 KV cache and a typical workload — sign in to run it and see GPU/cloud recommendations.
Size this model →