Google · Open weights
Gemma 4 12B Instruct
Google's Gemma 4 12B Instruct is an open-weight multimodal model for text and image understanding, with tool calling and structured output support.
- Tools
- Vision
- Structured output
- Input
- Text, Images
- Output
- Text
- Discovery tier
- Open weights
Pricing
USD per 1M tokens- Input
- $0.05
- Output
- $0.25
- Cache read
- $0.025
Reference prices as published by the model provider. API example
Specifications
- Context window
- 131.1K
- Max output
- 32.8K
- Average output (7 days)
- 162 tokens
- Longer than 16% of 468 models
Side by side
Gemma 4 12B Instruct compared
Specifications, prices, speed and benchmark scores next to each other. Benchmarks below show every compared model too.
| Gemma 4 12B InstructGoogleThis model | Mistral Medium 3.1MistralRemove | |
|---|---|---|
| Provider | Mistral | |
| Discovery tier | Open weights | Balanced |
| Added | Aug 1, 2026 | Sep 5, 2025 |
| Limits | ||
| Context window | 131.07K | 131.07K |
| Max output | 32.77K | 32.77K |
| Average output | 162 tokens | 60 tokens |
| Pricing, USD per 1M tokens | ||
| Input | $0.05Best | $0.40 |
| Output | $0.25Best | $2.00 |
| Cache read | $0.025 | — |
| Performance | ||
| Throughput | 23.5 tok/s | — |
| Time to first token | 2.5 s | — |
| Request success (24h) | 98.6% | 100.0%Best |
| Benchmark indexes | ||
| Intelligence Index | 9.4Best | 9.2 |
| Coding Index | — | 20.5 |
| Math Index | — | 38.3 |
| Capabilities | ||
| Tools | Yes | No |
| Vision | Yes | No |
| Structured output | Yes | No |
| Reasoning effort | — | — |
Public evaluations
Benchmarks
Independent evaluation results, with how each score ranks among the models in the Noumen catalog.
- Gemma 4 12B Instruct
- Mistral Medium 3.1
Intelligence Index
- Gemma 4 12B Instruct9.4
- Mistral Medium 3.19.2
Gemma 4 12B Instruct: Better than 19% of 326 scored models
Indexes
Coding Index
Composite of the coding evaluations
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 20.5
Math Index
Composite of the math evaluations
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 38.3
Agentic work
AutomationBench-AA
Workflow automation, penalized for guardrail violations
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 4.1%
AutomationBench-AA tasks completed
Workflows finished end to end without a guardrail violation
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 0.2%
AA-Briefcasev1.1
Agentic knowledge work, rated head to head
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 530 Elo
GDPval-AAv2.1
Economically valuable occupational tasks, rated head to head
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 385 Elo
Document reasoning
GDP.pdf
Professional reasoning over PDF documents, all-pass rate
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 1.2%
AA-LCRv1.1
Reasoning across long documents
- Gemma 4 12B Instruct: 35.0%
- Mistral Medium 3.1: 42.7%Top
MLCR-AA
Long-context reasoning over medical records
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 2.8%
Reasoning and knowledge
Humanity’s Last Exam
Expert-level questions across academic fields
- Gemma 4 12B Instruct: 6.3%Top
- Mistral Medium 3.1: 4.7%
GPQA Diamond
Graduate-level science questions
- Gemma 4 12B Instruct: 66.1%Top
- Mistral Medium 3.1: 58.8%
MMLU-Pro
Multi-subject knowledge and reasoning
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 68.3%
IFBench
Following precise output instructions
- Gemma 4 12B Instruct: 45.2%Top
- Mistral Medium 3.1: 39.8%
Coding
Terminal-Benchv4.0
Practical software engineering in a terminal
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 0.0%
SciCode
Scientific computing in Python
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 32.4%
LiveCodeBench
Recent competitive programming problems
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 40.6%
Math
AIME 2025
Competition mathematics, 2025 problems
- Gemma 4 12B Instruct: —
- Mistral Medium 3.1: 38.3%
Legacy evaluations
Terminal-Bench Hard
Earlier terminal task suite
- Gemma 4 12B Instruct: 11.4%Top
- Mistral Medium 3.1: 10.6%
τ²-Bench Telecom
Tool use in a customer-support simulation
- Gemma 4 12B Instruct: 31.9%
- Mistral Medium 3.1: 40.6%Top
AA-LCR (unversioned)
Long-context reasoning, earlier grading
- Gemma 4 12B Instruct: 35.0%
- Mistral Medium 3.1: 42.7%Top
Scores from Artificial Analysis, updated Oct 26, 2026. Artificial Analysis
- Gemma 4 12B Instruct
- Mistral Medium 3.1
| Leaderboard | Gemma 4 12B Instruct | Mistral Medium 3.1 |
|---|---|---|
| Vision Style Control · Creative Writing | Not in top 20 | #8 · 1204.1 |
Ranks among Noumen catalog models on LMArena leaderboards. Only top-20 placements are listed.LMArena
Live routing data
Performance and reliability
Recent averages across requests routed for this model. Your own speed depends on prompt length, reasoning effort and load.
- Throughput
- 23.5 tok/s
- Faster than 15% of 254 models
- Time to first token
- 2.5 s
- Quicker than 67% of 254 models
- Prompt cache hits
- 30.2%
- About $0.0425 per 1M input tokens with caching
- Request success (24h)
- 98.5%
- Reliable
Cache hits are the share of input tokens recently served from a prompt cache across all requests. They are not a forecast of your own hit rate.
- Healthy (98% or more)
- Degraded (90–98%)
- Down (under 90%)
- No requests
Developers
Use it through the API
Noumen’s API is OpenAI-compatible: any Chat Completions client works with a base URL and a key.
curl https://www.noumen.xyz/api/v1/chat/completions \
-H "Authorization: Bearer $NOUMEN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gemma-4-12b-it",
"messages": [{ "role": "user", "content": "Explain this stack trace." }]
}'import os
from openai import OpenAI
client = OpenAI(base_url="https://www.noumen.xyz/api/v1", api_key=os.environ["NOUMEN_API_KEY"])
reply = client.chat.completions.create(
model="gemma-4-12b-it",
messages=[{"role": "user", "content": "Explain this stack trace."}],
)
print(reply.choices[0].message.content)Related models
Other Google models, starting with the same family.
