Groq

By Groq

Ultra-fast inference API for open models on custom LPU hardware

Website ↗ Docs ↗

APIProprietary

#llm #inference #fast #api

Free tier Paid

Doc version 1.0 · Updated June 28, 2026

Related tools

Overview

Groq is an inference provider that runs open-source models on its custom LPU hardware, delivering some of the fastest tokens-per-second available.
It exposes an OpenAI-compatible API with a generous free developer tier, so it drops into existing code through LiteLLM or the OpenAI SDK.

flowchart LR
  prompt[prompt] --> groq[(Groq LPU)]
  groq --> fast[very fast tokens]

The Code samples tab shows a LiteLLM-routed call.

When to use it

Choose Groq when speed is the priority and an open model (Llama, Qwen, and others) fits the task — especially real-time agents, where its throughput shortens each reasoning step.

Pricing

Free developer tier (rate-limited, no card). Paid is pay-as-you-go per token. Prices per million tokens (input / output)

Plan	Price	Notes
Developer (free)	$0	Rate-limited, no card
Llama 3.3 70B	$0.59 / $0.79	Example paid rate

Source ↗ · as of 2026-06-27

Example

Python · via LiteLLM

When to use

Reach for Groq when latency matters — interactive agents and chat where tokens-per-second is the bottleneck.

from litellm import completion

# GROQ_API_KEY in the environment. Groq serves open models (Llama, etc.)
# on its LPU hardware — the same OpenAI-style call, just very fast.
resp = completion(
    model="groq/llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Say hello in one short sentence."}],
)
print(resp.choices[0].message.content)