DSPy

By Stanford NLP

Program — don't prompt — language models; declare modules and let DSPy compile and optimize the prompts.

Website ↗ Repository ↗ Docs ↗

v3.2.12026-05-05PythonMIT

#prompt-optimization #modular #stanford #python

36.3k Completely free Open source

Doc version 1.0 · Updated June 28, 2026

Overview

DSPy (from Stanford NLP) lets you write LM pipelines as declarative Python modules with typed signatures (question -> answer) instead of hand-tuned prompt strings.
Optimizers (“teleprompters”) then compile those modules — searching prompts and few-shot examples against a metric — so quality improves without manual prompt fiddling.

flowchart LR
  sig["question -> answer"] --> module[DSPy module]
  module --> lm[(LM via LiteLLM)]
  lm --> out[answer]

The Code samples tab shows declaring a module and compiling it with an optimizer — pick from the selector to compare.

When to use it

Choose DSPy when you’d rather optimize a pipeline against a metric than maintain brittle prompt strings — especially for multi-stage reasoning or RAG systems.

Example

Module · ChainOfThought

When to use

Reach for a module when you'd rather declare the task as a signature than hand-write a prompt string. ChainOfThought adds a reasoning step before the answer.

How it flows

DSPy builds the prompt from the signature, calls the LM through LiteLLM, then fills in the answer field.

Structure

flowchart LR
  sig["question -> answer"] --> cot[ChainOfThought]
  cot --> lm[(LM via LiteLLM)]
  lm --> ans[answer]

import dspy

dspy.configure(lm=dspy.LM("anthropic/claude-opus-4-8"))

# The signature "question -> answer" is the whole spec; DSPy writes the prompt.
qa = dspy.ChainOfThought("question -> answer")

result = qa(question="Why is the sky blue?")
print(result.answer)

For a runnable build of this module, see dspy_1 in the Implementation tab.

When to use

Reach for an optimizer when you'd rather tune a pipeline against a metric than hand-tune prompts.

How it flows

BootstrapFewShot searches few-shot demos that maximize the metric on the trainset, producing a compiled module.

Structure

flowchart LR
  train[(trainset)] --> opt[BootstrapFewShot]
  qa[ChainOfThought] --> opt
  opt -->|metric| tuned[compiled module]

import dspy
from dspy.teleprompt import BootstrapFewShot

dspy.configure(lm=dspy.LM("anthropic/claude-opus-4-8"))

qa = dspy.ChainOfThought("question -> answer")

# A few labeled examples to optimize against.
trainset = [
    dspy.Example(question="2+2?", answer="4").with_inputs("question"),
    dspy.Example(question="Capital of France?", answer="Paris").with_inputs("question"),
]

def exact_match(example, pred, trace=None):
    return example.answer.lower() in pred.answer.lower()

# Search few-shot demos that maximize the metric, then compile a tuned module.
tuned = BootstrapFewShot(metric=exact_match).compile(qa, trainset=trainset)

Example

DSPy — a self-optimizing predictor

A real, runnable mini-project. Download it and run with Docker.

View samples/dspy_1 on GitHub ↗

June 28, 2026

DSPy — a self-optimizing predictor

A tiny DSPy program: it declares a typed signature (question -> answer), wraps it in a ChainOfThought module, and prints the answer. DSPy builds the prompt — you only describe the task. The LM is routed through LiteLLM, so the same code works with Anthropic Claude, OpenAI, or Google AI Studio (Gemini) — just change MODEL in .env.

Configure

cd samples/dspy_1
cp .env.sample .env
# edit .env: set MODEL and the matching provider key

MODEL picks the provider:

Provider	`MODEL` example	Key in `.env`
Anthropic Claude	`claude-opus-4-8`	`ANTHROPIC_API_KEY`
OpenAI	`gpt-4o`	`OPENAI_API_KEY`
Google AI Studio	`gemini/gemini-2.5-flash`	`GEMINI_API_KEY`
Ollama (local)	`ollama_chat/qwen3.5:9b`	`OLLAMA_API_BASE`

.env is gitignored — only .env.sample is committed.

Ollama (local models): first pull the model on the host — ollama pull qwen3.5:9b (or ollama run qwen3.5:9b). Then set MODEL=ollama_chat/qwen3.5:9b and point OLLAMA_API_BASE at the server — no API key needed. In a devcontainer with DooD the container reaches the host's Ollama at http://host.docker.internal:11434; running locally use http://localhost:11434.

Run with Docker

cd samples/dspy_1
docker build -t aas-dspy .
docker run --rm --env-file .env aas-dspy "Why is the sky blue?"

Run with Docker (in a devcontainer with DooD)

In a dev container that talks to the host Docker daemon (Docker-outside-of-Docker), the foreground docker run above often prints nothing and exits 0 — but the run itself succeeds. The script runs to completion and Docker captures all of its output; only the live attached stream drops it over the VM boundary. You can confirm this: docker logs on the same container shows the full output, the container exits 0, and it is not an OOM. Run detached and follow the logs instead:

cd samples/dspy_1
docker build -t aas-dspy .
docker logs -f "$(docker run -d --env-file .env aas-dspy \
  "Why is the sky blue?")"

Run locally

cd samples/dspy_1
pip install -r requirements.txt
python app.py "Why is the sky blue?"

python-dotenv loads .env automatically. Get keys from Anthropic, OpenAI, or Google AI Studio.

Example run

Output varies by model and run — LLMs are non-deterministic, so the exact wording (and an agent's steps) differ each time. Below is one run with claude-opus-4-8.

The sky is blue because of a process called Rayleigh scattering. Sunlight is made up of all colors, but as it passes through Earth's atmosphere, the shorter blue wavelengths are scattered in all directions by air molecules much more than the longer red wavelengths. This scattered blue light fills the sky, making it appear blue to our eyes.

Files

.env.sample

# Copy this file to `.env` and fill in the key for the provider you want.
#   cp .env.sample .env
#
# One codebase, many providers — DSPy routes the request via LiteLLM based on MODEL.

# Pick the model. Examples:
#   claude-opus-4-8              Anthropic Claude   -> needs ANTHROPIC_API_KEY
#   gpt-4o                       OpenAI             -> needs OPENAI_API_KEY
#   gemini/gemini-2.5-flash      Google AI Studio   -> needs GEMINI_API_KEY
#   ollama_chat/qwen3.5:9b       Ollama (local)     -> needs OLLAMA_API_BASE, no key
MODEL=claude-opus-4-8

# Set only the key for the provider you chose; leave the rest blank.
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GEMINI_API_KEY=

# Ollama only — where the Ollama server runs (pull the model first, e.g.
# `ollama pull qwen3.5:9b`). Other providers ignore this line.
#   Local, no Docker:           http://localhost:11434
#   Docker / DooD on the host:  http://host.docker.internal:11434
OLLAMA_API_BASE=http://host.docker.internal:11434

app.py

"""DSPy: declarative LM modules instead of hand-tuned prompts.

A typed signature (`question -> answer`) is the whole spec — DSPy builds the
prompt for you. The LM is routed through LiteLLM, so MODEL in `.env` picks the
provider (claude-opus-4-8 / gpt-4o / gemini/gemini-2.5-flash) without touching
the code.

Run:
    docker build -t aas-dspy .
    docker run --rm --env-file .env aas-dspy "Why is the sky blue?"
"""

import os
import sys

import dspy
from dotenv import load_dotenv

# Load .env when running locally; in Docker the keys come from --env-file.
load_dotenv()


DEFAULT_QUESTION = "Why is the sky blue? Answer in one sentence."


def main() -> None:
    question = " ".join(sys.argv[1:]) or DEFAULT_QUESTION

    # MODEL chooses the provider; dspy.LM routes it through LiteLLM.
    model = os.environ.get("MODEL", "claude-opus-4-8")
    dspy.configure(lm=dspy.LM(model=model))

    # ChainOfThought adds a reasoning step before producing the answer field.
    qa = dspy.ChainOfThought("question -> answer")
    result = qa(question=question)

    print(result.answer)


if __name__ == "__main__":
    main()

Dockerfile

# Minimal image for the DSPy sample.
FROM python:3.13-slim

# Stream stdout/stderr unbuffered so the output appears live.
ENV PYTHONUNBUFFERED=1

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

# Provide your config at run time (.env is NOT baked into the image):
#   docker run --rm --env-file .env aas-dspy "<question>"
ENTRYPOINT ["python", "app.py"]

Overview

When to use it

Pricing

When to use

How it flows

When to use

How it flows

Configure

Run with Docker

Run with Docker (in a devcontainer with DooD)

Run locally

Example run