NeMo Guardrails

By NVIDIA

Programmable rails for LLM apps — keep conversations safe, on-topic, and on-policy with Colang flows.

Website ↗ Repository ↗ Docs ↗

v0.23.02026-07-01PythonApache-2.0

#guardrails #safety #dialogue #python

6.8k Completely free Open source

Doc version 1.0 · Updated June 28, 2026

Related concepts

Harness Engineering Doc version 1.0 · Updated June 28, 2026 Wrapping an LLM in scaffolding — a control loop, tools, a sandbox, guardrails, and evaluation — so it behaves reliably in production.

Related tools

Guardrails AI

Guardrails & Safety

Overview

NeMo Guardrails is an open-source toolkit for adding programmable guardrails to LLM-based conversational apps.
You define rails in Colang — input, dialog, retrieval, and output checks — and the library enforces them around every model call so replies stay safe and on-topic.

flowchart LR
  user[user] --> rails[NeMo Guardrails]
  rails --> policy[Colang rails]
  rails --> llm[LLM]
  llm --> safe[on-topic reply]

The Code samples tab shows loading a config and generating a guarded reply.

When to use it

Choose NeMo Guardrails when you need programmable, testable controls over an LLM conversation — blocking off-topic or unsafe turns and steering dialog flows — rather than relying on prompt instructions alone.

Example

Python · add rails

When to use

Reach for NeMo Guardrails when an LLM app must stay on-topic and on-policy, with input, dialog, and output rails enforced before and after the model.

How it flows

RailsConfig loads your Colang rails; generate runs the user turn through them and the LLM, returning a checked, on-topic reply.

Structure

flowchart LR
  user[user] --> rails[NeMo Guardrails]
  rails --> policy[Colang rails]
  rails --> llm[LLM]
  llm --> safe[on-topic reply]

from nemoguardrails import LLMRails, RailsConfig

# Load config.yml + Colang flows from a directory.
config = RailsConfig.from_path("./config")
rails = LLMRails(config)

response = rails.generate(messages=[
    {"role": "user", "content": "Tell me about your refund policy."},
])
print(response["content"])

Example

NeMo Guardrails — implementation sample

A real, runnable mini-project. Download it and run with Docker.

View samples/nemo-guardrails_1 on GitHub ↗

June 28, 2026

NeMo Guardrails — implementation sample

A tiny NeMo Guardrails sample: it loads a Colang greeting rail and generates a reply, so the bot answers through a fixed, approved message instead of free-form output. The main and embedding models are both OpenAI, so no local model download is needed.

Configure

cd samples/nemo-guardrails_1
cp .env.sample .env
# edit .env: set OPENAI_API_KEY (used for the main + embedding models)

Run with Docker

cd samples/nemo-guardrails_1
docker build -t aas-nemo .
docker run --rm --env-file .env aas-nemo

Run with Docker (in a devcontainer with DooD)

The foreground docker run may print nothing under Docker-outside-of-Docker — run detached and follow the logs:

docker logs -f "$(docker run -d --env-file .env aas-nemo)"

Run locally

cd samples/nemo-guardrails_1
pip install -r requirements.txt
python app.py

Example run

The greeting rail maps the user's "hi there" to a fixed bot message, so the reply is the same every run:

bot: Hello! I'm a guardrailed assistant — I only respond through approved rails.

Files

.env.sample

# Copy to `.env` and add your OpenAI API key (used for the main + embedding models).
#   cp .env.sample .env
OPENAI_API_KEY=

MODEL=gpt-4o-mini

app.py

"""NeMo Guardrails: put programmable rails around an LLM.

Loads a tiny rails config — a greeting flow written in Colang — and generates a
reply. The rail maps a user greeting to a fixed, safe bot message instead of
free-form output, showing how rails steer a conversation. Both the main and the
embedding model are OpenAI, so only OPENAI_API_KEY is needed (no local download).

Run:
    docker build -t aas-nemo .
    docker run --rm --env-file .env aas-nemo
"""

import os

from dotenv import load_dotenv
from nemoguardrails import LLMRails, RailsConfig

load_dotenv()

MODEL = os.environ.get("MODEL", "gpt-4o-mini")

YAML_CONFIG = f"""
models:
  - type: main
    engine: openai
    model: {MODEL}
  - type: embeddings
    engine: openai
    model: text-embedding-3-small
"""

COLANG_CONFIG = """
define user express greeting
  "hello"
  "hi there"
  "hey"

define bot express greeting
  "Hello! I'm a guardrailed assistant — I only respond through approved rails."

define flow greeting
  user express greeting
  bot express greeting
"""


def main() -> None:
    config = RailsConfig.from_content(
        yaml_content=YAML_CONFIG,
        colang_content=COLANG_CONFIG,
    )
    rails = LLMRails(config)

    response = rails.generate(messages=[{"role": "user", "content": "hi there"}])
    print("bot:", response["content"])


if __name__ == "__main__":
    main()

Dockerfile

FROM python:3.12-slim

ENV PYTHONUNBUFFERED=1

WORKDIR /app

# NeMo Guardrails pulls `annoy`, which ships no wheels and compiles from source.
RUN apt-get update \
    && apt-get install -y --no-install-recommends build-essential \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

ENTRYPOINT ["python", "app.py"]