DocsUse it from6 min read

Frameworks#

Copy-paste recipes for putting shad0w in front of the LLM call your framework already makes. shad0w needs one thing from a framework: a way to ask the LLM for an answer. Every recipe below is that, plus a line or two.

Pick a recipe#

You use	Easiest route	Code change
any Python function that asks an LLM	`@shad0w.decide`	a decorator
LangChain / LangGraph	`llm_teacher(complete=...)` or `shad0w_runnable`	a few lines
LiteLLM (SDK or proxy)	`shad0w.integrations.litellm`	one import, or a callback
Vercel AI SDK	`shad0wMiddleware` / `decisionModel`	wrap the model
Pydantic AI and other decision-model clients	the proxy	`base_url` only
FastAPI, Django, Flask	create once, call per request	a few lines
any OpenAI SDK, any language	the proxy	`base_url` only

Every recipe has the same behaviour: the table (shad0w's small learned model) answers what it is certified on (tested on answers it never trained on, with a bound on how often it disagrees with your LLM); everything else goes to your LLM and the answer is logged for the next train().

Any function: the decorator#

If you already have a function that asks your LLM, annotate its return type with the options:

from typing import Literal
import shad0w

@shad0w.decide()
def route(text: str) -> Literal["billing", "tech", "sales", "other"]:
    return my_llm_call(text)          # Instructor, Pydantic AI, LangChain, raw SDK: anything

route("my invoice is wrong")          # from the table once trained and certified, else from my_llm_call (logged)
route.shadow.train()                  # after ~1,000 logged answers

-> bool makes a yes/no question, and an Enum return type returns enum members. To let shad0w call the LLM itself, pass it to the decorator: @shad0w.decide(llm="openai/gpt-6-luna"), and leave the body as .... See Python.

LangChain / LangGraph#

Use the chat model you already have as the teacher. shad0w builds the prompt and maps the reply to one option:

import shad0w
from langchain_openai import ChatOpenAI
from langchain_core.runnables import RunnableLambda

llm = ChatOpenAI(model="gpt-6-luna", temperature=0)
teacher = shad0w.llm_teacher(["billing", "tech", "sales", "other"], question="team",
                             complete=lambda messages: llm.invoke(messages).content)
team = shad0w.decision("team", llm=teacher)

route = RunnableLambda(lambda x: team(x["text"]).answer)        # use it in a chain...

def pick_next(state):                                           # ...or as a LangGraph conditional edge
    return team(state["messages"][-1].content).answer
graph.add_conditional_edges("triage", pick_next, {"billing": "billing_agent", "tech": "tech_agent", "sales": "sales_agent", "other": "human"})

Or wrap a chain you already have (your own prompt and model) as a Runnable. The table answers what it is certified on; otherwise the chain runs, its reply is mapped to one option and logged:

import shad0w
from shad0w.integrations.langchain import shad0w_runnable

team = shad0w.decision("team", options=["billing", "tech", "sales", "other"])   # no llm: the chain supplies it
router = shad0w_runnable(team, prompt | llm)        # .invoke(text | {"text": ...} | messages), .ainvoke(...) too
router.invoke({"text": "my invoice is wrong"})      # -> "billing"

With shad0w_runnable, shad0w never calls the model itself, so there are no background spot checks; the first recipe has them.

LiteLLM#

In-process, any of LiteLLM's providers as the teacher:

import litellm, shad0w

complete = lambda messages: litellm.completion(model="anthropic/claude-haiku-4-5", messages=messages,
                                               temperature=0).choices[0].message.content
intent = shad0w.decision("intent", llm=shad0w.llm_teacher(["refund", "lost_card", "other"], question="intent", complete=complete))

Keep calling LiteLLM and let shad0w short-circuit certified answers. They come back as normal ModelResponse objects, through LiteLLM's mock_response:

import shad0w
from shad0w.integrations.litellm import completion

intent = shad0w.decision("intent", options=["refund", "lost_card", "other"])
resp = completion(intent, model="gpt-6-luna", messages=[{"role": "user", "content": "my card was stolen"}])

LiteLLM proxy. Register a callback in a file next to your config.yaml:

# shad0w_hooks.py
import shad0w
from shad0w.integrations.litellm import Shad0wLogger

handler = Shad0wLogger({"intent": shad0w.decision("intent", options=["refund", "lost_card", "other"])})

# config.yaml
litellm_settings:
  callbacks: shad0w_hooks.handler

A request is a decision when its metadata names the question, extra_body={"metadata": {"shad0w_question": "intent"}}, or when the handler has only one question. Shad0wLogger(shadows, question_of=fn) lets you pick the question from the request yourself. Answers from the model are logged for the next train(). These adapters are tested against stand-ins for LiteLLM's interfaces, not against a running LiteLLM proxy.

Or put the shad0w proxy in front of a LiteLLM proxy. It speaks OpenAI, so nothing else changes:

shad0w proxy --upstream http://localhost:4000 --port 8010

Vercel AI SDK#

Wrap the model: certified answers skip it, everything else is logged for training.

import { wrapLanguageModel, generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { decision, shad0wMiddleware } from "shad0wllm";

const intent = await decision("intent", { options: ["refund", "lost_card", "balance", "other"],
                                          bundle: "shad0w/intent/bundle", log: "shad0w/intent/log.jsonl" });
const model = wrapLanguageModel({ model: openai("gpt-6-luna"), middleware: shad0wMiddleware(intent, { specificationVersion: "v3" }) });

export async function POST(req: Request) {
  const { message } = await req.json();
  const { text } = await generateText({ model, system: "Answer with one of: refund, lost_card, balance, other.", prompt: message });
  return Response.json({ intent: text });
}

specificationVersion must match your ai major version (ai 5 → "v2", ai 6 → "v3", ai 7 → "v4"). With experimental_decide, use decisionModel(intent, { fallback: openai.decisionModel("gpt-6-luna") }) as the model: the table answers the questions it certifies and the fallback answers the rest. See JavaScript.

Pydantic AI and other decision-model clients#

Point the client's base URL at shad0w proxy. Requests to /v1/decisions (OpenAI Decisions API) and /v1/systemone (Jev, Kev, Laya) are recognised with no marking. See Decisions API & System One.

FastAPI#

import shad0w
from fastapi import FastAPI

app = FastAPI()
intent = shad0w.decision("intent", options=["refund", "lost_card", "balance", "other"], llm="openai/gpt-6-luna",
                         auto_train=2000)                  # retrain in the background every 2,000 new answers

@app.post("/route")
async def route(body: dict):
    d = await intent.adecide(body["text"])                 # never blocks the event loop on a table answer
    return {"intent": d.answer, "source": d.source, "why": d.why, "us": d.latency_us}

@app.get("/shad0w")
def stats():
    return intent.stats()

For Prometheus, share one shad0w.Metrics() across your deciders (metrics= keyword) and expose metrics.prometheus() on /metrics. See Monitoring.

Django / Flask#

Create the decider once, at import time (module level). It is thread-safe. Call intent(text) in your views.

Cloudflare Workers, Deno, Bun#

See JavaScript. The same decision() works everywhere fetch does.

Anything with an OpenAI client#

Run the proxy and set base_url. That covers Ruby, Go, Java, .NET, PHP and every other language with an OpenAI SDK.

Updated 2026-10-09Suggest an edit on GitHub View as Markdown