# shad0w

> shad0w (pip install shad0wllm, npm i shad0wllm) learns the fixed-choice decisions an application sends to an LLM
> (intents, routing, moderation, agent tool choice), and answers the ones it can certify from a small table on the CPU in
> microseconds. Every other request still goes to the LLM, and its answer is logged so the table keeps learning. The
> certificate (Learn-then-Test with Clopper-Pearson bounds) states that, among served answers, disagreement with the LLM
> is at most alpha (default 5%) with 90% confidence. It works in front of chat models, the OpenAI Decisions API and System
> One decision models (Jev, Kev, Laya), from Python, JavaScript, C or HTTP, or with zero code through an OpenAI-compatible
> proxy. Apache-2.0, by Gaurav Chauhan.

## Docs
- [Get started in 5 minutes](https://2796gaurav.github.io/shad0w/docs/): Markdown at https://2796gaurav.github.io/shad0w/docs/index.md
- [Decisions API & System One (zero code)](https://2796gaurav.github.io/shad0w/docs/decisions-api.html): Markdown at https://2796gaurav.github.io/shad0w/docs/decisions-api.md
- [Connect your LLM](https://2796gaurav.github.io/shad0w/docs/connect-your-llm.html): Markdown at https://2796gaurav.github.io/shad0w/docs/connect-your-llm.md
- [The proxy](https://2796gaurav.github.io/shad0w/docs/proxy.html): Markdown at https://2796gaurav.github.io/shad0w/docs/proxy.md
- [Python](https://2796gaurav.github.io/shad0w/docs/python.html): Markdown at https://2796gaurav.github.io/shad0w/docs/python.md
- [JavaScript & TypeScript](https://2796gaurav.github.io/shad0w/docs/javascript.html): Markdown at https://2796gaurav.github.io/shad0w/docs/javascript.md
- [HTTP](https://2796gaurav.github.io/shad0w/docs/http.html): Markdown at https://2796gaurav.github.io/shad0w/docs/http.md
- [C & other languages](https://2796gaurav.github.io/shad0w/docs/c.html): Markdown at https://2796gaurav.github.io/shad0w/docs/c.md
- [LangChain, LiteLLM, AI SDK…](https://2796gaurav.github.io/shad0w/docs/frameworks.html): Markdown at https://2796gaurav.github.io/shad0w/docs/frameworks.md
- [Configuration](https://2796gaurav.github.io/shad0w/docs/configuration.html): Markdown at https://2796gaurav.github.io/shad0w/docs/configuration.md
- [Tuning & fine-tuning](https://2796gaurav.github.io/shad0w/docs/tuning.html): Markdown at https://2796gaurav.github.io/shad0w/docs/tuning.md
- [Changing your options](https://2796gaurav.github.io/shad0w/docs/options.html): Markdown at https://2796gaurav.github.io/shad0w/docs/options.md
- [Running without an LLM](https://2796gaurav.github.io/shad0w/docs/without-llm.html): Markdown at https://2796gaurav.github.io/shad0w/docs/without-llm.md
- [Rollout: shadow → canary → serve](https://2796gaurav.github.io/shad0w/docs/rollout.html): Markdown at https://2796gaurav.github.io/shad0w/docs/rollout.md
- [Monitoring](https://2796gaurav.github.io/shad0w/docs/observability.html): Markdown at https://2796gaurav.github.io/shad0w/docs/observability.md
- [Training & retraining](https://2796gaurav.github.io/shad0w/docs/training.html): Markdown at https://2796gaurav.github.io/shad0w/docs/training.md
- [Command line](https://2796gaurav.github.io/shad0w/docs/cli.html): Markdown at https://2796gaurav.github.io/shad0w/docs/cli.md
- [Troubleshooting](https://2796gaurav.github.io/shad0w/docs/troubleshooting.html): Markdown at https://2796gaurav.github.io/shad0w/docs/troubleshooting.md
- [The guarantee](https://2796gaurav.github.io/shad0w/docs/guarantee.html): Markdown at https://2796gaurav.github.io/shad0w/docs/guarantee.md
- [Scale: options, size, speed](https://2796gaurav.github.io/shad0w/docs/scale.html): Markdown at https://2796gaurav.github.io/shad0w/docs/scale.md
- [Benchmarks & evidence](https://2796gaurav.github.io/shad0w/docs/benchmarks.html): Markdown at https://2796gaurav.github.io/shad0w/docs/benchmarks.md
- [How it compares](https://2796gaurav.github.io/shad0w/docs/compare.html): Markdown at https://2796gaurav.github.io/shad0w/docs/compare.md
- [Limits](https://2796gaurav.github.io/shad0w/docs/limits.html): Markdown at https://2796gaurav.github.io/shad0w/docs/limits.md
- [FAQ](https://2796gaurav.github.io/shad0w/docs/faq.html): Markdown at https://2796gaurav.github.io/shad0w/docs/faq.md
- [Reference](https://2796gaurav.github.io/shad0w/docs/reference.html): Markdown at https://2796gaurav.github.io/shad0w/docs/reference.md
- [Changelog](https://2796gaurav.github.io/shad0w/docs/changelog.html): Markdown at https://2796gaurav.github.io/shad0w/docs/changelog.md

## Blog
- [Zero-code System Zero for the OpenAI Decisions API](https://2796gaurav.github.io/shad0w/blog/decisions-api-proxy.html): Put a certified reflex in front of client.decisions.create with three commands and no code change. Shadow mode first, the dashboard, the agreement check, the cost maths and the limits.
- [Snake at 120 moves a second: System 0 vs System One models, at game speed](https://2796gaurav.github.io/shad0w/blog/game-speed.html): shad0w learned snake and Pong from an LLM's answers. Then it raced three System One decision models on an Apple M5 Pro's GPU and a hosted LLM through the same games, with every decision timed. Here is what happened, and the Pong you can play.
- [94.3%, 51.3%, 0%: honest numbers from shadowing a hosted LLM](https://2796gaurav.github.io/shad0w/blog/honest-numbers.html): We put shad0w in front of a hosted LLM through the official OpenAI client and measured how much shad0w answered, how often it agreed with the LLM, how fast, and where it answered nothing at all.
- [Adding a category without breaking production](https://2796gaurav.github.io/shad0w/blog/adding-a-category.html): A trained shortcut and a changing list of options are a dangerous mix. The rule shad0w follows, and what happens when you add, remove or rename an option.
- [System Zero: the reflex layer below your LLM](https://2796gaurav.github.io/shad0w/blog/system-zero.html): shad0w is the shadow of your LLM. It watches your LLM, learns to answer like it, and takes the repeat traffic, so the LLM only gets what shad0w is not sure about. What System Zero is, why a cache is not enough, and where shad0w stays silent.
- [How shad0w proves it agrees with your LLM](https://2796gaurav.github.io/shad0w/blog/how-the-certificate-works.html): The certificate behind shad0w, in plain words: a held-out test, a confidence line, an allowance for bad luck, and what the promise does and does not cover.

## Watch and play
- [How shad0w works, in two minutes (film)](https://2796gaurav.github.io/shad0w/#film): MP4 at https://2796gaurav.github.io/shad0w/assets/shad0w-film.mp4
- [Snake arena](https://2796gaurav.github.io/shad0w/arena/): shad0w, small decision models and a hosted LLM play the same snake games at game speed
- [Play Pong against shad0w](https://2796gaurav.github.io/shad0w/pong/) · [Playground: a 77-intent table in your browser](https://2796gaurav.github.io/shad0w/demo/)

## Author
- Gaurav Chauhan · [LinkedIn](https://www.linkedin.com/in/gauravc2708/) · [GitHub](https://github.com/2796gaurav) · gauravc2708@gmail.com

## Code
- [GitHub](https://github.com/2796gaurav/shad0w)
- [PyPI: shad0wllm](https://pypi.org/project/shad0wllm/)
- [npm: shad0wllm](https://www.npmjs.com/package/shad0wllm)

## Optional
- [All docs in one Markdown file](https://2796gaurav.github.io/shad0w/llms-full.txt)