A real 77-intent shad0w,
deciding as you type.
This shad0w watched a small open model (bge-small, 68% accurate here) answer the public BANKING77 dataset, learned to answer like it, and was certified at α = 5% on 3,000 answers it never trained on. Everything runs in this tab: no server, no network after load.
Loading the model…
The certificate, in plain words
- loading…
The promise is about agreeing with the model shad0w learned from (here an open model that is 68% accurate), not about the truth. In production, your own LLM is the one it learns from.
Replay real traffic
Stream 800 held-out BANKING77 test messages through shad0w. Messages it is sure of are answered here; the rest would go to the LLM. These 800 are a random sample of the 3,080 test messages, so the rates move a little around the figures above. On a sample this small, a disagreement a bit over α now and then is expected; the promise is about the long run.
Build your own: Getting started · GitHub