General / Getting Started
What is Botscanner?
Botscanner is the home of our LLM Router (beta). It sends each prompt to the model that answers it best for the least money, so you get state-of-the-art quality at less than 10% of the cost of always calling the top frontier model. That is a saving of over 90%.
Use the router as a chat, or as an OpenAI-compatible API. The Playground is still there when you want to compare the answers themselves: one prompt, many LLMs, and the answers ranked by other LLMs.
Powered by
AutoBench.
Who is Botscanner for?
Anyone who uses LLMs and would rather not pay frontier prices on every prompt: developers, researchers, students, and teams shipping an AI product or an agent.
Use the router when you want the answer. Use the Playground when you want to read how several models answered the same prompt, and see those answers ranked by LLMs.
How do I get started?
Go on botscanner.ai and request an invitation code, then sign up. New accounts receive $1 in free credits. No credit card is required to start.
Open the router to chat, or create an API key and point your OpenAI client at https://api.autobench.org/v1. Free accounts start on the Cost knob. Quality and Balanced unlock at Tier 1.
The Botscanner Router
What is the Botscanner Router?
Botscanner Router is a router that reads each prompt and picks the model that is good enough to answer it, weighing quality, cost, and speed. Simple prompts go to a cheaper model and harder ones go to a stronger model, so you stop paying frontier prices for questions that don’t need them. You can see which model answered and what it cost.
It is the same product in two places: the chat on Botscanner, and an OpenAI-compatible API at https://api.autobench.org/v1. You choose a knob (Quality, Balanced, or Cost). The router chooses the model. Powered by
AutoBench.
How does the Router decide?
The Router decides in two steps.
1. A quick check for easy prompts. Short, clear prompts for simple math, code, or instructions go straight to a cheaper model, so an easy question never pays for a stronger one.
2. A trained router for everything else. It reads the prompt and predicts how well each of its three models (low, mid, and high cost) would answer it. It learned this from
AutoBench evaluations, where a committee of LLMs judges the answers, rather than one lab grading its own model. Your setting then decides how far up the cost ladder the router goes. Quality moves up to stronger models most readily, Cost stays cheaper, and Balanced sits in between. Each reply shows which model answered.
What are Quality, Balanced, and Cost?
Three settings, same router.
Quality aims for the best answers we can route.
Balanced is the default: the best trade-off of quality and cost.
Cost keeps the bill as low as we can while the answers still hold up.
Free accounts use Cost. Quality and Balanced unlock at Tier 1, once your credit purchases add up to $10. We update what sits behind each knob as better and cheaper models appear. The way you call it does not change.
Does the Router cut my costs without lowering answer quality?
The Botscanner Router is built to match state-of-the-art answers while you pay less than 10% of the cost of always calling the top frontier model. That is a saving of over 90% against that baseline.
The figure is a single-turn measurement on the
AutoBench router benchmark. A committee of LLM judges scores each answer, and we compare the routed answer with always calling the strongest model in the pool. Current numbers are on
autobench.org. We publish a new measurement when we update the router, so the page there is the live one.
Multi-turn chats and tool calls work. The published figure is the single-turn benchmark, not a promise that every prompt will land on that number. The model that answered, and what it cost, is shown on each reply.
Will the models change?
Yes. We update the router and the models it can choose as better and cheaper models appear. The way you use it stays the same: same chat, same API, same three knobs. Each answer names the model that produced it.
Can I choose the model myself?
Yes, you can choose the model yourself when you use the Playground. When you use the Router, the Botscanner Router will choose for you the best model for your prompt and will tell you which model answered.
Can I attach files in the Router?
Yes. The Router chat accepts text, Markdown, CSV, JSON, source code, PDF, and Word documents. Up to 5 files and 10 MB per turn. Images and audio are not supported. We extract the text and send it with your prompt.
Why is the Router called beta?
The Router is live and billed like the rest of Botscanner, and we are still improving it. The model pool and the routing policy will change. If something looks wrong, tell us from the Contact page.
API
How do I call the API?
Create a key on the API keys page, then point any OpenAI client at https://api.autobench.org/v1. The model id is the knob:
autobench/beta-quality, autobench/beta-balanced, or autobench/beta-cost.
from openai import OpenAI
client = OpenAI(base_url="https://api.autobench.org/v1", api_key="ab_live_YOUR_KEY")
client.chat.completions.create(
model="autobench/beta-balanced",
messages=[{"role": "user", "content": "..."}],
)
Is the API OpenAI-compatible?
Yes, for chat completions. You can pass temperature, max_tokens (or max_completion_tokens), stop, stream, tools, and tool_choice. Reasoning effort is locked, so the measured quality still holds. A request is rejected if it asks for more than one choice, logprobs, images, audio, or a specific model instead of a knob.
Are there rate limits and spend caps?
Each key is limited to 60 requests per minute. You can set a monthly spend cap on the key, either a preset or your own amount. The Usage page shows the month's tokens and cost.
Where do I see what the API spent?
The Usage page graphs tokens and cost for your keys. Router chat and API calls share one credit balance, and both show up in Billing.
Pricing & Credits
How does pricing work?
Botscanner is pay-as-you-go credits, in US dollars. One balance pays for the Router, the API, and the Playground.
What does the Router cost?
You pay the upstream model's cost plus a 50% markup. Choosing the route is free: you only pay for the answer. The chat and the API use the same price, taken from one credit balance.
Set against always calling the top frontier model, that bill is less than 10% of the cost. You save over 90%. The 50% is a markup on the cheap routed call, not a share of what the frontier model would have charged.
The Playground is separate. There you pay the answering models and the ranking models you picked, by the tokens they use, and you see the breakdown after each task.
What does the Playground cost?
In the Playground you pay for two jobs: the LLMs that write the answers, and the LLMs that rank those answers. The price follows the tokens they use. A cost breakdown is shown after each task, so you can see which model cost what.
It is the same credit balance as the router and the API. Comparing several answers is more expensive than one routed answer, because you asked for several.
Are there free credits to start?
Yes. New accounts receive $1 in free credits. No credit card is required to start.
How can I purchase more credits?
Open
Billing and pay with Stripe. Purchases are between $5 and $1,000 per transaction. You review the amount before you pay.
Your balance updates when Stripe confirms the payment, usually within moments. The same balance pays for the router, the API, and the Playground. You can also turn on auto-recharge from the billing page, so a payment is taken when your balance falls below a threshold you set.
What is Tier 1?
Tier 1 unlocks the Quality and Balanced knobs on the Router, and the higher models in the Playground. It turns on automatically once your credit purchases add up to $10. Free accounts use the Cost knob.
Playground: How It Works
How does the Playground work?
The Playground compares answers, not just model names. You ask once. Several LLMs answer. Other LLMs rank those answers.
1. Enter the question or prompt.
2. Select the answering LLMs, and the ranking LLMs that will judge the answers.
3. Botscanner sends your prompt to the answering models in parallel.
4. Each ranking model reads the prompt and the answers, and scores them. You get every answer, ordered from best to worst, with the cost and the token usage.
Use this when you want to see how different models actually responded. Use the router when you want one answer, already chosen.
How does the Playground rank the answers?
Ranking is done by LLMs, not by a fixed rubric we wrote by hand. You pick the ranking models. Each one reads your prompt and every answer, and scores accuracy, relevance, clarity, and how well the answer follows the prompt, on a simple 1 to 5 scale.
Botscanner combines those scores and shows the best answer first. You still see the others, in rank order, so a close second is not hidden. The rank is that committee's judgment of the answer, not a claim that one model is best at everything.
How fast is a Playground task?
Requests go out in parallel. Some models answer in seconds. Stronger models, especially reasoning models, can take minutes. Answers show up as they arrive, so you can read the best ones already on the page. Stalled tasks are monitored.
Playground: Models & Tiers
Which models does the Playground support?
A wide set of models from major providers. The Models page is the current list, including which tier each model needs and whether it can answer, rank, or both.
What are Tiers and how do they work?
Tiers control which Playground models you can select, and which Router knobs you can use. Free accounts use the Cost knob and a wide set of Playground models. Tier 1 unlocks Quality and Balanced on the Router, and the more demanding Playground models. Your tier rises with the credits you have purchased.
How do I know which tier a model belongs to?
The Models page lists the required tier for each model. Your current tier is on your profile.
Can all Playground models answer and rank?
Not always. Some are better at answering, some at ranking, and many do both. The Models page marks each model as answer, rank, or both.
Account Management
How do I manage my account?
After you log in you can open your profile, credit balance, tier, billing, API keys, usage, and support tickets.
Is my password secure?
Passwords are stored with a strong hash (Werkzeug). We never store them in plain text. API keys are shown once, when you create them, and only a hash is kept after that.
Features
What are Model Presets?
Presets are saved sets of answering and ranking models for the Playground. You can save your own (for example "My writing setup") or use the built-in ones. They load with one click and stay on your account. The Router does not use presets: it picks the model.
What if I need help?
Open Support and create a ticket. You can follow the conversation and the status on the site.
Is there a guide for new users?
The first time you open the Playground you can take a short tour of that page. You can skip it and start it again later from your profile.
Troubleshooting & Support
I paid for credits, but my balance hasn't updated. What should I do?
Stripe usually confirms a payment within moments. If the balance is still unchanged after a few minutes, open a support ticket with the time and amount. Never include your full card number. The same payment cannot be credited twice.
A Playground task is taking a very long time. What's happening?
Time depends on the models you picked and on the providers behind them. If a model fails or times out, that part is not charged. If it keeps happening, open a support ticket.
A Router answer is taking a long time. Is it stuck?
Some routed answers, especially on harder prompts, can take up to about three minutes. The page shows when the prompt has been routed and which model is answering. You can stop the turn; you are billed only for what was already produced. If it fails before an answer, you are not charged.
Where can I find answers to other questions?
This page covers the common ones. If yours is not here, open a support ticket or use the Contact page.
About AutoBench & Botscanner
Who is behind Botscanner?
Botscanner is a service of
AutoBench, a project of eZecute S.R.L. in Rome. AutoBench is the evaluation layer. The Botscanner Router is the product you use, powered by that layer.
The same app is at botscanner.ai and botscanner.autobench.org.
Is AutoBench a separate company?
Not yet.
AutoBench is a project of eZecute S.R.L. and is planned to become its own company. Until then, eZecute operates Botscanner and is the company named in the Terms of Service and the Privacy Policy.
Is the Router in beta?
Yes. The Botscanner Router is in beta: you can chat and call the API today, and we will keep updating the router and the models behind it. The knobs stay the same when the models change. If something breaks, tell us from the Contact page.
What is AutoBench?
AutoBench is the evaluation layer under Botscanner. A committee of LLMs scores each answer (Collective-LLM-as-a-Judge), so the measurement is not one lab grading its own model. Those judgments are the traces the Botscanner Router is trained on.
Public runs have evaluated 120+ models and produced 1.3M+ judgments. The benchmark methodology was independently validated with DIAG, La Sapienza University of Rome (
arXiv:2510.22593). Leaderboards and the current router numbers are on
autobench.org.
What is your vision?
The useful question is no longer "which model is best". It is "which model should answer this prompt". Agents need that choice on every step, and a model lab cannot honestly route traffic away from its own models.
AutoBench measures, as a neutral layer between models and applications. The Botscanner Router is that choice, cheap enough to run on every request, with the saving left mostly with you.