The tool-caller that knows when not to call.
A function-calling model fine-tuned with AutoScientist. It refuses, clarifies, and calls — instead of hallucinating tools. Trained on the exact failure modes every other dataset ignores.
pip install git+https://github.com/ankit25bcs10610/adaption-ai-lab.git- +0.0%
- dataset quality · Adaptive Data
- 0
- abstain cases · recovered from 8
- 0B
- base · Qwen2.5-Coder
Measured — Adaptive Data quality grade (C→B) and the audited dataset composition.
// user has no matching tool
"Write me a poem about the monsoon."
"action": "refuse",
"message": "No available tool
can handle this request."
}
No hallucinated call. This is the win.
the core idea
Three behaviors. One JSON envelope.
Most datasets only teach a model to call tools. Ours teaches it to call, refuse, and clarify — the decisions that separate a reliable agent from a confident liar.
Call
When a tool fits and all args are present, emit a schema-valid call.
{"action":"call",
"calls":[{"name":
"get_weather",
"arguments":{"city":
"Mumbai"}}]}Refuse
When no available tool can help, decline — never invent one.
{"action":"refuse",
"message":"No tool
can satisfy this
request."}Clarify
When a required arg is missing or the choice is ambiguous, ask.
{"action":"clarify",
"message":"Which city?
I need it before I
can proceed."}the moat
Hard negatives
~40% of the training set is cases where the right answer is not a plain tool call — refuse, clarify, disambiguate, resist over-refusal, or complete every call. That's exactly where baselines score ~50%, and where we win.
no_tool
nothing applies → refuse
missing_arg
required arg absent → clarify
ambiguous
two tools fit → clarify
over_refusal
hedged but doable → still call
partial_parallel
two intents → two calls
try it live
Watch it decide, in your browser.
Toggle a tool off and re-run — a valid call becomes a refusal. That's the whole point: it won't invent a tool it doesn't have.
Available tools
Try a scenario
Model output
This playground runs a faithful, deterministic simulation of the model's decision logic in your browser (no download). The released weights produce the same JSON envelope.
try it · voice + agent
Tell the agent what to do — by voice or text.
A live agent that plans, calls the right tool, and refuses or asks when it shouldn’t guess — the exact discipline this dataset teaches. Runs in your browser (no key, no server); upgrades to a real Claude-powered agent when an API key is configured. Tap the mic to speak; toggle for spoken replies.
the pipeline
A data-centric loop, end to end.
AutoScientist automates training — so the whole game is the data. Every stage is reproducible, seeded, and one command (scripts/run_all.sh).
- 01
Build the dataset
Curate ToolACE to schema-valid positives, synthesize hard negatives + execution-verified env examples.
- 02
Dedup + decontaminate
Slice-aware MinHash + semantic dedup, cross-split leakage removed, and a decontamination pass vs public probes.
- 03
Honest baseline
Eval the raw base model first — identical greedy decoding, so the comparison is fair.
- 04
Train on AutoScientist
Upload, co-optimize data + recipe, get grade_before / grade_after / improvement (real: +15.7%, C→B).
- 05
DPO on the moat
Preference pairs: chosen = verified refuse/clarify/call, rejected = the hardest confirmed-wrong near-miss.
- 06
Release + demo
Manifest + preflight, then publish to HF + Kaggle with an auto-filled model card.
results
Measured wins, honest targets.
The dataset is the product, so we lead with what the platform actually measured — and clearly mark what still needs a training run.
Dataset-quality grade
A data-centric platform grades the dataset. This is the improvement it reported.
The data-quality audit
What a bug hunt on the build pipeline found — and fixed. The moat had nearly shipped empty.
- Refuse cases (no_tool)8→239the moat was starved by a dedup bug
- Clarify cases (miss_param)1→36generator selected argless tools
- Disambiguate cases0→133templated queries collapsed under dedup
- Schema-invalid gold calls36%→0%type/enum-aware value synthesis
Base vs. fine-tuned model
Illustrative targets for the behavior the moat trains. Same greedy decoding, bootstrapped SE.
Illustrative targets — the model table becomes real after training on the improved dataset (a GPU step). The one-command harness (scripts/run_all.sh) runs baseline → multi-seed eval → paired significance → gap decomposition → report.
second track · data visualization
Charts, read in English and Hindi.
A multimodal chart-understanding model trained on a self-verifying synthetic dataset — every answer computed from the underlying data, so the number on the chart matches the gold by construction. Plus a Devanagari + romanized slice for the HackIndia track, and a text-only Vega-Lite spec-reading modality.
- Correct by constructionGround-truth is computed from the data, not labeled — no noise to cap accuracy.
- Hindi + romanizedSame ground-truth, localized labels & questions; paired en/hi twins for a matched-pair Δ.
- Two modalitiesRendered images and text-only Vega-Lite specs — chart comprehension with and without a VLM.
- Wide, measurable gapOn CharXiv, GPT-4o scores ~47% on reasoning vs ~80% human — room to close.
open release
Weights, data, and demo. All open.
Everything is public — model card, dataset card, eval harness, and a live Space. Reproduce it, fork it, beat it.