Fine-tuned example

Trained on 2,183 refined examples in about 4 minutes

1,331 headline labels + 592 maths answers + 260 right-vs-wrong pairs, 1 epoch on one L4 GPU. Llama 3.2 1B, no human labels, tested on questions it never saw.

base model (untrained)trained on refined data

Average across 6 unseen sentiment datasets (Yelp, Amazon, SST-2, IMDB, PhraseBank, TweetEval)

Maths · 300 GSM-Symbolic problems

News sentiment · 300 human-labelled headlines

Negation · 200 trick sentences like "not bad at all"

Words like "not" flip the meaning; small models often miss it. Base got 3 examples to help it.

592refined maths answers

260right-vs-wrong pairs

1 epoch~4 min of training

News: trained on 1,331 refined headline labels; much of the jump is the model learning to answer with a clear label.

That small amount of refined data moved a model Meta trained on trillions of words. Same model, same 300 unseen problems. Our product is the refined dataset; this fine-tune is one example of what it does.

Six unseen sentiment datasets

Trained only on refined financial headlines, then tested on other people's text it never saw.

base modelfine-tuned on refined data

Why refined data beats regular data

Regular datasets carry hidden mistakes, and models learn them. The refinery removes them.

+3.6points: refined vs raw data, same model and problems (Llama 8B)

+4.0points: refined vs GSM8K's own human-written answers (Llama 8B)

30% → 2%wrong labels before and after refining (Error Detection Test)

Even well-known benchmark datasets contain wrong labels (MIT, "Pervasive Label Errors", 2021); scraped data is worse. Cleaning by hand costs ₹10–50+ per item; the refinery does it for about ₹2–6 with volunteer GPUs. Gains are largest on messy data and grow with stronger voter models.