Applied Will · Training data for AI labs

RL environments, country by country.

Replicas of the apps people use in each country, with tasks graded so models can't cheat. Polish first.

What we build

Every task ships with the evidence.

Environments to train in, experts who check the work, and data from the companies that do it.

  • Environments

    RL environments

    Replicas of the apps people use in each country, packaged for your agent to train in. Every task carries its rollouts, grader attacks and blind expert solves.

    Polish first · from $200/task · pilots of 25

  • Expert data

    Expert data

    Two blind solves and a sealed review on every task. For construction, takeoffs drawn by a concrete estimator.

    74,750 sq ft of takeoffs hand-labeled by our founder

    Sample · demo run 8 Oct 2026

    What you get for one task

    nw-all-above-eight · PL

    Rollouts
    0/6 · agent never solved it
    Grader attacks
    10 tried · 0 got through
    Blind solves
    Solve 1 ✓ · Solve 2 ✓ Answers agree
    Sealed review
    Accepted
    Provenance
    2480e8…44ae6a

    Source: the unit's provenance manifest; rollouts from our first batch, same task.

  • Partnerships

    Data partnerships

    Companies license the records of how their work gets done, on terms they set. We turn them into environments and tasks for AI labs.

    You choose what's in, what's redacted, what never leaves

  • Audits

    Task audits

    Send tasks you already have. We attack each grader and solve every task blind.

    Free 10-task sample · web-shop tasks today

The problem

RL tasks fail quietly.

A leaked answer or a loose grader doesn't crash anything. The model just learns to cheat.

The lab's own check: under 2% flagged

An outside audit: 67% of its open-sourced coding tasks leaked the answer

Source: Vals AI, Oct 2026

How it works

Machines triage. Experts decide.

  1. 01

    Model

    Generate

    A model drafts the task and grader.

  2. 02

    Automated

    Roll out

    A cheap agent tries it six times.

  3. 03

    Automated

    Grader checks

    Accepts the answer, rejects doing nothing.

  4. 04

    Automated

    Grader attacks

    One working exploit rejects the task.

  5. 05

    Experts

    Two blind solves, sealed review

    Two experts solve blind. Verdicts stay sealed.

Research

What our runs show.

  • ConcreteTakeoff-Bench

    Footing counts on real drawings: text/OCR vs. vision models.

    Accuracy, dev split

    • Claude Haiku 5.5100%
    • Claude Sonnet 5.5100%
    • Text/OCR79%
    • Jev (TypeSafe)79%
    • Qwen3-VL-8B61%
  • Planted broken tasks

    3/3 caught before any person looked. Here's what caught each.

    Caught by

    • Loose graderGrader attacks
    • Wrong reference answerBlind solve
    • Ambiguous promptBlind solve
  • Our first batch

    23 tasks, scored one by one.

    Agent rollouts, 6 per task

    • Unsolved13
    • In range6
    • Too easy4

All research