Environments
Real jobs packaged as sandboxes with a grader, for model labs to train and evaluate on.
What a pack holds
Everything a lab needs to run the tasks at scale, and nothing it has to rebuild.
TasksReal jobs, with their data and files
SandboxA container definition with read-only files
ToolsShell, file read and edit, as the task needs
GraderLine by line, by code wherever possible
SplitsA training set and a held-out evaluation set
LicenceNon-exclusive, or exclusive by agreement
Why these tasks
They come from the domains the group’s products work in.
Each pack comes from a domain a product in the Ambitious Machines group works in: pay transparency, discovery review, company records. The tasks are written by the people who build those products, and every answer is checked by code before a task ships.