Scope: can an agent
use what you made?
You publish a small library that supplies weather data. Can a coding agent find its instructions and use it correctly? Scope is Microsoft’s evaluation platform for studying that sort of task. You choose a task and success criteria; it records agent runs so you can inspect the steps behind a result. [27] [29]
Make success visible
For our imagined library, supply a fictional forecast and ask for a display. Check whether the agent imports the right function, handles a missing temperature and produces the expected screen. A failed step may point to an unclear example or error message in your own software. This is an illustrative assignment.
What its setup involves
Scope describes workers packaged in containers: software environments holding the tools they need. Kubernetes manages those workers across machines. That infrastructure supports running experiments at scale; deploying it is a separate task from choosing a good question. We have read the documentation and have yet to deploy Scope. [27]