- Your application embeds a client library (for example the Python SDK) or shells out to the
agentserviceCLI for coding agents like Claude Code or Cursor. That code starts runs, supplies tool results when the model pauses, registers schemas, posts feedback, and otherwise drives the HTTP API. - The agent-service backend hosts the workflow runner. It stores the agents as a schema. For each run it walks the agent graph, calls models, records metrics and tool traces, applies validators, and—when you configure them—runs graders and other eval hooks tied to that run.
- Sudoiq platform agents (analyst / optimization agents) read those runs: telemetry, grader output, stored tests, and user or admin feedback. They can propose workflow or rubric changes; in the product, those proposals are typically performance-tested before you adopt them broadly—either by replaying against historical or fixed examples (backtest-style regression) or by running A/B experiments on live traffic.
agentservice CLI.
Example: one run, then evals and feedback
A minimal pattern: start a run from your app; the platform attaches task identity; you record use feedback on that task; optimization agents use that signal together with other runs. (Exact symbols match your installedasapi / agentserviceapi package; see Getting started.)
How it works
- Run agents — Start a run with inputs; wait for structured or free-text results.
- Add tools — Register tool schemas on the service and attach app-defined tools so your code runs when the model calls a tool.
- Tune quality — Attach tests and evaluations, collect feedback on runs, and feed that signal back into workflow improvements (see Evals, tests, and feedback).
- Iterate safely — Use versions, validators, and feedback loops to reduce regressions as graphs change.
Libraries and tools
- Python client (
asapiimport, oragentserviceapidepending on packaging) — SDK for runs, tools, validators, tests, graders, and feedback. agentservice(CLI) — Agent editing, execute, tasks, tools/validators push, and demos.- Node.js — SDK coming soon; see Node.js client.
Embedding in your app (Python / CLI)
- Python 3.10+ recommended for the CLI and examples.
- Dependencies (with the client):
httpx,pydantic(installed automatically).
Next steps
- Getting started — virtualenv, API key, first successful run.
- What is an agent?
- How runs work
- Python client — includes adding a tool.
- CLI
- Evals, tests, and feedback