Use fixed answers
Checked-in test data gives a valid agent or user answer for each step.
FIRST-PARTY TESTS · METHOD 1.1
Latest verified result · August 8, 2026
40/40
Every owned skill workflow test finished with the expected result.
What this result says
Yield ran the exact workflow steps.
These tests cover the part Yield controls: running code, keeping step order, saving results, and resuming. They do not claim Yield is better than prompts or judge whether a model’s answer is true.
40 / 40skill workflow tests passed
10 × 4patterns × supported SDKs
8 / 8runtime checks passed
0third-party comparisons
01 / Skill workflow tests
The same ten flows. Four SDKs.
Each PASS is a real yskill test run. The suite gives fixed answers for agent and user steps. Yield still runs command steps itself.
| Owned skill workflow | TypeScript | Python | Go | Rust |
|---|---|---|---|---|
| Review a branch | PASS | PASS | PASS | PASS |
| Investigate a failure | PASS | PASS | PASS | PASS |
| QA a web change | PASS | PASS | PASS | PASS |
| Release a package | PASS | PASS | PASS | PASS |
| Triage an issue | PASS | PASS | PASS | PASS |
| Repair CI | PASS | PASS | PASS | PASS |
| Upgrade a dependency | PASS | PASS | PASS | PASS |
| Migrate a database | PASS | PASS | PASS | PASS |
| Audit a security change | PASS | PASS | PASS | PASS |
| Publish an iOS build | PASS | PASS | PASS | PASS |
02 / Runtime
Eight runtime checks.
These checks cover responses, replay, choices, changed code, and failed rules.
- 01Resume and finish
A saved answer moves the run to its expected final result.
- 02Commit one response
Concurrent identical responses create one completion event.
- 03Recover an exact retry
The same committed response recovers; different content is refused.
- 04Keep answers inside choices
Every SDK rejects a value outside the options declared by the workflow.
- 05Repeat the same run
The saved history produces the same next step.
- 06Stop on changed behavior
If the workflow changed, Yield stops instead of using the wrong saved result.
- 07Stop on a failed rule
A failed check ends as blocked and saves the reason.
- 08Handle changed code
New workflow code is refused until the user accepts the change.
03 / Method
Test the code on its own.
The suite uses fixed model answers. If a test fails, the problem is in the SDK, runtime, protocol, or workflow code.
Run the workflow
yskill test runs commands, submits each answer, and requires the expected final result.
Check the runtime
Focused tests cover concurrent responses, exact retries, choices, replay, blocked runs, and changed code.
04 / What it proves
A useful result with a clear limit.
We show what the tests cover and what they do not cover.
This proves
- All four SDKs can run the ten tested workflows.
- Yield runs commands, keeps step order, saves state, checks rules, and records the final result in these tests.
- The tested runtime stops when behavior changes or a rule fails.
This does not prove
- Yield is better than prose or any other tool.
- An agent’s diagnosis, review, or recommendation is correct.
- The example commands are safe for every production repository.
Run it yourself.
The published result is accepted only when the tests run again with the same source hash and test list.
git clone https://github.com/operatorstack/yield.git
cd yield/evals
git checkout 66decd17098acf2e16e90cb7286d7f6949d4c158
npm ci
npm testExact first-party test source
Source hash
9a3f7e52d3321fc41bd5792f185db8d9aac235f4031cc1d0651bb60e8de076c1Yield commit
66decd17098acf2e16e90cb7286d7f6949d4c158Generated . The hash covers the CLI, runtime, protocol, SDKs, skill workflows, test data, and test script. It does not include build caches or independently test agent judgment.