nixmac E2E Report - live_openrouter_full_mac_journey
0 passed
1 assertion failed
0 infra/not-run
1/1 selected scenarios produced reports
Live OpenRouter real-Mac journey
live_openrouter_full_mac_journey
full-mac on macos-e2e (full-mac)
Real-Mac companion - Attached to Live OpenRouter evolve smoke
- Commit
33f37e696bb5d5f1217b458e722d756ffb3d1c64- Duration
- 411s
- Replay
tests/e2e/run.sh live_openrouter_full_mac_journey
Full-Mac lane: real macOS desktop automation with full-screen recording evidence.
What this checks
Runs the shipped app on a real Mac with a real OpenRouter key, submits a constrained prompt, and verifies a live-model diff reaches review without applying.
Coverage
- OpenRouter key/model are seeded into the released app settings on the Mac runner
- Prompt submission calls a live OpenRouter-backed provider path
- Live model/tool-call loop reaches evolve review
- Temporary config repo diff includes pkgs.jq in flake.nix
- Flow stops at review without applying or rebuilding
- Publishes screenshots and a full-screen recording as adjacent real-desktop proof
Known gaps / not covered
- Calls a live model and can fail for provider outages, rate limits, account credit, or prompt nondeterminism.
- Stops at evolve review; it does not apply, rebuild, or verify final macOS state.
Failure: Live OpenRouter diff did not contain pkgs.jq
- What happened
- Live OpenRouter diff did not contain pkgs.jq
- Next action
- Open the full report and workflow logs for the failing phase, then rerun the replay command after fixing the cause.
| Phase | Status | Duration | Summary |
|---|---|---|---|
| Nix installed (already present) | passed | 0s | |
| Live provider settings seeded | passed | 0s | |
| Live OpenRouter diff did not contain pkgs.jq | failed | 0s | Live OpenRouter diff did not contain pkgs.jq |