nixmac E2E Report - provider_resilience_full_mac_journey
1 passed
0 assertion failed
0 infra/not-run
1/1 selected scenarios produced reports
Provider resilience real-Mac journey
provider_resilience_full_mac_journey
full-mac on macos-e2e (full-mac)
Real-Mac companion - Attached to Provider validation blocks prompt, Provider failure recovery
- Commit
33f37e696bb5d5f1217b458e722d756ffb3d1c64- Duration
- 257s
- Replay
tests/e2e/run.sh provider_resilience_full_mac_journey
Full-Mac lane: real macOS desktop automation with full-screen recording evidence.
What this checks
Runs the shipped app on a real Mac through invalid vLLM configuration and mock provider failure states, recording visible recovery/error behavior.
Coverage
- vLLM selected without a base URL renders provider validation before submission
- Prompt suggestion can fill the input while provider validation remains visible
- Open AI Models settings recovery copy is visible
- Mock provider billing/credits failure surfaces visibly in the widget
- Reads settings.json to prove which provider configuration each phase used
- Publishes screenshots and a full-screen recording as adjacent real-desktop proof
Known gaps / not covered
- Uses deterministic local provider states; it does not enumerate every live provider failure shape.
- Does not call external providers.
View additional proof artifacts (4)
| Phase | Status | Duration | Summary |
|---|---|---|---|
| Nix installed (already present) | passed | 0s | |
| Provider validation block verified | passed | 0s | |
| Provider failure recovery verified | passed | 0s |