nixmac E2E Report - settings_state_full_mac_journey
0 passed
1 assertion failed
0 infra/not-run
1/1 selected scenarios produced reports
Settings state real-Mac journey
settings_state_full_mac_journey
full-mac on macos-e2e (full-mac)
Real-Mac companion - Attached to Settings and provider tabs, Settings controls persistence
- Commit
33f37e696bb5d5f1217b458e722d756ffb3d1c64- Duration
- 368s
- Replay
tests/e2e/run.sh settings_state_full_mac_journey
Full-Mac lane: real macOS desktop automation with full-screen recording evidence.
What this checks
Runs the shipped app on a real Mac, opens Settings, records Preferences/API Keys/AI Models, and verifies representative Preferences mutations persist to the real app-support settings.json.
Coverage
- Configured prompt screen launches from persisted macOS app-support settings
- Preferences tab renders and representative confirmation toggles persist via UI interaction
- API Keys tab renders OpenRouter, OpenAI, Ollama, and vLLM settings from the seeded settings.json
- AI Models tab renders provider and limit controls from the seeded settings.json
- Uses polling against the real settings.json to avoid persistence races
- Publishes screenshots and a full-screen recording as adjacent real-desktop proof
Known gaps / not covered
- Real-Mac companion proof; WDIO remains the deterministic authority for exact select/input values and full settings-form mutation coverage.
- Only Preferences toggles are mutated through UI in this full-Mac journey; API Keys and AI Models are hydrated/recorded from seeded settings.
- Does not call live providers or validate external API credentials.
Failure: Preferences controls did not render
- What happened
- Preferences controls did not render
- Next action
- Open the full report and workflow logs for the failing phase, then rerun the replay command after fixing the cause.
| Phase | Status | Duration | Summary |
|---|---|---|---|
| Nix installed (already present) | passed | 0s | |
| Configured state seeded | passed | 0s | |
| Settings dialog launched | passed | 0s | |
| Preferences controls did not render | failed | 0s | Preferences controls did not render |