Compare commits

..
Author SHA1 Message Date
Bailey Dixon 447ec356d7 Merge pull request #359 from Codename-11/dev
release(android): android-v1.9.0
2026-08-14 21:32:50 -04:00
Bailey Dixon ab359e5efb release(android): android-v1.9.0 2026-08-14 21:01:51 -04:00
Bailey Dixon 86b8161cb7 feat(android): refine sessions and messaging 2026-08-14 20:58:31 -04:00
Bailey Dixon e401fdb0c7 fix(android): persist landed message reactions 2026-08-14 17:04:54 -04:00
Bailey Dixon d4325d5aab fix(android): restore compact chat interactions 2026-08-14 16:57:19 -04:00
Bailey Dixon c734d75484 feat(android): add desktop-style session profiles 2026-08-14 16:57:18 -04:00
Bailey Dixon 0411804780 fix(android): accept compatible session flags 2026-08-14 16:57:18 -04:00
Bailey Dixon 8f89c2841d fix(android): keep dashboard teardown off main thread 2026-08-14 16:57:18 -04:00
Bailey Dixon 933c1842a0 Merge pull request #358 from Codename-11/chore/backmerge-desktop-beta3
chore: back-merge Desktop beta.3 release
2026-08-14 16:31:24 -04:00
Bailey Dixon dfe1b53327 chore: back-merge desktop beta.3 release 2026-08-14 16:30:37 -04:00
Bailey Dixon d36580a983 Merge pull request #356 from Codename-11/dev
release: Desktop beta.3 process-containment patch
2026-08-14 16:11:35 -04:00
Bailey Dixon 2d178ff884 Merge pull request #357 from Codename-11/release/desktop-0.4.0-beta.3
fix(desktop): close process containment race
2026-08-14 16:05:00 -04:00
Bailey Dixon d61955f82a fix(desktop): close process containment race 2026-08-14 15:58:57 -04:00
Bailey Dixon adec6ed2c0 Merge pull request #355 from Codename-11/release/desktop-0.4.0-beta.3
release(desktop): desktop-v0.4.0-beta.3
2026-08-14 15:44:24 -04:00
Bailey Dixon e9e44c8bb2 release(desktop): desktop-v0.4.0-beta.3 2026-08-14 15:36:53 -04:00
Bailey Dixon c6b0732a02 Merge pull request #354 from Codename-11/fix/desktop-tray-process-containment
fix(desktop): contain tray subprocess storms
2026-08-14 15:22:21 -04:00
Bailey Dixon d919115788 fix(desktop): contain tray subprocess storms 2026-08-14 15:15:12 -04:00
Bailey Dixon 49da085ae8 Merge pull request #353 from Codename-11/chore/backmerge-desktop-beta2-server-1.8.0
chore: backmerge Desktop beta.2 and Server 1.8.0 releases
2026-08-14 15:05:21 -04:00
Bailey Dixon 889b6f0858 chore: merge desktop beta.2 and server 1.8.0 release history into dev 2026-08-14 15:05:12 -04:00
Bailey Dixon 5100c524f6 Merge pull request #351 from Codename-11/dev
release: Desktop beta.2 and Server 1.8.0
2026-08-14 15:04:03 -04:00
Bailey Dixon c609ae867f Merge pull request #352 from Codename-11/chore/backmerge-desktop-beta1
chore: back-merge Desktop beta.1 release history
2026-08-14 14:49:23 -04:00
Bailey Dixon 107f8c7720 chore: merge desktop beta.1 release history into dev 2026-08-14 14:49:08 -04:00
Bailey Dixon 8963e4fafd Merge pull request #350 from Codename-11/release/server-1.8.0
release(server): server-v1.8.0
2026-08-14 14:46:57 -04:00
Bailey Dixon 5d9624e2ef release(server): server-v1.8.0 2026-08-14 14:35:01 -04:00
Bailey Dixon 4b063d3fd7 Merge pull request #349 from Codename-11/release/desktop-0.4.0-beta.2
release(desktop): desktop-v0.4.0-beta.2
2026-08-14 12:56:04 -04:00
Bailey Dixon 9b9d7b654c release(desktop): desktop-v0.4.0-beta.2 2026-08-14 12:45:56 -04:00
Bailey Dixon a26e17e72c Merge pull request #348 from Codename-11/fix/cua-windows-health-compat
feat(desktop): enhance activity and control diagnostics
2026-08-14 12:38:12 -04:00
Bailey Dixon a3a6a9bb13 fix(desktop): satisfy tray release lint 2026-08-14 12:36:46 -04:00
Bailey Dixon 169bd09559 merge: sync desktop activity work with dev
# Conflicts:
#	CHANGELOG.md
2026-08-14 11:32:56 -04:00
Bailey Dixon 45d631e7ac feat(desktop): enhance activity and control diagnostics 2026-08-14 11:30:36 -04:00
Bailey Dixon eb6a6c95d2 merge: integrate official desktop relay plugin 2026-08-14 07:58:10 -04:00
Bailey Dixon 3b102663c2 feat(plugin): add official desktop relay surface 2026-08-14 07:56:26 -04:00
Bailey Dixon 0e5fc4c606 Merge branch 'fix/android-proactive-thread-entry' into dev 2026-08-14 07:50:07 -04:00
Bailey Dixon c3189f2cbb fix(android): open proactive messages as threads 2026-08-14 07:49:39 -04:00
Bailey Dixon 0d6c3bd6b0 Merge pull request #346 from Codename-11/dev
release(desktop): desktop-v0.4.0-beta.1
2026-08-13 21:18:41 -04:00
Bailey Dixon 06d88ad40a Merge pull request #345 from Codename-11/release/desktop-0.4.0-beta.1
release(desktop): desktop-v0.4.0-beta.1
2026-08-13 21:11:57 -04:00
Bailey Dixon 8ae5b3fbc2 release(desktop): desktop-v0.4.0-beta.1 2026-08-13 21:04:07 -04:00
Bailey Dixon 39b7a8f108 Merge pull request #344 from Codename-11/fix/desktop-updater-cua-hardening
feat(desktop): adopt CUA as primary control backend
2026-08-13 20:57:50 -04:00
Bailey Dixon af6e167692 fix(desktop): normalize Windows installer paths 2026-08-13 20:51:17 -04:00
Bailey Dixon 274bd6ae98 fix(desktop): honor CUA health schema 2026-08-13 20:45:58 -04:00
Bailey Dixon 9fc55b379a fix(desktop): clarify CUA readiness fallback 2026-08-13 20:34:45 -04:00
Bailey Dixon 559a0ffdc8 feat(desktop): make CUA the primary control backend 2026-08-13 20:16:03 -04:00
Bailey Dixon 75bcd9180f feat(desktop): add optional CUA control engine 2026-08-13 19:33:11 -04:00
Bailey Dixon 9c995a443d fix(desktop): harden bundle updates 2026-08-13 19:06:25 -04:00
129 changed files with 11692 additions and 731 deletions
+37 -2
View File
@@ -80,7 +80,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: '1.3.x'
bun-version-file: 'desktop/.bun-version'
- name: Install deps
run: npm ci
@@ -153,6 +153,40 @@ jobs:
desktop/dist/bin/hermes-relay-darwin-arm64
retention-days: 7
smoke-windows-cli-release-asset:
name: Smoke exact Windows CLI release asset
runs-on: windows-latest
needs:
- validate-release
- build-cli-binaries
steps:
- uses: actions/download-artifact@v8
with:
name: cli-binaries
path: release-assets
- name: Repeated launch and process cleanup gate
shell: pwsh
env:
EXPECTED_DESKTOP_VERSION: ${{ needs.validate-release.outputs.version }}
run: |
$ErrorActionPreference = 'Stop'
$exe = (Resolve-Path 'release-assets/hermes-relay-win-x64.exe').Path
1..20 | ForEach-Object {
$output = & $exe --version
if ($LASTEXITCODE -ne 0) { throw "Windows CLI smoke failed with exit $LASTEXITCODE" }
if ($output -ne "hermes-relay $env:EXPECTED_DESKTOP_VERSION") {
throw "Unexpected Windows CLI version output: $output"
}
}
Start-Sleep -Milliseconds 500
$leftovers = Get-CimInstance Win32_Process | Where-Object {
$_.ExecutablePath -eq $exe
}
if ($leftovers) {
throw "Windows CLI smoke left $(@($leftovers).Count) process(es) behind"
}
build-windows-tray-installer:
name: Build Windows tray installer
runs-on: windows-latest
@@ -175,7 +209,7 @@ jobs:
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: '1.3.x'
bun-version-file: 'desktop/.bun-version'
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable
@@ -375,6 +409,7 @@ jobs:
runs-on: ubuntu-latest
needs:
- build-cli-binaries
- smoke-windows-cli-release-asset
- build-windows-tray-installer
steps:
# Needed so CLI_RELEASE_NOTES.md is available to render into the release body
+42
View File
@@ -6,19 +6,61 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
## [Unreleased]
## [1.9.0] - 2026-08-14
### Added
- **Android session browsing matches Hermes Desktop's recent organization model.** The primary session drawer can toggle between the active profile and all profiles, group by recency, project, status, or profile, order by supported session metrics, and narrow rows by status, project, profile, or pull-request state without collapsing duplicate IDs across profile stores. Named profiles receive stable identity-color badges with locally persisted color overrides.
- **Android can edit current Hermes profiles through the standard Gateway.** The Profile Inspector capability-gates `profiles.describe` and `profiles.configure`, keeps Relay-only memory editing and older-Hermes fallback intact, and reports partial section saves without discarding failed drafts.
- **Android sessions show their coding context when Hermes supplies it.** Session rows can display repository, Git branch, and the current state of the pull request created by that session while older hosts remain unchanged.
- Android Manage can now finish host-owned backup workflows, edit or remove learning nodes with explicit recovery guidance, configure and activate memory providers, and complete profile-scoped WhatsApp QR onboarding through the authenticated upstream Dashboard contracts.
### Fixed
- **Android network clients shut down safely during route changes.** Replacing an authenticated Dashboard client now moves OkHttp connection-pool eviction off the main thread, preventing a live TLS socket close from crashing the app with `NetworkOnMainThreadException`. (#334)
- **Android preserves authoritative Gateway outcomes.** Protected-file cards cannot offer forbidden persistent scopes, compression no-ops show the server result, bounded resume failures do not create context-free replacement sessions, and edit/regenerate retains durable row identities across consecutive rewinds.
- **Android routes and uploads against live upstream truth.** Multiplex API fallback trusts `served_profiles` instead of installed profiles, and generic documents carry the Gateway-issued `@file:` reference into ordinary and queued prompts.
- **Android clarify cards preserve upstream decision semantics.** Multi-select prompts keep independent selections and submit one exact list, while server expiry events—not an invented local deadline—retire unanswered cards.
- **Android keeps profile management and retained automation truthful.** Custom Endpoint list and mutation routes now follow the selected Hermes profile, while completed one-shot cron jobs show their retained outcome and expose only valid Runs/Delete actions.
- **Android and Relay recover more generated media reliably.** Android accepts upstream-valid wrapped, punctuated, adjacent, spaced, and Windows `MEDIA:` markers without consuming fenced examples, and Relay translates Docker-visible workspace, home, cache, and configured-mount paths before applying its existing credential, sandbox, and size checks.
- **Android keeps cross-profile sessions with their owning agent.** Opening a session from All Profiles hydrates, resumes, sends, and renders with that session's profile without changing the global profile selection; New Chat from that view starts with the default profile.
- **Android reactions and standard voice follow the active conversation.** Reactions resolve durable rows for both user and assistant messages, while Vanilla Hermes voice remains on the authenticated Gateway instead of requiring the optional API fallback.
- **Android session navigation behaves predictably.** The drawer closes on outside taps, uses an ungrouped recent-session list by default, retains project grouping as an explicit option, and exposes secondary actions in All Profiles mode.
## [0.4.0-beta.3] - 2026-08-14
### Fixed
- **Windows tray polling can no longer accumulate unbounded helper processes.** Grant discovery now uses lightweight local state, management refreshes are single-flight and visibility-aware, and child probes have hard timeouts, bounded output, tree cleanup, caching, and backoff. A dedicated bounded `tray.log` records sanitized operational failures without mixing them into daemon logs.
- **Concurrent Desktop lifecycle requests cannot start duplicate daemons.** Cross-process lifecycle and runtime ownership locks serialize startup and recovery while preserving stale-owner cleanup.
## [1.8.0] - 2026-08-14
### Added
- **Official Hermes Desktop can surface Relay through its supported runtime Plugin SDK.** The unified plugin package now includes an opt-in, profile-scoped Desktop pane for Relay status, paired devices, bridge activity, media, pairing, revocation, and remote-access management. Loading, startup, reconnects, profile changes, and updates never open it; only labeled sidebar, status-bar, or command-palette actions register and reveal the movable native pane.
## [0.4.0-beta.2] - 2026-08-14
### Added
- **Desktop Activity now keeps inspectable local evidence.** Commands, files, devices, connection lifecycle, and computer control share a truthful event stepper with dedicated failure details; screenshot events can retain bounded local PNG evidence and open it in a larger borderless viewer. Settings controls retention as Off, 1 day, 7 days, or 30 days and shows local file usage.
### Fixed
- **Tunnel state stays responsive through interruption and retry.** The CLI UI distinguishes connected, reconnecting, and stopped states, exposes retry attempt/timing and a Retry now action, records connection failures and recovery in Activity, and shows compact connection cards only while the main UI is hidden.
- **Windows CUA readiness no longer depends on the flaky whole-desktop health scan.** Hermes-Relay verifies the canonical runtime, manifest, required tools, daemon, and safe permission mode before starting structured sessions, while accessibility health remains an explicit CLI/UI diagnostic that can be rechecked without forcing the compatibility backend. This temporary workaround is scoped to the upstream fixed-timeout issue and keeps individual actions fail-closed.
## [0.4.0-beta.1] - 2026-08-14
### Added
- **CUA Driver is the preferred Windows structured-control engine.** New local settings prefer a verified CUA runtime for window-targeted background actions, fresh snapshot tokens, and optional per-session animated agent cursors without moving the physical pointer; Windows Input is the explicit compatibility backend and backend choice is fixed for each control session. Full-display observation remains on the read-only system capture path. CLI and UI can explicitly install, check, or update the canonical CUA package after verifying the upstream release manifest and installer checksum; nothing is bundled or updated automatically, driver telemetry stays off for Hermes sessions, and activity records contain only bounded, redacted control metadata.
### Fixed
- **Windows bundle updates fail closed when installed processes retain a binary lock.** Setup waits for the invoking CLI, quiesces the tray and its short-lived CLI children, checks every payload extraction before writing release metadata, preserves custom install directories, and returns a failure instead of reporting a mixed-version installation.
- **CUA readiness follows the published driver contract.** Hermes accepts the documented `ok` health state, distinguishes an installed-but-degraded runtime from a missing installation, and constructs trusted Windows installer paths consistently across verification environments.
## [1.7.0] - 2026-08-13
+8 -15
View File
@@ -1,30 +1,23 @@
# Hermes-Relay CLI v__VERSION__
**Release Date:** 2026-08-13
**Release Date:** 2026-08-14
This alpha makes the compact **Hermes-Relay CLI UI** responsive during connection changes, expands host and capability management, and presents live route security and diagnostics without turning the tray into a full desktop client.
This patch prevents the Windows management tray from accumulating Hermes-Relay, registry, and ADB helper processes when a refresh is slow or fails, and adds bounded diagnostics for future recovery.
**Experimental phase.** Assets remain unsigned, so Windows SmartScreen and macOS Gatekeeper may warn on first launch. Standalone CLI binaries ship for Windows x64, Linux x64, and macOS x64/arm64; the management tray is Windows-only.
**Beta phase.** Assets remain unsigned, so Windows SmartScreen and macOS Gatekeeper may warn on first launch. Standalone CLI binaries ship for Windows x64, Linux x64, and macOS x64/arm64; the management UI is Windows-only.
## What's changed
### Added
- **Host management hub.** Each paired Hermes host has local identity, pairing facts, access and capability controls, authorized-client revocation, re-pairing, and guarded removal.
- **Evidence-first activity.** Overview shows the latest three events and detailed views retain bounded command, output, exit, duration, and truncation evidence.
- **Live route details.** The Agent-to-PC path shows route type, encryption state, endpoint, packet motion, and a connection test with reachability and latency.
### Changed
- **Connection lifecycle stays responsive.** Connect, disconnect, and snapshot work run outside the UI thread with immediate transition feedback and single-flight live polling.
- **Access presets are explicit.** Restricted, Ask Every Time, Standard, Full Access, and Custom map visibly onto individual command, file, screen/input, and hardware capabilities.
- **Tailscale is the recommended remote route.** Direct TLS and Hermes Secure Link remain supported; Hermes Reach is marked experimental and stays below supported routes.
- **Grant discovery stays lightweight.** The approval window reads local bridge state instead of rebuilding the complete management snapshot, schedules each refresh only after the previous one finishes, and pauses idle polling while hidden.
- **Management refreshes are visibility-aware and resilient.** Concurrent snapshot requests share one bounded result, optional static checks are cached, and repeated failures back off instead of creating more work.
### Fixed
- **Legacy LAN and Tailscale routes no longer appear as Custom VPN.** Route testing infers generic saved roles from the endpoint and reports the correct network path.
- **PowerShell output remains complete.** Scalar, pipeline, JSON, native output, errors, exit status, and truncation metadata return reliably through desktop RPC.
- **Tray placement follows the real notification area.** Responsive geometry uses the tray monitor and DPI and remains anchored above the icon.
- **Windows helper processes are contained.** Tray-launched commands have hard deadlines, bounded output capture, descendant cleanup, and suppressed loader-error dialogs, preventing stalled probes from growing into a process storm.
- **Daemon startup is serialized across launchers.** Cross-process lifecycle and runtime ownership locks prevent concurrent start or restart requests from leaving duplicate daemons behind.
- **Tray failures are diagnosable without exposing command data.** A rotated, sanitized `tray.log` records bounded probe and lifecycle outcomes separately from `daemon.log`.
## Install
+6 -12
View File
@@ -1,8 +1,8 @@
# Hermes-Relay-Server v__VERSION__
**Release Date:** August 13, 2026
**Release Date:** August 14, 2026
This release adds an operator-owned secure ingress path, promotes Tailscale as the easiest supported remote route, and introduces Hermes Reach as a disabled-by-default experimental broker for outbound-only environments.
This release adds an official, opt-in Relay pane for Hermes Desktop through the supported runtime Plugin SDK. It keeps Relay management profile-scoped and user-invoked without opening a pane during startup, reconnects, profile changes, or plugin updates.
Standard chat, session history, and Vanilla Hermes voice remain upstream-owned and do not require this plugin.
@@ -10,19 +10,13 @@ Standard chat, session history, and Vanilla Hermes voice remain upstream-owned a
### Added
- **Hermes Secure Link.** A pinned-TLS ingress can expose Relay, API, and Dashboard namespaces through one self-hosted endpoint while retaining each service's native authentication.
- **Experimental Hermes Reach.** An optional self-hosted rendezvous broker forwards opaque inner Secure Link TLS records, with hashed credentials, replay protection, bounded streams, persistence, and revocation.
- **Remote-access status.** Dashboard and pairing metadata distinguish reachability from transport protection and show supported services without exposing certificate pins.
- **Official Hermes Desktop pane.** The unified plugin package registers a movable native pane for Relay status, paired devices, bridge activity, media, pairing, revocation, and remote-access management.
- **Explicit entry points.** Labeled sidebar, status-bar, and command-palette actions register and reveal the pane lazily; repeated opens reuse the same surface.
- **Profile-scoped state.** Cached Relay state follows the active Hermes profile and is disposed cleanly when the plugin unloads.
### Changed
- **Tailscale is recommended for remote access.** Tailscale Serve remains the simplest supported path; direct TLS and Secure Link are self-hosted alternatives, while Reach stays advanced and experimental.
- **Pairing carries reviewed transport trust.** QR payloads include the Secure Link endpoint and pin before the first network request; rotation requires explicit re-pairing.
### Fixed
- **Route credentials remain scoped and revocable.** Reach credentials are issued only through trusted Secure Link ingress, replaced atomically per Relay session, bounded by session expiry, and removed on revocation.
- **Proxy namespaces preserve credential isolation.** API and Dashboard headers, cookies, redirects, methods, sizes, timeouts, and loopback authority are constrained independently.
- **Plugin loading stays passive.** Loading, startup, reconnects, profile changes, and updates never reveal the pane or perform pane-owned network work.
## Install / update
+8
View File
@@ -219,6 +219,14 @@ It pairs against the **same relay and credential store** as the Android app —
On Windows, the default installer adds the optional compact **Hermes-Relay CLI UI** tray popup for host selection and pairing, connection and daemon state, per-host Ask/Trusted/Full Access, local grant dialogs, authorized-client revocation, activity, settings, and emergency stop. It is a management surface only—chat, TUI, plugins, voice, and agent sessions remain CLI/upstream concerns.
Structured Windows computer control prefers a compatible local CUA Driver
runtime for window-targeted background actions and virtual per-session agent
cursors. It remains behind Hermes host policy, grants, targeting, audit, and
emergency stop; Windows input is an explicit compatibility backend. CUA is not
bundled or updated automatically, but the local CLI/UI can explicitly install,
check, or update its verified canonical package. It is never exposed as a raw
remote tool surface. See the [desktop tools guide](https://hermes-relay.dev/docs/desktop/tools.html#computer-use-engines).
- **Docs:** [CLI guide](https://hermes-relay.dev/docs/desktop/) · [`desktop/README.md`](desktop/README.md)
- **AI-agent setup recipe:** `/hermes-relay-desktop-setup`
+3
View File
@@ -128,6 +128,7 @@ optional Windows installer.
| File | Purpose |
|---|---|
| `desktop/package.json` | canonical CLI version |
| `desktop/.bun-version` | exact Bun compiler/runtime for standalone binaries |
| `desktop/package-lock.json` | npm root/workspace package metadata |
| `desktop/src/version.ts` | compiled CLI runtime version |
| `desktop/tray/Cargo.toml` | native systray package version |
@@ -152,6 +153,8 @@ manually, run `npm run sync:version` before checking. `npm run verify` is the
single Windows release-parity gate: version sync, type-check, tests, TypeScript
build, compiled CLI smoke, and tray formatting, Clippy, check, and tests. CI runs
the portable portions on every desktop change and the Windows tray gates separately.
Release jobs read `desktop/.bun-version`; cross-built and Windows-built artifacts
must not silently embed different Bun runtime versions.
## Branching policy
+27 -13
View File
@@ -1,10 +1,10 @@
# Hermes-Relay-Android v1.8.1
# Hermes-Relay-Android v1.9.0
**Release Date:** August 9, 2026
**Release Date:** August 14, 2026
## Download
> Installing on your phone? Download `hermes-relay-1.8.1-sideload-release.apk` and tap it for the full feature set, or install the conservative build from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
> Installing on your phone? Download `hermes-relay-1.9.0-sideload-release.apk` and tap it for the full feature set, or install the conservative build from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
The `.aab` file is a Play Console upload bundle and cannot be installed by tapping it on a phone.
@@ -12,22 +12,36 @@ Verify the download against `SHA256SUMS.txt`. See the [sideload guide](https://h
## Summary
This patch keeps long Hermes conversations complete and aligns Android's
Gateway behavior with current upstream turn contracts.
This release makes multi-profile session browsing feel native, keeps reactions
and voice attached to the correct conversation, and expands standard Gateway
management without requiring the optional Relay plugin.
## Added
- Browse one profile or all profiles, customize sorting and filters, and
optionally group sessions by project, recency, status, or profile.
- See profile identity, repository, branch, and pull-request context directly
in session rows when Hermes supplies it.
- Edit current Hermes profiles and complete more Manage workflows through the
authenticated standard Gateway.
## Fixed
- Complete transcript reads page explicitly across both API-server and
profile-scoped Dashboard routes, so sessions beyond Hermes' latest-500
default retain stable history, sharing, retry, edit, and recovery anchors.
- Gateway submit rejections preserve the authoritative server message without
silently falling through to SSE.
- Gateway event envelopes reconcile consistently, and edit-and-regenerate
requests send the required truncation confirmation.
- Cross-profile sessions hydrate and resume with their owning agent while All
Profiles remains selected; New Chat from that view respects the default
profile.
- Reactions pin to both user and assistant messages using durable message rows.
- Vanilla Hermes voice stays on the Gateway instead of depending on the
optional API fallback.
- Session navigation defaults to an ungrouped list, keeps project grouping
opt-in, restores secondary actions, and closes on outside taps.
- Route changes shut down network clients off the main thread, and Gateway
outcomes, uploads, clarify cards, automation state, and media recovery follow
authoritative upstream behavior.
## Install / Verify
- App version: **1.8.1** (versionCode **42**).
- App version: **1.9.0** (versionCode **43**).
- Standard Chat, sessions, Manage, and Vanilla Hermes voice continue to work
against unmodified upstream Hermes.
- The optional Relay plugin is not required for standard Android chat or hosted
+42 -2
View File
@@ -6,6 +6,27 @@ For shipped work, see `DEVLOG.md`. For architectural decisions, see `docs/decisi
---
## Certify the official Desktop Relay plugin
The unified `plugin/desktop/plugin.js` implementation is covered by source-level
SDK contract, packaging, explicit-open, no-auto-open, close, unload, and profile
cache-isolation tests. A physical official Hermes Desktop session is still
required before calling the UX live-certified:
- Test default and named local profiles, ordinary authenticated remote mode,
and SSH mode with differently named local/remote profile mapping.
- In two full app windows, prove enabling, registration, explicit open,
requests, close/reopen, hot reload, and disable/unload remain window-local.
- Prove startup, reconnect, profile change, layout restore/reset, update, and
background events never open or focus Relay.
- Drag and dock the pane across native zones, close it, reopen it from all three
labeled actions, and verify no private-hook fallback is needed.
- Exercise Relay running/unreachable, zero/one/multiple devices, pairing,
revocation, bridge activity, media, remote access, and renderer error logging
without exposing credentials, pairing payloads, filesystem paths, or tokens.
---
## Structured desktop hardware capabilities
Structured access and per-host USB policy now ship with typed, serial-bound ADB
@@ -935,7 +956,7 @@ Phase 1 (end-to-end spine) shipped on `Codename-11/phone-platform` — `send_mes
- **Outbound buffering — ✅ relay-side DONE (2026-06-29).** `ProactiveChannel.push()` now queues agent→phone messages in a bounded deque (drop-oldest, 24 h TTL) when no phone is subscribed and returns `{queued: true}` (not 503); `_flush_outbound` delivers FIFO on the next subscribe (stale pruned). Inspect/cancel via `peek_outbound`/`cancel_outbound` + loopback `GET`/`DELETE /phone/outbound`. **UI surfacing of the queued state** (host-side, since the queue exists while the phone is OFFLINE): (a) ✅ **desktop CLI `relay queue` / `relay queue --clear` / `--cancel <id>` DONE (2026-06-29)** over the new endpoints (loopback-only — run on the relay host); a dashboard Relay-tab view is the optional GUI equivalent; (b) **remaining** — in the threaded agent surface, mark messages that arrived-while-away, and show the user's OWN pending replies (the Phase 3 reply queue) with a sending/Cancel affordance — that's where phone-side "queued + cancel" belongs.
- **Threads surface (unified-session model — see ADR 12 + the Refinement above).** Build order, each shippable: **(1)** source tags in the session drawer (`source=phone` → clean **Threads** chip + thread-spool icon, NOT a phone glyph) — also delivers the "source attribution in Chat" goal; **(2)** open a Thread in Chat from its session-store history (reuse the existing message-history path); **(3)** route the live `proactive` push into the session view + notification + unread, demoting `ProactiveInboxStore` to cache/outbox; **(4)** reply from the Chat composer via `proactive.reply` + persist the user turn + local `Sending/Queued/Failed` status — **MVP**; **(5)** a **Threads capability row** in the best-path UI + a pinned **Threads** entry atop the drawer (thread-spool icon, shown only when relay-paired + opted-in) + retire `HermesInboxScreen`, re-point the notification deep-link + Settings "View messages"; **(6)** outbox/retry on reconnect; **(7)** relay `proactive.reply.ack` (honest Delivered) + `proactive.cancel`; **(8)** multi-thread `chat_id` (named/project Threads). **Verify gate before (1):** confirm the app's session-list/history path surfaces a `source=phone` session cleanly (upstream `session.list` returns all sources flat, so it should — but check whether the drawer currently filters it out). Honesty call: do NOT show "Delivered" until (7) lands (can't confirm it client-side before the ack).
- **Status (2026-06-29, implemented UNBUILT — verify in Studio):** **CODE-COMPLETE on `dev`:** slice **1** (drawer source tags + `ThreadSpoolGlyph` + Threads filter), **2** (open a Thread from history — free via the existing `loadSessionHistory` path), **3-parse** (carry `reply_to` on `ProactiveMessage`), **4** (composer reply in a `source=phone` session routes over `proactive.reply`; `MessageDeliveryStatus` SENDING→DELIVERED/FAILED on the bubble), **5** (Threads capability row in `SessionPathCard` + `threadsCapabilityActive` drawer wiring), **7** (relay `proactive.reply.ack` + `proactive.cancel` — 25/25 `unittest` green — and client ack handling). **DONE since (2026-06-29, built + on phone):** live **in-thread reply rendering** (an agent reply lands in the open Thread as an ASSISTANT bubble, suppressing the notification/inbox — `injectIntoThread`); **user-created named Threads** ("+ New Thread"); **retire `HermesInboxScreen`** (deleted; route + nav removed; notification tap + Settings "View messages" re-pointed to Chat; surface renamed "Hermes messages" → **"Threads"**); relay slice-7 ack/cancel **DEPLOYED** to the host so **"Delivered" is live**. **DEFERRED (reasons):** per-session **unread badge**; **outbox/retry** (needs multiplexer connection-state); **exact-Thread deep-link** from the notification (opens Chat today, not the specific thread — needs select-session-on-entry); **remove the now-orphaned `ProactiveInboxStore`** (viewer-less write-only log); **agent-initiated** named Threads (upstream `send_message` thread param). On-device verifies for the create-flow: fresh-`chat_id` auto-create, the `…:dm:<chat_id>` id form, `renameSession` on a phone session.
- **Status (2026-06-29, implemented UNBUILT — verify in Studio):** **CODE-COMPLETE on `dev`:** slice **1** (drawer source tags + `ThreadSpoolGlyph` + Threads filter), **2** (open a Thread from history — free via the existing `loadSessionHistory` path), **3-parse** (carry `reply_to` on `ProactiveMessage`), **4** (composer reply in a `source=phone` session routes over `proactive.reply`; `MessageDeliveryStatus` SENDING→DELIVERED/FAILED on the bubble), **5** (Threads capability row in `SessionPathCard` + `threadsCapabilityActive` drawer wiring), **7** (relay `proactive.reply.ack` + `proactive.cancel` — 25/25 `unittest` green — and client ack handling). **DONE since (2026-06-29, built + on phone):** live **in-thread reply rendering**; **user-created named Threads** ("+ New Thread"); **retire `HermesInboxScreen`**; relay slice-7 ack/cancel **DEPLOYED** to the host so **"Delivered" is live**. **DONE (2026-08-14):** notification taps survive cold start and open the exact `chat_id`; agent-initiated outbound messages appear as connection-scoped provisional Threads backed by the bounded proactive store, then promote to the real `source=phone` session after the first reply. **DEFERRED:** per-session **unread badge**; **outbox/retry** (needs multiplexer connection-state); **agent-initiated** named Threads (upstream `send_message` thread param). On-device verifies for the create-flow: fresh-`chat_id` auto-create, the `…:dm:<chat_id>` id form, `renameSession` on a phone session.
- **User-created Threads (slice 8, Discord-style) — CODE-COMPLETE on `dev` (built + installed 2026-06-29; on-device behavior pending).** "+ New Thread" in the drawer's Threads view → name dialog → `ChatViewModel.startNewThread` mints a fresh `chat_id`; the first composer message opens it over `proactive.reply` (gateway auto-creates the `source=phone` session) → `switchToCreatedThread` polls + switches to the real session + applies the name. Existing-thread replies route by the `chat_id` parsed from the session id (`…:dm:<chat_id>`; opaque id → home fallback). **On-device verifies:** (1) a fresh-`chat_id` no-`reply_to` inbound creates a new `source=phone` session; (2) the phone session id carries the `…:dm:<chat_id>` form the client parses; (3) `renameSession` titles a phone session. **Remaining slice-8:** AGENT-initiated named Threads (the upstream `send_message` thread/chat_id param so the agent can open its own named Threads).
- **`chat_id` not exposed by `/api/sessions` (root cause of the 2026-06-29 on-device create-flow bugs — fixed client-side).** Confirmed on the host: a phone session's `id` is a timestamp (e.g. `20260629_204755_94f391d6`); the real `chat_id` lives in the `session_key` (`agent:main:phone:dm:<chat_id>`) and a `chat_id` column — but `/api/sessions` returns **neither `chat_id` nor `session_key`**, only `source` + the timestamp `id`. So the client could not map a session ↔ its `chat_id`, which broke create-thread switch/rename + reply routing + in-thread injection. **Client workaround shipped:** find a created thread by session-list **diff** (the new `source=phone` session), keep an in-memory `sessionId → chat_id` map (learned at creation + from incoming `phone.message`s) for reply routing, and inject by source (+ learned chat_id) rather than a parsed id. **Limitation:** for a thread the app didn't create *this* session (agent-created, another device, or after an app restart) `chat_id` is unknown until a message arrives while viewing it → its replies fall back to the home channel until then. **RESOLVED via the plugin (2026-06-29, per upstream-or-plugin policy):** the relay now exposes `GET /phone/threads` (`plugin/relay/session_store.py` reads the gateway store read-only → `[{session_id, chat_id, title}]`; `server.py` `handle_phone_threads`, bearer for the app / loopback for diag; 5 unit tests). The app (`RelayHttpClient.fetchPhoneThreads` → `ConnectionViewModel.phoneThreadChatIds` on every `auth.ok` → `ChatViewModel.seedThreadChatIds`, authoritative over the learned map) now routes replies correctly for **any** Thread — incl. ones it didn't create + after restart. Deployed + verified live. **Still-nice-to-have (lower priority): the upstream PR** to add `chat_id`/`session_key` to `/api/sessions` (the standard-path proper fix; the relay route then becomes redundant + the client prefers upstream when present).
- **Threads as named/project conversations (Discord-parity — folds into multi-thread #8).** A stable *named* `chat_id` per project = a persistent, agent-reachable project Thread (Discord named-thread parity for "persist a session for a project"). Enables: the agent **opening** a new named Thread for a background job/topic (a relay/gateway "open thread" affordance + a `send_message`-adjacent tool); cron/job updates landing in their own Thread; and replying to a Thread from any surface (desktop CLI / dashboard) since it is just a gateway session. Also evaluate per-Thread profile binding (a project Thread uses the "work" profile — ties to profile=contact).
@@ -964,7 +985,7 @@ The gateway-platform model is the *correct + sufficient architecture* (the phone
session store; the relay buffer is only the live/offline-delivery layer, not a
parallel history database.
- **Profile = contact mapping (new idea, fold in).** Multiple Hermes **profiles** (distinct agent personas/configs) could each be a distinct thread *source*/"contact" — DMing different agents. Maps cleanly onto the per-thread `chat_id` + source-attribution work; lets the app feel like a contact list of agents.
- **Per-thread notification controls + deep-link (Discord-parity affordances).** Per-thread notification channels, mute/DND/quiet-hours (Phase 3 partially), and a notification that **deep-links into the exact thread** (tap → land in that conversation) so dipping in/out while multitasking is frictionless.
- **Per-thread notification controls (Discord-parity affordances).** Exact-thread notification deep-linking is shipped. Remaining: per-thread notification channels and mute/DND/quiet-hours controls (Phase 3 partially).
- **Agent-initiated rich content.** Agent → phone thread with **images/cards** (relay media infra + `InboundAttachmentCard`/`HermesCardBubble` already exist on the chat side — reuse). Inbound (phone → agent) reply media stays deferred (text-first), but outbound rich content is low-cost parity.
- **In-thread "agent is working" indicator.** A typing/working state in the thread while the agent thinks/runs tools (Discord typing-dots parity) — the chat surface already has thinking indicators to reuse.
@@ -1230,6 +1251,25 @@ When the answer becomes clearer, this section becomes either an ADR in `docs/dec
## Smaller deferred items
- **Certify the preferred CUA Driver backend (ADR 56).** The canonical-runtime
probe, bounded adapter, server-owned control-session envelope, per-session
grant state, local engine/status controls, telemetry-off process environment,
and Hermes snapshot-token primitives now exist. Before graduating the engine,
finish end-to-end enforcement of app/display/folder scopes and sensitive
pixel/accessibility denial or redaction, harden the grant-bridge ACL and nonce
lifecycle, and complete live Windows certification proving the physical cursor and
foreground app stay unchanged, stale or cross-window tokens fail, two remote
control sessions receive isolated animated cursors, and foreground escalation
never happens implicitly. Exercise revoke on grant expiry, disconnect,
re-pair, policy downgrade, emergency stop, Windows-session change, and daemon
shutdown. The explicit local CUA install/update surface now verifies upstream
manifest identity and installer SHA-256; add Windows publisher verification
when upstream signs the installer. Keep raw CUA tools, configuration,
recording, replay, and JavaScript outside the remote agent surface.
Remove the temporary Windows readiness/health split once
[trycua/cua#3103](https://github.com/trycua/cua/issues/3103) ships in the
supported CUA range; restore a mandatory health gate only if the upstream
probe is bounded and cannot leave UI Automation falsely busy.
- **MediaProjection consent flow** — wired in MainActivity (2026-04-12), needs end-to-end test on a real device
- **WorkManager upgrade for auto-disable timer** — currently a coroutine `Job + delay()` in `AutoDisableWorker.kt`; documented at top of file. Upgrade when androidx.work joins the classpath
- **Wave 3 voice-bridge multi-turn confirmation** — currently a 5s TTS countdown with cancel; conversational confirmation is the follow-up
@@ -1 +1 @@
Long sessions now retain complete history beyond Hermes' latest-500 default, keeping edit, retry, sharing, and recovery anchors stable. Gateway submit rejections preserve the server's message without unintended SSE fallback, while event envelopes and edit-and-regenerate requests follow current upstream contracts.
Browse sessions across profiles without losing their owning agent or your selected scope. Reactions now pin to both user and assistant messages, Vanilla Hermes voice stays on the authenticated Gateway, and New Chat in All Profiles respects the default profile. Sessions are ungrouped by default, with project grouping available in Customize Sessions.
+28
View File
@@ -1,5 +1,33 @@
{
"versions": [
{
"version": "1.9.0",
"title": "Better sessions, reactions, and voice",
"date": "2026-08-14",
"sections": [
{
"header": "Sessions keep their identity",
"bullets": [
"Browse one profile or all profiles, customize sorting and filters, and optionally group sessions by project, recency, status, or profile.",
"Cross-profile sessions hydrate, resume, and send with their owning agent without changing the global profile selection; New Chat in All Profiles uses the default profile."
]
},
{
"header": "Conversation controls stay attached",
"bullets": [
"Reactions pin to durable rows on both user and assistant messages.",
"Vanilla Hermes voice stays on the authenticated Gateway instead of requiring the optional API fallback."
]
},
{
"header": "Context without clutter",
"bullets": [
"Session rows show profile, project, branch, and pull-request context when Hermes supplies it, while the default view remains ungrouped.",
"The session drawer restores secondary actions in All Profiles and closes when you tap outside it."
]
}
]
},
{
"version": "1.8.1",
"title": "Complete, reliable transcripts",
+6 -5
View File
@@ -1,6 +1,7 @@
v1.8.1 - Complete, reliable transcripts
v1.9.0 - Better sessions, reactions, and voice
* Keep complete history in long sessions beyond Hermes' latest-500 default.
* Preserve stable edit, retry, sharing, and recovery anchors while paging history.
* Show authoritative Gateway rejection messages without an unintended fallback.
* Reconcile Gateway events and edit-and-regenerate requests with current upstream contracts.
* Browse all profiles without switching away from the selected scope.
* Start new chats with the default profile and hydrate history with the session owner.
* Pin reactions to both user and assistant messages.
* Keep Vanilla Hermes voice on the authenticated Gateway.
* Use an ungrouped session list by default, with project grouping available in Customize Sessions.
@@ -147,6 +147,12 @@ data class ChatMessage(
* never used as a Compose key or synthesized client-side.
*/
val rowId: Long? = null,
/**
* Durable iOS-style tapbacks attached to this server message. Hermes keeps
* one reaction per author in the message's display metadata; the UI also
* updates this list optimistically while a reaction write is in flight.
*/
val reactions: List<MessageReaction> = emptyList(),
/**
* Mixture-of-Agents advisor responses surfaced during the live turn.
* Unavailable advisors retain only neutral state, never their raw failure
@@ -156,6 +162,29 @@ data class ChatMessage(
val moaReferences: List<MoaReference> = emptyList(),
)
data class MessageReaction(
val emoji: String,
val author: String,
/** Epoch seconds, matching the Gateway/Desktop contract. */
val at: Double,
)
/** Apply Hermes' one-reaction-per-author, re-tap-to-retract semantics. */
internal fun applyMessageReaction(
reactions: List<MessageReaction>,
emoji: String?,
author: String = "user",
at: Double = System.currentTimeMillis() / 1000.0,
): List<MessageReaction> {
val previous = reactions.firstOrNull { it.author == author }
val withoutAuthor = reactions.filterNot { it.author == author }
return if (emoji.isNullOrBlank() || previous?.emoji == emoji) {
withoutAuthor
} else {
withoutAuthor + MessageReaction(emoji = emoji, author = author, at = at)
}
}
data class MoaReference(
val index: Int,
val count: Int?,
@@ -412,6 +441,11 @@ data class ChatSession(
val title: String?,
val model: String?,
val messageCount: Int = 0,
val inputTokens: Int = 0,
val outputTokens: Int = 0,
val actualCostUsd: Double? = null,
val estimatedCostUsd: Double? = null,
val isActive: Boolean = false,
val updatedAt: Long = 0L,
val startedAt: Long = 0L,
val lastActivityAt: Long = 0L,
@@ -436,6 +470,12 @@ data class ChatSession(
val pullRequestState: String? = null,
val pullRequestDraft: Boolean = false,
) {
val totalTokens: Int
get() = inputTokens + outputTokens
val costUsd: Double
get() = actualCostUsd ?: estimatedCostUsd ?: 0.0
val activityTimestamp: Long
get() = firstPositive(lastActivityAt, updatedAt, startedAt)
@@ -34,6 +34,8 @@ data class ProactiveInboxEntry(
* field).
*/
val chatId: String? = null,
/** Owning saved connection. Null only for entries written by older builds. */
val connectionId: String? = null,
)
private val Context.proactiveInboxStore: DataStore<Preferences> by
@@ -49,10 +51,10 @@ private const val MAX_ENTRIES = 100
* newest-first, deduped by id (so a re-delivered message doesn't double up), and
* capped at [MAX_ENTRIES]. Survives app restart.
*
* Demoted (2026-06-29): the agent conversation now lives as a Thread in Chat (the
* gateway session is the durable history), so the in-app inbox view is retired.
* This store is only fed for messages NOT shown in an open Thread; it currently
* has no viewer and is fully retireable — see TODO.
* Demoted (2026-06-29): once a phone gateway session exists, it is the durable
* history. Outbound agent messages arrive before that session exists, so this
* bounded store also backs the provisional Thread until the user's first reply
* promotes it to a real `source=phone` session.
*/
class ProactiveInboxRepository(private val context: Context) {
@@ -9,6 +9,7 @@ import androidx.datastore.preferences.preferencesDataStore
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.map
import kotlinx.serialization.builtins.ListSerializer
import kotlinx.serialization.builtins.MapSerializer
import kotlinx.serialization.builtins.serializer
import kotlinx.serialization.json.Json
@@ -16,6 +17,8 @@ import kotlinx.serialization.json.Json
data class ProfilePresentation(
val order: List<String> = emptyList(),
val hidden: Set<String> = emptySet(),
/** Local-only named-profile accent overrides, stored as normalized RGB hex. */
val colors: Map<String, String> = emptyMap(),
)
/**
@@ -63,14 +66,17 @@ class ProfilePresentationStore(
private val json = Json { ignoreUnknownKeys = true }
private val listSerializer = ListSerializer(String.serializer())
private val mapSerializer = MapSerializer(String.serializer(), String.serializer())
private fun orderKey(connectionId: String) = stringPreferencesKey("order_$connectionId")
private fun hiddenKey(connectionId: String) = stringPreferencesKey("hidden_$connectionId")
private fun colorsKey(connectionId: String) = stringPreferencesKey("colors_$connectionId")
fun presentationFlow(connectionId: String): Flow<ProfilePresentation> = dataStore.data.map { prefs ->
ProfilePresentation(
order = decode(prefs[orderKey(connectionId)]),
hidden = decode(prefs[hiddenKey(connectionId)]).toSet(),
colors = decodeMap(prefs[colorsKey(connectionId)]),
)
}
@@ -82,10 +88,18 @@ class ProfilePresentationStore(
dataStore.edit { it[hiddenKey(connectionId)] = json.encodeToString(listSerializer, hidden.sorted()) }
}
suspend fun setColors(connectionId: String, colors: Map<String, String>) {
dataStore.edit {
if (colors.isEmpty()) it.remove(colorsKey(connectionId))
else it[colorsKey(connectionId)] = json.encodeToString(mapSerializer, colors.toSortedMap())
}
}
suspend fun clear(connectionId: String) {
dataStore.edit {
it.remove(orderKey(connectionId))
it.remove(hiddenKey(connectionId))
it.remove(colorsKey(connectionId))
}
}
@@ -98,6 +112,12 @@ class ProfilePresentationStore(
} else {
runCatching { json.decodeFromString(listSerializer, raw) }.getOrDefault(emptyList())
}
private fun decodeMap(raw: String?): Map<String, String> = if (raw == null) {
emptyMap()
} else {
runCatching { json.decodeFromString(mapSerializer, raw) }.getOrDefault(emptyMap())
}
}
internal val Context.profilePresentationDataStore: DataStore<Preferences>
@@ -148,6 +148,8 @@ class ConnectionManager(
private val reconnectJitterUnit: () -> Double = { kotlin.random.Random.nextDouble() },
/** Exact-authority pinned client for a plugin-proxy WSS URL. */
private val proxyClientProvider: ((String) -> OkHttpClient?)? = null,
/** Test seam for observing lifecycle teardown without opening a socket. */
private val okHttpClientFactory: (() -> OkHttpClient)? = null,
) {
private val supervisorJob = SupervisorJob()
private val scope = CoroutineScope(supervisorJob + Dispatchers.IO)
@@ -158,6 +160,7 @@ class ConnectionManager(
}
private fun buildClient(url: String? = null): OkHttpClient {
okHttpClientFactory?.let { return it() }
val builder = OkHttpClient.Builder()
// OkHttp's 10s default connectTimeout is LAN-tuned; a Tailscale
// DERP-relayed cold-start handshake can exceed it, and a failed
@@ -61,8 +61,8 @@ class ProactiveMessageHandler(
/**
* Show an inbound message inline in the Chat **Thread** it belongs to, when
* that Thread is currently open. Returns true if it was shown there — in
* which case the message is NOT also notified or added to the inbox (you're
* already looking at the conversation). The unified-Threads counterpart of
* which case the message is persisted but not also notified (you're already
* looking at the conversation). The unified-Threads counterpart of
* [toSession]; wired after construction.
*/
var injectIntoThread: ((ProactiveMessage) -> Boolean)? = null,
@@ -94,13 +94,15 @@ class ProactiveMessageHandler(
/** Route a parsed message: into the open Thread if it belongs there, else
* the durable inbox log + the surface its hint selects. */
private fun dispatch(msg: ProactiveMessage) {
// Unified Threads: if this message belongs to the Thread currently open
// in Chat, render it inline there and STOP — no notification, no inbox
// entry (you're already looking at the conversation).
if (injectIntoThread?.invoke(msg) == true) return
// Otherwise the inbox is the durable log of agent-initiated messages and
// the surfacing hint selects the additional surface.
// Persist first even when the currently open Thread consumes the live
// message. Agent-initiated outbound sends do not create a gateway
// session until the phone replies, so this cache is the provisional
// Thread transcript during that gap.
toInbox?.invoke(msg)
// Unified Threads: if this message belongs to the Thread currently open
// in Chat, render it inline there and stop before raising a notification.
if (injectIntoThread?.invoke(msg) == true) return
// The surfacing hint selects the additional surface.
when (msg.surfacing?.lowercase()) {
"inbox" -> { /* inbox only — already recorded above */ }
"session" -> {
@@ -1520,7 +1520,8 @@ class ChatHandler {
// this as the same visible row across the post-turn reload.
prior.copy(
id = messageId,
rowId = item.rowId,
rowId = item.resolvedRowId,
reactions = item.reactions,
role = role,
content = cleanedContent,
attachments = carriedAttachments,
@@ -1551,7 +1552,8 @@ class ChatHandler {
// nothing local to carry).
ChatMessage(
id = messageId,
rowId = item.rowId,
rowId = item.resolvedRowId,
reactions = item.reactions,
role = role,
content = cleanedContent,
attachments = carriedAttachments,
@@ -2050,6 +2052,11 @@ class ChatHandler {
title = resolvedTitle,
model = item.model,
messageCount = item.messageCount ?: 0,
inputTokens = item.inputTokens ?: 0,
outputTokens = item.outputTokens ?: 0,
actualCostUsd = item.actualCostUsd,
estimatedCostUsd = item.estimatedCostUsd,
isActive = item.isActive,
updatedAt = activityAtMs,
startedAt = startedAtMs,
lastActivityAt = lastActivityAtMs,
@@ -1661,7 +1661,7 @@ class GatewayChatClient(
* id. This follows the upstream gateway contract, which resolves the row
* atomically inside the active session. A null emoji removes the reaction.
*/
suspend fun reactToNewest(role: String, emoji: String?): Result<JsonObject> {
suspend fun reactToMessage(rowId: Long?, role: String, emoji: String?): Result<JsonObject> {
require(role == "user" || role == "assistant") { "unsupported reaction role" }
val sid = liveSessionId
?: return Result.failure(GatewayRpcException("no live session"))
@@ -1669,7 +1669,7 @@ class GatewayChatClient(
"message.react",
buildJsonObject {
put("session_id", sid)
put("newest_role", role)
if (rowId != null) put("row_id", rowId) else put("newest_role", role)
if (emoji == null) put("emoji", JsonNull) else put("emoji", emoji)
put("author", "user")
},
@@ -436,7 +436,8 @@ class HermesApiClient(
private val json: Json = Json {
ignoreUnknownKeys = true
isLenient = true
}
},
okHttpClient: OkHttpClient? = null,
) {
@Volatile
private var lastCapabilities: ServerCapabilities? = null
@@ -480,7 +481,7 @@ class HermesApiClient(
private val mainHandler = Handler(Looper.getMainLooper())
private val client: OkHttpClient = httpClient ?: OkHttpClient.Builder()
private val client: OkHttpClient = httpClient ?: okHttpClient ?: OkHttpClient.Builder()
.readTimeout(5, TimeUnit.MINUTES)
.connectTimeout(10, TimeUnit.SECONDS)
.build()
@@ -1,19 +1,24 @@
package com.hermesandroid.relay.network.upstream.models
import com.hermesandroid.relay.data.MessageReaction
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.KSerializer
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.SerializationException
import kotlinx.serialization.descriptors.PrimitiveKind
import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonDecoder
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonEncoder
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.doubleOrNull
import kotlinx.serialization.json.jsonArray
import kotlinx.serialization.json.jsonPrimitive
import kotlinx.serialization.json.longOrNull
@@ -97,6 +102,36 @@ object FlexibleLongSerializer : KSerializer<Long?> {
}
}
/**
* Dashboard versions may expose SQLite JSON columns either as an object or as
* their raw JSON string. Normalize both shapes so persisted presentation data
* (notably message reactions) survives a history reload on every supported
* upstream version.
*/
object FlexibleJsonObjectSerializer : KSerializer<JsonObject?> {
override val descriptor = JsonObject.serializer().descriptor
override fun deserialize(decoder: Decoder): JsonObject? {
return try {
val jsonDecoder = decoder as? JsonDecoder ?: return null
when (val element = jsonDecoder.decodeJsonElement()) {
is JsonObject -> element
is JsonPrimitive -> element.content.takeIf { it.isNotBlank() }
?.let { Json.parseToJsonElement(it) as? JsonObject }
else -> null
}
} catch (_: Exception) {
null
}
}
override fun serialize(encoder: Encoder, value: JsonObject?) {
val jsonEncoder = encoder as? JsonEncoder
?: throw SerializationException("FlexibleJsonObjectSerializer requires JSON")
jsonEncoder.encodeJsonElement(value ?: JsonNull)
}
}
/** Timestamp serializer for Hermes session metadata.
*
* Upstream currently returns epoch seconds for `started_at` / `last_active`;
@@ -134,6 +169,32 @@ object FlexibleTimestampSerializer : KSerializer<Double?> {
}
}
/**
* Boolean serializer for session flags backed by SQLite integer columns.
*
* Older Dashboard responses can expose those columns as `0` / `1` instead of
* JSON booleans. Accept the equivalent primitive forms without making arbitrary
* numbers or strings truthy, and always serialize back to a real JSON boolean.
*/
object FlexibleBooleanSerializer : KSerializer<Boolean> {
override val descriptor = PrimitiveSerialDescriptor("FlexibleBoolean", PrimitiveKind.BOOLEAN)
override fun deserialize(decoder: Decoder): Boolean {
val jsonDecoder = decoder as? JsonDecoder ?: return decoder.decodeBoolean()
val element = jsonDecoder.decodeJsonElement()
val value = (element as? JsonPrimitive)?.content?.trim()?.lowercase()
return when (value) {
"true", "1" -> true
"false", "0" -> false
else -> throw SerializationException("Expected a boolean-compatible value, got $element")
}
}
override fun serialize(encoder: Encoder, value: Boolean) {
encoder.encodeBoolean(value)
}
}
// --- Session CRUD responses ---
@Serializable
@@ -187,9 +248,16 @@ data class SessionItem(
@SerialName("tool_call_count") val toolCallCount: Int? = null,
@SerialName("input_tokens") val inputTokens: Int? = null,
@SerialName("output_tokens") val outputTokens: Int? = null,
@SerialName("has_model_config") val hasModelConfig: Boolean = false,
@SerialName("actual_cost_usd") val actualCostUsd: Double? = null,
@SerialName("estimated_cost_usd") val estimatedCostUsd: Double? = null,
@SerialName("is_active") val isActive: Boolean = false,
@SerialName("has_model_config")
@Serializable(with = FlexibleBooleanSerializer::class)
val hasModelConfig: Boolean = false,
/** Durable flags returned by current Dashboard and API-server session resources. */
@Serializable(with = FlexibleBooleanSerializer::class)
val pinned: Boolean = false,
@Serializable(with = FlexibleBooleanSerializer::class)
val archived: Boolean = false,
/** Optional workspace metadata added by newer Dashboard session lists. */
val cwd: String? = null,
@@ -327,7 +395,9 @@ data class MessageItem(
val timestamp: Double? = null,
@SerialName("finish_reason") val finishReason: String? = null,
@SerialName("display_kind") val displayKind: String? = null,
@SerialName("display_metadata") val displayMetadata: JsonObject? = null,
@SerialName("display_metadata")
@Serializable(with = FlexibleJsonObjectSerializer::class)
val displayMetadata: JsonObject? = null,
// Reasoning persisted with the assistant message (upstream serializes
// both names; reasoning is the canonical one). Restored into
// ChatMessage.thinkingContent so the Thought-process block survives a
@@ -335,11 +405,24 @@ data class MessageItem(
val reasoning: String? = null,
@SerialName("reasoning_content") val reasoningContent: String? = null,
) {
/**
* Dashboard history uses the SQLite row id as numeric `id`; Gateway
* history exposes the same value explicitly as `row_id`. Match Desktop by
* accepting either representation so persisted rows remain directly
* reactable after reload.
*/
val resolvedRowId: Long?
get() = rowId ?: id?.toLongOrNull()
/** Reasoning text under whichever field name the server used. */
val resolvedReasoning: String?
get() = reasoning?.takeIf { it.isNotBlank() }
?: reasoningContent?.takeIf { it.isNotBlank() }
/** Persisted tapbacks stored by Hermes in display_metadata.reactions. */
val reactions: List<MessageReaction>
get() = parseMessageReactions(displayMetadata?.get("reactions"))
/** Extract content as plain text string. Handles both string and array-of-parts formats. */
val contentText: String?
get() = when (content) {
@@ -367,6 +450,21 @@ data class MessageItem(
}
}
fun parseMessageReactions(element: JsonElement?): List<MessageReaction> =
(element as? JsonArray).orEmpty().mapNotNull { raw ->
val reaction = raw as? JsonObject ?: return@mapNotNull null
val emoji = (reaction["emoji"] as? JsonPrimitive)?.content?.takeIf { it.isNotBlank() }
?: return@mapNotNull null
val author = (reaction["author"] as? JsonPrimitive)?.content
?.takeIf { it == "user" || it == "agent" }
?: return@mapNotNull null
MessageReaction(
emoji = emoji,
author = author,
at = (reaction["at"] as? JsonPrimitive)?.doubleOrNull ?: 0.0,
)
}
// --- SSE streaming events from /api/sessions/{id}/chat/stream ---
//
// Hermes WebAPI event types (from server source):
@@ -9,6 +9,7 @@ import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
import android.os.Build
import android.net.Uri
import android.util.Log
import androidx.core.app.NotificationCompat
import androidx.core.app.NotificationManagerCompat
@@ -47,10 +48,13 @@ object ProactiveMessageNotifier {
/**
* Tap route — opens Chat, where the message lives as a Thread. Must match
* `Screen.Chat.route()` in RelayApp. Routed via the EXTRA_NAV_ROUTE deep-link
* path (MainActivity → NavRouteRequest → RelayApp collector). Opening the
* exact Thread by chat_id is a follow-up (see TODO).
* path (MainActivity → NavRouteRequest → RelayApp collector), carrying the
* `chat_id` so RelayApp opens the exact real or provisional Thread.
*/
private const val TAP_ROUTE = "chat"
private fun tapRoute(chatId: String?): String =
chatId?.takeIf { it.isNotBlank() }
?.let { "chat?proactiveChatId=${Uri.encode(it)}" }
?: "chat"
/**
* Post (or replace) a proactive-message notification.
@@ -80,7 +84,7 @@ object ProactiveMessageNotifier {
val tapIntent = Intent(context, MainActivity::class.java).apply {
flags = Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_CLEAR_TOP
putExtra(MainActivity.EXTRA_NAV_ROUTE, TAP_ROUTE)
putExtra(MainActivity.EXTRA_NAV_ROUTE, tapRoute(chatId))
}
val pendingFlags = PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE
// Distinct requestCode per slot so each notification gets its own
@@ -176,7 +176,9 @@ internal class HermesRuntimeBinder(
chat.setDisplayProfileProvider { connection.effectiveDisplayProfile.value }
chat.setDisplayAliasProvider { connection.profileDisplayAlias.value }
chat.setProfileSessionLister { connection.listProfileScopedSessions() }
chat.setProfileMessageLoaderWithMode(connection::loadProfileScopedMessages)
chat.setProfileMessageLoaderWithMode { profileName, sessionId, mode ->
connection.loadProfileScopedMessages(profileName, sessionId, mode)
}
chat.setDashboardConfigLoader { connection.loadActiveDashboardConfig() }
chat.profileSessionDeleter = connection::deleteProfileScopedSession
chat.profileSessionRenamer = connection::renameProfileScopedSession
@@ -336,17 +336,20 @@ sealed class Screen(
// NavHost, and the NavigationBarItem click must navigate via [route]()
// so no unresolved `{openAgentSheet}` leaks into the destination.
data object Chat : Screen(
"chat?openAgentSheet={openAgentSheet}&sessionId={sessionId}&profile={profile}",
"chat?openAgentSheet={openAgentSheet}&sessionId={sessionId}&profile={profile}" +
"&proactiveChatId={proactiveChatId}",
"Chat",
Icons.AutoMirrored.Filled.Chat,
) {
const val ARG_OPEN_AGENT_SHEET: String = "openAgentSheet"
const val ARG_SESSION_ID: String = "sessionId"
const val ARG_PROFILE: String = "profile"
const val ARG_PROACTIVE_CHAT_ID: String = "proactiveChatId"
fun route(
openAgentSheet: Boolean = false,
sessionId: String? = null,
profile: String? = null,
proactiveChatId: String? = null,
): String {
val params = buildList {
if (openAgentSheet) add("$ARG_OPEN_AGENT_SHEET=true")
@@ -356,6 +359,9 @@ sealed class Screen(
profile?.takeIf { it.isNotBlank() }?.let {
add("$ARG_PROFILE=${android.net.Uri.encode(it)}")
}
proactiveChatId?.takeIf { it.isNotBlank() }?.let {
add("$ARG_PROACTIVE_CHAT_ID=${android.net.Uri.encode(it)}")
}
}
return if (params.isEmpty()) "chat" else "chat?${params.joinToString("&")}"
}
@@ -1685,6 +1691,11 @@ fun RelayApp() {
nullable = true
defaultValue = null
},
navArgument(Screen.Chat.ARG_PROACTIVE_CHAT_ID) {
type = NavType.StringType
nullable = true
defaultValue = null
},
),
) { backStackEntry ->
// Responsive bubble width based on screen width. The "Blend"
@@ -1715,6 +1726,35 @@ fun RelayApp() {
val requestedProfileRoute = backStackEntry.arguments
?.getString(Screen.Chat.ARG_PROFILE)
?.takeIf { it.isNotBlank() }
val requestedProactiveChatId = backStackEntry.arguments
?.getString(Screen.Chat.ARG_PROACTIVE_CHAT_ID)
?.takeIf { it.isNotBlank() }
val proactiveInboxEntries by connectionViewModel.inboxMessages.collectAsState()
val phoneThreadChatIds by connectionViewModel.phoneThreadChatIds.collectAsState()
LaunchedEffect(
requestedProactiveChatId,
proactiveInboxEntries,
phoneThreadChatIds,
) {
val chatId = requestedProactiveChatId ?: return@LaunchedEffect
val realSessionId = phoneThreadChatIds.entries
.firstOrNull { it.value == chatId }
?.key
if (realSessionId != null) {
chatViewModel.switchSession(realSessionId)
} else {
val entries = proactiveInboxEntries.filter {
(it.connectionId == null || it.connectionId == activeConnectionId) &&
(it.chatId ?: "phone") == chatId
}
if (entries.isEmpty()) return@LaunchedEffect
chatViewModel.openProactiveThread(chatId, entries)
}
backStackEntry.arguments?.putString(
Screen.Chat.ARG_PROACTIVE_CHAT_ID,
null,
)
}
LaunchedEffect(
requestedSessionId,
requestedProfileRoute,
@@ -1739,7 +1739,10 @@ fun ActiveCardRoutesSection(
var routeEditorOriginal by remember(connection.id) {
mutableStateOf<EndpointCandidate?>(null)
}
val hasTailscaleRoute = endpoints.any { it.role.equals("tailscale", ignoreCase = true) }
val hasTailscaleRoute = hasConfiguredTailscaleRoute(
endpoints = endpoints,
primaryEndpointUrl = connection.primaryEndpointUrl,
)
val tailscalePreferred = preferredRole?.equals("tailscale", ignoreCase = true) == true
val routeNeedsAttention = activeEndpoint == null && liveState != RelayUiState.Connected
val showTailscaleUnavailableHint =
@@ -2171,6 +2174,12 @@ fun ActiveCardRoutesSection(
}
}
internal fun hasConfiguredTailscaleRoute(
endpoints: List<EndpointCandidate>,
primaryEndpointUrl: String,
): Boolean = endpoints.any { it.role.equals("tailscale", ignoreCase = true) } ||
Connection.inferRouteRole(primaryEndpointUrl) == "tailscale"
/**
* Numbered step row for the Manual pairing code fallback. Tightly
* coupled to its Card 3 layout — step badge sizing + content shape —
@@ -94,7 +94,7 @@ import kotlinx.coroutines.delay
*/
enum class ChatInputTrailing { SEND, VOICE, STOP, STEER, QUEUE }
private val ChatComposerShape = RoundedCornerShape(18.dp)
private val ChatComposerShape = RoundedCornerShape(26.dp)
private val ChatInputChipShape = RoundedCornerShape(12.dp)
internal const val CHAT_INPUT_FIELD_TEST_TAG = "chat-input-field"
@@ -315,7 +315,7 @@ fun ChatInputBar(
border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant),
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 8.dp, vertical = 6.dp)
.padding(horizontal = 8.dp, vertical = 3.dp)
.then(surfaceModifier),
) {
Column {
@@ -341,15 +341,15 @@ fun ChatInputBar(
}
Column(
modifier = Modifier.padding(horizontal = 8.dp, vertical = 6.dp),
modifier = Modifier.padding(horizontal = 6.dp, vertical = 3.dp),
) {
BasicTextField(
value = value,
onValueChange = { if (it.length <= charLimit) onValueChange(it) },
modifier = Modifier
.fillMaxWidth()
.heightIn(min = 34.dp)
.padding(horizontal = 8.dp, vertical = 4.dp)
.heightIn(min = 30.dp)
.padding(horizontal = 10.dp, vertical = 2.dp)
// Keep directional keys inside the editor. Compose's
// BasicTextField owns normal caret/selection movement;
// cancelling focus traversal prevents a boundary arrow
@@ -408,7 +408,7 @@ fun ChatInputBar(
Row(
modifier = Modifier
.fillMaxWidth()
.heightIn(min = 48.dp),
.heightIn(min = 44.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(4.dp),
) {
@@ -27,6 +27,7 @@ import androidx.compose.foundation.layout.fillMaxHeight
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.offset
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.layout.widthIn
@@ -43,6 +44,7 @@ import androidx.compose.material.icons.filled.FormatQuote
import androidx.compose.material.icons.filled.Stop
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
@@ -90,6 +92,7 @@ import java.text.SimpleDateFormat
import java.util.Date
internal const val CHAT_PET_IDENTITY_OBSTACLE_PREFIX = "chat-message-identity:"
private val MESSAGE_REACTIONS = listOf("❤️", "👍", "👎", "😂", "‼️", "❓")
@OptIn(ExperimentalFoundationApi::class)
@Composable
@@ -455,11 +458,51 @@ fun MessageBubble(
val showEditAction = onEditMessage != null && isUser
val showSpeakAction = shouldShowSpeakResponseAction(message, onSpeakMessage != null)
val showStopSpeakingAction = shouldShowStopSpeakingAction(message, onStopSpeaking != null)
val selectedUserReaction = message.reactions.firstOrNull { it.author == "user" }?.emoji
if (onQuoteMessage != null || onReact != null || showEditAction || showSpeakAction || showStopSpeakingAction) {
DropdownMenu(
expanded = showMessageActions,
onDismissRequest = { showMessageActions = false },
shape = RoundedCornerShape(24.dp),
containerColor = MaterialTheme.colorScheme.surfaceContainerHigh,
tonalElevation = 3.dp,
shadowElevation = 8.dp,
) {
if (onReact != null) {
Row(
horizontalArrangement = Arrangement.SpaceEvenly,
verticalAlignment = Alignment.CenterVertically,
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 4.dp, vertical = 6.dp),
) {
MESSAGE_REACTIONS.forEach { emoji ->
IconButton(
onClick = {
showMessageActions = false
onReact(emoji)
},
modifier = Modifier.background(
color = if (selectedUserReaction == emoji) {
MaterialTheme.colorScheme.secondaryContainer
} else {
Color.Transparent
},
shape = CircleShape,
),
) {
Text(
text = emoji,
fontSize = 24.sp,
modifier = Modifier.semantics {
contentDescription = "React with $emoji"
},
)
}
}
}
HorizontalDivider()
}
DropdownMenuItem(
text = { Text(stringResource(R.string.msg_bubble_copy)) },
onClick = {
@@ -515,8 +558,22 @@ fun MessageBubble(
},
)
}
if (onReact != null && selectedUserReaction != null) {
DropdownMenuItem(
text = { Text("Remove reaction") },
onClick = {
showMessageActions = false
onReact(null)
},
)
}
}
}
Box(
modifier = Modifier.padding(
bottom = if (message.reactions.isNotEmpty()) 8.dp else 0.dp,
),
) {
Surface(
shape = bubbleShape,
color = backgroundColor,
@@ -558,7 +615,7 @@ fun MessageBubble(
// same tactile confirm every chat app fires.
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
if (
onQuoteMessage != null || showEditAction || showSpeakAction ||
onQuoteMessage != null || onReact != null || showEditAction || showSpeakAction ||
showStopSpeakingAction
) {
showMessageActions = true
@@ -645,25 +702,6 @@ fun MessageBubble(
SelectionContainer { messageTextContent() }
}
}
if (onReact != null) {
listOf("👍", "❤️", "😂").forEach { emoji ->
DropdownMenuItem(
text = { Text("React $emoji") },
onClick = {
showMessageActions = false
onReact(emoji)
},
)
}
DropdownMenuItem(
text = { Text("Remove reaction") },
onClick = {
showMessageActions = false
onReact(null)
},
)
}
if (onSessionReference != null && sessionReferences.isNotEmpty()) {
sessionReferences.forEach { reference ->
TextButton(
@@ -838,6 +876,19 @@ fun MessageBubble(
}
}
}
if (message.reactions.isNotEmpty()) {
MessageReactionBadge(
reactions = message.reactions.map { it.emoji },
onOpen = onReact?.let { { showMessageActions = true } },
modifier = Modifier
.align(if (isUser) Alignment.BottomEnd else Alignment.BottomStart)
.offset(
x = if (isUser) (-10).dp else 10.dp,
y = 9.dp,
),
)
}
}
val inlineActions: @Composable () -> Unit = {
MessageInlineActions(
showQuote = onQuoteMessage != null,
@@ -890,6 +941,41 @@ fun MessageBubble(
} // end CompositionLocalProvider(LocalMediaBlurMode)
}
@OptIn(ExperimentalFoundationApi::class)
@Composable
private fun MessageReactionBadge(
reactions: List<String>,
onOpen: (() -> Unit)?,
modifier: Modifier = Modifier,
) {
val description = "Reactions: ${reactions.joinToString(" ")}"
Surface(
shape = RoundedCornerShape(12.dp),
color = MaterialTheme.colorScheme.surfaceContainerHighest,
tonalElevation = 2.dp,
shadowElevation = 2.dp,
modifier = modifier
.then(
if (onOpen != null) {
Modifier.combinedClickable(onClick = onOpen, onLongClick = onOpen)
} else {
Modifier
},
)
.semantics { contentDescription = description },
) {
Row(
horizontalArrangement = Arrangement.spacedBy(2.dp),
verticalAlignment = Alignment.CenterVertically,
modifier = Modifier.padding(horizontal = 7.dp, vertical = 3.dp),
) {
reactions.forEach { emoji ->
Text(text = emoji, fontSize = 14.sp, lineHeight = 16.sp)
}
}
}
}
@Composable
private fun MessageInlineActions(
showQuote: Boolean,
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,167 @@
package com.hermesandroid.relay.ui.components
import com.hermesandroid.relay.data.ChatSession
import com.hermesandroid.relay.data.SessionActivityState
import java.util.Locale
internal enum class SessionDrawerGrouping {
None,
Updated,
Project,
Status,
Profile,
}
internal enum class SessionDrawerOrdering {
Updated,
Created,
Title,
Status,
Tokens,
Cost,
}
internal enum class SessionDrawerStatus {
NeedsInput,
Working,
Idle,
}
internal enum class SessionDrawerPrState {
Open,
Draft,
Merged,
Closed,
None,
}
internal data class SessionDrawerViewOptions(
val grouping: SessionDrawerGrouping = SessionDrawerGrouping.None,
val ordering: SessionDrawerOrdering = SessionDrawerOrdering.Updated,
val statuses: Set<SessionDrawerStatus> = emptySet(),
val profiles: Set<String> = emptySet(),
val projects: Set<String> = emptySet(),
val pullRequests: Set<SessionDrawerPrState> = emptySet(),
val showProfile: Boolean = false,
val showUpdated: Boolean = true,
val showTokens: Boolean = false,
val showCost: Boolean = false,
)
internal data class SessionDrawerGroup(
val key: String,
val label: String?,
val rows: List<ProfileSessionRow>,
)
internal fun sessionRowKey(row: ProfileSessionRow): String =
"${row.profile.lowercase(Locale.ROOT)}:${row.session.sessionId}"
internal fun sessionProjectLabel(session: ChatSession): String {
val raw = (session.gitRepoRoot ?: session.workingDirectory)
?.trim()
?.trimEnd('/', '\\')
.orEmpty()
if (raw.isBlank()) return "No project"
return raw.substringAfterLast('/').substringAfterLast('\\').ifBlank { raw }
}
internal fun sessionDrawerStatus(
row: ProfileSessionRow,
activityStates: Map<String, SessionActivityState>,
): SessionDrawerStatus = when (
activityStates[sessionRowKey(row)] ?: activityStates[row.session.sessionId]
) {
SessionActivityState.NeedsInput -> SessionDrawerStatus.NeedsInput
SessionActivityState.Working -> SessionDrawerStatus.Working
null -> if (row.session.isActive) SessionDrawerStatus.Working else SessionDrawerStatus.Idle
}
internal fun sessionDrawerPrState(session: ChatSession): SessionDrawerPrState = when {
session.pullRequestNumber == null -> SessionDrawerPrState.None
session.pullRequestDraft -> SessionDrawerPrState.Draft
session.pullRequestState.equals("merged", ignoreCase = true) -> SessionDrawerPrState.Merged
session.pullRequestState.equals("closed", ignoreCase = true) -> SessionDrawerPrState.Closed
else -> SessionDrawerPrState.Open
}
internal fun filterAndSortSessionRows(
rows: List<ProfileSessionRow>,
options: SessionDrawerViewOptions,
activityStates: Map<String, SessionActivityState> = emptyMap(),
): List<ProfileSessionRow> {
val filtered = rows.asSequence()
.filter { options.statuses.isEmpty() || sessionDrawerStatus(it, activityStates) in options.statuses }
.filter { options.profiles.isEmpty() || it.profile in options.profiles }
.filter { options.projects.isEmpty() || sessionProjectLabel(it.session) in options.projects }
.filter { options.pullRequests.isEmpty() || sessionDrawerPrState(it.session) in options.pullRequests }
.toList()
val statusRank = mapOf(
SessionDrawerStatus.NeedsInput to 0,
SessionDrawerStatus.Working to 1,
SessionDrawerStatus.Idle to 2,
)
val comparator = when (options.ordering) {
SessionDrawerOrdering.Updated -> compareByDescending<ProfileSessionRow> { it.session.activityTimestamp }
SessionDrawerOrdering.Created -> compareByDescending { it.session.startTimestamp }
SessionDrawerOrdering.Title -> compareBy { it.session.title.orEmpty().lowercase(Locale.ROOT) }
SessionDrawerOrdering.Status -> compareBy { statusRank.getValue(sessionDrawerStatus(it, activityStates)) }
SessionDrawerOrdering.Tokens -> compareByDescending { it.session.totalTokens }
SessionDrawerOrdering.Cost -> compareByDescending { it.session.costUsd }
}
return filtered.sortedWith(
compareByDescending<ProfileSessionRow> { it.session.pinned }
.then(comparator)
.thenBy { it.session.title.orEmpty().lowercase(Locale.ROOT) },
)
}
internal fun groupSessionRows(
rows: List<ProfileSessionRow>,
grouping: SessionDrawerGrouping,
activityStates: Map<String, SessionActivityState> = emptyMap(),
nowMillis: Long = System.currentTimeMillis(),
): List<SessionDrawerGroup> {
if (rows.isEmpty()) return emptyList()
val grouped = rows.groupBy { row ->
when (grouping) {
SessionDrawerGrouping.None -> null
SessionDrawerGrouping.Updated -> updatedBucket(row.session.activityTimestamp, nowMillis)
SessionDrawerGrouping.Project -> sessionProjectLabel(row.session)
SessionDrawerGrouping.Status -> sessionDrawerStatus(row, activityStates).displayLabel
SessionDrawerGrouping.Profile -> row.profile
}
}
val groups = grouped.map { (label, groupRows) ->
SessionDrawerGroup(key = "${grouping.name}:$label", label = label, rows = groupRows)
}
return if (grouping == SessionDrawerGrouping.Project) {
groups.sortedWith(
compareBy<SessionDrawerGroup> { it.label != "No project" }
.thenByDescending { group -> group.rows.maxOfOrNull { it.session.activityTimestamp } ?: 0L }
.thenBy { it.label.orEmpty().lowercase(Locale.ROOT) },
)
} else {
groups
}
}
private val SessionDrawerStatus.displayLabel: String
get() = when (this) {
SessionDrawerStatus.NeedsInput -> "Needs input"
SessionDrawerStatus.Working -> "Working"
SessionDrawerStatus.Idle -> "Idle"
}
private fun updatedBucket(timestamp: Long, nowMillis: Long): String {
if (timestamp <= 0L) return "Older"
val age = (nowMillis - timestamp).coerceAtLeast(0L)
return when {
age < DAY_MILLIS -> "Today"
age < 2 * DAY_MILLIS -> "Yesterday"
age < 7 * DAY_MILLIS -> "Last 7 days"
else -> "Older"
}
}
private const val DAY_MILLIS = 24L * 60L * 60L * 1_000L
@@ -173,6 +173,7 @@ import com.hermesandroid.relay.data.HermesCardAction
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.data.PhysicalKeyboardEnterBehavior
import com.hermesandroid.relay.data.ProfilePresentationPolicy
import com.hermesandroid.relay.data.ProactiveInboxEntry
import com.hermesandroid.relay.data.SessionActivityState
import com.hermesandroid.relay.data.VoicePresentationMode
import com.hermesandroid.relay.data.hermesProcessNotificationOrNull
@@ -232,6 +233,7 @@ import com.hermesandroid.relay.ui.components.ThinkingMatrixColor
import com.hermesandroid.relay.ui.components.ThinkingMatrixPattern
import com.hermesandroid.relay.ui.components.SessionDrawerContent
import com.hermesandroid.relay.ui.components.ProfileSessionRow
import com.hermesandroid.relay.ui.components.ProvisionalThreadRow
import com.hermesandroid.relay.ui.components.ProfileDisplayManagerDialog
import com.hermesandroid.relay.ui.components.ProfileShelf
import com.hermesandroid.relay.ui.components.ProfileSwitcherSheet
@@ -298,14 +300,12 @@ internal fun resolveSessionActivityStates(
internal fun resolveChatHeaderSubtitle(
isStreaming: Boolean,
statusText: String,
projectName: String?,
personalityName: String?,
modelName: String?,
): String = if (isStreaming) {
statusText
} else {
listOfNotNull(
projectName?.takeIf { it.isNotBlank() },
personalityName?.takeIf { it.isNotBlank() },
modelName?.takeIf { it.isNotBlank() },
).joinToString(" \u00B7 ").ifBlank { statusText }
@@ -821,6 +821,27 @@ fun ChatScreen(
// has been made (the /api/config fallback is more useful than the bare
// connection label).
val agentProfiles by connectionViewModel.agentProfiles.collectAsState()
var allProfileSessions by remember { mutableStateOf<List<ProfileSessionRow>>(emptyList()) }
var allProfileSessionsLoading by remember { mutableStateOf(false) }
val openedSessionProfileName by chatViewModel.openedSessionProfileName.collectAsState()
val conversationProfile = openedSessionProfileName?.let { owner ->
agentProfiles.firstOrNull { it.name.equals(owner, ignoreCase = true) }
?: allProfileSessions.firstOrNull {
it.profile.equals(owner, ignoreCase = true) &&
it.session.sessionId == currentSessionId
}?.session?.let { session ->
com.hermesandroid.relay.data.Profile(
name = owner,
model = session.model.orEmpty(),
description = owner,
)
}
?: com.hermesandroid.relay.data.Profile(
name = owner,
model = "",
description = owner,
)
} ?: effectiveProfile
val profileDisplayAlias by connectionViewModel.profileDisplayAlias.collectAsState()
val activeConnection by connectionViewModel.activeConnection.collectAsState()
val serverModelName by chatViewModel.serverModelName.collectAsState()
@@ -832,7 +853,6 @@ fun ChatScreen(
val selectedProviderOverride by chatViewModel.selectedProviderOverride.collectAsState()
val gatewayCurrentModel by chatViewModel.gatewayCurrentModel.collectAsState()
val gatewayCurrentProvider by chatViewModel.gatewayCurrentProvider.collectAsState()
val gatewayProjectName by chatViewModel.gatewayProjectName.collectAsState()
val selectedReasoningEffort by chatViewModel.selectedReasoningEffort.collectAsState()
val currentSession = remember(sessions, currentSessionId) {
sessions.firstOrNull { it.sessionId == currentSessionId }
@@ -844,8 +864,8 @@ fun ChatScreen(
gatewayModel = gatewayCurrentModel,
gatewayProvider = gatewayCurrentProvider,
persistedSessionModel = currentSession?.model,
profileDefaultModel = effectiveProfile?.model,
serverDefaultModel = serverModelName,
profileDefaultModel = conversationProfile?.model,
serverDefaultModel = serverModelName.takeIf { openedSessionProfileName == null },
)
val sessionPickerProvider = sessionModelState.pickerProvider
?: sessionModelState.pickerModel?.let { model ->
@@ -1136,8 +1156,6 @@ fun ChatScreen(
}
val listState = rememberLazyListState()
val drawerState = rememberDrawerState(DrawerValue.Closed)
var allProfileSessions by remember { mutableStateOf<List<ProfileSessionRow>>(emptyList()) }
var allProfileSessionsLoading by remember { mutableStateOf(false) }
PetInteractionLayer(
owner = "chat-interaction-layer",
active = shouldHideChatPet(
@@ -1995,7 +2013,7 @@ fun ChatScreen(
// four keys, which missed updates in some cases (most notably a
// profile switch while the ConnectionInfoSheet was open, where the
// ambient sheet scope appeared to swallow the key comparison).
val agentDisplayName by remember(
val globalSelectedAgentDisplayName by remember(
effectiveProfile,
selectedPersonality,
defaultPersonality,
@@ -2003,13 +2021,31 @@ fun ChatScreen(
activeConnection?.label,
) {
derivedStateOf {
val profile = effectiveProfile
AgentDisplay.agentName(
profile = effectiveProfile,
selectedPersonality = selectedPersonality,
defaultPersonality = defaultPersonality,
connectionLabel = activeConnection?.label,
localDisplayAlias = profileDisplayAlias,
)
}
}
val agentDisplayName by remember(
conversationProfile,
selectedPersonality,
defaultPersonality,
profileDisplayAlias,
openedSessionProfileName,
activeConnection?.label,
) {
derivedStateOf {
val profile = conversationProfile
AgentDisplay.agentName(
profile = profile,
selectedPersonality = selectedPersonality,
defaultPersonality = defaultPersonality,
connectionLabel = activeConnection?.label,
localDisplayAlias = profileDisplayAlias,
localDisplayAlias = profileDisplayAlias.takeIf { openedSessionProfileName == null },
)
}
}
@@ -2038,13 +2074,13 @@ fun ChatScreen(
ModalNavigationDrawer(
drawerState = drawerState,
// Disable the drawer's edge-swipe while voice mode is up so the
// overlay reads as a true modal — the swipe gesture lives on the
// drawer itself, so the overlay's pointer scrim alone can't block it.
gesturesEnabled = !voiceUiState.voiceMode,
// Material routes scrim taps through the drawer's gesture handler.
// Keep it enabled so tapping outside always dismisses the drawer; the
// voice overlay already owns input while voice mode is visible.
gesturesEnabled = true,
drawerContent = {
val drawerTitle = if (effectiveProfile != null) {
stringResource(R.string.chat_profile_sessions, agentDisplayName)
stringResource(R.string.chat_profile_sessions, globalSelectedAgentDisplayName)
} else {
stringResource(R.string.chat_server_default_sessions)
}
@@ -2066,12 +2102,29 @@ fun ChatScreen(
val threadsCapabilityActive = threadsProactiveEnabled &&
threadsAuthState is com.hermesandroid.relay.auth.AuthState.Paired
val hiddenSources by connectionViewModel.hiddenSources.collectAsState()
val proactiveInboxEntries by connectionViewModel.inboxMessages.collectAsState()
val phoneThreadChatIds by connectionViewModel.phoneThreadChatIds.collectAsState()
val provisionalThreadEntries = buildProvisionalThreadRows(
entries = proactiveInboxEntries,
activeConnectionId = activeConnection?.id,
realThreadChatIds = phoneThreadChatIds.values,
)
val provisionalThreads = provisionalThreadEntries.map { (chatId, entries) ->
val latest = entries.maxBy { it.receivedAt }
ProvisionalThreadRow(
chatId = chatId,
title = latest.title.ifBlank { "Hermes" },
messageCount = entries.size,
lastActivityAt = latest.receivedAt,
)
}
SessionDrawerContent(
sessions = sessions,
currentSessionId = currentSessionId,
scopeTitle = drawerTitle,
scopeSubtitle = drawerSubtitle,
activeProfileName = effectiveProfile?.name ?: "default",
isLoading = isLoadingSessions,
isOpen = drawerState.isOpen,
activityStates = sessionActivityStates,
@@ -2083,6 +2136,24 @@ fun ChatScreen(
chatViewModel.createNewChat()
scope.launch { drawerState.close() }
},
onNewDefaultChat = {
val defaultProfile = agentProfiles.firstOrNull {
it.name.equals("default", ignoreCase = true)
} ?: com.hermesandroid.relay.data.Profile(
name = "default",
model = "",
description = "Default",
)
chatViewModel.createProfileChat(
profileName = "default",
profile = defaultProfile,
contextKey = AgentDisplay.profileContextKey(
connectionId = activeConnection?.id,
profileName = "default",
),
)
scope.launch { drawerState.close() }
},
onSelectSession = { sessionId ->
chatViewModel.switchSession(sessionId)
scope.launch { drawerState.close() }
@@ -2112,12 +2183,23 @@ fun ChatScreen(
chatViewModel.startNewThread(name)
scope.launch { drawerState.close() }
},
provisionalThreads = provisionalThreads,
onSelectProvisionalThread = { chatId ->
chatViewModel.openProactiveThread(
chatId,
provisionalThreadEntries[chatId].orEmpty(),
)
scope.launch { drawerState.close() }
},
hiddenSources = hiddenSources,
onToggleSourceHidden = { source, hidden ->
connectionViewModel.setSourceHidden(source, hidden)
},
allProfilesSupported = !activeConnection?.resolvedDashboardUrl.isNullOrBlank(),
allProfileSessions = allProfileSessions,
allProfileSessionsLoading = allProfileSessionsLoading,
profileColors = profilePresentation.colors,
onProfileColorChange = connectionViewModel::setProfileColor,
onRefreshAllProfiles = {
if (!allProfileSessionsLoading) scope.launch {
allProfileSessionsLoading = true
@@ -2134,6 +2216,11 @@ fun ChatScreen(
title = item.title ?: item.preview,
model = item.model,
messageCount = item.messageCount ?: 0,
inputTokens = item.inputTokens ?: 0,
outputTokens = item.outputTokens ?: 0,
actualCostUsd = item.actualCostUsd,
estimatedCostUsd = item.estimatedCostUsd,
isActive = item.isActive,
startedAt = ((item.startedAt ?: 0.0) * 1000).toLong(),
lastActivityAt = ((item.resolvedLastActivity ?: 0.0) * 1000).toLong(),
source = item.source,
@@ -2164,9 +2251,29 @@ fun ChatScreen(
it.name.equals(profileName, ignoreCase = true)
}
if (target != null || profileName.equals("default", ignoreCase = true)) {
connectionViewModel.selectProfile(target)
chatViewModel.activateGatewayProfile(target)
chatViewModel.switchSession(sessionId)
val ownerProfile = target ?: allProfileSessions.firstOrNull {
it.profile.equals(profileName, ignoreCase = true) &&
it.session.sessionId == sessionId
}?.session?.let { session ->
com.hermesandroid.relay.data.Profile(
name = profileName,
model = session.model.orEmpty(),
description = profileName,
)
} ?: com.hermesandroid.relay.data.Profile(
name = profileName,
model = "",
description = profileName,
)
chatViewModel.openProfileSession(
profileName = profileName,
profile = ownerProfile,
contextKey = AgentDisplay.profileContextKey(
connectionId = activeConnection?.id,
profileName = profileName,
),
sessionId = sessionId,
)
scope.launch { drawerState.close() }
} else {
scope.launch {
@@ -2174,6 +2281,54 @@ fun ChatScreen(
}
}
},
onDeleteProfileSession = { profileName, sessionId ->
scope.launch {
if (connectionViewModel.deleteSession(profileName, sessionId)) {
allProfileSessions = allProfileSessions.filterNot {
it.profile == profileName && it.session.sessionId == sessionId
}
}
}
},
onRenameProfileSession = { profileName, sessionId, title ->
scope.launch {
if (connectionViewModel.renameSession(profileName, sessionId, title)) {
allProfileSessions = allProfileSessions.map { row ->
if (row.profile == profileName && row.session.sessionId == sessionId) {
row.copy(session = row.session.copy(title = title))
} else {
row
}
}
}
}
},
onSetProfileSessionPinned = { profileName, sessionId, pinned ->
scope.launch {
if (connectionViewModel.setSessionPinned(profileName, sessionId, pinned)) {
allProfileSessions = allProfileSessions.map { row ->
if (row.profile == profileName && row.session.sessionId == sessionId) {
row.copy(session = row.session.copy(pinned = pinned))
} else {
row
}
}
}
}
},
onSetProfileSessionArchived = { profileName, sessionId, archived ->
scope.launch {
if (connectionViewModel.setSessionArchived(profileName, sessionId, archived)) {
allProfileSessions = allProfileSessions.map { row ->
if (row.profile == profileName && row.session.sessionId == sessionId) {
row.copy(session = row.session.copy(archived = archived))
} else {
row
}
}
}
}
},
)
}
) {
@@ -2279,7 +2434,6 @@ fun ChatScreen(
resolveChatHeaderSubtitle(
isStreaming = isStreaming,
statusText = statusText,
projectName = gatewayProjectName,
personalityName = nonDefaultPersonality,
modelName = modelName,
)
@@ -2586,7 +2740,7 @@ fun ChatScreen(
selectedProfile = selectedProfile,
resolvedProfile = effectiveProfile,
presentation = profilePresentation,
activeDisplayName = agentDisplayName,
activeDisplayName = globalSelectedAgentDisplayName,
isProfileLocked = isProfileLocked,
lockedProfileName = lockedProfileName,
switchEnabled = profileSwitchEnabled,
@@ -3112,9 +3266,12 @@ fun ChatScreen(
isGatewayTransport &&
messageReactionsSupported &&
!message.isStreaming &&
message.uiKey in newestReactableMessageKeys
(
message.rowId != null ||
message.uiKey in newestReactableMessageKeys
)
) {
{ emoji -> chatViewModel.reactToNewest(message.role, emoji) }
{ emoji -> chatViewModel.reactToMessage(message, emoji) }
} else {
null
},
@@ -4230,6 +4387,15 @@ fun ChatScreen(
}
}
internal fun buildProvisionalThreadRows(
entries: List<ProactiveInboxEntry>,
activeConnectionId: String?,
realThreadChatIds: Collection<String>,
): Map<String, List<ProactiveInboxEntry>> = entries
.filter { it.connectionId == null || it.connectionId == activeConnectionId }
.groupBy { it.chatId ?: "phone" }
.filterKeys { it !in realThreadChatIds }
// --- Helper functions ---
@Composable
@@ -0,0 +1,51 @@
package com.hermesandroid.relay.ui.theme
import androidx.compose.ui.graphics.Color
import java.util.Locale
import kotlin.math.abs
import kotlin.math.roundToInt
private const val PROFILE_SATURATION = 0.68f
private const val PROFILE_LIGHTNESS = 0.58f
/** The same evenly-spaced 12-hue picker model used by Hermes Desktop. */
val ProfileAccentSwatches: List<String> = (0 until 12).map { index ->
hslColor(index * 30f, PROFILE_SATURATION, PROFILE_LIGHTNESS).toRgbHex()
}
/** Desktop-compatible deterministic profile identity color; default remains neutral. */
fun resolveProfileAccent(name: String?, overrides: Map<String, String>): Color? {
val key = name?.trim().orEmpty()
if (key.isBlank() || key.equals("default", ignoreCase = true)) return null
return accentColor(overrides[key]) ?: deterministicProfileAccent(key)
}
internal fun deterministicProfileAccent(name: String): Color {
var hash = 0u
name.forEach { character -> hash = hash * 31u + character.code.toUInt() }
return hslColor((hash % 360u).toFloat(), PROFILE_SATURATION, PROFILE_LIGHTNESS)
}
private fun hslColor(hue: Float, saturation: Float, lightness: Float): Color {
val chroma = (1f - abs(2f * lightness - 1f)) * saturation
val section = (hue / 60f) % 6f
val x = chroma * (1f - abs(section % 2f - 1f))
val (red, green, blue) = when {
section < 1f -> Triple(chroma, x, 0f)
section < 2f -> Triple(x, chroma, 0f)
section < 3f -> Triple(0f, chroma, x)
section < 4f -> Triple(0f, x, chroma)
section < 5f -> Triple(x, 0f, chroma)
else -> Triple(chroma, 0f, x)
}
val match = lightness - chroma / 2f
return Color(red + match, green + match, blue + match)
}
private fun Color.toRgbHex(): String = String.format(
Locale.ROOT,
"#%02X%02X%02X",
(red * 255f).roundToInt(),
(green * 255f).roundToInt(),
(blue * 255f).roundToInt(),
)
@@ -1,8 +1,8 @@
package com.hermesandroid.relay.util
import kotlinx.coroutines.flow.MutableSharedFlow
import kotlinx.coroutines.flow.SharedFlow
import kotlinx.coroutines.flow.asSharedFlow
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.receiveAsFlow
/**
* Cross-layer one-shot navigation requests.
@@ -18,23 +18,21 @@ import kotlinx.coroutines.flow.asSharedFlow
* `BridgeSafetySettingsScreen` instead of dropping the user on `MainActivity`'s
* home screen.
*
* Buffer: `extraBufferCapacity = 4` so back-to-back tryEmit calls during
* `onCreate → setContent` don't drop on the floor before `RelayApp`'s
* collector subscribes. Replay 0 — late subscribers shouldn't replay stale
* navigation intents from prior process lifetimes.
* A buffered [Channel] is intentional here: notification taps are consumed in
* `MainActivity.onCreate` before Compose installs RelayApp's collector. A
* replay-0 SharedFlow drops those cold-start requests when no subscriber exists.
* The channel retains up to four one-shot routes and hands each to the single
* app-root collector exactly once.
*/
object NavRouteRequest {
private val _requests = MutableSharedFlow<String>(
replay = 0,
extraBufferCapacity = 4,
)
private val channel = Channel<String>(capacity = 4)
val requests: SharedFlow<String> = _requests.asSharedFlow()
val requests: Flow<String> = channel.receiveAsFlow()
/** Fire-and-forget emit. Safe to call from any thread, including the main thread. */
fun tryRequest(route: String): Boolean = _requests.tryEmit(route)
fun tryRequest(route: String): Boolean = channel.trySend(route).isSuccess
suspend fun request(route: String) {
_requests.emit(route)
channel.send(route)
}
}
@@ -31,8 +31,10 @@ import com.hermesandroid.relay.data.MediaSettings
import com.hermesandroid.relay.data.MediaSettingsRepository
import com.hermesandroid.relay.data.MessageDeliveryStatus
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.data.applyMessageReaction
import com.hermesandroid.relay.data.parseChatQuotedPrompt
import com.hermesandroid.relay.data.Profile
import com.hermesandroid.relay.data.ProactiveInboxEntry
import com.hermesandroid.relay.data.RealtimeConversationContextMessage
import com.hermesandroid.relay.data.RealtimeTurnTrace
import com.hermesandroid.relay.data.SessionActivityState
@@ -92,6 +94,7 @@ import com.hermesandroid.relay.network.upstream.ChatHandler
import com.hermesandroid.relay.network.shared.LocalDispatchResult
import com.hermesandroid.relay.network.upstream.formatPhoneActionResult
import com.hermesandroid.relay.network.upstream.models.MessageItem
import com.hermesandroid.relay.network.upstream.models.parseMessageReactions
import com.hermesandroid.relay.network.upstream.SESSION_MESSAGE_PAGE_SIZE
import com.hermesandroid.relay.network.upstream.SessionMessageLoadMode
import com.hermesandroid.relay.network.upstream.models.SessionItem
@@ -135,6 +138,7 @@ import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.contentOrNull
import kotlinx.serialization.json.intOrNull
import kotlinx.serialization.json.longOrNull
import okhttp3.sse.EventSource
import java.util.UUID
import java.util.concurrent.ConcurrentHashMap
@@ -832,7 +836,7 @@ class ChatViewModel : ViewModel() {
_approvalModeCapability.value = GatewayApprovalModeCapability.Unknown
return
}
val launchProfileOwned = sessionProfileNameProvider() == null
val launchProfileOwned = currentSessionProfileName() == null
_approvalModeWritable.value = launchProfileOwned
_approvalModeReadOnlyForProfile.value = !launchProfileOwned
if (!launchProfileOwned) {
@@ -982,7 +986,7 @@ class ChatViewModel : ViewModel() {
if (pairs.isEmpty()) return
val generation = relayCapabilityGeneration.incrementAndGet()
val profileKey = reasoningCapabilityContextKey()
val profile = sessionProfileNameProvider()
val profile = currentSessionProfileName()
viewModelScope.launch {
val result = relay.fetchModelCapabilities(
models = pairs.map {
@@ -1153,7 +1157,7 @@ class ChatViewModel : ViewModel() {
// here skips the setModel below, leaving the gateway on its true
// server-configured default.
val defaultModel = AgentDisplay.requestModelName(
(effectiveProfileProvider() ?: selectedProfileProvider())?.model
(openedSessionDisplayProfile ?: effectiveProfileProvider() ?: selectedProfileProvider())?.model
?: _serverModelName.value,
)
if (!defaultModel.isNullOrBlank()) {
@@ -1461,7 +1465,8 @@ class ChatViewModel : ViewModel() {
* new profile's agent. SSE turns already carry the profile per-request as
* `profileName`. [profile] = the new pick; null = the default profile.
*/
fun activateGatewayProfile(profile: Profile?) {
fun activateGatewayProfile(profile: Profile?, refreshModelOptions: Boolean = true) {
clearOpenedSessionOwner()
val gateway = gatewayClient ?: return
if (streamingEndpoint != "gateway") return
// A profile switch is a UI/context detach, not a Stop action. Preserve
@@ -1530,11 +1535,13 @@ class ChatViewModel : ViewModel() {
// config.get would read the launch/global profile's effort (wrong
// scope). It's left unknown above and confirmed by session.info on the
// first turn — the same honesty discipline yolo/fast use.
viewModelScope.launch {
gateway.modelOptions().onSuccess {
_modelProviders.value = it.providers
_gatewayCurrentModel.value = it.currentModel
_gatewayCurrentProvider.value = it.currentProvider
if (refreshModelOptions) {
viewModelScope.launch {
gateway.modelOptions().onSuccess {
_modelProviders.value = it.providers
_gatewayCurrentModel.value = it.currentModel
_gatewayCurrentProvider.value = it.currentProvider
}
}
}
refreshActiveAgentName()
@@ -1662,7 +1669,7 @@ class ChatViewModel : ViewModel() {
}
// Bind each gateway session.create/resume to the currently-selected
// profile (pulled live) — the upstream gateway builds the agent from it.
client?.sessionProfileProvider = { sessionProfileNameProvider() }
client?.sessionProfileProvider = { currentSessionProfileName() }
// Bind the in-chat picks onto each fresh session.create so a brand-new
// chat actually runs on the picked model/provider AND the chosen
// reasoning effort / fast tier (not the global default) — and so setting
@@ -2322,15 +2329,33 @@ class ChatViewModel : ViewModel() {
private val _transientNotice = MutableSharedFlow<String>(extraBufferCapacity = 8)
val transientNotice: SharedFlow<String> = _transientNotice.asSharedFlow()
fun reactToNewest(role: MessageRole, emoji: String?) {
fun reactToMessage(message: ChatMessage, emoji: String?) {
val gateway = gatewayClient ?: return
val role = message.role
if (role != MessageRole.USER && role != MessageRole.ASSISTANT) return
val handler = chatHandler ?: return
val snapshot = messages.value.firstOrNull { it.uiKey == message.uiKey }?.reactions
?: message.reactions
handler.mutateMessage(message.uiKey) { current ->
current.copy(reactions = applyMessageReaction(current.reactions, emoji))
}
viewModelScope.launch {
gateway.reactToNewest(role.name.lowercase(), emoji).fold(
onSuccess = {
gateway.reactToMessage(message.rowId, role.name.lowercase(), emoji).fold(
onSuccess = { result ->
val persisted = parseMessageReactions(result["reactions"])
val rowId = (result["row_id"] as? JsonPrimitive)?.longOrNull
handler.mutateMessage(message.uiKey) { current ->
current.copy(
rowId = rowId ?: current.rowId,
reactions = persisted,
)
}
_transientNotice.tryEmit(if (emoji == null) "Reaction removed." else "Reaction added.")
},
onFailure = { error ->
handler.mutateMessage(message.uiKey) { current ->
current.copy(reactions = snapshot)
}
if ((error as? GatewayRpcException)?.code == -32601) {
_messageReactionsSupported.value = false
_transientNotice.tryEmit("Message reactions aren't supported by this gateway.")
@@ -2491,6 +2516,22 @@ class ChatViewModel : ViewModel() {
}
private var displayAliasProvider: () -> String? = { null }
private var activeProfileContextKey: String? = null
private val _openedSessionProfileName = MutableStateFlow<String?>(null)
val openedSessionProfileName: StateFlow<String?> = _openedSessionProfileName.asStateFlow()
private var openedSessionDisplayProfile: Profile? = null
/**
* Profile namespace owned by the conversation currently on screen. Opening a
* row from the global All Profiles browser must not mutate the persistent
* profile selector, but resume/history/send still need the row's exact owner.
*/
private fun currentSessionProfileName(): String? =
_openedSessionProfileName.value ?: sessionProfileNameProvider()
private fun clearOpenedSessionOwner() {
_openedSessionProfileName.value = null
openedSessionDisplayProfile = null
}
/** Process ownership is profile+session scoped; stored IDs alone are not globally unique. */
private fun selectBackgroundProcessSession(
@@ -2594,14 +2635,14 @@ class ChatViewModel : ViewModel() {
* read that profile's own DB. Returns `null` off the dashboard surface.
*/
private var profileMessageLoader:
(suspend (String, SessionMessageLoadMode) -> Result<List<MessageItem>>?)? = null
(suspend (String?, String, SessionMessageLoadMode) -> Result<List<MessageItem>>?)? = null
fun setProfileMessageLoader(loader: suspend (String) -> Result<List<MessageItem>>?) {
profileMessageLoader = { sessionId, _ -> loader(sessionId) }
profileMessageLoader = { _, sessionId, _ -> loader(sessionId) }
}
fun setProfileMessageLoaderWithMode(
loader: suspend (String, SessionMessageLoadMode) -> Result<List<MessageItem>>?,
loader: suspend (String?, String, SessionMessageLoadMode) -> Result<List<MessageItem>>?,
) {
profileMessageLoader = loader
}
@@ -2616,9 +2657,10 @@ class ChatViewModel : ViewModel() {
sessionId: String,
requireProfileScope: Boolean = false,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
profileName: String? = currentSessionProfileName(),
): List<MessageItem> {
if (streamingEndpoint == "gateway") {
return loadGatewaySessionHistory(sessionId, requireProfileScope, mode)
return loadGatewaySessionHistory(sessionId, requireProfileScope, mode, profileName)
}
return apiClient?.getMessages(sessionId, mode) ?: emptyList()
}
@@ -2633,8 +2675,9 @@ class ChatViewModel : ViewModel() {
sessionId: String,
requireProfileScope: Boolean = false,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
profileName: String? = currentSessionProfileName(),
): List<MessageItem> {
val scoped = profileMessageLoader?.invoke(sessionId, mode)
val scoped = profileMessageLoader?.invoke(profileName, sessionId, mode)
if (scoped != null) {
// A gateway profile owns a distinct state.db. Never fall through to
// the launch/default API database when its scoped read fails: an
@@ -2791,7 +2834,7 @@ class ChatViewModel : ViewModel() {
approvalModeRevision.incrementAndGet()
_approvalMode.value = mode
_approvalModeCapability.value = GatewayApprovalModeCapability.Supported
val launchProfileOwned = sessionProfileNameProvider() == null
val launchProfileOwned = currentSessionProfileName() == null
_approvalModeWritable.value = launchProfileOwned
_approvalModeReadOnlyForProfile.value = !launchProfileOwned
}
@@ -2932,6 +2975,12 @@ class ChatViewModel : ViewModel() {
fun injectThreadMessage(msg: ProactiveMessage): Boolean {
val handler = chatHandler ?: return false
val msgChatId = msg.chatId?.takeIf { it.isNotBlank() }
pendingThread?.let { pending ->
if (msgChatId == null || msgChatId == pending.chatId) {
handler.addAgentThreadMessage(msg.text, msg.messageId, msg.title)
return true
}
}
// A freshly-created thread whose real session we're still switching to:
// show the agent's first reply in the draft view now (the switch
// reconciles it from history). Covers the gap before currentSessionId is
@@ -3245,7 +3294,50 @@ class ChatViewModel : ViewModel() {
}
}
fun openProfileSession(
profileName: String,
profile: Profile?,
contextKey: String,
sessionId: String,
) {
// Detach the old live gateway session without reading launch/global
// model options: session.info for the resumed owner is authoritative.
activateGatewayProfile(profile, refreshModelOptions = false)
_openedSessionProfileName.value = profileName
openedSessionDisplayProfile = profile
refreshActiveAgentName(profile, relabelGenericMessages = true)
switchProfileContextInternal(contextKey, sessionId, persistLastSession = false)
}
/**
* Start a fresh draft owned by an explicit profile without changing the
* persistent profile selector. The All Profiles browser uses this for its
* New chat action, where upstream's literal `default` profile must win over
* the server's sticky active profile.
*/
fun createProfileChat(
profileName: String,
profile: Profile?,
contextKey: String,
) {
activateGatewayProfile(profile, refreshModelOptions = false)
_openedSessionProfileName.value = profileName
openedSessionDisplayProfile = profile
refreshActiveAgentName(profile, relabelGenericMessages = true)
switchProfileContextInternal(contextKey, sessionId = null, persistLastSession = false)
AppAnalytics.onSessionCreated()
}
fun switchProfileContext(contextKey: String, sessionId: String?) {
clearOpenedSessionOwner()
switchProfileContextInternal(contextKey, sessionId)
}
private fun switchProfileContextInternal(
contextKey: String,
sessionId: String?,
persistLastSession: Boolean = true,
) {
val handler = chatHandler ?: return
val isInitialContextBinding = activeProfileContextKey == null
handler.activeAgentName = currentAgentDisplayName()
@@ -3279,6 +3371,7 @@ class ChatViewModel : ViewModel() {
if (!isInitialContextBinding) releaseTurnForNavigation(handler)
cancelAnswerRecovery(settleUi = false)
val loadGeneration = historyLoadGeneration.incrementAndGet()
val sessionProfileName = currentSessionProfileName()
sessionRefreshGeneration.incrementAndGet()
sessionRefreshJob?.cancel()
_isLoadingSessions.value = false
@@ -3328,7 +3421,7 @@ class ChatViewModel : ViewModel() {
handler.setSessionId(sessionId)
publishQueuedMessages()
selectBackgroundProcessSession(sessionId, contextKey)
if (sessionId != null) {
if (sessionId != null && persistLastSession) {
onSessionChanged?.invoke(sessionId)
}
@@ -3361,7 +3454,7 @@ class ChatViewModel : ViewModel() {
false
}
if (!recovered) {
val messages = loadSessionHistory(sessionId)
val messages = loadSessionHistory(sessionId, profileName = sessionProfileName)
if (stillCurrent()) {
handler.loadMessageHistory(messages)
if (streamingEndpoint == "gateway") gatewayClient?.prewarm(sessionId)
@@ -3550,6 +3643,9 @@ class ChatViewModel : ViewModel() {
fun createNewChat() {
val handler = chatHandler ?: return
clearOpenedSessionOwner()
pendingThread = null
creatingThread = null
// Gateway turns continue as detached siblings; SSE remains exclusive.
releaseTurnForNavigation(handler)
@@ -3666,6 +3762,43 @@ class ChatViewModel : ViewModel() {
onSessionChanged?.invoke(null)
}
/**
* Open an agent-initiated Thread before the gateway has a `source=phone`
* session for it. Outbound platform sends do not create gateway sessions;
* the first phone reply does. Until then the durable proactive inbox is the
* provisional transcript. The existing pending-thread send path promotes
* this draft to the real gateway session after the user's first reply.
*/
fun openProactiveThread(chatId: String, entries: List<ProactiveInboxEntry>) {
val handler = chatHandler ?: return
val normalizedChatId = chatId.ifBlank { "phone" }
val ordered = entries
.filter { (it.chatId ?: "phone") == normalizedChatId }
.sortedBy { it.receivedAt }
if (ordered.isEmpty()) return
releaseTurnForNavigation(handler)
cancelAnswerRecovery(settleUi = false)
historyLoadGeneration.incrementAndGet()
pendingThread = PendingThread(
chatId = normalizedChatId,
name = ordered.last().title.ifBlank { "Hermes" },
)
creatingThread = null
gatewayClient?.clearSession()
handler.setSessionId(null)
selectBackgroundProcessSession(null)
handler.clearMessages()
ordered.forEach { entry ->
handler.addAgentThreadMessage(entry.text, entry.id, entry.title)
}
_contextUsage.value = null
_contextWindow.value = null
dismissPendingAskNotification()
_pendingAsk.value = null
onSessionChanged?.invoke(null)
}
/**
* After a "+ New Thread" first send, poll the session list until the gateway
* has created the new `source=phone` session, then switch to it (loading its
@@ -3710,7 +3843,10 @@ class ChatViewModel : ViewModel() {
fun switchSession(sessionId: String) {
val handler = chatHandler ?: return
clearOpenedSessionOwner()
if (streamingEndpoint != "gateway" && apiClient == null) return
pendingThread = null
creatingThread = null
// Keep a Gateway sibling alive and detach its callbacks. SSE remains a
// single exclusive stream and is interrupted on navigation.
@@ -4767,7 +4903,7 @@ class ChatViewModel : ViewModel() {
private fun maybeNotifyInteraction(
sessionId: String,
ask: GatewayAsk,
profile: String? = sessionProfileNameProvider(),
profile: String? = currentSessionProfileName(),
) {
val context = appContext ?: return
InteractionRequestNotifier.notify(
@@ -4783,7 +4919,7 @@ class ChatViewModel : ViewModel() {
private fun cancelInteractionNotification(
sessionId: String,
ask: GatewayAsk,
profile: String? = sessionProfileNameProvider(),
profile: String? = currentSessionProfileName(),
) {
val context = appContext ?: return
InteractionRequestNotifier.cancel(context, sessionId, ask, profile)
@@ -7674,49 +7810,17 @@ class ChatViewModel : ViewModel() {
val gateway = gatewayClient
_steerableTurn.value = false
// Voice turns must deliver their interface + spoken-output-formatting
// context to the model, but the gateway prompt.submit RPC has NO
// system-message slot (it is bare text — see the else branch). Prepending
// to the user text would persist the instruction into the transcript.
// The SSE endpoints carry it in the non-persisted system_message field
// instead, so force any turn that has a per-turn interface context
// (set only by sendVoiceMessage) onto SSE. resolveSseFallback picks the
// best available SSE route.
// Gateway prompt.submit has no system-message slot, so its voice turn
// cannot carry the optional spoken-output formatting hint. Keep the turn
// on Gateway anyway: the API server is an optional fallback and may not
// be reachable from the phone. A working vanilla Gateway turn is more
// important than silently making standard voice depend on port 8642.
// SSE-selected connections still receive the interface context normally.
//
// Synthetic voice-intent and provider-answered realtime traces have the
// same gateway limitation — prompt.submit can't carry them — but on a
// gateway-primary phone "leave them for the next SSE turn" means *never*.
// Drain those voice-only traces by forcing this one turn onto the sessions
// SSE route, but ONLY when that is strictly safe:
// - an existing session id + the sessions fallback route (a stateless
// completions/runs detour would drop THIS turn from the transcript
// to save a trace — worse than deferring), and
// - the default profile (a non-default profile's gateway session lives
// in that profile's own state.db, which the shared api_server surface
// can't see — the sessions POST would 404 and fail the user's turn).
// Cost when it fires: one turn without live gateway thinking. The synced
// traces persist server-side, so this happens at most once per batch.
//
// Rich-card actions are different: the action itself is the current user
// turn. Keep it on the selected Gateway and defer its optional synthetic
// audit trace until a naturally selected SSE turn. A card must never
// activate or depend on the optional API fallback.
val sseDrainEndpoint = resolveSseFallback(handler)
val forceSseForVoiceTraceDrain =
client != null &&
(hasVoiceIntents || hasRealtimeTurns) &&
streamingEndpoint == "gateway" &&
profileName == null &&
sseDrainEndpoint == "sessions"
val effectiveEndpoint =
if (client != null &&
(interfaceContextPrompt != null || forceSseForVoiceTraceDrain) &&
streamingEndpoint == "gateway"
) {
sseDrainEndpoint
} else {
streamingEndpoint
}
// Synthetic local traces also wait for a naturally selected SSE turn.
// They are supplemental context and must never make a connected Gateway
// turn depend on the optional API server.
val effectiveEndpoint = streamingEndpoint
updateTurnCheckpointTransport(effectiveEndpoint)
// Remember whether this turn runs on the gateway client (vs an SSE
// EventSource) so a mid-turn route handoff doesn't cancel it — only the
@@ -7904,6 +8008,7 @@ class ChatViewModel : ViewModel() {
): String? {
val selectedProfile = selectedProfileProvider()
val effectiveProfile = effectiveProfileOverride
?: openedSessionDisplayProfile
?: displayProfileProvider()
?: effectiveProfileProvider()
?: selectedProfile
@@ -7912,7 +8017,11 @@ class ChatViewModel : ViewModel() {
selectedPersonality = _selectedPersonality.value,
defaultPersonality = _defaultPersonality.value,
connectionLabel = null,
localDisplayAlias = displayAliasProvider(),
localDisplayAlias = if (_openedSessionProfileName.value == null) {
displayAliasProvider()
} else {
null
},
).ifBlank { null }
}
@@ -1571,11 +1571,30 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
suspend fun listAllProfileSessions(limit: Int = 200): Result<List<SessionItem>>? =
profileController.listAllProfileSessions(limit)
suspend fun deleteSession(profileName: String, sessionId: String): Boolean =
profileController.deleteSession(profileName, sessionId)
suspend fun renameSession(profileName: String, sessionId: String, title: String): Boolean =
profileController.renameSession(profileName, sessionId, title)
suspend fun setSessionPinned(profileName: String, sessionId: String, pinned: Boolean): Boolean =
profileController.setSessionPinned(profileName, sessionId, pinned)
suspend fun setSessionArchived(profileName: String, sessionId: String, archived: Boolean): Boolean =
profileController.setSessionArchived(profileName, sessionId, archived)
suspend fun loadProfileScopedMessages(
sessionId: String,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
): Result<List<MessageItem>>? = profileController.loadProfileScopedMessages(sessionId, mode)
suspend fun loadProfileScopedMessages(
profileName: String?,
sessionId: String,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
): Result<List<MessageItem>>? =
profileController.loadProfileScopedMessages(profileName, sessionId, mode)
/**
* Delete a session scoped to the ACTIVE PROFILE via the dashboard
* `DELETE /api/sessions/{id}?profile=` surface — the write twin of
@@ -1624,6 +1643,9 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
fun setProfileHidden(profileName: String?, hidden: Boolean) =
profileController.setProfileHidden(profileName, hidden)
fun setProfileColor(profileName: String, colorHex: String?) =
profileController.setProfileColor(profileName, colorHex)
fun resetProfilePresentation() = profileController.resetProfilePresentation()
/**
@@ -2362,6 +2384,7 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
text = msg.text,
receivedAt = msg.sentAt ?: System.currentTimeMillis(),
chatId = msg.chatId,
connectionId = connectionStore.activeConnectionId.value,
),
)
}
@@ -350,6 +350,38 @@ class ProfileController(
return dashboardClientFactory(connectionId, dashboardUrl).listAllProfileSessions(limit)
}
suspend fun deleteSession(profileName: String, sessionId: String): Boolean {
val connectionId = activeConnectionId.value ?: return false
val dashboardUrl = activeDashboardUrlProvider() ?: return false
return dashboardClientFactory(connectionId, dashboardUrl)
.deleteSession(sessionId, profileName)
.isSuccess
}
suspend fun renameSession(profileName: String, sessionId: String, title: String): Boolean {
val connectionId = activeConnectionId.value ?: return false
val dashboardUrl = activeDashboardUrlProvider() ?: return false
return dashboardClientFactory(connectionId, dashboardUrl)
.renameSession(sessionId, title, profileName)
.isSuccess
}
suspend fun setSessionPinned(profileName: String, sessionId: String, pinned: Boolean): Boolean {
val connectionId = activeConnectionId.value ?: return false
val dashboardUrl = activeDashboardUrlProvider() ?: return false
return dashboardClientFactory(connectionId, dashboardUrl)
.setSessionPinned(sessionId, pinned, profileName)
.isSuccess
}
suspend fun setSessionArchived(profileName: String, sessionId: String, archived: Boolean): Boolean {
val connectionId = activeConnectionId.value ?: return false
val dashboardUrl = activeDashboardUrlProvider() ?: return false
return dashboardClientFactory(connectionId, dashboardUrl)
.setSessionArchived(sessionId, archived, profileName)
.isSuccess
}
/**
* A session's transcript, scoped to the active profile via the dashboard
* `/api/sessions/{id}/messages?profile=`. Returns `null` off the dashboard
@@ -358,10 +390,19 @@ class ProfileController(
suspend fun loadProfileScopedMessages(
sessionId: String,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
): Result<List<MessageItem>>? = loadProfileScopedMessages(
profileName = resolveSessionProfileName(),
sessionId = sessionId,
mode = mode,
)
suspend fun loadProfileScopedMessages(
profileName: String?,
sessionId: String,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
): Result<List<MessageItem>>? {
val connectionId = activeConnectionId.value ?: return null
val dashboardUrl = activeDashboardUrlProvider() ?: return null
val profileName = resolveSessionProfileName()
return dashboardClientFactory(connectionId, dashboardUrl)
.getSessionMessages(sessionId, profileName, mode)
}
@@ -583,6 +624,24 @@ class ProfileController(
}
}
/** Persist or clear a local cosmetic accent for a named profile. */
fun setProfileColor(profileName: String, colorHex: String?) {
val connectionId = activeConnectionId.value ?: return
val key = profileName.trim()
if (key.isBlank() || key.equals("default", ignoreCase = true)) return
scope.launch {
profilePresentationWriteMutex.withLock {
val updated = profilePresentationStore
.presentationFlow(connectionId)
.first()
.colors
.toMutableMap()
.apply { if (colorHex == null) remove(key) else put(key, colorHex) }
profilePresentationStore.setColors(connectionId, updated)
}
}
}
fun resetProfilePresentation() {
val connectionId = activeConnectionId.value ?: return
scope.launch {
@@ -16,6 +16,7 @@ import com.hermesandroid.relay.network.upstream.isNativeDashboardTransportEligib
import com.hermesandroid.relay.network.upstream.ServerCapabilities
import com.hermesandroid.relay.network.upstream.resolveStreamingEndpointPreference
import com.hermesandroid.relay.network.upstream.trustedDashboardBearerAuthOrNull
import com.hermesandroid.relay.network.shutdownOffMainThread
import java.util.concurrent.ConcurrentHashMap
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
@@ -80,6 +81,10 @@ class UpstreamTransportController(
/** Applies pairing-bound TLS to a standard authenticated client when needed. */
private val pinnedClientProvider: (String, okhttp3.OkHttpClient) -> okhttp3.OkHttpClient? =
{ _, _ -> null },
private val dashboardHttpClientFactory:
(DashboardCookieStore, DashboardBearerAuth?) -> okhttp3.OkHttpClient = { cookieStore, bearerAuth ->
DashboardApiClient.defaultClient(cookieStore, bearerAuth)
},
) {
// --- Per-connection dashboard cookie stores ----------------------------
@@ -151,9 +156,9 @@ class UpstreamTransportController(
* scoping, the gateway client, standard-API setup probe) route through.
*/
fun dashboardClientFor(connectionId: String, dashboardUrl: String): DashboardApiClient {
val base = DashboardApiClient.defaultClient(
cookieStore = dashboardCookieStoreFor(connectionId),
bearerAuth = bearerAuthForTrustedDashboard(connectionId, dashboardUrl),
val base = dashboardHttpClientFactory(
dashboardCookieStoreFor(connectionId),
bearerAuthForTrustedDashboard(connectionId, dashboardUrl),
)
return DashboardApiClient(
baseUrl = dashboardUrl,
@@ -167,9 +172,9 @@ class UpstreamTransportController(
* no active connection (the standard-voice probe path).
*/
fun dashboardClientForActive(dashboardUrl: String): DashboardApiClient {
val base = DashboardApiClient.defaultClient(
cookieStore = activeDashboardCookieStore() ?: InMemoryDashboardCookieStore(),
bearerAuth = activeConnectionIdProvider()?.let {
val base = dashboardHttpClientFactory(
activeDashboardCookieStore() ?: InMemoryDashboardCookieStore(),
activeConnectionIdProvider()?.let {
bearerAuthForTrustedDashboard(it, dashboardUrl)
},
)
@@ -219,9 +224,9 @@ class UpstreamTransportController(
disposeDashboardHttpClient(client)
dashboardHttpClientCache = null
}
val base = DashboardApiClient.defaultClient(
cookieStore = activeDashboardCookieStore() ?: InMemoryDashboardCookieStore(),
bearerAuth = activeConnectionIdProvider()?.let { activeId ->
val base = dashboardHttpClientFactory(
activeDashboardCookieStore() ?: InMemoryDashboardCookieStore(),
activeConnectionIdProvider()?.let { activeId ->
bearerAuthForTrustedDashboard(activeId, dashboardUrl)
},
)
@@ -250,10 +255,12 @@ class UpstreamTransportController(
}
private fun disposeDashboardHttpClient(client: okhttp3.OkHttpClient) {
client.dispatcher.cancelAll()
client.connectionPool.evictAll()
runCatching { client.cache?.close() }
client.dispatcher.executorService.shutdown()
shutdownOffMainThread("DashboardHttpClient-shutdown") {
client.dispatcher.cancelAll()
client.connectionPool.evictAll()
runCatching { client.cache?.close() }
client.dispatcher.executorService.shutdown()
}
}
// --- Gateway availability ----------------------------------------------
@@ -3884,6 +3884,35 @@
<string name="dashboard_whatsapp_saved">WhatsApp ativado; o Hermes iniciou a reinicialização do gateway.</string>
<string name="drawer_all_profiles">Todos os perfis</string>
<string name="drawer_no_profile_sessions">Nenhuma sessão de perfil correspondente.</string>
<string name="drawer_customize_sessions">Personalizar sessões</string>
<string name="drawer_group_by">Agrupar por</string>
<string name="drawer_order_by">Ordenar por</string>
<string name="drawer_show_metadata">Mostrar nas linhas</string>
<string name="drawer_show_details">Mostrar detalhes</string>
<string name="drawer_filters">Filtros</string>
<string name="drawer_project_home">Início</string>
<string name="drawer_expand_project">Expandir %1$s</string>
<string name="drawer_collapse_project">Recolher %1$s</string>
<plurals name="drawer_project_session_count">
<item quantity="one">%1$d sessão</item>
<item quantity="other">%1$d sessões</item>
</plurals>
<plurals name="drawer_profile_count">
<item quantity="one">%1$d perfil</item>
<item quantity="other">%1$d perfis</item>
</plurals>
<string name="drawer_filter_status">Status</string>
<string name="drawer_filter_profile">Perfil</string>
<string name="drawer_filter_project">Projeto</string>
<string name="drawer_filter_pull_request">Pull request</string>
<string name="drawer_option_updated">Atualizado</string>
<string name="drawer_option_profile">Perfil</string>
<string name="drawer_option_tokens">Tokens</string>
<string name="drawer_option_cost">Custo</string>
<string name="drawer_reset_filters">Redefinir</string>
<string name="drawer_profile_colors">Cores dos perfis</string>
<string name="drawer_profile_color_auto">Automática</string>
<string name="drawer_profile_color_set">Definir a cor do perfil %1$s como %2$s</string>
<string name="drawer_close">Fechar</string>
<string name="profile_inspector_gateway_settings">Configurações do perfil do Gateway</string>
<string name="profile_inspector_gateway_settings_hint">Salvo diretamente pelo Hermes para este perfil selecionado.</string>
@@ -3972,6 +3972,33 @@
<string name="dashboard_whatsapp_saved">WhatsApp 已启用;Hermes 已开始重启网关。</string>
<string name="drawer_all_profiles">所有配置文件</string>
<string name="drawer_no_profile_sessions">没有匹配的配置文件会话。</string>
<string name="drawer_customize_sessions">自定义会话</string>
<string name="drawer_group_by">分组方式</string>
<string name="drawer_order_by">排序方式</string>
<string name="drawer_show_metadata">在行中显示</string>
<string name="drawer_show_details">显示详细信息</string>
<string name="drawer_filters">筛选器</string>
<string name="drawer_project_home">主页</string>
<string name="drawer_expand_project">展开%1$s</string>
<string name="drawer_collapse_project">折叠%1$s</string>
<plurals name="drawer_project_session_count">
<item quantity="other">%1$d 个会话</item>
</plurals>
<plurals name="drawer_profile_count">
<item quantity="other">%1$d 个配置文件</item>
</plurals>
<string name="drawer_filter_status">状态</string>
<string name="drawer_filter_profile">配置文件</string>
<string name="drawer_filter_project">项目</string>
<string name="drawer_filter_pull_request">拉取请求</string>
<string name="drawer_option_updated">更新时间</string>
<string name="drawer_option_profile">配置文件</string>
<string name="drawer_option_tokens">令牌</string>
<string name="drawer_option_cost">费用</string>
<string name="drawer_reset_filters">重置</string>
<string name="drawer_profile_colors">配置文件颜色</string>
<string name="drawer_profile_color_auto">自动</string>
<string name="drawer_profile_color_set">将 %1$s 配置文件颜色设为 %2$s</string>
<string name="drawer_close">关闭</string>
<string name="profile_inspector_gateway_settings">Gateway 配置文件设置</string>
<string name="profile_inspector_gateway_settings_hint">通过 Hermes 直接保存到当前所选配置文件。</string>
+29
View File
@@ -4044,6 +4044,35 @@
<string name="dashboard_whatsapp_saved">WhatsApp aktiviert; Hermes hat einen Gateway-Neustart gestartet.</string>
<string name="drawer_all_profiles">Alle Profile</string>
<string name="drawer_no_profile_sessions">Keine passenden Profilsitzungen.</string>
<string name="drawer_customize_sessions">Sitzungen anpassen</string>
<string name="drawer_group_by">Gruppieren nach</string>
<string name="drawer_order_by">Sortieren nach</string>
<string name="drawer_show_metadata">In Zeilen anzeigen</string>
<string name="drawer_show_details">Details anzeigen</string>
<string name="drawer_filters">Filter</string>
<string name="drawer_project_home">Startseite</string>
<string name="drawer_expand_project">%1$s erweitern</string>
<string name="drawer_collapse_project">%1$s reduzieren</string>
<plurals name="drawer_project_session_count">
<item quantity="one">%1$d Sitzung</item>
<item quantity="other">%1$d Sitzungen</item>
</plurals>
<plurals name="drawer_profile_count">
<item quantity="one">%1$d Profil</item>
<item quantity="other">%1$d Profile</item>
</plurals>
<string name="drawer_filter_status">Status</string>
<string name="drawer_filter_profile">Profil</string>
<string name="drawer_filter_project">Projekt</string>
<string name="drawer_filter_pull_request">Pull Request</string>
<string name="drawer_option_updated">Aktualisiert</string>
<string name="drawer_option_profile">Profil</string>
<string name="drawer_option_tokens">Tokens</string>
<string name="drawer_option_cost">Kosten</string>
<string name="drawer_reset_filters">Zurücksetzen</string>
<string name="drawer_profile_colors">Profilfarben</string>
<string name="drawer_profile_color_auto">Automatisch</string>
<string name="drawer_profile_color_set">Profilfarbe von %1$s auf %2$s setzen</string>
<string name="drawer_close">Schließen</string>
<string name="profile_inspector_gateway_settings">Gateway-Profileinstellungen</string>
<string name="profile_inspector_gateway_settings_hint">Wird für dieses ausgewählte Profil direkt über Hermes gespeichert.</string>
+29
View File
@@ -3729,6 +3729,35 @@
<string name="dashboard_whatsapp_saved">WhatsApp activado; Hermes inició un reinicio del gateway.</string>
<string name="drawer_all_profiles">Todos los perfiles</string>
<string name="drawer_no_profile_sessions">No hay sesiones de perfil coincidentes.</string>
<string name="drawer_customize_sessions">Personalizar sesiones</string>
<string name="drawer_group_by">Agrupar por</string>
<string name="drawer_order_by">Ordenar por</string>
<string name="drawer_show_metadata">Mostrar en las filas</string>
<string name="drawer_show_details">Mostrar detalles</string>
<string name="drawer_filters">Filtros</string>
<string name="drawer_project_home">Inicio</string>
<string name="drawer_expand_project">Expandir %1$s</string>
<string name="drawer_collapse_project">Contraer %1$s</string>
<plurals name="drawer_project_session_count">
<item quantity="one">%1$d sesión</item>
<item quantity="other">%1$d sesiones</item>
</plurals>
<plurals name="drawer_profile_count">
<item quantity="one">%1$d perfil</item>
<item quantity="other">%1$d perfiles</item>
</plurals>
<string name="drawer_filter_status">Estado</string>
<string name="drawer_filter_profile">Perfil</string>
<string name="drawer_filter_project">Proyecto</string>
<string name="drawer_filter_pull_request">Solicitud de incorporación</string>
<string name="drawer_option_updated">Actualizado</string>
<string name="drawer_option_profile">Perfil</string>
<string name="drawer_option_tokens">Tokens</string>
<string name="drawer_option_cost">Coste</string>
<string name="drawer_reset_filters">Restablecer</string>
<string name="drawer_profile_colors">Colores de perfil</string>
<string name="drawer_profile_color_auto">Automático</string>
<string name="drawer_profile_color_set">Establecer el color del perfil %1$s en %2$s</string>
<string name="drawer_close">Cerrar</string>
<string name="profile_inspector_gateway_settings">Ajustes del perfil de Gateway</string>
<string name="profile_inspector_gateway_settings_hint">Se guarda directamente mediante Hermes para este perfil seleccionado.</string>
+29
View File
@@ -4043,6 +4043,35 @@
<string name="dashboard_whatsapp_saved">WhatsApp を有効にしました。Hermes がゲートウェイの再起動を開始しました。</string>
<string name="drawer_all_profiles">すべてのプロファイル</string>
<string name="drawer_no_profile_sessions">一致するプロファイルセッションはありません。</string>
<string name="drawer_customize_sessions">セッションをカスタマイズ</string>
<string name="drawer_group_by">グループ化</string>
<string name="drawer_order_by">並び順</string>
<string name="drawer_show_metadata">行に表示</string>
<string name="drawer_show_details">詳細を表示</string>
<string name="drawer_filters">フィルター</string>
<string name="drawer_project_home">ホーム</string>
<string name="drawer_expand_project">%1$sを展開</string>
<string name="drawer_collapse_project">%1$sを折りたたむ</string>
<plurals name="drawer_project_session_count">
<item quantity="one">%1$d件のセッション</item>
<item quantity="other">%1$d件のセッション</item>
</plurals>
<plurals name="drawer_profile_count">
<item quantity="one">%1$d件のプロファイル</item>
<item quantity="other">%1$d件のプロファイル</item>
</plurals>
<string name="drawer_filter_status">ステータス</string>
<string name="drawer_filter_profile">プロフィール</string>
<string name="drawer_filter_project">プロジェクト</string>
<string name="drawer_filter_pull_request">プルリクエスト</string>
<string name="drawer_option_updated">更新日時</string>
<string name="drawer_option_profile">プロフィール</string>
<string name="drawer_option_tokens">トークン</string>
<string name="drawer_option_cost">コスト</string>
<string name="drawer_reset_filters">リセット</string>
<string name="drawer_profile_colors">プロフィールの色</string>
<string name="drawer_profile_color_auto">自動</string>
<string name="drawer_profile_color_set">%1$s のプロフィール色を %2$s に設定</string>
<string name="drawer_close">閉じる</string>
<string name="profile_inspector_gateway_settings">Gateway プロファイル設定</string>
<string name="profile_inspector_gateway_settings_hint">選択中のプロファイルに Hermes 経由で直接保存します。</string>
+33
View File
@@ -3765,6 +3765,39 @@
<string name="dashboard_whatsapp_saved">WhatsApp включён; Hermes начал перезапуск шлюза.</string>
<string name="drawer_all_profiles">Все профили</string>
<string name="drawer_no_profile_sessions">Нет подходящих сеансов профиля.</string>
<string name="drawer_customize_sessions">Настроить сеансы</string>
<string name="drawer_group_by">Группировать по</string>
<string name="drawer_order_by">Сортировать по</string>
<string name="drawer_show_metadata">Показывать в строках</string>
<string name="drawer_show_details">Показывать подробности</string>
<string name="drawer_filters">Фильтры</string>
<string name="drawer_project_home">Главная</string>
<string name="drawer_expand_project">Развернуть %1$s</string>
<string name="drawer_collapse_project">Свернуть %1$s</string>
<plurals name="drawer_project_session_count">
<item quantity="one">%1$d сеанс</item>
<item quantity="few">%1$d сеанса</item>
<item quantity="many">%1$d сеансов</item>
<item quantity="other">%1$d сеанса</item>
</plurals>
<plurals name="drawer_profile_count">
<item quantity="one">%1$d профиль</item>
<item quantity="few">%1$d профиля</item>
<item quantity="many">%1$d профилей</item>
<item quantity="other">%1$d профиля</item>
</plurals>
<string name="drawer_filter_status">Статус</string>
<string name="drawer_filter_profile">Профиль</string>
<string name="drawer_filter_project">Проект</string>
<string name="drawer_filter_pull_request">Запрос на слияние</string>
<string name="drawer_option_updated">Обновлено</string>
<string name="drawer_option_profile">Профиль</string>
<string name="drawer_option_tokens">Токены</string>
<string name="drawer_option_cost">Стоимость</string>
<string name="drawer_reset_filters">Сбросить</string>
<string name="drawer_profile_colors">Цвета профилей</string>
<string name="drawer_profile_color_auto">Автоматически</string>
<string name="drawer_profile_color_set">Установить цвет профиля %1$s: %2$s</string>
<string name="drawer_close">Закрыть</string>
<string name="profile_inspector_gateway_settings">Настройки профиля Gateway</string>
<string name="profile_inspector_gateway_settings_hint">Сохраняются напрямую через Hermes для выбранного профиля.</string>
+30 -1
View File
@@ -4063,8 +4063,37 @@
<string name="support_bundle_copied">Support information copied</string>
<string name="support_bundle_no_share">Support information copied — no app found to share to</string>
<string name="support_bundle_share_title">Share Hermes-Relay support information</string>
<string name="drawer_all_profiles">All profiles</string>
<string name="drawer_all_profiles">All Profiles</string>
<string name="drawer_no_profile_sessions">No matching profile sessions.</string>
<string name="drawer_customize_sessions">Customize sessions</string>
<string name="drawer_group_by">Group by</string>
<string name="drawer_order_by">Order by</string>
<string name="drawer_show_metadata">Show on rows</string>
<string name="drawer_show_details">Show details</string>
<string name="drawer_filters">Filters</string>
<string name="drawer_project_home">Home</string>
<string name="drawer_expand_project">Expand %1$s</string>
<string name="drawer_collapse_project">Collapse %1$s</string>
<plurals name="drawer_project_session_count">
<item quantity="one">%1$d session</item>
<item quantity="other">%1$d sessions</item>
</plurals>
<plurals name="drawer_profile_count">
<item quantity="one">%1$d profile</item>
<item quantity="other">%1$d profiles</item>
</plurals>
<string name="drawer_filter_status">Status</string>
<string name="drawer_filter_profile">Profile</string>
<string name="drawer_filter_project">Project</string>
<string name="drawer_filter_pull_request">Pull request</string>
<string name="drawer_option_updated">Updated</string>
<string name="drawer_option_profile">Profile</string>
<string name="drawer_option_tokens">Tokens</string>
<string name="drawer_option_cost">Cost</string>
<string name="drawer_reset_filters">Reset</string>
<string name="drawer_profile_colors">Profile colors</string>
<string name="drawer_profile_color_auto">Auto</string>
<string name="drawer_profile_color_set">Set %1$s profile color to %2$s</string>
<string name="drawer_close">Close</string>
<string name="profile_inspector_gateway_settings">Gateway profile settings</string>
<string name="profile_inspector_gateway_settings_hint">Saved directly through upstream Hermes for this selected profile.</string>
@@ -12,6 +12,18 @@ import org.junit.Test
*/
class ChatMessageTest {
@Test
fun messageReaction_replacesAndRetractsTheUsersTapback() {
val agent = MessageReaction("🔥", "agent", 1.0)
val heart = applyMessageReaction(listOf(agent), "❤️", at = 2.0)
assertEquals(listOf(agent, MessageReaction("❤️", "user", 2.0)), heart)
val replaced = applyMessageReaction(heart, "😂", at = 3.0)
assertEquals(listOf(agent, MessageReaction("😂", "user", 3.0)), replaced)
assertEquals(listOf(agent), applyMessageReaction(replaced, "😂", at = 4.0))
}
// --- ChatMessage creation with defaults ---
@Test
@@ -25,9 +25,14 @@ class ProfilePresentationStoreTest {
fun orderAndHiddenProfilesRoundTripPerConnection() = runBlocking {
store.setOrder("one", listOf("beta", "alpha"))
store.setHidden("one", setOf("gamma"))
store.setColors("one", mapOf("alpha" to "#356CFF"))
assertEquals(
ProfilePresentation(order = listOf("beta", "alpha"), hidden = setOf("gamma")),
ProfilePresentation(
order = listOf("beta", "alpha"),
hidden = setOf("gamma"),
colors = mapOf("alpha" to "#356CFF"),
),
store.presentationFlow("one").first(),
)
assertEquals(ProfilePresentation(), store.presentationFlow("two").first())
@@ -104,6 +109,7 @@ class ProfilePresentationStoreTest {
fun clearRemovesOnlyOneConnectionsPreferences() = runBlocking {
store.setOrder("one", listOf("beta"))
store.setHidden("one", setOf("alpha"))
store.setColors("one", mapOf("beta" to "#ED4E8B"))
store.setOrder("two", listOf("gamma"))
store.clear("one")
@@ -0,0 +1,130 @@
package com.hermesandroid.relay.network
import android.content.Context
import android.os.Looper
import com.hermesandroid.relay.network.relay.ChannelMultiplexer
import com.hermesandroid.relay.network.relay.ConnectionManager
import com.hermesandroid.relay.network.upstream.DashboardApiClient
import com.hermesandroid.relay.network.upstream.HermesApiClient
import com.hermesandroid.relay.viewmodel.connection.UpstreamTransportController
import io.mockk.mockk
import java.util.concurrent.AbstractExecutorService
import java.util.concurrent.CountDownLatch
import java.util.concurrent.ExecutorService
import java.util.concurrent.TimeUnit
import java.util.concurrent.atomic.AtomicBoolean
import java.util.concurrent.atomic.AtomicReference
import okhttp3.Dispatcher
import okhttp3.OkHttpClient
import org.junit.Assert.assertNotSame
import org.junit.Assert.assertSame
import org.junit.Assert.assertTrue
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
import org.robolectric.annotation.Config
/** Owner-level coverage for every production ConnectionPool.evictAll() teardown. */
@RunWith(RobolectricTestRunner::class)
@Config(sdk = [34])
class NetworkShutdownOwnerTest {
@Test
fun hermesApiClient_shutdownLeavesMainThread() {
val teardown = TrackingExecutorService()
val client = HermesApiClient(
baseUrl = "https://hermes.example.test",
apiKey = "test-key",
okHttpClient = clientWith(teardown),
)
client.shutdown()
teardown.assertShutdownOffMainThread()
}
@Test
fun dashboardApiClient_shutdownLeavesMainThread() {
val teardown = TrackingExecutorService()
val client = DashboardApiClient(
baseUrl = "https://hermes.example.test",
okHttpClient = clientWith(teardown),
)
client.shutdown()
teardown.assertShutdownOffMainThread()
}
@Test
fun connectionManager_shutdownLeavesMainThread() {
val teardown = TrackingExecutorService()
val manager = ConnectionManager(
multiplexer = ChannelMultiplexer(),
okHttpClientFactory = { clientWith(teardown) },
)
manager.shutdown()
teardown.assertShutdownOffMainThread()
}
@Test
fun upstreamTransportController_connectionSwitchResetLeavesMainThread() {
val dashboardUrl = "https://hermes.example.test"
val teardown = TrackingExecutorService()
val controller = UpstreamTransportController(
context = mockk<Context>(relaxed = true),
activeConnectionIdProvider = { null },
dashboardUrlProvider = { dashboardUrl },
gatewayKeepAliveProvider = { false },
dashboardHttpClientFactory = { _, _ -> clientWith(teardown) },
)
controller.dashboardHttpClientForActive(dashboardUrl)
controller.resetGatewayForConnectionSwitch()
teardown.assertShutdownOffMainThread()
}
private fun clientWith(executor: ExecutorService): OkHttpClient =
OkHttpClient.Builder()
.dispatcher(Dispatcher(executor))
.build()
private class TrackingExecutorService : AbstractExecutorService() {
private val shutdown = AtomicBoolean(false)
private val shutdownLatch = CountDownLatch(1)
private val shutdownThread = AtomicReference<Thread>()
override fun shutdown() {
shutdownThread.compareAndSet(null, Thread.currentThread())
shutdown.set(true)
shutdownLatch.countDown()
}
override fun shutdownNow(): MutableList<Runnable> {
shutdown()
return mutableListOf()
}
override fun isShutdown(): Boolean = shutdown.get()
override fun isTerminated(): Boolean = shutdown.get()
override fun awaitTermination(timeout: Long, unit: TimeUnit): Boolean =
shutdownLatch.await(timeout, unit)
override fun execute(command: Runnable) = command.run()
fun assertShutdownOffMainThread() {
assertSame(Looper.myLooper(), Looper.getMainLooper())
assertTrue("owner did not shut its OkHttp dispatcher down", shutdownLatch.await(5, TimeUnit.SECONDS))
assertNotSame(
"owner performed OkHttp teardown on the main thread",
Looper.getMainLooper().thread,
shutdownThread.get(),
)
}
}
}
@@ -975,6 +975,49 @@ class ChatHandlerTest {
assertEquals(42L, handler.messages.value.last().rowId)
}
@Test
fun loadMessageHistory_hydratesPersistedReactionsAndLetsServerWin() {
fun metadata(emoji: String) = buildJsonObject {
put("reactions", buildJsonArray {
add(buildJsonObject {
put("emoji", emoji)
put("author", "user")
put("at", 1_700_000_000.0)
})
})
}
handler.loadMessageHistory(
listOf(
MessageItem(
id = "assistant-1",
rowId = 42L,
role = "assistant",
content = JsonPrimitive("Reply"),
displayMetadata = metadata("❤️"),
),
),
)
assertEquals("❤️", handler.messages.value.single().reactions.single().emoji)
handler.mutateMessage("assistant-1") { message ->
message.copy(reactions = listOf(com.hermesandroid.relay.data.MessageReaction("👍", "user", 2.0)))
}
handler.loadMessageHistory(
listOf(
MessageItem(
id = "assistant-1",
rowId = 42L,
role = "assistant",
content = JsonPrimitive("Reply"),
displayMetadata = metadata("😂"),
),
),
)
assertEquals("😂", handler.messages.value.single().reactions.single().emoji)
}
@Test
fun rebindSurvivorUserRowIds_replacesPrefixAndClearsUnboundTurns() {
handler.loadMessageHistory(
@@ -1986,7 +1986,7 @@ class GatewayChatClientTest {
harness.awaitRpc("session.resume")
harness.awaitRpc("prompt.submit")
val result = runBlocking { client.reactToNewest("assistant", "👍") }
val result = runBlocking { client.reactToMessage(null, "assistant", "👍") }
assertTrue(result.isSuccess)
val params = harness.awaitRpc("message.react")
@@ -1997,6 +1997,20 @@ class GatewayChatClientTest {
assertFalse("row_id" in params)
}
@Test
fun `message reaction targets durable row when history provides it`() {
client.sendTurn("stored-1", "hi", null, Recorder().callbacks) {}
harness.awaitRpc("session.resume")
harness.awaitRpc("prompt.submit")
val result = runBlocking { client.reactToMessage(42L, "assistant", "❤️") }
assertTrue(result.isSuccess)
val params = harness.awaitRpc("message.react")
assertEquals(42L, (params["row_id"] as? JsonPrimitive)?.longOrNull)
assertFalse("newest_role" in params)
}
@Test
fun `pet thumbnail connects on demand and sends upstream params`() {
client.sessionProfileProvider = { "work" }
@@ -1,14 +1,17 @@
package com.hermesandroid.relay.network.upstream.models
import kotlinx.serialization.encodeToString
import kotlinx.serialization.SerializationException
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertNotNull
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Assert.assertThrows
import org.junit.Before
import org.junit.Test
@@ -71,7 +74,10 @@ class SessionModelsTest {
"message_count": 5,
"tool_call_count": 2,
"input_tokens": 100,
"output_tokens": 200
"output_tokens": 200,
"actual_cost_usd": 1.25,
"estimated_cost_usd": 1.50,
"is_active": true
}
""".trimIndent()
@@ -82,6 +88,11 @@ class SessionModelsTest {
assertEquals(1700000900.0, item.resolvedLastActivity!!, 0.001)
assertEquals(5, item.messageCount)
assertEquals(2, item.toolCallCount)
assertEquals(100, item.inputTokens)
assertEquals(200, item.outputTokens)
assertEquals(1.25, item.actualCostUsd!!, 0.001)
assertEquals(1.50, item.estimatedCostUsd!!, 0.001)
assertTrue(item.isActive)
}
@Test
@@ -94,6 +105,42 @@ class SessionModelsTest {
assertTrue(item.archived)
}
@Test
fun sessionItem_deserializesNumericSessionFlags() {
val item = json.decodeFromString<SessionItem>(
"""{"id":"s1","has_model_config":1,"pinned":0,"archived":1}""",
)
assertTrue(item.hasModelConfig)
assertFalse(item.pinned)
assertTrue(item.archived)
}
@Test
fun sessionItem_deserializesStringBooleanSessionFlags() {
val item = json.decodeFromString<SessionItem>(
"""{"id":"s1","has_model_config":"false","pinned":"1","archived":"0"}""",
)
assertFalse(item.hasModelConfig)
assertTrue(item.pinned)
assertFalse(item.archived)
}
@Test
fun sessionItem_rejectsNonBooleanSessionFlagValues() {
assertThrows(SerializationException::class.java) {
json.decodeFromString<SessionItem>(
"""{"id":"s1","pinned":2}""",
)
}
assertThrows(SerializationException::class.java) {
json.decodeFromString<SessionItem>(
"""{"id":"s1","archived":"yes"}""",
)
}
}
@Test
fun sessionItem_deserializesOptionalWorkspaceMetadata() {
val item = json.decodeFromString<SessionItem>(
@@ -320,6 +367,25 @@ class SessionModelsTest {
"""{"role":"user","row_id":{"unexpected":true}}""",
).rowId,
)
assertEquals(
75L,
json.decodeFromString<MessageItem>(
"""{"id":75,"role":"assistant"}""",
).resolvedRowId,
)
}
@Test
fun messageItem_readsReactionsFromObjectOrRawJsonMetadata() {
val objectBacked = json.decodeFromString<MessageItem>(
"""{"role":"assistant","display_metadata":{"reactions":[{"emoji":"👍","author":"user","at":1.0}]}}""",
)
val stringBacked = json.decodeFromString<MessageItem>(
"""{"role":"assistant","display_metadata":"{\"reactions\":[{\"emoji\":\"❤️\",\"author\":\"user\",\"at\":2.0}]}"}""",
)
assertEquals("👍", objectBacked.reactions.single().emoji)
assertEquals("❤️", stringBacked.reactions.single().emoji)
}
@Test
@@ -17,6 +17,7 @@ import androidx.test.ext.junit.runners.AndroidJUnit4
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.ChatQuoteReference
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.data.MessageReaction
import com.hermesandroid.relay.data.buildChatQuotedPrompt
import org.junit.Assert.assertEquals
import org.junit.Rule
@@ -117,6 +118,65 @@ class MessageBubbleInteractionTest {
compose.onNodeWithText("Quote in reply").assertIsDisplayed().assertHasClickAction()
}
@Test
fun reactionsStayOutsideBubbleAndOpenAsFloatingTapbacks() {
val reactions = mutableListOf<String?>()
val message = ChatMessage(
id = "assistant-reactions",
role = MessageRole.ASSISTANT,
content = "React to this response.",
timestamp = 1_700_000_000_000L,
)
compose.setContent {
MaterialTheme {
MessageBubble(
message = message,
onReact = { reactions += it },
)
}
}
compose.onNodeWithContentDescription("React with 👍").assertDoesNotExist()
compose.onNodeWithText("Remove reaction").assertDoesNotExist()
compose.onNodeWithContentDescription("assistant message: ${message.content}")
.performTouchInput { longClick() }
compose.onNodeWithContentDescription("React with 👍")
.assertIsDisplayed()
.assertHasClickAction()
.performClick()
compose.runOnIdle { assertEquals(listOf("👍"), reactions) }
compose.onNodeWithContentDescription("React with 👍").assertDoesNotExist()
}
@Test
fun landedReactionStaysPinnedToTheBubbleAndReopensPicker() {
val message = ChatMessage(
id = "assistant-landed-reaction",
role = MessageRole.ASSISTANT,
content = "A reacted response.",
timestamp = 1_700_000_000_000L,
reactions = listOf(MessageReaction("❤️", "user", 1_700_000_000.0)),
)
compose.setContent {
MaterialTheme {
MessageBubble(message = message, onReact = {})
}
}
compose.onNodeWithContentDescription("Reactions: ❤️")
.assertIsDisplayed()
.assertHasClickAction()
.performClick()
compose.onNodeWithContentDescription("React with ❤️").assertIsDisplayed()
compose.onNodeWithText("Remove reaction").assertIsDisplayed()
}
@Test
fun quotedReplyRendersStructuredReferenceAndKeepsMarkupOutOfActions() {
var quotedContent: String? = null
@@ -0,0 +1,135 @@
package com.hermesandroid.relay.ui.components
import com.hermesandroid.relay.data.ChatSession
import com.hermesandroid.relay.data.SessionActivityState
import org.junit.Assert.assertEquals
import org.junit.Test
class SessionDrawerPolicyTest {
@Test
fun `sessions are ungrouped by default`() {
assertEquals(SessionDrawerGrouping.None, SessionDrawerViewOptions().grouping)
}
@Test
fun `cross profile rows retain composite identity`() {
val alpha = row("alpha", "same")
val beta = row("beta", "same")
assertEquals("alpha:same", sessionRowKey(alpha))
assertEquals("beta:same", sessionRowKey(beta))
}
@Test
fun `profile project status and pull request filters compose`() {
val wanted = row(
profile = "work",
id = "wanted",
repo = "/src/hermes-relay",
prNumber = 347,
prState = "open",
)
val wrongProfile = row("personal", "other", repo = "/src/hermes-relay", prNumber = 22)
val wrongProject = row("work", "notes", repo = "/src/notes", prNumber = 23)
val states = mapOf(sessionRowKey(wanted) to SessionActivityState.NeedsInput)
val filtered = filterAndSortSessionRows(
rows = listOf(wrongProfile, wrongProject, wanted),
options = SessionDrawerViewOptions(
profiles = setOf("work"),
projects = setOf("hermes-relay"),
statuses = setOf(SessionDrawerStatus.NeedsInput),
pullRequests = setOf(SessionDrawerPrState.Open),
),
activityStates = states,
)
assertEquals(listOf("wanted"), filtered.map { it.session.sessionId })
}
@Test
fun `token and cost ordering use authoritative session metrics`() {
val small = row("default", "small", inputTokens = 10, outputTokens = 20, cost = 3.0)
val large = row("default", "large", inputTokens = 500, outputTokens = 600, cost = 1.0)
assertEquals(
listOf("large", "small"),
filterAndSortSessionRows(
listOf(small, large),
SessionDrawerViewOptions(ordering = SessionDrawerOrdering.Tokens),
).map { it.session.sessionId },
)
assertEquals(
listOf("small", "large"),
filterAndSortSessionRows(
listOf(small, large),
SessionDrawerViewOptions(ordering = SessionDrawerOrdering.Cost),
).map { it.session.sessionId },
)
}
@Test
fun `desktop style grouping supports project profile status and updated buckets`() {
val now = 10 * DAY
val working = row("work", "working", repo = "/src/hermes-relay", updatedAt = now - 1_000)
val idle = row("personal", "idle", repo = "/src/notes", updatedAt = now - 3 * DAY)
val states = mapOf(sessionRowKey(working) to SessionActivityState.Working)
assertEquals(
listOf("hermes-relay", "notes"),
groupSessionRows(listOf(working, idle), SessionDrawerGrouping.Project, states, now)
.mapNotNull { it.label },
)
assertEquals(
listOf("work", "personal"),
groupSessionRows(listOf(working, idle), SessionDrawerGrouping.Profile, states, now)
.mapNotNull { it.label },
)
assertEquals(
listOf("Working", "Idle"),
groupSessionRows(listOf(working, idle), SessionDrawerGrouping.Status, states, now)
.mapNotNull { it.label },
)
assertEquals(
listOf("Today", "Last 7 days"),
groupSessionRows(listOf(working, idle), SessionDrawerGrouping.Updated, states, now)
.mapNotNull { it.label },
)
assertEquals(
listOf(null),
groupSessionRows(listOf(working, idle), SessionDrawerGrouping.None, states, now)
.map { it.label },
)
}
private fun row(
profile: String,
id: String,
repo: String? = null,
prNumber: Int? = null,
prState: String? = null,
inputTokens: Int = 0,
outputTokens: Int = 0,
cost: Double? = null,
updatedAt: Long = 0L,
) = ProfileSessionRow(
profile = profile,
session = ChatSession(
sessionId = id,
title = id,
model = null,
gitRepoRoot = repo,
pullRequestNumber = prNumber,
pullRequestState = prState,
inputTokens = inputTokens,
outputTokens = outputTokens,
actualCostUsd = cost,
lastActivityAt = updatedAt,
),
)
private companion object {
const val DAY = 24L * 60L * 60L * 1_000L
}
}
@@ -8,10 +8,15 @@ import androidx.compose.ui.test.assertIsDisplayed
import androidx.compose.ui.test.hasText
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onNodeWithTag
import androidx.compose.ui.test.onNodeWithContentDescription
import androidx.compose.ui.test.onAllNodesWithContentDescription
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.performClick
import androidx.compose.ui.test.performScrollTo
import androidx.compose.ui.test.performScrollToNode
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.hermesandroid.relay.data.ChatSession
import com.hermesandroid.relay.ui.theme.ProfileAccentSwatches
import org.junit.Rule
import org.junit.Test
import org.junit.Assert.assertEquals
@@ -66,6 +71,42 @@ class SessionDrawerTest {
listOf("hermes-relay", "feature/android-session-context", "PR #134 · Open"),
sessionWorkLabels(session),
)
assertEquals(
listOf(
SessionWorkBadgeKind.PROJECT,
SessionWorkBadgeKind.BRANCH,
SessionWorkBadgeKind.PULL_REQUEST,
),
sessionWorkBadges(session).map(SessionWorkBadge::kind),
)
}
@Test
fun `session rows identify project and branch badges`() {
compose.setContent {
MaterialTheme {
SessionDrawerContent(
sessions = listOf(
ChatSession(
sessionId = "coding-1",
title = "Ship it",
model = null,
gitRepoRoot = "/work/hermes-relay",
gitBranch = "feature/chat-polish",
),
),
currentSessionId = null,
onNewChat = {},
onSelectSession = {},
onDeleteSession = {},
onRenameSession = { _, _ -> },
)
}
}
compose.onNodeWithText("hermes-relay").assertIsDisplayed()
compose.onNodeWithContentDescription("Project: hermes-relay").assertIsDisplayed()
compose.onNodeWithContentDescription("Branch: feature/chat-polish").assertIsDisplayed()
}
@Test
@@ -120,4 +161,168 @@ class SessionDrawerTest {
compose.onNodeWithText("Now latest").assertIsDisplayed()
}
@Test
fun `all profiles toggle renders duplicate ids together in the primary list`() {
var pinned: Triple<String, String, Boolean>? = null
compose.setContent {
MaterialTheme {
SessionDrawerContent(
sessions = listOf(ChatSession("same", "Current", null)),
currentSessionId = null,
activeProfileName = "alpha",
allProfilesSupported = true,
allProfileSessions = listOf(
ProfileSessionRow("alpha", ChatSession("same", "Alpha session", null)),
ProfileSessionRow("beta", ChatSession("same", "Beta session", null)),
),
onRefreshAllProfiles = {},
onSelectProfileSession = { _, _ -> },
onSetProfileSessionPinned = { profile, sessionId, value ->
pinned = Triple(profile, sessionId, value)
},
onNewChat = {},
onSelectSession = {},
onDeleteSession = {},
onRenameSession = { _, _ -> },
)
}
}
compose.onNodeWithText("All Profiles").performClick()
compose.onNodeWithText("Alpha session").assertIsDisplayed()
compose.onNodeWithText("Beta session").assertIsDisplayed()
compose.onAllNodesWithContentDescription("Session actions")[0]
.assertIsDisplayed()
.performClick()
compose.onNodeWithText("Pin session").performClick()
compose.runOnIdle { assertEquals(Triple("alpha", "same", true), pinned) }
}
@Test
fun `opening an owned session keeps all profiles browsing selected`() {
var scopeTitle by mutableStateOf("Mizu Sessions")
compose.setContent {
MaterialTheme {
SessionDrawerContent(
sessions = emptyList(),
currentSessionId = null,
scopeTitle = scopeTitle,
activeProfileName = "mizu",
allProfilesSupported = true,
allProfileSessions = listOf(
ProfileSessionRow("mizu", ChatSession("m", "Mizu chat", null)),
ProfileSessionRow("x-bot", ChatSession("x", "X Bot chat", null)),
),
onRefreshAllProfiles = {},
onSelectProfileSession = { _, _ -> scopeTitle = "X Bot Sessions" },
onNewChat = {},
onSelectSession = {},
onDeleteSession = {},
onRenameSession = { _, _ -> },
)
}
}
compose.onNodeWithText("All Profiles").performClick()
compose.onNodeWithText("X Bot chat").performClick()
compose.onNodeWithText("Mizu chat").assertIsDisplayed()
compose.onNodeWithText("X Bot Sessions").assertDoesNotExist()
compose.onNodeWithText("2 profiles · 2 sessions").assertIsDisplayed()
}
@Test
fun `new chat from all profiles requests an explicit default draft`() {
var scopedNewChats = 0
var defaultNewChats = 0
compose.setContent {
MaterialTheme {
SessionDrawerContent(
sessions = emptyList(),
currentSessionId = null,
allProfilesSupported = true,
allProfileSessions = listOf(
ProfileSessionRow("default", ChatSession("d", "Default chat", null)),
),
onRefreshAllProfiles = {},
onSelectProfileSession = { _, _ -> },
onNewChat = { scopedNewChats++ },
onNewDefaultChat = { defaultNewChats++ },
onSelectSession = {},
onDeleteSession = {},
onRenameSession = { _, _ -> },
)
}
}
compose.onNodeWithText("All Profiles").performClick()
compose.waitForIdle()
compose.onNodeWithText("New Chat").performClick()
compose.runOnIdle {
assertEquals(0, scopedNewChats)
assertEquals(1, defaultNewChats)
}
}
@Test
fun `customize sessions exposes desktop backed view variants`() {
compose.setContent {
MaterialTheme {
SessionDrawerContent(
sessions = listOf(ChatSession("one", "One", null)),
currentSessionId = null,
onNewChat = {},
onSelectSession = {},
onDeleteSession = {},
onRenameSession = { _, _ -> },
)
}
}
compose.onNodeWithText("Customize sessions").performClick()
compose.onNodeWithText("Group by").assertIsDisplayed()
compose.onNodeWithText("Order by").assertIsDisplayed()
compose.onNodeWithText("Show details").assertIsDisplayed()
compose.onNodeWithText("Filters").assertIsDisplayed()
}
@Test
fun `all profiles customization can override a profile identity color`() {
var changed: Pair<String, String?>? = null
compose.setContent {
MaterialTheme {
SessionDrawerContent(
sessions = emptyList(),
currentSessionId = null,
allProfilesSupported = true,
allProfileSessions = listOf(
ProfileSessionRow("alpha", ChatSession("a", "Alpha session", null)),
ProfileSessionRow("beta", ChatSession("b", "Beta session", null)),
),
onProfileColorChange = { profile, color -> changed = profile to color },
onRefreshAllProfiles = {},
onSelectProfileSession = { _, _ -> },
onNewChat = {},
onSelectSession = {},
onDeleteSession = {},
onRenameSession = { _, _ -> },
)
}
}
compose.onNodeWithText("All Profiles").performClick()
compose.onNodeWithText("Customize sessions").performClick()
compose.onNodeWithText("Profile colors").performScrollTo().assertIsDisplayed()
compose.onNodeWithContentDescription(
"Set alpha profile color to ${ProfileAccentSwatches.first()}",
).performScrollTo().performClick()
compose.runOnIdle {
assertEquals("alpha" to ProfileAccentSwatches.first(), changed)
}
}
}
@@ -0,0 +1,44 @@
package com.hermesandroid.relay.ui.components
import com.hermesandroid.relay.data.ApiEndpoint
import com.hermesandroid.relay.data.EndpointCandidate
import org.junit.Assert.assertFalse
import org.junit.Assert.assertTrue
import org.junit.Test
class TailscaleRoutePresenceTest {
@Test
fun primaryTailnetIpCountsAsConfiguredRoute() {
assertTrue(
hasConfiguredTailscaleRoute(
endpoints = emptyList(),
primaryEndpointUrl = "http://100.75.1.2:9119",
),
)
}
@Test
fun explicitTailscaleCandidateStillCounts() {
assertTrue(
hasConfiguredTailscaleRoute(
endpoints = listOf(
EndpointCandidate(
role = "tailscale",
api = ApiEndpoint("server.ts.net", 8642),
),
),
primaryEndpointUrl = "http://192.168.1.2:9119",
),
)
}
@Test
fun ordinaryLanDoesNotCount() {
assertFalse(
hasConfiguredTailscaleRoute(
endpoints = emptyList(),
primaryEndpointUrl = "http://192.168.1.2:9119",
),
)
}
}
@@ -11,7 +11,6 @@ class ChatHeaderSubtitleTest {
resolveChatHeaderSubtitle(
isStreaming = true,
statusText = "Streaming",
projectName = "Hermes Relay",
personalityName = "Victor",
modelName = "GPT-5.6",
),
@@ -19,13 +18,12 @@ class ChatHeaderSubtitleTest {
}
@Test
fun `idle subtitle retains project personality and model metadata`() {
fun `idle subtitle retains only personality and model metadata`() {
assertEquals(
"Hermes Relay \u00B7 Victor \u00B7 GPT-5.6",
"Victor \u00B7 GPT-5.6",
resolveChatHeaderSubtitle(
isStreaming = false,
statusText = "Connected",
projectName = "Hermes Relay",
personalityName = "Victor",
modelName = "GPT-5.6",
),
@@ -0,0 +1,48 @@
package com.hermesandroid.relay.ui.screens
import com.hermesandroid.relay.data.ProactiveInboxEntry
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertTrue
import org.junit.Test
class ProvisionalThreadRowsTest {
@Test
fun rowsAreConnectionScopedAndDisappearWhenRealThreadExists() {
val entries = listOf(
entry("one", connectionId = "connection-a", chatId = "phone"),
entry("two", connectionId = "connection-b", chatId = "phone"),
entry("three", connectionId = "connection-a", chatId = "project"),
)
val rows = buildProvisionalThreadRows(
entries = entries,
activeConnectionId = "connection-a",
realThreadChatIds = listOf("project"),
)
assertEquals(listOf("one"), rows.getValue("phone").map { it.id })
assertFalse("project" in rows)
}
@Test
fun legacyUnscopedEntriesRemainVisibleOnTheActiveConnection() {
val rows = buildProvisionalThreadRows(
entries = listOf(entry("legacy", connectionId = null, chatId = null)),
activeConnectionId = "connection-a",
realThreadChatIds = emptyList(),
)
assertTrue("phone" in rows)
}
private fun entry(id: String, connectionId: String?, chatId: String?) =
ProactiveInboxEntry(
id = id,
title = "Hermes",
text = id,
receivedAt = 1L,
chatId = chatId,
connectionId = connectionId,
)
}
@@ -0,0 +1,25 @@
package com.hermesandroid.relay.ui.theme
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotEquals
import org.junit.Assert.assertNull
import org.junit.Test
class ProfileAccentTest {
@Test
fun namedProfilesReceiveStableDistinctColors() {
assertEquals(deterministicProfileAccent("alpha"), deterministicProfileAccent("alpha"))
assertNotEquals(deterministicProfileAccent("alpha"), deterministicProfileAccent("beta"))
}
@Test
fun overrideWinsAndDefaultRemainsNeutral() {
assertEquals(accentColor("#356CFF"), resolveProfileAccent("alpha", mapOf("alpha" to "#356CFF")))
assertNull(resolveProfileAccent("default", mapOf("default" to "#356CFF")))
}
@Test
fun pickerOffersDesktopSizedPalette() {
assertEquals(12, ProfileAccentSwatches.distinct().size)
}
}
@@ -0,0 +1,23 @@
package com.hermesandroid.relay.util
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.withTimeout
import kotlinx.coroutines.withTimeoutOrNull
class NavRouteRequestTest {
@Test
fun requestBeforeCollector_isRetainedAndConsumedOnce() = runBlocking {
assertTrue(NavRouteRequest.tryRequest("chat?proactiveChatId=phone"))
assertEquals(
"chat?proactiveChatId=phone",
withTimeout(1_000) { NavRouteRequest.requests.first() },
)
assertNull(withTimeoutOrNull(50) { NavRouteRequest.requests.first() })
}
}
@@ -12,6 +12,7 @@ import com.hermesandroid.relay.data.ChatTurnToolCheckpoint
import com.hermesandroid.relay.data.ChatTurnUserCheckpoint
import com.hermesandroid.relay.data.HermesCardDispatch
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.data.Profile
import com.hermesandroid.relay.network.upstream.ChatHandler
import com.hermesandroid.relay.network.upstream.DashboardApiClient
import com.hermesandroid.relay.network.upstream.GatewayChatClient
@@ -148,6 +149,75 @@ class ChatViewModelGatewayInboundTurnTest {
assertEquals("victor", gatewayClient.sessionProfileProvider())
}
@Test
fun allProfilesOpenKeepsGlobalSelectionButScopesHistoryResumeAndSendToOwner() {
val global = Profile(name = "mizu", model = "grok-4.5", description = "Mizu")
val owner = Profile(name = "x-bot", model = "grok-4.3", description = "X Bot")
var loadedProfile: String? = null
var persistedSession = "unchanged"
viewModel.setSelectedProfileProvider { global }
viewModel.setSessionProfileNameProvider { global.name }
viewModel.onSessionChanged = { persistedSession = it ?: "cleared" }
viewModel.setProfileMessageLoaderWithMode { profileName, _, _ ->
loadedProfile = profileName
Result.success(emptyList())
}
viewModel.openProfileSession(
profileName = owner.name,
profile = owner,
contextKey = AgentDisplay.profileContextKey("connection-a", owner.name),
sessionId = "x-bot-session",
)
awaitCondition { loadedProfile == owner.name }
assertEquals(owner.name, viewModel.openedSessionProfileName.value)
assertEquals(owner.name, gatewayClient.sessionProfileProvider())
assertEquals("X-bot", handler.activeAgentName)
assertEquals("unchanged", persistedSession)
viewModel.switchProfileContext(
AgentDisplay.profileContextKey("connection-a", global.name),
sessionId = null,
)
assertEquals(null, viewModel.openedSessionProfileName.value)
assertEquals(global.name, gatewayClient.sessionProfileProvider())
viewModel.openProfileSession(
profileName = owner.name,
profile = owner,
contextKey = AgentDisplay.profileContextKey("connection-a", owner.name),
sessionId = "x-bot-session",
)
assertEquals(owner.name, viewModel.openedSessionProfileName.value)
viewModel.createNewChat()
assertEquals(null, viewModel.openedSessionProfileName.value)
assertEquals(global.name, gatewayClient.sessionProfileProvider())
}
@Test
fun allProfilesNewChatUsesLiteralDefaultWithoutChangingGlobalSelection() {
val global = Profile(name = "victor", model = "grok-4.5", description = "Victor")
val rootDefault = Profile(name = "default", model = "gpt-5.5", description = "Hermes")
var persistedSession = "unchanged"
viewModel.setSelectedProfileProvider { global }
viewModel.setSessionProfileNameProvider { global.name }
viewModel.onSessionChanged = { persistedSession = it ?: "cleared" }
viewModel.createProfileChat(
profileName = "default",
profile = rootDefault,
contextKey = AgentDisplay.profileContextKey("connection-a", "default"),
)
assertEquals("default", viewModel.openedSessionProfileName.value)
assertEquals("default", gatewayClient.sessionProfileProvider())
assertEquals(null, handler.currentSessionId.value)
assertEquals("Hermes", handler.activeAgentName)
assertEquals("unchanged", persistedSession)
}
@Test
fun gatewaySessionCreateProviderPreservesExplicitFastFalse() {
serverWs.send(
@@ -1383,38 +1453,12 @@ class ChatViewModelGatewayInboundTurnTest {
}
@Test
fun unsolicitedTurnDoesNotReplaceAnActiveForcedSseTurn() {
holdCompletionsStream = true
fun gatewayVoiceTurnDoesNotRequireApiFallback() {
viewModel.sendVoiceMessage("local voice turn", "Respond for spoken playback")
awaitCondition { handler.isStreaming.value }
val localPlaceholderId = handler.messages.value.last().id
val params = gatewayHarness.awaitRpc("prompt.submit")
serverWs.send(gatewayHarness.eventFrame("message.start", null, "live-resumed"))
serverWs.send(
gatewayHarness.eventFrame(
"message.delta",
buildJsonObject { put("text", BACKGROUND_ANSWER) },
"live-resumed",
),
)
persistedHistory = persistedAnswerHistory()
serverWs.send(
gatewayHarness.eventFrame(
"message.complete",
buildJsonObject { put("text", BACKGROUND_ANSWER) },
"live-resumed",
),
)
Thread.sleep(150)
shadowOf(Looper.getMainLooper()).idle()
assertTrue("the forced SSE turn must still own streaming", handler.isStreaming.value)
assertTrue(handler.messages.value.any { it.id == localPlaceholderId && it.isStreaming })
assertFalse(handler.messages.value.any { it.content == BACKGROUND_ANSWER })
viewModel.cancelStream()
awaitCondition { !handler.isStreaming.value }
awaitCondition { handler.messages.value.any { it.content == BACKGROUND_ANSWER } }
assertEquals(JsonPrimitive("local voice turn"), params["text"])
assertEquals(0, apiCompletionsRequestCount.get())
}
@Test
@@ -241,6 +241,35 @@ class ProfileControllerLockTest {
}
}
@Test
fun explicitProfileSessionMutations_useOwningProfileWithoutChangingSelection() {
dashboardUrl = "https://dashboard.example"
coEvery { dashboardClient.deleteSession("session-2", profile = "coder") } returns
Result.success(buildJsonObject { })
coEvery { dashboardClient.renameSession("session-2", "Updated", profile = "coder") } returns
Result.success(buildJsonObject { })
coEvery { dashboardClient.setSessionPinned("session-2", true, profile = "coder") } returns
Result.success(buildJsonObject { })
coEvery { dashboardClient.setSessionArchived("session-2", true, profile = "coder") } returns
Result.success(buildJsonObject { })
assertTrue(runBlocking { controller.deleteSession("coder", "session-2") })
assertTrue(runBlocking { controller.renameSession("coder", "session-2", "Updated") })
assertTrue(runBlocking { controller.setSessionPinned("coder", "session-2", true) })
assertTrue(runBlocking { controller.setSessionArchived("coder", "session-2", true) })
coVerify(exactly = 1) { dashboardClient.deleteSession("session-2", profile = "coder") }
coVerify(exactly = 1) {
dashboardClient.renameSession("session-2", "Updated", profile = "coder")
}
coVerify(exactly = 1) {
dashboardClient.setSessionPinned("session-2", true, profile = "coder")
}
coVerify(exactly = 1) {
dashboardClient.setSessionArchived("session-2", true, profile = "coder")
}
}
// --- selectProfile gating while locked ----------------------------------
@Test
@@ -6,6 +6,7 @@ import org.junit.Assert.assertNotSame
import org.junit.Assert.assertSame
import org.junit.Assert.assertTrue
import org.junit.Test
import java.util.concurrent.TimeUnit
class UpstreamTransportControllerAuthClientTest {
@Test
@@ -25,6 +26,10 @@ class UpstreamTransportControllerAuthClientTest {
dashboardUrl = "https://hermes.example.test/alternate"
val moved = controller.dashboardHttpClientForActive(dashboardUrl)
assertNotSame(first, moved)
assertTrue(first.dispatcher.executorService.isShutdown)
val deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(5)
while (!first.dispatcher.executorService.isShutdown && System.nanoTime() < deadline) {
Thread.yield()
}
assertTrue("replaced dashboard client was not disposed", first.dispatcher.executorService.isShutdown)
}
}
+1
View File
@@ -0,0 +1 @@
1.3.14
+101 -5
View File
@@ -50,7 +50,9 @@ for existing installs until explicitly enabled. Settings also exposes **Open
terminal**, **Open Hermes CLI**, **View daemon log**, and **Run diagnostics**,
and manages Desktop release updates. **Help &
About** reports the UI, CLI, and connected Relay versions and links to the docs,
troubleshooting, release notes, logs, and diagnostics. The installer download is
troubleshooting, release notes, logs, and diagnostics. Tray lifecycle and child-
process failures are written to `~/.hermes/tray.log`; daemon connection and tool-
router events remain in `~/.hermes/daemon.log`. The installer download is
verified against the release
`SHA256SUMS.txt`, preserves the startup preference, restores a previously
running daemon, and relaunches the tray after the silent replacement.
@@ -80,11 +82,15 @@ aborts, and non-zero process exits; and keeps request context collapsed until
explicitly expanded. Events record handler duration and request ID where
available. The compact Overview still shows only the three newest events.
Settings also keeps activity compact: it previews the three newest events and
opens a dedicated Activity page. Selecting an event opens bounded request,
stdout, stderr, result, exit, timing, and truncation evidence; sensitive request
inputs are excluded. Handler failures and aborts are **Issues**; non-zero process exits are
opens a dedicated Activity page. Selecting an event opens a truthful lifecycle
stepper plus bounded request, stdout, stderr, result, exit, timing, and
truncation evidence; sensitive request inputs are excluded. Screenshot events
can retain an opaque local PNG outside the JSON log and open it in a larger
borderless viewer. **Settings → Activity → Screenshot evidence** controls this
as Off, 1 day, 7 days (default), or 30 days, shows local file count/usage, and
caps storage at 20 files and 10 MB per image. Handler failures and aborts are **Issues**; non-zero process exits are
shown separately because probing commands may legitimately use them. **Clear**
removes both current and rotated local audit history after confirmation.
removes current/rotated audit history and retained screenshot evidence after confirmation.
Clicking a card under **Hosts** opens that host's detail page; it does not change
the active connection. The detail page is the per-host hub for its local display
@@ -441,6 +447,81 @@ assist/control grant is active because approved input inherits that privilege.
Default computer-use policy blocks password managers, credential prompts, banking/payment/crypto surfaces, OS security/admin settings, and private-key/token material. `~/.hermes/desktop-control.json` lets operators tighten or extend that baseline.
#### Preferred CUA Driver engine
On Windows, Hermes-Relay prefers a compatible local
[CUA Driver](https://github.com/trycua/cua) runtime for structured
computer-control engine. It stays behind the same `desktop_computer_*` tools:
the agent does not receive CUA's raw tool surface, configuration, updater,
recording, replay, JavaScript, application-launch, or process-termination
operations.
Windows input is the explicit compatibility backend. A backend is selected once
when each authenticated control session starts and cannot change mid-session;
changing the setting affects only new sessions. If preferred CUA is unavailable
before a session starts, that session can use compatibility mode.
Inspect the detected runtime and selected/effective engine with:
```powershell
hermes-relay computer-use status --json
hermes-relay computer-use cua status
hermes-relay computer-use cua health # explicit accessibility recheck
hermes-relay computer-use cua check-update
hermes-relay computer-use cua install --yes
hermes-relay computer-use cua update --yes
hermes-relay computer-use engine cua # preferred; requires a ready runtime
hermes-relay computer-use engine legacy # explicit compatibility backend
hermes-relay computer-use cursor on
```
The management UI exposes the same controls under **Settings → Computer
control**. CUA is selected only when its canonical Windows package resolves from
`%USERPROFILE%\.cua-driver\packages\current\cua-driver.exe`, its supported
version and manifest agree, its required tools are present, and its permission mode
is not unrestricted. The live health report is an explicit diagnostic while the
temporary Windows workaround for trycua/cua#3103 is active. Hermes ignores an unrelated
or stale `cua-driver.exe` found earlier on `PATH`.
Background dispatch is mandatory. If an application cannot accept a
background action, the action fails instead of silently stealing focus.
**Allow foreground escalation** is reserved for a later explicitly approved
path; this release always reports it off and dispatches CUA actions in the
background. **Animated agent cursor** shows a virtual, session-scoped pointer
for agent activity; it does not move the operator's physical Windows cursor and
is not another hardware pointer. Hermes binds driver sessions and snapshot
tokens to its own control authority, target PID/window, grant, and fresh
snapshot; an element token cannot be reused across windows or after it is
consumed or expires.
CUA Driver is not bundled with the Hermes-Relay installer. The explicit
`computer-use cua install|update --yes` commands use the canonical upstream
GitHub release manifest and installer. Hermes verifies the manifest's
repository/product/version and installer SHA-256 before execution under a
sanitized child-process environment, then checks
the canonical binary's path, version, own manifest, tool surface, and permission
mode. Accessibility health can be rechecked separately. This is release-metadata/checksum validation—not a Windows
publisher signature. A native update newer than the supported `>=0.19.3,
<0.20.0` range is displayed but refused. There is no silent install/update,
and every child invocation forces CUA telemetry off. `hermes-relay update`
continues to manage only the CLI and management UI.
Window-scoped snapshots and semantic actions use the selected control backend.
The existing full-display screenshot remains a separate read-only
`system_capture` path, so observation does not cause a mid-session backend
switch. The local audit and UI Activity timeline record bounded high-level
evidence such as backend, background dispatch, control session, target
application/window identifiers, action, phase, and verification state.
Accessibility text, screenshot bytes, entered values, and raw CUA responses are
excluded from that drilldown.
CUA improves structured screen/input isolation, but it is not a sandbox for
general commands. If Commands, PowerShell, or terminal execution is allowed,
that trusted path can still use ordinary operating-system automation. Disable
raw command access when CUA's scoped UI-control boundary is part of the security
model. Full Access can remove ordinary task prompts, but it does not bypass
authenticated targeting, sensitive-surface checks, snapshot freshness, UAC or
Windows-session boundaries, audit, driver health, or emergency stop.
### Devices — server-side session management
```sh
@@ -633,6 +714,21 @@ Precedence for credentials: `--token` → `HERMES_RELAY_TOKEN` → `--code` →
## Troubleshooting
- **Many `Bun` / `hermes-relay.exe` processes, or Windows error `0xc0000142` from `reg.exe`, `adb.exe`, or `hermes-relay.exe`** — quit **Hermes-Relay CLI UI** first, then run `hermes-relay daemon stop` from a fresh PowerShell. If the CLI cannot start, inspect exact executable paths before stopping only Hermes-Relay-owned processes:
```powershell
$relayBin = [IO.Path]::GetFullPath("$env:USERPROFILE\.hermes\bin\")
$relayProcesses = Get-CimInstance Win32_Process | Where-Object {
$_.ExecutablePath -and
[IO.Path]::GetFullPath($_.ExecutablePath).StartsWith($relayBin, [StringComparison]::OrdinalIgnoreCase) -and
$_.Name -in @('hermes-relay.exe', 'hermes-relay-tray.exe')
}
$relayProcesses | Select-Object ProcessId, ParentProcessId, Name, ExecutablePath, CommandLine
# Review the rows above before stopping them:
$relayProcesses | ForEach-Object { Stop-Process -Id $_.ProcessId }
```
Do not broadly stop every process named `Bun`: unrelated development tools may use the same runtime name. After recovery, inspect `~/.hermes/tray.log` for snapshot, subprocess timeout, launch, and exit failures. Use `~/.hermes/daemon.log` for the single long-running daemon's authentication, transport, and tool-router lifecycle. If unrelated Windows programs still fail to initialize, restart Windows before relaunching the tray.
- **`auth timed out after 15000ms`** — the relay subprocess takes 15–30 s on first attach because it initializes the full agent. Bump the timeout: `HERMES_RELAY_AUTH_TIMEOUT_MS=30000 hermes-relay …`.
- **`relay rejected credentials: auth failed`** — your stored token expired or was revoked. Re-pair: `hermes-relay pair --remote ws://…`.
- **`RelayTransport: global WebSocket not available`** — your Node is too old. Need >=21.
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "@hermes-relay/cli",
"version": "0.4.0-alpha.8",
"version": "0.4.0-beta.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@hermes-relay/cli",
"version": "0.4.0-alpha.8",
"version": "0.4.0-beta.3",
"license": "MIT",
"bin": {
"hermes-relay": "bin/hermes-relay.js"
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@hermes-relay/cli",
"version": "0.4.0-alpha.8",
"version": "0.4.0-beta.3",
"description": "Thin-client CLI for Hermes-Relay — talk to a remote Hermes agent over WSS with pairing auth, stream-renders tool calls and responses to plain stdout.",
"type": "module",
"bin": {
+118
View File
@@ -0,0 +1,118 @@
[CmdletBinding()]
param(
[switch]$Run,
[string]$CuaDriver = "$env:USERPROFILE\.cua-driver\packages\current\cua-driver.exe"
)
$ErrorActionPreference = 'Stop'
if (-not $Run) {
Write-Host 'Hermes CUA live acceptance is opt-in because it opens and controls two Calculator windows.'
Write-Host 'Re-run with: powershell -ExecutionPolicy Bypass -File scripts/test-cua-windows.ps1 -Run'
exit 0
}
if (-not (Test-Path -LiteralPath $CuaDriver -PathType Leaf)) {
throw "Canonical CUA Driver was not found at $CuaDriver"
}
Add-Type -TypeDefinition @'
using System;
using System.Runtime.InteropServices;
public static class HermesCuaAcceptanceNative {
[StructLayout(LayoutKind.Sequential)] public struct POINT { public int X; public int Y; }
[DllImport("user32.dll")] public static extern bool GetCursorPos(out POINT point);
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
}
'@
function Invoke-Cua([string]$Tool, [hashtable]$Payload) {
$json = $Payload | ConvertTo-Json -Depth 12 -Compress
$raw = $json | & $CuaDriver call $Tool
if ($LASTEXITCODE -ne 0) { throw "CUA $Tool failed with exit code $LASTEXITCODE" }
$result = $raw | ConvertFrom-Json
if ($result.isError -eq $true) { throw "CUA $Tool rejected the request: $raw" }
return $result
}
function Get-DesktopSentinel {
$point = New-Object HermesCuaAcceptanceNative+POINT
[void][HermesCuaAcceptanceNative]::GetCursorPos([ref]$point)
[pscustomobject]@{
CursorX = $point.X
CursorY = $point.Y
Foreground = [HermesCuaAcceptanceNative]::GetForegroundWindow().ToInt64()
}
}
function Assert-Unchanged([object]$Before, [string]$Step) {
$after = Get-DesktopSentinel
if ($after.CursorX -ne $Before.CursorX -or $after.CursorY -ne $Before.CursorY) {
throw "$Step moved the physical cursor from ($($Before.CursorX),$($Before.CursorY)) to ($($after.CursorX),$($after.CursorY))"
}
if ($after.Foreground -ne $Before.Foreground) {
throw "$Step changed the foreground HWND from $($Before.Foreground) to $($after.Foreground)"
}
}
function Get-Window([object]$Launch) {
$window = @($Launch.windows | Where-Object { $_.is_on_screen -ne $false }) | Select-Object -First 1
if (-not $window) { throw "CUA launch did not return a usable window: $($Launch | ConvertTo-Json -Depth 8 -Compress)" }
return $window
}
$sessionA = "hermes-live-a-$([guid]::NewGuid().ToString('N'))"
$sessionB = "hermes-live-b-$([guid]::NewGuid().ToString('N'))"
$opened = @()
try {
$health = Invoke-Cua health_report @{}
if ($health.overall -ne 'healthy') { throw "CUA health must be healthy, got $($health.overall)" }
[void](Invoke-Cua start_session @{ session = $sessionA; capture_scope = 'window' })
[void](Invoke-Cua start_session @{ session = $sessionB; capture_scope = 'window' })
$sentinel = Get-DesktopSentinel
$launchA = Invoke-Cua launch_app @{ aumid = 'Microsoft.WindowsCalculator_8wekyb3d8bbwe!App'; creates_new_application_instance = $true; session = $sessionA }
$launchB = Invoke-Cua launch_app @{ aumid = 'Microsoft.WindowsCalculator_8wekyb3d8bbwe!App'; creates_new_application_instance = $true; session = $sessionB }
$opened = @($launchA.pid, $launchB.pid)
Start-Sleep -Milliseconds 800
Assert-Unchanged $sentinel 'background launch'
$windowA = Get-Window $launchA
$windowB = Get-Window $launchB
if ($windowA.window_id -eq $windowB.window_id) { throw 'isolated sessions resolved to the same Calculator window' }
$beforeA = Invoke-Cua get_window_state @{ pid = $launchA.pid; window_id = $windowA.window_id; session = $sessionA; query = 'Seven'; include_screenshot = $false }
$seven = @($beforeA.elements | Where-Object { $_.label -eq 'Seven' -or $_.name -eq 'Seven' }) | Select-Object -First 1
if (-not $seven.element_token) { throw 'Calculator Seven element did not expose an opaque token' }
[void](Invoke-Cua click @{ pid = $launchA.pid; window_id = $windowA.window_id; element_token = $seven.element_token; session = $sessionA; scope = 'window'; delivery_mode = 'background' })
Assert-Unchanged $sentinel 'background element action'
$afterA = Invoke-Cua get_window_state @{ pid = $launchA.pid; window_id = $windowA.window_id; session = $sessionA; query = 'Display'; include_screenshot = $false }
if ($afterA.snapshot_id -eq $beforeA.snapshot_id) { throw 'post-action snapshot did not advance its generation' }
$staleRaw = (@{ pid = $launchA.pid; window_id = $windowA.window_id; element_token = $seven.element_token; session = $sessionA; scope = 'window'; delivery_mode = 'background' } | ConvertTo-Json -Compress) | & $CuaDriver call click
$stale = $staleRaw | ConvertFrom-Json
if ($stale.isError -ne $true) { throw 'stale element token was accepted after a newer snapshot' }
$cursorA = Invoke-Cua get_agent_cursor_state @{ session = $sessionA }
$cursorB = Invoke-Cua get_agent_cursor_state @{ session = $sessionB }
if (($cursorA | ConvertTo-Json -Depth 8 -Compress) -eq ($cursorB | ConvertTo-Json -Depth 8 -Compress)) {
throw 'two control sessions did not expose isolated cursor state'
}
[void](Invoke-Cua end_session @{ session = $sessionA })
$endedRaw = (@{ session = $sessionA } | ConvertTo-Json -Compress) | & $CuaDriver call get_session_state
$ended = $endedRaw | ConvertFrom-Json
if ($ended.code -ne 'session_not_started') { throw 'ended session remained active' }
Write-Host 'PASS: physical cursor and foreground stayed unchanged.'
Write-Host 'PASS: background Calculator action was bracketed by snapshots.'
Write-Host 'PASS: stale token rejection and two isolated cursor sessions were verified.'
Write-Host 'PASS: ending a session immediately revoked its CUA state.'
} finally {
foreach ($session in @($sessionA, $sessionB)) {
try { [void](Invoke-Cua end_session @{ session = $session }) } catch { Write-Warning $_ }
}
Write-Host "Calculator processes created by this acceptance run: $($opened -join ', '). Close them manually after inspection."
}
+34 -3
View File
@@ -5,7 +5,8 @@
// the audit flagged as the biggest desktop-tools transparency gap.
import type { ParsedArgs } from '../cli.js'
import { auditLogPath, readRecentAudit } from '../lib/auditLog.js'
import { auditLogPath, auditScreenshotEvidenceStatus, clearAuditScreenshotEvidence, pruneAuditScreenshotEvidence, readRecentAudit } from '../lib/auditLog.js'
import { readDesktopUseSettings, setActivityScreenshotRetention } from '../lib/desktopUseSettings.js'
import { renderTable } from '../lib/table.js'
import { SYMBOLS, theme as makeTheme } from '../lib/theme.js'
import { printUsage, type UsageSpec } from '../lib/usage.js'
@@ -13,10 +14,12 @@ import { printUsage, type UsageSpec } from '../lib/usage.js'
const AUDIT_USAGE: UsageSpec = {
name: 'audit',
summary: 'show recent desktop-tool activity the agent ran on this machine',
usage: ['audit [--limit <n>] [--json]'],
usage: ['audit [--limit <n>] [--json]', 'audit screenshots [on|off] [--days <1|7|30>] [--yes] [--json]'],
flags: [
{ flag: '--limit <n>', desc: 'How many recent entries to show (default 50)' },
{ flag: '--json', desc: 'Emit raw audit entries as JSON' }
{ flag: '--json', desc: 'Emit raw audit entries as JSON' },
{ flag: '--days <1|7|30>', desc: 'Local screenshot retention period' },
{ flag: '--yes', desc: 'Confirm a retention change' }
],
examples: ['hermes-relay audit', 'hermes-relay audit --limit 20']
}
@@ -36,6 +39,34 @@ export async function auditCommand(args: ParsedArgs): Promise<number> {
return 0
}
if (args.positional[0] === 'screenshots') {
const settings = await readDesktopUseSettings()
const mode = args.positional[1]
if (mode === 'on' || mode === 'off') {
if (args.flags.yes !== true) {
process.stderr.write('audit screenshots: retention changes require --yes\n')
return 1
}
const rawDays = typeof args.flags.days === 'string' ? Number(args.flags.days) : settings.activity_screenshot_retention_days
if (rawDays !== 1 && rawDays !== 7 && rawDays !== 30) {
process.stderr.write('audit screenshots: --days must be 1, 7, or 30\n')
return 1
}
await setActivityScreenshotRetention(mode === 'on', rawDays)
if (mode === 'off') await clearAuditScreenshotEvidence()
else await pruneAuditScreenshotEvidence(rawDays)
} else if (mode) {
process.stderr.write('audit screenshots: expected on or off\n')
return 1
}
const current = await readDesktopUseSettings()
const evidence = await auditScreenshotEvidenceStatus()
const result = { enabled: current.activity_screenshot_retention_enabled, days: current.activity_screenshot_retention_days, ...evidence }
if (args.flags.json) process.stdout.write(JSON.stringify(result, null, 2) + '\n')
else process.stdout.write(`Screenshot evidence: ${result.enabled ? `${result.days} days` : 'off'} · ${result.count} file${result.count === 1 ? '' : 's'}\n`)
return 0
}
const rawLimit = typeof args.flags.limit === 'string' ? parseInt(args.flags.limit, 10) : 50
const limit = Number.isFinite(rawLimit) && rawLimit > 0 ? rawLimit : 50
+163 -4
View File
@@ -5,11 +5,19 @@ import { readDaemonStatus, isDaemonProcessAlive } from '../lib/daemonStatus.js'
import {
readDesktopUseSettings,
requestComputerGrantCancellation,
setComputerControlSettings,
setDesktopUseEnabled
} from '../lib/desktopUseSettings.js'
import { listPendingGrantRequests } from '../lib/grantBridge.js'
import { theme as makeTheme } from '../lib/theme.js'
import { printUsage, type UsageSpec, unknownSubcommand } from '../lib/usage.js'
import { CuaDriverAdapter, type CuaRuntimeStatus } from '../tools/cuaDriver.js'
import {
checkCuaUpdate,
getCuaManagementStatus,
installCuaDriver,
updateCuaDriver
} from '../tools/cuaManagement.js'
const COMPUTER_USE_USAGE: UsageSpec = {
name: 'computer-use',
@@ -18,21 +26,31 @@ const COMPUTER_USE_USAGE: UsageSpec = {
'computer-use status [--json]',
'computer-use enable [--yes]',
'computer-use disable',
'computer-use cancel'
'computer-use cancel',
'computer-use engine <legacy|cua>',
'computer-use cursor <on|off>',
'computer-use cua <status|health|install|check-update|update> [--json] [--yes]'
],
subcommands: [
{ verb: 'status', desc: 'Show preference, daemon state, active grant, and pending requests' },
{ verb: 'enable', desc: 'Persist desktop-use enablement after explicit confirmation' },
{ verb: 'disable', desc: 'Disable desktop use and request cancellation of any active grant' },
{ verb: 'cancel', desc: 'Cancel the active task-scoped desktop grant' }
{ verb: 'cancel', desc: 'Cancel the active task-scoped desktop grant' },
{ verb: 'engine', desc: 'Choose legacy Windows input or a ready CUA Driver backend' },
{ verb: 'cursor', desc: 'Show or hide the CUA virtual agent cursor' },
{ verb: 'cua', desc: 'Manage the canonical CUA Driver package and recheck accessibility health' }
],
flags: [
{ flag: '--json', desc: 'Emit machine-readable status' },
{ flag: '--yes', desc: 'Confirm enablement non-interactively' }
{ flag: '--yes', desc: 'Confirm enablement, CUA installation, or CUA update explicitly' }
],
examples: [
'hermes-relay computer-use status',
'hermes-relay computer-use enable',
'hermes-relay computer-use cua status',
'hermes-relay computer-use cua health',
'hermes-relay computer-use cua check-update',
'hermes-relay computer-use cua install --yes',
'hermes-relay computer-use cancel',
'hermes-relay computer-use disable'
]
@@ -62,6 +80,24 @@ async function statusPayload(): Promise<Record<string, unknown>> {
const activeGrant = daemonAlive && daemon?.computer_grant?.active === true
? daemon.computer_grant
: null
let cua: CuaRuntimeStatus | null = null
try {
cua = await CuaDriverAdapter.status()
} catch {
// The optional backend must not make ordinary desktop-use status fail.
}
const cuaReason = cua?.reason?.toLowerCase() ?? ''
const cuaState = !cua?.available
? 'not_installed'
: cua.ready
? 'ready'
: /(?:incompatible|unsupported|version|manifest|permission mode|missing required tools)/.test(cuaReason)
? 'incompatible'
: /(?:degraded|health)/.test(cuaReason)
? 'degraded'
: 'error'
const cuaReady = cuaState === 'ready'
const lifecycle = daemonAlive ? daemon?.computer_control : undefined
return {
enabled: settings.computer_use_enabled,
daemon_alive: daemonAlive,
@@ -69,7 +105,32 @@ async function statusPayload(): Promise<Record<string, unknown>> {
daemon_computer_use_enabled: daemonAlive ? (daemon?.computer_use_enabled ?? false) : false,
active_grant: activeGrant,
pending_grants: pending.length,
restart_required: daemonAlive && daemon?.computer_use_enabled !== settings.computer_use_enabled
restart_required: daemonAlive && daemon?.computer_use_enabled !== settings.computer_use_enabled,
computer_control_engine: {
selected: settings.computer_control_engine,
effective: settings.computer_control_engine === 'cua'
? lifecycle?.active_backend === 'cua'
? 'cua'
: lifecycle?.active_backend === 'legacy_compat'
? 'legacy'
: cuaReady ? 'cua' : 'legacy'
: 'legacy',
available: cua?.available === true,
state: cuaState,
version: cua?.binaryVersion ?? null,
health: cua?.health ?? null,
path: cua?.binaryPath ?? null,
cursor_enabled: settings.cua_cursor_enabled,
active_sessions: lifecycle?.active_sessions ?? 0,
active_backend: lifecycle?.active_backend ?? 'idle',
last_action: lifecycle?.last_action ?? null,
foreground_escalation_enabled: false,
message: settings.computer_control_engine === 'cua' && !cuaReady
? cuaState === 'degraded'
? `CUA Driver is installed, but UI Automation is degraded; new sessions use Windows Input compatibility mode. ${cua?.reason ?? ''}`.trim()
: `CUA Driver is unavailable before control starts; new sessions use Windows Input compatibility mode. ${cua?.reason ?? ''}`.trim()
: cua?.reason ?? null
}
}
}
@@ -81,6 +142,68 @@ export async function computerUseCommand(args: ParsedArgs): Promise<number> {
}
const subcommand = args.positional[0] ?? 'status'
if (subcommand === 'cua') {
const action = args.positional[1] ?? 'status'
const json = args.flags.json === true
try {
if (action === 'health') {
const health = await CuaDriverAdapter.healthStatus()
if (json) process.stdout.write(JSON.stringify(health, null, 2) + '\n')
else {
process.stdout.write(t.bold('CUA Driver accessibility health') + `\n state: ${health.state}\n`)
if (health.reason) process.stdout.write(t.warnLine(` ${health.reason}`) + '\n')
process.stdout.write(t.muted(' This diagnostic does not disable the runtime while the temporary Windows compatibility policy is active.') + '\n')
}
// The probe result is data, not command failure. Callers (including the
// tray) inspect state while still receiving the JSON for degradation.
return 0
}
const payload = action === 'status'
? await getCuaManagementStatus()
: action === 'check-update'
? await checkCuaUpdate()
: action === 'install'
? args.flags.yes === true
? await installCuaDriver()
: null
: action === 'update'
? args.flags.yes === true
? await updateCuaDriver()
: null
: undefined
if (payload === undefined) {
process.stderr.write(t.err('cua action must be status, health, install, check-update, or update') + '\n')
return 1
}
if (payload === null) {
process.stderr.write(t.err(`CUA ${action} requires explicit confirmation with --yes`) + '\n')
return 1
}
if (json) {
process.stdout.write(JSON.stringify(payload, null, 2) + '\n')
return 0
}
const version = payload.current_version ?? 'not installed'
process.stdout.write(t.bold('CUA Driver') + `\n version: ${version}\n`)
process.stdout.write(` package: ${payload.canonical_path ?? 'not installed'}\n`)
process.stdout.write(` compatibility: ${payload.compatible ? 'supported' : payload.compatibility_reason ?? 'not ready'}\n`)
if (payload.stale_path_shim) {
process.stdout.write(t.warnLine(` PATH resolves a competing copy: ${payload.discovered_path}`) + '\n')
process.stdout.write(t.muted(' Hermes uses the canonical package/current install instead.') + '\n')
}
if (payload.update) {
if (payload.update.error) process.stdout.write(t.warnLine(` update check: ${payload.update.error}`) + '\n')
else if (payload.update.update_available) {
process.stdout.write(` update: ${payload.update.latest_version}${payload.update.compatible ? ' available' : ' available but unsupported'}\n`)
} else process.stdout.write(' update: up to date\n')
}
if (payload.operation) process.stdout.write(t.okLine(`CUA ${payload.operation.kind} completed`) + '\n')
return 0
} catch (error) {
process.stderr.write(t.err(error instanceof Error ? error.message : String(error)) + '\n')
return 1
}
}
if (subcommand === 'status') {
const payload = await statusPayload()
if (args.flags.json) {
@@ -129,6 +252,42 @@ export async function computerUseCommand(args: ParsedArgs): Promise<number> {
return 0
}
if (subcommand === 'engine') {
const engine = args.positional[1]
if (engine !== 'legacy' && engine !== 'cua') {
process.stderr.write(t.err('engine must be legacy or cua') + '\n')
return 1
}
if (engine === 'cua') {
const payload = await statusPayload()
const status = payload.computer_control_engine as { state?: string }
if (status.state !== 'ready') {
process.stderr.write(t.err('CUA Driver is not ready; engine selection was not changed') + '\n')
return 1
}
}
await setComputerControlSettings({ computer_control_engine: engine })
process.stdout.write(t.okLine(`computer control engine set to ${engine}`) + '\n')
return 0
}
if (subcommand === 'cursor') {
const value = args.positional[1]
if (value !== 'on' && value !== 'off') {
process.stderr.write(t.err(`${subcommand} must be on or off`) + '\n')
return 1
}
const payload = await statusPayload()
const status = payload.computer_control_engine as { selected?: string; state?: string }
if (status.selected !== 'cua' || status.state !== 'ready') {
process.stderr.write(t.err(`CUA Driver must be selected and ready before changing ${subcommand}`) + '\n')
return 1
}
await setComputerControlSettings({ cua_cursor_enabled: value === 'on' })
process.stdout.write(t.okLine(`CUA ${subcommand} ${value}`) + '\n')
return 0
}
return unknownSubcommand(COMPUTER_USE_USAGE, subcommand, t)
}
+240 -13
View File
@@ -31,6 +31,7 @@
import { spawn } from 'node:child_process'
import { closeSync, openSync, promises as fs } from 'node:fs'
import { randomUUID } from 'node:crypto'
import * as os from 'node:os'
import * as path from 'node:path'
@@ -50,6 +51,7 @@ import {
type DaemonStatus
} from '../lib/daemonStatus.js'
import { rpcErrorMessage, asRpcResult } from '../lib/rpc.js'
import { appendAudit } from '../lib/auditLog.js'
import { effectiveHostAccessMode, effectiveHostCapabilityPolicies, getHostAccessMode, getHostCapabilityPolicies } from '../lib/hostAccessPolicy.js'
import { theme as makeTheme } from '../lib/theme.js'
import { printUsage, type UsageSpec } from '../lib/usage.js'
@@ -62,12 +64,13 @@ import {
shouldAdvertiseComputerUse
} from '../tools/handlerSet.js'
import {
cancelComputerGrant,
cancelAllComputerGrants,
configureComputerUseRuntime,
getActiveComputerGrant,
expireComputerControlSessions,
setComputerGrantChangeListener,
type ComputerGrant
} from '../tools/computerGrants.js'
import { closeCuaControlSession, setComputerControlLifecycleListener } from '../tools/cuaDriver.js'
import { DesktopToolRouter } from '../tools/router.js'
import { configureCapabilityPolicies } from '../tools/capabilityRuntime.js'
import { adbBackendAvailable } from '../tools/handlers/adb.js'
@@ -85,6 +88,156 @@ const VOICE_DISCOVERY_FILE = 'desktop-voice.json'
const STATUS_HEARTBEAT_MS = 30_000
const DETACHED_START_TIMEOUT_MS = 20_000
const DETACHED_START_POLL_MS = 100
const LIFECYCLE_LOCK_TIMEOUT_MS = 25_000
const INSTANCE_LOCK_TIMEOUT_MS = 2_000
const LOCK_POLL_MS = 50
const INCOMPLETE_LOCK_GRACE_MS = 1_000
interface ProcessLockOwner {
pid: number
process_name: string
token: string
created_at: number
purpose: string
}
interface ProcessLock {
owner: ProcessLockOwner
release: () => Promise<void>
}
interface ProcessLockOptions {
timeoutMs: number
purpose: string
now?: () => number
sleep?: (ms: number) => Promise<void>
ownerAlive?: (owner: ProcessLockOwner) => boolean
}
function daemonLockPath(kind: 'lifecycle' | 'instance'): string {
return path.join(os.homedir(), '.hermes', `daemon-${kind}.lock`)
}
async function readProcessLockOwner(lockPath: string): Promise<ProcessLockOwner | null> {
try {
const parsed = JSON.parse(await fs.readFile(path.join(lockPath, 'owner.json'), 'utf8')) as Partial<ProcessLockOwner>
if (
typeof parsed.pid !== 'number' ||
typeof parsed.process_name !== 'string' ||
typeof parsed.token !== 'string' ||
typeof parsed.created_at !== 'number' ||
typeof parsed.purpose !== 'string'
) return null
return parsed as ProcessLockOwner
} catch {
return null
}
}
function processLockOwnerAlive(owner: ProcessLockOwner): boolean {
// PID liveness is deliberately authoritative here. Unlike status/stop, a
// conservative false positive only causes a bounded lock timeout; a false
// negative could let a second daemon start. It also avoids spawning tasklist
// or ps from the startup hot path (and executable-name truncation on Unix).
return isPidAlive(owner.pid)
}
async function releaseProcessLock(lockPath: string, token: string): Promise<void> {
const owner = await readProcessLockOwner(lockPath)
if (owner?.token !== token) return
try {
// Remove the ownership record before the directory. mkdir remains blocked
// until rmdir succeeds, so a newer owner cannot appear between the token
// check and release.
await fs.unlink(path.join(lockPath, 'owner.json'))
await fs.rmdir(lockPath)
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error
}
}
/** Atomic directory lock shared by compiled Bun binaries and Node-based dev
* invocations. Stale recovery quarantines an observed directory before it is
* removed, while token-checked release never recursively removes the path. */
async function acquireProcessLock(lockPath: string, options: ProcessLockOptions): Promise<ProcessLock> {
const now = options.now ?? Date.now
const sleep = options.sleep ?? (ms => new Promise(resolve => setTimeout(resolve, ms)))
const ownerAlive = options.ownerAlive ?? processLockOwnerAlive
const deadline = now() + options.timeoutMs
const owner: ProcessLockOwner = {
pid: process.pid,
process_name: path.basename(process.execPath),
token: randomUUID(),
created_at: now(),
purpose: options.purpose
}
await fs.mkdir(path.dirname(lockPath), { recursive: true })
while (true) {
try {
await fs.mkdir(lockPath)
try {
await fs.writeFile(
path.join(lockPath, 'owner.json'),
JSON.stringify(owner) + '\n',
{ encoding: 'utf8', flag: 'wx', mode: 0o600 }
)
} catch (error) {
await fs.rm(lockPath, { recursive: true, force: true }).catch(() => undefined)
throw error
}
return { owner, release: () => releaseProcessLock(lockPath, owner.token) }
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'EEXIST') throw error
}
const observed = await readProcessLockOwner(lockPath)
let stale = observed !== null && !ownerAlive(observed)
if (!observed) {
try {
const stat = await fs.stat(lockPath)
stale = now() - stat.mtimeMs >= INCOMPLETE_LOCK_GRACE_MS
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') continue
throw error
}
}
if (stale) {
const quarantinePath = `${lockPath}.stale-${process.pid}-${randomUUID()}`
try {
await fs.rename(lockPath, quarantinePath)
await fs.rm(quarantinePath, { recursive: true, force: true })
continue
} catch (error) {
if (['ENOENT', 'EEXIST', 'ENOTEMPTY'].includes((error as NodeJS.ErrnoException).code ?? '')) continue
throw error
}
}
if (now() >= deadline) {
const detail = observed
? `owner pid ${observed.pid} (${observed.purpose})`
: 'owner metadata is still being created'
throw new Error(`timed out after ${options.timeoutMs}ms waiting for ${options.purpose} lock; ${detail}`)
}
await sleep(Math.min(LOCK_POLL_MS, Math.max(1, deadline - now())))
}
}
async function withDaemonLifecycleLock<T>(operation: string, fn: () => Promise<T>): Promise<T> {
const lock = await acquireProcessLock(daemonLockPath('lifecycle'), {
timeoutMs: LIFECYCLE_LOCK_TIMEOUT_MS,
purpose: `daemon ${operation}`
})
try {
return await fn()
} finally {
try {
await lock.release()
} catch (error) {
process.stderr.write(`daemon: lifecycle_lock_release_failed: ${rpcErrorMessage(error)}\n`)
}
}
}
const DAEMON_USAGE: UsageSpec = {
name: 'daemon',
@@ -478,7 +631,7 @@ async function runElevatedDaemonLifecycle(
/** `daemon start` / `--detach` — spawn the foreground daemon as a detached
* background process (no console window on Windows), logging to a file. */
async function startDetachedDaemon(args: ParsedArgs): Promise<number> {
async function startDetachedDaemonLocked(args: ParsedArgs): Promise<number> {
const t = makeTheme({ noColor: !!args.flags['no-color'] })
const existing = await readDaemonStatus()
@@ -574,8 +727,17 @@ async function startDetachedDaemon(args: ParsedArgs): Promise<number> {
return 0
}
async function startDetachedDaemon(args: ParsedArgs): Promise<number> {
try {
return await withDaemonLifecycleLock('start', () => startDetachedDaemonLocked(args))
} catch (error) {
process.stderr.write(`daemon: lifecycle_lock_failed: ${rpcErrorMessage(error)}\n`)
return 1
}
}
/** `daemon stop` — terminate the running background daemon by its status pid. */
async function stopDaemon(args: ParsedArgs): Promise<number> {
async function stopDaemonLocked(args: ParsedArgs): Promise<number> {
const t = makeTheme({ noColor: !!args.flags['no-color'] })
const status = await readDaemonStatus()
if (!status) {
@@ -600,7 +762,16 @@ async function stopDaemon(args: ParsedArgs): Promise<number> {
return 0
}
async function restartDaemon(args: ParsedArgs): Promise<number> {
async function stopDaemon(args: ParsedArgs): Promise<number> {
try {
return await withDaemonLifecycleLock('stop', () => stopDaemonLocked(args))
} catch (error) {
process.stderr.write(`daemon: lifecycle_lock_failed: ${rpcErrorMessage(error)}\n`)
return 1
}
}
async function restartDaemonLocked(args: ParsedArgs): Promise<number> {
const t = makeTheme({ noColor: !!args.flags['no-color'] })
const existing = await readDaemonStatus()
if (existing && isDaemonProcessAlive(existing)) {
@@ -623,7 +794,16 @@ async function restartDaemon(args: ParsedArgs): Promise<number> {
}
}
await clearDaemonStatus()
return startDetachedDaemon(args)
return startDetachedDaemonLocked(args)
}
async function restartDaemon(args: ParsedArgs): Promise<number> {
try {
return await withDaemonLifecycleLock('restart', () => restartDaemonLocked(args))
} catch (error) {
process.stderr.write(`daemon: lifecycle_lock_failed: ${rpcErrorMessage(error)}\n`)
return 1
}
}
async function restartDaemonAsUser(args: ParsedArgs): Promise<number> {
@@ -717,6 +897,26 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
const human = humanFlag || (!args.flags['log-json'] && !!process.stderr.isTTY)
const log = makeLogger(human)
let instanceLock: ProcessLock
try {
instanceLock = await acquireProcessLock(daemonLockPath('instance'), {
timeoutMs: INSTANCE_LOCK_TIMEOUT_MS,
purpose: 'daemon runtime'
})
log.info({
event: 'instance_lock_acquired',
lock_path: daemonLockPath('instance'),
pid: process.pid
})
} catch (error) {
log.error({
event: 'instance_lock_failed',
message: rpcErrorMessage(error),
lock_path: daemonLockPath('instance')
})
return 1
}
// URL resolution mirrors chat/shell — explicit --remote / HERMES_RELAY_URL
// win, otherwise fall back to a stored session. resolveFirstRunUrl with
// nonInteractive:true auto-picks when exactly one session exists, throws a
@@ -735,6 +935,7 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
(e instanceof Error ? e.message : String(e)) +
' Pass --remote <url>, set HERMES_RELAY_URL, or pair first with `hermes-relay pair`.'
})
await instanceLock.release()
return 1
}
}
@@ -787,6 +988,7 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
url,
message: 'no session token. Run `hermes-relay pair --remote <url>` once, then start the daemon.'
})
await instanceLock.release()
return 1
}
@@ -853,17 +1055,26 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
? (info as { attempt?: number; delayMs?: number })
: {}
log.warn({ event: 'reconnecting', attempt: attempt ?? null, delay_ms: delayMs ?? null })
updateStatus({ state: 'reconnecting', last_event: 'reconnecting' })
const retryAt = nowSec() + Math.ceil((delayMs ?? 0) / 1000)
updateStatus({ state: 'reconnecting', last_event: 'reconnecting', reconnect_attempt: attempt ?? null, retry_at: retryAt, last_error: 'Relay connection interrupted' })
if ((attempt ?? 1) === 1) {
void appendAudit({
ts: Date.now(), kind: 'connection.state', tool: 'daemon.reconnecting', category: 'system', ok: false,
host_url: configuredUrl, summary: 'Automatic reconnect started', error: 'Relay connection interrupted'
})
}
})
relay.on('reconnected', () => {
log.info({ event: 'reconnected' })
updateStatus({ state: 'connected', last_event: 'reconnected' })
updateStatus({ state: 'connected', last_event: 'reconnected', reconnect_attempt: null, retry_at: null, last_error: null })
void appendAudit({ ts: Date.now(), kind: 'connection.state', tool: 'daemon.reconnected', category: 'system', ok: true, host_url: configuredUrl, summary: 'Relay tunnel restored' })
})
relay.on('exit', (code: unknown) => {
// Transport gave up (auth.fail, reconnect gate returned false, or
// reconnect attempts exhausted). Daemon exits non-zero so the
// service manager decides whether to restart.
log.error({ event: 'transport_exited', code: typeof code === 'number' ? code : null })
void appendAudit({ ts: Date.now(), kind: 'connection.state', tool: 'daemon.disconnected', category: 'system', ok: false, host_url: configuredUrl, summary: 'Relay transport stopped', error: 'Automatic reconnect stopped' })
// Defer exit so the log line flushes before the process dies.
setImmediate(() => process.exit(1))
})
@@ -873,11 +1084,14 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
const outcome = await relay.whenAuthResolved()
if (!outcome.ok) {
log.error({ event: 'auth_failed', reason: outcome.reason })
updateStatus({ state: 'stopped', last_event: 'auth_failed', last_error: outcome.reason, reconnect_attempt: null, retry_at: null })
void appendAudit({ ts: Date.now(), kind: 'connection.state', tool: 'daemon.auth_failed', category: 'system', ok: false, host_url: configuredUrl, summary: 'Relay authentication failed', error: outcome.reason })
try {
relay.kill()
} catch {
/* ignore */
}
await instanceLock.release()
return 1
}
@@ -886,7 +1100,7 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
server_version: relay.serverVersion ?? null,
transport: relay.authMeta?.transportHint ?? null
})
updateStatus({ state: 'connected', server_version: relay.serverVersion ?? null, last_event: 'authed' })
updateStatus({ state: 'connected', server_version: relay.serverVersion ?? null, last_event: 'authed', reconnect_attempt: null, retry_at: null, last_error: null })
// Signal downstream handlers that we're running headless. The router
// also checks this env var in its detectInteractive() fallback, so any
@@ -925,8 +1139,14 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
expires_at: grant?.expires_at ?? null,
reason: grant?.reason
})
const restoreGrantListener = setComputerGrantChangeListener(grant => {
const restoreGrantListener = setComputerGrantChangeListener((grant, controlSessionId) => {
updateStatus({ computer_grant: toDaemonGrantStatus(grant), last_event: 'grant_changed' })
if (!grant && controlSessionId) {
void closeCuaControlSession(controlSessionId, 'computer grant ended')
}
})
const restoreControlLifecycleListener = setComputerControlLifecycleListener(computerControl => {
updateStatus({ computer_control: computerControl })
})
let cancellationCheckRunning = false
@@ -936,10 +1156,10 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
try {
const request = await consumeComputerGrantCancellation()
if (request) {
const result = cancelComputerGrant(request.reason)
const result = { cancelled: cancelAllComputerGrants(request.reason) }
log.info({ event: 'computer_grant_cancelled_locally', reason: request.reason, result })
} else {
getActiveComputerGrant()
expireComputerControlSessions()
}
} catch (error) {
log.warn({ event: 'computer_grant_control_failed', message: rpcErrorMessage(error) })
@@ -1027,8 +1247,9 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
clearInterval(statusHeartbeat)
clearInterval(grantControlInterval)
restoreGrantListener()
restoreControlLifecycleListener()
try {
await clearDaemonStatus()
await clearDaemonStatus(process.pid)
} catch {
/* ignore */
}
@@ -1052,6 +1273,11 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
} catch {
/* ignore */
}
try {
await instanceLock.release()
} catch (error) {
log.warn({ event: 'instance_lock_release_failed', message: rpcErrorMessage(error) })
}
}
setupGracefulExit({ cleanups: [cleanup] })
@@ -1073,6 +1299,7 @@ export {
daemonStatusIsReady as __daemonStatusIsReadyForTests,
buildDaemonChildArgs as __buildDaemonChildArgsForTests,
buildElevationLaunchPlan as __buildElevationLaunchPlanForTests,
acquireProcessLock as __acquireProcessLockForTests,
quoteWindowsArgument as __quoteWindowsArgumentForTests,
makeLogger as __makeLoggerForTests,
readDetachedStartupFailure as __readDetachedStartupFailureForTests,
+145 -3
View File
@@ -9,7 +9,8 @@
//
// Best-effort by design: a logging failure must never break a tool dispatch.
import { appendFile, mkdir, readFile, rename, stat } from 'node:fs/promises'
import { appendFile, mkdir, readFile, readdir, rename, stat, unlink, writeFile } from 'node:fs/promises'
import { createHash } from 'node:crypto'
import { homedir } from 'node:os'
import { dirname, join } from 'node:path'
@@ -17,7 +18,7 @@ export interface AuditEntry {
/** Epoch milliseconds when the command completed. */
ts: number
/** Stable event type for consumers that do not want to infer it from fields. */
kind?: 'tool.completed'
kind?: 'tool.completed' | 'connection.state'
tool: string
category?: AuditCategory
ok: boolean
@@ -25,6 +26,20 @@ export interface AuditEntry {
request_id?: string
/** Relay identity that issued the action, when known. */
host_url?: string
relay_session_id?: string
requester_device_id?: string
run_id?: string
target_device_id?: string
backend?: 'cua' | 'legacy_compat' | 'system_capture'
dispatch?: 'background' | 'foreground_compatibility'
control_session_id?: string
target_app?: string
target_title?: string
target_pid?: number
target_window_id?: number
action?: string
verification?: 'snapshot_captured' | 'not_requested' | 'failed'
phase?: 'structured_primary' | 'explicit_compatibility' | 'pre_session_safe_fallback'
/** Handler wall time, excluding relay transport latency. */
duration_ms?: number
/** Process exit code when the handler returns one. Non-zero is attention-worthy. */
@@ -45,6 +60,11 @@ export interface AuditEntry {
/** Short success summary (path / exit code / first stdout line). */
summary?: string
error?: string
/** Opaque local evidence identifier. Screenshot pixels never enter JSONL. */
screenshot_evidence_id?: string
screenshot_mime_type?: 'image/png'
screenshot_width?: number
screenshot_height?: number
}
export type AuditCategory = 'command' | 'files' | 'screen' | 'input' | 'devices' | 'system' | 'other'
@@ -52,11 +72,109 @@ export type AuditCategory = 'command' | 'files' | 'screen' | 'input' | 'devices'
/** Rotate the log once it crosses ~1 MB, keeping a single `.1` backup. */
const MAX_BYTES = 1_000_000
const MAX_DETAIL_BYTES = 32_768
const MAX_SCREENSHOT_BYTES = 10_000_000
const MAX_SCREENSHOT_FILES = 20
export function auditLogPath(): string {
return join(homedir(), '.hermes', 'desktop-audit.jsonl')
}
export function auditEvidenceDirectory(): string {
return process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR ?? join(homedir(), '.hermes', 'activity-evidence')
}
type ScreenshotEvidence = Pick<AuditEntry, 'screenshot_evidence_id' | 'screenshot_mime_type' | 'screenshot_width' | 'screenshot_height'>
function screenshotPayload(result: unknown): { base64: string; width?: number; height?: number } | null {
if (!result || typeof result !== 'object' || Array.isArray(result)) return null
const record = result as Record<string, unknown>
const nested = record.after_screenshot
if (nested && typeof nested === 'object' && !Array.isArray(nested)) {
const found = screenshotPayload(nested)
if (found) return found
}
const base64 = typeof record.screenshot_base64 === 'string'
? record.screenshot_base64
: typeof record.bytes_base64 === 'string'
? record.bytes_base64
: null
if (!base64) return null
const display = record.display && typeof record.display === 'object' && !Array.isArray(record.display)
? record.display as Record<string, unknown>
: null
const width = typeof record.screenshot_width === 'number' ? record.screenshot_width
: typeof display?.width === 'number' ? display.width : undefined
const height = typeof record.screenshot_height === 'number' ? record.screenshot_height
: typeof display?.height === 'number' ? display.height : undefined
return { base64, width, height }
}
async function pruneScreenshotEvidence(directory: string, retentionDays: number): Promise<void> {
const now = Date.now()
const maxAgeMs = retentionDays * 24 * 60 * 60 * 1000
const records = await Promise.all((await readdir(directory, { withFileTypes: true }))
.filter(entry => entry.isFile() && /^[a-f0-9]{32}\.png$/.test(entry.name))
.map(async entry => ({ name: entry.name, stats: await stat(join(directory, entry.name)) })))
records.sort((left, right) => right.stats.mtimeMs - left.stats.mtimeMs)
await Promise.all(records
.filter((record, index) => index >= MAX_SCREENSHOT_FILES || now - record.stats.mtimeMs > maxAgeMs)
.map(record => unlink(join(directory, record.name)).catch(() => {})))
}
export async function pruneAuditScreenshotEvidence(retentionDays: 1 | 7 | 30): Promise<void> {
try {
await pruneScreenshotEvidence(auditEvidenceDirectory(), retentionDays)
} catch {
/* Missing or unreadable evidence is equivalent to an empty store. */
}
}
/** Retain a bounded, local screenshot for the activity evidence viewer. The
* audit log stores only an opaque identifier; clearing activity removes the
* evidence directory too. */
export async function persistAuditScreenshot(result: unknown, requestId: string, retentionDays = 7): Promise<ScreenshotEvidence> {
const payload = screenshotPayload(result)
if (!payload) return {}
let bytes: Buffer
try {
bytes = Buffer.from(payload.base64, 'base64')
} catch {
return {}
}
if (!bytes.length || bytes.length > MAX_SCREENSHOT_BYTES || bytes.subarray(0, 8).toString('hex') !== '89504e470d0a1a0a') return {}
const id = createHash('sha256').update(`${requestId}:${Date.now()}`).digest('hex').slice(0, 32)
const directory = auditEvidenceDirectory()
try {
await mkdir(directory, { recursive: true })
await writeFile(join(directory, `${id}.png`), bytes, { mode: 0o600, flag: 'wx' })
await pruneScreenshotEvidence(directory, retentionDays)
return {
screenshot_evidence_id: id,
screenshot_mime_type: 'image/png',
...(payload.width ? { screenshot_width: payload.width } : {}),
...(payload.height ? { screenshot_height: payload.height } : {})
}
} catch {
return {}
}
}
export async function clearAuditScreenshotEvidence(): Promise<void> {
const { rm } = await import('node:fs/promises')
await rm(auditEvidenceDirectory(), { recursive: true, force: true })
}
export async function auditScreenshotEvidenceStatus(): Promise<{ count: number; bytes: number }> {
try {
const records = await Promise.all((await readdir(auditEvidenceDirectory(), { withFileTypes: true }))
.filter(entry => entry.isFile() && /^[a-f0-9]{32}\.png$/.test(entry.name))
.map(entry => stat(join(auditEvidenceDirectory(), entry.name))))
return { count: records.length, bytes: records.reduce((total, value) => total + value.size, 0) }
} catch {
return { count: 0, bytes: 0 }
}
}
export async function appendAudit(entry: AuditEntry): Promise<void> {
const path = auditLogPath()
try {
@@ -127,7 +245,8 @@ function boundedText(value: string): { text: string; truncated: boolean } {
* bodies, environment values, or unbounded process output. */
export function auditDetails(
args: Record<string, unknown>,
result?: unknown
result?: unknown,
options: { redactComputerContent?: boolean } = {}
): Pick<AuditEntry, 'request_detail' | 'stdout' | 'stderr' | 'result_detail' | 'request_truncated' | 'stdout_truncated' | 'stderr_truncated' | 'result_truncated'> {
const safeRequest: Record<string, unknown> = {}
for (const key of [
@@ -141,6 +260,29 @@ export function auditDetails(
request_detail: request.text,
request_truncated: request.truncated || undefined
}
if (options.redactComputerContent) {
if (result && typeof result === 'object' && !Array.isArray(result)) {
const record = result as Record<string, unknown>
const target = record.target && typeof record.target === 'object' && !Array.isArray(record.target)
? record.target as Record<string, unknown>
: {}
const safeResult = {
backend: record.backend,
action: record.action,
status: record.status,
code: record.code,
target: {
pid: target.pid,
windowId: target.windowId
},
redacted: true
}
const value = boundedText(JSON.stringify(safeResult, null, 2))
details.result_detail = value.text
details.result_truncated = value.truncated || undefined
}
return details
}
if (!result || typeof result !== 'object') {
if (result !== undefined) {
const value = boundedText(String(result))
+26 -7
View File
@@ -14,6 +14,7 @@ import { execFileSync } from 'node:child_process'
import type { ProcessPrivilege } from './processPrivilege.js'
import type { HostAccessMode } from './hostAccessPolicy.js'
import type { ComputerControlLifecycleStatus } from '../tools/cuaDriver.js'
export type DaemonState = 'starting' | 'connected' | 'reconnecting' | 'stopped'
@@ -44,26 +45,39 @@ export interface DaemonStatus {
advertised_tools?: number
voice_url?: string | null
last_event?: string
reconnect_attempt?: number | null
retry_at?: number | null
last_error?: string | null
username?: string
privilege?: ProcessPrivilege
computer_use_enabled?: boolean
access_mode?: HostAccessMode
tools_enabled?: boolean
computer_grant?: DaemonComputerGrantStatus
computer_control?: ComputerControlLifecycleStatus
}
export function daemonStatusPath(): string {
return join(homedir(), '.hermes', 'daemon-status.json')
}
let statusWriteChain: Promise<void> = Promise.resolve()
export async function writeDaemonStatus(status: DaemonStatus): Promise<void> {
const filePath = daemonStatusPath()
try {
await fs.mkdir(dirname(filePath), { recursive: true })
await fs.writeFile(filePath, JSON.stringify(status, null, 2) + '\n', { mode: 0o600 })
} catch {
// Best-effort — never let status bookkeeping take down the daemon.
}
const snapshot = JSON.stringify(status, null, 2) + '\n'
statusWriteChain = statusWriteChain.then(async () => {
const temporaryPath = `${filePath}.${process.pid}.${Date.now()}.tmp`
try {
await fs.mkdir(dirname(filePath), { recursive: true })
await fs.writeFile(temporaryPath, snapshot, { mode: 0o600 })
await fs.rename(temporaryPath, filePath)
} catch {
// Best-effort — never let status bookkeeping take down the daemon.
await fs.unlink(temporaryPath).catch(() => undefined)
}
})
await statusWriteChain
}
export async function readDaemonStatus(): Promise<DaemonStatus | null> {
@@ -75,8 +89,13 @@ export async function readDaemonStatus(): Promise<DaemonStatus | null> {
}
}
export async function clearDaemonStatus(): Promise<void> {
export async function clearDaemonStatus(expectedPid?: number): Promise<void> {
await statusWriteChain
try {
if (expectedPid !== undefined) {
const current = await readDaemonStatus()
if (current?.pid !== expectedPid) return
}
await fs.unlink(daemonStatusPath())
} catch {
/* missing — fine */
+51 -3
View File
@@ -7,6 +7,10 @@ import { grantBridgeDir } from './grantBridge.js'
export interface DesktopUseSettings {
computer_use_enabled: boolean
computer_control_engine: 'legacy' | 'cua'
cua_cursor_enabled: boolean
activity_screenshot_retention_enabled: boolean
activity_screenshot_retention_days: 1 | 7 | 30
updated_at?: string
}
@@ -27,22 +31,36 @@ export function computerGrantCancellationPath(): string {
function normalizeSettings(value: unknown): DesktopUseSettings {
if (!value || typeof value !== 'object' || Array.isArray(value)) {
return { computer_use_enabled: false }
return defaultSettings()
}
const raw = value as Partial<DesktopUseSettings>
return {
computer_use_enabled: raw.computer_use_enabled === true,
computer_control_engine: raw.computer_control_engine === 'legacy' ? 'legacy' : 'cua',
cua_cursor_enabled: raw.cua_cursor_enabled === true,
activity_screenshot_retention_enabled: raw.activity_screenshot_retention_enabled !== false,
activity_screenshot_retention_days: raw.activity_screenshot_retention_days === 1 || raw.activity_screenshot_retention_days === 30 ? raw.activity_screenshot_retention_days : 7,
updated_at: typeof raw.updated_at === 'string' ? raw.updated_at : undefined
}
}
function defaultSettings(): DesktopUseSettings {
return {
computer_use_enabled: false,
computer_control_engine: 'cua',
cua_cursor_enabled: false,
activity_screenshot_retention_enabled: true,
activity_screenshot_retention_days: 7
}
}
export function readDesktopUseSettingsSync(
filePath = desktopUseSettingsPath()
): DesktopUseSettings {
try {
return normalizeSettings(JSON.parse(readFileSync(filePath, 'utf8')))
} catch {
return { computer_use_enabled: false }
return defaultSettings()
}
}
@@ -52,7 +70,7 @@ export async function readDesktopUseSettings(
try {
return normalizeSettings(JSON.parse(await readFile(filePath, 'utf8')))
} catch {
return { computer_use_enabled: false }
return defaultSettings()
}
}
@@ -68,6 +86,7 @@ export async function setDesktopUseEnabled(
filePath = desktopUseSettingsPath()
): Promise<DesktopUseSettings> {
const settings: DesktopUseSettings = {
...await readDesktopUseSettings(filePath),
computer_use_enabled: enabled,
updated_at: new Date().toISOString()
}
@@ -75,6 +94,35 @@ export async function setDesktopUseEnabled(
return settings
}
export async function setComputerControlSettings(
update: Partial<Pick<DesktopUseSettings,
'computer_control_engine' | 'cua_cursor_enabled'>>,
filePath = desktopUseSettingsPath()
): Promise<DesktopUseSettings> {
const settings: DesktopUseSettings = {
...await readDesktopUseSettings(filePath),
...update,
updated_at: new Date().toISOString()
}
await writeJsonAtomic(filePath, settings)
return settings
}
export async function setActivityScreenshotRetention(
enabled: boolean,
days: 1 | 7 | 30,
filePath = desktopUseSettingsPath()
): Promise<DesktopUseSettings> {
const settings: DesktopUseSettings = {
...await readDesktopUseSettings(filePath),
activity_screenshot_retention_enabled: enabled,
activity_screenshot_retention_days: days,
updated_at: new Date().toISOString()
}
await writeJsonAtomic(filePath, settings)
return settings
}
export async function requestComputerGrantCancellation(
reason = 'cancelled from local desktop controls',
filePath = computerGrantCancellationPath()
@@ -0,0 +1,152 @@
import { randomUUID } from 'node:crypto'
export const LEGACY_CONTROL_SESSION_ID = 'legacy-local-control-session'
export interface ComputerControlAuthority {
/** Server-attested identity for one attached desktop router lifecycle. */
controlSessionId: string
/** Canonical relay URL when known. Never supplied by remote tool arguments. */
hostUrl?: string
/** Request currently being dispatched. Bound by the router, not the caller. */
requestId?: string
/** Reserved for authenticated relay identity once the wire protocol supplies it. */
relaySessionId?: string
requesterDeviceId?: string
chatSessionId?: string
runId?: string
targetDeviceId?: string
}
export interface ComputerTarget {
pid: number
windowId: number
app?: string
title?: string
executable?: string
display?: string
folder?: string
}
export interface SnapshotBindingInput {
authority: ComputerControlAuthority
grantId: string | null
target: ComputerTarget
snapshotGeneration: string
driverElementToken?: string
ttlMs?: number
}
interface SnapshotBinding extends SnapshotBindingInput {
expiresAt: number
}
export type SensitiveTargetDecision =
| { allowed: true; reason: 'not_sensitive' }
| { allowed: false; reason: 'missing_target_identity' | 'sensitive_target'; matched?: string }
const DEFAULT_SENSITIVE_PATTERNS = Object.freeze([
/\b(password|passkey|credential|authenticator|security key)\b/i,
/\b(1password|bitwarden|keepass|lastpass|dashlane|proton pass)\b/i,
/\b(bank|banking|payment|checkout|wallet|crypto|cryptocurrency)\b/i,
/\b(certificate|private[ _-]?key|api[ _-]?key|access[ _-]?token|secret)\b/i,
/\b(user account control|windows security|credential manager|security settings)\b/i
])
/**
* Hermes-owned state for a single router lifecycle. It intentionally does not
* execute input. It binds remote requests, local grants, snapshots, and opaque
* element handles before an execution backend (legacy or CUA) is invoked.
*/
export class ComputerControlSecurityState {
private readonly seenRequestIds = new Map<string, { at: number; controlSessionId: string }>()
private readonly snapshotBindings = new Map<string, SnapshotBinding>()
constructor(
readonly authority: ComputerControlAuthority,
private readonly maxRememberedRequests = 2_048
) {}
bindRequest(
requestId: string,
authority: ComputerControlAuthority = this.authority
): ComputerControlAuthority | null {
const normalized = requestId.trim()
if (!normalized || normalized.length > 256 || this.seenRequestIds.has(normalized)) return null
this.seenRequestIds.set(normalized, { at: Date.now(), controlSessionId: authority.controlSessionId })
while (this.seenRequestIds.size > this.maxRememberedRequests) {
const oldest = this.seenRequestIds.keys().next().value as string | undefined
if (!oldest) break
this.seenRequestIds.delete(oldest)
}
return { ...authority, requestId: normalized }
}
issueSnapshotToken(input: SnapshotBindingInput): string {
if (!input.authority.controlSessionId.trim()) throw new Error('snapshot authority requires a control session id')
const token = `hermes-snapshot-${randomUUID()}`
const ttlMs = Math.max(1_000, Math.min(input.ttlMs ?? 60_000, 300_000))
this.snapshotBindings.set(token, { ...input, expiresAt: Date.now() + ttlMs })
return token
}
consumeSnapshotToken(
token: string,
expected: Omit<SnapshotBindingInput, 'ttlMs' | 'driverElementToken' | 'snapshotGeneration'> & {
snapshotGeneration?: string
}
): SnapshotBinding | null {
const binding = this.snapshotBindings.get(token)
this.snapshotBindings.delete(token)
if (!binding || binding.expiresAt <= Date.now()) return null
if (binding.authority.controlSessionId !== expected.authority.controlSessionId) return null
if (binding.grantId !== expected.grantId) return null
if (binding.target.pid !== expected.target.pid || binding.target.windowId !== expected.target.windowId) return null
if (expected.snapshotGeneration && binding.snapshotGeneration !== expected.snapshotGeneration) return null
return binding
}
revoke(): void {
this.seenRequestIds.clear()
this.snapshotBindings.clear()
}
/** Revoke only artifacts owned by one concurrent relay control session. */
revokeAuthority(controlSessionId: string): void {
for (const [requestId, binding] of this.seenRequestIds) {
if (binding.controlSessionId === controlSessionId) this.seenRequestIds.delete(requestId)
}
for (const [token, binding] of this.snapshotBindings) {
if (binding.authority.controlSessionId === controlSessionId) this.snapshotBindings.delete(token)
}
}
}
export function hasAuthenticatedControlIdentity(
authority: ComputerControlAuthority | undefined
): authority is ComputerControlAuthority & Required<Pick<ComputerControlAuthority,
'relaySessionId' | 'requesterDeviceId' | 'runId' | 'targetDeviceId'>> {
return !!authority && [
authority.controlSessionId,
authority.relaySessionId,
authority.requesterDeviceId,
authority.runId,
authority.targetDeviceId
].every(value => typeof value === 'string' && value.trim().length > 0)
}
/** Fail closed when a brokered action does not identify its app/window. */
export function evaluateSensitiveTarget(
target: Pick<ComputerTarget, 'app' | 'title' | 'executable'>,
extraPatterns: readonly RegExp[] = []
): SensitiveTargetDecision {
const identity = [target.app, target.title, target.executable]
.filter((value): value is string => typeof value === 'string' && value.trim().length > 0)
.join(' | ')
if (!identity) return { allowed: false, reason: 'missing_target_identity' }
for (const pattern of [...DEFAULT_SENSITIVE_PATTERNS, ...extraPatterns]) {
pattern.lastIndex = 0
const match = pattern.exec(identity)
if (match) return { allowed: false, reason: 'sensitive_target', matched: match[0] }
}
return { allowed: true, reason: 'not_sensitive' }
}
+128 -40
View File
@@ -1,9 +1,16 @@
import { randomUUID } from 'node:crypto'
import {
DEFAULT_CAPABILITY_POLICIES,
presetCapabilityPolicies,
type CapabilityPolicies,
type HostAccessMode
} from '../lib/hostAccessPolicy.js'
import {
LEGACY_CONTROL_SESSION_ID,
type ComputerControlAuthority,
type ComputerTarget
} from './computerControlSecurity.js'
export type ComputerGrantMode = 'observe' | 'assist' | 'control'
@@ -30,14 +37,34 @@ export interface ComputerUseRuntime {
capabilities: CapabilityPolicies
}
let activeGrant: ComputerGrant | null = null
let grantChangeListener: ((grant: ComputerGrant | null) => void) | null = null
let runtime: ComputerUseRuntime = {
interface ControlSessionState {
activeGrant: ComputerGrant | null
runtime: ComputerUseRuntime
}
const defaultRuntime = (): ComputerUseRuntime => ({
url: null,
computerUseConsented: false,
consentSource: 'none',
accessMode: 'ask',
capabilities: { ...DEFAULT_CAPABILITY_POLICIES }
})
const controlSessions = new Map<string, ControlSessionState>()
let grantChangeListener: ((grant: ComputerGrant | null, controlSessionId?: string) => void) | null = null
function authorityKey(authority?: ComputerControlAuthority): string {
return authority?.controlSessionId || LEGACY_CONTROL_SESSION_ID
}
function sessionState(authority?: ComputerControlAuthority): ControlSessionState {
const key = authorityKey(authority)
let state = controlSessions.get(key)
if (!state) {
state = { activeGrant: null, runtime: defaultRuntime() }
controlSessions.set(key, state)
}
return state
}
function nowMs(): number {
@@ -45,7 +72,7 @@ function nowMs(): number {
}
function newGrantId(): string {
return `computer-grant-${nowMs().toString(36)}-${Math.random().toString(36).slice(2, 8)}`
return `computer-grant-${randomUUID()}`
}
function parseScope(value: unknown): ComputerGrantScope {
@@ -82,18 +109,19 @@ export function normalizeComputerGrantDurationSeconds(value: unknown): number {
: 900
}
function expireIfNeeded(): void {
if (!activeGrant) {
function expireIfNeeded(authority?: ComputerControlAuthority): void {
const state = sessionState(authority)
if (!state.activeGrant) {
return
}
if (Date.parse(activeGrant.expires_at) <= nowMs()) {
activeGrant = null
grantChangeListener?.(null)
if (Date.parse(state.activeGrant.expires_at) <= nowMs()) {
state.activeGrant = null
grantChangeListener?.(null, authorityKey(authority))
}
}
export function setComputerGrantChangeListener(
listener: ((grant: ComputerGrant | null) => void) | null
listener: ((grant: ComputerGrant | null, controlSessionId?: string) => void) | null
): () => void {
const previous = grantChangeListener
grantChangeListener = listener
@@ -102,24 +130,25 @@ export function setComputerGrantChangeListener(
}
}
export function getActiveComputerGrant(): ComputerGrant | null {
expireIfNeeded()
return activeGrant
export function getActiveComputerGrant(authority?: ComputerControlAuthority): ComputerGrant | null {
expireIfNeeded(authority)
return sessionState(authority).activeGrant
}
export function getComputerGrantSummary(): Record<string, unknown> {
if (runtime.capabilities.screen_input === 'allow') {
export function getComputerGrantSummary(authority?: ComputerControlAuthority): Record<string, unknown> {
const state = sessionState(authority)
if (state.runtime.capabilities.screen_input === 'allow') {
return {
active: true,
mode: runtime.accessMode === 'full_access' ? 'full_access' : 'capability_allow',
mode: state.runtime.accessMode === 'full_access' ? 'full_access' : 'capability_allow',
expires_at: null,
scope: null,
reason: runtime.accessMode === 'full_access'
reason: state.runtime.accessMode === 'full_access'
? 'This host has Full Access.'
: 'Screen and input are allowed by this host policy.'
}
}
const grant = getActiveComputerGrant()
const grant = getActiveComputerGrant(authority)
if (!grant) {
return {
active: false,
@@ -138,16 +167,18 @@ export function getComputerGrantSummary(): Record<string, unknown> {
}
}
export function configureComputerUseRuntime(next: Partial<ComputerUseRuntime>): void {
const capabilities = next.capabilities ?? (next.accessMode ? presetCapabilityPolicies(next.accessMode) : runtime.capabilities)
runtime = {
...runtime,
export function configureComputerUseRuntime(next: Partial<ComputerUseRuntime>, authority?: ComputerControlAuthority): void {
const state = sessionState(authority)
const capabilities = next.capabilities ?? (next.accessMode ? presetCapabilityPolicies(next.accessMode) : state.runtime.capabilities)
state.runtime = {
...state.runtime,
...next,
capabilities
}
}
export function getComputerUseRuntimeSummary(): Record<string, unknown> {
export function getComputerUseRuntimeSummary(authority?: ComputerControlAuthority): Record<string, unknown> {
const runtime = sessionState(authority).runtime
return {
url: runtime.url,
consented: runtime.computerUseConsented,
@@ -165,12 +196,14 @@ export interface RequestComputerGrantInput {
reason?: unknown
}
export function requestComputerGrant(input: RequestComputerGrantInput): Record<string, unknown> {
export function requestComputerGrant(input: RequestComputerGrantInput, authority?: ComputerControlAuthority): Record<string, unknown> {
const state = sessionState(authority)
const runtime = state.runtime
if (runtime.capabilities.screen_input === 'allow') {
return {
ok: true,
full_access: runtime.accessMode === 'full_access',
grant: getComputerGrantSummary(),
grant: getComputerGrantSummary(authority),
message: 'This host already has Full Access; no task grant is required.'
}
}
@@ -184,7 +217,7 @@ export function requestComputerGrant(input: RequestComputerGrantInput): Record<s
code: 'computer_use_consent_required',
message:
'Assist/control grants require local desktop-tool consent for this relay URL before task-scoped input grants can be created.',
grant: getComputerGrantSummary()
grant: getComputerGrantSummary(authority)
}
}
@@ -197,12 +230,12 @@ export function requestComputerGrant(input: RequestComputerGrantInput): Record<s
created_at: createdAt.toISOString(),
expires_at: new Date(createdAt.getTime() + durationSeconds * 1000).toISOString()
}
activeGrant = grant
grantChangeListener?.(grant)
state.activeGrant = grant
grantChangeListener?.(grant, authorityKey(authority))
return {
ok: true,
grant: getComputerGrantSummary(),
grant: getComputerGrantSummary(authority),
message:
mode === 'observe'
? 'Observe grant active. Screenshot/status tools may run.'
@@ -210,29 +243,84 @@ export function requestComputerGrant(input: RequestComputerGrantInput): Record<s
}
}
export function cancelComputerGrant(reason = 'cancelled'): Record<string, unknown> {
const previous = getActiveComputerGrant()
activeGrant = null
if (previous) grantChangeListener?.(null)
export function cancelComputerGrant(reason = 'cancelled', authority?: ComputerControlAuthority): Record<string, unknown> {
const state = sessionState(authority)
const previous = getActiveComputerGrant(authority)
state.activeGrant = null
if (previous) grantChangeListener?.(null, authorityKey(authority))
return {
ok: true,
cancelled: previous !== null,
previous_grant: previous,
reason,
grant: getComputerGrantSummary()
grant: getComputerGrantSummary(authority)
}
}
export function hasComputerInputGrant(): boolean {
export function hasComputerInputGrant(authority?: ComputerControlAuthority): boolean {
const runtime = sessionState(authority).runtime
if (runtime.capabilities.screen_input === 'allow') return true
const grant = getActiveComputerGrant()
const grant = getActiveComputerGrant(authority)
return grant?.mode === 'assist' || grant?.mode === 'control'
}
export function hasComputerObserveGrant(): boolean {
return runtime.capabilities.screen_input === 'allow' || getActiveComputerGrant() !== null
export function hasComputerObserveGrant(authority?: ComputerControlAuthority): boolean {
const runtime = sessionState(authority).runtime
return runtime.capabilities.screen_input === 'allow' || getActiveComputerGrant(authority) !== null
}
export function hasFullHostAccess(): boolean {
return runtime.accessMode === 'full_access'
/** Seed a new router lifecycle from the legacy process-level configuration. */
export function initializeComputerControlSession(authority: ComputerControlAuthority): void {
if (controlSessions.has(authorityKey(authority))) return
const legacy = sessionState().runtime
controlSessions.set(authorityKey(authority), {
activeGrant: null,
runtime: { ...legacy, capabilities: { ...legacy.capabilities } }
})
}
export function hasFullHostAccess(authority?: ComputerControlAuthority): boolean {
return sessionState(authority).runtime.accessMode === 'full_access'
}
export function revokeComputerControlSession(authority: ComputerControlAuthority, reason = 'control session ended'): void {
cancelComputerGrant(reason, authority)
controlSessions.delete(authorityKey(authority))
}
export function cancelAllComputerGrants(_reason = 'cancelled locally'): number {
let cancelled = 0
for (const [controlSessionId, state] of controlSessions) {
if (!state.activeGrant) continue
state.activeGrant = null
cancelled += 1
grantChangeListener?.(null, controlSessionId)
}
return cancelled
}
export function expireComputerControlSessions(): void {
for (const controlSessionId of controlSessions.keys()) {
expireIfNeeded({ controlSessionId })
}
}
/** Enforce the scope shown in the approval prompt before a broker acts. */
export function computerGrantAllowsTarget(authority: ComputerControlAuthority, target: ComputerTarget): boolean {
const runtime = sessionState(authority).runtime
if (runtime.capabilities.screen_input === 'allow') return true
const grant = getActiveComputerGrant(authority)
if (!grant) return false
if (grant.scope.display && grant.scope.display !== target.display) return false
if (grant.scope.app) {
const actual = [target.app, target.executable].filter(Boolean).join(' ').toLowerCase()
if (!actual || !actual.includes(grant.scope.app.toLowerCase())) return false
}
if (grant.scope.folder) {
if (!target.folder) return false
const expected = grant.scope.folder.replaceAll('\\', '/').replace(/\/+$/, '').toLowerCase()
const actual = target.folder.replaceAll('\\', '/').toLowerCase()
if (actual !== expected && !actual.startsWith(`${expected}/`)) return false
}
return true
}
+866
View File
@@ -0,0 +1,866 @@
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process'
import { createHash } from 'node:crypto'
import { access, realpath } from 'node:fs/promises'
import { homedir } from 'node:os'
import { basename, dirname, join, relative, resolve, sep } from 'node:path'
import { readDesktopUseSettingsSync } from '../lib/desktopUseSettings.js'
const SUPPORTED_MIN_VERSION = [0, 19, 3] as const
const SUPPORTED_MAX_VERSION = [0, 20, 0] as const
const DEFAULT_TIMEOUT_MS = 8_000
const MAX_OUTPUT_BYTES = 4 * 1024 * 1024
const REQUIRED_TOOLS = Object.freeze([
'health_report',
'start_session',
'end_session',
'list_windows',
'get_window_state',
'click',
'set_value',
'press_key',
'scroll'
])
const ALLOWED_TOOLS = Object.freeze([...REQUIRED_TOOLS, 'set_agent_cursor_enabled'])
export interface CuaProcessResult {
stdout: string
stderr: string
exitCode: number
}
export interface CuaProcessRunner {
run(
executable: string,
args: readonly string[],
options?: { stdin?: string; timeoutMs?: number; signal?: AbortSignal; env?: NodeJS.ProcessEnv }
): Promise<CuaProcessResult>
}
export interface CuaRuntimeStatus {
available: boolean
ready: boolean
binaryPath?: string
binaryVersion?: string
permissionMode?: 'standard' | 'bounded'
health?: 'ok' | 'not_checked'
reason?: string
}
export interface CuaHealthStatus {
state: 'healthy' | 'degraded' | 'error'
checkedAt: string
overall?: string
reason?: string
temporaryWindowsCompatibility: true
}
export interface CuaControlSessionIdentity {
controlSessionId: string
targetDeviceId: string
hostUrl?: string
requestId?: string
relaySessionId?: string
requesterDeviceId?: string
chatSessionId?: string
runId?: string
}
export interface CuaWindowTarget {
pid: number
windowId: number
}
export interface CuaSnapshotOptions extends CuaWindowTarget {
query?: string
includeScreenshot?: boolean
maxElements?: number
maxDepth?: number
}
export interface CuaElementTarget extends CuaWindowTarget {
elementToken: string
}
export type CuaToolResult = Record<string, unknown>
export type ComputerControlBackend = 'cua' | 'legacy_compat'
export interface ComputerControlBackendSelection {
backend: ComputerControlBackend
reason: 'cua_ready' | 'explicit_compatibility' | 'cua_unavailable_before_session'
selectedAt: string
cursorEnabled: boolean
}
export interface CuaActionEvent {
action: string
target_app?: string
target_pid?: number
target_window_id?: number
verification: 'snapshot_captured' | 'not_requested' | 'failed'
occurred_at: string
}
export interface ComputerControlLifecycleStatus {
active_sessions: number
cursor_enabled: boolean
active_backend: 'cua' | 'legacy_compat' | 'mixed' | 'idle'
last_action: CuaActionEvent | null
}
interface CuaManifest {
schema_version?: unknown
binary_version?: unknown
binary_path?: unknown
}
interface CuaHealthReport {
schema_version?: unknown
driver_version?: unknown
overall?: unknown
}
interface JsonRpcResponse {
id?: number
result?: unknown
error?: { message?: unknown }
}
/**
* CUA is a separately maintained child process. Do not copy the relay daemon's
* environment into it: that environment can contain pairing tokens, provider
* credentials, and other secrets unrelated to desktop control.
*/
export function cuaDriverEnvironment(extra: NodeJS.ProcessEnv = {}): NodeJS.ProcessEnv {
const allowed = [
'SystemRoot', 'WINDIR', 'COMSPEC', 'PATHEXT', 'PATH', 'TEMP', 'TMP',
'LOCALAPPDATA', 'APPDATA', 'USERPROFILE', 'HOMEDRIVE', 'HOMEPATH',
'PROCESSOR_ARCHITECTURE', 'PROCESSOR_ARCHITEW6432', 'NUMBER_OF_PROCESSORS'
]
const env: NodeJS.ProcessEnv = {}
for (const key of allowed) {
const value = process.env[key]
if (value !== undefined) env[key] = value
}
return { ...env, ...extra, CUA_DRIVER_RS_TELEMETRY_ENABLED: '0' }
}
class CuaMcpClient {
private readonly child: ChildProcessWithoutNullStreams
private readonly pending = new Map<number, { resolve(value: unknown): void; reject(error: Error): void; timer: NodeJS.Timeout }>()
private nextId = 1
private stdout = ''
private closed = false
private constructor(binaryPath: string) {
this.child = spawn(binaryPath, ['mcp', '--socket', '\\\\.\\pipe\\cua-driver'], {
stdio: ['pipe', 'pipe', 'pipe'],
windowsHide: true,
env: cuaDriverEnvironment()
})
this.child.stdout.on('data', (chunk: Buffer) => this.receive(chunk))
this.child.on('error', error => this.failAll(new CuaRuntimeError(`CUA MCP transport failed: ${error.message}`, 'transport')))
this.child.on('close', code => this.failAll(new CuaRuntimeError(`CUA MCP transport closed (${code ?? 'unknown'})`, 'transport')))
}
static async connect(binaryPath: string, expectedVersion: string): Promise<CuaMcpClient> {
const client = new CuaMcpClient(binaryPath)
const initialized = await client.request('initialize', {
protocolVersion: '2025-06-18',
capabilities: {},
clientInfo: { name: 'hermes-relay-desktop', version: '1' }
}) as { protocolVersion?: unknown; serverInfo?: { version?: unknown } }
if (initialized.protocolVersion !== '2025-06-18') {
client.close()
throw new CuaRuntimeError('CUA MCP protocol version is incompatible', 'incompatible')
}
if (initialized.serverInfo?.version !== expectedVersion) {
client.close()
throw new CuaRuntimeError(
`CUA MCP daemon version ${String(initialized.serverInfo?.version ?? 'unknown')} does not match the verified binary ${expectedVersion}`,
'incompatible'
)
}
client.notify('notifications/initialized', {})
return client
}
async call(tool: string, args: Record<string, unknown>): Promise<CuaToolResult> {
const result = await this.request('tools/call', { name: tool, arguments: args }) as {
isError?: unknown
structuredContent?: unknown
content?: Array<{ type?: unknown; text?: unknown; data?: unknown; mimeType?: unknown }>
}
if (result.isError === true) {
// Driver errors can quote UI labels or entered values. Treat the broker
// result as sensitive and expose only the stable operation name.
throw new CuaRuntimeError(`CUA Driver ${tool} rejected the action`, 'transport')
}
if (result.structuredContent && typeof result.structuredContent === 'object' && !Array.isArray(result.structuredContent)) {
const structured = { ...(result.structuredContent as CuaToolResult) }
const image = result.content?.find(item => item.type === 'image' && typeof item.data === 'string')
if (image) {
structured.screenshot_base64 = image.data
structured.screenshot_mime_type = image.mimeType
}
return structured
}
const text = result.content?.find(item => item.type === 'text' && typeof item.text === 'string')?.text
return text ? parseJsonObject(String(text), `CUA Driver ${tool}`) : {}
}
close(): void {
if (this.closed) return
this.closed = true
this.child.stdin.end()
const timer = setTimeout(() => this.child.kill('SIGKILL'), 1_000)
timer.unref?.()
}
private request(method: string, params: Record<string, unknown>): Promise<unknown> {
if (this.closed) return Promise.reject(new CuaRuntimeError('CUA MCP transport is closed', 'closed'))
const id = this.nextId++
return new Promise((resolvePromise, reject) => {
const timer = setTimeout(() => {
this.pending.delete(id)
reject(new CuaRuntimeError(`CUA MCP ${method} timed out`, 'transport'))
}, DEFAULT_TIMEOUT_MS)
timer.unref?.()
this.pending.set(id, { resolve: resolvePromise, reject, timer })
this.child.stdin.write(`${JSON.stringify({ jsonrpc: '2.0', id, method, params })}\n`)
})
}
private notify(method: string, params: Record<string, unknown>): void {
this.child.stdin.write(`${JSON.stringify({ jsonrpc: '2.0', method, params })}\n`)
}
private receive(chunk: Buffer): void {
this.stdout += chunk.toString('utf8')
if (Buffer.byteLength(this.stdout, 'utf8') > MAX_OUTPUT_BYTES) {
this.failAll(new CuaRuntimeError('CUA MCP response exceeded the output limit', 'transport'))
this.child.kill('SIGKILL')
return
}
let newline = this.stdout.indexOf('\n')
while (newline >= 0) {
const line = this.stdout.slice(0, newline).trim()
this.stdout = this.stdout.slice(newline + 1)
if (line) this.resolveLine(line)
newline = this.stdout.indexOf('\n')
}
}
private resolveLine(line: string): void {
let message: JsonRpcResponse
try { message = JSON.parse(line) as JsonRpcResponse } catch { return }
if (typeof message.id !== 'number') return
const pending = this.pending.get(message.id)
if (!pending) return
this.pending.delete(message.id)
clearTimeout(pending.timer)
if (message.error) pending.reject(new CuaRuntimeError(`CUA MCP error: ${String(message.error.message ?? 'unknown')}`, 'transport'))
else pending.resolve(message.result)
}
private failAll(error: Error): void {
if (this.closed && this.pending.size === 0) return
this.closed = true
for (const pending of this.pending.values()) {
clearTimeout(pending.timer)
pending.reject(error)
}
this.pending.clear()
}
}
export interface CuaRuntimeOptions {
platform?: NodeJS.Platform
homeDir?: string
runner?: CuaProcessRunner
}
export class CuaRuntimeError extends Error {
constructor(
message: string,
readonly code: 'unavailable' | 'incompatible' | 'degraded' | 'transport' | 'closed'
) {
super(message)
this.name = 'CuaRuntimeError'
}
}
export class SpawnCuaProcessRunner implements CuaProcessRunner {
run(
executable: string,
args: readonly string[],
options: { stdin?: string; timeoutMs?: number; signal?: AbortSignal; env?: NodeJS.ProcessEnv } = {}
): Promise<CuaProcessResult> {
return new Promise((resolvePromise, reject) => {
const child = spawn(executable, [...args], {
stdio: ['pipe', 'pipe', 'pipe'],
windowsHide: true,
env: options.env ?? cuaDriverEnvironment()
})
let stdout = ''
let stderr = ''
let settled = false
let outputBytes = 0
const finish = (error?: Error, exitCode = -1) => {
if (settled) return
settled = true
clearTimeout(timer)
options.signal?.removeEventListener('abort', abort)
if (error) reject(error)
else resolvePromise({ stdout, stderr, exitCode })
}
const abort = () => {
child.kill('SIGKILL')
finish(new CuaRuntimeError('CUA Driver process was cancelled', 'transport'))
}
const append = (current: string, chunk: Buffer): string => {
outputBytes += chunk.length
if (outputBytes > MAX_OUTPUT_BYTES) {
child.kill('SIGKILL')
finish(new CuaRuntimeError('CUA Driver response exceeded the output limit', 'transport'))
return current
}
return current + chunk.toString('utf8')
}
const timer = setTimeout(() => {
child.kill('SIGKILL')
finish(new CuaRuntimeError('CUA Driver process timed out', 'transport'))
}, options.timeoutMs ?? DEFAULT_TIMEOUT_MS)
timer.unref?.()
options.signal?.addEventListener('abort', abort, { once: true })
child.stdout.on('data', (chunk: Buffer) => { stdout = append(stdout, chunk) })
child.stderr.on('data', (chunk: Buffer) => { stderr = append(stderr, chunk) })
child.on('error', error => finish(new CuaRuntimeError(`CUA Driver process failed: ${error.message}`, 'transport')))
child.on('close', code => finish(undefined, code ?? -1))
child.stdin.end(options.stdin ?? '')
})
}
}
function compareVersion(left: readonly number[], right: readonly number[]): number {
for (let i = 0; i < 3; i += 1) {
const delta = (left[i] ?? 0) - (right[i] ?? 0)
if (delta !== 0) return delta
}
return 0
}
function parseVersion(value: string): [number, number, number] | null {
const match = /(?:^|\s)(\d+)\.(\d+)\.(\d+)(?:\s|$)/.exec(value.trim())
return match ? [Number(match[1]), Number(match[2]), Number(match[3])] : null
}
function parseJsonObject(text: string, label: string): Record<string, unknown> {
let parsed: unknown
try {
parsed = JSON.parse(text)
} catch {
throw new CuaRuntimeError(`${label} returned invalid JSON`, 'transport')
}
if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
throw new CuaRuntimeError(`${label} returned a non-object response`, 'transport')
}
return parsed as Record<string, unknown>
}
function validatePositiveInteger(value: number, name: string): void {
if (!Number.isSafeInteger(value) || value <= 0) {
throw new CuaRuntimeError(`${name} must be a positive integer`, 'transport')
}
}
function validateElementToken(value: string): void {
if (!/^e[0-9a-f]+$/i.test(value)) {
throw new CuaRuntimeError('elementToken is not a valid opaque CUA element token', 'transport')
}
}
function derivedSessionId(identity: CuaControlSessionIdentity): string {
const fields = [
identity.controlSessionId,
identity.targetDeviceId,
identity.hostUrl ?? '',
identity.relaySessionId ?? '',
identity.requesterDeviceId ?? '',
identity.chatSessionId ?? '',
identity.runId ?? ''
]
if (!identity.controlSessionId.trim() || !identity.targetDeviceId.trim()) {
throw new CuaRuntimeError('CUA control session identity is incomplete', 'transport')
}
return `hermes-${createHash('sha256').update(fields.join('\u0000')).digest('hex').slice(0, 32)}`
}
function controlIdentityFingerprint(identity: CuaControlSessionIdentity): string {
return createHash('sha256').update([
identity.controlSessionId,
identity.targetDeviceId,
identity.hostUrl ?? '',
identity.relaySessionId ?? '',
identity.requesterDeviceId ?? '',
identity.chatSessionId ?? '',
identity.runId ?? ''
].join('\u0000')).digest('hex')
}
async function canonicalBinary(options: CuaRuntimeOptions): Promise<string> {
const platform = options.platform ?? process.platform
if (platform !== 'win32') {
throw new CuaRuntimeError(`CUA Driver adapter is not enabled on ${platform}`, 'unavailable')
}
const home = resolve(options.homeDir ?? homedir())
const packages = join(home, '.cua-driver', 'packages')
const releases = join(packages, 'releases')
const candidate = join(packages, 'current', 'cua-driver.exe')
let resolvedCandidate: string
let resolvedReleases: string
try {
;[resolvedCandidate, resolvedReleases] = await Promise.all([realpath(candidate), realpath(releases)])
await access(resolvedCandidate)
} catch {
throw new CuaRuntimeError('Canonical CUA Driver package is not installed', 'unavailable')
}
const inside = relative(resolvedReleases, resolvedCandidate)
if (!inside || inside.startsWith(`..${sep}`) || inside === '..' || resolve(resolvedCandidate) === resolve(resolvedReleases)) {
throw new CuaRuntimeError('Canonical CUA Driver package resolved outside its release store', 'incompatible')
}
if (basename(resolvedCandidate).toLowerCase() !== 'cua-driver.exe' || basename(dirname(resolvedCandidate)).length === 0) {
throw new CuaRuntimeError('Canonical CUA Driver executable path is invalid', 'incompatible')
}
return resolvedCandidate
}
export class CuaDriverAdapter {
readonly binaryPath: string
readonly binaryVersion: string
readonly permissionMode: 'standard' | 'bounded'
private readonly sessions = new Map<string, CuaControlSession>()
private constructor(
binaryPath: string,
binaryVersion: string,
permissionMode: 'standard' | 'bounded',
private readonly runner: CuaProcessRunner,
private readonly usePersistentMcp: boolean,
private readonly supportsCursorToggle: boolean
) {
this.binaryPath = binaryPath
this.binaryVersion = binaryVersion
this.permissionMode = permissionMode
}
static async connect(options: CuaRuntimeOptions = {}): Promise<CuaDriverAdapter> {
const runner = options.runner ?? new SpawnCuaProcessRunner()
const binaryPath = await canonicalBinary(options)
const run = async (args: readonly string[], stdin?: string): Promise<string> => {
const result = await runner.run(binaryPath, args, {
stdin,
env: cuaDriverEnvironment()
})
if (result.exitCode !== 0) {
throw new CuaRuntimeError(`CUA Driver probe ${args[0] ?? 'command'} failed`, 'transport')
}
return result.stdout.trim()
}
const versionText = await run(['--version'])
const versionTuple = parseVersion(versionText)
if (!versionTuple || compareVersion(versionTuple, SUPPORTED_MIN_VERSION) < 0 || compareVersion(versionTuple, SUPPORTED_MAX_VERSION) >= 0) {
throw new CuaRuntimeError(`Unsupported CUA Driver version: ${versionText || 'unknown'}`, 'incompatible')
}
const manifest = parseJsonObject(await run(['manifest', '--pretty']), 'CUA Driver manifest') as CuaManifest
if (manifest.schema_version !== '1' || manifest.binary_version !== versionTuple.join('.')) {
throw new CuaRuntimeError('CUA Driver manifest schema or version is incompatible', 'incompatible')
}
const manifestPath = typeof manifest.binary_path === 'string' ? await realpath(manifest.binary_path).catch(() => '') : ''
if (resolve(manifestPath).toLowerCase() !== resolve(binaryPath).toLowerCase()) {
throw new CuaRuntimeError('CUA Driver manifest identifies a different executable', 'incompatible')
}
const toolNames = new Set((await run(['list-tools'])).split(/\r?\n/).map(line => line.split(':', 1)[0]?.trim()).filter(Boolean))
const missingTools = REQUIRED_TOOLS.filter(tool => !toolNames.has(tool))
if (missingTools.length > 0) {
throw new CuaRuntimeError(`CUA Driver is missing required tools: ${missingTools.join(', ')}`, 'incompatible')
}
const statusText = await run(['status'])
const permissionMatch = /permission mode:\s*(standard|bounded|unrestricted)\b/i.exec(statusText)
if (!permissionMatch || permissionMatch[1]?.toLowerCase() === 'unrestricted') {
throw new CuaRuntimeError('CUA Driver permission mode is unavailable or unrestricted', 'incompatible')
}
const permissionMode = permissionMatch[1]!.toLowerCase() as 'standard' | 'bounded'
// Temporary Windows compatibility policy for trycua/cua#3103. The global
// health_report performs a whole-desktop UIA walk with a fixed timeout and
// can poison the driver's busy flag after a false timeout. Runtime
// readiness is therefore based on the canonical binary, manifest, tool
// contract, daemon status, and safe permission mode. Individual structured
// actions still fail closed. Keep health_report as an explicit diagnostic
// via healthStatus(), and remove this split when upstream fixes #3103.
return new CuaDriverAdapter(
binaryPath,
versionTuple.join('.'),
permissionMode,
runner,
options.runner === undefined,
toolNames.has('set_agent_cursor_enabled')
)
}
static async status(options: CuaRuntimeOptions = {}): Promise<CuaRuntimeStatus> {
try {
const adapter = await CuaDriverAdapter.connect(options)
return {
available: true,
ready: true,
binaryPath: adapter.binaryPath,
binaryVersion: adapter.binaryVersion,
permissionMode: adapter.permissionMode,
health: 'not_checked',
reason: 'Runtime checks passed; Windows accessibility health is checked separately.'
}
} catch (error) {
const reason = error instanceof Error ? error.message : String(error)
return { available: error instanceof CuaRuntimeError && error.code !== 'unavailable', ready: false, reason }
}
}
static async healthStatus(options: CuaRuntimeOptions = {}): Promise<CuaHealthStatus> {
const checkedAt = new Date().toISOString()
try {
const adapter = await CuaDriverAdapter.connect(options)
const runner = options.runner ?? new SpawnCuaProcessRunner()
const result = await runner.run(adapter.binaryPath, ['call', 'health_report'], {
stdin: '{}',
timeoutMs: DEFAULT_TIMEOUT_MS,
env: cuaDriverEnvironment()
})
if (result.exitCode !== 0) {
return {
state: 'error', checkedAt, temporaryWindowsCompatibility: true,
reason: `CUA Driver health probe exited ${result.exitCode}`
}
}
const health = parseJsonObject(result.stdout.trim(), 'CUA Driver health report') as CuaHealthReport
if (health.schema_version !== '1' || health.driver_version !== adapter.binaryVersion) {
return {
state: 'error', checkedAt, temporaryWindowsCompatibility: true,
reason: 'CUA Driver health report schema or version is incompatible'
}
}
const overall = String(health.overall ?? 'unknown')
return overall === 'ok'
? { state: 'healthy', checkedAt, overall, temporaryWindowsCompatibility: true }
: {
state: 'degraded', checkedAt, overall, temporaryWindowsCompatibility: true,
reason: `CUA Driver reported ${overall}; runtime remains available and actions still fail closed.`
}
} catch (error) {
return {
state: 'error', checkedAt, temporaryWindowsCompatibility: true,
reason: error instanceof Error ? error.message : String(error)
}
}
}
async openSession(identity: CuaControlSessionIdentity, signal?: AbortSignal, cursorEnabled = false): Promise<CuaControlSession> {
const id = derivedSessionId(identity)
if (this.sessions.has(id)) {
throw new CuaRuntimeError('CUA control session is already active', 'transport')
}
if (signal?.aborted) throw new CuaRuntimeError('CUA control session was cancelled', 'transport')
const mcp = this.usePersistentMcp ? await CuaMcpClient.connect(this.binaryPath, this.binaryVersion) : null
const invoke = async (tool: string, args: Record<string, unknown>, invokeSignal?: AbortSignal): Promise<CuaToolResult> => {
if (invokeSignal?.aborted) throw new CuaRuntimeError('CUA action was cancelled', 'transport')
if (!ALLOWED_TOOLS.includes(tool)) throw new CuaRuntimeError('Attempted to invoke a non-allowlisted CUA Driver tool', 'transport')
if (mcp) return mcp.call(tool, args)
const result = await this.runner.run(this.binaryPath, ['call', tool], {
stdin: JSON.stringify(args),
timeoutMs: DEFAULT_TIMEOUT_MS,
signal: invokeSignal,
env: cuaDriverEnvironment()
})
if (result.exitCode !== 0) {
throw new CuaRuntimeError(`CUA Driver ${tool} failed`, 'transport')
}
const payload = parseJsonObject(result.stdout.trim(), `CUA Driver ${tool}`)
if (payload.isError === true) throw new CuaRuntimeError(`CUA Driver ${tool} rejected the action`, 'transport')
return payload
}
await invoke('start_session', { session: id, capture_scope: 'window' }, signal).catch(error => {
mcp?.close()
throw error
})
if (this.supportsCursorToggle) {
await invoke('set_agent_cursor_enabled', { session: id, enabled: cursorEnabled }, signal).catch(error => {
mcp?.close()
throw error
})
}
const session = new CuaControlSession(
id,
invoke,
() => mcp?.close(),
() => this.sessions.delete(id)
)
this.sessions.set(id, session)
return session
}
async closeAll(reason = 'Hermes control authority ended'): Promise<void> {
const sessions = [...this.sessions.values()]
await Promise.allSettled(sessions.map(session => session.close(reason)))
}
}
export class CuaControlSession {
private closed = false
constructor(
private readonly id: string,
private readonly invoke: (tool: string, args: Record<string, unknown>, signal?: AbortSignal) => Promise<CuaToolResult>,
private readonly closeTransport: () => void,
private readonly onClose: () => void
) {}
private ensureOpen(): void {
if (this.closed) throw new CuaRuntimeError('CUA control session is closed', 'closed')
}
async listWindows(pid?: number, signal?: AbortSignal): Promise<CuaToolResult> {
this.ensureOpen()
if (pid !== undefined) validatePositiveInteger(pid, 'pid')
return this.invoke('list_windows', pid === undefined ? {} : { pid }, signal)
}
async snapshot(options: CuaSnapshotOptions, signal?: AbortSignal): Promise<CuaToolResult> {
this.ensureOpen()
validatePositiveInteger(options.pid, 'pid')
validatePositiveInteger(options.windowId, 'windowId')
return this.invoke('get_window_state', {
pid: options.pid,
window_id: options.windowId,
session: this.id,
query: options.query,
include_screenshot: options.includeScreenshot,
max_elements: options.maxElements ?? 1_000,
max_depth: options.maxDepth ?? 20
}, signal)
}
async clickElement(target: CuaElementTarget, signal?: AbortSignal): Promise<CuaToolResult> {
this.ensureOpen()
validatePositiveInteger(target.pid, 'pid')
validatePositiveInteger(target.windowId, 'windowId')
validateElementToken(target.elementToken)
return this.invoke('click', {
pid: target.pid,
window_id: target.windowId,
element_token: target.elementToken,
session: this.id,
scope: 'window',
delivery_mode: 'background'
}, signal)
}
async setElementValue(target: CuaElementTarget, value: string, signal?: AbortSignal): Promise<CuaToolResult> {
this.ensureOpen()
validatePositiveInteger(target.pid, 'pid')
validatePositiveInteger(target.windowId, 'windowId')
validateElementToken(target.elementToken)
return this.invoke('set_value', {
pid: target.pid,
window_id: target.windowId,
element_token: target.elementToken,
value,
session: this.id
}, signal)
}
async pressKey(target: CuaWindowTarget, key: string, signal?: AbortSignal): Promise<CuaToolResult> {
this.ensureOpen()
validatePositiveInteger(target.pid, 'pid')
validatePositiveInteger(target.windowId, 'windowId')
if (!/^(?:return|tab|escape|up|down|left|right|space|delete|home|end|pageup|pagedown|f(?:[1-9]|1[0-2])|[a-z0-9])$/i.test(key)) {
throw new CuaRuntimeError('key is not in the allowlisted CUA key vocabulary', 'transport')
}
return this.invoke('press_key', {
pid: target.pid,
window_id: target.windowId,
key: key.toLowerCase(),
session: this.id,
scope: 'window',
delivery_mode: 'background'
}, signal)
}
async scroll(target: CuaElementTarget, direction: 'up' | 'down' | 'left' | 'right', amount: number, signal?: AbortSignal): Promise<CuaToolResult> {
this.ensureOpen()
validatePositiveInteger(target.pid, 'pid')
validatePositiveInteger(target.windowId, 'windowId')
validateElementToken(target.elementToken)
if (!Number.isSafeInteger(amount) || amount < 1 || amount > 20) {
throw new CuaRuntimeError('scroll amount must be an integer from 1 to 20', 'transport')
}
return this.invoke('scroll', {
pid: target.pid,
window_id: target.windowId,
element_token: target.elementToken,
direction,
amount,
session: this.id,
delivery_mode: 'background'
}, signal)
}
async close(_reason = 'Hermes control session ended'): Promise<void> {
if (this.closed) return
this.closed = true
this.onClose()
try {
await this.invoke('end_session', { session: this.id })
} finally {
this.closeTransport()
}
}
}
let sharedAdapter: Promise<CuaDriverAdapter> | null = null
const sharedSessions = new Map<string, Promise<CuaControlSession>>()
const sharedSessionIdentities = new Map<string, string>()
const backendSelections = new Map<string, ComputerControlBackendSelection>()
let lastActionEvent: CuaActionEvent | null = null
let lifecycleListener: ((status: ComputerControlLifecycleStatus) => void) | null = null
export function setComputerControlLifecycleListener(
listener: ((status: ComputerControlLifecycleStatus) => void) | null
): () => void {
lifecycleListener = listener
listener?.(computerControlLifecycleStatus())
return () => { if (lifecycleListener === listener) lifecycleListener = null }
}
function publishLifecycle(): void {
lifecycleListener?.(computerControlLifecycleStatus())
}
let testSessionFactory: ((identity: CuaControlSessionIdentity) => Promise<CuaControlSession>) | null = null
/** Test-only dependency seam; production callers must never configure this. */
export function setCuaControlSessionFactoryForTests(
factory: ((identity: CuaControlSessionIdentity) => Promise<CuaControlSession>) | null
): void {
testSessionFactory = factory
sharedSessions.clear()
sharedSessionIdentities.clear()
backendSelections.clear()
sharedAdapter = null
}
/** Select once per outer Hermes control session; an active session never changes backend. */
export async function selectComputerControlBackend(
controlSessionId: string,
selected: 'legacy' | 'cua',
cursorEnabled: boolean
): Promise<ComputerControlBackendSelection> {
const existing = backendSelections.get(controlSessionId)
if (existing) return existing
let selection: ComputerControlBackendSelection
if (selected === 'legacy') {
selection = {
backend: 'legacy_compat',
reason: 'explicit_compatibility',
selectedAt: new Date().toISOString(),
cursorEnabled: false
}
} else if (testSessionFactory) {
selection = { backend: 'cua', reason: 'cua_ready', selectedAt: new Date().toISOString(), cursorEnabled }
} else {
try {
sharedAdapter ??= CuaDriverAdapter.connect()
await sharedAdapter
selection = { backend: 'cua', reason: 'cua_ready', selectedAt: new Date().toISOString(), cursorEnabled }
} catch {
sharedAdapter = null
selection = {
backend: 'legacy_compat',
reason: 'cua_unavailable_before_session',
selectedAt: new Date().toISOString(),
cursorEnabled: false
}
}
}
backendSelections.set(controlSessionId, selection)
publishLifecycle()
return selection
}
export function recordCuaActionEvent(event: Omit<CuaActionEvent, 'occurred_at'>): void {
lastActionEvent = { ...event, occurred_at: new Date().toISOString() }
publishLifecycle()
}
export function computerControlLifecycleStatus(): ComputerControlLifecycleStatus {
const active = [...backendSelections.values()]
const backends = new Set(active.map(item => item.backend))
return {
active_sessions: active.length,
cursor_enabled: active.some(item => item.backend === 'cua' && item.cursorEnabled),
active_backend: active.length === 0 ? 'idle' : backends.size > 1 ? 'mixed' : active[0]!.backend,
last_action: lastActionEvent
}
}
/** Return the one bounded CUA session owned by an authenticated Hermes control session. */
export async function getCuaControlSession(identity: CuaControlSessionIdentity): Promise<CuaControlSession> {
const key = identity.controlSessionId
const fingerprint = controlIdentityFingerprint(identity)
const existingFingerprint = sharedSessionIdentities.get(key)
if (existingFingerprint && existingFingerprint !== fingerprint) {
throw new CuaRuntimeError('CUA control session identity changed after session creation', 'incompatible')
}
let session = sharedSessions.get(key)
if (!session) {
if (testSessionFactory) {
session = testSessionFactory(identity)
} else {
sharedAdapter ??= CuaDriverAdapter.connect()
const settings = readDesktopUseSettingsSync()
const selection = backendSelections.get(key)
session = sharedAdapter.then(adapter => adapter.openSession(
identity,
undefined,
selection?.backend === 'cua' ? selection.cursorEnabled : settings.cua_cursor_enabled
))
}
sharedSessions.set(key, session)
sharedSessionIdentities.set(key, fingerprint)
void session.catch(() => {
sharedSessions.delete(key)
sharedSessionIdentities.delete(key)
})
}
return session
}
/** Revoke a CUA cursor/session when its outer Hermes authority ends. */
export async function closeCuaControlSession(controlSessionId: string, reason?: string): Promise<void> {
const session = sharedSessions.get(controlSessionId)
sharedSessions.delete(controlSessionId)
sharedSessionIdentities.delete(controlSessionId)
backendSelections.delete(controlSessionId)
publishLifecycle()
if (session) await session.then(value => value.close(reason)).catch(() => undefined)
}
export async function closeAllCuaControlSessions(reason?: string): Promise<void> {
const sessions = [...sharedSessions.values()]
sharedSessions.clear()
sharedSessionIdentities.clear()
backendSelections.clear()
publishLifecycle()
await Promise.allSettled(sessions.map(session => session.then(value => value.close(reason))))
sharedAdapter = null
}
+390
View File
@@ -0,0 +1,390 @@
import { createHash } from 'node:crypto'
import { createReadStream } from 'node:fs'
import { access, mkdtemp, realpath, rm, writeFile } from 'node:fs/promises'
import { homedir, tmpdir } from 'node:os'
import { delimiter, join, relative, resolve, sep, win32 } from 'node:path'
import {
CuaDriverAdapter,
SpawnCuaProcessRunner,
cuaDriverEnvironment,
type CuaProcessRunner
} from './cuaDriver.js'
export const CUA_SUPPORTED_MIN_VERSION = '0.19.3'
export const CUA_SUPPORTED_MAX_EXCLUSIVE = '0.20.0'
const TRUSTED_REPOSITORY = 'trycua/cua'
const TRUSTED_PRODUCT = 'cua-driver-rs'
const RELEASE_BASE = 'https://github.com/trycua/cua/releases/download'
const MAX_INSTALLER_BYTES = 2 * 1024 * 1024
interface FetchResponse {
ok: boolean
status: number
arrayBuffer(): Promise<ArrayBuffer>
json(): Promise<unknown>
}
export type CuaManagementFetch = (url: string) => Promise<FetchResponse>
export interface CuaUpdateStatus {
checked_at?: string
current_version?: string
latest_version?: string
update_available: boolean
compatible: boolean
error?: string
release_notes_url?: string
}
export interface CuaManagementStatus {
installed: boolean
canonical_path: string | null
discovered_path: string | null
stale_path_shim: boolean
current_version: string | null
compatible: boolean
compatibility_reason: string | null
binary_sha256: string | null
release_source: typeof TRUSTED_REPOSITORY
telemetry_enabled: false
bundled: false
supported_range: {
minimum: typeof CUA_SUPPORTED_MIN_VERSION
maximum_exclusive: typeof CUA_SUPPORTED_MAX_EXCLUSIVE
}
update?: CuaUpdateStatus
operation?: {
kind: 'install' | 'update'
state: 'completed'
version: string
release_manifest_verified: true
installer_checksum_verified: true
runtime_verified: true
}
}
export interface CuaManagementOptions {
platform?: NodeJS.Platform
arch?: string
homeDir?: string
path?: string
runner?: CuaProcessRunner
fetch?: CuaManagementFetch
systemRoot?: string
}
interface TrustedRelease {
version: string
installerUrl: string
installerSha256: string
}
function versionTuple(value: string): [number, number, number] | null {
const match = /(?:^|\s|v)(\d+)\.(\d+)\.(\d+)(?:\s|$)/.exec(value.trim())
return match ? [Number(match[1]), Number(match[2]), Number(match[3])] : null
}
function compareVersion(left: string, right: string): number | null {
const a = versionTuple(left)
const b = versionTuple(right)
if (!a || !b) return null
for (let index = 0; index < 3; index += 1) {
const delta = a[index] - b[index]
if (delta !== 0) return delta
}
return 0
}
export function isSupportedCuaVersion(value: string): boolean {
const minimum = compareVersion(value, CUA_SUPPORTED_MIN_VERSION)
const maximum = compareVersion(value, CUA_SUPPORTED_MAX_EXCLUSIVE)
return minimum !== null && maximum !== null && minimum >= 0 && maximum < 0
}
function canonicalPaths(homeDir: string): { executable: string; releases: string } {
const packages = join(resolve(homeDir), '.cua-driver', 'packages')
return {
executable: join(packages, 'current', 'cua-driver.exe'),
releases: join(packages, 'releases')
}
}
async function resolveCanonical(options: CuaManagementOptions): Promise<string | null> {
if ((options.platform ?? process.platform) !== 'win32') return null
const paths = canonicalPaths(options.homeDir ?? homedir())
try {
const [binary, releases] = await Promise.all([
realpath(paths.executable),
realpath(paths.releases)
])
await access(binary)
const inside = relative(releases, binary)
if (!inside || inside === '..' || inside.startsWith(`..${sep}`)) return null
return binary
} catch {
return null
}
}
async function firstPathCandidate(options: CuaManagementOptions): Promise<string | null> {
if ((options.platform ?? process.platform) !== 'win32') return null
for (const entry of (options.path ?? process.env.PATH ?? '').split(delimiter)) {
const trimmed = entry.trim().replace(/^"|"$/g, '')
if (!trimmed) continue
const candidate = join(trimmed, 'cua-driver.exe')
try {
await access(candidate)
return await realpath(candidate)
} catch {
// Continue to the next PATH entry.
}
}
return null
}
async function sha256File(path: string): Promise<string> {
return new Promise((resolvePromise, reject) => {
const hash = createHash('sha256')
const stream = createReadStream(path)
stream.on('data', chunk => hash.update(chunk))
stream.on('error', reject)
stream.on('end', () => resolvePromise(hash.digest('hex')))
})
}
function safeError(error: unknown): string {
return (error instanceof Error ? error.message : String(error)).slice(0, 500)
}
async function binaryVersion(binary: string, runner: CuaProcessRunner): Promise<string | null> {
const result = await runner.run(binary, ['--version'], {
timeoutMs: 8_000,
env: cuaDriverEnvironment()
})
if (result.exitCode !== 0) return null
const tuple = versionTuple(result.stdout)
return tuple?.join('.') ?? null
}
export async function getCuaManagementStatus(
options: CuaManagementOptions = {}
): Promise<CuaManagementStatus> {
const runner = options.runner ?? new SpawnCuaProcessRunner()
const [canonical, discovered] = await Promise.all([
resolveCanonical(options),
firstPathCandidate(options)
])
let currentVersion: string | null = null
let binarySha256: string | null = null
let compatibilityReason: string | null = null
if (canonical) {
try {
;[currentVersion, binarySha256] = await Promise.all([
binaryVersion(canonical, runner),
sha256File(canonical)
])
} catch (error) {
compatibilityReason = safeError(error)
}
}
const compatible = !!currentVersion && isSupportedCuaVersion(currentVersion)
if (canonical && !compatibilityReason && !compatible) {
compatibilityReason = currentVersion
? `CUA Driver ${currentVersion} is outside the supported range`
: 'CUA Driver version could not be verified'
}
return {
installed: canonical !== null,
canonical_path: canonical,
discovered_path: discovered,
stale_path_shim: !!canonical && !!discovered && resolve(canonical).toLowerCase() !== resolve(discovered).toLowerCase(),
current_version: currentVersion,
compatible,
compatibility_reason: compatibilityReason,
binary_sha256: binarySha256,
release_source: TRUSTED_REPOSITORY,
telemetry_enabled: false,
bundled: false,
supported_range: {
minimum: CUA_SUPPORTED_MIN_VERSION,
maximum_exclusive: CUA_SUPPORTED_MAX_EXCLUSIVE
}
}
}
function normalizeUpdatePayload(payload: Record<string, unknown>): CuaUpdateStatus {
const latest = typeof payload.latest_version === 'string' ? payload.latest_version : undefined
const error = typeof payload.error === 'string' && payload.error.trim() ? payload.error.slice(0, 500) : undefined
return {
checked_at: typeof payload.checked_at === 'string' ? payload.checked_at : undefined,
current_version: typeof payload.current_version === 'string' ? payload.current_version : undefined,
latest_version: latest,
update_available: payload.update_available === true,
compatible: !!latest && isSupportedCuaVersion(latest),
error,
release_notes_url: typeof payload.release_notes_url === 'string' ? payload.release_notes_url : undefined
}
}
export async function checkCuaUpdate(
options: CuaManagementOptions = {}
): Promise<CuaManagementStatus> {
const runner = options.runner ?? new SpawnCuaProcessRunner()
let status = await getCuaManagementStatus({ ...options, runner })
for (let attempt = 0; !status.installed && attempt < 2; attempt += 1) {
await new Promise(resolvePromise => setTimeout(resolvePromise, 50))
status = await getCuaManagementStatus({ ...options, runner })
}
if (!status.canonical_path) {
return { ...status, update: { update_available: false, compatible: false, error: 'CUA Driver is not installed' } }
}
const result = await runner.run(status.canonical_path, ['check-update', '--json', '--no-cache'], {
timeoutMs: 30_000,
env: cuaDriverEnvironment()
})
if (result.exitCode !== 0) {
return {
...status,
update: {
update_available: false,
compatible: false,
error: result.stderr.trim().slice(0, 500) || `check-update exited ${result.exitCode}`
}
}
}
try {
const payload = JSON.parse(result.stdout) as Record<string, unknown>
return { ...status, update: normalizeUpdatePayload(payload) }
} catch {
return { ...status, update: { update_available: false, compatible: false, error: 'CUA Driver returned invalid update JSON' } }
}
}
async function trustedRelease(version: string, fetchImpl: CuaManagementFetch): Promise<TrustedRelease> {
if (!isSupportedCuaVersion(version)) {
throw new Error(`CUA Driver ${version} is outside the Hermes-supported range`)
}
const tag = `cua-driver-rs-v${version}`
const base = `${RELEASE_BASE}/${tag}`
const manifestResponse = await fetchImpl(`${base}/release-manifest.json`)
if (!manifestResponse.ok) throw new Error(`CUA release manifest request failed (${manifestResponse.status})`)
const manifest = await manifestResponse.json() as Record<string, unknown>
if (
manifest.schemaVersion !== 1 || manifest.repository !== TRUSTED_REPOSITORY ||
manifest.product !== TRUSTED_PRODUCT || manifest.version !== version || manifest.tag !== tag
) {
throw new Error('CUA release publisher or version manifest is invalid')
}
const assets = Array.isArray(manifest.assets) ? manifest.assets : []
const installer = assets.find(asset =>
!!asset && typeof asset === 'object' && (asset as Record<string, unknown>).name === 'install.ps1'
) as Record<string, unknown> | undefined
const checksum = typeof installer?.sha256 === 'string' ? installer.sha256.toLowerCase() : ''
if (!/^[0-9a-f]{64}$/.test(checksum)) throw new Error('CUA release installer checksum is missing')
return { version, installerUrl: `${base}/install.ps1`, installerSha256: checksum }
}
async function applyTrustedRelease(
release: TrustedRelease,
runner: CuaProcessRunner,
fetchImpl: CuaManagementFetch,
systemRootOverride?: string
): Promise<void> {
const response = await fetchImpl(release.installerUrl)
if (!response.ok) throw new Error(`CUA installer request failed (${response.status})`)
const script = Buffer.from(await response.arrayBuffer())
if (script.byteLength === 0 || script.byteLength > MAX_INSTALLER_BYTES) {
throw new Error('CUA installer size is invalid')
}
const actual = createHash('sha256').update(script).digest('hex')
if (actual !== release.installerSha256) throw new Error('CUA installer checksum verification failed')
const directory = await mkdtemp(join(tmpdir(), 'hermes-cua-install-'))
const path = join(directory, 'install.ps1')
try {
await writeFile(path, script, { mode: 0o600 })
const configuredSystemRoot = systemRootOverride ?? process.env.SystemRoot ?? process.env.WINDIR
if (!configuredSystemRoot || !configuredSystemRoot.match(/^[A-Za-z]:[\\/]/)) {
throw new Error('Windows system PowerShell path is unavailable')
}
const powershell = win32.join(win32.resolve(configuredSystemRoot), 'System32', 'WindowsPowerShell', 'v1.0', 'powershell.exe')
const result = await runner.run(powershell, [
'-NoProfile',
'-NonInteractive',
'-ExecutionPolicy',
'Bypass',
'-File',
path
], {
timeoutMs: 15 * 60_000,
env: cuaDriverEnvironment({
CUA_DRIVER_RS_VERSION: release.version,
CUA_DRIVER_INSTALL_CHANNEL: 'install_script'
})
})
if (result.exitCode !== 0) {
throw new Error(result.stderr.trim().slice(0, 500) || `CUA installer exited ${result.exitCode}`)
}
} finally {
await rm(directory, { recursive: true, force: true })
}
}
async function mutateCua(
kind: 'install' | 'update',
version: string,
options: CuaManagementOptions
): Promise<CuaManagementStatus> {
if ((options.platform ?? process.platform) !== 'win32') throw new Error('CUA management is currently available on Windows only')
const runner = options.runner ?? new SpawnCuaProcessRunner()
const fetchImpl = options.fetch ?? (globalThis.fetch as unknown as CuaManagementFetch)
const release = await trustedRelease(version, fetchImpl)
await applyTrustedRelease(release, runner, fetchImpl, options.systemRoot)
const status = await getCuaManagementStatus({ ...options, runner })
if (!status.installed || status.current_version !== version || !status.compatible) {
throw new Error(
`CUA canonical package did not pass post-install version verification ` +
`(installed=${status.installed}, version=${status.current_version ?? 'unknown'}, compatible=${status.compatible})`
)
}
const runtime = await CuaDriverAdapter.status({
platform: options.platform,
homeDir: options.homeDir,
runner
})
if (
!runtime.ready || runtime.binaryVersion !== version ||
!runtime.binaryPath || !status.canonical_path ||
resolve(runtime.binaryPath).toLowerCase() !== resolve(status.canonical_path).toLowerCase()
) {
throw new Error(runtime.reason ?? 'CUA canonical manifest, path, or health verification failed')
}
return {
...status,
operation: {
kind,
state: 'completed',
version,
release_manifest_verified: true,
installer_checksum_verified: true,
runtime_verified: true
}
}
}
export async function installCuaDriver(options: CuaManagementOptions = {}): Promise<CuaManagementStatus> {
const existing = await getCuaManagementStatus(options)
if (existing.installed) return existing
return mutateCua('install', CUA_SUPPORTED_MIN_VERSION, options)
}
export async function updateCuaDriver(options: CuaManagementOptions = {}): Promise<CuaManagementStatus> {
const checked = await checkCuaUpdate(options)
const update = checked.update
if (!update || update.error) throw new Error(update?.error ?? 'CUA update check failed')
if (!update.update_available) return checked
if (!update.latest_version || !update.compatible) {
throw new Error(`CUA Driver ${update.latest_version ?? 'unknown'} is available but unsupported by this Hermes Relay build`)
}
return mutateCua('update', update.latest_version, options)
}
+243 -19
View File
@@ -2,6 +2,7 @@ import { spawn } from 'node:child_process'
import {
cancelComputerGrant,
computerGrantAllowsTarget,
getActiveComputerGrant,
getComputerGrantSummary,
getComputerUseRuntimeSummary,
@@ -13,12 +14,22 @@ import {
requestComputerGrant,
type ComputerGrantMode
} from '../computerGrants.js'
import { evaluateSensitiveTarget, hasAuthenticatedControlIdentity, type ComputerTarget } from '../computerControlSecurity.js'
import {
CuaDriverAdapter,
closeCuaControlSession,
computerControlLifecycleStatus,
getCuaControlSession,
recordCuaActionEvent,
selectComputerControlBackend
} from '../cuaDriver.js'
import { approveComputerGrant } from '../computerActionApproval.js'
import {
runComputerInputAction,
validateComputerAction
} from '../computerInput.js'
import type { ToolHandler } from '../router.js'
import type { ToolContext, ToolHandler } from '../router.js'
import { readDesktopUseSettingsSync } from '../../lib/desktopUseSettings.js'
import { screenshotHandler } from './screenshot.js'
const STATUS_TIMEOUT_MS = 5_000
@@ -212,13 +223,18 @@ function pngDimensions(base64: string): { width: number; height: number } | null
}
}
function failure(code: string, message: string, extra: Record<string, unknown> = {}): Record<string, unknown> {
function failure(
code: string,
message: string,
extra: Record<string, unknown> = {},
authority?: ToolContext['controlSession']
): Record<string, unknown> {
return {
ok: false,
code,
message,
...EXPERIMENTAL_META,
grant: getComputerGrantSummary(),
grant: getComputerGrantSummary(authority),
...extra
}
}
@@ -230,17 +246,178 @@ function parseGrantMode(value: unknown): ComputerGrantMode | null {
return null
}
function positiveInteger(value: unknown): number | null {
return typeof value === 'number' && Number.isSafeInteger(value) && value > 0 ? value : null
}
function cuaTarget(args: Record<string, unknown>): Pick<ComputerTarget, 'pid' | 'windowId'> | null {
const pid = positiveInteger(args.pid)
const windowId = positiveInteger(args.window_id)
if (pid === null || windowId === null) return null
return { pid, windowId }
}
async function cuaPreflight(args: Record<string, unknown>, ctx: ToolContext): Promise<
| { ok: true; target: ComputerTarget; grantId: string | null; session: Awaited<ReturnType<typeof getCuaControlSession>> }
| { ok: false; result: Record<string, unknown> }
> {
if (!hasAuthenticatedControlIdentity(ctx.controlSession) || !ctx.controlSecurity) {
return { ok: false, result: failure('authenticated_control_session_required', 'Structured CUA control requires server-attested relay, requester, run, and target identity.', {}, ctx.controlSession) }
}
const requested = cuaTarget(args)
if (!requested) return { ok: false, result: failure('invalid_target', 'Structured CUA control requires positive pid and window_id.', {}, ctx.controlSession) }
const session = await getCuaControlSession({ ...ctx.controlSession, targetDeviceId: ctx.controlSession.targetDeviceId })
const listed = await session.listWindows(requested.pid, ctx.abortSignal)
const windows = Array.isArray(listed.windows) ? listed.windows : []
const exact = windows.find(item => isObject(item) && item.window_id === requested.windowId)
if (!isObject(exact)) {
return { ok: false, result: failure('target_not_found', 'CUA Driver did not report the exact requested PID and window.', {}, ctx.controlSession) }
}
const target: ComputerTarget = {
...requested,
app: typeof exact.app_name === 'string' ? exact.app_name : undefined,
title: typeof exact.title === 'string' ? exact.title : undefined,
executable: typeof exact.executable === 'string' ? exact.executable : undefined
}
const sensitive = evaluateSensitiveTarget(target)
if (!sensitive.allowed) {
return { ok: false, result: failure('sensitive_target_blocked', 'Structured control is blocked for missing or sensitive application identity.', { reason: sensitive.reason }, ctx.controlSession) }
}
if (!computerGrantAllowsTarget(ctx.controlSession, target)) {
return { ok: false, result: failure('grant_target_mismatch', 'The active computer grant does not cover this application target.', {}, ctx.controlSession) }
}
return { ok: true, target, grantId: getActiveComputerGrant(ctx.controlSession)?.id ?? null, session }
}
async function cuaSnapshot(args: Record<string, unknown>, ctx: ToolContext): Promise<Record<string, unknown>> {
const check = await cuaPreflight(args, ctx)
if (!check.ok) return check.result
const authority = ctx.controlSession!
const raw = await check.session.snapshot({
pid: check.target.pid,
windowId: check.target.windowId,
query: argString(args.query).trim() || undefined,
includeScreenshot: args.include_screenshot !== false,
maxElements: typeof args.max_elements === 'number' ? args.max_elements : undefined,
maxDepth: typeof args.max_depth === 'number' ? args.max_depth : undefined
}, ctx.abortSignal)
const generation = typeof raw.snapshot_id === 'string' ? raw.snapshot_id : ''
if (!generation) return failure('invalid_backend_response', 'CUA Driver snapshot did not include a generation.', {}, authority)
const elements = Array.isArray(raw.elements) ? raw.elements : []
const safeElements = elements.slice(0, 1_000).map(item => {
if (!isObject(item)) return item
const driverToken = typeof item.element_token === 'string' ? item.element_token : undefined
const output = { ...item }
delete output.element_token
if (driverToken) {
output.snapshot_token = ctx.controlSecurity!.issueSnapshotToken({
authority,
grantId: check.grantId,
target: check.target,
snapshotGeneration: generation,
driverElementToken: driverToken
})
}
return output
})
return {
ok: true,
...EXPERIMENTAL_META,
backend: 'cua_driver',
snapshot_generation: generation,
target: check.target,
elements: safeElements,
tree_markdown: raw.tree_markdown,
screenshot_base64: raw.screenshot_base64,
screenshot_width: raw.screenshot_width,
screenshot_height: raw.screenshot_height,
truncated: elements.length > safeElements.length
}
}
async function cuaAction(args: Record<string, unknown>, ctx: ToolContext): Promise<Record<string, unknown>> {
const check = await cuaPreflight(args, ctx)
if (!check.ok) return check.result
const authority = ctx.controlSession!
const token = argString(args.snapshot_token).trim()
const binding = token ? ctx.controlSecurity!.consumeSnapshotToken(token, {
authority,
grantId: check.grantId,
target: check.target,
...(typeof args.snapshot_generation === 'string' ? { snapshotGeneration: args.snapshot_generation } : {})
}) : null
if (!binding?.driverElementToken) {
return failure('invalid_or_stale_snapshot', 'A fresh one-use snapshot_token for this exact session, grant, PID, and window is required.', {}, authority)
}
const element = { pid: check.target.pid, windowId: check.target.windowId, elementToken: binding.driverElementToken }
const action = argString(args.action).trim()
let result: Record<string, unknown>
if (action === 'click_element') result = await check.session.clickElement(element, ctx.abortSignal)
else if (action === 'set_value') result = await check.session.setElementValue(element, argString(args.value), ctx.abortSignal)
else if (action === 'press_key') result = await check.session.pressKey(check.target, argString(args.key), ctx.abortSignal)
else if (action === 'scroll_element') {
const direction = argString(args.direction) as 'up' | 'down' | 'left' | 'right'
if (!['up', 'down', 'left', 'right'].includes(direction)) return failure('invalid_request', 'direction must be up, down, left, or right.', {}, authority)
result = await check.session.scroll(element, direction, positiveInteger(args.amount) ?? 1, ctx.abortSignal)
} else return failure('invalid_request', 'Unsupported structured CUA action.', {}, authority)
const after = await check.session.snapshot({ pid: check.target.pid, windowId: check.target.windowId, includeScreenshot: false }, ctx.abortSignal)
recordCuaActionEvent({
action,
target_app: check.target.app,
target_pid: check.target.pid,
target_window_id: check.target.windowId,
verification: 'snapshot_captured'
})
return {
ok: true,
...EXPERIMENTAL_META,
backend: 'cua',
dispatch: 'background',
control_session_id: authority.controlSessionId,
target_app: check.target.app,
target_title: check.target.title,
target_pid: check.target.pid,
target_window_id: check.target.windowId,
action,
verification: 'snapshot_captured',
phase: 'structured_primary',
result,
verification_snapshot: after
}
}
export const computerStatusHandler: ToolHandler = async (_args, ctx) => {
const grant = getActiveComputerGrant()
const runtime = getComputerUseRuntimeSummary()
const grant = getActiveComputerGrant(ctx.controlSession)
const runtime = getComputerUseRuntimeSummary(ctx.controlSession)
const inputBackendReady = runtime.consented === true && process.platform === 'win32'
const fullAccess = runtime.full_access === true
const settings = readDesktopUseSettingsSync()
const cua = settings.computer_control_engine === 'cua' ? await CuaDriverAdapter.status() : null
const lifecycle = computerControlLifecycleStatus()
return {
ok: true,
...EXPERIMENTAL_META,
platform: process.platform,
displays: await getDisplays(),
runtime,
computer_control_engine: {
...lifecycle,
selected: settings.computer_control_engine,
effective: lifecycle.active_backend === 'cua'
? 'cua'
: lifecycle.active_backend === 'legacy_compat'
? 'legacy'
: settings.computer_control_engine === 'cua' && cua?.ready ? 'cua' : 'legacy',
cursor_enabled: settings.cua_cursor_enabled,
foreground_escalation_enabled: false,
cua,
message: settings.computer_control_engine === 'cua' && !cua?.ready
? cua?.available
? `CUA Driver is installed, but not ready; new sessions use Windows Input compatibility mode. ${cua.reason ?? ''}`.trim()
: `CUA Driver is unavailable before control starts; new sessions use Windows Input compatibility mode. ${cua?.reason ?? ''}`.trim()
: null
},
permissions: {
screenshot: fullAccess ? 'full_access' : grant ? 'granted' : 'grant_required',
input: fullAccess || grant?.mode === 'assist' || grant?.mode === 'control'
@@ -248,9 +425,9 @@ export const computerStatusHandler: ToolHandler = async (_args, ctx) => {
? fullAccess ? 'full_access' : 'granted_until_expiry'
: 'grant_active_but_input_backend_unavailable'
: 'not_granted',
accessibility: 'not_implemented'
accessibility: cua?.ready ? 'available' : 'unavailable'
},
grant: getComputerGrantSummary(),
grant: getComputerGrantSummary(ctx.controlSession),
overlay: {
visible: ctx.interactive,
state: ctx.interactive ? 'cli_grant_prompt_available' : 'not_available',
@@ -267,13 +444,25 @@ export const computerStatusHandler: ToolHandler = async (_args, ctx) => {
}
export const computerScreenshotHandler: ToolHandler = async (args, ctx) => {
if (!hasComputerObserveGrant()) {
if (!hasComputerObserveGrant(ctx.controlSession)) {
return failure(
'grant_required',
'Screenshot observe mode requires an active observe/assist/control grant. Call desktop_computer_grant_request first.'
'Screenshot observe mode requires an active observe/assist/control grant. Call desktop_computer_grant_request first.',
{},
ctx.controlSession
)
}
const settings = readDesktopUseSettingsSync()
const selection = ctx.controlSession
? await selectComputerControlBackend(ctx.controlSession.controlSessionId, settings.computer_control_engine, settings.cua_cursor_enabled)
: null
if (selection?.backend === 'cua') {
// Window-scoped observation belongs to CUA. A caller may still request the
// existing read-only display capture; it is not an input-backend fallback.
if (args.pid !== undefined || args.window_id !== undefined) return cuaSnapshot(args, ctx)
}
if (args.region !== undefined && args.region !== null) {
return failure(
'not_implemented',
@@ -299,6 +488,7 @@ export const computerScreenshotHandler: ToolHandler = async (args, ctx) => {
ok: true,
...EXPERIMENTAL_META,
mode: 'observe',
backend: 'system_capture',
format: result.format,
bytes_base64: result.bytes_base64,
saved_path: result.saved_path,
@@ -318,7 +508,7 @@ export const computerScreenshotHandler: ToolHandler = async (args, ctx) => {
applied: false,
reason: 'Sensitive-window redaction is planned but not implemented yet.'
},
grant: getComputerGrantSummary()
grant: getComputerGrantSummary(ctx.controlSession)
}
}
@@ -327,17 +517,39 @@ export const computerActionHandler: ToolHandler = async (args, ctx) => {
if (!action) {
return failure('invalid_request', 'desktop_computer_action requires an action name.')
}
const settings = readDesktopUseSettingsSync()
const selection = ctx.controlSession
? await selectComputerControlBackend(ctx.controlSession.controlSessionId, settings.computer_control_engine, settings.cua_cursor_enabled)
: null
if (selection?.backend === 'cua') {
if (!['click_element', 'set_value', 'press_key', 'scroll_element'].includes(action)) {
return failure(
'cua_structured_action_required',
'CUA is selected; legacy coordinate and foreground-routed input is disabled. Take a structured snapshot and use an element action.',
{ action },
ctx.controlSession
)
}
if (!hasComputerInputGrant(ctx.controlSession)) {
return failure('grant_required', 'Structured host input requires an active assist/control grant.', { action }, ctx.controlSession)
}
return cuaAction(args, ctx)
}
if (['click_element', 'set_value', 'press_key', 'scroll_element'].includes(action)) {
return failure('cua_unavailable', 'Structured CUA actions are unavailable in the compatibility backend.', { action }, ctx.controlSession)
}
const displays = await getDisplays()
const validation = validateComputerAction(args, displays)
if (!validation.ok) {
return failure(validation.code, validation.message, { action })
}
if (!hasComputerInputGrant()) {
if (!hasComputerInputGrant(ctx.controlSession)) {
return failure(
'grant_required',
'Host input is disabled. Request and locally approve an assist/control grant first.',
{ action }
{ action },
ctx.controlSession
)
}
@@ -352,12 +564,21 @@ export const computerActionHandler: ToolHandler = async (args, ctx) => {
performed_at: new Date().toISOString(),
duration_ms: Date.now() - started,
input_backend: inputResult.backend,
backend: 'legacy_compat',
dispatch: 'foreground_compatibility',
phase: selection?.reason === 'cua_unavailable_before_session' ? 'pre_session_safe_fallback' : 'explicit_compatibility',
control_session_id: ctx.controlSession?.controlSessionId,
verification: normalized.returnScreenshot ? 'snapshot_captured' : 'not_requested',
platform: inputResult.platform,
grant: getComputerGrantSummary()
grant: getComputerGrantSummary(ctx.controlSession)
}
if (normalized.returnScreenshot) {
response.after_screenshot = await computerScreenshotHandler({ display: args.display ?? 'primary' }, ctx)
}
recordCuaActionEvent({
action: normalized.action,
verification: normalized.returnScreenshot ? 'snapshot_captured' : 'not_requested'
})
return response
}
@@ -366,7 +587,7 @@ export const computerGrantRequestHandler: ToolHandler = async (args, ctx) => {
if (!mode) {
return failure('invalid_request', 'mode must be one of observe, assist, or control.')
}
const runtime = getComputerUseRuntimeSummary()
const runtime = getComputerUseRuntimeSummary(ctx.controlSession)
if (runtime.full_access === true) {
return {
...requestComputerGrant({
@@ -374,7 +595,7 @@ export const computerGrantRequestHandler: ToolHandler = async (args, ctx) => {
scope: args.scope,
duration_seconds: args.duration_seconds,
reason: args.reason
}),
}, ctx.controlSession),
...EXPERIMENTAL_META
}
}
@@ -382,7 +603,9 @@ export const computerGrantRequestHandler: ToolHandler = async (args, ctx) => {
if (runtime.consented !== true) {
return failure(
'computer_use_consent_required',
'Assist/control grants require local desktop-tool consent for this relay URL before task-scoped input grants can be created.'
'Assist/control grants require local desktop-tool consent for this relay URL before task-scoped input grants can be created.',
{},
ctx.controlSession
)
}
const approval = await approveComputerGrant({
@@ -406,17 +629,18 @@ export const computerGrantRequestHandler: ToolHandler = async (args, ctx) => {
scope: args.scope,
duration_seconds: args.duration_seconds,
reason: args.reason
}),
}, ctx.controlSession),
...EXPERIMENTAL_META
}
}
export const computerCancelHandler: ToolHandler = async (args) => {
export const computerCancelHandler: ToolHandler = async (args, ctx) => {
const reason = typeof args.reason === 'string' && args.reason.trim()
? args.reason.trim()
: 'cancelled by desktop_computer_cancel'
if (ctx.controlSession) await closeCuaControlSession(ctx.controlSession.controlSessionId, reason)
return {
...cancelComputerGrant(reason),
...cancelComputerGrant(reason, ctx.controlSession),
...EXPERIMENTAL_META
}
}
+186 -9
View File
@@ -30,18 +30,31 @@ import {
auditDetails,
categorizeTool,
previewArgs,
persistAuditScreenshot,
resultExitCode,
summarizeResult
} from '../lib/auditLog.js'
import { VERSION } from '../version.js'
import { desktopDeviceId } from '../deviceIdentity.js'
import { getComputerGrantSummary, getComputerUseRuntimeSummary } from './computerGrants.js'
import { readDesktopUseSettingsSync } from '../lib/desktopUseSettings.js'
import {
getComputerGrantSummary,
getComputerUseRuntimeSummary,
initializeComputerControlSession,
revokeComputerControlSession
} from './computerGrants.js'
import {
ComputerControlSecurityState,
type ComputerControlAuthority
} from './computerControlSecurity.js'
import { closeAllCuaControlSessions, closeCuaControlSession } from './cuaDriver.js'
/** The payload shape server → client for a single tool invocation. */
export interface ToolCallPayload {
request_id: string
tool: string
args: Record<string, unknown>
control_session?: unknown
}
/** Either a success with a free-form result, or a failure with an error
@@ -67,6 +80,10 @@ export interface ToolContext {
cwd: string
abortSignal: AbortSignal
interactive: boolean
/** Server-attested control identity. Optional only for legacy handlers. */
controlSession?: ComputerControlAuthority
/** Hermes-owned snapshot/token binding state for this router lifecycle. */
controlSecurity?: ComputerControlSecurityState
}
/** A tool handler. Throws → router responds with `{ok:false, error}`. */
@@ -122,6 +139,98 @@ function isToolCallPayload(x: unknown): x is ToolCallPayload {
)
}
interface ControlSessionEndPayload {
version: 1
id: string
target_device_id: string
reason?: string
}
function parseControlSessionEnd(value: unknown): ControlSessionEndPayload | null {
if (!value || typeof value !== 'object' || Array.isArray(value)) return null
const raw = value as Record<string, unknown>
const field = (key: string): string | null => {
const value = raw[key]
if (typeof value !== 'string') return null
const normalized = value.trim()
return normalized && normalized.length <= 256 && !/[\u0000-\u001f\u007f]/u.test(normalized) ? normalized : null
}
if (raw.version !== 1) return null
const id = field('id')
const targetDeviceId = field('target_device_id')
if (!id || !targetDeviceId) return null
const reason = raw.reason === undefined ? undefined : field('reason') ?? undefined
return { version: 1, id, target_device_id: targetDeviceId, ...(reason ? { reason } : {}) }
}
function parseControlAuthority(
value: unknown,
requestId: string,
hostUrl: string | undefined
): ComputerControlAuthority | null {
if (!value || typeof value !== 'object' || Array.isArray(value)) return null
const raw = value as Record<string, unknown>
const required = (field: string): string | null => {
const current = raw[field]
if (typeof current !== 'string') return null
const normalized = current.trim()
if (!normalized || normalized.length > 256 || /[\u0000-\u001f\u007f]/u.test(normalized)) return null
return normalized
}
if (raw.version !== 1) return null
const id = required('id')
const boundRequestId = required('request_id')
const requesterDeviceId = required('requester_device_id')
const targetDeviceId = required('target_device_id')
const runId = required('run_id')
if (
!id || !boundRequestId || boundRequestId !== requestId || !requesterDeviceId ||
!targetDeviceId || targetDeviceId !== desktopDeviceId() || !runId
) return null
const optional = (field: string): string | undefined => {
const current = raw[field]
if (typeof current !== 'string') return undefined
const normalized = current.trim()
return normalized && normalized.length <= 256 && !/[\u0000-\u001f\u007f]/u.test(normalized)
? normalized
: undefined
}
return {
controlSessionId: id,
// The relay-owned control session is the authenticated execution-session
// identity currently available on the wire. Keep the adapter contract
// explicit while avoiding any identity supplied through tool args.
relaySessionId: id,
requesterDeviceId,
targetDeviceId,
requestId,
chatSessionId: optional('chat_session_id'),
runId,
...(hostUrl ? { hostUrl } : {})
}
}
function computerAuditMetadata(result: unknown): Record<string, unknown> {
if (!result || typeof result !== 'object' || Array.isArray(result)) return {}
const source = result as Record<string, unknown>
const metadata: Record<string, unknown> = {}
// Window titles frequently contain document names, account names, or page
// content. Keep the app and numeric target for drilldown, but never persist
// the title in the local activity log.
for (const key of ['backend', 'dispatch', 'control_session_id', 'target_app', 'action', 'verification', 'phase']) {
if (typeof source[key] === 'string') metadata[key] = String(source[key]).slice(0, 256)
}
for (const key of ['target_pid', 'target_window_id']) {
if (typeof source[key] === 'number' && Number.isSafeInteger(source[key])) metadata[key] = source[key]
}
return metadata
}
export function toolResultSucceeded(tool: string, result: unknown): boolean {
if (!tool.startsWith('desktop_computer_')) return true
return !result || typeof result !== 'object' || Array.isArray(result) || (result as Record<string, unknown>).ok !== false
}
/** Default interactive detection — true iff stdin is a TTY AND we're not
* flagged as the daemon subcommand. The daemon command sets
* HERMES_RELAY_DAEMON=1 in its own process.env before constructing the
@@ -151,6 +260,9 @@ export class DesktopToolRouter {
* timeout, abort). Surfaced in the heartbeat so an agent or the dashboard
* can ask "is this client healthy?" without parsing transcript history. */
private lastError: { message: string; tool: string; ts: number } | null = null
private readonly controlAuthority: ComputerControlAuthority
private readonly controlSecurity: ComputerControlSecurityState
private readonly activeControlAuthorities = new Map<string, ComputerControlAuthority>()
constructor(opts: DesktopToolRouterOpts) {
this.handlers = opts.handlers
@@ -161,6 +273,13 @@ export class DesktopToolRouter {
// capture it once at construct time so a short shell session that
// happens to get its stdin redirected mid-flight doesn't flip mode.
this.interactive = opts.interactive ?? detectInteractive()
this.controlAuthority = {
controlSessionId: `router-${desktopDeviceId()}-${this.startedAtMs}`,
...(this.hostUrl ? { hostUrl: this.hostUrl } : {})
}
this.controlSecurity = new ComputerControlSecurityState(this.controlAuthority)
initializeComputerControlSession(this.controlAuthority)
this.activeControlAuthorities.set(this.controlAuthority.controlSessionId, this.controlAuthority)
}
/** Install the `onChannel('desktop')` listener and start heartbeats.
@@ -188,6 +307,17 @@ export class DesktopToolRouter {
void this.dispatch(payload)
return
}
if (type === 'desktop.control_session_end') {
const ended = parseControlSessionEnd(payload)
if (!ended || ended.target_device_id !== desktopDeviceId()) return
const authority = this.activeControlAuthorities.get(ended.id)
if (!authority || authority.targetDeviceId !== ended.target_device_id) return
this.activeControlAuthorities.delete(ended.id)
this.controlSecurity.revokeAuthority(ended.id)
revokeComputerControlSession(authority, ended.reason ?? 'relay control session ended')
void closeCuaControlSession(ended.id, ended.reason ?? 'relay control session ended')
return
}
// Unknown desktop.* types — ignore; server may extend later.
})
@@ -202,6 +332,7 @@ export class DesktopToolRouter {
/** Remove the channel listener and stop heartbeats. Idempotent. */
detach(): void {
if (!this.attached) {
this.revokeControlAuthority()
return
}
this.attached = false
@@ -215,6 +346,16 @@ export class DesktopToolRouter {
/* ignore */
}
this.relay = null
this.revokeControlAuthority()
}
private revokeControlAuthority(): void {
this.controlSecurity.revoke()
for (const authority of this.activeControlAuthorities.values()) {
revokeComputerControlSession(authority, 'desktop router detached')
}
this.activeControlAuthorities.clear()
void closeAllCuaControlSessions('desktop router detached')
}
/** Broadcast the advertised-tools heartbeat. Safe to call when detached
@@ -241,8 +382,8 @@ export class DesktopToolRouter {
device_name: os.hostname()
}
if (this.advertisedTools.some(name => name.startsWith('desktop_computer_'))) {
const runtime = getComputerUseRuntimeSummary()
const grant = getComputerGrantSummary()
const runtime = getComputerUseRuntimeSummary(this.controlAuthority)
const grant = getComputerGrantSummary(this.controlAuthority)
const inputGrantActive =
grant.active === true && (grant.mode === 'assist' || grant.mode === 'control')
payload.computer_use = {
@@ -289,6 +430,22 @@ export class DesktopToolRouter {
return
}
const relayAuthority = parseControlAuthority(cmd.control_session, request_id, this.hostUrl)
if (tool.startsWith('desktop_computer_') && cmd.control_session !== undefined && !relayAuthority) {
this.sendResponse({ request_id, ok: false, error: 'invalid server control_session binding' })
return
}
const requestAuthority = relayAuthority ?? this.controlAuthority
if (relayAuthority) {
initializeComputerControlSession(relayAuthority)
this.activeControlAuthorities.set(relayAuthority.controlSessionId, relayAuthority)
}
const controlSession = this.controlSecurity.bindRequest(request_id, requestAuthority)
if (!controlSession) {
this.sendResponse({ request_id, ok: false, error: 'duplicate or invalid desktop request_id' })
return
}
const controller = new AbortController()
const timeoutMs = tool.startsWith('desktop_computer_') || tool.startsWith('desktop_adb_') || tool.startsWith('desktop_usb_')
? COMPUTER_USE_HANDLER_TIMEOUT_MS
@@ -303,7 +460,9 @@ export class DesktopToolRouter {
const ctx: ToolContext = {
cwd: process.cwd(),
abortSignal: controller.signal,
interactive: this.interactive
interactive: this.interactive,
controlSession,
controlSecurity: this.controlSecurity
}
try {
@@ -314,20 +473,34 @@ export class DesktopToolRouter {
// work was completable.
this.sendResponse({ request_id, ok: true, result })
// Local audit trail — fire-and-forget so logging never delays the reply.
void appendAudit({
const retention = readDesktopUseSettingsSync()
const canRetainScreenshot = tool.includes('screenshot') || tool === 'desktop_computer_action'
const screenshotEvidence = retention.activity_screenshot_retention_enabled && canRetainScreenshot
? persistAuditScreenshot(result, request_id, retention.activity_screenshot_retention_days)
: Promise.resolve({})
void screenshotEvidence.then(screenshotEvidence => appendAudit({
ts: Date.now(),
kind: 'tool.completed',
tool,
category: categorizeTool(tool),
ok: true,
ok: toolResultSucceeded(tool, result),
request_id,
host_url: this.hostUrl,
duration_ms: Date.now() - startedAt,
exit_code: resultExitCode(result),
args_preview: previewArgs(args),
summary: summarizeResult(result),
...auditDetails(args, result)
})
...(controlSession.relaySessionId ? { relay_session_id: controlSession.relaySessionId } : {}),
...(controlSession.requesterDeviceId ? { requester_device_id: controlSession.requesterDeviceId } : {}),
...(controlSession.runId ? { run_id: controlSession.runId } : {}),
...(controlSession.targetDeviceId ? { target_device_id: controlSession.targetDeviceId } : {}),
...computerAuditMetadata(result),
...screenshotEvidence,
...(!toolResultSucceeded(tool, result) && result && typeof result === 'object' && !Array.isArray(result) && typeof (result as Record<string, unknown>).code === 'string'
? { error: String((result as Record<string, unknown>).code).slice(0, 128) }
: {}),
...auditDetails(args, result, { redactComputerContent: tool.startsWith('desktop_computer_') })
}))
} catch (e) {
clearTimeout(timeoutTimer)
// Distinguish aborts (timeout or transport teardown) from genuine
@@ -355,7 +528,11 @@ export class DesktopToolRouter {
host_url: this.hostUrl,
duration_ms: Date.now() - startedAt,
args_preview: previewArgs(args),
...auditDetails(args),
...(controlSession.relaySessionId ? { relay_session_id: controlSession.relaySessionId } : {}),
...(controlSession.requesterDeviceId ? { requester_device_id: controlSession.requesterDeviceId } : {}),
...(controlSession.runId ? { run_id: controlSession.runId } : {}),
...(controlSession.targetDeviceId ? { target_device_id: controlSession.targetDeviceId } : {}),
...auditDetails(args, undefined, { redactComputerContent: tool.startsWith('desktop_computer_') }),
error: message
})
}
+1 -1
View File
@@ -1,2 +1,2 @@
// Regenerated from package.json by gen:version script. Do not edit by hand.
export const VERSION = "0.4.0-alpha.8" as const
export const VERSION = "0.4.0-beta.3" as const
+6 -1
View File
@@ -13,6 +13,7 @@ export interface WindowsInstallerLaunchOptions {
cliPath?: string
trayPath?: string
delayMs?: number
callerPid?: number
}
/**
@@ -30,9 +31,12 @@ export function windowsInstallerLaunchPlan(
installDir = '',
cliPath = '',
trayPath = '',
delayMs = 1200
delayMs = 1200,
callerPid = process.pid
} = options
const script = [
'$callerPid = [int]$env:HERMES_RELAY_SETUP_CALLER_PID',
'if ($callerPid -gt 0) { Wait-Process -Id $callerPid -ErrorAction SilentlyContinue }',
`Start-Sleep -Milliseconds ${Math.max(0, Math.floor(delayMs))}`,
'$installer = $env:HERMES_RELAY_SETUP_PATH',
"$installerArgs = if ($env:HERMES_RELAY_SETUP_SILENT -eq '1') { @('/S') } else { @() }",
@@ -64,6 +68,7 @@ export function windowsInstallerLaunchPlan(
env: {
...process.env,
HERMES_RELAY_SETUP_PATH: installerPath,
HERMES_RELAY_SETUP_CALLER_PID: String(callerPid),
HERMES_RELAY_SETUP_SILENT: silent ? '1' : '0',
HERMES_RELAY_SETUP_RESTART_DAEMON: restartDaemon ? '1' : '0',
HERMES_RELAY_SETUP_INSTALL_DIR: installDir,
+47 -1
View File
@@ -1,7 +1,11 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import { mkdtemp, readFile, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { auditDetails, categorizeTool, resultExitCode, summarizeResult } from '../src/lib/auditLog.js'
import { auditDetails, categorizeTool, persistAuditScreenshot, resultExitCode, summarizeResult } from '../src/lib/auditLog.js'
import { toolResultSucceeded } from '../src/tools/router.js'
test('audit events classify the activity surfaces used by the tray', () => {
assert.equal(categorizeTool('desktop_powershell'), 'command')
@@ -31,3 +35,45 @@ test('audit events preserve process exit outcome separately from dispatch succes
assert.equal(summarizeResult(result), 'exit 17')
assert.equal(resultExitCode({ path: 'C:\\temp\\file.txt' }), undefined)
})
test('computer-use audit redacts screenshots, UI trees, labels, and entered values', () => {
const details = auditDetails(
{ pid: 42, window_id: 7, value: 'top-secret-password' },
{
backend: 'cua_driver',
target: { pid: 42, windowId: 7, title: 'Password Manager' },
tree_markdown: 'Password: hunter2',
screenshot_base64: 'sensitive-pixels',
elements: [{ label: 'API token sk-live-secret' }]
},
{ redactComputerContent: true }
)
const serialized = JSON.stringify(details)
assert.doesNotMatch(serialized, /hunter2|sensitive-pixels|sk-live-secret|top-secret-password|Password Manager/)
assert.match(details.result_detail ?? '', /"redacted": true/)
assert.match(details.result_detail ?? '', /"pid": 42/)
})
test('semantic computer-control rejection is audited as failed without changing transport semantics', () => {
assert.equal(toolResultSucceeded('desktop_computer_action', { ok: false, code: 'sensitive_target_blocked' }), false)
assert.equal(toolResultSucceeded('desktop_computer_action', { ok: true }), true)
assert.equal(toolResultSucceeded('desktop_read_file', { ok: false }), true)
})
test('screenshot evidence is retained as a private opaque PNG instead of JSON content', async () => {
const directory = await mkdtemp(join(tmpdir(), 'hermes-activity-evidence-'))
const previous = process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR
process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR = directory
try {
const png = Buffer.from('89504e470d0a1a0a00000000', 'hex').toString('base64')
const evidence = await persistAuditScreenshot({ screenshot_base64: png, screenshot_width: 640, screenshot_height: 480 }, 'request-1', 7)
assert.match(evidence.screenshot_evidence_id ?? '', /^[a-f0-9]{32}$/)
assert.equal(evidence.screenshot_width, 640)
assert.equal((await readFile(join(directory, `${evidence.screenshot_evidence_id}.png`))).subarray(0, 8).toString('hex'), '89504e470d0a1a0a')
assert.equal(JSON.stringify(evidence).includes(png), false)
} finally {
if (previous === undefined) delete process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR
else process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR = previous
await rm(directory, { recursive: true, force: true })
}
})
@@ -0,0 +1,190 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import {
computerGrantAllowsTarget,
configureComputerUseRuntime,
getComputerGrantSummary,
initializeComputerControlSession,
cancelAllComputerGrants,
requestComputerGrant,
revokeComputerControlSession
} from '../src/tools/computerGrants.js'
import {
ComputerControlSecurityState,
evaluateSensitiveTarget,
hasAuthenticatedControlIdentity,
type ComputerControlAuthority
} from '../src/tools/computerControlSecurity.js'
const authority = (id: string): ComputerControlAuthority => ({
controlSessionId: id,
hostUrl: 'wss://relay.example'
})
test('control request ids are accepted once per router lifecycle', () => {
const state = new ComputerControlSecurityState(authority('router-a'))
assert.equal(state.bindRequest('request-1')?.requestId, 'request-1')
assert.equal(state.bindRequest('request-1'), null)
assert.equal(state.bindRequest(' '), null)
})
test('snapshot tokens are opaque, one-use, and bound to grant and target', () => {
const auth = authority('router-snapshot')
const state = new ComputerControlSecurityState(auth)
const token = state.issueSnapshotToken({
authority: auth,
grantId: 'grant-a',
target: { pid: 42, windowId: 7, app: 'Notepad' },
snapshotGeneration: 'generation-1',
driverElementToken: 'driver-private-token'
})
assert.match(token, /^hermes-snapshot-/)
assert.equal(state.consumeSnapshotToken(token, {
authority: auth,
grantId: 'grant-b',
target: { pid: 42, windowId: 7 }
}), null)
assert.equal(state.consumeSnapshotToken(token, {
authority: auth,
grantId: 'grant-a',
target: { pid: 42, windowId: 7 }
}), null)
})
test('snapshot tokens return their backend binding only to the exact target', () => {
const auth = authority('router-target')
const state = new ComputerControlSecurityState(auth)
const token = state.issueSnapshotToken({
authority: auth,
grantId: null,
target: { pid: 9, windowId: 11 },
snapshotGeneration: 'g1',
driverElementToken: 'opaque-driver-token'
})
const binding = state.consumeSnapshotToken(token, {
authority: auth,
grantId: null,
target: { pid: 9, windowId: 11 },
snapshotGeneration: 'g1'
})
assert.equal(binding?.driverElementToken, 'opaque-driver-token')
assert.equal(state.consumeSnapshotToken(token, {
authority: auth,
grantId: null,
target: { pid: 9, windowId: 11 }
}), null)
})
test('authority revocation removes only that concurrent session artifacts', () => {
const first = authority('router-first')
const second = authority('router-second')
const state = new ComputerControlSecurityState(first)
assert.ok(state.bindRequest('request-first', first))
assert.ok(state.bindRequest('request-second', second))
const firstToken = state.issueSnapshotToken({
authority: first, grantId: null, target: { pid: 1, windowId: 1 }, snapshotGeneration: 'g1'
})
const secondToken = state.issueSnapshotToken({
authority: second, grantId: null, target: { pid: 2, windowId: 2 }, snapshotGeneration: 'g2'
})
state.revokeAuthority(first.controlSessionId)
assert.ok(state.bindRequest('request-first', first))
assert.equal(state.bindRequest('request-second', second), null)
assert.equal(state.consumeSnapshotToken(firstToken, { authority: first, grantId: null, target: { pid: 1, windowId: 1 } }), null)
assert.ok(state.consumeSnapshotToken(secondToken, { authority: second, grantId: null, target: { pid: 2, windowId: 2 } }))
})
test('sensitive target policy fails closed without identity and blocks baseline surfaces', () => {
assert.deepEqual(evaluateSensitiveTarget({}), { allowed: false, reason: 'missing_target_identity' })
assert.equal(evaluateSensitiveTarget({ app: 'Bitwarden' }).allowed, false)
assert.equal(evaluateSensitiveTarget({ title: 'Windows Security settings' }).allowed, false)
assert.deepEqual(evaluateSensitiveTarget({ app: 'Notepad', title: 'notes.txt' }), {
allowed: true,
reason: 'not_sensitive'
})
})
test('authenticated control identity requires relay-attested requester/run/target fields', () => {
assert.equal(hasAuthenticatedControlIdentity(authority('local-only')), false)
assert.equal(hasAuthenticatedControlIdentity({
controlSessionId: 'control-1',
relaySessionId: 'relay-1',
requesterDeviceId: 'requester-1',
runId: 'run-1',
targetDeviceId: 'desktop-1'
}), true)
})
test('router control sessions inherit host policy but keep grants isolated', () => {
configureComputerUseRuntime({
url: 'wss://relay.example',
computerUseConsented: true,
consentSource: 'stored',
accessMode: 'ask',
capabilities: {
commands: 'ask', files: 'ask', screen_input: 'ask', usb: 'ask', microphone: 'disabled', camera: 'disabled'
}
})
const first = authority('router-first')
const second = authority('router-second')
initializeComputerControlSession(first)
initializeComputerControlSession(second)
requestComputerGrant({
mode: 'control',
scope: { app: 'Notepad', display: 'primary', folder: 'C:\\work' },
duration_seconds: 60,
reason: 'edit notes'
}, first)
assert.equal(getComputerGrantSummary(first).active, true)
assert.equal(getComputerGrantSummary(second).active, false)
assert.equal(computerGrantAllowsTarget(first, {
pid: 1,
windowId: 2,
app: 'Windows Notepad',
display: 'primary',
folder: 'C:\\work\\notes'
}), true)
assert.equal(computerGrantAllowsTarget(first, {
pid: 1,
windowId: 2,
app: 'Browser',
display: 'primary',
folder: 'C:\\work\\notes'
}), false)
revokeComputerControlSession(first)
revokeComputerControlSession(second)
})
test('Full Access configured before router creation is preserved without sharing grants', () => {
configureComputerUseRuntime({
computerUseConsented: true,
accessMode: 'full_access'
})
const full = authority('router-full')
initializeComputerControlSession(full)
assert.equal(getComputerGrantSummary(full).mode, 'full_access')
revokeComputerControlSession(full)
configureComputerUseRuntime({ accessMode: 'ask' })
})
test('local emergency cancellation revokes every active control session', () => {
configureComputerUseRuntime({
computerUseConsented: true,
accessMode: 'ask',
capabilities: {
commands: 'ask', files: 'ask', screen_input: 'ask', usb: 'ask', microphone: 'disabled', camera: 'disabled'
}
})
const first = authority('emergency-first')
const second = authority('emergency-second')
initializeComputerControlSession(first)
initializeComputerControlSession(second)
requestComputerGrant({ mode: 'control', reason: 'first' }, first)
requestComputerGrant({ mode: 'observe', reason: 'second' }, second)
assert.equal(cancelAllComputerGrants('emergency stop'), 2)
assert.equal(getComputerGrantSummary(first).active, false)
assert.equal(getComputerGrantSummary(second).active, false)
revokeComputerControlSession(first)
revokeComputerControlSession(second)
})
+260
View File
@@ -0,0 +1,260 @@
import assert from 'node:assert/strict'
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import test from 'node:test'
import { ComputerControlSecurityState, type ComputerControlAuthority } from '../src/tools/computerControlSecurity.js'
import {
cancelComputerGrant,
configureComputerUseRuntime,
requestComputerGrant
} from '../src/tools/computerGrants.js'
import {
setCuaControlSessionFactoryForTests,
type CuaControlSession,
type CuaControlSessionIdentity
} from '../src/tools/cuaDriver.js'
import {
computerActionHandler,
computerCancelHandler,
computerScreenshotHandler
} from '../src/tools/handlers/computer.js'
import type { ToolContext } from '../src/tools/router.js'
interface FakeCall {
name: string
args?: unknown
}
class FakeCuaSession {
readonly calls: FakeCall[] = []
closed = false
snapshotNumber = 0
appName = 'Calculator'
async listWindows(pid?: number): Promise<Record<string, unknown>> {
this.calls.push({ name: 'listWindows', args: pid })
return {
windows: [{
window_id: 200,
app_name: this.appName,
title: this.appName,
executable: 'C:\\WindowsApps\\CalculatorApp.exe'
}]
}
}
async snapshot(args: unknown): Promise<Record<string, unknown>> {
this.snapshotNumber += 1
this.calls.push({ name: 'snapshot', args })
return {
snapshot_id: `s0000000${this.snapshotNumber}`,
elements: [{ element_index: 7, element_token: 'e1234abcd', role: 'button', label: 'Seven' }],
tree_markdown: '[7] button Seven'
}
}
async clickElement(args: unknown): Promise<Record<string, unknown>> {
this.calls.push({ name: 'clickElement', args })
return { clicked: true }
}
async setElementValue(args: unknown, value: string): Promise<Record<string, unknown>> {
this.calls.push({ name: 'setElementValue', args: { args, value } })
return { changed: true }
}
async pressKey(args: unknown, key: string): Promise<Record<string, unknown>> {
this.calls.push({ name: 'pressKey', args: { args, key } })
return { pressed: true }
}
async scroll(args: unknown, direction: string, amount: number): Promise<Record<string, unknown>> {
this.calls.push({ name: 'scroll', args: { args, direction, amount } })
return { scrolled: true }
}
async close(reason?: string): Promise<void> {
this.closed = true
this.calls.push({ name: 'close', args: reason })
}
}
async function fixture(): Promise<{
authority: ComputerControlAuthority
ctx: ToolContext
session: FakeCuaSession
cleanup(): Promise<void>
}> {
const dir = await mkdtemp(join(tmpdir(), 'hermes-cua-handlers-'))
const settingsPath = join(dir, 'desktop-settings.json')
await writeFile(settingsPath, JSON.stringify({
computer_use_enabled: true,
computer_control_engine: 'cua',
cua_cursor_enabled: true,
cua_foreground_escalation_enabled: false
}))
process.env.HERMES_RELAY_DESKTOP_SETTINGS_PATH = settingsPath
const authority: ComputerControlAuthority = {
controlSessionId: 'control-handler-test',
requestId: 'request-handler-test',
relaySessionId: 'relay-session',
requesterDeviceId: 'requester-device',
runId: 'run-1',
targetDeviceId: 'desktop-target'
}
const session = new FakeCuaSession()
setCuaControlSessionFactoryForTests(async (_identity: CuaControlSessionIdentity) => session as unknown as CuaControlSession)
configureComputerUseRuntime({
url: 'wss://relay.example.test',
computerUseConsented: true,
consentSource: 'stored',
accessMode: 'ask'
}, authority)
requestComputerGrant({ mode: 'control', scope: { app: 'Calculator' }, duration_seconds: 60 }, authority)
const ctx: ToolContext = {
cwd: dir,
abortSignal: new AbortController().signal,
interactive: false,
controlSession: authority,
controlSecurity: new ComputerControlSecurityState(authority)
}
return {
authority,
ctx,
session,
cleanup: async () => {
cancelComputerGrant('test cleanup', authority)
setCuaControlSessionFactoryForTests(null)
delete process.env.HERMES_RELAY_DESKTOP_SETTINGS_PATH
await rm(dir, { recursive: true, force: true })
}
}
}
test('CUA handlers issue a Hermes token, execute once, and verify with a fresh snapshot', async () => {
const item = await fixture()
try {
const observed = await computerScreenshotHandler({ pid: 100, window_id: 200 }, item.ctx) as {
ok: boolean
backend: string
elements: Array<{ snapshot_token: string; element_token?: string }>
}
assert.equal(observed.ok, true)
assert.equal(observed.backend, 'cua_driver')
assert.equal(observed.elements[0]!.element_token, undefined)
assert.match(observed.elements[0]!.snapshot_token, /^hermes-snapshot-/)
const acted = await computerActionHandler({
action: 'click_element',
pid: 100,
window_id: 200,
snapshot_token: observed.elements[0]!.snapshot_token,
snapshot_generation: 's00000001'
}, item.ctx) as { ok: boolean; backend: string; verification_snapshot: { snapshot_id: string } }
assert.equal(acted.ok, true)
assert.equal(acted.backend, 'cua')
assert.equal(acted.verification_snapshot.snapshot_id, 's00000002')
assert.deepEqual(item.session.calls.map(call => call.name), [
'listWindows', 'snapshot', 'listWindows', 'clickElement', 'snapshot'
])
const replayed = await computerActionHandler({
action: 'click_element',
pid: 100,
window_id: 200,
snapshot_token: observed.elements[0]!.snapshot_token
}, item.ctx) as { ok: boolean; code: string }
assert.equal(replayed.ok, false)
assert.equal(replayed.code, 'invalid_or_stale_snapshot')
assert.equal(item.session.calls.filter(call => call.name === 'clickElement').length, 1)
} finally {
await item.cleanup()
}
})
test('CUA handlers reject unauthenticated and mismatched targets before input', async () => {
const item = await fixture()
try {
const unauthenticatedAuthority = { controlSessionId: 'legacy-only' }
requestComputerGrant({ mode: 'observe', duration_seconds: 60 }, unauthenticatedAuthority)
const unauthenticated = await computerScreenshotHandler(
{ pid: 100, window_id: 200 },
{ ...item.ctx, controlSession: unauthenticatedAuthority }
) as { ok: boolean; code: string }
assert.equal(unauthenticated.ok, false)
assert.equal(unauthenticated.code, 'authenticated_control_session_required')
cancelComputerGrant('test cleanup', unauthenticatedAuthority)
const missing = await computerScreenshotHandler({ pid: 100, window_id: 201 }, item.ctx) as {
ok: boolean
code: string
}
assert.equal(missing.ok, false)
assert.equal(missing.code, 'target_not_found')
requestComputerGrant({ mode: 'control', scope: { app: 'Notepad' }, duration_seconds: 60 }, item.authority)
const wrongGrant = await computerScreenshotHandler({ pid: 100, window_id: 200 }, item.ctx) as {
ok: boolean
code: string
}
assert.equal(wrongGrant.ok, false)
assert.equal(wrongGrant.code, 'grant_target_mismatch')
item.session.appName = 'Windows Security'
configureComputerUseRuntime({
url: 'wss://relay.example.test',
computerUseConsented: true,
consentSource: 'stored',
accessMode: 'full_access'
}, item.authority)
const sensitive = await computerScreenshotHandler({ pid: 100, window_id: 200 }, item.ctx) as {
ok: boolean
code: string
}
assert.equal(sensitive.ok, false)
assert.equal(sensitive.code, 'sensitive_target_blocked')
assert.equal(item.session.calls.some(call => call.name === 'clickElement'), false)
} finally {
await item.cleanup()
}
})
test('desktop computer cancel closes the matching CUA session', async () => {
const item = await fixture()
try {
await computerScreenshotHandler({ pid: 100, window_id: 200 }, item.ctx)
const result = await computerCancelHandler({ reason: 'operator stop' }, item.ctx) as { ok: boolean }
assert.equal(result.ok, true)
assert.equal(item.session.closed, true)
assert.deepEqual(item.session.calls.at(-1), { name: 'close', args: 'operator stop' })
} finally {
await item.cleanup()
}
})
test('selected CUA retains system display observation but never falls back to legacy input', async () => {
const item = await fixture()
try {
if (process.platform === 'win32') {
const screenshot = await computerScreenshotHandler({ display: 'primary' }, item.ctx) as {
ok: boolean
backend: string
}
assert.equal(screenshot.ok, true)
assert.equal(screenshot.backend, 'system_capture')
}
const coordinate = await computerActionHandler({
action: 'left_click',
x: 100,
y: 200
}, item.ctx) as { ok: boolean; code: string }
assert.equal(coordinate.ok, false)
assert.equal(coordinate.code, 'cua_structured_action_required')
assert.equal(item.session.calls.some(call => call.name === 'clickElement'), false)
} finally {
await item.cleanup()
}
})
+167
View File
@@ -0,0 +1,167 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import { desktopDeviceId } from '../src/deviceIdentity.js'
import type { RelayTransport } from '../src/transport/RelayTransport.js'
import { configureComputerUseRuntime, getComputerGrantSummary, requestComputerGrant } from '../src/tools/computerGrants.js'
import {
DesktopToolRouter,
type ToolContext
} from '../src/tools/router.js'
type ChannelListener = (type: string, payload: Record<string, unknown>) => void
function harness(handler: (ctx: ToolContext) => void) {
let listener: ChannelListener | null = null
const sent: Array<{ channel: string; type: string; payload: Record<string, unknown> }> = []
const relay = {
onChannel(channel: string, next: ChannelListener | null) {
assert.equal(channel, 'desktop')
listener = next
},
sendChannel(channel: string, type: string, payload: Record<string, unknown>) {
sent.push({ channel, type, payload })
}
} as unknown as RelayTransport
const router = new DesktopToolRouter({
consentGranted: true,
interactive: false,
hostUrl: 'wss://relay.example',
handlers: {
desktop_computer_snapshot: async (_args, ctx) => {
handler(ctx)
return { ok: true }
}
}
})
router.attach(relay)
return {
router,
sent,
emit(type: string, payload: Record<string, unknown>) {
assert.ok(listener)
listener(type, payload)
}
}
}
function nextTurn(): Promise<void> {
return new Promise(resolve => setImmediate(resolve))
}
test('router passes a strictly validated server control session to handlers', async () => {
let received: ToolContext | undefined
const testHarness = harness(ctx => { received = ctx })
const requestId = 'request-1'
testHarness.emit('desktop.command', {
request_id: requestId,
tool: 'desktop_computer_snapshot',
args: {},
control_session: {
version: 1,
id: 'control-1',
request_id: requestId,
requester_device_id: 'paired-agent-1',
target_device_id: desktopDeviceId(),
chat_session_id: 'chat-1',
run_id: 'run-1'
}
})
await nextTurn()
assert.equal(received?.controlSession?.controlSessionId, 'control-1')
assert.equal(received?.controlSession?.relaySessionId, 'control-1')
assert.equal(received?.controlSession?.requestId, requestId)
assert.equal(received?.controlSession?.requesterDeviceId, 'paired-agent-1')
assert.equal(received?.controlSession?.targetDeviceId, desktopDeviceId())
assert.equal(received?.controlSession?.runId, 'run-1')
assert.equal(received?.controlSession?.chatSessionId, 'chat-1')
assert.ok(testHarness.sent.some(item =>
item.type === 'desktop.response' && item.payload.request_id === requestId && item.payload.ok === true
))
testHarness.router.detach()
})
test('router rejects a supplied control session whose request binding is invalid', async () => {
let invoked = false
const testHarness = harness(() => { invoked = true })
testHarness.emit('desktop.command', {
request_id: 'request-outer',
tool: 'desktop_computer_snapshot',
args: {},
control_session: {
version: 1,
id: 'control-1',
request_id: 'request-other',
requester_device_id: 'paired-agent-1',
target_device_id: desktopDeviceId(),
run_id: 'run-1'
}
})
await nextTurn()
assert.equal(invoked, false)
const response = testHarness.sent.find(item =>
item.type === 'desktop.response' && item.payload.request_id === 'request-outer'
)
assert.equal(response?.payload.ok, false)
assert.equal(response?.payload.error, 'invalid server control_session binding')
testHarness.router.detach()
})
test('router preserves legacy computer commands when the server omits identity', async () => {
let received: ToolContext | undefined
const testHarness = harness(ctx => { received = ctx })
testHarness.emit('desktop.command', {
request_id: 'legacy-request',
tool: 'desktop_computer_snapshot',
args: {}
})
await nextTurn()
assert.match(received?.controlSession?.controlSessionId ?? '', /^router-/)
assert.equal(received?.controlSession?.relaySessionId, undefined)
testHarness.router.detach()
})
test('relay control-session end revokes only the exact local target authority', async () => {
let received: ToolContext | undefined
const testHarness = harness(ctx => {
received = ctx
configureComputerUseRuntime({ computerUseConsented: true, accessMode: 'ask' }, ctx.controlSession)
requestComputerGrant({ mode: 'control', duration_seconds: 60 }, ctx.controlSession)
})
const requestId = 'request-ending'
testHarness.emit('desktop.command', {
request_id: requestId,
tool: 'desktop_computer_snapshot',
args: {},
control_session: {
version: 1,
id: 'control-ending',
request_id: requestId,
requester_device_id: 'paired-agent-1',
target_device_id: desktopDeviceId(),
run_id: 'run-ending'
}
})
await nextTurn()
assert.equal(getComputerGrantSummary(received?.controlSession).active, true)
testHarness.emit('desktop.control_session_end', {
version: 1,
id: 'control-ending',
target_device_id: 'another-desktop',
reason: 'wrong target'
})
assert.equal(getComputerGrantSummary(received?.controlSession).active, true)
testHarness.emit('desktop.control_session_end', {
version: 1,
id: 'control-ending',
target_device_id: desktopDeviceId(),
reason: 'run ended'
})
assert.equal(getComputerGrantSummary(received?.controlSession).active, false)
testHarness.router.detach()
})
+201
View File
@@ -0,0 +1,201 @@
import assert from 'node:assert/strict'
import { mkdtemp, mkdir, rm, symlink, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import test from 'node:test'
import {
CuaDriverAdapter,
CuaRuntimeError,
closeAllCuaControlSessions,
getCuaControlSession,
selectComputerControlBackend,
setCuaControlSessionFactoryForTests,
type CuaControlSession,
type CuaProcessResult,
type CuaProcessRunner
} from '../src/tools/cuaDriver.js'
const REQUIRED_TOOL_LINES = [
'health_report', 'start_session', 'end_session', 'list_windows', 'get_window_state',
'click', 'set_value', 'press_key', 'scroll'
].map(name => `${name}: test tool`).join('\n')
class FakeRunner implements CuaProcessRunner {
readonly calls: Array<{ executable: string; args: readonly string[]; stdin?: string }> = []
constructor(
private readonly binaryPath: string,
private readonly health = 'ok',
private readonly version = '0.19.3',
private readonly permissionMode = 'standard'
) {}
async run(executable: string, args: readonly string[], options = {}): Promise<CuaProcessResult> {
this.calls.push({ executable, args, stdin: options.stdin })
const command = args.join(' ')
if (command === '--version') return ok(`cua-driver ${this.version}`)
if (command === 'manifest --pretty') {
return ok(JSON.stringify({ schema_version: '1', binary_version: this.version, binary_path: this.binaryPath }))
}
if (command === 'list-tools') return ok(REQUIRED_TOOL_LINES)
if (command === 'status') return ok(`Cua Driver daemon is running\n permission mode: ${this.permissionMode} (test)`)
if (command === 'call health_report') {
return ok(JSON.stringify({ schema_version: '1', driver_version: this.version, overall: this.health }))
}
return ok(JSON.stringify({ ok: true, tool: args[1] }))
}
}
function ok(stdout: string): CuaProcessResult {
return { stdout, stderr: '', exitCode: 0 }
}
async function fakeInstall(): Promise<{ home: string; binary: string; cleanup(): Promise<void> }> {
const home = await mkdtemp(join(tmpdir(), 'hermes-cua-'))
const releases = join(home, '.cua-driver', 'packages', 'releases')
const release = join(releases, '0.19.3-x86_64-pc-windows-msvc')
const current = join(home, '.cua-driver', 'packages', 'current')
await mkdir(release, { recursive: true })
const binary = join(release, 'cua-driver.exe')
await writeFile(binary, '')
await symlink(release, current, 'junction')
return { home, binary, cleanup: () => rm(home, { recursive: true, force: true }) }
}
test('discovers only the canonical package/current executable and negotiates readiness', async () => {
const install = await fakeInstall()
try {
const runner = new FakeRunner(install.binary)
const adapter = await CuaDriverAdapter.connect({ platform: 'win32', homeDir: install.home, runner })
assert.equal(adapter.binaryPath, install.binary)
assert.equal(adapter.binaryVersion, '0.19.3')
assert.equal(adapter.permissionMode, 'standard')
assert.equal(runner.calls[0]?.executable, install.binary)
assert.deepEqual(runner.calls.map(call => call.args.join(' ')).slice(0, 4), [
'--version', 'manifest --pretty', 'list-tools', 'status'
])
assert.equal(runner.calls.some(call => call.args.join(' ') === 'call health_report'), false)
} finally {
await install.cleanup()
}
})
test('keeps runtime ready when global health is degraded but still rejects unrestricted permission mode', async () => {
const install = await fakeInstall()
try {
const adapter = await CuaDriverAdapter.connect({
platform: 'win32', homeDir: install.home, runner: new FakeRunner(install.binary, 'degraded')
})
assert.equal(adapter.binaryVersion, '0.19.3')
await assert.rejects(
CuaDriverAdapter.connect({ platform: 'win32', homeDir: install.home, runner: new FakeRunner(install.binary, 'ok', '0.19.3', 'unrestricted') }),
(error: unknown) => error instanceof CuaRuntimeError && error.code === 'incompatible'
)
} finally {
await install.cleanup()
}
})
test('explicit health recheck reports degradation without changing runtime readiness', async () => {
const install = await fakeInstall()
try {
const runner = new FakeRunner(install.binary, 'degraded')
const health = await CuaDriverAdapter.healthStatus({ platform: 'win32', homeDir: install.home, runner })
assert.equal(health.state, 'degraded')
assert.equal(health.overall, 'degraded')
assert.equal(health.temporaryWindowsCompatibility, true)
assert.equal(runner.calls.filter(call => call.args.join(' ') === 'call health_report').length, 1)
const runtime = await CuaDriverAdapter.status({ platform: 'win32', homeDir: install.home, runner })
assert.equal(runtime.ready, true)
assert.equal(runtime.health, 'not_checked')
} finally {
await install.cleanup()
}
})
test('uses a locally derived session and exposes only typed background actions', async () => {
const install = await fakeInstall()
try {
const runner = new FakeRunner(install.binary)
const adapter = await CuaDriverAdapter.connect({ platform: 'win32', homeDir: install.home, runner })
const session = await adapter.openSession({ controlSessionId: 'control-1', targetDeviceId: 'desktop-1', runId: 'run-1' })
await session.snapshot({ pid: 123, windowId: 456, includeScreenshot: false })
await session.clickElement({ pid: 123, windowId: 456, elementToken: 'e123abc' })
await session.pressKey({ pid: 123, windowId: 456 }, 'escape')
await session.close()
const invocations = runner.calls.filter(call => call.args[0] === 'call' && call.args[1] !== 'health_report')
assert.deepEqual(invocations.map(call => call.args[1]), [
'start_session', 'get_window_state', 'click', 'press_key', 'end_session'
])
const start = JSON.parse(invocations[0]!.stdin!) as { session: string; capture_scope: string }
assert.match(start.session, /^hermes-[0-9a-f]{32}$/)
assert.equal(start.capture_scope, 'window')
const click = JSON.parse(invocations[2]!.stdin!) as { session: string; delivery_mode: string; scope: string }
assert.equal(click.session, start.session)
assert.equal(click.delivery_mode, 'background')
assert.equal(click.scope, 'window')
await assert.rejects(session.listWindows(), /closed/)
} finally {
await install.cleanup()
}
})
test('rejects invalid tokens and keys before invoking the driver', async () => {
const install = await fakeInstall()
try {
const runner = new FakeRunner(install.binary)
const adapter = await CuaDriverAdapter.connect({ platform: 'win32', homeDir: install.home, runner })
const session = await adapter.openSession({ controlSessionId: 'control-2', targetDeviceId: 'desktop-1' })
const before = runner.calls.length
await assert.rejects(session.clickElement({ pid: 1, windowId: 2, elementToken: '../bad' }), /elementToken/)
await assert.rejects(session.pressKey({ pid: 1, windowId: 2 }, 'win+r'), /allowlisted/)
assert.equal(runner.calls.length, before)
await session.close()
} finally {
await install.cleanup()
}
})
test('shared control sessions reject identity changes under the same authority id', async () => {
const fake = { close: async () => undefined } as unknown as CuaControlSession
setCuaControlSessionFactoryForTests(async () => fake)
try {
await getCuaControlSession({
controlSessionId: 'control-bound',
targetDeviceId: 'desktop-1',
relaySessionId: 'relay-1',
requesterDeviceId: 'agent-1',
runId: 'run-1'
})
await assert.rejects(getCuaControlSession({
controlSessionId: 'control-bound',
targetDeviceId: 'desktop-1',
relaySessionId: 'relay-1',
requesterDeviceId: 'agent-2',
runId: 'run-1'
}), /identity changed/)
} finally {
await closeAllCuaControlSessions('test cleanup')
setCuaControlSessionFactoryForTests(null)
}
})
test('control backend selection is immutable for the active session', async () => {
const fake = { close: async () => undefined } as unknown as CuaControlSession
setCuaControlSessionFactoryForTests(async () => fake)
try {
const primary = await selectComputerControlBackend('backend-cua', 'cua', true)
const attemptedDowngrade = await selectComputerControlBackend('backend-cua', 'legacy', false)
assert.equal(primary.backend, 'cua')
assert.deepEqual(attemptedDowngrade, primary)
const compatibility = await selectComputerControlBackend('backend-legacy', 'legacy', false)
const attemptedUpgrade = await selectComputerControlBackend('backend-legacy', 'cua', true)
assert.equal(compatibility.backend, 'legacy_compat')
assert.deepEqual(attemptedUpgrade, compatibility)
} finally {
await closeAllCuaControlSessions('test cleanup')
setCuaControlSessionFactoryForTests(null)
}
})
+149
View File
@@ -0,0 +1,149 @@
import assert from 'node:assert/strict'
import { mkdir, mkdtemp, rm, symlink, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import test from 'node:test'
import {
CuaDriverAdapter,
CuaRuntimeError,
type CuaProcessResult,
type CuaProcessRunner
} from '../src/tools/cuaDriver.js'
const tools = [
'health_report', 'start_session', 'end_session', 'list_windows', 'get_window_state',
'click', 'set_value', 'press_key', 'scroll'
]
class StatefulFakeCua implements CuaProcessRunner {
readonly calls: Array<{ args: readonly string[]; payload: Record<string, unknown> | null; signal?: AbortSignal }> = []
rejectClicksAsStale = false
constructor(private readonly binary: string) {}
async run(
_executable: string,
args: readonly string[],
options: { stdin?: string; signal?: AbortSignal } = {}
): Promise<CuaProcessResult> {
const payload = options.stdin ? JSON.parse(options.stdin) as Record<string, unknown> : null
this.calls.push({ args: [...args], payload, signal: options.signal })
if (options.signal?.aborted) throw new CuaRuntimeError('fake action cancelled', 'transport')
const command = args.join(' ')
if (command === '--version') return ok('cua-driver 0.19.3')
if (command === 'manifest --pretty') {
return ok(JSON.stringify({ schema_version: '1', binary_version: '0.19.3', binary_path: this.binary }))
}
if (command === 'list-tools') return ok(tools.map(tool => `${tool}: fake`).join('\n'))
if (command === 'status') return ok('permission mode: bounded')
if (command === 'call health_report') {
return ok(JSON.stringify({ schema_version: '1', driver_version: '0.19.3', overall: 'ok' }))
}
if (command === 'call get_window_state') {
return ok(JSON.stringify({
snapshot_id: 's1234abcd',
elements: [{ element_index: 7, element_token: 'e1234abcd', role: 'button', label: 'Seven' }]
}))
}
if (command === 'call click' && this.rejectClicksAsStale) {
return ok(JSON.stringify({ isError: true, code: 'stale_element_token' }))
}
return ok(JSON.stringify({ ok: true }))
}
}
function ok(stdout: string): CuaProcessResult {
return { stdout, stderr: '', exitCode: 0 }
}
async function harness(): Promise<{
adapter: CuaDriverAdapter
runner: StatefulFakeCua
cleanup(): Promise<void>
}> {
const home = await mkdtemp(join(tmpdir(), 'hermes-cua-integration-'))
const release = join(home, '.cua-driver', 'packages', 'releases', '0.19.3-test')
await mkdir(release, { recursive: true })
const binary = join(release, 'cua-driver.exe')
await writeFile(binary, '')
await symlink(release, join(home, '.cua-driver', 'packages', 'current'), 'junction')
const runner = new StatefulFakeCua(binary)
const adapter = await CuaDriverAdapter.connect({ platform: 'win32', homeDir: home, runner })
return { adapter, runner, cleanup: () => rm(home, { recursive: true, force: true }) }
}
function identity(id: string) {
return { controlSessionId: id, targetDeviceId: 'desktop-target', runId: 'run-1' }
}
test('semantic CUA action can be bracketed by fresh snapshots without foreground dispatch', async () => {
const { adapter, runner, cleanup } = await harness()
try {
const session = await adapter.openSession(identity('control-snapshot'))
const before = await session.snapshot({ pid: 100, windowId: 200, includeScreenshot: true })
const token = (before.elements as Array<{ element_token: string }>)[0]!.element_token
await session.clickElement({ pid: 100, windowId: 200, elementToken: token })
await session.snapshot({ pid: 100, windowId: 200, includeScreenshot: false })
await session.close()
const calls = runner.calls.filter(call => call.args[0] === 'call' && call.args[1] !== 'health_report')
assert.deepEqual(calls.map(call => call.args[1]), [
'start_session', 'get_window_state', 'click', 'get_window_state', 'end_session'
])
assert.deepEqual(calls[2]!.payload, {
pid: 100,
window_id: 200,
element_token: 'e1234abcd',
session: calls[0]!.payload!.session,
scope: 'window',
delivery_mode: 'background'
})
} finally {
await cleanup()
}
})
test('stale driver token errors and cancellation both fail closed', async () => {
const { adapter, runner, cleanup } = await harness()
try {
const session = await adapter.openSession(identity('control-failure'))
runner.rejectClicksAsStale = true
await assert.rejects(
session.clickElement({ pid: 100, windowId: 200, elementToken: 'edeadbeef' }),
/rejected the action/
)
const controller = new AbortController()
controller.abort()
await assert.rejects(
session.pressKey({ pid: 100, windowId: 200 }, 'escape', controller.signal),
/cancelled/
)
const key = runner.calls.find(call => call.args[1] === 'press_key')
assert.equal(key, undefined)
await session.close()
} finally {
await cleanup()
}
})
test('two control authorities get isolated virtual cursor sessions and independent teardown', async () => {
const { adapter, runner, cleanup } = await harness()
try {
const first = await adapter.openSession(identity('control-one'))
const second = await adapter.openSession(identity('control-two'))
const starts = runner.calls.filter(call => call.args[1] === 'start_session')
assert.equal(starts.length, 2)
assert.notEqual(starts[0]!.payload!.session, starts[1]!.payload!.session)
assert.match(String(starts[0]!.payload!.session), /^hermes-[0-9a-f]{32}$/)
assert.equal(starts[0]!.payload!.capture_scope, 'window')
assert.equal(starts[1]!.payload!.capture_scope, 'window')
await first.close()
await assert.rejects(first.snapshot({ pid: 100, windowId: 200 }), /closed/)
await second.snapshot({ pid: 300, windowId: 400 })
await second.close()
const ends = runner.calls.filter(call => call.args[1] === 'end_session').map(call => call.payload!.session)
assert.deepEqual(new Set(ends), new Set(starts.map(call => call.payload!.session)))
} finally {
await cleanup()
}
})
+326
View File
@@ -0,0 +1,326 @@
import assert from 'node:assert/strict'
import { createHash } from 'node:crypto'
import { access, mkdir, mkdtemp, rm, symlink, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import test from 'node:test'
import {
checkCuaUpdate,
getCuaManagementStatus,
installCuaDriver,
isSupportedCuaVersion,
updateCuaDriver,
type CuaManagementFetch
} from '../src/tools/cuaManagement.js'
import { CuaDriverAdapter, type CuaProcessResult, type CuaProcessRunner } from '../src/tools/cuaDriver.js'
import { computerUseCommand } from '../src/commands/computerUse.js'
const ok = (stdout = ''): CuaProcessResult => ({ stdout, stderr: '', exitCode: 0 })
async function packageHome(version = '0.19.3'): Promise<{ root: string; binary: string }> {
const root = await mkdtemp(join(tmpdir(), 'hermes-cua-management-'))
const release = join(root, '.cua-driver', 'packages', 'releases', `${version}-x86_64-pc-windows-msvc`)
const current = join(root, '.cua-driver', 'packages', 'current')
await mkdir(release, { recursive: true })
const binary = join(release, 'cua-driver.exe')
await writeFile(binary, `fake-cua-${version}`)
await symlink(release, current, 'junction')
return { root, binary }
}
class ProbeRunner implements CuaProcessRunner {
constructor(
readonly version = '0.19.3',
readonly latest = version,
readonly onRun?: (executable: string, args: readonly string[], env?: NodeJS.ProcessEnv) => Promise<void> | void,
readonly health: string = 'ok'
) {}
async run(executable: string, args: readonly string[], options: { env?: NodeJS.ProcessEnv } = {}): Promise<CuaProcessResult> {
await this.onRun?.(executable, args, options.env)
if (args[0] === '--version') return ok(`cua-driver ${this.version}`)
if (args[0] === 'check-update') return ok(JSON.stringify({
checked_at: '2026-08-13T00:00:00Z',
current_version: this.version,
latest_version: this.latest,
update_available: this.latest !== this.version,
error: null
}))
if (args[0] === 'manifest') return ok(JSON.stringify({
schema_version: '1',
binary_version: this.version,
binary_path: executable
}))
if (args[0] === 'list-tools') return ok([
'health_report:', 'start_session:', 'end_session:', 'list_windows:',
'get_window_state:', 'click:', 'set_value:', 'press_key:', 'scroll:'
].join('\n'))
if (args[0] === 'status') return ok('permission mode: standard')
if (args[0] === 'call' && args[1] === 'health_report') return ok(JSON.stringify({
schema_version: '1', driver_version: this.version, overall: this.health
}))
return ok()
}
}
test('supported CUA range is bounded to the adapter contract', () => {
assert.equal(isSupportedCuaVersion('0.19.2'), false)
assert.equal(isSupportedCuaVersion('0.19.3'), true)
assert.equal(isSupportedCuaVersion('0.19.99'), true)
assert.equal(isSupportedCuaVersion('0.20.0'), false)
})
test('status prefers canonical package/current and reports a competing PATH shim', async () => {
const home = await packageHome()
const stale = await mkdtemp(join(tmpdir(), 'hermes-cua-stale-'))
await writeFile(join(stale, 'cua-driver.exe'), 'stale')
try {
const status = await getCuaManagementStatus({
platform: 'win32', homeDir: home.root, path: stale, runner: new ProbeRunner()
})
assert.equal(status.installed, true)
assert.equal(status.current_version, '0.19.3')
assert.equal(status.compatible, true)
assert.equal(status.stale_path_shim, true)
assert.equal(status.canonical_path, home.binary)
assert.equal(status.release_source, 'trycua/cua')
assert.equal(status.telemetry_enabled, false)
} finally {
await rm(home.root, { recursive: true, force: true })
await rm(stale, { recursive: true, force: true })
}
})
test('check-update exposes a newer incompatible release without applying it', async () => {
const home = await packageHome()
try {
const status = await checkCuaUpdate({
platform: 'win32', homeDir: home.root, path: '', runner: new ProbeRunner('0.19.3', '0.20.0')
})
assert.equal(status.update?.update_available, true)
assert.equal(status.update?.latest_version, '0.20.0')
assert.equal(status.update?.compatible, false)
} finally {
await rm(home.root, { recursive: true, force: true })
}
})
test('update refuses an unsupported latest release before any download or apply', async () => {
const home = await packageHome()
let fetches = 0
let powershellRuns = 0
try {
await assert.rejects(updateCuaDriver({
platform: 'win32',
homeDir: home.root,
path: '',
runner: new ProbeRunner('0.19.3', '0.20.0', executable => {
if (executable.toLowerCase() === 'powershell.exe') powershellRuns += 1
}),
fetch: async () => {
fetches += 1
throw new Error('must not fetch')
}
}), /available but unsupported/)
assert.equal(fetches, 0)
assert.equal(powershellRuns, 0)
} finally {
await rm(home.root, { recursive: true, force: true })
}
})
test('install verifies trusted release metadata/checksum and sanitizes installer environment', async () => {
const root = await mkdtemp(join(tmpdir(), 'hermes-cua-install-test-'))
const script = Buffer.from('Write-Output canonical-installer')
const scriptSha = createHash('sha256').update(script).digest('hex')
const fetchImpl: CuaManagementFetch = async url => ({
ok: true,
status: 200,
async arrayBuffer() { return script.buffer.slice(script.byteOffset, script.byteOffset + script.byteLength) },
async json() {
assert.match(url, /release-manifest\.json$/)
return {
schemaVersion: 1,
repository: 'trycua/cua',
product: 'cua-driver-rs',
version: '0.19.3',
tag: 'cua-driver-rs-v0.19.3',
assets: [{ name: 'install.ps1', sha256: scriptSha }]
}
}
})
let installerEnvironment: NodeJS.ProcessEnv | undefined
let installerPath: string | undefined
const runner = new ProbeRunner('0.19.3', '0.19.3', async (executable, args, env) => {
if (!executable.toLowerCase().endsWith('\\windows\\system32\\windowspowershell\\v1.0\\powershell.exe')) return
installerEnvironment = env
installerPath = args[args.indexOf('-File') + 1]
const release = join(root, '.cua-driver', 'packages', 'releases', '0.19.3-x86_64-pc-windows-msvc')
await mkdir(release, { recursive: true })
await writeFile(join(release, 'cua-driver.exe'), 'installed')
await symlink(release, join(root, '.cua-driver', 'packages', 'current'), 'junction')
})
const priorSecret = process.env.OPENAI_API_KEY
process.env.OPENAI_API_KEY = 'must-not-leak'
try {
const status = await installCuaDriver({
platform: 'win32', homeDir: root, path: '', runner, fetch: fetchImpl, systemRoot: 'C:\\Windows'
})
assert.equal(status.operation?.release_manifest_verified, true)
assert.equal(status.operation?.installer_checksum_verified, true)
assert.equal(status.operation?.runtime_verified, true)
assert.equal(installerEnvironment?.OPENAI_API_KEY, undefined)
assert.equal(installerEnvironment?.CUA_DRIVER_RS_TELEMETRY_ENABLED, '0')
assert.equal(installerEnvironment?.CUA_DRIVER_RS_VERSION, '0.19.3')
assert.ok(installerPath)
await assert.rejects(access(installerPath!))
} finally {
if (priorSecret === undefined) delete process.env.OPENAI_API_KEY
else process.env.OPENAI_API_KEY = priorSecret
await rm(root, { recursive: true, force: true })
}
})
test('install rejects invalid release identity metadata before downloading the installer', async () => {
const root = await mkdtemp(join(tmpdir(), 'hermes-cua-invalid-manifest-'))
let requests = 0
try {
await assert.rejects(installCuaDriver({
platform: 'win32', homeDir: root, path: '', runner: new ProbeRunner(),
fetch: async url => {
requests += 1
assert.match(url, /release-manifest\.json$/)
return {
ok: true,
status: 200,
async arrayBuffer() { return new ArrayBuffer(0) },
async json() {
return {
schemaVersion: 1,
repository: 'attacker/fork',
product: 'cua-driver-rs',
version: '0.19.3',
tag: 'cua-driver-rs-v0.19.3',
assets: [{ name: 'install.ps1', sha256: 'a'.repeat(64) }]
}
}
}
}
}), /publisher or version manifest is invalid/)
assert.equal(requests, 1)
} finally {
await rm(root, { recursive: true, force: true })
}
})
test('install rejects an installer whose bytes do not match release metadata', async () => {
const root = await mkdtemp(join(tmpdir(), 'hermes-cua-bad-checksum-'))
let powershellRuns = 0
try {
await assert.rejects(installCuaDriver({
platform: 'win32', homeDir: root, path: '',
runner: new ProbeRunner('0.19.3', '0.19.3', executable => {
if (executable.toLowerCase() === 'powershell.exe') powershellRuns += 1
}),
fetch: async url => ({
ok: true,
status: 200,
async arrayBuffer() { return Buffer.from('tampered').buffer },
async json() {
assert.match(url, /release-manifest\.json$/)
return {
schemaVersion: 1, repository: 'trycua/cua', product: 'cua-driver-rs',
version: '0.19.3', tag: 'cua-driver-rs-v0.19.3',
assets: [{ name: 'install.ps1', sha256: 'a'.repeat(64) }]
}
}
})
}), /checksum verification failed/)
assert.equal(powershellRuns, 0)
} finally {
await rm(root, { recursive: true, force: true })
}
})
test('post-install runtime verification succeeds while explicit health remains degraded', async () => {
const root = await mkdtemp(join(tmpdir(), 'hermes-cua-degraded-'))
const script = Buffer.from('installer')
const checksum = createHash('sha256').update(script).digest('hex')
const runner = new ProbeRunner('0.19.3', '0.19.3', async executable => {
if (!executable.toLowerCase().endsWith('\\windows\\system32\\windowspowershell\\v1.0\\powershell.exe')) return
const release = join(root, '.cua-driver', 'packages', 'releases', '0.19.3-x86_64-pc-windows-msvc')
await mkdir(release, { recursive: true })
await writeFile(join(release, 'cua-driver.exe'), 'installed')
await symlink(release, join(root, '.cua-driver', 'packages', 'current'), 'junction')
}, 'degraded')
try {
const installed = await installCuaDriver({
platform: 'win32', homeDir: root, path: '', runner, systemRoot: 'C:\\Windows',
fetch: async url => ({
ok: true,
status: 200,
async arrayBuffer() { return script.buffer.slice(script.byteOffset, script.byteOffset + script.byteLength) },
async json() {
assert.match(url, /release-manifest\.json$/)
return {
schemaVersion: 1, repository: 'trycua/cua', product: 'cua-driver-rs',
version: '0.19.3', tag: 'cua-driver-rs-v0.19.3',
assets: [{ name: 'install.ps1', sha256: checksum }]
}
}
})
})
assert.equal(installed.operation?.runtime_verified, true)
const health = await CuaDriverAdapter.healthStatus({ platform: 'win32', homeDir: root, runner })
assert.equal(health.state, 'degraded')
} finally {
await rm(root, { recursive: true, force: true })
}
})
test('CLI install and update require explicit confirmation', async () => {
let stderr = ''
const original = process.stderr.write
process.stderr.write = ((chunk: string | Uint8Array) => {
stderr += chunk.toString()
return true
}) as typeof process.stderr.write
try {
assert.equal(await computerUseCommand({ command: 'computer-use', flags: {}, positional: ['cua', 'install'] }), 1)
assert.match(stderr, /requires explicit confirmation with --yes/)
} finally {
process.stderr.write = original
}
})
test('install rejects an unverified installer before process execution', async () => {
const root = await mkdtemp(join(tmpdir(), 'hermes-cua-bad-installer-'))
let processStarted = false
const fetchImpl: CuaManagementFetch = async url => ({
ok: true,
status: 200,
async arrayBuffer() { return Buffer.from('tampered').buffer },
async json() {
assert.match(url, /release-manifest\.json$/)
return {
schemaVersion: 1,
repository: 'trycua/cua',
product: 'cua-driver-rs',
version: '0.19.3',
tag: 'cua-driver-rs-v0.19.3',
assets: [{ name: 'install.ps1', sha256: '0'.repeat(64) }]
}
}
})
const runner = new ProbeRunner('0.19.3', '0.19.3', () => { processStarted = true })
try {
await assert.rejects(
installCuaDriver({ platform: 'win32', homeDir: root, path: '', runner, fetch: fetchImpl }),
/checksum verification failed/
)
assert.equal(processStarted, false)
} finally {
await rm(root, { recursive: true, force: true })
}
})
+54
View File
@@ -6,6 +6,7 @@ import test from 'node:test'
import type { DaemonStatus } from '../src/lib/daemonStatus.js'
import {
__acquireProcessLockForTests as acquireProcessLock,
__buildDaemonChildArgsForTests as buildDaemonChildArgs,
__buildElevationLaunchPlanForTests as buildElevationLaunchPlan,
__daemonStatusIsReadyForTests as daemonStatusIsReady,
@@ -44,6 +45,59 @@ test('detached startup ignores stale status and succeeds only for the child pid'
assert.equal(result.outcome === 'ready' ? result.status.pid : null, 42)
})
test('concurrent daemon starts elect exactly one cross-process lock owner', async t => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'hermes-daemon-lock-'))
t.after(() => fs.rm(dir, { recursive: true, force: true }))
const lockPath = path.join(dir, 'daemon-lifecycle.lock')
const attempts = await Promise.allSettled([
acquireProcessLock(lockPath, { timeoutMs: 0, purpose: 'concurrent start A' }),
acquireProcessLock(lockPath, { timeoutMs: 0, purpose: 'concurrent start B' })
])
const winners = attempts.filter(
(attempt): attempt is PromiseFulfilledResult<Awaited<ReturnType<typeof acquireProcessLock>>> =>
attempt.status === 'fulfilled'
)
const losers = attempts.filter(attempt => attempt.status === 'rejected')
assert.equal(winners.length, 1)
assert.equal(losers.length, 1)
assert.match(String((losers[0] as PromiseRejectedResult).reason), /timed out.*lock/s)
await winners[0]!.value.release()
})
test('daemon lock recovers a dead owner and release cannot remove a replacement owner', async t => {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'hermes-daemon-stale-lock-'))
t.after(() => fs.rm(dir, { recursive: true, force: true }))
const lockPath = path.join(dir, 'daemon-instance.lock')
await fs.mkdir(lockPath)
await fs.writeFile(path.join(lockPath, 'owner.json'), JSON.stringify({
pid: 999_999,
process_name: 'dead-hermes-relay.exe',
token: 'dead-owner',
created_at: 1,
purpose: 'daemon runtime'
}))
const lock = await acquireProcessLock(lockPath, {
timeoutMs: 100,
purpose: 'daemon runtime replacement',
ownerAlive: () => false
})
const replacement = {
...lock.owner,
token: 'newer-owner'
}
await fs.writeFile(path.join(lockPath, 'owner.json'), JSON.stringify(replacement))
await lock.release()
assert.equal((await fs.stat(lockPath)).isDirectory(), true)
assert.equal(
JSON.parse(await fs.readFile(path.join(lockPath, 'owner.json'), 'utf8')).token,
'newer-owner'
)
})
test('detached startup reports log evidence before generic child-exit failure', async () => {
let now = 0
const result = await waitForDetachedStartup(42, 1_000, {
+41
View File
@@ -9,6 +9,8 @@ import {
readDesktopUseSettings,
readDesktopUseSettingsSync,
requestComputerGrantCancellation,
setActivityScreenshotRetention,
setComputerControlSettings,
setDesktopUseEnabled
} from '../src/lib/desktopUseSettings.js'
import { shouldAdvertiseComputerUse } from '../src/tools/handlerSet.js'
@@ -34,6 +36,45 @@ test('desktop-use preference defaults off and persists explicit changes', async
}
})
test('computer control settings default fail-closed and survive desktop-use changes', async () => {
const dir = await mkdtemp(join(tmpdir(), 'hermes-computer-control-'))
const settingsPath = join(dir, 'desktop-settings.json')
try {
assert.deepEqual(await readDesktopUseSettings(settingsPath), {
computer_use_enabled: false,
computer_control_engine: 'cua',
cua_cursor_enabled: false,
activity_screenshot_retention_enabled: true,
activity_screenshot_retention_days: 7
})
await setComputerControlSettings({
computer_control_engine: 'cua',
cua_cursor_enabled: true
}, settingsPath)
await setDesktopUseEnabled(true, settingsPath)
const settings = await readDesktopUseSettings(settingsPath)
assert.equal(settings.computer_use_enabled, true)
assert.equal(settings.computer_control_engine, 'cua')
assert.equal(settings.cua_cursor_enabled, true)
} finally {
await rm(dir, { recursive: true, force: true })
}
})
test('screenshot evidence retention is explicit and survives other desktop setting changes', async () => {
const dir = await mkdtemp(join(tmpdir(), 'hermes-screenshot-retention-'))
const settingsPath = join(dir, 'desktop-settings.json')
try {
await setActivityScreenshotRetention(false, 30, settingsPath)
await setDesktopUseEnabled(true, settingsPath)
const settings = await readDesktopUseSettings(settingsPath)
assert.equal(settings.activity_screenshot_retention_enabled, false)
assert.equal(settings.activity_screenshot_retention_days, 30)
} finally {
await rm(dir, { recursive: true, force: true })
}
})
test('grant cancellation bridge is consumed exactly once', async () => {
const dir = await mkdtemp(join(tmpdir(), 'hermes-desktop-cancel-'))
const cancelPath = join(dir, 'cancel-active.json')
+30 -1
View File
@@ -31,6 +31,8 @@ test('installer launch is delayed until the running CLI can exit', () => {
assert.equal(plan.options.env?.HERMES_RELAY_SETUP_PATH, 'C:\\Temp\\hermes setup.exe')
assert.equal(plan.options.env?.HERMES_RELAY_SETUP_SILENT, '1')
assert.equal(plan.options.env?.HERMES_RELAY_SETUP_INSTALL_DIR, 'C:\\Hermes custom')
assert.equal(plan.options.env?.HERMES_RELAY_SETUP_CALLER_PID, String(process.pid))
assert.match(plan.args.join(' '), /Wait-Process -Id \$callerPid/)
assert.match(plan.args.join(' '), /Start-Sleep/)
assert.match(plan.args.join(' '), /\/D=/)
assert.match(plan.args.join(' '), /Remove-Item -LiteralPath \$installer/)
@@ -44,7 +46,7 @@ test('PowerShell launches an installer whose path contains spaces via environmen
const fakeInstaller = join(scratch, 'fake setup.exe')
await copyFile(join(process.env.WINDIR ?? 'C:\\Windows', 'System32', 'whoami.exe'), fakeInstaller)
try {
const plan = windowsInstallerLaunchPlan(fakeInstaller, { silent: false, delayMs: 0 })
const plan = windowsInstallerLaunchPlan(fakeInstaller, { silent: false, delayMs: 0, callerPid: 0 })
const result = spawnSync(plan.program, plan.args, { ...plan.options, detached: false, stdio: 'pipe' })
assert.equal(result.status, 0, result.stderr?.toString())
} finally {
@@ -76,6 +78,25 @@ test('NSIS bundle cleans cooperative-update and local-development backups', asyn
}
})
test('NSIS bundle quiesces tray children and retries failed payload extraction once', async () => {
const script = await readFile(new URL('../tray/installer/hermes-relay.nsi', import.meta.url), 'utf8')
const core = script.slice(script.indexOf('Section "Hermes-Relay CLI and management UI"'), script.indexOf('Section "Start tray when I sign in"'))
const trayStop = core.indexOf('/IM hermes-relay-tray.exe /T /F')
const cliDrain = core.indexOf('/IM hermes-relay.exe /T /F')
const firstInstall = core.indexOf('File /oname=hermes-relay.exe "${CLI_EXE}"')
const retry = core.indexOf('install_attempt_failed:')
assert.ok(trayStop >= 0, 'installer should terminate the tray process tree')
assert.ok(cliDrain < firstInstall, 'installed processes must be drained before replacement')
assert.ok(retry > firstInstall, 'installer should retry the first failed extraction attempt')
assert.match(core, /StrCpy \$InstallAttempt "1"[\s\S]*Goto install_attempt/)
assert.match(core, /ClearErrors[\s\S]*File \/oname=hermes-relay\.exe[\s\S]*File \/oname=hermes-relay-tray\.exe[\s\S]*IfErrors install_attempt_failed/)
assert.match(core, /SetErrorLevel 1\s+Quit/)
assert.match(core, /MessageBox MB_ICONSTOP .* \/SD IDOK/)
assert.match(core, /taskkill\.exe" \/IM hermes-relay-tray\.exe \/T \/F/)
assert.match(core, /taskkill\.exe" \/IM hermes-relay\.exe \/T \/F/)
})
test('local tray installation embeds production assets instead of loading devUrl', async () => {
const script = await readFile(new URL('../scripts/dev-install-tray.mjs', import.meta.url), 'utf8')
assert.match(script, /'--features', 'custom-protocol'/)
@@ -87,6 +108,14 @@ test('CLI opens the GUI process without forcing a hidden Windows startup state',
assert.doesNotMatch(source, /windowsHide:\s*true/)
})
test('tray update helper preserves the current install directory and cleans its installer', async () => {
const source = await readFile(new URL('../tray/src/main.rs', import.meta.url), 'utf8')
assert.match(source, /\.env\("HERMES_UPDATE_INSTALL_DIR", install_dir\)/)
assert.match(source, /\$installerArgs=@\('\/S',\('\/D=' \+ \$env:HERMES_UPDATE_INSTALL_DIR\)\)/)
assert.match(source, /Remove-Item -LiteralPath \$env:HERMES_UPDATE_INSTALLER -Force/)
assert.match(source, /exit \$exitCode/)
})
test('POSIX installer only advertises artifacts produced by the release workflow', async () => {
const script = await readFile(new URL('../scripts/install.sh', import.meta.url), 'utf8')
assert.doesNotMatch(script, /hermes-relay-linux-arm64/)
+2 -1
View File
@@ -1232,8 +1232,9 @@ checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "hermes-relay-tray"
version = "0.4.0-alpha.8"
version = "0.4.0-beta.3"
dependencies = [
"base64 0.22.1",
"serde",
"serde_json",
"tauri",
+6 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "hermes-relay-tray"
version = "0.4.0-alpha.8"
version = "0.4.0-beta.3"
description = "Compact Windows management UI for Hermes-Relay CLI"
authors = ["Axiom Labs"]
edition = "2021"
@@ -14,6 +14,7 @@ path = "src/main.rs"
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
base64 = "0.22"
tauri = { version = "2", features = ["tray-icon", "image-png"] }
[features]
@@ -25,6 +26,10 @@ windows = { version = "0.61.3", features = [
"Win32_Graphics_Gdi",
"Win32_Security",
"Win32_System_Com",
"Win32_System_Diagnostics_Debug",
"Win32_System_Diagnostics_ToolHelp",
"Win32_System_IO",
"Win32_System_JobObjects",
"Win32_System_Ole",
"Win32_System_Threading",
"Win32_System_Variant",
+32 -4
View File
@@ -19,6 +19,7 @@ VIAddVersionKey "FileVersion" "${VERSION}"
VIAddVersionKey "LegalCopyright" "MIT License"
Var ExistingStartup
Var InstallAttempt
Function .onInit
ReadRegStr $ExistingStartup HKCU "Software\Microsoft\Windows\CurrentVersion\Run" "HermesRelayTray"
@@ -42,20 +43,47 @@ FunctionEnd
Section "Hermes-Relay CLI and management UI" SEC_CORE
SectionIn RO
IfFileExists "$INSTDIR\hermes-relay.exe" 0 tray_stop
; Stop the tray first so its snapshot polling cannot launch another CLI
; process while setup is trying to replace hermes-relay.exe. The following
; /T also terminates a poll already in flight before payload extraction.
nsExec::ExecToLog '"$SYSDIR\taskkill.exe" /IM hermes-relay-tray.exe /T /F'
IfFileExists "$INSTDIR\hermes-relay.exe" 0 processes_quiesced
nsExec::ExecToLog '"$INSTDIR\hermes-relay.exe" daemon stop'
tray_stop:
nsExec::ExecToLog '"$SYSDIR\taskkill.exe" /IM hermes-relay-tray.exe /F'
Sleep 250
nsExec::ExecToLog '"$SYSDIR\taskkill.exe" /IM hermes-relay.exe /T /F'
processes_quiesced:
; Drain the daemon and any short-lived tray/management CLI children that
; raced with shutdown. An explicit bundle update is the lifecycle boundary
; for installed Hermes-Relay processes.
Sleep 350
Delete "$INSTDIR\hermes-relay.new.exe"
Delete "$INSTDIR\hermes-relay.old.exe"
Delete "$INSTDIR\hermes-relay.exe.bak"
Delete "$INSTDIR\hermes-relay-tray.exe.bak"
StrCpy $InstallAttempt "0"
install_attempt:
SetOutPath "$INSTDIR"
ClearErrors
File /oname=hermes-relay.exe "${CLI_EXE}"
File /oname=hermes-relay-tray.exe "${TRAY_EXE}"
File /oname=hermes-relay-path.ps1 "${PATH_HELPER}"
File /oname=hermes-relay-ui.cmd "${UI_SHIM}"
IfErrors install_attempt_failed
; ClearErrors/IfErrors covers both CLI and tray writes, so setup cannot
; publish registry metadata after a locked or partial replacement.
Goto cli_verified
install_attempt_failed:
StrCmp $InstallAttempt "0" 0 cli_verification_failed
StrCpy $InstallAttempt "1"
nsExec::ExecToLog '"$SYSDIR\taskkill.exe" /IM hermes-relay-tray.exe /T /F'
nsExec::ExecToLog '"$SYSDIR\taskkill.exe" /IM hermes-relay.exe /T /F'
Sleep 350
Goto install_attempt
cli_verification_failed:
MessageBox MB_ICONSTOP "Hermes-Relay CLI UI ${VERSION} could not replace the running installation. Close Hermes-Relay processes and run setup again." /SD IDOK
SetErrorLevel 1
Quit
cli_verified:
WriteUninstaller "$INSTDIR\uninstall-hermes-relay.exe"
WriteRegStr HKCU "Software\HermesRelay" "InstallDir" "$INSTDIR"
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "@hermes-relay/tray-ui",
"version": "0.4.0-alpha.8",
"version": "0.4.0-beta.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@hermes-relay/tray-ui",
"version": "0.4.0-alpha.8",
"version": "0.4.0-beta.3",
"dependencies": {
"@tauri-apps/api": "^2.8.0",
"lucide-react": "^0.468.0",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@hermes-relay/tray-ui",
"private": true,
"version": "0.4.0-alpha.8",
"version": "0.4.0-beta.3",
"type": "module",
"scripts": {
"dev": "vite --host 127.0.0.1",
+575
View File
@@ -0,0 +1,575 @@
//! Bounded execution for short-lived Windows helper commands.
//!
//! Each invocation owns a Job Object. Closing it terminates any processes that
//! the command spawned, so a timed-out CLI cannot leave an orphaned process
//! tree behind. Output is drained concurrently while retaining only a bounded
//! prefix for diagnostics and JSON parsing.
use std::{
fmt,
io::{self, Read},
mem::size_of,
os::windows::{io::AsRawHandle, process::CommandExt},
process::{Command, ExitStatus, Stdio},
sync::{
atomic::{AtomicBool, Ordering},
Arc, Mutex,
},
thread::{self, JoinHandle},
time::{Duration, Instant},
};
use windows::{
core::{Error as WindowsError, PCWSTR},
Win32::{
Foundation::{CloseHandle, HANDLE},
System::{
Diagnostics::ToolHelp::{
CreateToolhelp32Snapshot, Thread32First, Thread32Next, TH32CS_SNAPTHREAD,
THREADENTRY32,
},
JobObjects::{
AssignProcessToJobObject, CreateJobObjectW, JobObjectExtendedLimitInformation,
SetInformationJobObject, JOBOBJECT_EXTENDED_LIMIT_INFORMATION,
JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE,
},
Threading::{
OpenThread, ResumeThread, CREATE_NO_WINDOW, CREATE_SUSPENDED, THREAD_SUSPEND_RESUME,
},
IO::CancelSynchronousIo,
},
},
};
const WAIT_POLL_INTERVAL: Duration = Duration::from_millis(10);
#[derive(Clone, Copy, Debug)]
pub(crate) struct RunOptions {
pub(crate) timeout: Duration,
/// Maximum number of bytes retained from each output stream.
pub(crate) max_capture_bytes: usize,
pub(crate) process_tree: ProcessTreePolicy,
}
impl RunOptions {
pub(crate) const fn new(timeout: Duration, max_capture_bytes: usize) -> Self {
Self {
timeout,
max_capture_bytes,
process_tree: ProcessTreePolicy::KillDescendants,
}
}
pub(crate) const fn direct_child_only(mut self) -> Self {
self.process_tree = ProcessTreePolicy::DirectChildOnly;
self
}
}
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub(crate) enum ProcessTreePolicy {
/// Contain the invocation in a Job Object and end every descendant when the
/// direct child exits or times out. This is the safe default for probes.
KillDescendants,
/// Allow a successful launcher to leave a deliberately detached daemon.
/// On timeout, only the direct child can be terminated.
DirectChildOnly,
}
#[derive(Clone, Debug, Default)]
pub(crate) struct CapturedOutput {
pub(crate) bytes: Vec<u8>,
pub(crate) total_bytes: u64,
pub(crate) truncated: bool,
}
#[derive(Debug)]
pub(crate) struct ProcessOutcome {
pub(crate) status: ExitStatus,
pub(crate) stdout: CapturedOutput,
pub(crate) stderr: CapturedOutput,
pub(crate) timed_out: bool,
pub(crate) duration: Duration,
}
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub(crate) enum ProcessStage {
CreateJob,
ConfigureJob,
Spawn,
AssignJob,
Resume,
Wait,
ReadStdout,
ReadStderr,
}
fn resume_suspended_process(process_id: u32, started: Instant) -> Result<(), ProcessError> {
let snapshot = unsafe { CreateToolhelp32Snapshot(TH32CS_SNAPTHREAD, 0) }
.map_err(|error| ProcessError::new(ProcessStage::Resume, started.elapsed(), error))?;
let mut entry = THREADENTRY32 {
dwSize: size_of::<THREADENTRY32>() as u32,
..THREADENTRY32::default()
};
let thread_id = (|| {
unsafe { Thread32First(snapshot, &mut entry) }
.map_err(|error| ProcessError::new(ProcessStage::Resume, started.elapsed(), error))?;
loop {
if entry.th32OwnerProcessID == process_id {
return Ok(entry.th32ThreadID);
}
if unsafe { Thread32Next(snapshot, &mut entry) }.is_err() {
return Err(ProcessError::new(
ProcessStage::Resume,
started.elapsed(),
io::Error::new(
io::ErrorKind::NotFound,
"suspended process thread was not found",
),
));
}
}
})();
unsafe {
let _ = CloseHandle(snapshot);
}
let thread_id = thread_id?;
let thread = unsafe { OpenThread(THREAD_SUSPEND_RESUME, false, thread_id) }
.map_err(|error| ProcessError::new(ProcessStage::Resume, started.elapsed(), error))?;
let resume_result = unsafe { ResumeThread(thread) };
unsafe {
let _ = CloseHandle(thread);
}
if resume_result == u32::MAX {
return Err(ProcessError::new(
ProcessStage::Resume,
started.elapsed(),
WindowsError::from_win32(),
));
}
Ok(())
}
#[derive(Debug)]
pub(crate) struct ProcessError {
pub(crate) stage: ProcessStage,
pub(crate) duration: Duration,
source: Box<dyn std::error::Error + Send + Sync>,
}
impl ProcessError {
fn new(
stage: ProcessStage,
duration: Duration,
source: impl std::error::Error + Send + Sync + 'static,
) -> Self {
Self {
stage,
duration,
source: Box::new(source),
}
}
}
impl fmt::Display for ProcessError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
formatter,
"process {:?} failed after {:?}: {}",
self.stage, self.duration, self.source
)
}
}
impl std::error::Error for ProcessError {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
Some(self.source.as_ref())
}
}
struct KillOnCloseJob(Option<HANDLE>);
impl KillOnCloseJob {
fn create(started: Instant) -> Result<Self, ProcessError> {
let handle = unsafe { CreateJobObjectW(None, PCWSTR::null()) }.map_err(|error| {
ProcessError::new(ProcessStage::CreateJob, started.elapsed(), error)
})?;
let mut limits = JOBOBJECT_EXTENDED_LIMIT_INFORMATION::default();
limits.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;
let configured = unsafe {
SetInformationJobObject(
handle,
JobObjectExtendedLimitInformation,
(&limits as *const JOBOBJECT_EXTENDED_LIMIT_INFORMATION).cast(),
size_of::<JOBOBJECT_EXTENDED_LIMIT_INFORMATION>() as u32,
)
};
if let Err(error) = configured {
unsafe {
let _ = CloseHandle(handle);
}
return Err(ProcessError::new(
ProcessStage::ConfigureJob,
started.elapsed(),
error,
));
}
Ok(Self(Some(handle)))
}
fn assign(&self, process: HANDLE, started: Instant) -> Result<(), ProcessError> {
unsafe { AssignProcessToJobObject(self.0.expect("open job handle"), process) }
.map_err(|error| ProcessError::new(ProcessStage::AssignJob, started.elapsed(), error))
}
/// Closing a KILL_ON_JOB_CLOSE job is the process-tree termination signal.
fn close(&mut self) {
if let Some(handle) = self.0.take() {
unsafe {
let _ = CloseHandle(handle);
}
}
}
}
impl Drop for KillOnCloseJob {
fn drop(&mut self) {
self.close();
}
}
/// Runs a command with bounded lifetime, process-tree ownership, and output.
///
/// The command's stdin is disconnected and stdout/stderr are replaced with
/// pipes. The retained bytes are a prefix; readers continue draining after the
/// limit so verbose children cannot deadlock on a full pipe.
pub(crate) fn run(
command: &mut Command,
options: RunOptions,
) -> Result<ProcessOutcome, ProcessError> {
let started = Instant::now();
let mut job = match options.process_tree {
ProcessTreePolicy::KillDescendants => Some(KillOnCloseJob::create(started)?),
ProcessTreePolicy::DirectChildOnly => None,
};
let creation_flags = match options.process_tree {
ProcessTreePolicy::KillDescendants => CREATE_NO_WINDOW | CREATE_SUSPENDED,
ProcessTreePolicy::DirectChildOnly => CREATE_NO_WINDOW,
};
command
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.creation_flags(creation_flags.0);
let mut child = command
.spawn()
.map_err(|error| ProcessError::new(ProcessStage::Spawn, started.elapsed(), error))?;
let process_handle = HANDLE(child.as_raw_handle());
if let Some(job) = job.as_ref() {
if let Err(error) = job.assign(process_handle, started) {
let _ = child.kill();
let _ = child.wait();
return Err(error);
}
if let Err(error) = resume_suspended_process(child.id(), started) {
let _ = child.kill();
let _ = child.wait();
return Err(error);
}
}
// Start both readers only after containment succeeds, but before waiting.
let stdout = child.stdout.take().expect("stdout was configured as piped");
let stderr = child.stderr.take().expect("stderr was configured as piped");
let stdout_reader = OutputReader::spawn(stdout, options.max_capture_bytes);
let stderr_reader = OutputReader::spawn(stderr, options.max_capture_bytes);
let deadline = started.checked_add(options.timeout);
let mut timed_out = false;
let status = loop {
match child.try_wait() {
Ok(Some(status)) => break status,
Ok(None) => {}
Err(error) => {
if let Some(job) = job.as_mut() {
job.close();
} else {
let _ = child.kill();
}
let _ = child.wait();
return Err(ProcessError::new(
ProcessStage::Wait,
started.elapsed(),
error,
));
}
}
if match deadline {
Some(deadline) => Instant::now() >= deadline,
None => true,
} {
timed_out = true;
if let Some(job) = job.as_mut() {
job.close();
} else {
let _ = child.kill();
}
break child.wait().map_err(|error| {
ProcessError::new(ProcessStage::Wait, started.elapsed(), error)
})?;
}
let remaining = deadline
.and_then(|deadline| deadline.checked_duration_since(Instant::now()))
.unwrap_or(Duration::ZERO);
thread::sleep(WAIT_POLL_INTERVAL.min(remaining));
};
// End any descendants that outlived the direct child, then collect EOF from
// both pipes. This also prevents successful helper commands from leaking.
if let Some(job) = job.as_mut() {
job.close();
}
let allow_detached_descendants = options.process_tree == ProcessTreePolicy::DirectChildOnly;
let stdout = stdout_reader.finish(
ProcessStage::ReadStdout,
started,
allow_detached_descendants,
)?;
let stderr = stderr_reader.finish(
ProcessStage::ReadStderr,
started,
allow_detached_descendants,
)?;
Ok(ProcessOutcome {
status,
stdout,
stderr,
timed_out,
duration: started.elapsed(),
})
}
struct OutputReader {
thread: Option<JoinHandle<io::Result<()>>>,
output: Arc<Mutex<CapturedOutput>>,
stop: Arc<AtomicBool>,
}
impl OutputReader {
fn spawn<R>(mut reader: R, max_capture_bytes: usize) -> Self
where
R: Read + Send + 'static,
{
let output = Arc::new(Mutex::new(CapturedOutput {
bytes: Vec::with_capacity(max_capture_bytes.min(8 * 1024)),
..CapturedOutput::default()
}));
let stop = Arc::new(AtomicBool::new(false));
let worker_output = Arc::clone(&output);
let worker_stop = Arc::clone(&stop);
let thread = thread::spawn(move || {
let mut buffer = [0_u8; 8 * 1024];
loop {
if worker_stop.load(Ordering::Acquire) {
return Ok(());
}
let read = match reader.read(&mut buffer) {
Ok(read) => read,
Err(_) if worker_stop.load(Ordering::Acquire) => return Ok(()),
Err(error) => return Err(error),
};
if read == 0 {
return Ok(());
}
let mut output = worker_output
.lock()
.map_err(|_| io::Error::other("output capture lock was poisoned"))?;
output.total_bytes = output.total_bytes.saturating_add(read as u64);
let remaining = max_capture_bytes.saturating_sub(output.bytes.len());
output
.bytes
.extend_from_slice(&buffer[..read.min(remaining)]);
output.truncated = output.total_bytes > output.bytes.len() as u64;
}
});
Self {
thread: Some(thread),
output,
stop,
}
}
fn finish(
mut self,
stage: ProcessStage,
started: Instant,
allow_detached_descendants: bool,
) -> Result<CapturedOutput, ProcessError> {
let thread = self.thread.take().expect("reader thread is present");
if allow_detached_descendants && !thread.is_finished() {
// Give the direct child a moment to flush. If a deliberately
// detached descendant inherited the pipe, cancel this thread's
// pending read so the bounded runner does not wait for that daemon.
let grace_deadline = Instant::now() + Duration::from_millis(25);
while !thread.is_finished() && Instant::now() < grace_deadline {
thread::sleep(Duration::from_millis(1));
}
self.stop.store(true, Ordering::Release);
let cancel_deadline = Instant::now() + Duration::from_millis(100);
while !thread.is_finished() && Instant::now() < cancel_deadline {
unsafe {
let _ = CancelSynchronousIo(HANDLE(thread.as_raw_handle()));
}
thread::sleep(Duration::from_millis(1));
}
}
if !thread.is_finished() && allow_detached_descendants {
// Cancellation is best-effort at the Win32 boundary. Return the
// bounded snapshot instead of allowing an inherited pipe to turn a
// timeout into an unbounded wait; the stop flag retires the reader
// after its pending operation returns.
return self.snapshot(stage, started);
}
match thread.join() {
Ok(Ok(())) => self.snapshot(stage, started),
Ok(Err(error)) => Err(ProcessError::new(stage, started.elapsed(), error)),
Err(_) => Err(ProcessError::new(
stage,
started.elapsed(),
io::Error::other("output reader thread panicked"),
)),
}
}
fn snapshot(
&self,
stage: ProcessStage,
started: Instant,
) -> Result<CapturedOutput, ProcessError> {
self.output
.lock()
.map(|output| output.clone())
.map_err(|_| {
ProcessError::new(
stage,
started.elapsed(),
io::Error::other("output capture lock was poisoned"),
)
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::fs;
fn cmd(script: &str) -> Command {
let mut command = Command::new("cmd.exe");
command.args(["/D", "/S", "/C", script]);
command
}
#[test]
fn captures_stdout_and_stderr() {
let outcome = run(
&mut cmd("echo stdout-text & echo stderr-text 1>&2"),
RunOptions::new(Duration::from_secs(2), 1024),
)
.expect("command should run");
assert!(outcome.status.success());
assert!(!outcome.timed_out);
assert!(String::from_utf8_lossy(&outcome.stdout.bytes).contains("stdout-text"));
assert!(String::from_utf8_lossy(&outcome.stderr.bytes).contains("stderr-text"));
}
#[test]
fn drains_output_after_capture_limit() {
let outcome = run(
&mut cmd("for /L %i in (1,1,2000) do @echo 12345678901234567890"),
RunOptions::new(Duration::from_secs(5), 64),
)
.expect("verbose command should not deadlock");
assert!(outcome.status.success());
assert_eq!(outcome.stdout.bytes.len(), 64);
assert!(outcome.stdout.truncated);
assert!(outcome.stdout.total_bytes > outcome.stdout.bytes.len() as u64);
}
#[test]
fn times_out_and_reaps_child() {
let outcome = run(
&mut cmd("ping 127.0.0.1 -n 30 >nul"),
RunOptions::new(Duration::from_millis(100), 1024),
)
.expect("timed out command should still return an outcome");
assert!(outcome.timed_out);
assert!(outcome.duration < Duration::from_secs(5));
assert!(outcome.status.code().is_some());
}
#[test]
fn direct_child_mode_times_out_and_reaps_child() {
let mut command = Command::new("ping.exe");
command.args(["127.0.0.1", "-n", "30"]);
let outcome = run(
&mut command,
RunOptions::new(Duration::from_millis(100), 1024).direct_child_only(),
)
.expect("timed out direct child should still return an outcome");
assert!(outcome.timed_out);
assert!(outcome.duration < Duration::from_secs(5));
assert!(outcome.status.code().is_some());
}
#[test]
fn timeout_reaps_a_spawned_descendant() {
let pid_path = std::env::temp_dir().join(format!(
"hermes-bounded-process-descendant-{}-{}.txt",
std::process::id(),
Instant::now().elapsed().as_nanos()
));
let escaped_path = pid_path.display().to_string().replace('\'', "''");
let script = format!(
"$child = Start-Process ping.exe -ArgumentList '127.0.0.1 -n 30' -WindowStyle Hidden -PassThru; Set-Content -LiteralPath '{escaped_path}' -Value $child.Id; Wait-Process -Id $child.Id"
);
let mut command = Command::new("powershell.exe");
command.args(["-NoProfile", "-Command", &script]);
let outcome = run(&mut command, RunOptions::new(Duration::from_secs(8), 1024))
.expect("timed out process tree should return an outcome");
assert!(outcome.timed_out);
let pid = fs::read_to_string(&pid_path)
.expect("descendant pid should be recorded")
.trim()
.parse::<u32>()
.expect("descendant pid should be numeric");
let probe = Command::new("powershell.exe")
.args([
"-NoProfile",
"-Command",
&format!("if (Get-Process -Id {pid} -ErrorAction SilentlyContinue) {{ exit 1 }}"),
])
.status()
.expect("descendant liveness probe should run");
let _ = fs::remove_file(pid_path);
assert!(probe.success(), "descendant process {pid} survived timeout");
}
}
+939 -113
View File
File diff suppressed because it is too large Load Diff
+45 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Hermes-Relay CLI UI",
"version": "0.4.0-alpha.8",
"version": "0.4.0-beta.3",
"identifier": "com.axiomlabs.hermes-relay-tray",
"build": {
"beforeDevCommand": "npm run dev",
@@ -47,6 +47,50 @@
"alwaysOnTop": true,
"skipTaskbar": true,
"center": false
},
{
"label": "notice",
"title": "Hermes-Relay connection status",
"width": 360,
"height": 126,
"resizable": false,
"fullscreen": false,
"decorations": false,
"transparent": true,
"backgroundColor": [
0,
0,
0,
0
],
"shadow": false,
"visible": false,
"alwaysOnTop": true,
"skipTaskbar": true,
"center": false
},
{
"label": "evidence",
"title": "Hermes-Relay screenshot evidence",
"width": 900,
"height": 650,
"minWidth": 560,
"minHeight": 420,
"resizable": true,
"fullscreen": false,
"decorations": false,
"transparent": true,
"backgroundColor": [
0,
0,
0,
0
],
"shadow": true,
"visible": false,
"alwaysOnTop": true,
"skipTaskbar": true,
"center": true
}
],
"security": {
@@ -0,0 +1,6 @@
#![cfg(windows)]
// The tray is a binary crate. Include the module directly so its focused unit
// tests remain runnable independently of main.rs integration.
#[path = "../src/bounded_process.rs"]
mod bounded_process;
+91 -4
View File
@@ -102,6 +102,30 @@ fn management_window_owns_the_expected_narrow_surfaces() {
"Experimental",
"open_management_from_grant",
"Open in UI",
"computer_control_engine",
"Computer control",
"CUA Driver",
"Not installed",
"Incompatible",
"Degraded",
"Ready",
"Animated agent cursor",
"Background only",
"Preferred structured engine",
"Compatibility",
"Active session",
"Event timeline",
"Post-action snapshot captured",
"Authenticated control session",
"computer_cua_status",
"computer_cua_health",
"computer_cua_install",
"computer_cua_check_update",
"computer_cua_update",
"verified compatible driver",
"set_computer_control_engine",
"set_cua_cursor_enabled",
"visual overlays—not additional Windows hardware pointers",
] {
assert!(
source.contains(required) || security.contains(required),
@@ -128,8 +152,11 @@ fn grants_use_the_dedicated_card_and_host_changes_reconcile_daemon_truth() {
.and_then(|windows| windows.iter().find(|window| window["label"] == "grant"))
.expect("grant window configuration");
assert!(ui.contains("isGrantWindow ? <GrantWindow /> : <ManagementApp />"));
assert!(ui.contains("isGrantWindow ? <GrantWindow />"));
assert!(ui.contains("isNoticeWindow ? <ConnectionNoticeWindow />"));
assert!(ui.contains("isEvidenceWindow ? <EvidenceWindow />"));
assert!(ui.contains("present_grant_window"));
assert!(ui.contains("get_pending_grant_context"));
assert!(ui.contains("Remote access request"));
assert!(native.contains("start_grant_watcher"));
assert!(native.contains("get_webview_window(\"grant\")"));
@@ -140,7 +167,8 @@ fn grants_use_the_dedicated_card_and_host_changes_reconcile_daemon_truth() {
serde_json::json!([0, 0, 0, 0])
);
assert_eq!(grant_window["shadow"], false);
assert!(ui.contains("(snapshot.daemon.configured_url ?? snapshot.daemon.url) === host.url"));
assert!(ui.contains("const daemonTargetsHost"));
assert!(ui.contains("(snapshot?.daemon.configured_url ?? snapshot?.daemon.url) === host.url"));
assert!(ui.contains("formatGrantScope(grant.scope)"));
assert!(ui.contains("grantAction(grant.scope)"));
assert!(ui.contains("Requested action"));
@@ -149,6 +177,10 @@ fn grants_use_the_dedicated_card_and_host_changes_reconcile_daemon_truth() {
assert!(native.contains("management.completed"));
assert!(native.contains("append_management_event"));
assert!(native.contains("fn clear_activity"));
assert!(native.contains("fn get_activity_screenshot"));
assert!(native.contains("fn set_activity_screenshot_retention"));
assert!(native.contains("fn start_connection_watcher"));
assert!(native.contains("fn present_connection_notice"));
assert!(native.contains("skip_serializing_if = \"Option::is_none\""));
assert!(native.contains("popup_position"));
assert!(native.contains("window.outer_size()"));
@@ -157,9 +189,22 @@ fn grants_use_the_dedicated_card_and_host_changes_reconcile_daemon_truth() {
assert!(native.contains("mpsc::channel::<TrayAction>()"));
assert!(native.contains("app.run_on_main_thread"));
assert!(native.contains("async fn get_snapshot"));
assert!(native.contains("spawn_blocking(build_snapshot)"));
assert!(native.contains("spawn_blocking(build_snapshot_coalesced)"));
assert!(native.contains("async fn check_desktop_update"));
assert!(native.contains("async fn install_desktop_update"));
for command in [
"async fn computer_cua_status",
"async fn computer_cua_health",
"async fn computer_cua_install",
"async fn computer_cua_check_update",
"async fn computer_cua_update",
] {
assert!(
native.contains(command),
"blocking CUA management command: {command}"
);
}
assert!(native.matches("spawn_blocking(||").count() >= 6);
assert!(native.contains("fn set_host_capability"));
assert!(native.contains("fn hardware_availability"));
assert!(native.contains("HERMES_RELAY_CODE"));
@@ -190,6 +235,44 @@ fn grants_use_the_dedicated_card_and_host_changes_reconcile_daemon_truth() {
assert!(!native.contains("TRAY_SLOT_LOGICAL_WIDTH"));
}
#[test]
fn grant_window_uses_narrow_single_flight_visibility_aware_polling() {
let ui = include_str!("../ui/App.tsx");
let grant_window = ui
.split("function GrantWindow()")
.nth(1)
.and_then(|source| source.split("function ActivityList(").next())
.expect("GrantWindow source");
assert!(grant_window.contains("call<PendingGrantContext>('get_pending_grant_context')"));
assert!(grant_window.contains("const incoming = next.grant"));
assert!(grant_window.contains("activeUrl ? displayHost(activeUrl)"));
assert!(!grant_window.contains("get_snapshot"));
assert!(!grant_window.contains("setInterval"));
assert!(grant_window.contains("if (running)"));
assert!(grant_window.contains("schedule(activeId.current ? 1000 : 5000)"));
assert!(grant_window.contains("document.visibilityState === 'visible'"));
assert!(grant_window.contains("document.addEventListener('visibilitychange', wake)"));
assert!(grant_window.contains("window.addEventListener('focus', wake)"));
}
#[test]
fn tray_subprocesses_are_bounded_cached_and_do_not_inherit_the_bun_relauncher() {
let native = include_str!("../src/main.rs");
let runner = include_str!("../src/bounded_process.rs");
assert!(native.contains("bounded_process::run(command, options)"));
assert!(native.contains("env(\"NODE_USE_SYSTEM_CA\", \"1\")"));
assert!(native.contains("build_snapshot_coalesced"));
assert!(native.contains("HARDWARE_AVAILABILITY_CACHE"));
assert!(native.contains("CLI_DETAILS_CACHE"));
assert!(!native.contains(".output()"));
assert!(!native.contains(".status()"));
assert!(runner.contains("JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE"));
assert!(runner.contains("max_capture_bytes"));
assert!(runner.contains("timed_out"));
}
#[test]
fn release_build_embeds_the_ui_instead_of_using_the_vite_server() {
let cargo = include_str!("../Cargo.toml");
@@ -214,7 +297,7 @@ fn management_window_keeps_the_reviewed_compact_geometry() {
assert!(config.contains("\"minWidth\": 340"));
assert!(config.contains("\"minHeight\": 460"));
assert!(config.contains("\"resizable\": false"));
assert_eq!(config.matches("\"alwaysOnTop\": true").count(), 2);
assert_eq!(config.matches("\"alwaysOnTop\": true").count(), 4);
assert!(!ui.contains("toggleMaximize"));
assert!(!ui.contains("data-tauri-drag-region"));
assert!(!capability.contains("allow-start-dragging"));
@@ -227,6 +310,10 @@ fn management_window_keeps_the_reviewed_compact_geometry() {
assert!(ui.contains("packet packet-outbound"));
assert!(ui.contains("packet packet-inbound"));
assert!(ui.contains("connectionTransition"));
assert!(ui.contains("daemon.reconnect_attempt"));
assert!(ui.contains("Retry now"));
assert!(ui.contains("Screenshot evidence"));
assert!(ui.contains("View larger"));
assert!(ui.contains("refreshInFlight"));
assert!(ui.contains("Starting daemon and opening relay tunnel"));
assert!(ui.contains("Stopping the local daemon"));
+318 -39
View File
@@ -6,10 +6,10 @@ import {
CircleHelp, Clock3, Download, ExternalLink, Eye, FileText, FolderOpen, Home, Info, Laptop, Link2,
Copy, LoaderCircle, LogOut, Monitor, MousePointer2, Power, Radio, RefreshCw, Server,
Settings, ShieldCheck, TerminalSquare, Trash2, Unplug, UserRoundX, X, Usb,
LockKeyhole, SlidersHorizontal, Mic, Video
LockKeyhole, SlidersHorizontal, Mic, Video, MousePointerClick, Maximize2, RotateCcw
} from 'lucide-react'
import logo from '../icons/icon-256.png'
import type { AccessMode, Activity, AuthorizedClient, Capability, CapabilityMode, Host, PendingGrantRequest, Snapshot, UpdateReport } from './types'
import type { AccessMode, Activity, AuthorizedClient, Capability, CapabilityMode, CuaHealthStatus, CuaManagementStatus, Host, PendingGrantRequest, Snapshot, UpdateReport } from './types'
import { describeTransportSecurity } from '../../src/transportSecurity'
import { displayLabel as displayRouteLabel, inferEndpointRole } from '../../src/endpoint'
@@ -17,8 +17,12 @@ type Page = 'overview' | 'access' | 'capabilities' | 'hosts' | 'pair-host' | 'ho
type PendingAction = { type: 'access'; mode: AccessMode } | { type: 'capability'; capability: Capability; mode: CapabilityMode } | { type: 'revoke'; client: AuthorizedClient; remote: string } | { type: 'repair' | 'forget'; host: Host } | { type: 'clear-activity' } | null
type RouteTestResult = { label: string; url: string; reachable: boolean; elapsed_ms: number; encrypted: boolean; security: string; error?: string | null }
type RouteTestReport = { best?: RouteTestResult | null; routes?: RouteTestResult[] }
type PendingGrantContext = { grant: PendingGrantRequest | null; active_url?: string | null }
const isGrantWindow = '__TAURI_INTERNALS__' in window && getCurrentWindow().label === 'grant'
const windowLabel = '__TAURI_INTERNALS__' in window ? getCurrentWindow().label : 'main'
const isGrantWindow = windowLabel === 'grant'
const isNoticeWindow = windowLabel === 'notice'
const isEvidenceWindow = windowLabel === 'evidence'
const demo: Snapshot = {
hosts: [{ url: 'wss://home-hermes.local:8767', name: 'Docker-Server', server_version: '1.6.3', endpoint_role: 'tailscale', paired_at: 1786458000, is_active: true, access_mode: 'full-access', capabilities: { commands: 'allow', files: 'allow', screen_input: 'allow', usb: 'allow', microphone: 'allow', camera: 'allow' } }],
@@ -30,6 +34,11 @@ const demo: Snapshot = {
cli_version: '0.4.0-alpha.4',
cli_path: 'C:\\Program Files\\Hermes-Relay CLI\\hermes-relay.exe',
hardware_availability: { usb: true, adb: true, microphone: false, camera: false },
activity_screenshot_retention: { enabled: true, days: 7, count: 2, bytes: 842_000 },
computer_control_engine: {
selected: 'legacy', effective: 'legacy', available: false, state: 'not_installed',
foreground_escalation_enabled: false, message: 'CUA Driver is not installed. Legacy input remains active.'
},
pending_grants: [],
activity: [
{ ts: Date.now() - 110_000, tool: 'desktop_powershell', ok: true, summary: 'exit 0', request_detail: '{\n "script": "Get-Process | Select-Object -First 5"\n}', stdout: 'Handles NPM(K) PM(K) WS(K) CPU(s) Id ProcessName\n------- ------ ----- ----- ------ -- -----------\n 412 31 74248 98312 4.18 812 powershell' },
@@ -41,6 +50,7 @@ const demo: Snapshot = {
async function call<T>(command: string, args?: Record<string, unknown>): Promise<T> {
if (!('__TAURI_INTERNALS__' in window)) {
if (command === 'get_snapshot') return demo as T
if (command === 'get_pending_grant_context') return { grant: demo.pending_grants[0] ?? null, active_url: demo.active_url } as T
if (command === 'list_authorized_clients') return [
{ token_prefix: 'f83a21c4', device_name: 'WORKSTATION', last_seen: Math.floor(Date.now() / 1000), transport_hint: 'desktop', is_current: true, grants: { chat: null, tools: null } },
{ token_prefix: '9d210b7e', device_name: 'Pixel 10 Pro', last_seen: Math.floor(Date.now() / 1000) - 420, transport_hint: 'android', grants: { chat: null } }
@@ -48,6 +58,8 @@ async function call<T>(command: string, args?: Record<string, unknown>): Promise
if (command === 'check_desktop_update') return { current: '0.4.0-alpha.3', up_to_date: true, ahead_of_latest: true, latest_version: '0.4.0-alpha.2', installed: false, needs_restart: false } as T
if (command === 'install_desktop_update') return { current: '0.4.0-alpha.3', up_to_date: true, ahead_of_latest: false, installed: true, needs_restart: true } as T
if (command === 'test_host_route') return { best: { label: 'LAN', url: 'ws://172.16.24.250:8767', reachable: true, elapsed_ms: 36, encrypted: false, security: 'Unencrypted relay connection' }, routes: [] } as T
if (command === 'computer_cua_status') return { installed: false, stale_path_shim: false, compatible: false, compatibility_reason: 'CUA Driver is not installed', supported_range: { minimum: '0.19.3', maximum_exclusive: '0.20.0' } } as T
if (command === 'computer_cua_health') return { state: 'degraded', checkedAt: new Date().toISOString(), overall: 'degraded', reason: 'UI Automation desktop enumeration exceeded 2000ms.', temporaryWindowsCompatibility: true } as T
return undefined as T
}
return invoke<T>(command, args)
@@ -116,11 +128,56 @@ function activityName(tool: string): string {
'host.access': 'Changed host access', 'host.pair': 'Pair host',
'client.revoke': 'Deauthorized client', 'grant.resolve': 'Resolved access request',
'daemon.start': 'Connected daemon', 'daemon.stop': 'Disconnected daemon',
'daemon.reconnecting': 'Connection interrupted', 'daemon.reconnected': 'Tunnel restored',
'daemon.disconnected': 'Tunnel disconnected', 'daemon.auth_failed': 'Connection failed',
'daemon.restart': 'Restarted daemon', 'startup.change': 'Changed startup setting'
}
return names[tool] ?? tool.replace(/^desktop[._]/, '').replaceAll('_', ' ').replaceAll('.', ' ').replace(/\b\w/g, value => value.toUpperCase())
}
function isComputerControl(entry: Activity): boolean {
return entry.backend === 'cua' || entry.backend === 'legacy_compat' || entry.tool.startsWith('desktop_computer_')
}
function controlActionLabel(entry: Activity): string {
return (entry.action ?? activityName(entry.tool)).replaceAll('_', ' ').replace(/\b\w/g, value => value.toUpperCase())
}
function controlVerificationLabel(value?: string): string {
return value === 'snapshot_captured' ? 'Post-action snapshot captured'
: value === 'failed' ? 'Verification failed'
: value ? value.replaceAll('_', ' ') : 'Not reported'
}
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`
if (bytes < 1024 * 1024) return `${Math.round(bytes / 1024)} KB`
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
}
type ActivityStep = { title: string; detail: string; state: 'done' | 'failed' | 'pending' }
function activitySteps(entry: Activity): ActivityStep[] {
if (isComputerControl(entry)) return [
{ title: 'Authorized session', detail: entry.control_session_id ? 'Authenticated control session' : 'Authenticated by Hermes', state: 'done' },
{ title: controlActionLabel(entry), detail: `${entry.backend === 'cua' ? 'CUA structured engine' : 'Windows input · Compatibility'} · ${entry.dispatch ?? 'background'}`, state: entry.ok ? 'done' : 'failed' },
{ title: 'Verification', detail: controlVerificationLabel(entry.verification), state: entry.verification === 'failed' ? 'failed' : entry.verification ? 'done' : 'pending' }
]
const category = activityCategory(entry)
if (category === 'system') {
const reconnecting = entry.tool === 'daemon.reconnecting'
return [
{ title: 'Connection state changed', detail: entry.summary ?? activityName(entry.tool), state: entry.ok ? 'done' : 'failed' },
...(reconnecting ? [{ title: 'Automatic retry', detail: 'Relay transport is retrying with backoff', state: 'pending' as const }] : [])
]
}
return [
{ title: 'Request received', detail: entry.args_preview ?? 'Validated local request', state: 'done' },
{ title: activityName(entry.tool), detail: entry.aborted ? 'Stopped before completion' : entry.error ?? entry.summary ?? 'Local execution', state: entry.ok ? 'done' : 'failed' },
{ title: 'Result recorded', detail: entry.ok ? (isNonZeroExit(entry) ? `Process exited ${activityExitCode(entry)}` : 'Evidence saved to local activity') : 'Failure details recorded', state: entry.ok ? 'done' : 'failed' }
]
}
function age(ts?: number): string {
if (!ts) return 'Not seen yet'
const seconds = Math.max(0, Math.floor(Date.now() / 1000) - ts)
@@ -196,7 +253,54 @@ function hostAccessLabel(host: Host): string {
}
export default function App() {
return isGrantWindow ? <GrantWindow /> : <ManagementApp />
return isGrantWindow ? <GrantWindow /> : isNoticeWindow ? <ConnectionNoticeWindow /> : isEvidenceWindow ? <EvidenceWindow /> : <ManagementApp />
}
type ConnectionNotice = { tone: 'connected' | 'warning' | 'offline'; title: string; detail: string }
function ConnectionNoticeWindow() {
const [notice, setNotice] = useState<ConnectionNotice | null>(null)
const hideTimer = useRef<number | null>(null)
useEffect(() => {
const receive = (event: Event) => {
const detail = (event as CustomEvent<ConnectionNotice>).detail
setNotice(detail)
if (hideTimer.current) window.clearTimeout(hideTimer.current)
hideTimer.current = window.setTimeout(() => void getCurrentWindow().hide(), detail.tone === 'warning' ? 6500 : 4200)
}
window.addEventListener('hermes-connection-notice', receive)
return () => { window.removeEventListener('hermes-connection-notice', receive); if (hideTimer.current) window.clearTimeout(hideTimer.current) }
}, [])
if (!notice) return null
return <div className={`connection-notice-shell ${notice.tone}`}>
<section className="connection-notice-card" role="status" aria-live="polite">
<span className="connection-notice-icon">{notice.tone === 'connected' ? <Check /> : notice.tone === 'warning' ? <RotateCcw /> : <Unplug />}</span>
<span><small>Hermes-Relay tunnel</small><strong>{notice.title}</strong><p>{notice.detail}</p></span>
<button className="connection-notice-open" onClick={() => call('open_management_from_notice')}>Open</button>
<button className="connection-notice-close" aria-label="Dismiss" onClick={() => getCurrentWindow().hide()}><X /></button>
</section>
</div>
}
function EvidenceWindow() {
const [evidenceId, setEvidenceId] = useState<string | null>(null)
const [source, setSource] = useState<string | null>(null)
const [error, setError] = useState<string | null>(null)
useEffect(() => {
const receive = (event: Event) => {
const id = (event as CustomEvent<{ evidenceId: string }>).detail.evidenceId
setEvidenceId(id); setSource(null); setError(null)
void call<string>('get_activity_screenshot', { evidenceId: id }).then(setSource).catch(value => setError(String(value)))
}
const close = (event: KeyboardEvent) => { if (event.key === 'Escape') void getCurrentWindow().hide() }
window.addEventListener('hermes-screenshot-evidence', receive)
window.addEventListener('keydown', close)
return () => { window.removeEventListener('hermes-screenshot-evidence', receive); window.removeEventListener('keydown', close) }
}, [])
return <div className="evidence-shell">
<header><span><Eye /><strong>Screenshot evidence</strong><small>Stored locally with this activity event</small></span><button aria-label="Close screenshot" onClick={() => getCurrentWindow().hide()}><X /></button></header>
<main>{source ? <img src={source} alt="Retained desktop screenshot" /> : error ? <div className="evidence-error"><AlertTriangle /><strong>Screenshot unavailable</strong><small>{error}</small></div> : <div className="evidence-loading"><LoaderCircle className="spin" /><span>{evidenceId ? 'Loading screenshot…' : 'Preparing viewer…'}</span></div>}</main>
</div>
}
function ManagementApp() {
@@ -225,7 +329,7 @@ function ManagementApp() {
const refreshInFlight = useRef(false)
const refresh = useCallback(async () => {
if (refreshInFlight.current) return
if (refreshInFlight.current) return true
refreshInFlight.current = true
try {
const next = await call<Snapshot>('get_snapshot')
@@ -247,15 +351,60 @@ function ManagementApp() {
setSnapshot(next)
setSelectedUrl(current => current && next.hosts.some(h => h.url === current) ? current : next.active_url ?? next.hosts[0]?.url ?? null)
setError(null)
} catch (e) { setError(String(e)) }
return true
} catch (e) { setError(String(e)); return false }
finally { refreshInFlight.current = false }
}, [])
const daemonRetrying = Boolean(snapshot?.daemon.running && snapshot.daemon.state === 'reconnecting')
useEffect(() => {
refresh()
const timer = window.setInterval(refresh, connectionTransition ? 350 : 5000)
return () => window.clearInterval(timer)
}, [refresh, connectionTransition])
let disposed = false
let enabled = false
let running = false
let failures = 0
let timer: number | null = null
const stop = () => {
enabled = false
if (timer !== null) window.clearTimeout(timer)
timer = null
}
const schedule = (delay: number) => {
if (disposed || !enabled) return
if (timer !== null) window.clearTimeout(timer)
timer = window.setTimeout(poll, delay)
}
const poll = async () => {
if (disposed || !enabled || running) return
running = true
timer = null
const ok = await refresh()
failures = ok ? 0 : Math.min(failures + 1, 4)
running = false
const baseDelay = connectionTransition ? 350 : daemonRetrying ? 1000 : 5000
schedule(ok ? baseDelay : Math.min(30_000, baseDelay * (2 ** failures)))
}
const start = () => {
if (disposed) return
enabled = true
if (timer === null && !running) void poll()
}
const visibilityChanged = () => {
if (document.visibilityState === 'visible') start()
else stop()
}
window.addEventListener('hermes-show', start)
window.addEventListener('hermes-hide', stop)
document.addEventListener('visibilitychange', visibilityChanged)
void getCurrentWindow().isVisible().then(visible => { if (visible) start() })
return () => {
disposed = true
stop()
window.removeEventListener('hermes-show', start)
window.removeEventListener('hermes-hide', stop)
document.removeEventListener('visibilitychange', visibilityChanged)
}
}, [refresh, connectionTransition, daemonRetrying])
useEffect(() => {
const close = (event: MouseEvent) => {
@@ -266,7 +415,11 @@ function ManagementApp() {
}, [])
const host = useMemo(() => snapshot?.hosts.find(item => item.url === selectedUrl) ?? null, [snapshot, selectedUrl])
const connected = Boolean(snapshot?.daemon.running && snapshot.daemon.state === 'connected' && host && (snapshot.daemon.configured_url ?? snapshot.daemon.url) === host.url)
const daemonTargetsHost = Boolean(host && (snapshot?.daemon.configured_url ?? snapshot?.daemon.url) === host.url)
const daemonActive = Boolean(snapshot?.daemon.running && daemonTargetsHost)
const connected = Boolean(daemonActive && snapshot?.daemon.state === 'connected')
const reconnecting = Boolean(daemonActive && snapshot?.daemon.state === 'reconnecting')
const retrySeconds = reconnecting && snapshot?.daemon.retry_at ? Math.max(0, snapshot.daemon.retry_at - Math.floor(Date.now() / 1000)) : null
useEffect(() => {
if (page !== 'host-detail' || !detailUrl) return
@@ -287,8 +440,8 @@ function ManagementApp() {
async function changeConnection() {
if (busy) return
const command = connected ? 'disconnect_daemon' : 'connect_daemon'
const transition = connected ? 'disconnecting' : 'connecting'
const command = daemonActive ? 'disconnect_daemon' : 'connect_daemon'
const transition = daemonActive ? 'disconnecting' : 'connecting'
setBusy(command)
setConnectionTransition(transition)
setError(null)
@@ -303,6 +456,16 @@ function ManagementApp() {
}
}
async function retryConnection() {
if (busy) return
setBusy('restart_daemon')
setConnectionTransition('connecting')
setError(null)
try { await call('restart_daemon'); await refresh() }
catch (e) { setError(String(e)) }
finally { setConnectionTransition(null); setBusy(null) }
}
async function testRoute(remote: string) {
setBusy('test_host_route')
setRouteTest(null)
@@ -379,6 +542,10 @@ function ManagementApp() {
}, 145)
}, [])
const requestHide = useCallback(() => {
window.dispatchEvent(new Event('hermes-hide'))
}, [])
useEffect(() => {
const show = () => setWindowVisible(true)
const visibilityChanged = () => {
@@ -412,13 +579,13 @@ function ManagementApp() {
<div className="brand"><img src={logo} alt="" /><span>Hermes-Relay CLI UI</span></div>
<div className="window-controls">
<button aria-label="Help and About" title="Help and About" onClick={() => setPage('help')}><CircleHelp /></button>
<button aria-label="Hide window" onClick={hideWindow}><X /></button>
<button aria-label="Hide window" onClick={requestHide}><X /></button>
</div>
</header>
<main className="content" ref={contentRef}>
{page === 'overview' && <>
<section className={`connection-route ${connected ? 'online' : 'offline'} ${connectionTransition ?? ''} ${snapshot.daemon.active_route === 'plugin_proxy' ? 'secure-link' : ''}`} aria-busy={connectionTransition !== null}>
<section className={`connection-route ${connected ? 'online' : reconnecting ? 'retrying' : 'offline'} ${connectionTransition ?? ''} ${snapshot.daemon.active_route === 'plugin_proxy' ? 'secure-link' : ''}`} aria-busy={connectionTransition !== null || reconnecting}>
{snapshot.hosts.length === 0 ?
<button className="empty-pair" onClick={() => openPair()}><Link2 /><span><strong>Pair host</strong><small>Connect this PC to a Hermes instance</small></span><ChevronRight /></button> :
<div className="route-grid" ref={selectorRef}>
@@ -445,10 +612,11 @@ function ManagementApp() {
const activeRole = snapshot.daemon.active_route ?? host?.endpoint_role ?? inferEndpointRole(snapshot.daemon.url ?? host?.url ?? '')
const security = describeTransportSecurity(snapshot.daemon.url ?? host?.url ?? '', activeRole)
return <div className={`route-status ${connected && !security.encrypted ? 'insecure' : ''}`} aria-live="polite" aria-atomic="true">
<strong>{connectionTransition === 'connecting' ? 'Connecting' : connectionTransition === 'disconnecting' ? 'Disconnecting' : connected ? 'Connected' : 'Disconnected'}</strong>
<strong>{connectionTransition === 'connecting' ? 'Connecting' : connectionTransition === 'disconnecting' ? 'Disconnecting' : reconnecting ? 'Reconnecting' : connected ? 'Connected' : 'Disconnected'}</strong>
{connected && <button className={`route-badge ${security.kind}`} aria-expanded={routeDetailsOpen} onClick={() => setRouteDetailsOpen(open => !open)}><ShieldCheck />{displayRouteLabel(activeRole ?? 'custom')}<ChevronDown /></button>}
{connectionTransition && <small>{connectionTransition === 'connecting' ? 'Starting daemon and opening relay tunnel' : 'Closing relay tunnel'}</small>}
{!connected && !connectionTransition && <small>Relay connection offline</small>}
{reconnecting && !connectionTransition && <><small>Attempt {snapshot.daemon.reconnect_attempt ?? 1}{retrySeconds !== null ? ` · retry in ${retrySeconds}s` : ' · retry scheduled'}</small><button className="retry-now" disabled={busy !== null} onClick={() => void retryConnection()}><RefreshCw /> Retry now</button></>}
{!connected && !reconnecting && !connectionTransition && <small>{snapshot.daemon.last_error ?? 'Relay connection offline'}</small>}
{connected && routeDetailsOpen && <aside className="route-detail-card"><div><ShieldCheck /><span><strong>{displayRouteLabel(activeRole ?? 'custom')}</strong><small>{security.detail}</small></span></div><dl><div><dt>Security</dt><dd>{security.label}</dd></div><div><dt>Endpoint</dt><dd title={snapshot.daemon.url ?? undefined}>{snapshot.daemon.url ?? 'Not reported'}</dd></div></dl><button className="route-test-button" disabled={busy === 'test_host_route'} onClick={() => host && testRoute(host.url)}>{busy === 'test_host_route' ? <LoaderCircle className="spin" /> : routeTest ? <RefreshCw /> : <ActivityIcon />}<span>{busy === 'test_host_route' ? <><strong>Testing connection…</strong><small>Checking every saved route</small></> : <><strong>{routeTest ? 'Test again' : 'Test connection'}</strong><small>Measure reachability and latency</small></>}</span></button>{routeTest && <div className={`route-test-result ${routeTest.best ? 'reachable' : 'unreachable'}`} aria-live="polite">{routeTest.best ? <><div className="route-test-summary"><span><Check /></span><strong>{routeTest.best.label} reachable</strong><em>{routeTest.best.elapsed_ms} ms</em></div><dl><div><dt>Protection</dt><dd className={routeTest.best.encrypted ? 'secure' : 'warning'}>{routeTest.best.security}</dd></div><div><dt>Tested endpoint</dt><dd title={routeTest.best.url}>{routeTest.best.url}</dd></div></dl><small>{Math.max(1, routeTest.routes?.length ?? 0)} saved route{(routeTest.routes?.length ?? 0) === 1 ? '' : 's'} checked</small></> : <div className="route-test-summary"><span><X /></span><strong>No route reachable</strong><em>Check host</em></div>}</div>}</aside>}
</div>
})()}
@@ -470,9 +638,9 @@ function ManagementApp() {
</button>
</section>}
<button className={`tunnel-button ${connectionTransition ? 'pending' : ''}`} disabled={busy !== null} aria-busy={connectionTransition !== null} onClick={() => void changeConnection()}>
{connectionTransition ? <LoaderCircle className="spin" /> : <Power />}
<span><strong>{connectionTransition === 'connecting' ? 'Connecting…' : connectionTransition === 'disconnecting' ? 'Disconnecting…' : connected ? 'Disconnect Tunnel' : 'Connect Tunnel'}</strong>{connectionTransition && <small>{connectionTransition === 'connecting' ? 'Waiting for the relay' : 'Stopping the local daemon'}</small>}</span>
<button className={`tunnel-button ${connectionTransition || reconnecting ? 'pending' : ''}`} disabled={busy !== null} aria-busy={connectionTransition !== null} onClick={() => void changeConnection()}>
{connectionTransition ? <LoaderCircle className="spin" /> : reconnecting ? <Unplug /> : <Power />}
<span><strong>{connectionTransition === 'connecting' ? 'Connecting…' : connectionTransition === 'disconnecting' ? 'Disconnecting…' : reconnecting ? 'Disconnect Tunnel' : connected ? 'Disconnect Tunnel' : 'Connect Tunnel'}</strong>{(connectionTransition || reconnecting) && <small>{connectionTransition === 'connecting' ? 'Waiting for the relay' : connectionTransition === 'disconnecting' ? 'Stopping the local daemon' : 'Automatic retry remains active'}</small>}</span>
</button>
<section className="activity-section">
@@ -489,7 +657,7 @@ function ManagementApp() {
{page === 'hosts' && <HostsPage hosts={snapshot.hosts} selected={host} onOpen={url => { setDetailUrl(url); setSelectedUrl(url); setPage('host-detail') }} onPair={() => openPair()} />}
{page === 'pair-host' && <PairHostPage initialUrl={pairInitialUrl} busy={busy === 'pair_host'} onBack={() => setPage('hosts')} onPair={pairHost} />}
{page === 'host-detail' && <HostDetailPage host={snapshot.hosts.find(item => item.url === detailUrl) ?? null} clients={clients} busy={busy !== null} onBack={() => setPage('hosts')} onConnect={connectHost} onRename={(remote, name) => action('rename_host', { remote, name })} onAccess={() => { setPolicyBack('host-detail'); setPage('access') }} onCapabilities={() => { setPolicyBack('host-detail'); setPage('capabilities') }} onRevoke={(remote, client) => setPending({ type: 'revoke', client, remote })} onRepair={host => setPending({ type: 'repair', host })} onForget={host => setPending({ type: 'forget', host })} />}
{page === 'settings' && <SettingsPage daemon={snapshot.daemon} startup={snapshot.startup_enabled} daemonAutostart={snapshot.daemon_autostart_enabled ?? false} activity={snapshot.activity} onAction={action} onStartup={value => action('set_startup', { enabled: value })} onDaemonAutostart={value => action('set_daemon_autostart', { enabled: value })} onHelp={() => setPage('help')} onViewActivity={() => { setActivityBack('settings'); setPage('activity') }} onOpenActivity={entry => { setSelectedActivity(entry); setActivityDetailBack('settings'); setPage('activity-detail') }} />}
{page === 'settings' && <SettingsPage daemon={snapshot.daemon} computerControl={snapshot.computer_control_engine ?? null} startup={snapshot.startup_enabled} daemonAutostart={snapshot.daemon_autostart_enabled ?? false} activity={snapshot.activity} screenshotRetention={snapshot.activity_screenshot_retention} onAction={action} onStartup={value => action('set_startup', { enabled: value })} onDaemonAutostart={value => action('set_daemon_autostart', { enabled: value })} onHelp={() => setPage('help')} onViewActivity={() => { setActivityBack('settings'); setPage('activity') }} onOpenActivity={entry => { setSelectedActivity(entry); setActivityDetailBack('settings'); setPage('activity-detail') }} />}
{page === 'help' && <HelpPage snapshot={snapshot} host={host} onBack={() => { setSelectedUrl(snapshot.active_url ?? null); setPage('settings') }} onAction={action} />}
{page === 'activity' && <ActivityPage entries={snapshot.activity} host={host} onBack={() => setPage(activityBack)} onClear={() => setPending({ type: 'clear-activity' })} onOpen={entry => { setSelectedActivity(entry); setActivityDetailBack('activity'); setPage('activity-detail') }} />}
{page === 'activity-detail' && <ActivityDetailPage entry={selectedActivity} host={host} onBack={() => setPage(activityDetailBack)} />}
@@ -521,7 +689,7 @@ function ManagementApp() {
}
function GrantWindow() {
const [snapshot, setSnapshot] = useState<Snapshot | null>(null)
const [activeUrl, setActiveUrl] = useState<string | null>(null)
const [grant, setGrant] = useState<PendingGrantRequest | null>(null)
const [expanded, setExpanded] = useState(false)
const [visible, setVisible] = useState(false)
@@ -530,9 +698,9 @@ function GrantWindow() {
const refreshGrant = useCallback(async () => {
try {
const next = await call<Snapshot>('get_snapshot')
const incoming = next.pending_grants[0] ?? null
setSnapshot(next)
const next = await call<PendingGrantContext>('get_pending_grant_context')
const incoming = next.grant
setActiveUrl(next.active_url ?? null)
if (!incoming) {
if (activeId.current) {
activeId.current = null
@@ -554,9 +722,56 @@ function GrantWindow() {
}, [])
useEffect(() => {
void refreshGrant()
const timer = window.setInterval(refreshGrant, 1000)
return () => window.clearInterval(timer)
let stopped = false
let running = false
let rerunRequested = false
let timer: number | null = null
const schedule = (delay: number) => {
if (stopped) return
if (timer !== null) window.clearTimeout(timer)
timer = window.setTimeout(poll, delay)
}
const poll = async () => {
if (stopped) return
if (running) {
rerunRequested = true
return
}
running = true
if (timer !== null) {
window.clearTimeout(timer)
timer = null
}
try { await refreshGrant() }
finally {
running = false
if (stopped) return
if (rerunRequested) {
rerunRequested = false
schedule(0)
} else if (activeId.current || document.visibilityState === 'visible') {
schedule(activeId.current ? 1000 : 5000)
}
}
}
const wake = () => {
if (document.visibilityState === 'visible') void poll()
else if (!activeId.current && timer !== null) {
window.clearTimeout(timer)
timer = null
}
}
document.addEventListener('visibilitychange', wake)
window.addEventListener('focus', wake)
void poll()
return () => {
stopped = true
if (timer !== null) window.clearTimeout(timer)
document.removeEventListener('visibilitychange', wake)
window.removeEventListener('focus', wake)
}
}, [refreshGrant])
async function toggleExpanded() {
@@ -580,9 +795,8 @@ function GrantWindow() {
}
}
if (!grant || !snapshot) return <div className="grant-shell" />
const hostName = snapshot.hosts.find(item => item.url === (snapshot.daemon.configured_url ?? snapshot.daemon.url))?.name
?? (snapshot.daemon.url ? displayHost(snapshot.daemon.url) : 'Connected host')
if (!grant) return <div className="grant-shell" />
const hostName = activeUrl ? displayHost(activeUrl) : 'Connected host'
const scope = formatGrantScope(grant.scope)
const action = grantAction(grant.scope)
const minutes = Math.max(1, Math.round(grant.duration_seconds / 60))
@@ -623,12 +837,14 @@ function ActivityList({ entries, host, onOpen }: { entries: Activity[]; host: Ho
const warning = isNonZeroExit(entry)
const key = entry.request_id ?? `${entry.ts}-${i}`
const Icon = category === 'command' ? TerminalSquare : category === 'files' ? FileText : category === 'screen' ? Eye : category === 'input' ? MousePointer2 : category === 'devices' ? Usb : category === 'system' ? LogOut : ActivityIcon
const detail = entry.error ?? entry.summary ?? (entry.aborted ? 'Request aborted' : 'Completed')
const computerControl = isComputerControl(entry)
const target = entry.target_app ?? entry.target_title
const detail = entry.error ?? (computerControl ? `${entry.backend === 'cua' ? 'CUA' : 'Compatibility'} · ${entry.dispatch ?? 'background'}${target ? ` · ${target}` : ''}` : entry.summary) ?? (entry.aborted ? 'Request aborted' : 'Completed')
const eventHost = entry.host_url ? displayHost(entry.host_url) : host?.name ?? 'Local daemon'
return <article className="activity-item" key={key}>
<button className="activity-row" disabled={!onOpen} onClick={() => onOpen?.(entry)}>
<span className={`activity-icon ${attention || warning ? 'amber' : category === 'system' ? 'green' : 'violet'}`}><Icon /></span>
<span className="activity-copy"><strong>{activityName(entry.tool)}</strong><small className={attention ? 'attention' : warning ? 'warning' : ''}>{detail} · {eventHost}</small></span>
<span className="activity-copy"><strong>{computerControl ? controlActionLabel(entry) : activityName(entry.tool)}</strong><small className={attention ? 'attention' : warning ? 'warning' : ''}>{detail} · {eventHost}</small></span>
<span className="activity-tail"><time>{formatTime(entry.ts)}</time>{onOpen && <ChevronRight />}</span>
</button>
</article>
@@ -663,22 +879,29 @@ function ActivityPage({ entries, host, onBack, onClear, onOpen }: { entries: Act
}
function ActivityDetailPage({ entry, host, onBack }: { entry: Activity | null; host: Host | null; onBack: () => void }) {
const [evidenceError, setEvidenceError] = useState<string | null>(null)
if (!entry) return <section className="page-panel"><button className="back-button" onClick={onBack}><ArrowLeft /> Back to Activity</button><div className="large-empty"><ActivityIcon /><h2>Event unavailable</h2></div></section>
const attention = needsAttention(entry)
const warning = isNonZeroExit(entry)
const status = entry.aborted ? 'Aborted' : !entry.ok ? 'Failed' : warning ? `Exit ${activityExitCode(entry)}` : 'Completed'
const eventHost = entry.host_url ? displayHost(entry.host_url) : host?.name ?? 'Local daemon'
const computerControl = isComputerControl(entry)
const steps = activitySteps(entry)
const blocks = [
['Request', entry.request_detail ?? entry.args_preview, entry.request_truncated],
['Standard output', entry.stdout, entry.stdout_truncated],
['Standard error', entry.stderr, entry.stderr_truncated],
['Result', entry.result_detail, entry.result_truncated],
['Error', entry.error, false]
['Result', entry.result_detail, entry.result_truncated]
] as const
return <section className="page-panel activity-detail-page">
<button className="back-button" onClick={onBack}><ArrowLeft /> Back to Activity</button>
<div className="activity-detail-title"><span className={`activity-icon ${attention || warning ? 'amber' : 'violet'}`}><TerminalSquare /></span><span><p>{activityCategory(entry)}</p><h1>{activityName(entry.tool)}</h1><small>{eventHost}</small></span></div>
<div className="activity-detail-title"><span className={`activity-icon ${attention || warning ? 'amber' : 'violet'}`}>{computerControl ? <MousePointerClick /> : <TerminalSquare />}</span><span><p>{computerControl ? 'Computer control' : activityCategory(entry)}</p><h1>{computerControl ? controlActionLabel(entry) : activityName(entry.tool)}</h1><small>{eventHost}</small></span></div>
<dl className="activity-detail-meta"><div><dt>Status</dt><dd className={attention ? 'attention' : warning ? 'warning' : 'success'}>{status}</dd></div><div><dt>When</dt><dd>{formatDateTime(entry.ts)}</dd></div><div><dt>Duration</dt><dd>{formatDuration(entry.duration_ms)}</dd></div></dl>
<section className="control-timeline" aria-label="Event timeline">{steps.map((step, index) => <div className={step.state} key={`${step.title}-${index}`}><i /><span><strong>{step.title}</strong><small>{step.detail}</small></span></div>)}</section>
{computerControl && (entry.target_app || entry.target_title || entry.target_pid || entry.target_window_id) && <dl className="control-target"><div><dt>Application</dt><dd>{entry.target_app ?? 'Not reported'}</dd></div><div><dt>Window</dt><dd title={entry.target_title}>{entry.target_title ?? 'Not reported'}</dd></div><div><dt>Target</dt><dd>{entry.target_pid ? `PID ${entry.target_pid}` : 'PID —'} · {entry.target_window_id ? `Window ${entry.target_window_id}` : 'Window —'}</dd></div></dl>}
{entry.error && <aside className="activity-error-callout" role="alert"><AlertTriangle /><span><strong>{entry.aborted ? 'Action stopped' : 'Action failed'}</strong><small>{entry.error}</small></span></aside>}
{entry.screenshot_evidence_id && <button className="screenshot-evidence-card" onClick={() => { setEvidenceError(null); void call('present_activity_screenshot', { evidenceId: entry.screenshot_evidence_id }).catch(error => setEvidenceError(String(error))) }}><span><Eye /><strong>Screenshot captured</strong><small>{entry.screenshot_width && entry.screenshot_height ? `${entry.screenshot_width} × ${entry.screenshot_height} PNG` : 'Retained local evidence'}</small></span><em><Maximize2 /> View larger</em></button>}
{evidenceError && <aside className="activity-error-callout"><AlertTriangle /><span><strong>Screenshot unavailable</strong><small>{evidenceError}</small></span></aside>}
<div className="activity-output-list">{blocks.filter(([, value]) => value).map(([label, value, truncated]) => <section key={label}><header><strong>{label}</strong>{truncated && <em>Truncated</em>}</header><pre>{value}</pre></section>)}</div>
{entry.request_id && <div className="activity-request-id">Request ID {entry.request_id}</div>}
</section>
@@ -783,10 +1006,32 @@ function HostDetailPage({ host, clients, busy, onBack, onConnect, onRename, onAc
</section>
}
function SettingsPage({ daemon, startup, daemonAutostart, activity, onAction, onStartup, onDaemonAutostart, onHelp, onViewActivity, onOpenActivity }: { daemon: Snapshot['daemon']; startup: boolean; daemonAutostart: boolean; activity: Activity[]; onAction: (name: string, args?: Record<string, unknown>) => Promise<unknown>; onStartup: (value: boolean) => void; onDaemonAutostart: (value: boolean) => void; onHelp: () => void; onViewActivity: () => void; onOpenActivity: (entry: Activity) => void }) {
function SettingsPage({ daemon, computerControl, startup, daemonAutostart, activity, screenshotRetention, onAction, onStartup, onDaemonAutostart, onHelp, onViewActivity, onOpenActivity }: { daemon: Snapshot['daemon']; computerControl: Snapshot['computer_control_engine']; startup: boolean; daemonAutostart: boolean; activity: Activity[]; screenshotRetention: Snapshot['activity_screenshot_retention']; onAction: (name: string, args?: Record<string, unknown>) => Promise<unknown>; onStartup: (value: boolean) => void; onDaemonAutostart: (value: boolean) => void; onHelp: () => void; onViewActivity: () => void; onOpenActivity: (entry: Activity) => void }) {
const [update, setUpdate] = useState<UpdateReport | null>(null)
const [updateBusy, setUpdateBusy] = useState<'check' | 'install' | null>(null)
const [updateError, setUpdateError] = useState<string | null>(null)
const [cuaManagement, setCuaManagement] = useState<CuaManagementStatus | null>(null)
const [cuaHealth, setCuaHealth] = useState<CuaHealthStatus | null>(null)
const [cuaBusy, setCuaBusy] = useState<'status' | 'health' | 'install' | 'check' | 'update' | null>(null)
const [cuaError, setCuaError] = useState<string | null>(null)
const cuaOperation = useCallback(async (operation: 'status' | 'install' | 'check' | 'update') => {
setCuaBusy(operation)
try {
setCuaManagement(await call<CuaManagementStatus>(operation === 'status' ? 'computer_cua_status' : operation === 'install' ? 'computer_cua_install' : operation === 'check' ? 'computer_cua_check_update' : 'computer_cua_update'))
setCuaError(null)
} catch (error) { setCuaError(String(error).replace(/^Error:\s*/i, '')) }
finally { setCuaBusy(null) }
}, [])
const recheckCuaHealth = useCallback(async () => {
setCuaBusy('health')
try {
setCuaHealth(await call<CuaHealthStatus>('computer_cua_health'))
setCuaError(null)
} catch (error) { setCuaError(String(error).replace(/^Error:\s*/i, '')) }
finally { setCuaBusy(null) }
}, [])
const checkUpdate = useCallback(async () => {
setUpdateBusy('check')
@@ -803,6 +1048,7 @@ function SettingsPage({ daemon, startup, daemonAutostart, activity, onAction, on
}, [])
useEffect(() => { void checkUpdate() }, [checkUpdate])
useEffect(() => { void cuaOperation('status') }, [cuaOperation])
const updateSummary = updateError
? updateError
@@ -816,12 +1062,44 @@ function SettingsPage({ daemon, startup, daemonAutostart, activity, onAction, on
? `${update.current} → ${update.latest_version} available`
: 'Check the desktop release channel.'
const cuaReady = computerControl?.available === true && computerControl.state === 'ready'
const engineState = computerControl?.state ?? 'not_installed'
const engineLabel = engineState === 'not_installed' ? 'Not installed' : engineState === 'incompatible' ? 'Incompatible' : engineState === 'degraded' ? 'Degraded' : engineState === 'ready' ? 'Ready' : 'Unavailable'
const engineDetail = computerControl?.message
?? (engineState === 'not_installed' ? 'CUA Driver is not installed. Windows input remains available for compatibility.'
: engineState === 'incompatible' ? 'The installed CUA Driver version is not compatible with this CLI.'
: engineState === 'degraded' ? 'CUA Driver reported a health problem. Windows compatibility input remains active.'
: engineState === 'ready' ? `CUA Driver ${computerControl?.version ?? ''} is compatible and healthy.`.trim()
: 'CUA Driver status could not be verified. Hermes uses the safe fallback.')
const effectiveEngine = computerControl?.effective === 'cua' ? 'CUA Driver' : 'Windows input'
const engineRole = computerControl?.effective === 'cua' ? 'Preferred structured engine' : 'Compatibility'
const activeSessions = computerControl?.active_sessions ?? 0
const activeBackend = computerControl?.active_backend === 'cua' ? 'CUA active'
: computerControl?.active_backend === 'legacy_compat' ? 'Compatibility active'
: computerControl?.active_backend === 'mixed' ? 'Mixed backends' : 'Idle'
const healthLabel = cuaHealth?.state === 'healthy' ? 'Healthy' : cuaHealth?.state === 'degraded' ? 'Degraded' : cuaHealth?.state === 'error' ? 'Check failed' : 'Not checked'
const healthDetail = cuaHealth?.reason
?? (cuaHealth?.state === 'healthy'
? 'The latest explicit accessibility check passed.'
: 'Optional diagnostic; it does not disable the runtime while the temporary Windows workaround is active.')
return <section className="page-panel settings-page"><div className="page-title"><div><p>Local management</p><h1>Settings</h1></div></div>
<div className="settings-group"><h2>Relay daemon</h2><div className="settings-card"><div className="setting-row"><span><strong>Daemon status</strong><small>{daemon.running ? `${daemon.state} · ${daemon.privilege ?? 'user'}` : 'Stopped'}</small></span><button className="compact-button" onClick={() => onAction('restart_daemon')}><RefreshCw /> Restart</button></div><div className="setting-row"><span><strong>{daemon.privilege === 'administrator' ? 'Administrator mode' : 'User mode'}</strong><small>{daemon.privilege === 'administrator' ? 'Remote actions inherit elevated rights.' : 'Recommended for normal operation.'}</small></span><button className={`compact-button privilege-action ${daemon.privilege === 'administrator' ? '' : 'admin-action'}`} onClick={() => onAction(daemon.privilege === 'administrator' ? 'restart_daemon_as_user' : 'restart_daemon_as_administrator')}>{daemon.privilege === 'administrator' ? 'Return to user mode' : 'Restart as Administrator…'}</button></div><label className="setting-row toggle-row"><span><strong>Start UI at sign-in</strong><small>Launch the tray after you sign in.</small></span><input type="checkbox" checked={startup} onChange={e => onStartup(e.target.checked)} /><i /></label><label className="setting-row toggle-row"><span><strong>Start daemon with UI</strong><small>Connect remote access when the tray starts.</small></span><input type="checkbox" checked={daemonAutostart} onChange={e => onDaemonAutostart(e.target.checked)} /><i /></label></div></div>
<div className="settings-group"><h2>CLI & diagnostics</h2><div className="settings-card quick-action-grid"><button onClick={() => onAction('open_terminal')}><TerminalSquare /><span>Open terminal</span></button><button onClick={() => onAction('open_cli_terminal')}><Bot /><span>Open Hermes CLI</span></button><button onClick={() => onAction('open_logs')}><FolderOpen /><span>View daemon log</span></button><button onClick={() => onAction('run_diagnostics')}><ActivityIcon /><span>Run diagnostics</span></button></div></div>
<div className="settings-group"><h2>Computer control</h2><div className={`settings-card engine-card engine-${engineState}`}>
<div className="engine-status"><span className="setting-icon"><MousePointerClick /></span><span><strong>{effectiveEngine}</strong><small>{engineRole} · {engineDetail}</small></span><em>{engineLabel}</em></div>
{cuaReady && <div className="engine-options">
<div className="engine-choice"><span><strong>Control engine</strong><small>CUA is preferred; Windows input is the compatibility fallback.</small></span><div role="radiogroup" aria-label="Computer control engine"><button role="radio" aria-checked={computerControl?.selected === 'cua'} className={computerControl?.selected === 'cua' ? 'active' : ''} onClick={() => onAction('set_computer_control_engine', { engine: 'cua' })}>CUA</button><button role="radio" aria-checked={computerControl?.selected !== 'cua'} className={computerControl?.selected !== 'cua' ? 'active' : ''} onClick={() => onAction('set_computer_control_engine', { engine: 'legacy' })}>Compatibility</button></div></div>
<div className="engine-live"><span><strong>{activeSessions}</strong><small>Active session{activeSessions === 1 ? '' : 's'}</small></span><em className={activeSessions ? 'active' : ''}><i /> {activeBackend}</em></div>
<label className="setting-row toggle-row"><span><strong>Animated agent cursor</strong><small>Labeled · smooth glide · click pulse. It does not move your physical mouse.</small></span><input type="checkbox" disabled={computerControl?.selected !== 'cua'} checked={computerControl?.selected === 'cua' && computerControl.cursor_enabled === true} onChange={e => onAction('set_cua_cursor_enabled', { enabled: e.target.checked })} /><i /></label>
<div className="setting-row background-only"><span><strong>Window interaction</strong><small>CUA actions stay in the background and never bring an app forward.</small></span><em>Background only</em></div>
<div className="setting-row cua-health"><span><strong>Accessibility health</strong><small>{healthDetail}</small></span><div><em className={`health-${cuaHealth?.state ?? 'unchecked'}`}>{healthLabel}</em><button className="compact-button" disabled={cuaBusy !== null} onClick={() => void recheckCuaHealth()}>{cuaBusy === 'health' ? <LoaderCircle className="spin" /> : <RefreshCw />} Recheck</button></div></div>
</div>}
<div className="cua-maintenance"><span><strong>{cuaManagement?.installed ? `CUA Driver ${cuaManagement.current_version ?? ''}`.trim() : 'CUA Driver'}</strong><small>{cuaError ?? cuaManagement?.update?.error ?? cuaManagement?.compatibility_reason ?? (cuaManagement?.update?.update_available ? `${cuaManagement.update.latest_version} available` : cuaManagement?.installed ? 'Installed from the verified upstream release.' : 'Install the verified compatible driver explicitly.')}</small></span><div>{!cuaManagement?.installed ? <button disabled={cuaBusy !== null} onClick={() => void cuaOperation('install')}>{cuaBusy === 'install' ? <LoaderCircle className="spin" /> : <Download />} Install</button> : cuaManagement.update?.update_available && cuaManagement.update.compatible ? <button disabled={cuaBusy !== null} onClick={() => void cuaOperation('update')}>{cuaBusy === 'update' ? <LoaderCircle className="spin" /> : <Download />} Update</button> : <button disabled={cuaBusy !== null} onClick={() => void cuaOperation('check')}>{cuaBusy === 'check' || cuaBusy === 'status' ? <LoaderCircle className="spin" /> : <RefreshCw />} Check</button>}</div></div>
</div><p className="group-help engine-help"><ShieldCheck /> CUA is the preferred structured engine. Hermes permissions, grants, audit, and emergency stop remain in control.</p></div>
<div className="settings-group"><h2>CLI & diagnostics</h2><div className="settings-card quick-action-grid"><button onClick={() => onAction('open_terminal')}><TerminalSquare /><span>Open terminal</span></button><button onClick={() => onAction('open_cli_terminal')}><Bot /><span>Open Hermes CLI</span></button><button onClick={() => onAction('open_logs')}><FolderOpen /><span>View daemon log</span></button><button onClick={() => onAction('open_tray_logs')}><FolderOpen /><span>View UI log</span></button><button onClick={() => onAction('run_diagnostics')}><ActivityIcon /><span>Run diagnostics</span></button></div></div>
<div className="settings-group"><h2>Updates</h2><div className={`settings-card update-card ${updateError ? 'error' : update?.ahead_of_latest ? 'ahead' : update?.up_to_date ? 'current' : ''}`}><div className="setting-row update-row"><span><strong>Hermes-Relay CLI UI</strong><small>{updateSummary}</small></span>{update && !update.up_to_date && !update.ahead_of_latest && !update.installed ? <button className="compact-button update-button" disabled={updateBusy !== null} onClick={installUpdate}>{updateBusy === 'install' ? <LoaderCircle className="spin" /> : <Download />} Install</button> : <button className="compact-button" disabled={updateBusy !== null} onClick={checkUpdate}>{updateBusy === 'check' ? <LoaderCircle className="spin" /> : <RefreshCw />} Check</button>}</div></div><p className="group-help update-help">Updates the management UI and CLI together, then restarts the tray automatically.</p></div>
<button className="about-link" onClick={onHelp}><span className="setting-icon"><Info /></span><span><strong>Help & About</strong><small>Versions, documentation and troubleshooting.</small></span><ChevronRight /></button>
<div className="settings-group"><div className="settings-group-heading"><h2>Activity</h2><button onClick={onViewActivity}>View all <ChevronRight /></button></div><p className="group-help">Recent remote actions recorded on this PC.</p><div className="settings-card padded"><ActivityList entries={activity.slice(-3).reverse()} host={null} onOpen={onOpenActivity} /></div></div>
<div className="settings-group"><div className="settings-group-heading"><h2>Activity</h2><button onClick={onViewActivity}>View all <ChevronRight /></button></div><p className="group-help">Recent remote actions recorded on this PC.</p><div className="settings-card activity-retention-card"><div className="setting-row"><span><strong>Screenshot evidence</strong><small>{screenshotRetention.count} retained · {formatBytes(screenshotRetention.bytes)} · stored only on this PC</small></span><div className="retention-options" role="radiogroup" aria-label="Screenshot evidence retention">{([{ label: 'Off', enabled: false, days: 7 }, { label: '1d', enabled: true, days: 1 }, { label: '7d', enabled: true, days: 7 }, { label: '30d', enabled: true, days: 30 }] as const).map(option => <button key={option.label} role="radio" aria-checked={screenshotRetention.enabled === option.enabled && (!option.enabled || screenshotRetention.days === option.days)} className={screenshotRetention.enabled === option.enabled && (!option.enabled || screenshotRetention.days === option.days) ? 'active' : ''} onClick={() => onAction('set_activity_screenshot_retention', { enabled: option.enabled, days: option.days })}>{option.label}</button>)}</div></div></div><div className="settings-card padded"><ActivityList entries={activity.slice(-3).reverse()} host={null} onOpen={onOpenActivity} /></div></div>
</section>
}
@@ -834,6 +1112,7 @@ function HelpPage({ snapshot, host, onBack, onAction }: { snapshot: Snapshot; ho
return <section className="page-panel help-page"><button className="back-button" onClick={onBack}><ArrowLeft /> Back to Settings</button><div className="about-hero"><img src={logo} alt="" /><span><p>Desktop companion</p><h1>Hermes-Relay CLI UI</h1><small>Compact control for this PC</small></span></div>
<div className="settings-group"><h2>About</h2><div className="settings-card about-facts"><dl><div><dt>UI version</dt><dd>{snapshot.ui_version ?? 'Unknown'}</dd></div><div><dt>CLI version</dt><dd>{snapshot.cli_version ?? 'Unknown'}</dd></div><div><dt>CLI path</dt><dd title={snapshot.cli_path ?? undefined}>{snapshot.cli_path ?? 'Not reported'}</dd></div><div><dt>Relay server</dt><dd>{host?.server_version ?? 'Not connected'}</dd></div></dl></div></div>
<div className="settings-group"><h2>Get help</h2><div className="settings-card management-links">{links.map(([label, url]) => <button key={url} onClick={() => onAction('open_external_url', { url })}><span><strong>{label}</strong></span><ExternalLink /></button>)}</div></div>
<div className="settings-group"><h2>Computer control</h2><div className="settings-card help-note"><ShieldCheck /><span><strong>CUA Driver is an optional control engine</strong><small>When compatible and healthy, it can use app-aware background control and separate animated agent cursors. These are visual overlays—not additional Windows hardware pointers. Foreground switching is not available; control remains background only. Full Access never bypasses targeting, sensitive-surface rules, audit, or emergency stop.</small></span></div></div>
<button className="diagnostic-action" onClick={() => onAction('run_diagnostics')}><ActivityIcon /><span><strong>Run diagnostics</strong><small>Check the daemon, installation and active relay.</small></span><ChevronRight /></button>
</section>
}
+127
View File
@@ -68,6 +68,8 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.route-endpoint small { font-size: 10px; font-weight: 550; white-space: nowrap; }
.route-link { position: relative; z-index: 0; display: block; height: 1px; border-top: 1.5px dashed #4fdf72; opacity: .9; }
.connection-route.offline .route-link { border-color: #66717d; opacity: .55; }
.connection-route.retrying .route-link { border-color: var(--amber); opacity: .7; animation: retry-link 1.4s ease-in-out infinite; }
@keyframes retry-link { 50% { opacity: .28; } }
.route-traffic { position: absolute; z-index: 1; pointer-events: none; left: 49px; right: 49px; top: 29px; height: 1px; overflow: hidden; }
.packet { position: absolute; top: -2px; left: 0; width: 9px; height: 5px; border-radius: 2px; background: #a3f9b2; box-shadow: 0 0 4px #72ed89, 0 0 10px #52df70; opacity: 0; }
.packet::after { content: ''; position: absolute; inset: 1px 2px; border-radius: 1px; background: #effff2; }
@@ -107,6 +109,10 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.connection-route.offline .route-status strong { color: #8d98a3; }
.connection-route.connecting .route-status strong { color: var(--violet); }
.connection-route.disconnecting .route-status strong { color: var(--amber); }
.connection-route.retrying .route-status > strong { color: var(--amber); }
.retry-now { border: 0; background: transparent; color: #d3a84f; padding: 2px 5px; display: inline-flex; align-items: center; gap: 4px; font-size: 9.5px; cursor: pointer; }
.retry-now:hover { color: #ffd77b; }
.retry-now svg { width: 11px; height: 11px; }
.connection-route.connecting .route-link { opacity: .9; animation-duration: .65s; }
.connection-route.disconnecting .route-link, .connection-route.disconnecting .route-traffic { opacity: .38; transition: opacity .18s ease; }
.route-status span, .route-status small { color: var(--muted); font-size: 11px; }
@@ -283,6 +289,34 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.activity-output-list header strong { font-size: 10px; font-weight: 550; }
.activity-output-list header em { color: var(--amber); font-size: 8px; font-style: normal; text-transform: uppercase; letter-spacing: .5px; }
.activity-output-list pre { max-height: 155px; margin: 0; padding: 9px; overflow: auto; color: #cdd2d7; font: 10px/1.45 'Cascadia Mono', Consolas, monospace; white-space: pre-wrap; overflow-wrap: anywhere; user-select: text; }
.control-timeline { margin: 0 0 10px; border: 1px solid var(--line); border-radius: 7px; background: #0d151c; padding: 8px 10px; display: grid; }
.control-timeline > div { min-height: 39px; position: relative; display: grid; grid-template-columns: 14px 1fr; gap: 8px; }
.control-timeline > div:not(:last-child)::after { content: ''; position: absolute; left: 5px; top: 15px; bottom: -2px; border-left: 1px dashed #4b5864; }
.control-timeline i { width: 11px; height: 11px; margin-top: 3px; border: 2px solid #53606c; border-radius: 50%; background: #0d151c; z-index: 1; }
.control-timeline .done i { border-color: var(--green); box-shadow: 0 0 6px #50e57755; }
.control-timeline .failed i { border-color: var(--amber); }
.control-timeline span { display: grid; align-content: start; gap: 1px; }
.control-timeline strong { font-size: 10.5px; font-weight: 560; }
.control-timeline small { color: var(--muted); font-size: 9.5px; overflow-wrap: anywhere; }
.control-timeline .pending i { border-color: #6d7782; border-style: dashed; }
.activity-error-callout { margin: 0 0 10px; padding: 9px 10px; border: 1px solid #77502c; border-radius: 7px; background: #2b1b0f; color: #f4c878; display: flex; gap: 8px; align-items: flex-start; }
.activity-error-callout > svg { width: 17px; flex: 0 0 17px; margin-top: 1px; }
.activity-error-callout span { min-width: 0; display: grid; gap: 2px; }
.activity-error-callout strong { font-size: 11px; }
.activity-error-callout small { color: #ddb986; font-size: 10px; line-height: 1.35; user-select: text; overflow-wrap: anywhere; }
.screenshot-evidence-card { width: 100%; min-height: 53px; margin: 0 0 10px; border: 1px solid #4c3d69; border-radius: 8px; background: linear-gradient(110deg, #171329, #101922); padding: 8px 10px; display: flex; justify-content: space-between; align-items: center; text-align: left; cursor: pointer; }
.screenshot-evidence-card:hover { border-color: #8e62d5; background: linear-gradient(110deg, #201638, #111b24); }
.screenshot-evidence-card > span { min-width: 0; display: grid; grid-template-columns: 25px 1fr; grid-template-rows: auto auto; column-gap: 7px; }
.screenshot-evidence-card > span svg { grid-row: 1 / 3; width: 20px; color: var(--violet); align-self: center; }
.screenshot-evidence-card strong { font-size: 11.5px; }
.screenshot-evidence-card small { color: var(--muted); font-size: 9.5px; }
.screenshot-evidence-card em { color: #c39aff; font-size: 10px; font-style: normal; display: flex; align-items: center; gap: 4px; white-space: nowrap; }
.screenshot-evidence-card em svg { width: 13px; }
.control-target { margin: 0 0 10px; display: grid; grid-template-columns: .8fr 1.2fr 1fr; border: 1px solid var(--line); border-radius: 7px; overflow: hidden; }
.control-target div { min-width: 0; padding: 7px 8px; border-right: 1px solid var(--line); display: grid; gap: 2px; }
.control-target div:last-child { border: 0; }
.control-target dt { color: var(--faint); font-size: 8px; text-transform: uppercase; letter-spacing: .55px; }
.control-target dd { margin: 0; color: #c8cdd2; font-size: 9.5px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.admin-warning { min-height: 44px; border: 1px solid #5b4615; border-radius: 6px; background: #3c2d0d3d; margin-top: 11px; padding: 0 12px; display: flex; align-items: center; gap: 11px; font-size: 12px; }
.admin-warning svg { color: var(--amber); width: 22px; }
.admin-warning span { flex: 1; }
@@ -445,6 +479,60 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.settings-group h2 span { color: var(--faint); font-weight: 400; margin-left: 4px; }
.group-help { margin: -2px 0 8px; }
.settings-card { border: 1px solid var(--line); border-radius: 8px; background: #111922a8; overflow: hidden; }
.engine-card { position: relative; }
.engine-card::before { content: ''; position: absolute; inset: 0 auto 0 0; width: 2px; background: var(--faint); }
.engine-card.engine-ready::before { background: var(--green); box-shadow: 0 0 10px #50e57788; }
.engine-card.engine-degraded::before, .engine-card.engine-incompatible::before { background: var(--amber); }
.engine-status { min-height: 67px; padding: 10px 11px 10px 13px; display: flex; align-items: center; gap: 9px; }
.engine-status > span:nth-child(2) { min-width: 0; flex: 1; display: grid; gap: 3px; }
.engine-status strong { font-size: 13px; font-weight: 570; }
.engine-status small { color: var(--muted); font-size: 10px; line-height: 1.35; }
.engine-status em { flex: 0 0 auto; align-self: flex-start; margin-top: 2px; border: 1px solid #46515d; border-radius: 10px; color: var(--muted); padding: 2px 6px; font-size: 8.5px; font-style: normal; text-transform: uppercase; letter-spacing: .5px; }
.engine-ready .engine-status em { border-color: #346443; color: var(--green); background: #163c2524; }
.engine-degraded .engine-status em, .engine-incompatible .engine-status em { border-color: #705528; color: var(--amber); background: #251e12; }
.engine-options { border-top: 1px solid var(--line); }
.engine-choice { min-height: 68px; padding: 9px 11px 9px 13px; display: flex; align-items: center; gap: 10px; border-bottom: 1px solid var(--line); }
.engine-choice > span { min-width: 0; flex: 1; display: grid; gap: 3px; }
.engine-choice strong { font-size: 12.5px; font-weight: 550; }
.engine-choice small { color: var(--muted); font-size: 9.5px; line-height: 1.25; }
.engine-choice > div { flex: 0 0 auto; display: grid; grid-template-columns: 1fr 1fr; border: 1px solid #46515d; border-radius: 6px; overflow: hidden; }
.engine-choice button { min-width: 54px; height: 28px; border: 0; border-right: 1px solid #46515d; background: #121b24; color: var(--muted); font-size: 10px; cursor: pointer; }
.engine-choice button:last-child { border-right: 0; }
.engine-choice button.active { background: #6137bd; color: #fff; }
.engine-live { min-height: 38px; padding: 6px 11px 6px 13px; border-bottom: 1px solid var(--line); display: flex; align-items: center; justify-content: space-between; }
.engine-live > span { display: flex; align-items: baseline; gap: 5px; }
.engine-live strong { font-size: 15px; color: #dfe4e9; }
.engine-live small { color: var(--muted); font-size: 9.5px; }
.engine-live em { color: var(--faint); font-size: 8.5px; font-style: normal; text-transform: uppercase; letter-spacing: .45px; }
.engine-live em i { display: inline-block; width: 5px; height: 5px; border-radius: 50%; background: var(--faint); margin-right: 3px; }
.engine-live em.active { color: var(--green); }
.engine-live em.active i { background: var(--green); box-shadow: 0 0 6px var(--green); }
.engine-options .setting-row { min-height: 64px; padding-left: 13px; }
.engine-options .setting-row strong { font-size: 12.5px; }
.engine-options .setting-row small { max-width: 250px; font-size: 9.5px; line-height: 1.3; }
.background-only { background: #5c41150f; }
.background-only em { flex: 0 0 auto; color: #b5bdc6; font-size: 9px; font-style: normal; text-transform: uppercase; letter-spacing: .45px; }
.cua-health { background: #101820; }
.cua-health > div { flex: 0 0 auto; display: flex; align-items: center; gap: 7px; }
.cua-health em { font-size: 8.5px; font-style: normal; text-transform: uppercase; letter-spacing: .4px; color: var(--faint); }
.cua-health em.health-healthy { color: var(--green); }
.cua-health em.health-degraded, .cua-health em.health-error { color: var(--amber); }
.cua-health .compact-button { min-width: 68px; height: 27px; padding: 4px 7px; font-size: 9px; }
.cua-health .compact-button svg { width: 11px; height: 11px; }
.cua-maintenance { min-height: 49px; padding: 7px 10px 7px 13px; border-top: 1px solid var(--line); display: flex; align-items: center; gap: 8px; }
.cua-maintenance > span { min-width: 0; flex: 1; display: grid; gap: 2px; }
.cua-maintenance strong { font-size: 10.5px; font-weight: 560; }
.cua-maintenance small { color: var(--muted); font-size: 9px; line-height: 1.25; }
.cua-maintenance button { min-width: 60px; height: 27px; border: 1px solid #4c5762; border-radius: 5px; background: #17212a; color: #dce1e6; display: flex; align-items: center; justify-content: center; gap: 4px; font-size: 9.5px; cursor: pointer; }
.cua-maintenance button:disabled { opacity: .55; cursor: wait; }
.cua-maintenance button svg { width: 12px; height: 12px; }
.engine-help { display: flex; align-items: flex-start; gap: 5px; margin: 6px 2px 0; color: var(--faint); font-size: 9.5px; line-height: 1.35; }
.engine-help svg { width: 13px; height: 13px; flex: 0 0 13px; color: var(--violet); }
.help-note { min-height: 92px; padding: 11px; display: flex; align-items: flex-start; gap: 9px; }
.help-note > svg { width: 18px; flex: 0 0 18px; color: var(--violet); }
.help-note span { display: grid; gap: 4px; }
.help-note strong { font-size: 12px; }
.help-note small { color: var(--muted); font-size: 10px; line-height: 1.45; }
.settings-card.padded { padding: 0 11px; }
.activity-panel { display: grid; gap: 8px; }
.activity-summary { display: grid; grid-template-columns: 1fr 1fr auto; gap: 7px; align-items: stretch; }
@@ -466,6 +554,11 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.clear-activity:disabled { opacity: .45; cursor: default; }
.activity-card { padding: 0; }
.activity-card .empty-state { min-height: 90px; }
.activity-retention-card { margin-bottom: 8px; }
.activity-retention-card .setting-row { align-items: center; }
.retention-options { flex: 0 0 auto; display: grid; grid-template-columns: repeat(4, auto); gap: 2px; padding: 2px; border: 1px solid #35414c; border-radius: 7px; background: #0b1218; }
.retention-options button { min-width: 29px; height: 24px; padding: 0 5px; border: 0; border-radius: 5px; background: transparent; color: var(--muted); font-size: 9.5px; cursor: pointer; }
.retention-options button.active { background: #7042d6; color: white; box-shadow: inset 0 1px #c9a5ff33; }
.settings-group-heading { display: flex; align-items: center; justify-content: space-between; }
.settings-group-heading h2 { margin-bottom: 7px; }
.settings-group-heading button { border: 0; background: transparent; color: var(--violet); display: flex; align-items: center; gap: 2px; padding: 0 0 7px; font-size: 11.5px; cursor: pointer; }
@@ -617,6 +710,40 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.grant-actions button:hover:not(:disabled) { filter: brightness(1.1); }
.grant-actions button:disabled { opacity: .6; cursor: wait; }
.connection-notice-shell { position: fixed; inset: 0; padding: 7px; display: grid; align-items: stretch; animation: notice-in .2s cubic-bezier(.2,.8,.2,1); }
@keyframes notice-in { from { opacity: 0; transform: translateY(12px) scale(.98); } }
.connection-notice-card { position: relative; border: 1px solid #3c4854; border-radius: 12px; background: #101922f5; box-shadow: 0 15px 38px #000b, inset 0 1px #ffffff08; padding: 13px 56px 12px 13px; display: grid; grid-template-columns: 38px minmax(0,1fr); gap: 10px; align-items: center; overflow: hidden; }
.connection-notice-card::before { content: ''; position: absolute; inset: 0 auto 0 0; width: 3px; background: var(--green); box-shadow: 0 0 13px var(--green); }
.connection-notice-shell.warning .connection-notice-card::before { background: var(--amber); box-shadow: 0 0 13px var(--amber); }
.connection-notice-shell.offline .connection-notice-card::before { background: #8c96a0; box-shadow: none; }
.connection-notice-icon { width: 36px; height: 36px; border-radius: 10px; background: #21432b; color: var(--green); display: grid; place-items: center; }
.connection-notice-shell.warning .connection-notice-icon { background: #493817; color: var(--amber); }
.connection-notice-shell.offline .connection-notice-icon { background: #29323b; color: #aeb7c0; }
.connection-notice-icon svg { width: 21px; }
.connection-notice-card > span:nth-child(2) { min-width: 0; display: grid; gap: 1px; }
.connection-notice-card small { color: var(--violet); font-size: 8.5px; text-transform: uppercase; letter-spacing: .8px; }
.connection-notice-card strong { font-size: 13px; }
.connection-notice-card p { margin: 1px 0 0; color: var(--muted); font-size: 10px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.connection-notice-open { position: absolute; right: 12px; bottom: 12px; border: 0; background: transparent; color: #c59aff; font-size: 9.5px; cursor: pointer; }
.connection-notice-close { position: absolute; right: 8px; top: 8px; width: 25px; height: 25px; border: 0; border-radius: 6px; background: transparent; color: #8f99a4; display: grid; place-items: center; cursor: pointer; }
.connection-notice-close:hover { color: white; background: #ffffff0c; }
.connection-notice-close svg { width: 14px; }
.evidence-shell { position: fixed; inset: 0; border: 1px solid #3a4652; border-radius: 11px; background: #0b1218; box-shadow: 0 24px 70px #000d; display: grid; grid-template-rows: 52px minmax(0,1fr); overflow: hidden; }
.evidence-shell > header { border-bottom: 1px solid var(--line); background: #111a23; padding: 0 13px 0 16px; display: flex; align-items: center; justify-content: space-between; }
.evidence-shell > header > span { display: grid; grid-template-columns: 28px auto; grid-template-rows: auto auto; column-gap: 8px; }
.evidence-shell > header svg { grid-row: 1 / 3; align-self: center; width: 20px; color: var(--violet); }
.evidence-shell > header strong { font-size: 13px; }
.evidence-shell > header small { color: var(--muted); font-size: 9.5px; }
.evidence-shell > header button { width: 34px; height: 34px; border: 0; border-radius: 7px; background: transparent; color: #aab2bb; display: grid; place-items: center; cursor: pointer; }
.evidence-shell > header button:hover { color: white; background: #c42b1c; }
.evidence-shell > main { min-width: 0; min-height: 0; padding: 14px; display: grid; place-items: center; background: radial-gradient(circle at center, #18232c, #080e13 68%); }
.evidence-shell img { max-width: 100%; max-height: 100%; object-fit: contain; border-radius: 5px; box-shadow: 0 8px 30px #000a; user-select: none; }
.evidence-loading, .evidence-error { color: var(--muted); display: grid; place-items: center; gap: 8px; text-align: center; }
.evidence-error svg { color: var(--amber); width: 30px; }
.evidence-error strong { color: var(--ink); font-size: 14px; }
.evidence-error small { max-width: 420px; font-size: 11px; }
@media (max-width: 410px) {
.content { padding-left: 20px; padding-right: 20px; }
.access-control button { gap: 5px; }

Some files were not shown because too many files have changed in this diff Show More