feat(desktop): enhance activity and control diagnostics

This commit is contained in:
Bailey Dixon
2026-08-14 11:30:36 -04:00
parent 8ae5b3fbc2
commit 45d631e7ac
28 changed files with 1027 additions and 104 deletions
+3
View File
@@ -8,12 +8,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
### Added
- **Desktop Activity now keeps inspectable local evidence.** Commands, files, devices, connection lifecycle, and computer control share a truthful event stepper with dedicated failure details; screenshot events can retain bounded local PNG evidence and open it in a larger borderless viewer. Settings controls retention as Off, 1 day, 7 days, or 30 days and shows local file usage.
- **Android can edit current Hermes profiles through the standard Gateway.** The Profile Inspector capability-gates `profiles.describe` and `profiles.configure`, keeps Relay-only memory editing and older-Hermes fallback intact, and reports partial section saves without discarding failed drafts.
- **Android sessions show their coding context when Hermes supplies it.** Session rows can display repository, Git branch, and the current state of the pull request created by that session while older hosts remain unchanged.
- Android Manage can now finish host-owned backup workflows, edit or remove learning nodes with explicit recovery guidance, configure and activate memory providers, and complete profile-scoped WhatsApp QR onboarding through the authenticated upstream Dashboard contracts.
### Fixed
- **Tunnel state stays responsive through interruption and retry.** The CLI UI distinguishes connected, reconnecting, and stopped states, exposes retry attempt/timing and a Retry now action, records connection failures and recovery in Activity, and shows compact connection cards only while the main UI is hidden.
- **Windows CUA readiness no longer depends on the flaky whole-desktop health scan.** Hermes-Relay verifies the canonical runtime, manifest, required tools, daemon, and safe permission mode before starting structured sessions, while accessibility health remains an explicit CLI/UI diagnostic that can be rechecked without forcing the compatibility backend. This temporary workaround is scoped to the upstream fixed-timeout issue and keeps individual actions fail-closed.
- **Android preserves authoritative Gateway outcomes.** Protected-file cards cannot offer forbidden persistent scopes, compression no-ops show the server result, bounded resume failures do not create context-free replacement sessions, and edit/regenerate retains durable row identities across consecutive rewinds.
- **Android routes and uploads against live upstream truth.** Multiplex API fallback trusts `served_profiles` instead of installed profiles, and generic documents carry the Gateway-issued `@file:` reference into ordinary and queued prompts.
- **Android clarify cards preserve upstream decision semantics.** Multi-select prompts keep independent selections and submit one exact list, while server expiry events—not an invented local deadline—retire unanswered cards.
+4
View File
@@ -1245,6 +1245,10 @@ When the answer becomes clearer, this section becomes either an ADR in `docs/dec
manifest identity and installer SHA-256; add Windows publisher verification
when upstream signs the installer. Keep raw CUA tools, configuration,
recording, replay, and JavaScript outside the remote agent surface.
Remove the temporary Windows readiness/health split once
[trycua/cua#3103](https://github.com/trycua/cua/issues/3103) ships in the
supported CUA range; restore a mandatory health gate only if the upstream
probe is bounded and cannot leave UI Automation falsely busy.
- **MediaProjection consent flow** — wired in MainActivity (2026-04-12), needs end-to-end test on a real device
- **WorkManager upgrade for auto-disable timer** — currently a coroutine `Job + delay()` in `AutoDisableWorker.kt`; documented at top of file. Upgrade when androidx.work joins the classpath
- **Wave 3 voice-bridge multi-turn confirmation** — currently a 5s TTS countdown with cancel; conversational confirmation is the follow-up
+14 -8
View File
@@ -80,11 +80,15 @@ aborts, and non-zero process exits; and keeps request context collapsed until
explicitly expanded. Events record handler duration and request ID where
available. The compact Overview still shows only the three newest events.
Settings also keeps activity compact: it previews the three newest events and
opens a dedicated Activity page. Selecting an event opens bounded request,
stdout, stderr, result, exit, timing, and truncation evidence; sensitive request
inputs are excluded. Handler failures and aborts are **Issues**; non-zero process exits are
opens a dedicated Activity page. Selecting an event opens a truthful lifecycle
stepper plus bounded request, stdout, stderr, result, exit, timing, and
truncation evidence; sensitive request inputs are excluded. Screenshot events
can retain an opaque local PNG outside the JSON log and open it in a larger
borderless viewer. **Settings → Activity → Screenshot evidence** controls this
as Off, 1 day, 7 days (default), or 30 days, shows local file count/usage, and
caps storage at 20 files and 10 MB per image. Handler failures and aborts are **Issues**; non-zero process exits are
shown separately because probing commands may legitimately use them. **Clear**
removes both current and rotated local audit history after confirmation.
removes current/rotated audit history and retained screenshot evidence after confirmation.
Clicking a card under **Hosts** opens that host's detail page; it does not change
the active connection. The detail page is the per-host hub for its local display
@@ -459,6 +463,7 @@ Inspect the detected runtime and selected/effective engine with:
```powershell
hermes-relay computer-use status --json
hermes-relay computer-use cua status
hermes-relay computer-use cua health # explicit accessibility recheck
hermes-relay computer-use cua check-update
hermes-relay computer-use cua install --yes
hermes-relay computer-use cua update --yes
@@ -470,8 +475,9 @@ hermes-relay computer-use cursor on
The management UI exposes the same controls under **Settings → Computer
control**. CUA is selected only when its canonical Windows package resolves from
`%USERPROFILE%\.cua-driver\packages\current\cua-driver.exe`, its supported
version and manifest agree, its required tools are present, its permission mode
is not unrestricted, and its live health report is healthy. Hermes ignores an unrelated
version and manifest agree, its required tools are present, and its permission mode
is not unrestricted. The live health report is an explicit diagnostic while the
temporary Windows workaround for trycua/cua#3103 is active. Hermes ignores an unrelated
or stale `cua-driver.exe` found earlier on `PATH`.
Background dispatch is mandatory. If an application cannot accept a
@@ -490,8 +496,8 @@ CUA Driver is not bundled with the Hermes-Relay installer. The explicit
GitHub release manifest and installer. Hermes verifies the manifest's
repository/product/version and installer SHA-256 before execution under a
sanitized child-process environment, then checks
the canonical binary's path, version, own manifest, tool surface, permission
mode, and health. This is release-metadata/checksum validation—not a Windows
the canonical binary's path, version, own manifest, tool surface, and permission
mode. Accessibility health can be rechecked separately. This is release-metadata/checksum validation—not a Windows
publisher signature. A native update newer than the supported `>=0.19.3,
<0.20.0` range is displayed but refused. There is no silent install/update,
and every child invocation forces CUA telemetry off. `hermes-relay update`
+34 -3
View File
@@ -5,7 +5,8 @@
// the audit flagged as the biggest desktop-tools transparency gap.
import type { ParsedArgs } from '../cli.js'
import { auditLogPath, readRecentAudit } from '../lib/auditLog.js'
import { auditLogPath, auditScreenshotEvidenceStatus, clearAuditScreenshotEvidence, pruneAuditScreenshotEvidence, readRecentAudit } from '../lib/auditLog.js'
import { readDesktopUseSettings, setActivityScreenshotRetention } from '../lib/desktopUseSettings.js'
import { renderTable } from '../lib/table.js'
import { SYMBOLS, theme as makeTheme } from '../lib/theme.js'
import { printUsage, type UsageSpec } from '../lib/usage.js'
@@ -13,10 +14,12 @@ import { printUsage, type UsageSpec } from '../lib/usage.js'
const AUDIT_USAGE: UsageSpec = {
name: 'audit',
summary: 'show recent desktop-tool activity the agent ran on this machine',
usage: ['audit [--limit <n>] [--json]'],
usage: ['audit [--limit <n>] [--json]', 'audit screenshots [on|off] [--days <1|7|30>] [--yes] [--json]'],
flags: [
{ flag: '--limit <n>', desc: 'How many recent entries to show (default 50)' },
{ flag: '--json', desc: 'Emit raw audit entries as JSON' }
{ flag: '--json', desc: 'Emit raw audit entries as JSON' },
{ flag: '--days <1|7|30>', desc: 'Local screenshot retention period' },
{ flag: '--yes', desc: 'Confirm a retention change' }
],
examples: ['hermes-relay audit', 'hermes-relay audit --limit 20']
}
@@ -36,6 +39,34 @@ export async function auditCommand(args: ParsedArgs): Promise<number> {
return 0
}
if (args.positional[0] === 'screenshots') {
const settings = await readDesktopUseSettings()
const mode = args.positional[1]
if (mode === 'on' || mode === 'off') {
if (args.flags.yes !== true) {
process.stderr.write('audit screenshots: retention changes require --yes\n')
return 1
}
const rawDays = typeof args.flags.days === 'string' ? Number(args.flags.days) : settings.activity_screenshot_retention_days
if (rawDays !== 1 && rawDays !== 7 && rawDays !== 30) {
process.stderr.write('audit screenshots: --days must be 1, 7, or 30\n')
return 1
}
await setActivityScreenshotRetention(mode === 'on', rawDays)
if (mode === 'off') await clearAuditScreenshotEvidence()
else await pruneAuditScreenshotEvidence(rawDays)
} else if (mode) {
process.stderr.write('audit screenshots: expected on or off\n')
return 1
}
const current = await readDesktopUseSettings()
const evidence = await auditScreenshotEvidenceStatus()
const result = { enabled: current.activity_screenshot_retention_enabled, days: current.activity_screenshot_retention_days, ...evidence }
if (args.flags.json) process.stdout.write(JSON.stringify(result, null, 2) + '\n')
else process.stdout.write(`Screenshot evidence: ${result.enabled ? `${result.days} days` : 'off'} · ${result.count} file${result.count === 1 ? '' : 's'}\n`)
return 0
}
const rawLimit = typeof args.flags.limit === 'string' ? parseInt(args.flags.limit, 10) : 50
const limit = Number.isFinite(rawLimit) && rawLimit > 0 ? rawLimit : 50
+16 -3
View File
@@ -29,7 +29,7 @@ const COMPUTER_USE_USAGE: UsageSpec = {
'computer-use cancel',
'computer-use engine <legacy|cua>',
'computer-use cursor <on|off>',
'computer-use cua <status|install|check-update|update> [--json] [--yes]'
'computer-use cua <status|health|install|check-update|update> [--json] [--yes]'
],
subcommands: [
{ verb: 'status', desc: 'Show preference, daemon state, active grant, and pending requests' },
@@ -38,7 +38,7 @@ const COMPUTER_USE_USAGE: UsageSpec = {
{ verb: 'cancel', desc: 'Cancel the active task-scoped desktop grant' },
{ verb: 'engine', desc: 'Choose legacy Windows input or a ready CUA Driver backend' },
{ verb: 'cursor', desc: 'Show or hide the CUA virtual agent cursor' },
{ verb: 'cua', desc: 'Manage the canonical CUA Driver package explicitly' }
{ verb: 'cua', desc: 'Manage the canonical CUA Driver package and recheck accessibility health' }
],
flags: [
{ flag: '--json', desc: 'Emit machine-readable status' },
@@ -48,6 +48,7 @@ const COMPUTER_USE_USAGE: UsageSpec = {
'hermes-relay computer-use status',
'hermes-relay computer-use enable',
'hermes-relay computer-use cua status',
'hermes-relay computer-use cua health',
'hermes-relay computer-use cua check-update',
'hermes-relay computer-use cua install --yes',
'hermes-relay computer-use cancel',
@@ -145,6 +146,18 @@ export async function computerUseCommand(args: ParsedArgs): Promise<number> {
const action = args.positional[1] ?? 'status'
const json = args.flags.json === true
try {
if (action === 'health') {
const health = await CuaDriverAdapter.healthStatus()
if (json) process.stdout.write(JSON.stringify(health, null, 2) + '\n')
else {
process.stdout.write(t.bold('CUA Driver accessibility health') + `\n state: ${health.state}\n`)
if (health.reason) process.stdout.write(t.warnLine(` ${health.reason}`) + '\n')
process.stdout.write(t.muted(' This diagnostic does not disable the runtime while the temporary Windows compatibility policy is active.') + '\n')
}
// The probe result is data, not command failure. Callers (including the
// tray) inspect state while still receiving the JSON for degradation.
return 0
}
const payload = action === 'status'
? await getCuaManagementStatus()
: action === 'check-update'
@@ -159,7 +172,7 @@ export async function computerUseCommand(args: ParsedArgs): Promise<number> {
: null
: undefined
if (payload === undefined) {
process.stderr.write(t.err('cua action must be status, install, check-update, or update') + '\n')
process.stderr.write(t.err('cua action must be status, health, install, check-update, or update') + '\n')
return 1
}
if (payload === null) {
+15 -3
View File
@@ -50,6 +50,7 @@ import {
type DaemonStatus
} from '../lib/daemonStatus.js'
import { rpcErrorMessage, asRpcResult } from '../lib/rpc.js'
import { appendAudit } from '../lib/auditLog.js'
import { effectiveHostAccessMode, effectiveHostCapabilityPolicies, getHostAccessMode, getHostCapabilityPolicies } from '../lib/hostAccessPolicy.js'
import { theme as makeTheme } from '../lib/theme.js'
import { printUsage, type UsageSpec } from '../lib/usage.js'
@@ -854,17 +855,26 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
? (info as { attempt?: number; delayMs?: number })
: {}
log.warn({ event: 'reconnecting', attempt: attempt ?? null, delay_ms: delayMs ?? null })
updateStatus({ state: 'reconnecting', last_event: 'reconnecting' })
const retryAt = nowSec() + Math.ceil((delayMs ?? 0) / 1000)
updateStatus({ state: 'reconnecting', last_event: 'reconnecting', reconnect_attempt: attempt ?? null, retry_at: retryAt, last_error: 'Relay connection interrupted' })
if ((attempt ?? 1) === 1) {
void appendAudit({
ts: Date.now(), kind: 'connection.state', tool: 'daemon.reconnecting', category: 'system', ok: false,
host_url: configuredUrl, summary: 'Automatic reconnect started', error: 'Relay connection interrupted'
})
}
})
relay.on('reconnected', () => {
log.info({ event: 'reconnected' })
updateStatus({ state: 'connected', last_event: 'reconnected' })
updateStatus({ state: 'connected', last_event: 'reconnected', reconnect_attempt: null, retry_at: null, last_error: null })
void appendAudit({ ts: Date.now(), kind: 'connection.state', tool: 'daemon.reconnected', category: 'system', ok: true, host_url: configuredUrl, summary: 'Relay tunnel restored' })
})
relay.on('exit', (code: unknown) => {
// Transport gave up (auth.fail, reconnect gate returned false, or
// reconnect attempts exhausted). Daemon exits non-zero so the
// service manager decides whether to restart.
log.error({ event: 'transport_exited', code: typeof code === 'number' ? code : null })
void appendAudit({ ts: Date.now(), kind: 'connection.state', tool: 'daemon.disconnected', category: 'system', ok: false, host_url: configuredUrl, summary: 'Relay transport stopped', error: 'Automatic reconnect stopped' })
// Defer exit so the log line flushes before the process dies.
setImmediate(() => process.exit(1))
})
@@ -874,6 +884,8 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
const outcome = await relay.whenAuthResolved()
if (!outcome.ok) {
log.error({ event: 'auth_failed', reason: outcome.reason })
updateStatus({ state: 'stopped', last_event: 'auth_failed', last_error: outcome.reason, reconnect_attempt: null, retry_at: null })
void appendAudit({ ts: Date.now(), kind: 'connection.state', tool: 'daemon.auth_failed', category: 'system', ok: false, host_url: configuredUrl, summary: 'Relay authentication failed', error: outcome.reason })
try {
relay.kill()
} catch {
@@ -887,7 +899,7 @@ export async function daemonCommand(args: ParsedArgs): Promise<number> {
server_version: relay.serverVersion ?? null,
transport: relay.authMeta?.transportHint ?? null
})
updateStatus({ state: 'connected', server_version: relay.serverVersion ?? null, last_event: 'authed' })
updateStatus({ state: 'connected', server_version: relay.serverVersion ?? null, last_event: 'authed', reconnect_attempt: null, retry_at: null, last_error: null })
// Signal downstream handlers that we're running headless. The router
// also checks this env var in its detectInteractive() fallback, so any
+106 -2
View File
@@ -9,7 +9,8 @@
//
// Best-effort by design: a logging failure must never break a tool dispatch.
import { appendFile, mkdir, readFile, rename, stat } from 'node:fs/promises'
import { appendFile, mkdir, readFile, readdir, rename, stat, unlink, writeFile } from 'node:fs/promises'
import { createHash } from 'node:crypto'
import { homedir } from 'node:os'
import { dirname, join } from 'node:path'
@@ -17,7 +18,7 @@ export interface AuditEntry {
/** Epoch milliseconds when the command completed. */
ts: number
/** Stable event type for consumers that do not want to infer it from fields. */
kind?: 'tool.completed'
kind?: 'tool.completed' | 'connection.state'
tool: string
category?: AuditCategory
ok: boolean
@@ -59,6 +60,11 @@ export interface AuditEntry {
/** Short success summary (path / exit code / first stdout line). */
summary?: string
error?: string
/** Opaque local evidence identifier. Screenshot pixels never enter JSONL. */
screenshot_evidence_id?: string
screenshot_mime_type?: 'image/png'
screenshot_width?: number
screenshot_height?: number
}
export type AuditCategory = 'command' | 'files' | 'screen' | 'input' | 'devices' | 'system' | 'other'
@@ -66,11 +72,109 @@ export type AuditCategory = 'command' | 'files' | 'screen' | 'input' | 'devices'
/** Rotate the log once it crosses ~1 MB, keeping a single `.1` backup. */
const MAX_BYTES = 1_000_000
const MAX_DETAIL_BYTES = 32_768
const MAX_SCREENSHOT_BYTES = 10_000_000
const MAX_SCREENSHOT_FILES = 20
export function auditLogPath(): string {
return join(homedir(), '.hermes', 'desktop-audit.jsonl')
}
export function auditEvidenceDirectory(): string {
return process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR ?? join(homedir(), '.hermes', 'activity-evidence')
}
type ScreenshotEvidence = Pick<AuditEntry, 'screenshot_evidence_id' | 'screenshot_mime_type' | 'screenshot_width' | 'screenshot_height'>
function screenshotPayload(result: unknown): { base64: string; width?: number; height?: number } | null {
if (!result || typeof result !== 'object' || Array.isArray(result)) return null
const record = result as Record<string, unknown>
const nested = record.after_screenshot
if (nested && typeof nested === 'object' && !Array.isArray(nested)) {
const found = screenshotPayload(nested)
if (found) return found
}
const base64 = typeof record.screenshot_base64 === 'string'
? record.screenshot_base64
: typeof record.bytes_base64 === 'string'
? record.bytes_base64
: null
if (!base64) return null
const display = record.display && typeof record.display === 'object' && !Array.isArray(record.display)
? record.display as Record<string, unknown>
: null
const width = typeof record.screenshot_width === 'number' ? record.screenshot_width
: typeof display?.width === 'number' ? display.width : undefined
const height = typeof record.screenshot_height === 'number' ? record.screenshot_height
: typeof display?.height === 'number' ? display.height : undefined
return { base64, width, height }
}
async function pruneScreenshotEvidence(directory: string, retentionDays: number): Promise<void> {
const now = Date.now()
const maxAgeMs = retentionDays * 24 * 60 * 60 * 1000
const records = await Promise.all((await readdir(directory, { withFileTypes: true }))
.filter(entry => entry.isFile() && /^[a-f0-9]{32}\.png$/.test(entry.name))
.map(async entry => ({ name: entry.name, stats: await stat(join(directory, entry.name)) })))
records.sort((left, right) => right.stats.mtimeMs - left.stats.mtimeMs)
await Promise.all(records
.filter((record, index) => index >= MAX_SCREENSHOT_FILES || now - record.stats.mtimeMs > maxAgeMs)
.map(record => unlink(join(directory, record.name)).catch(() => {})))
}
export async function pruneAuditScreenshotEvidence(retentionDays: 1 | 7 | 30): Promise<void> {
try {
await pruneScreenshotEvidence(auditEvidenceDirectory(), retentionDays)
} catch {
/* Missing or unreadable evidence is equivalent to an empty store. */
}
}
/** Retain a bounded, local screenshot for the activity evidence viewer. The
* audit log stores only an opaque identifier; clearing activity removes the
* evidence directory too. */
export async function persistAuditScreenshot(result: unknown, requestId: string, retentionDays = 7): Promise<ScreenshotEvidence> {
const payload = screenshotPayload(result)
if (!payload) return {}
let bytes: Buffer
try {
bytes = Buffer.from(payload.base64, 'base64')
} catch {
return {}
}
if (!bytes.length || bytes.length > MAX_SCREENSHOT_BYTES || bytes.subarray(0, 8).toString('hex') !== '89504e470d0a1a0a') return {}
const id = createHash('sha256').update(`${requestId}:${Date.now()}`).digest('hex').slice(0, 32)
const directory = auditEvidenceDirectory()
try {
await mkdir(directory, { recursive: true })
await writeFile(join(directory, `${id}.png`), bytes, { mode: 0o600, flag: 'wx' })
await pruneScreenshotEvidence(directory, retentionDays)
return {
screenshot_evidence_id: id,
screenshot_mime_type: 'image/png',
...(payload.width ? { screenshot_width: payload.width } : {}),
...(payload.height ? { screenshot_height: payload.height } : {})
}
} catch {
return {}
}
}
export async function clearAuditScreenshotEvidence(): Promise<void> {
const { rm } = await import('node:fs/promises')
await rm(auditEvidenceDirectory(), { recursive: true, force: true })
}
export async function auditScreenshotEvidenceStatus(): Promise<{ count: number; bytes: number }> {
try {
const records = await Promise.all((await readdir(auditEvidenceDirectory(), { withFileTypes: true }))
.filter(entry => entry.isFile() && /^[a-f0-9]{32}\.png$/.test(entry.name))
.map(entry => stat(join(auditEvidenceDirectory(), entry.name))))
return { count: records.length, bytes: records.reduce((total, value) => total + value.size, 0) }
} catch {
return { count: 0, bytes: 0 }
}
}
export async function appendAudit(entry: AuditEntry): Promise<void> {
const path = auditLogPath()
try {
+3
View File
@@ -45,6 +45,9 @@ export interface DaemonStatus {
advertised_tools?: number
voice_url?: string | null
last_event?: string
reconnect_attempt?: number | null
retry_at?: number | null
last_error?: string | null
username?: string
privilege?: ProcessPrivilege
computer_use_enabled?: boolean
+22 -1
View File
@@ -9,6 +9,8 @@ export interface DesktopUseSettings {
computer_use_enabled: boolean
computer_control_engine: 'legacy' | 'cua'
cua_cursor_enabled: boolean
activity_screenshot_retention_enabled: boolean
activity_screenshot_retention_days: 1 | 7 | 30
updated_at?: string
}
@@ -36,6 +38,8 @@ function normalizeSettings(value: unknown): DesktopUseSettings {
computer_use_enabled: raw.computer_use_enabled === true,
computer_control_engine: raw.computer_control_engine === 'legacy' ? 'legacy' : 'cua',
cua_cursor_enabled: raw.cua_cursor_enabled === true,
activity_screenshot_retention_enabled: raw.activity_screenshot_retention_enabled !== false,
activity_screenshot_retention_days: raw.activity_screenshot_retention_days === 1 || raw.activity_screenshot_retention_days === 30 ? raw.activity_screenshot_retention_days : 7,
updated_at: typeof raw.updated_at === 'string' ? raw.updated_at : undefined
}
}
@@ -44,7 +48,9 @@ function defaultSettings(): DesktopUseSettings {
return {
computer_use_enabled: false,
computer_control_engine: 'cua',
cua_cursor_enabled: false
cua_cursor_enabled: false,
activity_screenshot_retention_enabled: true,
activity_screenshot_retention_days: 7
}
}
@@ -102,6 +108,21 @@ export async function setComputerControlSettings(
return settings
}
export async function setActivityScreenshotRetention(
enabled: boolean,
days: 1 | 7 | 30,
filePath = desktopUseSettingsPath()
): Promise<DesktopUseSettings> {
const settings: DesktopUseSettings = {
...await readDesktopUseSettings(filePath),
activity_screenshot_retention_enabled: enabled,
activity_screenshot_retention_days: days,
updated_at: new Date().toISOString()
}
await writeJsonAtomic(filePath, settings)
return settings
}
export async function requestComputerGrantCancellation(
reason = 'cancelled from local desktop controls',
filePath = computerGrantCancellationPath()
+56 -6
View File
@@ -43,10 +43,18 @@ export interface CuaRuntimeStatus {
binaryPath?: string
binaryVersion?: string
permissionMode?: 'standard' | 'bounded'
health?: 'ok'
health?: 'ok' | 'not_checked'
reason?: string
}
export interface CuaHealthStatus {
state: 'healthy' | 'degraded' | 'error'
checkedAt: string
overall?: string
reason?: string
temporaryWindowsCompatibility: true
}
export interface CuaControlSessionIdentity {
controlSessionId: string
targetDeviceId: string
@@ -484,10 +492,13 @@ export class CuaDriverAdapter {
throw new CuaRuntimeError('CUA Driver permission mode is unavailable or unrestricted', 'incompatible')
}
const permissionMode = permissionMatch[1]!.toLowerCase() as 'standard' | 'bounded'
const health = parseJsonObject(await run(['call', 'health_report'], '{}'), 'CUA Driver health report') as CuaHealthReport
if (health.schema_version !== '1' || health.driver_version !== versionTuple.join('.') || health.overall !== 'ok') {
throw new CuaRuntimeError(`CUA Driver health is ${String(health.overall ?? 'unknown')}`, 'degraded')
}
// Temporary Windows compatibility policy for trycua/cua#3103. The global
// health_report performs a whole-desktop UIA walk with a fixed timeout and
// can poison the driver's busy flag after a false timeout. Runtime
// readiness is therefore based on the canonical binary, manifest, tool
// contract, daemon status, and safe permission mode. Individual structured
// actions still fail closed. Keep health_report as an explicit diagnostic
// via healthStatus(), and remove this split when upstream fixes #3103.
return new CuaDriverAdapter(
binaryPath,
versionTuple.join('.'),
@@ -507,7 +518,8 @@ export class CuaDriverAdapter {
binaryPath: adapter.binaryPath,
binaryVersion: adapter.binaryVersion,
permissionMode: adapter.permissionMode,
health: 'ok'
health: 'not_checked',
reason: 'Runtime checks passed; Windows accessibility health is checked separately.'
}
} catch (error) {
const reason = error instanceof Error ? error.message : String(error)
@@ -515,6 +527,44 @@ export class CuaDriverAdapter {
}
}
static async healthStatus(options: CuaRuntimeOptions = {}): Promise<CuaHealthStatus> {
const checkedAt = new Date().toISOString()
try {
const adapter = await CuaDriverAdapter.connect(options)
const runner = options.runner ?? new SpawnCuaProcessRunner()
const result = await runner.run(adapter.binaryPath, ['call', 'health_report'], {
stdin: '{}',
timeoutMs: DEFAULT_TIMEOUT_MS,
env: cuaDriverEnvironment()
})
if (result.exitCode !== 0) {
return {
state: 'error', checkedAt, temporaryWindowsCompatibility: true,
reason: `CUA Driver health probe exited ${result.exitCode}`
}
}
const health = parseJsonObject(result.stdout.trim(), 'CUA Driver health report') as CuaHealthReport
if (health.schema_version !== '1' || health.driver_version !== adapter.binaryVersion) {
return {
state: 'error', checkedAt, temporaryWindowsCompatibility: true,
reason: 'CUA Driver health report schema or version is incompatible'
}
}
const overall = String(health.overall ?? 'unknown')
return overall === 'ok'
? { state: 'healthy', checkedAt, overall, temporaryWindowsCompatibility: true }
: {
state: 'degraded', checkedAt, overall, temporaryWindowsCompatibility: true,
reason: `CUA Driver reported ${overall}; runtime remains available and actions still fail closed.`
}
} catch (error) {
return {
state: 'error', checkedAt, temporaryWindowsCompatibility: true,
reason: error instanceof Error ? error.message : String(error)
}
}
}
async openSession(identity: CuaControlSessionIdentity, signal?: AbortSignal, cursorEnabled = false): Promise<CuaControlSession> {
const id = derivedSessionId(identity)
if (this.sessions.has(id)) {
+10 -2
View File
@@ -30,11 +30,13 @@ import {
auditDetails,
categorizeTool,
previewArgs,
persistAuditScreenshot,
resultExitCode,
summarizeResult
} from '../lib/auditLog.js'
import { VERSION } from '../version.js'
import { desktopDeviceId } from '../deviceIdentity.js'
import { readDesktopUseSettingsSync } from '../lib/desktopUseSettings.js'
import {
getComputerGrantSummary,
getComputerUseRuntimeSummary,
@@ -471,7 +473,12 @@ export class DesktopToolRouter {
// work was completable.
this.sendResponse({ request_id, ok: true, result })
// Local audit trail — fire-and-forget so logging never delays the reply.
void appendAudit({
const retention = readDesktopUseSettingsSync()
const canRetainScreenshot = tool.includes('screenshot') || tool === 'desktop_computer_action'
const screenshotEvidence = retention.activity_screenshot_retention_enabled && canRetainScreenshot
? persistAuditScreenshot(result, request_id, retention.activity_screenshot_retention_days)
: Promise.resolve({})
void screenshotEvidence.then(screenshotEvidence => appendAudit({
ts: Date.now(),
kind: 'tool.completed',
tool,
@@ -488,11 +495,12 @@ export class DesktopToolRouter {
...(controlSession.runId ? { run_id: controlSession.runId } : {}),
...(controlSession.targetDeviceId ? { target_device_id: controlSession.targetDeviceId } : {}),
...computerAuditMetadata(result),
...screenshotEvidence,
...(!toolResultSucceeded(tool, result) && result && typeof result === 'object' && !Array.isArray(result) && typeof (result as Record<string, unknown>).code === 'string'
? { error: String((result as Record<string, unknown>).code).slice(0, 128) }
: {}),
...auditDetails(args, result, { redactComputerContent: tool.startsWith('desktop_computer_') })
})
}))
} catch (e) {
clearTimeout(timeoutTimer)
// Distinguish aborts (timeout or transport teardown) from genuine
+22 -1
View File
@@ -1,7 +1,10 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import { mkdtemp, readFile, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { auditDetails, categorizeTool, resultExitCode, summarizeResult } from '../src/lib/auditLog.js'
import { auditDetails, categorizeTool, persistAuditScreenshot, resultExitCode, summarizeResult } from '../src/lib/auditLog.js'
import { toolResultSucceeded } from '../src/tools/router.js'
test('audit events classify the activity surfaces used by the tray', () => {
@@ -56,3 +59,21 @@ test('semantic computer-control rejection is audited as failed without changing
assert.equal(toolResultSucceeded('desktop_computer_action', { ok: true }), true)
assert.equal(toolResultSucceeded('desktop_read_file', { ok: false }), true)
})
test('screenshot evidence is retained as a private opaque PNG instead of JSON content', async () => {
const directory = await mkdtemp(join(tmpdir(), 'hermes-activity-evidence-'))
const previous = process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR
process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR = directory
try {
const png = Buffer.from('89504e470d0a1a0a00000000', 'hex').toString('base64')
const evidence = await persistAuditScreenshot({ screenshot_base64: png, screenshot_width: 640, screenshot_height: 480 }, 'request-1', 7)
assert.match(evidence.screenshot_evidence_id ?? '', /^[a-f0-9]{32}$/)
assert.equal(evidence.screenshot_width, 640)
assert.equal((await readFile(join(directory, `${evidence.screenshot_evidence_id}.png`))).subarray(0, 8).toString('hex'), '89504e470d0a1a0a')
assert.equal(JSON.stringify(evidence).includes(png), false)
} finally {
if (previous === undefined) delete process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR
else process.env.HERMES_RELAY_ACTIVITY_EVIDENCE_DIR = previous
await rm(directory, { recursive: true, force: true })
}
})
+25 -7
View File
@@ -72,20 +72,21 @@ test('discovers only the canonical package/current executable and negotiates rea
assert.equal(adapter.binaryVersion, '0.19.3')
assert.equal(adapter.permissionMode, 'standard')
assert.equal(runner.calls[0]?.executable, install.binary)
assert.deepEqual(runner.calls.map(call => call.args.join(' ')).slice(0, 5), [
'--version', 'manifest --pretty', 'list-tools', 'status', 'call health_report'
assert.deepEqual(runner.calls.map(call => call.args.join(' ')).slice(0, 4), [
'--version', 'manifest --pretty', 'list-tools', 'status'
])
assert.equal(runner.calls.some(call => call.args.join(' ') === 'call health_report'), false)
} finally {
await install.cleanup()
}
})
test('fails closed for degraded health and unrestricted permission mode', async () => {
test('keeps runtime ready when global health is degraded but still rejects unrestricted permission mode', async () => {
const install = await fakeInstall()
try {
await assert.rejects(
CuaDriverAdapter.connect({ platform: 'win32', homeDir: install.home, runner: new FakeRunner(install.binary, 'degraded') }),
(error: unknown) => error instanceof CuaRuntimeError && error.code === 'degraded'
)
const adapter = await CuaDriverAdapter.connect({
platform: 'win32', homeDir: install.home, runner: new FakeRunner(install.binary, 'degraded')
})
assert.equal(adapter.binaryVersion, '0.19.3')
await assert.rejects(
CuaDriverAdapter.connect({ platform: 'win32', homeDir: install.home, runner: new FakeRunner(install.binary, 'ok', '0.19.3', 'unrestricted') }),
(error: unknown) => error instanceof CuaRuntimeError && error.code === 'incompatible'
@@ -95,6 +96,23 @@ test('fails closed for degraded health and unrestricted permission mode', async
}
})
test('explicit health recheck reports degradation without changing runtime readiness', async () => {
const install = await fakeInstall()
try {
const runner = new FakeRunner(install.binary, 'degraded')
const health = await CuaDriverAdapter.healthStatus({ platform: 'win32', homeDir: install.home, runner })
assert.equal(health.state, 'degraded')
assert.equal(health.overall, 'degraded')
assert.equal(health.temporaryWindowsCompatibility, true)
assert.equal(runner.calls.filter(call => call.args.join(' ') === 'call health_report').length, 1)
const runtime = await CuaDriverAdapter.status({ platform: 'win32', homeDir: install.home, runner })
assert.equal(runtime.ready, true)
assert.equal(runtime.health, 'not_checked')
} finally {
await install.cleanup()
}
})
test('uses a locally derived session and exposes only typed background actions', async () => {
const install = await fakeInstall()
try {
+7 -4
View File
@@ -13,7 +13,7 @@ import {
updateCuaDriver,
type CuaManagementFetch
} from '../src/tools/cuaManagement.js'
import type { CuaProcessResult, CuaProcessRunner } from '../src/tools/cuaDriver.js'
import { CuaDriverAdapter, type CuaProcessResult, type CuaProcessRunner } from '../src/tools/cuaDriver.js'
import { computerUseCommand } from '../src/commands/computerUse.js'
const ok = (stdout = ''): CuaProcessResult => ({ stdout, stderr: '', exitCode: 0 })
@@ -243,7 +243,7 @@ test('install rejects an installer whose bytes do not match release metadata', a
}
})
test('post-install degraded health fails the canonical runtime gate', async () => {
test('post-install runtime verification succeeds while explicit health remains degraded', async () => {
const root = await mkdtemp(join(tmpdir(), 'hermes-cua-degraded-'))
const script = Buffer.from('installer')
const checksum = createHash('sha256').update(script).digest('hex')
@@ -255,7 +255,7 @@ test('post-install degraded health fails the canonical runtime gate', async () =
await symlink(release, join(root, '.cua-driver', 'packages', 'current'), 'junction')
}, 'degraded')
try {
await assert.rejects(installCuaDriver({
const installed = await installCuaDriver({
platform: 'win32', homeDir: root, path: '', runner, systemRoot: 'C:\\Windows',
fetch: async url => ({
ok: true,
@@ -270,7 +270,10 @@ test('post-install degraded health fails the canonical runtime gate', async () =
}
}
})
}), /health is degraded/)
})
assert.equal(installed.operation?.runtime_verified, true)
const health = await CuaDriverAdapter.healthStatus({ platform: 'win32', homeDir: root, runner })
assert.equal(health.state, 'degraded')
} finally {
await rm(root, { recursive: true, force: true })
}
+18 -1
View File
@@ -9,6 +9,7 @@ import {
readDesktopUseSettings,
readDesktopUseSettingsSync,
requestComputerGrantCancellation,
setActivityScreenshotRetention,
setComputerControlSettings,
setDesktopUseEnabled
} from '../src/lib/desktopUseSettings.js'
@@ -42,7 +43,9 @@ test('computer control settings default fail-closed and survive desktop-use chan
assert.deepEqual(await readDesktopUseSettings(settingsPath), {
computer_use_enabled: false,
computer_control_engine: 'cua',
cua_cursor_enabled: false
cua_cursor_enabled: false,
activity_screenshot_retention_enabled: true,
activity_screenshot_retention_days: 7
})
await setComputerControlSettings({
computer_control_engine: 'cua',
@@ -58,6 +61,20 @@ test('computer control settings default fail-closed and survive desktop-use chan
}
})
test('screenshot evidence retention is explicit and survives other desktop setting changes', async () => {
const dir = await mkdtemp(join(tmpdir(), 'hermes-screenshot-retention-'))
const settingsPath = join(dir, 'desktop-settings.json')
try {
await setActivityScreenshotRetention(false, 30, settingsPath)
await setDesktopUseEnabled(true, settingsPath)
const settings = await readDesktopUseSettings(settingsPath)
assert.equal(settings.activity_screenshot_retention_enabled, false)
assert.equal(settings.activity_screenshot_retention_days, 30)
} finally {
await rm(dir, { recursive: true, force: true })
}
})
test('grant cancellation bridge is consumed exactly once', async () => {
const dir = await mkdtemp(join(tmpdir(), 'hermes-desktop-cancel-'))
const cancelPath = join(dir, 'cancel-active.json')
+1
View File
@@ -1234,6 +1234,7 @@ checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
name = "hermes-relay-tray"
version = "0.4.0-beta.1"
dependencies = [
"base64 0.22.1",
"serde",
"serde_json",
"tauri",
+1
View File
@@ -14,6 +14,7 @@ path = "src/main.rs"
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
base64 = "0.22"
tauri = { version = "2", features = ["tray-icon", "image-png"] }
[features]
+341 -16
View File
@@ -5,6 +5,7 @@ compile_error!("hermes-relay-tray is a Windows-only optional systray");
#[cfg(windows)]
mod app {
use base64::{engine::general_purpose::STANDARD as BASE64_STANDARD, Engine as _};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::{
@@ -225,6 +226,10 @@ mod app {
privilege: Option<String>,
username: Option<String>,
updated_at: Option<u64>,
last_event: Option<String>,
reconnect_attempt: Option<u64>,
retry_at: Option<u64>,
last_error: Option<String>,
}
fn stopped() -> String {
@@ -314,6 +319,14 @@ mod app {
result_truncated: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
error: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
screenshot_evidence_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
screenshot_mime_type: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
screenshot_width: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
screenshot_height: Option<u64>,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
@@ -332,6 +345,7 @@ mod app {
active_url: Option<String>,
daemon: DaemonStatus,
activity: Vec<Activity>,
activity_screenshot_retention: ActivityScreenshotRetention,
pending_grants: Vec<PendingGrantRequest>,
startup_enabled: bool,
daemon_autostart_enabled: bool,
@@ -342,6 +356,33 @@ mod app {
cli_path: Option<String>,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
struct ActivityScreenshotRetention {
#[serde(default)]
enabled: bool,
#[serde(default = "default_retention_days")]
days: u64,
#[serde(default)]
count: u64,
#[serde(default)]
bytes: u64,
}
fn default_retention_days() -> u64 {
7
}
impl Default for ActivityScreenshotRetention {
fn default() -> Self {
Self {
enabled: true,
days: 7,
count: 0,
bytes: 0,
}
}
}
#[derive(Debug, Clone, Deserialize, Serialize)]
struct ComputerControlEngine {
selected: String,
@@ -598,17 +639,36 @@ mod app {
let Ok(path) = home_dir().map(|h| h.join(".hermes").join("desktop-audit.jsonl")) else {
return Vec::new();
};
let Ok(text) = fs::read_to_string(path) else {
return Vec::new();
};
text.lines()
.rev()
.take(30)
.filter_map(|line| serde_json::from_str(line).ok())
.collect::<Vec<_>>()
let text = [path.with_extension("jsonl.1"), path]
.into_iter()
.rev()
.collect()
.filter_map(|file| fs::read_to_string(file).ok())
.collect::<Vec<_>>()
.join("\n");
let mut activity = text
.lines()
.filter_map(|line| serde_json::from_str(line).ok())
.collect::<Vec<Activity>>();
let evidence_directory = home_dir()
.ok()
.map(|home| home.join(".hermes").join("activity-evidence"));
for entry in &mut activity {
let available = entry
.screenshot_evidence_id
.as_ref()
.zip(evidence_directory.as_ref())
.is_some_and(|(id, directory)| directory.join(format!("{id}.png")).is_file());
if !available {
entry.screenshot_evidence_id = None;
entry.screenshot_mime_type = None;
entry.screenshot_width = None;
entry.screenshot_height = None;
}
}
activity.sort_by_key(|entry| entry.ts);
if activity.len() > 200 {
activity.drain(..activity.len() - 200);
}
activity
}
fn append_management_event(
@@ -656,6 +716,90 @@ mod app {
fs::remove_file(&backup)
.map_err(|error| format!("cannot remove rotated activity: {error}"))?;
}
let evidence = directory.join("activity-evidence");
if evidence.exists() {
fs::remove_dir_all(&evidence)
.map_err(|error| format!("cannot clear screenshot evidence: {error}"))?;
}
Ok(())
}
fn append_management_error(tool: &str, summary: &str, error: &str) {
let Ok(directory) = home_dir().map(|home| home.join(".hermes")) else {
return;
};
if fs::create_dir_all(&directory).is_err() {
return;
}
let timestamp = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_millis() as u64;
let event = serde_json::json!({
"ts": timestamp,
"kind": "management.completed",
"tool": tool,
"category": "system",
"ok": false,
"summary": summary,
"error": error.chars().take(512).collect::<String>(),
});
if let Ok(mut file) = OpenOptions::new()
.create(true)
.append(true)
.open(directory.join("desktop-audit.jsonl"))
{
let _ = writeln!(file, "{event}");
}
}
#[tauri::command]
fn get_activity_screenshot(evidence_id: String) -> Result<String, String> {
if evidence_id.len() != 32 || !evidence_id.bytes().all(|byte| byte.is_ascii_hexdigit()) {
return Err("invalid screenshot evidence identifier".to_string());
}
let path = home_dir()?
.join(".hermes")
.join("activity-evidence")
.join(format!("{}.png", evidence_id.to_ascii_lowercase()));
let bytes =
fs::read(path).map_err(|_| "screenshot evidence is no longer available".to_string())?;
if bytes.is_empty()
|| bytes.len() > 10_000_000
|| !bytes.starts_with(&[0x89, b'P', b'N', b'G', 0x0d, 0x0a, 0x1a, 0x0a])
{
return Err("screenshot evidence is invalid".to_string());
}
Ok(format!(
"data:image/png;base64,{}",
BASE64_STANDARD.encode(bytes)
))
}
#[tauri::command]
fn set_activity_screenshot_retention(enabled: bool, days: u64) -> Result<(), String> {
if !matches!(days, 1 | 7 | 30) {
return Err("screenshot retention must be 1, 7, or 30 days".to_string());
}
let days = days.to_string();
run_cli_checked(&[
"audit",
"screenshots",
if enabled { "on" } else { "off" },
"--days",
&days,
"--yes",
])?;
append_management_event(
"activity.retention",
if enabled {
"Screenshot evidence retention changed"
} else {
"Screenshot evidence retention disabled"
},
None,
None,
);
Ok(())
}
@@ -721,11 +865,16 @@ mod app {
.unwrap_or_default();
let (cli_version, cli_path) = cli_details();
let computer_control_engine = cached_computer_control_engine();
let activity_screenshot_retention = run_json(&["audit", "screenshots", "--json"])
.ok()
.and_then(|value| serde_json::from_value(value).ok())
.unwrap_or_default();
Ok(Snapshot {
hosts,
active_url: selected,
daemon,
activity: read_activity(),
activity_screenshot_retention,
pending_grants,
startup_enabled: startup_enabled(),
daemon_autostart_enabled: daemon_autostart_enabled(),
@@ -760,6 +909,15 @@ mod app {
.map_err(|error| format!("CUA status task failed: {error}"))?
}
#[tauri::command]
async fn computer_cua_health() -> Result<Value, String> {
tauri::async_runtime::spawn_blocking(|| {
run_json(&["computer-use", "cua", "health", "--json"])
})
.await
.map_err(|error| format!("CUA health task failed: {error}"))?
}
#[tauri::command]
async fn computer_cua_install() -> Result<Value, String> {
tauri::async_runtime::spawn_blocking(|| {
@@ -1298,7 +1456,10 @@ mod app {
#[tauri::command]
async fn connect_daemon() -> Result<(), String> {
tauri::async_runtime::spawn_blocking(|| {
run_cli_checked(&["daemon", "start"])?;
if let Err(error) = run_cli_checked(&["daemon", "start"]) {
append_management_error("daemon.start", "Relay daemon failed to connect", &error);
return Err(error);
}
append_management_event("daemon.start", "Relay daemon connected", None, None);
Ok(())
})
@@ -1308,7 +1469,10 @@ mod app {
#[tauri::command]
async fn disconnect_daemon() -> Result<(), String> {
tauri::async_runtime::spawn_blocking(|| {
run_cli_checked(&["daemon", "stop"])?;
if let Err(error) = run_cli_checked(&["daemon", "stop"]) {
append_management_error("daemon.stop", "Relay daemon failed to disconnect", &error);
return Err(error);
}
append_management_event("daemon.stop", "Relay daemon disconnected", None, None);
Ok(())
})
@@ -1316,10 +1480,17 @@ mod app {
.map_err(|error| format!("disconnect daemon task failed: {error}"))?
}
#[tauri::command]
fn restart_daemon() -> Result<(), String> {
run_cli_checked(&["daemon", "restart"])?;
append_management_event("daemon.restart", "Relay daemon restarted", None, None);
Ok(())
async fn restart_daemon() -> Result<(), String> {
tauri::async_runtime::spawn_blocking(|| {
if let Err(error) = run_cli_checked(&["daemon", "restart"]) {
append_management_error("daemon.restart", "Relay daemon failed to restart", &error);
return Err(error);
}
append_management_event("daemon.restart", "Relay daemon restarted", None, None);
Ok(())
})
.await
.map_err(|error| format!("restart daemon task failed: {error}"))?
}
#[tauri::command]
@@ -1656,6 +1827,154 @@ mod app {
present_grant_window_inner(&app, expanded, &tray_position)
}
#[derive(Serialize)]
struct ConnectionNotice<'a> {
tone: &'a str,
title: &'a str,
detail: String,
}
fn daemon_status_from_file() -> Option<DaemonStatus> {
let path = home_dir().ok()?.join(".hermes").join("daemon-status.json");
serde_json::from_slice(&fs::read(path).ok()?).ok()
}
fn present_connection_notice(app: &AppHandle, notice: ConnectionNotice<'_>) {
if app
.get_webview_window("main")
.is_some_and(|window| window.is_visible().unwrap_or(false))
{
if let Some(window) = app.get_webview_window("notice") {
let _ = window.hide();
}
return;
}
let Some(window) = app.get_webview_window("notice") else {
return;
};
let Some(monitor) = window
.current_monitor()
.ok()
.flatten()
.or_else(|| window.primary_monitor().ok().flatten())
else {
return;
};
let scale = monitor.scale_factor();
let size = PhysicalSize::new(
(360.0 * scale).round() as u32,
(126.0 * scale).round() as u32,
);
let _ = window.set_size(Size::Physical(size));
let _ = window.set_position(bottom_right_position(&monitor.work_area(), size, scale));
let Ok(payload) = serde_json::to_string(&notice) else {
return;
};
let _ = window.eval(&format!("window.dispatchEvent(new CustomEvent('hermes-connection-notice', {{ detail: {payload} }}))"));
let _ = window.show();
}
fn start_connection_watcher(app: AppHandle) {
thread::spawn(move || {
let mut previous = None::<String>;
loop {
let status = daemon_status_from_file();
let state = status
.as_ref()
.map(|value| value.state.clone())
.unwrap_or_else(|| "stopped".to_string());
if let Some(before) = previous.as_deref() {
if before != state {
let notice = match state.as_str() {
"connected" => Some(ConnectionNotice {
tone: "connected",
title: if before == "reconnecting" {
"Tunnel restored"
} else {
"Tunnel connected"
},
detail: status
.as_ref()
.and_then(|value| value.url.clone())
.unwrap_or_else(|| "Remote access is ready".to_string()),
}),
"reconnecting" => Some(ConnectionNotice {
tone: "warning",
title: "Connection interrupted",
detail: format!(
"Retrying automatically · attempt {}",
status
.as_ref()
.and_then(|value| value.reconnect_attempt)
.unwrap_or(1)
),
}),
"stopped"
if before == "connected"
|| before == "reconnecting"
|| before == "starting" =>
{
Some(ConnectionNotice {
tone: "offline",
title: "Tunnel disconnected",
detail: status
.as_ref()
.and_then(|value| value.last_error.clone())
.unwrap_or_else(|| "Remote access is offline".to_string()),
})
}
_ => None,
};
if let Some(notice) = notice {
let handle = app.clone();
let _ = app.run_on_main_thread(move || {
present_connection_notice(&handle, notice)
});
}
}
}
previous = Some(state);
thread::sleep(Duration::from_millis(500));
}
});
}
#[tauri::command]
fn open_management_from_notice(app: AppHandle, tray_position: State<'_, TrayPositionState>) {
if let Some(window) = app.get_webview_window("notice") {
let _ = window.hide();
}
let anchor = tray_position.0.lock().ok().and_then(|value| *value);
reveal_main_window(&app, anchor);
}
#[tauri::command]
fn present_activity_screenshot(app: AppHandle, evidence_id: String) -> Result<(), String> {
let _ = get_activity_screenshot(evidence_id.clone())?;
let window = app
.get_webview_window("evidence")
.ok_or_else(|| "screenshot viewer is unavailable".to_string())?;
let payload = serde_json::to_string(&serde_json::json!({ "evidenceId": evidence_id }))
.map_err(|error| error.to_string())?;
window.eval(&format!("window.dispatchEvent(new CustomEvent('hermes-screenshot-evidence', {{ detail: {payload} }}))")).map_err(|error| error.to_string())?;
let monitor = app
.get_webview_window("main")
.and_then(|main| main.current_monitor().ok().flatten())
.or_else(|| window.current_monitor().ok().flatten())
.or_else(|| window.primary_monitor().ok().flatten());
if let (Some(monitor), Ok(size)) = (monitor, window.outer_size()) {
let work = monitor.work_area();
let x =
work.position.x + ((work.size.width as i64 - size.width as i64).max(0) / 2) as i32;
let y = work.position.y
+ ((work.size.height as i64 - size.height as i64).max(0) / 2) as i32;
let _ = window.set_position(PhysicalPosition::new(x, y));
}
window.show().map_err(|error| error.to_string())?;
let _ = window.set_focus();
Ok(())
}
fn start_grant_watcher(app: AppHandle, tray_position: TrayPositionState) {
thread::spawn(move || {
let mut active_id = None::<String>;
@@ -1721,6 +2040,7 @@ mod app {
pair_host,
test_host_route,
open_management_from_grant,
open_management_from_notice,
forget_host,
connect_daemon,
disconnect_daemon,
@@ -1737,10 +2057,14 @@ mod app {
set_computer_control_engine,
set_cua_cursor_enabled,
computer_cua_status,
computer_cua_health,
computer_cua_install,
computer_cua_check_update,
computer_cua_update,
clear_activity,
get_activity_screenshot,
set_activity_screenshot_retention,
present_activity_screenshot,
present_grant_window
])
.setup(|app| {
@@ -1749,6 +2073,7 @@ mod app {
let tray_anchor = anchor.clone();
let menu_anchor = anchor.clone();
start_grant_watcher(app.handle().clone(), anchor);
start_connection_watcher(app.handle().clone());
start_activation_watcher(app.handle().clone());
let tray_actions = start_tray_action_worker(app.handle().clone());
let click_actions = tray_actions.clone();
+34
View File
@@ -47,6 +47,40 @@
"alwaysOnTop": true,
"skipTaskbar": true,
"center": false
},
{
"label": "notice",
"title": "Hermes-Relay connection status",
"width": 360,
"height": 126,
"resizable": false,
"fullscreen": false,
"decorations": false,
"transparent": true,
"backgroundColor": [0, 0, 0, 0],
"shadow": false,
"visible": false,
"alwaysOnTop": true,
"skipTaskbar": true,
"center": false
},
{
"label": "evidence",
"title": "Hermes-Relay screenshot evidence",
"width": 900,
"height": 650,
"minWidth": 560,
"minHeight": 420,
"resizable": true,
"fullscreen": false,
"decorations": false,
"transparent": true,
"backgroundColor": [0, 0, 0, 0],
"shadow": true,
"visible": false,
"alwaysOnTop": true,
"skipTaskbar": true,
"center": true
}
],
"security": {
+17 -4
View File
@@ -114,10 +114,11 @@ fn management_window_owns_the_expected_narrow_surfaces() {
"Preferred structured engine",
"Compatibility",
"Active session",
"Computer control timeline",
"Event timeline",
"Post-action snapshot captured",
"Authenticated control session",
"computer_cua_status",
"computer_cua_health",
"computer_cua_install",
"computer_cua_check_update",
"computer_cua_update",
@@ -151,7 +152,9 @@ fn grants_use_the_dedicated_card_and_host_changes_reconcile_daemon_truth() {
.and_then(|windows| windows.iter().find(|window| window["label"] == "grant"))
.expect("grant window configuration");
assert!(ui.contains("isGrantWindow ? <GrantWindow /> : <ManagementApp />"));
assert!(ui.contains("isGrantWindow ? <GrantWindow />"));
assert!(ui.contains("isNoticeWindow ? <ConnectionNoticeWindow />"));
assert!(ui.contains("isEvidenceWindow ? <EvidenceWindow />"));
assert!(ui.contains("present_grant_window"));
assert!(ui.contains("Remote access request"));
assert!(native.contains("start_grant_watcher"));
@@ -163,7 +166,8 @@ fn grants_use_the_dedicated_card_and_host_changes_reconcile_daemon_truth() {
serde_json::json!([0, 0, 0, 0])
);
assert_eq!(grant_window["shadow"], false);
assert!(ui.contains("(snapshot.daemon.configured_url ?? snapshot.daemon.url) === host.url"));
assert!(ui.contains("const daemonTargetsHost"));
assert!(ui.contains("(snapshot?.daemon.configured_url ?? snapshot?.daemon.url) === host.url"));
assert!(ui.contains("formatGrantScope(grant.scope)"));
assert!(ui.contains("grantAction(grant.scope)"));
assert!(ui.contains("Requested action"));
@@ -172,6 +176,10 @@ fn grants_use_the_dedicated_card_and_host_changes_reconcile_daemon_truth() {
assert!(native.contains("management.completed"));
assert!(native.contains("append_management_event"));
assert!(native.contains("fn clear_activity"));
assert!(native.contains("fn get_activity_screenshot"));
assert!(native.contains("fn set_activity_screenshot_retention"));
assert!(native.contains("fn start_connection_watcher"));
assert!(native.contains("fn present_connection_notice"));
assert!(native.contains("skip_serializing_if = \"Option::is_none\""));
assert!(native.contains("popup_position"));
assert!(native.contains("window.outer_size()"));
@@ -185,6 +193,7 @@ fn grants_use_the_dedicated_card_and_host_changes_reconcile_daemon_truth() {
assert!(native.contains("async fn install_desktop_update"));
for command in [
"async fn computer_cua_status",
"async fn computer_cua_health",
"async fn computer_cua_install",
"async fn computer_cua_check_update",
"async fn computer_cua_update",
@@ -249,7 +258,7 @@ fn management_window_keeps_the_reviewed_compact_geometry() {
assert!(config.contains("\"minWidth\": 340"));
assert!(config.contains("\"minHeight\": 460"));
assert!(config.contains("\"resizable\": false"));
assert_eq!(config.matches("\"alwaysOnTop\": true").count(), 2);
assert_eq!(config.matches("\"alwaysOnTop\": true").count(), 4);
assert!(!ui.contains("toggleMaximize"));
assert!(!ui.contains("data-tauri-drag-region"));
assert!(!capability.contains("allow-start-dragging"));
@@ -262,6 +271,10 @@ fn management_window_keeps_the_reviewed_compact_geometry() {
assert!(ui.contains("packet packet-outbound"));
assert!(ui.contains("packet packet-inbound"));
assert!(ui.contains("connectionTransition"));
assert!(ui.contains("daemon.reconnect_attempt"));
assert!(ui.contains("Retry now"));
assert!(ui.contains("Screenshot evidence"));
assert!(ui.contains("View larger"));
assert!(ui.contains("refreshInFlight"));
assert!(ui.contains("Starting daemon and opening relay tunnel"));
assert!(ui.contains("Stopping the local daemon"));
+142 -26
View File
@@ -6,10 +6,10 @@ import {
CircleHelp, Clock3, Download, ExternalLink, Eye, FileText, FolderOpen, Home, Info, Laptop, Link2,
Copy, LoaderCircle, LogOut, Monitor, MousePointer2, Power, Radio, RefreshCw, Server,
Settings, ShieldCheck, TerminalSquare, Trash2, Unplug, UserRoundX, X, Usb,
LockKeyhole, SlidersHorizontal, Mic, Video, MousePointerClick
LockKeyhole, SlidersHorizontal, Mic, Video, MousePointerClick, Maximize2, RotateCcw
} from 'lucide-react'
import logo from '../icons/icon-256.png'
import type { AccessMode, Activity, AuthorizedClient, Capability, CapabilityMode, CuaManagementStatus, Host, PendingGrantRequest, Snapshot, UpdateReport } from './types'
import type { AccessMode, Activity, AuthorizedClient, Capability, CapabilityMode, CuaHealthStatus, CuaManagementStatus, Host, PendingGrantRequest, Snapshot, UpdateReport } from './types'
import { describeTransportSecurity } from '../../src/transportSecurity'
import { displayLabel as displayRouteLabel, inferEndpointRole } from '../../src/endpoint'
@@ -18,7 +18,10 @@ type PendingAction = { type: 'access'; mode: AccessMode } | { type: 'capability'
type RouteTestResult = { label: string; url: string; reachable: boolean; elapsed_ms: number; encrypted: boolean; security: string; error?: string | null }
type RouteTestReport = { best?: RouteTestResult | null; routes?: RouteTestResult[] }
const isGrantWindow = '__TAURI_INTERNALS__' in window && getCurrentWindow().label === 'grant'
const windowLabel = '__TAURI_INTERNALS__' in window ? getCurrentWindow().label : 'main'
const isGrantWindow = windowLabel === 'grant'
const isNoticeWindow = windowLabel === 'notice'
const isEvidenceWindow = windowLabel === 'evidence'
const demo: Snapshot = {
hosts: [{ url: 'wss://home-hermes.local:8767', name: 'Docker-Server', server_version: '1.6.3', endpoint_role: 'tailscale', paired_at: 1786458000, is_active: true, access_mode: 'full-access', capabilities: { commands: 'allow', files: 'allow', screen_input: 'allow', usb: 'allow', microphone: 'allow', camera: 'allow' } }],
@@ -30,6 +33,7 @@ const demo: Snapshot = {
cli_version: '0.4.0-alpha.4',
cli_path: 'C:\\Program Files\\Hermes-Relay CLI\\hermes-relay.exe',
hardware_availability: { usb: true, adb: true, microphone: false, camera: false },
activity_screenshot_retention: { enabled: true, days: 7, count: 2, bytes: 842_000 },
computer_control_engine: {
selected: 'legacy', effective: 'legacy', available: false, state: 'not_installed',
foreground_escalation_enabled: false, message: 'CUA Driver is not installed. Legacy input remains active.'
@@ -53,6 +57,7 @@ async function call<T>(command: string, args?: Record<string, unknown>): Promise
if (command === 'install_desktop_update') return { current: '0.4.0-alpha.3', up_to_date: true, ahead_of_latest: false, installed: true, needs_restart: true } as T
if (command === 'test_host_route') return { best: { label: 'LAN', url: 'ws://172.16.24.250:8767', reachable: true, elapsed_ms: 36, encrypted: false, security: 'Unencrypted relay connection' }, routes: [] } as T
if (command === 'computer_cua_status') return { installed: false, stale_path_shim: false, compatible: false, compatibility_reason: 'CUA Driver is not installed', supported_range: { minimum: '0.19.3', maximum_exclusive: '0.20.0' } } as T
if (command === 'computer_cua_health') return { state: 'degraded', checkedAt: new Date().toISOString(), overall: 'degraded', reason: 'UI Automation desktop enumeration exceeded 2000ms.', temporaryWindowsCompatibility: true } as T
return undefined as T
}
return invoke<T>(command, args)
@@ -121,6 +126,8 @@ function activityName(tool: string): string {
'host.access': 'Changed host access', 'host.pair': 'Pair host',
'client.revoke': 'Deauthorized client', 'grant.resolve': 'Resolved access request',
'daemon.start': 'Connected daemon', 'daemon.stop': 'Disconnected daemon',
'daemon.reconnecting': 'Connection interrupted', 'daemon.reconnected': 'Tunnel restored',
'daemon.disconnected': 'Tunnel disconnected', 'daemon.auth_failed': 'Connection failed',
'daemon.restart': 'Restarted daemon', 'startup.change': 'Changed startup setting'
}
return names[tool] ?? tool.replace(/^desktop[._]/, '').replaceAll('_', ' ').replaceAll('.', ' ').replace(/\b\w/g, value => value.toUpperCase())
@@ -140,6 +147,35 @@ function controlVerificationLabel(value?: string): string {
: value ? value.replaceAll('_', ' ') : 'Not reported'
}
function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`
if (bytes < 1024 * 1024) return `${Math.round(bytes / 1024)} KB`
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`
}
type ActivityStep = { title: string; detail: string; state: 'done' | 'failed' | 'pending' }
function activitySteps(entry: Activity): ActivityStep[] {
if (isComputerControl(entry)) return [
{ title: 'Authorized session', detail: entry.control_session_id ? 'Authenticated control session' : 'Authenticated by Hermes', state: 'done' },
{ title: controlActionLabel(entry), detail: `${entry.backend === 'cua' ? 'CUA structured engine' : 'Windows input · Compatibility'} · ${entry.dispatch ?? 'background'}`, state: entry.ok ? 'done' : 'failed' },
{ title: 'Verification', detail: controlVerificationLabel(entry.verification), state: entry.verification === 'failed' ? 'failed' : entry.verification ? 'done' : 'pending' }
]
const category = activityCategory(entry)
if (category === 'system') {
const reconnecting = entry.tool === 'daemon.reconnecting'
return [
{ title: 'Connection state changed', detail: entry.summary ?? activityName(entry.tool), state: entry.ok ? 'done' : 'failed' },
...(reconnecting ? [{ title: 'Automatic retry', detail: 'Relay transport is retrying with backoff', state: 'pending' as const }] : [])
]
}
return [
{ title: 'Request received', detail: entry.args_preview ?? 'Validated local request', state: 'done' },
{ title: activityName(entry.tool), detail: entry.aborted ? 'Stopped before completion' : entry.error ?? entry.summary ?? 'Local execution', state: entry.ok ? 'done' : 'failed' },
{ title: 'Result recorded', detail: entry.ok ? (isNonZeroExit(entry) ? `Process exited ${activityExitCode(entry)}` : 'Evidence saved to local activity') : 'Failure details recorded', state: entry.ok ? 'done' : 'failed' }
]
}
function age(ts?: number): string {
if (!ts) return 'Not seen yet'
const seconds = Math.max(0, Math.floor(Date.now() / 1000) - ts)
@@ -215,7 +251,54 @@ function hostAccessLabel(host: Host): string {
}
export default function App() {
return isGrantWindow ? <GrantWindow /> : <ManagementApp />
return isGrantWindow ? <GrantWindow /> : isNoticeWindow ? <ConnectionNoticeWindow /> : isEvidenceWindow ? <EvidenceWindow /> : <ManagementApp />
}
type ConnectionNotice = { tone: 'connected' | 'warning' | 'offline'; title: string; detail: string }
function ConnectionNoticeWindow() {
const [notice, setNotice] = useState<ConnectionNotice | null>(null)
const hideTimer = useRef<number | null>(null)
useEffect(() => {
const receive = (event: Event) => {
const detail = (event as CustomEvent<ConnectionNotice>).detail
setNotice(detail)
if (hideTimer.current) window.clearTimeout(hideTimer.current)
hideTimer.current = window.setTimeout(() => void getCurrentWindow().hide(), detail.tone === 'warning' ? 6500 : 4200)
}
window.addEventListener('hermes-connection-notice', receive)
return () => { window.removeEventListener('hermes-connection-notice', receive); if (hideTimer.current) window.clearTimeout(hideTimer.current) }
}, [])
if (!notice) return null
return <div className={`connection-notice-shell ${notice.tone}`}>
<section className="connection-notice-card" role="status" aria-live="polite">
<span className="connection-notice-icon">{notice.tone === 'connected' ? <Check /> : notice.tone === 'warning' ? <RotateCcw /> : <Unplug />}</span>
<span><small>Hermes-Relay tunnel</small><strong>{notice.title}</strong><p>{notice.detail}</p></span>
<button className="connection-notice-open" onClick={() => call('open_management_from_notice')}>Open</button>
<button className="connection-notice-close" aria-label="Dismiss" onClick={() => getCurrentWindow().hide()}><X /></button>
</section>
</div>
}
function EvidenceWindow() {
const [evidenceId, setEvidenceId] = useState<string | null>(null)
const [source, setSource] = useState<string | null>(null)
const [error, setError] = useState<string | null>(null)
useEffect(() => {
const receive = (event: Event) => {
const id = (event as CustomEvent<{ evidenceId: string }>).detail.evidenceId
setEvidenceId(id); setSource(null); setError(null)
void call<string>('get_activity_screenshot', { evidenceId: id }).then(setSource).catch(value => setError(String(value)))
}
const close = (event: KeyboardEvent) => { if (event.key === 'Escape') void getCurrentWindow().hide() }
window.addEventListener('hermes-screenshot-evidence', receive)
window.addEventListener('keydown', close)
return () => { window.removeEventListener('hermes-screenshot-evidence', receive); window.removeEventListener('keydown', close) }
}, [])
return <div className="evidence-shell">
<header><span><Eye /><strong>Screenshot evidence</strong><small>Stored locally with this activity event</small></span><button aria-label="Close screenshot" onClick={() => getCurrentWindow().hide()}><X /></button></header>
<main>{source ? <img src={source} alt="Retained desktop screenshot" /> : error ? <div className="evidence-error"><AlertTriangle /><strong>Screenshot unavailable</strong><small>{error}</small></div> : <div className="evidence-loading"><LoaderCircle className="spin" /><span>{evidenceId ? 'Loading screenshot…' : 'Preparing viewer…'}</span></div>}</main>
</div>
}
function ManagementApp() {
@@ -270,11 +353,13 @@ function ManagementApp() {
finally { refreshInFlight.current = false }
}, [])
const daemonRetrying = Boolean(snapshot?.daemon.running && snapshot.daemon.state === 'reconnecting')
useEffect(() => {
refresh()
const timer = window.setInterval(refresh, connectionTransition ? 350 : 5000)
const timer = window.setInterval(refresh, connectionTransition ? 350 : daemonRetrying ? 1000 : 5000)
return () => window.clearInterval(timer)
}, [refresh, connectionTransition])
}, [refresh, connectionTransition, daemonRetrying])
useEffect(() => {
const close = (event: MouseEvent) => {
@@ -285,7 +370,11 @@ function ManagementApp() {
}, [])
const host = useMemo(() => snapshot?.hosts.find(item => item.url === selectedUrl) ?? null, [snapshot, selectedUrl])
const connected = Boolean(snapshot?.daemon.running && snapshot.daemon.state === 'connected' && host && (snapshot.daemon.configured_url ?? snapshot.daemon.url) === host.url)
const daemonTargetsHost = Boolean(host && (snapshot?.daemon.configured_url ?? snapshot?.daemon.url) === host.url)
const daemonActive = Boolean(snapshot?.daemon.running && daemonTargetsHost)
const connected = Boolean(daemonActive && snapshot?.daemon.state === 'connected')
const reconnecting = Boolean(daemonActive && snapshot?.daemon.state === 'reconnecting')
const retrySeconds = reconnecting && snapshot?.daemon.retry_at ? Math.max(0, snapshot.daemon.retry_at - Math.floor(Date.now() / 1000)) : null
useEffect(() => {
if (page !== 'host-detail' || !detailUrl) return
@@ -306,8 +395,8 @@ function ManagementApp() {
async function changeConnection() {
if (busy) return
const command = connected ? 'disconnect_daemon' : 'connect_daemon'
const transition = connected ? 'disconnecting' : 'connecting'
const command = daemonActive ? 'disconnect_daemon' : 'connect_daemon'
const transition = daemonActive ? 'disconnecting' : 'connecting'
setBusy(command)
setConnectionTransition(transition)
setError(null)
@@ -322,6 +411,16 @@ function ManagementApp() {
}
}
async function retryConnection() {
if (busy) return
setBusy('restart_daemon')
setConnectionTransition('connecting')
setError(null)
try { await call('restart_daemon'); await refresh() }
catch (e) { setError(String(e)) }
finally { setConnectionTransition(null); setBusy(null) }
}
async function testRoute(remote: string) {
setBusy('test_host_route')
setRouteTest(null)
@@ -437,7 +536,7 @@ function ManagementApp() {
<main className="content" ref={contentRef}>
{page === 'overview' && <>
<section className={`connection-route ${connected ? 'online' : 'offline'} ${connectionTransition ?? ''} ${snapshot.daemon.active_route === 'plugin_proxy' ? 'secure-link' : ''}`} aria-busy={connectionTransition !== null}>
<section className={`connection-route ${connected ? 'online' : reconnecting ? 'retrying' : 'offline'} ${connectionTransition ?? ''} ${snapshot.daemon.active_route === 'plugin_proxy' ? 'secure-link' : ''}`} aria-busy={connectionTransition !== null || reconnecting}>
{snapshot.hosts.length === 0 ?
<button className="empty-pair" onClick={() => openPair()}><Link2 /><span><strong>Pair host</strong><small>Connect this PC to a Hermes instance</small></span><ChevronRight /></button> :
<div className="route-grid" ref={selectorRef}>
@@ -464,10 +563,11 @@ function ManagementApp() {
const activeRole = snapshot.daemon.active_route ?? host?.endpoint_role ?? inferEndpointRole(snapshot.daemon.url ?? host?.url ?? '')
const security = describeTransportSecurity(snapshot.daemon.url ?? host?.url ?? '', activeRole)
return <div className={`route-status ${connected && !security.encrypted ? 'insecure' : ''}`} aria-live="polite" aria-atomic="true">
<strong>{connectionTransition === 'connecting' ? 'Connecting' : connectionTransition === 'disconnecting' ? 'Disconnecting' : connected ? 'Connected' : 'Disconnected'}</strong>
<strong>{connectionTransition === 'connecting' ? 'Connecting' : connectionTransition === 'disconnecting' ? 'Disconnecting' : reconnecting ? 'Reconnecting' : connected ? 'Connected' : 'Disconnected'}</strong>
{connected && <button className={`route-badge ${security.kind}`} aria-expanded={routeDetailsOpen} onClick={() => setRouteDetailsOpen(open => !open)}><ShieldCheck />{displayRouteLabel(activeRole ?? 'custom')}<ChevronDown /></button>}
{connectionTransition && <small>{connectionTransition === 'connecting' ? 'Starting daemon and opening relay tunnel' : 'Closing relay tunnel'}</small>}
{!connected && !connectionTransition && <small>Relay connection offline</small>}
{reconnecting && !connectionTransition && <><small>Attempt {snapshot.daemon.reconnect_attempt ?? 1}{retrySeconds !== null ? ` · retry in ${retrySeconds}s` : ' · retry scheduled'}</small><button className="retry-now" disabled={busy !== null} onClick={() => void retryConnection()}><RefreshCw /> Retry now</button></>}
{!connected && !reconnecting && !connectionTransition && <small>{snapshot.daemon.last_error ?? 'Relay connection offline'}</small>}
{connected && routeDetailsOpen && <aside className="route-detail-card"><div><ShieldCheck /><span><strong>{displayRouteLabel(activeRole ?? 'custom')}</strong><small>{security.detail}</small></span></div><dl><div><dt>Security</dt><dd>{security.label}</dd></div><div><dt>Endpoint</dt><dd title={snapshot.daemon.url ?? undefined}>{snapshot.daemon.url ?? 'Not reported'}</dd></div></dl><button className="route-test-button" disabled={busy === 'test_host_route'} onClick={() => host && testRoute(host.url)}>{busy === 'test_host_route' ? <LoaderCircle className="spin" /> : routeTest ? <RefreshCw /> : <ActivityIcon />}<span>{busy === 'test_host_route' ? <><strong>Testing connection…</strong><small>Checking every saved route</small></> : <><strong>{routeTest ? 'Test again' : 'Test connection'}</strong><small>Measure reachability and latency</small></>}</span></button>{routeTest && <div className={`route-test-result ${routeTest.best ? 'reachable' : 'unreachable'}`} aria-live="polite">{routeTest.best ? <><div className="route-test-summary"><span><Check /></span><strong>{routeTest.best.label} reachable</strong><em>{routeTest.best.elapsed_ms} ms</em></div><dl><div><dt>Protection</dt><dd className={routeTest.best.encrypted ? 'secure' : 'warning'}>{routeTest.best.security}</dd></div><div><dt>Tested endpoint</dt><dd title={routeTest.best.url}>{routeTest.best.url}</dd></div></dl><small>{Math.max(1, routeTest.routes?.length ?? 0)} saved route{(routeTest.routes?.length ?? 0) === 1 ? '' : 's'} checked</small></> : <div className="route-test-summary"><span><X /></span><strong>No route reachable</strong><em>Check host</em></div>}</div>}</aside>}
</div>
})()}
@@ -489,9 +589,9 @@ function ManagementApp() {
</button>
</section>}
<button className={`tunnel-button ${connectionTransition ? 'pending' : ''}`} disabled={busy !== null} aria-busy={connectionTransition !== null} onClick={() => void changeConnection()}>
{connectionTransition ? <LoaderCircle className="spin" /> : <Power />}
<span><strong>{connectionTransition === 'connecting' ? 'Connecting…' : connectionTransition === 'disconnecting' ? 'Disconnecting…' : connected ? 'Disconnect Tunnel' : 'Connect Tunnel'}</strong>{connectionTransition && <small>{connectionTransition === 'connecting' ? 'Waiting for the relay' : 'Stopping the local daemon'}</small>}</span>
<button className={`tunnel-button ${connectionTransition || reconnecting ? 'pending' : ''}`} disabled={busy !== null} aria-busy={connectionTransition !== null} onClick={() => void changeConnection()}>
{connectionTransition ? <LoaderCircle className="spin" /> : reconnecting ? <Unplug /> : <Power />}
<span><strong>{connectionTransition === 'connecting' ? 'Connecting…' : connectionTransition === 'disconnecting' ? 'Disconnecting…' : reconnecting ? 'Disconnect Tunnel' : connected ? 'Disconnect Tunnel' : 'Connect Tunnel'}</strong>{(connectionTransition || reconnecting) && <small>{connectionTransition === 'connecting' ? 'Waiting for the relay' : connectionTransition === 'disconnecting' ? 'Stopping the local daemon' : 'Automatic retry remains active'}</small>}</span>
</button>
<section className="activity-section">
@@ -508,7 +608,7 @@ function ManagementApp() {
{page === 'hosts' && <HostsPage hosts={snapshot.hosts} selected={host} onOpen={url => { setDetailUrl(url); setSelectedUrl(url); setPage('host-detail') }} onPair={() => openPair()} />}
{page === 'pair-host' && <PairHostPage initialUrl={pairInitialUrl} busy={busy === 'pair_host'} onBack={() => setPage('hosts')} onPair={pairHost} />}
{page === 'host-detail' && <HostDetailPage host={snapshot.hosts.find(item => item.url === detailUrl) ?? null} clients={clients} busy={busy !== null} onBack={() => setPage('hosts')} onConnect={connectHost} onRename={(remote, name) => action('rename_host', { remote, name })} onAccess={() => { setPolicyBack('host-detail'); setPage('access') }} onCapabilities={() => { setPolicyBack('host-detail'); setPage('capabilities') }} onRevoke={(remote, client) => setPending({ type: 'revoke', client, remote })} onRepair={host => setPending({ type: 'repair', host })} onForget={host => setPending({ type: 'forget', host })} />}
{page === 'settings' && <SettingsPage daemon={snapshot.daemon} computerControl={snapshot.computer_control_engine ?? null} startup={snapshot.startup_enabled} daemonAutostart={snapshot.daemon_autostart_enabled ?? false} activity={snapshot.activity} onAction={action} onStartup={value => action('set_startup', { enabled: value })} onDaemonAutostart={value => action('set_daemon_autostart', { enabled: value })} onHelp={() => setPage('help')} onViewActivity={() => { setActivityBack('settings'); setPage('activity') }} onOpenActivity={entry => { setSelectedActivity(entry); setActivityDetailBack('settings'); setPage('activity-detail') }} />}
{page === 'settings' && <SettingsPage daemon={snapshot.daemon} computerControl={snapshot.computer_control_engine ?? null} startup={snapshot.startup_enabled} daemonAutostart={snapshot.daemon_autostart_enabled ?? false} activity={snapshot.activity} screenshotRetention={snapshot.activity_screenshot_retention} onAction={action} onStartup={value => action('set_startup', { enabled: value })} onDaemonAutostart={value => action('set_daemon_autostart', { enabled: value })} onHelp={() => setPage('help')} onViewActivity={() => { setActivityBack('settings'); setPage('activity') }} onOpenActivity={entry => { setSelectedActivity(entry); setActivityDetailBack('settings'); setPage('activity-detail') }} />}
{page === 'help' && <HelpPage snapshot={snapshot} host={host} onBack={() => { setSelectedUrl(snapshot.active_url ?? null); setPage('settings') }} onAction={action} />}
{page === 'activity' && <ActivityPage entries={snapshot.activity} host={host} onBack={() => setPage(activityBack)} onClear={() => setPending({ type: 'clear-activity' })} onOpen={entry => { setSelectedActivity(entry); setActivityDetailBack('activity'); setPage('activity-detail') }} />}
{page === 'activity-detail' && <ActivityDetailPage entry={selectedActivity} host={host} onBack={() => setPage(activityDetailBack)} />}
@@ -684,29 +784,29 @@ function ActivityPage({ entries, host, onBack, onClear, onOpen }: { entries: Act
}
function ActivityDetailPage({ entry, host, onBack }: { entry: Activity | null; host: Host | null; onBack: () => void }) {
const [evidenceError, setEvidenceError] = useState<string | null>(null)
if (!entry) return <section className="page-panel"><button className="back-button" onClick={onBack}><ArrowLeft /> Back to Activity</button><div className="large-empty"><ActivityIcon /><h2>Event unavailable</h2></div></section>
const attention = needsAttention(entry)
const warning = isNonZeroExit(entry)
const status = entry.aborted ? 'Aborted' : !entry.ok ? 'Failed' : warning ? `Exit ${activityExitCode(entry)}` : 'Completed'
const eventHost = entry.host_url ? displayHost(entry.host_url) : host?.name ?? 'Local daemon'
const computerControl = isComputerControl(entry)
const steps = activitySteps(entry)
const blocks = [
['Request', entry.request_detail ?? entry.args_preview, entry.request_truncated],
['Standard output', entry.stdout, entry.stdout_truncated],
['Standard error', entry.stderr, entry.stderr_truncated],
['Result', entry.result_detail, entry.result_truncated],
['Error', entry.error, false]
['Result', entry.result_detail, entry.result_truncated]
] as const
return <section className="page-panel activity-detail-page">
<button className="back-button" onClick={onBack}><ArrowLeft /> Back to Activity</button>
<div className="activity-detail-title"><span className={`activity-icon ${attention || warning ? 'amber' : 'violet'}`}>{computerControl ? <MousePointerClick /> : <TerminalSquare />}</span><span><p>{computerControl ? 'Computer control' : activityCategory(entry)}</p><h1>{computerControl ? controlActionLabel(entry) : activityName(entry.tool)}</h1><small>{eventHost}</small></span></div>
<dl className="activity-detail-meta"><div><dt>Status</dt><dd className={attention ? 'attention' : warning ? 'warning' : 'success'}>{status}</dd></div><div><dt>When</dt><dd>{formatDateTime(entry.ts)}</dd></div><div><dt>Duration</dt><dd>{formatDuration(entry.duration_ms)}</dd></div></dl>
{computerControl && <section className="control-timeline" aria-label="Computer control timeline">
<div className="done"><i /><span><strong>Authorized session</strong><small>{entry.control_session_id ? 'Authenticated control session' : 'Authenticated by Hermes'}</small></span></div>
<div className={entry.ok ? 'done' : 'failed'}><i /><span><strong>{controlActionLabel(entry)}</strong><small>{entry.backend === 'cua' ? 'CUA structured engine' : 'Windows input · Compatibility'} · {entry.dispatch ?? 'background'}</small></span></div>
<div className={entry.verification === 'failed' ? 'failed' : entry.verification ? 'done' : ''}><i /><span><strong>Verification</strong><small>{controlVerificationLabel(entry.verification)}</small></span></div>
</section>}
<section className="control-timeline" aria-label="Event timeline">{steps.map((step, index) => <div className={step.state} key={`${step.title}-${index}`}><i /><span><strong>{step.title}</strong><small>{step.detail}</small></span></div>)}</section>
{computerControl && (entry.target_app || entry.target_title || entry.target_pid || entry.target_window_id) && <dl className="control-target"><div><dt>Application</dt><dd>{entry.target_app ?? 'Not reported'}</dd></div><div><dt>Window</dt><dd title={entry.target_title}>{entry.target_title ?? 'Not reported'}</dd></div><div><dt>Target</dt><dd>{entry.target_pid ? `PID ${entry.target_pid}` : 'PID —'} · {entry.target_window_id ? `Window ${entry.target_window_id}` : 'Window —'}</dd></div></dl>}
{entry.error && <aside className="activity-error-callout" role="alert"><AlertTriangle /><span><strong>{entry.aborted ? 'Action stopped' : 'Action failed'}</strong><small>{entry.error}</small></span></aside>}
{entry.screenshot_evidence_id && <button className="screenshot-evidence-card" onClick={() => { setEvidenceError(null); void call('present_activity_screenshot', { evidenceId: entry.screenshot_evidence_id }).catch(error => setEvidenceError(String(error))) }}><span><Eye /><strong>Screenshot captured</strong><small>{entry.screenshot_width && entry.screenshot_height ? `${entry.screenshot_width} × ${entry.screenshot_height} PNG` : 'Retained local evidence'}</small></span><em><Maximize2 /> View larger</em></button>}
{evidenceError && <aside className="activity-error-callout"><AlertTriangle /><span><strong>Screenshot unavailable</strong><small>{evidenceError}</small></span></aside>}
<div className="activity-output-list">{blocks.filter(([, value]) => value).map(([label, value, truncated]) => <section key={label}><header><strong>{label}</strong>{truncated && <em>Truncated</em>}</header><pre>{value}</pre></section>)}</div>
{entry.request_id && <div className="activity-request-id">Request ID {entry.request_id}</div>}
</section>
@@ -811,12 +911,13 @@ function HostDetailPage({ host, clients, busy, onBack, onConnect, onRename, onAc
</section>
}
function SettingsPage({ daemon, computerControl, startup, daemonAutostart, activity, onAction, onStartup, onDaemonAutostart, onHelp, onViewActivity, onOpenActivity }: { daemon: Snapshot['daemon']; computerControl: Snapshot['computer_control_engine']; startup: boolean; daemonAutostart: boolean; activity: Activity[]; onAction: (name: string, args?: Record<string, unknown>) => Promise<unknown>; onStartup: (value: boolean) => void; onDaemonAutostart: (value: boolean) => void; onHelp: () => void; onViewActivity: () => void; onOpenActivity: (entry: Activity) => void }) {
function SettingsPage({ daemon, computerControl, startup, daemonAutostart, activity, screenshotRetention, onAction, onStartup, onDaemonAutostart, onHelp, onViewActivity, onOpenActivity }: { daemon: Snapshot['daemon']; computerControl: Snapshot['computer_control_engine']; startup: boolean; daemonAutostart: boolean; activity: Activity[]; screenshotRetention: Snapshot['activity_screenshot_retention']; onAction: (name: string, args?: Record<string, unknown>) => Promise<unknown>; onStartup: (value: boolean) => void; onDaemonAutostart: (value: boolean) => void; onHelp: () => void; onViewActivity: () => void; onOpenActivity: (entry: Activity) => void }) {
const [update, setUpdate] = useState<UpdateReport | null>(null)
const [updateBusy, setUpdateBusy] = useState<'check' | 'install' | null>(null)
const [updateError, setUpdateError] = useState<string | null>(null)
const [cuaManagement, setCuaManagement] = useState<CuaManagementStatus | null>(null)
const [cuaBusy, setCuaBusy] = useState<'status' | 'install' | 'check' | 'update' | null>(null)
const [cuaHealth, setCuaHealth] = useState<CuaHealthStatus | null>(null)
const [cuaBusy, setCuaBusy] = useState<'status' | 'health' | 'install' | 'check' | 'update' | null>(null)
const [cuaError, setCuaError] = useState<string | null>(null)
const cuaOperation = useCallback(async (operation: 'status' | 'install' | 'check' | 'update') => {
@@ -828,6 +929,15 @@ function SettingsPage({ daemon, computerControl, startup, daemonAutostart, activ
finally { setCuaBusy(null) }
}, [])
const recheckCuaHealth = useCallback(async () => {
setCuaBusy('health')
try {
setCuaHealth(await call<CuaHealthStatus>('computer_cua_health'))
setCuaError(null)
} catch (error) { setCuaError(String(error).replace(/^Error:\s*/i, '')) }
finally { setCuaBusy(null) }
}, [])
const checkUpdate = useCallback(async () => {
setUpdateBusy('check')
try { setUpdate(await call<UpdateReport>('check_desktop_update')); setUpdateError(null) }
@@ -872,6 +982,11 @@ function SettingsPage({ daemon, computerControl, startup, daemonAutostart, activ
const activeBackend = computerControl?.active_backend === 'cua' ? 'CUA active'
: computerControl?.active_backend === 'legacy_compat' ? 'Compatibility active'
: computerControl?.active_backend === 'mixed' ? 'Mixed backends' : 'Idle'
const healthLabel = cuaHealth?.state === 'healthy' ? 'Healthy' : cuaHealth?.state === 'degraded' ? 'Degraded' : cuaHealth?.state === 'error' ? 'Check failed' : 'Not checked'
const healthDetail = cuaHealth?.reason
?? (cuaHealth?.state === 'healthy'
? 'The latest explicit accessibility check passed.'
: 'Optional diagnostic; it does not disable the runtime while the temporary Windows workaround is active.')
return <section className="page-panel settings-page"><div className="page-title"><div><p>Local management</p><h1>Settings</h1></div></div>
<div className="settings-group"><h2>Relay daemon</h2><div className="settings-card"><div className="setting-row"><span><strong>Daemon status</strong><small>{daemon.running ? `${daemon.state} · ${daemon.privilege ?? 'user'}` : 'Stopped'}</small></span><button className="compact-button" onClick={() => onAction('restart_daemon')}><RefreshCw /> Restart</button></div><div className="setting-row"><span><strong>{daemon.privilege === 'administrator' ? 'Administrator mode' : 'User mode'}</strong><small>{daemon.privilege === 'administrator' ? 'Remote actions inherit elevated rights.' : 'Recommended for normal operation.'}</small></span><button className={`compact-button privilege-action ${daemon.privilege === 'administrator' ? '' : 'admin-action'}`} onClick={() => onAction(daemon.privilege === 'administrator' ? 'restart_daemon_as_user' : 'restart_daemon_as_administrator')}>{daemon.privilege === 'administrator' ? 'Return to user mode' : 'Restart as Administrator…'}</button></div><label className="setting-row toggle-row"><span><strong>Start UI at sign-in</strong><small>Launch the tray after you sign in.</small></span><input type="checkbox" checked={startup} onChange={e => onStartup(e.target.checked)} /><i /></label><label className="setting-row toggle-row"><span><strong>Start daemon with UI</strong><small>Connect remote access when the tray starts.</small></span><input type="checkbox" checked={daemonAutostart} onChange={e => onDaemonAutostart(e.target.checked)} /><i /></label></div></div>
@@ -882,13 +997,14 @@ function SettingsPage({ daemon, computerControl, startup, daemonAutostart, activ
<div className="engine-live"><span><strong>{activeSessions}</strong><small>Active session{activeSessions === 1 ? '' : 's'}</small></span><em className={activeSessions ? 'active' : ''}><i /> {activeBackend}</em></div>
<label className="setting-row toggle-row"><span><strong>Animated agent cursor</strong><small>Labeled · smooth glide · click pulse. It does not move your physical mouse.</small></span><input type="checkbox" disabled={computerControl?.selected !== 'cua'} checked={computerControl?.selected === 'cua' && computerControl.cursor_enabled === true} onChange={e => onAction('set_cua_cursor_enabled', { enabled: e.target.checked })} /><i /></label>
<div className="setting-row background-only"><span><strong>Window interaction</strong><small>CUA actions stay in the background and never bring an app forward.</small></span><em>Background only</em></div>
<div className="setting-row cua-health"><span><strong>Accessibility health</strong><small>{healthDetail}</small></span><div><em className={`health-${cuaHealth?.state ?? 'unchecked'}`}>{healthLabel}</em><button className="compact-button" disabled={cuaBusy !== null} onClick={() => void recheckCuaHealth()}>{cuaBusy === 'health' ? <LoaderCircle className="spin" /> : <RefreshCw />} Recheck</button></div></div>
</div>}
<div className="cua-maintenance"><span><strong>{cuaManagement?.installed ? `CUA Driver ${cuaManagement.current_version ?? ''}`.trim() : 'CUA Driver'}</strong><small>{cuaError ?? cuaManagement?.update?.error ?? cuaManagement?.compatibility_reason ?? (cuaManagement?.update?.update_available ? `${cuaManagement.update.latest_version} available` : cuaManagement?.installed ? 'Installed from the verified upstream release.' : 'Install the verified compatible driver explicitly.')}</small></span><div>{!cuaManagement?.installed ? <button disabled={cuaBusy !== null} onClick={() => void cuaOperation('install')}>{cuaBusy === 'install' ? <LoaderCircle className="spin" /> : <Download />} Install</button> : cuaManagement.update?.update_available && cuaManagement.update.compatible ? <button disabled={cuaBusy !== null} onClick={() => void cuaOperation('update')}>{cuaBusy === 'update' ? <LoaderCircle className="spin" /> : <Download />} Update</button> : <button disabled={cuaBusy !== null} onClick={() => void cuaOperation('check')}>{cuaBusy === 'check' || cuaBusy === 'status' ? <LoaderCircle className="spin" /> : <RefreshCw />} Check</button>}</div></div>
</div><p className="group-help engine-help"><ShieldCheck /> CUA is the preferred structured engine. Hermes permissions, grants, audit, and emergency stop remain in control.</p></div>
<div className="settings-group"><h2>CLI & diagnostics</h2><div className="settings-card quick-action-grid"><button onClick={() => onAction('open_terminal')}><TerminalSquare /><span>Open terminal</span></button><button onClick={() => onAction('open_cli_terminal')}><Bot /><span>Open Hermes CLI</span></button><button onClick={() => onAction('open_logs')}><FolderOpen /><span>View daemon log</span></button><button onClick={() => onAction('run_diagnostics')}><ActivityIcon /><span>Run diagnostics</span></button></div></div>
<div className="settings-group"><h2>Updates</h2><div className={`settings-card update-card ${updateError ? 'error' : update?.ahead_of_latest ? 'ahead' : update?.up_to_date ? 'current' : ''}`}><div className="setting-row update-row"><span><strong>Hermes-Relay CLI UI</strong><small>{updateSummary}</small></span>{update && !update.up_to_date && !update.ahead_of_latest && !update.installed ? <button className="compact-button update-button" disabled={updateBusy !== null} onClick={installUpdate}>{updateBusy === 'install' ? <LoaderCircle className="spin" /> : <Download />} Install</button> : <button className="compact-button" disabled={updateBusy !== null} onClick={checkUpdate}>{updateBusy === 'check' ? <LoaderCircle className="spin" /> : <RefreshCw />} Check</button>}</div></div><p className="group-help update-help">Updates the management UI and CLI together, then restarts the tray automatically.</p></div>
<button className="about-link" onClick={onHelp}><span className="setting-icon"><Info /></span><span><strong>Help & About</strong><small>Versions, documentation and troubleshooting.</small></span><ChevronRight /></button>
<div className="settings-group"><div className="settings-group-heading"><h2>Activity</h2><button onClick={onViewActivity}>View all <ChevronRight /></button></div><p className="group-help">Recent remote actions recorded on this PC.</p><div className="settings-card padded"><ActivityList entries={activity.slice(-3).reverse()} host={null} onOpen={onOpenActivity} /></div></div>
<div className="settings-group"><div className="settings-group-heading"><h2>Activity</h2><button onClick={onViewActivity}>View all <ChevronRight /></button></div><p className="group-help">Recent remote actions recorded on this PC.</p><div className="settings-card activity-retention-card"><div className="setting-row"><span><strong>Screenshot evidence</strong><small>{screenshotRetention.count} retained · {formatBytes(screenshotRetention.bytes)} · stored only on this PC</small></span><div className="retention-options" role="radiogroup" aria-label="Screenshot evidence retention">{([{ label: 'Off', enabled: false, days: 7 }, { label: '1d', enabled: true, days: 1 }, { label: '7d', enabled: true, days: 7 }, { label: '30d', enabled: true, days: 30 }] as const).map(option => <button key={option.label} role="radio" aria-checked={screenshotRetention.enabled === option.enabled && (!option.enabled || screenshotRetention.days === option.days)} className={screenshotRetention.enabled === option.enabled && (!option.enabled || screenshotRetention.days === option.days) ? 'active' : ''} onClick={() => onAction('set_activity_screenshot_retention', { enabled: option.enabled, days: option.days })}>{option.label}</button>)}</div></div></div><div className="settings-card padded"><ActivityList entries={activity.slice(-3).reverse()} host={null} onOpen={onOpenActivity} /></div></div>
</section>
}
+67 -1
View File
@@ -68,6 +68,8 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.route-endpoint small { font-size: 10px; font-weight: 550; white-space: nowrap; }
.route-link { position: relative; z-index: 0; display: block; height: 1px; border-top: 1.5px dashed #4fdf72; opacity: .9; }
.connection-route.offline .route-link { border-color: #66717d; opacity: .55; }
.connection-route.retrying .route-link { border-color: var(--amber); opacity: .7; animation: retry-link 1.4s ease-in-out infinite; }
@keyframes retry-link { 50% { opacity: .28; } }
.route-traffic { position: absolute; z-index: 1; pointer-events: none; left: 49px; right: 49px; top: 29px; height: 1px; overflow: hidden; }
.packet { position: absolute; top: -2px; left: 0; width: 9px; height: 5px; border-radius: 2px; background: #a3f9b2; box-shadow: 0 0 4px #72ed89, 0 0 10px #52df70; opacity: 0; }
.packet::after { content: ''; position: absolute; inset: 1px 2px; border-radius: 1px; background: #effff2; }
@@ -107,6 +109,10 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.connection-route.offline .route-status strong { color: #8d98a3; }
.connection-route.connecting .route-status strong { color: var(--violet); }
.connection-route.disconnecting .route-status strong { color: var(--amber); }
.connection-route.retrying .route-status > strong { color: var(--amber); }
.retry-now { border: 0; background: transparent; color: #d3a84f; padding: 2px 5px; display: inline-flex; align-items: center; gap: 4px; font-size: 9.5px; cursor: pointer; }
.retry-now:hover { color: #ffd77b; }
.retry-now svg { width: 11px; height: 11px; }
.connection-route.connecting .route-link { opacity: .9; animation-duration: .65s; }
.connection-route.disconnecting .route-link, .connection-route.disconnecting .route-traffic { opacity: .38; transition: opacity .18s ease; }
.route-status span, .route-status small { color: var(--muted); font-size: 11px; }
@@ -291,7 +297,21 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.control-timeline .failed i { border-color: var(--amber); }
.control-timeline span { display: grid; align-content: start; gap: 1px; }
.control-timeline strong { font-size: 10.5px; font-weight: 560; }
.control-timeline small { color: var(--muted); font-size: 9.5px; text-transform: capitalize; }
.control-timeline small { color: var(--muted); font-size: 9.5px; overflow-wrap: anywhere; }
.control-timeline .pending i { border-color: #6d7782; border-style: dashed; }
.activity-error-callout { margin: 0 0 10px; padding: 9px 10px; border: 1px solid #77502c; border-radius: 7px; background: #2b1b0f; color: #f4c878; display: flex; gap: 8px; align-items: flex-start; }
.activity-error-callout > svg { width: 17px; flex: 0 0 17px; margin-top: 1px; }
.activity-error-callout span { min-width: 0; display: grid; gap: 2px; }
.activity-error-callout strong { font-size: 11px; }
.activity-error-callout small { color: #ddb986; font-size: 10px; line-height: 1.35; user-select: text; overflow-wrap: anywhere; }
.screenshot-evidence-card { width: 100%; min-height: 53px; margin: 0 0 10px; border: 1px solid #4c3d69; border-radius: 8px; background: linear-gradient(110deg, #171329, #101922); padding: 8px 10px; display: flex; justify-content: space-between; align-items: center; text-align: left; cursor: pointer; }
.screenshot-evidence-card:hover { border-color: #8e62d5; background: linear-gradient(110deg, #201638, #111b24); }
.screenshot-evidence-card > span { min-width: 0; display: grid; grid-template-columns: 25px 1fr; grid-template-rows: auto auto; column-gap: 7px; }
.screenshot-evidence-card > span svg { grid-row: 1 / 3; width: 20px; color: var(--violet); align-self: center; }
.screenshot-evidence-card strong { font-size: 11.5px; }
.screenshot-evidence-card small { color: var(--muted); font-size: 9.5px; }
.screenshot-evidence-card em { color: #c39aff; font-size: 10px; font-style: normal; display: flex; align-items: center; gap: 4px; white-space: nowrap; }
.screenshot-evidence-card em svg { width: 13px; }
.control-target { margin: 0 0 10px; display: grid; grid-template-columns: .8fr 1.2fr 1fr; border: 1px solid var(--line); border-radius: 7px; overflow: hidden; }
.control-target div { min-width: 0; padding: 7px 8px; border-right: 1px solid var(--line); display: grid; gap: 2px; }
.control-target div:last-child { border: 0; }
@@ -492,6 +512,13 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.engine-options .setting-row small { max-width: 250px; font-size: 9.5px; line-height: 1.3; }
.background-only { background: #5c41150f; }
.background-only em { flex: 0 0 auto; color: #b5bdc6; font-size: 9px; font-style: normal; text-transform: uppercase; letter-spacing: .45px; }
.cua-health { background: #101820; }
.cua-health > div { flex: 0 0 auto; display: flex; align-items: center; gap: 7px; }
.cua-health em { font-size: 8.5px; font-style: normal; text-transform: uppercase; letter-spacing: .4px; color: var(--faint); }
.cua-health em.health-healthy { color: var(--green); }
.cua-health em.health-degraded, .cua-health em.health-error { color: var(--amber); }
.cua-health .compact-button { min-width: 68px; height: 27px; padding: 4px 7px; font-size: 9px; }
.cua-health .compact-button svg { width: 11px; height: 11px; }
.cua-maintenance { min-height: 49px; padding: 7px 10px 7px 13px; border-top: 1px solid var(--line); display: flex; align-items: center; gap: 8px; }
.cua-maintenance > span { min-width: 0; flex: 1; display: grid; gap: 2px; }
.cua-maintenance strong { font-size: 10.5px; font-weight: 560; }
@@ -527,6 +554,11 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.clear-activity:disabled { opacity: .45; cursor: default; }
.activity-card { padding: 0; }
.activity-card .empty-state { min-height: 90px; }
.activity-retention-card { margin-bottom: 8px; }
.activity-retention-card .setting-row { align-items: center; }
.retention-options { flex: 0 0 auto; display: grid; grid-template-columns: repeat(4, auto); gap: 2px; padding: 2px; border: 1px solid #35414c; border-radius: 7px; background: #0b1218; }
.retention-options button { min-width: 29px; height: 24px; padding: 0 5px; border: 0; border-radius: 5px; background: transparent; color: var(--muted); font-size: 9.5px; cursor: pointer; }
.retention-options button.active { background: #7042d6; color: white; box-shadow: inset 0 1px #c9a5ff33; }
.settings-group-heading { display: flex; align-items: center; justify-content: space-between; }
.settings-group-heading h2 { margin-bottom: 7px; }
.settings-group-heading button { border: 0; background: transparent; color: var(--violet); display: flex; align-items: center; gap: 2px; padding: 0 0 7px; font-size: 11.5px; cursor: pointer; }
@@ -678,6 +710,40 @@ button:focus-visible, input:focus-visible { outline: 2px solid var(--violet); ou
.grant-actions button:hover:not(:disabled) { filter: brightness(1.1); }
.grant-actions button:disabled { opacity: .6; cursor: wait; }
.connection-notice-shell { position: fixed; inset: 0; padding: 7px; display: grid; align-items: stretch; animation: notice-in .2s cubic-bezier(.2,.8,.2,1); }
@keyframes notice-in { from { opacity: 0; transform: translateY(12px) scale(.98); } }
.connection-notice-card { position: relative; border: 1px solid #3c4854; border-radius: 12px; background: #101922f5; box-shadow: 0 15px 38px #000b, inset 0 1px #ffffff08; padding: 13px 56px 12px 13px; display: grid; grid-template-columns: 38px minmax(0,1fr); gap: 10px; align-items: center; overflow: hidden; }
.connection-notice-card::before { content: ''; position: absolute; inset: 0 auto 0 0; width: 3px; background: var(--green); box-shadow: 0 0 13px var(--green); }
.connection-notice-shell.warning .connection-notice-card::before { background: var(--amber); box-shadow: 0 0 13px var(--amber); }
.connection-notice-shell.offline .connection-notice-card::before { background: #8c96a0; box-shadow: none; }
.connection-notice-icon { width: 36px; height: 36px; border-radius: 10px; background: #21432b; color: var(--green); display: grid; place-items: center; }
.connection-notice-shell.warning .connection-notice-icon { background: #493817; color: var(--amber); }
.connection-notice-shell.offline .connection-notice-icon { background: #29323b; color: #aeb7c0; }
.connection-notice-icon svg { width: 21px; }
.connection-notice-card > span:nth-child(2) { min-width: 0; display: grid; gap: 1px; }
.connection-notice-card small { color: var(--violet); font-size: 8.5px; text-transform: uppercase; letter-spacing: .8px; }
.connection-notice-card strong { font-size: 13px; }
.connection-notice-card p { margin: 1px 0 0; color: var(--muted); font-size: 10px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.connection-notice-open { position: absolute; right: 12px; bottom: 12px; border: 0; background: transparent; color: #c59aff; font-size: 9.5px; cursor: pointer; }
.connection-notice-close { position: absolute; right: 8px; top: 8px; width: 25px; height: 25px; border: 0; border-radius: 6px; background: transparent; color: #8f99a4; display: grid; place-items: center; cursor: pointer; }
.connection-notice-close:hover { color: white; background: #ffffff0c; }
.connection-notice-close svg { width: 14px; }
.evidence-shell { position: fixed; inset: 0; border: 1px solid #3a4652; border-radius: 11px; background: #0b1218; box-shadow: 0 24px 70px #000d; display: grid; grid-template-rows: 52px minmax(0,1fr); overflow: hidden; }
.evidence-shell > header { border-bottom: 1px solid var(--line); background: #111a23; padding: 0 13px 0 16px; display: flex; align-items: center; justify-content: space-between; }
.evidence-shell > header > span { display: grid; grid-template-columns: 28px auto; grid-template-rows: auto auto; column-gap: 8px; }
.evidence-shell > header svg { grid-row: 1 / 3; align-self: center; width: 20px; color: var(--violet); }
.evidence-shell > header strong { font-size: 13px; }
.evidence-shell > header small { color: var(--muted); font-size: 9.5px; }
.evidence-shell > header button { width: 34px; height: 34px; border: 0; border-radius: 7px; background: transparent; color: #aab2bb; display: grid; place-items: center; cursor: pointer; }
.evidence-shell > header button:hover { color: white; background: #c42b1c; }
.evidence-shell > main { min-width: 0; min-height: 0; padding: 14px; display: grid; place-items: center; background: radial-gradient(circle at center, #18232c, #080e13 68%); }
.evidence-shell img { max-width: 100%; max-height: 100%; object-fit: contain; border-radius: 5px; box-shadow: 0 8px 30px #000a; user-select: none; }
.evidence-loading, .evidence-error { color: var(--muted); display: grid; place-items: center; gap: 8px; text-align: center; }
.evidence-error svg { color: var(--amber); width: 30px; }
.evidence-error strong { color: var(--ink); font-size: 14px; }
.evidence-error small { max-width: 420px; font-size: 11px; }
@media (max-width: 410px) {
.content { padding-left: 20px; padding-right: 20px; }
.access-control button { gap: 5px; }
+18 -1
View File
@@ -23,6 +23,10 @@ export interface DaemonStatus {
privilege?: string | null
username?: string | null
updated_at?: number | null
last_event?: string | null
reconnect_attempt?: number | null
retry_at?: number | null
last_error?: string | null
}
export interface ComputerControlEngine {
@@ -62,9 +66,17 @@ export interface CuaManagementStatus {
operation?: { kind: 'install' | 'update'; state: 'completed'; version: string }
}
export interface CuaHealthStatus {
state: 'healthy' | 'degraded' | 'error'
checkedAt: string
overall?: string
reason?: string
temporaryWindowsCompatibility: true
}
export interface Activity {
ts: number
kind?: 'tool.completed' | 'management.completed'
kind?: 'tool.completed' | 'management.completed' | 'connection.state'
tool: string
category?: 'command' | 'files' | 'screen' | 'input' | 'devices' | 'system' | 'other'
ok: boolean
@@ -94,6 +106,10 @@ export interface Activity {
stderr_truncated?: boolean
result_truncated?: boolean
error?: string
screenshot_evidence_id?: string
screenshot_mime_type?: 'image/png'
screenshot_width?: number
screenshot_height?: number
}
export interface Snapshot {
@@ -101,6 +117,7 @@ export interface Snapshot {
active_url?: string | null
daemon: DaemonStatus
activity: Activity[]
activity_screenshot_retention: { enabled: boolean; days: number; count: number; bytes: number }
pending_grants: PendingGrantRequest[]
startup_enabled: boolean
daemon_autostart_enabled?: boolean
+14 -5
View File
@@ -3073,12 +3073,13 @@ state—while omitting accessibility text, screenshots, entered values, and raw
driver responses.
Hermes now owns an explicit Windows lifecycle surface without bundling the
driver: `computer-use cua status|install|check-update|update`, with `--yes`
driver: `computer-use cua status|health|install|check-update|update`, with `--yes`
required for install/update. Mutations accept only supported upstream releases
(`>=0.19.3 <0.20.0`), verify the `trycua/cua` product/version manifest and
installer SHA-256 before running a temporary installer under a sanitized
environment, then verify the canonical `packages/current` binary, manifest,
version, path, permission mode, and health. There is no automatic install or
version, path, and permission mode. Accessibility health is an explicit
diagnostic. There is no automatic install or
update.
**Context.** The first Windows input backend uses PowerShell, `SetCursorPos`,
@@ -3103,7 +3104,7 @@ dispatch, application launch/termination, JavaScript execution, recording,
replay, configuration, and driver updates require separate, explicit local
authority. Full Access may bypass ordinary task prompts, but never authenticated
targeting, sensitive-surface blocks, UAC/session boundaries, audit, emergency
stop, or driver health checks.
stop, runtime validation, or per-action failures.
Every semantic element action requires a fresh pre-action window snapshot, an
opaque element token bound to its snapshot generation, authenticated principal,
@@ -3123,10 +3124,18 @@ change, or daemon shutdown ends the corresponding driver session immediately.
CUA runs in the interactive user's logon session, never Windows Session 0.
Initial packaging remains optional and resolves the canonical installed package
rather than an untrusted PATH entry. Readiness uses the driver's live manifest,
schema, permission mode, and health report and fails closed on an absent,
degraded, or incompatible backend. Telemetry and driver updates remain explicit
schema, tool surface, daemon status, and permission mode and fails closed on an
absent or incompatible backend. Telemetry and driver updates remain explicit
operator choices.
**Temporary Windows implementation note (2026-08-14).** Until
`trycua/cua#3103` is fixed in the supported driver range, the whole-desktop
`health_report` is not a session-start gate: its fixed UIA timeout can report a
false degradation and leave the driver temporarily busy. Operators can re-run
that diagnostic from the CLI or UI. Structured actions retain their existing
target, grant, snapshot, timeout, and fail-closed checks. Remove this exception
when the upstream probe is bounded and cannot poison later actions.
**Consequences.** Hermes can gain background, element-aware control and clean
per-agent animated cursors without replacing its Relay protocol or permission
model. The existing PowerShell/User32 backend remains a compatibility fallback
+3 -1
View File
@@ -73,6 +73,7 @@ choice. Read-only status and update checks never install anything:
```powershell
hermes-relay computer-use cua status
hermes-relay computer-use cua health
hermes-relay computer-use cua check-update
hermes-relay computer-use cua install --yes
hermes-relay computer-use cua update --yes
@@ -84,7 +85,8 @@ to apply it if it falls outside Hermes-Relay's supported range (`>=0.19.3,
<0.20.0`). Hermes downloads the versioned GitHub release manifest and installer,
checks the manifest repository/product/version and the installer's SHA-256, and
then verifies the canonical binary path, version, driver manifest, required
tools, permission mode, and health report. These are release-metadata and
tools, and permission mode. Accessibility health remains an explicit recheck
while the temporary Windows compatibility workaround is active. These are release-metadata and
checksum integrity checks, not a Windows publisher signature.
The UI provides the same explicit **Install**, **Check**, and **Update** actions
+9
View File
@@ -296,6 +296,7 @@ hermes-relay computer-use engine cua # prefer structured CUA for new sessio
hermes-relay computer-use engine legacy # explicit Windows-input compatibility
hermes-relay computer-use cursor on # virtual per-session cursor
hermes-relay computer-use cua status
hermes-relay computer-use cua health # recheck accessibility health
hermes-relay computer-use cua check-update
hermes-relay computer-use cua install --yes
hermes-relay computer-use cua update --yes
@@ -332,8 +333,16 @@ Show what the remote agent has run on **this** machine through the desktop tools
hermes-relay audit # last 50 desktop-tool calls
hermes-relay audit --limit 20 # fewer
hermes-relay audit --json # raw entries for scripting
hermes-relay audit screenshots --json
hermes-relay audit screenshots on --days 7 --yes
hermes-relay audit screenshots off --yes
```
Screenshot evidence is local and separate from the JSONL log. Retention defaults
to seven days and is capped at 20 PNG files and 10 MB per file. Choose 1, 7, or
30 days, or turn it off; disabling retention removes existing screenshot
evidence without deleting the remaining activity history.
```
Desktop-tool activity (3 most recent)
+15 -4
View File
@@ -238,6 +238,7 @@ hermes-relay computer-use status --json
hermes-relay computer-use engine cua
hermes-relay computer-use cursor on
hermes-relay computer-use cua status
hermes-relay computer-use cua health
hermes-relay computer-use cua check-update
hermes-relay computer-use cua install --yes
hermes-relay computer-use cua update --yes
@@ -246,8 +247,9 @@ hermes-relay computer-use cua update --yes
The management UI exposes the same choices under **Settings → Computer
control**. Selecting `cua` succeeds only when Hermes finds the canonical runtime
at `%USERPROFILE%\.cua-driver\packages\current\cua-driver.exe` and verifies its
supported version, manifest identity, allowlisted tools, non-unrestricted
permission mode, and live health report. Hermes does not trust whichever `cua-driver.exe`
supported version, manifest identity, allowlisted tools, and non-unrestricted
permission mode. Accessibility health is an explicit diagnostic while the
temporary Windows workaround for trycua/cua#3103 is active. Hermes does not trust whichever `cua-driver.exe`
happens to appear first on `PATH`. If the runtime is missing, incompatible, or
degraded, the UI explains why and a new control session can use Windows input
compatibility. Backend selection is made once per authenticated control session;
@@ -276,7 +278,9 @@ the canonical upstream package, validates versioned GitHub release metadata and
installer SHA-256, and refuses updates outside `>=0.19.3,<0.20.0`. These checks
do not claim a Windows publisher signature. Hermes executes the verified
temporary installer under a sanitized environment, then validates the canonical
`packages/current` binary, driver manifest, permission mode, and health. Hermes forces CUA telemetry off for
`packages/current` binary, driver manifest, and permission mode. The UI and
`computer-use cua health` can recheck accessibility health without changing
the selected backend. Hermes forces CUA telemetry off for
its child invocations; any future telemetry opt-in belongs to the local
operator. `hermes-relay update` continues to manage only the Hermes-Relay CLI
and Windows UI.
@@ -356,7 +360,14 @@ If `connected: true` but the agent still says the tool is missing:
`hermes-relay daemon` runs the WSS connection + tool router headless, so the agent can reach your machine while you're in another window or VS Code or off making coffee. Use `hermes-relay daemon start` to run it in the **background** (no console window, survives closing the terminal), `daemon status` to check it, and `daemon stop` to stop it. See [Subcommands → daemon](./subcommands.md#hermes-relay-daemon) for full lifecycle/log details.
Want to see what the agent actually ran on your machine? `hermes-relay audit` lists recent `desktop_*` activity from a local log. The management UI previews the latest three events and opens each event into bounded request, stdout, stderr, result, exit, timing, and truncation details. Sensitive request inputs are excluded.
Want to see what the agent actually ran on your machine? `hermes-relay audit` lists recent `desktop_*` activity from a local log. The management UI previews the latest three events and opens each event into a lifecycle stepper plus bounded request, stdout, stderr, result, exit, timing, and truncation details. Errors have a dedicated failure panel. Screenshot events may retain bounded local evidence for a larger viewer; Settings controls Off/1-day/7-day/30-day retention and clearing Activity removes it. Sensitive request inputs remain excluded from the JSON audit log.
The daemon records one interruption event when automatic reconnect begins and a
recovery event when it succeeds rather than adding one row per backoff attempt.
The Overview shows retry attempt/timing and supports **Retry now** or an explicit
disconnect. When the management UI is hidden, connection loss and restoration
use the same compact local-card language as permission requests; no duplicate
card appears while the UI is already open.
`daemon start` covers "background, this session." On Windows, **Start UI at sign-in** registers the optional tray as a per-user login entry. **Start daemon with UI** separately opts into connecting remote access when the tray launches and defaults off for existing installs. Neither is a Windows service. For Linux/macOS or a machine-level lifetime, wrap foreground `hermes-relay daemon` with your service manager of choice.
+10 -5
View File
@@ -97,12 +97,17 @@ hermes-relay computer-use status --json
is intentionally ignored.
- **Incompatible** means the executable, manifest, supported version, required
tool set, or permission mode did not match the Hermes adapter contract.
- **Degraded** means the live CUA health report was not healthy. On Windows,
confirm the driver is running in the interactive user's logon session rather
than Session 0, then run the upstream `cua-driver doctor` command.
- **Accessibility health** is a separate, explicit diagnostic on Windows. Use
**Recheck** in the UI or run `hermes-relay computer-use cua health`. A
degraded result does not disable the runtime while the temporary workaround
for the upstream fixed-timeout issue is active; canonical runtime checks and
each structured action still fail closed. Confirm the driver is running in
the interactive user's logon session rather than Session 0 before deeper
diagnosis.
CUA is preferred for new structured-control sessions. If it is not ready before
a session begins, Hermes can select the Windows input compatibility backend;
CUA is preferred for new structured-control sessions. If its executable,
manifest, required tool set, daemon status, or safe permission mode is not
ready before a session begins, Hermes can select the Windows input compatibility backend;
it never changes backend in the middle of a control session. Re-check with
`hermes-relay computer-use cua status`, repair explicitly with
`computer-use cua install --yes`, or use `computer-use cua check-update` followed