381 KiB
Hermes-Relay — Dev Log
2026-04-19 — MorphingSphere: pure-core extraction + browser preview + runtime parity proof
Context. The sphere had grown into a 494-line Compose file with Android Paint + Typeface + nativeCanvas.drawText wired into the same function that did the math. That made it impossible to iterate on the algorithm without building to a device — and blocked any future port to a non-Android surface (user site, Hermes TUI, Compose Desktop). Branch claude/ui-dev-preview-exploration-vKvzr split the file into a pure algorithm + a Compose renderer and added a zero-dep browser harness; this session added the parity proof.
Layout — three artifacts, one seam.
MorphingSphereCore.kt(new, 412 lines) —kotlin.mathonly. OwnsSphereState,SphereParams,SphereColors,SphereFrame,SphereCell,paramsFor(),colorsFor(),forEachSphereCell(). No Android, no Compose, noPaint.MorphingSphere.kt(494 → ~45 lines of renderer +@Previewdecorators) — Compose Canvas binding. Swapped legacyPaint+Typeface+nativeCanvas.drawTextforrememberTextMeasurer()+ ComposedrawText. Owns animation state (animateFloatAsState,rememberInfiniteTransition) and feeds aSphereFrameinto the core per tick.preview/web/sphere.js— line-for-line JS mirror ofMorphingSphereCore.kt.Math.imul+|0to match Kotlin'sIntoverflow in the hash,((x % n) + n) % nfor Kotlin's floored-positive.mod(),Math.truncfor.toInt()truncation-toward-zero.
Browser harness. preview/web/index.html — live HTML+canvas, python3 -m http.server --directory preview/web, no build step. Panel exposes State / Voice mode / Voice amp / Intensity / Tool burst / Pause / reset t, plus a Layout section (Cols, Rows, Fill %, Aspect, Char size) added this session with a phone 9:16 preset that pins canvas aspect to 0.5625 to match Compose @Preview(widthDp=360, heightDp=640). Hitting 1..6 cycles state; Space pauses.
Runtime parity proof. Line-by-line code audit was first, but proving parity needs evidence, not inspection. Added:
preview/web/parity-check.mjs— Node harness running sphere.js at the 8 Compose@Previewfixtures (Idle/Thinking/Streaming/Error/Compact/Listening/Speaking-low/Speaking-peak), emitting two FNV-1a 32-bit checksums per fixture.structhashes only(row, col, charCode)— discrete, robust to Float/Double precision drift.fullhashes the same plus color/alpha rounded to 3 decimals.app/src/test/kotlin/com/hermesandroid/relay/ui/components/MorphingSphereCoreParityTest.kt— JVM unit test (no Android deps, runs ontestGooglePlayDebugUnitTest), mirrors the JS harness exactly: same 8 fixtures, same FNV-1a impl via KotlinUInt, same%.3fformatting viaLocale.ROOT, same tuple layout.
Result. 8 / 8 structural checksums match. 8 / 8 zone histograms match (inside/glow/ring counts identical). 6 / 8 full checksums match — Listening and Speaking-low drift on color/alpha at the 3rd decimal, which is expected Float (Kotlin) vs Double (JS) mantissa precision in compound expressions like (colR + lightBoost + warmth).coerceIn(0, 1). Speaking-peak avoids the drift because amp=0.95 puts lerp() results near the endpoints where Float mantissa has room.
Decision. MorphingSphereCore is declared the single source of truth going forward. The Compose MorphingSphere composable keeps its public API (same params, same defaults) so call sites — VoiceModeOverlay, the chat empty state — need no changes. Future edits to the algorithm go in MorphingSphereCore.kt, mirror into sphere.js, and MorphingSphereCoreParityTest catches drift between sides.
Reusable surface. The core is now droppable into:
- A terminal TUI (Hermes CLI) — swap Compose
drawTextfor ANSI-coloredprint, keep everything else. - The user site (codename-11.dev) — reuse
sphere.jsdirectly with an HTML<canvas>host. - Compose Desktop — reuse
MorphingSphere.ktunchanged once the host project adds theui/componentsdir to its source set.
Worktree gotcha. git worktree add doesn't copy .gitignored files. local.properties (SDK path + keystore creds) has to be recreated in any new worktree before gradle tasks run. For the parity test a minimal sdk.dir=... is enough — signing creds stay in the main checkout.
2026-04-19 — Multi-endpoint pairing + first-class Tailscale (ADR 24 + 25)
Branches: Wave 1 + Wave 2 landed on dev. ADRs 24 and 25 written and committed. Docs pass (this entry) closes the work out.
Shipped.
- ADR 24 — Multi-endpoint pairing.
plugin/pair.pynow emits an orderedendpointsarray (lan/tailscale/public/ operator-defined roles) in a newhermes: 3QR schema;hermes: 2stays the default when no endpoints are present. New CLI flags--mode {auto,lan,tailscale,public}and--public-url <url>drive candidate emission;--mode autoautodetects LAN IP + Tailscale status and composes them with an optional public URL.plugin/relay/qr_sign.pycanonical form preserves array order and role strings verbatim — HMAC round-trip test pins this against future refactors.plugin/relay/server.pyhandle_pairing_mint/handle_pairing_registeraccept the optionalendpointsbody field. Phone side: newdata/Endpoint.kt(EndpointCandidate/ApiEndpoint/RelayEndpoint/displayLabel()),data/PairingPreferences.ktper-device endpoint store,ui/components/QrPairingScanner.ktparses the new field with a v1/v2 synthesizer for back-compat,auth/AuthManager.ktpersists the endpoint list onauth.ok, andviewmodel/ConnectionViewModel.ktstages endpoints at pair time. Wave 2 Kt-Probe addedConnectionManager.resolveBestEndpoint()+NetworkCallbackre-probing andRelayUiState.activeEndpointRole. - ADR 25 — First-class Tailscale helper. New
plugin/relay/tailscale.pywithstatus()/enable(port)/disable(port)/canonical_upstream_present()— all shell out to thetailscaleCLI with short timeouts, return structured dicts, never raise. Newplugin/relay/tailscale_cli.pyargparse wrapper +scripts/hermes-relay-tailscaleshell shim mirroring thehermes-pairpattern.install.shgains optional step [7/7] offering Tailscale enablement; honoursTS_DECLINE=1/TS_AUTO=1. - Dashboard Remote Access tab (Wave 2 Dashboard-R4).
plugin/dashboard/plugin_api.py+ React UI + committeddist/index.js— operator can enable/disable the helper, mint multi-endpoint QRs, and inspect which modes are active without dropping to a shell. - Docs pass. New
docs/remote-access.md(263 lines) — decision matrix + per-mode setup recipes + troubleshooting. Updateddocs/security.mdwith a "Remote connectivity" subsection + top-of-list Tailscale recommendation. Updateddocs/relay-server.mdwithTS_AUTO/TS_DECLINEenv vars, the dashboard plugin proxy-route table, and a Tailscale-helper subsection. Updateddocs/spec.md§3.3 + §3.3.1 for v3 QR schema + endpoints array. UpdatedREADME.md"What's new" with the one-line connect-from-anywhere pitch. UpdatedCHANGELOG.md[Unreleased]with Added / Changed / Backward compatible subsections. UpdatedCLAUDE.mdKey Files + Integration Points (hygiene pass only).
Key architectural decisions (restated from the ADRs).
- Strict priority, not reachability-weighted. Priority 0 wins when reachable; reachability is a tiebreaker among equal priorities, never a promoter across priority boundaries. Matches DNS SRV semantics — nothing new to debate, and keeps operator intent authoritative.
- Open-string
role, not a closed enum.wireguard,zerotier,netbird-eu, etc. render as generic "Custom VPN ()" without a release. HMAC canonicalization preserves the exact emitted string. - Tailscale helper auto-retires on upstream merge.
canonical_upstream_present()probeshermes gateway run --help | grep tailscale; once PR #9295 lands, the helper prints a log line pointing at the canonical flag and exits 0. Same removal pattern ashermes_relay_bootstrap/after PR #8556. - No second crypto layer. The operator already owns both endpoints and the transport (Tailscale / Caddy / WireGuard / Cloudflare Tunnel) is TLS-terminated by the operator's chosen path. Adding Noise / libsignal over WSS would add complexity without defending any threat in the trust model.
What's next.
- Monitor upstream PR #9295 for merge. When it lands, verify
canonical_upstream_present()detection works on a vanilla hermes-agent install, update the helper to print the retirement log line, and schedule the helper's removal PR (one file + the install.sh step + the shim). - Bailey's Studio build +
./gradlew lintpass before the multi-endpoint work gets pushed from Kotlin side. No Python blockers.
Blockers. None. Awaiting Bailey's lint + build pass on the Kotlin changes before the feature branches merge into dev.
Same-day follow-up — --prefer priority override + regression fix
Two commits landed on dev after the initial ADR 24 + 25 bundle:
-
feat(pairing): --prefer priority override on all pair surfaces(e914810). Adds explicit "promote this role to priority 0" control so operators can force a specific endpoint path without re-ordering defaults globally. Surfaces: CLIhermes-pair --prefer tailscale, the/hermes-relay-pairskill (SKILL.md updated), and a dropdown below the Endpoint Preview card on the dashboard's Remote Access tab. Open-vocab role string; case-insensitive matching; unknown role emits a stderr warning and keeps natural order. 6 newBuildEndpointCandidatesPreferTestscover the happy paths and edge cases. Dashboard bundle rebuilt to 61.3 KB. -
fix(relay): restore profile PUT handlers clobbered by ADR 24 commit(ee653d4). The ADR 24 bundle (fae8ccd) had an Edit that replaced a wider chunk ofplugin/relay/server.pythan intended while adding the endpoints passthrough tohandle_pairing_mint/handle_pairing_register— collaterally deleting ~479 lines ofhandle_profile_soul_put/handle_profile_memory_put+_extract_write_content. CI — Relay went from 2 pre-existing failures (test_profile_discovery, unrelated) → 27 on the bundle push → back down to the same 2 after the fix. Restored by resettingserver.pyto the pre-ADR-24 state (47667bd) and re-applying only the intended ~30-line endpoints passthrough. Full suite: 673 pass / 6 skipped locally. Same bug class as theAuthManager.profilesUpdatedEventscollateral wipe caught mid-deploy — worth a note in any future "agent-assisted refactor" guidance: tightold_stringanchors + verify-by-compile after every agent's work. -
Server deploy (
~/.hermes/hermes-relay/ondev, PID restarted forhermes-relay+hermes-gateway+hermes-dashboard) verified at each step:{"status": "ok"}health,tailscale.status()returning the livedocker-server.tail6f460.ts.net/100.71.8.56, andPOST /api/plugins/hermes-relay/pairing {"mode":"auto","prefer":"tailscale"}returning[(0, 'tailscale'), (1, 'lan')]over the wire. Pre-existingtest_profile_discoveryfailures on Linux CI runners (Windows-local passes) are a separate Bailey in-progress item — not introduced by this work. -
Docs sync pass.
docs/remote-access.md→ added "Promoting a role to priority 0 —--prefer" subsection under Combining modes.user-docs/features/connections.md→ new "Multi-endpoint pairing" section (end-user facing).user-docs/guide/getting-started.md→ new "Connecting from Anywhere (Tailscale, VPN, Public URL)" section between Relay Server and Verify Connection.CHANGELOG.md[Unreleased]gains--preferunder Added + the PUT-handler restore under Fixed.
2026-04-18 — Profile Inspector UI (v0.7.0)
Branch: feature/profile-inspector-ui. Kotlin-worker slice of the v0.7.0 Profile Inspector feature — Python worker runs in parallel and owns the /soul + /memory relay endpoints. Two feature commits + docs.
Shipped (Kotlin).
RelayProfileInspectorClient+ wire models. New read-only HTTP client for the four inspector endpoints (/api/profiles/{name}/config,/skills,/soul,/memory) mirroringRelayHttpClient's constructor shape (OkHttp, lazy bearer-token provider,ws://→http://URL flip). All four fetch methods returnResult<T>and hop toDispatchers.IObefore any network I/O — reinforcing the v0.6.0 post-mortem fix forNetworkOnMainThreadExceptionin the voice client. Profile names are URL-encoded before being spliced into the path so names with spaces / non-ASCII characters don't blow up the route. Wire models (ProfileConfigResponse,ProfileSkillsResponse,ProfileSoulResponse,ProfileMemoryResponse) are@Serializablewith snake_case → camelCase mapping via@SerialName. Optional wire fields (truncated,readonly,enabled) default to safe values so pre-v0.7.0 relays that omit them deserialize cleanly.ProfileInspectorScreen(4 tabs). Config renders as a collapsible JSON tree (nested objects click to expand, monospace leaf values, 120-char truncation). SOUL is a scrollable monospace box with byte-size caption + truncation banner when the server flags the content as sliced; absent SOUL renders an empty-state with the expected path. Memory is a list of expandable cards per entry (filename + size in the header, content revealed on tap, per-entry truncation banner). Skills groups by category with a "(disabled)" label on skills whereenabled=false. Top-bar Refresh icon firesloadAll(); each pane has its own inline Retry button on error state. PullToRefresh was skipped in favour of the explicit Refresh icon + per-section Retry — matchesPairedDevicesScreen's "still-experimental PullRefresh" note.ProfileInspectorViewModel. Four independentLoadState<T>flows — one per section — so a slow/memoryfetch never blocks the already-arrived/configtab. Lazy (no fetch untilloadAll()) and stateful per-section (refreshSection(InspectorSection.Config)re-fetches just that tab). Profile name comes in viaSavedStateHandleso a process-death restore inspects the same profile. The VM is keyed off the profile name in the nav backstack so entering a different profile produces a fresh VM rather than reusing stale state.ProfileInspectorCardin Settings. Sits directly underActiveAgentCardso the "active agent → inspect it" reads naturally top-to-bottom. When no profile is selected, the card renders at 50% alpha with "No active agent" and becomes a no-op — kept visible (not hidden) so the feature stays discoverable before a pair-and-pick happens.Screen.ProfileInspectornav destination. Typed String path arg, registered via a tinyViewModelProvider.Factorythat pullsSavedStateHandleout ofCreationExtrasso nav args propagate into the VM constructor cleanly. Back navigation pops the destination and the VM is GC'd with it.
Key architectural decisions.
- Four independent load states, not one "screen state". Each section's flow transitions Idle → Loading → Loaded/Error independently. One combined state would have meant the fastest-arriving tab gets blocked by the slowest — unnecessary UX cost given the 3 - 4 tabs the user flips between.
- URL-encoded profile-name splice, not an OkHttp path segment. We use
URLEncoder.encode(..., "UTF-8").replace("+", "%20")before splicing into the literal path string, then build theHttpUrlfrom the full string. OkHttp'saddPathSegmentwould re-encode and produce double-encoding for profile names with%already in them (pathological but possible). A single round of encoding is the safe choice. - ViewModel keyed on nav arg.
viewModel(key = "profile-inspector-$name", ...)means the same profile navigated to twice yields the same VM (back-stack reuse), but switching to profile B produces a fresh VM — avoids the "I opened inspector for axiom, changed profiles, reopened and saw axiom's data" class of bug.
Deferred.
- Pull-to-refresh gesture. Explicit Refresh button is enough for v0.7.0; if users ask for the gesture we'll revisit once Material3's
PullToRefreshBoxgraduates out of experimental. - Edit-in-place. Inspector is strictly read-only by design. Editing is still "SSH to the server and
$EDITOR config.yaml" territory. A "Copy to clipboard" affordance on each section is a possible follow-up if users ask for it. - MockWebServer integration tests.
testImplementationdoesn't include MockWebServer and the spec forbids adding deps for this slice — we covered wire-contract parsing + URL encoding + required-field enforcement via JVM-local tests, and the client's actual network path is exercised by the on-device relay smoke test.
Upstream dependency. Soul and Memory endpoints land via the parallel Python worker on the same branch. The Kotlin side is fetch-and-render — if either endpoint returns a 5xx / 404 before the Python change merges, the relevant tab just shows a Retry error state and the rest of the inspector keeps working.
2026-04-18 — Profile metadata + read-only config API (v0.7.0 groundwork)
Branch: feature/profile-config-readonly. Kotlin-worker slice of the v0.7.0 groundwork (Python-worker slice runs in parallel, owns the relay-side wire contract). Three feature commits + docs.
Shipped (Kotlin).
- Extended
Profiledata class. Added three optional fields —gatewayRunning: Boolean,hasSoul: Boolean,skillCount: Int— mapped via@SerialNameto the snake_case wire keys (gateway_running,has_soul,skill_count) the relay will emit inauth.okprofiles entries. All three default to safe zero-values so pre-v0.7 relays deserialize cleanly.AuthManager.parseAgentProfilesreads them withbooleanOrNull/intOrNullfallbacks — malformed values fall through to defaults rather than crashing the pairing handshake. - Runtime-metadata indicators in the agent sheet. Each Profile row in
AgentInfoSheetnow carries a 6 dp status dot (green when gateway_running, grey otherwise), an optional "N skills" chip (hidden when skill_count == 0), and an optional "SOUL" badge (primary-container, hidden when has_soul is false). Gateway-off profiles stay selectable at 50% alpha — the probe is best-effort, and a stale dot shouldn't lock a user out.ProfileRadioRowgrew three optional params (contentAlpha,leadingDotColor,secondaryTrailingslot) without changing the Default/personality-row call sites. Also added an inline "Profile SOUL overrides personality while active" caption under the personality section when a profile-with-SOUL + non-default personality are both selected, mirroring the existing caption in the Profile section. - Persisted
selectedProfileper Connection. NewProfileSelectionStore— its own DataStore (profile_selections) keyed by connectionId — lets each Connection remember its last-picked profile across app restart.ConnectionViewModel.selectProfilewrites through. Connection switch clears_selectedProfilefirst (so a stale A pick never dangles on B) then loads B's persisted name and resolves it againstagentProfilesonce the post-switchauth.okrepopulates the list.removeConnectioncallsstore.clear(connectionId)after the switch-away to avoid racing the unmounted store.
Key architectural decisions.
- Name-based persistence, not Profile-object persistence. The persisted value is the profile
name: String, resolved at read time against the server's freshagentProfileslist. Profiles can be renamed, removed, or re-modelled on the server between app launches; persisting a fullProfilesnapshot risks silently using a stale model override. Resolution-at-read yields null when the name no longer exists, and the UI falls through to the Default row. - Dedicated DataStore.
profile_selectionsis separate fromrelay_settingsso clearing or migrating one doesn't threaten the other. A new per-connection key prefix (selected_profile_<id>) lets us clear-per-connection cleanly on removal. - Gateway-off profiles remain selectable. Spec explicitly allows the user to pick a profile whose gateway probe is grey. Rationale: the probe is best-effort, can miss a recent restart, and hard-disabling a row based on a stale probe would confuse users who just ran
hermes profile useand haven't restarted the relay.
Deferred.
- Migration of any old "ephemeral" pick into the new store. N/A — the prior behaviour was to always reset on restart, so there's nothing to migrate from. First boot on v0.7.0 starts clean; next
selectProfilecall writes through. - UI for inspecting the gateway-running timestamp. Useful for debugging stale probes but noise for the common path. Track in a future dashboard pass if it's actually asked about.
Upstream dependency. The three new profile fields are optional on the wire and fall back to defaults, so this PR is safe to land ahead of the Python-worker relay change. Pairing against a pre-v0.7 relay continues to work — the phone just renders every dot grey and hides every badge.
2026-04-18 — v0.6.0: Multi-Connection + Agent Profiles
Branch: feature/multi-profile-connections. Landing as v0.6.0. Two orthogonal pieces of work that compose into a three-layer agent model (Connection → Profile → Personality), plus a top-bar + Settings UX consolidation.
Shipped.
- Multi-Connection (
docs/decisions.md§19). Renamed the original internal "profile" concept to Connection — a paired Hermes server.ConnectionStorepersists N connections in DataStore with the active one, migration seeds connection 0 from the existinghermes_companion_auth_hwstore (zero re-pair, transparent to the user),SessionTokenStoretakes aprefsNameparameter so each Connection gets its ownEncryptedSharedPreferencesfile,AuthManagergains aconnectionIdctor. Switch is a heavy context swap: cancel in-flight SSE, disconnect relay WSS, rebind provider StateFlows (RelayHttpClient/RelayVoiceClientre-read lazily), rebuildHermesApiClient, reconnect, reprobe capabilities, reload sessions + personalities + profiles, restore per-Connection last-active session. Top-bar Connection chip +ConnectionSwitcherSheetfor switching; Settings → Connections for CRUD (rename, re-pair viaConnectionWizard, revoke, remove). Per-connection scope: sessions, memory, personalities, skills, profiles, relay cert pin, voice endpoints, last-active session. Global scope: theme, bridge safety prefs, TOFU cert-pin map, notification companion state. - Agent Profiles (
docs/decisions.md§21). Directory-scan + overlay. Relay walks~/.hermes/profiles/*/, reads each profile'sconfig.yaml+SOUL.md, and advertises the list inauth.okas{name, model, description, system_message}(plus a synthetic "default" entry for the root config). Phone sendsmodeloverride +system_messagefromSOUL.mdon chat turns when a profile is selected. Gated byRELAY_PROFILE_DISCOVERY_ENABLED=1(default on). Shipped asoverlay-not-isolation: memory, sessions,.env, skills, and cron jobs still ride the Connection's default gateway. For full isolation, run the profile's own gateway on its own port and pair as a separate Connection. - Consolidated agent sheet. Deleted standalone
ProfilePicker.ktandPersonalityPicker.kttop-bar chips in favor of a single bottom sheet opened from the top-bar agent name. Sheet holds Profile list, Personality list, and session info + analytics (message count, tokens in/out, avg TTFT). Scrollable. Toast confirmations on switch. Reclaims top-bar real estate and reduces the visual layer count (one tap instead of two chips). - Settings "Active agent" card — top-of-Settings summary of the current Connection / Profile / Personality with a
openAgentSheetnav arg that navigates to Chat and auto-opens the sheet. Closes the "how do I change my agent" discoverability gap for Settings-originating users. - Polish pass (
7031115). Pair wizard URL-scheme cross-validation (inline hint when API field haswss://); pair-stamp of the active Connection's metadata on successful auth (no stale state after re-pair);ConnectionStatusBadgetop-aligns on multi-line rows; Settings treats paired + briefly-disconnected Connection as Connecting (amber) rather than Disconnected (red).
Key architectural decisions.
- Directory-scan for profiles. First pass parsed a fictional top-level
profiles:/agents:list inconfig.yaml. Upstream has never used that shape — profiles upstream are isolated directory instances at~/.hermes/profiles/<name>/, each with its own config,.env,SOUL.md, memory, sessions, and (optionally) its own gateway daemon. Old path always returned empty on real installs. Rewrite: walk~/.hermes/profiles/*/, read config + SOUL, report what's really there. See the "Earlier (abandoned) design" paragraph indocs/decisions.md§21 and commits0303a4f/b9d2914/ec7559c. - Overlay-not-isolation for v1. A profile overrides
model+system_message; everything else (memory, sessions, skills, API keys) rides the Connection's gateway. Rationale: a real per-profile isolation layer is "run the profile's gateway as its own service and pair it as a separate Connection" — we already have the plumbing for that via Multi-Connection. Building a second isolation layer inside one gateway would double the attack surface for no UX win. - Three-layer model = Connection → Profile → Personality. Connection picks the server. Profile picks the agent directory on that server. Personality picks a system-prompt preset within the agent's config. Hierarchy flows top to bottom; picking a Connection resets Profile (Profile is ephemeral and server-scoped).
docs/decisions.md§8 now carries a terminology-note block making this explicit, since the §8 title still says "Profiles" in the legacy sense. - Kept scope intentionally small for v0.6.0. No per-Connection Profile persistence, no gateway-running probe, no mode where one Connection hosts multiple profile-gateways behind one pairing. Those are §21 follow-ups (see Deferred).
Deferred to v0.7+.
- True per-profile isolation via separate Connections. Doc how
hermes -p <name> platform start api --port 8643+ pair-as-new-Connection achieves this today, in user-docs. - Persisted profile selection per Connection. Currently resets on app restart and Connection switch. ~30-line extension: add
lastSelectedProfileNameto theConnectiondata class and restore on switch. - Gateway-running probe (hermes-desktop-inspired). Would let the Connection health indicator distinguish "server unreachable" from "paired, awake, responding" without waiting for a first request to fail. Low-priority; the existing probe behavior is adequate for v0.6.0.
Docs pass.
CHANGELOG.md— new[0.6.0]section above the existing (v0.5.x-work-in-progress) entries.docs/spec.md— Chat top-bar section rewritten around the three-chip reality;auth.okprofiles[]field documented.docs/decisions.md— §8 gained a terminology-note block pointing forward to §19 and §21.user-docs/features/connections.md,profiles.md,personalities.md— top-bar chip references updated to the agent sheet.index.mdfeature grid picked up Connections + Profiles rows.user-docs/guide/chat.md— Personalities section expanded into "Agent Sheet — Profile + Personality" + Connection Chip subsection.getting-started.mdgained a "Multiple Hermes servers" tip pointing atfeatures/connections.md.user-docs/architecture/decisions.md— new ADR-14 (Multi-Connection) + ADR-15 (Agent Profile picker) mirroringdocs/decisions.md.README.md— new "What's new in v0.6.0" block above the feature list; added a feature bullet for multi-Connection + profiles.
2026-04-18 — Dashboard pairing: /pairing/mint schema rework + grants render fix
Context. Bailey hit a silent pairing failure scanning QRs minted by the dashboard's "Pair new device" flow. Logcat showed the app attempting pairing with serverUrl=http://172.16.24.250:8767 (relay's port, not API's) and an empty relay block (relayUrl= and code= both blank), causing applyServerIssuedCodeAndReset to bail with "empty code, returning early — authState NOT reset" and the WSS never handshook.
Root cause. handle_pairing_mint in plugin/relay/server.py was written as if the QR was relay-only: it defaulted top-level host/port/tls to server.config.host/port (which is the RELAY's bind — 0.0.0.0:8767), put the freshly-minted pairing code in top-level key, and emitted only session metadata inside the relay block (ttl_seconds, grants, transport_hint — no url, no code). But the wire format documented at docs/spec.md §3.3.1 and implemented by QrPairingScanner.kt expects top-level = API server (port 8642 default) with relay.{url,code} nested. The CLI path (pair.py → build_payload at line 762) was correct; the endpoint was the outlier. The dashboard plugin's "editable pair URL" feature (commit d7e5fc8) inherited the broken semantics because its request body forwards host/port/tls verbatim to the endpoint.
Fix. handle_pairing_mint now mirrors the CLI shape exactly:
- Top-level
host/port/tlsdefault fromRelayConfig.webapi_url(parsed viaurllib.parse.urlparse; host resolved throughpair._resolve_lan_ipsolocalhost/0.0.0.0become the machine's LAN IP, which is what the phone needs) - Body overrides:
host/port/tls/api_key relay_block["url"]derived from_relay_lan_base_url(server.config.host, server.config.port, tls=bool(server.config.ssl_cert))— the relay knows its own WSS addressrelay_block["code"]carries the minted value (used to live at top-levelkey;keyis now the optional API bearer)
Regression test at plugin/tests/test_pairing_mint_schema.py — 8 AioHTTP cases pinning the payload shape that QrPairingScanner.kt parses, including that the minted code lands in relay.code (not top-level key) and top-level port defaults to 8642 (not 8767). All pass. Test file uses unittest per CLAUDE.md — pytest tripped on conftest.py's responses import on the server venv.
Doc sync. docs/spec.md §3.3.1 already documented the correct wire format — this fix brought the server code back into line. Bumped the Updated stamp from 2026-04-13 to 2026-04-18 and added a line under Implementation references pointing at handle_pairing_mint + the new regression test. CLAUDE.md Key Files rows for plugin/relay/server.py + plugin/dashboard/plugin_api.py updated to reflect the API-server-at-top-level semantics.
Deployment hazard discovered along the way. The server had two parallel checkouts: ~/hermes-relay/ (a dev clone, looked authoritative because SYSTEM.md lists ~/ as the project root) and ~/.hermes/hermes-relay/ (the actual symlinked install per CLAUDE.md). Running git pull in the wrong one updated nothing visible — restart-and-check reported stale version. Verified via systemctl --user cat hermes-relay | grep WorkingDirectory that the installed path is the .hermes/ copy; then pulled + restarted on that one. Also noticed the installed repo was on a long-dead feature/dashboard-plugin branch with 10 local WIP commits (bisect diagnostics from the dashboard build-up). Switched to main (fast-forward clean, 45 commits) after confirming the local feature branch's meaningful work was already on origin/main through the merged PR. The dead branch stays in local refs for history; origin has no matching ref.
Bonus fix in the same branch. Dashboard's Relay Management tab was crashing with minified React error #31 (object with keys {chat, terminal, bridge}) when rendering the paired-sessions list. RelayManagement.jsx:172 wrapped a dict-shaped s.grants in a 1-element array then rendered each entry as a React child — Badge doesn't accept objects. Fix: Object.keys(s.grants) when the value is dict-shaped so Badge children are always channel-name strings. Rebuilt plugin/dashboard/dist/index.js via npm run build (the dashboard loads the pre-built IIFE verbatim, source edits without a rebuild are invisible).
Branch + PR. fix/pairing-mint-schema — two commits (4f0affa, ca50524), pushed, deployed to ~/.hermes/hermes-relay/, restarted, verified live curl /pairing/mint round-trip produces the correct shape. PR open at the origin-suggested URL — merge to land in v0.6.0.
Principle. The original endpoint divergence would have been caught immediately by a schema round-trip test between the server's minted payload and the Android parser's data class. Added that test now as guardrail, not post-mortem — the two implementations will drift again eventually, this time CI yells instead of a silent field-mapping failure.
2026-04-18 — v0.5.1 release prep: lint iterations + VoicePlayerTest deferred
PR #31 (feature/voice-quality-pass → main, v0.5.1 candidate) — CI iteration, no app-behavior changes beyond the preceding v0.5.1 feature commits.
Lint round. PR CI first failed on three latent Android lint errors, which I fixed one-at-a-time because Android lint prints only the first failure before aborting:
UnsafeOptInUsageErroratVoicePlayer.kt:88— Media3ExoPlayer.audioSessionIdis@UnstableApi. Fix: class-level@OptIn(UnstableApi::class). First attempt used Kotlin's@OptIn(implicit import) which Android lint'sUnsafeOptInUsageErrorrule does not recognize; had to explicitlyimport androidx.annotation.OptInto get the AndroidX variant. This distinction is non-obvious from the error text.- Second lint pass then surfaced
FlowOperatorInvokedInCompositionatRelayApp.kt:421and:424— pre-existing onmain, masked behind the first-failure abort. Fix: wrap.map { }chains inremember(repo) { repo.settings.map { ... } }so the Flow is stable across recompositions. - Bailey called out that I should have run
./gradlew lintlocally before pushing instead of burning CI iterations. Codified the rule in CLAUDE.md step 4 of "Typical Dev Loop" + a cross-session feedback memory. Both changes land in PR #31.
Test round — VoicePlayerTest deferred. The voice-quality-pass branch added VoicePlayerTest.kt with mockkConstructor(ExoPlayer.Builder::class) + mockk<ExoPlayer>(). Both trip MockK's Objenesis instantiator into loading Media3's ExoPlayer class, whose static init chain references android.os.Looper — unavailable in JVM unit tests. CI hung for 7+ minutes before we cancelled.
Tried three fixes:
-
Factory seam on
VoicePlayer— addedexoPlayerFactory: (Context) -> ExoPlayer = ::defaultExoPlayerconstructor param so tests inject a mock directly instead of instrumentingBuilder. Production behavior identical; listener attach moved from.also { player -> }toinit { }so it binds to the storedexoPlayerfield. This is kept — clean code win regardless of test outcome. -
Robolectric 4.14.1 dependency +
@RunWith(RobolectricTestRunner::class)+@Config(sdk = [34])— gives the test JVM shadow Android framework classes. Standalone./gradlew test --tests VoicePlayerTestpassed in 2m53s, but the full unit test suite hung indefinitely. Reverted — see below. -
forkEvery = 1on the unit test task to isolate Robolectric's classloader per test class — hypothesis was that Robolectric's shadow framework was bleeding into sibling test JVMs and deadlocking them on coroutine/async code. Full suite hung anyway. Reverted.
Decision. @Ignore VoicePlayerTest for v0.5.1 with strong tracking — kdoc TODO + this DEVLOG entry + deferred_items.md + a GitHub issue linking to commit 456f91e which is the last state where Robolectric was wired. Follow-up PR will likely move VoicePlayer tests into a separate source set (src/robolectricTest/kotlin/) with its own Gradle Test task so it doesn't contaminate the main suite's JVM. Kept the VoicePlayer factory seam refactor (pure win). Reverted Robolectric dep + testOptions config to keep main lean for v0.5.1; the follow-up PR adds them back with the source-set split.
Principle. Bailey flagged during this session: "If we ignore or defer any tests to bypass, document in TODO so we don't forget." Honored via four tracking surfaces (code, DEVLOG, memory, GitHub issue). The alternative — blocking v0.5.1 on test-infra iteration — isn't a better outcome because the app works on device and the tests for it were written on a bad premise (trying to unit-test Media3 without Android framework on the classpath).
Next session. Open the follow-up PR to finish the Robolectric wiring: separate source set, dedicated Gradle task, CI step calling the new task. Then un-@Ignore VoicePlayerTest.
2026-04-18 — Silence auto-stop + bootstrap middleware fix
Context. Bailey asked about STT / silence detection while reviewing the voice stack post-barge-in. Turned out the uplink mic path has no endpointing at all — VoiceRecorder uses MediaRecorder with AudioSource.MIC (no AEC/NS, no VAD), and stopListening() has exactly one caller: ChatScreen.kt:1319 on mic release. The VoicePreferences.silenceThresholdMs preference was wired to a Settings slider and persisted to DataStore but never consumed by any code. Cosmetic setting. Worst case was Continuous mode: after TTS finishes, it re-arms the mic and waits forever for a manual stop — ambient noise gets included, user walks away, recording goes on indefinitely.
Decision. Wire the existing preference. Cheapest of the four options I laid out (wire silenceThresholdMs → AudioSource.VOICE_RECOGNITION → second Silero on uplink → server-side VAD via gpt-4o-transcribe). The other three stay on the shelf; this one is self-contained in the ViewModel.
Implementation.
- Promoted
voicePreferencesto a VM field (previously captured only insideinitialize's closure). - Added
silenceWatchdogJob+startSilenceWatchdog()— snapshotssilenceThresholdMsat turn start, pollsVoiceRecorder.amplitudeevery 150 ms, reusesRESUME_SILENCE_THRESHOLD = 0.08fas the silence floor (same floor the post-barge-in resume watchdog already uses — one tuning knob, not two). Grace window: auto-stop only fires after at least one above-floor frame, so "user taps mic, doesn't speak" never insta-closes the turn. - Skip in
HoldToTalk— the physical release is the authoritative stop there, auto-stop on silence while the user is still holding would be surprising behavior. - Cancelled in
stopListening(),interruptSpeaking(), andonCleared().
No UI change needed — the Settings slider and "Auto-stop listening after this much silence" copy already exist. The slider is now load-bearing instead of cosmetic.
Bootstrap crash (separate fix, same day). Teammate debugging a different project hit 'tuple' object has no attribute 'freeze' on gateway startup. Root cause in hermes_relay_bootstrap/_command_middleware.py: maybe_install_middleware() was doing app._middlewares = (*existing, middleware) — replacing the aiohttp FrozenList with a plain tuple. When AppRunner.setup() later called _middlewares.freeze(), tuples don't have that method and the gateway crashed. Switched to in-place app._middlewares.append(middleware) — the FrozenList is still mutable at middleware-install time (freeze happens later in setup). Also updated the test mock to use [] instead of () so it matches real aiohttp behavior. 31/31 tests pass.
Deferred (from the silence-detection discussion).
- Uplink
AudioSource.VOICE_RECOGNITION— engages Android's built-in NS/AEC tuned for STT. Single-line change, needs a release to validate on the Samsung. - Second Silero on the uplink path — real VAD-based endpointing instead of raw-amplitude. Barge-in infra is already there to copy.
gpt-4o-transcribe/ OpenAI Realtime — server-side endpointing + lower latency, but bigger pivot. Park until we hit real limits with the amplitude approach.- "Off" option on the silence slider — the current slider range is 1000–10000 ms with no way to disable. Future UI change if a user wants fully manual control (e.g., pause-to-think flows). Treating thresholdMs <= 0 as "off" is already supported in the watchdog for when that UI lands.
Next session. Bailey rebuilds, smoke-tests Continuous + TapToTalk at the default 3s threshold. Validation: (1) natural pauses mid-utterance don't prematurely auto-stop at 1s/2s thresholds — if they do, tune the floor up or add a longer default; (2) the watchdog never fires when the user is manually holding for HoldToTalk; (3) ambient noise in a moderately-noisy room (TV, fan) doesn't prevent auto-stop because amp never drops below 0.08. If (3) bites, that's when AudioSource.VOICE_RECOGNITION becomes non-optional.
2026-04-17 — Voice barge-in (agent-team single-session, stacked on voice-quality-pass)
Motivation. After the voice-quality-pass work (see entry below), Bailey noted that conversation mode still lagged ChatGPT / Claude app in a specific way: you can't interrupt the agent by speaking. You have to wait for the full response, or tap the mic to force a new turn — neither of which matches the "normal turn-taking" pattern users expect from modern voice UIs. The industry-standard recipe is duplex audio + VAD + AEC + hysteresis + optional resume-from-sentence-boundary. All five pieces implementable on Android if you accept that AEC quality varies across OEMs (the trap that kills most naive attempts).
Design session. Bailey asked for a "quick win" barge-in implementation; I pushed back that the real engineering is the false-positive defense stack — the VAD is the easy part, fighting AEC variance across Samsung / Pixel / Motorola devices is the hard part. Landed on a three-layer defense: (1) AcousticEchoCanceler attached to the ExoPlayer audio session, (2) VAD hysteresis requiring 2–3 consecutive speech frames (on top of the Silero library's own attack/release windows), (3) soft ducking on single-frame positives before hard cutoff, so a stray frame only briefly dips the volume. Plus a compatibility-warning UI strip for devices without AEC so users know why quality varies.
Bailey also wanted the feature to match our existing voice-settings pattern — always-visible toggle with a sensitivity picker and a resume sub-toggle, not a hidden flag. Good call — means the control surface is discoverable and the setting can be tuned per user preference rather than requiring a global "works for everyone" default that doesn't exist on Android's mic landscape.
Plan written to docs/plans/2026-04-17-voice-barge-in.md, 6 work units (B1 prefs, B2 VAD, B3 duplex listener, B4 integration, B5 UI, B6 ducking helpers) + Doc2. Branched from feature/voice-quality-pass rather than origin/main because barge-in architecturally depends on V4's supervisor-scoped cancellation + pendingTtsFiles cleanup, V5's VoicePlayer.audioSessionId exposure (needed for AcousticEchoCanceler.create()), and V3's sentence-chunk tracking (needed for resume-from-next-sentence). Merges after voice-quality-pass lands; stacked PRs.
Execution. Worktree at ../hermes-android-barge-in from feature/voice-quality-pass. Branch feature/voice-barge-in. Seven commits on top of the plan:
- Wave 1 (parallel, disjoint files). B1
c4fadf3BargeInPreferences DataStore + B23d011d9Silero VAD engine + B6de925c8VoicePlayer ducking helpers. Three subagents, clean touch surface. B2 agent discovered the library's actual frame-size constraint at 16 kHz is{512, 1024, 1536}(plan said 640 — that was wrong, from an older library version), pinned2.0.10not2.0.8(latest stable), added jitpack repo tosettings.gradle.ktssince android-vad isn't on Maven Central, and noted the library exposes threshold via aModeenum (OFF / NORMAL / AGGRESSIVE / VERY_AGGRESSIVE) not a float — so our user-facingOff/Low/Default/Highpicker has to invert ("aggressive" means less sensitive in the library). Parallel agents also hit a staging race where one agent'sgit commitinitially swept another's files; recovered viagit reset+ explicit pathspec. Noted for future parallel-worktree playbooks — always commit with explicit pathspecs when multiple agents share a tree. - Wave 2 (parallel, disjoint files). B3
48259e1duplex listener + B5c65f1feVoice Settings UI. B3 agent introduced anAudioFrameSourceinterface seam so tests can inject a scripted frame queue without Robolectric —AudioRecordis a final class with native state, impossible to mock cleanly without the seam. Also added areaderDispatcherparam sorunTestvirtual time controls the reader loop deterministically. B5 agent found no existingVoiceSettingsViewModeland created one following theBridgeViewModelpattern; choseSingleChoiceSegmentedButtonRowfor the 4-option sensitivity picker (matching theChatSettingsScreenprecedent, not a dropdown). Section heading "Barge-in" chosen over "Interruption" because every other Voice Settings section is feature-named ("Text-to-Speech", "Speech-to-Text"). AEC compatibility badge renders always (not gated on toggle enabled) so users see the device capability before they flip it. - Wave 3 (serial, VoiceViewModel integration). B4
3e39663. ThreadedcurrentChunkIndexthrough V4's play worker without touching V4's supervisor topology — addedMutableStateFlow<Int>on the VM and wired it via V4's existingonFileReadycallback lambda, so the top-levelrunTtsPlayWorkercontract stays unchanged. Chunk tracking is layered in VM-owned lambdas, not the workers. For the "user still speaking" cancel signal on the resume watchdog, the agent usedVoiceRecorder.amplitude(perceptually-scaled StateFlow, already exposed) rather than re-subscribing a fresh AudioRecord — simpler and cheaper, since the recorder is already hot from the barge-in pre-warm. Added three@VisibleForTesting internaltest seams (startBargeInListenerForTest,seedSpeakingStateForTest,drainTtsQueueForTest) so the coordinator's 7-case test matrix runs without pushing real MP3s through the V4 pipeline. Found one additional race: a synthesize call in flight when interrupt fires can complete after the cancel signal and add a file topendingTtsFilesthat the immediate cleanup misses — closed via the supervisor'sfinallyblock (same pattern V4 established). - Wave 4. Doc2 (this entry) — DEVLOG, CHANGELOG
[Unreleased]gains an Added section above the voice-quality-pass Changed section,docs/spec.mdPhase V gets barge-in bullets,user-docs/features/voice.mdgains a "Barge-in (interrupt the agent)" section and updates the Tap-mode description (the existing "tap to interrupt" is now one of three ways to interrupt).
Results — expected and TBD. Net change on top of voice-quality-pass: ~3,000 insertions / 5 deletions across 16 files. New tests: VAD engine (scripted frame sequences), duplex listener (via AudioFrameSource seam), ducking helpers, preferences round-trip, and the 7-case VoiceViewModel coordinator suite. Bailey's on-device smoke test still pending — key things to validate: (1) toggling barge-in on + speaking during TTS cuts playback within ~100 ms on headphones, (2) 600 ms of silence after interrupt resumes from next sentence if resume is enabled, (3) devices without AEC show the compatibility warning badge and (presumably) trigger more false positives in practice. Plan to leave barge-in default-off through first release; telemetry or Bailey's own use will inform whether we change the default.
Deferred. Same Up1 from voice-quality-pass — upstream PR exposing VoiceSettings in hermes-agent/tools/tts_tool.py::_generate_elevenlabs. Not blocked by barge-in. Also deferred: live sensitivity reactivity mid-Speaking (the coordinator rebuilds VAD on next listener start rather than re-applying sensitivity on an already-running listener — simpler, and users flipping sensitivity during a turn is a niche workflow).
Open questions for next session.
- Does the ducking feedback feel natural on-device, or should we skip the
maybeSpeechstage entirely and just hard-cut on hysteresis pass? The 30 % ducking level is a guess. - Should the resume watchdog be shorter than 600 ms? A breath is more like 300–400 ms; 600 ms might feel sluggish. Tune after Bailey's smoke test.
- Telemetry for AEC-unavailable devices — if a non-trivial portion of users see the badge, is the UX degraded enough that we should gate-off barge-in entirely on those devices, rather than just warning?
- Integration with voice-intent direct-dispatch (from 2026-04-15): if the user barges in with "cancel" during a destructive-action countdown, does that route through the existing cancel handler or the new barge-in path? Probably needs a small coordination test.
Next session. Bailey smoke-tests. If quality holds: rebase onto main after voice-quality-pass merges, open stacked PR feature/voice-barge-in → main. If quality is poor on his device: tune RESUME_SILENCE_THRESHOLD, hysteresis consecutive-frame counts, or default-off sensitivity preset.
2026-04-17 — Voice quality pass (agent-team single-session)
Motivation. Bailey's ongoing voice-mode testing with an ElevenLabs-configured Hermes backend surfaced a cluster of quality issues that individually read like different bugs but shared compounding root causes: voice output noticeably switching between crisp and muffled, volume drifting between sentences, audible pauses mid-response, and the occasional jumbled-letter spell-out when the agent emitted markdown, URLs, or tool-annotation tokens. ChatGPT- and Claude-app-class voice experiences don't exhibit this — diagnosis focused on what they do differently.
Diagnosis chain (2026-04-16). Traced the voice pipeline end-to-end across the Windows checkout, the relay's plugin/relay/voice.py, and the upstream ~/.hermes/hermes-agent/tools/tts_tool.py on hermes-host. Five compounding layers:
- Server config used
eleven_multilingual_v2— ElevenLabs' expressive long-form model, which re-interprets prosody per call. Wrong model for a chunked per-sentence pipeline; every sentence is a fresh TTS context with independent encoder gain and EQ. Accounts for the bulk of the "clear↔muffled" perception. - Upstream
_generate_elevenlabs()(tts_tool.py:222) passes noVoiceSettings— all defaults, includingstabilitythat's fine for long-form but too low for chunked output.use_speaker_boost(auto-normalizing) is off. Accounts for inter-chunk volume drift. - Upstream has
_strip_markdown_for_tts()(tts_tool.py:1053) but it's only called from the CLIstream_tts_to_speaker()path.text_to_speech_tool(), which our relay calls, does NOT run the stripper. Our relay's/voice/synthesizepasses raw assistant text to ElevenLabs — tool-annotation tokens like`💻 terminal`and URLs likehttps://github.com/foo/barget spoken character-by-character. Accounts for jumbled letters. VoicePlayer.play(file)on Android tore down and rebuilt aMediaPlayerper sentence — codec context reset, audio-track re-init, audible seam/pop. Accounts for playback-layer portion of the muffled switching.VoiceViewModel.startTtsConsumerwas strictly serial:synthesize → play → awaitCompletion → next synthesize. Every sentence boundary cost one full network round-trip. Accounts for the inter-sentence pause.
Plan written to docs/plans/2026-04-16-voice-quality-pass.md covering 8 work units across 5 waves, scoped to a single feature branch in a worktree.
Execution (2026-04-17). Worktree at ../hermes-android-voice from origin/main. Single branch feature/voice-quality-pass. Six commits on top of the plan:
- Wave 0 (operator, pre-session).
Cfg1—~/.hermes/config.yamlon hermes-host flipped fromeleven_multilingual_v2→eleven_flash_v2_5;hermes-gateway.servicerestarted. Voice ID unchanged. - Wave 1 (parallel, disjoint files).
V13618bcdrelay TTS sanitizer +V5b24a37eExoPlayer gapless playback. Two subagents, completely disjoint touch surface (plugin/relay/*vsapp/src/.../audio/VoicePlayer.kt). No conflicts. V5 agent flagged that the plan'sConcatenatingMediaSourceis deprecated in Media3 1.x and usedaddMediaItemon a single persistent ExoPlayer instead — same effect, current API. - Wave 2.
V2a13dc3b— client-sidesanitizeForTtsmirroring V1's regex set, applied per delta before sentenceBuffer append, with multi-delta code-fence deferral (apendingRawDelta: StringBuilderthat buffers until opening/closing fence counts balance). V2 agent caught a real subtle bug: the plan's Python[emoji1 emoji2 ...]character class translates badly tojava.util.regex, which operates on UTF-16 code units — non-BMP emojis would match individual surrogate halves and leave orphan low-surrogates in the output. Switched to(?:A|B|C|...)alternation groups. Kotlin testability pattern used top-levelinternalfunctions (matching existingextractNextSentence) rather than@VisibleForTestingclass members. - Wave 3.
V3f916b22—MIN_COALESCE_LEN=40+MAX_BUFFER_LEN=400secondary-break escape + 800 ms idle-delta timer. SpikedICU4J BreakIterator.getSentenceInstance()per the plan's suggestion but rejected it: its locale-sensitive abbreviation handling is unpredictable on partial buffers left by V2's sanitizer, and it can't be parameterized with thestreamComplete/coalescing semantics without reinventing the loop on top of it. Extended hand-rolled regex is deterministic and cheap. Secondary-break pattern requires comma-followed-by-whitespace so"3,000"survives. Added@Volatile streamComplete: BooleanonVoiceViewModel, set bystartStreamObserverwhen the assistant stream flipsisStreaming=false, reset in turn-start /exitVoiceMode/interruptSpeaking. The idle-flush timer passesstreamComplete=truetoextractNextSentencefor that call only — does NOT mutate the class flag, because the SSE stream may still be live and the next delta should return to conservative coalescing. - Wave 4.
V48fcec6b—startTtsConsumersplit into twosupervisorScope-rooted coroutines joined byChannel<File>(capacity=2). Synth worker runs one file ahead. V4 agent chose Option B (explicit Channel) over the plan's recommended Option A (rely on ExoPlayer's internal queue for backpressure) becauseVoicePlayer.mediaItemCountisn't public and V4 was guardrailed out of touching V5's work. Play worker still callsawaitCompletion()per file (V5's queue-drained semantic), so ExoPlayer's internal queue never exceeds 1 — a single queuing layer, not redundant. Found and guarded a real race:ttsConsumerJob?.cancel()is non-blocking, so a synthesize call in flight can complete AFTER cancel was signalled and add a file topendingTtsFilesthatinterruptSpeaking's immediate cleanup would miss. The supervisor'sfinally { deletePendingSynthFiles() }catches late arrivals. Consumer is now cancellable+restartable — interrupt/exit cancel the job, clean up files, and immediately callstartTtsConsumer()for the next turn. - Wave 5.
Doc1(this entry) — DEVLOG, CHANGELOG[Unreleased], spec Phase V section updated to reflect the new architecture.
Deferred. Up1 — upstream PR exposing VoiceSettings (stability / similarity_boost / use_speaker_boost) in hermes-agent/tools/tts_tool.py::_generate_elevenlabs. Useful once merged but not blocking; the flash model already reduces inter-chunk variance enough that the missing settings don't dominate the remaining gap.
Results — expected and TBD. Net change: +2,998 / -197 lines across 15 files. New tests: 33 relay sanitizer unit tests (all green) + 4 new client test files (sanitization parity, chunking semantics, prefetch pipelining timing + cancellation cleanup, ExoPlayer queue behavior — verified by reading, not by gradle test per project convention). Bailey's on-device smoke test (3-sentence response, 1 code-block response, 1 multi-paragraph response) still pending; will capture subjective before/after in the PR description.
Rebase note (2026-04-17). This branch was rebased onto main at v0.5.0 (from earlier base b811da1). Conflicts resolved: a single-import collision in VoiceViewModel.kt (kept both supervisorScope + contentOrNull), DEVLOG + CHANGELOG prepends (combined both days' entries in reverse chronological order, this entry placed above the v0.4.1 bridge polish entry since VQP is the newer work on 2026-04-17), and a docs/spec.md auto-merge. No code logic changed during rebase.
2026-04-18 — Dashboard plugin
Context. The upstream Dashboard Plugin System landed on hermes-agent's axiom branch in three commits (01214a7f plugin system, 3f6c4346 theme, 247929b0 OAuth providers). The gateway now scans ~/.hermes/plugins/<name>/dashboard/manifest.json at startup and exposes registered plugins as tabs in its web UI. Since ~/.hermes/plugins/hermes-relay already points at our plugin/ subtree (from install.sh), we can ship a relay-specific dashboard plugin by only adding a plugin/dashboard/ subtree plus a few relay HTTP routes — no hermes-agent fork needed. The four deferred-audit items that only the relay knows about — paired-device state, bridge command history, push delivery (future), media-registry tokens — map cleanly onto four tabs of a single plugin.
Decision summary.
- Single plugin with internal shadcn
Tabs, not four plugins. Manifest allows onetab.pathper plugin; four plugins would fragment the nav. Recorded as ADR 19 indocs/decisions.md. - Pre-built IIFE bundle committed to git. Upstream's example plugin uses plain
React.createElement(no build), but four non-trivial tabs are painful to maintain that way. Source lives underplugin/dashboard/src/, bundled with esbuild to a single ~16 KB IIFE atplugin/dashboard/dist/index.js. Dashboard never runs the build — operators get a ready-to-serve bundle. - Loopback-only for the new relay routes.
/bridge/activity,/media/inspect,/relay/infoare gated the same way/bridge/statusand/pairing/registeralready are. The plugin backend runs inside the gateway process (also localhost) and calls the relay athttp://127.0.0.1:8767/...— no bearer minting. Also added a loopback-exempt branch on/sessionsso the plugin proxy can list paired devices without one. - Dashboard backend is a thin proxy.
plugin/dashboard/plugin_api.pyexposes five routes at/api/plugins/hermes-relay/*and forwards to the relay. No business logic in the plugin — relay stays source of truth. - Push Console is a real tab, stub data. Keeps the four-tab nav layout correct for when FCM lands; swapping in real data is additive.
Implementation (Wave-by-wave).
- Wave 1 (relay state plumbing).
2212fbc — feat(relay): add MediaRegistry.list_all for dashboard inspectoradded the lock-guarded snapshot method that strips absolute paths, returns basename-onlyfile_namefields, and filters expired entries by default (+8 unit tests).777a06a — feat(relay): add bridge command ring buffer for dashboard activity feedadded theBridgeCommandRecorddataclass +deque(maxlen=100)onBridgeHandler, wired append/update intohandle_command()/handle_response(), and taughtget_recent()to redact params keyed on{password, token, secret, otp, bearer}(+9 unit tests covering append, update, timeout path, redaction, ring-buffer eviction). - Wave 2 (relay HTTP routes).
4370806 — feat(relay): add /bridge/activity /media/inspect /relay/info for dashboardadded the three loopback-gated handlers inplugin/relay/server.pyadjacent to/bridge/status(factored through a_require_loopback()helper), plus the tinyhandle_sessions_listloopback-exempt branch so the dashboard proxy can list paired devices without a bearer. Route tests extendtest_bridge_activity.py+test_media_inspect.pyand a newtest_relay_info.pycovers the aggregate status shape. - Wave 3 (dashboard plugin body).
b51940c — feat(dashboard): add plugin_api.py proxy to relay HTTPadded the FastAPI router with five routes (/overview,/sessions,/bridge-activity,/media,/push), a shared_proxy_get()helper, and structured 502 translation on relay connect-error / timeout / 5xx so the UI can show "relay unreachable" (+10 unit tests inplugin/dashboard/test_plugin_api.py).087149e — feat(dashboard): add React UI for four relay tabsadded theplugin/dashboard/{src,dist}/subtree — JSX sources undersrc/(index + four tab components +lib/api.js+lib/formatters.js), esbuild toolchain (package.json+build.sh), and the committed ~16 KB IIFE bundle atdist/index.jsthat registers aswindow.__HERMES_PLUGINS__.register("hermes-relay", …)via the SDK global. - Wave 4 (manifest wiring).
78c209e — feat(dashboard): add plugin manifest + plan fileaddedplugin/dashboard/manifest.json—name: "hermes-relay",label: "Relay",icon: "Activity"(from the 20-name Lucide whitelist),tab.path: "/relay",tab.position: "after:skills",entry: "dist/index.js",api: "plugin_api.py". Verified the existing~/.hermes/plugins/hermes-relaysymlink resolvesdashboard/manifest.jsoncorrectly.
Files touched (this session — Doc1 wave only).
CHANGELOG.md— new[Unreleased]"Added — Dashboard plugin" bullet groupDEVLOG.md— this entryREADME.md— one-line mention under Quick StartCLAUDE.md—plugin/dashboard/added to Repository Layout; four Key Files rows under a new "Plugin — Dashboard" groupdocs/spec.md— "Dashboard plugin" subsection appended to §10 Hermes Integration Pointsdocs/decisions.md— new ADR 19 "Dashboard plugin: single plugin with internal tabs + pre-built IIFE bundle"docs/relay-server.md— three new rows in the HTTP Routes table + loopback-branch note on/sessionsuser-docs/features/dashboard.md— new user-facing pageuser-docs/.vitepress/config.mts— sidebar nav entry for dashboard.md
Deferred.
- Push Console needs FCM. The tab renders a static "FCM not configured" banner from
GET /api/plugins/hermes-relay/push. Swapping in real data is additive — onlyPushConsole.jsx+plugin_api.py::get_pushchange. Tracked under thedeferred_itemsmemory entry. - Session revoke button is a placeholder. The Relay Management tab exposes "Revoke" buttons per paired device but they currently log to the console — the plugin proxy would need a new
DELETE /api/plugins/hermes-relay/sessions/{prefix}route forwarded to the relay's existingDELETE /sessions/{token_prefix}. Blocked on deciding the auth story (loopback-only? dashboard session token?). Keeps the UI placement correct for when the proxy route lands. - Session extend button not yet built; would follow the same proxy pattern against
PATCH /sessions/{token_prefix}.
Next session. Bailey restarts hermes-gateway on the server, verifies GET /api/dashboard/plugins includes hermes-relay, loads the dashboard UI, and exercises all four tabs against a live paired phone. Screenshot the tabs so we can drop real images into user-docs/features/dashboard.md in place of the placeholder captions.
No blockers.
2026-04-17 — v0.4.1 Bridge page polish pass
Motivation. The Bridge tab had accreted cards without an information hierarchy — the master toggle, a separate status card, the permission checklist, unattended access, a standalone keyguard warning chip, and the safety summary all sat at the same visual weight, and tapping the master Switch when Accessibility wasn't granted was a silent no-op (stock Android disabled-switch behavior). Unattended Access lacked a master dependency, and once ON there was no in-app affordance while the user was on another tab. The LLM also had to learn reactively that commands wouldn't reach apps on a sleeping/locked device (via keyguard_blocked errors) instead of being told upfront.
What landed on feature/v0.4.1-integration:
- Bridge card hierarchy rewrite. New order: Master → Permission Checklist → [Advanced divider] → Unattended Access → Safety Summary → Activity Log. Master toggle copy now leads with "Master switch —" and carries a
MASTERpill so its parent-gate role is legible at a glance. The oldBridgeStatusCardwas dropped from the layout (its status rows already live inside the master card); the component file stays in-tree for now. - Master-toggle feedback fix. Tapping the Switch when
HermesAccessibilityServiceisn't granted now surfaces a snackbar ("Accessibility Service must be enabled first.") with an "Open Settings" action that deep-links toACTION_ACCESSIBILITY_SETTINGS, instead of the previous silent-drop disabled-switch behavior. The master-toggle info dialog also gained a paragraph attributing the "Hermes has device control" persistent notification to the master switch. - Unattended Access gated on master.
UnattendedAccessRow's Switch isenabled = masterEnabledand shows "Requires Agent Control — enable the master switch above first." when master is off. The standaloneKeyguardDetectedChipwas inlined as aKeyguardDetectedAlertSurface band inside the Unattended Access card so the credential-lock warning lives next to the thing that triggers it. Unattended scary-dialog copy no longer implies the unattended toggle owns the persistent notification — attributes it correctly to the master switch. UnattendedGlobalBanner— in-app banner vs. system chip split. New 28dp amber strip renders at the top ofRelayApp's scaffold on every tab when master + unattended are both on (sideload only). Theme-aware amber (amber-on-dark vs. dark-amber-on-pale in light mode), pulsing dot, "Unattended access ON — agent can wake and drive this device" copy, chevron → navigates to the Bridge tab. The existing WindowManagerBridgeStatusOverlayChipcontinues to handle the backgrounded-app case (foreground-gating of the chip is in flight on a sibling branch). Net: banner when the user is IN Hermes-Relay, system chip when the app is backgrounded.- Agent-awareness upgrades to
PhoneSnapshot. Three new fields:unattendedEnabled,credentialLockDetected,screenOn.PhoneStatusPromptBuilder.buildBridgeLine()appends explicit guidance so the LLM knows upfront whether commands will land while the user is away, instead of finding out reactively viakeyguard_blockedresponses. - Bug fixes bundled in. Permission checklist Optional pill no longer wraps mid-pill on narrow titles (switched to
FlowRow+softWrap=false); runtime-permission rows (Mic, Camera, Contacts, SMS, Phone, Location) now fall back toSettings.ACTION_APPLICATION_DETAILS_SETTINGSwhen a permission has been permanently denied, instead of silently no-opping.
Next. Bailey's on-device verification of the reordered layout, master-snackbar, global banner, and PhoneSnapshot fields on the Samsung S24 / Android 14 sideload build. After that, version bump to v0.4.1 release tag via bash scripts/bump-version.sh 0.4.1 → push tag → CI.
No blockers.
2026-04-16 — v0.4.1: Tiered permission checklist + JIT permission-denied surfacing
Motivation. Two v0.4.x fast-follows from ROADMAP.md, scoped to ship together because they share the same UX axis ("the user understands which permission is missing and what to do about it"). Until now the Bridge tab's checklist was a flat 4-row layout that lumped optional and required perms together, and android_search_contacts / android_send_sms / android_call / android_location failures bubbled up as opaque error strings — the LLM had to pattern-match the phrasing to figure out it was a permission issue. Both surfaces now make the missing-permission state legible to humans AND to the agent.
What landed.
- Tiered checklist UI (
BridgePermissionChecklist.ktrewrite). Four explicit sections — Core bridge (required), Notification companion (optional), Voice & camera (optional), Sideload features (optional, sideload-only). Each row uses arememberLauncherForActivityResult(ActivityResultContracts.RequestPermission)for runtime dangerous perms and the existing intent helpers for special perms (Accessibility, Notification Listener, Overlay, Screen Capture). Optional rows render a "Optional" Material 3 pill so users don't perceive them as urgent. Section headers use a primary-coloured label + caption + thin divider. Hides sideload-only sections on googlePlay via the existingBuildFlavor.isSideloadgate. BridgePermissionStatusextended withmicrophonePermitted,cameraPermitted,contactsPermitted,smsPermitted,phonePermitted,locationPermitted.refreshPermissionStatus()probes each viaContextCompat.checkSelfPermission. Re-runs on everyLifecycle.Event.ON_RESUME(existing pattern).- Manifest verification. All required
<uses-permission>declarations were already in place:RECORD_AUDIO+CAMERAinapp/src/main/AndroidManifest.xml,READ_CONTACTS+SEND_SMS+CALL_PHONE+ACCESS_FINE_LOCATION(+ACCESS_COARSE_LOCATION) inapp/src/sideload/AndroidManifest.xml. No manifest changes needed for this PR. ResolveResulttyped-union shipped inplugin/tools/resolve_result.py—Found(value)/NotFound(detail)/PermissionDenied(permission, reason)dataclasses with afrom_bridge_response(response, *, found_value=None)constructor that classifies a bridge response as one of the three variants. Reads both canonical (code/permission, v0.4.1) and legacy (error_code/required_permission, pre-v0.4.1) wire-key spellings so the rollout is forwards/backwards compatible across mixed-version installs.- Bridge response envelope extended in
BridgeCommandHandler.kt::respondFromResultto emit the canonicalcode+permissionaliases ALONGSIDE the existingerror_code+required_permissionfields.LocalDispatchResultparsing also accepts either spelling. The phone now produces{"ok": false, "error": "...", "code": "permission_denied", "permission": "android.permission.READ_CONTACTS", "error_code": "permission_denied", "required_permission": "android.permission.READ_CONTACTS"}. - Tier C agent-tool wrappers upgrade permission errors to JIT structured responses.
android_location/android_search_contacts/android_send_sms/android_callinplugin/tools/android_tool.pynow run their bridge response through_maybe_jit_permission_responseafter the HTTP round-trip. Oncode: permission_deniedthe wrapper returns{"ok": false, "error": "User has not granted Contacts permission (android.permission.READ_CONTACTS). They can enable it in Settings > Apps > Hermes Relay > Permissions. Tool: android_search_contacts.", "code": "permission_denied", "permission": "..."}— deterministic LLM-readable copy with the exact Settings deep-link path embedded. - Voice-mode JIT chip in
VoiceModeOverlay.kt. NewPermissionDeniedChipcomposable surfaces above the mic button when the most recent voice intent dispatch returnederrorCode == "permission_denied". Tap deep-links toSettings.ACTION_APPLICATION_DETAILS_SETTINGSforBuildConfig.APPLICATION_ID(so both flavors land on their own package's permission page).VoiceUiState.permissionDeniedCallout: PermissionDeniedCallout?carries the chip state.buildPermissionDeniedCallout(label, result)reads the structuredpermissionfield offresult.resultJsonand builds a copy line ("I need Contacts to Search contacts here. Tap to open Settings."). Cleared on chip tap and on the next mic-tap (fresh turn). - Voice TTS on permission_denied was already in place from the 2026-04-15 voice fixes session —
speakDispatchResultalready says "Permission needed. {hint}" in this branch. The chip is purely additive. - 17 new Python tests in
plugin/tests/test_resolve_result.py. Covers the ResolveResult classifier, both canonical/legacy/mixed wire-key spellings, success passthrough, non-permission error passthrough, and JIT upgrades for all four Tier C wrappers. All 17 pass; existing 39 Tier-C tests still pass with no regressions.
Branch. feature/tiered-permissions (forked from main, with feature/bridge-notifications-permission merged in first per the spec — that branch's POST_NOTIFICATIONS row sits inside the new Core-bridge tier without modification). Pushed to origin, PR-ready.
Files touched (summary).
app/src/main/kotlin/com/hermesandroid/relay/ui/components/BridgePermissionChecklist.kt(rewrite — tiered layout)app/src/main/kotlin/com/hermesandroid/relay/viewmodel/BridgeViewModel.kt(status fields + probes)app/src/main/kotlin/com/hermesandroid/relay/ui/screens/BridgeScreen.kt(6 new permission launchers, wired into checklist)app/src/main/kotlin/com/hermesandroid/relay/network/handlers/BridgeCommandHandler.kt(canonical alias emission, dual-key parse)app/src/main/kotlin/com/hermesandroid/relay/viewmodel/VoiceViewModel.kt(PermissionDeniedCallout state, buildPermissionDeniedCallout, clearPermissionDeniedCallout)app/src/main/kotlin/com/hermesandroid/relay/ui/components/VoiceModeOverlay.kt(PermissionDeniedChip composable + signature param)app/src/main/kotlin/com/hermesandroid/relay/ui/screens/ChatScreen.kt(chip-tap callback wiring deep-link to Settings)plugin/tools/resolve_result.py(NEW — typed-union dataclass hierarchy + classifier)plugin/tools/android_tool.py(_maybe_jit_permission_responsehelper + 4 wrappers updated)plugin/tests/test_resolve_result.py(NEW — 17 unit tests)ROADMAP.md,CHANGELOG.md,DEVLOG.md(this entry)
Notable decisions.
- Did not introduce a server-side resolver layer. The prompt suggested Python resolvers like
resolveContactPhone, but the actual contact resolution lives in Kotlin (ActionExecutor.searchContacts). Implementing the spec verbatim — PythonResolveResulttypes — still gave value as the agent-tool wrapper's classifier of bridge responses, which is the layer that needed the structured permission-denied surfacing for the LLM. The KotlinContactResolutionsealed-class equivalent already exists from 2026-04-15. - Both wire-key spellings emitted by the phone (canonical + legacy). Forwards/backwards compatibility across the v0.4.x APK rollout window. Drop the legacy aliases after v0.5.0 once the 0.4.0 APKs are estimated to be off the field.
- JIT chip uses errorContainer colour, not a notification-style banner. Voice mode is a focused single-purpose modality; banner-treatment would compete with the destructive-countdown row. Chip is explicit and tappable but doesn't block the voice flow.
- Optional rows use a neutral status tint when not granted. Original layout used error-red for any not-granted row, which made optional perms feel urgent — the v0.4.1 layout treats neutral-grey + optional-pill as the "this is fine if you skip it" affordance.
- Sideload-only rows hidden entirely on googlePlay rather than rendered as disabled. Showing a permission row for a manifest entry that doesn't exist would confuse users and potentially flag review on the Play track.
Verification.
python -m unittest plugin.tests.test_resolve_result— 17 tests pass.python -m unittest plugin.tests.test_android_call plugin.tests.test_android_send_sms plugin.tests.test_android_search_contacts plugin.tests.test_android_location— 39 existing tests still pass.python -m py_compile plugin/tools/resolve_result.py plugin/tools/android_tool.py— clean.- Kotlin code reviewed for type correctness and import resolution; no
gradle buildper project convention (Bailey builds from Studio).
Next session. Bailey's on-device retest of the tiered checklist + JIT chip flow on Samsung S24 / Android 14 sideload APK. After that, version bump to v0.4.1 + cut release.
2026-04-15 — Bootstrap command middleware (slash-command hallucination fix)
Motivation. When a user types /help or /model into the Android app's chat, the message reaches the LLM verbatim on vanilla upstream hermes-agent installs because APIServerAdapter doesn't connect to the GatewayRouter. The LLM hallucinates a plausible-sounding wrong reply. The upstream fix lives in gateway/platforms/api_server_slash.py (Stage 1 PR), but vanilla installs won't get it until that PR merges. This session implements the bootstrap-middleware sibling that ships the fix now.
What was built:
-
hermes_relay_bootstrap/_command_middleware.py(~420 LOC) — aiohttp middleware that:- Intercepts
POST /v1/chat/completionsandPOST /v1/runs(zero-cost skip for all other paths) - Runs auth check (
adapter._check_auth) before any command logic - Extracts the user message from each endpoint's body format
- Resolves against
hermes_cli.commands.resolve_command()from the central command registry - Stateless commands (
/help,/commands,/profile,/provider) are dispatched locally using the same helpers as upstream - Stateful commands (
/model,/new,/retry, etc.) return a deterministic decline notice - Unknown and CLI-only commands fall through to the LLM
- For
/v1/chat/completions: returns synthetic SSE stream (role + content + finish +[DONE]) or JSONchat.completion - For
/v1/runs: injectsmessage.delta+run.completed+ sentinel intoadapter._run_streamsqueue, returns{"run_id": ..., "status": "started"}with 202, so the events endpoint drains the queue normally - Feature-detects: if
gateway.platforms.api_server_slashexists (upstream PR landed), middleware is skipped - Fail-open: any exception falls through to the original handler
- Intercepts
-
hermes_relay_bootstrap/_patch.py— added_maybe_install_command_middleware()call in the__setitem__hook alongside the existing route injection, using the same timing window whereapp._middlewaresis still mutable. -
hermes_relay_bootstrap/__init__.py— updated docstring to document both injection layers (routes + middleware). -
plugin/tests/test_command_middleware.py(31 tests) — unit tests for message extraction, command resolution, response builders, feature detection, plus full aiohttp integration tests viaTestClient/TestServercovering both endpoints, streaming, auth rejection, and fall-through behavior.
Test results: 31 new tests pass. Full existing suite (450 tests) still passes with no regressions.
Android client impact: None. The synthetic responses match the exact shapes the client already parses (chat.completion.chunk for completions SSE, message.delta/run.completed for runs events).
Next: Merge to main via PR, then deploy to server (git pull && systemctl --user restart hermes-gateway). Phone re-pair after restart.
2026-04-16 — v0.4.1 unattended access mode (sideload-only)
Motivation. First v0.4.1 fast-follow off the ROADMAP. Bailey wants to
walk away from the phone and let the agent finish whatever task is in
flight without the screen having to stay manually awake. Without
unattended access, every bridge action that lands on a sleeping screen
silently no-ops because dispatchGesture runs against the dimmed UI.
Scope landed on feature/unattended-access:
UnattendedAccessManager(new file underbridge/) — process singleton holding the SCREEN_BRIGHT wake lock + orchestratingKeyguardManager.requestDismissKeyguard. Hosts aWakeOutcomeenum (Success / SuccessNoKeyguardChange / KeyguardBlocked / Disabled) and two StateFlows (enabled,credentialLockDetected). Initialized once fromHermesRelayApp.onCreate. The host Activity is registered inMainActivity.onResumeand cleared inonPauseso we don't leak it past the lifecycle.- DataStore additions to
BridgeSafetyPreferences—unattendedAccessEnabled(the user opt-in) andunattendedWarningSeen(latch for the one-time scary dialog). Both default false. Setters keepKEY_SAFETY_INITIALIZEDin sync so user-clear-blocklist semantics are preserved. BridgeViewModelexposure —unattendedEnabled,unattendedWarningSeen,credentialLockDetectedStateFlows;setUnattendedAccessEnabled(boolean)+markUnattendedWarningSeen(). An init-time collector mirrors the persisted setting intoUnattendedAccessManager.setEnabled()so the dispatch fast-path can read state synchronously without DataStore.onScreenResumednow also re-probes the keyguard state.UnattendedAccessRowcomponent — Compose card with the toggle, a one-time scaryAlertDialog(security model + credential-lock limitation + how to disable), and a persistentCardchip when the device has a credential lock detected. Only rendered insideBridgeScreenwhenBuildFlavor.isSideloadis true.- Bridge dispatch pre-gate —
BridgeCommandHandler.dispatchcallsUnattendedAccessManager.acquireForAction()after the safety-rails pass and before the actionwhenblock, for any path NOT in the newREAD_ONLY_PATHSset. KeyguardBlocked outcome short-circuits with HTTP 423 +error_code = "keyguard_blocked"+ afinal = trueflag so the LLM doesn't blindly retry against the lock screen. classifyGestureFailure()— small helper onActionExecutorthat wraps gesture-dispatch failure messages with a keyguard-aware hint when the live keyguard state is locked. Tap / swipe / drag / long_press failure paths use it.BridgeCommandHandler.classifyBridgeErrornow routes "keyguard"-bearing strings to thekeyguard_blockederror_code so both pre-gate and gesture-failure paths converge on the same structured response.BridgeStatusOverlayChipvariant — added anunattended: Booleanparameter so the chip can render in an amber-dot "Unattended ON" variant.BridgeStatusOverlay.setChipVisiblegained anunattendedargument and a tear-down-and-rebuild path when the flag flips between attached state.BridgeViewModel's overlay-toggle collector now combines master / overlayPref / unattendedEnabled and forces the chip on whenever unattended is active even if the user disabled the regular overlay preference.- Lifecycle hardening — Master-toggle-off and relay-disconnect
both call
UnattendedAccessManager.release(). The persistedunattendedAccessEnabledvalue is preserved across master toggle cycles (the user's "I want unattended when bridge is on" preference shouldn't be wiped by every disconnect). - Manifest —
DISABLE_KEYGUARDdeclared inapp/src/sideload/AndroidManifest.xml. WAKE_LOCK already lives in the main manifest for the existing PARTIAL_WAKE_LOCK gesture scope and covers SCREEN_BRIGHT acquires too — no new top-level permission.
Tests landed. app/src/test/kotlin/com/hermesandroid/relay/bridge/ UnattendedAccessManagerTest.kt covers state transitions (enabled ↔
disabled, credential lock detection from isDeviceSecure,
acquireForAction outcomes for the four reachable cases without a
real PowerManager: Disabled / SuccessNoKeyguardChange / Success /
KeyguardBlocked) using MockK to stub Context.getSystemService and
reflection to reset the singleton between tests. Also added
BridgeSafetySettingsTest.kt to lock in the false-by-default contract
on the two new preference fields.
Decisions documented (NOT re-litigated this session):
- No WiFi-disconnect failsafe. Tailscale / VPN invalidate the
"leaving WiFi = leaving LAN" assumption; relay-disconnect detection
- auto-disable timer cover the gap.
- Default auto-disable timer stays at 30 minutes. No unattended-mode override.
- Credential lock limitation is structural — Android does not let
third-party apps dismiss PIN / pattern / biometric. Surface it via
the warning dialog + persistent chip +
keyguard_blockedrather than try to work around it.
Wake-lock flag note. SCREEN_BRIGHT_WAKE_LOCK has been
deprecated since API 17 in favor of Window.FLAG_KEEP_SCREEN_ON /
Activity.setTurnScreenOn(true). We accept the deprecation warning
under explicit @Suppress annotations because the Activity-bound
alternatives don't apply to a background bridge that doesn't own
its own window — the bridge runs from a foreground service, gestures
are dispatched via AccessibilityService, and the only "UI surface"
we own off-Activity is the WindowManager overlay chip which can't
drive screen wake-up. Same story for requestDismissKeyguard —
it's API 26+ (our minSdk is 26 so always available) and reachable
only when an Activity is registered, hence the
MainActivity.setHostActivity plumbing.
Files touched:
- new:
app/src/main/kotlin/com/hermesandroid/relay/bridge/UnattendedAccessManager.kt - new:
app/src/main/kotlin/com/hermesandroid/relay/ui/components/UnattendedAccessRow.kt - new:
app/src/test/kotlin/com/hermesandroid/relay/bridge/UnattendedAccessManagerTest.kt - new:
app/src/test/kotlin/com/hermesandroid/relay/data/BridgeSafetySettingsTest.kt - mod:
app/src/main/kotlin/com/hermesandroid/relay/HermesRelayApp.kt - mod:
app/src/main/kotlin/com/hermesandroid/relay/MainActivity.kt - mod:
app/src/main/kotlin/com/hermesandroid/relay/data/BridgeSafetyPreferences.kt - mod:
app/src/main/kotlin/com/hermesandroid/relay/viewmodel/BridgeViewModel.kt - mod:
app/src/main/kotlin/com/hermesandroid/relay/ui/screens/BridgeScreen.kt - mod:
app/src/main/kotlin/com/hermesandroid/relay/bridge/BridgeStatusOverlay.kt - mod:
app/src/main/kotlin/com/hermesandroid/relay/ui/components/DestructiveVerbConfirmDialog.kt(chip variant) - mod:
app/src/main/kotlin/com/hermesandroid/relay/network/handlers/BridgeCommandHandler.kt - mod:
app/src/main/kotlin/com/hermesandroid/relay/accessibility/ActionExecutor.kt - mod:
app/src/sideload/AndroidManifest.xml - mod:
ROADMAP.md(marked v0.4.1 unattended access shipped) - mod:
CHANGELOG.md(added unreleased section)
Next. Bailey builds + installs from Android Studio + flips the
toggle on a Samsung S24 with a PIN lock. Expected behaviours:
(a) scary dialog appears on first toggle-on, (b) keyguard-detected
chip stays visible while unattended is ON, (c) bridge actions return
keyguard_blocked when fired against a locked screen, (d) toggling
master off drops the wake lock immediately. After verification, cut
v0.4.1 release tag.
Known residuals / follow-ups:
- The
WakeOutcome.Successpath doesn't actually wait on therequestDismissKeyguardcallback — we predict success based onisDeviceSecure == false. Predict-don't-wait is fine for None / Swipe locks (which always succeed) but a Wave 2 refinement could plumb the callback for richer telemetry. - The host-activity registration is single-Activity by design.
If a future MainActivity refactor introduces a second Activity
surface, the
setHostActivity(null)call in onPause would clobber the wrong reference. Document this constraint where it matters. READ_ONLY_PATHSis hand-maintained — adding a new bridge route requires updating both the dispatchwhenand the read-only set. A linter rule or test-time enforcement would catch drift.
2026-04-16 — Voice intent → server session sync (v0.4.1)
Motivation. Bailey's 2026-04-14 on-device repro: speak "open Chrome", phone opens Chrome (good), then type "did that work?" → LLM responds "I have no prior context for what you're asking about." Voice intents dispatch in-process via BridgeCommandHandler.handleLocalCommand (correct — voice actions are phone-local) and append a local-only trace bubble to chat (good UX), but the server-side Hermes session never absorbs the action so the gateway-side LLM has zero memory of it. Followups felt like the chat had "reset."
Approach. Synthesize OpenAI-format assistant (with tool_calls) + tool (with tool_call_id) message pairs from unsynced voice-intent traces and pass them under a new optional messages field on the existing /v1/runs and /api/sessions/{id}/chat/stream payloads. LLMs are trained on this exact shape — they read it as natural conversation history, not a side-channel system-prompt note. Lower retry risk than a bracketed system-message prefix. Frontend-only — zero server changes.
Implementation summary.
data/ChatMessage.kt— addedvoiceIntent: VoiceIntentTrace?field (default null). The newVoiceIntentTraceclass capturestoolName(must startandroid_*),argumentsJson(OpenAI tool-call args, JSON-encoded string),success,resultJson(full result envelope includingok/error/error_code), andsyncedToServerflag. Default null so every existingChatMessage(...)call site keeps compiling without touching this field.voice/VoiceIntentSyncBuilder.kt(new file, ~170 LOC) — pure-function builder that walks chat history, filters to unsynced voice-intent traces, mintscall_voiceintent_<uuid>IDs, and emits a JsonArray of syntheticassistant+toolmessage pairs in chronological order.hasUnsynced()short-circuits the empty-array allocation on the common-case turn.successResultJson()/failureResultJson()helpers normalize the tool-response payload shape so call sites don't hand-roll JSON. No Android dependencies — JVM-pure for cheap unit testing.network/HermesApiClient.kt—sendChatStreamandsendRunStreamboth gained an optionalvoiceIntentMessages: JsonArray? = nullparameter. When non-empty, splices it under a top-levelmessagesfield on the request body. Additive, OpenAI-compat — older servers ignore unrecognised body fields.viewmodel/ChatViewModel.kt—startStreamsnapshots history, callsVoiceIntentSyncBuilder.buildSyntheticMessages, threads the result into both API client paths, then callshandler.markVoiceIntentsSynced()after the API client takes ownership. Idempotent — already-synced traces are skipped.recordVoiceIntent/recordVoiceIntentResultsignatures now accept an optionalvoiceIntent: VoiceIntentTrace?so VoiceViewModel can attach the structured payload.network/handlers/ChatHandler.kt—appendLocalVoiceIntentTraceandappendLocalVoiceIntentResultnow accept an optionalvoiceIntentparameter; newmarkVoiceIntentsSynced()method flipssyncedToServer=trueon every trace currently in state.voice/VoiceBridgeIntentHandler.kt—IntentResult.HandledgainedandroidToolName: String?(defaults to null) andandroidToolArgsJson: String(defaults to "{}"). Sideload classifier populates both per intent soVoiceIntentSyncBuildercan synthesize a structured tool_call.- Sideload
VoiceBridgeIntentHandlerImpl.kt— everytryHandlebranch now passesandroidToolName+androidToolArgsJsontohandleSafe/handleDestructive. The dispatch callbacks (onDispatchResult) carry these forward to VoiceViewModel. MapsSendSms→android_send_sms,OpenApp→android_open_app,Tap→android_tap_text,Back/Home→android_press_key. Args mirror what the gateway-side LLM tool wrappers inplugin/tools/android_tool.pywould emit for the same actions, so the synthetic tool_call looks identical to the real one. VoiceIntentResultCallbacktypealias — extended in BOTHgooglePlay/andsideload/flavor source sets to(intentLabel, result, androidToolName, androidToolArgsJson) -> Unit. Play APK never invokes the callback (no destructive intents in the no-op handler) but typealias parity keepsVoiceViewModelcompilable against either flavor with no#ifgating.viewmodel/VoiceViewModel.kt— extended dispatch callback wiring builds aVoiceIntentTracefrom the post-dispatch outcome (usingLocalDispatchResult.isSuccess+resultJson+errorMessage+errorCode) and threads it intochatViewModel.recordVoiceIntentResult(label, result, voiceTrace). Trace attaches to the post-dispatch RESULT bubble (not the pre-dispatch action bubble) because that's the moment the dispatch outcome is authoritative — pre-dispatch was either pending (destructive countdown) or not-yet-known.
Tests. app/src/test/kotlin/com/hermesandroid/relay/voice/VoiceIntentSyncBuilderTest.kt (12 cases):
- empty history → empty output
- no voice-intent messages in history → empty output
- single success trace → exactly one assistant+tool pair, correct shape
- failure trace → tool message content carries
ok:false+error+error_code - already-synced traces are skipped
- multiple traces preserve chronological order
- non-
android_*tool name is filtered (defence-in-depth) - blank args is filtered (defence-in-depth)
- assistant
tool_calls[].idand tooltool_call_idreference the same minted ID hasUnsyncedhappy paths (empty / unsynced / all-synced)successResultJson/failureResultJsonhelpers behave correctly
ChatHandlerTest.kt gained 4 new cases for trace storage + markVoiceIntentsSynced flag flip + idempotency on already-synced traces + safety on plain non-trace messages.
Files touched (10):
app/src/main/kotlin/com/hermesandroid/relay/data/ChatMessage.ktapp/src/main/kotlin/com/hermesandroid/relay/voice/VoiceIntentSyncBuilder.kt(new)app/src/main/kotlin/com/hermesandroid/relay/network/HermesApiClient.ktapp/src/main/kotlin/com/hermesandroid/relay/network/handlers/ChatHandler.ktapp/src/main/kotlin/com/hermesandroid/relay/viewmodel/ChatViewModel.ktapp/src/main/kotlin/com/hermesandroid/relay/viewmodel/VoiceViewModel.ktapp/src/main/kotlin/com/hermesandroid/relay/voice/VoiceBridgeIntentHandler.ktapp/src/sideload/kotlin/com/hermesandroid/relay/voice/VoiceBridgeIntentFactory.ktapp/src/sideload/kotlin/com/hermesandroid/relay/voice/VoiceBridgeIntentHandlerImpl.ktapp/src/googlePlay/kotlin/com/hermesandroid/relay/voice/VoiceBridgeIntentFactory.ktapp/src/test/kotlin/com/hermesandroid/relay/voice/VoiceIntentSyncBuilderTest.kt(new)app/src/test/kotlin/com/hermesandroid/relay/network/handlers/ChatHandlerTest.kt
Branch. feature/voice-session-sync. Not merged. Pushed for review.
What's next. On-device verification: send a voice intent ("open Chrome"), then type "did that work?" — the LLM should describe the action with grounded context instead of hallucinating. If the upstream gateway logs the synthetic messages array as additional context (visible in journalctl --user -u hermes-gateway -f), the wire integration is clean. If the gateway 400s on the new field name (e.g. wants additional_messages or history), we may need to peek at gateway/platforms/api_server.py upstream to confirm the exact field name. The CHANGELOG note about "additive — OpenAI-compat" reflects best-current-knowledge but isn't yet wire-verified.
Blockers. None. Ships as a self-contained refactor.
2026-04-15 — Voice → bridge → agent pipeline end-to-end fixes
Motivation. Bailey's on-device voice SMS testing surfaced a stack of bugs that individually looked small but together blocked the voice → bridge → agent pipeline from working end-to-end. Over a single long session we diagnosed and fixed six distinct layers: voice classifier gaps, a safety-modal lifecycle bug, chat history clobbering, a plugin check_fn pointed at the wrong relay endpoint, missing LLM tool wrappers for the direct-dispatch phone actions, and two Android OEM-interaction issues (MediaProjection notification persistence + activity recreation on warm reopen). Landed across four commits on feature/bridge-feature-expansion.
Summary of changes (in narrative order, not commit order):
- Voice intent classifier narrowed + phrasing-tolerant. Removed the
Scrollvoice intent (nobody says "scroll down" aloud;/scrollroute stays for server-sideandroid_scrolltool calls). AddedSMS_INDIRECTregex to catch natural indirect-object phrasing ("send Hannah a text saying smoke test") plusmessageas a direct verb. Added a phone-number literal bypass so "text +1 555 1234 saying hi" skips contact lookup. - Honest error classification replaces "couldn't find contact" catch-all.
resolveContactPhonenow returns aContactResolutionsealed type distinguishingFound/ServiceMissing/PermissionMissing/NotFound/NoPhoneNumber/OtherError, with voice mode speaking a specific actionable message per category ("I need Contacts permission to look up that number" vs "I couldn't find a contact called Hannah").resolveAppPackagegot the same treatment viaAppResolution. Pre-flight SEND_SMS check short-circuits before the 5s countdown if permission is missing. Multi-contact matches surface total-count + matched-name in the spoken confirmation so the user knows one of several got picked. - Safety-modal threading + savedstate lifecycle fixes.
BridgeSafetyManager.awaitConfirmationnow wrapshost.showConfirmationinwithContext(Dispatchers.Main.immediate)— the ComposeView creation + setContent must run on Main, but was being called fromDispatchers.Defaultvia the voice local-dispatch path, threw, and got swallowed as "likely overlay permission missing" (which misled on-device triage for a full test cycle).OverlayLifecycleOwner.start()init order flipped:performRestore(null)must run while lifecycle state is still INITIALIZED, not after advancing to CREATED — current androidx.savedstate assertsperformAttachrequires INITIALIZED. The old order worked against an earlier androidx.savedstate version and broke silently when the Compose BOM bumped. - Voice mode UX: post-dispatch feedback, TTS, visual countdown, fall-through visibility.
handleLocalCommandwent from fire-and-forget to capturing the response via anAtomicReferencesink on theLocalDispatchcoroutine context element, returningLocalDispatchResult(status, errorMessage, errorCode, resultJson). Voice mode emits a follow-up chat bubble ("Send SMS — sent ✓" / "Send SMS — cancelled by you" / permission/service errors) AND speaks the outcome via the existing TTS queue. Voice-in-voice cancel detects "cancel / stop / never mind" during the 5-second countdown and routes it straight tocancelPending()instead of classifying it as a new turn. Visual countdown:DestructiveCountdownStateonVoiceUiState+onCountdownStartcallback +LinearProgressIndicatorinVoiceModeOverlaysynced to the real delay. Classifier fall-through immediately flipsVoiceState.Thinkingso the UI shows progress during SSE connect latency instead of appearing to drop the utterance. - Voice mode transcript with chat parity.
VoiceModeOverlaynow observesChatViewModel.messagesand renders the last 6 viaCompactTranscriptRow+StreamingResponseRow. Voice-action traces render viaMarkdownContent. User messages keep the "YOU" caption (fixed a caption mislabel where voice-intent user messages were showing as "ACTION" due to an id-prefix check that didn't filter by role).ChatHandler.loadMessageHistorypreserves anyvoice-intent-*messages across server reloads so "Opened Chrome" bubbles don't vanish when session_end reload fires after voice fall-through to LLM. - Chat parity: structured result bubbles for android_ tool calls on the runs API.*
ChatHandlernow interceptstool.completedSSE events from/v1/runsfor action tools (send_sms, call, search_contacts, open_app, return_to_hermes, screenshot, press_key, setup — read-only tools skipped because they'd add noise on top of the existingToolProgressCard) and emits the same structured markdown bubble the voice fast-path does.agentName = "Phone action"distinguishes chat-originated from voice-originated ("Voice action").MessageBubble.ktpicked up a subtle visual marker for both action-type bubbles so they read as distinct from LLM replies when interleaved. - Plugin
_check_requirementspointed at the right endpoint with a three-gate rule. Was hitting/pingwhich returns{pong, ts}and looking forphone_connected/accessibilityServicefields that don't exist there — result: the gate always returned False and hid all 13 non-setup tools from every gateway platform. Victor's "no android_* tools available" reports were this. Now hits/bridge/statusand requires all three:phone_connected AND bridge.accessibility_granted AND bridge.master_enabled. Tools vanish from the LLM's schema entirely when the master toggle is off (or accessibility is revoked post-Studio-reinstall), giving the model a clean "no tools" signal instead of an error-interpretation race. Trade-off accepted: tools disappear mid-session if the user flips the toggle — desired behavior per "stop the agent from controlling my phone" intent. - 4 new direct-dispatch plugin tools.
android_search_contacts,android_send_sms,android_call,android_return_to_hermesall wrap phone routes (/search_contacts,/send_sms,/call, new/return_to_hermes) that were already fully implemented (safety modal, directSmsManager/CALL_PHONEdispatch) but had never been exposed to the agent, forcing it to drive Messages / Phone / Contacts UI step-by-step viatap+read_screen. Tool count 14 → 18. The/return_to_hermesroute short-circuits whenservice.currentApp == service.packageName(i.e. Hermes is already foreground) so agent wrap-up calls are benign no-ops in voice mode. Master-toggle check exempts/return_to_hermesso the agent can always wrap up cleanly. - Honest error_code in bridge.response JSON.
respondFromResultsubstring-classifies ActionExecutor errors and emitserror_code+required_permissionfields alongside the existing free-texterror, forpermission_denied/service_unavailable/user_denied/bridge_disabled. LLM gets both human-readable text AND a machine-readable classification. Clearer 403 English on thebridge_disabledpath explicitly says "this is NOT a pairing problem" + names the exact toggle — earlier Victor was walking users through re-pairing when the master toggle was off. - Plugin tool descriptions carry the "prefer direct dispatch + return_to_hermes when done" guidance instead of Victor's personality. Initial implementation edited
~/.hermes/config.yamlon the server, which only works for Bailey's one install — reverted. Guidance now lives inandroid_send_sms/android_call/android_open_appdescriptions where it travels with the plugin to any hermes-agent install. Portable across deploys. - MediaProjection notification persists bug fixed.
BridgeForegroundService.onTaskRemovednow revokes the MediaProjection + downgrades the FGS type back to SPECIAL_USE-only when the user swipes the app from recents. The bridge itself keeps running (agent phone control via WSS still works), but the system-level screen-cast icon goes away — which is what the user expects when they "close" the app. Before this fix the icon persisted indefinitely because foreground services legitimately survive task removal and the projection stayed bound. - Activity recreation on warm reopen fixed.
MainActivityin the manifest gainedlaunchMode="singleTask"+configChanges="uiMode|fontScale|locale|density|orientation|screenSize|screenLayout|keyboardHidden". Before this the activity was being recreated on config changes and on certain resume paths, triggeringinstallSplashScreen()again and showing the splash on warm reopen. The residual case — Samsung OneUI aggressively killing backgrounded apps even with a foreground service — needs user-side battery whitelisting (Settings → Device Care → App battery management → Hermes Relay → Unrestricted) and can't be fixed in code.
Commits landed on feature/bridge-feature-expansion:
5c763eb— honest errors, modal lifecycle, missing LLM tools (11 files, +974/-158)536a131— post-dispatch feedback + clearer bridge-disabled error (7 files, +325/-37)- (pending, this session) — agent-team work: voice UX polish (TTS, cancel, countdown, phone-number bypass, multi-contact), chat structured feedback parity, plugin three-gate rule,
/return_to_hermesshort-circuit, tool description guidance, MediaProjection cleanup, activity recreation hardening
Server state. hermes-gateway restarted cleanly twice this session. _check_requirements() now returns True/False deterministically based on live /bridge/status (verified via Python import). Victor personality reverted to clean default — no plugin-specific coupling. All 18 tools register with no missing/orphan handlers.
Known residual items for follow-up (not blocking, documented for future-us):
- Full multi-turn voice disambiguation for multi-contact matches — current fix just hints "found N, using first". Wave 3 multi-turn state machine deferred.
- Samsung OneUI process-death UX — add an in-app nudge to whitelist battery optimization if running on Samsung + not currently whitelisted. Polish.
- Tool-description guidance relies on LLM cooperation (non-deterministic) — monitor whether Victor actually picks up
android_send_smsover UI automation in future tests. If not, we may need to reinforce via plugin.yaml or a new upstream plugin-metadata field. loadMessageHistoryrace with voice-intent-result bubbles self-heals via timestamp sort but has a brief visual flicker window.android_return_to_hermesno-op in voice mode is benign but wastes a tool call — agent should detect voice mode and skip. Minor.
Next session. Waiting on Bailey's retest with the bundled agent-team work. Once that passes, bump version (voice intent loop + chat parity is v0.4.0-worthy) and start the release cut.
2026-04-15 — Gateway slash-command dispatch: architectural finding + three-PR strategy
Motivation. Bailey typed /model into the Android app's chat on the feature/bridge-feature-expansion branch to switch the model for the session. The message reached the LLM as plain text and got a hallucinated reply: "Switching model — /model is a client-side command, I don't execute it. Just send it as its own message (not wrapped in chat) and Hermes will swap the model for the session." The reply is confidently wrong on two counts — the command is not "client-side", and nothing about the Android client's wire format would prevent Hermes from intercepting it. The LLM was filling a gap it didn't know existed.
Root cause (confirmed via upstream code archaeology by a subagent oriented in a fresh clone of Codename-11/hermes-agent on feat/session-api):
- Gateway slash commands (
/model,/new,/retry, the 29 inhermes_cli/commands.py::GATEWAY_KNOWN_COMMANDS) are intercepted by in-process platform adapters — Discord, Telegram, Slack, Matrix, Signal, BlueBubbles, etc. All of them wrap inbound messages intoMessageEvents withMessageType.COMMANDand route them throughBasePlatformAdapter.handle_message()→GatewayRouter._handle_message(). That router method contains a ~300-lineif canonical == "new"/"help"/"model"/...dispatch chain atgateway/run.py:2645–2929, and the command handlers (_handle_model_commandat 4226,_handle_status_commandat 4065,_handle_help_commandat 4151, and ~25 others) are instance methods onGatewayRouterthat mutate router-owned state:self._session_model_overrides,self._agent_cache,self._agent_cache_lock, plus per-adapter interactive pickers viaself.adapters[platform].send_model_picker(...). APIServerAdapter(gateway/platforms/api_server.py) does not connect to the router. Its__init__(atapi_server.py:331) takes only aPlatformConfig— noGatewayRouterreference._handle_chat_completionsand_handle_runscallself._run_agent(...)directly, which callsself._create_agent(...)to build a fresh agent per request with no persistent session state. Upstream confirms this is intentional:run.py:3148comments that api_server is an "excluded platform" from the router notification path.- Therefore
/modelcannot be made to work on/v1/runswith a local preprocessor. A fresh agent per request means there is no persistent session to switch the model on. Same story for/new,/retry,/undo,/compress,/title,/resume,/branch,/rollback,/approve,/deny,/background,/stop,/yolo,/fast,/reasoning,/personality— they all mutate router-owned state that api_server doesn't maintain. What can run statelessly:/helpand/commandsviagateway_help_lines()athermes_cli/commands.py:340, and possibly/profile,/provider,/usage,/insights,/statusdepending on their implementation.
Strategy — a three-PR arc, each independently reviewable, each with a matching bootstrap-middleware sibling shipping earlier:
-
PR #8556 (already submitted,
feat/session-api→NousResearch/hermes-agent:main). "feat(api-server): add session management API for frontend clients." Scope is broader than the title: sessions CRUD + search + fork + messages,/api/sessions/{id}/chat+/api/sessions/{id}/chat/stream, memory CRUD, skills, config, available-models. Verified 2026-04-15 viagh pr diff 8556. This is the missing session primitives — once merged, api_server has a durable place where stateful state can live. -
Stage 1 PR — slash-command preprocessor on the stateless endpoints. Sibling follow-up to #8556, same file (
gateway/platforms/api_server.py), stacks onfeat/session-apiduring review so the diff shows both together. Detects known gateway commands on/v1/runs+/v1/chat/completions, dispatches the small stateless subset (/help,/commands) via existing helpers, returns a deterministic synthetic-SSE "use a channel with session state" notice for the stateful majority. Respects upstream's intentional api_server design and does not touchGatewayRouter. Being prepared today inC:/Users/Bailey/Desktop/Open-Projects/hermes-agent-pr-prep/on branchfeat/api-server-gateway-commandsvia a background subagent; Option B scope (stateless dispatch + polite decline) confirmed after the subagent stopped on the architectural finding instead of writing Option C code that would have fought upstream's design. The subagent is preparing the full diff + PR body for human review before anygh pr createruns. Matching bootstrap-middleware sibling (hermes_relay_bootstrap/_command_middleware.py) is filed as a v0.4.1 roadmap item so the hallucination fix ships to vanilla-upstream installs without waiting for the upstream PR. -
Stage 2 PR — stateful slash-command dispatch on
/api/sessions/{id}/chat/stream. Blocked on #8556 merging. Once session primitives ship upstream, a smaller separate PR adds a preprocessor scoped to the session chat stream endpoint only, using the URL'ssession_idas the persistence handle. Stateful commands become session-scoped dict writes (session.model_override = new_model) without refactoringGatewayRouteror plumbing api_server into the router. Clean partition:/v1/*stays stateless, statefulness lives on/api/sessions/*.
Why not a single giant "Option C" PR that refactors GatewayRouter to expose dispatch_slash_command() + wires APIServerAdapter to the router + adds per-session state to api_server: (a) it would withdraw #8556 to restart bigger, which burns reviewer goodwill and signals indecision to Nous, (b) it would touch 10+ files across subsystems normally owned separately, reviewing far worse than three small PRs, (c) it would fight the documented "api_server is excluded from router" design intent rather than extending it, (d) #8556 is already doing most of the work (session primitives); a parallel session substrate would duplicate it, (e) review slots on respected org PRs are hard-won; don't give one back.
Docs updated this session:
CLAUDE.md— fork branch reference corrected tofeat/session-api+ scope clarification on #8556.docs/decisions.md— same fix in the upstream-notes section (line 85).docs/upstream-contributions.md— new §5 documenting the Stage 1 + Stage 2 upstream PR plan and the bootstrap-middleware workaround.ROADMAP.md— v0.4.1 bootstrap-middleware entry rewritten to reflect the Option B ceiling (cannot make/modelactually switch models on/v1/runs— that's Stage 2, blocked on #8556) and the architectural reasoning.TODO.md— three new entries covering Stage 1 upstream PR, Stage 1 bootstrap middleware, and Stage 2 (blocked on #8556).hermes_relay_bootstrap/_handlers.py,hermes_relay_bootstrap/_patch.py— docstring branch-name corrections (feat/api-server-enhancements→feat/session-api).
Not touched this session:
- No Kotlin changes. The Android client at
HermesApiClient.kt:655-715already parsesmessage.delta/response.output_text.delta/reasoning.available/tool.started/tool.completed/run.completed/[DONE], so when Stage 1 or the bootstrap middleware lands with a synthetic SSE stream matching those shapes, the client needs no changes. - No changes to
gateway/platforms/api_server.pyor any file inhermes-agent-pr-prep/. The subagent has orientation-only state onfeat/api-server-gateway-commands; implementation begins on its next wake. - The
feature/bridge-feature-expansionbranch is untouched by this session and continues with its voice-bridge work. The slash-command work is a separate branch arc and will not land on the bridge feature branch.
2026-04-13 — Play Console migration to Axiom-Labs, LLC (applicationId → com.axiomlabs.hermesrelay)
Moved the Play Store listing from Bailey's personal Play Console account (where v0.1.x–v0.3.0 shipped under Internal testing) to the Axiom-Labs, LLC D-U-N-S-verified organization account. This unblocks straight-to-production rollout — personal accounts created after late 2023 are subject to Google's 14-day closed-testing rule (≥12 opted-in testers for 14 continuous days before production promotion); D-U-N-S-verified org accounts are exempt. See Google's policy.
applicationId change. com.hermesandroid.relay → com.axiomlabs.hermesrelay (googlePlay flavor) and com.axiomlabs.hermesrelay.sideload (sideload flavor, unchanged suffix pattern). Play Store package names are permanently reserved once used, so the old ID can never be reclaimed — the previous Internal-testing listing is retired and existing installs won't auto-upgrade to the new listing. Blast radius is one tester (Bailey), so manual reinstall is fine.
Kotlin namespace stays at com.hermesandroid.relay. AGP decouples namespace (build-time R-class / BuildConfig / on-disk source layout) from applicationId (runtime install identity), so we don't need to mass-rename the 130+ Kotlin files, their package declarations, proguard rules, or test FQCNs. The source tree stays stable; only the Play/Android install identity moves. scripts/dev.bat and scripts/dev.sh already use the split FQCN form adb am start -n com.axiomlabs.hermesrelay/com.hermesandroid.relay.MainActivity to bridge the two namespaces at launch time.
Keystore identity is unchanged. Same CN=Bailey Dixon, Codename-11 upload cert, same SHA256 fingerprint, same HERMES_KEYSTORE_* GitHub Secrets, same CI signing flow. Play App Signing mints a new server-side app signing key per listing, but that's invisible to us since App Signing is enabled. No CI or keystore changes required.
Files touched in this migration:
app/build.gradle.kts—applicationId = "com.axiomlabs.hermesrelay"+ multi-paragraph comment block explaining the namespace/applicationId decoupling and the migration rationaleRELEASE.md— new "Google Play Console developer account" section documenting the Axiom-Labs account, the D-U-N-S exemption from the 14-day rule, and a "Historical note (2026-04-13 migration)" block for future maintainersROADMAP.md— "Current — Axiom-Labs migration" section added at the topREADME.md— Play Store badge URL →?id=com.axiomlabs.hermesrelay; copyright line → "Axiom-Labs"CLAUDE.md— "applicationId" bullet updated to call out both flavors undercom.axiomlabs.*scripts/dev.bat,scripts/dev.sh—am startcommands updated to the new applicationIdscripts/bridge-smoke.sh— sideload package test payload →com.axiomlabs.hermesrelay.sideloaduser-docs/guide/getting-started.md,user-docs/guide/release-tracks.md— Play Store URLs + both applicationIds
Play Console listing creation (for future reference):
| Field | Value |
|---|---|
| App name | Hermes-Relay |
| Package name | com.axiomlabs.hermesrelay |
| Default language | English (US) |
| App/game | App |
| Free/paid | Free |
| Developer account | Axiom-Labs, LLC |
Nothing to rebuild or re-sign beyond the normal v0.3.1+ release flow.
2026-04-12 — Manual pairing fallback — hermes-pair --register-code <code>
Wired the host-side half of the in-app fallback pairing flow. The phone has been generating a local 6-char code in Settings → Connection → Manual pairing code (fallback) and AuthManager.authenticate() already sent that code as pairing_code when no server-issued code was present, but there was no convenient way for an operator to pre-register an arbitrary code with the relay without going through the QR flow. Now there is.
Use case: Bailey wants to pair a phone he's already SSH'd in from, where there's no second device with a camera to scan a QR off the host's display. The QR flow is unusable in that scenario; the new flag is the only path.
plugin/pair.py additions:
- New
normalize_pairing_code(code)helper — upper-cases, strips, validates againstPAIRING_ALPHABET(A-Z / 0-9) andPAIRING_CODE_LENGTH(6). RaisesInvalidPairingCodeError(aValueErrorsubclass) with a clear message on length or alphabet mismatches so the CLI can fail fast instead of letting the operator find out via an HTTP 400. - New
register_code_command(args)— separate code path from the QR pipeline. Validates the code, parses--ttl/--grants(same parser as the QR flow, default30d), resolves the relay port fromread_relay_config(), probes/healthfirst so we can give a precise "relay not running" message instead of a generic post failure, then calls the existingregister_relay_code()helper with all the optional metadata. Prints a clean success block listing the code, transport hint, session TTL, grants, and the exact 3-step "tap Connect in the app" instructions. Distinct exit codes:0success,1relay unreachable / rejected,2argument validation failed. pair_command()short-circuits toregister_code_commandwhenargs.register_codeis set, so the QR pipeline is skipped entirely (no relay code minted, no QR rendered, no API config probed).- New CLI flags on the standalone
python -m plugin.pairargparser:--register-code <CODE>(the trigger),--transport-hint {ws,wss}(override the auto-detected transport hint when running behind an external TLS proxy that the host can't see). All existing flags (--ttl,--grants,--host,--port, etc.) compose with--register-codeexactly the way they compose with the QR flow.
New CLI shape:
hermes-pair --register-code ABCD12
hermes-pair --register-code ABCD12 --ttl 30d --grants terminal=7d,bridge=1d
hermes-pair --register-code ABCD12 --ttl never --transport-hint wss
The relay endpoint (POST /pairing/register in plugin/relay/server.py) and PairingManager.register_code were not touched — they already accepted user-supplied codes with optional TTL/grants/transport-hint metadata. This change just exposes that capability via a CLI flag so the operator doesn't have to hand-craft a curl POST.
plugin/tests/test_register_code.py — 25 tests, stdlib unittest only. Three test classes:
NormalizePairingCodeTests(13 tests) — happy path (uppercase / lowercase / whitespace-stripped / all-digit / all-letter), every rejection branch (empty / whitespace-only /None/ too short / too long / dash / punctuation / unicode).RegisterCodeCommandTests(10 tests) — happy path with default TTL (verifies it posts30 * 24 * 3600seconds andtransport_hint="ws"); happy path with--ttl 7d --grants terminal=1d,bridge=1dandtls=Truein the relay config (verifiestransport_hint="wss"and the auto-uppercase ofabcd12 → ABCD12); explicit--transport-hint wssoverride; every error path (invalid chars, wrong length, empty, relay unreachable → exit 1, relay rejection → exit 1, invalid--ttl→ exit 2, invalid--grants→ exit 2). Each error-path test asserts the network was NOT touched if validation failed first.RegisterRelayCodeWireShapeTests(2 tests) — patchesurllib.request.urlopendirectly and verifies the bytes the manual flow puts on the wire match whathandle_pairing_registeralready parses:{"code": ..., "ttl_seconds": ..., "grants": ..., "transport_hint": ...}. Also covers the minimal-body case (just{"code": ...}when no metadata is supplied) so we don't accidentally start sendingnullfields the server's_parse_pairing_metadatawould have to filter.
Run via python -m unittest plugin.tests.test_register_code — 25 passed in 0.006s.
Docs touched:
skills/devops/hermes-relay-pair/SKILL.md— added--register-codeto the "Useful flags" bullet list and a full "Manual fallback (--register-code)" section between Procedure and Pitfalls covering when to use it, the 3-step workflow, how--ttl/--grants/--transport-hintcompose, and the exit codes.skills/devops/hermes-relay-self-setup/SKILL.md— added an "If you can't scan a QR" subsection inside section D pointing operators at thehermes-relay-pairskill's manual-fallback recipe.user-docs/reference/configuration.md— expanded the "Manual pairing code (fallback)" bullet from a one-liner into a full 3-step workflow walkthrough so users can find this without leaving the app's docs.user-docs/guide/getting-started.md— added a "Camera unavailable? Use manual pairing" callout under the "Choosing session lifetime + channel grants" section.README.md— added a one-line bullet under the install section listinghermes-pair --register-codealongside the slash command and the dashed shim.
Conventions: No Kotlin touched (parallel agent owns the in-app UI cleanup). Did not modify plugin/relay/server.py or PairingManager — endpoint already supported user-supplied codes. Tests run via stdlib unittest to bypass the conftest.py that imports responses.
2026-04-12 — Connection UX — Manual pairing code walkthrough + per-channel grant revoke
Manual pairing code card. Settings → Connection → "Manual pairing code (fallback)" used to be a bare-bones code + copy/regen stub that left users guessing what to do with the 6-character code. Replaced the body with a proper three-step walkthrough: (1) copy the code (with both an instant copy button and the existing regenerate button), (2) on the host, run hermes-pair --register-code <code> rendered in a copyable monospace shell-command surface, (3) tap Connect to fire the pair flow. Step 3 is a real Button that calls applyServerIssuedCodeAndReset → disconnectRelay → connectRelay (mirroring the existing dialog flow), tracks an in-flight card3ConnectInProgress flag with a CircularProgressIndicator + "Connecting…" label, and reports success/failure through LocalSnackbarHost.showHumanError with the "pair" classifier context. Below the steps is an expandable "How does this work?" explainer that spells out when this is the right flow vs. the QR scan, and reminds users that bridge control is gated by the master toggle on the Bridge tab — not by the pairing code itself. New private ManualPairStep helper renders the numbered step badges. Pairs cleanly with the parallel agent's host-side hermes-pair --register-code work above. Markers: // === MANUAL-PAIR-FOLLOWUP: ... === / // === END MANUAL-PAIR-FOLLOWUP ===.
Per-channel grant revoke. Paired Devices → device card grant chips are now individually revocable. Each chip got an inline Close icon button (a small clickable Box instead of IconButton to avoid the 48dp touch-target inflation that would blow up the FlowRow). Tapping the x opens an AlertDialog confirming "Revoke access for ?" with an explicit reminder that the session itself stays paired and other channels keep their current expiry. New ConnectionViewModel.revokeChannelGrant(tokenPrefix, channel) builds the full grants map by reading the device's current PairedDeviceInfo.grants, converting absolute-epoch grants back to seconds-from-now (clamped to ≥ 1 since 0 means "never expire" on the relay side), and replacing only the target channel with 1L (≈ instantly expired by the time the PATCH lands). Then it reuses RelayHttpClient.extendSession with ttlSeconds = null, grants = rebuilt and refreshes the device list on success. The chip label also got a relative TTL helper (formatRelativeTtl) that renders "never" / "in 6d" / "in 23h" / "expired" instead of the previous absolute short date. The full per-session "Revoke" button stays — per-channel chips are additive, not a replacement. Markers: // === PER-CHANNEL-REVOKE: ... === / // === END PER-CHANNEL-REVOKE ===.
2026-04-12 — Phase 3 / status — relay GET /bridge/status endpoint + expanded BridgeStatusReporter
Backend half of the android_phone_status() work. The phone (BridgeStatusReporter.kt) now pushes a structured status envelope every 30 s with three nested groups:
device—Build.MODEL, battery,PowerManager.isInteractive,HermesAccessibilityService.instance?.currentAppbridge—master_enabled,accessibility_granted,screen_capture_granted,overlay_granted,notification_listener_grantedsafety— blocklist count, destructive verb count, auto-disable timer minutes,auto_disable_at_ms
Old flat keys kept alongside the new structure for backwards compat. New pushNow() method on the reporter for out-of-band emissions; ConnectionViewModel calls it whenever the master toggle flips so the relay-side cache refreshes immediately instead of waiting up to 30 s.
Relay side: BridgeHandler gained latest_status: dict | None and last_seen_at: float | None, populated in handle_status. New handle_bridge_status route at GET /bridge/status, gated to loopback only (mirrors /pairing/register). Empty cache → 503; populated → 200 with last_seen_seconds_ago computed at response time. Stale-cache disconnects return 200 — the cached snapshot is still useful.
Tests: plugin/tests/test_bridge_status.py — 7 stdlib unittest cases (empty cache 503, populated 200, disconnected, non-loopback rejection, IPv6 ::1, snapshot-not-reference regression, dual-field stamping). All pass; existing test_bridge_channel.py still green.
Markers: # === PHASE3-status: ... === / # === END PHASE3-status === for the new Python blocks; // === PHASE3-status: ... === for Kotlin.
2026-04-12 — Phase 3 / bridge UI hardening — master gate, MediaProjection consent, label disambiguation
Three follow-ups discovered while smoke-testing the merged Phase 3 build on a real device.
Master gate fix. HermesAccessibilityService.cachedMasterEnabled was supposed to be fed by an external updateMasterEnabledCache(...) writer, but nothing was calling it — the cache was permanently false and BridgeCommandHandler was 403'ing every command except /ping and /current_app regardless of the toggle state. Fix: the service now starts a serviceScope coroutine on onServiceConnected that observes masterEnabledFlow(this) and pumps every emission into the cache. Service-owned scope is cancelled on onUnbind/onDestroy and re-created on each connect (cancelled SupervisorJobs can't be reused). Cache is reset to false on teardown so a stale-but-bound state can't leak through.
MediaProjection consent flow. MediaProjectionHolder.onGranted(...) existed but nothing called it — there was no ActivityResultLauncher registered anywhere. Fix: MainActivity now registers a launcher in onCreate and a new ScreenCaptureRequester process-singleton holds the launch closure (installed on Activity create, cleared on destroy). BridgeViewModel.requestScreenCapture() calls ScreenCaptureRequester.request() to ask the user; the system consent dialog fires; the result callback feeds MediaProjectionHolder.onGranted(...) which closes the loop with the existing ScreenCapture.kt. Bridge tab's Screen Capture row is now tappable instead of inert; the row's checkmark reflects MediaProjectionHolder.projection != null.
Notification Listener Test button parity. Added BridgeViewModel.testNotificationListener() and an onTestNotificationListener lambda through BridgePermissionChecklist so the row gets a Test button matching the other three. The dedicated functional test on NotificationCompanionSettingsScreen still ships.
Launcher label disambiguation per flavor. With googlePlayDebug and sideloadDebug installed side-by-side (same icon, same name), there was no way to tell which install you were tapping in the launcher / recents / Settings → Apps. Fix: sideload's res/values/strings.xml now overrides three strings:
app_name→Hermes Dev(10 chars, fits launcher icon, mirrorsSlack Dev/Chrome Devconvention)a11y_service_label→Hermes-Bridge Dev(Accessibility settings list)notification_companion_label→Hermes Dev notification companion
googlePlay's a11y_service_label also flipped to Hermes-Bridge (with hyphen) for consistency. The Play track inherits all other strings from main.
Other small fixes: BridgeForegroundService.kt Intent.flags = ... → addFlags(...) (the property setter form doesn't compile because Intent.setFlags returns Intent, not void). BridgeViewModel.kt removed StateFlow.distinctUntilChanged() (deprecated no-op since StateFlow already deduplicates).
Markers: new code blocks tagged // === PHASE3-bridge-ui-followup: ... === / // === END PHASE3-bridge-ui-followup ===.
2026-04-12 — Phase 3 / status trio — android_phone_status tool + hermes-status CLI + /hermes-relay-status skill
Symmetric read-only counterpart to the pair trio. The relay ships GET /bridge/status (loopback, in a parallel agent's scope) that exposes live phone state — connection, battery, screen, foreground app, bridge permission flags, safety-rail config. This change builds the three consumers that wrap it so the agent, operators, and chat users all have a clean entry point.
plugin/tools/android_phone_status.py. New Hermes tool registered via the canonical tools.registry import pattern, modeled after plugin/tools/android_notifications.py. Zero args. Calls http://127.0.0.1:8767/bridge/status over loopback (no bearer — same trust model as /media/register and /pairing/register), returns {"status": "ok"|"error", "phone_status": {...}|null, "error": str|null}. Stdlib urllib.request only — no requests/httpx. Handles three canonical paths:
- 200 OK →
status=okwith the parsed relay body asphone_status. - 503 (relay alive but no phone has ever connected) →
status=okwithphone_status={"phone_connected": false, ...}. From the agent's perspective the phone not being connected isn't an error — it's just the current state, and the LLM should render it as prose. - URLError/OSError/timeout →
status=errorwitherror="relay unreachable". This is the only case where the agent should bail.
The tool's description is explicit about when to call it: before attempting any bridge operation (android_tap, android_type, android_screenshot) so the agent can check phone_connected + bridge.*_granted upfront instead of eating failures one by one. check_fn=lambda: True because gating status on bridge connectivity would be circular — it IS the connectivity check.
plugin/status.py. New CLI module modeled after plugin/pair.py. Probes the same endpoint, pretty-prints with a small ANSI _Palette helper that's a no-op when stdout isn't a TTY. Three distinct exit codes so shell scripts can distinguish the cases: 0 success + connected, 1 relay unreachable, 2 relay alive but no phone. --json flag emits raw JSON pass-through (always a stable envelope so callers can jq over it even on failure). --port N overrides RELAY_PORT. Invokable via python -m plugin.status or the hermes-status shim. fetch_status(port) is factored out as a pure-function core so the tests can exercise it without mocking argparse.
skills/devops/hermes-relay-status/SKILL.md. New slash-command skill mirroring hermes-relay-pair's structure verbatim — same frontmatter shape, same metadata.hermes.category: devops, same "When to Use / Prerequisites / Procedure / Pitfalls / Verification" section layout. Explains how to interpret the three exit codes, when to re-pair vs. when to restart the relay, and how to read the permission flags when the user's "accessibility is granted" but status says otherwise (OS killed the service). Picked up automatically by the external_dirs scan installer registers — no config change needed on upgrade.
install.sh / uninstall.sh — hermes-status shim. Added step [5/6] sibling block that writes ~/.local/bin/hermes-status alongside hermes-pair. Same template: a tiny bash shim that execs $HERMES_VENV_PY -m plugin.status "$@" with a friendly error if the venv python can't be found. Uninstaller mirrors it for removal — both shims come out together under [5/6]. Header comments in both scripts updated to reference the plural "shims".
plugin/tests/test_android_phone_status.py — 19 tests, stdlib unittest only. Covers the tool's success path, 503 mapping, 503 with empty body, connection refused (URLError), raw OSError, other HTTP errors (500), non-JSON body, non-object JSON ([1,2,3], 42), schema sanity (parameters.required == [], properties == {}), and the registry handler wrapper. Also covers the CLI's fetch_status return-triple shape across all three exit codes + main(["--json"]) happy path + all three failure modes. The rendering pass has two smoke tests: one verifies the happy path includes all the expected fields (device name, battery %, current app, blocklist count, destructive verbs, permission labels), one verifies the disconnected short-circuit omits the Device/Bridge/Safety sections entirely. Mocks urllib.request.urlopen via unittest.mock.patch, uses a tiny _FakeResponse context-manager stub and _http_error(code, body) helper that builds a real urllib.error.HTTPError whose .read() returns the fixture body (critical: the 503 handler reads the body to extract the phone-state envelope, so the mock has to wrap a BytesIO in the fp slot, not just pass bytes). All tests run green:
python -m unittest plugin.tests.test_android_phone_status -v
Ran 19 tests in 0.010s
OK
Files touched: new plugin/tools/android_phone_status.py, new plugin/status.py, new skills/devops/hermes-relay-status/SKILL.md, new plugin/tests/test_android_phone_status.py, edits to install.sh + uninstall.sh for the shim pair. No relay-side changes — the /bridge/status endpoint is owned by a parallel agent working in the plugin/relay/server.py scope. This change is pure consumer.
2026-04-12 — Phase 3 / safety-rails followup — deep link, overlay nag, in-app permission Test buttons
Three small UX wins on top of the merged Wave 2 safety rails. All gate cleaner first-run smoke testing on a real device.
Deep link from foreground notification → BridgeSafetySettingsScreen. BridgeForegroundService.ACTION_OPEN_SETTINGS previously dropped the user on MainActivity's home screen — they had to navigate Settings → Bridge safety manually. New app/src/main/kotlin/com/hermesandroid/relay/util/NavRouteRequest.kt is a tiny MutableSharedFlow<String> singleton that any external launcher (foreground service, broadcast receiver, shortcut intent) can post route requests to. MainActivity reads EXTRA_NAV_ROUTE in onCreate + onNewIntent and forwards via NavRouteRequest.tryRequest. RelayApp adds a LaunchedEffect(navController) that collects NavRouteRequest.requests and forwards each emission to navController.navigate(route) { launchSingleTop = true }. BridgeForegroundService now sets MainActivity.EXTRA_NAV_ROUTE = "settings/bridge_safety" on its launch intent. Single observer at the app root, no per-screen plumbing.
Overlay-permission nag banner on BridgeScreen. When the master toggle is on but Settings.canDrawOverlays(context) is false, a prominent red errorContainer-colored card appears at the top of BridgeScreen warning the user that confirmation prompts can't display. Without the banner, BridgeSafetyManager.awaitConfirmation was failing-closed silently — destructive verbs would just be denied with no UX hint about why. New private OverlayPermissionNagCard Composable in BridgeScreen.kt with a Warning icon, plain-language explanation, and a tap action that opens Settings.ACTION_MANAGE_OVERLAY_PERMISSION for our package. Goes away on its own once the permission is granted (BridgeViewModel re-probes on ON_RESUME).
In-app permission Test buttons in BridgePermissionChecklist. Matches the pattern ε already shipped on NotificationCompanionSettingsScreen. Each row in the bridge permission checklist now has an optional "Test" text-button next to its status icon. Tapping it runs a side-effect-free diagnostic on the underlying runtime state and surfaces the result via the global LocalSnackbarHost. New methods in BridgeViewModel:
testAccessibilityService()— checksHermesAccessibilityService.instanceis non-null ANDinstance.rootInActiveWindowis non-null. Three result strings cover "not bound", "bound but no active window", and "OK".testScreenCapture()— checksMediaProjectionHolder.projectionis non-null. Reports either "active grant" or "no grant — bridge will request consent next /screenshot".testOverlayPermission()— checksSettings.canDrawOverlays(context). Reports granted vs not-granted with a hint to tap the row.
Each method emits a one-shot human-readable string on a new MutableSharedFlow<String> (testEvents) that BridgeScreen collects in a LaunchedEffect and shows via snackbarHost.showSnackbar(message). Notification listener row keeps onTest = null because the existing dedicated test on NotificationCompanionSettingsScreen already covers it.
Why diagnostic-only, not full functional tests (e.g., actually capture a screenshot or actually flash an overlay): the diagnostic checks need zero side effects, no extra dependencies, no permission flows mid-test. They answer the "is this thing actually wired through?" question 80% of the time — full functional tests are a Wave 3 follow-up if the diagnostics aren't enough in practice.
Files touched: new util/NavRouteRequest.kt, new private composables in BridgeScreen.kt, edits to MainActivity.kt, BridgeForegroundService.kt, RelayApp.kt, BridgePermissionChecklist.kt, BridgeViewModel.kt. All marker blocks use PHASE3-safety-rails-followup.
2026-04-12 — Rename Phase 3 agent codenames from Greek letters to ASCII slugs
Bulk rename across the repo + Obsidian plan: replaced the Greek-letter agent codenames I'd been using since the original Phase 3 plan (α β accessibility bridge-ui notif-listener safety-rails voice-intents vision-nav — wait, only the first four were Greek; the actual original set was α β γ δ ε ζ η θ) with descriptive ASCII slugs. The Greek letters were a math/CS-paper convention and they sort nicely, but they're a pain to type, render badly in some terminals, and made commit-history search awkward.
Mapping:
| Old | New |
|---|---|
| α | bridge-server |
| flavor-split | flavor-split |
| accessibility | accessibility |
| bridge-ui | bridge-ui |
| notif-listener | notif-listener |
| safety-rails | safety-rails |
| voice-intents | voice-intents |
| vision-nav | vision-nav |
Scope: 42 repo files (DEVLOG, CLAUDE, Kotlin sources, Python sources, manifests, Vue docs) + the canonical Obsidian plan at Plans/Phase 3 — Bridge Channel.md. Implementation was a single sed pass per file with all 8 substitutions in one invocation, applied via a bash for-loop. Verified with grep -rc '[αβγδεζηθ]' returning 0 across the affected trees.
Git history is not rewritten. Existing commits and merge commits keep their original Greek-letter subjects (e.g., merge(phase3-α): migrate legacy bridge relay). Renaming history would require a force push and would invalidate every commit hash since the divergence point — not worth the cost for a cosmetic change. Going forward, new branches, commit messages, and marker blocks all use the ASCII slugs.
Marker block convention going forward: // === PHASE3-bridge-server: ... === / // === END PHASE3-bridge-server === (and the analogous Python # === ... === and XML <!-- === ... === --> forms). Followup blocks use PHASE3-<slug>-followup.
2026-04-12 — Phase 3 / Wave 1 hotfix — dedupe AccessibilityService entry; Gradle deprecation cleanup
Two small fixes discovered while smoke-testing Wave 1 in Android Studio.
Duplicate AccessibilityService entry in Android Settings. Bailey noticed Settings → Accessibility → Installed services listed two Hermes entries (Hermes-Relay and Hermes Bridge). Root cause: a coordination miss between agents flavor-split and accessibility during the Wave 1 agent-team run. flavor-split created the flavor manifests first and stubbed <service android:name=".accessibility.BridgeAccessibilityService" tools:ignore="MissingClass"> blocks in both flavor overlays, anticipating that accessibility would name the service class BridgeAccessibilityService. accessibility later named it HermesAccessibilityService instead. Manifest merger only collapses <service> elements with matching android:name, so two different names → two separate <service> entries in the merged manifest. The flavor-split-stub entry pointed at a class that doesn't exist anywhere — enabling that row in Android Settings would have crashed the bind.
Fix: removed the stub <service> blocks from both app/src/googlePlay/AndroidManifest.xml and app/src/sideload/AndroidManifest.xml entirely (the flavor manifests are now empty <application /> overlays kept around for future flavor-specific permissions / activities). The flavor distinction now lives purely at the resource layer — each flavor's res/xml/accessibility_service_config.xml carries its own use-case description and flag bitset, and Gradle's resource merger picks the right file at build time. Added a11y_service_label ("Hermes-Relay Bridge") to app/src/main/res/values/strings.xml so the single canonical service entry in app/src/main/AndroidManifest.xml can be labeled distinctly from the launcher icon.
Gradle deprecation warning. AGP printed android.dependency.excludeLibraryComponentsFromConstraints=true is deprecated, will be removed in version 10.0. The two flags useConstraints=true + excludeLibraryComponentsFromConstraints=true were doing the work of one — the second flag overrode the first to mean "actually, exclude library components from constraint resolution." AGP's recommended migration is useConstraints=false. Collapsed two lines in gradle.properties into one. Semantics unchanged.
Lessons learned (for future agent teams): when an agent that creates a manifest stub names a class to be filled in by another agent, the two agents need to agree on the class name up front — manifest merger does not auto-collapse <service> blocks with different android:name values, even if everything else matches. Putting the canonical class name in the plan's "Wire format" section would have prevented this.
2026-04-12 — Onboarding/Settings unification + lifecycle-aware health checks
Closed two long-standing UX gaps that compounded each other: (1) onboarding's "Connect" page only configured the API server side and discarded the QR's relay block (code, TTL, grants, transport hint), so users walked out of onboarding in a "half-paired" state and had to re-do setup in Settings; (2) connection status badges showed stale Connected/Disconnected for up to 30 s after foregrounding because nothing kicked a re-probe on Lifecycle.Event.ON_RESUME — the StateFlow snapshot was just preserved across backgrounding even when the underlying server had died or the network had flipped.
New: app/src/main/kotlin/com/hermesandroid/relay/ui/components/ConnectionWizard.kt — shared three-step pairing wizard (Scan → Confirm → Verify) with a circular step indicator at the top. Used by both OnboardingScreen and ConnectionSettingsScreen's "Pair with QR" button so first-run and re-pair stay in lockstep forever. Step 1 launches QrPairingScanner via the camera permission flow. Step 2 displays the scanned details (server URL, relay URL, grants, transport security badge), an inline TTL radio list, and a red "plain ws://" warning when the relay is insecure. Step 3 observes AuthState with a 15s withTimeout, surfaces classified errors via Retry/Back/Cancel, and calls onComplete() on AuthState.Paired. The wizard's "Skip for now" affordance only renders when the caller passes showSkip = true (onboarding sets that, Settings doesn't).
New: ConnectionViewModel.applyPairingPayload(payload, ttlSeconds) — single entry point for "user just confirmed a scanned QR + chose a TTL." Does the full credential dance: API URL + key, relay URL + insecure mode, applyServerIssuedCodeAndReset (wipes TOFU pin + applies code), setPendingGrants, setPendingTtlSeconds, disconnectRelay() + connectRelay(), then revalidate(). Replaces the ~50-line confirm-callback that used to live in ConnectionSettingsScreen.kt::SessionTtlPickerDialog.onConfirm. The wizard is the only caller now.
New: ConnectionViewModel.HealthStatus (Unknown / Probing / Reachable / Unreachable) + apiServerHealth + relayServerHealth StateFlows. Distinct from the existing apiServerReachable: Boolean so the UI can render a dedicated "Probing" pose right after foreground / network change. The boolean stays in place for legacy callers; new UI consumes the tri-state flow.
New: ConnectionViewModel.revalidate() — single entry point for "the world might have changed — re-check everything." Flips both health flows to Probing immediately so the badge poses don't flash stale Connected/Disconnected, then fires API + relay /health probes in parallel and joins them, and kicks reconnectIfStale() to bring the WSS back if it was holding a paired-but-disconnected state. Idempotent — guarded by a revalidationJob so rapid foreground/background cycles don't pile up parallel probes.
New: lifecycle hook in RelayApp.kt — DisposableEffect(lifecycleOwner) registers a LifecycleEventObserver that calls connectionViewModel.revalidate() on ON_RESUME. This is the fix for the resume-lag flash. Single observer at the app root so every screen gets fresh badges on foreground without each one wiring its own.
New: connectivity reaction inside ConnectionViewModel.init — the existing ConnectivityObserver.observe() flow was previously exposed as networkStatus but never read. Now a collector inside init calls revalidate() on every Available transition (with drop(1) to skip the seed value so we don't double-probe on first composition). Airplane-mode flips, network handoffs, and Tailscale up/down events now actually trigger a re-probe instead of being purely advisory.
New: periodic relay health loop — mirrors the existing 30s API health loop. Uses RelayHttpClient.probeHealth() (unauthenticated, 3s timeout, no rate-limit cost) to update relayServerHealth independently of the WSS heartbeat. Plugs the historical gap where relay status was only verified by WSS or manual Save & Test taps.
New: fourth state on ConnectionStatusBadge — BadgeState.Probing — gray pulsing dot at 1.2s cadence, distinct from the amber Connecting pulse at 0.8s and the green Connected pulse at 1.5s. Added a new state: BadgeState overload of both ConnectionStatusBadge and ConnectionStatusRow. The boolean overloads are preserved (with a new optional isProbing: Boolean = false parameter) so legacy call sites — chat header, terminal screen, bridge cards — keep working without churn. Only call sites that want a Probing state need to migrate.
Edited: app/src/main/kotlin/com/hermesandroid/relay/ui/onboarding/OnboardingScreen.kt — replaced the bespoke ConnectPage (which discarded the relay block, never walked the user through TTL picking, and let users exit onboarding in a half-paired state) and RelayPage (a bare URL text field) with a single ConnectPage that embeds ConnectionWizard. Pages list is now: Welcome → Chat → (Terminal → Bridge if relay enabled) → Connect (wizard). Onboarding's onComplete callback collapsed from 4 args (apiServerUrl, apiKey, relayUrl, relayPairingCode) to zero — the wizard owns credential application via applyPairingPayload, so the callback only needs to mark complete + navigate. The pager's bottom Next/Back navigation is hidden on the Connect page so the wizard owns its own affordances. Top-bar Skip is also hidden on the Connect page so the wizard's "Skip for now" is the only skip affordance.
Edited: app/src/main/kotlin/com/hermesandroid/relay/ui/screens/ConnectionSettingsScreen.kt — "Scan Pairing QR" + "Guided setup" buttons collapsed into a single "Pair with QR" button that opens the wizard inside a full-screen Dialog. "Re-pair" button (shown when paired) now opens the wizard instead of launching the camera directly. Removed: the cameraPermissionLauncher (wizard owns permission), pendingQrPayload state, the inline SessionTtlPickerDialog confirm-callback block (~50 lines), and the showPairingWalkthrough/PairingWalkthroughDialog dialog. The API/Relay/Session status rows now consume apiServerHealth and relayServerHealth so they render the new Probing pose during the post-resume window.
Edited: app/src/main/kotlin/com/hermesandroid/relay/ui/screens/SettingsScreen.kt — root Connection summary card's API and Relay rows updated to consume the new health flows + render Probing on resume. Status text reads "Checking…" during the probe instead of flashing stale "Connected"/"Unreachable".
Deleted: app/src/main/kotlin/com/hermesandroid/relay/ui/components/PairingWalkthroughDialog.kt (~370 lines) — the old 4-step dialog-counter wizard. Wizard supersedes it. SessionTtlPickerDialog.kt is preserved because the wizard reuses its ttlPickerOptions() and defaultTtlSeconds() helpers.
The end result: one canonical pairing flow used by both onboarding and Settings, badges that flip to "Checking…" instead of flashing stale state on resume, and an explicit Probing pose so the user can tell the difference between "we don't know yet" and "we know it's down."
2026-04-12 — Docs: two-track explainer + FeatureMatrix component
Closed the user-doc gap left by Phase 3's googlePlay/sideload Gradle flavor split. Before this, the user-docs site framed sideloading as just an alternative install path — there was no signal that the sideload APK actually unlocks tier 3/4/6 features (voice→bridge intent routing, vision-driven android_navigate, workflow recording) that are compiled out of the Play Store APK by BuildFlavor.bridgeTier{3,4,6} checks.
New: user-docs/.vitepress/theme/components/FeatureMatrix.vue — polished feature comparison component that matches the design language of HermesFlow.vue / InstallSection.vue / HeroDemo.vue (Space Grotesk + Space Mono, --vp-c-brand-1 purple accent, flat + border-separated, no gradients/shadows/blur). Renders a semantic <table> with one row per feature (tier numbers are an implementation detail users don't care about), grouped into Chat & voice / Bridge — read / Bridge — control / Safety rails / Install & updates sections. Three support states per cell (full / limited / none) with inline SVG icons inheriting currentColor from the cell's class. Sideload-only rows fade to --vp-c-text-3 opacity 0.6; "limited" cells get a • see note suffix that surfaces a per-row footnote about the read-only Play accessibility surface. Responsive: above 720px shows both columns side-by-side with a brand-soft tinted Sideload column; below 720px collapses to a single visible column with role=tablist mobile tab switcher (the semantic table stays intact for screen readers — the inactive column is hidden via display:none on mobile-only cells, not removed). Below 480px the text labels collapse to icon-only with the support state on the cell's aria-label. Zero npm deps — all icons are inline SVG.
New: user-docs/guide/release-tracks.md — canonical prose explainer. Plain-language opener ("Hermes-Relay ships in two flavors built from the same codebase"), TL;DR up top, "Why two tracks?" section that explains Google Play's accessibility-service scrutiny without jargon, embedded <FeatureMatrix />, decision guide ("Want X? → Track Y" bullets), "Can I switch later?" section explaining the side-by-side applicationIdSuffix = ".sideload" story, install instructions for both tracks (linking to existing getting-started.md content rather than duplicating), and a "Safety rails — always on, in both tracks" closer to make clear the floor isn't a sideload feature. Wired into the /guide/ sidebar between Installation & Setup and Chat.
Edited: user-docs/features/index.md — added a "Bridge — Phone Control" table with a Track column and a <span class="track-badge track-badge--sideload">Sideload only</span> convention for compiled-out features (voice→bridge, vision navigation, workflow recording). Added a "Bridge — Safety Rails" table to make the floor explicit. Removed Bridge from "Coming Soon" since Phase 3 is shipping. Added a "Choose your track" section near the bottom with <FeatureMatrix /> embedded and a link to release-tracks.md. Top-of-page intro now flags the two flavors and the badge convention.
Edited: user-docs/guide/getting-started.md — replaced the single-line "download the APK or wait for Play Store" sentence in step 1 with a short two-flavor pitch and a link to release-tracks.md. Did not duplicate the matrix here — just points to it.
Edited: user-docs/.vitepress/config.mts — added Release tracks to the /guide/ sidebar between Installation & Setup and Chat.
Edited: user-docs/.vitepress/theme/index.ts — registered FeatureMatrix as a global Vue component so MD files can use <FeatureMatrix /> without per-file imports (matches the existing HermesFlow registration pattern).
Build verified locally with npx vitepress build — clean compile, all pages render, no errors.
2026-04-12 — Phase 3 / Wave 2 / safety-rails: bridge safety rails
Branch: feature/phase3-zeta-safety-rails (off main @ 8d86a62 which has Wave 1 merged).
What landed. Tier 5 safety for the bridge channel — five enforcement components that together make it OK to actually ship agent device control:
-
Per-app blocklist (
BridgeSafetyPreferences.kt) — DataStore-backed set of package names. Ships with a conservative default covering common banking apps (Chase / Wells Fargo / BoA / Revolut / Monzo / Starling), payment apps (Venmo / Cash App / PayPal / Coinbase), password managers (1Password / Bitwarden / LastPass / Dashlane / Keeper), and 2FA apps (Google Authenticator / Authy / Duo). Users edit viaBridgeSafetySettingsScreen— searchable LazyColumn of installed apps with checkboxes. Enforcement is inBridgeSafetyManager.checkPackageAllowed(packageName: String?), called byBridgeCommandHandleragainstHermesAccessibilityService.currentApp; a blocked package fails fast with HTTP 403{"error": "blocked package <name>"}. -
Destructive-verb confirmation modal (
DestructiveVerbConfirmDialog.kt+BridgeStatusOverlay.kt) — when/tap_textor/typecarries a body that matches one of the configurable destructive verbs (default: send, pay, delete, transfer, confirm, submit, post, publish, buy, purchase, charge, withdraw) on a word-boundary regex,BridgeSafetyManager.awaitConfirmation(method, text)suspends the handler coroutine, shows a full-screen Compose modal through aSYSTEM_ALERT_WINDOWoverlay (necessary because the agent can act while the app is backgrounded), and waits on aCompletableDeferred<Boolean>under awithTimeout. Timeout default 30s; timeout = DENY. Fail-closed: if the overlay permission is missing, the check returns false. -
Auto-disable timer (
AutoDisableWorker.kt) — a coroutineJobowned byBridgeSafetyManagerwith adelay(minutes.toMillis()). Rescheduled on every accepted bridge command. On fire: flips the master toggle off viaHermesAccessibilityService.setMasterEnabled(context, false)and posts a one-shot "Bridge auto-disabled after idle" notification throughNotificationManagerCompat. Default idle window: 30 min, slider 5..120 min. Deviation from spec: spec called for WorkManager, butandroidx.workis not in the classpath and adding the dep is out of scope for this slice. The coroutine-job approach is documented inAutoDisableWorker.ktas the canonical upgrade path if WorkManager is added later. -
Persistent foreground service (
BridgeForegroundService.kt) — plainServicewithstartForeground(NOTIFICATION_ID, notification, FOREGROUND_SERVICE_TYPE_SPECIAL_USE)on Android 14+, graceful fallback below. Notification shows "Hermes agent has device control" with two action buttons: Disable (broadcasts back into the service which flips the master toggle) and Settings (opensMainActivity; TODO followup to wire a deep-link extra for direct nav toBridgeSafetySettingsScreen). Started/stopped fromBridgeViewModelvia amasterToggle.distinctUntilChanged().collect {}observer. -
Optional status overlay chip (
BridgeStatusOverlay.kt) — tiny floating "Hermes active" pill in the top-right corner. Off by default, opt-in viaBridgeSafetySettingsScreen. Uses the sameSYSTEM_ALERT_WINDOWpipeline as the confirmation modal — only oneWindowManagerattachment point per process. The overlay host implementsConfirmationOverlayHostso the safety manager can reach both the chip and the modal through the same singleton.
Files created (8):
app/src/main/kotlin/com/hermesandroid/relay/data/BridgeSafetyPreferences.ktapp/src/main/kotlin/com/hermesandroid/relay/bridge/BridgeSafetyManager.ktapp/src/main/kotlin/com/hermesandroid/relay/bridge/BridgeForegroundService.ktapp/src/main/kotlin/com/hermesandroid/relay/bridge/BridgeStatusOverlay.ktapp/src/main/kotlin/com/hermesandroid/relay/bridge/AutoDisableWorker.ktapp/src/main/kotlin/com/hermesandroid/relay/ui/screens/BridgeSafetySettingsScreen.ktapp/src/main/kotlin/com/hermesandroid/relay/ui/components/DestructiveVerbConfirmDialog.kt(also hostsBridgeStatusOverlayChip)app/src/main/kotlin/com/hermesandroid/relay/ui/components/BridgeSafetySummaryCard.kt
Shared-concern files touched (6) — all marked with // === PHASE3-safety-rails: ... === / === END PHASE3-safety-rails ===:
BridgeCommandHandler.kt— injectssafetyManager: BridgeSafetyManager?, runs the three-stage check (blocklist → confirmation → reschedule timer) before dispatching each action.AndroidManifest.xml— addsSYSTEM_ALERT_WINDOW+FOREGROUND_SERVICE_SPECIAL_USEuses-permission + a<service android:name=".bridge.BridgeForegroundService" android:foregroundServiceType="specialUse"/>declaration with thePROPERTY_SPECIAL_USE_FGS_SUBTYPEjustification property.BridgeScreen.kt— replaces theSafetyPlaceholderCardstub with the realBridgeSafetySummaryCarddriven byBridgeSafetyManager.peek()?.settingsand.autoDisableAtMs. NewonNavigateToBridgeSafetyparameter.SettingsScreen.kt— addsonNavigateToBridgeSafetycallback and a "Bridge safety"SettingsCategoryRowbetween Notification companion and Media.RelayApp.kt—Screen.BridgeSafetySettingsdata object, composable(route), wires the nav callback from Settings + from Bridge screen.BridgeViewModel.kt— observesmasterToggleand callsBridgeForegroundService.start/stop, reschedules/cancels the auto-disable timer on toggle changes, and combines the master toggle withstatusOverlayEnabledto drive the chip visibility.ConnectionViewModel.kt(wiring only, marked) — callsBridgeSafetyManager.install()+BridgeStatusOverlay.install()at construction and passes the manager intoBridgeCommandHandler.
Key design notes / blockers / decisions.
-
WorkManager not in classpath. Went with a coroutine-owned timer instead of adding a new dependency.
AutoDisableWorker.ktdocuments the upgrade path. This is acceptable because the toggle is in-process — no inter-process scheduling is required, and process death already implies the service is disconnected. -
SYSTEM_ALERT_WINDOW UX. Users must grant the permission through
Settings.ACTION_MANAGE_OVERLAY_PERMISSIONbefore either the confirmation modal or the status chip can display.BridgeSafetySettingsScreendetects viaSettings.canDrawOverlays(context)and kicks off the grant flow on first switch tap. Fail-closed: if the permission is missing when a destructive verb fires,BridgeSafetyManager.awaitConfirmationdenies the action outright — better to drop a command than to silently allow it. Phase 3 Wave 3 should add a prominent "grant overlay permission" nag on the BridgeScreen when the user enables the master toggle without having granted it. -
Compose in a WindowManager overlay.
ComposeViewattached viaWindowManager.addViewdoesn't automatically get aViewTreeLifecycleOwner— we synthesize an always-RESUMEDOverlayLifecycleOwner+ViewModelStoreOwner. SkippedSavedStateRegistryOwnerbecause the dialog only usesremember, notrememberSaveable, andandroidx.savedstateisn't pinned in the version catalog. If future overlay content needsrememberSaveable, pin that artifact and extendOverlayLifecycleOwner. -
Confirmation coroutine wiring. The safety manager keeps a
ConcurrentHashMap<Long, PendingConfirmation>keyed by a monotonic request id.awaitConfirmationregisters an entry, asks the overlay host to show the modal, thenwithTimeout(...) { deferred.await() }. The overlay's Allow / Deny buttons call back with the result. Timeouts dismiss the overlay programmatically. The suspendingBridgeCommandHandler.dispatchcall is what holds the slot — the relay sees a slow response instead of a premature 403, which matches the intended UX (user gets 30s to react, agent waits). -
accessibility/bridge-ui API dependencies. Relied on:
HermesAccessibilityService.currentApp(accessibility, @Volatile String?),HermesAccessibilityService.setMasterEnabled(context, enabled)(accessibility),HermesAccessibilityService.isMasterEnabled()(accessibility),BridgePreferencesRepositorypattern (bridge-ui),relayDataStoreextension (existing),BridgeViewModel.masterToggle: StateFlow<Boolean>(bridge-ui). All used as-is — no edits in accessibility/bridge-ui territory.
What's next.
- Wire a deep-link extra on
MainActivitysoBridgeForegroundService.ACTION_OPEN_SETTINGSlands directly onBridgeSafetySettingsScreen(currently opens the app and the user taps through). - Record
BridgeActivityEntryrows for blocked-package / denied-confirmation events so the bridge activity log shows why an agent command failed. - Wave 3 can add tests: unit tests for the destructive-verb regex word-boundary matching, an instrumented test for the foreground-service lifecycle, and a fake overlay host for
awaitConfirmationtimeout behavior.
2026-04-12 — Phase 3 / Wave 2 / voice-intents: voice→bridge intent routing (sideload-only, Tier 3)
Wired voice mode to optionally route transcribed utterances through the bridge channel instead of chat. When the user holds the voice button and says "text Sam I'll be 10 min late" on a sideload build, a lightweight keyword classifier recognizes the SMS intent, the voice layer speaks a confirmation ("About to text Sam: I'll be 10 min late. Say cancel to stop."), and a bridge envelope is queued for dispatch after a 5-second cancel window. "Open camera" / "scroll down" / "go back" / etc. execute immediately with no countdown.
Cross-flavor compile pattern. This is the first Wave 2 surface to exercise the googlePlay vs sideload flavor split that agent flavor-split is wiring up in parallel. VoiceBridgeIntentHandler is an interface + sealed IntentResult in main/ so VoiceViewModel has a stable compile-time reference. Two separate implementations ship per-flavor with matching FQCNs and a shared top-level factory function createVoiceBridgeIntentHandler(multiplexer):
app/src/googlePlay/kotlin/.../VoiceBridgeIntentHandlerImpl.kt—NoopVoiceBridgeIntentHandleralways returnsIntentResult.NotApplicable, never touches the bridge, never references any device-control / accessibility class. Keeps the Play APK honest with its conservative feature description.app/src/sideload/kotlin/.../VoiceBridgeIntentHandlerImpl.kt—RealVoiceBridgeIntentHandler+VoiceIntentClassifierwith keyword/regex patterns for SendSms / OpenApp / Tap / Scroll / Back / Home. Destructive intents go through the 5 s countdown; safe intents dispatch immediately.
VoiceViewModel.initialize() calls the factory exactly once; the ViewModel only ever holds the interface reference. No reflection, no runtime flavor check — Gradle picks the right impl at build time via the source-set flavor. The routing call site in processVoiceInput tries the handler first and falls through to chatVm.sendMessage(text) on NotApplicable.
Classifier heuristic. Six intents, first-match-wins, all case-insensitive with a filler-stripping preamble ("hey", "okay", "please", "can you", …):
| Intent | Pattern example | Parse |
|---|---|---|
| SendSms (with separator) | text <contact> saying <body> / text <contact>: <body> / send a message to <contact> saying <body> |
contact, body |
| SendSms (no separator) | text <contact> <body≥2chars> — conservative: 1-2 word contact, requires a body |
contact, body |
| OpenApp | `open | launch |
| Tap | `tap | press |
| Scroll | `scroll (to the)? up | down |
| Back | `(go | navigate)? back` |
| Home | `(press | go)? home(?: screen)?` |
Design bias: false negatives > false positives. "Can you text me when you're done?" falls through to chat because the SMS regex can't parse a contact + body from it. Anything that doesn't cleanly split into a structured action becomes a chat message. Each pattern is a single named-group regex so Wave 3 tuning is mechanical.
v1 confirmation simplification. Full conversational confirmation ("About to text… say yes or cancel") would need a multi-turn voice state machine we don't have. Instead: destructive intents speak the confirmation via the existing sentence-TTS queue, start a 5 s countdown coroutine, and dispatch unless VoiceViewModel.cancelPendingBridgeIntent() is called first. The overlay's Cancel button wires into that method. Full voice-confirmation is a Wave 3 follow-up (see Phase 3 plan row voice-bridge-confirmation).
Bridge envelope wire shape. Simple and documented so Wave 2 sibling safety-rails (bridge-safety-rails) owns the authoritative schema:
{ channel: "bridge",
type: "tool.call",
payload: { tool: "android_send_sms" | "android_open_app" | ... ,
args: { ... },
requires_confirmation: true|false,
source: "voice" } }
Emission goes through ChannelMultiplexer.send(envelope) — the new bridgeMultiplexer: ChannelMultiplexer? = null parameter on VoiceViewModel.initialize() is optional so existing call sites keep compiling until they're updated to pass the instance. The googlePlay factory ignores the multiplexer entirely.
Files:
New (main):
app/src/main/kotlin/com/hermesandroid/relay/voice/VoiceBridgeIntentHandler.kt— interface +sealed class IntentResult { Handled, NotApplicable }
New (googlePlay flavor):
app/src/googlePlay/kotlin/com/hermesandroid/relay/voice/VoiceBridgeIntentHandlerImpl.kt—NoopVoiceBridgeIntentHandlerapp/src/googlePlay/kotlin/com/hermesandroid/relay/voice/VoiceBridgeIntentFactory.kt— factory returning the no-op
New (sideload flavor):
app/src/sideload/kotlin/com/hermesandroid/relay/voice/VoiceBridgeIntentHandlerImpl.kt—RealVoiceBridgeIntentHandlerwith v1 countdown + envelope buildersapp/src/sideload/kotlin/com/hermesandroid/relay/voice/VoiceBridgeIntentFactory.kt— factory returning the real implapp/src/sideload/kotlin/com/hermesandroid/relay/voice/VoiceIntentClassifier.kt— regex-based classifier +VoiceIntentsealed class
Touched (main):
app/src/main/kotlin/com/hermesandroid/relay/viewmodel/VoiceViewModel.kt— newbridgeMultiplexerparam oninitialize(); factory call at the bottom of init; intent-routing block at the top ofprocessVoiceInput(between the Thinking-state update and the sentence-buffer reset); newcancelPendingBridgeIntent()public method for the overlay's Cancel button. All changes wrapped in// === PHASE3-voice-intents: ... === / // === END PHASE3-voice-intents ===markers.
Known cross-worktree dependency. This worktree assumes agent flavor-split's productFlavors { googlePlay; sideload } + BuildFlavor.bridgeTier3 constant lands in build.gradle.kts / FeatureFlags.kt before this branch merges. Without flavor-split's flavor config, Gradle won't resolve the createVoiceBridgeIntentHandler(...) reference from VoiceViewModel because neither flavor source set is active. Order of merges: flavor-split first, then voice-intents (and safety-rails + vision-nav).
Next steps (Wave 2 sibling + Wave 3):
- safety-rails owns the
bridgeenvelope contract — if the wire-level field names change (tool→name, etc.), updateVoiceBridgeIntentHandlerImpl.buildXxxEnvelopein one place. - Overlay wire-up (Bailey or a future agent): show
IntentResult.Handled.intentLabelin the voice overlay, add a Cancel button that callscancelPendingBridgeIntent()whenrequiresConfirmation=true. - Wave 3
voice-bridge-confirmation: replace the 5 s countdown with a proper multi-turn confirmation ("say yes or cancel" → STT → classifier for yes/no). - Classifier unit tests:
extractNextSentence-style unit tests forVoiceIntentClassifier.classify(). Top-level function already testable without an Application context.
2026-04-12 — Phase 3 / Wave 2 / vision-nav — android_navigate vision-driven navigation tool
Lands the Tier 4 "burner-phone trick, done sanely" plan row. New Hermes tool android_navigate(intent, max_iterations=5) closes the loop on high-level bridge automation: instead of the agent having to know exact node IDs or coordinates, it passes a natural-language intent ("compose a tweet about rainy weather") and the tool loops screenshot → vision model → action → repeat until the model emits done or hits the iteration cap.
Files added:
plugin/tools/android_navigate.py— main loop + registry registration. Reuses the same_bridge_url()/_auth_headers()/ANDROID_BRIDGE_URLtransport layer asandroid_tool.pyso pairing config stays single-sourced. Dispatches parsed actions to/tap_text,/tap,/type,/swipe,/press_key(the HTTP routes Wave 1 stood up). Screenshot per iteration goes through the bridge/screenshotendpoint, then triesplugin.relay.client.register_media()for an opaque token and gracefully falls back to afile://marker if the relay isn't reachable. Every step records{step, action, params, screenshot_token, reasoning, result}into a trace the agent gets back on both success and failure envelopes. Iteration cap defaults to 5, hard-clamped toABSOLUTE_MAX_ITERATIONS = 20so the agent can't accidentally burn capture budget.plugin/tools/android_navigate_prompt.py— prompt template + response parser. Parser is case-insensitive, tolerates trailing punctuation on the verb, allowsdoneto omit the PARAMS line, enforces per-verb param shape (e.g.tapneeds either(x,y)ints ornode_idstring), and returnsParsedAction(action="error", reasoning=<why>)on any malformed input so the loop can surface structural failures cleanly.plugin/tests/test_android_navigate.py— 35 stdlib-onlyunittesttests covering all 6 valid actions, every malformed-input branch (empty / missing lines / bad JSON / wrong shape / unknown verb / list-not-object / bad swipe direction), success path (single-step and multi-step), iteration cap, screenshot failure, parse error, action-execution failure,llm_gapenvelope, empty intent short-circuit,max_iterationsclamping, and schema sanity. Uses onlyunittest.mockso it runs cleanly viapython -m unittest plugin.tests.test_android_navigatewithout the pytestconftest.pythat imports theresponsesmodule.
LLM integration — known gap: The plan specifies "vision model integration" but doesn't pick a provider, and the plugin has no published LLM client surface for tools to call. The gateway's run loop is not re-entrant (calling the agent from inside a tool deadlocks the run). Rather than invent a new LLM client and ship an opinionated default, the loop has a call_vision_model injection point. Tests patch it; production either (a) sets HERMES_NAVIGATE_STUB_REPLY for smoke testing, or (b) swaps _default_vision_model for a real client (e.g. Anthropic messages.create with a base64 image block). Until (b) lands, calling android_navigate against a live phone returns a clean {"status": "error", "reason": "llm_gap", ...} envelope so the agent sees exactly why the loop can't run — no silent fakes, no import-time crashes. The _default_vision_model docstring documents the replacement contract.
Tool guardrails (from the plan):
- Never continuous capture — exactly one
/screenshotper iteration, only when the tool is invoked. - Default 5 iterations, hard-clamped ceiling of 20.
- 200 ms settle delay between action and next screenshot; callers needing longer waits use
android_waitexplicitly. intentmust be non-empty; empty/whitespace returns an error envelope without touching the bridge.
Files NOT touched: plugin/relay/*, plugin/relay/channels/bridge.py, plugin/tools/android_tool.py, plugin/tools/android_relay.py, plugin/relay/auth.py, plugin/relay/media.py, plugin/relay/voice.py, anything under app/. The slice is self-contained in plugin/tools/ as scoped.
Test results: python -m unittest plugin.tests.test_android_navigate → 35 tests, all pass in ~0.05 s.
2026-04-12 — Dynamic phone-status system prompt block
Replaced the single hardcoded "app context prompt" sentence with a transparent, granular block that reflects real Phase 3 bridge/permission state. The old toggle promised far more than it delivered — one static line, regardless of anything actually happening on the phone. The new block is a master toggle + 4 sub-toggles with a live preview card in Chat Settings.
New file: app/src/main/kotlin/com/hermesandroid/relay/util/PhoneStatusPromptBuilder.kt — pure function buildPromptBlock(settings, snapshot) returning String?. Returns null when master is off so no empty system message is sent. Defines AppContextSettings (5 booleans) and PhoneSnapshot (11 nullable fields). Output capped under ~100 words; the brief suggests calling android_phone_status for full detail, keeping per-turn token cost down while giving the agent a permission-lit entrypoint.
ConnectionViewModel: Four new DataStore keys alongside the existing KEY_APP_CONTEXT — KEY_APP_CONTEXT_BRIDGE_STATE (default true), KEY_APP_CONTEXT_CURRENT_APP (default false — privacy), KEY_APP_CONTEXT_BATTERY (default false — privacy), KEY_APP_CONTEXT_SAFETY_STATUS (default true). Each gets a StateFlow + setter mirroring the appContextEnabled pattern.
ChatViewModel: Deleted APP_CONTEXT_PROMPT constant, replaced var appContextEnabled: Boolean with var appContextSettings: AppContextSettings. send() now calls buildPromptBlock(appContextSettings, capturePhoneSnapshot()). The capturePhoneSnapshot() helper uses guarded reflection (runCatching on every lookup) to read Phase 3 classes — HermesAccessibilityService.instance, MediaProjectionHolder.projection, BridgeSafetyManager.peek(). The reflection path exists so ChatViewModel compiles cleanly before those classes land in this worktree (they're being built by a parallel agent); once they exist, reads light up with zero further code changes here. Non-Phase-3 sources (battery, overlay permission, notification listener flat-string) use direct platform APIs.
RelayApp.kt: Single LaunchedEffect now keys on all 5 toggles and writes a fresh AppContextSettings into chatViewModel.appContextSettings on any change.
ChatSettingsScreen.kt: Master toggle renamed to "Share phone status with agent". When enabled, AnimatedVisibility reveals 4 sub-toggle rows + a "Preview" Card. The preview uses remember(master, bridgeState, currentApp, battery, safetyStatus) to regenerate exact output text on every toggle change, calling the same buildPromptBlock with a neutral empty snapshot so the user sees the shape without leaking current phone state into the settings screen. Shows "(no system message will be sent)" when builder returns null.
Privacy model: Default configuration sends "mobile-friendly preamble + bridge permission summary + safety rails count" — no package names, no battery level. Users opt in explicitly to each privacy-sensitive field. The android_phone_status tool (being built in a parallel worktree) is the disclosure path for full detail so per-turn cost stays bounded.
Markers: all new blocks tagged // === PHASE3-status: ... === / // === END PHASE3-status === for grep.
2026-04-12 — Add canonical uninstall.sh + bootstrap docs
Companion to the bootstrap injection work below. There was no formal uninstall path before — install.sh is idempotent so most update flows worked, but cleanly removing the plugin (e.g., to test that install.sh works on a truly fresh state) required manually undoing 6 install steps. New uninstall.sh reverses them in opposite order:
- Stops + disables
hermes-relay.service, removes the systemd unit, daemon-reloads - Removes
~/.local/bin/hermes-pairshim - Scrubs the relay's
skills.external_dirsentry from~/.hermes/config.yamlvia the same yaml parsing pattern install.sh uses, with a.bakbackup before write — preserves all other entries - Removes
~/.hermes/plugins/hermes-relaysymlink + any legacy stales (hermes-android, etc.) - Removes
hermes_relay_bootstrap.pthfrom venv site-packages,pip uninstall hermes-relay - Removes
~/.hermes/hermes-relayclone (sanity-checked: refuses to delete a directory that doesn't have.git+install.sh)
What it never touches: ~/.hermes/.env (other tools authenticate against this), ~/.hermes/state.db (sessions DB shared with the gateway), ~/.hermes/hermes-agent/ (the agent itself), ~/.hermes/hermes-agent/venv/ (only our .pth is removed, not the venv core), and ~/.hermes/hermes-relay-qr-secret (kept by default — the QR signing identity is precious; opt in to wipe with --remove-secret).
Flags: --dry-run previews without changing anything, --keep-clone leaves the git tree in place, --remove-secret wipes the QR secret. Help text: bash uninstall.sh --help.
install.sh header docs updated to mention bootstrap injection (step 2) and the uninstall path. Success summary now prints both git pull && bash install.sh for updates and bash uninstall.sh --dry-run for previewing removal. README.md and user-docs/guide/getting-started.md got equivalent updates with the bootstrap explanation + uninstall flags.
2026-04-12 — Bootstrap injection: vanilla upstream hermes-agent now works with the plugin
Closed the "you must run our hermes-agent fork to get full features" gap. The Codename-11 fork (feat/api-server-enhancements, 13 commits, submitted as PR #8556) adds 20 management endpoints — /api/sessions/* CRUD, /api/memory, /api/skills, /api/config, /api/available-models, /api/sessions/{id}/chat/stream — that the Android app depends on. Until #8556 merges and reaches a release, vanilla upstream users were missing the sessions browser, conversation-history-on-restart, personality picker, command palette, and memory management.
The fix: a single .pth file that runs at Python interpreter startup. New hermes_relay_bootstrap/ package ships with the plugin, gets loaded by Python's site module before anything in hermes-agent imports aiohttp.web. The bootstrap installs a sys.meta_path finder that wraps the loader for aiohttp.web. When the import resolves, our wrapper replaces web.Application with a thin subclass. The subclass overrides __setitem__ to detect app["api_server_adapter"] = self — the line at gateway/platforms/api_server.py:1735 where the gateway gives us a reference to the adapter while the router is still mutable. At that moment we feature-detect by route path and bind ~14 management handlers from _handlers.py directly onto the same router. The gateway then continues with its own route registrations and starts the server. From the outside, vanilla upstream now serves all the fork's management endpoints.
What is NOT injected and why: the chat-stream handler (/api/sessions/{id}/chat/stream). It depends on _create_agent and agent.run_conversation with multimodal content — the riskiest cross-cutting upstream methods that the fork may have implicitly modified. Instead, chat goes through standard upstream /v1/runs, which already emits structured tool.started/tool.completed SSE events. This is arguably an upgrade — /v1/runs has live tool events whereas the sessions chat-stream path required a post-stream message-history reload to render tool cards. The Android client adapts via a new streamingEndpoint = "auto" mode (default for new installs).
Files added:
hermes_relay_bootstrap/__init__.py(~30 lines) — installs the meta_path finderhermes_relay_bootstrap/_patch.py(~170 lines) —_AioHttpWebFinder,_PatchingLoader,_PatchedApplication,_maybe_register_routeswith feature detection by route pathhermes_relay_bootstrap/_handlers.py(~500 lines) — 14 ported management handlers + helpers (sessions CRUD, memory CRUD, skills, config, available-models). Handlers takeadapteras a closure parameter rather than being bound methods, so we don't pollute upstream's class.hermes_relay_bootstrap.pth— single line:import hermes_relay_bootstrap
Files changed:
pyproject.toml— addedhermes_relay_bootstrap*to packages.find include listinstall.shstep 2 — copies the.pthinto the venv'ssite-packages/afterpip install -e. Verified empirically: setuptools' editable install does NOT shipdata-filesto site-packages reliably (it puts them invenv/data/instead, where Python'ssitemodule never looks). Manual copy is necessary.app/src/main/kotlin/.../HermesApiClient.kt— newServerCapabilitiesdata class +probeCapabilities()method that returns per-endpoint presence. Uses HEAD-method probes (initially tried OPTIONS but the gateway's CORS middleware intercepts preflight and returns 403 for both existing and missing paths — discovered during the live install test on 2026-04-12). Treats any non-404 status as "route exists."detectChatMode()becomes a thin compatibility wrapper aroundprobeCapabilities().toChatMode().app/src/main/kotlin/.../ConnectionViewModel.kt— exposesserverCapabilities: StateFlow<ServerCapabilities>, populates it fromprobeCapabilities()inrebuildApiClient(), addsresolveStreamingEndpoint(preference)helper that collapses"auto"to a concrete"sessions"or"runs"based on the latest snapshot. Default endpoint preference for new installs flipped from"sessions"to"auto".app/src/main/kotlin/.../ChatViewModel.kt—streamingEndpointdefault flipped from"sessions"to"runs"(the safer fallback before RelayApp pushes the resolved value).app/src/main/kotlin/.../ui/RelayApp.kt—LaunchedEffect(streamingEndpoint, serverCapabilities)recomputes the resolved endpoint when either changes, pushes into ChatViewModel.app/src/main/kotlin/.../ui/screens/ChatSettingsScreen.kt— Settings → Streaming endpoint dropdown gains an "Auto" option (alongside Sessions/Runs). Helper text dynamically shows which path Auto is currently using.CLAUDE.md— non-standard endpoints table rewritten to show all three "provided by" mechanisms (fork, bootstrap, upstream-merged), plus key files entries for bootstrap + capability detection.docs/decisions.md— new ADR 16 covering the runtime injection rationale, options considered (A/B/C/D), risks accepted, removal path.vault/Hermes-Relay.md— new "Bootstrap Injection Architecture" section with the compatibility matrix; updated "Hermes Integration" table to show two install paths (fork or bootstrap).
Compatibility matrix (all three combinations safe to ship the bootstrap with):
| Gateway version | Bootstrap behavior | Result |
|---|---|---|
Codename-11 fork (axiom) |
Detects existing /api/sessions, no-ops |
Fork serves everything natively ✓ |
| Vanilla upstream main | Detects no /api/sessions, injects routes |
Bootstrap-injected endpoints serve ✓ |
| Post-PR-#8556 upstream-merged | Detects existing /api/sessions, no-ops |
Upstream serves everything natively ✓ |
Removal path (when PR #8556 reaches a released hermes-agent version): delete hermes_relay_bootstrap/, delete hermes_relay_bootstrap.pth, remove the .pth drop block from install.sh. The Android client probeCapabilities() + streamingEndpoint = "auto" plumbing stays — it's permanent infrastructure that handles mixed-version deployments.
2026-04-12 — Phase 3 / Wave 1.5 — bridge-server-followup + notif-listener-followup (post-merge polish)
Two small follow-ups discovered during the Wave 1 agent-team merge. Both gate clean smoke testing of the merged Wave 1 work in Android Studio.
bridge-server-followup — POST /media/upload route on the relay. Agent accessibility's ScreenCapture posts screenshot bytes via multipart to /media/upload, but bridge-server only ported the existing /media/register (loopback + path-based). Phone has no shared filesystem with the relay, so /media/register was unusable for the bridge screenshot path. The new endpoint accepts a file multipart field with bearer auth (every paired phone has a session token), streams the bytes to a NamedTemporaryFile under tempfile.gettempdir() (which is in the default MediaRegistry.allowed_roots), enforces MediaRegistry.max_size_bytes while reading (returns 413 on overflow), then hands the path off to MediaRegistry.register() so token issuance / expiry / LRU eviction / GET /media/{token} are byte-for-byte identical to the loopback path. Marker block: # === PHASE3-bridge-server-followup: /media/upload === / # === END PHASE3-bridge-server-followup === in plugin/relay/server.py. tempfile import added at the top. Route registered next to /media/register. py_compile clean.
notif-listener-followup — wire HermesNotificationCompanion.multiplexer + Settings nav row. Agent notif-listener flagged two small touch-ups in its handoff:
ConnectionViewModel.initnow setsHermesNotificationCompanion.multiplexer = multiplexeronce, immediately after the bridge handler registration. The companion service buffers up to 50 envelopes in its ownpendingEnvelopesqueue while the slot is null, so wiring it from here (rather than at service-bind time) is safe — the buffer drains on the nextonNotificationPostedonce the slot is set. Marker:// === PHASE3-notif-listener-followup: notification companion multiplexer wiring === / // === END PHASE3-notif-listener-followup ===.SettingsScreengains a newonNavigateToNotificationCompanion: () -> Unitcallback parameter and aSettingsCategoryRowbetween Voice mode and Media (Icons.Filled.Notifications, "Notification companion", "Let your assistant triage notifications you've shared").RelayAppaddsScreen.NotificationCompanionSettings(routesettings/notifications), wires the new callback in theSettingsScreencall site, and registers acomposable(...)that hostsNotificationCompanionSettingsScreen(onBack = popBackStack). Both new imports added.
After this entry the Bridge tab + Notification companion screen are both reachable through the normal navigation tree, and accessibility's screenshot pipeline has a working server-side endpoint. Wave 2 (safety-rails safety, voice-intents voice-bridge, vision-nav vision) is unblocked once Bailey confirms both build flavors compile in Android Studio.
2026-04-12 — Phase 3 / Wave 1 / bridge-server — Migrated legacy bridge relay into unified relay (port 8767)
Retired the standalone bridge relay (plugin/tools/android_relay.py + the duplicate top-level plugin/android_relay.py, both listening on port 8766) and folded its functionality into the unified Hermes-Relay on port 8767 as the bridge channel. The wire protocol (bridge.command / bridge.response / bridge.status) stays byte-for-byte identical — only the transport changed. Agents accessibility, bridge-ui, notif-listener can now build against a single port.
plugin/relay/channels/bridge.py— replaced the stub with a realBridgeHandler. One handler instance perRelayServer; holdsphone_ws+pending: dict[request_id, Future]behind anasyncio.Lock.handle_command(method, path, params, body)mints a request_id, registers the future, sends abridge.commandenvelope, and awaits a response with 30s timeout (matches the legacyandroid_relay._RESPONSE_TIMEOUT).handle(ws, envelope)opportunistically latchesphone_wsand dispatchesbridge.response/bridge.status.detach_ws(ws, reason)fails all pending futures withConnectionErrorso HTTP callers don't hang when the phone drops.plugin/relay/server.py— added 14 HTTP routes (/ping,/screen,/screenshot,/get_apps,/apps[legacy alias],/current_app,/tap,/tap_text,/type,/swipe,/open_app,/press_key,/scroll,/wait,/setup) each delegating to_bridge_dispatch → server.bridge.handle_command. BridgeError → 503/504/502 depending on message. Wiredserver.bridge.detach_ws(ws)into_on_disconnectso phone drops instantly fail in-flight commands instead of hanging to the 30s timeout. All additions are bracketed by# === PHASE3-bridge-server: ... === / # === END PHASE3-bridge-server ===markers so the Agent notif-listener notification-listener merges are mechanical.plugin/android_tool.py+plugin/tools/android_tool.py— BRIDGE_URL default changed fromhttp://localhost:8766tohttp://localhost:8767(both copies;plugin/android_tool.pyis the one imported byplugin/__init__.py,plugin/tools/android_tool.pyis the standalone-toolset copy)._relay_port()falls back throughANDROID_RELAY_PORT → RELAY_PORT → 8767.android_setup()rewritten: no longer imports the deletedandroid_relaymodule, instead probeshttp://localhost:<port>/healthto verify the unified relay is up and returns a structured error if not. Env-var side effects preserved so the existingtest_android_tool.py::TestSetupstill passes.plugin/android_relay.py+plugin/tools/android_relay.py— DELETED. Both copies of the standalone relay are gone.plugin/tests/test_bridge_channel.py— new unittest suite (7 tests) covering envelope routing, future resolution, timeout cleanup, disconnect cleanup, send-failure cleanup, and the legacy-timeout regression guard. Uses a_FakeWsstand-in and bypasses the pytestconftest.pythat importsresponses. Run withpython -m unittest plugin.tests.test_bridge_channel. All 7 green locally; existingtest_relay_media_routes/test_qr_sign/test_session_grants/test_media_registrystill pass (22 + 47 assertions) socreate_app+ route registration hold.
Auth model judgment call: the bridge HTTP routes are unauthenticated at the HTTP layer, matching the legacy standalone relay. Defensible because (a) the trust boundary is unchanged — only same-host processes reach localhost:8767, (b) a disconnected/unpaired phone naturally causes every call to fail with 503, and (c) the bridge grant is already tracked per-session in Session.grants["bridge"] so Wave 2 (safety-rails) can add a bearer wrapper without touching the handler. Noted inline in the PHASE3-bridge-server block so Agent safety-rails can find it.
Branch: feature/phase3-alpha-bridge-server-migration.
2026-04-12 — Phase 3 / Wave 1 / flavor-split — googlePlay + sideload build flavors
Agent flavor-split (bridge-flavor-split) adds the Gradle flavor split that the Phase 3 Bridge channel needs before any accessibility code lands. Google Play reviews AccessibilityService heavily, so Phase 3 ships two parallel release tracks from one codebase: a conservative googlePlay flavor with a notifications-and-confirmations use-case description, and a sideload flavor with the full agent-control description for GitHub Releases / F-Droid / ADB distribution.
Scope of this change:
app/build.gradle.kts—flavorDimensions += "track"with two product flavors. Thesideloadflavor carriesapplicationIdSuffix = ".sideload"andversionNameSuffix = "-sideload"so both tracks can coexist on the same device during Phase 3 testing. ThegooglePlayflavor keeps the canonicalcom.hermesandroid.relayapplicationId so existing v0.2.0 Play Store installs upgrade cleanly and Play Console keeps its release history. Decision trade-off: power users with both installed will see two launcher icons until we differentiate labels in a follow-up.app/src/googlePlay/*— flavor manifest overlay declaring.accessibility.BridgeAccessibilityService(owned by Agent accessibility insrc/main/), conservativeaccessibility_service_config.xml(event subset:typeWindowStateChanged|typeWindowContentChanged|typeViewClicked,flagDefaultonly, no gestures,canRetrieveWindowContent=true), anda11y_description_googleplaytargeted at Play Store policy review.app/src/sideload/*— flavor manifest overlay, full-capabilityaccessibility_service_config.xml(typeAllMask,flagRetrieveInteractiveWindows|flagReportViewIds|flagRequestTouchExplorationMode,canPerformGestures=true), anda11y_description_sideloadwith explicit voice/vision/logging language.FeatureFlags.kt— newBuildFlavorobject withcurrent/displayName/ sixbridgeTier1..6flags. Tiers 1, 2, 5 are baseline-true for both tracks. Tiers 3 (voice-first), 4 (vision-first), 6 (ambitious future) areget() = current == SIDELOADso UI code can do a singleif (BuildFlavor.bridgeTier3)check and R8 can fold the branch at release-build time for the Play track.AboutScreen.kt— added a small "Track: Google Play" / "Track: Sideload" row under the existing Version row (which owns the 7-tap dev-options reveal). Marked with=== PHASE3-flavor-split ===banners so bridge-ui and notif-listener can land their own additions without merge conflicts.
Not shipped in this change: the actual BridgeAccessibilityService class (Agent accessibility) and any tier-gated UI surfaces (Agents bridge-ui/notif-listener). The manifests reference .accessibility.BridgeAccessibilityService with tools:ignore="MissingClass" so the Gradle + AAPT check doesn't block Agent accessibility's landing.
Branch: feature/phase3-beta-bridge-flavor-split.
2026-04-12 — Phase 3 / Wave 1 / accessibility — accessibility runtime (service + reader + executor + capture)
Wave 1 Agent accessibility (accessibility-runtime) landing the phone-side execution layer for the bridge channel. Five new files under a new com.hermesandroid.relay.accessibility package plus BridgeCommandHandler under network/handlers:
HermesAccessibilityService— masterAccessibilityServicesubclass. Self-registers as a@Volatilesingleton ononServiceConnected, clears on unbind/destroy. Caches the foregrounded package fromTYPE_WINDOW_STATE_CHANGEDevents (the only event type we consume — content-change events fire thousands/min and we read the tree on demand viarootInActiveWindow). Master enable flag lives in DataStore (bridge_master_enabled) so UI + safety rails can toggle it without killing the service.ScreenReader— UI tree →ScreenContent(rootBounds, nodes[], truncated).@Serializableoutput; oneScreenNodeper interesting node (non-blank text, content description, clickable/longClickable/scrollable/editable). Hard caps:MAX_NODES=512,MAX_TEXT_LEN=2000per field.findNodeBoundsByTextandfindFocusedInputhelpers for the executor. Recycles child nodes in atry/finallypattern per-iteration.ActionExecutor—tap/tapText/swipe/scrollviaGestureDescriptionwrapped insuspendCancellableCoroutineso the suspend form actually waits forGestureResultCallback.onCompleted.typeTextusesACTION_SET_TEXTwith aBundlearg.pressKeymaps a curated string vocabulary (home/back/recents/notifications/quick_settings/power_dialog/lock_screen) toAccessibilityService.GLOBAL_ACTION_*constants — we deliberately don't accept rawKeyEventcodes so agents can't inject arbitrary keypresses.wait(ms)clamped to 15s. Every method returnsActionResult(ok, data, error)so the handler can map 1:1 to HTTP-style status codes.ScreenCapture—MediaProjection→VirtualDisplay→ImageReader→ PNG bytes → multipart upload toPOST /media/uploadon the relay. CropsrowStridepadding beforeBitmap.copyPixelsFromBuffer. 2.5s capture timeout.MediaProjectionHoldersingleton holds the per-session grant — Bridge UI (Agent bridge-ui) is responsible for theActivityResultLauncherflow that callsonGranted(resultCode, data). Registers aMediaProjection.Callbackto null the holder when the system revokes.BridgeStatusReporter— coroutine that emitsbridge.statusenvelopes every 30s withscreen_on(viaPowerManager.isInteractive),battery(BatteryManager.BATTERY_PROPERTY_CAPACITYwith a sticky-intent fallback for OEM quirks),current_app(from the service singleton), andaccessibility_enabled(true when the service instance is non-null). Owned byConnectionViewModel.BridgeCommandHandler— routes inboundbridge.commandenvelopes to the executor and emitsbridge.response. Wire paths:/ping(works without the service),/tap,/tap_text,/type,/swipe,/scroll,/press_key,/wait,/screen,/screenshot,/current_app. Gates everything except/pingand/current_appon the master-enable toggle, returning 503 if the service isn't connected or 403 if the soft master is off./screenserializes the fullScreenContentviakotlinx.serialization.json.
Wired into ChannelMultiplexer.kt via a // === PHASE3-accessibility === marked section (simplified the existing bridge branch that was stubbed with a TODO), and ConnectionViewModel.kt gets a matching marker block that instantiates ScreenCapture, BridgeCommandHandler, and BridgeStatusReporter, registers the handler, and starts the reporter. AndroidManifest.xml declares the service with BIND_ACCESSIBILITY_SERVICE permission + intent filter + @xml/accessibility_service_config meta-data (the XML itself is flavor-provided by Agent flavor-split). Added FOREGROUND_SERVICE, FOREGROUND_SERVICE_MEDIA_PROJECTION, and POST_NOTIFICATIONS for the Wave 2 persistent-notification work.
Known blocker for bridge-ui to wire: MediaProjectionManager.createScreenCaptureIntent() requires an Activity-scoped result. ScreenCapture.createConsentIntent() exposes the intent; the Bridge screen needs an ActivityResultLauncher<Intent> that forwards the result to MediaProjectionHolder.onGranted(context, resultCode, data). Until that lands, /screenshot returns 503 with a clear error message.
Known blocker for bridge-server to fix: the relay's /media/register endpoint is loopback-only and path-based (see plugin/relay/media.py). The phone can't use it — there's no shared filesystem. ScreenCapture.uploadViaMultipart POSTs to /media/upload (new endpoint; mirrors the /voice/transcribe multipart pattern) which doesn't exist yet. Until bridge-server ships it, /screenshot surfaces a 404 with the exact message "relay /media/upload endpoint not found — server needs Phase 3 bridge-server migration". Token extraction uses the same {"ok": true, "token": "..."} shape as /media/register.
Branch: feature/phase3-accessibility-runtime.
2026-04-12 — Phase 3 / Wave 1 / bridge-ui — BridgeScreen rewrite (bridge-screen-ui)
Replaced the BridgeScreen "Coming Soon" placeholder with the real Tier-1
control surface from the Phase 3 plan. Wave 1 Agent bridge-ui (bridge-screen-ui)
deliverable — the UI scaffold is now ready to render whatever state Agent accessibility's
HermesAccessibilityService exposes once its runtime lands.
New files:
app/src/main/kotlin/.../data/BridgePreferences.kt— DataStore repo withbridge_master_enabledboolean and serializedbridge_activity_logJSON list (capped at 100 entries). MirrorsVoicePreferences.kt/MediaSettings.ktstyle. Uses lenientJsonfor forward-compat.app/src/main/kotlin/.../viewmodel/BridgeViewModel.kt— AndroidViewModel exposingmasterToggle,bridgeStatus,permissionStatus,activityLogStateFlows. Reads a11y service enablement viaSettings.Secure.ENABLED_ACCESSIBILITY_SERVICES, overlay viaSettings.canDrawOverlays, notification listener viaenabled_notification_listeners. SeedsBridgeStatuson init fromPowerManager.isInteractive+BatteryManager.BATTERY_PROPERTY_CAPACITYso the card isn't empty before accessibility lands. Four explicitTODO(accessibility-handoff)markers documenting the StateFlow/class-name/activity-writer surface that accessibility needs to expose for final wiring.app/src/main/kotlin/.../ui/components/BridgeMasterToggle.kt— headline Switch card with status inlines (device / battery / screen / current app), explanation dialog (required by Play Store a11y review), and a11y-granted gate so users can't flip it on before enabling the service.app/src/main/kotlin/.../ui/components/BridgeStatusCard.kt— standalone status card withConnectionStatusBadgeintegration. Usable independently of the master toggle so safety-rails can re-arrange the cards without losing state.app/src/main/kotlin/.../ui/components/BridgePermissionChecklist.kt— four-row checklist (Accessibility / Screen Capture / Overlay / Notification Listener) with tap-to-open Intent launchers wrapped inrunCatchingfor OEM skin safety.ACTION_ACCESSIBILITY_SETTINGS,ACTION_MANAGE_OVERLAY_PERMISSIONwith package URI, and the directenabled_notification_listenersintent string.app/src/main/kotlin/.../ui/components/BridgeActivityLog.kt— scrollableLazyColumn(bounded atheightIn(max = 320.dp)) with tap-to-expand row showing full timestamp, status, result text, and optional screenshot token.java.time.DateTimeFormatterforHH:mm:ss/yyyy-MM-dd HH:mm:ss. Stubbed screenshot thumbnail rendering with a TODO pointing at accessibility's MediaRegistry upload path.
Shared-concern edits (marked with PHASE3-bridge-ui: banners):
app/src/main/kotlin/.../ui/RelayApp.kt— the existingBridgeScreen()call wraps in the bridge-ui marker. Signature compatible:BridgeScreendefaults its ViewModel viaviewModel(), so no new nav-graph plumbing needed.
Each new component carries a @Preview (or two — e.g.,
BridgeMasterTogglePreviewOn vs PreviewBlocked) so Bailey can iterate in
Android Studio's preview pane without rebuilding the whole app.
accessibility-handoff points (in priority order, documented in BridgeViewModel KDoc):
bridgeStatusStateFlow — bridge-ui stubs a best-effort read from system APIs; accessibility replaces with the real HermesAccessibilityService status flow.A11Y_SERVICE_CLASSconstant — bridge-ui hard-codes"com.hermesandroid.relay.accessibility.HermesAccessibilityService". accessibility confirms the final FQCN.recordActivitywrite path — accessibility's command dispatcher callsBridgeViewModel.recordActivity(entry)on everybridge.commandto populate the activity log. Until then the log stays empty and the empty- state copy shows.- Master toggle read from accessibility — accessibility's service reads
bridge_master_enabledfromBridgePreferencesRepository.settingsand treats it as the runtime disable switch. No extra wiring needed from bridge-ui; accessibility just imports the repo.
Build-flavor flags (BuildFlavor.bridgeTierN from flavor-split) are not referenced yet
— this worktree pre-dates flavor-split's landing. Once flavor-split's object lands on main, bridge-ui's
components remain compatible (they don't tier-gate anything; Tier 1 is
always-on in both flavors per the plan).
Safety card is a placeholder stub — Agent safety-rails (Wave 2) owns the real safety
UI. bridge-ui's SafetyPlaceholderCard just says "Configure in Bridge Safety
Settings" with a Wave-2 teaser subtitle.
Branch: feature/phase3-delta-bridge-screen-ui.
2026-04-12 — Phase 3 / Wave 1 / notif-listener — notification companion (opt-in triage helper)
Adds an opt-in helper that lets the user's Hermes assistant read notifications they've explicitly granted access to via Android's standard NotificationListenerService API — the same one Wear OS, Android Auto, and Tasker have used for over a decade. Disabled by default; the user controls grant + revoke via Android Settings → Notification access.
Three pieces, all marked with PHASE3-notif-listener block markers in shared files:
-
Phone —
NotificationListenerService+ Compose settings screen. Newapp/src/main/kotlin/.../notifications/package withHermesNotificationCompanion(the bound service) +NotificationModels(@Serializable NotificationEntry) +ui/screens/NotificationCompanionSettingsScreen(About / Status / Test sections, mirrorsVoiceSettingsScreenstyle). The service buffers up to 50 envelopes in aConcurrentLinkedQueueifcompanion.multiplexerisn't wired yet (cold-start ordering), drains on the nextonNotificationPosted, then sends viamultiplexer.sendNotification(envelope)— a thin new wrapper inChannelMultiplexerthat fast-paths to no-op whensendCallback == null(relay offline). Notifications with empty title+text are skipped (background-sync placeholders that just confuse the LLM). TheStatusrow usesLifecycleEventObserver(ON_RESUME)so the grant state updates immediately when the user comes back from Android Settings. -
Server — bounded in-memory deque. New
plugin/relay/channels/notifications.py::NotificationsChannelwithrecent: collections.deque[dict]capped at 100 entries viamaxlen(LRU-by-time eviction for free).handle()dispatchesnotification.postedenvelopes tohandle_envelope()which appends;get_recent(limit)returns newest-first withlimitclamped to[1, max_entries]. Wired intoRelayServer.__init__asself.notificationsand dispatched in_on_messageforchannel == "notifications". The cache is in-memory only and lost on restart by design — same semantics as a smartwatch out of range. -
Agent tool —
android_notifications_recent(limit=20). Newplugin/tools/android_notifications.pyregisters the tool into thetools.registry(with thetry/except ImportErrorpattern matchingandroid_tool.py). It hitshttp://127.0.0.1:8767/notifications/recent?limit=Nover loopback via stdliburllib.request— no auth needed becausehandle_notifications_recentskips bearer for loopback callers (matches the/media/registerand/pairing/registertrust model). Remote callers still go through_require_bearer_session. The tool docstring explicitly frames it as "List recent notifications the user has shared with this assistant" so the LLM treats absence as "not granted yet" rather than an error.
Files touched:
- New:
plugin/relay/channels/notifications.py,plugin/tools/android_notifications.py,app/src/main/kotlin/.../notifications/HermesNotificationCompanion.kt,app/src/main/kotlin/.../notifications/NotificationModels.kt,app/src/main/kotlin/.../ui/screens/NotificationCompanionSettingsScreen.kt - Edited (additive only, marker-blocked):
plugin/relay/server.py(import + handler init + HTTP route + dispatch + route registration),app/src/main/kotlin/.../network/ChannelMultiplexer.kt(sendNotificationwrapper),app/src/main/AndroidManifest.xml(service entry withBIND_NOTIFICATION_LISTENER_SERVICEpermission),app/src/main/res/values/strings.xml(notification_companion_label) - Docs:
CLAUDE.mdKey Files table (8 new rows / additive notes),DEVLOG.md(this entry)
Decisions:
- No new session grant type. Spec explicitly said don't add
notificationstoauth.pygrants — this is opt-in via Android system permission, not via per-channel session grant. Reuses the existingchatgrant trust boundary. - Loopback bypass for the tool. The route's spec said "bearer auth gated like /media/*", but the tool is in-process on the host, so we mirror
/media/register's loopback gate: 127.0.0.1 callers skip the bearer check, remote callers still require a session token. This means the tool doesn't need to grovel through the relay's session store to mint a token. - Drop on relay-offline rather than buffer at the multiplexer. A wearable doesn't replay notifications it missed while out of range; we don't either. The cold-start buffer in the service is for the much shorter "service bound but multiplexer not wired yet" window.
activeNotificationsfor the Test button, not a relay round-trip. Lets the user verify the listener is bound even if the relay is unreachable, which is the more common failure mode at first-grant time.
Validation: python -m py_compile plugin/relay/channels/notifications.py plugin/tools/android_notifications.py plugin/relay/server.py → OK. Kotlin compile happens on Bailey's next Android Studio run.
Branch: feature/phase3-epsilon-notification-companion (off worktree-agent-a84b51cc).
Next: wire HermesNotificationCompanion.multiplexer from ConnectionViewModel after relay handshake (one-line touch), add a Settings entry-point row that navigates to NotificationCompanionSettingsScreen, and (Phase 3 follow-up) consider a per-package allow/deny list so the user can mute social-media spam from the listener pipeline before it ever hits the relay.
2026-04-12 — Fix: TTS waveform stays alive through multi-sentence playback
The waveform was flatlinining after the first sentence while audio kept playing. Root cause: maybeAutoResume() fired after every sentence in the TTS consumer loop. The SSE stream finishes before TTS plays all queued sentences, so streamObserverJob?.isActive was already false → state flipped to Idle → amplitude bridge stopped → waveform died. Fix: restructured TTS consumer from for loop to while + tryReceive peek. maybeAutoResume only fires when the queue is actually drained (tryReceive returns failure), not between sentences. Between sentences within the same response, tryReceive succeeds immediately and the consumer skips the Idle transition. Additionally, the consumer re-asserts Speaking state before each synthesis call to handle the edge case where the queue was briefly empty between observer pushes.
2026-04-12 — Classified error feedback across voice/chat/settings/pairing
Ended the "error: unknown" era. New RelayErrorClassifier.kt converts any Throwable → HumanError(title, body, retryable, actionLabel) based on exception type + context tag. Branch order: UnknownHostException → ConnectException → SocketTimeoutException → SSLException → SecurityException → IllegalStateException → IOException (message-scan for HTTP 401/403/404/413/500/503) → default. SSL before IOException is load-bearing since SSLPeerUnverifiedException extends IOException. Context tags (transcribe, synthesize, voice_config, record, pair, save_and_test, media_fetch, send_message) shape the title and drive 404 body specialization.
Global SnackbarHost via LocalSnackbarHost: CompositionLocal<SnackbarHostState> at RelayApp scope — every screen can toast via LocalSnackbarHost.current.showHumanError(err). Both VoiceViewModel and ChatViewModel gained errorEvents: SharedFlow<HumanError> (one-shot events, replay=0, buffer=4, DROP_OLDEST). 5 voice error sites + 4 chat error sites converted to use the classifier. Mic permission banner in ChatScreen rebuilt with "Open Settings" action. ConnectionSettingsScreen Save & Test and manual pair now show classified errors.
2026-04-12 — Voice polish: reactive waveform, pill edges, stop semantics, enter/exit chimes
Comprehensive polish pass on voice mode addressing issues surfaced during live testing:
- Waveform sensitivity: four layers of damping were fighting us. Fixed at every stage: perceptual curve in VoiceRecorder (noise-floor + speech-ceiling + sqrt), attack/release envelope follower in VoiceViewModel (0.75/0.10 at 60Hz), killed the Compose spring in VoiceWaveform, amplitude-driven phase velocity via
withFrameNanosticker (speeds up 3.5× at peak amplitude). - Pill edges: dual-technique merge — geometric
sin(π·t)taper forces wave to centerY at endpoints +BlendMode.DstInhorizontal gradient mask insaveLayer. Studied Conjure's pill approach (dark gradient overlay against solid background) — won't work on our translucent overlay, hence the layer technique. - TTS replying to wrong turn:
ignoreAssistantIdcaptures the pre-send last-assistant-id so the stream observer doesn't replay the previous turn's response. Root cause: StateFlow.collect replays current value on subscribe, and the current value at subscribe time still has the previous assistant message. - Stop semantics:
interruptSpeaking()rewritten to drain queue + stop player +chatViewModel.cancelStream()+ cancel all jobs → Idle (not Listening). Old version only paused playback; SSE kept feeding ttsQueue. - Enter/exit chimes: new
VoiceSfxPlayer.kt— pre-synthesized 200ms PCM sweeps (440→660 Hz ascending enter, mirror exit) viaAudioTrack.MODE_STATIC. Phase-accumulated to avoid chirp artifacts. - Stop button color: hardcoded vivid red
Color(0xFFE53935)for both Listening and Speaking states (Material 3 darkcolorScheme.errorresolved to pale pink). - Scrollable response: overlay response Column now
weight(1f, fill=false) + verticalScroll. - NaN guards:
VoicePlayer.computeRmsandVoiceViewModel.sanitizeAmplitude—Float.coerceInsilently passes NaN per IEEE 754. - Gradle logcat task:
silenceAndroidViewLogsExec task hooked viafinalizedByon allinstall*tasks — runsadb shell setprop log.tag.View SILENTto suppress Compose's Android 15setRequestedFrameRate=NaNspam.
2026-04-12 — Relay startup: Python-side .env bootstrap + systemd user service
Fixed a drift-class bug where the relay would 500 on /voice/transcribe with STT provider 'openai' configured but no API key available after any restart that wasn't preceded by a manual source ~/.hermes/.env. Root cause: the relay was run as a detached nohup process, which inherits whatever env the launching shell happens to have exported — not what's in ~/.hermes/.env. The gateway already solves this via hermes_cli/main.py:144 calling load_hermes_dotenv(project_env=...) at import time, which is why its user systemd unit carries no EnvironmentFile= directive. The relay was missing that pattern entirely.
New: plugin/relay/_env_bootstrap.py
A tiny helper (55 lines) exposing load_hermes_env() -> list[Path]. Preferred path: from hermes_cli.env_loader import load_hermes_dotenv and call it with defaults — keeps precedence and encoding fallbacks (latin-1 on UnicodeDecodeError) in exact lockstep with hermes-agent. Fallback path: direct python-dotenv against $HERMES_HOME/.env (or ~/.hermes/.env when unset) with override=True. Silent no-op in stripped-down containers that explicitly provide env via docker run -e …. Both entry points — plugin/relay/__main__.py and the legacy relay_server/__main__.py shim — call load_hermes_env() before importing .server, so anything in the module-import chain that reads os.getenv at module level sees the same environment regardless of launcher.
New: systemd user unit matching the gateway's shape
Rewrote relay_server/hermes-relay.service from scratch. The old template was a system-level unit hardcoded to /home/bailey/hermes-relay with /usr/bin/python3 — nobody was using it correctly. New template is a user unit with %h expansion so it's user-agnostic:
[Service]
Type=simple
ExecStart=%h/.hermes/hermes-agent/venv/bin/python -m plugin.relay --no-ssl --log-level INFO
WorkingDirectory=%h/.hermes/hermes-relay
Environment="PATH=%h/.hermes/hermes-agent/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
Environment="VIRTUAL_ENV=%h/.hermes/hermes-agent/venv"
Environment="HERMES_HOME=%h/.hermes"
Restart=on-failure
RestartSec=30
...
[Install]
WantedBy=default.target
No EnvironmentFile= on purpose — _env_bootstrap.py handles that. Matches the gateway unit's env directives (PATH, VIRTUAL_ENV, HERMES_HOME) + restart policy (StartLimitIntervalSec=600, StartLimitBurst=5) + log destination (StandardOutput=journal). User target so it lives in ~/.config/systemd/user/ and doesn't need sudo.
install.sh step [6/6] — optional systemd install
Added a final installer step that idempotently drops the unit into ~/.config/systemd/user/, runs daemon-reload, and enable --nows the service. Skipped gracefully on:
- Hosts without
systemctl(macOS, some BSDs) - Hosts where
systemctl --user show-environmentfails (bare chroots, WSL without systemd, containers without a user bus) $HERMES_RELAY_NO_SYSTEMDset (explicit opt-out for users who prefer their own process supervisor)
If an orphan nohup-launched python -m plugin.relay is already holding :8767, the installer warns the user to kill it first rather than racing. A parting hint about loginctl enable-linger $USER for users who want the relay to survive SSH logout.
After install, the update cycle is now a single command:
cd ~/.hermes/hermes-relay && git pull
systemctl --user restart hermes-relay
No pkill / nohup / disown dance, no manual env sourcing, no stale-key drift on restart.
Docs surface updated
docs/relay-server.md— new Quick Start withinstall.shas the recommended path, manual-run and Docker sections preserved, new.env auto-loadingsubsection explaining precedence + why there's noEnvironmentFile=.user-docs/reference/relay-server.md— mirror of the above in user-facing style, plus two new troubleshooting entries (voice 500 "no API key available" → check.env+ restart, and service stops on SSH logout →loginctl enable-linger).CLAUDE.mdServer Deployment section — relay is now listed ashermes-relay.service(user unit), the restart command issystemctl --user restart hermes-relay, the verification step iscat /proc/$PID/environviasystemctl show -p MainPID, and the old nohup instructions carry a history note pointing at this entry.DEVLOG.md— this entry.
Why this is the right shape for the general plugin path
The installer is the only user-facing install surface for hermes-relay and already owns plugin registration, skill discovery, and the hermes-pair shim. Adding systemd-user install to it means any Linux user who runs the one-liner gets the relay in the same canonical place as hermes-gateway (~/.config/systemd/user/) with the same management commands (systemctl --user …). Non-systemd hosts fall back to manual python -m plugin.relay --no-ssl and the Python-side env loader still does the right thing. Docker users mount ~/.hermes and the bootstrap finds .env at its canonical path inside the container. There's no setup branching that varies by user, distro, or launch method — the env load happens at import time, which is the one place every launch path passes through.
Fix scope: 3 Python files touched (2 entry points + 1 new helper), 1 systemd template rewritten, 1 installer stanza added, 4 docs updated. No breaking changes to existing callers — manual python -m plugin.relay still works and is actually more robust than before because it now auto-loads .env.
2026-04-12 — Voice mode: end-to-end voice conversation via relay TTS/STT endpoints
Shipped voice mode in a single session via a four-agent team in a shared worktree (feature/voice-mode). Plan came from the Obsidian vault at Hermes-Relay/Plans/Voice Mode.md — a 4-phase spec (V1 server endpoints → V2 app audio pipeline → V3 orb animation → V4 polish). All four phases landed in parallel; V2b UI waited on V2a's locked ViewModel contract, everything else ran concurrently. Net: ~2750 lines across 9 new files + 6 modified files, no upstream hermes-agent changes required.
Architecture
Voice lives entirely in the relay plugin, not upstream hermes-agent. The relay is editable-installed into the hermes-agent venv, so plugin/relay/voice.py imports tools.tts_tool.text_to_speech_tool and tools.transcription_tools.transcribe_audio directly from the server's configured providers. Both upstream functions are sync, so the aiohttp handlers wrap them in asyncio.to_thread(...) to avoid blocking the event loop. Config (provider, voice, model) is read internally by the tools from ~/.hermes/config.yaml — the relay doesn't pass provider arguments, which means Bailey can swap TTS providers (currently ElevenLabs) or STT providers (currently OpenAI whisper-1) on the server without touching the phone.
Three new routes on the relay alongside /media/*:
POST /voice/transcribe— multipart audio →{text, provider}POST /voice/synthesize—{text}JSON →audio/mpegfile responseGET /voice/config— provider availability + current settings
All three gated on the same bearer auth as /media/* (local helper _require_bearer_session, not the private server.py version, to avoid circular imports). Fourteen unit tests in plugin/tests/test_voice_routes.py, all passing on the server (68/68 across voice + existing media/sessions suites). Upstream tool imports are lazy inside each handler so voice.py can be imported on Windows (where the hermes-agent venv doesn't exist); tests inject fake modules into sys.modules at collect time.
Android audio pipeline
app/.../audio/VoiceRecorder.kt wraps MediaRecorder with MPEG_4/AAC output to .m4a (not WebM — the plan suggested WebM but m4a is Android-native and whisper-1 handles it fine via the upstream _validate_audio_file path). Amplitude is polled at ~60fps from mediaRecorder.maxAmplitude / 32767f and exposed as a StateFlow<Float> for the orb.
app/.../audio/VoicePlayer.kt wraps MediaPlayer + android.media.audiofx.Visualizer for real-time amplitude during TTS playback. The Visualizer construction is in a try/catch — some OEM devices refuse to construct one even with MODIFY_AUDIO_SETTINGS granted, and rather than crashing the voice session on those devices we fall back to a flat-zero amplitude and log. Voice still works; the sphere just won't pulse during Speaking on those devices.
app/.../network/RelayVoiceClient.kt mirrors RelayHttpClient's ctor shape — same okHttpClient / relayUrlProvider / sessionTokenProvider pattern, same error handling. Transcribe uses MultipartBody.Builder, synthesize uses JSON POST with the response bytes streamed to cacheDir/voice_tts_<ts>.mp3. Added a third method getVoiceConfig() for the Voice Settings screen to read provider availability.
app/.../viewmodel/VoiceViewModel.kt orchestrates the state machine: Idle → Listening → Transcribing → Thinking → Speaking → Idle. Sentence-boundary detection lives in a top-level internal fun extractNextSentence(StringBuilder) that drains complete sentences (whitespace-lookahead for e.g., min length 6 so tiny fragments don't get synthesized) into a Channel<String> consumed by a dedicated coroutine that synthesizes + plays + awaits completion per sentence. This gives streaming TTS without needing chunked-Opus support on the server — the latency win comes from client-side chunking of the SSE stream.
ChatViewModel integration — cleaner than the plan
The plan called for a // VOICE HOOK callback added to ChatViewModel so VoiceViewModel could observe streaming deltas. Instead, VoiceViewModel.startStreamObserver collects chatVm.messages: StateFlow<List<ChatMessage>> and diffs the last assistant message's content length on each emission. Zero changes to ChatViewModel / ChatHandler. The transcribed user text is routed through the normal chatVm.sendMessage(text) path, so voice utterances appear as regular user messages in chat history — the same message shows up if you reload the session on another device.
The trade-off is documented in a KDoc comment on the observer: this assumes "the last isStreaming=true message is the current turn," which is true today. If ChatViewModel ever streams multiple assistant messages concurrently (agent hand-off, multi-agent runs) this would need a dedicated per-turn flow. Flagged for Phase 3+ review.
MorphingSphere: Listening + Speaking states
Two new SphereState values added additively — all five existing call sites (RelayApp, BridgeScreen, ChatScreen ×3) compile unchanged because voiceAmplitude: Float = 0f and voiceMode: Boolean = false are defaulted. The plan spec had exact formulas for amplitude → visual parameter mapping (lines 338-367 of Voice Mode.md) and the sphere-animator teammate mapped them to the actual render variables:
- Listening — soft blue/purple (#597EF2 ↔ #A573F2, cooler than Idle's green/purple).
breatheSpeedlerps 1.0→1.3× at half amplitude,turbulenceAmpgets +0.15×amp, perimeter-noisewobbleAmplitudescales up 30% max. Subtle — the orb "breathes with the user." - Speaking — vivid green/teal (#40EB8C ↔ #4DD9E0) with
coreWarmth(a new synthetic Float, not a refactor — spliced into the existing warmth term at line ~400) lerping 0.3→1.0 on amplitude for a white-hot core at peaks.breatheSpeedup to 2×,turbulenceAmp+0.5×amp,wobbleAmplitude+80%,dataRingSpeedup to 4× on peak amplitude. Dramatic — the orb performs the voice.
Radius scale for voiceMode=true is capped at 1.08×, not 1.0× as the plan suggested. The existing baseRadius is already 0.60× half-extent and the data ring at 1.55× would overflow the canvas past 1.1×. Expansion is subtle by physical necessity. Three @Preview functions (MorphingSphereListeningPreview, MorphingSphereSpeakingLowPreview, MorphingSphereSpeakingPeakPreview) with deterministic frames so Bailey can scrub amplitude values in Studio without a device.
Voice mode UI
VoiceModeOverlay.kt is the full-screen experience: top bar with interaction-mode dropdown + close X, centered sphere at 60% height with voiceMode=true, transcribed + response text with AnimatedContent fades, mic button at bottom with pulseScale driven by amplitude. The mic button respects interactionMode:
- TapToTalk — click starts recording; auto-stops on silence (threshold from
VoicePreferences, default 3000ms); click again to stop manually; click during Speaking routes tointerruptSpeaking(). - HoldToTalk —
awaitEachGesture { awaitFirstDown() + waitForUpOrCancellation() }starts on press and stops on release. - Continuous — after TTS finishes speaking,
maybeAutoResume()auto-starts listening again. Current detection usesstreamObserverJob?.isActive != trueas the "turn done" signal becauseChannel.isEmptyisn't public API — may resume very slightly early on the last sentence before the final observer tick, but it's imperceptible in practice.
Haptics: HapticFeedbackType.LongPress on record start, HapticFeedbackType.TextHandleMove on record stop. Error banner with retry button handles mic permission denial, voice-config failure (shows "Unknown" provider rows), and transcribe/synthesize errors.
ChatScreen now hosts a mic FAB in the bottom-right corner (hidden while voice mode is active) that triggers the permission flow. Existing chat content fades to 40% alpha via animateFloatAsState when voice mode opens. VoiceSettingsScreen.kt is a new sub-screen off Settings with four sections: Voice Mode (interaction mode + silence threshold slider + auto-TTS placeholder), Text-to-Speech (read-only provider/voice labels), Speech-to-Text (read-only provider + language picker stored for future use), and a Test Voice button that calls the new VoiceViewModel.testVoice(sample) extension. VoicePreferences.kt is a new DataStore-backed repo mirroring MediaSettings.kt.
Things left as follow-ups
- Silence-detector wiring — V2a's recorder exposes the amplitude StateFlow, and
VoicePreferences.silenceThresholdMsis persisted, but the ViewModel doesn't yet collect recorder amplitude and auto-callstopListening()on silence. V2b flagged this as a boundary question (expose a preferences setter on VoiceViewModel vs. observe DataStore directly). Not a voice-mode blocker — tap-to-stop works — but the auto-stop UX promised in the plan isn't there yet. Small follow-up task. - Auto-resume precision — the
streamObserverJob.isActivesignal for continuous mode is slightly imprecise (see above). If Bailey reports stalled turns in continuous mode this is where to look. - Voice config write-through — reads
GET /voice/config, but the Voice Settings screen can't yet write changes back to~/.hermes/config.yaml. That would need a management API endpoint (Phase M territory per the plan). For now, Bailey edits the yaml and restarts hermes-gateway. - Dedicated OkHttpClient for voice —
voiceClienthas its own OkHttpClient instance (2-min read timeout) rather than sharingConnectionViewModel.relayOkHttp(which is private). Minor duplication; would clean up if we exposed that field. - ChatScreen box/column indent —
Box { Column {...} }with same-level indentation in ChatScreen is slightly ugly. Compiles fine; purely cosmetic.
Files
New (server): plugin/relay/voice.py, plugin/tests/test_voice_routes.py.
New (app): app/.../audio/VoiceRecorder.kt, app/.../audio/VoicePlayer.kt, app/.../network/RelayVoiceClient.kt, app/.../viewmodel/VoiceViewModel.kt, app/.../ui/components/VoiceModeOverlay.kt, app/.../ui/screens/VoiceSettingsScreen.kt, app/.../data/VoicePreferences.kt, app/src/test/.../viewmodel/SentenceExtractionTest.kt.
Modified: plugin/relay/server.py (+21 lines — imports, handler wiring, 3 route registrations), app/src/main/AndroidManifest.xml (RECORD_AUDIO + MODIFY_AUDIO_SETTINGS), app/.../ui/components/MorphingSphere.kt (+145 lines — Listening/Speaking states, voiceAmplitude modifier, coreWarmth spliced into existing warmth term, voiceMode scale, 3 preview functions), app/.../ui/RelayApp.kt (+49 lines — VoiceViewModel wiring + VoiceSettings nav route), app/.../ui/screens/ChatScreen.kt (+115 lines — mic FAB + permission flow + chat alpha + overlay mount), app/.../ui/screens/SettingsScreen.kt (+46 lines — nav entry).
Dev workflow note: the team actually worked
Four agents, shared worktree, 1 blocker (V2b waited on V2a's locked VoiceUiState contract). Agents reported via SendMessage → mailbox → delivered as team idle notifications. The lead (me) did: SSH recon (real upstream signatures from ~/.hermes/hermes-agent/tools/), plan review, task list with explicit blockedBy, parallel agent spawn with complete self-contained briefs (including full upstream signatures so no agent needed to re-SSH), integration spot-checks on diffs, then this docs pass. Server-voice teammate even SCP'd test files to the server, ran them via unittest, and cleaned up its stray files on the server side when done. Zero Kotlin gradle runs — every agent respected the "Bailey compiles in Studio" rule.
Next time: for a feature this size, the worktree + team approach is the right default. The alternative (single-session linear work) would have been 3–4× slower.
2026-04-11 — clearSession reconnect leak → self-inflicted rate limit
Bailey reported "unable to pair via QR code" after hitting Revoke on his own device in Paired Devices. Worked again after an app restart.
Root cause (from ~/hermes-relay.log)
20:38:29 DELETE /sessions/f75cfb14 → 200 (self-revoke)
20:38:29 GET /sessions → 401 (phone loadPairedDevices with dead token)
20:38:32 Client disconnected
[relay restart]
20:39:03 WebSocket from 172.16.24.13 → Auth failed: Invalid pairing code or session token
20:39:04 WebSocket from 172.16.24.13 → Auth failed
20:39:05 WebSocket from 172.16.24.13 → Auth failed
20:39:06 WebSocket from 172.16.24.13 → Auth failed
20:39:07 WebSocket from 172.16.24.13 → Auth failed
20:39:07 [WARNING] IP 172.16.24.13 blocked for 300s after 5 failed auth attempts
20:39:08+ GET /ws → 429 (blocked)
Five Auth failed in four seconds. The rate limiter did exactly what it was built to do.
The bug — three state machines disagree
AuthManager.clearSession()wipes the stored token, setsauthState = Unpaired, and regenerates a fresh local pairing code (_pairingCode.value = generatePairingCode()).ConnectionManager's internal reconnect loop (scheduleReconnect→doConnect) runs entirely inside the network module — it bypassesConnectionViewModel.connectRelayand itshasPairContextgate completely.shouldReconnectstaystrueuntil someone callsdisconnect().ConnectionViewModel.clearSession()calledauthManager.clearSession()and returned — never calleddisconnectRelay(). So the reconnect scheduler kept firing after the state wipe.
Sequence of collapse:
- Self-revoke succeeds (server deletes session)
- Phone's Paired Devices UI calls
clearSession()→ state wiped, reconnect loop still alive - WS disconnects (because the relay closes the socket after revoke / the relay restart fires)
onClosed→scheduleReconnect→ backoff →doConnectwith freshly-regenerated local pair code in the auth envelope- Server:
"Invalid pairing code or session token"(the local code isn't registered) - Loop fires 5 times in ~4 seconds (exponential backoff starts at 1s)
- Rate limiter blocks the IP for 5 minutes
- User scans a fresh QR →
/pairing/registerclears the block — but the phone's next WS connect bounces off whatever delayed retry is still in flight (and also possibly a 429 from the still-valid block depending on exact timing) - User restarts app → cold start has
shouldReconnect = falseand empty session store, no auto-connect, fresh QR scan succeeds
Fix
network/ConnectionManager.kt:
- New constructor parameter
reconnectGate: () -> Boolean = { true }(defense-in-depth gate for the internal auto-reconnect loop). scheduleReconnect()callsreconnectGate()both before scheduling and after the backoff delay. If either returnsfalse, the retry is silently dropped andconnectionStateis set toDisconnected. Rationale: the delay window is where state flips happen — user hits Revoke while the previous attempt was sleeping, and we need to re-check.- Logs
"scheduleReconnect: gate says no pair context — aborting retry"at INFO for traceability. - Default value preserves backwards compat with tests that construct a bare
ConnectionManager(multiplexer).
viewmodel/ConnectionViewModel.kt:
ConnectionManagernow constructed withreconnectGate = { authManager.hasPairContext }— samehasPairContextpredicate introduced for the Option B Save & Test gate.clearSession()rewritten to calldisconnectRelay()beforeauthManager.clearSession(). Order matters: tear down the reconnect loop before wiping the state it depends on, so in-flight retries return cleanly instead of firing with half-wiped state. KDoc explains the 2026-04-11 rate-limit incident as the why.
Both fixes together: clearSession stops the immediate loop, and the reconnectGate is a safety net for any other code path that wipes state without calling disconnect. Either alone would technically fix the reported bug; together they harden against future regressions.
Test plan (on-device)
- Go to Settings → Paired Devices → open the current device's card → Revoke → confirm. Snackbar shows "This device was unpaired". App navigates to pair screen.
- Immediately scan a fresh QR (from
/hermes-relay-pairon the server). Should succeed — no "unable to pair" error, no 429 on the relay's/ws. - Check
~/hermes-relay.logafter revoke: should see theDELETE /sessions/...line but NOT a burst ofAuth failedentries. Phone should stay quiet until the user scans the new QR. - Bonus: toggle airplane mode while paired (forces disconnect with pair context still valid), confirm reconnect still works normally after re-enabling network.
Server deploy
Phone-only change. No server restart needed. Bailey rebuilds from Android Studio.
2026-04-11 — Paired Devices JSON unwrap fix + /media/by-path permissive sandbox (B+C)
Two bugs surfaced on-device:
1. Paired Devices crash — "Expected start of the array '[', but had '{'"
RelayHttpClient.listSessions was parsing the response body as a bare List<PairedDeviceInfo>, but the server returns {"sessions": [...]} (plugin/relay/server.py::handle_sessions_list, line 406). Classic wire-format mismatch. The phone saw JSON starting with { and crashed kotlinx.serialization's list deserializer.
Fix: parse the body to JsonElement, pull out the sessions field, then decode the array via Json.decodeFromJsonElement(ListSerializer(PairedDeviceInfo.serializer()), arrayElement). If the sessions field is missing, return a descriptive IOException instead of the kotlinx parse error so the UI shows a meaningful message.
One concept to internalize: Json.decodeFromJsonElement(deserializer, element) is a member function on the Json instance — no extension import needed. The reified-generic form (Json.decodeFromJsonElement<T>(element)) needs the import kotlinx.serialization.json.decodeFromJsonElement extension import; the explicit-deserializer form doesn't.
2. "Path not allowed by relay sandbox" for legitimate LLM emits
The /media/by-path route enforced allowed_roots against every phone-side fetch. When the LLM ran search_files and found an image somewhere like ~/projects/claw3d/readme.png, the phone hit the sandbox and rendered a "Path not allowed" error card — even though the agent had already read the file's bytes via its own tools and would happily paste them into chat if asked.
Decision: B + C (C as default).
- C (conceptual flip): drop the allowlist on
/media/by-pathby default. The trust boundary is the bearer-auth'd paired phone; the LLM can already exfiltrate bytes via plain text responses, so the sandbox was defense-in-depth with a high false-positive rate in practice. The token path (loopback-only/media/register) keeps its strict allowlist because it's trivially enforceable there. - B (config knob):
RELAY_MEDIA_STRICT_SANDBOX=1(orRelayConfig.media_strict_sandbox = True) re-enables the allowlist enforcement on the by-path route for operators who want the tighter default back.
Changes
plugin/relay/media.py:
validate_media_path(path, allowed_roots, max_size_bytes)—allowed_rootsnow acceptslist[str] | None. When None, the root-under-allowlist check is skipped entirely. All other checks (absolute path, realpath, exists, regular file, size cap) remain unconditional.MediaRegistry.__init__gainsstrict_sandbox: bool = Falseparameter, stored asself.strict_sandbox. Logged at init time alongside the existing max_entries/ttl/max_size summary.
plugin/relay/config.py:
RelayConfig.media_strict_sandbox: bool = Falsefield.RELAY_MEDIA_STRICT_SANDBOXenv var parsed infrom_env()— accepts1/true/yes/on(case-insensitive).
plugin/relay/server.py:
RelayServer.__init__passesstrict_sandbox=config.media_strict_sandboxto theMediaRegistry.handle_media_by_pathcomputesroots_for_check = server.media.allowed_roots if server.media.strict_sandbox else Noneand passes that tovalidate_media_path. Permissive mode ⇒ only file-level checks (absolute, exists, regular, size). Strict mode ⇒ full allowlist enforcement.- Docstring on
handle_media_by_pathrewritten to explain the permissive-by-default model and the opt-in path.
app/src/main/kotlin/.../network/RelayHttpClient.kt:
listSessions()— unwraps{"sessions": [...]}before decoding the array. Missing-field path surfaces as a cleanIOException("Relay response missing 'sessions' field").
plugin/tests/test_relay_media_routes.py:
- Existing
RelayMediaRoutesTestsclass pinnedconfig.media_strict_sandbox = Trueso its legacy assertions (outside-sandbox → 403, etc.) still hold. - New sibling class
RelayMediaByPathPermissiveTestsexercises the production default:test_by_path_outside_allowlist_still_streams_in_permissive_mode— the regression-guard test. A file in a totally unrelated tmpdir streams successfully with a valid bearer. If this breaks, Bailey's claw3d workflow breaks again.test_by_path_still_rejects_relative_in_permissive_mode— absolute-path check is unconditional.test_by_path_still_404s_nonexistent_in_permissive_mode— file-existence check is unconditional.test_register_still_enforces_allowlist_in_permissive_mode— the token path (loopbackPOST /media/register) stays strict regardless of the flag.
CLAUDE.md:
- Updated the
plugin/relay/media.pyand "Inbound media fetch (path)" rows in the Key Files / Integration Points tables to document the permissive default and the opt-inRELAY_MEDIA_STRICT_SANDBOXknob.
Server deploy
Python-side changes. Bailey needs to git pull on the server and restart the relay (pkill -TERM -f "python -m plugin.relay" + nohup ... & disown per the standard recipe) before the new by-path semantics take effect. Phone-side listSessions fix ships with the next Android Studio build.
Test plan (on-device after next build)
- Paired Devices screen: open → loads without the "Couldn't load paired devices" error card. Should show the current device with transport badge, expiry, and grant chips.
- Chat with
find an image: LLM finds something in~/projects/**→ card renders inline (not "Path not allowed by relay sandbox"). - Optional:
RELAY_MEDIA_STRICT_SANDBOX=1on the server → same image request now renders the "Path not allowed" card (strict-mode opt-in works).
2026-04-11 — Option B: Save & Test as HTTP health probe + pair-context gate on connectRelay
Bailey flagged a trap in the Settings → Manual configuration flow: the Connect button fired connectRelay(url) unconditionally, even when the phone had no session token and no pending server-issued pair code. The WSS handshake would open, the phone would send an auth envelope with whatever code was lying around (usually the locally-generated phone→host Phase 3 code, which isn't registered on the relay), auth would fail, and after 5 such failures in 60 seconds the relay's rate limiter would block the IP for 5 minutes. Users fumbling with manual config could lock themselves out of pairing entirely.
Reviewed three fix options; picked B (split reachability from connect). Rejected C (two buttons — "Test" + "Connect") because the second button is redundant with the existing Reconnect button / auto-reconnect paths and would be disabled-when-unpaired / duplicated-when-paired. The right answer is one button that does a reachability probe, not two buttons fighting over WSS semantics.
Changes
Server: none. This is entirely phone-side.
plugin/relay unchanged except indirectly — the new phone-side probeHealth method hits the relay's existing GET /health endpoint (already implemented at plugin/relay/server.py::handle_health).
auth/AuthManager.kt — new hasPairContext: Boolean getter. Returns true when any of:
authStateisPaired(stored session token)authStateisPairing(mid-handshake)serverIssuedCode != null(fresh QR scan or manual code entry just landed)
Crucially does NOT count the locally-generated _pairingCode.value as valid — that's the phone→host Phase 3 code and is meaningless to the relay side, so sending it yields guaranteed rate-limit hits. This is the fix for the entire class of "connect when unpaired" bugs.
network/RelayHttpClient.kt — new probeHealth(relayUrl): Result<RelayHealth> method:
- Converts
ws:///wss://→http:///https://via the same regex used byfetchMedia - Unauthenticated
GET /healthwith a 3-second timeout (viaokHttpClient.newBuilder().connectTimeout(3, SECONDS).readTimeout(3, SECONDS).build()— no need for a separate long-lived client) - Parses the JSON body and validates it looks like a hermes-relay health response:
status == "ok"AND a non-blankversionfield. Anything else fails with a descriptive error. Prevents a random HTTP server on port 8767 from falsely passing. - Returns
RelayHealth(version, clients, sessions)on success so the UI can render"✓ Reachable — hermes-relay v0.2.0 (0 clients, 1 session)"— actual metadata, not just a boolean. - Error messages differentiate:
"Connection refused — is the relay running on this URL?","Relay is not responding (3s timeout)","Relay responded HTTP 404","Relay returned non-JSON: ...", etc.
viewmodel/ConnectionViewModel.kt:
- Rewrote
connectRelay(url)/connectRelay()to delegate to a privateconnectRelayInternalthat gates onauthManager.hasPairContext. When the gate fails, log an informational message and return silently — the UI doesn't get any error state because this path should only be hit as a user-initiated mistake, not a programmatic one. The four legitimate entry points (pair walkthrough, stale row tap, Reconnect button, QR confirm) all set pair context before calling connectRelay, so the gate passes. - New sealed interface
RelayReachable { Probing, Ok(version, clients, sessions), Fail(message) }+relayReachableResult: StateFlow<RelayReachable?>+testRelayReachable(url)method that saves the URL to DataStore and probes/health. The save-before-probe order is deliberate: a failed probe still leaves the URL persisted so the user can edit and retry without losing their typing. clearRelayReachableResult()— called by the Relay URL text field'sonValueChangeso stale probe results vanish when the user edits the URL.- Deleted the old callback-based
testRelayReachability(wsUrl, onResult: (Boolean) -> Unit)— it was a thinner version ofprobeHealththat spun up its own OkHttpClient per call and returned only a boolean. Single caller in SettingsScreen migrated to the new state-flow API. - Removed the now-unused
okhttp3.Requestimport (was only referenced by the deleted method).
ui/screens/SettingsScreen.kt — Manual configuration button row rewritten:
- Connect button deleted. It was the leak source. Reconnecting a paired session is handled by the existing Reconnect button (in the Connection card, gated on
Paired), the stale row tap, andreconnectIfStale()on screen entry. No fourth "Connect" button is needed. - "Test" button renamed to "Save & Test" — wired to
testRelayReachable(relayUrlInput). Label change matches the new behavior (the button also persists the URL to DataStore). Disabled when the URL is blank or a probe is in flight. - Disconnect button kept — still useful for debugging / forcing reconnect cycles.
- Result row rewritten to read from
relayReachableResult: StateFlowinstead of the old localrelayTestInProgress/relayTestResultvars (both removed). Shows:Probing /health…with spinner during the probe✓ Reachable — hermes-relay v0.2.0 (0 clients, 1 session)(green) on success, with version + client count so the user knows they're pointing at an actual relay✗ <specific error>(red) on any failure- Nothing when idle (so the row collapses cleanly when not in use)
- Relay URL text field's
onValueChangenow clears the probe result — stale "✓ Reachable" indicators won't hang around after the user starts typing a different URL.
Bonus fix — Settings QR confirm never connected WSS
While tracing the pair-context gate I noticed the Settings → Scan Pairing QR → TTL picker confirm flow stashed the server-issued code + grants + TTL but never actually called connectRelay. The WSS stayed disconnected until the user either left and re-entered Settings (firing reconnectIfStale, which still wouldn't fire since authState is still Unpaired) or manually tapped Reconnect. This was a pre-existing bug dating from the pairing+security architecture cycle — the walkthrough dialog path (which has its own inline connectRelay call) masked it.
Added the missing disconnectRelay() + connectRelay(relay.url) pair to the TTL picker's onConfirm callback, mirroring the manual code pair path at submitPairing / authManager.applyServerIssuedCodeAndReset / connectRelay(...). Since applyServerIssuedCodeAndReset has just set serverIssuedCode, the new pair-context gate passes cleanly.
Bonus fix — Connection status rows didn't look tappable
Second Bailey UX feedback from the same cycle: the API Server / Relay / Session status rows in the Connection section open drawer sheets on tap, but there was no visual affordance that they were tappable — users would stumble onto the tap target by accident.
Fixed in ConnectionStatusRow:
- New
onClick: (() -> Unit)? = nullparameter. When non-null, the component applies a Material ripple + rounded-corner clip + internalclickablemodifier, so the whole row is a proper list-item tap target. - Trailing chevron icon (
Icons.AutoMirrored.Filled.KeyboardArrowRight) rendered after the status text wheneveronClick != null(and noonTestbutton is competing for the trailing slot). Gives users the standard Settings-list-item visual cue that the row opens something.
All three call sites in SettingsScreen migrated from modifier = Modifier.fillMaxWidth().clickable { ... } to onClick = { ... } + modifier = Modifier.fillMaxWidth(). Old pattern still works for legacy call sites (the component applies the interactive wrapping additively).
Files changed
Server: none.
Phone:
app/src/main/kotlin/com/hermesandroid/relay/auth/AuthManager.kt—hasPairContextgetterapp/src/main/kotlin/com/hermesandroid/relay/network/RelayHttpClient.kt—probeHealth()+RelayHealthdata class +jsonObjectimportapp/src/main/kotlin/com/hermesandroid/relay/viewmodel/ConnectionViewModel.kt—RelayReachablesealed interface,relayReachableResultflow,testRelayReachable(),clearRelayReachableResult(),connectRelayInternal()with pair-context gate, deletedtestRelayReachabilityapp/src/main/kotlin/com/hermesandroid/relay/ui/screens/SettingsScreen.kt— Connect button removed, Save & Test rewired, result-row reads from ViewModel flow, onValueChange clears probe, QR confirm flow adds missingconnectRelay, three status rows migrated toonClickparameterapp/src/main/kotlin/com/hermesandroid/relay/ui/components/ConnectionStatusBadge.kt—ConnectionStatusRowgainsonClick+ chevron affordance
Docs: this DEVLOG entry.
Test plan
On-device, after Bailey's next Android Studio build:
- Fresh Manual Config reachability probe — Settings → Connection → Manual configuration → type a relay URL → tap Save & Test. Verify:
- Spinner + "Probing /health…" appears briefly
- On a live relay: green "✓ Reachable — hermes-relay vX.Y.Z (N client, M session)"
- On a dead URL: red "✗ " (refused / timeout / non-JSON / etc)
- Editing the URL clears the result immediately
- Pair-context gate doesn't regress pairing — scan a fresh
/hermes-relay-pairQR → TTL picker confirms → verify the WSS connects +authStatereachesPairedwithout any manual intervention - Manual code entry still works — Settings → Already configured? Enter code only → type a valid code → verify it pairs and WSS connects
- Drawer affordance visible — open Settings → Connection section → verify API Server / Relay / Session rows show a chevron at the right edge + have a ripple on tap → tapping each opens its respective info sheet
- Flicker still fixed — open Settings with a stale session → verify the row shows "Reconnecting..." for the whole transition, not the old "Stale → Connecting → Connected" flash
Known caveats
- The Connect button removal is a UX change for power users who were using it to force-reconnect. They should use the Reconnect button in the Connection card (gated on Paired) or the stale row tap. If this breaks a workflow Bailey relies on, the button can come back gated on
authManager.hasPairContext. - The new
probeHealthgives 3 seconds before timing out. On a very slow LAN this might be too tight; consider bumping to 5s if probes start false-failing. probeHealthdoesn't follow redirects (OkHttp default). A relay behind a reverse proxy that 301s/healthwould fail the probe. None of the current deployments do that.
2026-04-11 — Relay status flicker fix + Dev Workflow docs in CLAUDE.md
Two small follow-ups from the same session:
1. Connection card flicker on Settings entry. When opening Settings with a stored session token, the Relay status row flashed through 3 rapid states: red "Disconnected" → amber "Stale — tap to reconnect" → amber "Connecting..." → green "Connected". Users saw the middle two as a flicker.
Root cause: authState default is Unpaired until the EncryptedSharedPreferences read finishes (~50ms async). During that gap the row shows red "Disconnected" which is actually correct. Once authState flips to Paired, isRelayStale becomes true and the row shows "Stale — tap to reconnect". The LaunchedEffect(Unit) that fires reconnectIfStale() then triggers ConnectionState.Connecting, changing the label to "Connecting...". These rapid transitions are the flicker.
Fix: added a screen-local isAutoReconnecting flag that's true for up to 5 seconds on screen entry (or until ConnectionState.Connected lands, whichever is first). During that window the status text unifies all the non-Connected sub-states into one consistent "Reconnecting..." label. After 5s the flag drops and the row falls through to the normal state machine — so the "Stale — tap to reconnect" affordance still works for cases where the user is genuinely sitting in a stale state (backgrounding the app, network flapping, etc.). Only the initial-entry transition gets the unified label.
The fix is purely presentational — the underlying state machine is unchanged. isConnecting is wired to treat isAutoReconnecting the same as the real Connecting / Reconnecting states so the ConnectionStatusBadge's pulse ring animates continuously through the window.
2. Dev workflow documented in CLAUDE.md.
Bailey asked for the typical dev loop to be documented in CLAUDE.md so Claude doesn't have to rediscover the flow each session. Added a new "Typical Dev Loop" subsection + "Server Deployment" table + "Where Python vs. Kotlin changes land" table. Covers:
- Edit-in-Windows, test-on-Linux-server split
- Python plugin edits via
pip install -eeditable path →git pullon server picks them up hermes-gateway.servicesystemctl user-unit restart when tool code changes- Manual
nohup/setsidrestart for the relay process (which isn't a systemd service on this deployment) - Running tests via
python -m unittest plugin.tests.test_<name>(avoiding the pre-existingconftest.pythat imports the uninstalledresponsesmodule) - The hard convention: Claude never builds/installs APKs — Bailey uses Android Studio's run button
- Where sensitive info lives (
~/SYSTEM.mdon the server,~/.hermes/.env) — explicitly NOT in this repo - The PATH conventions (venv, plugin symlink, relay log, config yaml, qr-secret)
Keeps the doc free of any host-specific identifiers (IP, username, SSH key) — those stay on the server side.
Also included in this entry but landed in a separate earlier commit: fix(auth): extend without grants regenerates from defaults, not absolute-time clamp (c209c99). The first cut of SessionManager.update_session tried to preserve existing grants via a _clamp_grants_to_lifetime helper when only TTL changed. That made sense for shortening (clip grants that exceed the new session) but produced nonsense for extending: a 1h bridge grant made 30 minutes ago, extended to 90d, would still have an absolute expiry of 30 minutes from now — the user would tap Extend and find bridge was about to expire anyway. Corrected to regenerate grants from _default_grants(new_ttl, now) when only TTL changes, giving fresh allocations for the new lifetime. Deleted the now-unused _clamp_grants_to_lifetime helper. Caught by test_extend_ttl_zero_means_never which asserts that extending a finite session to never-expire produces all-null (never) grants.
Files changed:
app/src/main/kotlin/com/hermesandroid/relay/ui/screens/SettingsScreen.kt—isAutoReconnectingflag + updatedConnectionStatusRowwiringCLAUDE.md— new Dev Workflow subsections (Typical Dev Loop, Server Deployment, Where Python vs. Kotlin changes land)plugin/relay/auth.py— (separate commitc209c99)update_sessionsemantic fix
No server side changes in this entry — the flicker fix is phone-only and the docs don't deploy.
2026-04-11 — Grant renewal action (PATCH /sessions/{prefix} + "Extend" button)
Why this exists: Pairing/security overhaul shipped a Paired Devices screen with list + revoke, but no way to change a session's TTL or grants after initial pair. Users who paired with "1 day" when trying out the feature and decided to keep it had to revoke + re-pair. This closes that gap with the smallest possible surface area: one new route, one new client method, one new button, one new dialog reuse. Scope #4 from the gap list — small and concrete enough to do directly rather than via agent team.
Semantics decision: TTL restarts the clock. "Extend by 30 days" = "30 days from now", not "add 30 days to the existing expiry." This matches what users mean when they tap Extend: they want the session to be valid for another 30 days starting today. It also means Extend can SHORTEN a session (pick a shorter duration or "Never" → fewer) — the button is labeled "Extend" for the common case but the semantics are really "update TTL policy".
Grant handling:
- If caller passes
grants: re-materialize from scratch via_materialize_grants(uses defaults for channels the caller didn't specify, clamps each to the new session lifetime). - If caller passes ONLY
ttl_seconds: keep existing grants but re-clamp them via a new_clamp_grants_to_lifetimehelper so a shorter TTL correctly clips any grant that would outlive the session. A longer TTL leaves grants unchanged (they were already ≤ the old expiry, which is now ≤ the new longer expiry, so no clipping needed). - If caller passes both: apply TTL first (new session expiry), then materialize the provided grants clamped to the new expiry.
- If caller passes neither: 400. No-op calls are bugs.
UX:
Reuses the existing SessionTtlPickerDialog directly — no new component needed. The dialog's "Keep this pairing for…" title works for both the pair flow and the extend flow (tense-neutral). Preselected option is computed from the session's current remaining lifetime rounded to the nearest picker option — never-expire sessions preselect "Never", 1-hour-remaining sessions preselect "1 day" (shortest real option), 25-day-remaining sessions preselect "30 days", etc.
Files
Server (plugin/relay/):
auth.py— newSessionManager.update_session(token, ttl_seconds?, grants?) -> Session | Nonethat restarts-the-clock on TTL and re-materializes/re-clamps grants. New module-level_clamp_grants_to_lifetime(grants, ttl_seconds, now)helper alongside the existing_materialize_grants. Expired sessions are NOT silently resurrected — caller should re-pair instead.server.py— newhandle_sessions_extend(request)handler doing full validation (non-negativettl_seconds, non-negative numeric grant values, at least one field present) + the prefix → session → update dance that mirrorshandle_sessions_revoke. Route registered viaapp.router.add_patch("/sessions/{token_prefix}", handle_sessions_extend)— PATCH is a distinct method on the same pattern, no ordering collision with existing GET/DELETE.plugin/tests/test_sessions_routes.py— 10 new tests: bearer-required, nonexistent prefix → 404, empty body → 400, negative TTL → 400, bad grant shape → 400, TTL-only restarts the clock (assertsnew_expiry ≈ now + new_ttl, NOT old_expiry + new_ttl),ttl_seconds=0→ never (null grants), grants-only leaves expiry alone, shorter TTL clips grants, self-extend, helper_settle()for the 1ms async delay needed on fast machines to make timestamp assertions meaningful.
Phone (app/src/main/kotlin/.../):
network/RelayHttpClient.kt— newextendSession(tokenPrefix, ttlSeconds?, grants?)method. Hand-rolls the small JSON body to avoid pulling in another serializer branch (two optional fields, trivial to write, auditable). 400/401/404/409/5xx are all mapped to user-facing error messages. 404 is a hard failure here (unlike revoke where "already gone" → success) — if you're extending an active session and it's gone, that's surprising and should surface.viewmodel/ConnectionViewModel.kt— newsuspend fun extendDevice(tokenPrefix, ttlSeconds): Boolean, mirrorsrevokeDevice's shape (refresh list on success, surface error viapairedDevicesError). Grants intentionally not exposed on this path — MVP UX is "pick a new duration", server-side re-clamping handles the rest. Power users can callRelayHttpClient.extendSessiondirectly for grant editing.ui/screens/PairedDevicesScreen.kt—pendingExtend: PairedDeviceInfo?state alongside the existingpendingRevoke. New dialog branch rendersSessionTtlPickerDialogwith the current remaining lifetime preselected (expires_at - now, clamped to ≥ 0, or 0 for never-expire sessions). Confirming callsconnectionViewModel.extendDevice(...)inside a coroutine + shows a snackbar.DeviceCardaction row becomes a 50/50 split between "Extend" and "Revoke" buttons (instead of a single full-width revoke). Both buttons areOutlinedButton; the revoke button keeps the error color tint.
Docs:
docs/relay-server.md+user-docs/reference/relay-server.md— new PATCH /sessions/{token_prefix} route rowuser-docs/reference/configuration.md— Extend button description in the Paired Devices subsectionCLAUDE.md— Integration Points table row for the new route- this DEVLOG entry
Not changed (intentionally)
__version__stays at 0.2.0 per prior direction (we never released 0.2.0)- No grant-editing UI — power users can hit the endpoint directly via a future tool
- No "extend by increment" UX (e.g. "+30 days on top of current") — semantics are always "set new TTL from now". Easier to reason about, matches what the picker already shows.
- No ADR bump — this is a small follow-up to ADR 15's architecture, not a new decision. ADR 15 already documents the session mutation model at a high level.
Test plan (on-device)
- Pair with
--ttl 1dso you have a short session - Open Paired Devices → find the current device → tap Extend
- Picker should show "1 day" preselected. Pick "30 days" → confirm.
- Verify the card now shows "Expires ~30 days from now" and the snackbar says "Session expiry updated"
- Tap Extend again → pick "Never expire" → confirm. Verify the card now shows "Never" and all channel grant chips show "never" too
- Tap Extend again → pick "1 day" → confirm. Verify the card now shows ~1 day from now AND the grant chips are now all ≤ 1 day (clamped by the shortened session)
- Try to Extend a non-current session (if you have multiple paired) → should work the same way
2026-04-11 — Pairing + Security Architecture Overhaul (grants, TTL, Keystore, TOFU, devices)
Why this exists:
The existing pairing model has been minimal since day one: pairing codes (one-shot, 10 min TTL) → session tokens (30 days hardcoded, single expiry, no per-channel grants, stored in EncryptedSharedPreferences). After the inbound media work exposed the "paired phone gets rate-limited to death on relay restart" gap, Bailey flagged the entire pairing story as ready for a pass. Explicit asks:
- Secure transport the default, insecure opt-in with clear UI
- User-chosen TTL at pair time (1d/7d/30d/90d/1y/never)
- Separate defaults by channel (terminal + bridge shorter than chat)
- Never-expire ALWAYS selectable — "don't force check, just allow based on user intent" (per Bailey)
- Tailscale detection — informational only, not opinionated about defaults
- Hardware-backed token storage
- TOFU cert pinning with explicit reset on re-pair
- Device revocation UI (Paired Devices screen)
- QR payload signing
- Phase 3 bidirectional pairing foundation
Delivered by two parallel background agents (server + phone) + local docs pass. No upstream hermes-agent changes — everything lives in files we own.
Server side — plugin/relay/
Data model (auth.py):
Sessiongainsgrants: dict[str, float](per-channel expiry timestamps),transport_hint: str("wss"/"ws"/"unknown"),first_seen: float, and handlesmath.inffor never-expire (is_expiredis False for inf; JSON serializes inf →null).- New
PairingMetadatadataclass carriesttl_seconds,grants, andtransport_hintthrough the pairing flow._PairingEntry.metadatastores it;PairingManager.register_code(..., ttl_seconds, grants, transport_hint)accepts it;consume_codereturns the metadata dataclass (orNone) instead of a bool — existing boolean callers useis not None. SessionManager.create_sessionnow takesttl_seconds,grants,transport_hintparams (all with backwards-compatible defaults)._materialize_grantshelper resolves seconds-from-now durations to absolute expiries and clamps each grant to the overall session lifetime — no terminal grant can outlive its session. Default caps: terminal 30 days, bridge 7 days, chat = session lifetime. Constants:DEFAULT_TERMINAL_CAP,DEFAULT_BRIDGE_CAP.SessionManager.list_sessions,SessionManager.find_by_prefix— used by the new routes below.RateLimiter.clear_all_blocks()— new method that resets_blockedand_failuresdicts. Called unconditionally when/pairing/registersucceeds, because the operator is explicitly re-pairing and stale rate-limit state is noise. This fixes the "phone rate-limited for 5 minutes after relay restart" bug that was biting Bailey right when this session started.
Routes (server.py):
handle_pairing_register— now accepts optionalttl_seconds/grants/transport_hintin the JSON body and forwards them to the_PairingEntry.metadata. Host-registered metadata takes precedence over phone-sent metadata (operator policy is authoritative). Callsserver.rate_limiter.clear_all_blocks()on success.handle_pairing_approve— newPOST /pairing/approve, Phase 3 bidirectional pairing stub. Loopback-only, same shape as/pairing/register. Marked with# TODO(Phase 3):— full flow needs a pending-codes store so operators review rather than rubber-stamp. Route + wire shape locked in now so the Android agent has something to target.handle_sessions_list— newGET /sessions, bearer-auth'd. Returns{"sessions": [...]}where each entry carriestoken_prefix(first 8 chars, never the full token),device_name,device_id,created_at,last_seen,expires_at(null for never),grants,transport_hint,is_current. Full tokens are NEVER included — only the prefix — so a caller can't extract another session's credential.handle_sessions_revoke— newDELETE /sessions/{token_prefix}, bearer-auth'd. Matches on first-N-char prefix (≥ 4 chars). Returns 200 on exact match, 404 on zero matches, 409 on ambiguous (2+) matches with the count in the body. Self-revoke is allowed and flagged viarevoked_self: trueso the phone knows to wipe local state._authenticate— extended to thread pairing metadata into the new session + includeexpires_at,grants,transport_hintin theauth.okpayload.math.inf→nullon the wire._detect_transport_hinthelper — sniffsrequest.transport.get_extra_info('ssl_object')(non-None →"wss"), falls back torequest.scheme, defaults to"unknown". Runs only when pairing metadata didn't already supply a hint.
QR signing (new qr_sign.py):
load_or_create_secret(path)— reads~/.hermes/hermes-relay-qr-secretif present (32 bytes,0o600, owner-only). On first run generates + writes viasecrets.token_bytes(32)withos.umasksafety.canonicalize(payload)— JSON-encode withsort_keys=True, separators=(",", ":"),allow_nan=False(so accidentally signingmath.infcrashes loudly). Explicitly strips thesigfield before canonicalization so signing a signed payload is idempotent.sign_payload(payload, secret) -> str— base64 HMAC-SHA256.verify_payload(payload, sig, secret) -> bool— constant-time compare viahmac.compare_digest.- 13 test assertions covering canonicalization order-independence,
sigexclusion, round-trip, tamper + wrong-secret + malformed-sig rejection, file perm check.
Pair CLI (plugin/pair.py + plugin/cli.py):
- New flags:
--ttl DURATION(parses1d/7d/30d/1y/never) and--grants SPEC(terminal=7d,bridge=1d). build_payload(..., sign=True)— auto-bumpshermes: 1 → 2when any v2 field is present, embedsttl_seconds/grants/transport_hintin therelayblock, computes and attaches the HMAC signature.read_relay_confignow readsRELAY_SSL_CERTto determine TLS status;_relay_lan_base_url(tls=True)emitswss://when set.register_relay_codesends the new fields so/pairing/register's metadata attaches to the code.render_text_blockshowsPair: for 30 days/Pair: indefinitely+ per-channel grant labels.
Tests added (all unittest.IsolatedAsyncioTestCase / AioHTTPTestCase, runs via python -m unittest):
test_qr_sign.py— 13 assertions covering canonicalize / sign / verify / load_or_create_secret.test_session_grants.py— default TTL + grant caps, 1-day session clamping terminal/bridge, never-expire (math.inf everywhere), explicit grants clamped to session,grant=0semantics, transport_hint, unknown-channel → expired, pairing metadata register/consume/one-shot/format-reject, list_sessions / find_by_prefix / revoke.test_sessions_routes.py— 401 on missing + invalid bearer, GET shape (no full-token leak, 8-char prefix,is_currentcorrect, null for never-expire), DELETE 404 / 200 / 409 (ambiguous) / self-revoke.test_rate_limit_clear.py— unitclear_all_blocks+ integration/pairing/registerclears pre-existing blocks + metadata round-trip + invalid-ttl / bad-transport rejection +/pairing/approveloopback gate.
Phone side — app/src/main/kotlin/.../
Token storage + TOFU (new auth/SessionTokenStore.kt, auth/CertPinStore.kt):
SessionTokenStoreinterface with two implementations:KeystoreTokenStore— StrongBox-preferred viasetRequestStrongBoxBacked(true)whenFEATURE_STRONGBOX_KEYSTOREis present (Android 9+). Best-effort:tryCreateswallows exceptions so broken OEM keystores don't brick pairing; falls back to the legacy store.LegacyEncryptedPrefsTokenStore— existingEncryptedSharedPreferencespath, TEE-backed viaMasterKey.AES256_GCM.
- One-shot migration: on first launch post-upgrade, reads
session_token/device_id/api_server_key/ paired-metadata blob from the legacyhermes_companion_authfile, writes them to the new store, clears the legacy. If Keystore is unavailable, legacy stays as the active store and no migration happens — users never lose their session. hasHardwareBackedStorage: Booleanflag — true only for the StrongBox path. Legacy TEE-backed reports false. Surfaced in the Session info sheet as "Hardware (StrongBox)" vs "Hardware (TEE)".CertPinStore— DataStore-backed map ofhost:port→ SHA-256 SPKI fingerprints.recordPinIfAbsenton first successful wss connect;buildPinnerSnapshotproduces an OkHttpCertificatePinner.ConnectionManagertakes a fresh snapshot on everydoConnect, so aremovePinFor(host)during re-pair is honored on the next connect — no coordination needed betweenAuthManagerandConnectionManager. Plaintextws://short-circuits viaisPinnableUrl.applyServerIssuedCodeAndReset(code, relayUrl?)now wipes the TOFU pin for the target host — a QR re-pair is explicit consent to a possibly-new cert fingerprint.
Auth + session model (auth/AuthManager.kt, new auth/PairedSession.kt):
PairedSessiondata class:token, deviceName, expiresAt: Long?, grants: Map<String, Long?>, transportHint, firstSeen, hasHardwareStorage.PairedDeviceInfowire model mirrors the server'sGET /sessionsresponse —token_prefix, device_name, device_id, created_at, last_seen, expires_at, grants, transport_hint, is_current.AuthManager.currentPairedSession: StateFlow<PairedSession?>— UI reads this for pair metadata without poking at prefs.handleAuthOkparsesexpires_at/grants/transport_hintfrom the payload, tolerates both int and float epoch seconds, persists alongside the token.authenticate(ttlSeconds)injectsttl_seconds+grantsinto pairing-mode auth envelopes. Session-mode re-auth (existing session token present) does NOT re-send these — the server keeps the grant table keyed on the original pair.
QR payload v2 (ui/components/QrPairingScanner.kt):
HermesPairingPayloadgainssig: String?(parsed, stored, NOT verified — the phone doesn't have the HMAC secret).RelayPairinggainsttlSeconds: Long?,grants: Map<String, Long>?,transportHint: String?— all optional with defaults.parseHermesPairingQrno longer rejects on version mismatch — anyhermes >= 1with a non-blank host decodes. Future v3+ still parses viaignoreUnknownKeys = true. v1 QRs with nohermesfield also parse.- Signature verification is a
// TODOin the parser — full verification requires a phone-side secret distribution path the protocol doesn't yet define.
TTL picker (new ui/components/SessionTtlPickerDialog.kt):
- Radio list: 1 day / 7 days / 30 days / 90 days / 1 year / Never expire.
defaultTtlSeconds(qrTtl, transportHint, tailscale)logic: QR operator value wins; else wss OR Tailscale → 30d; plain ws → 7d; unknown → 30d.- Never expire is always selectable per Bailey's explicit "don't force check" direction. Shows an inline warning — "This device will stay paired until you revoke it manually. Only choose this if you control the network — LAN, Tailscale, VPN, or TLS." — but does NOT gate.
- Dialog ALWAYS opens on QR scan so the user confirms the TTL — the trust model is the user's judgment, not the QR's.
- Last user pick persists to
PairingPreferences.pairTtlSecondsand becomes the preselected default on future pairs.
Transport security UI (new ui/components/TransportSecurityBadge.kt, ui/components/InsecureConnectionAckDialog.kt):
- Badge renders in 3 states (secure green 🔒 / amber insecure-with-reason / red insecure-unknown) and 3 sizes (Chip / Row / Large). Rendered in Settings → Connection, in the Session info sheet, and on each Paired Device card.
- Insecure ack dialog shown first time the user toggles insecure mode on. Body: plain-language threat model. Radio buttons for reason — "LAN only" / "Tailscale or VPN" / "Local development only". Only dismissible after selecting a reason + tapping "I understand". Marks
insecure_ack_seen = true. Reason is stored for display, NOT gating.
Tailscale detection (new util/TailscaleDetector.kt):
- Checks
NetworkInterface.getNetworkInterfaces()fortailscale0interface or an address in100.64.0.0/10(Tailscale CGNAT range), plus the configured relay URL host (.ts.net/ 100.x.y.z). StateFlow<Boolean>refreshed on network changes viaConnectivityManagercallback.- Purely informational — shown as a "Tailscale detected" green chip in the Connection section. Does NOT auto-change any defaults per Bailey's #5 requirement.
Paired Devices screen (new ui/screens/PairedDevicesScreen.kt):
- Nav destination reached from Settings → Connection → "Paired Devices".
- Fetches via
RelayHttpClient.listSessions()(new method, GET/sessionswith bearer auth). Each card: device name + ID, "Current device" badge ifisCurrent, transport security badge, expiry ("Expires Apr 18, 2026" or "Never"), per-channel grant chips, revoke button. - Revoke button → confirmation dialog →
RelayHttpClient.revokeSession(tokenPrefix). Revoking the current device wipes local session token + redirects to pairing flow. - Pull-to-refresh. Empty state "No paired devices". Graceful 404 handling — if the server hasn't shipped
/sessionsyet, renders empty list instead of crashing.
network/RelayHttpClient.kt — two new methods:
listSessions(): Result<List<PairedDeviceInfo>>— GET/sessionswith bearer auth. 404 treated as "endpoint not implemented yet" → empty list.revokeSession(tokenPrefix): Result<Unit>— DELETE/sessions/{prefix}with bearer auth. 404 treated as "already gone" → success.
network/ConnectionManager.kt — TOFU integration:
- Optional
CertPinStoreparam. RebuildsOkHttpClienton every connect with a freshCertificatePinnersnapshot (so re-pair pin wipes take effect immediately). Records peer cert fingerprint inonOpenwhen the handshake is TLS. - Connect logic moved to IO dispatcher so the DataStore read for pin snapshot doesn't run on the caller's thread.
viewmodel/ConnectionViewModel.kt:
- Wires
AuthManagerbeforeConnectionManagerso the pin store is available. - Exposes
currentPairedSession,pairedDevices+pairedDevicesLoading+pairedDevicesError,isTailscaleDetected,insecureAckSeen,insecureReason. loadPairedDevices(),revokeDevice(prefix),setInsecureAckComplete(reason).- Shuts down Tailscale detector on
onCleared.
ui/screens/SettingsScreen.kt — integration:
onNavigateToPairedDevicescallback added to the signature.- Connection card: new TransportSecurityBadge row, Tailscale chip (if detected), hardware-storage chip, Paired Devices navigation row.
- Insecure toggle now routes through the first-time ack dialog. Cancel leaves the toggle off; confirm persists the reason and flips the mode.
- QR scan handoff now stages the payload into
pendingQrPayloadand opensSessionTtlPickerDialog. Picker confirm applies URLs/key/code, callsapplyServerIssuedCodeAndReset(code, relayUrl)(wipes the TOFU pin), storespendingGrants+pendingTtlSeconds, then tests the connection.
ui/components/ConnectionInfoSheet.kt:
SessionInfoSheetnow readscurrentPairedSessionand displays Expires / Channel grants / Transport / Key storage rows when paired.
ui/RelayApp.kt:
- New
Screen.PairedDevicesnav destination wired into the nav graph with back + re-pair callbacks. Reachable only from Settings — no bottom-nav slot.
New DataStore keys (data/PairingPreferences.kt):
pair_ttl_seconds(Long, user's last-selected TTL)insecure_ack_seen(Boolean)insecure_reason(String:"lan_only"/"tailscale_vpn"/"local_dev"/"")tofu_pins(string-encoded map)
Security review
The expansion of what a paired phone can do is:
- Listing all sessions (prefixes only, not full tokens) — a compromised phone was already able to see its own session, this adds visibility into other paired devices' metadata. Acceptable because the paired-phone population is explicitly trusted by the operator (they approved each QR pair).
- Revoking other sessions — any paired phone can revoke any other paired device. This is DoS territory: a compromised phone could lock out all other devices. Mitigation: revocation is auditable via the relay logs, and re-pairing is always possible from the host. For most deployments (one operator, 1-2 phones) this is acceptable. For multi-user deployments it'll need per-device role model (admin / user).
- QR payload signing — raises the bar for QR tampering (attacker can't inject their own pairing code via a modified QR photo), but it's defensive: the phone doesn't verify signatures yet. The server-side infrastructure is there so phone-side verification can land in a follow-up once the secret distribution model is defined.
- Never-expire sessions — Bailey's explicit call: allow based on user intent, not gated. Users who pick this are accepting the risk. The Paired Devices screen makes revocation trivial.
- TOFU pinning — protects against MITM of a self-signed wss cert AFTER the first connect. Does not protect the first connect itself (trust-on-first-use). Acceptable for the LAN / Tailscale / VPN deployment model.
- StrongBox storage — improves attacker cost for on-device token extraction on Android 9+ devices with StrongBox hardware. Best-effort; older devices fall back to TEE-backed EncryptedSharedPreferences which is still strong.
Files created/modified
Server (11 files):
- New:
plugin/relay/qr_sign.py,plugin/tests/test_qr_sign.py,plugin/tests/test_rate_limit_clear.py,plugin/tests/test_session_grants.py,plugin/tests/test_sessions_routes.py - Modified:
plugin/cli.py,plugin/pair.py,plugin/relay/auth.py,plugin/relay/server.py - Unchanged:
plugin/relay/__init__.py(version stays0.2.0per Bailey's direction — never released)
Phone (17 files):
- New:
auth/CertPinStore.kt,auth/PairedSession.kt,auth/SessionTokenStore.kt,data/PairingPreferences.kt,ui/components/InsecureConnectionAckDialog.kt,ui/components/SessionTtlPickerDialog.kt,ui/components/TransportSecurityBadge.kt,ui/screens/PairedDevicesScreen.kt,util/TailscaleDetector.kt - Modified:
auth/AuthManager.kt,network/ConnectionManager.kt,network/RelayHttpClient.kt,ui/RelayApp.kt,ui/components/ConnectionInfoSheet.kt,ui/components/QrPairingScanner.kt,ui/screens/SettingsScreen.kt,viewmodel/ConnectionViewModel.kt
Docs: this DEVLOG entry, ADR 15 in docs/decisions.md, §3.3 / §3.3.1 / §3.4 rewrites in docs/spec.md, route tables in docs/relay-server.md + user-docs/reference/relay-server.md, config sections in user-docs/reference/configuration.md, Key Files + Integration Points in CLAUDE.md, pair flow in user-docs/guide/getting-started.md.
Known gaps / follow-ups
- Phone-side QR signature verification — parsing + storing the
sigfield works; full verification requires a secret distribution mechanism (pre-shared key? on-device enrollment?) the protocol doesn't yet define. - Bidirectional pairing full UX —
POST /pairing/approveis a working stub. Real Phase 3 work needs a pending-codes store + operator approval UI. - Per-device role model — all paired devices currently have equal revoke rights. Multi-user / admin-vs-user split is a future refactor.
- Grant renewal UI — the Paired Devices screen shows expiry but has no "extend this grant" action. Pair-again from the host or just revoke + re-pair.
- Build verification — no gradle run in this session. Bailey deploys from Android Studio. If a compile bug slipped through a KDoc or Compose misuse, it'll surface on the first build attempt (like the
text/*comment bug earlier today).
2026-04-11 — Inbound Media v2: bare-path fetch + session-reload re-parse
Why this exists:
On-device testing of the v1 inbound media pipeline (shipped earlier today, commits 1195778 + 8f61262) surfaced two bugs that showed up in the same screenshot:
- Placeholder flicker. The
⚠️ Image unavailablecard rendered for a split second and then vanished, replaced by rawMEDIA:/tmp/...text in the bubble. - Blank-looking attachment bubbles. Related symptoms of the same underlying issue — messages re-rendered inconsistently during the turn-end reload.
Both root-caused to the session_end reload pattern documented in CLAUDE.md: when a streaming turn completes, ChatViewModel.onCompleteCb calls client.getMessages() → ChatHandler.loadMessageHistory(), which wholesale-replaces _messages.value with fresh ChatMessages built from item.contentText. The streaming-time media marker parser had stripped the markers from the client-side copy and injected attachments, but NEITHER mutation was visible to loadMessageHistory — the server-stored text still contained the raw markers, and the client-injected attachments were gone.
Fix 1: re-run the media marker parser on reloaded history. loadMessageHistory now scans each loaded assistant message's content, strips matched lines, and queues onMediaAttachmentRequested / onMediaBarePathRequested callbacks to fire after the wholesale _messages.value assignment (so mutateMessage lookups hit the newly-loaded IDs). dispatchedMediaMarkers is cleared at the same time since pre-reload dedupe keys are meaningless against post-reload message IDs. Extracted into extractMediaMarkersFromContent — a pure helper that doesn't touch mutable buffer state. Shipped in commit 272a3c5.
While digging into the flicker bug I noticed something bigger and more important in Bailey's screenshot:
The LLM is emitting MEDIA:/tmp/... in its free-form text completions, not via the tool. Upstream hermes-agent/agent/prompt_builder.py:266 explicitly instructs the model in its system prompt: "include MEDIA:/absolute/path/to/file in your response. The file..." So the LLM treats MEDIA markers as a first-class way to request file delivery — in free-form completions, not through tool calls. Our v1 fix only intercepted markers emitted by tools that called register_media() — which covers android_screenshot but not the much larger set of LLM free-form emissions. For those, the marker form is always bare-path (MEDIA:/tmp/foo.jpg), and our phone-side handler treated bare-path as "unavailable" no matter what.
Fix 2: GET /media/by-path on the relay + phone-side fetchMediaByPath. Adds a second fetch route alongside the existing /media/{token}. Key points:
- Shared path validation — extracted
validate_media_path(path, allowed_roots, max_size_bytes) -> (real_path, size)at the top ofplugin/relay/media.py. BothMediaRegistry.register(the loopback-only tool path) and the newhandle_media_by_path(the phone-auth'd direct fetch) call it, so the sandbox rules can't drift. - Same trust model as
/media/{token}— bearer auth against the existingSessionManager. Only a paired phone with a valid relay session token can fetch; 401 for everyone else. - Path sandboxing — absolute path →
os.path.realpath→ must resolve under an allowed root (tempfile.gettempdir()+HERMES_WORKSPACE+RELAY_MEDIA_ALLOWED_ROOTS) → must exist → must be a regular file → must fit underRELAY_MEDIA_MAX_SIZE_MB. Symlink escape is blocked by therealpathpre-check. 403 for any violation; 404 if the file is missing; 400 if thepathquery param is absent. - Content-Type negotiation — if the phone passes
?content_type=<...>it's honored; otherwise the server guesses via Pythonmimetypes.guess_type(). Falls back toapplication/octet-stream. - Route ordering —
/media/by-pathis registered before/media/{token}increate_appor aiohttp swallows the literal path as a token and 404s. Commented in the source as a reminder. - Phone-side rename for clarity —
onUnavailableMediaMarker→onMediaBarePathRequested(ChatHandler callback and ChatViewModel method). The name "unavailable" made sense in v1 when bare-path was always terminal; now bare-path is the primary LLM format, so it's a request-to-fetch like the token form. The failure branch still produces the⚠️ Image unavailablecard, but only when the fetch actually fails. - Shared fetch pipeline —
performFetch→performFetchWith(handler, messageId, fetchKey, settings, fetch: suspend () -> Result<FetchedMedia>). Takes the fetch lambda as a parameter so both the token path (relay.fetchMedia(token)) and the bare-path (relay.fetchMediaByPath(path)) share the same size-cap / cache / state-flip logic. ThefetchKeystored inAttachment.relayTokendisambiguates via the leading/—secrets.token_urlsafenever produces a/, so the prefix check is unambiguous.manualFetchAttachment(retry CTA) uses the same discriminator.
Security review (in ADR 14 addendum):
Adding /media/by-path widens what a paired phone can request by one degree — it can now read any file in the allowed-roots whitelist without host-local tool cooperation. This does NOT widen the trust boundary because (1) the whitelist is the same, (2) /tmp on Linux is already world-readable to same-user processes, (3) bearer auth still requires a valid session token, and (4) realpath symlink-resolves before the whitelist check so symlink escape is still blocked. Operators who want a tighter sandbox should narrow RELAY_MEDIA_ALLOWED_ROOTS, not disable the endpoint.
Tests added (plugin/tests/test_relay_media_routes.py):
test_by_path_without_authorization_returns_401test_by_path_with_invalid_bearer_returns_401test_by_path_missing_path_param_returns_400test_by_path_outside_sandbox_returns_403test_by_path_nonexistent_in_sandbox_returns_404test_by_path_relative_path_returns_403test_by_path_happy_path_streams_bytes(verifies auto-guessedContent-Type: image/pngfrom.pngextension)test_by_path_content_type_hint_overrides_guess(verifies?content_type=application/jsonwins over extension guess)test_by_path_oversized_returns_403(uses try/finally to restoremax_size_bytesso other tests in the class aren't affected —AioHTTPTestCasereuses one app instance)
Files created/modified:
Server (Python):
plugin/relay/media.py— newvalidate_media_path()module-level helper +_is_under_any_root()private helper;MediaRegistry.registerrefactored to call the new helper; duplicate_is_under_allowed_rootmethod removed.plugin/relay/server.py— newhandle_media_by_pathroute handler, new imports (mimetypes,os,validate_media_path), route registration increate_app(order-sensitive —by-pathbefore{token}).plugin/tests/test_relay_media_routes.py— 9 new tests for the by-path endpoint.
Phone (Kotlin):
app/src/main/kotlin/com/hermesandroid/relay/network/RelayHttpClient.kt— newfetchMediaByPath(path, contentTypeHint)method, usesokhttp3.HttpUrlbuilder for correct query-param encoding (paths with slashes / spaces / unicode).app/src/main/kotlin/com/hermesandroid/relay/network/handlers/ChatHandler.kt— renameonUnavailableMediaMarker→onMediaBarePathRequested, updated KDoc explaining the new semantics.app/src/main/kotlin/com/hermesandroid/relay/viewmodel/ChatViewModel.kt— rename + rewrite method body (now inserts LOADING, applies cellular gate, callsperformFetchWith { relay.fetchMediaByPath(path) });performFetch→performFetchWithsignature change (takesfetch: suspend () -> Result<FetchedMedia>lambda);manualFetchAttachmentdispatches to token-vs-path branch byfetchKey.startsWith("/").
Docs:
docs/decisions.md— ADR 14 addendum covering the bare-path fetch endpoint, upstream-prompt rationale, and security reviewdocs/relay-server.md+user-docs/reference/relay-server.md— new/media/by-pathroute entrydocs/spec.md— §6.2a updated with three-route listing + bare-path flowuser-docs/reference/configuration.md— honest description of the bare-path-as-primary-format modelCLAUDE.md— Integration Points table split into token / path fetch + tool / LLM marker rowsDEVLOG.md— this entry
Known gaps still filed for later:
- Session replay across relay restarts (phone-side persistent cache by token/hash-indexed)
- Auto-fetch threshold slider enforcement (currently persisted-not-enforced placeholder)
- Build verification — I haven't run gradle against the phone side; relying on type-checks and by-eye review. Bailey builds from Studio.
Next cycle discussion point: Bailey raised a broader design question about pairing security and TTL policy — bidirectional pairing, secure-by-default, user-selectable pair duration at initial pair, separate defaults for terminal vs bridge, opt-in never-expire for secured transports. Deferred to next design pass; not in this commit.
2026-04-11 — Inbound Media Pipeline (agent → phone, Discord-style file rendering)
Done:
- Root cause surfaced. The
android_screenshottool has always returnedMEDIA:/tmp/...in its response text, assuming hermes-agent's gateway would extract and deliver the file as a native attachment. Upstream verification againstgateway/platforms/api_server.pyshowedAPIServerAdapter.send()is an explicit no-op ("API server uses HTTP request/response, not send()") and_write_sse_chat_completionstreams raw deltas without ever invokingextract_media(). The upstream extract-media / send_document machinery (gateway/run.py:4570,4747) is wired for push platforms only (Telegram, Feishu, WeChat). On our HTTP pull adapter, theMEDIA:tag has always passed through to the phone as literal text. No existing upstream path exists for delivering files over the HTTP API surface without a platform-adapter PR. - Workaround landed: plugin-owned file-serving on the relay. Added a
MediaRegistryand two new routes to the plugin's existing relay server. Media-producing tools POST to a loopback-onlyPOST /media/registerwith a file path + content type, get back an opaquesecrets.token_urlsafe(16)token, and emitMEDIA:hermes-relay://<token>in their chat response text instead of the bare path. The phone'sChatHandlerparses the marker out of the SSE stream, fires a ViewModel callback, andRelayHttpClientfetches the bytes overGET /media/{token}withAuthorization: Bearer <session_token>(reusing the existingSessionManager). Bytes land incacheDir/hermes-media/, get shared viaFileProvider(${applicationId}.fileprovider), and render inline via a newInboundAttachmentCardcomponent. Result: zero LLM context bloat (token is ~25 chars), no upstream fork, no new auth model. - Registry design. In-memory
OrderedDictLRU withasyncio.Lockfor thread-safety. Defaults: 24-hour TTL (chosen to cover within-a-day session scrollback — the real human use case; anything longer is wasted since SessionManager is in-memory and relay restarts invalidate all tokens regardless), 500-entry LRU cap (prevents runaway memory/disk under screenshot spam), 100 MB file-size cap (guards against/media/registerbeing handed a 10 GB file). Path sandboxing: file must be absolute,os.path.realpath()resolve under an allowed root (default:tempfile.gettempdir()+HERMES_WORKSPACEor~/.hermes/workspace/+ anyRELAY_MEDIA_ALLOWED_ROOTSentries), exist, be a regular file, and fit under the size cap. The token → path mapping is held server-side — the client only ever presents an opaque token on GET, so there's zero path-traversal surface on the fetch endpoint. - Fallback when relay isn't running. The tool calls
register_media()via stdliburllib.requestwith a 5s timeout; on any failure (relay down, connection refused, non-200 response) it returns the legacyMEDIA:<tmp_path>form with a logger warning. The phone'sChatHandlerrecognizes the bare-path form via a second regex and firesonUnavailableMediaMarker, which inserts a FAILEDAttachmentplaceholder rendering⚠️ Image unavailable — relay offline. No regression versus today's behavior; the placeholder is just tidier than raw marker text. - Discord-style rendering on the phone. New
AttachmentState { LOADING, LOADED, FAILED }andAttachmentRenderMode { IMAGE, VIDEO, AUDIO, PDF, TEXT, GENERIC }on the existingAttachmentdata class.InboundAttachmentCarddispatches by(state × renderMode): images render inline from the cached URI (decoded viaBitmapFactory.decodeByteArray+asImageBitmap, matching the existing outbound-attachment render path — no Coil/Glide added); video/audio/pdf/text/generic render as tap-to-open file cards that fireACTION_VIEWwithFLAG_GRANT_READ_URI_PERMISSION. The same component now handles outbound attachments too (they default tostate=LOADED), soMessageBubble.ktno longer has a separate outbound-only render branch. - Cellular gate. If
autoFetchOnCellular == false(default) and the device is on a cellular network, the attachment stays in LOADING state witherrorMessage = "Tap to download"— the user taps to triggermanualFetchAttachment(), which re-issues the fetch ignoring the cellular gate. Encoded via existing enum + errorMessage slot rather than adding a new state value to keep the data class surface small. - Dedup.
ChatHandler.dispatchedMediaMarkersis a per-session set that prevents double-firing between real-time streaming scans (scanForMediaMarkerscalled fromonTextDelta) and the post-stream reconciliation pass (finalizeMediaMarkerscalled fromonTurnComplete/onStreamComplete). Marker parsing runs unconditionally — not gated on theparseToolAnnotationsfeature flag. - Settings UI. New "Inbound media" subsection in Settings (between Chat and Appearance) exposes four DataStore-backed knobs: max inbound attachment size (5–100 MB, default 25), auto-fetch threshold (0–50 MB, default 2 — persisted but not currently enforced; only the cellular toggle gates fetches today, with the threshold reserved for forward-compatibility), auto-fetch on cellular (default off), and cached media cap (50–500 MB, default 200) with a "Clear cached media" button that calls
MediaCacheWriter.clear()and shows a Toast with the freed byte count. LRU eviction on the cache is by file mtime. - Auth parity. The media GET endpoint uses the same relay session token that gates the WSS channel itself — no stronger, no weaker. User raised the question of whether the media endpoint needed its own auth given that chat is optionally unauthenticated; answer is the relay session token (issued at pairing, stored in
EncryptedSharedPreferences) is a separate and always-required credential, so/media/<token>inherits exactly the WSS trust level and adds unguessable per-file entropy on top. Opt-in insecure (ws://) mode intentionally does nothing to strengthen this — it matches the existing "trusted LAN" assumption for local dev. - Tests. 11 registry tests (happy path, expiry, LRU eviction, LRU reorder on get, relative path rejection, nonexistent path rejection, directory rejection, outside-allowed-roots rejection, symlink-escape rejection [skipped on Windows without symlink priv], oversized rejection, empty content_type rejection) + 8 route tests (
/media/registernon-loopback 403, happy path 200, validation 400, bad JSON 400;/media/{token}no auth 401, bad bearer 401, valid + streamed 200, expired 404, unknown 404). Usesunittest.IsolatedAsyncioTestCase+aiohttp.test_utils.AioHTTPTestCase(no pytest-asyncio dep required).
Why this wasn't Option A (inline base64 in tool output):
- Inline base64 bloats the LLM context on every call (~135 KB per 1080p screenshot, growing with history), matters for video/audio scalability, and forces the agent to pay for bytes it's just routing to the phone. User explicitly rejected that tradeoff.
- Option B (plugin-owned file endpoint) decouples the wire format from the file bytes: tokens are ~25 chars, bytes flow out-of-band over a separate authenticated HTTP channel. Costs: new endpoint surface area, new phone-side fetch path, FileProvider plumbing — but all of it lives in files we already own.
Files created:
Server (Python):
plugin/relay/media.py—MediaRegistry,_MediaEntry,MediaRegistrationError,_default_allowed_roots()plugin/relay/client.py— stdliburllib.request-basedregister_media()+_post_loopback()helper (kept separate fromplugin/pair.py's existingregister_relay_codeto avoid weakening that function's narrower error surface)plugin/tests/test_media_registry.py— 11 tests,unittest.IsolatedAsyncioTestCaseplugin/tests/test_relay_media_routes.py— 8 tests,aiohttp.test_utils.AioHTTPTestCase
Phone (Kotlin):
app/src/main/kotlin/com/hermesandroid/relay/network/RelayHttpClient.kt— OkHttp GET, ws→http URL rewrite, Content-Disposition filename parse, Resultapp/src/main/kotlin/com/hermesandroid/relay/data/MediaSettings.kt— DataStore-backedMediaSettings+MediaSettingsRepositoryapp/src/main/kotlin/com/hermesandroid/relay/util/MediaCacheWriter.kt— LRU-capped cache atcacheDir/hermes-media/, FileProvider URI generation, MIME→ext mapapp/src/main/kotlin/com/hermesandroid/relay/ui/components/InboundAttachmentCard.kt— single component dispatching onstate × renderModeapp/src/main/res/xml/file_provider_paths.xml—<cache-path name="hermes-media" path="hermes-media/"/>
Files modified:
Server:
plugin/relay/config.py— 4 new fields (media_max_size_mb,media_ttl_seconds,media_lru_cap,media_allowed_roots),from_env()parsingplugin/relay/server.py—self.media = MediaRegistry(...)inRelayServer.__init__,handle_media_register+handle_media_get+ route registration increate_appplugin/tools/android_tool.py—android_screenshot()callsregister_media()→ emitshermes-relay://<token>on success, falls back to bare path with alogging.warningon failureplugin/android_tool.py— identical change to the top-level duplicate copy
Phone:
app/src/main/AndroidManifest.xml—<provider>forandroidx.core.content.FileProviderwith authority${applicationId}.fileproviderapp/src/main/kotlin/com/hermesandroid/relay/data/ChatMessage.kt—AttachmentState+AttachmentRenderModeenums, extendedAttachmentwithstate/errorMessage/relayToken/cachedUri,textLikeMimescompanion,renderModecomputed propertyapp/src/main/kotlin/com/hermesandroid/relay/network/handlers/ChatHandler.kt—mediaRelayRegex+mediaBarePathRegex,onMediaAttachmentRequested+onUnavailableMediaMarkerasvarcallbacks (not ctor params),mediaLineBuffer+dispatchedMediaMarkersdedupe set,scanForMediaMarkerscalled unconditionally fromonTextDelta,finalizeMediaMarkerscalled fromonTurnComplete/onStreamComplete,mutateMessagehelper exposed so the ViewModel can flip attachment state on the private_messagesStateFlowapp/src/main/kotlin/com/hermesandroid/relay/viewmodel/ChatViewModel.kt— newinitializeMedia(context, relayHttpClient, mediaSettingsRepo, mediaCacheWriter),onMediaAttachmentRequested,performFetch,manualFetchAttachment,onUnavailableMediaMarker,MEDIA_TAP_TO_DOWNLOADcompanion constantapp/src/main/kotlin/com/hermesandroid/relay/viewmodel/ConnectionViewModel.kt— owns media singletons (mediaSettingsRepo,mediaCacheWriter,relayHttpClient), sharedOkHttpClient,_cachedMediaCapMbmirror loop so the writer's cap lambda is synchronousapp/src/main/kotlin/com/hermesandroid/relay/ui/RelayApp.kt—chatViewModel.initializeMedia(...)wired inside the existingLaunchedEffect(apiClient)blockapp/src/main/kotlin/com/hermesandroid/relay/ui/components/MessageBubble.kt— replaced outbound-only attachment rendering withattachments.forEachIndexed { InboundAttachmentCard(...) }, addedonAttachmentRetry+onAttachmentManualFetchparamsapp/src/main/kotlin/com/hermesandroid/relay/ui/screens/ChatScreen.kt— empty-bubble skip now respectsattachments.isNotEmpty(), wiresmanualFetchAttachmentto both retry + manual-fetch slotsapp/src/main/kotlin/com/hermesandroid/relay/ui/screens/SettingsScreen.kt— newInboundMediaSection(connectionViewModel)composable between Chat and Appearance (coexists with the other team's unified Connection section; no collision)
Files NOT touched (other team owns them or out-of-scope): AuthManager.kt (other team added applyServerIssuedCodeAndReset for the manual-code-entry dialog), ConnectionInfoSheet.kt (other team's new bottom-sheet component for Connection rows), plugin/pair.py, anything under relay_server/ (thin shim — untouched), any upstream hermes-agent code.
Next:
- Wire the auto-fetch threshold slider to the actual fetch logic — currently only the cellular toggle gates fetches, and the threshold is persisted-but-unused as a forward-compatibility placeholder. Real enforcement would need either a HEAD preflight to get the size before committing to the fetch, or we accept the post-hoc reject (byte-count comparison after the body lands, wasted bytes on oversize).
- Phone-side persistence of fetched media so session replay works across relay restarts. Currently the
FileProvidercache is opaque toChatHandler— if the user scrolls back into a session from yesterday, the tokens in the stored message text are stale (relay registry is in-memory) and the fetch 404s. Phone-side token-or-hash-indexed cache would survive this. - Consider wiring the same pipeline into any future tools that want to emit files (voice, plots, reports). The
MediaRegistry+register_media()helper is tool-agnostic — onlyandroid_screenshotuses it today. - Unit-test coverage for the Kotlin side:
ChatHandlermarker parsing,RelayHttpClientURL-rewrite,MediaCacheWriterLRU eviction. The Python side has 19 tests; the Kotlin side currently has none for the media pipeline. - Possible upstream contribution to
hermes-agent: makegateway/platforms/api_server.py's_write_sse_chat_completionroute deltas throughGatewayStreamConsumerso the_MEDIA_REstripper ingateway/stream_consumer.py:188engages. That would at least keep rawMEDIA:tags out of the chat display for other HTTP-API clients that don't implement their own phone-side parser. Would not solve the actual file-delivery problem (still nosend_documentimpl) but would at least stop the leakage. Track indocs/upstream-contributions.md.
Blockers:
- None. The feature is ready for on-device testing.
Test plan (for on-device smoke):
- Start relay (
scripts/dev.bat relayor equivalent), pair phone, open chat. - Invoke a tool that produces a screenshot (e.g., via an agent command that triggers
android_screenshot). Verify the screenshot renders inline as an image, not as raw text. - Kill the relay mid-session, trigger another screenshot, verify the
⚠️ Image unavailable — relay offlineplaceholder renders. - In Settings → Inbound media: adjust the max-size slider, toggle cellular, hit "Clear cached media", verify toast with freed bytes.
- Tap a non-image attachment (test with a PDF tool result if available) and verify
ACTION_VIEWopens an external app with a validcontent://URI.
2026-04-11 — Install Flow Canonicalization (external_dirs + pip install -e + skill category layout)
Done:
- Install flow rewritten to match Hermes canonical distribution patterns (per
~/.hermes/hermes-agent/website/docs/user-guide/features/skills.md). The newinstall.shclones the repo to~/.hermes/hermes-relay/(override with$HERMES_RELAY_HOME) instead of a throwaway tmpdir,pip install -es the package into the hermes-agent venv, and registers the clone'sskills/directory underskills.external_dirsin~/.hermes/config.yamlvia an idempotent YAML edit. The plugin is symlinked into~/.hermes/plugins/hermes-relay, and a thin~/.local/bin/hermes-pairshim execspython -m plugin.pairin the venv. - Updates are now
cd ~/.hermes/hermes-relay && git pull— one command updates plugin (editable install picks up changes automatically) + skill (external_dirsis scanned fresh on every hermes-agent invocation) + docs. Nohermes skills updatestep — that only applies to hub-installed skills, notexternal_dirs-scanned ones. - Skill directory now follows canonical category layout —
skills/devops/hermes-relay-pair/SKILL.md(category subdir matching themetadata.hermes.category: devopsfrontmatter), not the old flatskills/hermes-relay-pair/. skills/hermes-pairing-qr/deleted entirely — the pre-plugin bash script + SKILL.md. Replaced byskills/devops/hermes-relay-pair/+plugin/pair.py(Python module) +hermes-pairshell shim.plugin/skill.mddeleted — old lowercase-s flat-file artifact from before the skill system existed.- Documented the upstream CLI gap — hermes-agent v0.8.0's
PluginContext.register_cli_command()is wired on the plugin side, buthermes_cli/main.py:5236only readsplugins.memory.discover_plugin_cli_commands()and never consults the generic_cli_commandsdict. Third-party plugin CLI commands never reach the top-level argparser. Docs no longer promisehermes pair(with a space) works — only/hermes-relay-pair(slash command via the skill) andhermes-pair(dashed shell shim) are documented as working entry points.
Files changed:
README.md— Quick Start replaceshermes pairwith/hermes-relay-pair+hermes-pair, adds update-via-git pullnote, updates repo structure to showskills/devops/hermes-relay-pair/docs/relay-server.md— pairing description and/pairing/registerrow updated to reference the new entry pointsdocs/decisions.md— new ADR 13 on skill distribution viaexternal_dirsuser-docs/guide/getting-started.md— full install-flow rewrite covering the 5-step canonical installer, update mechanism, slash command vs shell shim entry points, upstream CLI gap warninguser-docs/reference/configuration.md— newSkills (external_dirs)subsection, command references updateduser-docs/reference/relay-server.md— pairing model + troubleshooting updatedCLAUDE.md— Repo Layout showsskills/devops/hermes-relay-pair/; Key Files gainsinstall.sh, drops deprecatedhermes-pairing-qrrows andplugin/skill.mdreferences; integration points updatedAGENTS.md— Setup steps rewritten around the canonical installerDEVLOG.md— this entry
Files NOT touched (main session owns them): plugin/**, relay_server/**, app/**, pyproject.toml, skills/devops/hermes-relay-pair/SKILL.md, install.sh. The deleted skills/hermes-pairing-qr/ and plugin/skill.md paths are referenced only as historical deletions in this entry and ADR 13.
Next:
- Verify
/hermes-relay-pairrenders correctly once the skill is atskills/devops/hermes-relay-pair/SKILL.mdand hermes-agent reloads fromexternal_dirs. - Confirm
install.sh's YAML edit is actually idempotent against a pre-existingexternal_dirslist with a trailing comment — regression-test with a pathological config. - Upstream patch to
hermes_cli/main.pythat dispatches to the generic_cli_commandsdict — would let us restorehermes pairas a first-class CLI verb. Track indocs/upstream-contributions.md.
Blockers:
- Upstream argparser doesn't forward to plugin CLI dict (see above). Not blocking the install flow — the slash command + shell shim cover the same surface.
2026-04-11 — Settings Connection UX Rework (QR-first, collapsible manual + bridge)
Done:
- Unified Connection section on the Settings screen. Replaced the three separate top-level cards (API Server, Relay Server, Pairing) with a single Connection section containing three stacked cards:
- Pair with your server — always visible, primary entry point. Large Scan Pairing QR button + a unified status summary line showing API Server (Reachable / Unreachable), Relay (Connected / Disconnected), and Session (Paired / Unpaired). This is the one-button flow: scan the QR from
hermes pairon the host and everything is configured. - Manual configuration — collapsible. Starts collapsed when the user is already paired and reachable, expanded otherwise. Holds the manual-entry fields (API Server URL, API Key, Relay URL, Insecure Mode toggle) and the Save & Test button. Power-user / troubleshooting path.
- Bridge pairing code — collapsible, gated by the
relayEnabledfeature flag, starts collapsed. Shows the locally-generated 6-char pairing code with copy / regenerate icons. Explicitly labelled "For the Phase 3 bridge feature — the host approves this code to enable Android tool control. Not used for initial pairing." Replaces the old Pairing card, which was visually prominent but semantically misleading in the new QR-driven flow.
- Pair with your server — always visible, primary entry point. Large Scan Pairing QR button + a unified status summary line showing API Server (Reachable / Unreachable), Relay (Connected / Disconnected), and Session (Paired / Unpaired). This is the one-button flow: scan the QR from
- Why. The old layout buried the QR button inside the API Server card next to Save & Test, so new users couldn't tell which button was the primary setup path. The old Pairing card prominently displayed a phone-generated code that's no longer used for initial pairing — only for the future Phase 3 bridge direction. The rework makes the happy path (one QR scan → chat + relay) the obvious default and demotes both manual config and the bridge code to collapsibles for users who actually need them.
- User docs updated.
user-docs/guide/getting-started.md(Manual Pairing section now walks through Settings → Connection → Manual configuration),user-docs/reference/configuration.md(Onboarding Settings renamed to Connection Settings + describes the three-card layout), and theCLAUDE.mdKey Files entry forSettingsScreen.kt.
Files changed:
app/src/main/kotlin/com/hermesandroid/relay/ui/screens/SettingsScreen.kt— three-card Connection section, collapsible state, unified status summaryuser-docs/guide/getting-started.md— Manual Pairing section updated for Settings → Connection layoutuser-docs/reference/configuration.md— Onboarding Settings → Connection Settings, three-card layout describedCLAUDE.md— Key FilesSettingsScreen.ktentry updatedDEVLOG.md— this entry
Next:
- Update splash / onboarding completion screen so the "you can change this later in Settings" hint points at the Connection section, not the old API Server card.
- Screenshot pass for Play Store listing — the old screenshots still show the three-section layout.
- Consider whether the Bridge pairing code card should be hidden entirely (not just collapsed) until Phase 3 lands, to avoid confusing users who enable the relay feature flag for terminal alone.
Blockers:
- None.
2026-04-11 — QR-Driven Relay Pairing (one scan → chat + relay)
Done:
- Extended QR payload schema —
HermesPairingPayload(inplugin/pair.py+app/.../QrPairingScanner.kt) now carries an optionalrelayblock alongside the existing API server fields:{ "hermes": 1, "host", "port", "key", "tls", "relay": { "url": "ws://host:port", "code": "ABCD12" } }. Therelayfield is nullable andkotlinx.serializationruns withignoreUnknownKeys = true, so old API-only QRs still parse cleanly — no migration required. - New relay endpoint
POST /pairing/register(plugin/relay/server.py→handle_pairing_register) — Pre-registers an externally-provided pairing code with the running relay. Accepts{"code": "ABCD12"}, returns{"ok": true, "code": "ABCD12"}. Gated to loopback callers only (127.0.0.1/::1) — any non-localrequest.remotegets HTTP 403. Matches the trust model: only a process with host shell access can inject codes; a LAN attacker cannot. Validation delegates toPairingManager.register_code()which enforces the 6-charA-Z / 0-9format. hermes pairprobes + pre-registers the relay — When invoked, the command callsprobe_relay()againsthttp://127.0.0.1:RELAY_PORT/health; on success, mints a fresh 6-char code (random.SystemRandom, alphabetstring.ascii_uppercase + string.digits), posts it to/pairing/register, and embeds{url, code}in the QR. If the relay isn't running it prints an[info]pointing athermes relay startand renders an API-only QR. If registration fails it prints a[warn]and also falls back. New--no-relayflag skips the probe entirely for operators who only want direct chat.- Output format —
render_text_block()now renders a second "Relay (terminal + bridge)" section when a relay block is present, showing thews://host:portURL and the pairing code (with "expires in 10 min, one-shot" note) alongside the existing "Server" section. Unified warning at the bottom notes the QR contains credentials whenever an API key OR a relay code is present. - Pairing alphabet widened —
plugin/relay/config.py—PAIRING_ALPHABETwent from"ABCDEFGHJKLMNPQRSTUVWXYZ23456789"(32 chars, no ambiguous 0/O/1/I) to"ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"(36 chars). The phone-sidePAIRING_CODE_CHARS = ('A'..'Z') + ('0'..'9')inAuthManager.ktcould previously emit codes thatPairingManager.register_code()silently rejected as "invalid format". The old restriction only mattered when a human had to retype a code from a display; with QR + HTTP the full alphabet is correct. - Relay config env vars —
RELAY_HOST/RELAY_PORTare now consumed byplugin/pair.py'sread_relay_config()too (in addition toplugin/relay/config.py), sohermes pairandhermes relay startagree on where the relay lives. - Phase 3 note — Phone-side
generatePairingCode()inAuthManager.ktis retained. The bridge channel (Phase 3) will use the opposite flow — phone generates, host approves — andPOST /pairing/registeris written generically enough to serve both directions.
Files changed/added:
plugin/relay/server.py—handle_pairing_register+ route registration on/pairing/registerplugin/relay/auth.py—PairingManager.register_code()validation helperplugin/relay/config.py— widenedPAIRING_ALPHABET, comment explaining whyplugin/pair.py—probe_relay(),register_relay_code(),_generate_relay_code(),_relay_lan_base_url(),read_relay_config(); extendedbuild_payload()/render_text_block()/pair_command()plugin/cli.py—--no-relayflag onhermes pairapp/src/main/kotlin/com/hermesandroid/relay/ui/components/QrPairingScanner.kt—RelayPairingdata class + nullablerelayfield onHermesPairingPayloadREADME.md— Quick Start pairing section now mentions one-scan chat + relaydocs/spec.md— pairing flow section and QR wire formatdocs/decisions.md— new ADR entry on QR-carries-both-credentials trust modeldocs/relay-server.md— routes table includes/pairing/register, loopback restriction noteuser-docs/guide/getting-started.md— updated pairing stepsuser-docs/reference/relay-server.md— routes + pairing modeluser-docs/reference/configuration.md—RELAY_HOST/RELAY_PORT+ alphabet noteCLAUDE.md— Integration Points + Repo Layout references updated toplugin/relay/DEVLOG.md— this entry
Next:
- End-to-end test: start relay →
hermes pair→ scan QR on phone → verify both API and relay auto-configure → verify terminal tab attaches without asking for a pairing code. - If the phone's stored relay session token is still valid from a previous pairing, the new code should be a no-op (session reconnect takes priority over pairing in
_authenticate()). Verify that path doesn't accidentally consume the freshly-registered code.
Blockers:
- None.
2026-04-11 — Phase 2: Terminal Channel MVP (server + app)
Done:
- Server-side
TerminalHandler(relay_server/channels/terminal.py) — Replaced the stub with a real PTY-backed shell handler. Usespty.openpty()+fork+TIOCSCTTY(notpty.fork()) so we can setO_NONBLOCKon the master fd before handing it toloop.add_reader(). Output is batched on a ~16 ms window (vialoop.call_later) or flushed immediately on 4 KiB buffer — that keeps 60 fps refresh from a shell dumping megabytes without flooding the WebSocket. Supportsterminal.attach/input/resize/detach/list. Resize usesTIOCSWINSZioctl for SIGWINCH. Graceful teardown on disconnect: flush pending buffer → remove reader → SIGHUP →waitpid(WNOHANG)loop (up to 1 s grace) → SIGKILL fallback →os.close. Shell resolution checks absolute-path candidates (request → config →$SHELL→/bin/bash→/bin/sh) and rejects relative paths. Per-client cap of 4 concurrent sessions. Child getsTERM=xterm-256color,COLORTERM=truecolor, andHERMES_RELAY_TERMINAL=<session_name>as a debug marker. Unix-only:pty/termios/fcntlimports are guarded withtry/except ImportErrorso the relay still starts on Windows — attach attempts return a cleanterminal.errorinstead of crashing the whole server at import time. - Config — Added
terminal_shell: str | NonetoRelayConfig(RELAY_TERMINAL_SHELLenv var,None= auto-detect). Wired intoTerminalHandler(default_shell=...)inrelay.py. - xterm.js asset bundle (
app/src/main/assets/terminal/) — Downloaded@xterm/xterm@5.5.0+@xterm/addon-fit@0.10.0+@xterm/addon-web-links@0.11.0from jsDelivr intoassets/terminal/. Wroteindex.htmlwith a Hermes-themed palette (navy#1A1A2Ebackground, purple#B794F4cursor/magenta, magenta/cyan/green ANSI mapping that matches the app's Material 3 primary). Disables autocorrect/overscroll/zoom. Uses base64-encoded output payloads (window.writeTerminal('<b64>')) to avoid JS string-escape headaches with control bytes and escape sequences. TerminalViewModel.kt— AndroidViewModel mirroringChatViewModelinit pattern. Registers aChannelMultiplexerhandler for"terminal". State flow tracks attached/sessionName/pid/shell/cols/rows/tmuxAvailable/ctrlActive/altActive/error. Output flows on aMutableSharedFlow<String>(replay=0, buffer=256) — explicitly not a StateFlow because terminal chunks must be delivered exactly once; StateFlow would conflate rapid deltas and drop output. Sticky CTRL translates a–z/A–Z +[\]to their control bytes; sticky ALT prefixes ESC. Both auto-clear after the next keypress. Pending-attach queue: if the WebView signals ready before the relay connects, the cols/rows are held and the attach fires onceConnectionState.Connectedlands.TerminalWebView.kt— Compose WebView wrapper. Loadsfile:///android_asset/terminal/index.html, installsAndroidBridge@JavascriptInterface (onReady/onInput/onResize/onLink).viewModel.outputFlowis collected in aLaunchedEffecton the UI thread and piped intowebView.evaluateJavascript("window.writeTerminal('$b64')").DisposableEffecttears down the WebView cleanly on recomposition out. Uses the modernshouldOverrideUrlLoading(WebView, WebResourceRequest)signature (minSdk 26), routes non-asset URLs to the system browser viaACTION_VIEW.ExtraKeysToolbar.kt—RowScope-extensionToolbarKeycomposable for the 8-key bottom toolbar: ESC, TAB, CTRL (sticky), ALT (sticky), ←↓↑→. Active state highlights withprimary.copy(alpha=0.22f)background + primary border. HapticLongPressfeedback on every tap.TerminalScreen.kt— Replaced the "Coming Soon" placeholder. TopAppBar with monospace subtitle line that shows session name / "attaching…" / "relay disconnected" / error.ConnectionStatusBadgein the actions slot (green when attached, amber when attaching/reconnecting, red otherwise) +RefreshIconButton for manual reattach. WebView fillsweight(1f),ExtraKeysToolbaris anchored at the bottom withnavigationBarsPadding() + imePadding()so it slides up with the IME. Overlay card appears when relay is disconnected or there's an error, explaining state and pointing at Settings.RelayApp.ktwiring — ImportedTerminalViewModel, addedviewModel()instance, one-timeLaunchedEffectcallsterminalViewModel.initialize(multiplexer, relayConnectionState)so the channel handler registers and auto-attaches on reconnect.Screen.Terminalcomposable now passes both view models intoTerminalScreen.
Files changed/added:
relay_server/channels/terminal.py(rewritten — 560 lines of real PTY handling)relay_server/config.py(newterminal_shellfield + env var)relay_server/relay.py(passdefault_shellintoTerminalHandler)app/src/main/assets/terminal/index.html(new)app/src/main/assets/terminal/xterm.js+xterm.css+addon-fit.js+addon-web-links.js(new — ~300 KB bundled, no CDN dependency at runtime)app/src/main/kotlin/com/hermesandroid/relay/viewmodel/TerminalViewModel.kt(new)app/src/main/kotlin/com/hermesandroid/relay/ui/components/TerminalWebView.kt(new)app/src/main/kotlin/com/hermesandroid/relay/ui/components/ExtraKeysToolbar.kt(new)app/src/main/kotlin/com/hermesandroid/relay/ui/screens/TerminalScreen.kt(rewritten)app/src/main/kotlin/com/hermesandroid/relay/ui/RelayApp.kt(import + instantiate + init + pass to screen)DEVLOG.md(this entry)
Build: gradlew :app:assembleDebug — BUILD SUCCESSFUL in 1m 59s. Only pre-existing deprecation warnings remain; no warnings or errors from new code. Server-side Python is not covered by CI; change is additive and gated by _PTY_AVAILABLE on Windows hosts so the existing chat/bridge channels remain unaffected.
Not yet tested on real hardware. This session produced compiling code, not verified feature behavior. Before declaring Phase 2 MVP shipped we need:
- A Linux/macOS host running the relay server + tmux (or not — raw PTY fallback is what we actually built) with a shell the host user can actually log into.
- Deploy the debug APK, connect the relay, open the Terminal tab, verify: prompt appears → soft keyboard typing reaches the shell → arrow keys work → CTRL+C interrupts → resize on rotation / IME show reflows prompt correctly → htop renders with box chars → disconnect/reconnect reattaches cleanly.
- Check for WebView keyboard quirks on at least two devices (the plan flags this as the highest device-side risk).
Deferred from the Phase 2 plan (will land in follow-up sessions):
- Plugin consolidation —
relay_server/is still a separate process; the plan wants it absorbed intoplugin/relay/with a unifiedhermes relayCLI. Pure refactor, no user-visible change. Separate session. - tmux session persistence —
self.tmux_availableis detected and surfaced interminal.attachedpayloads but we're not using libtmux yet. Current implementation is raw PTY only. Adding tmux is additive (same envelope protocol, swap the spawn path). - P1/P2 polish — pinch-to-zoom, mouse reporting (needed for htop/vim mouse), font bundling (JetBrains Mono NF), multiple themes, settings screen entries, visual bell, scroll-to-bottom FAB, URL-detection config, multi-session picker dropdown, hardware keyboard edge cases.
- CLI commands —
hermes relay status/sessions/killare spec'd but not wired. Nothing to wire them to until plugin consolidation lands.
Next:
- Smoke-test on a real device with the relay running against a real Linux host.
- Fix whatever that surfaces (WebView keyboard oddities, resize timing, PTY race conditions we haven't seen yet).
- Decide whether to ship MVP as-is under a feature flag or continue straight through 2B polish → tmux → consolidation before any user sees it.
Blockers:
- None in code. Need a Linux/macOS relay host to exercise the PTY path end-to-end.
2026-04-10 — v0.1.0 Play Store Release (Internal Testing)
Done:
- Keystore — Generated
release.keystore(RSA 2048, SHA384withRSA, 10000-day validity, aliashermes-relay) viakeytool -genkey. Certificate subject:CN=Bailey Dixon, OU=Hermes-Relay, O=Codename-11, L=Tampa, ST=Florida, C=US. SHA1 fingerprintC9:8E:1B:74:A6:D8:A6:6E:0A:3A:C9:00:96:C2:0B:B7:44:B0:B7:FC; SHA256A9:A4:2D:94:20:8B:94:B3:68:5B:01:93:E3:94:9B:90:50:AD:80:60:56:E7:16:3C:FC:E5:11:AF:68:0D:79:4B. Stored at repo root asrelease.keystore(gitignored via.gitignore:31). Password stored in password manager; must back up file + password to a separate encrypted location before closing this session. - local.properties — Added
hermes.keystore.path,hermes.keystore.password,hermes.key.alias,hermes.key.passwordlines pointing at the absolute path (Gradle'sfile()resolves relative paths against theapp/module, not repo root, so absolute path with forward slashes is required on Windows). - Local bundle build —
gradlew bundleRelease— BUILD SUCCESSFUL in 4m 41s, producedapp/build/outputs/bundle/release/app-release.aab(19,071,575 bytes / 18.2 MB). Fingerprint-verified withkeytool -printcert -jarfile— AAB is release-signed with the correct cert (serialeaaf7de55766c57e), not the debug fallback. - GitHub Secrets — Set all four via
gh secret setCLI so.github/workflows/release.ymlwill release-sign CI artifacts on future tags:HERMES_KEYSTORE_BASE64(frombase64 -w 0 release.keystore),HERMES_KEYSTORE_PASSWORD,HERMES_KEY_ALIAS=hermes-relay,HERMES_KEY_PASSWORD. Usedprintf '%s'(no trailing newline) piped togh secret setfor the non-base64 values — a trailing\nwould get baked into the secret and cause "Keystore was tampered with" failures in CI. - Play Console upload — Uploaded the local
app-release.aabto the Internal testing track on Google Play Console.versionCode=1,versionName=0.1.0. Enrolled in Play App Signing (Google re-signs installs with their HSM-held key; the keystore is now only an upload key with reset-via-support recovery). Release rolled out successfully. One non-blocking warning about missing native debug symbols (deferred — see Next section). - Git tag —
git tag -a v0.1.0pushed toorigin, triggering.github/workflows/release.ymlto build APK + AAB +SHA256SUMS.txtand attach them to a GitHub Release namedv0.1.0. Tag landed on commit089e011(theplay-publisher 4.0.0AGP 9 compat bump from a parallel session), which means the CI-built AAB is byte-different from the Play Console AAB (differentlibs.versions.toml) but functionally identical and signed with the same cert — acceptable for Internal testing because only the Play Console artifact reaches testers; the GitHub Release is a secondary distribution channel.
Files changed:
local.properties(gitignored) — added release signing propertiesrelease.keystore(gitignored) — newDEVLOG.md— this entry
Next:
- Back up
release.keystore+ password to an encrypted off-machine location before closing this session. Losing both = losing ability to submit future upload keys (Play App Signing reset flow takes ~2 days). - Add native debug symbols for v0.1.1 — Add
ndk { debugSymbolLevel = "SYMBOL_TABLE" }to thereleasebuild type inapp/build.gradle.kts. Fixes the Play Console warning and gives readable native stack traces for crashes in transitive deps (ML Kit, CameraX, OkHttp BoringSSL, Compose/Skia). - Promote through tracks — New personal Play accounts need 14 continuous days of Closed testing with ≥12 opted-in testers before production rollout. Create a Closed testing track and recruit testers ASAP if the production timeline matters.
- Verify GitHub Release assets — Once
release.ymlfinishes, confirm the Release has APK + AAB +SHA256SUMS.txtand that the workflow summary says "Release-signed" (not the debug-signed warning banner).
Blockers:
- None.
2026-04-10 — Smooth Chat Auto-Scroll Fix + Compose Deprecation Cleanup
Done:
- Smooth chat auto-scroll — Rewrote
ChatScreen.ktauto-scroll logic to fix five bugs surfaced while recording the demo video. (1) TheLaunchedEffectkeys only watchedmessages.sizeandlastMessage?.content?.length, so growth of the reasoning block (thinkingContent) and tool-card additions (toolCalls) silently froze auto-follow during long thinking and tool execution phases. (2)animateScrollToItem(messages.size - 1)defaulted toscrollOffset = 0, which aligns the top of the item with the top of the viewport — for tall streaming bubbles this snapped the user back to the start of the message instead of staying with the latest token. (3) There was no "user is reading history" gate, so any delta would yank a user reading scrollback back to the bottom. (4) TheisStreamingflag was a snapshot key, so the stream-complete transition (true → false) re-triggeredanimateScrollToItemeven when no content actually changed — producing a visible jiggle. (5) Sessions endpoint reloads the entire message list on stream complete vialoadMessageHistory(), and the resultinganimateItem()placement animations on every bubble fought with our concurrentanimateScrollToItem— producing a flash where the viewport visibly settled twice. - The fix — Added a
ChatScrollSnapshotdata class that captures all five streaming-state fields (message count, content length, thinking length, tool-call count, isStreaming). AsnapshotFlowover the snapshot, debounced withdistinctUntilChanged, drives auto-scroll viacollectLatest(which cancels in-flight animations when newer deltas arrive — prevents pile-ups during rapid SSE bursts). The scroll target is(totalItemsCount - 1, scrollOffset = Int.MAX_VALUE)so Compose pins the absolute end of the list to the bottom of the viewport regardless of how tall the streaming bubble has grown. AuserScrolledAwayflag, tracked via a separatesnapshotFlowonlistState.isScrollInProgress to isAtBottom, gates the auto-follow — scrolling up to read history pauses it; scrolling back to the bottom (or tapping the FAB) resumes it. The FAB now uses!listState.canScrollForwardfor itsisAtBottomderivation (replaces the off-by-onelastVisible < messages.size - 2arithmetic) and clearsuserScrolledAwayon tap so live-follow resumes immediately even before the scroll animation settles. ThecollectLatestlambda also tracks the previous snapshot in a coroutine-scoped var, which lets it (a) skip "state-only" deltas where only theisStreamingflag changed (the viewport was already correct from the last content delta — re-animating causes a jiggle on stream complete) and (b) detect "list rebuild" deltas (sessions-modeloadMessageHistorycollapsing one streaming message into multiple final messages with proper boundaries) and use the instantscrollToItempath instead ofanimateScrollToItemso the items can run theiranimateItem()placement animations without competing with our scroll animation. - Compose deprecation cleanup — Migrated
Icons.Filled.Chat→Icons.AutoMirrored.Filled.ChatinRelayApp.kt(auto-mirrors for RTL locales). MigratedLocalClipboardManager→LocalClipboard(suspend-basedClipboard.setClipEntry(ClipEntry)API) in bothChatScreen.ktandSettingsScreen.kt. Both clipboard call sites now wrap in the existingrememberCoroutineScope().launch {}since the new API is suspend; the underlying clipboard write now runs off the UI thread, which is a small responsiveness win on lower-end devices. Removed the now-unusedAnnotatedStringimports from both files. Build is now warning-clean. - Settings toggle — Added
smoothAutoScrollboolean preference toConnectionViewModel(DataStore keysmooth_auto_scroll, defaulttrue), mirroring the existinganimationEnabled/animationBehindChatpattern. New row in Settings > Chat under "Show reasoning". When disabled, the entire auto-scrollLaunchedEffectearly-returns and the chat is fully manual. - Docs — Updated
README.mdfeatures list,docs/spec.mdSettings Tab section, anduser-docs/guide/chat.md(new "Smooth Auto-Scroll" subsection explaining the pause-on-scroll-up behavior). - Brand rename — "Hermes Relay" → "Hermes-Relay" across 57 files (docs, app strings, scripts, workflows, plugin, relay server). Aligns the display name with the canonical repo slug. The Android
app_nameinstrings.xmlis nowHermes-Relay; PascalCase code identifiers (HermesRelayApp,HermesRelayTheme, logcat tagHermesRelay) were intentionally left alone since they're internal symbols, not user-facing text. - Docs landing redesign — Restructured
user-docs/index.mdto put install above features. CreatedInstallSection.vue(mounted via VitePresshome-hero-afterslot) andHeroDemo.vue(mounted viahome-hero-imageslot). Hero is now a phone-framed<video>that autoplays the demo, then crossfades to the brand logo after 12s for the rest of the session via Vue<Transition mode="out-in">— bandwidth fetches stop when the video unmounts. Fixed two raw-HTML/guide/getting-startedhrefs that VitePress base-rewriting silently skipped (they were 404ing on the deployed site under the/hermes-relay/base path). - Demo video pipeline — Re-encoded source
chat_demo.mp4from 20.5 MB / 102 fps / 1080×2340 down to 1.95 MB / 30 fps / 720×1560 via ffmpeg (-vf scale=720:-2,fps=30 -crf 28 -preset slow -an -movflags +faststart) — 90% size reduction, mostly from dropping the wildly oversampled framerate. Extracted poster JPEG from first frame at 0.5s for instant LCP. Embedded with autoplay+muted+playsinline+preload=metadata in the docs hero, withcontrolsin Getting Started's "Verify Connection" section, and as a<video>tag in README pointing at the GitHub raw URL. Used portableimageio-ffmpegPython package since system ffmpeg wasn't installed. - Release infrastructure hardening —
.github/workflows/release.ymlnow builds APK + AAB in one Gradle invocation, decodes aHERMES_KEYSTORE_BASE64GitHub Secret to$RUNNER_TEMP/release.keystore, signs with the release keystore when secrets are set, falls back to debug signing with a warning banner in$GITHUB_STEP_SUMMARYwhen they're not. Generates SHA256 checksums for both artifacts and attaches all three (APK, AAB, SHA256SUMS.txt) to the GitHub Release. - gradle-play-publisher plugin — Added
com.github.triplet.playv3.13.0 (latest stable compatible with AGP 8.13.2 — v4.0.0 requires AGP 9). Configured inapp/build.gradle.ktswithtrack=internal,releaseStatus=DRAFT,defaultToAppBundles=true, reading credentials from<repo-root>/play-service-account.json(gitignored). Plugin is fully optional —assembleRelease/bundleReleasework without the JSON; only the explicitpublishReleaseBundle/promoteReleaseArtifacttasks require it. Verifiedsettings.gradle.ktsalready hasgradlePluginPortal()inpluginManagement. - RELEASE.md — New canonical doc (312 lines) covering: SemVer versioning conventions with optional
-alpha/-beta/-rc.Nprereleases and monotonicappVersionCode; one-time setup (keystore generation viakeytool,local.propertiesconfig, base64 encoding for CI, Play Console account + 14-day closed testing rule for new personal accounts, Play Developer API service account creation in Google Cloud Console, GitHub Actions secrets); the 7-step release process (bump → notes → build → verify → tag/push → upload → post-release); CI behavior; hotfix recipe; troubleshooting.CLAUDE.md"Dev Workflow" section now references it.
Files changed:
app/src/main/kotlin/com/hermesandroid/relay/viewmodel/ConnectionViewModel.kt(preference key + StateFlow + setter)app/src/main/kotlin/com/hermesandroid/relay/ui/screens/SettingsScreen.kt(new toggle row in Chat section + LocalClipboard migration)app/src/main/kotlin/com/hermesandroid/relay/ui/screens/ChatScreen.kt(data class, scroll-tracking effect, rewritten auto-scroll effect with previous-snapshot diffing, FAB fix, LocalClipboard migration)app/src/main/kotlin/com/hermesandroid/relay/ui/RelayApp.kt(Icons.AutoMirrored.Filled.Chat)README.md,docs/spec.md,user-docs/guide/chat.md,DEVLOG.md- Rebrand — 57 files across
app/src/main/res/,docs/,user-docs/,scripts/,plugin/,relay_server/,skills/,.github/workflows/ - Docs landing —
user-docs/index.md(frontmatter trim),user-docs/.vitepress/theme/index.ts(slot wiring + dead-link fix), newuser-docs/.vitepress/theme/components/{InstallSection,HeroDemo}.vue - Demo video —
assets/chat_demo.mp4+assets/chat_demo_poster.jpg(canonical),user-docs/public/chat_demo.mp4+user-docs/public/chat_demo_poster.jpg(VitePress-served copies),README.md(<video>tag),user-docs/guide/getting-started.md(full video in Verify Connection section) - Release infra —
.github/workflows/release.yml(AAB build + keystore decode + checksums + summary),gradle/libs.versions.toml(play-publisher = 3.13.0),app/build.gradle.kts(alias(libs.plugins.play.publisher)+play { }block),.gitignore(play-service-account.json + keystore.properties) - Release docs — new
RELEASE.md(312 lines),CLAUDE.md(Release Process subsection)
Next:
- Test on device with a long streaming response (verify auto-follow during reasoning + tool cards + text deltas)
- Test the pause-on-scroll-up gesture and the FAB resume
- Verify the disabled-pref path leaves the chat fully manual
Blockers:
- None —
compileDebugKotlinpasses clean (only pre-existingLocalClipboardManagerdeprecation warnings unrelated to this change)
2026-04-09 — Play Store Prep, Plugin CLI Migration, One-Line Install
Done:
- Play Store listing prep (v0.1.0) — 512x512 hi-res icon + 1024x500 feature graphic rendered from
assets/logo.svgviascripts/gen-store-assets.mjs(pure-Node,@resvg/resvg-js). Feature graphic shows centered logo + three channel labels (Chat, Terminal, Bridge). Listing doc atdocs/play-store-listing.mdwith short/full descriptions, release notes, and category. Interactivescripts/screenshots.batTUI for capturing clean device screenshots with Android demo mode enabled. - Privacy policy page —
user-docs/privacy.mdpublished at/privacyfor Google Play's required URL. Formal language (effective date, COPPA disclosure, contact, change policy). Added Privacy link to top nav. - Plugin CLI migration —
hermes-pair→hermes pair— Rewrote the standalone bash script as a Python module (plugin/pair.py) withplugin/cli.pyregisteringhermes pairvia the v0.8.0register_cli(subparser)convention. Pure-Python QR rendering viasegno(noqrencodebinary). Always shows connection details as plain text alongside the QR, sohermes pairworks inside Hermes Rich TUI and over limited SSH sessions. Cross-platform LAN IP detection via socket trick (replaces Linux-onlyip route). Config fallback chain:config.yaml→~/.hermes/.env→ env vars → defaults. Wrappedctx.register_cli_command()call in try/except so 14android_*tools still register cleanly on hermes-agent v0.7.0. - One-line server install —
install.shat repo root:curl -fsSL .../install.sh | bashclones plugin, installs Python deps (requests,aiohttp,segno), and prints next steps. Deleted the oldplugin/install.shsince its own URL comment pointed to the root. SupportsHERMES_HOMEandHERMES_RELAY_BRANCHenv overrides. - README + docs restructure — Replaced README
## Installwith## Quick Startleading with thecurl | bashone-liner. Homepage (user-docs/index.md) gets a custom "Install in 30 seconds" block below feature cards. Guide landing page (user-docs/guide/index.md) leads with the same one-liner. Getting Started page restructured into 3-step Quick Start. VitePress copy buttons on code blocks already in place. - Tool annotations default OFF — The chat parse tool annotations feature was causing messages to wait for stream end before displaying. Defaulted to
falseinChatHandler.ktandConnectionViewModel.kt; subtitle now warns about the streaming delay behavior. - Deprecated the standalone skill —
skills/hermes-pairing-qr/SKILL.mdandhermes-pairscript keep working for v0.7.0 users but print deprecation warnings pointing to the plugin.
Files changed:
plugin/pair.py(new),plugin/cli.py(new),plugin/__init__.py(wire CLI registration),plugin/setup.py/pyproject.toml/requirements.txt/plugin.yaml(bumped to 0.3.0 + segno)install.sh(new at root, replacesplugin/install.sh)docs/play-store-listing.md(new),assets/play-store-icon-512.png(new),assets/play-store-feature-1024x500.png(new),scripts/gen-store-assets.mjs(new),scripts/screenshots.bat(new)user-docs/privacy.md(new),user-docs/.vitepress/config.mts(nav link),user-docs/index.md(install section),user-docs/guide/index.md(Quick Install),user-docs/guide/getting-started.md(restructured)README.md(Quick Start up top),CLAUDE.md(key files updated)skills/hermes-pairing-qr/SKILL.md+hermes-pair(deprecation notices)
Next:
- Test
hermes pairagainst local hermes-agent v0.8.0 once available - Finalize Play Store screenshots (curate ones without real server IPs/keys)
- Submit v0.1.0 to Google Play Console
Blockers:
- hermes-agent v0.8.0 not yet released at time of writing — the
register_cli_commandplumbing exists in v0.7.0 but general plugin CLI commands aren't wired into the main argparse yet. Plugin still installs and registers tools fine on v0.7.0; users fall back to the deprecated standalone script for pairing until v0.8.0 ships.
2026-04-08 — Tool Call Reload on Stream Complete, Keyboard Gap Fix, Placeholder Dedup
Done:
- Tool call reload on stream complete — Sessions endpoint doesn't emit structured tool events during streaming; tool calls only exist as
tool_callsJSON on stored server messages. Added server history reload inonCompleteCbwhen using sessions mode — replaces the single streaming message with the server's authoritative multi-message structure (proper message boundaries + tool call cards). Queue drain deferred until reload completes. - Annotation finalization pass — Added
finalizeAnnotations()as a post-stream reconciliation hook inonStreamComplete()andonTurnComplete(). Re-scans final message content for surviving annotation text, strips it, creates missingToolCallobjects, and marks incomplete annotation tools as completed. Safety net for servers that emit inline annotation markers. - Placeholder message dedup — When
message.startedfires with a server-assigned ID, the empty placeholder message's ID is now replaced viareplaceMessageId()(only acts on empty+streaming messages). Prevents orphan placeholder bubbles showing duplicate streaming dots alongside the real message. - Keyboard gap fix — Bottom navigation bar now hides when keyboard is visible (
WindowInsets.ime.getBottom > 0). Eliminates the gap between input bar and keyboard caused byinnerPadding(bottom nav height) stacking withimePadding()(keyboard height).
Files changed:
network/handlers/ChatHandler.kt— AddedreplaceMessageId(),finalizeAnnotations(),matchAnnotationToolName(). Wired finalization intoonStreamComplete()andonTurnComplete().viewmodel/ChatViewModel.kt—onMessageStartedCbcallsreplaceMessageId().onCompleteCbreloads session history for sessions mode.ui/RelayApp.kt— Bottom nav hidden when keyboard visible via IME insets check.
2026-04-07 — ASCII Morphing Sphere, Ambient Mode, Animation Settings, Polish Fixes
Done:
- ASCII morphing sphere — animated visualization on the empty chat screen, inspired by AMP Code CLI. Pure Compose Canvas rendering (no OpenGL). Characters
. : - = + * # % @form a sphere shape with 3D lighting. Color pulses green to purple. Contained in square aspect ratio box above "Start a conversation" text. - Ambient mode — toggle button (AutoAwesome icon) in chat header bar hides messages and shows the sphere fullscreen. Tap the ChatBubble icon to return to chat.
- Animation behind messages — sphere renders at 15% opacity behind the chat message list as a subtle ambient background. Toggleable in Settings.
- Animation settings — new section in Settings under Appearance: "ASCII sphere" toggle (on by default), "Behind messages" toggle (on by default, disabled when animation is off).
- Parse tool annotations — marked as "Experimental" badge, disabled/dimmed when streaming endpoint is "Runs" mode (only relevant for Sessions mode).
- Empty bubble fix — messages with blank content and no tool calls are now hidden from chat.
- App icon fix — adaptive icon foreground scaled to 75% via
<group>transform for proper safe zone padding. - Dev scripts — added
release,bundle,versioncommands toscripts/dev.bat. - MCP tooling — android-tools-mcp v0.1.1 (IDE/build layer) + mobile-mcp (device/runtime layer) configured as companion MCP servers. Full reference in
docs/mcp-tooling.md. - Audit fixes — MIT LICENSE added, orphaned
companion/andcompanion_relay/removed,FOREGROUND_SERVICEpermission removed, CHANGELOG URLs fixed, version refs updated to 0.1.0, .gitignore updated, plugin refs updated from raulvidis to Codename-11.
New files:
ui/components/MorphingSphere.kt— ASCII morphing sphere composable (Canvas-based, 3D lighting, color pulse)
Files changed:
ui/screens/ChatScreen.kt— Empty state sphere, ambient mode toggle, behind-messages background layerui/screens/SettingsScreen.kt— Animation settings section (ASCII sphere toggle, behind messages toggle), parse tool annotations experimental badgeviewmodel/ConnectionViewModel.kt— Animation preference DataStore keys/flowsapp/build.gradle.kts— Version and build config updatesres/mipmap-anydpi-v26/ic_launcher.xml— 75% scale group transform on foregroundscripts/dev.bat— Added release, bundle, version commands
2026-04-07 — Token Tracking Fix, Stats Enhancements, Keyboard Gap, Configurable Limits
Done:
- Token tracking fix — Root cause: OpenAI-format SSE events (e.g.
chat.completion.chunk) have notype/eventfield, so theval eventType = type ?: event.resolvedType ?: returnline exited before usage was ever checked. Moved usage extraction before the type resolution in bothsendChatStream()andsendRunStream(). Also addedprompt_tokens/completion_tokens(OpenAI naming) support inUsageInfoviaresolvedInputTokens/resolvedOutputTokenshelper properties. - Stats for Nerds enhancements — Reset button with confirmation dialog, tokens per message average in summary line (
~Xk/msg), peak TTFT and slowest completion times (tertiary color),formatMsWithSeconds()helper shows1234ms (1.2s)for all time displays >= 1s. - Configurable limits — Expandable "Limits" section in Chat settings with segmented button rows for max attachment size (1/5/10/25/50 MB, default 10) and max message length (1K/2K/4K/8K/16K chars, default 4K). Persisted to DataStore, read reactively in ChatScreen.
- Keyboard gap fix — Set
contentWindowInsets = WindowInsets(0)on the Scaffold in RelayApp.kt. The Scaffold was adding system bar padding toinnerPaddingthat stacked with ChatScreen'simePadding(), causing a visible gap between input bar and keyboard.
Files changed:
network/HermesApiClient.kt— Usage check moved before eventType resolution in both streaming methodsnetwork/models/SessionModels.kt—UsageInfonow acceptsprompt_tokens/completion_tokens, addedresolvedInputTokens/resolvedOutputTokens/resolvedTotalTokensviewmodel/ChatViewModel.kt— Usesusage.resolvedInputTokensetcviewmodel/ConnectionViewModel.kt—maxAttachmentMb+maxMessageLengthDataStore keys/flows/settersui/components/StatsForNerds.kt— Reset button+dialog, tokens/msg, peak/slowest times,formatMsWithSeconds()ui/screens/SettingsScreen.kt— Expandable "Limits" section with segmented buttonsui/screens/ChatScreen.kt— ReadscharLimit/maxAttachmentMbfrom settingsui/RelayApp.kt—contentWindowInsets = WindowInsets(0)on Scaffold
2026-04-07 — File Attachments
Done:
- Generic file attachments — users can attach any file type via
+button in the input bar. Uses AndroidOpenMultipleDocumentspicker (accepts*/*). Files base64-encoded and sent in the Hermes APIattachmentsarray ({contentType, content}). - Attachment preview strip — horizontal scrollable row above input bar showing pending attachments. Image attachments show decoded thumbnails, other files show document icon + filename + size. Each attachment has a remove (X) button.
- Attachment rendering in bubbles — user messages display attached images inline (decoded from base64), non-image attachments show as file badge with name. Forward-compatible with agent-sent images.
- 10 MB file size limit — enforced client-side with toast warning.
- Send with attachments only — send button enabled when attachments are present even without text. Sends
[attachment]as placeholder text. - API integration —
attachmentsparameter added to bothsendChatStream()andsendRunStream()in HermesApiClient. Serialized as JSON array matching Hermes WebAPI spec. - Message history support —
MessageItem.imageUrlsextractsimage_urlcontent blocks from OpenAI-format content arrays for future server-side image rendering.
Files changed:
data/ChatMessage.kt— AddedAttachmentdata class,attachmentsfield onChatMessagenetwork/models/SessionModels.kt— AddedimageUrlsproperty toMessageItemnetwork/HermesApiClient.kt—attachmentsparam onsendChatStream()+sendRunStream(), JSON array serializationviewmodel/ChatViewModel.kt—_pendingAttachmentsStateFlow, add/remove/clear, snapshot-and-clear on send, pass through to APIui/screens/ChatScreen.kt—+button,OpenMultipleDocumentspicker, attachment preview strip,formatFileSize()helperui/components/MessageBubble.kt— Inline image rendering (base64 decode), file badge for non-images
2026-04-07 — Client-Side Message Queuing
Done:
- Message queuing — Users can now send messages while the agent is streaming. Messages are queued locally and auto-sent when the current stream completes. Queue drains one at a time, maintaining proper ordering.
- Input bar redesign — During streaming, both Stop and Send buttons are visible side by side. Send button uses
tertiarycolor during streaming to indicate "queue" mode. Placeholder changes to "Queue a message..." when streaming. - Queue indicator — Animated bar above the input field shows queued message count ("1 message queued" / "3 messages queued") with a Clear button to discard the queue. Uses
AnimatedVisibilityfor smooth entrance/exit. - Queue lifecycle — Queue is cleared on stream cancellation (Stop button) and on stream error, preventing stale messages from auto-sending after failures.
Design decisions:
- Client-side queuing (not server-side
/queuecommand) because the Hermes HTTP API doesn't support concurrent SSE streams to the same session. The gateway's/queueis a CLI-level feature, not an HTTP endpoint. - Queue drains automatically — no manual "send next" required. Provides a seamless conversation flow.
- No purple glow on Send button during streaming — visual distinction between "send now" and "queue for later".
Files changed:
viewmodel/ChatViewModel.kt—_queuedMessagesStateFlow,sendMessage()queues during streaming,sendMessageInternal()extracted,drainQueue()on complete,clearQueue(), queue cleared on error/cancelui/screens/ChatScreen.kt— Queue indicator row, input bar with both Stop+Send buttons, tertiary send tint during streaming, "Queue a message..." placeholder
2026-04-07 — Feature Gating, MCP Tooling, v0.1.0 Release Prep
Done:
- Feature gating system —
FeatureFlags.ktsingleton with compile-time defaults (BuildConfig.DEV_MODE) and runtime DataStore overrides. Debug builds have all features unlocked; release builds gate experimental features behind Developer Options. - Developer Options — Hidden settings section activated by tapping version number 7 times (same UX as Android system Developer Options). Contains relay features toggle and lock button. Uses
tertiarycolor scheme for visual distinction. - Gated relay/pairing settings — Relay Server and Pairing sections in Settings hidden by default in release builds. Only visible when relay feature flag is enabled via Developer Options.
- Gated onboarding pages — Terminal, Bridge, and Relay pages dynamically excluded from onboarding flow when relay feature is disabled. Page count and indices adjust automatically.
- Version bump —
0.1.0-beta→0.1.0for Google Play submission. - BuildConfig.DEV_MODE —
truefor debug,falsefor release. Used by FeatureFlags as compile-time default. - android-tools-mcp v0.1.1 — Fixed MCP server path, built plugin from fork, committed wrapper jar fix, repo cleanup (fork attribution, VM option name fix, cross-platform release script), released to GitHub.
- mobile-mcp added — Added
mobile-next/mobile-mcpas companion MCP server for device/runtime testing (tap, swipe, screenshot, app management). Configured with telemetry disabled. - MCP tooling docs — Created
docs/mcp-tooling.mdwith full reference for both MCP servers (setup, prerequisites, 40 tools listed, when-to-use guide, overlap analysis).
New files:
data/FeatureFlags.kt— Feature flag singletondocs/mcp-tooling.md— MCP tooling reference
Files changed:
app/build.gradle.kts— AddedDEV_MODEBuildConfig field,buildConfig = truegradle/libs.versions.toml— Version0.1.0ui/screens/SettingsScreen.kt— Feature-gated relay/pairing, added Developer Options section with tap-to-unlockui/onboarding/OnboardingScreen.kt— Dynamic page list based on feature flagsCLAUDE.md— Updated current state, added FeatureFlags to key files, MCP tooling section, related projects
Next:
- Build release APK and submit to Google Play (closed testing track)
- Test feature gating on release build (relay settings hidden, dev options tap unlock)
- Phase 2: Terminal channel
- Phase 3: Bridge channel
2026-04-07 — Session Management Audit, Play Store Release Prep
Done:
- Session management audit — Full review of session CRUD, persistence, capability detection, error handling. Implementation is complete and solid against upstream Hermes API (both
/api/sessionsnon-standard and/v1/runsstandard endpoints). - Fixed SessionDrawer highlight bug —
backgroundColorvariable was computed but never applied to the Row modifier. Active sessions now properly highlighted withsecondaryContainer. - Added Privacy Policy link — New "Privacy Policy" button in Settings → About, linking to GitHub-hosted
docs/privacy.md. Required for Google Play Store submission. - Fixed privacy.md inaccuracies — Added CAMERA permission to the permissions table (used for QR scanning, declared
required="false"). Corrected network security description to accurately reflect cleartext policy. - Fixed RELEASE_NOTES.md URL — Changed generic
user/hermes-androidto actualCodename-11/hermes-android. - Improved network_security_config.xml docs — Expanded comment explaining why cleartext is globally permitted (Android doesn't support IP range restrictions, users connect to arbitrary LAN IPs) and how security is enforced at the application layer (insecure mode toggle + warning badge).
Session management features confirmed working:
- List/create/switch/rename/delete sessions with optimistic updates + rollback
- Message history loading with tool call reconstruction
- Auto-session creation on first message send with auto-title
- Session ID persistence via DataStore across app restarts
- Capability detection (
detectChatMode()) with graceful degradation - Both Sessions and Runs streaming endpoints
Play Store readiness:
- Signing config loads from env vars / local.properties ✅
- ProGuard rules comprehensive ✅
- Release workflow with version validation ✅
- Privacy policy link in app ✅
- Network security documented ✅
- No hardcoded debug flags in release ✅
- Version:
0.1.0-beta(versionCode 1) — ready for open testing track
Files changed:
ui/components/SessionDrawer.kt— Addedbackgroundimport + appliedbackgroundColorto Rowui/screens/SettingsScreen.kt— Added Shield icon import + Privacy Policy button in About carddocs/privacy.md— Added CAMERA permission, fixed network security descriptionRELEASE_NOTES.md— Fixed issues URLres/xml/network_security_config.xml— Expanded documentation comment
2026-04-07 — Chat UI Polish, Annotation Stripping, Reasoning Extraction
Done:
- Scroll-to-bottom FAB — SmallFloatingActionButton appears when scrolled up from bottom. Animated fade + slide. Haptic on click. Positioned bottom-end of message area.
- Message entrance animations —
animateItem()on all LazyColumn items (messages, spacers, streaming dots). Smooth fade + slide when items appear/reorder. - Date separators — "Today", "Yesterday", or "EEE, MMM d" chips between messages from different calendar days. Subtle surfaceVariant pill style.
- Message grouping — Consecutive same-sender messages have tighter spacing (2dp base + 1dp vs 6dp padding), suppressed agent name on non-first messages, grouped bubble corner shapes (flat edges where messages meet).
- Pre-first-token indicator — Placeholder assistant message with streaming dots appears immediately after send, before any SSE delta. Fills naturally when first delta arrives.
- Copy feedback toast — Snackbar "Copied to clipboard" on long-press copy. Previously only haptic with no visual confirmation.
- Annotation stripping — When the tool annotation parser matches inline text (
`💻 terminal`), it now strips that text from the message content. Previously the raw annotation text remained visible alongside the ToolCall card. - Inline reasoning extraction —
<think>/<thinking>tags in assistant text are detected and redirected tothinkingContentfor the ThinkingBlock. Handles tags split across streaming deltas. Resets on stream complete.
Files changed:
ui/screens/ChatScreen.kt— FAB, date separators, grouping, snackbar, animation modifiers, Box wrapper for message areaui/components/MessageBubble.kt—isFirstInGroup/isLastInGroupparams, grouped bubble shapes, conditional agent namenetwork/handlers/ChatHandler.kt—addPlaceholderMessage(),stripLineFromContent(),processInlineReasoning(), thinking tag parser,parseAnnotationLinereturns Booleanviewmodel/ChatViewModel.kt— placeholder message before stream start
Note: Code block copy button already existed (MarkdownContent.kt → CodeBlockWithCopyButton).
2026-04-07 — Tool Call Rendering Fix, Runs API, SSE Architecture Correction
Done:
- Fixed premature stream completion —
assistant.completedwas callingonComplete(), terminating the stream before tool events arrived in multi-turn agent loops. Now onlyrun.completed/doneend the stream.assistant.completedcalls newonTurnComplete()which marks one message done without stopping the stream. - Added
message.startedhandling — Server-assigned message IDs now tracked viaonMessageStartedcallback. Enables proper multi-turn message tracking (each assistant turn gets its own message). - Dynamic message ID tracking —
ChatViewModel.startStream()usescurrentMessageIdvariable that updates when the server sends new message IDs, instead of hardcoding one UUID for the whole stream. - Rewrote tool annotation parser — Regex patterns now match actual Hermes format:
`💻 terminal`(any emoji + tool name in backticks). Uses state tracking: first occurrence = start, second = complete. Also handles explicit completion/failure emojis (✅/❌) and verbose format (🔧 Running: tool_name). - Fixed message history tool calls —
loadMessageHistory()now reconstructsToolCallobjects from assistant messages'tool_callsfield and matches tool results fromrole:"tool"messages. Previously skipped all tool data. - Runs API event coverage — Added
message.delta,reasoning.available,run.failedevent handling. UpdatedHermesSseEventmodel witheventfield (alias fortype),toolfield (Runs API format),duration,output,text,timestamp. AddedresolvedTypeandresolvedToolNamehelpers. - SSE debug logging — All events logged with
HermesApiClienttag. Filter withadb logcat -s HermesApiClientto see what the server actually sends. - Updated decisions.md — Documented the two streaming endpoints (Sessions vs Runs), tool call transparency differences, upstream API notes.
- Updated settings description — Streaming endpoint toggle now explains the difference.
Architecture correction (from upstream research):
/api/sessionsCRUD endpoints are NOT in upstream hermes-agent source. They may be version-specific (v0.7.0). Standard endpoints are/v1/chat/completions,/v1/responses,/v1/runs./v1/chat/completionsstreaming embeds tool calls as inline markdown text (`💻 terminal`), NOT as separate SSE events. The annotation parser is the primary detection path./v1/runs+/v1/runs/{run_id}/eventsprovides structured lifecycle events with realtool.started/tool.completed— this is the correct endpoint for rich tool display.- Hermes has no "channels" API (Discord/Telegram-style). The
channel_directory.pyis for cross-platform message routing, not a chat API.
Files changed:
network/HermesApiClient.kt— new callbacks, fixed completion flow, debug loggingnetwork/handlers/ChatHandler.kt—onTurnComplete(), annotation rewrite, history tool callsnetwork/models/SessionModels.kt— new fields for Runs API compatibilityviewmodel/ChatViewModel.kt— dynamic message ID tracking, new callback wiringui/screens/SettingsScreen.kt— updated endpoint toggle descriptiondocs/decisions.md— corrected API architecture documentation
Next:
- Deploy to device and test tool call rendering with
adb logcat -s HermesApiClient - Test with both "Sessions" and "Runs" endpoint modes
- Verify annotation parser matches actual Hermes verbose output
- If Runs API works well, consider making it the default endpoint
Blockers:
- Need a running hermes-agent server with tools configured to validate tool event flow end-to-end
2026-04-06 — Personality System, Command Palette, QR Pairing, Chat Header
Done:
- Personality system fix —
getProfiles()was reading wrong JSON path and returning empty list. Replaced withgetPersonalities()readingconfig.agent.personalities+config.display.personality. Server default personality shown first in picker. Switching sends personality's system prompt viasystem_message(previousprofilefield was ignored by server). - Agent name on chat bubbles — Added
agentNamefield toChatMessage. Active personality name displayed above assistant messages. - Chat header redesign — Messaging-app style: avatar circle with initial letter +
ConnectionStatusBadgepulse overlay, agent name (titleMedium), model name subtitle from/api/config. - Command palette — Searchable bottom sheet with category filter chips (2-row limit, expandable), 29 gateway built-in commands + dynamic personality commands + 90+ server skills from
GET /api/skills./button on input bar opens palette. - Inline autocomplete improved — Extracted to
InlineAutocompletecomponent withLazyColumn, 2-line descriptions, up to 8 results. - QR code pairing — ML Kit barcode scanner + CameraX. Detects
{"hermes":1,...}payload, auto-fills server URL + API key, triggers connection test. Available in Settings and Onboarding. hermes-pairskill — Added toskills/hermes-pairing-qr/for users to install on their server. Generator script + SKILL.md.- ConnectionStatusBadge — Reusable animated status indicator with pulse ring (green connected, amber connecting, red disconnected). Wired into Settings, Onboarding, and chat header.
- Relay server docs —
docs/relay-server.md,relay_server/Dockerfile,relay_server/hermes-relay.service,relay_server/SKILL.md. - Upstream contributions doc —
docs/upstream-contributions.md— proposedGET /api/commands,personalityparameter, terminal HTTP API.
Corrections to previous session:
- "Server profile picker" was actually fetching from wrong path — now correctly reads
config.agent.personalities - "Sends
profilefield" — server ignores this; now sendssystem_messagewith personality prompt - "13 personality commands" were hardcoded — now generated dynamically from server config
- ProfilePicker renamed to PersonalityPicker
2026-04-06 — v0.1.0-beta Polish, Profiles, Analytics, Splash
Done:
- Package rename —
com.hermesandroid.companion→com.hermesandroid.relay. All files moved, manifest updated, app name changed to "Hermes-Relay". - Server profile picker — Replaced hardcoded 8-personality system with dynamic server profiles fetched from
GET /api/config. ProfilePicker in top bar shows Default + server-configured profiles. Sendsprofilefield in chat requests. - Personality switching — 13 built-in Hermes personalities available via
/personality <name>slash commands (server-side, session-level switching). - Slash command autocomplete — Type
/in chat input to see built-in commands (/help,/verbose,/clear,/status) + 13 personality commands + dynamically fetched server skills viaGET /api/skills. Filterable dropdown overlay. - In-app analytics (Stats for Nerds) —
AppAnalyticssingleton tracking response times (TTFT, completion), token usage, health check latency, stream success rates. Canvas bar charts in Settings with purple gradient. Accessible via Settings > Chat > Stats for Nerds. - Tool call display config — Off/Compact/Detailed modes in Settings.
CompactToolCallinline component for compact mode.ToolProgressCardauto-expands while tool is running, auto-collapses on complete. - App context prompt — Toggleable system message telling the agent the user is on mobile. Enabled by default in Settings > Chat.
- Animated splash screen —
AnimatedVectorDrawablewith scale + overshoot + fade animation. Icon background color matches theme. Hold-while-loading (stays until DataStore ready). Smooth fade-out exit transition. Separatesplash_icon.xmlat 0.9x scale. - Chat empty state — Logo + "Start a conversation" + suggestion chips that populate input.
- Animated streaming dots — Replaces static "streaming..." text with pulsing 3-dot animation.
- Haptic feedback — On send, copy, stream complete, error.
- About section redesign — Logo on dark background, dynamic version from
BuildConfig, Source + Docs link buttons, credits line. - Hermes docs links — In onboarding welcome page, API key help dialog, and Settings About section.
- Release signing config — Environment variables +
local.propertiesfallback with graceful debug-signing fallback. - Centralized versioning —
libs.versions.tomlas single source of truth (appVersionName,appVersionCode). - Logo fix — Removed vertical H bars from ghost layer, now matches actual SVG (V-crossbar + diagonal feathers only).
- SSE debug logging — Unhandled event types now logged for diagnostics.
- Release infrastructure (from ARC patterns) — 3-job release workflow (validate → CI → release) reading from
libs.versions.toml. Claude automation workflows (issue triage, fix, code review). Dependabot auto-merge. CHANGELOG.md + RELEASE_NOTES.md for v0.1.0-beta. Updated PR template with Android checklist.
New files:
data/AppAnalytics.kt— In-app analytics singletonui/components/StatsForNerds.kt— Canvas bar charts for analyticsui/components/CompactToolCall.kt— Inline compact tool call displaynetwork/models/SessionModels.kt— Session, message, SSE event modelsres/drawable/splash_icon.xml— Static splash icon (0.9x scale)res/drawable/splash_icon_animated.xml— Animated splash vectorres/animator/— Splash animation resources.github/workflows/claude.yml— Claude automation.github/workflows/claude-code-review.yml— Claude code review.github/workflows/dependabot-auto-merge.yml— Dependabot auto-merge
Next:
- Build and test against running Hermes API server
- Test on emulator and physical device (S25 Ultra)
- Set up keystore/signing secrets for release CI
- Deploy docs site (GitHub Pages or similar)
- Phase 2: Terminal channel (xterm.js in WebView, tmux integration)
- Phase 3: Bridge channel migration
Blockers:
- None — ready for on-device testing
2026-04-05 — Project Scaffolding
Done:
- Wrote spec (docs/spec.md) and decisions (docs/decisions.md)
- Created CLAUDE.md handoff for agent development
- Created DEVLOG.md
2026-04-05 — MVP Phase 0 + Phase 1 Implementation
Done:
- Created Android app — full Jetpack Compose project (30+ files, 2500+ lines)
- Bottom nav scaffold with 4 tabs (Chat, Terminal, Bridge, Settings)
- WSS connection manager (OkHttp, auto-reconnect with exponential backoff)
- Channel multiplexer (typed envelope protocol)
- Auth flow (6-char pairing code → session token in EncryptedSharedPreferences)
- Chat UI: message bubbles, streaming text, tool progress cards, profile selector
- Settings UI: server URL, connection status, theme selector
- Material 3 + Material You dynamic theming
- @Preview composables for MessageBubble and ToolProgressCard
- Terminal and Bridge tabs stubbed for Phase 2/3
- Created relay server — Python aiohttp WSS server (10 files, 1500+ lines)
- WSS server on port 8767 with health check
- Auth: pairing codes (10min expiry), session tokens (30-day expiry), rate limiting
- Chat channel: proxies to Hermes WebAPI SSE, re-emits as WS envelopes
- Terminal/bridge channel stubs
- Runnable via
python -m relay_server
- Created CI/CD — GitHub Actions
- CI: lint → build (debug APK) → test, relay syntax check
- Release: tag-triggered, version validation, signed APK → GitHub Release
2026-04-05 — Repo Restructure + Build Fixes
Done:
- Promoted Android project to repo root (Android Studio opens root directly)
- Removed
hermes-android-bridge/(upstream absorbed into Compose rewrite) - Renamed
hermes-android-plugin/→plugin/ - Moved root
tools/,skills/,tests/intoplugin/ - Resolved build issues: gradle.properties, launcher icons, SDK path, compileSdk 36
- Pinned AGP 8.13.2 + Gradle 8.13 + JVM toolchain 17
- Added dev scripts (
scripts/dev.sh+scripts/dev.bat) - Updated all docs (README, CLAUDE.md, AGENTS.md, DEVLOG.md, spec directory structure)
- Build verified: debug APK builds successfully
Current state:
- Phase 0 + Phase 1 complete
- Android Studio opens and syncs from repo root
- Debug APK builds and deploys to emulator/device
- Dev scripts ready for build/install/run/test/relay workflows
2026-04-05 — Direct API Chat Refactor
Done:
- Refactored chat to connect directly to Hermes API Server (
/v1/chat/completions)- Chat no longer routes through relay server — bypasses it entirely
- Uses OpenAI-compatible HTTP/SSE with
X-Hermes-Session-Idfor session continuity - Auth via
Authorization: Bearer <API_SERVER_KEY>stored in EncryptedSharedPreferences
- New
HermesApiClient— OkHttp-SSE client with health check and streaming chat - New
ApiModels.kt— OpenAI-format request/response models (ChatCompletionChunk, etc.) - Refactored
ChatHandler— removed envelope-based dispatch, added typed SSE entry points - Refactored
ConnectionViewModel— dual connection model:- API Server (HTTP) for chat — URL, key, health check, reachability state
- Relay Server (WSS) for bridge/terminal — separate URL, connect/disconnect
- Refactored
ChatViewModel— sends viaHermesApiClient.sendChatStream()with cancel support - Updated Onboarding — collects API Server URL + API Key (required) + Relay URL (optional)
- Updated Settings — split into "API Server" and "Relay Server" cards
- Updated ChatScreen — gates on API reachability, added stop button for streaming
- Updated DataManager — backup format v2 with separate apiServerUrl/relayUrl fields
- Updated docs/decisions.md with ADR for direct API chat
- Updated docs/spec.md with new chat architecture
Architecture change:
Before: Phone (WSS) → Relay (:8767) → WebAPI (:8642) [everything]
After: Phone (HTTP/SSE) → API Server (:8642) [chat — direct]
Phone (WSS) → Relay (:8767) [bridge, terminal]
2026-04-05 — Edge Case Fixes + CI/CD Hardening
Done:
- Fixed SSE thread safety — all callbacks dispatched to main thread via Handler
- Fixed overlapping streams — previous stream cancelled before new send
- Fixed tool call completion — now matches by toolCallId instead of first incomplete
- Fixed onboarding test connection — client properly cleaned up on exception via try/finally
- Fixed health check loop — only runs when API client is configured
- Added
network_security_config.xml— cleartext restricted to localhost/127.0.0.1/emulator - Added ProGuard rules for
okhttp-sse(okhttp3.sse., okhttp3.internal.sse.) - Added
idfield to ToolCall data class for proper matching - SSE read timeout set to 5 minutes (was 0/infinite — detects dead connections)
- OkHttpClient.shutdown() now uses awaitTermination for clean teardown
- Used AtomicBoolean for completeCalled flag (thread-safe)
- Created CHANGELOG.md (Keep a Changelog format)
- Created RELEASE_NOTES.md (used as GitHub Release body)
- Updated release.yml — uses RELEASE_NOTES.md body, SHA256 checksums, prerelease detection
- Created .github/dependabot.yml (Gradle + GitHub Actions, weekly, grouped)
- Created .github/PULL_REQUEST_TEMPLATE.md with checklist
- Added in-app "What's New" dialog in Settings (reads from bundled whats_new.txt asset)
- Bumped versionCode=2, versionName=0.2.0
2026-04-05 — Session Management + What's New Auto-Show
Done:
- Switched chat streaming from
/v1/chat/completionsto/api/sessions/{id}/chat/stream- Proper Hermes session API — not the undocumented X-Hermes-Session-Id header
- Parses Hermes-native SSE events (assistant.delta, tool.started, tool.completed, assistant.completed)
- Full session CRUD in HermesApiClient:
listSessions(),createSession(),deleteSession(),renameSession(),getMessages()
- Session drawer UI (ModalNavigationDrawer):
- List sessions with title, timestamp, message count
- New Chat button, switch sessions, rename/delete with confirmation dialogs
- Hamburger menu icon in ChatScreen top bar
- Session lifecycle:
- Auto-creates session on first message if none active
- Auto-titles session from first user message (truncated to 50 chars)
- Message history loads when switching sessions
- Last session ID persisted to DataStore — resumes on app restart
- Optimistic deletes and renames
- What's New auto-show:
- Tracks last seen version in DataStore (KEY_LAST_SEEN_VERSION)
- Shows WhatsNewDialog automatically when version changes
- Dismisses and records current version
- New models: SessionModels.kt (SessionItem, MessageItem, HermesSseEvent, etc.)
- Updated ChatHandler with session management methods (updateSessions, removeSession, etc.)
- Updated ChatMessage.ChatSession with messageCount and updatedAt fields
- Updated whats_new.txt with session management features
2026-04-05 — MVP Polish: Markdown, Reasoning, Tokens, Personalities, UX
Done:
- Markdown rendering — assistant messages render with full markdown (code blocks, bold, italic, links, lists) via mikepenz multiplatform-markdown-renderer-m3
- Message copy — long-press on any message bubble to copy text to clipboard
- Reasoning/thinking display — collapsible ThinkingBlock above assistant responses when agent uses extended thinking; toggle in Settings
- Token & cost tracking — per-message token count (↑input ↓output) and estimated cost displayed below timestamp
- Personality picker — dropdown in chat top bar with 8 built-in personalities (default, concise, creative, technical, teacher, formal, pirate, kawaii); injects system_message into chat stream
- Error retry button — "Retry" button in error banner re-sends last failed message
- Offline detection — ConnectivityObserver via ConnectivityManager; shows offline banner when network is lost
- Streaming state fix — onStreamError now clears isStreaming/isThinkingStreaming on all affected messages
- Haptic feedback — on send, stream complete, error, and message copy
- Input character limit — 4096 char limit with counter shown near the limit
- Responsive layout — bubble width adapts by screen width: 300dp (phone), 480dp (medium), 600dp (tablet)
- Enriched tool cards — tool-type-specific icons (terminal, search, file, tap, keyboard, etc.), duration tracking ("Completed in X.Xs")
- Accessibility — content descriptions on message bubbles, tool cards, thinking blocks, all interactive elements
- Dead code cleanup — deleted unused ApiModels.kt and ChatModels.kt
- Settings: Chat section — new "Show reasoning" toggle
- Added ACCESS_NETWORK_STATE permission to AndroidManifest
New files:
ui/components/MarkdownContent.kt— Compose markdown renderer wrapperui/components/ThinkingBlock.kt— collapsible reasoning/thinking displayui/components/TokenDisplay.kt— per-message token + cost displayui/components/PersonalityPicker.kt— personality selection dropdownnetwork/ConnectivityObserver.kt— reactive network connectivity listener
New dependencies:
com.mikepenz:multiplatform-markdown-renderer-m3:0.30.0com.mikepenz:multiplatform-markdown-renderer-code:0.30.0material3-window-size-class(responsive layout)
2026-04-05 — Full Audit + Bug Fixes + VitePress Docs Site
Audit findings fixed (9 bugs):
- CRITICAL: Added ProGuard rules for mikepenz markdown renderer + intellij-markdown parser
- CRITICAL: Cached fallback StateFlows in ChatViewModel (was creating new instances per access)
- MAJOR: Fixed MessageItem.id type from Int? to String? (server returns string IDs)
- HIGH: Added !isStreaming guard to send button (prevented double-send)
- HIGH: Added personality fallback when ID not found (prevents null system message)
- MEDIUM: Session rename dialog remember key now includes session (prevents stale state)
- MEDIUM: ToolProgressCard duration guard against negative values
- LOW: Error banner text limited to 2 lines with ellipsis
- BUILD: Unified AGP version to 8.13.2 in libs.versions.toml
VitePress documentation site created:
- 20 pages across 4 sections: Guide, Features, Architecture, Reference
- Landing page with hero section and 8 feature cards
- Full user guide: getting started, chat, sessions, troubleshooting
- Feature docs: direct API, markdown, reasoning, personalities, tokens, tools
- Architecture docs: overview diagram, ADRs adapted from docs/decisions.md, security model
- API reference with all endpoints and request/response examples
- Configuration reference with all DataStore keys and settings
- VitePress config with nav, sidebar, local search
- Run with:
npm install && npm run dev:docs
Next:
- Build and test against running Hermes API server
- Test on emulator and physical device (S25 Ultra)
- Set up keystore/signing secrets for release CI
- Deploy docs site (GitHub Pages or similar)
- Phase 2: Terminal channel (xterm.js in WebView, tmux integration)
- Phase 3: Bridge channel migration