Compare commits

...
187 Commits
Author SHA1 Message Date
Bailey DixonandClaude Opus 4.6 e06deace37 fix: update stale version refs 0.1.0 → 0.4.0 in app + user-docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 23:00:08 -04:00
Bailey DixonandClaude Opus 4.6 7420704560 fix(manifest): flip FGS type override direction for lint compliance
CI lint failed on googlePlayDebug: the main manifest declared
foregroundServiceType="specialUse|mediaProjection" but the
FOREGROUND_SERVICE_MEDIA_PROJECTION permission was in the sideload
manifest only. Lint checks the main manifest pre-merge and flags
the type without the permission as an error.

Fix: main declares specialUse only (the safe default for googlePlay).
Sideload manifest overrides to specialUse|mediaProjection via
tools:replace — sideload has the MEDIA_PROJECTION permission and
the /screenshot route. googlePlay inherits the main default directly
with no override needed.

This is the correct direction: main = conservative (matches Play),
sideload = additive (opts into more capabilities). The previous
approach had it backwards — main had both types and googlePlay
stripped one via tools:replace, which tripped lint because the
permission removal didn't propagate to the main manifest's lint pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:31:39 -04:00
Bailey DixonandClaude Opus 4.6 7bed7bee48 release: v0.4.0
Version bump 0.3.0 → 0.4.0 across all three sources:
- gradle/libs.versions.toml (appVersionName + appVersionCode 3→4)
- pyproject.toml
- plugin/relay/__init__.py

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:23:02 -04:00
Bailey Dixon 7e73d6a71f Merge branch 'feature/bridge-feature-expansion' into main
v0.4.0 — Voice → bridge → agent pipeline, Play Store hardening,
18 android_* tools, and end-to-end safety modal system.

65 commits on the feature branch covering:
- Voice intent classifier (SMS direct/indirect, open app, tap, back, home)
- In-process local dispatch (bypasses WSS round-trip for voice intents)
- Safety modal threading fix (Dispatchers.Main.immediate) + savedstate
  lifecycle init order fix (performRestore before CREATED)
- Post-dispatch feedback (LocalDispatchResult capture, TTS, chat bubbles)
- Voice mode transcript (last 6 chat messages, CompactTranscriptRow)
- 4 new plugin tools (android_send_sms, android_call, android_search_contacts,
  android_return_to_hermes) with structured error_code responses
- _check_requirements fixed (hit /bridge/status not /ping, single-gate)
- Honest error classification (sealed result types, per-category messages)
- Recursive JSON serialization in respondFromResult
- Structured contact phones ({number, type, label}, sorted, Watch heuristic)
- /tap + /long_press destructive-verb gate via ScreenReader node text
- userDeniedResponse with "denial is final" semantics
- Trust-model + denial-retry-guard language in all tool descriptions
- MediaProjection cleanup on onTaskRemoved
- Activity launchMode=singleTask + configChanges
- Play Store hardening: route whitelist (fail-closed), manifest split
  (FOREGROUND_SERVICE_MEDIA_PROJECTION → sideload only), FGS type override,
  UI gating (Bridge Mode label, hidden safety/overlay/screenshot rows)
- Flavor context in BridgeStatusReporter (device.flavor + application_id)
- 3 new user-docs pages (voice-intents, phone-control-tools, flavor-differences)
- All docs updated (tool count 14→18, scroll removed from voice, etc.)
2026-04-15 22:22:27 -04:00
Bailey DixonandClaude Opus 4.6 00a9887ab4 chore: commit pre-session working tree changes
Six files modified outside the voice-intents session — roadmap
updates, TODO adjustments, upstream contribution docs, decisions
doc corrections, and bootstrap handler/patch tweaks. Committing
to clean the working tree before the v0.4.0 merge to main.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:20:57 -04:00
Bailey DixonandClaude d3c41e3f75 fix(docs): use doc layout for privacy policy page (#30)
The privacy page used `layout: page` which renders content without the
`vp-doc` CSS class wrapper, breaking all document styling (tables,
typography, headings, custom blocks). Switch to `layout: doc`.

https://claude.ai/code/session_01C4typoCNZQBLCqqaZuUDqC

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-15 22:18:53 -04:00
Bailey DixonandClaude Opus 4.6 513de394d5 docs + fix: add 3 user-docs pages + expose flavor in bridge status
New user-docs pages (350 lines total):
- features/voice-intents.md — voice intent classifier, supported
  intents table, 5s countdown + voice cancel, contact resolution
  with multi-phone preference, post-dispatch TTS/chat feedback,
  phone-number literal bypass, classifier fall-through behavior
- features/phone-control-tools.md — all 18 android_* tools with
  category/sideload markers, check_requirements single-gate model,
  direct dispatch tools, trust model, denial-is-final policy,
  structured error_code responses, contact phones format
- architecture/flavor-differences.md — build system, source set
  layout, manifest split, accessibility config comparison,
  BridgeCommandHandler Play whitelist, UI differences table,
  BuildFlavor object, plugin cross-flavor behavior

Fix: BridgeStatusReporter now includes device.flavor and
device.application_id in the bridge.status envelope pushed to the
relay every 30s. The relay's /bridge/status HTTP endpoint passes
these through, so the LLM agent can see which build flavor is
connected BEFORE attempting sideload-only tools. Without this the
agent was flavor-blind until a 403 sideload_only hit — wasting a
tool call and requiring error-recovery reasoning.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:17:23 -04:00
Bailey DixonandClaude Opus 4.6 160dda049a docs(user-docs): update release tracks, relay routes, security for v0.4.0
release-tracks.md:
- Google Play TL;DR updated: explicitly "read-only bridge", lists
  what can't happen (no tap/type/swipe/SMS/call), notes "Bridge Mode"
  toggle label. Sideload: mentions 18 tools + "Agent Control" label.
- Safety rails section rewritten: explains that Play track has no
  safety modals because action routes are code-blocked (not flag-
  disabled). Sideload section documents /tap + /long_press verb
  gate, denial-is-final semantics, and the full safety stack.

relay-server.md:
- Route count 27 → 31 (30 excluding legacy /apps alias). Lists the
  4 new routes (send_sms, call, search_contacts, return_to_hermes).
  Notes the googlePlay whitelist gate.

security.md:
- Destructive-verb confirmation section expanded: now covers /tap +
  /long_press by nodeId (via ScreenReader text resolution), not just
  /tap_text + /type. Documents the denial-is-final semantics with
  error_code=user_denied + "do not retry" instruction.
- Master-enable bypass list updated: added /return_to_hermes. Added
  note about the googlePlay whitelist gate that fires before the
  master-enable check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:07:56 -04:00
Bailey DixonandClaude Opus 4.6 29610d25fc docs: update tool count 14 → 18, remove scroll from voice intents
Seven docs files referenced the old "14 android_* tool handlers"
count and/or listed scroll as a voice intent. Updated to reflect
the v0.4.0 state: 18 tools (added android_send_sms, android_call,
android_search_contacts, android_return_to_hermes), scroll removed
from the voice intent classifier (server-side android_scroll tool
handles it instead via the LLM tool-calling path).

Files updated:
- CLAUDE.md (repo layout + key files table)
- README.md (repo layout + install instructions)
- CONTRIBUTING.md (install verification instructions)
- CHANGELOG.md (v0.3.0 tool count in bridge section)
- RELEASE_NOTES.md (voice intent list + relay route inventory)
- relay_server/SKILL.md (install step comment)
- user-docs/guide/getting-started.md (feature list)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:53:21 -04:00
Bailey DixonandClaude Opus 4.6 aa67e09e66 fix(bridge): Play UI gating — toggle label, safety cards, permission rows
Gate sideload-only UI elements behind BuildFlavor.isSideload so the
googlePlay APK's visible interface matches its declared capability.

BridgeScreen.kt:
- Master toggle label: "Agent Control" on sideload, "Bridge Mode"
  on googlePlay. Subtitle: "agent can interact/control" vs "screen
  reading for chat context". Label shapes user expectation and is
  what Play reviewers read alongside the a11y description.
- Overlay permission nag banner: hidden on googlePlay — no
  destructive-verb modal means SYSTEM_ALERT_WINDOW isn't needed.
- Safety summary card (destructive verb count, blocklist size,
  auto-disable timer): hidden on googlePlay — all action routes are
  blocked at the whitelist so safety configuration is irrelevant.

BridgeMasterToggle.kt:
- New `label: String = "Agent Control"` parameter so the call site
  can pass "Bridge Mode" on googlePlay. Subtitle auto-adjusts based
  on label context.

BridgePermissionChecklist.kt:
- "Display over other apps" row: hidden on googlePlay — no safety
  modal to render, permission not needed.
- (Screen Capture row was already hidden in a prior commit.)
- Accessibility Service subtitle: "dispatch taps/types" on sideload
  vs "Read screen content for chat context" on googlePlay.

Net result for googlePlay Bridge tab: users see "Bridge Mode" toggle
+ accessibility service row + notification listener row + activity
log. No mention of agent control, screen capture, overlays, or
safety configuration. Clean surface for review.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:46:30 -04:00
Bailey DixonandClaude Opus 4.6 be181664f8 fix(bridge): Play Store hardening — route whitelist, manifest split, UI gate
Prepare the googlePlay flavor for Play Store review by ensuring the
APK's code capabilities match its declared accessibility use case
("read on-screen content and summarize notifications").

== BridgeCommandHandler Play route whitelist ==

Added a fail-closed whitelist BEFORE the per-path dispatch. On
googlePlay, only read-only routes pass: /current_app, /screen,
/get_apps, /apps, /clipboard (GET only), /return_to_hermes. All
action routes (/tap, /type, /swipe, /scroll, /open_app, /press_key,
/long_press, /drag, /screenshot, /find_nodes, /screen_hash,
/diff_screen, /describe_node, /media, /send_intent, /broadcast,
/events, /send_sms, /call, /search_contacts, /location, /clipboard
POST) return 403 with error_code=sideload_only.

The whitelist is fail-closed so any NEW route added to the when block
defaults to sideload-only unless explicitly whitelisted — future
routes can't accidentally widen the Play capability surface.

/clipboard POST has its own sideload gate inside the clipboard
handler since /clipboard is whitelisted for GET (read).

Early-return routes (/ping, /events read, /setup) are above the
gate and work on both flavors — they're harmless probes.

== Manifest split ==

- FOREGROUND_SERVICE_MEDIA_PROJECTION permission moved from main
  manifest to sideload manifest. googlePlay doesn't need screen
  recording (/screenshot is gated) and declaring the permission
  would flag review.

- googlePlay manifest overlay gained a tools:replace override on
  BridgeForegroundService's foregroundServiceType, narrowing it
  from specialUse|mediaProjection to specialUse-only. Without
  this the merged manifest would carry the mediaProjection type
  from main and Play review would flag it.

- SPECIAL_USE FGS property description rewritten to be flavor-
  neutral and match the read-only use case: "Maintains a persistent
  WebSocket connection to the user's Hermes server for real-time
  chat relay and notification mirroring."

== Accessibility description ==

googlePlay strings.xml updated: removed "replying with your
confirmation" (since /type and /tap are gated) and added explicit
"read-only — it does not perform taps, type text, or control other
apps" so the description matches the code capabilities exactly.

== UI gate ==

BridgePermissionChecklist.kt: Screen Capture row hidden on
googlePlay via BuildFlavor.isSideload check. Accessibility Service
subtitle changed from "dispatch taps/types" to "Read screen content
for chat context" on googlePlay. Users and reviewers see capabilities
that match the APK.

== Also in this commit (from the P0/P1 audit) ==

- /tap + /long_press destructive-verb gate (extractDestructiveVerbText)
- Galaxy Watch label-heuristic in sortPhonesByPreference
- userDeniedResponse helper with structured error_code
- anyToJsonElement recursive JSON serializer
- Denial-retry guard in all UI-automation tool descriptions
- Structured phones list in ActionExecutor.searchContacts
- sideload_only error_code on all four sideload-gated 403s

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:41:57 -04:00
Bailey DixonandClaude Opus 4.6 b7dab4c496 fix(voice-intents): close UI-automation bypass + structured contact phones
Follow-up pass from the P0/P1 audit after Bailey's 2026-04-15 on-device
test where Victor hit two distinct gaps: (1) he denied an SMS modal
and the agent immediately fell back to driving the Messages app UI by
android_open_app + android_tap, bypassing the denial, (2) the contact
Hannah Dixon had a Galaxy Watch phone AND a mobile, and the voice
auto-picker chose the Watch.

== P0: Close the android_tap destructive-verb bypass ==

BridgeSafetyManager.requiresConfirmation now fires on /tap and
/long_press in addition to /tap_text and /type. BridgeCommandHandler
adds extractDestructiveVerbText() which resolves the tapped node's
text via ScreenReader.findNodeById when the caller passes a nodeId
(the common LLM pattern after android_read_screen). The text is
pattern-matched against the user's destructive-verb list the same
way /tap_text does, so tapping a button whose label is "Send" /
"Delete" / "Pay" / "Confirm" now fires the safety modal regardless
of which tool the agent used to locate it.

Fail-open for coordinate-only taps (no nodeId) and for any case
where the snapshot/findNodeById fails — matches pre-0.4.0 semantics
for /tap so we're strictly ADDING coverage, not converting taps to
fail-closed. Coordinate hit-testing is a P0.5 follow-up and rarely
matters in practice because modern LLMs prefer nodeId after
android_read_screen.

Recycles window roots + the resolved node on the gate path to avoid
leaking AccessibilityNodeInfo handles.

== P1: Server-side phone preference sort + Galaxy Watch label heuristic ==

ActionExecutor.searchContacts now sorts each contact's phones list
in preference order before returning: mobile > main > home > work >
other > everything else (custom, watch, fax, pager). Critically, a
label-substring heuristic detects "watch" / "pager" / "smartwatch" /
"fax" in the phone's human-readable label and demotes those entries
to the bottom tier REGARDLESS of their canonical type. This catches
Galaxy Watch entries that Samsung Health registers as TYPE_MOBILE
with a "Watch" label, which the type-only ranker from the previous
commit couldn't distinguish from real mobiles.

Stable sort via rank*1000 + originalIndex so within a tier the
insertion order from the content provider is preserved.

The voice handler's pickPreferredPhone simplifies to "return the
first non-empty entry" since the list is pre-sorted server-side.
This centralizes ranking in one place and — crucially — means the
LLM tool-calling path benefits without needing tool-description
cooperation: the LLM can just use phones[0] blindly and get the
right number.

== P1: Denial-retry warnings in UI-automation tool descriptions ==

android_open_app, android_tap, android_tap_text, android_type, and
android_call all gained explicit DENIAL-RETRY GUARD paragraphs
telling the LLM not to use these tools to replicate a destructive
action that android_send_sms or android_call just received a
user_denied response for. android_send_sms already had this in the
previous commit; now all the fallback-path tools carry the same
warning so the LLM sees it no matter which alternate route it
considers. Belt and braces — the real enforcement is in the code
(the /tap verb gate now fires on the Messages app's Send button),
but the tool-description guidance helps the LLM fail gracefully
instead of hammering the modal.

== Structural cleanup: recursive JSON serialization ==

BridgeCommandHandler.respondFromResult went from "primitives-only
with toString fallback" to fully recursive anyToJsonElement for
nested Lists and Maps. Pre-fix, the search_contacts response
serialized the `contacts` field as a Kotlin list-repr string
("[{id=9, name=Hannah, phones=...}]") — the LLM had to parse
pseudo-JSON. Now it gets real JSON with the structured phones
list as actual arrays of objects. Same recursive serializer
handles future ActionResult fields that carry nested structure.

== userDeniedResponse helper ==

Three denial paths in BridgeCommandHandler (/tap_text verb gate,
/call, /send_sms) all now return the same canonical shape via a
userDeniedResponse(contextText) helper:

  {
    "error": "<context>. This is a FINAL denial. Do NOT retry...",
    "error_code": "user_denied",
    "reason": "confirmation_denied_or_timeout",
    "final": true,
    "instruction": "Do not retry via UI automation. Denial is terminal."
  }

Structured error_code + explicit instruction field give the LLM
both machine-readable classification and a literal no-fallback
directive in the JSON payload. The free-text error carries the
contextual details (SMS recipient, call number, which verb fired
the tap gate).

== sideload_only 403s carry error_code + flavor ==

The four sideload-only 403s (/location, /search_contacts, /call,
/send_sms on a googlePlay build) now include error_code="sideload_only"
and flavor="googlePlay" so the LLM can classify them cleanly and
(per the android_send_sms tool description) ask the user for
explicit consent before falling back to UI automation, rather than
silently switching paths. This fixes the secondary bug where Victor
paraphrased a denial as "Direct SMS is blocked on this build" — on
sideload it was actually a user_denied, but Victor's loose free-text
interpretation let it slide into "build limitation" territory and
justified a UI-automation fallback.

== classifyBridgeError ==

Expanded to catch "user denied" (broader than "user denied
destructive action"), "this is a final denial", and "sideload-only"
substrings. Loosened user_denied pattern in case future error text
varies.

== Files ==

- app/src/main/kotlin/.../accessibility/ActionExecutor.kt
  - fetchPhonesForContact returns List<Map> with {number, type, label}
  - sortPhonesByPreference helper (new)
  - phoneTypeKey + phoneTypeDisplayLabel helpers (new)
  - searchContacts applies sortPhonesByPreference server-side

- app/src/main/kotlin/.../network/handlers/BridgeCommandHandler.kt
  - extractDestructiveVerbText helper (new) — resolves node text
    for /tap + /long_press by nodeId via ScreenReader
  - userDeniedResponse helper (new) — canonical 403 shape
  - anyToJsonElement helper (new) — recursive JSON serialization
  - respondFromResult uses recursive serializer
  - Three awaitConfirmation denial paths use userDeniedResponse
  - Four sideload_only 403s carry error_code + flavor fields
  - classifyBridgeError expanded patterns

- app/src/main/kotlin/.../bridge/BridgeSafetyManager.kt
  - requiresConfirmation accepts /tap and /long_press paths

- app/src/sideload/kotlin/.../voice/VoiceBridgeIntentHandlerImpl.kt
  - pickPreferredPhone simplified (server pre-sorts)
  - ContactResolution.Found gained phoneType + phoneLabel +
    totalPhones fields for richer voice preview
  - SendSms branch speaks phone qualifier when totalPhones > 1
    ("texting Hannah's Mobile at ...")

- plugin/android_tool.py
  - android_search_contacts description documents new structured
    phones shape + disambiguation rules
  - android_send_sms description carries TRUST MODEL + DENIAL
    IS FINAL + sideload_only fallback guidance (~3000 chars)
  - android_call gained DENIAL IS FINAL block
  - android_open_app / android_tap / android_tap_text / android_type
    gained DENIAL-RETRY GUARD warnings

Server synced + gateway restarted, all 18 tools still register,
check_requirements still returns True when phone_connected=true.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:27:21 -04:00
Bailey DixonandClaude Opus 4.6 ddd6f706df fix(voice-intents): trust-model language in tool descriptions
Victor was double-confirming before every SMS action, asking the user
"who is Hannah Dixon, is she in your contacts, confirm the wording" in
chat BEFORE calling any tool, even though:

1. android_search_contacts exists specifically so the LLM can resolve
   names autonomously
2. The phone's destructive-verb safety modal is a hardcoded final
   on-device Allow/Deny checkpoint that blocks every SMS send

Claude-family models have a trained safety reflex around messaging
actions — especially emotionally loaded content and especially to
names the model doesn't recognize. That reflex is generally good but
redundant here because the on-device modal is the real checkpoint.
The fix is to make the trust model explicit in the tool descriptions
so the model knows:

- The user gave it phone control explicitly via the master toggle
- The on-device modal is sufficient confirmation on its own
- Chat-side double-confirmation is redundant AND frustrating
- Contact names should be resolved via search_contacts, not by
  bouncing lookup questions back to the user

android_search_contacts, android_send_sms, and android_call all got
expanded descriptions with a TRUST MODEL block explaining the
on-device checkpoint and explicitly prohibiting the double-confirm
anti-pattern.

Tool description lengths:
- android_search_contacts: 754 chars
- android_send_sms: 1837 chars
- android_call: 991 chars

These are long for tool descriptions, but Claude reads them carefully
and the behavioral override is worth the token cost. Alternative
approaches (personality prompt, plugin.yaml directive, system-level
instructions) all couple us to specific hermes-agent deploy config
or violate the "guidance travels with the plugin" principle we
established in c5cdb45.

Caught 2026-04-15 by Bailey on-device: he enabled Agent Control and
asked Victor to text Hannah Dixon; Victor responded with a
multi-question interrogation instead of calling the search tool.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 21:01:25 -04:00
Bailey DixonandClaude Opus 4.6 1323dc3dc3 fix(voice-intents): single-gate _check_requirements + structured 503
Reverting the three-gate rule from the previous commit. Bailey hit the
real-world cost on-device: with the master toggle off, Victor saw no
android_* tools and hallucinated a reason for their absence — "Phone
bridge isn't connected right now, pair the Hermes-Relay app" — then
asked the user for a pairing code when the phone was already paired.
The phone WAS connected. Victor filled the absence-of-tools with a
guessed explanation and steered the user to the wrong action.

Root cause is a fundamental LLM-interaction principle: hidden tools
don't mean "no capability" to the model, they mean "invent a narrative
for why this capability isn't here." The clean-no-tools signal I
rationalized in the last commit sounded good in theory but fails in
practice because LLMs reason about presence/absence by making things
up.

Fix: gate check_fn ONLY on phone_connected. Let downstream return
structured errors for the other failure modes:

- No a11y → HTTP 503 + error_code=service_unavailable + explicit
  "the phone IS paired, this is NOT a pairing problem" text +
  required_action field naming the exact Settings destination
- Master toggle off → HTTP 403 + error_code=bridge_disabled +
  similar structured fields (already landed in 5c763eb)

Both error paths now carry enough context that the LLM can relay
accurate instructions. check_fn is reserved for the ONE case where
tools literally cannot function: no WSS session at all. In that case
the LLM sees only android_setup and correctly deduces "we need to
pair."

Plugin docstring expanded with the three-version history so the next
person who considers adding gates to check_fn understands why this
specific design was chosen.

Server verification with live bridge status showing phone_connected=True
but a11y=False and master=False:

  check_requirements(): True
  tool count: 18

All tools visible, all 18 gated behind a single phone_connected check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:48:22 -04:00
Bailey DixonandClaude Opus 4.6 c5cdb45cdb fix(voice-intents): UX polish, chat parity, OEM hardening, portable guidance
Agent-team follow-up after Bailey's 2026-04-15 voice SMS test session.
Fifteen files touched, ranging from voice-mode UX to Android manifest
hardening to plugin tool descriptions. See DEVLOG for the full story
and "known residual items" list.

Voice mode UX:
- TTS speaks the post-dispatch result ("Text sent", "Cancelled",
  "Permission needed") via the existing ttsQueue pipeline
- Voice-in-voice cancel: "cancel", "stop", "never mind", "abort",
  "forget it", "wait" spoken DURING the 5s countdown now routes
  straight to cancelPending() instead of being classified as a
  fresh turn that goes to the LLM. VoiceBridgeIntentHandler interface
  gained hasPendingDestructive() so VoiceViewModel can intercept
  cancel utterances before the classifier runs
- Visual countdown: DestructiveCountdownState on VoiceUiState +
  onCountdownStart callback + LinearProgressIndicator in the overlay
  synced to the real delay, so the user sees the 5s window tick down
  instead of staring at a static UI
- Phone-number literal bypass: "text +1 555 123 4567 saying hi" no
  longer fails contact resolution; PHONE_NUMBER_REGEX detects
  phone-shaped contacts and skips resolveContactPhone
- Multi-contact match hint: "Found 3 contacts matching John. Using
  John Smith." when resolveContactPhone returns more than one hit
  so the user knows one of several got picked (full multi-turn
  disambiguation deferred to Wave 3)

Chat parity:
- ChatHandler intercepts tool.completed SSE events on /v1/runs for
  android_* action tools (send_sms, call, search_contacts, open_app,
  return_to_hermes, screenshot, press_key, setup) and emits a
  structured follow-up bubble with the same per-category formatting
  voice mode uses. Read-only + UI-micro-action tools skipped to
  avoid doubling up with the existing ToolProgressCard
- New appendLocalVoiceIntentResult(description, agentName) signature
  so chat-originated action bubbles get a distinct "Phone action"
  caption vs voice-originated "Voice action"
- MessageBubble renders a subtle visual marker (tertiary-tinted
  leading border) for both action-bubble types so they read as
  distinct from LLM replies when interleaved

Plugin + phone-side:
- _check_requirements gates on all three: phone_connected AND
  bridge.accessibility_granted AND bridge.master_enabled. Tools
  vanish from the LLM's schema entirely when the master toggle
  is off, giving the model a clean "no tools" signal instead of
  an error-interpretation race. Trade-off: tools disappear mid-
  session if the user flips the toggle — desired per "stop the
  agent from controlling my phone" intent
- /return_to_hermes short-circuits when service.currentApp ==
  service.packageName (already foreground), returning 200 with
  a "note: already foreground" instead of re-firing the launch
  intent. Benign for voice mode where Hermes is always foreground
- Tool descriptions carry the "prefer direct dispatch" +
  "call return_to_hermes as final step" guidance inline on
  android_open_app / android_send_sms / android_call. Portable
  across hermes-agent installs; replaces the earlier Victor
  personality prompt edit (reverted — only worked for one install)

Android OEM hardening:
- BridgeForegroundService.onTaskRemoved revokes MediaProjection
  and downgrades the FGS type back to SPECIAL_USE-only when the
  user swipes the app from recents. Fixes the bug where the system
  screen-cast icon persisted in the status bar indefinitely after
  app close because foreground services legitimately survive task
  removal and the projection stayed bound. Bridge itself keeps
  running so agent phone control over WSS still works
- MainActivity manifest entry gained launchMode="singleTask" and
  configChanges="uiMode|fontScale|locale|density|orientation|
  screenSize|screenLayout|keyboardHidden". Before this the activity
  was being recreated on config changes and certain resume paths,
  triggering installSplashScreen() every time and showing the
  splash on warm reopen. Samsung OneUI's aggressive backgrounded-
  app kills (even with FGS) is the residual case that needs
  user-side battery whitelisting and can't be fixed in code

DEVLOG.md updated with the full session narrative.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:41:59 -04:00
Bailey DixonandClaude Opus 4.6 536a131e58 fix(voice-intents): post-dispatch feedback + clearer bridge-disabled error
Two follow-ups to Bailey's 2026-04-15 on-device voice SMS test:

1. Voice actions had no post-dispatch feedback
   After the safety modal resolved and the SMS actually sent (or failed),
   the voice mode transcript still showed only the pre-dispatch preview
   ("Send SMS -- awaiting confirmation"). User had no in-context way to
   tell whether the message went through, was denied, or hit a permission
   error. BridgeCommandHandler.handleLocalCommand was fire-and-forget:
   it ran dispatch() under the LocalDispatch context element, which told
   respond() to drop the payload instead of sending a WSS envelope. The
   response JSON was built and thrown away.

   Fix: add a capture mechanism to the LocalDispatch context element
   (AtomicReference<JsonObject?> for the result body, AtomicInteger for
   the status code) and have respond() write into them in the local path.
   handleLocalCommand now returns LocalDispatchResult(status, errorMessage,
   errorCode, resultJson) carrying everything voice mode needs to render
   an accurate follow-up. The LocalBridgeDispatcher typealias flips from
   'suspend (Envelope) -> Unit' to 'suspend (Envelope) -> LocalDispatchResult'
   in both the sideload and googlePlay factories -- the Play flavor still
   never invokes it but the shared VoiceViewModel call site compiles
   against either source set.

   RealVoiceBridgeIntentHandler threads a new VoiceIntentResultCallback
   through the factory; handleDestructive and handleSafe call it after
   each dispatch with the captured LocalDispatchResult. VoiceViewModel
   wires the callback to ChatViewModel.recordVoiceIntentResult, a new
   method that renders a markdown bubble per outcome category:

     - success             -> **Send SMS -- sent** check
     - user_denied         -> **Send SMS -- cancelled by you**
     - bridge_disabled     -> **Send SMS -- agent control is off** + hint
     - permission_denied   -> **Send SMS -- permission needed** + detail
     - service_unavailable -> **Send SMS -- bridge offline** + hint
     - cancelled           -> **Send SMS -- cancelled before dispatch**
     - other failure       -> **Send SMS -- failed** + error text

   New ChatHandler.appendLocalVoiceIntentResult writes the bubble with
   id prefix 'voice-intent-result-' so it survives loadMessageHistory
   reloads (same preservation filter as the pre-dispatch trace) and
   renders via MarkdownContent in CompactTranscriptRow (the prefix
   already matches 'voice-intent-' so no overlay changes needed).

2. Paired-but-disabled error text confused the LLM
   When the phone was paired + accessibility granted but the master
   toggle was flipped off, BridgeCommandHandler's 403 response said:

     'Bridge is disabled -- enable Agent Control in the Bridge tab'

   Victor repeatedly read this as 'bridge not paired' and walked the
   user through re-pairing instead of the toggle. LLMs are unreliable
   around ambiguous phrasing. Fix: explicit error text that names the
   exact action needed and declares it is NOT a pairing problem, plus
   a structured error_code = 'bridge_disabled' and a required_action
   field. classifyBridgeError in respondFromResult also picks up the
   new substring so the error_code propagates through any call site
   that routes through that helper.

Seven files touched, no plugin changes (those are already live on the
server from 5c763eb). handleLocalCommand signature change ripples to:
BridgeCommandHandler (method + LocalDispatch class + respond + new
data class + classifier), sideload and googlePlay factories (typealias +
factory signatures + new VoiceIntentResultCallback typealias),
VoiceBridgeIntentHandlerImpl (new field + handleDestructive/handleSafe +
dispatch helper return type), VoiceViewModel (factory call site),
ChatViewModel (new recordVoiceIntentResult + formatVoiceIntentResult),
ChatHandler (new appendLocalVoiceIntentResult).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:10:42 -04:00
Bailey DixonandClaude Opus 4.6 5c763eb265 fix(voice-intents): honest errors, modal lifecycle, and missing LLM tools
End-to-end fix for the voice -> bridge -> agent pipeline. Twelve changes
across phone, plugin, and bridge layers so the voice fast-path, the LLM
tool-calling path, and the safety modal all actually work.

Voice fast-path:
- Remove Scroll voice intent (nobody says it aloud; /scroll route stays
  for LLM android_scroll tool calls).
- Add SMS_INDIRECT regex to catch "send Hannah a text saying hi" phrasing
  and "message" as a direct verb ("message Sam saying hi").
- Classify resolver failures via ContactResolution / AppResolution sealed
  types so voice speaks specific per-category messages (permission,
  service, not-found, no-phone, other) instead of "couldn't find" for
  every failure mode.
- Pre-check SEND_SMS before the 5s countdown so permission-denied doesn't
  silent-fail at the end of the confirmation flow.
- Flip voice state to Thinking immediately on classifier fall-through so
  the UI shows progress during SSE connect latency.
- Enrich IntentResult.Handled with details: Map<String, String> for
  structured chat-trace rendering (app label, package, match tier,
  contact, resolved number, body, error code).
- Rewrite chat-trace formatter to render markdown per category.

Bridge safety modal:
- Fix showConfirmation threading -- ComposeView setContent must run on
  Main, was called from Dispatchers.Default via voice local-dispatch,
  threw, and got swallowed as "likely overlay permission missing".
- Fix OverlayLifecycleOwner.start() init order -- current androidx.savedstate
  asserts performAttach runs while lifecycle is still INITIALIZED; the
  old code advanced to CREATED first and tripped the assertion, killing
  every destructive-verb modal attempt silently.

Voice mode UI:
- Replace single responseText slot with compact rolling transcript
  observing ChatViewModel.messages (last 6), rendered via new
  CompactTranscriptRow + StreamingResponseRow composables.
- Voice-action traces render via MarkdownContent. User messages keep
  the "YOU" caption (fix mislabeling where voice-intent user messages
  were captioned "ACTION").
- Preserve local voice-intent trace messages across
  ChatHandler.loadMessageHistory reloads so "Opened Chrome" bubbles
  don't vanish when session_end reload fires after fall-through.

LLM bridge path -- honest errors:
- respondFromResult emits structured error_code + required_permission
  alongside the existing free-text error when ActionExecutor errors
  match known patterns (permission_denied, service_unavailable,
  user_denied). LLM gets both human-readable text AND a machine-readable
  classification.

LLM bridge path -- missing tools:
- Fix plugin _check_requirements: was hitting /ping which returns
  {pong, ts}, looking for phone_connected and accessibilityService
  fields that do not exist there. Result: the gate always returned
  False and hid all 13 non-setup tools from every gateway platform.
  Now hits /bridge/status and requires BOTH phone_connected AND
  bridge.accessibility_granted -- tools vanish from the LLM's schema
  when a11y is revoked (common post-Studio-reinstall) instead of
  letting the LLM confidently dispatch commands that 503.
- Add 4 new plugin tools: android_search_contacts, android_send_sms,
  android_call, android_return_to_hermes. The first three wrap phone
  routes that were fully implemented (with safety modals and direct
  SmsManager / CALL_PHONE dispatch) but never exposed to the agent,
  forcing it to drive the Messages app UI step-by-step. The fourth
  lets the agent foreground Hermes Relay as the final step of a
  phone-control task so the user sees the reply in-context without
  manually switching apps.
- New /return_to_hermes route on BridgeCommandHandler, exempt from
  the master-toggle check so the agent can always wrap up cleanly.

Tool count goes from 14 to 18. Plugin + gateway verified on the server
(systemctl --user restart hermes-gateway, _check_requirements() returns
True, all 18 tools register with no missing or orphan handlers).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 19:20:19 -04:00
Bailey DixonandClaude Opus 4.6 2e51ba9115 fix(voice-intents): leave a local chat trace so voice actions are visible
Voice intents dispatch in-process (per the local dispatch loop fix in
0c9221f) and used to return silently from VoiceViewModel.processTurn
without ever touching chat history. Symptom: user says "open Chrome"
via voice (works), then says "proceed" (or any follow-up) and the LLM
responds "no prior context" because chat scrollback is empty for the
voice turn entirely. The user perceives this as "the chat is resetting
on voice".

Fix: append a local-only voice-intent trace to chat history when an
intent is handled. Two messages back-to-back:
  - user message with the raw transcribed text
  - assistant message labelled "Voice action" with the action description

Both are local-only — they live in `_messages` but never hit the
server-side session, so the gateway LLM still doesn't see prior voice
actions in its session memory. Server-side session sync is documented
as a v0.4.1 follow-up in ROADMAP.md (three approaches outlined: gateway
log endpoint, fait-accompli prompt prefix, or routing through sendMessage
with an idempotency token).

Local trace is enough to address the user-visible "chat is resetting"
perception for v0.4.0. Follow-up turns within the same voice session
will see the trace in chat scrollback and feel coherent again. Cross-
turn LLM context is the v0.4.1 problem.

## Files

- ChatHandler.kt: new `appendLocalVoiceIntentTrace(userText, action)`
  appends both messages with stable IDs and an "agentName: Voice action"
  marker so the assistant bubble doesn't render under the current
  personality's avatar.
- ChatViewModel.kt: new `recordVoiceIntent(userText, action)` wrapper
  delegates to ChatHandler.
- VoiceViewModel.kt: in processTurn, after `IntentResult.Handled`, call
  `chatVm.recordVoiceIntent` BEFORE the existing speak/idle UI update so
  the trace is in history by the time the user moves on.
- ROADMAP.md: new v0.4.1 entry "Voice intent → server session sync"
  documenting the three paths to full LLM context.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 21:43:13 -04:00
Bailey DixonandClaude Opus 4.6 0c9221f359 fix(voice-intents): in-process bridge dispatch via handleLocalCommand
Voice intents on the phone were building bridge.command envelopes and
sending them over the WSS multiplexer to the relay, which correctly
rejected them with `ignoring unexpected bridge.command from phone`. The
bridge.command wire protocol is server→phone only — phone-originated
commands have no inbound route on the relay side. Caught by Bailey's
on-device test 2026-04-14 after the multiplexer-wiring fix (a568366)
unblocked the dispatch path enough to surface this protocol mismatch.

Voice intents are phone-local: the classifier, resolver, action executor,
and accessibility service all run in this process. Round-tripping over
WSS adds latency, burns bandwidth, and (as we discovered) doesn't even
work. The fix is to dispatch in-process while still going through the
existing BridgeCommandHandler dispatch + Tier 5 safety pipeline so
destructive-verb modals, blocklist gating, and idle auto-disable timer
still apply.

## Approach

1. Add a coroutine-context-element marker `LocalDispatch` in
   BridgeCommandHandler.kt. Per-coroutine, so concurrent WSS-incoming
   dispatches are unaffected.

2. Add a public `suspend fun handleLocalCommand(envelope)` entry point
   that wraps the existing `dispatch()` in `withContext(LocalDispatch())`.
   In-process callers (voice today, anything else later) call this
   instead of going through the multiplexer.

3. Make `respond` and `respondFromResult` suspend (mechanical change —
   all 87 call sites are already inside the suspend `dispatch()`
   function). In `respond()`, check for the LocalDispatch context
   element and skip `multiplexer.send` when present — voice doesn't
   need the response payload, and sending it would just bounce back
   through the same WSS protocol mismatch.

4. Wire the local dispatcher through:
   - Add `LocalBridgeDispatcher` typealias to both flavors'
     VoiceBridgeIntentFactory.kt (sideload uses it; googlePlay defines
     it for signature parity but the no-op factory ignores it)
   - `RealVoiceBridgeIntentHandler` takes an optional
     `localBridgeDispatcher` and prefers it over `multiplexer.send`
     in `dispatch()`. Falls back to the multiplexer (with a WARN log)
     if null — known broken in production but preserves test harness
     compatibility.
   - `VoiceViewModel.initialize()` takes the dispatcher and passes
     through to the factory
   - `RelayApp` wires
     `connectionViewModel.bridgeCommandHandler::handleLocalCommand` as
     the dispatcher
   - `ConnectionViewModel.bridgeCommandHandler` is now public so
     RelayApp can grab the method reference

## Thread safety

The LocalDispatch context element is per-coroutine, not per-instance,
so a WSS-incoming dispatch in flight at the same time as a voice
dispatch will NOT see the LocalDispatch marker on its own coroutine
context. The two paths are fully independent. Verified by reading
through the dispatch flow — no shared mutable state between the two
entry points other than the BridgeSafetyManager singleton, which is
already thread-safe by design.

## What this preserves

Every guarantee the existing bridge command pipeline provides:
- Tier 5 blocklist check (banking apps etc still blocked from voice)
- Destructive verb modal (voice "text Sam saying X" still requires
  user tap on the overlay before SMS leaves)
- Idle auto-disable timer reschedule on every accepted command
- Master-toggle gate (bridge disabled = voice intents return 403)

What's different vs the WSS path: no `bridge.response` envelope sent
back. Voice doesn't read responses, so this is a no-op functionally.
The full action result is still logged via the executor's own logging
for debugging.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 21:30:00 -04:00
Bailey DixonandClaude Opus 4.6 8d418efba0 docs(roadmap): expand v0.4.1 — voice local dispatch + tiered perms + JIT errors
Three follow-ups discovered during the v0.4.0 on-device voice flow test:

1. Voice intent local dispatch loop — the v0.4 voice handler builds
   bridge.command envelopes and routes them through the multiplexer →
   WSS → relay path, but the relay correctly rejects them with
   "ignoring unexpected bridge.command from phone" because bridge.command
   is server→phone-only by design. Voice intents are phone-local and
   should dispatch in-process via a new BridgeCommandHandler.handleLocalCommand
   entry point that preserves the safety modal pipeline.

2. Tiered permission checklist — current BridgePermissionChecklist only
   covers 4 special perms (Accessibility, Screen Capture, Overlay,
   Notification Listener). The sideload manifest declares READ_CONTACTS,
   SEND_SMS, CALL_PHONE, ACCESS_FINE_LOCATION but they're never surfaced
   anywhere in the UI — users have to know to grant them via Android
   Settings. Tiered design with sideload-only sections + optional badges.

3. JIT permission errors with agent recommendations — split resolver
   returns into a sealed class so PermissionDenied is distinct from
   NotFound, add a `code` field to bridge tool error envelopes, and the
   voice/chat flows surface "open Settings to grant" with deep-link
   instead of generic "I couldn't find" messages. Agent gets enough
   structured context to recommend the fix instead of hallucinating.

All three caught by on-device test 2026-04-14 after the multiplexer
wiring fix (a568366) unblocked voice dispatch and revealed the wire
protocol mismatch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 21:20:03 -04:00
Bailey DixonandClaude Opus 4.6 a568366e6c fix(voice-intents): wire bridge multiplexer into VoiceViewModel.initialize
The sideload voice→bridge intent handler was silently no-op'ing on every
classified utterance: the classifier matched ("open Chrome" → OpenApp),
the resolver found the package, the envelope was built — but
`dispatch()` bailed on its null-guard because nothing ever passed a
ChannelMultiplexer to `VoiceViewModel.initialize()`. The optional-with-
default-null parameter was introduced "to keep existing call sites
compiling" but the call site in RelayApp was never updated.

Symptom from the on-device test:
- Voice mode showed "Open App: Open Chrome." in the recognition row
  (correct — that's the VoiceViewModel display format for a successful
  classification with a null spokenConfirmation)
- Chrome never actually launched
- No /open_app envelope hit the relay journal at all
- Same bug for /send_sms — except contact resolution would have failed
  first anyway (separate issue, READ_CONTACTS permission likely missing
  — to be diagnosed after this fix lands)

Fix: pass `connectionViewModel.multiplexer` into the initialize() call
in RelayApp. Documented inline because the optional default is a
foot-gun that we should remove in a follow-up (make the parameter
required, with the googlePlay no-op factory taking a non-null mux it
ignores) but for v0.4.0 the inline note + working call site is enough.

Caught by Bailey's on-device voice flow test 2026-04-14 — the smoke
test cannot catch this class of bug because it exercises the bridge
HTTP endpoints directly, not the voice→bridge integration path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 21:07:59 -04:00
Bailey DixonandClaude Opus 4.6 85cf6b4136 fix(bridge-smoke): foreground hermes app for clipboard round-trip
Android 10+ blocks background clipboard READS — only the app with
current window focus can call ClipboardManager.getPrimaryClip(). Writes
are unrestricted. The smoke test was hitting this: the write succeeded
with length: 23 but the subsequent read returned "" because the device
had been pressed-home before the round-trip and the hermes app no longer
had focus.

Fix: foreground com.axiomlabs.hermesrelay.sideload via /open_app right
before the write, then drop back to home after the read for cleanup.
The new /open_app call counts as its own smoke path, so total becomes
17 (was 16). Cleanup home press is the 18th.

Verified locally that this is a system policy, not a Kotlin handler bug:
the previous run's first /clipboard read returned the sentinel from the
prior smoke run, proving the read path itself works.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 20:40:03 -04:00
Bailey DixonandClaude Opus 4.6 5cc8036f3c chore: fold parallel main-worktree WIP into bridge feature branch
Folds an in-progress parallel reorg from the main worktree into the
v0.4 bridge feature branch so it ships as one release. The WIP was
never committed on main; merged via stash → feature branch.

Major surface-area changes:
- pairing UX rewrite: ConnectionWizard (+597), QrPairingScanner (+494),
  OnboardingScreen/Page (+281), new PairScreen.kt
- ScreenCapture rework (+369) — kept compatible with v0.4 smoke-tested
  bridge handlers (verified 14/16 paths green on hermes-host)
- BridgeForegroundService rework (+164)
- AuthManager additions (+82)
- new ComposeArrWorkaround util + hookup in BridgeStatusOverlay
  (additive — the v0.4 SavedStateRegistryOwner fix is preserved)
- new .githooks/ scripts, CONTRIBUTING.md, hermes-relay-doctor skill
- AGENTS.md doc rewrite (+428), assorted user-docs touch-ups
- removed legacy plugin/skills/android/SKILL.md (replaced by
  skills/devops/hermes-relay-doctor)

Conflict resolution: 4 files (README.md, install.sh, two user-docs
pages) had overlapping edits between this WIP and the feature branch.
Resolved to feature-branch HEAD (Updated upstream) per Bailey — keeps
v0.4 install.sh refspec-widening + branch-flag ergonomics + the
detailed v0.4 README capability list.

AGENTS.md.bak intentionally left untracked.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 20:12:08 -04:00
Bailey DixonandClaude Opus 4.6 63a60ea077 fix(bridge-smoke): bash brace-parsing corrupts every POST body
The smoke script has been silently sending broken JSON bodies on every
POST since I wrote it. Discovered during the v0.4 on-device test when
every /press_key /open_app /clipboard write came back from the relay
with `400 missing 'X' in body`, even though the script's body literal
was correct.

The bug
- The curl_path helper used `-d "${body:-{}}"` to default an empty body
  to `{}`. Bash's parameter expansion parser closes `${body:-{}` at the
  FIRST `}`, then treats the second `}` as a literal character outside
  the expansion. So:
    body='{"key":"home"}'
    echo "${body:-{}}"
    # prints: {"key":"home"}}        (note the doubled trailing brace)
- The double-brace JSON is malformed. The relay's request.json() raises
  ValueError, the except branch falls back to body={}, the phone gets
  an empty body, and every handler that requires a body field returns
  400. The relay's `bridge >>>` log line confirms this ground truth:
    bridge >>> POST /press_key body={}
- When body is genuinely empty/unset, the expansion accidentally
  produces `{}` (open + close brace), which parses as a valid empty
  JSON object — so the bug is invisible for paths with no required
  body fields. That's why /find_nodes "passed" — its filters are all
  optional with safe defaults.

The fix
- Replace `${body:-{}}` with an explicit if/else that doesn't put a
  literal `{` inside a `${var:-default}` expansion. The default is
  assigned to a separate variable first, then dereferenced normally
  inside the curl arg.
- Long comment in the function body documenting the gotcha + the live
  symptom + the date so the next person to touch this file doesn't
  reintroduce the same shortcut.

Caught by the v0.4 on-device smoke run (Samsung S938U / hermes-host
Docker-Server). Direct curl from the host's command line worked fine
when the body was inline-quoted, which is how I confirmed the script
itself was at fault rather than the relay or the phone.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 19:45:31 -04:00
Bailey DixonandClaude Opus 4.6 040d6a18a1 fix(action-executor): narrow service param to HermesAccessibilityService
Compile error blocking the v0.4 sideload build:

  e: ActionExecutor.kt:557:29 Unresolved reference 'reader'.

Wave 1/2/3 added the M4 long-press resolver at line 556-557:

    val target: AccessibilityNodeInfo? =
        service.reader.findNodeById(ownedRoots, nodeId!!)

…but `service` was typed as the framework `AccessibilityService` base
class, which doesn't have a `reader` property. The `reader` getter lives
on our `HermesAccessibilityService` subclass at line 139 of that file:

    val reader: ScreenReader get() = screenReader

Nobody caught this because nobody compiled the integration branch end-
to-end before the on-device test today — kotlinc never ran on this
file in the wave 1/2/3 worktrees, presumably because the per-agent
worktrees were each isolated to their own slice of the codebase.
Caught the moment we tried `gradlew assembleSideloadDebug` against the
full integration tree.

Fix
- Narrow the constructor parameter type from `AccessibilityService` to
  `HermesAccessibilityService`. The single caller at
  HermesAccessibilityService.actionExecutor passes `this`, so the
  narrowed type is always satisfied — no runtime cast, no behavior
  change.
- All existing `service.dispatchGesture(...)`, `service.performGlobalAction(...)`,
  `service.startActivity(...)`, etc. calls in this file still compile
  because HermesAccessibilityService extends AccessibilityService.
- Added a one-paragraph comment above the class explaining the
  narrowing rationale so the next reader knows why we don't accept the
  framework base class.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 19:37:52 -04:00
Bailey DixonandClaude Opus 4.6 612f253b57 fix(bridge-overlay): wire SavedStateRegistryOwner on OverlayLifecycleOwner
Crash on enabling the persistent status chip in the Bridge Safety screen:

  java.lang.IllegalStateException: Composed into the View which doesn't
      propagateViewTreeSavedStateRegistryOwner!
    at androidx.compose.ui.platform.AndroidComposeView.onAttachedToWindow(AndroidComposeView.android.kt:2234)
    at com.hermesandroid.relay.bridge.BridgeStatusOverlay.setChipVisible:124
    at com.hermesandroid.relay.viewmodel.BridgeViewModel$3$3.emit:199

Captured on Samsung S24 / Android 14 on 2026-04-14 during the v0.4
on-device verification session. Every overlay attach hit this —
including the destructive-verb confirmation modal, not just the
chip — so the Phase 3 safety-rails modal was also un-shippable as of
the pre-fix build.

Root cause
- OverlayLifecycleOwner implemented LifecycleOwner + ViewModelStoreOwner
  only. It deliberately skipped SavedStateRegistryOwner on the theory
  "the overlay composables use plain `remember`, not `rememberSaveable`,
  so no saved state is needed."
- That theory is wrong. `AndroidComposeView.onAttachedToWindow` runs a
  hard precondition check for ViewTreeSavedStateRegistryOwner on the
  attached view's tree, regardless of whether the composable body ever
  reads or writes saved state. The check throws IllegalStateException
  on null — no fallback, no graceful degradation.
- The KDoc and CLAUDE.md entry both claimed SavedStateRegistryOwner was
  "deliberately skipped" with the above rationale. Both were wrong and
  have been updated.

Fix
- Add `androidx.savedstate` imports: SavedStateRegistry,
  SavedStateRegistryController, SavedStateRegistryOwner, and the
  View.setViewTreeSavedStateRegistryOwner extension.
- Add SavedStateRegistryOwner to OverlayLifecycleOwner's interface list.
- Create a `SavedStateRegistryController.create(this)` backing field and
  expose it via `override val savedStateRegistry`.
- Update `start()` to follow the required init sequence:
    registry.currentState = CREATED        // required BEFORE performRestore
    savedStateController.performRestore(null)  // empty bundle = fresh state
    registry.currentState = RESUMED
  Calling performRestore while still at INITIALIZED trips a second
  assertion inside SavedStateRegistryController.
- In `attachLifecycle`, add the third call:
    view.setViewTreeSavedStateRegistryOwner(owner)
  Applies to both the chip ComposeView (line 106) and the confirmation
  modal ComposeView (line 164) since they share the same attach helper.

Artifact availability
- `androidx.savedstate.savedstate` is a transitive dependency of
  `androidx.lifecycle:lifecycle-runtime-ktx:2.10.0` (already in the
  version catalog), so no new dependency line in libs.versions.toml is
  required. The extension function
  `View.setViewTreeSavedStateRegistryOwner` ships in the same artifact.

Docs
- CLAUDE.md BridgeStatusOverlay row rewritten to document all three
  tree owners + the init sequence constraint + the crash provenance.
- BridgeStatusOverlay.kt class KDoc and OverlayLifecycleOwner KDoc
  updated to reflect the actual requirement instead of the skipped-
  on-purpose folk wisdom.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 18:40:41 -04:00
Bailey DixonandClaude Opus 4.6 c65851aa37 fix(bridge-smoke): /events is GET, not POST
Caught during the first live smoke run on hermes-host: my script was
sending POST /events with a body, which the relay's HTTP router rejects
with 405 Method Not Allowed in ~11ms (fast because it never round-trips
to the phone).

The relay registers /events as:

    app.router.add_get("/events", handle_bridge_events_recent)

…matching the Python tool's `_get(f"/events?limit={limit}&since={since}")`
call shape. Query params, not JSON body.

Swap the smoke call to GET with an inline query string. No other test
paths need fixing — verified against the full route registration
block in plugin/relay/server.py (GET: /ping /screen /screenshot
/get_apps /apps /current_app /clipboard /screen_hash /events. POST:
everything else on the bridge surface).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 18:36:00 -04:00
Bailey Dixon 27fc43ec10 Merge origin/main into feature/bridge-feature-expansion: pick up 4de05c9 bootstrap fix 2026-04-14 18:25:04 -04:00
Bailey DixonandClaude Sonnet 4.6 4de05c9458 fix(bootstrap): drop skills_categories import removed by upstream
Upstream commit 8d023e43 removed `skills_categories` from
`tools/skills_tool.py` as dead code. The bootstrap's `_resolve_upstream()`
imported it unconditionally, which raised ImportError and silently prevented
all injected /api/* routes from registering on post-sync vanilla upstream
(the try/except in _maybe_register_routes caught it, gateway started fine,
but sessions/memory/skills/config were all missing).

Fix: remove the import and the /api/skills/categories route entirely. The
app never calls this endpoint — skill browsing uses /api/skills?category=.
Upstream removed it as dead code; we don't re-introduce it.

Updated _INJECTED_PATHS in _patch.py and module docstring in _handlers.py
to document both deliberate omissions (chat/stream and skills/categories).
Updated docs/decisions.md and docs/HERMES-WEBAPI-REFERENCE.md to reflect
the upstream alignment rationale.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 18:15:14 -04:00
Bailey DixonandClaude Opus 4.6 e0866ba475 fix(install.sh): widen remote refspec so --branch works on existing clones
Pre-v0.4 installs cloned the repo with `git clone --single-branch`, which
pins the remote refspec to `+refs/heads/main:refs/remotes/origin/main`.
That means on an existing clone:

  git fetch origin feature/foo   # populates FETCH_HEAD, fine
  git checkout feature/foo       # FAILS — no remote-tracking ref exists
                                 #   and no local branch of that name

…and install.sh's step [1/6] line does exactly that sequence, so
`HERMES_RELAY_BRANCH=feature/foo hermes-relay-update` and
`hermes-relay-update --branch feature/foo` both silently break on any
clone that pre-dates this fix.

Caught live on hermes-host (Docker-Server) on 2026-04-14 while
bootstrap-installing the feature/bridge-feature-expansion branch for
v0.4 on-device testing. The manual workaround was:

  cd ~/.hermes/hermes-relay
  git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
  git fetch origin
  git checkout feature/bridge-feature-expansion

After which install.sh worked cleanly. This commit rolls that
workaround into install.sh itself as an idempotent step so:
  * Existing single-branch clones get widened on next `install.sh` run
  * `git config` is a no-op when the refspec is already wide, so
    running this on a normal clone doesn't disturb anything
  * Future branch switches via --branch / HERMES_RELAY_BRANCH work
    without any user-side surgery

Also drops `--single-branch` from the fresh-clone path on line 240.
New installs now get the standard wide refspec from the start. Cost is
a few extra KB of refs data per clone, which is negligible next to the
ergonomic win of "switching branches just works" for every future
installation.

Out of scope: the pre-v0.4 shim on already-installed hosts still
hardcodes the main URL for install.sh, so they can't use --branch
directly via `hermes-relay-update`. Those hosts need one of:
  1. `HERMES_RELAY_BRANCH=<branch> hermes-relay-update` (the env var
     form bypasses the shim's lack of --branch parsing — main's
     install.sh honors HERMES_RELAY_BRANCH natively)
  2. Direct curl: `curl -fsSL .../feature/foo/install.sh | bash -s -- --branch feature/foo`

Either path gets them to a post-fix state where regular
`hermes-relay-update --branch <whatever>` works going forward.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 17:04:53 -04:00
Bailey DixonandClaude Opus 4.6 12204fddc0 fix(v0.4): close C4 + H5 voice intent TODOs, add install.sh --branch, test_android_read_screen
Three open TODOs from the v0.4 hand-off, plus an install.sh ergonomic
improvement that came up while planning on-device verification.

C4 — voice contact-name → phone resolution
- RealVoiceBridgeIntentHandler now resolves the spoken contact name to
  a real phone number BEFORE dispatching /send_sms, instead of passing
  the raw name (which the phone-side regex check rejected, surfacing
  as a user-visible error every time).
- New private helper resolveContactPhone() calls
  HermesAccessibilityService.instance.actionExecutor.searchContacts(
  contactName, limit=5) directly. Same code path as the /search_contacts
  bridge route but invoked locally — no response-correlation plumbing
  needed because both the voice handler and the accessibility service
  are in the same process.
- searchContacts returns phones as a comma-joined string ("+1555..., +1555...")
  per the C2 wire shape; the resolver splits on `,`, trims whitespace,
  and takes the first non-blank entry.
- Wrapped in withContext(Dispatchers.IO) — ContactsContract queries
  hit the on-device content provider and can block.
- If the resolver returns null (service unbound, contacts permission
  missing, no matching contact, or matched contact has no phone), the
  intent surfaces as a friendly spoken response ("I couldn't find a
  contact called Sam.") instead of dispatching a broken envelope.
- The destructive-verb confirmation modal still fires on the
  BridgeCommandHandler side, so the resolver is purely additive — it
  doesn't bypass any safety gate.

H5 — voice app-name → package resolution
- Same shape as C4 but for /open_app. New resolveAppPackage() helper
  calls service.packageManager.queryIntentActivities(ACTION_MAIN +
  CATEGORY_LAUNCHER) and fuzzy-matches the spoken app name against
  the launchable-app inventory.
- Three-tier match (case-insensitive, first hit wins):
    1. Exact label match — "spotify" → "Spotify"
    2. Prefix match — "chro" → "Chrome Beta"
    3. Substring match — "google" → "Google Maps"
  Order matters: tier 1 + 2 prevent accidental substring hits like
  "messages" matching "Google Messages" instead of the literal
  Messages app.
- Requires the <queries><intent action=MAIN category=LAUNCHER/></queries>
  manifest declaration that landed in the previous commit (7755851) —
  without it Android 11+ silently returns a near-empty candidate list.
- Wrapped in withContext(Dispatchers.IO) — PackageManager queries
  can be slow on devices with many apps.
- Same null/error surfacing pattern as C4. BridgeCommandHandler still
  blocklist-checks the resolved target package before launching, so
  voice-utterance intents to open blocked banking apps stay blocked.

buildSmsEnvelope() and buildOpenAppEnvelope() now take 2 args (the
intent + the resolved value). Both call sites in tryHandle updated.

install.sh --branch flag (+ HERMES_RELAY_INSTALL_URL on the shim)
- Adds a CLI flag to install.sh that overrides HERMES_RELAY_BRANCH.
  Flag wins over env var wins over the default ("main"). Same
  precedence pattern as HERMES_RELAY_HOME / HERMES_VENV_PY.
- The hermes-relay-update shim now reads HERMES_RELAY_INSTALL_URL to
  override the install.sh source URL. Bootstrap caveat documented in
  the shim header: switching to a feature branch BEFORE that branch
  is merged to main needs the env-var override because the shim
  normally curls install.sh from main, which won't have the
  --branch flag yet.
- After the bootstrap install completes the host has the new
  install.sh on disk + the shim handles all subsequent updates,
  including switching back to main via `hermes-relay-update`
  (no flag needed, defaults to main).
- Documented the flag and the bootstrap escape hatch in the shim
  doc-comment + the install.sh header.

plugin/tests/test_android_read_screen.py
- 11 stdlib-unittest cases mirroring the test_android_search_contacts
  pattern (no pytest, no responses, runs as
  `python -m unittest plugin.tests.test_android_read_screen`).
- Coverage: happy path with default include_bounds=False, explicit
  include_bounds=True/False, empty node list, service-not-connected
  503 passthrough, ConnectionError network failure, requests.Timeout,
  schema registration sanity, and handler dispatch from the
  _HANDLERS dict with both default and explicit args.
- All 11 tests pass locally (`python -m unittest plugin.tests.test_android_read_screen`).
- Caught a documentation drift along the way: android_read_screen
  in the integration branch sends `?include_bounds=` (wave 1/2/3
  rename) but main still sends `?bounds=`. The test asserts the
  current branch shape; the rename will land in main as part of the
  v0.4 merge.

Out of v0.4 scope, NOT touched
- The original C4 doc-comment proposed a response-correlation rewrite
  of ChannelMultiplexer to wire bridge.command → bridge.response
  request_id matching. Local resolution sidesteps that requirement
  entirely. The correlation pattern is still useful for other
  cross-channel flows (e.g. a future agent-driven UI test framework)
  and is tracked separately rather than deferred under the C4 label.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 13:46:08 -04:00
Bailey DixonandClaude Opus 4.6 b1f4822ce3 test(bridge): add scripts/bridge-smoke.sh end-to-end smoke runner
End-to-end smoke test for the bridge channel HTTP surface. Curls every
documented bridge route via the unified relay (default localhost:8767)
to verify the round-trip relay → WSS → phone → handler → response works
after a code change or relay restart. Exists specifically to catch the
silent-drop class of regression that bit us in v0.3.0 → v0.4.0 (Python
side registered /open_app /get_apps /setup, Kotlin dispatcher had no
matching `when (path)` branch, commands fell through to the unknown-
path 404 with no warning).

Why a shell script not pytest
- The whole point is verifying the live WSS round-trip with a real
  phone — there's nothing meaningful to mock. pytest discovery on a
  CI box that has no phone is dead weight.
- Shell is faster to iterate on, output is paste-friendly into chat,
  zero dependencies beyond bash + curl.
- Plays well with the existing scripts/ convention (sibling of
  dev.sh, gen-dev-cert.sh).

Why hermes-host not local PC
- The relay binds localhost:8767 for tool callers; running curl from
  hermes-host mirrors exactly how plugin/tools/android_tool.py talks
  to the relay. Local PC would need to punch the bearer token across
  the LAN, which is a different threat model than what's actually
  exercised in production.
- The bearer token already lives in ~/.hermes/.env on the host, so
  there's nothing to copy.

Test inventory
- Read-only suite (always runs, no side effects):
    /setup, /current_app, /get_apps, /apps (legacy), /screen,
    /screen_hash, /find_nodes, /clipboard GET, /events, /screenshot
- Destructive suite (default ON, --no-destructive to skip):
    /press_key home, /open_app com.android.chrome, /press_key back,
    /press_key home (cleanup), /clipboard write+read round-trip
- Clipboard round-trip is the cheapest "did the WSS path actually
  work" sanity check — write a unique sentinel, read it back, fail
  if the buffer mismatches. Catches dispatcher/session bugs a status-
  code-only check would miss.

Flags
- --pair CODE pre-registers a pairing code via /pairing/register
  (clears rate-limit blocks per ADR 15) and continues. Token still
  comes from .env — the phone-side claim is out of band.
- --filter PATTERN re-runs a single test by regex match — handy for
  iterating on a single broken handler without going through the
  whole suite.
- --token / --relay overrides for one-off testing against a non-
  default setup.
- --quiet collapses passes to one-line bullets, keeps failures verbose.

Output and exit codes
- Per-test pass/fail with HTTP status, elapsed ms, and 100-char body
  snippet (color-coded if stdout is a tty, plain otherwise so logs
  paste cleanly into chat / GitHub issues).
- Failed test names collected and re-printed at the bottom with a
  copy-paste --filter command for re-running just the broken one.
- Exit 0 = all pass, 1 = at least one test failed, 2 = preflight
  failed (relay down, no token, phone not connected). The 2 vs 1
  split lets shell wrappers distinguish "infrastructure broken" from
  "feature broken".

Preflight gate
- /health (no auth)
- token loaded from ~/.hermes/.env (or --token override)
- /ping with bearer (verifies a phone is actually connected over WSS,
  not just that the relay process is alive)

CLAUDE.md updated with a Dev Workflow → "Bridge smoke test" subsection
documenting the canonical post-relay-restart workflow.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 12:59:46 -04:00
Bailey DixonandClaude Opus 4.6 77558517c1 fix(v0.4): port missing /open_app /get_apps /setup Kotlin handlers + docs sweep
Latent v0.3.0 regression surfaced by the v0.4 gap-fix audit on 2026-04-14:
the Python relay registered POST /open_app, GET /get_apps (+ /apps legacy
alias), and POST /setup during Phase 3 Wave 1, but the Kotlin
BridgeCommandHandler never wired the matching `when (path) ->` branches.
Commands silent-dropped into the unknown-path 404, so android_open_app()
and android_get_apps() have been broken since v0.3.0 shipped — discovered
when the gap-fix agent cross-referenced the Python tool surface against
the Kotlin dispatcher.

Bridge handler changes
- /open_app: getLaunchIntentForPackage(pkg) + FLAG_ACTIVITY_NEW_TASK +
  service.startActivity(intent). Defense-in-depth blocklist check on the
  *target* package (mirrors the /send_intent + /broadcast B4 pattern,
  since the existing gate only checks the foreground app). Returns 400
  on missing package, 403 on blocked, 404 on no-launch-intent, 500 on
  startActivity failure.
- /get_apps + /apps (legacy alias): enumerates launchable apps via
  PackageManager.queryIntentActivities(ACTION_MAIN + CATEGORY_LAUNCHER),
  returns {apps: [{package, label}], count}. Pure-read so it inherits
  the master + foreground-blocklist gate already on the dispatch path.
- /setup: 200 no-op early-return next to /ping and /events. The Python
  android_setup() in plugin/tools/android_tool.py is host-side only —
  it writes ANDROID_BRIDGE_TOKEN to ~/.hermes/.env and never forwards
  to the phone — but the route is still registered on the relay, so
  answering 200 instead of 404 keeps the audit clean.

Manifest
- Added <queries><intent action=MAIN category=LAUNCHER/></queries> to
  app/src/main/AndroidManifest.xml. Required on Android 11+ (API 30+)
  for queryIntentActivities to return real data — without it the call
  silently returns a near-empty list. Play-policy-safe (no
  QUERY_ALL_PACKAGES). Also retroactively fixes a latent gap in
  BridgeSafetySettingsScreen.kt:399 where the blocklist UI uses the
  same call from a regular Activity context and was likely showing a
  truncated set on real devices.

Docs (user-docs sweep + CHANGELOG)
- CLAUDE.md: rewrote the BridgeCommandHandler row to list the full
  post-v0.4 path inventory (was stuck at the v0.2 inventory — wave
  1/2/3 added ~16 paths without updating it).
- CHANGELOG.md: drafted [0.4.0] - 2026-04-14 entry covering all 19
  integration commits, broken into Read / Act / Tier-C / Docs / Fixed
  sections.
- user-docs/reference/relay-server.md: added the full 27-route bridge
  HTTP inventory, /notifications/recent and /bridge/status rows, plus
  a Tier 5 gating note.
- user-docs/architecture/index.md + HermesFlow.vue: retired the stale
  ":8766 standalone bridge relay" — bridge is unified on :8767 since
  Phase 3 Wave 1.
- user-docs/architecture/decisions.md: rewrote ADR-3 (unified relay,
  with v0.2→v0.3 history), added ADR-9 (five-stage safety gate),
  ADR-10 (wake-scope), ADR-11 (event stream), ADR-12 (Android 14
  MediaProjection FGS type), ADR-13 (build flavors). Removed "voice
  mode" from deferrals (shipped in v0.3).
- user-docs/architecture/security.md: rewrote "Bridge Security" as
  "Five-Stage Safety Gate" with the Tier 5 pipeline, bypass list, and
  sideload-only tier caveats.
- user-docs/features/index.md: removed the orphaned "Calendar read"
  bullet — no android_calendar tool, no /calendar route, doc rot from
  the doc2 sweep.

RELEASE_NOTES.md is intentionally NOT touched in this commit — it
still holds the v0.3.0 release notes and gets rewritten as part of the
v0.4 release-cut commit (matches the v0.3 pattern). A draft is
available in the docs subagent transcript.

Notes for on-device verification
- Build the sideload flavor and call android_get_apps() — should
  return the full launchable app list rather than the truncated
  v0.3.0 set.
- android_open_app("com.android.chrome") (or any installed package)
  should actually launch the app rather than silently no-op.
- BridgeSafetySettingsScreen blocklist picker should now show the
  full installed-app list after the manifest <queries> change.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 11:23:46 -04:00
Bailey DixonandClaude Opus 4.6 8fe34dc41b fix(v0.4): code-review gap fixes for bridge feature expansion
HIGH
- H1/M5 (relay GET dispatch): merge query params into bridge envelope
  body so phone-side handlers see them. Fixes android_events limit/since
  and android_read_screen include_bounds silently defaulting. Also
  renamed the android_read_screen query key from `bounds` to
  `include_bounds` so the Python tool key matches what the Kotlin
  handler reads.
- H2 (searchNodes walker): hoist nextIndex counter outside the per-window
  loop AND switch the increment predicate to match walk/findNodeById's
  canonical "interesting" filter so IDs from find_nodes match the global
  scheme readAllWindows + findNodeById use. Previously nodeIds from
  find_nodes silently resolved to the wrong node on multi-window screens
  (and could drift even on single-window screens because the increment
  fired on every visit, not only on emitted nodes).
- H3 (scroll leak): recycle rootInActiveWindow node in a try/finally.
- H4 (SMS multipart): cache divideMessage result, don't call twice.
- H5 (voice intents): convert OpenApp/Tap/Scroll/Back/Home builders to
  bridge.command envelope shape matching SendSms. Previously these
  dispatched tool.call envelopes that BridgeCommandHandler silently
  dropped — the intents appeared to succeed but never reached the phone.

MEDIUM
- M1 (/call /send_sms): return 503 when safetyManager is null instead
  of silently bypassing the confirmation modal.
- M2 (EventStore TOCTOU): re-check isStreaming inside the lock in
  append so setStreaming(false) → clear is atomic.
- M4 (longPress nodeId): use reader.findNodeById instead of
  viewIdResourceName match, matching /tap and /scroll semantics and
  the Python schema's advertised contract. Removed the now-unused
  findNodeByResourceId helper.

LOW
- L3 (WakeLockManager.initialize): synchronized check-and-set to
  close the theoretical concurrent-init race.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:46:39 -04:00
Bailey DixonandClaude Opus 4.6 3dc41947c1 Merge feature/doc-spec-status-refresh: 15-item spec rot cleanup
Resolved conflicts in docs/spec.md by keeping Doc3's authoritative
§6.4 v0.4 rewrite (tool surface tables + architectural patterns)
and consolidating the Phase 3 checklist to include both the
ADR-15-era security items and the v0.4 expansion bullet.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:31:52 -04:00
Bailey Dixon fece0a1ff6 Merge feature/doc3-spec-bridge-surface: spec.md + decisions.md updates 2026-04-13 22:28:38 -04:00
Bailey Dixon 2cc8b76ee6 Merge feature/doc2-readme-features: README + features doc sweep 2026-04-13 22:28:34 -04:00
Bailey DixonandClaude Opus 4.6 407efc6d63 Merge feature/C1-C4-sideload-perms: location/contacts/call/sms tools
Resolved conflicts:
- ActionExecutor.kt imports merged (added annotation/PendingIntent/
  BroadcastReceiver/IntentFilter/PackageManager/Location/LocationManager/
  Build/ContextCompat/withTimeoutOrNull); Tier C constants joined the
  long_press/parent_walk constants in the companion. Functions
  location/searchContacts/makeCall/sendSms not wrapped in wakeForAction.
- BridgeCommandHandler.kt imports added BuildFlavor + EventStore from
  both branches; /location/search_contacts/call/send_sms cases added
  after /clipboard /media. /call and /send_sms unconditionally call
  safetyManager.awaitConfirmation.
- server.py routes + handlers (additive).
- android_tool.py docstring + function defs + schemas + handlers.

Other C1-C4 files (FeatureFlags BuildFlavor.isSideload helper, sideload
manifest permissions, VoiceBridgeIntentHandlerImpl /send_sms wiring)
auto-merged cleanly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:28:29 -04:00
Bailey DixonandClaude Opus 4.6 937580d072 Merge feature/B1-event-stream: EventStore + events/event_stream tools
Resolved conflicts: server.py handlers + routes (additive), and
android_tool.py (docstring, function defs, schema block, handlers
map). ConflResolution kept all pre-existing tools and appended
android_events + android_event_stream to the end of each list.

EventStore.kt is a new file (no conflict). HermesAccessibilityService
auto-merged (onAccessibilityEvent hook + EventStore.append call).
BridgeCommandHandler auto-merged (/events + /events/stream cases).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:20:48 -04:00
Bailey DixonandClaude Opus 4.6 2da98cd0ab Merge feature/B4-send-intent: raw Intent + broadcast escape hatches
Resolved conflicts:
- ActionExecutor.kt imports merged (ActivityNotFoundException,
  ComponentName, Uri joined with A6's ClipData/Manager/Context and
  A9's Rect, keeping WakeLockManager import). sendIntent/sendBroadcast
  are NOT wrapped in wakeForAction — they're Intent dispatches, not
  gesture strokes.
- server.py handlers (both function defs + route registrations).
- android_tool.py docstring, function defs, schema block (macro +
  clipboard + screen_hash/diff_screen + send_intent + broadcast all
  preserved additively), and handlers map.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:16:46 -04:00
Bailey DixonandClaude Opus 4.6 25970c3e24 Merge feature/A4-describe-node: describe_node + nodeId tap/scroll
Resolved conflicts:
- ActionExecutor.scroll gained optional centerX/centerY params (A4)
  while preserving the A8 WakeLockManager.wakeForAction wrapper.
- BridgeCommandHandler docstring + /tap + /scroll + /describe_node
  cases auto-merged (additive).
- ScreenReader.findNodeById walker ALIGNED to P1 interesting-only
  emission semantics. A4's original pre-order-total counter would
  have broken nodeId round-trip against P1's `w<win>:${out.size}`
  scheme. Rewritten walkForId to replicate P1's `interesting`
  predicate exactly (text/contentDesc/clickable/longClickable/
  scrollable/editable AND non-empty bounds) and share the counter
  GLOBALLY across windows, matching readAllWindows' shared
  `collected` list. Earlier/later windows still contribute to the
  global counter but can only claim the match when currentWindow
  == wantedWindow.
- android_tool.py docstring, function defs, schema block, handler
  map — kept all additive entries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:11:37 -04:00
Bailey DixonandClaude Opus 4.6 3210d382d7 docs(spec): additional rot pass — 7 more stale items
Follow-up to previous spec status refresh. Fixes:
- §4 Tech Stack: 8766 → 8767 with Phase 3 Wave 1 migration note
- §4 Tech Stack: DI decision made (manual, not Hilt) + storage
  updated to Keystore-first with EncryptedSharedPreferences fallback
- §5 Settings Tab: describe unified Pair-with-your-server card
  (replaces separate API Server / Relay Server entries), plus new
  Voice and Notification Companion sub-screens
- §6.4 Bridge Channel: narrative updated — android_relay.py retired,
  functionality migrated to android_tool.py + channels/bridge.py
- §3.2 chat note: clarified relay scope — chat bypasses but voice,
  bridge, terminal, notifications, media go through
- §7 Phase 6 Future: check off notification listener (shipped
  v0.3.0) and clipboard bridge (shipped on v0.4 branch); add
  reverse file transfer, multi-device routing, on-device fallback,
  iOS client evaluation
- §9 Dependencies: refresh table from current gradle/libs.versions.toml
  — AGP 8.13.2, Kotlin 2.3.20, Compose BOM 2026.03.01, OkHttp 5.3.2,
  etc. Add note that this is a snapshot, not authoritative.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:11:35 -04:00
Bailey DixonandClaude Opus 4.6 f8b0195a91 Merge feature/A5-screen-hash: SHA-256 screen fingerprint + diff tool
Resolved conflicts: android_tool.py (4 spots — docstring, function
defs, schemas block where clipboard_write boundary collided with
screen_hash/diff_screen, and handlers map), server.py (handler +
route), BridgeCommandHandler.kt docstring. ScreenHasher.kt is a new
file so no conflict. Kept all additive entries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:07:20 -04:00
Bailey DixonandClaude Opus 4.6 501d0cca8c Merge feature/A3-find-nodes: filtered searchNodes tool
Resolved conflicts: ScreenReader.kt — kept A3's searchNodes +
searchWalk functions above P1's findNodeBoundsByText (not below).
server.py and android_tool.py conflicts were additive (wave-local
handler + route + tool). Test file count assertion kept as >= 14.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:04:49 -04:00
Bailey DixonandClaude Opus 4.6 b6487215be Merge feature/A9-tap-text-fallback: three-tier tapText cascade
Resolved conflicts: ActionExecutor.tapText — manually composed A9's
direct-click/parent-walk/coordinate-fallback cascade INSIDE A8's
WakeLockManager.wakeForAction wrapper, operating on P1's
snapshotAllWindows() ownedRoots (iterating each window root to find
the first match). Kept A9's private findNodeByText/findFirstNode
helpers. Kept PARENT_WALK_MAX=8 constant alongside A1's long-press
constants. Recycling contract preserves matched node through Tier 3
and releases all window roots in outer finally.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:02:43 -04:00
Bailey DixonandClaude Opus 4.6 8e402dd158 docs(spec): refresh v0.1.0-frozen status + check off shipped phases
Address 8 stale items flagged during v0.4 bridge expansion review:
- Bump header status line to current (post-v0.3.0, v0.4 in flight)
- §7 Phase 2 Terminal: check off preview shipped
- §7 Phase 4 Security: check off ADR #15 deliverables (TOFU, Keystore,
  session TTL, paired devices, transport badge, HMAC QR signing)
- §7 Phase 5 Polish/CI: check off Play Store submission, GH Actions
  release, Material You, splash
- §8 MVP Scope: preserve as historical snapshot appendix
- §3.2 auth.ok envelope: add expires_at / grants / transport_hint
  fields from ADR #15
- §8 Non-goals: remove shipped items, keep only genuine gaps
  (biometric lock, push notifications, iOS, reverse file transfer)
- §5 Bridge Tab: rewrite in present tense matching current
  BridgeScreen.kt implementation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:01:11 -04:00
Bailey DixonandClaude Opus 4.6 51dead05a3 Merge feature/A7-media: media playback broadcast
Resolved conflicts: import lists, BridgeCommandHandler case, server.py
handler+route pair, test count assertion relaxed to >= 14, tool list
docstring. ActionExecutor.mediaControl auto-merged cleanly (not wrapped
in wakeForAction since it's a broadcast, not a gesture).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 22:00:33 -04:00
Bailey DixonandClaude Opus 4.6 c2d9eb3e21 Merge feature/A1-long-press: long-press gesture
Resolved conflicts: ActionExecutor.kt — kept drag() + longPress() both
after swipe, kept Dispatchers import from A6/clipboard side. Upgraded
A1's single-root snapshotRoot() nodeId lookup to P1 multi-window
snapshotAllWindows() pattern (matches tapText). longPress takes a
viewIdResourceName nodeId (distinct from P1 walk IDs) so its own
helper findNodeByResourceId is kept as-is.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:58:55 -04:00
Bailey DixonandClaude Opus 4.6 cccafa912e Merge feature/A2-drag: drag gesture
Resolved conflicts: ActionExecutor.kt — kept Dispatchers/withContext imports (needed by clipboard).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:56:52 -04:00
Bailey DixonandClaude Opus 4.6 c5ff028000 Merge feature/A6-clipboard: clipboard read/write bridge
Resolved conflicts:
- ActionExecutor.kt imports: kept WakeLockManager + Dispatchers
- android_tool.py _SCHEMAS: both android_macro and clipboard entries
- android_tool.py _HANDLERS: added both dispatchers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:53:39 -04:00
Bailey Dixon 0424e7fafb Merge feature/A10-macro: batched workflow dispatcher 2026-04-13 21:51:29 -04:00
Bailey DixonandClaude Opus 4.6 21795a3562 Merge feature/P1-all-windows: multi-window ScreenReader
Resolved conflict in ActionExecutor.kt: combine A8 wakeForAction
wrapper with P1 multi-window snapshotAllWindows logic in both
tapText and typeText. typeText promoted to suspend fun to sit
inside wakeForAction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:51:23 -04:00
Bailey Dixon a732f86393 Merge feature/A8-wake-lock: WakeLockManager wake scope wrapper 2026-04-13 21:50:00 -04:00
Bailey Dixon 844dc6ab26 Merge feature/A11-android-skill: per-app playbook 2026-04-13 21:49:56 -04:00
Bailey DixonandClaude Opus 4.6 35808e7905 feat(C1-C4): sideload-only tier - location, contacts, call, send_sms
C1 android_location: last-known GPS via LocationManager across
providers, staleness warning, no fresh-fix requests from background.

C2 android_search_contacts: ContactsContract filter + phone number
resolution, privacy-respecting logging.

C3 android_call: auto-dial via ACTION_CALL on sideload, fallback to
ACTION_DIAL on googlePlay / permission-denied. Destructive-verb
confirmation modal gates every call.

C4 android_send_sms: direct SmsManager.sendTextMessage +
sendMultipartTextMessage with PendingIntent result callback (actual
wait for send completion, not fire-and-forget). API-version-aware
SmsManager retrieval. Voice-to-bridge SendSms intent handler now
emits a real /send_sms bridge.command envelope instead of a
malformed tool.call payload; contact->number resolution marked
TODO(C4) with a sketch since fire-and-forget dispatch lacks
response correlation. Destructive-verb confirmation modal gates
every send.

All four permissions added to app/src/sideload/AndroidManifest.xml
only - NOT the main manifest. Flavor gate via
FeatureFlags.BuildFlavor.isSideload in BridgeCommandHandler. Tools
return 'sideload-only' errors on googlePlay devices. Every call/send
logs the full payload to the safety-rails activity log via the
confirmation modal's method + text fields.

Tests: 39 stdlib-unittest cases across
plugin/tests/test_android_{location,search_contacts,call,send_sms}.py
- happy / denied / timeout / schema coverage for each tool. Existing
test_android_tool.py tool-count assertion bumped 14 -> 18.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:45:43 -04:00
Bailey DixonandClaude Opus 4.6 af1a862235 docs(doc3): document v0.4 bridge surface + new architectural patterns
Update docs/spec.md android_* tool surface with the 16 new tools
(Tier A gestures, clipboard, media, macro, Tier B events + raw
intent, Tier C sideload-only location/contacts/call/send_sms).
Add architectural-patterns subsection covering WakeLockManager
wake-scope wrapping, P1 multi-window ScreenReader, A9 three-tier
tapText cascade, and ScreenHasher content fingerprinting. Update
docs/decisions.md with ADR(s) for the load-bearing pattern
adoption.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:41:33 -04:00
Bailey DixonandClaude Opus 4.6 9bab4356aa feat(B1): add android_events + android_event_stream
Real-time AccessibilityEvent ring buffer (500 entries, thread-safe,
throttled to 1 event per type+package per 100ms). Hooks
HermesAccessibilityService.onAccessibilityEvent when streaming is
enabled (off by default — explicit opt-in via android_event_stream).
Two tools: android_events(limit, since) polls recent entries,
android_event_stream(enabled) toggles capture and clears buffer on
disable. Signal-rich event types only: click, text changed, window
content/state changed, scroll.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:39:50 -04:00
Bailey DixonandClaude Opus 4.6 241e0dc135 docs(doc2): update README + user-docs features for v0.4 bridge expansion
Refresh the Features section with user-facing language for the new
bridge tool surface: long-press/drag gestures, smarter tap
fallbacks, filtered node search, screen change detection,
clipboard bridge, system media control, macro batching,
real-time event streaming, raw Intent escape hatch. Sideload-only
additions (location, contacts, calling, SMS) called out
separately. Keeps the Features list under ~10 bullets — elevator
pitch, not full catalog.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:37:43 -04:00
Bailey DixonandClaude Opus 4.6 1ec7a2e8ee feat(B4): add android_send_intent and android_broadcast
Power-user escape hatch for launching arbitrary Activities and
sending broadcasts. Both take action/data/package/extras; send_intent
also accepts component + category. FLAG_ACTIVITY_NEW_TASK added for
Activity launches. Gated through BridgeSafetyManager package
blocklist — a blocklisted target package refuses. ActivityNotFound
and SecurityException are soft-failed into ActionResult errors,
not crashes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:20:58 -04:00
Bailey DixonandClaude Opus 4.6 1e5ad146f4 feat(A4): android_describe_node + wire nodeId for /tap and /scroll
New describe_node tool returns the full property bag (bounds, classes,
text, state flags, hintText, viewIdResourceName) for a nodeId from
the P1 stable-ID scheme. Also resolves that P1 emitted nodeIds but
/tap and /scroll ignored them — BridgeCommandHandler now parses
nodeId, resolves via ScreenReader.findNodeById, dispatches against
the node's bounds center. Closes the end-to-end nodeId contract.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:20:36 -04:00
Bailey DixonandClaude Opus 4.6 3543e6679b feat(A5): add android_screen_hash + android_diff_screen
Cheap change detection for navigation loops. SHA-256 over per-node
fingerprints (className + text + contentDescription + bounds +
viewIdResourceName) across the full multi-window accessibility
tree. diff_screen reports changed + new hash + node_count in one
call so the agent can update its reference without an extra
round-trip. ~100x cheaper than re-reading the full tree for
'did anything change?' polling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:19:07 -04:00
Bailey DixonandClaude Opus 4.6 d7ed77eebb feat(A9): three-tier tapText fallback cascade
Real-world Android apps wrap clickable content in non-clickable
text/image views all the time (Uber, Spotify, Instagram, Tinder).
Our old tapText gave up when the matched text node wasn't clickable.
New cascade: direct click (tier 1) -> parent walk up to 8 levels
(tier 2) -> coordinate tap at bounds center (tier 3). Careful node
recycling through the parent chain, bounds captured before recycling
the original. Returns a descriptive ActionResult indicating which
tier succeeded.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:18:47 -04:00
Bailey DixonandClaude Opus 4.6 9c4acbcf0f feat(A6): add android_clipboard_read and android_clipboard_write
Clipboard bridge via ClipboardManager. Label ClipData as "hermes" so
other apps can see the source. Handle empty clipboard as empty
string (not error). On API 31+ the system shows a privacy toast on
write, documented in the tool description.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:18:30 -04:00
Bailey DixonandClaude Opus 4.6 e2ba96b192 feat(A10): add android_macro batched workflow tool
Pure-Python orchestrator that dispatches to other android_* tools
in order, stopping on first failure. Returns a structured trace
with completed-count, per-step results, and error details.
Complements android_navigate (vision-driven) for known workflows
where the steps are deterministic and batching cuts round-trips.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:18:19 -04:00
Bailey DixonandClaude Opus 4.6 dd61db78d9 feat(A7): add android_media system-wide playback control
Control whatever media app is currently playing (Spotify, YouTube
Music, Pocket Casts, etc.) via ACTION_MEDIA_BUTTON broadcast with
KEYCODE_MEDIA_* keycodes. DOWN+UP ordered broadcast pair. Actions:
play, pause, toggle, next, previous. No special permissions — media
button is a system-wide interface every compliant player handles.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:17:47 -04:00
Bailey DixonandClaude Opus 4.6 164cdfb5c1 feat(A1): add android_long_press gesture tool
Long-press at coordinates or on an accessibility node via
ACTION_LONG_CLICK / GestureDescription. Wrapped in WakeLockManager
wake scope and BridgeSafetyManager package gate. Duration clamped
to 100-3000ms.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:17:26 -04:00
Bailey DixonandClaude Opus 4.6 83bfa23f6b feat(A3): add android_find_nodes filtered search
Targeted node search by text/class/clickable criteria across all
accessibility windows. Avoids dumping the full tree for simple
existence queries. Reuses the P1 multi-window walker and emits
nodes with stable w<N>:<M> IDs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:17:16 -04:00
Bailey DixonandClaude Opus 4.6 52843410d3 feat(A2): add android_drag gesture tool
Drag gesture from (startX, startY) to (endX, endY) via a single-stroke
GestureDescription. Wrapped in WakeLockManager wake scope and
BridgeSafetyManager package gate. Duration clamped to 100-3000ms.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:16:07 -04:00
Bailey DixonandClaude Opus 4.6 a042a4e2e0 feat(P1): ScreenReader walks all accessibility windows
Switch from rootInActiveWindow to service.windows.mapNotNull { it.root }
so we catch system overlays, popup menus, and notification shade
content. Node IDs prefixed with a window index to disambiguate. Careful
recycling of every window root we fetch. MAX_NODES=512 cap applies to
the combined tree, not per-window.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:01:53 -04:00
Bailey DixonandClaude Opus 4.6 0ace580be8 feat(A11): add skills/android/SKILL.md agent playbook
Per-app procedures for Uber, WhatsApp, Spotify, Maps, Settings,
Tinder plus a hard "do not loop" rule for bounded tool-call
budgets. Registered via skills.external_dirs by the installer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 21:00:04 -04:00
Bailey DixonandClaude Opus 4.6 78d44edb2a feat(A8): add WakeLockManager wake-scope wrapper for gesture dispatch
Wrap ActionExecutor.tap/tapText/typeText/swipe/scroll in a
screen-bright wake lock so bridge commands don't silently fail
when the phone is idle. Ref-counted, 10s timeout, manifest
WAKE_LOCK permission confirmed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 20:58:49 -04:00
Bailey DixonandClaude Opus 4.6 67c44543f4 docs: add ROADMAP.md + bridge feature expansion plan
Lean classic roadmap at root (Vision / Shipped / Current / Next / Future)
plus detailed v0.4 bridge-feature-expansion implementation plan under
docs/plans/. Removes stale docs/STATUS.md and docs/plan.md (superseded
by CHANGELOG/DEVLOG/ROADMAP).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 20:52:46 -04:00
Bailey DixonandClaude Opus 4.6 1e72ff4a9d chore: migrate applicationId to com.axiomlabs.hermesrelay + AGP 9.1.1
Switch the Play Store / Android runtime identity from
`com.hermesandroid.relay` to `com.axiomlabs.hermesrelay` as part of the
move from a personal Play Console account to the Axiom-Labs, LLC
DUNS-verified org account. The Kotlin namespace stays at
`com.hermesandroid.relay` — `namespace` and `applicationId` are
decoupled, so all 130+ source files, class FQCNs, Class.forName
lookups, and broadcast action strings keep working unchanged.

Why: the DUNS-verified org account is exempt from Google Play's 14-day
/ 12-tester closed-testing rule that applies to new personal dev
accounts, unblocking straight-to-production rollout. Play Store
package names are permanently reserved once used, so the old
`com.hermesandroid.relay` listing on the personal account is being
retired rather than transferred.

Changes:
- app/build.gradle.kts: applicationId → com.axiomlabs.hermesrelay;
  update flavor-split + migration comments
- scripts/dev.{bat,sh}: `adb shell am start` now uses explicit FQCN
  `com.axiomlabs.hermesrelay/com.hermesandroid.relay.MainActivity` —
  the `.MainActivity` shorthand breaks once applicationId ≠ namespace
- README.md, user-docs/guide/{getting-started,release-tracks}.md:
  Play Store links → new package ID
- CLAUDE.md: split "Package" into explicit Namespace vs applicationId
  entries documenting the decoupling
- RELEASE.md: rewrite Play Console section with Axiom-Labs org-account
  context, 14-day-rule exemption, keystore-continuity note, and
  historical migration callout
- build.gradle.kts: AGP 9.1.0 → 9.1.1 (Android Studio catalog update,
  rolled in for convenience)

Runtime verification: FileProvider auto-follows via
`\${applicationId}.fileprovider`; BridgeViewModel's a11y service check
uses `ComponentName(ctx.packageName, A11Y_SERVICE_CLASS)` which adapts
at runtime; proguard `-keep class com.hermesandroid.relay.**` rule is
namespace-based and still matches. Upload keystore + SHA256 fingerprint
are preserved — existing GitHub Secrets need no changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 19:03:51 -04:00
Bailey DixonandClaude Opus 4.6 722b87a3bf release: v0.3.0 — version-tagged artifacts + notes rewrite
Re-cut of v0.3.0. The prior tag was built before product flavors
landed the way they did, so its artifacts shipped as
`app-<flavor>-release.{apk,aab}` with no version in the filename
and the CI/release workflow globs were looking in pre-flavor paths
(`apk/release/`, `bundle/release/`) that no longer exist after the
googlePlay/sideload split.

This commit lands everything the re-cut needs in one atomic
release-prep push:

build: `base { archivesName.set("hermes-relay-<version>") }` in
  app/build.gradle.kts injects the app version into every APK and
  AAB filename. Outputs are now self-identifying at every stage
  of the pipeline — e.g.
  `hermes-relay-0.3.0-sideload-release.apk`,
  `hermes-relay-0.3.0-googlePlay-release.aab`.

ci: ci.yml upload-artifact glob fixed to `apk/*/debug/*.apk` so
  both `googlePlay` and `sideload` flavor debug APKs are picked
  up. Adds `if-no-files-found: error` so any future regression
  fails loudly instead of logging a warning that blends into
  green runs.

docs: RELEASE_NOTES.md fully rewritten in the upstream hermes-agent
  release-notes format (emoji-sectioned headers, Highlights bullet
  list with **bold title** — description pattern, `Since vX.Y.Z`
  stats line, tagline blockquote, contributors section, compare
  link footer). Content refocused on Phase 3 bridge channel as
  the headline.

docs: README.md, RELEASE.md, CLAUDE.md, user-docs/guide/getting-started.md,
  user-docs/guide/release-tracks.md, user-docs/.vitepress/theme/components/InstallSection.vue
  all swept to reference the new filenames. User-facing install
  prose uses the `-sideload-release.apk` / `-googlePlay-release.aab`
  suffix convention so it stays version-agnostic; shell command
  examples use `hermes-relay-*-sideload-release.apk` globs so
  copy-paste works at any version.

The release workflow was already updated in c1f1d17 to use
flavor-aware path globs (apk/*/release/*.apk and
bundle/*Release/*.aab), so re-pushing the v0.3.0 tag against
this HEAD will rebuild, sign, checksum, and attach the new
version-tagged artifacts automatically.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 16:10:31 -04:00
Bailey DixonandClaude Opus 4.6 c1f1d173f9 fix(ci): release workflow output paths for flavored builds
v0.2.0's release workflow assumed the pre-flavor AGP output layout:
  app/build/outputs/apk/release/*.apk
  app/build/outputs/bundle/release/*.aab

Phase 3 added `flavorDimensions += "track"` with googlePlay + sideload
product flavors, which moves the outputs to flavor-qualified paths:
  app/build/outputs/apk/googlePlay/release/app-googlePlay-release.apk
  app/build/outputs/apk/sideload/release/app-sideload-release.apk
  app/build/outputs/bundle/googlePlayRelease/app-googlePlay-release.aab
  app/build/outputs/bundle/sideloadRelease/app-sideload-release.aab

Note the APK vs AAB path quirk: APKs get a nested `/<flavor>/release/`
segment but AABs use concatenated `/<flavor>Release/`. Both documented
AGP conventions, both different from each other. Glob patterns updated
to match.

The v0.3.0 release workflow "Build release artifacts" step actually
succeeded — assembleRelease + bundleRelease are flavor-wide task
aliases that build ALL four artifacts in one gradle run. The failure
was downstream in the checksum + upload steps, which still used the
pre-flavor paths and got "No such file or directory" from sha256sum
on a glob that matched nothing.

Fixes in release.yml:

- Generate checksums — glob updated to `apk/*/release/*.apk` and
  `bundle/*Release/*.aab`, catches both flavors
- Create GitHub Release `files:` — same glob update, all four
  artifacts attached
- Release summary — replaced the fixed-path `ls -la` with a
  `find … -exec ls -la` that works for any flavor layout
- New diagnostic step "List produced artifacts (debug aid)" runs
  BEFORE checksums and prints every APK + AAB path. If this fails
  again, the next run's log will show exact paths up front instead
  of having to guess at AGP conventions

Also updated RELEASE_NOTES.md Download section — v0.2.0's text said
"grab app-release.apk" but that filename no longer exists. Now calls
out the four flavored names explicitly and explains which one to
download for sideload vs which goes to Play Console.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 15:29:05 -04:00
Bailey DixonandClaude Opus 4.6 92d15e8f6d fix(deps): pin markdown-renderer to 0.30.0 (0.40.2 breaks compile)
Dependabot auto-merged markdown-renderer 0.30.0 → 0.40.2 on 2026-04-13
(commit 299c98f) which silently broke CI. The bump introduces breaking
API changes that MarkdownContent.kt hasn't been updated for:

- markdownColor() no longer accepts codeText / linkText parameters
- MarkdownCodeBlock / MarkdownCodeFence inner lambdas now take a 3rd
  TextStyle argument (Function2 → Function3)
- MarkdownHighlightedCode's 3rd parameter is now TextStyle, not
  Highlights.Builder

Every CI run since the bump has been red, including v0.3.0's Release
workflow. Pinning back to 0.30.0 to unblock the release; TODO.md now
tracks the proper API-update task and a suggestion to add dependabot
ignore rules for packages we know need manual attention on major bumps.

Also flags a broader investigation: .github/workflows/dependabot-auto-
merge.yml somehow merged this despite CI failing. Needs a CI-gate fix
or a major-bump ignore rule.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 13:19:37 -04:00
Bailey DixonandClaude Opus 4.6 9d47673137 release: v0.3.0
Phase 3 bridge channel + voice mode + notification companion + agent
introspection tools + self-install skill + manual pair fallback +
per-channel grant revoke + installer TUI pass + Android 14 MediaProjection
compliance + new feature-branch workflow.

Also introduces the team workflow going forward:
- Feature branches + --no-ff merges as the house style
- Version bumps gated to release-prep commits on main via new
  scripts/bump-version.sh (atomic across libs.versions.toml +
  pyproject.toml + plugin/relay/__init__.py::__version__)
- Branch protection on main with release-prep carve-out

Files:
- scripts/bump-version.sh (new) — atomic three-source version bump
  with SemVer validation, monotonic appVersionCode increment, post-bump
  sanity check, diff output, and next-steps printout
- RELEASE.md — new "The three version sources" + "Branching policy"
  sections covering branch name prefixes, --no-ff merge style, version
  bumps never on feature branches, and branch protection rules. Release
  Process step 1 now points at bump-version.sh; step 4 enumerates all
  three version files in the git add command
- CLAUDE.md — Git section expanded with the new branching policy,
  conventional commits examples, and a reference to bump-version.sh
- CHANGELOG.md — full [0.3.0] - 2026-04-13 section pulled from the
  DEVLOG history since 0.2.0. Keep-a-Changelog sections: Added /
  Changed / Fixed / Docs.
- RELEASE_NOTES.md — rewritten for v0.3.0. Highlights the Phase 3
  bridge channel as the headline, then voice mode / notification
  companion / agent introspection tools / self-install skill / manual
  pair fallback / per-channel revoke / installer polish / workflow
  changes. Same Download section layout as v0.2.0.

All three version sources already synced at 0.3.0 via earlier commit
f86b7ce — no bump-version.sh run needed, but the script is in this
commit so future releases can use it.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 12:51:18 -04:00
dependabot[bot] d6bf22442d chore(deps): bump gradle-wrapper from 9.3.1 to 9.4.1 (#28)
Bumps [gradle-wrapper](https://github.com/gradle/gradle) from 9.3.1 to 9.4.1.
- [Release notes](https://github.com/gradle/gradle/releases)
- [Commits](https://github.com/gradle/gradle/compare/v9.3.1...v9.4.1)

---
updated-dependencies:
- dependency-name: gradle-wrapper
  dependency-version: 9.4.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-13 12:44:33 +00:00
dependabot[bot] 299c98fe9a chore(deps): bump markdown-renderer from 0.30.0 to 0.40.2 (#26)
Bumps `markdown-renderer` from 0.30.0 to 0.40.2.

Updates `com.mikepenz:multiplatform-markdown-renderer-m3` from 0.30.0 to 0.40.2
- [Release notes](https://github.com/mikepenz/multiplatform-markdown-renderer/releases)
- [Changelog](https://github.com/mikepenz/multiplatform-markdown-renderer/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/mikepenz/multiplatform-markdown-renderer/compare/v0.30.0...v0.40.2)

Updates `com.mikepenz:multiplatform-markdown-renderer-code` from 0.30.0 to 0.40.2
- [Release notes](https://github.com/mikepenz/multiplatform-markdown-renderer/releases)
- [Changelog](https://github.com/mikepenz/multiplatform-markdown-renderer/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/mikepenz/multiplatform-markdown-renderer/compare/v0.30.0...v0.40.2)

---
updated-dependencies:
- dependency-name: com.mikepenz:multiplatform-markdown-renderer-m3
  dependency-version: 0.40.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: com.mikepenz:multiplatform-markdown-renderer-code
  dependency-version: 0.40.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-13 12:44:20 +00:00
dependabot[bot] 37ede0cf06 chore(deps): bump camera from 1.4.1 to 1.6.0 (#24)
Bumps `camera` from 1.4.1 to 1.6.0.

Updates `androidx.camera:camera-core` from 1.4.1 to 1.6.0

Updates `androidx.camera:camera-camera2` from 1.4.1 to 1.6.0

Updates `androidx.camera:camera-lifecycle` from 1.4.1 to 1.6.0

Updates `androidx.camera:camera-view` from 1.4.1 to 1.6.0

---
updated-dependencies:
- dependency-name: androidx.camera:camera-core
  dependency-version: 1.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: androidx.camera:camera-camera2
  dependency-version: 1.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: androidx.camera:camera-lifecycle
  dependency-version: 1.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: androidx.camera:camera-view
  dependency-version: 1.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-13 12:43:19 +00:00
dependabot[bot] 48231a5bd2 chore(deps): bump org.jetbrains.kotlinx:kotlinx-coroutines-test (#22)
Bumps [org.jetbrains.kotlinx:kotlinx-coroutines-test](https://github.com/Kotlin/kotlinx.coroutines) from 1.9.0 to 1.10.2.
- [Release notes](https://github.com/Kotlin/kotlinx.coroutines/releases)
- [Changelog](https://github.com/Kotlin/kotlinx.coroutines/blob/master/CHANGES.md)
- [Commits](https://github.com/Kotlin/kotlinx.coroutines/compare/1.9.0...1.10.2)

---
updated-dependencies:
- dependency-name: org.jetbrains.kotlinx:kotlinx-coroutines-test
  dependency-version: 1.10.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-13 12:43:02 +00:00
dependabot[bot] d0a85bc79f chore(deps): bump haze from 1.3.0 to 1.7.2 (#21)
Bumps `haze` from 1.3.0 to 1.7.2.

Updates `dev.chrisbanes.haze:haze` from 1.3.0 to 1.7.2
- [Release notes](https://github.com/chrisbanes/haze/releases)
- [Changelog](https://github.com/chrisbanes/haze/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chrisbanes/haze/compare/1.3.0...1.7.2)

Updates `dev.chrisbanes.haze:haze-materials` from 1.3.0 to 1.7.2
- [Release notes](https://github.com/chrisbanes/haze/releases)
- [Changelog](https://github.com/chrisbanes/haze/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chrisbanes/haze/compare/1.3.0...1.7.2)

---
updated-dependencies:
- dependency-name: dev.chrisbanes.haze:haze
  dependency-version: 1.7.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: dev.chrisbanes.haze:haze-materials
  dependency-version: 1.7.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-13 12:42:55 +00:00
Bailey DixonandClaude Opus 4.6 f86b7ced88 chore(version): sync libs.versions.toml + pyproject + __version__ at 0.3.0
The three version sources had drifted into three different values:
- gradle/libs.versions.toml: 0.2.0 (canonical source, last released)
- pyproject.toml: 0.5.0 (drifted — never matched a real release)
- plugin/relay/__init__.py::__version__: 0.2.0 (stale constant) →
  0.5.0 (my wrong bump earlier today when I trusted pyproject)

Bailey flagged this: the actual in-progress target is 0.3.0. Everything
now matches.

- gradle/libs.versions.toml — appVersionName 0.2.0 → 0.3.0, appVersionCode
  2 → 3 (Play Console requires monotonic increment across all uploads)
- pyproject.toml — version 0.5.0 → 0.3.0 (reset the drifted value)
- plugin/relay/__init__.py::__version__ — 0.5.0 → 0.3.0 + expanded the
  sync comment to name libs.versions.toml as the canonical source and
  reference the drift incident so future contributors don't repeat it

CHANGELOG.md left alone — the [Unreleased] section is where in-progress
work lives until a real release freezes it into [0.3.0] - <date>.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 08:11:40 -04:00
Bailey DixonandClaude Opus 4.6 8b5ee18613 chore: bump relay __version__ 0.2.0 → 0.5.0 + document update shim in CLAUDE.md
Two pieces of doc/version drift caught while running the canonical update
on Docker-Server tonight:

1. plugin/relay/__init__.py::__version__ was hardcoded to "0.2.0" but
   pyproject.toml has been at "0.5.0" for a while. The /health endpoint
   reports __version__, which made it look like the running process was
   stale (it wasn't — it was the constant) and confused both the
   troubleshooting agent and me. Bumped + added a comment reminding
   future contributors to keep this in sync with pyproject.

2. CLAUDE.md was missing entries for `hermes-relay-update`,
   `--register-code`, and the install.sh `enable --now` → `restart` fix.
   Added file-table rows for the new shim + the `register_code_command`,
   updated the install.sh row to reflect step 5's three shims + step 6b
   gateway restart logic, and rewrote the "Standard update cycle"
   section to document the canonical one-liner, the SSH env-var-passing
   gotcha (env vars set on `cmd1 | cmd2` only apply to cmd1), and the
   manual piecewise path as fallback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 23:11:43 -04:00
Bailey DixonandClaude Opus 4.6 158bbcd455 feat(install): add hermes-relay-update shim
We had hermes-pair and hermes-status as discoverable shell shims but
no equivalent for "update Hermes-Relay". Bailey caught this — there's
no reason a user should have to remember the curl one-liner when the
two related commands are in their PATH.

New ~/.local/bin/hermes-relay-update is a thin shim that re-runs the
canonical curl-pipe installer with arg passthrough via `bash -s --`.
install.sh is already fully idempotent so re-running it IS the update —
this just gives that operation a memorable name.

Honors HERMES_RELAY_RESTART_GATEWAY / HERMES_RELAY_NO_RESTART_GATEWAY
naturally (the env vars pass straight through), and any future install.sh
flags will work via the shim's `"$@"` forwarding without changes here.

- install.sh — UPDATE_SHIM_PATH config, third shim writer in step 5,
  closing message lists hermes-relay-update first under "Update later",
  header docs comment block updated to describe all three shims
- uninstall.sh — UPDATE_SHIM_PATH config, removal block in step 5,
  header docs comment updated
- README.md — Updating paragraph rewritten to mention hermes-relay-update
  as the shortest path, with a note that re-running the curl pipe is
  equivalent

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 22:24:33 -04:00
Bailey DixonandClaude Opus 4.6 d32c70138d feat(pair): finish wiring manual pairing fallback + per-channel revoke UI
Closes the loop on the half-wired in-app fallback pairing flow that's been
sitting since Phase 3 landed, and adds per-channel grant revoke to the
Paired Devices screen so users can yank one channel without nuking the
whole session.

## Manual pair fallback — host side (plugin/pair.py)

New `--register-code <CODE>` flag composes with the existing TTL / grants /
transport-hint flags. Skips the QR rendering pipeline entirely and just
pre-registers the supplied code with the local relay over the same loopback
`/pairing/register` endpoint the QR flow already uses.

- `normalize_pairing_code()` — 6-char A-Z/0-9 validator with clear errors
- `register_code_command()` — probes relay, pre-registers, prints "tap
  Connect" instructions
- `pair_command()` short-circuits to the new path when --register-code is set

Composes with all existing flags:
  hermes-pair --register-code ABCD12 --ttl 30d --grants chat:never,bridge:7d

Exit codes: 0 success, 1 relay unreachable / rejection, 2 arg validation.

Tests: plugin/tests/test_register_code.py — 25 stdlib unittest cases across
3 classes (validator branches, command happy + error paths, wire-shape
assertions via patched urlopen). All pass in 0.004s.

## Manual pair fallback — phone side UX polish

ConnectionSettingsScreen Card 3 (was the bare "code + copy + regenerate"
stub) is now a numbered three-step walkthrough:
  1. Copy the code (with copy + regenerate icon buttons inline)
  2. Run `hermes-pair --register-code <CODE>` (rendered in a tappable
     monospace span — one-tap copies the full command)
  3. Real "Connect" button that fires applyServerIssuedCodeAndReset →
     disconnectRelay → connectRelay, with in-flight spinner state +
     LocalSnackbarHost result feedback via classifyError("pair", ...)

Below the steps: an expandable "How does this work?" explainer covering
when this is the right flow (no camera / SSH-only / single-device pair),
how the code mechanism works end-to-end, and the reminder that bridge
control is gated by the master toggle on the Bridge tab — not by this
code. New private ManualPairStep composable for the numbered step badges.

## Per-channel grant revoke — Paired Devices screen

Each device card's GrantChip now has an inline x icon. Tapping it opens
an AlertDialog confirming "Revoke <channel> access for <device>?" with a
reminder that the session itself stays paired and other channels keep
their existing expiry.

New ConnectionViewModel.revokeChannelGrant(tokenPrefix, channel) which:
- Reads cached PairedDeviceInfo grants
- Rebuilds the full grants map converting absolute-epoch back to seconds-
  from-now (clamped to >= 1 because the relay-side _materialize_grants
  interprets 0 as "never expire")
- Replaces the target channel with 1L (~instantly expired by the time
  the PATCH lands)
- Calls relayHttpClient.extendSession(tokenPrefix, ttlSeconds=null,
  grants=rebuilt)
- Snackbar + device list refresh on success

GrantChip rewritten to show relative TTL ("never" / "in 6d" / "in 23h" /
"expired") instead of absolute short date. Used a small clickable Box
instead of IconButton for the chip x because IconButton's 48dp minimum
inflates the FlowRow visually.

The full-session "Revoke" button is unchanged — per-channel chips are
additive, not replacements.

## Docs

- skills/devops/hermes-relay-pair/SKILL.md — Manual fallback section
  between Procedure and Pitfalls with workflow + composition rules
- skills/devops/hermes-relay-self-setup/SKILL.md — "If you can't scan
  a QR" subsection in section D
- user-docs/reference/configuration.md — expanded Manual pairing code
  bullet into a 3-step walkthrough
- user-docs/guide/getting-started.md — camera-unavailable tip callout
- README.md — one-line bullet under the install section
- DEVLOG.md — full entries for both halves under 2026-04-12

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 22:09:24 -04:00
Bailey DixonandClaude Opus 4.6 7f7d92955a docs: rename misleading 'Bridge pairing code' → 'Manual pairing code (fallback)'
Three coordinated text-only edits across plugin tool docstring, in-app
Settings card, and user-docs prose. No logic changes, no behavior changes.

The in-app card and matching configuration.md doc described the locally-
generated 6-char code as the "Phase 3 bridge feature" approval mechanism.
That was speculative when written and turned out to be wrong: Phase 3's
bridge gate is the master toggle in BridgeViewModel + the foreground
service notification, not an in-app code-approval flow. The locally-
generated code is actually the auth fallback path in AuthManager.
authenticate() — used when no QR-issued code is present, requiring the
host to pre-register the matching code with the relay. Renamed accordingly.

android_setup tool docstring + parameter rename:
- First line now says "FALLBACK helper" so LLMs reading the tool registry
  get the right signal. Was previously "Configure the Android bridge to
  point at the unified Hermes-Relay" which sounded canonical
- Parameter renamed `pairing_code` → `bridge_session_token`. The function
  stores the value in ANDROID_BRIDGE_TOKEN which is sent as the bearer
  token on every bridge HTTP call — it expects a long-lived session token,
  not a one-shot pairing code. The old name was misleading
- Clarified user_instructions: stop telling users to "scan the QR with this
  pairing code" (mixing flows). Just say "run hermes-pair, scan the QR"

Phase 3 status table cleanup:
- user-docs/guide/index.md and user-docs/reference/relay-server.md status
  tables previously said "Phase 3" for Bridge. Phase 3 is now in production
  on the sideload track (with safety rails, Tier 5 master toggle, accessibility
  service, MediaProjection consent flow). Updated to "Beta (sideload track)".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:49:42 -04:00
Bailey DixonandClaude Opus 4.6 05d266eb68 fix(ci): drop BIND_ACCESSIBILITY_SERVICE uses-permission (system-only)
Lint's [ProtectedPermissions] check correctly flags this as a
system-only permission. The system grants it to services that declare
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE" on
their <service> tag — which BridgeAccessibilityService already does
(line 84 of the same manifest). The redundant top-level uses-permission
was harmless at runtime but breaks the release lint gate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:45:18 -04:00
Bailey DixonandClaude Opus 4.6 9c489bb0cf fix(ci): suppress MissingPermission lint on AutoDisableWorker.notify
Lint can't trace through [hasPostNotificationsPermission] to see that
postNotification early-returns when the POST_NOTIFICATIONS runtime
grant isn't held, even though the gate exists at the top of the method
and the notify() call itself is wrapped in runCatching as a belt-and-
braces. The helper exists so the same gate can grow more conditions
later without each call site re-implementing it — preferable to
inlining the check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:40:21 -04:00
Bailey DixonandClaude Opus 4.6 b1b80e429a fix(auth): self-heal corrupted EncryptedSharedPreferences
After Studio reinstalls, the EncryptedSharedPreferences master key can
get rotated out from under the existing prefs file, leaving every
decrypt failing with AEADBadTagException forever. Until now the only
fix was a manual factory-reset — which manifested as "have to re-pair
every Studio rebuild".

Both KeystoreTokenStore and LegacyEncryptedPrefsTokenStore now wrap
every read/write in try/catch and call a new resetPrefs() helper on
failure: best-effort prefs.edit().clear(), then deleteSharedPreferences,
then a fresh buildPrefs() instance swapped into the now-mutable prefs
field. Reads return null/false on failure, writes retry once after
reset, clearAll falls through to a file delete.

KeystoreTokenStore.tryCreate also fires a one-shot read probe via the
instance's own contains() so a pre-corrupted file from a prior install
heals during construction rather than at first user-visible call. By
the time AuthManager.init reads KEY_SESSION_TOKEN, the store is in a
clean working state.

Net: if the master key gets rotated, the user loses the previous
session token but the next pair flow works without manual intervention
instead of every read silently returning null forever.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:38:01 -04:00
Bailey DixonandClaude Opus 4.6 792c02281e fix(voice): preserve response text on stop + add test voice toasts
interruptSpeaking no longer clears responseText on the way to Idle.
"Stop" was evaporating the visible response under the user's hand even
though chat history was preserved server-side — the bug was the voice
overlay's local copy getting blanked. The next startListening already
resets responseText at the moment the user explicitly starts a new
turn, so old text only sticks until they choose to move on.

testVoice now fires three Toasts so the user knows what's happening:
"Testing voice…" on trigger, "Voice test successful" on completion,
"Voice test failed: <reason>" on every failure path (pipeline missing,
synthesize failure, no audio returned, playback exception). The
trigger toast is held in a local var and cancelled before the result
toast fires so the two don't briefly stack.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:37:46 -04:00
Bailey DixonandClaude Opus 4.6 ba6f6bbf07 feat(voice): in-flow STT block, auto-scroll, fade edges, sized sphere
Voice overlay UX overhaul:

- Drop AnimatedContent wrapper around responseText so streaming tokens
  accrete in place instead of fade-flickering on every delta. Empty-state
  hint keeps AnimatedContent because that's a real discrete swap.
- Auto-scroll the response area to its tail on every length change so
  streaming content stays visible without manual drag.
- Top + bottom fade gradient via graphicsLayer Offscreen + BlendMode.DstIn
  so overflow is visually obvious without a scrollbar.
- TextAlign.Start for response paragraphs, Center retained for hint.
- Replace SpaceBetween + fillMaxHeight(0.6f) with weighted slots (sphere
  1.5f / response 1f) so sphere/waveform stop drifting as response grows
  and sphere holds its ~60% column share regardless of content.
- Move the "you said" block from the top of the column to directly above
  the response text. Old top-anchored chip split user gaze between the
  mic button at the bottom and the chip at the top after every turn;
  in-flow placement gives a single linear eye path mic → up → STT →
  response. Restyled as left-aligned "YOU" caption + body text so it
  pairs with the response below via Gestalt proximity.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:37:35 -04:00
Bailey DixonandClaude Opus 4.6 ef746d1f8f fix(bridge): Android 14+ MediaProjection grant evaporates without mediaProjection FGS
On Android 14 (API 34) and above, MediaProjectionManager.getMediaProjection()
returns a projection that the system auto-revokes within frames UNLESS a
foreground service has already called startForeground(type=mediaProjection)
BEFORE the call. Without the FGS slot declared, the consent dialog appears,
the user taps Allow, the dialog closes, and MediaProjectionHolder.projection
remains permanently null. Sample-tested on a Samsung S24 / Android 14
(2026-04-12).

Three coordinated edits:

1. AndroidManifest.xml — change BridgeForegroundService's foregroundServiceType
   from `specialUse` to `specialUse|mediaProjection`. Both subtypes share
   the same notification + same lifecycle + same service. The required
   FOREGROUND_SERVICE_MEDIA_PROJECTION permission was already declared
   from an earlier Phase 3 commit.

2. BridgeForegroundService.kt — startForegroundNotification() now ORs
   ServiceInfo.FOREGROUND_SERVICE_TYPE_SPECIAL_USE with
   ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION on API 34+.
   The Q..T branch is unchanged (AGP attaches the manifest type).

3. BridgeViewModel.requestScreenCapture() — gate the consent flow on
   the master toggle. BridgeForegroundService only runs while the
   toggle is on, so firing the consent dialog with the FGS not yet
   running would let Android revoke the grant on the way back. New
   path: if !masterToggle.value, emit a snackbar telling the user to
   enable Allow Agent Control first, then re-tap Screen Capture.

This also fixes the broader "background works properly" gap — with
the mediaProjection FGS slot declared, the projection survives short
backgrounding events that previously killed it.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:32:16 -04:00
Bailey DixonandClaude Opus 4.6 0055445d3e fix(install): actually restart hermes-relay when it's already active
`systemctl --user enable --now` is a no-op on services that are already
active — it only `start`s if the service isn't running. That meant every
install.sh re-run on a host with an already-running relay reported "[ok]
hermes-relay is running" but never actually picked up the new code from
the editable install. Spent way too long on Docker-Server tonight chasing
"why is /bridge/status still 404 after install.sh".

New behavior: detect the already-active case and call `restart` explicitly,
with a spinner + clear "picking up new code" message. Otherwise fall back
to the existing enable-and-start path for first-time installs.

Also normalized the warn/info paths through the new warn() helper so they
get the yellow ⚠ glyph + dim hints consistently.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:30:10 -04:00
Bailey DixonandClaude Opus 4.6 41f518625a feat(skill): hermes-relay-self-setup — single-source agent install recipe
Ship a canonical, agent-readable setup recipe with two delivery modes from
one file. Solves the chicken-and-egg discovery problem for AI assistants
helping users install/maintain Hermes-Relay.

New file: skills/devops/hermes-relay-self-setup/SKILL.md
- Pre-install: agents fetch the raw GitHub URL of the SKILL.md, follow it
  to verify hermes-agent prerequisites, run the installer, pair the phone,
  verify the result. No Hermes needed at fetch time.
- Post-install: same file is auto-discovered as a Hermes skill via the
  existing skills/external_dirs registration. Users invoke
  /hermes-relay-self-setup from any chat for re-setup, troubleshooting,
  "is everything wired correctly?" checks.
- Single source, two delivery modes, zero drift. No llms.txt duplication.

README.md "For AI Agents" section
- New 20-line copy-paste prompt block right after the install one-liner
- Tells the agent its role, points at the SKILL.md raw URL, lists the
  high-level steps, includes safety reminders (confirm before running,
  never restart hermes-gateway without asking)
- Cross-references the post-install /hermes-relay-self-setup slash command

user-docs vitepress home view (InstallSection.vue)
- New "For AI Agents" subsection cleanly below the install-extras grid
- Same agent prompt as the README, with copy button
- Note explaining the dual-mode pattern
- Styles match existing install-section visual language

install.sh closing message
- New "Self-setup / troubleshoot" section listing /hermes-relay-self-setup
  alongside the existing pair commands so post-install users discover the
  slash command without reading docs

TODO.md (new file at root)
- Captures the bigger open question Bailey raised: proper Hermes plugin/
  skill/tool distribution. Currently we ship a custom install.sh; should
  upstream have a canonical plugin registry? Should skills install
  independently? Should we propose this pattern to upstream? Notes the
  hermes-relay-self-setup SKILL.md as a precedent worth generalizing.
- Also lists the smaller deferred items (MediaProjection consent test,
  WorkManager upgrade, voice-bridge multi-turn, vision-nav LLM client,
  release-tracks screenshots) so they don't get lost.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 21:14:48 -04:00
Bailey DixonandClaude Opus 4.6 b9cd86bdf7 feat(install): TUI polish — colors, banner, spinner, unicode bullets
Replace the plain-text installer with a TTY-aware experience that matches
the polish of modern installers (rustup, homebrew, openclaw):

- ANSI color helpers via tput, with NO_COLOR=1 + non-TTY fallbacks to
  empty strings + ASCII glyphs so curl | bash logs and CI capture stay
  grep-friendly
- Boxed banner at start ("Hermes-Relay Installer · Phase 3 — Bridge…")
  and at end ("✓ Hermes-Relay installed")
- New step() helper for [N/6] section headers (bold cyan ▶ marker +
  bold step counter + bold title)
- New spin() helper that runs while a backgrounded PID is alive — used
  for the long pip install (5-30s) and the optional gateway restart.
  No-op (silent wait) when stdout isn't a TTY
- New warn() helper for the yellow ⚠ caution lines
- Polished gateway-restart prompt: explains WHY a restart helps, what
  the cost is (~2s of interrupted chats), and defaults to no
- Restructured closing message into clear sections (Pair / Update /
  Manage / Uninstall) with bold commands and dim explanations,
  including the new hermes-status shim

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 20:57:16 -04:00
Bailey DixonandClaude Opus 4.6 98b5e9f0c2 fix(install): make hermes-gateway restart opt-in, never forced
The previous version unconditionally restarted hermes-gateway whenever it
was running as a user service, which interrupts active chat sessions and
feels presumptuous from a third-party plugin installer.

New behavior:
- HERMES_RELAY_RESTART_GATEWAY=1   → restart unconditionally (scripted)
- HERMES_RELAY_NO_RESTART_GATEWAY=1 → skip silently (paranoid default)
- Interactive shell (TTY)          → prompt [y/N], default no
- Non-interactive (curl | bash)    → print hint, skip

The user is always in control. The gateway hint is clear about WHY a
restart is needed (re-import new plugin tools) and how to do it manually.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 20:54:12 -04:00
Bailey DixonandClaude Opus 4.6 103fe70e9d fix(install): auto-restart hermes-gateway so new plugin tools re-import
The gateway caches plugin tools and skills at process import time, so a
fresh `git pull` of the plugin doesn't take effect until the gateway is
restarted. Without this, running install.sh on an existing install left
new tools (e.g. android_phone_status) silently invisible — the relay
restarted but the gateway kept serving stale imports.

New step 6b after the relay-service restart: if hermes-gateway is running
as a user systemd service, restart it. If it's not (manual invocation,
container, etc.), print a hint instead of guessing.

Also rewrote the closing "To update later" message to point users at the
canonical one-liner curl pipe and explain what re-running install.sh does
end-to-end (pull, refresh editable install, re-create shims, restart relay,
restart gateway).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 20:52:21 -04:00
Bailey DixonandClaude Opus 4.6 990870cee3 feat(phase3-status): dynamic phone-status prompt + android_phone_status tool + bridge UI hardening
Three-layer feature so the agent finally knows what the phone can do, plus
follow-up fixes discovered while smoke-testing the merged Phase 3 build.

Layer 1 — phone-side dynamic system prompt (replaces the static one-liner):
- New PhoneStatusPromptBuilder.kt builds a transparent block from real
  bridge/permission state, capped under 100 words, returns null when
  everything is off (no empty system messages)
- 4 new sub-toggles in ConnectionViewModel: bridge state, current app,
  battery, safety status. Privacy-sensitive ones (current_app, battery)
  default OFF
- ChatSettingsScreen gains a live preview Card showing exactly what
  will be sent on the next message
- ChatViewModel deletes APP_CONTEXT_PROMPT, calls the builder via a
  guarded-reflection capturePhoneSnapshot() helper

Layer 2 — relay backend (GET /bridge/status, loopback only):
- BridgeStatusReporter expanded to push the full nested device/bridge/safety
  contract every 30s; new pushNow() method called on master toggle flips
- BridgeHandler caches latest_status + last_seen_at
- New handle_bridge_status route mirrors /pairing/register loopback gate
- 7 new stdlib unittest tests in test_bridge_status.py — all green

Layer 2 — symmetric trio (mirrors the pair feature):
- New plugin/tools/android_phone_status.py — Hermes tool, stdlib only
- New plugin/status.py + hermes-status shim — operator CLI with --json/--port,
  three exit codes (0/1/2 for connected/relay-down/no-phone)
- New skills/devops/hermes-relay-status/SKILL.md — slash command
- install.sh + uninstall.sh updated for the second shim
- 19 new stdlib unittest tests — all green

Bridge UI hardening (master gate, MediaProjection, labels):
- HermesAccessibilityService now feeds cachedMasterEnabled itself via a
  service-scoped DataStore observer. The previous push-from-outside
  pattern was never wired and the cache stayed false forever, 403'ing
  every command except /ping and /current_app
- MainActivity registers an ActivityResultLauncher for MediaProjection;
  new ScreenCaptureRequester process-singleton bridges non-Activity
  callers (BridgeViewModel.requestScreenCapture()). Bridge tab's Screen
  Capture row is now tappable instead of inert
- BridgeViewModel.testNotificationListener() + an onTestNotificationListener
  lambda through BridgePermissionChecklist for parity with the other rows
- Sideload flavor strings: app_name → "Hermes Dev", a11y_service_label →
  "Hermes-Bridge Dev", notification_companion_label → "Hermes Dev …".
  googlePlay's a11y_service_label flipped to "Hermes-Bridge" (with hyphen)
  for consistency. Disambiguates side-by-side installs in launcher /
  recents / Settings → Apps
- BridgeForegroundService: Intent.flags = … → addFlags(…) (the property
  setter form fails because Intent.setFlags returns Intent, not void)
- BridgeViewModel: removed deprecated StateFlow.distinctUntilChanged()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 20:43:58 -04:00
Bailey Dixon eec9f60b63 feat(bridge): post-merge follow-ups — deep link, overlay nag, in-app Test buttons
Three small UX wins on top of merged Wave 2:

1. Foreground-notification "Settings" action now deep-links straight to
   BridgeSafetySettingsScreen instead of dropping the user on MainActivity
   home. New util/NavRouteRequest.kt singleton SharedFlow lets any external
   launcher (foreground service, broadcast receiver, shortcut intent) post
   nav requests. MainActivity reads EXTRA_NAV_ROUTE in onCreate / onNewIntent
   and pumps to NavRouteRequest. RelayApp's LaunchedEffect(navController)
   forwards each request to navController.navigate.

2. Overlay-permission nag banner on BridgeScreen — when the master toggle
   is on but Settings.canDrawOverlays is false, a prominent red card warns
   the user that confirmation prompts can't display. Without it, the
   BridgeSafetyManager.awaitConfirmation fail-closed was silent.

3. In-app permission Test buttons in BridgePermissionChecklist. Matches
   the notification companion's existing Test pattern. Each row gets an
   optional "Test" affordance next to the status icon; BridgeViewModel
   exposes testAccessibilityService / testScreenCapture / testOverlayPermission
   methods that run side-effect-free diagnostic checks and emit a one-shot
   result on a new testEvents SharedFlow. BridgeScreen collects and shows
   via LocalSnackbarHost.

All marker blocks use PHASE3-safety-rails-followup convention.
2026-04-12 19:03:45 -04:00
Bailey Dixon 7a5e7e31f8 chore(rename): replace Greek-letter agent codenames with ASCII slugs
Bulk rename across the repo + Obsidian plan: agents previously identified
by α β γ δ ε ζ η θ now use descriptive ASCII slugs (bridge-server,
flavor-split, accessibility, bridge-ui, notif-listener, safety-rails,
voice-intents, vision-nav). Greek letters were a math-paper convention
that sorts nicely but renders badly in some terminals, can't be typed
without a special keyboard, and made commit-history search awkward.

Single sed pass per file, applied via bash for-loop across 42 repo files
+ the canonical Obsidian plan. Verified with grep -rc '[αβγδεζηθ]' = 0.

Git history is not rewritten — existing commit subjects keep their Greek
letters because rewriting them would require a force push and invalidate
every commit hash since the divergence point. Going forward, branches,
commit messages, and marker blocks all use ASCII.

Marker block convention going forward:
  // === PHASE3-<slug>: ... ===
  // === END PHASE3-<slug> ===

Followup blocks use PHASE3-<slug>-followup.
2026-04-12 19:00:09 -04:00
Bailey Dixon 5b2d9f8611 fix(devlog): remove orphaned conflict marker (followup to θ merge) 2026-04-12 18:51:10 -04:00
Bailey Dixon b13374e49f merge(phase3-θ): android_navigate vision-driven navigation tool
# Conflicts:
#	DEVLOG.md
2026-04-12 18:50:19 -04:00
Bailey Dixon ed3e8729ef fix(devlog): remove stray HEAD conflict marker from ζ merge resolution 2026-04-12 18:49:56 -04:00
Bailey Dixon e6a150facc merge(phase3-η): voice→bridge intent routing (sideload-only, Tier 3)
# Conflicts:
#	DEVLOG.md
2026-04-12 18:49:32 -04:00
Bailey Dixon 2808d0e7c7 merge(phase3-ζ): bridge safety rails (blocklist, destructive-verb confirm, auto-disable, foreground service)
# Conflicts:
#	DEVLOG.md
2026-04-12 18:49:03 -04:00
Bailey Dixon 98f4c909df merge: docs two-track explainer + ConnectionWizard refactor + Phase 3 manifest dedup hotfix 2026-04-12 18:48:13 -04:00
Bailey Dixon 08bdb0d1d0 merge: pick up ad7db94 (docs: uninstall mention on guide index + relay-server reference) 2026-04-12 18:48:08 -04:00
Bailey Dixon 7fae9a3931 docs: DEVLOG entry for Phase 3 manifest dedup hotfix 2026-04-12 18:47:09 -04:00
Bailey Dixon 261a169c62 fix(phase3): dedupe AccessibilityService entry; collapse Gradle deprecation
Wave 1 agent β stubbed a phantom <service android:name=".accessibility.BridgeAccessibilityService">
in both flavor manifests anticipating γ would name the class BridgeAccessibilityService.
γ named it HermesAccessibilityService. Manifest merger kept both as separate
<service> entries because the names differ — one real (γ's), one phantom (β's,
no implementing class). Android Settings showed two rows; enabling the phantom
would have crashed the bind.

Removed both stub <service> blocks from the flavor manifests entirely. The
flavor distinction now lives at the resource layer (accessibility_service_config.xml
+ flavor strings.xml), where Gradle's resource merger picks the right files
per variant. Flavor manifests kept as empty <application /> overlays for
future flavor-specific permissions / activities. Added a11y_service_label
to main strings.xml so the canonical service entry can be labeled distinctly
from the launcher icon.

Also collapsed gradle.properties android.dependency.{useConstraints=true,
excludeLibraryComponentsFromConstraints=true} into a single
useConstraints=false (the AGP-recommended migration for the deprecation
warning), unchanged semantics.
2026-04-12 18:46:12 -04:00
Bailey Dixon 3171cb2165 feat(onboarding,connection): unified ConnectionWizard + lifecycle-aware health probes
Replaces the bespoke onboarding ConnectPage and ConnectionSettings pairing
walkthrough with a single shared three-step ConnectionWizard (Scan →
Confirm → Verify), and adds the missing on-resume revalidation that was
causing badges to flash stale Connected/Disconnected for ~30s after
foregrounding.

- New ConnectionWizard.kt: shared by OnboardingScreen and ConnectionSettings
- New ConnectionViewModel.applyPairingPayload() — single entry point for
  "user confirmed a scanned QR + chose a TTL" (replaces the ~50-line
  inline confirm callback)
- New HealthStatus tri-state (Unknown/Probing/Reachable/Unreachable) +
  apiServerHealth + relayServerHealth StateFlows
- New revalidate() — flips both health flows to Probing immediately so
  badges don't flash stale state, then probes API + relay /health in
  parallel and joins
- Lifecycle hook in RelayApp wires DisposableEffect → ON_RESUME →
  connectionViewModel.revalidate() at the app root, single observer
  for the whole tree
- Connectivity reaction: ConnectivityObserver flow now drives revalidate()
  on Available transitions (drop(1) to skip seed)
- Periodic 30s relay /health loop mirroring the existing API loop
- ConnectionStatusBadge gains Probing state (gray pulse @ 1.2s),
  preserves boolean overloads for legacy call sites
- OnboardingScreen.ConnectPage replaced; pages list collapses Skip and
  next/back nav while wizard is active
- ConnectionSettingsScreen "Scan Pairing QR" + "Guided setup" buttons
  collapsed into one "Pair with QR" button + full-screen dialog wizard
- PairingWalkthroughDialog.kt deleted (~370 lines, superseded)
2026-04-12 18:45:39 -04:00
Bailey DixonandClaude Opus 4.6 8dd0b2d86c feat(phase3-ζ): bridge safety rails (blocklist, confirmation, auto-disable, foreground service)
Tier 5 safety enforcement for Phase 3 Wave 2:

- Per-app blocklist (DataStore-backed, ships with ~30 banking/payments/
  password-manager/2FA defaults). Enforced in BridgeCommandHandler via
  BridgeSafetyManager.checkPackageAllowed against currentApp.
- Destructive-verb confirmation modal (word-boundary regex, default verbs
  send/pay/delete/transfer/confirm/submit/post/publish/buy/purchase/charge/
  withdraw). Suspends BridgeCommandHandler on a CompletableDeferred under
  withTimeout; shown via SYSTEM_ALERT_WINDOW overlay ComposeView. Fail-closed
  on missing overlay permission or timeout.
- Auto-disable idle timer (5..120 min, default 30) rescheduled on every
  accepted command. Implemented as a coroutine-owned Job (androidx.work is
  not in the classpath) with AutoDisableWorker.kt documenting the upgrade
  path. Fires HermesAccessibilityService.setMasterEnabled(false) and posts
  a one-shot notification.
- Persistent BridgeForegroundService with specialUse foregroundServiceType
  on Android 14+; Disable / Settings notification actions. Lifecycle driven
  by BridgeViewModel's masterToggle observer.
- Optional floating status chip overlay (SYSTEM_ALERT_WINDOW, off by
  default). Shares the WindowManager attachment point with the confirmation
  modal via ConfirmationOverlayHost.

Files created:
- data/BridgeSafetyPreferences.kt
- bridge/BridgeSafetyManager.kt
- bridge/BridgeForegroundService.kt
- bridge/BridgeStatusOverlay.kt
- bridge/AutoDisableWorker.kt
- ui/screens/BridgeSafetySettingsScreen.kt
- ui/components/DestructiveVerbConfirmDialog.kt (+ BridgeStatusOverlayChip)
- ui/components/BridgeSafetySummaryCard.kt

Files edited (marked PHASE3-ζ):
- network/handlers/BridgeCommandHandler.kt (safety enforcement injection)
- AndroidManifest.xml (SYSTEM_ALERT_WINDOW, FGS_SPECIAL_USE, service)
- ui/screens/BridgeScreen.kt (replace SafetyPlaceholderCard)
- ui/screens/SettingsScreen.kt (Bridge safety entry-point row)
- ui/RelayApp.kt (BridgeSafetySettings route)
- viewmodel/BridgeViewModel.kt (foreground service lifecycle observer)
- viewmodel/ConnectionViewModel.kt (install safety manager + overlay host)

CLAUDE.md Key Files updated with 8 new rows + BridgeCommandHandler addendum.
DEVLOG.md entry for 2026-04-12 Phase 3 Wave 2 ζ.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 18:28:50 -04:00
Bailey DixonandClaude Opus 4.6 4c87d12bc3 docs(user-docs): add two-track release model explainer + FeatureMatrix component
- New FeatureMatrix.vue component matches HermesFlow/InstallSection design
  language (Space Grotesk + Space Mono, --vp-c-brand-1 accent, flat +
  border-separated). Semantic <table>, three support states with inline
  SVG icons, responsive collapse to single-column with role=tablist mobile
  switcher below 720px, icon-only below 480px. Zero npm deps.
- New release-tracks.md page explains the googlePlay vs sideload split in
  plain language: why two tracks (Play accessibility-service scrutiny),
  what's in each (embedded FeatureMatrix), how to choose, can-I-switch
  (yes — applicationIdSuffix lets them coexist), install + update flows.
- features/index.md adds Bridge — Phone Control + Bridge — Safety Rails
  tables with a Sideload-only badge convention, removes Bridge from
  Coming Soon, embeds the matrix near the bottom.
- getting-started.md step 1 now flags both flavors and links to release-
  tracks instead of pretending sideloading is just an alternative install.
- Sidebar nav adds Release tracks under /guide/.
- Theme registers FeatureMatrix as a global Vue component.
- Verified with npx vitepress build — clean compile, all pages render.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 18:25:24 -04:00
Bailey DixonandClaude Opus 4.6 a8b2822802 feat(phase3-η): voice→bridge intent routing (sideload-only, Tier 3)
Wires voice mode to optionally route transcribed text through the bridge
channel instead of chat. Held voice → STT → keyword classifier → bridge
tool call for phone-control intents, fall through to chat for everything
else.

Cross-flavor compile pattern (first surface to exercise β's flavor split):
- Interface + sealed IntentResult in main/
- googlePlay flavor: NoopVoiceBridgeIntentHandler (always NotApplicable,
  never references any bridge / accessibility class — keeps the Play APK
  honest with its conservative feature description)
- sideload flavor: RealVoiceBridgeIntentHandler + VoiceIntentClassifier
  (regex-based, six intents, false-negative-biased)
- Both flavors export createVoiceBridgeIntentHandler(multiplexer) at the
  same FQCN; VoiceViewModel calls it once in initialize(). No reflection.

v1 confirmation (Wave 3 follow-up swaps in real conversational confirm):
- Destructive intents (SendSms) speak a confirmation via the existing
  sentence-TTS queue, start a 5s countdown coroutine, and dispatch
  unless VoiceViewModel.cancelPendingBridgeIntent() is called.
- Safe intents (OpenApp, Tap, Scroll, Back, Home) dispatch immediately.

Classifier patterns (strip fillers first, then first-match-wins):
- SendSms: 'text/send a message to X saying|:,Y' + no-separator fallback
- OpenApp: 'open|launch|start [the] <app> [app]'
- Tap:     'tap|press|click [on] [the] <target> [button]'
- Scroll:  'scroll [to the] up|down|top|bottom'
- Back:    '[go|navigate] back'
- Home:    '[press|go] home [screen]'

Envelope wire shape (simple + documented; ζ owns the schema):
  channel: 'bridge'
  type:    'tool.call'
  payload: { tool, args, requires_confirmation, source: 'voice' }
Dispatched via ChannelMultiplexer.send(envelope). New optional
bridgeMultiplexer param on VoiceViewModel.initialize() so existing call
sites keep compiling until they're updated to pass the instance.

Files:
- main:       voice/VoiceBridgeIntentHandler.kt (interface + IntentResult)
- googlePlay: voice/VoiceBridgeIntentHandlerImpl.kt (no-op)
- googlePlay: voice/VoiceBridgeIntentFactory.kt
- sideload:   voice/VoiceBridgeIntentHandlerImpl.kt (real)
- sideload:   voice/VoiceBridgeIntentFactory.kt
- sideload:   voice/VoiceIntentClassifier.kt
- main:       viewmodel/VoiceViewModel.kt (PHASE3-η marker blocks)

Known cross-worktree dependency: this assumes β's productFlavors
{ googlePlay; sideload } lands in build.gradle.kts before merge.
Order of merges: β first, then η (+ ζ + θ).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 18:16:44 -04:00
Bailey Dixon 1a2e3ff613 feat(phase3-θ): android_navigate vision-driven navigation tool
Adds the Phase 3 Tier 4 `android_navigate(intent, max_iterations=5)`
tool — a close-the-loop wrapper around the Wave 1 bridge HTTP routes
that takes a natural-language intent, screenshots the phone, asks a
vision model for the next action, executes it, and repeats until the
model emits `done` or the iteration cap is hit. Default cap 5,
hard-clamped to 20.

Hard plan constraint honoured: never continuous capture. Exactly one
/screenshot per iteration, only when the tool is invoked.

- plugin/tools/android_navigate.py — main loop, action dispatch, and
  registry registration. Shares _bridge_url()/_auth_headers() with
  android_tool.py so pairing config stays single-sourced.
- plugin/tools/android_navigate_prompt.py — prompt template + response
  parser. Kept separate so the parser tests don't need requests.
- plugin/tests/test_android_navigate.py — 35 stdlib unittest cases
  covering every valid action, every malformed-input branch, and the
  full loop (success / iteration cap / screenshot failure / parse
  error / action failure / llm_gap envelope / empty intent / clamping
  / schema sanity). Runs via `python -m unittest` without pulling in
  the conftest.py `responses` dependency.

LLM integration is a known gap documented in the module docstring: the
plan doesn't pick a vision provider, the plugin has no published LLM
client surface for tools, and the gateway's run loop is not re-entrant.
The loop defines a `call_vision_model` injection point — tests patch
it, production either sets HERMES_NAVIGATE_STUB_REPLY for smoke runs
or swaps _default_vision_model for a real Anthropic/OpenAI client.
Until that's wired, live calls return a clean {status: error, reason:
llm_gap} envelope instead of silently faking actions or crashing.

DEVLOG + CLAUDE.md Key Files rows updated for the three new files.
2026-04-12 18:16:02 -04:00
Bailey DixonandClaude Opus 4.6 ad7db94d07 docs(user): mention uninstall + full-features promise on guide index + relay-server reference
Two doc surfaces still mentioned install.sh without their uninstall
counterpart:

- user-docs/guide/index.md (the "What is Hermes-Relay?" landing page)
  now states "One command, full features" and lists the uninstall
  one-liner alongside the install one.
- user-docs/reference/relay-server.md adds a uninstall recipe in the
  Quick Start section right next to the systemctl management commands.

Both surfaces now match the install/uninstall coverage already in
README.md, install.sh, uninstall.sh, getting-started.md, and api.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 18:05:12 -04:00
Bailey Dixon 8d86a62440 merge: phase 3 wave 1 bridge team (α β γ δ ε + followups)
# Conflicts:
#	DEVLOG.md
2026-04-12 17:57:48 -04:00
Bailey DixonandClaude Opus 4.6 9f37047490 fix(client): probeCapabilities uses HEAD not OPTIONS to bypass CORS middleware
End-to-end install test on the production gateway revealed that the
OPTIONS-method probes I'd written never reach the router — hermes-agent's
security_headers_middleware intercepts OPTIONS preflight requests and
returns 403 for both existing AND missing paths. That made every probe
report 403, which my code treated as "endpoint missing," which would
have caused Auto mode to mis-route every install.

Fix: switch to HEAD method, treat any non-404 status as "route exists."
HEAD bypasses the CORS middleware path and surfaces the actual router
status (200/204/401/403/405 for present, 404 for missing). Verified
against the production gateway:
  HEAD /api/sessions                       → 200 (sessions CRUD present)
  HEAD /api/sessions/probe/chat/stream     → 405 (POST-only handler present)
  HEAD /v1/runs                            → 405 (POST-only handler present)
  HEAD /v1/models                          → 200 (OpenAI compat present)
  HEAD /this-route-does-not-exist          → 404 (genuinely missing)

Also extracted the probe into a tiny inner `routeExists(path)` helper
inside probeCapabilities so the four endpoint checks read as four
identical lines instead of four blocks of try/catch.

CLAUDE.md key files entry + DEVLOG.md narrative updated to reflect
the HEAD switch + the empirical discovery of the CORS-middleware
behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 17:53:31 -04:00
Bailey Dixon 3e80f4abf3 feat(phase3-followups): /media/upload route + notification companion wiring
α-followup: POST /media/upload bridges γ's phone-side ScreenCapture to the
relay's MediaRegistry. Multipart file field, bearer auth, streams to a
sandboxed tempfile under tempfile.gettempdir() (in the default
MediaRegistry allowed_roots), enforces max_size_bytes during read, then
hands the path to MediaRegistry.register so token issuance / expiry /
LRU / GET /media/{token} are identical to the loopback path.

ε-followup: ConnectionViewModel.init now sets
HermesNotificationCompanion.multiplexer once after bridge handler
registration; the service's pendingEnvelopes queue drains on the next
onNotificationPosted. SettingsScreen gains a Notification companion
entry-point row and RelayApp registers Screen.NotificationCompanionSettings
+ a composable route hosting NotificationCompanionSettingsScreen.

Both touch-ups gate clean Wave 1 smoke testing in Android Studio.
2026-04-12 17:47:28 -04:00
Bailey DixonandClaude Opus 4.6 b6db95caf9 feat(plugin): runtime API bootstrap + capability auto-detect + canonical uninstaller
Adds hermes_relay_bootstrap/ — a Python package shipped via .pth in the
hermes-agent venv site-packages — that monkey-patches aiohttp.web.Application
at interpreter startup. When the gateway builds its app, the bootstrap
intercepts app["api_server_adapter"] = self and injects 14 management
handlers onto the same router: /api/sessions/* CRUD, /api/memory,
/api/skills, /api/config, /api/available-models. Feature-detects on route
path and no-ops cleanly when these routes are already present, so it's
safe to ship across all hermes-agent versions.

Chat streaming continues to use standard upstream /v1/runs (which already
emits structured tool events). The Android client gains a new
ServerCapabilities probe + streamingEndpoint = "auto" default that picks
the best chat path automatically based on what each server actually
exposes (Auto/Sessions/Runs).

Also adds canonical uninstall.sh — reverses every install.sh step in the
opposite order, idempotent, never touches state shared with other Hermes
tools (.env, sessions DB, hermes-agent venv core). Flags: --dry-run,
--keep-clone, --remove-secret. install.sh header + success summary +
README + user-docs/guide/getting-started.md + user-docs/reference/api.md
all updated to mention the uninstall path.

Verified end-to-end on the server: bootstrap loads via .pth, intercepts
aiohttp.web import, injects 11 unique paths onto a vanilla aiohttp app,
GET /api/sessions returns real production data via SessionDB, OPTIONS
probes return the expected 405/404 codes for capability detection, and
feature detection no-ops cleanly when routes already exist.

See docs/decisions.md ADR 16 for full rationale and DEVLOG.md
2026-04-12 entries for the work breakdown.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 17:42:58 -04:00
Bailey Dixon 2896627b75 merge(phase3-ε): add notification companion service for opt-in notification triage
# Conflicts:
#	CLAUDE.md
#	DEVLOG.md
#	app/src/main/AndroidManifest.xml
#	plugin/relay/server.py
2026-04-12 17:33:09 -04:00
Bailey DixonandClaude Opus 4.6 1e92138460 feat(phase3-ε): add notification companion service for opt-in notification triage
Phase 3 / Wave 1 / ε. Adds an opt-in helper that lets the user's
Hermes assistant read notifications they've explicitly granted
access to via Android's NotificationListenerService — the same
public API Wear OS, Android Auto, and Tasker have used for over
a decade. Disabled by default; user controls grant/revoke via
Android Settings → Notification access.

Three pieces (all PHASE3-ε marker-blocked in shared files):

* Phone — HermesNotificationCompanion (NotificationListenerService
  subclass) + NotificationModels + NotificationCompanionSettingsScreen
  (About / Status / Test sections, mirrors VoiceSettingsScreen). Cold-
  start buffer up to 50 envelopes, drops on relay-offline (matches
  smartwatch-out-of-range semantics). Skips notifications with no
  human-readable content.

* Server — NotificationsChannel with collections.deque(maxlen=100)
  for LRU-by-time eviction. Wired into RelayServer __init__ + _on_message
  dispatch. In-memory only, lost on restart by design.

* Tool — android_notifications_recent(limit=20) registers via
  tools.registry, hits /notifications/recent over loopback (no auth
  needed for loopback callers, matches /media/register trust model).
  Stdlib urllib.request only.

No new session grant type (reuses existing chat grant trust boundary
per spec). ChannelMultiplexer.sendNotification() wrapper for the
outbound path. python -m py_compile clean on all touched/new files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 17:28:12 -04:00
Bailey Dixon c80dcfbebb merge(phase3-δ): rewrite BridgeScreen with master toggle, permissions, status, activity log
# Conflicts:
#	DEVLOG.md
2026-04-12 17:20:45 -04:00
Bailey Dixon 61637bdf7e merge(phase3-γ): build phone-side accessibility runtime (service + reader + executor + capture)
# Conflicts:
#	DEVLOG.md
2026-04-12 17:20:10 -04:00
Bailey Dixon f8889835b3 merge(phase3-α): migrate legacy bridge relay into unified relay (port 8767)
# Conflicts:
#	DEVLOG.md
2026-04-12 17:19:28 -04:00
Bailey Dixon 62fdfb021e merge(phase3-β): add googlePlay + sideload build flavors with flavor-specific a11y manifests 2026-04-12 17:17:41 -04:00
Bailey DixonandClaude Opus 4.6 dc2256b3fe feat(phase3-γ): build phone-side accessibility runtime (service + reader + executor + capture)
Wave 1 Agent γ — phone-side execution layer for the bridge channel.

New package com.hermesandroid.relay.accessibility:
- HermesAccessibilityService: AccessibilityService subclass with @Volatile
  singleton, current-app tracking via TYPE_WINDOW_STATE_CHANGED, DataStore-
  backed master enable flag.
- ScreenReader: UI tree → @Serializable ScreenContent(rootBounds, nodes[],
  truncated). MAX_NODES=512 cap. findNodeBoundsByText + findFocusedInput
  helpers for the executor.
- ActionExecutor: tap/tapText/swipe/scroll via GestureDescription wrapped
  as suspend fn via suspendCancellableCoroutine. typeText via ACTION_SET_TEXT.
  pressKey maps curated vocab to GLOBAL_ACTION_*. wait clamped to 15s.
- ScreenCapture: MediaProjection → VirtualDisplay → ImageReader → PNG →
  multipart POST /media/upload. Co-located MediaProjectionHolder singleton
  for the per-session grant.
- BridgeStatusReporter: 30s coroutine emitting bridge.status with screen_on,
  battery, current_app, accessibility_enabled.

New network/handlers/BridgeCommandHandler.kt: routes bridge.command envelopes
to ActionExecutor, emits bridge.response. Paths /ping, /tap, /tap_text,
/type, /swipe, /scroll, /press_key, /wait, /screen, /screenshot,
/current_app. Gated on master enable + service availability.

Wired into ChannelMultiplexer (marked with PHASE3-γ comments), and
ConnectionViewModel instantiates the handler + reporter and starts ticking.
AndroidManifest declares the service with BIND_ACCESSIBILITY_SERVICE
permission, intent filter, and meta-data pointing at the flavor-provided
@xml/accessibility_service_config. Adds FOREGROUND_SERVICE,
FOREGROUND_SERVICE_MEDIA_PROJECTION, POST_NOTIFICATIONS for Wave 2.

Known blockers (documented in DEVLOG + file docstrings):
- Agent δ owns the MediaProjection consent ActivityResultLauncher flow;
  ScreenCapture.createConsentIntent() + MediaProjectionHolder.onGranted()
  are the integration points.
- Agent α owns POST /media/upload on the relay; current /media/register is
  loopback-only + path-based so the phone can't use it. /screenshot
  surfaces a clear 404 message until that lands.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 17:09:36 -04:00
Bailey Dixon 99835344d3 feat(phase3-δ): rewrite BridgeScreen with master toggle, permissions, status, activity log
Phase 3 Wave 1 Agent δ (bridge-screen-ui) — replaces the Bridge tab placeholder
with the four-card control surface from the Phase 3 plan. Cards: master toggle
with live device status, standalone status card, permission checklist with
tap-to-Settings launchers, scrollable activity log with tap-to-expand rows,
and an inert Safety placeholder owned by Wave 2 Agent ζ.

New:
- data/BridgePreferences.kt      — DataStore repo, JSON activity log (cap 100)
- viewmodel/BridgeViewModel.kt    — AndroidViewModel, system-probe permission state
- ui/components/BridgeMasterToggle.kt
- ui/components/BridgeStatusCard.kt
- ui/components/BridgePermissionChecklist.kt
- ui/components/BridgeActivityLog.kt
- ui/screens/BridgeScreen.kt      — full rewrite, ON_RESUME permission re-probe

γ-handoff stubs (TODO markers in BridgeViewModel):
- _bridgeStatus MutableStateFlow — γ replaces with HermesAccessibilityService flow
- A11Y_SERVICE_CLASS constant    — γ confirms final FQCN
- recordActivity write path       — γ's command dispatcher calls this
- master_enabled DataStore key    — γ's service reads as runtime disable switch

Each new component carries @Preview annotations for Android Studio iteration.
DEVLOG and CLAUDE.md Key Files updated.
2026-04-12 17:09:32 -04:00
Bailey DixonandClaude Opus 4.6 985632dafe feat(phase3-α): migrate legacy bridge relay into unified relay (port 8767)
Retires the standalone bridge relay (plugin/tools/android_relay.py +
the duplicate top-level plugin/android_relay.py, both listening on port
8766) and folds its functionality into the unified Hermes-Relay on port
8767 as the bridge channel. Wire protocol (bridge.command /
bridge.response / bridge.status) is unchanged — only the transport moved.

- plugin/relay/channels/bridge.py: real BridgeHandler with phone_ws +
  pending[request_id]→Future, asyncio.Lock-protected. handle_command()
  mints request_id, sends bridge.command, awaits bridge.response with
  30s timeout (matches legacy android_relay._RESPONSE_TIMEOUT).
  detach_ws() fails all pending futures with ConnectionError on phone
  disconnect so HTTP callers fail fast instead of hanging to timeout.

- plugin/relay/server.py: 14 HTTP routes (/ping, /screen, /screenshot,
  /get_apps, /apps legacy alias, /current_app, /tap, /tap_text, /type,
  /swipe, /open_app, /press_key, /scroll, /wait, /setup) delegate to
  _bridge_dispatch → BridgeHandler.handle_command. BridgeError →
  503/504/502 based on message. server.bridge.detach_ws(ws) wired into
  _on_disconnect so phone drops instantly fail in-flight commands.
  All additions bracketed by # === PHASE3-α: ... === / # === END
  PHASE3-α === markers for mechanical merges with Agent ε (notification-
  listener).

- plugin/android_tool.py + plugin/tools/android_tool.py: BRIDGE_URL
  default 8766 → 8767. _relay_port() falls back through
  ANDROID_RELAY_PORT → RELAY_PORT → 8767. android_setup() rewritten —
  no longer imports the deleted android_relay module, instead probes
  /health to verify the unified relay is up.

- plugin/android_relay.py + plugin/tools/android_relay.py: DELETED.

- plugin/tests/test_bridge_channel.py: new unittest suite (7 tests,
  all passing) covering envelope routing, future resolution, timeout
  cleanup, disconnect cleanup, send-failure cleanup, and the legacy-
  timeout regression guard. Run with:
    python -m unittest plugin.tests.test_bridge_channel

- DEVLOG.md + CLAUDE.md: Phase 3 / Wave 1 / α entry + Key Files row for
  plugin/relay/channels/bridge.py. Repo layout block trimmed
  android_relay.py.

Judgment call: bridge HTTP routes are unauthenticated at the HTTP layer,
matching the legacy relay. Trust boundary unchanged (localhost-only);
disconnected phone naturally 503s every call; bridge grant already
tracked in Session.grants["bridge"] so Wave 2 safety-rails can wrap
without touching this handler.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 17:05:43 -04:00
Bailey DixonandClaude Opus 4.6 dd52e2b4cb feat(phase3-β): add googlePlay and sideload build flavors with flavor-specific a11y manifests
Phase 3 Wave 1 / Agent β — introduces the Gradle flavor split the Bridge
channel needs before any AccessibilityService code lands. Google Play
reviews AccessibilityService heavily, so Phase 3 ships two release tracks
from one codebase:

  googlePlay — conservative use-case description ("read notifications,
               summarize messages, reply with your confirmation"),
               conservative event-type subset, no gestures, canonical
               applicationId for clean Play Store upgrades from v0.2.0.

  sideload   — full agent-control description, typeAllMask, gestures,
               flagRetrieveInteractiveWindows/flagReportViewIds/
               flagRequestTouchExplorationMode, applicationIdSuffix
               `.sideload` so both tracks coexist on the same device.

FeatureFlags.BuildFlavor exposes `current` / `displayName` / six
bridgeTier1..6 flags. Tiers 1, 2, 5 are baseline-true for both tracks;
tiers 3 (voice-first), 4 (vision-first), 6 (ambitious future) are
`get() = current == SIDELOAD` so R8 can fold the branch away in the
Play release build.

AboutScreen shows a small "Track: Google Play" / "Track: Sideload"
badge under the existing Version row, bracketed with PHASE3-β banners
so agents δ and ε can land follow-ups without merge conflicts.

The flavor manifests reference `.accessibility.BridgeAccessibilityService`
with `tools:ignore="MissingClass"` — the actual service class is Agent
γ's territory and will land in `app/src/main/kotlin/.../accessibility/`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 17:00:53 -04:00
Bailey DixonandClaude Opus 4.6 44732f5502 fix(voice): newline is always a sentence boundary regardless of next char
extractNextSentence required the char after a terminator to be whitespace
or end-of-buffer before accepting it as a sentence boundary. This works
for periods (avoids splitting "e.g.") but fails for newlines — "Hello
there\nmore text" didn't extract because '\n' was followed by 'm'.
Newlines are inherently sentence boundaries; no lookahead needed.

Fixes SentenceExtractionTest.`accepts sentence ending in newline`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 00:44:32 -04:00
Bailey DixonandClaude Opus 4.6 c482e8b2a3 fix(settings): ChevronRight → KeyboardArrowRight — icon not in compose-icons-extended
ChevronRight doesn't exist in the Material Icons Extended artifact this
project depends on (despite being listed on fonts.google.com). It was
introduced by the settings refactor agents and has been failing CI since
f733d93. KeyboardArrowRight is visually identical and definitely exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 00:39:41 -04:00
Bailey DixonandClaude Opus 4.6 99a8a3d7b1 release: v0.2.0
Voice mode, terminal (Phase 2), pairing + security architecture,
inbound media pipeline, settings refactor, classified error feedback,
relay .env autoload + systemd user service. 54 commits since v0.1.0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 00:35:20 -04:00
Bailey DixonandClaude Opus 4.6 985126ed0b fix(settings): revert ChevronRight to AutoMirrored — Filled.ChevronRight doesn't exist
The previous commit changed Icons.AutoMirrored.Filled.ChevronRight to
Icons.Filled.ChevronRight, but ChevronRight only exists under the
AutoMirrored namespace in Material Icons Extended. CI failed with
"Unresolved reference 'ChevronRight'" on all three usages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 00:11:21 -04:00
Bailey DixonandClaude Opus 4.6 26e7011fa8 chore: add app screenshots, .claude to gitignore, fix ChevronRight icon
- Untrack assets/screenshots/ from .gitignore and commit 8 app
  screenshots (splash, voice, chat, sessions, commands, terminal)
- Add .claude/ to .gitignore (Claude Code internal state — worktrees,
  image cache, conversation logs — should never be committed)
- SettingsScreen.kt: Icons.AutoMirrored.Filled.ChevronRight →
  Icons.Filled.ChevronRight (3 instances)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 00:10:36 -04:00
Bailey DixonandClaude Opus 4.6 d414b434d0 docs: sync CLAUDE.md key files + DEVLOG entries for voice/error/relay commits
CLAUDE.md Key Files table:
- Added VoiceSfxPlayer.kt, VoiceWaveform.kt, RelayErrorClassifier.kt,
  _env_bootstrap.py entries
- Updated VoiceRecorder.kt (perceptual curve), VoicePlayer.kt (NaN guard),
  VoiceViewModel.kt (ignoreAssistantId, errorEvents, envelope, tryReceive
  consumer), VoiceModeOverlay.kt (scroll, stop button, errorEvents param),
  install.sh (6 steps) descriptions

DEVLOG.md:
- Added entries for voice polish (cca41bf), error feedback (0c8a035),
  and TTS waveform fix (5e093b7) — all three were committed without
  DEVLOG entries during the rapid session

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 00:09:19 -04:00
Bailey DixonandClaude Opus 4.6 5e093b7bc8 fix(voice): waveform stays alive through multi-sentence TTS playback
The waveform was flatlinining after the first sentence of a multi-
sentence response while audio kept playing. Root cause: maybeAutoResume()
was called after EVERY sentence in the TTS consumer loop. The SSE stream
almost always finishes before TTS plays all queued sentences, so
streamObserverJob?.isActive was already false by the time sentence 1
completed — maybeAutoResume transitioned to Idle, the amplitude bridge
stopped forwarding player output, and the waveform died.

Fix: restructured the TTS consumer from a for-loop to a while-loop with
tryReceive peek. maybeAutoResume only fires when the queue is actually
drained (tryReceive returns failure), not after every sentence. Between
sentences of a multi-sentence response, tryReceive succeeds immediately
(next sentence is already queued) and the consumer skips maybeAutoResume
entirely — state stays Speaking, amplitude bridge stays wired, waveform
keeps rendering.

Additionally, the consumer re-asserts Speaking state before each
synthesis call. If the queue was briefly empty between sentences (race
between the stream observer pushing and the consumer pulling),
maybeAutoResume may have transiently toggled to Idle — the re-assertion
corrects it before the new sentence's audio starts playing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 00:05:35 -04:00
Bailey DixonandClaude Opus 4.6 0c8a035b19 feat(ux): classified error feedback across voice/chat/settings/pairing
Ends the era of user-hostile "error: unknown" strings. Every network
or permission failure now names what broke and — where it makes sense —
offers a retry or "Open Settings" action. Built on a single classifier
+ a global SnackbarHost so any ViewModel can surface a human error
without owning its own banner.

Foundation (util/RelayErrorClassifier.kt + ui/RelayApp.kt)
- New HumanError(title, body, retryable, actionLabel) data class +
  classifyError(Throwable?, context: String?) helper. Pure Kotlin,
  no Compose, no Android deps.
- Branch order is load-bearing: UnknownHostException → ConnectException
  → SocketTimeoutException → SSLPeerUnverifiedException / SSLException
  → SecurityException → IllegalStateException → IOException (with
  message-substring scan for 401/403/404/413/500/503) → default. SSL
  before IOException matters because SSLException extends IOException;
  if someone reshuffles the when, cert mismatches silently become
  generic "Network error" retries. Comment in the file flags this.
- Context tags ("transcribe", "synthesize", "voice_config", "record",
  "pair", "save_and_test", "media_fetch", "send_message") shape the
  title and the 404 body (e.g., context=voice_config + 404 → "The
  relay doesn't have voice endpoints — it may be an older version").
- New top-level LocalSnackbarHost: CompositionLocal<SnackbarHostState>
  in RelayApp.kt, seeded with a single remembered state at RelayApp
  scope and wired into the Scaffold's snackbarHost slot. NavHost
  content is wrapped in CompositionLocalProvider so every screen
  reaches it via `LocalSnackbarHost.current`.
- New extension `suspend fun SnackbarHostState.showHumanError(err)`
  picks SnackbarDuration based on err.retryable.

ViewModel pattern (VoiceViewModel, ChatViewModel)
- Each ViewModel gets a MutableSharedFlow<HumanError>(replay=0,
  extraBufferCapacity=4, onBufferOverflow=DROP_OLDEST) exposed as
  val errorEvents: SharedFlow<HumanError>. One-shot event semantics —
  collectors don't re-fire on recomposition.
- Private helper (surfaceError / emitError) wraps
  `classifyError(t, context) → tryEmit → _uiState.error`. Generic
  try/catch bodies that used to build "${e.message ?: "unknown"}"
  strings now call surfaceError(e, context = "…") — one line.
- setError(String) still exists for control-flow messages that don't
  have a Throwable ("No speech detected", "Recorder not initialized",
  "Voice pipeline not initialized"). Exceptions go through the
  classifier; sentinel paths stay as explicit strings.

Voice (VoiceViewModel + VoiceModeOverlay + VoiceSettingsScreen)
- 5 error sites converted to surfaceError:
    processVoiceInput transcribe failure → context="transcribe"
    startTtsConsumer synthesize failure  → context="synthesize"
    testVoice synthesize failure         → context="synthesize"
    startListening MediaRecorder failure → context="record"
    stopListening MediaRecorder failure  → context="record"
- VoiceModeOverlay gains an optional trailing `errorEvents:
  SharedFlow<HumanError>? = null` parameter with a LaunchedEffect at
  the top that forwards each emission to
  LocalSnackbarHost.current.showHumanError(err). Default-null so the
  ChatScreen call site compiles without rewrite.
- VoiceSettingsScreen feeds `voiceConfigError` through classifyError
  (context="voice_config") and collects voiceViewModel.errorEvents
  so Test Voice button failures surface as snackbars while on-screen.
- Inline error banner (AnimatedVisibility over uiState.error) kept as
  the longer-lived display — belt and suspenders with the transient
  snackbar.

Chat (ChatViewModel + ChatScreen)
- ChatViewModel gets the same errorEvents SharedFlow + emitError helper.
- Converted error sites:
    performFetchWith onSuccess cache catch → context="media_fetch"
    performFetchWith onFailure branch     → context="media_fetch"
    startStream onErrorCb                 → context="send_message"
    sendMessageInternal "Failed to create chat session" fallback
                                          → context="send_message"
  Inbound attachment cards still show their in-place error state AND
  the snackbar pops — attachments survive scroll, snackbar is the
  immediate attention-grabber.
- ChatScreen wires `LaunchedEffect(chatViewModel) { errorEvents.collect
  { snackbarHost.showHumanError(it) } }` immediately after the voice
  state hoist. VoiceModeOverlay is now invoked with
  `errorEvents = voiceViewModel.errorEvents` so overlay-scope errors
  also reach the global snackbar.

Microphone permission affordance (ChatScreen)
- The existing "microphone permission denied" AnimatedVisibility banner
  was a single Text row. Rebuilt as a Column with:
    titleSmall "Microphone permission needed"
    bodySmall  "Tap Open Settings and grant Microphone access to use
                voice mode."
    Row(Dismiss, Open Settings)
  "Open Settings" fires
  Intent(Settings.ACTION_APPLICATION_DETAILS_SETTINGS,
         Uri.fromParts("package", context.packageName, null))
  + FLAG_ACTIVITY_NEW_TASK. Users can now grant the permission without
  hunting through Android's settings app. Banner location + voice
  overlay mounting + smart-swap input bar all untouched.

Connection Settings (ConnectionSettingsScreen)
- Save & Test → RelayReachable.Fail branch now renders
  classifyError(Exception(r.message), context="save_and_test") as a
  HumanError title + body. TODO left for the ViewModel refactor that
  would thread the raw Throwable through RelayReachable.Fail so we can
  classify by exception type instead of message-string rewrap.
- Manual pairing code entry: "Error: unknown" replaced with
  `classifyError(e, context="pair").body` — distinguishes QR parse
  failures, relay unreachable, and code rejection for the first time.
  Existing "Server rejected code: …" path (terminal.reason) left
  alone since it's already specific.
- API Save & Test (the "API server reachable" / "Cannot reach API
  server" Toast next to the relay probe) was out-of-scope for this
  pass — flagged in the agent report for a follow-up.

Scope / non-goals
- No global rewrite of every try/catch(_) — 23 sites still exist, most
  safe (SFX playback, file cleanup) but the audit flagged a handful
  that deserve individual review. Deferred to a targeted follow-up.
- ConnectionViewModel.testRelayReachable still discards the Throwable
  and stores only err.message on RelayReachable.Fail. Save & Test
  classification works but loses fidelity vs. a raw-Throwable path.
  Leaving as-is for now per "don't refactor ViewModel APIs mid-pass".

No new Gradle dependencies. Existing import surface preserved
(animateFloat / rememberInfiniteTransition weren't regressed; every
existing @Preview still compiles against the same inputs).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 23:55:51 -04:00
Bailey DixonandClaude Opus 4.6 1ddb1b35dc feat(relay): .env autoload in Python + systemd user service via install.sh
Fixes the drift bug where /voice/transcribe would 500 with "STT provider
'openai' configured but no API key available" after any relay restart
that wasn't preceded by a manual `source ~/.hermes/.env`. Mirrors how
hermes_cli/main.py bootstraps env for the gateway — which is why the
gateway systemd unit has no EnvironmentFile= directive — so the relay
can now live in the same canonical systemd user-service shape.

plugin/relay/_env_bootstrap.py (new)
- load_hermes_env() helper. Prefers hermes_cli.env_loader.load_hermes_dotenv
  when importable so precedence and encoding fallbacks match the rest
  of Hermes; falls back to direct python-dotenv against
  \$HERMES_HOME/.env (or ~/.hermes/.env) with override=True; silent
  no-op if neither is available (stripped containers that provide env
  via docker run -e ...).

plugin/relay/__main__.py + relay_server/__main__.py
- Both entry points call load_hermes_env() BEFORE importing .server,
  so any module in the import chain that reads os.getenv at module
  level sees the same env regardless of launcher. noqa: E402 on the
  server import — order is deliberate.

relay_server/hermes-relay.service (rewritten)
- Was a stale system-level unit hardcoded to /home/bailey/hermes-relay
  using /usr/bin/python3 — nobody was using it correctly. Rewritten as
  a user unit matching hermes-gateway.service exactly:
    ExecStart=%h/.hermes/hermes-agent/venv/bin/python -m plugin.relay ...
    WorkingDirectory=%h/.hermes/hermes-relay
    Environment="PATH=%h/.hermes/hermes-agent/venv/bin:..."
    Environment="VIRTUAL_ENV=%h/.hermes/hermes-agent/venv"
    Environment="HERMES_HOME=%h/.hermes"
    Restart=on-failure / RestartSec=30
    StartLimitIntervalSec=600 / StartLimitBurst=5
    StandardOutput=journal
    WantedBy=default.target
- systemd's %h expands to the user's home, so the template is
  user-agnostic — no hand-editing required on a default install.
- NO EnvironmentFile= on purpose: _env_bootstrap.py handles it. Any
  future `systemctl --user restart hermes-relay` picks up fresh values
  from .env without sourcing anything in the shell.

install.sh step [6/6] (new)
- Idempotent. Drops the unit into ~/.config/systemd/user/, runs
  daemon-reload, and `enable --now`s the service.
- Skipped gracefully on hosts without a systemd user session:
    - systemctl not in PATH (macOS, some BSDs)
    - systemctl --user show-environment fails (bare chroots, WSL
      without systemd, containers without a user bus)
    - HERMES_RELAY_NO_SYSTEMD env var set (explicit opt-out)
- Detects orphan nohup-launched python -m plugin.relay processes
  holding :8767 and tells the user to kill them first rather than
  racing the installer.
- Parting hint about `loginctl enable-linger \$USER` for users who
  want the relay to survive SSH logout.
- Final echo updated with the systemctl --user / journalctl --user
  management commands.

Docs
- docs/relay-server.md — new Quick Start with install.sh as the
  recommended path, manual-run + Docker preserved, new ".env
  auto-loading" subsection explaining precedence and why there's no
  EnvironmentFile=. File table now lists _env_bootstrap.py and flags
  the service template as user-unit / no-EnvironmentFile.
- user-docs/reference/relay-server.md — mirror of the above in
  user-facing style. Two new troubleshooting entries: voice 500
  "no API key available" → check .env + restart, and service stops
  on SSH logout → loginctl enable-linger.
- CLAUDE.md — Server Deployment section updated to list
  hermes-relay.service as a user systemd unit, restart command is
  `systemctl --user restart hermes-relay`, env verification via
  `cat /proc/\$PID/environ` through systemctl show -p MainPID,
  old nohup instructions carry a history pointer to the DEVLOG entry.
  Table row for Relay log now points at journalctl --user -u
  hermes-relay (legacy ~/hermes-relay.log noted as historical only).
- DEVLOG.md — new entry at the top covering the full fix: root cause,
  new files, unit rewrite, installer step, docs updated, rationale
  for why this shape is right for the general plugin path.

No breaking changes to existing callers — manual `python -m plugin.relay`
still works and is actually more robust than before because it now
auto-loads .env the same way the installed service does.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 23:25:10 -04:00
Bailey DixonandClaude Opus 4.6 cca41bf0b0 feat(voice): reactive waveform, pill edges, stop-to-idle, enter/exit chimes
Polish pass on voice mode addressing everything that surfaced during live
testing: waveform felt dead at normal speech levels, edges cut off hard,
stop button during Speaking was ambiguous, agent voice sometimes replied
to the previous turn, long responses clipped in the overlay with no scroll,
and there was no audible cue for entering/exiting voice mode.

Waveform (VoiceWaveform.kt + VoiceRecorder.kt + VoiceViewModel.kt)
- Perceptual amplitude curve at the source (VoiceRecorder): subtract a
  NOISE_FLOOR, rescale against a SPEECH_CEILING, sqrt-boost. Normal speech
  (raw 3000/32767 ≈ 0.09) now maps to ~0.48 instead of ~0.22.
- Attack-fast / release-slow envelope follower in VoiceViewModel
  (bridgeAmplitudeFlows). Attack 0.75 / release 0.10 at 60 Hz gives
  ~30 ms peak attack and ~350 ms decay — the Siri "spike on speech,
  trail on silence" feel. Replaces the Compose spring that was adding
  ~200 ms of double-smoothing lag.
- Killed the animateFloatAsState(spring) smoothing and the downstream
  pow(0.5) boost — both already done upstream. Amplitude now flows
  straight from mic to pixels.
- Replaced rememberInfiniteTransition + three animateFloat phase clocks
  with a single withFrameNanos ticker (rememberAmplitudeDrivenPhases).
  Phase velocity scales with amplitude via PHASE_AMP_BOOST (1 + amp*2.5),
  so silence runs at base tempo and loud speech pushes cycles 3.5× faster.
  Makes the wave visibly "surge" when the user speaks.
- Faster base phase durations (2000/1400/950 ms from 5200/3300/2100).
- Pill/lens edge merge via dual technique: (1) geometric sin(π*t) taper
  on the per-sample envelope forces wave excursion to zero at both
  endpoints — the line meets centerY naturally at x=0 and x=width, (2)
  BlendMode.DstIn horizontal gradient mask inside a drawIntoCanvas
  saveLayer as a belt-and-suspenders alpha fade over the outer 12%.
  Conjure's pill approach paints a dark gradient over the wave against
  a solid pill background; that doesn't work on our translucent overlay,
  hence the layer + DstIn.
- NaN guards in VoicePlayer.computeRms (NaN propagates through coerceIn
  silently per IEEE 754) and VoiceViewModel.sanitizeAmplitude to prevent
  Compose's Android 15 variable-refresh-rate math from spamming
  setRequestedFrameRate=NaN on every draw pass.

Stream observer (VoiceViewModel)
- Capture ignoreAssistantId = the current last assistant-message id
  BEFORE calling chatVm.sendMessage. The stream observer early-returns
  on any emission whose lastAssistant.id matches that baseline. Without
  it, StateFlow.collect replays the previous turn's response as a giant
  "delta" for the new turn and TTS speaks the wrong answer.
- interruptSpeaking() rewritten to actually stop everything: drain
  ttsQueue → player.stop → chatViewModel.cancelStream (so the agent
  stops generating tokens) → cancel streamObserverJob + currentTurnJob
  → reset sentenceBuffer / lastObservedMessageId / lastObservedContentLength
  / speakEnvelope → transition to Idle (not Listening — Bailey's mental
  model is "stop = ready to start new turn"). The old version only
  paused playback, so the observer kept feeding ttsQueue from the
  still-running SSE stream.

VoiceSfxPlayer.kt (new, 146 lines)
- Pre-synthesized 200 ms PCM chimes played via AudioTrack.MODE_STATIC
  with AudioAttributes USAGE_ASSISTANT + CONTENT_TYPE_SONIFICATION.
- Enter chime = 440→660 Hz ascending perfect-fifth sweep. Exit chime
  is the descending mirror. 15 ms linear attack, hold, 40 ms half-cosine
  decay — click-free at both ends. Peak 0.35 of full-scale so it reads
  as UI feedback, not a notification.
- Phase accumulated from instantaneous frequency (phase += 2π·f(t)/sr)
  rather than sin(2π·f(t)·t) to avoid chirp artifacts when f itself
  varies with t.
- Failed AudioTrack construction on weird OEM devices becomes a silent
  no-op rather than crashing voice mode — the fallback is "voice works,
  just no chime."
- Wired into VoiceViewModel via a new 5th initialize() parameter.
  enterVoiceMode calls playEnter before the state update; exitVoiceMode
  calls playExit before teardown so the AudioTrack release doesn't
  cut it off.

VoiceModeOverlay.kt (scroll + stop UX)
- Response text Column now weight(1f, fill=false) + verticalScroll so
  long agent responses scroll inside the remaining space between the
  waveform and mic button without clipping off-screen.
- Speaking-state mic button container is Color(0xFFE53935) (same
  Material Red 600 as Listening) and the icon is Icons.Filled.Stop —
  users can now see they can tap to interrupt. Material 3 dark-theme
  colorScheme.error resolves to a soft pink (#F2B8B5) which reads as
  "pale" not "STOP" on a circular mic button, so the red is hardcoded
  for this affordance.

ChatScreen.kt (smart-swap send/mic)
- Trailing input-bar button smart-swaps between Mic (empty input → tap
  to enter voice mode) and Send (any text/attachment → tap to send).
  Removes the floating Mic FAB that was colliding with the scroll-to-
  bottom button. Stop button stays as a separate IconButton visible
  during streaming.

build.gradle.kts
- silenceAndroidViewLogs Gradle task registered with group=hermes,
  hooked via afterEvaluate → install* finalizedBy. Runs adb shell
  setprop log.tag.View SILENT on every Android Studio install to work
  around Compose's Android 15 setRequestedFrameRate=NaN log spam at
  ~120 entries/sec. Resolves the Android SDK via local.properties
  sdk.dir + \$ANDROID_HOME + \$ANDROID_SDK_ROOT fallback; skipped via
  onlyIf when adb isn't found. Nice-to-have, not a build requirement.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 23:24:25 -04:00
Bailey Dixon f733d932ae feat(settings): replace mega-Column root with category-list landing page
Phase C of the SettingsScreen split. The mega 2609-line SettingsScreen.kt
is now ~270 lines — a compact category list matching the VoiceSettingsScreen
pattern. Top-level structure:

  - Connection quick-look card with live API / Relay / Session status
    rows (uses the existing ConnectionStatusRow component) and taps
    through to the Connection sub-screen where the full pairing /
    manual-config UX now lives.
  - Category rows for Chat / Voice / Media / Appearance / Paired Devices /
    Analytics, each navigating to its dedicated sub-screen.
  - Developer options row, gated on FeatureFlags.devOptionsUnlocked so it
    only appears after the tap-7x version unlock.
  - About row always visible at the bottom.

RelayApp.kt's Settings composable route passes nine navigation callbacks
to plumb each category into its dedicated route. The old
onNavigateToPairedDevices / onNavigateToVoiceSettings pair is kept
(routed the same way) so behavior doesn't regress.

All inline content that used to live in the mega-Column — pairing card,
manual-configuration expandable, bridge pairing code, chat toggles, media
sliders, appearance picker, stats for nerds, data management, developer
feature flags, about, dialogs — now lives in the per-category sub-screens
extracted in Phase B (commits 86123e3, 3c4bfc9, 9786dc5). This commit is
purely the root rewrite + navigation wiring; none of the behavior moves
further.
2026-04-11 23:21:09 -04:00
Bailey Dixon e0592b1d84 merge: fill Appearance/Analytics/Developer/About sub-screens (agent-polish) 2026-04-11 23:18:05 -04:00
Bailey Dixon a56a728326 merge: fill Chat + Media sub-screens (agent-chat-media) 2026-04-11 23:18:05 -04:00
Bailey Dixon 1992788460 merge: fill Connection sub-screen (agent-conn) 2026-04-11 23:17:59 -04:00
Bailey Dixon 9786dc5add feat(settings): fill Appearance/Analytics/Developer/About sub-screens
Phase B extraction. Four polish sub-screens filled in with content
extracted verbatim from the mega SettingsScreen:
- Appearance: theme picker + font-scale + animation toggles
- Analytics: Stats for Nerds display + reset
- Developer: feature flags + DataManagementSection unfolded inline
- About: version info + links + tap-7x developer-options unlock

SettingsScreen.kt untouched; Phase C deletes the relocated content and
replaces the mega-Column root with a category-list layout.
2026-04-11 23:17:16 -04:00
Bailey Dixon 86123e3f4b feat(settings): fill Connection sub-screen with extracted section
Phase B extraction of the Connection section from the mega SettingsScreen
into ConnectionSettingsScreen. Pairing card, manual configuration
expandable, bridge pairing-code expandable, connection status row,
action buttons, and all associated dialogs now live in the new dedicated
file. Behavior is unchanged — this is a pure relocation.

SettingsScreen.kt is untouched in this commit; Phase C removes the
extracted section from the mega file and replaces the root with a
category-list layout.
2026-04-11 23:17:16 -04:00
Bailey Dixon 3c4bfc96e2 feat(settings): fill Chat and Media sub-screens with extracted sections
Phase B extraction. Chat-behavior knobs (show reasoning, smooth auto-
scroll, tool display, message length, attachment size, endpoint
preference) now live in ChatSettingsScreen. Inbound media knobs (max
size, auto-fetch threshold, auto-fetch on cellular, cache cap, clear
cache) now live in MediaSettingsScreen. InboundMediaSection private
helper unfolded inline; formatBytesHuman copied into the media file.
SettingsScreen.kt untouched; Phase C deletes the relocated content.
2026-04-11 23:15:14 -04:00
Bailey Dixon b1b647e183 feat(settings): scaffold per-category sub-screens + navigation routes
Phase A of the SettingsScreen split. Adds:
- 7 stub sub-screens (Connection / Chat / Media / Appearance / Analytics /
  Developer / About), each with a Scaffold + back-arrow TopAppBar matching
  the existing VoiceSettingsScreen pattern. Bodies are TODO placeholders
  to be filled in by Phase B extraction.
- 7 new Screen sealed-class entries and composable() routes in RelayApp.kt
  so the stubs are reachable from navigation even before content lands.
- SettingsExpandableCard hoisted from SettingsScreen.kt into its own public
  component file so every sub-screen can reuse it without copy-paste.

The existing SettingsScreen.kt mega-file is unchanged in this commit —
the old single-Column layout is still the root Settings destination until
Phase C rewires it to a category-list root. Phase B agents will fill in
the sub-screen bodies by copying sections from SettingsScreen.kt into
their targets; Phase C will replace the mega-Column with the category list.
2026-04-11 23:10:48 -04:00
Bailey Dixon ad5ee21842 merge: global font-scale preference (agent-fontscale)
Phase 2 of the terminal feature drop. Font-size setting in
Settings → Appearance with four stops (0.85x/1.0x/1.15x/1.3x).

- ConnectionViewModel gets KEY_FONT_SCALE + fontScale StateFlow +
  setFontScale(). Value persisted in DataStore and restored on startup.
- HermesRelayTheme takes a fontScale parameter; when != 1.0f it wraps
  MaterialTheme in a CompositionLocalProvider(LocalDensity) that scales
  the existing density's fontScale so every Text/TextField across the
  app scales uniformly without a typography rewrite.
- TerminalWebView observes the flow and calls window.setFontSize via
  evaluateJavascript, which then triggers fitAddon.fit() + our
  layout-listener refit for a clean resize.

Agent: general-purpose, worktree-agent-a0d1a86d, commit d8e7c01.

# Conflicts:
#	app/src/main/kotlin/com/hermesandroid/relay/ui/components/TerminalWebView.kt
#	app/src/main/kotlin/com/hermesandroid/relay/ui/screens/TerminalScreen.kt
2026-04-11 23:00:47 -04:00
Bailey Dixon 030b00acfa merge: terminal tabs + scrollback search + session info sheet (agent-terminalui)
Phases 3-5 of the terminal feature drop.

- Tabs: TerminalViewModel refactored to a per-tab TabState list with a
  TabOutput-tagged SharedFlow. Up to 4 concurrent sessions, each sending
  a stable session_name (hermes-<device_id>-tabN) that composes with
  agent-tmux's tmux backend for per-tab persistence.
- Search: vendored @xterm/addon-search@0.15.0; window.searchNext /
  searchPrev / clearSearch exposed from index.html. New TerminalSearchBar
  Compose component, toggled from the TopAppBar.
- Session info sheet: tappable top-bar title opens a ModalBottomSheet
  (TerminalSessionInfoSheet) with per-tab metadata, transport badge,
  grant chips, and reattach/detach actions.

Load-bearing layout listener + onPageFinished refit + auth-gate combine
flow + window.refit(w, h) + ResizeObserver all preserved verbatim per
the pre-merge contract.

Agent: general-purpose, worktree-agent-addacf8d, commit cafa2c5.
2026-04-11 22:58:51 -04:00
Bailey Dixon 001ce82d8b merge: tmux-backed terminal persistence (agent-tmux)
Phase 1 of the terminal feature drop. Shells spawn inside tmux when
available (`tmux -u new-session -A -s hermes-<name>`) so phone disconnects
no longer kill the shell. Reattach lands in the same running session with
cwd, env, running processes, and scrollback intact. Client-detach / ws-drop
/ eof / shutdown all use _close_session(preserve_shell=True) to skip the
SIGHUP/reap path on the tmux branch. Bare-bash fallback unchanged.

Agent: general-purpose, worktree-agent-a7684d99, commit 57e1c08.
2026-04-11 22:58:29 -04:00
Bailey Dixon cafa2c5093 feat(terminal): tabs + scrollback search + tappable session info sheet
Three polish features on the now-working terminal tab:
- Up to 4 concurrent sessions with a Chrome-style tab strip. Each tab
  sends a stable session_name (hermes-<device_id>-tabN) over the wire so
  tmux-backed persistence lands in the same shell on reconnect.
- Scrollback search via @xterm/addon-search. Search icon in the TopAppBar
  reveals an inline search row; prev/next highlight matches in xterm.
- Tappable top-bar title opens a ModalBottomSheet with full session
  metadata (session name, pid, shell, grid, transport, grants, expiry)
  matching the Chat tab's agent-info overlay pattern. Detach / reattach
  buttons on the sheet for quick control.

All per-tab WebViews preserve the addOnLayoutChangeListener + onPageFinished
refit behavior that unblocks xterm rendering on Compose layout changes.
2026-04-11 22:56:33 -04:00
Bailey Dixon d8e7c017a4 feat(settings): global font-scale preference applied to Compose + xterm
New "Font size" control next to the theme picker in Settings → Appearance.
Four discrete stops (0.85x / 1.0x / 1.15x / 1.3x) persisted in DataStore.
Compose typography scales via LocalDensity.fontScale at the theme root, so
every Text/TextField across the app scales uniformly. xterm's fontSize is
pushed to the WebView via a LaunchedEffect observing the StateFlow, using
the existing window.setFontSize helper in index.html.
2026-04-11 22:48:12 -04:00
Bailey Dixon 57e1c085e9 feat(relay/terminal): tmux-backed shell for cross-reconnect persistence
Shells now run inside `tmux -u new-session -A -s <name>` when tmux is
available, so phone disconnects no longer kill the shell. Reattach on
the same session_name re-enters the running session with cwd, env,
running processes, and scrollback intact. Bare-bash fallback retained
for hosts without tmux. Client-initiated detach preserves the tmux
session; only WS-drop / EOF / server-shutdown in the bare-bash path
still hup+kill.
2026-04-11 22:46:15 -04:00
Bailey Dixon 182d5a4ca5 fix(terminal): force xterm fit from Kotlin layout listener + gate attach on auth
Two entangled bugs were hiding behind "terminal tab connects but no command
output appears":

1. Chromium WebView on Android does NOT reliably propagate native-side
   resize into the HTML viewport. `html, body { height: 100% }` resolves
   against a stale 0-height viewport on the first composition pass (the
   WebView is measured and HTML-loaded before Compose has finished
   laying out the AnimatedContent tab switch), so fitAddon latches at
   rows=1. Output scrolls straight off the single-row viewport.

   Fix: bypass Chromium's viewport math entirely. An OnLayoutChangeListener
   on the WebView posts `evaluateJavascript("window.refit(w, h)")` every
   time Compose lays out the view, passing the real CSS-pixel dimensions
   derived from resources.displayMetrics.density. The new `window.refit`
   force-sets `document.documentElement`, body, and #terminal widths/heights
   explicitly before calling `fitAddon.fit()`, so fit reads a sane
   clientHeight and computes proper cols/rows. Also re-fired on
   onPageFinished in case the layout listener raced the HTML load.

2. On every reconnect TerminalViewModel.sendAttach won a race against
   AuthManager.authenticate() and sent `terminal.attach` before the
   `system/auth` envelope. Relay rejected with "expected system/auth,
   got terminal/terminal.attach" and the phone's AuthState flipped to
   Failed — which the UI rendered as "pair cleared, re-pair required"
   on every app rebuild.

   Fix: TerminalViewModel.initialize now takes a StateFlow<AuthState> and
   uses combine(connectionState, authState) to only fire attach when BOTH
   Connected AND Paired. Introduced isReadyForChannelMessages() helper;
   onTerminalReady and reattach route through it.

Also drops the LAYER_TYPE_HARDWARE hint on the WebView (Samsung's GPU
layer occasionally fails to propagate xterm DOM updates) and adds
Log.i lines for onTerminalReady / resize cols/rows + a console.log
diagnostic inside writeTerminal so the next on-device test shows the
real geometry in logcat.
2026-04-11 22:41:28 -04:00
Bailey Dixon 309f132d61 fix(pair): drop QR error-correction to low for smaller terminal render
The signed payload is long enough (~500-800 bytes with HMAC signature)
that error="m" was producing a high QR version number and a visibly
oversized terminal render. Dropping to error="l" saves 2-3 QR versions
(~16-24 fewer modules across), which is the single biggest lever left
after compact=True + border=1. Applied to both terminal and PNG paths
so they stay consistent.

Scannability is unaffected on a phone camera at normal distances —
error="l" still has 7% redundancy, well above the noise floor for a
direct-scan use case where the QR is rendered cleanly on screen.
2026-04-11 22:18:54 -04:00
Bailey Dixon f3f8fd2fc2 fix(pair): render terminal QR compactly with border=1
Shrinks the hermes-pair / /hermes-relay-pair QR so it fits a normal
terminal window. Was already using compact=True (half-blocks); adds
border=1 to drop the default 4-module quiet zone. Content, signing,
and scan reliability unchanged.
2026-04-11 21:55:09 -04:00
Bailey Dixon 28ff502bf1 Merge branch 'feature/voice-mode' — real-time voice mode via relay TTS/STT
4-phase voice conversation feature built in a shared worktree by a 4-agent team:
V1 relay /voice/* endpoints + V2a audio pipeline + V2b UI/settings + V3 sphere
Listening/Speaking states. Full session notes in DEVLOG.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

# Conflicts:
#	DEVLOG.md
2026-04-11 21:20:02 -04:00
Bailey DixonandClaude Opus 4.6 672fa8acf0 feat(voice): real-time voice mode via relay TTS/STT endpoints
Adds end-to-end voice conversation. Tap the mic FAB in chat to enter
voice mode: MorphingSphere expands to fill the screen, listens while
you speak, and performs the agent's response as it streams back via
sentence-level client-chunked streaming TTS.

Server (plugin/relay):
- POST /voice/transcribe — multipart audio → text, wraps
  tools.transcription_tools.transcribe_audio in asyncio.to_thread
- POST /voice/synthesize — JSON text → audio/mpeg file, wraps
  tools.tts_tool.text_to_speech_tool
- GET /voice/config — provider availability from ~/.hermes/config.yaml
- Bearer auth matching /media/* pattern
- 14 unit tests (plugin/tests/test_voice_routes.py), all passing

Android (app):
- VoiceRecorder (MediaRecorder/AAC/.m4a, amplitude StateFlow at 60fps)
- VoicePlayer (MediaPlayer+Visualizer, OEM fallback to flat amplitude)
- RelayVoiceClient (OkHttp multipart+JSON, mirrors RelayHttpClient)
- VoiceViewModel (turn state machine, sentence detection via
  extractNextSentence helper, Channel<String> TTS queue consumer)
- VoiceModeOverlay (full-screen UI, tap/hold/continuous modes,
  haptics, interruption)
- VoiceSettingsScreen + VoicePreferences (DataStore)
- 11 sentence-extraction unit tests

MorphingSphere:
- New Listening state — soft blue/purple, subtle wobble with user
  amplitude (±15%)
- New Speaking state — vivid green/teal, dramatic core-warmth pulse
  with agent amplitude (±80%), data ring spin up to 4× on peak
- voiceAmplitude + voiceMode params, both defaulted so existing call
  sites unchanged
- 3 @Preview functions for Studio iteration

Integration:
- VoiceViewModel observes ChatHandler.messages StateFlow directly —
  zero changes to ChatViewModel. Transcribed text routes through
  normal chatVm.sendMessage() so voice utterances appear as regular
  user messages in chat history.
- Voice is a modality on top of chat, not a separate channel.

Docs:
- DEVLOG, CHANGELOG, README, CLAUDE.md updated
- docs/spec.md — new Phase V section replacing the future bullet
- docs/decisions.md — ADR covering relay-hosted vs upstream,
  buffer-not-stream TTS, m4a vs webm, ChatViewModel observation
- docs/relay-server.md + user-docs/reference/relay-server.md —
  /voice/* endpoint reference
- user-docs/features/voice.md (NEW) — end-user feature doc
- user-docs/guide/chat.md — voice mode section
- user-docs/features/index.md — promoted from "Future" to shipped

Known follow-ups (not blockers):
- Silence detector wiring (amplitude StateFlow exists, pref persists,
  but stopListening() auto-call not yet hooked)
- Continuous auto-resume uses streamObserverJob.isActive as the
  "turn done" signal — slightly imprecise, replace if Bailey sees
  stalled turns
- Voice config write-through from phone (Phase M territory)

Built in worktree feature/voice-mode via a four-agent team: one
server-side agent (V1), one audio-pipeline agent (V2a), one UI agent
(V2b+V4), one sphere-animation agent (V3). Full session notes in
DEVLOG.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 21:14:31 -04:00
Bailey Dixon 423eae6a4f chore(terminal): trace logs on input/output path for on-device diagnosis
Temporary diagnostic logging to narrow down why typing into the Terminal
tab clears the line with no output visible. Logs the four hops:
  1. WebView bridge.onInput (Kotlin)
  2. TerminalViewModel.sendInput (Kotlin)
  3. relay terminal.input write (Python — info level so default INFO run catches it)
  4. relay terminal.output flush + Android writeTerminal (both sides)

Will be removed once the root cause is identified.
2026-04-11 20:56:31 -04:00
Bailey DixonandClaude Opus 4.6 6b5da398aa fix(auth): clearSession tears down reconnect loop + reconnect gate
Self-revoke on Paired Devices blocked the phone's IP for 5 minutes. Root
cause: ConnectionManager's internal scheduleReconnect loop bypasses
ConnectionViewModel.connectRelay's hasPairContext gate entirely, and
AuthManager.clearSession doesn't call disconnect — so after a self-revoke
the reconnect loop kept firing with a freshly-regenerated *local* pair
code, hit "Invalid pairing code" 5 times in 4 seconds, and the rate
limiter did its job.

Two complementary fixes:

1. ConnectionViewModel.clearSession now calls disconnectRelay() before
   authManager.clearSession(). Order matters — tear down the reconnect
   loop before wiping the state it depends on.

2. ConnectionManager takes a new reconnectGate: () -> Boolean parameter,
   called both before scheduling the retry and after the backoff delay.
   ConnectionViewModel wires it as { authManager.hasPairContext } so the
   auto-reconnect stops firing if auth state says we shouldn't be trying.
   Default value preserves backwards compat with test call sites.

Either fix alone solves Bailey's immediate bug; together they harden
against future code paths that wipe auth state without calling disconnect.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 20:47:05 -04:00
Bailey DixonandClaude Opus 4.6 2cdf446db4 fix: paired devices JSON unwrap + permissive /media/by-path sandbox
Two on-device bugs shipped in one commit since they surfaced together.

1. Paired Devices crash — RelayHttpClient.listSessions was parsing the
   response body as a bare List<PairedDeviceInfo>, but the server returns
   {"sessions": [...]}. kotlinx.serialization crashed with "Expected start
   of the array '[', but had '{'". Fix: parse to JsonElement, pull out the
   sessions field, then decode via Json.decodeFromJsonElement(ListSerializer,
   element). Missing-field surfaces as a clean IOException.

2. "Path not allowed by relay sandbox" on legitimate LLM emits — the
   /media/by-path route enforced allowed_roots against every fetch, so
   when the LLM ran search_files and found an image in ~/projects/... the
   phone rendered a "Path not allowed" error card. The trust boundary is
   already the bearer-auth'd paired phone, and the LLM can exfiltrate
   bytes via plain text anyway, so the allowlist was defense-in-depth
   with a high false-positive rate.

   Flipped /media/by-path to permissive by default (C). File-level checks
   (absolute, exists, regular, size cap) remain unconditional. Added
   RELAY_MEDIA_STRICT_SANDBOX env var + RelayConfig.media_strict_sandbox
   flag (B) to re-enable allowlist enforcement for operators who want it.
   Token path (loopback POST /media/register) stays strict regardless.

Tests: existing RelayMediaRoutesTests class pinned strict=True to preserve
its legacy assertions. New RelayMediaByPathPermissiveTests class covers
the production default — most importantly the regression-guard test that
fetches a file outside any allowed root with a valid bearer and expects
200 + bytes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 20:38:04 -04:00
Bailey DixonandClaude Opus 4.6 4f2babfa34 feat(auth): Save & Test as HTTP health probe + pair-context gate + row affordance
Three related fixes from one testing cycle:

## 1. Save & Test is now a /health probe, not a WSS connect attempt

The Manual configuration button row had a Connect button that fired
connectRelay(url) unconditionally. Users with no session token who
tapped it to "test if my URL works" would trigger a WSS handshake,
send an auth envelope with a meaningless code, fail auth, and after
5 failures in 60s the relay's rate limiter would block their IP for
5 minutes.

Fix splits reachability from connect:

- New AuthManager.hasPairContext getter — true when authState is
  Paired/Pairing OR serverIssuedCode is non-null. Does NOT count the
  locally-generated phone->host Phase 3 code as valid pair context.

- New RelayHttpClient.probeHealth(relayUrl) — unauthenticated GET
  /health with 3s timeout, parses the response body, validates
  status=ok + non-blank version so a random HTTP server on 8767
  doesn't falsely pass. Returns version/clients/sessions metadata.

- ConnectionViewModel.connectRelay() now delegates to private
  connectRelayInternal which gates on hasPairContext. All legit
  entry points (pair walkthrough, stale row tap, Reconnect button,
  QR confirm, manual code entry) set pair context before calling
  connectRelay so the gate passes.

- New RelayReachable sealed interface + relayReachableResult state
  flow + testRelayReachable(url) method. Saves the URL THEN probes
  so a failed probe still leaves the URL persisted.

- Connect button deleted from Manual Configuration. Reconnecting is
  covered by the existing Reconnect button / stale row tap /
  reconnectIfStale on screen entry.

- Test button renamed to Save & Test. Result row reads from the new
  state flow. OutlinedTextField onValueChange clears stale results.

- Deleted the old callback-based testRelayReachability method and
  its now-unused okhttp3.Request import.

## 2. Bonus: Settings QR confirm never actually connected WSS

Pre-existing bug found during exploration. The Settings Scan Pairing
QR -> TTL picker confirm flow stashed serverIssuedCode + grants +
TTL but never called connectRelay. The WSS stayed disconnected until
the user manually tapped Reconnect or left and re-entered Settings.
Added the missing disconnectRelay() + connectRelay(relay.url) pair
to the TTL picker onConfirm. Pair-context gate passes because
applyServerIssuedCodeAndReset has just set serverIssuedCode.

## 3. Bonus: Connection status rows didn't look tappable

Bailey flagged that API Server / Relay / Session rows open info
drawers on tap but had no visual affordance. Users had to stumble
onto the tap target.

Fixed in ConnectionStatusRow:
- New onClick parameter. When non-null, the component applies a
  Material ripple + rounded-corner clip + internal clickable
  modifier so the whole row is a proper list-item tap target.
- Trailing chevron icon (AutoMirrored KeyboardArrowRight) whenever
  onClick is set. Standard Settings list-item cue.

All three SettingsScreen call sites migrated from
`modifier = Modifier.fillMaxWidth().clickable { ... }` to
`onClick = { ... }` + `modifier = Modifier.fillMaxWidth()`.
Legacy .clickable callers still work — the new interactive
wrapping is additive.

## Files

Phone only (no server changes):
- auth/AuthManager.kt: hasPairContext getter
- network/RelayHttpClient.kt: probeHealth + RelayHealth + jsonObject
  import
- viewmodel/ConnectionViewModel.kt: RelayReachable sealed interface,
  testRelayReachable, clearRelayReachableResult,
  connectRelayInternal with gate, deleted testRelayReachability
- ui/components/ConnectionStatusBadge.kt: onClick param + chevron
  affordance on ConnectionStatusRow
- ui/screens/SettingsScreen.kt: Save & Test wiring, Connect button
  removed, QR confirm connect fix, three status rows migrated to
  onClick pattern
- DEVLOG.md + CLAUDE.md updates

## Caveats

- Connect-button removal is a UX change. If Bailey was using it to
  force-reconnect when paired, use the Reconnect button in the
  Connection card instead (same path, gated on Paired).
- probeHealth timeout is 3s. On a very slow LAN this might false-
  fail; bump to 5s if needed.
- probeHealth doesn't follow redirects. A relay behind a reverse
  proxy that 301s /health would fail the probe. Current deployments
  don't do that.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 20:20:26 -04:00
Bailey DixonandClaude Opus 4.6 f0e851637a fix(settings): unify relay status on Settings entry as 'Reconnecting...' to kill flicker
Also adds a 'Typical Dev Loop' + 'Server Deployment' section to
CLAUDE.md so the Windows-edit / Linux-server-test workflow, the
hermes-gateway + relay restart commands, the unittest-not-pytest
escape for the pre-existing responses conftest, and the
Studio-only-APK-install rule don't have to be rediscovered each
session. No host-specific identifiers in the doc — sensitive
info lives in ~/SYSTEM.md on the server.

## Flicker

Opening Settings with a stored session token was flashing three
rapid states: red Disconnected → amber Stale → amber Connecting
→ green Connected. Root cause: authState defaults to Unpaired until
the EncryptedSharedPreferences read completes (~50ms), during
which isRelayStale is false and the row shows Disconnected. Then
authState flips to Paired → isRelayStale becomes true →
'Stale — tap to reconnect'. Then LaunchedEffect(Unit) fires
reconnectIfStale() → ConnectionState.Connecting → 'Connecting...'.
Users see the middle two transitions as a flicker.

Fix: screen-local isAutoReconnecting flag true for up to 5s on
screen entry (or until Connected lands). During that window the
status text is unified to 'Reconnecting...' for all non-Connected
sub-states. After 5s the flag drops and the row falls through to
the normal state machine, so the 'Stale — tap to reconnect'
affordance still works for genuine stale cases (backgrounding,
network flap, etc).

Purely presentational — underlying state machine unchanged.
isConnecting flag also treats isAutoReconnecting as in-flight so
the ConnectionStatusBadge pulse ring animates through the window.

## CLAUDE.md Dev Workflow

New subsections under 'Dev Workflow':
- Typical Dev Loop — 8-step flow for edit-in-Windows + test-on-
  server (python syntax check locally, unit tests on server, phone
  testing via Android Studio run button)
- Server Deployment — canonical paths (hermes-agent venv, hermes-
  relay clone, plugin symlink, relay log, qr-secret, config yaml)
  + the standard update cycle (git pull, systemctl restart gateway,
  pkill+nohup restart relay, unittest command)
- Where Python vs. Kotlin changes land — quick lookup table for
  what needs to be restarted/rebuilt when each kind of file changes

Explicitly notes: never install APKs from Claude (Bailey uses
Studio), never pytest without the unittest escape (conftest imports
the uninstalled 'responses' module), always nohup+disown or
setsid -f when launching the relay over SSH (otherwise ssh session
exit kills the detached process before it fully backgrounds).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 19:42:33 -04:00
Bailey DixonandClaude Opus 4.6 c209c9981e fix(auth): extend without grants regenerates from defaults, not absolute-time clamp
The first pass of SessionManager.update_session preserved old grants
via a _clamp_grants_to_lifetime helper when only ttl_seconds changed.
That made sense for shortening (clip grants that would outlive the
new session) but produced nonsense for extending:

Scenario: user pairs for 30d with default grants (terminal=30d,
bridge=7d). An hour later they tap Extend and pick 90d. With the
old code the grants kept their ORIGINAL absolute-time expiries —
terminal still at pair_time + 30d, bridge at pair_time + 7d — so
bridge effectively only got ~7d from the pair moment, not 7d from
'now'. Users would tap Extend expecting a fresh allocation and
find their grants were already stale.

Worse: for a 1h session with 1h grants, extending to 90d left all
grants at pair_time + 1h = roughly now, i.e. immediately expired.

Fix: when only ttl_seconds changes (grants is None), regenerate
grants from _default_grants(new_ttl, now). Handles both shorter
(grants correctly clipped to new cap) and longer (grants stretch
to sensible defaults for the new lifetime) uniformly. Users who
want to preserve custom grants across an extend must pass them
explicitly.

Deletes the now-unused _clamp_grants_to_lifetime helper.

Caught by test_extend_ttl_zero_means_never asserting that extending
a finite session to never-expire produces never-expire grants — the
old code left the finite grants in place.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 19:37:24 -04:00
Bailey DixonandClaude Opus 4.6 c4865edf49 feat(auth): grant renewal — PATCH /sessions/{prefix} + Extend button
Closes gap #4 from the pairing architecture follow-up list: no way to
change a session's TTL after initial pair without revoking and
re-pairing.

## Server

- auth.py: new SessionManager.update_session(token, ttl_seconds?,
  grants?). TTL restarts the clock from now (not old_expiry + new_ttl)
  so Extend by 30 days means 30 days from now. Grants semantics:
  passing grants re-materializes from scratch clamped to the new
  session lifetime; omitting grants re-clamps the existing ones via
  a new _clamp_grants_to_lifetime helper so shorter TTLs correctly
  clip grants that would outlive the session. Expired sessions are
  NOT silently resurrected.

- server.py: new handle_sessions_extend PATCH /sessions/{token_prefix}
  handler. Bearer auth, prefix-match-then-update pattern mirroring
  handle_sessions_revoke. Full body validation (non-negative integer
  ttl_seconds, non-negative numeric grant values, at least one field
  present) → 400 on any violation. Route registered via add_patch,
  no ordering collision with existing GET/DELETE on the same pattern.

- tests: 10 new tests covering bearer-required, 404 on prefix miss,
  400 on empty/invalid body, TTL-only restarts the clock (asserts
  new_expiry ~= now + new_ttl, NOT old_expiry + new_ttl),
  ttl_seconds=0 -> never, grants-only leaves expiry alone, shorter
  TTL clips grants, self-extend. Helper _settle() for the 1ms async
  delay needed on fast machines to make timestamp assertions
  meaningful.

## Phone

- RelayHttpClient.kt: new extendSession(tokenPrefix, ttlSeconds?,
  grants?) method. Hand-rolled JSON body (two optional fields,
  trivial and auditable). Error mapping 400/401/404/409/5xx to
  user-facing messages. 404 is a hard failure here (unlike revoke
  where already gone = success) - if you're extending an active
  session and it's gone, that's surprising.

- ConnectionViewModel.kt: new extendDevice(tokenPrefix, ttlSeconds):
  Boolean. Same shape as revokeDevice (refresh list on success,
  error via pairedDevicesError). Grants intentionally not exposed
  on this path - MVP UX is "pick a new duration", server-side
  re-clamping handles the rest.

- PairedDevicesScreen.kt: pendingExtend state alongside pendingRevoke.
  Reuses SessionTtlPickerDialog directly (title "Keep this pairing
  for..." is tense-neutral). Preselects the session's current
  remaining lifetime (expires_at - now, clamped to >= 0, or 0 for
  never-expire sessions). DeviceCard action row is now a 50/50 split
  between Extend and Revoke buttons.

## Not changed

- __version__ stays at 0.2.0 per prior direction
- No grant-editing UI in the MVP path - power users hit the endpoint
  directly
- No "extend by increment" - semantics are always "set new TTL from
  now", easier to reason about
- No new ADR - this is a small follow-up to ADR 15's architecture,
  not a new decision

## Docs

- docs/relay-server.md + user-docs/reference/relay-server.md: new
  PATCH /sessions/{token_prefix} route row
- user-docs/reference/configuration.md: Extend button description
  in the Paired Devices subsection
- CLAUDE.md: Integration Points table row
- DEVLOG entry with rationale, semantics, test plan

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 19:34:39 -04:00
Bailey DixonandClaude Opus 4.6 c1eb7d6c5d fix(auth): use getApplication<>() inside setInsecureAckComplete
The constructor parameter 'application: Application' is not declared
as a property (no 'val' prefix), so it's in scope during property
initializers but NOT inside method bodies. AndroidViewModel.application
is private in the base class, so accessing it directly fails at
compile time.

Every other method in ConnectionViewModel already uses
getApplication<Application>() for this reason. setInsecureAckComplete
was the sole exception — the Android agent's new code called
PairingPreferences.setInsecureAckSeen(application, true) inside a
viewModelScope.launch block, which doesn't compile.

Local 'val ctx = getApplication<Application>()' at the top of the
launch scope matches the pattern used elsewhere in the file.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 19:14:54 -04:00
Bailey DixonandClaude Opus 4.6 ff863c70ec test(auth): fix lowercase rejection test — register_code normalizes, doesn't reject
register_code has always called .upper() before validating against
PAIRING_ALPHABET, so lowercase input is accepted and normalized — not
rejected. The Python agent's new test_register_rejects_bad_format
asserted the opposite. Corrected the test to match actual behavior
and added test_register_normalizes_case as a regression guard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 19:07:05 -04:00
Bailey DixonandClaude Opus 4.6 11bf512eaa feat(auth): pairing + security architecture — grants, TTL picker, Keystore, TOFU, Paired Devices
Full overhaul of the pairing model + session security based on Bailey's
design request. Everything lives in files we own; no upstream patches.

## Why

The existing model was minimal: one-shot pairing codes → fixed-30-day
session tokens → no channel separation → EncryptedSharedPreferences
storage. After the inbound media work exposed the "phone rate-limited
to death on relay restart" gap, the entire pairing story was ready
for a pass. Bailey asked for: user-chosen TTL at pair time (including
"never"), per-channel grants, secure-by-default transport UX, device
revocation UI, Android Keystore storage, TOFU cert pinning, QR signing,
Phase 3 bidirectional pairing foundation, and Tailscale detection.

Never expire is explicitly always selectable per Bailey's direction:
dont force check, just allow based on user intent. User agency over
policy.

## Server - plugin/relay/

- auth.py: Session gains grants + transport_hint + first_seen; math.inf
  for never-expire (serializes as null); PairingMetadata dataclass
  threaded through PairingManager.register_code + consume_code (returns
  metadata or None, not bool); SessionManager create_session accepts
  ttl_seconds + grants + transport_hint; _materialize_grants clamps
  per-channel grants to session lifetime; list_sessions + find_by_prefix
  + revoke for the new routes; RateLimiter.clear_all_blocks() called
  unconditionally when /pairing/register succeeds (fixes the stale
  block prevents re-pair bug). Default caps: terminal 30d, bridge 7d.

- server.py: /pairing/register accepts ttl_seconds/grants/transport_hint
  metadata (host wins over phone); new /pairing/approve (Phase 3 stub);
  new GET /sessions (tokens masked to 8-char prefix, is_current flag);
  new DELETE /sessions/{prefix} (200/404/409, self-revoke flagged);
  _authenticate threads pairing metadata into new session;
  _detect_transport_hint sniffs SSL from request transport; auth.ok
  payload carries expires_at + grants + transport_hint (inf to null).

- qr_sign.py NEW: HMAC-SHA256 over canonicalized JSON. Canonical form
  strips sig so signing is idempotent. load_or_create_secret reads or
  creates ~/.hermes/hermes-relay-qr-secret (32 bytes, 0o600).
  allow_nan=False so signing math.inf crashes loudly.

- pair.py + cli.py: new --ttl and --grants flags. parse_duration for
  1d/7d/30d/90d/1y/never + loose patterns. parse_grants for
  terminal=7d,bridge=1d. build_payload(sign=True) auto-bumps hermes 1
  to 2 when v2 fields present and attaches HMAC signature.
  read_relay_config detects TLS via RELAY_SSL_CERT. render_text_block
  shows Pair for 30 days / indefinitely plus grant labels.

- __version__ stays at 0.2.0 per Bailey: we never released 0.2.0.

- Tests: 4 new test modules covering qr_sign, session_grants,
  sessions_routes, rate_limit_clear. Existing media tests still pass.

## Phone - app/src/main/kotlin/.../

- auth/SessionTokenStore.kt NEW: interface + KeystoreTokenStore
  (StrongBox-preferred) + LegacyEncryptedPrefsTokenStore fallback.
  tryCreate swallows exceptions. One-shot lossless migration.
  hasHardwareBackedStorage flag surfaced in UI.

- auth/CertPinStore.kt NEW: TOFU cert pinning. SHA-256 SPKI fingerprints
  per host:port in DataStore. recordPinIfAbsent on first successful wss
  handshake; buildPinnerSnapshot produces an OkHttp CertificatePinner.
  removePinFor called by applyServerIssuedCodeAndReset on QR re-pair.

- auth/PairedSession.kt NEW: PairedSession state class + PairedDeviceInfo
  wire model.

- auth/AuthManager.kt: lazy-init SessionTokenStore picker with
  migration; currentPairedSession StateFlow; parses expires_at/grants/
  transport_hint from auth.ok; authenticate(ttlSeconds) injects
  ttl_seconds + grants into pairing-mode auth envelope;
  applyServerIssuedCodeAndReset wipes TOFU pin for target host.

- data/PairingPreferences.kt NEW: DataStore keys pair_ttl_seconds,
  insecure_ack_seen, insecure_reason, tofu_pins.

- util/TailscaleDetector.kt NEW: NetworkInterface scan for tailscale0
  plus 100.64.0.0/10 CGNAT plus relay URL host check. Informational
  only.

- ui/components/SessionTtlPickerDialog.kt NEW: radio list
  1d/7d/30d/90d/1y/Never. Defaults: QR wins, wss/Tailscale 30d,
  ws 7d, unknown 30d. Never warning inline, NOT gated.

- ui/components/TransportSecurityBadge.kt NEW: three states, three
  sizes.

- ui/components/InsecureConnectionAckDialog.kt NEW: first-time consent
  with reason picker. Reason persists for display, not gating.

- ui/screens/PairedDevicesScreen.kt NEW: full list with transport
  badge + expiry + grant chips + revoke button. Self-revoke wipes
  local state + redirects to pair.

- network/RelayHttpClient.kt: new listSessions and
  revokeSession(tokenPrefix) methods. 404 handled as endpoint not yet
  implemented for forward-compat.

- network/ConnectionManager.kt: optional CertPinStore param. Rebuilds
  OkHttpClient on every connect with fresh CertificatePinner snapshot.

- ui/components/QrPairingScanner.kt: HermesPairingPayload gains sig
  field and defaults; RelayPairing gains ttlSeconds/grants/
  transportHint. parseHermesPairingQr no longer rejects on version
  mismatch.

- viewmodel/ConnectionViewModel.kt: wires AuthManager before
  ConnectionManager so pin store is available. Exposes
  currentPairedSession, pairedDevices, isTailscaleDetected,
  insecureAckSeen, insecureReason.

- ui/RelayApp.kt: new Screen.PairedDevices route.

- ui/screens/SettingsScreen.kt: TransportSecurityBadge + Tailscale
  chip + hardware storage chip + Paired Devices row in Connection
  card. Insecure toggle routes through ack dialog. QR scan handoff
  opens SessionTtlPickerDialog.

- ui/components/ConnectionInfoSheet.kt: SessionInfoSheet shows
  Expires / Channel grants / Transport / Key storage rows.

## Docs

DEVLOG entry, ADR 15, spec.md section 3.3/3.3.1/3.4 rewrites, relay
route tables in docs + user-docs, Paired Devices subsection in
configuration.md, pair flow extension in getting-started.md, CLAUDE.md
Current State + Key Files + Integration Points.

## Known gaps filed for follow-up

- Phone-side QR signature verification (parsing works; full verify
  needs secret distribution path)
- Bidirectional pairing full UX tied to Phase 3 bridge
- Per-device role model (admin vs user)
- Grant renewal UI on Paired Devices
- Build verification: no gradle run this session; Bailey deploys from
  Android Studio

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 19:06:04 -04:00
Bailey DixonandClaude Opus 4.6 af39525e33 feat(chat): inbound media v2 — /media/by-path + LLM bare-path fetch
On-device testing of inbound media v1 (commits 1195778 + 8f61262)
surfaced two bugs that both root-caused to the same missing piece.

## Bug 1: bare-path LLM emissions leak as raw text

Upstream hermes-agent/agent/prompt_builder.py:266 explicitly instructs
the LLM to "include MEDIA:/absolute/path/to/file in your response. The
file..." — so the LLM emits MEDIA markers as free-form completion text,
not as tool output. v1's token-registration flow (register_media + emit
MEDIA:hermes-relay://<token>) only covered markers emitted BY tools.
LLM free-form emissions — which are the common case, per the upstream
prompt — stayed as bare-path and rendered as raw text on the phone
because the phone's handler treated bare-path as a permanent "unavailable"
state.

## Bug 2: session-reload wipes injected attachments

Already fixed in 272a3c5 — ChatViewModel.onCompleteCb reloads history
at every turn end, and loadMessageHistory didn't re-run the marker
parser so client-injected attachments were wiped.

## Fix: new relay route GET /media/by-path + phone-side rewiring

Server-side — plugin/relay/:

- media.py: extracted validate_media_path(path, allowed_roots,
  max_size_bytes) -> (real_path, size) as a module-level helper. Both
  MediaRegistry.register (loopback-only tool path) and the new
  handle_media_by_path (phone-auth'd direct fetch) call it — sandbox
  rules can't drift between the two entry points.

- server.py: new handle_media_by_path route handler. Takes ?path=<abs>
  (required) and ?content_type=<optional hint>. Bearer auth against
  the existing SessionManager — same token the WSS channel uses.
  Content-Type: phone hint wins, else guessed via mimetypes.guess_type.
  Route registered BEFORE /media/{token} in create_app or aiohttp
  swallows the literal "by-path" as a token (commented).
  Error shapes: 400 missing path; 401 auth; 403 sandbox/too-large;
  404 missing file.

- 9 new route tests in test_relay_media_routes.py covering 401s,
  missing-path 400, outside-sandbox 403, relative-path 403, missing
  file 404, happy path with auto-guessed image/png content type,
  content_type hint override, and oversized 403 (with try/finally
  to restore max_size_bytes since AioHTTPTestCase reuses the app).

Phone-side — app/:

- RelayHttpClient.fetchMediaByPath(path, contentTypeHint): new
  method. URL built via okhttp3.HttpUrl.Builder so query encoding
  handles paths with slashes / spaces / unicode correctly. Error
  mapping (401/403/404/5xx → human messages suitable for the
  Attachment.errorMessage field).

- ChatHandler.onUnavailableMediaMarker → onMediaBarePathRequested.
  The "Unavailable" naming made sense in v1 when bare-path was
  always a terminal state. Now bare-path is the primary LLM format
  and fires a fetch, so the rename reflects the new semantics.
  Updated KDoc explains the upstream prompt_builder context.

- ChatViewModel: rename + rewrite method body. Now inserts a LOADING
  placeholder with Attachment.relayToken set to the absolute path
  (reusing relayToken as a generic inbound-fetch key — since
  secrets.token_urlsafe never produces a leading '/', the prefix
  disambiguates token-vs-path downstream). Applies cellular gate,
  then calls performFetchWith { relay.fetchMediaByPath(path) }.

- performFetch → performFetchWith: signature now takes a suspend
  () -> Result<FetchedMedia> fetch lambda so both paths (token and
  bare-path) share the same size-cap / cache / state-flip logic.
  manualFetchAttachment dispatches to token vs by-path via the
  same startsWith("/") heuristic.

## Security review (in ADR 14 addendum)

Adding /media/by-path widens what a paired phone can request by one
degree — it can now read any file in the allowed-roots whitelist
without host-local tool cooperation. This does NOT widen the trust
boundary because (1) the whitelist is identical to /media/register,
(2) /tmp on Linux is already world-readable to same-user processes,
(3) bearer auth still gates access to paired peers only, and (4)
realpath symlink-resolves before the whitelist check so symlink
escape is still blocked. Operators who want a tighter sandbox
should narrow RELAY_MEDIA_ALLOWED_ROOTS, not disable the route.

## Docs

- decisions.md ADR 14 addendum covering the bare-path endpoint and
  the security review above
- relay-server.md + user-docs/reference/relay-server.md route tables
- spec.md §6.2a updated (three routes now, bare-path flow described)
- user-docs/reference/configuration.md: bare-path as primary format
- CLAUDE.md Integration Points split into token/path + tool/LLM rows
- DEVLOG entry with full rationale

## Known gaps still filed

- Auto-fetch threshold slider enforcement (persisted, not wired)
- Session replay across relay restarts (in-memory registry, fix
  is phone-side persistent cache — future)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 17:35:30 -04:00
Bailey DixonandClaude Opus 4.6 5c0875d9ea feat(settings): Connection UX v2 — stale-state amber + reactive API key flag
Second pass of Connection section polish from the settings team, landed
in the same working tree as the inbound media v2 work. Kept separate
for history hygiene.

- AuthManager.apiKeyPresent: StateFlow<Boolean>
  Reactive flag updated by setApiKey / clearApiKey so Settings can
  render "API key: already set — leave blank to keep" without having
  to call the suspend getApiKey() from inside a composable. Seeded
  from persisted prefs on init.

- ConnectionViewModel.reconnectIfStale()
  No-op unless (paired AND disconnected AND has URL). Called from
  Settings screen entry and the "tap to reconnect" action on the
  Relay status row. Avoids duplicate connect calls that would
  interrupt an in-flight auth.

- SettingsScreen: stale-state amber handling
  New isRelayStale local = (Paired AND Disconnected). Relay status
  row renders "Stale — tap to reconnect" in amber instead of red
  "Disconnected" — the fix is a single reconnect, not a re-pair.
  LaunchedEffect(Unit) calls reconnectIfStale() on screen entry so
  90% of users never see the stale state at all.

- PairingWalkthroughDialog + ConnectionInfoSheet: UX polish wiring
  for the above (dialog state handoff, sheet tap targets).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 17:34:40 -04:00
Bailey DixonandClaude Opus 4.6 272a3c5c3e fix(chat): re-run media marker parser in session-reload path
The "session_end reload" pattern in ChatViewModel.onCompleteCb calls
client.getMessages() → handler.loadMessageHistory() at the end of
every /api/sessions/{id}/chat/stream turn, and loadMessageHistory
was wholesale-replacing _messages.value with ChatMessages built from
raw item.contentText without running the media marker parser.

Two consequences:
  1. Any FAILED/LOADING Attachment the streaming path injected for a
     MEDIA: marker was immediately wiped — the ⚠️ "Image unavailable"
     placeholder flickered for a frame and vanished.
  2. The raw "MEDIA:/tmp/foo.jpg" text became visible in the bubble
     because server-stored content still contains the marker (the
     streaming strip was phone-local only).

Fix: loadMessageHistory now scans each loaded assistant message's
content for media markers, strips matched lines, and queues callback
hits to fire *after* the wholesale _messages.value = loaded assignment
(so the ViewModel's mutateMessage lookups find the newly-loaded IDs).
dispatchedMediaMarkers is cleared at the same time since pre-reload
dedupe keys are meaningless against post-reload message IDs.

Extracted into a pure helper `extractMediaMarkersFromContent` + a
private sealed MediaMarkerHit so the reload path doesn't have to
share mutable buffer state with the streaming path.

Tested paths:
  - /v1/runs + /api/sessions/*/chat/stream both call loadMessageHistory
    via ChatViewModel's onCompleteCb (ChatViewModel.kt:485-487) and via
    the initial session-open load (ChatViewModel.kt:269-270).
  - Streaming path unchanged.

Known follow-up (not in this commit): upstream hermes-agent's
agent/prompt_builder.py instructs the LLM to "include MEDIA:/absolute/
path/to/file in your response" — so bare-path markers leak into chat
from the LLM's free-form text, not just from tools. The current
placeholder-based handling treats them as "unavailable" even when the
relay is running. Proper fix is a /media/by-path endpoint on the relay
that auth'd phones can fetch directly. Discussing with user before
scoping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 15:12:51 -04:00
Bailey DixonandClaude Opus 4.6 8f61262cf0 feat(chat): inbound media pipeline — relay MediaRegistry + phone fetcher + Discord-style rendering
Closes the gap where tool-produced files (screenshots today, video/audio/PDF
in the future) were leaking into chat as raw "MEDIA:/tmp/..." text instead
of rendering inline.

Root cause is upstream: APIServerAdapter.send() in hermes-agent's
gateway/platforms/api_server.py is an explicit no-op, and
_write_sse_chat_completion streams raw deltas without ever invoking
extract_media(). Upstream's extract_media() / send_document() pipeline
only fires for push platforms (Telegram, Feishu, WeChat). Our HTTP pull
adapter has no file-delivery path at all.

Fix: plugin-owned file-serving on the relay, opaque-token markers in chat
text, out-of-band bearer-auth'd HTTPS fetch for bytes. Zero LLM context
cost (token is ~25 chars; bytes never travel through the chat stream).

## Server (Python)

- plugin/relay/media.py — MediaRegistry (asyncio.Lock-guarded OrderedDict
  LRU, 24h TTL, 500-entry cap, 100 MB per-file cap), _MediaEntry,
  MediaRegistrationError. Path sandboxing: absolute, realpath-resolves
  under an allowed root (default tmpdir + HERMES_WORKSPACE +
  RELAY_MEDIA_ALLOWED_ROOTS), exists, regular file, under size cap.
- plugin/relay/client.py — stdlib urllib.request register_media() helper
  for in-process tool callers.
- plugin/relay/server.py — handle_media_register (loopback-only, mirrors
  /pairing/register trust model) + handle_media_get (Bearer auth via the
  existing SessionManager, web.FileResponse stream with Content-Type +
  Content-Disposition). Routes registered in create_app.
- plugin/relay/config.py — 4 new env vars: RELAY_MEDIA_MAX_SIZE_MB,
  RELAY_MEDIA_TTL_SECONDS, RELAY_MEDIA_LRU_CAP, RELAY_MEDIA_ALLOWED_ROOTS.
- plugin/tools/android_tool.py + plugin/android_tool.py — android_screenshot()
  calls register_media() after writing the temp file, emits
  MEDIA:hermes-relay://<token> on success. On any failure (relay down,
  timeout) falls back to bare MEDIA:<path> with a warning; the phone
  handles the bare form as an "unavailable" placeholder.
- plugin/tests/test_media_registry.py — 11 tests (happy path, TTL expiry,
  LRU eviction + reorder on get, relative/nonexistent/directory path
  rejection, allowed-roots whitelist, symlink-escape rejection, oversize,
  empty content_type).
- plugin/tests/test_relay_media_routes.py — 8 tests (/media/register
  loopback gate, happy path, validation 400, bad JSON; /media/{token}
  401 without/with bad bearer, 200 streams bytes, 404 expired, 404 unknown).

## Phone (Kotlin)

- network/RelayHttpClient.kt — OkHttp GET /media/{token}, Bearer auth,
  ws→http URL rewrite, Content-Disposition filename parse.
- data/ChatMessage.kt — AttachmentState (LOADING/LOADED/FAILED),
  AttachmentRenderMode (IMAGE/VIDEO/AUDIO/PDF/TEXT/GENERIC), extended
  Attachment with state/errorMessage/relayToken/cachedUri. Outbound
  attachments default to LOADED (backward-compat).
  (NB: also fixes a Kotlin block-comment nesting bug — the KDoc used
  "text/*" as a MIME wildcard, which Kotlin's nested-comment lexer treated
  as an unclosed /* opening and swallowed the rest of the file. Rephrased.)
- data/MediaSettings.kt — DataStore-backed: maxInboundSizeMb (25),
  autoFetchThresholdMb (2, persisted-not-enforced placeholder),
  autoFetchOnCellular (off), cachedMediaCapMb (200).
- util/MediaCacheWriter.kt — LRU-capped cache at cacheDir/hermes-media/,
  mtime eviction, MIME→extension map, returns FileProvider content:// URIs.
- network/handlers/ChatHandler.kt — mediaRelayRegex + mediaBarePathRegex,
  scanForMediaMarkers called unconditionally from onTextDelta (not gated
  on parseToolAnnotations), dispatchedMediaMarkers dedupe set, new
  onMediaAttachmentRequested + onUnavailableMediaMarker var callbacks,
  mutateMessage helper exposed so ChatViewModel can flip attachment state.
- viewmodel/ChatViewModel.kt — initializeMedia wiring, LOADING placeholder
  insertion on marker dispatch, fetch via RelayHttpClient, size cap check
  post-download, cache via MediaCacheWriter, state flip to LOADED/FAILED.
  Cellular gate encoded as LOADING + errorMessage="Tap to download" (no
  new enum value needed); manualFetchAttachment() retries ignoring the gate.
- viewmodel/ConnectionViewModel.kt — owns media singletons, shared
  OkHttpClient, cached-cap mirror loop so the writer's cap lambda is
  synchronous.
- ui/RelayApp.kt — initializeMedia wired inside the existing
  LaunchedEffect(apiClient) block.
- ui/components/MessageBubble.kt — attachments loop dispatches through
  InboundAttachmentCard regardless of direction (no separate outbound
  render path).
- ui/components/InboundAttachmentCard.kt — single Compose component
  dispatched on (state × renderMode). IMAGE decodes from cachedUri via
  BitmapFactory + asImageBitmap (matches existing outbound image path,
  no Coil/Glide added). VIDEO/AUDIO/PDF/TEXT/GENERIC render as tap-to-open
  cards firing ACTION_VIEW with FLAG_GRANT_READ_URI_PERMISSION.
- ui/screens/ChatScreen.kt — empty-bubble skip respects
  attachments.isNotEmpty, wires manualFetchAttachment to retry + manual-fetch.
- ui/screens/SettingsScreen.kt — new InboundMediaSection between Chat and
  Appearance (max size, auto-fetch threshold, cellular toggle, cached cap,
  clear button). Coexists with the Connection UX landed in the previous
  commit: both features share this file and this commit is where their
  shared edits land.
- res/xml/file_provider_paths.xml + AndroidManifest.xml — FileProvider
  declaration with authority ${applicationId}.fileprovider.

## Docs

- DEVLOG.md — full session entry with root cause, design, files, known
  gaps, test plan.
- docs/decisions.md — ADR 14 on plugin-owned media endpoint, trust model,
  resource bounds, alternatives rejected.
- docs/spec.md — new §6.2a Inbound Media covering wire contract, server
  routes, phone parse/fetch/render flow, known gaps.
- docs/relay-server.md + user-docs/reference/relay-server.md — new
  RELAY_MEDIA_* env vars, new /media/register + /media/{token} routes.
- user-docs/reference/configuration.md — new "Inbound Media Settings"
  section with honest notes on the two known gaps (auto-fetch threshold
  not enforced, session replay breaks across relay restarts).
- CLAUDE.md — Key Files + Integration Points updated.

## Known gaps (filed as DEVLOG follow-ups)

- Auto-fetch threshold slider is persisted but not enforced today —
  only the cellular toggle + the hard max cap actually gate fetches.
- Session replay breaks across relay restarts. MediaRegistry is in-memory;
  phone-side persistent cache (indexed by token or content hash) is the
  right layer for durability and is out of scope for this pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 15:00:30 -04:00
Bailey DixonandClaude Opus 4.6 11957788b7 feat(settings): Connection row bottom sheets + manual code entry
Adds the three core components of the reworked Connection section that
the settings team landed this session:

- ConnectionInfoSheet — per-row (API / Relay / Session) bottom sheets
  surfacing current state and next-step affordances.
- PairingWalkthroughDialog — manual 6-char pairing code entry with
  soft-keyboard auto-pop, Go-key dispatch, and red-state clearing on
  edit after an error.
- AuthManager.applyServerIssuedCodeAndReset — atomically applies a
  user-entered code AND wipes any stale session token, avoiding the
  race where authenticate() would otherwise reuse the old token
  instead of consuming the new code.

SettingsScreen wiring that imports and renders these components lands
in the follow-up commit alongside the inbound media section — both
features share that file and are committed together to keep the tree
building at every commit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 14:59:21 -04:00
Bailey DixonandClaude Opus 4.6 5edaa751da feat(install): canonical Hermes skill deployment via external_dirs
Aligns the install/deploy flow with Hermes's canonical skill distribution
patterns (per ~/.hermes/hermes-agent/website/docs/user-guide/features/skills.md).
Before this commit our install.sh was pre-skill-system: a throwaway `cp -r`
into ~/.hermes/plugins/ with no pip install, no skill install, no shell
shim, and next-steps pointing at `hermes pair` (which is blocked on an
upstream CLI argparser gap).

## Repo layout — match canonical Hermes skill layout

- `skills/hermes-relay-pair/` → `skills/devops/hermes-relay-pair/`
  (category subdirectory matches metadata.hermes.category and the pattern
   documented in the Hermes skill system guide)
- **Deleted** `skills/hermes-pairing-qr/` (the pre-plugin bash-script skill
  that was already marked deprecated — superseded by plugin/pair.py +
  /hermes-relay-pair skill + hermes-pair shim)
- **Deleted** `plugin/skill.md` (lowercase-s flat-file artifact from before
  the skill system existed)

## install.sh — full rewrite to canonical flow

  1. Clone to ~/.hermes/hermes-relay/ (override with HERMES_RELAY_HOME).
     Idempotent — existing git clones get `git pull --ff-only`.
  2. `pip install -e $RELAY_HOME/` into the hermes-agent venv (editable,
     so `git pull` is the update mechanism — no reinstall needed).
  3. Symlink ~/.hermes/plugins/hermes-relay → $RELAY_HOME/plugin. Removes
     any stale hermes-android-named symlinks to prevent double-registration.
  4. Register $RELAY_HOME/skills in ~/.hermes/config.yaml under
     `skills.external_dirs` via an idempotent YAML edit (pyyaml). Saves
     a .bak next to the original before writing because yaml.safe_dump
     round-trips lose inline comments — users with hand-edited configs
     can restore. This is the path Hermes documents for dev-repo skills:
     external_dirs are scanned fresh on each invocation, so `git pull`
     instantly updates all skills in the clone.
  5. Install ~/.local/bin/hermes-pair shell shim that execs
     `<venv-python> -m plugin.pair "$@"`. Override the venv python path
     via $HERMES_VENV_PY. Since `hermes pair` (with space) is blocked on
     an upstream gap, this is the canonical shell-side entry point.
  6. Print next steps — /hermes-relay-pair (slash command in any Hermes
     chat), hermes-pair (shell), or python -m plugin.pair (direct). Never
     `hermes pair` (doesn't work on vanilla Hermes v0.8.0).

## docs/upstream-contributions.md

Added two new items documenting the upstream gaps I discovered during
this session:

  - **Item 3** — third-party plugin CLI commands aren't wired into the
    top-level argparser. Concrete ~10-line patch to `main.py:5236` that
    would unblock `hermes pair`, `hermes relay start`, and every other
    plugin that uses `ctx.register_cli_command()`.
  - **Item 4** — skill discovery doesn't follow symlinks, forcing
    external_dirs usage for dev-repo skills. Worth upstream if someone
    wants to contribute a fix.

## Kotlin + docs sync (via parallel subagents)

- `SettingsScreen.kt:214` — hint text in the "Pair with your server" card
  no longer references `hermes pair`; now points at `/hermes-relay-pair`
  and `hermes-pair` as the two working entry points.
- README, AGENTS.md, CLAUDE.md, docs/{spec,decisions,security,relay-server}.md,
  user-docs/{guide,reference,architecture}/**, user-docs home-page
  InstallSection.vue — all pair-command references swept to the new
  canonical flow. New ADR-13 documents the external_dirs distribution
  decision.
- DEVLOG — new 2026-04-11 entry at the top of the day.

Verified locally:
  bash -n install.sh                        → syntax clean
  gradlew :app:compileDebugKotlin           → BUILD SUCCESSFUL
  pip install -e .                          → hermes-relay 0.5.0
  python -c "import plugin.pair" from /c/   → imports cleanly

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 13:38:59 -04:00
Bailey DixonandClaude Opus 4.6 e9c3f824ba feat: /hermes-relay-pair skill + rename plugin hermes-android → hermes-relay
Three parallel workstreams landed together.

## 1. Skill authoring — new /hermes-relay-pair slash command

skills/hermes-relay-pair/SKILL.md (98 lines, agentskills.io-compatible):
- Proper YAML frontmatter (name, description, version, author, license,
  platforms: [linux, macos], metadata.hermes tags/category/homepage)
- Body sections: When to Use / Prerequisites / Procedure / Pitfalls /
  Verification — terse imperatives suitable for the agent's context window
- Tells the agent to run `python -m plugin.pair`, explains the venv path
  trap, walks through host-side verification (curl /health, clients
  count), warns about code expiration + QR terminal rendering gotchas
- Once installed to ~/.hermes/skills/, auto-registers as the
  /hermes-relay-pair slash command in every chat session + messaging
  platform.

## 2. Rename hermes-android → hermes-relay (user-facing, not Python)

The repo was rebranded on 2026-04-08 but the Python *package* / plugin
distribution name still said `hermes-android`. It's visible in
`hermes plugins list`, pypi-style imports, and a dozen user-docs /
README / install-snippet references. Renamed everywhere it's a package
name or user-visible string; Python module path `plugin` stays exactly
as-is (imports unchanged).

- pyproject.toml: name "hermes-android" → "hermes-relay",
  version "0.4.0" → "0.5.0", description rewritten
- plugin/plugin.yaml: name + version aligned with pyproject
- plugin/__init__.py: docstring updated
- install.sh: PLUGIN_NAME target dir is now ~/.hermes/plugins/hermes-relay
- README.md, AGENTS.md, docs/{security,relay-server,upstream-contributions}.md,
  user-docs/guide/getting-started.md, user-docs/reference/{relay-server,configuration}.md:
  install snippets + package references swept
- plugin/{android_tool,tools/android_tool}.py: package name strings updated
- relay_server/SKILL.md, skills/hermes-pairing-qr/*: deprecation notices
  updated to reference the new name
- Historical DEVLOG entries, plan.md build plan, and Python import
  paths left untouched on purpose (history and correctness)

## 3. user-docs additive pass for the slash command

user-docs/guide/getting-started.md and README.md now mention
/hermes-relay-pair as the primary "from a Hermes chat session" path
alongside the `hermes pair` CLI. Narrow additive edits only — no
section rewrites.

## Upstream gap discovered

Hermes v0.8.0 has `PluginContext.register_cli_command()` for third-party
plugins but main.py only wires `plugins.memory.discover_plugin_cli_commands()`
into the top-level argparser — the generic `_cli_commands` dict is
populated correctly but never consulted. Result: our `hermes pair` and
`hermes relay` sub-commands register cleanly but aren't callable from
the CLI until an upstream fix lands. The `register_cli_command` calls
in plugin/__init__.py are left in place so they'll start working the
moment main.py is patched.

Practical workaround: use /hermes-relay-pair (this skill) or a shell
shim at ~/.local/bin/hermes-pair that execs `python -m plugin.pair`
(deploy step — not in this commit).

Verified locally:
  pip install -e .           → hermes-relay 0.5.0
  python -c "import plugin.pair"
  python -m plugin.pair --help

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 13:17:28 -04:00
Bailey DixonandClaude Opus 4.6 d4ca65099a fix(plugin): move pyproject.toml to repo root so plugin.* imports work
The previous pyproject.toml lived inside `plugin/` (flat layout). With
setuptools auto-discovery, that caused pip install to expose `relay/`,
`tests/`, `tools/` as TOP-LEVEL packages instead of `plugin.relay`,
`plugin.tests`, `plugin.tools`, and `plugin/__init__.py` + `pair.py` +
`cli.py` were never registered as importable modules at all.

End result: after `pip install -e plugin/`, you couldn't do
`python -m plugin.pair` from any directory (not even the repo root),
and `hermes pair` as a CLI sub-command never worked because the plugin
loader couldn't import `plugin` itself.

## The fix

1. Move pyproject.toml from `plugin/pyproject.toml` to repo root.
2. Delete `plugin/setup.py` (redundant with pyproject.toml on modern
   setuptools).
3. Add explicit `[tool.setuptools.packages.find]` that includes
   `plugin*` and `relay_server*`, with the excludes needed to keep
   `app/`, `docs/`, `user-docs/`, `scripts/`, `plugin.tests`,
   `plugin.skills` out of the installed wheel.
4. Add `[tool.setuptools.package-data]` so `plugin/skill.md`,
   `plugin/plugin.yaml`, and the android SKILL files ship with the
   install.

After `pip install -e .` from the repo root:

  python -c "import plugin.pair, plugin.relay.server"  # works from anywhere
  python -m plugin.pair --help                          # works from anywhere
  hermes pair --help                                    # plugin CLI registers

Verified locally with a fresh editable install and a cross-directory
import + module-invocation smoke test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:57:17 -04:00
Bailey DixonandClaude Opus 4.6 32046be16e feat(settings): unified Connection section — primary QR scan + collapsibles
Restructures the Settings pairing surface. Previously three top-level
cards ("API Server", "Relay Server", "Pairing") with the QR scanner
buried inside the API Server card next to Save & Test, and a prominent
phone-generated pairing code display that no longer matters for initial
setup in the new QR-driven flow.

Now:

- **Pair with your server** — primary card, always visible. Full-width
  "Scan Pairing QR" button + unified 3-line status (API Server / Relay /
  Session) using ConnectionStatusRow with animated pulse indicators.
  Shows "Clear Session" and "Re-pair" secondary actions only when
  AuthState is Paired.
- **Manual configuration** — collapsible SettingsExpandableCard.
  Contains all the API URL / API Key / Relay URL / Connect + Disconnect
  / Insecure mode fields for power users and troubleshooting. Seeded
  expanded when not-yet-paired, collapsed when the user is already
  reachable + paired. rememberSaveable preserves user intent across
  recompositions so manual collapse doesn't get overridden when the
  connection drops.
- **Bridge pairing code** — collapsible, gated behind relayEnabled,
  collapsed by default. Shows the locally-generated pairing code with
  copy + regenerate. Labeled explicitly "For Phase 3 bridge feature —
  not used for initial pairing" so users don't confuse it with the
  QR-scanned terminal/chat credentials.

New helper: `SettingsExpandableCard` — private composable at file scope,
reuses the same gradientBorder + surfaceVariant styling as the other
Settings cards. Tap target scoped to the header Row so interacting with
fields inside the expanded body doesn't toggle collapse.

Expand-state default reads `apiReachable` and `authState` at first
composition. On cold start both come back as their `initial` values
(false / Unpaired), so the card opens by default — conservative fail-
open so new users see the fields instead of a collapsed mystery.

## Docs

- `user-docs/guide/getting-started.md` — Manual Pairing section now
  describes both onboarding and Settings → Connection paths.
- `user-docs/reference/configuration.md` — "Onboarding Settings"
  renamed to "Connection Settings" with the three-card layout.
- `CLAUDE.md` — SettingsScreen.kt entry expanded.
- `DEVLOG.md` — 2026-04-11 entry above the QR pairing flow entry.

Verified: gradlew :app:assembleDebug — BUILD SUCCESSFUL.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:48:20 -04:00
Bailey DixonandClaude Opus 4.6 8a65ed19f1 docs: additional pairing-flow updates (security + architecture)
Follow-up from the QR pairing flow commit — landed separately because the
docs subagent was still running while the primary commit went out.

- docs/security.md — relay auth no longer describes the broken
  "phone-generated code" model; now reflects the QR-driven flow.
- user-docs/architecture/{index,security,decisions}.md — alphabet
  sentence updated, ADR-7 rewritten, relay auth steps aligned with the
  new pairing model.
- CLAUDE.md — repo layout clarification (plugin/relay canonical,
  relay_server/ shim paths).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:34:20 -04:00
Bailey DixonandClaude Opus 4.6 6fc366b6af feat(pairing): QR carries relay credentials — one scan, both channels
Reworks the relay pairing flow so it matches the API-side flow we already
had: the operator runs `hermes pair` on the host (the trust anchor), the
relay pre-registers a fresh pairing code via a new loopback-only endpoint,
and the QR payload carries both API credentials AND relay credentials.
The phone scans once and is fully configured for chat + terminal.

## Wire format

HermesPairingPayload now has an optional nested `relay` object:

    {
      "hermes": 1,
      "host": "<api-ip>", "port": 8642, "key": "<bearer>", "tls": false,
      "relay": { "url": "ws://<ip>:8767", "code": "ABCD12" }
    }

Old API-only QRs still parse cleanly (kotlinx.serialization
`ignoreUnknownKeys = true`, nullable field). The `relay` block is only
present when `hermes pair` found a running relay and pre-registration
succeeded.

## Server side

- `POST /pairing/register` on the relay: accepts `{"code":"…"}`, calls
  `PairingManager.register_code()`, returns `{"ok":true}` on success.
  Gated to `127.0.0.1` / `::1` — a remote LAN attacker cannot inject
  pairing codes; only host-shell processes can.
- `plugin/relay/config.py` — `PAIRING_ALPHABET` broadened from the
  no-ambiguous-chars 32-char set to the full 36-char `A-Z0-9` set so it
  matches `AuthManager.PAIRING_CODE_CHARS` on the app side. The earlier
  restriction silently rejected valid codes containing `0/O/1/I`.

## `hermes pair` changes

- Probes `http://127.0.0.1:<relay_port>/health` on startup.
- If the relay is up: mints a 6-char code, POSTs `/pairing/register`,
  embeds `relay.{url,code}` in the QR payload.
- If not: emits an `[info]` line pointing at `hermes relay start` and
  renders an API-only QR (backward compat).
- New `--no-relay` flag to force API-only even when a relay is running.
- Text block now has a "Relay (terminal + bridge)" section alongside
  "Server" so manual entry is possible when QR scanning fails.
- Warning is now gated to "any credential present" rather than just
  "API key present" — both types trigger the "don't share screenshots"
  notice.

## App side

- `HermesPairingPayload` gains optional `RelayPairing(url, code)`.
- `AuthManager.applyServerIssuedCode(code)` stores a one-shot override
  that trumps the locally-generated code on the next `authenticate()`
  call. Cleared automatically on `auth.ok` so subsequent reconnects use
  the long-lived session token. Local generation stays as the fallback
  and as the Phase-3 bridge direction trust anchor (phone issues code,
  host approves).
- `OnboardingScreen` — `onComplete` signature grew a `relayPairingCode`
  param; `ConnectPage` gets an `onRelayPairingDetected` callback that
  propagates scanned relay fields up to the parent. Applied to
  AuthManager in `RelayApp.kt` before nav transition.
- `SettingsScreen` — scanner handler auto-configures the relay URL,
  flips insecure mode on for `ws://` URLs, and calls
  `applyServerIssuedCode()`. Toast message distinguishes "API + relay
  configured" from "API only".
- `ConnectionManager.connect()` — URL normalizer appends `/ws` if the
  path is empty, so a bare `ws://host:port` still upgrades cleanly.
  Pairs with the server-side `/` → `/ws` alias added earlier.

## Docs

README, docs/{spec,decisions,relay-server}.md,
user-docs/{guide/getting-started,reference/relay-server}.md, and DEVLOG
all updated to describe the new single-scan flow + `/pairing/register`
endpoint + `--no-relay` flag + alphabet change.

Verified locally:
  gradlew :app:assembleDebug — BUILD SUCCESSFUL
  python -m py_compile plugin/relay/server.py plugin/pair.py
  python -m plugin.pair --help  / python -m plugin.relay --help
  adb install -r app-debug.apk — Success

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:31:41 -04:00
Bailey DixonandClaude Opus 4.6 118a2058a8 feat(relay): PTY terminal backend + consolidate relay_server into plugin/relay
Two changes packaged together since the PTY backend landed at the new
plugin/relay/channels/terminal.py location:

1. **PTY terminal backend** — rewrites the terminal channel stub into a
   real pty.openpty() + fork + TIOCSCTTY handler with non-blocking master
   fd plugged into asyncio.loop.add_reader(). Output batched on ~16ms
   frames (or 4 KiB overflow). TIOCSWINSZ resize, graceful SIGHUP →
   WNOHANG reap → SIGKILL teardown. Unix-only; import-guarded so the
   relay still starts on Windows with a clean "not supported" error on
   attach. Adds terminal_shell config field (RELAY_TERMINAL_SHELL env var).
   Per-client session cap of 4.

2. **Plugin consolidation (Phase 2 plan step)** — relay_server/ now lives
   at plugin/relay/ as the canonical location. relay_server/ is kept as a
   thin compat shim so `python -m relay_server` and existing docs keep
   working. Adds `hermes relay start` CLI sub-command wired through the
   plugin CLI registration system. Bumps plugin version 0.3.0 → 0.4.0,
   adds pyyaml to install_requires, libtmux to optional extras.

Deferred (still in the plan, not this commit): libtmux session
persistence, hermes relay status/sessions/kill sub-commands, bridge
channel protocol rewrite (still HTTP↔WS in plugin/android_relay.py on
its own port).

Verified locally:
  python -m plugin.relay --help
  python -m relay_server --help
  python -m py_compile plugin/relay/**/*.py
  gradlew :app:compileDebugKotlin (UP-TO-DATE)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:58:22 -04:00
Bailey DixonandClaude Opus 4.6 1ea2d1a9f8 feat(phase2): terminal tab — xterm.js WebView + sticky-modifier toolbar
Replaces the Terminal tab placeholder with a real Compose WebView hosting
xterm.js 5.5.0 (bundled at app/src/main/assets/terminal/, no CDN at runtime).
TerminalViewModel registers with the shared ChannelMultiplexer, routes
terminal.output envelopes into the WebView via base64-encoded
window.writeTerminal calls, and handles sticky CTRL/ALT translation. The
ExtraKeysToolbar provides ESC/TAB/CTRL/ALT/arrows with haptic feedback.

Paired with the Python PTY backend + plugin consolidation in the follow-up
commit. Build verified via gradlew :app:assembleDebug. Not yet exercised
on a real device + real relay.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:56:35 -04:00
Bailey Dixon 94d54d4481 docs: add Star History chart 2026-04-10 18:13:51 -04:00
Bailey Dixon 27fcb52b34 fix: add native debug symbols for Play Console 2026-04-10 17:56:44 -04:00
Bailey DixonandClaude Opus 4.6 b91aa4f6a4 docs: v0.1.0 release polish — sideload APK + friendly issue CTAs
- Add "Sideload APK" path across the docs site and README, linking to
  /releases/latest so it stays current across future releases. Full
  install + SHA256 verification (bash + PowerShell) in the getting
  started guide; condensed 4-step version in README.
- Add "Found a bug? Let us know!" CTAs with warm indie-dev tone on the
  docs home, README, RELEASE_NOTES template, and the theme's doc-footer
  bar. No more corporate "Report Issue" language.
- Fix stale upstream repo links: VitePress navbar, social links, hero
  "View on GitHub" button, and doc-footer CTA were all pointing at
  NousResearch/hermes-agent instead of Codename-11/hermes-relay. Also
  enabled Issues + Discussions on the repo (neither was on), which is
  why the old links would have 404'd anyway.
- Update live v0.1.0 GitHub Release body via gh release edit with a
  "Which file do I download?" section explaining app-release.apk vs
  .aab. Propagated the same section into RELEASE_NOTES.md so v0.1.1+
  inherit it automatically.
- RELEASE.md: note the one-time v0.1.0 release-body retrofit and add
  gh release edit fallback for future releases.
- DEVLOG.md: entry for v0.1.0 Play Store Internal testing + GitHub
  Release cut (keystore generation, CI secrets, tag/push, fingerprint
  verification of the CI-built AAB).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 17:06:31 -04:00
Bailey DixonandClaude Opus 4.6 78777cc08e feat(docs): add OG image + social preview meta tags
Crawlers for Twitter/X, Facebook/Messenger, Slack, Discord, and LinkedIn
need absolute URLs in meta tags — VitePress's base-prefix rewriting does
not apply to meta content attributes. Add a complete set:

- og:image (+ width/height/type/alt/secure_url) pointing at a new
  og-image.png in public/ (1024x500, reused from the Play Store feature
  graphic for brand consistency)
- og:url, og:site_name, canonical link
- twitter:card upgraded from "summary" to "summary_large_image" so the
  wide hero graphic renders full-width

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 07:26:07 -04:00
Bailey DixonandClaude Opus 4.6 ceca250124 feat(docs): add copy buttons to install/pair commands on home
VitePress only attaches its theme copy button to markdown code fences
processed through its pipeline, so the hand-written <pre><code> blocks
in InstallSection.vue had no copy affordance. Add a tiny dependency-free
copy button per block backed by navigator.clipboard with a 2s "Copied!"
state flash.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 22:24:33 -04:00
270 changed files with 60571 additions and 7819 deletions
+40
View File
@@ -0,0 +1,40 @@
#!/bin/bash
# @subframe-version 0.15.1-beta
# @subframe-managed
#
# SubFrame pre-commit hook
# Auto-updates STRUCTURE.json when JS files in src/ are committed.
#
# Check if any JS files in src/ are staged
STAGED_JS=$(git diff --cached --name-only --diff-filter=ACMRD | grep -E '^src/.*\.(js|ts|tsx|jsx)$' || true)
# Also check for deleted source files
DELETED_JS=$(git diff --cached --name-only --diff-filter=D | grep -E '^src/.*\.(js|ts|tsx|jsx)$' || true)
if [ -z "$STAGED_JS" ] && [ -z "$DELETED_JS" ]; then
exit 0
fi
# Only update if .subframe/STRUCTURE.json exists (SubFrame project)
if [ ! -f ".subframe/STRUCTURE.json" ]; then
exit 0
fi
# Only update if the updater script exists
UPDATER=".githooks/update-structure.js"
if [ ! -f "$UPDATER" ]; then
exit 0
fi
echo "[SubFrame] Source files changed, updating .subframe/STRUCTURE.json..."
# Run the updater with staged/deleted file lists as env vars
STAGED_FILES="$STAGED_JS" DELETED_FILES="$DELETED_JS" node "$UPDATER"
# Stage the updated .subframe/STRUCTURE.json
git add .subframe/STRUCTURE.json
echo "[SubFrame] .subframe/STRUCTURE.json updated and staged."
exit 0
+22
View File
@@ -0,0 +1,22 @@
#!/bin/bash
# @subframe-version 0.15.1-beta
# @subframe-managed
# SubFrame pre-push hook
# Triggers pipeline workflows configured with "on: { push: true }"
# To bypass: git push --no-verify
SUBFRAME_DIR=".subframe"
PIPELINES_DIR="$SUBFRAME_DIR/pipelines"
TRIGGER_FILE="$PIPELINES_DIR/.pre-push-trigger"
# Only trigger if SubFrame is initialized
if [ ! -d "$SUBFRAME_DIR" ]; then
exit 0
fi
# Write trigger file for SubFrame to detect
mkdir -p "$PIPELINES_DIR"
echo "{\"trigger\": \"pre-push\", \"timestamp\": \"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"}" > "$TRIGGER_FILE"
# Don't block the push — pipeline runs async via SubFrame UI
exit 0
+127
View File
@@ -0,0 +1,127 @@
#!/usr/bin/env node
// @subframe-version 0.15.1-beta
// @subframe-managed
/**
* SubFrame STRUCTURE.json Updater
* Called by .githooks/pre-commit when source files in src/ are staged.
* Reads STAGED_FILES and DELETED_FILES from environment variables.
*/
const fs = require('fs');
const path = require('path');
const ROOT = process.cwd();
const STRUCTURE_FILE = path.join(ROOT, '.subframe', 'STRUCTURE.json');
const SRC_DIR = path.join(ROOT, 'src');
// Strip any JS/TS extension for module key
function stripExt(p) {
return p.replace(/\.(js|ts|tsx|jsx)$/, '');
}
// Load existing STRUCTURE.json
let structure;
try {
structure = JSON.parse(fs.readFileSync(STRUCTURE_FILE, 'utf-8'));
} catch (e) {
process.exit(0);
}
if (!structure.modules) {
structure.modules = {};
}
const files = (process.env.STAGED_FILES || '').split('\n').filter(Boolean);
const deleted = (process.env.DELETED_FILES || '').split('\n').filter(Boolean);
// Remove deleted modules
for (const file of deleted) {
const key = stripExt(path.relative(SRC_DIR, path.join(ROOT, file)))
.replace(/\\/g, '/');
if (structure.modules[key]) {
delete structure.modules[key];
}
}
// Parse each staged file
for (const file of files) {
const fullPath = path.join(ROOT, file);
if (!fs.existsSync(fullPath)) continue;
let content;
try {
content = fs.readFileSync(fullPath, 'utf-8');
} catch (e) {
continue;
}
const key = stripExt(path.relative(SRC_DIR, fullPath))
.replace(/\\/g, '/');
// Extract description from top JSDoc comment
let description = '';
const docMatch = content.match(/^\/\*\*\s*\n\s*\*\s*([^\n]+)/);
if (docMatch) description = docMatch[1].trim();
// Extract exports — CJS (module.exports) and ESM (export { ... }, export function)
const xports = [];
const cjsMatch = content.match(/module\.exports\s*=\s*\{([^}]+)\}/);
if (cjsMatch) {
cjsMatch[1].split(',').forEach(function(s) {
const name = s.trim().split(':')[0].trim();
if (name && !name.startsWith('//')) xports.push(name);
});
}
// ESM named exports: export { foo, bar } or export function foo
const esmExportRe = /^export\s+(?:function|const|let|class|async\s+function)\s+(\w+)/gm;
let em;
while ((em = esmExportRe.exec(content)) !== null) {
if (!xports.includes(em[1])) xports.push(em[1]);
}
// Extract dependencies — CJS require() and ESM import
const deps = [];
const reqRe = /require\s*\(\s*['"]([^'"]+)['"]\s*\)/g;
let m;
while ((m = reqRe.exec(content)) !== null) {
const dep = m[1];
if (dep.startsWith('./') || dep.startsWith('../')) {
deps.push(stripExt(dep.replace(/^\.+\//, '')));
} else {
deps.push(dep);
}
}
const importRe = /import\s+.*?from\s+['"]([^'"]+)['"]/g;
while ((m = importRe.exec(content)) !== null) {
const dep = m[1];
if (dep.startsWith('./') || dep.startsWith('../')) {
deps.push(stripExt(dep.replace(/^\.+\//, '')));
} else {
deps.push(dep);
}
}
// Extract function names with line numbers
const functions = {};
const fnRe = /^(?:export\s+)?(?:async\s+)?function\s+(\w+)\s*\(/gm;
while ((m = fnRe.exec(content)) !== null) {
const lineNum = content.substring(0, m.index).split('\n').length;
functions[m[1]] = { line: lineNum };
}
const existing = structure.modules[key] || {};
structure.modules[key] = {
file: file,
description: description || existing.description || '',
exports: xports,
depends: deps.filter(function(v, i, a) { return a.indexOf(v) === i; }),
functions: Object.keys(functions).length > 0 ? functions : (existing.functions || {})
};
}
// Update timestamp and save
structure.lastUpdated = new Date().toISOString().split('T')[0];
if (structure._frame_metadata) {
structure._frame_metadata.lastUpdated = structure.lastUpdated;
}
fs.writeFileSync(STRUCTURE_FILE, JSON.stringify(structure, null, 2) + '\n');
+14 -4
View File
@@ -77,7 +77,10 @@ jobs:
uses: actions/upload-artifact@v7
with:
name: debug-apk
path: app/build/outputs/apk/debug/*.apk
# Product flavors (googlePlay, sideload) nest APKs under
# app/build/outputs/apk/<flavor>/debug/ — the `*` matches both.
path: app/build/outputs/apk/*/debug/*.apk
if-no-files-found: error
retention-days: 14
# ──────────────────────────────────────────────
@@ -130,9 +133,16 @@ jobs:
- name: Install dependencies
run: pip install -r relay_server/requirements.txt
- name: Syntax check
run: python -m py_compile relay_server/relay.py
- name: Syntax check (plugin.relay — canonical location)
run: |
python -m py_compile plugin/relay/server.py
python -m py_compile plugin/relay/channels/terminal.py
python -m py_compile plugin/relay/channels/chat.py
python -m py_compile plugin/relay/channels/bridge.py
- name: Syntax check (relay_server shim)
run: python -m py_compile relay_server/__init__.py relay_server/__main__.py
# TODO: Add pytest step when relay tests exist
# - name: Run tests
# run: pytest relay_server/tests/
# run: pytest plugin/relay/tests/
+32 -4
View File
@@ -94,12 +94,32 @@ jobs:
HERMES_KEYSTORE_PASSWORD: ${{ secrets.HERMES_KEYSTORE_PASSWORD }}
HERMES_KEY_ALIAS: ${{ secrets.HERMES_KEY_ALIAS }}
HERMES_KEY_PASSWORD: ${{ secrets.HERMES_KEY_PASSWORD }}
# `assembleRelease` and `bundleRelease` are flavor-wide task aliases
# (added by `flavorDimensions += "track"` in app/build.gradle.kts), so
# this one line builds ALL four artifacts at once. Filenames come from
# `archivesName` (set in app/build.gradle.kts) which injects the app
# version, so `<version>` below is `libs.versions.appVersionName`:
# app/build/outputs/apk/googlePlay/release/hermes-relay-<version>-googlePlay-release.apk
# app/build/outputs/apk/sideload/release/hermes-relay-<version>-sideload-release.apk
# app/build/outputs/bundle/googlePlayRelease/hermes-relay-<version>-googlePlay-release.aab
# app/build/outputs/bundle/sideloadRelease/hermes-relay-<version>-sideload-release.aab
run: ./gradlew bundleRelease assembleRelease
- name: List produced artifacts (debug aid)
run: |
echo "=== APK outputs ==="
find app/build/outputs/apk -name '*.apk' -print 2>/dev/null || true
echo "=== AAB outputs ==="
find app/build/outputs/bundle -name '*.aab' -print 2>/dev/null || true
- name: Generate checksums
# Flavor dimension adds an extra path segment to the AGP output layout.
# APKs live under `apk/<flavor>/release/`, AABs under `bundle/<flavor>Release/`
# (note the concatenated camelCase — AGP path quirk, documented but
# different between APK and AAB). The globs below match both flavors.
run: |
cd app/build/outputs
sha256sum apk/release/*.apk bundle/release/*.aab > SHA256SUMS.txt
sha256sum apk/*/release/*.apk bundle/*Release/*.aab > SHA256SUMS.txt
cat SHA256SUMS.txt
- name: Create GitHub Release
@@ -108,9 +128,16 @@ jobs:
name: v${{ needs.validate.outputs.version }}
body_path: RELEASE_NOTES.md
prerelease: ${{ contains(needs.validate.outputs.version, '-') }}
# Attach all four flavored artifacts — users sideload the
# `hermes-relay-<version>-sideload-release.apk` for the full
# Phase 3 / Tier 3/4/6 feature set; the
# `hermes-relay-<version>-googlePlay-release.aab` is what gets
# uploaded to Play Console. APK twin of the googlePlay flavor
# and AAB twin of the sideload flavor are included for parity
# (useful for diff tooling, not primary downloads).
files: |
app/build/outputs/apk/release/*.apk
app/build/outputs/bundle/release/*.aab
app/build/outputs/apk/*/release/*.apk
app/build/outputs/bundle/*Release/*.aab
app/build/outputs/SHA256SUMS.txt
- name: Release summary
@@ -127,5 +154,6 @@ jobs:
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "### Artifacts" >> "$GITHUB_STEP_SUMMARY"
echo '```' >> "$GITHUB_STEP_SUMMARY"
ls -la app/build/outputs/apk/release/ app/build/outputs/bundle/release/ >> "$GITHUB_STEP_SUMMARY"
find app/build/outputs/apk -name '*.apk' -exec ls -la {} + >> "$GITHUB_STEP_SUMMARY" 2>/dev/null || true
find app/build/outputs/bundle -name '*.aab' -exec ls -la {} + >> "$GITHUB_STEP_SUMMARY" 2>/dev/null || true
echo '```' >> "$GITHUB_STEP_SUMMARY"
+2 -2
View File
@@ -54,8 +54,8 @@ package-lock.json
# Local upstream reference
hermes-agent-upstream/
# Captured screenshots (may contain sensitive server IPs/keys — curate manually before committing)
assets/screenshots/
# Claude Code internal state (worktrees, image cache, conversation logs)
.claude/
# Kotlin compiler cache
.kotlin/
+373 -40
View File
@@ -1,53 +1,386 @@
# hermes-android
<!-- @subframe-version 0.15.1-beta -->
<!-- @subframe-managed -->
# hermes-android - SubFrame Project
## Overview
This extension adds Android device control to hermes-agent via the `android` toolset.
It communicates with the Hermes-Relay app running on an Android device over WSS.
This project is managed with **SubFrame**. AI assistants should follow the rules below to keep documentation up to date.
## Setup
> **Note:** This file is named `AGENTS.md` to be AI-tool agnostic. CLAUDE.md and GEMINI.md contain a reference to this file.
### Quick start (relay + plugin)
---
```bash
pip install aiohttp pyyaml && python -m relay_server --no-ssl # start relay
cp -r plugin ~/.hermes/plugins/hermes-android # install plugin
## Core Working Principle
**Only do what the user asks.** Do not go beyond the scope of the request.
- Implement exactly what the user requested — nothing more, nothing less.
- Do not change business logic, flow, or architecture unless the user explicitly asks for it.
- If a user asks for a design change, only change the design. Do not refactor, restructure, or modify functionality alongside it.
- If you have additional suggestions or improvements, **present them as suggestions** to the user. Never implement them without approval.
- The user's request must be completed first. Additional ideas come after, as proposals.
---
## Relationship to Native AI Tools
SubFrame **enhances** native AI coding tools — it does not replace them.
**Claude Code** works exactly as normal. Built-in features (`/init`, `/commit`, `/review-pr`, `/compact`, `/memory`, CLAUDE.md) are fully supported. CLAUDE.md is Claude Code's native instruction file — users can add their own tool-specific instructions freely. SubFrame adds a small backlink reference pointing to this AGENTS.md file using HTML comment markers (`<!-- SUBFRAME:BEGIN -->` / `<!-- SUBFRAME:END -->`). SubFrame will never overwrite user content in CLAUDE.md.
**Gemini CLI** works exactly as normal. Built-in features (`/init`, `/model`, `/memory`, `/compress`, `/settings`, GEMINI.md) are fully supported. GEMINI.md is Gemini CLI's native instruction file — same backlink approach as CLAUDE.md. Users can add their own instructions freely and SubFrame won't overwrite them.
**Codex CLI** gets SubFrame context via a wrapper script at `.subframe/bin/codex` that injects AGENTS.md as an initial prompt.
**This file (AGENTS.md)** contains SubFrame-specific rules that apply across all tools:
- Sub-Task management (`.subframe/tasks/*.md`, index at `.subframe/tasks.json`)
- Codebase mapping (`.subframe/STRUCTURE.json`)
- Context preservation (`.subframe/PROJECT_NOTES.md`)
- Internal docs and changelog (`.subframe/docs-internal/`)
- Session notes and decision tracking
---
## Session Start
**Read these files at the start of each session:**
1. **`.subframe/STRUCTURE.json`** — Module map, file locations, architecture notes
2. **`.subframe/PROJECT_NOTES.md`** — Project vision, past decisions, session notes
3. **`.subframe/tasks.json`** — Sub-task index (pending, in-progress, completed)
This gives you full project context before making any changes. The session-start hook (if configured) automatically injects pending/in-progress sub-tasks into your context, but you should still read these files for deeper understanding.
### Concurrent Work & Worktrees
Before making changes, check whether other AI sessions or agent teams are already working on this repository. Signs of concurrent work include:
- In-progress sub-tasks you didn't start (check `.subframe/tasks.json`)
- Recent uncommitted changes in `git status` that aren't yours
- Lock files or active worktrees (`git worktree list`)
**If concurrent work is detected**, ask the user: "Another session appears to be working on this project. Should I use a git worktree to avoid conflicts?"
**Git worktrees** create an isolated copy of the repo on a separate branch, allowing parallel work without merge conflicts:
- Each worktree has its own working directory and branch
- Changes in one worktree don't affect others until merged
- Use worktrees when multiple agents or sessions work on different features simultaneously
**When to suggest a worktree:**
- Agent teams spawning multiple workers on the same repo
- User asks to work on a feature while another is in progress
- The session-start hook flags concurrent sessions
**When worktrees are NOT needed:**
- Single-session work with no concurrent agents
- Read-only exploration or research tasks
- Quick fixes that won't conflict with in-progress work
---
## Hooks (Automatic Awareness)
SubFrame can configure project-level hooks that automate sub-task awareness. These hooks fire automatically — no manual intervention needed.
| Hook | When it fires | What it does |
|------|---------------|--------------|
| **SessionStart** | Startup, resume, after compaction | Injects pending/in-progress sub-tasks into context |
| **UserPromptSubmit** | Each user prompt | Fuzzy-matches prompt against pending sub-tasks, suggests starting a match |
| **Stop** | When AI finishes responding | Reminds about in-progress sub-tasks; flags untracked work if source files changed |
| **PreToolUse** | Before tool execution | Project-specific guardrails (if configured) |
| **PostToolUse** | After tool execution | Project-specific follow-ups (if configured) |
These hooks ensure sub-task awareness even after context compaction. Hook configuration lives in `.claude/settings.json`.
---
## Skills (Slash Commands)
SubFrame provides optional slash commands for AI coding tools that support them (e.g., Claude Code):
| Skill | Purpose |
|-------|---------|
| `/sub-tasks` | Interactive sub-task management — list, start, complete, add, archive |
| `/sub-docs` | Sync all SubFrame documentation after feature work (changelog, CLAUDE.md, PROJECT_NOTES, STRUCTURE) |
| `/sub-audit` | Code review + documentation audit on recent changes |
| `/onboard` | Bootstrap SubFrame files from existing codebase context |
Skills are deployed to `.claude/skills/` and enhance the workflow — but direct file editing always works as a fallback. If your AI tool doesn't support skills, follow the manual instructions in each section below.
---
## Sub-Task Management
> **Terminology:** "Sub-Tasks" are SubFrame's project task tracking system. The name plays on "Sub" from SubFrame and disambiguates from Claude Code's internal todo tools. When the user says "sub-task", they mean this system.
### Sub-Task File Format
Each sub-task lives in its own markdown file at `.subframe/tasks/<id>.md` with YAML frontmatter:
```yaml
---
id: task-abc12345
title: Short and clear title (max 60 characters)
status: pending | in_progress | completed
priority: high | medium | low
category: feature | fix | refactor | docs | test | chore
description: AI's detailed explanation — what, how, which files affected
userRequest: User's original prompt/request — copy exactly
acceptanceCriteria: When is this task done? Concrete testable criteria
blockedBy: [] # task IDs this depends on
blocks: [] # task IDs that depend on this
createdAt: ISO timestamp
updatedAt: ISO timestamp
completedAt: ISO timestamp | null
---
## Notes
[YYYY-MM-DD] Session notes, alternatives considered, dependencies.
## Steps
- [x] Completed step
- [ ] Pending step
```
Then restart hermes-agent. See [docs/relay-server.md](docs/relay-server.md) for Docker, systemd, TLS, and configuration options.
A generated index is kept at `.subframe/tasks.json` for hooks and quick lookups. After creating or modifying task `.md` files, regenerate the index by reading all `.subframe/tasks/*.md` files (excluding `archive/`) and building the JSON with tasks grouped by status.
### Full setup
1. Install the Hermes-Relay APK on the Android device (build via `scripts/dev.bat build`)
2. Grant the app Accessibility Service permission in Settings > Accessibility
3. Grant SYSTEM_ALERT_WINDOW permission
4. Start the relay server: `pip install aiohttp pyyaml && python -m relay_server --no-ssl`
5. Install the plugin: `cp -r plugin ~/.hermes/plugins/hermes-android`
6. Restart hermes-agent
### Sub-Task Recognition Rules
## Tool usage patterns
**These ARE SUB-TASKS:**
- When the user requests a feature or change
- Decisions like "Let's do this", "Let's add this", "Improve this"
- Deferred work: "We'll do this later", "Let's leave it for now"
- Gaps or improvement opportunities discovered while coding
- Situations requiring bug fixes
### Read before act
ALWAYS call android_read_screen before tapping. Never guess coordinates.
**These are NOT SUB-TASKS:**
- Error messages and debugging sessions
- Questions, explanations, information exchange
- Temporary experiments and tests
- Work already completed and closed
- Instant fixes (like typo fixes)
### Prefer text over coordinates
Use android_tap_text("Continue") over android_tap(x=540, y=1200).
### Sub-Task Creation Flow
### Wait after navigation
After opening an app or tapping a button that triggers loading,
always call android_wait with expected text before next action.
1. Detect sub-task patterns during conversation
2. **Check existing sub-tasks first** — read `.subframe/tasks.json` to avoid duplicates
3. Ask the user: "I identified these sub-tasks from our conversation, should I add them?"
4. If approved, create `.subframe/tasks/<id>.md` with all required frontmatter fields
5. Regenerate the `.subframe/tasks.json` index
### Confirmation pattern for destructive actions
Before confirming a purchase, ride, or send action — always report
to the user what you're about to do and wait for approval.
Example: "I'm about to confirm an Uber ride to [destination] for [price].
Reply 'yes' to confirm."
### Sub-Task Content Rules
## Common package names
- com.ubercab — Uber
- com.bolt.client — Bolt
- com.whatsapp — WhatsApp
- com.spotify.music — Spotify
- com.google.android.apps.maps — Google Maps
- com.android.chrome — Chrome
- com.google.android.gm — Gmail
- com.instagram.android — Instagram
- com.twitter.android — X/Twitter
**title:** Short, action-oriented
- OK: "Add tasks button to terminal toolbar"
- Bad: "Tasks"
**description:** AI's detailed technical explanation
- What will be done, how, which files affected
- Minimum 2-3 sentences
**userRequest:** User's original words — copy verbatim for context preservation
**acceptanceCriteria:** Concrete, testable completion criteria
### Sub-Task Status Updates
**Before starting any work**, check `.subframe/tasks.json` for an existing sub-task that matches. If found, set it to `in_progress` — do not create a duplicate.
- `pending` → `in_progress` — immediately when you begin working (update `updatedAt`)
- `in_progress` → `completed` — when done and verified (set `completedAt`, update `updatedAt`)
- `completed` → `pending` — when reopening, add a note explaining why
- After commit: check and update the status of all related sub-tasks
- **Incomplete work:** If partially done at session end, leave as `in_progress` and add a notes entry
### Sub-Task Lifecycle
- If a sub-task grows beyond its original scope, split it — create new sub-tasks and reference the parent ID in notes
- Cross-reference relevant commit hashes or PR numbers in notes
- Update the description if the approach changes significantly
### Priority Guidelines
- **high** — Blocking other work or explicitly flagged as urgent by the user
- **medium** — Normal feature work and standard bug fixes
- **low** — Nice-to-have improvements, deferred items, minor polish
---
## .subframe/PROJECT_NOTES.md Rules
### When to Update?
- When an important architectural decision is made
- When a technology choice is made
- When an important problem is solved and the solution method is noteworthy
- When an approach is determined together with the user
### Format
Free format. Date + title is sufficient:
```markdown
### [YYYY-MM-DD] Topic title
Conversation/decision as is, with its context...
```
### Update Flow
- Update immediately after a decision is made
- You can add without asking the user (for important decisions)
- You can accumulate small decisions and add them in bulk
### Organization Rules
- Keep **"Project Vision"** at the top, then **"Session Notes"** in chronological order
- Notes should capture the **why** (decisions, trade-offs, alternatives rejected), not the **what** (code structure belongs in STRUCTURE.json)
- When the same topic spans multiple sessions, consolidate related notes under the original heading rather than creating duplicates
- When notes grow beyond ~500 lines, consider archiving older session notes or grouping by month
---
## Context Preservation (Automatic Note Taking)
SubFrame's core purpose is to prevent context loss. Capture important moments and ask the user.
### When to Ask?
Ask the user: **"Should I add this to .subframe/PROJECT_NOTES.md?"** when:
- A sub-task is successfully completed
- An important architectural/technical decision is made
- A bug is fixed and the solution method is noteworthy
- "Let's do this later" is said (also add as a sub-task)
- A new pattern or best practice is discovered
### Importance Threshold
**Would it take more than 5 minutes to re-derive or re-explain in a future session?** If yes, capture it.
**Always capture:** Architecture decisions, technology choices, approach changes, user preferences discovered during work.
**Never capture:** Routine debugging steps, simple config changes, typo fixes.
**Note failed approaches too** — a brief "We tried X, it didn't work because Y" prevents future re-exploration of dead ends.
### Completion Detection
Pay attention to these signals:
- User approval: "okay", "done", "it worked", "nice", "fixed", "yes"
- Moving from one topic to another
- User continuing after build/run succeeds
### How to Add?
1. **DON'T write a summary** — Add the conversation as is, with its context
2. **Add date** — In `### [YYYY-MM-DD] Title` format
3. **Add to Session Notes section** — At the end of PROJECT_NOTES.md
### When NOT to Ask
- For every small change (it becomes spam)
- Typo fixes, simple corrections
- If the user already said "no" or "not needed", don't ask again for that topic
### If User Says "No"
No problem, continue. The user can also say what they consider important themselves: "add this to notes"
---
## .subframe/STRUCTURE.json Rules
**This file is the map of the codebase.**
### When to Update?
- When a new file/folder is created
- When a file/folder is deleted or moved
- When module dependencies change
- When an IPC channel is added or changed
- When an important architectural pattern is discovered (architectureNotes)
### Full Schema
```json
{
"modules": {
"main/moduleName": {
"file": "src/main/moduleName.ts",
"description": "What this module does",
"exports": ["init", "loadData"],
"depends": ["fs", "path", "shared/ipcChannels"],
"functions": {
"init": { "line": 15 },
"loadData": { "line": 42 }
}
}
},
"ipcChannels": {
"CHANNEL_NAME": {
"direction": "renderer → main",
"handler": "main/moduleName"
}
},
"architectureNotes": {
"topicName": {
"issue": "Description of the pattern or concern",
"solution": "How it was resolved"
}
}
}
```
### Update Rules
- The pre-commit hook (if configured) auto-updates STRUCTURE.json when source files in `src/` are committed
- When deleting files, remove their entries from `modules` and update any `depends` arrays that referenced them
- When adding IPC channels, also add them to the `ipcChannels` section with `direction` and `handler`
- `architectureNotes` is for **structural patterns** (e.g., circular dependency workarounds, init ordering). Use PROJECT_NOTES.md for **decisions and session context**
- If function line numbers drift significantly after edits, re-run the pre-commit hook or update manually
---
## .subframe/docs-internal/ Directory
This directory holds project documentation that doesn't belong in the root:
| File | Purpose |
|------|---------|
| `changelog.md` | Track changes under `## [Unreleased]`, grouped by Added/Changed/Fixed/Removed |
| `*.md` (ADRs) | Architecture Decision Records for significant design choices |
**What goes here:** Changelog entries, architecture decision records, internal reference docs.
**What does NOT go here:** User-facing docs (those go in `docs/` or project root), task files (those go in `.subframe/tasks/`).
---
## .subframe/QUICKSTART.md Rules
### When to Update?
- When installation steps change
- When new requirements are added
- When important commands change
---
## Before Ending Work
After significant work (code changes, architecture decisions), verify SubFrame files are in sync:
1. **Sub-Tasks** — Was this work tracked? Check `.subframe/tasks.json` → create/complete as needed
2. **PROJECT_NOTES.md** — Any decisions worth preserving? Ask the user
3. **Changelog** — Does `.subframe/docs-internal/changelog.md` reflect the changes?
4. **STRUCTURE.json** — Source files changed? The pre-commit hook handles this automatically if configured; otherwise update manually
The stop hook (if configured) will flag untracked work automatically.
---
## General Rules
1. **Language:** Write documentation in English (except code examples)
2. **Date Format:** ISO 8601 (YYYY-MM-DDTHH:mm:ssZ)
3. **After Commit:** Check sub-tasks (`.subframe/tasks/*.md`) and `.subframe/STRUCTURE.json`
4. **Session Start:** Read STRUCTURE.json, PROJECT_NOTES.md, and tasks.json before making changes
5. **Don't Duplicate:** Always check existing sub-tasks before creating new ones
---
*This file was automatically created by SubFrame.*
*Creation date: 2026-04-14*
<!-- subframe-template-version: 1 -->
+343
View File
@@ -6,6 +6,349 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
## [Unreleased]
## [0.4.0] - 2026-04-14
### Added — Bridge feature expansion (the big one)
v0.4 roughly triples the bridge surface. The agent can now do everything
v0.3 could do, plus long-press, drag, full clipboard access, system-wide
media control, raw Android Intents, an accessibility-event stream, app
launching, app listing, multi-window screen reads, filtered node search,
screen-hash change detection, stable per-node IDs, three-tier
`tap_text` fallback, a batched macro dispatcher, wake-lock-guarded
gesture dispatch, and a per-app skill playbook for common flows. The
sideload track additionally ships direct SMS, contact lookup, one-tap
dialing, and location awareness.
**Read surface**
- **`/long_press`** (A1) — long-press gesture by coordinate or node ID,
covering context menus, text selection, and widget rearranging
- **`/drag`** (A2) — drag gesture from point A → point B over a
configurable duration
- **`/find_nodes`** (A3) — filtered accessibility-tree search (text,
clickable flag, class name, resource ID) instead of returning the
whole tree
- **`/describe_node`** (A4) — full property bag for a stable node ID,
plus `nodeId` wiring for `/tap` and `/scroll` so the agent can hand
IDs forward without re-resolving coordinates
- **`/screen_hash`** + **`/diff_screen`** (A5) — cheap SHA-256 screen
fingerprint and diff tools for "wait until this screen changes"
loops without re-downloading the full accessibility tree
- **`/events`** + **`/events/stream`** (B1) — accessibility-event
stream. In-memory `EventStore` buffers recent `AccessibilityEvent`
objects so the agent can poll for UI events or wait for a specific
trigger instead of hammering `/screen`. Toggle capture on/off via
`/events/stream`.
- **Multi-window `ScreenReader`** (P1) — `/screen` now walks every
accessibility window (system UI, popups, notification shade) instead
of only the active app's window
**Act surface**
- **`/clipboard`** (A6) — bidirectional system clipboard read/write
- **`/media`** (A7) — system-wide playback control (play, pause, next,
previous, volume) via `MediaSessionManager`
- **`/send_intent`** + **`/broadcast`** (B4) — raw Android Intent /
broadcast escape hatch for apps that expose deep-link actions
- **Three-tier `tap_text` cascade** (A9) — exact match → clickable
ancestor walk → substring fallback, fixes apps that wrap labels in
non-clickable parents
- **`android_macro`** (A10) — batched workflow dispatcher runs a
sequence of bridge commands as one call with configurable pacing,
no round-trip per step
- **`WakeLockManager`** (A8) — `PARTIAL_WAKE_LOCK` scope wrapper around
gesture dispatch so commands still land on dim or idle screens.
Scoped try/finally semantics, never a stale hold.
**Tier C — sideload-only phone utilities**
- **`/location`** (C1) — GPS last-known-location read for "where am
I?" and location-scoped commands
- **`/search_contacts`** (C2) — contact lookup by name → phone number
for voice intents like "text Mom"
- **`/call`** (C3) — direct call via `ACTION_CALL`, with an
`ACTION_DIAL` fallback where the flavor can't hold `CALL_PHONE`
- **`/send_sms`** (C4) — direct SMS send via `SmsManager` with
send-result confirmation (no dialer bounce)
**Docs + skills**
- **`skills/android/SKILL.md`** (A11) — per-app playbook with reusable
flows for common apps, agent-discoverable via the Hermes skills
system
- **`docs/spec.md` + `docs/decisions.md`** — v0.4 bridge surface
documented, Phase 3 status marked shipped, 15-item spec rot pass
### Fixed
- **Missing Kotlin handlers for `/open_app`, `/get_apps`, `/apps`,
and `/setup`** — latent v0.3.0 regression. The Python relay side had
the routes and the plugin tools were calling them, but the in-app
`BridgeCommandHandler` had never wired the corresponding `when (path)
->` branches. Commands silent-dropped until this release.
- **Android 11+ package visibility for `/get_apps`** — added a
`<queries>` element to the main manifest so
`PackageManager.queryIntentActivities(ACTION_MAIN + CATEGORY_LAUNCHER)`
returns the full launchable app list. Without this, the tool returned
an empty list on modern Android targets.
### Docs
- **`user-docs` expansion** — added the full 27-route bridge HTTP
inventory to `reference/relay-server.md`, rewrote
`architecture/security.md` around the five-stage safety gate + Tier 5
rails, added ADR-9 through ADR-13 (bridge safety gate, wake-scope,
event stream, MediaProjection FGS type, build flavors), and retired
all remaining "Bridge :8766" references now that the bridge is
unified on `:8767`.
## [0.3.0] - 2026-04-13
### Added
**Bridge channel (the big one)** — the agent can now read the phone's
screen, tap, type, swipe, and take screenshots. Gated behind a
deliberate in-app master toggle, per-channel session grants, Android
Accessibility Service permission, MediaProjection consent, and the
safety rails system (blocklist, destructive-verb confirmation modal,
idle auto-disable timer, optional persistent status overlay).
- **`HermesAccessibilityService`** — Android `AccessibilityService`
subclass that reads the active window's UI tree, dispatches taps /
types / swipes / scrolls / key presses via `GestureDescription` and
`ACTION_SET_TEXT`, and caches the foregrounded package
- **`ScreenCapture.kt`** — `MediaProjection` → `VirtualDisplay` →
`ImageReader` → PNG bytes, uploaded to the relay via `/media/upload`
- **`BridgeCommandHandler`** — routes inbound `bridge.command` envelopes
to the executor, with the three-stage safety check
(blocklist → destructive-verb confirmation → auto-disable reschedule)
- **`BridgeSafetyManager`** — process-wide safety enforcement singleton
with DataStore-backed blocklist (30 default banking/payments/2FA
apps), destructive verb list (`send`/`pay`/`delete`/`transfer`/etc.),
auto-disable timer, and confirmation timeout
- **`BridgeForegroundService`** — persistent "Hermes has device control"
notification with Disable + Settings actions, deep-linked to the
Bridge Safety settings screen
- **`BridgeStatusOverlay`** — `WindowManager` overlay host for the
destructive-verb confirmation modal and optional floating status chip
- **Bridge UI** — new Bridge tab with master toggle, permission
checklist (accessibility / screen capture / overlay / notification
listener), status card, activity log, and safety summary card
- **Bridge Safety settings screen** — blocklist editor (searchable
package picker), destructive verb editor, auto-disable timer slider,
status overlay toggle, confirmation timeout slider
- **18 `android_*` plugin tools** routed through the new unified bridge (14 baseline + send_sms, call, search_contacts, return_to_hermes added in v0.4.0)
channel (migrated from the legacy standalone `android_relay.py`)
- **`android_navigate`** — vision-driven close-the-loop navigation tool
(sideload track only)
- **`android_phone_status`** — agent-callable introspection tool that
reports live bridge state (`device`, `bridge` permissions, `safety`
config) via the new `/bridge/status` relay endpoint
**Voice mode** — real-time voice conversation via relay TTS/STT:
- Tap the mic in the chat bar to enter voice mode with the ASCII
morphing sphere, layered-sine-wave waveform visualizer, and
streaming sentence-level TTS playback
- Three interaction modes (Tap-to-talk, Hold-to-talk, Continuous)
- Sphere voice states — Listening (blue/purple), Speaking (green/teal)
- Voice settings screen (interaction mode, silence threshold, provider
info, Test Voice)
- New relay endpoints — `POST /voice/transcribe`, `POST /voice/synthesize`,
`GET /voice/config`
- **Voice-to-bridge intent routing** (sideload track only) —
spoken commands like "text Mom saying on my way" route to the bridge
channel instead of the chat channel, with destructive-verb
confirmation flow
**Notification companion** — `HermesNotificationCompanion`
(`NotificationListenerService`) reads posted notifications and forwards
them to the relay over a new `notifications` channel for agent
summaries. Opt-in via the standard Android notification-access grant.
**Self-setup skill** — `/hermes-relay-self-setup` and
`skills/devops/hermes-relay-self-setup/SKILL.md`. Single-source agent
install recipe — raw URL fetch for pre-install users, Hermes skill
discovery for post-install users. Zero drift.
**Manual pairing fallback** — `hermes-pair --register-code ABCD12`
pre-registers an arbitrary 6-char code with the relay for the rare
"no camera available" case (SSH-only, single-device pair). Phone-side
"Manual pairing code (fallback)" card in Settings → Connection walks
the user through the three-step workflow with a real Connect button.
**Per-channel grant revoke** — each device on the Paired Devices screen
now has tappable per-channel grant chips (chat / voice / terminal /
bridge) with an inline x icon. Revoking a single channel leaves the
other channels' expiries intact.
**`hermes-status`** and **`hermes-relay-update`** shell shims — alongside
`hermes-pair`, these three shims give full discoverable CLI coverage for
pair / status / update.
**`/health` + `/bridge/status` relay endpoints** — loopback-only
structured phone-status endpoint (device, bridge permissions, safety
state) that backs `hermes-status`, `android_phone_status()`, and the
`/hermes-relay-status` skill.
**ConnectionWizard + onboarding unification** — shared three-step
pairing wizard (Scan → Confirm → Verify) used by both first-run
onboarding and re-pair from Settings. Eliminates the "half-paired" state
where onboarding configured the API side but dropped the relay block.
**Lifecycle-aware health checks** — `ConnectionViewModel.revalidate()`
fires on `ON_RESUME` and on `ConnectivityObserver` `Available`
transitions, with a new `Probing` tri-state and a new gray pulsing
`ConnectionStatusBadge` pose. Kills the "foreground lag flash" where
badges showed stale Connected/Disconnected for 30s after foregrounding.
**Two build flavors** — `googlePlay` (Play Store track, conservative
Accessibility use case) and `sideload` (`.sideload` applicationId
suffix, full feature set including voice-to-bridge intents and
`android_navigate`). `sideload` shows as "Hermes Dev" in the launcher
for side-by-side disambiguation.
**TOFU cert pinning**, **Android Keystore session token storage**
(StrongBox-preferred with `EncryptedSharedPreferences` fallback),
**transport security badge**, **session TTL picker dialog** (1d / 7d /
30d / 90d / 1y / never), **Paired Devices screen** with full revoke
flow, **Tailscale detector**, **insecure-mode ack dialog** with reason
picker.
### Changed
- **`install.sh` TUI polish** — ANSI colors (TTY-aware, `NO_COLOR`
respected), boxed banner, unicode step bullets, spinner for the long
pip install, polished closing message with structured Pair / Update /
Manage / Uninstall sections
- **`install.sh` restart semantics** — the restart-relay path now uses
explicit `systemctl --user restart` instead of `enable --now` (the
latter is a no-op on already-active services and silently left
editable-install code refreshes stranded)
- **`install.sh` step 6b** — offer (don't force) hermes-gateway restart
so new plugin tools re-import. Interactive prompt (default no), env
var opt-in via `HERMES_RELAY_RESTART_GATEWAY=1`, or flag opt-out
- **Connection settings card rename** — "Bridge pairing code" → "Manual
pairing code (fallback)" with a walkthrough UX instead of a bare
code display. The old label implied bridge-specific 2FA; it's
actually the auth fallback for the whole handshake.
- **Sideload flavor strings** — `app_name` → `Hermes Dev`,
`a11y_service_label` → `Hermes-Bridge Dev`, notification companion
label → `Hermes Dev notification companion`. Disambiguates side-by-side
installs in launcher / recents / Settings → Apps.
- **Google Play flavor a11y label** → `Hermes-Bridge` (with hyphen)
for consistency with the sideload naming
- **`BridgeStatusReporter`** — pushed envelope now includes the full
nested `device` / `bridge` / `safety` contract instead of four flat
keys, with a new `pushNow()` method for out-of-band emission on
master toggle flips
- **`hermes-relay.service` systemd unit** — runs the relay on port
8767 with `--no-ssl --log-level INFO`, loads `~/.hermes/.env` via
`_env_bootstrap.py` at import time (no `EnvironmentFile=` needed)
### Fixed
- **Android 14 MediaProjection grant evaporation** — on API 34+,
`getMediaProjection()` returned projections the system auto-revoked
within frames because `BridgeForegroundService` was declared as
`specialUse` only. Added `mediaProjection` to the FGS type slot,
updated `startForeground()` to OR both type constants, and gated
`requestScreenCapture()` on the master toggle so the FGS is
guaranteed running before consent fires.
- **Master toggle gate broken end-to-end** — `cachedMasterEnabled` was
never written because nothing called `updateMasterEnabledCache`. The
cache was permanently `false` and `BridgeCommandHandler` 403'd every
command except `/ping` and `/current_app`. Service now owns a
coroutine that observes the DataStore flow and feeds the cache.
- **MediaProjection consent flow never wired** — `MediaProjectionHolder.
onGranted` existed but no `ActivityResultLauncher` was registered.
`MainActivity` now registers a launcher and a new
`ScreenCaptureRequester` process-singleton bridges non-Activity
callers (`BridgeViewModel.requestScreenCapture()`).
- **Manifest dedupe** — duplicate `HermesAccessibilityService` entry in
Android Settings caused by a stub `<service>` block in the flavor
manifests that pointed at a class that didn't exist
- **Gradle deprecation** — `android.dependency.
excludeLibraryComponentsFromConstraints=true` collapsed into
`useConstraints=false`
- **Version drift** — `pyproject.toml` had speculatively bumped to
`0.5.0` and `plugin/relay/__init__.py::__version__` was stuck at
`0.2.0`. Both synced to `0.3.0` via the new `bump-version.sh` script.
### Docs
- **`hermes-relay-self-setup`**, **`hermes-relay-pair`**, and
**`hermes-relay-status`** skills — agent-readable setup / pair /
status recipes via the Hermes skills system
- **`RELEASE.md`** — expanded with the three-source version contract,
feature-branch workflow, `--no-ff` merge style, branch protection
policy, and `bump-version.sh` recipe
- **`CLAUDE.md`** — updated Git section with the new branching policy,
added file-table entries for `hermes-relay-update`,
`register_code_command`, and the expanded `install.sh`
- **`TODO.md`** — captures open research questions around proper
Hermes plugin/skill/tool distribution
- **`user-docs` vitepress site** — new "For AI Agents" copy-paste
block on the home view, Feature Matrix component, two-track explainer,
manual-pair workflow walkthrough in configuration.md
## [0.2.0] - 2026-04-12
### Added
- **Voice mode** — real-time voice conversation via relay TTS/STT endpoints. Tap the mic in the chat bar to enter voice mode with the sphere, waveform visualizer, and streaming sentence-level TTS playback
- Three interaction modes (Tap-to-talk, Hold-to-talk, Continuous)
- Streaming TTS with sentence-boundary detection
- Interrupt: tap stop during Speaking to cancel TTS + SSE stream
- Sphere voice states: Listening (blue/purple), Speaking (green/teal)
- Voice settings screen (interaction mode, silence threshold, provider info, Test Voice)
- Relay endpoints: `POST /voice/transcribe`, `POST /voice/synthesize`, `GET /voice/config`
- 6 TTS + 5 STT providers via hermes-agent config
- Voice messages appear as normal chat messages in session history
- **Reactive layered-sine waveform** — three overlapping waves with amplitude-driven phase velocity (`withFrameNanos` ticker), pill-shaped edge merge (geometric `sin(πt)` taper + `BlendMode.DstIn` gradient mask), color-keyed to voice state
- **Enter/exit voice chimes** — synthesized 200ms PCM sweeps via AudioTrack (440→660 Hz enter, mirror exit)
- **Terminal (preview)** — tmux-backed persistent shells with tabs, scrollback search, and session info sheet
- **Session TTL picker** — choose 1d / 7d / 30d / 90d / 1y / Never at pair time
- **Per-channel grants** — control terminal/bridge access per paired device
- **Android Keystore token storage** — StrongBox-preferred hardware-backed encrypted storage with TEE fallback
- **TOFU certificate pinning** — SHA-256 SPKI fingerprints per host:port, wiped on re-pair
- **Paired Devices screen** — list all paired devices with metadata, extend sessions, revoke access
- **Transport security badges** — three-state visual indicator (secure / insecure-with-reason / insecure-unknown)
- **HMAC-SHA256 QR signing** — pairing QR codes signed via host-local secret
- **Insecure connection acknowledgment dialog** — first-time consent with threat model explanation + reason picker
- **Inbound media pipeline** — agent-produced files via relay `MediaRegistry` with opaque tokens, Discord-style rendering for image/video/audio/PDF/text/generic attachments
- **`/media/by-path` endpoint** — LLM-emitted `MEDIA:/path` markers fetched directly by the phone
- **Settings refactor** — category-list landing page with dedicated sub-screens (Connection, Chat, Voice, Media, Appearance, Paired Devices, Analytics, Developer)
- **Global font-scale preference** — applies to both chat and terminal
- **RelayErrorClassifier** — converts any `Throwable` into a user-facing `HumanError(title, body, retryable, actionLabel)` with context-aware titles
- **Global SnackbarHost** — `LocalSnackbarHost` CompositionLocal at RelayApp scope so any screen can surface classified errors
- **Mic permission banner** — rebuilt with "Open Settings" action button instead of a toast
- **Relay `.env` autoload** — `plugin/relay/_env_bootstrap.py` loads `~/.hermes/.env` at Python import time, matching the gateway pattern
- **systemd user service** — `install.sh` step [6/6] installs and enables `hermes-relay.service` automatically
- **Save & Test health probe** — relay connection verification with classified error feedback
- **Gradle logcat task** — `silenceAndroidViewLogs` auto-runs `adb shell setprop log.tag.View SILENT` after every install to suppress Compose Android 15 VRR spam
- **App screenshots** in `assets/screenshots/`
### Fixed
- **Voice replying to wrong turn** — `ignoreAssistantId` baseline prevents the stream observer from replaying the previous turn's response as TTS for the new question
- **Waveform flatline between sentences** — TTS consumer restructured from `for` loop to `while` + `tryReceive` so `maybeAutoResume` only fires when the queue is actually drained, not after every sentence
- **Stop button during Speaking** — `interruptSpeaking()` now cancels the SSE stream via `chatViewModel.cancelStream()`, drains TTS queue, and returns to Idle (previously only paused playback)
- **Waveform unresponsive to speech** — perceptual amplitude curve at the source (noise-floor subtraction + speech-ceiling rescale + sqrt boost), attack/release envelope follower (0.75/0.10 at 60Hz), killed Compose spring double-smoothing
- **Stop button color** — hardcoded vivid red `Color(0xFFE53935)` for Listening + Speaking (Material 3 dark `colorScheme.error` resolved to pale pink)
- **NaN amplitude propagation** — guards in VoicePlayer.computeRms and VoiceViewModel.sanitizeAmplitude (IEEE 754: `Float.coerceIn` silently passes NaN)
- **Relay voice 500 on restart** — `.env` not loaded into relay process when started via nohup/systemd without shell sourcing
- **Rate-limit block on re-pair** — `/pairing/register` clears all rate-limit blocks on success
- **Paired devices JSON unwrap** — permissive `/media/by-path` sandbox
- **Settings status flicker** — unified relay status as "Reconnecting..." on Settings entry
### Changed
- Smart-swap trailing input button (empty → Mic, text → Send) replacing the floating Mic FAB
- Voice overlay is fully opaque surface (was 0.95 alpha — "phantom pencil" bleed-through from chat)
- Bottom nav hidden during voice mode
- Scrollable response text in voice overlay (long responses no longer clip)
- `install.sh` is now 6 steps (was 5) — new step [6/6] for systemd user service
- Relay restart is `systemctl --user restart hermes-relay` (nohup era ended)
## [0.1.0] - 2026-04-07
### Added
+185 -112
View File
@@ -4,9 +4,9 @@
## What This Is
A native Android app for Hermes agent. Chat connects directly to the Hermes API Server. Bridge and terminal channels use a relay server over WSS. The app is Kotlin + Jetpack Compose. The server relay is Python + aiohttp.
A native Android app (Kotlin + Jetpack Compose) paired with a Python relay server (aiohttp) for the Hermes agent platform. Chat connects directly to the Hermes API Server via HTTP/SSE; bridge and terminal use a relay over WSS.
**Current state:** v0.1.0 (Google Play). Phase 0 + Phase 1 complete with direct API chat, session management, markdown rendering, messaging-style chat header (avatar + agent name + model subtitle), personality picker with agent name on bubbles, searchable command palette (29 gateway commands + dynamic personalities + server skills), QR code pairing, ConnectionStatusBadge (animated pulse ring), in-app analytics (Stats for Nerds with reset, peak times, tokens/msg), animated splash screen, tool display configuration, client-side message queuing (send while streaming), file attachments (images, documents, any file type via base64), configurable limits (attachment size, message length), feature gating with Developer Options, ASCII morphing sphere animation (empty chat state + ambient mode + behind-messages background), and animation settings in Settings. The relay server handles bridge (Phase 3) and terminal (Phase 2) via WSS. Auth uses optional Bearer token for API, pairing code for relay. Relay/pairing settings are hidden in production behind Developer Options (tap version 7x to unlock).
**Current state:** v0.4.x — Phase 0–3 complete. Direct API chat, session management, pairing + security, inbound media, voice mode, bridge/accessibility control, notification companion, and safety rails. Two product flavors: `googlePlay` (conservative) and `sideload` (full-capability).
## Architecture
@@ -33,58 +33,62 @@ Chat goes directly to the API server via HTTP/SSE. The API key (Bearer token) is
| `GET /health` | Health check | — |
| `GET/POST/PATCH/DELETE /api/jobs/*` | Cron job management | — |
**Non-standard endpoints (may be version-specific):**
**Non-standard endpoints (provided by fork OR by plugin bootstrap):**
These endpoints work on our hermes-agent v0.7.0 but are **not in the upstream source**. They may be fork-specific, version-specific, or added by plugins. Always use `detectChatMode()` to probe availability.
These endpoints are not in stock upstream `gateway/platforms/api_server.py`. There are three ways a hermes-agent install can serve them:
| Endpoint | Purpose | Fallback |
|----------|---------|----------|
| `POST /api/sessions/{id}/chat/stream` | Session-based SSE chat | Use `/v1/runs` or `/v1/chat/completions` |
| `GET/POST/PATCH/DELETE /api/sessions` | Session CRUD | Use `X-Hermes-Session-Id` header with `/v1/chat/completions` |
| `GET /api/skills` | Skill discovery | Hardcoded command list |
| `GET /api/config` | Server config (personalities, model) | No fallback — personality picker empty |
1. **Codename-11 fork** (`feat/session-api` branch, deployed on the `axiom` branch) — adds them natively. Submitted upstream as PR [#8556](https://github.com/NousResearch/hermes-agent/pull/8556) *"feat(api-server): add session management API for frontend clients"* — scope is broader than the title: sessions CRUD + session chat/stream + memory + skills + config + available-models.
2. **Bootstrap injection** (`hermes_relay_bootstrap/`) — monkey-patches aiohttp on startup via `.pth` file. Does NOT inject `/api/sessions/{id}/chat/stream` — use `/v1/runs` for chat.
3. **Upstream-merged** (post PR #8556) — bootstrap auto-detects and no-ops.
| Endpoint | Purpose | Provided by |
|----------|---------|-------------|
| `GET /api/sessions` (CRUD) | Session list/create/rename/delete/fork | Fork OR bootstrap OR upstream-merged |
| `GET /api/sessions/{id}/messages` | Conversation history | Fork OR bootstrap OR upstream-merged |
| `GET /api/sessions/search` | Full-text message search | Fork OR bootstrap OR upstream-merged |
| `POST /api/sessions/{id}/chat/stream` | Session-based SSE chat | Fork OR upstream-merged ONLY (NOT bootstrap) |
| `GET /api/config`, `PATCH /api/config` | Personalities + model config | Fork OR bootstrap OR upstream-merged |
| `GET /api/skills`, `/categories`, `/{name}` | Skill discovery | Fork OR bootstrap OR upstream-merged |
| `GET/POST/PATCH/DELETE /api/memory` | Memory CRUD | Fork OR bootstrap OR upstream-merged |
| `GET /api/available-models` | Provider model list | Fork OR bootstrap OR upstream-merged |
The Android client probes per-endpoint capability via `HermesApiClient.probeCapabilities()` (returns `ServerCapabilities`). When `streamingEndpoint = "auto"`, `ConnectionViewModel.resolveStreamingEndpoint()` picks `sessions` or `runs` based on the capability snapshot.
**Tool call rendering paths:**
1. **Runs API** (`/v1/runs`) — Best for tool display. Emits `tool.started`/`tool.completed` as real SSE events → rendered as ToolProgressCards in real-time.
2. **Sessions API** (`/api/sessions/{id}/chat/stream`) — Does NOT emit structured tool events during streaming. Tool calls are stored server-side as `tool_calls` JSON on each message. On stream complete, `ChatViewModel.onCompleteCb` reloads message history via `getMessages()` → `loadMessageHistory()` to get proper message boundaries + tool call cards. This is the "session_end reload" pattern.
3. **Annotation parser** (`ChatHandler.parseAnnotationLine` + `finalizeAnnotations`) — Fallback for servers that inject inline markdown annotations (`` `💻 terminal` ``). Parses during streaming + reconciliation pass on stream end. If your Hermes version uses a different format, check `adb logcat -s HermesApiClient` for raw SSE events and update the regex.
1. **Runs API** — Emits `tool.started`/`tool.completed` as real SSE events → `ToolProgressCard` in real-time.
2. **Sessions API** — No structured tool events during streaming; reloads message history on stream complete ("session_end reload" pattern).
3. **Annotation parser** — Fallback for servers emitting inline markdown annotations (`` `💻 terminal` ``).
## Key Instructions
- **Always verify upstream before assuming an endpoint exists.** Check `gateway/platforms/api_server.py` in hermes-agent. If an endpoint isn't there, document it as non-standard and implement a fallback.
- When building features that interface with hermes-agent, reference the upstream source — not just our spec docs. Our spec may be aspirational or based on a specific server version.
- If we use a non-standard endpoint, mark it clearly in code comments and ensure `detectChatMode()` handles its absence gracefully.
- **Always verify upstream before assuming an endpoint exists.** Check `gateway/platforms/api_server.py` in hermes-agent. If an endpoint isn't there, document whether bootstrap injects it or it requires the fork.
- If we use a non-standard endpoint, ensure `probeCapabilities()` covers it and the auto-resolver degrades gracefully.
- **Bootstrap maintenance:** Remove `hermes_relay_bootstrap/` in one PR once PR #8556 merges. It's no-op-compatible, so leaving it in place during rollout is harmless.
## Repository Layout
```
hermes-android/ ← Android Studio opens this root
├── app/ ← Android app module (Compose)
│ ├── src/main/kotlin/com/hermesandroid/relay/
│ │ ├── ui/ # Screens, components, theme
│ │ ├── network/ # ConnectionManager, ChannelMultiplexer, handlers
│ │ ├── auth/ # AuthManager (pairing + tokens)
│ │ ├── viewmodel/ # ChatViewModel, ConnectionViewModel
│ │ └── data/ # ChatMessage, ToolCall models
│ └── build.gradle.kts
├── build.gradle.kts ← Root Gradle (AGP, Kotlin plugins)
├── settings.gradle.kts
├── gradle/ ← Wrapper (8.13) + version catalog
├── scripts/ ← Dev scripts (build, install, run, test, relay)
├── relay_server/ ← Python WSS relay server
│ ├── relay.py # Main aiohttp WSS server
│ ├── auth.py # Pairing + session management
│ ├── channels/ # chat.py, terminal.py (stub), bridge.py (stub)
│ └── config.py
├── plugin/ ← Hermes agent plugin (14 android_* tools)
│ ├── android_tool.py
│ ├── android_relay.py
│ ├── tools/ # Standalone toolset
│ ├── skills/ # Agent skills
│ └── tests/
├── skills/ ← Installable Hermes skills
│ └── hermes-pairing-qr/ # (DEPRECATED) QR pairing — use `hermes pair` from the plugin instead
├── docs/ ← spec, decisions, security
└── .github/workflows/ ← CI + release
hermes-android/
├── app/src/main/kotlin/com/hermesandroid/relay/
│ ├── ui/ # Screens, components, theme
│ ├── network/ # ConnectionManager, ChannelMultiplexer, handlers
│ ├── auth/ # AuthManager (pairing + tokens)
│ ├── viewmodel/ # ChatViewModel, ConnectionViewModel
│ ├── data/ # ChatMessage, ToolCall models, FeatureFlags
│ ├── audio/ # VoiceRecorder, VoicePlayer, VoiceSfxPlayer
│ ├── voice/ # VoiceViewModel, VoiceBridgeIntentHandler
│ ├── accessibility/ # HermesAccessibilityService, ScreenReader, ActionExecutor
│ ├── bridge/ # BridgeSafetyManager, BridgeForegroundService, BridgeStatusOverlay
│ └── notifications/ # HermesNotificationCompanion
├── plugin/ ← Hermes agent plugin
│ ├── android_tool.py # 18 android_* tool handlers
│ ├── pair.py # QR pairing implementation
│ ├── relay/ # Canonical WSS relay (server.py, auth.py, channels/, media.py, voice.py)
│ └── tools/ # android_navigate.py, android_notifications.py
├── relay_server/ ← Thin compat shim → plugin.relay (legacy entrypoint)
├── hermes_relay_bootstrap/ ← Runtime patch for vanilla upstream; removable after PR #8556
├── skills/devops/hermes-relay-pair/ ← /hermes-relay-pair slash command
├── scripts/ ← dev.bat, bridge-smoke.sh, bump-version.sh
└── docs/ ← spec, decisions, security, relay-server, mcp-tooling
```
## Project Conventions
@@ -93,29 +97,32 @@ hermes-android/ ← Android Studio opens this root
- **Root-level:** README.md, CLAUDE.md, AGENTS.md, DEVLOG.md, .gitignore
- **docs/** — spec, decisions, security, and any other long-form documentation
- **DEVLOG.md** — update at end of each work session with what was done, what's next, blockers
- **CLAUDE.md hygiene:** Key Files entries must stay one line — implementation detail belongs in the file or `docs/`. Run `/revise-claude-md` after feature-heavy sessions to trim drift.
### Code Style — Android (Kotlin)
- **Jetpack Compose** — no XML layouts. Material 3 / Material You.
- **kotlinx.serialization** — not Gson. Type-safe, faster.
- **OkHttp** for WebSocket + SSE — `okhttp` for WSS relay, `okhttp-sse` for API streaming
- **Single-activity** — Compose Navigation for all routing
- **Package:** `com.hermesandroid.relay`
- **Min SDK 26, Target SDK 35, Compile SDK 36**
- **Kotlin 2.0+**, JVM toolchain 17
- **Namespace (Kotlin source tree):** `com.hermesandroid.relay` — stable, drives on-disk layout + class FQCNs
- **applicationId:** `com.axiomlabs.hermesrelay` (googlePlay), `com.axiomlabs.hermesrelay.sideload` (sideload)
- **Min SDK 26, Target SDK 35, Compile SDK 36** / **Kotlin 2.0+**, JVM toolchain 17
### Code Style — Server (Python)
- **aiohttp** for the WSS relay — async, matches existing Hermes relay patterns
- **aiohttp** — async, matches existing Hermes relay patterns
- **Type hints everywhere** — Python 3.11+ syntax
- **asyncio** for concurrency — no threading
- **Structured logging** — use `logging` module, not print()
- **asyncio** — no threading; **structured logging** — use `logging`, not print()
### Git
- **Commit messages:** `type: description` — e.g. `feat: add chat channel UI`, `fix: WSS reconnect race condition`
- **Branch from main** — feature branches for anything non-trivial
- **Conventional Commits:** `feat`, `fix`, `docs`, `refactor`, `test`, `chore`
- **Feature branches** as of 2026-04-13. Straight-to-main for single-file typos only.
- **Merge style:** `git merge --no-ff` — no squash. Preserves per-commit trail for agent-team branches.
- **Version bumps on `main` only.** Use `bash scripts/bump-version.sh <new-version>` to bump all three sources atomically (`gradle/libs.versions.toml`, `pyproject.toml`, `plugin/relay/__init__.py`).
- **Branch protection** on `main` since 0.3.0 — PRs must pass CI; direct push blocked except `release: vX.Y.Z`.
### Testing
- **Android:** JUnit + Compose testing for UI, MockK for mocks
- **Python:** pytest for relay tests
- **Python:** `python -m unittest plugin.tests.test_<name>` — avoid bare `pytest` (conftest imports `responses` which may not be installed in the venv)
- **CI runs on every push** — build must pass before merge
## Key Files
@@ -124,48 +131,69 @@ hermes-android/ ← Android Studio opens this root
|------|-----|
| `docs/spec.md` | Full specification — protocol, UI layouts, phases, dependencies |
| `docs/decisions.md` | Architecture decisions — framework choice, channel design, auth model |
| `app/src/main/kotlin/.../ui/RelayApp.kt` | Main scaffold — bottom nav, navigation |
| `app/src/main/kotlin/.../network/HermesApiClient.kt` | Direct HTTP/SSE client — `sendChatStream()` for sessions endpoint, `sendRunStream()` for runs endpoint, `detectChatMode()` for capability probing |
| `app/src/main/kotlin/.../network/ConnectionManager.kt` | WSS connection with auto-reconnect (relay) |
| `app/src/main/kotlin/.../network/ChannelMultiplexer.kt` | Envelope routing by channel (relay) |
| `app/src/main/kotlin/.../network/ConnectivityObserver.kt` | Reactive network connectivity listener |
| `app/src/main/kotlin/.../network/handlers/ChatHandler.kt` | Chat message state, streaming events, tool annotation parser (inline markdown → ToolCall) |
| `app/src/main/kotlin/.../network/models/SessionModels.kt` | Session, message, SSE event data models |
| `app/src/main/kotlin/.../data/FeatureFlags.kt` | Feature gating — compile-time defaults (DEV_MODE) + runtime DataStore overrides |
| `app/src/main/kotlin/.../data/AppAnalytics.kt` | In-app analytics singleton (TTFT, tokens, health, stream rates) |
| `app/src/main/kotlin/.../ui/screens/ChatScreen.kt` | Chat UI — streaming messages, slash commands, tool cards |
| `app/src/main/kotlin/.../ui/screens/SettingsScreen.kt` | Settings — connection, chat, appearance, analytics, about |
| `app/src/main/kotlin/.../ui/components/StatsForNerds.kt` | Canvas bar charts for analytics display |
| `app/src/main/kotlin/.../ui/components/CompactToolCall.kt` | Inline compact tool call display |
| `app/src/main/kotlin/.../ui/components/PersonalityPicker.kt` | Personality picker dropdown (from config.agent.personalities) |
| `app/src/main/kotlin/.../ui/components/CommandPalette.kt` | Searchable command palette (bottom sheet) + inline autocomplete |
| `app/src/main/kotlin/.../ui/components/ConnectionStatusBadge.kt` | Animated pulse ring status indicator (connected/connecting/disconnected) |
| `app/src/main/kotlin/.../ui/components/MorphingSphere.kt` | ASCII morphing sphere — 3D lit character sphere with color pulse, used in empty chat state, ambient mode, and behind-messages background |
| `app/src/main/kotlin/.../ui/components/MessageBubble.kt` | Message bubbles with markdown, tokens, tool cards |
| `app/src/main/kotlin/.../ui/components/ToolProgressCard.kt` | Expandable tool execution card (auto-expand/collapse) |
| `app/src/main/kotlin/.../viewmodel/ChatViewModel.kt` | Chat orchestration — send, stream, cancel, slash commands |
| `app/src/main/kotlin/.../viewmodel/ConnectionViewModel.kt` | Dual connection model (API + relay) |
| `app/src/main/res/drawable/splash_icon.xml` | Splash screen icon (0.9x scale) |
| `app/src/main/res/drawable/splash_icon_animated.xml` | Animated splash (scale + overshoot + fade) |
| `relay_server/relay.py` | Relay server — main WSS server (bridge/terminal only) |
| `relay_server/SKILL.md` | Hermes skill reference for relay self-setup |
| `relay_server/Dockerfile` | Container image for relay server |
| `relay_server/hermes-relay.service` | Systemd unit file for persistent deployment |
| `docs/relay-server.md` | Relay server setup, config, Docker, systemd, TLS reference |
| `app/src/main/kotlin/.../ui/components/QrPairingScanner.kt` | QR code scanner + Hermes pairing payload parser |
| `plugin/pair.py` | QR pairing logic (pure-Python, uses segno) — replaces deprecated bash script |
| `plugin/cli.py` | Registers `hermes pair` CLI sub-command via v0.8.0 plugin CLI API |
| `skills/hermes-pairing-qr/SKILL.md` | (DEPRECATED) QR pairing skill — use `hermes pair` from the plugin instead |
| `skills/hermes-pairing-qr/hermes-pair` | (DEPRECATED) QR generator script — use `hermes pair` from the plugin instead |
| `AGENTS.md` | Tool usage patterns for the `android_*` toolset |
| `docs/mcp-tooling.md` | MCP server setup — android-tools-mcp + mobile-mcp |
| **App — Core** | |
| `ui/RelayApp.kt` | Main scaffold — bottom nav, Compose navigation |
| `viewmodel/ChatViewModel.kt` | Chat orchestration — send, stream, cancel, slash commands |
| `viewmodel/ConnectionViewModel.kt` | Dual connection model (API + relay); `resolveStreamingEndpoint()` |
| `network/HermesApiClient.kt` | Direct HTTP/SSE — `sendRunStream()`, `sendChatStream()`, `probeCapabilities()` |
| `network/ConnectionManager.kt` | WSS to relay with auto-reconnect; rebuilds OkHttpClient with fresh CertPinner on connect |
| `network/ChannelMultiplexer.kt` | Envelope routing by channel; `sendNotification()` for notification outbound |
| `network/handlers/ChatHandler.kt` | Chat message state, streaming events, tool annotation parser |
| `network/models/SessionModels.kt` | Session, message, SSE event data models |
| `data/FeatureFlags.kt` | Feature gating — DEV_MODE + DataStore overrides; `BuildFlavor` (googlePlay/sideload Tier flags) |
| **App — Auth** | |
| `auth/AuthManager.kt` | Wires SessionTokenStore + CertPinStore; parses auth.ok; `applyServerIssuedCodeAndReset()` |
| `auth/SessionTokenStore.kt` | Keystore (StrongBox) + EncryptedSharedPrefs fallback; lossless migration on upgrade |
| `auth/CertPinStore.kt` | TOFU cert pinning — SHA-256 SPKI per host:port in DataStore |
| `auth/PairedSession.kt` | PairedSession state + PairedDeviceInfo wire model |
| `network/RelayHttpClient.kt` | OkHttp for /media, /sessions (list/revoke/extend), /health |
| **App — Bridge** | |
| `network/handlers/BridgeCommandHandler.kt` | Routes `bridge.command` → ActionExecutor; full path inventory + safety-rail integration |
| `viewmodel/BridgeViewModel.kt` | BridgeScreen VM — masterToggle, bridgeStatus, permissionStatus, activityLog |
| `bridge/BridgeSafetyManager.kt` | Blocklist + destructive-verb confirmation + auto-disable timer; fails-closed on /call and /send_sms |
| `data/BridgeSafetyPreferences.kt` | DataStore for blocklist, destructive verbs, auto-disable minutes, confirmation timeout |
| `ui/screens/BridgeScreen.kt` | Bridge UI — master toggle, status, permission checklist, safety summary, activity log |
| `bridge/BridgeStatusOverlay.kt` | WindowManager overlay; `ConfirmationOverlayHost`; requires `SavedStateRegistryOwner` init order (CREATED→restore→RESUMED) |
| `accessibility/HermesAccessibilityService.kt` | AccessibilityService subclass; `@Volatile instance` singleton for BridgeCommandHandler |
| `accessibility/ScreenReader.kt` | UI tree → ScreenContent; `findNodeBoundsByText()`, `findFocusedInput()` |
| `accessibility/ActionExecutor.kt` | Gesture/text dispatch via GestureDescription + ACTION_SET_TEXT; pressKey maps vocab only |
| **App — Voice** | |
| `voice/VoiceViewModel.kt` | Voice turn state machine; TTS queue; `ignoreAssistantId`; `errorEvents: SharedFlow` |
| `audio/VoiceRecorder.kt` | MediaRecorder wrapper; perceptual amplitude curve; `.m4a` at 16kHz/64kbps |
| `audio/VoicePlayer.kt` | MediaPlayer + Visualizer; amplitude StateFlow; `awaitCompletion()` via coroutine |
| `network/RelayVoiceClient.kt` | OkHttp for `/voice/transcribe`, `/synthesize`, `/config` |
| `voice/VoiceBridgeIntentHandler.kt` | Interface routing voice utterances to bridge; impls per flavor via factory |
| `voice/VoiceIntentClassifier.kt` | Regex phone-control classifier (sideload only); false-negatives preferred over false-positives |
| `ui/components/VoiceModeOverlay.kt` | Full-screen voice UI — MorphingSphere + VoiceWaveform + mic button |
| **App — Media + Notifications** | |
| `util/MediaCacheWriter.kt` | `cacheDir/hermes-media/` LRU writer; returns FileProvider URIs |
| `ui/components/InboundAttachmentCard.kt` | Discord-style attachment card for images/video/audio/pdf/text/generic |
| `notifications/HermesNotificationCompanion.kt` | NotificationListenerService; cold-start buffer (50); forwards via ChannelMultiplexer |
| `util/RelayErrorClassifier.kt` | `classifyError(Throwable, context) → HumanError`; used by Voice/Chat/Connection |
| **Relay — Server** | |
| `plugin/relay/server.py` | Canonical relay — WSS + HTTP routes; bridge, media, voice, session, pairing handlers |
| `plugin/relay/auth.py` | PairingManager, SessionManager, RateLimiter; `math.inf` for never-expire |
| `plugin/relay/channels/bridge.py` | Bridge handler — `handle_command()` mints request_id, awaits response, 30s timeout |
| `plugin/relay/channels/notifications.py` | Bounded deque (100) of notification metadata; in-memory only |
| `plugin/relay/media.py` | MediaRegistry — LRU token store; `strict_sandbox` off by default for `/media/by-path` |
| `plugin/relay/voice.py` | Voice endpoints — transcribe, synthesize, voice_config; lazy tool imports |
| `plugin/relay/qr_sign.py` | HMAC-SHA256 QR signing; secret at `~/.hermes/hermes-relay-qr-secret` |
| `plugin/relay/_env_bootstrap.py` | Loads `~/.hermes/.env` before relay imports; called from both entry points |
| **Plugin — Tools + Installer** | |
| `plugin/tools/android_tool.py` | 18 `android_*` tool handlers (14 baseline + send_sms, call, search_contacts, return_to_hermes); `android_screenshot` first consumer of `register_media()` |
| `plugin/tools/android_navigate.py` | Vision-driven navigation loop; up to 20 iterations; `llm_gap` error until vision client wired |
| `plugin/pair.py` | QR payload builder + CLI; `build_payload(sign=True)`; `--register-code` fallback |
| `install.sh` | Canonical installer — 6 steps; idempotent; drops `hermes-relay-update` shim |
| `uninstall.sh` | Canonical uninstaller; reverses install.sh; never touches `.env` or `state.db` |
| `hermes_relay_bootstrap/` | Runtime patch for vanilla upstream; no-op on fork/upstream-merged; remove after PR #8556 |
## What NOT to Do
- **Don't use XML layouts** — Compose only
- **Don't use Gson** — kotlinx.serialization
- **Don't use Ktor for networking** — OkHttp for WebSocket
- **Don't build terminal or bridge channels yet** — Phase 2 and 3. Stubbed with `TODO`.
- **Don't use plaintext WebSocket** — `wss://` only, even in development
- **Don't put documentation in root** — long-form docs go in `docs/`
- **Don't forget DEVLOG.md** — update it
@@ -179,8 +207,6 @@ Two MCP servers are configured for AI-assisted development. See `docs/mcp-toolin
| `android-tools-mcp` | IDE/Build — Compose previews, Gradle, code search, Android docs | Android Studio running with project open |
| `mobile-mcp` | Device/Runtime — tap, swipe, screenshot, app management | ADB + connected device/emulator |
Together they cover the full loop: code → preview → build → deploy → interact → screenshot.
## Dev Workflow
```bash
@@ -193,39 +219,86 @@ scripts/dev.bat version # Show current version from libs.versions.toml
scripts/dev.bat relay # Start relay server (dev mode, no SSL)
```
Open repo root in Android Studio for Compose previews and device deployment.
### Bridge smoke test (run on hermes-host, not local PC)
```bash
scripts/bridge-smoke.sh # full suite, destructive ON
scripts/bridge-smoke.sh --no-destructive # read-only paths only
scripts/bridge-smoke.sh --filter open_app # re-run a single test
scripts/bridge-smoke.sh --pair ABCDEF # register pairing code first
```
Curls every bridge HTTP route via `localhost:8767`. Catches the silent-drop regression class (Python relay registers a route but Kotlin dispatcher's `when (path)` has no matching branch). Run after every relay restart.
### Typical Dev Loop
1. **Edit locally** — Windows checkout. Both plugin (`plugin/`) and app (`app/`) live here.
2. **Python syntax check** — `python -m py_compile plugin/<file>.py`. Full tests run on the server.
3. **Kotlin changes** — do NOT run `gradle build`. Bailey builds via Android Studio's ▶ button. Never `adb install` from Claude.
4. **Commit + push** — feature branch for anything >1-2 commits.
5. **Pull + restart on server** — see Server Deployment below.
6. **Test on phone** — Bailey builds from Studio, installs to Samsung device, pairs via `/hermes-relay-pair`.
### Server Deployment
Server is a Linux box running hermes-agent with hermes-relay editable-installed (`pip install -e`). Sensitive details (IP, user, secrets) in `~/SYSTEM.md` on the server — not in this repo.
| What | Where |
|---|---|
| hermes-agent repo | `~/.hermes/hermes-agent/` |
| hermes-relay clone | `~/.hermes/hermes-relay/` |
| Plugin symlink | `~/.hermes/plugins/hermes-relay` → `~/.hermes/hermes-relay/plugin` |
| Config | `~/.hermes/config.yaml` + `~/.hermes/.env` |
| Relay log | `journalctl --user -u hermes-relay -f` |
**Update:** `hermes-relay-update` (idempotent, re-fetches install.sh). Or manually: `git pull --ff-only && systemctl --user restart hermes-relay`.
**Key conventions:**
- Phone re-pairs after each relay restart (SessionManager is in-memory; wiped on restart)
- Use `python -m unittest` not `pytest` — conftest imports `responses` which may not be installed
- `_env_bootstrap.py` loads `~/.hermes/.env` on every relay start — no stale API keys
### Where Python vs. Kotlin changes land
| Change type | Who restarts? | Command |
|---|---|---|
| Plugin tool (`android_tool.py` etc.) | `hermes-gateway.service` | `systemctl --user restart hermes-gateway` |
| Relay code (`plugin/relay/*.py`) | `hermes-relay.service` | `systemctl --user restart hermes-relay` |
| Pair CLI / skill files | — | No restart — fresh process / scanned on invocation |
| Android app | Bailey (Studio) | Studio run button |
### Release Process
See [RELEASE.md](RELEASE.md) for the full release recipe — versioning conventions, keystore setup, Play Console upload (manual + automated via `gradle-play-publisher`), GitHub release workflow, and troubleshooting.
See [RELEASE.md](RELEASE.md) for the full recipe.
Quick reference:
- **Version source of truth**: `gradle/libs.versions.toml` (`appVersionName`, `appVersionCode`)
- **SemVer with optional prereleases**: `v0.1.0`, `v0.1.1`, `v0.2.0-beta.1`, `v1.0.0-rc.1`
- **`appVersionCode` is monotonic** — always increment, even across prereleases (Play Console rejects collisions)
- **Build AAB locally**: `scripts/dev.bat bundle` → `app/build/outputs/bundle/release/app-release.aab`
- **Verify signing**: `keytool -list -printcert -jarfile <aab>` — must NOT show `CN=Android Debug`
- **Cut a release**: bump version → commit → `git tag vMAJOR.MINOR.PATCH` → `git push origin <tag>` → CI builds APK + AAB and creates GitHub Release
- **Required GitHub Secrets** for signed CI builds: `HERMES_KEYSTORE_BASE64`, `HERMES_KEYSTORE_PASSWORD`, `HERMES_KEY_ALIAS`, `HERMES_KEY_PASSWORD` (without these the workflow still runs but produces debug-signed artifacts that Play Console will reject)
- **Optional automated upload**: `gradlew publishReleaseBundle --track=internal` (requires `play-service-account.json` at repo root, gitignored)
- **Version source:** `gradle/libs.versions.toml` (`appVersionName`, `appVersionCode`)
- **Bump atomically:** `bash scripts/bump-version.sh <new-version>` — updates all three sources
- **`appVersionCode` is monotonic** — always increment across prereleases
- **Cut a release:** bump → commit → `git tag vMAJOR.MINOR.PATCH` → push tag → CI builds + GitHub Release
- **Required secrets:** `HERMES_KEYSTORE_BASE64`, `HERMES_KEYSTORE_PASSWORD`, `HERMES_KEY_ALIAS`, `HERMES_KEY_PASSWORD`
## Integration Points
| Surface | Standard Endpoint | Non-Standard Fallback |
|---------|-------------------|----------------------|
| Chat streaming | `POST /v1/runs` → `GET /v1/runs/{id}/events` (structured tool events) | `POST /api/sessions/{id}/chat/stream` (inline tool text) |
| Chat (OpenAI compat) | `POST /v1/chat/completions` (stream=true) | — |
| Session CRUD | `X-Hermes-Session-Id` header on `/v1/chat/completions` | `GET/POST/PATCH/DELETE /api/sessions` (non-standard) |
| Personalities | Read from `~/.hermes/config.yaml` | `GET /api/config` (non-standard) |
| Server skills | — | `GET /api/skills` (non-standard) |
| Health check | `GET /health` or `GET /v1/health` | — |
| Models | `GET /v1/models` | — |
| Plugin tools | `android_*` via `plugin/` | — |
| Surface | Endpoint | Notes |
|---------|----------|-------|
| Chat streaming | `POST /v1/runs` → `GET /v1/runs/{id}/events` | Structured tool events; preferred |
| Chat (sessions) | `POST /api/sessions/{id}/chat/stream` | No live tool events; reloads history on stream complete |
| Chat (compat) | `POST /v1/chat/completions` (stream=true) | Inline tool annotations only |
| Session CRUD | `GET/POST/PATCH/DELETE /api/sessions` | Non-standard; bootstrap or fork |
| Pairing (QR) | `POST /pairing/register` (loopback only) | Via `/hermes-relay-pair` or `hermes-pair` shim |
| Pairing auth | WSS `auth.ok` payload | Includes `expires_at`, `grants`, `transport_hint` |
| Inbound media (token) | `GET /media/{token}` | Bearer auth; 24h TTL |
| Inbound media (path) | `GET /media/by-path?path=<abs>` | Permissive by default; `RELAY_MEDIA_STRICT_SANDBOX=1` to restrict |
| Session management | `GET /sessions`, `DELETE /sessions/{prefix}`, `PATCH /sessions/{prefix}` | List/revoke/extend; RelayHttpClient |
| Voice transcribe | `POST /voice/transcribe` | multipart/form-data; bearer auth |
| Voice synthesize | `POST /voice/synthesize` | JSON → audio/mpeg; max 5000 chars |
| Voice config | `GET /voice/config` | Returns current tts/stt provider info |
| Notifications | `GET /notifications/recent?limit=N` | Loopback callers skip bearer |
| Relay health | `GET /health` on `:8767` | Used by `RelayHttpClient.probeHealth()` |
| Capabilities | `HEAD /api/sessions`, `HEAD /v1/runs`, etc. | HEAD avoids CORS 403 on OPTIONS preflight |
## Upstream References
When working on features that interface with hermes-agent, consult these source files directly:
| Topic | Upstream File |
|-------|--------------|
| API endpoints | `gateway/platforms/api_server.py` — all registered HTTP routes |
+109
View File
@@ -0,0 +1,109 @@
# Contributing to Hermes-Relay
Thanks for your interest in contributing! Hermes-Relay is an indie, open-source project and every contribution — code, bug reports, docs tweaks, feature ideas — genuinely shapes where it goes next.
This guide covers the developer setup. For the release recipe see [RELEASE.md](RELEASE.md); for architecture context see [docs/spec.md](docs/spec.md) and [docs/decisions.md](docs/decisions.md).
## Quick Start (Android)
1. **File > Open** the repo root in Android Studio
2. Wait for Gradle sync
3. **Run** (Shift+F10) to deploy to emulator or device
That's it — no extra setup or credentials required for a debug build.
## Dev Scripts
Helper scripts for common development tasks:
```bash
scripts/dev.bat build # Build debug APK
scripts/dev.bat release # Build signed release APK
scripts/dev.bat bundle # Build release AAB for Google Play
scripts/dev.bat run # Build + install + launch + logcat
scripts/dev.bat test # Run unit tests
scripts/dev.bat version # Show current version
scripts/dev.bat relay # Start relay server (dev, no TLS)
```
Linux/macOS equivalent lives at `scripts/dev.sh`.
## Repository Structure
```
hermes-relay/
├── app/ # Android app (Kotlin + Jetpack Compose)
├── plugin/ # Hermes agent plugin + relay server (Python + aiohttp)
│ ├── relay/ # Canonical relay server (channels, auth, media, voice)
│ ├── tools/ # android_* tool implementations
│ └── pair.py # QR pairing CLI
├── skills/ # Hermes agent skills (pair, self-setup)
├── user-docs/ # VitePress documentation site
├── docs/ # Spec, architecture decisions, security notes
├── scripts/ # Dev helper scripts
├── .github/workflows/ # CI + release pipelines
└── gradle/ # Wrapper + version catalog
```
The legacy `relay_server/` directory is a thin compatibility shim around `plugin.relay` that keeps the `python -m relay_server` entry point working.
## Tech Stack
| Component | Stack |
|-----------|-------|
| **Android App** | Kotlin 2.0, Jetpack Compose, Material 3, OkHttp |
| **Relay Server** | Python 3.11+, aiohttp |
| **Serialization** | kotlinx.serialization |
| **Build** | AGP 9, Gradle 8.13, JVM toolchain 17 |
| **CI/CD** | GitHub Actions (lint, build, test, signed APK artifacts) |
| **Min SDK** | 26 (Android 8.0) / Target SDK 35 |
## Running the Relay Locally
Only needed if you're working on the bridge, voice, notifications, or media features. Chat alone doesn't need the relay.
```bash
# From the hermes-agent venv (if you installed via the one-liner):
hermes relay start --no-ssl
# Or from a repo checkout:
python -m plugin.relay --no-ssl
```
See [docs/relay-server.md](docs/relay-server.md) for TLS, systemd, Docker, and full configuration.
## Plugin Development
End users should install via the one-liner in the README. For local development from a clone:
```bash
# One-shot copy:
cp -r plugin ~/.hermes/plugins/hermes-relay
# Or symlink for live edits:
ln -s "$PWD/plugin" ~/.hermes/plugins/hermes-relay
```
After the plugin is in place, restart hermes and verify pairing with `hermes-pair` (shell shim) or `/hermes-relay-pair` in any Hermes chat surface. The 18 `android_*` tools register regardless of hermes-agent version.
> **Note:** A top-level `hermes pair` CLI sub-command is not currently exposed — hermes-agent v0.8.0's top-level argparser doesn't yet forward to third-party plugins' `register_cli_command()` dict. Use the slash command or the dashed shim instead.
## Commit Conventions
We follow [Conventional Commits](https://www.conventionalcommits.org/): `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`.
Feature branches are the house style — `feature/<name>`, `fix/<name>`, `docs/<name>`, `chore/<name>` — merged into `main` via `--no-ff` merge commits so the per-branch history stays visible in `git log --graph`. Straight-to-`main` is reserved for single-file typo fixes.
Release-prep commits (version bump + tag) are allowed to push directly to `main` via a branch-protection carve-out — see [RELEASE.md](RELEASE.md) for the full release process.
## Testing
- **Android unit tests:** `scripts/dev.bat test` (runs JUnit + MockK + Compose testing)
- **Python tests:** `python -m unittest plugin.tests.test_<name>` from the repo root with the hermes-agent venv active. `pytest` works too but the pre-existing `conftest.py` imports a module that isn't always installed — `unittest` avoids that entirely.
CI (`.github/workflows/ci.yml`) runs lint, Android build, Android unit tests, and a Python relay syntax check on every push.
## Questions?
- **Architecture context?** [docs/spec.md](docs/spec.md) covers protocols, UI layouts, and the channel model. [docs/decisions.md](docs/decisions.md) covers the forks in the road and why we picked what we did.
- **Something unclear?** [Open an issue](https://github.com/Codename-11/hermes-relay/issues/new) — we read every one, and "this contributing guide is confusing" is a completely fair bug report.
+1665
View File
File diff suppressed because it is too large Load Diff
+82 -27
View File
@@ -36,11 +36,22 @@ Two steps: install the Android app on your phone, then install the plugin on you
### 1. Install the Android app
<!-- TODO: Uncomment when Play Store listing is live
<a href="https://play.google.com/store/apps/details?id=com.hermesandroid.relay"><img src="https://play.google.com/intl/en_us/badges/static/images/badges/en_badge_web_generic.png" alt="Get it on Google Play" height="80"></a>
<a href="https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay"><img src="https://play.google.com/intl/en_us/badges/static/images/badges/en_badge_web_generic.png" alt="Get it on Google Play" height="80"></a>
-->
- **Google Play** — coming soon
- **APK** — download from [GitHub Releases](https://github.com/Codename-11/hermes-relay/releases)
- **Google Play** — coming soon (currently on Internal testing)
- **APK** — download from [GitHub Releases](https://github.com/Codename-11/hermes-relay/releases/latest)
#### Sideload APK (GitHub Releases)
Prefer not to wait for Google Play? Grab the signed APK directly:
1. Download the file ending in **`-sideload-release.apk`** from [the latest release](https://github.com/Codename-11/hermes-relay/releases/latest) — that's the full-featured "Hermes Dev" build. (Skip any `.aab` file — those are the Google Play bundle format and won't install directly.)
2. On your phone: **Settings → Apps → Special app access → Install unknown apps** and allow your browser (first time only).
3. Open the APK from your downloads and tap **Install**.
4. Optionally verify integrity against `SHA256SUMS.txt` from the same release (`sha256sum` on macOS/Linux, `Get-FileHash -Algorithm SHA256` on Windows).
Full walkthrough, including signing-certificate fingerprint: [Sideload guide](https://codename-11.github.io/hermes-relay/guide/getting-started.html#sideload-apk).
### 2. Install the server plugin (one-liner)
@@ -50,15 +61,40 @@ On the machine running your Hermes agent:
curl -fsSL https://raw.githubusercontent.com/Codename-11/hermes-relay/main/install.sh | bash
```
This installs the `hermes-android` plugin — 14 `android_*` device control tools plus the `hermes pair` CLI command. After restarting hermes, generate a pairing QR code:
The installer clones Hermes-Relay to `~/.hermes/hermes-relay/` (override with `$HERMES_RELAY_HOME`), `pip install -e`s the package into the hermes-agent venv, registers the `skills/` directory in your `~/.hermes/config.yaml` under `skills.external_dirs` (so updates flow through `git pull`), symlinks the plugin into `~/.hermes/plugins/hermes-relay`, drops a thin `hermes-pair` shim into `~/.local/bin/`, and (optionally) installs a systemd user service for the WSS relay. After restart, pair your phone via either of these equivalent entry points:
```bash
hermes pair
- **From any Hermes chat surface** (CLI, Discord, Telegram, etc.): type `/hermes-relay-pair` and the `hermes-relay-pair` skill renders the QR inline. Shortest path if you're already chatting with the agent.
- **From a shell**: `hermes-pair` (dashed) — a thin wrapper around `python -m plugin.pair` in the hermes-agent venv. Use this in scripts or when you want the raw output.
- **No camera?** `hermes-pair --register-code ABCD12` — manual fallback for SSH-only / camera-less setups. Read the 6-char code from the app's **Settings → Connection → Manual pairing code (fallback)** card, pre-register it on the host with this command, then tap **Connect** in the app. Composes with `--ttl` / `--grants`.
Scan the QR from the Android app's onboarding screen and you're connected. One scan configures **both** the direct-chat API server **and** the WSS relay (for terminal/bridge) — if a local relay is running at `localhost:8767`, the pair command pre-registers a fresh 6-char pairing code with it and embeds the relay URL + code in the same QR. If you only want direct chat, pass `--no-relay` (or just don't start the relay). Plain-text connection details are always printed alongside the QR so you can copy values by hand if your terminal can't render QR blocks.
**Updating:** `hermes-relay-update` (shortest path — installed as part of the one-liner) or re-run the same `curl … | bash` from above. Both are equivalent and fully idempotent: pulls latest main, refreshes the editable install, recreates all three shims, restarts `hermes-relay`, and prompts before restarting `hermes-gateway`. Set `HERMES_RELAY_RESTART_GATEWAY=1` to opt into the gateway restart non-interactively. For routine plugin/skill updates without restarting anything, a plain `cd ~/.hermes/hermes-relay && git pull` is enough — the editable install picks up the new code on next process start.
**Uninstalling:** `bash ~/.hermes/hermes-relay/uninstall.sh` reverses every install step in the opposite order. Idempotent, never touches state shared with other Hermes tools (`.env`, sessions DB, hermes-agent venv core). Flags: `--dry-run`, `--keep-clone`, `--remove-secret`. Or pull the script via curl if you've already removed the clone.
**Requirements:** Android 8.0+ (SDK 26), [hermes-agent](https://github.com/NousResearch/hermes-agent) v0.8.0+, Python 3.11+.
### For AI Agents
If you have an AI assistant (Claude, GPT, etc.) and want it to install or maintain Hermes-Relay for you, paste the block below into the chat. The agent will fetch the canonical setup recipe from this repo and walk you through it — verification, pairing, troubleshooting included.
```text
You are helping me install and maintain Hermes-Relay (https://github.com/Codename-11/hermes-relay), a native Android client + Python plugin for the Hermes AI agent platform.
Read the canonical setup recipe before acting:
https://raw.githubusercontent.com/Codename-11/hermes-relay/main/skills/devops/hermes-relay-self-setup/SKILL.md
Then guide me through:
- Verifying hermes-agent is already installed (it's a prerequisite — Hermes-Relay is a plugin, not standalone)
- Running the install one-liner: `curl -fsSL https://raw.githubusercontent.com/Codename-11/hermes-relay/main/install.sh | bash`
- Pairing my phone via `hermes-pair` or `/hermes-relay-pair`
- Verifying with `hermes-status`
Always confirm before running shell commands. Never restart hermes-gateway without asking. If any step fails, consult the Troubleshooting section in the SKILL.md and ask me for the exact error.
```
Scan it from the Android app's onboarding screen and you're connected. The command also prints the server URL and API key as plain text, so you can pair manually if your terminal can't render QR blocks.
**Requirements:** Android 8.0+ (SDK 26), [hermes-agent](https://github.com/NousResearch/hermes-agent) v0.8.0+ (for the `hermes pair` CLI), Python 3.11+.
Already have Hermes-Relay installed? The same recipe is auto-loaded as a Hermes skill — invoke it from any chat with `/hermes-relay-self-setup` for re-setup, troubleshooting, or "is everything wired correctly?" checks. Single source, two delivery modes (raw URL pre-install + Hermes skill post-install), no drift.
## What It Does
@@ -67,22 +103,23 @@ Talk to your Hermes agent from anywhere. Direct API streaming, session history,
| Channel | What | Status |
|---------|------|--------|
| **Chat** | Stream conversations to Hermes via HTTP/SSE | Available |
| **Voice** | Real-time voice conversation via relay TTS/STT | Available |
| **Bridge** | Agent reads the screen and performs UI actions (tap, long-press, drag, type, clipboard, media, macros, events) | Available |
| **Terminal** | Secure remote shell via tmux | Phase 2 |
| **Bridge** | Agent controls the phone — taps, types, screenshots | Phase 3 |
## Features
- **Streaming chat** — Direct SSE to the Hermes API Server with real-time markdown rendering
- **Smooth auto-scroll** — Live-follow streaming responses with a "scrolled up to read" pause/resume gesture
- **Session management** — Create, switch, rename, delete chat sessions
- **Tool visualization** — See agent tool calls as they execute (compact or detailed cards)
- **Personalities** — Switch between agent personalities with a picker
- **Slash commands** — 29+ gateway commands, searchable command palette
- **File attachments** — Send images, documents, any file type
- **Message queuing** — Send messages while the agent is still streaming
- **Analytics** — Stats for Nerds with TTFT, token usage, stream health
- **Security** — Encrypted local storage (AES-256-GCM), HTTPS enforced
- **QR pairing** — Scan a QR code to auto-configure your server connection
- **Streaming chat** — Direct SSE to the Hermes API Server with real-time markdown rendering, session history, tool-call visualization, personality picker, searchable command palette (29+ gateway commands), file attachments, and send-while-streaming message queuing
- **Voice mode** — Real-time voice conversation via the relay; the sphere listens with you and performs the agent's reply as it speaks. Uses your server's configured TTS/STT providers (Edge TTS, ElevenLabs, OpenAI, MiniMax, Mistral, NeuTTS / faster-whisper, Groq, OpenAI Whisper)
- **Phone control (bridge)** — The agent can read what's on screen and act on it — tap, long-press, drag, swipe, scroll, type, and press system keys — plus take screenshots, read/write the clipboard, and control system-wide media playback. Gesture reliability is hardened for dim/idle screens, and a smarter tap-fallback cascade handles apps where labels sit inside non-clickable wrappers
- **Screen understanding** — Filtered accessibility-tree search, per-node property lookups with stable IDs, cheap screen-hash change detection, and multi-window reads (system overlays, popups, notification shade) so the agent can reason about UI without guessing
- **Workflow automation** — Batched macro execution for multi-step flows, real-time accessibility event streaming for "wait until something happens" waits, and a raw-Intent escape hatch for apps that expose deep-link actions
- **Notification companion** — Opt-in notification access so the agent can triage, summarize, and route incoming notifications
- **Bridge safety rails** — Per-app blocklist (banking, payments, 2FA default-blocked), destructive-verb confirmation modal (send, pay, delete, transfer…), idle auto-disable timer, optional persistent-status overlay, full activity log
- **Security & pairing** — QR-code pairing, Android Keystore session storage (StrongBox-preferred), TOFU cert pinning, per-channel time-bound grants, user-chosen session TTL
- **Analytics** — Stats for Nerds with TTFT, token usage, stream health, and peak-time charts
> Sideload builds add direct SMS, contact search, one-tap dialing, and location awareness — handy for fully hands-free voice intents like "text Sam I'll be 10 minutes late". See [Release tracks](https://codename-11.github.io/hermes-relay/guide/release-tracks) for the full sideload capability matrix.
## Getting Started
@@ -140,8 +177,10 @@ scripts/dev.bat relay # Start relay server (dev, no TLS)
hermes-relay/
├── app/ # Android app (Kotlin + Jetpack Compose)
├── relay_server/ # WSS relay server (Python + aiohttp)
├── plugin/ # Hermes agent plugin (14 android_* tools)
├── skills/ # Hermes agent skills (QR pairing)
├── plugin/ # Hermes agent plugin (18 android_* tools + pair module)
├── skills/ # Hermes agent skills
│ └── devops/
│ └── hermes-relay-pair/ # /hermes-relay-pair slash-command skill
├── user-docs/ # VitePress documentation site
├── docs/ # Spec, decisions, security
├── scripts/ # Dev helper scripts
@@ -163,7 +202,9 @@ hermes-relay/
### Relay Server (optional — terminal/bridge only)
```bash
pip install aiohttp pyyaml && python -m relay_server --no-ssl
hermes relay start --no-ssl # if you installed the plugin
# or from a repo checkout:
python -m plugin.relay --no-ssl
```
Or with Docker:
@@ -179,17 +220,31 @@ See [docs/relay-server.md](docs/relay-server.md) for TLS, systemd, and full setu
End users should install via the [one-liner](#2-install-the-server-plugin-one-liner) at the top. For local development from a clone:
```bash
cp -r plugin ~/.hermes/plugins/hermes-android
cp -r plugin ~/.hermes/plugins/hermes-relay
# Or symlink for live edits:
ln -s "$PWD/plugin" ~/.hermes/plugins/hermes-android
ln -s "$PWD/plugin" ~/.hermes/plugins/hermes-relay
```
Then restart hermes and run `hermes pair` to test the CLI command. The 14 `android_*` tools register regardless of hermes-agent version; the `hermes pair` CLI requires v0.8.0+.
Then restart hermes and run `hermes-pair` (dashed shell shim) or type `/hermes-relay-pair` in any Hermes chat surface to verify pairing. The 14 `android_*` tools register regardless of hermes-agent version. **Note:** a top-level `hermes pair` CLI sub-command is *not* currently exposed — hermes-agent v0.8.0's top-level argparser doesn't yet forward to third-party plugins' `register_cli_command()` dict. Use the slash command or the dashed shim instead.
## Hermes Agent
Hermes-Relay is built for [Hermes Agent](https://github.com/NousResearch/hermes-agent) — an open-source AI agent platform by [Nous Research](https://nousresearch.com). See the [Hermes Agent docs](https://hermes-agent.nousresearch.com) for server setup, gateway configuration, and plugin development.
## Found a bug? Let us know!
This is an indie project and every report helps shape where it goes next. If something feels off, broken, or just weird — [open an issue](https://github.com/Codename-11/hermes-relay/issues/new). We read every one, and even a one-line "this didn't work on my Pixel 7" is genuinely useful.
## Star History
<a href="https://www.star-history.com/?repos=Codename-11%2Fhermes-relay&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=Codename-11/hermes-relay&type=date&theme=dark&legend=top-left" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=Codename-11/hermes-relay&type=date&legend=top-left" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=Codename-11/hermes-relay&type=date&legend=top-left" />
</picture>
</a>
## License
[MIT](LICENSE) — Copyright (c) 2026 [Axiom-Labs](https://codename-11.dev)
+173 -26
View File
@@ -44,6 +44,89 @@ Never decrement `appVersionCode` — Play Console rejects any upload whose
code is lower than or equal to a previous upload on the same track. Confirm
current values with `scripts\dev.bat version`.
### The three version sources (MUST stay in lockstep)
There are **three** places the version lives, and they MUST all match on
every release commit. Drift is silent and painful — we chased a "why does
/health say 0.2.0" bug for hours on 2026-04-12 because `pyproject.toml`
had drifted to `0.5.0` speculatively and `plugin/relay/__init__.py` was
still at a stale `0.2.0`.
| File | Line | Written by |
|---|---|---|
| `gradle/libs.versions.toml` | `appVersionName = "…"` | You (canonical) |
| `pyproject.toml` | `version = "…"` | You (Python package) |
| `plugin/relay/__init__.py` | `__version__ = "…"` | You (runtime, reported by `/health`) |
**Always bump them atomically via `scripts/bump-version.sh`**:
```bash
bash scripts/bump-version.sh 0.3.0
```
The script validates SemVer, bumps `appVersionCode` monotonically, rewrites
all three files, runs a post-bump sanity grep, prints the diff, and tells
you the next steps. It deliberately does NOT commit, tag, or touch
`CHANGELOG.md` / `RELEASE_NOTES.md` — those need human prose.
## Branching policy
Hermes-Relay uses **feature branches + no-ff merges**, with version bumps
gated to release-prep commits on `main`. `main` is always "last release +
unreleased features," never mid-refactor.
### Branch names
| Prefix | When | Example |
|---|---|---|
| `feature/<name>` | New feature (>1-2 commits) | `feature/bridge-scroll-tool` |
| `fix/<name>` | Focused bug fix | `fix/media-projection-fgs` |
| `docs/<name>` | Docs-only changes larger than a typo | `docs/sideload-guide` |
| `chore/<name>` | Cleanup / refactor / tooling | `chore/sync-version-sources` |
Straight-to-main is still OK for **single-file typo fixes** and **tiny
one-liner tweaks**. Judgment call — if in doubt, branch.
### Merge style: `--no-ff`
Always merge with `git merge --no-ff <branch>` (or the "Create a merge
commit" option in the GitHub PR UI). This preserves the branch context
as a visible merge commit in `git log --graph`, which is valuable when:
- An agent team pushed several commits to a branch — the per-commit trail
is useful for "which agent did what"
- `git bisect` needs to treat the whole branch as one unit
- Someone reviews history in 6 months and wants to know "what was the
bundle of changes that introduced feature X"
Squash merges lose that detail and are **not** the house style.
### Version bumps happen at release-prep, NOT on feature branches
Feature branches **never** touch `gradle/libs.versions.toml`,
`pyproject.toml`, or `plugin/relay/__init__.py`. If two feature branches
both bumped the version, they'd collide on `appVersionCode` (which must
be monotonic) and you'd hit a merge conflict for no good reason.
The version-bump commit lives on `main`, created via
`scripts/bump-version.sh`, immediately before `git tag`. It's a dedicated
commit with the message `release: vX.Y.Z` that also lands the CHANGELOG
and RELEASE_NOTES updates.
### Branch protection on `main`
Light branch protection is enabled on `main` to enforce the above:
- Direct pushes blocked (must go through PR)
- PR must pass CI (build + unit tests) before merge
- Force push and branch deletion blocked
- Signed commits + review approval NOT required (solo-dev overhead)
Release-prep commits (`release: vX.Y.Z`) are an exception — they're the
one time direct push is allowed via a short-lived bypass, because they
include the version bump + tag push in one transaction. Everything else
goes through a PR.
## One-time Setup
### 1. Release signing keystore
@@ -100,17 +183,40 @@ file afterward.
### 2. Google Play Console developer account
Hermes-Relay ships under the **Axiom-Labs, LLC** Play Console account
(D-U-N-S verified organization). The applicationId is
`com.axiomlabs.hermesrelay` (googlePlay flavor) and
`com.axiomlabs.hermesrelay.sideload` (sideload flavor — not shipped through
Play at all). The Kotlin namespace / source tree stays at
`com.hermesandroid.relay` for historical reasons; see `app/build.gradle.kts`
for the decoupling rationale.
If you're setting up a fresh account (for a fork or a new downstream):
1. Register at <https://play.google.com/console/signup> ($25 one-time fee).
2. Complete identity verification (personal accounts need a government ID;
organization accounts need a D-U-N-S number).
3. Create the app listing: name, language, free/paid, declarations.
**New personal accounts only**: Google requires an app to run in
**closed testing** with **at least 12 opted-in testers** for **14
continuous days** before it can be promoted to production. Internal testing
does NOT satisfy this requirement — only the closed testing track starts
the 14-day clock. Organization (D-U-N-S) accounts are exempt. See
[Google's policy](https://support.google.com/googleplay/android-developer/answer/14151465).
**The 14-day closed-testing rule does NOT apply to Hermes-Relay.** Google
requires *new personal* developer accounts to run an app in closed testing
with ≥12 opted-in testers for 14 continuous days before promotion to
production. Organization accounts with a verified D-U-N-S number are exempt
from this policy, and Axiom-Labs is a D-U-N-S-verified org account. See
[Google's policy](https://support.google.com/googleplay/android-developer/answer/14151465)
for the full text.
> **Historical note (2026-04-13 migration):** v0.1.x through v0.3.0 shipped
> on Internal testing under Bailey's personal Play Console account with
> applicationId `com.hermesandroid.relay`. That listing was retired as part
> of the org-account migration. Play Store package names are permanently
> reserved once used — `com.hermesandroid.relay` can never be reclaimed —
> so all releases from v0.3.1 onwards ship fresh under the new
> `com.axiomlabs.hermesrelay` listing. The upload keystore identity is
> unchanged (same `CN=Bailey Dixon, Codename-11` cert, same SHA256
> fingerprint), so existing GitHub Secrets and the CI signing flow need no
> changes. Google Play App Signing mints a new server-side app signing key
> per listing — that's invisible to us since App Signing is enabled.
### 3. Play Developer API service account (optional)
@@ -142,33 +248,43 @@ artifacts will not be accepted by Play Console.
## Release Process
### 1. Bump the version
### 1. Bump the version (atomic across all three sources)
Edit `gradle/libs.versions.toml`:
Use `scripts/bump-version.sh` — it rewrites `libs.versions.toml`,
`pyproject.toml`, AND `plugin/relay/__init__.py::__version__` in lockstep,
increments `appVersionCode` monotonically, and runs a sanity check. Don't
edit the files by hand; drift is silent and painful.
```toml
[versions]
appVersionName = "0.1.1" # bump per SemVer
appVersionCode = "2" # ALWAYS increment, even for prereleases
```bash
bash scripts/bump-version.sh 0.3.0
```
Confirm:
Confirm the bump:
```bat
scripts\dev.bat version
```
The script's diff output should show exactly three files changed and all
three carrying the new version string.
### 2. Update release notes and changelog
- `RELEASE_NOTES.md` — body of the GitHub Release for this version
(rewritten each release; the workflow uses this as-is).
(rewritten each release; the workflow uses this as-is). Keep the
**Download** section near the top — it should spell out which file
to grab by its `-sideload-release.apk` / `-googlePlay-release.aab`
suffix (every artifact is version-tagged as
`hermes-relay-<version>-<flavor>-<buildType>` via `archivesName`
in `app/build.gradle.kts`) and link to the sideload guide.
The v0.3.0 body is a good template.
- `CHANGELOG.md` — cumulative history; append a new section.
### 3. Build and verify locally
```bat
scripts\dev.bat bundle
keytool -list -printcert -jarfile app\build\outputs\bundle\release\app-release.aab
keytool -list -printcert -jarfile app\build\outputs\bundle\googlePlayRelease\hermes-relay-*-googlePlay-release.aab
```
The `keytool` output must show your release certificate (the CN/OU/O
@@ -176,32 +292,51 @@ values you entered during `keytool -genkey`). If it shows
`CN=Android Debug, O=Android, C=US`, the keystore wasn't picked up —
recheck `local.properties` before continuing.
Product flavors (`googlePlay`, `sideload`) nest outputs under a flavor
directory: APKs live in `app/build/outputs/apk/<flavor>/release/` and
AABs live in `app/build/outputs/bundle/<flavor>Release/`. Every file is
prefixed `hermes-relay-<version>-` via `archivesName` in
`app/build.gradle.kts`.
Optional device smoke test: `scripts\dev.bat release` then
`adb install -r app\build\outputs\apk\release\app-release.apk`.
`adb install -r app\build\outputs\apk\sideload\release\hermes-relay-*-sideload-release.apk`.
### 4. Commit and tag
The release-prep commit is one of the few allowed direct-to-main pushes
(see Branching Policy above — branch protection exempts the
`release: vX.Y.Z` pattern because tagging + bumping must be atomic):
```bash
git add gradle/libs.versions.toml RELEASE_NOTES.md CHANGELOG.md
git commit -m "release: v0.1.1"
git add gradle/libs.versions.toml pyproject.toml plugin/relay/__init__.py \
RELEASE_NOTES.md CHANGELOG.md
git commit -m "release: v0.3.0"
git push origin main
git tag v0.1.1
git push origin v0.1.1
git tag v0.3.0
git push origin v0.3.0
```
Pushing a tag matching `v*` triggers `.github/workflows/release.yml`,
which builds, signs, checksums, and creates a GitHub Release. Watch the
run under the **Actions** tab.
> **Why all three files in the commit?** See "The three version sources"
> above — `bump-version.sh` rewrites them atomically, so they must be
> staged + committed atomically too. Missing one creates the same drift
> the script was built to prevent.
### 5. Upload to Play Console
**Manual upload (default):**
1. Download `app-release.aab` from the GitHub Release assets, or use your
local build at `app\build\outputs\bundle\release\app-release.aab`.
2. In Play Console: **Release > Testing > Internal testing** (or **Closed
testing** for the 14-day clock).
1. Download the file ending in `-googlePlay-release.aab` from the GitHub
Release assets (for example, `hermes-relay-0.3.0-googlePlay-release.aab`),
or use your local build at
`app\build\outputs\bundle\googlePlayRelease\hermes-relay-<version>-googlePlay-release.aab`.
2. In Play Console: **Release > Testing > Internal testing** (the 14-day
closed-testing rule does NOT apply to this account — see "Google Play
Console developer account" above).
3. **Create new release** > upload the AAB.
4. Paste `RELEASE_NOTES.md` into the release notes field.
5. **Review release** > **Start rollout.**
@@ -229,8 +364,9 @@ gradlew promoteReleaseArtifact --from-track=internal --promote-track=alpha
Typical path:
1. **Internal testing** — personal smoke test (no tester or time minimum)
2. **Closed testing (alpha)** — starts the 14-day clock for new personal
accounts; needs at least 12 opted-in testers
2. **Closed testing (alpha)** — optional for staged rollout; Axiom-Labs'
org account is exempt from the 14-day / 12-tester rule, so you can skip
straight from Internal to Production if the build is ready
3. **Open testing (beta)** — optional public beta
4. **Production** — live on the Play Store
@@ -239,6 +375,17 @@ Promote via the Play Console UI or `gradlew promoteReleaseArtifact`.
### 7. After release
- Verify the GitHub Release has APK, AAB, and `SHA256SUMS.txt` attached.
- Confirm the release body includes the **Download** section that tells
users which asset to grab. If you kept the structure from
`RELEASE_NOTES.md` this will already be baked in. If for some reason
it's missing, edit the body with:
```bash
gh release view vX.Y.Z --repo Codename-11/hermes-relay --json body --jq .body > /tmp/body.md
# edit /tmp/body.md to add/fix the Download section
gh release edit vX.Y.Z --repo Codename-11/hermes-relay --notes-file /tmp/body.md
```
(This step was only needed as a retrofit for v0.1.0 — v0.1.1+ inherit
the Download section automatically from `RELEASE_NOTES.md`.)
- Confirm Play Console shows the new versionCode on the target track.
- Update `DEVLOG.md` with a short entry for the release.
+159 -30
View File
@@ -1,41 +1,170 @@
# Hermes-Relay v0.1.0
# Hermes-Relay v0.3.0
First release — a native Android client for the Hermes agent platform with direct API chat, session management, and a full Material 3 Compose UI.
**Release Date:** April 13, 2026
**Since v0.2.0:** 56 commits · 8 major feature merges · 2 new build flavors · 1 new agent-control channel
## Highlights
> **Bridge channel.** The agent can now read your screen, tap, type, swipe, and take screenshots on your phone — gated behind a five-stage safety rails system, a master toggle, the Android Accessibility Service, MediaProjection consent, and per-channel session grants. Plus full voice-mode polish, a notification companion, and two new agent-introspection tools so the agent stops flying blind.
- **Direct API chat** — connects to your Hermes API Server via SSE streaming
- **Session management** — create, switch, rename, delete sessions with full message history
- **Markdown rendering** — code blocks, bold, italic, links, lists in assistant messages
- **Reasoning display** — collapsible thinking blocks when the agent uses extended thinking
- **Token tracking** — per-message input/output token count and estimated cost
- **Personality picker** — dynamic personalities from server config, agent name on chat bubbles
- **Command palette** — searchable command browser with 29 gateway commands + server skills
- **QR code pairing** — scan `hermes-pair` QR to auto-configure connection
- **Material You theming** — dynamic colors with light/dark/auto support
- **File attachments** — attach images, documents, and other files to messages
- **Message queuing** — send follow-up messages while the agent is still responding
- **Offline detection** — graceful degradation when network connectivity is lost
- **Feature gating** — Developer Options (tap version 7x) for experimental features
- **Configurable limits** — adjustable attachment size and message length in Settings
- **In-app analytics** — Stats for Nerds with response times, token usage, peak times, reset
---
## Install
## 📥 Download
Download from Google Play or install the APK from the release assets below.
v0.3.0 ships in **two build flavors**. APK filenames are version-tagged, so every file carries its release number:
## Requirements
| Flavor | File | Who it's for |
|---|---|---|
| **sideload** (recommended) | `hermes-relay-0.3.0-sideload-release.apk` | Full feature set — bridge channel, voice-to-bridge intents, vision-driven `android_navigate`. Installs alongside the Play Store build with a `.sideload` applicationId. |
| **Google Play** | `hermes-relay-0.3.0-googlePlay-release.aab` | Uploaded to Play Console for Internal testing. Conservative feature set (chat, voice, safety rails — no agent device control) to match Play Store's Accessibility policy. |
| googlePlay APK | `hermes-relay-0.3.0-googlePlay-release.apk` | Parity + diff tooling — not the primary download. |
| sideload AAB | `hermes-relay-0.3.0-sideload-release.aab` | Parity + diff tooling — not the primary download. |
- Android 8.0+ (API 26)
- A running [Hermes agent](https://github.com/NousResearch/hermes-agent) instance
**Verify integrity** with `SHA256SUMS.txt` from the same release before installing. See the [Sideload guide](https://codename-11.github.io/hermes-relay/guide/getting-started.html#sideload-apk) for the step-by-step install walkthrough.
## What's Next
> **Why two flavors?** The `googlePlay` build stays inside Play Store's Accessibility Service policy review. The `sideload` build unlocks the full agent-control feature set and installs with a `.sideload` applicationId suffix so both can coexist on the same device — the sideload launcher is labelled **"Hermes Dev"** for disambiguation. For a capability-by-capability breakdown, see the [Release tracks comparison](https://codename-11.github.io/hermes-relay/guide/release-tracks.html).
- Terminal channel via tmux (Phase 2)
- Bridge channel migration (Phase 3)
- Push notifications
- Agent-initiated image rendering (MEDIA: tags)
---
## Feedback
## ✨ Highlights
- Issues: https://github.com/Codename-11/hermes-relay/issues
- **Bridge Channel** — The agent can read the phone's screen, tap, type, swipe, and take screenshots via a new `HermesAccessibilityService` + `MediaProjection` pipeline. Five independent safety gates must all be green before a single command executes: session grant → master toggle → Accessibility permission → MediaProjection consent → safety rails.
- **Voice Mode polish** — Full-screen voice UI with an ASCII morphing sphere (Listening = blue/purple, Speaking = green/teal), reactive layered-sine waveform visualizer, pill-edge merge, tap/hold/continuous interaction modes, and sentence-boundary streaming through a TTS queue. Backed by three new relay endpoints — `POST /voice/transcribe`, `POST /voice/synthesize`, `GET /voice/config` — with 6 TTS and 5 STT providers available via `~/.hermes/config.yaml`.
- **Voice-to-bridge intents** *(sideload only)* — Spoken commands like *"text Mom saying on my way"* route to the bridge channel with destructive-verb confirmation instead of falling through to chat. Regex-based intent classifier covers send-sms / open-app / tap / back / home (scroll removed in v0.4.0 — server-side android_scroll tool handles it instead).
- **Notification Companion** — `HermesNotificationCompanion` (`NotificationListenerService`) forwards posted notifications to the relay over a new `notifications` channel so the agent can summarize them or act on them. Opt-in via the standard Android notification-access grant.
- **Agent introspection tools** — Two new Hermes tools close the "agent has no idea what permissions are granted" loop: `android_phone_status()` returns the full structured phone state (battery, screen, current app, bridge permission flags, safety state) via a new loopback-only `/bridge/status` relay endpoint, and `hermes-status` is a matching CLI shim for operators with three exit codes so shell scripts can tell connected / relay-down / no-phone apart.
- **Per-channel grant revoke** — Paired Devices screen now shows per-channel grant chips with relative TTL labels and an inline x icon. Tap the x → confirm → revoke just that channel. The full-session Revoke button still nukes the whole session.
- **Manual pairing fallback** — For setups without a QR path (phone is the only camera / host is SSH-only): Settings → Connection → **Manual pairing code (fallback)** → copy the 6-char code → run `hermes-pair --register-code ABCD12 [--ttl 30d --grants chat:never,bridge:7d]` on the host → tap **Connect**.
- **Self-install skill for AI agents** — `/hermes-relay-self-setup` is a single-source agent-readable install recipe. Pre-install users paste the raw GitHub URL into any AI; post-install users get it as a slash command. The README has a copy-paste prompt block to hand off setup to Claude / GPT / any agent.
- **Version-tagged release artifacts** — Every APK and AAB in this release is named `hermes-relay-<version>-<flavor>-<buildType>`, so `which-file-is-which` is obvious at a glance and multiple versions don't overwrite each other in your Downloads folder.
---
## 📱 Bridge Channel
The headline feature. Everything in this section is gated behind the five-stage safety system documented above.
### Bridge Tab UI
- **Master toggle card** — "Allow Agent Control" is the load-bearing user-facing gate
- **Bridge status card** — live bridge-connected indicator (reuses `ConnectionStatusBadge`'s pulsing ring)
- **Permission checklist** — Accessibility / Screen Capture / Overlay / Notification Listener rows with in-app **Test** buttons that fire single-command smoke tests instead of requiring the agent to be online
- **Activity log** — tap-to-expand entries with timestamps, status, result text, and optional screenshot tokens (capped at 100 entries)
- **Safety summary card** — live countdown to auto-disable, blocklist/verb counts at a glance
### Safety Rails
- **App blocklist** — 30 default banking / payments / password-manager / 2FA apps pre-seeded; searchable `PackageManager.queryIntentActivities(CATEGORY_LAUNCHER)` picker for custom entries
- **Destructive-verb confirmation modal** — word-boundary regex match against `/tap_text` + `/type` payloads. Default verbs: `send`, `pay`, `delete`, `transfer`, `confirm`, `submit`, `post`, `publish`, `buy`, `purchase`, `charge`, `withdraw`. Modal rendered via a `WindowManager` overlay so it's visible even when Hermes isn't in the foreground.
- **Idle auto-disable** — 5-120 min slider. Any command resets the timer; process death clears state so a stale grant can't survive a crash.
- **Optional persistent status overlay** — small floating "Hermes active" pill via `SYSTEM_ALERT_WINDOW`, gated behind the overlay-permission walk-through
- **Confirmation timeout** — 10-60s slider; fails-closed if missing overlay permission
### AccessibilityService Pipeline
- `HermesAccessibilityService` — `@Volatile` singleton so `BridgeCommandHandler` reaches the live service without DI
- `ScreenReader` — UI tree → `ScreenContent(rootBounds, nodes[], truncated)` with node recycling and `MAX_NODES=512` cap
- `ActionExecutor` — `tap`/`tapText`/`swipe`/`scroll` via `GestureDescription` wrapped in `suspendCancellableCoroutine` so suspend form actually waits for completion; `typeText` via `ACTION_SET_TEXT`; `pressKey` mapped to a curated string vocab (no raw KeyEvent codes)
- `BridgeForegroundService` — persistent "Hermes has device control" notification with **Disable** + **Settings** action buttons; declared as `foregroundServiceType=specialUse|mediaProjection` per Android 14+ requirements
### Relay Bridge Server
- 18 HTTP routes registered on `plugin/relay/server.py` — `/ping`, `/screen`, `/screenshot`, `/get_apps`, `/current_app`, `/tap`, `/tap_text`, `/type`, `/swipe`, `/open_app`, `/press_key`, `/scroll`, `/wait`, `/setup`, `/send_sms`, `/call`, `/search_contacts`, `/return_to_hermes` (last 4 added in v0.4.0)
- Wire protocol migrated from the legacy standalone `plugin/tools/android_relay.py` (port 8766) into the unified relay on port 8767. Envelope fields match the legacy relay byte-for-byte.
- 30s per-command timeout, fail-fast on phone disconnect so HTTP callers don't wedge
---
## 🎙️ Voice Mode
- **Three interaction modes** — Tap-to-talk / Hold-to-talk / Continuous, configurable in Voice Settings
- **Reactive layered-sine waveform visualizer** — three overlapping waves at co-prime frequencies (1.2 / 2.1 / 3.4) with amplitude-driven phase velocity, pill-edge merge via `BlendMode.DstIn` + geometric `sin(π·t)` taper
- **Sphere voice states** — `SphereState.Listening` (soft blue/purple, subtle amplitude wobble) and `SphereState.Speaking` (vivid green/teal, dramatic core-warmth pulse, data ring spin up to 4× on peak)
- **In-flow STT display** — "YOU"-labelled transcribed text lives between the waveform and the response area so the eye flow is one linear motion (mic → up → STT → response), not split top↔bottom
- **Auto-scroll response** — text area follows new tokens via `LaunchedEffect(responseText.length) { scrollState.animateScrollTo(maxValue) }` with a fade-to-transparent gradient mask at top and bottom edges (`graphicsLayer { compositingStrategy = CompositingStrategy.Offscreen }` + `drawWithContent + BlendMode.DstIn`)
- **Stop preserves history** — tapping Stop while the agent is speaking freezes the current response text on screen instead of clearing it; voice mode stays open
- **Voice SFX chimes** — pre-synthesized 200 ms PCM sweeps (ascending 440→660 Hz enter, descending mirror exit) via `AudioTrack.MODE_STATIC` with `USAGE_ASSISTANT`
- **Sentence-boundary TTS queue** — top-level `extractNextSentence(StringBuilder)` helper with whitespace-lookahead for abbreviations (`e.g.`, `i.e.`, `Dr.`, etc.), dedicated consumer coroutine that only triggers auto-resume when the queue is actually drained (waveform stays alive through multi-sentence playback)
- **Attack-fast/release-slow envelope follower** — 0.75 / 0.10 at 60 Hz replaces the old Compose spring so the sphere and waveform respond instantly to speech onsets
- **Voice test toasts** — three-toast lifecycle (Testing / Success / Failed: reason) with explicit `cancel()` between trigger and result so toasts don't stack
---
## 🔔 Notifications & Agent Introspection
- **`HermesNotificationCompanion`** — `NotificationListenerService` subclass with the same opt-in flow as Wear OS / Android Auto / Tasker
- **Cold-start buffer** — `pendingEnvelopes` queue capped at 50 entries preserves ordering when notifications fire before the multiplexer is wired
- **In-memory bounded deque** on the relay side — `NotificationsChannel` holds the most recent 100 entries, wiped on relay restart by design (matches smartwatch-companion semantics)
- **`android_notifications_recent(limit=20)`** — Hermes tool registered by `plugin/tools/android_notifications.py`, calls the loopback `GET /notifications/recent` endpoint
- **Notification Companion settings screen** — status indicator, test notification dump (pulls `service.activeNotifications` directly for end-to-end verification without a relay round-trip), open-Android-settings button
- **`android_phone_status()`** — new agent tool returning the full phone state via loopback `/bridge/status`
- **`hermes-status`** — CLI shim with three exit codes for shell-scriptable bridge state queries
---
## 🔐 Security & Pairing
- **Android 14+ MediaProjection** — `foregroundServiceType=mediaProjection` added to `BridgeForegroundService` so the screen-capture grant survives backgrounding and Android 14+'s auto-revocation window (symptom prior to fix: consent dialog appears, user allows, dialog closes, grant evaporates within a frame)
- **Master toggle gate fix** — `cachedMasterEnabled` was never written in v0.2.0, so the gate was no-op; service now owns the observer directly
- **MediaProjection consent flow** — `MainActivity` hosts the `ActivityResultLauncher` + a process-singleton `MediaProjectionHolder` rendezvous for non-Activity callers
- **Self-healing EncryptedSharedPreferences** — `KeystoreTokenStore` and `LegacyEncryptedPrefsTokenStore` catch `AEADBadTagException` on master-key rotation (happens automatically on every Android Studio reinstall) and rebuild the prefs file instead of leaving the user permanently unable to decrypt their session token
- **Strict-mode sandbox opt-in** — `RELAY_MEDIA_STRICT_SANDBOX=1` re-enables the allowlist enforcement on `/media/by-path`; permissive-by-default since 2026-04-11 (path-token route still always enforces sandbox)
- **Per-channel grant revoke API** — `PATCH /sessions/{token_prefix}` restarts the clock from now; `DELETE /sessions/{token_prefix}` matches on first-4+ chars, 200 exact / 404 zero / 409 ambiguous, self-revoke flagged via `revoked_self: true`
---
## 🛠️ Installer & Developer Workflow
- **`install.sh` TUI pass** — ANSI colors, boxed banner, unicode step bullets, spinner for the long pip install step, structured closing message
- **`install.sh` restart actually restarts** — fixed the subtle bug where `enable --now` on an already-active systemd service was a no-op (the source of the 2026-04-12 "install ran clean but relay is still on stale code" debug session)
- **Optional `hermes-gateway` restart prompt** — gates on TTY, respects `HERMES_RELAY_RESTART_GATEWAY=1` for non-interactive runs so it's automation-safe
- **`hermes-relay-update` shim** — two-line wrapper around the canonical curl pipe, re-fetches the latest `install.sh` on every invocation so improvements to the installer itself take effect immediately
- **Three version sources in lockstep** — `scripts/bump-version.sh` atomically bumps `gradle/libs.versions.toml`, `pyproject.toml`, and `plugin/relay/__init__.py::__version__` with SemVer validation and monotonic `appVersionCode` enforcement
- **Branch protection on `main`** — direct push blocked except for the `release: vX.Y.Z` pattern, PR must pass CI before merge, force push + branch deletion blocked
- **Feature branches + `--no-ff` merges** — direct-to-main reserved for single-file typos; agent-team branches get per-commit traces in `git log --graph`
- **`base { archivesName }` in `app/build.gradle.kts`** — injects the app version into every APK/AAB filename so release artifacts are self-identifying at every stage of the pipeline
---
## 🐛 Notable Bug Fixes
- **Fix: Android 14 MediaProjection grant evaporation** — missing `foregroundServiceType=mediaProjection` declaration
- **Fix: master toggle gate broken end-to-end** — `cachedMasterEnabled` never written
- **Fix: re-pair required after every Android Studio rebuild** — self-heal corrupted `EncryptedSharedPreferences` on master-key rotation
- **Fix: voice response text cleared on Stop** — `interruptSpeaking()` was wiping `responseText`; now freezes on-screen
- **Fix: auto-scroll didn't follow streaming tokens** — added `LaunchedEffect(length)` driver
- **Fix: STT bubble split user gaze top↔bottom** — moved to in-flow position between waveform and response
- **Fix: sphere too small in voice mode** — Compose `weight()` divides *remaining* space; bumped sphere weight to 1.5f vs response 1f for ~60% share
- **Fix: `install.sh` restart was a no-op on already-active services** — explicit `systemctl --user restart` detection
- **Fix: flavored APK upload paths in CI** — `apk/*/debug/*.apk` glob matches both `googlePlay` and `sideload` flavor directories
- **Fix: `BIND_ACCESSIBILITY_SERVICE` as `uses-permission`** — lint `ProtectedPermissions` violation; already declared correctly on the `<service>` tag
- **Fix: `AutoDisableWorker.notify()` lint `MissingPermission`** — suppression with documented helper gate
- **Fix: stray pairing rate-limit blocks surviving relay restart** — `/pairing/register` now clears all blocks on success
- **Fix: `MissingFeature` newline not treated as sentence boundary in voice TTS chunker**
- **Fix: `ChevronRight` icon missing** — reverted to `AutoMirrored.Filled.ChevronRight`
---
## 📚 Documentation & Skills
- **Installer README + DEVLOG update** — canonical update cycle documented top-to-bottom
- **`docs/spec.md` + `docs/decisions.md`** — bridge pipeline, safety rails architecture, two-flavor rationale
- **`/hermes-relay-self-setup` skill** — single-source agent-readable install recipe (dual-mode: pre-install via raw URL, post-install via slash command)
- **`/hermes-relay-pair` skill** — canonical category layout (`devops`), matches `metadata.hermes.category` frontmatter
- **`user-docs` flavor comparison** — "Which build should I pick?" decision guide on the Release Tracks page
- **Android Studio dev loop** — Bailey's testing convention documented in CLAUDE.md so future sessions don't try to `adb install` from the tool side
---
## 👥 Contributors
Primary development by **@Codename-11** (Bailey Dixon). Implementation assisted by Claude Code on isolated feature branches with `--no-ff` merges to preserve the per-component history.
Dependency bumps via Dependabot: markdown-renderer, gradle-wrapper, haze, camera, kotlinx-coroutines-test.
---
**Full Changelog**: [v0.2.0...v0.3.0](https://github.com/Codename-11/hermes-relay/compare/v0.2.0...v0.3.0)
**See also**: [RELEASE.md](https://github.com/Codename-11/hermes-relay/blob/main/RELEASE.md) for the release recipe, [CHANGELOG.md](https://github.com/Codename-11/hermes-relay/blob/main/CHANGELOG.md) for cumulative history.
+113
View File
@@ -0,0 +1,113 @@
# Hermes-Relay Roadmap
> Where Hermes-Relay is headed. Short, high-level, grouped by release milestone. For detailed implementation plans of active work see [`docs/plans/`](docs/plans/); for shipped work see [`CHANGELOG.md`](CHANGELOG.md); for the session-by-session narrative see [`DEVLOG.md`](DEVLOG.md).
## Vision
Native Android companion for the [Hermes agent platform](https://github.com/NousResearch/hermes-agent) — chat, voice, and full phone control in one app. We're building toward a world where your AI agent has safe, graceful hands on your phone for the tasks where that matters most: messaging, navigation, music, day-to-day automation, and anything else that's currently a tap-through chore.
## Shipped
- **v0.3.0** — Bridge channel (sideload), voice mode, notification companion, two build flavors, full safety rails system. [CHANGELOG](CHANGELOG.md#030---2026-04-13)
- **v0.2.0** — Voice mode foundation, terminal preview, TOFU cert pinning, Paired Devices screen. [CHANGELOG](CHANGELOG.md)
- **v0.1.0** — Chat, sessions, QR pairing, encrypted storage, Play Store submission.
## Current — Axiom-Labs migration
Moving the Play Store listing from a personal account to the DUNS-verified Axiom-Labs LLC org account. Unblocks straight-to-production rollout (no 14-day closed-testing requirement). New applicationId `com.axiomlabs.hermesrelay`; keystore identity + SHA256 fingerprint preserved. In progress — waiting on Google DUNS verification.
## Next — v0.4: Bridge feature expansion
Detailed plan: [`docs/plans/2026-04-13-bridge-feature-expansion.md`](docs/plans/2026-04-13-bridge-feature-expansion.md).
Expands the bridge channel's tool surface substantially, ports reliability patterns from the broader Hermes-Android ecosystem, and ships a per-app playbook skill so the agent has ready-made procedures for common apps out of the box.
**Core gestures.** Long press, drag, and pinch — foundational interactions currently missing from the toolkit.
**Screen efficiency.** Lightweight screen hashing + diff for cheap change detection in navigation loops; targeted node search (`find_nodes`) to avoid dumping full accessibility trees; detailed node introspection (`describe_node`) for richer LLM context.
**System integration.** Clipboard bridge (read + write), system-wide media playback control (play / pause / next / previous), sequential macro execution for batched workflows.
**Reliability.** Wake-lock wrapping on gesture-dispatching actions, three-tier `tap_text` fallback cascade for apps that wrap clickable content in non-clickable views, multi-window accessibility tree traversal.
**Per-app playbook skill.** `skills/android/SKILL.md` gives the LLM ready-made step-by-step procedures for Uber, WhatsApp, Spotify, Maps, Settings, and Tinder — plus a hard "do not loop" rule for bounded tool-call budgets.
**Sideload-only additions.** Location (`ACCESS_FINE_LOCATION`), contact search (`READ_CONTACTS`), direct SMS (`SEND_SMS`), and direct-dial calling (`CALL_PHONE`). All gated behind the existing sideload flavor to preserve Play Store policy compliance on the `googlePlay` track.
## Next-next — v0.4.1: Bridge fast-follows
Small follow-ons to v0.4 deliberately deferred to keep the v0.4.0 release surface focused.
**Unattended access mode** *(sideload-only).* Opt-in toggle on the Bridge tab that acquires `FULL_WAKE_LOCK + ACQUIRE_CAUSES_WAKEUP`, raises `SCREEN_OFF_TIMEOUT` to max while active, and requests `KeyguardManager.requestDismissKeyguard()` so the agent can drive the device while the user is away. Hard-bounded by the existing bridge auto-disable timer, fronted by a persistent foreground-service notification + status-overlay chip, and gated behind a scary opt-in dialog that explains the security model. **Documented hard limit:** Android does not let third-party apps dismiss credential locks (PIN / pattern / biometric) — the user has to set their lock screen to **None** or **Swipe** themselves for the wake to land them past the keyguard. Without that, the screen wakes but stays on the lock screen, and the bridge gracefully reports `keyguard_blocked`. Auto-revokes when the pairing token expires or the device leaves WiFi (failsafe).
**Voice intent local dispatch loop.** The v0.4 voice intent handler builds `bridge.command` envelopes and routes them through the `ChannelMultiplexer` → WSS → relay → back-to-phone path, which the relay correctly rejects with `ignoring unexpected bridge.command from phone` (the wire protocol is server→phone only by design). Voice intents are phone-local, so the dispatch should be local: extend `BridgeCommandHandler` with a `handleLocalCommand(envelope)` entry point that runs the existing `when(path)` dispatch + the full Tier 5 safety check pipeline (blocklist → destructive verb modal → action executor) in-process, and have `RealVoiceBridgeIntentHandler.dispatch()` call it instead of `multiplexer.send()`. Single source of truth for "bridge command → action" preserved; safety modals still fire for destructive verbs; no WSS round-trip for an action that's happening on the same device. Caught by Bailey's on-device test 2026-04-14 after the multiplexer-wiring fix unblocked the dispatch path.
**Tiered permission checklist with JIT permission errors** *(sideload-only sections gated on `BuildFlavor.SIDELOAD`).* Extend `BridgePermissionChecklist.kt` from the current 4-row design to a tiered surface with explicit grant affordances for every dangerous permission Hermes-Relay actually declares:
- **Core bridge** (both flavors) — Accessibility, Screen Capture, Overlay, **Notifications (Android 13+, currently missing)**
- **Notification companion** (both flavors, optional) — Notification Listener
- **Voice & camera** (both flavors) — Microphone, Camera
- **Sideload features** (sideload-only, optional) — Contacts, SMS, Phone, Location
Each row uses `rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission)` for runtime grants and `ACTION_APPLICATION_DETAILS_SETTINGS` deep-links for special perms. Status re-probes on `Lifecycle.Event.ON_RESUME` (same pattern as the existing 4 rows). Optional perms get an "Optional" badge so users don't feel pressured to grant the full set.
**JIT permission-denied surfacing.** When a tool fails because of a missing runtime permission (e.g., `resolveContactPhone` returns null because `READ_CONTACTS` is denied), the failure path returns a structured `permission_denied` error code instead of generic "I couldn't find...". The voice flow speaks **"I need Contacts permission to look up Sam — open Settings to grant"** with a tap-target that deep-links to the app's permission page. The chat flow surfaces the same structured error to the agent so the LLM can recommend the fix in plain language instead of hallucinating about why the tool failed. Requires (a) splitting the resolver return type into a `sealed class ResolveResult<T> { Found / NotFound / PermissionDenied(perm, reason) }`, (b) adding a `code` field to bridge tool error envelopes, (c) the agent tool wrapper interpreting `code: permission_denied` and feeding it back to the LLM with context.
**Voice intent → server session sync.** Voice intents currently dispatch in-process (good for latency) and append a **local-only** trace to chat history (good for visual continuity), but the server-side session never sees them — so the gateway LLM has no memory of prior voice actions when the user follows up via text or voice. Symptom: user says "open Chrome" via voice (works), then says "did that work?" → LLM responds "I have no prior context for what you're asking about". Fix needs one of: (a) a new gateway endpoint `POST /api/sessions/{id}/messages` that injects a message into the session log without firing an LLM completion (cheap, server-side change in hermes-agent), (b) a fait-accompli prompt prefix where the next chat message includes a synthetic system note `[Earlier in this session, the user used voice intent to: open Chrome]` (no server change, prompt-side only), or (c) routing voice intents through `chatVm.sendMessage()` in addition to the local dispatch with an idempotency token so the LLM's server-side bridge tools don't double-fire. Pick one after measuring the LLM's tendency to retry actions when given fait-accompli context. Caught by Bailey's on-device test 2026-04-14: "The chat is resetting on voice or with our tools?" — actually voice intents bypass chat entirely, but the user-visible effect is the same.
**Gateway slash-command preprocessor — bootstrap middleware (Option B scope).** Built-in Hermes slash commands (`/model`, `/new`, `/retry`, the 29 in `hermes_cli/commands.py::GATEWAY_KNOWN_COMMANDS`) are intercepted by in-process platform adapters like Discord, Telegram, Slack, etc. — all of which route inbound `MessageEvent`s through `GatewayRouter._handle_message` at `gateway/run.py:2645–2929`, where a ~300-line dispatch chain mutates router-owned state (`_session_model_overrides`, `_agent_cache`). But `APIServerAdapter` **does not connect to the router** — it's intentionally excluded from the router notification path (see comment at `run.py:3148`), calls `_run_agent` directly, and **creates a fresh agent per request with no persistent session state**. This is the design intent of the OpenAI-compatible endpoint, not an oversight. The practical effect: on `POST /v1/runs` and `POST /v1/chat/completions`, slash commands pass through to the LLM verbatim; the LLM hallucinates a plausible-sounding but wrong reply ("`/model` is a client-side command"); the user is confused. Caught by Bailey on 2026-04-15 during a Hermes chat test from the Android app.
**What the middleware can do (near-term, ships via install.sh).** New aiohttp middleware in `hermes_relay_bootstrap/_command_middleware.py`, installed at the same `_PatchedApplication.__setitem__` hook as the current route injection so it lands before `AppRunner.setup()` freezes the app. Filters by `request.path in ("/v1/runs", "/v1/chat/completions")` — zero-cost fast path for everything else. On chat paths: parses the body, lazy-imports `GATEWAY_KNOWN_COMMANDS` + `resolve_command()` + `gateway_help_lines()` from `hermes_cli.commands`, and splits on command type:
- **Stateless commands** (`/help`, `/commands`, and any others the upstream Option B PR ends up supporting without router state) — actually dispatch, emit a synthetic SSE stream matching the runs handler's existing event shape so the Android client at `HermesApiClient.kt:655-715` renders it as a normal assistant turn.
- **Stateful commands** (`/model`, `/new`, `/retry`, `/undo`, `/compress`, `/title`, `/resume`, `/branch`, `/rollback`, `/yolo`, `/reasoning`, `/personality`, etc. — most of the registry) — emit a synthetic SSE stream whose content is a short, helpful notice: *"The `/model` command requires a persistent session and isn't available on the stateless `/v1/runs` endpoint. Use `/api/sessions/{id}/chat/stream` (post-PR-#8556) or a channel with session state. For commands that work here, type `/help`."* This replaces the LLM hallucination with a deterministic, accurate message that points the user at the real fix.
**On no match** (unknown command, cli-only command, or plain text): falls through to `handler(request)` unchanged. Fork-detects the same way the existing injection does — if the upstream preprocessor PR lands first, the middleware no-ops.
**Ceiling.** This middleware can **never** make `/model` actually switch models on `/v1/runs`, because there is no persistent session on that endpoint to switch. That's a Phase 2 follow-up (below), not a flaw in the middleware.
**Files.** New `hermes_relay_bootstrap/_command_middleware.py` (~150 LOC), one-line append in `_patch.py` inside `_maybe_register_routes`, stdlib `unittest` coverage in `plugin/tests/test_bootstrap_command_middleware.py` mirroring the existing `test_bootstrap_patch.py` harness. Mirrors the upstream Option B PR exactly so the two can be reviewed side-by-side.
**Phase 2 — stateful dispatch on the session chat stream endpoint (post PR #8556).** Once PR #8556 merges and `/api/sessions/{id}/chat/stream` ships natively in upstream, a separate middleware (or a follow-up upstream PR) can add a preprocessor **scoped to that endpoint only**, leveraging the `session_id` in the URL as the persistence handle. At that point stateful commands become a dict write against session-scoped state — `session.model_override = new_model` — without needing to refactor `GatewayRouter` or plumb api_server into the router. Much smaller than a full router refactor, and it matches upstream's partition: `/v1/*` stays stateless, statefulness lives on `/api/sessions/*`. Blocked on #8556 landing.
## Future — v0.5+
Shape subject to change. Each theme needs a separate design + plan pass before implementation; file design notes as research matures.
### Observability & introspection
- Real-time accessibility event streaming for reactive workflows (`android_events`, `android_event_stream`)
- On-device text-to-speech through the phone's system speaker for hands-free responses (distinct from the in-app voice mode)
- Short MP4 screen recording for visual bug reports and "show me what happens when you tap this" flows
- Annotated failure screenshots — auto-capture a screenshot with the intended target highlighted when a tap or wait fails, so the LLM can self-correct with visual context
- Generalized loop guardrails across all bridge tools — extend `android_navigate`'s `max_iterations` cap into a per-session rolling counter covering every bridge tool call
- Raw Intent / Broadcast escape hatch for power workflows
### Automation & triggers
- **Scheduled automations** — "every weekday at 7am, open Maps, check my commute, report the time." Wires bridge commands to hermes-agent's existing cron tool
- **Event-triggered actions** — "when a notification from X arrives, do Y." Reactive rule engine on top of the notification companion + accessibility event stream
- **User-recorded macros** — watch a workflow once, replay it on demand via `android_macro`
- **Multi-phone pairing UX** — explicit "add another device" flow, per-device routing for tool calls
- **Phone → server file transfer** — reverse direction of the existing inbound media pipeline ("fetch my latest photo")
### Voice assistant
- Always-listening wake-word mode (Porcupine or equivalent), off-by-default with explicit opt-in
- Phone call handling — agent answers incoming calls, speaks via TTS, transcribes incoming audio, takes messages ("answer my phone, take a message, tell them I'll call back")
### Research horizon
- **On-device local model execution** — Gemma / Qwen running directly on the phone via MediaPipe or llama.cpp, for offline fallback and hybrid routing (simple tasks local, complex tasks remote)
- **Cross-app workflow execution** with inter-step state carry-over — "find a restaurant on Maps, share it on WhatsApp, book an Uber there"
- **Web dashboard** for monitoring bridge activity server-side
- **iOS support** via Shortcuts + accessibility bridge + App Intents (evaluate feasibility before committing)
- **Developer-mode embedded HTTP server** on the phone — a Ktor/Netty server on a local port for direct-to-phone testing over USB or LAN without routing through the relay (dev ergonomics, not user-facing)
### Vision
Dedicated **"Hermes Phone"** — a device (or phone ROM) that boots straight into agent mode, where the OS itself is the agent. Long-term north star, not a concrete deliverable.
---
## How this roadmap evolves
New ideas enter via: direct proposals in GitHub issues, comparison passes against similar projects, community feedback from users and contributors, or internal research that turns into a shipped prototype.
Active work waves (like the v0.4 bridge feature expansion above) get their detailed implementation plans in [`docs/plans/`](docs/plans/). When a plan wave ships, its plan file is archived or removed and the items migrate into [`CHANGELOG.md`](CHANGELOG.md).
Have an idea? [Open an issue](https://github.com/Codename-11/hermes-relay/issues/new) — every one is read.
+42
View File
@@ -0,0 +1,42 @@
# Hermes-Relay — TODO
Open items that don't fit a formal Phase plan but shouldn't be lost. Items move from here into a Plan in `docs/spec.md` or an Obsidian Phase plan once they're ready to schedule.
For shipped work, see `DEVLOG.md`. For architectural decisions, see `docs/decisions.md`.
---
## Research / open questions
### Proper Hermes plugin / skill / tool distribution
**Status:** open question, no plan yet.
We currently distribute Hermes-Relay via a one-shot `install.sh` that clones the repo, `pip install -e`s the package into the user's hermes-agent venv, and registers `skills/` via the `external_dirs` config knob. This works but it's a custom protocol — every project that wants to ship a Hermes plugin reinvents it.
Things to look into:
- **Does upstream hermes-agent have or plan a canonical plugin registry / package format?** If yes, we should migrate to it. If no, we may want to propose one upstream so third-party plugins (ours and others) get a standard install path.
- **Skill distribution as separate from plugin distribution** — right now skills ride along with the plugin install via `external_dirs`. Should skills be installable independently (e.g. `hermes skill install <git-url>`)? Would that fragment maintenance or improve reuse?
- **Tool registration discoverability** — `android_*` tools register at gateway import time. There's no canonical "list installed plugin tools" API. Would adding one to upstream make sense, or is `gateway tool list` already enough?
- **Versioning + compatibility ranges** — `pip install -e` doesn't enforce version pins between hermes-agent and our plugin. A breaking change in upstream's plugin loader could silently break us. Do we need a `hermes_compat: ">=0.8.0,<1.0.0"` field somewhere?
- **`hermes-relay-self-setup` SKILL.md as a precedent** — we just shipped a self-installing skill that an LLM can fetch from a raw GitHub URL and execute. Does this pattern generalize? Could it become a recommended way for any third-party Hermes project to ship setup automation?
- **Bootstrap injection** — `hermes_relay_bootstrap/` monkey-patches `aiohttp.web.Application` to inject endpoints into vanilla upstream. This is intentional but feels like a hack. Upstream PR #8556 (`feat/session-api`) will eventually let us delete it — verified 2026-04-15 that its scope covers the full bootstrap surface (sessions, memory, skills, config, available-models). Track that PR's status periodically.
- **Gateway slash-command preprocessor — upstream Stage 1 PR.** Sibling follow-up to #8556. Intercepts known gateway commands on `/v1/runs` + `/v1/chat/completions`, dispatches the stateless ones (`/help`, `/commands`) via `gateway_help_lines()`, returns a deterministic "use a channel with session state" notice for the stateful majority. Currently being prepared in `C:/Users/Bailey/Desktop/Open-Projects/hermes-agent-pr-prep/` on branch `feat/api-server-gateway-commands`; awaiting subagent's code + draft PR body before pushing. See `docs/upstream-contributions.md` §5.
- **Gateway slash-command preprocessor — bootstrap middleware (Stage 1 equivalent).** Sibling shim in `hermes_relay_bootstrap/_command_middleware.py` that mirrors the upstream Stage 1 PR as an aiohttp middleware injected at bootstrap time. Ships the hallucination fix to vanilla-upstream installs before the upstream PR lands. Planned for v0.4.1, after the current bridge feature branch wraps. See `ROADMAP.md` v0.4.1 entry.
- **Stage 2 — stateful slash-command dispatch on `/api/sessions/{id}/chat/stream`.** Blocked on PR #8556 merging. Once session primitives ship upstream, add a preprocessor scoped to the session chat stream endpoint only, using `session_id` as the persistence handle. Separate upstream PR + matching bootstrap middleware. See `docs/upstream-contributions.md` §5 ("Stage 2").
When the answer becomes clearer, this section becomes either an ADR in `docs/decisions.md` or a Plan under `Plans/`.
---
## Smaller deferred items
- **MediaProjection consent flow** — wired in MainActivity (2026-04-12), needs end-to-end test on a real device
- **WorkManager upgrade for auto-disable timer** — currently a coroutine `Job + delay()` in `AutoDisableWorker.kt`; documented at top of file. Upgrade when androidx.work joins the classpath
- **Wave 3 voice-bridge multi-turn confirmation** — currently a 5s TTS countdown with cancel; conversational confirmation is the follow-up
- **LLM client wiring for `android_navigate`** — `_default_vision_model` is stubbed; production swap to a real Anthropic/OpenAI vision client
- **Real screenshots of each flavor's a11y permission dialog** — for `user-docs/guide/release-tracks.md`
- **`llms.txt` standard** — explicitly skipped in favor of the `hermes-relay-self-setup` SKILL.md path; revisit if the standard gains traction in the agent ecosystem
- **`markdown-renderer` 0.40.x API update** — pinned at `0.30.0` in `gradle/libs.versions.toml` because 0.40.2 introduced breaking API changes that `app/src/main/kotlin/com/hermesandroid/relay/ui/components/MarkdownContent.kt` hasn't been updated for. Specifically: `markdownColor()` drops `codeText`/`linkText`, `MarkdownCodeBlock`/`MarkdownCodeFence` inner lambdas now take a 3rd `TextStyle` arg, and `MarkdownHighlightedCode`'s 3rd param is now `TextStyle` instead of `Highlights.Builder`. Dependabot auto-merged the bump on 2026-04-13 which silently broke CI; reverted for the v0.3.0 release. Update requires reading the new library API docs and testing in Studio — not a blind fix. Consider adding a dependabot ignore rule for `markdown-renderer` major bumps until this is handled.
- **Dependabot auto-merge guardrails** — Dependabot merged breaking bumps despite CI failing. Investigate why `.github/workflows/dependabot-auto-merge.yml` isn't gating on CI status, and consider adding an ignore rule for packages we know need manual attention on major bumps (`markdown-renderer`, compose BOM, activity-compose).
+69 -1
View File
@@ -7,12 +7,35 @@ plugins {
alias(libs.plugins.play.publisher)
}
// Rename output artifacts to include the app version. AGP respects
// `archivesName` for both APK (assemble*) and AAB (bundle*) outputs, so
// this single line produces `hermes-relay-<version>-<flavor>-<buildType>`
// filenames across debug, release, and per-flavor variants. The version
// is pulled from libs.versions.toml so bumping via scripts/bump-version.sh
// keeps artifact names in sync with no second source of truth.
base {
archivesName.set("hermes-relay-${libs.versions.appVersionName.get()}")
}
android {
// Kotlin package / on-disk source layout / R class namespace. Decoupled
// from `applicationId` below as of the Axiom-Labs org-account migration:
// the repo's source tree stays under `com.hermesandroid.relay` (so all
// 130+ Kotlin files and their package declarations keep working) while
// the Play Store / Android-system identity lives under `com.axiomlabs.*`.
// This is an AGP-supported pattern — `namespace` is a build-time concept
// and `applicationId` is the runtime install identity; they don't have
// to match.
namespace = "com.hermesandroid.relay"
compileSdk = 36
defaultConfig {
applicationId = "com.hermesandroid.relay"
// Axiom-Labs, LLC Play Console listing. Changed from the original
// `com.hermesandroid.relay` on 2026-04-13 during the org-account
// migration. The old Internal-testing listing under Bailey's personal
// account is being deleted; the DUNS-verified Axiom-Labs account is
// exempt from Play's 14-day closed-testing rule. See RELEASE.md.
applicationId = "com.axiomlabs.hermesrelay"
minSdk = 26
targetSdk = 35
versionCode = libs.versions.appVersionCode.get().toInt()
@@ -45,12 +68,56 @@ android {
}
}
// ─── Phase 3 — Bridge channel release tracks ────────────────────────────────
// Google Play scrutinizes AccessibilityService heavily (policy review + manual
// appeals are common), so Phase 3 ships two distinct tracks via flavor-merged
// manifests + flavor-scoped strings + flavor-scoped accessibility configs:
//
// googlePlay — conservative use-case description targeted at Play Store
// policy review. Subset of event types + flagDefault only.
// No gestures, no interactive-window reporting. Feature gates
// in BuildFlavor.kt hide tier 3/4/6 surfaces in the UI.
//
// sideload — full agent-control description for users who install the
// APK directly (GitHub Releases, F-Droid, ADB). typeAllMask,
// gestures, interactive windows, view-id reporting. All six
// tiers enabled.
//
// applicationIdSuffix decision: sideload gets `.sideload` so both tracks can
// coexist on the same device. The Play build keeps the base
// `com.axiomlabs.hermesrelay` applicationId as the canonical Play Store
// install; sideload becomes `com.axiomlabs.hermesrelay.sideload`. Cost:
// anyone with both installed sees two launcher icons — we differentiate
// via the flavored strings.xml label suffix.
//
// Note: the previous `com.hermesandroid.relay` applicationId (Internal
// testing under Bailey's personal Play account) is being retired as part
// of the Axiom-Labs org-account migration. Play Store package names are
// permanently reserved once used, so the old ID can never be reclaimed —
// existing Internal-testing installs won't auto-upgrade to the new listing
// and will need a manual reinstall (limited blast radius, single tester).
flavorDimensions += "track"
productFlavors {
create("googlePlay") {
dimension = "track"
// No applicationIdSuffix — this IS the canonical Play Store install.
}
create("sideload") {
dimension = "track"
applicationIdSuffix = ".sideload"
versionNameSuffix = "-sideload"
}
}
buildTypes {
debug {
buildConfigField("boolean", "DEV_MODE", "true")
}
release {
isMinifyEnabled = true
ndk {
debugSymbolLevel = "SYMBOL_TABLE"
}
proguardFiles(
getDefaultProguardFile("proguard-android-optimize.txt"),
"proguard-rules.pro"
@@ -176,3 +243,4 @@ dependencies {
debugImplementation(libs.compose.ui.tooling)
debugImplementation(libs.compose.ui.test.manifest)
}
+28
View File
@@ -0,0 +1,28 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Google Play flavor manifest overlay.
Merged on top of `app/src/main/AndroidManifest.xml` by AGP when the
`googlePlayDebug` / `googlePlayRelease` variants are built.
The AccessibilityService is declared exactly once in `app/src/main/AndroidManifest.xml`.
The flavor distinction is purely at the resource layer: this flavor's
`res/xml/accessibility_service_config.xml` carries the conservative use-case
description required for Google Play policy review, and `res/values/strings.xml`
carries the description string. Gradle's resource merger picks the right
files at build time, so we don't need to redeclare the <service> here.
This file is intentionally kept as an empty overlay so future flavor-specific
permissions / activities have an obvious home. Mirror structural additions
in `app/src/sideload/AndroidManifest.xml` unless the change is intentionally
track-specific.
-->
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<!-- googlePlay inherits main manifest's specialUse-only FGS type
directly — no override needed. The sideload manifest ADDS
mediaProjection via tools:replace; googlePlay gets the safe
default. -->
<application />
</manifest>
@@ -0,0 +1,48 @@
package com.hermesandroid.relay.voice
import com.hermesandroid.relay.network.ChannelMultiplexer
import com.hermesandroid.relay.network.handlers.LocalDispatchResult
import com.hermesandroid.relay.network.models.Envelope
/**
* Local in-process bridge dispatcher type. The Play flavor never invokes
* this — phone-control actions are sideload-only — but the typealias has
* to exist in the googlePlay source set so the shared `VoiceViewModel`
* call site compiles regardless of active flavor.
*
* Return type mirrors the sideload flavor's updated shape so the shared
* typealias binding site in VoiceViewModel compiles against either source
* set without #if gating.
*/
typealias LocalBridgeDispatcher = suspend (Envelope) -> LocalDispatchResult
/** @see com.hermesandroid.relay.voice.VoiceIntentResultCallback in the sideload flavor. */
typealias VoiceIntentResultCallback = (intentLabel: String, result: LocalDispatchResult) -> Unit
/** @see com.hermesandroid.relay.voice.VoiceIntentCountdownCallback in the sideload flavor. */
typealias VoiceIntentCountdownCallback = (intentLabel: String, durationMs: Long) -> Unit
/**
* === PHASE3-voice-intents (googlePlay flavor): factory ===
*
* Flavor-specific factory function that [com.hermesandroid.relay.viewmodel.VoiceViewModel]
* calls exactly once during `initialize()`. The Play build always returns
* the no-op handler.
*
* Both the `googlePlay` and `sideload` flavors export a function with this
* exact signature + package so `VoiceViewModel` has a single static call
* site and no reflection.
*
* All parameters accepted for signature parity with sideload and silently
* ignored — the Play APK deliberately never references any bridge or
* accessibility class so the conservative Play feature description stays
* honest.
*/
fun createVoiceBridgeIntentHandler(
multiplexer: ChannelMultiplexer?,
localBridgeDispatcher: LocalBridgeDispatcher? = null,
onDispatchResult: VoiceIntentResultCallback? = null,
onCountdownStart: VoiceIntentCountdownCallback? = null,
): VoiceBridgeIntentHandler = NoopVoiceBridgeIntentHandler()
// === END PHASE3-voice-intents (googlePlay) ===
@@ -0,0 +1,31 @@
package com.hermesandroid.relay.voice
/**
* === PHASE3-voice-intents (googlePlay flavor): no-op voice→bridge handler ===
*
* On the Google Play track we ship the conservative feature description:
* no accessibility-service usage, no device-control voice routing.
*
* This implementation never references any bridge / accessibility class and
* always returns [IntentResult.NotApplicable] so [VoiceViewModel] falls
* through to normal chat for every utterance.
*
* Sibling: `app/src/sideload/kotlin/.../VoiceBridgeIntentHandlerImpl.kt`
* ships the real classifier.
*/
internal class NoopVoiceBridgeIntentHandler : VoiceBridgeIntentHandler {
override suspend fun tryHandle(transcribedText: String): IntentResult {
// Play APK path: every utterance is chat. No classification runs,
// no bridge envelopes are sent. Nothing to cancel either.
return IntentResult.NotApplicable
}
override fun cancelPending() {
// no-op — nothing to cancel on Play.
}
override fun hasPendingDestructive(): Boolean = false
}
// === END PHASE3-voice-intents (googlePlay) ===
+18
View File
@@ -0,0 +1,18 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Google Play flavor strings.
`a11y_description_googleplay` is the user-facing description shown in
Android's Accessibility settings when enabling the Hermes Bridge service.
It is ALSO what Play Store reviewers read when evaluating our
AccessibilityService use-case declaration, so phrasing matters: stay
narrowly scoped, emphasize user confirmation, emphasize dormancy until
the user opts in inside the app.
Do not reference voice or vision features here — tier 3/4/6 are gated
off for this flavor via FeatureFlags.BuildFlavor.
-->
<resources>
<string name="a11y_service_label">Hermes-Bridge</string>
<string name="a11y_description_googleplay">Hermes assists you by reading on-screen content and summarizing notifications. The service is read-only — it does not perform taps, type text, or control other apps. It is dormant until you explicitly enable Bridge mode in the app.</string>
</resources>
@@ -0,0 +1,20 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Google Play AccessibilityService configuration.
Conservative event-type subset targeted at the "read notifications,
summarize messages, reply with confirmation" use case Play Store policy
review expects. Does NOT subscribe to typeAllMask, does NOT request
gestures, does NOT request flagRetrieveInteractiveWindows.
Keep these attributes aligned with the description in strings.xml
(`a11y_description_googleplay`) — if the description widens, reviewers
will expect the config to widen too.
-->
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:description="@string/a11y_description_googleplay"
android:accessibilityEventTypes="typeWindowStateChanged|typeWindowContentChanged|typeViewClicked"
android:accessibilityFlags="flagDefault"
android:accessibilityFeedbackType="feedbackGeneric"
android:notificationTimeout="100"
android:canRetrieveWindowContent="true" />
+155
View File
@@ -4,9 +4,76 @@
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<!-- === A8 wake-lock: keep CPU awake while dispatching bridge gestures === -->
<!-- Normal-protection permission (no runtime prompt). Held only inside
WakeLockManager.wakeForAction { ... }, with a 10s hard timeout and
ref-counted release. -->
<uses-permission android:name="android.permission.WAKE_LOCK" />
<!-- === PHASE3-accessibility: AccessibilityService + bridge permissions === -->
<!-- BIND_ACCESSIBILITY_SERVICE is intentionally NOT declared as a
<uses-permission> here — it's a system-only permission granted to
services that declare android:permission on their <service> tag
(see the BridgeAccessibilityService entry below). Declaring it as
a uses-permission trips lint's [ProtectedPermissions] check.
FOREGROUND_SERVICE* are for the persistent notification that
Agent safety-rails will wire in Wave 2 for MediaProjection-backed
screenshots. POST_NOTIFICATIONS is required on API 33+ for that
same foreground-service notification. -->
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<!-- FOREGROUND_SERVICE_MEDIA_PROJECTION moved to sideload manifest.
googlePlay doesn't need screen recording: /screenshot route is
gated sideload-only in BridgeCommandHandler. Declaring the
permission on the Play track would flag review since our
accessibility use-case ("read-only screen reading") doesn't
justify screen capture. -->
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<!-- === END PHASE3-accessibility === -->
<!-- === PHASE3-safety-rails: safety service + overlay === -->
<!-- SYSTEM_ALERT_WINDOW is user-granted via Settings.ACTION_MANAGE_OVERLAY_PERMISSION.
Used for (a) the destructive-verb confirmation modal that must be
visible even when Hermes isn't in the foreground, and (b) the optional
floating "Hermes active" status chip. The permission is declared here
so the user-visible grant flow triggers, but the overlay itself only
appears when the user has explicitly consented.
FOREGROUND_SERVICE_SPECIAL_USE is required on Android 14+ for the
persistent "Bridge active" notification (BridgeForegroundService),
because the specialUse type needs its own declared permission. -->
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_SPECIAL_USE" />
<!-- === END PHASE3-safety-rails === -->
<uses-feature android:name="android.hardware.camera" android:required="false" />
<!-- === PHASE3-baseline-handlers: package visibility for /get_apps + /open_app === -->
<!-- On Android 11+ (API 30+), apps can only see other packages that
are implicitly visible (own UID, system apps, etc.) unless they
declare a <queries> filter or hold QUERY_ALL_PACKAGES. The
BridgeCommandHandler /get_apps route uses
queryIntentActivities(ACTION_MAIN + CATEGORY_LAUNCHER) to enumerate
launchable apps, and BridgeSafetySettingsScreen uses the same call
to populate the blocklist UI — both need this declaration to see
the full launcher set. Without it queryIntentActivities silently
returns a near-empty list (typical symptom: blocklist UI shows
only Hermes-Relay itself + a handful of system apps).
Declaring an <intent> filter with ACTION_MAIN + CATEGORY_LAUNCHER
is the Play-policy-safe approach — it does NOT require the
restricted QUERY_ALL_PACKAGES permission, which Play would
otherwise demand a policy declaration for. -->
<queries>
<intent>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent>
</queries>
<!-- === END PHASE3-baseline-handlers === -->
<application
android:name=".HermesRelayApp"
android:allowBackup="true"
@@ -20,6 +87,8 @@
<activity
android:name=".MainActivity"
android:exported="true"
android:launchMode="singleTask"
android:configChanges="uiMode|fontScale|locale|density|orientation|screenSize|screenLayout|keyboardHidden"
android:theme="@style/Theme.HermesRelay.Splash">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
@@ -27,6 +96,92 @@
</intent-filter>
</activity>
<provider
android:name="androidx.core.content.FileProvider"
android:authorities="${applicationId}.fileprovider"
android:exported="false"
android:grantUriPermissions="true">
<meta-data
android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/file_provider_paths" />
</provider>
<!-- === PHASE3-accessibility: AccessibilityService declaration === -->
<!-- The @xml/accessibility_service_config resource is provided by
the flavor-specific source sets (app/src/googlePlay/ and
app/src/sideload/) owned by Agent flavor-split. Each flavor declares its
own accessibility use-case description and flag bitset — the
googlePlay track declares a conservative "notifications + reply
with confirmation" use case, the sideload track declares the
full agent-control use case. Gradle merges the flavor XML into
main at build time.
-->
<service
android:name=".accessibility.HermesAccessibilityService"
android:exported="true"
android:label="@string/a11y_service_label"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="@xml/accessibility_service_config" />
</service>
<!-- === END PHASE3-accessibility === -->
<!-- === PHASE3-notif-listener: notification companion service === -->
<service
android:name=".notifications.HermesNotificationCompanion"
android:label="@string/notification_companion_label"
android:permission="android.permission.BIND_NOTIFICATION_LISTENER_SERVICE"
android:exported="true">
<intent-filter>
<action android:name="android.service.notification.NotificationListenerService" />
</intent-filter>
</service>
<!-- === END PHASE3-notif-listener === -->
<!-- === PHASE3-safety-rails: safety service + overlay === -->
<!-- BridgeForegroundService is a plain (non-exported) foreground
service driven by BridgeViewModel based on the master toggle.
It owns the persistent "Hermes agent has device control"
notification.
foregroundServiceType is the OR of two API 34+ subtypes:
- specialUse — backs the persistent "bridge active"
indicator we shipped with Tier 5 safety rails. Comes
with the SPECIAL_USE foreground-service permission and
the Play Console policy declaration.
- mediaProjection — REQUIRED by Android 14+ before any
call to MediaProjectionManager.getMediaProjection().
Without this declaration, getMediaProjection() returns
a projection that the system auto-revokes within a
frame, leaving us with a permanently-null
MediaProjectionHolder.projection. Symptom on the
device: consent dialog appears, user allows full
screen, dialog closes, grant evaporates. Sample-tested
on a Samsung S24 / Android 14 on 2026-04-12.
Both types share the same notification + same lifecycle —
one service, one notification, two type slots.
Android 14+ requires a <property> tag justifying the
specialUse subtype. The mediaProjection subtype does NOT
need a property tag because it has its own dedicated
permission (FOREGROUND_SERVICE_MEDIA_PROJECTION). -->
<service
android:name=".bridge.BridgeForegroundService"
android:exported="false"
android:foregroundServiceType="specialUse">
<property
android:name="android.app.PROPERTY_SPECIAL_USE_FGS_SUBTYPE"
android:value="Maintains a persistent WebSocket connection to the user's Hermes server for real-time chat relay and notification mirroring. The service is dormant until the user explicitly enables Bridge mode in the app." />
</service>
<!-- === END PHASE3-safety-rails === -->
</application>
</manifest>
@@ -0,0 +1,2 @@
!function(e,t){"object"==typeof exports&&"object"==typeof module?module.exports=t():"function"==typeof define&&define.amd?define([],t):"object"==typeof exports?exports.FitAddon=t():e.FitAddon=t()}(self,(()=>(()=>{"use strict";var e={};return(()=>{var t=e;Object.defineProperty(t,"__esModule",{value:!0}),t.FitAddon=void 0,t.FitAddon=class{activate(e){this._terminal=e}dispose(){}fit(){const e=this.proposeDimensions();if(!e||!this._terminal||isNaN(e.cols)||isNaN(e.rows))return;const t=this._terminal._core;this._terminal.rows===e.rows&&this._terminal.cols===e.cols||(t._renderService.clear(),this._terminal.resize(e.cols,e.rows))}proposeDimensions(){if(!this._terminal)return;if(!this._terminal.element||!this._terminal.element.parentElement)return;const e=this._terminal._core,t=e._renderService.dimensions;if(0===t.css.cell.width||0===t.css.cell.height)return;const r=0===this._terminal.options.scrollback?0:e.viewport.scrollBarWidth,i=window.getComputedStyle(this._terminal.element.parentElement),o=parseInt(i.getPropertyValue("height")),s=Math.max(0,parseInt(i.getPropertyValue("width"))),n=window.getComputedStyle(this._terminal.element),l=o-(parseInt(n.getPropertyValue("padding-top"))+parseInt(n.getPropertyValue("padding-bottom"))),a=s-(parseInt(n.getPropertyValue("padding-right"))+parseInt(n.getPropertyValue("padding-left")))-r;return{cols:Math.max(2,Math.floor(a/t.css.cell.width)),rows:Math.max(1,Math.floor(l/t.css.cell.height))}}}})(),e})()));
//# sourceMappingURL=addon-fit.js.map
File diff suppressed because one or more lines are too long
@@ -0,0 +1,2 @@
!function(e,t){"object"==typeof exports&&"object"==typeof module?module.exports=t():"function"==typeof define&&define.amd?define([],t):"object"==typeof exports?exports.WebLinksAddon=t():e.WebLinksAddon=t()}(self,(()=>(()=>{"use strict";var e={6:(e,t)=>{function n(e){try{const t=new URL(e),n=t.password&&t.username?`${t.protocol}//${t.username}:${t.password}@${t.host}`:t.username?`${t.protocol}//${t.username}@${t.host}`:`${t.protocol}//${t.host}`;return e.toLocaleLowerCase().startsWith(n.toLocaleLowerCase())}catch(e){return!1}}Object.defineProperty(t,"__esModule",{value:!0}),t.LinkComputer=t.WebLinkProvider=void 0,t.WebLinkProvider=class{constructor(e,t,n,o={}){this._terminal=e,this._regex=t,this._handler=n,this._options=o}provideLinks(e,t){const n=o.computeLink(e,this._regex,this._terminal,this._handler);t(this._addCallbacks(n))}_addCallbacks(e){return e.map((e=>(e.leave=this._options.leave,e.hover=(t,n)=>{if(this._options.hover){const{range:o}=e;this._options.hover(t,n,o)}},e)))}};class o{static computeLink(e,t,r,i){const s=new RegExp(t.source,(t.flags||"")+"g"),[a,c]=o._getWindowedLineStrings(e-1,r),l=a.join("");let d;const p=[];for(;d=s.exec(l);){const e=d[0];if(!n(e))continue;const[t,s]=o._mapStrIdx(r,c,0,d.index),[a,l]=o._mapStrIdx(r,t,s,e.length);if(-1===t||-1===s||-1===a||-1===l)continue;const h={start:{x:s+1,y:t+1},end:{x:l,y:a+1}};p.push({range:h,text:e,activate:i})}return p}static _getWindowedLineStrings(e,t){let n,o=e,r=e,i=0,s="";const a=[];if(n=t.buffer.active.getLine(e)){const e=n.translateToString(!0);if(n.isWrapped&&" "!==e[0]){for(i=0;(n=t.buffer.active.getLine(--o))&&i<2048&&(s=n.translateToString(!0),i+=s.length,a.push(s),n.isWrapped&&-1===s.indexOf(" ")););a.reverse()}for(a.push(e),i=0;(n=t.buffer.active.getLine(++r))&&n.isWrapped&&i<2048&&(s=n.translateToString(!0),i+=s.length,a.push(s),-1===s.indexOf(" ")););}return[a,o]}static _mapStrIdx(e,t,n,o){const r=e.buffer.active,i=r.getNullCell();let s=n;for(;o;){const e=r.getLine(t);if(!e)return[-1,-1];for(let n=s;n<e.length;++n){e.getCell(n,i);const s=i.getChars();if(i.getWidth()&&(o-=s.length||1,n===e.length-1&&""===s)){const e=r.getLine(t+1);e&&e.isWrapped&&(e.getCell(0,i),2===i.getWidth()&&(o+=1))}if(o<0)return[t,n]}t++,s=0}return[t,s]}}t.LinkComputer=o}},t={};function n(o){var r=t[o];if(void 0!==r)return r.exports;var i=t[o]={exports:{}};return e[o](i,i.exports,n),i.exports}var o={};return(()=>{var e=o;Object.defineProperty(e,"__esModule",{value:!0}),e.WebLinksAddon=void 0;const t=n(6),r=/(https?|HTTPS?):[/]{2}[^\s"'!*(){}|\\\^<>`]*[^\s"':,.!?{}|\\\^~\[\]`()<>]/;function i(e,t){const n=window.open();if(n){try{n.opener=null}catch{}n.location.href=t}else console.warn("Opening link blocked as opener could not be cleared")}e.WebLinksAddon=class{constructor(e=i,t={}){this._handler=e,this._options=t}activate(e){this._terminal=e;const n=this._options,o=n.urlRegex||r;this._linkProvider=this._terminal.registerLinkProvider(new t.WebLinkProvider(this._terminal,o,this._handler,n))}dispose(){this._linkProvider?.dispose()}}})(),o})()));
//# sourceMappingURL=addon-web-links.js.map
+308
View File
@@ -0,0 +1,308 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no, viewport-fit=cover">
<meta name="format-detection" content="telephone=no">
<title>Hermes-Relay Terminal</title>
<link rel="stylesheet" href="xterm.css">
<style>
html, body {
margin: 0;
padding: 0;
width: 100%;
height: 100%;
background: #1A1A2E;
overflow: hidden;
font-family: 'JetBrains Mono', 'Fira Code', 'Cascadia Mono', 'Roboto Mono', monospace;
-webkit-user-select: none;
user-select: none;
-webkit-touch-callout: none;
-webkit-tap-highlight-color: transparent;
}
#terminal {
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
padding: 8px 6px 0 8px;
box-sizing: border-box;
}
.xterm .xterm-viewport {
background-color: #1A1A2E !important;
}
/* Let text inside the terminal be selectable so copy/paste works. */
.xterm-rows, .xterm-selection-layer {
-webkit-user-select: text;
user-select: text;
}
</style>
</head>
<body>
<div id="terminal"></div>
<script src="xterm.js"></script>
<script src="addon-fit.js"></script>
<script src="addon-web-links.js"></script>
<script src="addon-search.js"></script>
<script>
(function () {
// ── Hermes-Relay theme (matches app primary palette) ──────────────
const hermesTheme = {
background: '#1A1A2E',
foreground: '#E8E8F0',
cursor: '#B794F4',
cursorAccent: '#1A1A2E',
selectionBackground: '#4A3A7A',
black: '#1A1A2E',
red: '#FF6B9D',
green: '#7EE787',
yellow: '#FFD166',
blue: '#79C0FF',
magenta: '#B794F4',
cyan: '#56D4DD',
white: '#E8E8F0',
brightBlack: '#4A4A6A',
brightRed: '#FF8FAE',
brightGreen: '#A8F0B0',
brightYellow: '#FFE29A',
brightBlue: '#A5D4FF',
brightMagenta: '#D0B3FF',
brightCyan: '#7FE5EE',
brightWhite: '#FFFFFF',
};
const term = new Terminal({
cursorBlink: true,
cursorStyle: 'block',
fontFamily: "'JetBrains Mono', 'Fira Code', 'Cascadia Mono', 'Roboto Mono', monospace",
fontSize: 13,
lineHeight: 1.15,
letterSpacing: 0,
scrollback: 10000,
theme: hermesTheme,
allowProposedApi: true,
allowTransparency: false,
convertEol: false,
macOptionIsMeta: true,
rightClickSelectsWord: false,
});
const fitAddon = new FitAddon.FitAddon();
const webLinksAddon = new WebLinksAddon.WebLinksAddon(function (event, uri) {
// Delegate to Android so taps open the system browser.
if (window.AndroidBridge && window.AndroidBridge.onLink) {
window.AndroidBridge.onLink(uri);
}
});
// Search addon — vendored from @xterm/addon-search@0.15.0. Loaded
// here so the Compose-side TerminalSearchBar can call findNext /
// findPrevious through the window.search* shims defined below.
// Wrapped in a try because if vendoring ever fails or the global
// shape changes, we want the rest of the terminal to keep working
// rather than turning the whole tab into a JS error.
let searchAddon = null;
try {
if (window.SearchAddon && window.SearchAddon.SearchAddon) {
searchAddon = new SearchAddon.SearchAddon();
}
} catch (err) {
console.warn('SearchAddon init failed: ' + err);
}
term.loadAddon(fitAddon);
term.loadAddon(webLinksAddon);
if (searchAddon) {
try { term.loadAddon(searchAddon); } catch (err) {
console.warn('SearchAddon loadAddon failed: ' + err);
searchAddon = null;
}
}
term.open(document.getElementById('terminal'));
try { fitAddon.fit(); } catch (_) { /* container not measured yet */ }
// ── Outbound: terminal → Android ──────────────────────────────────
term.onData(function (data) {
if (window.AndroidBridge && window.AndroidBridge.onInput) {
window.AndroidBridge.onInput(data);
}
});
term.onBinary(function (data) {
// Binary input (very rare — only used for mouse tracking byte paths).
if (window.AndroidBridge && window.AndroidBridge.onInput) {
window.AndroidBridge.onInput(data);
}
});
let lastCols = 0;
let lastRows = 0;
term.onResize(function (size) {
if (size.cols === lastCols && size.rows === lastRows) return;
lastCols = size.cols;
lastRows = size.rows;
if (window.AndroidBridge && window.AndroidBridge.onResize) {
window.AndroidBridge.onResize(size.cols, size.rows);
}
});
// ── Inbound: Android → terminal ───────────────────────────────────
// Base64-encoded payloads avoid JS string-escaping headaches when the
// stream contains control characters, raw escape sequences, or bytes
// that would need careful quoting in evaluateJavascript.
let writeCount = 0;
window.writeTerminal = function (b64) {
if (!b64) return;
try {
const binary = atob(b64);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i++) {
bytes[i] = binary.charCodeAt(i);
}
term.write(bytes);
writeCount++;
// Diagnostic: dump xterm geometry + container metrics every few
// writes so we can see from logcat whether xterm has a sane
// viewport and rows. Routed through onConsoleMessage → the
// TerminalWebView tag.
if (writeCount <= 2 || writeCount % 5 === 0) {
const el = document.getElementById('terminal');
console.log('xterm state: cols=' + term.cols +
' rows=' + term.rows +
' container=' + el.clientWidth + 'x' + el.clientHeight +
' bytesIn=' + bytes.length);
}
} catch (err) {
console.error('writeTerminal decode error: ' + err);
}
};
// Called by Kotlin's addOnLayoutChangeListener with the WebView's
// actual dimensions in CSS pixels. We force body + #terminal to those
// sizes explicitly because Chromium WebView's internal viewport does
// NOT reliably track native view resizes on Android — `height: 100%`
// resolves against a stale 0-height viewport on the first composition
// pass and never recovers without this manual override. With the
// explicit style set, xterm's fit() reads the correct clientHeight
// and computes a sane cols/rows.
window.refit = function (widthCss, heightCss) {
try {
if (typeof widthCss === 'number' && typeof heightCss === 'number' &&
widthCss > 0 && heightCss > 0) {
document.documentElement.style.width = widthCss + 'px';
document.documentElement.style.height = heightCss + 'px';
document.body.style.width = widthCss + 'px';
document.body.style.height = heightCss + 'px';
const el = document.getElementById('terminal');
if (el) {
el.style.width = widthCss + 'px';
el.style.height = heightCss + 'px';
}
}
fitAddon.fit();
const el2 = document.getElementById('terminal');
console.log('refit: cols=' + term.cols + ' rows=' + term.rows +
' container=' + (el2 ? el2.clientWidth + 'x' + el2.clientHeight : '?') +
' requested=' + widthCss + 'x' + heightCss);
} catch (err) {
console.error('fit error: ' + err);
}
};
window.setFontSize = function (size) {
try {
term.options.fontSize = size;
fitAddon.fit();
} catch (_) {}
};
window.focusTerminal = function () {
try { term.focus(); } catch (_) {}
};
window.clearTerminal = function () {
try { term.clear(); } catch (_) {}
};
window.pasteToTerminal = function (text) {
if (!text) return;
try {
term.paste(text);
} catch (_) {
// Fallback — send as plain input
if (window.AndroidBridge && window.AndroidBridge.onInput) {
window.AndroidBridge.onInput(text);
}
}
};
// Refit on any container size change. ResizeObserver is more reliable
// than `window.resize` on Android WebView — the window doesn't always
// fire `resize` when Compose resizes the parent View, so the initial
// fit can latch at rows=1 (before Compose has measured) and never
// recompute. ResizeObserver watches the element's content box directly
// and fires whenever Compose grows/shrinks the WebView. Without this,
// `ls` output scrolls straight off a 1-row viewport and users see
// "typed a command, everything cleared, no output".
let resizeTimeout = null;
const terminalEl = document.getElementById('terminal');
const scheduleFit = function () {
if (resizeTimeout) clearTimeout(resizeTimeout);
resizeTimeout = setTimeout(function () {
try { fitAddon.fit(); } catch (_) {}
}, 80);
};
if (typeof ResizeObserver !== 'undefined') {
const resizeObserver = new ResizeObserver(scheduleFit);
resizeObserver.observe(terminalEl);
}
// Keep the window.resize listener too — it still fires on orientation
// change and covers edge cases where ResizeObserver isn't available.
window.addEventListener('resize', scheduleFit);
// Signal readiness to Android after two rAFs so the initial layout
// has a chance to settle. ResizeObserver above will keep re-fitting
// if the container grows after this first onReady call, and the
// `resize(cols, rows)` bridge call syncs the server side.
requestAnimationFrame(function () {
try { fitAddon.fit(); } catch (_) {}
requestAnimationFrame(function () {
if (window.AndroidBridge && window.AndroidBridge.onReady) {
window.AndroidBridge.onReady(term.cols, term.rows);
}
});
});
// Diagnostic hook — Kotlin can read back the current grid size.
window.getTerminalGeometry = function () {
return JSON.stringify({ cols: term.cols, rows: term.rows });
};
// ── Search shims — driven by TerminalSearchBar.kt ────────────────
// Each function returns a boolean so the Kotlin side can check
// whether the search actually matched (for "no results" UX).
// The try/catch is defensive: if the addon is missing or its API
// changes, the terminal stays usable instead of throwing.
window.searchNext = function (text) {
if (!text || !searchAddon) return false;
try { return !!searchAddon.findNext(text); } catch (_) { return false; }
};
window.searchPrev = function (text) {
if (!text || !searchAddon) return false;
try { return !!searchAddon.findPrevious(text); } catch (_) { return false; }
};
window.clearSearch = function () {
if (!searchAddon) return;
try {
if (typeof searchAddon.clearDecorations === 'function') {
searchAddon.clearDecorations();
}
} catch (_) { /* old addon-search builds */ }
};
})();
</script>
</body>
</html>
+218
View File
@@ -0,0 +1,218 @@
/**
* Copyright (c) 2014 The xterm.js authors. All rights reserved.
* Copyright (c) 2012-2013, Christopher Jeffrey (MIT License)
* https://github.com/chjj/term.js
* @license MIT
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*
* Originally forked from (with the author's permission):
* Fabrice Bellard's javascript vt100 for jslinux:
* http://bellard.org/jslinux/
* Copyright (c) 2011 Fabrice Bellard
* The original design remains. The terminal itself
* has been extended to include xterm CSI codes, among
* other features.
*/
/**
* Default styles for xterm.js
*/
.xterm {
cursor: text;
position: relative;
user-select: none;
-ms-user-select: none;
-webkit-user-select: none;
}
.xterm.focus,
.xterm:focus {
outline: none;
}
.xterm .xterm-helpers {
position: absolute;
top: 0;
/**
* The z-index of the helpers must be higher than the canvases in order for
* IMEs to appear on top.
*/
z-index: 5;
}
.xterm .xterm-helper-textarea {
padding: 0;
border: 0;
margin: 0;
/* Move textarea out of the screen to the far left, so that the cursor is not visible */
position: absolute;
opacity: 0;
left: -9999em;
top: 0;
width: 0;
height: 0;
z-index: -5;
/** Prevent wrapping so the IME appears against the textarea at the correct position */
white-space: nowrap;
overflow: hidden;
resize: none;
}
.xterm .composition-view {
/* TODO: Composition position got messed up somewhere */
background: #000;
color: #FFF;
display: none;
position: absolute;
white-space: nowrap;
z-index: 1;
}
.xterm .composition-view.active {
display: block;
}
.xterm .xterm-viewport {
/* On OS X this is required in order for the scroll bar to appear fully opaque */
background-color: #000;
overflow-y: scroll;
cursor: default;
position: absolute;
right: 0;
left: 0;
top: 0;
bottom: 0;
}
.xterm .xterm-screen {
position: relative;
}
.xterm .xterm-screen canvas {
position: absolute;
left: 0;
top: 0;
}
.xterm .xterm-scroll-area {
visibility: hidden;
}
.xterm-char-measure-element {
display: inline-block;
visibility: hidden;
position: absolute;
top: 0;
left: -9999em;
line-height: normal;
}
.xterm.enable-mouse-events {
/* When mouse events are enabled (eg. tmux), revert to the standard pointer cursor */
cursor: default;
}
.xterm.xterm-cursor-pointer,
.xterm .xterm-cursor-pointer {
cursor: pointer;
}
.xterm.column-select.focus {
/* Column selection mode */
cursor: crosshair;
}
.xterm .xterm-accessibility:not(.debug),
.xterm .xterm-message {
position: absolute;
left: 0;
top: 0;
bottom: 0;
right: 0;
z-index: 10;
color: transparent;
pointer-events: none;
}
.xterm .xterm-accessibility-tree:not(.debug) *::selection {
color: transparent;
}
.xterm .xterm-accessibility-tree {
user-select: text;
white-space: pre;
}
.xterm .live-region {
position: absolute;
left: -9999px;
width: 1px;
height: 1px;
overflow: hidden;
}
.xterm-dim {
/* Dim should not apply to background, so the opacity of the foreground color is applied
* explicitly in the generated class and reset to 1 here */
opacity: 1 !important;
}
.xterm-underline-1 { text-decoration: underline; }
.xterm-underline-2 { text-decoration: double underline; }
.xterm-underline-3 { text-decoration: wavy underline; }
.xterm-underline-4 { text-decoration: dotted underline; }
.xterm-underline-5 { text-decoration: dashed underline; }
.xterm-overline {
text-decoration: overline;
}
.xterm-overline.xterm-underline-1 { text-decoration: overline underline; }
.xterm-overline.xterm-underline-2 { text-decoration: overline double underline; }
.xterm-overline.xterm-underline-3 { text-decoration: overline wavy underline; }
.xterm-overline.xterm-underline-4 { text-decoration: overline dotted underline; }
.xterm-overline.xterm-underline-5 { text-decoration: overline dashed underline; }
.xterm-strikethrough {
text-decoration: line-through;
}
.xterm-screen .xterm-decoration-container .xterm-decoration {
z-index: 6;
position: absolute;
}
.xterm-screen .xterm-decoration-container .xterm-decoration.xterm-decoration-top-layer {
z-index: 7;
}
.xterm-decoration-overview-ruler {
z-index: 8;
position: absolute;
top: 0;
right: 0;
pointer-events: none;
}
.xterm-decoration-top {
z-index: 2;
position: relative;
}
File diff suppressed because one or more lines are too long
+28 -28
View File
@@ -1,32 +1,32 @@
v0.1.0 — First Release
v0.4.0 — Bridge Feature Expansion
Chat
• Direct API chat via SSE streaming
• Markdown rendering — code blocks, bold, italic, links
• Session management — create, switch, rename, delete
• Message history with auto-titles
• Reasoning display (collapsible thinking blocks)
• Personality picker with dynamic server personalities
• Command palette — 29+ commands + server skills
• Token & cost tracking per message
• File attachments — images, documents, any file type
• Message queuing — send while agent is streaming
Bridge Channel
• Long-press and drag gestures via ActionExecutor
• Clipboard read/write over bridge protocol
• Intent send (open URLs, share, launch activities)
• Location, contacts, call, and SMS bridge commands
• Multi-window screen reading in ScreenReader
• Macro recording and playback support
• Expanded BridgeCommandHandler path inventory
Animation
• ASCII morphing sphere on empty chat screen
• Ambient mode — fullscreen sphere (toggle in header)
• Subtle sphere behind messages at 15% opacity
• Animation controls in Settings > Appearance
Voice
• Voice-to-bridge intent routing (sideload)
• VoiceBridgeIntentHandler with per-flavor factory
• VoiceIntentClassifier regex phone-control detection
• Local chat trace for voice actions
App
• Material You theming (light/dark/auto)
• QR code pairing for quick setup
• Stats for Nerds — response times, health metrics
• Offline detection with reconnect
• Developer Options — tap version 7x to unlock experimental features
• Configurable limits — attachment size, message length
Safety
• Tiered permission system per bridge command category
• JIT error surfacing for missing permissions
• BridgeSafetyManager blocklist + destructive-verb confirmation
• Auto-disable timer for bridge sessions
Security
• API keys in EncryptedSharedPreferences
• Network security config for localhost
• Feature gating for unfinished features
Notifications
• HermesNotificationCompanion — opt-in NotificationListenerService
• Notification forwarding to agent via ChannelMultiplexer
• Bounded relay-side deque (100 entries)
Prior releases
• v0.3.0 — Bridge channel, voice mode, notification companion, two build flavors
• v0.2.0 — Voice foundation, terminal preview, TOFU cert pinning
• v0.1.0 — Chat, sessions, QR pairing, encrypted storage
@@ -1,14 +1,37 @@
package com.hermesandroid.relay
import android.app.Application
import android.os.Build
import androidx.compose.ui.ComposeUiFlags
import androidx.compose.ui.ExperimentalComposeUiApi
import com.hermesandroid.relay.data.AppAnalytics
import com.hermesandroid.relay.power.WakeLockManager
class HermesRelayApp : Application() {
@OptIn(ExperimentalComposeUiApi::class)
override fun attachBaseContext(base: android.content.Context?) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.VANILLA_ICE_CREAM) {
ComposeUiFlags.isAdaptiveRefreshRateEnabled = false
}
super.attachBaseContext(base)
}
@OptIn(ExperimentalComposeUiApi::class)
override fun onCreate() {
super.onCreate()
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.VANILLA_ICE_CREAM) {
// Compose's adaptive refresh-rate hint path on API 35 can emit
// `setRequestedFrameRate frameRate=NaN` from inside AndroidComposeView
// on every draw pass. Disable ARR globally until the upstream fix lands.
ComposeUiFlags.isAdaptiveRefreshRateEnabled = false
}
instance = this
AppAnalytics.initialize(this)
// A8 — wire the bridge-gesture wake-lock wrapper so
// ActionExecutor.tap/tapText/typeText/swipe/scroll can hold
// a partial wake lock while dispatching.
WakeLockManager.initialize(this)
}
companion object {
@@ -1,22 +1,63 @@
package com.hermesandroid.relay
import android.animation.ObjectAnimator
import android.content.Context
import android.content.Intent
import android.media.projection.MediaProjectionManager
import android.os.Bundle
import android.util.Log
import android.view.View
import android.view.animation.DecelerateInterpolator
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.enableEdgeToEdge
import androidx.activity.result.contract.ActivityResultContracts
import androidx.activity.viewModels
import androidx.core.animation.doOnEnd
import androidx.core.splashscreen.SplashScreen.Companion.installSplashScreen
import com.hermesandroid.relay.accessibility.ScreenCaptureRequester
import com.hermesandroid.relay.bridge.BridgeForegroundService
import com.hermesandroid.relay.ui.RelayApp
import com.hermesandroid.relay.util.ComposeArrWorkaround
import com.hermesandroid.relay.util.NavRouteRequest
import com.hermesandroid.relay.viewmodel.ConnectionViewModel
class MainActivity : ComponentActivity() {
private val connectionViewModel: ConnectionViewModel by viewModels()
// === PHASE3-bridge-ui-followup: MediaProjection consent flow ===
// ActivityResultLauncher for the system screen-capture consent dialog.
// Must be registered BEFORE the activity reaches STARTED state, hence
// declared as a property (registerForActivityResult is safe to call
// from a property initializer on ComponentActivity).
//
// We do NOT call MediaProjectionHolder directly from here. On Android
// 14+, getMediaProjection() must run from inside a foreground service
// that has already called startForeground(type=mediaProjection), and
// that startForeground call must happen AFTER consent. So we hand the
// result off to BridgeForegroundService, which:
// 1. Upgrades its FGS type to SPECIAL_USE | MEDIA_PROJECTION
// 2. Calls MediaProjectionHolder.acceptGrantInsideForegroundService
// 3. Stores the projection in the holder's StateFlow
// BridgeViewModel observes that flow and refreshes the UI immediately.
//
// ScreenCaptureRequester is a process-singleton rendezvous so the
// BridgeViewModel (which has no Activity reference) can ask us to
// launch the dialog without leaking this Activity through the VM.
private val mediaProjectionLauncher = registerForActivityResult(
ActivityResultContracts.StartActivityForResult()
) { result ->
val data = result.data
if (result.resultCode == RESULT_OK && data != null) {
Log.i(TAG, "MediaProjection consent granted — handing off to FGS")
BridgeForegroundService.grantMediaProjection(this, result.resultCode, data)
} else {
Log.i(TAG, "MediaProjection consent rejected (resultCode=${result.resultCode})")
}
}
// === END PHASE3-bridge-ui-followup ===
override fun onCreate(savedInstanceState: Bundle?) {
val splashScreen = installSplashScreen()
@@ -41,8 +82,75 @@ class MainActivity : ComponentActivity() {
super.onCreate(savedInstanceState)
enableEdgeToEdge()
// === PHASE3-bridge-ui-followup: install MediaProjection requester ===
// Hand the launcher to the process-singleton rendezvous so
// BridgeViewModel.requestScreenCapture() can fire the consent
// dialog without holding an Activity reference.
ScreenCaptureRequester.install {
val mgr = getSystemService(Context.MEDIA_PROJECTION_SERVICE)
as MediaProjectionManager
try {
mediaProjectionLauncher.launch(mgr.createScreenCaptureIntent())
} catch (t: Throwable) {
Log.w(TAG, "failed to launch MediaProjection consent: ${t.message}")
}
}
// === END PHASE3-bridge-ui-followup ===
// === PHASE3-safety-rails-followup: deep-link nav route from external intents ===
// Foreground services, broadcast receivers, and shortcut intents can
// attach EXTRA_NAV_ROUTE to request that RelayApp navigate to a
// specific Compose route on launch. The actual navigation happens
// in RelayApp's NavRouteRequest collector — we just pump the request
// into the SharedFlow here.
consumeNavRouteIntent(intent)
// === END PHASE3-safety-rails-followup ===
setContent {
RelayApp()
}
window.decorView.post {
ComposeArrWorkaround.disableForViewTree(window.decorView)
}
}
override fun onNewIntent(intent: Intent) {
super.onNewIntent(intent)
// === PHASE3-safety-rails-followup: deep-link nav route on re-launch ===
// Same as onCreate but for the singleTask / FLAG_ACTIVITY_CLEAR_TOP
// path: when the app is already running and the foreground service's
// PendingIntent re-launches us, the new intent comes through here
// instead of onCreate. RelayApp's collector handles both cases.
setIntent(intent)
consumeNavRouteIntent(intent)
// === END PHASE3-safety-rails-followup ===
}
private fun consumeNavRouteIntent(intent: Intent?) {
val route = intent?.getStringExtra(EXTRA_NAV_ROUTE) ?: return
if (route.isBlank()) return
NavRouteRequest.tryRequest(route)
}
override fun onDestroy() {
// === PHASE3-bridge-ui-followup: clear MediaProjection requester ===
// Drop the launcher closure so we don't hold a stale Activity ref
// after destroy. ScreenCaptureRequester.request() will return false
// until the next MainActivity instance reinstalls itself.
ScreenCaptureRequester.uninstall()
// === END PHASE3-bridge-ui-followup ===
super.onDestroy()
}
companion object {
private const val TAG = "MainActivity"
/**
* Intent extra carrying a Compose nav route. Set by foreground services
* (and any other external launcher) on the `Intent(this, MainActivity::class.java)`
* they fire to request RelayApp navigate to a specific destination on
* launch / re-launch.
*/
const val EXTRA_NAV_ROUTE = "com.hermesandroid.relay.EXTRA_NAV_ROUTE"
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,297 @@
package com.hermesandroid.relay.accessibility
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.os.BatteryManager
import android.os.Build
import android.os.PowerManager
import android.provider.Settings
import android.util.Log
import com.hermesandroid.relay.bridge.BridgeSafetyManager
import com.hermesandroid.relay.network.ChannelMultiplexer
import com.hermesandroid.relay.network.models.Envelope
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.isActive
import kotlinx.coroutines.launch
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
/**
* Phase 3 — accessibility `accessibility-runtime`
*
* Coroutine-driven status reporter. Emits a `bridge.status` envelope every
* [TICK_MS] (default 30 seconds) describing the phone's current state.
*
* ## Wire format (Phase 3 `phase3-status` expansion)
*
* The envelope payload is the full structured status contract that the
* relay-side [plugin.relay.channels.bridge.BridgeHandler] caches and
* serves via `GET /bridge/status`. It MUST match the JSON contract used
* by `android_phone_status()`:
*
* ```json
* {
* "device": {
* "name": "SM-S921U",
* "battery_percent": 78,
* "screen_on": true,
* "current_app": "com.android.chrome"
* },
* "bridge": {
* "master_enabled": true,
* "accessibility_granted": true,
* "screen_capture_granted": true,
* "overlay_granted": true,
* "notification_listener_granted": true
* },
* "safety": {
* "blocklist_count": 30,
* "destructive_verbs_count": 12,
* "auto_disable_minutes": 30,
* "auto_disable_at_ms": null
* }
* }
* ```
*
* The legacy top-level keys (`screen_on`, `battery`, `current_app`,
* `accessibility_enabled`, `ts`) are ALSO emitted for backwards
* compatibility with any consumer that hasn't been updated to read the
* nested `device` / `bridge` / `safety` groups yet. The fields are
* cheap and additive, and the relay caches the whole payload verbatim.
*
* ## Lifecycle
*
* The reporter is a no-op until [start] is called, and [stop] is
* idempotent. [com.hermesandroid.relay.viewmodel.ConnectionViewModel]
* owns it and ties the lifecycle to the WSS connection — reporting
* while disconnected is a waste of battery and the multiplexer would
* drop the envelopes silently anyway.
*
* ## Out-of-band pushes
*
* Callers may invoke [pushNow] to force an immediate emission outside
* the 30 s tick — used by `ConnectionViewModel` when the master toggle
* flips, so the relay-side cache updates immediately instead of waiting
* up to 30 s for the next periodic tick.
*/
class BridgeStatusReporter(
private val context: Context,
private val multiplexer: ChannelMultiplexer,
private val scope: CoroutineScope,
) {
companion object {
private const val TAG = "BridgeStatusReporter"
private const val TICK_MS = 30_000L
}
private var job: Job? = null
/**
* Start the reporter. Safe to call multiple times — if a job is
* already running we log and return.
*/
fun start() {
if (job?.isActive == true) {
Log.v(TAG, "already running")
return
}
job = scope.launch {
// Send an immediate first tick so the agent sees fresh status
// as soon as the WSS connection comes up, rather than waiting
// up to 30s for the first periodic tick.
while (isActive) {
try {
emitTick()
} catch (t: Throwable) {
Log.w(TAG, "status emit failed: ${t.message}")
}
delay(TICK_MS)
}
}
}
fun stop() {
job?.cancel()
job = null
}
/**
* Force an immediate status emission outside the [TICK_MS] cadence.
* Useful when state changes that matter to the agent (master toggle
* flip, accessibility service connected, etc.) — rather than waiting
* up to 30 s for the relay cache to refresh.
*
* No-op if [start] hasn't been called yet — in that case the next
* start() will fire a tick anyway.
*/
fun pushNow() {
if (job?.isActive != true) {
Log.v(TAG, "pushNow() called while stopped — no-op")
return
}
scope.launch {
try {
emitTick()
} catch (t: Throwable) {
Log.w(TAG, "pushNow emit failed: ${t.message}")
}
}
}
/**
* Build one status envelope from live phone state and push it through
* the multiplexer. Exposed as internal-ish so unit tests can drive
* a single tick without spinning the coroutine.
*/
internal fun emitTick() {
val screenOn = try {
val pm = context.getSystemService(Context.POWER_SERVICE) as PowerManager
pm.isInteractive
} catch (t: Throwable) {
Log.v(TAG, "screenOn probe failed: ${t.message}")
false
}
val battery = try {
// Prefer the BatteryManager property API — it's the only
// always-accurate source on modern Android. The legacy sticky
// intent path is fine too but we'd have to parse two fields.
val bm = context.getSystemService(Context.BATTERY_SERVICE) as BatteryManager
bm.getIntProperty(BatteryManager.BATTERY_PROPERTY_CAPACITY)
} catch (t: Throwable) {
Log.v(TAG, "battery probe failed: ${t.message}")
-1
}
// If the property API returns 0 (some OEM firmwares do) fall back
// to the sticky intent read.
val batteryFinal = if (battery <= 0) readBatteryViaIntent() else battery
val currentApp = HermesAccessibilityService.instance?.currentApp
val accessibilityGranted = HermesAccessibilityService.instance != null
val masterEnabled = HermesAccessibilityService.instance?.isMasterEnabled() ?: false
// Screen-capture grant — the process-singleton holder is non-null
// iff the user granted MediaProjection consent this session.
val screenCaptureGranted = try {
MediaProjectionHolder.projection != null
} catch (t: Throwable) {
Log.v(TAG, "screen_capture probe failed: ${t.message}")
false
}
val overlayGranted = try {
Settings.canDrawOverlays(context)
} catch (t: Throwable) {
Log.v(TAG, "overlay probe failed: ${t.message}")
false
}
val notificationListenerGranted = try {
val enabled = Settings.Secure.getString(
context.contentResolver,
"enabled_notification_listeners"
)
enabled?.contains(context.packageName) ?: false
} catch (t: Throwable) {
Log.v(TAG, "notification_listener probe failed: ${t.message}")
false
}
// Safety snapshot — read lazily through the process-singleton
// so we don't take a DI dep here. Falls back to zeros if the
// safety manager hasn't been built yet (cold start).
val safetyManager = BridgeSafetyManager.peek()
val safetySnapshot = safetyManager?.settings?.value
val blocklistCount = safetySnapshot?.blocklist?.size ?: 0
val destructiveVerbsCount = safetySnapshot?.destructiveVerbs?.size ?: 0
val autoDisableMinutes = safetySnapshot?.autoDisableMinutes ?: 0
val autoDisableAtMs = safetyManager?.autoDisableAtMs?.value
val deviceName = Build.MODEL ?: "unknown"
val envelope = Envelope(
channel = "bridge",
type = "bridge.status",
payload = buildJsonObject {
// Nested structured groups matching the relay HTTP contract.
put("device", buildJsonObject {
put("name", deviceName)
put("battery_percent", batteryFinal)
put("screen_on", screenOn)
put("current_app", currentApp ?: "unknown")
// Flavor context so the LLM knows upfront which build
// is connected and can avoid attempting sideload-only
// tools on a googlePlay phone. Without this the agent
// is flavor-blind until it tries android_send_sms and
// gets a 403 sideload_only — by which point it's
// already wasted a tool call and has to recover.
put("flavor", com.hermesandroid.relay.data.BuildFlavor.current)
put("application_id", com.hermesandroid.relay.data.BuildFlavor.run {
// BuildConfig.APPLICATION_ID is the resolved
// applicationId from the active flavor variant.
try {
@Suppress("KotlinConstantConditions")
com.hermesandroid.relay.BuildConfig.APPLICATION_ID
} catch (_: Throwable) {
"unknown"
}
})
})
put("bridge", buildJsonObject {
put("master_enabled", masterEnabled)
put("accessibility_granted", accessibilityGranted)
put("screen_capture_granted", screenCaptureGranted)
put("overlay_granted", overlayGranted)
put("notification_listener_granted", notificationListenerGranted)
})
put("safety", buildJsonObject {
put("blocklist_count", blocklistCount)
put("destructive_verbs_count", destructiveVerbsCount)
put("auto_disable_minutes", autoDisableMinutes)
if (autoDisableAtMs == null) {
put("auto_disable_at_ms", JsonNull)
} else {
put("auto_disable_at_ms", autoDisableAtMs)
}
})
// ── Legacy top-level fields (backwards compat) ────────
// Kept so pre-phase3-status consumers still see the
// same fields they're already parsing. New consumers
// should read from the nested `device` / `bridge`
// groups above.
put("screen_on", screenOn)
put("battery", batteryFinal)
put("current_app", currentApp ?: "unknown")
put("accessibility_enabled", accessibilityGranted)
put("ts", System.currentTimeMillis())
}
)
multiplexer.send(envelope)
}
/**
* Legacy fallback for BatteryManager.getIntProperty returning 0 —
* reads the sticky `ACTION_BATTERY_CHANGED` intent and computes
* percentage manually.
*/
private fun readBatteryViaIntent(): Int = try {
val filter = IntentFilter(Intent.ACTION_BATTERY_CHANGED)
@Suppress("UNUSED_VARIABLE")
val placeholder: BroadcastReceiver? = null
val battery = context.registerReceiver(null, filter)
val level = battery?.getIntExtra(BatteryManager.EXTRA_LEVEL, -1) ?: -1
val scale = battery?.getIntExtra(BatteryManager.EXTRA_SCALE, -1) ?: -1
if (level >= 0 && scale > 0) (level * 100 / scale) else -1
} catch (t: Throwable) {
Log.v(TAG, "battery intent fallback failed: ${t.message}")
-1
}
}
@@ -0,0 +1,323 @@
package com.hermesandroid.relay.accessibility
import android.accessibilityservice.AccessibilityService
import android.content.Context
import android.content.Intent
import android.os.Build
import android.util.Log
import android.view.accessibility.AccessibilityEvent
import android.view.accessibility.AccessibilityNodeInfo
import androidx.datastore.preferences.core.booleanPreferencesKey
import androidx.datastore.preferences.core.edit
import com.hermesandroid.relay.data.relayDataStore
import com.hermesandroid.relay.event.EventStore
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.map
import kotlinx.coroutines.launch
/**
* Phase 3 — accessibility `accessibility-runtime`
*
* Hermes's master `AccessibilityService` subclass. Provides the phone-side
* execution layer for the bridge channel: the agent reads the screen,
* taps/types/swipes, and captures screenshots through this service.
*
* # Lifecycle & singleton pattern
*
* Android instantiates `AccessibilityService` subclasses itself (through the
* manifest declaration + user opt-in in Settings → Accessibility), so we
* can't pass collaborators in via a constructor. The canonical workaround
* is a weak-referenced singleton: on [onServiceConnected] the instance
* registers itself with [Companion.instance], and on [onUnbind]/destroy it
* clears itself. Any other code that needs to dispatch gestures (the
* `BridgeCommandHandler`) asks for [instance] and bails out if it's null
* (service not running / user hasn't granted permission).
*
* # Master enable / disable
*
* The Android system toggle in `Settings → Accessibility → Hermes Relay` is
* the hard switch — if it's off we never receive events. On top of that the
* user can flip a soft master in Settings (`bridge_master_enabled`); when
* that's false we still run (Android requires it to stay connected) but we
* refuse to execute commands. [isMasterEnabled] is a StateFlow the UI
* observes and the command handler checks before dispatching actions.
*
* # Event handling
*
* We subscribe to a minimal event set — `TYPE_WINDOW_STATE_CHANGED` to
* track the foreground package (for [currentApp] status), and nothing else.
* The XML resource (flavor-provided by Agent flavor-split) controls the exact flag
* bitset. We deliberately do NOT process text / content-change events —
* those fire thousands of times a minute and are pointless for our use
* case (we read the UI tree on demand via `rootInActiveWindow`).
*/
class HermesAccessibilityService : AccessibilityService() {
companion object {
private const val TAG = "HermesA11yService"
/** Master-enable DataStore key — read + toggled from Settings UI. */
val KEY_BRIDGE_MASTER_ENABLED = booleanPreferencesKey("bridge_master_enabled")
/**
* Static reference to the live service instance, or null if the
* service is not running. Written on [onServiceConnected],
* cleared on [onUnbind] / [onDestroy].
*
* Read by [com.hermesandroid.relay.network.handlers.BridgeCommandHandler]
* and by the Bridge UI screen (bridge-ui) to check live status.
*/
@Volatile
var instance: HermesAccessibilityService? = null
private set
/**
* Observe the DataStore-backed master enable flag for the bridge.
* UI can collect this to drive the master-toggle switch; the service
* itself also polls it via [isMasterEnabled] before executing commands.
*/
fun masterEnabledFlow(context: Context): Flow<Boolean> =
context.applicationContext.relayDataStore.data
.map { prefs -> prefs[KEY_BRIDGE_MASTER_ENABLED] ?: false }
/**
* Persist a new master-enable value. Called from Settings UI when the
* user flips the switch, and from [BridgeStatusReporter] / safety
* rails when auto-disable timers fire.
*/
suspend fun setMasterEnabled(context: Context, enabled: Boolean) {
context.applicationContext.relayDataStore.edit { prefs ->
prefs[KEY_BRIDGE_MASTER_ENABLED] = enabled
}
}
}
private val screenReader = ScreenReader()
private val screenHasher = ScreenHasher()
private var _actionExecutor: ActionExecutor? = null
/**
* Service-scoped coroutine context. Started fresh in
* [onServiceConnected], cancelled in [onUnbind] / [onDestroy]. Used to
* observe the DataStore-backed master toggle and pump it into
* [cachedMasterEnabled] so [BridgeCommandHandler] can do a non-suspend
* gate check on every inbound command.
*
* Without this collector the cache stays at its `false` default and
* the gate refuses every command — that was the original Phase 3
* "bridge always disabled" bug.
*
* Re-created on each connect because cancelled `SupervisorJob`s can't
* be reused, and Android may rebind the same service instance after
* an unbind on rare config changes.
*/
@Volatile
private var serviceScope: CoroutineScope? = null
/**
* Cached package name of the currently-foregrounded app. Updated on
* every `TYPE_WINDOW_STATE_CHANGED` event. Read by the status reporter
* for the `current_app` field in `bridge.status`.
*/
@Volatile
var currentApp: String? = null
private set
/**
* Public accessor for the lazily-constructed [ActionExecutor]. The
* executor needs a back-reference to the service (for `dispatchGesture`),
* so we can only build it after Android has fully constructed us.
*/
val actionExecutor: ActionExecutor
get() = _actionExecutor ?: ActionExecutor(this).also { _actionExecutor = it }
/** Convenience wrapper — the service uses its own [ScreenReader] instance. */
val reader: ScreenReader get() = screenReader
/**
* Convenience wrapper for the service's [ScreenHasher] instance.
* Used by `BridgeCommandHandler` for `/screen_hash` + `/diff_screen`.
*/
val hasher: ScreenHasher get() = screenHasher
override fun onServiceConnected() {
super.onServiceConnected()
instance = this
Log.i(TAG, "HermesAccessibilityService connected")
// Feed the master-toggle cache for the lifetime of this service
// binding. Re-created on each connect — see [serviceScope] KDoc.
val scope = CoroutineScope(Dispatchers.Default + SupervisorJob())
serviceScope = scope
scope.launch {
masterEnabledFlow(this@HermesAccessibilityService).collect { enabled ->
cachedMasterEnabled = enabled
Log.d(TAG, "master toggle cached: $enabled")
}
}
}
override fun onAccessibilityEvent(event: AccessibilityEvent?) {
if (event == null) return
when (event.eventType) {
AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED -> {
val pkg = event.packageName?.toString()
if (!pkg.isNullOrBlank()) {
currentApp = pkg
}
}
else -> {
// Other event types are declared in the config XML for
// future safety rails (blocklist enforcement via
// content-change events) but we deliberately no-op here
// today. Filtering happens inside the config flag bitset
// so we never even receive most events.
}
}
// === PHASE3-event-stream: B1 android_events / android_event_stream ===
// Feed signal-rich events into the bounded ring buffer when the
// agent has explicitly opted in via android_event_stream(true).
// EventStore does its own type-filter + throttle + thread-safety
// — we just hand it the raw event.
if (EventStore.isStreaming) {
EventStore.append(event)
}
// === END PHASE3-event-stream ===
}
override fun onInterrupt() {
// Called by the system when it wants us to drop any in-flight work.
// We don't queue long-running operations — every bridge command is
// fire-and-forget with its own callback — so there's nothing to
// cancel here.
Log.i(TAG, "onInterrupt — accessibility service asked to stop work")
}
override fun onUnbind(intent: Intent?): Boolean {
Log.i(TAG, "HermesAccessibilityService unbinding")
if (instance === this) instance = null
serviceScope?.cancel()
serviceScope = null
cachedMasterEnabled = false
return super.onUnbind(intent)
}
override fun onDestroy() {
if (instance === this) instance = null
serviceScope?.cancel()
serviceScope = null
cachedMasterEnabled = false
super.onDestroy()
}
/**
* Soft master-toggle cache. Fed by the [serviceScope] collector started
* in [onServiceConnected] — DO NOT write directly. Read by
* [BridgeCommandHandler] on every inbound command via [isMasterEnabled].
*
* Volatile because the writer runs on Dispatchers.Default and the
* reader runs on whichever multiplexer thread the bridge envelope
* arrives on.
*/
@Volatile
private var cachedMasterEnabled: Boolean = false
fun isMasterEnabled(): Boolean = cachedMasterEnabled
/**
* Snapshot the current root node of the active window. Returns null if
* no window is focused or the system refuses access (e.g. IME window).
*
* Callers must `recycle()` the returned node when done.
*
* On API 34+, [AccessibilityNodeInfo.recycle] is deprecated but still
* safe to call — the system just no-ops. We support min SDK 26 so we
* keep calling it for the older branch.
*
* Prefer [snapshotAllWindows] for /screen + tap_text / type_text — it
* catches system overlays, popup menus, and the notification shade.
* This single-root form is retained for callers that genuinely only
* care about the foregrounded app window (e.g. [ActionExecutor.scroll],
* which uses the active window's bounds as the scroll gesture's frame).
*/
fun snapshotRoot(): AccessibilityNodeInfo? = try {
rootInActiveWindow
} catch (t: Throwable) {
Log.w(TAG, "rootInActiveWindow threw: ${t.message}")
null
}
/**
* P1 — Snapshot the root nodes of **all** live accessibility windows,
* top-of-stack first. Catches system overlays, popup menus, the
* notification shade when pulled down, permission dialogs, and
* multi-window split-screen state — all of which [snapshotRoot] misses.
*
* Every [android.view.accessibility.AccessibilityWindowInfo.getRoot]
* returns a fresh [AccessibilityNodeInfo] that the caller MUST
* `recycle()` when done. Recycling is the single biggest landmine in
* this area — leaking window roots makes subsequent gesture dispatches
* fail silently because the system runs out of node handles.
*
* # Fallback semantics
*
* `service.windows` returns an empty list unless the accessibility
* config XML requests `flagRetrieveInteractiveWindows`. That flag is
* **only** set in the `sideload` flavor — the `googlePlay` flavor
* deliberately runs on the conservative config subset to pass Play
* Store policy review. When `windows` is empty (or throws, or every
* window's root is null) we fall back to a single-element list
* wrapping [rootInActiveWindow], preserving pre-P1 behaviour on
* `googlePlay` builds.
*
* Returns an empty list only if the service cannot read any window
* root at all (e.g. lock screen, master-off state). Callers should
* treat an empty return the same as `snapshotRoot() == null`.
*/
fun snapshotAllWindows(): List<AccessibilityNodeInfo> {
// Try the full multi-window path first. `service.windows` is
// available on API 21+ and we target min SDK 26, so no version
// guard needed.
val windowList: List<android.view.accessibility.AccessibilityWindowInfo> = try {
this.windows ?: emptyList()
} catch (t: Throwable) {
Log.w(TAG, "service.windows threw: ${t.message}")
emptyList()
}
if (windowList.isNotEmpty()) {
val roots = ArrayList<AccessibilityNodeInfo>(windowList.size)
for (wi in windowList) {
val root: AccessibilityNodeInfo? = try {
wi.root
} catch (t: Throwable) {
Log.w(TAG, "AccessibilityWindowInfo.getRoot threw: ${t.message}")
null
}
if (root != null) roots.add(root)
}
if (roots.isNotEmpty()) return roots
// Else fall through — all window roots were null, try the
// single-window fallback in case it can still see the active
// window.
}
// googlePlay fallback (or empty-windows edge case): mimic the
// pre-P1 single-root behaviour.
val active = snapshotRoot() ?: return emptyList()
return listOf(active)
}
/**
* Best-effort indicator — `true` when the runtime is >= API 26 (always
* true on this app, we target 26+). Exposed for completeness so the
* Bridge UI can render a compile-time capabilities badge without
* reading BuildConfig directly.
*/
val supportsGestures: Boolean get() = Build.VERSION.SDK_INT >= Build.VERSION_CODES.O
}
@@ -0,0 +1,631 @@
package com.hermesandroid.relay.accessibility
import android.content.Context
import android.content.Intent
import android.graphics.Bitmap
import android.graphics.PixelFormat
import android.hardware.display.DisplayManager
import android.hardware.display.VirtualDisplay
import android.media.Image
import android.media.ImageReader
import android.media.projection.MediaProjection
import android.media.projection.MediaProjectionManager
import android.os.Handler
import android.os.HandlerThread
import android.util.DisplayMetrics
import android.util.Log
import android.view.WindowManager
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.MultipartBody
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.RequestBody.Companion.toRequestBody
import java.io.ByteArrayOutputStream
import java.io.File
import java.io.IOException
import java.util.concurrent.TimeUnit
/**
* Phase 3 — accessibility `accessibility-runtime`
*
* Captures the phone's screen via the Android [MediaProjection] API, encodes
* it as PNG, and publishes it to the relay so the agent can fetch it.
*
* ## Permission flow (blocker for Agent bridge-ui / UI to wire)
*
* `MediaProjection` cannot be granted by the app itself — it needs an
* explicit user consent dialog per session, launched via
* [MediaProjectionManager.createScreenCaptureIntent] from an `Activity`.
* The resulting `Intent` is then passed to [MediaProjectionManager.getMediaProjection]
* to build an actual projection.
*
* Because the grant lives on an `Activity` result, this class can only
* provide the capture loop — **the consent flow must be wired by the
* Bridge UI screen (Agent bridge-ui)**. Suggested contract:
*
* 1. `BridgeScreen` holds an `ActivityResultLauncher<Intent>` registered
* with `ActivityResultContracts.StartActivityForResult()`.
* 2. On "Enable screenshots" tap, bridge-ui calls
* `MediaProjectionManager.createScreenCaptureIntent()` and launches it.
* 3. On result, bridge-ui passes `(resultCode, data)` into a central holder
* (e.g. a ViewModel singleton or [MediaProjectionHolder]).
* 4. [ScreenCapture] reads from that holder on each capture call and
* rebuilds a `MediaProjection` when needed. The projection will need
* to be backed by a foreground service on Android 10+ — Agent safety-rails
* owns the persistent-notification service declaration.
*
* Until that wiring lands, this class will fail gracefully with
* `Result.failure(IllegalStateException("MediaProjection not granted"))`
* and the agent will see the error text in the `bridge.response` body.
*
* ## Upload path — current limitation
*
* The relay's existing `/media/register` endpoint is **loopback-only and
* path-based** (`plugin/relay/media.py`) — it registers a file path that
* already exists on the relay host, with an optional content type and
* filename. The phone is by definition not on the relay host, so it has no
* usable path to register.
*
* Two options exist for bridging this gap:
*
* 1. **New relay endpoint** (preferred) — `POST /media/upload` accepts
* `multipart/form-data` with the PNG bytes, writes to a sandboxed tmp
* dir (`tempfile.gettempdir()`), then calls `MediaRegistry.register()`
* on the resulting path. Wire shape mirrors `/voice/transcribe`. This
* is a clean server-side change that Agent bridge-server could land in parallel.
*
* 2. **Local-only screenshots** (fallback) — the phone writes the PNG to
* its own cache dir, emits `MEDIA:file://<cache_path>` in the response,
* and the agent fetches it via an on-device tool (N/A — the agent runs
* on the host, not the phone). So option 2 doesn't actually work.
*
* This class implements option 1 via [uploadViaMultipart]. If the endpoint
* returns 404 (not yet deployed), we surface the error to the agent with a
* clear message. **Agent bridge-server owns the `POST /media/upload` endpoint** — it's
* the only remaining server-side work to complete Tier 1 screenshots.
*
* ## Thread model
*
* `ImageReader` delivers frames on a background `HandlerThread` we own.
* The PNG encode runs on [Dispatchers.IO] via [withContext]. The HTTP
* upload is also IO-dispatched. All three can be cancelled by the caller.
*/
class ScreenCapture(
private val context: Context,
private val httpClient: OkHttpClient,
private val relayUrlProvider: () -> String?,
private val sessionTokenProvider: suspend () -> String?,
private val mediaProjectionProvider: () -> MediaProjection?,
) {
companion object {
private const val TAG = "ScreenCapture"
/** PNG quality is a no-op for PNG, but Bitmap.compress expects the arg. */
private const val PNG_QUALITY = 100
/**
* ImageReader buffer count — we only need the latest frame, but the
* reader requires at least 2 slots so the producer (VirtualDisplay)
* can keep writing while we acquire the previous one.
*/
private const val MAX_IMAGES = 2
/** Capture timeout — if no frame arrives in this window, fail loudly. */
private const val CAPTURE_TIMEOUT_MS = 2_500L
}
// === PHASE3-bridge-ui-followup: MediaProjection reuse fix ===
//
// Starting in Android 14 (API 34), each MediaProjection instance supports
// exactly ONE createVirtualDisplay() call per session. Calling it a
// second time throws with the error:
// "Don't re-use the resultData... Don't take multiple captures by
// invoking MediaProjection#createVirtualDisplay multiple times on
// the same instance."
//
// The old implementation built a fresh VirtualDisplay + ImageReader on
// EVERY screenshot call and released it after, which worked on Android
// 13 and below but breaks the second /screenshot request on 14+.
//
// Fix: keep the VirtualDisplay + ImageReader + HandlerThread alive
// across captures, keyed by the MediaProjection instance. Rebuild only
// when the projection reference changes (fresh consent grant) or the
// dimensions change (orientation flip). The ImageReader's
// setOnImageAvailableListener drains the buffer continuously; each
// captureAndUpload() installs a one-shot [pendingCapture] callback
// that fires on the next frame.
//
// Thread model:
// - `captureMutex` serializes concurrent captureAndUpload() calls
// - `cacheLock` protects the cached-state fields against the listener
// thread (which runs on `captureThread.looper`) racing with rebuild
// - The listener always acquires the latest frame; the pendingCapture
// deferred is completed with the encoded PNG bytes inside the
// listener callback on the capture thread.
private val captureMutex = kotlinx.coroutines.sync.Mutex()
private val cacheLock = Any()
private var cachedProjection: MediaProjection? = null
private var cachedReader: ImageReader? = null
private var cachedDisplay: VirtualDisplay? = null
private var cachedThread: HandlerThread? = null
private var cachedHandler: Handler? = null
private var cachedWidth: Int = 0
private var cachedHeight: Int = 0
private var cachedDensity: Int = 0
/**
* Pending capture request, populated on [captureAndUpload] entry and
* completed by the persistent ImageReader listener on the next frame.
* `@Volatile` so the listener thread sees assignments made from the
* capture coroutine. AtomicReference-style swap semantics via
* [pendingCaptureRef] avoid a stale completion racing a new request.
*/
private val pendingCaptureRef = java.util.concurrent.atomic.AtomicReference<
kotlinx.coroutines.CompletableDeferred<ByteArray>?
>(null)
// === END PHASE3-bridge-ui-followup ===
/**
* Build the consent intent that `BridgeScreen` launches via an
* `ActivityResultLauncher`. Callers should launch the intent with
* `StartActivityForResult` and on success route the result to
* `BridgeForegroundService.grantMediaProjection(...)`, which handles
* the Android 14+ FGS-type-upgrade dance and stores the projection
* inside the holder. Calling
* [MediaProjectionHolder.acceptGrantInsideForegroundService] from
* outside a foreground service is a known footgun — see that method's
* docstring for the full explanation.
*/
fun createConsentIntent(): Intent =
(context.getSystemService(Context.MEDIA_PROJECTION_SERVICE) as MediaProjectionManager)
.createScreenCaptureIntent()
/**
* Capture one frame and upload it to the relay. Returns the inbound
* media marker (`MEDIA:hermes-relay://<token>`) on success so the
* bridge command handler can embed it directly in the `bridge.response`
* result.
*
* Fails fast and with clear messaging on every expected error path:
*
* - `MediaProjection not granted` → bridge-ui needs to run the consent flow
* - `relay URL not configured` / `session token missing` → pair first
* - `relay upload endpoint not found` → bridge-server needs to ship `/media/upload`
* - `capture timeout` → the virtual display never emitted a frame
*/
suspend fun captureAndUpload(): Result<String> = withContext(Dispatchers.IO) {
val projection = mediaProjectionProvider()
?: return@withContext Result.failure(
IllegalStateException(
"MediaProjection not granted — enable Bridge screenshots in the app"
)
)
// Serialize concurrent capture requests so only one pendingCapture
// is in flight at a time. The bridge command handler is the usual
// caller and it's single-threaded per /screenshot request, but the
// mutex keeps us honest if anything ever parallelizes.
val pngBytes = try {
captureMutex.withLock {
captureFrame(projection)
}
} catch (e: Exception) {
Log.w(TAG, "captureFrame failed: ${e.message}")
return@withContext Result.failure(e)
}
uploadViaMultipart(pngBytes)
}
/**
* Release any cached VirtualDisplay / ImageReader / HandlerThread. Call
* this when the MediaProjection is revoked (the holder's `onStop`
* callback, or an explicit revoke) so a subsequent grant starts with
* a clean slate. Safe to call multiple times.
*
* NOTE: this does NOT stop the MediaProjection itself — that's the
* holder's responsibility. We only own the capture pipeline built on
* top of the projection.
*/
fun releaseCache() {
synchronized(cacheLock) {
runCatching { cachedDisplay?.release() }
runCatching { cachedReader?.close() }
runCatching { cachedThread?.quitSafely() }
cachedDisplay = null
cachedReader = null
cachedThread = null
cachedHandler = null
cachedProjection = null
cachedWidth = 0
cachedHeight = 0
cachedDensity = 0
}
// Fail any pending capture with a descriptive error so the caller
// doesn't hang for the timeout.
pendingCaptureRef.getAndSet(null)?.takeIf { it.isActive }?.completeExceptionally(
IOException("capture pipeline released before frame arrived")
)
}
/**
* Capture one frame from the cached VirtualDisplay + ImageReader,
* rebuilding them if the projection reference changed or dimensions
* drifted (orientation flip). Returns the PNG-encoded bytes.
*
* The ImageReader's persistent listener is set up once inside
* [ensureCacheFor]. Each call here installs a fresh
* [pendingCaptureRef] deferred that the listener completes on the
* next frame; the listener drains non-waiting frames so the buffer
* doesn't back up while nothing is asking for screenshots.
*/
private suspend fun captureFrame(projection: MediaProjection): ByteArray {
val metrics = DisplayMetrics()
@Suppress("DEPRECATION")
(context.getSystemService(Context.WINDOW_SERVICE) as WindowManager)
.defaultDisplay.getRealMetrics(metrics)
val width = metrics.widthPixels
val height = metrics.heightPixels
val densityDpi = metrics.densityDpi
ensureCacheFor(projection, width, height, densityDpi)
val deferred = kotlinx.coroutines.CompletableDeferred<ByteArray>()
// Replace any stale pending capture (shouldn't exist because of
// the mutex, but defensive). If there's a previous one, fail it
// so nobody ends up stuck.
val previous = pendingCaptureRef.getAndSet(deferred)
if (previous != null && previous.isActive) {
previous.completeExceptionally(
IOException("capture superseded by a newer request")
)
}
return try {
kotlinx.coroutines.withTimeout(CAPTURE_TIMEOUT_MS) { deferred.await() }
} catch (e: kotlinx.coroutines.TimeoutCancellationException) {
pendingCaptureRef.compareAndSet(deferred, null)
throw IOException("screen capture timed out")
} catch (t: Throwable) {
pendingCaptureRef.compareAndSet(deferred, null)
throw t
}
}
/**
* Build (or reuse) the cached VirtualDisplay + ImageReader + HandlerThread
* for this projection. Rebuilds when:
*
* - The projection reference has changed (new consent grant landed)
* - The captured dimensions don't match the current display (orientation
* flipped, foldable opened/closed, display switched)
*
* Must be called while [captureMutex] is held so the cached fields
* aren't racing another capture.
*/
private fun ensureCacheFor(
projection: MediaProjection,
width: Int,
height: Int,
densityDpi: Int,
) {
synchronized(cacheLock) {
val projectionChanged = cachedProjection !== projection
val dimensionsChanged = width != cachedWidth || height != cachedHeight
if (!projectionChanged && !dimensionsChanged && cachedDisplay != null && cachedReader != null) {
return
}
// Tear down any stale cache before building fresh.
runCatching { cachedDisplay?.release() }
runCatching { cachedReader?.close() }
runCatching { cachedThread?.quitSafely() }
val thread = HandlerThread("HermesScreenCapture").apply { start() }
val handler = Handler(thread.looper)
val reader = ImageReader.newInstance(
width, height, PixelFormat.RGBA_8888, MAX_IMAGES
)
// Persistent listener — fires on every frame the VirtualDisplay
// produces. If there's a pending capture request, we encode
// the frame and complete it; otherwise we just drain the image
// so the ImageReader buffer stays clear.
reader.setOnImageAvailableListener({ r ->
val waiter = pendingCaptureRef.get()
if (waiter == null || !waiter.isActive) {
// Drain-and-drop — nobody's asking for a screenshot
// right now but frames are still arriving.
runCatching { r.acquireLatestImage() }.getOrNull()?.close()
return@setOnImageAvailableListener
}
var image: Image? = null
try {
image = r.acquireLatestImage()
?: return@setOnImageAvailableListener
val png = imageToPngBytes(image, width, height)
// Only complete the EXACT deferred we latched onto,
// so a stale listener firing after supersession doesn't
// resolve a new request.
if (pendingCaptureRef.compareAndSet(waiter, null)) {
waiter.complete(png)
}
} catch (t: Throwable) {
if (pendingCaptureRef.compareAndSet(waiter, null)) {
waiter.completeExceptionally(t)
}
} finally {
runCatching { image?.close() }
}
}, handler)
val display = try {
projection.createVirtualDisplay(
"hermes-bridge-capture",
width,
height,
densityDpi,
DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
reader.surface,
null,
handler,
)
} catch (t: Throwable) {
// Build failed — roll back so the next attempt tries fresh.
runCatching { reader.close() }
runCatching { thread.quitSafely() }
throw t
}
// Register the MediaProjection.Callback so if the system stops
// this projection out from under us, we release our cache
// instead of holding dead handles. The holder's own callback
// is separate — it clears projectionFlow; ours clears the
// capture pipeline. Both are safe and complementary.
try {
projection.registerCallback(object : MediaProjection.Callback() {
override fun onStop() {
releaseCache()
}
}, handler)
} catch (_: Throwable) {
// Some OEMs log but don't throw if the callback is already
// registered by another party (e.g. the holder). Ignore.
}
cachedProjection = projection
cachedReader = reader
cachedDisplay = display
cachedThread = thread
cachedHandler = handler
cachedWidth = width
cachedHeight = height
cachedDensity = densityDpi
Log.i(TAG, "screen capture pipeline built ${width}x$height dpi=$densityDpi")
}
}
/**
* Convert an [Image] from `ImageReader` into a PNG byte array. The
* plane's `rowStride` may be wider than `width * 4` — we must crop
* the stride padding before [Bitmap.copyPixelsFromBuffer].
*/
private fun imageToPngBytes(image: Image, width: Int, height: Int): ByteArray {
val plane = image.planes[0]
val buffer = plane.buffer
val pixelStride = plane.pixelStride
val rowStride = plane.rowStride
val rowPadding = rowStride - pixelStride * width
val bitmapWidth = width + rowPadding / pixelStride
val bitmap = Bitmap.createBitmap(bitmapWidth, height, Bitmap.Config.ARGB_8888)
bitmap.copyPixelsFromBuffer(buffer)
// Crop to the exact screen width if rowStride padding widened us.
val cropped = if (bitmapWidth != width) {
Bitmap.createBitmap(bitmap, 0, 0, width, height).also {
bitmap.recycle()
}
} else bitmap
val out = ByteArrayOutputStream(256 * 1024)
cropped.compress(Bitmap.CompressFormat.PNG, PNG_QUALITY, out)
cropped.recycle()
return out.toByteArray()
}
/**
* Upload PNG bytes to the relay via `POST /media/upload` (multipart).
*
* **Blocker:** this endpoint does not exist server-side yet (see class
* docstring). When it ships, it should accept a single `file` part and
* return `{"ok": true, "token": "..."}` — the same JSON shape as
* `/media/register`, minus the loopback restriction and with the
* server sandboxing the temp-file write internally.
*/
private suspend fun uploadViaMultipart(pngBytes: ByteArray): Result<String> {
val relayUrl = relayUrlProvider()?.trim().orEmpty()
if (relayUrl.isEmpty()) {
return Result.failure(IllegalStateException("Relay URL not configured"))
}
val sessionToken = sessionTokenProvider()
if (sessionToken.isNullOrBlank()) {
return Result.failure(
IllegalStateException("Relay not paired — session token missing")
)
}
val httpBase = relayUrl
.replace(Regex("^wss://", RegexOption.IGNORE_CASE), "https://")
.replace(Regex("^ws://", RegexOption.IGNORE_CASE), "http://")
.trimEnd('/')
val url = "$httpBase/media/upload"
val body = MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormDataPart(
"file",
"hermes-screenshot-${System.currentTimeMillis()}.png",
pngBytes.toRequestBody("image/png".toMediaType())
)
.build()
val fastClient = httpClient.newBuilder()
.callTimeout(15, TimeUnit.SECONDS)
.build()
val request = Request.Builder()
.url(url)
.post(body)
.header("Authorization", "Bearer $sessionToken")
.header("Accept", "application/json")
.build()
return try {
fastClient.newCall(request).execute().use { response ->
when (response.code) {
200 -> {
val raw = response.body?.string().orEmpty()
val token = extractToken(raw)
if (token.isNullOrBlank()) {
Result.failure(
IOException("relay returned success but no token")
)
} else {
Result.success("MEDIA:hermes-relay://$token")
}
}
404 -> Result.failure(
IOException(
"relay /media/upload endpoint not found — server needs " +
"Phase 3 bridge-server migration"
)
)
401, 403 -> Result.failure(
IOException("unauthorized — re-pair with the relay")
)
413 -> Result.failure(
IOException("screenshot too large for relay media cap")
)
in 500..599 -> Result.failure(
IOException("relay error (HTTP ${response.code})")
)
else -> Result.failure(
IOException("HTTP ${response.code}: ${response.message}")
)
}
}
} catch (e: IOException) {
Log.w(TAG, "uploadViaMultipart failed: ${e.message}")
Result.failure(e)
}
}
/**
* Minimal JSON token extractor — the response body is small and has a
* single interesting key. We avoid pulling in a full `Json` parse here
* because `ScreenCapture` is already a heavy dependency graph (Android
* media + OkHttp) and we don't want to add kotlinx.serialization
* wiring for a 40-byte response.
*/
private fun extractToken(body: String): String? {
val match = Regex("""\"token\"\s*:\s*\"([^\"]+)\"""").find(body)
return match?.groupValues?.get(1)
}
}
/**
* Holds the per-session [MediaProjection] grant.
*
* # Android 14+ rule
*
* `MediaProjectionManager.getMediaProjection()` MUST be called only after a
* foreground service has called `startForeground()` with type
* `FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION`, and that call must happen
* AFTER the user has granted the consent dialog. Calling it before — even
* if you're inside the launcher result callback — gives you a projection
* that the system auto-revokes within a frame, with no error visible to
* the app. Symptom: consent dialog appears, user allows, dialog closes,
* grant evaporates. Sample-tested on Samsung S24 / Android 14, 2026-04-12.
*
* Because of that rule, this holder no longer constructs the projection
* itself — it can only be populated from inside a foreground service that
* has already called `startForeground(type=mediaProjection)`. The phone-side
* entry point is `BridgeForegroundService.handleGrantedIntent`, which is
* dispatched from `MainActivity.mediaProjectionLauncher`.
*
* The projection state is exposed as a [StateFlow] so the UI can react to
* grants/revocations without polling. [BridgeViewModel] observes this and
* calls `refreshPermissionStatus()` on every emission, so the green check
* lights up immediately rather than waiting for the next lifecycle resume.
*
* Cleared on [revoke] (user disabled screenshots) or when the projection's
* own `onStop` callback fires (system revoked it).
*/
object MediaProjectionHolder {
private val _projectionFlow = kotlinx.coroutines.flow.MutableStateFlow<MediaProjection?>(null)
/**
* Reactive view of the current projection. Emits a fresh value every
* time the holder is populated or cleared; null means "no active grant."
*/
val projectionFlow: kotlinx.coroutines.flow.StateFlow<MediaProjection?> = _projectionFlow
/**
* Synchronous read used by [ScreenCapture] on each capture call. Always
* matches the latest [projectionFlow] value.
*/
val projection: MediaProjection? get() = _projectionFlow.value
/**
* Build a [MediaProjection] from a consent intent result and store it.
* **Caller must already be inside a foreground service that has called
* `startForeground(type=mediaProjection)`** — otherwise Android 14+ will
* silently auto-revoke the projection. The canonical caller is
* [com.hermesandroid.relay.bridge.BridgeForegroundService.handleGrantedIntent].
*
* Returns true on success, false on user-rejected consent or any
* downstream API error.
*/
fun acceptGrantInsideForegroundService(
context: Context,
resultCode: Int,
data: Intent?,
): Boolean {
if (resultCode != android.app.Activity.RESULT_OK || data == null) {
Log.i("MediaProjectionHolder", "consent rejected (resultCode=$resultCode)")
return false
}
val manager = context.getSystemService(Context.MEDIA_PROJECTION_SERVICE)
as MediaProjectionManager
val newProjection = try {
manager.getMediaProjection(resultCode, data)
} catch (t: Throwable) {
Log.w(
"MediaProjectionHolder",
"getMediaProjection threw: ${t.message} — is this called inside a " +
"foreground service that already did startForeground(mediaProjection)?"
)
null
} ?: return false
newProjection.registerCallback(object : MediaProjection.Callback() {
override fun onStop() {
_projectionFlow.value = null
}
}, Handler(android.os.Looper.getMainLooper()))
_projectionFlow.value = newProjection
Log.i("MediaProjectionHolder", "MediaProjection grant accepted and stored")
return true
}
fun revoke() {
try { _projectionFlow.value?.stop() } catch (_: Throwable) {}
_projectionFlow.value = null
}
}
@@ -0,0 +1,76 @@
package com.hermesandroid.relay.accessibility
/**
* Phase 3 — bridge-ui follow-up
*
* Process-singleton bridge between non-Activity code (BridgeViewModel,
* settings screens) and the Activity-scoped `ActivityResultLauncher` that
* fires the system MediaProjection consent dialog.
*
* # Why a singleton
*
* `MediaProjectionManager.createScreenCaptureIntent()` must be launched via
* an `ActivityResultLauncher` registered against a `ComponentActivity` —
* there is no way to invoke it from a ViewModel directly. We can't pass
* the launcher into the ViewModel either, because the ViewModel outlives
* the Activity across configuration changes and we'd leak the old Activity.
*
* The singleton pattern: `MainActivity` registers a launcher in `onCreate`,
* stores a closure that calls `launcher.launch(...)` here, and clears the
* closure in `onDestroy`. Anything that wants to ask the user for the
* MediaProjection grant calls [request] — if the Activity is alive, the
* system dialog appears; if not, the call returns false and the caller
* should surface "open the app first".
*
* # Why not just inject the launcher into the ViewModel
*
* `ActivityResultLauncher` is bound to the ViewModel store of the host
* Activity, not the ViewModel. Crossing that boundary either leaks the
* Activity (bad) or works only until the first rotation (worse). The
* lifecycle-respecting way is to keep the launcher Activity-scoped and
* route requests to it through a process-wide rendezvous like this.
*/
object ScreenCaptureRequester {
@Volatile
private var launchAction: (() -> Unit)? = null
/**
* Called by `MainActivity.onCreate` (or any ComponentActivity that
* wants to host the consent flow) with a closure that launches its
* pre-registered `ActivityResultLauncher` for the
* `ACTION_MEDIA_PROJECTION` intent.
*/
fun install(launch: () -> Unit) {
launchAction = launch
}
/** Called by `MainActivity.onDestroy` so we don't hold a stale Activity ref. */
fun uninstall() {
launchAction = null
}
/**
* Trigger the consent dialog. Returns `true` if a host Activity is
* currently installed and the request was dispatched, `false` if no
* Activity is alive (caller should fall back to "open the app first").
*
* The actual grant arrives asynchronously via the launcher's result
* callback — see `MainActivity.mediaProjectionLauncher`, which hands
* the result to `BridgeForegroundService.grantMediaProjection` so the
* grant lands inside a foreground service that's already running with
* `startForeground(type=mediaProjection)` (Android 14+ requirement).
*/
fun request(): Boolean {
val action = launchAction ?: return false
return try {
action.invoke()
true
} catch (t: Throwable) {
false
}
}
/** Quick poll for UI — true when a host Activity is currently installed. */
val isAvailable: Boolean get() = launchAction != null
}
@@ -0,0 +1,258 @@
package com.hermesandroid.relay.accessibility
import android.graphics.Rect
import android.view.accessibility.AccessibilityNodeInfo
import com.hermesandroid.relay.accessibility.ScreenReader.Companion.MAX_NODES
import java.security.MessageDigest
import kotlinx.serialization.Serializable
/**
* Phase 3 — bridge feature expansion, work unit A5.
*
* Cheap change detection for agent navigation loops. Computes a SHA-256
* hash over the full multi-window accessibility tree so the agent can
* ask "did anything change since last iteration?" with ~100x less data
* than a full [ScreenReader.readScreen] round-trip.
*
* # Fingerprint field set (hash stability is load-bearing)
*
* Each interesting node contributes:
*
* className | text | contentDescription | bounds | viewIdResourceName
*
* Joined per-node with `|`, and joined between nodes with `\u001e`
* (ASCII record separator) so a literal `|` in text can't collide with
* field separators. The concatenated string is hashed with SHA-256 and
* returned as lowercase hex.
*
* ## Why these fields
* * **className** — structural identity of the widget
* * **text** — what the user sees; primary content change signal
* * **contentDescription** — screen-reader label, relevant for icon
* buttons whose visible text is empty
* * **bounds** — layout geometry; catches appearance of new dialogs,
* bottom sheets, and popups
* * **viewIdResourceName** — stable across recompositions, lets us
* distinguish nodes that happen to share text (e.g. two "OK" buttons)
*
* ## Why NOT these
* * **isFocused / accessibility focus** — keyboard navigation toggles
* focus without any visible content change; would cause the hash to
* churn on arrow-key presses
* * **isSelected** — same reasoning as focus; Material chips etc.
* flip selected state during the ripple animation
* * **isEnabled / isClickable** — these almost always co-vary with
* text/bounds and dragging them in adds noise without signal
* * **timestamps** — no timestamps in the fingerprint, obviously
* * **window index (w<N>:<M> nodeId)** — the window index is stable
* across reads within a snapshot, but including it in the fingerprint
* is redundant with bounds (windows are geographically disjoint) and
* would make the hash brittle to window-list reorderings the agent
* doesn't care about
*
* ## Known limitation
* Apps that put a live counter in a text field (e.g. "Downloading…
* 3s", scrolling tickers, animated progress %) will churn the hash
* every frame. The agent's calling tool documents this and recommends
* `android_read_screen` for those edge cases.
*
* # Traversal
*
* Walks each provided root with the same child-recycling contract as
* [ScreenReader.walk] — children are `.recycle()`'d in `try/finally`,
* the input roots are NOT recycled (the caller owns their lifetime,
* matching [ScreenReader.readScreen]'s convention).
*
* Node count is capped at [MAX_NODES] total across all windows so a
* pathological grid can't OOM the hasher; if the cap is hit we still
* return a hash of what we collected and set [ScreenHashResult.truncated]
* = true.
*
* # Usage pattern
*
* ```kotlin
* // A5 — when P1 multi-window lands, this will be
* // service.snapshotAllWindows() -> List<AccessibilityNodeInfo>
* // Until then callers pass a singleton list of the active root.
* val roots: List<AccessibilityNodeInfo> = listOfNotNull(service.snapshotRoot())
* val hash = ScreenHasher().screenHash(roots)
* ```
*/
class ScreenHasher {
companion object {
/** ASCII record separator (0x1E) — unlikely to appear in UI text. */
private const val RECORD_SEPARATOR = '\u001E'
/** Intra-node field separator. */
private const val FIELD_SEPARATOR = '|'
/** SHA-256 digest length in hex chars (64). Used by tests. */
const val HASH_HEX_LENGTH = 64
}
@Serializable
data class ScreenHashResult(
val hash: String,
val nodeCount: Int,
val truncated: Boolean = false,
)
@Serializable
data class DiffScreenResult(
val changed: Boolean,
val hash: String,
val nodeCount: Int,
val truncated: Boolean = false,
)
/**
* Walk the multi-window tree under [roots] and return a stable
* SHA-256 fingerprint.
*
* Does NOT recycle [roots] — caller owns them, same contract as
* [ScreenReader.readScreen].
*/
fun screenHash(roots: List<AccessibilityNodeInfo>): ScreenHashResult {
val buf = StringBuilder(2048)
var count = 0
var truncated = false
for (root in roots) {
if (count >= MAX_NODES) {
truncated = true
break
}
val hit = walk(root, buf, count)
count = hit.count
if (hit.truncated) {
truncated = true
break
}
}
val digest = MessageDigest.getInstance("SHA-256")
.digest(buf.toString().toByteArray(Charsets.UTF_8))
return ScreenHashResult(
hash = digest.toHex(),
nodeCount = count,
truncated = truncated,
)
}
/**
* Compute the current hash and compare against [previousHash].
* Always returns both the new hash and the node count so the agent
* can update its reference without a second round-trip.
*/
fun diffScreen(
roots: List<AccessibilityNodeInfo>,
previousHash: String,
): DiffScreenResult {
val current = screenHash(roots)
return DiffScreenResult(
changed = current.hash != previousHash,
hash = current.hash,
nodeCount = current.nodeCount,
truncated = current.truncated,
)
}
// ── Internals ──────────────────────────────────────────────────────
/** Result of a walk — the running node count and a truncation flag. */
private data class WalkResult(val count: Int, val truncated: Boolean)
/**
* Append fingerprints for [node] and all descendants into [out].
* Mirrors [ScreenReader.walk]'s recycle contract exactly.
*/
private fun walk(
node: AccessibilityNodeInfo?,
out: StringBuilder,
startCount: Int,
): WalkResult {
if (node == null) return WalkResult(startCount, false)
if (startCount >= MAX_NODES) return WalkResult(startCount, true)
var count = startCount
val rect = Rect()
node.getBoundsInScreen(rect)
val text = node.text?.toString()?.takeIf { it.isNotBlank() }
val contentDesc = node.contentDescription?.toString()?.takeIf { it.isNotBlank() }
val clickable = node.isClickable
val longClickable = node.isLongClickable
val scrollable = node.isScrollable
// Same "interesting" predicate as ScreenReader so the hash
// covers exactly the nodes the agent can actually see/interact
// with. Otherwise a layout container reshuffle would churn the
// hash without any visible change.
val interesting = (
text != null || contentDesc != null ||
clickable || longClickable || scrollable || node.isEditable
) && rect.width() > 0 && rect.height() > 0
if (interesting) {
appendFingerprint(
out = out,
className = node.className?.toString(),
text = text,
contentDescription = contentDesc,
rect = rect,
viewId = node.viewIdResourceName,
)
count += 1
}
val childCount = node.childCount
for (i in 0 until childCount) {
if (count >= MAX_NODES) return WalkResult(count, true)
val child = node.getChild(i) ?: continue
try {
val hit = walk(child, out, count)
count = hit.count
if (hit.truncated) return WalkResult(count, true)
} finally {
@Suppress("DEPRECATION")
try { child.recycle() } catch (_: Throwable) { }
}
}
return WalkResult(count, false)
}
private fun appendFingerprint(
out: StringBuilder,
className: String?,
text: String?,
contentDescription: String?,
rect: Rect,
viewId: String?,
) {
if (out.isNotEmpty()) out.append(RECORD_SEPARATOR)
out.append(className.orEmpty()).append(FIELD_SEPARATOR)
out.append(text.orEmpty()).append(FIELD_SEPARATOR)
out.append(contentDescription.orEmpty()).append(FIELD_SEPARATOR)
// Compact bounds form — matches what the agent would see in
// ScreenNode.bounds.toString() but cheaper to build.
out.append(rect.left).append(',')
.append(rect.top).append(',')
.append(rect.right).append(',')
.append(rect.bottom).append(FIELD_SEPARATOR)
out.append(viewId.orEmpty())
}
private fun ByteArray.toHex(): String {
val sb = StringBuilder(this.size * 2)
for (b in this) {
val v = b.toInt() and 0xff
sb.append(HEX[v ushr 4])
sb.append(HEX[v and 0x0f])
}
return sb.toString()
}
}
private val HEX = "0123456789abcdef".toCharArray()
@@ -0,0 +1,807 @@
package com.hermesandroid.relay.accessibility
import android.graphics.Rect
import android.os.Build
import android.view.accessibility.AccessibilityNodeInfo
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonNull
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
/**
* Phase 3 — accessibility `accessibility-runtime`
*
* Walks the `AccessibilityNodeInfo` trees of every active accessibility
* window and produces a single structured, serializable snapshot the agent
* can reason about.
*
* The output shape is deliberately flat: the agent overwhelmingly cares
* about *"what text can I see, and where is it?"* — not the exact widget
* hierarchy. We emit one [ScreenNode] per interesting node (anything with
* non-blank text, content description, or a click / long-click action) and
* include its screen bounds so [ActionExecutor.tapText] can hand them to
* `dispatchGesture` without re-walking the tree.
*
* # Multi-window walk (P1)
*
* As of P1 (`feature/P1-all-windows`) the reader walks *every* live
* accessibility window, not just `rootInActiveWindow`. This catches system
* overlays, popup menus, the notification shade when pulled down,
* permission dialogs, and multi-window split-screen state — all of which
* were previously invisible to the agent.
*
* ## Tree-merging strategy
*
* We flatten all windows into **one** `ScreenContent` rather than returning
* per-window trees. This is deliberate: the existing public contract
* (consumed by `BridgeCommandHandler` and the Python-side LLM prompt) is
* "one JSON object with a `nodes[]` array". Preserving that contract means
* no wire-format change and no consumer refactor.
*
* To disambiguate nodes that belong to different windows we prefix every
* `nodeId` with `w<windowIndex>:<sequentialIndex>`. `windowIndex` is the
* position in `service.windows` (top-of-stack first, per Android docs),
* `sequentialIndex` is the 0-based collection order within the combined
* walk. The root bounds emitted on `ScreenContent.rootBounds` are the
* **union** of every window's root bounds, so geometric callers still see
* a single enclosing rectangle.
*
* ## MAX_NODES cap
*
* The 512-node cap applies to the *combined* tree, not per-window. If the
* first window alone exceeds the cap we short-circuit the whole walk and
* set `truncated = true` without descending into later windows — the agent
* already has more than it can reasonably use.
*
* ## Node recycling landmine
*
* Every `AccessibilityWindowInfo.getRoot()` returns a **fresh**
* `AccessibilityNodeInfo` that must be recycled by the caller. Every
* `info.getChild(i)` also returns a fresh ref. The public methods here
* do NOT recycle their input roots — the caller
* ([HermesAccessibilityService.snapshotAllWindows] in practice) owns the
* lifetime of the roots it produced. Child nodes fetched during the walk
* are recycled per-iteration in `try/finally`.
*
* The tree is bounded by [MAX_NODES] to prevent pathological apps (grids
* with thousands of cells) from producing multi-megabyte screen dumps.
* When the cap is hit we short-circuit traversal and set
* [ScreenContent.truncated] = true so the agent knows to scroll if it
* needs more.
*/
class ScreenReader {
companion object {
/**
* Hard cap on node count. 512 is comfortable for a typical app
* screen (most have 30–150 interesting nodes) while keeping the
* wire payload under ~40 KB in the worst case. With P1 the cap
* now spans the *combined* tree across all live windows.
*/
const val MAX_NODES = 512
/** Hard cap on individual text-field length before truncation. */
const val MAX_TEXT_LEN = 2000
}
/**
* Structured representation of a screen, ready for JSON serialization.
* The [rootBounds] are in absolute screen pixels (what `dispatchGesture`
* uses). When P1 merges multiple windows, [rootBounds] is the union of
* every window's root bounds.
*/
@Serializable
data class ScreenContent(
val packageName: String?,
val rootBounds: Bounds,
val nodes: List<ScreenNode>,
val truncated: Boolean = false,
val nodeCount: Int = nodes.size,
)
@Serializable
data class ScreenNode(
/**
* Stable, walk-scoped identifier for this node. Format:
* `w<windowIndex>:<sequentialIndex>` — e.g. `w0:42` for the 43rd
* emitted node from the top-of-stack window, `w1:7` for the 8th
* emitted node of the next window down. Always present when the
* node was produced by [readAllWindows]; may be null for callers
* that bypass the multi-window entry point (legacy tests).
*
* Also re-used by A3 `searchNodes` so filtered results feed back
* into `tap nodeId` and other node-ID-addressable commands.
*
* The Python `android_tool` layer has long advertised a `nodeId`
* field in its doc-comments; P1 is the first version that actually
* emits it on the wire.
*/
val nodeId: String? = null,
val text: String? = null,
val contentDescription: String? = null,
val className: String? = null,
val viewId: String? = null,
val bounds: Bounds,
val clickable: Boolean = false,
val longClickable: Boolean = false,
val scrollable: Boolean = false,
val editable: Boolean = false,
val focused: Boolean = false,
val selected: Boolean = false,
val enabled: Boolean = true,
)
@Serializable
data class Bounds(
val left: Int,
val top: Int,
val right: Int,
val bottom: Int,
) {
val centerX: Int get() = (left + right) / 2
val centerY: Int get() = (top + bottom) / 2
val width: Int get() = right - left
val height: Int get() = bottom - top
val isEmpty: Boolean get() = width <= 0 || height <= 0
}
/**
* Back-compat single-root entry point. Delegates to [readAllWindows]
* with a single-element list so the walk semantics (cap, recycling,
* node-id prefix) stay identical regardless of which public method
* the caller picked.
*
* This method does NOT recycle [rootNode] — the caller owns it.
*
* @param includeBounds when false, we still collect bounds for
* traversal decisions but zero them in the output to shrink the
* payload. The agent almost always wants bounds so this defaults
* true.
*/
fun readScreen(
rootNode: AccessibilityNodeInfo,
includeBounds: Boolean = true,
): ScreenContent = readAllWindows(listOf(rootNode), includeBounds)
/**
* P1 multi-window entry point. Walks every root in [windowRoots] in
* order (top-of-stack first) and emits a single flat [ScreenContent]
* with node IDs prefixed by window index.
*
* The [windowRoots] list is NOT recycled by this method — callers
* ([HermesAccessibilityService.snapshotAllWindows] + finally blocks
* in `BridgeCommandHandler`) own the roots they produced. Child nodes
* fetched during traversal ARE recycled per-iteration.
*
* @param includeBounds see [readScreen].
*/
fun readAllWindows(
windowRoots: List<AccessibilityNodeInfo>,
includeBounds: Boolean = true,
): ScreenContent {
val collected = ArrayList<ScreenNode>(128)
var truncated = false
// Union of every window's root bounds. Starts empty and absorbs
// each window's root rect via Rect.union(), so one-window callers
// see identical output to the pre-P1 code path.
val unionRect = Rect()
var unionInitialized = false
var packageName: String? = null
for ((windowIndex, rootNode) in windowRoots.withIndex()) {
if (collected.size >= MAX_NODES) {
// Cap already exhausted by an earlier window — do not
// descend. This is intentional: the agent has more than
// it can reason about and we'd rather be honest about it
// than silently clip the later-window content.
truncated = true
break
}
// Latch the first window's package name as the "primary" one.
// Multi-window state with different packages (e.g. a popup from
// a different process) is rare, and the agent can still see
// the full package-name string on each node via className if
// it needs finer-grained attribution.
if (packageName == null) {
packageName = rootNode.packageName?.toString()
}
val rootRect = Rect()
rootNode.getBoundsInScreen(rootRect)
if (!unionInitialized) {
unionRect.set(rootRect)
unionInitialized = true
} else {
unionRect.union(rootRect)
}
val hit = walk(
node = rootNode,
windowIndex = windowIndex,
out = collected,
includeBounds = includeBounds,
)
if (hit) {
truncated = true
break
}
}
val rootBounds = if (unionInitialized) unionRect.toBoundsOrZero() else ZERO_BOUNDS
return ScreenContent(
packageName = packageName,
rootBounds = rootBounds,
nodes = collected,
truncated = truncated,
)
}
/**
* Recursive walker. Returns `true` if the cap was hit (signaling
* caller to mark output as truncated).
*
* [windowIndex] is the position in the parent walk's `windowRoots`
* list and becomes the `w<n>:` prefix on every emitted node ID.
*/
private fun walk(
node: AccessibilityNodeInfo?,
windowIndex: Int,
out: MutableList<ScreenNode>,
includeBounds: Boolean,
): Boolean {
if (node == null) return false
if (out.size >= MAX_NODES) return true
// Skip invisible nodes early — no visual, no interaction target.
if (!node.isVisibleToUser) {
// still walk children, some containers aren't marked visible
// but host visible descendants
}
val rect = Rect()
node.getBoundsInScreen(rect)
val bounds = if (includeBounds) rect.toBoundsOrZero() else ZERO_BOUNDS
val text = node.text?.toString()?.takeIf { it.isNotBlank() }?.take(MAX_TEXT_LEN)
val contentDesc = node.contentDescription?.toString()
?.takeIf { it.isNotBlank() }
?.take(MAX_TEXT_LEN)
val clickable = node.isClickable
val longClickable = node.isLongClickable
val scrollable = node.isScrollable
val interesting = (text != null || contentDesc != null ||
clickable || longClickable || scrollable || node.isEditable) &&
!bounds.isEmpty
if (interesting) {
// Node ID is `w<windowIndex>:<sequentialIndex>`. Using the
// sequential emission index (not a hash of the node) keeps
// IDs stable within a single snapshot and compact on the
// wire, at the cost of being snapshot-scoped (not durable
// across successive /screen calls). That's the right tradeoff
// — the agent re-reads the screen before each action anyway.
val nodeId = "w$windowIndex:${out.size}"
out.add(
ScreenNode(
nodeId = nodeId,
text = text,
contentDescription = contentDesc,
className = node.className?.toString(),
viewId = node.viewIdResourceName,
bounds = bounds,
clickable = clickable,
longClickable = longClickable,
scrollable = scrollable,
editable = node.isEditable,
focused = node.isFocused,
selected = node.isSelected,
enabled = node.isEnabled,
)
)
}
val childCount = node.childCount
for (i in 0 until childCount) {
if (out.size >= MAX_NODES) return true
val child = node.getChild(i) ?: continue
try {
val hit = walk(child, windowIndex, out, includeBounds)
if (hit) return true
} finally {
@Suppress("DEPRECATION")
try { child.recycle() } catch (_: Throwable) { }
}
}
return false
}
/**
* Filtered search across all provided window roots. Used by
* `android_find_nodes` — returns up to [limit] matches without
* dumping the whole accessibility tree.
*
* Filter semantics (all optional, all ANDed):
* - [text]: case-insensitive substring match against each node's
* `text` OR `contentDescription`.
* - [className]: exact match against `node.className.toString()`.
* - [clickable]: when non-null, filters on `node.isClickable`.
*
* The underlying traversal honors [MAX_NODES] as a safety rail even
* when [limit] is higher — we stop walking after visiting 512 nodes
* regardless of how many matched. Node recycling follows the same
* per-child `try/finally` pattern as [walk], so this function is
* leak-free w.r.t. the accessibility node pool.
*
* Returns a list of [ScreenNode] in the same shape that
* [readAllWindows] / [readScreen] emits, including the P1 `nodeId`
* field (`"w<windowIndex>:<sequentialIndex>"`) so callers can feed
* results back into `tap nodeId`.
*/
fun searchNodes(
roots: List<AccessibilityNodeInfo>,
text: String? = null,
className: String? = null,
clickable: Boolean? = null,
limit: Int = 20,
): List<ScreenNode> {
if (limit <= 0) return emptyList()
val loweredText = text?.takeIf { it.isNotBlank() }?.lowercase()
val effectiveLimit = limit.coerceAtLeast(0)
val out = ArrayList<ScreenNode>(effectiveLimit.coerceAtMost(64))
// Shared visit counter across all windows — the MAX_NODES cap is
// global, matching how `readAllWindows` (P1) bounds traversal.
val visited = intArrayOf(0)
// H2 fix: hoist the emission-id counter OUTSIDE the per-window loop
// so it mirrors `walk`/`findNodeById`'s GLOBAL "interesting" counter
// exactly. Previously this was scoped to each window, producing IDs
// like `w1:0` when the canonical scheme is `w1:N` (N = global). Round
// trips through `find_nodes → tap nodeId` then resolved to the wrong
// node on any screen with >1 window.
val nextIndex = intArrayOf(0)
for ((windowIndex, root) in roots.withIndex()) {
if (out.size >= effectiveLimit) break
if (visited[0] >= MAX_NODES) break
searchWalk(
node = root,
windowIndex = windowIndex,
nextIndex = nextIndex,
visited = visited,
loweredText = loweredText,
className = className,
clickable = clickable,
limit = effectiveLimit,
out = out,
)
}
return out
}
/**
* Recursive walker for [searchNodes]. Returns nothing; accumulates
* matches into [out] and respects both [MAX_NODES] (via [visited])
* and [limit] (via `out.size`).
*
* [nextIndex] is the per-window sequential counter used to build the
* `w<windowIndex>:<N>` node ID — mirrors the scheme P1 introduced in
* [readAllWindows] so the two surfaces produce stable, comparable IDs.
*/
private fun searchWalk(
node: AccessibilityNodeInfo?,
windowIndex: Int,
nextIndex: IntArray,
visited: IntArray,
loweredText: String?,
className: String?,
clickable: Boolean?,
limit: Int,
out: MutableList<ScreenNode>,
) {
if (node == null) return
if (out.size >= limit) return
if (visited[0] >= MAX_NODES) return
visited[0] += 1
val nodeText = node.text?.toString()?.takeIf { it.isNotBlank() }?.take(MAX_TEXT_LEN)
val contentDesc = node.contentDescription?.toString()
?.takeIf { it.isNotBlank() }
?.take(MAX_TEXT_LEN)
val nodeClassName = node.className?.toString()
val nodeClickable = node.isClickable
val nodeLongClickable = node.isLongClickable
val nodeScrollable = node.isScrollable
val nodeEditable = node.isEditable
// H2 fix: nextIndex must mirror `walk`/`findNodeById`'s emission
// counter exactly — increment ONLY for nodes that pass the canonical
// "interesting" predicate (text || contentDesc || clickable ||
// longClickable || scrollable || editable, with non-empty bounds),
// not for every visited node. Otherwise the IDs handed back from
// find_nodes drift relative to readAllWindows + findNodeById and
// round-trip lookups silently resolve to the wrong node.
val rect = Rect()
node.getBoundsInScreen(rect)
val bounds = rect.toBoundsOrZero()
val canonicalInteresting = (nodeText != null || contentDesc != null ||
nodeClickable || nodeLongClickable || nodeScrollable || nodeEditable) &&
!bounds.isEmpty
val thisNodeIndex: Int
if (canonicalInteresting) {
thisNodeIndex = nextIndex[0]
nextIndex[0] += 1
} else {
thisNodeIndex = -1
}
val matchesText = loweredText == null ||
(nodeText?.lowercase()?.contains(loweredText) == true) ||
(contentDesc?.lowercase()?.contains(loweredText) == true)
val matchesClass = className == null || nodeClassName == className
val matchesClickable = clickable == null || nodeClickable == clickable
// Only emit nodes that BOTH match the search filter AND are
// canonically-interesting (i.e. would also be emitted by `walk`).
// Restricting emission to canonical nodes is what makes the round
// trip find_nodes → tap nodeId reliable.
if (canonicalInteresting && matchesText && matchesClass && matchesClickable) {
if (!bounds.isEmpty || loweredText != null || className != null) {
out.add(
ScreenNode(
nodeId = "w$windowIndex:$thisNodeIndex",
text = nodeText,
contentDescription = contentDesc,
className = nodeClassName,
viewId = node.viewIdResourceName,
bounds = bounds,
clickable = nodeClickable,
longClickable = node.isLongClickable,
scrollable = node.isScrollable,
editable = node.isEditable,
focused = node.isFocused,
selected = node.isSelected,
enabled = node.isEnabled,
)
)
if (out.size >= limit) return
}
}
val childCount = node.childCount
for (i in 0 until childCount) {
if (out.size >= limit) return
if (visited[0] >= MAX_NODES) return
val child = node.getChild(i) ?: continue
try {
searchWalk(
node = child,
windowIndex = windowIndex,
nextIndex = nextIndex,
visited = visited,
loweredText = loweredText,
className = className,
clickable = clickable,
limit = limit,
out = out,
)
} finally {
@Suppress("DEPRECATION")
try { child.recycle() } catch (_: Throwable) { }
}
}
}
/**
* Find the first node across [windowRoots] whose text or
* content-description contains [needle] (case-insensitive). Used by
* [ActionExecutor.tapText]. Returns the node's bounds, or null if no
* match anywhere.
*
* We walk fresh (not against a cached [ScreenContent]) so the result
* is always current — tapping into stale bounds is the single most
* common "bridge tapped the wrong thing" bug.
*/
fun findNodeBoundsByText(
windowRoots: List<AccessibilityNodeInfo>,
needle: String,
): Bounds? {
if (needle.isBlank()) return null
val lowered = needle.lowercase()
for (root in windowRoots) {
val hit = findFirst(root) { node ->
val text = node.text?.toString()?.lowercase()
val desc = node.contentDescription?.toString()?.lowercase()
(text?.contains(lowered) == true) || (desc?.contains(lowered) == true)
}
if (hit != null) {
val r = Rect()
hit.getBoundsInScreen(r)
return r.toBoundsOrZero()
}
}
return null
}
/**
* Back-compat single-root overload. Delegates to the multi-window
* form so tests and any legacy call site still compile unchanged.
*/
fun findNodeBoundsByText(rootNode: AccessibilityNodeInfo, needle: String): Bounds? =
findNodeBoundsByText(listOf(rootNode), needle)
/**
* Find the currently-focused input node across [windowRoots] (for
* [ActionExecutor.typeText]). Prefers `FOCUS_INPUT` focus on each
* window in order, falls back to the first editable node found in any
* window.
*
* Returned node is caller-owned — must be `recycle()`d on API < 33.
*/
fun findFocusedInput(windowRoots: List<AccessibilityNodeInfo>): AccessibilityNodeInfo? {
for (root in windowRoots) {
root.findFocus(AccessibilityNodeInfo.FOCUS_INPUT)?.let { return it }
}
for (root in windowRoots) {
findFirst(root) { it.isEditable }?.let { return it }
}
return null
}
/**
* Back-compat single-root overload. Delegates to the multi-window
* form.
*/
fun findFocusedInput(rootNode: AccessibilityNodeInfo): AccessibilityNodeInfo? =
findFocusedInput(listOf(rootNode))
private fun findFirst(
node: AccessibilityNodeInfo?,
predicate: (AccessibilityNodeInfo) -> Boolean,
): AccessibilityNodeInfo? {
if (node == null) return null
if (predicate(node)) return node
val childCount = node.childCount
for (i in 0 until childCount) {
val child = node.getChild(i) ?: continue
val hit = findFirst(child, predicate)
if (hit != null) return hit
@Suppress("DEPRECATION")
try { child.recycle() } catch (_: Throwable) { }
}
return null
}
// ─── A4: describe_node + stable nodeId lookup ────────────────────────────
//
// `findNodeById` re-walks the window tree every call. We deliberately do
// NOT cache IDs between calls — the UI changes, nodes come and go, and a
// cached lookup table would be stale the moment the user scrolled. The
// walker assigns the same `w<windowIndex>:<sequentialIndex>` IDs that the
// P1 multi-window [walk] emits, so a nodeId from `read_screen` round-trips
// cleanly into `describe_node`, `/tap`, and `/scroll` during the same
// screen dwell.
//
// IMPORTANT: the counter MUST mirror P1 semantics exactly:
// 1. Only "interesting" nodes (text/contentDesc/clickable/longClickable/
// scrollable/editable AND non-empty bounds) get an ID.
// 2. The sequential index is a GLOBAL pre-order emission counter
// shared across all windows (not per-window). Window 0 emits a
// handful of nodes at 0..N-1, then window 1's first emission is
// N, not 0. This matches how readAllWindows feeds a single
// `collected` list into multiple `walk()` calls.
//
// IMPORTANT: the caller takes ownership of the returned node and is
// responsible for `node.recycle()`. We stop recycling at the match frontier
// and let it bubble up. `describeNode` below handles this contract.
/**
* Walk every window root in [roots] and return the first node whose
* assigned stable ID matches [nodeId]. Returns `null` if no match.
*
* ID format is `w<windowIndex>:<sequentialIndex>` — the same scheme the
* P1 multi-window [walk] emits on [ScreenNode.nodeId]. The sequential
* index is a 0-based GLOBAL emission counter (shared across windows)
* that increments only for "interesting" nodes — matching the P1 filter
* in [walk] (`interesting = has text/desc/clickable/longClickable/
* scrollable/editable AND non-empty bounds`).
*
* The caller takes ownership of the returned [AccessibilityNodeInfo] and
* MUST recycle it (on API <= 33) when done. We stop recycling at the
* match frontier so the node survives the return trip.
*/
fun findNodeById(
roots: List<AccessibilityNodeInfo>,
nodeId: String,
): AccessibilityNodeInfo? {
if (nodeId.isBlank()) return null
// Parse `w<windowIdx>:<seqIdx>`. Reject malformed IDs up front so we
// don't spend O(tree) walking when the input can't possibly match.
val colonIdx = nodeId.indexOf(':')
if (colonIdx <= 1 || nodeId[0] != 'w') return null
val wantedWindow = nodeId.substring(1, colonIdx).toIntOrNull() ?: return null
val wantedSeq = nodeId.substring(colonIdx + 1).toIntOrNull() ?: return null
if (wantedWindow < 0 || wantedWindow >= roots.size || wantedSeq < 0) return null
// GLOBAL counter, shared across every window's walk to match
// readAllWindows' shared `collected` list ordering.
val counter = IntArray(1)
for ((windowIndex, root) in roots.withIndex()) {
// Cheap pre-filter: the wantedSeq is global, so we still have to
// walk earlier windows to drain their interesting-node emission
// counts, BUT once we're at or past the wanted window we can
// look for the match. We only RETURN a match when we're on the
// correct windowIndex AND the global counter reaches wantedSeq.
val hit = walkForId(root, wantedSeq, counter, windowIndex, wantedWindow)
if (hit != null) return hit
if (counter[0] > wantedSeq) return null
}
return null
}
/**
* Recursive node-id walker. Increments [counter] only for "interesting"
* nodes (matching P1's [walk] semantics). Returns a non-null match
* (caller-owned and responsible for recycling) OR null if this subtree
* doesn't contain it.
*
* [currentWindow] is the index of the window currently being walked;
* [wantedWindow] is the window portion of the parsed nodeId. We only
* return a hit when they match — earlier and later windows still
* contribute to the global counter but can't claim the match.
*
* Recycling rules:
* - Children that don't contain the match are recycled in-place.
* - The matched node bubbles up un-recycled — the outermost caller owns it.
* - The root node itself is never recycled here; the caller of
* `findNodeById` owns window roots (same contract as `snapshotAllWindows`).
*/
private fun walkForId(
node: AccessibilityNodeInfo?,
wantedSeq: Int,
counter: IntArray,
currentWindow: Int,
wantedWindow: Int,
): AccessibilityNodeInfo? {
if (node == null) return null
if (counter[0] > wantedSeq) return null
// Determine whether this node would be emitted by P1's [walk].
// Must mirror the `interesting` predicate there exactly or the
// counter drifts and the round-trip breaks.
val rect = Rect()
node.getBoundsInScreen(rect)
val bounds = rect.toBoundsOrZero()
val nodeText = node.text?.toString()?.takeIf { it.isNotBlank() }
val contentDesc = node.contentDescription?.toString()?.takeIf { it.isNotBlank() }
val clickable = node.isClickable
val longClickable = node.isLongClickable
val scrollable = node.isScrollable
val interesting = (nodeText != null || contentDesc != null ||
clickable || longClickable || scrollable || node.isEditable) &&
!bounds.isEmpty
if (interesting) {
val mySeq = counter[0]
counter[0] = mySeq + 1
if (currentWindow == wantedWindow && mySeq == wantedSeq) {
// Match. Bubble up without recycling.
return node
}
}
val childCount = node.childCount
for (i in 0 until childCount) {
if (counter[0] > wantedSeq) return null
val child = node.getChild(i) ?: continue
val hit = walkForId(child, wantedSeq, counter, currentWindow, wantedWindow)
if (hit != null) {
// Match in this subtree — don't recycle the hit.
return hit
}
@Suppress("DEPRECATION")
try { child.recycle() } catch (_: Throwable) { }
}
return null
}
/**
* A4: result of a `describe_node` lookup. Serialized directly into the
* `bridge.response` result payload by [BridgeCommandHandler].
*
* When [found] is false, [properties] is null and [error] explains why.
*/
data class DescribeNodeResult(
val found: Boolean,
val properties: JsonObject? = null,
val error: String? = null,
)
/**
* A4: return the full property bag for [nodeId] on the current window
* set. Props: `nodeId`, `bounds`, `className`, `text`, `contentDescription`,
* `hintText` (API 26+), `viewIdResourceName`, `childCount`, plus a dozen
* state flags. `checked` is null when the node isn't checkable so callers
* can distinguish "not a toggle" from "unchecked toggle".
*
* Walks the tree via [findNodeById], builds the JSON, and recycles the
* resolved node before returning.
*/
fun describeNode(
roots: List<AccessibilityNodeInfo>,
nodeId: String,
): DescribeNodeResult {
val node = findNodeById(roots, nodeId)
?: return DescribeNodeResult(found = false, error = "node not found: $nodeId")
try {
val rect = Rect().also { node.getBoundsInScreen(it) }
val bounds = rect.toBoundsOrZero()
// hintText is API 26+. minSdk on this project is 26, so in practice
// it's always available — but we guard anyway to keep the property
// out of the payload on devices where the API call would throw.
val hintText: String? = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
node.hintText?.toString()
} else null
// Local helper to collapse blank/null strings to JsonNull. A
// local lambda rather than an extension `put` overload so we
// don't shadow `JsonObjectBuilder.put(String, String?)`.
fun strOrNull(raw: String?): JsonElement =
if (raw.isNullOrBlank()) JsonNull else JsonPrimitive(raw)
val props: JsonObject = buildJsonObject {
put("nodeId", nodeId)
put("bounds", buildJsonObject {
put("left", bounds.left)
put("top", bounds.top)
put("right", bounds.right)
put("bottom", bounds.bottom)
put("centerX", bounds.centerX)
put("centerY", bounds.centerY)
put("width", bounds.width)
put("height", bounds.height)
})
put("className", strOrNull(node.className?.toString()))
put("text", strOrNull(node.text?.toString()))
put("contentDescription", strOrNull(node.contentDescription?.toString()))
put("hintText", strOrNull(hintText))
put("viewIdResourceName", strOrNull(node.viewIdResourceName))
put("childCount", node.childCount)
put("clickable", node.isClickable)
put("longClickable", node.isLongClickable)
put("focusable", node.isFocusable)
put("focused", node.isFocused)
put("editable", node.isEditable)
put("scrollable", node.isScrollable)
put("checkable", node.isCheckable)
// Null vs false is load-bearing: null = "not a toggle",
// false = "unchecked toggle".
put("checked", if (node.isCheckable) JsonPrimitive(node.isChecked) else JsonNull)
put("enabled", node.isEnabled)
put("selected", node.isSelected)
put("password", node.isPassword)
}
return DescribeNodeResult(found = true, properties = props)
} finally {
@Suppress("DEPRECATION")
try { node.recycle() } catch (_: Throwable) { }
}
}
private fun Rect.toBoundsOrZero(): Bounds =
Bounds(left = left, top = top, right = right, bottom = bottom)
private val ZERO_BOUNDS = Bounds(0, 0, 0, 0)
}
@@ -0,0 +1,191 @@
package com.hermesandroid.relay.audio
import android.media.MediaPlayer
import android.media.audiofx.Visualizer
import android.util.Log
import kotlinx.coroutines.suspendCancellableCoroutine
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import java.io.File
import kotlin.coroutines.resume
import kotlin.math.sqrt
/**
* Plays a TTS audio file emitted by the relay's `/voice/synthesize` endpoint
* and exposes a live [amplitude] flow for the MorphingSphere / UI meter.
*
* Amplitude is computed from [Visualizer] PCM waveform RMS — one waveform
* snapshot is captured every ~40 ms (the visualizer's default capture rate)
* and reduced to a 0..1 float. On devices that don't allow Visualizer
* construction (missing MODIFY_AUDIO_SETTINGS or OEM quirks) we log and
* continue with amplitude pinned at 0 rather than crashing the voice session.
*
* One player instance owns at most one active playback. Calling [play] again
* while something is playing stops the previous file first.
*/
class VoicePlayer {
companion object {
private const val TAG = "VoicePlayer"
private const val VISUALIZER_SIZE_BYTES = 1024
}
private val _amplitude = MutableStateFlow(0f)
val amplitude: StateFlow<Float> = _amplitude.asStateFlow()
private var mediaPlayer: MediaPlayer? = null
private var visualizer: Visualizer? = null
private var completionListener: (() -> Unit)? = null
/**
* Start playback of [audioFile]. Returns immediately; completion is
* delivered via [awaitCompletion]. If another file is already playing,
* [stop]s it first.
*/
fun play(audioFile: File) {
if (mediaPlayer != null) {
Log.w(TAG, "play called while another file is playing — stopping first")
stop()
}
val player = MediaPlayer()
try {
player.setDataSource(audioFile.absolutePath)
player.prepare()
player.setOnCompletionListener {
_amplitude.value = 0f
completionListener?.invoke()
}
player.setOnErrorListener { _, what, extra ->
Log.e(TAG, "MediaPlayer error: what=$what extra=$extra")
_amplitude.value = 0f
completionListener?.invoke()
true
}
player.start()
} catch (e: Exception) {
Log.e(TAG, "MediaPlayer setup failed: ${e.message}")
try { player.release() } catch (_: Exception) { /* ignore */ }
throw e
}
mediaPlayer = player
attachVisualizer(player)
}
/**
* Suspend until the current playback completes or errors. Cancellable —
* if the caller cancels, playback is left running (use [stop] for a
* hard teardown).
*/
suspend fun awaitCompletion(): Unit = suspendCancellableCoroutine { cont ->
if (mediaPlayer == null) {
cont.resume(Unit)
return@suspendCancellableCoroutine
}
completionListener = {
completionListener = null
if (cont.isActive) cont.resume(Unit)
}
cont.invokeOnCancellation {
completionListener = null
}
}
/**
* Hard teardown: stop playback, release visualizer + player, reset
* amplitude. Safe to call repeatedly.
*/
fun stop() {
completionListener = null
visualizer?.let { v ->
try { v.enabled = false } catch (_: Exception) { /* ignore */ }
try { v.release() } catch (_: Exception) { /* ignore */ }
}
visualizer = null
mediaPlayer?.let { p ->
try {
if (p.isPlaying) p.stop()
} catch (_: Exception) { /* ignore */ }
try { p.reset() } catch (_: Exception) { /* ignore */ }
try { p.release() } catch (_: Exception) { /* ignore */ }
}
mediaPlayer = null
_amplitude.value = 0f
}
/**
* True if there's an active [MediaPlayer]. Doesn't check `isPlaying` —
* that would race with the completion listener.
*/
fun isPlaying(): Boolean = mediaPlayer != null
private fun attachVisualizer(player: MediaPlayer) {
try {
val viz = Visualizer(player.audioSessionId)
viz.captureSize = VISUALIZER_SIZE_BYTES.coerceIn(
Visualizer.getCaptureSizeRange()[0],
Visualizer.getCaptureSizeRange()[1],
)
viz.setDataCaptureListener(
object : Visualizer.OnDataCaptureListener {
override fun onWaveFormDataCapture(
visualizer: Visualizer?,
waveform: ByteArray?,
samplingRate: Int,
) {
if (waveform == null || waveform.isEmpty()) return
_amplitude.value = computeRms(waveform)
}
override fun onFftDataCapture(
visualizer: Visualizer?,
fft: ByteArray?,
samplingRate: Int,
) { /* unused */ }
},
Visualizer.getMaxCaptureRate() / 2,
true, // waveform
false, // fft
)
viz.enabled = true
visualizer = viz
} catch (e: Exception) {
// Some devices refuse Visualizer (MODIFY_AUDIO_SETTINGS denied,
// OEM restrictions). Fall back to flat-zero amplitude rather
// than killing the voice session.
Log.w(TAG, "Visualizer unavailable — amplitude stuck at 0: ${e.message}")
_amplitude.value = 0f
visualizer = null
}
}
/**
* RMS of an 8-bit unsigned PCM waveform, mapped to 0..1. Visualizer
* emits signed bytes centered at 128 (0x80), so the first step is to
* re-center at 0.
*
* Guards: empty waveform short-circuits to 0 to avoid `0.0 / 0 = NaN`,
* and the final result is NaN-filtered before returning. A single NaN
* amplitude frame would otherwise propagate through `animateFloatAsState`
* and cause Android 15 to log `setRequestedFrameRate frameRate=NaN` on
* every draw pass.
*/
private fun computeRms(waveform: ByteArray): Float {
if (waveform.isEmpty()) return 0f
var sumSq = 0.0
for (b in waveform) {
val sample = (b.toInt() and 0xFF) - 128
sumSq += (sample * sample).toDouble()
}
val rms = sqrt(sumSq / waveform.size)
// 128 is the theoretical max deviation for re-centered 8-bit PCM.
val normalized = (rms / 128.0).toFloat()
return if (normalized.isNaN() || normalized.isInfinite()) 0f
else normalized.coerceIn(0f, 1f)
}
}
@@ -0,0 +1,224 @@
package com.hermesandroid.relay.audio
import android.annotation.SuppressLint
import android.content.Context
import android.media.MediaRecorder
import android.os.Build
import android.util.Log
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.isActive
import kotlinx.coroutines.launch
import java.io.File
import kotlin.math.sqrt
/**
* Captures the user's voice into an `.m4a` (AAC-in-MP4) file for V2a voice
* mode. The relay's `/voice/transcribe` endpoint feeds this to whisper-1 via
* OpenAI, which accepts m4a/mp4 natively.
*
* A live [amplitude] flow is exposed for the UI (MorphingSphere + meter) —
* driven by polling `MediaRecorder.maxAmplitude` every ~16 ms. The polling
* coroutine runs on the caller-supplied [scope] so it dies with the owning
* ViewModel.
*
* One recorder instance owns at most one active recording at a time. Calling
* [startRecording] again while a recording is in flight will stop the
* previous one first. [stopRecording] is safe to call when nothing is
* running (it just returns the last file, or throws if there never was one).
*/
class VoiceRecorder(
private val context: Context,
private val scope: CoroutineScope,
) {
companion object {
private const val TAG = "VoiceRecorder"
private const val SAMPLE_RATE = 16_000
private const val BIT_RATE = 64_000
private const val AMPLITUDE_POLL_MS = 16L
private const val MAX_AMPLITUDE_SHORT = 32_767f
// Perceptual amplitude mapping constants. Raw PCM peak values from
// MediaRecorder.maxAmplitude for a phone at arm's length:
// silence / ambient : 100..500 (≤0.015 of max)
// quiet speech : 500..3000 (0.015..0.09)
// normal speech : 3000..8000 (0.09..0.24)
// loud speech : 8000..18000 (0.24..0.55)
// shout / clipping : 18000..32767 (0.55..1.0)
//
// Linear 0..1 puts normal conversation between 0.09 and 0.24 — the
// meter barely moves. Subtract a noise floor, rescale into the
// speech-ceiling window, then apply a sqrt curve so quiet speech
// still registers visually without drowning loud speech at the top.
private const val NOISE_FLOOR = 0.01f
private const val SPEECH_CEILING = 0.35f
}
private val _amplitude = MutableStateFlow(0f)
val amplitude: StateFlow<Float> = _amplitude.asStateFlow()
private var mediaRecorder: MediaRecorder? = null
private var currentOutputFile: File? = null
private var pollJob: Job? = null
/**
* Begin a new recording. Returns the output [File] that will receive the
* audio once [stopRecording] is called. Throws on permission failure or
* encoder init failure — callers should catch and surface to the UI.
*/
@SuppressLint("MissingPermission")
fun startRecording(): File {
// Defensive: if a recording is somehow still running, tear it down
// before starting a new one. MediaRecorder transitions are strict.
if (mediaRecorder != null) {
Log.w(TAG, "startRecording called while another recording is in flight — stopping it first")
try {
stopRecording()
} catch (_: Exception) {
// Swallow — we're about to overwrite state anyway.
releaseRecorder()
}
}
val outFile = File(context.cacheDir, "voice_rec_${System.currentTimeMillis()}.m4a")
currentOutputFile = outFile
val recorder = buildRecorder()
try {
recorder.setAudioSource(MediaRecorder.AudioSource.MIC)
recorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC)
recorder.setAudioSamplingRate(SAMPLE_RATE)
recorder.setAudioEncodingBitRate(BIT_RATE)
recorder.setAudioChannels(1)
recorder.setOutputFile(outFile.absolutePath)
recorder.prepare()
recorder.start()
} catch (e: IllegalStateException) {
Log.e(TAG, "MediaRecorder failed to start: ${e.message}")
try {
recorder.reset()
} catch (_: Exception) { /* ignore */ }
recorder.release()
mediaRecorder = null
currentOutputFile = null
throw e
} catch (e: Exception) {
Log.e(TAG, "MediaRecorder setup failed: ${e.message}")
try {
recorder.reset()
} catch (_: Exception) { /* ignore */ }
recorder.release()
mediaRecorder = null
currentOutputFile = null
throw e
}
mediaRecorder = recorder
startAmplitudePolling()
return outFile
}
/**
* Stop the active recording, flush the encoder, and return the completed
* output [File]. Safe to call when nothing is recording — in that case
* it returns the last file produced, or throws if there never was one.
*/
fun stopRecording(): File {
val file = currentOutputFile
?: throw IllegalStateException("stopRecording called with no active recording")
stopAmplitudePolling()
val recorder = mediaRecorder
if (recorder != null) {
try {
recorder.stop()
} catch (e: IllegalStateException) {
// MediaRecorder.stop throws if called before any audio was
// captured (sub-300ms recordings). Treat as recoverable —
// the output file may be 0 bytes but the caller can check.
Log.w(TAG, "MediaRecorder.stop threw — recording may be empty: ${e.message}")
} catch (e: RuntimeException) {
Log.w(TAG, "MediaRecorder.stop runtime error: ${e.message}")
} finally {
releaseRecorder()
}
}
_amplitude.value = 0f
return file
}
/**
* True if a recording is currently active. Cheap — just checks whether
* we have a live [MediaRecorder] reference.
*/
fun isRecording(): Boolean = mediaRecorder != null
/**
* Release any recorder resources without returning a file. Safe fallback
* for error paths where the output file is known-invalid.
*/
fun cancel() {
stopAmplitudePolling()
mediaRecorder?.let { r ->
try {
r.stop()
} catch (_: Exception) { /* ignore */ }
}
releaseRecorder()
currentOutputFile?.let { f ->
try { f.delete() } catch (_: Exception) { /* ignore */ }
}
currentOutputFile = null
_amplitude.value = 0f
}
private fun releaseRecorder() {
mediaRecorder?.let { r ->
try { r.reset() } catch (_: Exception) { /* ignore */ }
try { r.release() } catch (_: Exception) { /* ignore */ }
}
mediaRecorder = null
}
@Suppress("DEPRECATION")
private fun buildRecorder(): MediaRecorder =
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
MediaRecorder(context)
} else {
MediaRecorder()
}
private fun startAmplitudePolling() {
pollJob?.cancel()
pollJob = scope.launch(Dispatchers.Default) {
while (isActive) {
val recorder = mediaRecorder ?: break
val raw = try {
recorder.maxAmplitude
} catch (e: IllegalStateException) {
// Recorder torn down under us — exit quietly.
break
}
val raw01 = (raw.toFloat() / MAX_AMPLITUDE_SHORT).coerceIn(0f, 1f)
val floored = ((raw01 - NOISE_FLOOR) / (SPEECH_CEILING - NOISE_FLOOR))
.coerceIn(0f, 1f)
_amplitude.value = sqrt(floored)
delay(AMPLITUDE_POLL_MS)
}
}
}
private fun stopAmplitudePolling() {
pollJob?.cancel()
pollJob = null
}
}
@@ -0,0 +1,146 @@
package com.hermesandroid.relay.audio
import android.content.Context
import android.media.AudioAttributes
import android.media.AudioFormat
import android.media.AudioTrack
import android.util.Log
import kotlin.math.PI
import kotlin.math.sin
/**
* Tiny synthesized-PCM chime player for voice-mode enter/exit feedback.
*
* Two buffers are generated once at construction — an ascending sweep
* (440 → 660 Hz, a perfect fifth) for enter, and its mirror (660 → 440 Hz)
* for exit. Both use the same attack / hold / decay envelope so the pair
* sounds symmetrical. Played via [AudioTrack] in [AudioTrack.MODE_STATIC]
* so replays are low-latency — the buffer is written once and we just
* rewind with [AudioTrack.reloadStaticData] on each trigger.
*
* Construction and every runtime call is wrapped in try/catch: some OEM
* devices refuse AudioTrack under unusual states (busy audio HAL, no route)
* and we never want a UI chime to crash voice mode.
*/
class VoiceSfxPlayer(@Suppress("UNUSED_PARAMETER") context: Context) {
private val enterTrack: AudioTrack? = buildChimeTrack(ascending = true)
private val exitTrack: AudioTrack? = buildChimeTrack(ascending = false)
fun playEnter() {
playTrack(enterTrack)
}
fun playExit() {
playTrack(exitTrack)
}
fun release() {
try { enterTrack?.release() } catch (e: Exception) { Log.w(TAG, "enter release failed: ${e.message}") }
try { exitTrack?.release() } catch (e: Exception) { Log.w(TAG, "exit release failed: ${e.message}") }
}
// ---------------------------------------------------------------------
private fun playTrack(track: AudioTrack?) {
if (track == null) return
try {
// Rewind the static buffer before each play so repeat invocations
// start from frame 0. stop() is a no-op if it's already stopped.
if (track.playState == AudioTrack.PLAYSTATE_PLAYING) {
track.pause()
}
track.stop()
track.reloadStaticData()
track.play()
} catch (e: Exception) {
Log.w(TAG, "chime play failed: ${e.message}")
}
}
private fun buildChimeTrack(ascending: Boolean): AudioTrack? {
return try {
val pcm = synthesizeSweep(ascending)
val bytes = pcm.size * 2 // 16-bit = 2 bytes/sample
val attrs = AudioAttributes.Builder()
// Assistant chime — closest semantic match for a voice-mode ping.
.setUsage(AudioAttributes.USAGE_ASSISTANT)
.setContentType(AudioAttributes.CONTENT_TYPE_SONIFICATION)
.build()
val format = AudioFormat.Builder()
.setSampleRate(SAMPLE_RATE)
.setEncoding(AudioFormat.ENCODING_PCM_16BIT)
.setChannelMask(AudioFormat.CHANNEL_OUT_MONO)
.build()
val track = AudioTrack.Builder()
.setAudioAttributes(attrs)
.setAudioFormat(format)
.setBufferSizeInBytes(bytes)
.setTransferMode(AudioTrack.MODE_STATIC)
.build()
val written = track.write(pcm, 0, pcm.size)
if (written < 0) {
Log.w(TAG, "AudioTrack.write returned $written; skipping chime")
track.release()
return null
}
track
} catch (e: Exception) {
Log.w(TAG, "AudioTrack construction failed: ${e.message}")
null
}
}
/**
* Generate a 200 ms sine sweep with a fast attack / smooth decay envelope.
* Peak amplitude capped at 0.35 of full-scale so it reads as a subtle UI
* chime rather than a notification.
*/
private fun synthesizeSweep(ascending: Boolean): ShortArray {
val totalSamples = (SAMPLE_RATE * DURATION_SEC).toInt()
val attackSamples = (SAMPLE_RATE * ATTACK_SEC).toInt()
val decaySamples = (SAMPLE_RATE * DECAY_SEC).toInt()
val out = ShortArray(totalSamples)
val startHz = if (ascending) LOW_HZ else HIGH_HZ
val endHz = if (ascending) HIGH_HZ else LOW_HZ
// Accumulate phase from instantaneous frequency so the sweep is
// continuous — computing sin(2π·f(t)·t) directly produces chirp
// artifacts because f is itself a function of t.
var phase = 0.0
for (i in 0 until totalSamples) {
val t = i.toDouble() / totalSamples
val freq = startHz + (endHz - startHz) * t
phase += 2.0 * PI * freq / SAMPLE_RATE
// Envelope: linear ramp up over attack, smooth cosine fade over decay.
val env = when {
i < attackSamples -> i.toFloat() / attackSamples
i >= totalSamples - decaySamples -> {
val d = (totalSamples - i).toFloat() / decaySamples
// Half-cosine ease: 0 → 1 as d goes 0 → 1.
(0.5f * (1f - kotlin.math.cos(PI.toFloat() * d)))
}
else -> 1f
}
val sample = sin(phase).toFloat() * env * PEAK_AMPLITUDE
out[i] = (sample * Short.MAX_VALUE).toInt()
.coerceIn(Short.MIN_VALUE.toInt(), Short.MAX_VALUE.toInt())
.toShort()
}
return out
}
companion object {
private const val TAG = "VoiceSfxPlayer"
private const val SAMPLE_RATE = 44100
private const val DURATION_SEC = 0.20f
private const val ATTACK_SEC = 0.015f
private const val DECAY_SEC = 0.040f
private const val LOW_HZ = 440.0
private const val HIGH_HZ = 660.0
private const val PEAK_AMPLITUDE = 0.35f
}
}
@@ -1,9 +1,8 @@
package com.hermesandroid.relay.auth
import android.content.Context
import android.content.SharedPreferences
import androidx.security.crypto.EncryptedSharedPreferences
import androidx.security.crypto.MasterKey
import android.util.Log
import com.hermesandroid.relay.data.PairingPreferences
import com.hermesandroid.relay.network.ChannelMultiplexer
import com.hermesandroid.relay.network.models.Envelope
import kotlinx.coroutines.CoroutineScope
@@ -16,10 +15,15 @@ import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.contentOrNull
import kotlinx.serialization.json.doubleOrNull
import kotlinx.serialization.json.jsonArray
import kotlinx.serialization.json.jsonObject
import kotlinx.serialization.json.jsonPrimitive
import kotlinx.serialization.json.longOrNull
import kotlinx.serialization.json.put
sealed class AuthState {
@@ -29,6 +33,27 @@ sealed class AuthState {
data class Failed(val reason: String) : AuthState()
}
/**
* Orchestrates pairing + session token lifecycle for the relay channel.
*
* **Security overhaul (2026-04-11):**
*
* - Token storage is abstracted behind [SessionTokenStore]. On first run
* we try [KeystoreTokenStore] (StrongBox-preferred) and fall back to
* [LegacyEncryptedPrefsTokenStore] when Keystore init fails. Existing
* tokens are migrated one-shot from the legacy prefs file on first
* launch so users don't need to re-pair after the upgrade.
*
* - TTL + grants + transport_hint are now parsed from the server's
* `auth.ok` payload and exposed via [currentPairedSession] so the UI can
* display expiry badges and per-channel grant chips.
*
* - The next [authenticate] call sends a user-selected `ttl_seconds` from
* the [SessionTtlPickerDialog]. See [setPendingTtlSeconds].
*
* - TOFU cert pinning lives in [CertPinStore]; re-pair flows call
* [CertPinStore.removePinFor] to reset trust for the target host.
*/
class AuthManager(
private val context: Context,
private val multiplexer: ChannelMultiplexer,
@@ -36,79 +61,312 @@ class AuthManager(
) : ChannelMultiplexer.ChannelHandler {
companion object {
private const val PREFS_NAME = "hermes_companion_auth"
private const val TAG = "AuthManager"
private const val KEY_SESSION_TOKEN = "session_token"
private const val KEY_DEVICE_ID = "device_id"
private const val KEY_API_KEY = "api_server_key"
private const val KEY_PAIRED_META = "paired_session_meta_json"
private const val PAIRING_CODE_LENGTH = 6
private val PAIRING_CODE_CHARS = ('A'..'Z') + ('0'..'9')
}
private val json = Json { ignoreUnknownKeys = true }
// Thread-safe lazy-init crypto on first access (off main thread)
private var _prefs: SharedPreferences? = null
private val prefsMutex = Mutex()
private suspend fun prefs(): SharedPreferences {
_prefs?.let { return it }
return prefsMutex.withLock {
// Double-check inside lock
_prefs?.let { return it }
// --- Token store (new, hardware-backed preferred) -----------------------
private var _store: SessionTokenStore? = null
private val storeMutex = Mutex()
/**
* Lazily construct the best available token store. First tries
* [KeystoreTokenStore] — if that fails on broken OEM keystores we fall
* back to [LegacyEncryptedPrefsTokenStore]. The chosen store is cached
* for the lifetime of this manager.
*
* After picking a store we run [migrateFromLegacyIfNeeded] once so any
* existing session token lands in the new location without forcing the
* user to re-pair.
*/
private suspend fun store(): SessionTokenStore {
_store?.let { return it }
return storeMutex.withLock {
_store?.let { return it }
withContext(Dispatchers.IO) {
val masterKey = MasterKey.Builder(context)
.setKeyScheme(MasterKey.KeyScheme.AES256_GCM)
.build()
EncryptedSharedPreferences.create(
context,
PREFS_NAME,
masterKey,
EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM
).also { _prefs = it }
val picked: SessionTokenStore =
KeystoreTokenStore.tryCreate(context) ?: LegacyEncryptedPrefsTokenStore(context)
migrateFromLegacyIfNeeded(picked)
_store = picked
picked
}
}
}
/**
* One-shot migration: copy session token + device ID + API key from the
* legacy [EncryptedSharedPreferences] file into the new store, then
* clear the legacy file. No-op when [picked] is itself the legacy store
* (nothing to migrate from — they're the same file).
*/
private fun migrateFromLegacyIfNeeded(picked: SessionTokenStore) {
if (picked is LegacyEncryptedPrefsTokenStore) return
val legacy = try {
LegacyEncryptedPrefsTokenStore(context)
} catch (_: Exception) {
return
}
val keysToMigrate = listOf(KEY_SESSION_TOKEN, KEY_DEVICE_ID, KEY_API_KEY, KEY_PAIRED_META)
var migrated = false
for (k in keysToMigrate) {
val existing = legacy.getString(k) ?: continue
if (!picked.contains(k)) {
picked.putString(k, existing)
migrated = true
}
}
if (migrated) {
// Wipe the legacy file so we don't keep cleartext-equivalent
// backup copies of the session token lying around.
legacy.clearAll()
}
}
/** Cert pin store — shared across all relay connections. */
val certPinStore: CertPinStore = CertPinStore(context)
/** True when the resolved token store reports StrongBox-backed storage. */
val hasHardwareBackedStorage: Boolean
get() = _store?.hasHardwareBackedStorage ?: false
private val _authState = MutableStateFlow<AuthState>(AuthState.Unpaired)
val authState: StateFlow<AuthState> = _authState.asStateFlow()
private val _pairingCode = MutableStateFlow(generatePairingCode())
val pairingCode: StateFlow<String> = _pairingCode.asStateFlow()
/**
* Snapshot of the current paired session metadata — token, expiry,
* grants, transport hint. `null` when unpaired. Exposed to the UI via
* [com.hermesandroid.relay.viewmodel.ConnectionViewModel.currentPairedSession].
*/
private val _currentPairedSession = MutableStateFlow<PairedSession?>(null)
val currentPairedSession: StateFlow<PairedSession?> = _currentPairedSession.asStateFlow()
/**
* When non-null, the next [authenticate] call sends this code instead
* of [_pairingCode]. Populated via [applyServerIssuedCode] when the
* user scans a QR that carries a server-issued relay pairing code.
*
* Cleared automatically after a successful `auth.ok` so we fall back
* to the locally-generated code (which stays as the trust anchor for
* the phone → server direction used by the Phase 3 bridge channel).
*/
private var serverIssuedCode: String? = null
/**
* Read-only view of whether this AuthManager currently has something to
* say in an auth envelope — either a pending server-issued pair code
* from a fresh QR scan or a stored session token (via [authState]).
*
* Used by [com.hermesandroid.relay.viewmodel.ConnectionViewModel.connectRelay]
* to gate WSS connect attempts. Without a pair context, firing the WSS
* handshake just sends a doomed auth envelope, which makes the relay
* tick its rate limiter 5 times in 60 seconds and then block the IP
* for 5 minutes — exactly the scenario that bit us on the first cycle
* of the inbound media test.
*
* Returns true if **any** of:
* 1. `authState` is [AuthState.Paired] (we have a session token)
* 2. `authState` is [AuthState.Pairing] (mid-handshake)
* 3. [serverIssuedCode] is non-null (fresh QR scan just landed, about
* to authenticate)
*
* Does NOT consider the locally-generated [_pairingCode] as a valid
* context — that code is for the Phase 3 phone→host direction and is
* NOT registered on the relay side, so sending it yields guaranteed
* auth failures.
*/
val hasPairContext: Boolean
get() = _authState.value is AuthState.Paired ||
_authState.value is AuthState.Pairing ||
serverIssuedCode != null
/**
* TTL the user picked at the [SessionTtlPickerDialog]. Included in the
* next auth envelope's payload as `ttl_seconds`. `null` means "use
* server default" (equivalent to the pre-overhaul behavior — the
* server's own policy kicks in). `0` means "never expire".
*/
private var pendingTtlSeconds: Long? = null
/**
* Optional per-channel grant preferences from the QR. If set, included
* in the auth envelope so the server can issue shorter-lived tokens for
* specific channels. Null means "use the server's default for each
* channel".
*/
private var pendingGrants: Map<String, Long>? = null
private val _profiles = MutableStateFlow<List<String>>(emptyList())
val profiles: StateFlow<List<String>> = _profiles.asStateFlow()
/**
* Whether an API key is currently stored. Updated reactively by
* [setApiKey] / [clearApiKey] so composables can show "API key: already
* set — leave blank to keep" without suspending.
*/
private val _apiKeyPresent = MutableStateFlow(false)
val apiKeyPresent: StateFlow<Boolean> = _apiKeyPresent.asStateFlow()
init {
// Register as system channel handler for auth messages
multiplexer.registerHandler("system", this)
// Check for existing session token off main thread
scope.launch {
val existingToken = prefs().getString(KEY_SESSION_TOKEN, null)
val s = store()
val existingToken = s.getString(KEY_SESSION_TOKEN)
if (existingToken != null) {
_authState.value = AuthState.Paired(existingToken)
_currentPairedSession.value = loadStoredMetadata(existingToken)
Log.i(
TAG,
"init: hydrated existing session_token=${existingToken.take(8)}… " +
"→ authState=Paired (stale-at-startup unless this is a real continuous session)"
)
} else {
Log.i(TAG, "init: no stored session_token → authState stays Unpaired")
}
_apiKeyPresent.value = !s.getString(KEY_API_KEY).isNullOrBlank()
}
}
/**
* Re-hydrate a [PairedSession] snapshot from the JSON blob we persisted
* alongside the session token. Returns a minimal fallback when metadata
* is missing (old installs without the blob).
*/
private suspend fun loadStoredMetadata(token: String): PairedSession {
val s = store()
val rawMeta = s.getString(KEY_PAIRED_META)
val now = System.currentTimeMillis() / 1000L
val defaults = PairedSession(
token = token,
deviceName = android.os.Build.MODEL,
expiresAt = null,
grants = emptyMap(),
transportHint = null,
firstSeen = now,
hasHardwareStorage = s.hasHardwareBackedStorage,
)
if (rawMeta.isNullOrBlank()) return defaults
return try {
val obj = json.decodeFromString<JsonObject>(rawMeta)
val expiresAt = obj["expires_at"]?.jsonPrimitive?.longOrNull
?: obj["expires_at"]?.jsonPrimitive?.doubleOrNull?.toLong()
val grants = obj["grants"]?.jsonObject?.mapValues { (_, v) ->
(v as? JsonPrimitive)?.longOrNull
?: (v as? JsonPrimitive)?.doubleOrNull?.toLong()
} ?: emptyMap()
val transportHint = obj["transport_hint"]?.jsonPrimitive?.contentOrNull
val firstSeen = obj["first_seen"]?.jsonPrimitive?.longOrNull ?: now
val deviceName = obj["device_name"]?.jsonPrimitive?.contentOrNull
?: android.os.Build.MODEL
PairedSession(
token = token,
deviceName = deviceName,
expiresAt = expiresAt,
grants = grants,
transportHint = transportHint,
firstSeen = firstSeen,
hasHardwareStorage = s.hasHardwareBackedStorage,
)
} catch (_: Exception) {
defaults
}
}
private suspend fun persistPairedSession(session: PairedSession) {
val s = store()
val meta = buildJsonObject {
put("device_name", session.deviceName)
session.expiresAt?.let { put("expires_at", it) }
put("transport_hint", session.transportHint ?: "")
put("first_seen", session.firstSeen)
val grantsObj = buildJsonObject {
for ((k, v) in session.grants) {
if (v != null) put(k, v)
}
}
put("grants", grantsObj)
}
s.putString(KEY_PAIRED_META, json.encodeToString(JsonObject.serializer(), meta))
}
private suspend fun getDeviceId(): String {
val p = prefs()
val existing = p.getString(KEY_DEVICE_ID, null)
val s = store()
val existing = s.getString(KEY_DEVICE_ID)
if (existing != null) return existing
val newId = java.util.UUID.randomUUID().toString()
p.edit().putString(KEY_DEVICE_ID, newId).apply()
s.putString(KEY_DEVICE_ID, newId)
return newId
}
/**
* Send auth envelope when connection is established.
* Public accessor for the stable device ID. Used by [TerminalViewModel]
* to compose deterministic per-tab `session_name` values
* (`hermes-<deviceId>-tabN`) so the relay's tmux-backed terminal channel
* can re-attach the same shell across reconnects.
*/
fun authenticate() {
suspend fun getOrCreateDeviceId(): String = getDeviceId()
/**
* Set the TTL the user picked at [SessionTtlPickerDialog]. `0` → never,
* `null` → defer to server default. Persisted across [authenticate]
* calls and mirrored into [PairingPreferences] for the next pair's
* preselection default.
*/
fun setPendingTtlSeconds(ttlSeconds: Long?) {
pendingTtlSeconds = ttlSeconds
if (ttlSeconds != null) {
scope.launch {
PairingPreferences.setPairTtlSeconds(context, ttlSeconds)
}
}
}
fun setPendingGrants(grants: Map<String, Long>?) {
pendingGrants = grants
}
/**
* Send auth envelope when connection is established.
*
* Pairing-code precedence when unpaired:
* 1. [serverIssuedCode] if set — comes from a scanned QR. This is the
* canonical path: operator runs `hermes pair` on the host, which
* pre-registers the code with the running relay, then renders it
* into the QR. The phone scans once and consumes it here.
* 2. [_pairingCode] (locally-generated). Fallback for manual setups
* and Phase 3's bridge direction (phone-issues-code, host-approves).
*
* Pending TTL + grants are serialized into the auth payload when set,
* letting the [SessionTtlPickerDialog] selection flow through to the
* server.
*/
fun authenticate(ttlSeconds: Long? = null) {
if (ttlSeconds != null) pendingTtlSeconds = ttlSeconds
scope.launch {
val currentState = _authState.value
val deviceId = getDeviceId()
val payload = when (currentState) {
is AuthState.Paired -> {
Log.i(
TAG,
"authenticate: sending session_token (state=Paired, token=${currentState.token.take(8)}…)"
)
buildJsonObject {
put("session_token", currentState.token)
put("device_id", deviceId)
@@ -117,10 +375,24 @@ class AuthManager(
}
else -> {
_authState.value = AuthState.Pairing
val codeToSend = serverIssuedCode ?: _pairingCode.value
val serverSource = if (serverIssuedCode != null) "QR" else "local-fallback"
Log.i(
TAG,
"authenticate: sending pairing_code=$codeToSend source=$serverSource " +
"ttl=$pendingTtlSeconds grants=${pendingGrants?.keys}"
)
buildJsonObject {
put("pairing_code", _pairingCode.value)
put("pairing_code", codeToSend)
put("device_id", deviceId)
put("device_name", android.os.Build.MODEL)
pendingTtlSeconds?.let { put("ttl_seconds", it) }
pendingGrants?.let { grants ->
val obj = buildJsonObject {
for ((k, v) in grants) put(k, v)
}
put("grants", obj)
}
}
}
}
@@ -135,7 +407,60 @@ class AuthManager(
}
}
/**
* Store a server-issued pairing code (from a scanned QR) for use on the
* next [authenticate] call. Overrides the locally-generated code; cleared
* automatically on successful `auth.ok`.
*
* Also mirrors the code into [_pairingCode] so any UI that displays the
* code shows the one actually being used — avoids a confusing "QR said
* ABCD12 but the app shows XYZ123" gap during the pairing moment.
*/
fun applyServerIssuedCode(code: String) {
val normalized = code.trim().uppercase()
if (normalized.isEmpty()) return
serverIssuedCode = normalized
_pairingCode.value = normalized
}
/**
* Apply a server-issued code AND wipe any existing session token in one
* atomic step. Used by the manual-code entry dialog and by the QR flow
* whenever the user re-pairs.
*
* Also wipes the TOFU cert pin for the target relay host (if provided)
* so the next wss handshake re-TOFUs against whatever cert is currently
* being presented. Without this step, a legit cert rotation looks
* identical to a MITM attack.
*/
fun applyServerIssuedCodeAndReset(code: String, relayUrl: String? = null) {
val normalized = code.trim().uppercase()
if (normalized.isEmpty()) {
Log.w(TAG, "applyServerIssuedCodeAndReset: empty code, returning early — authState NOT reset")
return
}
val prevState = _authState.value
serverIssuedCode = normalized
_pairingCode.value = normalized
_authState.value = AuthState.Unpaired
_currentPairedSession.value = null
Log.i(
TAG,
"applyServerIssuedCodeAndReset: code=$normalized relayUrl=$relayUrl " +
"prevState=${prevState::class.simpleName} → Unpaired"
)
scope.launch {
val s = store()
s.remove(KEY_SESSION_TOKEN)
s.remove(KEY_PAIRED_META)
if (relayUrl != null) {
certPinStore.removePinFor(relayUrl)
}
}
}
override fun onMessage(envelope: Envelope) {
Log.i(TAG, "onMessage channel=${envelope.channel} type=${envelope.type}")
when (envelope.type) {
"auth.ok" -> handleAuthOk(envelope)
"auth.fail" -> handleAuthFail(envelope)
@@ -148,29 +473,34 @@ class AuthManager(
fun clearSession() {
scope.launch {
prefs().edit().remove(KEY_SESSION_TOKEN).apply()
val s = store()
s.remove(KEY_SESSION_TOKEN)
s.remove(KEY_PAIRED_META)
_authState.value = AuthState.Unpaired
_currentPairedSession.value = null
_pairingCode.value = generatePairingCode()
}
}
// --- API Key storage (for direct Hermes API Server auth) ---
suspend fun getApiKey(): String? {
return prefs().getString(KEY_API_KEY, null)
}
suspend fun getApiKey(): String? = store().getString(KEY_API_KEY)
suspend fun setApiKey(key: String) {
val trimmed = key.trim()
val s = store()
if (trimmed.isBlank()) {
prefs().edit().remove(KEY_API_KEY).apply()
s.remove(KEY_API_KEY)
_apiKeyPresent.value = false
} else {
prefs().edit().putString(KEY_API_KEY, trimmed).apply()
s.putString(KEY_API_KEY, trimmed)
_apiKeyPresent.value = true
}
}
suspend fun clearApiKey() {
prefs().edit().remove(KEY_API_KEY).apply()
store().remove(KEY_API_KEY)
_apiKeyPresent.value = false
}
val isPaired: Boolean
@@ -182,9 +512,64 @@ class AuthManager(
val payload = envelope.payload
val token = payload["session_token"]?.jsonPrimitive?.contentOrNull
if (token == null) {
Log.w(
TAG,
"handleAuthOk: payload missing session_token — authState NOT transitioned to Paired. " +
"Payload keys: ${payload.keys}"
)
}
if (token != null) {
prefs().edit().putString(KEY_SESSION_TOKEN, token).apply()
val s = store()
s.putString(KEY_SESSION_TOKEN, token)
_authState.value = AuthState.Paired(token)
Log.i(TAG, "handleAuthOk: Paired(token=${token.take(8)}…)")
// Server-issued code is one-shot — drop it once the
// upgrade to a long-lived session token has landed.
serverIssuedCode = null
// --- Parse new security fields -------------------------
//
// expires_at: epoch seconds or null (never expires).
// Accepts both integer and float representations —
// Python servers often return time.time() which is a
// float. We go through jsonPrimitive so `null` literal
// in the payload surfaces as a null Long here.
val expiresAtElem = payload["expires_at"]
val expiresAt: Long? = expiresAtElem?.jsonPrimitive?.let { prim ->
prim.longOrNull ?: prim.doubleOrNull?.toLong()
}
// grants: map of channel name → epoch seconds | null
val grantsObj = payload["grants"] as? JsonObject
val grantsMap: Map<String, Long?> = grantsObj
?.mapValues { (_, v) ->
val prim = v as? JsonPrimitive
prim?.longOrNull ?: prim?.doubleOrNull?.toLong()
}
?: emptyMap()
val transportHint = payload["transport_hint"]
?.jsonPrimitive?.contentOrNull
val paired = PairedSession(
token = token,
deviceName = android.os.Build.MODEL,
expiresAt = expiresAt,
grants = grantsMap,
transportHint = transportHint,
firstSeen = System.currentTimeMillis() / 1000L,
hasHardwareStorage = s.hasHardwareBackedStorage,
)
_currentPairedSession.value = paired
persistPairedSession(paired)
// Pending TTL/grants are consumed — the server has
// either honored or overridden them and the next
// auth round-trip should not resend stale values.
pendingTtlSeconds = null
pendingGrants = null
}
val profilesArray = payload["profiles"]?.jsonArray
@@ -199,13 +584,51 @@ class AuthManager(
private fun handleAuthFail(envelope: Envelope) {
try {
val reason = envelope.payload["reason"]?.jsonPrimitive?.contentOrNull ?: "Unknown error"
_authState.value = AuthState.Failed(reason)
val rawReason = envelope.payload["reason"]?.jsonPrimitive?.contentOrNull
?: "Unknown error"
val humanized = humanizeAuthFailReason(rawReason)
Log.w(TAG, "handleAuthFail: raw=$rawReason humanized=$humanized")
_authState.value = AuthState.Failed(humanized)
} catch (e: Exception) {
Log.w(TAG, "handleAuthFail: exception parsing payload", e)
_authState.value = AuthState.Failed("Authentication failed")
}
}
/**
* Map common relay `auth.fail` reasons to user-friendly short strings.
* The wizard VerifyStep surfaces the returned text directly, so this is
* what the user reads when a pair fails.
*
* Pass-through for anything we don't recognize so server-side debug
* output isn't clobbered.
*/
private fun humanizeAuthFailReason(raw: String): String {
val lower = raw.lowercase()
return when {
// "pairing code not recognized", "invalid pairing code",
// "unknown pairing code", etc. — the code is no longer on the
// relay, which almost always means the QR was already used.
"pairing" in lower && ("not recogniz" in lower ||
"invalid" in lower ||
"unknown" in lower ||
"consumed" in lower ||
"already used" in lower) ->
"That pairing code was already used. Generate a fresh QR " +
"from `hermes-pair` and scan again."
"rate" in lower && "limit" in lower ->
"Too many pair attempts — the relay temporarily blocked your " +
"IP. Wait ~5 minutes and try again."
"expired" in lower ->
"The pairing code expired. Generate a fresh QR and scan again."
"session" in lower && "expired" in lower ->
"Your session expired. Re-pair to get a new one."
"session_token" in lower || "token" in lower ->
"The server rejected your saved session. Re-pair to get a new one."
else -> raw
}
}
private fun generatePairingCode(): String {
return (1..PAIRING_CODE_LENGTH)
.map { PAIRING_CODE_CHARS.random() }
@@ -0,0 +1,150 @@
package com.hermesandroid.relay.auth
import android.content.Context
import android.util.Base64
import android.util.Log
import com.hermesandroid.relay.data.PairingPreferences
import kotlinx.coroutines.runBlocking
import okhttp3.CertificatePinner
import java.net.URI
import java.security.MessageDigest
import java.security.cert.Certificate
import java.security.cert.X509Certificate
/**
* Trust-on-first-use (TOFU) certificate pinning store.
*
* Records the SHA-256 fingerprint of the peer certificate the first time we
* successfully connect over wss to a given host:port. Subsequent connects
* build an OkHttp [CertificatePinner] with the stored pin. If the cert
* changes, OkHttp will refuse the connection and our reconnect loop will
* surface it as a loud error.
*
* Scope and limitations:
* - Only applies to `wss://` — `ws://` is unencrypted, pinning is moot.
* - Pin format is OkHttp's `sha256/<base64>` — matches what
* [CertificatePinner.pin] expects.
* - Pins are keyed by `host:port` (lowercase). If the user re-pairs via QR
* we wipe the pin for that host via [removePinFor] so the next connect
* re-TOFUs against the new cert.
* - Wildcards and SAN matching are NOT supported — only exact host matches.
* Relay operators running a single host per port is the common case.
*
* Storage lives in DataStore via [PairingPreferences.setTofuPin].
*/
class CertPinStore(private val context: Context) {
companion object {
private const val TAG = "CertPinStore"
/**
* Compute the OkHttp-compatible `sha256/<base64>` pin string for an
* [X509Certificate]. Mirrors [CertificatePinner.pin] internally.
*/
fun fingerprint(cert: X509Certificate): String {
val spki = cert.publicKey.encoded
val digest = MessageDigest.getInstance("SHA-256").digest(spki)
val b64 = Base64.encodeToString(digest, Base64.NO_WRAP)
return "sha256/$b64"
}
/** Extract `host:port` from a ws/wss URL. Returns null on malformed input. */
fun hostPortFromUrl(wsUrl: String): String? {
return try {
val uri = URI(wsUrl.trim())
val host = uri.host ?: return null
val port = when {
uri.port > 0 -> uri.port
uri.scheme.equals("wss", ignoreCase = true) -> 443
uri.scheme.equals("https", ignoreCase = true) -> 443
else -> 80
}
"${host.lowercase()}:$port"
} catch (_: Exception) {
null
}
}
/** True when the URL is secure (wss/https) and pinning is meaningful. */
fun isPinnableUrl(url: String): Boolean {
val trimmed = url.trim().lowercase()
return trimmed.startsWith("wss://") || trimmed.startsWith("https://")
}
}
/**
* Snapshot of all stored pins. Blocks on the DataStore Flow — caller
* should be off the main thread. Used by [buildPinnerSnapshot] which is
* called from OkHttp builder setup.
*/
private fun getPinsBlocking(): Map<String, String> =
runBlocking { PairingPreferences.getTofuPins(context) }
/**
* Build an OkHttp [CertificatePinner] from the current pin store. Empty
* pin store → returns [CertificatePinner.DEFAULT], which allows any cert
* (so first-time TOFU still works — we record on first successful connect).
*/
fun buildPinnerSnapshot(): CertificatePinner {
val pins = getPinsBlocking()
if (pins.isEmpty()) return CertificatePinner.DEFAULT
val builder = CertificatePinner.Builder()
for ((hostPort, pin) in pins) {
val host = hostPort.substringBefore(':')
builder.add(host, pin)
}
return builder.build()
}
/**
* Record a pin for a host. Called from the WebSocket listener's `onOpen`
* when we have a successful connection and can read the peer certs from
* the response handshake. Idempotent — if the pin already matches, we
* simply overwrite with the same value.
*
* If there's already a pin for this host and the new fingerprint differs,
* this logs a loud warning but still overwrites. The intended safe path
* for cert rotation is a re-pair through the QR flow, which calls
* [removePinFor] before the next connect.
*/
suspend fun recordPinIfAbsent(url: String, peerCerts: List<Certificate>) {
if (!isPinnableUrl(url)) return
val hostPort = hostPortFromUrl(url) ?: return
val leaf = peerCerts.firstOrNull() as? X509Certificate ?: return
val newPin = fingerprint(leaf)
val existing = PairingPreferences.getTofuPins(context)[hostPort]
if (existing == null) {
Log.i(TAG, "TOFU: recording new pin for $hostPort -> $newPin")
PairingPreferences.setTofuPin(context, hostPort, newPin)
} else if (existing != newPin) {
// If a mismatched pin survives into this code path, something
// bypassed the OkHttp CertificatePinner check. Log it loudly so
// it shows up in bug reports. Keep the existing pin — do NOT
// silently overwrite.
Log.e(
TAG,
"TOFU: pin MISMATCH for $hostPort (stored=$existing, peer=$newPin). " +
"Keeping stored pin. Re-pair to accept the new certificate."
)
}
}
/**
* Wipe the stored pin for a host. Called when the user explicitly re-pairs
* from a QR — the new pairing is an implicit trust-reset.
*/
suspend fun removePinFor(url: String) {
val hostPort = hostPortFromUrl(url) ?: return
Log.i(TAG, "TOFU: removing pin for $hostPort (re-pair)")
PairingPreferences.removeTofuPin(context, hostPort)
}
/**
* Snapshot the list of currently pinned hosts. Suspending version for UI
* use — backs [PairedDevicesScreen]'s security badge hints.
*/
suspend fun listPinnedHosts(): List<String> =
PairingPreferences.getTofuPins(context).keys.toList()
}
@@ -0,0 +1,87 @@
package com.hermesandroid.relay.auth
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
/**
* Snapshot of the currently-paired session held by [AuthManager].
*
* Populated from the server's `auth.ok` payload after a successful pair
* and persisted alongside the session token. Exposed to the UI via
* [AuthManager.currentPairedSession] so screens can display expiry, grants,
* and transport posture without reaching into [android.content.SharedPreferences].
*
* @property token the long-lived relay session token. Same value carried in
* the legacy `AuthState.Paired(token)`.
* @property deviceName the human-readable device name the phone sent during
* pairing. Mirrored back here for UI display.
* @property expiresAt epoch seconds at which the server says the session
* expires — or `null` when the user chose "never expire" at the
* TTL picker. `null` is a first-class value, not a missing field.
* @property grants per-channel expiry map. Keys: `"chat"`, `"terminal"`,
* `"bridge"` (and any future channel the server adds). Values are
* epoch seconds or `null` for "never". Missing channels = server
* didn't grant that channel to this device.
* @property transportHint the transport the server advises the phone to
* use, for UX labeling only. `"wss"` / `"ws"` / `null` when the
* server didn't provide a hint.
* @property firstSeen epoch seconds of when this session record was created
* locally (from `System.currentTimeMillis() / 1000`).
* @property hasHardwareStorage whether the [SessionTokenStore] that persists
* the token reports StrongBox-backed storage. Drives the 🛡 badge
* in the paired devices screen.
*/
data class PairedSession(
val token: String,
val deviceName: String,
val expiresAt: Long?,
val grants: Map<String, Long?>,
val transportHint: String?,
val firstSeen: Long,
val hasHardwareStorage: Boolean,
) {
/** True when the session has no expiry — the user picked "Never" at pair time. */
val neverExpires: Boolean get() = expiresAt == null
/**
* True when the server-issued `expires_at` is in the past. Computed on
* read (not stored) so the UI can render "Expired" bands the moment the
* clock rolls over without needing an observer.
*/
fun isExpired(nowSeconds: Long = System.currentTimeMillis() / 1000L): Boolean {
val expiry = expiresAt ?: return false
return expiry < nowSeconds
}
}
/**
* A single paired device as returned by the relay's `GET /sessions` endpoint.
*
* One phone = one entry. When the phone is looking at its own entry,
* [isCurrent] is true. Used by [PairedDevicesScreen] to render revocable
* device cards.
*
* Wire contract: matches the sibling Python agent's response shape. Extra
* fields are tolerated via `ignoreUnknownKeys = true` so the server can grow
* the shape without breaking existing phones.
*/
@Serializable
data class PairedDeviceInfo(
@SerialName("token_prefix")
val tokenPrefix: String,
@SerialName("device_name")
val deviceName: String = "",
@SerialName("device_id")
val deviceId: String = "",
@SerialName("created_at")
val createdAt: Double? = null,
@SerialName("last_seen")
val lastSeen: Double? = null,
@SerialName("expires_at")
val expiresAt: Double? = null,
val grants: Map<String, Double?> = emptyMap(),
@SerialName("transport_hint")
val transportHint: String? = null,
@SerialName("is_current")
val isCurrent: Boolean = false,
)
@@ -0,0 +1,322 @@
package com.hermesandroid.relay.auth
import android.content.Context
import android.content.SharedPreferences
import android.os.Build
import android.util.Log
import androidx.security.crypto.EncryptedSharedPreferences
import androidx.security.crypto.MasterKey
/**
* Abstraction over the storage backend for the relay session token + API key
* + device ID.
*
* Two implementations exist:
*
* - [KeystoreTokenStore] — preferred. Uses [MasterKey] with
* `setRequestStrongBoxBacked(true)` on Android P+ devices that report a
* dedicated secure element (`PackageManager.FEATURE_STRONGBOX_KEYSTORE`).
* Falls back to AES256_GCM on TEE when StrongBox isn't available.
*
* - [LegacyEncryptedPrefsTokenStore] — fallback. The plain
* `MasterKey.Builder().setKeyScheme(AES256_GCM)` path that the pre-
* overhaul [AuthManager] used. Still strong — AES256_GCM is hardware-
* backed on almost every shipping device — but lacks the StrongBox
* attestation guarantee.
*
* Why an interface: we want to migrate existing installs off the legacy path
* without forcing users to re-pair. [AuthManager] picks the best available
* store, one-shot copies the legacy prefs in, and clears them. See
* `migrateFromLegacyIfNeeded` in [AuthManager].
*
* The store reports whether it's actually hardware-backed via
* [hasHardwareBackedStorage] so the UI can show a "🛡 hardware-backed" badge
* on the paired-device card.
*/
interface SessionTokenStore {
/**
* True when the underlying [MasterKey] is backed by a dedicated secure
* element (StrongBox). A `false` here doesn't mean "insecure" — it just
* means the key is in TEE or software-emulated, depending on the device.
*/
val hasHardwareBackedStorage: Boolean
fun getString(key: String): String?
fun putString(key: String, value: String)
fun remove(key: String)
fun contains(key: String): Boolean
/** Wipe every entry — used by `resetAppData` / "factory reset" in Settings. */
fun clearAll()
}
// ---------------------------------------------------------------------------
// Keystore-backed implementation
// ---------------------------------------------------------------------------
/**
* [SessionTokenStore] using [EncryptedSharedPreferences] backed by a
* [MasterKey] requesting StrongBox when available.
*
* On Android P+ devices with `FEATURE_STRONGBOX_KEYSTORE`, the master key
* lives in the dedicated secure element and all crypto ops happen there.
* On older devices or devices without StrongBox, this silently falls back
* to TEE-backed AES256_GCM — still hardware-backed on virtually every real
* phone.
*/
class KeystoreTokenStore private constructor(
private val context: Context,
private val wantsStrongBox: Boolean,
override val hasHardwareBackedStorage: Boolean
) : SessionTokenStore {
// Mutable so [resetPrefs] can swap in a fresh instance after a corrupted
// file is deleted. Built lazily via [buildPrefs] so the constructor can't
// throw — [tryCreate] still controls the "is this device usable at all"
// decision via its init probe below.
private var prefs: SharedPreferences = buildPrefs()
private fun buildPrefs(): SharedPreferences {
val builder = MasterKey.Builder(context)
.setKeyScheme(MasterKey.KeyScheme.AES256_GCM)
if (wantsStrongBox) {
try {
builder.setRequestStrongBoxBacked(true)
} catch (e: Exception) {
Log.w(TAG, "StrongBox request failed, falling back: ${e.message}")
}
}
val masterKey = builder.build()
return EncryptedSharedPreferences.create(
context,
PREFS_NAME,
masterKey,
EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM
)
}
/**
* Nuke a corrupted EncryptedSharedPreferences file and rebuild a fresh
* one. Triggered from any read/write that throws — the typical failure
* mode is the master key getting rotated out from under us during a
* Studio reinstall, after which every decrypt fails with
* `AEADBadTagException` (or sometimes a wrapped `GeneralSecurityException`)
* forever. The cure is to delete the file so the next pair flow re-stores
* everything against a fresh key.
*
* Best-effort: swallow exceptions from the `clear()` and
* `deleteSharedPreferences` calls themselves, since they can also throw
* when the underlying state is wedged.
*/
private fun resetPrefs() {
try {
prefs.edit().clear().apply()
} catch (_: Exception) { /* expected on a wedged file */ }
try {
context.deleteSharedPreferences(PREFS_NAME)
} catch (e: Exception) {
Log.w(TAG, "deleteSharedPreferences($PREFS_NAME) failed: ${e.message}")
}
prefs = buildPrefs()
}
companion object {
private const val TAG = "KeystoreTokenStore"
private const val PREFS_NAME = "hermes_companion_auth_hw"
/**
* Try to build a [KeystoreTokenStore] on the current device. Returns
* null when any step throws (some older OEM ROMs have broken
* AndroidKeystore implementations — we don't want the app to brick
* itself trying to create a master key).
*
* Also fires a one-shot read probe so a pre-corrupted file from a
* previous install gets healed during construction rather than on the
* first user-driven read. The probe routes through the instance's
* own [getString], so if it throws, [resetPrefs] runs and we end up
* with a fresh empty prefs file — not a permanently broken store.
*/
fun tryCreate(context: Context): KeystoreTokenStore? {
return try {
val wantsStrongBox = Build.VERSION.SDK_INT >= Build.VERSION_CODES.P &&
context.packageManager.hasSystemFeature(
android.content.pm.PackageManager.FEATURE_STRONGBOX_KEYSTORE
)
val store = KeystoreTokenStore(
context = context.applicationContext,
wantsStrongBox = wantsStrongBox,
hasHardwareBackedStorage = wantsStrongBox,
)
// Force a read so a wedged file from a prior install heals
// here rather than at the first user-visible call.
store.contains("__init_probe__")
store
} catch (e: Exception) {
// Broken Keystore, expired key, etc. — fall back to legacy.
Log.w(TAG, "KeystoreTokenStore init failed: ${e.message}")
null
}
}
}
override fun getString(key: String): String? {
return try {
prefs.getString(key, null)
} catch (e: Exception) {
Log.w(TAG, "getString($key) failed — wiping corrupted prefs: ${e.message}")
resetPrefs()
null
}
}
override fun putString(key: String, value: String) {
try {
prefs.edit().putString(key, value).apply()
} catch (e: Exception) {
Log.w(TAG, "putString($key) failed — rebuilding prefs and retrying: ${e.message}")
resetPrefs()
try {
prefs.edit().putString(key, value).apply()
} catch (e2: Exception) {
Log.w(TAG, "putString($key) retry after reset failed: ${e2.message}")
}
}
}
override fun remove(key: String) {
try {
prefs.edit().remove(key).apply()
} catch (e: Exception) {
Log.w(TAG, "remove($key) failed: ${e.message}")
resetPrefs()
}
}
override fun contains(key: String): Boolean {
return try {
prefs.contains(key)
} catch (e: Exception) {
Log.w(TAG, "contains($key) failed — wiping corrupted prefs: ${e.message}")
resetPrefs()
false
}
}
override fun clearAll() {
try {
prefs.edit().clear().apply()
} catch (e: Exception) {
Log.w(TAG, "clearAll failed — falling back to file delete: ${e.message}")
resetPrefs()
}
}
}
// ---------------------------------------------------------------------------
// Legacy fallback implementation
// ---------------------------------------------------------------------------
/**
* The original [EncryptedSharedPreferences] path — uses the same file name
* as pre-overhaul [AuthManager]. Instances of this class serve a dual role:
* (a) fallback for devices where [KeystoreTokenStore.tryCreate] fails, and
* (b) migration source for reading existing session tokens out of the legacy
* prefs on first launch after the update.
*/
class LegacyEncryptedPrefsTokenStore(context: Context) : SessionTokenStore {
companion object {
const val LEGACY_PREFS_NAME = "hermes_companion_auth"
private const val TAG = "LegacyEncryptedPrefs"
}
private val appContext: Context = context.applicationContext
// Mutable so [resetPrefs] can swap in a fresh instance after a corrupted
// file is deleted. See [KeystoreTokenStore.resetPrefs] for the rationale.
private var prefs: SharedPreferences = buildPrefs()
private fun buildPrefs(): SharedPreferences {
val masterKey = MasterKey.Builder(appContext)
.setKeyScheme(MasterKey.KeyScheme.AES256_GCM)
.build()
return EncryptedSharedPreferences.create(
appContext,
LEGACY_PREFS_NAME,
masterKey,
EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM
)
}
private fun resetPrefs() {
try {
prefs.edit().clear().apply()
} catch (_: Exception) { /* expected on a wedged file */ }
try {
appContext.deleteSharedPreferences(LEGACY_PREFS_NAME)
} catch (e: Exception) {
Log.w(TAG, "deleteSharedPreferences($LEGACY_PREFS_NAME) failed: ${e.message}")
}
prefs = buildPrefs()
}
// AES256_GCM via MasterKey is hardware-backed (TEE) on essentially every
// shipping Android device — but we don't have the StrongBox attestation
// guarantee, so we report false here. The UI uses this to decide whether
// to render the shield badge.
override val hasHardwareBackedStorage: Boolean = false
override fun getString(key: String): String? {
return try {
prefs.getString(key, null)
} catch (e: Exception) {
Log.w(TAG, "getString($key) failed — wiping legacy prefs: ${e.message}")
resetPrefs()
null
}
}
override fun putString(key: String, value: String) {
try {
prefs.edit().putString(key, value).apply()
} catch (e: Exception) {
Log.w(TAG, "putString($key) failed — rebuilding legacy prefs and retrying: ${e.message}")
resetPrefs()
try {
prefs.edit().putString(key, value).apply()
} catch (e2: Exception) {
Log.w(TAG, "putString($key) retry after reset failed: ${e2.message}")
}
}
}
override fun remove(key: String) {
try {
prefs.edit().remove(key).apply()
} catch (e: Exception) {
Log.w(TAG, "remove($key) failed: ${e.message}")
resetPrefs()
}
}
override fun contains(key: String): Boolean {
return try {
prefs.contains(key)
} catch (e: Exception) {
Log.w(TAG, "contains($key) failed — wiping legacy prefs: ${e.message}")
resetPrefs()
false
}
}
override fun clearAll() {
try {
prefs.edit().clear().apply()
} catch (e: Exception) {
Log.w(TAG, "clearAll failed — falling back to file delete: ${e.message}")
resetPrefs()
}
}
}
@@ -0,0 +1,129 @@
package com.hermesandroid.relay.bridge
import android.Manifest
import android.annotation.SuppressLint
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
import android.os.Build
import android.util.Log
import androidx.core.app.NotificationCompat
import androidx.core.app.NotificationManagerCompat
import androidx.core.content.ContextCompat
import com.hermesandroid.relay.MainActivity
import com.hermesandroid.relay.R
import com.hermesandroid.relay.accessibility.HermesAccessibilityService
/**
* Phase 3 — safety-rails `bridge-safety-rails`
*
* Canonical "turn the bridge off after idle" unit of work. Not a real
* `androidx.work.CoroutineWorker` — the project intentionally does not
* depend on androidx.work — but its shape mirrors one exactly: a single
* suspend [run] method that performs the work and returns.
*
* Why this pattern instead of dropping a WorkManager dep:
* - Auto-disable is a pure in-memory decision: the toggle lives in our
* own DataStore, no inter-process scheduling is required.
* - Android's AlarmManager / WorkManager are needed when the work must
* survive process death. For bridge, process death already implies
* the service is disconnected and the master toggle re-evaluates
* fresh on the next launch. So a coroutine-owned `delay` does it.
* - Every command reschedules the timer, so the idle window is always
* reset against wall clock. No drift concerns.
*
* When WorkManager is added later (say, if notif-listener needs background-posted
* notifications on a schedule), this file is a natural upgrade point:
* change the class to `CoroutineWorker(appContext, params)` and have
* [BridgeSafetyManager.rescheduleAutoDisable] enqueue a [OneTimeWorkRequest]
* instead of launching a local coroutine.
*/
class AutoDisableWorker(private val context: Context) {
companion object {
private const val TAG = "AutoDisableWorker"
private const val CHANNEL_ID = "bridge_auto_disable"
private const val CHANNEL_NAME = "Bridge auto-disable"
private const val NOTIFICATION_ID = 3821
}
/**
* Execute the auto-disable: flip the master toggle off and post a
* one-shot "bridge paused" notification. Idempotent — safe to call
* twice (the second call just re-writes the same DataStore value
* and overrides the existing notification).
*/
suspend fun run() {
try {
HermesAccessibilityService.setMasterEnabled(context, false)
} catch (t: Throwable) {
Log.w(TAG, "run: failed to flip master toggle", t)
}
postNotification()
}
// Lint can't trace through [hasPostNotificationsPermission] to see that
// we early-return when the runtime grant isn't held, and the notify()
// call is also wrapped in runCatching to swallow SecurityException as
// a belt-and-braces. Suppress here rather than inlining the check —
// the helper exists so the same gate can grow more conditions later
// without each call site re-implementing it.
@SuppressLint("MissingPermission")
private fun postNotification() {
ensureChannel()
if (!hasPostNotificationsPermission()) {
Log.i(TAG, "POST_NOTIFICATIONS not granted — skipping auto-disable notification")
return
}
val tapIntent = Intent(context, MainActivity::class.java).apply {
flags = Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_CLEAR_TOP
}
val pendingFlags = PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE
val tapPending = PendingIntent.getActivity(context, 0, tapIntent, pendingFlags)
val builder = NotificationCompat.Builder(context, CHANNEL_ID)
.setSmallIcon(R.mipmap.ic_launcher)
.setContentTitle("Bridge auto-disabled")
.setContentText("Paused after idle — tap to re-enable in the Bridge tab.")
.setStyle(NotificationCompat.BigTextStyle().bigText(
"Hermes bridge was idle for too long, so device control has been turned off " +
"automatically. Open the Bridge tab to turn it back on if you still need it."
))
.setContentIntent(tapPending)
.setAutoCancel(true)
.setOnlyAlertOnce(true)
.setPriority(NotificationCompat.PRIORITY_DEFAULT)
runCatching {
NotificationManagerCompat.from(context).notify(NOTIFICATION_ID, builder.build())
}.onFailure { Log.w(TAG, "postNotification: notify failed", it) }
}
private fun ensureChannel() {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.O) return
val nm = context.getSystemService(NotificationManager::class.java) ?: return
val existing = nm.getNotificationChannel(CHANNEL_ID)
if (existing != null) return
val channel = NotificationChannel(
CHANNEL_ID,
CHANNEL_NAME,
NotificationManager.IMPORTANCE_DEFAULT,
).apply {
description = "Fires once when the bridge auto-disables after being idle."
setShowBadge(false)
}
nm.createNotificationChannel(channel)
}
private fun hasPostNotificationsPermission(): Boolean {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.TIRAMISU) return true
return ContextCompat.checkSelfPermission(
context,
Manifest.permission.POST_NOTIFICATIONS
) == PackageManager.PERMISSION_GRANTED
}
}
@@ -0,0 +1,396 @@
package com.hermesandroid.relay.bridge
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.app.Service
import android.content.Context
import android.content.Intent
import android.content.pm.ServiceInfo
import android.os.Build
import android.os.IBinder
import android.util.Log
import androidx.core.app.NotificationCompat
import com.hermesandroid.relay.MainActivity
import com.hermesandroid.relay.R
import com.hermesandroid.relay.accessibility.HermesAccessibilityService
import com.hermesandroid.relay.accessibility.MediaProjectionHolder
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.launch
/**
* Phase 3 — safety-rails `bridge-safety-rails`
*
* Persistent foreground service that signals "Hermes agent has device
* control" to the user whenever the bridge master toggle is on. The
* notification is:
*
* - Ongoing (can't be swiped away)
* - Two actions: "Disable" (broadcast into the master toggle) and
* "Settings" (deep-link into BridgeSafetySettingsScreen)
* - Channel `bridge_foreground` at DEFAULT importance (we do NOT want
* heads-up because that would pop over every screen the agent taps)
*
* # Foreground service type
*
* On Android 10+ foreground services must declare a `foregroundServiceType`.
* For Tier 5 we use `specialUse` with the justification string
* "Persistent indicator that the Hermes agent has device control, per
* Tier 5 safety rails." Play Store review allowance is declared in the
* manifest `<property name="...SPECIAL_USE"/>`.
*
* We deliberately do NOT use `FOREGROUND_SERVICE_MEDIA_PROJECTION` here
* even though accessibility's `ScreenCapture.kt` uses MediaProjection — that
* permission is already declared, and binding the foreground service to
* mediaProjection would couple its lifecycle to the current screen-grant,
* which doesn't match our "always on while bridge is active" semantics.
*
* # Lifecycle wiring
*
* [BridgeViewModel] observes the master-toggle StateFlow and calls
* [start] / [stop] on transitions. The service itself does not observe
* the toggle — it's a passive "I'm on" indicator, and bundling the flow
* subscription into the service would mean the service owns its own
* coroutine scope + we'd need to worry about bind/unbind timing. Simpler
* to have the ViewModel drive it.
*/
class BridgeForegroundService : Service() {
companion object {
private const val TAG = "BridgeForegroundSvc"
const val CHANNEL_ID = "bridge_foreground"
private const val CHANNEL_NAME = "Bridge active"
const val NOTIFICATION_ID = 4712
const val ACTION_START = "com.hermesandroid.relay.bridge.START"
const val ACTION_STOP = "com.hermesandroid.relay.bridge.STOP"
const val ACTION_DISABLE = "com.hermesandroid.relay.bridge.DISABLE"
const val ACTION_OPEN_SETTINGS = "com.hermesandroid.relay.bridge.OPEN_SETTINGS"
// === PHASE3-bridge-ui-followup: MediaProjection upgrade action ===
// Fired by MainActivity.mediaProjectionLauncher after the user has
// granted the system consent dialog. Carries the resultCode + data
// Intent the launcher received. The service handles this by:
// 1. Calling startForeground AGAIN with the dual SPECIAL_USE |
// MEDIA_PROJECTION type bitmask (legal NOW because consent
// has been granted).
// 2. Calling MediaProjectionHolder.acceptGrantInsideForegroundService
// to construct and store the projection.
// Splitting the FGS type slot out of the initial start-up is
// critical: Android 14+ silently auto-revokes any projection created
// by a service that called startForeground(type=mediaProjection)
// BEFORE the consent dialog was granted. The previous code had
// mediaProjection in the initial type bitmask the moment the master
// toggle flipped on, which is exactly that violation. Symptom:
// "I tap Allow but the row never turns green" — Bailey, 2026-04-13.
const val ACTION_GRANT_PROJECTION = "com.hermesandroid.relay.bridge.GRANT_PROJECTION"
const val EXTRA_RESULT_CODE = "com.hermesandroid.relay.bridge.RESULT_CODE"
const val EXTRA_RESULT_DATA = "com.hermesandroid.relay.bridge.RESULT_DATA"
// === END PHASE3-bridge-ui-followup ===
fun start(context: Context) {
val intent = Intent(context.applicationContext, BridgeForegroundService::class.java)
.setAction(ACTION_START)
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
context.applicationContext.startForegroundService(intent)
} else {
context.applicationContext.startService(intent)
}
}
fun stop(context: Context) {
// Use stopService() rather than startService(ACTION_STOP) — on
// Android 15+ (target SDK 35), the system routes ANY intent to
// a service with foregroundServiceType through the foreground
// watchdog and demands a startForeground call within 5s, even
// if the intent is an internal "please shut down" message. By
// going through stopService() we bypass onStartCommand entirely
// and call onDestroy directly — clean shutdown, no watchdog.
val intent = Intent(context.applicationContext, BridgeForegroundService::class.java)
context.applicationContext.stopService(intent)
}
/**
* Fire-and-forget: hand the consent result off to the foreground
* service so it can upgrade its FGS type to include MEDIA_PROJECTION
* and construct the projection. Called from
* `MainActivity.mediaProjectionLauncher` immediately after the user
* grants the system consent dialog.
*
* The service must already be running (master toggle on) — that is
* guaranteed by `BridgeViewModel.requestScreenCapture()` which gates
* the consent flow on the master toggle being on.
*/
fun grantMediaProjection(context: Context, resultCode: Int, data: Intent) {
val intent = Intent(context.applicationContext, BridgeForegroundService::class.java)
.setAction(ACTION_GRANT_PROJECTION)
.putExtra(EXTRA_RESULT_CODE, resultCode)
.putExtra(EXTRA_RESULT_DATA, data)
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
context.applicationContext.startForegroundService(intent)
} else {
context.applicationContext.startService(intent)
}
}
}
// True after we've successfully called startForeground with the
// mediaProjection type slot (i.e. after consent + grant). Drives the
// type bitmask passed to subsequent startForeground calls so we don't
// accidentally drop the slot on a re-start.
private var hasMediaProjectionType: Boolean = false
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
override fun onBind(intent: Intent?): IBinder? = null
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
Log.i(TAG, "onStartCommand action=${intent?.action} flags=$flags startId=$startId")
// === PHASE3-bridge-ui-followup: always startForeground first ===
// CRITICAL: on Android 15+ (target SDK 35), ANY intent delivered to
// a service that declares foregroundServiceType in the manifest —
// including intents dispatched via Context.startService() — gets
// tracked by the system's foreground-service watchdog. The service
// has 5 seconds to call startForeground() or the system throws
// ForegroundServiceDidNotStartInTimeException and crashes the app.
//
// Symptom we hit on 2026-04-13: opening the Bridge tab → BridgeViewModel
// collector fires masterToggle (initial value false) → calls
// BridgeForegroundService.stop(ctx) → startService(ACTION_STOP) →
// service onStartCommand handles ACTION_STOP, calls stopForeground
// and stopSelf without ever calling startForeground → 5s later the
// system kills the process.
//
// The fix: ALWAYS call startForeground at the top of onStartCommand,
// before any action branching. The brief notification flash for
// stop-only paths is acceptable; the alternative (using a bound
// service or broadcast receiver for control commands) is a much
// bigger refactor for the same outcome.
startForegroundNotification()
// === END PHASE3-bridge-ui-followup ===
when (intent?.action) {
ACTION_STOP -> {
Log.i(TAG, "ACTION_STOP → stopping foreground service")
hasMediaProjectionType = false
stopForeground(STOP_FOREGROUND_REMOVE)
stopSelf()
return START_NOT_STICKY
}
ACTION_DISABLE -> {
Log.i(TAG, "ACTION_DISABLE → flipping master toggle off")
scope.launch {
runCatching {
HermesAccessibilityService.setMasterEnabled(applicationContext, false)
}.onFailure { Log.w(TAG, "master toggle write failed", it) }
}
// Keep the service alive until BridgeViewModel sees the
// toggle flip and calls stop(); that's the canonical path.
return START_STICKY
}
ACTION_OPEN_SETTINGS -> {
Log.i(TAG, "ACTION_OPEN_SETTINGS → launching MainActivity with deep-link to bridge safety")
// PHASE3-safety-rails-followup: deep-link to BridgeSafetySettingsScreen.
// MainActivity reads EXTRA_NAV_ROUTE in onCreate / onNewIntent
// and emits it on the NavRouteRequest SharedFlow, which RelayApp
// collects and forwards to the NavController. The route string
// is hardcoded here on purpose to avoid pulling the entire
// ui.RelayApp graph into the bridge service classpath — if you
// change Screen.BridgeSafetySettings.route in RelayApp.kt,
// change it here too.
val launch = Intent(this, MainActivity::class.java).apply {
addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_CLEAR_TOP)
putExtra(MainActivity.EXTRA_NAV_ROUTE, "settings/bridge_safety")
}
runCatching { startActivity(launch) }
return START_STICKY
}
ACTION_GRANT_PROJECTION -> {
// === PHASE3-bridge-ui-followup: post-consent FGS type upgrade ===
// The launcher result has just landed in MainActivity. The
// top-of-onStartCommand call already brought us into the
// foreground (with SPECIAL_USE only). Now flip the type
// flag and call startForeground AGAIN to upgrade to
// SPECIAL_USE | MEDIA_PROJECTION — legal NOW because the
// consent has been granted — then construct the projection
// from inside the foreground state. Two startForeground
// calls on the same service is well-supported; the second
// just changes the type bitmask.
Log.i(TAG, "ACTION_GRANT_PROJECTION → upgrading FGS type and accepting grant")
hasMediaProjectionType = true
startForegroundNotification()
val resultCode = intent.getIntExtra(EXTRA_RESULT_CODE, 0)
val data: Intent? = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) {
intent.getParcelableExtra(EXTRA_RESULT_DATA, Intent::class.java)
} else {
@Suppress("DEPRECATION")
intent.getParcelableExtra(EXTRA_RESULT_DATA)
}
val accepted = MediaProjectionHolder.acceptGrantInsideForegroundService(
this, resultCode, data
)
if (!accepted) {
// Rolling back the type slot keeps us honest: if the
// user actually denied (or the API failed), we shouldn't
// claim a grant we don't have. Re-run startForeground
// with SPECIAL_USE only so the FGS type matches reality.
Log.w(TAG, "grant not accepted — reverting FGS to SPECIAL_USE only")
hasMediaProjectionType = false
startForegroundNotification()
}
return START_STICKY
// === END PHASE3-bridge-ui-followup ===
}
}
// Fall-through for ACTION_START / null intent — startForegroundNotification
// was already called at the top of this method.
return START_STICKY
}
/**
* Foreground services legitimately survive task removal — the service
* instance stays alive even after the user swipes the app from recents,
* which is the behavior we want for the bridge itself (agent-driven
* phone control via WSS should keep working when the user "closes" the
* app). BUT the MediaProjection that powers screen capture should NOT
* outlive task removal: the system-level screen-cast indicator icon in
* Android's status bar stays visible for the duration of the projection,
* and a user who swiped the app away would understandably read that
* icon as "the app I just closed is still recording my screen." That's
* the exact class of trust failure we can't afford for an
* accessibility-controlling app.
*
* Fix: on task removal, revoke only the projection (the bridge stays
* alive) and downgrade the FGS type back to SPECIAL_USE-only so
* startForeground reflects reality and the MEDIA_PROJECTION system
* indicator disappears. Next time the user reopens the app and
* re-enables screenshots, a fresh consent dialog appears — which is
* the correct UX for a privacy-sensitive capability.
*
* Caught 2026-04-15 by Bailey: the screen-cast icon persisted in the
* status bar even after force-stopping the app from recents.
*/
override fun onTaskRemoved(rootIntent: Intent?) {
super.onTaskRemoved(rootIntent)
Log.i(
TAG,
"onTaskRemoved: user swiped app — revoking MediaProjection, " +
"keeping bridge alive for agent control",
)
runCatching { MediaProjectionHolder.revoke() }
if (hasMediaProjectionType) {
hasMediaProjectionType = false
runCatching { startForegroundNotification() }
}
}
override fun onDestroy() {
// Reset state so a fresh service instance starts in the
// SPECIAL_USE-only configuration. Also drop any held MediaProjection
// — a projection without an active bridge is meaningless and the
// next bridge enable should always prompt for fresh consent.
hasMediaProjectionType = false
runCatching { MediaProjectionHolder.revoke() }
scope.cancel()
super.onDestroy()
}
private fun startForegroundNotification() {
ensureChannel()
val notification = buildNotification()
try {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
// === PHASE3-bridge-ui-followup: gated MediaProjection type slot ===
// CRITICAL: on Android 14+, startForeground(type=mediaProjection)
// is only legal AFTER the user has granted the system consent
// dialog. Calling it before — even if you intend to "wait
// until consent arrives" — makes the eventual projection
// get auto-revoked by the system within a frame, with no
// app-visible error.
//
// So we start with SPECIAL_USE only when the bridge first
// comes up (master toggle on, no projection yet), and the
// ACTION_GRANT_PROJECTION handler upgrades us to
// SPECIAL_USE | MEDIA_PROJECTION right after consent and
// before getMediaProjection. That's why this method reads
// [hasMediaProjectionType] instead of always OR-ing both.
//
// Both subtypes share this single notification + this single
// service. Manifest lists both in `foregroundServiceType`.
val typeMask = if (hasMediaProjectionType) {
ServiceInfo.FOREGROUND_SERVICE_TYPE_SPECIAL_USE or
ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION
} else {
ServiceInfo.FOREGROUND_SERVICE_TYPE_SPECIAL_USE
}
startForeground(NOTIFICATION_ID, notification, typeMask)
// === END PHASE3-bridge-ui-followup ===
} else if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
// Q..U: the type arg is required on Q+ too, but
// FOREGROUND_SERVICE_TYPE_SPECIAL_USE is Android 14+.
// Fall through to the plain startForeground on Q..T —
// AGP will attach the manifest-declared type automatically.
startForeground(NOTIFICATION_ID, notification)
} else {
startForeground(NOTIFICATION_ID, notification)
}
} catch (t: Throwable) {
Log.w(TAG, "startForeground failed — bridge indicator will not be visible", t)
stopSelf()
}
}
private fun buildNotification(): android.app.Notification {
val tapIntent = Intent(this, MainActivity::class.java).apply {
flags = Intent.FLAG_ACTIVITY_CLEAR_TOP or Intent.FLAG_ACTIVITY_SINGLE_TOP
}
val pendingFlags = PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE
val tapPending = PendingIntent.getActivity(this, 0, tapIntent, pendingFlags)
val disableIntent = Intent(this, BridgeForegroundService::class.java)
.setAction(ACTION_DISABLE)
val disablePending = PendingIntent.getService(this, 1, disableIntent, pendingFlags)
val settingsIntent = Intent(this, BridgeForegroundService::class.java)
.setAction(ACTION_OPEN_SETTINGS)
val settingsPending = PendingIntent.getService(this, 2, settingsIntent, pendingFlags)
return NotificationCompat.Builder(this, CHANNEL_ID)
.setSmallIcon(R.mipmap.ic_launcher)
.setContentTitle("Hermes agent has device control")
.setContentText("Bridge is active — tap Disable to stop at any time.")
.setStyle(NotificationCompat.BigTextStyle().bigText(
"The Hermes agent can currently read the screen and perform " +
"actions on your behalf through the accessibility service. " +
"Tap Disable to turn this off immediately."
))
.setContentIntent(tapPending)
.setOngoing(true)
.setOnlyAlertOnce(true)
.setPriority(NotificationCompat.PRIORITY_DEFAULT)
.setCategory(NotificationCompat.CATEGORY_SERVICE)
.addAction(0, "Disable", disablePending)
.addAction(0, "Settings", settingsPending)
.build()
}
private fun ensureChannel() {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.O) return
val nm = getSystemService(NotificationManager::class.java) ?: return
if (nm.getNotificationChannel(CHANNEL_ID) != null) return
val channel = NotificationChannel(
CHANNEL_ID,
CHANNEL_NAME,
NotificationManager.IMPORTANCE_DEFAULT,
).apply {
description = "Persistent indicator while the Hermes agent has device control."
setShowBadge(false)
}
nm.createNotificationChannel(channel)
}
}
@@ -0,0 +1,387 @@
package com.hermesandroid.relay.bridge
import android.content.Context
import android.util.Log
import com.hermesandroid.relay.data.BridgeSafetyPreferencesRepository
import com.hermesandroid.relay.data.BridgeSafetySettings
import kotlinx.coroutines.CompletableDeferred
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.TimeoutCancellationException
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.launch
import kotlinx.coroutines.plus
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.atomic.AtomicLong
/**
* Phase 3 — safety-rails `bridge-safety-rails`
*
* Central enforcement point for Tier 5 safety: per-app blocklist, destructive
* verb confirmation, and idle-based auto-disable. Owned as a singleton-per-
* process by [ConnectionViewModel] and injected into [BridgeCommandHandler].
*
* # Integration surface
*
* - [checkPackageAllowed] — called with the currently foregrounded package
* (from `HermesAccessibilityService.currentApp`). Returns false if the
* user has that package on the blocklist. Caller maps false → HTTP 403.
*
* - [requiresConfirmation] — cheap synchronous predicate: does the agent's
* requested action (`/tap_text` / `/type`) carry one of the user's
* destructive verbs in its body? Returns false for every other path.
*
* - [awaitConfirmation] — suspend function that shows a system-overlay
* modal and waits for the user to tap Allow / Deny. Returns true on
* allow, false on deny or on [BridgeSafetySettings.confirmationTimeoutSeconds]
* timeout. The suspending `BridgeCommandHandler` action coroutine is
* what's held here — the phone's agent call stays open until the user
* reacts, which is exactly the UX we want (the server sees a slow
* response, not a denial race).
*
* - [rescheduleAutoDisable] — every accepted command bumps the idle timer
* forward; after [BridgeSafetySettings.autoDisableMinutes] of silence
* the master toggle flips off and a one-shot notification fires.
* [cancelAutoDisable] cancels the pending timer (called when the master
* toggle flips off manually, so we don't race the timer against the
* user).
*
* # Overlay <-> coroutine wiring
*
* The confirmation modal lives in a [SYSTEM_ALERT_WINDOW]-backed overlay
* (see [ConfirmationOverlayHost]). The `awaitConfirmation` coroutine
* registers a pending entry keyed by a monotonic request id, shows the
* overlay, and then waits on the entry's `CompletableDeferred<Boolean>`
* under a `withTimeout`. The overlay's Allow / Deny buttons complete the
* deferred. On timeout we dismiss the overlay from the manager side and
* return false ("treat silence as deny"). If the overlay host is not
* available — e.g. the user hasn't granted `SYSTEM_ALERT_WINDOW` yet — we
* fail-closed and return false so no destructive action slips through.
*
* # No WorkManager
*
* The Android app does not depend on androidx.work. [AutoDisableWorker]
* documents the canonical pattern, but the live path is a coroutine
* `Job` owned by this manager, delayed by the configured minutes. This is
* acceptable because we are the in-memory owner of the master-toggle flow
* — no inter-process or cross-restart scheduling is needed. On process
* death the master toggle is simply evaluated fresh from DataStore, and
* any command not explicitly sent within the idle window never actually
* happens because the app isn't running.
*/
class BridgeSafetyManager(
context: Context,
private val scope: CoroutineScope,
) {
companion object {
private const val TAG = "BridgeSafetyMgr"
@Volatile
private var INSTANCE: BridgeSafetyManager? = null
/**
* Process-wide accessor. Both [BridgeCommandHandler] and the overlay
* host need to reach the same manager instance without a DI graph;
* [ConnectionViewModel] calls [install] once at init time.
*/
fun peek(): BridgeSafetyManager? = INSTANCE
fun install(context: Context, scope: CoroutineScope): BridgeSafetyManager {
val existing = INSTANCE
if (existing != null) return existing
val created = BridgeSafetyManager(context.applicationContext, scope)
INSTANCE = created
return created
}
}
private val appContext: Context = context.applicationContext
private val prefsRepo = BridgeSafetyPreferencesRepository(appContext)
/** Latest settings snapshot — UI + checks read this via [settings]. */
private val _settings = MutableStateFlow(BridgeSafetySettings())
val settings: StateFlow<BridgeSafetySettings> = _settings.asStateFlow()
/** True once the DataStore collector has ticked at least once. */
@Volatile
private var settingsHydrated: Boolean = false
/**
* Pending confirmation requests keyed by a monotonic id. The overlay's
* Allow / Deny callbacks complete the deferred by looking up the id the
* overlay was opened for. [ConcurrentHashMap] because the completion
* happens on the Compose overlay thread and registration on the bridge
* scope's dispatcher.
*/
private val pendingConfirmations = ConcurrentHashMap<Long, PendingConfirmation>()
private val nextRequestId = AtomicLong(0L)
/** Coroutine job that fires auto-disable after idle. */
@Volatile
private var autoDisableJob: Job? = null
/**
* Remaining time (epoch millis) for the current auto-disable job, or
* null when idle. BridgeSafetySummaryCard reads this as a countdown.
*/
private val _autoDisableAtMs = MutableStateFlow<Long?>(null)
val autoDisableAtMs: StateFlow<Long?> = _autoDisableAtMs.asStateFlow()
init {
// Eagerly observe DataStore — writes from the settings screen flow
// into the cache so checkPackageAllowed / requiresConfirmation can
// read synchronously without a suspend hop.
scope.launch {
prefsRepo.settings.collect { latest ->
_settings.value = latest
settingsHydrated = true
}
}
}
// ── Blocklist ────────────────────────────────────────────────────────
/**
* Returns false iff [packageName] is explicitly blocklisted. A null
* or blank package (accessibility service hasn't seen a window yet)
* is treated as allowed — we can't block what we don't know.
*/
suspend fun checkPackageAllowed(packageName: String?): Boolean {
if (packageName.isNullOrBlank()) return true
val snapshot = currentSettings()
return packageName !in snapshot.blocklist
}
// ── Destructive verbs ────────────────────────────────────────────────
suspend fun requiresConfirmation(method: String, text: String?): Boolean {
if (text.isNullOrBlank()) return false
// Pre-0.4.0 only /tap_text and /type were gated — node-id taps
// slipped past because the gate never looked up the tapped
// node's text. Bailey 2026-04-15 hit this: after denying an
// SMS, the agent fell back to android_open_app + android_tap
// (by nodeId) on the Messages app's "Send" button, bypassing
// the verb modal. BridgeCommandHandler.extractDestructiveVerbText
// now resolves the node's text for /tap + /long_press too, and
// this method gates on any of the four paths as long as the
// caller supplies a text argument.
if (method != "/tap_text" &&
method != "/type" &&
method != "/tap" &&
method != "/long_press"
) return false
val verbs = currentSettings().destructiveVerbs
if (verbs.isEmpty()) return false
return containsDestructiveVerb(text, verbs)
}
/**
* Show the confirmation overlay and suspend until the user reacts or
* the timeout elapses. Returns true to allow, false to deny.
*
* Fail-closed: if the SYSTEM_ALERT_WINDOW permission hasn't been
* granted, or if the overlay host can't be reached for any reason, we
* return false and log a warning. Better a missed agent command than
* a silently-allowed destructive action.
*/
suspend fun awaitConfirmation(method: String, text: String?): Boolean {
val snapshot = currentSettings()
val timeoutMs = snapshot.confirmationTimeoutSeconds * 1000L
val requestId = nextRequestId.incrementAndGet()
val deferred = CompletableDeferred<Boolean>()
val pending = PendingConfirmation(
id = requestId,
method = method,
text = text.orEmpty(),
verb = text?.let { firstMatchedVerb(it, snapshot.destructiveVerbs) }.orEmpty(),
deferred = deferred,
)
pendingConfirmations[requestId] = pending
val host = ConfirmationOverlayHost.instance
if (host == null) {
Log.w(TAG, "awaitConfirmation: no overlay host installed — failing closed (deny)")
pendingConfirmations.remove(requestId)
return false
}
// ComposeView creation + setContent inside BridgeStatusOverlay.showConfirmation
// must run on the Main thread. This awaitConfirmation call can originate
// from either the WSS-incoming BridgeCommandHandler path (already on Main
// via ChannelMultiplexer's dispatcher) OR from the in-process voice-intent
// local-dispatch path (Dispatchers.Default, via RealVoiceBridgeIntentHandler's
// own scope). Using Dispatchers.Main.immediate makes the first case a no-op
// and only schedules on Main for the second — one line handles both callers.
//
// Before this fix the off-thread call threw from ComposeView.setContent,
// the outer runCatching swallowed it, and the log message mislabelled the
// cause as "likely overlay permission missing" which sent debugging up a
// wrong tree (2026-04-15 on-device test with overlay permission granted
// but voice SMS still never showed the modal).
val shown = runCatching {
withContext(Dispatchers.Main.immediate) {
host.showConfirmation(pending) { resolution ->
resolveConfirmation(requestId, resolution)
}
}
}
if (shown.isFailure) {
Log.w(TAG, "awaitConfirmation: host.showConfirmation threw — denying", shown.exceptionOrNull())
pendingConfirmations.remove(requestId)
return false
}
return try {
withTimeout(timeoutMs) { deferred.await() }
} catch (_: TimeoutCancellationException) {
Log.i(TAG, "awaitConfirmation: timed out after ${timeoutMs}ms — denying")
host.dismissConfirmation(requestId)
pendingConfirmations.remove(requestId)
false
} catch (t: Throwable) {
Log.w(TAG, "awaitConfirmation: unexpected failure — denying", t)
host.dismissConfirmation(requestId)
pendingConfirmations.remove(requestId)
false
}
}
/**
* Called from the overlay UI when the user taps Allow / Deny (or when
* the overlay is dismissed programmatically). Safe to call for an
* unknown id (we just drop on the floor).
*/
fun resolveConfirmation(requestId: Long, allowed: Boolean) {
val pending = pendingConfirmations.remove(requestId) ?: return
pending.deferred.complete(allowed)
}
// ── Auto-disable timer ───────────────────────────────────────────────
/**
* Cancel any pending timer and arm a fresh one. Called on every accepted
* bridge command — an actively-used bridge never auto-disables.
*/
fun rescheduleAutoDisable() {
val minutes = _settings.value.autoDisableMinutes
val delayMs = minutes * 60_000L
val fireAt = System.currentTimeMillis() + delayMs
autoDisableJob?.cancel()
_autoDisableAtMs.value = fireAt
autoDisableJob = (scope + SupervisorJob()).launch {
try {
delay(delayMs)
Log.i(TAG, "Auto-disable fired after $minutes min of idle")
// Hand off to the canonical worker so both code paths look
// identical from a behavioral standpoint (notification +
// master-toggle flip).
AutoDisableWorker(appContext).run()
} catch (_: Throwable) {
// Cancellation is expected on reschedule — swallow quietly.
} finally {
if (_autoDisableAtMs.value == fireAt) _autoDisableAtMs.value = null
}
}
}
fun cancelAutoDisable() {
autoDisableJob?.cancel()
autoDisableJob = null
_autoDisableAtMs.value = null
}
// ── Internals ────────────────────────────────────────────────────────
private suspend fun currentSettings(): BridgeSafetySettings {
// Prefer the cached value once the DataStore collector has ticked
// at least once. Before that, fall back to a one-shot read of
// DataStore so the very first command after install() doesn't
// race the collector and see stale defaults.
if (settingsHydrated) return _settings.value
return try {
val first = prefsRepo.settings.first()
_settings.value = first
settingsHydrated = true
first
} catch (t: Throwable) {
Log.w(TAG, "currentSettings: DataStore read failed — using defaults", t)
BridgeSafetySettings()
}
}
private fun containsDestructiveVerb(text: String, verbs: Set<String>): Boolean {
if (text.isBlank()) return false
val lower = text.lowercase()
for (verb in verbs) {
val v = verb.lowercase()
if (v.isBlank()) continue
// \b<verb>\b — word-boundary match, so "send" matches "send it"
// but not "sender" or "sendmail". Kotlin's Regex `\b` uses the
// JVM Pattern engine; we pre-escape the verb in case a user
// added something like `pay.` via the settings screen.
val pattern = Regex("\\b${Regex.escape(v)}\\b", RegexOption.IGNORE_CASE)
if (pattern.containsMatchIn(lower)) return true
}
return false
}
private fun firstMatchedVerb(text: String, verbs: Set<String>): String? {
if (text.isBlank()) return null
val lower = text.lowercase()
return verbs.firstOrNull { v ->
val vl = v.lowercase()
vl.isNotBlank() && Regex("\\b${Regex.escape(vl)}\\b", RegexOption.IGNORE_CASE)
.containsMatchIn(lower)
}
}
}
/**
* One in-flight confirmation modal. Held in [BridgeSafetyManager.pendingConfirmations]
* and surfaced to the overlay host so the Compose dialog can render the
* full context.
*/
data class PendingConfirmation(
val id: Long,
val method: String,
val text: String,
val verb: String,
val deferred: CompletableDeferred<Boolean>,
)
/**
* Abstraction the safety manager talks to when it needs a modal on screen.
* The live implementation lives in [BridgeStatusOverlay] (it's the same
* [WindowManager] pipeline that hosts the ambient status dot, so we only
* hold one SYSTEM_ALERT_WINDOW attachment per process).
*/
interface ConfirmationOverlayHost {
/**
* Render the confirmation modal. Must be idempotent per [request.id] —
* calling twice with the same id is a no-op. [onResult] is invoked once
* when the user reacts (or when the modal is dismissed externally, in
* which case pass `false`).
*/
fun showConfirmation(
request: PendingConfirmation,
onResult: (allowed: Boolean) -> Unit,
)
/** Dismiss a modal without resolving the deferred (the manager side does that). */
fun dismissConfirmation(requestId: Long)
companion object {
@Volatile
var instance: ConfirmationOverlayHost? = null
}
}
@@ -0,0 +1,294 @@
package com.hermesandroid.relay.bridge
import android.annotation.SuppressLint
import android.content.Context
import android.graphics.PixelFormat
import android.os.Build
import android.provider.Settings
import android.util.Log
import android.view.Gravity
import android.view.View
import android.view.WindowManager
import androidx.compose.material3.MaterialTheme
import androidx.compose.ui.platform.ComposeView
import androidx.compose.ui.platform.ViewCompositionStrategy
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleOwner
import androidx.lifecycle.LifecycleRegistry
import androidx.lifecycle.ViewModelStore
import androidx.lifecycle.ViewModelStoreOwner
import androidx.lifecycle.setViewTreeLifecycleOwner
import androidx.lifecycle.setViewTreeViewModelStoreOwner
import androidx.savedstate.SavedStateRegistry
import androidx.savedstate.SavedStateRegistryController
import androidx.savedstate.SavedStateRegistryOwner
import androidx.savedstate.setViewTreeSavedStateRegistryOwner
import com.hermesandroid.relay.ui.components.BridgeStatusOverlayChip
import com.hermesandroid.relay.ui.components.DestructiveVerbConfirmDialog
import com.hermesandroid.relay.util.ComposeArrWorkaround
import java.util.concurrent.ConcurrentHashMap
/**
* Phase 3 — safety-rails `bridge-safety-rails`
*
* WindowManager-backed overlay host. Serves two jobs in one place so we
* only ever attach a single `SYSTEM_ALERT_WINDOW` View per process:
*
* 1. A small floating status chip ("Hermes is active") shown when the
* user has opted in via [BridgeSafetySettings.statusOverlayEnabled].
* 2. A full-width centered destructive-verb confirmation modal that
* [BridgeSafetyManager] fires from [BridgeSafetyManager.awaitConfirmation].
*
* The two overlays are independent [ComposeView] attachments — they
* don't share layout params. That keeps the chip a tiny permanent hole
* in the gesture layer while the modal is a modal rectangle that
* intercepts touches only when present.
*
* # Lifecycle plumbing for ComposeView
*
* `ComposeView` attached via `WindowManager` does not automatically get
* a `ViewTreeLifecycleOwner` (normally the containing Activity provides
* one). Compose requires one to run its recomposer, so we attach a
* minimal [OverlayLifecycleOwner] that's always in the RESUMED state
* while the view is attached. Same deal for SavedStateRegistryOwner
* (required by SavedStateHandle inside composables) and ViewModelStoreOwner.
*/
class BridgeStatusOverlay(context: Context) : ConfirmationOverlayHost {
companion object {
private const val TAG = "BridgeStatusOverlay"
@Volatile
private var INSTANCE: BridgeStatusOverlay? = null
fun install(context: Context): BridgeStatusOverlay {
val existing = INSTANCE
if (existing != null) return existing
val created = BridgeStatusOverlay(context.applicationContext)
INSTANCE = created
ConfirmationOverlayHost.instance = created
return created
}
fun peek(): BridgeStatusOverlay? = INSTANCE
}
private val appContext: Context = context.applicationContext
private val wm: WindowManager =
appContext.getSystemService(Context.WINDOW_SERVICE) as WindowManager
private var chipView: View? = null
private val activeConfirmations = ConcurrentHashMap<Long, View>()
// ── Status chip ──────────────────────────────────────────────────────
/**
* Show or hide the floating status chip. No-op if the overlay
* permission hasn't been granted — [BridgeSafetySettingsScreen] is
* responsible for walking the user through the grant flow.
*/
@SuppressLint("InflateParams")
fun setChipVisible(visible: Boolean) {
if (!visible) {
chipView?.let {
runCatching { wm.removeView(it) }
.onFailure { Log.w(TAG, "removeView(chip) failed", it) }
}
chipView = null
return
}
if (chipView != null) return // already showing
if (!Settings.canDrawOverlays(appContext)) {
Log.w(TAG, "setChipVisible: SYSTEM_ALERT_WINDOW not granted — skipping chip")
return
}
val compose = ComposeView(appContext).apply {
setContent {
MaterialTheme { BridgeStatusOverlayChip() }
}
}
attachLifecycle(compose)
val params = WindowManager.LayoutParams(
WindowManager.LayoutParams.WRAP_CONTENT,
WindowManager.LayoutParams.WRAP_CONTENT,
overlayType(),
WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE or
WindowManager.LayoutParams.FLAG_NOT_TOUCH_MODAL or
WindowManager.LayoutParams.FLAG_LAYOUT_IN_SCREEN or
WindowManager.LayoutParams.FLAG_LAYOUT_NO_LIMITS,
PixelFormat.TRANSLUCENT,
).apply {
gravity = Gravity.TOP or Gravity.END
x = 24
y = 96
}
runCatching { wm.addView(compose, params) }
.onFailure {
Log.w(TAG, "addView(chip) failed", it)
return
}
compose.post { ComposeArrWorkaround.disableForViewTree(compose) }
chipView = compose
}
// ── Confirmation modal ───────────────────────────────────────────────
override fun showConfirmation(
request: PendingConfirmation,
onResult: (allowed: Boolean) -> Unit,
) {
if (activeConfirmations.containsKey(request.id)) return
if (!Settings.canDrawOverlays(appContext)) {
Log.w(TAG, "showConfirmation: SYSTEM_ALERT_WINDOW not granted — denying")
onResult(false)
return
}
val compose = ComposeView(appContext).apply {
setViewCompositionStrategy(ViewCompositionStrategy.DisposeOnDetachedFromWindow)
setContent {
MaterialTheme {
DestructiveVerbConfirmDialog(
method = request.method,
verb = request.verb,
fullText = request.text,
onAllow = {
onResult(true)
dismissConfirmation(request.id)
},
onDeny = {
onResult(false)
dismissConfirmation(request.id)
},
)
}
}
}
attachLifecycle(compose)
val params = WindowManager.LayoutParams(
WindowManager.LayoutParams.MATCH_PARENT,
WindowManager.LayoutParams.MATCH_PARENT,
overlayType(),
WindowManager.LayoutParams.FLAG_DIM_BEHIND or
WindowManager.LayoutParams.FLAG_LAYOUT_IN_SCREEN,
PixelFormat.TRANSLUCENT,
).apply {
dimAmount = 0.6f
gravity = Gravity.CENTER
}
val added = runCatching { wm.addView(compose, params) }.isSuccess
if (!added) {
Log.w(TAG, "addView(confirm) failed — denying")
onResult(false)
return
}
compose.post { ComposeArrWorkaround.disableForViewTree(compose) }
activeConfirmations[request.id] = compose
}
override fun dismissConfirmation(requestId: Long) {
val view = activeConfirmations.remove(requestId) ?: return
runCatching { wm.removeView(view) }
.onFailure { Log.w(TAG, "removeView(confirm $requestId) failed", it) }
}
// ── Internals ────────────────────────────────────────────────────────
private fun overlayType(): Int =
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY
} else {
@Suppress("DEPRECATION")
WindowManager.LayoutParams.TYPE_PHONE
}
private fun attachLifecycle(view: View) {
val owner = OverlayLifecycleOwner().also { it.start() }
view.setViewTreeLifecycleOwner(owner)
view.setViewTreeViewModelStoreOwner(owner)
// REQUIRED even when the overlay content uses only plain `remember`:
// `AndroidComposeView.onAttachedToWindow` hard-fails with
// `IllegalStateException: Composed into the View which doesn't
// propagateViewTreeSavedStateRegistryOwner` if this tree owner is
// missing, regardless of whether the composable actually reads
// saved state. Confirmed empirically on Samsung S24 / Android 14
// / Compose BOM 2024.12 when enabling the persistent status chip
// from the Bridge Safety screen (Phase 3 safety-rails).
view.setViewTreeSavedStateRegistryOwner(owner)
}
}
/**
* Minimal always-RESUMED lifecycle owner for ComposeViews we attach to
* a `WindowManager`. Compose's `Recomposer` refuses to run inside a view
* that has no `ViewTreeLifecycleOwner`; `viewModel()` calls inside the
* overlay need a `ViewModelStore`; and — as of recent Compose versions —
* `AndroidComposeView.onAttachedToWindow` hard-requires a
* `ViewTreeSavedStateRegistryOwner` even for composables that never read
* saved state. So this class implements all three.
*
* Earlier versions of this file deliberately skipped
* `SavedStateRegistryOwner` on the assumption that "no `rememberSaveable`
* → no saved state needed". That assumption was wrong: the onAttach gate
* in `AndroidComposeView` doesn't inspect the composable body, it just
* checks for the tree owner and throws. The crash was
* [IllegalStateException] at `AndroidComposeView.onAttachedToWindow:2234`
* on every overlay attach.
*
* ## Init sequence — DO NOT REORDER
*
* Current androidx.savedstate requires:
*
* 1. `savedStateController.performRestore(null)` — while the owner is
* still in [Lifecycle.State.INITIALIZED]. Internally this calls
* `performAttach()` which hard-asserts `currentState == INITIALIZED`
* and throws `IllegalStateException: Restarter must be created only
* during owner's initialization stage` if you've already advanced
* past it.
* 2. `registry.currentState = CREATED`
* 3. `registry.currentState = RESUMED`
*
* An older androidx.savedstate release required the OPPOSITE order
* (CREATED → performRestore → RESUMED) and this file shipped with that
* code, matching the KDoc. The 2026-04-15 Compose BOM bump flipped the
* contract and the overlay started throwing on every destructive-verb
* confirmation attempt. Caught by Bailey's on-device voice→SMS test
* that same day — see the `BridgeSafetyMgr` stack trace in the session
* log. The chip path didn't trigger it because it was never exercised
* in the same build + flavor combo; only the confirmation modal path
* hit the assertion.
*/
private class OverlayLifecycleOwner :
LifecycleOwner,
ViewModelStoreOwner,
SavedStateRegistryOwner {
private val registry = LifecycleRegistry(this)
override val lifecycle: Lifecycle get() = registry
private val store = ViewModelStore()
override val viewModelStore: ViewModelStore get() = store
private val savedStateController = SavedStateRegistryController.create(this)
override val savedStateRegistry: SavedStateRegistry
get() = savedStateController.savedStateRegistry
fun start() {
// Restore saved state FIRST — must run while currentState is still
// INITIALIZED or performAttach() throws. See KDoc above for the
// assertion story.
savedStateController.performRestore(null)
registry.currentState = Lifecycle.State.CREATED
registry.currentState = Lifecycle.State.RESUMED
}
fun stop() {
registry.currentState = Lifecycle.State.DESTROYED
store.clear()
}
}
@@ -0,0 +1,127 @@
package com.hermesandroid.relay.data
import android.content.Context
import androidx.datastore.preferences.core.booleanPreferencesKey
import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.core.stringPreferencesKey
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.map
import kotlinx.serialization.Serializable
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
/**
* User-tunable bridge mode preferences + persisted activity log.
*
* Phase 3 Wave 1 — owned by Agent bridge-ui (`bridge-screen-ui`).
*
* - [masterEnabled] — the headline "Allow Agent Control" switch on BridgeScreen.
* Compose-layer gate only; the real safety fence is that the user must also
* enable `HermesAccessibilityService` in Android Settings (which is the
* Tier-5 "honest" trust boundary Google cares about). Persisting it here lets
* us survive restarts and lets Agent accessibility's `HermesAccessibilityService` query
* the same source of truth without needing its own DataStore file.
*
* - [activityLog] — rolling window of recent [BridgeActivityEntry] rows that
* back the Activity Log card. Capped at [MAX_LOG_ENTRIES] on every append
* so we don't grow the DataStore preferences file unbounded (this is a Prefs
* datastore, not Room — large blob values hurt commit latency). Serialized
* as a single JSON array string under one key so the whole read/update is
* atomic. That's cheaper than Proto DataStore for a cap this small and
* matches the `VoicePreferences.kt` / `MediaSettings.kt` style already in
* the tree.
*
* The log schema intentionally does NOT carry the screenshot bytes — those
* live in `MediaRegistry` on the relay side and are fetched via
* `RelayHttpClient.fetchMedia(token)` when the user expands the row. Storing
* `thumbnailToken: String?` as an opaque reference keeps DataStore small and
* reuses the existing MediaRegistry LRU-cap + FileProvider cache story.
*/
@Serializable
data class BridgeActivityEntry(
/** Epoch millis when the command was received from the relay. */
val timestampMs: Long,
/** Short method name — `tap`, `tap_text`, `type`, `swipe`, `read_screen`, etc. */
val method: String,
/** Free-form one-line summary of the args — `"(540, 1200)"`, `"send"`, `"Chrome"`. */
val summary: String,
/** Execution status — Pending, Success, Failed, Blocked (by Tier-5 safety rails). */
val status: BridgeActivityStatus,
/** Optional longer result text — set on Success / Failed to show in the expanded row. */
val resultText: String? = null,
/**
* Optional screenshot MediaRegistry token (`hermes-relay://<token>` or the
* raw token stem). The UI resolves this through the existing InboundAttachmentCard
* pipeline — bridge-ui deliberately does not invent a second thumbnail cache.
* Null for commands that don't carry a screenshot.
*/
val thumbnailToken: String? = null,
/** Unique ID used as the LazyColumn key — lets Compose animate inserts cleanly. */
val id: String,
)
@Serializable
enum class BridgeActivityStatus { Pending, Success, Failed, Blocked }
data class BridgeSettings(
val masterEnabled: Boolean = false,
)
class BridgePreferencesRepository(private val context: Context) {
companion object {
private val KEY_MASTER_ENABLED = booleanPreferencesKey("bridge_master_enabled")
private val KEY_ACTIVITY_LOG = stringPreferencesKey("bridge_activity_log")
/** Hard cap on persisted entries. See file-level KDoc for rationale. */
const val MAX_LOG_ENTRIES = 100
const val DEFAULT_MASTER_ENABLED = false
}
// Lenient JSON — ignore unknown keys so we can evolve the schema without
// breaking installed users on app upgrade, mirroring how PairingPreferences
// and MediaSettings handle forward-compat.
private val json = Json {
ignoreUnknownKeys = true
encodeDefaults = true
}
val settings: Flow<BridgeSettings> = context.relayDataStore.data.map { prefs ->
BridgeSettings(
masterEnabled = prefs[KEY_MASTER_ENABLED] ?: DEFAULT_MASTER_ENABLED,
)
}
val activityLog: Flow<List<BridgeActivityEntry>> = context.relayDataStore.data.map { prefs ->
val raw = prefs[KEY_ACTIVITY_LOG] ?: return@map emptyList()
runCatching { json.decodeFromString<List<BridgeActivityEntry>>(raw) }
.getOrDefault(emptyList())
}
suspend fun setMasterEnabled(enabled: Boolean) {
context.relayDataStore.edit { it[KEY_MASTER_ENABLED] = enabled }
}
/**
* Prepend a new entry and trim to [MAX_LOG_ENTRIES]. Idempotent on
* duplicate ids — if an entry with the same id already exists we replace
* it in-place (used when a Pending entry transitions to Success/Failed).
*/
suspend fun appendEntry(entry: BridgeActivityEntry) {
context.relayDataStore.edit { prefs ->
val current = prefs[KEY_ACTIVITY_LOG]?.let {
runCatching { json.decodeFromString<List<BridgeActivityEntry>>(it) }
.getOrDefault(emptyList())
} ?: emptyList()
val deduped = current.filterNot { it.id == entry.id }
val updated = (listOf(entry) + deduped).take(MAX_LOG_ENTRIES)
prefs[KEY_ACTIVITY_LOG] = json.encodeToString(updated)
}
}
suspend fun clearLog() {
context.relayDataStore.edit { prefs ->
prefs[KEY_ACTIVITY_LOG] = json.encodeToString(emptyList<BridgeActivityEntry>())
}
}
}
@@ -0,0 +1,242 @@
package com.hermesandroid.relay.data
import android.content.Context
import androidx.datastore.preferences.core.booleanPreferencesKey
import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.core.intPreferencesKey
import androidx.datastore.preferences.core.stringPreferencesKey
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.map
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
/**
* User-tunable Tier 5 bridge-safety preferences.
*
* Phase 3 Wave 2 — owned by Agent safety-rails (`bridge-safety-rails`).
*
* The five knobs this file backs match the Tier 5 spec:
*
* - [blocklist] — per-app package-name allowlist-inverse. If the currently
* foregrounded package is in this set, `BridgeCommandHandler` refuses
* every action with HTTP 403. Defaults ship with a conservative list of
* common banking apps + password managers so first-run users are safe
* from "agent taps Transfer in my banking app" accidents before they
* ever touch the settings screen. Users can edit freely.
*
* - [destructiveVerbs] — words that trigger a confirmation modal when they
* appear in `/tap_text` or `/type` payloads. Seeded with a set of verbs
* that carry irreversible or high-stakes consequences. Editable.
*
* - [autoDisableMinutes] — idle timeout after which the master toggle
* auto-flips to false. Rescheduled on every command so an active agent
* never triggers it; a runaway agent that stops sending commands for
* this long loses bridge access automatically.
*
* - [statusOverlayEnabled] — opt-in floating-dot indicator (like the
* screen-recording red dot) that's visible while bridge is active.
* Off by default — some users hate persistent overlays, foreground-
* service notification already signals liveness.
*
* - [confirmationTimeoutSeconds] — how long the destructive-verb modal
* waits before treating silence as DENY. Default 30s.
*
* Matches the `BridgePreferences.kt` / `VoicePreferences.kt` / `MediaSettings.kt`
* style: a single DataStore (`relayDataStore`), lists serialized as JSON
* strings under one key each, lenient deserialization so schema evolution
* doesn't break installed users.
*/
data class BridgeSafetySettings(
val blocklist: Set<String> = DEFAULT_BLOCKLIST,
val destructiveVerbs: Set<String> = DEFAULT_DESTRUCTIVE_VERBS,
val autoDisableMinutes: Int = DEFAULT_AUTO_DISABLE_MINUTES,
val statusOverlayEnabled: Boolean = DEFAULT_STATUS_OVERLAY_ENABLED,
val confirmationTimeoutSeconds: Int = DEFAULT_CONFIRMATION_TIMEOUT_SECONDS,
)
/** Seeded blocklist — conservative high-stakes packages shipped out of the box. */
val DEFAULT_BLOCKLIST: Set<String> = setOf(
// Banking — US / UK / generic
"com.chase.sig.android",
"com.wf.wellsfargomobile",
"com.bankofamerica.digitalwallet",
"com.usaa.mobile.android.usaa",
"com.konylabs.capitalone",
"com.americanexpress.android.acctsvcs.us",
"com.discoverfinancial.mobile",
"com.infonow.bofa",
"uk.co.hsbc.hsbcukmobilebanking",
"com.barclays.android.barclaysmobilebanking",
"com.monzo.android",
"co.uk.getmondo",
"co.revolut.app",
"com.starlingbank.android",
// Payments / crypto
"com.venmo",
"com.squareup.cash",
"com.paypal.android.p2pmobile",
"com.coinbase.android",
"co.mona.android",
// Password managers
"com.lastpass.lpandroid",
"com.agilebits.onepassword",
"com.x8bit.bitwarden",
"com.dashlane.frozenaccount",
"com.keepersecurity.passwordmanager",
"com.bitwarden.authenticator",
// 2FA apps
"com.google.android.apps.authenticator2",
"com.authy.authy",
"com.duosecurity.duomobile",
)
/**
* Seeded destructive-verb list. Matched case-insensitively as whole words
* (via regex `\b<verb>\b`) inside `tap_text` / `type` payloads. Users can
* add/remove via [BridgeSafetySettingsScreen].
*/
val DEFAULT_DESTRUCTIVE_VERBS: Set<String> = setOf(
"send",
"pay",
"delete",
"transfer",
"confirm",
"submit",
"post",
"publish",
"buy",
"purchase",
"charge",
"withdraw",
)
const val DEFAULT_AUTO_DISABLE_MINUTES: Int = 30
const val MIN_AUTO_DISABLE_MINUTES: Int = 5
const val MAX_AUTO_DISABLE_MINUTES: Int = 120
const val DEFAULT_STATUS_OVERLAY_ENABLED: Boolean = false
const val DEFAULT_CONFIRMATION_TIMEOUT_SECONDS: Int = 30
const val MIN_CONFIRMATION_TIMEOUT_SECONDS: Int = 10
const val MAX_CONFIRMATION_TIMEOUT_SECONDS: Int = 60
class BridgeSafetyPreferencesRepository(private val context: Context) {
companion object {
private val KEY_BLOCKLIST = stringPreferencesKey("bridge_blocklist")
private val KEY_DESTRUCTIVE_VERBS = stringPreferencesKey("bridge_destructive_verbs")
private val KEY_AUTO_DISABLE_MINUTES = intPreferencesKey("bridge_auto_disable_minutes")
private val KEY_STATUS_OVERLAY = booleanPreferencesKey("bridge_status_overlay_enabled")
private val KEY_CONFIRMATION_TIMEOUT =
intPreferencesKey("bridge_confirmation_timeout_seconds")
/** Sentinel key we set the first time settings get written. Used to
* tell "user cleared the blocklist" from "user has never touched it". */
private val KEY_SAFETY_INITIALIZED = booleanPreferencesKey("bridge_safety_initialized")
}
private val json = Json {
ignoreUnknownKeys = true
encodeDefaults = true
}
val settings: Flow<BridgeSafetySettings> = context.relayDataStore.data.map { prefs ->
val initialized = prefs[KEY_SAFETY_INITIALIZED] ?: false
val blocklist = prefs[KEY_BLOCKLIST]?.let { decodeSet(it) }
?: if (initialized) emptySet() else DEFAULT_BLOCKLIST
val verbs = prefs[KEY_DESTRUCTIVE_VERBS]?.let { decodeSet(it) }
?: if (initialized) emptySet() else DEFAULT_DESTRUCTIVE_VERBS
BridgeSafetySettings(
blocklist = blocklist,
destructiveVerbs = verbs,
autoDisableMinutes = (prefs[KEY_AUTO_DISABLE_MINUTES] ?: DEFAULT_AUTO_DISABLE_MINUTES)
.coerceIn(MIN_AUTO_DISABLE_MINUTES, MAX_AUTO_DISABLE_MINUTES),
statusOverlayEnabled = prefs[KEY_STATUS_OVERLAY] ?: DEFAULT_STATUS_OVERLAY_ENABLED,
confirmationTimeoutSeconds = (prefs[KEY_CONFIRMATION_TIMEOUT]
?: DEFAULT_CONFIRMATION_TIMEOUT_SECONDS)
.coerceIn(MIN_CONFIRMATION_TIMEOUT_SECONDS, MAX_CONFIRMATION_TIMEOUT_SECONDS),
)
}
suspend fun setBlocklist(packages: Set<String>) {
context.relayDataStore.edit { prefs ->
prefs[KEY_BLOCKLIST] = json.encodeToString(packages.toList().sorted())
prefs[KEY_SAFETY_INITIALIZED] = true
}
}
suspend fun addToBlocklist(packageName: String) {
context.relayDataStore.edit { prefs ->
val current = prefs[KEY_BLOCKLIST]?.let { decodeSet(it) } ?: DEFAULT_BLOCKLIST
val next = (current + packageName).toList().sorted()
prefs[KEY_BLOCKLIST] = json.encodeToString(next)
prefs[KEY_SAFETY_INITIALIZED] = true
}
}
suspend fun removeFromBlocklist(packageName: String) {
context.relayDataStore.edit { prefs ->
val current = prefs[KEY_BLOCKLIST]?.let { decodeSet(it) } ?: DEFAULT_BLOCKLIST
val next = (current - packageName).toList().sorted()
prefs[KEY_BLOCKLIST] = json.encodeToString(next)
prefs[KEY_SAFETY_INITIALIZED] = true
}
}
suspend fun setDestructiveVerbs(verbs: Set<String>) {
context.relayDataStore.edit { prefs ->
prefs[KEY_DESTRUCTIVE_VERBS] = json.encodeToString(
verbs.map { it.trim().lowercase() }.filter { it.isNotEmpty() }.toSet().toList().sorted()
)
prefs[KEY_SAFETY_INITIALIZED] = true
}
}
suspend fun addDestructiveVerb(verb: String) {
val normalized = verb.trim().lowercase()
if (normalized.isEmpty()) return
context.relayDataStore.edit { prefs ->
val current = prefs[KEY_DESTRUCTIVE_VERBS]?.let { decodeSet(it) } ?: DEFAULT_DESTRUCTIVE_VERBS
val next = (current + normalized).toList().sorted()
prefs[KEY_DESTRUCTIVE_VERBS] = json.encodeToString(next)
prefs[KEY_SAFETY_INITIALIZED] = true
}
}
suspend fun removeDestructiveVerb(verb: String) {
val normalized = verb.trim().lowercase()
context.relayDataStore.edit { prefs ->
val current = prefs[KEY_DESTRUCTIVE_VERBS]?.let { decodeSet(it) } ?: DEFAULT_DESTRUCTIVE_VERBS
val next = (current - normalized).toList().sorted()
prefs[KEY_DESTRUCTIVE_VERBS] = json.encodeToString(next)
prefs[KEY_SAFETY_INITIALIZED] = true
}
}
suspend fun setAutoDisableMinutes(minutes: Int) {
context.relayDataStore.edit { prefs ->
prefs[KEY_AUTO_DISABLE_MINUTES] =
minutes.coerceIn(MIN_AUTO_DISABLE_MINUTES, MAX_AUTO_DISABLE_MINUTES)
prefs[KEY_SAFETY_INITIALIZED] = true
}
}
suspend fun setStatusOverlayEnabled(enabled: Boolean) {
context.relayDataStore.edit { prefs ->
prefs[KEY_STATUS_OVERLAY] = enabled
prefs[KEY_SAFETY_INITIALIZED] = true
}
}
suspend fun setConfirmationTimeoutSeconds(seconds: Int) {
context.relayDataStore.edit { prefs ->
prefs[KEY_CONFIRMATION_TIMEOUT] =
seconds.coerceIn(MIN_CONFIRMATION_TIMEOUT_SECONDS, MAX_CONFIRMATION_TIMEOUT_SECONDS)
prefs[KEY_SAFETY_INITIALIZED] = true
}
}
private fun decodeSet(raw: String): Set<String> =
runCatching { json.decodeFromString<List<String>>(raw).toSet() }
.getOrDefault(emptySet())
}
@@ -1,5 +1,27 @@
package com.hermesandroid.relay.data
/**
* Lifecycle state of an inbound attachment (media fetched from the relay).
*
* Outbound attachments authored by the user are always [LOADED].
* Inbound attachments start as [LOADING], then transition to [LOADED] (bytes
* cached + content:// URI available) or [FAILED] (network error, size cap
* exceeded, relay offline, etc).
*/
enum class AttachmentState { LOADING, LOADED, FAILED }
/**
* How the UI should render a loaded attachment. Derived from the MIME type.
* - [IMAGE] inline image (decode bytes / load URI).
* - [VIDEO] file card with play icon, tap opens ACTION_VIEW.
* - [AUDIO] file card with audio icon, tap opens ACTION_VIEW.
* - [PDF] file card with document icon, tap opens ACTION_VIEW.
* - [TEXT] file card with text icon (used for any `text/...` MIME and
* text-like application types: json, xml, yaml, toml, etc).
* - [GENERIC] generic file card for unknown or binary types.
*/
enum class AttachmentRenderMode { IMAGE, VIDEO, AUDIO, PDF, TEXT, GENERIC }
data class ChatMessage(
val id: String,
val role: MessageRole,
@@ -23,15 +45,69 @@ data class ChatMessage(
/**
* A file attachment sent with a message.
* Matches the Hermes API format: { contentType, content (base64) }
*
* Two shapes:
* 1. Outbound — user picks a file, it's base64'd into [content] with a known
* [contentType]. [state] defaults to [AttachmentState.LOADED] so the
* render pipeline treats it like a "ready" attachment.
* 2. Inbound — tool output emitted a `MEDIA:hermes-relay://<token>` marker.
* Starts as [AttachmentState.LOADING] with [relayToken] set. Once the
* bytes land via [RelayHttpClient.fetchMedia], [cachedUri] is populated
* with a `content://` URI from the FileProvider and state flips to
* [AttachmentState.LOADED]. On failure state is [AttachmentState.FAILED]
* and [errorMessage] holds a human-readable reason.
*
* Matches the Hermes API outbound format: { contentType, content (base64) }.
*/
data class Attachment(
val contentType: String, // MIME type (e.g. "image/png", "application/pdf", "text/plain")
val content: String, // Base64-encoded file content
val content: String, // Base64-encoded file content (outbound) or empty (inbound)
val fileName: String? = null,
val fileSize: Long? = null
val fileSize: Long? = null,
// --- Inbound fetch state ---
val state: AttachmentState = AttachmentState.LOADED,
val errorMessage: String? = null,
/** Opaque token from `MEDIA:hermes-relay://<token>` — identifies the file on the relay. */
val relayToken: String? = null,
/** content:// URI from the FileProvider once bytes are cached to disk. */
val cachedUri: String? = null
) {
val isImage: Boolean get() = contentType.startsWith("image/")
/**
* How the UI should render this attachment, derived from [contentType].
* Falls back to [AttachmentRenderMode.GENERIC] for unknown types.
*/
val renderMode: AttachmentRenderMode
get() = when {
contentType.startsWith("image/") -> AttachmentRenderMode.IMAGE
contentType.startsWith("video/") -> AttachmentRenderMode.VIDEO
contentType.startsWith("audio/") -> AttachmentRenderMode.AUDIO
contentType == "application/pdf" -> AttachmentRenderMode.PDF
contentType.startsWith("text/") || contentType in textLikeMimes -> AttachmentRenderMode.TEXT
else -> AttachmentRenderMode.GENERIC
}
companion object {
/**
* MIME types that are text-like even though they don't start with `text/`.
* Used by [renderMode] to route these to [AttachmentRenderMode.TEXT].
*/
val textLikeMimes = setOf(
"application/json",
"application/xml",
"application/yaml",
"application/x-yaml",
"application/toml",
"application/javascript",
"application/x-sh",
"text/plain",
"text/markdown",
"text/html",
"text/css",
"text/csv"
)
}
}
data class ToolCall(
@@ -60,3 +60,59 @@ object FeatureFlags {
}
}
}
/**
* Compile-time gating based on the active Gradle product flavor.
*
* Phase 3 ships Bridge on two tracks with very different AccessibilityService
* scope: the `googlePlay` flavor carries a conservative event-type subset and
* a "notifications + confirmations" description for Play Store policy review,
* and the `sideload` flavor carries the full agent-control surface. The tier
* flags below let UI code hide tier 3/4/6 surfaces on the Play build without
* a runtime check — Kotlin's `val … get() = current == SIDELOAD` resolves at
* each call site, but because `current` is a compile-time string, R8 is able
* to fold the check away in release builds.
*
* Tier definitions (see `Phase 3 — Bridge Channel.md` in the vault):
* 1. baseline — both tracks (app open, tap, navigate within app)
* 2. notifications — both tracks (read notifications, summarize, reply)
* 3. voice-first — sideload only (always-on voice capture)
* 4. vision-first — sideload only (always-on screen reading)
* 5. safety rails — both tracks (confirmation dialogs, action log)
* 6. ambitious future — sideload only (cross-app macros, scheduling)
*/
object BuildFlavor {
const val GOOGLE_PLAY = "googlePlay"
const val SIDELOAD = "sideload"
val current: String get() = BuildConfig.FLAVOR
/**
* True when the current build is the sideload track. Kept as a property
* getter (not a compile-time `val`) so the call site reads cleanly —
* `BuildFlavor.isSideload` is easier to eyeball in `BridgeCommandHandler`
* than `BuildFlavor.current == BuildFlavor.SIDELOAD`. R8 folds the
* comparison away in release builds because `current` resolves to a
* compile-time `BuildConfig.FLAVOR` string constant.
*
* Used by Tier C (C1-C4) tool gates — `android_location`,
* `android_search_contacts`, `android_call`, `android_send_sms` — to
* return `"sideload-only"` 403 responses on googlePlay builds instead
* of crashing on a missing permission declaration.
*/
val isSideload: Boolean get() = current == SIDELOAD
val bridgeTier1: Boolean = true // baseline — both tracks
val bridgeTier2: Boolean = true // notifications, calendar — both tracks
val bridgeTier3: Boolean get() = current == SIDELOAD // voice-first
val bridgeTier4: Boolean get() = current == SIDELOAD // vision-first
val bridgeTier5: Boolean = true // safety rails — always on
val bridgeTier6: Boolean get() = current == SIDELOAD // future ambitious
/** Human-readable badge label for the Settings → About version row. */
val displayName: String
get() = when (current) {
GOOGLE_PLAY -> "Google Play"
SIDELOAD -> "Sideload"
else -> current.ifBlank { "Unknown" }
}
}
@@ -0,0 +1,73 @@
package com.hermesandroid.relay.data
import android.content.Context
import androidx.datastore.preferences.core.booleanPreferencesKey
import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.core.intPreferencesKey
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.map
/**
* User-tunable limits for inbound media attachments fetched from the relay.
*
* - [maxInboundSizeMb] hard cap — anything larger is rejected after download
* and the attachment flips to FAILED with a "too large" message.
* - [autoFetchThresholdMb] soft threshold — files up to this size auto-fetch
* on any network. Above it, still auto-fetch on Wi-Fi but show a manual
* "tap to download" CTA on cellular (unless [autoFetchOnCellular] is set).
* (Current implementation auto-fetches everything up to the max cap when
* the connection isn't cellular; the threshold gates cellular behavior.)
* - [autoFetchOnCellular] master switch: when false, the cellular-network
* case always inserts a manual-download placeholder.
* - [cachedMediaCapMb] LRU cap on the `hermes-media/` cache directory.
*/
data class MediaSettings(
val maxInboundSizeMb: Int = 25,
val autoFetchThresholdMb: Int = 2,
val autoFetchOnCellular: Boolean = false,
val cachedMediaCapMb: Int = 200
)
/**
* DataStore-backed settings repository for inbound media behavior.
* Shares the app-wide [relayDataStore] so all preferences live in one file.
*/
class MediaSettingsRepository(private val context: Context) {
companion object {
private val KEY_MAX_INBOUND_MB = intPreferencesKey("media_max_inbound_mb")
private val KEY_AUTO_FETCH_THRESHOLD_MB = intPreferencesKey("media_auto_fetch_threshold_mb")
private val KEY_AUTO_FETCH_ON_CELLULAR = booleanPreferencesKey("media_auto_fetch_on_cellular")
private val KEY_CACHED_MEDIA_CAP_MB = intPreferencesKey("media_cached_cap_mb")
const val DEFAULT_MAX_INBOUND_MB = 25
const val DEFAULT_AUTO_FETCH_THRESHOLD_MB = 2
const val DEFAULT_AUTO_FETCH_ON_CELLULAR = false
const val DEFAULT_CACHED_MEDIA_CAP_MB = 200
}
val settings: Flow<MediaSettings> = context.relayDataStore.data.map { prefs ->
MediaSettings(
maxInboundSizeMb = prefs[KEY_MAX_INBOUND_MB] ?: DEFAULT_MAX_INBOUND_MB,
autoFetchThresholdMb = prefs[KEY_AUTO_FETCH_THRESHOLD_MB] ?: DEFAULT_AUTO_FETCH_THRESHOLD_MB,
autoFetchOnCellular = prefs[KEY_AUTO_FETCH_ON_CELLULAR] ?: DEFAULT_AUTO_FETCH_ON_CELLULAR,
cachedMediaCapMb = prefs[KEY_CACHED_MEDIA_CAP_MB] ?: DEFAULT_CACHED_MEDIA_CAP_MB
)
}
suspend fun setMaxInboundSize(mb: Int) {
context.relayDataStore.edit { it[KEY_MAX_INBOUND_MB] = mb.coerceAtLeast(1) }
}
suspend fun setAutoFetchThreshold(mb: Int) {
context.relayDataStore.edit { it[KEY_AUTO_FETCH_THRESHOLD_MB] = mb.coerceAtLeast(0) }
}
suspend fun setAutoFetchOnCellular(enabled: Boolean) {
context.relayDataStore.edit { it[KEY_AUTO_FETCH_ON_CELLULAR] = enabled }
}
suspend fun setCachedMediaCap(mb: Int) {
context.relayDataStore.edit { it[KEY_CACHED_MEDIA_CAP_MB] = mb.coerceAtLeast(10) }
}
}
@@ -0,0 +1,142 @@
package com.hermesandroid.relay.data
import android.content.Context
import androidx.datastore.preferences.core.booleanPreferencesKey
import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.core.longPreferencesKey
import androidx.datastore.preferences.core.stringPreferencesKey
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.flow.map
/**
* DataStore-backed preferences for the pairing + security overhaul introduced
* 2026-04-11.
*
* Stores:
* - `pair_ttl_seconds` — user's last-selected session TTL (0 = never). Used
* to preselect the [SessionTtlPickerDialog] on the next pair attempt.
* - `insecure_ack_seen` — whether the user has dismissed the one-time
* [InsecureConnectionAckDialog] explaining the threat model of ws://.
* - `insecure_reason` — the reason the user selected in that dialog. Used
* for display only (drives the transport security badge label) — never
* gating. Empty string when unset.
* - `tofu_pins` — stringified `host:port=sha256:hex|host:port=sha256:hex`
* map of TOFU-pinned certificate fingerprints. Simple delimited encoding
* keeps us out of `Gson`/full-object serialization while still handling
* multiple hosts.
*
* Lives alongside [FeatureFlags] on the same DataStore instance
* (`relay_settings`) so all settings share one file.
*/
object PairingPreferences {
// 30 days — same default the TTL picker uses for wss/Tailscale setups.
const val DEFAULT_TTL_SECONDS: Long = 30L * 24L * 60L * 60L
// Sentinel — matches the wire contract ("never expire" when ttl_seconds
// is 0 or missing).
const val TTL_NEVER: Long = 0L
private val KEY_PAIR_TTL_SECONDS = longPreferencesKey("pair_ttl_seconds")
private val KEY_INSECURE_ACK_SEEN = booleanPreferencesKey("insecure_ack_seen")
private val KEY_INSECURE_REASON = stringPreferencesKey("insecure_reason")
private val KEY_TOFU_PINS = stringPreferencesKey("tofu_pins")
// --- Pair TTL -----------------------------------------------------------
/**
* Observe the user's last-selected TTL (epoch-seconds delta). Defaults to
* [DEFAULT_TTL_SECONDS] on first run.
*
* Callers that need to preselect the [SessionTtlPickerDialog] should use
* this. A value of [TTL_NEVER] means "never expire".
*/
fun pairTtlSeconds(context: Context): Flow<Long> =
context.relayDataStore.data.map { prefs ->
prefs[KEY_PAIR_TTL_SECONDS] ?: DEFAULT_TTL_SECONDS
}
suspend fun getPairTtlSeconds(context: Context): Long =
pairTtlSeconds(context).first()
suspend fun setPairTtlSeconds(context: Context, ttlSeconds: Long) {
context.relayDataStore.edit { prefs ->
prefs[KEY_PAIR_TTL_SECONDS] = ttlSeconds
}
}
// --- Insecure acknowledgment -------------------------------------------
fun insecureAckSeen(context: Context): Flow<Boolean> =
context.relayDataStore.data.map { it[KEY_INSECURE_ACK_SEEN] ?: false }
suspend fun setInsecureAckSeen(context: Context, seen: Boolean) {
context.relayDataStore.edit { it[KEY_INSECURE_ACK_SEEN] = seen }
}
/**
* Reason the user selected when they flipped insecure mode on.
*
* Expected values:
* - `"lan_only"` — "LAN only (trusted network)"
* - `"tailscale_vpn"` — "Tailscale or VPN"
* - `"local_dev"` — "Local development only"
* - `""` — not yet acknowledged
*
* Used for display in [TransportSecurityBadge] — never gates behavior.
*/
fun insecureReason(context: Context): Flow<String> =
context.relayDataStore.data.map { it[KEY_INSECURE_REASON] ?: "" }
suspend fun setInsecureReason(context: Context, reason: String) {
context.relayDataStore.edit { it[KEY_INSECURE_REASON] = reason }
}
// --- TOFU pins ----------------------------------------------------------
//
// Stored as a compact `host:port=fingerprint|host:port=fingerprint` string
// so we can handle multiple hosts without pulling in json-schema or
// bumping to Proto DataStore. Hosts are normalized lowercase; fingerprints
// are OkHttp CertificatePinner-compatible (`sha256/<base64>`).
fun tofuPins(context: Context): Flow<Map<String, String>> =
context.relayDataStore.data.map { prefs ->
decodePins(prefs[KEY_TOFU_PINS] ?: "")
}
suspend fun getTofuPins(context: Context): Map<String, String> =
tofuPins(context).first()
suspend fun setTofuPin(context: Context, hostPort: String, pin: String) {
val normalized = hostPort.lowercase()
context.relayDataStore.edit { prefs ->
val current = decodePins(prefs[KEY_TOFU_PINS] ?: "").toMutableMap()
current[normalized] = pin
prefs[KEY_TOFU_PINS] = encodePins(current)
}
}
suspend fun removeTofuPin(context: Context, hostPort: String) {
val normalized = hostPort.lowercase()
context.relayDataStore.edit { prefs ->
val current = decodePins(prefs[KEY_TOFU_PINS] ?: "").toMutableMap()
current.remove(normalized)
prefs[KEY_TOFU_PINS] = encodePins(current)
}
}
private fun decodePins(raw: String): Map<String, String> {
if (raw.isBlank()) return emptyMap()
return raw.split('|')
.mapNotNull { entry ->
val idx = entry.indexOf('=')
if (idx <= 0 || idx == entry.lastIndex) null
else entry.substring(0, idx) to entry.substring(idx + 1)
}
.toMap()
}
private fun encodePins(pins: Map<String, String>): String =
pins.entries.joinToString("|") { (host, pin) -> "$host=$pin" }
}
@@ -0,0 +1,67 @@
package com.hermesandroid.relay.data
import android.content.Context
import androidx.datastore.preferences.core.booleanPreferencesKey
import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.core.longPreferencesKey
import androidx.datastore.preferences.core.stringPreferencesKey
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.map
/**
* User-tunable voice mode preferences.
*
* - [interactionMode] how the mic button behaves: "tap" | "hold" | "continuous".
* Drives the VoiceViewModel's InteractionMode enum at startup.
* - [silenceThresholdMs] auto-stop threshold for listening: after this many
* ms of amplitude below the silence floor, stopListening() is called.
* - [autoTts] future — read TTS on every non-voice assistant message.
* - [language] STT language hint. Stored; not yet wired to /voice/transcribe
* (V1 doesn't accept a language param).
*/
data class VoiceSettings(
val interactionMode: String = "tap",
val silenceThresholdMs: Long = 3000L,
val autoTts: Boolean = false,
val language: String = "",
)
class VoicePreferencesRepository(private val context: Context) {
companion object {
private val KEY_INTERACTION_MODE = stringPreferencesKey("voice_interaction_mode")
private val KEY_SILENCE_THRESHOLD_MS = longPreferencesKey("voice_silence_threshold_ms")
private val KEY_AUTO_TTS = booleanPreferencesKey("voice_auto_tts")
private val KEY_LANGUAGE = stringPreferencesKey("voice_language")
const val DEFAULT_INTERACTION_MODE = "tap"
const val DEFAULT_SILENCE_THRESHOLD_MS = 3000L
const val DEFAULT_AUTO_TTS = false
const val DEFAULT_LANGUAGE = ""
}
val settings: Flow<VoiceSettings> = context.relayDataStore.data.map { prefs ->
VoiceSettings(
interactionMode = prefs[KEY_INTERACTION_MODE] ?: DEFAULT_INTERACTION_MODE,
silenceThresholdMs = prefs[KEY_SILENCE_THRESHOLD_MS] ?: DEFAULT_SILENCE_THRESHOLD_MS,
autoTts = prefs[KEY_AUTO_TTS] ?: DEFAULT_AUTO_TTS,
language = prefs[KEY_LANGUAGE] ?: DEFAULT_LANGUAGE,
)
}
suspend fun setInteractionMode(mode: String) {
context.relayDataStore.edit { it[KEY_INTERACTION_MODE] = mode }
}
suspend fun setSilenceThresholdMs(ms: Long) {
context.relayDataStore.edit { it[KEY_SILENCE_THRESHOLD_MS] = ms.coerceAtLeast(500L) }
}
suspend fun setAutoTts(enabled: Boolean) {
context.relayDataStore.edit { it[KEY_AUTO_TTS] = enabled }
}
suspend fun setLanguage(language: String) {
context.relayDataStore.edit { it[KEY_LANGUAGE] = language }
}
}
@@ -0,0 +1,279 @@
package com.hermesandroid.relay.event
import android.view.accessibility.AccessibilityEvent
/**
* Phase 3 — `B1 event-stream`
*
* Process-wide bounded ring buffer of recent [AccessibilityEvent]s,
* exposed via the `android_events(limit, since)` polling tool on the
* host. Off by default — capture only runs when
* [isStreaming] is true, which the agent toggles explicitly via
* `android_event_stream(true)`.
*
* # Why a ring buffer
*
* `onAccessibilityEvent` can fire hundreds of times per second during a
* scroll or text-change burst. Even with per-`(type, package)` throttling
* we can easily accumulate thousands of entries an hour, so we cap the
* store at [MAX_ENTRIES] and evict the oldest in FIFO order. The `since`
* parameter on the poll tool lets the agent efficiently ask "what's new
* since the last time I checked" without re-downloading history.
*
* # Why we filter event types
*
* Android emits a deep taxonomy of events (focus, selection, hover,
* gesture-detection, touch-exploration, view-scrolled-sub-tree, etc.).
* The overwhelming majority are low-signal for a reactive-agent use
* case — we keep only the five that answer "what is the user DOING":
*
* * click — the user pressed something
* * text changed — the user typed / pasted
* * window state changed — the foreground activity switched
* * window content changed — something on the current screen updated
* * scroll — the user scrolled a list
*
* Everything else is dropped at [append] so the buffer stays dense.
*
* # Throttling semantics
*
* We keep one event per `(eventTypeInt, packageName)` per
* [THROTTLE_WINDOW_MS]. This collapses scroll bursts and text-change
* storms into a single entry per app per 100ms, which is the natural
* human-observable granularity anyway. The throttle map is garbage-
* collected every [THROTTLE_GC_INTERVAL_MS] to keep its size bounded
* even if the user flips through many packages.
*
* # Thread safety
*
* Reads and writes both take the single [lock]. Android's accessibility
* service pumps events on its own binder thread; the poll tool flows in
* over the WSS multiplexer thread. A simple `synchronized` block is
* plenty — we're not in a hot rendering path.
*
* # Privacy
*
* Events can contain search queries, typed messages, password form
* values, and anything else that touches an EditText. The buffer is
* cleared automatically on any [setStreaming] transition (both
* enable→disable and disable→enable) so the agent cannot observe
* history from a previous "on" interval without the user's explicit
* second opt-in.
*/
object EventStore {
/** Maximum number of entries retained in the ring buffer. */
const val MAX_ENTRIES: Int = 500
/** Throttle window per `(eventTypeInt, packageName)` pair. */
const val THROTTLE_WINDOW_MS: Long = 100L
/** Sweep interval for the throttle map (keeps memory bounded). */
const val THROTTLE_GC_INTERVAL_MS: Long = 5_000L
/**
* One recorded accessibility event. Snake-case fields on the wire
* so the Python tool client sees a consistent envelope across every
* `android_*` surface.
*/
data class Entry(
val timestamp: Long,
val eventType: String,
val packageName: String?,
val className: String?,
val text: String?,
val contentDescription: String?,
val source: String,
)
private val lock = Any()
private val buffer: ArrayDeque<Entry> = ArrayDeque(MAX_ENTRIES)
/**
* Per-`(eventTypeInt, packageName)` last-appended timestamp for the
* [THROTTLE_WINDOW_MS] gate. Swept by [maybeGcThrottleMap] whenever
* [append] runs and the clock has advanced past the last sweep by
* [THROTTLE_GC_INTERVAL_MS].
*/
private val throttleMap: MutableMap<Pair<Int, String>, Long> = HashMap()
private var lastThrottleGcAt: Long = 0L
@Volatile
var isStreaming: Boolean = false
private set
/**
* Enable or disable event capture. Always clears the buffer on a
* state change (both directions) to avoid two kinds of leak:
*
* * on disable → the buffer is wiped so a subsequent re-enable
* cannot replay stale "off"-interval events.
* * on enable → we start with a fresh slate so no pre-existing
* state from a prior session is mixed with the new stream.
*
* A no-op call (same value) leaves the buffer alone.
*/
fun setStreaming(enabled: Boolean) {
synchronized(lock) {
if (isStreaming == enabled) return
isStreaming = enabled
buffer.clear()
throttleMap.clear()
lastThrottleGcAt = 0L
}
}
/**
* Append an AccessibilityEvent to the buffer if it survives the
* type filter and the `(type, package)` throttle. Silently drops
* events when [isStreaming] is false — the caller in
* `HermesAccessibilityService.onAccessibilityEvent` can still call
* us unconditionally.
*/
fun append(event: AccessibilityEvent) {
// M2 fix: previously the `isStreaming` check happened OUTSIDE the
// lock, creating a TOCTOU window. A binder-thread append could read
// `isStreaming == true` and then enter the lock AFTER `setStreaming
// (false)` had already cleared the buffer, adding a stale entry.
// Move the check INSIDE the synchronized block so the streaming gate
// and the buffer mutation are atomic. The @Volatile on `isStreaming`
// stays as a cheap fast-path hint outside the hot lock.
if (!isStreaming) return
val eventTypeInt = event.eventType
val humanType = humanEventType(eventTypeInt) ?: return
val pkg = event.packageName?.toString()
val now = System.currentTimeMillis()
synchronized(lock) {
// Re-check under the lock so a concurrent setStreaming(false)
// can't race a stale entry past the volatile read above.
if (!isStreaming) return
// Throttle: drop if we saw a same-kind event in the last window.
val key = Pair(eventTypeInt, pkg ?: "")
val last = throttleMap[key]
if (last != null && (now - last) < THROTTLE_WINDOW_MS) {
return
}
throttleMap[key] = now
maybeGcThrottleMap(now)
val entry = Entry(
timestamp = now,
eventType = humanType,
packageName = pkg,
className = event.className?.toString(),
text = concatEventText(event),
contentDescription = event.contentDescription?.toString(),
source = sourceForEventType(eventTypeInt),
)
if (buffer.size >= MAX_ENTRIES) {
buffer.removeFirst()
}
buffer.addLast(entry)
}
}
/**
* Return up to [limit] most-recent entries (oldest-first inside
* the returned list, matching poll semantics of "play these back in
* chronological order"). If [since] is non-zero, only entries with
* `timestamp > since` are returned.
*/
fun recent(limit: Int = 50, since: Long = 0L): List<Entry> {
if (limit <= 0) return emptyList()
synchronized(lock) {
if (buffer.isEmpty()) return emptyList()
// Collect chronologically (buffer is already oldest → newest)
// filtered by since, then tail-trimmed to limit.
val filtered = if (since > 0L) {
buffer.filter { it.timestamp > since }
} else {
buffer.toList()
}
return if (filtered.size <= limit) {
filtered
} else {
filtered.subList(filtered.size - limit, filtered.size).toList()
}
}
}
/** Clear both the buffer and the throttle map. */
fun clear() {
synchronized(lock) {
buffer.clear()
throttleMap.clear()
lastThrottleGcAt = 0L
}
}
/** Test / diag helper — current number of buffered entries. */
fun size(): Int = synchronized(lock) { buffer.size }
// ── Internals ───────────────────────────────────────────────────
/**
* Map an Android event-type int to the short human string the
* polling tool returns. Returning `null` is the drop signal — the
* five accepted types are the signal-rich ones per the Phase 3
* event-stream plan; every other type is silently ignored.
*/
private fun humanEventType(eventType: Int): String? = when (eventType) {
AccessibilityEvent.TYPE_VIEW_CLICKED -> "click"
AccessibilityEvent.TYPE_VIEW_TEXT_CHANGED -> "text_changed"
AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED -> "window_content_changed"
AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED -> "window_state_changed"
AccessibilityEvent.TYPE_VIEW_SCROLLED -> "scroll"
else -> null
}
/**
* Stable `TYPE_*` name for the `source` field — mirrors the Android
* constant name so downstream consumers can correlate with logcat.
*/
private fun sourceForEventType(eventType: Int): String = when (eventType) {
AccessibilityEvent.TYPE_VIEW_CLICKED -> "TYPE_VIEW_CLICKED"
AccessibilityEvent.TYPE_VIEW_TEXT_CHANGED -> "TYPE_VIEW_TEXT_CHANGED"
AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED -> "TYPE_WINDOW_CONTENT_CHANGED"
AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED -> "TYPE_WINDOW_STATE_CHANGED"
AccessibilityEvent.TYPE_VIEW_SCROLLED -> "TYPE_VIEW_SCROLLED"
else -> "TYPE_UNKNOWN_$eventType"
}
/**
* Concatenate [AccessibilityEvent.getText] into one display string,
* truncated to 200 chars so a runaway EditText doesn't blow the
* buffer. Returns null when the event had no text at all (we'd
* rather emit `null` than empty string so the JSON stays tight).
*/
private fun concatEventText(event: AccessibilityEvent): String? {
val parts = event.text ?: return null
if (parts.isEmpty()) return null
val joined = parts.joinToString(separator = " ") { it?.toString().orEmpty() }.trim()
if (joined.isEmpty()) return null
return if (joined.length > 200) joined.substring(0, 200) else joined
}
/**
* Amortised GC of the throttle map. Called from [append] under the
* lock. Walks the map only every [THROTTLE_GC_INTERVAL_MS] to avoid
* O(n) work per event.
*/
private fun maybeGcThrottleMap(now: Long) {
if (lastThrottleGcAt == 0L) {
lastThrottleGcAt = now
return
}
if ((now - lastThrottleGcAt) < THROTTLE_GC_INTERVAL_MS) return
lastThrottleGcAt = now
val cutoff = now - THROTTLE_WINDOW_MS
val it = throttleMap.entries.iterator()
while (it.hasNext()) {
val e = it.next()
if (e.value < cutoff) it.remove()
}
}
}
@@ -67,10 +67,21 @@ class ChannelMultiplexer {
// TODO: Phase 2 — terminal channel handler
handlers["terminal"]?.onMessage(envelope)
}
"bridge" -> {
// TODO: Phase 3 — bridge channel handler
handlers["bridge"]?.onMessage(envelope)
}
// === PHASE3-accessibility: bridge channel routing ===
// bridge.command envelopes come FROM the server and are
// dispatched to a [BridgeCommandHandler] which hands them to
// the [HermesAccessibilityService]'s [ActionExecutor]. Responses
// (bridge.response / bridge.status) flow back through [send]
// directly — the handler never consumes its own responses.
//
// This branch is intentionally symmetric with "chat" and
// "terminal": route inbound envelopes to whatever handler is
// registered. The handler registration itself happens in
// [ConnectionViewModel] so the ViewModel controls whether
// bridge routing is active (Bridge can be gated by build
// flavor or by the master enable toggle in the UI).
"bridge" -> handlers["bridge"]?.onMessage(envelope)
// === END PHASE3-accessibility ===
else -> {
// Unknown channel — ignore
}
@@ -91,6 +102,28 @@ class ChannelMultiplexer {
sendCallback?.invoke(envelope)
}
// === PHASE3-notif-listener: notification outbound routing ===
//
// `HermesNotificationCompanion` is a system-bound
// `NotificationListenerService` that lives outside the ViewModel
// scope. To push posted-notification envelopes onto the WSS
// connection, it grabs the live multiplexer reference (set by
// `ConnectionViewModel` via the static companion `multiplexer`
// slot on the service) and calls [sendNotification].
//
// This is a thin wrapper over [send] with a no-op fast path when
// no send callback is wired yet (relay disconnected). We drop on
// the floor at this layer rather than buffering — the service
// owns the cold-start buffer in its `pendingEnvelopes` queue, and
// dropping when the relay is offline matches the smartwatch
// companion semantics (a wearable doesn't replay notifications
// it missed while out of range either).
fun sendNotification(envelope: Envelope) {
val cb = sendCallback ?: return
cb.invoke(envelope)
}
// === END PHASE3-notif-listener ===
/**
* Handle system channel messages (auth, ping/pong).
*/
@@ -1,6 +1,7 @@
package com.hermesandroid.relay.network
import android.util.Log
import com.hermesandroid.relay.auth.CertPinStore
import com.hermesandroid.relay.network.models.Envelope
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
@@ -12,6 +13,7 @@ import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.launch
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
import okhttp3.CertificatePinner
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.Response
@@ -27,7 +29,37 @@ enum class ConnectionState {
}
class ConnectionManager(
private val multiplexer: ChannelMultiplexer
private val multiplexer: ChannelMultiplexer,
/**
* Optional TOFU certificate pin store. When provided, ConnectionManager
* builds its [OkHttpClient] with a snapshot of the current pins each
* connect — so subsequent wss connects refuse mismatched certs. On a
* successful `onOpen` we call back to record the peer cert fingerprint
* for the first-time TOFU case. See [CertPinStore] for the contract.
*
* Nullable for backwards-compat with unit tests that construct a bare
* ConnectionManager without auth wiring.
*/
private val certPinStore: CertPinStore? = null,
/**
* Defense-in-depth guard for the internal auto-reconnect loop. Called
* from [scheduleReconnect] both before scheduling the delayed retry and
* after the backoff delay expires — if it returns `false`, the retry is
* silently dropped.
*
* The canonical wiring is `{ authManager.hasPairContext }` so the phone
* never fires a reconnect with no session token and no pending pair
* code. Without this gate, ConnectionManager's internal retry loop
* completely bypasses [ConnectionViewModel.connectRelay]'s gate (the
* primary gate introduced in the 2026-04-11 "Option B" commit), and
* stale credentials get fed into the auth envelope after clearSession
* wipes state → relay returns "Invalid pairing code or session token"
* → rate limiter blocks the IP after 5 attempts → user can't re-pair.
*
* Defaults to always-allow for tests and legacy call sites. Production
* wiring passes the AuthManager gate from [ConnectionViewModel].
*/
private val reconnectGate: () -> Boolean = { true }
) {
private val supervisorJob = SupervisorJob()
private val scope = CoroutineScope(supervisorJob + Dispatchers.IO)
@@ -37,10 +69,26 @@ class ConnectionManager(
encodeDefaults = true
}
private val client = OkHttpClient.Builder()
.pingInterval(30, TimeUnit.SECONDS)
.readTimeout(0, TimeUnit.MILLISECONDS)
.build()
private fun buildClient(): OkHttpClient {
val builder = OkHttpClient.Builder()
.pingInterval(30, TimeUnit.SECONDS)
.readTimeout(0, TimeUnit.MILLISECONDS)
// Swap in the current pin snapshot on every connect. We DON'T hold a
// long-lived OkHttpClient with a stale pinner — otherwise a re-pair
// that wipes a pin would still be subject to the pre-wipe rules.
certPinStore?.let { store ->
try {
builder.certificatePinner(store.buildPinnerSnapshot())
} catch (e: Exception) {
Log.w(TAG, "CertificatePinner build failed: ${e.message}")
builder.certificatePinner(CertificatePinner.DEFAULT)
}
}
return builder.build()
}
@Volatile
private var client: OkHttpClient = buildClient()
private var webSocket: WebSocket? = null
private var serverUrl: String? = null
@@ -82,15 +130,37 @@ class ConnectionManager(
return
}
// Normalize: append /ws if the user gave us a bare host:port with no
// path. The relay routes the WebSocket handler at /ws; a bare URL
// hits the HTTP root and comes back as 404 Not Found during the
// upgrade handshake. We still accept an explicit path if present.
val normalized = normalizeRelayUrl(url)
_isInsecureConnection.value = isInsecure
if (isInsecure) {
Log.w(TAG, "⚠ Connecting over INSECURE ws:// to: $url")
Log.w(TAG, "⚠ Connecting over INSECURE ws:// to: $normalized")
}
serverUrl = url
serverUrl = normalized
shouldReconnect = true
reconnectAttempt = 0
doConnect(url)
doConnect(normalized)
}
private fun normalizeRelayUrl(url: String): String {
// Strip scheme to reason about the path portion cheaply.
val schemeEnd = url.indexOf("://")
if (schemeEnd < 0) return url
val afterScheme = url.substring(schemeEnd + 3)
val pathStart = afterScheme.indexOf('/')
return if (pathStart < 0) {
// No path at all — append /ws
"$url/ws"
} else {
val path = afterScheme.substring(pathStart)
// Empty or root path — append ws
if (path == "/" || path.isEmpty()) "${url.trimEnd('/')}/ws" else url
}
}
fun disconnect() {
@@ -120,14 +190,44 @@ class ConnectionManager(
ConnectionState.Connecting
}
scope.launch { doConnectInternal(url) }
}
private fun doConnectInternal(url: String) {
// Rebuild the client so the CertificatePinner picks up the current
// pin store snapshot — crucial right after applyServerIssuedCodeAndReset
// wipes a pin for re-pair. buildClient() does a tiny DataStore read
// via runBlocking, so it runs on the IO dispatcher inside [scope].
client = buildClient()
val request = Request.Builder()
.url(url)
.build()
Log.i(TAG, "doConnect: opening WSS to $url")
webSocket = client.newWebSocket(request, object : WebSocketListener() {
override fun onOpen(webSocket: WebSocket, response: Response) {
reconnectAttempt = 0
_connectionState.value = ConnectionState.Connected
Log.i(TAG, "onOpen: WSS handshake complete ($url)")
// TOFU: record the peer cert fingerprint if we don't have one
// yet. OkHttp populates response.handshake when the connection
// was upgraded over TLS; ws:// plaintext connections skip this.
certPinStore?.let { store ->
val handshake = response.handshake
val peerCerts = handshake?.peerCertificates
if (peerCerts != null && peerCerts.isNotEmpty()) {
scope.launch {
try {
store.recordPinIfAbsent(url, peerCerts)
} catch (e: Exception) {
Log.w(TAG, "recordPinIfAbsent failed: ${e.message}")
}
}
}
}
multiplexer.onConnected()
}
@@ -141,17 +241,19 @@ class ConnectionManager(
}
override fun onClosing(webSocket: WebSocket, code: Int, reason: String) {
Log.i(TAG, "onClosing: code=$code reason=$reason")
webSocket.close(code, reason)
}
override fun onClosed(webSocket: WebSocket, code: Int, reason: String) {
Log.i(TAG, "onClosed: code=$code reason=$reason")
_connectionState.value = ConnectionState.Disconnected
scheduleReconnect()
}
override fun onFailure(webSocket: WebSocket, t: Throwable, response: Response?) {
Log.w(TAG, "onFailure: ${t.javaClass.simpleName}: ${t.message} (responseCode=${response?.code})")
_connectionState.value = ConnectionState.Disconnected
t.printStackTrace()
scheduleReconnect()
}
})
@@ -159,6 +261,17 @@ class ConnectionManager(
private fun scheduleReconnect() {
if (!shouldReconnect) return
// Defense-in-depth: if auth state says we shouldn't be reconnecting
// (no session token, no pending pair code), abort before we spend
// an attempt. This catches the "clearSession wiped auth state but
// the reconnect scheduler didn't get the memo" class of bug —
// without this, we'd fire invalid-credential auth envelopes into
// the rate limiter and block ourselves.
if (!reconnectGate()) {
Log.i(TAG, "scheduleReconnect: gate says no pair context — aborting retry")
_connectionState.value = ConnectionState.Disconnected
return
}
val url = serverUrl ?: return
reconnectAttempt++
@@ -168,8 +281,14 @@ class ConnectionManager(
scope.launch {
delay(backoffMs)
if (shouldReconnect) {
// Re-check the gate after the backoff — by the time the delay
// expires, auth state may have changed (e.g., user hit Revoke
// during the retry window).
if (shouldReconnect && reconnectGate()) {
doConnect(url)
} else if (!reconnectGate()) {
Log.i(TAG, "scheduleReconnect: gate turned false during backoff — aborting retry")
_connectionState.value = ConnectionState.Disconnected
}
}
}
@@ -50,6 +50,53 @@ enum class ChatMode {
DISCONNECTED
}
/**
* Per-endpoint capability snapshot. Populated by [HermesApiClient.probeCapabilities].
*
* The Android client uses this to pick the best chat path automatically when
* `streamingEndpoint = "auto"`. The bootstrap-injected vanilla-upstream case
* is the interesting one: `sessionsApi=true` (we injected it) but
* `sessionsChatStream=false` (we deliberately didn't inject the chat
* handler — runs is better). The auto-resolver picks `runs` for chat in that
* case while still using sessions endpoints for browse/rename/delete.
*/
data class ServerCapabilities(
/** `/api/sessions` (CRUD) — true on fork, upstream-merged, OR bootstrap-injected. */
val sessionsApi: Boolean,
/** `/api/sessions/{id}/chat/stream` (SSE) — true ONLY on fork or upstream-merged. */
val sessionsChatStream: Boolean,
/** `/v1/runs` (structured-event SSE) — standard upstream chat path. */
val runs: Boolean,
/** `/v1/chat/completions` — OpenAI-compatible fallback. */
val portable: Boolean,
/** `/health` — basic reachability. */
val healthy: Boolean,
) {
/** Resolve `streamingEndpoint = "auto"` to the best concrete choice. */
fun preferredChatEndpoint(): String = when {
sessionsChatStream -> "sessions"
runs -> "runs"
else -> "sessions" // last-resort: try sessions, will surface a clear error
}
fun toChatMode(): ChatMode = when {
!healthy -> ChatMode.DISCONNECTED
sessionsApi -> ChatMode.ENHANCED_HERMES
portable || runs -> ChatMode.PORTABLE
else -> ChatMode.DISCONNECTED
}
companion object {
val DISCONNECTED = ServerCapabilities(
sessionsApi = false,
sessionsChatStream = false,
runs = false,
portable = false,
healthy = false,
)
}
}
/**
* Direct HTTP/SSE client for the Hermes API Server.
*
@@ -770,37 +817,77 @@ class HermesApiClient(
/**
* Probe the server to determine which chat API is available.
* Checks /health, then /api/sessions (enhanced), then /v1/models (portable).
* Convenience wrapper around [probeCapabilities] that collapses the
* per-endpoint result into the older 3-state ChatMode enum for callers
* that don't need the detail.
*/
suspend fun detectChatMode(): ChatMode = withContext(Dispatchers.IO) {
// 1. Basic connectivity
try {
val healthReq = authRequest("$baseUrl/health").get().build()
client.newCall(healthReq).execute().use { response ->
if (!response.isSuccessful) return@withContext ChatMode.DISCONNECTED
}
suspend fun detectChatMode(): ChatMode = probeCapabilities().toChatMode()
/**
* Probe each endpoint we care about and return a per-route capability
* snapshot. This is the source of truth for "which chat path should we
* use" — see [ServerCapabilities.preferredChatEndpoint].
*
* Probe order:
* 1. `/health` — if this fails, everything else is moot.
* 2. `HEAD /api/sessions?limit=1` — sessions CRUD (true on fork OR
* bootstrap-injected upstream).
* 3. `HEAD /api/sessions/probe/chat/stream` — chat-stream handler
* presence. The handler only accepts POST, so HEAD returns 405
* (Method Not Allowed) when the route is registered. 404 means
* the route doesn't exist at all.
* 4. `HEAD /v1/runs` — runs endpoint presence (same 405-vs-404 logic).
* 5. `HEAD /v1/models` — OpenAI-compat reachability.
*
* **Why HEAD instead of OPTIONS:** The hermes-agent gateway runs CORS
* middleware (`security_headers_middleware`) that intercepts OPTIONS
* preflight requests and returns 403 for both existing AND missing
* paths — making OPTIONS useless as a probe. HEAD bypasses the CORS
* middleware path and surfaces the actual router status (200/401/405
* for present, 404 for missing). Verified empirically against the
* production hermes-agent gateway on 2026-04-12.
*
* **Success criterion:** any HTTP response code that isn't 404 means
* the route is registered. We accept 200, 204, 401, 403, 405, 415,
* etc. as positive — even quirky middleware responses count, because
* the alternative (404) is the only signal that means "no such path."
*
* Network errors (connection refused, DNS failure, etc.) count as
* "missing" since we can't differentiate from a server-down case.
*/
suspend fun probeCapabilities(): ServerCapabilities = withContext(Dispatchers.IO) {
// 1. Health
val healthy = try {
val req = authRequest("$baseUrl/health").get().build()
client.newCall(req).execute().use { it.isSuccessful }
} catch (_: Exception) {
return@withContext ChatMode.DISCONNECTED
false
}
if (!healthy) return@withContext ServerCapabilities.DISCONNECTED
// Reusable HEAD probe — returns true if the route is registered
// (any status except 404 + network errors). Already inside the
// Dispatchers.IO context from the outer withContext, so the
// blocking OkHttp calls are safe here.
fun routeExists(path: String): Boolean = try {
val req = authRequest("$baseUrl$path").head().build()
client.newCall(req).execute().use { response -> response.code != 404 }
} catch (_: Exception) {
false
}
// 2. Try enhanced sessions API
try {
val sessionsReq = authRequest("$baseUrl/api/sessions?limit=1").get().build()
client.newCall(sessionsReq).execute().use { response ->
if (response.isSuccessful) return@withContext ChatMode.ENHANCED_HERMES
}
} catch (_: Exception) { /* fall through */ }
val sessionsApi = routeExists("/api/sessions?limit=1")
val sessionsChatStream = routeExists("/api/sessions/probe/chat/stream")
val runs = routeExists("/v1/runs")
val portable = routeExists("/v1/models")
// 3. Try OpenAI-compatible models endpoint
try {
val modelsReq = authRequest("$baseUrl/v1/models").get().build()
client.newCall(modelsReq).execute().use { response ->
if (response.isSuccessful) return@withContext ChatMode.PORTABLE
}
} catch (_: Exception) { /* fall through */ }
// Server is reachable but neither API is available
ChatMode.DISCONNECTED
ServerCapabilities(
sessionsApi = sessionsApi,
sessionsChatStream = sessionsChatStream,
runs = runs,
portable = portable,
healthy = true,
)
}
// --- Lifecycle ---
@@ -0,0 +1,683 @@
package com.hermesandroid.relay.network
import android.util.Log
import com.hermesandroid.relay.auth.PairedDeviceInfo
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import kotlinx.serialization.builtins.ListSerializer
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.jsonObject
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.RequestBody.Companion.toRequestBody
import java.io.IOException
/**
* HTTP client for the Hermes relay media endpoint.
*
* The chat SSE stream can emit tool output containing a marker of the form
* `MEDIA:hermes-relay://<opaque-token>`
* [ChatHandler][com.hermesandroid.relay.network.handlers.ChatHandler] parses
* the marker, and [ChatViewModel][com.hermesandroid.relay.viewmodel.ChatViewModel]
* calls [fetchMedia] to pull the actual bytes over plain HTTP(S). The relay
* base URL is the WSS relay URL with `ws`/`wss` swapped for `http`/`https`.
*
* Authentication reuses the relay session token (same token used to authorize
* the WSS channel). It's supplied lazily via [sessionTokenProvider] because
* the token is backed by EncryptedSharedPreferences and requires a suspend
* call on first access.
*
* This client deliberately does NOT wire into the existing [HermesApiClient]
* — that one is scoped to the Hermes API server (chat, sessions, etc.) and
* uses a separate auth token (the optional Hermes Bearer API key). The relay
* and the API server are independent services even when they're co-located.
*/
class RelayHttpClient(
private val okHttpClient: OkHttpClient,
private val relayUrlProvider: () -> String?,
private val sessionTokenProvider: suspend () -> String?
) {
companion object {
private const val TAG = "RelayHttpClient"
private val sessionsJson = Json {
ignoreUnknownKeys = true
isLenient = true
coerceInputValues = true
explicitNulls = false
}
}
/**
* The result of a successful [fetchMedia] call.
*
* @property contentType MIME type parsed from the `Content-Type` header,
* falling back to `application/octet-stream` when absent.
* @property bytes raw response body.
* @property fileName best-effort filename parsed from
* `Content-Disposition: inline; filename="..."`, or null.
*/
data class FetchedMedia(
val contentType: String,
val bytes: ByteArray,
val fileName: String?
) {
override fun equals(other: Any?): Boolean {
if (this === other) return true
if (other !is FetchedMedia) return false
return contentType == other.contentType &&
bytes.contentEquals(other.bytes) &&
fileName == other.fileName
}
override fun hashCode(): Int {
var result = contentType.hashCode()
result = 31 * result + bytes.contentHashCode()
result = 31 * result + (fileName?.hashCode() ?: 0)
return result
}
}
/**
* Fetch `GET /media/<token>` from the relay over HTTP(S). Returns a
* [Result] — success carries a [FetchedMedia], failure wraps the
* underlying exception with a human-readable message suitable for
* surfacing in the attachment's `errorMessage` field.
*/
suspend fun fetchMedia(token: String): Result<FetchedMedia> = withContext(Dispatchers.IO) {
val relayUrl = relayUrlProvider()?.trim().orEmpty()
if (relayUrl.isEmpty()) {
return@withContext Result.failure(
IllegalStateException("Relay URL not configured")
)
}
val sessionToken = sessionTokenProvider()
if (sessionToken.isNullOrBlank()) {
return@withContext Result.failure(
IllegalStateException("Relay not paired — session token missing")
)
}
val httpBase = relayUrl
.replace(Regex("^wss://", RegexOption.IGNORE_CASE), "https://")
.replace(Regex("^ws://", RegexOption.IGNORE_CASE), "http://")
.trimEnd('/')
val url = "$httpBase/media/$token"
val request = Request.Builder()
.url(url)
.get()
.header("Authorization", "Bearer $sessionToken")
.header("Accept", "*/*")
.build()
try {
okHttpClient.newCall(request).execute().use { response ->
if (!response.isSuccessful) {
val reason = when (response.code) {
401, 403 -> "Unauthorized — re-pair with the relay"
404 -> "File expired or not found on relay"
413 -> "File too large for relay"
in 500..599 -> "Relay error (HTTP ${response.code})"
else -> "HTTP ${response.code}: ${response.message.ifBlank { "request failed" }}"
}
return@withContext Result.failure(IOException(reason))
}
val contentType = response.header("Content-Type")
?.substringBefore(';')
?.trim()
?.ifBlank { null }
?: "application/octet-stream"
val fileName = parseContentDispositionFilename(
response.header("Content-Disposition")
)
val body = response.body
if (body == null) {
return@withContext Result.failure(IOException("Empty response body"))
}
val bytes = body.bytes()
Result.success(FetchedMedia(contentType, bytes, fileName))
}
} catch (e: IOException) {
Log.w(TAG, "fetchMedia failed for $token: ${e.message}")
Result.failure(e)
} catch (e: Exception) {
Log.w(TAG, "fetchMedia unexpected error for $token: ${e.message}")
Result.failure(e)
}
}
/**
* Fetch `GET /media/by-path?path=<abs>` from the relay.
*
* Used when the agent's LLM freeform-emits a bare `MEDIA:/abs/path.ext`
* marker in its response text (upstream `prompt_builder.py` explicitly
* instructs the LLM to emit this form). The relay validates the path
* against the same sandbox that `/media/register` uses — no token
* round-trip is needed because the file is identified by its absolute
* path directly.
*
* Auth is the same relay session token used by [fetchMedia]. If the
* fetch fails for any reason the returned [Result] wraps an [IOException]
* with a human-readable message suitable for [com.hermesandroid.relay.data.Attachment.errorMessage].
*
* @param path absolute path on the relay host — passed verbatim as a
* query parameter (OkHttp URL-encodes it correctly).
* @param contentTypeHint optional MIME hint. If null, the server guesses
* from the file extension via Python's [mimetypes].
*/
suspend fun fetchMediaByPath(
path: String,
contentTypeHint: String? = null,
): Result<FetchedMedia> = withContext(Dispatchers.IO) {
val relayUrl = relayUrlProvider()?.trim().orEmpty()
if (relayUrl.isEmpty()) {
return@withContext Result.failure(
IllegalStateException("Relay URL not configured")
)
}
val sessionToken = sessionTokenProvider()
if (sessionToken.isNullOrBlank()) {
return@withContext Result.failure(
IllegalStateException("Relay not paired — session token missing")
)
}
val httpBase = relayUrl
.replace(Regex("^wss://", RegexOption.IGNORE_CASE), "https://")
.replace(Regex("^ws://", RegexOption.IGNORE_CASE), "http://")
.trimEnd('/')
// Build the URL via OkHttp's HttpUrl builder so query-param encoding
// handles paths with slashes, spaces, and non-ASCII characters
// correctly. A naive string-concat would double-encode or mis-encode.
val url = try {
"$httpBase/media/by-path".toHttpUrl().newBuilder()
.addQueryParameter("path", path)
.apply {
if (contentTypeHint != null) {
addQueryParameter("content_type", contentTypeHint)
}
}
.build()
} catch (e: IllegalArgumentException) {
return@withContext Result.failure(
IOException("Invalid relay URL: ${e.message}")
)
}
val request = Request.Builder()
.url(url)
.get()
.header("Authorization", "Bearer $sessionToken")
.header("Accept", "*/*")
.build()
try {
okHttpClient.newCall(request).execute().use { response ->
if (!response.isSuccessful) {
val reason = when (response.code) {
401 -> "Unauthorized — re-pair with the relay"
403 -> "Path not allowed by relay sandbox"
404 -> "File not found on relay: $path"
400 -> "Bad request — missing path"
in 500..599 -> "Relay error (HTTP ${response.code})"
else -> "HTTP ${response.code}: ${response.message.ifBlank { "request failed" }}"
}
return@withContext Result.failure(IOException(reason))
}
val contentType = response.header("Content-Type")
?.substringBefore(';')
?.trim()
?.ifBlank { null }
?: "application/octet-stream"
val fileName = parseContentDispositionFilename(
response.header("Content-Disposition")
)
val body = response.body
if (body == null) {
return@withContext Result.failure(IOException("Empty response body"))
}
val bytes = body.bytes()
Result.success(FetchedMedia(contentType, bytes, fileName))
}
} catch (e: IOException) {
Log.w(TAG, "fetchMediaByPath failed for $path: ${e.message}")
Result.failure(IOException("Relay unreachable: ${e.message ?: "IO error"}"))
} catch (e: Exception) {
Log.w(TAG, "fetchMediaByPath unexpected error for $path: ${e.message}")
Result.failure(e)
}
}
// ------------------------------------------------------------------
// Paired-device management (2026-04-11 security overhaul)
// ------------------------------------------------------------------
//
// The sibling Python agent is adding two new relay endpoints:
// GET /sessions → list all paired devices
// DELETE /sessions/{token_prefix} → revoke a specific device
//
// Both are bearer-auth'd with the same session token we use for the
// WSS channel. These methods are *defensive* — if the server hasn't
// been updated yet, they'll come back with 404 and the UI renders an
// empty list instead of crashing. See [PairedDevicesScreen] for the
// consumer.
/**
* Fetch the list of currently-paired devices from the relay.
*
* @return [Result.success] with a list of [PairedDeviceInfo] (possibly
* empty), or [Result.failure] with a diagnostic exception. A 404
* is treated as "endpoint not implemented yet" → empty list.
*/
suspend fun listSessions(): Result<List<PairedDeviceInfo>> = withContext(Dispatchers.IO) {
val relayUrl = relayUrlProvider()?.trim().orEmpty()
if (relayUrl.isEmpty()) {
return@withContext Result.failure(
IllegalStateException("Relay URL not configured")
)
}
val sessionToken = sessionTokenProvider()
if (sessionToken.isNullOrBlank()) {
return@withContext Result.failure(
IllegalStateException("Relay not paired — session token missing")
)
}
val httpBase = relayUrl
.replace(Regex("^wss://", RegexOption.IGNORE_CASE), "https://")
.replace(Regex("^ws://", RegexOption.IGNORE_CASE), "http://")
.trimEnd('/')
val url = "$httpBase/sessions"
val request = Request.Builder()
.url(url)
.get()
.header("Authorization", "Bearer $sessionToken")
.header("Accept", "application/json")
.build()
try {
okHttpClient.newCall(request).execute().use { response ->
if (response.code == 404) {
// Server hasn't shipped the endpoint yet — degrade to
// empty list so the UI can render "No paired devices"
// without exploding.
Log.i(TAG, "listSessions: relay returned 404, endpoint not implemented")
return@withContext Result.success(emptyList())
}
if (!response.isSuccessful) {
val reason = when (response.code) {
401, 403 -> "Unauthorized — re-pair with the relay"
in 500..599 -> "Relay error (HTTP ${response.code})"
else -> "HTTP ${response.code}: ${response.message.ifBlank { "request failed" }}"
}
return@withContext Result.failure(IOException(reason))
}
val body = response.body?.string().orEmpty()
// Server shape: `{"sessions": [...]}`. Unwrap the array first —
// parsing as a bare list fails with "Expected start of array '['
// but had '{'" (exactly the crash from 2026-04-11 on-device).
val root = sessionsJson.parseToJsonElement(body).jsonObject
val arrayElement = root["sessions"]
?: return@withContext Result.failure(
IOException("Relay response missing 'sessions' field")
)
val devices = sessionsJson.decodeFromJsonElement(
ListSerializer(PairedDeviceInfo.serializer()),
arrayElement
)
Result.success(devices)
}
} catch (e: IOException) {
Log.w(TAG, "listSessions failed: ${e.message}")
Result.failure(e)
} catch (e: Exception) {
Log.w(TAG, "listSessions parse error: ${e.message}")
Result.failure(e)
}
}
/**
* Revoke a paired device by its token prefix.
*
* Token prefixes are the first N characters of the session token —
* enough to uniquely identify a device without transmitting the full
* token. The server looks up and deletes the matching record.
*
* Revoking the CURRENT device (i.e. the phone making the request) is
* valid — the caller should follow up by wiping local state and
* redirecting to the pairing screen.
*/
suspend fun revokeSession(tokenPrefix: String): Result<Unit> = withContext(Dispatchers.IO) {
val relayUrl = relayUrlProvider()?.trim().orEmpty()
if (relayUrl.isEmpty()) {
return@withContext Result.failure(
IllegalStateException("Relay URL not configured")
)
}
val sessionToken = sessionTokenProvider()
if (sessionToken.isNullOrBlank()) {
return@withContext Result.failure(
IllegalStateException("Relay not paired — session token missing")
)
}
val httpBase = relayUrl
.replace(Regex("^wss://", RegexOption.IGNORE_CASE), "https://")
.replace(Regex("^ws://", RegexOption.IGNORE_CASE), "http://")
.trimEnd('/')
val url = try {
"$httpBase/sessions/".toHttpUrl().newBuilder()
.addPathSegment(tokenPrefix)
.build()
} catch (e: IllegalArgumentException) {
return@withContext Result.failure(
IOException("Invalid relay URL: ${e.message}")
)
}
val request = Request.Builder()
.url(url)
.delete()
.header("Authorization", "Bearer $sessionToken")
.header("Accept", "application/json")
.build()
try {
okHttpClient.newCall(request).execute().use { response ->
if (response.code == 404) {
// Already gone — treat as success so the UI can just
// drop the row on the next refresh.
return@withContext Result.success(Unit)
}
if (!response.isSuccessful) {
val reason = when (response.code) {
401, 403 -> "Unauthorized — re-pair with the relay"
in 500..599 -> "Relay error (HTTP ${response.code})"
else -> "HTTP ${response.code}: ${response.message.ifBlank { "request failed" }}"
}
return@withContext Result.failure(IOException(reason))
}
Result.success(Unit)
}
} catch (e: IOException) {
Log.w(TAG, "revokeSession failed: ${e.message}")
Result.failure(e)
} catch (e: Exception) {
Log.w(TAG, "revokeSession unexpected error: ${e.message}")
Result.failure(e)
}
}
/**
* Extend (or update) a paired device's session TTL and/or per-channel
* grants.
*
* Backs the "Extend" button on the Paired Devices card. At least one
* of [ttlSeconds] / [grants] must be non-null — both null is an
* immediate `Result.failure` without hitting the network.
*
* * [ttlSeconds] — new session lifetime in seconds, `0` means never
* expire. When provided, the server restarts the clock from now
* (i.e. "extend by 30 days" = "30 days from now", not "add 30 days
* to the existing expiry"). `null` leaves session expiry alone.
* * [grants] — seconds-from-now per channel. When provided, grants
* are re-materialized and clamped to the (possibly new) session
* lifetime. `null` leaves grants alone, though they'll be re-clamped
* server-side if [ttlSeconds] was provided and shortens the session.
*
* Returns [Result.success] on HTTP 200. 404 is a hard failure here
* (unlike revoke — "already gone" is a surprise when you're trying to
* extend an active session).
*/
suspend fun extendSession(
tokenPrefix: String,
ttlSeconds: Long? = null,
grants: Map<String, Long>? = null,
): Result<Unit> = withContext(Dispatchers.IO) {
if (ttlSeconds == null && grants == null) {
return@withContext Result.failure(
IllegalArgumentException("extendSession requires at least one of ttlSeconds or grants")
)
}
val relayUrl = relayUrlProvider()?.trim().orEmpty()
if (relayUrl.isEmpty()) {
return@withContext Result.failure(
IllegalStateException("Relay URL not configured")
)
}
val sessionToken = sessionTokenProvider()
if (sessionToken.isNullOrBlank()) {
return@withContext Result.failure(
IllegalStateException("Relay not paired — session token missing")
)
}
val httpBase = relayUrl
.replace(Regex("^wss://", RegexOption.IGNORE_CASE), "https://")
.replace(Regex("^ws://", RegexOption.IGNORE_CASE), "http://")
.trimEnd('/')
val url = try {
"$httpBase/sessions/".toHttpUrl().newBuilder()
.addPathSegment(tokenPrefix)
.build()
} catch (e: IllegalArgumentException) {
return@withContext Result.failure(
IOException("Invalid relay URL: ${e.message}")
)
}
// Hand-rolling the JSON body to keep the serializer tree thin —
// kotlinx.serialization's JsonObjectBuilder would pull in another
// dependency branch. The body is 1-2 fields so the hand form is
// trivial and auditable.
val bodyJson = buildString {
append('{')
var first = true
if (ttlSeconds != null) {
append("\"ttl_seconds\":").append(ttlSeconds)
first = false
}
if (grants != null) {
if (!first) append(',')
append("\"grants\":{")
var g = true
for ((k, v) in grants) {
if (!g) append(',')
append('"').append(k.replace("\"", "\\\"")).append("\":").append(v)
g = false
}
append('}')
}
append('}')
}
val request = Request.Builder()
.url(url)
.patch(bodyJson.toRequestBody("application/json".toMediaType()))
.header("Authorization", "Bearer $sessionToken")
.header("Accept", "application/json")
.build()
try {
okHttpClient.newCall(request).execute().use { response ->
if (!response.isSuccessful) {
val reason = when (response.code) {
400 -> "Invalid extend request (check TTL/grants)"
401, 403 -> "Unauthorized — re-pair with the relay"
404 -> "Session not found — it may have expired"
409 -> "Ambiguous token prefix — retry with more chars"
in 500..599 -> "Relay error (HTTP ${response.code})"
else -> "HTTP ${response.code}: ${response.message.ifBlank { "request failed" }}"
}
return@withContext Result.failure(IOException(reason))
}
Result.success(Unit)
}
} catch (e: IOException) {
Log.w(TAG, "extendSession failed: ${e.message}")
Result.failure(e)
} catch (e: Exception) {
Log.w(TAG, "extendSession unexpected error: ${e.message}")
Result.failure(e)
}
}
/**
* Result of a [probeHealth] call.
*
* @property version the relay's reported version string (e.g. `"0.2.0"`)
* @property clients number of currently-connected WS clients
* @property sessions number of active [SessionManager] entries
*/
data class RelayHealth(
val version: String,
val clients: Int,
val sessions: Int,
)
/**
* Probe a relay URL for reachability via an unauthenticated `GET /health`.
*
* This is the **"is this URL pointing at a live relay"** check behind the
* Settings → Manual configuration → **Save & Test** button. It:
*
* * uses HTTP (converting `ws://`/`wss://` → `http://`/`https://`)
* * sends NO `Authorization` header — health is public
* * times out fast (3 seconds) so the UI doesn't hang
* * validates that the response body actually looks like a
* hermes-relay health response (`{"status": "ok", "version": ...}`)
* so a random HTTP server on port 8767 doesn't falsely pass
*
* Unlike [fetchMedia] / [listSessions], this method does NOT consult
* [relayUrlProvider] or [sessionTokenProvider] — the caller passes the
* URL to probe directly. That's deliberate: the user might be testing a
* URL they've typed into the manual-config field but haven't saved yet,
* so we can't read it back from stored settings.
*
* Returns [Result.success] with parsed metadata on a valid hermes-relay
* health response; [Result.failure] wrapping an [IOException] with a
* human-readable message on any failure (network, non-200, bad body,
* doesn't-look-like-hermes-relay).
*/
suspend fun probeHealth(relayUrl: String): Result<RelayHealth> = withContext(Dispatchers.IO) {
val trimmed = relayUrl.trim()
if (trimmed.isEmpty()) {
return@withContext Result.failure(
IllegalArgumentException("Relay URL is empty")
)
}
val httpBase = trimmed
.replace(Regex("^wss://", RegexOption.IGNORE_CASE), "https://")
.replace(Regex("^ws://", RegexOption.IGNORE_CASE), "http://")
.trimEnd('/')
val url = try {
"$httpBase/health".toHttpUrl()
} catch (e: IllegalArgumentException) {
return@withContext Result.failure(
IOException("Invalid relay URL: ${e.message}")
)
}
// Fast-timeout client — we don't want Save & Test to hang the UI
// for 10 seconds on a dead URL.
val fastClient = okHttpClient.newBuilder()
.connectTimeout(3, java.util.concurrent.TimeUnit.SECONDS)
.readTimeout(3, java.util.concurrent.TimeUnit.SECONDS)
.writeTimeout(3, java.util.concurrent.TimeUnit.SECONDS)
.build()
val request = Request.Builder()
.url(url)
.get()
.header("Accept", "application/json")
.build()
try {
fastClient.newCall(request).execute().use { response ->
if (!response.isSuccessful) {
return@withContext Result.failure(
IOException("Relay responded HTTP ${response.code}")
)
}
val body = response.body?.string().orEmpty()
if (body.isBlank()) {
return@withContext Result.failure(
IOException("Relay returned an empty response")
)
}
// Parse the JSON and verify it looks like a hermes-relay
// health response (status=ok + version field).
val parsed: Map<String, kotlinx.serialization.json.JsonElement> = try {
sessionsJson.parseToJsonElement(body).jsonObject
} catch (e: Exception) {
return@withContext Result.failure(
IOException("Relay returned non-JSON: ${e.message ?: "parse error"}")
)
}
val status = (parsed["status"] as? kotlinx.serialization.json.JsonPrimitive)?.content
if (status != "ok") {
return@withContext Result.failure(
IOException("Relay reports status=${status ?: "missing"} (expected 'ok')")
)
}
val version = (parsed["version"] as? kotlinx.serialization.json.JsonPrimitive)?.content
if (version.isNullOrBlank()) {
return@withContext Result.failure(
IOException("Response doesn't look like a hermes-relay — missing 'version' field")
)
}
val clients = (parsed["clients"] as? kotlinx.serialization.json.JsonPrimitive)
?.content?.toIntOrNull() ?: 0
val sessions = (parsed["sessions"] as? kotlinx.serialization.json.JsonPrimitive)
?.content?.toIntOrNull() ?: 0
Result.success(RelayHealth(version = version, clients = clients, sessions = sessions))
}
} catch (e: java.net.SocketTimeoutException) {
Log.w(TAG, "probeHealth timeout: ${e.message}")
Result.failure(IOException("Relay is not responding (3s timeout)"))
} catch (e: java.net.ConnectException) {
Log.w(TAG, "probeHealth connect refused: ${e.message}")
Result.failure(IOException("Connection refused — is the relay running on this URL?"))
} catch (e: IOException) {
Log.w(TAG, "probeHealth IO error: ${e.message}")
Result.failure(IOException("Network error: ${e.message ?: "unreachable"}"))
} catch (e: Exception) {
Log.w(TAG, "probeHealth unexpected error: ${e.message}")
Result.failure(e)
}
}
/**
* Extract `filename` from a `Content-Disposition` header. Handles the
* common `inline; filename="foo.png"` and `attachment; filename=foo.png`
* shapes. RFC 5987 `filename*` encoding is not supported — if the relay
* ever needs non-ASCII names it'll need extending.
*/
private fun parseContentDispositionFilename(header: String?): String? {
if (header.isNullOrBlank()) return null
val match = Regex("""filename\s*=\s*"?([^";]+)"?""", RegexOption.IGNORE_CASE).find(header)
return match?.groupValues?.get(1)?.trim()?.ifBlank { null }
}
}
@@ -0,0 +1,264 @@
package com.hermesandroid.relay.network
import android.content.Context
import android.util.Log
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.jsonObject
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.MultipartBody
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.RequestBody.Companion.asRequestBody
import okhttp3.RequestBody.Companion.toRequestBody
import java.io.File
import java.io.IOException
/**
* HTTP client for the Hermes relay **voice** endpoints (V1 contract).
*
* POST /voice/transcribe — multipart upload, returns `{"text": "..."}`
* POST /voice/synthesize — JSON `{"text": "..."}`, returns audio/mpeg bytes
*
* Auth, URL conversion (`ws` ↔ `http`), and error shape mirror
* [RelayHttpClient] — same bearer token, same `{relayUrl, sessionToken}`
* providers. We take the providers as constructor args instead of sharing
* a [RelayHttpClient] instance so the classes stay decoupled.
*/
class RelayVoiceClient(
private val context: Context,
private val okHttpClient: OkHttpClient,
private val relayUrlProvider: () -> String?,
private val sessionTokenProvider: suspend () -> String?,
) {
companion object {
private const val TAG = "RelayVoiceClient"
private val json = Json { ignoreUnknownKeys = true; isLenient = true }
private val JSON_MEDIA_TYPE = "application/json".toMediaType()
private val MP4_AUDIO = "audio/mp4".toMediaType()
}
/**
* Upload [audioFile] to `/voice/transcribe` and return the transcribed
* text. Expects a JSON response of the form
* `{"text": "...", "provider": "...", "success": true}`.
*/
suspend fun transcribe(audioFile: File): Result<String> = withContext(Dispatchers.IO) {
val httpBase = resolveHttpBase()
?: return@withContext Result.failure(IllegalStateException("Relay URL not configured"))
val token = sessionTokenProvider()
if (token.isNullOrBlank()) {
return@withContext Result.failure(
IllegalStateException("Relay not paired — session token missing")
)
}
if (!audioFile.exists() || audioFile.length() == 0L) {
return@withContext Result.failure(
IOException("Audio file missing or empty: ${audioFile.name}")
)
}
val body = MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormDataPart(
name = "audio",
filename = audioFile.name,
body = audioFile.asRequestBody(MP4_AUDIO),
)
.build()
val request = Request.Builder()
.url("$httpBase/voice/transcribe")
.post(body)
.header("Authorization", "Bearer $token")
.header("Accept", "application/json")
.build()
try {
okHttpClient.newCall(request).execute().use { response ->
if (!response.isSuccessful) {
return@withContext Result.failure(IOException(describeHttpError(response.code, response.message)))
}
val raw = response.body?.string().orEmpty()
if (raw.isBlank()) {
return@withContext Result.failure(IOException("Empty transcription response"))
}
val obj = try {
json.parseToJsonElement(raw).jsonObject
} catch (e: Exception) {
return@withContext Result.failure(IOException("Transcribe returned non-JSON: ${e.message ?: "parse error"}"))
}
val text = (obj["text"] as? JsonPrimitive)?.content
if (text.isNullOrEmpty()) {
val success = (obj["success"] as? JsonPrimitive)?.content
return@withContext Result.failure(
IOException("Transcription empty (success=$success)")
)
}
Result.success(text)
}
} catch (e: IOException) {
Log.w(TAG, "transcribe failed: ${e.message}")
Result.failure(IOException("Voice transcribe failed: ${e.message ?: "network error"}"))
} catch (e: Exception) {
Log.w(TAG, "transcribe unexpected error: ${e.message}")
Result.failure(e)
}
}
/**
* POST [text] to `/voice/synthesize`, stream the audio/mpeg response
* bytes to a temp file in [Context.getCacheDir], and return the file.
*
* The TTS endpoint is not streamed — the relay buffers the full mp3 and
* returns it in one shot. Caller is responsible for deleting the file
* when done (typical pattern: keep the last N mp3s in the cache dir and
* let the OS reclaim on cache pressure).
*/
suspend fun synthesize(text: String): Result<File> = withContext(Dispatchers.IO) {
val httpBase = resolveHttpBase()
?: return@withContext Result.failure(IllegalStateException("Relay URL not configured"))
val token = sessionTokenProvider()
if (token.isNullOrBlank()) {
return@withContext Result.failure(
IllegalStateException("Relay not paired — session token missing")
)
}
if (text.isBlank()) {
return@withContext Result.failure(IllegalArgumentException("synthesize: text is blank"))
}
// Hand-rolled JSON — one field, easier to audit than pulling in
// JsonObjectBuilder just for this call.
val escaped = text
.replace("\\", "\\\\")
.replace("\"", "\\\"")
.replace("\n", "\\n")
.replace("\r", "\\r")
.replace("\t", "\\t")
val bodyJson = "{\"text\":\"$escaped\"}"
val request = Request.Builder()
.url("$httpBase/voice/synthesize")
.post(bodyJson.toRequestBody(JSON_MEDIA_TYPE))
.header("Authorization", "Bearer $token")
.header("Accept", "audio/mpeg")
.build()
try {
okHttpClient.newCall(request).execute().use { response ->
if (!response.isSuccessful) {
return@withContext Result.failure(IOException(describeHttpError(response.code, response.message)))
}
val body = response.body
?: return@withContext Result.failure(IOException("Empty synthesize response body"))
val outFile = File(context.cacheDir, "voice_tts_${System.currentTimeMillis()}.mp3")
body.byteStream().use { input ->
outFile.outputStream().use { output ->
input.copyTo(output)
}
}
if (outFile.length() == 0L) {
outFile.delete()
return@withContext Result.failure(IOException("Synthesize returned 0 bytes"))
}
Result.success(outFile)
}
} catch (e: IOException) {
Log.w(TAG, "synthesize failed: ${e.message}")
Result.failure(IOException("Voice synthesize failed: ${e.message ?: "network error"}"))
} catch (e: Exception) {
Log.w(TAG, "synthesize unexpected error: ${e.message}")
Result.failure(e)
}
}
/**
* Probe `/voice/config` to discover which TTS/STT providers the relay
* has active. Used by VoiceSettingsScreen to show read-only provider
* labels and by VoiceViewModel to surface a "no voice providers"
* error when the backend isn't configured.
*/
suspend fun getVoiceConfig(): Result<VoiceConfig> = withContext(Dispatchers.IO) {
val httpBase = resolveHttpBase()
?: return@withContext Result.failure(IllegalStateException("Relay URL not configured"))
val token = sessionTokenProvider()
if (token.isNullOrBlank()) {
return@withContext Result.failure(
IllegalStateException("Relay not paired — session token missing")
)
}
val request = Request.Builder()
.url("$httpBase/voice/config")
.get()
.header("Authorization", "Bearer $token")
.header("Accept", "application/json")
.build()
try {
okHttpClient.newCall(request).execute().use { response ->
if (!response.isSuccessful) {
return@withContext Result.failure(IOException(describeHttpError(response.code, response.message)))
}
val raw = response.body?.string().orEmpty()
if (raw.isBlank()) {
return@withContext Result.failure(IOException("Empty voice config response"))
}
try {
Result.success(json.decodeFromString(VoiceConfig.serializer(), raw))
} catch (e: Exception) {
Result.failure(IOException("Voice config parse failed: ${e.message ?: "parse error"}"))
}
}
} catch (e: IOException) {
Log.w(TAG, "getVoiceConfig failed: ${e.message}")
Result.failure(IOException("Voice config failed: ${e.message ?: "network error"}"))
} catch (e: Exception) {
Log.w(TAG, "getVoiceConfig unexpected error: ${e.message}")
Result.failure(e)
}
}
private fun resolveHttpBase(): String? {
val relayUrl = relayUrlProvider()?.trim().orEmpty()
if (relayUrl.isEmpty()) return null
return relayUrl
.replace(Regex("^wss://", RegexOption.IGNORE_CASE), "https://")
.replace(Regex("^ws://", RegexOption.IGNORE_CASE), "http://")
.trimEnd('/')
}
private fun describeHttpError(code: Int, message: String): String = when (code) {
401, 403 -> "Unauthorized — re-pair with the relay"
404 -> "Voice endpoint not available on this relay"
413 -> "Audio too large for relay"
503 -> "Voice provider unavailable (check relay config)"
in 500..599 -> "Relay error (HTTP $code)"
else -> "HTTP $code: ${message.ifBlank { "request failed" }}"
}
}
/**
* Wire shape of `GET /voice/config`. Providers are returned as nested
* objects describing the currently-active STT and TTS backend. Extra
* fields are tolerated via `ignoreUnknownKeys = true`.
*/
@Serializable
data class VoiceConfig(
val tts: VoiceProviderInfo? = null,
val stt: VoiceProviderInfo? = null,
)
@Serializable
data class VoiceProviderInfo(
val provider: String? = null,
val model: String? = null,
val voice: String? = null,
val available: Boolean = true,
)
@@ -11,10 +11,15 @@ import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.update
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.boolean
import kotlinx.serialization.json.booleanOrNull
import kotlinx.serialization.json.contentOrNull
import kotlinx.serialization.json.jsonObject
import kotlinx.serialization.json.jsonPrimitive
/**
* Manages chat message state, session list, and streaming events.
@@ -55,6 +60,15 @@ class ChatHandler {
private val toolAnnotationVerboseFailedRegex = Regex(
"""([❌✗])\s+(?:Failed(?:\s*:\s*|\s+)|Error(?:\s*:\s*|\s+))(\w[\w\s]*)"""
)
// Inbound media markers emitted by tool results (e.g. android_screenshot).
//
// Primary form (server with relay): `MEDIA:hermes-relay://<token>` — the
// relay has the file, we fetch it over HTTP and render it inline.
// Fallback form (no relay): `MEDIA:/absolute/path` — relay wasn't
// reachable when the tool fired, so we render an "unavailable"
// placeholder instead of attempting a fetch.
private val mediaRelayRegex = Regex("""MEDIA:hermes-relay://([A-Za-z0-9_-]+)""")
private val mediaBarePathRegex = Regex("""^\s*MEDIA:(/\S+)\s*$""")
// Known completion/failure emojis — if these appear in backtick format, it's a completion
private val completionEmojis = setOf("✅", "✓", "☑")
private val failureEmojis = setOf("❌", "✗", "⚠")
@@ -66,6 +80,33 @@ class ChatHandler {
/** Active personality/agent name — set by ChatViewModel before each stream. Included on new assistant messages. */
var activeAgentName: String? = null
/**
* Fired when the text stream contains `MEDIA:hermes-relay://<token>`. The
* ViewModel should insert a LOADING [com.hermesandroid.relay.data.Attachment]
* on the matching message and kick off a relay fetch.
* Default is a no-op so tests and legacy callers don't have to wire it.
*/
var onMediaAttachmentRequested: (messageId: String, token: String) -> Unit = { _, _ -> }
/**
* Fired when the text stream contains a bare-path `MEDIA:/path` marker.
*
* The bare-path form is the PRIMARY format LLMs emit — upstream
* `hermes-agent/agent/prompt_builder.py` explicitly instructs the model
* to "include MEDIA:/absolute/path/to/file in your response" — so this
* callback fires for most inbound media, not just fallback / unavailable
* cases.
*
* The ViewModel inserts a LOADING placeholder and kicks off a direct
* fetch via `RelayHttpClient.fetchMediaByPath`, which hits the
* bearer-auth'd `/media/by-path` relay route. The route enforces the
* same path sandbox as `/media/register`. The placeholder flips to
* LOADED on success, FAILED on any fetch error (relay offline → "Relay
* unreachable", sandbox violation → "Path not allowed", missing file →
* "File not found", etc).
*/
var onMediaBarePathRequested: (messageId: String, originalPath: String) -> Unit = { _, _ -> }
/**
* Buffer for incomplete lines during streaming. Tool annotations are line-oriented
* (backtick + emoji + tool_name + backtick), so we accumulate text until we see a
@@ -74,6 +115,21 @@ class ChatHandler {
*/
private var annotationLineBuffer = StringBuilder()
/**
* Separate line buffer used exclusively for media-marker scanning.
*
* Media parsing runs unconditionally (unlike tool-annotation parsing which
* is gated behind [parseToolAnnotations]), so it needs its own buffer so
* disabling the tool-annotation path doesn't silently drop partial media
* lines that hadn't yet hit a newline.
*
* Tracks already-fired tokens per message to avoid duplicate attachments
* when a marker happens to appear in both real-time streaming and the
* post-stream finalize reconciliation pass.
*/
private var mediaLineBuffer = StringBuilder()
private val dispatchedMediaMarkers = mutableSetOf<String>()
/**
* Tracks which tool names currently have an active (in-progress) annotation-based
* ToolCall, keyed by "messageId:toolName" → toolCallId. This lets us match a
@@ -104,6 +160,254 @@ class ChatHandler {
}
}
/**
* Append a local-only voice-intent trace to the chat scroll. Used by
* the sideload voice intent flow (`RealVoiceBridgeIntentHandler`) so
* phone-control utterances like "open Chrome" or "text Sam" leave a
* visible record in chat history rather than vanishing into a side-
* channel the user can't see.
*
* Adds two messages back-to-back:
* - a user message with the raw transcribed text
* - an assistant message with the action description
*
* Both are local-only — they're injected straight into [_messages]
* without touching the server-side session, so the gateway-side LLM
* does NOT see them in its session memory. That means follow-up
* questions like "did the SMS send?" still go to the LLM with only
* the bare follow-up as context.
*
* Server-side session sync (so the LLM can reason about voice actions
* from prior turns) is a v0.4.1 follow-up — it needs either a new
* "log message without LLM round-trip" gateway endpoint or a fait-
* accompli prompt prefix scheme. See ROADMAP.md.
*
* @param userText The raw transcribed voice utterance.
* @param actionDescription Human-readable description of what the
* bridge layer did (or is about to do). Shown verbatim in the
* assistant message bubble.
*/
fun appendLocalVoiceIntentTrace(userText: String, actionDescription: String) {
val ts = System.currentTimeMillis()
val userMsg = ChatMessage(
id = "voice-intent-user-$ts",
role = MessageRole.USER,
content = userText,
timestamp = ts,
)
val assistantMsg = ChatMessage(
id = "voice-intent-action-$ts",
role = MessageRole.ASSISTANT,
content = actionDescription,
timestamp = ts + 1,
// Mark the personality so the bubble doesn't render under the
// current personality's avatar (which would imply the LLM said
// it). The chat UI's existing agentName plumbing handles the
// alternate label automatically.
agentName = "Voice action",
)
_messages.update { list ->
(list + userMsg + assistantMsg).let {
if (it.size > MAX_MESSAGES) it.drop(it.size - MAX_MESSAGES) else it
}
}
}
/**
* Append ONLY an assistant-role bubble showing the post-dispatch
* outcome of a voice intent action (e.g. "SMS sent", "user denied",
* "permission missing"). Called by [ChatViewModel.recordVoiceIntentResult]
* after the phone-side executor returns, so the user sees the actual
* result of the destructive-verb flow instead of just the pre-dispatch
* preview. ID prefix `voice-intent-result-` makes this survive
* [loadMessageHistory] reloads the same way the pre-dispatch trace
* does, and also lets [CompactTranscriptRow] in voice mode render it
* via MarkdownContent.
*
* [agentName] defaults to "Voice action" for the voice-mode origin
* (classifier → sideload handler path) but chat mode tool-call
* parity passes "Phone action" so the label reflects that the bubble
* is a structured trace of an LLM-initiated android_* tool call
* rather than a user utterance classified and dispatched locally.
*/
fun appendLocalVoiceIntentResult(
description: String,
agentName: String = "Voice action",
) {
val ts = System.currentTimeMillis()
val resultMsg = ChatMessage(
id = "voice-intent-result-$ts",
role = MessageRole.ASSISTANT,
content = description,
timestamp = ts,
agentName = agentName,
)
_messages.update { list ->
(list + resultMsg).let {
if (it.size > MAX_MESSAGES) it.drop(it.size - MAX_MESSAGES) else it
}
}
}
/**
* Reusable lenient JSON parser for tool-result previews. [Json { ... }]
* is cheap to construct but we share one instance so per-tool-completion
* parsing doesn't churn allocations.
*/
private val phoneActionResultJson = Json {
ignoreUnknownKeys = true
isLenient = true
}
/**
* Inspect a just-completed android_* tool call and, if it's an ACTION
* tool (not a read-only probe or UI micro-action), synthesize a
* [LocalDispatchResult]-shaped outcome from [resultPreview] and emit a
* structured follow-up bubble via [appendLocalVoiceIntentResult]. This
* gives chat-mode tool calls the same post-dispatch feedback the voice
* flow got in 0.4.1, so the user always sees a visible success/failure
* trace even when the LLM's narration is lossy or quiet.
*
* [isFailure] true means we're being called from [onToolCallFailed] —
* in that case [resultPreview] is the error string, not a JSON blob,
* and we skip the parse.
*/
private fun maybeEmitPhoneActionBubble(
toolName: String,
resultPreview: String?,
isFailure: Boolean,
) {
val label = labelForAndroidTool(toolName) ?: return
val synthetic = if (isFailure) {
LocalDispatchResult(
status = 500,
errorMessage = resultPreview?.ifBlank { null },
errorCode = null,
resultJson = null,
)
} else {
parseAndroidToolResult(resultPreview)
}
val description = formatPhoneActionResult(label, synthetic)
appendLocalVoiceIntentResult(description, agentName = "Phone action")
}
/**
* Map an `android_*` tool name to a short human-readable action label,
* or null if the tool is read-only / UI-micro and should NOT emit a
* result bubble.
*
* Philosophy: only meaningful action-completion states earn a bubble.
* Read probes (`android_read_screen`, `android_get_apps`) and UI
* micro-actions (`android_tap`, `android_swipe`) already render as a
* [com.hermesandroid.relay.data.ToolCall] card on the assistant
* message, so emitting a second bubble for each would just spam the
* scrollback.
*/
private fun labelForAndroidTool(toolName: String): String? {
if (!toolName.startsWith("android_")) return null
return when (toolName) {
"android_send_sms" -> "Send SMS"
"android_call" -> "Call"
"android_search_contacts" -> "Search Contacts"
"android_open_app" -> "Open App"
"android_return_to_hermes" -> "Return to Hermes"
"android_screenshot" -> "Screenshot"
"android_press_key" -> "Key Press"
"android_setup" -> "Bridge Setup"
// Read-only / UI micro-actions — intentionally skipped.
// The ToolProgressCard on the assistant bubble already
// surfaces these inline; an extra result bubble would be
// noise, not signal.
"android_read_screen",
"android_find_nodes",
"android_tap",
"android_tap_text",
"android_long_press",
"android_type",
"android_swipe",
"android_scroll",
"android_drag",
"android_wait",
"android_get_apps",
"android_current_app",
"android_describe_node",
"android_ping",
"android_clipboard_read",
"android_clipboard_write",
"android_media",
"android_screen_hash",
"android_diff_screen",
"android_events",
"android_event_stream",
"android_location",
"android_macro",
"android_send_intent",
"android_broadcast" -> null
// Unknown android_* tools still get a generic label so new
// additions don't vanish silently — the catchall in
// [formatPhoneActionResult] handles them.
else -> toolName
.removePrefix("android_")
.replace('_', ' ')
.replaceFirstChar { it.uppercase() }
}
}
/**
* Parse an `android_*` tool result JSON preview into a
* [LocalDispatchResult]-shaped struct so [formatPhoneActionResult]
* can reuse the voice-mode formatter verbatim. Plugin handlers in
* `plugin/tools/android_tool.py` return `json.dumps(data)` with
* `ok: bool` / `error: str` fields, so we just look those up.
*
* Fails open: if the preview is missing, blank, truncated, or
* otherwise unparseable we return a success result with no error
* message. A streaming tool completion should never crash chat just
* because the result preview was malformed.
*/
private fun parseAndroidToolResult(resultPreview: String?): LocalDispatchResult {
val raw = resultPreview?.trim().orEmpty()
if (raw.isEmpty() || !raw.startsWith("{")) {
return LocalDispatchResult(
status = 200,
errorMessage = null,
errorCode = null,
resultJson = null,
)
}
return try {
val obj = phoneActionResultJson.parseToJsonElement(raw).jsonObject
val ok = obj["ok"]?.jsonPrimitive?.booleanOrNull
val errorMsg = obj["error"]?.jsonPrimitive?.contentOrNull
?: obj["message"]?.jsonPrimitive?.contentOrNull
val errorCode = obj["error_code"]?.jsonPrimitive?.contentOrNull
?: obj["code"]?.jsonPrimitive?.contentOrNull
val status = when {
ok == true -> 200
ok == false -> 400
errorMsg != null -> 400
else -> 200
}
LocalDispatchResult(
status = status,
errorMessage = errorMsg,
errorCode = errorCode,
resultJson = obj,
)
} catch (e: Exception) {
Log.w(TAG, "parseAndroidToolResult: unparseable preview (${e.message})")
LocalDispatchResult(
status = 200,
errorMessage = null,
errorCode = null,
resultJson = null,
)
}
}
/**
* Add a placeholder assistant message immediately after the user sends,
* showing streaming dots before the first SSE delta arrives.
@@ -118,6 +422,12 @@ class ChatHandler {
fun clearMessages() {
_messages.value = emptyList()
// Drop any pending line buffers / dedupe state so a fresh session
// doesn't inherit leftovers from the previous one.
mediaLineBuffer.clear()
dispatchedMediaMarkers.clear()
annotationLineBuffer.clear()
activeAnnotationTools.clear()
}
/**
@@ -139,10 +449,38 @@ class ChatHandler {
_error.value = null
}
/**
* Apply [transform] to the first message matching [messageId]. Used by
* [ChatViewModel][com.hermesandroid.relay.viewmodel.ChatViewModel] to
* mutate attachment state (LOADING → LOADED / FAILED) without having
* direct access to the private [_messages] StateFlow.
*
* No-op if no message matches (e.g. it was trimmed from the rolling
* MAX_MESSAGES buffer between the callback being queued and executing).
*/
fun mutateMessage(messageId: String, transform: (ChatMessage) -> ChatMessage) {
_messages.update { messages ->
messages.map { msg ->
if (msg.id == messageId) transform(msg) else msg
}
}
}
/**
* Load message history from API response into the messages list.
* Replaces current messages with the loaded history.
* Reconstructs tool calls from assistant messages' tool_calls field.
*
* Server-persisted message content still contains raw `MEDIA:...` markers
* (the streaming-time stripping is a phone-local operation that never
* mutated server storage), so this pass re-runs the marker parser on each
* loaded assistant message: strips the marker lines from displayed content
* and re-fires [onMediaAttachmentRequested] / [onMediaBarePathRequested]
* so the ViewModel can inject attachments against the freshly-loaded
* message IDs. Without this, the session_end reload — which happens at
* every stream complete — would wipe the placeholder the streaming path
* just injected, and the raw marker text would become visible in the
* bubble. See the placeholder-flicker fix in DEVLOG 2026-04-11.
*/
fun loadMessageHistory(items: List<MessageItem>) {
// Build a map of tool result messages (role:"tool") keyed by tool_call_id
@@ -150,6 +488,11 @@ class ChatHandler {
val toolResults = items.filter { it.role == "tool" }
.associateBy { it.toolCallId }
// Accumulator for media markers we find in loaded content — fired AFTER
// the wholesale `_messages.value = ...` assignment so the ViewModel's
// mutateMessage lookups find the newly-loaded messages.
val pendingMediaHits = mutableListOf<Pair<String, MediaMarkerHit>>()
val loaded = items.mapNotNull { item ->
val role = when (item.role) {
"user" -> MessageRole.USER
@@ -169,16 +512,129 @@ class ChatHandler {
emptyList()
}
val messageId = item.id?.toString() ?: java.util.UUID.randomUUID().toString()
val rawContent = item.contentText ?: ""
// Run the media marker parser on assistant content; strip matched
// lines and queue hits for post-assignment dispatch.
val cleanedContent = if (role == MessageRole.ASSISTANT && rawContent.isNotEmpty()) {
extractMediaMarkersFromContent(messageId, rawContent, pendingMediaHits)
} else {
rawContent
}
ChatMessage(
id = item.id?.toString() ?: java.util.UUID.randomUUID().toString(),
id = messageId,
role = role,
content = item.contentText ?: "",
content = cleanedContent,
timestamp = timestampMs,
isStreaming = false,
toolCalls = toolCalls
)
}
_messages.value = if (loaded.size > MAX_MESSAGES) loaded.takeLast(MAX_MESSAGES) else loaded
// Reload swaps the entire message list — any stale dedupe entries keyed
// on pre-reload message IDs are meaningless now. Clear so the hits we
// just collected against the reloaded IDs are guaranteed to fire.
dispatchedMediaMarkers.clear()
// === PHASE3-voice-intents-chathistory ===
// Preserve local-only voice-intent trace messages across a reload.
// These messages are injected by [appendLocalVoiceIntentTrace] with
// IDs prefixed "voice-intent-" and never reach the server-side
// session, so a wholesale `_messages.value = loaded` assignment
// would wipe them. Bailey hit this 2026-04-15: voice fall-through
// ("proceed" → not a recognized intent → chat.sendMessage) triggered
// a history reload on stream complete and the previous voice trace
// vanished, making it look like "the chat cleared". Server-side
// sync (so these traces reach the LLM's session memory too) is
// still a v0.4.1 follow-up, but preserving them client-side is
// enough to fix the disappearing-scrollback bug today.
val preservedVoiceTraces = _messages.value.filter {
it.id.startsWith("voice-intent-")
}
val merged = if (preservedVoiceTraces.isEmpty()) {
loaded
} else {
// Merge by timestamp so voice traces interleave with the
// reloaded server messages in chronological order. The voice
// trace IDs carry `System.currentTimeMillis()` in their suffix
// (see appendLocalVoiceIntentTrace), so ChatMessage.timestamp
// is the source of truth here.
(loaded + preservedVoiceTraces).sortedBy { it.timestamp }
}
_messages.value = if (merged.size > MAX_MESSAGES) merged.takeLast(MAX_MESSAGES) else merged
// === END PHASE3-voice-intents-chathistory ===
// Now that the reloaded messages are in state, fire callbacks so the
// ViewModel can insert LOADING/FAILED attachments via mutateMessage.
for ((messageId, hit) in pendingMediaHits) {
when (hit) {
is MediaMarkerHit.RelayToken -> {
val dedupeKey = "$messageId:relay:${hit.token}"
if (dispatchedMediaMarkers.add(dedupeKey)) {
Log.d(TAG, "Media marker (relay, reload): token=${hit.token}")
onMediaAttachmentRequested(messageId, hit.token)
}
}
is MediaMarkerHit.BarePath -> {
val dedupeKey = "$messageId:bare:${hit.path}"
if (dispatchedMediaMarkers.add(dedupeKey)) {
Log.d(TAG, "Media marker (bare-path, reload): ${hit.path}")
onMediaBarePathRequested(messageId, hit.path)
}
}
}
}
}
/**
* Marker hit collected during [loadMessageHistory] for post-assignment dispatch.
*/
private sealed interface MediaMarkerHit {
data class RelayToken(val token: String) : MediaMarkerHit
data class BarePath(val path: String) : MediaMarkerHit
}
/**
* Scan loaded (non-streaming) message content line-by-line for media
* markers, append hits to [out], and return the content with matched
* lines removed. Pure function — does NOT mutate [_messages] or fire
* callbacks. Called from [loadMessageHistory].
*/
private fun extractMediaMarkersFromContent(
messageId: String,
content: String,
out: MutableList<Pair<String, MediaMarkerHit>>,
): String {
var cleaned = content
for (rawLine in content.lines()) {
val trimmed = rawLine.trim()
if (trimmed.isEmpty()) continue
val relayMatch = mediaRelayRegex.find(trimmed)
if (relayMatch != null) {
out.add(messageId to MediaMarkerHit.RelayToken(relayMatch.groupValues[1]))
cleaned = cleaned
.replace("\n$rawLine\n", "\n")
.replace("\n$rawLine", "")
.replace("$rawLine\n", "")
.replace(rawLine, "")
continue
}
val bareMatch = mediaBarePathRegex.find(trimmed)
if (bareMatch != null) {
out.add(messageId to MediaMarkerHit.BarePath(bareMatch.groupValues[1]))
cleaned = cleaned
.replace("\n$rawLine\n", "\n")
.replace("\n$rawLine", "")
.replace("$rawLine\n", "")
.replace(rawLine, "")
}
}
return cleaned.trim()
}
/**
@@ -332,6 +788,10 @@ class ChatHandler {
if (parseToolAnnotations) {
scanForToolAnnotations(messageId, processedDelta)
}
// Always scan for media markers — inbound attachments are a first-class
// feature and shouldn't be gated behind the tool-annotation flag.
scanForMediaMarkers(messageId, processedDelta)
}
/**
@@ -440,6 +900,74 @@ class ChatHandler {
}
}
/**
* Accumulate incoming text in a dedicated media line buffer and scan
* completed lines for media markers. Runs independently of the
* tool-annotation pipeline so inbound files always render, regardless of
* whether the user has enabled `parseToolAnnotations`.
*
* Matched markers fire [onMediaAttachmentRequested] / [onMediaBarePathRequested]
* and the raw line is stripped from the visible message content (via
* [stripLineFromContent]) so the user sees the rendered attachment card
* instead of the literal `MEDIA:...` text.
*/
private fun scanForMediaMarkers(messageId: String, delta: String) {
mediaLineBuffer.append(delta)
while (true) {
val newlineIndex = mediaLineBuffer.indexOf('\n')
if (newlineIndex == -1) break
val line = mediaLineBuffer.substring(0, newlineIndex)
mediaLineBuffer.delete(0, newlineIndex + 1)
val trimmed = line.trim()
if (trimmed.isEmpty()) continue
if (tryDispatchMediaMarker(messageId, trimmed)) {
stripLineFromContent(messageId, trimmed)
}
}
}
/**
* Inspect [line] for a media marker and dispatch the appropriate callback.
*
* - `MEDIA:hermes-relay://<token>` → [onMediaAttachmentRequested]
* - bare `MEDIA:/path` (the whole line, no other content) →
* [onMediaBarePathRequested]
*
* De-dupes via [dispatchedMediaMarkers] so the same token doesn't fire
* twice (e.g. once during streaming and again during finalize).
*
* Returns true when a marker was matched so the caller can strip the line.
*/
private fun tryDispatchMediaMarker(messageId: String, line: String): Boolean {
val relayMatch = mediaRelayRegex.find(line)
if (relayMatch != null) {
val token = relayMatch.groupValues[1]
val dedupeKey = "$messageId:relay:$token"
if (dispatchedMediaMarkers.add(dedupeKey)) {
Log.d(TAG, "Media marker (relay): token=$token")
onMediaAttachmentRequested(messageId, token)
}
return true
}
val bareMatch = mediaBarePathRegex.find(line)
if (bareMatch != null) {
val path = bareMatch.groupValues[1]
val dedupeKey = "$messageId:bare:$path"
if (dispatchedMediaMarkers.add(dedupeKey)) {
Log.d(TAG, "Media marker (bare-path, unavailable): $path")
onMediaBarePathRequested(messageId, path)
}
return true
}
return false
}
/**
* Remove a matched annotation line from the message's displayed content.
* This prevents the raw annotation text (e.g., `💻 terminal`) from showing
@@ -483,6 +1011,13 @@ class ChatHandler {
private fun parseAnnotationLine(messageId: String, line: String): Boolean {
if (line.isEmpty()) return false
// --- Media markers (run first so they're always detected, even when
// parseToolAnnotations is off — the scanForMediaMarkers path handles
// the streaming case, this branch handles finalize reconciliation) ---
if (tryDispatchMediaMarker(messageId, line)) {
return true
}
// --- Format 1: Backtick-wrapped `<emoji> <tool_name>` ---
toolAnnotationBacktickRegex.find(line)?.let { match ->
val emojiToken = match.groupValues[1]
@@ -557,6 +1092,50 @@ class ChatHandler {
return false
}
/**
* Flush any remaining partial line in the media buffer and re-scan the
* final message content for any media markers that survived real-time
* stripping. Called unconditionally from [onStreamComplete] and
* [onTurnComplete] — inbound media is a first-class feature regardless
* of whether tool-annotation parsing is enabled.
*/
private fun finalizeMediaMarkers(messageId: String) {
// Drain any partial line (last media marker without trailing newline).
if (mediaLineBuffer.isNotEmpty()) {
val remaining = mediaLineBuffer.toString().trim()
mediaLineBuffer.clear()
if (remaining.isNotEmpty() && tryDispatchMediaMarker(messageId, remaining)) {
stripLineFromContent(messageId, remaining)
}
}
// Post-stream reconciliation: re-scan the final content for markers
// that raced with stripLineFromContent during streaming.
_messages.update { messages ->
messages.map { msg ->
if (msg.id != messageId || msg.role != MessageRole.ASSISTANT) return@map msg
var cleaned = msg.content
var changed = false
for (rawLine in msg.content.lines()) {
val trimmed = rawLine.trim()
if (trimmed.isEmpty()) continue
if (mediaRelayRegex.containsMatchIn(trimmed) ||
mediaBarePathRegex.containsMatchIn(trimmed)
) {
tryDispatchMediaMarker(messageId, trimmed)
cleaned = cleaned
.replace("\n$rawLine\n", "\n")
.replace("\n$rawLine", "")
.replace("$rawLine\n", "")
.replace(rawLine, "")
changed = true
}
}
if (changed) msg.copy(content = cleaned.trim()) else msg
}
}
}
/**
* Flush any remaining partial line in the annotation buffer.
* Called when the stream ends so we don't miss annotations that
@@ -702,6 +1281,14 @@ class ChatHandler {
}
fun onToolCallComplete(messageId: String, toolCallId: String, resultPreview: String? = null) {
// Snapshot the matching tool call's name BEFORE mutating — we need it
// to decide whether to emit a phone-action result bubble below.
val toolName = _messages.value
.firstOrNull { it.id == messageId && it.role == MessageRole.ASSISTANT }
?.toolCalls
?.firstOrNull { it.id == toolCallId }
?.name
_messages.update { messages ->
messages.map { msg ->
if (msg.id == messageId && msg.role == MessageRole.ASSISTANT) {
@@ -723,9 +1310,28 @@ class ChatHandler {
}
}
}
// Chat parity: emit a structured follow-up bubble for android_*
// action tools the LLM called, mirroring the voice-mode post-
// dispatch feedback path. Only fires for ACTION tools — read-only
// and UI micro-actions are skipped (see [labelForAndroidTool]) so
// chat parity doesn't spam the scrollback with `android_read_screen`
// or `android_tap` bubbles that the existing ToolProgressCard
// already surfaces inline on the assistant message.
if (toolName != null) {
maybeEmitPhoneActionBubble(toolName, resultPreview, isFailure = false)
}
}
fun onToolCallFailed(messageId: String, toolCallId: String, error: String?) {
// Snapshot tool name before the update so the phone-action bubble
// can label the failure correctly.
val toolName = _messages.value
.firstOrNull { it.id == messageId && it.role == MessageRole.ASSISTANT }
?.toolCalls
?.firstOrNull { it.id == toolCallId }
?.name
_messages.update { messages ->
messages.map { msg ->
if (msg.id == messageId && msg.role == MessageRole.ASSISTANT) {
@@ -747,6 +1353,14 @@ class ChatHandler {
}
}
}
if (toolName != null) {
maybeEmitPhoneActionBubble(
toolName = toolName,
resultPreview = error,
isFailure = true,
)
}
}
/**
@@ -775,6 +1389,9 @@ class ChatHandler {
if (parseToolAnnotations) {
finalizeAnnotations(messageId)
}
// Finalize media markers unconditionally (not gated by parseToolAnnotations)
finalizeMediaMarkers(messageId)
// Note: do NOT set _isStreaming to false — the run is still active
}
@@ -806,6 +1423,9 @@ class ChatHandler {
if (parseToolAnnotations) {
finalizeAnnotations(messageId)
}
// Finalize media markers unconditionally
finalizeMediaMarkers(messageId)
}
fun onStreamError(message: String) {
@@ -873,3 +1493,60 @@ class ChatHandler {
_lastSentMessage.value = text
}
}
/**
* Markdown-formatted description of a [LocalDispatchResult] for rendering
* in a chat bubble. Shared between the voice post-dispatch feedback path
* ([com.hermesandroid.relay.viewmodel.ChatViewModel.recordVoiceIntentResult])
* and the chat-mode android_* tool-completion path ([ChatHandler.onToolCallComplete])
* so both origins render identical-looking bubbles for identical outcomes.
*
* The label parameter is the short human-readable action name
* ("Send SMS", "Open App", "Call", etc). Error-code branches mirror the
* `error_code` strings [com.hermesandroid.relay.network.handlers.BridgeCommandHandler]
* emits on destructive-verb rejections.
*/
internal fun formatPhoneActionResult(
label: String,
result: LocalDispatchResult,
): String = when {
result.isSuccess -> when (label) {
"Send SMS" -> "**$label — sent** ✓"
"Open App" -> "**$label — opened** ✓"
"Tap" -> "**$label — done** ✓"
"Navigate back" -> "**$label — done** ✓"
"Home" -> "**$label — done** ✓"
else -> "**$label — complete** ✓"
}
result.errorCode == "user_denied" -> buildString {
append("**$label — cancelled by you**")
}
result.errorCode == "bridge_disabled" -> buildString {
append("**$label — agent control is off**")
append('\n')
append("Enable Agent Control in the Hermes Bridge tab to retry.")
}
result.errorCode == "permission_denied" -> buildString {
append("**$label — permission needed**")
append('\n')
append(result.errorMessage ?: "The phone is missing a required runtime permission.")
}
result.errorCode == "service_unavailable" -> buildString {
append("**$label — bridge offline**")
append('\n')
append("The accessibility service isn't connected. Enable Hermes accessibility in Settings.")
}
result.errorCode == "cancelled" -> buildString {
append("**$label — cancelled before dispatch**")
}
result.errorMessage != null -> buildString {
append("**$label — failed**")
append('\n')
append(result.errorMessage)
}
else -> buildString {
append("**$label — failed**")
append('\n')
append("Status ${result.status}.")
}
}
@@ -0,0 +1,203 @@
package com.hermesandroid.relay.notifications
import android.content.ComponentName
import android.content.Context
import android.provider.Settings
import android.service.notification.NotificationListenerService
import android.service.notification.StatusBarNotification
import android.util.Log
import com.hermesandroid.relay.network.ChannelMultiplexer
import com.hermesandroid.relay.network.models.Envelope
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.encodeToJsonElement
import java.util.concurrent.ConcurrentLinkedQueue
/**
* Opt-in [NotificationListenerService] that forwards posted-notification
* metadata to the user's paired Hermes assistant over the existing WSS
* connection.
*
* This is the same Android API used by Wear OS, Android Auto, Tasker,
* and every smart-watch companion app — it's part of the public SDK
* and the user grants/revokes access via Android's system "Notification
* access" page (`Settings.ACTION_NOTIFICATION_LISTENER_SETTINGS`).
* The user can revoke at any time and Android shows the running
* listener in their system permissions list.
*
* Wire-up:
* 1. The user opens Settings → Notification companion in the app
* 2. They tap "Open Android Settings" and toggle Hermes-Relay on
* 3. Android binds this service automatically
* 4. [onListenerConnected] sets the static [active] reference
* 5. [onNotificationPosted] builds an [Envelope] and pushes it to
* the [ChannelMultiplexer] (set by `ConnectionViewModel` once
* the relay handshake completes)
* 6. Multiplexer hands the envelope to ConnectionManager → WSS → relay
*
* Pattern: a static `companion object` reference to the bound service
* instance + a static [multiplexer] slot that the ViewModel injects.
* This is the standard Android service-to-app handoff (matches
* `HermesAccessibilityService` in agent accessibility's worktree).
*/
class HermesNotificationCompanion : NotificationListenerService() {
/**
* Buffer for entries that arrive before [multiplexer] has been
* wired up (e.g. notifications during app cold-start). Bounded so
* we don't OOM if the multiplexer is never set. Drained on the
* next [onNotificationPosted] call once a multiplexer is present.
*/
private val pendingEnvelopes = ConcurrentLinkedQueue<Envelope>()
override fun onListenerConnected() {
super.onListenerConnected()
active = this
Log.i(TAG, "NotificationListener bound — companion is live")
}
override fun onListenerDisconnected() {
super.onListenerDisconnected()
if (active === this) {
active = null
}
Log.i(TAG, "NotificationListener disconnected")
}
override fun onDestroy() {
if (active === this) {
active = null
}
super.onDestroy()
}
override fun onNotificationPosted(sbn: StatusBarNotification?) {
if (sbn == null) return
val entry = sbn.toEntry() ?: return
val envelope = entry.toEnvelope()
// Drain any backlog first so order is preserved.
val mux = multiplexer
if (mux == null) {
pendingEnvelopes.offer(envelope)
// Cap the buffer at a sensible size — drop oldest on overflow.
while (pendingEnvelopes.size > MAX_PENDING) {
pendingEnvelopes.poll()
}
Log.d(
TAG,
"Buffered notification (no multiplexer yet, pending=${pendingEnvelopes.size})",
)
return
}
// Drain pending first (in arrival order) then send the new one.
while (true) {
val next = pendingEnvelopes.poll() ?: break
try {
mux.sendNotification(next)
} catch (t: Throwable) {
Log.w(TAG, "Failed to send buffered notification: ${t.message}")
}
}
try {
mux.sendNotification(envelope)
} catch (t: Throwable) {
Log.w(TAG, "Failed to send notification: ${t.message}")
}
}
/**
* We don't act on notification removal — the cache on the relay is
* append-only with LRU eviction. Could be added later if the LLM
* needs to know "this one was dismissed".
*/
override fun onNotificationRemoved(sbn: StatusBarNotification?) {
// Intentionally no-op for now.
}
// ── Helpers ───────────────────────────────────────────────────────
private fun StatusBarNotification.toEntry(): NotificationEntry? {
val n = notification ?: return null
val extras = n.extras ?: return null
val title = extras.getCharSequence(android.app.Notification.EXTRA_TITLE)?.toString()
val text = extras.getCharSequence(android.app.Notification.EXTRA_TEXT)?.toString()
val sub = extras.getCharSequence(android.app.Notification.EXTRA_SUB_TEXT)?.toString()
// Skip notifications with no human-readable content — they're
// usually background sync placeholders that just confuse the LLM.
if (title.isNullOrBlank() && text.isNullOrBlank()) return null
return NotificationEntry(
packageName = packageName,
title = title,
text = text,
subText = sub,
postedAt = postTime,
key = key,
)
}
private fun NotificationEntry.toEnvelope(): Envelope {
val payload = JSON.encodeToJsonElement(NotificationEntry.serializer(), this) as JsonObject
return Envelope(
channel = "notifications",
type = "notification.posted",
payload = payload,
)
}
companion object {
private const val TAG = "HermesNotifCompanion"
private const val MAX_PENDING = 50
private val JSON = Json {
encodeDefaults = true
ignoreUnknownKeys = true
}
/**
* The currently bound service instance, or null if the user
* has not granted notification access (or has revoked it).
* Set in [onListenerConnected]. Use [isAccessGranted] to check
* the system permission state without depending on this flag,
* since the service may not have bound yet right after the
* user grants access.
*/
@Volatile
var active: HermesNotificationCompanion? = null
private set
/**
* Multiplexer reference injected by `ConnectionViewModel` once
* the relay handshake completes. Service buffers envelopes
* until this is set so notifications that arrive during cold
* start aren't dropped.
*/
@Volatile
var multiplexer: ChannelMultiplexer? = null
/**
* True if the user has granted notification-access permission
* to this app in Android Settings. Cheap synchronous check
* against `enabled_notification_listeners` — safe to call from
* Compose recomposition.
*/
fun isAccessGranted(context: Context): Boolean {
val pkg = context.packageName
val flat = Settings.Secure.getString(
context.contentResolver,
"enabled_notification_listeners",
) ?: return false
val expected = ComponentName(
pkg,
HermesNotificationCompanion::class.java.name,
).flattenToString()
// The flat string is a colon-separated list of component names.
return flat.split(':').any { it.equals(expected, ignoreCase = true) }
}
}
}
@@ -0,0 +1,32 @@
package com.hermesandroid.relay.notifications
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
/**
* Wire model for a single notification entry forwarded by
* [HermesNotificationCompanion] over the WSS connection.
*
* Mirrors the Python-side payload shape that
* `plugin/relay/channels/notifications.py::NotificationsChannel`
* caches in its bounded deque, so the relay can deserialize without
* any per-field translation.
*
* The fields are intentionally minimal — no icon, no big-text expansion,
* no actions — because the smartwatch-companion use case is "tell me
* what came in" not "let me interact with it from my LLM". Adding
* fields later is a non-breaking change as long as Python's deque
* stays a `dict[str, Any]`.
*/
@Serializable
data class NotificationEntry(
@SerialName("package_name")
val packageName: String,
val title: String? = null,
val text: String? = null,
@SerialName("sub_text")
val subText: String? = null,
@SerialName("posted_at")
val postedAt: Long,
val key: String,
)
@@ -0,0 +1,176 @@
package com.hermesandroid.relay.power
import android.content.Context
import android.os.PowerManager
import android.util.Log
/**
* A8 — Wake-scope wrapper for bridge gesture dispatch.
*
* ## Why this exists
*
* When the phone is idle (screen dimmed or the CPU has drifted into a
* low-power state), `AccessibilityService.dispatchGesture` and
* `ACTION_SET_TEXT` sometimes silently fail or land after a multi-second
* lag — the gesture's `GestureResultCallback` never fires because the
* looper stalls. The symptom on-device is a bridge command that "worked"
* over the wire but produced no visible effect.
*
* Wrapping the gesture-dispatching entry points of [com.hermesandroid.relay.accessibility.ActionExecutor]
* in [wakeForAction] holds a wake lock just long enough for the gesture
* to be issued, dispatched, and its completion callback delivered, then
* releases. Ref-counted so nested calls (e.g. `tapText` → `tap`) don't
* release each other's lock prematurely.
*
* ## Wake-lock flag choice
*
* We use [PowerManager.PARTIAL_WAKE_LOCK]. The classic "screen stays on"
* flags (`SCREEN_BRIGHT_WAKE_LOCK`, `FULL_WAKE_LOCK`) have been deprecated
* since API 17 — Google's guidance is that anything needing the screen
* awake should use `Window.FLAG_KEEP_SCREEN_ON` or
* `Activity.setTurnScreenOn(true)` / `KeyguardManager.requestDismissKeyguard`
* from a visible surface. We are in a background service with no window,
* so those APIs don't apply — but we also don't need the screen bright;
* we just need the CPU to stay scheduled long enough for the gesture to
* land and its callback to fire. `PARTIAL_WAKE_LOCK` does exactly that
* and is the only non-deprecated wake-lock flag remaining under our
* minSdk = 26 / targetSdk = 35 constraints.
*
* Note: this does **not** wake a fully-off screen. If the device is
* genuinely locked, the gesture will still no-op against the lock screen
* UI — that's a `KeyguardManager` / foreground-activity problem, not a
* wake-lock problem, and is intentionally out of scope for this unit.
*
* ## Safety rails
*
* - **Ref-counted** under a `synchronized` block so nested calls share
* one physical wake lock.
* - **10-second hard timeout** on the underlying [PowerManager.WakeLock]
* so a crashed or long-stalled gesture can never pin the CPU awake.
* Even a bridge command that somehow hangs for minutes will see the
* lock self-release after 10s.
* - **`try { block() } finally { release }`** so throwing gesture code
* still releases the lock.
* - **Non-reference-counted underlying [PowerManager.WakeLock]** — we
* manage the count ourselves in [lockCount] rather than letting
* `WakeLock.acquire/release` do it, because PowerManager's ref-count
* is a process-global footgun that can outlive our suspend frames on
* coroutine cancellation.
*/
object WakeLockManager {
private const val TAG = "WakeLockManager"
private const val WAKE_LOCK_TAG = "HermesRelay::BridgeAction"
/**
* Hard cap — the lock will self-release after this many ms even if
* the block is still running. Long-held wake locks are a classic
* battery-drain bug; gesture dispatch should never take this long.
*/
private const val TIMEOUT_MS: Long = 10_000L
@Volatile
private var wakeLock: PowerManager.WakeLock? = null
private val countLock = Any()
private var lockCount: Int = 0
/**
* One-shot initializer. Call from `Application.onCreate` with the
* application context so we can build the underlying wake lock
* without leaking an Activity. Idempotent — subsequent calls are
* no-ops.
*/
fun initialize(context: Context) {
// L3 fix: previously the `if (wakeLock != null) return` check and
// the subsequent assignment were not synchronized, so two concurrent
// initialize() calls could each create their own WakeLock and the
// second would overwrite the first. Benign in practice (only called
// from Application.onCreate which is main-thread) but trivial to
// harden by reusing the existing countLock.
synchronized(countLock) {
if (wakeLock != null) return
val pm = context.applicationContext.getSystemService(Context.POWER_SERVICE) as? PowerManager
if (pm == null) {
Log.w(TAG, "PowerManager unavailable — WakeLockManager will no-op")
return
}
wakeLock = pm.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, WAKE_LOCK_TAG).apply {
// We manage ref-counting ourselves; setReferenceCounted(false)
// means release() always fully releases, regardless of how many
// acquire() calls preceded it.
setReferenceCounted(false)
}
}
}
/**
* Run [block] with a partial wake lock held. Ref-counted so nested
* calls (e.g. `tapText` inside `tap`) share one physical lock. The
* lock is released in a `finally` so exceptions still clean up.
*
* If [initialize] was never called (e.g. the manager was never
* wired up in `Application.onCreate`), this falls through to simply
* running [block] with no wake lock held — the safer failure mode
* is "bridge works but may glitch when idle", not "bridge crashes".
*/
suspend fun <T> wakeForAction(block: suspend () -> T): T {
val acquired = tryAcquire()
return try {
block()
} finally {
if (acquired) {
tryRelease()
}
}
}
/**
* Acquire the wake lock if this is the outermost call. Returns true
* if the caller is responsible for releasing (i.e. they bumped the
* ref count — this matters for the finally block in [wakeForAction]).
*/
private fun tryAcquire(): Boolean {
val lock = wakeLock ?: return false
synchronized(countLock) {
if (lockCount == 0) {
try {
lock.acquire(TIMEOUT_MS)
} catch (t: Throwable) {
Log.w(TAG, "wakeLock.acquire threw: ${t.message}")
return false
}
}
lockCount += 1
return true
}
}
/**
* Decrement the ref count and release the underlying lock when the
* last waiter drops off. Tolerant of already-timed-out locks —
* `isHeld` check avoids an `IllegalStateException` on release of a
* lock that the 10-second timeout already reaped.
*/
private fun tryRelease() {
val lock = wakeLock ?: return
synchronized(countLock) {
if (lockCount <= 0) {
// Shouldn't happen, but if it does, don't let an
// underflow wedge the counter.
lockCount = 0
return
}
lockCount -= 1
if (lockCount == 0) {
try {
if (lock.isHeld) {
lock.release()
}
} catch (t: Throwable) {
Log.w(TAG, "wakeLock.release threw: ${t.message}")
}
}
}
}
}
@@ -29,14 +29,23 @@ import androidx.compose.material3.NavigationBar
import androidx.compose.material3.NavigationBarItem
import androidx.compose.material3.NavigationBarItemDefaults
import androidx.compose.material3.Scaffold
import androidx.compose.material3.SnackbarDuration
import androidx.compose.material3.SnackbarHost
import androidx.compose.material3.SnackbarHostState
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.runtime.staticCompositionLocalOf
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.compose.LocalLifecycleOwner
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.vector.ImageVector
@@ -52,15 +61,53 @@ import androidx.navigation.compose.currentBackStackEntryAsState
import androidx.navigation.compose.rememberNavController
import com.hermesandroid.relay.ui.components.MorphingSphere
import com.hermesandroid.relay.ui.components.WhatsNewDialog
import com.hermesandroid.relay.util.HumanError
import kotlinx.coroutines.delay
import com.hermesandroid.relay.ui.onboarding.OnboardingScreen
import com.hermesandroid.relay.ui.screens.AboutScreen
import com.hermesandroid.relay.ui.screens.AnalyticsScreen
import com.hermesandroid.relay.ui.screens.AppearanceSettingsScreen
import com.hermesandroid.relay.ui.screens.BridgeScreen
// === PHASE3-safety-rails: bridge safety route ===
import com.hermesandroid.relay.ui.screens.BridgeSafetySettingsScreen
// === END PHASE3-safety-rails ===
import com.hermesandroid.relay.ui.screens.ChatScreen
import com.hermesandroid.relay.ui.screens.ChatSettingsScreen
import com.hermesandroid.relay.ui.screens.ConnectionSettingsScreen
import com.hermesandroid.relay.ui.screens.DeveloperSettingsScreen
import com.hermesandroid.relay.ui.screens.MediaSettingsScreen
import com.hermesandroid.relay.ui.screens.PairedDevicesScreen
import com.hermesandroid.relay.ui.screens.SettingsScreen
import com.hermesandroid.relay.ui.screens.TerminalScreen
import com.hermesandroid.relay.ui.screens.NotificationCompanionSettingsScreen
import com.hermesandroid.relay.ui.screens.VoiceSettingsScreen
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import com.hermesandroid.relay.viewmodel.ChatViewModel
import com.hermesandroid.relay.viewmodel.ConnectionViewModel
import com.hermesandroid.relay.viewmodel.TerminalViewModel
import com.hermesandroid.relay.viewmodel.VoiceViewModel
import com.hermesandroid.relay.audio.VoicePlayer
import com.hermesandroid.relay.audio.VoiceRecorder
import com.hermesandroid.relay.audio.VoiceSfxPlayer
import com.hermesandroid.relay.network.RelayVoiceClient
import com.hermesandroid.relay.auth.AuthState
import androidx.lifecycle.viewModelScope
// Global snackbar host so any screen can surface a HumanError without
// plumbing a host through every ViewModel. Provided by RelayApp below.
val LocalSnackbarHost = staticCompositionLocalOf<SnackbarHostState> {
error("LocalSnackbarHost not provided — wrap your UI in RelayApp's CompositionLocalProvider")
}
// Short-lived snackbar by default; retryable errors get Long so users have
// time to tap the action before it auto-dismisses.
suspend fun SnackbarHostState.showHumanError(err: HumanError) {
showSnackbar(
message = err.body,
actionLabel = err.actionLabel,
duration = if (err.retryable) SnackbarDuration.Long else SnackbarDuration.Short,
)
}
sealed class Screen(
val route: String,
@@ -72,6 +119,33 @@ sealed class Screen(
data object Terminal : Screen("terminal", "Terminal", Icons.Filled.Code)
data object Bridge : Screen("bridge", "Bridge", Icons.Filled.PhoneAndroid)
data object Settings : Screen("settings", "Settings", Icons.Filled.Settings)
// Non-bottom-nav destinations — reached by explicit navigation, not the
// NavigationBar. Paired Devices is opened from Settings → Connection.
data object PairedDevices : Screen("paired_devices", "Paired Devices", Icons.Filled.Settings)
// Full-screen pair wizard route. Replaces the old in-Settings Dialog
// launch so the chooser + Confirm + Verify steps + the camera viewport
// get a real fullscreen surface (the Dialog wasn't actually filling the
// window — Settings cards were leaking through behind it).
data object Pair : Screen("pair", "Pair", Icons.Filled.Settings)
data object VoiceSettings : Screen("voice_settings", "Voice", Icons.Filled.Settings)
// === PHASE3-notif-listener-followup ===
data object NotificationCompanionSettings :
Screen("settings/notifications", "Notification companion", Icons.Filled.Settings)
// === END PHASE3-notif-listener-followup ===
// === PHASE3-safety-rails: bridge safety route ===
data object BridgeSafetySettings :
Screen("settings/bridge_safety", "Bridge safety", Icons.Filled.Settings)
// === END PHASE3-safety-rails ===
// Per-category settings sub-screens — split out of the mega SettingsScreen
// following the VoiceSettingsScreen pattern (see DEVLOG 2026-04-11).
data object ConnectionSettings : Screen("settings/connection", "Connection", Icons.Filled.Settings)
data object ChatSettings : Screen("settings/chat", "Chat", Icons.Filled.Settings)
data object MediaSettings : Screen("settings/media", "Media", Icons.Filled.Settings)
data object AppearanceSettings : Screen("settings/appearance", "Appearance", Icons.Filled.Settings)
data object Analytics : Screen("settings/analytics", "Analytics", Icons.Filled.Settings)
data object DeveloperSettings : Screen("settings/developer", "Developer", Icons.Filled.Settings)
data object About : Screen("settings/about", "About", Icons.Filled.Settings)
}
private val bottomNavScreens = listOf(
@@ -85,17 +159,114 @@ private val bottomNavScreens = listOf(
fun RelayApp() {
val connectionViewModel: ConnectionViewModel = viewModel()
val chatViewModel: ChatViewModel = viewModel()
val terminalViewModel: TerminalViewModel = viewModel()
val voiceViewModel: VoiceViewModel = viewModel()
// One-time init: the terminal channel ViewModel registers with the shared
// multiplexer and observes the relay connection state so it can attach/
// reattach automatically on network changes.
LaunchedEffect(Unit) {
terminalViewModel.initialize(
multiplexer = connectionViewModel.multiplexer,
connectionState = connectionViewModel.relayConnectionState,
authState = connectionViewModel.authState,
authManager = connectionViewModel.authManager
)
}
// Lifecycle-aware revalidation. ON_RESUME (every time the app comes
// to the foreground) flips both health badges to Probing and fires a
// fresh API + relay /health probe. Without this hook, badges showed
// stale Connected/Disconnected for up to 30s after foregrounding —
// the entire StateFlow snapshot was preserved across backgrounding
// even when the underlying server had died or the network had flipped.
val lifecycleOwner = LocalLifecycleOwner.current
DisposableEffect(lifecycleOwner) {
val observer = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_RESUME) {
connectionViewModel.revalidate()
}
}
lifecycleOwner.lifecycle.addObserver(observer)
onDispose {
lifecycleOwner.lifecycle.removeObserver(observer)
}
}
// Initialize ChatViewModel reactively when API client becomes available
val apiClient by connectionViewModel.apiClient.collectAsState()
val lastSessionId by connectionViewModel.lastSessionId.collectAsState()
var sessionResumed by remember { mutableStateOf(false) }
val mediaContext = androidx.compose.ui.platform.LocalContext.current
// Voice pipeline wiring — mirrors ChatViewModel.initializeMedia (above).
// We build a dedicated OkHttpClient so voice requests don't contend with
// media fetches on the same dispatcher queue, then hand VoiceViewModel
// the client + recorder + player it needs for the turn state machine.
val voiceClient = remember {
RelayVoiceClient(
context = mediaContext,
okHttpClient = okhttp3.OkHttpClient.Builder()
.readTimeout(2, java.util.concurrent.TimeUnit.MINUTES)
.connectTimeout(15, java.util.concurrent.TimeUnit.SECONDS)
.build(),
relayUrlProvider = { connectionViewModel.relayUrl.value },
sessionTokenProvider = {
(connectionViewModel.authState.value as? AuthState.Paired)?.token
},
)
}
// Remembered so the AudioTrack buffers are synthesized once per process.
// VoiceSfxPlayer is internally crash-proof — failed AudioTrack builds
// become null-tracks that no-op — so we don't need an outer try/catch.
val voiceSfxPlayer = remember { VoiceSfxPlayer(mediaContext) }
LaunchedEffect(Unit) {
val recorder = VoiceRecorder(mediaContext, voiceViewModel.viewModelScope)
val player = VoicePlayer()
voiceViewModel.initialize(
voiceClient = voiceClient,
chatViewModel = chatViewModel,
recorder = recorder,
player = player,
sfxPlayer = voiceSfxPlayer,
// === PHASE3-voice-intents-localdispatch ===
// Wire the local in-process dispatcher so voice intents go
// through `BridgeCommandHandler.handleLocalCommand` (same
// dispatch + Tier 5 safety pipeline as WSS-incoming commands)
// instead of round-tripping through the relay. The relay
// correctly rejects phone-originated bridge.command envelopes
// as "unexpected from phone" — the wire protocol is server→
// phone for commands, and voice intents are phone-local.
//
// The multiplexer is still passed for non-bridge envelope use
// cases and as a debug fallback (with a WARN log) if the
// local dispatcher is somehow null at runtime.
//
// Discovered + fixed 2026-04-14 — see ROADMAP.md v0.4.1
// "voice intent local dispatch loop" entry and the multiplexer
// wiring fix in commit a568366 that unblocked the dispatch
// path enough to surface this protocol mismatch.
bridgeMultiplexer = connectionViewModel.multiplexer,
localBridgeDispatcher = connectionViewModel.bridgeCommandHandler::handleLocalCommand,
// === END PHASE3-voice-intents-localdispatch ===
)
}
LaunchedEffect(apiClient) {
apiClient?.let { client ->
chatViewModel.initialize(client, connectionViewModel.chatHandler)
chatViewModel.updateApiClient(client)
// Wire inbound-media dependencies. Safe to call on every reinit —
// idempotent rewire of the ChatHandler callbacks.
chatViewModel.initializeMedia(
context = mediaContext,
relayHttpClient = connectionViewModel.relayHttpClient,
mediaSettingsRepo = connectionViewModel.mediaSettingsRepo,
mediaCacheWriter = connectionViewModel.mediaCacheWriter
)
// Wire session persistence callback
chatViewModel.onSessionChanged = { sessionId ->
connectionViewModel.saveLastSessionId(sessionId)
@@ -109,11 +280,28 @@ fun RelayApp() {
}
}
// Sync app context toggle from settings to chat
// === PHASE3-status: sync granular phone-status settings to chat ===
val appContextEnabled by connectionViewModel.appContextEnabled.collectAsState()
LaunchedEffect(appContextEnabled) {
chatViewModel.appContextEnabled = appContextEnabled
val appContextBridgeState by connectionViewModel.appContextBridgeState.collectAsState()
val appContextCurrentApp by connectionViewModel.appContextCurrentApp.collectAsState()
val appContextBattery by connectionViewModel.appContextBattery.collectAsState()
val appContextSafetyStatus by connectionViewModel.appContextSafetyStatus.collectAsState()
LaunchedEffect(
appContextEnabled,
appContextBridgeState,
appContextCurrentApp,
appContextBattery,
appContextSafetyStatus,
) {
chatViewModel.appContextSettings = com.hermesandroid.relay.util.AppContextSettings(
master = appContextEnabled,
bridgeState = appContextBridgeState,
currentApp = appContextCurrentApp,
battery = appContextBattery,
safetyStatus = appContextSafetyStatus,
)
}
// === END PHASE3-status ===
// Sync tool annotation parsing toggle to ChatHandler
val parseAnnotations by connectionViewModel.parseToolAnnotations.collectAsState()
@@ -121,10 +309,14 @@ fun RelayApp() {
connectionViewModel.chatHandler.parseToolAnnotations = parseAnnotations
}
// Sync streaming endpoint preference to chat
// Sync streaming endpoint preference to chat. Resolves "auto" against the
// current server capabilities so vanilla upstream + bootstrap-injected
// sessions API picks /v1/runs for chat (which has live tool events)
// while still using /api/sessions/* for browse/rename/delete.
val streamingEndpoint by connectionViewModel.streamingEndpoint.collectAsState()
LaunchedEffect(streamingEndpoint) {
chatViewModel.streamingEndpoint = streamingEndpoint
val serverCapabilities by connectionViewModel.serverCapabilities.collectAsState()
LaunchedEffect(streamingEndpoint, serverCapabilities) {
chatViewModel.streamingEndpoint = connectionViewModel.resolveStreamingEndpoint(streamingEndpoint)
}
// What's New auto-show
@@ -144,8 +336,9 @@ fun RelayApp() {
// Observe theme preference
val themePreference by connectionViewModel.theme.collectAsState()
val fontScale by connectionViewModel.fontScale.collectAsState()
HermesRelayTheme(themePreference = themePreference) {
HermesRelayTheme(themePreference = themePreference, fontScale = fontScale) {
// Brief sphere intro after system splash fades
var introComplete by remember { mutableStateOf(false) }
LaunchedEffect(Unit) {
@@ -155,6 +348,22 @@ fun RelayApp() {
val navController = rememberNavController()
// === PHASE3-safety-rails-followup: cross-layer deep-link nav ===
// Collect navigation requests posted by external launchers (e.g., the
// BridgeForegroundService notification's "Settings" action). The
// service sets EXTRA_NAV_ROUTE on its launch intent → MainActivity's
// onCreate / onNewIntent reads it and pumps it onto NavRouteRequest →
// we forward each emission to the NavController. Single observer at
// the app root so every screen benefits.
LaunchedEffect(navController) {
com.hermesandroid.relay.util.NavRouteRequest.requests.collect { route ->
navController.navigate(route) {
launchSingleTop = true
}
}
}
// === END PHASE3-safety-rails-followup ===
val startDestination = if (onboardingCompleted) Screen.Chat.route else Screen.Onboarding.route
val navBackStackEntry by navController.currentBackStackEntryAsState()
@@ -165,12 +374,23 @@ fun RelayApp() {
val imeBottom = WindowInsets.ime.getBottom(density)
val isKeyboardVisible = imeBottom > 0
// Voice mode is a full-screen modality — while it's active we hide the
// bottom navigation bar so the voice overlay can own the entire screen
// without the Chat/Terminal/Bridge/Settings tabs peeking through below.
val voiceUiState by voiceViewModel.uiState.collectAsState()
// Single snackbar host for the whole app — exposed via LocalSnackbarHost
// so voice/chat/settings screens can call showHumanError from their
// error-collector LaunchedEffects without threading state downwards.
val snackbarHostState = remember { SnackbarHostState() }
Box(modifier = Modifier.fillMaxSize()) {
Scaffold(
modifier = Modifier.fillMaxSize(),
contentWindowInsets = WindowInsets(0),
snackbarHost = { SnackbarHost(snackbarHostState) },
bottomBar = {
if (!isOnboarding && !isKeyboardVisible) {
if (!isOnboarding && !isKeyboardVisible && !voiceUiState.voiceMode) {
NavigationBar(
containerColor = if (isDarkTheme) {
Color(0xFF1A1A2E).copy(alpha = 0.9f)
@@ -230,21 +450,31 @@ fun RelayApp() {
}
}
) { innerPadding ->
CompositionLocalProvider(LocalSnackbarHost provides snackbarHostState) {
NavHost(
navController = navController,
startDestination = startDestination,
modifier = Modifier.padding(innerPadding)
) {
composable(Screen.Onboarding.route) {
// The wizard inside OnboardingScreen now owns credential
// application via ConnectionViewModel.applyPairingPayload,
// so the callback collapses to "mark complete + navigate
// to chat". The legacy 4-arg signature was discarding the
// relay block entirely.
//
// CRITICAL: pass the Activity-scoped connectionViewModel
// explicitly instead of letting OnboardingScreen fetch
// its own via `viewModel()`. A bare `viewModel()` call
// inside a `composable(...)` block binds to the
// NavBackStackEntry's store, so the onboarding VM gets
// destroyed by `popUpTo(Onboarding) { inclusive = true }`
// on navigation to Chat — taking the freshly-minted
// session token with it. See the full writeup on the
// OnboardingScreen function definition.
OnboardingScreen(
onComplete = { apiServerUrl, apiKey, relayUrl ->
connectionViewModel.updateApiServerUrl(apiServerUrl)
if (apiKey.isNotBlank()) {
connectionViewModel.updateApiKey(apiKey)
}
if (relayUrl.isNotBlank()) {
connectionViewModel.updateRelayUrl(relayUrl)
}
connectionViewModel = connectionViewModel,
onComplete = {
connectionViewModel.completeOnboarding()
navController.navigate(Screen.Chat.route) {
popUpTo(Screen.Onboarding.route) { inclusive = true }
@@ -265,19 +495,164 @@ fun RelayApp() {
ChatScreen(
chatViewModel = chatViewModel,
connectionViewModel = connectionViewModel,
voiceViewModel = voiceViewModel,
maxBubbleWidth = maxBubbleWidth
)
}
composable(Screen.Terminal.route) {
TerminalScreen()
TerminalScreen(
terminalViewModel = terminalViewModel,
connectionViewModel = connectionViewModel
)
}
composable(Screen.Bridge.route) {
BridgeScreen()
// === PHASE3-bridge-ui: BridgeScreen wiring ===
// BridgeScreen owns its own BridgeViewModel via the
// default `viewModel()` parameter — no shared state with
// ChatViewModel / ConnectionViewModel is plumbed through
// here yet. Once Agent accessibility lands HermesAccessibilityService
// and we need to observe its runtime state from RelayApp
// scope, a shared holder or explicit VM param gets added
// here.
BridgeScreen(
// === PHASE3-safety-rails: bridge safety route ===
onNavigateToBridgeSafety = {
navController.navigate(Screen.BridgeSafetySettings.route)
},
// === END PHASE3-safety-rails ===
)
// === END PHASE3-bridge-ui ===
}
composable(Screen.Settings.route) {
SettingsScreen(connectionViewModel = connectionViewModel)
SettingsScreen(
connectionViewModel = connectionViewModel,
onNavigateToConnectionSettings = {
navController.navigate(Screen.ConnectionSettings.route)
},
onNavigateToChatSettings = {
navController.navigate(Screen.ChatSettings.route)
},
onNavigateToMediaSettings = {
navController.navigate(Screen.MediaSettings.route)
},
onNavigateToAppearanceSettings = {
navController.navigate(Screen.AppearanceSettings.route)
},
onNavigateToAnalytics = {
navController.navigate(Screen.Analytics.route)
},
onNavigateToVoiceSettings = {
navController.navigate(Screen.VoiceSettings.route)
},
onNavigateToNotificationCompanion = {
navController.navigate(Screen.NotificationCompanionSettings.route)
},
// === PHASE3-safety-rails: bridge safety route ===
onNavigateToBridgeSafety = {
navController.navigate(Screen.BridgeSafetySettings.route)
},
// === END PHASE3-safety-rails ===
onNavigateToPairedDevices = {
navController.navigate(Screen.PairedDevices.route)
},
onNavigateToDeveloperSettings = {
navController.navigate(Screen.DeveloperSettings.route)
},
onNavigateToAbout = {
navController.navigate(Screen.About.route)
}
)
}
composable(Screen.VoiceSettings.route) {
VoiceSettingsScreen(
voiceViewModel = voiceViewModel,
voiceClient = voiceClient,
onBack = { navController.popBackStack() }
)
}
// === PHASE3-notif-listener-followup: notification companion route ===
composable(Screen.NotificationCompanionSettings.route) {
NotificationCompanionSettingsScreen(
onBack = { navController.popBackStack() }
)
}
// === END PHASE3-notif-listener-followup ===
// === PHASE3-safety-rails: bridge safety route ===
composable(Screen.BridgeSafetySettings.route) {
BridgeSafetySettingsScreen(
onBack = { navController.popBackStack() }
)
}
// === END PHASE3-safety-rails ===
composable(Screen.PairedDevices.route) {
PairedDevicesScreen(
connectionViewModel = connectionViewModel,
onBack = { navController.popBackStack() },
onRequestRepair = {
// Pop back to Settings so the user lands on the
// "Scan Pairing QR" button rather than getting
// stranded on an empty devices list.
navController.popBackStack(Screen.Settings.route, inclusive = false)
}
)
}
composable(Screen.ConnectionSettings.route) {
ConnectionSettingsScreen(
connectionViewModel = connectionViewModel,
onBack = { navController.popBackStack() },
onNavigateToPairedDevices = {
navController.navigate(Screen.PairedDevices.route)
},
onNavigateToPair = {
navController.navigate(Screen.Pair.route)
}
)
}
composable(Screen.Pair.route) {
com.hermesandroid.relay.ui.screens.PairScreen(
connectionViewModel = connectionViewModel,
onComplete = { navController.popBackStack() },
onCancel = { navController.popBackStack() }
)
}
composable(Screen.ChatSettings.route) {
ChatSettingsScreen(
connectionViewModel = connectionViewModel,
onBack = { navController.popBackStack() }
)
}
composable(Screen.MediaSettings.route) {
MediaSettingsScreen(
connectionViewModel = connectionViewModel,
onBack = { navController.popBackStack() }
)
}
composable(Screen.AppearanceSettings.route) {
AppearanceSettingsScreen(
connectionViewModel = connectionViewModel,
onBack = { navController.popBackStack() }
)
}
composable(Screen.Analytics.route) {
AnalyticsScreen(
connectionViewModel = connectionViewModel,
onBack = { navController.popBackStack() }
)
}
composable(Screen.DeveloperSettings.route) {
DeveloperSettingsScreen(
connectionViewModel = connectionViewModel,
onBack = { navController.popBackStack() }
)
}
composable(Screen.About.route) {
AboutScreen(
connectionViewModel = connectionViewModel,
onBack = { navController.popBackStack() }
)
}
}
} // end CompositionLocalProvider
}
// Sphere intro overlay — fades out after 1.5s to reveal main UI
@@ -0,0 +1,316 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Block
import androidx.compose.material.icons.filled.CheckCircle
import androidx.compose.material.icons.filled.Error
import androidx.compose.material.icons.filled.HourglassEmpty
import androidx.compose.material.icons.filled.InsertPhoto
import androidx.compose.material.icons.filled.Timeline
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.data.BridgeActivityEntry
import com.hermesandroid.relay.data.BridgeActivityStatus
import java.time.Instant
import java.time.LocalDateTime
import java.time.ZoneId
import java.time.format.DateTimeFormatter
/**
* Scrollable activity log card — rolling view of the most recent bridge
* commands with timestamp, method, status icon, and tap-to-expand detail.
*
* Phase 3 Wave 1 — bridge-ui (`bridge-screen-ui`). The log is populated via
* [com.hermesandroid.relay.data.BridgePreferencesRepository.appendEntry];
* accessibility's command dispatcher is the producer once its runtime is wired. Until
* then this card shows the "no activity yet" empty state.
*
* Height-bounded with [heightIn] to keep the log from pushing the safety
* card off the fold on phone-portrait screens — the activity log is
* internally scrollable within its own [LazyColumn].
*/
@Composable
fun BridgeActivityLog(
entries: List<BridgeActivityEntry>,
onClear: () -> Unit,
modifier: Modifier = Modifier,
) {
Card(
modifier = modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp)
) {
Row(
verticalAlignment = Alignment.CenterVertically,
modifier = Modifier.fillMaxWidth()
) {
Icon(
imageVector = Icons.Filled.Timeline,
contentDescription = null,
tint = MaterialTheme.colorScheme.primary,
modifier = Modifier.size(18.dp)
)
Spacer(modifier = Modifier.size(6.dp))
Text(
text = "Activity Log",
style = MaterialTheme.typography.titleSmall,
fontWeight = FontWeight.SemiBold,
color = MaterialTheme.colorScheme.primary,
modifier = Modifier.weight(1f)
)
if (entries.isNotEmpty()) {
TextButton(onClick = onClear) { Text("Clear") }
}
}
HorizontalDivider(color = MaterialTheme.colorScheme.outline.copy(alpha = 0.15f))
if (entries.isEmpty()) {
Text(
text = "No bridge commands yet. Every tap, type, and " +
"screenshot the agent performs will show up here.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
} else {
// Hard cap the visible height so the log doesn't eat the whole
// screen — internally scrollable for history.
LazyColumn(
modifier = Modifier.heightIn(max = 320.dp),
verticalArrangement = Arrangement.spacedBy(4.dp)
) {
items(entries, key = { it.id }) { entry ->
ActivityRow(entry = entry)
}
}
}
}
}
}
@Composable
private fun ActivityRow(entry: BridgeActivityEntry) {
var expanded by remember(entry.id) { mutableStateOf(false) }
Column(
modifier = Modifier
.fillMaxWidth()
.clip(RoundedCornerShape(8.dp))
.clickable { expanded = !expanded }
.padding(vertical = 6.dp, horizontal = 4.dp),
) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
modifier = Modifier.fillMaxWidth()
) {
StatusDot(status = entry.status)
Text(
text = formatTime(entry.timestampMs),
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
fontFamily = FontFamily.Monospace
)
Text(
text = entry.method,
style = MaterialTheme.typography.bodyMedium,
fontWeight = FontWeight.Medium,
)
Text(
text = entry.summary,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.weight(1f)
)
if (entry.thumbnailToken != null) {
Icon(
imageVector = Icons.Filled.InsertPhoto,
contentDescription = "Has screenshot",
tint = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.size(14.dp)
)
}
}
AnimatedVisibility(visible = expanded) {
Column(
modifier = Modifier.padding(start = 28.dp, top = 6.dp, bottom = 2.dp),
verticalArrangement = Arrangement.spacedBy(4.dp)
) {
Text(
text = "Full timestamp: ${formatFullTime(entry.timestampMs)}",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
fontFamily = FontFamily.Monospace
)
Text(
text = "Status: ${entry.status.name}",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
if (!entry.resultText.isNullOrBlank()) {
Text(
text = entry.resultText,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurface
)
}
if (entry.thumbnailToken != null) {
// TODO(accessibility-handoff): wire this to InboundAttachmentCard
// once accessibility's ScreenCapture.kt is uploading real screenshots
// to MediaRegistry. Until then we show a placeholder token
// label so the expand affordance still communicates the
// shape of the future feature.
Text(
text = "Screenshot token: ${entry.thumbnailToken}",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
fontFamily = FontFamily.Monospace
)
}
}
}
}
}
@Composable
private fun StatusDot(status: BridgeActivityStatus) {
val (icon, tint) = when (status) {
BridgeActivityStatus.Pending -> Icons.Filled.HourglassEmpty to
MaterialTheme.colorScheme.onSurfaceVariant
BridgeActivityStatus.Success -> Icons.Filled.CheckCircle to Color(0xFF4CAF50)
BridgeActivityStatus.Failed -> Icons.Filled.Error to
MaterialTheme.colorScheme.error
BridgeActivityStatus.Blocked -> Icons.Filled.Block to Color(0xFFFFA726)
}
Icon(
imageVector = icon,
contentDescription = status.name,
tint = tint,
modifier = Modifier.size(14.dp)
)
}
// ── Formatting helpers ──────────────────────────────────────────────────
private val TIME_FMT: DateTimeFormatter = DateTimeFormatter.ofPattern("HH:mm:ss")
private val FULL_FMT: DateTimeFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")
private fun formatTime(epochMs: Long): String = try {
LocalDateTime.ofInstant(Instant.ofEpochMilli(epochMs), ZoneId.systemDefault())
.format(TIME_FMT)
} catch (_: Exception) {
"--:--:--"
}
private fun formatFullTime(epochMs: Long): String = try {
LocalDateTime.ofInstant(Instant.ofEpochMilli(epochMs), ZoneId.systemDefault())
.format(FULL_FMT)
} catch (_: Exception) {
"—"
}
// ── Previews ────────────────────────────────────────────────────────────
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeActivityLogPreviewFilled() {
val now = System.currentTimeMillis()
val sample = listOf(
BridgeActivityEntry(
id = "1",
timestampMs = now,
method = "tap",
summary = "(540, 1200)",
status = BridgeActivityStatus.Success,
resultText = "Dispatched gesture at (540, 1200)",
),
BridgeActivityEntry(
id = "2",
timestampMs = now - 2_000,
method = "read_screen",
summary = "root → 42 nodes",
status = BridgeActivityStatus.Success,
thumbnailToken = "hermes-relay://ab12cd34",
),
BridgeActivityEntry(
id = "3",
timestampMs = now - 5_500,
method = "open_app",
summary = "Chrome",
status = BridgeActivityStatus.Blocked,
resultText = "Blocked by safety rails: chrome is in user blocklist",
),
BridgeActivityEntry(
id = "4",
timestampMs = now - 9_000,
method = "type",
summary = "\"hello\"",
status = BridgeActivityStatus.Failed,
resultText = "No focused input field",
),
BridgeActivityEntry(
id = "5",
timestampMs = now - 12_000,
method = "ping",
summary = "→ 12ms",
status = BridgeActivityStatus.Success,
),
)
MaterialTheme {
BridgeActivityLog(
entries = sample,
onClear = {},
modifier = Modifier.padding(16.dp)
)
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeActivityLogPreviewEmpty() {
MaterialTheme {
BridgeActivityLog(
entries = emptyList(),
onClear = {},
modifier = Modifier.padding(16.dp)
)
}
}
@@ -0,0 +1,255 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.BatteryFull
import androidx.compose.material.icons.filled.Info
import androidx.compose.material.icons.filled.PhoneAndroid
import androidx.compose.material.icons.filled.ScreenLockPortrait
import androidx.compose.material.icons.filled.Smartphone
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.viewmodel.BridgeStatus
/**
* Master "Allow Agent Control" card — the headline of the Bridge tab.
*
* Phase 3 Wave 1 — bridge-ui (`bridge-screen-ui`). Visual style mirrors the status
* cards in `PairedDevicesScreen`: surfaceVariant background, 16dp padding,
* 10dp row spacing. Uses [ConnectionStatusBadge] for the pulsing status dot
* so the Bridge tab looks visually consistent with the Settings → Connection
* section.
*
* The headline switch is `enabled = allowEnable` so users can't flip it on
* when the a11y service isn't granted — we instead bounce them to the
* permission checklist. Tapping the info icon opens an explanation dialog
* (required by Google Play's a11y review process, per the Phase 3 plan's
* Play Store Strategy section).
*/
@Composable
fun BridgeMasterToggle(
enabled: Boolean,
status: BridgeStatus?,
accessibilityGranted: Boolean,
onToggle: (Boolean) -> Unit,
modifier: Modifier = Modifier,
label: String = "Agent Control",
) {
var showExplain by remember { mutableStateOf(false) }
// Subtitle text reflects the flavor context. Sideload: "agent can
// interact / control". googlePlay (label != "Agent Control"):
// "connected / disconnected" since the Play flavor is read-only.
val isSideloadLabel = label == "Agent Control"
val subtitle = if (isSideloadLabel) {
if (enabled) "Active — agent can interact with this device"
else "Off — agent cannot control this device"
} else {
if (enabled) "Active — screen reading for chat context"
else "Off — bridge is not reading screen content"
}
Card(
modifier = modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
) {
Column(modifier = Modifier.weight(1f)) {
Text(
text = label,
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.primary,
fontWeight = FontWeight.SemiBold
)
Text(
text = subtitle,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
}
IconButton(onClick = { showExplain = true }) {
Icon(
imageVector = Icons.Filled.Info,
contentDescription = "What does this do?",
tint = MaterialTheme.colorScheme.onSurfaceVariant
)
}
Switch(
checked = enabled && accessibilityGranted,
onCheckedChange = { onToggle(it) },
enabled = accessibilityGranted || enabled,
)
}
if (!accessibilityGranted) {
Text(
text = "Grant the Accessibility Service permission below to enable.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.error
)
}
if (status != null) {
Spacer(modifier = Modifier.height(2.dp))
StatusInlineRow(
icon = Icons.Filled.PhoneAndroid,
label = "Device",
value = status.deviceName
)
StatusInlineRow(
icon = Icons.Filled.BatteryFull,
label = "Battery",
value = status.batteryPercent?.let { "$it%" } ?: "—"
)
StatusInlineRow(
icon = Icons.Filled.ScreenLockPortrait,
label = "Screen",
value = if (status.screenOn) "ON" else "OFF"
)
StatusInlineRow(
icon = Icons.Filled.Smartphone,
label = "Current app",
value = status.currentApp ?: "—"
)
}
}
}
if (showExplain) {
AlertDialog(
onDismissRequest = { showExplain = false },
title = { Text("About Agent Control") },
text = {
Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text(
"Agent Control lets your Hermes agent read what's on " +
"your screen and interact with apps on your behalf " +
"(tap, type, scroll, screenshot).",
style = MaterialTheme.typography.bodyMedium
)
Text(
"This uses Android's Accessibility Service API, which " +
"is the same permission screen readers use. You must " +
"enable it in Android Settings before this switch works.",
style = MaterialTheme.typography.bodyMedium
)
Text(
"You can turn Agent Control off at any time from this " +
"screen or by disabling the service in Android " +
"Settings. All bridge commands are logged in the " +
"Activity Log below.",
style = MaterialTheme.typography.bodyMedium
)
}
},
confirmButton = {
TextButton(onClick = { showExplain = false }) { Text("Got it") }
}
)
}
}
@Composable
private fun StatusInlineRow(
icon: androidx.compose.ui.graphics.vector.ImageVector,
label: String,
value: String,
) {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Icon(
imageVector = icon,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.height(16.dp)
)
Text(
text = label,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.weight(1f)
)
Text(
text = value,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurface,
)
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeMasterTogglePreviewOn() {
MaterialTheme {
BridgeMasterToggle(
enabled = true,
status = BridgeStatus(
deviceName = "Galaxy S24",
batteryPercent = 78,
screenOn = true,
currentApp = "Chrome",
accessibilityEnabled = true,
),
accessibilityGranted = true,
onToggle = {},
modifier = Modifier.padding(16.dp)
)
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeMasterTogglePreviewBlocked() {
MaterialTheme {
BridgeMasterToggle(
enabled = false,
status = BridgeStatus(
deviceName = "Pixel 9",
batteryPercent = 42,
screenOn = true,
currentApp = null,
accessibilityEnabled = false,
),
accessibilityGranted = false,
onToggle = {},
modifier = Modifier.padding(16.dp)
)
}
}
@@ -0,0 +1,301 @@
package com.hermesandroid.relay.ui.components
import android.content.Context
import android.content.Intent
import android.net.Uri
import android.provider.Settings
import com.hermesandroid.relay.data.BuildFlavor
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.KeyboardArrowRight
import androidx.compose.material.icons.filled.Accessibility
import androidx.compose.material.icons.filled.CheckCircle
import androidx.compose.material.icons.filled.Notifications
import androidx.compose.material.icons.filled.PictureInPicture
import androidx.compose.material.icons.filled.RadioButtonUnchecked
import androidx.compose.material.icons.filled.ScreenShare
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.viewmodel.BridgePermissionStatus
/**
* Permission checklist card — one row per required Android permission.
* Tapping a non-granted row fires an Intent to the corresponding Android
* Settings screen so the user can grant the permission without leaving
* their muscle-memory path.
*
* Phase 3 Wave 1 — bridge-ui (`bridge-screen-ui`). Uses vector Material icons to
* stay inside the already-shipped icon set (no dependency on
* compose-icons-extended, which has bitten us before — see
* `fix(settings): revert ChevronRight…`).
*/
@Composable
fun BridgePermissionChecklist(
status: BridgePermissionStatus,
modifier: Modifier = Modifier,
// === PHASE3-safety-rails-followup: in-app permission Test handlers ===
onTestAccessibility: (() -> Unit)? = null,
onTestScreenCapture: (() -> Unit)? = null,
onTestOverlay: (() -> Unit)? = null,
// === END PHASE3-safety-rails-followup ===
// === PHASE3-bridge-ui-followup: extended interactions ===
// Tapping the Screen Capture row launches the system MediaProjection
// consent dialog (no Settings page exists for this permission, so
// the row's onClick was previously null). Notification Listener gets
// its own Test button on the Bridge tab for parity with the others —
// the dedicated test on NotificationCompanionSettingsScreen still ships.
onRequestScreenCapture: (() -> Unit)? = null,
onTestNotificationListener: (() -> Unit)? = null,
// === END PHASE3-bridge-ui-followup ===
) {
val context = LocalContext.current
Card(
modifier = modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(6.dp)
) {
Text(
text = "Permissions",
style = MaterialTheme.typography.titleSmall,
fontWeight = FontWeight.SemiBold,
color = MaterialTheme.colorScheme.primary,
)
Text(
text = "Tap a row to open Android Settings · Tap Test to verify the permission works.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
HorizontalDivider(color = MaterialTheme.colorScheme.outline.copy(alpha = 0.15f))
PermissionRow(
icon = Icons.Filled.Accessibility,
title = "Accessibility Service",
subtitle = if (BuildFlavor.isSideload)
"Read screen content, dispatch taps/types"
else
"Read screen content for chat context",
granted = status.accessibilityServiceEnabled,
onClick = { openAccessibilitySettings(context) },
onTest = onTestAccessibility,
)
// Screen Capture — sideload only. googlePlay doesn't declare
// FOREGROUND_SERVICE_MEDIA_PROJECTION or the /screenshot route,
// and showing a consent toggle for a capability the APK can't
// use would confuse both users and Play reviewers.
if (BuildFlavor.isSideload) {
PermissionRow(
icon = Icons.Filled.ScreenShare,
title = "Screen Capture",
subtitle = if (status.screenCapturePermitted)
"Granted for this session — agent can take screenshots"
else
"Tap to grant — agent needs this for /screenshot",
granted = status.screenCapturePermitted,
// MediaProjection has no Android Settings page; tapping the
// row launches the system consent dialog directly via
// ScreenCaptureRequester. Falls back to inert (no chevron)
// if the parent didn't provide a launcher (e.g., previews).
onClick = onRequestScreenCapture,
onTest = onTestScreenCapture,
)
}
// Display over other apps — sideload only. googlePlay has no
// destructive-verb safety modal (action routes are blocked) and
// no status overlay chip, so the SYSTEM_ALERT_WINDOW permission
// isn't needed and showing the row would confuse users + reviewers.
if (BuildFlavor.isSideload) {
PermissionRow(
icon = Icons.Filled.PictureInPicture,
title = "Display over other apps",
subtitle = "Status overlay while bridge is active",
granted = status.overlayPermitted,
onClick = { openOverlaySettings(context) },
onTest = onTestOverlay,
)
}
PermissionRow(
icon = Icons.Filled.Notifications,
title = "Notification Listener",
subtitle = "Read notifications for agent summaries",
granted = status.notificationListenerPermitted,
onClick = { openNotificationListenerSettings(context) },
// Parity Test button on the Bridge tab. The full functional
// round-trip test still lives on NotificationCompanionSettingsScreen.
onTest = onTestNotificationListener,
)
}
}
}
@Composable
private fun PermissionRow(
icon: ImageVector,
title: String,
subtitle: String,
granted: Boolean,
onClick: (() -> Unit)?,
onTest: (() -> Unit)? = null,
) {
val rowModifier = if (onClick != null) {
Modifier
.fillMaxWidth()
.clickable(onClick = onClick)
.padding(vertical = 10.dp)
} else {
Modifier
.fillMaxWidth()
.padding(vertical = 10.dp)
}
Row(
modifier = rowModifier,
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
Icon(
imageVector = icon,
contentDescription = null,
tint = MaterialTheme.colorScheme.primary,
modifier = Modifier.size(22.dp),
)
Column(modifier = Modifier.weight(1f)) {
Text(
text = title,
style = MaterialTheme.typography.bodyMedium,
)
Text(
text = subtitle,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
// === PHASE3-safety-rails-followup: in-app Test button ===
// Compact text button next to the status icon. Tapping it runs a
// diagnostic check on the permission and surfaces the result via
// BridgeScreen → LocalSnackbarHost. Only renders when the parent
// provides an onTest lambda; null hides the button on rows where
// a meaningful diagnostic isn't available at this layer.
if (onTest != null) {
TextButton(
onClick = onTest,
contentPadding = androidx.compose.foundation.layout.PaddingValues(
horizontal = 8.dp,
vertical = 0.dp,
),
) {
Text(
text = "Test",
style = MaterialTheme.typography.labelSmall,
)
}
}
// === END PHASE3-safety-rails-followup ===
// Status icon: green check when granted, red empty circle otherwise.
Icon(
imageVector = if (granted) Icons.Filled.CheckCircle
else Icons.Filled.RadioButtonUnchecked,
contentDescription = if (granted) "Granted" else "Not granted",
tint = if (granted) Color(0xFF4CAF50) else MaterialTheme.colorScheme.error,
modifier = Modifier.size(22.dp),
)
if (onClick != null) {
Icon(
imageVector = Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.size(18.dp),
)
}
}
}
// ── Intent helpers ──────────────────────────────────────────────────────
//
// All three intent launchers guard with runCatching because Samsung / Xiaomi
// OEM skins occasionally ship without the standard ACTION_* constants, and
// we'd rather degrade to a no-op than crash the Bridge screen.
private fun openAccessibilitySettings(context: Context) {
runCatching {
val intent = Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS).apply {
addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
}
context.startActivity(intent)
}
}
private fun openOverlaySettings(context: Context) {
runCatching {
val intent = Intent(
Settings.ACTION_MANAGE_OVERLAY_PERMISSION,
Uri.parse("package:${context.packageName}")
).apply {
addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
}
context.startActivity(intent)
}
}
private fun openNotificationListenerSettings(context: Context) {
runCatching {
val intent = Intent("android.settings.ACTION_NOTIFICATION_LISTENER_SETTINGS").apply {
addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
}
context.startActivity(intent)
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgePermissionChecklistPreviewAllGranted() {
MaterialTheme {
BridgePermissionChecklist(
status = BridgePermissionStatus(
accessibilityServiceEnabled = true,
screenCapturePermitted = true,
overlayPermitted = true,
notificationListenerPermitted = true,
),
modifier = Modifier.padding(16.dp)
)
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgePermissionChecklistPreviewNoneGranted() {
MaterialTheme {
BridgePermissionChecklist(
status = BridgePermissionStatus(),
modifier = Modifier.padding(16.dp)
)
}
}
@@ -0,0 +1,188 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.KeyboardArrowRight
import androidx.compose.material.icons.filled.Security
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableLongStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.data.BridgeSafetySettings
import com.hermesandroid.relay.data.DEFAULT_BLOCKLIST
import com.hermesandroid.relay.data.DEFAULT_DESTRUCTIVE_VERBS
/**
* Phase 3 — safety-rails `bridge-safety-rails`
*
* Replaces the inert `SafetyPlaceholderCard` in [BridgeScreen]. Shows a
* live-updating summary of Tier 5 safety state:
*
* - Blocklist count ("12 apps blocked")
* - Destructive-verb count ("12 verbs need confirmation")
* - Auto-disable window ("Auto-off after 30 min idle")
* - Auto-disable countdown when a timer is active
*
* Tap → navigate to [BridgeSafetySettingsScreen].
*
* The card is self-sufficient: caller passes the settings snapshot +
* optional countdown millis (from `BridgeSafetyManager.autoDisableAtMs`)
* and wires the onClick to the nav controller.
*/
@Composable
fun BridgeSafetySummaryCard(
settings: BridgeSafetySettings,
autoDisableAtMs: Long? = null,
onManage: () -> Unit,
) {
// Tick a local clock every second when a countdown is active so the
// "in 12:34" label actually counts down. Recomposition is scoped to
// this card so the rest of the Bridge screen stays still.
var nowMs by remember { mutableLongStateOf(System.currentTimeMillis()) }
if (autoDisableAtMs != null) {
LaunchedEffect(autoDisableAtMs) {
while (true) {
nowMs = System.currentTimeMillis()
kotlinx.coroutines.delay(1000L)
}
}
}
Card(
modifier = Modifier
.fillMaxWidth()
.clickable(onClick = onManage),
shape = RoundedCornerShape(14.dp),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(10.dp)
) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
modifier = Modifier.fillMaxWidth()
) {
Icon(
imageVector = Icons.Filled.Security,
contentDescription = null,
tint = MaterialTheme.colorScheme.primary,
)
Text(
text = "Safety",
style = MaterialTheme.typography.titleSmall,
fontWeight = FontWeight.SemiBold,
color = MaterialTheme.colorScheme.primary,
modifier = Modifier.weight(1f),
)
Icon(
imageVector = Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = "Manage",
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
SafetySummaryRow(
label = "Apps blocked",
value = "${settings.blocklist.size}",
)
SafetySummaryRow(
label = "Destructive verbs",
value = "${settings.destructiveVerbs.size}",
)
SafetySummaryRow(
label = "Auto-disable",
value = if (autoDisableAtMs != null) {
val remainMs = (autoDisableAtMs - nowMs).coerceAtLeast(0L)
val remainMin = (remainMs / 60_000L).toInt()
val remainSec = ((remainMs % 60_000L) / 1000L).toInt()
"in ${remainMin}:${remainSec.toString().padStart(2, '0')}"
} else {
"${settings.autoDisableMinutes} min idle"
},
)
Text(
text = "Tap Manage to edit blocklist, destructive verbs, and timers.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
@Composable
private fun SafetySummaryRow(label: String, value: String) {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
) {
Text(
text = label,
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.weight(1f),
)
Text(
text = value,
style = MaterialTheme.typography.bodyMedium,
fontWeight = FontWeight.Medium,
color = MaterialTheme.colorScheme.onSurface,
)
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeSafetySummaryCardPreview() {
MaterialTheme {
Column(modifier = Modifier.padding(16.dp)) {
BridgeSafetySummaryCard(
settings = BridgeSafetySettings(
blocklist = DEFAULT_BLOCKLIST,
destructiveVerbs = DEFAULT_DESTRUCTIVE_VERBS,
),
autoDisableAtMs = null,
onManage = {},
)
}
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeSafetySummaryCardPreview_Countdown() {
MaterialTheme {
Column(modifier = Modifier.padding(16.dp)) {
BridgeSafetySummaryCard(
settings = BridgeSafetySettings(
blocklist = DEFAULT_BLOCKLIST,
destructiveVerbs = DEFAULT_DESTRUCTIVE_VERBS,
),
autoDisableAtMs = System.currentTimeMillis() + 12 * 60_000L + 34_000L,
onManage = {},
)
}
}
}
@@ -0,0 +1,147 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.viewmodel.BridgeStatus
/**
* Standalone status-only card — can render with or without [BridgeMasterToggle]
* above it. Used by `BridgeScreen` as a secondary "connection state at a
* glance" block when there's enough info to show but the user hasn't touched
* the master toggle yet.
*
* Phase 3 Wave 1 — bridge-ui (`bridge-screen-ui`). Kept distinct from
* [BridgeMasterToggle] so that Agent safety-rails in Wave 2 can relocate the master
* toggle without losing the status surface (and so we can reuse this card
* in the Settings → Connection section later if desired).
*/
@Composable
fun BridgeStatusCard(
status: BridgeStatus?,
isConnected: Boolean,
modifier: Modifier = Modifier,
) {
Card(
modifier = modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp)
) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
modifier = Modifier.fillMaxWidth()
) {
Text(
text = "Status",
style = MaterialTheme.typography.titleSmall,
fontWeight = FontWeight.SemiBold,
color = MaterialTheme.colorScheme.primary,
modifier = Modifier.weight(1f)
)
ConnectionStatusBadge(
isConnected = isConnected,
isConnecting = false,
)
Text(
text = if (isConnected) "Connected" else "Disconnected",
style = MaterialTheme.typography.labelMedium,
color = if (isConnected) MaterialTheme.colorScheme.onSurface
else MaterialTheme.colorScheme.onSurfaceVariant,
)
}
HorizontalDivider(color = MaterialTheme.colorScheme.outline.copy(alpha = 0.15f))
if (status == null) {
Text(
text = "Bridge runtime not yet reporting status. Enable " +
"Agent Control above to begin receiving device telemetry.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
} else {
StatusKeyValue("Device", status.deviceName)
StatusKeyValue(
"Battery",
status.batteryPercent?.let { "$it%" } ?: "Unknown"
)
StatusKeyValue("Screen", if (status.screenOn) "ON" else "OFF")
StatusKeyValue("Current app", status.currentApp ?: "—")
StatusKeyValue(
"Accessibility service",
if (status.accessibilityEnabled) "Enabled" else "Disabled"
)
}
}
}
}
@Composable
private fun StatusKeyValue(key: String, value: String) {
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween
) {
Text(
text = key,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
Text(
text = value,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurface
)
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeStatusCardPreviewConnected() {
MaterialTheme {
BridgeStatusCard(
status = BridgeStatus(
deviceName = "Galaxy S24",
batteryPercent = 78,
screenOn = true,
currentApp = "Chrome",
accessibilityEnabled = true,
),
isConnected = true,
modifier = Modifier.padding(16.dp)
)
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeStatusCardPreviewEmpty() {
MaterialTheme {
BridgeStatusCard(
status = null,
isConnected = false,
modifier = Modifier.padding(16.dp)
)
}
}
@@ -0,0 +1,533 @@
package com.hermesandroid.relay.ui.components
import android.content.ClipData
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.navigationBarsPadding
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.ContentCopy
import androidx.compose.material.icons.filled.Warning
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.ModalBottomSheet
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Text
import androidx.compose.material3.rememberModalBottomSheetState
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.ClipEntry
import androidx.compose.ui.platform.LocalClipboard
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.auth.AuthState
import com.hermesandroid.relay.data.FeatureFlags
import com.hermesandroid.relay.network.ConnectionState
import com.hermesandroid.relay.viewmodel.ConnectionViewModel
import kotlinx.coroutines.launch
// ---------------------------------------------------------------------------
// Shared helpers
// ---------------------------------------------------------------------------
@Composable
private fun InfoRow(
label: String,
value: String,
valueColor: Color = MaterialTheme.colorScheme.onSurface,
monospace: Boolean = false
) {
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically
) {
Text(
text = label,
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
Text(
text = value,
style = MaterialTheme.typography.bodyMedium,
color = valueColor,
fontFamily = if (monospace) FontFamily.Monospace else null
)
}
}
@Composable
private fun StatusChip(text: String, background: Color, contentColor: Color) {
Text(
text = text,
style = MaterialTheme.typography.labelMedium,
fontWeight = FontWeight.Medium,
color = contentColor,
modifier = Modifier
.clip(RoundedCornerShape(12.dp))
.background(background)
.padding(horizontal = 10.dp, vertical = 4.dp)
)
}
@Composable
private fun ChipRow(label: String, chip: @Composable () -> Unit) {
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically
) {
Text(
text = label,
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
chip()
}
}
@Composable
private fun connectionChip(state: ConnectionState) {
val (label, bg, fg) = when (state) {
ConnectionState.Connected -> Triple(
"Connected",
MaterialTheme.colorScheme.primaryContainer,
MaterialTheme.colorScheme.onPrimaryContainer
)
ConnectionState.Connecting -> Triple(
"Connecting\u2026",
MaterialTheme.colorScheme.tertiaryContainer,
MaterialTheme.colorScheme.onTertiaryContainer
)
ConnectionState.Reconnecting -> Triple(
"Reconnecting\u2026",
MaterialTheme.colorScheme.tertiaryContainer,
MaterialTheme.colorScheme.onTertiaryContainer
)
ConnectionState.Disconnected -> Triple(
"Disconnected",
MaterialTheme.colorScheme.surfaceVariant,
MaterialTheme.colorScheme.onSurfaceVariant
)
}
StatusChip(text = label, background = bg, contentColor = fg)
}
@Composable
private fun authStateChip(state: AuthState) {
val (label, bg, fg) = when (state) {
is AuthState.Unpaired -> Triple(
"Unpaired",
MaterialTheme.colorScheme.surfaceVariant,
MaterialTheme.colorScheme.onSurfaceVariant
)
is AuthState.Pairing -> Triple(
"Pairing\u2026",
MaterialTheme.colorScheme.tertiaryContainer,
MaterialTheme.colorScheme.onTertiaryContainer
)
is AuthState.Paired -> Triple(
"Paired",
MaterialTheme.colorScheme.primaryContainer,
MaterialTheme.colorScheme.onPrimaryContainer
)
is AuthState.Failed -> Triple(
"Failed: ${state.reason}",
MaterialTheme.colorScheme.errorContainer,
MaterialTheme.colorScheme.onErrorContainer
)
}
StatusChip(text = label, background = bg, contentColor = fg)
}
// ---------------------------------------------------------------------------
// 1. SessionInfoSheet
// ---------------------------------------------------------------------------
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun SessionInfoSheet(
connectionViewModel: ConnectionViewModel,
onDismiss: () -> Unit
) {
val sheetState = rememberModalBottomSheetState(skipPartiallyExpanded = false)
val scope = rememberCoroutineScope()
val clipboard = LocalClipboard.current
val authState by connectionViewModel.authState.collectAsState()
val relayUrl by connectionViewModel.relayUrl.collectAsState()
val relayConnectionState by connectionViewModel.relayConnectionState.collectAsState()
val pairingCode by connectionViewModel.pairingCode.collectAsState()
val profiles by connectionViewModel.authManager.profiles.collectAsState()
val pairedSession by connectionViewModel.currentPairedSession.collectAsState()
ModalBottomSheet(
onDismissRequest = onDismiss,
sheetState = sheetState
) {
Column(
modifier = Modifier
.padding(horizontal = 24.dp, vertical = 16.dp)
.navigationBarsPadding(),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
Text(
text = "Session",
style = MaterialTheme.typography.titleLarge
)
Text(
text = "Session authenticates the app with the relay server. " +
"The pairing code is consumed once, then the server issues a " +
"long-lived token stored locally.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
HorizontalDivider()
ChipRow(label = "State") { authStateChip(authState) }
InfoRow(label = "Relay URL", value = relayUrl, monospace = true)
ChipRow(label = "Connection") { connectionChip(relayConnectionState) }
HorizontalDivider()
// Pairing code — big monospace + copy
Column(verticalArrangement = Arrangement.spacedBy(4.dp)) {
Text(
text = "Pairing code",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically
) {
Text(
text = pairingCode,
style = MaterialTheme.typography.headlineSmall,
fontFamily = FontFamily.Monospace,
fontWeight = FontWeight.Bold
)
IconButton(onClick = {
scope.launch {
clipboard.setClipEntry(
ClipEntry(
ClipData.newPlainText("Pairing code", pairingCode)
)
)
}
}) {
Icon(
imageVector = Icons.Default.ContentCopy,
contentDescription = "Copy pairing code"
)
}
}
}
HorizontalDivider()
InfoRow(
label = "Device",
value = "${android.os.Build.MANUFACTURER} ${android.os.Build.MODEL}"
)
InfoRow(
label = "Session token present",
value = if (authState is AuthState.Paired) "Yes" else "No"
)
InfoRow(
label = "Profiles",
value = if (profiles.isEmpty()) "(none)" else profiles.joinToString(", ")
)
// Security overhaul (2026-04-11) — show expiry + grants + storage.
pairedSession?.let { paired ->
HorizontalDivider()
val expiryLabel = when {
paired.expiresAt == null -> "Never"
else -> java.text.DateFormat
.getDateInstance(java.text.DateFormat.MEDIUM)
.format(java.util.Date(paired.expiresAt * 1000L))
}
InfoRow(label = "Expires", value = expiryLabel)
if (paired.grants.isNotEmpty()) {
val grantsLabel = paired.grants.entries.joinToString(", ") { (k, v) ->
if (v == null) "$k: never" else k
}
InfoRow(label = "Channel grants", value = grantsLabel)
}
val transportLabel = paired.transportHint?.uppercase() ?: "—"
InfoRow(label = "Transport", value = transportLabel)
InfoRow(
label = "Key storage",
value = if (paired.hasHardwareStorage) "Hardware (StrongBox)" else "Hardware (TEE)"
)
}
Spacer(modifier = Modifier.height(4.dp))
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp)
) {
OutlinedButton(
onClick = {
connectionViewModel.clearSession()
onDismiss()
},
modifier = Modifier.fillMaxWidth(0.5f)
) {
Text("Clear Session")
}
OutlinedButton(
onClick = { connectionViewModel.regeneratePairingCode() },
modifier = Modifier.fillMaxWidth()
) {
Text("Regenerate Code")
}
}
}
}
}
// ---------------------------------------------------------------------------
// 2. ApiServerInfoSheet
// ---------------------------------------------------------------------------
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun ApiServerInfoSheet(
connectionViewModel: ConnectionViewModel,
onDismiss: () -> Unit
) {
val sheetState = rememberModalBottomSheetState(skipPartiallyExpanded = false)
val apiServerUrl by connectionViewModel.apiServerUrl.collectAsState()
val apiServerReachable by connectionViewModel.apiServerReachable.collectAsState()
val chatMode by connectionViewModel.chatMode.collectAsState()
val streamingEndpoint by connectionViewModel.streamingEndpoint.collectAsState()
// Reactive flag from AuthManager — updated whenever setApiKey/clearApiKey
// runs and seeded from stored prefs on init.
val apiKeyPresent by connectionViewModel.authManager.apiKeyPresent.collectAsState()
var testing by remember { mutableStateOf(false) }
var testResult by remember { mutableStateOf<String?>(null) }
ModalBottomSheet(
onDismissRequest = onDismiss,
sheetState = sheetState
) {
Column(
modifier = Modifier
.padding(horizontal = 24.dp, vertical = 16.dp)
.navigationBarsPadding(),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
Text(
text = "API server",
style = MaterialTheme.typography.titleLarge
)
Text(
text = "Chat traffic goes directly from the app to the Hermes " +
"API server over HTTP/SSE. The relay is only involved for " +
"terminal and bridge channels.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
HorizontalDivider()
InfoRow(label = "URL", value = apiServerUrl, monospace = true)
ChipRow(label = "Reachable") {
val (label, bg, fg) = if (apiServerReachable) {
Triple(
"Yes",
MaterialTheme.colorScheme.primaryContainer,
MaterialTheme.colorScheme.onPrimaryContainer
)
} else {
Triple(
"No",
MaterialTheme.colorScheme.errorContainer,
MaterialTheme.colorScheme.onErrorContainer
)
}
StatusChip(text = label, background = bg, contentColor = fg)
}
InfoRow(label = "Streaming mode", value = chatMode.toString())
InfoRow(label = "Endpoint preference", value = streamingEndpoint)
InfoRow(
label = "API key set",
value = if (apiKeyPresent) "Yes (hidden)" else "No"
)
InfoRow(
label = "Last health check",
value = if (apiServerReachable) "Just now (ok)" else "Just now (failed)"
)
if (testResult != null) {
Text(
text = testResult!!,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
}
Spacer(modifier = Modifier.height(4.dp))
OutlinedButton(
onClick = {
testing = true
testResult = "Testing\u2026"
connectionViewModel.testApiConnection { success ->
testing = false
testResult = if (success) "Connection OK" else "Connection failed"
}
},
enabled = !testing,
modifier = Modifier.fillMaxWidth()
) {
Text(if (testing) "Testing\u2026" else "Test connection")
}
}
}
}
// ---------------------------------------------------------------------------
// 3. RelayInfoSheet
// ---------------------------------------------------------------------------
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun RelayInfoSheet(
connectionViewModel: ConnectionViewModel,
onDismiss: () -> Unit
) {
val sheetState = rememberModalBottomSheetState(skipPartiallyExpanded = false)
val context = LocalContext.current
val relayUrl by connectionViewModel.relayUrl.collectAsState()
val relayConnectionState by connectionViewModel.relayConnectionState.collectAsState()
val isInsecureConnection by connectionViewModel.isInsecureConnection.collectAsState()
val insecureMode by connectionViewModel.insecureMode.collectAsState()
val relayEnabled by FeatureFlags.relayEnabled(context)
.collectAsState(initial = FeatureFlags.isDevBuild)
ModalBottomSheet(
onDismissRequest = onDismiss,
sheetState = sheetState
) {
Column(
modifier = Modifier
.padding(horizontal = 24.dp, vertical = 16.dp)
.navigationBarsPadding(),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
Text(
text = "Relay",
style = MaterialTheme.typography.titleLarge
)
Text(
text = "The relay is a WebSocket server that brokers terminal " +
"and bridge channels between the app and the host. Connect " +
"automatically after successful pairing.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
HorizontalDivider()
InfoRow(label = "URL", value = relayUrl, monospace = true)
ChipRow(label = "Connection state") { connectionChip(relayConnectionState) }
if (isInsecureConnection) {
Row(
modifier = Modifier
.fillMaxWidth()
.clip(RoundedCornerShape(8.dp))
.background(MaterialTheme.colorScheme.errorContainer)
.padding(horizontal = 12.dp, vertical = 8.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp)
) {
Icon(
imageVector = Icons.Default.Warning,
contentDescription = null,
tint = MaterialTheme.colorScheme.onErrorContainer
)
Text(
text = "ws:// \u2014 traffic not encrypted",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onErrorContainer
)
}
}
InfoRow(
label = "Insecure mode allowed",
value = if (insecureMode) "Yes" else "No"
)
InfoRow(
label = "Relay enabled (feature flag)",
value = if (relayEnabled) "Yes" else "No"
)
Spacer(modifier = Modifier.height(4.dp))
val isConnected = relayConnectionState == ConnectionState.Connected ||
relayConnectionState == ConnectionState.Connecting ||
relayConnectionState == ConnectionState.Reconnecting
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp)
) {
OutlinedButton(
onClick = { connectionViewModel.connectRelay() },
enabled = relayEnabled && !isConnected,
modifier = Modifier.fillMaxWidth(0.5f)
) {
Text("Connect")
}
OutlinedButton(
onClick = { connectionViewModel.disconnectRelay() },
enabled = relayEnabled && isConnected,
modifier = Modifier.fillMaxWidth()
) {
Text("Disconnect")
}
}
}
}
}
@@ -6,15 +6,22 @@ import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.tween
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.KeyboardArrowRight
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Text
import androidx.compose.ui.draw.clip
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
@@ -26,47 +33,80 @@ import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.unit.Dp
import androidx.compose.ui.unit.dp
/**
* Four-state connection indicator. The fourth state — [Probing] — is the
* one that fixes the resume-lag flash: when the app comes back to the
* foreground, every status starts as Probing instead of inheriting whatever
* the StateFlow happened to hold from the last session. The badge then
* resolves to Connected or Disconnected once a fresh health probe lands.
*
* Old call sites that still use the boolean overload keep working — they
* just never enter the Probing state, which is fine for the local-first
* indicators (e.g. terminal session badges) where there's nothing to revalidate.
*/
enum class BadgeState {
/** Verified connected by the most recent probe. Green pulse. */
Connected,
/** Connect/handshake in flight. Amber pulse. */
Connecting,
/** Verified disconnected by the most recent probe. Red, no pulse. */
Disconnected,
/** Status not yet verified — fresh foreground, no probe has landed. Gray pulse. */
Probing,
}
/**
* Animated connection status indicator — a colored dot with an optional pulsing ring.
*
* - **Connected (green):** solid dot + slow heartbeat pulse (1.5 s)
* - **Connecting / Reconnecting (amber):** solid dot + faster pulse (0.8 s)
* - **Probing (gray):** solid dot + slow pulse (1.2 s) — initial state on resume
* - **Disconnected (red):** solid dot, no pulse
*/
@Composable
fun ConnectionStatusBadge(
isConnected: Boolean,
isConnecting: Boolean = false,
state: BadgeState,
modifier: Modifier = Modifier,
size: Dp = 12.dp
size: Dp = 12.dp,
) {
val onSurfaceVariant = MaterialTheme.colorScheme.onSurfaceVariant
val errorColor = MaterialTheme.colorScheme.error
val dotColor: Color
val showPulse: Boolean
val pulseDurationMs: Int
val statusLabel: String
when {
isConnected -> {
when (state) {
BadgeState.Connected -> {
dotColor = Color(0xFF4CAF50) // Material green 500
showPulse = true
pulseDurationMs = 1500
statusLabel = "Connected"
}
isConnecting -> {
BadgeState.Connecting -> {
dotColor = Color(0xFFFFA726) // Material amber/orange 400
showPulse = true
pulseDurationMs = 800
statusLabel = "Connecting"
}
else -> {
dotColor = MaterialTheme.colorScheme.error
BadgeState.Probing -> {
dotColor = onSurfaceVariant
showPulse = true
pulseDurationMs = 1200
statusLabel = "Checking status"
}
BadgeState.Disconnected -> {
dotColor = errorColor
showPulse = false
pulseDurationMs = 1500 // unused but required for val init
statusLabel = "Disconnected"
}
}
// Pulse animation — only runs when showPulse is true
val pulseScale: Float
val pulseAlpha: Float
@@ -100,7 +140,6 @@ fun ConnectionStatusBadge(
.size(size)
.semantics { contentDescription = statusLabel }
.drawBehind {
// Pulse ring (expanding, fading circle outline)
if (showPulse && pulseAlpha > 0f) {
val ringRadius = (this.size.minDimension / 2f) * pulseScale
drawCircle(
@@ -109,34 +148,100 @@ fun ConnectionStatusBadge(
style = Stroke(width = 2.dp.toPx())
)
}
// Solid inner dot
drawCircle(color = dotColor)
}
)
}
/**
* A row showing [ConnectionStatusBadge] alongside a text label and optional status text / test button.
* Boolean overload — preserved for legacy call sites (terminal, bridge,
* chat header, etc.) that don't need a Probing state. New code should pass
* a [BadgeState] directly so it can express the "checking status" pose.
*/
@Composable
fun ConnectionStatusBadge(
isConnected: Boolean,
isConnecting: Boolean = false,
isProbing: Boolean = false,
modifier: Modifier = Modifier,
size: Dp = 12.dp,
) {
val state = when {
isConnected -> BadgeState.Connected
isConnecting -> BadgeState.Connecting
isProbing -> BadgeState.Probing
else -> BadgeState.Disconnected
}
ConnectionStatusBadge(state = state, modifier = modifier, size = size)
}
/**
* A row showing [ConnectionStatusBadge] alongside a text label and status text.
*
* When [onClick] is non-null, the row is rendered as a tappable surface with
* a trailing chevron so the user gets a clear "tap for details" affordance —
* otherwise users have no way to tell that the row opens a drawer.
* When null, the row is static and no chevron is shown.
*
* The old `onTest` trailing-button slot is still supported for call sites
* that want an inline Test button inside the row — distinct from the whole-row
* clickable behavior.
*/
@Composable
fun ConnectionStatusRow(
label: String,
isConnected: Boolean,
isConnecting: Boolean = false,
isProbing: Boolean = false,
statusText: String,
onTest: (() -> Unit)? = null,
onClick: (() -> Unit)? = null,
modifier: Modifier = Modifier
) {
Row(
val state = when {
isConnected -> BadgeState.Connected
isConnecting -> BadgeState.Connecting
isProbing -> BadgeState.Probing
else -> BadgeState.Disconnected
}
ConnectionStatusRow(
label = label,
state = state,
statusText = statusText,
onTest = onTest,
onClick = onClick,
modifier = modifier,
)
}
/**
* Four-state overload of [ConnectionStatusRow]. Prefer this for any new
* call site that needs to render the Probing state — the boolean overload
* is kept around for legacy call sites.
*/
@Composable
fun ConnectionStatusRow(
label: String,
state: BadgeState,
statusText: String,
onTest: (() -> Unit)? = null,
onClick: (() -> Unit)? = null,
modifier: Modifier = Modifier
) {
val interactiveModifier = if (onClick != null) {
Modifier
.clip(RoundedCornerShape(8.dp))
.clickable(onClick = onClick)
.padding(horizontal = 4.dp, vertical = 6.dp)
} else {
Modifier.padding(horizontal = 4.dp, vertical = 6.dp)
}
Row(
modifier = modifier.then(interactiveModifier),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp)
) {
ConnectionStatusBadge(
isConnected = isConnected,
isConnecting = isConnecting
)
ConnectionStatusBadge(state = state)
Text(
text = label,
@@ -147,10 +252,11 @@ fun ConnectionStatusRow(
Text(
text = statusText,
style = MaterialTheme.typography.bodyMedium,
color = when {
isConnected -> Color(0xFF4CAF50)
isConnecting -> Color(0xFFFFA726)
else -> MaterialTheme.colorScheme.error
color = when (state) {
BadgeState.Connected -> Color(0xFF4CAF50)
BadgeState.Connecting -> Color(0xFFFFA726)
BadgeState.Probing -> MaterialTheme.colorScheme.onSurfaceVariant
BadgeState.Disconnected -> MaterialTheme.colorScheme.error
}
)
@@ -160,5 +266,14 @@ fun ConnectionStatusRow(
Text("Test")
}
}
if (onClick != null && onTest == null) {
Icon(
imageVector = Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.size(18.dp)
)
}
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,252 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Warning
import androidx.compose.material3.Button
import androidx.compose.material3.ButtonDefaults
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
/**
* Phase 3 — safety-rails `bridge-safety-rails`
*
* Destructive-verb confirmation modal content. Rendered inside a
* [BridgeStatusOverlay] SYSTEM_ALERT_WINDOW ComposeView — NOT a Compose
* `Dialog`, because Dialogs require an Activity window and we're sitting
* on a WindowManager overlay. The parent overlay supplies the dim + the
* full-screen-touch-interceptor; this composable only draws the card.
*
* Shows the agent's exact requested action + the flagged verb so the
* user isn't guessing what they're allowing. Two buttons:
* - Deny (primary-tonal, safe default) — caller maps to false
* - Allow (tonal with a warning tint) — caller maps to true
*
* Kept UI-layer stateless: both `onAllow` and `onDeny` return directly.
* The callers in [BridgeStatusOverlay] update the overlay registry and
* resolve the pending `CompletableDeferred`.
*/
@Composable
fun DestructiveVerbConfirmDialog(
method: String,
verb: String,
fullText: String,
onAllow: () -> Unit,
onDeny: () -> Unit,
) {
// Root-fills-overlay-with-center-alignment. The overlay already applies
// FLAG_DIM_BEHIND so we only need to draw the card itself.
Box(
modifier = Modifier.fillMaxSize(),
contentAlignment = Alignment.Center,
) {
Card(
modifier = Modifier
.widthIn(max = 360.dp)
.fillMaxWidth(0.92f)
.padding(horizontal = 16.dp),
shape = RoundedCornerShape(20.dp),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surface,
),
elevation = CardDefaults.cardElevation(defaultElevation = 8.dp),
) {
Column(
modifier = Modifier
.fillMaxWidth()
.padding(20.dp),
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(10.dp),
) {
Icon(
imageVector = Icons.Filled.Warning,
contentDescription = null,
tint = Color(0xFFFFA726),
)
Text(
text = "Confirm destructive action",
style = MaterialTheme.typography.titleMedium,
fontWeight = FontWeight.SemiBold,
color = MaterialTheme.colorScheme.onSurface,
)
}
Text(
text = "The Hermes agent is about to ${verbPhrase(method, verb)}.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurface,
)
Box(
modifier = Modifier
.fillMaxWidth()
.background(
MaterialTheme.colorScheme.surfaceVariant,
RoundedCornerShape(10.dp)
)
.padding(12.dp)
) {
Column(verticalArrangement = Arrangement.spacedBy(4.dp)) {
Text(
text = labelFor(method),
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Text(
text = fullText.ifBlank { "(no text)" },
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurface,
)
}
}
Text(
text = "If you didn't expect this, tap Deny.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(modifier = Modifier.height(4.dp))
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(10.dp),
) {
OutlinedButton(
modifier = Modifier.weight(1f),
onClick = onDeny,
) {
Text("Deny")
}
Button(
modifier = Modifier.weight(1f),
onClick = onAllow,
colors = ButtonDefaults.buttonColors(
containerColor = Color(0xFFE53935),
contentColor = Color.White,
),
) {
Text("Allow")
}
}
}
}
}
}
private fun labelFor(method: String): String = when (method) {
"/tap_text" -> "Target text (tap)"
"/type" -> "Text to type"
else -> "Payload"
}
private fun verbPhrase(method: String, verb: String): String {
val action = when (method) {
"/tap_text" -> "tap a button containing"
"/type" -> "type text containing"
else -> "perform an action with"
}
return if (verb.isBlank()) "$action a destructive keyword" else "$action the word \"$verb\""
}
/**
* Small floating status chip shown in the top-right of the screen when
* the user has opted in via `BridgeSafetySettings.statusOverlayEnabled`.
* Pure visual — no gesture handling, the parent overlay is FLAG_NOT_FOCUSABLE
* so taps pass through to whatever's underneath.
*
* Kept in this file so [BridgeStatusOverlay] only imports one component
* package.
*/
@Composable
fun BridgeStatusOverlayChip() {
Box(
modifier = Modifier
.background(
color = Color(0xFF1A1A2E).copy(alpha = 0.85f),
shape = RoundedCornerShape(12.dp),
)
.padding(horizontal = 10.dp, vertical = 6.dp)
) {
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(6.dp),
) {
Box(
modifier = Modifier
.height(8.dp)
.widthIn(min = 8.dp, max = 8.dp)
.background(Color(0xFFE53935), RoundedCornerShape(50))
)
Text(
text = "Hermes active",
style = MaterialTheme.typography.labelSmall,
color = Color.White,
fontWeight = FontWeight.Medium,
)
}
}
}
@Preview(showBackground = true, backgroundColor = 0xFFFFFFFF)
@Composable
private fun BridgeStatusOverlayChipPreview() {
MaterialTheme {
Box(modifier = Modifier.padding(24.dp)) {
BridgeStatusOverlayChip()
}
}
}
@Preview(showBackground = true, backgroundColor = 0x99000000)
@Composable
private fun DestructiveVerbConfirmDialogPreview_TapText() {
MaterialTheme {
DestructiveVerbConfirmDialog(
method = "/tap_text",
verb = "Send",
fullText = "Send $500 to Alice",
onAllow = {},
onDeny = {},
)
}
}
@Preview(showBackground = true, backgroundColor = 0x99000000)
@Composable
private fun DestructiveVerbConfirmDialogPreview_Type() {
MaterialTheme {
DestructiveVerbConfirmDialog(
method = "/type",
verb = "delete",
fullText = "delete all messages in #general",
onAllow = {},
onDeny = {},
)
}
}
@@ -0,0 +1,174 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.LocalHapticFeedback
import androidx.compose.ui.hapticfeedback.HapticFeedbackType
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import com.hermesandroid.relay.viewmodel.TerminalViewModel.SpecialKey
/**
* Horizontal toolbar of keys that the Android soft keyboard doesn't offer:
* ESC, TAB, CTRL (sticky), ALT (sticky), and arrow keys.
*
* Sticky modifier behavior: tapping CTRL or ALT highlights the key and
* applies the modifier to the next character typed (via
* [TerminalViewModel.sendInput]'s translation path). The ViewModel clears
* the flag automatically once a character is sent, so tap → key → back to
* idle matches Termux/JuiceSSH convention.
*/
@Composable
fun ExtraKeysToolbar(
ctrlActive: Boolean,
altActive: Boolean,
onEsc: () -> Unit,
onTab: () -> Unit,
onCtrlToggle: () -> Unit,
onAltToggle: () -> Unit,
onArrow: (SpecialKey) -> Unit,
modifier: Modifier = Modifier
) {
val haptic = LocalHapticFeedback.current
val containerColor = MaterialTheme.colorScheme.surfaceContainerHigh
Row(
modifier = modifier
.fillMaxWidth()
.background(containerColor)
.padding(horizontal = 4.dp, vertical = 4.dp),
horizontalArrangement = Arrangement.spacedBy(4.dp),
verticalAlignment = Alignment.CenterVertically
) {
ToolbarKey(
label = "ESC",
active = false,
weight = 1.2f,
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
onEsc()
}
)
ToolbarKey(
label = "TAB",
active = false,
weight = 1.2f,
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
onTab()
}
)
ToolbarKey(
label = "CTRL",
active = ctrlActive,
weight = 1.3f,
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
onCtrlToggle()
}
)
ToolbarKey(
label = "ALT",
active = altActive,
weight = 1.2f,
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
onAltToggle()
}
)
Spacer(modifier = Modifier.width(4.dp))
ToolbarKey(
label = "\u2190",
active = false,
weight = 1f,
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
onArrow(SpecialKey.ARROW_LEFT)
}
)
ToolbarKey(
label = "\u2193",
active = false,
weight = 1f,
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
onArrow(SpecialKey.ARROW_DOWN)
}
)
ToolbarKey(
label = "\u2191",
active = false,
weight = 1f,
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
onArrow(SpecialKey.ARROW_UP)
}
)
ToolbarKey(
label = "\u2192",
active = false,
weight = 1f,
onClick = {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
onArrow(SpecialKey.ARROW_RIGHT)
}
)
}
}
@Composable
private fun androidx.compose.foundation.layout.RowScope.ToolbarKey(
label: String,
active: Boolean,
weight: Float,
onClick: () -> Unit
) {
val shape = RoundedCornerShape(6.dp)
val scheme = MaterialTheme.colorScheme
val bg = if (active) scheme.primary.copy(alpha = 0.22f) else scheme.surface
val fg = if (active) scheme.primary else scheme.onSurface
val borderColor = if (active) scheme.primary.copy(alpha = 0.6f) else scheme.outlineVariant.copy(alpha = 0.4f)
Box(
modifier = Modifier
.weight(weight)
.heightIn(min = 36.dp)
.height(36.dp)
.clip(shape)
.background(bg, shape)
.border(width = 1.dp, color = borderColor, shape = shape)
.clickable(onClick = onClick),
contentAlignment = Alignment.Center
) {
Text(
text = label,
color = fg,
fontSize = 13.sp,
fontWeight = if (active) FontWeight.SemiBold else FontWeight.Medium,
fontFamily = FontFamily.Monospace
)
}
}
@@ -0,0 +1,332 @@
package com.hermesandroid.relay.ui.components
import android.content.Intent
import android.net.Uri
import androidx.compose.foundation.Image
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.ImageBitmap
import androidx.compose.ui.graphics.asImageBitmap
import androidx.compose.ui.layout.ContentScale
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.Dp
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.data.Attachment
import com.hermesandroid.relay.data.AttachmentRenderMode
import com.hermesandroid.relay.data.AttachmentState
import com.hermesandroid.relay.viewmodel.ChatViewModel
/**
* Discord-style inline render for any attachment — outbound (user-authored)
* or inbound (fetched from the relay via a `MEDIA:hermes-relay://` marker).
*
* Dispatches on (state × renderMode):
* - LOADING → small spinner card, "Tap to download" CTA when
* [Attachment.errorMessage] equals [ChatViewModel.MEDIA_TAP_TO_DOWNLOAD].
* - FAILED → small warning card with retry tap target.
* - LOADED → IMAGE renders inline (bitmap decode), everything else
* renders as a tap-to-open file card that fires ACTION_VIEW
* on the cached content:// URI with FLAG_GRANT_READ_URI_PERMISSION.
*
* Outbound attachments always have [AttachmentState.LOADED] so they take the
* LOADED branch immediately — no behavior change relative to the legacy
* MessageBubble attachment code.
*/
@Composable
fun InboundAttachmentCard(
attachment: Attachment,
onRetry: () -> Unit,
onManualFetch: () -> Unit,
modifier: Modifier = Modifier,
maxWidth: Dp = 280.dp
) {
when (attachment.state) {
AttachmentState.LOADING -> LoadingCard(
attachment = attachment,
onManualFetch = onManualFetch,
modifier = modifier,
maxWidth = maxWidth
)
AttachmentState.FAILED -> FailedCard(
attachment = attachment,
onRetry = onRetry,
modifier = modifier,
maxWidth = maxWidth
)
AttachmentState.LOADED -> LoadedAttachment(
attachment = attachment,
modifier = modifier,
maxWidth = maxWidth
)
}
}
@Composable
private fun LoadingCard(
attachment: Attachment,
onManualFetch: () -> Unit,
modifier: Modifier,
maxWidth: Dp
) {
val isManualCta = attachment.errorMessage == ChatViewModel.MEDIA_TAP_TO_DOWNLOAD
Surface(
shape = RoundedCornerShape(10.dp),
color = MaterialTheme.colorScheme.surfaceVariant,
modifier = modifier
.widthIn(max = maxWidth)
.then(if (isManualCta) Modifier.clickable { onManualFetch() } else Modifier)
) {
Row(
modifier = Modifier.padding(12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(10.dp)
) {
if (isManualCta) {
Text(text = "\u2B07\uFE0F", style = MaterialTheme.typography.titleMedium)
} else {
CircularProgressIndicator(
strokeWidth = 2.dp,
modifier = Modifier.size(18.dp),
color = MaterialTheme.colorScheme.primary
)
}
Column(modifier = Modifier.weight(1f)) {
Text(
text = if (isManualCta) "Tap to download" else "Downloading…",
style = MaterialTheme.typography.bodyMedium,
fontWeight = FontWeight.Medium,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
val subtitle = attachment.fileName
?: attachment.contentType.takeIf { it.isNotBlank() && it != "application/octet-stream" }
if (subtitle != null) {
Text(
text = subtitle,
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.7f)
)
}
}
}
}
}
@Composable
private fun FailedCard(
attachment: Attachment,
onRetry: () -> Unit,
modifier: Modifier,
maxWidth: Dp
) {
Surface(
shape = RoundedCornerShape(10.dp),
color = MaterialTheme.colorScheme.errorContainer,
modifier = modifier
.widthIn(max = maxWidth)
.clickable { onRetry() }
) {
Row(
modifier = Modifier.padding(12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(10.dp)
) {
Text(text = "\u26A0\uFE0F", style = MaterialTheme.typography.titleMedium)
Column(modifier = Modifier.weight(1f)) {
Text(
text = attachment.errorMessage ?: "Attachment failed",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onErrorContainer
)
Text(
text = "Tap to retry",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onErrorContainer.copy(alpha = 0.7f)
)
}
}
}
}
@Composable
private fun LoadedAttachment(
attachment: Attachment,
modifier: Modifier,
maxWidth: Dp
) {
when (attachment.renderMode) {
AttachmentRenderMode.IMAGE -> ImageRender(attachment, modifier, maxWidth)
AttachmentRenderMode.VIDEO,
AttachmentRenderMode.AUDIO,
AttachmentRenderMode.PDF,
AttachmentRenderMode.TEXT,
AttachmentRenderMode.GENERIC -> FileCardRender(attachment, modifier, maxWidth)
}
}
/**
* Inline image render. Prefers [Attachment.cachedUri] (inbound attachments
* written to FileProvider cache), falls back to decoding base64 [Attachment.content]
* for outbound attachments authored by the user.
*/
@Composable
private fun ImageRender(
attachment: Attachment,
modifier: Modifier,
maxWidth: Dp
) {
val context = LocalContext.current
val bitmap: ImageBitmap? = remember(attachment.cachedUri, attachment.content) {
try {
val bytes: ByteArray? = when {
!attachment.cachedUri.isNullOrBlank() -> {
context.contentResolver.openInputStream(Uri.parse(attachment.cachedUri))
?.use { it.readBytes() }
}
attachment.content.isNotBlank() -> {
android.util.Base64.decode(attachment.content, android.util.Base64.DEFAULT)
}
else -> null
}
bytes?.let {
android.graphics.BitmapFactory.decodeByteArray(it, 0, it.size)?.asImageBitmap()
}
} catch (_: Exception) {
null
}
}
if (bitmap != null) {
Image(
bitmap = bitmap,
contentDescription = attachment.fileName,
modifier = modifier
.widthIn(max = maxWidth)
.clip(RoundedCornerShape(8.dp)),
contentScale = ContentScale.FillWidth
)
} else {
// Decode failed — degrade to a generic file card so at least the
// user can tap through to an external viewer.
FileCardRender(
attachment = attachment.copy(contentType = "application/octet-stream"),
modifier = modifier,
maxWidth = maxWidth
)
}
}
@Composable
private fun FileCardRender(
attachment: Attachment,
modifier: Modifier,
maxWidth: Dp
) {
val context = LocalContext.current
val (emoji, typeLabel) = emojiAndLabelFor(attachment.renderMode, attachment.contentType)
Surface(
shape = RoundedCornerShape(10.dp),
color = MaterialTheme.colorScheme.surfaceVariant,
modifier = modifier
.widthIn(max = maxWidth)
.clickable {
val uriStr = attachment.cachedUri
if (!uriStr.isNullOrBlank()) {
try {
val uri = Uri.parse(uriStr)
val intent = Intent(Intent.ACTION_VIEW).apply {
setDataAndType(uri, attachment.contentType)
addFlags(
Intent.FLAG_GRANT_READ_URI_PERMISSION or
Intent.FLAG_ACTIVITY_NEW_TASK
)
}
context.startActivity(intent)
} catch (_: Exception) {
// No viewer installed or malformed URI — silently ignore.
}
}
}
) {
Row(
modifier = Modifier.padding(12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(10.dp)
) {
Box(
modifier = Modifier.size(36.dp),
contentAlignment = Alignment.Center
) {
Text(text = emoji, style = MaterialTheme.typography.headlineSmall)
}
Column(modifier = Modifier.weight(1f)) {
Text(
text = attachment.fileName ?: typeLabel,
style = MaterialTheme.typography.bodyMedium,
fontWeight = FontWeight.Medium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
maxLines = 2
)
val sizeLabel = attachment.fileSize?.let { formatBytes(it) }
val subtitle = listOfNotNull(
typeLabel.takeIf { attachment.fileName != null },
sizeLabel
).joinToString(" \u00B7 ")
if (subtitle.isNotBlank()) {
Text(
text = subtitle,
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.7f)
)
}
}
}
}
}
private fun emojiAndLabelFor(
mode: AttachmentRenderMode,
contentType: String
): Pair<String, String> = when (mode) {
AttachmentRenderMode.IMAGE -> "\uD83D\uDDBC\uFE0F" to "Image"
AttachmentRenderMode.VIDEO -> "\uD83C\uDFAC" to "Video"
AttachmentRenderMode.AUDIO -> "\uD83C\uDFB5" to "Audio"
AttachmentRenderMode.PDF -> "\uD83D\uDCC4" to "PDF"
AttachmentRenderMode.TEXT -> "\uD83D\uDCDD" to contentTypeToLabel(contentType, fallback = "Text")
AttachmentRenderMode.GENERIC -> "\uD83D\uDCCE" to contentTypeToLabel(contentType, fallback = "File")
}
private fun contentTypeToLabel(contentType: String, fallback: String): String {
val bare = contentType.substringBefore(';').trim()
if (bare.isBlank() || bare == "application/octet-stream") return fallback
val subtype = bare.substringAfter('/', missingDelimiterValue = bare)
return subtype.uppercase().take(16)
}
private fun formatBytes(bytes: Long): String {
if (bytes <= 0) return ""
val kb = bytes / 1024.0
val mb = kb / 1024.0
return when {
mb >= 1.0 -> "%.1f MB".format(mb)
kb >= 1.0 -> "%.0f KB".format(kb)
else -> "$bytes B"
}
}
@@ -0,0 +1,136 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.selection.selectable
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Warning
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.RadioButton
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.semantics.Role
import androidx.compose.ui.unit.dp
/**
* One-time acknowledgment dialog shown the first time the user flips
* "Allow insecure connections" on. Explains the threat model in plain
* language and asks the user to confirm *why* they're enabling it.
*
* The selected reason is stored in
* [com.hermesandroid.relay.data.PairingPreferences.setInsecureReason]
* and drives the label on the [TransportSecurityBadge]. It is **not** used
* to gate anything — Bailey's explicit call is "trust the user's judgment,
* just display what they told us".
*
* State management is owned by the caller:
* - Caller tracks `showDialog` as a boolean
* - On dismiss without a reason, caller should revert the toggle
* - On confirm with a reason, caller persists both the reason and
* `insecureAckSeen = true`, then leaves the toggle enabled
*/
@Composable
fun InsecureConnectionAckDialog(
onConfirm: (reason: String) -> Unit,
onCancel: () -> Unit,
) {
var selectedReason by remember { mutableStateOf<String?>(null) }
val reasonOptions = listOf(
"lan_only" to "LAN only (trusted network)",
"tailscale_vpn" to "Tailscale or VPN",
"local_dev" to "Local development only",
)
AlertDialog(
onDismissRequest = onCancel,
icon = {
Icon(
imageVector = Icons.Filled.Warning,
contentDescription = null,
tint = MaterialTheme.colorScheme.error
)
},
title = {
Text(
text = "Allow insecure connections?",
style = MaterialTheme.typography.titleLarge
)
},
text = {
Column(
verticalArrangement = Arrangement.spacedBy(12.dp),
modifier = Modifier.fillMaxWidth()
) {
Text(
text = "Insecure mode lets this app connect over plain " +
"ws:// and http://. Anyone on the network between " +
"your phone and the server can read your chat " +
"messages, session tokens, and terminal traffic.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
Text(
text = "Only use this on networks you control. Pick " +
"the reason that best describes your setup:",
style = MaterialTheme.typography.bodyMedium
)
Spacer(Modifier.height(4.dp))
Column {
reasonOptions.forEach { (key, label) ->
Row(
modifier = Modifier
.fillMaxWidth()
.selectable(
selected = selectedReason == key,
onClick = { selectedReason = key },
role = Role.RadioButton
)
.padding(vertical = 4.dp),
verticalAlignment = Alignment.CenterVertically
) {
RadioButton(
selected = selectedReason == key,
onClick = { selectedReason = key }
)
Text(
text = label,
style = MaterialTheme.typography.bodyMedium,
modifier = Modifier.padding(start = 8.dp)
)
}
}
}
}
},
confirmButton = {
TextButton(
enabled = selectedReason != null,
onClick = { selectedReason?.let(onConfirm) }
) {
Text("I understand")
}
},
dismissButton = {
TextButton(onClick = onCancel) {
Text("Cancel")
}
}
)
}
@@ -6,17 +6,22 @@ import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.tween
import androidx.compose.foundation.ExperimentalFoundationApi
import androidx.compose.foundation.background
import androidx.compose.foundation.combinedClickable
import androidx.compose.foundation.isSystemInDarkTheme
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxHeight
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.ui.draw.clip
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.text.selection.SelectionContainer
import androidx.compose.material3.MaterialTheme
@@ -24,16 +29,13 @@ import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.draw.clip
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.semantics.contentDescription
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.unit.Dp
import androidx.compose.ui.unit.dp
import androidx.compose.ui.graphics.asImageBitmap
import androidx.compose.ui.unit.sp
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.MessageRole
@@ -51,15 +53,48 @@ fun MessageBubble(
showThinking: Boolean = true,
isFirstInGroup: Boolean = true,
isLastInGroup: Boolean = true,
onCopyMessage: (String) -> Unit = {}
onCopyMessage: (String) -> Unit = {},
/**
* Invoked when the user taps a FAILED inbound attachment card.
* `attachmentIndex` is the position in [ChatMessage.attachments] so the
* ViewModel can re-fetch the exact placeholder that needs re-trying.
*/
onAttachmentRetry: (messageId: String, attachmentIndex: Int) -> Unit = { _, _ -> },
/**
* Invoked when the user taps a LOADING+"Tap to download" placeholder
* (the cellular deferral case).
*/
onAttachmentManualFetch: (messageId: String, attachmentIndex: Int) -> Unit = { _, _ -> }
) {
val isUser = message.role == MessageRole.USER
val isSystem = message.role == MessageRole.SYSTEM
val backgroundColor = when (message.role) {
MessageRole.USER -> MaterialTheme.colorScheme.primary
MessageRole.ASSISTANT -> MaterialTheme.colorScheme.surfaceVariant
MessageRole.SYSTEM -> MaterialTheme.colorScheme.tertiaryContainer
// Phone/voice-origin action bubble marker.
//
// Voice mode (sideload classifier → RealVoiceBridgeIntentHandler) emits
// these with `agentName = "Voice action"` and an id prefixed
// `voice-intent-action-*` / `voice-intent-result-*`. Chat mode parity
// (ChatHandler.onToolCallComplete for android_* tools) emits them with
// `agentName = "Phone action"`. We match on either so a single render
// branch applies the accent to both origins.
//
// Chosen marker: subtle thin vertical accent bar on the leading edge of
// the bubble in colorScheme.tertiary, so an action bubble is
// immediately distinguishable from a regular LLM reply when they
// interleave in the scrollback. Subtle on purpose — the content still
// carries the signal, the bar just flags "this was a phone control
// action, not LLM narration".
val isActionBubble = !isUser && !isSystem && (
message.agentName == "Voice action" ||
message.agentName == "Phone action" ||
message.id.startsWith("voice-intent-")
)
val backgroundColor = when {
message.role == MessageRole.USER -> MaterialTheme.colorScheme.primary
message.role == MessageRole.SYSTEM -> MaterialTheme.colorScheme.tertiaryContainer
isActionBubble -> MaterialTheme.colorScheme.tertiaryContainer.copy(alpha = 0.45f)
else -> MaterialTheme.colorScheme.surfaceVariant
}
val textColor = when (message.role) {
@@ -110,12 +145,31 @@ fun MessageBubble(
)
}
// Message bubble
// Message bubble.
//
// Action bubbles (voice/phone origin) wrap the existing Surface in
// a Row with a thin leading tertiary-colored accent bar. The bar
// is rendered as a separate Box so it hugs the bubble's left edge
// regardless of content height (tall bubbles with multi-line
// markdown stretch the bar via fillMaxHeight + IntrinsicSize).
Row(
modifier = Modifier.widthIn(max = maxBubbleWidth),
verticalAlignment = Alignment.Top,
) {
if (isActionBubble) {
Box(
modifier = Modifier
.padding(top = 8.dp, bottom = 8.dp, end = 6.dp)
.width(3.dp)
.height(if (message.content.isBlank()) 14.dp else 24.dp)
.clip(CircleShape)
.background(MaterialTheme.colorScheme.tertiary.copy(alpha = 0.85f))
)
}
Surface(
shape = bubbleShape,
color = backgroundColor,
modifier = Modifier
.widthIn(max = maxBubbleWidth)
.then(
if (!isUser && !isSystem && isDarkTheme) {
Modifier.leftEdgeGlow(
@@ -151,46 +205,22 @@ fun MessageBubble(
}
}
// Attachments
// Attachments — dispatched through the unified InboundAttachmentCard
// so outbound and inbound attachments share the same render pipeline.
// Outbound attachments (user-authored) always have state=LOADED so
// they route straight to the LOADED branch; inbound attachments (via
// MEDIA markers) cycle through LOADING → LOADED / FAILED as the
// background fetch progresses.
if (message.attachments.isNotEmpty()) {
Spacer(modifier = Modifier.height(4.dp))
message.attachments.forEach { attachment ->
if (attachment.isImage) {
val imageBitmap = remember(attachment.content) {
try {
val bytes = android.util.Base64.decode(attachment.content, android.util.Base64.DEFAULT)
android.graphics.BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
?.asImageBitmap()
} catch (_: Exception) { null as androidx.compose.ui.graphics.ImageBitmap? }
}
if (imageBitmap != null) {
androidx.compose.foundation.Image(
bitmap = imageBitmap,
contentDescription = attachment.fileName,
modifier = Modifier
.widthIn(max = maxBubbleWidth - 24.dp)
.clip(RoundedCornerShape(8.dp))
.padding(vertical = 2.dp),
contentScale = androidx.compose.ui.layout.ContentScale.FillWidth
)
}
} else {
Row(
modifier = Modifier.padding(vertical = 2.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(4.dp)
) {
Text(
text = "\uD83D\uDCC4",
style = MaterialTheme.typography.bodyMedium
)
Text(
text = attachment.fileName ?: attachment.contentType,
style = MaterialTheme.typography.labelSmall,
color = textColor.copy(alpha = 0.8f)
)
}
}
message.attachments.forEachIndexed { index, attachment ->
InboundAttachmentCard(
attachment = attachment,
onRetry = { onAttachmentRetry(message.id, index) },
onManualFetch = { onAttachmentManualFetch(message.id, index) },
maxWidth = maxBubbleWidth - 24.dp,
modifier = Modifier.padding(vertical = 2.dp)
)
}
}
@@ -220,6 +250,7 @@ fun MessageBubble(
}
}
}
} // end Row (bubble + optional leading accent bar)
}
}
@@ -56,6 +56,10 @@ enum class SphereState {
Thinking,
/** Energy radiates outward, strong ripples, focused eye. Speaking. */
Streaming,
/** Voice mode — listening to user. Cool palette, subtle amplitude-driven motion. */
Listening,
/** Voice mode — speaking to user. Warm core, dramatic amplitude-driven motion. */
Speaking,
/** Red shift, erratic motion. Something wrong. */
Error
}
@@ -99,6 +103,20 @@ private fun paramsFor(state: SphereState) = when (state) {
coreTightness = 0.60f, turbulenceAmp = 0.08f,
rippleScale = 2.0f, heartbeatSpeed = 1.5f, radialFlowSpeed = 0.5f
)
SphereState.Listening -> SphereParams(
// Calm base — voiceAmplitude modulates on top (see render loop).
breatheSpeed = 0.55f, breatheAmp = 0.035f,
lightSpeedX = 0.22f, lightSpeedY = 0.16f, lightInfluence = 0.38f,
coreTightness = 0.78f, turbulenceAmp = 0.05f,
rippleScale = 0.9f, heartbeatSpeed = 1.2f, radialFlowSpeed = 0.18f
)
SphereState.Speaking -> SphereParams(
// Assertive base — amplitude pushes it dramatically further.
breatheSpeed = 0.45f, breatheAmp = 0.05f,
lightSpeedX = 0.20f, lightSpeedY = 0.14f, lightInfluence = 0.30f,
coreTightness = 0.55f, turbulenceAmp = 0.07f,
rippleScale = 1.8f, heartbeatSpeed = 1.8f, radialFlowSpeed = 0.45f
)
SphereState.Error -> SphereParams(
breatheSpeed = 1.2f, breatheAmp = 0.03f,
lightSpeedX = 0.7f, lightSpeedY = 0.6f, lightInfluence = 0.40f,
@@ -120,6 +138,17 @@ private fun colorsFor(state: SphereState) = when (state) {
0.20f, 0.90f, 0.50f, // green
0.25f, 0.80f, 0.85f // teal
)
SphereState.Listening -> SphereColors(
// Cool soft blue/purple — cooler than Idle's green/purple.
0.35f, 0.55f, 0.95f, // #597EF2 soft blue
0.65f, 0.45f, 0.95f // #A573F2 soft purple
)
SphereState.Speaking -> SphereColors(
// Vibrant green/teal — same family as Streaming but punchier.
// The render loop pushes core toward white as amplitude peaks.
0.25f, 0.92f, 0.55f, // #40EB8C vivid green
0.30f, 0.85f, 0.88f // #4DD9E0 teal
)
SphereState.Error -> SphereColors(
0.90f, 0.30f, 0.25f, // red
0.85f, 0.50f, 0.20f // orange
@@ -134,25 +163,71 @@ fun MorphingSphere(
state: SphereState = SphereState.Idle,
intensity: Float = 0f,
toolCallBurst: Float = 0f,
voiceAmplitude: Float = 0f,
voiceMode: Boolean = false,
fixedTime: Float? = null,
fixedColorPhase: Float? = null
) {
// Clamp amplitude once — downstream math assumes 0..1.
val amp = voiceAmplitude.coerceIn(0f, 1f)
// ── Animated state parameters (smooth 800ms transitions) ─────
val targetP = remember(state) { paramsFor(state) }
val targetC = remember(state) { colorsFor(state) }
val spec = tween<Float>(800, easing = FastOutSlowInEasing)
val breatheSpeed by animateFloatAsState(targetP.breatheSpeed, spec, label = "bSpd")
// ── voiceMode expansion scalar ───────────────────────────────
// 1.0 = normal, ~1.08 = expanded (bounded to avoid data-ring overflow).
val voiceRadiusScale by animateFloatAsState(
targetValue = if (voiceMode) 1.08f else 1.0f,
animationSpec = tween(600, easing = FastOutSlowInEasing),
label = "voiceExpand"
)
val baseBreatheSpeed by animateFloatAsState(targetP.breatheSpeed, spec, label = "bSpd")
val breatheAmp by animateFloatAsState(targetP.breatheAmp, spec, label = "bAmp")
val lightSpeedX by animateFloatAsState(targetP.lightSpeedX, spec, label = "lsX")
val lightSpeedY by animateFloatAsState(targetP.lightSpeedY, spec, label = "lsY")
val lightInfluence by animateFloatAsState(targetP.lightInfluence, spec, label = "lInf")
val coreTightness by animateFloatAsState(targetP.coreTightness, spec, label = "core")
val turbulenceAmp by animateFloatAsState(targetP.turbulenceAmp, spec, label = "turb")
val baseTurbulence by animateFloatAsState(targetP.turbulenceAmp, spec, label = "turb")
val rippleScale by animateFloatAsState(targetP.rippleScale, spec, label = "rip")
val heartbeatSpeed by animateFloatAsState(targetP.heartbeatSpeed, spec, label = "hb")
val radialFlowSpeed by animateFloatAsState(targetP.radialFlowSpeed, spec, label = "rf")
// ── Voice amplitude modulation ────────────────────────────────
// Listening = subtle (≤30% boost); Speaking = dramatic (up to 3×).
// Idle/Thinking/Streaming/Error ignore amplitude — existing behavior preserved.
val breatheSpeed = when (state) {
SphereState.Listening -> lerp(baseBreatheSpeed, baseBreatheSpeed * 1.3f, amp * 0.5f)
SphereState.Speaking -> lerp(baseBreatheSpeed, baseBreatheSpeed * 2.0f, amp)
else -> baseBreatheSpeed
}
val turbulenceAmp = when (state) {
SphereState.Listening -> baseTurbulence + amp * 0.15f
SphereState.Speaking -> baseTurbulence + amp * 0.5f
else -> baseTurbulence
}
// Core warmth: 0.30 is the existing constant baked into the render loop's
// warmth term (see line where `warmth = (1f - normDist^2) * 0.12f` is mixed).
// Speaking pushes this multiplier from 0.3 → 1.0 as amplitude rises, driving
// the core bright→white. Listening holds at 0.3 (no change vs. other states).
val coreWarmth = when (state) {
SphereState.Speaking -> lerp(0.30f, 1.0f, amp)
else -> 0.30f
}
// Perimeter wobble — existing code uses a fixed 0.06 multiplier.
val wobbleAmplitude = when (state) {
SphereState.Listening -> 0.06f * (1f + amp * 0.3f)
SphereState.Speaking -> 0.06f * (1f + amp * 0.8f)
else -> 0.06f
}
// Data ring orbit speed — existing code uses `t * 0.4f`.
val dataRingSpeed = when (state) {
SphereState.Speaking -> 0.4f * (1f + amp * 3f)
else -> 0.4f
}
val cr1 by animateFloatAsState(targetC.r1, spec, label = "cr1")
val cg1 by animateFloatAsState(targetC.g1, spec, label = "cg1")
val cb1 by animateFloatAsState(targetC.b1, spec, label = "cb1")
@@ -216,10 +291,12 @@ fun MorphingSphere(
val cy = rows / 2f
val charAspect = cellW / cellH
// Reduced from 0.72 so data ring (1.55x) fits within grid
// Reduced from 0.72 so data ring (1.55x) fits within grid.
// voiceRadiusScale is ~1.08 in voiceMode, 1.0 otherwise — bounded so the
// data ring outer edge (1.55x) still stays within the drawable region.
val maxRadiusFromRows = (rows / 2f) * 0.60f
val maxRadiusFromCols = (cols / 2f) * charAspect * 0.60f
val baseRadius = minOf(maxRadiusFromRows, maxRadiusFromCols)
val baseRadius = minOf(maxRadiusFromRows, maxRadiusFromCols) * voiceRadiusScale
val t = time
// ── Breathing ────────────────────────────────────────────
@@ -261,12 +338,12 @@ fun MorphingSphere(
val dist = sqrt(dx * dx + dy * dy)
val angle = atan2(dy, dx)
// ── Perimeter (subtle 6% wobble) ────────────────
// ── Perimeter (subtle 6% wobble — amplified by voice) ───
val perimeterNoise = fbm(
angle * 1.8f + t * 0.08f,
angle * 0.7f + t * 0.12f
) * 2f - 1f
val distortedRadius = breathingRadius * (1f + perimeterNoise * 0.06f)
val distortedRadius = breathingRadius * (1f + perimeterNoise * wobbleAmplitude)
val glowRadius = distortedRadius * 1.35f
val dataRingInner = distortedRadius * 1.40f
val dataRingOuter = distortedRadius * 1.55f
@@ -355,8 +432,10 @@ fun MorphingSphere(
val alpha = ((brightness * 0.4f + 0.6f) * edgeFade * scanline)
.coerceIn(0.1f, 1f)
// Core warmth (center bleeds towards white)
val warmth = (1f - normDist * normDist) * 0.12f
// Core warmth (center bleeds towards white).
// coreWarmth is 0.30 for all non-voice states (→ 0.12 multiplier,
// the historical value) and scales up to 1.0 when Speaking peaks.
val warmth = (1f - normDist * normDist) * (coreWarmth * 0.40f)
val lightBoost = directionalLight * 0.08f
paint.color = android.graphics.Color.argb(
@@ -406,8 +485,9 @@ fun MorphingSphere(
val ringT = (dist - dataRingInner) / (dataRingOuter - dataRingInner)
// Orbiting: offset angle by time (different layers at different speeds)
val orbitAngle = angle - t * 0.4f + ringT * 1.5f
// Orbiting: offset angle by time (different layers at different speeds).
// dataRingSpeed is 0.4 default, spun up to ~1.6 at Speaking peak.
val orbitAngle = angle - t * dataRingSpeed + ringT * 1.5f
// Sparsity: only render ~15% of ring positions
val ringNoise = fbm(
orbitAngle * 4f + t * 0.3f,
@@ -517,3 +597,48 @@ private fun PreviewCompact() {
MorphingSphere(Modifier.fillMaxSize(), fixedTime = 8f, fixedColorPhase = 2.0f)
}
}
@Preview(name = "Listening", showBackground = true, backgroundColor = 0xFF0A0A0A, widthDp = 360, heightDp = 640)
@Composable
fun MorphingSphereListeningPreview() {
Box(Modifier.fillMaxSize().background(Color(0xFF0A0A0A))) {
MorphingSphere(
modifier = Modifier.fillMaxSize(),
state = SphereState.Listening,
voiceAmplitude = 0.4f,
voiceMode = true,
fixedTime = 5f,
fixedColorPhase = 1.0f
)
}
}
@Preview(name = "Speaking (low)", showBackground = true, backgroundColor = 0xFF0A0A0A, widthDp = 360, heightDp = 640)
@Composable
fun MorphingSphereSpeakingLowPreview() {
Box(Modifier.fillMaxSize().background(Color(0xFF0A0A0A))) {
MorphingSphere(
modifier = Modifier.fillMaxSize(),
state = SphereState.Speaking,
voiceAmplitude = 0.2f,
voiceMode = true,
fixedTime = 5f,
fixedColorPhase = 2.0f
)
}
}
@Preview(name = "Speaking (peak)", showBackground = true, backgroundColor = 0xFF0A0A0A, widthDp = 360, heightDp = 640)
@Composable
fun MorphingSphereSpeakingPeakPreview() {
Box(Modifier.fillMaxSize().background(Color(0xFF0A0A0A))) {
MorphingSphere(
modifier = Modifier.fillMaxSize(),
state = SphereState.Speaking,
voiceAmplitude = 0.95f,
voiceMode = true,
fixedTime = 5f,
fixedColorPhase = 2.5f
)
}
}
@@ -7,11 +7,20 @@ import androidx.camera.core.ImageAnalysis
import androidx.camera.core.Preview
import androidx.camera.lifecycle.ProcessCameraProvider
import androidx.camera.view.PreviewView
import androidx.compose.animation.core.FastOutSlowInEasing
import androidx.compose.animation.core.RepeatMode
import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.animateFloatAsState
import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.spring
import androidx.compose.animation.core.tween
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.aspectRatio
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
@@ -27,6 +36,7 @@ import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
@@ -34,65 +44,285 @@ import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.StrokeCap
import androidx.compose.ui.layout.onSizeChanged
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.platform.LocalDensity
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.IntSize
import androidx.compose.ui.unit.dp
import androidx.compose.ui.viewinterop.AndroidView
import androidx.compose.foundation.Canvas
import androidx.core.content.ContextCompat
import androidx.lifecycle.compose.LocalLifecycleOwner
import com.google.mlkit.vision.barcode.BarcodeScanning
import com.google.mlkit.vision.barcode.common.Barcode
import com.google.mlkit.vision.common.InputImage
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import java.util.concurrent.atomic.AtomicBoolean
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.int
import kotlinx.serialization.json.intOrNull
import kotlinx.serialization.json.jsonPrimitive
import java.util.concurrent.Executors
import kotlin.math.max
/**
* Parsed result from a Hermes pairing QR code.
*
* QR payload format:
* **Supported versions:** v1 and v2.
*
* v1 (legacy, pre-2026-04-11):
* ```json
* {"hermes":1,"host":"172.16.24.250","port":8642,"key":"bearer-token","tls":false}
* {
* "hermes": 1,
* "host": "172.16.24.250",
* "port": 8642,
* "key": "bearer-token",
* "tls": false,
* "relay": { "url": "ws://172.16.24.250:8767", "code": "ABCD12" }
* }
* ```
*
* v2 (security overhaul, 2026-04-11):
* ```json
* {
* "hermes": 2,
* "host": "192.168.1.100",
* "port": 8642,
* "key": "optional-api-key",
* "tls": true,
* "relay": {
* "url": "ws://192.168.1.100:8767",
* "code": "ABC123",
* "ttl_seconds": 2592000,
* "grants": { "terminal": 2592000, "bridge": 604800 },
* "transport_hint": "wss"
* },
* "sig": "base64-hmac-sha256"
* }
* ```
*
* The top-level fields configure the direct-chat Hermes API server. The
* optional [relay] block configures the Hermes-Relay WSS connection used by
* the terminal and bridge channels.
*
* **Forward/backward compatibility:**
* - `hermes` now has a default of `1` so v1 QRs without the field parse.
* - `sig` is captured but **not verified** — we don't have the server's
* HMAC secret. Stored for future verification and for operator audit.
* TODO: once the server exposes a pairing public key, verify.
* - Unknown fields are tolerated via `ignoreUnknownKeys = true`. v3+ QRs
* will still parse on this phone.
* - The `ttl_seconds`, `grants`, and `transport_hint` fields on [RelayPairing]
* are nullable so v1 QRs with only `url` + `code` still deserialize.
*
* Old QRs without the relay block still parse cleanly because the field is
* nullable.
*/
@Serializable
data class HermesPairingPayload(
val hermes: Int,
val host: String,
val hermes: Int = 1,
val host: String = "",
val port: Int = 8642,
val key: String = "",
val tls: Boolean = false
val tls: Boolean = false,
val relay: RelayPairing? = null,
val sig: String? = null,
) {
/** Build the full server URL from host, port, and tls flag. */
/** Build the full API server URL from host, port, and tls flag. */
val serverUrl: String
get() = "${if (tls) "https" else "http"}://$host:$port"
}
private val json = Json { ignoreUnknownKeys = true }
/**
* Relay connection details carried in a Hermes pairing QR.
*
* - [url] is the full WebSocket URL the phone should connect to, e.g.
* `ws://172.16.24.250:8767` for dev or `wss://relay.example.com:8767`
* for a TLS-fronted relay.
* - [code] is a 6-char one-shot pairing code that the relay has already
* registered via its localhost-only `/pairing/register` endpoint. The
* phone sends this code in its first `system/auth` envelope; the relay
* consumes it and returns a long-lived session token for subsequent
* reconnects.
* - [ttlSeconds] is an operator-preselected session TTL. When non-null the
* [com.hermesandroid.relay.ui.components.SessionTtlPickerDialog] defaults
* to this value so users can override it if they want. `0` means "never
* expire"; `null`/missing means "use the phone-side default".
* - [grants] is an optional per-channel TTL map. Keys are channel names
* (`"chat"`, `"terminal"`, `"bridge"`); values are seconds. When the
* phone authenticates it includes these grants in its auth envelope so
* the relay can issue channel-specific tokens.
* - [transportHint] is `"wss"` / `"ws"` / `null`. Drives the default TTL
* selection and the [com.hermesandroid.relay.ui.components.TransportSecurityBadge]
* label.
*/
@Serializable
data class RelayPairing(
val url: String = "",
val code: String = "",
@SerialName("ttl_seconds")
val ttlSeconds: Long? = null,
val grants: Map<String, Long>? = null,
@SerialName("transport_hint")
val transportHint: String? = null,
)
private val json = Json {
ignoreUnknownKeys = true
isLenient = true
coerceInputValues = true
}
/**
* Try to parse a scanned string as a Hermes pairing QR payload.
* Returns null if it's not a valid Hermes QR (no "hermes":1 field).
*
* Accepts both v1 and v2 (or anything without a `hermes` field — we default
* to `1`). Returns null when the payload is not valid JSON, has no `host`
* field, or fails strict decoding.
*/
fun parseHermesPairingQr(raw: String): HermesPairingPayload? {
return try {
// Quick check: must contain "hermes" key with value 1
// Quick check: must contain a `host` field and be valid JSON. We no
// longer reject based on the `hermes` version int — future v3+ QRs
// should still parse on this phone so Bailey doesn't have to ship a
// whole release to keep up with wire-format growth.
val obj = json.decodeFromString<JsonObject>(raw)
val version = obj["hermes"]?.jsonPrimitive?.int ?: return null
if (version != 1) return null
json.decodeFromString<HermesPairingPayload>(raw)
val version = obj["hermes"]?.jsonPrimitive?.intOrNull ?: 1
if (version < 1) return null
val decoded = json.decodeFromString<HermesPairingPayload>(raw)
if (decoded.host.isBlank()) return null
// TODO(security): verify `decoded.sig` against the server's HMAC
// secret once the pairing protocol exposes a public verification
// path. For now we parse and store the signature but do not reject
// unsigned payloads — the phone has no way to fetch the server's
// secret in-band.
decoded
} catch (_: Exception) {
null
}
}
/**
* A bounding rect in *viewport* pixel coordinates (top-left origin), produced
* by mapping a barcode's image-space bounding box through the camera rotation
* + FILL_CENTER scale of the PreviewView. Used to drive the dynamic
* "snap-to-QR" corner brackets in [ScannerCornersOverlay].
*/
private data class ViewportRect(
val left: Float,
val top: Float,
val right: Float,
val bottom: Float,
)
/**
* One L-shaped corner bracket — origin point + the two arm endpoints + its
* core/glow colors. Pulled out to a top-level class so the draw loop can be
* a regular `for` over a typed list (Kotlin local data classes inside
* lambdas have edge-case restrictions; safer to declare here).
*/
private data class CornerBracket(
val origin: Offset,
val horiz: Offset,
val vert: Offset,
val core: Color,
val glow: Color,
)
/**
* Map a barcode bounding box in **image buffer coordinates** through the
* camera rotation and FILL_CENTER scaling of a square viewport, returning
* the rect in viewport pixel coordinates.
*
* Math notes:
* - The camera buffer arrives in sensor orientation (typically landscape
* e.g. 1280×720), with [rotationDegrees] indicating how many degrees the
* image needs to be rotated CW to display upright on the device.
* - We rotate the bounding box first, then scale-and-offset it into the
* viewport. FILL_CENTER picks the *larger* of (vp/imgW, vp/imgH) so the
* image fully covers the viewport (cropping the longer side).
* - For 90°/270° rotations the post-rotation dimensions are swapped.
*/
private fun mapBoxToViewport(
box: android.graphics.Rect,
imgW: Int,
imgH: Int,
rotationDegrees: Int,
viewportSize: IntSize,
): ViewportRect {
// Rotate the box into display orientation.
val rotated = when (rotationDegrees) {
90 -> floatArrayOf(
(imgH - box.bottom).toFloat(),
box.left.toFloat(),
(imgH - box.top).toFloat(),
box.right.toFloat(),
)
180 -> floatArrayOf(
(imgW - box.right).toFloat(),
(imgH - box.bottom).toFloat(),
(imgW - box.left).toFloat(),
(imgH - box.top).toFloat(),
)
270 -> floatArrayOf(
box.top.toFloat(),
(imgW - box.right).toFloat(),
box.bottom.toFloat(),
(imgW - box.left).toFloat(),
)
else -> floatArrayOf(
box.left.toFloat(),
box.top.toFloat(),
box.right.toFloat(),
box.bottom.toFloat(),
)
}
val rotW = if (rotationDegrees == 90 || rotationDegrees == 270) imgH else imgW
val rotH = if (rotationDegrees == 90 || rotationDegrees == 270) imgW else imgH
// FILL_CENTER: the image is scaled to fully cover the viewport, then
// centered. The visible portion is the central `viewport`-sized window
// of the scaled image. We map by applying the scale + the centering offset.
val vpW = viewportSize.width.toFloat()
val vpH = viewportSize.height.toFloat()
val scale = max(vpW / rotW, vpH / rotH)
val scaledW = rotW * scale
val scaledH = rotH * scale
val offsetX = (vpW - scaledW) / 2f
val offsetY = (vpH - scaledH) / 2f
return ViewportRect(
left = (rotated[0] * scale + offsetX).coerceIn(0f, vpW),
top = (rotated[1] * scale + offsetY).coerceIn(0f, vpH),
right = (rotated[2] * scale + offsetX).coerceIn(0f, vpW),
bottom = (rotated[3] * scale + offsetY).coerceIn(0f, vpH),
)
}
/**
* Full-screen QR code scanner overlay.
* Detects Hermes pairing QR codes and calls [onPairingDetected] with the parsed payload.
*
* Layout:
* - Header bar with a Close button + "Scan Hermes QR" title
* - Square camera viewport at 50% of the screen width, with rounded corners
* - Sci-fi L-bracket overlay drawn on top of the viewport. When no QR is
* in frame the brackets sit at a centered "ready" position with a slow
* pulse animation. When a barcode is detected the brackets snap (with
* a spring) to the bounding box of the QR — defining a live "lock-on"
* indicator. Brackets release back to the centered ready state ~600ms
* after the QR leaves the frame.
* - Instruction copy below
*
* Detects Hermes pairing QR codes and calls [onPairingDetected] with the
* parsed payload after a brief delay, so the user actually sees the lock-on
* snap animation before the screen transitions away.
*/
@Composable
fun QrPairingScanner(
@@ -105,6 +335,30 @@ fun QrPairingScanner(
val hasDetected = remember { AtomicBoolean(false) }
val cameraProviderRef = remember { mutableStateOf<ProcessCameraProvider?>(null) }
// Viewport is sized at 50% of the screen width via Modifier.fillMaxWidth(0.5f)
// below — comfortable scan target without dominating the screen, and
// matches the "futuristic scan port" aesthetic the brackets are drawn around.
// Live viewport pixel size — captured via onSizeChanged so the analyzer
// thread can compute viewport-space coordinates for the corner brackets.
var viewportSizePx by remember { mutableStateOf(IntSize.Zero) }
// Latest detected QR bounding box in viewport pixel coordinates. Updated
// continuously by the analyzer for any successfully decoded QR (not just
// valid Hermes ones). null = no current detection → brackets fall back
// to centered ready position.
var detectedBox by remember { mutableStateOf<ViewportRect?>(null) }
// Frame counter from the analyzer — bumped every analyzed frame so the
// "release back to ready position" timer can detect when detections stop
// arriving. Volatile because it's written from the camera executor thread
// and read from the main thread coroutine.
var lastDetectionAtMs by remember { mutableStateOf(0L) }
// Lock-on state — set true when we've parsed a valid Hermes payload.
// Drives the brief settle delay before navigating away so the user sees
// the snap animation actually land on the QR.
var lockedPayload by remember { mutableStateOf<HermesPairingPayload?>(null) }
val cameraExecutor = remember { Executors.newSingleThreadExecutor() }
DisposableEffect(Unit) {
onDispose {
@@ -113,6 +367,26 @@ fun QrPairingScanner(
}
}
// Release the brackets back to the centered ready position when no
// detection has arrived for ~600ms. Otherwise a stale detection from
// a frame ago would keep the brackets "stuck" off-center after the QR
// has left the frame.
LaunchedEffect(lastDetectionAtMs) {
if (detectedBox == null) return@LaunchedEffect
kotlinx.coroutines.delay(600)
if (System.currentTimeMillis() - lastDetectionAtMs >= 600) {
detectedBox = null
}
}
// After we lock on a valid Hermes payload, hold the snap animation for
// ~450ms so the user perceives the lock-on, then forward to onPairingDetected.
LaunchedEffect(lockedPayload) {
val payload = lockedPayload ?: return@LaunchedEffect
kotlinx.coroutines.delay(450)
onPairingDetected(payload)
}
Box(
modifier = Modifier
.fillMaxSize()
@@ -146,13 +420,18 @@ fun QrPairingScanner(
)
}
Spacer(modifier = Modifier.height(32.dp))
Spacer(modifier = Modifier.height(24.dp))
// Camera preview
// Camera preview viewport (75% of screen width, square). Wider
// than the original 50% pass — a generous scan target makes
// framing the QR effortless and gives the bracket animations
// more room to read as a "lock-on" instead of a tiny pop.
Box(
modifier = Modifier
.size(280.dp)
.clip(RoundedCornerShape(16.dp)),
.fillMaxWidth(0.75f)
.aspectRatio(1f)
.clip(RoundedCornerShape(20.dp))
.onSizeChanged { viewportSizePx = it },
contentAlignment = Alignment.Center
) {
AndroidView(
@@ -183,32 +462,56 @@ fun QrPairingScanner(
.also { analysis ->
analysis.setAnalyzer(cameraExecutor) { imageProxy ->
val mediaImage = imageProxy.image
if (mediaImage != null && !hasDetected.get()) {
val inputImage = InputImage.fromMediaImage(
mediaImage,
imageProxy.imageInfo.rotationDegrees
)
barcodeScanner.process(inputImage)
.addOnSuccessListener { barcodes ->
for (barcode in barcodes) {
if (barcode.valueType == Barcode.TYPE_TEXT ||
barcode.valueType == Barcode.TYPE_UNKNOWN
) {
val rawValue = barcode.rawValue ?: continue
val payload = parseHermesPairingQr(rawValue)
if (payload != null && hasDetected.compareAndSet(false, true)) {
onPairingDetected(payload)
return@addOnSuccessListener
}
if (mediaImage == null || hasDetected.get()) {
imageProxy.close()
return@setAnalyzer
}
val rotation = imageProxy.imageInfo.rotationDegrees
val imgW = mediaImage.width
val imgH = mediaImage.height
val inputImage = InputImage.fromMediaImage(
mediaImage,
rotation
)
barcodeScanner.process(inputImage)
.addOnSuccessListener { barcodes ->
// Drive the brackets off ANY decoded QR so
// the lock-on snap is visible even before
// we've parsed it as a valid Hermes payload.
val first = barcodes.firstOrNull { b ->
b.boundingBox != null &&
(b.valueType == Barcode.TYPE_TEXT ||
b.valueType == Barcode.TYPE_UNKNOWN)
}
val box = first?.boundingBox
val vpSize = viewportSizePx
if (box != null && vpSize.width > 0 && vpSize.height > 0) {
detectedBox = mapBoxToViewport(
box = box,
imgW = imgW,
imgH = imgH,
rotationDegrees = rotation,
viewportSize = vpSize,
)
lastDetectionAtMs = System.currentTimeMillis()
}
// Then try to parse for the actual lock.
for (barcode in barcodes) {
if (barcode.valueType == Barcode.TYPE_TEXT ||
barcode.valueType == Barcode.TYPE_UNKNOWN
) {
val rawValue = barcode.rawValue ?: continue
val payload = parseHermesPairingQr(rawValue)
if (payload != null && hasDetected.compareAndSet(false, true)) {
lockedPayload = payload
return@addOnSuccessListener
}
}
}
.addOnCompleteListener {
imageProxy.close()
}
} else {
imageProxy.close()
}
}
.addOnCompleteListener {
imageProxy.close()
}
}
}
@@ -229,6 +532,15 @@ fun QrPairingScanner(
},
modifier = Modifier.fillMaxSize()
)
// Sci-fi L-bracket overlay. When detectedBox is null the
// brackets sit at a centered ready inset; when present they
// spring to the bounding box of the live detection.
ScannerCornersOverlay(
detected = detectedBox,
locked = lockedPayload != null,
modifier = Modifier.fillMaxSize(),
)
}
Spacer(modifier = Modifier.height(24.dp))
@@ -261,3 +573,229 @@ fun QrPairingScanner(
}
}
}
/**
* Sci-fi L-bracket overlay drawn on top of the camera viewport. Renders four
* corner brackets that:
*
* - Sit at a centered "ready" inset (~12% of viewport from each edge) when
* no QR is detected, with a slow breathing pulse on alpha.
* - Spring to the bounding box of a live detection when [detected] is non-null
* — animated independently per side so the snap reads as a genuine "lock-on"
* rather than a translation.
* - Switch from the primary cyan tint to a vivid green when [locked] is true,
* so the brief settle delay before navigation reads as confirmation.
*
* The brackets themselves are drawn with `Stroke(cap = StrokeCap.Round)` so
* the L-corners blend cleanly. Two passes — a soft outer glow at low alpha
* + a crisp inner stroke — give the futuristic glow without needing actual
* blur shaders.
*/
@Composable
private fun ScannerCornersOverlay(
detected: ViewportRect?,
locked: Boolean,
modifier: Modifier = Modifier,
) {
// Themed idle: two-tone gradient between primary (top-left/bottom-right)
// and tertiary (top-right/bottom-left). Both are brand purples in this
// theme, so the corners read as cohesive but not flat.
val primary = MaterialTheme.colorScheme.primary
val tertiary = MaterialTheme.colorScheme.tertiary
val onPrimary = MaterialTheme.colorScheme.onPrimary
// Vivid Material A400 success green — much more saturated than the
// generic 500-shade we had before, reads as "lock-on confirmed" instead
// of "neutral status indicator".
val successCore = Color(0xFF00E676)
val successGlow = Color(0xFF69F0AE)
// Slow breathing pulse on alpha when idle. Locked state stays solid +
// gets its own one-shot ramp so the green burst is unmistakable.
val infiniteTransition = rememberInfiniteTransition(label = "scan-corners")
val idlePulse by infiniteTransition.animateFloat(
initialValue = 0.45f,
targetValue = 1f,
animationSpec = infiniteRepeatable(
animation = tween(1400, easing = FastOutSlowInEasing),
repeatMode = RepeatMode.Reverse,
),
label = "idle-pulse",
)
// One-shot ramp that fires when `locked` flips true. Drives the
// outward scale pop on the corners + the green tint flash overlay.
val lockRamp by animateFloatAsState(
targetValue = if (locked) 1f else 0f,
animationSpec = if (locked) {
spring(dampingRatio = 0.55f, stiffness = 220f)
} else {
tween(180)
},
label = "lock-ramp",
)
var size by remember { mutableStateOf(IntSize.Zero) }
val density = LocalDensity.current
// Compute the target rect (left/top/right/bottom in px). When idle we
// inset from the viewport edges by ~10%; when detected we use the
// detected box. Each side animates independently with a snappy spring.
val readyInsetFrac = 0.10f
val targetLeft: Float
val targetTop: Float
val targetRight: Float
val targetBottom: Float
if (detected != null) {
targetLeft = detected.left
targetTop = detected.top
targetRight = detected.right
targetBottom = detected.bottom
} else if (size.width > 0 && size.height > 0) {
targetLeft = size.width * readyInsetFrac
targetTop = size.height * readyInsetFrac
targetRight = size.width * (1f - readyInsetFrac)
targetBottom = size.height * (1f - readyInsetFrac)
} else {
targetLeft = 0f
targetTop = 0f
targetRight = 0f
targetBottom = 0f
}
val springSpec = spring<Float>(
dampingRatio = 0.7f,
stiffness = 280f,
)
val animLeft by animateFloatAsState(targetLeft, springSpec, label = "snap-l")
val animTop by animateFloatAsState(targetTop, springSpec, label = "snap-t")
val animRight by animateFloatAsState(targetRight, springSpec, label = "snap-r")
val animBottom by animateFloatAsState(targetBottom, springSpec, label = "snap-b")
Canvas(
modifier = modifier.onSizeChanged { size = it }
) {
if (animRight <= animLeft || animBottom <= animTop) return@Canvas
// On lock, push the brackets outward by ~10dp so they pop OUT past
// the QR boundary like a "got it" flourish, then settle.
val popPx = with(density) { 10.dp.toPx() } * lockRamp
val left = animLeft - popPx
val top = animTop - popPx
val right = animRight + popPx
val bottom = animBottom + popPx
val w = right - left
val h = bottom - top
// Corner arm length scales with the smaller box side so the brackets
// stay proportional whether snapped to a small QR or sitting at the
// ready inset. Bumped from 22% → 26% for a more pronounced sci-fi look.
val arm = (kotlin.math.min(w, h) * 0.26f).coerceAtLeast(with(density) { 18.dp.toPx() })
val coreStroke = with(density) { 4.dp.toPx() }
val glowStroke = with(density) { 14.dp.toPx() }
val pipRadius = with(density) { 3.dp.toPx() }
// Idle alpha breathes; detected/locked are solid + amped by the lockRamp.
val baseAlpha = if (detected != null || locked) 1f else idlePulse
val glowAlpha = if (detected != null || locked) {
0.55f + 0.25f * lockRamp
} else {
idlePulse * 0.35f
}
// Diagonal pairing: TL+BR get the primary; TR+BL get the tertiary.
// Gives a cohesive two-tone "diagonal scan" feel. When locked, all
// four corners flip to the success green.
val tlBrCore = if (locked) successCore.copy(alpha = baseAlpha) else primary.copy(alpha = baseAlpha)
val trBlCore = if (locked) successCore.copy(alpha = baseAlpha) else tertiary.copy(alpha = baseAlpha)
val tlBrGlow = if (locked) successGlow.copy(alpha = glowAlpha) else primary.copy(alpha = glowAlpha)
val trBlGlow = if (locked) successGlow.copy(alpha = glowAlpha) else tertiary.copy(alpha = glowAlpha)
val pipColor = if (locked) successGlow.copy(alpha = baseAlpha) else onPrimary.copy(alpha = baseAlpha * 0.85f)
val corners = listOf(
CornerBracket(
origin = Offset(left, top),
horiz = Offset(left + arm, top),
vert = Offset(left, top + arm),
core = tlBrCore,
glow = tlBrGlow,
),
CornerBracket(
origin = Offset(right, top),
horiz = Offset(right - arm, top),
vert = Offset(right, top + arm),
core = trBlCore,
glow = trBlGlow,
),
CornerBracket(
origin = Offset(left, bottom),
horiz = Offset(left + arm, bottom),
vert = Offset(left, bottom - arm),
core = trBlCore,
glow = trBlGlow,
),
CornerBracket(
origin = Offset(right, bottom),
horiz = Offset(right - arm, bottom),
vert = Offset(right, bottom - arm),
core = tlBrCore,
glow = tlBrGlow,
),
)
// Pass 1 — wide soft glow underneath (low alpha, fat stroke)
for (c in corners) {
drawLine(
color = c.glow,
start = c.origin,
end = c.horiz,
strokeWidth = glowStroke,
cap = StrokeCap.Round,
)
drawLine(
color = c.glow,
start = c.origin,
end = c.vert,
strokeWidth = glowStroke,
cap = StrokeCap.Round,
)
}
// Pass 2 — crisp core stroke
for (c in corners) {
drawLine(
color = c.core,
start = c.origin,
end = c.horiz,
strokeWidth = coreStroke,
cap = StrokeCap.Round,
)
drawLine(
color = c.core,
start = c.origin,
end = c.vert,
strokeWidth = coreStroke,
cap = StrokeCap.Round,
)
}
// Pass 3 — pip dots at each L-corner origin. Tiny detail that reads
// as "targeting reticle" rather than "rounded rectangle".
for (c in corners) {
drawCircle(
color = pipColor,
radius = pipRadius,
center = c.origin,
)
}
// Lock flash — brief green tint over the entire viewport that fades
// out as lockRamp settles. Driven by the same spring as the corner
// pop so they read as one event.
if (lockRamp > 0f) {
drawRect(
color = successCore.copy(alpha = 0.18f * lockRamp),
topLeft = Offset.Zero,
size = this.size,
)
}
}
}
@@ -0,0 +1,233 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.selection.selectable
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Schedule
import androidx.compose.material.icons.filled.Warning
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.RadioButton
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.semantics.Role
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.data.PairingPreferences
/**
* TTL picker shown after a successful QR parse and *before* the WSS auth
* handshake kicks off. Lets the user pick how long the pairing should last.
*
* **Design philosophy (Bailey's explicit calls):**
*
* - **Never is always selectable.** We do NOT gate "Never expire" on
* transport security. Users on LAN, Tailscale, VPN, or TLS all need the
* option, and we trust the user's judgment. A brief warning sits under
* the option; we do not block it.
*
* - **Tailscale is informational only.** [isTailscaleDetected] changes the
* helper line ("Transport: Tailscale detected") and nudges the default
* selection for fresh installs, but does not change what's available.
*
* - **Last selection persists.** The user's previous choice is seeded via
* [initialTtlSeconds] which the caller pulls from
* [PairingPreferences.getPairTtlSeconds]. On confirm the caller persists
* the new choice back.
*
* **Default selection logic** (caller should compute via
* [defaultTtlSeconds] before opening the dialog):
* 1. If the QR payload's relay block has `ttlSeconds`, use that
* 2. Else if transport hint is `"wss"` OR Tailscale is detected → 30 days
* 3. Else if `ws://` without Tailscale → 7 days
* 4. Fall back → 30 days
*
* The picker always shows so the user can override — the user's trust model
* is the only one that matters and we force a confirmation step.
*/
@Composable
fun SessionTtlPickerDialog(
initialTtlSeconds: Long,
isTailscaleDetected: Boolean,
transportHint: String?,
onConfirm: (ttlSeconds: Long) -> Unit,
onCancel: () -> Unit,
) {
val options = ttlPickerOptions()
val startIndex = options.indexOfFirst { it.seconds == initialTtlSeconds }
.coerceAtLeast(defaultOptionIndex(options))
var selectedIndex by remember { mutableStateOf(startIndex) }
AlertDialog(
onDismissRequest = onCancel,
icon = {
Icon(
imageVector = Icons.Filled.Schedule,
contentDescription = null,
tint = MaterialTheme.colorScheme.primary
)
},
title = {
Text(
text = "Keep this pairing for…",
style = MaterialTheme.typography.titleLarge
)
},
text = {
Column(
verticalArrangement = Arrangement.spacedBy(8.dp),
modifier = Modifier.fillMaxWidth()
) {
Text(
text = "Your phone will reconnect automatically during " +
"this window. After it expires you'll need to re-pair.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
// Transport hint / Tailscale helper line
val helperLine = when {
isTailscaleDetected -> "Transport: Tailscale detected"
transportHint.equals("wss", ignoreCase = true) -> "Transport: TLS (wss://)"
transportHint.equals("ws", ignoreCase = true) -> "Transport: plain ws://"
else -> null
}
if (helperLine != null) {
Text(
text = helperLine,
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.primary
)
}
Spacer(Modifier.height(4.dp))
Column {
options.forEachIndexed { index, option ->
Row(
modifier = Modifier
.fillMaxWidth()
.selectable(
selected = selectedIndex == index,
onClick = { selectedIndex = index },
role = Role.RadioButton
)
.padding(vertical = 4.dp),
verticalAlignment = Alignment.CenterVertically
) {
RadioButton(
selected = selectedIndex == index,
onClick = { selectedIndex = index }
)
Text(
text = option.label,
style = MaterialTheme.typography.bodyMedium,
modifier = Modifier.padding(start = 8.dp)
)
}
}
}
// Never-expire warning — inline, not a gate. Shown only when
// the user has Never selected.
val neverIndex = options.indexOfFirst { it.seconds == PairingPreferences.TTL_NEVER }
if (selectedIndex == neverIndex) {
Row(
verticalAlignment = Alignment.Top,
horizontalArrangement = Arrangement.spacedBy(6.dp),
modifier = Modifier.padding(top = 4.dp)
) {
Icon(
imageVector = Icons.Filled.Warning,
contentDescription = null,
tint = MaterialTheme.colorScheme.tertiary,
modifier = Modifier
.padding(top = 2.dp)
.height(16.dp)
)
Text(
text = "This device will stay paired until you " +
"revoke it manually from Paired Devices. " +
"Only choose this if you control the network " +
"— LAN, Tailscale, VPN, or TLS.",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
}
}
}
},
confirmButton = {
TextButton(
onClick = { onConfirm(options[selectedIndex].seconds) }
) {
Text("Pair")
}
},
dismissButton = {
TextButton(onClick = onCancel) {
Text("Cancel")
}
}
)
}
/**
* A single entry in the TTL picker. `seconds == 0` means "never expire"
* (wire contract alignment — matches `ttl_seconds: 0` on the QR payload).
*/
data class TtlOption(val label: String, val seconds: Long)
/** The canonical set of TTL options shown in the picker. */
fun ttlPickerOptions(): List<TtlOption> = listOf(
TtlOption("1 day", 24L * 60 * 60),
TtlOption("7 days", 7L * 24 * 60 * 60),
TtlOption("30 days", 30L * 24 * 60 * 60),
TtlOption("90 days", 90L * 24 * 60 * 60),
TtlOption("1 year", 365L * 24 * 60 * 60),
TtlOption("Never expire", PairingPreferences.TTL_NEVER),
)
/** Fallback default when no previous selection and no QR hint — 30 days. */
private fun defaultOptionIndex(options: List<TtlOption>): Int =
options.indexOfFirst { it.seconds == PairingPreferences.DEFAULT_TTL_SECONDS }
.coerceAtLeast(0)
/**
* Compute the default TTL for a new pair based on:
* - QR payload's `ttlSeconds` (operator intent via `hermes-pair --ttl`)
* - Transport hint (`"wss"` → 30d, `"ws"` → 7d)
* - Tailscale detected → 30d
* - Fallback → 30d
*
* Called by [com.hermesandroid.relay.ui.screens.SettingsScreen] right before
* opening the picker, passed in as `initialTtlSeconds`.
*/
fun defaultTtlSeconds(
qrTtlSeconds: Long?,
transportHint: String?,
isTailscaleDetected: Boolean,
): Long {
if (qrTtlSeconds != null) return qrTtlSeconds
val isWss = transportHint.equals("wss", ignoreCase = true)
val isWs = transportHint.equals("ws", ignoreCase = true)
return when {
isWss || isTailscaleDetected -> 30L * 24 * 60 * 60
isWs -> 7L * 24 * 60 * 60
else -> PairingPreferences.DEFAULT_TTL_SECONDS
}
}
@@ -0,0 +1,82 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.ExpandLess
import androidx.compose.material.icons.filled.ExpandMore
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.ui.theme.gradientBorder
/**
* Reusable expandable card used across the Settings sub-screens. Header row
* is always visible and toggles the body on tap. Previously lived as a
* private helper inside `SettingsScreen.kt`; hoisted to a public component
* when the mega-settings file was split into per-category sub-screens
* following the VoiceSettingsScreen pattern.
*/
@Composable
fun SettingsExpandableCard(
title: String,
expanded: Boolean,
onToggle: () -> Unit,
isDarkTheme: Boolean,
modifier: Modifier = Modifier,
content: @Composable () -> Unit
) {
Card(
modifier = modifier
.fillMaxWidth()
.gradientBorder(
shape = RoundedCornerShape(12.dp),
isDarkTheme = isDarkTheme
),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(modifier = Modifier.padding(16.dp)) {
// Header row — always visible, tappable to toggle
Row(
modifier = Modifier
.fillMaxWidth()
.clickable(onClick = onToggle),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically
) {
Text(
text = title,
style = MaterialTheme.typography.titleSmall
)
Icon(
imageVector = if (expanded) Icons.Filled.ExpandLess else Icons.Filled.ExpandMore,
contentDescription = if (expanded) "Collapse" else "Expand"
)
}
// Expandable content
AnimatedVisibility(visible = expanded) {
Column(
modifier = Modifier.padding(top = 12.dp),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
content()
}
}
}
}
}
@@ -0,0 +1,128 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Close
import androidx.compose.material.icons.filled.KeyboardArrowDown
import androidx.compose.material.icons.filled.KeyboardArrowUp
import androidx.compose.material.icons.filled.Search
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.focus.FocusRequester
import androidx.compose.ui.focus.focusRequester
import androidx.compose.ui.text.input.ImeAction
import androidx.compose.ui.unit.dp
import androidx.compose.foundation.text.KeyboardActions
import androidx.compose.foundation.text.KeyboardOptions
/**
* Inline search row that appears below the terminal top app bar.
*
* Owns its own query state — the parent only deals in find-next /
* find-previous events plus a close-and-clear callback. The query is
* not persisted across show/hide cycles, mirroring how Chrome's find
* bar resets on close.
*
* - Auto-focuses the text field when the bar appears so the user can
* start typing immediately. The keyboard pops up via the IME without
* needing an extra tap.
* - Pressing Enter (IME action Search) triggers find-next, matching
* Chrome / VS Code conventions.
* - Up / Down arrow IconButtons step through prev / next results.
* - Close button hides the bar AND fires `onClose` so the parent can
* invoke `window.clearSearch()` on the active tab to wipe decorations.
*
* The search itself is JS-side via `window.searchNext('text')` etc on the
* active tab's WebView. This component knows nothing about WebViews — it
* just sends string queries upward.
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun TerminalSearchBar(
onSearchNext: (String) -> Unit,
onSearchPrev: (String) -> Unit,
onClose: () -> Unit,
modifier: Modifier = Modifier,
) {
var query by remember { mutableStateOf("") }
val focusRequester = remember { FocusRequester() }
LaunchedEffect(Unit) {
// Defer one frame so the OutlinedTextField has been attached before
// requesting focus — otherwise the call no-ops on first composition.
try {
focusRequester.requestFocus()
} catch (_: Exception) { /* not yet attached */ }
}
Row(
modifier = modifier
.fillMaxWidth()
.background(MaterialTheme.colorScheme.surface)
.padding(horizontal = 8.dp, vertical = 6.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(4.dp),
) {
OutlinedTextField(
value = query,
onValueChange = { query = it },
modifier = Modifier
.weight(1f)
.focusRequester(focusRequester),
placeholder = { Text("Search scrollback") },
leadingIcon = {
Icon(
imageVector = Icons.Filled.Search,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
},
singleLine = true,
keyboardOptions = KeyboardOptions(imeAction = ImeAction.Search),
keyboardActions = KeyboardActions(
onSearch = { if (query.isNotEmpty()) onSearchNext(query) }
),
)
IconButton(
onClick = { if (query.isNotEmpty()) onSearchPrev(query) },
enabled = query.isNotEmpty(),
) {
Icon(
imageVector = Icons.Filled.KeyboardArrowUp,
contentDescription = "Previous match",
)
}
IconButton(
onClick = { if (query.isNotEmpty()) onSearchNext(query) },
enabled = query.isNotEmpty(),
) {
Icon(
imageVector = Icons.Filled.KeyboardArrowDown,
contentDescription = "Next match",
)
}
IconButton(onClick = onClose) {
Icon(
imageVector = Icons.Filled.Close,
contentDescription = "Close search",
)
}
}
}
@@ -0,0 +1,307 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.ExperimentalLayoutApi
import androidx.compose.foundation.layout.FlowRow
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.ModalBottomSheet
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.material3.rememberModalBottomSheetState
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.auth.PairedSession
import com.hermesandroid.relay.viewmodel.TerminalViewModel
import java.text.DateFormat
import java.util.Date
/**
* Bottom-sheet dialog showing full metadata for a single terminal tab plus
* quick-action buttons.
*
* Mirrors the UX of Chat's agent-info dialog (tap title → open info), but
* uses a [ModalBottomSheet] instead of an AlertDialog because terminal
* sessions have substantially more metadata (PID, shell, grid, transport,
* grants, expiry) and the bottom-sheet form factor handles the longer
* content gracefully without forcing scroll-inside-modal weirdness.
*
* The sheet reads its data from two sources:
*
* - [tab] — per-tab state from [TerminalViewModel.activeTab]. Drives the
* tab-specific rows: tab number, session name, PID, shell, grid size,
* tmux availability.
*
* - [pairedSession] — the relay-wide session metadata from
* [com.hermesandroid.relay.auth.AuthManager.currentPairedSession]. Drives
* transport security, expiry, and grant chips. May be null briefly during
* re-pair.
*
* Action buttons:
* - **Reattach** — calls [onReattach] then dismisses. The reattach itself is
* no-op safe if the tab is already attached (the ViewModel queues a fresh
* `terminal.attach` envelope).
* - **Close tab** — only enabled when there's more than one tab open
* ([TerminalViewModel.closeTab] no-ops on the last tab anyway). Calls
* [onCloseTab] then dismisses the sheet.
* - **Done** — pure dismiss.
*/
@OptIn(ExperimentalMaterial3Api::class, ExperimentalLayoutApi::class)
@Composable
fun TerminalSessionInfoSheet(
tab: TerminalViewModel.TabState,
pairedSession: PairedSession?,
canCloseTab: Boolean,
onReattach: () -> Unit,
onCloseTab: () -> Unit,
onDismiss: () -> Unit,
) {
val sheetState = rememberModalBottomSheetState(skipPartiallyExpanded = true)
ModalBottomSheet(
onDismissRequest = onDismiss,
sheetState = sheetState,
containerColor = MaterialTheme.colorScheme.surface,
) {
Column(
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 24.dp, vertical = 12.dp),
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
// Header — tab number + truncated session name. The session_name
// is monospaced because it's an opaque-ish identifier and looks
// wrong in a proportional font.
Column {
Text(
text = "Tab ${tab.tabId}",
style = MaterialTheme.typography.titleLarge,
fontWeight = FontWeight.SemiBold,
)
Text(
text = tab.sessionName,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
fontFamily = FontFamily.Monospace,
)
}
// Connection status chips — attach state + tmux availability.
FlowRow(
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
modifier = Modifier.fillMaxWidth(),
) {
StatusChip(
label = when {
tab.attached -> "Attached"
tab.attaching -> "Attaching…"
tab.error != null -> "Error"
else -> "Detached"
},
isPositive = tab.attached,
)
if (tab.tmuxAvailable) {
StatusChip(label = "tmux", isPositive = true)
}
}
HorizontalDivider()
// Per-tab metadata.
InfoRow("PID", tab.pid?.toString() ?: "—")
InfoRow("Shell", tab.shell ?: "—", monospace = true)
InfoRow("Grid", "${tab.cols} × ${tab.rows}")
if (tab.error != null) {
InfoRow("Last error", tab.error, valueColor = MaterialTheme.colorScheme.error)
}
HorizontalDivider()
// Relay session metadata — pulled from the shared paired session
// because terminal piggybacks on the same session token as chat.
// The transport badge is the same component used in
// PairedDevicesScreen + ConnectionInfoSheet for visual parity.
if (pairedSession != null) {
Text(
text = "Relay session",
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
TransportSecurityBadge(
isSecure = pairedSession.transportHint == "wss",
reason = null,
size = TransportSecuritySize.Row,
)
InfoRow(
label = "Expires",
value = formatExpiry(pairedSession.expiresAt),
)
if (pairedSession.grants.isNotEmpty()) {
Text(
text = "Channel grants",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
FlowRow(
horizontalArrangement = Arrangement.spacedBy(6.dp),
verticalArrangement = Arrangement.spacedBy(6.dp),
modifier = Modifier.fillMaxWidth(),
) {
for ((channel, expiry) in pairedSession.grants) {
GrantChipLocal(channel = channel, expiresAt = expiry)
}
}
}
} else {
Text(
text = "No paired session",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
Spacer(modifier = Modifier.height(8.dp))
// Action row.
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
OutlinedButton(
onClick = {
onReattach()
onDismiss()
},
modifier = Modifier.weight(1f),
) {
Text("Reattach")
}
OutlinedButton(
onClick = {
onCloseTab()
onDismiss()
},
enabled = canCloseTab,
modifier = Modifier.weight(1f),
) {
Text("Close tab")
}
}
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.End,
) {
TextButton(onClick = onDismiss) {
Text("Done")
}
}
Spacer(modifier = Modifier.height(8.dp))
}
}
}
@Composable
private fun StatusChip(label: String, isPositive: Boolean) {
val (bg, fg) = if (isPositive) {
MaterialTheme.colorScheme.primary.copy(alpha = 0.15f) to MaterialTheme.colorScheme.primary
} else {
MaterialTheme.colorScheme.surfaceVariant to MaterialTheme.colorScheme.onSurfaceVariant
}
Box(
modifier = Modifier
.clip(RoundedCornerShape(8.dp))
.background(bg)
.padding(horizontal = 10.dp, vertical = 4.dp),
) {
Text(
text = label,
color = fg,
style = MaterialTheme.typography.labelSmall,
fontWeight = FontWeight.Medium,
)
}
}
@Composable
private fun InfoRow(
label: String,
value: String,
monospace: Boolean = false,
valueColor: androidx.compose.ui.graphics.Color = MaterialTheme.colorScheme.onSurface,
) {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.Top,
) {
Text(
text = label,
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.width(96.dp),
)
Text(
text = value,
style = MaterialTheme.typography.bodyMedium,
color = valueColor,
fontFamily = if (monospace) FontFamily.Monospace else FontFamily.Default,
)
}
}
@Composable
private fun GrantChipLocal(channel: String, expiresAt: Long?) {
Box(
modifier = Modifier
.clip(RoundedCornerShape(8.dp))
.background(MaterialTheme.colorScheme.surfaceVariant)
.padding(horizontal = 8.dp, vertical = 4.dp),
) {
Text(
text = buildString {
append(channel)
append(" · ")
append(if (expiresAt == null) "never" else formatExpiryShort(expiresAt))
},
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
private fun formatExpiry(epochSeconds: Long?): String {
if (epochSeconds == null) return "Never"
return try {
DateFormat.getDateTimeInstance(DateFormat.MEDIUM, DateFormat.SHORT)
.format(Date(epochSeconds * 1000L))
} catch (_: Exception) {
"—"
}
}
private fun formatExpiryShort(epochSeconds: Long): String {
return try {
DateFormat.getDateInstance(DateFormat.SHORT).format(Date(epochSeconds * 1000L))
} catch (_: Exception) {
"—"
}
}
@@ -0,0 +1,172 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.combinedClickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Add
import androidx.compose.material.icons.filled.Close
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.foundation.ExperimentalFoundationApi
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.viewmodel.TerminalViewModel
/**
* Chrome-style tab strip for the terminal screen.
*
* One numbered button per active tab plus a `+` button to open new tabs
* (disabled when [TerminalViewModel.MAX_TABS] is reached).
*
* - Active tab: primary background, on-primary text.
* - Inactive tab: surface-variant background, on-surface-variant text.
* - The active tab also shows a small × icon to close it without needing
* a long-press; long-press still works on any tab so users can close
* background tabs without switching to them first.
* - Closing the last remaining tab is silently a no-op (handled in
* [TerminalViewModel.closeTab]) — there's always at least one terminal.
*/
@OptIn(ExperimentalFoundationApi::class)
@Composable
fun TerminalTabBar(
tabs: List<TerminalViewModel.TabState>,
activeTabId: Int,
onSelectTab: (Int) -> Unit,
onCloseTab: (Int) -> Unit,
onNewTab: () -> Unit,
modifier: Modifier = Modifier,
) {
Row(
modifier = modifier
.fillMaxWidth()
.background(MaterialTheme.colorScheme.surface)
.padding(horizontal = 8.dp, vertical = 6.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(6.dp),
) {
for (tab in tabs) {
val isActive = tab.tabId == activeTabId
TerminalTabChip(
tab = tab,
isActive = isActive,
canClose = tabs.size > 1,
onClick = { onSelectTab(tab.tabId) },
onLongPress = { if (tabs.size > 1) onCloseTab(tab.tabId) },
onCloseClick = { if (tabs.size > 1) onCloseTab(tab.tabId) },
)
}
Spacer(modifier = Modifier.width(2.dp))
// New-tab button — visually distinct from numbered tabs (rounder,
// plain icon) so it doesn't get mistaken for a fifth tab when the
// user has 4 already.
val canAddMore = tabs.size < TerminalViewModel.MAX_TABS
IconButton(
onClick = onNewTab,
enabled = canAddMore,
modifier = Modifier.size(32.dp),
) {
Icon(
imageVector = Icons.Filled.Add,
contentDescription = "New terminal tab",
tint = if (canAddMore) {
MaterialTheme.colorScheme.onSurface
} else {
MaterialTheme.colorScheme.onSurface.copy(alpha = 0.3f)
},
)
}
}
}
@OptIn(ExperimentalFoundationApi::class)
@Composable
private fun TerminalTabChip(
tab: TerminalViewModel.TabState,
isActive: Boolean,
canClose: Boolean,
onClick: () -> Unit,
onLongPress: () -> Unit,
onCloseClick: () -> Unit,
) {
val bg = if (isActive) {
MaterialTheme.colorScheme.primary
} else {
MaterialTheme.colorScheme.surfaceVariant
}
val fg = if (isActive) {
MaterialTheme.colorScheme.onPrimary
} else {
MaterialTheme.colorScheme.onSurfaceVariant
}
val shape = RoundedCornerShape(8.dp)
Row(
modifier = Modifier
.height(32.dp)
.clip(shape)
.background(bg)
.border(
width = 1.dp,
color = if (isActive) Color.Transparent else fg.copy(alpha = 0.15f),
shape = shape,
)
.combinedClickable(
onClick = onClick,
onLongClick = onLongPress,
)
.padding(horizontal = 12.dp, vertical = 4.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(6.dp),
) {
Text(
text = tab.tabId.toString(),
color = fg,
fontWeight = if (isActive) FontWeight.SemiBold else FontWeight.Medium,
fontFamily = FontFamily.Monospace,
style = MaterialTheme.typography.labelMedium,
)
// Show a small × on the active tab so users can close it without
// discovering the long-press gesture. Hidden when canClose is false
// (last remaining tab) or on background tabs to keep the strip
// compact.
if (isActive && canClose) {
Box(
modifier = Modifier
.size(16.dp)
.clip(RoundedCornerShape(50))
.combinedClickable(
onClick = onCloseClick,
onLongClick = onCloseClick,
),
contentAlignment = Alignment.Center,
) {
Icon(
imageVector = Icons.Filled.Close,
contentDescription = "Close tab ${tab.tabId}",
tint = fg,
modifier = Modifier.size(12.dp),
)
}
}
}
}
@@ -0,0 +1,276 @@
package com.hermesandroid.relay.ui.components
import android.annotation.SuppressLint
import android.content.Intent
import android.net.Uri
import android.view.View
import android.webkit.ConsoleMessage
import android.webkit.JavascriptInterface
import android.webkit.WebChromeClient
import android.webkit.WebResourceRequest
import android.webkit.WebView
import android.webkit.WebViewClient
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.viewinterop.AndroidView
import com.hermesandroid.relay.viewmodel.TerminalViewModel
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.filter
/**
* Composable hosting xterm.js inside a WebView, scoped to a single terminal
* tab.
*
* Loads `file:///android_asset/terminal/index.html`, installs a JS bridge
* named `AndroidBridge`, and pipes [TerminalViewModel.outputFlow] entries
* tagged with [tabId] into the terminal via `window.writeTerminal`.
*
* Bridge methods are invoked on a WebView-owned binder thread — the ViewModel
* it forwards to uses thread-safe state flows, so no extra synchronization
* is needed here.
*
* @param tabId the tab number this WebView is bound to. Used for both
* outbound bridge calls (so the ViewModel knows which tab the input
* came from) and inbound output filtering.
* @param onWebViewReady invoked once on construction with the underlying
* WebView so the parent screen can keep a reference for things like
* `window.searchNext('...')` evaluation. Optional — pass null if you
* don't need it.
*/
@Composable
fun TerminalWebView(
viewModel: TerminalViewModel,
tabId: Int,
modifier: Modifier = Modifier,
fontScale: StateFlow<Float> = remember { MutableStateFlow(1.0f) },
onWebViewReady: ((WebView) -> Unit)? = null,
) {
val context = LocalContext.current
// Background matches xterm's theme so there's no white flash while the
// HTML boots. Kept as a constant rather than pulling from MaterialTheme
// because the WebView host color must match the xterm theme exactly.
val webViewBackground = 0xFF1A1A2E.toInt()
// Device density — we pass CSS-pixel dimensions to xterm explicitly in
// the layout listener below because Chromium's internal viewport on
// Android WebView doesn't reliably update when Compose resizes the
// parent View. Without this, xterm latches at rows=1 on first layout
// and never grows, so command output scrolls straight off the viewport.
val density = context.resources.displayMetrics.density
// Each tab gets its own WebView instance. We key the `remember` block on
// tabId so reusing the same composition slot for a different tab (which
// shouldn't happen with our `key(tabId)` parent, but is defensive) yields
// a fresh WebView rather than re-binding the JS bridge to the wrong tab.
val webView = remember(tabId) {
@SuppressLint("SetJavaScriptEnabled")
WebView(context).apply {
setBackgroundColor(webViewBackground)
settings.javaScriptEnabled = true
settings.domStorageEnabled = true
settings.allowFileAccess = false
settings.allowContentAccess = false
settings.mediaPlaybackRequiresUserGesture = false
settings.loadWithOverviewMode = false
settings.useWideViewPort = false
settings.textZoom = 100 // prevent system font-size from scaling xterm
overScrollMode = View.OVER_SCROLL_NEVER
isVerticalScrollBarEnabled = false
isHorizontalScrollBarEnabled = false
isFocusable = true
isFocusableInTouchMode = true
// Intentionally NOT setting LAYER_TYPE_HARDWARE — on Samsung WebView
// it can latch the first rendered frame on the GPU layer and fail
// to propagate subsequent xterm DOM updates, so the initial prompt
// draws but command output never appears. Default layer type lets
// Chromium pick its own compositing path.
webViewClient = object : WebViewClient() {
override fun shouldOverrideUrlLoading(
view: WebView,
request: WebResourceRequest
): Boolean {
val url = request.url?.toString() ?: return false
// Any navigation away from our asset bundle goes to the system browser.
if (!url.startsWith("file:///android_asset/terminal/")) {
try {
context.startActivity(
Intent(Intent.ACTION_VIEW, Uri.parse(url))
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
)
} catch (_: Exception) { /* no browser available */ }
return true
}
return false
}
override fun onPageFinished(view: WebView, url: String) {
// Re-fire refit once the JS side is actually defined.
// The layout listener above may have fired before
// `window.refit` existed; fire it here with current
// dimensions so the first sensible fit lands regardless
// of event ordering.
val widthCss = (view.width / density).toInt()
val heightCss = (view.height / density).toInt()
if (widthCss > 0 && heightCss > 0) {
view.post {
view.evaluateJavascript(
"if (window.refit) { window.refit($widthCss, $heightCss); }",
null
)
}
}
}
}
webChromeClient = object : WebChromeClient() {
override fun onConsoleMessage(message: ConsoleMessage?): Boolean {
message ?: return false
android.util.Log.d(
"TerminalWebView",
"tab=$tabId ${message.messageLevel()} [${message.sourceId()}:${message.lineNumber()}] ${message.message()}"
)
return true
}
}
addJavascriptInterface(TerminalBridge(viewModel, this, tabId), "AndroidBridge")
// Force xterm to refit every time Compose gives us a new layout.
// Compose often measures the WebView at ~0-pixel-tall during the
// AnimatedContent tab-switch animation and during IME transitions,
// so the initial fit latches at rows=1. We forward the real CSS
// dimensions explicitly here because Chromium's WebView doesn't
// reliably propagate native-side resize to the HTML viewport — if
// we rely on CSS `height:100%` or the `window.resize` event, the
// internal viewport stays frozen at the first bad measurement.
//
// PRESERVED FROM SINGLE-TAB IMPLEMENTATION (commit 182d5a4) — every
// per-tab WebView MUST install this listener with the same density
// conversion and the same post-to-main pattern. Removing it
// re-introduces the rows=1 latch bug.
addOnLayoutChangeListener { view, left, top, right, bottom,
oldLeft, oldTop, oldRight, oldBottom ->
val widthPx = right - left
val heightPx = bottom - top
if (widthPx <= 0 || heightPx <= 0) return@addOnLayoutChangeListener
val widthCss = (widthPx / density).toInt()
val heightCss = (heightPx / density).toInt()
// Defer to the next UI frame so layout has settled and the
// WebView is attached to the Compose hierarchy before we run
// JS against it.
view.post {
(view as WebView).evaluateJavascript(
"if (window.refit) { window.refit($widthCss, $heightCss); }",
null
)
}
}
loadUrl("file:///android_asset/terminal/index.html")
}
}
// Hand the WebView to the parent so it can drive search-bar JS calls
// against the active tab. Fired once per tabId so the parent's map of
// tab -> WebView stays correct on tab churn.
LaunchedEffect(webView) {
onWebViewReady?.invoke(webView)
}
// Stream outputs from the ViewModel into the WebView, filtered to this
// tab only. Each per-tab WebView ignores chunks destined for other tabs.
LaunchedEffect(webView, viewModel, tabId) {
viewModel.outputFlow
.filter { it.tabId == tabId }
.collect { tabOutput ->
android.util.Log.d(
"TerminalWebView",
"tab=$tabId writeTerminal: ${tabOutput.b64.length} b64 chars"
)
// evaluateJavascript must run on the UI thread; LaunchedEffect's
// dispatcher is Main.
webView.evaluateJavascript("window.writeTerminal('${tabOutput.b64}');", null)
}
}
// Push the user's global font scale into xterm. The base size matches
// index.html (`fontSize: 13`), and `window.setFontSize` already calls
// `fitAddon.fit()` so the layout listener picks up the resulting resize
// automatically. We coerce to a sensible minimum so a tiny scale (or a
// future smaller stop) can never produce sub-6px terminal glyphs.
LaunchedEffect(webView, fontScale) {
fontScale.collect { scale ->
val target = (13 * scale).toInt().coerceAtLeast(6)
webView.evaluateJavascript(
"if (window.setFontSize) window.setFontSize($target);",
null
)
}
}
DisposableEffect(webView) {
onDispose {
try {
webView.loadUrl("about:blank")
webView.stopLoading()
webView.removeJavascriptInterface("AndroidBridge")
webView.destroy()
} catch (_: Exception) { /* best-effort teardown */ }
}
}
AndroidView(
factory = { webView },
modifier = modifier
)
}
/**
* JS → Kotlin bridge. Methods marked `@JavascriptInterface` are callable from
* the HTML side as `AndroidBridge.<method>(...)`. Every call happens on a
* WebView-owned binder thread — route straight into the ViewModel, which is
* thread-safe.
*
* Each bridge instance is bound to a specific [tabId] at construction time
* so the ViewModel always knows which tab a given input/resize/ready call
* came from. The HTML side never sees or sends a tab id — the binding is
* one-WebView-per-tab on the Kotlin side.
*/
private class TerminalBridge(
private val viewModel: TerminalViewModel,
private val webView: WebView,
private val tabId: Int,
) {
@JavascriptInterface
fun onReady(cols: Int, rows: Int) {
viewModel.onTerminalReady(tabId, cols, rows)
}
@JavascriptInterface
fun onInput(data: String) {
android.util.Log.d("TerminalWebView", "tab=$tabId bridge.onInput: ${data.length} bytes")
viewModel.sendInput(tabId, data)
}
@JavascriptInterface
fun onResize(cols: Int, rows: Int) {
viewModel.resize(tabId, cols, rows)
}
@JavascriptInterface
fun onLink(url: String) {
val context = webView.context ?: return
try {
context.startActivity(
Intent(Intent.ACTION_VIEW, Uri.parse(url))
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
)
} catch (_: Exception) { /* no browser available */ }
}
}
@@ -0,0 +1,172 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.background
import androidx.compose.foundation.border
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Lock
import androidx.compose.material.icons.filled.LockOpen
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
/**
* Visual badge for the current relay transport security posture.
*
* Three states:
*
* - **Secure (TLS)** — rendered in green when the relay URL is `wss://`
* and either the connection is live or the user just configured it.
* Users see the shield and understand that traffic is encrypted +
* optionally TOFU-pinned.
*
* - **Insecure (reason)** — amber when `isSecure == false` and the user has
* acknowledged the [InsecureConnectionAckDialog] with a known reason
* (`"lan_only"`, `"tailscale_vpn"`, `"local_dev"`). The reason comes from
* [PairingPreferences.insecureReason].
*
* - **Insecure (network unknown)** — red when `isSecure == false` and the
* user has NOT yet acknowledged the dialog. This is a louder warning to
* prompt them through the ack flow.
*
* Three size variants:
*
* - [TransportSecuritySize.Chip] — small pill used inline in Settings rows
* - [TransportSecuritySize.Row] — full-width row for [ConnectionInfoSheet]
* - [TransportSecuritySize.Large] — prominent version for
* [PairedDevicesScreen] card headers
*
* Colors are chosen to match existing [ConnectionStatusBadge] palette so
* the UI feels consistent — green ≈ connected, amber ≈ connecting/stale,
* red ≈ error.
*/
enum class TransportSecuritySize { Chip, Row, Large }
@Composable
fun TransportSecurityBadge(
isSecure: Boolean,
reason: String?,
modifier: Modifier = Modifier,
size: TransportSecuritySize = TransportSecuritySize.Chip,
) {
val (label, bg, fg) = resolveAppearance(isSecure, reason)
val icon = if (isSecure) Icons.Filled.Lock else Icons.Filled.LockOpen
val shape = RoundedCornerShape(
when (size) {
TransportSecuritySize.Chip -> 8.dp
TransportSecuritySize.Row -> 10.dp
TransportSecuritySize.Large -> 12.dp
}
)
val verticalPad = when (size) {
TransportSecuritySize.Chip -> 4.dp
TransportSecuritySize.Row -> 8.dp
TransportSecuritySize.Large -> 10.dp
}
val horizontalPad = when (size) {
TransportSecuritySize.Chip -> 8.dp
TransportSecuritySize.Row -> 12.dp
TransportSecuritySize.Large -> 14.dp
}
val iconSize = when (size) {
TransportSecuritySize.Chip -> 14.dp
TransportSecuritySize.Row -> 18.dp
TransportSecuritySize.Large -> 20.dp
}
Row(
modifier = modifier
.clip(shape)
.background(bg)
.border(1.dp, fg.copy(alpha = 0.35f), shape)
.padding(horizontal = horizontalPad, vertical = verticalPad),
horizontalArrangement = Arrangement.spacedBy(6.dp),
verticalAlignment = Alignment.CenterVertically
) {
Icon(
imageVector = icon,
contentDescription = null,
tint = fg,
modifier = Modifier.size(iconSize)
)
Text(
text = label,
color = fg,
fontWeight = FontWeight.Medium,
style = when (size) {
TransportSecuritySize.Chip -> MaterialTheme.typography.labelSmall
TransportSecuritySize.Row -> MaterialTheme.typography.bodySmall
TransportSecuritySize.Large -> MaterialTheme.typography.bodyMedium
}
)
}
}
/** Build the user-facing label for a given insecure reason code. */
fun insecureReasonLabel(reason: String?): String = when (reason) {
"lan_only" -> "Insecure (LAN)"
"tailscale_vpn" -> "Insecure (Tailscale)"
"local_dev" -> "Insecure (dev)"
else -> "Insecure (network unknown)"
}
private data class BadgeAppearance(val label: String, val bg: Color, val fg: Color)
@Composable
private fun resolveAppearance(isSecure: Boolean, reason: String?): BadgeAppearance {
// Secure — green, matches ConnectionStatusBadge connected palette.
if (isSecure) {
val green = Color(0xFF2E7D32)
return BadgeAppearance(
label = "Secure (TLS)",
bg = green.copy(alpha = 0.14f),
fg = green,
)
}
val hasKnownReason = when (reason) {
"lan_only", "tailscale_vpn", "local_dev" -> true
else -> false
}
return if (hasKnownReason) {
// Amber — user has acknowledged + picked a reason, UX is informational.
val amber = Color(0xFFF9A825)
BadgeAppearance(
label = insecureReasonLabel(reason),
bg = amber.copy(alpha = 0.16f),
fg = amber,
)
} else {
// Red — no ack yet, louder warning.
val red = MaterialTheme.colorScheme.error
BadgeAppearance(
label = insecureReasonLabel(reason),
bg = red.copy(alpha = 0.16f),
fg = red,
)
}
}
/**
* Utility: given a URL (ws/wss/http/https), return whether it's secure.
* Used by callers that want to derive a badge state from a raw URL string.
*/
fun isUrlSecure(url: String?): Boolean {
if (url.isNullOrBlank()) return false
val lower = url.trim().lowercase()
return lower.startsWith("wss://") || lower.startsWith("https://")
}
@@ -0,0 +1,747 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.animation.AnimatedContent
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.core.Animatable
import androidx.compose.animation.core.animateFloatAsState
import androidx.compose.animation.core.LinearEasing
import androidx.compose.animation.core.tween
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import androidx.compose.animation.togetherWith
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.foundation.gestures.waitForUpOrCancellation
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.statusBarsPadding
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Close
import androidx.compose.material.icons.filled.GraphicEq
import androidx.compose.material.icons.filled.Mic
import androidx.compose.material.icons.filled.Refresh
import androidx.compose.material.icons.filled.Stop
import androidx.compose.material3.DropdownMenu
import androidx.compose.material3.DropdownMenuItem
import androidx.compose.material3.FilledTonalIconButton
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.IconButtonDefaults
import androidx.compose.material3.LinearProgressIndicator
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.draw.drawWithContent
import androidx.compose.ui.graphics.BlendMode
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.CompositingStrategy
import androidx.compose.ui.graphics.graphicsLayer
import androidx.compose.ui.hapticfeedback.HapticFeedbackType
import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.platform.LocalHapticFeedback
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.ui.LocalSnackbarHost
import com.hermesandroid.relay.ui.showHumanError
import com.hermesandroid.relay.util.HumanError
import com.hermesandroid.relay.viewmodel.DestructiveCountdownState
import com.hermesandroid.relay.viewmodel.InteractionMode
import com.hermesandroid.relay.viewmodel.VoiceState
import com.hermesandroid.relay.viewmodel.VoiceUiState
import kotlinx.coroutines.flow.SharedFlow
/**
* Full-screen voice-mode overlay. Renders the MorphingSphere in its voiceMode
* expanded state plus a mic button that drives the [VoiceViewModel] turn
* cycle. Dispatch rules per interaction mode live on the mic button — the
* overlay itself is a thin Compose shell over the locked VoiceUiState.
*/
@Composable
fun VoiceModeOverlay(
uiState: VoiceUiState,
onMicTap: () -> Unit,
onMicRelease: () -> Unit,
onInterrupt: () -> Unit,
onDismiss: () -> Unit,
onModeChange: (InteractionMode) -> Unit,
onClearError: () -> Unit,
modifier: Modifier = Modifier,
// Nullable so existing call sites compile; when wired, voice errors
// surface as global snackbars in addition to the inline banner below.
errorEvents: SharedFlow<HumanError>? = null,
// === PHASE3-voice-mode-transcript ===
// Compact rolling transcript of the last N chat messages — the
// caller (ChatScreen) passes `chatViewModel.messages.takeLast(6)`.
// Voice mode is voice-first but wasn't giving the user any visibility
// into conversation history during a session, so this adds a compact
// strip below the sphere that observes the same ChatHandler message
// flow the chat tab uses. Includes local-only voice-intent traces
// (agentName = "Voice action", id prefix "voice-intent-") rendered
// via MarkdownContent so the bold + inline code in traces like
// "**Opened Chrome** `com.android.chrome`" shows correctly.
//
// Default empty-list so existing call sites compile; the empty state
// hint ("Tap the mic to speak") still renders when the transcript
// is empty.
transcriptMessages: List<ChatMessage> = emptyList(),
// === END PHASE3-voice-mode-transcript ===
) {
val surface = MaterialTheme.colorScheme.surface
val haptic = LocalHapticFeedback.current
var modeMenuOpen by remember { mutableStateOf(false) }
// Pipe classified voice errors to the app-wide snackbar host. The inline
// error banner stays as a belt-and-suspenders for longer-lived messages.
val snackbarHost = LocalSnackbarHost.current
LaunchedEffect(errorEvents) {
errorEvents?.collect { err ->
snackbarHost.showHumanError(err)
}
}
Box(
modifier = modifier
.fillMaxSize()
// Fully opaque surface — at 0.95f the chat content behind the
// overlay was bleeding through (the "phantom pencil" effect). Voice
// mode is a modality, not a dim-over; the transition animation
// already sells the mode change, no translucency needed.
.background(surface)
) {
// Top bar — close + mode selector.
// `statusBarsPadding()` pushes the row below the system status bar so
// the close button isn't crammed against battery/signal icons and
// Android's gesture area can't swallow taps on buttons near the edge.
Row(
modifier = Modifier
.fillMaxWidth()
.statusBarsPadding()
.padding(horizontal = 12.dp, vertical = 8.dp)
.align(Alignment.TopCenter),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically,
) {
Box {
TextButton(onClick = { modeMenuOpen = true }) {
Text(uiState.interactionMode.label())
}
DropdownMenu(
expanded = modeMenuOpen,
onDismissRequest = { modeMenuOpen = false },
) {
InteractionMode.values().forEach { mode ->
DropdownMenuItem(
text = { Text(mode.label()) },
onClick = {
onModeChange(mode)
modeMenuOpen = false
},
)
}
}
}
// Filled tonal background gives the close button a clear "tappable"
// affordance. Plain IconButton was visually indistinguishable from
// the surface and users couldn't find it at a glance.
FilledTonalIconButton(
onClick = onDismiss,
colors = IconButtonDefaults.filledTonalIconButtonColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant,
contentColor = MaterialTheme.colorScheme.onSurfaceVariant,
),
) {
Icon(
imageVector = Icons.Filled.Close,
contentDescription = "Close voice mode",
)
}
}
// Center column: pinned "you said" chip → sphere → waveform → scrolling transcript.
//
// Layout uses fixed-weight slots (sphere 1.5, transcript 1f) instead
// of SpaceBetween so the sphere/waveform don't drift as the transcript
// grows. The transcript area owns its own scroll state with auto-
// scroll-to-tail and a top/bottom fade mask for overflow indication.
//
// Transcript content sources:
// - transcriptMessages: last N chat messages (includes voice-intent
// traces rendered via MarkdownContent)
// - uiState.responseText: the in-flight streaming response for the
// current turn (a bubble that hasn't yet been finalized into a
// ChatMessage by the time the user sees it mid-stream)
val transcriptScrollState = rememberScrollState()
// Auto-scroll to the tail whenever: (a) a new message arrives in the
// transcript, (b) the streaming responseText grows. Both conditions
// fire the same smooth animateScrollTo so the user's eye never has
// to chase token churn.
LaunchedEffect(transcriptMessages.size, uiState.responseText.length) {
transcriptScrollState.animateScrollTo(transcriptScrollState.maxValue)
}
// Vertical fade brush for the scroll area: top + bottom edges fade
// into the surface so overflow is visually obvious without showing a
// scrollbar. Applied via BlendMode.DstIn inside an offscreen
// compositing layer so the gradient multiplies the alpha of the
// text underneath rather than tinting the surface behind.
val fadeBrush = remember {
Brush.verticalGradient(
0f to Color.Transparent,
0.08f to Color.Black,
0.92f to Color.Black,
1f to Color.Transparent,
)
}
Column(
modifier = Modifier
.fillMaxSize()
.padding(top = 64.dp, bottom = 160.dp, start = 24.dp, end = 24.dp),
horizontalAlignment = Alignment.CenterHorizontally,
) {
// Sphere — dominant share of the column. Compose's `weight()`
// splits *remaining* space (after fixed-size children) among
// weighted children, so 1.5f vs the response's 1f gives the
// sphere ~60% of the available area — matching the old
// `fillMaxHeight(0.6f)` look without drifting as the response
// grows. weight(fill=true) keeps it from collapsing.
Box(
modifier = Modifier
.fillMaxWidth()
.weight(1.5f),
contentAlignment = Alignment.Center,
) {
MorphingSphere(
modifier = Modifier.fillMaxSize(),
state = voiceStateToSphereState(uiState.state),
voiceAmplitude = uiState.amplitude,
voiceMode = true,
)
}
VoiceWaveform(
amplitude = uiState.amplitude,
state = uiState.state,
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 32.dp, vertical = 8.dp),
)
Spacer(Modifier.height(8.dp))
// Destructive-intent countdown indicator. Fades in while a
// SendSms (or future destructive intent) is waiting on the v1
// 5 s confirmation window and fades out the moment dispatch
// completes or the user cancels. Kept deliberately subtle —
// voice mode is voice-first; this is a secondary hint, not
// the primary safety gate (that's still the spoken preview
// + the cancel vocabulary).
DestructiveCountdownRow(
countdown = uiState.destructiveCountdown,
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 32.dp),
)
Spacer(Modifier.height(4.dp))
// "You said" block — sits directly above the agent response so
// the eye flows from the waveform straight down through the
// transcribed text into the answer in one motion. The old top-
// anchored chip forced the user to look up after stopping the
// mic, then back down to read the response — splitting attention.
// Left-aligned + small uppercase caption so it visually pairs
// with the response below it like a mini chat thread.
AnimatedVisibility(
visible = !uiState.transcribedText.isNullOrBlank(),
enter = fadeIn(),
exit = fadeOut(),
) {
Column(
modifier = Modifier
.fillMaxWidth()
.padding(bottom = 6.dp),
horizontalAlignment = Alignment.Start,
) {
Text(
text = "YOU",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.primary,
)
Spacer(Modifier.height(2.dp))
Text(
text = uiState.transcribedText.orEmpty(),
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = TextAlign.Start,
modifier = Modifier.fillMaxWidth(),
)
}
}
// Transcript area. Shows the last N chat messages in a compact
// rolling list, plus the in-flight streaming responseText for
// the current turn. Empty-state hint renders when BOTH the
// transcript AND the live response are empty (first launch,
// fresh voice session).
Box(
modifier = Modifier
.fillMaxWidth()
.weight(1f, fill = true),
) {
val hasTranscript = transcriptMessages.isNotEmpty()
val hasLiveResponse = uiState.responseText.isNotBlank()
if (hasTranscript || hasLiveResponse) {
Column(
modifier = Modifier
.fillMaxSize()
.graphicsLayer { compositingStrategy = CompositingStrategy.Offscreen }
.drawWithContent {
drawContent()
drawRect(brush = fadeBrush, blendMode = BlendMode.DstIn)
}
.verticalScroll(transcriptScrollState)
.padding(vertical = 12.dp),
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
transcriptMessages.forEach { msg ->
CompactTranscriptRow(msg)
}
// Live streaming response for the current turn —
// rendered as a trailing row so it sits below the
// committed history. Once the turn completes and
// the message is appended to chat history,
// responseText clears and this row collapses back
// into the transcript list naturally.
if (hasLiveResponse) {
StreamingResponseRow(uiState.responseText)
}
}
} else {
Box(
modifier = Modifier.fillMaxSize(),
contentAlignment = Alignment.Center,
) {
AnimatedContent(
targetState = stateHint(uiState.state),
transitionSpec = {
fadeIn(tween(200)) togetherWith fadeOut(tween(200))
},
label = "stateHint",
) { hint ->
Text(
text = hint,
style = MaterialTheme.typography.labelLarge,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = TextAlign.Center,
modifier = Modifier.fillMaxWidth(),
)
}
}
}
}
}
// Error banner
AnimatedVisibility(
visible = uiState.error != null,
enter = fadeIn(),
exit = fadeOut(),
modifier = Modifier
.align(Alignment.TopCenter)
.padding(top = 64.dp, start = 16.dp, end = 16.dp),
) {
Surface(
shape = RoundedCornerShape(12.dp),
color = MaterialTheme.colorScheme.errorContainer,
tonalElevation = 2.dp,
) {
Row(
modifier = Modifier.padding(12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = uiState.error.orEmpty(),
color = MaterialTheme.colorScheme.onErrorContainer,
style = MaterialTheme.typography.bodySmall,
modifier = Modifier.weight(1f, fill = false),
)
TextButton(
onClick = {
onClearError()
onMicTap()
},
) {
Icon(
imageVector = Icons.Filled.Refresh,
contentDescription = null,
modifier = Modifier.size(16.dp),
)
Spacer(Modifier.size(4.dp))
Text("Retry")
}
}
}
}
// Mic button — dispatch by current voice state.
//
// Listening → tap stops recording (feeds the audio to STT)
// Speaking → tap interrupts TTS and starts fresh recording
// Idle/Error → tap starts recording
// Transcribing/Thinking → tap is a no-op; the state machine is busy
//
// The bug before was that Listening fell through to the else branch
// which called onMicTap (= startListening) a second time instead of
// stopping the in-progress recording. User ended up with a button
// that visually said "Stop" but was actually wired to "Start."
VoiceMicButton(
uiState = uiState,
onTap = {
when (uiState.state) {
VoiceState.Listening -> {
try {
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
} catch (_: Exception) { /* ignore */ }
onMicRelease()
}
VoiceState.Speaking -> onInterrupt()
VoiceState.Transcribing, VoiceState.Thinking -> {
// Busy — ignore tap. Avoids double-stop / double-send races.
}
VoiceState.Idle, VoiceState.Error -> {
try {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
} catch (_: Exception) { /* ignore */ }
onMicTap()
}
}
},
onPress = {
try {
haptic.performHapticFeedback(HapticFeedbackType.LongPress)
} catch (_: Exception) { /* ignore */ }
onMicTap()
},
onRelease = {
try {
haptic.performHapticFeedback(HapticFeedbackType.TextHandleMove)
} catch (_: Exception) { /* ignore */ }
onMicRelease()
},
modifier = Modifier
.align(Alignment.BottomCenter)
.padding(bottom = 48.dp),
)
}
}
@Composable
private fun VoiceMicButton(
uiState: VoiceUiState,
onTap: () -> Unit,
onPress: () -> Unit,
onRelease: () -> Unit,
modifier: Modifier = Modifier,
) {
val pulseScale by animateFloatAsState(
targetValue = when (uiState.state) {
VoiceState.Listening -> 1f + uiState.amplitude * 0.15f
VoiceState.Speaking -> 1f + uiState.amplitude * 0.08f
else -> 1f
},
animationSpec = tween(120),
label = "micPulse",
)
// Hardcoded vivid red for Listening AND Speaking. Material 3's dark-theme
// `colorScheme.error` role resolves to a soft pink (#F2B8B5) which reads
// as "pale" rather than "STOP" on a circular mic button — the universal
// "record/stop" language is a saturated red, so bypass the role and use
// Material Red 600. Speaking needs the same red because tapping it
// interrupts TTS; the old tertiary (green-ish) read as "playing" and
// users didn't realize they could stop the agent.
val containerColor = when (uiState.state) {
VoiceState.Listening -> Color(0xFFE53935)
VoiceState.Speaking -> Color(0xFFE53935)
VoiceState.Error -> MaterialTheme.colorScheme.errorContainer
else -> MaterialTheme.colorScheme.primary
}
val icon = when (uiState.state) {
VoiceState.Listening -> Icons.Filled.Stop
VoiceState.Transcribing, VoiceState.Thinking -> Icons.Filled.GraphicEq
// Stop icon makes the "tap to interrupt TTS" affordance obvious;
// VolumeUp looked decorative and users didn't try tapping it.
VoiceState.Speaking -> Icons.Filled.Stop
else -> Icons.Filled.Mic
}
val gestureModifier = when (uiState.interactionMode) {
InteractionMode.HoldToTalk -> Modifier.pointerInput(Unit) {
awaitEachGesture {
awaitFirstDown()
onPress()
waitForUpOrCancellation()
onRelease()
}
}
else -> Modifier.clickable { onTap() }
}
Surface(
modifier = modifier
.size((72 * pulseScale).dp)
.clip(CircleShape)
.then(gestureModifier),
shape = CircleShape,
color = containerColor,
shadowElevation = 6.dp,
) {
Box(contentAlignment = Alignment.Center) {
Icon(
imageVector = icon,
contentDescription = "Voice mic",
tint = Color.White,
modifier = Modifier.size(32.dp),
)
}
}
}
private fun voiceStateToSphereState(state: VoiceState): SphereState = when (state) {
VoiceState.Listening -> SphereState.Listening
VoiceState.Speaking -> SphereState.Speaking
VoiceState.Thinking, VoiceState.Transcribing -> SphereState.Thinking
VoiceState.Error -> SphereState.Error
VoiceState.Idle -> SphereState.Idle
}
private fun stateHint(state: VoiceState): String = when (state) {
VoiceState.Idle -> "Tap the mic to speak"
VoiceState.Listening -> "Listening..."
VoiceState.Transcribing -> "Transcribing..."
VoiceState.Thinking -> "Thinking..."
VoiceState.Speaking -> "Speaking..."
VoiceState.Error -> ""
}
private fun InteractionMode.label(): String = when (this) {
InteractionMode.TapToTalk -> "Tap to talk"
InteractionMode.HoldToTalk -> "Hold to talk"
InteractionMode.Continuous -> "Continuous"
}
/**
* Compact transcript row for voice mode. Rendering rules:
*
* - `id.startsWith("voice-intent-")` → voice-action trace, rendered via
* [MarkdownContent] unbounded (these are the rich bridge-intent bubbles
* like "**Opened Chrome** `com.android.chrome` — exact match", which
* need the bold + inline code styling to read well).
* - `role == USER` → small italic caption "YOU" + body in bodySmall,
* two-line truncation.
* - `role == ASSISTANT` → caption "AGENT" + body in bodyMedium,
* four-line truncation.
* - `role == SYSTEM` → skipped entirely (voice mode is a conversation
* surface, not a system-message debug view).
*
* The caller is responsible for bounding the list length (voice mode
* uses `takeLast(6)`).
*/
@Composable
private fun CompactTranscriptRow(message: ChatMessage) {
if (message.role == MessageRole.SYSTEM) return
// A voice-intent trace is a PAIR of messages added by
// ChatHandler.appendLocalVoiceIntentTrace — one USER with the raw
// utterance ("voice-intent-user-$ts") and one ASSISTANT with the
// action description ("voice-intent-action-$ts"). Only the assistant
// half should carry the "ACTION" caption + MarkdownContent rendering;
// the user half should stay labeled "YOU" so the transcript reads as
// a normal user→assistant exchange. Pre-fix we keyed off the id
// prefix alone and mislabeled the user half as ACTION (Bailey
// 2026-04-15 screenshot: duplicate ACTION row with the raw utterance).
val isVoiceActionBubble = message.role == MessageRole.ASSISTANT &&
message.id.startsWith("voice-intent-")
val caption = when {
isVoiceActionBubble -> "ACTION"
message.role == MessageRole.USER -> "YOU"
else -> "AGENT"
}
val captionColor = when {
isVoiceActionBubble -> MaterialTheme.colorScheme.tertiary
message.role == MessageRole.USER -> MaterialTheme.colorScheme.primary
else -> MaterialTheme.colorScheme.secondary
}
Column(
modifier = Modifier.fillMaxWidth(),
horizontalAlignment = Alignment.Start,
) {
Text(
text = caption,
style = MaterialTheme.typography.labelSmall,
color = captionColor,
)
Spacer(Modifier.height(2.dp))
when {
isVoiceActionBubble -> MarkdownContent(
content = message.content,
textColor = MaterialTheme.colorScheme.onSurface,
)
message.role == MessageRole.USER -> Text(
text = message.content,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
maxLines = 2,
overflow = TextOverflow.Ellipsis,
)
else -> Text(
text = message.content,
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurface,
maxLines = 4,
overflow = TextOverflow.Ellipsis,
)
}
}
}
/**
* Renders the 5-second destructive-intent confirmation countdown as a
* thin horizontal progress bar with a short label. The bar fills from
* 0 → 1 over [DestructiveCountdownState.durationMs] using a local
* [Animatable] keyed on the countdown's `startedAtMs` so a fresh
* countdown always restarts cleanly. When [countdown] is null, the row
* fades out entirely.
*
* Visual treatment is intentionally muted (tertiary color, caption-sized
* label) — voice mode is voice-first and the real safety rails live in
* the spoken preview and the cancel vocabulary. This is just "something
* is about to happen" scaffolding for sighted users.
*/
@Composable
private fun DestructiveCountdownRow(
countdown: DestructiveCountdownState?,
modifier: Modifier = Modifier,
) {
// Hold the most-recent non-null countdown so the row can keep rendering
// its fade-out transition after the VM clears the state. Without this
// the label would snap to "" at the same frame the fade begins.
var lastShown by remember { mutableStateOf<DestructiveCountdownState?>(null) }
LaunchedEffect(countdown) {
if (countdown != null) lastShown = countdown
}
val progress = remember { Animatable(0f) }
LaunchedEffect(countdown?.startedAtMs) {
if (countdown != null) {
progress.snapTo(0f)
progress.animateTo(
targetValue = 1f,
animationSpec = tween(
durationMillis = countdown.durationMs.toInt().coerceAtLeast(100),
easing = LinearEasing,
),
)
} else {
progress.snapTo(0f)
}
}
AnimatedVisibility(
visible = countdown != null,
enter = fadeIn(tween(150)),
exit = fadeOut(tween(200)),
modifier = modifier,
) {
val label = (countdown ?: lastShown)?.intentLabel ?: ""
val durationSec = ((countdown ?: lastShown)?.durationMs ?: 0L) / 1000
Column(
modifier = Modifier.fillMaxWidth(),
horizontalAlignment = Alignment.Start,
) {
Text(
text = if (label.isNotBlank()) {
"$label in ${durationSec}s — say cancel to stop"
} else {
"Confirming…"
},
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.tertiary,
modifier = Modifier.fillMaxWidth(),
)
Spacer(Modifier.height(4.dp))
LinearProgressIndicator(
progress = { progress.value.coerceIn(0f, 1f) },
modifier = Modifier.fillMaxWidth(),
color = MaterialTheme.colorScheme.tertiary,
trackColor = MaterialTheme.colorScheme.surfaceVariant,
)
}
}
}
/**
* Row for the in-flight streaming response — `uiState.responseText` that
* hasn't yet been committed to chat history. Separate composable so it
* can render slightly differently from committed [CompactTranscriptRow]
* (larger type, onSurface instead of onSurfaceVariant) matching the
* pre-transcript single-line "current response" visual weight, and so
* streaming tokens accrete in place without the AnimatedContent flicker
* we had before transcript mode.
*/
@Composable
private fun StreamingResponseRow(responseText: String) {
Column(
modifier = Modifier.fillMaxWidth(),
horizontalAlignment = Alignment.Start,
) {
Text(
text = "AGENT",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.secondary,
)
Spacer(Modifier.height(2.dp))
Text(
text = responseText,
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurface,
textAlign = TextAlign.Start,
modifier = Modifier.fillMaxWidth(),
)
}
}
@@ -0,0 +1,437 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.animation.animateColorAsState
import androidx.compose.animation.core.tween
import androidx.compose.foundation.Canvas
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.runtime.setValue
import androidx.compose.runtime.withFrameNanos
import androidx.compose.ui.Modifier
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.graphics.BlendMode
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.Path
import androidx.compose.ui.graphics.StrokeCap
import androidx.compose.ui.graphics.drawscope.Stroke
import androidx.compose.ui.graphics.drawscope.drawIntoCanvas
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.viewmodel.VoiceState
import kotlin.math.PI
import kotlin.math.max
import kotlin.math.sin
/**
* Reactive layered-sine-wave visualizer.
*
* Mounted below the [MorphingSphere] in voice-mode overlay. Renders three
* overlapping sine waves at different frequencies, coupled to the live mic
* amplitude, and color-keyed to the current [VoiceState] so the waveform
* shifts between the user's voice (soft blue/purple) and the agent's voice
* (vivid green/teal) automatically.
*
* Design goals:
* - Siri / ChatGPT voice smooth, not a spiky equalizer
* - Subtle accent to the sphere, not a competing focal point
* - Color palette must exactly match MorphingSphere's Listening/Speaking
* states so sphere + waveform read as one cohesive element
* - Never fully disappears during silence — baseline ~10% envelope keeps
* the component alive so the UI doesn't flicker on/off with voice gaps
*
* Pure Compose Canvas — no external libraries.
*/
// Exact hex values from MorphingSphere palette (see colorsFor() in that file).
// Listening poles (cool soft blue/purple) and Speaking poles (vivid green/teal).
private val ListeningPrimary = Color(0xFF597EF2) // soft blue
private val ListeningSecondary = Color(0xFFA573F2) // soft purple
private val SpeakingPrimary = Color(0xFF40EB8C) // vivid green
private val SpeakingSecondary = Color(0xFF4DD9E0) // teal
// Three sine layers. Base frequencies in "full-canvas-widths per cycle" units:
// wave 1 at 1.2 fits roughly one and a bit crests across the component, wave 3
// at 3.4 fits three-ish. Tuned by eye in the previews — they drift against
// each other because the phase accumulators each run on independent clocks.
private val baseFreqs = floatArrayOf(1.2f, 2.1f, 3.4f)
// Relative amplitude weight per layer. Wave 1 is the dominant shape, 2 and 3
// add detail and shimmer on top.
private val layerScales = floatArrayOf(1.0f, 0.70f, 0.40f)
// Relative saturation per layer. Back layers ride a little dimmer so the
// front layer reads as the primary contour.
private val layerAlphas = floatArrayOf(1.0f, 0.80f, 0.60f)
// Phase-cycle durations in millis at silence. Co-prime-ish so layers never
// phase-lock. The amplitude-driven phase driver multiplies velocity by up to
// PHASE_AMP_BOOST at amplitude=1, so at a full-throated speech peak wave 1
// does a cycle in ~570 ms — the characteristic Siri "surge."
private val phaseDurationsMs = intArrayOf(2000, 1400, 950)
// Multiplier applied to phase velocity at amplitude=1.0. 1 + amp*2.5 means
// silence runs at base tempo, normal speech (amp≈0.5) runs ~2.25× faster,
// loud speech (amp=1.0) runs 3.5× faster. This is what makes the wave feel
// like it's being pushed by your voice instead of drifting on its own clock.
private const val PHASE_AMP_BOOST = 2.5f
// Envelope math: each wave's max excursion is `amp * layerScale * (h * peak)`.
// `peak = 0.55` means at amplitude=1.0 the dominant wave reaches 55 % of the
// half-height — nearly the full canvas span minus stroke headroom.
private const val PEAK_FRACTION = 0.55f
// Minimum envelope at amplitude=0. Keeps the component faintly alive at
// silence so it doesn't collapse to a dead straight line between turns.
private const val MIN_ENVELOPE = 0.04f
// Sample one path vertex every N pixels. Lower = smoother but more work per
// frame. 2 px is plenty smooth at this stroke width.
private const val SAMPLE_STEP_PX = 2f
// Stroke width in dp.
private const val STROKE_WIDTH_DP = 2.5f
// Fraction of the canvas width devoted to each edge's alpha falloff. Conjure's
// pill overlay masks the outer ~15% of each side with an opaque gradient; we
// replicate it as an alpha mask here since we're sitting on a translucent
// overlay, not a solid pill. 0.12 = 12 % fade per side, 76 % solid middle.
private const val EDGE_FADE_FRACTION = 0.12f
@Composable
fun VoiceWaveform(
amplitude: Float,
state: VoiceState,
modifier: Modifier = Modifier,
) {
// No downstream smoothing. The VoiceViewModel already runs an
// attack/release envelope follower on the recorder/player amplitude
// flow, so we get an instantly-responsive-on-peaks / slow-fade-on-tail
// signal here. Stacking a Compose spring on top would re-introduce the
// very lag that made the waveform feel dead during the V2b round.
val displayAmplitude = amplitude.coerceIn(0f, 1f)
// State-driven color pair. animateColorAsState gives us a ~400ms crossfade
// between palettes so transitioning from Listening → Thinking → Speaking
// doesn't look abrupt.
val dim = MaterialTheme.colorScheme.onSurfaceVariant
val errorColor = MaterialTheme.colorScheme.error
val targetPrimary = when (state) {
VoiceState.Idle -> dim.copy(alpha = 0.3f)
VoiceState.Listening -> ListeningPrimary
VoiceState.Transcribing -> ListeningPrimary.copy(alpha = 0.6f)
VoiceState.Thinking -> dim.copy(alpha = 0.5f)
VoiceState.Speaking -> SpeakingPrimary
VoiceState.Error -> errorColor.copy(alpha = 0.8f)
}
val targetSecondary = when (state) {
VoiceState.Idle -> dim.copy(alpha = 0.3f)
VoiceState.Listening -> ListeningSecondary
VoiceState.Transcribing -> dim.copy(alpha = 0.6f)
VoiceState.Thinking -> dim.copy(alpha = 0.5f)
VoiceState.Speaking -> SpeakingSecondary
VoiceState.Error -> errorColor.copy(alpha = 0.8f)
}
val primaryColor by animateColorAsState(
targetValue = targetPrimary,
animationSpec = tween(durationMillis = 400),
label = "wavePrimary",
)
val secondaryColor by animateColorAsState(
targetValue = targetSecondary,
animationSpec = tween(durationMillis = 400),
label = "waveSecondary",
)
// Three phase accumulators driven by a single per-frame ticker. Phase
// velocity scales with the current amplitude so the wave visibly surges
// when the user speaks instead of running on its own fixed clock.
val phases = rememberAmplitudeDrivenPhases(phaseDurationsMs, displayAmplitude)
Canvas(
modifier = modifier
.fillMaxWidth()
.height(56.dp),
) {
val width = size.width
val height = size.height
if (width <= 0f || height <= 0f) return@Canvas
val centerY = height / 2f
val peakPixels = height * PEAK_FRACTION
val strokePx = STROKE_WIDTH_DP.dp.toPx()
// Sample every SAMPLE_STEP_PX pixels across the canvas. steps+1 vertices.
val steps = max(2, (width / SAMPLE_STEP_PX).toInt())
// Offscreen layer so we can mask the stroke alpha with DstIn and the
// mask never bleeds onto whatever's underneath the waveform. Without
// saveLayer the DstIn would punch a hole in the parent background.
drawIntoCanvas { canvas ->
canvas.saveLayer(
bounds = androidx.compose.ui.geometry.Rect(0f, 0f, width, height),
paint = androidx.compose.ui.graphics.Paint(),
)
// Draw back-to-front so wave 1 (front, full saturation) sits on top.
for (layer in 2 downTo 0) {
val freq = baseFreqs[layer]
val phase = phases[layer]
val layerScale = layerScales[layer]
val layerAlpha = layerAlphas[layer]
// Amplitude envelope for this layer. Uses the gain-curved
// displayAmplitude so low speech levels still move the wave.
// Baseline of MIN_ENVELOPE keeps the component faintly alive at
// true silence (amplitude = 0).
val rawEnvelope = displayAmplitude * layerScale * peakPixels
val minEnvelope = MIN_ENVELOPE * peakPixels * layerScale
val envelope = max(rawEnvelope, minEnvelope)
val path = Path()
path.moveTo(0f, centerY)
for (i in 0..steps) {
val x = i * (width / steps)
val t = x / width
val angle = (t * freq * 2f * PI).toFloat() + phase
// Geometric tuck-in: sin(πt) goes 0 → 1 → 0 across the
// canvas so the sine excursion is forced to zero at both
// edges. This pulls both endpoints down to centerY and
// makes the wave form a natural pill/lens silhouette
// without any hard cut at the canvas boundary.
val taper = sin(PI.toFloat() * t)
val y = centerY + sin(angle) * envelope * taper
if (i == 0) path.moveTo(x, y) else path.lineTo(x, y)
}
drawPath(
path = path,
brush = Brush.horizontalGradient(
listOf(
primaryColor.copy(alpha = primaryColor.alpha * layerAlpha),
secondaryColor.copy(alpha = secondaryColor.alpha * layerAlpha),
),
),
style = Stroke(width = strokePx, cap = StrokeCap.Round),
)
}
// Alpha-mask the edges. Conjure paints the pill background over
// the outer 15 % of each side; on a translucent overlay we instead
// modulate the stroke alpha directly with DstIn so only the wave
// pixels get feathered — the parent background is untouched.
// Belt-and-suspenders with the geometric tuck above: the mask
// catches any stray stroke width near the edges that the taper
// leaves behind.
drawRect(
brush = Brush.horizontalGradient(
colorStops = arrayOf(
0f to Color.Transparent,
EDGE_FADE_FRACTION to Color.Black,
1f - EDGE_FADE_FRACTION to Color.Black,
1f to Color.Transparent,
),
),
topLeft = Offset.Zero,
size = size,
blendMode = BlendMode.DstIn,
)
canvas.restore()
}
}
}
/**
* Per-frame phase ticker for the three wave layers. Runs one [withFrameNanos]
* loop and updates all three phases atomically so they can't tear against
* each other. Phase velocity = base-omega × (1 + amplitude × [PHASE_AMP_BOOST]),
* so the wave accelerates when the user speaks.
*/
@Composable
private fun rememberAmplitudeDrivenPhases(
baseDurationsMs: IntArray,
amplitude: Float,
): FloatArray {
val ampRef = rememberUpdatedState(amplitude)
var phases by remember { mutableStateOf(FloatArray(baseDurationsMs.size)) }
LaunchedEffect(Unit) {
val twoPi = (2f * PI).toFloat()
// Precompute base angular velocities (rad/s) so we don't divide on
// every frame. Each wave's full cycle at silence = durationMs.
val baseOmega = FloatArray(baseDurationsMs.size) { i ->
twoPi / (baseDurationsMs[i] / 1000f)
}
var prevNanos = 0L
while (true) {
withFrameNanos { nanos ->
if (prevNanos == 0L) {
prevNanos = nanos
return@withFrameNanos
}
val dtSec = (nanos - prevNanos) / 1_000_000_000f
prevNanos = nanos
val boost = 1f + ampRef.value * PHASE_AMP_BOOST
val current = phases
val next = FloatArray(current.size)
for (i in current.indices) {
var p = current[i] + baseOmega[i] * dtSec * boost
// Keep phase in [0, 2π) so sin() stays in its efficient
// range and the float doesn't drift to a large magnitude
// over long voice sessions.
if (p >= twoPi) p -= twoPi
next[i] = p
}
phases = next
}
}
}
return phases
}
// ── Previews ─────────────────────────────────────────────────────────
@Preview(
name = "Idle",
showBackground = true,
backgroundColor = 0xFF0A0A0A,
widthDp = 360,
heightDp = 80,
)
@Composable
private fun VoiceWaveformIdlePreview() {
Box(
Modifier
.fillMaxSize()
.background(Color(0xFF0A0A0A)),
) {
VoiceWaveform(
amplitude = 0.0f,
state = VoiceState.Idle,
modifier = Modifier.padding(16.dp),
)
}
}
@Preview(
name = "Listening low",
showBackground = true,
backgroundColor = 0xFF0A0A0A,
widthDp = 360,
heightDp = 80,
)
@Composable
private fun VoiceWaveformListeningLowPreview() {
Box(
Modifier
.fillMaxSize()
.background(Color(0xFF0A0A0A)),
) {
VoiceWaveform(
amplitude = 0.25f,
state = VoiceState.Listening,
modifier = Modifier.padding(16.dp),
)
}
}
@Preview(
name = "Listening peak",
showBackground = true,
backgroundColor = 0xFF0A0A0A,
widthDp = 360,
heightDp = 80,
)
@Composable
private fun VoiceWaveformListeningPeakPreview() {
Box(
Modifier
.fillMaxSize()
.background(Color(0xFF0A0A0A)),
) {
VoiceWaveform(
amplitude = 0.9f,
state = VoiceState.Listening,
modifier = Modifier.padding(16.dp),
)
}
}
@Preview(
name = "Speaking peak",
showBackground = true,
backgroundColor = 0xFF0A0A0A,
widthDp = 360,
heightDp = 80,
)
@Composable
private fun VoiceWaveformSpeakingPeakPreview() {
Box(
Modifier
.fillMaxSize()
.background(Color(0xFF0A0A0A)),
) {
VoiceWaveform(
amplitude = 0.9f,
state = VoiceState.Speaking,
modifier = Modifier.padding(16.dp),
)
}
}
@Preview(
name = "Thinking",
showBackground = true,
backgroundColor = 0xFF0A0A0A,
widthDp = 360,
heightDp = 80,
)
@Composable
private fun VoiceWaveformThinkingPreview() {
Box(
Modifier
.fillMaxSize()
.background(Color(0xFF0A0A0A)),
) {
VoiceWaveform(
amplitude = 0.1f,
state = VoiceState.Thinking,
modifier = Modifier.padding(16.dp),
)
}
}
@Preview(
name = "Error",
showBackground = true,
backgroundColor = 0xFF0A0A0A,
widthDp = 360,
heightDp = 80,
)
@Composable
private fun VoiceWaveformErrorPreview() {
Box(
Modifier
.fillMaxSize()
.background(Color(0xFF0A0A0A)),
) {
VoiceWaveform(
amplitude = 0.3f,
state = VoiceState.Error,
modifier = Modifier.padding(16.dp),
)
}
}
@@ -1,6 +1,10 @@
package com.hermesandroid.relay.ui.onboarding
import androidx.compose.foundation.background
import androidx.compose.foundation.isSystemInDarkTheme
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.BoxScope
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.ColumnScope
import androidx.compose.foundation.layout.Spacer
@@ -8,19 +12,27 @@ import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Chat
import androidx.compose.material.icons.automirrored.filled.Chat
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import com.hermesandroid.relay.ui.theme.gradientBorder
@Composable
fun OnboardingPage(
@@ -28,43 +40,124 @@ fun OnboardingPage(
title: String,
description: String,
modifier: Modifier = Modifier,
heroContent: @Composable BoxScope.() -> Unit = {
FeatureHero(
icon = icon,
title = title,
)
},
content: @Composable ColumnScope.() -> Unit = {}
) {
val isDarkTheme = isSystemInDarkTheme()
val heroShape = RoundedCornerShape(30.dp)
val bodyShape = RoundedCornerShape(26.dp)
val heroBrush = Brush.radialGradient(
colors = if (isDarkTheme) {
listOf(
MaterialTheme.colorScheme.surfaceContainerHighest.copy(alpha = 0.96f),
MaterialTheme.colorScheme.surfaceContainer.copy(alpha = 0.92f),
MaterialTheme.colorScheme.surfaceContainerLow.copy(alpha = 0.98f),
)
} else {
listOf(
MaterialTheme.colorScheme.primaryContainer.copy(alpha = 0.70f),
MaterialTheme.colorScheme.surface.copy(alpha = 0.98f),
MaterialTheme.colorScheme.secondaryContainer.copy(alpha = 0.60f),
)
}
)
Column(
modifier = modifier
.fillMaxWidth()
.padding(horizontal = 32.dp),
.widthIn(max = 560.dp)
.padding(horizontal = 24.dp, vertical = 12.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.Center
) {
Icon(
imageVector = icon,
contentDescription = title,
modifier = Modifier.size(72.dp),
tint = MaterialTheme.colorScheme.primary
)
Card(
modifier = Modifier
.fillMaxWidth()
.height(232.dp)
.gradientBorder(shape = heroShape, isDarkTheme = isDarkTheme),
shape = heroShape,
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceContainer
)
) {
Box(
modifier = Modifier
.fillMaxWidth()
.height(232.dp)
.background(heroBrush),
contentAlignment = Alignment.Center
) {
heroContent()
}
}
Spacer(modifier = Modifier.height(16.dp))
Spacer(modifier = Modifier.height(18.dp))
Text(
text = title,
style = MaterialTheme.typography.headlineMedium,
textAlign = TextAlign.Center,
color = MaterialTheme.colorScheme.onSurface
)
Card(
modifier = Modifier
.fillMaxWidth()
.gradientBorder(shape = bodyShape, isDarkTheme = isDarkTheme),
shape = bodyShape,
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(
modifier = Modifier.padding(horizontal = 24.dp, vertical = 22.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(14.dp)
) {
Text(
text = title,
style = MaterialTheme.typography.headlineMedium,
textAlign = TextAlign.Center,
color = MaterialTheme.colorScheme.onSurface
)
Spacer(modifier = Modifier.height(16.dp))
Text(
text = description,
style = MaterialTheme.typography.bodyLarge,
textAlign = TextAlign.Center,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
Text(
text = description,
style = MaterialTheme.typography.bodyLarge,
textAlign = TextAlign.Center,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
content()
}
}
}
}
Spacer(modifier = Modifier.height(16.dp))
content()
@Composable
private fun FeatureHero(
icon: ImageVector,
title: String,
) {
val isDarkTheme = isSystemInDarkTheme()
Box(
modifier = Modifier.fillMaxWidth(),
contentAlignment = Alignment.Center
) {
Box(
modifier = Modifier
.size(136.dp)
.clip(CircleShape)
.background(
MaterialTheme.colorScheme.primary.copy(alpha = if (isDarkTheme) 0.14f else 0.10f)
),
contentAlignment = Alignment.Center
) {
Icon(
imageVector = icon,
contentDescription = title,
modifier = Modifier.size(74.dp),
tint = MaterialTheme.colorScheme.primary
)
}
}
}
@@ -73,7 +166,7 @@ fun OnboardingPage(
private fun OnboardingPagePreview() {
HermesRelayTheme {
OnboardingPage(
icon = Icons.Filled.Chat,
icon = Icons.AutoMirrored.Filled.Chat,
title = "Talk to Your Agent",
description = "Stream conversations with any Hermes profile. Ask questions, run tasks, and collaborate in real time."
)
@@ -85,7 +178,7 @@ private fun OnboardingPagePreview() {
private fun OnboardingPageWithContentPreview() {
HermesRelayTheme {
OnboardingPage(
icon = Icons.Filled.Chat,
icon = Icons.AutoMirrored.Filled.Chat,
title = "Let's Connect",
description = "Enter your relay server URL to get started."
) {
@@ -1,10 +1,12 @@
package com.hermesandroid.relay.ui.onboarding
import android.content.Intent
import android.net.Uri
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import android.content.Intent
import android.net.Uri
import androidx.compose.foundation.Image
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
@@ -17,30 +19,20 @@ import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.pager.HorizontalPager
import androidx.compose.foundation.pager.rememberPagerState
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.HelpOutline
import android.widget.Toast
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
import androidx.compose.material.icons.filled.QrCodeScanner
import androidx.compose.material.icons.filled.Visibility
import androidx.compose.material.icons.filled.VisibilityOff
import androidx.compose.material.icons.outlined.Dns
import androidx.compose.material.icons.automirrored.outlined.MenuBook
import androidx.compose.material.icons.outlined.Forum
import androidx.compose.material.icons.outlined.Hub
import androidx.compose.material.icons.outlined.PhonelinkSetup
import androidx.compose.material.icons.outlined.RocketLaunch
import androidx.compose.material.icons.outlined.Terminal
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
@@ -53,41 +45,63 @@ import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.input.PasswordVisualTransformation
import androidx.compose.ui.text.input.VisualTransformation
import androidx.compose.ui.draw.clip
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.res.painterResource
import androidx.compose.ui.text.style.TextDecoration
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import androidx.compose.runtime.collectAsState
import androidx.lifecycle.viewmodel.compose.viewModel
import com.hermesandroid.relay.R
import com.hermesandroid.relay.data.FeatureFlags
import com.hermesandroid.relay.ui.components.ConnectionStatusBadge
import com.hermesandroid.relay.ui.components.QrPairingScanner
import com.hermesandroid.relay.ui.components.MorphingSphere
import com.hermesandroid.relay.ui.components.SphereState
import com.hermesandroid.relay.ui.components.ConnectionWizard
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import kotlinx.coroutines.Dispatchers
import com.hermesandroid.relay.viewmodel.ConnectionViewModel
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
import okhttp3.OkHttpClient
import okhttp3.Request
import java.io.IOException
import java.net.ConnectException
import java.net.SocketTimeoutException
import java.net.UnknownHostException
import javax.net.ssl.SSLException
import java.util.concurrent.TimeUnit
private sealed interface TestResult {
data object Success : TestResult
data class Failure(val message: String) : TestResult
}
/** Page identifiers for dynamic onboarding flow. */
private enum class OnboardingPage { Welcome, Chat, Terminal, Bridge, Connect, Relay }
private enum class OnboardingPage { Welcome, Chat, Terminal, Bridge, Connect }
/**
* Three-stage onboarding:
*
* 1. **Feature pages** (Welcome / Chat / Terminal / Bridge) — informational
* swipe-able introduction. Bridge / Terminal pages only show when the
* relay feature is enabled (Developer Options).
* 2. **Connect page** — embeds the shared [ConnectionWizard] so onboarding
* uses the exact same scan → confirm → verify flow as Settings → Connection.
* The wizard owns credential application; on success / skip it calls back
* into [onComplete] to finish onboarding and navigate to chat.
*
* The previous separate "ConnectPage" + "RelayPage" pair has been removed —
* it discarded the QR's relay block, never applied per-channel grants, never
* walked the user through the TTL picker, and let users exit onboarding in a
* "half-paired" state that broke the relay/voice/bridge features.
*/
@Composable
fun OnboardingScreen(
onComplete: (apiServerUrl: String, apiKey: String, relayUrl: String) -> Unit
// === onboarding-pair-bug-fix (2026-04-13) ===
// CRITICAL: the ConnectionViewModel MUST be passed in from the top
// level of RelayApp, not fetched here via `viewModel()`. A bare
// `viewModel()` call inside a composable that lives under a
// `composable(Screen.Onboarding.route) { ... }` block binds to the
// NavBackStackEntry's ViewModelStore, not the Activity's. When
// onboarding completes and we `popUpTo(Onboarding) { inclusive = true }`
// during navigation to Chat, that backstack entry is destroyed and
// the scoped VM's `onCleared()` runs → `connectionManager.shutdown()`
// → WSS `close(1000)` → the freshly-minted session token is thrown
// away. Meanwhile Chat uses the Activity-scoped instance (a DIFFERENT
// ConnectionViewModel), which never saw the pair and has no token.
// Symptom: "onboarding reports success but only the API URL survived."
//
// Passing the VM in explicitly forces us to share the Activity-scoped
// instance that Chat / Settings / Bridge all use, so the pair state
// lands on the right VM and survives the Onboarding→Chat transition.
connectionViewModel: ConnectionViewModel,
onComplete: () -> Unit,
) {
val context = LocalContext.current
val relayEnabled by FeatureFlags.relayEnabled(context).collectAsState(initial = FeatureFlags.isDevBuild)
@@ -102,9 +116,6 @@ fun OnboardingScreen(
add(OnboardingPage.Bridge)
}
add(OnboardingPage.Connect)
if (relayEnabled) {
add(OnboardingPage.Relay)
}
}
}
val pageCount = pages.size
@@ -112,9 +123,6 @@ fun OnboardingScreen(
val pagerState = rememberPagerState(pageCount = { pageCount })
val coroutineScope = rememberCoroutineScope()
var apiServerUrl by rememberSaveable { mutableStateOf("http://localhost:8642") }
var apiKey by rememberSaveable { mutableStateOf("") }
var relayUrl by rememberSaveable { mutableStateOf("wss://localhost:8767") }
var showSkipConfirm by rememberSaveable { mutableStateOf(false) }
Surface(
@@ -126,12 +134,12 @@ fun OnboardingScreen(
onDismissRequest = { showSkipConfirm = false },
title = { Text("Skip setup?") },
text = {
Text("You can configure your server connection later in Settings. Without an API key, your connection will not be authenticated.")
Text("You can configure your server connection later in Settings → Connection. Without pairing, chat and voice features won't work yet.")
},
confirmButton = {
TextButton(onClick = {
showSkipConfirm = false
onComplete(apiServerUrl, apiKey, relayUrl)
onComplete()
}) {
Text("Skip anyway")
}
@@ -144,10 +152,9 @@ fun OnboardingScreen(
)
}
Column(
modifier = Modifier.fillMaxSize()
) {
// Top bar with Skip
Column(modifier = Modifier.fillMaxSize()) {
// Top bar with Skip — only on informational pages, not the wizard
// (which has its own "Skip for now" affordance).
Row(
modifier = Modifier
.fillMaxWidth()
@@ -155,13 +162,14 @@ fun OnboardingScreen(
horizontalArrangement = Arrangement.End,
verticalAlignment = Alignment.CenterVertically
) {
// Skip button - always visible
TextButton(onClick = { showSkipConfirm = true }) {
Text(
text = "Skip",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
if (pages[pagerState.currentPage] != OnboardingPage.Connect) {
TextButton(onClick = { showSkipConfirm = true }) {
Text(
text = "Skip",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
}
}
}
@@ -180,80 +188,65 @@ fun OnboardingScreen(
OnboardingPage.Terminal -> TerminalPage()
OnboardingPage.Bridge -> BridgePage()
OnboardingPage.Connect -> ConnectPage(
apiServerUrl = apiServerUrl,
onApiServerUrlChange = { apiServerUrl = it },
apiKey = apiKey,
onApiKeyChange = { apiKey = it }
)
OnboardingPage.Relay -> RelayPage(
relayUrl = relayUrl,
onRelayUrlChange = { relayUrl = it }
connectionViewModel = connectionViewModel,
onComplete = onComplete,
onSkip = { showSkipConfirm = true },
)
}
}
}
// Bottom section: page indicator + navigation buttons
Column(
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 32.dp)
.padding(bottom = 48.dp),
horizontalAlignment = Alignment.CenterHorizontally
) {
// Page indicator
PageIndicator(
pageCount = pageCount,
currentPage = pagerState.currentPage
)
Spacer(modifier = Modifier.height(24.dp))
// Navigation buttons row: Back (left) + Next/Get Started (right)
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically
// Bottom navigation only on informational pages — the wizard
// owns its own back/pair affordances.
if (pages[pagerState.currentPage] != OnboardingPage.Connect) {
Column(
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 32.dp)
.padding(bottom = 48.dp),
horizontalAlignment = Alignment.CenterHorizontally
) {
// Back button — only visible when not on first page
AnimatedVisibility(
visible = pagerState.currentPage > 0,
enter = fadeIn(),
exit = fadeOut()
PageIndicator(
pageCount = pageCount,
currentPage = pagerState.currentPage
)
Spacer(modifier = Modifier.height(24.dp))
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically
) {
TextButton(
onClick = {
coroutineScope.launch {
pagerState.animateScrollToPage(pagerState.currentPage - 1)
}
}
AnimatedVisibility(
visible = pagerState.currentPage > 0,
enter = fadeIn(),
exit = fadeOut()
) {
Text(text = "Back")
TextButton(
onClick = {
coroutineScope.launch {
pagerState.animateScrollToPage(pagerState.currentPage - 1)
}
}
) {
Text(text = "Back")
}
}
if (pagerState.currentPage == 0) {
Spacer(modifier = Modifier.width(1.dp))
}
}
// Invisible spacer when Back is hidden so Next/Get Started stays right-aligned
if (pagerState.currentPage == 0) {
Spacer(modifier = Modifier.width(1.dp))
}
// Next / Get Started button
if (pagerState.currentPage < lastPage) {
Button(
onClick = {
coroutineScope.launch {
pagerState.animateScrollToPage(pagerState.currentPage + 1)
pagerState.animateScrollToPage(
(pagerState.currentPage + 1).coerceAtMost(lastPage)
)
}
}
) {
Text(text = "Next")
}
} else {
Button(
onClick = { onComplete(apiServerUrl, apiKey, relayUrl) },
enabled = apiServerUrl.isNotBlank()
) {
Text(text = "Get Started")
Text(text = if (pagerState.currentPage == lastPage - 1) "Connect" else "Next")
}
}
}
@@ -268,9 +261,95 @@ private fun WelcomePage() {
OnboardingPage(
icon = Icons.Outlined.RocketLaunch,
title = "Hermes-Relay",
description = "Your Hermes agent, in your pocket."
description = "Your Hermes agent, in your pocket.",
heroContent = {
Box(modifier = Modifier.fillMaxSize()) {
MorphingSphere(
modifier = Modifier
.fillMaxSize()
.padding(horizontal = 12.dp, vertical = 6.dp),
state = SphereState.Idle,
intensity = 0.12f,
)
Box(
modifier = Modifier
.align(Alignment.TopStart)
.padding(16.dp)
.clip(RoundedCornerShape(999.dp))
.background(MaterialTheme.colorScheme.surface.copy(alpha = 0.80f))
.padding(horizontal = 12.dp, vertical = 6.dp)
) {
Text(
text = "Welcome",
style = MaterialTheme.typography.labelLarge,
color = MaterialTheme.colorScheme.onSurface,
)
}
Box(
modifier = Modifier
.align(Alignment.BottomCenter)
.padding(bottom = 18.dp)
.size(60.dp)
.clip(RoundedCornerShape(18.dp))
.background(MaterialTheme.colorScheme.surface.copy(alpha = 0.88f)),
contentAlignment = Alignment.Center
) {
Image(
painter = painterResource(R.drawable.ic_launcher_foreground),
contentDescription = "Hermes-Relay logo",
modifier = Modifier.size(42.dp)
)
}
}
}
) {
Spacer(modifier = Modifier.height(12.dp))
Text(
text = "Read the app guide, browse the repo, or jump to Hermes Agent docs while you finish server setup.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(10.dp)
) {
OutlinedButton(
onClick = {
context.startActivity(
Intent(Intent.ACTION_VIEW, Uri.parse("https://codename-11.github.io/hermes-relay/"))
)
},
modifier = Modifier.weight(1f)
) {
Icon(
imageVector = Icons.AutoMirrored.Outlined.MenuBook,
contentDescription = null,
modifier = Modifier.size(16.dp)
)
Spacer(modifier = Modifier.width(6.dp))
Text("User Guide")
}
OutlinedButton(
onClick = {
context.startActivity(
Intent(Intent.ACTION_VIEW, Uri.parse("https://github.com/Codename-11/hermes-relay"))
)
},
modifier = Modifier.weight(1f)
) {
Icon(
painter = painterResource(R.drawable.ic_github),
contentDescription = null,
modifier = Modifier.size(16.dp)
)
Spacer(modifier = Modifier.width(6.dp))
Text("GitHub")
}
}
Text(
text = "hermes-agent.nousresearch.com",
style = MaterialTheme.typography.bodySmall.copy(
@@ -313,359 +392,35 @@ private fun BridgePage() {
@Composable
private fun ConnectPage(
apiServerUrl: String,
onApiServerUrlChange: (String) -> Unit,
apiKey: String,
onApiKeyChange: (String) -> Unit
connectionViewModel: ConnectionViewModel,
onComplete: () -> Unit,
onSkip: () -> Unit,
) {
val context = LocalContext.current
val coroutineScope = rememberCoroutineScope()
var apiKeyVisible by rememberSaveable { mutableStateOf(false) }
var testResult by remember { mutableStateOf<TestResult?>(null) }
var isTesting by rememberSaveable { mutableStateOf(false) }
var showApiKeyHelp by rememberSaveable { mutableStateOf(false) }
var showQrScanner by remember { mutableStateOf(false) }
val cameraPermissionLauncher = rememberLauncherForActivityResult(
contract = ActivityResultContracts.RequestPermission()
) { granted ->
if (granted) {
showQrScanner = true
} else {
Toast.makeText(context, "Camera permission needed to scan QR codes", Toast.LENGTH_SHORT).show()
}
}
// API Key help dialog
if (showApiKeyHelp) {
AlertDialog(
onDismissRequest = { showApiKeyHelp = false },
title = { Text("Where to find your API key") },
text = {
Column(verticalArrangement = Arrangement.spacedBy(12.dp)) {
Text(
"Your API key is the API_SERVER_KEY value from your Hermes server configuration.",
style = MaterialTheme.typography.bodyMedium
)
Text(
"Location: ~/.hermes/.env",
style = MaterialTheme.typography.bodyMedium,
fontFamily = FontFamily.Monospace
)
Text(
"If you haven't set a key yet, add one to your server config — it secures all API communication.",
style = MaterialTheme.typography.bodyMedium
)
Text(
"Default port: 8642",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
Spacer(modifier = Modifier.height(4.dp))
val helpContext = LocalContext.current
Text(
text = "View full setup guide",
style = MaterialTheme.typography.bodySmall.copy(
textDecoration = TextDecoration.Underline
),
color = MaterialTheme.colorScheme.primary,
modifier = Modifier.clickable {
helpContext.startActivity(
Intent(Intent.ACTION_VIEW, Uri.parse("https://hermes-agent.nousresearch.com"))
)
}
)
}
},
confirmButton = {
TextButton(onClick = { showApiKeyHelp = false }) {
Text("Got it")
}
}
)
}
OnboardingPage(
icon = Icons.Outlined.Dns,
title = "Connect",
description = "Enter your Hermes server address and API key to connect securely."
Box(
modifier = Modifier
.fillMaxSize()
.padding(top = 8.dp),
contentAlignment = Alignment.TopCenter,
) {
Spacer(modifier = Modifier.height(8.dp))
// API Server URL
OutlinedTextField(
value = apiServerUrl,
onValueChange = {
onApiServerUrlChange(it)
testResult = null
},
label = { Text("API Server URL") },
placeholder = { Text("http://your-server:8642") },
singleLine = true,
modifier = Modifier.fillMaxWidth()
)
Spacer(modifier = Modifier.height(8.dp))
// API Key with help icon
OutlinedTextField(
value = apiKey,
onValueChange = {
onApiKeyChange(it)
testResult = null
},
label = {
Row(verticalAlignment = Alignment.CenterVertically) {
Text("API Key")
Spacer(modifier = Modifier.width(4.dp))
Icon(
imageVector = Icons.AutoMirrored.Filled.HelpOutline,
contentDescription = "API key help",
modifier = Modifier
.size(18.dp)
.clickable { showApiKeyHelp = true },
tint = MaterialTheme.colorScheme.onSurfaceVariant
)
}
},
placeholder = { Text("Your API_SERVER_KEY value") },
singleLine = true,
visualTransformation = if (apiKeyVisible) {
VisualTransformation.None
} else {
PasswordVisualTransformation()
},
trailingIcon = {
IconButton(onClick = { apiKeyVisible = !apiKeyVisible }) {
Icon(
imageVector = if (apiKeyVisible) {
Icons.Filled.VisibilityOff
} else {
Icons.Filled.Visibility
},
contentDescription = if (apiKeyVisible) "Hide" else "Show"
)
}
},
modifier = Modifier.fillMaxWidth()
)
// "How do I set this up?" clickable text
Text(
text = "How do I set this up?",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.primary,
modifier = Modifier
.align(Alignment.Start)
.clickable { showApiKeyHelp = true }
.padding(top = 4.dp)
)
Spacer(modifier = Modifier.height(12.dp))
// Test Connection button + result
Row(
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp)
) {
OutlinedButton(
onClick = {
if (apiServerUrl.isNotBlank()) {
isTesting = true
testResult = null
coroutineScope.launch {
testResult = performHealthCheck(apiServerUrl, apiKey)
isTesting = false
}
}
},
enabled = apiServerUrl.isNotBlank() && !isTesting
) {
Text("Test Connection")
}
if (isTesting) {
CircularProgressIndicator(modifier = Modifier.size(20.dp), strokeWidth = 2.dp)
}
testResult?.let { result ->
when (result) {
is TestResult.Success -> {
ConnectionStatusBadge(
isConnected = true,
modifier = Modifier.size(14.dp),
size = 14.dp
)
Spacer(modifier = Modifier.width(4.dp))
Text(
text = "Connected",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.primary
)
}
is TestResult.Failure -> {
ConnectionStatusBadge(
isConnected = false,
modifier = Modifier.size(14.dp),
size = 14.dp
)
}
}
}
}
// Error message text below the row
testResult?.let { result ->
if (result is TestResult.Failure) {
Text(
text = result.message,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.error,
modifier = Modifier
.align(Alignment.Start)
.padding(top = 4.dp)
)
}
}
Spacer(modifier = Modifier.height(8.dp))
// QR code scanning option
OutlinedButton(
onClick = {
cameraPermissionLauncher.launch(android.Manifest.permission.CAMERA)
},
modifier = Modifier.fillMaxWidth()
) {
Icon(
imageVector = Icons.Filled.QrCodeScanner,
contentDescription = null,
modifier = Modifier.size(18.dp)
)
Spacer(modifier = Modifier.width(8.dp))
Text("Scan QR Code")
}
Text(
text = "Run hermes-pair on your server to generate a QR code",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.align(Alignment.Start)
)
}
// QR Scanner overlay
if (showQrScanner) {
QrPairingScanner(
onPairingDetected = { payload ->
onApiServerUrlChange(payload.serverUrl)
onApiKeyChange(payload.key)
showQrScanner = false
// Auto-trigger test
isTesting = true
testResult = null
coroutineScope.launch {
testResult = performHealthCheck(payload.serverUrl, payload.key)
isTesting = false
}
},
onDismiss = { showQrScanner = false }
ConnectionWizard(
connectionViewModel = connectionViewModel,
onComplete = onComplete,
onCancel = onSkip,
showSkip = true,
)
}
}
@Composable
private fun RelayPage(
relayUrl: String,
onRelayUrlChange: (String) -> Unit
) {
OnboardingPage(
icon = Icons.Outlined.Hub,
title = "Relay Server",
description = "Optional — for Bridge and Terminal features. You can set this up later in Settings."
) {
Spacer(modifier = Modifier.height(8.dp))
OutlinedTextField(
value = relayUrl,
onValueChange = onRelayUrlChange,
label = { Text("Relay URL (optional)") },
placeholder = { Text("wss://your-server:8767") },
singleLine = true,
modifier = Modifier.fillMaxWidth()
)
Spacer(modifier = Modifier.height(4.dp))
Text(
text = "Needed for Bridge and Terminal features",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
}
}
/**
* Performs a direct OkHttp health check against the Hermes API server.
* Returns a [TestResult] with either success or a descriptive error message.
*/
private suspend fun performHealthCheck(apiServerUrl: String, apiKey: String): TestResult =
withContext(Dispatchers.IO) {
val client = OkHttpClient.Builder()
.connectTimeout(10, TimeUnit.SECONDS)
.readTimeout(10, TimeUnit.SECONDS)
.build()
try {
val builder = Request.Builder()
.url("$apiServerUrl/health")
.get()
if (apiKey.isNotBlank()) {
builder.header("Authorization", "Bearer $apiKey")
}
val request = builder.build()
client.newCall(request).execute().use { response ->
when {
response.isSuccessful -> TestResult.Success
response.code == 401 -> TestResult.Failure("Unauthorized \u2014 check your API key")
else -> TestResult.Failure("Server returned HTTP ${response.code}")
}
}
} catch (e: SSLException) {
// User entered https:// but server runs plain HTTP, or vice versa
if (apiServerUrl.startsWith("https://", ignoreCase = true)) {
TestResult.Failure("TLS handshake failed \u2014 try http:// if your server doesn't use HTTPS")
} else {
TestResult.Failure("SSL error: ${e.message}")
}
} catch (e: ConnectException) {
TestResult.Failure("Connection refused \u2014 check the URL and port")
} catch (e: UnknownHostException) {
TestResult.Failure("Server not found \u2014 check the hostname")
} catch (e: SocketTimeoutException) {
TestResult.Failure("Connection timed out \u2014 is the server running?")
} catch (e: IOException) {
val msg = e.message ?: ""
when {
msg.contains("401") -> TestResult.Failure("Unauthorized \u2014 check your API key")
msg.contains("tls", ignoreCase = true) || msg.contains("ssl", ignoreCase = true) ->
TestResult.Failure("TLS error \u2014 try http:// if your server doesn't use HTTPS")
else -> TestResult.Failure("Connection failed: $msg")
}
} catch (e: Exception) {
TestResult.Failure("Connection failed: ${e.message}")
} finally {
client.dispatcher.executorService.shutdown()
client.connectionPool.evictAll()
}
}
@Preview(showSystemUi = true)
@Composable
private fun OnboardingScreenPreview() {
HermesRelayTheme {
// Preview uses whatever ViewModelStoreOwner Compose-Preview mocks;
// at preview time there's no NavHost so the scope collision that
// the production entry point has to avoid doesn't apply here.
OnboardingScreen(
onComplete = { _, _, _ -> }
connectionViewModel = viewModel(),
onComplete = {},
)
}
}
@@ -675,7 +430,8 @@ private fun OnboardingScreenPreview() {
private fun OnboardingScreenDarkPreview() {
HermesRelayTheme(themePreference = "dark") {
OnboardingScreen(
onComplete = { _, _, _ -> }
connectionViewModel = viewModel(),
onComplete = {},
)
}
}
@@ -0,0 +1,336 @@
package com.hermesandroid.relay.ui.screens
import android.content.Intent
import android.net.Uri
import android.widget.Toast
import androidx.compose.foundation.Image
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.isSystemInDarkTheme
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ArrowBack
import androidx.compose.material.icons.automirrored.outlined.MenuBook
import androidx.compose.material.icons.outlined.Code
import androidx.compose.material.icons.outlined.Shield
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.material3.TopAppBar
import androidx.compose.material3.TopAppBarDefaults
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.res.painterResource
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.R
import com.hermesandroid.relay.data.BuildFlavor
import com.hermesandroid.relay.data.FeatureFlags
import com.hermesandroid.relay.ui.components.WhatsNewDialog
import com.hermesandroid.relay.ui.theme.gradientBorder
import com.hermesandroid.relay.viewmodel.ConnectionViewModel
import kotlinx.coroutines.launch
/**
* Dedicated About screen. Hosts version info, build metadata, credits,
* license, GitHub + docs links, and the tap-7x version-number unlock for
* Developer options.
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun AboutScreen(
connectionViewModel: ConnectionViewModel,
onBack: () -> Unit,
onUnlockDeveloperOptions: () -> Unit = {},
) {
val context = LocalContext.current
val scope = rememberCoroutineScope()
val isDarkTheme = isSystemInDarkTheme()
// Tap-7x unlock state (mirrors the old SettingsScreen locals)
val devOptionsUnlocked by FeatureFlags.devOptionsUnlocked(context).collectAsState(initial = FeatureFlags.isDevBuild)
var versionTapCount by remember { mutableStateOf(0) }
var lastTapTime by remember { mutableStateOf(0L) }
var showWhatsNew by remember { mutableStateOf(false) }
Scaffold(
topBar = {
TopAppBar(
title = { Text("About") },
navigationIcon = {
IconButton(onClick = onBack) {
Icon(
imageVector = Icons.AutoMirrored.Filled.ArrowBack,
contentDescription = "Back",
)
}
},
colors = TopAppBarDefaults.topAppBarColors(
containerColor = MaterialTheme.colorScheme.surface,
),
)
},
) { innerPadding ->
Column(
modifier = Modifier
.fillMaxSize()
.padding(innerPadding)
.verticalScroll(rememberScrollState())
.padding(horizontal = 16.dp, vertical = 16.dp),
verticalArrangement = Arrangement.spacedBy(16.dp),
) {
// About section
Text(
text = "About",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.primary
)
Card(
modifier = Modifier
.fillMaxWidth()
.gradientBorder(
shape = RoundedCornerShape(12.dp),
isDarkTheme = isDarkTheme
),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(
modifier = Modifier.padding(16.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(8.dp)
) {
// Logo
Box(
modifier = Modifier
.size(80.dp)
.clip(RoundedCornerShape(20.dp))
.background(Color(0xFF1A1A2E)),
contentAlignment = Alignment.Center
) {
Image(
painter = painterResource(R.drawable.ic_launcher_foreground),
contentDescription = "Hermes-Relay",
modifier = Modifier.size(80.dp)
)
}
Text(
text = "Hermes-Relay",
style = MaterialTheme.typography.titleLarge,
textAlign = TextAlign.Center
)
Text(
text = "Native Android client for Hermes Agent",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = TextAlign.Center
)
Spacer(modifier = Modifier.height(4.dp))
// Version info (dynamic)
val versionName = remember {
try {
context.packageManager.getPackageInfo(context.packageName, 0).versionName ?: "—"
} catch (_: Exception) { "—" }
}
val versionCode = remember {
try {
@Suppress("DEPRECATION")
context.packageManager.getPackageInfo(context.packageName, 0).versionCode.toString()
} catch (_: Exception) { "—" }
}
Row(
modifier = Modifier
.fillMaxWidth()
.clickable {
if (devOptionsUnlocked) return@clickable
val now = System.currentTimeMillis()
if (now - lastTapTime > 2000) {
versionTapCount = 1
} else {
versionTapCount++
}
lastTapTime = now
val remaining = 7 - versionTapCount
when {
remaining <= 0 -> {
scope.launch { FeatureFlags.unlockDevOptions(context) }
Toast.makeText(context, "Developer options unlocked", Toast.LENGTH_SHORT).show()
versionTapCount = 0
onUnlockDeveloperOptions()
}
remaining <= 3 -> {
Toast.makeText(context, "$remaining taps to unlock developer options", Toast.LENGTH_SHORT).show()
}
}
},
horizontalArrangement = Arrangement.SpaceBetween
) {
Text(
text = "Version",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
Text(
text = "$versionName ($versionCode)",
style = MaterialTheme.typography.bodyMedium
)
}
// === PHASE3-flavor-split: build flavor badge ===
// Surfaces which release track the user is running. Tier 3/4/6
// Bridge surfaces differ between googlePlay and sideload, so it
// helps bug reports to see the active flavor right next to the
// version. Small, unobtrusive — shares the same row style as
// the Version label above.
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween
) {
Text(
text = "Track",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
Text(
text = BuildFlavor.displayName,
style = MaterialTheme.typography.bodyMedium
)
}
// === END PHASE3-flavor-split ===
HorizontalDivider()
// Links
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp)
) {
OutlinedButton(
onClick = {
val intent = Intent(Intent.ACTION_VIEW, Uri.parse("https://github.com/Codename-11/hermes-relay"))
context.startActivity(intent)
},
modifier = Modifier.weight(1f)
) {
Icon(
painter = painterResource(R.drawable.ic_github),
contentDescription = null,
modifier = Modifier.size(16.dp)
)
Spacer(modifier = Modifier.size(6.dp))
Text("GitHub")
}
OutlinedButton(
onClick = {
val intent = Intent(Intent.ACTION_VIEW, Uri.parse("https://codename-11.github.io/hermes-relay/"))
context.startActivity(intent)
},
modifier = Modifier.weight(1f)
) {
Icon(
imageVector = Icons.AutoMirrored.Outlined.MenuBook,
contentDescription = null,
modifier = Modifier.size(16.dp)
)
Spacer(modifier = Modifier.size(6.dp))
Text("App Docs")
}
}
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(8.dp)
) {
OutlinedButton(
onClick = {
val intent = Intent(Intent.ACTION_VIEW, Uri.parse("https://hermes-agent.nousresearch.com"))
context.startActivity(intent)
},
modifier = Modifier.weight(1f)
) {
Icon(
imageVector = Icons.Outlined.Code,
contentDescription = null,
modifier = Modifier.size(16.dp)
)
Spacer(modifier = Modifier.size(6.dp))
Text("Hermes Docs")
}
}
// Privacy policy link
OutlinedButton(
onClick = {
val intent = Intent(Intent.ACTION_VIEW, Uri.parse("https://github.com/Codename-11/hermes-relay/blob/main/docs/privacy.md"))
context.startActivity(intent)
},
modifier = Modifier.fillMaxWidth()
) {
Icon(
imageVector = Icons.Outlined.Shield,
contentDescription = null,
modifier = Modifier.size(16.dp)
)
Spacer(modifier = Modifier.size(6.dp))
Text("Privacy Policy")
}
// What's New
TextButton(onClick = { showWhatsNew = true }) {
Text("What's New in This Version")
}
// Credits
Text(
text = "Axiom Labs \u2764\uFE0F Hermes Agent · Nous Research",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant.copy(alpha = 0.6f),
textAlign = TextAlign.Center,
modifier = Modifier.fillMaxWidth()
)
}
}
}
}
// What's New dialog
if (showWhatsNew) {
WhatsNewDialog(onDismiss = { showWhatsNew = false })
}
}
@@ -0,0 +1,72 @@
package com.hermesandroid.relay.ui.screens
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ArrowBack
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Text
import androidx.compose.material3.TopAppBar
import androidx.compose.material3.TopAppBarDefaults
import androidx.compose.runtime.Composable
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.ui.components.StatsForNerds
import com.hermesandroid.relay.viewmodel.ConnectionViewModel
/**
* Dedicated Analytics screen — "Stats for Nerds". Hosts TTFT chart, tokens/
* message, peak times, stream rate, health metrics, and the reset button.
* Read-only display of in-app analytics from AppAnalytics.
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun AnalyticsScreen(
connectionViewModel: ConnectionViewModel,
onBack: () -> Unit,
) {
Scaffold(
topBar = {
TopAppBar(
title = { Text("Analytics") },
navigationIcon = {
IconButton(onClick = onBack) {
Icon(
imageVector = Icons.AutoMirrored.Filled.ArrowBack,
contentDescription = "Back",
)
}
},
colors = TopAppBarDefaults.topAppBarColors(
containerColor = MaterialTheme.colorScheme.surface,
),
)
},
) { innerPadding ->
Column(
modifier = Modifier
.fillMaxSize()
.padding(innerPadding)
.verticalScroll(rememberScrollState())
.padding(horizontal = 16.dp, vertical = 16.dp),
verticalArrangement = Arrangement.spacedBy(16.dp),
) {
// Stats for Nerds section
Text(
text = "Stats for Nerds",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.primary
)
StatsForNerds()
}
}
}
@@ -0,0 +1,258 @@
package com.hermesandroid.relay.ui.screens
import androidx.compose.foundation.isSystemInDarkTheme
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ArrowBack
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Scaffold
import androidx.compose.material3.SegmentedButton
import androidx.compose.material3.SegmentedButtonDefaults
import androidx.compose.material3.SingleChoiceSegmentedButtonRow
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.material3.TopAppBar
import androidx.compose.material3.TopAppBarDefaults
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.alpha
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.ui.theme.gradientBorder
import com.hermesandroid.relay.viewmodel.ConnectionViewModel
/**
* Dedicated Appearance settings screen. Hosts theme picker (auto/light/dark),
* font-scale preference, and animation toggles (sphere background, idle
* animation, etc.).
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun AppearanceSettingsScreen(
connectionViewModel: ConnectionViewModel,
onBack: () -> Unit,
) {
val theme by connectionViewModel.theme.collectAsState()
val isDarkTheme = isSystemInDarkTheme()
Scaffold(
topBar = {
TopAppBar(
title = { Text("Appearance") },
navigationIcon = {
IconButton(onClick = onBack) {
Icon(
imageVector = Icons.AutoMirrored.Filled.ArrowBack,
contentDescription = "Back",
)
}
},
colors = TopAppBarDefaults.topAppBarColors(
containerColor = MaterialTheme.colorScheme.surface,
),
)
},
) { innerPadding ->
Column(
modifier = Modifier
.fillMaxSize()
.padding(innerPadding)
.verticalScroll(rememberScrollState())
.padding(horizontal = 16.dp, vertical = 16.dp),
verticalArrangement = Arrangement.spacedBy(16.dp),
) {
// Theme section
Text(
text = "Appearance",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.primary
)
Card(
modifier = Modifier
.fillMaxWidth()
.gradientBorder(
shape = RoundedCornerShape(12.dp),
isDarkTheme = isDarkTheme
),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
Text(
text = "Theme",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
val themeOptions = listOf("auto", "light", "dark")
val themeLabels = listOf("Auto", "Light", "Dark")
val selectedIndex = themeOptions.indexOf(theme).coerceAtLeast(0)
SingleChoiceSegmentedButtonRow(modifier = Modifier.fillMaxWidth()) {
themeOptions.forEachIndexed { index, option ->
SegmentedButton(
shape = SegmentedButtonDefaults.itemShape(
index = index,
count = themeOptions.size
),
onClick = { connectionViewModel.setTheme(option) },
selected = index == selectedIndex
) {
Text(themeLabels[index])
}
}
}
// ── Font size ──────────────────────────────────────────
//
// Discrete stops applied globally via LocalDensity.fontScale
// at the Compose theme root, plus pushed to xterm via
// window.setFontSize through TerminalWebView.
val fontScale by connectionViewModel.fontScale.collectAsState()
val fontScaleOptions = listOf(0.85f, 1.0f, 1.15f, 1.3f)
val fontScaleLabels = listOf("Small", "Normal", "Large", "Larger")
// Match the closest stop — float equality is fragile.
val selectedFontScaleIndex = fontScaleOptions
.withIndex()
.minByOrNull { kotlin.math.abs(it.value - fontScale) }
?.index
?: 1
Text(
text = "Font size",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
SingleChoiceSegmentedButtonRow(modifier = Modifier.fillMaxWidth()) {
fontScaleOptions.forEachIndexed { index, option ->
SegmentedButton(
shape = SegmentedButtonDefaults.itemShape(
index = index,
count = fontScaleOptions.size
),
onClick = { connectionViewModel.setFontScale(option) },
selected = index == selectedFontScaleIndex
) {
Text(fontScaleLabels[index])
}
}
}
// Subtle preview at the chosen scale. We multiply the
// current bodyMedium fontSize by the selected stop so the
// preview reflects the user's choice immediately, even
// though everything else in the app already scales via
// LocalDensity once they tap a stop.
val previewBase = MaterialTheme.typography.bodyMedium
Text(
text = "Aa — sample text",
style = previewBase.copy(
fontSize = previewBase.fontSize * fontScaleOptions[selectedFontScaleIndex]
),
color = MaterialTheme.colorScheme.onSurfaceVariant
)
}
}
// Animation section
Text(
text = "Animation",
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.primary
)
Card(
modifier = Modifier
.fillMaxWidth()
.gradientBorder(
shape = RoundedCornerShape(12.dp),
isDarkTheme = isDarkTheme
),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
val animEnabled by connectionViewModel.animationEnabled.collectAsState()
val animBehindChat by connectionViewModel.animationBehindChat.collectAsState()
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
// Animation enabled toggle
Row(
modifier = Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically
) {
Column(modifier = Modifier.weight(1f)) {
Text(
text = "ASCII sphere",
style = MaterialTheme.typography.bodyMedium
)
Text(
text = "Show animated sphere on empty chat screen and ambient mode",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
}
Switch(
checked = animEnabled,
onCheckedChange = { connectionViewModel.setAnimationEnabled(it) }
)
}
HorizontalDivider()
// Behind chat toggle
Row(
modifier = Modifier
.fillMaxWidth()
.then(if (!animEnabled) Modifier.alpha(0.5f) else Modifier),
horizontalArrangement = Arrangement.SpaceBetween,
verticalAlignment = Alignment.CenterVertically
) {
Column(modifier = Modifier.weight(1f)) {
Text(
text = "Behind messages",
style = MaterialTheme.typography.bodyMedium
)
Text(
text = "Show subtle sphere animation behind chat messages",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
}
Switch(
checked = animBehindChat && animEnabled,
onCheckedChange = { connectionViewModel.setAnimationBehindChat(it) },
enabled = animEnabled
)
}
}
}
}
}
}
@@ -0,0 +1,435 @@
package com.hermesandroid.relay.ui.screens
import android.content.Context
import android.content.Intent
import android.net.Uri
import android.provider.Settings
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ArrowBack
import androidx.compose.material.icons.filled.Add
import androidx.compose.material.icons.filled.Close
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.Checkbox
import androidx.compose.material3.DividerDefaults
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.InputChip
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Slider
import androidx.compose.material3.Switch
import androidx.compose.material3.Text
import androidx.compose.material3.TopAppBar
import androidx.compose.material3.TopAppBarDefaults
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.compose.LocalLifecycleOwner
import com.hermesandroid.relay.data.BridgeSafetyPreferencesRepository
import com.hermesandroid.relay.data.BridgeSafetySettings
import com.hermesandroid.relay.data.DEFAULT_BLOCKLIST
import com.hermesandroid.relay.data.MAX_AUTO_DISABLE_MINUTES
import com.hermesandroid.relay.data.MAX_CONFIRMATION_TIMEOUT_SECONDS
import com.hermesandroid.relay.data.MIN_AUTO_DISABLE_MINUTES
import com.hermesandroid.relay.data.MIN_CONFIRMATION_TIMEOUT_SECONDS
import kotlinx.coroutines.launch
/**
* Phase 3 — safety-rails `bridge-safety-rails`
*
* Compose screen for Tier 5 safety configuration. Five sections:
*
* 1. Blocklist — searchable list of installed apps with a checkbox per
* row. Uses `PackageManager.getInstalledApplications` + a filter
* dropping system apps with no launcher intent so the list is a few
* dozen entries not five hundred.
* 2. Destructive verbs — input-chip list with an "Add" text field.
* 3. Auto-disable timer — slider 5..120 min.
* 4. Status overlay — toggle; if flipped on without SYSTEM_ALERT_WINDOW
* permission, tapping the switch kicks off the permission grant flow.
* 5. Confirmation timeout — slider 10..60s.
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun BridgeSafetySettingsScreen(onBack: () -> Unit) {
val context = LocalContext.current
val scope = rememberCoroutineScope()
val repo = remember { BridgeSafetyPreferencesRepository(context) }
val settings by repo.settings.collectAsState(initial = BridgeSafetySettings())
// Overlay-permission live check — recompute on resume so returning
// from Settings flips the switch's availability without nav churn.
var canDrawOverlays by remember {
mutableStateOf(Settings.canDrawOverlays(context))
}
val lifecycleOwner = LocalLifecycleOwner.current
LaunchedEffect(lifecycleOwner) {
val observer = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_RESUME) {
canDrawOverlays = Settings.canDrawOverlays(context)
}
}
lifecycleOwner.lifecycle.addObserver(observer)
}
// Cached installed-app list (expensive to call, so we do it once).
var installedApps by remember { mutableStateOf<List<InstalledApp>>(emptyList()) }
var appSearch by remember { mutableStateOf("") }
LaunchedEffect(Unit) {
installedApps = loadInstalledApps(context)
}
Scaffold(
topBar = {
TopAppBar(
title = { Text("Bridge safety") },
navigationIcon = {
IconButton(onClick = onBack) {
Icon(
imageVector = Icons.AutoMirrored.Filled.ArrowBack,
contentDescription = "Back"
)
}
},
colors = TopAppBarDefaults.topAppBarColors(
containerColor = MaterialTheme.colorScheme.surface
)
)
}
) { inner ->
Column(
modifier = Modifier
.fillMaxSize()
.padding(inner)
.verticalScroll(rememberScrollState())
.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(16.dp),
) {
// ── Blocklist ───────────────────────────────────────────────
SectionCard(title = "Blocked apps") {
Text(
text = "The agent cannot act on any app in this list. Defaults ship with banking apps and password managers.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.size(8.dp))
OutlinedTextField(
value = appSearch,
onValueChange = { appSearch = it },
modifier = Modifier.fillMaxWidth(),
singleLine = true,
label = { Text("Search apps") },
)
Spacer(Modifier.size(8.dp))
val filtered = remember(installedApps, appSearch, settings.blocklist) {
val term = appSearch.trim().lowercase()
val sorted = installedApps.sortedWith(
compareByDescending<InstalledApp> { it.packageName in settings.blocklist }
.thenBy { it.label.lowercase() }
)
if (term.isEmpty()) sorted
else sorted.filter {
it.label.lowercase().contains(term) ||
it.packageName.lowercase().contains(term)
}
}
LazyColumn(
modifier = Modifier
.fillMaxWidth()
.heightIn(max = 340.dp),
verticalArrangement = Arrangement.spacedBy(4.dp),
) {
items(filtered, key = { it.packageName }) { app ->
val checked = app.packageName in settings.blocklist
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
) {
Checkbox(
checked = checked,
onCheckedChange = { nowChecked ->
scope.launch {
if (nowChecked) repo.addToBlocklist(app.packageName)
else repo.removeFromBlocklist(app.packageName)
}
},
)
Column(modifier = Modifier.weight(1f)) {
Text(
text = app.label,
style = MaterialTheme.typography.bodyMedium,
)
Text(
text = app.packageName,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
}
}
}
// ── Destructive verbs ───────────────────────────────────────
SectionCard(title = "Destructive verbs") {
Text(
text = "Tap_text / type commands whose text contains these words (word-boundary match) trigger a confirmation modal.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.size(8.dp))
var newVerb by remember { mutableStateOf("") }
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
) {
OutlinedTextField(
value = newVerb,
onValueChange = { newVerb = it },
modifier = Modifier.weight(1f),
singleLine = true,
label = { Text("Add a verb") },
)
IconButton(onClick = {
val v = newVerb.trim()
if (v.isNotEmpty()) {
scope.launch { repo.addDestructiveVerb(v) }
newVerb = ""
}
}) {
Icon(Icons.Filled.Add, contentDescription = "Add verb")
}
}
Spacer(Modifier.size(6.dp))
Column(verticalArrangement = Arrangement.spacedBy(4.dp)) {
settings.destructiveVerbs.sorted().chunked(3).forEach { row ->
Row(
horizontalArrangement = Arrangement.spacedBy(6.dp),
) {
row.forEach { verb ->
InputChip(
selected = false,
onClick = {},
label = { Text(verb) },
trailingIcon = {
IconButton(
onClick = {
scope.launch { repo.removeDestructiveVerb(verb) }
},
modifier = Modifier.size(18.dp),
) {
Icon(
Icons.Filled.Close,
contentDescription = "Remove",
modifier = Modifier.size(14.dp),
)
}
},
)
}
}
}
}
}
// ── Auto-disable timer ──────────────────────────────────────
SectionCard(title = "Auto-disable after idle") {
Text(
text = "Master toggle flips off after this many minutes with no bridge commands. Rescheduled on every command.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.size(4.dp))
Text(
text = "${settings.autoDisableMinutes} min",
style = MaterialTheme.typography.titleMedium,
fontWeight = FontWeight.SemiBold,
)
Slider(
value = settings.autoDisableMinutes.toFloat(),
onValueChange = { v ->
scope.launch { repo.setAutoDisableMinutes(v.toInt()) }
},
valueRange = MIN_AUTO_DISABLE_MINUTES.toFloat()..MAX_AUTO_DISABLE_MINUTES.toFloat(),
steps = (MAX_AUTO_DISABLE_MINUTES - MIN_AUTO_DISABLE_MINUTES) / 5 - 1,
)
}
// ── Status overlay ──────────────────────────────────────────
SectionCard(title = "Status overlay") {
Row(
verticalAlignment = Alignment.CenterVertically,
modifier = Modifier.fillMaxWidth(),
) {
Column(modifier = Modifier.weight(1f)) {
Text(
text = "Show floating indicator",
style = MaterialTheme.typography.bodyLarge,
)
Text(
text = if (canDrawOverlays) {
"Permission granted."
} else {
"Tap to grant overlay permission."
},
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
Switch(
checked = settings.statusOverlayEnabled && canDrawOverlays,
onCheckedChange = { wanted ->
if (wanted && !canDrawOverlays) {
// Kick off the SYSTEM_ALERT_WINDOW grant flow.
runCatching {
val intent = Intent(
Settings.ACTION_MANAGE_OVERLAY_PERMISSION,
Uri.parse("package:${context.packageName}")
).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
context.startActivity(intent)
}
} else {
scope.launch { repo.setStatusOverlayEnabled(wanted) }
}
}
)
}
}
// ── Confirmation timeout ────────────────────────────────────
SectionCard(title = "Confirmation timeout") {
Text(
text = "Destructive-verb modal waits this long for your response before defaulting to Deny.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.size(4.dp))
Text(
text = "${settings.confirmationTimeoutSeconds} s",
style = MaterialTheme.typography.titleMedium,
fontWeight = FontWeight.SemiBold,
)
Slider(
value = settings.confirmationTimeoutSeconds.toFloat(),
onValueChange = { v ->
scope.launch { repo.setConfirmationTimeoutSeconds(v.toInt()) }
},
valueRange = MIN_CONFIRMATION_TIMEOUT_SECONDS.toFloat()..MAX_CONFIRMATION_TIMEOUT_SECONDS.toFloat(),
steps = (MAX_CONFIRMATION_TIMEOUT_SECONDS - MIN_CONFIRMATION_TIMEOUT_SECONDS) / 5 - 1,
)
}
Spacer(modifier = Modifier.size(24.dp))
}
}
}
@Composable
private fun SectionCard(title: String, content: @Composable () -> Unit) {
Card(
modifier = Modifier.fillMaxWidth(),
shape = RoundedCornerShape(14.dp),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.surfaceVariant
)
) {
Column(modifier = Modifier.padding(16.dp)) {
Text(
text = title,
style = MaterialTheme.typography.titleMedium,
fontWeight = FontWeight.SemiBold,
color = MaterialTheme.colorScheme.primary,
)
HorizontalDivider(
modifier = Modifier.padding(vertical = 8.dp),
thickness = DividerDefaults.Thickness,
color = MaterialTheme.colorScheme.outline.copy(alpha = 0.2f),
)
content()
}
}
}
private data class InstalledApp(
val packageName: String,
val label: String,
)
/**
* Enumerate installed apps that have a launcher intent + all apps in the
* default blocklist (even if they're not installed — so users still see
* which entries the defaults cover). Deduped by package name.
*/
private fun loadInstalledApps(context: Context): List<InstalledApp> {
val pm = context.packageManager
val launcher = Intent(Intent.ACTION_MAIN).apply { addCategory(Intent.CATEGORY_LAUNCHER) }
val launcherable = runCatching { pm.queryIntentActivities(launcher, 0) }
.getOrDefault(emptyList())
.mapNotNull { resolve ->
val info = resolve.activityInfo?.applicationInfo ?: return@mapNotNull null
val label = runCatching { pm.getApplicationLabel(info).toString() }
.getOrDefault(info.packageName ?: return@mapNotNull null)
InstalledApp(packageName = info.packageName, label = label)
}
val defaults = DEFAULT_BLOCKLIST.map { pkg ->
val label = runCatching {
val ai = pm.getApplicationInfo(pkg, 0)
pm.getApplicationLabel(ai).toString()
}.getOrDefault(pkg.substringAfterLast('.'))
InstalledApp(packageName = pkg, label = label)
}
return (launcherable + defaults)
.distinctBy { it.packageName }
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeSafetySettingsScreenPreview() {
MaterialTheme {
// The real screen needs a Context for PackageManager lookups, so
// this preview just shows one section to verify styling.
Column(Modifier.padding(16.dp)) {
SectionCard(title = "Destructive verbs") {
Text(
text = "Tap_text / type commands whose text contains these words trigger a confirmation modal.",
style = MaterialTheme.typography.bodySmall,
)
}
}
}
}
@@ -1,129 +1,313 @@
package com.hermesandroid.relay.ui.screens
import android.content.Intent
import android.net.Uri
import android.provider.Settings
import com.hermesandroid.relay.data.BuildFlavor
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.aspectRatio
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Schedule
import androidx.compose.material3.AssistChip
import androidx.compose.material.icons.filled.Warning
import androidx.compose.material3.Card
import androidx.compose.material3.CardDefaults
import androidx.compose.material3.ExperimentalMaterial3Api
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Text
import androidx.compose.material3.TopAppBar
import androidx.compose.material3.TopAppBarDefaults
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.tooling.preview.Preview
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.ui.components.MorphingSphere
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleEventObserver
import androidx.lifecycle.compose.LocalLifecycleOwner
import androidx.lifecycle.viewmodel.compose.viewModel
// === PHASE3-safety-rails: safety summary card ===
import com.hermesandroid.relay.bridge.BridgeSafetyManager
import com.hermesandroid.relay.data.BridgeSafetySettings
import com.hermesandroid.relay.ui.components.BridgeSafetySummaryCard
// === END PHASE3-safety-rails ===
import com.hermesandroid.relay.ui.LocalSnackbarHost
import com.hermesandroid.relay.ui.components.BridgeActivityLog
import com.hermesandroid.relay.ui.components.BridgeMasterToggle
import com.hermesandroid.relay.ui.components.BridgePermissionChecklist
import com.hermesandroid.relay.ui.components.BridgeStatusCard
import com.hermesandroid.relay.viewmodel.BridgeViewModel
/**
* Bridge tab — phase 3 Wave 1 rewrite (Agent bridge-ui, `bridge-screen-ui`).
*
* Replaces the Phase 0 "Coming Soon" placeholder with the real control
* surface described in `Plans/Phase 3 — Bridge Channel.md` §5. Four stacked
* cards in a verticalScroll column:
*
* 1. [BridgeMasterToggle] — "Allow Agent Control" + live status
* 2. [BridgePermissionChecklist] — accessibility / capture / overlay / notif
* 3. [BridgeActivityLog] — scrollable recent-command history
* 4. Safety placeholder — stub owned by Agent safety-rails in Wave 2
*
* State comes from [BridgeViewModel] which in turn reads from the
* [com.hermesandroid.relay.data.BridgePreferencesRepository] DataStore for
* anything persistent, and stubs the live bridge-runtime state until
* Agent accessibility's `HermesAccessibilityService` exposes it. See [BridgeViewModel]'s
* KDoc for the exact accessibility-handoff surface.
*
* Lifecycle: we re-probe permission status on every ON_RESUME so that
* returning from Android Settings immediately flips the accessibility
* checklist row from red to green without needing to navigate away and back.
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun BridgeScreen() {
Column(modifier = Modifier.fillMaxSize()) {
TopAppBar(
title = { Text("Bridge") },
colors = TopAppBarDefaults.topAppBarColors(
containerColor = MaterialTheme.colorScheme.surface
)
)
fun BridgeScreen(
viewModel: BridgeViewModel = viewModel(),
// === PHASE3-safety-rails: safety summary card ===
onNavigateToBridgeSafety: () -> Unit = {},
// === END PHASE3-safety-rails ===
) {
val masterToggle by viewModel.masterToggle.collectAsState()
val permissionStatus by viewModel.permissionStatus.collectAsState()
val bridgeStatus by viewModel.bridgeStatus.collectAsState()
val activityLog by viewModel.activityLog.collectAsState()
Box(
// === PHASE3-safety-rails-followup: surface permission Test results via snackbar ===
val snackbarHost = LocalSnackbarHost.current
LaunchedEffect(viewModel) {
viewModel.testEvents.collect { message ->
snackbarHost.showSnackbar(message)
}
}
// === END PHASE3-safety-rails-followup ===
val context = LocalContext.current
// === PHASE3-safety-rails: safety summary card ===
// Pulls live safety settings + countdown off the process-wide
// BridgeSafetyManager singleton. No ViewModel wiring required — the
// manager is installed by ConnectionViewModel at app start.
val safetyManager = BridgeSafetyManager.peek()
val safetySettings by (safetyManager?.settings
?: remember { kotlinx.coroutines.flow.MutableStateFlow(BridgeSafetySettings()) })
.collectAsState()
val autoDisableAtMs by (safetyManager?.autoDisableAtMs
?: remember { kotlinx.coroutines.flow.MutableStateFlow<Long?>(null) })
.collectAsState()
// === END PHASE3-safety-rails ===
// Re-run permission + system-status probes whenever the screen resumes.
val lifecycleOwner = LocalLifecycleOwner.current
DisposableEffect(lifecycleOwner) {
val observer = LifecycleEventObserver { _, event ->
if (event == Lifecycle.Event.ON_RESUME) {
viewModel.onScreenResumed()
}
}
lifecycleOwner.lifecycle.addObserver(observer)
onDispose { lifecycleOwner.lifecycle.removeObserver(observer) }
}
Scaffold(
topBar = {
TopAppBar(
title = { Text("Bridge") },
colors = TopAppBarDefaults.topAppBarColors(
containerColor = MaterialTheme.colorScheme.surface
)
)
}
) { innerPadding ->
Column(
modifier = Modifier
.fillMaxSize()
.padding(horizontal = 32.dp),
contentAlignment = Alignment.Center
.padding(innerPadding)
.verticalScroll(rememberScrollState())
.padding(horizontal = 16.dp, vertical = 16.dp),
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
Column(
modifier = Modifier.widthIn(max = 300.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.Center
// === PHASE3-safety-rails-followup: overlay-permission nag banner ===
// Bridge is enabled but the user hasn't granted Display Over Other
// Apps. Without that permission, the destructive-verb confirmation
// modal can't render and BridgeSafetyManager.awaitConfirmation
// fails closed (denies the action) — silently from the user's
// perspective. Show a prominent banner so the failure isn't
// mysterious. Sideload only — googlePlay has no destructive-verb
// modal (action routes are blocked) so the overlay permission isn't
// needed and the nag would confuse users + reviewers.
if (BuildFlavor.isSideload &&
masterToggle &&
!permissionStatus.overlayPermitted
) {
// Sphere replaces static icon
Box(
modifier = Modifier
.fillMaxWidth()
.aspectRatio(1.2f)
) {
MorphingSphere(modifier = Modifier.fillMaxSize())
}
Spacer(modifier = Modifier.height(16.dp))
Text(
text = "Device Bridge",
style = MaterialTheme.typography.headlineSmall,
color = MaterialTheme.colorScheme.onSurface
)
Spacer(modifier = Modifier.height(8.dp))
Text(
text = "Let your Hermes agent interact with your phone \u2014 " +
"tap, type, read the screen, and automate workflows.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = TextAlign.Center
)
Spacer(modifier = Modifier.height(16.dp))
AssistChip(
onClick = { },
label = {
Text(
text = "Coming Soon",
style = MaterialTheme.typography.labelSmall
)
OverlayPermissionNagCard(
onTap = {
runCatching {
val intent = Intent(
Settings.ACTION_MANAGE_OVERLAY_PERMISSION,
Uri.parse("package:${context.packageName}"),
).apply { addFlags(Intent.FLAG_ACTIVITY_NEW_TASK) }
context.startActivity(intent)
}
},
leadingIcon = {
Icon(
imageVector = Icons.Filled.Schedule,
contentDescription = null,
modifier = Modifier.size(16.dp)
)
}
)
}
// === END PHASE3-safety-rails-followup ===
Spacer(modifier = Modifier.height(20.dp))
BridgeMasterToggle(
enabled = masterToggle,
status = bridgeStatus,
accessibilityGranted = permissionStatus.accessibilityServiceEnabled,
onToggle = { viewModel.setMasterEnabled(it) },
// googlePlay: the toggle is "Bridge Mode" (read-only screen
// reading). Sideload: "Agent Control" (full phone control).
// The label shapes user expectation + is what reviewers read.
label = if (BuildFlavor.isSideload)
"Allow Agent Control"
else
"Enable Bridge Mode",
)
val plannedFeatures = listOf(
"Agent-controlled device interaction",
"Accessibility service integration",
"Activity log and command history",
"Permission management"
BridgeStatusCard(
status = bridgeStatus,
// TODO(accessibility-handoff): once accessibility exposes a `bridgeConnected`
// StateFlow from HermesAccessibilityService, drive this off
// that instead of the a11y-granted flag.
isConnected = permissionStatus.accessibilityServiceEnabled && masterToggle,
)
BridgePermissionChecklist(
status = permissionStatus,
// === PHASE3-safety-rails-followup: in-app permission Test handlers ===
onTestAccessibility = { viewModel.testAccessibilityService() },
onTestScreenCapture = { viewModel.testScreenCapture() },
onTestOverlay = { viewModel.testOverlayPermission() },
// === END PHASE3-safety-rails-followup ===
// === PHASE3-bridge-ui-followup: extended interactions ===
onRequestScreenCapture = { viewModel.requestScreenCapture() },
onTestNotificationListener = { viewModel.testNotificationListener() },
// === END PHASE3-bridge-ui-followup ===
)
BridgeActivityLog(
entries = activityLog,
onClear = { viewModel.clearActivityLog() }
)
// === PHASE3-safety-rails: safety summary card ===
// Sideload only — googlePlay has no action routes so safety
// settings (destructive verbs, blocklist, auto-disable) are
// irrelevant. Showing them would confuse users and imply
// capabilities the Play APK doesn't have.
if (BuildFlavor.isSideload) {
BridgeSafetySummaryCard(
settings = safetySettings,
autoDisableAtMs = autoDisableAtMs,
onManage = onNavigateToBridgeSafety,
)
}
// === END PHASE3-safety-rails ===
Spacer(modifier = Modifier.height(16.dp))
}
}
}
/**
* Shown at the top of [BridgeScreen] when the master toggle is on but the
* user hasn't granted SYSTEM_ALERT_WINDOW. Without that permission,
* `BridgeSafetyManager.awaitConfirmation` fails closed and destructive
* actions get silently denied — the user has no way to know unless we
* tell them. Tap to open the overlay-permission Settings page.
*
* Phase 3 / safety-rails followup. Goes away on its own once
* `Settings.canDrawOverlays(context)` flips true.
*/
@Composable
private fun OverlayPermissionNagCard(onTap: () -> Unit) {
Card(
modifier = Modifier
.fillMaxWidth()
.clickable(onClick = onTap),
shape = RoundedCornerShape(14.dp),
colors = CardDefaults.cardColors(
containerColor = MaterialTheme.colorScheme.errorContainer,
),
) {
Row(
modifier = Modifier.padding(16.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
Icon(
imageVector = Icons.Filled.Warning,
contentDescription = null,
tint = MaterialTheme.colorScheme.onErrorContainer,
modifier = Modifier.size(28.dp),
)
Column(modifier = Modifier.weight(1f)) {
Text(
text = "Grant 'Display over other apps'",
style = MaterialTheme.typography.titleSmall,
fontWeight = FontWeight.SemiBold,
color = MaterialTheme.colorScheme.onErrorContainer,
)
Text(
text = "Without this, confirmation prompts can't show when " +
"the agent acts. Destructive actions will be silently denied. " +
"Tap to grant.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onErrorContainer,
)
Column(
verticalArrangement = Arrangement.spacedBy(4.dp)
) {
plannedFeatures.forEach { feature ->
Text(
text = "\u2022 $feature",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant
)
}
}
}
}
}
}
@Preview(showBackground = true)
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeScreenPreview() {
private fun OverlayPermissionNagCardPreview() {
MaterialTheme {
BridgeScreen()
Column(modifier = Modifier.padding(16.dp)) {
OverlayPermissionNagCard(onTap = {})
}
}
}
@Preview(showBackground = true, backgroundColor = 0xFF1A1A2E)
@Composable
private fun BridgeScreenPreview() {
// Previews can't construct a real AndroidViewModel without an Application
// context — we just render the summary card on its own to verify spacing.
// Individual components have their own full previews.
MaterialTheme {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(12.dp)
) {
BridgeSafetySummaryCard(
settings = BridgeSafetySettings(),
autoDisableAtMs = null,
onManage = {},
)
}
}
}

Some files were not shown because too many files have changed in this diff Show More