Compare commits

..
Author SHA1 Message Date
Bailey Dixon c932adbc7b release(desktop): desktop-v0.4.0-beta.7 2026-09-02 21:34:28 -04:00
Bailey Dixon 23cec497ce release(android): android-v1.15.1 2026-09-02 21:34:28 -04:00
Bailey Dixon 1739dc36a2 Merge pull request #538 from Codename-11/fix/android-voice-error-dialog
fix(android): polish voice errors and steer/queue controls
2026-09-02 21:21:05 -04:00
Bailey Dixon c917a94fcb fix(android): polish voice errors and steer/queue controls 2026-09-02 21:09:32 -04:00
Bailey Dixon e1a72ee8af Merge pull request #537 from Codename-11/fix/android-session-refresh-oom
fix(android): harden chat sessions and media rendering
2026-09-02 13:49:02 -04:00
Bailey Dixon 94f9ba4305 fix(android): harden chat sessions and media rendering 2026-09-02 13:34:14 -04:00
Bailey Dixon 3186afdde8 Merge pull request #536 from Codename-11/fix/android-attachment-viewer-rotation-salvage
fix(android): preserve attachment previews across rotation
2026-09-02 06:59:34 -04:00
Bailey Dixon 0261a342cf merge: refresh attachment viewer salvage from dev
# Conflicts:
#	CHANGELOG.md
2026-09-01 22:16:20 -04:00
Bailey DixonandYuzhe Wang f4c212f800 fix(android): preserve attachment previews across rotation
Hoist full-screen media viewers above transcript rows so responsive portrait/landscape reflow cannot dismiss an active image, attachment, or gallery preview. Reset the host when its connection, session, or policy owner changes.

Render video through Media3's fitted Compose content frame, retain playback position and controls across player recreation, and respect the user's system rotation preference.

Add focused state-restoration and viewer-host coverage for attachments, images, galleries, and owner changes.

Co-authored-by: Yuzhe Wang <o_xkenshin@icloud.com>
2026-09-01 22:15:56 -04:00
Bailey Dixon 3bb294c6c3 Merge pull request #535 from Codename-11/fix/android-sherpa-r8-jni
fix(android): preserve sherpa JNI names through R8
2026-09-01 22:03:15 -04:00
Bailey Dixon dc73cc74b1 Merge remote-tracking branch 'origin/dev' into fix/android-sherpa-r8-jni 2026-09-01 21:52:27 -04:00
Bailey Dixon 607bc95b73 Merge chore/repository-organization-20260901 into integration/repository-organization-20260901 2026-09-01 21:47:50 -04:00
Bailey Dixon 3b4da0305e docs: organize project records 2026-09-01 21:37:16 -04:00
Bailey Dixon 3af0092b8f Merge remote-tracking branch 'origin/dev' into fix/android-sherpa-r8-jni
# Conflicts:
#	CHANGELOG.md
2026-09-01 21:14:38 -04:00
Bailey DixonandMattyB01 9cfa4c8b8c fix(android): preserve sherpa JNI names through R8
Co-authored-by: MattyB01 <141138642+MattyB01@users.noreply.github.com>
2026-09-01 21:14:15 -04:00
Bailey Dixon a7c860a2e3 Merge pull request #532 from Codename-11/fix/android-stream-dashboard-media
fix(android): stream dashboard media to disk
2026-09-01 21:02:39 -04:00
Bailey Dixon ed71b02b01 Merge remote-tracking branch 'origin/dev' into fix/android-stream-dashboard-media
# Conflicts:
#	CHANGELOG.md
2026-09-01 19:25:15 -04:00
Bailey Dixon a90dc85466 Merge pull request #533 from Codename-11/fix/android-gateway-owner-rejection
fix(android): wait for gateway session readiness
2026-09-01 19:08:41 -04:00
Bailey Dixon 67a60143cb fix(android): wait for gateway session readiness 2026-09-01 18:50:46 -04:00
Bailey Dixon c3276fa7e5 chore: quarantine Quest experiment 2026-09-01 18:17:22 -04:00
Bailey Dixon 99ac20de35 fix(android): stream dashboard media to disk 2026-09-01 13:06:15 -04:00
Bailey Dixon 0650102230 docs: organize localized readme entrypoints 2026-09-01 12:19:03 -04:00
Bailey Dixon 8d4f324717 feat(release): promote exact-tree artifacts and CI evidence (#530) 2026-09-01 11:00:43 -04:00
180 changed files with 3799 additions and 872 deletions
+1 -1
View File
@@ -14,7 +14,7 @@ function classifyCiPaths(paths) {
const under = (prefixes) => paths.some((path) => prefixes.some((prefix) => path.startsWith(prefix)));
return {
android: forceAll || under(['app/', 'relay-core/', 'relay-ui/', 'ui-preview/', 'quest/', 'gradle/']) || exact([
android: forceAll || under(['app/', 'gradle/']) || exact([
'build.gradle.kts', 'settings.gradle.kts', 'gradle.properties', 'gradlew', 'gradlew.bat',
'scripts/check-android-locales.py', 'scripts/android-locale-harness.py',
'scripts/check-android-collection-apis.py', 'scripts/check-android-native-compat.py',
+1 -1
View File
@@ -16,7 +16,7 @@ const none = {
assert.deepEqual(classifyCiPaths(['README.md']), none);
assert.deepEqual(classifyCiPaths(['desktop/src/cli.ts']), { ...none, desktop: true });
assert.deepEqual(classifyCiPaths(['relay-core/src/main/kotlin/Wire.kt']), { ...none, android: true });
assert.deepEqual(classifyCiPaths(['experiments/quest/src/main/kotlin/Quest.kt']), none);
assert.deepEqual(classifyCiPaths(['scripts/check-android-release-notes.py']), { ...none, android: true });
assert.deepEqual(classifyCiPaths(['scripts/check-android-native-compat.py']), { ...none, android: true });
assert.deepEqual(classifyCiPaths(['scripts/android_release_artifacts.py']), { ...none, android: true });
-4
View File
@@ -20,10 +20,6 @@ on:
branches: [main, dev]
paths:
- "app/**"
- "relay-core/**"
- "relay-ui/**"
- "ui-preview/**"
- "quest/**"
- "gradle/**"
- "build.gradle.kts"
- "settings.gradle.kts"
+1 -4
View File
@@ -24,10 +24,7 @@ Thumbs.db
local.properties
/build/
/app/build/
/relay-core/build/
/relay-ui/build/
/ui-preview/build/
/quest/build/
/experiments/quest/**/build/
/app/release/
*.apk
*.aab
+1 -1
View File
@@ -17,7 +17,7 @@ contract here and in `RELEASE.md`.
- Android local/cloud verification → **[docs/android-build-lane.md](docs/android-build-lane.md)**
- Android emulator lanes → **[docs/android-emulator-testing.md](docs/android-emulator-testing.md)** — suggest the smallest relevant API 36 lanes; never run the full matrix automatically
- `android_*` toolset + MCP → **[docs/mcp-tooling.md](docs/mcp-tooling.md)**
- Follow-ups / deferred work / known gaps → **[TODO.md](TODO.md)** (the single home for "what's next" — never DEVLOG, never scattered code comments)
- Follow-ups / deferred work / known gaps → **[docs/project/TODO.md](docs/project/TODO.md)** (the single home for "what's next" — never DEVLOG, never scattered code comments)
## Branch contract
+20 -4
View File
@@ -6,13 +6,29 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
## [Unreleased]
## [Android 1.15.1] - 2026-09-02
### Changed
- **Android Chat and Voice use tablet space intentionally.** Expanded layouts keep introductions, transcripts, composer controls, and status chrome on readable centered rails, while landscape Voice Focus separates identity controls from conversation activity without changing phone or portrait interaction behavior.
- Chat and Bot Chat offer a compact Correct now / Queue next tray behind the composer. Chat settings sets the default; each message can override it. Stop pauses pending work until Resume, and editing or removing queued messages preserves the remaining order.
- Wider Chat and Voice layouts keep text and controls centered and readable, including landscape Voice Focus.
### Fixed
- **Windows desktop updates keep the installed CLI and UI on one release.** `hermes-relay update` now detects a colocated management UI, reports both installed versions, and uses the verified bundle installer to replace and restart the affected surfaces together. Explicit CLI-only installations retain the standalone binary updater.
- Delivery and correction labels remain readable inside user-message bubbles.
- Voice errors use a scrollable dialog with separate Retry and Dismiss actions.
- Attachment previews stay open through rotation, and videos retain their original proportions. (#483)
- Release builds preserve the native configuration names required for wake-word startup. (#444)
- Standard Hermes attachments stream to disk while enforcing download size limits. (#531)
- Session refresh no longer sustains a request loop. History loads, chat rendering, image previews, and media exports keep memory use bounded.
- Image-generation progress remains visible between interim replies and media delivery.
- New Gateway chats wait for session readiness before the first prompt; ownership refusals preserve the retryable prompt and server error.
## [0.4.0-beta.7] - 2026-09-02
### Fixed
- Windows updates detect a colocated management UI, report both installed versions, and update the CLI and UI together through the verified bundle installer. CLI-only installations keep their standalone updater.
## [Android 1.15.0] - 2026-08-31
@@ -1279,7 +1295,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
- **Voice-exit chime firing on every Add-connection tap.** `ConnectionSwitchCoordinator.switchConnection` fires the `voiceStopCallback` unconditionally at step 3 (correct for connection-to-connection switches while voice is active), but `beginAddConnection` also routes through `switchConnection` to bind the placeholder Connection's auth store before the pair wizard runs — and `VoiceViewModel.exitVoiceMode()` was playing `sfxPlayer.playExit()` regardless of whether voice mode was actually on. Logcat confirmed the chime on every Add-connection FAB tap. Fix adds an idempotence guard at the top of `exitVoiceMode()`: early-return when `_uiState.value.voiceMode` is already false. Teardown is still safe to skip because every inner statement is null-guarded + try/catch-wrapped and would be a no-op on an already-stopped voice session; the only meaningful line is the `playExit()` SFX, which is what we're silencing.
- **500 ms freeze on every Add-connection tap.** `ConnectionSwitchCoordinator.switchConnection` runs a `withTimeoutOrNull(AUTH_HYDRATE_TIMEOUT_MS = 500L)` block at step 10 to wait for the freshly-bound `AuthManager` to flip `AuthState` from `Loading` to `Paired`. The comment acknowledged Add-connection is the common path and the 500 ms was meant to be "imperceptible," but on-device it wasn't — the user perceived the delay (and the voice chime masking it) on every tap. The placeholder Connection created by `beginAddConnection` has `pairedAt == null` and an empty EncryptedSharedPreferences store, so `AuthState` will NEVER reach `Paired` — the 500 ms is pure stall. Fix short-circuits the hydrate wait when `target.pairedAt == null`: skip `withTimeoutOrNull` entirely for placeholders and log at DEBUG instead of the misleading "auth hydrate timeout" INFO. Real paired-to-paired switches still run the full hydrate wait because both sides have `pairedAt != null`.
- **KDoc nested-comment trap in `ConnectionViewModel.relayReady` doc block.** A literal `/voice/*` path pattern inside the `relayReady` KDoc opened a nested block comment (Kotlin supports nested `/* */`, Java does not) whose `*/` then closed only the nested level — leaving the outer `/**` open for the remaining ~2200 lines of the file. Symptom: `MainActivity.kt:67` "Unresolved reference 'isReady'" plus ~50 cascading "Cannot infer type" errors across `PairedDevicesScreen`, `SettingsScreen`, `TerminalScreen`. Real errors (`Missing '}`, `Unclosed comment`) were the last two lines of `./gradlew compileGooglePlayDebugKotlin` output, easy to miss. Fix was a two-character rewrite: path patterns now wrapped in backticks AND `/*` → `/...` so the glob-looking character isn't in a block-comment position. Lesson logged in `DEVLOG.md` 2026-04-21; worth a sweep of other KDoc blocks for shell/regex-looking patterns before the next large diff.
- **KDoc nested-comment trap in `ConnectionViewModel.relayReady` doc block.** A literal `/voice/*` path pattern inside the `relayReady` KDoc opened a nested block comment (Kotlin supports nested `/* */`, Java does not) whose `*/` then closed only the nested level — leaving the outer `/**` open for the remaining ~2200 lines of the file. Symptom: `MainActivity.kt:67` "Unresolved reference 'isReady'" plus ~50 cascading "Cannot infer type" errors across `PairedDevicesScreen`, `SettingsScreen`, `TerminalScreen`. Real errors (`Missing '}`, `Unclosed comment`) were the last two lines of `./gradlew compileGooglePlayDebugKotlin` output, easy to miss. Fix was a two-character rewrite: path patterns now wrapped in backticks AND `/*` → `/...` so the glob-looking character isn't in a block-comment position. Lesson logged in `docs/project/DEVLOG.md` 2026-04-21; worth a sweep of other KDoc blocks for shell/regex-looking patterns before the next large diff.
- **Orphan placeholder connections from abandoned Add-connection flows.** The `beginAddConnection` path pre-creates a placeholder Connection and switches to it before the pair wizard runs — so `applyPairingPayload` lands the token in the right auth store. Previously, cleanup of the placeholder was wired only to the explicit Cancel button and TopAppBar back arrow. System back (gesture back / predictive back) bypassed that branch, leaving the placeholder in the connection list forever. Two-part fix: (a) `PairScreen` now installs a `BackHandler` that routes system back through the same `onCancel` → `discardPlaceholderConnection` branch the explicit back arrow uses; (b) `ConnectionViewModel.init` sweeps for any existing orphans (tuple: `pairedAt == null && apiServerUrl.isBlank() && label == PLACEHOLDER_LABEL`) on cold start and removes them — the tuple cannot be produced by any real pairing, so the sweep is safe without a dry-run. If the active connection at startup points at an orphan, the sweep switches to the first surviving real connection before deleting. Fixes the "why does my chip say 'New connection…'" symptom on devices that were affected pre-fix.
- **Pair flow now auto-starts the camera on Add connection.** `ConnectionWizard` gains an `autoStart: String?` param (currently only `"scan"` is honored). The Add-connection FAB on `ConnectionsSettingsScreen` passes it so the wizard fires the camera permission launcher on first composition instead of forcing users through the Method chooser — one obvious next step, one-tap flow. Re-pair surfaces intentionally leave `autoStart` null so the full Scan / Enter code / Show code chooser stays available there. The deep-link arg is plumbed through `Screen.Pair`'s route (`pair?connectionId=...&autoStart=...`) and `PairScreen`'s new `autoStart` param; unrecognized values fall through to the default Method step so future builds can add more targets without breaking old ones.
@@ -2096,7 +2112,7 @@ picker.
- **`CLAUDE.md`** — updated Git section with the new branching policy,
added file-table entries for `hermes-relay-update`,
`register_code_command`, and the expanded `install.sh`
- **`TODO.md`** — captures open research questions around proper
- **`docs/project/TODO.md`** — captures open research questions around proper
Hermes plugin/skill/tool distribution
- **`user-docs` vitepress site** — new "For AI Agents" copy-paste
block on the home view, Feature Matrix component, two-track explainer,
+5 -9
View File
@@ -1,22 +1,18 @@
# Hermes-Relay CLI+UI v__VERSION__
**Release Date:** 2026-08-31
**Release Date:** 2026-09-02
This beta preserves complete multi-route pairing while preventing Desktop from dialing Dashboard-ingress Relay routes before Dashboard WebSocket ticket support is available. (Related: #399)
This beta fixes Windows updates so the installed CLI and management UI advance together. Explicit CLI-only installations keep their standalone update path.
**Beta phase.** Assets remain unsigned, so Windows SmartScreen and macOS Gatekeeper may warn on first launch. Standalone CLI binaries ship for Windows x64, Linux x64/arm64, and macOS x64/arm64; the management UI is Windows-only.
## What's changed
### Changed
- **Saved hosts retain the full route topology.** Dashboard, Relay, optional API, route priority, transport protection, certificate pins, and the selected host survive LAN, Tailscale, and public-route changes without creating duplicate hosts.
- **API-less pairing is first-class.** Dashboard plus direct Relay can pair without inventing an optional API server, while secure-first ranking retains plain LAN as the final fallback.
### Fixed
- **Dashboard-ingress Relay routes fail closed on Desktop.** The daemon, host selector, and Relay transport reject ingress that requires a Dashboard WebSocket ticket and choose a compatible direct Relay fallback instead of attempting an unauthenticated dial.
- **Pairing accepts the current v3 candidate shape.** Optional API records, same-origin Dashboard/Relay routes, and legacy top-level payloads remain compatible without collapsing route ownership.
- `hermes-relay update` detects an installed management UI beside the CLI and reports both installed versions.
- Bundle installations use the checksum-verified Windows installer to update and restart the affected CLI and UI together.
- Explicit CLI-only installations continue to use the standalone binary updater.
## Install
+10 -6
View File
@@ -259,16 +259,20 @@ translations may ship as `ai-translated`; do not claim fluent review unless a
review reference is recorded. Focused correction PRs from fluent contributors
are the canonical way to improve wording and can advance a locale to
`community-reviewed` or `verified` under `docs/translation-playbook.md`.
Translated READMEs use separate `README.<locale>.md` files; `README.md` remains
the canonical project description. User docs may be added incrementally under
`user-docs/<locale>/`, with links back to canonical English reference material.
Translated README entrypoints live under `docs/readme/` as
`README.<locale>.md`; root `README.md` remains the canonical project
description. Keep translated entrypoints concise: summarize onboarding and
core capabilities, link to localized user docs where available, and link back
to English for fast-moving architecture, security, and operator detail. User
docs may be added incrementally under `user-docs/<locale>/`, with links back to
canonical English reference material.
## Changelog & writing conventions
This is a **public repo** — `CHANGELOG.md`, `DEVLOG.md`, the README, and everything under `docs/` ship publicly. Keep them clean:
This is a **public repo** — `CHANGELOG.md`, `docs/project/DEVLOG.md`, the README, and everything under `docs/` ship publicly. Keep them clean:
- **`CHANGELOG.md`** follows [Keep a Changelog](https://keepachangelog.com/) (Added / Changed / Fixed). Append your change to the `## [Unreleased]` block in the PR. Entries can carry detail while they accumulate, but at release-prep the version block is **condensed to crisp public bullets** (1–2 lines each) — the deep "how we debugged it" narrative belongs in commit messages and `DEVLOG.md`, not the public changelog.
- **`DEVLOG.md`** is a factual engineering log — what changed, why, and how it was verified. Keep it depersonalized and third-person; it's a record, not a diary.
- **`CHANGELOG.md`** follows [Keep a Changelog](https://keepachangelog.com/) (Added / Changed / Fixed). Append your change to the `## [Unreleased]` block in the PR. Entries can carry detail while they accumulate, but at release-prep the version block is **condensed to crisp public bullets** (1–2 lines each) — the deep "how we debugged it" narrative belongs in commit messages and `docs/project/DEVLOG.md`, not the public changelog.
- **`docs/project/DEVLOG.md`** is a factual engineering log — what changed, why, and how it was verified. Keep it depersonalized and third-person; it's a record, not a diary.
- **No non-public wording anywhere committed:** no personal names (attribute impersonally — identity lives in git history), no real server hostnames/IPs or internal deployment names, no AI/assistant process self-narration, no fork/branch plumbing in user-facing notes. Generic example IPs in setup docs are fine.
Release notes (`RELEASE_NOTES.md`, `app/src/main/assets/whats_new.txt`, `docs/play-store-listing.md`) are theme-framed and user-facing; see [RELEASE.md](RELEASE.md) §2 "Scrub for public distribution" for the full checklist.
+7 -1
View File
@@ -21,7 +21,13 @@
</p>
<p align="center">
<strong>English</strong> · <a href="README.zh-CN.md">简体中文</a><br>
<strong>English</strong> ·
<a href="docs/readme/README.de.md">Deutsch</a> ·
<a href="docs/readme/README.es.md">Español</a> ·
<a href="docs/readme/README.ja.md">日本語</a> ·
<a href="docs/readme/README.pt-BR.md">Português (Brasil)</a> ·
<a href="docs/readme/README.ru.md">Русский</a> ·
<a href="docs/readme/README.zh-CN.md">简体中文</a><br>
<a href="https://hermes-relay.dev/docs/">Documentation</a> ·
<a href="https://github.com/Codename-11/hermes-relay/releases">Releases</a> ·
<a href="https://github.com/Codename-11/hermes-relay/discussions">Discussions</a> ·
-109
View File
@@ -1,109 +0,0 @@
<p align="center">
<img src="assets/play-store-feature-1024x500.png" alt="Hermes-Relay — 随身携带您的 Hermes 代理" width="800">
</p>
<p align="center">
<strong>运行在您的电脑上,连接到您的设备。</strong><br>
Hermes-Relay 是 <a href="https://github.com/NousResearch/hermes-agent">Hermes Agent</a> 的原生 Android 客户端,提供流式聊天、免手动语音和代理管理;另有单文件 CLI,让代理在已配对的电脑上安全使用终端、文件和截图工具。
</p>
<p align="center">
<strong>简体中文</strong> · <a href="README.md">English</a><br>
<a href="https://hermes-relay.dev/docs/zh-CN/">中文文档</a> ·
<a href="https://github.com/Codename-11/hermes-relay/releases">版本下载</a> ·
<a href="https://github.com/Codename-11/hermes-relay/discussions">社区讨论</a> ·
<a href="CHANGELOG.md">更新日志</a>
</p>
> 英文 [README.md](README.md) 是最新、完整的项目说明。本页维护中文安装入口和核心功能摘要;协议、架构和维护者文档以英文版本为准。
## 功能简介
- **Android 应用**:流式聊天、会话历史、文件附件、Hermes 管理、语音模式、原生插件页面、Petdex 悬浮宠物、多连接和配置文件;也可将 Hermes 设为 Android 助手。
- **无需插件的标准路径**:聊天、管理和标准语音可直接连接未修改的上游 Hermes Agent。
- **可选 Relay 插件**:增加终端、手机控制、媒体传输、通知助手、Relay 语音、电脑工具,以及需确认的代理创建插件页面草稿。
- **安全连接**:二维码配对、Android Keystore、证书固定、按通道授权和可配置会话有效期。
- **远程使用**:可配置 Tailscale 或 HTTPS 地址,在家庭局域网和远程路由之间自动切换。
- **两种 Android 发行渠道**:Google Play 版本适合日常使用;sideload 版本包含完整手机控制能力。
## 快速开始
### 1. 安装 Android 应用
- [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay):自动更新,包含聊天、语音、管理、终端、媒体和通知功能。
- [GitHub Releases](https://github.com/Codename-11/hermes-relay/releases):下载最新 `android-v*` 版本中以 `-sideload-release.apk` 结尾的文件,获得完整手机控制功能。
### 2. 启动 Hermes API 服务
手机需要能够访问 Hermes API 服务,并使用 API 密钥进行身份验证:
```bash
hermes setup --portal
mkdir -p ~/.hermes
API_SERVER_KEY="$(openssl rand -hex 32)"
cat >> ~/.hermes/.env <<EOF
API_SERVER_ENABLED=true
API_SERVER_HOST=0.0.0.0
API_SERVER_PORT=8642
API_SERVER_KEY=$API_SERVER_KEY
EOF
chmod 600 ~/.hermes/.env
echo "Android API URL: http://<电脑IP>:8642 key: $API_SERVER_KEY"
hermes gateway
```
`0.0.0.0` 会让同一网络中的设备访问 API。请保留强密钥;离开可信局域网时,应使用 Tailscale 或 HTTPS 反向代理,不要直接把端口暴露到互联网。
### 3. 在手机上连接
打开应用后,可以:
- 扫描局域网中的 Hermes;
- 手动输入 `http://<主机>:8642` 和 API 密钥;
- 扫描包含 API、Dashboard 和可选 Relay 地址的设置二维码。
如需在手机上管理模型、密钥、技能和配置文件,请运行 Hermes Dashboard,并在应用的 **管理** 页面登录一次。同一登录会话也会启用标准语音。
### 4. 可选:安装 Relay
仅在需要终端、手机控制、媒体路由、Relay 会话、实时语音、电脑工具或代理创建插件页面草稿时安装:
```bash
hermes plugins install Codename-11/hermes-relay/plugin --enable
hermes relay doctor
hermes relay start --no-ssl
hermes pair
```
已安装的 Hermes 插件可通过已认证的 Dashboard 向 Android 提供由应用安全渲染的原生页面,无需在手机上运行插件代码。Relay 1.5.0 另支持需用户确认的代理创建页面草稿。
完整说明请阅读[中文快速开始](https://hermes-relay.dev/docs/zh-CN/guide/quick-start);远程访问、协议和高级配置暂时链接到英文参考文档。
安装问题、早期想法、一般交流和作品分享请使用 [GitHub Discussions](https://github.com/Codename-11/hermes-relay/discussions)。可复现的错误和明确、可执行的功能请求请提交到 [Issues](https://github.com/Codename-11/hermes-relay/issues/new)。
## 中文界面
<table>
<tr>
<td align="center" width="33%"><img src="assets/screenshots/Zh01.jpg" alt="中文设置界面" width="100%"><br><sub><b>设置</b></sub></td>
<td align="center" width="33%"><img src="assets/screenshots/Zh02.jpg" alt="中文管理界面" width="100%"><br><sub><b>管理</b></sub></td>
<td align="center" width="33%"><img src="assets/screenshots/Zh03.jpg" alt="中文导航界面" width="100%"><br><sub><b>导航</b></sub></td>
</tr>
</table>
## 参与翻译
Android 英文资源是规范来源。新增语言必须保持资源名称、类型和格式参数一致,并通过:
```bash
python scripts/check-android-locales.py
./gradlew lint
```
翻译规范、目录命名、复数和占位符规则见 [docs/localization.md](docs/localization.md)。
## 许可证
[MIT](LICENSE) — Copyright (c) 2026 [Axiom-Labs](https://codename-11.dev)
+1 -1
View File
@@ -924,7 +924,7 @@ gradlew promoteReleaseArtifact --from-track=internal --promote-track=production
(This step was only needed as a retrofit for v0.1.0 — v0.1.1+ inherit
the Download section automatically from `RELEASE_NOTES.md`.)
- Confirm Play Console shows the new versionCode on the target track.
- Update `DEVLOG.md` with a short entry for the release.
- Update `docs/project/DEVLOG.md` with a short entry for the release.
## CI Behavior
+20 -23
View File
@@ -1,10 +1,10 @@
# Hermes-Relay Android v1.15.0
# Hermes-Relay Android v1.15.1
**Release Date:** August 31, 2026
**Release Date:** September 2, 2026
## Download
> Installing on your phone? Download `hermes-relay-1.15.0-sideload-release.apk` and tap it for the full feature set, or install the conservative build from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
> Installing on your phone? Download `hermes-relay-1.15.1-sideload-release.apk` and tap it for the full feature set, or install the conservative build from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
The `.aab` file is a Play Console upload bundle and cannot be installed by tapping it on a phone.
@@ -12,32 +12,29 @@ Verify the download against `SHA256SUMS.txt`. See the [sideload guide](https://h
## Summary
This release makes the everyday Android experience work through current upstream Hermes without treating the optional Hermes-Relay Plugin as a prerequisite. It also strengthens parent access, cross-client activity, transport ownership, recovery, and release visibility.
This patch improves chat, media, and voice reliability. It reduces memory-heavy work, keeps attachment previews stable through rotation, and makes follow-up message behavior and voice errors easier to understand.
## Changed
- **Upstream-first standard surfaces.** Chat attachments, inbound host files, current-session Git reads, Nous usage bars, and keyed Hermes notices use authenticated Dashboard/Gateway contracts first. Relay remains an additive compatibility and enhancement layer.
- **Clear Settings ownership.** Media sits with Chat and Voice under Hermes. Proactive Threads, Terminal, Notification Companion, Relay sessions, enhanced voice, and Device Control stay grouped under Relay tools.
- **App-specific supervised parent access.** Parents choose a PIN or password and receive a six-word recovery phrase; Android device credentials and biometrics no longer grant parent authority.
- **Complete What's New history.** Release highlights, the remaining improvements and fixes, compatibility notes, toast counts, and history now come from one structured inventory.
- Choose Correct now or Queue next from a slim tray behind the composer. Chat settings sets the default; the tray overrides one message. Stop pauses the queue, Resume continues it, and editing or removing an item preserves its successors.
- Chat and Voice use readable centered layouts on wider screens, including landscape Voice Focus.
## Fixed
- Host-local images, audio, video, and files download through the authenticated Dashboard and remain loaded across history reconciliation instead of flashing `Relay URL not configured` or returning to Loading.
- Standard voice remains usable after explicit Relay removal, while temporary Relay outages preserve configured choices and default-profile preferences stay isolated across connections.
- Android can display uniquely matched Desktop/TUI Working and Waiting activity without resuming, activating, or interrupting the external turn.
- Dashboard/Gateway chats preserve their transcript, draft, profile, and session through sign-out or outages instead of silently changing to Direct API.
- Completed text survives Dashboard-history authentication expiry and returns the existing sign-in recovery path.
- Long-running context compaction refreshes the turn watchdog instead of being interrupted as idle.
- Bot Chat history renders immediately after its route-owned handler binds.
- An exact idle live-session snapshot settles an Android-owned turn whose terminal frame was lost, reconciles history, and drains its queued follow-up.
- Supervised Add Gateway remains parent-owned across relock, back navigation, and pending setup cancellation.
- Completed generated images stay visible during marker persistence lag and retain the intended image-generation presentation.
- Correction and delivery labels remain visible inside user-message bubbles.
- Voice errors open in a scrollable dialog with separate Retry and Dismiss actions.
- Attachment previews remain open through rotation, and video previews preserve their proportions.
- Release optimization preserves the native speech configuration required for wake-word startup.
- Standard Hermes attachments download directly to disk with bounded size checks.
- Session refresh avoids repeated request loops; history loads, Markdown, image previews, and media exports keep memory use bounded.
- Image-generation progress stays visible through gaps between interim replies and returned media.
- The first prompt waits for Gateway session readiness. Ownership refusals retain the prompt for retry and show the original server error.
## Install / Verify
- App version: **1.15.0** (versionCode **53**).
- Standard Chat, sessions, profiles, Manage, standard voice, outbound attachments, and ordinary inbound files work against current unmodified upstream Hermes without the optional Hermes-Relay Plugin.
- Install Hermes-Relay Plugin v1.11.1 for Terminal, proactive Threads/offline delivery, Notification Companion, Relay sessions, provider-native/realtime voice, compatibility media metadata, Secure Link, and phone/device control.
- Explicit Direct API/API-only connections remain supported and are never used as a silent failover for a Dashboard-owned chat.
- Granular Device Control and the system Voice Focus overlay remain sideload-only; the Google Play build does not declare their restricted permissions.
- App version: **1.15.1** (versionCode **54**).
- Standard Chat, sessions, profiles, Manage, voice, and ordinary media use current upstream Hermes. Speech-to-text still requires a configured provider on the host.
- Hermes-Relay Plugin **1.11.1** remains the current optional plugin release; this Android patch does not require a new plugin version.
- Paused text queues can be restored. Attachment bytes are not persisted in preferences; unrestorable attachment queues must be reviewed and sent again.
- Explicit Direct API/API-only connections remain supported and are not used as silent failover for Dashboard-owned chats.
- Granular Device Control and the system Voice Focus overlay remain sideload-only.
+2 -1
View File
@@ -367,8 +367,9 @@ dependencies {
implementation(libs.okhttp)
implementation(libs.okhttp.sse)
// Media3 ExoPlayer — gapless TTS queue playback (replaces MediaPlayer in VoicePlayer)
// Media3 ExoPlayer + lifecycle-aware Compose video surface.
implementation(libs.media3.exoplayer)
implementation(libs.media3.ui.compose)
// android-vad Silero — on-device VAD for barge-in (B2)
// Bundled ONNX Silero model (~2.2 MB); pulled from JitPack.
+3
View File
@@ -55,6 +55,9 @@
-keep class androidx.camera.** { *; }
-dontwarn androidx.camera.**
# sherpa-onnx JNI resolves Kotlin configuration classes and fields by name.
-keep class com.k2fsa.sherpa.onnx.** { *; }
# ── General ──────────────────────────────────────────────────────────
-keepattributes SourceFile,LineNumberTable
-renamesourcefileattribute SourceFile
@@ -15,6 +15,10 @@ import androidx.compose.ui.test.assertTextEquals
import androidx.compose.ui.test.junit4.v2.createAndroidComposeRule
import androidx.compose.ui.test.onAllNodesWithText
import androidx.compose.ui.test.onNodeWithTag
import androidx.compose.ui.test.onAllNodesWithContentDescription
import androidx.compose.ui.test.onNodeWithContentDescription
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.performClick
import androidx.test.platform.app.InstrumentationRegistry
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.hermesandroid.relay.data.MessageRole
@@ -171,6 +175,81 @@ class GatewayExternalFixtureInstrumentedTest {
}
}
@Test
fun queuedStopResume_preservesWorkAcrossLifecycleAndUsesExplicitResume() {
val base = InstrumentationRegistry.getArguments().getString(ARG_FIXTURE_BASE_URL)?.trimEnd('/')
assumeTrue("Pass a queued_stop_resume fixture URL", !base.isNullOrBlank())
requireNotNull(base)
val http = OkHttpClient.Builder().callTimeout(10, TimeUnit.SECONDS).build()
assertEquals("queued_stop_resume", readFixtureJson(http, "$base/__fixture__/state")["scenario"]?.jsonString())
val dashboard = DashboardApiClient(base, http)
val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO).also { gatewayScope = it }
val gateway = GatewayChatClient(
initialDashboardClient = dashboard, okHttpClient = http, scope = scope,
callbackDispatcher = { block -> Handler(Looper.getMainLooper()).post(block) },
reconnectJitterUnit = { 0.0 },
).also { gatewayClient = it }
val handler = ChatHandler().also { it.setSessionId(STORED_SESSION_ID) }
val vm = ChatViewModel().also {
it.initialize(null, handler)
it.streamingEndpoint = "gateway"
it.setProfileMessageLoaderWithMode { profile, session, mode -> dashboard.getSessionMessages(session, profile, mode) }
it.updateGatewayClient(gateway)
it.switchProfileContext(com.hermesandroid.relay.data.AgentDisplay.profileContextKey("fixture-queue", null), STORED_SESSION_ID)
}.also { viewModel = it }
compose.setContent {
val queue by vm.queuedMessages.collectAsStateWithLifecycle()
val paused by vm.queuePaused.collectAsStateWithLifecycle()
com.hermesandroid.relay.ui.theme.HermesRelayTheme(themePreference = "dark") {
androidx.compose.material3.Surface {
Column {
com.hermesandroid.relay.ui.components.ChatBusyActionSelector(
com.hermesandroid.relay.data.BusyMessageAction.QueueNext, {}, onStop = vm::cancelStream,
)
com.hermesandroid.relay.ui.components.ChatMessageQueue(
queue, paused, vm::resumeQueue, vm::clearQueue, {}, vm::removeQueuedAt, canEdit = true,
)
}
}
}
}
assertTrue(runBlocking { gateway.prewarmAwait(STORED_SESSION_ID) })
compose.runOnIdle { vm.sendMessage("Original work") }
compose.waitUntil(10_000) { handler.isStreaming.value && vm.steerableTurn.value }
compose.runOnIdle {
vm.sendMessage("Remove this follow-up", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
vm.sendMessage("Keep this follow-up", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
}
compose.onAllNodesWithContentDescription("Remove queued message")[0].performClick()
compose.onNodeWithContentDescription("Stop streaming").performClick()
compose.onNodeWithText("Queue paused").assertIsDisplayed()
assertEquals(listOf("Keep this follow-up"), vm.queuedMessages.value)
compose.activityRule.scenario.moveToState(androidx.lifecycle.Lifecycle.State.CREATED)
compose.activityRule.scenario.moveToState(androidx.lifecycle.Lifecycle.State.RESUMED)
compose.onNodeWithText("Keep this follow-up").assertIsDisplayed()
compose.onNodeWithText("Resume").performClick()
try {
compose.waitUntil(15_000) {
vm.queuedMessages.value.isEmpty() && !handler.isStreaming.value &&
handler.messages.value.any { it.content == "Resumed follow-up." }
}
} catch (error: androidx.compose.ui.test.ComposeTimeoutException) {
throw AssertionError(
"Synthetic queue fixture did not settle: queued=${vm.queuedMessages.value.size}, " +
"paused=${vm.queuePaused.value}, streaming=${handler.isStreaming.value}, " +
"messages=${handler.messages.value.map { it.role to it.content }}, " +
"error=${handler.error.value}",
error,
)
}
val evidence = readFixtureJson(http, "$base/__fixture__/evidence")["entries"] as JsonArray
assertEquals(2, evidence.rpcCount("prompt.submit"))
assertEquals(1, evidence.rpcCount("session.interrupt"))
assertEquals(0, evidence.rpcCount("session.redirect"))
assertEquals("gateway", vm.streamingEndpoint)
}
private fun JsonArray.rpcCount(method: String): Int = count { element ->
val entry = element as? JsonObject ?: return@count false
entry["kind"]?.jsonString() == "rpc" && entry["method"]?.jsonString() == method
@@ -1,3 +1,3 @@
v1.15.0 - Standard Hermes first, with clearer Relay boundaries
v1.15.1 - Steadier chat, media, and voice
Standard Chat, Voice, attachments, returned files, current-session Git, usage, and Hermes notices now prefer upstream Dashboard and Gateway support without requiring Relay. Returned media stays loaded, voice survives Relay removal, and Settings clearly separates standard Hermes from Relay tools. Supervised Mode also gains app-specific parent access and recovery.
More reliable chats and media: fewer memory-heavy refreshes, smoother large histories, and attachment previews that survive rotation. Choose whether follow-ups correct the current response or wait in a queue. Voice errors are easier to read, image-generation progress stays visible, and wake-word startup and first-message readiness are fixed.
+77
View File
@@ -1,6 +1,83 @@
{
"schema": 3,
"versions": [
{
"version": "1.15.1",
"title": "Steadier chat, media, and voice",
"date": "2026-09-02",
"summary": "Chats use less memory, attachment previews stay in place, and voice failures are easier to recover from. Follow-up controls make it clear whether a message changes the current response or waits for the next turn.",
"changes": [
{
"id": "follow-up-controls",
"kind": "improved",
"title": "Choose when follow-up messages are sent",
"summary": "A slim tray behind the composer offers Correct now or Queue next. Chat settings sets the default, and a composer choice applies to one message. Stop pauses pending work until Resume; editing or removing an item keeps the remaining queue usable.",
"highlight": true
},
{
"id": "tablet-layouts",
"kind": "improved",
"title": "Make better use of wider screens",
"summary": "Chat and Voice keep text and controls on readable centered layouts. Landscape Voice Focus separates identity controls from conversation activity."
},
{
"id": "delivery-labels",
"kind": "fixed",
"title": "Read message delivery status clearly",
"summary": "Correction and delivery labels use contrasting text instead of disappearing into the message bubble."
},
{
"id": "voice-error-dialog",
"kind": "fixed",
"title": "Read and dismiss voice errors",
"summary": "Voice errors open in a contained dialog with scrollable details and separate Retry and Dismiss actions, without overlapping chat controls."
},
{
"id": "attachment-previews",
"kind": "fixed",
"title": "Keep attachment previews open through rotation",
"summary": "Image, video, audio, PDF, text, and file previews stay open as the screen rotates. Videos retain their original proportions.",
"highlight": true
},
{
"id": "wake-word-startup",
"kind": "fixed",
"title": "Fix wake-word startup in release builds",
"summary": "Release optimization now preserves the native speech configuration names needed to initialize wake-word detection."
},
{
"id": "attachment-downloads",
"kind": "fixed",
"title": "Download attachments with less memory",
"summary": "Standard Hermes attachments stream into the on-disk cache while download size limits remain enforced."
},
{
"id": "chat-memory-safety",
"kind": "fixed",
"title": "Keep large chats and media manageable",
"summary": "Automatic session refresh no longer loops. Routine history loads, chat rendering, image previews, and media exports use bounded memory instead of allocating entire large responses.",
"highlight": true
},
{
"id": "image-progress",
"kind": "fixed",
"title": "Keep image-generation progress visible",
"summary": "The working indicator stays visible between interim replies and the generated image, including gateways that omit tool activity events."
},
{
"id": "first-message-readiness",
"kind": "fixed",
"title": "Wait for new chats to be ready",
"summary": "The first message waits for the Gateway session to initialize. Ownership refusals keep the prompt retryable and show the server's error."
}
],
"compatibility": [
"Standard Chat, sessions, media, and voice continue to use upstream Hermes. Voice transcription still requires a configured speech-to-text provider on the Hermes host.",
"Paused text queues can be restored. Attachment bytes are not stored in preferences; an attachment queue that cannot be restored must be reviewed and sent again."
],
"playNotes": "More reliable chats and media: fewer memory-heavy refreshes, smoother large histories, and attachment previews that survive rotation. Choose whether follow-ups correct the current response or wait in a queue. Voice errors are easier to read, image-generation progress stays visible, and wake-word startup and first-message readiness are fixed.",
"sections": []
},
{
"version": "1.15.0",
"title": "Standard Hermes first, with clearer Relay boundaries",
+14 -19
View File
@@ -1,29 +1,24 @@
v1.15.0 - Standard Hermes first, with clearer Relay boundaries
v1.15.1 - Steadier chat, media, and voice
Summary
* Chat, voice, attachments, inbound files, current-session Git, usage, and Hermes notices now prefer current upstream Dashboard and Gateway support. Relay stays optional for compatibility and the tools it uniquely provides.
* Chats use less memory, attachment previews stay in place, and voice failures are easier to recover from. Follow-up controls make it clear whether a message changes the current response or waits for the next turn.
Highlights
* Use standard Hermes without Relay prompts — Chat attachments, inbound files, current-session Git, Nous usage, and Hermes notices use upstream routes first.
* Keep returned files loaded — Images, audio, video, and documents download through the Dashboard and no longer flash Relay errors or return to Loading.
* Use app-specific parent access — A parent PIN or password plus recovery phrase protects Supervised Mode without trusting the phone unlock credential.
* Choose when follow-up messages are sent — A slim tray behind the composer offers Correct now or Queue next. Chat settings sets the default, and a composer choice applies to one message. Stop pauses pending work until Resume; editing or removing an item keeps the remaining queue usable.
* Keep attachment previews open through rotation — Image, video, audio, PDF, text, and file previews stay open as the screen rotates. Videos retain their original proportions.
* Keep large chats and media manageable — Automatic session refresh no longer loops. Routine history loads, chat rendering, image previews, and media exports use bounded memory instead of allocating entire large responses.
Improved
* See which features need Relay — Media sits with standard Hermes settings while Threads, Terminal, notifications, enhanced voice, and device tools stay under Relay tools.
* Read the complete release record — What's New shows one release summary, selected highlights, every remaining change, and relevant compatibility notes.
* Make better use of wider screens — Chat and Voice keep text and controls on readable centered layouts. Landscape Voice Focus separates identity controls from conversation activity.
Fixed
* Keep Standard voice after removing Relay — Dashboard voice remains ready, temporary outages preserve choices, and shared default-profile settings stay isolated.
* Observe another client's activity safely — A uniquely matched Desktop or TUI turn can show Working or Waiting without Android taking control.
* Keep each chat on its chosen transport — Dashboard chats preserve their transcript, draft, profile, and session through sign-out or outages instead of silently changing databases.
* Preserve completed replies at sign-in expiry — A Dashboard history authentication failure keeps completed text visible and opens the existing sign-in recovery path.
* Let long context compaction finish — Visible compaction activity refreshes the turn watchdog instead of being interrupted as idle.
* Render Bot Chat history immediately — Route-owned Bot Chats observe their bound history from first composition.
* Settle turns after a lost terminal frame — An exact idle live-session snapshot reconciles the Android-owned turn and drains its queued follow-up.
* Keep Gateway setup parent-owned — Relock and back navigation cancel the exact pending setup without bypassing parent authority.
* Keep completed generated images visible — Generated media survives marker persistence lag and retains its intended Chat animation.
* Read message delivery status clearly — Correction and delivery labels use contrasting text instead of disappearing into the message bubble.
* Read and dismiss voice errors — Voice errors open in a contained dialog with scrollable details and separate Retry and Dismiss actions, without overlapping chat controls.
* Fix wake-word startup in release builds — Release optimization now preserves the native speech configuration names needed to initialize wake-word detection.
* Download attachments with less memory — Standard Hermes attachments stream into the on-disk cache while download size limits remain enforced.
* Keep image-generation progress visible — The working indicator stays visible between interim replies and the generated image, including gateways that omit tool activity events.
* Wait for new chats to be ready — The first message waits for the Gateway session to initialize. Ownership refusals keep the prompt retryable and show the server's error.
Compatibility
* Current upstream Hermes provides standard Chat, sessions, Manage, voice, attachments, inbound files, current-session Git reads, usage, and notices without the optional Hermes-Relay Plugin.
* Hermes-Relay Plugin 1.11.1 remains required for Terminal, proactive Threads and offline delivery, Notification Companion, Relay sessions, enhanced voice, Secure Link, and phone or device control.
* Granular Device Control and the system Voice Focus overlay remain sideload-only.
* Standard Chat, sessions, media, and voice continue to use upstream Hermes. Voice transcription still requires a configured speech-to-text provider on the Hermes host.
* Paused text queues can be restored. Attachment bytes are not stored in preferences; an attachment queue that cannot be restored must be reviewed and sent again.
@@ -22,6 +22,26 @@ enum class PhysicalKeyboardEnterBehavior(val storedValue: String) {
}
}
/** Default intent for a message submitted while an agent is responding. */
enum class BusyMessageAction(val storedValue: String) {
CorrectNow("correct_now"),
QueueNext("queue_next");
companion object {
fun fromStoredValue(value: String?): BusyMessageAction =
entries.firstOrNull { it.storedValue == value } ?: CorrectNow
}
}
fun canCorrectBusyMessage(
steerable: Boolean,
hasAttachments: Boolean,
hasPendingInput: Boolean,
status: String?,
text: String,
): Boolean = steerable && !hasAttachments && !hasPendingInput &&
status?.contains("compact", ignoreCase = true) != true && !text.trimStart().startsWith("/")
/** Device-level chat input preferences shared by every Hermes profile. */
class ChatInputPreferencesRepository(
private val dataStore: DataStore<Preferences>,
@@ -29,6 +49,7 @@ class ChatInputPreferencesRepository(
constructor(context: Context) : this(context.relayDataStore)
companion object {
internal val KEY_BUSY_MESSAGE_ACTION = stringPreferencesKey("busy_message_action")
internal val KEY_PHYSICAL_KEYBOARD_ENTER =
stringPreferencesKey("physical_keyboard_enter_behavior")
internal val KEY_CONVERT_LARGE_PASTES =
@@ -45,6 +66,14 @@ class ChatInputPreferencesRepository(
}
.distinctUntilChanged()
val busyMessageAction: Flow<BusyMessageAction> = dataStore.data
.map { BusyMessageAction.fromStoredValue(it[KEY_BUSY_MESSAGE_ACTION]) }
.distinctUntilChanged()
suspend fun setBusyMessageAction(action: BusyMessageAction) {
dataStore.edit { it[KEY_BUSY_MESSAGE_ACTION] = action.storedValue }
}
val convertLargePastesToAttachments: Flow<Boolean> = dataStore.data
.map { preferences -> preferences[KEY_CONVERT_LARGE_PASTES] ?: true }
.distinctUntilChanged()
@@ -35,6 +35,9 @@ data class ChatTurnCheckpoint(
val baselineAssistantCount: Int,
val pendingAsk: ChatTurnAskCheckpoint? = null,
val queuedMessages: List<ChatQueuedMessageCheckpoint> = emptyList(),
val queuePaused: Boolean = false,
/** Only pending local work remains; never reattach the completed/stopped turn. */
val queueOnly: Boolean = false,
val startedAt: Long,
val updatedAt: Long,
) {
@@ -348,7 +348,7 @@ class BridgeCommandHandler(
* the multiplexer. The two paths are fully independent.
*
* Caught by Bailey's on-device test 2026-04-14 — see the v0.4.1
* "voice intent local dispatch loop" entry in ROADMAP.md.
* "voice intent local dispatch loop" entry in docs/project/ROADMAP.md.
*/
suspend fun handleLocalCommand(envelope: Envelope): LocalDispatchResult {
if (envelope.type != "bridge.command") {
@@ -49,10 +49,10 @@ import okhttp3.Request
import okhttp3.RequestBody
import okhttp3.RequestBody.Companion.toRequestBody
import okhttp3.Response
import java.io.ByteArrayOutputStream
import java.io.IOException
import java.io.InputStream
import java.io.OutputStream
import java.io.ByteArrayOutputStream
import java.net.URLEncoder
import java.util.concurrent.TimeUnit
import kotlin.coroutines.resume
@@ -294,6 +294,34 @@ internal fun copyBounded(
return written
}
internal fun copyMediaBounded(
input: InputStream,
output: OutputStream,
declaredLength: Long?,
limitBytes: Long,
): Long {
require(limitBytes > 0)
require(declaredLength == null || declaredLength >= 0)
if (declaredLength != null && declaredLength > limitBytes) {
throw IOException("File exceeds the configured download limit")
}
val buffer = ByteArray(DEFAULT_BUFFER_SIZE)
var written = 0L
while (true) {
val read = input.read(buffer)
if (read < 0) break
written += read
if (written > limitBytes) {
throw IOException("File exceeds the configured download limit")
}
output.write(buffer, 0, read)
}
if (declaredLength != null && written != declaredLength) {
throw IOException("Media file changed while it was being downloaded")
}
return written
}
/** One entry from `GET /api/audio/elevenlabs/voices` — non-secret voice metadata. */
data class ElevenLabsVoice(
val voiceId: String,
@@ -312,14 +340,14 @@ data class ElevenLabsVoices(
)
/**
* Bytes fetched from upstream's authenticated managed-files surface.
* Metadata for a file streamed from upstream's authenticated managed-files surface.
*
* The Dashboard applies its own managed-root, sensitive-file, and maximum-size
* policy before these bytes leave the Hermes host. Android applies the user's
* policy before the file leaves the Hermes host. Android applies the user's
* stricter inbound-media cap while reading the response as a second boundary.
*/
data class DashboardFetchedFile(
val bytes: ByteArray,
val sizeBytes: Long,
val contentType: String,
val fileName: String?,
)
@@ -401,13 +429,14 @@ class DashboardApiClient(
* This is the same authenticated `/api/files/download` route official
* Desktop uses for remote gateway files. The caller supplies its display
* cap so a malicious or stale Content-Length cannot cause an unbounded
* allocation. Audio/video are downloaded into Android's local media cache;
* allocation. The supplied output is normally Android's on-disk media cache;
* local playback supplies seeking, so `/api/files/stream` is unnecessary
* on this path.
*/
suspend fun downloadManagedFile(
serverPath: String,
maxBytes: Long,
output: OutputStream,
): Result<DashboardFetchedFile> = withContext(Dispatchers.IO) {
if (serverPath.isBlank()) {
return@withContext Result.failure(IOException("Media path is empty"))
@@ -428,29 +457,11 @@ class DashboardApiClient(
if (declaredLength != null && declaredLength > maxBytes) {
throw IOException("File exceeds the configured download limit")
}
val initialSize = declaredLength
?.coerceAtMost(Int.MAX_VALUE.toLong())
?.toInt()
?: DEFAULT_BUFFER_SIZE
val output = ByteArrayOutputStream(initialSize)
body.byteStream().use { input ->
val buffer = ByteArray(DEFAULT_BUFFER_SIZE)
var readTotal = 0L
while (true) {
val count = input.read(buffer)
if (count < 0) break
readTotal += count
if (readTotal > maxBytes) {
throw IOException("File exceeds the configured download limit")
}
output.write(buffer, 0, count)
}
if (declaredLength != null && readTotal != declaredLength) {
throw IOException("Media file changed while it was being downloaded")
}
val readTotal = body.byteStream().use { input ->
copyMediaBounded(input, output, declaredLength, maxBytes)
}
DashboardFetchedFile(
bytes = output.toByteArray(),
sizeBytes = readTotal,
contentType = response.header("Content-Type")
?.substringBefore(';')
?.trim()
@@ -1345,7 +1356,7 @@ class DashboardApiClient(
suspend fun getSessionMessages(
sessionId: String,
profile: String? = null,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
mode: SessionMessageLoadMode = SessionMessageLoadMode.LATEST,
): Result<List<MessageItem>> = withContext(Dispatchers.IO) {
val name = profile?.trim().orEmpty()
loadSessionMessages(mode) { page ->
@@ -1749,6 +1760,7 @@ class DashboardApiClient(
// Mirrors current upstream `_MANAGED_FILE_MAX_BYTES`; enforcing it
// client-side avoids uploading a body the Dashboard will reject.
internal const val MAX_BACKUP_TRANSFER_BYTES = 100L * 1024L * 1024L
internal const val MAX_JSON_RESPONSE_BYTES = 8L * 1024L * 1024L
fun pathSegment(value: String): String =
URLEncoder.encode(value, "UTF-8").replace("+", "%20")
@@ -2477,17 +2489,44 @@ data class StoredDashboardCookie(
}
private fun Response.readJsonObject(json: Json): JsonObject {
val raw = body.string()
val raw = body.readUtf8Bounded(DashboardApiClient.MAX_JSON_RESPONSE_BYTES)
if (raw.isBlank()) return JsonObject(emptyMap())
return json.parseToJsonElement(raw).jsonObject
}
private fun Response.readJsonElement(json: Json): JsonElement {
val raw = body.string()
val raw = body.readUtf8Bounded(DashboardApiClient.MAX_JSON_RESPONSE_BYTES)
if (raw.isBlank()) return JsonObject(emptyMap())
return json.parseToJsonElement(raw)
}
internal fun okhttp3.ResponseBody.readUtf8Bounded(maxBytes: Long): String {
require(maxBytes in 1..Int.MAX_VALUE.toLong()) { "Invalid response byte limit" }
val declaredLength = contentLength()
if (declaredLength > maxBytes) {
throw IOException("Dashboard response exceeds Android's bounded JSON limit")
}
val initialSize = when {
declaredLength in 1..maxBytes -> declaredLength.toInt()
else -> minOf(maxBytes, DEFAULT_BUFFER_SIZE.toLong()).toInt()
}
val output = ByteArrayOutputStream(initialSize)
byteStream().use { input ->
val buffer = ByteArray(DEFAULT_BUFFER_SIZE)
var total = 0L
while (true) {
val read = input.read(buffer)
if (read < 0) break
total += read
if (total > maxBytes) {
throw IOException("Dashboard response exceeds Android's bounded JSON limit")
}
output.write(buffer, 0, read)
}
}
return output.toString(Charsets.UTF_8.name())
}
private fun contentDispositionFileName(header: String): String? {
val encoded = Regex("""filename\*=UTF-8''([^;]+)""", RegexOption.IGNORE_CASE)
.find(header)
@@ -2576,7 +2615,11 @@ internal fun Throwable.isDashboardManagedFilesUnsupported(): Boolean {
}
private fun apiFailure(response: Response, operation: String): IOException {
val bodyDetail = runCatching { response.body.string() }.getOrDefault("")
val bodyDetail = try {
response.body.readUtf8Bounded(4L * 1024L)
} catch (_: Exception) {
""
}
val detail = bodyDetail.take(240).ifBlank { response.message }
return DashboardHttpException(
statusCode = response.code,
@@ -119,6 +119,8 @@ class GatewayChatClient(
private val promptSubmitTimeoutMs: Long = PROMPT_SUBMIT_REQUEST_TIMEOUT_MS,
/** Test seam — idle-progress watchdog base. Production keeps [TURN_TIMEOUT_MS]. */
private val turnIdleTimeoutMs: Long = TURN_TIMEOUT_MS,
/** Test seam — lazy session.create/resume readiness barrier. */
private val sessionReadyTimeoutMs: Long = SESSION_READY_TIMEOUT_MS,
/** Test seam — compaction idle lease. Production keeps [COMPACTING_TIMEOUT_MS]. */
private val compactingTimeoutMs: Long = COMPACTING_TIMEOUT_MS,
/** Random source for ordinary reconnect full-jitter. */
@@ -191,6 +193,7 @@ class GatewayChatClient(
* would have been abandoned server-side anyway.
*/
private const val PROMPT_SUBMIT_REQUEST_TIMEOUT_MS = 1_800_000L
private const val SESSION_READY_TIMEOUT_MS = 300_000L
private const val CONNECT_TIMEOUT_MS = 20_000L
/**
@@ -413,6 +416,10 @@ class GatewayChatClient(
@Volatile
private var webSocket: WebSocket? = null
/** Profile explicitly routed by the Dashboard `/api/ws?profile=` handshake. */
@Volatile
private var connectedSocketProfile: String? = null
/** Completed when the server's `gateway.ready` event arrives for the current socket. */
@Volatile
private var readySignal: CompletableDeferred<Unit>? = null
@@ -421,6 +428,9 @@ class GatewayChatClient(
/** Invalidates an older async prewarm when a newer session selection wins. */
private val prewarmRequestGeneration = AtomicLong(0)
private val pendingRpcs = ConcurrentHashMap<Long, CompletableDeferred<JsonObject>>()
private val lazyLiveSessions = ConcurrentHashMap.newKeySet<String>()
private val readyLiveSessions = ConcurrentHashMap.newKeySet<String>()
private val sessionReadyWaiters = ConcurrentHashMap<String, CompletableDeferred<Unit>>()
/** Monotonic client-local fence for lazy child watch open/close races. */
private val childWatchGeneration = AtomicLong(0)
@@ -832,7 +842,7 @@ class GatewayChatClient(
// through the normal failed-turn callback instead.
cleanupStagedAttachments(stagedImagePaths)
turn.tracer.done("submit-rejected")
turn.callbacks.onError(
turn.callbacks.onSubmitRejected(
submitError?.message ?: "Hermes rejected the new session",
)
return@launch
@@ -899,6 +909,9 @@ class GatewayChatClient(
liveSessionId = null
storedSessionId = null
liveSessionProfile = null
failSessionReadyWaiters("gateway session cleared")
lazyLiveSessions.clear()
readyLiveSessions.clear()
cancelledTurnDrain = null
_serverTools.value = null
}
@@ -3057,9 +3070,10 @@ class GatewayChatClient(
)
}
val ticketMs = (System.nanoTime() - connectStart) / 1_000_000
val socketProfile = currentSessionProfile()
val url = dashboardClient.gatewayWebSocketUrl(
ticket = ticket.ticket,
profile = currentSessionProfile(),
profile = socketProfile,
)
?: throw GatewayConnectAttemptException(
"could not build /api/ws URL",
@@ -3093,6 +3107,7 @@ class GatewayChatClient(
if (readyFailure != null) {
socket.cancel()
webSocket = null
connectedSocketProfile = null
throw (readyFailure as? GatewayConnectAttemptException
?: GatewayConnectAttemptException(
"gateway connection failed: ${readyFailure.message}",
@@ -3105,6 +3120,7 @@ class GatewayChatClient(
val wsMs = (System.nanoTime() - connectStart) / 1_000_000 - ticketMs
Log.i(TAG, "Gateway connected (/api/ws ready) — ticket=${ticketMs}ms ws=${wsMs}ms")
hasEverReachedReady = true
connectedSocketProfile = socketProfile
coldStartFailureEpisodes = 0
_reconnectDisposition.value = GatewayReconnectDisposition.None
_connectionState.value = GatewayConnectionState.Ready
@@ -3279,7 +3295,8 @@ class GatewayChatClient(
?.stringField("profile_name")
?.trim()
?.takeIf { it.isNotEmpty() }
if (actual != expected) {
val socketConfirmed = actual == null && connectedSocketProfile == expected
if (actual != expected && !socketConfirmed) {
val detail = actual?.let { "Hermes returned profile '$it'" }
?: "Hermes did not confirm profile ownership"
throw GatewayPreflightException(
@@ -3389,6 +3406,7 @@ class GatewayChatClient(
requestedStoredId != null &&
liveSessionProfile == requestedProfile
) {
awaitSessionReadyIfRequired(liveSessionId!!)
return
}
@@ -3418,6 +3436,7 @@ class GatewayChatClient(
liveSessionProfile = requestedProfile
updateCancelledDrainLiveSession(requestedStoredId, live)
applySessionResultInfo(result)
awaitSessionReadyIfRequired(live, result)
return
}
throw GatewayAuthoritativeResumeException(
@@ -3453,6 +3472,45 @@ class GatewayChatClient(
if (cancelledTurnDrain?.storedSessionId != stored) cancelledTurnDrain = null
if (settledTurnDrain?.storedSessionId != stored) settledTurnDrain = null
turn.callbacks.onSessionId(stored)
awaitSessionReadyIfRequired(live, created)
}
/**
* Wait for current upstream's authoritative deferred-build edge instead of
* racing a lazy session into compute-host turn isolation. Without this
* barrier, the parent runtime can claim the durable lease before the child
* rechecks it, causing the child to reject itself as a second live owner.
*/
private suspend fun awaitSessionReadyIfRequired(
liveId: String,
sessionResult: JsonObject? = null,
) {
val lazy = (sessionResult?.get("info") as? JsonObject)?.booleanField("lazy") == true
if (lazy) lazyLiveSessions += liveId
if (liveId !in lazyLiveSessions) return
if (readyLiveSessions.remove(liveId)) {
lazyLiveSessions.remove(liveId)
return
}
val waiter = sessionReadyWaiters.computeIfAbsent(liveId) { CompletableDeferred() }
if (readyLiveSessions.remove(liveId)) waiter.complete(Unit)
try {
withTimeout(sessionReadyTimeoutMs) { waiter.await() }
lazyLiveSessions.remove(liveId)
} catch (error: Exception) {
throw GatewayPreflightException(
error.message ?: "Hermes session initialization timed out",
)
} finally {
sessionReadyWaiters.remove(liveId, waiter)
}
}
private fun failSessionReadyWaiters(message: String) {
sessionReadyWaiters.values.forEach {
it.completeExceptionally(GatewayRpcException(message))
}
sessionReadyWaiters.clear()
}
private fun createListener(ready: CompletableDeferred<Unit>) = object : WebSocketListener() {
@@ -3631,6 +3689,33 @@ class GatewayChatClient(
return
}
// Lazy create/resume returns before its AIAgent exists. The deferred
// build publishes exact-session session.info when it is ready. Record
// that edge before live-session routing so a fast build cannot race
// the RPC response and strand the first submit.
if (type == "session.info" && !eventSessionId.isNullOrBlank() &&
payload?.booleanField("lazy") != true
) {
readyLiveSessions += eventSessionId
sessionReadyWaiters.remove(eventSessionId)?.complete(Unit)
}
// Deferred agent construction can fail after lazy session.create/
// resume returns but before prompt.submit. Fail the readiness barrier
// immediately so the optimistic prompt remains retryable/Not sent
// instead of waiting for the five-minute readiness timeout.
if (type == "error" && !eventSessionId.isNullOrBlank() &&
(eventSessionId in lazyLiveSessions || sessionReadyWaiters.containsKey(eventSessionId))
) {
val message = payload?.stringField("message")
?: "Hermes session initialization failed"
lazyLiveSessions.remove(eventSessionId)
readyLiveSessions.remove(eventSessionId)
sessionReadyWaiters.remove(eventSessionId)
?.completeExceptionally(GatewayRpcException(message))
return
}
// Upstream emits session.reclaimed process-wide, so it is identified
// by payload rather than params.session_id. Retire only an exact live
// runtime we own; preserve the durable id so the next send resumes it.
@@ -3945,6 +4030,7 @@ class GatewayChatClient(
// id on the shared gateway stream) keeps matching after reconnect.
val preservedLiveId = liveSessionId
webSocket = null
connectedSocketProfile = null
readySignal = null
liveSessionId = null
attachMethodForSocket = null
@@ -3958,6 +4044,9 @@ class GatewayChatClient(
it.completeExceptionally(GatewayRpcException("gateway connection lost"))
}
pendingRpcs.clear()
failSessionReadyWaiters("gateway connection lost")
lazyLiveSessions.clear()
readyLiveSessions.clear()
failChildWatches("Child watch disconnected from the gateway")
val turn = activeTurn
if (turn == null) {
@@ -4133,8 +4222,12 @@ class GatewayChatClient(
private fun closeSocket(reason: String) {
webSocket?.close(1000, reason)
webSocket = null
connectedSocketProfile = null
readySignal = null
liveSessionId = null
failSessionReadyWaiters("gateway socket closed")
lazyLiveSessions.clear()
readyLiveSessions.clear()
attachMethodForSocket = null
commandsCatalogCache = null
_processCapability.value = GatewayProcessCapability.Unknown
@@ -4811,6 +4904,9 @@ class GatewayChatClient(
onStatusClear = { kind -> dispatchIfCurrent(stillCurrent) { callbacks.onStatusClear(kind) } },
onNoticeShow = { notice -> dispatchIfCurrent(stillCurrent) { callbacks.onNoticeShow(notice) } },
onNoticeClear = { key -> dispatchIfCurrent(stillCurrent) { callbacks.onNoticeClear(key) } },
onSubmitRejected = { message ->
dispatchIfCurrent(stillCurrent) { callbacks.onSubmitRejected(message) }
},
)
private fun dispatchIfCurrent(stillCurrent: () -> Boolean, callback: () -> Unit) {
@@ -196,7 +196,8 @@ class GatewayEventMapper(
"tool.start" -> {
clearActivityStatuses()
val name = payload.string("name") ?: "unknown"
val identity = payload.effectiveToolIdentity()
val name = identity.name
// A pending generating placeholder for this name is adopted
// (consumed FIFO) whether or not the server sent a real id.
val adopted = generatingIdsByName[name]?.removeFirstOrNull()
@@ -208,7 +209,7 @@ class GatewayEventMapper(
}
else -> syntheticToolId(name)
}
val argsPreview = payload?.get("args")
val argsPreview = identity.argsPreview ?: payload?.get("args")
?.takeUnless { it is JsonPrimitive && it.contentOrNull.isNullOrBlank() }
?.toString()
?.takeIf { it.isNotBlank() && it != "null" }
@@ -220,7 +221,7 @@ class GatewayEventMapper(
"tool.complete" -> {
clearActivityStatuses()
val name = payload.string("name") ?: "unknown"
val name = payload.effectiveToolIdentity().name
val toolId = payload.string("tool_id")
?: openSyntheticIdsByName[name]?.removeFirstOrNull()
?: return
@@ -279,7 +280,12 @@ class GatewayEventMapper(
"error" -> {
turnEnded = true
callbacks.onError(payload.string("message") ?: "Gateway error")
val message = payload.string("message") ?: "Gateway error"
if (isSessionOwnershipRejection(message)) {
callbacks.onSubmitRejected(message)
} else {
callbacks.onError(message)
}
}
"subagent.spawn_requested", "subagent.start", "subagent.thinking", "subagent.tool",
@@ -469,6 +475,19 @@ class GatewayEventMapper(
private const val MAX_MOA_REFERENCE_CHARS = 16_000
private val OUTPUT_RISK_LEVELS = setOf("low", "medium", "high", "critical")
private val TERMINAL_EVENTS = setOf("message.complete", "error")
/**
* Isolated-turn paths can acknowledge `prompt.submit` and then emit
* the ownership refusal as a plain terminal `error` event. That event
* has no JSON-RPC code or structured reason, so match both stable
* clauses from upstream's canonical message.
*/
internal fun isSessionOwnershipRejection(message: String): Boolean {
val normalized = message.lowercase()
return "already has a live owner (" in normalized &&
"only one surface at a time may run a session" in normalized
}
internal fun isFailedMoaReference(text: String): Boolean {
val normalized = text.trimStart().lowercase()
return normalized.startsWith("[failed:") || normalized.startsWith("[skipped:")
@@ -609,6 +628,34 @@ private const val MAX_CLARIFY_CHOICES = 4
private const val SUDO_TIMEOUT_SECONDS = 120
private const val SECRET_TIMEOUT_SECONDS = 300
private data class GatewayToolIdentity(
val name: String,
val argsPreview: String?,
)
/**
* Older Tool Search gateways can expose the model-visible `tool_call` bridge
* instead of the effective tool identity that current upstream callbacks use.
* Unwrap only the bridge's explicit structured `{name, arguments}` envelope;
* never infer a tool from prompt text or result content.
*/
private fun JsonObject?.effectiveToolIdentity(): GatewayToolIdentity {
val outerName = string("name") ?: "unknown"
val outerArgs = this?.get("args") as? JsonObject
if (outerName != "tool_call" || outerArgs == null) {
return GatewayToolIdentity(outerName, null)
}
val effectiveName = outerArgs.string("name")
?.trim()
?.takeIf { it.isNotEmpty() }
?: return GatewayToolIdentity(outerName, null)
val effectiveArgs = outerArgs["arguments"]
?.takeUnless { it is JsonPrimitive && it.contentOrNull.isNullOrBlank() }
?.toString()
?.takeIf { it.isNotBlank() && it != "null" }
return GatewayToolIdentity(effectiveName, effectiveArgs)
}
private fun JsonObject?.string(key: String): String? =
(this?.get(key) as? JsonPrimitive)?.contentOrNull
@@ -741,6 +741,12 @@ class GatewayTurnCallbacks(
val onNoticeShow: (GatewayAgentNotice) -> Unit = { _ -> },
/** Exact-key dismissal for an upstream notice. */
val onNoticeClear: (key: String) -> Unit = { _ -> },
/**
* The Gateway authoritatively refused `prompt.submit` before a model turn
* began. The composed user row remains local and retryable; callers must
* not treat this as a dropped stream or reconcile it from history.
*/
val onSubmitRejected: (String) -> Unit = onError,
)
/**
@@ -726,10 +726,10 @@ class HermesApiClient(
suspend fun getMessages(
sessionId: String,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
mode: SessionMessageLoadMode = SessionMessageLoadMode.LATEST,
): List<MessageItem> = withContext(Dispatchers.IO) {
loadSessionMessages(mode) { page ->
runCatching {
try {
val url = "$baseUrl/api/sessions/$sessionId/messages".toHttpUrlOrNull()
?.newBuilder()
?.addQueryParameter("limit", page.limit.toString())
@@ -740,14 +740,20 @@ class HermesApiClient(
val request = authRequest(url.toString()).get().build()
client.newCall(request).execute().use { response ->
if (!response.isSuccessful) error("HTTP ${response.code}")
val body = response.body?.string() ?: error("empty response body")
val body = response.body.readUtf8Bounded(
DashboardApiClient.MAX_JSON_RESPONSE_BYTES,
)
val parsed = json.decodeFromString<MessageListResponse>(body)
SessionMessagePage(
messages = parsed.data ?: parsed.items ?: parsed.messages ?: emptyList(),
pagination = parsed.pagination,
payloadChars = body.length,
)
}
}.let { Result.success(it) }
} catch (error: CancellationException) {
throw error
} catch (error: Exception) {
Result.failure(error)
}
}.getOrElse { error ->
if (error is CancellationException) throw error
@@ -72,7 +72,7 @@ internal suspend fun loadSessionMessages(
Result.success(collected)
} catch (error: CancellationException) {
throw error
} catch (error: Throwable) {
} catch (error: Exception) {
Result.failure(error)
}
}
@@ -195,7 +195,7 @@ class StandardHermesVoiceClient(
.let { Result.success<VoiceSpeechStream?>(it) }
} catch (cancelled: CancellationException) {
throw cancelled
} catch (error: Throwable) {
} catch (error: Exception) {
Result.failure(error)
}
}
@@ -222,7 +222,9 @@ class StandardHermesVoiceClient(
if (!response.isSuccessful) {
return Result.failure(apiFailure(response, operation))
}
val body = response.body.string()
val body = response.body.readUtf8Bounded(
DashboardApiClient.MAX_JSON_RESPONSE_BYTES,
)
if (body.isBlank()) {
return Result.failure(IOException("$operation returned an empty response"))
}
@@ -245,7 +247,11 @@ class StandardHermesVoiceClient(
}
private fun apiFailure(response: Response, operation: String): IOException {
val body = runCatching { response.body.string() }.getOrDefault("")
val body = try {
response.body.readUtf8Bounded(4L * 1024L)
} catch (_: Exception) {
""
}
val detail = body.takeIf { it.isNotBlank() } ?: response.message
val message = when (response.code) {
400 -> "$operation rejected that input - ${detail.ifBlank { "bad request" }}"
@@ -117,6 +117,7 @@ import com.hermesandroid.relay.ui.components.pet.PetSafeAreaRegistry
import com.hermesandroid.relay.ui.components.pet.petPerchSurface
import com.hermesandroid.relay.ui.components.pet.platformModalOwnsPetLayer
import com.hermesandroid.relay.ui.components.ConnectionSwitcherSheet
import com.hermesandroid.relay.ui.components.ChatMediaViewerHost
import com.hermesandroid.relay.ui.components.ChatTransportStatusBadge
import com.hermesandroid.relay.ui.components.ChatTransportTier
import com.hermesandroid.relay.ui.components.ConnectionSecurityGlyph
@@ -2475,103 +2476,110 @@ fun RelayApp() {
val screenChatLabel = stringResource(R.string.screen_chat_label)
ChatScreen(
chatViewModel = chatViewModel,
connectionViewModel = connectionViewModel,
voiceViewModel = voiceViewModel,
voiceClient = voiceClient,
maxBubbleWidth = maxBubbleWidth,
voicePresentationMode = voicePresentationMode,
onVoicePresentationModeChange = { mode ->
connectionSwitchScope.launch {
voicePreferences.setPresentationMode(mode)
}
},
openAgentSheetOnEntry = openAgentSheetArg,
onAgentSheetArgConsumed = {
backStackEntry.arguments?.putBoolean(
Screen.Chat.ARG_OPEN_AGENT_SHEET, false,
)
},
// AgentInfoSheet footer jumps straight into the full
// Connections CRUD screen — saves a detour through
// Settings → Gateways.
onNavigateToConnections = {
navController.navigate(Screen.ConnectionsSettings.route)
},
onNavigateToConnect = {
navController.navigate(Screen.Pair.route()) {
launchSingleTop = true
}
},
onRepairConnection = {
navController.navigate(
Screen.Pair.route(
connectionId = activeConnectionId,
autoStart = "relay",
),
) {
launchSingleTop = true
}
},
// Empty-chat "needs connection" card also offers the offline
// demo, so a skipped / never-connected first run can explore
// without leaving Chat. Safe here — this state only shows when
// nothing is configured, so there's no placeholder in flight.
onTryDemo = enterDemo,
onNavigateToDashboardSignIn = {
navController.navigate(Screen.DashboardSignIn.route()) {
launchSingleTop = true
}
},
onNavigateToBridge = {
rememberBridgeReturn(
route = Screen.Chat.route(openAgentSheet = false),
label = screenChatLabel,
)
navController.navigate(Screen.Bridge.route) {
popUpTo(navController.graph.findStartDestination().id) {
saveState = true
ChatMediaViewerHost(
activeConnectionId,
effectiveSessionProfileName,
currentChatSessionId,
chatSupervisedPolicy,
) {
ChatScreen(
chatViewModel = chatViewModel,
connectionViewModel = connectionViewModel,
voiceViewModel = voiceViewModel,
voiceClient = voiceClient,
maxBubbleWidth = maxBubbleWidth,
voicePresentationMode = voicePresentationMode,
onVoicePresentationModeChange = { mode ->
connectionSwitchScope.launch {
voicePreferences.setPresentationMode(mode)
}
launchSingleTop = true
restoreState = true
}
},
onNavigateToTerminal = {
navController.navigate(Screen.Terminal.route) {
launchSingleTop = true
}
},
onNavigateToSettings = {
navController.navigate(Screen.Settings.route) {
launchSingleTop = true
}
},
onNavigateToAppearanceSettings = {
navController.navigate(Screen.AppearanceSettings.route) {
launchSingleTop = true
}
},
onNavigateToVoiceSettings = {
navController.navigate(Screen.VoiceSettings.route) {
launchSingleTop = true
}
},
onNavigateToProfileInspector = { profileName ->
navController.navigate(Screen.ProfileInspector.route(profileName)) {
launchSingleTop = true
}
},
supervisedPolicy = chatSupervisedPolicy,
onNavigateToBotMode = {
navController.navigate(Screen.BotMode.route) { launchSingleTop = true }
},
gitWorkspaceAvailable = gitWorkspaceAvailable,
gitWorkspaceSummary = gitWorkspaceSummary,
onNavigateToGitWorkspace = {
navController.navigate(Screen.GitState.route) { launchSingleTop = true }
},
)
},
openAgentSheetOnEntry = openAgentSheetArg,
onAgentSheetArgConsumed = {
backStackEntry.arguments?.putBoolean(
Screen.Chat.ARG_OPEN_AGENT_SHEET, false,
)
},
// AgentInfoSheet footer jumps straight into the full
// Connections CRUD screen — saves a detour through
// Settings → Gateways.
onNavigateToConnections = {
navController.navigate(Screen.ConnectionsSettings.route)
},
onNavigateToConnect = {
navController.navigate(Screen.Pair.route()) {
launchSingleTop = true
}
},
onRepairConnection = {
navController.navigate(
Screen.Pair.route(
connectionId = activeConnectionId,
autoStart = "relay",
),
) {
launchSingleTop = true
}
},
// Empty-chat "needs connection" card also offers the offline
// demo, so a skipped / never-connected first run can explore
// without leaving Chat. Safe here — this state only shows when
// nothing is configured, so there's no placeholder in flight.
onTryDemo = enterDemo,
onNavigateToDashboardSignIn = {
navController.navigate(Screen.DashboardSignIn.route()) {
launchSingleTop = true
}
},
onNavigateToBridge = {
rememberBridgeReturn(
route = Screen.Chat.route(openAgentSheet = false),
label = screenChatLabel,
)
navController.navigate(Screen.Bridge.route) {
popUpTo(navController.graph.findStartDestination().id) {
saveState = true
}
launchSingleTop = true
restoreState = true
}
},
onNavigateToTerminal = {
navController.navigate(Screen.Terminal.route) {
launchSingleTop = true
}
},
onNavigateToSettings = {
navController.navigate(Screen.Settings.route) {
launchSingleTop = true
}
},
onNavigateToAppearanceSettings = {
navController.navigate(Screen.AppearanceSettings.route) {
launchSingleTop = true
}
},
onNavigateToVoiceSettings = {
navController.navigate(Screen.VoiceSettings.route) {
launchSingleTop = true
}
},
onNavigateToProfileInspector = { profileName ->
navController.navigate(Screen.ProfileInspector.route(profileName)) {
launchSingleTop = true
}
},
supervisedPolicy = chatSupervisedPolicy,
onNavigateToBotMode = {
navController.navigate(Screen.BotMode.route) { launchSingleTop = true }
},
gitWorkspaceAvailable = gitWorkspaceAvailable,
gitWorkspaceSummary = gitWorkspaceSummary,
onNavigateToGitWorkspace = {
navController.navigate(Screen.GitState.route) { launchSingleTop = true }
},
)
}
}
composable(Screen.BotMode.route) {
BotModeScreen(
@@ -31,6 +31,7 @@ import androidx.compose.runtime.mutableStateMapOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
@@ -128,10 +129,11 @@ fun AttachmentGallery(
val exportAllowed = LocalImageExportAllowed.current
val scope = rememberCoroutineScope()
val blurMode = LocalMediaBlurMode.current
val viewerController = LocalChatMediaViewerController.current
val revealed = remember { mutableStateMapOf<String, Boolean>() }
var viewerStartIndex by remember { mutableStateOf<Int?>(null) }
var viewerStartIndex by rememberSaveable { mutableStateOf<Int?>(null) }
viewerStartIndex?.let { startIndex ->
viewerStartIndex?.takeIf { viewerController == null }?.let { startIndex ->
AttachmentGalleryViewer(
attachments = attachments,
initialIndex = startIndex.coerceIn(attachments.indices),
@@ -184,7 +186,21 @@ fun AttachmentGallery(
.testTag("attachment-gallery-tile-$galleryIndex")
.clip(RoundedCornerShape(GALLERY_CORNER))
.combinedClickable(
onClick = { viewerStartIndex = galleryIndex },
onClick = {
if (viewerController != null) {
viewerController.openGallery(
attachments = attachments,
initialIndex = galleryIndex,
initiallyRevealedKeys = revealed
.filterValues { it }
.keys,
blurMode = blurMode,
exportAllowed = exportAllowed,
)
} else {
viewerStartIndex = galleryIndex
}
},
onLongClick = { menuExpanded = true },
),
)
@@ -9,7 +9,6 @@ import android.media.audiofx.Visualizer
import android.net.Uri
import android.os.Build
import android.os.ParcelFileDescriptor
import android.view.SurfaceView
import android.widget.Toast
import androidx.annotation.OptIn
import androidx.compose.foundation.Image
@@ -63,10 +62,12 @@ import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableLongStateOf
import androidx.compose.runtime.mutableStateMapOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.runtime.staticCompositionLocalOf
import androidx.compose.ui.Alignment
@@ -86,13 +87,13 @@ import androidx.compose.ui.platform.testTag
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.IntSize
import androidx.compose.ui.viewinterop.AndroidView
import androidx.compose.ui.window.Dialog
import androidx.compose.ui.window.DialogProperties
import androidx.media3.common.MediaItem
import androidx.media3.common.Player
import androidx.media3.common.util.UnstableApi
import androidx.media3.exoplayer.ExoPlayer
import androidx.media3.ui.compose.ContentFrame
import coil3.compose.AsyncImage
import com.hermesandroid.relay.data.Attachment
import com.hermesandroid.relay.data.AttachmentRenderMode
@@ -290,7 +291,7 @@ fun Modifier.zoomable(maxScale: Float = 6f): Modifier {
* Dispatches on [Attachment.renderMode]:
* - IMAGE → zoomable image (Coil from the cached URI, or a decoded bitmap),
* honoring the sensitive blur gate.
* - VIDEO → ExoPlayer on a hand-wired [SurfaceView] with transport controls.
* - VIDEO → ExoPlayer through Media3's lifecycle-aware Compose content frame.
* - AUDIO → ExoPlayer mini-player with scrubber + a Visualizer amplitude meter.
* - PDF → [PdfRenderer] paginated pages in a LazyColumn.
* - TEXT → in-app monospace text viewer.
@@ -384,7 +385,8 @@ fun AttachmentViewer(
Box(
modifier = modifier
.fillMaxSize()
.background(Color.Black.copy(alpha = 0.96f)),
.background(Color.Black.copy(alpha = 0.96f))
.testTag("attachment-viewer"),
) {
// Body fills; toolbar floats on top. PDF/TEXT add their own top
// inset so the first line clears the toolbar.
@@ -737,17 +739,27 @@ private fun VideoBody(attachment: Attachment) {
return
}
var savedPosition by rememberSaveable(uri.toString()) { mutableLongStateOf(0L) }
var wantsToPlay by rememberSaveable(uri.toString()) { mutableStateOf(true) }
var muted by rememberSaveable(uri.toString()) { mutableStateOf(false) }
val player = remember(uri) {
ExoPlayer.Builder(context).build().apply {
setMediaItem(MediaItem.fromUri(uri))
if (savedPosition > 0L) seekTo(savedPosition)
playWhenReady = wantsToPlay
volume = if (muted) 0f else 1f
prepare()
playWhenReady = true
}
}
DisposableEffect(player) { onDispose { player.release() } }
DisposableEffect(player) {
onDispose {
savedPosition = player.currentPosition.coerceAtLeast(0L)
wantsToPlay = player.playWhenReady
player.release()
}
}
var isPlaying by remember { mutableStateOf(true) }
var muted by remember { mutableStateOf(false) }
var isPlaying by remember { mutableStateOf(player.isPlaying) }
var position by remember { mutableStateOf(0L) }
var duration by remember { mutableStateOf(0L) }
@@ -761,24 +773,30 @@ private fun VideoBody(attachment: Attachment) {
LaunchedEffect(player) {
while (true) {
position = player.currentPosition.coerceAtLeast(0L)
savedPosition = position
duration = player.duration.takeIf { it > 0L } ?: 0L
delay(250)
}
}
Column(modifier = Modifier.fillMaxSize(), verticalArrangement = Arrangement.Center) {
AndroidView(
factory = { ctx ->
SurfaceView(ctx).also { player.setVideoSurfaceView(it) }
},
modifier = Modifier.fillMaxWidth().weight(1f, fill = false).aspectRatio(16f / 9f),
)
Box(
modifier = Modifier.fillMaxWidth().weight(1f),
contentAlignment = Alignment.Center,
) {
ContentFrame(
player = player,
modifier = Modifier.fillMaxSize().testTag("attachment-video-content"),
contentScale = ContentScale.Fit,
)
}
PlaybackControls(
isPlaying = isPlaying,
position = position,
duration = duration,
onPlayPause = {
if (player.isPlaying) player.pause() else player.play()
wantsToPlay = !player.playWhenReady
player.playWhenReady = wantsToPlay
},
onSeek = { player.seekTo(it) },
trailing = {
@@ -0,0 +1,215 @@
package com.hermesandroid.relay.ui.components
import androidx.activity.compose.BackHandler
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.core.tween
import androidx.compose.animation.expandVertically
import androidx.compose.animation.shrinkVertically
import androidx.compose.animation.fadeIn
import androidx.compose.animation.fadeOut
import androidx.compose.animation.slideInVertically
import androidx.compose.animation.slideOutVertically
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.selection.selectable
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Close
import androidx.compose.material.icons.filled.Check
import androidx.compose.material.icons.filled.Stop
import androidx.compose.material.icons.filled.ExpandLess
import androidx.compose.material.icons.filled.ExpandMore
import androidx.compose.material3.Icon
import androidx.compose.material3.IconButton
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.setValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.testTag
import androidx.compose.ui.res.pluralStringResource
import androidx.compose.ui.res.stringResource
import androidx.compose.ui.semantics.Role
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.Dp
import androidx.compose.ui.zIndex
import com.hermesandroid.relay.ui.theme.appearanceTopRoundedCornerShape
import com.hermesandroid.relay.R
import com.hermesandroid.relay.data.BusyMessageAction
private val COMPOSER_TRAY_UNDERLAP = 12.dp
/** A separate rear surface tucked beneath the composer's foreground edge. */
@Composable
fun ChatComposerLayers(
action: BusyMessageAction?,
onActionChange: (BusyMessageAction) -> Unit,
correctionAvailable: Boolean,
onStop: (() -> Unit)?,
modifier: Modifier = Modifier,
content: @Composable () -> Unit,
) {
Column(
modifier = modifier.fillMaxWidth(),
verticalArrangement = Arrangement.spacedBy(-COMPOSER_TRAY_UNDERLAP),
) {
if (action != null) {
ChatBusyActionSelector(
action, onActionChange,
correctionAvailable = correctionAvailable,
onStop = onStop,
bottomUnderlap = COMPOSER_TRAY_UNDERLAP,
modifier = Modifier.fillMaxWidth().padding(horizontal = 20.dp).zIndex(0f),
)
}
Box(Modifier.fillMaxWidth().zIndex(1f).testTag("chatComposerForeground")) { content() }
}
}
/** Compact current intent; optional plain-text choices slide out from behind it. */
@Composable
fun ChatBusyActionSelector(
action: BusyMessageAction,
onActionChange: (BusyMessageAction) -> Unit,
modifier: Modifier = Modifier,
correctionAvailable: Boolean = true,
onStop: (() -> Unit)? = null,
bottomUnderlap: Dp = 0.dp,
) {
var expanded by remember { mutableStateOf(false) }
BackHandler(enabled = expanded) { expanded = false }
Surface(
modifier = modifier.testTag("chatBusyActionTray"),
color = MaterialTheme.colorScheme.surfaceContainerLow.copy(alpha = 0.82f),
shape = appearanceTopRoundedCornerShape(12.dp),
tonalElevation = 0.dp,
) {
Column(Modifier.padding(bottom = bottomUnderlap)) {
AnimatedVisibility(
visible = expanded,
enter = expandVertically(tween(220), expandFrom = Alignment.Bottom) +
slideInVertically(tween(220)) { it } + fadeIn(tween(160)),
exit = shrinkVertically(tween(180), shrinkTowards = Alignment.Bottom) +
slideOutVertically(tween(180)) { it } + fadeOut(tween(120)),
) {
Row(
Modifier.fillMaxWidth().testTag("chatBusyActionDrawer").padding(horizontal = 8.dp),
verticalAlignment = Alignment.CenterVertically,
) {
BusyMessageAction.entries.forEach { option ->
val available = option == BusyMessageAction.QueueNext || correctionAvailable
Row(
Modifier.weight(1f).heightIn(min = 48.dp)
.testTag("chatBusyAction-${option.storedValue}")
.selectable(selected = option == action, enabled = available, role = Role.RadioButton) {
onActionChange(option)
expanded = false
}.padding(horizontal = 8.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(6.dp),
) {
if (option == action) {
Icon(Icons.Default.Check, null, Modifier.size(14.dp),
tint = MaterialTheme.colorScheme.onSurface)
} else Spacer(Modifier.size(14.dp))
Text(
stringResource(if (option == BusyMessageAction.CorrectNow) R.string.chat_correct_now else R.string.chat_queue_next),
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onSurface.copy(alpha = if (available) 1f else 0.38f),
)
}
}
}
}
Row(
Modifier.fillMaxWidth().padding(horizontal = 12.dp),
verticalAlignment = Alignment.CenterVertically,
) {
Row(
Modifier.weight(1f).heightIn(min = 40.dp)
.testTag("chatBusyActionTrigger")
.clickable(role = Role.Button) { expanded = !expanded },
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(4.dp),
) {
Text(
stringResource(if (action == BusyMessageAction.CorrectNow) R.string.chat_correct_now else R.string.chat_queue_next),
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Icon(
if (expanded) Icons.Default.ExpandMore else Icons.Default.ExpandLess,
null, Modifier.size(14.dp), tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
if (onStop != null) {
IconButton(onClick = onStop) {
Icon(Icons.Default.Stop, stringResource(R.string.chat_input_stop_streaming), Modifier.size(18.dp))
}
}
}
}
}
}
@Composable
fun ChatMessageQueue(
messages: List<String>,
paused: Boolean,
onResume: () -> Unit,
onClear: () -> Unit,
onEdit: (Int) -> Unit,
onRemove: (Int) -> Unit,
canEdit: Boolean,
modifier: Modifier = Modifier,
) {
if (messages.isEmpty()) return
Column(modifier) {
Row(Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Text(
if (paused) stringResource(R.string.chat_queue_paused)
else pluralStringResource(R.plurals.chat_queue_count, messages.size, messages.size),
style = MaterialTheme.typography.labelMedium,
modifier = Modifier.weight(1f),
)
if (paused) TextButton(onClick = onResume) { Text(stringResource(R.string.chat_queue_resume)) }
TextButton(onClick = onClear) { Text(stringResource(R.string.chat_clear)) }
}
Column(
Modifier.heightIn(max = 144.dp).verticalScroll(rememberScrollState()),
verticalArrangement = Arrangement.spacedBy(2.dp),
) {
messages.forEachIndexed { index, text ->
Row(Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
Text(
text = text,
maxLines = 2,
overflow = androidx.compose.ui.text.style.TextOverflow.Ellipsis,
style = MaterialTheme.typography.bodySmall,
modifier = Modifier.weight(1f)
.clickable(enabled = canEdit, role = Role.Button) { onEdit(index) }
.padding(vertical = 12.dp),
)
IconButton(onClick = { onRemove(index) }) {
Icon(Icons.Default.Close, stringResource(R.string.chat_queue_remove))
}
}
}
}
}
}
@@ -53,7 +53,13 @@ fun ChatFailurePanel(
Spacer(Modifier.width(10.dp))
Column(modifier = Modifier.weight(1f)) {
Text(
text = stringResource(R.string.chat_failure_title),
text = stringResource(
if (isInferenceUnavailableFailure(failure.rawError)) {
R.string.chat_failure_inference_unavailable
} else {
R.string.chat_failure_title
},
),
style = MaterialTheme.typography.bodyMedium,
fontWeight = FontWeight.SemiBold,
)
@@ -148,3 +154,10 @@ internal fun failureIdentity(route: String, model: String?, provider: String?):
provider?.trim()?.takeIf { it.isNotEmpty() },
model?.trim()?.takeIf { it.isNotEmpty() },
).distinct().joinToString(" · ")
internal fun isInferenceUnavailableFailure(rawError: String): Boolean {
val message = rawError.lowercase()
return "agent init failed" in message ||
"no codex credentials stored" in message ||
"runtime resolution still failed" in message
}
@@ -2,6 +2,7 @@ package com.hermesandroid.relay.ui.components
import android.content.Intent
import android.graphics.BitmapFactory
import android.net.Uri
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
@@ -29,6 +30,7 @@ import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.runtime.staticCompositionLocalOf
import androidx.compose.ui.Alignment
@@ -97,9 +99,9 @@ private fun isSensitiveAltText(alt: String): Boolean {
/**
* Resolves a server-local image path — an absolute path the agent put in a
* markdown image `![alt](/abs/path)` — to raw bytes, via the relay's
* `/media/by-path` route when a relay session is paired. Returns null when no
* relay is available or the fetch fails, so the renderer falls back to the
* markdown image `![alt](/abs/path)` — to an on-disk cache URI, via the relay's
* authenticated Dashboard route (or Relay compatibility route). Returns a
* failure when no route is available or the fetch fails, so the renderer falls back to the
* "this image is on the server" notice. Provided by ChatScreen from
* [com.hermesandroid.relay.viewmodel.ChatViewModel.resolveServerImage]; the
* default is null, which preserves the standard (no-plugin) behavior where a
@@ -113,12 +115,12 @@ private fun isSensitiveAltText(alt: String): Boolean {
* `RelayServerImage` on app open with a server-local image in history).
*/
sealed interface ServerImageResult {
class Success(val bytes: ByteArray, val sensitive: Boolean = false) : ServerImageResult
class Success(val cachedUri: String, val sensitive: Boolean = false) : ServerImageResult
class Failure(val reason: String) : ServerImageResult
}
fun interface RelayServerImageResolver {
/** Fetch the server-local file's bytes over the relay, or a
/** Fetch the server-local file into the media cache, or a
* [ServerImageResult.Failure] whose reason explains why (unpaired /
* sandboxed / missing / decode) so the UI can surface it instead of a
* generic placeholder. */
@@ -293,8 +295,10 @@ private sealed interface DataUrlImagePhase {
@Composable
private fun DataUrlChatImage(image: ChatInlineImage, maxWidth: Dp) {
var phase by remember(image.src) { mutableStateOf<DataUrlImagePhase>(DataUrlImagePhase.Loading) }
var viewerOpen by remember(image.src) { mutableStateOf(false) }
var viewerOpen by rememberSaveable(image.src) { mutableStateOf(false) }
val blurMode = LocalMediaBlurMode.current
val exportAllowed = LocalImageExportAllowed.current
val viewerController = LocalChatMediaViewerController.current
var revealed by remember(image.src) { mutableStateOf(false) }
LaunchedEffect(image.src) {
phase = withInlineImageDecodeLock {
@@ -329,17 +333,17 @@ private fun DataUrlChatImage(image: ChatInlineImage, maxWidth: Dp) {
reason = "Unsupported or oversized inline image.",
)
is DataUrlImagePhase.Loaded -> {
if (viewerOpen) {
val viewerSource = ChatImageViewerSource.Bitmap(
bitmap = current.bitmap,
displayName = image.alt.ifBlank { "image" },
mime = current.mime,
// Decode the already-retained data URL only when the user
// requests Save/Share; don't retain a second byte array.
bytesProvider = { decodeInlineImageDataUrlOffMain(image.src)?.bytes },
)
if (viewerController == null && viewerOpen) {
ChatImageViewer(
source = ChatImageViewerSource.Bitmap(
bitmap = current.bitmap,
displayName = image.alt.ifBlank { "image" },
mime = current.mime,
// Decode the already-retained data URL only when the
// user requests Save/Share; don't keep a second 5 MiB
// byte array beside every thumbnail.
bytesProvider = { decodeInlineImageDataUrlOffMain(image.src)?.bytes },
),
source = viewerSource,
onDismiss = { viewerOpen = false },
sensitive = image.sensitive,
initiallyRevealed = revealed,
@@ -358,7 +362,19 @@ private fun DataUrlChatImage(image: ChatInlineImage, maxWidth: Dp) {
.widthIn(max = maxWidth)
.heightIn(max = 360.dp)
.clip(RoundedCornerShape(12.dp))
.clickable { viewerOpen = true },
.clickable {
if (viewerController != null) {
viewerController.openImage(
source = viewerSource,
sensitive = image.sensitive,
initiallyRevealed = revealed,
blurMode = blurMode,
exportAllowed = exportAllowed,
)
} else {
viewerOpen = true
}
},
)
}
}
@@ -368,18 +384,21 @@ private fun DataUrlChatImage(image: ChatInlineImage, maxWidth: Dp) {
@Composable
private fun RemoteChatImage(image: ChatInlineImage, maxWidth: Dp) {
var viewerOpen by remember { mutableStateOf(false) }
var viewerOpen by rememberSaveable(image.src) { mutableStateOf(false) }
val blurMode = LocalMediaBlurMode.current
val exportAllowed = LocalImageExportAllowed.current
val viewerController = LocalChatMediaViewerController.current
var revealed by remember(image.src) { mutableStateOf(false) }
val blurred = !revealed && shouldBlurImage(blurMode, image.sensitive)
if (viewerOpen) {
val viewerSource = ChatImageViewerSource.Coil(
model = image.src,
displayName = image.alt.ifBlank { "image" },
mime = "image/*",
bytesProvider = { MediaSaver.fetchRemoteBytes(image.src).first },
)
if (viewerController == null && viewerOpen) {
ChatImageViewer(
source = ChatImageViewerSource.Coil(
model = image.src,
displayName = image.alt.ifBlank { "image" },
mime = "image/*",
bytesProvider = { MediaSaver.fetchRemoteBytes(image.src).first },
),
source = viewerSource,
onDismiss = { viewerOpen = false },
sensitive = image.sensitive,
initiallyRevealed = revealed,
@@ -395,7 +414,19 @@ private fun RemoteChatImage(image: ChatInlineImage, maxWidth: Dp) {
.widthIn(max = maxWidth)
.heightIn(max = 360.dp)
.clip(RoundedCornerShape(12.dp))
.clickable { viewerOpen = true },
.clickable {
if (viewerController != null) {
viewerController.openImage(
source = viewerSource,
sensitive = image.sensitive,
initiallyRevealed = revealed,
blurMode = blurMode,
exportAllowed = exportAllowed,
)
} else {
viewerOpen = true
}
},
) {
val state by painter.state.collectAsState()
when (state) {
@@ -461,6 +492,7 @@ private fun RelayServerImage(
maxWidth: Dp,
resolver: RelayServerImageResolver,
) {
val context = LocalContext.current
var phase by remember(image.src) {
mutableStateOf<RelayImagePhase>(
cachedInlineImage(image.src)
@@ -473,20 +505,22 @@ private fun RelayServerImage(
phase = withContext(Dispatchers.IO) {
val result = try {
resolver.fetch(image.src)
} catch (t: Throwable) {
ServerImageResult.Failure(t.message ?: "relay fetch failed")
} catch (error: Exception) {
ServerImageResult.Failure(error.message ?: "relay fetch failed")
}
when (result) {
is ServerImageResult.Success -> {
val bytes = result.bytes
val bmp = runCatching {
decodeOrientedBitmap(bytes)
}.getOrNull()?.asImageBitmap()
val cachedUri = Uri.parse(result.cachedUri)
val bmp = try {
decodeBoundedOrientedBitmap(context, cachedUri)
} catch (_: Exception) {
null
}?.asImageBitmap()
if (bmp != null) {
putInlineImage(image.src, bmp, result.sensitive)
RelayImagePhase.Loaded(bmp, result.sensitive)
} else {
RelayImagePhase.Failed("fetched ${bytes.size} B but couldn't decode the image")
RelayImagePhase.Failed("fetched file could not be decoded as an image")
}
}
is ServerImageResult.Failure -> RelayImagePhase.Failed(result.reason)
@@ -527,23 +561,32 @@ private fun RelayServerImageContent(
maxWidth: Dp,
resolver: RelayServerImageResolver,
) {
var viewerOpen by remember { mutableStateOf(false) }
val context = LocalContext.current
var viewerOpen by rememberSaveable(image.src) { mutableStateOf(false) }
val blurMode = LocalMediaBlurMode.current
val exportAllowed = LocalImageExportAllowed.current
val viewerController = LocalChatMediaViewerController.current
var revealed by remember(image.src) { mutableStateOf(false) }
val sensitive = image.sensitive || fetchedSensitive
val blurred = !revealed && shouldBlurImage(blurMode, sensitive)
if (viewerOpen) {
val viewerSource = ChatImageViewerSource.Bitmap(
bitmap = bitmap,
displayName = image.alt.ifBlank {
image.src.substringAfterLast('/').ifBlank { "image" }
},
mime = "image/*",
// Save/Share re-fetch the original bytes on demand so we don't hold
// them in memory next to the decoded bitmap.
bytesProvider = {
(resolver.fetch(image.src) as? ServerImageResult.Success)?.let { fetched ->
context.contentResolver.openInputStream(Uri.parse(fetched.cachedUri))
?.use { it.readBytes() }
}
},
)
if (viewerController == null && viewerOpen) {
ChatImageViewer(
source = ChatImageViewerSource.Bitmap(
bitmap = bitmap,
displayName = image.alt.ifBlank {
image.src.substringAfterLast('/').ifBlank { "image" }
},
mime = "image/*",
// Save/Share re-fetch the original bytes on demand so we don't
// hold them in memory next to the decoded bitmap.
bytesProvider = { (resolver.fetch(image.src) as? ServerImageResult.Success)?.bytes },
),
source = viewerSource,
onDismiss = { viewerOpen = false },
sensitive = sensitive,
initiallyRevealed = revealed,
@@ -559,7 +602,19 @@ private fun RelayServerImageContent(
.widthIn(max = maxWidth)
.heightIn(max = 360.dp)
.clip(RoundedCornerShape(12.dp))
.clickable { viewerOpen = true },
.clickable {
if (viewerController != null) {
viewerController.openImage(
source = viewerSource,
sensitive = sensitive,
initiallyRevealed = revealed,
blurMode = blurMode,
exportAllowed = exportAllowed,
)
} else {
viewerOpen = true
}
},
)
}
}
@@ -170,7 +170,11 @@ fun ChatImageViewer(
onClick = {
scope.launch {
busy = true
val bytes = runCatching { source.bytesProvider() }.getOrNull()
val bytes = try {
source.bytesProvider()
} catch (_: Exception) {
null
}
busy = false
if (bytes == null) {
toast(context, errorMsg)
@@ -195,7 +199,11 @@ fun ChatImageViewer(
onClick = {
scope.launch {
busy = true
val bytes = runCatching { source.bytesProvider() }.getOrNull()
val bytes = try {
source.bytesProvider()
} catch (_: Exception) {
null
}
if (bytes == null) {
busy = false
toast(context, errorMsg)
@@ -1,5 +1,6 @@
package com.hermesandroid.relay.ui.components
import com.hermesandroid.relay.data.BusyMessageAction
import androidx.compose.animation.AnimatedContent
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.core.tween
@@ -191,6 +192,9 @@ fun ChatInputBar(
physicalEnterSends: Boolean = true,
largePasteThreshold: Int? = null,
onLargePaste: (String) -> Unit = {},
busyAction: BusyMessageAction? = null,
correctionAvailable: Boolean = true,
onBusyActionChange: (BusyMessageAction) -> Unit = {},
) {
val canSubmit = enabled && submitEnabled && trailing in setOf(
ChatInputTrailing.SEND,
@@ -236,7 +240,7 @@ fun ChatInputBar(
// Correction and queueing are materially different actions. Keep the
// explanation, but lead with a visible state pill so the distinction
// does not depend on the trailing icon or accent color alone.
AnimatedVisibility(visible = caption != null) {
AnimatedVisibility(visible = caption != null && busyAction == null) {
val detail = caption ?: lastCaption.orEmpty()
val actionLabel = when (trailing) {
ChatInputTrailing.STEER -> stringResource(R.string.chat_input_steer_response)
@@ -319,6 +323,12 @@ fun ChatInputBar(
)
}
ChatComposerLayers(
action = busyAction,
onActionChange = onBusyActionChange,
correctionAvailable = correctionAvailable,
onStop = onStop.takeUnless { trailing == ChatInputTrailing.STOP },
) {
Surface(
shape = appearanceComposerShape(),
color = MaterialTheme.colorScheme.surfaceContainerHigh,
@@ -646,6 +656,7 @@ fun ChatInputBar(
}
}
}
}
internal data class LargeTextInsertion(
val insertedText: String,
@@ -0,0 +1,146 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.runtime.Composable
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.runtime.Stable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.runtime.staticCompositionLocalOf
import com.hermesandroid.relay.data.Attachment
import com.hermesandroid.relay.data.BlurMode
private sealed interface ChatMediaViewerRequest {
val blurMode: BlurMode
val exportAllowed: Boolean
data class SingleAttachment(
val attachment: Attachment,
val initiallyRevealed: Boolean,
override val blurMode: BlurMode,
override val exportAllowed: Boolean,
) : ChatMediaViewerRequest
data class AttachmentGallery(
val attachments: List<Attachment>,
val initialIndex: Int,
val initiallyRevealedKeys: Set<String>,
override val blurMode: BlurMode,
override val exportAllowed: Boolean,
) : ChatMediaViewerRequest
data class InlineImage(
val source: ChatImageViewerSource,
val sensitive: Boolean,
val initiallyRevealed: Boolean,
override val blurMode: BlurMode,
override val exportAllowed: Boolean,
) : ChatMediaViewerRequest
}
@Stable
internal class ChatMediaViewerController {
private var activeRequest by mutableStateOf<ChatMediaViewerRequest?>(null)
internal fun openAttachment(
attachment: Attachment,
initiallyRevealed: Boolean,
blurMode: BlurMode,
exportAllowed: Boolean,
) {
activeRequest = ChatMediaViewerRequest.SingleAttachment(
attachment = attachment,
initiallyRevealed = initiallyRevealed,
blurMode = blurMode,
exportAllowed = exportAllowed,
)
}
internal fun openGallery(
attachments: List<Attachment>,
initialIndex: Int,
initiallyRevealedKeys: Set<String>,
blurMode: BlurMode,
exportAllowed: Boolean,
) {
activeRequest = ChatMediaViewerRequest.AttachmentGallery(
attachments = attachments,
initialIndex = initialIndex,
initiallyRevealedKeys = initiallyRevealedKeys,
blurMode = blurMode,
exportAllowed = exportAllowed,
)
}
internal fun openImage(
source: ChatImageViewerSource,
sensitive: Boolean,
initiallyRevealed: Boolean,
blurMode: BlurMode,
exportAllowed: Boolean,
) {
activeRequest = ChatMediaViewerRequest.InlineImage(
source = source,
sensitive = sensitive,
initiallyRevealed = initiallyRevealed,
blurMode = blurMode,
exportAllowed = exportAllowed,
)
}
internal fun dismiss() {
activeRequest = null
}
@Composable
internal fun RenderActiveViewer() {
val request = activeRequest ?: return
CompositionLocalProvider(
LocalMediaBlurMode provides request.blurMode,
LocalImageExportAllowed provides request.exportAllowed,
) {
when (request) {
is ChatMediaViewerRequest.SingleAttachment -> AttachmentViewer(
attachment = request.attachment,
onDismiss = ::dismiss,
initiallyRevealed = request.initiallyRevealed,
)
is ChatMediaViewerRequest.AttachmentGallery -> AttachmentGalleryViewer(
attachments = request.attachments,
initialIndex = request.initialIndex,
onDismiss = ::dismiss,
initiallyRevealedKeys = request.initiallyRevealedKeys,
)
is ChatMediaViewerRequest.InlineImage -> ChatImageViewer(
source = request.source,
onDismiss = ::dismiss,
sensitive = request.sensitive,
initiallyRevealed = request.initiallyRevealed,
)
}
}
}
}
internal val LocalChatMediaViewerController =
staticCompositionLocalOf<ChatMediaViewerController?> { null }
/**
* Owns the active full-screen preview above transcript rows so responsive
* portrait/landscape reflow cannot dispose the viewer with its source bubble.
*/
@Composable
internal fun ChatMediaViewerHost(
vararg ownerKeys: Any?,
content: @Composable () -> Unit,
) {
// A preview never follows a connection/session/policy owner change. Aside
// from avoiding stale media, this makes export permission fail closed when
// supervised policy changes while a viewer is open.
val controller = remember(*ownerKeys) { ChatMediaViewerController() }
CompositionLocalProvider(LocalChatMediaViewerController provides controller) {
content()
controller.RenderActiveViewer()
}
}
@@ -73,7 +73,7 @@ private const val SWIPE_DISMISS_THRESHOLD_PX = 80f
* progress" moment (pairing, long upload, a bridge action sequence). When first
* reused, decouple it from [ConnectionStatusSnapshot] and rename to a generic
* `StatusToast`. It also anchors [UpdateAvailableBanner]'s visual language + the
* shared [ConnectionStepRow]/[StepGlyph] helpers. See TODO.md.
* shared [ConnectionStepRow]/[StepGlyph] helpers. See docs/project/TODO.md.
*
* Rendered as a top-aligned overlay inside a `Box` — it slides down OVER the UI
* without shifting layout. Pair it with `AnimatedVisibility(enter =
@@ -14,6 +14,7 @@ import androidx.compose.foundation.Image
import androidx.compose.foundation.clickable
import androidx.compose.foundation.combinedClickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.aspectRatio
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
@@ -44,6 +45,7 @@ import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.rememberCoroutineScope
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
@@ -302,17 +304,21 @@ private fun ImageRender(
}
LaunchedEffect(attachment.cachedUri, attachment.content) {
val decoded = withContext(Dispatchers.IO) {
runCatching {
val bytes: ByteArray? = when {
try {
when {
!attachment.cachedUri.isNullOrBlank() ->
context.contentResolver.openInputStream(Uri.parse(attachment.cachedUri))
?.use { it.readBytes() }
decodeBoundedOrientedBitmap(context, Uri.parse(attachment.cachedUri))
?.asImageBitmap()
attachment.content.isNotBlank() ->
android.util.Base64.decode(attachment.content, android.util.Base64.DEFAULT)
android.util.Base64.decode(
attachment.content,
android.util.Base64.DEFAULT,
).let { decodeBoundedOrientedBitmap(it)?.asImageBitmap() }
else -> null
}
bytes?.let { decodeOrientedBitmap(it)?.asImageBitmap() }
}.getOrNull()
} catch (_: Exception) {
null
}
}
if (decoded != null) bitmap = decoded else decodeFailed = true
}
@@ -329,9 +335,14 @@ private fun ImageRender(
}
val blurMode = LocalMediaBlurMode.current
val viewerController = LocalChatMediaViewerController.current
var revealed by remember(attachment.cachedUri, attachment.content) { mutableStateOf(false) }
val blurred = !revealed && shouldBlurImage(blurMode, attachment.sensitive)
var viewerOpen by remember { mutableStateOf(false) }
var viewerOpen by rememberSaveable(
attachment.cachedUri,
attachment.relayToken,
attachment.fileName,
) { mutableStateOf(false) }
var menuExpanded by remember { mutableStateOf(false) }
val bmp = bitmap
@@ -352,7 +363,7 @@ private fun ImageRender(
return
}
if (viewerOpen) {
if (viewerController == null && viewerOpen) {
AttachmentViewer(
attachment = attachment,
onDismiss = { viewerOpen = false },
@@ -367,12 +378,25 @@ private fun ImageRender(
contentDescription = attachment.fileName,
modifier = Modifier
.widthIn(max = maxWidth)
.fillMaxWidth()
.aspectRatio(bmp.width.toFloat() / bmp.height.coerceAtLeast(1))
.clip(RoundedCornerShape(8.dp))
.combinedClickable(
onClick = { viewerOpen = true },
onClick = {
if (viewerController != null) {
viewerController.openAttachment(
attachment = attachment,
initiallyRevealed = revealed,
blurMode = blurMode,
exportAllowed = exportAllowed,
)
} else {
viewerOpen = true
}
},
onLongClick = { menuExpanded = true },
),
contentScale = ContentScale.FillWidth,
contentScale = ContentScale.Fit,
)
}
// One-tap save overlay — hidden while the blur cover is up so it
@@ -406,15 +430,20 @@ private fun FileCardRender(
val scope = rememberCoroutineScope()
val (emoji, typeLabel) = emojiAndLabelFor(attachment.renderMode, attachment.contentType)
var menuExpanded by remember { mutableStateOf(false) }
var viewerOpen by remember { mutableStateOf(false) }
var viewerOpen by rememberSaveable(
attachment.cachedUri,
attachment.relayToken,
attachment.fileName,
) { mutableStateOf(false) }
// Real thumbnail when one is cheap (video first frame, PDF first page);
// null falls back to the type emoji.
val thumbnail = rememberAttachmentThumbnail(attachment)
val blurMode = LocalMediaBlurMode.current
val viewerController = LocalChatMediaViewerController.current
val blurThumb = thumbnail != null && shouldBlurThumb(blurMode, attachment)
if (viewerOpen) {
if (viewerController == null && viewerOpen) {
// Non-image types don't blur in the viewer; open straight through.
AttachmentViewer(
attachment = attachment,
@@ -430,7 +459,18 @@ private fun FileCardRender(
.widthIn(max = maxWidth)
// Tap previews in-app; long-press surfaces Open-externally / Share / Save.
.combinedClickable(
onClick = { viewerOpen = true },
onClick = {
if (viewerController != null) {
viewerController.openAttachment(
attachment = attachment,
initiallyRevealed = true,
blurMode = blurMode,
exportAllowed = exportAllowed,
)
} else {
viewerOpen = true
}
},
onLongClick = { menuExpanded = true },
)
) {
@@ -648,7 +688,11 @@ private fun rememberAttachmentThumbnail(attachment: Attachment): ImageBitmap? {
}
LaunchedEffect(attachment.cachedUri, attachment.content, attachment.renderMode) {
thumb = withContext(Dispatchers.IO) {
runCatching { generateThumbnail(context, attachment) }.getOrNull()
try {
generateThumbnail(context, attachment)
} catch (_: Exception) {
null
}
}
}
return thumb
@@ -46,7 +46,11 @@ internal fun decodeInlineImageDataUrl(
// ceil(maxBytes / 3) * 4 plus at most two trailing padding characters.
val maxEncoded = ((maxBytes.toLong() + 2L) / 3L) * 4L
if (encoded.length.toLong() > maxEncoded) return null
val bytes = runCatching { Base64.getDecoder().decode(encoded) }.getOrNull() ?: return null
val bytes = try {
Base64.getDecoder().decode(encoded)
} catch (_: IllegalArgumentException) {
return null
}
if (bytes.isEmpty() || bytes.size > maxBytes || !matchesImageMagic(mime, bytes)) return null
return DecodedInlineImageData(bytes, mime)
}
@@ -73,7 +73,7 @@ fun MarkdownContent(
modifier: Modifier = Modifier
) {
ConfiguredMarkdownContent(
content = content,
content = markdownRenderWindow(content),
textColor = textColor,
modifier = modifier,
)
@@ -318,7 +318,7 @@ fun StreamingMarkdownContent(
var generation by remember { mutableIntStateOf(0) }
key(generation) {
NativeStreamingMarkdownGeneration(
content = content.withoutLeadingBlankLines(),
content = markdownRenderWindow(content.withoutLeadingBlankLines()),
textColor = textColor,
modifier = modifier,
onResetRequired = { generation += 1 },
@@ -353,6 +353,17 @@ private fun NativeStreamingMarkdownGeneration(
)
}
internal const val MAX_RENDERED_MARKDOWN_CHARS = 256_000
private const val MARKDOWN_TRUNCATION_NOTICE =
"\n\n---\n_Response display was shortened for Android memory safety._"
internal fun markdownRenderWindow(content: String): String =
if (content.length <= MAX_RENDERED_MARKDOWN_CHARS) {
content
} else {
content.take(MAX_RENDERED_MARKDOWN_CHARS) + MARKDOWN_TRUNCATION_NOTICE
}
internal data class StreamingMarkdownAppendPlan(
val resetRequired: Boolean,
val delta: String,
@@ -856,13 +856,13 @@ fun MessageBubble(
}
}
// Delivery status — only on agent-Thread reply bubbles (a user
// message routed over the relay proactive channel). Null on every
// ordinary chat message, which render nothing here.
// Delivery status for local user messages, including steering.
// Use the bubble's foreground: accent-on-accent hides the label.
message.deliveryStatus?.takeIf { isUser }?.let { status ->
Spacer(modifier = Modifier.height(2.dp))
MessageDeliveryIndicator(
status = status,
contentColor = textColor,
text = MessageDeliveryIndicatorText(
sending = stringResource(R.string.msg_bubble_sending),
queued = stringResource(R.string.msg_bubble_queued),
@@ -18,6 +18,7 @@ import androidx.compose.material3.minimumInteractiveComponentSize
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.semantics.LiveRegionMode
import androidx.compose.ui.semantics.Role
@@ -57,6 +58,7 @@ fun MessageDeliveryIndicator(
modifier: Modifier = Modifier,
failureMessage: String? = null,
onRetry: (() -> Unit)? = null,
contentColor: Color? = null,
) {
val retryAction = onRetry.takeIf { status == MessageDeliveryStatus.FAILED }
val retryable = retryAction != null
@@ -67,7 +69,7 @@ fun MessageDeliveryIndicator(
retryable = retryable,
)
val presentation = messageDeliveryPresentation(status)
val tint = when (presentation.colorRole) {
val tint = contentColor ?: when (presentation.colorRole) {
DeliveryColorRole.NEUTRAL -> MaterialTheme.colorScheme.onSurfaceVariant
DeliveryColorRole.ACCENT -> MaterialTheme.colorScheme.primary
DeliveryColorRole.QUEUED -> MaterialTheme.colorScheme.tertiary
@@ -11,16 +11,15 @@ import androidx.compose.ui.platform.LocalContext
/**
* Temporarily allow the device to rotate (portrait or landscape, following the
* sensor) while this composable is in the composition, overriding the app-wide
* portrait lock declared in the manifest (`MainActivity` screenOrientation).
* user's rotation preference) while this composable is in the composition,
* overriding the app-wide portrait lock declared in the manifest
* (`MainActivity` screenOrientation).
* Restores the previous orientation (portrait) when it leaves the composition.
*
* Used by the full-screen media viewers ([ChatImageViewer], [AttachmentViewer])
* so wide images and video can be viewed in landscape while the rest of the app
* stays portrait-locked. Uses `SENSOR` (portrait + both landscapes, no
* upside-down) so the viewer rotates with the device regardless of the system
* auto-rotate toggle; swap to `SCREEN_ORIENTATION_USER` to instead respect that
* toggle.
* stays portrait-locked. Uses `USER` so the viewer follows the device only when
* the user has enabled rotation instead of overriding their system preference.
*/
@Composable
fun AllowDeviceRotation() {
@@ -28,7 +27,7 @@ fun AllowDeviceRotation() {
DisposableEffect(Unit) {
val activity = context.findActivity()
val previous = activity?.requestedOrientation
activity?.requestedOrientation = ActivityInfo.SCREEN_ORIENTATION_SENSOR
activity?.requestedOrientation = ActivityInfo.SCREEN_ORIENTATION_USER
onDispose {
activity?.requestedOrientation =
previous ?: ActivityInfo.SCREEN_ORIENTATION_PORTRAIT
@@ -1,8 +1,10 @@
package com.hermesandroid.relay.ui.components
import android.content.Context
import android.graphics.Bitmap
import android.graphics.BitmapFactory
import android.graphics.Matrix
import android.net.Uri
import androidx.exifinterface.media.ExifInterface
import java.io.ByteArrayInputStream
@@ -25,6 +27,11 @@ internal fun imageOrientationTransform(orientation: Int): ImageOrientationTransf
else -> ImageOrientationTransform()
}
internal fun shouldApplyExifOrientation(mimeType: String?): Boolean = when (mimeType?.lowercase()) {
"image/png", "image/gif", "image/bmp" -> false
else -> true
}
/**
* Decode image bytes for display and apply their EXIF orientation. BitmapFactory
* returns the stored pixel layout and otherwise leaves portrait phone photos
@@ -41,14 +48,16 @@ internal fun decodeOrientedBitmap(
BitmapFactory.decodeByteArray(bytes, 0, bytes.size, options)
} ?: return null
val orientation = runCatching {
val orientation = try {
ByteArrayInputStream(bytes).use { stream ->
ExifInterface(stream).getAttributeInt(
ExifInterface.TAG_ORIENTATION,
ExifInterface.ORIENTATION_NORMAL,
)
}
}.getOrDefault(ExifInterface.ORIENTATION_NORMAL)
} catch (_: Exception) {
ExifInterface.ORIENTATION_NORMAL
}
val transform = imageOrientationTransform(orientation)
if (transform.rotationDegrees == 0f && !transform.flipHorizontal) return decoded
@@ -56,9 +65,81 @@ internal fun decodeOrientedBitmap(
if (transform.rotationDegrees != 0f) postRotate(transform.rotationDegrees)
if (transform.flipHorizontal) postScale(-1f, 1f)
}
return runCatching {
val oriented = try {
Bitmap.createBitmap(decoded, 0, 0, decoded.width, decoded.height, matrix, true)
}.getOrNull()?.also { oriented ->
if (oriented !== decoded) decoded.recycle()
} catch (_: Exception) {
null
}
return oriented?.also { bitmap ->
if (bitmap !== decoded) decoded.recycle()
} ?: decoded
}
/** Decode a cached content URI without copying the encoded file into RAM. */
internal fun decodeOrientedBitmap(context: Context, uri: Uri): Bitmap? =
decodeBoundedOrientedBitmap(context, uri)
internal fun decodeBoundedOrientedBitmap(context: Context, uri: Uri): Bitmap? {
val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
val boundsInput = context.contentResolver.openInputStream(uri) ?: return null
boundsInput.use { input ->
// Bounds-only decode deliberately returns null; the populated Options
// are the result and must not be treated as a decode failure.
BitmapFactory.decodeStream(input, null, bounds)
}
val sample = inlineImageSampleSize(bounds.outWidth, bounds.outHeight) ?: return null
val decoded = context.contentResolver.openInputStream(uri)?.use { input ->
BitmapFactory.decodeStream(
input,
null,
BitmapFactory.Options().apply {
inSampleSize = sample
inPreferredConfig = Bitmap.Config.ARGB_8888
},
)
} ?: return null
val orientation = if (shouldApplyExifOrientation(bounds.outMimeType)) {
try {
context.contentResolver.openInputStream(uri)?.use { input ->
ExifInterface(input).getAttributeInt(
ExifInterface.TAG_ORIENTATION,
ExifInterface.ORIENTATION_NORMAL,
)
}
} catch (_: Exception) {
null
} ?: ExifInterface.ORIENTATION_NORMAL
} else {
ExifInterface.ORIENTATION_NORMAL
}
val transform = imageOrientationTransform(orientation)
if (transform.rotationDegrees == 0f && !transform.flipHorizontal) return decoded
val matrix = Matrix().apply {
if (transform.rotationDegrees != 0f) postRotate(transform.rotationDegrees)
if (transform.flipHorizontal) postScale(-1f, 1f)
}
val oriented = try {
Bitmap.createBitmap(decoded, 0, 0, decoded.width, decoded.height, matrix, true)
} catch (_: Exception) {
null
}
return oriented?.also { bitmap ->
if (bitmap !== decoded) decoded.recycle()
} ?: decoded
}
/** Decode an encoded chat image only after bounding its decoded pixel footprint. */
internal fun decodeBoundedOrientedBitmap(bytes: ByteArray): Bitmap? {
if (bytes.isEmpty()) return null
val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
BitmapFactory.decodeByteArray(bytes, 0, bytes.size, bounds)
val sample = inlineImageSampleSize(bounds.outWidth, bounds.outHeight) ?: return null
return decodeOrientedBitmap(
bytes,
BitmapFactory.Options().apply {
inSampleSize = sample
inPreferredConfig = Bitmap.Config.ARGB_8888
},
)
}
@@ -296,12 +296,12 @@ internal fun formatAttachmentSize(bytes: Long): String {
private suspend fun decodePendingAttachmentImage(attachment: Attachment): ImageBitmap? =
withContext(Dispatchers.IO) {
runCatching {
try {
val bytes = Base64.decode(attachment.content, Base64.DEFAULT)
if (bytes.isEmpty()) return@runCatching null
if (bytes.isEmpty()) return@withContext null
val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
BitmapFactory.decodeByteArray(bytes, 0, bytes.size, bounds)
if (bounds.outWidth <= 0 || bounds.outHeight <= 0) return@runCatching null
if (bounds.outWidth <= 0 || bounds.outHeight <= 0) return@withContext null
var sample = 1
while (
@@ -314,7 +314,9 @@ private suspend fun decodePendingAttachmentImage(attachment: Attachment): ImageB
bytes,
BitmapFactory.Options().apply { inSampleSize = sample },
)?.asImageBitmap()
}.getOrNull()
} catch (_: Exception) {
null
}
}
private sealed interface PendingPreviewState {
@@ -27,6 +27,7 @@ import androidx.compose.foundation.layout.fillMaxHeight
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.heightIn
import androidx.compose.foundation.layout.navigationBarsPadding
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
@@ -39,6 +40,9 @@ import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.lazy.rememberLazyListState
import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.text.selection.SelectionContainer
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.CenterFocusStrong
import androidx.compose.material.icons.filled.Close
@@ -48,10 +52,11 @@ import androidx.compose.material.icons.filled.ExpandMore
import androidx.compose.material.icons.filled.GraphicEq
import androidx.compose.material.icons.filled.Image
import androidx.compose.material.icons.filled.Mic
import androidx.compose.material.icons.filled.Refresh
import androidx.compose.material.icons.filled.ErrorOutline
import androidx.compose.material.icons.filled.Settings
import androidx.compose.material.icons.filled.Stop
import androidx.compose.material3.Icon
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.IconButton
import androidx.compose.material3.HorizontalDivider
import androidx.compose.material3.LinearProgressIndicator
@@ -77,6 +82,7 @@ import androidx.compose.ui.input.pointer.pointerInput
import androidx.compose.ui.layout.ContentScale
import androidx.compose.ui.platform.LocalConfiguration
import androidx.compose.ui.platform.LocalHapticFeedback
import androidx.compose.ui.platform.LocalSoftwareKeyboardController
import androidx.compose.ui.platform.testTag
import androidx.compose.ui.semantics.Role
import androidx.compose.ui.semantics.contentDescription
@@ -145,8 +151,7 @@ fun VoiceModeOverlay(
onModeChange: (InteractionMode) -> Unit,
onClearError: () -> Unit,
modifier: Modifier = Modifier,
// Nullable so existing call sites compile; when wired, voice errors
// surface as global snackbars in addition to the inline banner below.
// Retained for callers; uiState.error owns the modal while voice is visible.
errorEvents: SharedFlow<HumanError>? = null,
// === PHASE3-voice-mode-transcript ===
// Compact rolling transcript of the last N chat messages — the
@@ -255,7 +260,7 @@ fun VoiceModeOverlay(
)
}
// Voice errors surface ONLY on the overlay's own inline top banner
// Voice errors surface ONLY in the overlay's dialog
// (uiState.error) while the overlay is up — we deliberately do NOT also pipe
// them to the app-wide bottom snackbar. Doing both duplicated the message
// and left a retry-only, un-dismissable toast at the bottom during long /
@@ -510,59 +515,15 @@ fun VoiceModeOverlay(
)
}
// Error banner
AnimatedVisibility(
visible = uiState.error != null,
enter = fadeIn(),
exit = fadeOut(),
modifier = Modifier
.align(Alignment.TopCenter)
.then(
responsiveLayout.chromeMaxWidth?.let {
Modifier.widthIn(max = it)
} ?: Modifier,
)
.fillMaxWidth()
.padding(top = 64.dp, start = 16.dp, end = 16.dp),
) {
Surface(
shape = appearanceRoundedCornerShape(12.dp),
color = MaterialTheme.colorScheme.errorContainer,
tonalElevation = 2.dp,
) {
Row(
modifier = Modifier.padding(12.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
Text(
text = uiState.error.orEmpty(),
color = MaterialTheme.colorScheme.onErrorContainer,
style = MaterialTheme.typography.bodySmall,
modifier = Modifier.weight(1f),
)
// Dismiss clears the error and returns to Idle without
// retrying, so a failed/timed-out turn never traps the user
// on a retry-only banner.
TextButton(onClick = { onClearError() }) {
Text(stringResource(R.string.common_dismiss))
}
TextButton(
onClick = {
onClearError()
onMicTap()
},
) {
Icon(
imageVector = Icons.Filled.Refresh,
contentDescription = null,
modifier = Modifier.size(16.dp),
)
Spacer(Modifier.size(4.dp))
Text(stringResource(R.string.voice_overlay_retry))
}
}
}
uiState.error?.let { error ->
VoiceErrorDialog(
error = error,
onDismiss = onClearError,
onRetry = {
onClearError()
onMicTap()
},
)
}
// Mic button — dispatch by current voice state.
@@ -589,6 +550,60 @@ fun VoiceModeOverlay(
}
}
@Composable
private fun VoiceErrorDialog(
error: String,
onDismiss: () -> Unit,
onRetry: () -> Unit,
) {
val keyboard = LocalSoftwareKeyboardController.current
LaunchedEffect(error) { keyboard?.hide() }
// Match ChatFailureDetailsDialog: keep provider detail selectable and inside
// a bounded surface, with actions outside the scroll area. A real dialog
// owns focus and blocks the underlying chat in both voice presentations.
AlertDialog(
onDismissRequest = onDismiss,
containerColor = MaterialTheme.colorScheme.surface,
icon = {
Icon(
imageVector = Icons.Default.ErrorOutline,
contentDescription = null,
tint = MaterialTheme.colorScheme.error,
)
},
title = { Text(stringResource(R.string.voice_overlay_error_title)) },
text = {
Surface(
color = MaterialTheme.colorScheme.surfaceVariant,
shape = MaterialTheme.shapes.small,
) {
SelectionContainer {
Text(
text = error,
style = MaterialTheme.typography.bodySmall,
modifier = Modifier
.fillMaxWidth()
.heightIn(max = 320.dp)
.verticalScroll(rememberScrollState())
.padding(12.dp),
)
}
}
},
confirmButton = {
TextButton(onClick = onRetry) {
Text(stringResource(R.string.voice_overlay_retry))
}
},
dismissButton = {
TextButton(onClick = onDismiss) {
Text(stringResource(R.string.common_dismiss))
}
},
)
}
@Composable
private fun VoiceFocusIdentityPane(
uiState: VoiceUiState,
@@ -1,5 +1,9 @@
package com.hermesandroid.relay.ui.screens
import com.hermesandroid.relay.data.BusyMessageAction
import com.hermesandroid.relay.data.canCorrectBusyMessage
import com.hermesandroid.relay.ui.components.ChatComposerLayers
import com.hermesandroid.relay.ui.components.ChatMessageQueue
import androidx.compose.foundation.BorderStroke
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
@@ -58,6 +62,7 @@ import com.hermesandroid.relay.network.upstream.DashboardApiClient
import com.hermesandroid.relay.network.upstream.GatewayChatClient
import com.hermesandroid.relay.network.upstream.SessionMessageLoadMode
import com.hermesandroid.relay.network.upstream.models.MessageItem
import com.hermesandroid.relay.ui.components.ChatMediaViewerHost
import com.hermesandroid.relay.ui.components.MessageBubble
import com.hermesandroid.relay.ui.theme.RelayRefresh
import com.hermesandroid.relay.viewmodel.ChatViewModel
@@ -84,24 +89,28 @@ fun BotChatScreen(
connectionViewModel: ConnectionViewModel,
onBack: () -> Unit,
) {
BotChatScreen(
route = route,
bot = bot,
sessionId = sessionId,
gatewayClient = gatewayClient,
dashboardClient = dashboardClient,
chatViewModel = chatViewModel,
onBack = onBack,
handlerFactory = ::ChatHandler,
historyLoader = { profileName, storedSessionId, mode ->
dashboardClient.getSessionMessages(
sessionId = storedSessionId,
profile = profileName,
mode = mode,
)
},
profileIconFlow = connectionViewModel::profileIconFlow,
)
val busyMessageAction by connectionViewModel.busyMessageAction.collectAsState()
ChatMediaViewerHost(route.key, sessionId) {
BotChatScreen(
route = route,
bot = bot,
sessionId = sessionId,
gatewayClient = gatewayClient,
dashboardClient = dashboardClient,
chatViewModel = chatViewModel,
onBack = onBack,
handlerFactory = ::ChatHandler,
historyLoader = { profileName, storedSessionId, mode ->
dashboardClient.getSessionMessages(
sessionId = storedSessionId,
profile = profileName,
mode = mode,
)
},
profileIconFlow = connectionViewModel::profileIconFlow,
busyMessageAction = busyMessageAction,
)
}
}
@OptIn(ExperimentalMaterial3Api::class)
@@ -117,6 +126,7 @@ internal fun BotChatScreen(
handlerFactory: () -> ChatHandler,
historyLoader: BotChatHistoryLoader,
profileIconFlow: BotChatProfileIconFlow,
busyMessageAction: BusyMessageAction = BusyMessageAction.CorrectNow,
) {
val handler = remember(route.key) { handlerFactory() }
val context = LocalContext.current
@@ -133,6 +143,21 @@ internal fun BotChatScreen(
.collectAsState(initial = null)
val listState = rememberLazyListState()
var composer by remember(route.key, sessionId) { mutableStateOf("") }
var editingQueuedMessage by remember(route.key, sessionId) { mutableStateOf(false) }
var busyActionOverride by remember(route.key, sessionId, busyMessageAction) {
mutableStateOf<BusyMessageAction?>(null)
}
val steerable by chatViewModel.steerableTurn.collectAsState()
val attachments by chatViewModel.pendingAttachments.collectAsState()
val pendingAsk by chatViewModel.pendingAsk.collectAsState()
val turnStatus by handler.turnStatus.collectAsState()
val queue by chatViewModel.queuedMessages.collectAsState()
val queuePaused by chatViewModel.queuePaused.collectAsState()
val correctionAvailable = canCorrectBusyMessage(
steerable, attachments.isNotEmpty(), pendingAsk != null, turnStatus, composer,
)
val effectiveBusyAction = if (correctionAvailable && !editingQueuedMessage) busyActionOverride ?: busyMessageAction
else BusyMessageAction.QueueNext
DisposableEffect(chatViewModel, gatewayClient, dashboardClient, route.key) {
chatViewModel.initialize(apiClient = null, chatHandler = handler)
@@ -234,49 +259,82 @@ internal fun BotChatScreen(
)
},
bottomBar = {
ChatComposerLayers(
action = effectiveBusyAction.takeIf { isStreaming },
onActionChange = {
editingQueuedMessage = false
busyActionOverride = it
},
correctionAvailable = correctionAvailable,
onStop = if (composer.isNotBlank()) chatViewModel::cancelStream else null,
modifier = Modifier.imePadding(),
) {
Surface(
color = MaterialTheme.colorScheme.surfaceContainer,
border = BorderStroke(1.dp, MaterialTheme.colorScheme.outlineVariant),
shape = RoundedCornerShape(topStart = 18.dp, topEnd = 18.dp),
modifier = Modifier.imePadding(),
) {
Row(
modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp),
verticalAlignment = Alignment.Bottom,
) {
OutlinedTextField(
value = composer,
onValueChange = { composer = it },
placeholder = { Text(stringResource(R.string.chat_placeholder_message)) },
modifier = Modifier.weight(1f),
minLines = 1,
maxLines = 6,
)
Spacer(Modifier.width(6.dp))
IconButton(
onClick = {
if (isStreaming) {
chatViewModel.cancelStream()
} else {
val text = composer.trim()
if (text.isNotEmpty()) {
composer = ""
chatViewModel.sendMessage(text)
}
Column {
ChatMessageQueue(
messages = queue,
paused = queuePaused,
onResume = chatViewModel::resumeQueue,
onClear = chatViewModel::clearQueue,
onRemove = chatViewModel::removeQueuedAt,
canEdit = composer.isBlank() && attachments.isEmpty(),
onEdit = { index ->
chatViewModel.takeQueuedForEdit(index)?.let {
composer = it
editingQueuedMessage = true
}
},
enabled = isStreaming || composer.isNotBlank(),
modifier = Modifier.padding(horizontal = 12.dp),
)
Row(
modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp),
verticalAlignment = Alignment.Bottom,
) {
Icon(
if (isStreaming) Icons.Filled.Stop else Icons.AutoMirrored.Filled.Send,
contentDescription = stringResource(
if (isStreaming) R.string.chat_input_stop_streaming
else R.string.chat_input_send_message,
),
OutlinedTextField(
value = composer,
onValueChange = { composer = it },
placeholder = { Text(stringResource(R.string.chat_placeholder_message)) },
modifier = Modifier.weight(1f),
minLines = 1,
maxLines = 6,
)
Spacer(Modifier.width(6.dp))
IconButton(
onClick = {
if (isStreaming && composer.isBlank()) {
chatViewModel.cancelStream()
} else {
val text = composer.trim()
if (text.isNotEmpty()) {
composer = ""
chatViewModel.sendMessage(text, effectiveBusyAction)
editingQueuedMessage = false
busyActionOverride = null
}
}
},
enabled = isStreaming || composer.isNotBlank(),
) {
Icon(
if (isStreaming && composer.isBlank()) Icons.Filled.Stop else Icons.AutoMirrored.Filled.Send,
contentDescription = stringResource(
when {
isStreaming && composer.isBlank() -> R.string.chat_input_stop_streaming
isStreaming && effectiveBusyAction == BusyMessageAction.CorrectNow -> R.string.chat_correct_now
isStreaming -> R.string.chat_queue_next
else -> R.string.chat_input_send_message
},
),
)
}
}
}
}
}
},
) { padding ->
Box(
@@ -2,6 +2,9 @@
package com.hermesandroid.relay.ui.screens
import com.hermesandroid.relay.data.BusyMessageAction
import com.hermesandroid.relay.data.canCorrectBusyMessage
import com.hermesandroid.relay.ui.components.ChatMessageQueue
import androidx.compose.animation.AnimatedVisibility
import androidx.compose.animation.animateContentSize
import androidx.compose.foundation.Canvas
@@ -1138,6 +1141,12 @@ fun ChatScreen(
val availableSkills by chatViewModel.availableSkills.collectAsState()
val queuedMessages by chatViewModel.queuedMessages.collectAsState()
val queuePaused by chatViewModel.queuePaused.collectAsState()
val busyMessageAction by connectionViewModel.busyMessageAction.collectAsState()
var busyActionOverride by remember(currentSessionId, busyMessageAction) {
mutableStateOf<BusyMessageAction?>(null)
}
var editingQueuedMessage by remember(currentSessionId) { mutableStateOf(false) }
val recentPrompts by chatViewModel.recentPrompts.collectAsState()
val recentPromptsEnabled by connectionViewModel.chatRecentPromptsEnabled.collectAsState()
// Effective approval-bypass (server-computed: ORs global approvals.mode=off,
@@ -1398,7 +1407,11 @@ fun ChatScreen(
}
var showCommandPalette by remember { mutableStateOf(false) }
var showTranscriptSearch by rememberSaveable { mutableStateOf(false) }
var pendingAttachmentPreview by remember { mutableStateOf<Attachment?>(null) }
var pendingAttachmentPreviewIndex by rememberSaveable(
composerDraftKey.connectionId,
composerDraftKey.profileId,
composerDraftKey.sessionId,
) { mutableStateOf<Int?>(null) }
var showModelSheet by remember { mutableStateOf(false) }
var showEffortSheet by remember { mutableStateOf(false) }
var showAgentInfo by remember { mutableStateOf(false) }
@@ -4110,69 +4123,21 @@ fun ChatScreen(
}
}
AnimatedVisibility(visible = queuedMessages.isNotEmpty()) {
Column(
modifier = Modifier
.fillMaxWidth()
.padding(horizontal = 20.dp),
) {
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween
) {
Text(
text = pluralStringResource(
R.plurals.chat_queue_count,
queuedMessages.size,
queuedMessages.size,
),
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.primary
)
TextButton(
onClick = { chatViewModel.clearQueue() },
modifier = Modifier.height(28.dp)
) {
Text(
stringResource(R.string.chat_clear),
style = MaterialTheme.typography.labelSmall
)
}
ChatMessageQueue(
messages = queuedMessages,
paused = queuePaused,
onResume = chatViewModel::resumeQueue,
onClear = chatViewModel::clearQueue,
onRemove = chatViewModel::removeQueuedAt,
canEdit = inputText.isBlank() && pendingAttachments.isEmpty(),
onEdit = { index ->
chatViewModel.takeQueuedForEdit(index)?.let {
inputText = it
editingQueuedMessage = true
}
// Per-item: tap the text to pull it back into the composer
// for editing; ✕ to drop just that one. (Reorder omitted —
// low value vs. drag-handle complexity on a transient queue.)
queuedMessages.forEachIndexed { index, msg ->
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
) {
Text(
text = msg,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
maxLines = 2,
overflow = androidx.compose.ui.text.style.TextOverflow.Ellipsis,
modifier = Modifier
.weight(1f)
.clickable {
chatViewModel.takeQueuedForEdit(index)?.let { t ->
inputText = if (inputText.isBlank()) t else "$inputText $t"
}
}
.padding(vertical = 4.dp),
)
TextButton(
onClick = { chatViewModel.removeQueuedAt(index) },
modifier = Modifier.height(28.dp),
) {
Text("✕", style = MaterialTheme.typography.labelSmall)
}
}
}
}
}
},
modifier = Modifier.fillMaxWidth().padding(horizontal = 20.dp),
)
val quoteYouLabel = stringResource(R.string.chat_quote_you)
val quoteHermesLabel = stringResource(R.string.chat_quote_hermes)
@@ -4218,15 +4183,17 @@ fun ChatScreen(
PendingAttachmentComposer(
attachments = pendingAttachments,
onPreview = { attachment, _ -> pendingAttachmentPreview = attachment },
onPreview = { _, index -> pendingAttachmentPreviewIndex = index },
onRemove = chatViewModel::removeAttachment,
onMove = chatViewModel::moveAttachment,
modifier = Modifier.padding(horizontal = 16.dp, vertical = 4.dp),
)
pendingAttachmentPreview?.let { attachment ->
pendingAttachmentPreviewIndex
?.let(pendingAttachments::getOrNull)
?.let { attachment ->
AttachmentViewer(
attachment = attachment,
onDismiss = { pendingAttachmentPreview = null },
onDismiss = { pendingAttachmentPreviewIndex = null },
initiallyRevealed = true,
)
}
@@ -4279,25 +4246,32 @@ fun ChatScreen(
// Gateway redirect is text-only. Attachment-bearing follow-ups must
// retain their files in the session-owned queue instead of showing
// a correction action that cannot carry them.
val canSteerCurrentMessage = steerableTurn && pendingAttachments.isEmpty() &&
val canSteerCurrentMessage = canCorrectBusyMessage(
steerableTurn, pendingAttachments.isNotEmpty(), pendingAsk != null, turnStatus, inputText,
) &&
(!supervised || supervisedPolicy.capabilities.steerResponse)
val effectiveBusyAction = if (canSteerCurrentMessage && !editingQueuedMessage) busyActionOverride ?: busyMessageAction
else BusyMessageAction.QueueNext
val correctCurrentMessage = canSteerCurrentMessage &&
effectiveBusyAction == BusyMessageAction.CorrectNow
val trailing = when {
!isStreaming && hasContent -> ChatInputTrailing.SEND
!isStreaming -> ChatInputTrailing.VOICE
isStreaming && !hasContent -> ChatInputTrailing.STOP
canSteerCurrentMessage -> ChatInputTrailing.STEER
correctCurrentMessage -> ChatInputTrailing.STEER
else -> ChatInputTrailing.QUEUE
}
val inputCaption = when {
isStreaming && hasContent && canSteerCurrentMessage ->
isStreaming && hasContent && correctCurrentMessage ->
stringResource(R.string.chat_sends_now)
isStreaming && hasContent && queuePaused -> stringResource(R.string.chat_queue_paused)
isStreaming && hasContent -> stringResource(R.string.chat_delivered_after_turn)
isStreaming && steerNotice != null -> steerNotice
else -> null
}
val inputPlaceholder = when {
editingMessage != null -> stringResource(R.string.chat_placeholder_edit)
isStreaming && canSteerCurrentMessage -> stringResource(R.string.chat_placeholder_steer)
isStreaming && correctCurrentMessage -> stringResource(R.string.chat_placeholder_steer)
isStreaming -> stringResource(R.string.chat_placeholder_queue)
else -> stringResource(R.string.chat_placeholder_message)
}
@@ -4549,6 +4523,12 @@ fun ChatScreen(
}
ChatInputBar(
busyAction = effectiveBusyAction.takeIf { isStreaming },
correctionAvailable = canSteerCurrentMessage,
onBusyActionChange = {
editingQueuedMessage = false
busyActionOverride = it
},
value = inputText,
onValueChange = { inputText = it },
placeholder = inputPlaceholder,
@@ -4577,7 +4557,9 @@ fun ChatScreen(
}
}
} else {
chatViewModel.sendMessage(outboundText)
chatViewModel.sendMessage(outboundText, effectiveBusyAction)
editingQueuedMessage = false
busyActionOverride = null
inputText = ""
finishSuccessfulSend()
}
@@ -1,5 +1,6 @@
package com.hermesandroid.relay.ui.screens
import com.hermesandroid.relay.ui.components.ChatBusyActionSelector
import android.os.Build
import androidx.activity.compose.rememberLauncherForActivityResult
import androidx.activity.result.contract.ActivityResultContracts
@@ -499,6 +500,22 @@ fun ChatSettingsScreen(
HorizontalDivider()
val busyMessageAction by connectionViewModel.busyMessageAction.collectAsState()
Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text(stringResource(R.string.chat_busy_default_title), style = MaterialTheme.typography.bodyMedium)
Text(
stringResource(R.string.chat_busy_default_description),
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
ChatBusyActionSelector(
action = busyMessageAction,
onActionChange = connectionViewModel::setBusyMessageAction,
modifier = Modifier.fillMaxWidth(),
)
}
HorizontalDivider()
val physicalKeyboardEnterBehavior by
connectionViewModel.physicalKeyboardEnterBehavior.collectAsState()
Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
@@ -6,6 +6,7 @@ import androidx.core.content.FileProvider
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import java.io.File
import java.io.InputStream
import java.security.MessageDigest
/**
@@ -98,7 +99,7 @@ class MediaCacheWriter(
suspend fun cache(bytes: ByteArray, contentType: String, fileName: String?): Uri =
withContext(Dispatchers.IO) {
val dir = cacheRoot
val targetName = buildFilename(bytes, contentType, fileName)
val targetName = buildFilename(bytes.sha1(), contentType, fileName)
val file = File(dir, targetName)
file.writeBytes(bytes)
// Best-effort LRU enforcement — never fatal on a cache operation.
@@ -113,6 +114,45 @@ class MediaCacheWriter(
)
}
/**
* Move a completed streaming download into the media cache without ever
* materializing the file as a ByteArray. The caller retains ownership of
* [source] when this throws; successful calls consume or copy it.
*/
suspend fun cache(source: File, contentType: String, fileName: String?): Uri {
val target = cacheFile(source, contentType, fileName)
return withContext(Dispatchers.IO) {
FileProvider.getUriForFile(
context,
"${context.packageName}.fileprovider",
target,
)
}
}
internal suspend fun cacheFile(source: File, contentType: String, fileName: String?): File =
withContext(Dispatchers.IO) {
require(source.isFile) { "Media cache source does not exist" }
val dir = cacheRoot
val target = File(dir, buildFilename(source.sha1(), contentType, fileName))
if (source.canonicalFile != target.canonicalFile) {
if (target.exists()) {
source.delete()
target.setLastModified(System.currentTimeMillis())
} else if (!source.renameTo(target)) {
source.inputStream().use { input ->
target.outputStream().use { output -> input.copyTo(output) }
}
source.delete()
}
}
try {
enforceCap()
} catch (_: Exception) { /* swallow */ }
target
}
/**
* Wipe the entire `hermes-media/` directory. Returns the number of bytes
* freed so the caller can show a "Freed X MB" snackbar/toast.
@@ -133,15 +173,8 @@ class MediaCacheWriter(
// --- internals ---
private fun buildFilename(bytes: ByteArray, contentType: String, fileName: String?): String {
private fun buildFilename(sha1: String, contentType: String, fileName: String?): String {
val ext = mimeToExt[contentType.lowercase().substringBefore(';').trim()] ?: "bin"
// SHA-1 of the bytes — stable + collision-resistant enough for cache keys.
val sha1 = try {
MessageDigest.getInstance("SHA-1").digest(bytes).joinToString("") { "%02x".format(it) }
} catch (_: Exception) {
// fallback to a weaker but always-available hash
bytes.contentHashCode().toUInt().toString(16)
}
// If the server supplied a filename, keep the base (sanitized) but
// force the extension we derived from the MIME. This keeps the
@@ -162,6 +195,33 @@ class MediaCacheWriter(
}
}
private fun ByteArray.sha1(): String = try {
MessageDigest.getInstance("SHA-1").digest(this).toHexString()
} catch (_: Exception) {
contentHashCode().toUInt().toString(16)
}
private fun File.sha1(): String = try {
inputStream().use { input ->
val digest = MessageDigest.getInstance("SHA-1")
input.updateDigest(digest)
digest.digest().toHexString()
}
} catch (_: Exception) {
"${length().toUInt().toString(16)}-${lastModified().toUInt().toString(16)}"
}
private fun InputStream.updateDigest(digest: MessageDigest) {
val buffer = ByteArray(DEFAULT_BUFFER_SIZE)
while (true) {
val read = read(buffer)
if (read < 0) return
digest.update(buffer, 0, read)
}
}
private fun ByteArray.toHexString(): String = joinToString("") { "%02x".format(it) }
/**
* Delete oldest-mtime files until the cache directory fits under the cap.
* Called after every write; safe to call more often.
@@ -14,6 +14,9 @@ import kotlinx.coroutines.withContext
import okhttp3.OkHttpClient
import okhttp3.Request
import java.io.File
import java.io.ByteArrayOutputStream
import java.io.IOException
import java.io.InputStream
import java.util.concurrent.TimeUnit
/**
@@ -39,6 +42,7 @@ object MediaSaver {
private const val SAVE_SUBDIR = "Hermes-Relay"
private const val SHARE_CACHE_DIR = "hermes-media"
private const val MAX_IN_MEMORY_MEDIA_BYTES = 25L * 1024L * 1024L
private val httpClient: OkHttpClient by lazy {
OkHttpClient.Builder()
@@ -70,7 +74,10 @@ object MediaSaver {
httpClient.newCall(request).execute().use { resp ->
if (!resp.isSuccessful) error("HTTP ${resp.code}")
val contentType = resp.header("Content-Type")?.substringBefore(';')?.trim()
val body = resp.body.bytes()
val responseBody = resp.body
val body = responseBody.byteStream().use { input ->
readBoundedBytes(input, responseBody.contentLength())
}
body to contentType
}
}
@@ -78,11 +85,42 @@ object MediaSaver {
/** Read the bytes behind a `content://` (or `file://`) [uri]. */
suspend fun readUriBytes(context: Context, uri: Uri): ByteArray? =
withContext(Dispatchers.IO) {
runCatching {
context.contentResolver.openInputStream(uri)?.use { it.readBytes() }
}.getOrNull()
try {
val declaredLength = context.contentResolver.openAssetFileDescriptor(uri, "r")
?.use { it.length }
?.takeIf { it >= 0L }
?: -1L
context.contentResolver.openInputStream(uri)?.use { input ->
readBoundedBytes(input, declaredLength)
}
} catch (_: Exception) {
null
}
}
private fun readBoundedBytes(input: InputStream, declaredLength: Long): ByteArray {
if (declaredLength > MAX_IN_MEMORY_MEDIA_BYTES) {
throw IOException("Media exceeds Android's in-memory export limit")
}
val initialSize = when {
declaredLength in 1..MAX_IN_MEMORY_MEDIA_BYTES -> declaredLength.toInt()
else -> DEFAULT_BUFFER_SIZE
}
val output = ByteArrayOutputStream(initialSize)
val buffer = ByteArray(DEFAULT_BUFFER_SIZE)
var total = 0L
while (true) {
val read = input.read(buffer)
if (read < 0) break
total += read
if (total > MAX_IN_MEMORY_MEDIA_BYTES) {
throw IOException("Media exceeds Android's in-memory export limit")
}
output.write(buffer, 0, read)
}
return output.toByteArray()
}
// --- Save ---------------------------------------------------------------
suspend fun saveImage(
@@ -1,5 +1,7 @@
package com.hermesandroid.relay.viewmodel
import com.hermesandroid.relay.data.BusyMessageAction
import com.hermesandroid.relay.data.canCorrectBusyMessage
import android.content.Context
import android.net.ConnectivityManager
import android.net.NetworkCapabilities
@@ -173,6 +175,8 @@ import kotlinx.serialization.json.contentOrNull
import kotlinx.serialization.json.intOrNull
import kotlinx.serialization.json.longOrNull
import okhttp3.sse.EventSource
import java.io.File
import java.io.IOException
import java.io.InterruptedIOException
import java.util.UUID
import java.util.concurrent.ConcurrentHashMap
@@ -743,6 +747,7 @@ class ChatViewModel : ViewModel() {
private const val CHECKPOINT_WRITE_INTERVAL_MS = 750L
private const val SESSION_REFRESH_RETRY_DELAY_MS = 1_000L
private const val SESSION_REFRESH_MAX_READINESS_RETRIES = 2
private const val MAX_COALESCED_SESSION_REFRESH_PASSES = 2
private const val SESSION_DRAWER_FRESHNESS_WINDOW_MS = 5_000L
private const val PROFILE_SESSION_CACHE_CONTEXTS = 24
private const val PROFILE_SESSION_CACHE_ROWS = 50
@@ -909,6 +914,10 @@ class ChatViewModel : ViewModel() {
)
private val queuedMessageItems = mutableListOf<QueuedMessage>()
private val pausedQueueDestinations = mutableSetOf<Pair<String, String>>()
private val retainedQueueCheckpoints = mutableMapOf<Pair<String, String>, ChatTurnCheckpoint>()
private val _queuePaused = MutableStateFlow(false)
val queuePaused: StateFlow<Boolean> = _queuePaused.asStateFlow()
private val completedQueueOwnerRuns = ConcurrentHashMap.newKeySet<String>()
private val _queuedMessages = MutableStateFlow<List<String>>(emptyList())
val queuedMessages: StateFlow<List<String>> = _queuedMessages.asStateFlow()
@@ -3400,8 +3409,6 @@ class ChatViewModel : ViewModel() {
}
if (visibleSignature != serverSignature) {
handler.loadMessageHistory(serverMessages)
refreshSessions()
scheduleTitleReconcile(storedSessionId)
return@launch
}
if (attempt < 7 && retryUntilChanged) delay(250L)
@@ -4035,7 +4042,7 @@ class ChatViewModel : ViewModel() {
private suspend fun loadSessionHistory(
sessionId: String,
requireProfileScope: Boolean = false,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
mode: SessionMessageLoadMode = SessionMessageLoadMode.LATEST,
profileName: String? = currentSessionProfileName(),
): List<MessageItem> {
if (profileMessageLoader != null) {
@@ -4061,7 +4068,7 @@ class ChatViewModel : ViewModel() {
private suspend fun loadGatewaySessionHistory(
sessionId: String,
requireProfileScope: Boolean = false,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
mode: SessionMessageLoadMode = SessionMessageLoadMode.LATEST,
profileName: String? = currentSessionProfileName(),
): List<MessageItem> {
val scoped = profileMessageLoader?.invoke(profileName, sessionId, mode)
@@ -4799,6 +4806,8 @@ class ChatViewModel : ViewModel() {
publishBackgroundSessionActivity()
queuedMessageItems.clear()
completedQueueOwnerRuns.clear()
pausedQueueDestinations.clear()
retainedQueueCheckpoints.clear()
publishQueuedMessages()
_steerableTurn.value = false
_steerNotice.value = null
@@ -5329,11 +5338,13 @@ class ChatViewModel : ViewModel() {
val sessionPageLoadFailed: StateFlow<Boolean> = _sessionPageLoadFailed.asStateFlow()
val sessionListUnavailable: StateFlow<Boolean> = _sessionListUnavailable.asStateFlow()
fun refreshSessions() = refreshSessions(
allowReadinessRetry = true,
preserveFailurePresentation = false,
readinessRetryAttempt = 0,
)
fun refreshSessions() {
refreshSessions(
allowReadinessRetry = true,
preserveFailurePresentation = false,
readinessRetryAttempt = 0,
)
}
/**
* Drawer-open freshness gate. Process-owned startup binding may already be
@@ -5458,8 +5469,10 @@ class ChatViewModel : ViewModel() {
var retryUnavailable = false
var retryReadiness = false
var refreshPass = 0
try {
do {
refreshPass += 1
sessionRefreshPending = false
// A queued trailing refresh owns a new outcome. Never let
// an earlier timeout overwrite a later successful list.
@@ -5557,7 +5570,7 @@ class ChatViewModel : ViewModel() {
)
} catch (e: CancellationException) {
throw e
} catch (e: Throwable) {
} catch (e: Exception) {
if (isCurrentRefresh()) {
android.util.Log.w(
"ChatViewModel",
@@ -5580,7 +5593,11 @@ class ChatViewModel : ViewModel() {
}
}
}
} while (sessionRefreshPending && isCurrentRefresh())
} while (
refreshPass < MAX_COALESCED_SESSION_REFRESH_PASSES &&
sessionRefreshPending &&
isCurrentRefresh()
)
} finally {
val shouldRetry = allowReadinessRetry &&
retryReadiness &&
@@ -6301,7 +6318,11 @@ class ChatViewModel : ViewModel() {
// --- Message sending ---
fun sendMessage(text: String) {
fun sendMessage(
text: String,
busyAction: BusyMessageAction =
BusyMessageAction.CorrectNow,
) {
if (text.isBlank()) return
supervisedMessageBlockReason(supervisedModePolicy, text)?.let { reason ->
chatHandler?.addSystemNotice(reason)
@@ -6388,9 +6409,13 @@ class ChatViewModel : ViewModel() {
// Mid-turn: redirect on the gateway transport, queue everywhere else.
if (activeStream != null) {
if (
_steerableTurn.value &&
busyAction == BusyMessageAction.CorrectNow &&
(!supervisedModePolicy.enabled || supervisedModePolicy.capabilities.steerResponse) &&
streamingEndpoint == "gateway" &&
_pendingAttachments.value.isEmpty()
canCorrectBusyMessage(
_steerableTurn.value, _pendingAttachments.value.isNotEmpty(),
_pendingAsk.value != null, chatHandler?.turnStatus?.value, text,
)
) {
steerActiveTurn(text.trim())
} else {
@@ -7570,8 +7595,7 @@ class ChatViewModel : ViewModel() {
queuedMessages = queuedMessageItems
.filter {
it.contextKey == contextKey &&
it.sessionId == sessionId &&
it.ownerRunId == seed.userMessageId
it.sessionId == sessionId
}
.map { item ->
ChatQueuedMessageCheckpoint(
@@ -7583,17 +7607,22 @@ class ChatViewModel : ViewModel() {
hadAttachments = item.attachments.isNotEmpty(),
)
},
queuePaused = (contextKey to sessionId) in pausedQueueDestinations,
startedAt = seed.startedAt,
updatedAt = now,
)
}
private fun restoreQueuedMessages(checkpoint: ChatTurnCheckpoint) {
val destination = checkpoint.contextKey to checkpoint.sessionId
if (checkpoint.queuePaused || checkpoint.queueOnly) pausedQueueDestinations += destination
var droppedAttachmentQueue = false
val droppedOwners = mutableMapOf<String, String>()
checkpoint.queuedMessages.forEach { saved ->
if (queuedMessageItems.any { it.id == saved.id }) return@forEach
if (saved.hadAttachments) {
droppedAttachmentQueue = true
droppedOwners[saved.id] = droppedOwners[saved.ownerRunId] ?: saved.ownerRunId
return@forEach
}
queuedMessageItems += QueuedMessage(
@@ -7602,11 +7631,16 @@ class ChatViewModel : ViewModel() {
contextKey = checkpoint.contextKey,
sessionId = checkpoint.sessionId,
transport = saved.transport,
ownerRunId = saved.ownerRunId,
ownerRunId = droppedOwners[saved.ownerRunId] ?: saved.ownerRunId,
attachments = emptyList(),
interfaceContextPrompt = saved.interfaceContextPrompt,
)
}
if (checkpoint.queueOnly) {
val ids = queuedMessageItems.mapTo(mutableSetOf()) { it.id }
queuedMessageItems.filter { it.contextKey == checkpoint.contextKey && it.sessionId == checkpoint.sessionId }
.map { it.ownerRunId }.filterNot { it in ids }.forEach { completedQueueOwnerRuns += it }
}
publishQueuedMessages()
if (droppedAttachmentQueue) {
_transientNotice.tryEmit(
@@ -7641,10 +7675,12 @@ class ChatViewModel : ViewModel() {
}
}
private fun clearTurnCheckpoint() {
private fun clearTurnCheckpoint(preserveQueue: Boolean = false) {
val queueSnapshot = if (preserveQueue) buildQueueOnlyCheckpoint() else null
queueSnapshot?.let { retainedQueueCheckpoints[it.contextKey to it.sessionId] = it }
val key = activeTurnCheckpointSeed?.let { seed ->
seed.contextKey?.let { TurnCheckpointKey(it, seed.sessionId) }
}
} ?: queueSnapshot?.let { TurnCheckpointKey(it.contextKey, it.sessionId) }
activeTurnCheckpointSeed = null
activeQueueOwnerRunId = null
val generation = checkpointGeneration.incrementAndGet()
@@ -7661,7 +7697,8 @@ class ChatViewModel : ViewModel() {
viewModelScope.launch {
checkpointMutex.withLock {
if (generation == checkpointGeneration.get() && activeTurnCheckpointSeed == null) {
if (key != null) runCatching { store.remove(key.contextKey, key.sessionId) }
if (queueSnapshot != null) runCatching { store.write(queueSnapshot) }
else if (key != null) runCatching { store.remove(key.contextKey, key.sessionId) }
}
}
}
@@ -7791,6 +7828,11 @@ class ChatViewModel : ViewModel() {
backgroundNeedsInputKeys -= key
publishBackgroundSessionActivity()
if (checkpoint.transport !in setOf("gateway", "sessions")) return false
if (checkpoint.queueOnly) {
retainedQueueCheckpoints[contextKey to sessionId] = checkpoint
restoreQueuedMessages(checkpoint)
return false
}
if (activeStream != null) return true
if (streamRecovery != null) {
if (checkpoint.transport == "gateway" && client != null) {
@@ -7804,7 +7846,13 @@ class ChatViewModel : ViewModel() {
}
adoptTurnCheckpoint(checkpoint)
val initialHistory = runCatching { loadSessionHistory(sessionId) }.getOrDefault(emptyList())
val initialHistory = try {
loadSessionHistory(sessionId)
} catch (error: CancellationException) {
throw error
} catch (_: Exception) {
emptyList()
}
if (!ownsTurnCheckpoint(checkpoint, handler)) return true
if (initialHistory.isNotEmpty()) handler.loadMessageHistory(initialHistory)
handler.restoreInFlightTurn(checkpoint)
@@ -8217,7 +8265,7 @@ class ChatViewModel : ViewModel() {
if (completedOwner != null && queuedMessageItems.any { it.ownerRunId == completedOwner }) {
completedQueueOwnerRuns += completedOwner
}
clearTurnCheckpoint()
clearTurnCheckpoint(preserveQueue = true)
activeStream = null
_steerableTurn.value = false
_steerNotice.value = null
@@ -8461,6 +8509,106 @@ class ChatViewModel : ViewModel() {
it.contextKey == destination.first && it.sessionId == destination.second
}.map(QueuedMessage::text)
}
_queuePaused.value = _queuedMessages.value.isNotEmpty() && destination in pausedQueueDestinations
}
private fun persistQueueChanges() {
scheduleCheckpointWrite(immediate = true)
val destination = currentQueueDestination() ?: return
if (activeTurnCheckpointSeed?.let { it.contextKey to it.sessionId } == destination) return
val template = retainedQueueCheckpoints[destination] ?: return
val items = queuedMessageItems.filter { it.contextKey to it.sessionId == destination }
val snapshot = template.copy(
queuePaused = destination in pausedQueueDestinations,
queuedMessages = items.map { item ->
ChatQueuedMessageCheckpoint(
id = item.id,
text = item.text.take(MAX_CHECKPOINT_TEXT_CHARS),
transport = item.transport,
ownerRunId = item.ownerRunId,
interfaceContextPrompt = item.interfaceContextPrompt,
hadAttachments = item.attachments.isNotEmpty(),
)
},
updatedAt = System.currentTimeMillis(),
)
if (items.isEmpty()) retainedQueueCheckpoints.remove(destination)
else retainedQueueCheckpoints[destination] = snapshot
val generation = checkpointGeneration.get()
val store = chatTurnCheckpointStore ?: return
viewModelScope.launch {
checkpointMutex.withLock {
if (generation != checkpointGeneration.get()) return@withLock
runCatching {
if (items.isEmpty()) store.remove(destination.first, destination.second)
else store.write(snapshot)
}
}
}
}
private fun buildQueueOnlyCheckpoint(): ChatTurnCheckpoint? {
val destination = currentQueueDestination() ?: return null
val items = queuedMessageItems.filter { (it.contextKey to it.sessionId) == destination }
if (items.isEmpty()) return null
buildTurnCheckpoint()?.let { return it.copy(queueOnly = true, pendingAsk = null) }
// A server-initiated turn has no outgoing user checkpoint. Retain only
// queue ownership metadata; queueOnly restoration never renders these
// empty turn fields or attempts to reattach the cancelled runtime.
val now = System.currentTimeMillis()
return ChatTurnCheckpoint(
contextKey = destination.first,
sessionId = destination.second,
transport = items.first().transport,
user = ChatTurnUserCheckpoint(items.first().ownerRunId, "", now),
assistant = ChatTurnAssistantCheckpoint(id = items.first().ownerRunId, timestamp = now, isStreaming = false),
priorUserMessageCount = 0,
baselineAssistantCount = 0,
queuedMessages = items.map { item ->
ChatQueuedMessageCheckpoint(
item.id, item.text.take(MAX_CHECKPOINT_TEXT_CHARS), item.transport,
item.ownerRunId, item.interfaceContextPrompt, item.attachments.isNotEmpty(),
)
},
queuePaused = destination in pausedQueueDestinations,
queueOnly = true,
startedAt = now,
updatedAt = now,
)
}
/** Stop preserves pending work. Resume is explicit and always keeps its destination. */
fun resumeQueue() {
val destination = currentQueueDestination() ?: return
val first = queuedMessageItems.firstOrNull { it.contextKey to it.sessionId == destination } ?: return
if (activeStream != null) {
val owner = activeQueueOwnerRunId ?: return
val index = queuedMessageItems.indexOf(first)
queuedMessageItems[index] = first.copy(ownerRunId = owner)
} else {
completedQueueOwnerRuns += first.ownerRunId
}
pausedQueueDestinations -= destination
publishQueuedMessages()
persistQueueChanges()
drainQueue()
}
private fun removeQueuedItem(item: QueuedMessage) {
queuedMessageItems.removeAll { it.id == item.id }
// Splice the run chain: successors now wait for the removed item's
// predecessor, never for a deleted turn that can no longer complete.
queuedMessageItems.replaceAll { next ->
if (next.contextKey == item.contextKey && next.sessionId == item.sessionId && next.ownerRunId == item.id) {
next.copy(ownerRunId = item.ownerRunId)
} else next
}
if (queuedMessageItems.none { it.ownerRunId == item.ownerRunId }) completedQueueOwnerRuns -= item.ownerRunId
if (queuedMessageItems.none { it.contextKey == item.contextKey && it.sessionId == item.sessionId }) {
pausedQueueDestinations -= item.contextKey to item.sessionId
}
publishQueuedMessages()
persistQueueChanges()
}
private fun removeQueuedMessagesFor(contextKey: String?, sessionId: String): Int {
@@ -8486,6 +8634,9 @@ class ChatViewModel : ViewModel() {
fun clearQueue() {
val destination = currentQueueDestination() ?: return
removeQueuedMessagesFor(destination.first, destination.second)
pausedQueueDestinations -= destination
publishQueuedMessages()
persistQueueChanges()
}
/** Drop a single queued message by index (no-op if out of range). */
@@ -8495,11 +8646,7 @@ class ChatViewModel : ViewModel() {
it.contextKey == destination.first && it.sessionId == destination.second
}
val item = visible.getOrNull(index) ?: return
queuedMessageItems.removeAll { it.id == item.id }
if (queuedMessageItems.none { it.ownerRunId == item.ownerRunId }) {
completedQueueOwnerRuns -= item.ownerRunId
}
publishQueuedMessages()
removeQueuedItem(item)
}
/**
@@ -8507,14 +8654,15 @@ class ChatViewModel : ViewModel() {
* the composer can prefill it. Null if the index is out of range.
*/
fun takeQueuedForEdit(index: Int): String? {
if (_pendingAttachments.value.isNotEmpty()) {
_transientNotice.tryEmit("Finish the current attachment draft before editing a queued message.")
return null
}
val destination = currentQueueDestination() ?: return null
val item = queuedMessageItems.filter {
it.contextKey == destination.first && it.sessionId == destination.second
}.getOrNull(index) ?: return null
queuedMessageItems.removeAll { it.id == item.id }
if (queuedMessageItems.none { it.ownerRunId == item.ownerRunId }) {
completedQueueOwnerRuns -= item.ownerRunId
}
removeQueuedItem(item)
_pendingAttachments.value = item.attachments
nextInterfaceContextPrompt = item.interfaceContextPrompt
publishQueuedMessages()
@@ -8528,6 +8676,7 @@ class ChatViewModel : ViewModel() {
if (streamingEndpoint == "gateway" && gatewayClient == null && client == null) return
if (activeStream != null || streamRecovery != null) return
val destination = currentQueueDestination() ?: return
if (destination in pausedQueueDestinations) return
val next = queuedMessageItems.firstOrNull {
it.contextKey == destination.first &&
it.sessionId == destination.second &&
@@ -9758,6 +9907,11 @@ class ChatViewModel : ViewModel() {
handler.onTurnComplete(currentMessageId)
gatewayInterimMessageId = currentMessageId
gatewayInterimSealedCurrentMessage = true
// `message.interim` closes one assistant segment but not the
// agent run. Create the next blank owner immediately so the chat
// retains its in-conversation working indicator even when the
// next tool/reasoning event is delayed or suppressed upstream.
ensurePostInterimMessage()
scheduleCheckpointWrite(immediate = true)
}
val onInterimReconciledCb = { text: String ->
@@ -10123,12 +10277,16 @@ class ChatViewModel : ViewModel() {
(streamingEndpoint == "sessions" || streamingEndpoint == "gateway")
) {
viewModelScope.launch {
runCatching {
try {
// Profile-aware read — see onCompleteCb: a bare
// getMessages 404s for a non-default-profile session
// and silently empties the transcript.
val serverMessages = loadSessionHistory(errorSessionId)
handler.loadMessageHistory(serverMessages)
} catch (error: CancellationException) {
throw error
} catch (_: Exception) {
// The existing local failure remains authoritative.
}
}
}
@@ -10531,6 +10689,10 @@ class ChatViewModel : ViewModel() {
_steerableTurn.value = false
onPreflightErrorCb(Exception(reason))
},
onSubmitRejected = { reason ->
_steerableTurn.value = false
onPreflightErrorCb(Exception(reason))
},
onFailure = { failure ->
val confirmedModel = gateway.serverModel.value
?.takeIf { it.isNotBlank() }
@@ -10580,7 +10742,7 @@ class ChatViewModel : ViewModel() {
_steerableTurn.value = false
onPreflightErrorCb(Exception(reason))
},
onPreflightFailure = {
onPreflightFailure = { reason ->
_steerableTurn.value = false
if (intentionallyCancelled) {
// User cancelled while preflight was in flight —
@@ -10592,11 +10754,11 @@ class ChatViewModel : ViewModel() {
// Nothing started server-side. Keep this turn bound
// to Gateway and settle it as retryable local state;
// API sessions are a different owner/database.
onPreflightErrorCb(
IllegalStateException(
"Hermes Dashboard chat is unavailable. Sign in or reconnect, then retry.",
),
)
// Preserve the authoritative Gateway explanation.
// Inference initialization failures happen before
// prompt.submit, and replacing them with generic
// reconnect advice hides the actual operator fix.
onPreflightErrorCb(IllegalStateException(reason))
}
},
)
@@ -10669,6 +10831,13 @@ class ChatViewModel : ViewModel() {
fun cancelStream() {
intentionallyCancelled = true
currentQueueDestination()?.let { destination ->
if (queuedMessageItems.any { it.contextKey to it.sessionId == destination }) {
pausedQueueDestinations += destination
activeQueueOwnerRunId?.let { completedQueueOwnerRuns += it }
}
}
publishQueuedMessages()
// User Stop also aborts a dropped-stream answer recovery. settleUi
// false: the Stopped-badge block below finalizes the placeholder
// itself (completing it here first would hide it from findLast).
@@ -10679,13 +10848,12 @@ class ChatViewModel : ViewModel() {
activeStream = null
imageActivityJob?.cancel()
imageActivityJob = null
clearQueue()
_steerableTurn.value = false
_steerNotice.value = null
// Gateway cancel issues session.interrupt, which force-denies any
// blocked approval server-side — stamp the card to match.
clearPendingAskAfterInterrupt()
clearTurnCheckpoint()
clearTurnCheckpoint(preserveQueue = true)
AppAnalytics.onStreamCancelled()
chatHandler?.let { handler ->
val streamingMsg = handler.messages.value.findLast { it.isStreaming }
@@ -10723,7 +10891,7 @@ class ChatViewModel : ViewModel() {
* placeholder as actionable FAILED with [MEDIA_TAP_TO_DOWNLOAD] in
* [Attachment.errorMessage]. The UI renders a retry card.
* 4. Otherwise → kick off a background fetch, enforce the max size cap,
* cache the bytes via [MediaCacheWriter], and update the attachment
* stream into [MediaCacheWriter], and update the attachment
* to LOADED (or FAILED on any error).
*
* Note: the matching happens by (messageId + relayToken). If the same
@@ -10787,7 +10955,10 @@ class ChatViewModel : ViewModel() {
expectedHistoryGeneration = historyGeneration,
) { _ ->
if (relay.mediaUrlConfigured()) {
relay.fetchMedia(token, maxBytes = settings.maxInboundSizeMb.toLong().coerceAtLeast(1L) * 1024L * 1024L)
relay.fetchMedia(
token,
maxBytes = settings.maxInboundSizeMb.toLong().coerceAtLeast(1L) * 1024L * 1024L,
).asInboundFetchedMedia()
} else {
Result.failure(MediaRouteUnavailableException())
}
@@ -10850,7 +11021,7 @@ class ChatViewModel : ViewModel() {
fetchServerPath(fetchKey, maxBytes, upstreamMediaClient)
} else {
if (relay.mediaUrlConfigured()) {
relay.fetchMedia(fetchKey, maxBytes = maxBytes)
relay.fetchMedia(fetchKey, maxBytes = maxBytes).asInboundFetchedMedia()
} else {
Result.failure(MediaRouteUnavailableException())
}
@@ -10909,7 +11080,22 @@ class ChatViewModel : ViewModel() {
return ServerImageResult.Failure("Connection changed while loading image")
}
return result.fold(
onSuccess = { ServerImageResult.Success(it.bytes, it.sensitive) },
onSuccess = { fetched ->
try {
val cachedUri = fetched.cachedUri ?: mediaCacheWriter?.cache(
requireNotNull(fetched.bytes),
fetched.contentType,
fetched.fileName,
)?.toString()
if (cachedUri == null) {
ServerImageResult.Failure("Media cache is unavailable")
} else {
ServerImageResult.Success(cachedUri, fetched.sensitive)
}
} catch (error: Exception) {
ServerImageResult.Failure(error.message ?: "Image cache failed")
}
},
onFailure = { ServerImageResult.Failure(it.message ?: "Image unavailable") },
)
}
@@ -11031,17 +11217,39 @@ class ChatViewModel : ViewModel() {
serverPath: String,
maxBytes: Long,
upstream: DashboardApiClient?,
): Result<RelayHttpClient.FetchedMedia> {
): Result<InboundFetchedMedia> {
if (upstream != null) {
val upstreamResult = upstream.downloadManagedFile(serverPath, maxBytes)
.map { fetched ->
RelayHttpClient.FetchedMedia(
contentType = fetched.contentType,
bytes = fetched.bytes,
fileName = fetched.fileName,
sensitive = false,
val cache = mediaCacheWriter
?: return Result.failure(IOException("Media cache is unavailable"))
val context = appContext
?: return Result.failure(IOException("Application context is unavailable"))
val staging = File.createTempFile("hermes-media-", ".part", context.cacheDir)
val upstreamResult = try {
val fetchedResult = staging.outputStream().buffered().use { output ->
upstream.downloadManagedFile(serverPath, maxBytes, output)
}
if (fetchedResult.isFailure) {
Result.failure(fetchedResult.exceptionOrNull()!!)
} else {
val fetched = fetchedResult.getOrThrow()
val uri = cache.cache(staging, fetched.contentType, fetched.fileName)
Result.success(
InboundFetchedMedia(
contentType = fetched.contentType,
fileName = fetched.fileName,
sensitive = false,
sizeBytes = fetched.sizeBytes,
cachedUri = uri.toString(),
),
)
}
} catch (error: CancellationException) {
throw error
} catch (error: Exception) {
Result.failure(error)
} finally {
staging.delete()
}
if (upstreamResult.isSuccess) return upstreamResult
val upstreamFailure = upstreamResult.exceptionOrNull()
if (upstreamFailure?.isDashboardManagedFilesUnsupported() != true) {
@@ -11050,13 +11258,14 @@ class ChatViewModel : ViewModel() {
val relay = relayHttpClient
if (relay?.mediaUrlConfigured() == true) {
return relay.fetchMediaByPath(serverPath, maxBytes = maxBytes)
.asInboundFetchedMedia()
}
return Result.failure(MediaRouteUnavailableException(upstreamFailure))
}
val relay = relayHttpClient
return if (relay?.mediaUrlConfigured() == true) {
relay.fetchMediaByPath(serverPath, maxBytes = maxBytes)
relay.fetchMediaByPath(serverPath, maxBytes = maxBytes).asInboundFetchedMedia()
} else {
Result.failure(MediaRouteUnavailableException())
}
@@ -11096,7 +11305,7 @@ class ChatViewModel : ViewModel() {
expectedRole: MessageRole = MessageRole.ASSISTANT,
expectedContextKey: String? = activeProfileContextKey,
expectedHistoryGeneration: Int = historyLoadGeneration.get(),
fetch: suspend (maxBytes: Long) -> Result<RelayHttpClient.FetchedMedia>,
fetch: suspend (maxBytes: Long) -> Result<InboundFetchedMedia>,
) {
val cache = mediaCacheWriter ?: return
val maxBytes = settings.maxInboundSizeMb.toLong().coerceAtLeast(1) * 1024L * 1024L
@@ -11117,8 +11326,8 @@ class ChatViewModel : ViewModel() {
) return
result.fold(
onSuccess = { fetched ->
if (fetched.bytes.size > maxBytes) {
val sizeMb = fetched.bytes.size / (1024.0 * 1024.0)
if (fetched.sizeBytes > maxBytes) {
val sizeMb = fetched.sizeBytes / (1024.0 * 1024.0)
updateAttachmentByToken(handler, messageId, fetchKey, expectedRole = expectedRole) { att ->
att.copy(
state = AttachmentState.FAILED,
@@ -11127,22 +11336,26 @@ class ChatViewModel : ViewModel() {
),
contentType = fetched.contentType,
fileName = fetched.fileName ?: att.fileName,
fileSize = fetched.bytes.size.toLong()
fileSize = fetched.sizeBytes
)
}
return
}
try {
val uri = cache.cache(fetched.bytes, fetched.contentType, fetched.fileName)
val cachedUri = fetched.cachedUri ?: cache.cache(
requireNotNull(fetched.bytes),
fetched.contentType,
fetched.fileName,
).toString()
updateAttachmentByToken(handler, messageId, fetchKey, expectedRole = expectedRole) { att ->
att.copy(
state = AttachmentState.LOADED,
errorMessage = null,
contentType = fetched.contentType,
fileName = fetched.fileName ?: att.fileName,
fileSize = fetched.bytes.size.toLong(),
cachedUri = uri.toString(),
fileSize = fetched.sizeBytes,
cachedUri = cachedUri,
// Carry the relay-authoritative sensitivity bit
// (X-Media-Sensitive header) onto the LOADED
// attachment so the renderer can blur per the
@@ -11188,6 +11401,26 @@ class ChatViewModel : ViewModel() {
)
}
private data class InboundFetchedMedia(
val contentType: String,
val fileName: String?,
val sensitive: Boolean,
val sizeBytes: Long,
val bytes: ByteArray? = null,
val cachedUri: String? = null,
)
private fun Result<RelayHttpClient.FetchedMedia>.asInboundFetchedMedia(): Result<InboundFetchedMedia> =
map { fetched ->
InboundFetchedMedia(
contentType = fetched.contentType,
fileName = fetched.fileName,
sensitive = fetched.sensitive,
sizeBytes = fetched.bytes.size.toLong(),
bytes = fetched.bytes,
)
}
/**
* Append an attachment to a specific assistant message, matched by id.
* No-ops if the message can't be found (e.g. it was trimmed from the
@@ -1,5 +1,6 @@
package com.hermesandroid.relay.viewmodel
import com.hermesandroid.relay.data.BusyMessageAction
import android.app.Application
import android.os.Build
import android.content.Context
@@ -2387,13 +2388,13 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
suspend fun loadProfileScopedMessages(
sessionId: String,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
mode: SessionMessageLoadMode = SessionMessageLoadMode.LATEST,
): Result<List<MessageItem>>? = profileController.loadProfileScopedMessages(sessionId, mode)
suspend fun loadProfileScopedMessages(
profileName: String?,
sessionId: String,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
mode: SessionMessageLoadMode = SessionMessageLoadMode.LATEST,
): Result<List<MessageItem>>? =
profileController.loadProfileScopedMessages(profileName, sessionId, mode)
@@ -3090,6 +3091,13 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
}
}
val busyMessageAction = chatInputPreferencesRepository.busyMessageAction
.stateIn(viewModelScope, SharingStarted.Eagerly, BusyMessageAction.CorrectNow)
fun setBusyMessageAction(action: BusyMessageAction) {
viewModelScope.launch { chatInputPreferencesRepository.setBusyMessageAction(action) }
}
val physicalKeyboardEnterBehavior: StateFlow<PhysicalKeyboardEnterBehavior> =
chatInputPreferencesRepository.physicalKeyboardEnterBehavior
.stateIn(
@@ -909,7 +909,7 @@ class ProfileController(
*/
suspend fun loadProfileScopedMessages(
sessionId: String,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
mode: SessionMessageLoadMode = SessionMessageLoadMode.LATEST,
): Result<List<MessageItem>>? = loadProfileScopedMessages(
profileName = resolveSessionProfileName(),
sessionId = sessionId,
@@ -919,7 +919,7 @@ class ProfileController(
suspend fun loadProfileScopedMessages(
profileName: String?,
sessionId: String,
mode: SessionMessageLoadMode = SessionMessageLoadMode.COMPLETE,
mode: SessionMessageLoadMode = SessionMessageLoadMode.LATEST,
): Result<List<MessageItem>>? {
val connectionId = activeConnectionId.value ?: return null
val dashboardUrl = activeDashboardUrlProvider() ?: return null
@@ -122,6 +122,7 @@
<string name="chat_retry">Tentar novamente</string>
<string name="chat_dismiss">Dispensar</string>
<string name="chat_failure_title">O Hermes não conseguiu concluir esta resposta</string>
<string name="chat_failure_inference_unavailable">Inferência indisponível</string>
<string name="chat_failure_details">Detalhes</string>
<string name="chat_failure_details_title">Falha na resposta</string>
<string name="chat_failure_details_guidance">O Hermes relatou este erro. O app não mudará de rota nem de modelo automaticamente.</string>
@@ -4420,4 +4421,12 @@
<string name="active_section_unavailable">Unavailable</string>
<string name="detail_inactive_badge">Not active</string>
<string name="detail_overview_clarity_summary">What works now, the path this phone is using, and what can take over if it becomes unavailable.</string>
<string name="voice_overlay_error_title">Erro de voz</string>
<string name="chat_correct_now">Corrigir agora</string>
<string name="chat_queue_next">Enviar depois</string>
<string name="chat_busy_default_title">Durante a resposta</string>
<string name="chat_busy_default_description">Escolha o que Enviar faz enquanto o Hermes trabalha. Este padrão do dispositivo também vale no campo de mensagem. Anexos e correções indisponíveis entram na fila.</string>
<string name="chat_queue_paused">Fila pausada</string>
<string name="chat_queue_resume">Retomar</string>
<string name="chat_queue_remove">Remover mensagem da fila</string>
</resources>
@@ -136,6 +136,7 @@
<string name="chat_retry">重试</string>
<string name="chat_dismiss">关闭</string>
<string name="chat_failure_title">Hermes 无法完成此回复</string>
<string name="chat_failure_inference_unavailable">推理服务不可用</string>
<string name="chat_failure_details">详情</string>
<string name="chat_failure_details_title">回复失败</string>
<string name="chat_failure_details_guidance">Hermes 报告了此错误。应用不会自动切换路由或模型。</string>
@@ -4501,4 +4502,12 @@
<string name="active_section_unavailable">Unavailable</string>
<string name="detail_inactive_badge">Not active</string>
<string name="detail_overview_clarity_summary">What works now, the path this phone is using, and what can take over if it becomes unavailable.</string>
<string name="voice_overlay_error_title">语音错误</string>
<string name="chat_correct_now">立即纠正</string>
<string name="chat_queue_next">稍后发送</string>
<string name="chat_busy_default_title">回复期间</string>
<string name="chat_busy_default_description">选择 Hermes 工作时发送按钮的行为。此设备默认设置与输入框共用。附件及无法立即纠正的消息将进入队列。</string>
<string name="chat_queue_paused">队列已暂停</string>
<string name="chat_queue_resume">继续</string>
<string name="chat_queue_remove">移除队列消息</string>
</resources>
+9
View File
@@ -136,6 +136,7 @@
<string name="chat_retry">Erneut versuchen</string>
<string name="chat_dismiss">Schließen</string>
<string name="chat_failure_title">Hermes konnte diese Antwort nicht abschließen</string>
<string name="chat_failure_inference_unavailable">Inferenz nicht verfügbar</string>
<string name="chat_failure_details">Details</string>
<string name="chat_failure_details_title">Antwortfehler</string>
<string name="chat_failure_details_guidance">Hermes hat diesen Fehler gemeldet. Die App wechselt Routen oder Modelle nicht automatisch.</string>
@@ -4577,4 +4578,12 @@
<string name="active_section_unavailable">Unavailable</string>
<string name="detail_inactive_badge">Not active</string>
<string name="detail_overview_clarity_summary">What works now, the path this phone is using, and what can take over if it becomes unavailable.</string>
<string name="voice_overlay_error_title">Sprachfehler</string>
<string name="chat_correct_now">Jetzt korrigieren</string>
<string name="chat_queue_next">Danach senden</string>
<string name="chat_busy_default_title">Während der Antwort</string>
<string name="chat_busy_default_description">Wähle, was Senden während der Arbeit ausführt. Dieser geräteweite Standard gilt auch im Eingabefeld. Anhänge und nicht verfügbare Korrekturen werden eingereiht.</string>
<string name="chat_queue_paused">Warteschlange pausiert</string>
<string name="chat_queue_resume">Fortsetzen</string>
<string name="chat_queue_remove">Nachricht aus Warteschlange entfernen</string>
</resources>
+9
View File
@@ -113,6 +113,7 @@
<string name="chat_retry">Reintentar</string>
<string name="chat_dismiss">Descartar</string>
<string name="chat_failure_title">Hermes no pudo completar esta respuesta</string>
<string name="chat_failure_inference_unavailable">Inferencia no disponible</string>
<string name="chat_failure_details">Detalles</string>
<string name="chat_failure_details_title">Error de respuesta</string>
<string name="chat_failure_details_guidance">Hermes informó de este error. La aplicación no cambiará automáticamente de ruta ni de modelo.</string>
@@ -4268,4 +4269,12 @@
<string name="active_section_unavailable">Unavailable</string>
<string name="detail_inactive_badge">Not active</string>
<string name="detail_overview_clarity_summary">What works now, the path this phone is using, and what can take over if it becomes unavailable.</string>
<string name="voice_overlay_error_title">Error de voz</string>
<string name="chat_correct_now">Corregir ahora</string>
<string name="chat_queue_next">Enviar después</string>
<string name="chat_busy_default_title">Durante la respuesta</string>
<string name="chat_busy_default_description">Elige qué hace Enviar mientras Hermes trabaja. Este ajuste del dispositivo también se usa en el cuadro de mensaje. Los adjuntos y las correcciones no disponibles se ponen en cola.</string>
<string name="chat_queue_paused">Cola pausada</string>
<string name="chat_queue_resume">Reanudar</string>
<string name="chat_queue_remove">Quitar mensaje de la cola</string>
</resources>
+9
View File
@@ -136,6 +136,7 @@
<string name="chat_retry">リトライ</string>
<string name="chat_dismiss">却下する</string>
<string name="chat_failure_title">Hermes はこの応答を完了できませんでした</string>
<string name="chat_failure_inference_unavailable">推論を利用できません</string>
<string name="chat_failure_details">詳細</string>
<string name="chat_failure_details_title">応答エラー</string>
<string name="chat_failure_details_guidance">Hermes からこのエラーが報告されました。アプリがルートやモデルを自動的に切り替えることはありません。</string>
@@ -4572,4 +4573,12 @@
<string name="active_section_unavailable">Unavailable</string>
<string name="detail_inactive_badge">Not active</string>
<string name="detail_overview_clarity_summary">What works now, the path this phone is using, and what can take over if it becomes unavailable.</string>
<string name="voice_overlay_error_title">音声エラー</string>
<string name="chat_correct_now">今すぐ修正</string>
<string name="chat_queue_next">次に送信</string>
<string name="chat_busy_default_title">応答中の動作</string>
<string name="chat_busy_default_description">Hermesの作業中に送信する際の動作を選択します。この端末の設定は入力欄と共通です。添付ファイルや修正できない場合はキューに追加します。</string>
<string name="chat_queue_paused">キューを一時停止中</string>
<string name="chat_queue_resume">再開</string>
<string name="chat_queue_remove">キューからメッセージを削除</string>
</resources>
+9
View File
@@ -130,6 +130,7 @@
<string name="chat_retry">Повторить</string>
<string name="chat_dismiss">Отменить</string>
<string name="chat_failure_title">Hermes не смог завершить ответ</string>
<string name="chat_failure_inference_unavailable">Генерация ответа недоступна</string>
<string name="chat_failure_details">Подробности</string>
<string name="chat_failure_details_title">Ошибка ответа</string>
<string name="chat_failure_details_guidance">Hermes сообщил об этой ошибке. Приложение не будет автоматически менять маршрут или модель.</string>
@@ -4316,4 +4317,12 @@
<string name="active_section_unavailable">Unavailable</string>
<string name="detail_inactive_badge">Not active</string>
<string name="detail_overview_clarity_summary">What works now, the path this phone is using, and what can take over if it becomes unavailable.</string>
<string name="voice_overlay_error_title">Ошибка голосового режима</string>
<string name="chat_correct_now">Исправить сейчас</string>
<string name="chat_queue_next">Отправить позже</string>
<string name="chat_busy_default_title">Во время ответа</string>
<string name="chat_busy_default_description">Выберите действие отправки, пока Hermes работает. Настройка общая для устройства и поля ввода. Вложения и недоступные исправления добавляются в очередь.</string>
<string name="chat_queue_paused">Очередь приостановлена</string>
<string name="chat_queue_resume">Продолжить</string>
<string name="chat_queue_remove">Удалить сообщение из очереди</string>
</resources>
+9
View File
@@ -140,6 +140,7 @@
<string name="chat_retry">Retry</string>
<string name="chat_dismiss">Dismiss</string>
<string name="chat_failure_title">Hermes couldn’t complete this response</string>
<string name="chat_failure_inference_unavailable">Inference unavailable</string>
<string name="chat_failure_details">Details</string>
<string name="chat_failure_details_title">Response failure</string>
<string name="chat_failure_details_guidance">Hermes reported this error. The app won’t switch routes or models automatically.</string>
@@ -4593,4 +4594,12 @@
<string name="settings_git_workspace">Git workspace</string>
<string name="settings_git_workspace_desc">Current-session Git first, with optional host discovery</string>
<string name="settings_git_workspace_off_desc">Session repository only · Host discovery is opt-in</string>
<string name="voice_overlay_error_title">Voice error</string>
<string name="chat_correct_now">Correct now</string>
<string name="chat_queue_next">Queue next</string>
<string name="chat_busy_default_title">While responding</string>
<string name="chat_busy_default_description">Choose what Send does while Hermes is working. This device-wide default is shared with the composer. Attachments and unavailable corrections queue instead.</string>
<string name="chat_queue_paused">Queue paused</string>
<string name="chat_queue_resume">Resume</string>
<string name="chat_queue_remove">Remove queued message</string>
</resources>
@@ -15,6 +15,26 @@ import org.junit.Test
class ChatInputPreferencesTest {
@Test
fun `busy action defaults to correction and is shared after repository recreation`() = runTest {
val store = InMemoryChatInputDataStore()
val repository = ChatInputPreferencesRepository(store)
assertEquals(BusyMessageAction.CorrectNow, repository.busyMessageAction.first())
repository.setBusyMessageAction(BusyMessageAction.QueueNext)
assertEquals(BusyMessageAction.QueueNext, ChatInputPreferencesRepository(store).busyMessageAction.first())
assertEquals(BusyMessageAction.CorrectNow, BusyMessageAction.fromStoredValue("unknown"))
}
@Test
fun `correction availability rejects attachments blocked input compaction and slash commands`() {
assertEquals(true, canCorrectBusyMessage(true, false, false, null, "Correct this"))
assertEquals(false, canCorrectBusyMessage(false, false, false, null, "Correct this"))
assertEquals(false, canCorrectBusyMessage(true, true, false, null, "Correct this"))
assertEquals(false, canCorrectBusyMessage(true, false, true, null, "Correct this"))
assertEquals(false, canCorrectBusyMessage(true, false, false, "Compacting context", "Correct this"))
assertEquals(false, canCorrectBusyMessage(true, false, false, null, "/model next"))
}
@Test
fun `large paste conversion defaults on and round trips`() = runTest {
val store = InMemoryChatInputDataStore()
@@ -13,6 +13,8 @@ import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.jsonObject
import okhttp3.HttpUrl.Companion.toHttpUrl
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.ResponseBody.Companion.toResponseBody
import okhttp3.mockwebserver.MockResponse
import okhttp3.mockwebserver.MockWebServer
import okhttp3.mockwebserver.SocketPolicy
@@ -31,6 +33,23 @@ import java.util.concurrent.TimeUnit
class DashboardApiClientTest {
@Test
fun boundedJsonReaderRejectsDeclaredAndObservedOverflow() {
val declared = "{}".toResponseBody("application/json".toMediaType())
assertThrows(IOException::class.java) {
declared.readUtf8Bounded(1)
}
val chunked = object : okhttp3.ResponseBody() {
override fun contentType() = "application/json".toMediaType()
override fun contentLength() = -1L
override fun source() = Buffer().writeUtf8("{\"value\":123}")
}
assertThrows(IOException::class.java) {
chunked.readUtf8Bounded(8)
}
}
@Test
fun `multiplex API routing uses served profiles instead of installed inventory`() {
val status = DashboardStatus(
@@ -1466,7 +1485,7 @@ class DashboardApiClientTest {
assertEquals("mizu", url.queryParameter("profile"))
assertEquals("500", url.queryParameter("limit"))
assertEquals("0", url.queryParameter("offset"))
assertEquals("oldest", url.queryParameter("order"))
assertEquals("latest", url.queryParameter("order"))
assertEquals(2, messages.size)
assertEquals("user", messages[0].role)
assertEquals("assistant", messages[1].role)
@@ -1478,7 +1497,11 @@ class DashboardApiClientTest {
server.enqueue(messagePageResponse(key = "messages", start = 500, count = 1, returned = 1))
val messages = DashboardApiClient(baseUrl = server.url("/").toString())
.getSessionMessages("sess-a", profile = "mizu")
.getSessionMessages(
"sess-a",
profile = "mizu",
mode = SessionMessageLoadMode.COMPLETE,
)
.getOrThrow()
val first = server.takeRequest().requestUrl!!
@@ -2042,13 +2065,15 @@ class DashboardApiClientTest {
.setBody("audio-bytes"),
)
val output = ByteArrayOutputStream()
val fetched = DashboardApiClient(baseUrl = server.url("/").toString())
.downloadManagedFile("/tmp/Hermes audio/voice reply.mp3", 1024)
.downloadManagedFile("/tmp/Hermes audio/voice reply.mp3", 1024, output)
.getOrThrow()
assertEquals("audio/mpeg", fetched.contentType)
assertEquals("voice reply.mp3", fetched.fileName)
assertEquals("audio-bytes", fetched.bytes.decodeToString())
assertEquals(11L, fetched.sizeBytes)
assertEquals("audio-bytes", output.toString(Charsets.UTF_8.name()))
val request = server.takeRequest()
assertEquals("/api/files/download", request.requestUrl!!.encodedPath)
assertEquals("/tmp/Hermes audio/voice reply.mp3", request.requestUrl!!.queryParameter("path"))
@@ -2059,11 +2084,28 @@ class DashboardApiClientTest {
server.enqueue(MockResponse().setHeader("Content-Length", 2048))
val result = DashboardApiClient(baseUrl = server.url("/").toString())
.downloadManagedFile("/tmp/large.bin", 1024)
.downloadManagedFile("/tmp/large.bin", 1024, ByteArrayOutputStream())
assertTrue(result.isFailure)
}
@Test
fun downloadManagedFile_rejectsChunkedBodyAfterStreamingCap() = runTest {
server.enqueue(
MockResponse()
.setHeader("Content-Type", "application/octet-stream")
.setChunkedBody("seventeen-byte-doc", 3),
)
val output = ByteArrayOutputStream()
val result = DashboardApiClient(baseUrl = server.url("/").toString())
.downloadManagedFile("/tmp/growing.bin", 16, output)
assertTrue(result.isFailure)
assertTrue(result.exceptionOrNull()?.message.orEmpty().contains("configured download limit"))
assertTrue(output.size() <= 16)
}
@Test
fun managedFileFallbackClassificationDistinguishesMissingRouteFromMissingFile() {
assertTrue(
@@ -133,6 +133,9 @@ class GatewayClientHarness(
@Volatile
var createdSessionProfileName: String? = null
@Volatile
var createdSessionLazy = false
/** Config keys rejected with the older-gateway unknown-key response. */
val unsupportedConfigKeys: MutableSet<String> = ConcurrentHashMap.newKeySet()
@@ -329,6 +332,7 @@ class GatewayClientHarness(
?: (params["profile"] as? JsonPrimitive)?.contentOrNull
?: "default",
)
if (createdSessionLazy) put("lazy", true)
})
}
}
@@ -768,6 +772,7 @@ class GatewayChatClientTest {
val thinkingDeltas = ConcurrentLinkedQueue<String>()
val sessionIds = ConcurrentLinkedQueue<String>()
val errors = ConcurrentLinkedQueue<String>()
val submitRejections = ConcurrentLinkedQueue<String>()
val resumeFailures = ConcurrentLinkedQueue<String>()
val interactions = ConcurrentLinkedQueue<GatewayAsk>()
val interactionExpiries = ConcurrentLinkedQueue<GatewayAskExpiry>()
@@ -805,6 +810,7 @@ class GatewayChatClientTest {
onResumeFailure = { resumeFailures += it; completeLatch.countDown() },
onStatusUpdate = { _, _ -> },
onStatusClear = { },
onSubmitRejected = { submitRejections += it; completeLatch.countDown() },
)
}
@@ -812,6 +818,7 @@ class GatewayChatClientTest {
rpcTimeoutMs: Long = 15_000L,
promptSubmitTimeoutMs: Long = 1_800_000L,
turnIdleTimeoutMs: Long = 180_000L,
sessionReadyTimeoutMs: Long = 300_000L,
compactingTimeoutMs: Long = 600_000L,
callbackDispatcher: (block: () -> Unit) -> Unit = { it() },
ticketTimeoutMs: Long = 8_000L,
@@ -834,6 +841,7 @@ class GatewayChatClientTest {
rpcTimeoutMs = rpcTimeoutMs,
promptSubmitTimeoutMs = promptSubmitTimeoutMs,
turnIdleTimeoutMs = turnIdleTimeoutMs,
sessionReadyTimeoutMs = sessionReadyTimeoutMs,
compactingTimeoutMs = compactingTimeoutMs,
)
@@ -872,6 +880,7 @@ class GatewayChatClientTest {
rpcTimeoutMs: Long = 15_000L,
promptSubmitTimeoutMs: Long = 1_800_000L,
turnIdleTimeoutMs: Long = 180_000L,
sessionReadyTimeoutMs: Long = 300_000L,
compactingTimeoutMs: Long = 600_000L,
ticketTimeoutMs: Long = 8_000L,
) {
@@ -881,6 +890,7 @@ class GatewayChatClientTest {
rpcTimeoutMs = rpcTimeoutMs,
promptSubmitTimeoutMs = promptSubmitTimeoutMs,
turnIdleTimeoutMs = turnIdleTimeoutMs,
sessionReadyTimeoutMs = sessionReadyTimeoutMs,
compactingTimeoutMs = compactingTimeoutMs,
ticketTimeoutMs = ticketTimeoutMs,
)
@@ -2687,13 +2697,28 @@ class GatewayChatClientTest {
}
@Test
fun `session create rejects older gateway without profile ownership metadata`() {
fun `profile scoped socket accepts older session result without duplicate ownership metadata`() {
val r = Recorder()
client.sessionProfileProvider = { "mizu" }
harness.omitSessionProfileMetadata = true
client.sendTurn(null, "hi", null, r.callbacks) { r.preflightFailures += it }
harness.awaitServerSocket()
harness.awaitRpc("prompt.submit")
assertTrue(r.preflightFailures.isEmpty())
}
@Test
fun `unscoped socket still rejects missing named profile ownership metadata`() {
val r = Recorder()
harness.omitSessionProfileMetadata = true
client.observe()
harness.awaitServerSocket()
awaitCondition { client.connectionState.value == GatewayConnectionState.Ready }
client.sessionProfileProvider = { "mizu" }
client.sendTurn(null, "hi", null, r.callbacks) { r.preflightFailures += it }
awaitCondition { r.preflightFailures.isNotEmpty() }
assertTrue(r.preflightFailures.single().contains("did not confirm profile ownership"))
@@ -3944,6 +3969,65 @@ class GatewayChatClientTest {
assertEquals(true, (submit["queued"] as? JsonPrimitive)?.booleanOrNull)
}
@Test
fun `lazy fresh session waits for authoritative ready edge before submit`() {
harness.createdSessionLazy = true
val r = Recorder()
client.sendTurn(null, "hello", null, r.callbacks) { r.preflightFailures += it }
harness.awaitRpc("session.create")
val serverWs = harness.awaitServerSocket()
Thread.sleep(100)
assertEquals(0, harness.rpcLog.count { it.first == "prompt.submit" })
serverWs.send(
harness.eventFrame(
"session.info",
buildJsonObject { put("lazy", false) },
"live-1",
),
)
harness.awaitRpc("prompt.submit")
assertTrue(r.preflightFailures.isEmpty())
}
@Test
fun `lazy fresh session readiness timeout never submits`() {
rebuildClient(sessionReadyTimeoutMs = 50L)
harness.createdSessionLazy = true
val r = Recorder()
client.sendTurn(null, "hello", null, r.callbacks) { r.preflightFailures += it }
harness.awaitRpc("session.create")
waitUntil { r.preflightFailures.isNotEmpty() }
assertEquals(0, harness.rpcLog.count { it.first == "prompt.submit" })
}
@Test
fun `lazy session initialization error fails before prompt submit`() {
harness.createdSessionLazy = true
val r = Recorder()
client.sendTurn(null, "hello", null, r.callbacks) { r.preflightFailures += it }
harness.awaitRpc("session.create")
val serverWs = harness.awaitServerSocket()
serverWs.send(
harness.eventFrame(
"error",
buildJsonObject {
put("message", "agent init failed: No Codex credentials stored")
},
"live-1",
),
)
waitUntil { r.preflightFailures.isNotEmpty() }
assertTrue(r.preflightFailures.single().contains("No Codex credentials stored"))
assertEquals(0, harness.rpcLog.count { it.first == "prompt.submit" })
}
@Test
fun `authoritative prompt rejections surface server message without preflight fallback`() {
val cases = listOf(
@@ -3964,8 +4048,9 @@ class GatewayChatClientTest {
client.sendTurn(null, "hello-$code", null, r.callbacks) { r.preflightFailures += it }
harness.awaitRpc("prompt.submit")
waitUntil { r.errors.isNotEmpty() }
assertEquals(listOf(message), r.errors.toList())
waitUntil { r.submitRejections.isNotEmpty() }
assertEquals(listOf(message), r.submitRejections.toList())
assertTrue(r.errors.isEmpty())
assertTrue("$code must not trigger SSE fallback", r.preflightFailures.isEmpty())
assertEquals(index + 1, harness.rpcLog.count { it.first == "prompt.submit" })
}
@@ -44,6 +44,7 @@ class GatewayEventMapperTest {
var usage: UsageInfo? = null
var usageCalls = 0
val errors = mutableListOf<String>()
val submitRejections = mutableListOf<String>()
val callbacks = GatewayTurnCallbacks(
onSessionId = { sessionIds += it },
@@ -74,6 +75,7 @@ class GatewayEventMapperTest {
onStatusClear = { statusClears += it },
onNoticeShow = { notices += it },
onNoticeClear = { noticeClears += it },
onSubmitRejected = { submitRejections += it },
)
}
@@ -483,10 +485,31 @@ class GatewayEventMapperTest {
val mapper = mapperWith(r)
mapper.onEvent("error", obj("""{"message":"model exploded"}"""))
assertEquals(listOf("model exploded"), r.errors)
assertTrue(r.submitRejections.isEmpty())
assertTrue(mapper.turnEnded)
assertEquals(0, r.completes)
}
@Test
fun `session ownership error is an authoritative submit rejection`() {
val r = Recorder()
val mapper = mapperWith(r)
val message =
"Session fresh-1 already has a live owner (webui, pid 42, running 0m). " +
"Only one surface at a time may run a session, because a second one would reason from stale history."
mapper.onEvent(
"error",
obj(
"""{"message":"Session fresh-1 already has a live owner (webui, pid 42, running 0m). Only one surface at a time may run a session, because a second one would reason from stale history."}""",
),
)
assertEquals(listOf(message), r.submitRejections)
assertTrue(r.errors.isEmpty())
assertTrue(mapper.turnEnded)
}
// --- Tools ---
@Test
@@ -547,6 +570,24 @@ class GatewayEventMapperTest {
)
}
@Test
fun `wrapped image generation start exposes the effective tool identity`() {
val recorder = Recorder()
val mapper = mapperWith(recorder)
mapper.onEvent(
"tool.start",
obj(
"""{"tool_id":"img-1","name":"tool_call","args":{"name":"image_generate","arguments":{"prompt":"test","aspect_ratio":"landscape"}}}""",
),
)
assertEquals(listOf("img-1" to "image_generate"), recorder.toolStarts)
assertEquals(
listOf("img-1" to "{\"prompt\":\"test\",\"aspect_ratio\":\"landscape\"}"),
recorder.toolStartArgs,
)
}
@Test
fun `tool complete with error routes to failed`() {
val r = Recorder()
@@ -30,7 +30,7 @@ class HermesApiClientTest {
server.enqueue(apiMessagePageResponse(start = 500, count = 1, returned = 1))
val messages = HermesApiClient(server.url("/").toString(), "test-key")
.getMessages("sess-a")
.getMessages("sess-a", SessionMessageLoadMode.COMPLETE)
val first = server.takeRequest().requestUrl!!
val second = server.takeRequest().requestUrl!!
@@ -1,6 +1,8 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.mutableStateOf
import androidx.compose.ui.test.junit4.StateRestorationTester
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onAllNodesWithText
import androidx.compose.ui.test.assertIsNotEnabled
@@ -76,6 +78,173 @@ class AttachmentGalleryUiTest {
compose.onNodeWithText("2 / 3").assertExists()
}
@Test
fun `selected gallery page survives activity state restoration`() {
val attachments = listOf("one.png", "two.png", "three.png").map { name ->
Attachment(
contentType = "image/png",
content = ONE_PIXEL_PNG,
fileName = name,
)
}
val restoration = StateRestorationTester(compose)
restoration.setContent {
MaterialTheme {
AttachmentGallery(attachments = attachments)
}
}
compose.onNodeWithTag("attachment-gallery-tile-1").performClick()
compose.onNodeWithText("2 / 3").assertExists()
restoration.emulateSavedInstanceStateRestore()
compose.onNodeWithText("2 / 3").assertExists()
}
@Test
fun `generic attachment viewer survives activity state restoration`() {
val restoration = StateRestorationTester(compose)
restoration.setContent {
MaterialTheme {
InboundAttachmentCard(
attachment = Attachment(
contentType = "application/octet-stream",
content = "",
fileName = "notes.bin",
),
onRetry = {},
onManualFetch = {},
)
}
}
compose.onNodeWithText("notes.bin").performClick()
compose.onNodeWithTag("attachment-viewer").assertExists()
restoration.emulateSavedInstanceStateRestore()
compose.onNodeWithTag("attachment-viewer").assertExists()
}
@Test
fun `host keeps generic viewer mounted when its transcript row leaves composition`() {
val showSource = mutableStateOf(true)
compose.setContent {
MaterialTheme {
ChatMediaViewerHost {
if (showSource.value) {
InboundAttachmentCard(
attachment = Attachment(
contentType = "application/octet-stream",
content = "",
fileName = "notes.bin",
),
onRetry = {},
onManualFetch = {},
)
}
}
}
}
compose.onNodeWithText("notes.bin").performClick()
compose.onNodeWithTag("attachment-viewer").assertExists()
compose.runOnIdle { showSource.value = false }
compose.onNodeWithTag("attachment-viewer").assertExists()
}
@Test
fun `host keeps image viewer mounted when its transcript row leaves composition`() {
val showSource = mutableStateOf(true)
compose.setContent {
MaterialTheme {
ChatMediaViewerHost {
if (showSource.value) {
InboundAttachmentCard(
attachment = Attachment(
contentType = "image/png",
content = ONE_PIXEL_PNG,
fileName = "image.png",
),
onRetry = {},
onManualFetch = {},
)
}
}
}
}
compose.waitUntil(timeoutMillis = 5_000) {
runCatching {
compose.onNodeWithContentDescription("image.png").fetchSemanticsNode()
}.isSuccess
}
compose.onNodeWithContentDescription("image.png").performClick()
compose.onNodeWithTag("attachment-viewer").assertExists()
compose.runOnIdle { showSource.value = false }
compose.onNodeWithTag("attachment-viewer").assertExists()
}
@Test
fun `host keeps selected gallery page mounted when its source leaves composition`() {
val showSource = mutableStateOf(true)
val attachments = listOf("one.png", "two.png", "three.png").map { name ->
Attachment(
contentType = "image/png",
content = ONE_PIXEL_PNG,
fileName = name,
)
}
compose.setContent {
MaterialTheme {
ChatMediaViewerHost {
if (showSource.value) {
AttachmentGallery(attachments = attachments)
}
}
}
}
compose.onNodeWithTag("attachment-gallery-tile-1").performClick()
compose.onNodeWithText("2 / 3").assertExists()
compose.runOnIdle { showSource.value = false }
compose.onNodeWithText("2 / 3").assertExists()
}
@Test
fun `host dismisses active viewer when its chat owner changes`() {
val owner = mutableStateOf("session-a")
compose.setContent {
MaterialTheme {
ChatMediaViewerHost(owner.value) {
InboundAttachmentCard(
attachment = Attachment(
contentType = "application/octet-stream",
content = "",
fileName = "notes.bin",
),
onRetry = {},
onManualFetch = {},
)
}
}
}
compose.onNodeWithText("notes.bin").performClick()
compose.onNodeWithTag("attachment-viewer").assertExists()
compose.runOnIdle { owner.value = "session-b" }
compose.onNodeWithTag("attachment-viewer").assertDoesNotExist()
}
@Test
fun `sensitive page actions stay disabled until that page is revealed`() {
val attachments = listOf(
@@ -0,0 +1,121 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material3.Surface
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.test.assertIsDisplayed
import androidx.compose.ui.test.assertIsFocused
import androidx.compose.ui.test.hasSetTextAction
import androidx.compose.ui.test.isDialog
import androidx.compose.ui.test.assertIsNotEnabled
import androidx.compose.ui.test.assertIsSelected
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onNodeWithContentDescription
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.onNodeWithTag
import androidx.compose.ui.test.onRoot
import androidx.compose.ui.test.performClick
import androidx.compose.ui.test.performTextInput
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.github.takahirom.roborazzi.captureRoboImage
import com.hermesandroid.relay.data.BusyMessageAction
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(qualifiers = "w400dp-h800dp-xhdpi")
class ChatBusyControlsTest {
@get:Rule val compose = createComposeRule()
@Test fun selectorChangesIntentWithoutSendingAndKeepsStopSeparate() {
var selected by mutableStateOf(BusyMessageAction.CorrectNow)
var stopped = 0
compose.setContent {
HermesRelayTheme(themePreference = "dark") {
ChatBusyActionSelector(selected, { selected = it }, onStop = { stopped++ })
}
}
compose.onNodeWithText("Queue next").assertDoesNotExist()
compose.onNodeWithTag("chatBusyActionTrigger").performClick()
compose.onNodeWithTag("chatBusyAction-queue_next").performClick()
compose.waitForIdle()
compose.onNodeWithTag("chatBusyActionDrawer").assertDoesNotExist()
compose.onNodeWithText("Queue next").assertIsDisplayed()
assertEquals(BusyMessageAction.QueueNext, selected)
assertEquals(0, stopped)
compose.onNodeWithContentDescription("Stop streaming").performClick()
assertEquals(1, stopped)
}
@Test fun unavailableCorrectionKeepsQueueSelectable() {
compose.setContent {
HermesRelayTheme {
ChatBusyActionSelector(BusyMessageAction.QueueNext, {}, correctionAvailable = false)
}
}
compose.onNodeWithTag("chatBusyActionTrigger").performClick()
compose.onNodeWithTag("chatBusyAction-correct_now").assertIsNotEnabled()
compose.onNodeWithTag("chatBusyAction-queue_next").assertIsSelected()
compose.onNodeWithTag("chatBusyActionDrawer")
.captureRoboImage("build/ui-evidence/chat-busy-drawer.png")
}
@Test fun pausedQueueActionsAndComposerRenderTogether() {
var resumed = 0
var cleared = 0
var draft by mutableStateOf("A correction")
compose.setContent {
HermesRelayTheme(themePreference = "dark") {
Surface(Modifier.fillMaxSize()) {
Column(verticalArrangement = Arrangement.Bottom) {
ChatMessageQueue(listOf("A saved follow-up", "Another follow-up"), true,
{ resumed++ }, { cleared++ }, {}, {}, canEdit = true)
ChatInputBar(
value = draft, onValueChange = { draft = it }, placeholder = "Message",
trailing = ChatInputTrailing.STEER, onSend = {}, onVoice = {}, onStop = {},
onAttachPhotos = {}, onAttachFiles = {}, onAttachCamera = {}, onPasteImage = {},
onLongPressAttach = {}, charLimit = 20_000, caption = "Changes the current response",
voiceReady = false, showVoiceHint = false, onVoiceHintShown = {}, isDarkTheme = true,
busyAction = BusyMessageAction.CorrectNow,
)
}
}
}
}
compose.onNodeWithText("Resume").assertIsDisplayed().performClick()
compose.onNodeWithText("Clear").performClick()
assertEquals(1, resumed)
assertEquals(1, cleared)
compose.onRoot().captureRoboImage("build/ui-evidence/chat-busy-controls-dark.png")
val input = compose.onNode(hasSetTextAction())
input.performClick()
val inputBottom = input.fetchSemanticsNode().boundsInRoot.bottom
val composerHeight = compose.onNodeWithTag("chatComposerForeground").fetchSemanticsNode().boundsInRoot.height
compose.onNodeWithTag("chatBusyActionTrigger").performClick()
compose.onNodeWithTag("chatBusyActionDrawer").assertIsDisplayed()
compose.onNode(isDialog()).assertDoesNotExist()
input.assertIsDisplayed().assertIsFocused()
assertEquals(inputBottom, input.fetchSemanticsNode().boundsInRoot.bottom, 1f)
val foreground = compose.onNodeWithTag("chatComposerForeground").fetchSemanticsNode().boundsInRoot
val rearTray = compose.onNodeWithTag("chatBusyActionTray").fetchSemanticsNode().boundsInRoot
assertEquals(composerHeight, foreground.height, 1f)
assertTrue("Rear tray must tuck behind the foreground composer", rearTray.bottom > foreground.top)
assertTrue(compose.onNodeWithTag("chatBusyActionDrawer").fetchSemanticsNode().boundsInRoot.bottom <= foreground.top)
input.performTextInput(" updated")
assertTrue(draft.contains("updated"))
compose.onRoot().captureRoboImage("build/ui-evidence/chat-busy-composer-expanded.png")
}
}
@@ -10,4 +10,16 @@ class ChatFailurePanelTest {
assertEquals("Gateway", failureIdentity("Gateway", null, null))
assertEquals("", failureIdentity("", null, null))
}
@Test
fun `inference initialization errors get a specific presentation`() {
assertEquals(
true,
isInferenceUnavailableFailure(
"agent init failed: No Codex credentials stored. " +
"setup.status reports configured credentials, but runtime resolution still failed.",
),
)
assertEquals(false, isInferenceUnavailableFailure("gateway connect cooling down"))
}
}
@@ -0,0 +1,81 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.semantics.SemanticsActions
import androidx.compose.ui.test.assertIsDisplayed
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.onRoot
import androidx.compose.ui.test.performSemanticsAction
import androidx.compose.ui.text.TextLayoutResult
import androidx.compose.ui.unit.dp
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.github.takahirom.roborazzi.captureRoboImage
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.MessageDeliveryStatus
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotEquals
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(qualifiers = "w400dp-h800dp-xhdpi")
class MessageDeliveryContrastTest {
@get:Rule val compose = createComposeRule()
@Test fun darkUserBubblesUseContrastingDeliveryText() = verifyDeliveryColors("dark")
@Test fun lightUserBubblesUseContrastingDeliveryText() = verifyDeliveryColors("light")
private fun verifyDeliveryColors(theme: String) {
var foreground = Color.Unspecified
var background = Color.Unspecified
compose.setContent {
HermesRelayTheme(themePreference = theme) {
foreground = MaterialTheme.colorScheme.onPrimary
background = MaterialTheme.colorScheme.primary
Surface(Modifier.fillMaxSize()) {
Column(
modifier = Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
MessageDeliveryStatus.entries.forEach { status ->
MessageBubble(
message = ChatMessage(
id = status.name,
role = MessageRole.USER,
content = "Test",
timestamp = 1_700_000_000_000L,
deliveryStatus = status,
),
animationEnabled = false,
)
}
}
}
}
}
assertNotEquals(background, foreground)
listOf("Sending…", "Queued", "Correction sent", "Delivered", "Not sent").forEach { label ->
val layouts = mutableListOf<TextLayoutResult>()
compose.onNodeWithText(label, useUnmergedTree = true)
.assertIsDisplayed()
.performSemanticsAction(SemanticsActions.GetTextLayoutResult) { it(layouts) }
assertEquals("Delivery label must use the user bubble foreground: $label", foreground, layouts.single().layoutInput.style.color)
}
compose.onRoot().captureRoboImage("build/ui-evidence/message-delivery-$theme.png")
}
}
@@ -1,10 +1,15 @@
package com.hermesandroid.relay.ui.components
import android.graphics.Bitmap
import android.media.ExifInterface
import android.net.Uri
import androidx.test.ext.junit.runners.AndroidJUnit4
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotNull
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RuntimeEnvironment
import java.io.File
@RunWith(AndroidJUnit4::class)
class OrientedBitmapDecoderTest {
@@ -36,4 +41,34 @@ class OrientedBitmapDecoderTest {
imageOrientationTransform(ExifInterface.ORIENTATION_TRANSVERSE),
)
}
@Test
fun `cached raster orientation matches viewer mime policy`() {
assertEquals(false, shouldApplyExifOrientation("image/png"))
assertEquals(false, shouldApplyExifOrientation("image/gif"))
assertEquals(true, shouldApplyExifOrientation("image/jpeg"))
assertEquals(true, shouldApplyExifOrientation("image/heif"))
}
@Test
fun `content uri decoder reads encoded image without byte array handoff`() {
val context = RuntimeEnvironment.getApplication()
val source = File.createTempFile("oriented-bitmap-", ".png", context.cacheDir)
val bitmap = Bitmap.createBitmap(3, 2, Bitmap.Config.ARGB_8888)
try {
source.outputStream().use { output ->
bitmap.compress(Bitmap.CompressFormat.PNG, 100, output)
}
} finally {
bitmap.recycle()
}
val decoded = decodeOrientedBitmap(context, Uri.fromFile(source))
assertNotNull(decoded)
assertEquals(3, decoded?.width)
assertEquals(2, decoded?.height)
decoded?.recycle()
source.delete()
}
}
@@ -65,4 +65,15 @@ class StreamingMarkdownContentTest {
planStreamingMarkdownAppend("", "authoritative final response"),
)
}
@Test
fun oversizedMarkdownKeepsDataOutsideTheBoundedRenderWindow() {
val source = "x".repeat(MAX_RENDERED_MARKDOWN_CHARS + 1)
val rendered = markdownRenderWindow(source)
assertEquals(MAX_RENDERED_MARKDOWN_CHARS, rendered.takeWhile { it == 'x' }.length)
assertFalse(rendered.startsWith(source))
assertTrue(rendered.contains("shortened for Android memory safety"))
}
}
@@ -0,0 +1,149 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.semantics.SemanticsProperties
import androidx.compose.ui.test.assertIsDisplayed
import androidx.compose.ui.test.isDialog
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.performClick
import androidx.compose.ui.test.performTouchInput
import androidx.compose.ui.test.swipeUp
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.github.takahirom.roborazzi.captureRoboImage
import com.hermesandroid.relay.data.VoicePresentationMode
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import com.hermesandroid.relay.viewmodel.InteractionMode
import com.hermesandroid.relay.viewmodel.VoiceState
import com.hermesandroid.relay.viewmodel.VoiceUiState
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(qualifiers = "w360dp-h780dp-xhdpi")
class VoiceErrorDialogTest {
@get:Rule val compose = createComposeRule()
@Test
fun conversationError_isModalAndDismissesWithoutStartingMicrophone() {
val actions = mutableListOf<String>()
renderError(onClear = { actions += "clear" }, onMicTap = { actions += "mic" })
compose.onNode(isDialog()).assertExists()
compose.onNodeWithText("Voice error").assertIsDisplayed()
compose.onNodeWithText("Dismiss").assertIsDisplayed()
compose.onNodeWithText("Retry").assertIsDisplayed()
compose.onNode(isDialog()).captureRoboImage("build/ui-evidence/voice-error-conversation-dark.png")
compose.onNodeWithText("Dismiss").performClick()
compose.mainClock.advanceTimeByFrame()
compose.onNode(isDialog()).assertDoesNotExist()
compose.runOnIdle { assertEquals(listOf("clear"), actions) }
}
@Test
fun focusError_retryClearsErrorBeforeStartingMicrophone() {
val actions = mutableListOf<String>()
renderError(
mode = VoicePresentationMode.Focus,
onClear = { actions += "clear" },
onMicTap = { actions += "mic" },
)
compose.onNode(isDialog()).assertExists()
compose.onNodeWithText("Retry").performClick()
compose.mainClock.advanceTimeByFrame()
compose.onNode(isDialog()).assertDoesNotExist()
compose.runOnIdle { assertEquals(listOf("clear", "mic"), actions) }
}
@Test
@Config(qualifiers = "w780dp-h360dp-xhdpi")
fun landscapeLargeText_longErrorScrollsWithoutHidingActions() {
val longError = List(8) { PROVIDER_ERROR }.joinToString("\n\n")
renderError(error = longError, theme = "light", fontScale = 1.5f)
compose.onNodeWithText("Dismiss").assertIsDisplayed()
compose.onNodeWithText("Retry").assertIsDisplayed()
val detail = compose.onNodeWithText(longError)
val scrollRange = detail.fetchSemanticsNode().config[SemanticsProperties.VerticalScrollAxisRange]
assertTrue(scrollRange.maxValue() > 0f)
detail.performTouchInput { swipeUp() }
compose.runOnIdle { assertTrue(scrollRange.value() > 0f) }
compose.onNodeWithText("Dismiss").assertIsDisplayed()
compose.onNodeWithText("Retry").assertIsDisplayed()
compose.onNode(isDialog()).captureRoboImage("build/ui-evidence/voice-error-landscape-light-large-text.png")
}
@Test
fun noError_doesNotShowDialog() {
renderError(error = null)
compose.onNode(isDialog()).assertDoesNotExist()
}
private fun renderError(
error: String? = PROVIDER_ERROR,
mode: VoicePresentationMode = VoicePresentationMode.Conversation,
theme: String = "dark",
fontScale: Float = 1f,
onClear: () -> Unit = {},
onMicTap: () -> Unit = {},
) {
var state by mutableStateOf(
VoiceUiState(
voiceMode = true,
state = if (error == null) VoiceState.Idle else VoiceState.Error,
interactionMode = InteractionMode.TapToTalk,
error = error,
),
)
compose.mainClock.autoAdvance = false
compose.setContent {
HermesRelayTheme(themePreference = theme, fontScale = fontScale) {
Surface(Modifier.fillMaxSize()) {
Box {
Text("Chat remains behind the modal")
VoiceModeOverlay(
uiState = state,
presentationMode = mode,
onMicTap = onMicTap,
onMicRelease = {},
onInterrupt = {},
onDismiss = {},
onModeChange = {},
onClearError = {
onClear()
state = state.copy(error = null, state = VoiceState.Idle)
},
)
}
}
}
}
compose.mainClock.advanceTimeBy(500)
}
private companion object {
const val PROVIDER_ERROR = "Hermes audio transcribe rejected that input - " +
"{\"detail\":\"No STT provider available. Install faster-whisper for free local " +
"transcription, configure HERMES_LOCAL_STT_COMMAND or install a local whisper CLI, " +
"set GROQ_API_KEY for free Groq Whisper, set MISTRAL_API_KEY for Mistral Voxtral " +
"Transcribe, configure xAI OAuth or set XAI_API_KEY for xAI Grok STT, set " +
"ELEVENLABS_API_KEY for ElevenLabs Scribe, or set VOICE_TOOLS_OPENAI_KEY or " +
"OPENAI_API_KEY for the OpenAI Whisper API.\"}"
}
}
@@ -0,0 +1,30 @@
package com.hermesandroid.relay.util
import kotlinx.coroutines.test.runTest
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
import org.robolectric.RuntimeEnvironment
import org.robolectric.annotation.Config
import java.io.File
@RunWith(RobolectricTestRunner::class)
@Config(sdk = [34])
class MediaCacheWriterTest {
@Test
fun cacheFile_promotesStreamedFileWithoutByteArrayHandoff() = runTest {
val context = RuntimeEnvironment.getApplication()
val source = File.createTempFile("hermes-media-test-", ".part", context.cacheDir)
.apply { writeText("streamed-media") }
val writer = MediaCacheWriter(context) { 32 }
val cached = writer.cacheFile(source, "audio/mpeg", "voice reply.mp3")
assertFalse(source.exists())
assertEquals("streamed-media", cached.readText())
writer.clear()
}
}
@@ -517,6 +517,40 @@ class ChatViewModelGatewayInboundTurnTest {
)
}
@Test
fun ownershipErrorAfterFreshSubmitStaysVisibleWithoutHistoryRecovery() {
val ownershipError =
"Session 20260612_120000_abc123 already has a live owner (webui, pid 42, running 0m). " +
"Only one surface at a time may run a session, because a second one would reason from stale history."
val historyReads = AtomicInteger(0)
viewModel.setProfileMessageLoader {
historyReads.incrementAndGet()
Result.success(emptyList())
}
viewModel.createNewChat()
viewModel.sendMessage("Keep this retryable")
gatewayHarness.awaitRpc("prompt.submit")
serverWs.send(
gatewayHarness.eventFrame(
"error",
buildJsonObject { put("message", ownershipError) },
"live-1",
),
)
awaitCondition { viewModel.chatFailure.value?.rawError == ownershipError }
assertFalse(handler.isStreaming.value)
assertFalse(viewModel.recoveringAnswer.value)
assertEquals(0, historyReads.get())
assertTrue(handler.messages.value.any { it.content == "Keep this retryable" })
assertFalse(
handler.messages.value.any {
it.content.contains("unfinished message was not found", ignoreCase = true)
},
)
}
@After
fun tearDown() {
DiagnosticsLog.clear()
@@ -879,6 +913,28 @@ class ChatViewModelGatewayInboundTurnTest {
assertEquals(2, calls.get())
}
@Test
fun reentrantSessionPublicationCannotKeepTheRefreshBatchAlive() {
val calls = AtomicInteger(0)
viewModel.setProfileSessionLister {
calls.incrementAndGet()
// Reproduce a downstream publication observer requesting another
// refresh while every successful directory read is still active.
viewModel.refreshSessions()
Result.success(listOf(SessionItem(id = "stable", title = "Stable session")))
}
viewModel.refreshSessions()
awaitCondition { calls.get() >= 2 }
Thread.sleep(150)
val settledCalls = calls.get()
Thread.sleep(150)
assertEquals(settledCalls, calls.get())
assertTrue(settledCalls <= 3)
assertEquals("stable", handler.sessions.value.singleOrNull()?.sessionId)
}
@Test
fun coalescedTrailingSuccessClearsTheInitialFailure() {
val calls = AtomicInteger(0)
@@ -1927,6 +1983,33 @@ class ChatViewModelGatewayInboundTurnTest {
assertFalse(gatewayHarness.rpcLog.any { it.first == "prompt.submit" })
}
@Test
fun gatewayInterimImmediatelyLeavesAStreamingConversationOwner() {
viewModel.sendMessage("Generate an image")
gatewayHarness.awaitRpc("prompt.submit")
serverWs.send(gatewayHarness.eventFrame("message.start", null, "live-resumed"))
serverWs.send(
gatewayHarness.eventFrame(
"message.interim",
buildJsonObject {
put("text", "Generating it now.")
put("already_streamed", false)
},
"live-resumed",
),
)
awaitCondition {
handler.messages.value.any { it.content == "Generating it now." }
}
assertTrue(handler.isStreaming.value)
val activeOwner = handler.messages.value.last()
assertTrue(activeOwner.isStreaming)
assertEquals(MessageRole.ASSISTANT, activeOwner.role)
assertTrue(activeOwner.content.isBlank())
}
@Test
fun queuedMainDispatchAdmitsBackgroundStartAfterLocalCompletion() {
viewModel.sendMessage("Local gateway turn")
@@ -3104,6 +3187,87 @@ class ChatViewModelGatewayInboundTurnTest {
assertTrue(viewModel.queuedMessages.value.isEmpty())
}
@Test
fun explicitQueueChoiceDoesNotAttemptRedirect() {
startTurnForQueueTest()
viewModel.sendMessage("Run next", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
assertEquals(listOf("Run next"), viewModel.queuedMessages.value)
assertFalse(gatewayHarness.rpcLog.any { it.first == "session.redirect" || it.first == "session.steer" })
}
@Test
fun deletingQueuedHeadReconnectsSuccessorToActiveTurn() {
startTurnForQueueTest()
viewModel.sendMessage("Remove me", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
viewModel.sendMessage("Keep me", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
viewModel.removeQueuedAt(0)
completeQueueTestTurn()
awaitCondition { gatewayHarness.rpcLog.any { it.first == "prompt.submit" && it.second["text"] == JsonPrimitive("Keep me") } }
assertFalse(gatewayHarness.rpcLog.any { it.first == "prompt.submit" && it.second["text"] == JsonPrimitive("Remove me") })
}
@Test
fun editingQueuedHeadDoesNotStrandSuccessor() {
startTurnForQueueTest()
viewModel.sendMessage("Edit me", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
viewModel.sendMessage("Keep me", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
assertEquals("Edit me", viewModel.takeQueuedForEdit(0))
completeQueueTestTurn()
awaitCondition { gatewayHarness.rpcLog.any { it.first == "prompt.submit" && it.second["text"] == JsonPrimitive("Keep me") } }
}
@Test
fun stopPausesQueueUntilExplicitResumeAndPersistsWholeChain() {
val store = MemoryCheckpointStore()
viewModel.setChatTurnCheckpointStore(store)
startTurnForQueueTest()
viewModel.sendMessage("First pending", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
viewModel.sendMessage("Second pending", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
viewModel.cancelStream()
shadowOf(Looper.getMainLooper()).idle()
assertEquals(listOf("First pending", "Second pending"), viewModel.queuedMessages.value)
assertTrue(viewModel.queuePaused.value)
assertFalse(gatewayHarness.rpcLog.any { it.first == "prompt.submit" })
awaitCondition { store.checkpoint?.queueOnly == true }
assertEquals(true, store.checkpoint?.queuePaused)
assertEquals(2, store.checkpoint?.queuedMessages?.size)
viewModel.resumeQueue()
awaitCondition { gatewayHarness.rpcLog.any { it.first == "prompt.submit" && it.second["text"] == JsonPrimitive("First pending") } }
assertFalse(viewModel.queuePaused.value)
assertEquals(listOf("Second pending"), viewModel.queuedMessages.value)
}
@Test
fun pausedQueueCheckpointRestoresWithoutRecoveringStoppedTurn() {
val store = MemoryCheckpointStore()
viewModel.setChatTurnCheckpointStore(store)
startTurnForQueueTest()
viewModel.sendMessage("Keep paused", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
viewModel.cancelStream()
awaitCondition { store.checkpoint?.queueOnly == true }
val saved = requireNotNull(store.checkpoint)
viewModel.clearQueue()
shadowOf(Looper.getMainLooper()).idle()
store.checkpoint = saved
val activationsBefore = gatewayHarness.rpcLog.count { it.first == "session.activate" }
viewModel.prewarmGateway()
awaitCondition { viewModel.queuedMessages.value == listOf("Keep paused") }
assertTrue(viewModel.queuePaused.value)
assertFalse(handler.isStreaming.value)
assertEquals(activationsBefore, gatewayHarness.rpcLog.count { it.first == "session.activate" })
}
private fun startTurnForQueueTest() {
viewModel.switchProfileContext(PROFILE_CONTEXT, STORED_SESSION_ID)
serverWs.send(gatewayHarness.eventFrame("message.start", null, "live-resumed"))
awaitCondition { handler.isStreaming.value }
}
private fun completeQueueTestTurn() {
serverWs.send(gatewayHarness.eventFrame("message.complete", buildJsonObject { put("text", "Original complete") }, "live-resumed"))
}
@Test
fun queuedMessageStaysWithOriginWhileAnotherRunningSessionCompletes() {
val secondSession = "stored-session-b"
@@ -3249,6 +3413,29 @@ class ChatViewModelGatewayInboundTurnTest {
)
}
@Test
fun connectionSwitchKeepsRunningCheckpointWhenQueueExists() {
val store = MemoryCheckpointStore()
val switches = MutableSharedFlow<String>(extraBufferCapacity = 1)
viewModel.setChatTurnCheckpointStore(store)
viewModel.switchProfileContext(PROFILE_CONTEXT, STORED_SESSION_ID)
viewModel.observeConnectionSwitches(switches)
viewModel.sendMessage("Keep the running turn recoverable")
gatewayHarness.awaitRpc("prompt.submit")
viewModel.sendMessage("Follow up on this connection", com.hermesandroid.relay.data.BusyMessageAction.QueueNext)
awaitCondition { viewModel.queuedMessages.value.size == 1 }
switches.tryEmit("connection-b")
awaitCondition { viewModel.queuedMessages.value.isEmpty() }
shadowOf(Looper.getMainLooper()).idle()
val checkpoint = requireNotNull(store.checkpoint)
assertFalse("A detached running turn must not become queue-only", checkpoint.queueOnly)
assertEquals("Keep the running turn recoverable", checkpoint.user.content)
assertEquals(listOf("Follow up on this connection"), checkpoint.queuedMessages.map { it.text })
assertFalse(gatewayHarness.rpcLog.any { it.first == "session.interrupt" })
}
@Test
fun deletingDestinationCancelsItsQueueAndEditRestoresOnlyThatItem() {
gatewayHarness.redirectStatus = "rejected"
@@ -3296,7 +3483,7 @@ class ChatViewModelGatewayInboundTurnTest {
}
@Test
fun retryQueuedForActiveRunIsCancelledWithThatRun() {
fun retryQueuedForActiveRunIsPausedWhenThatRunStops() {
gatewayHarness.redirectStatus = "rejected"
viewModel.switchProfileContext(
AgentDisplay.profileContextKey("connection-a", null),
@@ -3311,7 +3498,8 @@ class ChatViewModelGatewayInboundTurnTest {
viewModel.cancelStream()
gatewayHarness.awaitRpc("session.interrupt")
assertTrue(viewModel.queuedMessages.value.isEmpty())
assertEquals(listOf("Retry this turn"), viewModel.queuedMessages.value)
assertTrue(viewModel.queuePaused.value)
serverWs.send(
gatewayHarness.eventFrame(
@@ -3754,6 +3942,7 @@ class ChatViewModelGatewayInboundTurnTest {
@Test
fun passiveForegroundObservationNeverClaimsOrInterruptsDesktopTurn() {
val directoryReads = AtomicInteger(0)
val observerProfile = Profile(
name = "observer",
model = "model-a",
@@ -3821,7 +4010,12 @@ class ChatViewModelGatewayInboundTurnTest {
viewModel.backgroundSessionActivityStates.value["observer:$STORED_SESSION_ID"] ==
SessionActivityState.Working
}
viewModel.setProfileSessionLister {
directoryReads.incrementAndGet()
Result.success(emptyList())
}
gatewayHarness.activeSessionListPayload = activeSessionPayload("waiting")
val directoryReadsBeforeHistoryPublication = directoryReads.get()
viewModel.requestSessionActivityRefresh()
awaitCondition {
viewModel.backgroundSessionActivityStates.value["observer:$STORED_SESSION_ID"] ==
@@ -3839,6 +4033,9 @@ class ChatViewModelGatewayInboundTurnTest {
handler.messages.value.singleOrNull()?.content ==
"Desktop completed without Android attachment."
}
shadowOf(Looper.getMainLooper()).idleFor(4, TimeUnit.SECONDS)
Thread.sleep(100)
assertEquals(directoryReadsBeforeHistoryPublication, directoryReads.get())
ownershipMethods.forEach { method ->
assertEquals(
"passive foreground sent $method",
@@ -11,6 +11,7 @@ import com.hermesandroid.relay.network.upstream.DashboardApiClient
import com.hermesandroid.relay.network.upstream.models.MessageItem
import com.hermesandroid.relay.util.MediaCacheWriter
import io.mockk.coEvery
import io.mockk.coVerify
import io.mockk.mockk
import kotlinx.serialization.json.JsonPrimitive
import okhttp3.OkHttpClient
@@ -25,6 +26,7 @@ import org.robolectric.RobolectricTestRunner
import org.robolectric.RuntimeEnvironment
import org.robolectric.Shadows.shadowOf
import org.robolectric.annotation.Config
import java.io.File
@RunWith(RobolectricTestRunner::class)
@Config(sdk = [34])
@@ -52,7 +54,9 @@ class ChatViewModelMediaStateTest {
pairedTokenSnapshot = { "paired-session" },
)
cache = mockk()
coEvery { cache.cache(any(), any(), any()) } returns
coEvery { cache.cache(any<ByteArray>(), any(), any()) } returns
Uri.parse("content://com.axiomlabs.hermesrelay.fileprovider/hermes-media/photo.jpg")
coEvery { cache.cache(any<File>(), any(), any()) } returns
Uri.parse("content://com.axiomlabs.hermesrelay.fileprovider/hermes-media/photo.jpg")
handler = ChatHandler()
viewModel = ChatViewModel().also {
@@ -197,6 +201,8 @@ class ChatViewModelMediaStateTest {
}.attachments.single()
assertEquals("audio/mpeg", loaded.contentType)
assertEquals("test-voice-message.mp3", loaded.fileName)
coVerify(exactly = 1) { cache.cache(any<File>(), "audio/mpeg", "test-voice-message.mp3") }
coVerify(exactly = 0) { cache.cache(any<ByteArray>(), any(), any()) }
val request = dashboardServer.takeRequest()
assertEquals("/api/files/download", request.requestUrl?.encodedPath)
assertEquals(path, request.requestUrl?.queryParameter("path"))
+1 -1
View File
@@ -751,7 +751,7 @@ one-time pairing, the interactive PTY/TUI shell, client-side tool routing,
auto-reconnect with TOFU cert pinning, server-side session management, the
headless daemon, local diagnostics, and the optional Windows management tray.
What's next (see [ROADMAP.md](../ROADMAP.md#desktop-track) for the full track):
What's next (see [docs/project/ROADMAP.md](../docs/project/ROADMAP.md#desktop-track) for the full track):
- Service installers — `install-service-{win,linux,mac}` to register the daemon with `sc.exe` / systemd user unit / `launchd` so it auto-starts on login.
- Multi-client server-side routing — today a connected desktop client is single-slot; allow laptop + home-desktop + work-box attached simultaneously with per-client tool dispatch via a new hermes-agent `ContextVar`.
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "@hermes-relay/cli",
"version": "0.4.0-beta.6",
"version": "0.4.0-beta.7",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@hermes-relay/cli",
"version": "0.4.0-beta.6",
"version": "0.4.0-beta.7",
"license": "MIT",
"bin": {
"hermes-relay": "bin/hermes-relay.js"
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@hermes-relay/cli",
"version": "0.4.0-beta.6",
"version": "0.4.0-beta.7",
"description": "Thin-client CLI for Hermes-Relay — talk to a remote Hermes agent over WSS with pairing auth, stream-renders tool calls and responses to plain stdout.",
"type": "module",
"bin": {
+1 -1
View File
@@ -1,2 +1,2 @@
// Regenerated from package.json by gen:version script. Do not edit by hand.
export const VERSION = "0.4.0-beta.6" as const
export const VERSION = "0.4.0-beta.7" as const
+1 -1
View File
@@ -1232,7 +1232,7 @@ checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "hermes-relay-tray"
version = "0.4.0-beta.6"
version = "0.4.0-beta.7"
dependencies = [
"base64 0.22.1",
"serde",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "hermes-relay-tray"
version = "0.4.0-beta.6"
version = "0.4.0-beta.7"
description = "Compact Windows management UI for Hermes-Relay CLI"
authors = ["Axiom Labs"]
edition = "2021"
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "@hermes-relay/tray-ui",
"version": "0.4.0-beta.6",
"version": "0.4.0-beta.7",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@hermes-relay/tray-ui",
"version": "0.4.0-beta.6",
"version": "0.4.0-beta.7",
"dependencies": {
"@tauri-apps/api": "^2.8.0",
"lucide-react": "^0.468.0",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@hermes-relay/tray-ui",
"private": true,
"version": "0.4.0-beta.6",
"version": "0.4.0-beta.7",
"type": "module",
"scripts": {
"dev": "vite --host 127.0.0.1",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "Hermes-Relay CLI UI",
"version": "0.4.0-beta.6",
"version": "0.4.0-beta.7",
"identifier": "com.axiomlabs.hermes-relay-tray",
"build": {
"beforeDevCommand": "npm run dev",
+6 -3
View File
@@ -1,6 +1,7 @@
# Hermes Relay Android UI Design Reference
Status: design reference for future Android, Quest, and pairing-flow work.
Status: design reference for Android and pairing-flow work. The quarantined
Quest prototype is retained under `experiments/quest/` for historical context.
Use this document when a future session asks for mobile UI polish, pairing flow changes, QR scanning, connection setup, onboarding, terminal cockpit, or "make it feel more like Orca." It is intentionally a reference, not an implementation task list.
@@ -11,7 +12,6 @@ This project should keep its native Android stack:
- Kotlin + Jetpack Compose + Material 3 for app chrome and interaction flows.
- CameraX + ML Kit for QR scanning.
- Android WebView only where it is already the correct tool, such as xterm.
- Meta Spatial SDK for Quest/XR surfaces where needed.
Do not propose a React Native or Expo migration just because Orca's mobile app uses that stack. The useful lesson from Orca is the visual discipline and pairing UX, not the framework choice.
@@ -205,7 +205,10 @@ Target layout:
### Quest / XR
Use this reference for Quest visual tone, but do not force the phone layout into XR. Quest should keep the spatial cockpit concept:
The inactive Quest prototype is quarantined under `experiments/quest/` and is
not part of the production Android build or normal CI. If development resumes,
use this reference for its visual tone, but do not force the phone layout into
XR. The spatial cockpit concept was:
- MorphingSphere and voice pipeline remain first-class.
- Terminal panels should be spatial, readable, and anchored.
+28 -28
View File
@@ -30,8 +30,8 @@ Status: **passed**
### Compared
- Source: `C:\Users\Bailey\.codex\generated_images\019f6640-e6dc-7c33-8aca-a1610e2a19f0\call_JzVlJKlG8KWTJRo2xup7oTlq.png`
- Implementation: `C:\Users\Bailey\.codex\visualizations\2026\07\25\agent-drawer-audit\passport-qa-final.png`
- Source: `<session-generated-reference>`
- Implementation: `<session-visualization-artifact>`
- Source size: 852 x 1846 px
- Device viewport: Android, 1080 x 2340 px, font scale 1.0
- State: Agent tab, Victor pinned profile, disconnected gateway
@@ -105,11 +105,11 @@ final result: passed
## Assistant overlay
- Reference: `C:\Users\Bailey\.codex\generated_images\019fb003-68ea-7052-85c7-3745fa7e838e\call_jPwpDr9QfaCDA6k2c01StBYe.png`
- Reference: `<session-generated-reference>`
- Implementation captures:
- `C:\Users\Bailey\.codex\visualizations\2026\07\29\019fb003-68ea-7052-85c7-3745fa7e838e\assistant-live-compact.png`
- `C:\Users\Bailey\.codex\visualizations\2026\07\29\019fb003-68ea-7052-85c7-3745fa7e838e\assistant-live-expanded.png`
- Combined comparison: `C:\Users\Bailey\.codex\visualizations\2026\07\29\019fb003-68ea-7052-85c7-3745fa7e838e\assistant-design-comparison.png`
- `<session-visualization-artifact>`
- `<session-visualization-artifact>`
- Combined comparison: `<session-visualization-artifact>`
- Device viewport: 1080 × 2340, portrait, Samsung SM-S938U
- States reviewed: system assistant compact overlay and expanded overlay over a non-Hermes app
@@ -146,18 +146,18 @@ final result: blocked
## Appearance customization and visual assets
- Selected reference: `C:\Users\Bailey\.codex\generated_images\019fe6f7-920d-7be1-b287-608a0bc2ff49\exec-facb08cb-3d8f-400a-b6ff-6f1a11a74d95.png`
- Selected reference: `<session-generated-reference>`
- Reference pixels: 852 x 1843.
- Final real-screen captures:
- `C:\Users\Bailey\.codex\worktrees\af0f\hermes-relay\app\build\store-shots\05_themes.png`
- `C:\Users\Bailey\.codex\worktrees\af0f\hermes-relay\app\build\store-shots\05_theme_customizer.png`
- `C:\Users\Bailey\.codex\worktrees\af0f\hermes-relay\app\build\store-shots\08_appearance.png`
- `app/build/store-shots/05_themes.png`
- `app/build/store-shots/05_theme_customizer.png`
- `app/build/store-shots/08_appearance.png`
- Implementation pixels: 1080 x 2160, portrait Robolectric/Roborazzi capture of the production Compose screen at its 360 dp Android viewport.
- Normalization: the selected reference was scaled to 1080 px wide and cropped to the same 2160 px viewport for the full-view comparison. The reference's taller aspect ratio remains visible as an expected viewport difference; the focused customizer capture verifies the below-fold editor content.
- Comparison evidence:
- Initial full-view comparison: `C:\Users\Bailey\.codex\visualizations\2026\08\09\019fe6f7-920d-7be1-b287-608a0bc2ff49\appearance-reference-v-current.png`
- Final normalized full-view comparison: `C:\Users\Bailey\.codex\visualizations\2026\08\09\019fe6f7-920d-7be1-b287-608a0bc2ff49\appearance-reference-v-current-2.png`
- Focused customizer comparison: `C:\Users\Bailey\.codex\visualizations\2026\08\09\019fe6f7-920d-7be1-b287-608a0bc2ff49\appearance-customizer-reference-v-current.png`
- Initial full-view comparison: `<session-visualization-artifact>`
- Final normalized full-view comparison: `<session-visualization-artifact>`
- Focused customizer comparison: `<session-visualization-artifact>`
- State: Hermes Relay dark preset, customizer expanded. The focused capture expands the real control by click and scrolls its Shape row into view.
### Comparison history
@@ -187,15 +187,15 @@ final result: passed
## Conversation voice dock
- Reference: `C:\Users\Bailey\.codex\generated_images\019fba42-1e93-78e1-9cd6-6919fb52d5bc\exec-89feab69-9caf-4084-b107-6bff43e511d6.png`
- Reference: `<session-generated-reference>`
- Implementation captures:
- `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit\implemented_conversation_final.png`
- `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit\conversation_refined_expanded.png`
- `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit\focus_final_expanded.png`
- Combined comparison: `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit\voice-dock-design-comparison-final.png`
- Expanded before/after comparison: `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit\voice-expanded-before-after.png`
- Entry transition recording: `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit\voice_entry_transition.mp4`
- Entry transition frame sequence: `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit\voice_entry_transition_frames.png`
- `<session-visualization-artifact>`
- `<session-visualization-artifact>`
- `<session-visualization-artifact>`
- Combined comparison: `<session-visualization-artifact>`
- Expanded before/after comparison: `<session-visualization-artifact>`
- Entry transition recording: `<session-visualization-artifact>`
- Entry transition frame sequence: `<session-visualization-artifact>`
- Reference size: 853 x 1844 px
- Device viewport: 1080 x 2340, portrait, Samsung SM-S938U
- States reviewed: collapsed idle dock, expanded Tap-mode controls, collapsed and expanded Focus header, Focus-to-Conversation transition
@@ -229,8 +229,8 @@ copying the mockup's illustrative content.
6. Compared the original and refined expanded Conversation and Focus states together; the final panels remove metadata fragmentation, header truncation, and avatar bleed-through.
7. Recorded a persisted-Focus voice entry, inspected the 30 fps frame sequence, and verified that Conversation appears first through a continuous composer expansion before Focus is offered as an explicit action.
8. Re-audited the installed Standard-mode drawer in both presentations. The final Conversation and Focus captures are:
- `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit-standard\21-final-expanded.png`
- `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit-standard\24-final-focus-expanded.png`
- `<session-visualization-artifact>`
- `<session-visualization-artifact>`
The route summary is now two clearly separated lines (`Standard mode · Victor` and `xAI TTS · Leo`), with duplicate provider/model aliases removed and configuration provenance left to Voice Settings.
9. Verified the Conversation long-press menu on-device in `22-final-speak-menu.png`: completed assistant messages expose Copy, Quote in reply, and Speak response without stacking Android's text-selection toolbar over the app menu.
10. Verified the connection footer remains present under the Conversation composer. Focus remains the only in-app voice presentation that hides it.
@@ -242,13 +242,13 @@ final result: passed
## System voice overlay redesign
- Selected reference: `C:\Users\Bailey\.codex\generated_images\019fba42-1e93-78e1-9cd6-6919fb52d5bc\exec-6a606b04-58c3-4a6d-b11b-cd25f99a047d.png`
- Selected reference: `<session-generated-reference>`
- Source concepts:
- `C:\Users\Bailey\.codex\generated_images\019fba42-1e93-78e1-9cd6-6919fb52d5bc\exec-31a5512a-9cb7-455a-b111-d5797ab09e93.png`
- `C:\Users\Bailey\.codex\generated_images\019fba42-1e93-78e1-9cd6-6919fb52d5bc\exec-bbfd616e-3847-4a2e-bbd0-65deb5e4e98c.png`
- `<session-generated-reference>`
- `<session-generated-reference>`
- Baseline captures:
- `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit-standard\29-current-overlay-expanded.png`
- `C:\Users\Bailey\.codex\visualizations\2026\07\31\019fba42-1e93-78e1-9cd6-6919fb52d5bc\voice-ui-audit-standard\30-current-overlay-expanded.png`
- `<session-visualization-artifact>`
- `<session-visualization-artifact>`
- Device viewport: 1080 x 2340, portrait, Samsung SM-S938U
- States to review: collapsed system overlay and expanded system overlay over a non-Hermes app
+10 -4
View File
@@ -267,7 +267,7 @@ not require an API endpoint or API bearer when dashboard chat is ready.
### 12. Android as a Hermes Platform/Channel — "Threads" (Shipped 2026-06-28; unified-session model 2026-06-29)
> **Status (2026-08-12):** SHIPPED as the `phone` platform plugin (`plugin/phone_platform.py`) — registered via `ctx.register_platform` with **no fork** (the ~16-file upstream change sketched below was avoided; the original research predates the open plugin-platform registry). Two-way reply is device-verified. Agent-facing entry is `send_message target=phone` (the stale `target=mobile:<device_id>` syntax below is superseded); standalone cron delivery uses the registered sender, and the adapter publishes its canonical home destination through upstream's channel directory. Live cron certification remains tracked in TODO.md.
> **Status (2026-08-12):** SHIPPED as the `phone` platform plugin (`plugin/phone_platform.py`) — registered via `ctx.register_platform` with **no fork** (the ~16-file upstream change sketched below was avoided; the original research predates the open plugin-platform registry). Two-way reply is device-verified. Agent-facing entry is `send_message target=phone` (the stale `target=mobile:<device_id>` syntax below is superseded); standalone cron delivery uses the registered sender, and the adapter publishes its canonical home destination through upstream's channel directory. Live cron certification remains tracked in docs/project/TODO.md.
>
> **Decision (2026-06-29) — unified-session "Threads", not a separate surface:** the proactive agent↔phone conversation is **not** a separate app lane/tab/segment. It is a **source-tagged session inside the one Chat surface** — a **Thread** (`source=phone`). The three things distinguishing a Thread from a normal gateway chat are *session properties*, not a separate UI: (a) the agent can initiate a turn, (b) it rides the relay `proactive` transport and is relay-gated, (c) it's a standing/named DM. **Scrollback = the gateway session store** (same read path Chat uses); **live receive = the relay `proactive` push** (→ notification); **send = `proactive.reply`**. The local `ProactiveInboxStore` is demoted to a live-push cache + outbox (no parallel history). The Thread capability is surfaced in the **connection best-path/capability UI** (a relay-tier capability, like terminal/bridge/voice) and as a clean **Threads** entry (thread-spool icon, NOT a phone glyph) pinned atop the session drawer when active — never a connection-wizard step. Degrades cleanly: no relay plugin → no `source=phone` sessions → Chat is unchanged (standard-path-safe). This **supersedes the earlier "separate Agent lane / 4th nav segment" sketch** and folds in the "show chat source/platform attribution in Chat" goal in one stroke.
>
@@ -3240,7 +3240,7 @@ escalation remains reserved and reports disabled. The optional driver is not
bundled or updated by Hermes-Relay, and Hermes forces driver telemetry off for
every child process it starts. The legacy engine remains the default and the
fail-closed fallback while the live Windows acceptance and remaining scope,
redaction, and grant-bridge hardening gates tracked in `TODO.md` stay open.
redaction, and grant-bridge hardening gates tracked in `docs/project/TODO.md` stay open.
**Second-phase refinement (2026-08-13).** CUA is the preferred/default setting
for new structured-control sessions; the original Windows input path is named
@@ -3743,7 +3743,7 @@ the session, duplicate submission, wrong-session events, or an arbitrary timer.
Normal terminal delivery is unchanged, queued turns retain their existing
ownership, Relay remains optional, and unmodified upstream compatibility is
preserved. Physical certification across the reported device/network matrix
remains tracked in `TODO.md`.
remains tracked in `docs/project/TODO.md`.
---
@@ -4118,7 +4118,7 @@ visible without mislabeling their parent turn. Older Gateways remain usable
but show Unavailable when no exact local terminal truth exists. Declarative
Gateway scenarios cover all four upstream states, complete-snapshot
disappearance, client-side profile isolation, and method-not-found; physical
and current-host certification remains tracked in `TODO.md`. An upstream
and current-host certification remains tracked in `docs/project/TODO.md`. An upstream
profile field/filter or explicitly owned aggregate activity route would remove
the remaining ambiguity for multi-profile clients.
@@ -4226,6 +4226,12 @@ longer current.
hidden-source exclusions server-side. Near-end scrolling appends subsequent
50-row `offset` pages under the same owner/generation; older rows never block
the first-open critical path.
- Routine transcript open, restore, and post-turn reconciliation request one
latest 500-row page, matching the rows Android can retain. Complete
oldest-first pagination is reserved for positional recovery that proves it
needs older anchors. Each Dashboard JSON response is rejected above the
Android byte limit before parsing, and aggregate complete reads retain their
independent row/payload ceilings.
- A refresh over existing rows is quiet. The exact connection/profile cache
remains visible until authoritative replacement rows arrive. An uncached
profile may show loading, but a failed read never means "no sessions."
+12
View File
@@ -68,9 +68,11 @@ the upstream contract identifiers it depends on.
|---|---|
| `initial_history_bind` | Durable, profile-scoped history is already available when the client resumes and first binds its rendered transcript |
| `ordinary_turn` | Normal message start, deltas, completion, and persisted history |
| `ownership_rejection` | A submit acknowledged before the defense-in-depth ownership check emits the canonical terminal refusal; no user/model row is persisted and clients must not enter history recovery |
| `compaction_status` | Compaction status is client-visible before terminal completion and may repeat as a heartbeat |
| `rapid_tools_interims` | Rapid chunks, reasoning, tool activity, and interim assistant boundaries |
| `queued_follow_up` | Two explicitly owned turns and ordered queue drainage |
| `queued_stop_resume` | Explicit queue choice, removal of a predecessor, Stop/pause, lifecycle return, and explicit Resume through production Android controls |
| `scope_rejection_inputs` | Exact, foreign, and unscoped event inputs |
| `terminal_gap_activate` | Socket closes after live output; replacement `session.activate` reports `running=false`; history is authoritative |
| `terminal_gap_active_list` | An exact Android-owned turn receives deltas but no terminal; `session.active_list` reports the same live/durable owner idle; history is authoritative |
@@ -123,6 +125,16 @@ unreviewed endpoint.
## Physical-device certification
The queue-control instrumentation uses the `queued_stop_resume` fixture on
loopback with an explicit ADB reverse. Run only
`GatewayExternalFixtureInstrumentedTest#queuedStopResume_preservesWorkAcrossLifecycleAndUsesExplicitResume`
with `gatewayFixtureBaseUrl` set to that fixture. It verifies two submits, one
interrupt, no redirect, and a single completed resumed reply. This is a separate
scenario from the terminal-gap certification runner below, whose evidence
contract requires socket loss and activation. The fixture emits the upstream
interrupted terminal before the next turn; omitting it would exercise a broken
server contract rather than the queue controls.
Inspect the exact plan first. The runner requires an explicit serial or
transport ID, targets only `com.axiomlabs.hermesrelay.sideload`, and performs no
installation unless its corresponding install flag is supplied.
+11 -11
View File
@@ -13,12 +13,12 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "227a53fc7a1fd3cf113b56b507c7d65372769db179f4772be892138ff0ac8060",
"main": "1efc1c169120e525ddd1484f9ac33c15d27f558d74b766e7c04f42abd8169255",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
"android": "complete",
"readme": "english-fallback",
"readme": "maintained-summary",
"user_docs": "core-pages",
"website": "complete"
},
@@ -48,12 +48,12 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "227a53fc7a1fd3cf113b56b507c7d65372769db179f4772be892138ff0ac8060",
"main": "1efc1c169120e525ddd1484f9ac33c15d27f558d74b766e7c04f42abd8169255",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
"android": "complete",
"readme": "english-fallback",
"readme": "maintained-summary",
"user_docs": "core-pages",
"website": "complete"
},
@@ -72,12 +72,12 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "227a53fc7a1fd3cf113b56b507c7d65372769db179f4772be892138ff0ac8060",
"main": "1efc1c169120e525ddd1484f9ac33c15d27f558d74b766e7c04f42abd8169255",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
"android": "complete",
"readme": "english-fallback",
"readme": "maintained-summary",
"user_docs": "core-pages",
"website": "complete"
},
@@ -96,12 +96,12 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "227a53fc7a1fd3cf113b56b507c7d65372769db179f4772be892138ff0ac8060",
"main": "1efc1c169120e525ddd1484f9ac33c15d27f558d74b766e7c04f42abd8169255",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
"android": "complete",
"readme": "english-fallback",
"readme": "maintained-summary",
"user_docs": "core-pages",
"website": "complete"
},
@@ -120,12 +120,12 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "227a53fc7a1fd3cf113b56b507c7d65372769db179f4772be892138ff0ac8060",
"main": "1efc1c169120e525ddd1484f9ac33c15d27f558d74b766e7c04f42abd8169255",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
"android": "complete",
"readme": "english-fallback",
"readme": "maintained-summary",
"user_docs": "english-fallback",
"website": "english-fallback"
}
@@ -135,7 +135,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "227a53fc7a1fd3cf113b56b507c7d65372769db179f4772be892138ff0ac8060",
"main": "1efc1c169120e525ddd1484f9ac33c15d27f558d74b766e7c04f42abd8169255",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
+11 -7
View File
@@ -78,21 +78,25 @@ structurally complete but semantically stale translation.
9. Test the locale on an emulator or device, including in-app and Android-system
language switching, process restart, text expansion, and
accessibility.
10. Add the language to the README language links and status registry. New
AI-assisted locales start as `ai-translated` with empty `review_refs`.
10. Add a compact `docs/readme/README.<locale>.md` entrypoint, link it from the
root README language list, and update the status registry. New AI-assisted
locales start as `ai-translated` with empty `review_refs`.
## README and user documentation
`README.md` remains canonical. Translations use separate files such as
`README.zh-CN.md` and link back to English. Product documentation can be rolled
out by locale under `user-docs/<locale>/`; untranslated technical references
should link to the canonical English page rather than copying stale content.
`README.md` remains canonical. Translated entrypoints live under
`docs/readme/` as `README.<locale>.md` and link back to English. They are
deliberately compact onboarding and core-feature summaries, not full copies of
the fast-moving English README. Product documentation can be rolled out by
locale under `user-docs/<locale>/`; untranslated technical references should
link to the canonical English page rather than copying stale content.
`docs/localization-status.json` is the authoritative per-locale and per-surface
status. README and user-documentation translations may follow app translation;
maintainer `docs/` and ADRs remain canonical English.
The public documentation currently localizes a deliberately bounded first-run
All shipped non-English Android locales have a compact translated README
entrypoint. The public documentation localizes a deliberately bounded first-run
set for German, Spanish, Japanese, Brazilian Portuguese, and Simplified Chinese.
Russian currently falls back to the canonical English documentation:
+1 -1
View File
@@ -513,7 +513,7 @@ reply completes. A reaction overlays whatever else would show (including
### Forthcoming behavior (designed, not yet rendered)
These are specified so authors can plan, but the renderer doesn't drive them
yet — they're tracked in `TODO.md`. Authoring the clips/flags now is harmless.
yet — they're tracked in `docs/project/TODO.md`. Authoring the clips/flags now is harmless.
- **`attention` reaction** — a one-shot when a notification arrives. Reserved: it
needs a host event the pet doesn't yet receive (unlike `greet`/`done`, which
@@ -2,14 +2,14 @@
> **Purpose.** Detailed implementation plan for the v0.4 bridge feature expansion wave. Each work unit here is self-contained enough that an agent can pick it up and execute independently, including per-unit Agent briefs, file lists, acceptance criteria, and reference-implementation pointers.
>
> **Why this file exists.** The high-level [`ROADMAP.md`](../../ROADMAP.md) at the repo root lists the milestones for this wave as brief bullets; this file is where the scoping detail lives. While this plan is active, it's the source of truth for the agent team. Once every unit here has shipped, the plan file gets archived or deleted and the shipped items live on in [`CHANGELOG.md`](../../CHANGELOG.md) and [`DEVLOG.md`](../../DEVLOG.md).
> **Why this file exists.** The high-level [`docs/project/ROADMAP.md`](../../docs/project/ROADMAP.md) lists the milestones for this wave as brief bullets; this file is where the scoping detail lives. While this plan is active, it's the source of truth for the agent team. Once every unit here has shipped, the plan file gets archived or deleted and the shipped items live on in [`CHANGELOG.md`](../../CHANGELOG.md) and [`docs/project/DEVLOG.md`](../../docs/project/DEVLOG.md).
>
> **Origin.** Compiled 2026-04-13 from a comparison pass against [raulvidis/hermes-android](https://github.com/raulvidis/hermes-android) plus prior research. Scope covers bridge-channel tool expansion, reliability patterns, and one documentation/skill addition — deferred and research-horizon items live in [`ROADMAP.md`](../../ROADMAP.md), not here.
> **Origin.** Compiled 2026-04-13 from a comparison pass against [raulvidis/hermes-android](https://github.com/raulvidis/hermes-android) plus prior research. Scope covers bridge-channel tool expansion, reliability patterns, and one documentation/skill addition — deferred and research-horizon items live in [`docs/project/ROADMAP.md`](../../docs/project/ROADMAP.md), not here.
>
> **Related files.**
> - [`ROADMAP.md`](../../ROADMAP.md) — milestone-level view of where this plan sits in the broader arc
> - [`docs/project/ROADMAP.md`](../../docs/project/ROADMAP.md) — milestone-level view of where this plan sits in the broader arc
> - [`CHANGELOG.md`](../../CHANGELOG.md) — cumulative release history
> - [`DEVLOG.md`](../../DEVLOG.md) — session-by-session narrative log
> - [`docs/project/DEVLOG.md`](../../docs/project/DEVLOG.md) — session-by-session narrative log
> - [`CLAUDE.md`](../../CLAUDE.md) — project conventions every implementing agent should read first
> - [`docs/spec.md`](../spec.md) — formal architecture spec
> - [`docs/decisions.md`](../decisions.md) — architecture decision records (ADRs)
@@ -30,7 +30,7 @@
| **P** | Architectural pattern / reliability improvement, not a new tool. |
| **Doc** | Documentation surface change. |
Longer-term / research / aspirational items live in [`ROADMAP.md`](../../ROADMAP.md), not in this plan.
Longer-term / research / aspirational items live in [`docs/project/ROADMAP.md`](../../docs/project/ROADMAP.md), not in this plan.
**Effort sizing** (scope, not duration): **S** = trivial single-file change, **M** = multi-file change touching 2–4 layers, **L** = new subsystem or cross-cutting work.
+7 -7
View File
@@ -9,16 +9,16 @@
> **Related files.**
> - [`CLAUDE.md`](../../CLAUDE.md) — project conventions every implementing agent reads first
> - [`docs/spec.md`](../spec.md) — protocol / voice surface reference
> - [`ROADMAP.md`](../../ROADMAP.md) — where this pass sits in the broader arc
> - [`docs/project/ROADMAP.md`](../../docs/project/ROADMAP.md) — where this pass sits in the broader arc
> - [`docs/plans/2026-04-13-bridge-feature-expansion.md`](./2026-04-13-bridge-feature-expansion.md) — precedent for plan-file format
> - [`DEVLOG.md`](../../DEVLOG.md) — append session entry on completion
> - [`docs/project/DEVLOG.md`](../../docs/project/DEVLOG.md) — append session entry on completion
> - Upstream reference: `~/.hermes/hermes-agent/tools/tts_tool.py` on hermes-host (`you@hermes-host`)
## How to use this file
1. **Bailey:** walk each work unit, mark status checkbox (`[x] Now` / `[x] Next` / `[x] Later` / `[x] Skip`). Leave inline notes to override scope.
2. **Orchestrator agent:** creates the worktree + branch per [Worktree setup](#worktree-setup), then dispatches work units per the [Sequencing](#sequencing--parallelism) plan. The orchestrator owns the working tree; subagents are given narrow, self-contained tasks and asked to return diffs or land commits on the shared branch.
3. **Session end:** run acceptance checklist, update `DEVLOG.md` + `CHANGELOG.md`, open one PR for the whole branch with all unit IDs in the body.
3. **Session end:** run acceptance checklist, update `docs/project/DEVLOG.md` + `CHANGELOG.md`, open one PR for the whole branch with all unit IDs in the body.
## Worktree setup
@@ -56,7 +56,7 @@ cd ../hermes-android-voice
| **Wave 2** | V2 | Serial | Extends `VoiceViewModel.stripMarkdown()` → must land before V3 because V3's coalesce logic runs against sanitized text. |
| **Wave 3** | V3 | Serial | Chunking changes to `VoiceViewModel.kt` — conflicts with V4. |
| **Wave 4** | V4 | Serial | Pipelining rewrite of `VoiceViewModel.startTtsConsumer()` — takes the output of V3's chunker as input. |
| **Wave 5** | Doc1 | Serial | `DEVLOG.md` + `CHANGELOG.md` + `user-docs/` updates after code stabilizes. |
| **Wave 5** | Doc1 | Serial | `docs/project/DEVLOG.md` + `CHANGELOG.md` + `user-docs/` updates after code stabilizes. |
**Shared-file hot-spot.** `VoiceViewModel.kt` is touched by V2, V3, and V4. These **must** serialize — do not parallelize them. The orchestrator should handle them in sequence (V2→V3→V4) as three commits from one agent session, not three concurrent subagents.
@@ -341,14 +341,14 @@ cd ../hermes-android-voice
**Summary.** Capture the wave in the append-only log, bump CHANGELOG with an `[Unreleased]` entry, refresh user-docs voice-mode troubleshooting, update spec if the TTS pipeline section exists.
**Scope / Acceptance criteria.**
- `DEVLOG.md`: append a 2026-04-16 session entry summarizing what shipped and the diagnosis chain that motivated it.
- `docs/project/DEVLOG.md`: append a 2026-04-16 session entry summarizing what shipped and the diagnosis chain that motivated it.
- `CHANGELOG.md`: `[Unreleased] — Changed` entry: "Voice output quality — switched to ElevenLabs flash streaming model, added client+relay text sanitization, coalesced short sentences, prefetched next sentence during playback, and swapped to ExoPlayer for gapless concatenation." Use conventional-changelog tone.
- `user-docs/`: if there's a voice-mode page, add a "What changed in voice mode" note. If not, don't create one just for this.
- `docs/spec.md`: if there's a voice-pipeline section, update the flow diagram or prose to reflect the prefetch + gapless architecture.
- No entry in `ROADMAP.md` — the wave is complete, not milestoned.
- No entry in `docs/project/ROADMAP.md` — the wave is complete, not milestoned.
**Files to touch.**
- `DEVLOG.md`
- `docs/project/DEVLOG.md`
- `CHANGELOG.md`
- `user-docs/*` (conditional)
- `docs/spec.md` (conditional)
+2 -2
View File
@@ -278,13 +278,13 @@
**Summary.** Capture the wave. Unlike the voice-quality-pass docs pass, this one DOES update `user-docs/features/voice.md` because barge-in is a user-facing feature with a settings UI they need to discover.
**Scope.**
- `DEVLOG.md`: append 2026-04-17 entry (second of the day) summarizing the barge-in stack.
- `docs/project/DEVLOG.md`: append 2026-04-17 entry (second of the day) summarizing the barge-in stack.
- `CHANGELOG.md`: add to `[Unreleased]` under an `### Added — Barge-in` section.
- `docs/spec.md` Phase V section: reference the new barge-in architecture (duplex audio, VAD, AEC, resume).
- `user-docs/features/voice.md`: add a new "## Barge-in (interrupt the agent)" section with screenshot placeholder, settings description, device compatibility note.
**Files to touch.**
- `DEVLOG.md`, `CHANGELOG.md`, `docs/spec.md`, `user-docs/features/voice.md`.
- `docs/project/DEVLOG.md`, `CHANGELOG.md`, `docs/spec.md`, `user-docs/features/voice.md`.
**Dependencies.** All B units committed.
+1 -1
View File
@@ -310,7 +310,7 @@ Add option (A) to R3's acceptance criteria: `handle_sessions_list` gains a loopb
**Root-level:**
- `CHANGELOG.md` — new `[Unreleased]` entry under `### Added`: "Dashboard plugin with four tabs: Relay Management, Bridge Activity, Push Console (stub), Media Inspector" + brief bullet list of the three new relay routes.
- `DEVLOG.md` — session entry for 2026-04-18 dashboard plugin work: Context, Decision summary, Implementation notes (one-liner per wave), Deferred (push console needs FCM).
- `docs/project/DEVLOG.md` — session entry for 2026-04-18 dashboard plugin work: Context, Decision summary, Implementation notes (one-liner per wave), Deferred (push console needs FCM).
- `README.md` — add one line under Quick Start mentioning the dashboard tab becomes available after gateway restart.
- `CLAUDE.md` — add `plugin/dashboard/` to Repository Layout; add four Key Files entries for the dashboard plugin (manifest.json, plugin_api.py, src/index.jsx, dist/index.js one-liners).
+1 -1
View File
@@ -100,7 +100,7 @@ Confirm exact invocation by reading `hermes_cli/main.py:1034` area (agent will d
### Phase 4 — Polish + docs *(post-smoke)*
- Update `~/.hermes/hermes-relay/README.md` with desktop install section.
- Update hermes-relay `CHANGELOG.md` `[Unreleased]`.
- Update hermes-relay `DEVLOG.md` with session summary.
- Update hermes-relay `docs/project/DEVLOG.md` with session summary.
- Bump version via `scripts/bump-version.sh 0.8.0-alpha`.
- User-docs page in `user-docs/guide/desktop.md`.
@@ -34,7 +34,7 @@
## Integration sequence
1. Update ROADMAP.md with the alpha.6 scope + deferred items.
1. Update docs/project/ROADMAP.md with the alpha.6 scope + deferred items.
2. Launch 6 parallel agents (A–F) with isolated file ownership.
3. I integrate cli.ts + desktop_tool.py after all agents report.
4. Expand `npm run smoke` to cover new subcommands.
@@ -262,7 +262,7 @@ Tool calls must appear live without leaving voice mode.
### Wave 7 - Documentation and operator guidance
- Update `user-docs/features/voice.md`.
- Update `TODO.md` to mark the realtime-agent TODO as planned/linked.
- Update `docs/project/TODO.md` to mark the realtime-agent TODO as planned/linked.
- Add troubleshooting for auth/session binding, provider disconnect fallback, and why tools still run through Hermes.
- Add DEVLOG entry after implementation lands.

Some files were not shown because too many files have changed in this diff Show More