Compare commits
23
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
77e34c2c02 | ||
|
|
f4ee409106 | ||
|
|
bfb608bea6 | ||
|
|
95a95fe7d2 | ||
|
|
1bdf2ae71b | ||
|
|
f9e7a2f320 | ||
|
|
988fac8522 | ||
|
|
d4832a6a38 | ||
|
|
f9c8736e5b | ||
|
|
a815dd33fa | ||
|
|
cc9c75a636 | ||
|
|
43179e03c0 | ||
|
|
693ac4ed64 | ||
|
|
f94d663ac4 | ||
|
|
d1a21bd42e | ||
|
|
7ee2d73010 | ||
|
|
682bde84fe | ||
|
|
16bdbe5f44 | ||
|
|
f43fba9fed | ||
|
|
d1745413fd | ||
|
|
1b7a8025c3 | ||
|
|
d92a87483e | ||
|
|
0bea626ed8 |
@@ -10,6 +10,31 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
|
||||
|
||||
- **Windows-trusted certificates work in the desktop CLI.** The packaged Windows binary and newer Node runtimes add the Windows certificate store without dropping bundled or operator-supplied roots, while TLS verification and Relay certificate pinning remain enforced.
|
||||
|
||||
## [Android 1.5.2] - 2026-07-28
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Dashboard sign-in completes across supported providers and network routes.** Self-hosted OIDC stays on the dashboard cookie flow, while Nous Portal opens in the system browser and completes standards-compatible PKCE through HTTPS, private-LAN, or Tailscale dashboard routes.
|
||||
- **Replayed chat updates no longer destabilize the conversation list.** Duplicate upstream message identifiers are coalesced before Compose renders them.
|
||||
|
||||
## [Android 1.5.1] - 2026-07-26
|
||||
|
||||
### Added
|
||||
|
||||
- **Voice supports focused and conversational layouts.** Focus keeps spoken turns, Markdown, tools, media, and actions in a compact voice surface, while Conversation opens the full Chat renderer without leaving the active voice session.
|
||||
- **Voice can speak only settled answers.** A global Voice setting keeps tool progress, service updates, and intermediate commentary visual while supported voice paths wait to speak the final Hermes answer.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Chat answers are easier to read in every theme.** Primary assistant text now uses the theme's full-contrast foreground, and chat prose uses a 15sp size with 21sp line height.
|
||||
- **Google Play builds target Android 16.** The app now targets API level 36 while retaining its existing minimum-device support.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Completed streamed answers render their formatting without losing the reading position.** Markdown headings, lists, emphasis, and code blocks replace the live text renderer only after completion, then the measured trailing edge remains anchored at the bottom.
|
||||
- **Standard Voice speaks completed assistant replies again.** Session and message fences no longer suppress a valid final answer during the handoff from generation to narration.
|
||||
- **Realtime background work no longer blocks the active voice controls.** A promoted task releases the foreground spinner and microphone while its progress, tools, cancellation, and final result remain available in the owning chat.
|
||||
|
||||
## [Server 1.4.3] - 2026-07-22
|
||||
|
||||
### Added
|
||||
|
||||
@@ -1,5 +1,51 @@
|
||||
# Hermes-Relay — Dev Log
|
||||
|
||||
## 2026-07-28 — Android 1.5.2 production release
|
||||
|
||||
Android 1.5.2 shipped from the approved `dev` to `main` release tree as
|
||||
versionCode 35. The release adds provider-aware Dashboard sign-in: Nous uses
|
||||
the advertised native PKCE system-browser flow, while compatible self-hosted
|
||||
providers retain cookie-backed full-page Dashboard authentication. Callback
|
||||
origin discovery remains server-driven, private-network HTTP compatibility is
|
||||
preserved, and arbitrary public HTTP redirects remain rejected.
|
||||
|
||||
The private Play preflight validated the exact application tree before release
|
||||
PR #265 merged. The immutable `android-v1.5.2` tag resolves to the resulting
|
||||
`main` tip, the production workflow promoted versionCode 35 to the completed
|
||||
Google Play production track, and the public GitHub release contains the
|
||||
signed AAB, sideload APK, and SHA-256 manifest. The published sideload APK
|
||||
checksum was independently verified; replacing the debug-signed phone build
|
||||
with the release-signed artifact requires an uninstall because Android
|
||||
correctly rejects cross-signature in-place updates.
|
||||
|
||||
## 2026-07-27 — Android replayed-message identity reconciliation
|
||||
|
||||
Android history reconciliation now collapses reconnect/rejoin replays of the
|
||||
same persisted message ID before publishing the transcript to Compose. The
|
||||
latest repeated snapshot replaces the value at the message's first transcript
|
||||
position, preserving stable ordering, distinct messages, and the LazyColumn
|
||||
identity contract without index- or random-key fallbacks.
|
||||
|
||||
Focused coverage reproduces the duplicate UUID condition and verifies that the
|
||||
authoritative final content wins while every rendered message keeps a unique
|
||||
stable UI key.
|
||||
|
||||
## 2026-07-26 — Android 1.5.1 patch reconciliation
|
||||
|
||||
Android 1.5.1 reconciles the post-1.5.0 voice and chat fixes into versionCode
|
||||
34. Voice now offers compact Focus and full Conversation presentation,
|
||||
Standard narration preserves valid completed replies, and promoted Realtime
|
||||
tasks release foreground voice controls while retaining progress and results.
|
||||
|
||||
Completed streamed answers promote from the stable live text node to full
|
||||
Markdown only after completion. The measured Markdown row is then positioned
|
||||
by its trailing edge until deferred code and attachment measurement settles,
|
||||
preventing the LazyColumn from restoring the start of a tall response.
|
||||
|
||||
The release also targets Android API level 36. Release notes, in-app What's
|
||||
New assets, localized Play notes, and the Play listing reference were updated
|
||||
for Android 1.5.1.
|
||||
|
||||
## 2026-07-25 — Immutable Android release dispatch repair
|
||||
|
||||
Android approval now dispatches the current release workflow definition from
|
||||
|
||||
+10
-21
@@ -1,10 +1,10 @@
|
||||
# Hermes-Relay-Android v1.5.0
|
||||
# Hermes-Relay-Android v1.5.2
|
||||
|
||||
**Release Date:** July 25, 2026
|
||||
**Release Date:** July 28, 2026
|
||||
|
||||
## Download
|
||||
|
||||
> Installing on your phone? Download `hermes-relay-1.5.0-sideload-release.apk` and tap it for the full feature set, or install the conservative build from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
|
||||
> Installing on your phone? Download `hermes-relay-1.5.2-sideload-release.apk` and tap it for the full feature set, or install the conservative build from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
|
||||
|
||||
The `.aab` file is a Play Console upload bundle and cannot be installed by tapping it on a phone.
|
||||
|
||||
@@ -12,28 +12,17 @@ Verify the download against `SHA256SUMS.txt`. See the [sideload guide](https://h
|
||||
|
||||
## Summary
|
||||
|
||||
This release makes the Hermes Dashboard and Gateway the clear standard connection, keeps active work reachable across the app and Android backgrounding, and gives profiles, voice, attachments, image generation, and approvals a more coherent native interface.
|
||||
|
||||
## Added
|
||||
|
||||
- The Agent Passport drawer combines live route and session context with explicit profile switching, personality, model, reasoning, approval-policy, chat-override, and processing-tier controls.
|
||||
- Secure browser-based Dashboard sign-in is scoped to the selected host. Chat, sessions, Manage, and Standard Voice share the same authenticated Gateway route while the API server remains an automatic fallback.
|
||||
- Standard and Realtime voice settings use focused provider, model, and voice cards with upstream-aware discovery, descriptions, inline previews, waveforms, and a browsable catalog. Standard replies can begin speaking completed segments before generation finishes.
|
||||
- User-started turns stay protected until every concurrent session settles. Privacy-safe notifications reopen the correct chat for approvals, questions, elevated permissions, or secure responses.
|
||||
- Onboarding finishes with a layered permission review: notifications are recommended deliberately, optional capabilities remain separate, and users can continue without granting phone access.
|
||||
- Chat surfaces one-turn model choices, approval modes, advisor progress, queued recovery, project labels, collapsible attachments, persisted images, interim Gateway events, and image-generation activity.
|
||||
This patch restores reliable dashboard sign-in for self-hosted OIDC and Nous Portal connections, including private-LAN and Tailscale routes, and prevents replayed chat events from destabilizing the conversation list.
|
||||
|
||||
## Fixed
|
||||
|
||||
- Gateway reconnects reactivate the original live session without resubmitting acknowledged prompts or duplicating session rows.
|
||||
- Tailscale, QR, and other remote routes move Dashboard, Gateway, sessions, Manage, Standard Voice, API fallback, and optional Relay together.
|
||||
- Dashboard authentication, model routing, recovery, and profile state stay scoped to the selected connection and session, including during cold start and rapid switching.
|
||||
- Promoted voice and background tasks keep their owning Chat row until the work settles.
|
||||
- User-installed certificate authorities work for self-hosted HTTPS/WSS while normal chain, hostname, and Relay-pin verification remain enforced.
|
||||
- Malformed syntax-highlighting ranges no longer crash Markdown rendering.
|
||||
- Developer Options no longer exposes the obsolete Relay feature flag; version-tap unlock, relock, backup, import, and reset actions now persist and report accurately.
|
||||
- Self-hosted OIDC returns through the dashboard cookie flow instead of a desktop-only loopback callback.
|
||||
- Nous Portal authentication opens in the system browser so provider security challenges can complete.
|
||||
- Native PKCE uses standards-compatible unpadded Base64URL and preserves the dashboard's canonical HTTPS callback origin while keeping tokens scoped to the active route.
|
||||
- Full-screen in-app sign-in remains available for compatible dashboard providers.
|
||||
- Replayed upstream chat events are coalesced before rendering, preventing duplicate message keys.
|
||||
|
||||
## Install / Verify
|
||||
|
||||
- App version: **1.5.0** (versionCode **33**).
|
||||
- App version: **1.5.2** (versionCode **35**).
|
||||
- Standard Chat and Vanilla Hermes voice continue to work against unmodified upstream Hermes.
|
||||
|
||||
@@ -717,9 +717,10 @@ Deferred:
|
||||
|
||||
A 5-agent audit compared the chat surface to Discord/Telegram/Messenger/iMessage/
|
||||
GitHub-mobile. **Shipped this pass (pending on-device verification):** a chat-tuned
|
||||
`markdownTypography()` ramp (headings were falling through to M3 display roles —
|
||||
h1=`displayLarge` 57sp in this app's scale — so a `#` was a billboard; now h1≈20sp
|
||||
scaling down, list/paragraph unified to 14sp, inline+fenced code 13sp, `textLink`
|
||||
`markdownTypography()` ramp (headings were falling through to M3 display roles —
|
||||
h1=`displayLarge` 57sp in this app's scale — so a `#` was a billboard; now h1≈20sp
|
||||
scaling down, list/paragraph unified to 15sp/21sp, primary assistant prose moved
|
||||
to the theme's full-contrast `onSurface`, inline+fenced code 13sp, `textLink`
|
||||
accent+underline) in `MarkdownContent.kt`; timestamp gated to `isLastInGroup` (was on
|
||||
every bubble) + grouping breaks on a >5min gap (`GROUP_GAP_MS`) so a resumed
|
||||
conversation gets its own beat; long-press haptic on the action menu; streaming dots
|
||||
@@ -731,10 +732,6 @@ gated to pre-first-token. Deferred:
|
||||
parses one full CommonMark document so global link references, indentation, and
|
||||
nested containers remain correct; the viewport now anchors that same remeasure.
|
||||
Verify lists, tables, quotes, HTML, nested fences, and reference links on-device.
|
||||
- **Bubble body 14sp → 15sp/21.** 14sp is the smallest body of the five reference
|
||||
apps. Bump markdown paragraph/text/list + the two plain `Text` sites
|
||||
(`MessageBubble.kt` user/system) together; keep ~1.4 leading so the ~272dp measure
|
||||
stays ~36–38 chars/line. Debatable/broad — left out of the certain heading win.
|
||||
- **Tail-corner on last-in-group only (design decision).** The audit flagged the
|
||||
per-bubble bottom tail as "half-implemented," but it's a deliberate aesthetic
|
||||
(every bubble tails). Switching to iMessage-style "tail on the last bubble only"
|
||||
|
||||
@@ -37,7 +37,7 @@ android {
|
||||
// exempt from Play's 14-day closed-testing rule. See RELEASE.md.
|
||||
applicationId = "com.axiomlabs.hermesrelay"
|
||||
minSdk = 26
|
||||
targetSdk = 35
|
||||
targetSdk = 36
|
||||
versionCode = libs.versions.appVersionCode.get().toInt()
|
||||
versionName = libs.versions.appVersionName.get()
|
||||
|
||||
|
||||
@@ -1 +1 @@
|
||||
Connect through secure Dashboard sign-in, switch profiles from the new Agent Passport, and keep multiple background chats active with actionable approval and question alerts. Image generation, attachments, model routing, voice, and Gateway recovery are clearer and more reliable. Setup now guides optional notification, camera, microphone, and companion permissions without blocking chat.
|
||||
Dashboard sign-in now completes reliably for self-hosted OIDC and Nous Portal, including private-LAN and Tailscale routes. Nous opens securely in the system browser, while compatible providers retain full-screen in-app sign-in. Replayed chat updates no longer duplicate conversation rows.
|
||||
|
||||
@@ -1 +1 @@
|
||||
通过安全的 Dashboard 登录连接,并在新的智能体护照中切换配置文件。多个后台对话可保持运行,审批或提问通知可直接返回正确会话。图像生成、附件、模型路由、语音和 Gateway 恢复更加清晰可靠。设置流程会说明可选的通知、相机、麦克风和通知伴侣权限,且不会阻止聊天。
|
||||
Hermes 仪表板登录现在可为自托管 OIDC 和 Nous Portal 可靠完成认证,并支持私有局域网与 Tailscale 路由。Nous 会在系统浏览器中安全打开,兼容的提供商仍可使用应用内全屏登录。重放的聊天更新不再产生重复会话行。
|
||||
|
||||
@@ -1,5 +1,53 @@
|
||||
{
|
||||
"versions": [
|
||||
{
|
||||
"version": "1.5.2",
|
||||
"title": "Sign in without detours",
|
||||
"date": "2026-07-28",
|
||||
"sections": [
|
||||
{
|
||||
"header": "Provider-compatible sign-in",
|
||||
"bullets": [
|
||||
"Self-hosted OIDC returns through the dashboard callback, while Nous Portal opens securely in the system browser.",
|
||||
"Private-LAN and Tailscale dashboard routes preserve the configured HTTPS callback and keep credentials scoped to the active connection."
|
||||
]
|
||||
},
|
||||
{
|
||||
"header": "Stable conversation updates",
|
||||
"bullets": [
|
||||
"Replayed upstream chat events are coalesced before rendering so duplicate message identifiers do not destabilize the conversation list."
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"version": "1.5.1",
|
||||
"title": "Voice and chat stay in place",
|
||||
"date": "2026-07-26",
|
||||
"sections": [
|
||||
{
|
||||
"header": "Voice at the right depth",
|
||||
"bullets": [
|
||||
"Use Voice Focus for a compact spoken-turn view or Conversation for the complete Chat renderer without leaving the active voice session.",
|
||||
"Keep intermediate work visual while supported voice paths wait to speak the settled final response."
|
||||
]
|
||||
},
|
||||
{
|
||||
"header": "Reliable narration and background work",
|
||||
"bullets": [
|
||||
"Standard Voice now speaks valid completed replies after generation hands off to narration.",
|
||||
"Realtime background tasks release foreground voice controls while their progress and results remain reachable."
|
||||
]
|
||||
},
|
||||
{
|
||||
"header": "Formatted answers stay readable",
|
||||
"bullets": [
|
||||
"Completed streams render headings, lists, emphasis, and code blocks without returning to the beginning of the answer.",
|
||||
"Assistant text uses stronger theme contrast and a more comfortable chat reading scale."
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"version": "1.5.0",
|
||||
"title": "Hermes, always in reach",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
v1.5.0 - Hermes, always in reach
|
||||
v1.5.2 - Sign in without detours
|
||||
|
||||
* Connect through secure Dashboard sign-in and switch profiles from the new Agent Passport.
|
||||
* Keep multiple background chats active and reopen the right session from approval or question alerts.
|
||||
* Follow richer attachments, image generation, model routing, Gateway recovery, and streaming voice.
|
||||
* Finish setup with clear, optional permission guidance that never blocks standard chat.
|
||||
* Complete self-hosted OIDC sign-in through the dashboard callback.
|
||||
* Open Nous Portal securely in the system browser.
|
||||
* Sign in over private-LAN and Tailscale dashboard routes.
|
||||
* Keep replayed chat updates from duplicating conversation rows.
|
||||
|
||||
@@ -247,6 +247,7 @@ data class Connection(
|
||||
apiServerUrl: String,
|
||||
relayUrl: String,
|
||||
extraApiUrls: List<Pair<String, String>> = emptyList(),
|
||||
dashboardUrl: String? = null,
|
||||
): List<EndpointCandidate> {
|
||||
val routes = buildList {
|
||||
endpointCandidateFromApiUrl(
|
||||
@@ -255,6 +256,7 @@ data class Connection(
|
||||
apiServerUrl = apiServerUrl,
|
||||
relayUrl = relayUrl.takeIf { it.isNotBlank() }
|
||||
?: deriveDefaultRelayUrl(apiServerUrl).orEmpty(),
|
||||
dashboardUrl = dashboardUrl,
|
||||
)?.let(::add)
|
||||
|
||||
extraApiUrls
|
||||
@@ -266,6 +268,7 @@ data class Connection(
|
||||
priority = index + 1,
|
||||
apiServerUrl = url,
|
||||
relayUrl = deriveDefaultRelayUrl(url).orEmpty(),
|
||||
dashboardUrl = dashboardUrl,
|
||||
)?.let(::add)
|
||||
}
|
||||
}
|
||||
@@ -340,6 +343,7 @@ data class Connection(
|
||||
priority: Int,
|
||||
apiServerUrl: String,
|
||||
relayUrl: String,
|
||||
dashboardUrl: String? = null,
|
||||
): EndpointCandidate? {
|
||||
val uri = runCatching { URI(apiServerUrl.trim().trimEnd('/')) }.getOrNull()
|
||||
?: return null
|
||||
@@ -363,12 +367,62 @@ data class Connection(
|
||||
role = role.ifBlank { inferRouteRole(apiServerUrl) },
|
||||
priority = priority,
|
||||
api = ApiEndpoint(host = host, port = port, tls = tls),
|
||||
dashboard = deriveDefaultDashboardUrl(apiServerUrl)
|
||||
dashboard = dashboardUrl
|
||||
?.trim()
|
||||
?.trimEnd('/')
|
||||
?.takeIf { it.isNotBlank() && urlsShareHost(it, apiServerUrl) }
|
||||
?.let { DashboardEndpoint(url = it) }
|
||||
?: deriveDefaultDashboardUrl(apiServerUrl)
|
||||
?.let { DashboardEndpoint(url = it) },
|
||||
relay = RelayEndpoint(url = resolvedRelayUrl, transportHint = transportHint),
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Reconcile stored API-derived routes with the Dashboard origin that
|
||||
* was actually verified during setup. Older app versions synthesized
|
||||
* `:9119` for every API route, even when the same host was reached
|
||||
* through an HTTPS reverse proxy on 443. Replace only that conventional
|
||||
* synthesized value (or a missing value); preserve explicit and
|
||||
* different-host LAN/Tailscale routes.
|
||||
*/
|
||||
fun reconcileDashboardRoutes(
|
||||
dashboardUrl: String?,
|
||||
candidates: List<EndpointCandidate>,
|
||||
): List<EndpointCandidate> {
|
||||
val explicitDashboard = dashboardUrl
|
||||
?.trim()
|
||||
?.trimEnd('/')
|
||||
?.takeIf { it.isNotBlank() }
|
||||
?: return candidates
|
||||
return candidates.map { candidate ->
|
||||
val apiUrl = candidate.api?.url ?: return@map candidate
|
||||
if (!urlsShareHost(explicitDashboard, apiUrl)) return@map candidate
|
||||
|
||||
val currentDashboard = candidate.dashboard?.url
|
||||
val derivedDashboard = deriveDefaultDashboardUrl(apiUrl)
|
||||
val canReplace = currentDashboard.isNullOrBlank() ||
|
||||
(
|
||||
derivedDashboard != null &&
|
||||
currentDashboard.trim().trimEnd('/')
|
||||
.equals(derivedDashboard, ignoreCase = true)
|
||||
)
|
||||
if (canReplace) {
|
||||
candidate.copy(dashboard = DashboardEndpoint(url = explicitDashboard))
|
||||
} else {
|
||||
candidate
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fun urlsShareHost(leftUrl: String, rightUrl: String): Boolean {
|
||||
val leftHost = runCatching { URI(leftUrl.trim()) }.getOrNull()?.host
|
||||
val rightHost = runCatching { URI(rightUrl.trim()) }.getOrNull()?.host
|
||||
return !leftHost.isNullOrBlank() &&
|
||||
!rightHost.isNullOrBlank() &&
|
||||
leftHost.equals(rightHost, ignoreCase = true)
|
||||
}
|
||||
|
||||
/**
|
||||
* De-duplication identity for rebuilding stored routes. Prefer the
|
||||
* legacy API authority when present so an older API-only candidate and
|
||||
|
||||
@@ -543,29 +543,6 @@ class ConnectionStore private constructor(
|
||||
}
|
||||
}
|
||||
|
||||
private fun Connection.withDashboardDefaults(): Connection {
|
||||
val derivedDashboardUrl = Connection.deriveDefaultDashboardUrl(apiServerUrl)
|
||||
val normalizedRoutes = routeCandidates.ifEmpty {
|
||||
Connection.buildRouteCandidates(apiServerUrl, relayUrl)
|
||||
}
|
||||
val normalizedPreferredRouteRole = preferredRouteRole?.takeIf { preferred ->
|
||||
normalizedRoutes.any { it.role.equals(preferred, ignoreCase = true) }
|
||||
}
|
||||
return if (
|
||||
(dashboardUrl.isNullOrBlank() && derivedDashboardUrl != null) ||
|
||||
normalizedRoutes != routeCandidates ||
|
||||
normalizedPreferredRouteRole != preferredRouteRole
|
||||
) {
|
||||
copy(
|
||||
dashboardUrl = dashboardUrl?.takeIf { it.isNotBlank() } ?: derivedDashboardUrl,
|
||||
routeCandidates = normalizedRoutes,
|
||||
preferredRouteRole = normalizedPreferredRouteRole,
|
||||
)
|
||||
} else {
|
||||
this
|
||||
}
|
||||
}
|
||||
|
||||
companion object {
|
||||
private const val TAG = "ConnectionStore"
|
||||
|
||||
@@ -585,3 +562,40 @@ class ConnectionStore private constructor(
|
||||
private const val DEFAULT_RELAY_URL = "ws://localhost:8767"
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Restore route defaults after loading a serialized connection. This remains
|
||||
* internal so focused persistence tests can exercise the same normalization
|
||||
* path used by [ConnectionStore].
|
||||
*/
|
||||
internal fun Connection.withDashboardDefaults(): Connection {
|
||||
val derivedDashboardUrl = Connection.deriveDefaultDashboardUrl(apiServerUrl)
|
||||
val effectiveDashboardUrl = dashboardUrl?.takeIf { it.isNotBlank() } ?: derivedDashboardUrl
|
||||
val storedOrDefaultRoutes = routeCandidates.ifEmpty {
|
||||
Connection.buildRouteCandidates(
|
||||
apiServerUrl = apiServerUrl,
|
||||
relayUrl = relayUrl,
|
||||
dashboardUrl = effectiveDashboardUrl,
|
||||
)
|
||||
}
|
||||
val normalizedRoutes = Connection.reconcileDashboardRoutes(
|
||||
dashboardUrl = effectiveDashboardUrl,
|
||||
candidates = storedOrDefaultRoutes,
|
||||
)
|
||||
val normalizedPreferredRouteRole = preferredRouteRole?.takeIf { preferred ->
|
||||
normalizedRoutes.any { it.role.equals(preferred, ignoreCase = true) }
|
||||
}
|
||||
return if (
|
||||
dashboardUrl != effectiveDashboardUrl ||
|
||||
normalizedRoutes != routeCandidates ||
|
||||
normalizedPreferredRouteRole != preferredRouteRole
|
||||
) {
|
||||
copy(
|
||||
dashboardUrl = effectiveDashboardUrl,
|
||||
routeCandidates = normalizedRoutes,
|
||||
preferredRouteRole = normalizedPreferredRouteRole,
|
||||
)
|
||||
} else {
|
||||
this
|
||||
}
|
||||
}
|
||||
|
||||
@@ -36,6 +36,14 @@ data class VoiceSettings(
|
||||
val audioRoute: String = VoiceAudioRoute.Auto.storageValue,
|
||||
val interactionMode: String = "tap",
|
||||
val silenceThresholdMs: Long = 1250L,
|
||||
/**
|
||||
* When true, voice keeps progress visual and waits for the settled Hermes
|
||||
* answer before speaking. Tool status, service updates, and intermediate
|
||||
* assistant commentary are not narrated.
|
||||
*/
|
||||
val finalAnswerOnly: Boolean = false,
|
||||
/** Presentation only; changing this never restarts or interrupts voice. */
|
||||
val presentationMode: String = VoicePresentationMode.Focus.storageValue,
|
||||
val realtimeTraceDetails: Boolean = false,
|
||||
/**
|
||||
* When true (default), Realtime Agent keeps one provider session/socket open
|
||||
@@ -122,6 +130,16 @@ enum class VoiceAudioRoute(val storageValue: String) {
|
||||
}
|
||||
}
|
||||
|
||||
enum class VoicePresentationMode(val storageValue: String) {
|
||||
Focus("focus"),
|
||||
Conversation("conversation");
|
||||
|
||||
companion object {
|
||||
fun fromStorage(value: String?): VoicePresentationMode =
|
||||
values().firstOrNull { it.storageValue == value } ?: Focus
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Active scope for per-profile voice prefs.
|
||||
*
|
||||
@@ -182,6 +200,8 @@ class VoicePreferencesRepository(private val dataStore: DataStore<Preferences>)
|
||||
// un-namespaced means switching profiles never churns these.
|
||||
private val KEY_INTERACTION_MODE = stringPreferencesKey("voice_interaction_mode")
|
||||
private val KEY_SILENCE_THRESHOLD_MS = longPreferencesKey("voice_silence_threshold_ms")
|
||||
private val KEY_FINAL_ANSWER_ONLY = booleanPreferencesKey("voice_final_answer_only")
|
||||
private val KEY_PRESENTATION_MODE = stringPreferencesKey("voice_presentation_mode")
|
||||
private val KEY_REALTIME_TRACE_DETAILS = booleanPreferencesKey("voice_realtime_trace_details")
|
||||
private val KEY_REALTIME_PERSISTENT_SESSION =
|
||||
booleanPreferencesKey("voice_realtime_persistent_session")
|
||||
@@ -191,6 +211,8 @@ class VoicePreferencesRepository(private val dataStore: DataStore<Preferences>)
|
||||
const val DEFAULT_INTERACTION_MODE = "tap"
|
||||
// 1250 ms matches hermes-desktop voice_mode `silenceMs` end-of-speech.
|
||||
const val DEFAULT_SILENCE_THRESHOLD_MS = 1250L
|
||||
const val DEFAULT_FINAL_ANSWER_ONLY = false
|
||||
const val DEFAULT_PRESENTATION_MODE = "focus"
|
||||
const val DEFAULT_REALTIME_TRACE_DETAILS = false
|
||||
const val DEFAULT_REALTIME_PERSISTENT_SESSION = true
|
||||
|
||||
@@ -258,6 +280,10 @@ class VoicePreferencesRepository(private val dataStore: DataStore<Preferences>)
|
||||
// --- global (shared across profiles) ---
|
||||
interactionMode = prefs[KEY_INTERACTION_MODE] ?: DEFAULT_INTERACTION_MODE,
|
||||
silenceThresholdMs = prefs[KEY_SILENCE_THRESHOLD_MS] ?: DEFAULT_SILENCE_THRESHOLD_MS,
|
||||
finalAnswerOnly = prefs[KEY_FINAL_ANSWER_ONLY] ?: DEFAULT_FINAL_ANSWER_ONLY,
|
||||
presentationMode = VoicePresentationMode.fromStorage(
|
||||
prefs[KEY_PRESENTATION_MODE] ?: DEFAULT_PRESENTATION_MODE,
|
||||
).storageValue,
|
||||
realtimeTraceDetails = prefs[KEY_REALTIME_TRACE_DETAILS]
|
||||
?: DEFAULT_REALTIME_TRACE_DETAILS,
|
||||
realtimePersistentSession = prefs[KEY_REALTIME_PERSISTENT_SESSION]
|
||||
@@ -367,6 +393,14 @@ class VoicePreferencesRepository(private val dataStore: DataStore<Preferences>)
|
||||
dataStore.edit { it[KEY_SILENCE_THRESHOLD_MS] = ms.coerceAtLeast(500L) }
|
||||
}
|
||||
|
||||
suspend fun setFinalAnswerOnly(enabled: Boolean) {
|
||||
dataStore.edit { it[KEY_FINAL_ANSWER_ONLY] = enabled }
|
||||
}
|
||||
|
||||
suspend fun setPresentationMode(mode: VoicePresentationMode) {
|
||||
dataStore.edit { it[KEY_PRESENTATION_MODE] = mode.storageValue }
|
||||
}
|
||||
|
||||
suspend fun setRealtimeTraceDetails(enabled: Boolean) {
|
||||
dataStore.edit { it[KEY_REALTIME_TRACE_DETAILS] = enabled }
|
||||
}
|
||||
|
||||
@@ -1278,6 +1278,7 @@ class RelayVoiceClient(
|
||||
model: String? = null,
|
||||
voice: String? = null,
|
||||
sampleRate: Int? = null,
|
||||
finalAnswerOnly: Boolean = false,
|
||||
onHandoff: (VoiceHandoffEvent) -> Unit = {},
|
||||
turnInputs: kotlinx.coroutines.channels.ReceiveChannel<RealtimeTurnInput>? = null,
|
||||
onTurnComplete: (RealtimeVoiceSummary) -> Unit = {},
|
||||
@@ -1309,6 +1310,7 @@ class RelayVoiceClient(
|
||||
model = model,
|
||||
voice = voice,
|
||||
sampleRate = sampleRate,
|
||||
finalAnswerOnly = finalAnswerOnly,
|
||||
)
|
||||
if (sessionResult.isFailure) {
|
||||
return@withContext Result.failure(sessionResult.exceptionOrNull() ?: IOException("Realtime agent session failed"))
|
||||
@@ -1822,6 +1824,28 @@ class RelayVoiceClient(
|
||||
if (event.type == "hermes.run.promoted") {
|
||||
longRunningTurn.set(true)
|
||||
Log.i(TAG, "Realtime agent turn marked long-running (run promoted); relaxing idle guard")
|
||||
if (persistent &&
|
||||
event.spokenHandoff == false &&
|
||||
activeTurn.compareAndSet(true, false)
|
||||
) {
|
||||
Log.i(
|
||||
TAG,
|
||||
"Realtime agent foreground turn ended at silent background promotion",
|
||||
)
|
||||
onTurnComplete(
|
||||
RealtimeVoiceSummary(
|
||||
provider = event.provider ?: session.provider,
|
||||
model = event.model ?: session.model,
|
||||
voice = event.voice ?: session.voice,
|
||||
sampleRate = session.sampleRate,
|
||||
audioChunks = audioChunks,
|
||||
audioBytes = audioBytes,
|
||||
firstAudioMs = event.firstAudioMs,
|
||||
responseDoneMs = event.responseDoneMs,
|
||||
eventLogPath = event.eventLogPath ?: session.eventLogPath,
|
||||
)
|
||||
)
|
||||
}
|
||||
}
|
||||
if (event.isAudioDelta) {
|
||||
audioChunks += 1
|
||||
@@ -1847,13 +1871,12 @@ class RelayVoiceClient(
|
||||
responseDoneMs = event.responseDoneMs,
|
||||
eventLogPath = event.eventLogPath ?: session.eventLogPath,
|
||||
)
|
||||
if (persistent) {
|
||||
if (persistent && activeTurn.compareAndSet(true, false)) {
|
||||
// Turn boundary, not session boundary: keep the socket
|
||||
// open for the next utterance.
|
||||
activeTurn.set(false)
|
||||
longRunningTurn.set(false)
|
||||
onTurnComplete(summary)
|
||||
} else {
|
||||
} else if (!persistent) {
|
||||
if (claimTerminalSocket(
|
||||
webSocket,
|
||||
generation,
|
||||
@@ -2679,6 +2702,7 @@ class RelayVoiceClient(
|
||||
model: String? = null,
|
||||
voice: String? = null,
|
||||
sampleRate: Int? = null,
|
||||
finalAnswerOnly: Boolean = false,
|
||||
): Result<RealtimeSessionResponse> {
|
||||
val body = buildJsonObject {
|
||||
putProfile()
|
||||
@@ -2694,6 +2718,9 @@ class RelayVoiceClient(
|
||||
sampleRate?.takeIf { it > 0 }?.let {
|
||||
put("sample_rate", JsonPrimitive(it))
|
||||
}
|
||||
if (finalAnswerOnly) {
|
||||
put("final_answer_only", JsonPrimitive(true))
|
||||
}
|
||||
chatSessionId?.trim()?.takeIf { it.isNotBlank() }?.let {
|
||||
put("chat_session_id", JsonPrimitive(it))
|
||||
}
|
||||
@@ -2980,6 +3007,9 @@ class RelayVoiceClient(
|
||||
responseDoneMs = (metrics?.get("response_done_ms") as? JsonPrimitive)?.doubleOrNull,
|
||||
tier = (obj["tier"] as? JsonPrimitive)?.contentOrNull,
|
||||
floor = (obj["floor"] as? JsonPrimitive)?.contentOrNull,
|
||||
spokenHandoff = (obj["spoken_handoff"] as? JsonPrimitive)
|
||||
?.contentOrNull
|
||||
?.toBooleanStrictOrNull(),
|
||||
activeToolName = (obj["active_tool_name"] as? JsonPrimitive)?.contentOrNull,
|
||||
completedToolCount = (obj["completed_tool_count"] as? JsonPrimitive)?.intOrNull
|
||||
?: (obj["tool_count"] as? JsonPrimitive)?.intOrNull,
|
||||
@@ -3374,6 +3404,7 @@ data class RealtimeVoiceEvent(
|
||||
// ADR 33: background-run promotion fields.
|
||||
val tier: String? = null,
|
||||
val floor: String? = null,
|
||||
val spokenHandoff: Boolean? = null,
|
||||
// hermes.run.progress extras — drive the live background-run chip.
|
||||
val activeToolName: String? = null,
|
||||
val completedToolCount: Int? = null,
|
||||
|
||||
@@ -1223,6 +1223,14 @@ class ChatHandler {
|
||||
// so we can attach results back to the originating assistant message's ToolCall
|
||||
val toolResults = items.filter { it.role == "tool" }
|
||||
.associateBy { it.toolCallId }
|
||||
// A reconnect/rejoin history response can repeat a persisted message row.
|
||||
// Chat's LazyColumn renders domain ids as stable keys (via ChatMessage.uiKey),
|
||||
// so allowing both copies through would crash Compose before either copy
|
||||
// could be reconciled. A domain id identifies one persisted message: retain
|
||||
// its first transcript position while adopting the latest repeated snapshot.
|
||||
// Rows without ids remain independent, and tool/hidden rows keep their
|
||||
// separate handling above/below.
|
||||
val renderedItems = coalesceRenderedHistoryItems(items)
|
||||
|
||||
// Accumulator for media markers we find in loaded content — fired AFTER
|
||||
// the wholesale `_messages.value = ...` assignment so the ViewModel's
|
||||
@@ -1240,8 +1248,8 @@ class ChatHandler {
|
||||
// silently misses those rows, so a gateway turn's tokens/badges survived
|
||||
// only if a content match happened to cover them. See
|
||||
// [reconcileLiveIdsToServer].
|
||||
val serverItemIds = items.mapNotNullTo(HashSet()) { it.id }
|
||||
val idRemap = reconcileLiveIdsToServer(items, serverItemIds)
|
||||
val serverItemIds = renderedItems.mapNotNullTo(HashSet()) { it.id }
|
||||
val idRemap = reconcileLiveIdsToServer(renderedItems, serverItemIds)
|
||||
|
||||
// Carry CLIENT-ONLY enrichment forward across the reload, keyed by the
|
||||
// RECONCILED message id. The server transcript (MessageItem) rebuilds
|
||||
@@ -1278,7 +1286,7 @@ class ChatHandler {
|
||||
// clientOnly bubbles (same exchange, pre-sync copy).
|
||||
val syncedRealtimeTurnContents = mutableSetOf<String>()
|
||||
|
||||
val loaded = items.mapNotNull { item ->
|
||||
val loaded = renderedItems.mapNotNull { item ->
|
||||
val displayKind = item.displayKind?.trim()?.lowercase()
|
||||
if (displayKind == "hidden") return@mapNotNull null
|
||||
val role = when {
|
||||
@@ -1540,6 +1548,34 @@ class ChatHandler {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Collapse replayed visible history rows by their authoritative message id.
|
||||
*
|
||||
* Replacing the value at its first-seen slot preserves transcript ordering;
|
||||
* the last repeated value wins so a later, more complete snapshot is not lost.
|
||||
* Null ids cannot be proven identical and therefore remain separate rows.
|
||||
*/
|
||||
private fun coalesceRenderedHistoryItems(items: List<MessageItem>): List<MessageItem> {
|
||||
val firstSlotById = HashMap<String, Int>()
|
||||
val coalesced = ArrayList<MessageItem>(items.size)
|
||||
for (item in items) {
|
||||
if (renderedRoleOf(item) == null) continue
|
||||
val id = item.id
|
||||
if (id == null) {
|
||||
coalesced += item
|
||||
continue
|
||||
}
|
||||
val existingSlot = firstSlotById[id]
|
||||
if (existingSlot == null) {
|
||||
firstSlotById[id] = coalesced.size
|
||||
coalesced += item
|
||||
} else {
|
||||
coalesced[existingSlot] = item
|
||||
}
|
||||
}
|
||||
return coalesced
|
||||
}
|
||||
|
||||
/** One adoptable server row during id reconciliation. `taken` enforces consume-once. */
|
||||
private class ReconcileSlot(
|
||||
val serverId: String,
|
||||
|
||||
+88
-3
@@ -108,13 +108,18 @@ class NativeDashboardAuthClient(
|
||||
provider: String? = null,
|
||||
): NativeDashboardAuthorization {
|
||||
requireStrictLoopbackRedirect(redirectUri)
|
||||
val verifier = randomBytes(32).base64Url()
|
||||
// RFC 7636 uses unpadded Base64URL. Okio's base64Url() preserves
|
||||
// trailing "=", which makes Hermes' standards-compliant S256
|
||||
// comparison fail even though both sides hashed the same bytes.
|
||||
val verifier = randomBytes(32).base64Url().trimEnd('=')
|
||||
val challenge = MessageDigest.getInstance("SHA-256")
|
||||
.digest(verifier.toByteArray(Charsets.US_ASCII))
|
||||
.toByteString()
|
||||
.base64Url()
|
||||
.trimEnd('=')
|
||||
val state = randomBytes(24).base64Url()
|
||||
val root = "$baseUrl/auth/native/authorize".toHttpUrlOrNull()
|
||||
val authorizationBaseUrl = resolveAuthorizationBaseUrl(provider)
|
||||
val root = "$authorizationBaseUrl/auth/native/authorize".toHttpUrlOrNull()
|
||||
?: throw IOException("Dashboard URL is not a valid http(s) address")
|
||||
val url = root.newBuilder()
|
||||
.addQueryParameter("code_challenge", challenge)
|
||||
@@ -130,6 +135,41 @@ class NativeDashboardAuthClient(
|
||||
return NativeDashboardAuthorization(url, verifier, state, generation)
|
||||
}
|
||||
|
||||
/**
|
||||
* A private-route dashboard may be configured with a canonical HTTPS
|
||||
* callback origin for its provider. Starting the browser on the private
|
||||
* origin would scope Hermes' temporary PKCE cookie to the wrong host, so
|
||||
* discover the provider's declared callback and start native auth there.
|
||||
* Token exchange still uses [baseUrl], keeping the resulting bearer bound
|
||||
* to the active connection route.
|
||||
*/
|
||||
private fun resolveAuthorizationBaseUrl(provider: String?): String {
|
||||
val configured = baseUrl.toHttpUrlOrNull() ?: return baseUrl
|
||||
if (
|
||||
!provider.equals("nous", ignoreCase = true) ||
|
||||
configured.scheme != "http" ||
|
||||
!isPrivateNetworkLiteral(configured.host)
|
||||
) {
|
||||
return baseUrl
|
||||
}
|
||||
val loginUrl = configured.newBuilder()
|
||||
.addPathSegments("auth/login")
|
||||
.addQueryParameter("provider", provider)
|
||||
.addQueryParameter("next", "/")
|
||||
.build()
|
||||
val discoveryClient = client.newBuilder()
|
||||
.followRedirects(false)
|
||||
.followSslRedirects(false)
|
||||
.build()
|
||||
val location = discoveryClient.newCall(
|
||||
Request.Builder().url(loginUrl).get().build(),
|
||||
).execute().use { response ->
|
||||
if (response.code !in 300..399) null else response.header("Location")
|
||||
}
|
||||
return canonicalDashboardBaseFromNousRedirect(location)
|
||||
?: throw IOException("Dashboard did not advertise a secure Nous callback origin")
|
||||
}
|
||||
|
||||
fun exchangeCallback(
|
||||
authorization: NativeDashboardAuthorization,
|
||||
callbackTarget: String,
|
||||
@@ -280,7 +320,52 @@ internal class NativeDashboardCallbackException(
|
||||
internal fun isNativeDashboardTransportEligible(baseUrl: String): Boolean {
|
||||
val url = baseUrl.trim().trimEnd('/').toHttpUrlOrNull() ?: return false
|
||||
return url.scheme == "https" ||
|
||||
(url.scheme == "http" && url.host == "127.0.0.1")
|
||||
(
|
||||
url.scheme == "http" &&
|
||||
(url.host == "127.0.0.1" || isPrivateNetworkLiteral(url.host))
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Hermes already permits explicitly configured HTTP dashboard sessions on
|
||||
* local routes. The brokered flow is no less protected than that cookie flow,
|
||||
* but remains unavailable to arbitrary cleartext Internet hosts.
|
||||
*/
|
||||
private fun isPrivateNetworkLiteral(host: String): Boolean {
|
||||
val octets = host.split('.').mapNotNull(String::toIntOrNull)
|
||||
if (octets.size != 4 || octets.any { it !in 0..255 }) return false
|
||||
val first = octets[0]
|
||||
val second = octets[1]
|
||||
return first == 10 ||
|
||||
(first == 172 && second in 16..31) ||
|
||||
(first == 192 && second == 168) ||
|
||||
(first == 100 && second in 64..127)
|
||||
}
|
||||
|
||||
internal fun canonicalDashboardBaseFromNousRedirect(location: String?): String? {
|
||||
val providerUrl = location?.toHttpUrlOrNull() ?: return null
|
||||
if (
|
||||
providerUrl.scheme != "https" ||
|
||||
!providerUrl.host.equals("portal.nousresearch.com", ignoreCase = true)
|
||||
) {
|
||||
return null
|
||||
}
|
||||
val callback = providerUrl.queryParameter("redirect_uri")
|
||||
?.toHttpUrlOrNull()
|
||||
?: return null
|
||||
if (callback.scheme != "https") return null
|
||||
val callbackSuffix = "/auth/callback"
|
||||
if (!callback.encodedPath.endsWith(callbackSuffix)) return null
|
||||
val basePath = callback.encodedPath
|
||||
.removeSuffix(callbackSuffix)
|
||||
.ifBlank { "/" }
|
||||
return callback.newBuilder()
|
||||
.encodedPath(basePath)
|
||||
.query(null)
|
||||
.fragment(null)
|
||||
.build()
|
||||
.toString()
|
||||
.trimEnd('/')
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
+18
@@ -33,6 +33,24 @@ internal fun dashboardRedirectAuthMode(authFlows: List<String>): DashboardRedire
|
||||
DashboardRedirectAuthMode.WebView
|
||||
}
|
||||
|
||||
/**
|
||||
* Nous Portal uses Cloudflare Turnstile and does not support embedded Android
|
||||
* WebViews. Keep self-hosted OIDC on the dashboard cookie flow, but use the
|
||||
* gateway's brokered system-browser flow for Nous when it is advertised.
|
||||
*/
|
||||
internal fun androidDashboardRedirectAuthMode(
|
||||
providerName: String,
|
||||
authFlows: List<String>,
|
||||
): DashboardRedirectAuthMode =
|
||||
if (
|
||||
providerName.equals("nous", ignoreCase = true) &&
|
||||
dashboardRedirectAuthMode(authFlows) == DashboardRedirectAuthMode.NativePkce
|
||||
) {
|
||||
DashboardRedirectAuthMode.NativePkce
|
||||
} else {
|
||||
DashboardRedirectAuthMode.WebView
|
||||
}
|
||||
|
||||
/**
|
||||
* Owns one native dashboard sign-in attempt.
|
||||
*
|
||||
|
||||
@@ -35,6 +35,7 @@ import androidx.compose.material3.Scaffold
|
||||
import androidx.compose.material3.SnackbarDuration
|
||||
import androidx.compose.material3.SnackbarHost
|
||||
import androidx.compose.material3.SnackbarHostState
|
||||
import androidx.compose.material3.SnackbarResult
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.CompositionLocalProvider
|
||||
@@ -108,6 +109,7 @@ import com.hermesandroid.relay.data.EnhancedVoiceOverrides
|
||||
import com.hermesandroid.relay.data.EndpointCandidate
|
||||
import com.hermesandroid.relay.data.VoiceAudioRoute
|
||||
import com.hermesandroid.relay.data.VoicePreferencesRepository
|
||||
import com.hermesandroid.relay.data.VoicePresentationMode
|
||||
import com.hermesandroid.relay.data.VoiceSettings
|
||||
import com.hermesandroid.relay.data.capabilities
|
||||
import com.hermesandroid.relay.data.displayLabel
|
||||
@@ -182,8 +184,8 @@ val LocalSnackbarHost = staticCompositionLocalOf<SnackbarHostState> {
|
||||
|
||||
// Short-lived snackbar by default; retryable errors get Long so users have
|
||||
// time to tap the action before it auto-dismisses.
|
||||
suspend fun SnackbarHostState.showHumanError(err: HumanError) {
|
||||
showSnackbar(
|
||||
suspend fun SnackbarHostState.showHumanError(err: HumanError): SnackbarResult {
|
||||
return showSnackbar(
|
||||
message = err.body,
|
||||
actionLabel = err.actionLabel,
|
||||
duration = if (err.retryable) SnackbarDuration.Long else SnackbarDuration.Short,
|
||||
@@ -1919,6 +1921,14 @@ fun RelayApp() {
|
||||
voiceViewModel = voiceViewModel,
|
||||
voiceClient = voiceClient,
|
||||
maxBubbleWidth = maxBubbleWidth,
|
||||
voicePresentationMode = VoicePresentationMode.fromStorage(
|
||||
voiceSettings.presentationMode,
|
||||
),
|
||||
onVoicePresentationModeChange = { mode ->
|
||||
connectionSwitchScope.launch {
|
||||
voicePreferences.setPresentationMode(mode)
|
||||
}
|
||||
},
|
||||
openAgentSheetOnEntry = openAgentSheetArg,
|
||||
onAgentSheetArgConsumed = {
|
||||
backStackEntry.arguments?.putBoolean(
|
||||
@@ -1936,6 +1946,16 @@ fun RelayApp() {
|
||||
launchSingleTop = true
|
||||
}
|
||||
},
|
||||
onRepairConnection = {
|
||||
navController.navigate(
|
||||
Screen.Pair.route(
|
||||
connectionId = activeConnectionId,
|
||||
autoStart = "relay",
|
||||
),
|
||||
) {
|
||||
launchSingleTop = true
|
||||
}
|
||||
},
|
||||
// Empty-chat "needs connection" card also offers the offline
|
||||
// demo, so a skipped / never-connected first run can explore
|
||||
// without leaving Chat. Safe here — this state only shows when
|
||||
|
||||
@@ -72,6 +72,7 @@ import androidx.compose.ui.text.input.ImeAction
|
||||
import androidx.compose.ui.text.style.TextOverflow
|
||||
import androidx.compose.ui.unit.Dp
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.unit.sp
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import com.hermesandroid.relay.data.ChatMessage
|
||||
import com.hermesandroid.relay.data.MessageRole
|
||||
@@ -261,8 +262,12 @@ fun AgentTextFlow(
|
||||
motionEnabled: Boolean,
|
||||
modifier: Modifier = Modifier,
|
||||
) {
|
||||
val flowStyle = MaterialTheme.typography.bodyMedium.copy(fontFamily = FontFamily.Monospace)
|
||||
val flowColor = MaterialTheme.colorScheme.onSurfaceVariant
|
||||
val flowStyle = MaterialTheme.typography.bodyMedium.copy(
|
||||
fontFamily = FontFamily.Monospace,
|
||||
fontSize = 15.sp,
|
||||
lineHeight = 21.sp,
|
||||
)
|
||||
val flowColor = MaterialTheme.colorScheme.onSurface
|
||||
|
||||
// Readable, non-faded mirror of the visible tail — used as the live-region
|
||||
// text on both paths so assistive tech hears the words.
|
||||
|
||||
@@ -67,6 +67,11 @@ fun MarkdownContent(
|
||||
modifier: Modifier = Modifier
|
||||
) {
|
||||
val isDarkTheme = LocalBrand.current.isDark
|
||||
val chatBodyStyle = MaterialTheme.typography.bodyMedium.copy(
|
||||
fontSize = 15.sp,
|
||||
lineHeight = 21.sp,
|
||||
color = textColor,
|
||||
)
|
||||
val highlightsBuilder = remember(isDarkTheme) {
|
||||
Highlights.Builder().theme(SyntaxThemes.atom(darkMode = isDarkTheme))
|
||||
}
|
||||
@@ -87,7 +92,8 @@ fun MarkdownContent(
|
||||
// ~45sp, h3=displaySmall 36sp) — a single `#` becomes a billboard inside the
|
||||
// ~272dp bubble. Here every level derives from bodyLarge/bodyMedium (so the
|
||||
// live font-picker still applies) and is capped so the largest heading is
|
||||
// ~1.4x the 14sp body, matching Discord / GitHub-mobile in-message headings.
|
||||
// proportionate to the 15sp body, matching Discord / GitHub-mobile
|
||||
// in-message headings.
|
||||
typography = markdownTypography(
|
||||
h1 = MaterialTheme.typography.bodyLarge.copy(
|
||||
fontSize = 20.sp, lineHeight = 26.sp, fontWeight = FontWeight.Bold, color = textColor,
|
||||
@@ -108,16 +114,17 @@ fun MarkdownContent(
|
||||
fontSize = 13.sp, fontWeight = FontWeight.SemiBold, letterSpacing = 0.4.sp,
|
||||
color = textColor.copy(alpha = 0.85f),
|
||||
),
|
||||
// Prose, list items, and quotes all sit at the 14sp body size so a
|
||||
// paragraph and the bullet list under it share one rhythm — the library
|
||||
// default 'text'/list role is bodyLarge (16sp), 2sp larger than paragraph.
|
||||
paragraph = MaterialTheme.typography.bodyMedium.copy(color = textColor),
|
||||
text = MaterialTheme.typography.bodyMedium.copy(color = textColor),
|
||||
bullet = MaterialTheme.typography.bodyMedium.copy(color = textColor),
|
||||
ordered = MaterialTheme.typography.bodyMedium.copy(color = textColor),
|
||||
list = MaterialTheme.typography.bodyMedium.copy(color = textColor),
|
||||
quote = MaterialTheme.typography.bodyMedium.copy(
|
||||
fontStyle = FontStyle.Italic, color = textColor.copy(alpha = 0.78f),
|
||||
// Prose, list items, and quotes share a 15sp/21sp reading rhythm.
|
||||
// The library default 'text'/list role is bodyLarge (16sp), while
|
||||
// bodyMedium was previously 14sp and unnecessarily small for long chat.
|
||||
paragraph = chatBodyStyle,
|
||||
text = chatBodyStyle,
|
||||
bullet = chatBodyStyle,
|
||||
ordered = chatBodyStyle,
|
||||
list = chatBodyStyle,
|
||||
quote = chatBodyStyle.copy(
|
||||
fontStyle = FontStyle.Italic,
|
||||
color = textColor.copy(alpha = 0.9f),
|
||||
),
|
||||
// Inline + fenced code at 13sp (one step under body, not two): monospace
|
||||
// + the tinted chip already signal "code" without also shrinking it, and
|
||||
@@ -125,7 +132,7 @@ fun MarkdownContent(
|
||||
code = MaterialTheme.typography.bodySmall.copy(
|
||||
fontSize = 13.sp, letterSpacing = 0.sp,
|
||||
fontFamily = FontFamily.Monospace,
|
||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||
color = textColor,
|
||||
),
|
||||
inlineCode = MaterialTheme.typography.bodyMedium.copy(
|
||||
fontSize = 13.sp, letterSpacing = 0.sp,
|
||||
@@ -302,7 +309,10 @@ fun StreamingMarkdownContent(
|
||||
// code and deliberately spaced prose are not altered.
|
||||
text = content.withoutLeadingBlankLines(),
|
||||
modifier = modifier,
|
||||
style = MaterialTheme.typography.bodyMedium,
|
||||
style = MaterialTheme.typography.bodyMedium.copy(
|
||||
fontSize = 15.sp,
|
||||
lineHeight = 21.sp,
|
||||
),
|
||||
color = textColor,
|
||||
)
|
||||
} else {
|
||||
|
||||
@@ -174,7 +174,7 @@ fun MessageBubble(
|
||||
|
||||
val textColor = when (message.role) {
|
||||
MessageRole.USER -> MaterialTheme.colorScheme.onPrimary
|
||||
MessageRole.ASSISTANT -> MaterialTheme.colorScheme.onSurfaceVariant
|
||||
MessageRole.ASSISTANT -> MaterialTheme.colorScheme.onSurface
|
||||
MessageRole.SYSTEM -> MaterialTheme.colorScheme.onTertiaryContainer
|
||||
}
|
||||
|
||||
@@ -469,7 +469,10 @@ fun MessageBubble(
|
||||
// Plain text for user and system messages
|
||||
Text(
|
||||
text = message.content,
|
||||
style = MaterialTheme.typography.bodyMedium,
|
||||
style = MaterialTheme.typography.bodyMedium.copy(
|
||||
fontSize = 15.sp,
|
||||
lineHeight = 21.sp,
|
||||
),
|
||||
color = textColor
|
||||
)
|
||||
} else {
|
||||
|
||||
@@ -41,6 +41,7 @@ import androidx.compose.material.icons.filled.Close
|
||||
import androidx.compose.material.icons.filled.ExpandLess
|
||||
import androidx.compose.material.icons.filled.ExpandMore
|
||||
import androidx.compose.material.icons.filled.GraphicEq
|
||||
import androidx.compose.material.icons.filled.Image
|
||||
import androidx.compose.material.icons.filled.Mic
|
||||
import androidx.compose.material.icons.filled.Refresh
|
||||
import androidx.compose.material.icons.filled.Settings
|
||||
@@ -73,8 +74,10 @@ import androidx.compose.ui.text.style.TextAlign
|
||||
import androidx.compose.ui.text.style.TextOverflow
|
||||
import androidx.compose.ui.unit.dp
|
||||
import com.hermesandroid.relay.data.ChatMessage
|
||||
import com.hermesandroid.relay.data.HermesCardAction
|
||||
import com.hermesandroid.relay.data.MessageRole
|
||||
import com.hermesandroid.relay.data.ToolCall
|
||||
import com.hermesandroid.relay.data.VoicePresentationMode
|
||||
import com.hermesandroid.relay.ui.components.avatar.AvatarRenderState
|
||||
import com.hermesandroid.relay.ui.components.avatar.LocalAgentAvatar
|
||||
import com.hermesandroid.relay.ui.LocalSnackbarHost
|
||||
@@ -146,7 +149,8 @@ fun VoiceModeOverlay(
|
||||
voiceOutputEnabled: Boolean? = null,
|
||||
voiceOutputFallbackEnabled: Boolean? = null,
|
||||
onOverlayRequest: () -> Unit = {},
|
||||
onCompactModeChange: (Boolean) -> Unit = {},
|
||||
presentationMode: VoicePresentationMode = VoicePresentationMode.Focus,
|
||||
onPresentationModeChange: (VoicePresentationMode) -> Unit = {},
|
||||
// === END PHASE3-voice-mode-transcript ===
|
||||
// === v0.4.1 JIT permission-denied chip ===
|
||||
// Tapped when the user clicks the permission-denied chip. Default no-op
|
||||
@@ -159,22 +163,21 @@ fun VoiceModeOverlay(
|
||||
onBackgroundRunCancel: () -> Unit = {},
|
||||
onBackgroundRunTap: () -> Unit = {},
|
||||
onHermesConfirmationAnswer: (String) -> Unit = {},
|
||||
onCardAction: (messageId: String, cardKey: String, action: HermesCardAction) -> Unit =
|
||||
{ _, _, _ -> },
|
||||
onCardInput: (messageId: String, cardKey: String, value: String) -> Unit =
|
||||
{ _, _, _ -> },
|
||||
// === END v0.4.1 ===
|
||||
) {
|
||||
val surface = MaterialTheme.colorScheme.surface
|
||||
val haptic = LocalHapticFeedback.current
|
||||
|
||||
var controlsExpanded by remember { mutableStateOf(false) }
|
||||
var focusMode by remember { mutableStateOf(true) }
|
||||
val focusMode = presentationMode == VoicePresentationMode.Focus
|
||||
val setFocusMode: (Boolean) -> Unit = { focused ->
|
||||
focusMode = focused
|
||||
onCompactModeChange(!focused)
|
||||
}
|
||||
|
||||
LaunchedEffect(uiState.voiceMode) {
|
||||
if (!uiState.voiceMode) {
|
||||
setFocusMode(true)
|
||||
}
|
||||
onPresentationModeChange(
|
||||
if (focused) VoicePresentationMode.Focus else VoicePresentationMode.Conversation,
|
||||
)
|
||||
}
|
||||
|
||||
// Voice errors surface ONLY on the overlay's own inline top banner
|
||||
@@ -193,7 +196,7 @@ fun VoiceModeOverlay(
|
||||
// chips, pill) didn't handle so stray taps/swipes don't fall
|
||||
// through to the chat + session drawer behind it. Children run on
|
||||
// the same Main pass leaf-first, so this only catches the gaps.
|
||||
// In compact mode the overlay is intentionally transparent and the
|
||||
// In Conversation the overlay is intentionally transparent and the
|
||||
// chat stays interactive, so no scrim is installed.
|
||||
.then(
|
||||
if (focusMode) {
|
||||
@@ -404,6 +407,11 @@ fun VoiceModeOverlay(
|
||||
message = msg,
|
||||
showThinking = showThinking,
|
||||
expanded = msg.id == latestId || msg.isStreaming,
|
||||
onViewConversation = {
|
||||
onPresentationModeChange(VoicePresentationMode.Conversation)
|
||||
},
|
||||
onCardAction = onCardAction,
|
||||
onCardInput = onCardInput,
|
||||
)
|
||||
}
|
||||
if (pendingTranscriptText != null) {
|
||||
@@ -418,6 +426,11 @@ fun VoiceModeOverlay(
|
||||
),
|
||||
showThinking = showThinking,
|
||||
expanded = true,
|
||||
onViewConversation = {
|
||||
onPresentationModeChange(VoicePresentationMode.Conversation)
|
||||
},
|
||||
onCardAction = onCardAction,
|
||||
onCardInput = onCardInput,
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -449,7 +462,7 @@ fun VoiceModeOverlay(
|
||||
}
|
||||
}
|
||||
|
||||
// Compact mode: the background-run chip must survive outside focus
|
||||
// Conversation: the background-run chip must survive outside focus
|
||||
// mode too — a running task with no visible presence reads as lost
|
||||
// (the chip previously existed ONLY in the focus layout).
|
||||
AnimatedVisibility(
|
||||
@@ -821,6 +834,15 @@ private fun VoiceSessionPill(
|
||||
overflow = TextOverflow.Ellipsis,
|
||||
modifier = Modifier.weight(1f),
|
||||
)
|
||||
if (!focusMode) {
|
||||
ConversationVoiceMicButton(
|
||||
uiState = uiState,
|
||||
onMicTap = onMicTap,
|
||||
onMicRelease = onMicRelease,
|
||||
onInterrupt = onInterrupt,
|
||||
onPauseAutoMode = onPauseAutoMode,
|
||||
)
|
||||
}
|
||||
Icon(
|
||||
imageVector = if (expanded) Icons.Filled.ExpandLess else Icons.Filled.ExpandMore,
|
||||
contentDescription = if (expanded) stringResource(R.string.voice_overlay_collapse_cd) else stringResource(R.string.voice_overlay_expand_cd),
|
||||
@@ -855,10 +877,9 @@ private fun VoiceSessionPill(
|
||||
.padding(top = 10.dp),
|
||||
verticalArrangement = Arrangement.spacedBy(8.dp),
|
||||
) {
|
||||
// Moved out of the collapsed header (4a): the current
|
||||
// interaction-mode pill plus the inline mic control. The
|
||||
// compact mic only appears in compact mode, where the
|
||||
// full-size bottom mic button is hidden.
|
||||
// The expanded body keeps the current interaction mode
|
||||
// visible; Conversation's persistent mic stays in the
|
||||
// collapsed header so it never disappears.
|
||||
Row(
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
verticalAlignment = Alignment.CenterVertically,
|
||||
@@ -866,15 +887,6 @@ private fun VoiceSessionPill(
|
||||
) {
|
||||
StatusPill(uiState.interactionMode.label())
|
||||
Spacer(Modifier.weight(1f))
|
||||
if (!focusMode) {
|
||||
CompactVoiceMicButton(
|
||||
uiState = uiState,
|
||||
onMicTap = onMicTap,
|
||||
onMicRelease = onMicRelease,
|
||||
onInterrupt = onInterrupt,
|
||||
onPauseAutoMode = onPauseAutoMode,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
Row(
|
||||
@@ -915,7 +927,11 @@ private fun VoiceSessionPill(
|
||||
modifier = Modifier.weight(1f),
|
||||
) {
|
||||
Text(
|
||||
if (focusMode) stringResource(R.string.voice_overlay_compact) else stringResource(R.string.voice_overlay_focus),
|
||||
if (focusMode) {
|
||||
stringResource(R.string.voice_overlay_conversation)
|
||||
} else {
|
||||
stringResource(R.string.voice_overlay_focus)
|
||||
},
|
||||
maxLines = 1,
|
||||
overflow = TextOverflow.Ellipsis,
|
||||
)
|
||||
@@ -958,7 +974,7 @@ private fun VoiceSessionPill(
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun CompactVoiceMicButton(
|
||||
private fun ConversationVoiceMicButton(
|
||||
uiState: VoiceUiState,
|
||||
onMicTap: () -> Unit,
|
||||
onMicRelease: () -> Unit,
|
||||
@@ -1224,6 +1240,9 @@ private fun CompactTranscriptRow(
|
||||
message: ChatMessage,
|
||||
showThinking: Boolean,
|
||||
expanded: Boolean,
|
||||
onViewConversation: () -> Unit,
|
||||
onCardAction: (messageId: String, cardKey: String, action: HermesCardAction) -> Unit,
|
||||
onCardInput: (messageId: String, cardKey: String, value: String) -> Unit,
|
||||
) {
|
||||
if (message.role == MessageRole.SYSTEM) return
|
||||
|
||||
@@ -1248,6 +1267,13 @@ private fun CompactTranscriptRow(
|
||||
message.role == MessageRole.USER -> MaterialTheme.colorScheme.primary
|
||||
else -> MaterialTheme.colorScheme.secondary
|
||||
}
|
||||
val (markdownBody, inlineImages) = remember(message.content, message.role) {
|
||||
if (message.role == MessageRole.ASSISTANT) {
|
||||
extractChatInlineImages(message.content)
|
||||
} else {
|
||||
message.content to emptyList()
|
||||
}
|
||||
}
|
||||
|
||||
Column(
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
@@ -1285,7 +1311,7 @@ private fun CompactTranscriptRow(
|
||||
}
|
||||
when {
|
||||
isVoiceActionBubble && hasText -> MarkdownContent(
|
||||
content = message.content,
|
||||
content = markdownBody,
|
||||
textColor = MaterialTheme.colorScheme.onSurface,
|
||||
)
|
||||
message.role == MessageRole.USER && hasText -> Text(
|
||||
@@ -1295,12 +1321,107 @@ private fun CompactTranscriptRow(
|
||||
maxLines = if (expanded) Int.MAX_VALUE else 3,
|
||||
overflow = TextOverflow.Ellipsis,
|
||||
)
|
||||
hasText -> Text(
|
||||
text = message.content,
|
||||
style = MaterialTheme.typography.bodyMedium,
|
||||
hasText -> MarkdownContent(
|
||||
content = markdownBody,
|
||||
textColor = MaterialTheme.colorScheme.onSurface,
|
||||
)
|
||||
}
|
||||
message.cards.forEachIndexed { index, card ->
|
||||
if (card.actions.isNotEmpty() || card.input != null) {
|
||||
Spacer(Modifier.height(6.dp))
|
||||
val cardKey = card.id ?: "idx:$index"
|
||||
HermesCardBubble(
|
||||
card = card,
|
||||
cardKey = cardKey,
|
||||
dispatches = message.cardDispatches,
|
||||
onActionTap = { key, action ->
|
||||
onCardAction(message.id, key, action)
|
||||
},
|
||||
onInputSubmit = { key, value ->
|
||||
onCardInput(message.id, key, value)
|
||||
},
|
||||
maxWidth = 360.dp,
|
||||
)
|
||||
}
|
||||
}
|
||||
if (
|
||||
message.attachments.isNotEmpty() ||
|
||||
message.cards.any { it.actions.isEmpty() && it.input == null } ||
|
||||
inlineImages.isNotEmpty()
|
||||
) {
|
||||
Spacer(Modifier.height(6.dp))
|
||||
VoiceRichResultAffordance(
|
||||
message = message,
|
||||
onViewConversation = onViewConversation,
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Composable
|
||||
private fun VoiceRichResultAffordance(
|
||||
message: ChatMessage,
|
||||
onViewConversation: () -> Unit,
|
||||
) {
|
||||
val inlineImages = remember(message.content) {
|
||||
extractChatInlineImages(message.content).second
|
||||
}
|
||||
val previewModel = remember(message.attachments, inlineImages) {
|
||||
message.attachments.firstOrNull { it.isImage && !it.cachedUri.isNullOrBlank() }?.cachedUri
|
||||
?: inlineImages.firstOrNull {
|
||||
it.src.startsWith("https://") || it.src.startsWith("http://")
|
||||
}?.src
|
||||
}
|
||||
val label = when {
|
||||
message.attachments.size + inlineImages.size > 1 ->
|
||||
stringResource(
|
||||
R.string.voice_overlay_rich_results_count,
|
||||
message.attachments.size + inlineImages.size,
|
||||
)
|
||||
message.attachments.isNotEmpty() || inlineImages.isNotEmpty() ->
|
||||
stringResource(R.string.voice_overlay_image_ready)
|
||||
else -> stringResource(R.string.voice_overlay_rich_result_ready)
|
||||
}
|
||||
|
||||
Surface(
|
||||
modifier = Modifier
|
||||
.fillMaxWidth()
|
||||
.clickable(onClick = onViewConversation),
|
||||
shape = RoundedCornerShape(10.dp),
|
||||
color = MaterialTheme.colorScheme.primaryContainer.copy(alpha = 0.48f),
|
||||
) {
|
||||
Row(
|
||||
modifier = Modifier.padding(horizontal = 10.dp, vertical = 8.dp),
|
||||
verticalAlignment = Alignment.CenterVertically,
|
||||
horizontalArrangement = Arrangement.spacedBy(8.dp),
|
||||
) {
|
||||
if (previewModel != null) {
|
||||
AsyncImage(
|
||||
model = previewModel,
|
||||
contentDescription = null,
|
||||
contentScale = ContentScale.Crop,
|
||||
modifier = Modifier
|
||||
.size(40.dp)
|
||||
.clip(RoundedCornerShape(8.dp)),
|
||||
)
|
||||
} else {
|
||||
Icon(
|
||||
imageVector = Icons.Filled.Image,
|
||||
contentDescription = null,
|
||||
tint = MaterialTheme.colorScheme.primary,
|
||||
modifier = Modifier.size(20.dp),
|
||||
)
|
||||
}
|
||||
Text(
|
||||
text = label,
|
||||
style = MaterialTheme.typography.labelMedium,
|
||||
color = MaterialTheme.colorScheme.onSurface,
|
||||
maxLines = if (expanded) Int.MAX_VALUE else 6,
|
||||
overflow = TextOverflow.Ellipsis,
|
||||
modifier = Modifier.weight(1f),
|
||||
)
|
||||
Text(
|
||||
text = stringResource(R.string.voice_overlay_view_conversation),
|
||||
style = MaterialTheme.typography.labelMedium,
|
||||
color = MaterialTheme.colorScheme.primary,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -139,6 +139,7 @@ import androidx.compose.material3.SmallFloatingActionButton
|
||||
import androidx.compose.material3.SnackbarHost
|
||||
import androidx.compose.material3.SnackbarHostState
|
||||
import androidx.compose.material3.SnackbarDuration
|
||||
import androidx.compose.material3.SnackbarResult
|
||||
import android.content.ClipData
|
||||
import android.content.Intent
|
||||
import android.net.Uri
|
||||
@@ -156,7 +157,9 @@ import com.hermesandroid.relay.data.AgentDisplay
|
||||
import com.hermesandroid.relay.data.Attachment
|
||||
import com.hermesandroid.relay.data.ChatMessage
|
||||
import com.hermesandroid.relay.data.Connection
|
||||
import com.hermesandroid.relay.data.HermesCardAction
|
||||
import com.hermesandroid.relay.data.MessageRole
|
||||
import com.hermesandroid.relay.data.VoicePresentationMode
|
||||
import com.hermesandroid.relay.data.hermesProcessNotificationOrNull
|
||||
import com.hermesandroid.relay.ui.components.AgentInfoSheet
|
||||
import com.hermesandroid.relay.ui.components.BackgroundTaskCard
|
||||
@@ -204,6 +207,7 @@ import com.hermesandroid.relay.ui.components.showsImageGenerationPlaceholder
|
||||
import com.hermesandroid.relay.ui.components.VoiceModeOverlay
|
||||
import com.hermesandroid.relay.ui.LocalSnackbarHost
|
||||
import com.hermesandroid.relay.ui.showHumanError
|
||||
import com.hermesandroid.relay.util.HumanErrorAction
|
||||
import com.hermesandroid.relay.ui.theme.RelayRefresh
|
||||
import kotlin.math.abs
|
||||
import com.hermesandroid.relay.ui.theme.relayGridTexture
|
||||
@@ -280,6 +284,17 @@ internal fun ChatScrollSnapshot.isCompletionAfter(previous: ChatScrollSnapshot?)
|
||||
previous.messageCount == messageCount &&
|
||||
previous.lastMessageUiKey == lastMessageUiKey
|
||||
|
||||
internal fun releaseRetainedLiveTail(
|
||||
retainedUiKey: String?,
|
||||
completedUiKey: String?,
|
||||
): String? = retainedUiKey?.takeUnless { it == completedUiKey }
|
||||
|
||||
internal fun tailEndScrollOffset(
|
||||
tailSizePx: Int,
|
||||
footerSizePx: Int,
|
||||
viewportSizePx: Int,
|
||||
): Int = (tailSizePx + footerSizePx - viewportSizePx).coerceAtLeast(0)
|
||||
|
||||
private class ChatTailTransitionRef(
|
||||
var snapshot: ChatScrollSnapshot? = null,
|
||||
)
|
||||
@@ -439,6 +454,8 @@ fun ChatScreen(
|
||||
voiceViewModel: VoiceViewModel,
|
||||
voiceClient: RelayVoiceClient? = null,
|
||||
maxBubbleWidth: Dp = 300.dp,
|
||||
voicePresentationMode: VoicePresentationMode = VoicePresentationMode.Focus,
|
||||
onVoicePresentationModeChange: (VoicePresentationMode) -> Unit = {},
|
||||
// Deep-link nudge from Settings → Active Agent card: when `true`, the
|
||||
// AgentInfoSheet auto-opens on first composition and [onAgentSheetArgConsumed]
|
||||
// fires so the host can clear the nav arg (prevents re-open on tab
|
||||
@@ -451,6 +468,7 @@ fun ChatScreen(
|
||||
// don't wire navigation.
|
||||
onNavigateToConnections: () -> Unit = {},
|
||||
onNavigateToConnect: () -> Unit = onNavigateToConnections,
|
||||
onRepairConnection: () -> Unit = onNavigateToConnect,
|
||||
// Offline demo entry, surfaced on the empty-chat "needs connection" card so a
|
||||
// skipped / never-connected first run can explore without a server. null hides it.
|
||||
onTryDemo: (() -> Unit)? = null,
|
||||
@@ -465,9 +483,10 @@ fun ChatScreen(
|
||||
) {
|
||||
val voiceUiState by voiceViewModel.uiState.collectAsState()
|
||||
val isDemoMode by connectionViewModel.isDemoMode.collectAsState()
|
||||
var voiceCompactMode by remember { mutableStateOf(false) }
|
||||
val chatAlpha by animateFloatAsState(
|
||||
targetValue = if (voiceUiState.voiceMode && !voiceCompactMode) 0.4f else 1f,
|
||||
targetValue = if (
|
||||
voiceUiState.voiceMode && voicePresentationMode == VoicePresentationMode.Focus
|
||||
) 0.4f else 1f,
|
||||
animationSpec = tween(300),
|
||||
label = "chatAlpha",
|
||||
)
|
||||
@@ -477,7 +496,13 @@ fun ChatScreen(
|
||||
val snackbarHost = LocalSnackbarHost.current
|
||||
LaunchedEffect(chatViewModel) {
|
||||
chatViewModel.errorEvents.collect { err ->
|
||||
snackbarHost.showHumanError(err)
|
||||
val result = snackbarHost.showHumanError(err)
|
||||
if (
|
||||
result == SnackbarResult.ActionPerformed &&
|
||||
err.action == HumanErrorAction.Repair
|
||||
) {
|
||||
onRepairConnection()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -788,6 +813,25 @@ fun ChatScreen(
|
||||
val clipboard = LocalClipboard.current
|
||||
val haptic = LocalHapticFeedback.current
|
||||
val snackbarHostState = remember { SnackbarHostState() }
|
||||
val handleCardAction: (String, String, HermesCardAction) -> Unit =
|
||||
remember(chatViewModel, context) {
|
||||
{ messageId, cardKey, action ->
|
||||
if (action.mode == HermesCardAction.Modes.OPEN_URL) {
|
||||
chatViewModel.dispatchCardAction(messageId, cardKey, action)
|
||||
com.hermesandroid.relay.ui.components.handleCardActionExternally(
|
||||
context,
|
||||
action,
|
||||
)
|
||||
} else {
|
||||
chatViewModel.dispatchCardAction(messageId, cardKey, action)
|
||||
}
|
||||
}
|
||||
}
|
||||
val handleCardInput: (String, String, String) -> Unit = remember(chatViewModel) {
|
||||
{ messageId, cardKey, value ->
|
||||
chatViewModel.answerAsk(messageId, cardKey, value)
|
||||
}
|
||||
}
|
||||
|
||||
// Ephemeral notices from the VM (model-switch warnings/errors, etc.) →
|
||||
// transient snackbar, never a chat bubble.
|
||||
@@ -885,7 +929,6 @@ fun ChatScreen(
|
||||
if (!voiceUiState.voiceMode) {
|
||||
voiceOverlayHost.hide()
|
||||
pendingVoiceOverlayPermission = false
|
||||
voiceCompactMode = false
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1130,10 +1173,12 @@ fun ChatScreen(
|
||||
derivedStateOf {
|
||||
val retainingVisibleTail = retainedLiveTailUiKey != null &&
|
||||
messages.lastOrNull()?.uiKey == retainedLiveTailUiKey
|
||||
val settlingVisibleTail = completionSettlingUiKey != null &&
|
||||
messages.lastOrNull()?.uiKey == completionSettlingUiKey
|
||||
messages.isNotEmpty() &&
|
||||
!isAtBottom &&
|
||||
!programmaticBottomScroll &&
|
||||
!((isStreaming || retainingVisibleTail) &&
|
||||
!((isStreaming || retainingVisibleTail || settlingVisibleTail) &&
|
||||
smoothAutoScroll &&
|
||||
!userScrolledAway)
|
||||
}
|
||||
@@ -1331,34 +1376,99 @@ fun ChatScreen(
|
||||
) {
|
||||
val settlingKey = completionSettlingUiKey ?: return@LaunchedEffect
|
||||
if (!smoothAutoScroll || userScrolledAway || isUserDragging) {
|
||||
// Retention is only a completion-transition aid. Never leave the
|
||||
// finalized tail on the plain streaming renderer just because the
|
||||
// user disabled follow-scroll or is reading above the bottom.
|
||||
retainedLiveTailUiKey = releaseRetainedLiveTail(
|
||||
retainedUiKey = retainedLiveTailUiKey,
|
||||
completedUiKey = settlingKey,
|
||||
)
|
||||
completionSettlingUiKey = null
|
||||
return@LaunchedEffect
|
||||
}
|
||||
|
||||
var settledFrames = 0
|
||||
repeat(6) {
|
||||
var previousMarkdownTailSize: Int? = null
|
||||
var previousMarkdownFooterSize: Int? = null
|
||||
val markdownWasAlreadyReleased = retainedLiveTailUiKey != settlingKey
|
||||
repeat(60) completionFrame@{
|
||||
withFrameNanos { }
|
||||
if (messages.lastOrNull()?.uiKey != settlingKey) {
|
||||
completionSettlingUiKey = null
|
||||
return@LaunchedEffect
|
||||
}
|
||||
|
||||
if (listState.canScrollForward) {
|
||||
settledFrames = 0
|
||||
val viewportHeight = listState.layoutInfo.viewportSize.height
|
||||
if (viewportHeight > 0) {
|
||||
listState.scroll(MutatePriority.Default) {
|
||||
scrollBy(viewportHeight.toFloat())
|
||||
if (!markdownWasAlreadyReleased && retainedLiveTailUiKey == settlingKey) {
|
||||
if (listState.canScrollForward) {
|
||||
settledFrames = 0
|
||||
val viewportHeight = listState.layoutInfo.viewportSize.height
|
||||
if (viewportHeight > 0) {
|
||||
listState.scroll(MutatePriority.Default) {
|
||||
scrollBy(viewportHeight.toFloat())
|
||||
}
|
||||
}
|
||||
return@completionFrame
|
||||
}
|
||||
} else {
|
||||
|
||||
settledFrames += 1
|
||||
if (settledFrames >= 2) {
|
||||
completionSettlingUiKey = null
|
||||
return@LaunchedEffect
|
||||
}
|
||||
if (settledFrames < 2) return@completionFrame
|
||||
retainedLiveTailUiKey = releaseRetainedLiveTail(
|
||||
retainedUiKey = retainedLiveTailUiKey,
|
||||
completedUiKey = settlingKey,
|
||||
)
|
||||
settledFrames = 0
|
||||
return@completionFrame
|
||||
}
|
||||
|
||||
// Once Markdown owns the row, position its measured trailing edge
|
||||
// explicitly. `canScrollForward` is insufficient here: LazyColumn
|
||||
// may preserve the leading edge of a tall item while reporting an
|
||||
// otherwise valid item anchor. Repeating catches deferred parsing,
|
||||
// highlighted code, and attachment measurement without competing
|
||||
// with the ordinary streaming-growth coroutine.
|
||||
val layout = listState.layoutInfo
|
||||
val tailIndex = messages.size // header item + zero-based messages
|
||||
val footerIndex = tailIndex + 1
|
||||
val tailInfo = layout.visibleItemsInfo.firstOrNull { it.index == tailIndex }
|
||||
val footerInfo = layout.visibleItemsInfo.firstOrNull { it.index == footerIndex }
|
||||
if (tailInfo == null) {
|
||||
listState.scrollToItem(tailIndex)
|
||||
settledFrames = 0
|
||||
return@completionFrame
|
||||
}
|
||||
|
||||
val viewportHeight = layout.viewportSize.height
|
||||
if (viewportHeight <= 0) return@completionFrame
|
||||
val desiredOffset = tailEndScrollOffset(
|
||||
tailSizePx = tailInfo.size,
|
||||
footerSizePx = footerInfo?.size ?: 0,
|
||||
viewportSizePx = viewportHeight,
|
||||
)
|
||||
if (desiredOffset == 0) {
|
||||
listState.scrollToItem(footerIndex)
|
||||
} else {
|
||||
listState.scrollToItem(tailIndex, desiredOffset)
|
||||
}
|
||||
val footerSize = footerInfo?.size ?: 0
|
||||
settledFrames = if (
|
||||
previousMarkdownTailSize == tailInfo.size &&
|
||||
previousMarkdownFooterSize == footerSize
|
||||
) {
|
||||
settledFrames + 1
|
||||
} else {
|
||||
0
|
||||
}
|
||||
previousMarkdownTailSize = tailInfo.size
|
||||
previousMarkdownFooterSize = footerSize
|
||||
if (settledFrames >= 12) {
|
||||
completionSettlingUiKey = null
|
||||
return@LaunchedEffect
|
||||
}
|
||||
}
|
||||
retainedLiveTailUiKey = releaseRetainedLiveTail(
|
||||
retainedUiKey = retainedLiveTailUiKey,
|
||||
completedUiKey = settlingKey,
|
||||
)
|
||||
completionSettlingUiKey = null
|
||||
}
|
||||
|
||||
@@ -2308,24 +2418,8 @@ fun ChatScreen(
|
||||
onAttachmentManualFetch = { msgId, idx ->
|
||||
chatViewModel.manualFetchAttachment(msgId, idx)
|
||||
},
|
||||
onCardAction = { msgId, cardKey, action ->
|
||||
// OPEN_URL is resolved at the UI layer
|
||||
// because launching ACTION_VIEW needs a
|
||||
// Context. Record the dispatch first so
|
||||
// the card collapses even if launch fails.
|
||||
if (action.mode == com.hermesandroid.relay.data.HermesCardAction.Modes.OPEN_URL) {
|
||||
chatViewModel.dispatchCardAction(msgId, cardKey, action)
|
||||
com.hermesandroid.relay.ui.components.handleCardActionExternally(
|
||||
context,
|
||||
action,
|
||||
)
|
||||
} else {
|
||||
chatViewModel.dispatchCardAction(msgId, cardKey, action)
|
||||
}
|
||||
},
|
||||
onCardInput = { msgId, cardKey, value ->
|
||||
chatViewModel.answerAsk(msgId, cardKey, value)
|
||||
},
|
||||
onCardAction = handleCardAction,
|
||||
onCardInput = handleCardInput,
|
||||
onEditMessage = if (
|
||||
isGatewayTransport &&
|
||||
!isStreaming &&
|
||||
@@ -3184,14 +3278,13 @@ fun ChatScreen(
|
||||
voiceConfigScope = activeVoiceScope,
|
||||
voiceOutputEnabled = activeVoiceEnabled,
|
||||
voiceOutputFallbackEnabled = voiceOutputConfig?.fallback_enabled,
|
||||
presentationMode = voicePresentationMode,
|
||||
onPresentationModeChange = onVoicePresentationModeChange,
|
||||
onOverlayRequest = showVoiceSystemOverlay,
|
||||
// Gear button in the overlay's expanded controls. The overlay
|
||||
// exits voice mode before invoking this, so navigation lands
|
||||
// on Voice Settings with no overlay left on top.
|
||||
onOpenSettings = onNavigateToVoiceSettings,
|
||||
onCompactModeChange = { compact ->
|
||||
voiceCompactMode = compact
|
||||
},
|
||||
// === v0.4.1 JIT permission-denied chip ===
|
||||
// Tap deep-links to Settings → Apps → Hermes-Relay →
|
||||
// Permissions for the running package. Use BuildConfig
|
||||
@@ -3215,6 +3308,8 @@ fun ChatScreen(
|
||||
onHermesConfirmationAnswer = { answer ->
|
||||
voiceViewModel.answerHermesConfirmation(answer)
|
||||
},
|
||||
onCardAction = handleCardAction,
|
||||
onCardInput = handleCardInput,
|
||||
// === END v0.4.1 ===
|
||||
)
|
||||
}
|
||||
|
||||
@@ -1,18 +1,19 @@
|
||||
package com.hermesandroid.relay.ui.screens
|
||||
|
||||
import android.webkit.CookieManager
|
||||
import android.webkit.WebChromeClient
|
||||
import android.webkit.WebResourceError
|
||||
import android.webkit.WebResourceRequest
|
||||
import android.webkit.WebView
|
||||
import android.webkit.WebViewClient
|
||||
import androidx.activity.compose.BackHandler
|
||||
import androidx.compose.foundation.layout.Arrangement
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.fillMaxSize
|
||||
import androidx.compose.foundation.layout.fillMaxWidth
|
||||
import androidx.compose.foundation.layout.heightIn
|
||||
import androidx.compose.foundation.layout.padding
|
||||
import androidx.compose.material.icons.Icons
|
||||
import androidx.compose.material.icons.automirrored.filled.ArrowBack
|
||||
import androidx.compose.material.icons.filled.Close
|
||||
import androidx.compose.material.icons.filled.Check
|
||||
import androidx.compose.material3.Button
|
||||
import androidx.compose.material3.Card
|
||||
@@ -21,6 +22,7 @@ import androidx.compose.material3.ExperimentalMaterial3Api
|
||||
import androidx.compose.material3.HorizontalDivider
|
||||
import androidx.compose.material3.Icon
|
||||
import androidx.compose.material3.IconButton
|
||||
import androidx.compose.material3.LinearProgressIndicator
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.material3.OutlinedTextField
|
||||
import androidx.compose.material3.Scaffold
|
||||
@@ -38,11 +40,11 @@ import androidx.compose.runtime.setValue
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.platform.LocalResources
|
||||
import androidx.compose.ui.res.stringResource
|
||||
import androidx.compose.ui.text.input.PasswordVisualTransformation
|
||||
import androidx.compose.ui.unit.dp
|
||||
import androidx.compose.ui.viewinterop.AndroidView
|
||||
import androidx.compose.ui.window.Dialog
|
||||
import com.hermesandroid.relay.R
|
||||
import com.hermesandroid.relay.ui.components.ConnectionSetupTimeline
|
||||
import com.hermesandroid.relay.ui.components.ConnectionSetupTimelineStep
|
||||
@@ -50,10 +52,10 @@ import com.hermesandroid.relay.network.upstream.DashboardApiClient
|
||||
import com.hermesandroid.relay.network.upstream.DashboardAuthProvider
|
||||
import com.hermesandroid.relay.network.upstream.DashboardAuthSession
|
||||
import com.hermesandroid.relay.network.upstream.DashboardCookieStore
|
||||
import com.hermesandroid.relay.network.upstream.EncryptedDashboardCookieStore
|
||||
import com.hermesandroid.relay.network.upstream.DashboardRedirectAuthMode
|
||||
import com.hermesandroid.relay.network.upstream.EncryptedDashboardCookieStore
|
||||
import com.hermesandroid.relay.network.upstream.NativeDashboardSignInCoordinator
|
||||
import com.hermesandroid.relay.network.upstream.dashboardRedirectAuthMode
|
||||
import com.hermesandroid.relay.network.upstream.androidDashboardRedirectAuthMode
|
||||
import com.hermesandroid.relay.network.upstream.importDashboardCookieHeader
|
||||
import com.hermesandroid.relay.network.upstream.isNativeDashboardTransportEligible
|
||||
import com.hermesandroid.relay.viewmodel.ConnectionViewModel
|
||||
@@ -62,6 +64,7 @@ import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.Job
|
||||
import kotlinx.coroutines.launch
|
||||
import kotlinx.coroutines.withContext
|
||||
import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
|
||||
|
||||
/**
|
||||
* Connection-level Dashboard authentication flow. It is deliberately outside
|
||||
@@ -76,6 +79,7 @@ fun DashboardSignInScreen(
|
||||
onAuthenticated: () -> Unit,
|
||||
) {
|
||||
val context = LocalContext.current
|
||||
val resources = LocalResources.current
|
||||
val appContext = context.applicationContext
|
||||
val scope = rememberCoroutineScope()
|
||||
val activeConnection by connectionViewModel.activeConnection.collectAsState()
|
||||
@@ -90,8 +94,8 @@ fun DashboardSignInScreen(
|
||||
var actionMessage by remember { mutableStateOf<String?>(null) }
|
||||
var actionIsError by remember { mutableStateOf(false) }
|
||||
var oauthProvider by remember { mutableStateOf<DashboardAuthProvider?>(null) }
|
||||
var redirectAuthMode by remember(dashboardUrl, connectionId) {
|
||||
mutableStateOf(DashboardRedirectAuthMode.WebView)
|
||||
var authFlows by remember(dashboardUrl, connectionId) {
|
||||
mutableStateOf<List<String>>(emptyList())
|
||||
}
|
||||
var nativeSignInJob by remember(dashboardUrl, connectionId) { mutableStateOf<Job?>(null) }
|
||||
var authenticationComplete by remember { mutableStateOf(false) }
|
||||
@@ -135,20 +139,20 @@ fun DashboardSignInScreen(
|
||||
LaunchedEffect(dashboardUrl, connectionId) {
|
||||
if (dashboardUrl.isBlank()) {
|
||||
loading = false
|
||||
actionMessage = context.getString(R.string.dashboard_no_url_configured)
|
||||
actionMessage = resources.getString(R.string.dashboard_no_url_configured)
|
||||
return@LaunchedEffect
|
||||
}
|
||||
val client = clientFactory()
|
||||
try {
|
||||
val status = client.getStatus().getOrElse {
|
||||
actionMessage = it.message ?: context.getString(R.string.dashboard_request_failed)
|
||||
actionMessage = it.message ?: resources.getString(R.string.dashboard_request_failed)
|
||||
actionIsError = true
|
||||
return@LaunchedEffect
|
||||
}
|
||||
providers = client.getAuthProviders().getOrNull()
|
||||
?.takeIf { it.isNotEmpty() }
|
||||
?: status.authProviderDetails
|
||||
redirectAuthMode = dashboardRedirectAuthMode(status.authFlows)
|
||||
authFlows = status.authFlows
|
||||
val session = if (status.authRequired) client.currentSession().getOrNull() else null
|
||||
connectionViewModel.recordDashboardStatus(
|
||||
status = status,
|
||||
@@ -179,11 +183,11 @@ fun DashboardSignInScreen(
|
||||
finishAuthentication()
|
||||
} else {
|
||||
actionMessage = result.exceptionOrNull()?.message
|
||||
?: context.getString(R.string.dashboard_signin_no_session)
|
||||
?: resources.getString(R.string.dashboard_signin_no_session)
|
||||
actionIsError = true
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
actionMessage = e.message ?: context.getString(R.string.dashboard_signin_failed)
|
||||
actionMessage = e.message ?: resources.getString(R.string.dashboard_signin_failed)
|
||||
actionIsError = true
|
||||
} finally {
|
||||
actionInFlight = false
|
||||
@@ -194,25 +198,28 @@ fun DashboardSignInScreen(
|
||||
|
||||
fun startRedirectSignIn(provider: DashboardAuthProvider) {
|
||||
if (actionInFlight || dashboardUrl.isBlank()) return
|
||||
if (redirectAuthMode == DashboardRedirectAuthMode.WebView) {
|
||||
if (
|
||||
androidDashboardRedirectAuthMode(provider.name, authFlows) ==
|
||||
DashboardRedirectAuthMode.WebView
|
||||
) {
|
||||
oauthProvider = provider
|
||||
return
|
||||
}
|
||||
if (!isNativeDashboardTransportEligible(dashboardUrl)) {
|
||||
actionMessage = context.getString(R.string.dashboard_native_signin_requires_https)
|
||||
actionMessage = resources.getString(R.string.dashboard_native_signin_requires_https)
|
||||
actionIsError = true
|
||||
return
|
||||
}
|
||||
val authClient = connectionViewModel.nativeDashboardAuthClientForActive(dashboardUrl)
|
||||
if (authClient == null) {
|
||||
actionMessage = context.getString(R.string.dashboard_native_signin_unavailable)
|
||||
actionMessage = resources.getString(R.string.dashboard_native_signin_unavailable)
|
||||
actionIsError = true
|
||||
return
|
||||
}
|
||||
|
||||
actionInFlight = true
|
||||
actionIsError = false
|
||||
actionMessage = context.getString(R.string.dashboard_native_signin_opening)
|
||||
actionMessage = resources.getString(R.string.dashboard_native_signin_opening)
|
||||
nativeSignInJob = scope.launch {
|
||||
try {
|
||||
NativeDashboardSignInCoordinator(authClient).signIn(provider.name) { authorizationUrl ->
|
||||
@@ -228,19 +235,19 @@ fun DashboardSignInScreen(
|
||||
}
|
||||
if (session?.authenticated == true) {
|
||||
actionMessage = session.provider?.let {
|
||||
context.getString(R.string.dashboard_signed_in_with, it)
|
||||
} ?: context.getString(R.string.dashboard_signed_in)
|
||||
resources.getString(R.string.dashboard_signed_in_with, it)
|
||||
} ?: resources.getString(R.string.dashboard_signed_in)
|
||||
actionIsError = false
|
||||
finishAuthentication()
|
||||
} else {
|
||||
actionMessage = context.getString(R.string.dashboard_signin_no_session)
|
||||
actionMessage = resources.getString(R.string.dashboard_signin_no_session)
|
||||
actionIsError = true
|
||||
}
|
||||
} catch (cancelled: CancellationException) {
|
||||
throw cancelled
|
||||
} catch (error: Exception) {
|
||||
actionMessage = error.message
|
||||
?: context.getString(R.string.dashboard_signin_failed)
|
||||
?: resources.getString(R.string.dashboard_signin_failed)
|
||||
actionIsError = true
|
||||
} finally {
|
||||
actionInFlight = false
|
||||
@@ -253,10 +260,8 @@ fun DashboardSignInScreen(
|
||||
onDispose { nativeSignInJob?.cancel() }
|
||||
}
|
||||
|
||||
oauthProvider
|
||||
?.takeIf { redirectAuthMode == DashboardRedirectAuthMode.WebView }
|
||||
?.let { provider ->
|
||||
DashboardOAuthDialog(
|
||||
oauthProvider?.let { provider ->
|
||||
DashboardOAuthScreen(
|
||||
dashboardUrl = dashboardUrl,
|
||||
provider = provider,
|
||||
cookieStoreFactory = cookieStoreFactory,
|
||||
@@ -272,8 +277,8 @@ fun DashboardSignInScreen(
|
||||
client.shutdown()
|
||||
}
|
||||
actionMessage = session.provider?.let {
|
||||
context.getString(R.string.dashboard_signed_in_with, it)
|
||||
} ?: context.getString(R.string.dashboard_signed_in)
|
||||
resources.getString(R.string.dashboard_signed_in_with, it)
|
||||
} ?: resources.getString(R.string.dashboard_signed_in)
|
||||
finishAuthentication()
|
||||
}
|
||||
},
|
||||
@@ -282,6 +287,7 @@ fun DashboardSignInScreen(
|
||||
actionIsError = true
|
||||
},
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
Scaffold(
|
||||
@@ -289,10 +295,7 @@ fun DashboardSignInScreen(
|
||||
TopAppBar(
|
||||
title = { Text(stringResource(R.string.dashboard_sign_in)) },
|
||||
navigationIcon = {
|
||||
IconButton(onClick = {
|
||||
nativeSignInJob?.cancel()
|
||||
onBack()
|
||||
}) {
|
||||
IconButton(onClick = onBack) {
|
||||
Icon(
|
||||
Icons.AutoMirrored.Filled.ArrowBack,
|
||||
contentDescription = stringResource(R.string.dashboard_back),
|
||||
@@ -321,13 +324,11 @@ fun DashboardSignInScreen(
|
||||
actionInFlight = actionInFlight,
|
||||
actionMessage = actionMessage,
|
||||
actionIsError = actionIsError,
|
||||
nativePkce = redirectAuthMode == DashboardRedirectAuthMode.NativePkce,
|
||||
nativeSignInInFlight = nativeSignInJob != null,
|
||||
nativeTransportEligible = isNativeDashboardTransportEligible(dashboardUrl),
|
||||
onSignIn = ::submitPassword,
|
||||
onOAuthSignIn = ::startRedirectSignIn,
|
||||
onCancelNativeSignIn = {
|
||||
actionMessage = context.getString(R.string.dashboard_native_signin_cancelled)
|
||||
actionMessage = resources.getString(R.string.dashboard_native_signin_cancelled)
|
||||
actionIsError = false
|
||||
nativeSignInJob?.cancel()
|
||||
},
|
||||
@@ -396,9 +397,7 @@ private fun DashboardSignInForm(
|
||||
actionInFlight: Boolean,
|
||||
actionMessage: String?,
|
||||
actionIsError: Boolean,
|
||||
nativePkce: Boolean,
|
||||
nativeSignInInFlight: Boolean,
|
||||
nativeTransportEligible: Boolean,
|
||||
onSignIn: (String, String, String) -> Unit,
|
||||
onOAuthSignIn: (DashboardAuthProvider) -> Unit,
|
||||
onCancelNativeSignIn: () -> Unit,
|
||||
@@ -427,18 +426,19 @@ private fun DashboardSignInForm(
|
||||
redirectProviders.forEach { provider ->
|
||||
Button(
|
||||
onClick = { onOAuthSignIn(provider) },
|
||||
enabled = !actionInFlight && (!nativePkce || nativeTransportEligible),
|
||||
enabled = !actionInFlight,
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
) {
|
||||
Text(stringResource(R.string.dashboard_signin_with_provider, provider.displayName ?: provider.name))
|
||||
}
|
||||
}
|
||||
if (nativePkce && !nativeTransportEligible && redirectProviders.isNotEmpty()) {
|
||||
Text(
|
||||
text = stringResource(R.string.dashboard_native_signin_requires_https),
|
||||
style = MaterialTheme.typography.bodySmall,
|
||||
color = MaterialTheme.colorScheme.error,
|
||||
)
|
||||
if (nativeSignInInFlight) {
|
||||
Button(
|
||||
onClick = onCancelNativeSignIn,
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
) {
|
||||
Text(stringResource(R.string.dashboard_cancel))
|
||||
}
|
||||
}
|
||||
if (passwordProvider != null || providers.isEmpty()) {
|
||||
if (redirectProviders.isNotEmpty()) HorizontalDivider()
|
||||
@@ -476,18 +476,11 @@ private fun DashboardSignInForm(
|
||||
},
|
||||
)
|
||||
}
|
||||
if (nativeSignInInFlight) {
|
||||
Button(
|
||||
onClick = onCancelNativeSignIn,
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
) {
|
||||
Text(stringResource(R.string.dashboard_cancel))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@OptIn(ExperimentalMaterial3Api::class)
|
||||
@Composable
|
||||
private fun DashboardOAuthDialog(
|
||||
private fun DashboardOAuthScreen(
|
||||
dashboardUrl: String,
|
||||
provider: DashboardAuthProvider,
|
||||
cookieStoreFactory: () -> DashboardCookieStore,
|
||||
@@ -504,6 +497,8 @@ private fun DashboardOAuthDialog(
|
||||
val verifyFailedStatus = stringResource(R.string.dashboard_oauth_verify_failed)
|
||||
var statusText by remember(initialStatus) { mutableStateOf(initialStatus) }
|
||||
var checking by remember { mutableStateOf(false) }
|
||||
var pageProgress by remember { mutableStateOf(0) }
|
||||
var webView by remember { mutableStateOf<WebView?>(null) }
|
||||
val loginUrl = remember(dashboardUrl, provider.name) {
|
||||
DashboardApiClient.authLoginUrl(
|
||||
baseUrl = dashboardUrl,
|
||||
@@ -512,14 +507,17 @@ private fun DashboardOAuthDialog(
|
||||
)
|
||||
}
|
||||
|
||||
fun maybeVerify(url: String?) {
|
||||
fun handleNavigation(url: String?) {
|
||||
val loadedUrl = url?.takeIf { it.isNotBlank() } ?: return
|
||||
val root = dashboardUrl.trim().trimEnd('/')
|
||||
val relative = loadedUrl.trim().removePrefix(root)
|
||||
val stillAuthenticating = relative.startsWith("/login", true) ||
|
||||
relative.startsWith("/auth/login", true) ||
|
||||
relative.startsWith("/auth/callback", true)
|
||||
if (!loadedUrl.startsWith(root, true) || stillAuthenticating) return
|
||||
when (dashboardWebViewAuthNavigation(dashboardUrl, loadedUrl)) {
|
||||
DashboardWebViewAuthNavigation.Continue -> return
|
||||
DashboardWebViewAuthNavigation.RejectLoopbackCallback -> {
|
||||
statusText = notAcceptedStatus
|
||||
onError(notAcceptedStatus)
|
||||
return
|
||||
}
|
||||
DashboardWebViewAuthNavigation.ImportAndVerify -> Unit
|
||||
}
|
||||
val manager = CookieManager.getInstance()
|
||||
manager.flush()
|
||||
val imported = importDashboardCookieHeader(
|
||||
@@ -549,39 +547,162 @@ private fun DashboardOAuthDialog(
|
||||
}
|
||||
}
|
||||
|
||||
Dialog(onDismissRequest = onDismiss) {
|
||||
Card(modifier = Modifier.fillMaxWidth().heightIn(max = 640.dp)) {
|
||||
Column(
|
||||
modifier = Modifier.padding(12.dp),
|
||||
verticalArrangement = Arrangement.spacedBy(8.dp),
|
||||
) {
|
||||
IconButton(onClick = onDismiss) {
|
||||
Icon(Icons.Filled.Close, contentDescription = stringResource(R.string.dashboard_close_signin))
|
||||
}
|
||||
Text(statusText, style = MaterialTheme.typography.bodySmall)
|
||||
AndroidView(
|
||||
modifier = Modifier.fillMaxWidth().weight(1f),
|
||||
factory = { viewContext ->
|
||||
CookieManager.getInstance().setAcceptCookie(true)
|
||||
WebView(viewContext).apply {
|
||||
settings.javaScriptEnabled = true
|
||||
settings.domStorageEnabled = true
|
||||
webViewClient = object : WebViewClient() {
|
||||
override fun shouldOverrideUrlLoading(
|
||||
view: WebView,
|
||||
request: WebResourceRequest,
|
||||
): Boolean = false
|
||||
BackHandler(onBack = onDismiss)
|
||||
|
||||
override fun onPageFinished(view: WebView, url: String?) {
|
||||
super.onPageFinished(view, url)
|
||||
maybeVerify(url)
|
||||
}
|
||||
}
|
||||
loadUrl(loginUrl)
|
||||
}
|
||||
},
|
||||
DisposableEffect(Unit) {
|
||||
onDispose {
|
||||
webView?.stopLoading()
|
||||
webView?.destroy()
|
||||
webView = null
|
||||
}
|
||||
}
|
||||
|
||||
Scaffold(
|
||||
topBar = {
|
||||
TopAppBar(
|
||||
title = {
|
||||
Column {
|
||||
Text(
|
||||
text = stringResource(
|
||||
R.string.dashboard_signin_with_provider,
|
||||
provider.displayName ?: provider.name,
|
||||
),
|
||||
style = MaterialTheme.typography.titleMedium,
|
||||
)
|
||||
Text(
|
||||
text = statusText,
|
||||
style = MaterialTheme.typography.labelSmall,
|
||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||
maxLines = 1,
|
||||
)
|
||||
}
|
||||
},
|
||||
navigationIcon = {
|
||||
IconButton(onClick = onDismiss) {
|
||||
Icon(
|
||||
Icons.AutoMirrored.Filled.ArrowBack,
|
||||
contentDescription = stringResource(R.string.dashboard_back),
|
||||
)
|
||||
}
|
||||
},
|
||||
)
|
||||
},
|
||||
) { innerPadding ->
|
||||
Column(
|
||||
modifier = Modifier
|
||||
.fillMaxSize()
|
||||
.padding(innerPadding),
|
||||
) {
|
||||
if (pageProgress in 0..99) {
|
||||
LinearProgressIndicator(
|
||||
progress = { pageProgress / 100f },
|
||||
modifier = Modifier.fillMaxWidth(),
|
||||
)
|
||||
}
|
||||
AndroidView(
|
||||
modifier = Modifier
|
||||
.fillMaxWidth()
|
||||
.weight(1f),
|
||||
factory = { viewContext ->
|
||||
CookieManager.getInstance().setAcceptCookie(true)
|
||||
WebView(viewContext).apply {
|
||||
settings.javaScriptEnabled = true
|
||||
settings.domStorageEnabled = true
|
||||
webChromeClient = object : WebChromeClient() {
|
||||
override fun onProgressChanged(view: WebView, newProgress: Int) {
|
||||
pageProgress = newProgress
|
||||
}
|
||||
}
|
||||
webViewClient = object : WebViewClient() {
|
||||
override fun shouldOverrideUrlLoading(
|
||||
view: WebView,
|
||||
request: WebResourceRequest,
|
||||
): Boolean {
|
||||
val target = request.url.toString()
|
||||
if (
|
||||
dashboardWebViewAuthNavigation(dashboardUrl, target) ==
|
||||
DashboardWebViewAuthNavigation.RejectLoopbackCallback
|
||||
) {
|
||||
handleNavigation(target)
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
override fun onReceivedError(
|
||||
view: WebView,
|
||||
request: WebResourceRequest,
|
||||
error: WebResourceError,
|
||||
) {
|
||||
super.onReceivedError(view, request, error)
|
||||
if (request.isForMainFrame) {
|
||||
val message = error.description?.toString()
|
||||
?.takeIf { it.isNotBlank() }
|
||||
?: verifyFailedStatus
|
||||
statusText = message
|
||||
onError(message)
|
||||
}
|
||||
}
|
||||
|
||||
override fun onPageFinished(view: WebView, url: String?) {
|
||||
super.onPageFinished(view, url)
|
||||
handleNavigation(url)
|
||||
}
|
||||
}
|
||||
webView = this
|
||||
loadUrl(loginUrl)
|
||||
}
|
||||
},
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
internal enum class DashboardWebViewAuthNavigation {
|
||||
Continue,
|
||||
ImportAndVerify,
|
||||
RejectLoopbackCallback,
|
||||
}
|
||||
|
||||
/**
|
||||
* Android redirect providers use the dashboard's cookie/OIDC flow. A foreign
|
||||
* loopback callback belongs to the desktop native-PKCE contract and must never
|
||||
* be followed, imported, or treated as an authenticated Android return.
|
||||
*/
|
||||
internal fun dashboardWebViewAuthNavigation(
|
||||
dashboardUrl: String,
|
||||
loadedUrl: String,
|
||||
): DashboardWebViewAuthNavigation {
|
||||
val dashboard = dashboardUrl.trim().trimEnd('/').toHttpUrlOrNull()
|
||||
?: return DashboardWebViewAuthNavigation.Continue
|
||||
val loaded = loadedUrl.trim().toHttpUrlOrNull()
|
||||
?: return DashboardWebViewAuthNavigation.Continue
|
||||
val sameOrigin = dashboard.scheme == loaded.scheme &&
|
||||
dashboard.host.equals(loaded.host, ignoreCase = true) &&
|
||||
dashboard.port == loaded.port
|
||||
if (!sameOrigin) {
|
||||
val foreignLoopback = loaded.scheme == "http" &&
|
||||
loaded.host in setOf("127.0.0.1", "localhost", "::1") &&
|
||||
loaded.encodedPath == "/callback"
|
||||
return if (foreignLoopback) {
|
||||
DashboardWebViewAuthNavigation.RejectLoopbackCallback
|
||||
} else {
|
||||
DashboardWebViewAuthNavigation.Continue
|
||||
}
|
||||
}
|
||||
|
||||
val basePath = dashboard.encodedPath.trimEnd('/')
|
||||
val relativePath = loaded.encodedPath
|
||||
.removePrefix(basePath)
|
||||
.ifBlank { "/" }
|
||||
return if (
|
||||
relativePath.equals("/login", ignoreCase = true) ||
|
||||
relativePath.equals("/auth/login", ignoreCase = true)
|
||||
) {
|
||||
DashboardWebViewAuthNavigation.Continue
|
||||
} else {
|
||||
// Includes the public /auth/callback response: import its cookies at
|
||||
// root scope, then verify the resulting session through /api/auth/me.
|
||||
DashboardWebViewAuthNavigation.ImportAndVerify
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3224,6 +3224,17 @@ private fun GlobalVoiceControlsCard(
|
||||
|
||||
HorizontalDivider(modifier = Modifier.padding(vertical = 8.dp))
|
||||
|
||||
SettingSwitchRow(
|
||||
title = stringResource(R.string.voice_settings_final_answer_only),
|
||||
detail = stringResource(R.string.voice_settings_final_answer_only_desc),
|
||||
checked = voiceSettings.finalAnswerOnly,
|
||||
onCheckedChange = { enabled ->
|
||||
scope.launch { prefsRepo.setFinalAnswerOnly(enabled) }
|
||||
},
|
||||
)
|
||||
|
||||
HorizontalDivider(modifier = Modifier.padding(vertical = 8.dp))
|
||||
|
||||
Text(
|
||||
text = stringResource(R.string.voice_settings_interaction_mode),
|
||||
style = MaterialTheme.typography.labelLarge,
|
||||
|
||||
@@ -22,11 +22,16 @@ import javax.net.ssl.SSLPeerUnverifiedException
|
||||
* showHumanError in RelayApp.kt.
|
||||
*/
|
||||
|
||||
enum class HumanErrorAction {
|
||||
Repair,
|
||||
}
|
||||
|
||||
data class HumanError(
|
||||
val title: String,
|
||||
val body: String,
|
||||
val retryable: Boolean = false,
|
||||
val actionLabel: String? = null,
|
||||
val action: HumanErrorAction? = null,
|
||||
)
|
||||
|
||||
private fun titlePrefix(context: String?, ctx: Context?): String = ctx?.let { c ->
|
||||
@@ -116,6 +121,7 @@ private fun classifyIoMessage(msg: String, context: String?, ctx: Context?): Hum
|
||||
body = "Your session is no longer valid — re-pair this device",
|
||||
retryable = false,
|
||||
actionLabel = ctx?.getString(R.string.error_classify_repair) ?: "Re-pair",
|
||||
action = HumanErrorAction.Repair,
|
||||
)
|
||||
"403" in msg || "forbidden" in msg -> HumanError(
|
||||
title = ctx?.getString(R.string.error_classify_not_allowed) ?: "Not allowed",
|
||||
@@ -271,6 +277,7 @@ private fun classifyErrorInternal(t: Throwable?, context: String?, ctx: Context?
|
||||
body = "The server certificate changed since you paired — re-pair to trust it",
|
||||
retryable = false,
|
||||
actionLabel = ctx?.getString(R.string.error_classify_repair) ?: "Re-pair",
|
||||
action = HumanErrorAction.Repair,
|
||||
)
|
||||
is SecurityException -> HumanError(
|
||||
title = ctx?.getString(R.string.error_classify_perm_needed) ?: "Permission needed",
|
||||
|
||||
@@ -5922,6 +5922,13 @@ class ChatViewModel : ViewModel() {
|
||||
queuedCount = event.queuedCount ?: 0,
|
||||
),
|
||||
)
|
||||
if (event.spokenHandoff == false) {
|
||||
// Silent promotion is the foreground turn boundary. The
|
||||
// background run keeps its owner/card and continues to
|
||||
// receive progress, cancellation, and delivery events.
|
||||
handler.onStreamComplete(assistantMessageId)
|
||||
activeStream = null
|
||||
}
|
||||
}
|
||||
"hermes.run.queued" -> {
|
||||
handler.updateBackgroundTask(assistantMessageId) { task ->
|
||||
|
||||
@@ -212,9 +212,12 @@ internal fun resolveEffectiveDashboardUrl(
|
||||
endpoint?.dashboard?.url
|
||||
?.takeIf { it.isNotBlank() }
|
||||
?.let { return it }
|
||||
endpoint?.api?.url
|
||||
?.let(Connection::deriveDefaultDashboardUrl)
|
||||
?.let { return it }
|
||||
endpoint?.api?.url?.let { apiUrl ->
|
||||
connection.dashboardUrl
|
||||
?.takeIf { it.isNotBlank() && Connection.urlsShareHost(it, apiUrl) }
|
||||
?.let { return it }
|
||||
Connection.deriveDefaultDashboardUrl(apiUrl)?.let { return it }
|
||||
}
|
||||
return connection.resolvedDashboardUrl
|
||||
}
|
||||
|
||||
@@ -788,6 +791,7 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
|
||||
apiServerUrl = apiServerUrl,
|
||||
relayUrl = relayUrl,
|
||||
extraApiUrls = extraApiUrls,
|
||||
dashboardUrl = activeConnection.value?.resolvedDashboardUrl,
|
||||
),
|
||||
existing = activeConnection.value?.routeCandidates.orEmpty(),
|
||||
)
|
||||
@@ -4686,17 +4690,21 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
|
||||
} else {
|
||||
current.dashboardUrl
|
||||
}
|
||||
val newRouteCandidates = Connection.reconcileDashboardRoutes(
|
||||
dashboardUrl = newDashboardUrl,
|
||||
candidates = payload.endpoints.orEmpty(),
|
||||
)
|
||||
val needsUpdate = current.apiServerUrl != payload.serverUrl ||
|
||||
current.relayUrl != newRelayUrl ||
|
||||
current.dashboardUrl != newDashboardUrl ||
|
||||
current.routeCandidates != payload.endpoints.orEmpty()
|
||||
current.routeCandidates != newRouteCandidates
|
||||
if (needsUpdate) {
|
||||
connectionStore.updateConnection(
|
||||
current.copy(
|
||||
apiServerUrl = payload.serverUrl,
|
||||
relayUrl = newRelayUrl,
|
||||
dashboardUrl = newDashboardUrl,
|
||||
routeCandidates = payload.endpoints.orEmpty(),
|
||||
routeCandidates = newRouteCandidates,
|
||||
preferredRouteRole = current.preferredRouteRole
|
||||
?.takeIf { preferred ->
|
||||
payload.endpoints.orEmpty().any {
|
||||
@@ -4927,6 +4935,22 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
|
||||
current.copy(
|
||||
label = nextLabel,
|
||||
dashboardUrl = normalized,
|
||||
routeCandidates = Connection.reconcileDashboardRoutes(
|
||||
dashboardUrl = normalized,
|
||||
candidates = current.routeCandidates.ifEmpty {
|
||||
listOfNotNull(
|
||||
Connection.endpointCandidateFromDashboardUrl(
|
||||
role = Connection.inferRouteRole(normalized),
|
||||
priority = 0,
|
||||
dashboardUrl = normalized,
|
||||
apiServerUrl = current.apiServerUrl
|
||||
.takeIf { it.isNotBlank() },
|
||||
relayUrl = current.relayUrl
|
||||
.takeIf { it.isNotBlank() },
|
||||
),
|
||||
)
|
||||
},
|
||||
),
|
||||
),
|
||||
)
|
||||
probeStandardVoice()
|
||||
@@ -6116,16 +6140,6 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
|
||||
) {
|
||||
val activeId = connectionStore.activeConnectionId.value ?: return
|
||||
val current = connectionStore.connections.value.firstOrNull { it.id == activeId } ?: return
|
||||
val nextRouteCandidates = routeCandidates ?: current.routeCandidates
|
||||
val nextPreferredRouteRole = when {
|
||||
preferredRouteRole != null -> preferredRouteRole.takeIf { it.isNotBlank() }
|
||||
routeCandidates != null &&
|
||||
current.preferredRouteRole != null &&
|
||||
nextRouteCandidates.none {
|
||||
it.role.equals(current.preferredRouteRole, ignoreCase = true)
|
||||
} -> null
|
||||
else -> current.preferredRouteRole
|
||||
}
|
||||
val nextDashboardUrl = when {
|
||||
dashboardUrlOverride != null -> {
|
||||
dashboardUrlOverride
|
||||
@@ -6139,6 +6153,19 @@ class ConnectionViewModel(application: Application) : AndroidViewModel(applicati
|
||||
}
|
||||
else -> current.dashboardUrl
|
||||
}
|
||||
val nextRouteCandidates = Connection.reconcileDashboardRoutes(
|
||||
dashboardUrl = nextDashboardUrl,
|
||||
candidates = routeCandidates ?: current.routeCandidates,
|
||||
)
|
||||
val nextPreferredRouteRole = when {
|
||||
preferredRouteRole != null -> preferredRouteRole.takeIf { it.isNotBlank() }
|
||||
routeCandidates != null &&
|
||||
current.preferredRouteRole != null &&
|
||||
nextRouteCandidates.none {
|
||||
it.role.equals(current.preferredRouteRole, ignoreCase = true)
|
||||
} -> null
|
||||
else -> current.preferredRouteRole
|
||||
}
|
||||
if (
|
||||
current.apiServerUrl == apiServerUrl &&
|
||||
current.relayUrl == relayUrl &&
|
||||
|
||||
@@ -111,7 +111,16 @@ internal data class AssistantSpeechBatch(
|
||||
val assistantMessages: List<ChatMessage>,
|
||||
val aggregateText: String,
|
||||
val hasTurnAssistant: Boolean,
|
||||
)
|
||||
) {
|
||||
/** Last non-empty assistant bubble: the settled answer after any tool commentary. */
|
||||
val finalAnswerText: String
|
||||
get() = assistantMessages
|
||||
.asReversed()
|
||||
.firstOrNull { it.content.isNotBlank() }
|
||||
?.content
|
||||
?.trim()
|
||||
.orEmpty()
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-voice-turn cursor over every assistant bubble created after the user
|
||||
@@ -323,9 +332,8 @@ enum class BackgroundRunPhase {
|
||||
DONE,
|
||||
}
|
||||
|
||||
internal fun realtimeTurnActiveAfterResponseDone(backgroundPhase: BackgroundRunPhase?): Boolean =
|
||||
backgroundPhase == BackgroundRunPhase.RUNNING ||
|
||||
backgroundPhase == BackgroundRunPhase.RECONNECTING
|
||||
internal fun realtimeTurnActiveAfterPromotion(spokenHandoff: Boolean?): Boolean =
|
||||
spokenHandoff != false
|
||||
|
||||
internal fun preserveRealtimeTurnOnStop(backgroundPhase: BackgroundRunPhase?): Boolean =
|
||||
backgroundPhase == BackgroundRunPhase.RUNNING ||
|
||||
@@ -648,6 +656,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
private var voicePreferences: VoicePreferencesRepository? = null
|
||||
private var voicePreferencesJob: Job? = null
|
||||
private var voiceEngineMode: VoiceEngineMode = VoiceEngineMode.HermesVoiceOutput
|
||||
private var finalAnswerOnly: Boolean = false
|
||||
private var realtimeTraceDetails: Boolean = false
|
||||
private var realtimePersistentSession: Boolean = true
|
||||
private var realtimeModel: String = ""
|
||||
@@ -1368,10 +1377,12 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
*/
|
||||
private fun applyVoiceSettingsSnapshot(settings: com.hermesandroid.relay.data.VoiceSettings) {
|
||||
val nextEngineMode = VoiceEngineMode.fromStorage(settings.engineMode)
|
||||
val finalAnswerPolicyChanged = finalAnswerOnly != settings.finalAnswerOnly
|
||||
val realtimeSelectionChanged =
|
||||
realtimeModel != settings.realtimeModel || realtimeVoice != settings.realtimeVoice
|
||||
if (
|
||||
voiceEngineMode != nextEngineMode ||
|
||||
finalAnswerPolicyChanged ||
|
||||
realtimeTraceDetails != settings.realtimeTraceDetails ||
|
||||
realtimeSelectionChanged
|
||||
) {
|
||||
@@ -1379,6 +1390,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
TAG,
|
||||
"Voice prefs updated engine=${nextEngineMode.storageValue} " +
|
||||
"interaction=${settings.interactionMode} " +
|
||||
"finalAnswerOnly=${settings.finalAnswerOnly} " +
|
||||
"realtimeTraceDetails=${settings.realtimeTraceDetails} " +
|
||||
"realtimeModel=${settings.realtimeModel.ifBlank { "relay-default" }} " +
|
||||
"realtimeVoice=${settings.realtimeVoice.ifBlank { "relay-default" }}",
|
||||
@@ -1390,11 +1402,13 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
(voiceEngineMode == VoiceEngineMode.RealtimeAgent &&
|
||||
nextEngineMode != VoiceEngineMode.RealtimeAgent) ||
|
||||
(realtimePersistentSession && !settings.realtimePersistentSession) ||
|
||||
finalAnswerPolicyChanged ||
|
||||
realtimeSelectionChanged
|
||||
) {
|
||||
closeRealtimeSession()
|
||||
}
|
||||
voiceEngineMode = nextEngineMode
|
||||
finalAnswerOnly = settings.finalAnswerOnly
|
||||
realtimeTraceDetails = settings.realtimeTraceDetails
|
||||
realtimePersistentSession = settings.realtimePersistentSession
|
||||
realtimeModel = settings.realtimeModel
|
||||
@@ -3162,17 +3176,21 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
resumeWatchdog?.cancel(); resumeWatchdog = null
|
||||
clearSpokenChunksState()
|
||||
|
||||
prepareStandardSpeechStream()
|
||||
|
||||
// Kick off streaming observer BEFORE sending the message so we don't
|
||||
// miss early deltas that arrive synchronously from the callback.
|
||||
startStreamObserver(chatVm)
|
||||
if (!finalAnswerOnly) {
|
||||
prepareStandardSpeechStream()
|
||||
}
|
||||
|
||||
// Route the transcribed text through the normal chat pipeline.
|
||||
// This will create a user message + kick off the SSE stream.
|
||||
// This creates the user row synchronously before kicking off the
|
||||
// transport, so bind the turn before observing the replaying StateFlows.
|
||||
// Starting the observer first leaves a small window where a legitimate
|
||||
// session adoption can be rejected before the submitted user key exists.
|
||||
// StateFlow replay preserves any assistant text that arrives before the
|
||||
// observer starts.
|
||||
val submittedUserUiKey =
|
||||
chatVm.sendVoiceMessage(userText, STABLE_VOICE_INTERFACE_CONTEXT)
|
||||
voiceTurnSessionFence?.bindSubmittedUser(submittedUserUiKey)
|
||||
startStreamObserver(chatVm)
|
||||
}
|
||||
|
||||
private suspend fun runVoiceRelayPreflight(engineLabel: String): Boolean {
|
||||
@@ -3299,7 +3317,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
)
|
||||
}
|
||||
}
|
||||
if (speak && (!audioSeen.get() || speakEvenAfterProviderAudio)) {
|
||||
if (speak && !finalAnswerOnly && (!audioSeen.get() || speakEvenAfterProviderAudio)) {
|
||||
// W3: per-turn throttle independent of the per-key dedupe above.
|
||||
// Suppress the TTS enqueue (UI state + diagnostics already
|
||||
// applied) when spoken status is too frequent or has hit the
|
||||
@@ -3371,6 +3389,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
conversationContext = conversationContext,
|
||||
model = realtimeModel,
|
||||
voice = realtimeVoice,
|
||||
finalAnswerOnly = finalAnswerOnly,
|
||||
onHandoff = { event -> recordRealtimeVoiceHandoff(sessionGeneration, event) },
|
||||
turnInputs = if (persistentOpen) realtimeTurnChannel else null,
|
||||
onTurnComplete = { summary ->
|
||||
@@ -3415,9 +3434,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
}
|
||||
if (suppressCommandResponse) {
|
||||
if (event.type == "voice.response.done") {
|
||||
providerRealtimeAgentTurnActive.set(
|
||||
realtimeTurnActiveAfterResponseDone(_uiState.value.backgroundRun?.phase),
|
||||
)
|
||||
providerRealtimeAgentTurnActive.set(false)
|
||||
suppressLocalCommandResponse = false
|
||||
realtimeAudioSuppressed = false
|
||||
}
|
||||
@@ -3638,10 +3655,12 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
}
|
||||
}
|
||||
"hermes.run.promoted" -> {
|
||||
providerRealtimeAgentTurnActive.set(true)
|
||||
// The run detached to the background; the provider speaks the
|
||||
// handoff. Surface a persistent chip so the user knows a long
|
||||
// task is still in flight (ADR 33 Tier B/C).
|
||||
providerRealtimeAgentTurnActive.set(
|
||||
realtimeTurnActiveAfterPromotion(event.spokenHandoff),
|
||||
)
|
||||
// The run detached to the background. A spoken handoff keeps
|
||||
// the foreground turn active until response.done; a silent
|
||||
// handoff ends it here. The task chip remains either way.
|
||||
val tier = event.tier ?: "promoted"
|
||||
Log.i(
|
||||
TAG,
|
||||
@@ -3845,9 +3864,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
hermesConfirmation = null,
|
||||
)
|
||||
}
|
||||
providerRealtimeAgentTurnActive.set(
|
||||
realtimeTurnActiveAfterResponseDone(_uiState.value.backgroundRun?.phase)
|
||||
)
|
||||
providerRealtimeAgentTurnActive.set(false)
|
||||
}
|
||||
"voice.error" -> {
|
||||
realtimeConfirmationControl = null
|
||||
@@ -4328,26 +4345,42 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
) { messages, runActive, sessionId -> Triple(messages, runActive, sessionId) }
|
||||
.collect { (messages, runActive, sessionId) ->
|
||||
if (!sessionFence.accepts(sessionId, messages)) {
|
||||
cancelStandardSpeechStream("chat session changed")
|
||||
streamObserverJob?.cancel()
|
||||
// Session id and message history are independent flows.
|
||||
// During session creation/adoption, combine can briefly
|
||||
// pair the new id with the old history (or vice versa).
|
||||
// Skip that inconsistent snapshot without permanently
|
||||
// killing narration; the next coherent emission is still
|
||||
// fenced by the submitted user row/session identity.
|
||||
return@collect
|
||||
}
|
||||
|
||||
val batch = cursor.poll(messages)
|
||||
batch.deltas.forEach { update ->
|
||||
if (update.startsNewBubble) {
|
||||
beginAssistantSpeechBubble()
|
||||
if (finalAnswerOnly) {
|
||||
if (batch.deltas.isNotEmpty()) {
|
||||
onVisualStreamDelta(batch.aggregateText)
|
||||
}
|
||||
} else {
|
||||
batch.deltas.forEach { update ->
|
||||
if (update.startsNewBubble) {
|
||||
beginAssistantSpeechBubble()
|
||||
}
|
||||
onStreamDelta(update.text, batch.aggregateText)
|
||||
}
|
||||
onStreamDelta(update.text, batch.aggregateText)
|
||||
}
|
||||
// Tool state can change without text growth.
|
||||
batch.assistantMessages.forEach(::observeHermesToolLoopForSpeech)
|
||||
if (!finalAnswerOnly) {
|
||||
batch.assistantMessages.forEach(::observeHermesToolLoopForSpeech)
|
||||
}
|
||||
|
||||
if (!runActive && batch.hasTurnAssistant) {
|
||||
streamComplete = true
|
||||
idleFlushJob?.cancel()
|
||||
idleFlushJob = null
|
||||
if (!finishStandardSpeechStream()) flushRemainingBuffer()
|
||||
if (finalAnswerOnly) {
|
||||
speakSettledFinalAnswer(batch.finalAnswerText)
|
||||
} else if (!finishStandardSpeechStream()) {
|
||||
flushRemainingBuffer()
|
||||
}
|
||||
streamObserverJob?.cancel()
|
||||
scheduleAgentAudioCompletionCheck()
|
||||
}
|
||||
@@ -4355,6 +4388,44 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
}
|
||||
}
|
||||
|
||||
private fun onVisualStreamDelta(fullContent: String) {
|
||||
_uiState.update {
|
||||
it.copy(
|
||||
state = VoiceState.Thinking,
|
||||
outputAudioActive = false,
|
||||
responseText = fullContent,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Final-only mode deliberately trades streaming latency for a clean spoken
|
||||
* result. The last non-empty assistant bubble is the settled answer; earlier
|
||||
* bubbles and tool states remain visible in Chat but never enter TTS.
|
||||
*/
|
||||
private fun speakSettledFinalAnswer(answer: String) {
|
||||
val spoken = sanitizeForTts(answer)
|
||||
if (spoken.isBlank()) return
|
||||
|
||||
_uiState.update {
|
||||
it.copy(
|
||||
state = VoiceState.Speaking,
|
||||
outputAudioActive = false,
|
||||
responseText = answer,
|
||||
)
|
||||
}
|
||||
|
||||
prepareStandardSpeechStream()
|
||||
if (offerStandardSpeechText(spoken)) {
|
||||
finishStandardSpeechStream()
|
||||
} else {
|
||||
sentenceBuffer = StringBuilder()
|
||||
pendingRawDelta = StringBuilder()
|
||||
appendSanitizedDelta(spoken)
|
||||
flushRemainingBuffer()
|
||||
}
|
||||
}
|
||||
|
||||
private fun beginAssistantSpeechBubble() {
|
||||
idleFlushJob?.cancel()
|
||||
idleFlushJob = null
|
||||
@@ -4393,7 +4464,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
|
||||
* device through the normal tool loop.
|
||||
*/
|
||||
private fun observeHermesToolLoopForSpeech(message: ChatMessage) {
|
||||
if (!_uiState.value.voiceMode || message.toolCalls.isEmpty()) return
|
||||
if (finalAnswerOnly || !_uiState.value.voiceMode || message.toolCalls.isEmpty()) return
|
||||
|
||||
var spokenForMessage = brokeredToolSpeechCounts[message.id] ?: 0
|
||||
message.toolCalls.forEach { tool ->
|
||||
|
||||
@@ -1502,6 +1502,8 @@
|
||||
<string name="voice_settings_save_realtime_agent">Salvar agente em tempo real</string>
|
||||
<string name="voice_settings_global_controls_title">Controles globais de voz</string>
|
||||
<string name="voice_settings_global_controls_desc">Estas configurações se aplicam aos dois mecanismos de voz em todos os perfis.</string>
|
||||
<string name="voice_settings_final_answer_only">Somente a resposta final</string>
|
||||
<string name="voice_settings_final_answer_only_desc">Fala apenas a resposta concluída. O progresso das ferramentas, as atualizações de serviço e os comentários intermediários permanecem visuais.</string>
|
||||
<string name="voice_settings_interaction_mode">Modo de interação</string>
|
||||
<string name="voice_settings_interaction_tap">Tocar para falar</string>
|
||||
<string name="voice_settings_interaction_hold">Manter pressionado para falar</string>
|
||||
@@ -1996,8 +1998,12 @@
|
||||
<string name="voice_overlay_collapse_cd">Recolher controles de voz</string>
|
||||
<string name="voice_overlay_expand_cd">Expandir controles de voz</string>
|
||||
<string name="voice_overlay_exit_cd">Sair do modo de voz</string>
|
||||
<string name="voice_overlay_compact">Compacto</string>
|
||||
<string name="voice_overlay_focus">Foco</string>
|
||||
<string name="voice_overlay_conversation">Conversa</string>
|
||||
<string name="voice_overlay_image_ready">Imagem pronta</string>
|
||||
<string name="voice_overlay_rich_result_ready">Resultado avançado pronto</string>
|
||||
<string name="voice_overlay_rich_results_count">%1$d resultados prontos</string>
|
||||
<string name="voice_overlay_view_conversation">Ver conversa</string>
|
||||
<string name="voice_overlay_overlay">Sobreposição</string>
|
||||
<string name="voice_overlay_exit">Sair</string>
|
||||
<string name="voice_overlay_settings_cd">Configurações de voz</string>
|
||||
|
||||
@@ -1563,6 +1563,8 @@
|
||||
<string name="voice_settings_save_realtime_agent">保存实时 Agent</string>
|
||||
<string name="voice_settings_global_controls_title">全局语音控制</string>
|
||||
<string name="voice_settings_global_controls_desc">这些设置适用于所有个人资料上的两个语音引擎。</string>
|
||||
<string name="voice_settings_final_answer_only">仅朗读最终答案</string>
|
||||
<string name="voice_settings_final_answer_only_desc">只朗读最终确定的答案。工具进度、服务更新和中间评论仍仅以视觉方式显示。</string>
|
||||
<string name="voice_settings_interaction_mode">交互模式</string>
|
||||
<string name="voice_settings_interaction_tap">点击说话</string>
|
||||
<string name="voice_settings_interaction_hold">按住说话</string>
|
||||
@@ -2088,8 +2090,12 @@
|
||||
<string name="voice_overlay_collapse_cd">收起语音控制</string>
|
||||
<string name="voice_overlay_expand_cd">展开语音控制</string>
|
||||
<string name="voice_overlay_exit_cd">退出语音模式</string>
|
||||
<string name="voice_overlay_compact">紧凑</string>
|
||||
<string name="voice_overlay_focus">专注</string>
|
||||
<string name="voice_overlay_conversation">对话</string>
|
||||
<string name="voice_overlay_image_ready">图片已就绪</string>
|
||||
<string name="voice_overlay_rich_result_ready">丰富结果已就绪</string>
|
||||
<string name="voice_overlay_rich_results_count">%1$d 个结果已就绪</string>
|
||||
<string name="voice_overlay_view_conversation">查看对话</string>
|
||||
<string name="voice_overlay_overlay">浮窗</string>
|
||||
<string name="voice_overlay_exit">退出</string>
|
||||
<string name="voice_overlay_settings_cd">语音设置</string>
|
||||
|
||||
@@ -1566,6 +1566,8 @@
|
||||
<string name="voice_settings_save_realtime_agent">Echtzeit-Agent speichern</string>
|
||||
<string name="voice_settings_global_controls_title">Globale Sprachsteuerung</string>
|
||||
<string name="voice_settings_global_controls_desc">Diese Einstellungen gelten für beide Sprach-Engines in jedem Profil.</string>
|
||||
<string name="voice_settings_final_answer_only">Nur endgültige Antwort</string>
|
||||
<string name="voice_settings_final_answer_only_desc">Spricht nur die abgeschlossene Antwort. Werkzeugfortschritt, Dienstmeldungen und Zwischenkommentare bleiben visuell.</string>
|
||||
<string name="voice_settings_interaction_mode">Interaktionsmodus</string>
|
||||
<string name="voice_settings_interaction_tap">Tippen zum Sprechen</string>
|
||||
<string name="voice_settings_interaction_hold">Halten zum Sprechen</string>
|
||||
@@ -2091,8 +2093,12 @@
|
||||
<string name="voice_overlay_collapse_cd">Sprachsteuerung einklappen</string>
|
||||
<string name="voice_overlay_expand_cd">Sprachsteuerung ausklappen</string>
|
||||
<string name="voice_overlay_exit_cd">Sprachmodus beenden</string>
|
||||
<string name="voice_overlay_compact">Kompakt</string>
|
||||
<string name="voice_overlay_focus">Fokus</string>
|
||||
<string name="voice_overlay_conversation">Gespräch</string>
|
||||
<string name="voice_overlay_image_ready">Bild bereit</string>
|
||||
<string name="voice_overlay_rich_result_ready">Rich-Ergebnis bereit</string>
|
||||
<string name="voice_overlay_rich_results_count">%1$d Ergebnisse bereit</string>
|
||||
<string name="voice_overlay_view_conversation">Gespräch anzeigen</string>
|
||||
<string name="voice_overlay_overlay">Overlay</string>
|
||||
<string name="voice_overlay_exit">Beenden</string>
|
||||
<string name="voice_overlay_settings_cd">Spracheinstellungen</string>
|
||||
|
||||
@@ -1442,6 +1442,8 @@
|
||||
<string name="voice_settings_save_realtime_agent">Guardar agente en tiempo real</string>
|
||||
<string name="voice_settings_global_controls_title">Controles de voz globales</string>
|
||||
<string name="voice_settings_global_controls_desc">Estas configuraciones se aplican a ambos motores de voz, en todos los perfiles.</string>
|
||||
<string name="voice_settings_final_answer_only">Solo la respuesta final</string>
|
||||
<string name="voice_settings_final_answer_only_desc">Reproduce únicamente la respuesta definitiva. El progreso de herramientas, las actualizaciones de servicio y los comentarios intermedios permanecen visuales.</string>
|
||||
<string name="voice_settings_interaction_mode">Modo de interacción</string>
|
||||
<string name="voice_settings_interaction_tap">Toca para hablar</string>
|
||||
<string name="voice_settings_interaction_hold">Espera para hablar</string>
|
||||
@@ -1903,8 +1905,12 @@
|
||||
<string name="voice_overlay_collapse_cd">Contraer controles de voz</string>
|
||||
<string name="voice_overlay_expand_cd">Ampliar los controles de voz</string>
|
||||
<string name="voice_overlay_exit_cd">Salir del modo de voz</string>
|
||||
<string name="voice_overlay_compact">Compacto</string>
|
||||
<string name="voice_overlay_focus">Enfocar</string>
|
||||
<string name="voice_overlay_conversation">Conversación</string>
|
||||
<string name="voice_overlay_image_ready">Imagen lista</string>
|
||||
<string name="voice_overlay_rich_result_ready">Resultado enriquecido listo</string>
|
||||
<string name="voice_overlay_rich_results_count">%1$d resultados listos</string>
|
||||
<string name="voice_overlay_view_conversation">Ver conversación</string>
|
||||
<string name="voice_overlay_overlay">Cubrir</string>
|
||||
<string name="voice_overlay_exit">Salida</string>
|
||||
<string name="voice_overlay_settings_cd">Configuraciones de voz</string>
|
||||
|
||||
@@ -1579,6 +1579,8 @@
|
||||
<string name="voice_settings_save_realtime_agent">リアルタイムエージェントを保存する</string>
|
||||
<string name="voice_settings_global_controls_title">グローバル音声コントロール</string>
|
||||
<string name="voice_settings_global_controls_desc">これらの設定は、すべてのプロファイルの両方の音声エンジンに適用されます。</string>
|
||||
<string name="voice_settings_final_answer_only">最終回答のみ</string>
|
||||
<string name="voice_settings_final_answer_only_desc">確定した回答だけを読み上げます。ツールの進行状況、サービス更新、中間コメントは画面表示のみになります。</string>
|
||||
<string name="voice_settings_interaction_mode">インタラクションモード</string>
|
||||
<string name="voice_settings_interaction_tap">タップして話す</string>
|
||||
<string name="voice_settings_interaction_hold">押し続けて話す</string>
|
||||
@@ -2104,8 +2106,12 @@
|
||||
<string name="voice_overlay_collapse_cd">音声コントロールを折りたたむ</string>
|
||||
<string name="voice_overlay_expand_cd">音声コントロールを拡張する</string>
|
||||
<string name="voice_overlay_exit_cd">音声モードを終了する</string>
|
||||
<string name="voice_overlay_compact">コンパクト</string>
|
||||
<string name="voice_overlay_focus">集中</string>
|
||||
<string name="voice_overlay_conversation">会話</string>
|
||||
<string name="voice_overlay_image_ready">画像の準備ができました</string>
|
||||
<string name="voice_overlay_rich_result_ready">リッチな結果の準備ができました</string>
|
||||
<string name="voice_overlay_rich_results_count">結果が%1$d件準備できました</string>
|
||||
<string name="voice_overlay_view_conversation">会話を表示</string>
|
||||
<string name="voice_overlay_overlay">かぶせる</string>
|
||||
<string name="voice_overlay_exit">出口</string>
|
||||
<string name="voice_overlay_settings_cd">音声設定</string>
|
||||
|
||||
@@ -1682,6 +1682,8 @@
|
||||
<string name="voice_settings_save_realtime_agent">Save realtime agent</string>
|
||||
<string name="voice_settings_global_controls_title">Global Voice Controls</string>
|
||||
<string name="voice_settings_global_controls_desc">These settings apply to both voice engines, on every profile.</string>
|
||||
<string name="voice_settings_final_answer_only">Final answer only</string>
|
||||
<string name="voice_settings_final_answer_only_desc">Speak only the settled answer. Tool progress, service updates, and intermediate commentary stay visual.</string>
|
||||
<string name="voice_settings_interaction_mode">Interaction mode</string>
|
||||
<string name="voice_settings_interaction_tap">Tap to talk</string>
|
||||
<string name="voice_settings_interaction_hold">Hold to talk</string>
|
||||
@@ -2211,8 +2213,12 @@
|
||||
<string name="voice_overlay_collapse_cd">Collapse voice controls</string>
|
||||
<string name="voice_overlay_expand_cd">Expand voice controls</string>
|
||||
<string name="voice_overlay_exit_cd">Exit voice mode</string>
|
||||
<string name="voice_overlay_compact">Compact</string>
|
||||
<string name="voice_overlay_focus">Focus</string>
|
||||
<string name="voice_overlay_conversation">Conversation</string>
|
||||
<string name="voice_overlay_focus">Voice focus</string>
|
||||
<string name="voice_overlay_image_ready">Image ready</string>
|
||||
<string name="voice_overlay_rich_result_ready">Rich result ready</string>
|
||||
<string name="voice_overlay_rich_results_count">%1$d results ready</string>
|
||||
<string name="voice_overlay_view_conversation">View conversation</string>
|
||||
<string name="voice_overlay_overlay">Overlay</string>
|
||||
<string name="voice_overlay_exit">Exit</string>
|
||||
<string name="voice_overlay_settings_cd">Voice settings</string>
|
||||
|
||||
@@ -111,6 +111,75 @@ class ConnectionDashboardFieldsTest {
|
||||
assertEquals("wss://hermes.tail1234.ts.net:8767", routes[1].relay?.url)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun buildRouteCandidates_preservesExplicitSameHostHttpsDashboard() {
|
||||
val routes = Connection.buildRouteCandidates(
|
||||
apiServerUrl = "https://hermes.example.com:8643",
|
||||
relayUrl = "wss://hermes.example.com:8767",
|
||||
dashboardUrl = "https://hermes.example.com:443",
|
||||
)
|
||||
|
||||
assertEquals(1, routes.size)
|
||||
assertEquals("https://hermes.example.com:443", routes.single().dashboard?.url)
|
||||
assertEquals("https://hermes.example.com:8643", routes.single().api?.url)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun reconcileDashboardRoutes_repairsStoredSameHostDerivedPort() {
|
||||
val stored = Connection.buildRouteCandidates(
|
||||
apiServerUrl = "https://hermes.example.com:8643",
|
||||
relayUrl = "wss://hermes.example.com:8767",
|
||||
)
|
||||
|
||||
val repaired = Connection.reconcileDashboardRoutes(
|
||||
dashboardUrl = "https://hermes.example.com:443",
|
||||
candidates = stored,
|
||||
)
|
||||
|
||||
assertEquals("https://hermes.example.com:443", repaired.single().dashboard?.url)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun reconcileDashboardRoutes_keepsDifferentHostRoamingDashboard() {
|
||||
val stored = Connection.buildRouteCandidates(
|
||||
apiServerUrl = "http://100.71.8.56:8642",
|
||||
relayUrl = "ws://100.71.8.56:8767",
|
||||
)
|
||||
|
||||
val repaired = Connection.reconcileDashboardRoutes(
|
||||
dashboardUrl = "https://hermes.example.com:443",
|
||||
candidates = stored,
|
||||
)
|
||||
|
||||
assertEquals("http://100.71.8.56:9119", repaired.single().dashboard?.url)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun persistedSecureDashboard_repairsDerivedGatewayRouteOnReload() {
|
||||
val stored = Connection(
|
||||
id = "conn-https",
|
||||
label = "Secure Hermes",
|
||||
apiServerUrl = "https://hermes.example.com:8643",
|
||||
relayUrl = "wss://hermes.example.com:8767",
|
||||
tokenStoreKey = "hermes_auth_https",
|
||||
dashboardUrl = "https://hermes.example.com:443",
|
||||
routeCandidates = Connection.buildRouteCandidates(
|
||||
apiServerUrl = "https://hermes.example.com:8643",
|
||||
relayUrl = "wss://hermes.example.com:8767",
|
||||
),
|
||||
)
|
||||
|
||||
val reloaded = json.decodeFromString<Connection>(
|
||||
json.encodeToString(Connection.serializer(), stored),
|
||||
).withDashboardDefaults()
|
||||
|
||||
assertEquals("https://hermes.example.com:443", reloaded.dashboardUrl)
|
||||
assertEquals(
|
||||
"https://hermes.example.com:443",
|
||||
reloaded.routeCandidates.single().dashboard?.url,
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun dashboardRouteBuilder_acceptsBareTailscaleHostWithoutOptionalSurfaces() {
|
||||
val route = Connection.endpointCandidateFromDashboardUrl(
|
||||
|
||||
@@ -11,6 +11,8 @@ import kotlinx.coroutines.flow.first
|
||||
import kotlinx.coroutines.test.runTest
|
||||
import org.junit.After
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Before
|
||||
import org.junit.Rule
|
||||
import org.junit.Test
|
||||
@@ -65,4 +67,44 @@ class VoicePreferencesRepositoryTest {
|
||||
assertEquals("grok-voice-think-fast-1.0", settings.realtimeModel)
|
||||
assertEquals("leo", settings.realtimeVoice)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun finalAnswerOnlyPersistsGloballyAcrossProfileScopes() = runTest {
|
||||
assertFalse(repository.settings.first().finalAnswerOnly)
|
||||
|
||||
repository.setFinalAnswerOnly(true)
|
||||
assertTrue(repository.settings.first().finalAnswerOnly)
|
||||
|
||||
repository.setActiveScope("connection-a", "coder")
|
||||
assertTrue(repository.settings.first().finalAnswerOnly)
|
||||
|
||||
repository.setActiveScope("connection-b", "writer")
|
||||
assertTrue(repository.settings.first().finalAnswerOnly)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun presentationModePersistsGloballyAcrossProfileScopes() = runTest {
|
||||
assertEquals(
|
||||
VoicePresentationMode.Focus.storageValue,
|
||||
repository.settings.first().presentationMode,
|
||||
)
|
||||
|
||||
repository.setPresentationMode(VoicePresentationMode.Conversation)
|
||||
assertEquals(
|
||||
VoicePresentationMode.Conversation.storageValue,
|
||||
repository.settings.first().presentationMode,
|
||||
)
|
||||
|
||||
repository.setActiveScope("connection-a", "coder")
|
||||
assertEquals(
|
||||
VoicePresentationMode.Conversation.storageValue,
|
||||
repository.settings.first().presentationMode,
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun unknownPresentationModeFallsBackToFocus() {
|
||||
assertEquals(VoicePresentationMode.Focus, VoicePresentationMode.fromStorage("unknown"))
|
||||
assertEquals(VoicePresentationMode.Focus, VoicePresentationMode.fromStorage(null))
|
||||
}
|
||||
}
|
||||
|
||||
+97
@@ -147,6 +147,28 @@ class RelayVoiceClientRoutingTest {
|
||||
assertEquals("leo", payload["voice"]?.jsonPrimitive?.content)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun realtimeAgentSessionSendsFinalAnswerOnlyPolicy() = runTest {
|
||||
val client = RelayVoiceClient(
|
||||
context = context,
|
||||
okHttpClient = httpClient,
|
||||
relayUrlProvider = { relayUrl(lanServer) },
|
||||
sessionTokenProvider = { "session-token" },
|
||||
)
|
||||
|
||||
val result = client.runRealtimeAgent(
|
||||
prompt = "Check Hermes quietly",
|
||||
inputPcm = ByteArray(0),
|
||||
finalAnswerOnly = true,
|
||||
) { _, _ -> }
|
||||
|
||||
assertTrue(result.exceptionOrNull()?.message, result.isSuccess)
|
||||
val request = lanServer.takeRequest(2, TimeUnit.SECONDS)
|
||||
?: error("missing realtime session request")
|
||||
val payload = Json.parseToJsonElement(request.body.readUtf8()).jsonObject
|
||||
assertEquals("true", payload["final_answer_only"]?.jsonPrimitive?.content)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun voiceOutputSessionResponseParsesResumeMetadata() {
|
||||
val response = Json.decodeFromString(
|
||||
@@ -2102,6 +2124,81 @@ class RelayVoiceClientRoutingTest {
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun persistentRealtimePromotionUsesSpokenHandoffAsForegroundBoundary() = runBlocking {
|
||||
val opened = CountDownLatch(1)
|
||||
val turns = Channel<RealtimeTurnInput>(Channel.UNLIMITED)
|
||||
val turnCompletions = Channel<Unit>(Channel.UNLIMITED)
|
||||
val followUpDelivered = CompletableDeferred<Result<Unit>>()
|
||||
lateinit var socket: ScriptedWebSocket
|
||||
lateinit var listener: WebSocketListener
|
||||
lanServer.dispatcher = sessionOnlyDispatcher(
|
||||
path = "/voice/realtime-agent/session",
|
||||
body = """
|
||||
{
|
||||
"success": true,
|
||||
"session_id": "realtime-agent-promotion-boundary-test",
|
||||
"websocket_path": "/voice/realtime-agent/session-test",
|
||||
"provider": "xai_realtime",
|
||||
"model": "grok-voice-latest",
|
||||
"voice": "leo",
|
||||
"sample_rate": 24000
|
||||
}
|
||||
""".trimIndent(),
|
||||
)
|
||||
val client = RelayVoiceClient(
|
||||
context = context,
|
||||
okHttpClient = httpClient,
|
||||
relayUrlProvider = { relayUrl(lanServer) },
|
||||
sessionTokenProvider = { "session-token" },
|
||||
webSocketFactory = { request, callback ->
|
||||
listener = callback
|
||||
socket = ScriptedWebSocket(request, callback) { true }
|
||||
callback.onOpen(socket, mockk(relaxed = true))
|
||||
opened.countDown()
|
||||
socket
|
||||
},
|
||||
)
|
||||
val sessionJob = async(Dispatchers.IO) {
|
||||
client.runRealtimeAgent(
|
||||
prompt = "Start a long task",
|
||||
inputPcm = ByteArray(0),
|
||||
turnInputs = turns,
|
||||
onTurnComplete = { turnCompletions.trySend(Unit) },
|
||||
) { _, _ -> }
|
||||
}
|
||||
|
||||
try {
|
||||
assertTrue(opened.await(2, TimeUnit.SECONDS))
|
||||
listener.onMessage(
|
||||
socket,
|
||||
"""{"type":"hermes.run.promoted","run_id":"run-silent","spoken_handoff":false}""",
|
||||
)
|
||||
withTimeout(2_000) { turnCompletions.receive() }
|
||||
|
||||
turns.send(
|
||||
RealtimeTurnInput(
|
||||
inputPcm = ByteArray(6_400) { 4 },
|
||||
deliveryResult = followUpDelivered,
|
||||
)
|
||||
)
|
||||
assertTrue(withTimeout(2_000) { followUpDelivered.await() }.isSuccess)
|
||||
|
||||
listener.onMessage(
|
||||
socket,
|
||||
"""{"type":"hermes.run.promoted","run_id":"run-spoken","spoken_handoff":true}""",
|
||||
)
|
||||
delay(100)
|
||||
assertTrue(turnCompletions.tryReceive().isFailure)
|
||||
|
||||
listener.onMessage(socket, """{"type":"voice.response.done"}""")
|
||||
withTimeout(2_000) { turnCompletions.receive() }
|
||||
} finally {
|
||||
turns.close()
|
||||
withTimeout(2_000) { sessionJob.await() }
|
||||
}
|
||||
}
|
||||
|
||||
private fun relayUrl(server: MockWebServer): String =
|
||||
"ws://${server.hostName}:${server.port}"
|
||||
|
||||
|
||||
@@ -1517,6 +1517,49 @@ class ChatHandlerTest {
|
||||
assertEquals(messages.size, messages.map { it.uiKey }.distinct().size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun loadMessageHistory_coalescesReplayedDomainIdWithoutLosingOrderOrContent() {
|
||||
val replayedId = "2c93af28-0b0b-436b-a112-7f164cac931d"
|
||||
|
||||
handler.loadMessageHistory(
|
||||
listOf(
|
||||
MessageItem(
|
||||
id = "user-1",
|
||||
role = "user",
|
||||
content = JsonPrimitive("question"),
|
||||
timestamp = 1.0,
|
||||
),
|
||||
MessageItem(
|
||||
id = replayedId,
|
||||
role = "assistant",
|
||||
content = JsonPrimitive("partial answer"),
|
||||
timestamp = 2.0,
|
||||
),
|
||||
MessageItem(
|
||||
id = "system-1",
|
||||
role = "system",
|
||||
content = JsonPrimitive("distinct visible content"),
|
||||
timestamp = 3.0,
|
||||
),
|
||||
// Rejoin replay of the same persisted message. The latest
|
||||
// snapshot is authoritative, but its first transcript position
|
||||
// and Compose identity must remain stable.
|
||||
MessageItem(
|
||||
id = replayedId,
|
||||
role = "assistant",
|
||||
content = JsonPrimitive("final answer"),
|
||||
timestamp = 4.0,
|
||||
),
|
||||
)
|
||||
)
|
||||
|
||||
val messages = handler.messages.value
|
||||
assertEquals(listOf("user-1", replayedId, "system-1"), messages.map { it.id })
|
||||
assertEquals("final answer", messages[1].content)
|
||||
assertEquals("distinct visible content", messages[2].content)
|
||||
assertEquals(messages.size, messages.map { it.uiKey }.distinct().size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun loadMessageHistory_secondReloadMatchesByIdAfterReconciliation() {
|
||||
// Once the first reload adopts the server id, subsequent reloads match by
|
||||
|
||||
+28
-1
@@ -68,10 +68,36 @@ class NativeDashboardAuthTest {
|
||||
assertEquals("http://127.0.0.1:43123/callback", query["redirect_uri"])
|
||||
assertEquals("nous", query["provider"])
|
||||
assertTrue(query.getValue("state").length >= 32)
|
||||
assertTrue(query.getValue("code_challenge").length >= 43)
|
||||
assertEquals(43, query.getValue("code_challenge").length)
|
||||
assertFalse(query.getValue("code_challenge").contains('='))
|
||||
assertNotEquals(query["state"], query["code_challenge"])
|
||||
}
|
||||
|
||||
@Test
|
||||
fun canonicalNousCallbackBase_usesSecurePublicOriginAndPreservesPrefix() {
|
||||
val location = "https://portal.nousresearch.com/oauth/authorize" +
|
||||
"?redirect_uri=https%3A%2F%2Fhermes.example.test%2Fgateway%2Fauth%2Fcallback"
|
||||
|
||||
assertEquals(
|
||||
"https://hermes.example.test/gateway",
|
||||
canonicalDashboardBaseFromNousRedirect(location),
|
||||
)
|
||||
assertEquals(
|
||||
null,
|
||||
canonicalDashboardBaseFromNousRedirect(
|
||||
"https://portal.nousresearch.com/oauth/authorize" +
|
||||
"?redirect_uri=http%3A%2F%2Fhermes.example.test%2Fauth%2Fcallback",
|
||||
),
|
||||
)
|
||||
assertEquals(
|
||||
null,
|
||||
canonicalDashboardBaseFromNousRedirect(
|
||||
"https://attacker.example/oauth/authorize" +
|
||||
"?redirect_uri=https%3A%2F%2Fhermes.example.test%2Fauth%2Fcallback",
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
@Test(expected = IllegalArgumentException::class)
|
||||
fun beginAuthorization_rejectsHostnameLoopback() {
|
||||
NativeDashboardAuthClient(server.url("/").toString(), store)
|
||||
@@ -103,6 +129,7 @@ class NativeDashboardAuthTest {
|
||||
.digest(verifier.toByteArray(Charsets.US_ASCII))
|
||||
.toByteString()
|
||||
.base64Url()
|
||||
.trimEnd('=')
|
||||
val authorizeChallenge = java.net.URI(authorization.authorizationUrl).rawQuery
|
||||
.split("&")
|
||||
.first { it.startsWith("code_challenge=") }
|
||||
|
||||
+21
@@ -115,7 +115,28 @@ class NativeDashboardSignInCoordinatorTest {
|
||||
)
|
||||
assertTrue(isNativeDashboardTransportEligible("https://hermes.example.test/prefix"))
|
||||
assertTrue(isNativeDashboardTransportEligible("http://127.0.0.1:9119"))
|
||||
assertTrue(isNativeDashboardTransportEligible("http://172.16.24.250:9119"))
|
||||
assertTrue(isNativeDashboardTransportEligible("http://100.71.8.56:9119"))
|
||||
assertFalse(isNativeDashboardTransportEligible("http://hermes.local:9119"))
|
||||
assertFalse(isNativeDashboardTransportEligible("http://203.0.113.10:9119"))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun androidRedirectMode_usesBrowserForNous_andCookieFlowForSelfHostedOidc() {
|
||||
val flows = listOf("cookie", "native_pkce")
|
||||
|
||||
assertEquals(
|
||||
DashboardRedirectAuthMode.NativePkce,
|
||||
androidDashboardRedirectAuthMode("nous", flows),
|
||||
)
|
||||
assertEquals(
|
||||
DashboardRedirectAuthMode.WebView,
|
||||
androidDashboardRedirectAuthMode("oidc", flows),
|
||||
)
|
||||
assertEquals(
|
||||
DashboardRedirectAuthMode.WebView,
|
||||
androidDashboardRedirectAuthMode("nous", listOf("cookie")),
|
||||
)
|
||||
}
|
||||
|
||||
private suspend fun completeSignIn(
|
||||
|
||||
+5
-7
@@ -12,7 +12,7 @@ import com.hermesandroid.relay.viewmodel.VoiceUiState
|
||||
import com.hermesandroid.relay.viewmodel.backgroundRunAfterCancelRequest
|
||||
import com.hermesandroid.relay.viewmodel.preserveRealtimeTurnOnStop
|
||||
import com.hermesandroid.relay.viewmodel.realtimeTranscriptState
|
||||
import com.hermesandroid.relay.viewmodel.realtimeTurnActiveAfterResponseDone
|
||||
import com.hermesandroid.relay.viewmodel.realtimeTurnActiveAfterPromotion
|
||||
import com.hermesandroid.relay.viewmodel.voiceSessionExitState
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertNull
|
||||
@@ -27,12 +27,10 @@ class VoiceModeOverlayStateTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
fun responseDone_keepsLogicalTurnActiveOnlyWhileBackgroundRunIsLive() {
|
||||
assertEquals(true, realtimeTurnActiveAfterResponseDone(BackgroundRunPhase.RUNNING))
|
||||
assertEquals(true, realtimeTurnActiveAfterResponseDone(BackgroundRunPhase.RECONNECTING))
|
||||
assertEquals(false, realtimeTurnActiveAfterResponseDone(BackgroundRunPhase.DELIVERING))
|
||||
assertEquals(false, realtimeTurnActiveAfterResponseDone(BackgroundRunPhase.DONE))
|
||||
assertEquals(false, realtimeTurnActiveAfterResponseDone(null))
|
||||
fun promotion_keepsForegroundBusyOnlyForSpokenHandoff() {
|
||||
assertEquals(false, realtimeTurnActiveAfterPromotion(spokenHandoff = false))
|
||||
assertEquals(true, realtimeTurnActiveAfterPromotion(spokenHandoff = true))
|
||||
assertEquals(true, realtimeTurnActiveAfterPromotion(spokenHandoff = null))
|
||||
}
|
||||
|
||||
@Test
|
||||
|
||||
@@ -1,11 +1,43 @@
|
||||
package com.hermesandroid.relay.ui.screens
|
||||
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Assert.assertNotEquals
|
||||
import org.junit.Assert.assertNull
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Test
|
||||
|
||||
class ChatScrollSnapshotTest {
|
||||
@Test
|
||||
fun `completion releases only the retained live tail`() {
|
||||
assertNull(releaseRetainedLiveTail("assistant-live", "assistant-live"))
|
||||
assertEquals(
|
||||
"assistant-live",
|
||||
releaseRetainedLiveTail("assistant-live", "different-message"),
|
||||
)
|
||||
assertNull(releaseRetainedLiveTail(null, "assistant-live"))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `tall markdown tail is positioned by its trailing edge`() {
|
||||
assertEquals(
|
||||
1_208,
|
||||
tailEndScrollOffset(
|
||||
tailSizePx = 2_400,
|
||||
footerSizePx = 8,
|
||||
viewportSizePx = 1_200,
|
||||
),
|
||||
)
|
||||
assertEquals(
|
||||
0,
|
||||
tailEndScrollOffset(
|
||||
tailSizePx = 600,
|
||||
footerSizePx = 8,
|
||||
viewportSizePx = 1_200,
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `same-tail stream completion requests an atomic bottom anchor`() {
|
||||
val streaming = snapshot(isStreaming = true)
|
||||
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
package com.hermesandroid.relay.ui.screens
|
||||
|
||||
import com.hermesandroid.relay.network.upstream.DashboardApiClient
|
||||
import com.hermesandroid.relay.network.upstream.DashboardCookieJar
|
||||
import com.hermesandroid.relay.network.upstream.InMemoryDashboardCookieStore
|
||||
import com.hermesandroid.relay.network.upstream.importDashboardCookieHeader
|
||||
import kotlinx.coroutines.test.runTest
|
||||
import okhttp3.OkHttpClient
|
||||
import okhttp3.mockwebserver.MockResponse
|
||||
import okhttp3.mockwebserver.MockWebServer
|
||||
import org.junit.After
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Before
|
||||
import org.junit.Test
|
||||
|
||||
class DashboardWebViewAuthPolicyTest {
|
||||
private lateinit var server: MockWebServer
|
||||
|
||||
@Before
|
||||
fun setUp() {
|
||||
server = MockWebServer()
|
||||
server.start()
|
||||
}
|
||||
|
||||
@After
|
||||
fun tearDown() {
|
||||
server.shutdown()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun selfHostedOidc_usesDashboardLoginWithoutNativeOrLoopbackParameters() {
|
||||
val url = DashboardApiClient.authLoginUrl(
|
||||
baseUrl = "https://hermes.example.test",
|
||||
provider = "self-hosted",
|
||||
next = "/",
|
||||
)
|
||||
|
||||
assertEquals(
|
||||
"https://hermes.example.test/auth/login?provider=self-hosted&next=%2F",
|
||||
url,
|
||||
)
|
||||
assertFalse(url.contains("/auth/native/authorize"))
|
||||
assertFalse(url.contains("redirect_uri"))
|
||||
assertFalse(url.contains("127.0.0.1"))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun publicDashboardCallback_importsCookieAndVerifiesAuthenticatedSession() = runTest {
|
||||
assertEquals(
|
||||
DashboardWebViewAuthNavigation.ImportAndVerify,
|
||||
dashboardWebViewAuthNavigation(
|
||||
"https://hermes.example.test",
|
||||
"https://hermes.example.test/auth/callback?code=public-code&state=public-state",
|
||||
),
|
||||
)
|
||||
|
||||
server.enqueue(
|
||||
MockResponse()
|
||||
.setHeader("Content-Type", "application/json")
|
||||
.setBody(
|
||||
"""{"authenticated":true,"username":"operator","provider":"self-hosted"}""",
|
||||
),
|
||||
)
|
||||
val store = InMemoryDashboardCookieStore()
|
||||
val callbackUrl = server.url("/auth/callback?code=public-code").toString()
|
||||
assertEquals(
|
||||
1,
|
||||
importDashboardCookieHeader(
|
||||
store = store,
|
||||
url = callbackUrl,
|
||||
cookieHeader = "hermes_session=authenticated",
|
||||
),
|
||||
)
|
||||
val client = DashboardApiClient(
|
||||
baseUrl = server.url("/").toString(),
|
||||
okHttpClient = OkHttpClient.Builder()
|
||||
.cookieJar(DashboardCookieJar(store))
|
||||
.build(),
|
||||
)
|
||||
|
||||
val session = client.currentSession().getOrThrow()
|
||||
|
||||
assertTrue(session.authenticated)
|
||||
val request = server.takeRequest()
|
||||
assertEquals("/api/auth/me", request.path)
|
||||
assertEquals("hermes_session=authenticated", request.getHeader("Cookie"))
|
||||
client.shutdown()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun foreignLoopbackCallbacksAreRejectedWhileProviderPagesContinue() {
|
||||
val dashboard = "https://hermes.example.test"
|
||||
listOf(
|
||||
"http://127.0.0.1:40179/callback?code=code",
|
||||
"http://localhost:40179/callback?code=code",
|
||||
"http://[::1]:40179/callback?code=code",
|
||||
).forEach { callback ->
|
||||
assertEquals(
|
||||
callback,
|
||||
DashboardWebViewAuthNavigation.RejectLoopbackCallback,
|
||||
dashboardWebViewAuthNavigation(dashboard, callback),
|
||||
)
|
||||
}
|
||||
assertEquals(
|
||||
DashboardWebViewAuthNavigation.Continue,
|
||||
dashboardWebViewAuthNavigation(
|
||||
dashboard,
|
||||
"https://auth.example.test/application/o/authorize/",
|
||||
),
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import java.io.IOException
|
||||
import java.net.ConnectException
|
||||
import java.net.SocketTimeoutException
|
||||
import java.net.UnknownHostException
|
||||
import javax.net.ssl.SSLException
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Assert.assertTrue
|
||||
@@ -60,6 +61,15 @@ class RelayErrorClassifierTest {
|
||||
|
||||
assertEquals("Session expired", err.title)
|
||||
assertTrue(err.body.contains("re-pair", ignoreCase = true))
|
||||
assertEquals(HumanErrorAction.Repair, err.action)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun certificateMismatchExposesRepairAction() {
|
||||
val err = classifyError(SSLException("certificate changed"))
|
||||
|
||||
assertEquals("Certificate mismatch", err.title)
|
||||
assertEquals(HumanErrorAction.Repair, err.action)
|
||||
}
|
||||
|
||||
@Test
|
||||
|
||||
@@ -33,6 +33,19 @@ class AssistantSpeechCursorTest {
|
||||
assertEquals(listOf("The check is complete."), second.deltas.map { it.text })
|
||||
assertTrue(second.deltas.single().startsNewBubble)
|
||||
assertEquals("I'll check that.\n\nThe check is complete.", second.aggregateText)
|
||||
assertEquals("The check is complete.", second.finalAnswerText)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `final answer skips blank tool bubbles and intermediate commentary`() {
|
||||
val cursor = AssistantSpeechCursor(emptyList())
|
||||
val interim = message("interim", MessageRole.ASSISTANT, "I'll check that.")
|
||||
val toolOnly = message("tool", MessageRole.ASSISTANT, " ")
|
||||
val final = message("final", MessageRole.ASSISTANT, " The settled answer. ")
|
||||
|
||||
val batch = cursor.poll(listOf(interim, toolOnly, final))
|
||||
|
||||
assertEquals("The settled answer.", batch.finalAnswerText)
|
||||
}
|
||||
|
||||
@Test
|
||||
|
||||
+68
@@ -240,6 +240,74 @@ class ChatViewModelRealtimeTurnTest {
|
||||
assertEquals(2, handler.messages.value.size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun silentBackgroundPromotionReleasesForegroundStreamButKeepsTaskOwnership() {
|
||||
val assistantId = viewModel.startRealtimeAgentTurn(
|
||||
userText = "Check release readiness",
|
||||
chatSessionId = "session-1",
|
||||
)
|
||||
|
||||
viewModel.applyRealtimeAgentEvent(
|
||||
assistantMessageId = assistantId,
|
||||
event = RealtimeVoiceEvent(
|
||||
type = "hermes.run.promoted",
|
||||
runId = "run-silent",
|
||||
tier = "durable",
|
||||
spokenHandoff = false,
|
||||
raw = "{}",
|
||||
),
|
||||
)
|
||||
|
||||
val promoted = handler.messages.value.single { it.id == assistantId }
|
||||
assertFalse(handler.isStreaming.value)
|
||||
assertFalse(promoted.isStreaming)
|
||||
assertEquals(BackgroundTaskPhase.RUNNING, promoted.backgroundTask?.phase)
|
||||
|
||||
viewModel.applyRealtimeAgentEvent(
|
||||
assistantMessageId = "newer-turn",
|
||||
event = RealtimeVoiceEvent(
|
||||
type = "hermes.run.progress",
|
||||
runId = "run-silent",
|
||||
message = "Checking Android",
|
||||
raw = "{}",
|
||||
),
|
||||
)
|
||||
|
||||
val updated = handler.messages.value.single { it.id == assistantId }
|
||||
assertEquals("Checking Android", updated.backgroundTask?.statusLine)
|
||||
assertEquals(BackgroundTaskPhase.RUNNING, updated.backgroundTask?.phase)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun spokenBackgroundPromotionWaitsForProviderResponseBoundary() {
|
||||
val assistantId = viewModel.startRealtimeAgentTurn(
|
||||
userText = "Check release readiness",
|
||||
chatSessionId = "session-1",
|
||||
)
|
||||
|
||||
viewModel.applyRealtimeAgentEvent(
|
||||
assistantMessageId = assistantId,
|
||||
event = RealtimeVoiceEvent(
|
||||
type = "hermes.run.promoted",
|
||||
runId = "run-spoken",
|
||||
spokenHandoff = true,
|
||||
raw = "{}",
|
||||
),
|
||||
)
|
||||
assertTrue(handler.isStreaming.value)
|
||||
|
||||
viewModel.applyRealtimeAgentEvent(
|
||||
assistantMessageId = assistantId,
|
||||
event = RealtimeVoiceEvent(type = "voice.response.done", raw = "{}"),
|
||||
)
|
||||
|
||||
assertFalse(handler.isStreaming.value)
|
||||
assertEquals(
|
||||
BackgroundTaskPhase.RUNNING,
|
||||
handler.messages.value.single { it.id == assistantId }.backgroundTask?.phase,
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun backgroundRunKeepsItsOwnerAfterANewerLocalVoiceCommand() {
|
||||
val backgroundAssistantId = viewModel.startRealtimeAgentTurn(
|
||||
|
||||
+19
-1
@@ -54,7 +54,25 @@ class EffectiveDashboardRouteTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `selected API-only route derives dashboard even when primary dashboard is explicit`() {
|
||||
fun `selected API-only route keeps explicit same-host secure dashboard`() {
|
||||
val connection = connection(
|
||||
dashboardUrl = "https://hermes.example.com:443",
|
||||
apiServerUrl = "https://hermes.example.com:8643",
|
||||
)
|
||||
val fallback = EndpointCandidate(
|
||||
role = "public",
|
||||
priority = 1,
|
||||
api = ApiEndpoint("hermes.example.com", 8643, tls = true),
|
||||
)
|
||||
|
||||
assertEquals(
|
||||
"https://hermes.example.com:443",
|
||||
resolveEffectiveDashboardUrl(connection, fallback),
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `selected API-only route derives dashboard for a different route host`() {
|
||||
val connection = connection(
|
||||
dashboardUrl = "http://192.168.1.20:9119",
|
||||
apiServerUrl = "http://192.168.1.20:8642",
|
||||
|
||||
@@ -42,6 +42,15 @@ class VoiceTurnSessionFenceTest {
|
||||
assertFalse(fence.accepts(sessionId = "other", messages = emptyList()))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `transient mismatch does not lose existing session binding`() {
|
||||
val fence = VoiceTurnSessionFence(initialSessionId = "active")
|
||||
fence.bindSubmittedUser("voice-user")
|
||||
|
||||
assertFalse(fence.accepts(sessionId = "other", messages = emptyList()))
|
||||
assertTrue(fence.accepts(sessionId = "active", messages = emptyList()))
|
||||
}
|
||||
|
||||
private fun user(uiKey: String) = ChatMessage(
|
||||
id = "id-$uiKey",
|
||||
role = MessageRole.USER,
|
||||
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
plugins {
|
||||
id("com.android.application") version "9.3.0" apply false
|
||||
id("com.android.library") version "9.3.0" apply false
|
||||
id("com.android.application") version "9.3.1" apply false
|
||||
id("com.android.library") version "9.3.1" apply false
|
||||
id("org.jetbrains.kotlin.plugin.compose") version "2.4.10" apply false
|
||||
id("org.jetbrains.kotlin.plugin.serialization") version "2.4.10" apply false
|
||||
}
|
||||
|
||||
+57
-2
@@ -1716,7 +1716,10 @@ override:
|
||||
**Protocol additions (relay <-> Android, additive).**
|
||||
|
||||
- `hermes.run.promoted` - run moved to background; carries `run_id`,
|
||||
`promote_after_ms`, `spoken_handoff`.
|
||||
`promote_after_ms`, `spoken_handoff`. A false `spoken_handoff` is also the
|
||||
foreground turn boundary; when true, the following `voice.response.done` is
|
||||
the boundary. In both cases the background socket, task card, cancellation,
|
||||
progress, and result delivery remain active.
|
||||
- `hermes.run.background_completed` - background run finished; precedes the
|
||||
provider/relay summary.
|
||||
- Extend `hermes.run.progress` with `tier` and `floor` so the client can render
|
||||
@@ -2166,7 +2169,7 @@ An API endpoint or Relay can be added later without recreating the connection.
|
||||
|
||||
## ADR 39 — Android dashboard redirect auth uses native PKCE
|
||||
|
||||
**Status:** Accepted (2026-07-25).
|
||||
**Status:** Superseded by ADR 40 (2026-07-27).
|
||||
|
||||
**Context.** Android originally completed redirect-provider dashboard sign-in
|
||||
inside a WebView and imported cookies. Current upstream Gateway can advertise a
|
||||
@@ -2199,3 +2202,55 @@ socket.
|
||||
is offered.
|
||||
- Older upstream versions remain usable through the explicitly identified
|
||||
WebView compatibility path.
|
||||
|
||||
---
|
||||
|
||||
## ADR 40 — Android dashboard redirect auth is provider-compatible
|
||||
|
||||
**Status:** Amended (2026-07-28).
|
||||
|
||||
**Context.** Upstream advertises `native_pkce` in `/api/status.auth_flows` for
|
||||
its desktop client. The corresponding `/auth/native/*` broker is explicitly a
|
||||
desktop system-browser flow: it redirects to a loopback listener owned by the
|
||||
desktop process and returns bearer tokens rather than dashboard cookies.
|
||||
Android incorrectly treated that server-wide capability as a platform-neutral
|
||||
mode selector, so redirect providers such as self-hosted OIDC were sent through
|
||||
the desktop loopback contract.
|
||||
|
||||
**Decision.** Android redirect-provider sign-in uses the upstream dashboard
|
||||
cookie flow by default:
|
||||
|
||||
- open `/auth/login?provider=...&next=...` in a full-screen embedded sign-in
|
||||
destination with a normal app bar rather than a modal WebView;
|
||||
- allow the provider to return through the dashboard's public
|
||||
`/auth/callback`;
|
||||
- import only cookies observed on the configured dashboard origin;
|
||||
- verify the imported session through `/api/auth/me`;
|
||||
- reject a foreign `http://127.0.0.1`, `localhost`, or `[::1]` `/callback`
|
||||
navigation instead of following or importing it.
|
||||
|
||||
Android does not select `/auth/native/authorize` merely because it appears in
|
||||
`auth_flows`. Self-hosted OIDC remains on the cookie contract above. Nous Portal
|
||||
is the narrow exception: its Cloudflare Turnstile challenge rejects embedded
|
||||
Android WebViews, so Android uses the gateway-brokered native PKCE route for
|
||||
that provider when advertised and opens it in a system Custom Tab. The
|
||||
ephemeral loopback listener, S256 verifier, state validation, encrypted bearer
|
||||
store, and exact-origin attachment remain app-owned. Public cleartext
|
||||
dashboards are rejected; explicitly configured RFC 1918 and Tailscale-IP
|
||||
dashboard routes retain the same HTTP allowance as their existing cookie
|
||||
sessions. If the provider redirect from a private route declares a canonical
|
||||
HTTPS dashboard callback, Android begins browser authorization on that
|
||||
canonical origin so the temporary PKCE cookie and callback remain same-origin;
|
||||
the one-time code exchange and resulting exact-origin bearer stay bound to the
|
||||
active private route.
|
||||
|
||||
**Consequences.**
|
||||
|
||||
- Self-hosted OIDC uses the same public callback registered for the dashboard.
|
||||
- Android Manage, Chat, Voice, and onboarding continue to share one verified
|
||||
dashboard cookie session.
|
||||
- A server-wide desktop capability can no longer switch Android into a
|
||||
loopback callback flow.
|
||||
- Android retains a full-screen embedded WebView for compatible dashboard
|
||||
cookie providers, while providers that prohibit embedding use the explicit
|
||||
brokered native route.
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
"verification": "ai-translated",
|
||||
"review_refs": [],
|
||||
"source_sha256": {
|
||||
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
|
||||
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
|
||||
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
|
||||
},
|
||||
"surfaces": {
|
||||
@@ -48,7 +48,7 @@
|
||||
"verification": "ai-translated",
|
||||
"review_refs": [],
|
||||
"source_sha256": {
|
||||
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
|
||||
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
|
||||
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
|
||||
},
|
||||
"surfaces": {
|
||||
@@ -72,7 +72,7 @@
|
||||
"verification": "ai-translated",
|
||||
"review_refs": [],
|
||||
"source_sha256": {
|
||||
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
|
||||
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
|
||||
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
|
||||
},
|
||||
"surfaces": {
|
||||
@@ -96,7 +96,7 @@
|
||||
"verification": "ai-translated",
|
||||
"review_refs": [],
|
||||
"source_sha256": {
|
||||
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
|
||||
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
|
||||
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
|
||||
},
|
||||
"surfaces": {
|
||||
@@ -120,7 +120,7 @@
|
||||
"verification": "ai-translated",
|
||||
"review_refs": [],
|
||||
"source_sha256": {
|
||||
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
|
||||
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
|
||||
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
|
||||
},
|
||||
"surfaces": {
|
||||
|
||||
@@ -85,12 +85,12 @@ This app is a community project and is not affiliated with or endorsed by NousRe
|
||||
Paste into Play Console → **What's new** (≤500 characters):
|
||||
|
||||
```
|
||||
v1.5.0 - Hermes, always in reach
|
||||
v1.5.2 - Sign in without detours
|
||||
|
||||
* Secure Dashboard sign-in and a clearer Agent Passport.
|
||||
* Active background chats with actionable approval and question alerts.
|
||||
* Better attachments, image generation, voice, routing, and recovery.
|
||||
* Optional permission guidance that never blocks chat.
|
||||
* Reliable self-hosted OIDC and Nous Portal sign-in.
|
||||
* Secure system-browser flow for Nous provider challenges.
|
||||
* Private-LAN and Tailscale dashboard route support.
|
||||
* Replayed chat updates no longer duplicate rows.
|
||||
```
|
||||
|
||||
## Category
|
||||
|
||||
+18
-13
@@ -170,16 +170,21 @@ Phone control — mirrors upstream relay protocol.
|
||||
|
||||
### 3.3 Auth Flow
|
||||
|
||||
Dashboard/Gateway redirect providers use the upstream native PKCE contract when
|
||||
`GET /api/status` advertises `native_pkce`. Android opens the selected provider
|
||||
in a Custom Tab and owns a single ephemeral callback on
|
||||
`http://127.0.0.1:<os-assigned-port>/callback`. PKCE verifier and CSRF state
|
||||
exist only for that sign-in coroutine. Access and refresh tokens are encrypted
|
||||
per connection and are attached only to the exact trusted dashboard base for
|
||||
Manage, Gateway tickets, and standard voice. Native exchange is allowed only
|
||||
for HTTPS dashboard addresses (plus literal loopback for development). A
|
||||
gateway without the capability uses the legacy cookie/WebView flow; a failed
|
||||
native attempt never silently downgrades.
|
||||
Dashboard/Gateway redirect authentication is provider-compatible. Nous Portal,
|
||||
which relies on a challenge that rejects embedded Android WebViews, uses the
|
||||
upstream brokered `native_pkce` flow in a system Custom Tab when the dashboard
|
||||
advertises it. The app owns an ephemeral loopback callback and stores the
|
||||
resulting bearer session only for that connection and exact dashboard origin.
|
||||
Self-hosted OIDC remains on the dashboard cookie flow: Android opens
|
||||
`/auth/login` in a full-screen embedded browser destination, lets the provider
|
||||
return through the public `/auth/callback`, imports only same-origin cookies,
|
||||
and verifies them through `/api/auth/me`. HTTPS is required on public routes;
|
||||
explicit private-LAN and Tailscale-IP dashboards may use their existing HTTP
|
||||
transport. When such a private route advertises a canonical HTTPS Nous callback,
|
||||
Android starts the browser on that canonical origin so Hermes' temporary PKCE
|
||||
cookie and the provider callback remain same-origin, then exchanges the
|
||||
one-time code through the active private route. The verified session is shared
|
||||
by Manage, Gateway tickets, and standard voice.
|
||||
|
||||
Pairing is QR-driven. The operator runs the pair command on the host — `hermes pair`, `/hermes-relay-pair` from any Hermes chat surface, or the compatibility `hermes-pair` shell shim. All share the same implementation in `plugin/pair.py`. The command probes for a running relay, generates a fresh 6-char code, pre-registers it with the relay via the loopback-only `POST /pairing/register` endpoint, then embeds the relay URL + code + **chosen TTL + per-channel grants + HMAC signature** (plus the API server credentials and optional dashboard URL) in a single QR payload. The phone scans once, **confirms the TTL and grants via a picker dialog**, and is configured for both chat AND terminal/bridge.
|
||||
|
||||
@@ -419,7 +424,7 @@ The bridge UI drives — and is driven by — Tier 5 safety-rails (`BridgeSafety
|
||||
- **Connections** (v0.6.0+) — lists every paired Hermes server with a per-card status chip. Actions: rename (inline), re-pair (reuses `ConnectionWizard` with `connectionId` nav arg), revoke, remove. Add-connection button launches the standard QR flow. Settings briefly treats a paired + disconnected relay as **Connecting** during the reconnect grace window, then promotes it to **Relay unreachable - tap to reconnect** if the live socket does not recover. API / Relay / Session detail sheets include compact sanitized recent-activity tails, and **Settings -> Diagnostics** shows the consolidated app-level API, relay, session, endpoint, and voice activity buffer. See `docs/decisions.md` §19.
|
||||
- **Connection (single-server settings)** — summary-first detail for one Hermes installation. Dashboard/Gateway health drives standard Chat, Manage, Sessions, and Voice readiness. API fallback and Relay extensions appear as independently optional capabilities. Advanced configuration exposes manual Dashboard, API, and Relay endpoints plus their native credentials; missing API or Relay settings never make a healthy Dashboard/Gateway connection look broken. Pairing-code and QR fallbacks remain available for Relay and compatibility setups. Transport security posture and paired-device grants remain visible without leading the normal setup flow with ports or bearer keys.
|
||||
- **Chat** — Show reasoning toggle, smooth auto-scroll toggle (live-follow streaming, default on), show token usage toggle, app context prompt toggle, tool call display (Off/Compact/Detailed), streaming endpoint selector (`auto` / `sessions` / `runs`), Stats for Nerds (analytics charts)
|
||||
- **Voice** — route-aware voice engine selector (`Vanilla Hermes` via dashboard audio, `Relay Voice Output`, and experimental `Realtime Agent`), global interaction mode (tap / hold / continuous), silence threshold slider, Auto-TTS toggle, selected-engine cards for dashboard or relay-backed settings, language picker, and a Test Current Engine card. Vanilla Hermes voice depends on Manage/dashboard auth; Relay-backed engines run a fast relay health preflight before uploading audio or opening a realtime provider session so a hung relay surfaces as a connection error instead of an indefinite Thinking state.
|
||||
- **Voice** — route-aware voice engine selector (`Vanilla Hermes` via dashboard audio, `Relay Voice Output`, and experimental `Realtime Agent`), global interaction mode (tap / hold / continuous), silence threshold slider, a final-answer-only speech policy, Auto-TTS toggle, selected-engine cards for dashboard or relay-backed settings, language picker, and a Test Current Engine card. Final-answer-only keeps tool/service progress and intermediate commentary visual while both voice engines wait to speak the settled answer; approvals, confirmation questions, and blocking failures remain actionable. Vanilla Hermes voice depends on Manage/dashboard auth; Relay-backed engines run a fast relay health preflight before uploading audio or opening a realtime provider session so a hung relay surfaces as a connection error instead of an indefinite Thinking state.
|
||||
- **Notification companion** — opt-in status, "Open Android Settings" action, test notification dump
|
||||
- **Permissions** — central permission/capability review screen linked from Settings and onboarding. It makes the Vanilla Hermes path explicit ("Chat and Manage" need no Android runtime grant), lists optional camera/microphone/notification access with current status and Android Settings links, and shows sideload-only Device Control requirements only in the sideload flavor.
|
||||
- **Appearance** — theme (auto/light/dark), dynamic colors toggle
|
||||
@@ -847,7 +852,7 @@ utilities.
|
||||
- `GET /voice/config` — provider availability + current settings from `tts:` / `stt:` in `~/.hermes/config.yaml`. When the basic TTS provider is Gemini or xAI, the response includes a `tts.enhanced` capability block (voices/models/audio-tag support + `supports_persona`/`supports_language` flags) so the app renders a per-request enhanced-voice picker. The Vanilla Hermes dashboard `/api/audio/speak` has no per-request surface — enhanced voice there stays config-only via Manage `PUT /api/config`.
|
||||
- `GET/PATCH /voice/output/config`, `POST /voice/output/session`, and `GET /voice/output/{session_id}` — relay-mediated streaming TTS renderer sessions. Android sends final Hermes text or brokered tool-status text and receives mono PCM deltas for direct `AudioTrack` playback. Session creation accepts optional provider/model/voice/sample-rate/language overrides for ephemeral draft previews; omitted values continue to resolve from the saved profile/relay defaults. Session responses include resumable-session metadata and PCM events carry `event_id`/`audio_event_id`, so short route changes during stable speech playback can resume and replay missed audio without re-rendering. Config responses include provider option metadata (`providers[].models`, `providers[].voices`, `providers[].languages`, `providers[].sample_rates`) for first-class dropdowns.
|
||||
- `GET/PATCH /voice/realtime/config`, `POST /voice/realtime/session`, and `GET /voice/realtime/{session_id}` — relay-mediated realtime provider-agent sessions for lab/dev experiments. Android can send PCM input events and receives mono PCM provider deltas for direct `AudioTrack` playback. Realtime config responses expose the same provider option shape where known.
|
||||
- `GET/PATCH /voice/realtime-agent/config`, `POST /voice/realtime-agent/session`, and `GET /voice/realtime-agent/{session_id}` — experimental Hermes-brokered Realtime Agent engine. The broker binds active profile/chat session/auth, streams Android mic PCM to a native realtime provider such as `xai_realtime` or `openai_realtime`, normalizes provider transcript/audio/function-call events, mirrors Hermes session/tool/confirmation events into Android, and returns compact Hermes tool results to the provider for concise spoken follow-up. Session responses include resumable-session metadata (`resume_token`, `resume_supported`, `resume_ttl_ms`); server events carry `event_id`, audio deltas carry `audio_event_id`, and Android can resume a detached session through the current `effectiveRelayUrl` after short Wi-Fi/cellular/LAN/Tailscale changes without starting a second Hermes run. A replacement route is usable only after relay `voice.session.resumed` confirmation; socket generation + resume-episode claims reject stale failure/close/fatal callbacks, unacknowledged input is replayed atomically, and each route-loss episode owns a bounded retry budget that starts at loss rather than session prewarm. Terminal exhaustion detaches session-owned reconnect UI so a stopped retry loop cannot leave an active task pill behind. The only provider-facing tool surface is `hermes_run_task`, `hermes_get_status`, `hermes_cancel`, and `hermes_confirm`.
|
||||
- `GET/PATCH /voice/realtime-agent/config`, `POST /voice/realtime-agent/session`, and `GET /voice/realtime-agent/{session_id}` — experimental Hermes-brokered Realtime Agent engine. The broker binds active profile/chat session/auth, streams Android mic PCM to a native realtime provider such as `xai_realtime` or `openai_realtime`, normalizes provider transcript/audio/function-call events, mirrors Hermes session/tool/confirmation events into Android, and returns compact Hermes tool results to the provider for concise spoken follow-up. Session creation accepts an ephemeral `final_answer_only` boolean; when enabled, the broker disables routine spoken handoffs and progress while preserving approval, confirmation, and blocking-failure prompts. Session responses include resumable-session metadata (`resume_token`, `resume_supported`, `resume_ttl_ms`); server events carry `event_id`, audio deltas carry `audio_event_id`, and Android can resume a detached session through the current `effectiveRelayUrl` after short Wi-Fi/cellular/LAN/Tailscale changes without starting a second Hermes run. A replacement route is usable only after relay `voice.session.resumed` confirmation; socket generation + resume-episode claims reject stale failure/close/fatal callbacks, unacknowledged input is replayed atomically, and each route-loss episode owns a bounded retry budget that starts at loss rather than session prewarm. Terminal exhaustion detaches session-owned reconnect UI so a stopped retry loop cannot leave an active task pill behind. The only provider-facing tool surface is `hermes_run_task`, `hermes_get_status`, `hermes_cancel`, and `hermes_confirm`.
|
||||
- `GET /voice/output/providers/{provider_id}/options`, `GET /voice/realtime/providers/{provider_id}/options`, and `GET /voice/realtime-agent/providers/{provider_id}/options` — provider-specific option refresh before saving. Android calls these when a provider is selected so dynamic account-backed choices can be fetched by the relay without exposing provider secrets. xAI refreshes built-in/paginated custom voices when API/OAuth auth is available; ElevenLabs refreshes voices/models/languages with its API key; OpenAI uses static documented voice choices. Realtime Agent provider payloads include `supports_realtime_agent_native` so render/lab-only realtime support is not confused with native speech-to-speech Hermes tooling. Responses include `schema_version`, grouped voice metadata, recommended/custom flags, and model/voice compatibility hints when known. Unknown or unauthenticated discovery falls back to static provider metadata plus manual entry.
|
||||
- `POST /voice/output/providers/{provider_id}/validate`, `POST /voice/realtime/providers/{provider_id}/validate`, and `POST /voice/realtime-agent/providers/{provider_id}/validate` — pre-save validation for provider/model/voice/sample-rate selections. Unknown manual IDs return warnings; explicit incompatibilities return blocking errors.
|
||||
- Voice-output provider defaults are relay-owned under `voice_output:` in `~/.hermes-relay/config.yaml` (or `RELAY_VOICE_OUTPUT_CONFIG`), then overridden by `RELAY_VOICE_OUTPUT_*` env vars for temporary tests. Authenticated operator clients may patch safe defaults (`enabled`, `provider`, `model`, `voice`, `sample_rate`, `language`, `codec`, `optimize_streaming_latency`, `text_normalization`, `auto_speech_tags`, `fallback_enabled`) through the relay. With `?profile=<name>`, the patch writes that profile's `voice_output:` section. Provider secrets and local auth paths stay server-side. `auto_speech_tags` is an xAI enhanced-voice control: when the renderer is `xai_tts` the relay applies `upstream_voice.apply_xai_speech_tags()` (upstream's inline/wrapping tone markers) to each chunk before rendering, so the streaming path matches the basic `/voice/synthesize` tone behavior. The `voice_lab` renderer set is xai/openai/elevenlabs — there is no Gemini streaming provider, so Gemini enhanced voice is `/voice/synthesize`-only.
|
||||
@@ -864,7 +869,7 @@ utilities.
|
||||
- Stable voice integrates with `ChatViewModel` by **observing** `messages: StateFlow`; transcribed text goes through normal `chatVm.sendMessage(text)` so voice utterances appear as regular user messages in chat history. Experimental Realtime Agent creates a mirrored chat turn and applies broker events directly so tool state, transcript text, assistant deltas, and final responses appear without leaving voice mode.
|
||||
- `VoiceModeOverlay` — full-screen UI with the MorphingSphere at 60% height in `voiceMode=true`, transcribed + response text, mic button supporting Tap / Hold / Continuous interaction modes.
|
||||
- `MorphingSphere` gains `SphereState.Listening` (soft blue/purple, subtle wobble with user amplitude) and `SphereState.Speaking` (vivid green/teal, dramatic core-warmth pulse with agent amplitude). Additive changes — existing call sites unchanged via defaulted `voiceAmplitude` / `voiceMode` params.
|
||||
- Voice Settings screen off the main Settings — Output / Listening / Advanced tabs split engine/provider selection from turn-taking controls and diagnostics. Output groups the provider summary and model/voice catalog, exposes inline no-save play/stop previews for the draft model and individual voices, shows the speaking waveform on the active row, and keeps Discard separate from Save. Dropdowns come from relay-advertised provider metadata, refresh through provider-specific options routes when the selected provider changes, become searchable/grouped for large voice catalogs, and validate compatibility before saving, with advanced manual entry for raw provider/model/voice IDs. Voice routes receive the selected Hermes profile; the relay reports whether values came from profile config, relay config, or global fallback. Test Current Engine remains under Advanced and uses `/voice/output/*` playback for stable mode and `/voice/realtime-agent/*` provider-native session playback for realtime mode; normal assistant speech uses the same streaming renderer PCM path when available.
|
||||
- Voice Settings screen off the main Settings — Output / Listening / Advanced tabs split engine/provider selection from turn-taking controls and diagnostics. Global controls include a final-answer-only policy shared by Standard and Realtime voice. Output groups the provider summary and model/voice catalog, exposes inline no-save play/stop previews for the draft model and individual voices, shows the speaking waveform on the active row, and keeps Discard separate from Save. Dropdowns come from relay-advertised provider metadata, refresh through provider-specific options routes when the selected provider changes, become searchable/grouped for large voice catalogs, and validate compatibility before saving, with advanced manual entry for raw provider/model/voice IDs. Voice routes receive the selected Hermes profile; the relay reports whether values came from profile config, relay config, or global fallback. Test Current Engine remains under Advanced and uses `/voice/output/*` playback for stable mode and `/voice/realtime-agent/*` provider-native session playback for realtime mode; normal assistant speech uses the same streaming renderer PCM path when available.
|
||||
|
||||
See `docs/decisions.md` → **Voice Mode — Architecture** for the historical baseline decisions. Current voice mode records PCM/WAV for STT, routes stable assistant speech through `/voice/output/*`, keeps `/voice/realtime/*` as a provider-agent lab path, and exposes `/voice/realtime-agent/*` as an experimental Hermes-brokered engine.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
[versions]
|
||||
appVersionName = "1.5.0"
|
||||
appVersionCode = "33"
|
||||
agp = "9.3.0"
|
||||
appVersionName = "1.5.2"
|
||||
appVersionCode = "35"
|
||||
agp = "9.3.1"
|
||||
kotlin = "2.4.10"
|
||||
compose-bom = "2026.06.01"
|
||||
navigation-compose = "2.9.8"
|
||||
@@ -15,7 +15,7 @@ security-crypto = "1.1.0"
|
||||
tink-android = "1.23.0"
|
||||
lifecycle = "2.11.0"
|
||||
activity-compose = "1.13.0"
|
||||
browser = "1.9.0"
|
||||
browser = "1.10.0"
|
||||
appcompat = "1.7.1"
|
||||
core-ktx = "1.19.0"
|
||||
datastore = "1.2.1"
|
||||
|
||||
@@ -259,6 +259,7 @@ class RealtimeAgentSession:
|
||||
# ADR 33 promotion state (populated from realtime_voice settings at create).
|
||||
promotion_enabled: bool = False
|
||||
promote_after_ms: int = 6000
|
||||
final_answer_only: bool = False
|
||||
spoken_handoff: bool = True
|
||||
result_delivery: str = "speak_verbatim"
|
||||
promoted_transcript: str | None = None
|
||||
@@ -402,6 +403,7 @@ class RealtimeAgentHandler:
|
||||
config=self.config,
|
||||
)
|
||||
context_messages = _parse_context_messages(payload.get("context_messages"))
|
||||
final_answer_only = _bool_value(payload.get("final_answer_only")) is True
|
||||
fetch_context_messages = getattr(self.hermes, "fetch_context_messages", None)
|
||||
if not context_messages and chat_session_id and callable(fetch_context_messages):
|
||||
context_messages = await self.hermes.fetch_context_messages(
|
||||
@@ -439,10 +441,19 @@ class RealtimeAgentHandler:
|
||||
),
|
||||
promotion_enabled=bool(settings.get("promotion_enabled", False)),
|
||||
promote_after_ms=int(settings.get("promote_after_ms", 6000)),
|
||||
spoken_handoff=bool(settings.get("spoken_handoff", True)),
|
||||
result_delivery=str(settings.get("result_delivery", "speak_verbatim")),
|
||||
final_answer_only=final_answer_only,
|
||||
spoken_handoff=(
|
||||
False if final_answer_only else bool(settings.get("spoken_handoff", True))
|
||||
),
|
||||
result_delivery=(
|
||||
"speak_verbatim"
|
||||
if final_answer_only
|
||||
else str(settings.get("result_delivery", "speak_verbatim"))
|
||||
),
|
||||
progress_spoken_after_seconds=(
|
||||
max(0, int(settings.get("progress_spoken_after_ms", 0))) / 1000.0
|
||||
0.0
|
||||
if final_answer_only
|
||||
else max(0, int(settings.get("progress_spoken_after_ms", 0))) / 1000.0
|
||||
),
|
||||
progress_repeat_seconds=(
|
||||
max(0, int(settings.get("progress_repeat_ms", 30000))) / 1000.0
|
||||
@@ -4805,6 +4816,19 @@ def _native_instructions(session: RealtimeAgentSession) -> str:
|
||||
current_timezone = interface_context["current_timezone"]
|
||||
context_block = _provider_context_block(session.context_messages)
|
||||
profile_block = _profile_prompt_block(session.profile_prompt_context)
|
||||
speech_policy = (
|
||||
"Final-answer-only speech is enabled. Do not speak acknowledgements, "
|
||||
"tool progress, service or status updates, or intermediate commentary. "
|
||||
"Call Hermes silently and wait to speak until its settled final answer is "
|
||||
"available. Approval or confirmation questions and blocking failures may "
|
||||
"still be spoken because the user must act on them. "
|
||||
if session.final_answer_only
|
||||
else (
|
||||
"You may speak one brief acknowledgement such as 'I'll check Hermes' "
|
||||
"or 'I'll check that' before the tool call, then call hermes_run_task "
|
||||
"immediately. The relay will provide restrained status while Hermes runs. "
|
||||
)
|
||||
)
|
||||
return (
|
||||
"You are the provider-native speech loop for Hermes Relay. Keep replies "
|
||||
"brief and conversational. Active interface: "
|
||||
@@ -4842,11 +4866,9 @@ def _native_instructions(session: RealtimeAgentSession) -> str:
|
||||
"information beyond what was already delivered. If the needed context is "
|
||||
"missing, stale, or requires fresh data or verification you do not "
|
||||
"already have, call hermes_run_task before answering. Do not say you lack "
|
||||
"context before a Hermes call. You "
|
||||
"may speak one brief acknowledgement such as 'I'll check Hermes' or "
|
||||
"'I'll check that' before the tool call, then call hermes_run_task "
|
||||
"immediately. Do not give a substantive answer until Hermes returns. "
|
||||
"The relay will provide restrained status while Hermes runs. "
|
||||
"context before a Hermes call. "
|
||||
f"{speech_policy}"
|
||||
"Do not give a substantive answer until Hermes returns. "
|
||||
"Route through Hermes for latest/recent/versioned data, device/desktop/app "
|
||||
"state, personal/session/project context, side effects, high-stakes or "
|
||||
"precision-sensitive answers, explicit check/verify/look-up requests, and "
|
||||
|
||||
@@ -630,6 +630,26 @@ class RealtimeAgentRoutesTests(AioHTTPTestCase):
|
||||
self.assertGreaterEqual(body["resume_ttl_ms"], 1000)
|
||||
self.assertTrue(body["experimental"])
|
||||
|
||||
async def test_final_answer_only_overrides_spoken_progress_for_one_session(self) -> None:
|
||||
token = await self._make_session()
|
||||
|
||||
resp = await self.client.post(
|
||||
"/voice/realtime-agent/session",
|
||||
json={"final_answer_only": True},
|
||||
headers=self._bearer(token),
|
||||
)
|
||||
|
||||
self.assertEqual(resp.status, 200)
|
||||
body = await resp.json()
|
||||
session = self._server().realtime_agent.sessions[body["session_id"]]
|
||||
self.assertTrue(session.final_answer_only)
|
||||
self.assertFalse(session.spoken_handoff)
|
||||
self.assertEqual(session.progress_spoken_after_seconds, 0.0)
|
||||
self.assertEqual(session.result_delivery, "speak_verbatim")
|
||||
instructions = broker_module._native_instructions(session)
|
||||
self.assertIn("Do not speak acknowledgements", instructions)
|
||||
self.assertIn("wait to speak until its settled final answer", instructions)
|
||||
|
||||
async def test_provider_native_instructions_include_recent_context(self) -> None:
|
||||
token = await self._make_session()
|
||||
fake_provider = FakeNativeProvider()
|
||||
|
||||
@@ -5,8 +5,8 @@ pluginManagement {
|
||||
gradlePluginPortal()
|
||||
}
|
||||
plugins {
|
||||
id("com.android.application") version "9.3.0"
|
||||
id("com.android.library") version "9.3.0"
|
||||
id("com.android.application") version "9.3.1"
|
||||
id("com.android.library") version "9.3.1"
|
||||
id("org.jetbrains.kotlin.plugin.compose") version "2.4.10"
|
||||
id("org.jetbrains.kotlin.plugin.serialization") version "2.4.10"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user