Compare commits

...
Author SHA1 Message Date
Bailey Dixon f9c8736e5b fix(android): keep dashboard OIDC on cookie flow 2026-07-27 08:57:17 -04:00
dependabot[bot] a815dd33fa build(deps): bump com.android.library from 9.3.0 to 9.3.1 (#261)
Bumps com.android.library from 9.3.0 to 9.3.1.

---
updated-dependencies:
- dependency-name: com.android.library
  dependency-version: 9.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 11:56:57 +00:00
dependabot[bot] cc9c75a636 build(deps): bump androidx.browser:browser from 1.9.0 to 1.10.0 (#260)
Bumps androidx.browser:browser from 1.9.0 to 1.10.0.

---
updated-dependencies:
- dependency-name: androidx.browser:browser
  dependency-version: 1.10.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 11:55:03 +00:00
dependabot[bot] 43179e03c0 build(deps): bump com.android.application from 9.3.0 to 9.3.1 (#259)
Bumps com.android.application from 9.3.0 to 9.3.1.

---
updated-dependencies:
- dependency-name: com.android.application
  dependency-version: 9.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 11:55:00 +00:00
Bailey Dixon 693ac4ed64 Merge pull request #256 from Codename-11/release/android-1.5.1-patch
release(android): android-v1.5.1
2026-07-26 15:44:19 -04:00
Bailey Dixon f94d663ac4 release(android): android-v1.5.1 2026-07-26 15:35:21 -04:00
Bailey Dixon d1a21bd42e fix(android): use Compose resources for sign-in copy 2026-07-26 15:35:20 -04:00
Bailey Dixon 7ee2d73010 fix(android): preserve formatted chat completion position 2026-07-26 15:10:48 -04:00
Bailey Dixon 682bde84fe fix(android): merge API 36 target 2026-07-26 09:38:22 -04:00
Bailey Dixon 16bdbe5f44 fix(android): target API 36 2026-07-26 09:38:00 -04:00
Bailey Dixon f43fba9fed feat(android): merge chat readability and final-only voice 2026-07-26 09:13:57 -04:00
Bailey Dixon d1745413fd fix(android): release realtime background foreground turns 2026-07-26 08:57:00 -04:00
Bailey Dixon 1b7a8025c3 fix(android): restore standard voice narration 2026-07-26 08:56:42 -04:00
Bailey Dixon d92a87483e feat(android): enhance voice conversation experience 2026-07-26 08:56:23 -04:00
Bailey Dixon 0bea626ed8 Merge pull request #253 from Codename-11/fix/android-release-dispatch
fix(android): repair immutable release dispatch
2026-07-25 18:30:27 -04:00
44 changed files with 1220 additions and 319 deletions
+18
View File
@@ -10,6 +10,24 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
- **Windows-trusted certificates work in the desktop CLI.** The packaged Windows binary and newer Node runtimes add the Windows certificate store without dropping bundled or operator-supplied roots, while TLS verification and Relay certificate pinning remain enforced.
## [Android 1.5.1] - 2026-07-26
### Added
- **Voice supports focused and conversational layouts.** Focus keeps spoken turns, Markdown, tools, media, and actions in a compact voice surface, while Conversation opens the full Chat renderer without leaving the active voice session.
- **Voice can speak only settled answers.** A global Voice setting keeps tool progress, service updates, and intermediate commentary visual while supported voice paths wait to speak the final Hermes answer.
### Changed
- **Chat answers are easier to read in every theme.** Primary assistant text now uses the theme's full-contrast foreground, and chat prose uses a 15sp size with 21sp line height.
- **Google Play builds target Android 16.** The app now targets API level 36 while retaining its existing minimum-device support.
### Fixed
- **Completed streamed answers render their formatting without losing the reading position.** Markdown headings, lists, emphasis, and code blocks replace the live text renderer only after completion, then the measured trailing edge remains anchored at the bottom.
- **Standard Voice speaks completed assistant replies again.** Session and message fences no longer suppress a valid final answer during the handoff from generation to narration.
- **Realtime background work no longer blocks the active voice controls.** A promoted task releases the foreground spinner and microphone while its progress, tools, cancellation, and final result remain available in the owning chat.
## [Server 1.4.3] - 2026-07-22
### Added
+16
View File
@@ -1,5 +1,21 @@
# Hermes-Relay — Dev Log
## 2026-07-26 — Android 1.5.1 patch reconciliation
Android 1.5.1 reconciles the post-1.5.0 voice and chat fixes into versionCode
34. Voice now offers compact Focus and full Conversation presentation,
Standard narration preserves valid completed replies, and promoted Realtime
tasks release foreground voice controls while retaining progress and results.
Completed streamed answers promote from the stable live text node to full
Markdown only after completion. The measured Markdown row is then positioned
by its trailing edge until deferred code and attachment measurement settles,
preventing the LazyColumn from restoring the start of a tall response.
The release also targets Android API level 36. Release notes, in-app What's
New assets, localized Play notes, and the Play listing reference were updated
for Android 1.5.1.
## 2026-07-25 — Immutable Android release dispatch repair
Android approval now dispatches the current release workflow definition from
+12 -18
View File
@@ -1,10 +1,10 @@
# Hermes-Relay-Android v1.5.0
# Hermes-Relay-Android v1.5.1
**Release Date:** July 25, 2026
**Release Date:** July 26, 2026
## Download
> Installing on your phone? Download `hermes-relay-1.5.0-sideload-release.apk` and tap it for the full feature set, or install the conservative build from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
> Installing on your phone? Download `hermes-relay-1.5.1-sideload-release.apk` and tap it for the full feature set, or install the conservative build from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
The `.aab` file is a Play Console upload bundle and cannot be installed by tapping it on a phone.
@@ -12,28 +12,22 @@ Verify the download against `SHA256SUMS.txt`. See the [sideload guide](https://h
## Summary
This release makes the Hermes Dashboard and Gateway the clear standard connection, keeps active work reachable across the app and Android backgrounding, and gives profiles, voice, attachments, image generation, and approvals a more coherent native interface.
This patch restores reliable narration and background behavior in Voice, adds focused and full-conversation voice layouts, and keeps richly formatted streamed answers anchored at their completed end.
## Added
- The Agent Passport drawer combines live route and session context with explicit profile switching, personality, model, reasoning, approval-policy, chat-override, and processing-tier controls.
- Secure browser-based Dashboard sign-in is scoped to the selected host. Chat, sessions, Manage, and Standard Voice share the same authenticated Gateway route while the API server remains an automatic fallback.
- Standard and Realtime voice settings use focused provider, model, and voice cards with upstream-aware discovery, descriptions, inline previews, waveforms, and a browsable catalog. Standard replies can begin speaking completed segments before generation finishes.
- User-started turns stay protected until every concurrent session settles. Privacy-safe notifications reopen the correct chat for approvals, questions, elevated permissions, or secure responses.
- Onboarding finishes with a layered permission review: notifications are recommended deliberately, optional capabilities remain separate, and users can continue without granting phone access.
- Chat surfaces one-turn model choices, approval modes, advisor progress, queued recovery, project labels, collapsible attachments, persisted images, interim Gateway events, and image-generation activity.
- Voice Focus keeps narration, Markdown, tools, media, and actionable cards in a compact voice-first view.
- Voice Conversation exposes the complete Chat renderer while preserving the active voice session.
- A final-answer speech preference keeps intermediate progress visual while supported voice paths wait for the settled response.
## Fixed
- Gateway reconnects reactivate the original live session without resubmitting acknowledged prompts or duplicating session rows.
- Tailscale, QR, and other remote routes move Dashboard, Gateway, sessions, Manage, Standard Voice, API fallback, and optional Relay together.
- Dashboard authentication, model routing, recovery, and profile state stay scoped to the selected connection and session, including during cold start and rapid switching.
- Promoted voice and background tasks keep their owning Chat row until the work settles.
- User-installed certificate authorities work for self-hosted HTTPS/WSS while normal chain, hostname, and Relay-pin verification remain enforced.
- Malformed syntax-highlighting ranges no longer crash Markdown rendering.
- Developer Options no longer exposes the obsolete Relay feature flag; version-tap unlock, relock, backup, import, and reset actions now persist and report accurately.
- Standard Voice narrates valid completed assistant responses instead of losing them during the generation-to-speech handoff.
- Realtime tasks promoted to background release the foreground spinner and microphone while progress and results remain reachable.
- Completed streamed answers switch to full Markdown and preserve the measured trailing edge instead of jumping to the start of the response.
- Assistant text uses the theme's full-contrast foreground with a more readable chat type scale.
## Install / Verify
- App version: **1.5.0** (versionCode **33**).
- App version: **1.5.1** (versionCode **34**).
- Standard Chat and Vanilla Hermes voice continue to work against unmodified upstream Hermes.
+4 -7
View File
@@ -717,9 +717,10 @@ Deferred:
A 5-agent audit compared the chat surface to Discord/Telegram/Messenger/iMessage/
GitHub-mobile. **Shipped this pass (pending on-device verification):** a chat-tuned
`markdownTypography()` ramp (headings were falling through to M3 display roles —
h1=`displayLarge` 57sp in this app's scale — so a `#` was a billboard; now h1≈20sp
scaling down, list/paragraph unified to 14sp, inline+fenced code 13sp, `textLink`
`markdownTypography()` ramp (headings were falling through to M3 display roles —
h1=`displayLarge` 57sp in this app's scale — so a `#` was a billboard; now h1≈20sp
scaling down, list/paragraph unified to 15sp/21sp, primary assistant prose moved
to the theme's full-contrast `onSurface`, inline+fenced code 13sp, `textLink`
accent+underline) in `MarkdownContent.kt`; timestamp gated to `isLastInGroup` (was on
every bubble) + grouping breaks on a >5min gap (`GROUP_GAP_MS`) so a resumed
conversation gets its own beat; long-press haptic on the action menu; streaming dots
@@ -731,10 +732,6 @@ gated to pre-first-token. Deferred:
parses one full CommonMark document so global link references, indentation, and
nested containers remain correct; the viewport now anchors that same remeasure.
Verify lists, tables, quotes, HTML, nested fences, and reference links on-device.
- **Bubble body 14sp → 15sp/21.** 14sp is the smallest body of the five reference
apps. Bump markdown paragraph/text/list + the two plain `Text` sites
(`MessageBubble.kt` user/system) together; keep ~1.4 leading so the ~272dp measure
stays ~36–38 chars/line. Debatable/broad — left out of the certain heading win.
- **Tail-corner on last-in-group only (design decision).** The audit flagged the
per-bubble bottom tail as "half-implemented," but it's a deliberate aesthetic
(every bubble tails). Switching to iMessage-style "tail on the last bubble only"
+1 -1
View File
@@ -37,7 +37,7 @@ android {
// exempt from Play's 14-day closed-testing rule. See RELEASE.md.
applicationId = "com.axiomlabs.hermesrelay"
minSdk = 26
targetSdk = 35
targetSdk = 36
versionCode = libs.versions.appVersionCode.get().toInt()
versionName = libs.versions.appVersionName.get()
@@ -1 +1 @@
Connect through secure Dashboard sign-in, switch profiles from the new Agent Passport, and keep multiple background chats active with actionable approval and question alerts. Image generation, attachments, model routing, voice, and Gateway recovery are clearer and more reliable. Setup now guides optional notification, camera, microphone, and companion permissions without blocking chat.
Choose compact Voice Focus or the complete Conversation layout. Standard Voice reliably speaks completed replies, Realtime background work no longer blocks voice controls, and streamed answers render Markdown while staying anchored at their completed end.
@@ -1 +1 @@
通过安全的 Dashboard 登录连接,并在新的智能体护照中切换配置文件。多个后台对话可保持运行,审批或提问通知可直接返回正确会话。图像生成、附件、模型路由、语音和 Gateway 恢复更加清晰可靠。设置流程会说明可选的通知、相机、麦克风和通知伴侣权限,且不会阻止聊天。
可选择精简的语音专注视图或完整的对话视图。标准语音现在会可靠朗读已完成的回复,实时后台任务不再阻塞语音控制,流式回复会渲染 Markdown 并停留在完成位置。
+28
View File
@@ -1,5 +1,33 @@
{
"versions": [
{
"version": "1.5.1",
"title": "Voice and chat stay in place",
"date": "2026-07-26",
"sections": [
{
"header": "Voice at the right depth",
"bullets": [
"Use Voice Focus for a compact spoken-turn view or Conversation for the complete Chat renderer without leaving the active voice session.",
"Keep intermediate work visual while supported voice paths wait to speak the settled final response."
]
},
{
"header": "Reliable narration and background work",
"bullets": [
"Standard Voice now speaks valid completed replies after generation hands off to narration.",
"Realtime background tasks release foreground voice controls while their progress and results remain reachable."
]
},
{
"header": "Formatted answers stay readable",
"bullets": [
"Completed streams render headings, lists, emphasis, and code blocks without returning to the beginning of the answer.",
"Assistant text uses stronger theme contrast and a more comfortable chat reading scale."
]
}
]
},
{
"version": "1.5.0",
"title": "Hermes, always in reach",
+5 -5
View File
@@ -1,6 +1,6 @@
v1.5.0 - Hermes, always in reach
v1.5.1 - Voice and chat stay in place
* Connect through secure Dashboard sign-in and switch profiles from the new Agent Passport.
* Keep multiple background chats active and reopen the right session from approval or question alerts.
* Follow richer attachments, image generation, model routing, Gateway recovery, and streaming voice.
* Finish setup with clear, optional permission guidance that never blocks standard chat.
* Choose a compact Voice Focus view or the complete Conversation renderer.
* Hear Standard Voice replies reliably after generation completes.
* Keep Realtime background work active without blocking voice controls.
* Read formatted streamed answers without jumping back to their beginning.
@@ -36,6 +36,14 @@ data class VoiceSettings(
val audioRoute: String = VoiceAudioRoute.Auto.storageValue,
val interactionMode: String = "tap",
val silenceThresholdMs: Long = 1250L,
/**
* When true, voice keeps progress visual and waits for the settled Hermes
* answer before speaking. Tool status, service updates, and intermediate
* assistant commentary are not narrated.
*/
val finalAnswerOnly: Boolean = false,
/** Presentation only; changing this never restarts or interrupts voice. */
val presentationMode: String = VoicePresentationMode.Focus.storageValue,
val realtimeTraceDetails: Boolean = false,
/**
* When true (default), Realtime Agent keeps one provider session/socket open
@@ -122,6 +130,16 @@ enum class VoiceAudioRoute(val storageValue: String) {
}
}
enum class VoicePresentationMode(val storageValue: String) {
Focus("focus"),
Conversation("conversation");
companion object {
fun fromStorage(value: String?): VoicePresentationMode =
values().firstOrNull { it.storageValue == value } ?: Focus
}
}
/**
* Active scope for per-profile voice prefs.
*
@@ -182,6 +200,8 @@ class VoicePreferencesRepository(private val dataStore: DataStore<Preferences>)
// un-namespaced means switching profiles never churns these.
private val KEY_INTERACTION_MODE = stringPreferencesKey("voice_interaction_mode")
private val KEY_SILENCE_THRESHOLD_MS = longPreferencesKey("voice_silence_threshold_ms")
private val KEY_FINAL_ANSWER_ONLY = booleanPreferencesKey("voice_final_answer_only")
private val KEY_PRESENTATION_MODE = stringPreferencesKey("voice_presentation_mode")
private val KEY_REALTIME_TRACE_DETAILS = booleanPreferencesKey("voice_realtime_trace_details")
private val KEY_REALTIME_PERSISTENT_SESSION =
booleanPreferencesKey("voice_realtime_persistent_session")
@@ -191,6 +211,8 @@ class VoicePreferencesRepository(private val dataStore: DataStore<Preferences>)
const val DEFAULT_INTERACTION_MODE = "tap"
// 1250 ms matches hermes-desktop voice_mode `silenceMs` end-of-speech.
const val DEFAULT_SILENCE_THRESHOLD_MS = 1250L
const val DEFAULT_FINAL_ANSWER_ONLY = false
const val DEFAULT_PRESENTATION_MODE = "focus"
const val DEFAULT_REALTIME_TRACE_DETAILS = false
const val DEFAULT_REALTIME_PERSISTENT_SESSION = true
@@ -258,6 +280,10 @@ class VoicePreferencesRepository(private val dataStore: DataStore<Preferences>)
// --- global (shared across profiles) ---
interactionMode = prefs[KEY_INTERACTION_MODE] ?: DEFAULT_INTERACTION_MODE,
silenceThresholdMs = prefs[KEY_SILENCE_THRESHOLD_MS] ?: DEFAULT_SILENCE_THRESHOLD_MS,
finalAnswerOnly = prefs[KEY_FINAL_ANSWER_ONLY] ?: DEFAULT_FINAL_ANSWER_ONLY,
presentationMode = VoicePresentationMode.fromStorage(
prefs[KEY_PRESENTATION_MODE] ?: DEFAULT_PRESENTATION_MODE,
).storageValue,
realtimeTraceDetails = prefs[KEY_REALTIME_TRACE_DETAILS]
?: DEFAULT_REALTIME_TRACE_DETAILS,
realtimePersistentSession = prefs[KEY_REALTIME_PERSISTENT_SESSION]
@@ -367,6 +393,14 @@ class VoicePreferencesRepository(private val dataStore: DataStore<Preferences>)
dataStore.edit { it[KEY_SILENCE_THRESHOLD_MS] = ms.coerceAtLeast(500L) }
}
suspend fun setFinalAnswerOnly(enabled: Boolean) {
dataStore.edit { it[KEY_FINAL_ANSWER_ONLY] = enabled }
}
suspend fun setPresentationMode(mode: VoicePresentationMode) {
dataStore.edit { it[KEY_PRESENTATION_MODE] = mode.storageValue }
}
suspend fun setRealtimeTraceDetails(enabled: Boolean) {
dataStore.edit { it[KEY_REALTIME_TRACE_DETAILS] = enabled }
}
@@ -1278,6 +1278,7 @@ class RelayVoiceClient(
model: String? = null,
voice: String? = null,
sampleRate: Int? = null,
finalAnswerOnly: Boolean = false,
onHandoff: (VoiceHandoffEvent) -> Unit = {},
turnInputs: kotlinx.coroutines.channels.ReceiveChannel<RealtimeTurnInput>? = null,
onTurnComplete: (RealtimeVoiceSummary) -> Unit = {},
@@ -1309,6 +1310,7 @@ class RelayVoiceClient(
model = model,
voice = voice,
sampleRate = sampleRate,
finalAnswerOnly = finalAnswerOnly,
)
if (sessionResult.isFailure) {
return@withContext Result.failure(sessionResult.exceptionOrNull() ?: IOException("Realtime agent session failed"))
@@ -1822,6 +1824,28 @@ class RelayVoiceClient(
if (event.type == "hermes.run.promoted") {
longRunningTurn.set(true)
Log.i(TAG, "Realtime agent turn marked long-running (run promoted); relaxing idle guard")
if (persistent &&
event.spokenHandoff == false &&
activeTurn.compareAndSet(true, false)
) {
Log.i(
TAG,
"Realtime agent foreground turn ended at silent background promotion",
)
onTurnComplete(
RealtimeVoiceSummary(
provider = event.provider ?: session.provider,
model = event.model ?: session.model,
voice = event.voice ?: session.voice,
sampleRate = session.sampleRate,
audioChunks = audioChunks,
audioBytes = audioBytes,
firstAudioMs = event.firstAudioMs,
responseDoneMs = event.responseDoneMs,
eventLogPath = event.eventLogPath ?: session.eventLogPath,
)
)
}
}
if (event.isAudioDelta) {
audioChunks += 1
@@ -1847,13 +1871,12 @@ class RelayVoiceClient(
responseDoneMs = event.responseDoneMs,
eventLogPath = event.eventLogPath ?: session.eventLogPath,
)
if (persistent) {
if (persistent && activeTurn.compareAndSet(true, false)) {
// Turn boundary, not session boundary: keep the socket
// open for the next utterance.
activeTurn.set(false)
longRunningTurn.set(false)
onTurnComplete(summary)
} else {
} else if (!persistent) {
if (claimTerminalSocket(
webSocket,
generation,
@@ -2679,6 +2702,7 @@ class RelayVoiceClient(
model: String? = null,
voice: String? = null,
sampleRate: Int? = null,
finalAnswerOnly: Boolean = false,
): Result<RealtimeSessionResponse> {
val body = buildJsonObject {
putProfile()
@@ -2694,6 +2718,9 @@ class RelayVoiceClient(
sampleRate?.takeIf { it > 0 }?.let {
put("sample_rate", JsonPrimitive(it))
}
if (finalAnswerOnly) {
put("final_answer_only", JsonPrimitive(true))
}
chatSessionId?.trim()?.takeIf { it.isNotBlank() }?.let {
put("chat_session_id", JsonPrimitive(it))
}
@@ -2980,6 +3007,9 @@ class RelayVoiceClient(
responseDoneMs = (metrics?.get("response_done_ms") as? JsonPrimitive)?.doubleOrNull,
tier = (obj["tier"] as? JsonPrimitive)?.contentOrNull,
floor = (obj["floor"] as? JsonPrimitive)?.contentOrNull,
spokenHandoff = (obj["spoken_handoff"] as? JsonPrimitive)
?.contentOrNull
?.toBooleanStrictOrNull(),
activeToolName = (obj["active_tool_name"] as? JsonPrimitive)?.contentOrNull,
completedToolCount = (obj["completed_tool_count"] as? JsonPrimitive)?.intOrNull
?: (obj["tool_count"] as? JsonPrimitive)?.intOrNull,
@@ -3374,6 +3404,7 @@ data class RealtimeVoiceEvent(
// ADR 33: background-run promotion fields.
val tier: String? = null,
val floor: String? = null,
val spokenHandoff: Boolean? = null,
// hermes.run.progress extras — drive the live background-run chip.
val activeToolName: String? = null,
val completedToolCount: Int? = null,
@@ -108,6 +108,7 @@ import com.hermesandroid.relay.data.EnhancedVoiceOverrides
import com.hermesandroid.relay.data.EndpointCandidate
import com.hermesandroid.relay.data.VoiceAudioRoute
import com.hermesandroid.relay.data.VoicePreferencesRepository
import com.hermesandroid.relay.data.VoicePresentationMode
import com.hermesandroid.relay.data.VoiceSettings
import com.hermesandroid.relay.data.capabilities
import com.hermesandroid.relay.data.displayLabel
@@ -1919,6 +1920,14 @@ fun RelayApp() {
voiceViewModel = voiceViewModel,
voiceClient = voiceClient,
maxBubbleWidth = maxBubbleWidth,
voicePresentationMode = VoicePresentationMode.fromStorage(
voiceSettings.presentationMode,
),
onVoicePresentationModeChange = { mode ->
connectionSwitchScope.launch {
voicePreferences.setPresentationMode(mode)
}
},
openAgentSheetOnEntry = openAgentSheetArg,
onAgentSheetArgConsumed = {
backStackEntry.arguments?.putBoolean(
@@ -72,6 +72,7 @@ import androidx.compose.ui.text.input.ImeAction
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.Dp
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.compose.ui.res.stringResource
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.MessageRole
@@ -261,8 +262,12 @@ fun AgentTextFlow(
motionEnabled: Boolean,
modifier: Modifier = Modifier,
) {
val flowStyle = MaterialTheme.typography.bodyMedium.copy(fontFamily = FontFamily.Monospace)
val flowColor = MaterialTheme.colorScheme.onSurfaceVariant
val flowStyle = MaterialTheme.typography.bodyMedium.copy(
fontFamily = FontFamily.Monospace,
fontSize = 15.sp,
lineHeight = 21.sp,
)
val flowColor = MaterialTheme.colorScheme.onSurface
// Readable, non-faded mirror of the visible tail — used as the live-region
// text on both paths so assistive tech hears the words.
@@ -67,6 +67,11 @@ fun MarkdownContent(
modifier: Modifier = Modifier
) {
val isDarkTheme = LocalBrand.current.isDark
val chatBodyStyle = MaterialTheme.typography.bodyMedium.copy(
fontSize = 15.sp,
lineHeight = 21.sp,
color = textColor,
)
val highlightsBuilder = remember(isDarkTheme) {
Highlights.Builder().theme(SyntaxThemes.atom(darkMode = isDarkTheme))
}
@@ -87,7 +92,8 @@ fun MarkdownContent(
// ~45sp, h3=displaySmall 36sp) — a single `#` becomes a billboard inside the
// ~272dp bubble. Here every level derives from bodyLarge/bodyMedium (so the
// live font-picker still applies) and is capped so the largest heading is
// ~1.4x the 14sp body, matching Discord / GitHub-mobile in-message headings.
// proportionate to the 15sp body, matching Discord / GitHub-mobile
// in-message headings.
typography = markdownTypography(
h1 = MaterialTheme.typography.bodyLarge.copy(
fontSize = 20.sp, lineHeight = 26.sp, fontWeight = FontWeight.Bold, color = textColor,
@@ -108,16 +114,17 @@ fun MarkdownContent(
fontSize = 13.sp, fontWeight = FontWeight.SemiBold, letterSpacing = 0.4.sp,
color = textColor.copy(alpha = 0.85f),
),
// Prose, list items, and quotes all sit at the 14sp body size so a
// paragraph and the bullet list under it share one rhythm — the library
// default 'text'/list role is bodyLarge (16sp), 2sp larger than paragraph.
paragraph = MaterialTheme.typography.bodyMedium.copy(color = textColor),
text = MaterialTheme.typography.bodyMedium.copy(color = textColor),
bullet = MaterialTheme.typography.bodyMedium.copy(color = textColor),
ordered = MaterialTheme.typography.bodyMedium.copy(color = textColor),
list = MaterialTheme.typography.bodyMedium.copy(color = textColor),
quote = MaterialTheme.typography.bodyMedium.copy(
fontStyle = FontStyle.Italic, color = textColor.copy(alpha = 0.78f),
// Prose, list items, and quotes share a 15sp/21sp reading rhythm.
// The library default 'text'/list role is bodyLarge (16sp), while
// bodyMedium was previously 14sp and unnecessarily small for long chat.
paragraph = chatBodyStyle,
text = chatBodyStyle,
bullet = chatBodyStyle,
ordered = chatBodyStyle,
list = chatBodyStyle,
quote = chatBodyStyle.copy(
fontStyle = FontStyle.Italic,
color = textColor.copy(alpha = 0.9f),
),
// Inline + fenced code at 13sp (one step under body, not two): monospace
// + the tinted chip already signal "code" without also shrinking it, and
@@ -125,7 +132,7 @@ fun MarkdownContent(
code = MaterialTheme.typography.bodySmall.copy(
fontSize = 13.sp, letterSpacing = 0.sp,
fontFamily = FontFamily.Monospace,
color = MaterialTheme.colorScheme.onSurfaceVariant,
color = textColor,
),
inlineCode = MaterialTheme.typography.bodyMedium.copy(
fontSize = 13.sp, letterSpacing = 0.sp,
@@ -302,7 +309,10 @@ fun StreamingMarkdownContent(
// code and deliberately spaced prose are not altered.
text = content.withoutLeadingBlankLines(),
modifier = modifier,
style = MaterialTheme.typography.bodyMedium,
style = MaterialTheme.typography.bodyMedium.copy(
fontSize = 15.sp,
lineHeight = 21.sp,
),
color = textColor,
)
} else {
@@ -174,7 +174,7 @@ fun MessageBubble(
val textColor = when (message.role) {
MessageRole.USER -> MaterialTheme.colorScheme.onPrimary
MessageRole.ASSISTANT -> MaterialTheme.colorScheme.onSurfaceVariant
MessageRole.ASSISTANT -> MaterialTheme.colorScheme.onSurface
MessageRole.SYSTEM -> MaterialTheme.colorScheme.onTertiaryContainer
}
@@ -469,7 +469,10 @@ fun MessageBubble(
// Plain text for user and system messages
Text(
text = message.content,
style = MaterialTheme.typography.bodyMedium,
style = MaterialTheme.typography.bodyMedium.copy(
fontSize = 15.sp,
lineHeight = 21.sp,
),
color = textColor
)
} else {
@@ -41,6 +41,7 @@ import androidx.compose.material.icons.filled.Close
import androidx.compose.material.icons.filled.ExpandLess
import androidx.compose.material.icons.filled.ExpandMore
import androidx.compose.material.icons.filled.GraphicEq
import androidx.compose.material.icons.filled.Image
import androidx.compose.material.icons.filled.Mic
import androidx.compose.material.icons.filled.Refresh
import androidx.compose.material.icons.filled.Settings
@@ -73,8 +74,10 @@ import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.HermesCardAction
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.data.ToolCall
import com.hermesandroid.relay.data.VoicePresentationMode
import com.hermesandroid.relay.ui.components.avatar.AvatarRenderState
import com.hermesandroid.relay.ui.components.avatar.LocalAgentAvatar
import com.hermesandroid.relay.ui.LocalSnackbarHost
@@ -146,7 +149,8 @@ fun VoiceModeOverlay(
voiceOutputEnabled: Boolean? = null,
voiceOutputFallbackEnabled: Boolean? = null,
onOverlayRequest: () -> Unit = {},
onCompactModeChange: (Boolean) -> Unit = {},
presentationMode: VoicePresentationMode = VoicePresentationMode.Focus,
onPresentationModeChange: (VoicePresentationMode) -> Unit = {},
// === END PHASE3-voice-mode-transcript ===
// === v0.4.1 JIT permission-denied chip ===
// Tapped when the user clicks the permission-denied chip. Default no-op
@@ -159,22 +163,21 @@ fun VoiceModeOverlay(
onBackgroundRunCancel: () -> Unit = {},
onBackgroundRunTap: () -> Unit = {},
onHermesConfirmationAnswer: (String) -> Unit = {},
onCardAction: (messageId: String, cardKey: String, action: HermesCardAction) -> Unit =
{ _, _, _ -> },
onCardInput: (messageId: String, cardKey: String, value: String) -> Unit =
{ _, _, _ -> },
// === END v0.4.1 ===
) {
val surface = MaterialTheme.colorScheme.surface
val haptic = LocalHapticFeedback.current
var controlsExpanded by remember { mutableStateOf(false) }
var focusMode by remember { mutableStateOf(true) }
val focusMode = presentationMode == VoicePresentationMode.Focus
val setFocusMode: (Boolean) -> Unit = { focused ->
focusMode = focused
onCompactModeChange(!focused)
}
LaunchedEffect(uiState.voiceMode) {
if (!uiState.voiceMode) {
setFocusMode(true)
}
onPresentationModeChange(
if (focused) VoicePresentationMode.Focus else VoicePresentationMode.Conversation,
)
}
// Voice errors surface ONLY on the overlay's own inline top banner
@@ -193,7 +196,7 @@ fun VoiceModeOverlay(
// chips, pill) didn't handle so stray taps/swipes don't fall
// through to the chat + session drawer behind it. Children run on
// the same Main pass leaf-first, so this only catches the gaps.
// In compact mode the overlay is intentionally transparent and the
// In Conversation the overlay is intentionally transparent and the
// chat stays interactive, so no scrim is installed.
.then(
if (focusMode) {
@@ -404,6 +407,11 @@ fun VoiceModeOverlay(
message = msg,
showThinking = showThinking,
expanded = msg.id == latestId || msg.isStreaming,
onViewConversation = {
onPresentationModeChange(VoicePresentationMode.Conversation)
},
onCardAction = onCardAction,
onCardInput = onCardInput,
)
}
if (pendingTranscriptText != null) {
@@ -418,6 +426,11 @@ fun VoiceModeOverlay(
),
showThinking = showThinking,
expanded = true,
onViewConversation = {
onPresentationModeChange(VoicePresentationMode.Conversation)
},
onCardAction = onCardAction,
onCardInput = onCardInput,
)
}
}
@@ -449,7 +462,7 @@ fun VoiceModeOverlay(
}
}
// Compact mode: the background-run chip must survive outside focus
// Conversation: the background-run chip must survive outside focus
// mode too — a running task with no visible presence reads as lost
// (the chip previously existed ONLY in the focus layout).
AnimatedVisibility(
@@ -821,6 +834,15 @@ private fun VoiceSessionPill(
overflow = TextOverflow.Ellipsis,
modifier = Modifier.weight(1f),
)
if (!focusMode) {
ConversationVoiceMicButton(
uiState = uiState,
onMicTap = onMicTap,
onMicRelease = onMicRelease,
onInterrupt = onInterrupt,
onPauseAutoMode = onPauseAutoMode,
)
}
Icon(
imageVector = if (expanded) Icons.Filled.ExpandLess else Icons.Filled.ExpandMore,
contentDescription = if (expanded) stringResource(R.string.voice_overlay_collapse_cd) else stringResource(R.string.voice_overlay_expand_cd),
@@ -855,10 +877,9 @@ private fun VoiceSessionPill(
.padding(top = 10.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
// Moved out of the collapsed header (4a): the current
// interaction-mode pill plus the inline mic control. The
// compact mic only appears in compact mode, where the
// full-size bottom mic button is hidden.
// The expanded body keeps the current interaction mode
// visible; Conversation's persistent mic stays in the
// collapsed header so it never disappears.
Row(
modifier = Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
@@ -866,15 +887,6 @@ private fun VoiceSessionPill(
) {
StatusPill(uiState.interactionMode.label())
Spacer(Modifier.weight(1f))
if (!focusMode) {
CompactVoiceMicButton(
uiState = uiState,
onMicTap = onMicTap,
onMicRelease = onMicRelease,
onInterrupt = onInterrupt,
onPauseAutoMode = onPauseAutoMode,
)
}
}
Row(
@@ -915,7 +927,11 @@ private fun VoiceSessionPill(
modifier = Modifier.weight(1f),
) {
Text(
if (focusMode) stringResource(R.string.voice_overlay_compact) else stringResource(R.string.voice_overlay_focus),
if (focusMode) {
stringResource(R.string.voice_overlay_conversation)
} else {
stringResource(R.string.voice_overlay_focus)
},
maxLines = 1,
overflow = TextOverflow.Ellipsis,
)
@@ -958,7 +974,7 @@ private fun VoiceSessionPill(
}
@Composable
private fun CompactVoiceMicButton(
private fun ConversationVoiceMicButton(
uiState: VoiceUiState,
onMicTap: () -> Unit,
onMicRelease: () -> Unit,
@@ -1224,6 +1240,9 @@ private fun CompactTranscriptRow(
message: ChatMessage,
showThinking: Boolean,
expanded: Boolean,
onViewConversation: () -> Unit,
onCardAction: (messageId: String, cardKey: String, action: HermesCardAction) -> Unit,
onCardInput: (messageId: String, cardKey: String, value: String) -> Unit,
) {
if (message.role == MessageRole.SYSTEM) return
@@ -1248,6 +1267,13 @@ private fun CompactTranscriptRow(
message.role == MessageRole.USER -> MaterialTheme.colorScheme.primary
else -> MaterialTheme.colorScheme.secondary
}
val (markdownBody, inlineImages) = remember(message.content, message.role) {
if (message.role == MessageRole.ASSISTANT) {
extractChatInlineImages(message.content)
} else {
message.content to emptyList()
}
}
Column(
modifier = Modifier.fillMaxWidth(),
@@ -1285,7 +1311,7 @@ private fun CompactTranscriptRow(
}
when {
isVoiceActionBubble && hasText -> MarkdownContent(
content = message.content,
content = markdownBody,
textColor = MaterialTheme.colorScheme.onSurface,
)
message.role == MessageRole.USER && hasText -> Text(
@@ -1295,12 +1321,107 @@ private fun CompactTranscriptRow(
maxLines = if (expanded) Int.MAX_VALUE else 3,
overflow = TextOverflow.Ellipsis,
)
hasText -> Text(
text = message.content,
style = MaterialTheme.typography.bodyMedium,
hasText -> MarkdownContent(
content = markdownBody,
textColor = MaterialTheme.colorScheme.onSurface,
)
}
message.cards.forEachIndexed { index, card ->
if (card.actions.isNotEmpty() || card.input != null) {
Spacer(Modifier.height(6.dp))
val cardKey = card.id ?: "idx:$index"
HermesCardBubble(
card = card,
cardKey = cardKey,
dispatches = message.cardDispatches,
onActionTap = { key, action ->
onCardAction(message.id, key, action)
},
onInputSubmit = { key, value ->
onCardInput(message.id, key, value)
},
maxWidth = 360.dp,
)
}
}
if (
message.attachments.isNotEmpty() ||
message.cards.any { it.actions.isEmpty() && it.input == null } ||
inlineImages.isNotEmpty()
) {
Spacer(Modifier.height(6.dp))
VoiceRichResultAffordance(
message = message,
onViewConversation = onViewConversation,
)
}
}
}
@Composable
private fun VoiceRichResultAffordance(
message: ChatMessage,
onViewConversation: () -> Unit,
) {
val inlineImages = remember(message.content) {
extractChatInlineImages(message.content).second
}
val previewModel = remember(message.attachments, inlineImages) {
message.attachments.firstOrNull { it.isImage && !it.cachedUri.isNullOrBlank() }?.cachedUri
?: inlineImages.firstOrNull {
it.src.startsWith("https://") || it.src.startsWith("http://")
}?.src
}
val label = when {
message.attachments.size + inlineImages.size > 1 ->
stringResource(
R.string.voice_overlay_rich_results_count,
message.attachments.size + inlineImages.size,
)
message.attachments.isNotEmpty() || inlineImages.isNotEmpty() ->
stringResource(R.string.voice_overlay_image_ready)
else -> stringResource(R.string.voice_overlay_rich_result_ready)
}
Surface(
modifier = Modifier
.fillMaxWidth()
.clickable(onClick = onViewConversation),
shape = RoundedCornerShape(10.dp),
color = MaterialTheme.colorScheme.primaryContainer.copy(alpha = 0.48f),
) {
Row(
modifier = Modifier.padding(horizontal = 10.dp, vertical = 8.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
if (previewModel != null) {
AsyncImage(
model = previewModel,
contentDescription = null,
contentScale = ContentScale.Crop,
modifier = Modifier
.size(40.dp)
.clip(RoundedCornerShape(8.dp)),
)
} else {
Icon(
imageVector = Icons.Filled.Image,
contentDescription = null,
tint = MaterialTheme.colorScheme.primary,
modifier = Modifier.size(20.dp),
)
}
Text(
text = label,
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onSurface,
maxLines = if (expanded) Int.MAX_VALUE else 6,
overflow = TextOverflow.Ellipsis,
modifier = Modifier.weight(1f),
)
Text(
text = stringResource(R.string.voice_overlay_view_conversation),
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.primary,
)
}
}
@@ -156,7 +156,9 @@ import com.hermesandroid.relay.data.AgentDisplay
import com.hermesandroid.relay.data.Attachment
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.Connection
import com.hermesandroid.relay.data.HermesCardAction
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.data.VoicePresentationMode
import com.hermesandroid.relay.data.hermesProcessNotificationOrNull
import com.hermesandroid.relay.ui.components.AgentInfoSheet
import com.hermesandroid.relay.ui.components.BackgroundTaskCard
@@ -280,6 +282,17 @@ internal fun ChatScrollSnapshot.isCompletionAfter(previous: ChatScrollSnapshot?)
previous.messageCount == messageCount &&
previous.lastMessageUiKey == lastMessageUiKey
internal fun releaseRetainedLiveTail(
retainedUiKey: String?,
completedUiKey: String?,
): String? = retainedUiKey?.takeUnless { it == completedUiKey }
internal fun tailEndScrollOffset(
tailSizePx: Int,
footerSizePx: Int,
viewportSizePx: Int,
): Int = (tailSizePx + footerSizePx - viewportSizePx).coerceAtLeast(0)
private class ChatTailTransitionRef(
var snapshot: ChatScrollSnapshot? = null,
)
@@ -439,6 +452,8 @@ fun ChatScreen(
voiceViewModel: VoiceViewModel,
voiceClient: RelayVoiceClient? = null,
maxBubbleWidth: Dp = 300.dp,
voicePresentationMode: VoicePresentationMode = VoicePresentationMode.Focus,
onVoicePresentationModeChange: (VoicePresentationMode) -> Unit = {},
// Deep-link nudge from Settings → Active Agent card: when `true`, the
// AgentInfoSheet auto-opens on first composition and [onAgentSheetArgConsumed]
// fires so the host can clear the nav arg (prevents re-open on tab
@@ -465,9 +480,10 @@ fun ChatScreen(
) {
val voiceUiState by voiceViewModel.uiState.collectAsState()
val isDemoMode by connectionViewModel.isDemoMode.collectAsState()
var voiceCompactMode by remember { mutableStateOf(false) }
val chatAlpha by animateFloatAsState(
targetValue = if (voiceUiState.voiceMode && !voiceCompactMode) 0.4f else 1f,
targetValue = if (
voiceUiState.voiceMode && voicePresentationMode == VoicePresentationMode.Focus
) 0.4f else 1f,
animationSpec = tween(300),
label = "chatAlpha",
)
@@ -788,6 +804,25 @@ fun ChatScreen(
val clipboard = LocalClipboard.current
val haptic = LocalHapticFeedback.current
val snackbarHostState = remember { SnackbarHostState() }
val handleCardAction: (String, String, HermesCardAction) -> Unit =
remember(chatViewModel, context) {
{ messageId, cardKey, action ->
if (action.mode == HermesCardAction.Modes.OPEN_URL) {
chatViewModel.dispatchCardAction(messageId, cardKey, action)
com.hermesandroid.relay.ui.components.handleCardActionExternally(
context,
action,
)
} else {
chatViewModel.dispatchCardAction(messageId, cardKey, action)
}
}
}
val handleCardInput: (String, String, String) -> Unit = remember(chatViewModel) {
{ messageId, cardKey, value ->
chatViewModel.answerAsk(messageId, cardKey, value)
}
}
// Ephemeral notices from the VM (model-switch warnings/errors, etc.) →
// transient snackbar, never a chat bubble.
@@ -885,7 +920,6 @@ fun ChatScreen(
if (!voiceUiState.voiceMode) {
voiceOverlayHost.hide()
pendingVoiceOverlayPermission = false
voiceCompactMode = false
}
}
@@ -1130,10 +1164,12 @@ fun ChatScreen(
derivedStateOf {
val retainingVisibleTail = retainedLiveTailUiKey != null &&
messages.lastOrNull()?.uiKey == retainedLiveTailUiKey
val settlingVisibleTail = completionSettlingUiKey != null &&
messages.lastOrNull()?.uiKey == completionSettlingUiKey
messages.isNotEmpty() &&
!isAtBottom &&
!programmaticBottomScroll &&
!((isStreaming || retainingVisibleTail) &&
!((isStreaming || retainingVisibleTail || settlingVisibleTail) &&
smoothAutoScroll &&
!userScrolledAway)
}
@@ -1331,34 +1367,99 @@ fun ChatScreen(
) {
val settlingKey = completionSettlingUiKey ?: return@LaunchedEffect
if (!smoothAutoScroll || userScrolledAway || isUserDragging) {
// Retention is only a completion-transition aid. Never leave the
// finalized tail on the plain streaming renderer just because the
// user disabled follow-scroll or is reading above the bottom.
retainedLiveTailUiKey = releaseRetainedLiveTail(
retainedUiKey = retainedLiveTailUiKey,
completedUiKey = settlingKey,
)
completionSettlingUiKey = null
return@LaunchedEffect
}
var settledFrames = 0
repeat(6) {
var previousMarkdownTailSize: Int? = null
var previousMarkdownFooterSize: Int? = null
val markdownWasAlreadyReleased = retainedLiveTailUiKey != settlingKey
repeat(60) completionFrame@{
withFrameNanos { }
if (messages.lastOrNull()?.uiKey != settlingKey) {
completionSettlingUiKey = null
return@LaunchedEffect
}
if (listState.canScrollForward) {
settledFrames = 0
val viewportHeight = listState.layoutInfo.viewportSize.height
if (viewportHeight > 0) {
listState.scroll(MutatePriority.Default) {
scrollBy(viewportHeight.toFloat())
if (!markdownWasAlreadyReleased && retainedLiveTailUiKey == settlingKey) {
if (listState.canScrollForward) {
settledFrames = 0
val viewportHeight = listState.layoutInfo.viewportSize.height
if (viewportHeight > 0) {
listState.scroll(MutatePriority.Default) {
scrollBy(viewportHeight.toFloat())
}
}
return@completionFrame
}
} else {
settledFrames += 1
if (settledFrames >= 2) {
completionSettlingUiKey = null
return@LaunchedEffect
}
if (settledFrames < 2) return@completionFrame
retainedLiveTailUiKey = releaseRetainedLiveTail(
retainedUiKey = retainedLiveTailUiKey,
completedUiKey = settlingKey,
)
settledFrames = 0
return@completionFrame
}
// Once Markdown owns the row, position its measured trailing edge
// explicitly. `canScrollForward` is insufficient here: LazyColumn
// may preserve the leading edge of a tall item while reporting an
// otherwise valid item anchor. Repeating catches deferred parsing,
// highlighted code, and attachment measurement without competing
// with the ordinary streaming-growth coroutine.
val layout = listState.layoutInfo
val tailIndex = messages.size // header item + zero-based messages
val footerIndex = tailIndex + 1
val tailInfo = layout.visibleItemsInfo.firstOrNull { it.index == tailIndex }
val footerInfo = layout.visibleItemsInfo.firstOrNull { it.index == footerIndex }
if (tailInfo == null) {
listState.scrollToItem(tailIndex)
settledFrames = 0
return@completionFrame
}
val viewportHeight = layout.viewportSize.height
if (viewportHeight <= 0) return@completionFrame
val desiredOffset = tailEndScrollOffset(
tailSizePx = tailInfo.size,
footerSizePx = footerInfo?.size ?: 0,
viewportSizePx = viewportHeight,
)
if (desiredOffset == 0) {
listState.scrollToItem(footerIndex)
} else {
listState.scrollToItem(tailIndex, desiredOffset)
}
val footerSize = footerInfo?.size ?: 0
settledFrames = if (
previousMarkdownTailSize == tailInfo.size &&
previousMarkdownFooterSize == footerSize
) {
settledFrames + 1
} else {
0
}
previousMarkdownTailSize = tailInfo.size
previousMarkdownFooterSize = footerSize
if (settledFrames >= 12) {
completionSettlingUiKey = null
return@LaunchedEffect
}
}
retainedLiveTailUiKey = releaseRetainedLiveTail(
retainedUiKey = retainedLiveTailUiKey,
completedUiKey = settlingKey,
)
completionSettlingUiKey = null
}
@@ -2308,24 +2409,8 @@ fun ChatScreen(
onAttachmentManualFetch = { msgId, idx ->
chatViewModel.manualFetchAttachment(msgId, idx)
},
onCardAction = { msgId, cardKey, action ->
// OPEN_URL is resolved at the UI layer
// because launching ACTION_VIEW needs a
// Context. Record the dispatch first so
// the card collapses even if launch fails.
if (action.mode == com.hermesandroid.relay.data.HermesCardAction.Modes.OPEN_URL) {
chatViewModel.dispatchCardAction(msgId, cardKey, action)
com.hermesandroid.relay.ui.components.handleCardActionExternally(
context,
action,
)
} else {
chatViewModel.dispatchCardAction(msgId, cardKey, action)
}
},
onCardInput = { msgId, cardKey, value ->
chatViewModel.answerAsk(msgId, cardKey, value)
},
onCardAction = handleCardAction,
onCardInput = handleCardInput,
onEditMessage = if (
isGatewayTransport &&
!isStreaming &&
@@ -3184,14 +3269,13 @@ fun ChatScreen(
voiceConfigScope = activeVoiceScope,
voiceOutputEnabled = activeVoiceEnabled,
voiceOutputFallbackEnabled = voiceOutputConfig?.fallback_enabled,
presentationMode = voicePresentationMode,
onPresentationModeChange = onVoicePresentationModeChange,
onOverlayRequest = showVoiceSystemOverlay,
// Gear button in the overlay's expanded controls. The overlay
// exits voice mode before invoking this, so navigation lands
// on Voice Settings with no overlay left on top.
onOpenSettings = onNavigateToVoiceSettings,
onCompactModeChange = { compact ->
voiceCompactMode = compact
},
// === v0.4.1 JIT permission-denied chip ===
// Tap deep-links to Settings → Apps → Hermes-Relay →
// Permissions for the running package. Use BuildConfig
@@ -3215,6 +3299,8 @@ fun ChatScreen(
onHermesConfirmationAnswer = { answer ->
voiceViewModel.answerHermesConfirmation(answer)
},
onCardAction = handleCardAction,
onCardInput = handleCardInput,
// === END v0.4.1 ===
)
}
@@ -27,7 +27,6 @@ import androidx.compose.material3.Scaffold
import androidx.compose.material3.Text
import androidx.compose.material3.TopAppBar
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
@@ -38,6 +37,7 @@ import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.Alignment
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.platform.LocalResources
import androidx.compose.ui.res.stringResource
import androidx.compose.ui.text.input.PasswordVisualTransformation
import androidx.compose.ui.unit.dp
@@ -51,17 +51,10 @@ import com.hermesandroid.relay.network.upstream.DashboardAuthProvider
import com.hermesandroid.relay.network.upstream.DashboardAuthSession
import com.hermesandroid.relay.network.upstream.DashboardCookieStore
import com.hermesandroid.relay.network.upstream.EncryptedDashboardCookieStore
import com.hermesandroid.relay.network.upstream.DashboardRedirectAuthMode
import com.hermesandroid.relay.network.upstream.NativeDashboardSignInCoordinator
import com.hermesandroid.relay.network.upstream.dashboardRedirectAuthMode
import com.hermesandroid.relay.network.upstream.importDashboardCookieHeader
import com.hermesandroid.relay.network.upstream.isNativeDashboardTransportEligible
import com.hermesandroid.relay.viewmodel.ConnectionViewModel
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.Job
import kotlinx.coroutines.launch
import kotlinx.coroutines.withContext
import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
/**
* Connection-level Dashboard authentication flow. It is deliberately outside
@@ -76,6 +69,7 @@ fun DashboardSignInScreen(
onAuthenticated: () -> Unit,
) {
val context = LocalContext.current
val resources = LocalResources.current
val appContext = context.applicationContext
val scope = rememberCoroutineScope()
val activeConnection by connectionViewModel.activeConnection.collectAsState()
@@ -90,10 +84,6 @@ fun DashboardSignInScreen(
var actionMessage by remember { mutableStateOf<String?>(null) }
var actionIsError by remember { mutableStateOf(false) }
var oauthProvider by remember { mutableStateOf<DashboardAuthProvider?>(null) }
var redirectAuthMode by remember(dashboardUrl, connectionId) {
mutableStateOf(DashboardRedirectAuthMode.WebView)
}
var nativeSignInJob by remember(dashboardUrl, connectionId) { mutableStateOf<Job?>(null) }
var authenticationComplete by remember { mutableStateOf(false) }
val cookieStoreFactory = remember(appContext, connectionId) {
@@ -135,20 +125,19 @@ fun DashboardSignInScreen(
LaunchedEffect(dashboardUrl, connectionId) {
if (dashboardUrl.isBlank()) {
loading = false
actionMessage = context.getString(R.string.dashboard_no_url_configured)
actionMessage = resources.getString(R.string.dashboard_no_url_configured)
return@LaunchedEffect
}
val client = clientFactory()
try {
val status = client.getStatus().getOrElse {
actionMessage = it.message ?: context.getString(R.string.dashboard_request_failed)
actionMessage = it.message ?: resources.getString(R.string.dashboard_request_failed)
actionIsError = true
return@LaunchedEffect
}
providers = client.getAuthProviders().getOrNull()
?.takeIf { it.isNotEmpty() }
?: status.authProviderDetails
redirectAuthMode = dashboardRedirectAuthMode(status.authFlows)
val session = if (status.authRequired) client.currentSession().getOrNull() else null
connectionViewModel.recordDashboardStatus(
status = status,
@@ -179,11 +168,11 @@ fun DashboardSignInScreen(
finishAuthentication()
} else {
actionMessage = result.exceptionOrNull()?.message
?: context.getString(R.string.dashboard_signin_no_session)
?: resources.getString(R.string.dashboard_signin_no_session)
actionIsError = true
}
} catch (e: Exception) {
actionMessage = e.message ?: context.getString(R.string.dashboard_signin_failed)
actionMessage = e.message ?: resources.getString(R.string.dashboard_signin_failed)
actionIsError = true
} finally {
actionInFlight = false
@@ -194,68 +183,13 @@ fun DashboardSignInScreen(
fun startRedirectSignIn(provider: DashboardAuthProvider) {
if (actionInFlight || dashboardUrl.isBlank()) return
if (redirectAuthMode == DashboardRedirectAuthMode.WebView) {
oauthProvider = provider
return
}
if (!isNativeDashboardTransportEligible(dashboardUrl)) {
actionMessage = context.getString(R.string.dashboard_native_signin_requires_https)
actionIsError = true
return
}
val authClient = connectionViewModel.nativeDashboardAuthClientForActive(dashboardUrl)
if (authClient == null) {
actionMessage = context.getString(R.string.dashboard_native_signin_unavailable)
actionIsError = true
return
}
actionInFlight = true
actionIsError = false
actionMessage = context.getString(R.string.dashboard_native_signin_opening)
nativeSignInJob = scope.launch {
try {
NativeDashboardSignInCoordinator(authClient).signIn(provider.name) { authorizationUrl ->
withContext(Dispatchers.Main.immediate) {
launchNativeDashboardAuthorization(context, authorizationUrl)
}
}
val client = clientFactory()
val session = try {
verifyAndRecord(client)
} finally {
client.shutdown()
}
if (session?.authenticated == true) {
actionMessage = session.provider?.let {
context.getString(R.string.dashboard_signed_in_with, it)
} ?: context.getString(R.string.dashboard_signed_in)
actionIsError = false
finishAuthentication()
} else {
actionMessage = context.getString(R.string.dashboard_signin_no_session)
actionIsError = true
}
} catch (cancelled: CancellationException) {
throw cancelled
} catch (error: Exception) {
actionMessage = error.message
?: context.getString(R.string.dashboard_signin_failed)
actionIsError = true
} finally {
actionInFlight = false
nativeSignInJob = null
}
}
// `native_pkce` is an upstream desktop capability. Android owns the
// dashboard cookie flow so the provider returns to the public
// /auth/callback, whose cookies are then verified via /api/auth/me.
oauthProvider = provider
}
DisposableEffect(dashboardUrl, connectionId) {
onDispose { nativeSignInJob?.cancel() }
}
oauthProvider
?.takeIf { redirectAuthMode == DashboardRedirectAuthMode.WebView }
?.let { provider ->
oauthProvider?.let { provider ->
DashboardOAuthDialog(
dashboardUrl = dashboardUrl,
provider = provider,
@@ -272,8 +206,8 @@ fun DashboardSignInScreen(
client.shutdown()
}
actionMessage = session.provider?.let {
context.getString(R.string.dashboard_signed_in_with, it)
} ?: context.getString(R.string.dashboard_signed_in)
resources.getString(R.string.dashboard_signed_in_with, it)
} ?: resources.getString(R.string.dashboard_signed_in)
finishAuthentication()
}
},
@@ -289,10 +223,7 @@ fun DashboardSignInScreen(
TopAppBar(
title = { Text(stringResource(R.string.dashboard_sign_in)) },
navigationIcon = {
IconButton(onClick = {
nativeSignInJob?.cancel()
onBack()
}) {
IconButton(onClick = onBack) {
Icon(
Icons.AutoMirrored.Filled.ArrowBack,
contentDescription = stringResource(R.string.dashboard_back),
@@ -321,16 +252,8 @@ fun DashboardSignInScreen(
actionInFlight = actionInFlight,
actionMessage = actionMessage,
actionIsError = actionIsError,
nativePkce = redirectAuthMode == DashboardRedirectAuthMode.NativePkce,
nativeSignInInFlight = nativeSignInJob != null,
nativeTransportEligible = isNativeDashboardTransportEligible(dashboardUrl),
onSignIn = ::submitPassword,
onOAuthSignIn = ::startRedirectSignIn,
onCancelNativeSignIn = {
actionMessage = context.getString(R.string.dashboard_native_signin_cancelled)
actionIsError = false
nativeSignInJob?.cancel()
},
)
}
}
@@ -396,12 +319,8 @@ private fun DashboardSignInForm(
actionInFlight: Boolean,
actionMessage: String?,
actionIsError: Boolean,
nativePkce: Boolean,
nativeSignInInFlight: Boolean,
nativeTransportEligible: Boolean,
onSignIn: (String, String, String) -> Unit,
onOAuthSignIn: (DashboardAuthProvider) -> Unit,
onCancelNativeSignIn: () -> Unit,
) {
var username by remember { mutableStateOf("") }
var password by remember { mutableStateOf("") }
@@ -427,19 +346,12 @@ private fun DashboardSignInForm(
redirectProviders.forEach { provider ->
Button(
onClick = { onOAuthSignIn(provider) },
enabled = !actionInFlight && (!nativePkce || nativeTransportEligible),
enabled = !actionInFlight,
modifier = Modifier.fillMaxWidth(),
) {
Text(stringResource(R.string.dashboard_signin_with_provider, provider.displayName ?: provider.name))
}
}
if (nativePkce && !nativeTransportEligible && redirectProviders.isNotEmpty()) {
Text(
text = stringResource(R.string.dashboard_native_signin_requires_https),
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.error,
)
}
if (passwordProvider != null || providers.isEmpty()) {
if (redirectProviders.isNotEmpty()) HorizontalDivider()
OutlinedTextField(
@@ -476,14 +388,6 @@ private fun DashboardSignInForm(
},
)
}
if (nativeSignInInFlight) {
Button(
onClick = onCancelNativeSignIn,
modifier = Modifier.fillMaxWidth(),
) {
Text(stringResource(R.string.dashboard_cancel))
}
}
}
@Composable
@@ -512,14 +416,17 @@ private fun DashboardOAuthDialog(
)
}
fun maybeVerify(url: String?) {
fun handleNavigation(url: String?) {
val loadedUrl = url?.takeIf { it.isNotBlank() } ?: return
val root = dashboardUrl.trim().trimEnd('/')
val relative = loadedUrl.trim().removePrefix(root)
val stillAuthenticating = relative.startsWith("/login", true) ||
relative.startsWith("/auth/login", true) ||
relative.startsWith("/auth/callback", true)
if (!loadedUrl.startsWith(root, true) || stillAuthenticating) return
when (dashboardWebViewAuthNavigation(dashboardUrl, loadedUrl)) {
DashboardWebViewAuthNavigation.Continue -> return
DashboardWebViewAuthNavigation.RejectLoopbackCallback -> {
statusText = notAcceptedStatus
onError(notAcceptedStatus)
return
}
DashboardWebViewAuthNavigation.ImportAndVerify -> Unit
}
val manager = CookieManager.getInstance()
manager.flush()
val imported = importDashboardCookieHeader(
@@ -570,11 +477,21 @@ private fun DashboardOAuthDialog(
override fun shouldOverrideUrlLoading(
view: WebView,
request: WebResourceRequest,
): Boolean = false
): Boolean {
val target = request.url.toString()
if (
dashboardWebViewAuthNavigation(dashboardUrl, target) ==
DashboardWebViewAuthNavigation.RejectLoopbackCallback
) {
handleNavigation(target)
return true
}
return false
}
override fun onPageFinished(view: WebView, url: String?) {
super.onPageFinished(view, url)
maybeVerify(url)
handleNavigation(url)
}
}
loadUrl(loginUrl)
@@ -585,3 +502,52 @@ private fun DashboardOAuthDialog(
}
}
}
internal enum class DashboardWebViewAuthNavigation {
Continue,
ImportAndVerify,
RejectLoopbackCallback,
}
/**
* Android redirect providers use the dashboard's cookie/OIDC flow. A foreign
* loopback callback belongs to the desktop native-PKCE contract and must never
* be followed, imported, or treated as an authenticated Android return.
*/
internal fun dashboardWebViewAuthNavigation(
dashboardUrl: String,
loadedUrl: String,
): DashboardWebViewAuthNavigation {
val dashboard = dashboardUrl.trim().trimEnd('/').toHttpUrlOrNull()
?: return DashboardWebViewAuthNavigation.Continue
val loaded = loadedUrl.trim().toHttpUrlOrNull()
?: return DashboardWebViewAuthNavigation.Continue
val sameOrigin = dashboard.scheme == loaded.scheme &&
dashboard.host.equals(loaded.host, ignoreCase = true) &&
dashboard.port == loaded.port
if (!sameOrigin) {
val foreignLoopback = loaded.scheme == "http" &&
loaded.host in setOf("127.0.0.1", "localhost", "::1") &&
loaded.encodedPath == "/callback"
return if (foreignLoopback) {
DashboardWebViewAuthNavigation.RejectLoopbackCallback
} else {
DashboardWebViewAuthNavigation.Continue
}
}
val basePath = dashboard.encodedPath.trimEnd('/')
val relativePath = loaded.encodedPath
.removePrefix(basePath)
.ifBlank { "/" }
return if (
relativePath.equals("/login", ignoreCase = true) ||
relativePath.equals("/auth/login", ignoreCase = true)
) {
DashboardWebViewAuthNavigation.Continue
} else {
// Includes the public /auth/callback response: import its cookies at
// root scope, then verify the resulting session through /api/auth/me.
DashboardWebViewAuthNavigation.ImportAndVerify
}
}
@@ -3224,6 +3224,17 @@ private fun GlobalVoiceControlsCard(
HorizontalDivider(modifier = Modifier.padding(vertical = 8.dp))
SettingSwitchRow(
title = stringResource(R.string.voice_settings_final_answer_only),
detail = stringResource(R.string.voice_settings_final_answer_only_desc),
checked = voiceSettings.finalAnswerOnly,
onCheckedChange = { enabled ->
scope.launch { prefsRepo.setFinalAnswerOnly(enabled) }
},
)
HorizontalDivider(modifier = Modifier.padding(vertical = 8.dp))
Text(
text = stringResource(R.string.voice_settings_interaction_mode),
style = MaterialTheme.typography.labelLarge,
@@ -5922,6 +5922,13 @@ class ChatViewModel : ViewModel() {
queuedCount = event.queuedCount ?: 0,
),
)
if (event.spokenHandoff == false) {
// Silent promotion is the foreground turn boundary. The
// background run keeps its owner/card and continues to
// receive progress, cancellation, and delivery events.
handler.onStreamComplete(assistantMessageId)
activeStream = null
}
}
"hermes.run.queued" -> {
handler.updateBackgroundTask(assistantMessageId) { task ->
@@ -111,7 +111,16 @@ internal data class AssistantSpeechBatch(
val assistantMessages: List<ChatMessage>,
val aggregateText: String,
val hasTurnAssistant: Boolean,
)
) {
/** Last non-empty assistant bubble: the settled answer after any tool commentary. */
val finalAnswerText: String
get() = assistantMessages
.asReversed()
.firstOrNull { it.content.isNotBlank() }
?.content
?.trim()
.orEmpty()
}
/**
* Per-voice-turn cursor over every assistant bubble created after the user
@@ -323,9 +332,8 @@ enum class BackgroundRunPhase {
DONE,
}
internal fun realtimeTurnActiveAfterResponseDone(backgroundPhase: BackgroundRunPhase?): Boolean =
backgroundPhase == BackgroundRunPhase.RUNNING ||
backgroundPhase == BackgroundRunPhase.RECONNECTING
internal fun realtimeTurnActiveAfterPromotion(spokenHandoff: Boolean?): Boolean =
spokenHandoff != false
internal fun preserveRealtimeTurnOnStop(backgroundPhase: BackgroundRunPhase?): Boolean =
backgroundPhase == BackgroundRunPhase.RUNNING ||
@@ -648,6 +656,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
private var voicePreferences: VoicePreferencesRepository? = null
private var voicePreferencesJob: Job? = null
private var voiceEngineMode: VoiceEngineMode = VoiceEngineMode.HermesVoiceOutput
private var finalAnswerOnly: Boolean = false
private var realtimeTraceDetails: Boolean = false
private var realtimePersistentSession: Boolean = true
private var realtimeModel: String = ""
@@ -1368,10 +1377,12 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
*/
private fun applyVoiceSettingsSnapshot(settings: com.hermesandroid.relay.data.VoiceSettings) {
val nextEngineMode = VoiceEngineMode.fromStorage(settings.engineMode)
val finalAnswerPolicyChanged = finalAnswerOnly != settings.finalAnswerOnly
val realtimeSelectionChanged =
realtimeModel != settings.realtimeModel || realtimeVoice != settings.realtimeVoice
if (
voiceEngineMode != nextEngineMode ||
finalAnswerPolicyChanged ||
realtimeTraceDetails != settings.realtimeTraceDetails ||
realtimeSelectionChanged
) {
@@ -1379,6 +1390,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
TAG,
"Voice prefs updated engine=${nextEngineMode.storageValue} " +
"interaction=${settings.interactionMode} " +
"finalAnswerOnly=${settings.finalAnswerOnly} " +
"realtimeTraceDetails=${settings.realtimeTraceDetails} " +
"realtimeModel=${settings.realtimeModel.ifBlank { "relay-default" }} " +
"realtimeVoice=${settings.realtimeVoice.ifBlank { "relay-default" }}",
@@ -1390,11 +1402,13 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
(voiceEngineMode == VoiceEngineMode.RealtimeAgent &&
nextEngineMode != VoiceEngineMode.RealtimeAgent) ||
(realtimePersistentSession && !settings.realtimePersistentSession) ||
finalAnswerPolicyChanged ||
realtimeSelectionChanged
) {
closeRealtimeSession()
}
voiceEngineMode = nextEngineMode
finalAnswerOnly = settings.finalAnswerOnly
realtimeTraceDetails = settings.realtimeTraceDetails
realtimePersistentSession = settings.realtimePersistentSession
realtimeModel = settings.realtimeModel
@@ -3162,17 +3176,21 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
resumeWatchdog?.cancel(); resumeWatchdog = null
clearSpokenChunksState()
prepareStandardSpeechStream()
// Kick off streaming observer BEFORE sending the message so we don't
// miss early deltas that arrive synchronously from the callback.
startStreamObserver(chatVm)
if (!finalAnswerOnly) {
prepareStandardSpeechStream()
}
// Route the transcribed text through the normal chat pipeline.
// This will create a user message + kick off the SSE stream.
// This creates the user row synchronously before kicking off the
// transport, so bind the turn before observing the replaying StateFlows.
// Starting the observer first leaves a small window where a legitimate
// session adoption can be rejected before the submitted user key exists.
// StateFlow replay preserves any assistant text that arrives before the
// observer starts.
val submittedUserUiKey =
chatVm.sendVoiceMessage(userText, STABLE_VOICE_INTERFACE_CONTEXT)
voiceTurnSessionFence?.bindSubmittedUser(submittedUserUiKey)
startStreamObserver(chatVm)
}
private suspend fun runVoiceRelayPreflight(engineLabel: String): Boolean {
@@ -3299,7 +3317,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
)
}
}
if (speak && (!audioSeen.get() || speakEvenAfterProviderAudio)) {
if (speak && !finalAnswerOnly && (!audioSeen.get() || speakEvenAfterProviderAudio)) {
// W3: per-turn throttle independent of the per-key dedupe above.
// Suppress the TTS enqueue (UI state + diagnostics already
// applied) when spoken status is too frequent or has hit the
@@ -3371,6 +3389,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
conversationContext = conversationContext,
model = realtimeModel,
voice = realtimeVoice,
finalAnswerOnly = finalAnswerOnly,
onHandoff = { event -> recordRealtimeVoiceHandoff(sessionGeneration, event) },
turnInputs = if (persistentOpen) realtimeTurnChannel else null,
onTurnComplete = { summary ->
@@ -3415,9 +3434,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
}
if (suppressCommandResponse) {
if (event.type == "voice.response.done") {
providerRealtimeAgentTurnActive.set(
realtimeTurnActiveAfterResponseDone(_uiState.value.backgroundRun?.phase),
)
providerRealtimeAgentTurnActive.set(false)
suppressLocalCommandResponse = false
realtimeAudioSuppressed = false
}
@@ -3638,10 +3655,12 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
}
}
"hermes.run.promoted" -> {
providerRealtimeAgentTurnActive.set(true)
// The run detached to the background; the provider speaks the
// handoff. Surface a persistent chip so the user knows a long
// task is still in flight (ADR 33 Tier B/C).
providerRealtimeAgentTurnActive.set(
realtimeTurnActiveAfterPromotion(event.spokenHandoff),
)
// The run detached to the background. A spoken handoff keeps
// the foreground turn active until response.done; a silent
// handoff ends it here. The task chip remains either way.
val tier = event.tier ?: "promoted"
Log.i(
TAG,
@@ -3845,9 +3864,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
hermesConfirmation = null,
)
}
providerRealtimeAgentTurnActive.set(
realtimeTurnActiveAfterResponseDone(_uiState.value.backgroundRun?.phase)
)
providerRealtimeAgentTurnActive.set(false)
}
"voice.error" -> {
realtimeConfirmationControl = null
@@ -4328,26 +4345,42 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
) { messages, runActive, sessionId -> Triple(messages, runActive, sessionId) }
.collect { (messages, runActive, sessionId) ->
if (!sessionFence.accepts(sessionId, messages)) {
cancelStandardSpeechStream("chat session changed")
streamObserverJob?.cancel()
// Session id and message history are independent flows.
// During session creation/adoption, combine can briefly
// pair the new id with the old history (or vice versa).
// Skip that inconsistent snapshot without permanently
// killing narration; the next coherent emission is still
// fenced by the submitted user row/session identity.
return@collect
}
val batch = cursor.poll(messages)
batch.deltas.forEach { update ->
if (update.startsNewBubble) {
beginAssistantSpeechBubble()
if (finalAnswerOnly) {
if (batch.deltas.isNotEmpty()) {
onVisualStreamDelta(batch.aggregateText)
}
} else {
batch.deltas.forEach { update ->
if (update.startsNewBubble) {
beginAssistantSpeechBubble()
}
onStreamDelta(update.text, batch.aggregateText)
}
onStreamDelta(update.text, batch.aggregateText)
}
// Tool state can change without text growth.
batch.assistantMessages.forEach(::observeHermesToolLoopForSpeech)
if (!finalAnswerOnly) {
batch.assistantMessages.forEach(::observeHermesToolLoopForSpeech)
}
if (!runActive && batch.hasTurnAssistant) {
streamComplete = true
idleFlushJob?.cancel()
idleFlushJob = null
if (!finishStandardSpeechStream()) flushRemainingBuffer()
if (finalAnswerOnly) {
speakSettledFinalAnswer(batch.finalAnswerText)
} else if (!finishStandardSpeechStream()) {
flushRemainingBuffer()
}
streamObserverJob?.cancel()
scheduleAgentAudioCompletionCheck()
}
@@ -4355,6 +4388,44 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
}
}
private fun onVisualStreamDelta(fullContent: String) {
_uiState.update {
it.copy(
state = VoiceState.Thinking,
outputAudioActive = false,
responseText = fullContent,
)
}
}
/**
* Final-only mode deliberately trades streaming latency for a clean spoken
* result. The last non-empty assistant bubble is the settled answer; earlier
* bubbles and tool states remain visible in Chat but never enter TTS.
*/
private fun speakSettledFinalAnswer(answer: String) {
val spoken = sanitizeForTts(answer)
if (spoken.isBlank()) return
_uiState.update {
it.copy(
state = VoiceState.Speaking,
outputAudioActive = false,
responseText = answer,
)
}
prepareStandardSpeechStream()
if (offerStandardSpeechText(spoken)) {
finishStandardSpeechStream()
} else {
sentenceBuffer = StringBuilder()
pendingRawDelta = StringBuilder()
appendSanitizedDelta(spoken)
flushRemainingBuffer()
}
}
private fun beginAssistantSpeechBubble() {
idleFlushJob?.cancel()
idleFlushJob = null
@@ -4393,7 +4464,7 @@ class VoiceViewModel(application: Application) : AndroidViewModel(application) {
* device through the normal tool loop.
*/
private fun observeHermesToolLoopForSpeech(message: ChatMessage) {
if (!_uiState.value.voiceMode || message.toolCalls.isEmpty()) return
if (finalAnswerOnly || !_uiState.value.voiceMode || message.toolCalls.isEmpty()) return
var spokenForMessage = brokeredToolSpeechCounts[message.id] ?: 0
message.toolCalls.forEach { tool ->
+7 -1
View File
@@ -1502,6 +1502,8 @@
<string name="voice_settings_save_realtime_agent">Salvar agente em tempo real</string>
<string name="voice_settings_global_controls_title">Controles globais de voz</string>
<string name="voice_settings_global_controls_desc">Estas configurações se aplicam aos dois mecanismos de voz em todos os perfis.</string>
<string name="voice_settings_final_answer_only">Somente a resposta final</string>
<string name="voice_settings_final_answer_only_desc">Fala apenas a resposta concluída. O progresso das ferramentas, as atualizações de serviço e os comentários intermediários permanecem visuais.</string>
<string name="voice_settings_interaction_mode">Modo de interação</string>
<string name="voice_settings_interaction_tap">Tocar para falar</string>
<string name="voice_settings_interaction_hold">Manter pressionado para falar</string>
@@ -1996,8 +1998,12 @@
<string name="voice_overlay_collapse_cd">Recolher controles de voz</string>
<string name="voice_overlay_expand_cd">Expandir controles de voz</string>
<string name="voice_overlay_exit_cd">Sair do modo de voz</string>
<string name="voice_overlay_compact">Compacto</string>
<string name="voice_overlay_focus">Foco</string>
<string name="voice_overlay_conversation">Conversa</string>
<string name="voice_overlay_image_ready">Imagem pronta</string>
<string name="voice_overlay_rich_result_ready">Resultado avançado pronto</string>
<string name="voice_overlay_rich_results_count">%1$d resultados prontos</string>
<string name="voice_overlay_view_conversation">Ver conversa</string>
<string name="voice_overlay_overlay">Sobreposição</string>
<string name="voice_overlay_exit">Sair</string>
<string name="voice_overlay_settings_cd">Configurações de voz</string>
@@ -1563,6 +1563,8 @@
<string name="voice_settings_save_realtime_agent">保存实时 Agent</string>
<string name="voice_settings_global_controls_title">全局语音控制</string>
<string name="voice_settings_global_controls_desc">这些设置适用于所有个人资料上的两个语音引擎。</string>
<string name="voice_settings_final_answer_only">仅朗读最终答案</string>
<string name="voice_settings_final_answer_only_desc">只朗读最终确定的答案。工具进度、服务更新和中间评论仍仅以视觉方式显示。</string>
<string name="voice_settings_interaction_mode">交互模式</string>
<string name="voice_settings_interaction_tap">点击说话</string>
<string name="voice_settings_interaction_hold">按住说话</string>
@@ -2088,8 +2090,12 @@
<string name="voice_overlay_collapse_cd">收起语音控制</string>
<string name="voice_overlay_expand_cd">展开语音控制</string>
<string name="voice_overlay_exit_cd">退出语音模式</string>
<string name="voice_overlay_compact">紧凑</string>
<string name="voice_overlay_focus">专注</string>
<string name="voice_overlay_conversation">对话</string>
<string name="voice_overlay_image_ready">图片已就绪</string>
<string name="voice_overlay_rich_result_ready">丰富结果已就绪</string>
<string name="voice_overlay_rich_results_count">%1$d 个结果已就绪</string>
<string name="voice_overlay_view_conversation">查看对话</string>
<string name="voice_overlay_overlay">浮窗</string>
<string name="voice_overlay_exit">退出</string>
<string name="voice_overlay_settings_cd">语音设置</string>
+7 -1
View File
@@ -1566,6 +1566,8 @@
<string name="voice_settings_save_realtime_agent">Echtzeit-Agent speichern</string>
<string name="voice_settings_global_controls_title">Globale Sprachsteuerung</string>
<string name="voice_settings_global_controls_desc">Diese Einstellungen gelten für beide Sprach-Engines in jedem Profil.</string>
<string name="voice_settings_final_answer_only">Nur endgültige Antwort</string>
<string name="voice_settings_final_answer_only_desc">Spricht nur die abgeschlossene Antwort. Werkzeugfortschritt, Dienstmeldungen und Zwischenkommentare bleiben visuell.</string>
<string name="voice_settings_interaction_mode">Interaktionsmodus</string>
<string name="voice_settings_interaction_tap">Tippen zum Sprechen</string>
<string name="voice_settings_interaction_hold">Halten zum Sprechen</string>
@@ -2091,8 +2093,12 @@
<string name="voice_overlay_collapse_cd">Sprachsteuerung einklappen</string>
<string name="voice_overlay_expand_cd">Sprachsteuerung ausklappen</string>
<string name="voice_overlay_exit_cd">Sprachmodus beenden</string>
<string name="voice_overlay_compact">Kompakt</string>
<string name="voice_overlay_focus">Fokus</string>
<string name="voice_overlay_conversation">Gespräch</string>
<string name="voice_overlay_image_ready">Bild bereit</string>
<string name="voice_overlay_rich_result_ready">Rich-Ergebnis bereit</string>
<string name="voice_overlay_rich_results_count">%1$d Ergebnisse bereit</string>
<string name="voice_overlay_view_conversation">Gespräch anzeigen</string>
<string name="voice_overlay_overlay">Overlay</string>
<string name="voice_overlay_exit">Beenden</string>
<string name="voice_overlay_settings_cd">Spracheinstellungen</string>
+7 -1
View File
@@ -1442,6 +1442,8 @@
<string name="voice_settings_save_realtime_agent">Guardar agente en tiempo real</string>
<string name="voice_settings_global_controls_title">Controles de voz globales</string>
<string name="voice_settings_global_controls_desc">Estas configuraciones se aplican a ambos motores de voz, en todos los perfiles.</string>
<string name="voice_settings_final_answer_only">Solo la respuesta final</string>
<string name="voice_settings_final_answer_only_desc">Reproduce únicamente la respuesta definitiva. El progreso de herramientas, las actualizaciones de servicio y los comentarios intermedios permanecen visuales.</string>
<string name="voice_settings_interaction_mode">Modo de interacción</string>
<string name="voice_settings_interaction_tap">Toca para hablar</string>
<string name="voice_settings_interaction_hold">Espera para hablar</string>
@@ -1903,8 +1905,12 @@
<string name="voice_overlay_collapse_cd">Contraer controles de voz</string>
<string name="voice_overlay_expand_cd">Ampliar los controles de voz</string>
<string name="voice_overlay_exit_cd">Salir del modo de voz</string>
<string name="voice_overlay_compact">Compacto</string>
<string name="voice_overlay_focus">Enfocar</string>
<string name="voice_overlay_conversation">Conversación</string>
<string name="voice_overlay_image_ready">Imagen lista</string>
<string name="voice_overlay_rich_result_ready">Resultado enriquecido listo</string>
<string name="voice_overlay_rich_results_count">%1$d resultados listos</string>
<string name="voice_overlay_view_conversation">Ver conversación</string>
<string name="voice_overlay_overlay">Cubrir</string>
<string name="voice_overlay_exit">Salida</string>
<string name="voice_overlay_settings_cd">Configuraciones de voz</string>
+7 -1
View File
@@ -1579,6 +1579,8 @@
<string name="voice_settings_save_realtime_agent">リアルタイムエージェントを保存する</string>
<string name="voice_settings_global_controls_title">グローバル音声コントロール</string>
<string name="voice_settings_global_controls_desc">これらの設定は、すべてのプロファイルの両方の音声エンジンに適用されます。</string>
<string name="voice_settings_final_answer_only">最終回答のみ</string>
<string name="voice_settings_final_answer_only_desc">確定した回答だけを読み上げます。ツールの進行状況、サービス更新、中間コメントは画面表示のみになります。</string>
<string name="voice_settings_interaction_mode">インタラクションモード</string>
<string name="voice_settings_interaction_tap">タップして話す</string>
<string name="voice_settings_interaction_hold">押し続けて話す</string>
@@ -2104,8 +2106,12 @@
<string name="voice_overlay_collapse_cd">音声コントロールを折りたたむ</string>
<string name="voice_overlay_expand_cd">音声コントロールを拡張する</string>
<string name="voice_overlay_exit_cd">音声モードを終了する</string>
<string name="voice_overlay_compact">コンパクト</string>
<string name="voice_overlay_focus">集中</string>
<string name="voice_overlay_conversation">会話</string>
<string name="voice_overlay_image_ready">画像の準備ができました</string>
<string name="voice_overlay_rich_result_ready">リッチな結果の準備ができました</string>
<string name="voice_overlay_rich_results_count">結果が%1$d件準備できました</string>
<string name="voice_overlay_view_conversation">会話を表示</string>
<string name="voice_overlay_overlay">かぶせる</string>
<string name="voice_overlay_exit">出口</string>
<string name="voice_overlay_settings_cd">音声設定</string>
+8 -2
View File
@@ -1682,6 +1682,8 @@
<string name="voice_settings_save_realtime_agent">Save realtime agent</string>
<string name="voice_settings_global_controls_title">Global Voice Controls</string>
<string name="voice_settings_global_controls_desc">These settings apply to both voice engines, on every profile.</string>
<string name="voice_settings_final_answer_only">Final answer only</string>
<string name="voice_settings_final_answer_only_desc">Speak only the settled answer. Tool progress, service updates, and intermediate commentary stay visual.</string>
<string name="voice_settings_interaction_mode">Interaction mode</string>
<string name="voice_settings_interaction_tap">Tap to talk</string>
<string name="voice_settings_interaction_hold">Hold to talk</string>
@@ -2211,8 +2213,12 @@
<string name="voice_overlay_collapse_cd">Collapse voice controls</string>
<string name="voice_overlay_expand_cd">Expand voice controls</string>
<string name="voice_overlay_exit_cd">Exit voice mode</string>
<string name="voice_overlay_compact">Compact</string>
<string name="voice_overlay_focus">Focus</string>
<string name="voice_overlay_conversation">Conversation</string>
<string name="voice_overlay_focus">Voice focus</string>
<string name="voice_overlay_image_ready">Image ready</string>
<string name="voice_overlay_rich_result_ready">Rich result ready</string>
<string name="voice_overlay_rich_results_count">%1$d results ready</string>
<string name="voice_overlay_view_conversation">View conversation</string>
<string name="voice_overlay_overlay">Overlay</string>
<string name="voice_overlay_exit">Exit</string>
<string name="voice_overlay_settings_cd">Voice settings</string>
@@ -11,6 +11,8 @@ import kotlinx.coroutines.flow.first
import kotlinx.coroutines.test.runTest
import org.junit.After
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertTrue
import org.junit.Before
import org.junit.Rule
import org.junit.Test
@@ -65,4 +67,44 @@ class VoicePreferencesRepositoryTest {
assertEquals("grok-voice-think-fast-1.0", settings.realtimeModel)
assertEquals("leo", settings.realtimeVoice)
}
@Test
fun finalAnswerOnlyPersistsGloballyAcrossProfileScopes() = runTest {
assertFalse(repository.settings.first().finalAnswerOnly)
repository.setFinalAnswerOnly(true)
assertTrue(repository.settings.first().finalAnswerOnly)
repository.setActiveScope("connection-a", "coder")
assertTrue(repository.settings.first().finalAnswerOnly)
repository.setActiveScope("connection-b", "writer")
assertTrue(repository.settings.first().finalAnswerOnly)
}
@Test
fun presentationModePersistsGloballyAcrossProfileScopes() = runTest {
assertEquals(
VoicePresentationMode.Focus.storageValue,
repository.settings.first().presentationMode,
)
repository.setPresentationMode(VoicePresentationMode.Conversation)
assertEquals(
VoicePresentationMode.Conversation.storageValue,
repository.settings.first().presentationMode,
)
repository.setActiveScope("connection-a", "coder")
assertEquals(
VoicePresentationMode.Conversation.storageValue,
repository.settings.first().presentationMode,
)
}
@Test
fun unknownPresentationModeFallsBackToFocus() {
assertEquals(VoicePresentationMode.Focus, VoicePresentationMode.fromStorage("unknown"))
assertEquals(VoicePresentationMode.Focus, VoicePresentationMode.fromStorage(null))
}
}
@@ -147,6 +147,28 @@ class RelayVoiceClientRoutingTest {
assertEquals("leo", payload["voice"]?.jsonPrimitive?.content)
}
@Test
fun realtimeAgentSessionSendsFinalAnswerOnlyPolicy() = runTest {
val client = RelayVoiceClient(
context = context,
okHttpClient = httpClient,
relayUrlProvider = { relayUrl(lanServer) },
sessionTokenProvider = { "session-token" },
)
val result = client.runRealtimeAgent(
prompt = "Check Hermes quietly",
inputPcm = ByteArray(0),
finalAnswerOnly = true,
) { _, _ -> }
assertTrue(result.exceptionOrNull()?.message, result.isSuccess)
val request = lanServer.takeRequest(2, TimeUnit.SECONDS)
?: error("missing realtime session request")
val payload = Json.parseToJsonElement(request.body.readUtf8()).jsonObject
assertEquals("true", payload["final_answer_only"]?.jsonPrimitive?.content)
}
@Test
fun voiceOutputSessionResponseParsesResumeMetadata() {
val response = Json.decodeFromString(
@@ -2102,6 +2124,81 @@ class RelayVoiceClientRoutingTest {
)
}
@Test
fun persistentRealtimePromotionUsesSpokenHandoffAsForegroundBoundary() = runBlocking {
val opened = CountDownLatch(1)
val turns = Channel<RealtimeTurnInput>(Channel.UNLIMITED)
val turnCompletions = Channel<Unit>(Channel.UNLIMITED)
val followUpDelivered = CompletableDeferred<Result<Unit>>()
lateinit var socket: ScriptedWebSocket
lateinit var listener: WebSocketListener
lanServer.dispatcher = sessionOnlyDispatcher(
path = "/voice/realtime-agent/session",
body = """
{
"success": true,
"session_id": "realtime-agent-promotion-boundary-test",
"websocket_path": "/voice/realtime-agent/session-test",
"provider": "xai_realtime",
"model": "grok-voice-latest",
"voice": "leo",
"sample_rate": 24000
}
""".trimIndent(),
)
val client = RelayVoiceClient(
context = context,
okHttpClient = httpClient,
relayUrlProvider = { relayUrl(lanServer) },
sessionTokenProvider = { "session-token" },
webSocketFactory = { request, callback ->
listener = callback
socket = ScriptedWebSocket(request, callback) { true }
callback.onOpen(socket, mockk(relaxed = true))
opened.countDown()
socket
},
)
val sessionJob = async(Dispatchers.IO) {
client.runRealtimeAgent(
prompt = "Start a long task",
inputPcm = ByteArray(0),
turnInputs = turns,
onTurnComplete = { turnCompletions.trySend(Unit) },
) { _, _ -> }
}
try {
assertTrue(opened.await(2, TimeUnit.SECONDS))
listener.onMessage(
socket,
"""{"type":"hermes.run.promoted","run_id":"run-silent","spoken_handoff":false}""",
)
withTimeout(2_000) { turnCompletions.receive() }
turns.send(
RealtimeTurnInput(
inputPcm = ByteArray(6_400) { 4 },
deliveryResult = followUpDelivered,
)
)
assertTrue(withTimeout(2_000) { followUpDelivered.await() }.isSuccess)
listener.onMessage(
socket,
"""{"type":"hermes.run.promoted","run_id":"run-spoken","spoken_handoff":true}""",
)
delay(100)
assertTrue(turnCompletions.tryReceive().isFailure)
listener.onMessage(socket, """{"type":"voice.response.done"}""")
withTimeout(2_000) { turnCompletions.receive() }
} finally {
turns.close()
withTimeout(2_000) { sessionJob.await() }
}
}
private fun relayUrl(server: MockWebServer): String =
"ws://${server.hostName}:${server.port}"
@@ -12,7 +12,7 @@ import com.hermesandroid.relay.viewmodel.VoiceUiState
import com.hermesandroid.relay.viewmodel.backgroundRunAfterCancelRequest
import com.hermesandroid.relay.viewmodel.preserveRealtimeTurnOnStop
import com.hermesandroid.relay.viewmodel.realtimeTranscriptState
import com.hermesandroid.relay.viewmodel.realtimeTurnActiveAfterResponseDone
import com.hermesandroid.relay.viewmodel.realtimeTurnActiveAfterPromotion
import com.hermesandroid.relay.viewmodel.voiceSessionExitState
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNull
@@ -27,12 +27,10 @@ class VoiceModeOverlayStateTest {
}
@Test
fun responseDone_keepsLogicalTurnActiveOnlyWhileBackgroundRunIsLive() {
assertEquals(true, realtimeTurnActiveAfterResponseDone(BackgroundRunPhase.RUNNING))
assertEquals(true, realtimeTurnActiveAfterResponseDone(BackgroundRunPhase.RECONNECTING))
assertEquals(false, realtimeTurnActiveAfterResponseDone(BackgroundRunPhase.DELIVERING))
assertEquals(false, realtimeTurnActiveAfterResponseDone(BackgroundRunPhase.DONE))
assertEquals(false, realtimeTurnActiveAfterResponseDone(null))
fun promotion_keepsForegroundBusyOnlyForSpokenHandoff() {
assertEquals(false, realtimeTurnActiveAfterPromotion(spokenHandoff = false))
assertEquals(true, realtimeTurnActiveAfterPromotion(spokenHandoff = true))
assertEquals(true, realtimeTurnActiveAfterPromotion(spokenHandoff = null))
}
@Test
@@ -1,11 +1,43 @@
package com.hermesandroid.relay.ui.screens
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertNotEquals
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
class ChatScrollSnapshotTest {
@Test
fun `completion releases only the retained live tail`() {
assertNull(releaseRetainedLiveTail("assistant-live", "assistant-live"))
assertEquals(
"assistant-live",
releaseRetainedLiveTail("assistant-live", "different-message"),
)
assertNull(releaseRetainedLiveTail(null, "assistant-live"))
}
@Test
fun `tall markdown tail is positioned by its trailing edge`() {
assertEquals(
1_208,
tailEndScrollOffset(
tailSizePx = 2_400,
footerSizePx = 8,
viewportSizePx = 1_200,
),
)
assertEquals(
0,
tailEndScrollOffset(
tailSizePx = 600,
footerSizePx = 8,
viewportSizePx = 1_200,
),
)
}
@Test
fun `same-tail stream completion requests an atomic bottom anchor`() {
val streaming = snapshot(isStreaming = true)
@@ -0,0 +1,114 @@
package com.hermesandroid.relay.ui.screens
import com.hermesandroid.relay.network.upstream.DashboardApiClient
import com.hermesandroid.relay.network.upstream.DashboardCookieJar
import com.hermesandroid.relay.network.upstream.InMemoryDashboardCookieStore
import com.hermesandroid.relay.network.upstream.importDashboardCookieHeader
import kotlinx.coroutines.test.runTest
import okhttp3.OkHttpClient
import okhttp3.mockwebserver.MockResponse
import okhttp3.mockwebserver.MockWebServer
import org.junit.After
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertTrue
import org.junit.Before
import org.junit.Test
class DashboardWebViewAuthPolicyTest {
private lateinit var server: MockWebServer
@Before
fun setUp() {
server = MockWebServer()
server.start()
}
@After
fun tearDown() {
server.shutdown()
}
@Test
fun selfHostedOidc_usesDashboardLoginWithoutNativeOrLoopbackParameters() {
val url = DashboardApiClient.authLoginUrl(
baseUrl = "https://hermes.example.test",
provider = "self-hosted",
next = "/",
)
assertEquals(
"https://hermes.example.test/auth/login?provider=self-hosted&next=%2F",
url,
)
assertFalse(url.contains("/auth/native/authorize"))
assertFalse(url.contains("redirect_uri"))
assertFalse(url.contains("127.0.0.1"))
}
@Test
fun publicDashboardCallback_importsCookieAndVerifiesAuthenticatedSession() = runTest {
assertEquals(
DashboardWebViewAuthNavigation.ImportAndVerify,
dashboardWebViewAuthNavigation(
"https://hermes.example.test",
"https://hermes.example.test/auth/callback?code=public-code&state=public-state",
),
)
server.enqueue(
MockResponse()
.setHeader("Content-Type", "application/json")
.setBody(
"""{"authenticated":true,"username":"operator","provider":"self-hosted"}""",
),
)
val store = InMemoryDashboardCookieStore()
val callbackUrl = server.url("/auth/callback?code=public-code").toString()
assertEquals(
1,
importDashboardCookieHeader(
store = store,
url = callbackUrl,
cookieHeader = "hermes_session=authenticated",
),
)
val client = DashboardApiClient(
baseUrl = server.url("/").toString(),
okHttpClient = OkHttpClient.Builder()
.cookieJar(DashboardCookieJar(store))
.build(),
)
val session = client.currentSession().getOrThrow()
assertTrue(session.authenticated)
val request = server.takeRequest()
assertEquals("/api/auth/me", request.path)
assertEquals("hermes_session=authenticated", request.getHeader("Cookie"))
client.shutdown()
}
@Test
fun foreignLoopbackCallbacksAreRejectedWhileProviderPagesContinue() {
val dashboard = "https://hermes.example.test"
listOf(
"http://127.0.0.1:40179/callback?code=code",
"http://localhost:40179/callback?code=code",
"http://[::1]:40179/callback?code=code",
).forEach { callback ->
assertEquals(
callback,
DashboardWebViewAuthNavigation.RejectLoopbackCallback,
dashboardWebViewAuthNavigation(dashboard, callback),
)
}
assertEquals(
DashboardWebViewAuthNavigation.Continue,
dashboardWebViewAuthNavigation(
dashboard,
"https://auth.example.test/application/o/authorize/",
),
)
}
}
@@ -33,6 +33,19 @@ class AssistantSpeechCursorTest {
assertEquals(listOf("The check is complete."), second.deltas.map { it.text })
assertTrue(second.deltas.single().startsNewBubble)
assertEquals("I'll check that.\n\nThe check is complete.", second.aggregateText)
assertEquals("The check is complete.", second.finalAnswerText)
}
@Test
fun `final answer skips blank tool bubbles and intermediate commentary`() {
val cursor = AssistantSpeechCursor(emptyList())
val interim = message("interim", MessageRole.ASSISTANT, "I'll check that.")
val toolOnly = message("tool", MessageRole.ASSISTANT, " ")
val final = message("final", MessageRole.ASSISTANT, " The settled answer. ")
val batch = cursor.poll(listOf(interim, toolOnly, final))
assertEquals("The settled answer.", batch.finalAnswerText)
}
@Test
@@ -240,6 +240,74 @@ class ChatViewModelRealtimeTurnTest {
assertEquals(2, handler.messages.value.size)
}
@Test
fun silentBackgroundPromotionReleasesForegroundStreamButKeepsTaskOwnership() {
val assistantId = viewModel.startRealtimeAgentTurn(
userText = "Check release readiness",
chatSessionId = "session-1",
)
viewModel.applyRealtimeAgentEvent(
assistantMessageId = assistantId,
event = RealtimeVoiceEvent(
type = "hermes.run.promoted",
runId = "run-silent",
tier = "durable",
spokenHandoff = false,
raw = "{}",
),
)
val promoted = handler.messages.value.single { it.id == assistantId }
assertFalse(handler.isStreaming.value)
assertFalse(promoted.isStreaming)
assertEquals(BackgroundTaskPhase.RUNNING, promoted.backgroundTask?.phase)
viewModel.applyRealtimeAgentEvent(
assistantMessageId = "newer-turn",
event = RealtimeVoiceEvent(
type = "hermes.run.progress",
runId = "run-silent",
message = "Checking Android",
raw = "{}",
),
)
val updated = handler.messages.value.single { it.id == assistantId }
assertEquals("Checking Android", updated.backgroundTask?.statusLine)
assertEquals(BackgroundTaskPhase.RUNNING, updated.backgroundTask?.phase)
}
@Test
fun spokenBackgroundPromotionWaitsForProviderResponseBoundary() {
val assistantId = viewModel.startRealtimeAgentTurn(
userText = "Check release readiness",
chatSessionId = "session-1",
)
viewModel.applyRealtimeAgentEvent(
assistantMessageId = assistantId,
event = RealtimeVoiceEvent(
type = "hermes.run.promoted",
runId = "run-spoken",
spokenHandoff = true,
raw = "{}",
),
)
assertTrue(handler.isStreaming.value)
viewModel.applyRealtimeAgentEvent(
assistantMessageId = assistantId,
event = RealtimeVoiceEvent(type = "voice.response.done", raw = "{}"),
)
assertFalse(handler.isStreaming.value)
assertEquals(
BackgroundTaskPhase.RUNNING,
handler.messages.value.single { it.id == assistantId }.backgroundTask?.phase,
)
}
@Test
fun backgroundRunKeepsItsOwnerAfterANewerLocalVoiceCommand() {
val backgroundAssistantId = viewModel.startRealtimeAgentTurn(
@@ -42,6 +42,15 @@ class VoiceTurnSessionFenceTest {
assertFalse(fence.accepts(sessionId = "other", messages = emptyList()))
}
@Test
fun `transient mismatch does not lose existing session binding`() {
val fence = VoiceTurnSessionFence(initialSessionId = "active")
fence.bindSubmittedUser("voice-user")
assertFalse(fence.accepts(sessionId = "other", messages = emptyList()))
assertTrue(fence.accepts(sessionId = "active", messages = emptyList()))
}
private fun user(uiKey: String) = ChatMessage(
id = "id-$uiKey",
role = MessageRole.USER,
+2 -2
View File
@@ -1,6 +1,6 @@
plugins {
id("com.android.application") version "9.3.0" apply false
id("com.android.library") version "9.3.0" apply false
id("com.android.application") version "9.3.1" apply false
id("com.android.library") version "9.3.1" apply false
id("org.jetbrains.kotlin.plugin.compose") version "2.4.10" apply false
id("org.jetbrains.kotlin.plugin.serialization") version "2.4.10" apply false
}
+45 -2
View File
@@ -1716,7 +1716,10 @@ override:
**Protocol additions (relay <-> Android, additive).**
- `hermes.run.promoted` - run moved to background; carries `run_id`,
`promote_after_ms`, `spoken_handoff`.
`promote_after_ms`, `spoken_handoff`. A false `spoken_handoff` is also the
foreground turn boundary; when true, the following `voice.response.done` is
the boundary. In both cases the background socket, task card, cancellation,
progress, and result delivery remain active.
- `hermes.run.background_completed` - background run finished; precedes the
provider/relay summary.
- Extend `hermes.run.progress` with `tier` and `floor` so the client can render
@@ -2166,7 +2169,7 @@ An API endpoint or Relay can be added later without recreating the connection.
## ADR 39 — Android dashboard redirect auth uses native PKCE
**Status:** Accepted (2026-07-25).
**Status:** Superseded by ADR 40 (2026-07-27).
**Context.** Android originally completed redirect-provider dashboard sign-in
inside a WebView and imported cookies. Current upstream Gateway can advertise a
@@ -2199,3 +2202,43 @@ socket.
is offered.
- Older upstream versions remain usable through the explicitly identified
WebView compatibility path.
---
## ADR 40 — Android dashboard redirect auth remains cookie/OIDC
**Status:** Accepted (2026-07-27).
**Context.** Upstream advertises `native_pkce` in `/api/status.auth_flows` for
its desktop client. The corresponding `/auth/native/*` broker is explicitly a
desktop system-browser flow: it redirects to a loopback listener owned by the
desktop process and returns bearer tokens rather than dashboard cookies.
Android incorrectly treated that server-wide capability as a platform-neutral
mode selector, so redirect providers such as self-hosted OIDC were sent through
the desktop loopback contract.
**Decision.** Android redirect-provider sign-in always uses the upstream
dashboard cookie flow:
- open `/auth/login?provider=...&next=...` in the embedded sign-in WebView;
- allow the provider to return through the dashboard's public
`/auth/callback`;
- import only cookies observed on the configured dashboard origin;
- verify the imported session through `/api/auth/me`;
- reject a foreign `http://127.0.0.1`, `localhost`, or `[::1]` `/callback`
navigation instead of following or importing it.
Android does not select `/auth/native/authorize` from `auth_flows`. Desktop
native PKCE remains an upstream desktop capability and is unchanged. Existing
encrypted native-token support is retained only so already-issued Android
sessions can be cleared or age out safely; it is not a sign-in entry point.
**Consequences.**
- Self-hosted OIDC uses the same public callback registered for the dashboard.
- Android Manage, Chat, Voice, and onboarding continue to share one verified
dashboard cookie session.
- A server-wide desktop capability can no longer switch Android into a
loopback callback flow.
- Android retains an embedded WebView for dashboard authentication; provider
compatibility and WebView security updates remain part of Android upkeep.
+5 -5
View File
@@ -13,7 +13,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -48,7 +48,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -72,7 +72,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -96,7 +96,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -120,7 +120,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "535984df5230a1120622c384f76056611c4edc73599b6e7f2311827c776eedd7",
"main": "91384aba5431a441e4ab2282d0efbdccd42f94b1a80d088971bcbe4ab12b23eb",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
+5 -5
View File
@@ -85,12 +85,12 @@ This app is a community project and is not affiliated with or endorsed by NousRe
Paste into Play Console → **What's new** (≤500 characters):
```
v1.5.0 - Hermes, always in reach
v1.5.1 - Voice and chat stay in place
* Secure Dashboard sign-in and a clearer Agent Passport.
* Active background chats with actionable approval and question alerts.
* Better attachments, image generation, voice, routing, and recovery.
* Optional permission guidance that never blocks chat.
* Voice Focus and full Conversation layouts.
* Reliable Standard Voice narration after generation.
* Realtime background work without blocked voice controls.
* Formatted streamed answers stay at their completed end.
```
## Category
+3 -3
View File
@@ -419,7 +419,7 @@ The bridge UI drives — and is driven by — Tier 5 safety-rails (`BridgeSafety
- **Connections** (v0.6.0+) — lists every paired Hermes server with a per-card status chip. Actions: rename (inline), re-pair (reuses `ConnectionWizard` with `connectionId` nav arg), revoke, remove. Add-connection button launches the standard QR flow. Settings briefly treats a paired + disconnected relay as **Connecting** during the reconnect grace window, then promotes it to **Relay unreachable - tap to reconnect** if the live socket does not recover. API / Relay / Session detail sheets include compact sanitized recent-activity tails, and **Settings -> Diagnostics** shows the consolidated app-level API, relay, session, endpoint, and voice activity buffer. See `docs/decisions.md` §19.
- **Connection (single-server settings)** — summary-first detail for one Hermes installation. Dashboard/Gateway health drives standard Chat, Manage, Sessions, and Voice readiness. API fallback and Relay extensions appear as independently optional capabilities. Advanced configuration exposes manual Dashboard, API, and Relay endpoints plus their native credentials; missing API or Relay settings never make a healthy Dashboard/Gateway connection look broken. Pairing-code and QR fallbacks remain available for Relay and compatibility setups. Transport security posture and paired-device grants remain visible without leading the normal setup flow with ports or bearer keys.
- **Chat** — Show reasoning toggle, smooth auto-scroll toggle (live-follow streaming, default on), show token usage toggle, app context prompt toggle, tool call display (Off/Compact/Detailed), streaming endpoint selector (`auto` / `sessions` / `runs`), Stats for Nerds (analytics charts)
- **Voice** — route-aware voice engine selector (`Vanilla Hermes` via dashboard audio, `Relay Voice Output`, and experimental `Realtime Agent`), global interaction mode (tap / hold / continuous), silence threshold slider, Auto-TTS toggle, selected-engine cards for dashboard or relay-backed settings, language picker, and a Test Current Engine card. Vanilla Hermes voice depends on Manage/dashboard auth; Relay-backed engines run a fast relay health preflight before uploading audio or opening a realtime provider session so a hung relay surfaces as a connection error instead of an indefinite Thinking state.
- **Voice** — route-aware voice engine selector (`Vanilla Hermes` via dashboard audio, `Relay Voice Output`, and experimental `Realtime Agent`), global interaction mode (tap / hold / continuous), silence threshold slider, a final-answer-only speech policy, Auto-TTS toggle, selected-engine cards for dashboard or relay-backed settings, language picker, and a Test Current Engine card. Final-answer-only keeps tool/service progress and intermediate commentary visual while both voice engines wait to speak the settled answer; approvals, confirmation questions, and blocking failures remain actionable. Vanilla Hermes voice depends on Manage/dashboard auth; Relay-backed engines run a fast relay health preflight before uploading audio or opening a realtime provider session so a hung relay surfaces as a connection error instead of an indefinite Thinking state.
- **Notification companion** — opt-in status, "Open Android Settings" action, test notification dump
- **Permissions** — central permission/capability review screen linked from Settings and onboarding. It makes the Vanilla Hermes path explicit ("Chat and Manage" need no Android runtime grant), lists optional camera/microphone/notification access with current status and Android Settings links, and shows sideload-only Device Control requirements only in the sideload flavor.
- **Appearance** — theme (auto/light/dark), dynamic colors toggle
@@ -847,7 +847,7 @@ utilities.
- `GET /voice/config` — provider availability + current settings from `tts:` / `stt:` in `~/.hermes/config.yaml`. When the basic TTS provider is Gemini or xAI, the response includes a `tts.enhanced` capability block (voices/models/audio-tag support + `supports_persona`/`supports_language` flags) so the app renders a per-request enhanced-voice picker. The Vanilla Hermes dashboard `/api/audio/speak` has no per-request surface — enhanced voice there stays config-only via Manage `PUT /api/config`.
- `GET/PATCH /voice/output/config`, `POST /voice/output/session`, and `GET /voice/output/{session_id}` — relay-mediated streaming TTS renderer sessions. Android sends final Hermes text or brokered tool-status text and receives mono PCM deltas for direct `AudioTrack` playback. Session creation accepts optional provider/model/voice/sample-rate/language overrides for ephemeral draft previews; omitted values continue to resolve from the saved profile/relay defaults. Session responses include resumable-session metadata and PCM events carry `event_id`/`audio_event_id`, so short route changes during stable speech playback can resume and replay missed audio without re-rendering. Config responses include provider option metadata (`providers[].models`, `providers[].voices`, `providers[].languages`, `providers[].sample_rates`) for first-class dropdowns.
- `GET/PATCH /voice/realtime/config`, `POST /voice/realtime/session`, and `GET /voice/realtime/{session_id}` — relay-mediated realtime provider-agent sessions for lab/dev experiments. Android can send PCM input events and receives mono PCM provider deltas for direct `AudioTrack` playback. Realtime config responses expose the same provider option shape where known.
- `GET/PATCH /voice/realtime-agent/config`, `POST /voice/realtime-agent/session`, and `GET /voice/realtime-agent/{session_id}` — experimental Hermes-brokered Realtime Agent engine. The broker binds active profile/chat session/auth, streams Android mic PCM to a native realtime provider such as `xai_realtime` or `openai_realtime`, normalizes provider transcript/audio/function-call events, mirrors Hermes session/tool/confirmation events into Android, and returns compact Hermes tool results to the provider for concise spoken follow-up. Session responses include resumable-session metadata (`resume_token`, `resume_supported`, `resume_ttl_ms`); server events carry `event_id`, audio deltas carry `audio_event_id`, and Android can resume a detached session through the current `effectiveRelayUrl` after short Wi-Fi/cellular/LAN/Tailscale changes without starting a second Hermes run. A replacement route is usable only after relay `voice.session.resumed` confirmation; socket generation + resume-episode claims reject stale failure/close/fatal callbacks, unacknowledged input is replayed atomically, and each route-loss episode owns a bounded retry budget that starts at loss rather than session prewarm. Terminal exhaustion detaches session-owned reconnect UI so a stopped retry loop cannot leave an active task pill behind. The only provider-facing tool surface is `hermes_run_task`, `hermes_get_status`, `hermes_cancel`, and `hermes_confirm`.
- `GET/PATCH /voice/realtime-agent/config`, `POST /voice/realtime-agent/session`, and `GET /voice/realtime-agent/{session_id}` — experimental Hermes-brokered Realtime Agent engine. The broker binds active profile/chat session/auth, streams Android mic PCM to a native realtime provider such as `xai_realtime` or `openai_realtime`, normalizes provider transcript/audio/function-call events, mirrors Hermes session/tool/confirmation events into Android, and returns compact Hermes tool results to the provider for concise spoken follow-up. Session creation accepts an ephemeral `final_answer_only` boolean; when enabled, the broker disables routine spoken handoffs and progress while preserving approval, confirmation, and blocking-failure prompts. Session responses include resumable-session metadata (`resume_token`, `resume_supported`, `resume_ttl_ms`); server events carry `event_id`, audio deltas carry `audio_event_id`, and Android can resume a detached session through the current `effectiveRelayUrl` after short Wi-Fi/cellular/LAN/Tailscale changes without starting a second Hermes run. A replacement route is usable only after relay `voice.session.resumed` confirmation; socket generation + resume-episode claims reject stale failure/close/fatal callbacks, unacknowledged input is replayed atomically, and each route-loss episode owns a bounded retry budget that starts at loss rather than session prewarm. Terminal exhaustion detaches session-owned reconnect UI so a stopped retry loop cannot leave an active task pill behind. The only provider-facing tool surface is `hermes_run_task`, `hermes_get_status`, `hermes_cancel`, and `hermes_confirm`.
- `GET /voice/output/providers/{provider_id}/options`, `GET /voice/realtime/providers/{provider_id}/options`, and `GET /voice/realtime-agent/providers/{provider_id}/options` — provider-specific option refresh before saving. Android calls these when a provider is selected so dynamic account-backed choices can be fetched by the relay without exposing provider secrets. xAI refreshes built-in/paginated custom voices when API/OAuth auth is available; ElevenLabs refreshes voices/models/languages with its API key; OpenAI uses static documented voice choices. Realtime Agent provider payloads include `supports_realtime_agent_native` so render/lab-only realtime support is not confused with native speech-to-speech Hermes tooling. Responses include `schema_version`, grouped voice metadata, recommended/custom flags, and model/voice compatibility hints when known. Unknown or unauthenticated discovery falls back to static provider metadata plus manual entry.
- `POST /voice/output/providers/{provider_id}/validate`, `POST /voice/realtime/providers/{provider_id}/validate`, and `POST /voice/realtime-agent/providers/{provider_id}/validate` — pre-save validation for provider/model/voice/sample-rate selections. Unknown manual IDs return warnings; explicit incompatibilities return blocking errors.
- Voice-output provider defaults are relay-owned under `voice_output:` in `~/.hermes-relay/config.yaml` (or `RELAY_VOICE_OUTPUT_CONFIG`), then overridden by `RELAY_VOICE_OUTPUT_*` env vars for temporary tests. Authenticated operator clients may patch safe defaults (`enabled`, `provider`, `model`, `voice`, `sample_rate`, `language`, `codec`, `optimize_streaming_latency`, `text_normalization`, `auto_speech_tags`, `fallback_enabled`) through the relay. With `?profile=<name>`, the patch writes that profile's `voice_output:` section. Provider secrets and local auth paths stay server-side. `auto_speech_tags` is an xAI enhanced-voice control: when the renderer is `xai_tts` the relay applies `upstream_voice.apply_xai_speech_tags()` (upstream's inline/wrapping tone markers) to each chunk before rendering, so the streaming path matches the basic `/voice/synthesize` tone behavior. The `voice_lab` renderer set is xai/openai/elevenlabs — there is no Gemini streaming provider, so Gemini enhanced voice is `/voice/synthesize`-only.
@@ -864,7 +864,7 @@ utilities.
- Stable voice integrates with `ChatViewModel` by **observing** `messages: StateFlow`; transcribed text goes through normal `chatVm.sendMessage(text)` so voice utterances appear as regular user messages in chat history. Experimental Realtime Agent creates a mirrored chat turn and applies broker events directly so tool state, transcript text, assistant deltas, and final responses appear without leaving voice mode.
- `VoiceModeOverlay` — full-screen UI with the MorphingSphere at 60% height in `voiceMode=true`, transcribed + response text, mic button supporting Tap / Hold / Continuous interaction modes.
- `MorphingSphere` gains `SphereState.Listening` (soft blue/purple, subtle wobble with user amplitude) and `SphereState.Speaking` (vivid green/teal, dramatic core-warmth pulse with agent amplitude). Additive changes — existing call sites unchanged via defaulted `voiceAmplitude` / `voiceMode` params.
- Voice Settings screen off the main Settings — Output / Listening / Advanced tabs split engine/provider selection from turn-taking controls and diagnostics. Output groups the provider summary and model/voice catalog, exposes inline no-save play/stop previews for the draft model and individual voices, shows the speaking waveform on the active row, and keeps Discard separate from Save. Dropdowns come from relay-advertised provider metadata, refresh through provider-specific options routes when the selected provider changes, become searchable/grouped for large voice catalogs, and validate compatibility before saving, with advanced manual entry for raw provider/model/voice IDs. Voice routes receive the selected Hermes profile; the relay reports whether values came from profile config, relay config, or global fallback. Test Current Engine remains under Advanced and uses `/voice/output/*` playback for stable mode and `/voice/realtime-agent/*` provider-native session playback for realtime mode; normal assistant speech uses the same streaming renderer PCM path when available.
- Voice Settings screen off the main Settings — Output / Listening / Advanced tabs split engine/provider selection from turn-taking controls and diagnostics. Global controls include a final-answer-only policy shared by Standard and Realtime voice. Output groups the provider summary and model/voice catalog, exposes inline no-save play/stop previews for the draft model and individual voices, shows the speaking waveform on the active row, and keeps Discard separate from Save. Dropdowns come from relay-advertised provider metadata, refresh through provider-specific options routes when the selected provider changes, become searchable/grouped for large voice catalogs, and validate compatibility before saving, with advanced manual entry for raw provider/model/voice IDs. Voice routes receive the selected Hermes profile; the relay reports whether values came from profile config, relay config, or global fallback. Test Current Engine remains under Advanced and uses `/voice/output/*` playback for stable mode and `/voice/realtime-agent/*` provider-native session playback for realtime mode; normal assistant speech uses the same streaming renderer PCM path when available.
See `docs/decisions.md` → **Voice Mode — Architecture** for the historical baseline decisions. Current voice mode records PCM/WAV for STT, routes stable assistant speech through `/voice/output/*`, keeps `/voice/realtime/*` as a provider-agent lab path, and exposes `/voice/realtime-agent/*` as an experimental Hermes-brokered engine.
+4 -4
View File
@@ -1,7 +1,7 @@
[versions]
appVersionName = "1.5.0"
appVersionCode = "33"
agp = "9.3.0"
appVersionName = "1.5.1"
appVersionCode = "34"
agp = "9.3.1"
kotlin = "2.4.10"
compose-bom = "2026.06.01"
navigation-compose = "2.9.8"
@@ -15,7 +15,7 @@ security-crypto = "1.1.0"
tink-android = "1.23.0"
lifecycle = "2.11.0"
activity-compose = "1.13.0"
browser = "1.9.0"
browser = "1.10.0"
appcompat = "1.7.1"
core-ktx = "1.19.0"
datastore = "1.2.1"
+30 -8
View File
@@ -259,6 +259,7 @@ class RealtimeAgentSession:
# ADR 33 promotion state (populated from realtime_voice settings at create).
promotion_enabled: bool = False
promote_after_ms: int = 6000
final_answer_only: bool = False
spoken_handoff: bool = True
result_delivery: str = "speak_verbatim"
promoted_transcript: str | None = None
@@ -402,6 +403,7 @@ class RealtimeAgentHandler:
config=self.config,
)
context_messages = _parse_context_messages(payload.get("context_messages"))
final_answer_only = _bool_value(payload.get("final_answer_only")) is True
fetch_context_messages = getattr(self.hermes, "fetch_context_messages", None)
if not context_messages and chat_session_id and callable(fetch_context_messages):
context_messages = await self.hermes.fetch_context_messages(
@@ -439,10 +441,19 @@ class RealtimeAgentHandler:
),
promotion_enabled=bool(settings.get("promotion_enabled", False)),
promote_after_ms=int(settings.get("promote_after_ms", 6000)),
spoken_handoff=bool(settings.get("spoken_handoff", True)),
result_delivery=str(settings.get("result_delivery", "speak_verbatim")),
final_answer_only=final_answer_only,
spoken_handoff=(
False if final_answer_only else bool(settings.get("spoken_handoff", True))
),
result_delivery=(
"speak_verbatim"
if final_answer_only
else str(settings.get("result_delivery", "speak_verbatim"))
),
progress_spoken_after_seconds=(
max(0, int(settings.get("progress_spoken_after_ms", 0))) / 1000.0
0.0
if final_answer_only
else max(0, int(settings.get("progress_spoken_after_ms", 0))) / 1000.0
),
progress_repeat_seconds=(
max(0, int(settings.get("progress_repeat_ms", 30000))) / 1000.0
@@ -4805,6 +4816,19 @@ def _native_instructions(session: RealtimeAgentSession) -> str:
current_timezone = interface_context["current_timezone"]
context_block = _provider_context_block(session.context_messages)
profile_block = _profile_prompt_block(session.profile_prompt_context)
speech_policy = (
"Final-answer-only speech is enabled. Do not speak acknowledgements, "
"tool progress, service or status updates, or intermediate commentary. "
"Call Hermes silently and wait to speak until its settled final answer is "
"available. Approval or confirmation questions and blocking failures may "
"still be spoken because the user must act on them. "
if session.final_answer_only
else (
"You may speak one brief acknowledgement such as 'I'll check Hermes' "
"or 'I'll check that' before the tool call, then call hermes_run_task "
"immediately. The relay will provide restrained status while Hermes runs. "
)
)
return (
"You are the provider-native speech loop for Hermes Relay. Keep replies "
"brief and conversational. Active interface: "
@@ -4842,11 +4866,9 @@ def _native_instructions(session: RealtimeAgentSession) -> str:
"information beyond what was already delivered. If the needed context is "
"missing, stale, or requires fresh data or verification you do not "
"already have, call hermes_run_task before answering. Do not say you lack "
"context before a Hermes call. You "
"may speak one brief acknowledgement such as 'I'll check Hermes' or "
"'I'll check that' before the tool call, then call hermes_run_task "
"immediately. Do not give a substantive answer until Hermes returns. "
"The relay will provide restrained status while Hermes runs. "
"context before a Hermes call. "
f"{speech_policy}"
"Do not give a substantive answer until Hermes returns. "
"Route through Hermes for latest/recent/versioned data, device/desktop/app "
"state, personal/session/project context, side effects, high-stakes or "
"precision-sensitive answers, explicit check/verify/look-up requests, and "
@@ -630,6 +630,26 @@ class RealtimeAgentRoutesTests(AioHTTPTestCase):
self.assertGreaterEqual(body["resume_ttl_ms"], 1000)
self.assertTrue(body["experimental"])
async def test_final_answer_only_overrides_spoken_progress_for_one_session(self) -> None:
token = await self._make_session()
resp = await self.client.post(
"/voice/realtime-agent/session",
json={"final_answer_only": True},
headers=self._bearer(token),
)
self.assertEqual(resp.status, 200)
body = await resp.json()
session = self._server().realtime_agent.sessions[body["session_id"]]
self.assertTrue(session.final_answer_only)
self.assertFalse(session.spoken_handoff)
self.assertEqual(session.progress_spoken_after_seconds, 0.0)
self.assertEqual(session.result_delivery, "speak_verbatim")
instructions = broker_module._native_instructions(session)
self.assertIn("Do not speak acknowledgements", instructions)
self.assertIn("wait to speak until its settled final answer", instructions)
async def test_provider_native_instructions_include_recent_context(self) -> None:
token = await self._make_session()
fake_provider = FakeNativeProvider()
+2 -2
View File
@@ -5,8 +5,8 @@ pluginManagement {
gradlePluginPortal()
}
plugins {
id("com.android.application") version "9.3.0"
id("com.android.library") version "9.3.0"
id("com.android.application") version "9.3.1"
id("com.android.library") version "9.3.1"
id("org.jetbrains.kotlin.plugin.compose") version "2.4.10"
id("org.jetbrains.kotlin.plugin.serialization") version "2.4.10"
}