Compare commits

...
Author SHA1 Message Date
Bailey Dixon 52f3f75658 Merge pull request #591 from Codename-11/dev
release: Android 1.17.0 and Plugin 1.11.3
2026-09-13 21:39:36 -04:00
Bailey Dixon 1d25985aca Merge pull request #592 from Codename-11/release/android-1.17.0-play-notes
release(android): reconcile Chinese Play notes for 1.17.0
2026-09-13 21:38:29 -04:00
Bailey Dixon 500cc83de6 release(android): reconcile Chinese Play notes for 1.17.0 2026-09-13 21:26:31 -04:00
Bailey Dixon 7d3c761c6a Merge pull request #590 from Codename-11/release/android-1.17.0-plugin-1.11.3
release: prepare Android 1.17.0 and Plugin 1.11.3
2026-09-13 21:25:20 -04:00
Bailey Dixon 2e097035fb release(android): android-v1.17.0 and server-v1.11.3 2026-09-13 21:14:30 -04:00
Bailey Dixon 280c20dbca Merge pull request #585 from Codename-11/fix/android-batch-clarify
fix(android): support batch Clarify and simplify chat cards
2026-09-13 20:33:18 -04:00
Bailey Dixon 86a570e72d chore: refresh chat UI cleanup against dev 2026-09-13 20:13:40 -04:00
Bailey Dixon cbe811fb52 fix(android): simplify chat bubble surfaces and footers 2026-09-13 20:13:15 -04:00
Bailey Dixon ac2508c5ce Merge pull request #588 from Codename-11/fix/android-gateway-context-audit
fix(android): show Gateway context-sharing limitations
2026-09-13 16:04:24 -04:00
Bailey Dixon 5eb5d198f4 chore: refresh batch Clarify against current dev 2026-09-13 13:29:57 -04:00
Bailey Dixon 52e0e38a81 chore: merge current dev into batch Clarify fix 2026-09-12 20:11:40 -04:00
Bailey Dixon 061512d330 fix(android): support upstream batch Clarify requests 2026-09-12 19:59:06 -04:00
Bailey Dixon 1a6f9cc10b Merge pull request #580 from Codename-11/dev
release(android): 1.16.1
2026-09-12 09:54:40 -04:00
Bailey Dixon 2865456761 Merge pull request #577 from Codename-11/dev
docs(release): reconcile 1.16.0 and 1.11.2 dates
2026-09-10 10:00:16 -04:00
64 changed files with 1687 additions and 132 deletions
+10 -1
View File
@@ -6,16 +6,25 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
## [Unreleased]
## [Android 1.17.0] - 2026-09-13
### Added
- Optional voice controls over other apps in Google Play, with contextual permission setup, a persistent Stop voice notification, and session shutdown on screen lock or permission loss. Phone control remains sideload-only.
### Fixed
- Android shows standalone response cards without an outer bubble, uses subtler assistant surfaces, and places delivery status beside message timestamps.
- Android answers upstream Clarify batches one question at a time, with independent choices, custom answers, and confirmed progress preserved across reconnects. (#474)
- Android context previews mark phone status and turn context as unavailable in Gateway chats instead of claiming they are sent. Settings clarify that automatic phone-status sharing applies to API-only chats. (#556)
- Relay Dashboard WebSockets work with current Hermes authentication helpers while preserving older-host compatibility, single-use tickets, Host/Origin/IP checks, and Relay session authentication.
- Android shows Hermes profile display names and groups the resolved server default under its agent identity, while preserving explicit profile selection and saved conversations.
## [Plugin 1.11.3] - 2026-09-13
### Fixed
- Relay Dashboard WebSockets work with current Hermes authentication helpers while preserving older-host compatibility, single-use tickets, Host/Origin/IP checks, and Relay session authentication.
## [Android 1.16.1] - 2026-09-12
### Fixed
+3 -6
View File
@@ -1,17 +1,14 @@
# Hermes-Relay Plugin v__VERSION__
**Release Date:** September 10, 2026
**Release Date:** September 13, 2026
## Summary
This patch makes Android and Desktop tool availability fast and reliable when Relay is unavailable, starts late-created Android bridge sessions without restarting Hermes, and restores compatibility with both current and legacy `android_setup` arguments. Standard Chat, Manage, standard voice, and ordinary inbound files remain upstream-owned.
Dashboard WebSocket connections work again with current Hermes authentication helpers, while older Hermes hosts remain supported.
## Fixed
- **Fast, accurate tool availability.** Android and Desktop tool checks use explicit IPv4 loopback and one bounded health snapshot instead of repeated per-tool connection attempts. Multi-PC capability advertisements remain isolated, and unavailable Relay clients continue to fail closed.
- **Late Android bridge recovery.** `android_*` calls retry profile-scoped and active bridge-session credentials after a stale token is rejected, so a phone connected after Hermes startup becomes usable without restarting the host.
- **Compatible Android setup arguments.** `android_setup` accepts the canonical `bridge_session_token` and `pairing_code` fields as well as their legacy aliases, with structured errors when no usable credential is supplied.
- **Isolated setup tests.** Android tool setup tests use a temporary Hermes home instead of writing bridge settings into the operator environment.
- Resolve WebSocket guards from their current upstream module and retain the older-host fallback. Single-use tickets, Host/Origin/IP checks, and independent Hermes-Relay session authentication remain enforced. Missing or incomplete helper contracts deny admission.
## Install / update
+20 -8
View File
@@ -1,10 +1,10 @@
# Hermes-Relay Android v1.16.1
# Hermes-Relay Android v1.17.0
**Release Date:** September 12, 2026
**Release Date:** September 13, 2026
## Download
> Installing on your phone? Download `hermes-relay-1.16.1-sideload-release.apk` and tap it for the full feature set, or install the conservative build from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
> Installing on your phone? Download `hermes-relay-1.17.0-sideload-release.apk` and tap it for the full feature set, or install from [Google Play](https://play.google.com/store/apps/details?id=com.axiomlabs.hermesrelay).
The `.aab` file is a Play Console upload bundle and cannot be installed by tapping it on a phone.
@@ -12,16 +12,28 @@ Verify the download against `SHA256SUMS.txt`. See the [sideload guide](https://h
## Summary
This patch fixes a remaining cold-start path that could leave a Dashboard-only connection waiting for Gateway readiness until the app resumed or its network route changed.
Google Play gains optional voice controls over other apps. This release also makes Clarify batches, profile identity, and chat context easier to follow while preserving confirmed answers and saved conversations.
## Added
- Start Voice Overlay from Voice Focus after granting microphone, notification, and display-over-other-apps access. Permission grants require a separate Start action. Stop voice from the overlay or persistent notification; screen lock, task removal, and permission loss end the session.
## Changed
- Standalone response cards use one surface, assistant bubbles are subtler, and timestamps share a row with delivery status.
## Fixed
- Dashboard-only connections now start the exact profile-scoped session directory before Gateway readiness, so the directory and passive Gateway socket no longer wait on each other during a cold launch.
- Answer upstream Clarify batches one question at a time, with independent choices, custom answers, and confirmed progress across reconnects. (#474)
- Context previews show that Gateway chats cannot send phone status or general turn context. Automatic phone-status sharing remains supported for API-only chats. (#556)
- Profiles display their Hermes names and group the resolved server default under its agent identity, preserving explicit selection and saved conversations.
## Install / Verify
- App version: **1.16.1** (versionCode **56**).
- App version: **1.17.0** (versionCode **57**).
- Standard Chat, sessions, profiles, Manage, voice, and ordinary media use current upstream Hermes. Speech-to-text still requires a configured provider on the host.
- Hermes-Relay Plugin **1.11.2** remains the matching optional release for Relay tools; this Gateway startup fix does not require it.
- Hermes-Relay Plugin **1.11.3** is the optional release for Hermes-Relay tools and current Dashboard WebSocket compatibility.
- Explicit Direct API/API-only connections remain supported and are not used as silent failover for Dashboard-owned chats.
- Granular Device Control and the system Voice Focus overlay remain sideload-only.
- Voice Overlay is available in Google Play and sideload builds. Device Control remains sideload-only.
- Gateway phone-status delivery and automatic Android identification remain unavailable pending upstream support.
- Physical Android 14-16 and OEM voice-overlay testing was not performed for this release. Code, rendered UI, existing emulator evidence, CI, and signed-package preflight provide the recorded verification.
@@ -0,0 +1,116 @@
package com.hermesandroid.relay.viewmodel
import android.os.Handler
import android.os.Looper
import androidx.activity.ComponentActivity
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.test.assertIsDisplayed
import androidx.compose.ui.test.junit4.v2.createAndroidComposeRule
import androidx.compose.ui.test.onNodeWithContentDescription
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.performClick
import androidx.compose.ui.test.performImeAction
import androidx.compose.ui.test.performTextInput
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.hermesandroid.relay.network.upstream.ChatHandler
import com.hermesandroid.relay.network.upstream.DashboardApiClient
import com.hermesandroid.relay.network.upstream.GatewayChatClient
import com.hermesandroid.relay.network.upstream.HermesApiClient
import com.hermesandroid.relay.ui.components.MessageBubble
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.runBlocking
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import okhttp3.OkHttpClient
import okhttp3.WebSocket
import org.junit.After
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Before
import org.junit.Rule
import org.junit.Test
/** Production socket, ViewModel, transcript, Compose and IME actions on a virtual device. */
class ClarifyBatchInstrumentedTest {
@get:Rule val compose = createAndroidComposeRule<ComponentActivity>()
private lateinit var fixture: AndroidGatewayContractFixture
private lateinit var scope: CoroutineScope
private lateinit var gateway: GatewayChatClient
private lateinit var viewModel: ChatViewModel
private lateinit var handler: ChatHandler
private lateinit var socket: WebSocket
@Before fun setUp() {
fixture = AndroidGatewayContractFixture()
scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
val http = OkHttpClient()
gateway = GatewayChatClient(
initialDashboardClient = DashboardApiClient(fixture.server.url("/").toString().trimEnd('/'), http),
okHttpClient = http, scope = scope,
callbackDispatcher = { Handler(Looper.getMainLooper()).post(it) },
)
handler = ChatHandler().also { it.setSessionId("20260821_120000_fixture") }
viewModel = ChatViewModel().also {
it.initialize(HermesApiClient(fixture.server.url("/").toString(), "fixture-key"), handler)
it.streamingEndpoint = "gateway"
it.setProfileMessageLoader { Result.success(emptyList()) }
it.updateGatewayClient(gateway)
}
compose.setContent {
val messages by handler.messages.collectAsStateWithLifecycle()
MaterialTheme {
LazyColumn(Modifier.fillMaxSize()) {
items(messages.size, key = { messages[it].id }) { index ->
MessageBubble(messages[index], showTimestamps = false,
onCardInput = viewModel::answerAsk, animationEnabled = false)
}
}
}
}
assertTrue(runBlocking { gateway.prewarmAwait("20260821_120000_fixture") })
socket = fixture.awaitServerSocket()
}
@After fun tearDown() {
viewModel.updateGatewayClient(null)
gateway.shutdown()
scope.cancel()
fixture.shutdown()
}
@Test fun confirmedProgressSurvivesLifecycleAndCustomAnswerUsesIme() {
compose.runOnIdle { viewModel.sendMessage("Ask two questions") }
fixture.awaitRpc("prompt.submit")
socket.send(fixture.event("clarify.request", Json.parseToJsonElement("""
{"request_id":"batch-device","questions":[
{"qid":"route/a","question":"Which route?","choices":["Canary","Immediate"]},
{"qid":"notes:b","question":"Anything else?","choices":null}
]}
""") as JsonObject, "fixture-live-1"))
compose.waitUntil(10_000) { viewModel.pendingAsk.value != null }
compose.onNodeWithText("Canary").performClick()
compose.waitUntil(10_000) { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "Canary" }
assertEquals(JsonPrimitive("route/a"), fixture.awaitRpc("clarify.respond")["question_id"])
compose.activityRule.scenario.moveToState(Lifecycle.State.STARTED)
compose.activityRule.scenario.moveToState(Lifecycle.State.RESUMED)
compose.onNodeWithText("Question 2 of 2").assertIsDisplayed()
compose.onNodeWithContentDescription("Type an answer…").apply {
performClick()
performTextInput(" Keep rollback ready ")
performImeAction()
}
compose.waitUntil(10_000) { viewModel.pendingAsk.value == null }
compose.onNodeWithText("All questions answered").assertIsDisplayed()
assertEquals(2, fixture.rpcCount("clarify.respond"))
assertEquals(1, fixture.rpcCount("prompt.submit"))
}
}
@@ -1,3 +1,3 @@
v1.16.1 - Dashboard-only cold starts recover
v1.17.0 - Voice over other apps and clearer conversations
Dashboard-only connections now prepare the selected profile before Gateway readiness, fixing a remaining cold-start path that could stay on waking or waiting for Gateway until the app resumed or its network route changed.
Use optional voice controls over other apps, with clear permission setup and an immediate Stop action. Answer Clarify batches one question at a time, with progress preserved across reconnects. Enjoy cleaner chat cards, recognizable profile names, and context previews that show what your connection supports. Phone control remains sideload-only.
@@ -1 +1,3 @@
新增监督模式:家长可配置并固定到指定配置文件,设置受设备身份验证保护。家长可限制附件、标准语音、生成媒体、历史记录、操作和技术详情。实时活动不可用时会话行保持中性显示,从家长设置返回时监督聊天也不再空白。
v1.17.0 - 跨应用语音与更清晰的对话
可选的语音悬浮控件让你在使用其他应用时继续语音会话,提供清晰的权限说明和随时停止操作。逐题回答 Clarify 批量问题,重连后保留已确认的进度。聊天卡片更简洁,配置文件显示名称更易辨认,上下文预览准确说明当前连接支持发送哪些信息。设备控制仍仅限侧载版本。
+19
View File
@@ -1,6 +1,25 @@
{
"schema": 3,
"versions": [
{
"version": "1.17.0",
"title": "Voice over other apps and clearer conversations",
"date": "2026-09-13",
"summary": "Google Play gains optional voice controls over other apps. Clarify questions, profile names, and chat context are easier to follow without losing confirmed answers or saved conversations.",
"changes": [
{"id": "play-voice-overlay", "kind": "added", "title": "Keep voice controls over other apps", "summary": "Start Voice Overlay from Voice Focus after granting microphone, notification, and display-over-other-apps access. Stop voice from its controls or persistent notification; screen lock and permission loss end the session.", "highlight": true},
{"id": "clarify-batches", "kind": "fixed", "title": "Answer Clarify questions one at a time", "summary": "Multi-question requests support separate choices and custom answers. Confirmed progress survives reconnects, and failed sends keep the current answer available.", "highlight": true},
{"id": "chat-card-surfaces", "kind": "improved", "title": "Read cleaner chat cards", "summary": "Standalone response cards lose the extra outer bubble, assistant messages use quieter surfaces, and delivery status sits beside the timestamp."},
{"id": "transport-context-preview", "kind": "fixed", "title": "See which context your chat can send", "summary": "Gateway previews mark phone status and turn context as unavailable. Automatic phone-status sharing is labeled for API-only chats, where it remains supported."},
{"id": "profile-display-names", "kind": "fixed", "title": "Recognize profiles by their display names", "summary": "Profiles use their Hermes display names, and the resolved server default appears under its agent identity. Explicit profile choices and saved conversations are preserved.", "highlight": true}
],
"compatibility": [
"Voice Overlay is now available in Google Play and sideload builds. Device Control remains sideload-only.",
"Standard Chat and voice use upstream Hermes without requiring Hermes-Relay Plugin. Gateway phone-status delivery and automatic Android identification remain unavailable."
],
"playNotes": "Use optional voice controls over other apps, with clear permission setup and an immediate Stop action. Answer Clarify batches one question at a time, with progress preserved across reconnects. Enjoy cleaner chat cards, recognizable profile names, and context previews that show what your connection supports. Phone control remains sideload-only.",
"sections": []
},
{
"version": "1.16.1",
"title": "Dashboard-only cold starts recover",
+13 -5
View File
@@ -1,11 +1,19 @@
v1.16.1 - Dashboard-only cold starts recover
v1.17.0 - Voice over other apps and clearer conversations
Summary
* Dashboard-only connections can now prepare the selected profile and open Gateway chat without waiting for a background, resume, or network-route change.
* Google Play gains optional voice controls over other apps. Clarify questions, profile names, and chat context are easier to follow without losing confirmed answers or saved conversations.
Highlights
* Open Gateway chat from a Dashboard-only cold start — The selected profile's session directory starts before Gateway readiness, so it cannot wait on the same passive socket that depends on its result.
* Keep voice controls over other apps — Start Voice Overlay from Voice Focus after granting microphone, notification, and display-over-other-apps access. Stop voice from its controls or persistent notification; screen lock and permission loss end the session.
* Answer Clarify questions one at a time — Multi-question requests support separate choices and custom answers. Confirmed progress survives reconnects, and failed sends keep the current answer available.
* Recognize profiles by their display names — Profiles use their Hermes display names, and the resolved server default appears under its agent identity. Explicit profile choices and saved conversations are preserved.
Improved
* Read cleaner chat cards — Standalone response cards lose the extra outer bubble, assistant messages use quieter surfaces, and delivery status sits beside the timestamp.
Fixed
* See which context your chat can send — Gateway previews mark phone status and turn context as unavailable. Automatic phone-status sharing is labeled for API-only chats, where it remains supported.
Compatibility
* Standard Dashboard and Gateway chat continue to use current upstream Hermes without requiring the optional Hermes-Relay Plugin.
* Hermes-Relay Plugin 1.11.2 remains the matching optional release for Relay tools.
* Voice Overlay is now available in Google Play and sideload builds. Device Control remains sideload-only.
* Standard Chat and voice use upstream Hermes without requiring Hermes-Relay Plugin. Gateway phone-status delivery and automatic Android identification remain unavailable.
@@ -5,6 +5,7 @@ import androidx.datastore.core.DataStore
import androidx.datastore.preferences.core.Preferences
import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.core.stringPreferencesKey
import com.hermesandroid.relay.network.upstream.GatewayClarifyQuestion
import kotlinx.coroutines.flow.first
import kotlinx.serialization.Serializable
import kotlinx.serialization.encodeToString
@@ -135,6 +136,9 @@ data class ChatTurnAskCheckpoint(
val text: String,
val choices: List<String>? = null,
val multiSelect: Boolean = false,
val questions: List<GatewayClarifyQuestion> = emptyList(),
val answers: Map<String, String> = emptyMap(),
val ownerId: String? = null,
val smartDenied: Boolean = false,
val envVar: String? = null,
val timeoutSeconds: Int,
@@ -71,6 +71,8 @@ data class HermesCard(
* actions.
*/
val input: HermesCardInput? = null,
/** Local Gateway batch; never an ordinary chat-message answer protocol. */
val clarifyBatch: HermesCardClarifyBatch? = null,
) {
object BuiltInTypes {
const val SKILL_RESULT = "skill_result"
@@ -96,6 +98,25 @@ data class HermesCard(
}
}
@Serializable
data class HermesCardClarifyBatch(
val questions: List<HermesCardClarifyQuestion>,
val expiresAtMillis: Long? = null,
)
@Serializable
data class HermesCardClarifyQuestion(
val key: String,
val question: String,
val input: HermesCardInput,
val answer: String? = null,
val submitting: Boolean = false,
)
/** Local callback identity. The RPC always uses the original qid, never this UI key. */
fun clarifyQuestionCardKey(cardKey: String, qid: String): String =
Json.encodeToString(listOf(cardKey, qid))
/**
* Interactive input slot on a [HermesCard]. The flags compose rather than
* branch — a sudo ask can be `masked + holdToConfirm` (password field whose
@@ -518,6 +518,16 @@ class ChatHandler {
}
}
/** Refresh only an existing local ask, preserving its dispatches and transcript position. */
fun updateAskCardMessage(messageId: String, card: HermesCard) {
_messages.update { list ->
list.map { message ->
if (message.clientOnly && message.matchesIdentity(messageId)) message.copy(cards = listOf(card))
else message
}
}
}
/**
* Edit-and-regenerate local truncation: drop [messageId] and everything
* after it. The gateway performs the authoritative truncation via
@@ -1586,6 +1586,7 @@ class GatewayChatClient(
claimedBackground?.pendingAsk?.let { ask ->
boundTurn.restoreInteraction(ask)
}
boundTurn.restorePendingClarify(response)
boundTurn.armWatchdog()
} else if (queued != null) {
synchronized(recoveryEventLock) { recoveryEvents = null }
@@ -1746,17 +1747,39 @@ class GatewayChatClient(
)
/** Answer a [GatewayAsk.Kind.CLARIFY] ask. */
suspend fun respondClarify(requestId: String, answer: String): Result<GatewayAskResponse> {
suspend fun respondClarify(
requestId: String,
answer: String,
questionId: String? = null,
): Result<GatewayAskResponse> {
val respondingTurn = activeTurn
if (questionId != null) {
val ask = respondingTurn?.pendingInteraction
// A lost acknowledgement is ambiguous until activation replays server progress.
// Never overwrite an accepted answer while reconnect is still reconciling it.
if (rejoinInProgress || ask?.kind != GatewayAsk.Kind.CLARIFY || ask.requestId != requestId ||
ask.questions.none { it.qid == questionId } || ask.ownershipToken.retired.get() ||
questionId in ask.ownershipToken.answers.get()
) return Result.failure(GatewayRpcException("Clarification is not ready for this question"))
}
val generation = respondingTurn?.interactionGeneration
val ownershipToken = respondingTurn?.pendingInteraction?.ownershipToken
return rpc(
"clarify.respond",
buildJsonObject {
put("request_id", requestId)
put("answer", answer)
questionId?.let { put("question_id", it) }
},
).map {
it.gatewayAskResponse().also {
respondingTurn?.acknowledgeInteraction(GatewayAskExpiry(GatewayAsk.Kind.CLARIFY, requestId))
if (generation != null) {
respondingTurn.acknowledgeClarify(requestId, questionId, answer, it == GatewayAskResponse.EXPIRED, generation)
}
val currentTurn = activeTurn
if (ownershipToken != null && currentTurn !== respondingTurn) {
currentTurn?.acknowledgeClarifyOwner(requestId, questionId, answer, it == GatewayAskResponse.EXPIRED, ownershipToken)
}
}
}
}
@@ -3854,7 +3877,10 @@ class GatewayChatClient(
val interactionRequest = GatewayEventMapper.interactionRequest(type, payload)
if (interactionRequest != null) {
val previous = backgroundTurn.pendingAsk
backgroundTurn.pendingAsk = interactionRequest
backgroundTurn.pendingAsk = if (previous?.kind == interactionRequest.kind &&
previous.requestId == interactionRequest.requestId && interactionRequest.questions.isNotEmpty()
) interactionRequest.withAnswers(previous.answers + interactionRequest.answers, previous)
else interactionRequest
if (previous?.kind != interactionRequest.kind ||
previous.requestId != interactionRequest.requestId
) {
@@ -3881,6 +3907,7 @@ class GatewayChatClient(
// be replayed or buffered. Only an authoritative expiry retires a
// detached ask; an explicit response is retired by its foreground VM.
if (explicitlyExpired) {
pendingAsk.ownershipToken.retired.set(true)
backgroundTurn.pendingAsk = null
callbackDispatcher {
backgroundInteractionListener?.invoke(
@@ -3998,6 +4025,9 @@ class GatewayChatClient(
}
return
}
if (type == "session.info" && eventSessionId != null && eventSessionId == liveSessionId) {
payload?.let(turn::restorePendingClarify)
}
turn.onEvent(type, payload)
if (turn.ended) {
if (activeTurn === turn) activeTurn = null
@@ -4204,6 +4234,7 @@ class GatewayChatClient(
activated.isSuccess -> {
activated.getOrNull()?.let { result ->
applySessionResultInfo(result)
turn.restorePendingClarify(result)
turn.settleFromAuthoritativeSessionState(
running = result.booleanField("running"),
source = "session.activate",
@@ -4506,6 +4537,19 @@ class GatewayChatClient(
fun acknowledgeInteraction(expiry: GatewayAskExpiry) {
mapper.acknowledgeInteraction(expiry)
}
val interactionGeneration: Long get() = mapper.interactionGeneration
fun acknowledgeClarify(requestId: String, questionId: String?, answer: String, expired: Boolean, generation: Long) {
mapper.acknowledgeClarify(requestId, questionId, answer, expired, generation)
}
fun acknowledgeClarifyOwner(requestId: String, questionId: String?, answer: String, expired: Boolean, owner: GatewayAskOwnership) {
mapper.acknowledgeClarifyOwner(requestId, questionId, answer, expired, owner)
}
fun restorePendingClarify(snapshot: JsonObject) {
val payload = snapshot["pending_clarify"] as? JsonObject
?: (snapshot["info"] as? JsonObject)?.get("pending_clarify") as? JsonObject
?: return
GatewayEventMapper.interactionRequest("clarify.request", payload)?.let(mapper::restoreInteraction)
}
private val deferredEventLock = Any()
private val deferredEvents = mutableListOf<Pair<String, JsonObject?>>()
private var eventsDeferred = deferEvents
@@ -32,19 +32,66 @@ class GatewayEventMapper(
var turnEnded: Boolean = false
private set
@get:Synchronized
internal val currentInteraction: GatewayAsk?
get() = pendingInteraction
@Volatile internal var interactionGeneration: Long = 0
private set
internal fun restoreInteraction(ask: GatewayAsk) {
val duplicate = pendingInteraction?.sameRequestAs(ask) == true
pendingInteraction = ask
if (!duplicate) callbacks.onInteractionRequest(ask)
@Synchronized internal fun restoreInteraction(ask: GatewayAsk) {
val previous = pendingInteraction
val duplicate = previous?.sameRequestAs(ask) == true
if (!duplicate) interactionGeneration++
val merged = if (duplicate && ask.questions.isNotEmpty()) {
ask.withAnswers(previous.answers + ask.answers, previous)
} else if (ask.questions.isNotEmpty()) ask.withAnswers(emptyMap()) else ask
if (merged.ownershipToken.retired.get() && !merged.clarifyComplete) {
if (duplicate) pendingInteraction = null
callbacks.onInteractionExpired(GatewayAskExpiry(merged.kind, merged.requestId))
if (pendingInteraction == null) drainDeferredTerminalEvent()
return
}
pendingInteraction = merged
if (!duplicate || previous != merged) callbacks.onInteractionRequest(merged)
if (merged.clarifyComplete) {
pendingInteraction = null
drainDeferredTerminalEvent()
}
}
@Synchronized internal fun acknowledgeClarify(
requestId: String,
questionId: String?,
answer: String,
expired: Boolean,
generation: Long = interactionGeneration,
) {
if (generation != interactionGeneration) return
val pending = pendingInteraction ?: return
if (pending.kind != GatewayAsk.Kind.CLARIFY || pending.requestId != requestId) return
if (!expired && questionId != null && pending.questions.none { it.qid == questionId }) return
if (!expired && questionId != null && pending.questions.any { it.qid == questionId }) {
val updated = pending.withAnswers(pending.answers + (questionId to answer))
if (updated.questions.any { it.qid !in updated.answers }) {
pendingInteraction = updated
return
}
}
acknowledgeInteraction(GatewayAskExpiry(GatewayAsk.Kind.CLARIFY, requestId))
}
@Synchronized internal fun acknowledgeClarifyOwner(
requestId: String, questionId: String?, answer: String, expired: Boolean, owner: GatewayAskOwnership,
) {
if (pendingInteraction?.ownershipToken !== owner) return
acknowledgeClarify(requestId, questionId, answer, expired)
}
/** Retire only the ask whose explicit respond RPC reached server truth. */
internal fun acknowledgeInteraction(expiry: GatewayAskExpiry) {
@Synchronized internal fun acknowledgeInteraction(expiry: GatewayAskExpiry) {
val pending = pendingInteraction ?: return
if (pending.matches(expiry)) {
pending.ownershipToken.retired.set(true)
pendingInteraction = null
drainDeferredTerminalEvent()
}
@@ -77,7 +124,7 @@ class GatewayEventMapper(
*/
private val generatingIdsByName = mutableMapOf<String, ArrayDeque<String>>()
fun onEvent(type: String, payload: JsonObject?) {
@Synchronized fun onEvent(type: String, payload: JsonObject?) {
if (turnEnded) return
interactionRequest(type, payload)?.let { ask ->
@@ -88,6 +135,7 @@ class GatewayEventMapper(
interactionExpiry(type, payload)?.let { expiry ->
val pending = pendingInteraction
if (pending != null && pending.matches(expiry)) {
pending.ownershipToken.retired.set(true)
pendingInteraction = null
}
callbacks.onInteractionExpired(expiry)
@@ -503,26 +551,7 @@ class GatewayEventMapper(
}
fun interactionRequest(type: String, payload: JsonObject?): GatewayAsk? = when (type) {
"clarify.request" -> {
val choices = (payload?.get("choices") as? JsonArray)
?.mapNotNull { (it as? JsonPrimitive)?.contentOrNull?.trim() }
?.filter { it.isNotEmpty() }
?.distinct()
?.take(MAX_CLARIFY_CHOICES)
?.takeIf { it.isNotEmpty() }
GatewayAsk(
kind = GatewayAsk.Kind.CLARIFY,
requestId = payload.string("request_id"),
text = payload.string("question") ?: "The agent needs clarification",
choices = choices,
multiSelect = payload.boolean("multi_select") == true && choices != null,
// Current upstream owns expiry through clarify.expire and
// does not advertise its configurable deadline. Never
// invent a local deadline; consume future additive
// metadata only when it is present and positive.
timeoutSeconds = payload.int("timeout_seconds")?.coerceAtLeast(0) ?: 0,
)
}
"clarify.request" -> clarifyRequest(payload)
"approval.request" -> GatewayAsk(
kind = GatewayAsk.Kind.APPROVAL,
@@ -555,6 +584,41 @@ class GatewayEventMapper(
else -> null
}
private fun clarifyRequest(payload: JsonObject?): GatewayAsk? {
val rawQuestions = payload?.get("questions")
if (rawQuestions != null && rawQuestions !is JsonArray) return null
val questions = rawQuestions?.map { value ->
val row = value as? JsonObject ?: return null
val qid = (row["qid"] as? JsonPrimitive)?.takeIf { it.isString }
?.contentOrNull?.takeIf(String::isNotBlank) ?: return null
val text = row.string("question")?.takeIf(String::isNotBlank) ?: return null
val choices = (row["choices"] as? JsonArray)?.mapNotNull {
(it as? JsonPrimitive)?.takeIf { option -> option.isString }
?.contentOrNull?.takeIf(String::isNotBlank)
}.orEmpty().take(MAX_CLARIFY_CHOICES)
GatewayClarifyQuestion(qid, text, choices, row.boolean("multi_select") == true && choices.isNotEmpty())
}.orEmpty()
if (questions.size > MAX_CLARIFY_QUESTIONS || questions.map { it.qid }.distinct().size != questions.size) return null
val choices = (payload?.get("choices") as? JsonArray)
?.mapNotNull { (it as? JsonPrimitive)?.contentOrNull?.trim() }
?.filter(String::isNotEmpty)?.distinct()?.take(MAX_CLARIFY_CHOICES)
?.takeIf { it.isNotEmpty() }
return GatewayAsk(
kind = GatewayAsk.Kind.CLARIFY,
requestId = payload.string("request_id"),
text = payload.string("question") ?: "The agent needs clarification",
choices = choices,
multiSelect = payload.boolean("multi_select") == true && choices != null,
// Upstream owns its configurable deadline; only consume advertised metadata.
timeoutSeconds = payload.int("timeout_seconds")?.coerceAtLeast(0) ?: 0,
questions = questions,
answers = (payload?.get("answers") as? JsonObject)?.mapNotNull { (qid, value) ->
(value as? JsonPrimitive)?.takeIf { it.isString }?.contentOrNull
?.takeIf { questions.any { q -> q.qid == qid } }?.let { qid to it }
}?.toMap().orEmpty(),
)
}
fun interactionExpiry(type: String, payload: JsonObject?): GatewayAskExpiry? = when (type) {
"clarify.expire" -> GatewayAskExpiry(
kind = GatewayAsk.Kind.CLARIFY,
@@ -634,6 +698,7 @@ class GatewayEventMapper(
// Upstream clarify tool accepts at most four choices. Sudo/secret retain fixed
// `_block()` timeouts; clarify is configurable and expires authoritatively.
private const val MAX_CLARIFY_CHOICES = 4
private const val MAX_CLARIFY_QUESTIONS = 5
private const val SUDO_TIMEOUT_SECONDS = 120
private const val SECRET_TIMEOUT_SECONDS = 300
@@ -2,11 +2,14 @@ package com.hermesandroid.relay.network.upstream
import com.hermesandroid.relay.network.upstream.models.MessageItem
import com.hermesandroid.relay.network.upstream.models.UsageInfo
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.booleanOrNull
import kotlinx.serialization.json.contentOrNull
import java.util.concurrent.atomic.AtomicBoolean
import java.util.concurrent.atomic.AtomicReference
/**
* Shared types for the Gateway chat transport — upstream hermes-agent's
@@ -230,10 +233,35 @@ data class GatewayAsk(
* authoritative `*.expire` event still retires the interaction.
*/
val timeoutSeconds: Int,
val questions: List<GatewayClarifyQuestion> = emptyList(),
val answers: Map<String, String> = emptyMap(),
) {
enum class Kind { CLARIFY, APPROVAL, SUDO, SECRET }
val clarifyComplete: Boolean get() = questions.isNotEmpty() && questions.all { it.qid in answers }
/** In-memory request incarnation shared when a live ask moves between turn mappers. */
internal var ownershipToken = GatewayAskOwnership(answers)
private set
internal fun withAnswers(answers: Map<String, String>, owner: GatewayAsk = this): GatewayAsk =
copy(answers = owner.ownershipToken.answers.updateAndGet { it + answers })
.also { it.ownershipToken = owner.ownershipToken }
}
/** A detached/reclaimed mapper shares confirmed progress with an RPC still owned by its predecessor. */
internal class GatewayAskOwnership(answers: Map<String, String>) {
val answers = AtomicReference(answers.toMap())
val retired = AtomicBoolean(false)
}
@Serializable
data class GatewayClarifyQuestion(
val qid: String,
val question: String,
val choices: List<String> = emptyList(),
val multiSelect: Boolean = false,
)
/**
* Server-side expiry of one blocking gateway interaction. Sudo/secret asks
* correlate by [requestId]; approvals remain session-scoped and therefore
@@ -0,0 +1,103 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.material3.LinearProgressIndicator
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.key
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalFocusManager
import androidx.compose.ui.res.stringResource
import androidx.compose.ui.semantics.LiveRegionMode
import androidx.compose.ui.semantics.heading
import androidx.compose.ui.semantics.liveRegion
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.R
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import kotlinx.serialization.json.Json
/** One mobile question surface; confirmed qids advance without resubmitting earlier answers. */
@Composable
internal fun ClarifyBatchContent(
batch: HermesCardClarifyBatch,
expired: Boolean,
onInputSubmit: (String, String) -> Unit,
) {
val answered = batch.questions.filter { it.answer != null }
val activeIndex = batch.questions.indexOfFirst { it.answer == null }
val active = batch.questions.getOrNull(activeIndex)
var showAnswers by rememberSaveable { mutableStateOf(false) }
val focusManager = LocalFocusManager.current
LaunchedEffect(active?.key, expired) { focusManager.clearFocus() }
Column(Modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text(
text = when {
expired -> stringResource(R.string.clarify_batch_expired)
active == null -> stringResource(R.string.clarify_batch_complete)
else -> stringResource(R.string.clarify_batch_progress, activeIndex + 1, batch.questions.size)
},
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.semantics { liveRegion = LiveRegionMode.Polite },
)
if (batch.questions.size > 1) {
LinearProgressIndicator(
progress = { answered.size.toFloat() / batch.questions.size },
modifier = Modifier.fillMaxWidth(),
)
}
if (active != null && !expired) {
key(active.key) {
Text(
active.question,
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface,
modifier = Modifier.semantics { heading() },
)
CardInputSlot(
input = active.input,
onSubmit = { if (!active.submitting) onInputSubmit(active.key, it) },
enabled = !active.submitting,
stackedChoices = true,
)
if (active.submitting) {
Text(stringResource(R.string.clarify_batch_sending), style = MaterialTheme.typography.labelMedium)
}
}
}
if (answered.isNotEmpty()) {
if (active == null || expired) {
Text(stringResource(R.string.clarify_batch_answered, answered.size), style = MaterialTheme.typography.labelLarge)
} else {
TextButton(onClick = { showAnswers = !showAnswers }) {
Text(stringResource(R.string.clarify_batch_answered, answered.size))
}
}
if (showAnswers || active == null || expired) {
answered.forEach { question ->
Text(question.question, style = MaterialTheme.typography.labelLarge)
val answer = if (question.input.multiSelect) {
runCatching { Json.decodeFromString<List<String>>(question.answer.orEmpty()).joinToString(", ") }
.getOrDefault(question.answer.orEmpty())
} else question.answer.orEmpty()
Text(answer, style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant)
Spacer(Modifier.height(4.dp))
}
}
}
}
}
@@ -59,6 +59,7 @@ import androidx.compose.runtime.mutableLongStateOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
@@ -145,7 +146,7 @@ fun HermesCardBubble(
// Expiry clock for timed asks. Ticks once a second while the deadline
// is ahead; freezes after. Keyed on the deadline so a re-used card id
// with a fresh expiry restarts the loop.
val expiresAt = card.input?.expiresAtMillis
val expiresAt = card.input?.expiresAtMillis ?: card.clarifyBatch?.expiresAtMillis
var nowMillis by remember(expiresAt) { mutableLongStateOf(System.currentTimeMillis()) }
LaunchedEffect(expiresAt) {
if (expiresAt == null) return@LaunchedEffect
@@ -252,6 +253,14 @@ fun HermesCardBubble(
// action button all collapse the same way.
val input = card.input
when {
card.clarifyBatch != null -> {
Spacer(Modifier.height(10.dp))
ClarifyBatchContent(
batch = card.clarifyBatch,
expired = expired || alreadyChosen?.actionValue == HermesCardDispatch.EXPIRED_STAMP,
onInputSubmit = onInputSubmit,
)
}
alreadyChosen != null -> {
Spacer(Modifier.height(10.dp))
val chosenAction = card.actions.firstOrNull {
@@ -408,15 +417,18 @@ private fun ChoseRow(
*/
@OptIn(ExperimentalLayoutApi::class)
@Composable
private fun CardInputSlot(
internal fun CardInputSlot(
input: HermesCardInput,
onSubmit: (String) -> Unit,
enabled: Boolean = true,
stackedChoices: Boolean = false,
) {
// Deliberately remember, not rememberSaveable — a typed secret must
// never be written into the saved-instance-state Bundle.
var answerText by remember { mutableStateOf("") }
// Batch drafts survive lazy-item disposal. Secrets never enter saved state.
var answerText by if (stackedChoices && !input.masked) rememberSaveable { mutableStateOf("") }
else remember { mutableStateOf("") }
var reveal by remember { mutableStateOf(false) }
var selectedChoices by remember(input.choices) { mutableStateOf(emptyList<String>()) }
var selectedChoices by if (stackedChoices) rememberSaveable(input.choices) { mutableStateOf(emptyList<String>()) }
else remember(input.choices) { mutableStateOf(emptyList<String>()) }
val isMultiSelect = input.multiSelect && input.choices.isNotEmpty()
val showFreeText = !input.masked && (
@@ -429,7 +441,7 @@ private fun CardInputSlot(
val submitFreeText = {
val customAnswer = answerText.trim()
if (customAnswer.isNotEmpty()) {
if (enabled && customAnswer.isNotEmpty()) {
onSubmit(
if (isMultiSelect) {
encodeClarifyMultiSelectAnswer(selectedChoices + customAnswer)
@@ -443,14 +455,13 @@ private fun CardInputSlot(
Column(modifier = Modifier.fillMaxWidth()) {
// Choice chips
if (input.choices.isNotEmpty()) {
FlowRow(
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
val choices: @Composable () -> Unit = {
input.choices.forEach { choice ->
if (isMultiSelect) {
val selected = choice in selectedChoices
FilterChip(
enabled = enabled,
modifier = if (stackedChoices) Modifier.fillMaxWidth() else Modifier,
selected = selected,
onClick = {
selectedChoices = if (selected) {
@@ -459,7 +470,11 @@ private fun CardInputSlot(
selectedChoices + choice
}
},
label = { Text(choice, style = MaterialTheme.typography.labelMedium) },
label = {
Text(choice,
style = if (stackedChoices) MaterialTheme.typography.bodyMedium else MaterialTheme.typography.labelMedium,
modifier = if (stackedChoices) Modifier.padding(vertical = 8.dp) else Modifier)
},
leadingIcon = if (selected) {
{
Icon(
@@ -477,9 +492,13 @@ private fun CardInputSlot(
)
} else {
AssistChip(
enabled = enabled,
modifier = if (stackedChoices) Modifier.fillMaxWidth() else Modifier,
onClick = { onSubmit(choice) },
label = {
Text(choice, style = MaterialTheme.typography.labelMedium)
Text(choice,
style = if (stackedChoices) MaterialTheme.typography.bodyMedium else MaterialTheme.typography.labelMedium,
modifier = if (stackedChoices) Modifier.padding(vertical = 8.dp) else Modifier)
},
colors = AssistChipDefaults.assistChipColors(
containerColor = MaterialTheme.colorScheme.surfaceContainerHighest,
@@ -489,6 +508,11 @@ private fun CardInputSlot(
}
}
}
if (stackedChoices) {
Column(Modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(4.dp)) { choices() }
} else {
FlowRow(horizontalArrangement = Arrangement.spacedBy(8.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) { choices() }
}
}
// Masked secret field
@@ -534,12 +558,13 @@ private fun CardInputSlot(
else R.string.card_answer_placeholder,
),
onSubmit = submitFreeText,
enabled = enabled,
modifier = Modifier.weight(1f),
)
if (!isMultiSelect) {
IconButton(
onClick = submitFreeText,
enabled = answerText.isNotBlank(),
enabled = enabled && answerText.isNotBlank(),
) {
Icon(
imageVector = Icons.AutoMirrored.Filled.Send,
@@ -559,7 +584,7 @@ private fun CardInputSlot(
Spacer(Modifier.height(10.dp))
Button(
onClick = { onSubmit(encodeClarifyMultiSelectAnswer(answers)) },
enabled = answers.isNotEmpty(),
enabled = enabled && answers.isNotEmpty(),
) {
Text(
stringResource(R.string.card_submit),
@@ -615,6 +640,7 @@ private fun InlineAnswerField(
placeholder: String,
onSubmit: () -> Unit,
modifier: Modifier = Modifier,
enabled: Boolean = true,
) {
val shape = appearanceRoundedCornerShape(16.dp)
Box(
@@ -632,6 +658,7 @@ private fun InlineAnswerField(
)
}
BasicTextField(
enabled = enabled,
value = value,
onValueChange = onValueChange,
textStyle = MaterialTheme.typography.bodyMedium.copy(
@@ -639,7 +666,7 @@ private fun InlineAnswerField(
),
cursorBrush = SolidColor(MaterialTheme.colorScheme.primary),
keyboardOptions = KeyboardOptions(imeAction = ImeAction.Send),
keyboardActions = KeyboardActions(onSend = { onSubmit() }),
keyboardActions = KeyboardActions(onSend = { onSubmit() }, onDone = { onSubmit() }),
maxLines = 3,
modifier = Modifier
.fillMaxWidth()
@@ -62,6 +62,7 @@ import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.RectangleShape
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.hapticfeedback.HapticFeedbackType
import androidx.compose.ui.platform.LocalContext
@@ -214,7 +215,7 @@ fun MessageBubble(
message.role == MessageRole.USER -> MaterialTheme.colorScheme.primary
message.role == MessageRole.SYSTEM -> MaterialTheme.colorScheme.tertiaryContainer
isActionBubble -> MaterialTheme.colorScheme.tertiaryContainer.copy(alpha = 0.45f)
else -> MaterialTheme.colorScheme.surfaceVariant
else -> MaterialTheme.colorScheme.surfaceContainerLow
}
val textColor = when (message.role) {
@@ -436,8 +437,8 @@ fun MessageBubble(
// rows) would otherwise paint a bare timestamp-only chip between the
// Thought-process block and the tool pill. The first-token working state
// is rendered directly in the conversation
// lane below, without an opaque bubble. Cards and attachments still own
// a normal bubble even when response prose has not arrived yet.
// lane below, without an opaque bubble. Standalone cards own their own
// surface; wrapping those in another filled bubble duplicates the chrome.
streamingStatusLabel?.takeIf { showWorkingStatus }?.let { streamingStatus ->
StandaloneStreamingStatus(
status = streamingStatus,
@@ -454,6 +455,10 @@ fun MessageBubble(
message.cards.isNotEmpty() ||
message.attachments.isNotEmpty() ||
inlineImages.isNotEmpty()
val standaloneCards = !isUser && !isSystem &&
visibleMessageContent.isBlank() && quoteEnvelope == null &&
message.cards.isNotEmpty() && message.attachments.isEmpty() &&
inlineImages.isEmpty() && !showImageGeneration
if (showBubble) {
Row(
modifier = Modifier.widthIn(max = maxBubbleWidth),
@@ -599,11 +604,11 @@ fun MessageBubble(
),
) {
Surface(
shape = bubbleShape,
color = backgroundColor,
shape = if (standaloneCards) RectangleShape else bubbleShape,
color = if (standaloneCards) Color.Transparent else backgroundColor,
modifier = Modifier
.then(
if (!isUser && !isSystem && isDarkTheme) {
if (!isUser && !isSystem && isDarkTheme && !standaloneCards) {
Modifier.leftEdgeGlow(
alpha = 0.12f,
width = 28.dp,
@@ -651,7 +656,8 @@ fun MessageBubble(
)
) {
Column(
modifier = Modifier.padding(horizontal = 14.dp, vertical = 9.dp),
modifier = if (standaloneCards) Modifier
else Modifier.padding(horizontal = 14.dp, vertical = 9.dp),
) {
quoteEnvelope?.let { envelope ->
ChatQuoteReferenceChip(
@@ -741,7 +747,7 @@ fun MessageBubble(
onInputSubmit = { key, value ->
onCardInput(message.id, key, value)
},
maxWidth = maxBubbleWidth - 24.dp,
maxWidth = if (standaloneCards) maxBubbleWidth else maxBubbleWidth - 24.dp,
modifier = Modifier.padding(vertical = 2.dp),
)
}
@@ -823,6 +829,7 @@ fun MessageBubble(
val hasTokenUsage = showUsage && !isUser &&
(message.inputTokens != null || message.outputTokens != null)
val deliveryStatus = message.deliveryStatus?.takeIf { isUser }
// Timestamp — only on the LAST bubble of a same-author run so a
// burst of fragments doesn't stack three near-touching time labels.
@@ -831,13 +838,14 @@ fun MessageBubble(
// This row is reserved from the first streaming frame. Completion
// can reveal both timestamp and token usage without adding a new
// footer line or changing the bubble's measured height.
if (isLastInGroup && (showTimestamps || hasTokenUsage)) {
if ((isLastInGroup && (showTimestamps || hasTokenUsage)) || deliveryStatus != null) {
Spacer(modifier = Modifier.height(2.dp))
Row(
horizontalArrangement = Arrangement.spacedBy(6.dp),
verticalAlignment = Alignment.CenterVertically,
modifier = if (standaloneCards) Modifier.padding(horizontal = 4.dp) else Modifier,
) {
if (showTimestamps) Text(
if (isLastInGroup && showTimestamps) Text(
text = timeFormat.format(Date(message.timestamp)),
style = MaterialTheme.typography.labelSmall,
color = textColor.copy(alpha = if (message.isStreaming) 0f else 0.6f),
@@ -847,33 +855,30 @@ fun MessageBubble(
Modifier
},
)
if (hasTokenUsage) {
if (isLastInGroup && hasTokenUsage) {
TokenDisplay(
inputTokens = message.inputTokens,
outputTokens = message.outputTokens,
)
}
// Share the footer line, retaining the user bubble's contrasting foreground.
deliveryStatus?.let { status ->
MessageDeliveryIndicator(
status = status,
contentColor = textColor,
text = MessageDeliveryIndicatorText(
sending = stringResource(R.string.msg_bubble_sending),
queued = stringResource(R.string.msg_bubble_queued),
steered = stringResource(R.string.msg_bubble_steered),
delivered = stringResource(R.string.msg_bubble_delivered),
failed = stringResource(R.string.msg_bubble_not_sent),
tapToRetry = stringResource(R.string.chat_retry),
),
)
}
}
}
// Delivery status for local user messages, including steering.
// Use the bubble's foreground: accent-on-accent hides the label.
message.deliveryStatus?.takeIf { isUser }?.let { status ->
Spacer(modifier = Modifier.height(2.dp))
MessageDeliveryIndicator(
status = status,
contentColor = textColor,
text = MessageDeliveryIndicatorText(
sending = stringResource(R.string.msg_bubble_sending),
queued = stringResource(R.string.msg_bubble_queued),
steered = stringResource(R.string.msg_bubble_steered),
delivered = stringResource(R.string.msg_bubble_delivered),
failed = stringResource(R.string.msg_bubble_not_sent),
tapToRetry = stringResource(R.string.chat_retry),
),
)
}
// Non-tail historical fragments have no reserved timestamp row.
// Preserve their existing standalone token metadata layout.
if (!isLastInGroup && hasTokenUsage) {
@@ -72,6 +72,9 @@ import com.hermesandroid.relay.network.upstream.ActiveTurnKeepAliveRegistry
import com.hermesandroid.relay.network.upstream.GatewayAsk
import com.hermesandroid.relay.network.upstream.GatewayAskExpiry
import com.hermesandroid.relay.network.upstream.GatewayAskResponse
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import com.hermesandroid.relay.data.HermesCardClarifyQuestion
import com.hermesandroid.relay.data.clarifyQuestionCardKey
import com.hermesandroid.relay.network.upstream.GatewayAgentNotice
import com.hermesandroid.relay.network.upstream.GatewayActiveSession
import com.hermesandroid.relay.network.upstream.GatewayActiveSessionStatus
@@ -3652,7 +3655,7 @@ class ChatViewModel : ViewModel() {
private val backgroundPendingInteractions =
ConcurrentHashMap<TurnCheckpointKey, BackgroundPendingInteraction>()
/** Ask cardKeys with a respond RPC in flight — blocks double-taps until it settles. */
/** Request-incarnation/card keys with a respond RPC in flight. */
private val answeredAskIds = mutableSetOf<String>()
/**
@@ -6774,7 +6777,18 @@ class ChatViewModel : ViewModel() {
existing.ask.kind == ask.kind &&
existing.ask.requestId == ask.requestId
) {
sessionId?.let { maybeNotifyInteraction(it, existing.ask) }
if (ask.questions.isNotEmpty()) {
val updated = existing.copy(ask = ask.copy(answers = existing.ask.answers + ask.answers))
_pendingAsk.value = updated
updateClarifyBatchCard(handler, updated)
if (updated.ask.clarifyComplete) {
_pendingAsk.value = null
updated.sessionId?.let { cancelInteractionNotification(it, updated.ask) }
activeTurnCheckpointKey()?.let { ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false) }
}
scheduleCheckpointWrite(immediate = true)
}
_pendingAsk.value?.let { pending -> sessionId?.let { maybeNotifyInteraction(it, pending.ask) } }
return
}
existing?.let { pending ->
@@ -6787,8 +6801,11 @@ class ChatViewModel : ViewModel() {
publishBackgroundSessionActivity()
}
val now = restored?.receivedAt ?: System.currentTimeMillis()
val cardKey = restored?.cardKey ?: ask.requestId
val proposedCardKey = restored?.cardKey ?: ask.requestId
?: "approval-${handler.currentSessionId.value ?: "session"}-$now"
val cardKey = if (restored == null && handler.messages.value.any { message ->
message.cards.any { it.id == proposedCardKey }
}) "$proposedCardKey-${java.util.UUID.randomUUID()}" else proposedCardKey
val expiresAt = ask.timeoutSeconds.takeIf { it > 0 }?.let { now + it * 1_000L }
val card = when (ask.kind) {
GatewayAsk.Kind.APPROVAL -> HermesCard(
@@ -6887,12 +6904,52 @@ class ChatViewModel : ViewModel() {
contextKey = contextKey,
sessionId = sessionId,
receivedAt = now,
ownerId = restored?.ownerId ?: java.util.UUID.randomUUID().toString(),
)
if (ask.questions.isNotEmpty()) updateClarifyBatchCard(handler, requireNotNull(_pendingAsk.value))
if (ask.clarifyComplete) {
_pendingAsk.value = null
activeKey?.let { ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false) }
scheduleCheckpointWrite(immediate = true)
return
}
activeKey?.let { ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), true) }
scheduleCheckpointWrite(immediate = true)
sessionId?.let { maybeNotifyInteraction(it, ask) }
}
private fun updateClarifyBatchCard(handler: ChatHandler, pending: PendingAsk) {
val ask = pending.ask
if (ask.questions.isEmpty()) return
handler.updateAskCardMessage(
pending.messageId,
HermesCard(
type = HermesCard.BuiltInTypes.ASK_CLARIFY,
title = appContext?.getString(R.string.chat_approval_clarify_title) ?: "Hermes needs clarification",
accent = HermesCard.Accents.INFO,
id = pending.cardKey,
clarifyBatch = HermesCardClarifyBatch(
questions = ask.questions.map { question ->
val key = clarifyQuestionCardKey(pending.cardKey, question.qid)
HermesCardClarifyQuestion(
key = key,
question = question.question,
input = HermesCardInput(
kind = if (question.choices.isEmpty()) HermesCardInput.Kinds.TEXT else HermesCardInput.Kinds.CHOICE,
choices = question.choices,
multiSelect = question.multiSelect,
allowFreeText = true,
),
answer = ask.answers[question.qid],
submitting = "${pending.ownerId}:$key" in answeredAskIds,
)
},
expiresAtMillis = ask.timeoutSeconds.takeIf { it > 0 }?.let { pending.receivedAt + it * 1_000L },
),
),
)
}
/**
* Supervised Chat never exposes approval, clarification, sudo, or secret
* inputs. Settle the upstream interaction immediately with its safest
@@ -6966,21 +7023,35 @@ class ChatViewModel : ViewModel() {
fun answerAsk(messageId: String, cardKey: String, value: String) {
val handler = chatHandler ?: return
val pending = _pendingAsk.value
val question = pending?.ask?.questions?.firstOrNull {
clarifyQuestionCardKey(pending.cardKey, it.qid) == cardKey
}
if (pending == null ||
pending.cardKey != cardKey ||
pending.messageId != messageId ||
(if (pending.ask.questions.isNotEmpty()) question == null else pending.cardKey != cardKey) ||
pending.contextKey != activeProfileContextKey ||
pending.sessionId != handler.currentSessionId.value
) {
handler.addSystemNotice("This request is no longer active.")
return
}
if (pending.ask.kind == GatewayAsk.Kind.CLARIFY && value.isBlank()) return
if (pending.ask.kind == GatewayAsk.Kind.CLARIFY && pending.ask.timeoutSeconds > 0 &&
System.currentTimeMillis() >= pending.receivedAt + pending.ask.timeoutSeconds * 1_000L
) {
expirePendingAsk(GatewayAskExpiry(pending.ask.kind, pending.ask.requestId))
return
}
if (question != null && question.qid in pending.ask.answers) return
val gateway = gatewayClient
if (gateway == null) {
emitError(Exception("Gateway is not connected"), context = "send_message")
return
}
// In-flight guard: one respond RPC per card at a time.
if (!answeredAskIds.add(cardKey)) return
// Include the request incarnation so a late completion cannot unlock a reused id.
val flightKey = "${pending.ownerId}:$cardKey"
if (!answeredAskIds.add(flightKey)) return
updateClarifyBatchCard(handler, pending)
val ask = pending.ask
val stampValue = when (ask.kind) {
// Empty sudo password = decline — stamp matches the Deny action
@@ -6992,11 +7063,19 @@ class ChatViewModel : ViewModel() {
else -> value
}
viewModelScope.launch {
fun ownsResponse(): Boolean = chatHandler === handler && gatewayClient === gateway &&
activeProfileContextKey == pending.contextKey &&
handler.currentSessionId.value == pending.sessionId &&
_pendingAsk.value?.ownerId == pending.ownerId
if (!ownsResponse()) {
answeredAskIds.remove(flightKey)
return@launch
}
val requestId = ask.requestId
val result = when (ask.kind) {
GatewayAsk.Kind.APPROVAL -> gateway.respondApproval(choice = value)
GatewayAsk.Kind.CLARIFY ->
requestId?.let { gateway.respondClarify(it, value) }
requestId?.let { gateway.respondClarify(it, value.trim(), question?.qid) }
?: Result.failure(GatewayRpcException("ask has no request id"))
GatewayAsk.Kind.SUDO ->
requestId?.let { gateway.respondSudo(it, value) }
@@ -7007,6 +7086,8 @@ class ChatViewModel : ViewModel() {
}
result.fold(
onSuccess = { response ->
answeredAskIds.remove(flightKey)
if (!ownsResponse()) return@fold
if (response == GatewayAskResponse.EXPIRED) {
expirePendingAsk(
GatewayAskExpiry(
@@ -7016,10 +7097,24 @@ class ChatViewModel : ViewModel() {
)
return@fold
}
if (question != null) {
val current = requireNotNull(_pendingAsk.value)
val updated = current.copy(ask = current.ask.copy(answers = current.ask.answers + (question.qid to value.trim())))
updateClarifyBatchCard(handler, updated)
if (updated.ask.questions.all { it.qid in updated.ask.answers }) {
updated.sessionId?.let { cancelInteractionNotification(it, updated.ask) }
_pendingAsk.value = null
activeTurnCheckpointKey()?.let { ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false) }
} else {
_pendingAsk.value = updated
}
scheduleCheckpointWrite(immediate = true)
return@fold
}
// Collapse only after the server confirms — a failed RPC
// must leave the card answerable for a retry.
handler.recordCardDispatch(pending.messageId, cardKey, stampValue)
if (_pendingAsk.value === pending) {
if (ownsResponse()) {
pending.sessionId?.let { cancelInteractionNotification(it, pending.ask) }
_pendingAsk.value = null
activeTurnCheckpointKey()?.let {
@@ -7029,7 +7124,9 @@ class ChatViewModel : ViewModel() {
}
},
onFailure = { e ->
answeredAskIds.remove(cardKey)
answeredAskIds.remove(flightKey)
if (!ownsResponse()) return@fold
updateClarifyBatchCard(handler, requireNotNull(_pendingAsk.value))
emitError(e, context = "send_message")
},
)
@@ -7054,7 +7151,6 @@ class ChatViewModel : ViewModel() {
activeTurnCheckpointKey()?.let {
ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false)
}
answeredAskIds.remove(pending.cardKey)
scheduleCheckpointWrite(immediate = true)
chatHandler?.recordCardDispatch(
pending.messageId,
@@ -7077,7 +7173,9 @@ class ChatViewModel : ViewModel() {
ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false)
}
scheduleCheckpointWrite(immediate = true)
if (pending.ask.kind == GatewayAsk.Kind.APPROVAL) {
if (pending.ask.kind == GatewayAsk.Kind.CLARIFY) {
chatHandler?.recordCardDispatch(pending.messageId, pending.cardKey, HermesCardDispatch.EXPIRED_STAMP)
} else if (pending.ask.kind == GatewayAsk.Kind.APPROVAL) {
chatHandler?.recordCardDispatch(pending.messageId, pending.cardKey, "deny")
}
}
@@ -7685,6 +7783,9 @@ class ChatViewModel : ViewModel() {
text = ask.ask.text,
choices = ask.ask.choices,
multiSelect = ask.ask.multiSelect,
questions = ask.ask.questions,
answers = ask.ask.answers,
ownerId = ask.ownerId,
smartDenied = ask.ask.smartDenied,
envVar = ask.ask.envVar,
timeoutSeconds = ask.ask.timeoutSeconds,
@@ -7893,6 +7994,8 @@ class ChatViewModel : ViewModel() {
text = saved.text,
choices = saved.choices,
multiSelect = saved.multiSelect,
questions = saved.questions,
answers = saved.answers,
smartDenied = saved.smartDenied,
envVar = saved.envVar,
timeoutSeconds = saved.timeoutSeconds,
@@ -11846,6 +11949,7 @@ data class PendingAsk(
/** Stored session id within [contextKey]; approvals are session-scoped upstream. */
val sessionId: String?,
val receivedAt: Long = System.currentTimeMillis(),
val ownerId: String = java.util.UUID.randomUUID().toString(),
)
/**
@@ -1,5 +1,10 @@
<?xml version='1.0' encoding='utf-8'?>
<resources>
<string name="clarify_batch_progress">Pergunta %1$d de %2$d</string>
<string name="clarify_batch_complete">Todas as perguntas respondidas</string>
<string name="clarify_batch_expired">Esta solicitação terminou</string>
<string name="clarify_batch_sending">Enviando resposta…</string>
<string name="clarify_batch_answered">Perguntas respondidas (%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">interface do agente</string>
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">问题 %1$d / %2$d</string>
<string name="clarify_batch_complete">所有问题均已回答</string>
<string name="clarify_batch_expired">此请求已结束</string>
<string name="clarify_batch_sending">正在发送回答…</string>
<string name="clarify_batch_answered">已回答的问题(%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">代理界面</string>
+5
View File
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">Frage %1$d von %2$d</string>
<string name="clarify_batch_complete">Alle Fragen beantwortet</string>
<string name="clarify_batch_expired">Diese Anfrage ist beendet</string>
<string name="clarify_batch_sending">Antwort wird gesendet…</string>
<string name="clarify_batch_answered">Beantwortete Fragen (%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">Agentenoberfläche</string>
+5
View File
@@ -1,5 +1,10 @@
<?xml version='1.0' encoding='utf-8'?>
<resources>
<string name="clarify_batch_progress">Pregunta %1$d de %2$d</string>
<string name="clarify_batch_complete">Todas las preguntas respondidas</string>
<string name="clarify_batch_expired">Esta solicitud ha finalizado</string>
<string name="clarify_batch_sending">Enviando respuesta…</string>
<string name="clarify_batch_answered">Preguntas respondidas (%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">interfaz del agente</string>
+5
View File
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">質問 %1$d / %2$d</string>
<string name="clarify_batch_complete">すべての質問に回答しました</string>
<string name="clarify_batch_expired">このリクエストは終了しました</string>
<string name="clarify_batch_sending">回答を送信中…</string>
<string name="clarify_batch_answered">回答済みの質問(%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">エージェントインターフェース</string>
+5
View File
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">Вопрос %1$d из %2$d</string>
<string name="clarify_batch_complete">Все вопросы отвечены</string>
<string name="clarify_batch_expired">Этот запрос завершён</string>
<string name="clarify_batch_sending">Отправка ответа…</string>
<string name="clarify_batch_answered">Отвеченные вопросы (%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">интерфейс агента</string>
+5
View File
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">Question %1$d of %2$d</string>
<string name="clarify_batch_complete">All questions answered</string>
<string name="clarify_batch_expired">This request has ended</string>
<string name="clarify_batch_sending">Sending answer…</string>
<string name="clarify_batch_answered">Answered questions (%1$d)</string>
<string name="chat_debug_title" translatable="false">Session diagnostics</string>
<string name="chat_debug_open" translatable="false">Open session diagnostics</string>
<string name="chat_debug_close" translatable="false">Close session diagnostics</string>
@@ -56,6 +56,21 @@ class ChatTurnCheckpointStoreTest {
assertEquals(checkpoint, store.read())
}
@Test
fun partialClarifyBatch_roundTripsWithExactQuestionOwnership() = runTest {
val checkpoint = sampleCheckpoint().copy(pendingAsk = ChatTurnAskCheckpoint(
kind = "CLARIFY", requestId = "batch", text = "", timeoutSeconds = 0,
messageId = "ask-batch", cardKey = "batch", receivedAt = now,
questions = listOf(
com.hermesandroid.relay.network.upstream.GatewayClarifyQuestion("route/a", "Which route?", listOf("Canary")),
com.hermesandroid.relay.network.upstream.GatewayClarifyQuestion("environment:b", "Which environments?", listOf("Stage", "Production"), true),
), answers = mapOf("route/a" to "Canary"),
))
store.write(checkpoint)
assertEquals(checkpoint, store.read())
assertEquals(mapOf("route/a" to "Canary"), store.read()?.pendingAsk?.answers)
}
@Test
fun corruptJson_isDiscarded() = runTest {
dataStore.edit { preferences ->
@@ -74,6 +74,8 @@ class GatewayClientHarness(
@Volatile
var recoveryRunning = false
@Volatile
var recoveryClarify: JsonObject? = null
@Volatile
var recoveryAssistant = ""
@@ -586,6 +588,7 @@ class GatewayClientHarness(
private val autoRespondEnabled = autoRespond
private fun recoveryPayload(sessionId: String, requestedProfile: String? = null): JsonObject = buildJsonObject {
recoveryClarify?.let { put("pending_clarify", it) }
put("session_id", sessionId)
put("running", recoveryRunning)
put("status", if (recoveryRunning) "streaming" else "idle")
@@ -3165,6 +3168,75 @@ class GatewayChatClientTest {
// --- Ask responders ---
@Test
fun `batch clarify reconnect adopts server answered qids without resubmitting`() {
val r = Recorder()
client.sendTurn(null, "hi", null, r.callbacks) { r.preflightFailures += it }
val socket = harness.awaitServerSocket()
harness.awaitRpc("prompt.submit")
harness.recoveryRunning = true
harness.recoveryClarify = harness.json.parseToJsonElement("""{
"request_id":"batch","questions":[{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}],
"answers":{"q0":"accepted before disconnect"}}
""") as JsonObject
socket.close(1001, "fixture reconnect")
val replacement = harness.awaitServerSocket()
harness.awaitRpc("session.activate")
val deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(5)
while (r.interactions.isEmpty() && System.nanoTime() < deadline) Thread.sleep(10)
assertEquals(mapOf("q0" to "accepted before disconnect"), r.interactions.last().answers)
assertTrue(harness.rpcLog.none { it.first == "clarify.respond" })
assertTrue(runBlocking { client.respondClarify("batch", "second", "q1") }.isSuccess)
replacement.send(harness.eventFrame("message.complete", buildJsonObject { put("text", "done") }, "live-1"))
assertTrue(r.completeLatch.await(5, TimeUnit.SECONDS))
}
@Test
fun `batch clarify responds with exact question id and JSON array answer`() {
val r = Recorder()
client.sendTurn(null, "hi", null, r.callbacks) { r.preflightFailures += it }
val socket = harness.awaitServerSocket()
harness.awaitRpc("prompt.submit")
socket.send(harness.eventFrame("clarify.request", harness.json.parseToJsonElement("""
{"request_id":"batch","questions":[{"qid":"env:b","question":"Which environments?","choices":["Stage","Production"],"multi_select":true}]}
""") as JsonObject, "live-1"))
awaitCondition { r.interactions.isNotEmpty() }
val answer = "[\"Stage\",\"Production\"]"
assertTrue(runBlocking { client.respondClarify("batch", answer, "env:b") }.isSuccess)
val respond = harness.awaitRpc("clarify.respond")
assertEquals("batch", (respond["request_id"] as? JsonPrimitive)?.contentOrNull)
assertEquals("env:b", (respond["question_id"] as? JsonPrimitive)?.contentOrNull)
assertEquals(answer, (respond["answer"] as? JsonPrimitive)?.contentOrNull)
}
@Test
fun `batch answer waits for reconnect replay and refuses an already answered qid`() {
val r = Recorder()
client.sendTurn(null, "hi", null, r.callbacks) { r.preflightFailures += it }
val socket = harness.awaitServerSocket()
harness.awaitRpc("prompt.submit")
val request = harness.json.parseToJsonElement("""{"request_id":"batch","questions":[
{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}]}""") as JsonObject
socket.send(harness.eventFrame("clarify.request", request, "live-1"))
awaitCondition { r.interactions.isNotEmpty() }
harness.recoveryRunning = true
harness.suppressAckMethods += "session.activate"
socket.close(1001, "fixture lost acknowledgement")
harness.awaitServerSocket()
val activation = harness.awaitPendingAck()
assertEquals("session.activate", activation.method)
assertTrue(runBlocking { client.respondClarify("batch", "retry", "q0") }.isFailure)
assertTrue(harness.rpcLog.none { it.first == "clarify.respond" })
harness.releaseAck(activation, buildJsonObject {
put("session_id", "live-1")
put("running", true)
put("pending_clarify", JsonObject(request + ("answers" to buildJsonObject { put("q0", "accepted") })))
})
awaitCondition { r.interactions.last().answers["q0"] == "accepted" }
assertTrue(runBlocking { client.respondClarify("batch", "overwrite", "q0") }.isFailure)
assertTrue(harness.rpcLog.none { it.first == "clarify.respond" })
}
@Test
fun `clarify respond carries request id and answer`() {
val r = Recorder()
@@ -16,6 +16,103 @@ import org.junit.Test
*/
class GatewayEventMapperTest {
@Test
fun `acknowledgement before reclaim remains in the shared request snapshot`() {
val original = mapperWith(Recorder())
original.onEvent("clarify.request", obj("""{"request_id":"batch","questions":[
{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}]}"""))
val detachedSnapshot = requireNotNull(original.currentInteraction)
original.acknowledgeClarify("batch", "q0", "first", false)
val recovered = mapperWith(Recorder())
recovered.restoreInteraction(detachedSnapshot)
assertEquals(mapOf("q0" to "first"), recovered.currentInteraction?.answers)
recovered.onEvent("message.complete", obj("""{"text":"done"}"""))
recovered.acknowledgeClarify("batch", "q1", "second", false)
assertTrue(recovered.turnEnded)
}
@Test
fun `expired detached snapshot cannot become pending again`() {
val original = mapperWith(Recorder())
original.onEvent("clarify.request", obj("""{"request_id":"batch","questions":[{"qid":"q0","question":"First?"}]}"""))
val snapshot = requireNotNull(original.currentInteraction)
original.acknowledgeClarify("batch", "q0", "ignored", true)
val r = Recorder()
val recovered = mapperWith(r)
recovered.restoreInteraction(snapshot)
assertNull(recovered.currentInteraction)
assertTrue(r.interactions.isEmpty())
assertEquals("batch", r.interactionExpiries.single().requestId)
}
@Test
fun `forwarded acknowledgement cannot touch a newer request incarnation`() {
val mapper = mapperWith(Recorder())
val request = obj("""{"request_id":"batch","questions":[{"qid":"q0","question":"First?"}]}""")
mapper.onEvent("clarify.request", request)
val oldOwner = requireNotNull(mapper.currentInteraction).ownershipToken
mapper.onEvent("clarify.expire", obj("""{"request_id":"batch"}"""))
mapper.onEvent("clarify.request", request)
mapper.acknowledgeClarifyOwner("batch", "q0", "stale", true, oldOwner)
assertTrue(requireNotNull(mapper.currentInteraction).answers.isEmpty())
assertFalse(mapper.turnEnded)
}
@Test
fun `partial acknowledgement holds terminal and merges replay without resurrecting an answer`() {
val r = Recorder()
val mapper = mapperWith(r)
val request = obj("""{"request_id":"batch","questions":[
{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}]}""")
mapper.onEvent("clarify.request", request)
mapper.onEvent("message.complete", obj("""{"text":"done"}"""))
mapper.acknowledgeClarify("batch", "q0", "first", false)
assertFalse(mapper.turnEnded)
assertEquals(mapOf("q0" to "first"), mapper.currentInteraction?.answers)
mapper.onEvent("clarify.request", request)
assertEquals(mapOf("q0" to "first"), mapper.currentInteraction?.answers)
mapper.acknowledgeClarify("old-batch", "q1", "stale", true)
assertFalse(mapper.turnEnded)
mapper.acknowledgeClarify("batch", "q1", "second", false)
assertNull(mapper.currentInteraction)
assertTrue(mapper.turnEnded)
}
@Test
fun `batch expiry retires partial progress only for exact request`() {
val r = Recorder()
val mapper = mapperWith(r)
mapper.onEvent("clarify.request", obj("""{"request_id":"batch","questions":[
{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}],"answers":{"q0":"done"}}"""))
mapper.onEvent("clarify.expire", obj("""{"request_id":"other"}"""))
assertEquals("done", mapper.currentInteraction?.answers?.get("q0"))
mapper.onEvent("clarify.expire", obj("""{"request_id":"batch"}"""))
assertNull(mapper.currentInteraction)
}
@Test
fun `batch clarify preserves exact qids question order choices and replayed answers`() {
val ask = GatewayEventMapper.interactionRequest("clarify.request", obj("""
{"request_id":"batch","questions":[
{"qid":"choice/a","question":"Which deployment?","choices":["Canary","Immediate"],"multi_select":false},
{"qid":"env:b","question":"Which environments?","choices":["Stage","Production"],"multi_select":true}
],"answers":{"choice/a":"Canary","foreign":"ignored"}}
"""))!!
assertEquals(listOf("choice/a", "env:b"), ask.questions.map { it.qid })
assertEquals("Which deployment?", ask.questions[0].question)
assertEquals(listOf("Stage", "Production"), ask.questions[1].choices)
assertTrue(ask.questions[1].multiSelect)
assertEquals(mapOf("choice/a" to "Canary"), ask.answers)
}
@Test
fun `one entry normalized clarify keeps its qid`() {
val ask = GatewayEventMapper.interactionRequest("clarify.request", obj("""
{"request_id":"one","questions":[{"qid":"q0","question":"Which file?","choices":null}]}
"""))!!
assertEquals(listOf("q0"), ask.questions.map { it.qid })
}
private class Recorder {
val textDeltas = mutableListOf<String>()
val interimMessages = mutableListOf<Pair<String, Boolean>>()
@@ -0,0 +1,121 @@
package com.hermesandroid.relay.screenshots
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalDensity
import androidx.compose.ui.semantics.SemanticsActions
import androidx.compose.ui.test.hasScrollAction
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.onRoot
import androidx.compose.ui.test.performSemanticsAction
import androidx.compose.ui.test.performTouchInput
import androidx.compose.ui.test.swipeUp
import androidx.compose.ui.text.TextLayoutResult
import androidx.compose.ui.unit.Density
import androidx.compose.ui.unit.dp
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.github.takahirom.roborazzi.captureRoboImage
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.HermesCard
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import com.hermesandroid.relay.data.HermesCardClarifyQuestion
import com.hermesandroid.relay.data.HermesCardInput
import com.hermesandroid.relay.data.MessageDeliveryStatus
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.ui.components.MessageBubble
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import java.io.File
import java.text.SimpleDateFormat
import java.util.Date
import java.util.Locale
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
class ChatBubblePolishScreenshotTest {
@get:Rule val compose = createComposeRule()
@Test @Config(qualifiers = "w360dp-h800dp-xhdpi")
fun darkConversation() = capture("dark-conversation", "dark", 1f)
@Test @Config(qualifiers = "w360dp-h800dp-xhdpi")
fun lightConversation() = capture("light-conversation", "light", 1f)
@Test @Config(qualifiers = "w320dp-h568dp-xhdpi")
fun narrowLargeText() = capture("narrow-large-text", "dark", 1.5f)
@Test @Config(qualifiers = "w720dp-h360dp-xhdpi")
fun landscape() = capture("landscape", "dark", 1f)
@Test @Config(qualifiers = "w360dp-h800dp-xhdpi")
fun correctionAndTimeShareOneLine() {
compose.setContent {
HermesRelayTheme(themePreference = "dark") {
MessageBubble(correction(), animationEnabled = false)
}
}
val time = SimpleDateFormat("h:mm a", Locale.US).format(Date(TIMESTAMP))
val timeBounds = compose.onNodeWithText(time, useUnmergedTree = true).fetchSemanticsNode().boundsInRoot
val status = compose.onNodeWithText("Correction sent", useUnmergedTree = true)
val statusBounds = status.fetchSemanticsNode().boundsInRoot
assertTrue("Time and correction should share a footer row", kotlin.math.abs(timeBounds.center.y - statusBounds.center.y) <= 2f)
val layouts = mutableListOf<TextLayoutResult>()
status.performSemanticsAction(SemanticsActions.GetTextLayoutResult) { it(layouts) }
assertEquals(1, layouts.single().lineCount)
}
private fun capture(name: String, theme: String, scale: Float) {
val card = HermesCard(
type = HermesCard.BuiltInTypes.ASK_CLARIFY,
title = "Hermes needs clarification",
id = "clarify",
clarifyBatch = HermesCardClarifyBatch(listOf(
HermesCardClarifyQuestion("q0", "Which accent color?", HermesCardInput(HermesCardInput.Kinds.TEXT), "Orange"),
HermesCardClarifyQuestion("q1", "Which sections?", HermesCardInput(HermesCardInput.Kinds.CHOICE, multiSelect = true), "[\"Calendar\",\"Tasks\"]"),
)),
)
val messages = listOf(
ChatMessage(id = "ask", role = MessageRole.ASSISTANT, content = "", timestamp = TIMESTAMP, cards = listOf(card), clientOnly = true),
ChatMessage(id = "reply", role = MessageRole.ASSISTANT, content = "Got both responses in one batch:\n\n- Accent: **Orange**\n- Sections: **Calendar and Tasks**", timestamp = TIMESTAMP),
correction(),
)
compose.setContent {
val density = LocalDensity.current
CompositionLocalProvider(LocalDensity provides Density(density.density, scale)) {
HermesRelayTheme(themePreference = theme) {
LazyColumn(
Modifier.fillMaxSize().background(MaterialTheme.colorScheme.background).padding(12.dp),
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
items(messages.size, key = { messages[it].id }) { index ->
MessageBubble(messages[index], showAgentIdentity = false, isFirstInGroup = index == 0 || index == 2,
isLastInGroup = index != 0, animationEnabled = false)
}
}
}
}
}
val directory = File("build/ui-evidence/bubble-polish").apply { mkdirs() }
compose.onRoot().captureRoboImage(File(directory, "$name-top.png").path)
repeat(3) { compose.onNode(hasScrollAction()).performTouchInput { swipeUp() } }
compose.onRoot().captureRoboImage(File(directory, "$name-bottom.png").path)
}
private fun correction() = ChatMessage(id = "correction", role = MessageRole.USER, content = "Test", timestamp = TIMESTAMP,
deliveryStatus = MessageDeliveryStatus.STEERED)
companion object { private const val TIMESTAMP = 1_700_000_000_000L }
}
@@ -0,0 +1,105 @@
package com.hermesandroid.relay.screenshots
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalDensity
import androidx.compose.ui.test.hasScrollAction
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onRoot
import androidx.compose.ui.test.performTouchInput
import androidx.compose.ui.test.swipeUp
import androidx.compose.ui.unit.Density
import androidx.compose.ui.unit.dp
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.github.takahirom.roborazzi.captureRoboImage
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.HermesCard
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import com.hermesandroid.relay.data.HermesCardClarifyQuestion
import com.hermesandroid.relay.data.HermesCardInput
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.ui.components.MessageBubble
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import java.io.File
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
class ClarifyBatchScreenshotTest {
@get:Rule val compose = createComposeRule()
@Test @Config(qualifiers = "w320dp-h568dp-xhdpi")
fun compactDark() = capture("compact-dark")
@Test @Config(qualifiers = "w320dp-h568dp-xhdpi")
fun compactLightLargeText() = capture("compact-light-font-1_5", theme = "light", scale = 1.5f)
@Test @Config(qualifiers = "w720dp-h360dp-xhdpi")
fun landscape() = capture("landscape", width = 540)
@Test @Config(qualifiers = "w330dp-h720dp-xhdpi")
fun narrowFoldable() = capture("foldable-pane", scale = 1.5f)
@Test @Config(qualifiers = "w360dp-h720dp-xhdpi")
fun partialProgress() = capture("partial", answered = 1)
@Test @Config(qualifiers = "w360dp-h720dp-xhdpi")
fun sending() = capture("sending", answered = 1, submitting = true)
@Test @Config(qualifiers = "w360dp-h720dp-xhdpi")
fun completed() = capture("completed", answered = 2)
@Test @Config(qualifiers = "w320dp-h568dp-xhdpi")
fun expiredPartial() = capture("expired-partial", answered = 1, expired = true)
private fun capture(
name: String, theme: String = "dark", scale: Float = 1f, width: Int = 300,
answered: Int = 0, submitting: Boolean = false, expired: Boolean = false,
) {
val options = listOf(
"Keep both systems running while traffic moves in measured stages (Recommended)",
"Migrate everything immediately and accept a short maintenance window",
"Pause until every downstream consumer has been verified",
"Use a reversible canary rollout with automatic rollback thresholds",
)
val questions = listOf(
HermesCardClarifyQuestion("q0", "Which deployment approach should I use for the migration, given the existing clients and the rollback requirements?",
HermesCardInput(HermesCardInput.Kinds.CHOICE, options, allowFreeText = true),
answer = options[0].takeIf { answered > 0 }),
HermesCardClarifyQuestion("q1", "Which environments should receive this change?",
HermesCardInput(HermesCardInput.Kinds.CHOICE, listOf("Staging", "Production"), multiSelect = true, allowFreeText = true),
answer = "[\"Staging\",\"Production\"]".takeIf { answered > 1 }, submitting = submitting),
)
val card = HermesCard(type = HermesCard.BuiltInTypes.ASK_CLARIFY, title = "Hermes needs clarification",
id = "batch", clarifyBatch = HermesCardClarifyBatch(questions, expiresAtMillis = if (expired) 1L else null))
compose.setContent {
val density = LocalDensity.current
CompositionLocalProvider(LocalDensity provides Density(density.density, scale)) {
HermesRelayTheme(themePreference = theme) {
LazyColumn(Modifier.fillMaxSize().background(MaterialTheme.colorScheme.background).padding(10.dp)) {
item {
MessageBubble(
message = ChatMessage(id = "batch-message", role = MessageRole.ASSISTANT, content = "",
timestamp = 0L, cards = listOf(card), clientOnly = true),
maxBubbleWidth = width.dp, showTimestamps = false, animationEnabled = false,
)
}
}
}
}
}
val directory = File("build/ui-evidence/clarify-batch").apply { mkdirs() }
compose.onRoot().captureRoboImage(File(directory, "$name-top.png").path)
repeat(4) { compose.onNode(hasScrollAction()).performTouchInput { swipeUp() } }
compose.onRoot().captureRoboImage(File(directory, "$name-bottom.png").path)
}
}
@@ -0,0 +1,120 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.setValue
import androidx.compose.ui.test.assertIsNotEnabled
import androidx.compose.ui.test.assertIsSelected
import androidx.compose.ui.test.assertIsFocused
import androidx.compose.ui.test.junit4.StateRestorationTester
import androidx.compose.ui.test.performKeyInput
import androidx.compose.ui.test.performSemanticsAction
import androidx.compose.ui.test.pressKey
import androidx.compose.ui.input.key.Key
import androidx.compose.ui.input.InputMode
import androidx.compose.ui.input.InputModeManager
import androidx.compose.ui.platform.LocalInputModeManager
import androidx.compose.ui.semantics.SemanticsActions
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onNodeWithContentDescription
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.performClick
import androidx.compose.ui.test.performImeAction
import androidx.compose.ui.test.performTextInput
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.hermesandroid.relay.data.HermesCard
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import com.hermesandroid.relay.data.HermesCardClarifyQuestion
import com.hermesandroid.relay.data.HermesCardInput
import org.junit.Assert.assertEquals
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(qualifiers = "w360dp-h720dp-xhdpi")
class ClarifyBatchInteractionTest {
@get:Rule val compose = createComposeRule()
@Test fun draftsAndSelectionSurviveSavedStateAndFocusFollowsQuestionOrder() {
val restoration = StateRestorationTester(compose)
lateinit var inputMode: InputModeManager
restoration.setContent {
inputMode = LocalInputModeManager.current
MaterialTheme {
HermesCardBubble(card().copy(clarifyBatch = HermesCardClarifyBatch(listOf(question))),
"batch", emptyList(), { _, _ -> }, { _, _ -> })
}
}
compose.onNodeWithText("Stage").performClick()
compose.onNodeWithContentDescription("Other (type your answer)…").performTextInput("Keep rollback")
restoration.emulateSavedInstanceStateRestore()
compose.onNodeWithText("Stage").assertIsSelected()
compose.onNodeWithText("Keep rollback").assertExists()
compose.runOnIdle { inputMode.requestInputMode(InputMode.Keyboard) }
compose.onNodeWithText("Stage").performSemanticsAction(SemanticsActions.RequestFocus) { it() }
compose.onNodeWithText("Stage").assertIsFocused()
compose.onNodeWithText("Stage").performKeyInput { pressKey(Key.Tab) }
compose.onNodeWithText("Production").assertIsFocused()
}
@Test fun multiSelectImeRetryKeepsChoicesAndDraftAndDisablesDuplicateActions() {
var submitting by mutableStateOf(false)
val answers = mutableListOf<Pair<String, String>>()
compose.setContent {
MaterialTheme {
HermesCardBubble(
card = card().copy(clarifyBatch = HermesCardClarifyBatch(listOf(question.copy(submitting = submitting)))),
cardKey = "batch", dispatches = emptyList(), onActionTap = { _, _ -> },
onInputSubmit = { key, value -> answers += key to value; submitting = true },
)
}
}
compose.onNodeWithText("Submit").assertIsNotEnabled()
compose.onNodeWithContentDescription("Other (type your answer)…").apply {
performTextInput(" ")
performImeAction()
}
compose.runOnIdle { assertEquals(0, answers.size) }
compose.onNodeWithText("Stage").performClick().assertIsSelected()
compose.onNodeWithContentDescription("Other (type your answer)…").apply {
performTextInput("custom")
performImeAction()
}
compose.onNodeWithText("Stage").assertIsNotEnabled()
compose.onNodeWithText("Submit").assertIsNotEnabled()
compose.runOnIdle {
assertEquals(listOf("qid-key" to "[\"Stage\",\"custom\"]"), answers)
submitting = false
}
compose.onNodeWithText("Stage").assertIsSelected()
compose.onNodeWithText("Submit").performClick()
compose.runOnIdle { assertEquals(answers[0], answers[1]) }
}
@Test fun confirmedQuestionAdvancesAndExpiredBatchRetainsAnswersWithoutInputs() {
var batch by mutableStateOf(HermesCardClarifyBatch(listOf(question,
question.copy(key = "second", question = "Anything else?", input = HermesCardInput(HermesCardInput.Kinds.TEXT, allowFreeText = true)))))
compose.setContent {
MaterialTheme {
HermesCardBubble(card().copy(clarifyBatch = batch), "batch", emptyList(), { _, _ -> }, { _, _ -> })
}
}
compose.runOnIdle { batch = batch.copy(questions = batch.questions.map { if (it.key == "qid-key") it.copy(answer = "[\"Stage\"]") else it }) }
compose.onNodeWithText("Question 2 of 2").assertExists()
compose.onNodeWithText("Anything else?").assertExists()
compose.onNodeWithText("Stage").assertDoesNotExist()
compose.runOnIdle { batch = batch.copy(expiresAtMillis = 1L) }
compose.onNodeWithText("This request has ended").assertExists()
compose.onNodeWithText("Stage").assertExists()
compose.onNodeWithContentDescription("Type an answer…").assertDoesNotExist()
}
private val question = HermesCardClarifyQuestion("qid-key", "Which environments?",
HermesCardInput(HermesCardInput.Kinds.CHOICE, listOf("Stage", "Production"), multiSelect = true, allowFreeText = true))
private fun card() = HermesCard(HermesCard.BuiltInTypes.ASK_CLARIFY, title = "Hermes needs clarification")
}
@@ -2747,6 +2747,121 @@ class ChatViewModelGatewayInboundTurnTest {
awaitCondition { viewModel.pendingAsk.value == null }
}
private fun presentBatchClarify(requestId: String = "batch") {
serverWs.send(gatewayHarness.eventFrame("clarify.request",
gatewayHarness.json.parseToJsonElement("""{"request_id":"$requestId","questions":[
{"qid":"route/a","question":"Which deployment?","choices":["Canary","Immediate"]},
{"qid":"environment:b","question":"Which environments?","choices":["Stage","Production"],"multi_select":true}
]}""") as kotlinx.serialization.json.JsonObject, "live-resumed"))
awaitCondition { viewModel.pendingAsk.value?.ask?.requestId == requestId }
}
@Test
fun batchClarifyInFlightAnswerSurvivesSessionRoundTripWithoutDuplicateRpc() {
val contextKey = AgentDisplay.profileContextKey("connection-a", null)
val store = MemoryCheckpointStore()
viewModel.setChatTurnCheckpointStore(store)
viewModel.switchProfileContext(contextKey, STORED_SESSION_ID)
viewModel.sendMessage("Ask before continuing")
gatewayHarness.awaitRpc("prompt.submit")
presentBatchClarify()
val pending = requireNotNull(viewModel.pendingAsk.value)
val key = com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "route/a")
gatewayHarness.suppressAckMethods += "clarify.respond"
viewModel.answerAsk(pending.messageId, key, "Canary")
shadowOf(Looper.getMainLooper()).idle()
val ack = gatewayHarness.awaitPendingAck()
viewModel.switchSession("other-session")
awaitCondition { handler.currentSessionId.value == "other-session" && store.checkpoint?.pendingAsk != null }
gatewayHarness.recoveryRunning = true
viewModel.switchSession(STORED_SESSION_ID)
awaitCondition { viewModel.pendingAsk.value != null && handler.currentSessionId.value == STORED_SESSION_ID }
val restored = requireNotNull(viewModel.pendingAsk.value)
assertEquals(pending.ownerId, restored.ownerId)
viewModel.answerAsk(restored.messageId, key, "Canary")
gatewayHarness.releaseAck(ack)
awaitCondition { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "Canary" }
assertEquals(1, gatewayHarness.rpcLog.count { it.first == "clarify.respond" })
gatewayHarness.suppressAckMethods -= "clarify.respond"
viewModel.answerAsk(restored.messageId,
com.hermesandroid.relay.data.clarifyQuestionCardKey(restored.cardKey, "environment:b"), "[\"Stage\"]")
awaitCondition { viewModel.pendingAsk.value == null }
serverWs.send(gatewayHarness.eventFrame("message.complete", buildJsonObject { put("text", "All answers received") }, "live-resumed"))
awaitCondition { !handler.isStreaming.value && !gatewayClient.hasActiveTurn() }
assertEquals(2, gatewayHarness.rpcLog.count { it.first == "clarify.respond" })
}
@Test
fun batchClarifyKeepsPartialProgressRejectsDuplicateAndNeverSendsChatAnswers() {
viewModel.sendMessage("Ask a batch")
gatewayHarness.awaitRpc("prompt.submit")
presentBatchClarify()
val pending = requireNotNull(viewModel.pendingAsk.value)
val key = com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "route/a")
gatewayHarness.suppressAckMethods += "clarify.respond"
viewModel.answerAsk(pending.messageId, key, "Canary")
viewModel.answerAsk(pending.messageId, key, "Canary")
shadowOf(Looper.getMainLooper()).idle()
val ack = gatewayHarness.awaitPendingAck()
assertEquals(1, gatewayHarness.rpcLog.count { it.first == "clarify.respond" })
gatewayHarness.releaseAck(ack)
awaitCondition { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "Canary" }
presentBatchClarify()
viewModel.answerAsk(pending.messageId, key, "Immediate")
assertEquals("Canary", viewModel.pendingAsk.value?.ask?.answers?.get("route/a"))
gatewayHarness.suppressAckMethods -= "clarify.respond"
val secondKey = com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "environment:b")
viewModel.answerAsk(pending.messageId, secondKey, "[\"Stage\",\"Production\"]")
awaitCondition { viewModel.pendingAsk.value == null }
val responses = gatewayHarness.rpcLog.filter { it.first == "clarify.respond" }.map { it.second }
assertEquals(listOf(JsonPrimitive("route/a"), JsonPrimitive("environment:b")), responses.map { it["question_id"] })
assertEquals(1, gatewayHarness.rpcLog.count { it.first == "prompt.submit" })
val card = handler.messages.value.single { it.id == pending.messageId }.cards.single()
assertEquals(listOf("Canary", "[\"Stage\",\"Production\"]"), card.clarifyBatch?.questions?.map { it.answer })
}
@Test
fun batchClarifyFailedRpcCanRetryAndWhitespaceCannotSubmit() {
viewModel.sendMessage("Ask a batch")
gatewayHarness.awaitRpc("prompt.submit")
presentBatchClarify()
val pending = requireNotNull(viewModel.pendingAsk.value)
val key = com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "route/a")
viewModel.answerAsk(pending.messageId, key, " ")
assertTrue(gatewayHarness.rpcLog.none { it.first == "clarify.respond" })
gatewayHarness.rpcErrors["clarify.respond"] = 5030 to "Try again"
viewModel.answerAsk(pending.messageId, key, "Canary")
awaitCondition { gatewayHarness.rpcLog.any { it.first == "clarify.respond" } &&
handler.messages.value.single { it.id == pending.messageId }.cards.single().clarifyBatch?.questions?.first()?.submitting == false }
assertTrue(requireNotNull(viewModel.pendingAsk.value).ask.answers.isEmpty())
gatewayHarness.rpcErrors.remove("clarify.respond")
viewModel.answerAsk(pending.messageId, key, " custom route ")
awaitCondition { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "custom route" }
assertEquals(2, gatewayHarness.rpcLog.count { it.first == "clarify.respond" })
}
@Test
fun staleExpiredBatchResponseCannotRetireReusedRequestId() {
viewModel.sendMessage("Ask a batch")
gatewayHarness.awaitRpc("prompt.submit")
presentBatchClarify()
val pending = requireNotNull(viewModel.pendingAsk.value)
gatewayHarness.suppressAckMethods += "clarify.respond"
viewModel.answerAsk(pending.messageId, com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "route/a"), "Canary")
shadowOf(Looper.getMainLooper()).idle()
val ack = gatewayHarness.awaitPendingAck()
serverWs.send(gatewayHarness.eventFrame("clarify.expire", buildJsonObject { put("request_id", "batch") }, "live-resumed"))
awaitCondition { viewModel.pendingAsk.value == null }
presentBatchClarify()
val newOwner = requireNotNull(viewModel.pendingAsk.value).ownerId
gatewayHarness.releaseAck(ack, buildJsonObject { put("status", "expired") })
gatewayHarness.suppressAckMethods -= "clarify.respond"
val current = requireNotNull(viewModel.pendingAsk.value)
viewModel.answerAsk(current.messageId, com.hermesandroid.relay.data.clarifyQuestionCardKey(current.cardKey, "route/a"), "Immediate")
awaitCondition { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "Immediate" }
assertEquals(newOwner, viewModel.pendingAsk.value?.ownerId)
}
@Test
fun authoritativeClarifyExpiryCollapsesCardAndRejectsLateAction() {
viewModel.sendMessage("Ask a question")
Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

+3
View File
@@ -68,6 +68,9 @@ the upstream contract identifiers it depends on.
|---|---|
| `initial_history_bind` | Durable, profile-scoped history is already available when the client resumes and first binds its rendered transcript |
| `ordinary_turn` | Normal message start, deltas, completion, and persisted history |
| `clarify_legacy` | Top-level single question and unkeyed `clarify.respond` |
| `clarify_normalized_single` | One normalized `questions[]` entry still requires its exact `qid` |
| `clarify_batch` | Independent qid responses, partial acknowledgement, and answered-question replay on reconnect |
| `session_initialization_failure` | Exact-session initialization error arrives before a lazy create acknowledgement; Android must fail the pending send without waiting for the readiness deadline |
| `subagent_child_preview` | Child activity continues after the parent terminal, followed by child completion and a separate completion wake; preview ownership remains on the same profile/session |
| `ownership_rejection` | A submit acknowledged before the defense-in-depth ownership check emits the canonical terminal refusal; no user/model row is persisted and clients must not enter history recovery |
+6 -6
View File
@@ -13,7 +13,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "03856583805b7bfa6f8134b9ff3bdc1c8f3bbd7b2a342dcd890ded66a04ee842",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -48,7 +48,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "03856583805b7bfa6f8134b9ff3bdc1c8f3bbd7b2a342dcd890ded66a04ee842",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -72,7 +72,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "03856583805b7bfa6f8134b9ff3bdc1c8f3bbd7b2a342dcd890ded66a04ee842",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -96,7 +96,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "03856583805b7bfa6f8134b9ff3bdc1c8f3bbd7b2a342dcd890ded66a04ee842",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -120,7 +120,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "03856583805b7bfa6f8134b9ff3bdc1c8f3bbd7b2a342dcd890ded66a04ee842",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -135,7 +135,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "03856583805b7bfa6f8134b9ff3bdc1c8f3bbd7b2a342dcd890ded66a04ee842",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
+2 -2
View File
@@ -91,9 +91,9 @@ This app is a community project and is not affiliated with or endorsed by NousRe
Paste into Play Console → **What's new** (≤500 characters):
```
v1.16.1 - Dashboard-only cold starts recover
v1.17.0 - Voice over other apps and clearer conversations
Dashboard-only connections now prepare the selected profile before Gateway readiness, fixing a remaining cold-start path that could stay on waking or waiting for Gateway until the app resumed or its network route changed.
Use optional voice controls over other apps, with clear permission setup and an immediate Stop action. Answer Clarify batches one question at a time, with progress preserved across reconnects. Enjoy cleaner chat cards, recognizable profile names, and context previews that show what your connection supports. Phone control remains sideload-only.
```
## Category
+6
View File
@@ -1,5 +1,11 @@
# Hermes-Relay — Dev Log
## 2026-09-13 — Android 1.17.0 and Plugin 1.11.3 release preparation
Prepared Android 1.17.0 (versionCode 57) with Google Play Voice Overlay, progressive Clarify batches, chat card presentation, transport-accurate context previews and profile display names. Prepared Plugin 1.11.3 for current and legacy Dashboard WebSocket guard ownership. CLI+UI remains 0.4.0-beta.7.
Reconciled the privacy policy with local chat recovery state, explicit API-only routing and public update checks. Android publication retains exact-tree signed Play preflight and policy/declaration gates. Physical Android 14-16/OEM voice-overlay testing is waived for this release and is not claimed as verification.
## 2026-09-02 — Android 1.15.1 and CLI+UI beta.7 release preparation
Prepared Android 1.15.1 (versionCode 54) and CLI+UI 0.4.0-beta.7 from the integrated release tree. Android notes cover chat memory bounds, media previews, Gateway readiness, follow-up controls, and voice recovery. CLI+UI notes cover the Windows unified updater. Plugin metadata remains at 1.11.1; its only unreleased change is a documentation-comment path.
+1 -1
View File
@@ -640,7 +640,7 @@ Bottom navigation bar with 4 tabs:
- **Agent Passport — profile inspection/configuration** — upstream Hermes profiles are selected from the Profile Shelf or the Passport's shared full switcher. Passport retains identity customization, model/personality/reasoning/safety configuration, inspection, and session analytics; it is not a second profile-picker implementation. See `docs/decisions.md` §21 and ADR 48.
- **Agent sheet — Personality section** — personalities fetched from `GET /api/config` (`config.agent.personalities`). Shows server default (from `config.display.personality`) + all configured. Active personality name shown on assistant chat bubbles.
- **Agent sheet — Approval controls** — gateway contract v3 exposes the profile-persisted `approvals.mode` policy (`manual` / `smart` / `off`) separately from YOLO. The launch/default profile gets the three-way control; multiplexed non-launch profiles reconcile `session.info.approval_mode` read-only until upstream config RPCs honor profile scope. The existing YOLO switch remains an explicit per-session override and never silently writes profile configuration. Older gateways keep chat and YOLO available while the profile control explains that an upstream update is required.
- **Interactive clarify cards** — ordinary upstream choices retain one-tap submission; `multi_select:true` choices toggle independently and require explicit submission as one JSON-array answer. Open text remains available for an Other answer. Android never invents a clarify deadline when upstream omits timeout metadata: the correlated `clarify.expire` event or an expired response retires the card authoritatively.
- **Interactive clarify cards** — ordinary upstream choices retain one-tap submission; `multi_select:true` choices toggle independently and require explicit submission as one JSON-array answer. Open text remains available for an Other answer. Upstream `questions[]` batches use one progressive card, retain exact qids, and answer each question through `clarify.respond` with `question_id`. Confirmed answers survive checkpoint recovery and merge with upstream replay without resubmission. Legacy top-level questions keep their original wire shape. Android never invents a clarify deadline when upstream omits timeout metadata: the correlated `clarify.expire` event or an expired response retires the card authoritatively.
- **Streaming dots** — animated pulsing 3-dot indicator replaces static "streaming..." text
- Displays: streaming delta text; quiet thinking/reasoning disclosures that open while live and collapse when settled; consecutive routine tool activity summarized as one live ticker or settled disclosure; standalone lifecycle surfaces for approvals, failures, generated media, file edits, and delegated work; per-message token counts + cost
+2 -2
View File
@@ -1,6 +1,6 @@
[versions]
appVersionName = "1.16.1"
appVersionCode = "56"
appVersionName = "1.17.0"
appVersionCode = "57"
agp = "9.4.0"
kotlin = "2.4.10"
compose-bom = "2026.08.00"
+1 -1
View File
@@ -3,7 +3,7 @@
"label": "Hermes-Relay",
"description": "Paired devices, Bridge activity, media tokens, and remote access for Hermes-Relay",
"icon": "Activity",
"version": "1.11.2",
"version": "1.11.3",
"tab": {
"path": "/relay",
"position": "after:skills"
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "hermes-relay-dashboard",
"version": "1.11.2",
"version": "1.11.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "hermes-relay-dashboard",
"version": "1.11.2",
"version": "1.11.3",
"devDependencies": {
"esbuild": "^0.25.12",
"qrcode": "^1.5.4"
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "hermes-relay-dashboard",
"version": "1.11.2",
"version": "1.11.3",
"private": true,
"description": "Hermes-Relay dashboard plugin frontend (IIFE bundle). Loaded verbatim by the hermes-agent dashboard via the Plugin SDK global.",
"scripts": {
+1 -1
View File
@@ -2,7 +2,7 @@ name: hermes-relay
# Temporary v1 shim for Hermes installers that reject manifests the runtime supports; see docs/project/TODO.md.
manifest_version: 1
api_version: 1
version: 1.11.2
version: 1.11.3
description: "Hermes-Relay plugin for QR pairing, relay sessions, dashboard management, remote desktop/phone tooling, and optional legacy compatibility diagnostics. Standard chat, Manage, and dashboard voice remain vanilla upstream Hermes surfaces."
author: Axiom Labs
license: MIT
+1 -1
View File
@@ -19,7 +19,7 @@ See ``plugin/relay/server.py`` for the aiohttp server,
# CLI+UI releases use desktop/package.json and desktop-v* tags. The /health endpoint
# reports this plugin version, and stale values make live diagnosis harder than
# it should be.
__version__ = "1.11.2"
__version__ = "1.11.3"
from .server import create_app, main # noqa: E402 — must come after __version__
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "hermes-relay"
version = "1.11.2"
version = "1.11.3"
description = "Hermes-Relay plugin — Android device control toolset, QR pairing CLI, and WSS relay server for hermes-agent"
requires-python = ">=3.11"
dependencies = [
@@ -36,7 +36,9 @@ SESSION_EXCLUSIVE_SUBMIT = "gateway.session_exclusive_submit"
SUBAGENT_CHILD_WATCH = "gateway.subagent_child_watch"
SESSION_INITIALIZATION = "gateway.session_initialization"
API_BOUNDARY = "api.fallback_boundary"
CLARIFY = "gateway.clarify"
ALL_CONTRACTS = (
CLARIFY,
GATEWAY_TERMINAL,
GATEWAY_SETTLED_INFO,
SESSION_ACTIVATE,
@@ -541,6 +543,27 @@ def load_requirements(manifest: Path | None) -> tuple[str, ...]:
return tuple(contract for contract in ALL_CONTRACTS if contract in requested)
def _check_clarify(server: SourceFile) -> CheckResult:
bridge = server.function("_clarify_block")
respond = server.function("_respond")
replay = server.function("_pending_clarify_request_payload")
block = server.function("_block")
required = (
{"questions", "qid", "question", "choices", "multi_select"} <= _string_constants(bridge)
and {"question_id", "request_id", "answers", "remaining", "expired"} <= _string_constants(respond)
and {"answers", "clarify.request"} <= _string_constants(replay)
and {"answers", "timed_out"} <= _string_constants(block)
)
return CheckResult(
CLARIFY, required,
tuple(server.evidence(node, label) for node, label in (
(bridge, "legacy and qid batch wire"), (respond, "per-question response"),
(replay, "answered-qid replay"), (block, "partial timeout"),
)),
None if required else "Clarify wire, response, replay, or partial-timeout contract changed",
)
def audit_sources(root: Path, requirements: Iterable[str]) -> list[CheckResult]:
server = SourceFile(root, SERVER)
methods = SourceFile(root, SESSION_METHODS)
@@ -558,6 +581,7 @@ def audit_sources(root: Path, requirements: Iterable[str]) -> list[CheckResult]:
raise ValueError("fork marker(s) found in upstream source: " + ", ".join(fork_hits))
checks = {
CLARIFY: lambda: _check_clarify(server),
GATEWAY_TERMINAL: lambda: _check_gateway_terminal(
SourceFile(root, "tui_gateway/prompt_turn.py")
if (root / "tui_gateway/prompt_turn.py").is_file() else server
@@ -17,6 +17,19 @@ SPEC.loader.exec_module(module)
SERVER_SOURCE = '''
def _clarify_block(sid, q, c, multi_select=False, questions=None):
return {"questions": [{"qid": "q0", "question": q, "choices": c, "multi_select": multi_select}]}
def _respond(rid, params, key):
return {"question_id": params.get("question_id"), "request_id": params.get("request_id"),
"answers": {}, "remaining": [], "status": "expired"}
def _pending_clarify_request_payload(sid):
return {"answers": {}, "event": "clarify.request"}
def _block(event, sid, payload):
return {"answers": {}, "timed_out": True}
def _session_info(agent, session=None):
return {"running": bool((session or {}).get("running"))}
@@ -164,6 +177,13 @@ async def _handle_runs(request):
class GatewayScenarioConformanceTest(unittest.TestCase):
def test_clarify_conformance_requires_question_ownership_and_replay(self) -> None:
results = module.audit_sources(self.root, [module.CLARIFY])
self.assertTrue(results[0].passed)
source = self.root / module.SERVER
source.write_text(SERVER_SOURCE.replace('"remaining"', '"other"'), encoding="utf-8")
self.assertFalse(module.audit_sources(self.root, [module.CLARIFY])[0].passed)
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.root = Path(self.temp.name)
+2 -1
View File
@@ -47,7 +47,7 @@ distribution, and trust installation are deliberately outside this fixture.
- `POST /api/auth/ws-ticket` mints a fresh, single-use 30-second ticket.
- `GET /api/ws?ticket=...` upgrades to WebSocket and sends `gateway.ready`.
- JSON-RPC methods: `session.create`, `session.resume`, `session.activate`,
`session.active_list`, `prompt.submit`, and `session.interrupt` when the
`session.active_list`, `prompt.submit`, `clarify.respond`, and `session.interrupt` when the
selected scenario enables them.
- `GET /api/sessions/{stored-id}/messages` returns persisted, paginated history
and accepts the upstream `profile`, `limit`, `offset`, and `order` query shape.
@@ -66,6 +66,7 @@ ordered `steps` list using these operations:
| Operation | Purpose |
|---|---|
| `event` | Send a Gateway event with `exact`, `foreign`, or `unscoped` identity. |
| `clarify` | Emit the supplied Clarify `payload` and wait for its legacy answer or every batch qid; activation replays confirmed answers. |
| `persist` | Append authoritative Dashboard history rows. |
| `sleep` | Create a bounded deterministic ordering window (maximum 5 seconds). |
| `set_running` | Change the authoritative session running state. |
@@ -18,6 +18,58 @@ from vanilla_gateway.evidence import EvidenceLog # noqa: E402
class FixtureTestCase(unittest.IsolatedAsyncioTestCase):
async def test_clarify_batch_requires_each_qid_and_replays_partial_progress(self) -> None:
fixture, base_url = await self.start("clarify_batch")
ws, _ = await self.connect(base_url)
await self.rpc(ws, 1, "prompt.submit")
frames = await self.frames_until(ws, lambda f: f.get("params", {}).get("type") == "clarify.request")
request = frames[-1]["params"]["payload"]
rid = request["request_id"]
await self.rpc(ws, 2, "clarify.respond", {"request_id": rid, "question_id": "foreign", "answer": "x"})
frames = await self.frames_until(ws, lambda f: f.get("id") == 2)
self.assertEqual(4002, frames[-1]["error"]["code"])
await self.rpc(ws, 3, "clarify.respond", {"request_id": rid, "question_id": "route/a", "answer": "Canary"})
frames = await self.frames_until(ws, lambda f: f.get("id") == 3)
self.assertEqual(["environment:b"], frames[-1]["result"]["remaining"])
self.assertTrue(fixture.running)
await ws.close()
ws, _ = await self.connect(base_url)
await self.rpc(ws, 4, "session.activate", {"session_id": fixture.scenario.live_session_id})
frames = await self.frames_until(ws, lambda f: f.get("id") == 4)
self.assertEqual({"route/a": "Canary"}, frames[-1]["result"]["pending_clarify"]["answers"])
await self.rpc(ws, 5, "clarify.respond", {
"request_id": rid, "question_id": "environment:b", "answer": '["Stage","Production"]',
})
frames = await self.frames_until(ws, lambda f: f.get("id") == 5)
self.assertEqual([], frames[-1]["result"]["remaining"])
self.assertEqual('["Stage","Production"]', fixture._clarify_answers["environment:b"])
frames = await self.frames_until(ws, lambda f: f.get("params", {}).get("type") == "message.complete")
self.assertEqual("message.complete", frames[-1]["params"]["type"])
async def test_clarify_legacy_keeps_unkeyed_response(self) -> None:
_, base_url = await self.start("clarify_legacy")
ws, _ = await self.connect(base_url)
await self.rpc(ws, 1, "prompt.submit")
frames = await self.frames_until(ws, lambda f: f.get("params", {}).get("type") == "clarify.request")
payload = frames[-1]["params"]["payload"]
self.assertNotIn("questions", payload)
await self.rpc(ws, 2, "clarify.respond", {"request_id": payload["request_id"], "answer": "Canary"})
frames = await self.frames_until(ws, lambda f: f.get("id") == 2)
self.assertEqual("ok", frames[-1]["result"]["status"])
async def test_one_normalized_question_is_still_qid_owned(self) -> None:
_, base_url = await self.start("clarify_normalized_single")
ws, _ = await self.connect(base_url)
await self.rpc(ws, 1, "prompt.submit")
frames = await self.frames_until(ws, lambda f: f.get("params", {}).get("type") == "clarify.request")
payload = frames[-1]["params"]["payload"]
self.assertEqual(1, len(payload["questions"]))
await self.rpc(ws, 2, "clarify.respond", {
"request_id": payload["request_id"], "question_id": payload["questions"][0]["qid"], "answer": "Canary",
})
frames = await self.frames_until(ws, lambda f: f.get("id") == 2)
self.assertEqual([], frames[-1]["result"]["remaining"])
async def asyncSetUp(self) -> None:
self.session = ClientSession()
self.runner: web.AppRunner | None = None
@@ -14,7 +14,7 @@ class ScenarioError(ValueError):
"""Raised when a scenario does not satisfy the fixture schema."""
_STEP_OPS = {"event", "persist", "sleep", "close", "set_running"}
_STEP_OPS = {"event", "persist", "sleep", "close", "set_running", "clarify"}
_LIVE_STATUSES = {"starting", "working", "waiting", "idle"}
_SAFE_NAME = re.compile(r"[A-Za-z0-9_.-]{1,120}")
@@ -68,6 +68,16 @@ class Scenario:
raise ScenarioError("event scope must be exact, foreign, or unscoped")
if step["op"] == "persist" and not isinstance(step.get("messages"), list):
raise ScenarioError("persist step requires a messages list")
if step["op"] == "clarify":
payload = step.get("payload")
if not isinstance(payload, dict) or not isinstance(payload.get("request_id"), str):
raise ScenarioError("clarify step requires a request_id payload")
questions = payload.get("questions", [])
if not isinstance(questions, list) or any(
not isinstance(q, dict) or not isinstance(q.get("qid"), str) or not q["qid"]
for q in questions
):
raise ScenarioError("clarify questions require exact qids")
if step["op"] == "set_running" and not isinstance(step.get("value"), bool):
raise ScenarioError("set_running step requires a boolean value")
if step["op"] == "sleep":
@@ -0,0 +1,52 @@
{
"name": "clarify_batch",
"live_session_id": "fixture-live-clarify",
"stored_session_id": "fixture-stored-clarify",
"profile": "default",
"contract_requirements": [
"gateway.clarify"
],
"turns": [
{
"steps": [
{
"op": "event",
"type": "message.start"
},
{
"op": "clarify",
"payload": {
"request_id": "clarify-batch",
"questions": [
{
"qid": "route/a",
"question": "Which deployment?",
"choices": [
"Canary",
"Immediate"
],
"multi_select": false
},
{
"qid": "environment:b",
"question": "Which environments?",
"choices": [
"Stage",
"Production"
],
"multi_select": true
}
]
}
},
{
"op": "event",
"type": "message.complete",
"payload": {
"text": "Clarification complete."
}
}
]
}
]
}
@@ -0,0 +1,38 @@
{
"name": "clarify_legacy",
"live_session_id": "fixture-live-clarify",
"stored_session_id": "fixture-stored-clarify",
"profile": "default",
"contract_requirements": [
"gateway.clarify"
],
"turns": [
{
"steps": [
{
"op": "event",
"type": "message.start"
},
{
"op": "clarify",
"payload": {
"request_id": "clarify-legacy",
"question": "Which deployment?",
"choices": [
"Canary",
"Immediate"
],
"multi_select": false
}
},
{
"op": "event",
"type": "message.complete",
"payload": {
"text": "Clarification complete."
}
}
]
}
]
}
@@ -0,0 +1,43 @@
{
"name": "clarify_normalized_single",
"live_session_id": "fixture-live-clarify",
"stored_session_id": "fixture-stored-clarify",
"profile": "default",
"contract_requirements": [
"gateway.clarify"
],
"turns": [
{
"steps": [
{
"op": "event",
"type": "message.start"
},
{
"op": "clarify",
"payload": {
"request_id": "clarify-one",
"questions": [
{
"qid": "route/a",
"question": "Which deployment?",
"choices": [
"Canary",
"Immediate"
],
"multi_select": false
}
]
}
},
{
"op": "event",
"type": "message.complete",
"payload": {
"text": "Clarification complete."
}
}
]
}
]
}
@@ -55,6 +55,10 @@ class GatewayFixture:
self._connection_sequence = 0
self._tasks: set[asyncio.Task[None]] = set()
self._sockets: set[web.WebSocketResponse] = set()
self._clarify: dict[str, Any] | None = None
self._clarify_answers: dict[str, str] = {}
self._clarify_done = asyncio.Event()
self._clarify_owner: tuple[web.WebSocketResponse, int] | None = None
@property
def running(self) -> bool:
@@ -135,16 +139,40 @@ class GatewayFixture:
await self._rpc_error(socket, request_id, 4040, "Stored session not found")
return
result = self._session_snapshot(include_stored=True)
if self._clarify is not None:
self._clarify_owner = socket, connection
elif method == "session.activate":
requested = params.get("session_id")
if requested != self.scenario.live_session_id:
await self._rpc_error(socket, request_id, 4041, "Live session not found")
return
result = self._session_snapshot(include_stored=True)
if self._clarify is not None:
self._clarify_owner = socket, connection
elif method == "prompt.submit":
await self._rpc_result(socket, request_id, {"ok": True})
await self._submit(socket, connection)
return
elif method == "clarify.respond":
pending = self._clarify
if pending is None or params.get("request_id") != pending["request_id"]:
result = {"status": "expired"}
else:
qids = [q["qid"] for q in pending.get("questions", [])]
qid = params.get("question_id")
if qids and qid:
if qid not in qids:
await self._rpc_error(socket, request_id, 4002, "unknown question_id")
return
self._clarify_answers[qid] = params.get("answer", "")
remaining = [q for q in qids if q not in self._clarify_answers]
result = {"status": "ok", "remaining": remaining}
if not remaining:
self._clarify_done.set()
else:
# Upstream's no-qid batch response cancels the whole request.
self._clarify_done.set()
result = {"status": "ok"}
elif method == "session.interrupt":
was_active = self._turn_active
tasks = tuple(self._tasks)
@@ -179,6 +207,9 @@ class GatewayFixture:
}
if include_stored:
snapshot["stored_session_id"] = self.scenario.stored_session_id
if self._clarify is not None:
snapshot["pending_clarify"] = dict(self._clarify, answers=dict(self._clarify_answers))
snapshot["info"]["pending_clarify"] = snapshot["pending_clarify"]
if self._running:
snapshot["inflight"] = {"user": "fixture turn", "assistant": "", "streaming": True}
return snapshot
@@ -218,6 +249,18 @@ class GatewayFixture:
operation = step["op"]
if operation == "sleep":
await asyncio.sleep(step["milliseconds"] / 1_000)
elif operation == "clarify":
self._clarify = dict(step["payload"])
self._clarify_answers = {}
self._clarify_owner = socket, connection
self._clarify_done.clear()
await self._send_event(
socket, connection, "clarify.request", self._clarify,
self.scenario.live_session_id,
)
await self._clarify_done.wait()
socket, connection = self._clarify_owner
self._clarify = None
elif operation == "set_running":
self._running = bool(step["value"])
self.evidence.add(
@@ -245,6 +288,8 @@ class GatewayFixture:
)
self.evidence.add("fault", connection=connection, outcome="socket_gap")
finally:
self._clarify = None
self._clarify_owner = None
self._turn_active = False
if self._running:
self._running = False
+7
View File
@@ -133,6 +133,13 @@ Sensitive prompts (sudo, secrets) are masked and hold-to-confirm. See
[Markdown Rendering → Rich Cards](/features/markdown#rich-cards) for the full
visual vocabulary.
When Hermes asks several questions together, the card shows one question at a
time and tracks your progress. Choose an option or type an **Other** answer;
questions that allow several choices have a **Submit** button. Each confirmed
answer stays recorded as you continue, including after a reconnect. If sending
fails, your selections remain available to retry. When a request ends, the card
keeps confirmed answers visible and stops accepting new ones.
## Context meter
A thin strip under the chat header tracks how full the conversation's context
+4 -4
View File
@@ -43,7 +43,7 @@
<p>The sideload build is a separate distribution track for users who intentionally install Device Control outside Google Play.</p>
<h2>Data storage</h2>
<p>All data is stored locally on your device in the app's private sandbox.</p>
<p>App settings, credentials, drafts and recovery state are stored locally in the app's private sandbox. Content sent to your configured Hermes server and AI providers follows their storage and retention settings.</p>
<table>
<thead><tr><th>Data</th><th>Storage method</th></tr></thead>
<tbody>
@@ -54,11 +54,11 @@
<tr><td>Pending Android Assistant context</td><td>App-private cache until one turn is accepted, cancellation/session exit, or one-hour stale cleanup; failed preflight retains it for retry</td></tr>
</tbody>
</table>
<p>Chat messages are not cached on your device. They are loaded from your Hermes server on demand and exist only in memory while the app is running.</p>
<p>Your Hermes server owns conversation history. The app retains local drafts, queued messages and bounded in-flight turn checkpoints to recover interrupted conversations; these can include message text and confirmed Clarify answers. Attachment and Android Assistant context caches support viewing, sending and retrying content.</p>
<h2>Network connections</h2>
<p>The app connects only to endpoints you configure: your Hermes Dashboard/Gateway for standard chat, management, voice, and inbound files; your optional API server for chat fallback; your relay server for terminal and TUI relay, Bridge Core status, explicit Relay media enhancements, notification companion, and session management; and your relay voice routes when you use enhanced Voice modes.</p>
<p>No connections are made to Google, Anthropic, or any other third-party service by the app. There is no telemetry, advertising, external crash reporting, DNS prefetching, or background network activity to hosted Hermes-Relay services. Your Hermes server may separately connect to AI providers according to its configuration; that server-side activity is outside the scope of this app.</p>
<p>Chat and voice content goes to endpoints you configure: your Hermes Dashboard/Gateway for standard chat, management, voice, and inbound files; an explicitly selected API-only connection for compatible chat; your relay server for terminal and TUI relay, Bridge Core status, explicit Relay media enhancements, notification companion, and session management; and your relay voice routes when you use enhanced Voice modes. Gateway-owned chats do not silently switch to an API server.</p>
<p>There is no telemetry, advertising or automatic external crash reporting. Your Hermes server may send content to AI providers according to its configuration. Optional update checks and links can contact public distribution and documentation services; these do not upload chat or voice content.</p>
<h2>Voice Overlay</h2>
<p>Voice Overlay is optional in both builds. Start it explicitly from Voice Focus while Hermes-Relay is visible and unlocked. It requires microphone access, display-over-other-apps access and an enabled microphone notification with Stop voice. Audio goes to the configured Hermes server; the overlay does not read or control other apps. Stop voice, closing the overlay, screen lock, task removal or loss of required access ends the overlay voice session. Returning to the app keeps foreground protection until the app is resumed. Granting permissions never starts a session.</p>