Compare commits

...
Author SHA1 Message Date
Bailey Dixon 86a570e72d chore: refresh chat UI cleanup against dev 2026-09-13 20:13:40 -04:00
Bailey Dixon cbe811fb52 fix(android): simplify chat bubble surfaces and footers 2026-09-13 20:13:15 -04:00
Bailey Dixon ac2508c5ce Merge pull request #588 from Codename-11/fix/android-gateway-context-audit
fix(android): show Gateway context-sharing limitations
2026-09-13 16:04:24 -04:00
Bailey Dixon 2ad7de4cc9 fix(android): make injected context previews transport-aware 2026-09-13 13:36:50 -04:00
Bailey Dixon 5eb5d198f4 chore: refresh batch Clarify against current dev 2026-09-13 13:29:57 -04:00
Bailey Dixon 7a86b9cee7 Merge pull request #583 from nicolasestrem/fix/dashboard-ws-guard-compat
fix(dashboard): resolve relocated upstream WebSocket guards
2026-09-12 20:14:26 -04:00
Bailey Dixon 60f93994f5 chore: merge current dev into dashboard WebSocket guard fix 2026-09-12 20:12:06 -04:00
Bailey Dixon 52e0e38a81 chore: merge current dev into batch Clarify fix 2026-09-12 20:11:40 -04:00
Bailey Dixon 471595240b Merge pull request #584 from Codename-11/feature/play-voice-overlay
feat(android): add user-started Play voice overlay
2026-09-12 20:03:40 -04:00
Bailey Dixon 061512d330 fix(android): support upstream batch Clarify requests 2026-09-12 19:59:06 -04:00
nicolasestrem af54cdddb3 fix(dashboard): resolve relocated upstream WebSocket guards
Preserve fail-closed admission and legacy compatibility. Add regression and real-upstream conformance tests plus transport documentation.
2026-09-12 18:21:33 +02:00
Bailey Dixon a5f671c06c Merge pull request #575 from Codename-11/dev
release: Android 1.16.0 and Plugin 1.11.2
2026-09-10 09:33:47 -04:00
Bailey Dixon 75feb55adf Merge pull request #540 from Codename-11/dev
release: Android 1.15.1 and CLI+UI 0.4.0-beta.7
2026-09-02 22:02:09 -04:00
Bailey Dixon f4b366389b release: Android 1.15.0, Plugin 1.11.1, CLI+UI 0.4.0-beta.6 (#525) 2026-08-31 23:45:58 -04:00
Bailey Dixon 2ec7b7aac9 release: Android 1.14.0 and Plugin 1.11.0 (#490) 2026-08-30 23:50:41 -04:00
60 changed files with 2118 additions and 165 deletions
+4
View File
@@ -12,6 +12,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
### Fixed
- Android shows standalone response cards without an outer bubble, uses subtler assistant surfaces, and places delivery status beside message timestamps.
- Android answers upstream Clarify batches one question at a time, with independent choices, custom answers, and confirmed progress preserved across reconnects. (#474)
- Android context previews mark phone status and turn context as unavailable in Gateway chats instead of claiming they are sent. Settings clarify that automatic phone-status sharing applies to API-only chats. (#556)
- Relay Dashboard WebSockets work with current Hermes authentication helpers while preserving older-host compatibility, single-use tickets, Host/Origin/IP checks, and Relay session authentication.
- Android shows Hermes profile display names and groups the resolved server default under its agent identity, while preserving explicit profile selection and saved conversations.
## [Android 1.16.1] - 2026-09-12
@@ -0,0 +1,116 @@
package com.hermesandroid.relay.viewmodel
import android.os.Handler
import android.os.Looper
import androidx.activity.ComponentActivity
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.test.assertIsDisplayed
import androidx.compose.ui.test.junit4.v2.createAndroidComposeRule
import androidx.compose.ui.test.onNodeWithContentDescription
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.performClick
import androidx.compose.ui.test.performImeAction
import androidx.compose.ui.test.performTextInput
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import com.hermesandroid.relay.network.upstream.ChatHandler
import com.hermesandroid.relay.network.upstream.DashboardApiClient
import com.hermesandroid.relay.network.upstream.GatewayChatClient
import com.hermesandroid.relay.network.upstream.HermesApiClient
import com.hermesandroid.relay.ui.components.MessageBubble
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.cancel
import kotlinx.coroutines.runBlocking
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import okhttp3.OkHttpClient
import okhttp3.WebSocket
import org.junit.After
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Before
import org.junit.Rule
import org.junit.Test
/** Production socket, ViewModel, transcript, Compose and IME actions on a virtual device. */
class ClarifyBatchInstrumentedTest {
@get:Rule val compose = createAndroidComposeRule<ComponentActivity>()
private lateinit var fixture: AndroidGatewayContractFixture
private lateinit var scope: CoroutineScope
private lateinit var gateway: GatewayChatClient
private lateinit var viewModel: ChatViewModel
private lateinit var handler: ChatHandler
private lateinit var socket: WebSocket
@Before fun setUp() {
fixture = AndroidGatewayContractFixture()
scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
val http = OkHttpClient()
gateway = GatewayChatClient(
initialDashboardClient = DashboardApiClient(fixture.server.url("/").toString().trimEnd('/'), http),
okHttpClient = http, scope = scope,
callbackDispatcher = { Handler(Looper.getMainLooper()).post(it) },
)
handler = ChatHandler().also { it.setSessionId("20260821_120000_fixture") }
viewModel = ChatViewModel().also {
it.initialize(HermesApiClient(fixture.server.url("/").toString(), "fixture-key"), handler)
it.streamingEndpoint = "gateway"
it.setProfileMessageLoader { Result.success(emptyList()) }
it.updateGatewayClient(gateway)
}
compose.setContent {
val messages by handler.messages.collectAsStateWithLifecycle()
MaterialTheme {
LazyColumn(Modifier.fillMaxSize()) {
items(messages.size, key = { messages[it].id }) { index ->
MessageBubble(messages[index], showTimestamps = false,
onCardInput = viewModel::answerAsk, animationEnabled = false)
}
}
}
}
assertTrue(runBlocking { gateway.prewarmAwait("20260821_120000_fixture") })
socket = fixture.awaitServerSocket()
}
@After fun tearDown() {
viewModel.updateGatewayClient(null)
gateway.shutdown()
scope.cancel()
fixture.shutdown()
}
@Test fun confirmedProgressSurvivesLifecycleAndCustomAnswerUsesIme() {
compose.runOnIdle { viewModel.sendMessage("Ask two questions") }
fixture.awaitRpc("prompt.submit")
socket.send(fixture.event("clarify.request", Json.parseToJsonElement("""
{"request_id":"batch-device","questions":[
{"qid":"route/a","question":"Which route?","choices":["Canary","Immediate"]},
{"qid":"notes:b","question":"Anything else?","choices":null}
]}
""") as JsonObject, "fixture-live-1"))
compose.waitUntil(10_000) { viewModel.pendingAsk.value != null }
compose.onNodeWithText("Canary").performClick()
compose.waitUntil(10_000) { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "Canary" }
assertEquals(JsonPrimitive("route/a"), fixture.awaitRpc("clarify.respond")["question_id"])
compose.activityRule.scenario.moveToState(Lifecycle.State.STARTED)
compose.activityRule.scenario.moveToState(Lifecycle.State.RESUMED)
compose.onNodeWithText("Question 2 of 2").assertIsDisplayed()
compose.onNodeWithContentDescription("Type an answer…").apply {
performClick()
performTextInput(" Keep rollback ready ")
performImeAction()
}
compose.waitUntil(10_000) { viewModel.pendingAsk.value == null }
compose.onNodeWithText("All questions answered").assertIsDisplayed()
assertEquals(2, fixture.rpcCount("clarify.respond"))
assertEquals(1, fixture.rpcCount("prompt.submit"))
}
}
@@ -5,6 +5,7 @@ import androidx.datastore.core.DataStore
import androidx.datastore.preferences.core.Preferences
import androidx.datastore.preferences.core.edit
import androidx.datastore.preferences.core.stringPreferencesKey
import com.hermesandroid.relay.network.upstream.GatewayClarifyQuestion
import kotlinx.coroutines.flow.first
import kotlinx.serialization.Serializable
import kotlinx.serialization.encodeToString
@@ -135,6 +136,9 @@ data class ChatTurnAskCheckpoint(
val text: String,
val choices: List<String>? = null,
val multiSelect: Boolean = false,
val questions: List<GatewayClarifyQuestion> = emptyList(),
val answers: Map<String, String> = emptyMap(),
val ownerId: String? = null,
val smartDenied: Boolean = false,
val envVar: String? = null,
val timeoutSeconds: Int,
@@ -71,6 +71,8 @@ data class HermesCard(
* actions.
*/
val input: HermesCardInput? = null,
/** Local Gateway batch; never an ordinary chat-message answer protocol. */
val clarifyBatch: HermesCardClarifyBatch? = null,
) {
object BuiltInTypes {
const val SKILL_RESULT = "skill_result"
@@ -96,6 +98,25 @@ data class HermesCard(
}
}
@Serializable
data class HermesCardClarifyBatch(
val questions: List<HermesCardClarifyQuestion>,
val expiresAtMillis: Long? = null,
)
@Serializable
data class HermesCardClarifyQuestion(
val key: String,
val question: String,
val input: HermesCardInput,
val answer: String? = null,
val submitting: Boolean = false,
)
/** Local callback identity. The RPC always uses the original qid, never this UI key. */
fun clarifyQuestionCardKey(cardKey: String, qid: String): String =
Json.encodeToString(listOf(cardKey, qid))
/**
* Interactive input slot on a [HermesCard]. The flags compose rather than
* branch — a sudo ask can be `masked + holdToConfirm` (password field whose
@@ -518,6 +518,16 @@ class ChatHandler {
}
}
/** Refresh only an existing local ask, preserving its dispatches and transcript position. */
fun updateAskCardMessage(messageId: String, card: HermesCard) {
_messages.update { list ->
list.map { message ->
if (message.clientOnly && message.matchesIdentity(messageId)) message.copy(cards = listOf(card))
else message
}
}
}
/**
* Edit-and-regenerate local truncation: drop [messageId] and everything
* after it. The gateway performs the authoritative truncation via
@@ -1586,6 +1586,7 @@ class GatewayChatClient(
claimedBackground?.pendingAsk?.let { ask ->
boundTurn.restoreInteraction(ask)
}
boundTurn.restorePendingClarify(response)
boundTurn.armWatchdog()
} else if (queued != null) {
synchronized(recoveryEventLock) { recoveryEvents = null }
@@ -1746,17 +1747,39 @@ class GatewayChatClient(
)
/** Answer a [GatewayAsk.Kind.CLARIFY] ask. */
suspend fun respondClarify(requestId: String, answer: String): Result<GatewayAskResponse> {
suspend fun respondClarify(
requestId: String,
answer: String,
questionId: String? = null,
): Result<GatewayAskResponse> {
val respondingTurn = activeTurn
if (questionId != null) {
val ask = respondingTurn?.pendingInteraction
// A lost acknowledgement is ambiguous until activation replays server progress.
// Never overwrite an accepted answer while reconnect is still reconciling it.
if (rejoinInProgress || ask?.kind != GatewayAsk.Kind.CLARIFY || ask.requestId != requestId ||
ask.questions.none { it.qid == questionId } || ask.ownershipToken.retired.get() ||
questionId in ask.ownershipToken.answers.get()
) return Result.failure(GatewayRpcException("Clarification is not ready for this question"))
}
val generation = respondingTurn?.interactionGeneration
val ownershipToken = respondingTurn?.pendingInteraction?.ownershipToken
return rpc(
"clarify.respond",
buildJsonObject {
put("request_id", requestId)
put("answer", answer)
questionId?.let { put("question_id", it) }
},
).map {
it.gatewayAskResponse().also {
respondingTurn?.acknowledgeInteraction(GatewayAskExpiry(GatewayAsk.Kind.CLARIFY, requestId))
if (generation != null) {
respondingTurn.acknowledgeClarify(requestId, questionId, answer, it == GatewayAskResponse.EXPIRED, generation)
}
val currentTurn = activeTurn
if (ownershipToken != null && currentTurn !== respondingTurn) {
currentTurn?.acknowledgeClarifyOwner(requestId, questionId, answer, it == GatewayAskResponse.EXPIRED, ownershipToken)
}
}
}
}
@@ -3854,7 +3877,10 @@ class GatewayChatClient(
val interactionRequest = GatewayEventMapper.interactionRequest(type, payload)
if (interactionRequest != null) {
val previous = backgroundTurn.pendingAsk
backgroundTurn.pendingAsk = interactionRequest
backgroundTurn.pendingAsk = if (previous?.kind == interactionRequest.kind &&
previous.requestId == interactionRequest.requestId && interactionRequest.questions.isNotEmpty()
) interactionRequest.withAnswers(previous.answers + interactionRequest.answers, previous)
else interactionRequest
if (previous?.kind != interactionRequest.kind ||
previous.requestId != interactionRequest.requestId
) {
@@ -3881,6 +3907,7 @@ class GatewayChatClient(
// be replayed or buffered. Only an authoritative expiry retires a
// detached ask; an explicit response is retired by its foreground VM.
if (explicitlyExpired) {
pendingAsk.ownershipToken.retired.set(true)
backgroundTurn.pendingAsk = null
callbackDispatcher {
backgroundInteractionListener?.invoke(
@@ -3998,6 +4025,9 @@ class GatewayChatClient(
}
return
}
if (type == "session.info" && eventSessionId != null && eventSessionId == liveSessionId) {
payload?.let(turn::restorePendingClarify)
}
turn.onEvent(type, payload)
if (turn.ended) {
if (activeTurn === turn) activeTurn = null
@@ -4204,6 +4234,7 @@ class GatewayChatClient(
activated.isSuccess -> {
activated.getOrNull()?.let { result ->
applySessionResultInfo(result)
turn.restorePendingClarify(result)
turn.settleFromAuthoritativeSessionState(
running = result.booleanField("running"),
source = "session.activate",
@@ -4506,6 +4537,19 @@ class GatewayChatClient(
fun acknowledgeInteraction(expiry: GatewayAskExpiry) {
mapper.acknowledgeInteraction(expiry)
}
val interactionGeneration: Long get() = mapper.interactionGeneration
fun acknowledgeClarify(requestId: String, questionId: String?, answer: String, expired: Boolean, generation: Long) {
mapper.acknowledgeClarify(requestId, questionId, answer, expired, generation)
}
fun acknowledgeClarifyOwner(requestId: String, questionId: String?, answer: String, expired: Boolean, owner: GatewayAskOwnership) {
mapper.acknowledgeClarifyOwner(requestId, questionId, answer, expired, owner)
}
fun restorePendingClarify(snapshot: JsonObject) {
val payload = snapshot["pending_clarify"] as? JsonObject
?: (snapshot["info"] as? JsonObject)?.get("pending_clarify") as? JsonObject
?: return
GatewayEventMapper.interactionRequest("clarify.request", payload)?.let(mapper::restoreInteraction)
}
private val deferredEventLock = Any()
private val deferredEvents = mutableListOf<Pair<String, JsonObject?>>()
private var eventsDeferred = deferEvents
@@ -32,19 +32,66 @@ class GatewayEventMapper(
var turnEnded: Boolean = false
private set
@get:Synchronized
internal val currentInteraction: GatewayAsk?
get() = pendingInteraction
@Volatile internal var interactionGeneration: Long = 0
private set
internal fun restoreInteraction(ask: GatewayAsk) {
val duplicate = pendingInteraction?.sameRequestAs(ask) == true
pendingInteraction = ask
if (!duplicate) callbacks.onInteractionRequest(ask)
@Synchronized internal fun restoreInteraction(ask: GatewayAsk) {
val previous = pendingInteraction
val duplicate = previous?.sameRequestAs(ask) == true
if (!duplicate) interactionGeneration++
val merged = if (duplicate && ask.questions.isNotEmpty()) {
ask.withAnswers(previous.answers + ask.answers, previous)
} else if (ask.questions.isNotEmpty()) ask.withAnswers(emptyMap()) else ask
if (merged.ownershipToken.retired.get() && !merged.clarifyComplete) {
if (duplicate) pendingInteraction = null
callbacks.onInteractionExpired(GatewayAskExpiry(merged.kind, merged.requestId))
if (pendingInteraction == null) drainDeferredTerminalEvent()
return
}
pendingInteraction = merged
if (!duplicate || previous != merged) callbacks.onInteractionRequest(merged)
if (merged.clarifyComplete) {
pendingInteraction = null
drainDeferredTerminalEvent()
}
}
@Synchronized internal fun acknowledgeClarify(
requestId: String,
questionId: String?,
answer: String,
expired: Boolean,
generation: Long = interactionGeneration,
) {
if (generation != interactionGeneration) return
val pending = pendingInteraction ?: return
if (pending.kind != GatewayAsk.Kind.CLARIFY || pending.requestId != requestId) return
if (!expired && questionId != null && pending.questions.none { it.qid == questionId }) return
if (!expired && questionId != null && pending.questions.any { it.qid == questionId }) {
val updated = pending.withAnswers(pending.answers + (questionId to answer))
if (updated.questions.any { it.qid !in updated.answers }) {
pendingInteraction = updated
return
}
}
acknowledgeInteraction(GatewayAskExpiry(GatewayAsk.Kind.CLARIFY, requestId))
}
@Synchronized internal fun acknowledgeClarifyOwner(
requestId: String, questionId: String?, answer: String, expired: Boolean, owner: GatewayAskOwnership,
) {
if (pendingInteraction?.ownershipToken !== owner) return
acknowledgeClarify(requestId, questionId, answer, expired)
}
/** Retire only the ask whose explicit respond RPC reached server truth. */
internal fun acknowledgeInteraction(expiry: GatewayAskExpiry) {
@Synchronized internal fun acknowledgeInteraction(expiry: GatewayAskExpiry) {
val pending = pendingInteraction ?: return
if (pending.matches(expiry)) {
pending.ownershipToken.retired.set(true)
pendingInteraction = null
drainDeferredTerminalEvent()
}
@@ -77,7 +124,7 @@ class GatewayEventMapper(
*/
private val generatingIdsByName = mutableMapOf<String, ArrayDeque<String>>()
fun onEvent(type: String, payload: JsonObject?) {
@Synchronized fun onEvent(type: String, payload: JsonObject?) {
if (turnEnded) return
interactionRequest(type, payload)?.let { ask ->
@@ -88,6 +135,7 @@ class GatewayEventMapper(
interactionExpiry(type, payload)?.let { expiry ->
val pending = pendingInteraction
if (pending != null && pending.matches(expiry)) {
pending.ownershipToken.retired.set(true)
pendingInteraction = null
}
callbacks.onInteractionExpired(expiry)
@@ -503,26 +551,7 @@ class GatewayEventMapper(
}
fun interactionRequest(type: String, payload: JsonObject?): GatewayAsk? = when (type) {
"clarify.request" -> {
val choices = (payload?.get("choices") as? JsonArray)
?.mapNotNull { (it as? JsonPrimitive)?.contentOrNull?.trim() }
?.filter { it.isNotEmpty() }
?.distinct()
?.take(MAX_CLARIFY_CHOICES)
?.takeIf { it.isNotEmpty() }
GatewayAsk(
kind = GatewayAsk.Kind.CLARIFY,
requestId = payload.string("request_id"),
text = payload.string("question") ?: "The agent needs clarification",
choices = choices,
multiSelect = payload.boolean("multi_select") == true && choices != null,
// Current upstream owns expiry through clarify.expire and
// does not advertise its configurable deadline. Never
// invent a local deadline; consume future additive
// metadata only when it is present and positive.
timeoutSeconds = payload.int("timeout_seconds")?.coerceAtLeast(0) ?: 0,
)
}
"clarify.request" -> clarifyRequest(payload)
"approval.request" -> GatewayAsk(
kind = GatewayAsk.Kind.APPROVAL,
@@ -555,6 +584,41 @@ class GatewayEventMapper(
else -> null
}
private fun clarifyRequest(payload: JsonObject?): GatewayAsk? {
val rawQuestions = payload?.get("questions")
if (rawQuestions != null && rawQuestions !is JsonArray) return null
val questions = rawQuestions?.map { value ->
val row = value as? JsonObject ?: return null
val qid = (row["qid"] as? JsonPrimitive)?.takeIf { it.isString }
?.contentOrNull?.takeIf(String::isNotBlank) ?: return null
val text = row.string("question")?.takeIf(String::isNotBlank) ?: return null
val choices = (row["choices"] as? JsonArray)?.mapNotNull {
(it as? JsonPrimitive)?.takeIf { option -> option.isString }
?.contentOrNull?.takeIf(String::isNotBlank)
}.orEmpty().take(MAX_CLARIFY_CHOICES)
GatewayClarifyQuestion(qid, text, choices, row.boolean("multi_select") == true && choices.isNotEmpty())
}.orEmpty()
if (questions.size > MAX_CLARIFY_QUESTIONS || questions.map { it.qid }.distinct().size != questions.size) return null
val choices = (payload?.get("choices") as? JsonArray)
?.mapNotNull { (it as? JsonPrimitive)?.contentOrNull?.trim() }
?.filter(String::isNotEmpty)?.distinct()?.take(MAX_CLARIFY_CHOICES)
?.takeIf { it.isNotEmpty() }
return GatewayAsk(
kind = GatewayAsk.Kind.CLARIFY,
requestId = payload.string("request_id"),
text = payload.string("question") ?: "The agent needs clarification",
choices = choices,
multiSelect = payload.boolean("multi_select") == true && choices != null,
// Upstream owns its configurable deadline; only consume advertised metadata.
timeoutSeconds = payload.int("timeout_seconds")?.coerceAtLeast(0) ?: 0,
questions = questions,
answers = (payload?.get("answers") as? JsonObject)?.mapNotNull { (qid, value) ->
(value as? JsonPrimitive)?.takeIf { it.isString }?.contentOrNull
?.takeIf { questions.any { q -> q.qid == qid } }?.let { qid to it }
}?.toMap().orEmpty(),
)
}
fun interactionExpiry(type: String, payload: JsonObject?): GatewayAskExpiry? = when (type) {
"clarify.expire" -> GatewayAskExpiry(
kind = GatewayAsk.Kind.CLARIFY,
@@ -634,6 +698,7 @@ class GatewayEventMapper(
// Upstream clarify tool accepts at most four choices. Sudo/secret retain fixed
// `_block()` timeouts; clarify is configurable and expires authoritatively.
private const val MAX_CLARIFY_CHOICES = 4
private const val MAX_CLARIFY_QUESTIONS = 5
private const val SUDO_TIMEOUT_SECONDS = 120
private const val SECRET_TIMEOUT_SECONDS = 300
@@ -2,11 +2,14 @@ package com.hermesandroid.relay.network.upstream
import com.hermesandroid.relay.network.upstream.models.MessageItem
import com.hermesandroid.relay.network.upstream.models.UsageInfo
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.JsonPrimitive
import kotlinx.serialization.json.booleanOrNull
import kotlinx.serialization.json.contentOrNull
import java.util.concurrent.atomic.AtomicBoolean
import java.util.concurrent.atomic.AtomicReference
/**
* Shared types for the Gateway chat transport — upstream hermes-agent's
@@ -230,10 +233,35 @@ data class GatewayAsk(
* authoritative `*.expire` event still retires the interaction.
*/
val timeoutSeconds: Int,
val questions: List<GatewayClarifyQuestion> = emptyList(),
val answers: Map<String, String> = emptyMap(),
) {
enum class Kind { CLARIFY, APPROVAL, SUDO, SECRET }
val clarifyComplete: Boolean get() = questions.isNotEmpty() && questions.all { it.qid in answers }
/** In-memory request incarnation shared when a live ask moves between turn mappers. */
internal var ownershipToken = GatewayAskOwnership(answers)
private set
internal fun withAnswers(answers: Map<String, String>, owner: GatewayAsk = this): GatewayAsk =
copy(answers = owner.ownershipToken.answers.updateAndGet { it + answers })
.also { it.ownershipToken = owner.ownershipToken }
}
/** A detached/reclaimed mapper shares confirmed progress with an RPC still owned by its predecessor. */
internal class GatewayAskOwnership(answers: Map<String, String>) {
val answers = AtomicReference(answers.toMap())
val retired = AtomicBoolean(false)
}
@Serializable
data class GatewayClarifyQuestion(
val qid: String,
val question: String,
val choices: List<String> = emptyList(),
val multiSelect: Boolean = false,
)
/**
* Server-side expiry of one blocking gateway interaction. Sudo/secret asks
* correlate by [requestId]; approvals remain session-scoped and therefore
@@ -0,0 +1,103 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.material3.LinearProgressIndicator
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.key
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalFocusManager
import androidx.compose.ui.res.stringResource
import androidx.compose.ui.semantics.LiveRegionMode
import androidx.compose.ui.semantics.heading
import androidx.compose.ui.semantics.liveRegion
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.unit.dp
import com.hermesandroid.relay.R
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import kotlinx.serialization.json.Json
/** One mobile question surface; confirmed qids advance without resubmitting earlier answers. */
@Composable
internal fun ClarifyBatchContent(
batch: HermesCardClarifyBatch,
expired: Boolean,
onInputSubmit: (String, String) -> Unit,
) {
val answered = batch.questions.filter { it.answer != null }
val activeIndex = batch.questions.indexOfFirst { it.answer == null }
val active = batch.questions.getOrNull(activeIndex)
var showAnswers by rememberSaveable { mutableStateOf(false) }
val focusManager = LocalFocusManager.current
LaunchedEffect(active?.key, expired) { focusManager.clearFocus() }
Column(Modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text(
text = when {
expired -> stringResource(R.string.clarify_batch_expired)
active == null -> stringResource(R.string.clarify_batch_complete)
else -> stringResource(R.string.clarify_batch_progress, activeIndex + 1, batch.questions.size)
},
style = MaterialTheme.typography.labelMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.semantics { liveRegion = LiveRegionMode.Polite },
)
if (batch.questions.size > 1) {
LinearProgressIndicator(
progress = { answered.size.toFloat() / batch.questions.size },
modifier = Modifier.fillMaxWidth(),
)
}
if (active != null && !expired) {
key(active.key) {
Text(
active.question,
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurface,
modifier = Modifier.semantics { heading() },
)
CardInputSlot(
input = active.input,
onSubmit = { if (!active.submitting) onInputSubmit(active.key, it) },
enabled = !active.submitting,
stackedChoices = true,
)
if (active.submitting) {
Text(stringResource(R.string.clarify_batch_sending), style = MaterialTheme.typography.labelMedium)
}
}
}
if (answered.isNotEmpty()) {
if (active == null || expired) {
Text(stringResource(R.string.clarify_batch_answered, answered.size), style = MaterialTheme.typography.labelLarge)
} else {
TextButton(onClick = { showAnswers = !showAnswers }) {
Text(stringResource(R.string.clarify_batch_answered, answered.size))
}
}
if (showAnswers || active == null || expired) {
answered.forEach { question ->
Text(question.question, style = MaterialTheme.typography.labelLarge)
val answer = if (question.input.multiSelect) {
runCatching { Json.decodeFromString<List<String>>(question.answer.orEmpty()).joinToString(", ") }
.getOrDefault(question.answer.orEmpty())
} else question.answer.orEmpty()
Text(answer, style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant)
Spacer(Modifier.height(4.dp))
}
}
}
}
}
@@ -59,6 +59,7 @@ import androidx.compose.runtime.mutableLongStateOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.runtime.saveable.rememberSaveable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
@@ -145,7 +146,7 @@ fun HermesCardBubble(
// Expiry clock for timed asks. Ticks once a second while the deadline
// is ahead; freezes after. Keyed on the deadline so a re-used card id
// with a fresh expiry restarts the loop.
val expiresAt = card.input?.expiresAtMillis
val expiresAt = card.input?.expiresAtMillis ?: card.clarifyBatch?.expiresAtMillis
var nowMillis by remember(expiresAt) { mutableLongStateOf(System.currentTimeMillis()) }
LaunchedEffect(expiresAt) {
if (expiresAt == null) return@LaunchedEffect
@@ -252,6 +253,14 @@ fun HermesCardBubble(
// action button all collapse the same way.
val input = card.input
when {
card.clarifyBatch != null -> {
Spacer(Modifier.height(10.dp))
ClarifyBatchContent(
batch = card.clarifyBatch,
expired = expired || alreadyChosen?.actionValue == HermesCardDispatch.EXPIRED_STAMP,
onInputSubmit = onInputSubmit,
)
}
alreadyChosen != null -> {
Spacer(Modifier.height(10.dp))
val chosenAction = card.actions.firstOrNull {
@@ -408,15 +417,18 @@ private fun ChoseRow(
*/
@OptIn(ExperimentalLayoutApi::class)
@Composable
private fun CardInputSlot(
internal fun CardInputSlot(
input: HermesCardInput,
onSubmit: (String) -> Unit,
enabled: Boolean = true,
stackedChoices: Boolean = false,
) {
// Deliberately remember, not rememberSaveable — a typed secret must
// never be written into the saved-instance-state Bundle.
var answerText by remember { mutableStateOf("") }
// Batch drafts survive lazy-item disposal. Secrets never enter saved state.
var answerText by if (stackedChoices && !input.masked) rememberSaveable { mutableStateOf("") }
else remember { mutableStateOf("") }
var reveal by remember { mutableStateOf(false) }
var selectedChoices by remember(input.choices) { mutableStateOf(emptyList<String>()) }
var selectedChoices by if (stackedChoices) rememberSaveable(input.choices) { mutableStateOf(emptyList<String>()) }
else remember(input.choices) { mutableStateOf(emptyList<String>()) }
val isMultiSelect = input.multiSelect && input.choices.isNotEmpty()
val showFreeText = !input.masked && (
@@ -429,7 +441,7 @@ private fun CardInputSlot(
val submitFreeText = {
val customAnswer = answerText.trim()
if (customAnswer.isNotEmpty()) {
if (enabled && customAnswer.isNotEmpty()) {
onSubmit(
if (isMultiSelect) {
encodeClarifyMultiSelectAnswer(selectedChoices + customAnswer)
@@ -443,14 +455,13 @@ private fun CardInputSlot(
Column(modifier = Modifier.fillMaxWidth()) {
// Choice chips
if (input.choices.isNotEmpty()) {
FlowRow(
horizontalArrangement = Arrangement.spacedBy(8.dp),
verticalArrangement = Arrangement.spacedBy(8.dp),
) {
val choices: @Composable () -> Unit = {
input.choices.forEach { choice ->
if (isMultiSelect) {
val selected = choice in selectedChoices
FilterChip(
enabled = enabled,
modifier = if (stackedChoices) Modifier.fillMaxWidth() else Modifier,
selected = selected,
onClick = {
selectedChoices = if (selected) {
@@ -459,7 +470,11 @@ private fun CardInputSlot(
selectedChoices + choice
}
},
label = { Text(choice, style = MaterialTheme.typography.labelMedium) },
label = {
Text(choice,
style = if (stackedChoices) MaterialTheme.typography.bodyMedium else MaterialTheme.typography.labelMedium,
modifier = if (stackedChoices) Modifier.padding(vertical = 8.dp) else Modifier)
},
leadingIcon = if (selected) {
{
Icon(
@@ -477,9 +492,13 @@ private fun CardInputSlot(
)
} else {
AssistChip(
enabled = enabled,
modifier = if (stackedChoices) Modifier.fillMaxWidth() else Modifier,
onClick = { onSubmit(choice) },
label = {
Text(choice, style = MaterialTheme.typography.labelMedium)
Text(choice,
style = if (stackedChoices) MaterialTheme.typography.bodyMedium else MaterialTheme.typography.labelMedium,
modifier = if (stackedChoices) Modifier.padding(vertical = 8.dp) else Modifier)
},
colors = AssistChipDefaults.assistChipColors(
containerColor = MaterialTheme.colorScheme.surfaceContainerHighest,
@@ -489,6 +508,11 @@ private fun CardInputSlot(
}
}
}
if (stackedChoices) {
Column(Modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(4.dp)) { choices() }
} else {
FlowRow(horizontalArrangement = Arrangement.spacedBy(8.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) { choices() }
}
}
// Masked secret field
@@ -534,12 +558,13 @@ private fun CardInputSlot(
else R.string.card_answer_placeholder,
),
onSubmit = submitFreeText,
enabled = enabled,
modifier = Modifier.weight(1f),
)
if (!isMultiSelect) {
IconButton(
onClick = submitFreeText,
enabled = answerText.isNotBlank(),
enabled = enabled && answerText.isNotBlank(),
) {
Icon(
imageVector = Icons.AutoMirrored.Filled.Send,
@@ -559,7 +584,7 @@ private fun CardInputSlot(
Spacer(Modifier.height(10.dp))
Button(
onClick = { onSubmit(encodeClarifyMultiSelectAnswer(answers)) },
enabled = answers.isNotEmpty(),
enabled = enabled && answers.isNotEmpty(),
) {
Text(
stringResource(R.string.card_submit),
@@ -615,6 +640,7 @@ private fun InlineAnswerField(
placeholder: String,
onSubmit: () -> Unit,
modifier: Modifier = Modifier,
enabled: Boolean = true,
) {
val shape = appearanceRoundedCornerShape(16.dp)
Box(
@@ -632,6 +658,7 @@ private fun InlineAnswerField(
)
}
BasicTextField(
enabled = enabled,
value = value,
onValueChange = onValueChange,
textStyle = MaterialTheme.typography.bodyMedium.copy(
@@ -639,7 +666,7 @@ private fun InlineAnswerField(
),
cursorBrush = SolidColor(MaterialTheme.colorScheme.primary),
keyboardOptions = KeyboardOptions(imeAction = ImeAction.Send),
keyboardActions = KeyboardActions(onSend = { onSubmit() }),
keyboardActions = KeyboardActions(onSend = { onSubmit() }, onDone = { onSubmit() }),
maxLines = 3,
modifier = Modifier
.fillMaxWidth()
@@ -23,14 +23,9 @@ import com.hermesandroid.relay.R
import com.hermesandroid.relay.viewmodel.ChatViewModel
/**
* Bottom-sheet audit of the exact extra context the agent is injected with on
* the next turn — opened by tapping the chat [ContextMeterBar].
*
* Renders the SAME [ChatViewModel.InjectedContext] the send path builds (via
* [ChatViewModel.previewInjectedContext] → `composeInjectedContext`), so it is
* a faithful audit, not a re-derivation that could drift. Empty blocks show a
* labeled note instead of vanishing, and the gateway's server-side persona is
* explicitly called out as not-sent-from-this-device.
* Transport-aware context preview, opened from the chat [ContextMeterBar].
* Unsupported blocks are labeled rather than represented as delivered.
* Server-reported configuration is separate from device-supplied context.
*/
@OptIn(ExperimentalMaterial3Api::class)
@Composable
@@ -53,7 +48,7 @@ fun InjectedContextSheet(
)
Spacer(Modifier.height(4.dp))
Text(
text = stringResource(R.string.injected_context_subtitle, context.transport),
text = stringResource(R.string.injected_context_subtitle),
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
@@ -71,6 +66,7 @@ fun InjectedContextSheet(
val mediaNoRelay = stringResource(R.string.injected_context_media_no_relay)
val relayNotSet = stringResource(R.string.injected_context_relay_not_set)
val turnNotSet = stringResource(R.string.injected_context_turn_not_set)
val unsupported = stringResource(R.string.injected_context_gateway_unsupported)
ContextSection(
title = personaTitle,
@@ -84,7 +80,7 @@ fun InjectedContextSheet(
ContextSection(
title = phoneStatusTitle,
body = context.appContext,
emptyNote = phoneStatusNotSet,
emptyNote = if (context.perTurnContextSupported) phoneStatusNotSet else unsupported,
)
ContextSection(
title = mediaTitle,
@@ -107,7 +103,7 @@ fun InjectedContextSheet(
ContextSection(
title = turnTitle,
body = context.interfaceContext,
emptyNote = turnNotSet,
emptyNote = if (context.perTurnContextSupported) turnNotSet else unsupported,
)
}
}
@@ -62,6 +62,7 @@ import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.RectangleShape
import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.hapticfeedback.HapticFeedbackType
import androidx.compose.ui.platform.LocalContext
@@ -214,7 +215,7 @@ fun MessageBubble(
message.role == MessageRole.USER -> MaterialTheme.colorScheme.primary
message.role == MessageRole.SYSTEM -> MaterialTheme.colorScheme.tertiaryContainer
isActionBubble -> MaterialTheme.colorScheme.tertiaryContainer.copy(alpha = 0.45f)
else -> MaterialTheme.colorScheme.surfaceVariant
else -> MaterialTheme.colorScheme.surfaceContainerLow
}
val textColor = when (message.role) {
@@ -436,8 +437,8 @@ fun MessageBubble(
// rows) would otherwise paint a bare timestamp-only chip between the
// Thought-process block and the tool pill. The first-token working state
// is rendered directly in the conversation
// lane below, without an opaque bubble. Cards and attachments still own
// a normal bubble even when response prose has not arrived yet.
// lane below, without an opaque bubble. Standalone cards own their own
// surface; wrapping those in another filled bubble duplicates the chrome.
streamingStatusLabel?.takeIf { showWorkingStatus }?.let { streamingStatus ->
StandaloneStreamingStatus(
status = streamingStatus,
@@ -454,6 +455,10 @@ fun MessageBubble(
message.cards.isNotEmpty() ||
message.attachments.isNotEmpty() ||
inlineImages.isNotEmpty()
val standaloneCards = !isUser && !isSystem &&
visibleMessageContent.isBlank() && quoteEnvelope == null &&
message.cards.isNotEmpty() && message.attachments.isEmpty() &&
inlineImages.isEmpty() && !showImageGeneration
if (showBubble) {
Row(
modifier = Modifier.widthIn(max = maxBubbleWidth),
@@ -599,11 +604,11 @@ fun MessageBubble(
),
) {
Surface(
shape = bubbleShape,
color = backgroundColor,
shape = if (standaloneCards) RectangleShape else bubbleShape,
color = if (standaloneCards) Color.Transparent else backgroundColor,
modifier = Modifier
.then(
if (!isUser && !isSystem && isDarkTheme) {
if (!isUser && !isSystem && isDarkTheme && !standaloneCards) {
Modifier.leftEdgeGlow(
alpha = 0.12f,
width = 28.dp,
@@ -651,7 +656,8 @@ fun MessageBubble(
)
) {
Column(
modifier = Modifier.padding(horizontal = 14.dp, vertical = 9.dp),
modifier = if (standaloneCards) Modifier
else Modifier.padding(horizontal = 14.dp, vertical = 9.dp),
) {
quoteEnvelope?.let { envelope ->
ChatQuoteReferenceChip(
@@ -741,7 +747,7 @@ fun MessageBubble(
onInputSubmit = { key, value ->
onCardInput(message.id, key, value)
},
maxWidth = maxBubbleWidth - 24.dp,
maxWidth = if (standaloneCards) maxBubbleWidth else maxBubbleWidth - 24.dp,
modifier = Modifier.padding(vertical = 2.dp),
)
}
@@ -823,6 +829,7 @@ fun MessageBubble(
val hasTokenUsage = showUsage && !isUser &&
(message.inputTokens != null || message.outputTokens != null)
val deliveryStatus = message.deliveryStatus?.takeIf { isUser }
// Timestamp — only on the LAST bubble of a same-author run so a
// burst of fragments doesn't stack three near-touching time labels.
@@ -831,13 +838,14 @@ fun MessageBubble(
// This row is reserved from the first streaming frame. Completion
// can reveal both timestamp and token usage without adding a new
// footer line or changing the bubble's measured height.
if (isLastInGroup && (showTimestamps || hasTokenUsage)) {
if ((isLastInGroup && (showTimestamps || hasTokenUsage)) || deliveryStatus != null) {
Spacer(modifier = Modifier.height(2.dp))
Row(
horizontalArrangement = Arrangement.spacedBy(6.dp),
verticalAlignment = Alignment.CenterVertically,
modifier = if (standaloneCards) Modifier.padding(horizontal = 4.dp) else Modifier,
) {
if (showTimestamps) Text(
if (isLastInGroup && showTimestamps) Text(
text = timeFormat.format(Date(message.timestamp)),
style = MaterialTheme.typography.labelSmall,
color = textColor.copy(alpha = if (message.isStreaming) 0f else 0.6f),
@@ -847,33 +855,30 @@ fun MessageBubble(
Modifier
},
)
if (hasTokenUsage) {
if (isLastInGroup && hasTokenUsage) {
TokenDisplay(
inputTokens = message.inputTokens,
outputTokens = message.outputTokens,
)
}
// Share the footer line, retaining the user bubble's contrasting foreground.
deliveryStatus?.let { status ->
MessageDeliveryIndicator(
status = status,
contentColor = textColor,
text = MessageDeliveryIndicatorText(
sending = stringResource(R.string.msg_bubble_sending),
queued = stringResource(R.string.msg_bubble_queued),
steered = stringResource(R.string.msg_bubble_steered),
delivered = stringResource(R.string.msg_bubble_delivered),
failed = stringResource(R.string.msg_bubble_not_sent),
tapToRetry = stringResource(R.string.chat_retry),
),
)
}
}
}
// Delivery status for local user messages, including steering.
// Use the bubble's foreground: accent-on-accent hides the label.
message.deliveryStatus?.takeIf { isUser }?.let { status ->
Spacer(modifier = Modifier.height(2.dp))
MessageDeliveryIndicator(
status = status,
contentColor = textColor,
text = MessageDeliveryIndicatorText(
sending = stringResource(R.string.msg_bubble_sending),
queued = stringResource(R.string.msg_bubble_queued),
steered = stringResource(R.string.msg_bubble_steered),
delivered = stringResource(R.string.msg_bubble_delivered),
failed = stringResource(R.string.msg_bubble_not_sent),
tapToRetry = stringResource(R.string.chat_retry),
),
)
}
// Non-tail historical fragments have no reserved timestamp row.
// Preserve their existing standalone token metadata layout.
if (!isLastInGroup && hasTokenUsage) {
@@ -3215,8 +3215,8 @@ fun ChatScreen(
)
}
if (!supervised && showContextSheet) {
// Live audit of the exact extra context the agent will be
// injected with on the next turn (transparency / auditability).
// Snapshot of supported client context and separately reported
// server configuration, not a delivery receipt.
InjectedContextSheet(
context = remember(showContextSheet) {
chatViewModel.previewInjectedContext()
@@ -72,6 +72,9 @@ import com.hermesandroid.relay.network.upstream.ActiveTurnKeepAliveRegistry
import com.hermesandroid.relay.network.upstream.GatewayAsk
import com.hermesandroid.relay.network.upstream.GatewayAskExpiry
import com.hermesandroid.relay.network.upstream.GatewayAskResponse
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import com.hermesandroid.relay.data.HermesCardClarifyQuestion
import com.hermesandroid.relay.data.clarifyQuestionCardKey
import com.hermesandroid.relay.network.upstream.GatewayAgentNotice
import com.hermesandroid.relay.network.upstream.GatewayActiveSession
import com.hermesandroid.relay.network.upstream.GatewayActiveSessionStatus
@@ -3652,7 +3655,7 @@ class ChatViewModel : ViewModel() {
private val backgroundPendingInteractions =
ConcurrentHashMap<TurnCheckpointKey, BackgroundPendingInteraction>()
/** Ask cardKeys with a respond RPC in flight — blocks double-taps until it settles. */
/** Request-incarnation/card keys with a respond RPC in flight. */
private val answeredAskIds = mutableSetOf<String>()
/**
@@ -6774,7 +6777,18 @@ class ChatViewModel : ViewModel() {
existing.ask.kind == ask.kind &&
existing.ask.requestId == ask.requestId
) {
sessionId?.let { maybeNotifyInteraction(it, existing.ask) }
if (ask.questions.isNotEmpty()) {
val updated = existing.copy(ask = ask.copy(answers = existing.ask.answers + ask.answers))
_pendingAsk.value = updated
updateClarifyBatchCard(handler, updated)
if (updated.ask.clarifyComplete) {
_pendingAsk.value = null
updated.sessionId?.let { cancelInteractionNotification(it, updated.ask) }
activeTurnCheckpointKey()?.let { ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false) }
}
scheduleCheckpointWrite(immediate = true)
}
_pendingAsk.value?.let { pending -> sessionId?.let { maybeNotifyInteraction(it, pending.ask) } }
return
}
existing?.let { pending ->
@@ -6787,8 +6801,11 @@ class ChatViewModel : ViewModel() {
publishBackgroundSessionActivity()
}
val now = restored?.receivedAt ?: System.currentTimeMillis()
val cardKey = restored?.cardKey ?: ask.requestId
val proposedCardKey = restored?.cardKey ?: ask.requestId
?: "approval-${handler.currentSessionId.value ?: "session"}-$now"
val cardKey = if (restored == null && handler.messages.value.any { message ->
message.cards.any { it.id == proposedCardKey }
}) "$proposedCardKey-${java.util.UUID.randomUUID()}" else proposedCardKey
val expiresAt = ask.timeoutSeconds.takeIf { it > 0 }?.let { now + it * 1_000L }
val card = when (ask.kind) {
GatewayAsk.Kind.APPROVAL -> HermesCard(
@@ -6887,12 +6904,52 @@ class ChatViewModel : ViewModel() {
contextKey = contextKey,
sessionId = sessionId,
receivedAt = now,
ownerId = restored?.ownerId ?: java.util.UUID.randomUUID().toString(),
)
if (ask.questions.isNotEmpty()) updateClarifyBatchCard(handler, requireNotNull(_pendingAsk.value))
if (ask.clarifyComplete) {
_pendingAsk.value = null
activeKey?.let { ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false) }
scheduleCheckpointWrite(immediate = true)
return
}
activeKey?.let { ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), true) }
scheduleCheckpointWrite(immediate = true)
sessionId?.let { maybeNotifyInteraction(it, ask) }
}
private fun updateClarifyBatchCard(handler: ChatHandler, pending: PendingAsk) {
val ask = pending.ask
if (ask.questions.isEmpty()) return
handler.updateAskCardMessage(
pending.messageId,
HermesCard(
type = HermesCard.BuiltInTypes.ASK_CLARIFY,
title = appContext?.getString(R.string.chat_approval_clarify_title) ?: "Hermes needs clarification",
accent = HermesCard.Accents.INFO,
id = pending.cardKey,
clarifyBatch = HermesCardClarifyBatch(
questions = ask.questions.map { question ->
val key = clarifyQuestionCardKey(pending.cardKey, question.qid)
HermesCardClarifyQuestion(
key = key,
question = question.question,
input = HermesCardInput(
kind = if (question.choices.isEmpty()) HermesCardInput.Kinds.TEXT else HermesCardInput.Kinds.CHOICE,
choices = question.choices,
multiSelect = question.multiSelect,
allowFreeText = true,
),
answer = ask.answers[question.qid],
submitting = "${pending.ownerId}:$key" in answeredAskIds,
)
},
expiresAtMillis = ask.timeoutSeconds.takeIf { it > 0 }?.let { pending.receivedAt + it * 1_000L },
),
),
)
}
/**
* Supervised Chat never exposes approval, clarification, sudo, or secret
* inputs. Settle the upstream interaction immediately with its safest
@@ -6966,21 +7023,35 @@ class ChatViewModel : ViewModel() {
fun answerAsk(messageId: String, cardKey: String, value: String) {
val handler = chatHandler ?: return
val pending = _pendingAsk.value
val question = pending?.ask?.questions?.firstOrNull {
clarifyQuestionCardKey(pending.cardKey, it.qid) == cardKey
}
if (pending == null ||
pending.cardKey != cardKey ||
pending.messageId != messageId ||
(if (pending.ask.questions.isNotEmpty()) question == null else pending.cardKey != cardKey) ||
pending.contextKey != activeProfileContextKey ||
pending.sessionId != handler.currentSessionId.value
) {
handler.addSystemNotice("This request is no longer active.")
return
}
if (pending.ask.kind == GatewayAsk.Kind.CLARIFY && value.isBlank()) return
if (pending.ask.kind == GatewayAsk.Kind.CLARIFY && pending.ask.timeoutSeconds > 0 &&
System.currentTimeMillis() >= pending.receivedAt + pending.ask.timeoutSeconds * 1_000L
) {
expirePendingAsk(GatewayAskExpiry(pending.ask.kind, pending.ask.requestId))
return
}
if (question != null && question.qid in pending.ask.answers) return
val gateway = gatewayClient
if (gateway == null) {
emitError(Exception("Gateway is not connected"), context = "send_message")
return
}
// In-flight guard: one respond RPC per card at a time.
if (!answeredAskIds.add(cardKey)) return
// Include the request incarnation so a late completion cannot unlock a reused id.
val flightKey = "${pending.ownerId}:$cardKey"
if (!answeredAskIds.add(flightKey)) return
updateClarifyBatchCard(handler, pending)
val ask = pending.ask
val stampValue = when (ask.kind) {
// Empty sudo password = decline — stamp matches the Deny action
@@ -6992,11 +7063,19 @@ class ChatViewModel : ViewModel() {
else -> value
}
viewModelScope.launch {
fun ownsResponse(): Boolean = chatHandler === handler && gatewayClient === gateway &&
activeProfileContextKey == pending.contextKey &&
handler.currentSessionId.value == pending.sessionId &&
_pendingAsk.value?.ownerId == pending.ownerId
if (!ownsResponse()) {
answeredAskIds.remove(flightKey)
return@launch
}
val requestId = ask.requestId
val result = when (ask.kind) {
GatewayAsk.Kind.APPROVAL -> gateway.respondApproval(choice = value)
GatewayAsk.Kind.CLARIFY ->
requestId?.let { gateway.respondClarify(it, value) }
requestId?.let { gateway.respondClarify(it, value.trim(), question?.qid) }
?: Result.failure(GatewayRpcException("ask has no request id"))
GatewayAsk.Kind.SUDO ->
requestId?.let { gateway.respondSudo(it, value) }
@@ -7007,6 +7086,8 @@ class ChatViewModel : ViewModel() {
}
result.fold(
onSuccess = { response ->
answeredAskIds.remove(flightKey)
if (!ownsResponse()) return@fold
if (response == GatewayAskResponse.EXPIRED) {
expirePendingAsk(
GatewayAskExpiry(
@@ -7016,10 +7097,24 @@ class ChatViewModel : ViewModel() {
)
return@fold
}
if (question != null) {
val current = requireNotNull(_pendingAsk.value)
val updated = current.copy(ask = current.ask.copy(answers = current.ask.answers + (question.qid to value.trim())))
updateClarifyBatchCard(handler, updated)
if (updated.ask.questions.all { it.qid in updated.ask.answers }) {
updated.sessionId?.let { cancelInteractionNotification(it, updated.ask) }
_pendingAsk.value = null
activeTurnCheckpointKey()?.let { ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false) }
} else {
_pendingAsk.value = updated
}
scheduleCheckpointWrite(immediate = true)
return@fold
}
// Collapse only after the server confirms — a failed RPC
// must leave the card answerable for a retry.
handler.recordCardDispatch(pending.messageId, cardKey, stampValue)
if (_pendingAsk.value === pending) {
if (ownsResponse()) {
pending.sessionId?.let { cancelInteractionNotification(it, pending.ask) }
_pendingAsk.value = null
activeTurnCheckpointKey()?.let {
@@ -7029,7 +7124,9 @@ class ChatViewModel : ViewModel() {
}
},
onFailure = { e ->
answeredAskIds.remove(cardKey)
answeredAskIds.remove(flightKey)
if (!ownsResponse()) return@fold
updateClarifyBatchCard(handler, requireNotNull(_pendingAsk.value))
emitError(e, context = "send_message")
},
)
@@ -7054,7 +7151,6 @@ class ChatViewModel : ViewModel() {
activeTurnCheckpointKey()?.let {
ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false)
}
answeredAskIds.remove(pending.cardKey)
scheduleCheckpointWrite(immediate = true)
chatHandler?.recordCardDispatch(
pending.messageId,
@@ -7077,7 +7173,9 @@ class ChatViewModel : ViewModel() {
ActiveTurnKeepAliveRegistry.setWaiting(it.keepAliveKey(), false)
}
scheduleCheckpointWrite(immediate = true)
if (pending.ask.kind == GatewayAsk.Kind.APPROVAL) {
if (pending.ask.kind == GatewayAsk.Kind.CLARIFY) {
chatHandler?.recordCardDispatch(pending.messageId, pending.cardKey, HermesCardDispatch.EXPIRED_STAMP)
} else if (pending.ask.kind == GatewayAsk.Kind.APPROVAL) {
chatHandler?.recordCardDispatch(pending.messageId, pending.cardKey, "deny")
}
}
@@ -7685,6 +7783,9 @@ class ChatViewModel : ViewModel() {
text = ask.ask.text,
choices = ask.ask.choices,
multiSelect = ask.ask.multiSelect,
questions = ask.ask.questions,
answers = ask.ask.answers,
ownerId = ask.ownerId,
smartDenied = ask.ask.smartDenied,
envVar = ask.ask.envVar,
timeoutSeconds = ask.ask.timeoutSeconds,
@@ -7893,6 +7994,8 @@ class ChatViewModel : ViewModel() {
text = saved.text,
choices = saved.choices,
multiSelect = saved.multiSelect,
questions = saved.questions,
answers = saved.answers,
smartDenied = saved.smartDenied,
envVar = saved.envVar,
timeoutSeconds = saved.timeoutSeconds,
@@ -9667,8 +9770,8 @@ class ChatViewModel : ViewModel() {
* 0. **Gateway transport** — the server owns the persona end-to-end: the
* session is bound to the selected profile (SOUL applied server-side) and
* the personality overlay rides `config.set`/`ephemeral_system_prompt`.
* The phone sends NO persona/profile prompt (only the phone-status block)
* so it can't double-apply. Cases 1–3 are the SSE-fallback rules.
* The phone sends no per-turn system context on Gateway.
* Cases 1–3 are the explicit API-only transport rules.
* 1. **Selected profile with a non-blank [Profile.systemMessage]** —
* profile wins outright. Profile is a richer, newer concept than
* personality: it bundles model + persona (from the profile's
@@ -9683,15 +9786,12 @@ class ChatViewModel : ViewModel() {
* configured default.
*
* The phone-status [appContextSettings] block is appended to whichever
* of the above wins (or sent alone in case 3), so the LLM always sees
* phone state regardless of persona source.
* of the API-only cases above wins (or sent alone in case 3).
*/
/**
* The exact `system_message` (`ephemeral_system_prompt`) injected for a
* turn, split into labeled blocks. Single source of truth shared by
* [startStream] (which sends [combinedSystemMessage]) and
* [previewInjectedContext] (which renders it in the chat audit sheet) so
* the preview can never drift from what is actually sent.
* Context prepared for the selected transport, split into labeled blocks.
* This is a preview, not a delivery receipt or the complete agent prompt.
* Gateway has no general per-turn system-context slot.
*/
data class InjectedContext(
val personaPrompt: String?,
@@ -9722,6 +9822,8 @@ class ChatViewModel : ViewModel() {
* audit UI labels that block "added server-side".
*/
val personaOwnedServerSide: Boolean,
/** False on Gateway; unsupported blocks are excluded from the payload and preview. */
val perTurnContextSupported: Boolean,
)
/**
@@ -9757,11 +9859,14 @@ class ChatViewModel : ViewModel() {
} else {
_sseToolNames.value
}
val appContextRaw = buildPromptBlock(
settings = appContextSettings,
snapshot = capturePhoneSnapshot(),
availableTools = availableTools,
)
val appContextRaw = if (!gateway && appContextSettings.master) {
buildPromptBlock(
settings = appContextSettings,
snapshot = capturePhoneSnapshot(),
availableTools = availableTools,
)
} else null
val interfaceContext = interfaceContextPrompt?.takeIf { !gateway && it.isNotBlank() }
// Gateway has no per-turn system slot. SSE carries the standard
// Dashboard route first, then the optional Relay enhancement.
val upstreamMediaAvailable = dashboardMediaClientProvider?.invoke() != null
@@ -9773,25 +9878,26 @@ class ChatViewModel : ViewModel() {
// block (a stable environment fact, like phone status) and before the
// per-turn interface context. The per-block fields below null out blanks
// only for display.
val combined = listOfNotNull(personaPrompt, appContextRaw, mediaCapability, interfaceContextPrompt)
val combined = listOfNotNull(personaPrompt, appContextRaw, mediaCapability, interfaceContext)
.joinToString("\n\n")
.ifBlank { null }
return InjectedContext(
personaPrompt = personaPrompt,
appContext = appContextRaw?.takeIf { it.isNotBlank() },
interfaceContext = interfaceContextPrompt?.takeIf { it.isNotBlank() },
interfaceContext = interfaceContext,
mediaCapability = mediaCapability,
relayServerBlocks = relayServerBlocks,
relayMediaAvailable = relayMediaAvailable,
combinedSystemMessage = combined,
transport = streamingEndpoint,
personaOwnedServerSide = gateway,
perTurnContextSupported = !gateway,
)
}
/**
* Live audit snapshot of what the agent will be injected with on the next
* turn — rendered by the chat context sheet. Per-turn voice context is null
* Preview of context supported by the selected transport, plus separately
* reported Relay configuration. Per-turn voice context is null
* here (it's set only on a spoken turn); the UI notes that.
*/
fun previewInjectedContext(): InjectedContext {
@@ -11843,6 +11949,7 @@ data class PendingAsk(
/** Stored session id within [contextKey]; approvals are session-scoped upstream. */
val sessionId: String?,
val receivedAt: Long = System.currentTimeMillis(),
val ownerId: String = java.util.UUID.randomUUID().toString(),
)
/**
+10 -4
View File
@@ -1,5 +1,10 @@
<?xml version='1.0' encoding='utf-8'?>
<resources>
<string name="clarify_batch_progress">Pergunta %1$d de %2$d</string>
<string name="clarify_batch_complete">Todas as perguntas respondidas</string>
<string name="clarify_batch_expired">Esta solicitação terminou</string>
<string name="clarify_batch_sending">Enviando resposta…</string>
<string name="clarify_batch_answered">Perguntas respondidas (%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">interface do agente</string>
@@ -757,7 +762,7 @@
<string name="chat_settings_notify_when_finishes">Alertas de chat em segundo plano</string>
<string name="chat_settings_notify_when_finishes_desc">Notifique quando o Hermes precisar de uma resposta ou terminar em segundo plano</string>
<string name="chat_settings_share_phone_status">Compartilhar o status do celular com o agente</string>
<string name="chat_settings_share_phone_status_desc">Inclua uma breve mensagem do sistema sobre o app e o celular em cada turno do chat</string>
<string name="chat_settings_share_phone_status_desc">Incluir contexto do app e do telefone em chats apenas por API. Não é enviado nos chats padrão do Gateway.</string>
<string name="chat_settings_bridge_permissions">Bridge + permissões</string>
<string name="chat_settings_bridge_permissions_desc">Se acessibilidade, captura de tela, sobreposição e notificações foram concedidas</string>
<string name="chat_settings_foreground_app">App em primeiro plano</string>
@@ -766,7 +771,7 @@
<string name="chat_settings_battery_level_desc">Porcentagem atual da bateria. Desativado por padrão.</string>
<string name="chat_settings_safety_rails">Proteções de segurança</string>
<string name="chat_settings_safety_rails_desc">Quantidade de itens bloqueados, quantidade de verbos destrutivos e temporizador de desativação automática</string>
<string name="chat_settings_preview">Prévia</string>
<string name="chat_settings_preview">Exemplo de chat apenas por API</string>
<string name="chat_settings_no_system_message">(nenhuma mensagem do sistema será enviada)</string>
<string name="chat_settings_parse_tool_annotations">Analisar anotações de ferramentas</string>
<string name="chat_settings_experimental">Experimental</string>
@@ -3196,10 +3201,10 @@
<string name="injected_context_phone_status_title">Status do celular</string>
<string name="injected_context_relay_not_set">Nenhum relay configurado</string>
<string name="injected_context_relay_title">Relay</string>
<string name="injected_context_subtitle">Contexto enviado com a próxima mensagem</string>
<string name="injected_context_subtitle">Prévia deste chat. O contexto do servidor é configurado separadamente.</string>
<string name="injected_context_title">Contexto injetado</string>
<string name="injected_context_turn_not_set">Não definido</string>
<string name="injected_context_turn_title">Modo de detecção de turno</string>
<string name="injected_context_turn_title">Contexto do turno</string>
<string name="model_picker_cd_clear">Limpar</string>
<string name="model_picker_count">%d modelos</string>
<string name="power_feature_back_desc">Voltar</string>
@@ -4447,4 +4452,5 @@
<string name="voice_overlay_setup_start">Iniciar sobreposição de voz</string>
<string name="voice_overlay_settings_hint">Opcional nas duas versões. Abra o foco de voz no Chat e escolha Sobreposição. As permissões sozinhas nunca iniciam a escuta.</string>
<string name="voice_overlay_reset_position">Redefinir posição</string>
<string name="injected_context_gateway_unsupported">Não é enviado no chat do Gateway. Esta conexão não oferece suporte a contexto adicional por turno.</string>
</resources>
+10 -4
View File
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">问题 %1$d / %2$d</string>
<string name="clarify_batch_complete">所有问题均已回答</string>
<string name="clarify_batch_expired">此请求已结束</string>
<string name="clarify_batch_sending">正在发送回答…</string>
<string name="clarify_batch_answered">已回答的问题(%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">代理界面</string>
@@ -796,7 +801,7 @@
<string name="chat_settings_notify_when_finishes">后台聊天提醒</string>
<string name="chat_settings_notify_when_finishes_desc">Hermes 在后台需要输入或完成回复时通知</string>
<string name="chat_settings_share_phone_status">与代理分享手机状态</string>
<string name="chat_settings_share_phone_status_desc">每轮聊天附带一条关于应用和手机的简短系统消息</string>
<string name="chat_settings_share_phone_status_desc">在仅使用 API 的聊天中包含应用和手机上下文。标准 Gateway 聊天不会发送此内容。</string>
<string name="chat_settings_bridge_permissions">Bridge + 权限</string>
<string name="chat_settings_bridge_permissions_desc">无障碍、屏幕捕获、悬浮窗、通知是否已授权</string>
<string name="chat_settings_foreground_app">前台应用</string>
@@ -805,7 +810,7 @@
<string name="chat_settings_battery_level_desc">当前电池百分比。默认关闭。</string>
<string name="chat_settings_safety_rails">安全护栏</string>
<string name="chat_settings_safety_rails_desc">黑名单数量、破坏性动词数量和自动禁用计时器</string>
<string name="chat_settings_preview">预览</string>
<string name="chat_settings_preview">仅使用 API 的聊天示例</string>
<string name="chat_settings_no_system_message">(不会发送系统消息)</string>
<string name="chat_settings_parse_tool_annotations">解析工具标注</string>
<string name="chat_settings_experimental">实验性</string>
@@ -3295,10 +3300,10 @@
<string name="injected_context_phone_status_title">手机状态</string>
<string name="injected_context_relay_not_set">未配置 Relay</string>
<string name="injected_context_relay_title">Relay</string>
<string name="injected_context_subtitle">随下一条消息发送的上下文</string>
<string name="injected_context_subtitle">此聊天的预览。服务器端上下文单独配置。</string>
<string name="injected_context_title">注入上下文</string>
<string name="injected_context_turn_not_set">未设置</string>
<string name="injected_context_turn_title">轮换检测模式</string>
<string name="injected_context_turn_title">本轮上下文</string>
<string name="model_picker_cd_clear">清空</string>
<string name="model_picker_count">%d 个模型</string>
<string name="power_feature_back_desc">返回</string>
@@ -4528,4 +4533,5 @@
<string name="voice_overlay_setup_start">启动语音悬浮窗</string>
<string name="voice_overlay_settings_hint">两个版本均可选择使用。在聊天的语音专注模式中选择悬浮窗。仅授予权限不会开始监听。</string>
<string name="voice_overlay_reset_position">重置位置</string>
<string name="injected_context_gateway_unsupported">Gateway 聊天不会发送此内容。此连接不支持每轮附加上下文。</string>
</resources>
+10 -4
View File
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">Frage %1$d von %2$d</string>
<string name="clarify_batch_complete">Alle Fragen beantwortet</string>
<string name="clarify_batch_expired">Diese Anfrage ist beendet</string>
<string name="clarify_batch_sending">Antwort wird gesendet…</string>
<string name="clarify_batch_answered">Beantwortete Fragen (%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">Agentenoberfläche</string>
@@ -799,7 +804,7 @@
<string name="chat_settings_notify_when_finishes">Chat-Benachrichtigungen im Hintergrund</string>
<string name="chat_settings_notify_when_finishes_desc">Benachrichtigen, wenn Hermes Eingaben benötigt oder im Hintergrund fertig wird</string>
<string name="chat_settings_share_phone_status">Smartphone-Status mit Agent teilen</string>
<string name="chat_settings_share_phone_status_desc">Bei jedem Chatdurchlauf eine kurze Systemnachricht über App und Smartphone einfügen</string>
<string name="chat_settings_share_phone_status_desc">App- und Smartphone-Kontext in reinen API-Chats mitsenden. Wird in Standard-Gateway-Chats nicht gesendet.</string>
<string name="chat_settings_bridge_permissions">Bridge + Berechtigungen</string>
<string name="chat_settings_bridge_permissions_desc">Ob Bedienungshilfe, Bildschirmaufnahme, Overlay und Benachrichtigungen erlaubt sind</string>
<string name="chat_settings_foreground_app">App im Vordergrund</string>
@@ -808,7 +813,7 @@
<string name="chat_settings_battery_level_desc">Aktueller Akkustand in Prozent. Standardmäßig aus.</string>
<string name="chat_settings_safety_rails">Schutzmechanismen</string>
<string name="chat_settings_safety_rails_desc">Anzahl gesperrter Einträge und destruktiver Verben sowie Timer zur automatischen Deaktivierung</string>
<string name="chat_settings_preview">Vorschau</string>
<string name="chat_settings_preview">Beispiel für einen reinen API-Chat</string>
<string name="chat_settings_no_system_message">(es wird keine Systemnachricht gesendet)</string>
<string name="chat_settings_parse_tool_annotations">Werkzeughinweise auswerten</string>
<string name="chat_settings_experimental">Experimentell</string>
@@ -3364,10 +3369,10 @@
<string name="injected_context_phone_status_title">Smartphone-Status</string>
<string name="injected_context_relay_not_set">Kein Relay konfiguriert</string>
<string name="injected_context_relay_title">Relay</string>
<string name="injected_context_subtitle">Kontext, der mit der nächsten Nachricht gesendet wird</string>
<string name="injected_context_subtitle">Vorschau für diesen Chat. Serverseitiger Kontext wird separat konfiguriert.</string>
<string name="injected_context_title">Eingefügter Kontext</string>
<string name="injected_context_turn_not_set">Nicht festgelegt</string>
<string name="injected_context_turn_title">Modus zur Durchlauferkennung</string>
<string name="injected_context_turn_title">Nachrichtenkontext</string>
<string name="model_picker_cd_clear">Leeren</string>
<string name="model_picker_count">%d Modelle</string>
<string name="power_feature_back_desc">Zurück</string>
@@ -4604,4 +4609,5 @@
<string name="voice_overlay_setup_start">Sprach-Overlay starten</string>
<string name="voice_overlay_settings_hint">In beiden Versionen optional. Öffne den Sprachfokus im Chat und wähle Overlay. Berechtigungen allein starten kein Zuhören.</string>
<string name="voice_overlay_reset_position">Position zurücksetzen</string>
<string name="injected_context_gateway_unsupported">Wird im Gateway-Chat nicht gesendet. Diese Verbindung unterstützt keinen zusätzlichen Kontext pro Nachricht.</string>
</resources>
+10 -4
View File
@@ -1,5 +1,10 @@
<?xml version='1.0' encoding='utf-8'?>
<resources>
<string name="clarify_batch_progress">Pregunta %1$d de %2$d</string>
<string name="clarify_batch_complete">Todas las preguntas respondidas</string>
<string name="clarify_batch_expired">Esta solicitud ha finalizado</string>
<string name="clarify_batch_sending">Enviando respuesta…</string>
<string name="clarify_batch_answered">Preguntas respondidas (%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">interfaz del agente</string>
@@ -724,7 +729,7 @@
<string name="chat_settings_notify_when_finishes">Alertas de chat en segundo plano</string>
<string name="chat_settings_notify_when_finishes_desc">Notificar cuando Hermes necesite información o termine en segundo plano</string>
<string name="chat_settings_share_phone_status">Compartir el estado del teléfono con el agente</string>
<string name="chat_settings_share_phone_status_desc">Incluya un breve mensaje del sistema sobre la aplicación y el teléfono en cada turno de chat.</string>
<string name="chat_settings_share_phone_status_desc">Incluir contexto de la app y el teléfono en chats solo por API. No se envía en los chats estándar de Gateway.</string>
<string name="chat_settings_bridge_permissions">Bridge + permisos</string>
<string name="chat_settings_bridge_permissions_desc">Si se concede accesibilidad, captura de pantalla, superposición o notificaciones</string>
<string name="chat_settings_foreground_app">Aplicación en primer plano</string>
@@ -733,7 +738,7 @@
<string name="chat_settings_battery_level_desc">Porcentaje de batería actual. Desactivado de forma predeterminada.</string>
<string name="chat_settings_safety_rails">Barandillas de seguridad</string>
<string name="chat_settings_safety_rails_desc">Recuento de listas de bloqueo, recuento de verbos destructivos y temporizador de desactivación automática</string>
<string name="chat_settings_preview">Avance</string>
<string name="chat_settings_preview">Ejemplo de chat solo por API</string>
<string name="chat_settings_no_system_message">(no se enviará ningún mensaje del sistema)</string>
<string name="chat_settings_parse_tool_annotations">Anotaciones de la herramienta de análisis</string>
<string name="chat_settings_experimental">Experimental</string>
@@ -3028,10 +3033,10 @@
<string name="injected_context_phone_status_title">Estado del teléfono</string>
<string name="injected_context_relay_not_set">Ningún relay configurado</string>
<string name="injected_context_relay_title">Relay</string>
<string name="injected_context_subtitle">Contexto enviado con el siguiente mensaje.</string>
<string name="injected_context_subtitle">Vista previa de este chat. El contexto del servidor se configura por separado.</string>
<string name="injected_context_title">Contexto inyectado</string>
<string name="injected_context_turn_not_set">No establecido</string>
<string name="injected_context_turn_title">Modo de detección de giro</string>
<string name="injected_context_turn_title">Contexto del turno</string>
<string name="model_picker_cd_clear">Borrar</string>
<string name="model_picker_count">Modelos %d</string>
<string name="power_feature_back_desc">Atrás</string>
@@ -4295,4 +4300,5 @@
<string name="voice_overlay_setup_start">Iniciar ventana de voz</string>
<string name="voice_overlay_settings_hint">Opcional en ambas versiones. Abre el enfoque de voz en Chat y elige Superposición. Los permisos por sí solos nunca inician la escucha.</string>
<string name="voice_overlay_reset_position">Restablecer posición</string>
<string name="injected_context_gateway_unsupported">No se envía en el chat de Gateway. Esta conexión no admite contexto adicional por turno.</string>
</resources>
+10 -4
View File
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">質問 %1$d / %2$d</string>
<string name="clarify_batch_complete">すべての質問に回答しました</string>
<string name="clarify_batch_expired">このリクエストは終了しました</string>
<string name="clarify_batch_sending">回答を送信中…</string>
<string name="clarify_batch_answered">回答済みの質問(%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">エージェントインターフェース</string>
@@ -796,7 +801,7 @@
<string name="chat_settings_notify_when_finishes">バックグラウンドのチャット通知</string>
<string name="chat_settings_notify_when_finishes_desc">バックグラウンドで Hermes が入力を必要としたとき、または完了したときに通知します</string>
<string name="chat_settings_share_phone_status">電話のステータスをエージェントと共有する</string>
<string name="chat_settings_share_phone_status_desc">すべてのチャット ターンにアプリと電話に関する短いシステム メッセージを含めます</string>
<string name="chat_settings_share_phone_status_desc">API のみのチャットにアプリと端末のコンテキストを含めます。標準の Gateway チャットでは送信されません。</string>
<string name="chat_settings_bridge_permissions">Bridge + 権限</string>
<string name="chat_settings_bridge_permissions_desc">アクセシビリティ、画面キャプチャ、オーバーレイ、通知が許可されているかどうか</string>
<string name="chat_settings_foreground_app">フォアグラウンドアプリ</string>
@@ -805,7 +810,7 @@
<string name="chat_settings_battery_level_desc">現在のバッテリーのパーセント。デフォルトではオフです。</string>
<string name="chat_settings_safety_rails">安全レール</string>
<string name="chat_settings_safety_rails_desc">ブロックリストの数、破壊的な動詞の数、および自動無効化タイマー</string>
<string name="chat_settings_preview">プレビュー</string>
<string name="chat_settings_preview">API のみのチャットの例</string>
<string name="chat_settings_no_system_message">(システムメッセージは送信されません)</string>
<string name="chat_settings_parse_tool_annotations">解析ツールの注釈</string>
<string name="chat_settings_experimental">実験的</string>
@@ -3371,10 +3376,10 @@
<string name="injected_context_phone_status_title">電話のステータス</string>
<string name="injected_context_relay_not_set">Relayが設定されていません</string>
<string name="injected_context_relay_title">Relay</string>
<string name="injected_context_subtitle">次のメッセージで送信されるコンテキスト</string>
<string name="injected_context_subtitle">このチャットのプレビューです。サーバー側のコンテキストは別途設定されます。</string>
<string name="injected_context_title">挿入されたコンテキスト</string>
<string name="injected_context_turn_not_set">未設定</string>
<string name="injected_context_turn_title">回転検出モード</string>
<string name="injected_context_turn_title">ターンのコンテキスト</string>
<string name="model_picker_cd_clear">クリア</string>
<string name="model_picker_count">%d モデル</string>
<string name="power_feature_back_desc">戻る</string>
@@ -4599,4 +4604,5 @@
<string name="voice_overlay_setup_start">音声オーバーレイを開始</string>
<string name="voice_overlay_settings_hint">両方のビルドで任意に利用できます。チャットの音声フォーカスでオーバーレイを選択します。権限の付与だけで録音は始まりません。</string>
<string name="voice_overlay_reset_position">位置をリセット</string>
<string name="injected_context_gateway_unsupported">Gateway チャットでは送信されません。この接続はターンごとの追加コンテキストに対応していません。</string>
</resources>
+10 -4
View File
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">Вопрос %1$d из %2$d</string>
<string name="clarify_batch_complete">Все вопросы отвечены</string>
<string name="clarify_batch_expired">Этот запрос завершён</string>
<string name="clarify_batch_sending">Отправка ответа…</string>
<string name="clarify_batch_answered">Отвеченные вопросы (%1$d)</string>
<string name="app_name">Hermes-Relay</string>
<string name="app_title">Hermes-Relay</string>
<string name="agent_interface">интерфейс агента</string>
@@ -773,7 +778,7 @@
<string name="chat_settings_notify_when_finishes">Уведомления о фоновом чате</string>
<string name="chat_settings_notify_when_finishes_desc">Уведомлять, когда Гермес требует ввода или заканчивает работу, когда приложение находится в фоновом режиме</string>
<string name="chat_settings_share_phone_status">Поделиться статусом телефона с агентом</string>
<string name="chat_settings_share_phone_status_desc">Включите короткое системное сообщение о приложении и телефоне в каждом обороте чата</string>
<string name="chat_settings_share_phone_status_desc">Добавлять контекст приложения и телефона в чаты только через API. В стандартных чатах Gateway он не отправляется.</string>
<string name="chat_settings_bridge_permissions">Мост + разрешения</string>
<string name="chat_settings_bridge_permissions_desc">Разрешения на доступность, захват экрана, наложение, уведомления</string>
<string name="chat_settings_foreground_app">Приложение на переднем плане</string>
@@ -782,7 +787,7 @@
<string name="chat_settings_battery_level_desc">Текущий процент заряда батареи. По умолчанию выключено.</string>
<string name="chat_settings_safety_rails">Ограждения безопасности</string>
<string name="chat_settings_safety_rails_desc">Количество в черном списке, количество деструктивных глаголов и таймер автоматического отключения</string>
<string name="chat_settings_preview">Предпросмотр</string>
<string name="chat_settings_preview">Пример чата только через API</string>
<string name="chat_settings_no_system_message">(системное сообщение не будет отправлено)</string>
<string name="chat_settings_parse_tool_annotations">Анализ аннотаций инструментов</string>
<string name="chat_settings_experimental">Экспериментальные</string>
@@ -3234,10 +3239,10 @@
<string name="injected_context_phone_status_title">Статус телефона</string>
<string name="injected_context_relay_not_set">плагин Relay не настроен</string>
<string name="injected_context_relay_title">плагин Relay</string>
<string name="injected_context_subtitle">Контекст отправляется с следующим сообщением</string>
<string name="injected_context_subtitle">Предпросмотр для этого чата. Контекст на стороне сервера настраивается отдельно.</string>
<string name="injected_context_title">Внедренный контекст</string>
<string name="injected_context_turn_not_set">Не установлено</string>
<string name="injected_context_turn_title">Режим обнаружения Turn</string>
<string name="injected_context_turn_title">Контекст хода</string>
<string name="model_picker_cd_clear">Очистить</string>
<string name="model_picker_count">%d моделей</string>
<string name="power_feature_back_desc">Назад</string>
@@ -4343,4 +4348,5 @@
<string name="voice_overlay_setup_start">Начать голосовой оверлей</string>
<string name="voice_overlay_settings_hint">Необязательно в обеих сборках. Откройте голосовой фокус в чате и выберите оверлей. Одни разрешения никогда не включают прослушивание.</string>
<string name="voice_overlay_reset_position">Сбросить положение</string>
<string name="injected_context_gateway_unsupported">Не отправляется в чате Gateway. Это подключение не поддерживает дополнительный контекст для каждого хода.</string>
</resources>
+10 -4
View File
@@ -1,5 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="clarify_batch_progress">Question %1$d of %2$d</string>
<string name="clarify_batch_complete">All questions answered</string>
<string name="clarify_batch_expired">This request has ended</string>
<string name="clarify_batch_sending">Sending answer…</string>
<string name="clarify_batch_answered">Answered questions (%1$d)</string>
<string name="chat_debug_title" translatable="false">Session diagnostics</string>
<string name="chat_debug_open" translatable="false">Open session diagnostics</string>
<string name="chat_debug_close" translatable="false">Close session diagnostics</string>
@@ -882,7 +887,7 @@
<string name="chat_settings_notify_when_finishes">Background chat alerts</string>
<string name="chat_settings_notify_when_finishes_desc">Notify when Hermes needs input or finishes while the app is in the background</string>
<string name="chat_settings_share_phone_status">Share phone status with agent</string>
<string name="chat_settings_share_phone_status_desc">Include a short system message about the app and phone on every chat turn</string>
<string name="chat_settings_share_phone_status_desc">Include app and phone context in API-only chats. Not sent in standard Gateway chats.</string>
<string name="chat_settings_bridge_permissions">Bridge + permissions</string>
<string name="chat_settings_bridge_permissions_desc">Whether accessibility, screen capture, overlay, notifications are granted</string>
<string name="chat_settings_foreground_app">Foreground app</string>
@@ -891,7 +896,7 @@
<string name="chat_settings_battery_level_desc">Current battery percent. Off by default.</string>
<string name="chat_settings_safety_rails">Safety rails</string>
<string name="chat_settings_safety_rails_desc">Blocklist count, destructive-verb count, and auto-disable timer</string>
<string name="chat_settings_preview">Preview</string>
<string name="chat_settings_preview">Example for API-only chat</string>
<string name="chat_settings_no_system_message">(no system message will be sent)</string>
<string name="chat_settings_parse_tool_annotations">Parse tool annotations</string>
<string name="chat_settings_experimental">Experimental</string>
@@ -3764,10 +3769,10 @@
<string name="injected_context_phone_status_title">Phone status</string>
<string name="injected_context_relay_not_set">No relay configured</string>
<string name="injected_context_relay_title">Relay</string>
<string name="injected_context_subtitle">Context sent with next message</string>
<string name="injected_context_subtitle">Preview for this chat. Server-side context is configured separately.</string>
<string name="injected_context_title">Injected context</string>
<string name="injected_context_turn_not_set">Not set</string>
<string name="injected_context_turn_title">Turn detection mode</string>
<string name="injected_context_turn_title">Turn context</string>
<string name="model_picker_cd_clear">Clear</string>
<string name="model_picker_count">%d models</string>
<string name="power_feature_back_desc">Back</string>
@@ -4662,4 +4667,5 @@
<string name="voice_overlay_setup_start">Start voice overlay</string>
<string name="voice_overlay_settings_hint">Optional in both builds. Open Voice Focus in Chat and choose Overlay to start. Permissions alone never start listening.</string>
<string name="voice_overlay_reset_position">Reset position</string>
<string name="injected_context_gateway_unsupported">Not sent in Gateway chat. This connection does not support extra per-turn context.</string>
</resources>
@@ -56,6 +56,21 @@ class ChatTurnCheckpointStoreTest {
assertEquals(checkpoint, store.read())
}
@Test
fun partialClarifyBatch_roundTripsWithExactQuestionOwnership() = runTest {
val checkpoint = sampleCheckpoint().copy(pendingAsk = ChatTurnAskCheckpoint(
kind = "CLARIFY", requestId = "batch", text = "", timeoutSeconds = 0,
messageId = "ask-batch", cardKey = "batch", receivedAt = now,
questions = listOf(
com.hermesandroid.relay.network.upstream.GatewayClarifyQuestion("route/a", "Which route?", listOf("Canary")),
com.hermesandroid.relay.network.upstream.GatewayClarifyQuestion("environment:b", "Which environments?", listOf("Stage", "Production"), true),
), answers = mapOf("route/a" to "Canary"),
))
store.write(checkpoint)
assertEquals(checkpoint, store.read())
assertEquals(mapOf("route/a" to "Canary"), store.read()?.pendingAsk?.answers)
}
@Test
fun corruptJson_isDiscarded() = runTest {
dataStore.edit { preferences ->
@@ -74,6 +74,8 @@ class GatewayClientHarness(
@Volatile
var recoveryRunning = false
@Volatile
var recoveryClarify: JsonObject? = null
@Volatile
var recoveryAssistant = ""
@@ -586,6 +588,7 @@ class GatewayClientHarness(
private val autoRespondEnabled = autoRespond
private fun recoveryPayload(sessionId: String, requestedProfile: String? = null): JsonObject = buildJsonObject {
recoveryClarify?.let { put("pending_clarify", it) }
put("session_id", sessionId)
put("running", recoveryRunning)
put("status", if (recoveryRunning) "streaming" else "idle")
@@ -3165,6 +3168,75 @@ class GatewayChatClientTest {
// --- Ask responders ---
@Test
fun `batch clarify reconnect adopts server answered qids without resubmitting`() {
val r = Recorder()
client.sendTurn(null, "hi", null, r.callbacks) { r.preflightFailures += it }
val socket = harness.awaitServerSocket()
harness.awaitRpc("prompt.submit")
harness.recoveryRunning = true
harness.recoveryClarify = harness.json.parseToJsonElement("""{
"request_id":"batch","questions":[{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}],
"answers":{"q0":"accepted before disconnect"}}
""") as JsonObject
socket.close(1001, "fixture reconnect")
val replacement = harness.awaitServerSocket()
harness.awaitRpc("session.activate")
val deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(5)
while (r.interactions.isEmpty() && System.nanoTime() < deadline) Thread.sleep(10)
assertEquals(mapOf("q0" to "accepted before disconnect"), r.interactions.last().answers)
assertTrue(harness.rpcLog.none { it.first == "clarify.respond" })
assertTrue(runBlocking { client.respondClarify("batch", "second", "q1") }.isSuccess)
replacement.send(harness.eventFrame("message.complete", buildJsonObject { put("text", "done") }, "live-1"))
assertTrue(r.completeLatch.await(5, TimeUnit.SECONDS))
}
@Test
fun `batch clarify responds with exact question id and JSON array answer`() {
val r = Recorder()
client.sendTurn(null, "hi", null, r.callbacks) { r.preflightFailures += it }
val socket = harness.awaitServerSocket()
harness.awaitRpc("prompt.submit")
socket.send(harness.eventFrame("clarify.request", harness.json.parseToJsonElement("""
{"request_id":"batch","questions":[{"qid":"env:b","question":"Which environments?","choices":["Stage","Production"],"multi_select":true}]}
""") as JsonObject, "live-1"))
awaitCondition { r.interactions.isNotEmpty() }
val answer = "[\"Stage\",\"Production\"]"
assertTrue(runBlocking { client.respondClarify("batch", answer, "env:b") }.isSuccess)
val respond = harness.awaitRpc("clarify.respond")
assertEquals("batch", (respond["request_id"] as? JsonPrimitive)?.contentOrNull)
assertEquals("env:b", (respond["question_id"] as? JsonPrimitive)?.contentOrNull)
assertEquals(answer, (respond["answer"] as? JsonPrimitive)?.contentOrNull)
}
@Test
fun `batch answer waits for reconnect replay and refuses an already answered qid`() {
val r = Recorder()
client.sendTurn(null, "hi", null, r.callbacks) { r.preflightFailures += it }
val socket = harness.awaitServerSocket()
harness.awaitRpc("prompt.submit")
val request = harness.json.parseToJsonElement("""{"request_id":"batch","questions":[
{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}]}""") as JsonObject
socket.send(harness.eventFrame("clarify.request", request, "live-1"))
awaitCondition { r.interactions.isNotEmpty() }
harness.recoveryRunning = true
harness.suppressAckMethods += "session.activate"
socket.close(1001, "fixture lost acknowledgement")
harness.awaitServerSocket()
val activation = harness.awaitPendingAck()
assertEquals("session.activate", activation.method)
assertTrue(runBlocking { client.respondClarify("batch", "retry", "q0") }.isFailure)
assertTrue(harness.rpcLog.none { it.first == "clarify.respond" })
harness.releaseAck(activation, buildJsonObject {
put("session_id", "live-1")
put("running", true)
put("pending_clarify", JsonObject(request + ("answers" to buildJsonObject { put("q0", "accepted") })))
})
awaitCondition { r.interactions.last().answers["q0"] == "accepted" }
assertTrue(runBlocking { client.respondClarify("batch", "overwrite", "q0") }.isFailure)
assertTrue(harness.rpcLog.none { it.first == "clarify.respond" })
}
@Test
fun `clarify respond carries request id and answer`() {
val r = Recorder()
@@ -16,6 +16,103 @@ import org.junit.Test
*/
class GatewayEventMapperTest {
@Test
fun `acknowledgement before reclaim remains in the shared request snapshot`() {
val original = mapperWith(Recorder())
original.onEvent("clarify.request", obj("""{"request_id":"batch","questions":[
{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}]}"""))
val detachedSnapshot = requireNotNull(original.currentInteraction)
original.acknowledgeClarify("batch", "q0", "first", false)
val recovered = mapperWith(Recorder())
recovered.restoreInteraction(detachedSnapshot)
assertEquals(mapOf("q0" to "first"), recovered.currentInteraction?.answers)
recovered.onEvent("message.complete", obj("""{"text":"done"}"""))
recovered.acknowledgeClarify("batch", "q1", "second", false)
assertTrue(recovered.turnEnded)
}
@Test
fun `expired detached snapshot cannot become pending again`() {
val original = mapperWith(Recorder())
original.onEvent("clarify.request", obj("""{"request_id":"batch","questions":[{"qid":"q0","question":"First?"}]}"""))
val snapshot = requireNotNull(original.currentInteraction)
original.acknowledgeClarify("batch", "q0", "ignored", true)
val r = Recorder()
val recovered = mapperWith(r)
recovered.restoreInteraction(snapshot)
assertNull(recovered.currentInteraction)
assertTrue(r.interactions.isEmpty())
assertEquals("batch", r.interactionExpiries.single().requestId)
}
@Test
fun `forwarded acknowledgement cannot touch a newer request incarnation`() {
val mapper = mapperWith(Recorder())
val request = obj("""{"request_id":"batch","questions":[{"qid":"q0","question":"First?"}]}""")
mapper.onEvent("clarify.request", request)
val oldOwner = requireNotNull(mapper.currentInteraction).ownershipToken
mapper.onEvent("clarify.expire", obj("""{"request_id":"batch"}"""))
mapper.onEvent("clarify.request", request)
mapper.acknowledgeClarifyOwner("batch", "q0", "stale", true, oldOwner)
assertTrue(requireNotNull(mapper.currentInteraction).answers.isEmpty())
assertFalse(mapper.turnEnded)
}
@Test
fun `partial acknowledgement holds terminal and merges replay without resurrecting an answer`() {
val r = Recorder()
val mapper = mapperWith(r)
val request = obj("""{"request_id":"batch","questions":[
{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}]}""")
mapper.onEvent("clarify.request", request)
mapper.onEvent("message.complete", obj("""{"text":"done"}"""))
mapper.acknowledgeClarify("batch", "q0", "first", false)
assertFalse(mapper.turnEnded)
assertEquals(mapOf("q0" to "first"), mapper.currentInteraction?.answers)
mapper.onEvent("clarify.request", request)
assertEquals(mapOf("q0" to "first"), mapper.currentInteraction?.answers)
mapper.acknowledgeClarify("old-batch", "q1", "stale", true)
assertFalse(mapper.turnEnded)
mapper.acknowledgeClarify("batch", "q1", "second", false)
assertNull(mapper.currentInteraction)
assertTrue(mapper.turnEnded)
}
@Test
fun `batch expiry retires partial progress only for exact request`() {
val r = Recorder()
val mapper = mapperWith(r)
mapper.onEvent("clarify.request", obj("""{"request_id":"batch","questions":[
{"qid":"q0","question":"First?"},{"qid":"q1","question":"Second?"}],"answers":{"q0":"done"}}"""))
mapper.onEvent("clarify.expire", obj("""{"request_id":"other"}"""))
assertEquals("done", mapper.currentInteraction?.answers?.get("q0"))
mapper.onEvent("clarify.expire", obj("""{"request_id":"batch"}"""))
assertNull(mapper.currentInteraction)
}
@Test
fun `batch clarify preserves exact qids question order choices and replayed answers`() {
val ask = GatewayEventMapper.interactionRequest("clarify.request", obj("""
{"request_id":"batch","questions":[
{"qid":"choice/a","question":"Which deployment?","choices":["Canary","Immediate"],"multi_select":false},
{"qid":"env:b","question":"Which environments?","choices":["Stage","Production"],"multi_select":true}
],"answers":{"choice/a":"Canary","foreign":"ignored"}}
"""))!!
assertEquals(listOf("choice/a", "env:b"), ask.questions.map { it.qid })
assertEquals("Which deployment?", ask.questions[0].question)
assertEquals(listOf("Stage", "Production"), ask.questions[1].choices)
assertTrue(ask.questions[1].multiSelect)
assertEquals(mapOf("choice/a" to "Canary"), ask.answers)
}
@Test
fun `one entry normalized clarify keeps its qid`() {
val ask = GatewayEventMapper.interactionRequest("clarify.request", obj("""
{"request_id":"one","questions":[{"qid":"q0","question":"Which file?","choices":null}]}
"""))!!
assertEquals(listOf("q0"), ask.questions.map { it.qid })
}
private class Recorder {
val textDeltas = mutableListOf<String>()
val interimMessages = mutableListOf<Pair<String, Boolean>>()
@@ -0,0 +1,121 @@
package com.hermesandroid.relay.screenshots
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalDensity
import androidx.compose.ui.semantics.SemanticsActions
import androidx.compose.ui.test.hasScrollAction
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.onRoot
import androidx.compose.ui.test.performSemanticsAction
import androidx.compose.ui.test.performTouchInput
import androidx.compose.ui.test.swipeUp
import androidx.compose.ui.text.TextLayoutResult
import androidx.compose.ui.unit.Density
import androidx.compose.ui.unit.dp
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.github.takahirom.roborazzi.captureRoboImage
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.HermesCard
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import com.hermesandroid.relay.data.HermesCardClarifyQuestion
import com.hermesandroid.relay.data.HermesCardInput
import com.hermesandroid.relay.data.MessageDeliveryStatus
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.ui.components.MessageBubble
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import java.io.File
import java.text.SimpleDateFormat
import java.util.Date
import java.util.Locale
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
class ChatBubblePolishScreenshotTest {
@get:Rule val compose = createComposeRule()
@Test @Config(qualifiers = "w360dp-h800dp-xhdpi")
fun darkConversation() = capture("dark-conversation", "dark", 1f)
@Test @Config(qualifiers = "w360dp-h800dp-xhdpi")
fun lightConversation() = capture("light-conversation", "light", 1f)
@Test @Config(qualifiers = "w320dp-h568dp-xhdpi")
fun narrowLargeText() = capture("narrow-large-text", "dark", 1.5f)
@Test @Config(qualifiers = "w720dp-h360dp-xhdpi")
fun landscape() = capture("landscape", "dark", 1f)
@Test @Config(qualifiers = "w360dp-h800dp-xhdpi")
fun correctionAndTimeShareOneLine() {
compose.setContent {
HermesRelayTheme(themePreference = "dark") {
MessageBubble(correction(), animationEnabled = false)
}
}
val time = SimpleDateFormat("h:mm a", Locale.US).format(Date(TIMESTAMP))
val timeBounds = compose.onNodeWithText(time, useUnmergedTree = true).fetchSemanticsNode().boundsInRoot
val status = compose.onNodeWithText("Correction sent", useUnmergedTree = true)
val statusBounds = status.fetchSemanticsNode().boundsInRoot
assertTrue("Time and correction should share a footer row", kotlin.math.abs(timeBounds.center.y - statusBounds.center.y) <= 2f)
val layouts = mutableListOf<TextLayoutResult>()
status.performSemanticsAction(SemanticsActions.GetTextLayoutResult) { it(layouts) }
assertEquals(1, layouts.single().lineCount)
}
private fun capture(name: String, theme: String, scale: Float) {
val card = HermesCard(
type = HermesCard.BuiltInTypes.ASK_CLARIFY,
title = "Hermes needs clarification",
id = "clarify",
clarifyBatch = HermesCardClarifyBatch(listOf(
HermesCardClarifyQuestion("q0", "Which accent color?", HermesCardInput(HermesCardInput.Kinds.TEXT), "Orange"),
HermesCardClarifyQuestion("q1", "Which sections?", HermesCardInput(HermesCardInput.Kinds.CHOICE, multiSelect = true), "[\"Calendar\",\"Tasks\"]"),
)),
)
val messages = listOf(
ChatMessage(id = "ask", role = MessageRole.ASSISTANT, content = "", timestamp = TIMESTAMP, cards = listOf(card), clientOnly = true),
ChatMessage(id = "reply", role = MessageRole.ASSISTANT, content = "Got both responses in one batch:\n\n- Accent: **Orange**\n- Sections: **Calendar and Tasks**", timestamp = TIMESTAMP),
correction(),
)
compose.setContent {
val density = LocalDensity.current
CompositionLocalProvider(LocalDensity provides Density(density.density, scale)) {
HermesRelayTheme(themePreference = theme) {
LazyColumn(
Modifier.fillMaxSize().background(MaterialTheme.colorScheme.background).padding(12.dp),
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
items(messages.size, key = { messages[it].id }) { index ->
MessageBubble(messages[index], showAgentIdentity = false, isFirstInGroup = index == 0 || index == 2,
isLastInGroup = index != 0, animationEnabled = false)
}
}
}
}
}
val directory = File("build/ui-evidence/bubble-polish").apply { mkdirs() }
compose.onRoot().captureRoboImage(File(directory, "$name-top.png").path)
repeat(3) { compose.onNode(hasScrollAction()).performTouchInput { swipeUp() } }
compose.onRoot().captureRoboImage(File(directory, "$name-bottom.png").path)
}
private fun correction() = ChatMessage(id = "correction", role = MessageRole.USER, content = "Test", timestamp = TIMESTAMP,
deliveryStatus = MessageDeliveryStatus.STEERED)
companion object { private const val TIMESTAMP = 1_700_000_000_000L }
}
@@ -0,0 +1,105 @@
package com.hermesandroid.relay.screenshots
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.lazy.LazyColumn
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalDensity
import androidx.compose.ui.test.hasScrollAction
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onRoot
import androidx.compose.ui.test.performTouchInput
import androidx.compose.ui.test.swipeUp
import androidx.compose.ui.unit.Density
import androidx.compose.ui.unit.dp
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.github.takahirom.roborazzi.captureRoboImage
import com.hermesandroid.relay.data.ChatMessage
import com.hermesandroid.relay.data.HermesCard
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import com.hermesandroid.relay.data.HermesCardClarifyQuestion
import com.hermesandroid.relay.data.HermesCardInput
import com.hermesandroid.relay.data.MessageRole
import com.hermesandroid.relay.ui.components.MessageBubble
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import java.io.File
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
class ClarifyBatchScreenshotTest {
@get:Rule val compose = createComposeRule()
@Test @Config(qualifiers = "w320dp-h568dp-xhdpi")
fun compactDark() = capture("compact-dark")
@Test @Config(qualifiers = "w320dp-h568dp-xhdpi")
fun compactLightLargeText() = capture("compact-light-font-1_5", theme = "light", scale = 1.5f)
@Test @Config(qualifiers = "w720dp-h360dp-xhdpi")
fun landscape() = capture("landscape", width = 540)
@Test @Config(qualifiers = "w330dp-h720dp-xhdpi")
fun narrowFoldable() = capture("foldable-pane", scale = 1.5f)
@Test @Config(qualifiers = "w360dp-h720dp-xhdpi")
fun partialProgress() = capture("partial", answered = 1)
@Test @Config(qualifiers = "w360dp-h720dp-xhdpi")
fun sending() = capture("sending", answered = 1, submitting = true)
@Test @Config(qualifiers = "w360dp-h720dp-xhdpi")
fun completed() = capture("completed", answered = 2)
@Test @Config(qualifiers = "w320dp-h568dp-xhdpi")
fun expiredPartial() = capture("expired-partial", answered = 1, expired = true)
private fun capture(
name: String, theme: String = "dark", scale: Float = 1f, width: Int = 300,
answered: Int = 0, submitting: Boolean = false, expired: Boolean = false,
) {
val options = listOf(
"Keep both systems running while traffic moves in measured stages (Recommended)",
"Migrate everything immediately and accept a short maintenance window",
"Pause until every downstream consumer has been verified",
"Use a reversible canary rollout with automatic rollback thresholds",
)
val questions = listOf(
HermesCardClarifyQuestion("q0", "Which deployment approach should I use for the migration, given the existing clients and the rollback requirements?",
HermesCardInput(HermesCardInput.Kinds.CHOICE, options, allowFreeText = true),
answer = options[0].takeIf { answered > 0 }),
HermesCardClarifyQuestion("q1", "Which environments should receive this change?",
HermesCardInput(HermesCardInput.Kinds.CHOICE, listOf("Staging", "Production"), multiSelect = true, allowFreeText = true),
answer = "[\"Staging\",\"Production\"]".takeIf { answered > 1 }, submitting = submitting),
)
val card = HermesCard(type = HermesCard.BuiltInTypes.ASK_CLARIFY, title = "Hermes needs clarification",
id = "batch", clarifyBatch = HermesCardClarifyBatch(questions, expiresAtMillis = if (expired) 1L else null))
compose.setContent {
val density = LocalDensity.current
CompositionLocalProvider(LocalDensity provides Density(density.density, scale)) {
HermesRelayTheme(themePreference = theme) {
LazyColumn(Modifier.fillMaxSize().background(MaterialTheme.colorScheme.background).padding(10.dp)) {
item {
MessageBubble(
message = ChatMessage(id = "batch-message", role = MessageRole.ASSISTANT, content = "",
timestamp = 0L, cards = listOf(card), clientOnly = true),
maxBubbleWidth = width.dp, showTimestamps = false, animationEnabled = false,
)
}
}
}
}
}
val directory = File("build/ui-evidence/clarify-batch").apply { mkdirs() }
compose.onRoot().captureRoboImage(File(directory, "$name-top.png").path)
repeat(4) { compose.onNode(hasScrollAction()).performTouchInput { swipeUp() } }
compose.onRoot().captureRoboImage(File(directory, "$name-bottom.png").path)
}
}
@@ -0,0 +1,52 @@
package com.hermesandroid.relay.screenshots
import androidx.compose.ui.test.junit4.createComposeRule
import androidx.compose.ui.test.onAllNodesWithText
import androidx.compose.ui.test.assertCountEquals
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.onRoot
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.github.takahirom.roborazzi.captureRoboImage
import com.hermesandroid.relay.ui.components.InjectedContextSheet
import com.hermesandroid.relay.ui.theme.HermesRelayTheme
import com.hermesandroid.relay.viewmodel.ChatViewModel
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(sdk = [34], qualifiers = "w360dp-h720dp-xhdpi")
class InjectedContextSheetTest {
@get:Rule val compose = createComposeRule()
@Test
fun gatewayPreviewLabelsUnsupportedContextWithoutShowingPhonePreamble() {
showPreview("gateway")
compose.onAllNodesWithText(
"Not sent in Gateway chat. This connection does not support extra per-turn context.",
).assertCountEquals(2)
compose.onNodeWithText("Hermes-Relay Android app", substring = true).assertDoesNotExist()
compose.onNodeWithText("Server-side persona").assertExists()
compose.onRoot().captureRoboImage("build/ui-regression/injected-context-gateway.png")
}
@Test
fun apiOnlyPreviewShowsThePreparedMobileContext() {
showPreview("sessions")
compose.onNodeWithText("Hermes-Relay Android app", substring = true).assertExists()
compose.onNodeWithText("Not sent in Gateway chat", substring = true).assertDoesNotExist()
compose.onRoot().captureRoboImage("build/ui-regression/injected-context-api.png")
}
private fun showPreview(endpoint: String) {
val preview = ChatViewModel().apply { streamingEndpoint = endpoint }.previewInjectedContext()
compose.setContent {
HermesRelayTheme(appThemeId = "hermes-relay", themePreference = "dark") {
InjectedContextSheet(context = preview, onDismiss = {})
}
}
}
}
@@ -0,0 +1,120 @@
package com.hermesandroid.relay.ui.components
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.setValue
import androidx.compose.ui.test.assertIsNotEnabled
import androidx.compose.ui.test.assertIsSelected
import androidx.compose.ui.test.assertIsFocused
import androidx.compose.ui.test.junit4.StateRestorationTester
import androidx.compose.ui.test.performKeyInput
import androidx.compose.ui.test.performSemanticsAction
import androidx.compose.ui.test.pressKey
import androidx.compose.ui.input.key.Key
import androidx.compose.ui.input.InputMode
import androidx.compose.ui.input.InputModeManager
import androidx.compose.ui.platform.LocalInputModeManager
import androidx.compose.ui.semantics.SemanticsActions
import androidx.compose.ui.test.junit4.v2.createComposeRule
import androidx.compose.ui.test.onNodeWithContentDescription
import androidx.compose.ui.test.onNodeWithText
import androidx.compose.ui.test.performClick
import androidx.compose.ui.test.performImeAction
import androidx.compose.ui.test.performTextInput
import androidx.test.ext.junit.runners.AndroidJUnit4
import com.hermesandroid.relay.data.HermesCard
import com.hermesandroid.relay.data.HermesCardClarifyBatch
import com.hermesandroid.relay.data.HermesCardClarifyQuestion
import com.hermesandroid.relay.data.HermesCardInput
import org.junit.Assert.assertEquals
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.annotation.Config
import org.robolectric.annotation.GraphicsMode
@RunWith(AndroidJUnit4::class)
@GraphicsMode(GraphicsMode.Mode.NATIVE)
@Config(qualifiers = "w360dp-h720dp-xhdpi")
class ClarifyBatchInteractionTest {
@get:Rule val compose = createComposeRule()
@Test fun draftsAndSelectionSurviveSavedStateAndFocusFollowsQuestionOrder() {
val restoration = StateRestorationTester(compose)
lateinit var inputMode: InputModeManager
restoration.setContent {
inputMode = LocalInputModeManager.current
MaterialTheme {
HermesCardBubble(card().copy(clarifyBatch = HermesCardClarifyBatch(listOf(question))),
"batch", emptyList(), { _, _ -> }, { _, _ -> })
}
}
compose.onNodeWithText("Stage").performClick()
compose.onNodeWithContentDescription("Other (type your answer)…").performTextInput("Keep rollback")
restoration.emulateSavedInstanceStateRestore()
compose.onNodeWithText("Stage").assertIsSelected()
compose.onNodeWithText("Keep rollback").assertExists()
compose.runOnIdle { inputMode.requestInputMode(InputMode.Keyboard) }
compose.onNodeWithText("Stage").performSemanticsAction(SemanticsActions.RequestFocus) { it() }
compose.onNodeWithText("Stage").assertIsFocused()
compose.onNodeWithText("Stage").performKeyInput { pressKey(Key.Tab) }
compose.onNodeWithText("Production").assertIsFocused()
}
@Test fun multiSelectImeRetryKeepsChoicesAndDraftAndDisablesDuplicateActions() {
var submitting by mutableStateOf(false)
val answers = mutableListOf<Pair<String, String>>()
compose.setContent {
MaterialTheme {
HermesCardBubble(
card = card().copy(clarifyBatch = HermesCardClarifyBatch(listOf(question.copy(submitting = submitting)))),
cardKey = "batch", dispatches = emptyList(), onActionTap = { _, _ -> },
onInputSubmit = { key, value -> answers += key to value; submitting = true },
)
}
}
compose.onNodeWithText("Submit").assertIsNotEnabled()
compose.onNodeWithContentDescription("Other (type your answer)…").apply {
performTextInput(" ")
performImeAction()
}
compose.runOnIdle { assertEquals(0, answers.size) }
compose.onNodeWithText("Stage").performClick().assertIsSelected()
compose.onNodeWithContentDescription("Other (type your answer)…").apply {
performTextInput("custom")
performImeAction()
}
compose.onNodeWithText("Stage").assertIsNotEnabled()
compose.onNodeWithText("Submit").assertIsNotEnabled()
compose.runOnIdle {
assertEquals(listOf("qid-key" to "[\"Stage\",\"custom\"]"), answers)
submitting = false
}
compose.onNodeWithText("Stage").assertIsSelected()
compose.onNodeWithText("Submit").performClick()
compose.runOnIdle { assertEquals(answers[0], answers[1]) }
}
@Test fun confirmedQuestionAdvancesAndExpiredBatchRetainsAnswersWithoutInputs() {
var batch by mutableStateOf(HermesCardClarifyBatch(listOf(question,
question.copy(key = "second", question = "Anything else?", input = HermesCardInput(HermesCardInput.Kinds.TEXT, allowFreeText = true)))))
compose.setContent {
MaterialTheme {
HermesCardBubble(card().copy(clarifyBatch = batch), "batch", emptyList(), { _, _ -> }, { _, _ -> })
}
}
compose.runOnIdle { batch = batch.copy(questions = batch.questions.map { if (it.key == "qid-key") it.copy(answer = "[\"Stage\"]") else it }) }
compose.onNodeWithText("Question 2 of 2").assertExists()
compose.onNodeWithText("Anything else?").assertExists()
compose.onNodeWithText("Stage").assertDoesNotExist()
compose.runOnIdle { batch = batch.copy(expiresAtMillis = 1L) }
compose.onNodeWithText("This request has ended").assertExists()
compose.onNodeWithText("Stage").assertExists()
compose.onNodeWithContentDescription("Type an answer…").assertDoesNotExist()
}
private val question = HermesCardClarifyQuestion("qid-key", "Which environments?",
HermesCardInput(HermesCardInput.Kinds.CHOICE, listOf("Stage", "Production"), multiSelect = true, allowFreeText = true))
private fun card() = HermesCard(HermesCard.BuiltInTypes.ASK_CLARIFY, title = "Hermes needs clarification")
}
@@ -1736,6 +1736,24 @@ class ChatViewModelGatewayInboundTurnTest {
assertEquals(false, gatewayClient.sessionModelProvider()?.fast)
}
@Test
fun injectedContextPreviewMatchesBareGatewayPayload() {
viewModel.appContextSettings = com.hermesandroid.relay.util.AppContextSettings(
master = true, battery = true, currentApp = true,
)
val preview = viewModel.previewInjectedContext()
assertFalse(preview.perTurnContextSupported)
assertNull(preview.combinedSystemMessage)
viewModel.sendMessage("Keep this message unchanged")
gatewayHarness.awaitRpc("prompt.submit")
val submitted = gatewayHarness.rpcLog.last { it.first == "prompt.submit" }.second
assertEquals(JsonPrimitive("Keep this message unchanged"), submitted["text"])
assertFalse(submitted.containsKey("system_message"))
assertFalse(submitted.containsKey("surface"))
assertEquals(0, apiCompletionsRequestCount.get())
}
@Test
fun dashboardOnlyConnectionCanSendWithoutApiClient() {
viewModel.updateGatewayClient(null)
@@ -2729,6 +2747,121 @@ class ChatViewModelGatewayInboundTurnTest {
awaitCondition { viewModel.pendingAsk.value == null }
}
private fun presentBatchClarify(requestId: String = "batch") {
serverWs.send(gatewayHarness.eventFrame("clarify.request",
gatewayHarness.json.parseToJsonElement("""{"request_id":"$requestId","questions":[
{"qid":"route/a","question":"Which deployment?","choices":["Canary","Immediate"]},
{"qid":"environment:b","question":"Which environments?","choices":["Stage","Production"],"multi_select":true}
]}""") as kotlinx.serialization.json.JsonObject, "live-resumed"))
awaitCondition { viewModel.pendingAsk.value?.ask?.requestId == requestId }
}
@Test
fun batchClarifyInFlightAnswerSurvivesSessionRoundTripWithoutDuplicateRpc() {
val contextKey = AgentDisplay.profileContextKey("connection-a", null)
val store = MemoryCheckpointStore()
viewModel.setChatTurnCheckpointStore(store)
viewModel.switchProfileContext(contextKey, STORED_SESSION_ID)
viewModel.sendMessage("Ask before continuing")
gatewayHarness.awaitRpc("prompt.submit")
presentBatchClarify()
val pending = requireNotNull(viewModel.pendingAsk.value)
val key = com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "route/a")
gatewayHarness.suppressAckMethods += "clarify.respond"
viewModel.answerAsk(pending.messageId, key, "Canary")
shadowOf(Looper.getMainLooper()).idle()
val ack = gatewayHarness.awaitPendingAck()
viewModel.switchSession("other-session")
awaitCondition { handler.currentSessionId.value == "other-session" && store.checkpoint?.pendingAsk != null }
gatewayHarness.recoveryRunning = true
viewModel.switchSession(STORED_SESSION_ID)
awaitCondition { viewModel.pendingAsk.value != null && handler.currentSessionId.value == STORED_SESSION_ID }
val restored = requireNotNull(viewModel.pendingAsk.value)
assertEquals(pending.ownerId, restored.ownerId)
viewModel.answerAsk(restored.messageId, key, "Canary")
gatewayHarness.releaseAck(ack)
awaitCondition { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "Canary" }
assertEquals(1, gatewayHarness.rpcLog.count { it.first == "clarify.respond" })
gatewayHarness.suppressAckMethods -= "clarify.respond"
viewModel.answerAsk(restored.messageId,
com.hermesandroid.relay.data.clarifyQuestionCardKey(restored.cardKey, "environment:b"), "[\"Stage\"]")
awaitCondition { viewModel.pendingAsk.value == null }
serverWs.send(gatewayHarness.eventFrame("message.complete", buildJsonObject { put("text", "All answers received") }, "live-resumed"))
awaitCondition { !handler.isStreaming.value && !gatewayClient.hasActiveTurn() }
assertEquals(2, gatewayHarness.rpcLog.count { it.first == "clarify.respond" })
}
@Test
fun batchClarifyKeepsPartialProgressRejectsDuplicateAndNeverSendsChatAnswers() {
viewModel.sendMessage("Ask a batch")
gatewayHarness.awaitRpc("prompt.submit")
presentBatchClarify()
val pending = requireNotNull(viewModel.pendingAsk.value)
val key = com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "route/a")
gatewayHarness.suppressAckMethods += "clarify.respond"
viewModel.answerAsk(pending.messageId, key, "Canary")
viewModel.answerAsk(pending.messageId, key, "Canary")
shadowOf(Looper.getMainLooper()).idle()
val ack = gatewayHarness.awaitPendingAck()
assertEquals(1, gatewayHarness.rpcLog.count { it.first == "clarify.respond" })
gatewayHarness.releaseAck(ack)
awaitCondition { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "Canary" }
presentBatchClarify()
viewModel.answerAsk(pending.messageId, key, "Immediate")
assertEquals("Canary", viewModel.pendingAsk.value?.ask?.answers?.get("route/a"))
gatewayHarness.suppressAckMethods -= "clarify.respond"
val secondKey = com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "environment:b")
viewModel.answerAsk(pending.messageId, secondKey, "[\"Stage\",\"Production\"]")
awaitCondition { viewModel.pendingAsk.value == null }
val responses = gatewayHarness.rpcLog.filter { it.first == "clarify.respond" }.map { it.second }
assertEquals(listOf(JsonPrimitive("route/a"), JsonPrimitive("environment:b")), responses.map { it["question_id"] })
assertEquals(1, gatewayHarness.rpcLog.count { it.first == "prompt.submit" })
val card = handler.messages.value.single { it.id == pending.messageId }.cards.single()
assertEquals(listOf("Canary", "[\"Stage\",\"Production\"]"), card.clarifyBatch?.questions?.map { it.answer })
}
@Test
fun batchClarifyFailedRpcCanRetryAndWhitespaceCannotSubmit() {
viewModel.sendMessage("Ask a batch")
gatewayHarness.awaitRpc("prompt.submit")
presentBatchClarify()
val pending = requireNotNull(viewModel.pendingAsk.value)
val key = com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "route/a")
viewModel.answerAsk(pending.messageId, key, " ")
assertTrue(gatewayHarness.rpcLog.none { it.first == "clarify.respond" })
gatewayHarness.rpcErrors["clarify.respond"] = 5030 to "Try again"
viewModel.answerAsk(pending.messageId, key, "Canary")
awaitCondition { gatewayHarness.rpcLog.any { it.first == "clarify.respond" } &&
handler.messages.value.single { it.id == pending.messageId }.cards.single().clarifyBatch?.questions?.first()?.submitting == false }
assertTrue(requireNotNull(viewModel.pendingAsk.value).ask.answers.isEmpty())
gatewayHarness.rpcErrors.remove("clarify.respond")
viewModel.answerAsk(pending.messageId, key, " custom route ")
awaitCondition { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "custom route" }
assertEquals(2, gatewayHarness.rpcLog.count { it.first == "clarify.respond" })
}
@Test
fun staleExpiredBatchResponseCannotRetireReusedRequestId() {
viewModel.sendMessage("Ask a batch")
gatewayHarness.awaitRpc("prompt.submit")
presentBatchClarify()
val pending = requireNotNull(viewModel.pendingAsk.value)
gatewayHarness.suppressAckMethods += "clarify.respond"
viewModel.answerAsk(pending.messageId, com.hermesandroid.relay.data.clarifyQuestionCardKey(pending.cardKey, "route/a"), "Canary")
shadowOf(Looper.getMainLooper()).idle()
val ack = gatewayHarness.awaitPendingAck()
serverWs.send(gatewayHarness.eventFrame("clarify.expire", buildJsonObject { put("request_id", "batch") }, "live-resumed"))
awaitCondition { viewModel.pendingAsk.value == null }
presentBatchClarify()
val newOwner = requireNotNull(viewModel.pendingAsk.value).ownerId
gatewayHarness.releaseAck(ack, buildJsonObject { put("status", "expired") })
gatewayHarness.suppressAckMethods -= "clarify.respond"
val current = requireNotNull(viewModel.pendingAsk.value)
viewModel.answerAsk(current.messageId, com.hermesandroid.relay.data.clarifyQuestionCardKey(current.cardKey, "route/a"), "Immediate")
awaitCondition { viewModel.pendingAsk.value?.ask?.answers?.get("route/a") == "Immediate" }
assertEquals(newOwner, viewModel.pendingAsk.value?.ownerId)
}
@Test
fun authoritativeClarifyExpiryCollapsesCardAndRejectsLateAction() {
viewModel.sendMessage("Ask a question")
@@ -0,0 +1,63 @@
package com.hermesandroid.relay.viewmodel
import com.hermesandroid.relay.data.Profile
import com.hermesandroid.relay.util.AppContextSettings
import org.junit.Assert.assertEquals
import org.junit.Assert.assertFalse
import org.junit.Assert.assertNull
import org.junit.Assert.assertTrue
import org.junit.Test
import org.junit.runner.RunWith
import org.robolectric.RobolectricTestRunner
import org.robolectric.annotation.Config
@RunWith(RobolectricTestRunner::class)
@Config(sdk = [34])
class InjectedContextTest {
@Test
fun gatewayPreviewExcludesPhoneContextEvenWhenSharingIsEnabled() {
val viewModel = ChatViewModel().apply {
streamingEndpoint = "gateway"
appContextSettings = AppContextSettings(master = true, battery = true, currentApp = true)
setSelectedProfileProvider { Profile(name = "writer", model = "test-model", systemMessage = "Profile persona") }
}
val preview = viewModel.previewInjectedContext()
assertFalse(preview.perTurnContextSupported)
assertTrue(preview.personaOwnedServerSide)
assertNull(preview.personaPrompt)
assertNull(preview.appContext)
assertNull(preview.interfaceContext)
assertNull(preview.mediaCapability)
assertNull(preview.combinedSystemMessage)
}
@Test
fun apiOnlyTransportsRetainOptedInPhoneContextAndMasterOffRemovesIt() {
for (endpoint in listOf("sessions", "runs", "completions")) {
val viewModel = ChatViewModel().apply { streamingEndpoint = endpoint }
val enabled = viewModel.previewInjectedContext()
assertTrue(enabled.perTurnContextSupported)
assertFalse(enabled.personaOwnedServerSide)
assertTrue(enabled.appContext!!.contains("Hermes-Relay Android app"))
assertEquals(enabled.appContext, enabled.combinedSystemMessage)
viewModel.appContextSettings = AppContextSettings(master = false, battery = true, currentApp = true)
val disabled = viewModel.previewInjectedContext()
assertNull(disabled.appContext)
assertNull(disabled.combinedSystemMessage)
}
}
@Test
fun changingTransportRebuildsPreviewWithoutChangingSharingPreference() {
val viewModel = ChatViewModel().apply { streamingEndpoint = "sessions" }
val apiPreview = viewModel.previewInjectedContext()
viewModel.streamingEndpoint = "gateway"
assertNull(viewModel.previewInjectedContext().appContext)
assertTrue(viewModel.appContextSettings.master)
viewModel.streamingEndpoint = "sessions"
assertEquals(apiPreview.appContext, viewModel.previewInjectedContext().appContext)
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

+3
View File
@@ -68,6 +68,9 @@ the upstream contract identifiers it depends on.
|---|---|
| `initial_history_bind` | Durable, profile-scoped history is already available when the client resumes and first binds its rendered transcript |
| `ordinary_turn` | Normal message start, deltas, completion, and persisted history |
| `clarify_legacy` | Top-level single question and unkeyed `clarify.respond` |
| `clarify_normalized_single` | One normalized `questions[]` entry still requires its exact `qid` |
| `clarify_batch` | Independent qid responses, partial acknowledgement, and answered-question replay on reconnect |
| `session_initialization_failure` | Exact-session initialization error arrives before a lazy create acknowledgement; Android must fail the pending send without waiting for the readiness deadline |
| `subagent_child_preview` | Child activity continues after the parent terminal, followed by child completion and a separate completion wake; preview ownership remains on the same profile/session |
| `ownership_rejection` | A submit acknowledged before the defense-in-depth ownership check emits the canonical terminal refusal; no user/model row is persisted and clients must not enter history recovery |
+6 -6
View File
@@ -13,7 +13,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "f174b8e5ff26511e81d31fc53249b03886a2067fd1aabac4bb2d5b027d5513ca",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -48,7 +48,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "f174b8e5ff26511e81d31fc53249b03886a2067fd1aabac4bb2d5b027d5513ca",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -72,7 +72,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "f174b8e5ff26511e81d31fc53249b03886a2067fd1aabac4bb2d5b027d5513ca",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -96,7 +96,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "f174b8e5ff26511e81d31fc53249b03886a2067fd1aabac4bb2d5b027d5513ca",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -120,7 +120,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "f174b8e5ff26511e81d31fc53249b03886a2067fd1aabac4bb2d5b027d5513ca",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
@@ -135,7 +135,7 @@
"verification": "ai-translated",
"review_refs": [],
"source_sha256": {
"main": "f174b8e5ff26511e81d31fc53249b03886a2067fd1aabac4bb2d5b027d5513ca",
"main": "8588c42385e783a2ddf888b515ca6b97a5df44a30d3619c048d3d03d91dd5278",
"sideload": "4abff4f1069091ec2de735c3037a7ec7d77699cb4321e8511a622437bceaf7c2"
},
"surfaces": {
+27
View File
@@ -58,6 +58,33 @@ standard upstream media/file delivery, and concise-response expectations. Once
that contract is available in the supported Hermes baseline, adopt it and add
Gateway conformance coverage proving the exact prompt bytes and session source.
Issues [#556](https://github.com/Codename-11/hermes-relay/issues/556) and
[#557](https://github.com/Codename-11/hermes-relay/issues/557) share this contract
gap. The Android audit fix labels unsupported context; automatic Android
identification and Gateway phone-status delivery still require upstream work.
Rechecked against upstream `5dea46d13deec9549bdc2ea703ae9201d733c28d`:
- [`methods_prompt.py`](https://github.com/NousResearch/hermes-agent/blob/5dea46d13deec9549bdc2ea703ae9201d733c28d/tui_gateway/methods_prompt.py)
accepts `surface` only for `hud` and `voice-live`; it has no general
per-turn system-context parameter. Neither surface means Android.
- [`session_notifications.py`](https://github.com/NousResearch/hermes-agent/blob/5dea46d13deec9549bdc2ea703ae9201d733c28d/tui_gateway/session_notifications.py)
adds those built-in surface notes to model input without changing the
persisted user row. This is the existing upstream seam to extend, with
explicit client capability negotiation, bounded opted-in context, and
turn-owned snapshots through queueing, retries, and client switches.
- [`server.py`](https://github.com/NousResearch/hermes-agent/blob/5dea46d13deec9549bdc2ea703ae9201d733c28d/tui_gateway/server.py)
accepts a caller-supplied session `source`, but
[`prompt_builder.py`](https://github.com/NousResearch/hermes-agent/blob/5dea46d13deec9549bdc2ea703ae9201d733c28d/agent/prompt_builder.py)
has no Android platform hint. A session-origin label is not verified
current-turn sender identity. Do not invent a platform or use a client hint
as authorization for phone tools.
- The API server's `POST /api/sessions` source field belongs to the explicit
API-only surface. Android standard chat creates/resumes sessions through
Gateway RPC; that REST field does not add per-turn context to `prompt.submit`.
Keep unknown/older hosts supported without sending private parameters,
rewriting the user transcript, overriding persona, or requiring Relay.
---
## Scope sensitive-media prompt guidance to capable clients
+3 -1
View File
@@ -597,6 +597,8 @@ Bottom navigation bar with 4 tabs:
```
### Chat Tab
- **Context preview** — the Injected context sheet describes the bound chat transport, not a delivery receipt. Standard Gateway chat has no general per-turn system-context slot: phone status, the Android client preamble, media hints, and interface-context blocks are omitted. Phone status and turn context show an explicit unsupported note. API-only chats retain the opted-in context and sharing controls; the Settings example is labeled API-only. Gateway persona and optional Relay-reported configuration remain separately server-owned. No context is prepended to user text or written into the personality slot to bypass this boundary.
- **Top bar and Profile Shelf (three-layer agent model).** Layout from left to right:
1. **Connection chip** — tap to open `ConnectionSwitcherSheet` (all paired servers + health indicator). Auto-hidden when you only have one Connection. See `docs/decisions.md` §19.
2. **Agent avatar/name region** — tap to expand or collapse the Profile Shelf immediately below the app bar. With only one visible effective identity, the shelf stays hidden and the same tap opens Agent Passport.
@@ -638,7 +640,7 @@ Bottom navigation bar with 4 tabs:
- **Agent Passport — profile inspection/configuration** — upstream Hermes profiles are selected from the Profile Shelf or the Passport's shared full switcher. Passport retains identity customization, model/personality/reasoning/safety configuration, inspection, and session analytics; it is not a second profile-picker implementation. See `docs/decisions.md` §21 and ADR 48.
- **Agent sheet — Personality section** — personalities fetched from `GET /api/config` (`config.agent.personalities`). Shows server default (from `config.display.personality`) + all configured. Active personality name shown on assistant chat bubbles.
- **Agent sheet — Approval controls** — gateway contract v3 exposes the profile-persisted `approvals.mode` policy (`manual` / `smart` / `off`) separately from YOLO. The launch/default profile gets the three-way control; multiplexed non-launch profiles reconcile `session.info.approval_mode` read-only until upstream config RPCs honor profile scope. The existing YOLO switch remains an explicit per-session override and never silently writes profile configuration. Older gateways keep chat and YOLO available while the profile control explains that an upstream update is required.
- **Interactive clarify cards** — ordinary upstream choices retain one-tap submission; `multi_select:true` choices toggle independently and require explicit submission as one JSON-array answer. Open text remains available for an Other answer. Android never invents a clarify deadline when upstream omits timeout metadata: the correlated `clarify.expire` event or an expired response retires the card authoritatively.
- **Interactive clarify cards** — ordinary upstream choices retain one-tap submission; `multi_select:true` choices toggle independently and require explicit submission as one JSON-array answer. Open text remains available for an Other answer. Upstream `questions[]` batches use one progressive card, retain exact qids, and answer each question through `clarify.respond` with `question_id`. Confirmed answers survive checkpoint recovery and merge with upstream replay without resubmission. Legacy top-level questions keep their original wire shape. Android never invents a clarify deadline when upstream omits timeout metadata: the correlated `clarify.expire` event or an expired response retires the card authoritatively.
- **Streaming dots** — animated pulsing 3-dot indicator replaces static "streaming..." text
- Displays: streaming delta text; quiet thinking/reasoning disclosures that open while live and collapse when settled; consecutive routine tool activity summarized as one live ticker or settled disclosure; standalone lifecycle surfaces for approvals, failures, generated media, file edits, and delegated work; per-message token counts + cost
+40
View File
@@ -63,6 +63,46 @@ Verified upstream source snapshot:
| `hermes relay doctor` | Hermes-Relay plugin CLI | No for diagnostics | Operator/agent diagnostics | Reports vanilla upstream Hermes route reachability (including `/v1/toolsets`), dashboard Nous/topology state, sanitized gateway event-loop heartbeat state, plugin layout, Relay loopback state, and legacy bootstrap presence. |
| `hermes_relay_bootstrap` routes | Legacy compatibility monkeypatch | No, but non-upstream | Fallback only | Installed via `.pth` by legacy installer. Injects only compatibility-only gaps: session search, memory, legacy skill detail/toggle, config, available-models, slash middleware. Sessions CRUD and skill/toolset lists are native upstream and retired from the bootstrap. Retained session-database work is offloaded (`AsyncSessionDB` when available, `asyncio.to_thread` fallback), and memory mutations reset newer upstream's request-local consolidation-failure budget. |
## Dashboard Relay WebSocket guard compatibility
The optional Relay ingress reuses the host's already-loaded
`_ws_request_is_allowed` and `_ws_auth_ok` pair. Current Hermes owns both in
`hermes_cli.web_server_chat`; older hosts expose them on `hermes_cli.web_server`
(or its legacy module alias). The current owner takes precedence. If it is
loaded but either guard is missing or non-callable, admission fails closed;
guards are never combined across modules or retried through a weaker fallback.
No Dashboard modules are imported just to discover guards.
Plugin enablement still uses the facade's `_get_dashboard_plugins` and the
host's enabled/disabled sets. A disabled plugin or rejected Host/Origin/IP is
checked before ticket consumption. Guard exceptions deny the upgrade, tickets
remain single-use, and Dashboard admission never replaces Relay's own
pairing/session authentication.
Run the hermetic transport regressions with:
```bash
python -m pytest plugin/dashboard/test_plugin_api.py plugin/tests/test_dashboard_ingress.py -q
```
An opt-in conformance lane exercises real current-upstream imports and ticket
consumption through FastAPI and an ephemeral loopback Relay, using a temporary
Hermes home rather than an installed service. Use a Python environment with the
Dashboard dependencies, provide the upstream checkout on `PYTHONPATH`, and run
this lane in its own process:
```bash
test_home=$(mktemp -d)
env -i PATH="$PATH" HOME="$test_home" HERMES_HOME="$test_home" \
HERMES_RELAY_TEST_UPSTREAM=1 PYTHONPATH="$PWD:/path/to/hermes-agent" \
python -m pytest plugin/dashboard/test_upstream_ws_guards.py -q
```
The explicit opt-in fails on missing upstream dependencies instead of silently
skipping. It covers query/subprotocol tickets, replay/expiry, policy and plugin
rejection before forwarding, loopback-only token auth, and inner Relay auth.
It does not certify a live reverse proxy or restart any installed service.
## Client capability gate (build flavor)
Route ownership above is a *server-side* contract. The Android client adds a
+10 -1
View File
@@ -364,8 +364,17 @@ def _dashboard_ws_guards() -> tuple[Any, Any] | None:
The current upstream plugin contract mounts routers but does not inject an
auth dependency for WebSockets. Using the host's already-loaded helpers
keeps ticket consumption and Host/Origin/IP policy identical to `/api/ws`.
If upstream moves either helper, this ingress fails closed.
Current hosts own both guards in web_server_chat; older hosts expose them
on web_server. Prefer the current owner and never combine guards from
different modules. A loaded current owner with an incomplete contract must
fail closed rather than fall back to potentially stale facade helpers.
"""
chat = sys.modules.get("hermes_cli.web_server_chat")
if chat is not None:
allowed = getattr(chat, "_ws_request_is_allowed", None)
authed = getattr(chat, "_ws_auth_ok", None)
return (allowed, authed) if callable(allowed) and callable(authed) else None
candidates = [sys.modules.get("hermes_cli.web_server")]
candidates.extend(
module for name, module in tuple(sys.modules.items())
+86
View File
@@ -293,6 +293,68 @@ class TransportIngressTests(PluginApiTestCase):
self.assertIn("size limit", response.json()["detail"])
class DashboardWebSocketGuardDiscoveryTests(unittest.TestCase):
def test_loaded_chat_guards_admit_without_legacy_facade_exports(self) -> None:
allowed = Mock(return_value=True)
authed = Mock(return_value=True)
modules = {
"hermes_cli.web_server": SimpleNamespace(),
"hermes_cli.web_server_chat": SimpleNamespace(
_ws_request_is_allowed=allowed, _ws_auth_ok=authed,
),
}
with patch.object(plugin_api.sys, "modules", modules):
self.assertEqual(plugin_api._dashboard_ws_guards(), (allowed, authed))
def test_legacy_hosts_retain_complete_guard_pair(self) -> None:
for name in ("hermes_cli.web_server", "legacy.web_server"):
with self.subTest(name=name):
guards = (Mock(), Mock())
module = SimpleNamespace(
_ws_request_is_allowed=guards[0], _ws_auth_ok=guards[1],
)
with patch.object(plugin_api.sys, "modules", {name: module}):
self.assertEqual(plugin_api._dashboard_ws_guards(), guards)
def test_current_owner_takes_precedence_over_stale_facade(self) -> None:
guards = (Mock(return_value=False), Mock(return_value=False))
modules = {
"hermes_cli.web_server_chat": SimpleNamespace(
_ws_request_is_allowed=guards[0], _ws_auth_ok=guards[1],
),
"hermes_cli.web_server": SimpleNamespace(
_ws_request_is_allowed=Mock(return_value=True),
_ws_auth_ok=Mock(return_value=True),
),
}
with patch.object(plugin_api.sys, "modules", modules):
self.assertEqual(plugin_api._dashboard_ws_guards(), guards)
def test_missing_or_partial_contracts_fail_closed_without_mixing(self) -> None:
complete = SimpleNamespace(_ws_request_is_allowed=Mock(), _ws_auth_ok=Mock())
for modules in (
{},
{"hermes_cli.web_server_chat": SimpleNamespace()},
{
"hermes_cli.web_server_chat": SimpleNamespace(_ws_auth_ok=Mock()),
"hermes_cli.web_server": complete,
},
{
"hermes_cli.web_server_chat": SimpleNamespace(
_ws_auth_ok=True, _ws_request_is_allowed=Mock(),
),
"hermes_cli.web_server": complete,
},
{
"hermes_cli.web_server": SimpleNamespace(_ws_auth_ok=Mock()),
"legacy.web_server": SimpleNamespace(_ws_request_is_allowed=Mock()),
},
):
with self.subTest(modules=tuple(modules)):
with patch.object(plugin_api.sys, "modules", modules):
self.assertIsNone(plugin_api._dashboard_ws_guards())
class TransportWebSocketAdmissionTests(unittest.IsolatedAsyncioTestCase):
class _Socket:
def __init__(self) -> None:
@@ -330,6 +392,30 @@ class TransportWebSocketAdmissionTests(unittest.IsolatedAsyncioTestCase):
request_allowed.assert_called_once_with(socket)
auth_ok.assert_called_once_with(socket)
async def test_request_denial_does_not_consume_ticket(self) -> None:
socket = self._Socket()
auth = Mock(return_value=True)
with patch.object(plugin_api, "_dashboard_plugin_is_enabled", return_value=True), patch.object(
plugin_api, "_dashboard_ws_guards", return_value=(lambda _ws: False, auth)
):
self.assertIsNone(await plugin_api._admit_transport_websocket(socket)) # type: ignore[arg-type]
self.assertEqual(socket.closed[0], 1008) # type: ignore[index]
auth.assert_not_called()
async def test_guard_exception_fails_closed_without_auth_fallback(self) -> None:
for failing_guard in (0, 1):
with self.subTest(failing_guard=failing_guard):
socket = self._Socket()
guards = [Mock(return_value=True), Mock(return_value=True)]
guards[failing_guard].side_effect = RuntimeError("guard unavailable")
with patch.object(plugin_api, "_dashboard_plugin_is_enabled", return_value=True), patch.object(
plugin_api, "_dashboard_ws_guards", return_value=tuple(guards)
):
self.assertIsNone(await plugin_api._admit_transport_websocket(socket)) # type: ignore[arg-type]
self.assertEqual(socket.closed[0], 1011) # type: ignore[index]
if failing_guard == 0:
guards[1].assert_not_called()
async def test_failed_dashboard_ticket_is_policy_close(self) -> None:
socket = self._Socket()
with patch.object(
+134
View File
@@ -0,0 +1,134 @@
"""Opt-in conformance against real Hermes Dashboard imports, never a live service.
Set HERMES_RELAY_TEST_UPSTREAM=1 and put a current hermes-agent checkout on
PYTHONPATH. Missing upstream dependencies then fail rather than silently skip.
"""
from __future__ import annotations
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from typing import Any
from fastapi import FastAPI
from fastapi.testclient import TestClient
from starlette.websockets import WebSocketDisconnect
from plugin.dashboard import plugin_api
from plugin.dashboard import test_plugin_api as transport_tests
@unittest.skipUnless(os.environ.get("HERMES_RELAY_TEST_UPSTREAM") == "1", "opt-in upstream conformance")
class UpstreamWebSocketGuardTests(unittest.TestCase):
path = "/api/plugins/hermes-relay/transport/ws"
def setUp(self) -> None:
home = Path(self.enterContext(tempfile.TemporaryDirectory()))
self.enterContext(patch.dict(os.environ, {
"HERMES_HOME": str(home),
"HERMES_RELAY_DASHBOARD_PROXY_SECRET": "integration-test-secret",
}))
self.enterContext(patch.object(Path, "home", return_value=home))
self.config_path = home / "config.yaml"
self.config_path.write_text("plugins:\n enabled: [hermes-relay]\n", encoding="utf-8")
from hermes_cli import web_server
from hermes_cli.dashboard_auth import ws_tickets
self.host = web_server
self.tickets = ws_tickets
self.tickets._reset_for_tests()
self.addCleanup(self.tickets._reset_for_tests)
app = FastAPI()
app.state.auth_required = True
app.state.bound_host = "127.0.0.1"
app.state.trusted_public_hosts = frozenset()
self.enterContext(patch.object(self.host, "app", app))
# Exercise the facade discovery function and real enabled/disabled config
# readers, but do not discover or execute the operator's installed plugins.
self.enterContext(patch.object(self.host, "_dashboard_plugins_cache", [{
"name": "hermes-relay", "source": "user", "_dir": str(home),
}]))
self.assertTrue(plugin_api._dashboard_plugin_is_enabled())
app.include_router(plugin_api.router, prefix="/api/plugins/hermes-relay")
self.client = self.enterContext(TestClient(
app, base_url="http://127.0.0.1", headers={"host": "127.0.0.1"},
))
self.relay = transport_tests._EphemeralRelay()
self.relay.start()
self.addCleanup(self.relay.stop)
self.enterContext(patch.object(plugin_api, "RELAY_PORT", self.relay.port))
def _ticket(self) -> str:
return self.tickets.mint_ticket(user_id="test-user", provider="test-provider")
def _assert_denied(self, path: str, **kwargs: Any) -> None:
# No connection to the loopback Relay is permitted on a failed guard.
with patch.object(plugin_api.aiohttp, "ClientSession", side_effect=AssertionError("Relay reached")):
with self.assertRaises(WebSocketDisconnect) as denied:
with self.client.websocket_connect(path, **kwargs):
pass
self.assertEqual(denied.exception.code, 1008)
def test_real_ticket_pairing_reconnect_subprotocol_and_replay(self) -> None:
self.assertTrue(self.relay.server.pairing.register_code("ABC123"))
ticket = self._ticket()
with self.client.websocket_connect(f"{self.path}?ticket={ticket}") as socket:
socket.send_json(transport_tests.TransportWebSocketIntegrationTests._auth_payload(pairing_code="ABC123"))
paired = socket.receive_json()
self.assertEqual(paired["type"], "auth.ok")
session_token = paired["payload"]["session_token"]
self._assert_denied(f"{self.path}?ticket={ticket}")
protocols = ["hermes-gateway-v1", f"hermes-gateway-ticket.{self._ticket()}"]
with self.client.websocket_connect(self.path, subprotocols=protocols) as socket:
self.assertEqual(socket.accepted_subprotocol, "hermes-gateway-v1")
socket.send_json(transport_tests.TransportWebSocketIntegrationTests._auth_payload(session_token=session_token))
self.assertEqual(socket.receive_json()["type"], "auth.ok")
self._assert_denied(self.path, subprotocols=protocols)
# Dashboard admission must not replace the inner Relay session boundary.
with self.client.websocket_connect(f"{self.path}?ticket={self._ticket()}") as socket:
socket.send_json(transport_tests.TransportWebSocketIntegrationTests._auth_payload(session_token="invalid"))
self.assertEqual(socket.receive_json()["type"], "auth.fail")
def test_real_ungated_token_keeps_loopback_peer_policy(self) -> None:
self.host.app.state.auth_required = False
path = f"{self.path}?token={self.host._SESSION_TOKEN}"
with self.client.websocket_connect(path) as socket:
socket.send_json(transport_tests.TransportWebSocketIntegrationTests._auth_payload(session_token="invalid"))
self.assertEqual(socket.receive_json()["type"], "auth.fail")
with TestClient(
self.host.app, headers={"host": "127.0.0.1"}, client=("203.0.113.8", 12345),
) as remote_client, patch.object(
plugin_api.aiohttp, "ClientSession", side_effect=AssertionError("Relay reached"),
):
with self.assertRaises(WebSocketDisconnect) as denied:
with remote_client.websocket_connect(path):
pass
self.assertEqual(denied.exception.code, 1008)
def test_real_policy_and_plugin_gate_reject_before_ticket_consumption(self) -> None:
for headers in ({"host": "untrusted.example"}, {"origin": "https://untrusted.example"}):
with self.subTest(headers=headers):
ticket = self._ticket()
self._assert_denied(f"{self.path}?ticket={ticket}", headers=headers)
self.assertEqual(self.tickets.consume_ticket(ticket)["user_id"], "test-user")
self._assert_denied(self.path)
self._assert_denied(f"{self.path}?ticket=invalid")
with patch.object(self.tickets.time, "time", return_value=0):
expired = self._ticket()
self._assert_denied(f"{self.path}?ticket={expired}")
for config in (
"plugins:\n enabled: []\n",
"plugins:\n enabled: [hermes-relay]\n disabled: [hermes-relay]\n",
):
self.config_path.write_text(config, encoding="utf-8")
self.assertFalse(plugin_api._dashboard_plugin_is_enabled())
ticket = self._ticket()
self._assert_denied(f"{self.path}?ticket={ticket}")
self.assertEqual(self.tickets.consume_ticket(ticket)["user_id"], "test-user")
+3
View File
@@ -21,6 +21,9 @@ import sys
REPO_ROOT = pathlib.Path(__file__).resolve().parents[1]
FOCUSED_TESTS = (
"com.hermesandroid.relay.viewmodel.InjectedContextTest",
"com.hermesandroid.relay.screenshots.InjectedContextSheetTest",
"com.hermesandroid.relay.viewmodel.ChatViewModelGatewayInboundTurnTest.injectedContextPreviewMatchesBareGatewayPayload",
"com.hermesandroid.relay.voice.VoiceOverlayLifecycleTest",
"com.hermesandroid.relay.voice.VoiceOverlayForegroundServiceTest",
"com.hermesandroid.relay.voice.VoiceOverlayPresentationTest",
@@ -36,7 +36,9 @@ SESSION_EXCLUSIVE_SUBMIT = "gateway.session_exclusive_submit"
SUBAGENT_CHILD_WATCH = "gateway.subagent_child_watch"
SESSION_INITIALIZATION = "gateway.session_initialization"
API_BOUNDARY = "api.fallback_boundary"
CLARIFY = "gateway.clarify"
ALL_CONTRACTS = (
CLARIFY,
GATEWAY_TERMINAL,
GATEWAY_SETTLED_INFO,
SESSION_ACTIVATE,
@@ -541,6 +543,27 @@ def load_requirements(manifest: Path | None) -> tuple[str, ...]:
return tuple(contract for contract in ALL_CONTRACTS if contract in requested)
def _check_clarify(server: SourceFile) -> CheckResult:
bridge = server.function("_clarify_block")
respond = server.function("_respond")
replay = server.function("_pending_clarify_request_payload")
block = server.function("_block")
required = (
{"questions", "qid", "question", "choices", "multi_select"} <= _string_constants(bridge)
and {"question_id", "request_id", "answers", "remaining", "expired"} <= _string_constants(respond)
and {"answers", "clarify.request"} <= _string_constants(replay)
and {"answers", "timed_out"} <= _string_constants(block)
)
return CheckResult(
CLARIFY, required,
tuple(server.evidence(node, label) for node, label in (
(bridge, "legacy and qid batch wire"), (respond, "per-question response"),
(replay, "answered-qid replay"), (block, "partial timeout"),
)),
None if required else "Clarify wire, response, replay, or partial-timeout contract changed",
)
def audit_sources(root: Path, requirements: Iterable[str]) -> list[CheckResult]:
server = SourceFile(root, SERVER)
methods = SourceFile(root, SESSION_METHODS)
@@ -558,6 +581,7 @@ def audit_sources(root: Path, requirements: Iterable[str]) -> list[CheckResult]:
raise ValueError("fork marker(s) found in upstream source: " + ", ".join(fork_hits))
checks = {
CLARIFY: lambda: _check_clarify(server),
GATEWAY_TERMINAL: lambda: _check_gateway_terminal(
SourceFile(root, "tui_gateway/prompt_turn.py")
if (root / "tui_gateway/prompt_turn.py").is_file() else server
@@ -17,6 +17,19 @@ SPEC.loader.exec_module(module)
SERVER_SOURCE = '''
def _clarify_block(sid, q, c, multi_select=False, questions=None):
return {"questions": [{"qid": "q0", "question": q, "choices": c, "multi_select": multi_select}]}
def _respond(rid, params, key):
return {"question_id": params.get("question_id"), "request_id": params.get("request_id"),
"answers": {}, "remaining": [], "status": "expired"}
def _pending_clarify_request_payload(sid):
return {"answers": {}, "event": "clarify.request"}
def _block(event, sid, payload):
return {"answers": {}, "timed_out": True}
def _session_info(agent, session=None):
return {"running": bool((session or {}).get("running"))}
@@ -164,6 +177,13 @@ async def _handle_runs(request):
class GatewayScenarioConformanceTest(unittest.TestCase):
def test_clarify_conformance_requires_question_ownership_and_replay(self) -> None:
results = module.audit_sources(self.root, [module.CLARIFY])
self.assertTrue(results[0].passed)
source = self.root / module.SERVER
source.write_text(SERVER_SOURCE.replace('"remaining"', '"other"'), encoding="utf-8")
self.assertFalse(module.audit_sources(self.root, [module.CLARIFY])[0].passed)
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.root = Path(self.temp.name)
+2 -1
View File
@@ -47,7 +47,7 @@ distribution, and trust installation are deliberately outside this fixture.
- `POST /api/auth/ws-ticket` mints a fresh, single-use 30-second ticket.
- `GET /api/ws?ticket=...` upgrades to WebSocket and sends `gateway.ready`.
- JSON-RPC methods: `session.create`, `session.resume`, `session.activate`,
`session.active_list`, `prompt.submit`, and `session.interrupt` when the
`session.active_list`, `prompt.submit`, `clarify.respond`, and `session.interrupt` when the
selected scenario enables them.
- `GET /api/sessions/{stored-id}/messages` returns persisted, paginated history
and accepts the upstream `profile`, `limit`, `offset`, and `order` query shape.
@@ -66,6 +66,7 @@ ordered `steps` list using these operations:
| Operation | Purpose |
|---|---|
| `event` | Send a Gateway event with `exact`, `foreign`, or `unscoped` identity. |
| `clarify` | Emit the supplied Clarify `payload` and wait for its legacy answer or every batch qid; activation replays confirmed answers. |
| `persist` | Append authoritative Dashboard history rows. |
| `sleep` | Create a bounded deterministic ordering window (maximum 5 seconds). |
| `set_running` | Change the authoritative session running state. |
@@ -18,6 +18,58 @@ from vanilla_gateway.evidence import EvidenceLog # noqa: E402
class FixtureTestCase(unittest.IsolatedAsyncioTestCase):
async def test_clarify_batch_requires_each_qid_and_replays_partial_progress(self) -> None:
fixture, base_url = await self.start("clarify_batch")
ws, _ = await self.connect(base_url)
await self.rpc(ws, 1, "prompt.submit")
frames = await self.frames_until(ws, lambda f: f.get("params", {}).get("type") == "clarify.request")
request = frames[-1]["params"]["payload"]
rid = request["request_id"]
await self.rpc(ws, 2, "clarify.respond", {"request_id": rid, "question_id": "foreign", "answer": "x"})
frames = await self.frames_until(ws, lambda f: f.get("id") == 2)
self.assertEqual(4002, frames[-1]["error"]["code"])
await self.rpc(ws, 3, "clarify.respond", {"request_id": rid, "question_id": "route/a", "answer": "Canary"})
frames = await self.frames_until(ws, lambda f: f.get("id") == 3)
self.assertEqual(["environment:b"], frames[-1]["result"]["remaining"])
self.assertTrue(fixture.running)
await ws.close()
ws, _ = await self.connect(base_url)
await self.rpc(ws, 4, "session.activate", {"session_id": fixture.scenario.live_session_id})
frames = await self.frames_until(ws, lambda f: f.get("id") == 4)
self.assertEqual({"route/a": "Canary"}, frames[-1]["result"]["pending_clarify"]["answers"])
await self.rpc(ws, 5, "clarify.respond", {
"request_id": rid, "question_id": "environment:b", "answer": '["Stage","Production"]',
})
frames = await self.frames_until(ws, lambda f: f.get("id") == 5)
self.assertEqual([], frames[-1]["result"]["remaining"])
self.assertEqual('["Stage","Production"]', fixture._clarify_answers["environment:b"])
frames = await self.frames_until(ws, lambda f: f.get("params", {}).get("type") == "message.complete")
self.assertEqual("message.complete", frames[-1]["params"]["type"])
async def test_clarify_legacy_keeps_unkeyed_response(self) -> None:
_, base_url = await self.start("clarify_legacy")
ws, _ = await self.connect(base_url)
await self.rpc(ws, 1, "prompt.submit")
frames = await self.frames_until(ws, lambda f: f.get("params", {}).get("type") == "clarify.request")
payload = frames[-1]["params"]["payload"]
self.assertNotIn("questions", payload)
await self.rpc(ws, 2, "clarify.respond", {"request_id": payload["request_id"], "answer": "Canary"})
frames = await self.frames_until(ws, lambda f: f.get("id") == 2)
self.assertEqual("ok", frames[-1]["result"]["status"])
async def test_one_normalized_question_is_still_qid_owned(self) -> None:
_, base_url = await self.start("clarify_normalized_single")
ws, _ = await self.connect(base_url)
await self.rpc(ws, 1, "prompt.submit")
frames = await self.frames_until(ws, lambda f: f.get("params", {}).get("type") == "clarify.request")
payload = frames[-1]["params"]["payload"]
self.assertEqual(1, len(payload["questions"]))
await self.rpc(ws, 2, "clarify.respond", {
"request_id": payload["request_id"], "question_id": payload["questions"][0]["qid"], "answer": "Canary",
})
frames = await self.frames_until(ws, lambda f: f.get("id") == 2)
self.assertEqual([], frames[-1]["result"]["remaining"])
async def asyncSetUp(self) -> None:
self.session = ClientSession()
self.runner: web.AppRunner | None = None
@@ -14,7 +14,7 @@ class ScenarioError(ValueError):
"""Raised when a scenario does not satisfy the fixture schema."""
_STEP_OPS = {"event", "persist", "sleep", "close", "set_running"}
_STEP_OPS = {"event", "persist", "sleep", "close", "set_running", "clarify"}
_LIVE_STATUSES = {"starting", "working", "waiting", "idle"}
_SAFE_NAME = re.compile(r"[A-Za-z0-9_.-]{1,120}")
@@ -68,6 +68,16 @@ class Scenario:
raise ScenarioError("event scope must be exact, foreign, or unscoped")
if step["op"] == "persist" and not isinstance(step.get("messages"), list):
raise ScenarioError("persist step requires a messages list")
if step["op"] == "clarify":
payload = step.get("payload")
if not isinstance(payload, dict) or not isinstance(payload.get("request_id"), str):
raise ScenarioError("clarify step requires a request_id payload")
questions = payload.get("questions", [])
if not isinstance(questions, list) or any(
not isinstance(q, dict) or not isinstance(q.get("qid"), str) or not q["qid"]
for q in questions
):
raise ScenarioError("clarify questions require exact qids")
if step["op"] == "set_running" and not isinstance(step.get("value"), bool):
raise ScenarioError("set_running step requires a boolean value")
if step["op"] == "sleep":
@@ -0,0 +1,52 @@
{
"name": "clarify_batch",
"live_session_id": "fixture-live-clarify",
"stored_session_id": "fixture-stored-clarify",
"profile": "default",
"contract_requirements": [
"gateway.clarify"
],
"turns": [
{
"steps": [
{
"op": "event",
"type": "message.start"
},
{
"op": "clarify",
"payload": {
"request_id": "clarify-batch",
"questions": [
{
"qid": "route/a",
"question": "Which deployment?",
"choices": [
"Canary",
"Immediate"
],
"multi_select": false
},
{
"qid": "environment:b",
"question": "Which environments?",
"choices": [
"Stage",
"Production"
],
"multi_select": true
}
]
}
},
{
"op": "event",
"type": "message.complete",
"payload": {
"text": "Clarification complete."
}
}
]
}
]
}
@@ -0,0 +1,38 @@
{
"name": "clarify_legacy",
"live_session_id": "fixture-live-clarify",
"stored_session_id": "fixture-stored-clarify",
"profile": "default",
"contract_requirements": [
"gateway.clarify"
],
"turns": [
{
"steps": [
{
"op": "event",
"type": "message.start"
},
{
"op": "clarify",
"payload": {
"request_id": "clarify-legacy",
"question": "Which deployment?",
"choices": [
"Canary",
"Immediate"
],
"multi_select": false
}
},
{
"op": "event",
"type": "message.complete",
"payload": {
"text": "Clarification complete."
}
}
]
}
]
}
@@ -0,0 +1,43 @@
{
"name": "clarify_normalized_single",
"live_session_id": "fixture-live-clarify",
"stored_session_id": "fixture-stored-clarify",
"profile": "default",
"contract_requirements": [
"gateway.clarify"
],
"turns": [
{
"steps": [
{
"op": "event",
"type": "message.start"
},
{
"op": "clarify",
"payload": {
"request_id": "clarify-one",
"questions": [
{
"qid": "route/a",
"question": "Which deployment?",
"choices": [
"Canary",
"Immediate"
],
"multi_select": false
}
]
}
},
{
"op": "event",
"type": "message.complete",
"payload": {
"text": "Clarification complete."
}
}
]
}
]
}
@@ -55,6 +55,10 @@ class GatewayFixture:
self._connection_sequence = 0
self._tasks: set[asyncio.Task[None]] = set()
self._sockets: set[web.WebSocketResponse] = set()
self._clarify: dict[str, Any] | None = None
self._clarify_answers: dict[str, str] = {}
self._clarify_done = asyncio.Event()
self._clarify_owner: tuple[web.WebSocketResponse, int] | None = None
@property
def running(self) -> bool:
@@ -135,16 +139,40 @@ class GatewayFixture:
await self._rpc_error(socket, request_id, 4040, "Stored session not found")
return
result = self._session_snapshot(include_stored=True)
if self._clarify is not None:
self._clarify_owner = socket, connection
elif method == "session.activate":
requested = params.get("session_id")
if requested != self.scenario.live_session_id:
await self._rpc_error(socket, request_id, 4041, "Live session not found")
return
result = self._session_snapshot(include_stored=True)
if self._clarify is not None:
self._clarify_owner = socket, connection
elif method == "prompt.submit":
await self._rpc_result(socket, request_id, {"ok": True})
await self._submit(socket, connection)
return
elif method == "clarify.respond":
pending = self._clarify
if pending is None or params.get("request_id") != pending["request_id"]:
result = {"status": "expired"}
else:
qids = [q["qid"] for q in pending.get("questions", [])]
qid = params.get("question_id")
if qids and qid:
if qid not in qids:
await self._rpc_error(socket, request_id, 4002, "unknown question_id")
return
self._clarify_answers[qid] = params.get("answer", "")
remaining = [q for q in qids if q not in self._clarify_answers]
result = {"status": "ok", "remaining": remaining}
if not remaining:
self._clarify_done.set()
else:
# Upstream's no-qid batch response cancels the whole request.
self._clarify_done.set()
result = {"status": "ok"}
elif method == "session.interrupt":
was_active = self._turn_active
tasks = tuple(self._tasks)
@@ -179,6 +207,9 @@ class GatewayFixture:
}
if include_stored:
snapshot["stored_session_id"] = self.scenario.stored_session_id
if self._clarify is not None:
snapshot["pending_clarify"] = dict(self._clarify, answers=dict(self._clarify_answers))
snapshot["info"]["pending_clarify"] = snapshot["pending_clarify"]
if self._running:
snapshot["inflight"] = {"user": "fixture turn", "assistant": "", "streaming": True}
return snapshot
@@ -218,6 +249,18 @@ class GatewayFixture:
operation = step["op"]
if operation == "sleep":
await asyncio.sleep(step["milliseconds"] / 1_000)
elif operation == "clarify":
self._clarify = dict(step["payload"])
self._clarify_answers = {}
self._clarify_owner = socket, connection
self._clarify_done.clear()
await self._send_event(
socket, connection, "clarify.request", self._clarify,
self.scenario.live_session_id,
)
await self._clarify_done.wait()
socket, connection = self._clarify_owner
self._clarify = None
elif operation == "set_running":
self._running = bool(step["value"])
self.evidence.add(
@@ -245,6 +288,8 @@ class GatewayFixture:
)
self.evidence.add("fault", connection=connection, outcome="socket_gap")
finally:
self._clarify = None
self._clarify_owner = None
self._turn_active = False
if self._running:
self._running = False
+1 -1
View File
@@ -121,7 +121,7 @@ Google Play builds do not include AccessibilityService-backed screen reading or
| Chat empty state | Logo + suggestion chips |
| Animated streaming dots | Pulsing 3-dot indicator during streaming |
| Haptic feedback | On send, copy, stream complete, error |
| App context prompt | Toggleable system message for mobile context |
| App context prompt | Opt-in phone details and mobile context for API-only chats; Gateway limitations shown in the context preview |
## Security
+23 -9
View File
@@ -133,6 +133,13 @@ Sensitive prompts (sudo, secrets) are masked and hold-to-confirm. See
[Markdown Rendering → Rich Cards](/features/markdown#rich-cards) for the full
visual vocabulary.
When Hermes asks several questions together, the card shows one question at a
time and tracks your progress. Choose an option or type an **Other** answer;
questions that allow several choices have a **Submit** button. Each confirmed
answer stays recorded as you continue, including after a reconnect. If sending
fails, your selections remain available to retry. When a request ends, the card
keeps confirmed answers visible and stops accepting new ones.
## Context meter
A thin strip under the chat header tracks how full the conversation's context
@@ -248,15 +255,22 @@ Each assistant message shows token usage below the timestamp:
## App context prompt
When enabled (**Settings → Chat → App context prompt**, on by default),
Hermes-Relay tells the agent it's talking to a phone so replies stay
mobile-friendly and concise, and can attach optional bridge/permission and
safety-rail summaries. On the standard (API-server) connection this rides an
invisible system message. The Gateway connection carries no app-context preamble
— its protocol has no hidden per-turn slot, and adding one would leave the text
in your saved chat history — so there the agent reads phone state on demand via
the `android_phone_status` tool. Privacy-sensitive fields (foreground app,
battery) default off and are only added when you opt in.
**Settings → Chat → Share phone status with agent** controls an extra system
message in API-only chats. It identifies Hermes-Relay Android to the agent and
can include optional phone status. Foreground app and battery sharing default
off and are included only when you opt in. Turning the master switch off also
removes the Android preamble.
Standard Gateway chats do not send this block: upstream has no general
per-turn context slot. The **Injected context** sheet marks phone status and
turn context as unsupported on Gateway, and Settings labels its sample as an
API-only example. The sheet is a preview, not proof of delivery or a complete
view of the agent prompt; persona and Relay configuration are server-owned.
The optional Relay `android_phone_status` tool can provide phone state when
enabled for the selected profile. Its presence does not identify who sent the
current message. Automatic Android identification on Gateway still needs an
upstream client-surface contract; pairing alone does not provide it.
## Persistent connection
+1 -1
View File
@@ -89,7 +89,7 @@ Available in **Settings > Chat**.
|---------|---------|-------------|
| Show reasoning | `true` | Display thinking/reasoning blocks above responses |
| Show token usage | `true` | Display input/output token counts and estimated cost |
| App context prompt | `true` | Send system message telling agent user is on mobile |
| Share phone status with agent | `true` | Include mobile context in API-only chats; not sent in standard Gateway chats |
| Tool call display | `Detailed` | How tool calls appear: Off, Compact, or Detailed |
| Personality | Server default | Active personality from `config.agent.personalities` via `GET /api/config` |