Three open TODOs from the v0.4 hand-off, plus an install.sh ergonomic
improvement that came up while planning on-device verification.
C4 — voice contact-name → phone resolution
- RealVoiceBridgeIntentHandler now resolves the spoken contact name to
a real phone number BEFORE dispatching /send_sms, instead of passing
the raw name (which the phone-side regex check rejected, surfacing
as a user-visible error every time).
- New private helper resolveContactPhone() calls
HermesAccessibilityService.instance.actionExecutor.searchContacts(
contactName, limit=5) directly. Same code path as the /search_contacts
bridge route but invoked locally — no response-correlation plumbing
needed because both the voice handler and the accessibility service
are in the same process.
- searchContacts returns phones as a comma-joined string ("+1555..., +1555...")
per the C2 wire shape; the resolver splits on `,`, trims whitespace,
and takes the first non-blank entry.
- Wrapped in withContext(Dispatchers.IO) — ContactsContract queries
hit the on-device content provider and can block.
- If the resolver returns null (service unbound, contacts permission
missing, no matching contact, or matched contact has no phone), the
intent surfaces as a friendly spoken response ("I couldn't find a
contact called Sam.") instead of dispatching a broken envelope.
- The destructive-verb confirmation modal still fires on the
BridgeCommandHandler side, so the resolver is purely additive — it
doesn't bypass any safety gate.
H5 — voice app-name → package resolution
- Same shape as C4 but for /open_app. New resolveAppPackage() helper
calls service.packageManager.queryIntentActivities(ACTION_MAIN +
CATEGORY_LAUNCHER) and fuzzy-matches the spoken app name against
the launchable-app inventory.
- Three-tier match (case-insensitive, first hit wins):
1. Exact label match — "spotify" → "Spotify"
2. Prefix match — "chro" → "Chrome Beta"
3. Substring match — "google" → "Google Maps"
Order matters: tier 1 + 2 prevent accidental substring hits like
"messages" matching "Google Messages" instead of the literal
Messages app.
- Requires the <queries><intent action=MAIN category=LAUNCHER/></queries>
manifest declaration that landed in the previous commit (7755851) —
without it Android 11+ silently returns a near-empty candidate list.
- Wrapped in withContext(Dispatchers.IO) — PackageManager queries
can be slow on devices with many apps.
- Same null/error surfacing pattern as C4. BridgeCommandHandler still
blocklist-checks the resolved target package before launching, so
voice-utterance intents to open blocked banking apps stay blocked.
buildSmsEnvelope() and buildOpenAppEnvelope() now take 2 args (the
intent + the resolved value). Both call sites in tryHandle updated.
install.sh --branch flag (+ HERMES_RELAY_INSTALL_URL on the shim)
- Adds a CLI flag to install.sh that overrides HERMES_RELAY_BRANCH.
Flag wins over env var wins over the default ("main"). Same
precedence pattern as HERMES_RELAY_HOME / HERMES_VENV_PY.
- The hermes-relay-update shim now reads HERMES_RELAY_INSTALL_URL to
override the install.sh source URL. Bootstrap caveat documented in
the shim header: switching to a feature branch BEFORE that branch
is merged to main needs the env-var override because the shim
normally curls install.sh from main, which won't have the
--branch flag yet.
- After the bootstrap install completes the host has the new
install.sh on disk + the shim handles all subsequent updates,
including switching back to main via `hermes-relay-update`
(no flag needed, defaults to main).
- Documented the flag and the bootstrap escape hatch in the shim
doc-comment + the install.sh header.
plugin/tests/test_android_read_screen.py
- 11 stdlib-unittest cases mirroring the test_android_search_contacts
pattern (no pytest, no responses, runs as
`python -m unittest plugin.tests.test_android_read_screen`).
- Coverage: happy path with default include_bounds=False, explicit
include_bounds=True/False, empty node list, service-not-connected
503 passthrough, ConnectionError network failure, requests.Timeout,
schema registration sanity, and handler dispatch from the
_HANDLERS dict with both default and explicit args.
- All 11 tests pass locally (`python -m unittest plugin.tests.test_android_read_screen`).
- Caught a documentation drift along the way: android_read_screen
in the integration branch sends `?include_bounds=` (wave 1/2/3
rename) but main still sends `?bounds=`. The test asserts the
current branch shape; the rename will land in main as part of the
v0.4 merge.
Out of v0.4 scope, NOT touched
- The original C4 doc-comment proposed a response-correlation rewrite
of ChannelMultiplexer to wire bridge.command → bridge.response
request_id matching. Local resolution sidesteps that requirement
entirely. The correlation pattern is still useful for other
cross-channel flows (e.g. a future agent-driven UI test framework)
and is tracked separately rather than deferred under the C4 label.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
168 lines
6.9 KiB
Python
168 lines
6.9 KiB
Python
"""
|
|
Unit tests for ``plugin.tools.android_tool.android_read_screen``.
|
|
|
|
Stdlib ``unittest`` + ``unittest.mock`` only — no ``pytest`` or
|
|
``responses``. Run via::
|
|
|
|
python -m unittest plugin.tests.test_android_read_screen
|
|
|
|
Why stdlib unittest: the existing ``plugin/tests/conftest.py`` imports
|
|
``responses`` which is not in every venv (notably the hermes-host venv).
|
|
``python -m unittest`` bypasses pytest's conftest discovery so the test
|
|
runs cleanly regardless of whether ``responses`` is installed.
|
|
|
|
Coverage:
|
|
* happy path — bridge returns a serialized ScreenContent payload
|
|
* include_bounds defaults to false and gets URL-encoded as such
|
|
* include_bounds=True flips the query string parameter
|
|
* empty screen — bridge returns an empty nodes list
|
|
* service-not-connected — bridge returns 503 error body
|
|
* network failure — _get raises, tool surfaces as JSON error
|
|
* schema registration sanity (params, defaults, handler dispatch)
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import sys
|
|
import unittest
|
|
from pathlib import Path
|
|
from unittest import mock
|
|
|
|
REPO_ROOT = Path(__file__).resolve().parents[2]
|
|
if str(REPO_ROOT) not in sys.path:
|
|
sys.path.insert(0, str(REPO_ROOT))
|
|
|
|
from plugin.tools import android_tool # noqa: E402
|
|
|
|
|
|
class TestAndroidReadScreenHappyPath(unittest.TestCase):
|
|
def test_returns_screen_content(self) -> None:
|
|
fake = {
|
|
"rootBounds": {"left": 0, "top": 0, "right": 1080, "bottom": 2400},
|
|
"nodes": [
|
|
{
|
|
"nodeId": "n1",
|
|
"text": "Settings",
|
|
"className": "android.widget.TextView",
|
|
"clickable": True,
|
|
},
|
|
{
|
|
"nodeId": "n2",
|
|
"text": "Network & internet",
|
|
"className": "android.widget.TextView",
|
|
"clickable": True,
|
|
},
|
|
],
|
|
"truncated": False,
|
|
}
|
|
with mock.patch.object(android_tool, "_get", return_value=fake) as m:
|
|
result = json.loads(android_tool.android_read_screen())
|
|
|
|
self.assertEqual(len(result["nodes"]), 2)
|
|
self.assertEqual(result["nodes"][0]["nodeId"], "n1")
|
|
self.assertFalse(result["truncated"])
|
|
|
|
# Default include_bounds=False should encode as ?bounds=false
|
|
m.assert_called_once()
|
|
called_path = m.call_args.args[0]
|
|
self.assertEqual(called_path, "/screen?include_bounds=false")
|
|
|
|
def test_include_bounds_true_passes_query(self) -> None:
|
|
fake = {"rootBounds": {}, "nodes": [], "truncated": False}
|
|
with mock.patch.object(android_tool, "_get", return_value=fake) as m:
|
|
android_tool.android_read_screen(include_bounds=True)
|
|
called_path = m.call_args.args[0]
|
|
self.assertEqual(called_path, "/screen?include_bounds=true")
|
|
|
|
def test_include_bounds_false_explicit_passes_query(self) -> None:
|
|
# Explicit False should still produce ?bounds=false (not omitted).
|
|
fake = {"rootBounds": {}, "nodes": [], "truncated": False}
|
|
with mock.patch.object(android_tool, "_get", return_value=fake) as m:
|
|
android_tool.android_read_screen(include_bounds=False)
|
|
called_path = m.call_args.args[0]
|
|
self.assertEqual(called_path, "/screen?include_bounds=false")
|
|
|
|
|
|
class TestAndroidReadScreenEmpty(unittest.TestCase):
|
|
def test_empty_node_list(self) -> None:
|
|
fake = {"rootBounds": {}, "nodes": [], "truncated": False}
|
|
with mock.patch.object(android_tool, "_get", return_value=fake):
|
|
result = json.loads(android_tool.android_read_screen())
|
|
self.assertEqual(result["nodes"], [])
|
|
self.assertFalse(result["truncated"])
|
|
|
|
|
|
class TestAndroidReadScreenErrorPaths(unittest.TestCase):
|
|
def test_service_not_connected_passthrough(self) -> None:
|
|
# When BridgeCommandHandler rejects with 503, the relay returns the
|
|
# error body verbatim. android_read_screen propagates the dict.
|
|
fake = {
|
|
"error": (
|
|
"Hermes AccessibilityService not connected — enable it in "
|
|
"Android Settings"
|
|
)
|
|
}
|
|
with mock.patch.object(android_tool, "_get", return_value=fake):
|
|
result = json.loads(android_tool.android_read_screen())
|
|
self.assertIn("error", result)
|
|
self.assertIn("AccessibilityService", result["error"])
|
|
|
|
def test_network_error_returns_error_json(self) -> None:
|
|
# _get raises (relay down, DNS failure, etc.) — tool catches and
|
|
# returns a JSON-serialized error envelope rather than crashing.
|
|
with mock.patch.object(
|
|
android_tool, "_get", side_effect=ConnectionError("relay unreachable")
|
|
):
|
|
result = json.loads(android_tool.android_read_screen())
|
|
self.assertIn("error", result)
|
|
self.assertIn("relay unreachable", result["error"])
|
|
|
|
def test_timeout_error(self) -> None:
|
|
# Distinct exception type to verify the catch-all behavior.
|
|
import requests
|
|
with mock.patch.object(
|
|
android_tool, "_get", side_effect=requests.Timeout("read timed out")
|
|
):
|
|
result = json.loads(android_tool.android_read_screen())
|
|
self.assertIn("error", result)
|
|
|
|
|
|
class TestAndroidReadScreenSchema(unittest.TestCase):
|
|
def test_registered(self) -> None:
|
|
self.assertIn("android_read_screen", android_tool._SCHEMAS)
|
|
self.assertIn("android_read_screen", android_tool._HANDLERS)
|
|
|
|
def test_schema_params(self) -> None:
|
|
schema = android_tool._SCHEMAS["android_read_screen"]
|
|
self.assertEqual(schema["name"], "android_read_screen")
|
|
self.assertIn("accessibility tree", schema["description"].lower())
|
|
props = schema["parameters"]["properties"]
|
|
self.assertIn("include_bounds", props)
|
|
self.assertEqual(props["include_bounds"]["type"], "boolean")
|
|
self.assertFalse(props["include_bounds"]["default"])
|
|
# include_bounds is optional — should NOT be in required.
|
|
self.assertEqual(schema["parameters"]["required"], [])
|
|
|
|
def test_handler_dispatch_with_default_args(self) -> None:
|
|
fake = {"rootBounds": {}, "nodes": [], "truncated": False}
|
|
with mock.patch.object(android_tool, "_get", return_value=fake):
|
|
out = android_tool._HANDLERS["android_read_screen"]({})
|
|
result = json.loads(out)
|
|
self.assertEqual(result["nodes"], [])
|
|
|
|
def test_handler_dispatch_with_include_bounds(self) -> None:
|
|
fake = {"rootBounds": {}, "nodes": [], "truncated": False}
|
|
with mock.patch.object(android_tool, "_get", return_value=fake) as m:
|
|
out = android_tool._HANDLERS["android_read_screen"](
|
|
{"include_bounds": True}
|
|
)
|
|
called_path = m.call_args.args[0]
|
|
self.assertEqual(called_path, "/screen?include_bounds=true")
|
|
result = json.loads(out)
|
|
self.assertEqual(result["nodes"], [])
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|