Is the IOS Bring-Your-Own Elevenreader broken or I'm setting it up wrong?

[SkiesOfTheKuth]

I've been trying to configure it for a few hours on and off and it isn't working.

It isn't calling any tools, or executing any task, and the only successful tool call 'messageClaudeCode' came back with this error:

"Client tool with name messageClaudeCode is not defined on client"

The voice works only in responding; the voice icon changes but the STT still remains active even when I leave the session and enter my home screen.

I'm not sure if it's a work-in-progress or I'm doing it wrong.

[SkiesOfTheKuth]

I got it working. I'm not sure if it's right but it's somewhat working.

processPermissionRequest

{
"type": "client",
"name": "processPermissionRequest",
"description": "Allows or denies the active Happy coding session permission request.",
"expects_response": true,
"response_timeout_secs": 10,
"parameters": [
{
"id": "requestId",
"type": "string",
"value_type": "llm_prompt",
"description": "The exact pending permission request ID from the Happy session context. Do not use the session ID here.",
"dynamic_variable": "",
"constant_value": "",
"enum": null,
"is_system_provided": false,
"required": true
},
{
"id": "decision",
"type": "string",
"value_type": "llm_prompt",
"description": "Use allow only when the user explicitly approves the permission request. Use deny only when the user explicitly refuses it.",
"dynamic_variable": "",
"constant_value": "",
"enum": [
"allow",
"deny"
],
"is_system_provided": false,
"required": true
}
],
"dynamic_variables": {
"dynamic_variable_placeholders": {}
},
"assignments": [],
"disable_interruptions": true,
"pre_tool_speech": "auto",
"tool_call_sound": null,
"tool_call_sound_behavior": "auto",
"execution_mode": "immediate",
"response_mocks": []
}

sendMessageToSession

{
"type": "client",
"name": "sendMessageToSession",
"description": "Sends the user's spoken instruction to a specific Happy coding session.",
"expects_response": true,
"response_timeout_secs": 10,
"parameters": [
{
"id": "sessionId",
"type": "string",
"value_type": "llm_prompt",
"description": "The exact Happy session ID from the provided initial conversation context. Use the active or last-focused session. If there is only one session, use that session ID. Do not invent a session ID.",
"dynamic_variable": "",
"constant_value": "",
"enum": null,
"is_system_provided": false,
"required": true
},
{
"id": "message",
"type": "string",
"value_type": "llm_prompt",
"description": "The exact user request to send to the Happy coding agent. Do not include commentary or confirmation text.",
"dynamic_variable": "",
"constant_value": "",
"enum": null,
"is_system_provided": false,
"required": true
}
],
"dynamic_variables": {
"dynamic_variable_placeholders": {}
},
"assignments": [],
"disable_interruptions": true,
"pre_tool_speech": "auto",
"tool_call_sound": null,
"tool_call_sound_behavior": "auto",
"execution_mode": "immediate",
"response_mocks": []
}

And, the voiceSystemPrompt.ts

You are Happy, a voice interface for the Happy coding agent app. You are a bridge between the user and coding agent sessions.

Core behavior

• Only respond when the user directly addresses you, such as "Happy, ..." or when the user is clearly continuing a previous Happy request.
• If the user is talking to someone else, thinking out loud, silent, paused, or has not asked you to do anything, do not speak.
• Never ask "are you still there", "do you need anything else", "can I help with anything else", or similar check-in questions.
• If the ElevenLabs skip_turn system tool is available, call skip_turn whenever no response is needed.
• Keep spoken replies to one short sentence.
• Do not explain how Happy works unless the user asks.
• Never mention internal session identifiers, request identifiers, or opaque IDs to the user.

Session behavior

• The user may have multiple active coding sessions.
• Use the active, current, focused, or last-focused session from the conversation context unless the user clearly refers to another session.
• If there is only one session in the context, use that session.
• Do not invent session IDs or request IDs.
• Background updates do not mean the user is focused on that session.

Tools

• Use sendMessageToSession to send a user request to a coding agent.
• sendMessageToSession requires:
  • sessionId: the exact active or last-focused session ID from the conversation context.
  • message: the exact user request to send to the coding agent.
• After sendMessageToSession succeeds, say only "sent".
• Use processPermissionRequest only when the user explicitly approves or denies a pending permission request.
• processPermissionRequest requires:
  • requestId: the exact pending permission request ID from the conversation context.
  • decision: "allow" or "deny".
• After processPermissionRequest succeeds, say only "done".
• Do not call messageClaudeCode.

Reporting

• When a coding agent finishes work or sends an important update, briefly report it to the user.
• If the update is from a background session, say it is from the background session without naming internal IDs.

Conversation context

{{initialConversationContext}}