Stream live call audio

Use the VoxML Stream verb to connect an answered call to a WebSocket service. It is intended for real-time transcription, conversational AI, quality monitoring, and other applications that need access to the call audio.

TryVox begins the stream while executing the call's VoxML response. There is currently no separate REST API for creating, listing, retrieving, or stopping stream objects on an already-running call.

Basic example

{
  "voxml_version": "1.0",
  "instructions": [
    {
      "verb": "Stream",
      "url": "wss://voice.example.com/calls",
      "track": "inbound_track",
      "parameters": {
        "agent_id": "support-assistant",
        "language": "en-IN"
      }
    }
  ]
}

Fields

outbound_track is not currently supported. TryVox returns a stream execution error instead of silently substituting another track.

Choosing a track

Use inbound_track for speech recognition or an AI assistant that should listen to the caller:

{
  "verb": "Stream",
  "url": "wss://voice.example.com/listen",
  "track": "inbound_track"
}

Use both_tracks for analytics, compliance, or transcription that needs the complete conversation:

{
  "verb": "Stream",
  "url": "wss://voice.example.com/analyse",
  "track": "both_tracks"
}

With both_tracks, the two directions are mixed into one mono channel. They are not delivered as separately labelled tracks.

WebSocket connection

Your server must:

• Accept a WebSocket client connection from TryVox.
• Accept the audio.drachtio.org WebSocket subprotocol.
• Handle one initial UTF-8 text frame followed by binary audio frames.
• Keep reads non-blocking and process audio continuously.

The connection sequence is:

1. TryVox opens the WebSocket.
2. TryVox sends the parameters object as a JSON text frame.
3. TryVox sends raw PCM audio in binary frames.
4. The connection closes when the call channel or stream ends.

For example, these parameters:

{
  "parameters": {
    "account_ref": "acct_42",
    "conversation_ref": "conv_91"
  }
}

arrive as the first text frame:

{
  "account_ref": "acct_42",
  "conversation_ref": "conv_91"
}

Do not treat the first text frame as audio.

Audio format

Audio sent from TryVox to your server uses:

The audio frames are binary WebSocket messages. They are not JSON objects and are not Base64 encoded.

If your speech or model provider expects 16 kHz audio, resample the PCM stream after receiving it. Do not reinterpret 8 kHz bytes as 16 kHz audio.

Authentication

The url may contain a short-lived, call-scoped token:

{
  "verb": "Stream",
  "url": "wss://voice.example.com/calls?token=eyJhbGciOi...",
  "track": "inbound_track"
}

Prefer an expiring token over a permanent API key. Validate it during the WebSocket upgrade and bind it to the expected account or call. Avoid placing reusable account credentials in parameters, because metadata travels inside the established WebSocket rather than authenticating the initial upgrade request.

Custom WebSocket request headers are not currently configurable through the Stream verb.

Stream termination

The WebSocket close event is the stream-level termination signal. Release model sessions, buffers, and temporary resources when it arrives.

For the authoritative call result, also configure the call's hangup_url. A stream can close because of a call hangup, a network failure, or media-server shutdown, so the close event alone does not contain the final call disposition.

TryVox does not currently emit separate Stream status callbacks or expose a Stream ID.

Sending audio toward the call

The same WebSocket can send audio back to the caller. Send a UTF-8 JSON text frame with type: "playAudio":

{
  "type": "playAudio",
  "data": {
    "audioContentType": "raw",
    "sampleRate": 8000,
    "audioContent": "BASE64_ENCODED_PCM_AUDIO"
  }
}

audioContent must be Base64-encoded mono signed linear PCM. For raw audio, sampleRate must be 8000 or 16000. You may instead send a complete WAV file by setting audioContentType to wave and Base64-encoding the file.

Each message is queued for playback on the call leg. Keep messages reasonably sized and send them in playback order.

TryVox does not currently define checkpoint, clearAudio, or stop WebSocket commands. Use the WebSocket close event for stream cleanup and the call API or VoxML for call control.

AI assistant example

{
  "voxml_version": "1.0",
  "instructions": [
    {
      "verb": "Say",
      "text": "Please wait while I connect the assistant."
    },
    {
      "verb": "Stream",
      "url": "wss://assistant.example.com/media?session=short-lived-token",
      "track": "inbound_track",
      "parameters": {
        "assistant": "customer-care",
        "locale": "en-IN"
      }
    }
  ]
}

Operational guidance

• Use wss:// with a publicly trusted certificate in production.
• Authorize the WebSocket during the upgrade, before accepting audio.
• Expect the first frame to be text and all subsequent audio frames to be binary.
• Apply backpressure and keep per-frame processing short.
• Resample explicitly when downstream services require another sample rate.
• Treat WebSocket disconnects as normal cleanup events and use hangup_url for the final call status.
• Do not log access tokens or raw call audio unless your retention and consent policies allow it.

Current limitations

• Streams can only be started through VoxML.
• The input format is fixed to 8 kHz L16.
• Only inbound_track and mono-mixed both_tracks are supported.
• Stream IDs, REST lifecycle operations, reconnect controls, status callbacks, and custom upgrade headers are not exposed.
• Bidirectional playback supports playAudio; checkpoint, clear, and stop control events are not exposed.