# Bring your own speech service

> Run a small bridge that speaks VoiceLayer's WebSocket protocol, and the agent's voice or transcriber runs on your service instead of one we list.

Source: https://docs.vlayers.ai/developers/custom-speech

Kind: Reference

This is the reference for bringing a speech service VoiceLayer does not list. It covers version 1 of the protocol: the handshake, the messages, the limits and the regions. A runnable example bridge in front of Deepgram ships with the SDK examples.

You run a small server (a "bridge") that speaks this protocol. VoiceLayer connects to it for a call: one WebSocket for the
transcriber and one for the voice, both to the URL on your connection. There is no other traffic: VoiceLayer never sends
your endpoint a test request, and the only other connection it ever makes is one short voice socket to make an apology clip
(`purpose: "clip"`, below).

## The connection

- The URL is `wss:` only. It is fixed on the connection, and its **host is sealed with each agent version that uses the
  connection**: if you point the connection at another host, calls on the older versions are refused (`connection_changed`)
  until you publish the agents again.
- The address must be public. VoiceLayer resolves the name once, refuses a private, loopback, link-local or metadata address,
  and connects to exactly that address; a redirect is refused. Put your bridge behind TLS with a certificate for the host.
- You choose a secret for the connection (the bearer secret). VoiceLayer also gives the connection a **signing secret**,
  shown once when the connection is created.

## The handshake: two credentials for two questions

| Header | Answers | Required |
|---|---|---|
| `Authorization: Bearer <secret>` | May this caller in? | yes |
| `x-voicelayer-signature: v1=<hex>` | Is it really VoiceLayer, for this call, now? | verifying it is your choice |
| `x-voicelayer-timestamp` | Unix seconds when VoiceLayer signed | with the signature |
| `x-voicelayer-call-id` | The call's id (the same as `callId` in `start`) | with the signature |
| `x-voicelayer-protocol` | The protocol number VoiceLayer speaks (`1`) | informational |

The signature is `HMAC-SHA256(signing secret, timestamp + "." + callId + "." + host)`, hex, prefixed `v1=`, where `host` is
the host of your connection's URL, lowercased, with the port when it is not the default. Recompute it and refuse a mismatch,
and refuse a timestamp more than five minutes from your clock. The bearer secret alone is enough to connect; the signature is
how an endpoint that is reachable by others knows a connection came from VoiceLayer for a call. The header carries a version
(`v1=`) so a public-key scheme can be added beside it without breaking you.

```ts
import { createHmac, timingSafeEqual } from 'node:crypto';

function verify(headers: Record<string, string | undefined>, host: string, secret: string): boolean {
  const ts = Number(headers['x-voicelayer-timestamp']);
  if (!Number.isFinite(ts) || Math.abs(Date.now() / 1000 - ts) > 300) return false;
  const want = 'v1=' + createHmac('sha256', secret).update(`${ts}.${headers['x-voicelayer-call-id']}.${host}`).digest('hex');
  const got = headers['x-voicelayer-signature'] ?? '';
  return got.length === want.length && timingSafeEqual(Buffer.from(got), Buffer.from(want));
}
```

## The first messages

Your **first message** is `ready`, echoing the protocol number you speak. A different number refuses your endpoint for the
call (VoiceLayer does not guess across versions). Then VoiceLayer sends `start`.

```
← {"type":"ready","protocol":1}
→ {"type":"start", ... }
```

An endpoint ignores a field it does not know. Fields are added, never repurposed.

## The transcriber

```
→ {"type":"start","leg":"stt","protocol":1,"callId":"…","encoding":"linear16","sampleRate":16000,
   "languages":["gu-IN","hi-IN","en-IN"],"switching":"listed"}
→ binary frames: caller audio, 16-bit little-endian PCM, mono, about 20 ms each
← {"type":"partial","text":"…"}                                          // optional, as the caller speaks
← {"type":"final","text":"…","language":"gu-IN","confidence":0.94}       // one per utterance
← {"type":"speech_start"} / {"type":"speech_end"}                         // optional
→ {"type":"stop"}                                                        // the call's audio is over
```

- `languages` lists the languages the call may hear (the agent's); `switching` is `listed` (callers move among them), `any`, or
  `off`. The first language is the one the call starts in.
- `language` on a `final` is optional. Without it the connection's `reportsLanguage` is false, and the call follows the
  caller's language by the reply's script only.
- If you send no `speech_start` / `speech_end`, VoiceLayer takes the first partial or final as the start of speech and a
  `final` as its end. If you send `speech_end`, send the `final` right after it.
- Audio toward you is buffered up to two seconds; past that, frames are dropped and counted on the call (a slow endpoint hears a
  gap, not a delay that grows).
- Time to the first result: counted from the moment 300 ms of the caller's voice has been sent. The default is 6 seconds; you
  may raise it for your own endpoint on the connection, up to 15 seconds. A silent call never starts the clock.

## The voice

```
→ {"type":"start","leg":"tts","protocol":1,"callId":"…","encoding":"linear16","sampleRate":24000,"purpose":"call"}
→ {"type":"speak","id":"u1","text":"…","language":"gu-IN","voice":"…"}
← binary frames: audio for the utterance in progress, 16-bit little-endian PCM, mono, at the `sampleRate` you were told
← {"type":"done","id":"u1"}
→ {"type":"cancel","id":"u1"}                                            // the caller interrupted
← {"type":"done","id":"u1"}                                              // acknowledge the cancel
```

- One utterance at a time (one sentence). All of an utterance's frames come before its `done`. The text has already passed
  VoiceLayer's output guard.
- `purpose` is `"call"` for the call's own voice and `"clip"` for the one short socket opened to make an apology clip (VoiceLayer
  speaks one fixed sentence once and stores the audio). Both carry a valid signature for the same call.
- Answer a `cancel` with `done` for that id. The next sentence waits up to a second for it, and then replaces the socket.
- Time to the first audio frame after `speak`: default 6 seconds, raisable on the connection up to 15.

## Errors

```
← {"type":"error","id":"u1","code":"voice_unknown","message":"…"}
```

An `error` ends the utterance (or, without an `id`, the stream). `code` is yours; it is shown on the call's record, so keep it short and
free of secrets. A refused handshake is an HTTP `401` on the upgrade.

## What VoiceLayer holds an endpoint to

| Limit | What happens |
|---|---|
| 3 seconds to connect and to send `ready` | The attempt fails |
| First result / first audio (defaults 6 s, up to 15 s) | The attempt fails |
| One reconnect per call (with a fresh signature), then nothing more | The call plays its stored apology |
| A frame from you over 64 KB | The stream is closed as `endpoint_misbehaving` |
| Audio faster than 3× real time over any 2 seconds | `endpoint_misbehaving` |
| A `final` over 2,000 characters, or more than 50 text messages a second | `endpoint_misbehaving` |
| Anything you send before VoiceLayer asks, beyond a small allowance | `endpoint_misbehaving` |

Every refusal is on the call's record with its reason. VoiceLayer does not judge your provider's quality; the call's record shows the
time to first result and first audio so you can.

## Regions

You declare where your endpoint processes speech (`in`, `eu` or `us`) on the connection. If the workspace limits speech processing
to a region, a call whose agent uses an endpoint outside it is refused at its start (`provider_outside_region`), and a fallback
outside it is not used. This says where the speech provider processes; it is not a claim about where VoiceLayer itself runs.
