Nuwa ASR real-time speech recognition protocol

1. Overview

Nuwa ASR provides real-time speech recognition over WebSocket. Clients stream audio to the ASR service and receive recognition results as they become available.

2. Before integration

• Obtain the WebSocket URL for the ASR service from the Nuwa platform.
• Audio is typically raw, mono PCM sampled at 16 kHz.
• Select the recognition engine with stype: doubao (default) or diy.

3. Connection URL

WebSocket connection

wss://wsapi.nuwaai.com/websk/humanAgent/realasr
wss://wsapi.nuwaai.com/websk/humanAgent/realasr/{stype}

Parameters

4. Connection example

const host = 'wss://your-asr-server.example.com';
const stype = 'doubao';
const ws = new WebSocket(`${host}/realasr/${stype}`);

ws.onopen = () => console.log('ASR WebSocket connected');
ws.onmessage = (event) => console.log('ASR message received:', JSON.parse(event.data));
ws.onclose = () => console.log('ASR WebSocket closed');
ws.onerror = (err) => console.error('ASR WebSocket error:', err);

import websocket
import json

host = 'wss://your-asr-server.example.com'
stype = 'doubao'
url = f'{host}/realasr/{stype}'

def on_open(ws):
    print('ASR WebSocket connected')

def on_message(ws, message):
    print('ASR message received:', json.loads(message))

def on_close(ws, close_status_code, close_msg):
    print('ASR WebSocket closed')

def on_error(ws, error):
    print('ASR WebSocket error:', error)

ws = websocket.WebSocketApp(url, on_open=on_open, on_message=on_message,
                            on_close=on_close, on_error=on_error)
ws.run_forever()

5. Client messages

5.1 Text control messages

Send a JSON message to configure recognition hot words:

{
  "type": "setHotWords",
  "data": { "hotWords": ["hello", "take a break", "how are you"] }
}

5.2 Binary audio messages

Send audio data as binary WebSocket messages:

• Send raw audio bytes (for example, a bytes/ArrayBuffer value).
• Audio is typically mono PCM at 16 kHz.

// Send an audio chunk
ws.send(audioArrayBuffer);

6. Server messages

The server pushes ASR results as JSON messages. Common message types are:

Example: recognition result

{ "type": "asr-text", "data": { "content": "The weather is nice today" } }

Example: final result

{ "type": "asr-commit", "data": { "content": "The weather is nice today" } }

Example: error notification

{ "type": "asr-error", "data": { "message": "Recognition failed", "code": 500 } }

7. Recognition service types

8. Recommended integration flow

1. Open a WebSocket connection to /realasr or /realasr/{stype}.
2. (Optional) Send setHotWords to configure hot words.
3. Stream binary audio data.
4. Continue receiving asr-text and asr-commit messages.
5. Close the WebSocket after recognition is complete.

sequenceDiagram
    participant C as Client
    participant S as ASR service
    C->>S: 1. Open WebSocket connection
    S-->>C: 2. Confirm connection
    C->>S: 3. Send setHotWords (optional)
    S-->>S: 4. Update hot-word configuration
    loop Stream and recognize audio
        C->>S: 5. Send binary audio data
        S-->>C: 6. Return asr-text partial result
        C->>S: 7. Continue sending audio
        S-->>C: 8. Return asr-commit final result
    end
    C->>S: 9. Close WebSocket connection