REST API: Live Video Streams & WebRTC WHEP Endpoints

The Streams API delivers real-time video, audio, and AI metadata using WebRTC WHEP, WebCodecs WebSocket, and Low-Latency HLS.

---

1. WebRTC WHEP Signaling (POST /stream/webrtc/whep/:id)

Initiates sub-500ms browser playback using standard IETF WHEP SDP offer/answer exchange.

• Request Headers:
  • Content-Type: application/sdp
  • Authorization: Bearer <TOKEN> (or query parameter ?token=... if token_auth: true)
• Request Body: Raw SDP Offer string generated by browser RTCPeerConnection.createOffer()

curl -X POST "http://localhost:8080/stream/webrtc/whep/cam-01" \
  -H "Content-Type: application/sdp" \
  --data-binary @client_offer.sdp

• Response (201 Created):
  • Content-Type: application/sdp
  • Body: Raw SDP Answer string from Pion WebRTC engine.

---

2. WebCodecs Binary WebSocket (GET /stream/ws/:id)

For ultra-low latency hardware-accelerated rendering in modern HTML5 WebCodecs players:

GET /stream/ws/cam-01?token=<STREAM_TOKEN>
Upgrade: websocket
Connection: Upgrade

Binary Frame Protocol:

• 0x01 (Codec Configuration Packet): Prepend byte 0x01 followed by SPS/PPS/VPS parameters for VideoDecoder.configure().
• 0x02 (Video Payload Packet): Prepend byte 0x02 followed by monotonically increasing timestamp (8 bytes) and raw NALU slices.

---

3. HTTP Live Streaming (HLS) Endpoints

3.1 Master Manifest (GET /stream/hls/:id/index.m3u8)

Returns the multi-variant master HLS manifest linking video and AI subtitle tracks:

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="AI Detections",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",URI="subs.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=4000000,SUBTITLES="subs"
stream.m3u8

---

3.2 Video Media Playlist (GET /stream/hls/:id/stream.m3u8)

Sliding window playlist listing available live virtual MPEG-TS chunks held in memory:

curl "http://localhost:8080/stream/hls/cam-01/stream.m3u8"

---

3.3 AI Metadata Subtitle Track (GET /stream/hls/:id/subs.m3u8)

Delivers synchronized WebVTT cues containing JSON bounding boxes from downstream neural network inference.

---

3.4 Live Segment Chunk (GET /stream/hls/:id/:segment)

Downloads an individual in-memory MPEG-TS video segment (e.g. segment_1042.ts).

Related Documentation

• REST API Reference
• WebRTC WHEP Streaming Guide
• HLS Streaming Guide
• AI Metadata Pipeline