Troubleshooting & Deep Dives Overview
When diagnosing anomalies in high-throughput video pipelines, a systematic diagnostic methodology is crucial. RUSEON Core's decoupled Ring Buffer architecture ensures that failures in one subsystem (such as slow disk writes) do not block or crash live WebRTC or HLS delivery.
Troubleshooting Guide Directory
| Issue Domain | Typical Symptoms | Deep-Dive Guide |
|---|---|---|
| Page Cache Thrashing & OOM | Linux memory usage climbs to 100%, system becomes unresponsive during heavy writes | Page Cache Thrashing Deep Dive |
| Corrupted MP4 & Power Loss | Recordings unplayable after sudden power outage or kernel panic | MP4 Atom Recovery Guide |
| Cameras & RTSP Ingest | Camera feeds dropping, 401 Unauthorized, RTSP TCP timeouts, frame distortion | Camera Troubleshooting |
| Streaming & WebRTC / HLS | WebRTC ICE connection failed, HLS playlist buffering, audio/video desync | Streaming Troubleshooting |
| Archive & Storage | Disk quota full, recording intervals missing, slow timeline queries | Archive Troubleshooting |
| Performance & CPU/RAM | High CPU usage, GC latency spikes, buffer frame drops under high load | Performance Troubleshooting |
| Common Error Catalog | Quick reference of error messages, HTTP status codes, and exact remediation steps | Common Errors Matrix |
4-Step Standard Diagnostic Workflow
- Check Live Probes:bash
curl -i http://localhost:9090/livez curl -i http://localhost:9090/readyz - Inspect Structured Logs:bash
# Filter for warnings and errors in real time sudo journalctl -u ruseon -f | grep -E '"level":"(warn|error|fatal)"' - Inspect Prometheus Buffer Drops: Check if
ruseon_buffer_drops_totalis incrementing, which indicates a slow subscriber channel. - Capture Live Profiling: If CPU or memory is abnormal, capture a pprof heap snapshot (
curl -s http://localhost:9090/debug/pprof/heap > heap.pprof).