[Bug]: Frigate's ffmpeg breaks loose if the stream is probed thru' go2rtc's API #13938
-
Checklist
Describe the problem you are havingI am experiencing weird problems with stream when using go2rtc restreaming. Everything works without hickups (tested for several hours so far) if I don't use the UI, but once I open the stream in full in Frigate's live view, the user interface makes request "GET /api/go2rtc/streams/<camera_name>" to Frigate, after which ffmpeg process breaks loose: it starts eating cpu and memory until OOM killer kills it. Log message "<camera_name> exceeded fps limit. Exiting ffmpeg..." appears in Frigate's log when this triggers. Steps to reproduce
Version0.14.1-f4f3cfa In which browser(s) are you experiencing the issue with?Firefox 129.0.2 (64-bit Win) Frigate config filemqtt:
enabled: true
host: REMOVED
user: frigate
password: REMOVED
topic_prefix: frigate
client_id: frigate
tls:
enabled: false
ffmpeg:
hwaccel_args: preset-vaapi
detectors:
coral:
type: edgetpu
device: pci
go2rtc:
streams:
piha:
- rtsp://motion:[email protected]:554/main
cameras:
piha:
enabled: true
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path: rtsp://REMOVED:8554/piha
input_args: preset-rtsp-restream
roles:
- detect
detect:
enabled: true
motion:
threshold: 40
contour_area: 10
improve_contrast: true
snapshots:
enabled: true
retain:
default: 30
record:
enabled: true
retain:
days: 7
mode: all
events:
retain:
default: 30
mode: motion
version: 0.14docker-compose file or Docker CLI commandversion: '3.7'
services:
frigate:
container_name: frigate
image: ghcr.io/blakeblackshear/frigate:stable
restart: unless-stopped
privileged: true
shm_size: "128mb"
devices:
- /dev/dri/renderD128:/dev/dri/renderD128
- /dev/apex_0:/dev/apex_0
volumes:
- /etc/localtime:/etc/localtime:ro
- /opt/frigate/config:/config
- /opt/frigate/storage:/media/frigate
- type: tmpfs
target: /tmp/cache
tmpfs:
size: 1000000000
ports:
- "5000:5000"
- "8971:8971"
environment:
FRIGATE_RTSP_PASSWORD: "REMOVED"Relevant Frigate log outputx.x.x.x - - [24/Sep/2024:15:50:07 +0300] "GET /api/go2rtc/streams/piha HTTP/1.1" 200 1972 "http://<redacted>:8971/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0" "-"
x.x.x.x - - [24/Sep/2024:15:50:09 +0300] "GET /api/piha/latest.webp?h=707&cache=1727182207653 HTTP/1.1" 200 213272 "http://<redacted>:8971/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0" "-"
[2024-09-24 15:50:16] watchdog.piha INFO : piha exceeded fps limit. Exiting ffmpeg...
[2024-09-24 15:50:16] watchdog.piha INFO : Waiting for ffmpeg to exit gracefully...
[... several minutes pass until OOM killer finally gets rid of Frigate's ffmpeg ...]
[2024-09-24 16:04:06] frigate.video ERROR : piha: Unable to read frames from ffmpeg process.
[2024-09-24 16:04:06] frigate.video ERROR : piha: ffmpeg process is not running. exiting capture thread...
[2024-09-24 16:04:16] watchdog.piha ERROR : Ffmpeg process crashed unexpectedly for piha.
[2024-09-24 16:04:16] watchdog.piha ERROR : The following ffmpeg logs include the last 100 lines prior to exit.
[2024-09-24 16:04:16] ffmpeg.piha.detect ERROR : [segment @ 0x55ec2ef43540] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[2024-09-24 16:04:16] ffmpeg.piha.detect ERROR : [segment @ 0x55ec2ef43540] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
[2024-09-24 16:04:16] ffmpeg.piha.detect ERROR : [rtsp @ 0x55ec2edba100] RTP: PT=60: bad cseq 1da5 expected=f049Relevant go2rtc log output2024-09-24 19:01:14.129321743 19:01:14.129 INF go2rtc platform=linux/amd64 revision=b2399f3 version=1.9.2
2024-09-24 19:01:14.129382721 19:01:14.129 INF config path=/dev/shm/go2rtc.yaml
2024-09-24 19:01:14.129784925 19:01:14.129 INF [rtsp] listen addr=:8554
2024-09-24 19:01:14.129888989 19:01:14.129 INF [webrtc] listen addr=:8555/tcp
2024-09-24 19:01:14.129915030 19:01:14.129 INF [api] listen addr=:1984Operating systemDebian Install methodDocker Compose Network connectionWired Camera make and modelMilesight MS-C3263-FPNA Screenshots of the Frigate UI's System metrics pages
Any other information that may be helpfulSince the problem seems to be related to go2rtc, I tried using external go2rtc as well. Unfortunately I was able to reproduce with both version 1.9.2 (used by Frigate 0.14.1) and the latest 1.9.4. Something happens to the already open restream used by Frigate when the stream is probed in go2rtc. Workaround for now: change Frigate's go2rtc's api port so that Frigate can't make requests there and run a separate go2rtc without any streams but still able to answer Frigate's ping so that Frigate's watchdog for go2rtc doesn't trigger :-D |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
I'm not able to reproduce this This may also be a bug in go2rtc considering that the probe using go2rtc api is what starts it. In any case, it is possible that this is fixed in dev already as we are using a newer version of ffmpeg that may perhaps be able to handle whatever error is occurring. |
Beta Was this translation helpful? Give feedback.
-
|
Either it is go2rtc or the camera. I found out that using ffmpeg with go2rtc seemed to also workaround the problem. Now probing the streams doesn't make Frigate's ffmpeg go crazy. Either probing causes go2rtc to somehow reconfigure or disturb the stream OR the camera behaves badly with open streams when go2rtc fetches something extra data from the camera during probing. |
Beta Was this translation helpful? Give feedback.
Either it is go2rtc or the camera. I found out that using ffmpeg with go2rtc seemed to also workaround the problem. Now probing the streams doesn't make Frigate's ffmpeg go crazy. Either probing causes go2rtc to somehow reconfigure or disturb the stream OR the camera behaves badly with open streams when go2rtc fetches something extra data from the camera during probing.