Reduce latency for RTSP streams

RVMedia support and discussion (components for displaying and controlling IP cameras, webcams, video conferencing, video chats, recording audio and video files)
Post Reply
kingdomsrl
Posts: 7
Joined: Thu May 11, 2023 2:42 pm

Reduce latency for RTSP streams

Post by kingdomsrl »

I'm building an user interface with multiple IP camera streams running at the same time. I'm using TRVCamMultiView with up to 16 viewers running. My problem is that the latency for all my cameras start from 1-2 seconds at program launch to 15-30 seconds after a couple of minutes.

This is my setup:
- Delphi 11.3 on Windows 11 x64
- FMX project, using FFMpeg
- Dahua PTZ IP camera, model DH-SD5A432XAN-HNR, supports H265, H264 and MJPEG over RTSP.

I'm using the substream 1 for these cameras for reduced bandwidth usage and this is how such stream is set up:
- Codec: H.264H (seems like an optimized codec)
- Resolution: 704*480
- FPS: 30
- Bitrate: 1024 Kb/S constant
- iFrame interval: 60

Things I have tried so far without improvement:
- Switching DeviceType between rvdtRTSP and rvdtIPCamera
- Switching VideoFormat between rvvfH264, rvvfAVI_H264 and rvvfMP4_H264
- FFMpegProperty.Audio:=False
- FFMpegProperty.TcpNodelay:=True
- FFMpegProperty.RecvBufferSize:=10000 (and smaller values)

Things I have tried but made the situation worse:
- FFMpegProperty.FrameDrop:=True (it causes the stream to end as soon as the first frame arrives)
- FFMpegProperty.RTSPTransport:=[rvpeUDP] (it causes the video to show artifacts, turns black and white, pixelated, jumping lots of frames)

I'm wondering why this latency "stabilizes" at 15-30 seconds and doesn't get larger while at the same time never recovering for real time video.
Sergey Tkachenko
Site Admin
Posts: 17253
Joined: Sat Aug 27, 2005 10:28 am
Contact:

Re: Reduce latency for RTSP streams

Post by Sergey Tkachenko »

1. Do you use the newest version of RVMedia?
2. Assign RVCamera.Latency = 0.
kingdomsrl
Posts: 7
Joined: Thu May 11, 2023 2:42 pm

Re: Reduce latency for RTSP streams

Post by kingdomsrl »

1. Do you use the newest version of RVMedia?
Yes, RVMedia FMX version 9.3
2. Assign RVCamera.Latency = 0
Done, video starts with almost no latency but in a couple of minutes it grows to 15-30 seconds again.

By the way the cameras have a propietary software (which I must replace) and there the latency seems minimal, like 1-3 seconds at most. When running both programs side-by-side video timestamps for propietary and RVMedia are the same, but after several minutes it grows to around 20 seconds (I have some screenshots but I don't know how to upload images here).

Something interesting I noted is that for this test I reduced the number of cameras from 16 to 6 and I took 10 minutes instead of 2 to reach that 20 sec latency. Will test with a single camera to see how it goes.

Edit: With a single camera latency between propietary and RVMedia is around 1 second and it stays the same.
Sergey Tkachenko
Site Admin
Posts: 17253
Joined: Sat Aug 27, 2005 10:28 am
Contact:

Re: Reduce latency for RTSP streams

Post by Sergey Tkachenko »

Theoretically, delay may be accumulated in the following places:
1. Inside the library that is used to get video (i.e., inside FFmpeg or GStreamer DLL)
2. In the code that works with this library and tries to keep a framerate of the original video.
3. Later, when decoded frames are buffered.

Delays at the step (3) must be eliminated when you assign RVCamera.Latency = 0.
Let's try to check the step (2). Open fmxMRVFFmpegObject.pas, and comment the lines:

Code: Select all

        if delay > 1 then
          Sleep(Round(delay));
Making delays between frames when decoding is important when displaying a prerecording video. But when displaying a stream from a camera, this delay is not necessary, because frames are received at real time.
Most probably, you use Windows 64-bit platform. In this case, your application is compiled with precompiled RVMedia units, and changes in source code can be applied only after recompiling packages. The simplest way to recompile them is running "Install RVMedia FMX in Delphi IDE" in Windows Start menu.

Delays at the step (1) may be reduced by assigning some FFMpegProperties. Or you can try using GStreamer instead of FFmpeg.
kingdomsrl
Posts: 7
Joined: Thu May 11, 2023 2:42 pm

Re: Reduce latency for RTSP streams

Post by kingdomsrl »

After commenting those pair of lines and recompiling packages (yes, I'm compiling for Win64) I can confirm latency is gone. RVMedia now is performing as good as the proprietary software, even when showing multiple streams simultaneously.
Thank you Sergey.
Post Reply