Google Device Access / WebRTC Memory Leak

[jflorchi]

Describe the bug
It seems that when Google Device Access opens new WebRTC connections, the older connections are not being closed. After a few days, the WebRTC plugin will report "Current Open Connections" over 400 and my LXC container will be using nearly 16 GB of RAM. When I restart the WebRTC plugin, RAM usage drops down to 3 GB.

To Reproduce
Use Google Device Access with WebRTC prebuffering and wait, the open connections and memory usage will steadily climb. I'm using a single Nest Doorbell Wired if it matters.

Expected behavior
WebRTC connections should be closed when no longer in use.

Screenshots

Server (please complete the following information):
Issue happens with Docker install, LXC install, Home Assistant, and running with node directly. Yes, I've tried all of them.

Hardware Model (please complete the following information):
Nest Doorbell Wired Gen 2

Client (please complete the following information, if applicable):
N/A

Additional context
I expect it has to do with GenerateWebRtcStream SDM renewal process. I lowered the "refreshAt" value from 4 minutes to 1 minute and saw significant improvement in time-to-oom, but it did not completely fix the issue. I also made this change, but it didn't seem to make a difference.

[tealtwo]

Describe the bug It seems that when Google Device Access opens new WebRTC connections, the older connections are not being closed. After a few days, the WebRTC plugin will report "Current Open Connections" over 400 and my LXC container will be using nearly 16 GB of RAM. When I restart the WebRTC plugin, RAM usage drops down to 3 GB.

To Reproduce Use Google Device Access with WebRTC prebuffering and wait, the open connections and memory usage will steadily climb. I'm using a single Nest Doorbell Wired if it matters.

Expected behavior WebRTC connections should be closed when no longer in use.

Screenshots

Server (please complete the following information): Issue happens with Docker install, LXC install, Home Assistant, and running with node directly. Yes, I've tried all of them.

Hardware Model (please complete the following information): Nest Doorbell Wired Gen 2

Client (please complete the following information, if applicable): N/A

Additional context I expect it has to do with GenerateWebRtcStream SDM renewal process. I lowered the "refreshAt" value from 4 minutes to 1 minute and saw significant improvement in time-to-oom, but it did not completely fix the issue. I also made this change, but it didn't seem to make a difference.

hey, so uhm the way Google’s SDM API work is unless the stream is renewed it’s automatically closed at the 5 minutes, and if it’s renewed after another 5 minutes it’ll close unless it’s renewed again. https://developers.google.com/nest/device-access/api/camera

This is NOT a Scrypted or GDA Plugin issue.
the image of details for the WebRTC stream that you sent is the details of the stream and it’s rebroadcast address, it has nothing to do with this. As someone who has used the GDA plugin with Scrypted, and still uses a custom WebRTC implementation this issue is unrelated to GDA/Scrypted’s WebRTC implementation.

[koush]

there's a teardown leak in the webrtc plugin. probably need some sort of activity health check.

[hkoci]

I've had this issue for a few months on a Nest 1st gen camera, I also use Ring and a generic RTSP camera.

I migrated from Proxmox to Synology DSM and added a 4gb memory restriction for the container, on Proxmox - in this LXC I added earlyoom to remove remnant ffmpeg processes - I saw when using htop, that some of these streams were over 2-3 days old.

[tealtwo]

Wouldn’t first gen Nest be RTSP?

[hkoci]

Wouldn’t first gen Nest be RTSP?

It's a good question - the documentation and previous knowledge was that it is RTSP with the Nest app, although with the Google Home migration and the Preview firmware after migrating the device from Nest to Google Home - this gives WebRTC feeds.

My current issue is rather the rebroadcasting add-on when set with ffmpeg, I used Surveillance Station and moved to Frigate - there are inactive RTSP streams that are not killed when the stream is lost.

Btw, I have a workaround - although I have moved away from Nest to Ring (I know just as bad using cloud devices haha), there's a plugin for Scrypted developed by apocaliss92 that can monitor plugins.

I use this setup to cleanup the remnant RPC objects every 6 am, as I couldn't get Nest to send the motion events, I had OpenCV with the video analysis also creating stagnant processes alongside the WebRTC, so adjust to what the memory leak is addressing. I previously used an earlyoom killer process on the Debian container running Scrypted; although this would also kill my Homekit plugin - so this is a much better solution.

I don't agree that killing processes is the answer but it is easier to manage for now.