ArgoCD sometimes fails to template Application manifest when using Helm Secrets

[Benjou02]

Checklist

• I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• I've included steps to reproduce the bug.
• I've pasted the output of argocd version.

Describe the bug

Sometimes, when syncing an app with some secrets:// values file that is indeed where it should be, the sync will fail with the following error, found in the UI and the repo server: Error: open secrets://secrets/prod.sops.yaml: no such file or directory.

(Trying) To Reproduce

We have not been able to reproduce consistently the issue, it seems to occur randomly when syncing such an app. However, here are some findings we discovered when trying to reproduce the behavior.

We did not manage to reproduce the bug manually, directly using the same helm template command (shown in the logs below) from within the repo server. However, the only way we found to reproduce this precise error message, was to launch the command without having installed the helm-secrets plugin.

When the file is missing and helm-secrets is installed, the message is different:

[helm-secrets] File does not exist: secrets/prod.sops.yaml 
Error: plugin "scripts/run.sh downloader" exited with error

The fact that this error seems random but always mentions the secrets:// file, combined with the fact that we could not obtain this exact error other than by removing the plugin, suggests that the problem is in the integration of the plugin.

I am opening this issue in the ArgoCD repository as it is the only place where we saw the problem occur, since we were not able to reproduce it using directly Helm.

Expected behavior

The sync should never fail with this error when the file is not missing.

Versions

argocd: v2.11.7+e4a0246
helm: version.BuildInfo{Version:"v3.14.4", GitCommit:"81c902a123462fd4052bc5e9aa9c513c4c8fc142", GitTreeState:"clean", GoVersion:"go1.21.9"}
helm-secrets: 4.2.2-dev

Logs

In the repo server:

time="2024-12-31T13:11:20Z" level=error msg="finished unary call with code Unknown" error="Manifest generation error (cached): `helm template . --name-template my-app --namespace app --kube-version 1.29 --values <path to cached source>/values.yaml --values secrets://secrets/prod.sops.yaml <api versions removed> --include-crds` failed exit status 1: Error: open secrets://secrets/prod.sops.yaml: no such file or directory" grpc.code=Unknown grpc.method=GenerateManifest grpc.service=repository.RepoServerService grpc.start_time="2024-12-31T13:11:20Z" grpc.time_ms=10.015 span.kind=server system=grpc