Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache cleanup does not clean cache entries for versions of gradle >8.9 #417

Closed
benkay opened this issue Oct 18, 2024 · 7 comments
Closed
Assignees
Labels
bug Something isn't working
Milestone

Comments

@benkay
Copy link

benkay commented Oct 18, 2024

The documentation implies that no further setup is required for the action to cleanup caches after builds.

I was investigating why our CI cache is steadily rising, and the cause seems to be that old transforms are not being cleaned up effectively.

We're building using Gradle 8.10.2 (via the wrapper), but this is the log output from the cache cleanup phase of the action

   Extracted Gradle 8.9 to /opt/actions-runner/_work/_temp/.gradle-actions/gradle-installations/installs/gradle-8.9
  Provisioned Gradle executable /opt/actions-runner/_work/_temp/.gradle-actions/gradle-installations/installs/gradle-8.9/bin/gradle
[..snip..]
  artifact cache (/home/runners/.gradle/caches/modules-2) cleaned up in 0.121 secs.
  Artifact transforms cache (/home/runners/.gradle/caches/8.9/transforms) removing files not accessed on or after Fri Oct 18 13:13:24 UTC 2024.
  Artifact transforms cache (/home/runners/.gradle/caches/8.9/transforms) cleanup deleted 0 files/directories.
  Artifact transforms cache (/home/runners/.gradle/caches/8.9/transforms) cleaned up in 0.0 secs.
  groovy-dsl (/home/runners/.gradle/caches/8.9/groovy-dsl) removing files not accessed on or after Fri Oct 18 13:13:24 UTC 2024.
  groovy-dsl (/home/runners/.gradle/caches/8.9/groovy-dsl) cleanup deleted 2 files/directories.
  groovy-dsl (/home/runners/.gradle/caches/8.9/groovy-dsl) cleaned up in 0.009 secs.
  jars (/home/runners/.gradle/caches/jars-9) cleanup deleted 0 files/directories.
  jars (/home/runners/.gradle/caches/jars-9) removing files not accessed on or after Fri Oct 18 13:13:24 UTC 2024.
  jars (/home/runners/.gradle/caches/jars-9) cleanup deleted 1 files/directories.
  jars (/home/runners/.gradle/caches/jars-9) cleaned up in 0.039 secs.

As the cache cleanup is provisioning Gradle 8.9, it looks like it's not cleaning up any transforms produced by newer versions.
The cache cleanup should either use the wrapper, or provision the latest version of Gradle so that transform caches are cleaned up.

@bigdaz
Copy link
Member

bigdaz commented Oct 31, 2024

Thanks for the report. This is likely an unwanted side-effect of #364, which performs cache-cleanup using the version of Gradle on PATH (at least 8.9) instead of always downloading the latest.

This means that cache cleanup won't work for a Gradle version > 8.9 unless that version (or newer) is available on the PATH.

Fixing this (without reverting to the old behaviour of always installing the latest Gradle) will involve detecting the newest Gradle version executed for a Job, and ensuring that cleanup is executed with at least that Gradle version.

@bigdaz bigdaz self-assigned this Nov 11, 2024
@bigdaz bigdaz added this to the v4.2.0 milestone Nov 11, 2024
@bigdaz bigdaz added the bug Something isn't working label Nov 11, 2024
bigdaz added a commit that referenced this issue Nov 11, 2024
The cache-cleanup API has changed, so the init-script that worked with
Gradle 8.9 no longer works with 8.11.
We now provision and use Gradle 8.11 for cache cleanup.

This provides a band-aid fix for #417 but that issue will still impact
any build configured to run with Gradle > 8.11
@bigdaz bigdaz changed the title Cache cleanup does not clean cache entries for versions of gradle >8.9 Cache cleanup may not clean cache entries for recent Gradle versions Nov 11, 2024
@bigdaz bigdaz changed the title Cache cleanup may not clean cache entries for recent Gradle versions Cache cleanup does not clean cache entries for newly released Gradle versions Nov 11, 2024
bigdaz added a commit that referenced this issue Nov 12, 2024
The cache-cleanup API has changed, so the init-script that worked with
Gradle 8.9 no longer works with 8.11.
We now provision and use Gradle 8.11 for cache cleanup.

This provides a band-aid fix for #417 but that issue will still impact
any build configured to run with Gradle > 8.11
bigdaz added a commit that referenced this issue Nov 12, 2024
The cache-cleanup API has changed, so the init-script that worked with
Gradle 8.9 no longer works with 8.11.
We now provision and use Gradle 8.11 for cache cleanup.

This provides a band-aid fix for #417 but that issue will still impact
any build configured to run with Gradle > 8.11
@ViacheslavKudinov
Copy link

ViacheslavKudinov commented Nov 12, 2024

Hi @bigdaz
Not sure if it is connected but post Setup Gradle fails since release of Gradle 8.11
with error

Provision Gradle 8.11
/root/gradle-installations/installs/gradle-8.11/bin/gradle -g /root/.gradle -I init.gradle --info --no-daemon --no-scan --build-cache -DGITHUB_DEPENDENCY_GRAPH_ENABLED=false noop
Initialized native services in: /root/.gradle/native
Initialized jansi services in: /root/.gradle/native
To honour the JVM settings for this build a single-use Daemon process will be forked. For more on this, please refer to https://docs.gradle.org/8.11/userguide/gradle_daemon.html#sec:disabling_the_daemon in the Gradle documentation.
Starting process 'Gradle build daemon'. Working directory: /root/.gradle/daemon/8.11 Command: /actions-runner/_work/_tool/Java_Temurin-Hotspot_jdk/21.0.5-11.0.LTS/x64/bin/java --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.xml/javax.xml.namespace=ALL-UNNAMED -XX:MaxMetaspaceSize=384m -XX:+HeapDumpOnOutOfMemoryError -Xms256m -Xmx512m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /root/gradle-installations/installs/gradle-8.11/lib/gradle-daemon-main-8.11.jar -javaagent:/root/gradle-installations/installs/gradle-8.11/lib/agents/gradle-instrumentation-agent-8.11.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 8.11
Successfully started process 'Gradle build daemon'
An attempt to start the daemon took 0.936 secs.
The client will now receive all logging from the daemon (pid: 6388). The daemon log file: /root/.gradle/daemon/8.11/daemon-6388.out.log
Daemon will be stopped at the end of the build 
Using 8 worker leases.
Received JVM installation metadata from '/actions-runner/_work/_tool/Java_Temurin-Hotspot_jdk/21.0.5-11.0.LTS/x64': {JAVA_HOME=/actions-runner/_work/_tool/Java_Temurin-Hotspot_jdk/21.0.5-11.0.LTS/x64, JAVA_VERSION=21.0.5, JAVA_VENDOR=Eclipse Adoptium, RUNTIME_NAME=OpenJDK Runtime Environment, RUNTIME_VERSION=21.0.5+11-LTS, VM_NAME=OpenJDK 64-Bit Server VM, VM_VERSION=21.0.5+11-LTS, VM_VENDOR=Eclipse Adoptium, OS_ARCH=amd64}
Watching the file system is configured to be enabled if available
Now considering [/actions-runner/_work/_temp/dummy-cleanup-project] as hierarchies to watch
File system watching is active
Caching disabled for Groovy DSL script compilation (cp_init) because:
  Build cache is disabled
  Not worth caching.
Compiling initialization script '/actions-runner/_work/_temp/dummy-cleanup-project/init.gradle' using SubsetScriptTransformer.
Caching disabled for Groovy DSL script compilation (init) because:
  Build cache is disabled
  Not worth caching.
Compiling initialization script '/actions-runner/_work/_temp/dummy-cleanup-project/init.gradle' using BuildScriptTransformer.
Starting Build

FAILURE: Build failed with an exception.

* Where:
Initialization script '/actions-runner/_work/_temp/dummy-cleanup-project/init.gradle' line: 8

* What went wrong:
Cannot get the value of write-only property 'removeUnusedEntriesOlderThan' for object of type org.gradle.api.internal.cache.DefaultCacheConfigurations$DefaultCacheResourceConfiguration.

Or just error

What went wrong:
Cannot get the value of write-only property 'removeUnusedEntriesOlderThan' for object of type org.gradle.api.internal.cache.DefaultCacheConfigurations$DefaultCacheResourceConfiguration.

worked well on 8.10.2 few days ago (probably before release 8.11)

Su5eD added a commit to Sinytra/Connector that referenced this issue Nov 12, 2024
@Su5eD
Copy link

Su5eD commented Nov 12, 2024

Running into the same issue here. I tried manually specifying the used gradle version but it looks like post-cleanup provisions its own gradle instance?

Is there an temporary workaround I can use until this is addressed?

@ViacheslavKudinov
Copy link

@Su5eD Please, try input cache-disabled to set false.
At least no failed post step for us.

@TemMax
Copy link

TemMax commented Nov 12, 2024

Changing the flag from true to false worked for me: gradle-home-cache-cleanup: false + removing cache-read-only: false

@bigdaz
Copy link
Member

bigdaz commented Nov 12, 2024

Thanks for the additional reports.
Yesterday I raised #431 for the specific issue of cache-cleanup failing on Gradle 8.11.
The fix has been merged and will be included in the next gradle/actions release, likely today.

@bigdaz bigdaz removed this from the v4.2.0 milestone Nov 12, 2024
@bigdaz bigdaz changed the title Cache cleanup does not clean cache entries for newly released Gradle versions Cache cleanup does not clean cache entries for versions of gradle >8.9 Nov 12, 2024
@bigdaz
Copy link
Member

bigdaz commented Nov 12, 2024

The recent release of [email protected] has addressed 2 issues related to cache cleanup:

There is still an issue with cache-cleanup failing to remove entries written by newer (not-yet-released) versions of Gradle. I've raised #436 for this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants