Dependency submission graph updates

[pexa-slee]

Hi all,

I was updating a spring dependency recently to resolve a CVE. The dependency submission action ran on main. Now when I look at the dependency graph, I see two versions of the same dependency. The alert was raised correctly on the old version of Spring MVC. But now it is resolved, I was expecting that the graph would be overridden with the new version and the dependabot alert would automatically close.

The version of the action we are using is v4 gradle/actions/dependency-submission@v4

Or is this the assumed behaviour?

Regards,

[bigdaz]

The most likely explanation is that you have 2 different versions of the dependency in your dependency graph.

[pexa-slee]

The most likely explanation is that you have 2 different versions of the dependency in your dependency graph.

Good point. Let me confirm by running the list of dependencies and see what are the transitives.

[pexa-slee]

I can confirm the repo in question is on 6.1.13

./gradlew -q dependencies > dependencies.txt

❯ cat dependencies.txt | grep spring-webmvc
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)
|    \--- org.springframework:spring-webmvc:6.1.13
|    |    \--- org.springframework:spring-webmvc:5.3.31 -> 6.1.13 (*)

[bigdaz]

Please read the FAQ linked above. In particular you'll need to follow the instructions in the linked HOW-TO.

Running gradle dependencies is not sufficient to determine all of the dependency versions in your project.

[pexa-slee]

I'm going down a bit of a rabbit hole here...

So I enabled debug logging for the task running off the main branch as stated here

I see in the logs references to org.springframework:spring-webmvc:6.1.11

Examples:

Detected dependency 'org.springframework:spring-webmvc:6.1.11': project = ':', configuration = 'compileClasspath'
Detected dependency 'org.springframework:spring-webmvc:6.1.11': project = ':', configuration = 'implementationDependenciesMetadata'
Detected dependency 'org.springframework:spring-webmvc:6.1.11': project = ':', configuration = 'productionRuntimeClasspath'

Now, locally I have run the recommended cmds to see if this dep does exist

Examples

 ./gradlew dependencyInsight --configuration  productionRuntimeClasspath --dependency org.springframework:spring-webmvc:6.1.11
 ./gradlew dependencyInsight --configuration  compileClasspath  --dependency org.springframework:spring-webmvc:6.1.11
 ./gradlew dependencyInsight --configuration  intTestCompileClasspath  --dependency org.springframework:spring-webmvc:6.1.11
 ./gradlew dependencyInsight --configuration  runtimeClasspath  --dependency org.springframework:spring-webmvc:6.1.11
 ./gradlew dependencyInsight --configuration  implementationDependenciesMetadata  --dependency org.springframework:spring-webmvc:6.1.11

I get no results back

Example response:

> Task :dependencyInsight
No dependencies matching given input were found in configuration ':compileClasspath'

Now if I run with version 6.1.13, I can see that it does exist:

./gradlew dependencyInsight --configuration  compileClasspath  --dependency org.springframework:spring-webmvc:6.1.13                  

> Task :dependencyInsight
org.springframework:spring-webmvc:6.1.13 (selected by rule)
  Variant apiElements:
    | Attribute Name                     | Provided     | Requested    |
    |------------------------------------|--------------|--------------|
    | org.gradle.status                  | release      |              |
    | org.gradle.category                | library      | library      |
    | org.gradle.dependency.bundling     | external     | external     |
    | org.gradle.jvm.environment         | standard-jvm | standard-jvm |
    | org.gradle.jvm.version             | 17           | 21           |
    | org.gradle.libraryelements         | jar          | classes      |
    | org.gradle.usage                   | java-api     | java-api     |
    | org.jetbrains.kotlin.platform.type | jvm          | jvm          |

org.springframework:spring-webmvc:6.1.13
\--- org.springframework.boot:spring-boot-starter-web:3.3.4
     \--- compileClasspath (requested org.springframework.boot:spring-boot-starter-web)

org.springframework:spring-webmvc:5.3.31 -> 6.1.13
\--- org.springdoc:springdoc-openapi-webmvc-core:1.8.0
     \--- org.springdoc:springdoc-openapi-ui:1.8.0
          \--- compileClasspath

[bigdaz]

That's very strange: I don't really understand how both versions can be resolved in the compileClasspath configuration for the same project.

The best and simplest way to track down this dependency would be to generate a Build Scan. This will quickly point to the cause of a dependency being resolved. If you can share a Build Scan link, then I can investigate further. There might be a bug in the dependency-graph plugin, but I'd need a way to reproduce it.

If you can't generate a Build Scan, then you might want to try running the buildEnvironment task as described here to see if it reveals anything.

[pexa-slee]

here are the results of the scan

[pexa-slee]

Hi @bigdaz , have you had a chance to look at my previous comment yet?

[bigdaz]

Sorry, just back from vacations.
Thanks for the build-scan screenshots. They clearly demonstrate that there's a disconnect between what's being reported in the scans and the generated dependency graph. At this stage I don't have any explanation for this.

We might be able to get a better idea by inspecting the full dependency graph that is submitted to GitHub. Unfortunately the full transitive graph is not visible in the GitHub UI, so we'll need to inspect the report that is submitted.

You can generate the report by changing your workflow to include dependency-graph: generate-and-upload as demonstrated here. You should be able to then download the dependency-graph file as a workflow artifact. If you can share that I may be able deduce what's going on. You can either remove any sensitive information and share it here, or send it directly to [email protected].

[pexa-slee]

Not a problem @bigdaz

I just emailed you with the report.

[bigdaz]

@pexa-slee Thanks for sending the report. The file contains org.springframework:spring-webmvc:6.1.13 but contains no mention of version 6.1.11. This means that the old version has been submitted to GitHub as part of the dependency graph, but the version has not been overwritten by the newer version.

GitHub uses the 'correlator' value in the submitted file to determine if it should append or replace the existing dependency graph with a newly submitted one. You can read more about 'correlator' under 'Properties of job' here.

So I can see 2 ways that the old dependency graph containing spring-webmvc:6.1.11 is retained:

1. There is another job submitting a different dependency graph with a different correlator.
2. The correlator changed between submitting the job with '6.1.11' and '6.1.13'

The current job correlator is service_build_and_test-test, so if you renamed the workflow or job that could cause the correlator to change.

Unfortunately, GitHub doesn't provide any way to see the correlator that submitted a particular dependency. However, the GItHub engineers informed me that the old dependency graph should eventually "time out" when new graphs are being submitted, even if these have a different correlator value.

Some questions:

• Is spring-webmvc:6.1.11 still listed in your dependency graph?
• Are there any other jobs that may also be submitting a dependency graph to GitHub?

[lislei]

I'm not sure if I have the same issue. We have observed duplicate dependencies as well - #482.
@pexa-slee have you integrated the dependency-review workflow as well?