Reload configuation endpoint is throwing 403 no valid crumb error

[SteveMaglio]

Jenkins and plugins versions report

Environment

Jenkins: 2.375.4
OS: Linux - 5.15.0-1051-azure
---
ace-editor:1.1
active-directory:2.30
ant:481.v7b_09e538fcca
antisamy-markup-formatter:155.v795fb_8702324
apache-httpcomponents-client-4-api:4.5.14-150.v7a_b_9d17134a_5
authentication-tokens:1.4
bitbucket:223.vd12f2bca5430
blueocean-commons:1.27.1
blueocean-rest:1.27.1
bootstrap5-api:5.2.1-3
bouncycastle-api:2.27
branch-api:2.1071.v1a_188a_562481
build-pipeline-plugin:1.5.8
build-timeout:1.24
build-user-vars-plugin:1.9
caffeine-api:2.9.3-65.v6a_47d0f4d1fe
checks-api:1.8.1
cloudbees-folder:6.800.v71307ca_b_986b
command-launcher:90.v669d7ccb_7c31
commons-lang3-api:3.12.0-36.vd97de6465d5b_
commons-text-api:1.10.0-36.vc008c8fcda_7b_
conditional-buildstep:1.4.2
configuration-as-code:1625.v27444588cc3d
credentials:1214.v1de940103927
credentials-binding:523.vd859a_4b_122e6
data-tables-api:1.12.1-4
display-url-api:2.3.7
docker-commons:1.21
docker-workflow:563.vd5d2e5c4007f
durable-task:504.vb10d1ae5ba2f
echarts-api:5.4.0-1
email-ext:2.99
envinject:2.892.v25453b_80e595
envinject-api:1.199.v3ce31253ed13
font-awesome-api:6.2.1-1
generic-webhook-trigger:1.86.2
git:5.0.0
git-client:4.1.0
git-parameter:0.9.18
git-server:99.va_0826a_b_cdfa_d
github:1.36.1
github-api:1.303-417.ve35d9dd78549
github-branch-source:1701.v00cc8184df93
gradle:2.2
handlebars:1.1.1
instance-identity:142.v04572ca_5b_265
ionicons-api:45.vf54fca_5d2154
jackson2-api:2.15.1-344.v6eb_55303dc3e
jakarta-activation-api:2.0.1-3
jakarta-mail-api:2.0.1-3
javadoc:226.v71211feb_e7e9
javax-activation-api:1.2.0-6
javax-mail-api:1.6.2-9
jaxb:2.3.8-1
jdk-tool:63.v62d2fd4b_4793
jjwt-api:0.11.5-77.v646c772fddb_0
job-dsl:1.77
jquery:1.12.4-0
jquery-detached:1.2.1
jquery3-api:3.6.0-4
jsch:0.1.55.61.va_e9ee26616e7
junit:1202.v79a_986785076
kubernetes:3893.v73d36f3b_9103
kubernetes-client-api:6.4.1-215.v2ed17097a_8e9
kubernetes-credentials:0.10.0
kubernetes-credentials-provider:1.209.v862c6e5fb_1ef
ldap:659.v8ca_b_a_fe79fa_d
lockable-resources:1131.vb_7c3d377e723
mailer:457.v3f72cb_e015e5
matrix-auth:3.1.6
matrix-project:789.v57a_725b_63c79
maven-plugin:3.21
mercurial:1260.vdfb_723cdcc81
metrics:4.2.18-439.v86a_20b_a_8318b_
mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_
mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_
momentjs:1.1.1
okhttp-api:4.10.0-125.v3593b_a_f8c97b_
openshift-client:1.0.38
openshift-k8s-credentials:67.ve6d4b_4f61295
pam-auth:1.10
parameterized-trigger:2.45
permissive-script-security:0.6
pipeline-build-step:487.va_823138eee8b_
pipeline-github-lib:38.v445716ea_edda_
pipeline-graph-analysis:202.va_d268e64deb_3
pipeline-groovy-lib:629.vb_5627b_ee2104
pipeline-input-step:466.v6d0a_5df34f81
pipeline-milestone-step:111.v449306f708b_7
pipeline-model-api:2.2118.v31fd5b_9944b_5
pipeline-model-declarative-agent:1.1.1
pipeline-model-definition:2.2118.v31fd5b_9944b_5
pipeline-model-extensions:2.2118.v31fd5b_9944b_5
pipeline-rest-api:2.31
pipeline-stage-step:305.ve96d0205c1c6
pipeline-stage-tags-metadata:2.2118.v31fd5b_9944b_5
pipeline-stage-view:2.31
plain-credentials:143.v1b_df8b_d3b_e48
plugin-util-api:2.20.0
popper2-api:2.11.6-2
resource-disposer:0.20
role-strategy:587.588.v850a_20a_30162
run-condition:1.5
scm-api:676.v886669a_199a_a_
script-security:1251.vfe552ed55f8d
snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4
ssh-agent:295.v9ca_a_1c7cc3a_a_
ssh-credentials:305.v8f4381501156
ssh-slaves:2.877.v365f5eb_a_b_eec
sshd:3.322.v159e91f6a_550
structs:324.va_f5d6774f3a_d
swarm:3.39
timestamper:1.22
token-macro:359.vb_cde11682e0c
trilead-api:2.84.v72119de229b_7
variant:59.vf075fe829ccb
workflow-aggregator:590.v6a_d052e5a_a_b_5
workflow-api:1208.v0cc7c6e0da_9e
workflow-basic-steps:1010.vf7a_b_98e847c1
workflow-cps:3618.v13db_a_21f0fcf
workflow-cps-global-lib:609.vd95673f149b_b
workflow-durable-task-step:1234.v019404b_3832a
workflow-job:1316.vd2290d3341a_f
workflow-multibranch:733.v109046189126
workflow-scm-step:400.v6b_89a_1317c9a_
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:839.v35e2736cfd5c
ws-cleanup:0.44

What Operating System are you using (both controller, and any agents involved in the problem)?

our jenkins master (controller) image is on "Debian GNU/Linux 11 (bullseye)"

the azure agent running the job i am trying is running "rockylinux:8"

Reproduction steps

1. Have two jenkins servers, A and B. each is configured with an env var CASC_RELOAD_TOKEN set to the same value.
2. update jenkins-config.yaml on server B
3. use server A to run these curl commands to reload config on server B

            sh (script: """
            export COOKIE_JAR=/tmp/cookies
            export JENKINS_CRUMB=$(curl --silent --cookie-jar $COOKIE_JAR -u $JENKINS_USERNAME:$JENKINS_PASSWORD \"$SERVER_B_URL/crumbIssuer/api/json\" | jq -r \".crumb\")"

            curl -X POST --cookie $COOKIE_JAR -H Jenkins-Crumb:$JENKINS_CRUMB \"$SERVER_B_URL/reload-configuration-as-code/?casc-reload-token=$CASC_RELOAD_TOKEN\"
            """)

5. receive 403 error No Valid Crumb appears

HTTP ERROR 403 No valid crumb was included in the request

URI:	/reload-configuration-as-code/
STATUS:	403
MESSAGE:	No valid crumb was included in the request
SERVLET:	Stapler

---

Powered by Jetty:// 10.0.12

---

`

Expected Results

200 success, with the configuration on Server B being updated upon refreshing the page (i am using new sytemMessages to see the new config changes).

Actual Results

403 error no valid crumb found, and the config does not update. essentially, i cannot get the endpoint to work.

Anything else?

I would rather use this JCASC_RELOAD_TOKEN instead of an individual user API token because the project i am working on has dozens of servers, and it seems impossible to configure API tokens (this issue regarding preconfiguration is still open #1830)