[Tracking] Context Propagation

[thisthat]

Goal

Propagate the W3C Trace Context and other metadata across promotion stages via KeptnApp CRDs.

Technical Details

I want to Propagate W3C trace-id across KeptnApp.
Propagate context through the deployment.
Any possible refactor: e.g., split KeptnApp release-matrix from pre-post tasks.

DoD

• I have a KeptnApp deployed in prod, I want to trace it back from the version that was deployed in dev, passing through hardening.
• I want to have pre-post tasks added after I started with an auto-discovered KeptnApp
• I want to attach metadata to the deployment of a KeptnApp and also customize that per-workload. This metadata must be:
  • visible in the Spans as span/event attributes.
  • passed to KeptnTasks and KeptnEvaluations at both: pre/post App and Workload levels.

---

Example:

kind: KeptnAppContext
name: "my-awesome-app" # this has to be the same value of part-of label
spec:
  preDeploymentTasks:
      - task1
      - task2
  postDeploymentTasks:
      - task1
      - task2
  preDeploymentEvaluations:
      - eval1
      - eval2
  postDeploymentEvaluations:
      - eval1
      - eval2
  metadata:
      - key: value
      - ...
  traceLinks:
    - "w3c trace id1"
    - "w3c trace id2"
--- 
kind: Deployment
spec:
 template:
   annotations:
     traceparent: "w3c parent trace"
     keptn.sh/metadata: "myMetadataKey1=myValue1,myMetadata2=myvalue2,..."
     k8s.io/part-of: my-awesome-app
     keptn.sh/pre-task: mytask1

Problems:

1. Having a CR with the context could create problems because of:

• apply order: we could start taking care of a Workload before the Ctx CR is applied
• different workloads might have different metadata: e.g. gitSHA could differ from each workload
• Clean-up of old data is tricky since we would need to either edit the same ctx CR or track old versions

2. Precedence of context between App and Workload

Possible solutions:

1. Add metadata information to the Workload directly via annotations

   • metadata is customizable per-workload
   • each workload could get linked to a different parent trace if they are bumped individually across stages
   • we maintain 1 writer per-CR

2. in the case of the same key, workload def wins

KEP

• KEP 87: Lifecycle Toolkit - Context Propagation for GitOps enhancement-proposals#89
• KEP 89: Lifecycle Toolkit - Context Information in Keptn Tasks  enhancement-proposals#91

Tasks

Preview Give feedback

1. Introduce v1beta1 lifecycle API #2573
   chore lifecycle-operator +2
   
2. Introduce KeptnAppContext CRD #2574
   +0
   
3. Use v1beta1 API in lifecycle-operator controllers #2575
   chore +1
   
4. Remove Pre/Post Deployment Tasks/Evaluations from KeptnApp in v1beta1 #2576
   lifecycle-operator +1
   
5. Add support for linked traces for KeptnAppVersion #2577
   lifecycle-operator +1
   
6. Propagate Context Metadata to KeptnAppVersion #2578
   +0
   
7. Apply metadata from pod annotations to KeptnWorkloadVersion #2579
   +0
   
8. Think about K8s default labels or used by other CNCF projects for metadata
9. Propagate and merge Context Metadata to KeptnWorkloadVersion #2580
   +0
   
10. Propagate Context Metadata and traceParent of current phase to KeptnTask containers #2581
    +0
    
11. Document how users should switch to KeptnAppContext from manually created KeptnApp #2582
    documentation on hold +2
    
12. Migration guide for KeptnAppContext #2810
    documentation +1
    
13. Create separate docs page from context section and enhance with new KeptnContext info #2813
    documentation +1
    
14. Create guide that shows how to link traces between different KeptnApp deployments #2814
    documentation +1
    
15. Update examples, kubebuilder sample files, getting started guides with new context features #2815
    documentation +1
    
16. CRD reference page for KeptnAppContext #2836
    documentation +1
    
17. Update auto app discovery and restartable app guides with new app context information #2869
    documentation +1
    
18. Update day2 operations to use KeptnAppContext #2883
    documentation +1
    
19. Update task and evaluation guides to use KeptnAppContext #2884
    documentation +1
    
20. Adapt KeptnTask CRD reference page #2895
    documentation +1
    
21. Adapt https://keptn.sh/latest/docs/components/lifecycle-operator/keptn-apps/ page according to new epic feature
22. add Xrefs between context pages
23. Docs: update links to CRD v1beta1
24. Create script for converting v1alpha3 KeptnApp to v1beta1 KeptnAppContext/KeptnApp

[heckelmann]

An additional thought on that:

If it would be possible to add an annotation to the deployment manifest like keptn.sh/metadata where I can store additional information (like a commit ID) which is propagated as an environment variable to the tasks, would enable me to use this information to send events to my observability platform and could also be used in the end-2-end trace.