Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ct-metrics): BPF implementation #1102

Merged
merged 16 commits into from
Dec 13, 2024
Merged

Conversation

SRodi
Copy link
Member

@SRodi SRodi commented Dec 5, 2024

Description

BPF implementation for connection tracking metrics. This is the data-plane work mentioned in this comment #1057 (comment)

Summary

  • feature flag enableConntrackMetrics
  • counters incremented within IFDEF in BPF
  • counters: packets forward/reply + bytes forward/reply
  • conntrack metadata includes metrics and is added to packets struct
  • add/update unit tests for conntrack_linux and packetparser_linux

Related Issue

#806

Checklist

  • I have read the contributing documentation.
  • I signed and signed-off the commits (git commit -S -s ...). See this documentation on signing commits.
  • I have correctly attributed the author(s) of the code.
  • I have tested the changes locally.
  • I have followed the project's style guidelines.
  • I have updated the documentation, if necessary.
  • I have added tests, if applicable.

Screenshots (if applicable) or Testing Completed

Please add any relevant screenshots or GIFs to showcase the changes made.

  1. enableConntrackMetrics=false

    # bpftool map dump id 994 -j | jq -r .[0]
    {
      "key": [
        ...
      ],
      "value": [
        ...
      ],
      "formatted": {
        "key": {
          ...
        },
        "value": {
          ...
          "conntrack_metadata": {
            "bytes_forward_count": 0,
            "bytes_reply_count": 0,
            "packets_forward_count": 0,
            "packets_reply_count": 0
          }
        }
      }
    }
  2. enableConntrackMetrics=true

    # bpftool map dump id 1019 -j | jq -r .[0]
    {
      "key": [
        ...
      ],
      "value": [
        ...
      ],
      "formatted": {
        "key": {
          ...
        },
        "value": {
          ...,
          "conntrack_metadata": {
            "bytes_forward_count": 13440,
            "bytes_reply_count": 56335,
            "packets_forward_count": 56,
            "packets_reply_count": 43
          }
        }
      }
    }

    At userland level I provisionally added a debug statement, just for this test, in packetparser_linux.go (without IP and proto translation)

    ❯ k logs -n kube-system retina-agent-chvdh | head -n 10 | grep metadata
    Defaulted container "retina" out of: retina, init-retina (init)
    ts=2024-12-13T10:37:08.881Z level=debug caller=packetparser/packetparser_linux.go:577 msg="Conntrack metadata" SrcIp=788657162 DstIp=2499867658 SrcPort=19117 DstPort=23313 Proto=6 PacketsForwardCount=73 PacketsReplyCount=83 BytesForwardCount=16068 BytesReplyCount=6936
    ts=2024-12-13T10:37:08.881Z level=debug caller=packetparser/packetparser_linux.go:577 msg="Conntrack metadata" SrcIp=788657162 DstIp=2499867658 SrcPort=19117 DstPort=23313 Proto=6 PacketsForwardCount=73 PacketsReplyCount=82 BytesForwardCount=16068 BytesReplyCount=6870

Additional Notes

Add any additional notes or context about the pull request here.


Please refer to the CONTRIBUTING.md file for more information on how to contribute to this project.

@SRodi SRodi added type/enhancement New feature or request area/ebpf lang/c The C Programming Language scope/M Change is Medium labels Dec 5, 2024
@SRodi SRodi self-assigned this Dec 5, 2024
@SRodi SRodi requested a review from a team as a code owner December 5, 2024 13:21
pkg/plugin/conntrack/_cprog/conntrack.c Outdated Show resolved Hide resolved
pkg/plugin/conntrack/_cprog/conntrack.c Show resolved Hide resolved
pkg/plugin/conntrack/_cprog/conntrack.c Outdated Show resolved Hide resolved
@vakalapa vakalapa requested a review from Copilot December 6, 2024 19:51

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 8 out of 10 changed files in this pull request and generated no suggestions.

Files not reviewed (2)
  • pkg/plugin/conntrack/_cprog/conntrack.c: Language not supported
  • pkg/plugin/packetparser/_cprog/packetparser.c: Language not supported
@SRodi SRodi force-pushed the ct/metrics-bpf branch 2 times, most recently from bdbcd8f to 1031c3f Compare December 9, 2024 16:09

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 8 out of 17 changed files in this pull request and generated no suggestions.

Files not reviewed (9)
  • pkg/plugin/conntrack/_cprog/conntrack.c: Language not supported
  • pkg/plugin/conntrack/_cprog/dynamic.h: Language not supported
  • pkg/plugin/packetparser/_cprog/dynamic.h: Language not supported
  • pkg/plugin/packetparser/_cprog/packetparser.c: Language not supported
  • pkg/plugin/packetparser/packetparser_bpfel_x86.go: Evaluated as low risk
  • pkg/plugin/packetparser/packetparser_linux.go: Evaluated as low risk
  • pkg/config/config.go: Evaluated as low risk
  • deploy/legacy/manifests/controller/helm/retina/templates/configmap.yaml: Evaluated as low risk
  • deploy/hubble/manifests/controller/helm/retina/templates/agent/configmap.yaml: Evaluated as low risk
@SRodi SRodi requested a review from ibezrukavyi December 9, 2024 17:09
@vakalapa
Copy link
Contributor

Overall looks good to me, punting to @nddq to review the conntrack pieces.

@SRodi SRodi requested a review from Copilot December 10, 2024 15:51

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 7 out of 19 changed files in this pull request and generated no suggestions.

Files not reviewed (12)
  • pkg/plugin/conntrack/_cprog/conntrack.c: Language not supported
  • pkg/plugin/conntrack/_cprog/dynamic.h: Language not supported
  • pkg/plugin/packetparser/_cprog/dynamic.h: Language not supported
  • pkg/plugin/packetparser/_cprog/packetparser.c: Language not supported
  • pkg/plugin/packetparser/packetparser_bpfel_arm64.go: Evaluated as low risk
  • pkg/plugin/packetparser/packetparser_linux.go: Evaluated as low risk
  • pkg/plugin/packetparser/packetparser_bpfel_x86.go: Evaluated as low risk
  • pkg/config/config.go: Evaluated as low risk
  • deploy/hubble/manifests/controller/helm/retina/templates/agent/configmap.yaml: Evaluated as low risk
  • pkg/plugin/conntrack/conntrack_bpfel_x86.go: Evaluated as low risk
  • deploy/legacy/manifests/controller/helm/retina/templates/configmap.yaml: Evaluated as low risk
  • pkg/plugin/conntrack/conntrack_bpfel_arm64.go: Evaluated as low risk
ibezrukavyi
ibezrukavyi previously approved these changes Dec 10, 2024
pkg/plugin/conntrack/_cprog/conntrack.c Outdated Show resolved Hide resolved
pkg/plugin/conntrack/_cprog/conntrack.c Outdated Show resolved Hide resolved
pkg/plugin/conntrack/_cprog/conntrack.c Show resolved Hide resolved
@SRodi SRodi requested review from rectified95 and nddq December 11, 2024 11:45
@SRodi SRodi requested a review from nddq December 13, 2024 10:46
@nddq nddq added this pull request to the merge queue Dec 13, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 13, 2024
@nddq nddq added this pull request to the merge queue Dec 13, 2024
Merged via the queue into microsoft:main with commit 47ccd8b Dec 13, 2024
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ebpf lang/c The C Programming Language scope/M Change is Medium type/enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants