Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release devtooling metrics v0 ready for external testing #2808

Open
ccerv1 opened this issue Jan 18, 2025 · 0 comments
Open

Release devtooling metrics v0 ready for external testing #2808

ccerv1 opened this issue Jan 18, 2025 · 0 comments
Assignees
Labels
c:analytics Analytics and metrics

Comments

@ccerv1
Copy link
Member

ccerv1 commented Jan 18, 2025

What is it?

This meta-issue aims to expose a mart model via the API to support S7 Retro Funding metrics, focusing on developer utility and adoption, and ultimately weighted evaluation models for badgeholder voting.

Eventually, everything should be exposed via a single endpoint, eg:

oso_optimismRetroFundingS7MetricsDevTooling_v0:
- metricId
- projectId
- sampleDate
- amount
- unit

This model should include all relevant values for each metricId : projectId pairing. Their should also be a metric that reflects the weighting of all metrics so as to produce a pre-normalized percent of the reward allocation. In this context, pre-normalized means that you could take the value, multiply it by the OP Reward for the measurement period, remove projects below the OP Reward minimum cap, and then re-normalize and redistribute the total reward across projects. (The business logic for doing this is already built by Agora and run by them.)

We want the following categories of impact:

Part 1: Developer Utility

We will need the following intermediate models for developer utility inputs: a model that contains weighted dependency graphs between builders and devtools; at least two OpenRank models that weight the graph; simple pivot of metrics derived from the graph.

Weighted Dependency Graphs

  • onchain_builder_project_id
  • dev_tooling_project_id
  • edge_type
  • edge_weight
  • weighting_algorithm

OpenRank Models

There should be at least two weighting algorithms derived from OpenRank developer rank models. Essentially, we create a reputation score for onchain developers and then apply that to weight their interactions with a devtooling project repo.

Metrics

We should also be able to derive simple metrics for each devtooling project in the graph, eg:

  • dev_tooling_project_id
  • num_npm_dependents
  • num_rust_dependents
  • num_dev_connections
  • num_solidity_dev_connections
  • avg_open_rank1_utility_score
  • avg_open_rank2_utility_score

Part 2: Developer Adoption

We also need intermediate models for deriving two types of onchain metrics:

  • Velocity (impact over the past month)
  • Acceleration (net change in impact over the past month)

Velocity

  • monthly_gas_sum
  • monthly_transaction_count
  • monthly_active_address_count

Acceleration

  • monthly_gas_sum_change
  • monthly_transaction_count_change
  • monthly_active_address_count_change

Final Evaluation Algo Models

We also need four models that ingest the graph and upstream metrics, and assign weights. These are:

  • Arcturus (adoption heavy)
  • Bellatrix (growth heavy)
  • Canopus (importance heavy)
  • Blended (even weighting of all three)

It should be possible to import the weighting coefficients from a private source.

@ccerv1 ccerv1 added the c:analytics Analytics and metrics label Jan 18, 2025
@ccerv1 ccerv1 self-assigned this Jan 18, 2025
@ccerv1 ccerv1 added this to the [c] OP Dev Tooling Metrics milestone Jan 18, 2025
@ccerv1 ccerv1 added this to OSO Jan 18, 2025
@github-project-automation github-project-automation bot moved this to Backlog in OSO Jan 18, 2025
@ccerv1 ccerv1 moved this from Backlog to Up Next in OSO Jan 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c:analytics Analytics and metrics
Projects
Status: Up Next
Development

No branches or pull requests

1 participant