Run benchmarks in test mode when not specifying verification functions in CI #48451
+510
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D67637754 |
rubennorte
added a commit
to rubennorte/react-native
that referenced
this pull request
Jan 2, 2025
…n functions in CI (facebook#48451) Summary: Changelog: [internal] Modifies the execution of benchmarks in CI to run benchmarks in test mode when they don't define a `verify` method. If a benchmark uses `verify`, the test is meant to make sure that the benchmark doesn't regress in CI. If it doesn't, then running the benchmark on CI doesn't provide much value. In that case, we run a single iteration of each test case just to make sure things don't break over time. Differential Revision: D67637754
rubennorte
force-pushed
the
export-D67637754
branch
from
f9c70ab to
2fc1fa7
Compare
rubennorte
added a commit
to rubennorte/react-native
that referenced
this pull request
Jan 3, 2025
…s in CI (facebook#48451) Summary: Changelog: [internal] Modifies the execution of benchmarks in CI to run benchmarks in test mode when they don't define a `verify` method. If a benchmark uses `verify`, the test is meant to make sure that the benchmark doesn't regress in CI. If it doesn't, then running the benchmark on CI doesn't provide much value. In that case, we run a single iteration of each test case just to make sure things don't break over time. Reviewed By: rshest Differential Revision: D67637754
rubennorte
added a commit
to rubennorte/react-native
that referenced
this pull request
Jan 3, 2025
…n functions in CI (facebook#48451) Summary: Changelog: [internal] Modifies the execution of benchmarks in CI to run benchmarks in test mode when they don't define a `verify` method. If a benchmark uses `verify`, the test is meant to make sure that the benchmark doesn't regress in CI. If it doesn't, then running the benchmark on CI doesn't provide much value. In that case, we run a single iteration of each test case just to make sure things don't break over time. Reviewed By: rshest Differential Revision: D67637754
|
This pull request was exported from Phabricator. Differential Revision: D67637754 |
rubennorte
force-pushed
the
export-D67637754
branch
from
2fc1fa7 to
526e0e0
Compare
rubennorte
added a commit
to rubennorte/react-native
that referenced
this pull request
Jan 4, 2025
…s in CI (facebook#48451) Summary: Changelog: [internal] Modifies the execution of benchmarks in CI to run benchmarks in test mode when they don't define a `verify` method. If a benchmark uses `verify`, the test is meant to make sure that the benchmark doesn't regress in CI. If it doesn't, then running the benchmark on CI doesn't provide much value. In that case, we run a single iteration of each test case just to make sure things don't break over time. Reviewed By: rshest Differential Revision: D67637754
rubennorte
added a commit
to rubennorte/react-native
that referenced
this pull request
Jan 4, 2025
…n functions in CI (facebook#48451) Summary: Changelog: [internal] Modifies the execution of benchmarks in CI to run benchmarks in test mode when they don't define a `verify` method. If a benchmark uses `verify`, the test is meant to make sure that the benchmark doesn't regress in CI. If it doesn't, then running the benchmark on CI doesn't provide much value. In that case, we run a single iteration of each test case just to make sure things don't break over time. Reviewed By: rshest Differential Revision: D67637754
|
This pull request was exported from Phabricator. Differential Revision: D67637754 |
Summary: Changelog: [internal] This implements a native module for Fantom to provide information about the CPU time used by the current process. This will be used by Fantom as the clock to run benchmarks more accurately. It provides 2 implementations: 1. One based on `clock_gettime` with `CLOCK_THREAD_CPUTIME_ID` that's available on Linux. This provides the CPU time for the current process with decent precision (tens of nanoseconds). 2. A fallback implementation that uses a monotonic clock (not actually CPU time). We can add a MacOS equivalent in a following diff. Reviewed By: rshest Differential Revision: D67596312
Summary: Changelog: [internal] Adds tinybench 3.1.0 (which has support for sync execution) and defined Flow types for the package. Reviewed By: dmytrorykun Differential Revision: D66698545
Summary:
Changelog: [internal]
Implements a basic API to run benchmarks with Fantom (using `tinybench` under the hood):
```
import {benchmark} from 'react-native/fantom';
benchmark
.suite('Suite name', {
// options
})
.add(
'Test name',
() => {
// code to benchmark
},
{
beforeAll: () => {},
beforeEach: () => {},
afterEach: () => {},
afterAll: () => {},
},
)
.verify(results => {
// check results and throw an error if the expectations fail
});
```
Features:
* Print benchmark results in the console as a table.
* It opts into optimized builds automatically
* Verifies that optimized build is used (unless manually opting out of the check via `disableOptimizedBuildCheck`).
* Supports verification of results (making expectations and making the test fail if the benchmark doesn't meet some expectations).
Reviewed By: rshest
Differential Revision: D66926183
…s in CI (facebook#48451) Summary: Changelog: [internal] Modifies the execution of benchmarks in CI to run benchmarks in test mode when they don't define a `verify` method. If a benchmark uses `verify`, the test is meant to make sure that the benchmark doesn't regress in CI. If it doesn't, then running the benchmark on CI doesn't provide much value. In that case, we run a single iteration of each test case just to make sure things don't break over time. Reviewed By: rshest Differential Revision: D67637754
rubennorte
added a commit
to rubennorte/react-native
that referenced
this pull request
Jan 6, 2025
…n functions in CI (facebook#48451) Summary: Changelog: [internal] Modifies the execution of benchmarks in CI to run benchmarks in test mode when they don't define a `verify` method. If a benchmark uses `verify`, the test is meant to make sure that the benchmark doesn't regress in CI. If it doesn't, then running the benchmark on CI doesn't provide much value. In that case, we run a single iteration of each test case just to make sure things don't break over time. Reviewed By: rshest Differential Revision: D67637754
rubennorte
force-pushed
the
export-D67637754
branch
from
526e0e0 to
db22a74
Compare
|
This pull request was exported from Phabricator. Differential Revision: D67637754 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Changelog: [internal]
Modifies the execution of benchmarks in CI to run benchmarks in test mode when they don't define a
verifymethod.If a benchmark uses
verify, the test is meant to make sure that the benchmark doesn't regress in CI. If it doesn't, then running the benchmark on CI doesn't provide much value. In that case, we run a single iteration of each test case just to make sure things don't break over time.Differential Revision: D67637754