Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add @rrweb/cutter and a Sync Replayer #1149

Draft
wants to merge 62 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
d17b08f
refactor: eliminate eslint errors as many as I can
YunFeng0817 Jul 3, 2022
be1aaa7
refactor: fix more eslint errors in the record module
YunFeng0817 Jul 3, 2022
2aa4e86
feat: define an interface of session cutter
YunFeng0817 Jul 3, 2022
dca528d
feat: implement a SyncReplayer and add session cut interface
YunFeng0817 Jul 3, 2022
fcb5446
feat: add a rrdom snapshot method
YunFeng0817 Jul 3, 2022
b6a6418
feat: finish a prototype of session cutter
YunFeng0817 Jul 3, 2022
2a7d769
fix: some inline css styles can't be generated by the cutter
YunFeng0817 Jul 3, 2022
6424d9b
refactor: move session-cutter from rrdom to rrweb-cutter
YunFeng0817 Jul 6, 2022
715bc78
test: add integration test case for sync replayer
YunFeng0817 Jul 10, 2022
fb29f20
Merge branch 'master' into trim
YunFeng0817 Jul 11, 2022
8530316
test: fix the broken unit test
YunFeng0817 Jul 11, 2022
bb20773
Merge branch 'master' into trim
YunFeng0817 Aug 10, 2022
d2af66c
Merge branch 'master' of https://github.com/rrweb-io/rrweb into trim
Juice10 Sep 27, 2022
cafe594
Apply formatting changes
Juice10 Sep 27, 2022
5f442b6
add pruneBranches
Juice10 Sep 27, 2022
247edce
use mutations to construct the pruneBranches tree as well
Juice10 Sep 28, 2022
b29b5df
Apply formatting changes
Juice10 Sep 28, 2022
ecc6332
Isolate tests by making sure inlineStyleEvents is unique each time
Juice10 Nov 10, 2022
b981c5c
Decrease the number of iterations needed
Juice10 Nov 10, 2022
5d3c905
Prune now also support moved nodes
Juice10 Nov 10, 2022
10b6c6d
Add `yarn prune-events` script.
Juice10 Nov 10, 2022
7f26979
Merge branch 'master' into trim
YunFeng0817 Feb 13, 2023
9605839
update dependencies and restore changes to rrdom
YunFeng0817 Feb 13, 2023
1caade6
Apply formatting changes
YunFeng0817 Feb 13, 2023
0164770
update sync-replayer with the new version of Replayer
YunFeng0817 Feb 13, 2023
e209748
fix import and upgrade jest version
YunFeng0817 Feb 13, 2023
c185364
fix failed tests
YunFeng0817 Feb 13, 2023
efdcbe8
Apply formatting changes
YunFeng0817 Feb 13, 2023
a490b2d
export SyncReplayer to entry/all.ts
YunFeng0817 Feb 13, 2023
0d8d0c6
Apply formatting changes
YunFeng0817 Feb 13, 2023
3297188
downgrade jest
YunFeng0817 Feb 13, 2023
25175cb
Merge branch 'master' into trim
YunFeng0817 Feb 14, 2023
e04aa23
rename rrweb-cutter
YunFeng0817 Feb 14, 2023
f27fed1
upgrade vite version for web extension
YunFeng0817 Feb 14, 2023
d75cb4f
Merge branch 'master' into trim
YunFeng0817 Feb 15, 2023
60c1f7a
improve code styles and quality
YunFeng0817 Feb 15, 2023
aabec36
add support for adoptedStylesheets
YunFeng0817 Feb 16, 2023
438df99
refactor: split index.ts into multiple files
YunFeng0817 Feb 17, 2023
7454590
add support for many incremental event type
YunFeng0817 Feb 20, 2023
67e58bb
handle custom events
YunFeng0817 Feb 20, 2023
1810f48
enable to build snapshot for iframes
YunFeng0817 Feb 20, 2023
3335897
refactor unit tests
YunFeng0817 Feb 20, 2023
ca3bec8
add support for sequential ID plugin
YunFeng0817 Feb 21, 2023
6318e26
update test case and fix bugs
YunFeng0817 Feb 21, 2023
f244127
add change log
YunFeng0817 Feb 21, 2023
cea319e
Merge branch 'master' into trim
YunFeng0817 Apr 10, 2023
09881bb
apply the change of https://github.com/rrweb-io/rrweb/pull/1145 to Sy…
YunFeng0817 Apr 10, 2023
16c6393
update rrweb dependencies
YunFeng0817 Apr 10, 2023
20efefb
add test cases for meta events and input events
YunFeng0817 Apr 24, 2023
7a23565
Merge branch 'master' into trim
YunFeng0817 Apr 24, 2023
f6e09c4
fix failed test cases and fix one bug in the session cutter
YunFeng0817 Apr 24, 2023
8aa4bb6
add test cases for cutting events with multiple meta events
YunFeng0817 Apr 24, 2023
27f6a2e
add test case for scroll events and fix a related bug
YunFeng0817 May 2, 2023
3115b1c
Merge branch 'master' into trim
YunFeng0817 May 2, 2023
4d682e8
update browser list db
YunFeng0817 May 2, 2023
73af8d6
init demo
YunFeng0817 Jun 14, 2023
0658578
Merge branch 'master' into trim
YunFeng0817 Dec 12, 2024
2c92fa3
Apply formatting changes
YunFeng0817 Dec 12, 2024
627cedb
upgrade cutter project setup
YunFeng0817 Dec 12, 2024
4327db3
restore
YunFeng0817 Dec 12, 2024
26db372
Apply formatting changes
YunFeng0817 Dec 12, 2024
491a175
adapt new changes of replayer to sync-replayer except media manager a…
YunFeng0817 Jan 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .changeset/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
"@rrweb/rrweb-plugin-sequential-id-record",
"@rrweb/rrweb-plugin-sequential-id-replay",
"@rrweb/rrweb-plugin-canvas-webrtc-record",
"@rrweb/rrweb-plugin-canvas-webrtc-replay"
"@rrweb/rrweb-plugin-canvas-webrtc-replay",
"@rrweb/cutter"
]
],
"linked": [],
Expand Down
8 changes: 8 additions & 0 deletions .changeset/gorgeous-eyes-agree.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
"@rrweb/cutter": patch
---

Feat: Add two tools to process the recording data

1. `session cutter` is a tool to cut the recording data into smaller pieces.
2. `pruneBranch` is a tool to prune branches from Dom tree, keeping only the nodes of interest.
5 changes: 5 additions & 0 deletions .changeset/popular-seals-buy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"rrweb": patch
---

Feat: Add a Sync Replayer which can be executed in an RRDom environment.
5 changes: 4 additions & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,11 @@ body:
options:
- rrweb
- rrweb-snapshot
- rrdom
- rrweb-player
- @rrweb/types
- rrweb-cutter
- rrdom
- rrdom-nodejs
- web-extension
- rrvideo
- Other (specify below)
Expand Down
5 changes: 4 additions & 1 deletion .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,11 @@ body:
options:
- rrweb
- rrweb-snapshot
- rrdom
- rrweb-player
- @rrweb/types
- rrweb-cutter
- rrdom
- rrdom-nodejs
- web-extension
- rrvideo
- Other (specify below)
Expand Down
4 changes: 4 additions & 0 deletions .vscode/rrweb-monorepo.code-workspace
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@
{
"name": "@rrweb/rrweb-plugin-canvas-webrtc-replay",
"path": "../packages/plugins/rrweb-plugin-canvas-webrtc-replay"
},
{
"name": "@rrweb/cutter (package)",
"path": "../packages/rrweb-cutter"
}
],
"settings": {
Expand Down
14 changes: 0 additions & 14 deletions packages/rrdom/jest.config.js

This file was deleted.

3 changes: 3 additions & 0 deletions packages/rrweb-cutter/.eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
vite.config.ts
jest.config.js
test
Empty file added packages/rrweb-cutter/README.md
Empty file.
Empty file.
2 changes: 2 additions & 0 deletions packages/rrweb-cutter/examples/server/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
yarn.lock
event.json
Empty file.
22 changes: 22 additions & 0 deletions packages/rrweb-cutter/examples/server/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "server",
"version": "1.0.0",
"description": "An example package of using rrweb-cutter with express",
"main": "dist/server.js",
"scripts": {
"start": "node dist/server.js",
"build": "tsc",
"dev": "nodemon src/index.ts"
},
"dependencies": {
"@rrweb-cutter": "../../",
"express": "^4.18.2"
},
"devDependencies": {
"@rrweb/types": "../../../types",
"@types/express": "^4.17.17",
"nodemon": "^2.0.22",
"ts-node": "^10.9.1",
"typescript": "^4.7.3"
}
}
25 changes: 25 additions & 0 deletions packages/rrweb-cutter/examples/server/src/client.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<!DOCTYPE html>
<html>

<head>
<meta charset="UTF-8">
<title>Replaying rrweb events after time</title>
<script src="https://cdn.jsdelivr.net/npm/rrweb/dist/rrweb.min.js"></script>
</head>

<body>
<div id="replay-container"></div>
<script>
const timepoint = Date.now() - 60000; // Fetch events that occurred in the last minute
fetch(`/rrweb-events-after-time/${timepoint}`)
.then(response => response.json())
.then(events => {
const replayer = new rrweb.Replayer(events, {
root: document.getElementById('replay-container'),
});
replayer.play();
});
</script>
</body>

</html>
28 changes: 28 additions & 0 deletions packages/rrweb-cutter/examples/server/src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import * as fs from 'fs';
import express, { Request, Response } from 'express';
import { eventWithTime } from '@rrweb/types';
import { cutSession } from '@rrweb/cutter';

const app = express();

// Sample events load from file.
// You can replace this with your own events loaded from other places (DataBase, Network, etc).
const events: eventWithTime[] = JSON.parse(
fs.readFileSync('./events1.json').toString(),
);

app.get('/events/:timepoint', (req: Request, res: Response) => {
const timepoint = parseInt(req.params.timepoint);
const sessions = cutSession(events, {
points: [timepoint],
});
if (sessions.length < 2)
return res.json({
error: 'Invalid timepoint',
});
return res.json(sessions[1].events);
});

app.listen(3000, () => {
console.log('Server started on port 3000');
});
25 changes: 25 additions & 0 deletions packages/rrweb-cutter/examples/server/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"compilerOptions": {
"rootDir": "src",
"outDir": "dist",
"declaration": true,
"declarationDir": "dist/types",
"target": "ES5",
"module": "commonjs",
"lib": ["ES6", "DOM"],
"moduleResolution": "Node",
"sourceMap": true,
"strictNullChecks": true,
"removeComments": true,
"preserveConstEnums": true,
"resolveJsonModule": true,
"isolatedModules": false,
"esModuleInterop": true,
"noImplicitReturns": true,
"noImplicitAny": true,
"skipLibCheck": true
},
"compileOnSave": true,
"include": ["src"],
"exclude": ["test"]
}
53 changes: 53 additions & 0 deletions packages/rrweb-cutter/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
{
"name": "@rrweb/cutter",
"version": "2.0.0-alpha.18",
"publishConfig": {
"access": "public"
},
"scripts": {
"dev": "vite build --watch",
"build": "yarn turbo run prepublish",
"check-types": "tsc -noEmit",
"test": "vitest run",
"prepublish": "tsc -noEmit && vite build",
"lint": "yarn eslint src/**/*.ts",
"prune-events": "npx vite-node scripts/prune.ts --"
},
"homepage": "https://github.com/rrweb-io/rrweb/tree/master/packages/rrweb-cutter#README.md",
"bugs": {
"url": "https://github.com/rrweb-io/rrweb/issues"
},
"repository": {
"type": "git",
"url": "git+https://github.com/rrweb-io/rrweb.git"
},
"author": "[email protected]",
"keywords": [
"rrweb",
"rrdom",
"session cutter"
],
"license": "MIT",
"type": "module",
"main": "dist/index.cjs",
"unpkg": "dist/index.iife.js",
"module": "dist/index.js",
"typings": "dist/index.d.ts",
"files": [
"dist",
"package.json"
],
"devDependencies": {
"@types/lodash.clonedeep": "^4.5.9",
"typescript": "^5.4.5",
"vite": "^5.3.1",
"vite-plugin-dts": "^3.9.1"
},
"dependencies": {
"@rrweb/types": "^2.0.0-alpha.18",
"lodash.clonedeep": "^4.5.0",
"rrdom": "^2.0.0-alpha.18",
"rrweb-snapshot": "^2.0.0-alpha.18",
"rrweb": "^2.0.0-alpha.18"
}
}
13 changes: 13 additions & 0 deletions packages/rrweb-cutter/scripts/prune.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import fs from 'fs';
import { pruneBranches } from '../src';

const myArgs = process.argv.slice(2);
if (myArgs.length < 3) {
console.log('Usage: yarn prune-events <input> <output> <id> (<id> ...)');
process.exit(1);
}

const [input, output, ...ids] = myArgs;
const events = JSON.parse(fs.readFileSync(input, 'utf8'));
const result = pruneBranches(events, { keep: ids.map((id) => parseInt(id)) });
fs.writeFileSync(output, JSON.stringify(result));
Loading
Loading