Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse IRI links #57

Merged
merged 75 commits into from
May 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
55d358c
starting to work with #16
farooqkz Jan 10, 2024
ee0f76c
ihier part complete
farooqkz Jan 14, 2024
af08d6a
well done!
farooqkz Jan 14, 2024
79718c1
implementing IRIs still
farooqkz Jan 17, 2024
53979e9
pretty pretty code
farooqkz Jan 17, 2024
133294a
latest update
farooqkz Jan 18, 2024
fbb2652
up
farooqkz Jan 29, 2024
cf90786
some fixes
farooqkz Feb 10, 2024
ed94a50
fixed many issues
farooqkz Feb 11, 2024
87993c2
some more fixes
farooqkz Feb 11, 2024
b3a0f71
some fixes
farooqkz Feb 17, 2024
f8ba2fe
some more fixes
farooqkz Feb 17, 2024
8c225a1
many other fixes
farooqkz Feb 21, 2024
d492bf0
another fix
farooqkz Feb 21, 2024
28cbdf7
taking slices from input
farooqkz Feb 22, 2024
ee0611d
fix semicolon meaning
farooqkz Mar 5, 2024
e3ac2ca
Fixing tons of things...
farooqkz Mar 5, 2024
f4dbe80
fix some stuff
farooqkz Mar 12, 2024
9e4f742
IT'S COMPILING, MAN \:D/
farooqkz Mar 12, 2024
6f16d66
fix some problems with is_in_one_of_ranges and updating the code:
farooqkz Mar 12, 2024
aa6d26d
started refactoring
farooqkz Mar 12, 2024
1ee692c
refactor nearly complete
farooqkz Mar 16, 2024
d0e9ace
reduce number of stacks required by inlining ihier part
farooqkz Mar 16, 2024
9d05d8a
update
farooqkz Mar 16, 2024
54a9bde
fix little bug
farooqkz Mar 17, 2024
d024101
fix loop...
farooqkz Mar 17, 2024
ffe9d6e
remove not_link_part_char fn
farooqkz Apr 2, 2024
e36f1de
fix clippy issues
farooqkz Apr 2, 2024
d6fae27
some more fixes
farooqkz Apr 2, 2024
d6b7dbc
correction and improvements
farooqkz Apr 2, 2024
df2842d
passing some more tests
farooqkz Apr 3, 2024
c08fb66
passing some more more tests
farooqkz Apr 3, 2024
124764c
some modifications
farooqkz Apr 4, 2024
33efac2
fix some more tests
farooqkz Apr 8, 2024
2f4aa9a
fixed another testcase
farooqkz Apr 8, 2024
8fab2c2
remove wrong testcases
farooqkz Apr 8, 2024
0ae4d6f
fixing more testcases
farooqkz Apr 9, 2024
b49797e
trying to pass last testcases, 2 left
farooqkz Apr 22, 2024
d4ab53a
fixed clippy issues
farooqkz Apr 22, 2024
18677d2
overall enhancements. more tbd
farooqkz Apr 25, 2024
c6567ad
fixing tests
farooqkz Apr 25, 2024
3b9f466
reconstructing tests
farooqkz Apr 25, 2024
0e1d971
I think I've fixed tests
farooqkz Apr 25, 2024
fde44ea
major refactoring of iri parsing. more TBD
farooqkz Apr 27, 2024
78c70b6
refactoring, continued
farooqkz Apr 28, 2024
3d77f75
refactoring complete. now let's pass tests
farooqkz Apr 28, 2024
1a631f7
fixed all testcases
farooqkz Apr 28, 2024
eb79b92
change link_url to parse_link
farooqkz Apr 28, 2024
db25cad
new data for benchmark
farooqkz Apr 28, 2024
e335cc4
correct formatting
farooqkz Apr 28, 2024
ff6fdfe
add new test case for internal labelled links
farooqkz Apr 29, 2024
79316e9
fix #66
farooqkz Apr 29, 2024
d946113
new test cases and fix formatting
farooqkz Apr 29, 2024
94ad0d9
micro optimization by inlining
farooqkz Apr 30, 2024
29d3ff8
add new test cases
farooqkz Apr 30, 2024
9a8a6ea
add commments about parse_labelled method of LinkDestination
farooqkz Apr 30, 2024
a843a14
add benchmarking links
farooqkz Apr 30, 2024
bf909a1
correct formatting
farooqkz Apr 30, 2024
060f57c
a fix for rustdoc errors
farooqkz Apr 30, 2024
513876d
fix clippy issues
farooqkz Apr 30, 2024
b7fef4e
improve tests
farooqkz Apr 30, 2024
9638381
Fix another clippy thing
farooqkz Apr 30, 2024
305f6f9
Fix clippy issues
farooqkz Apr 30, 2024
829ba3d
fix formatting
farooqkz Apr 30, 2024
a0203f4
upgrade rust toolchain to 1.77.2
farooqkz Apr 30, 2024
abe9bec
don't parse internal markdown links. see #66 in the repo for details
farooqkz May 5, 2024
8bc5b05
fix conflict
farooqkz May 5, 2024
46fe115
move parenthesis, bracket and angle parsing into dedicated function
farooqkz May 5, 2024
7c6ff77
remove unused clippy ignore
farooqkz May 5, 2024
ef13980
fix clippy issues
farooqkz May 5, 2024
3c7726e
fix delimited link
Simon-Laux May 7, 2024
957f08f
cargo fmt
Simon-Laux May 7, 2024
63628d8
fix parenthesis in target of labeled link
Simon-Laux May 7, 2024
3f82aad
remove dead commented out code and update doc comment
Simon-Laux May 7, 2024
f58dac6
add links to link parsing standards to spec
Simon-Laux May 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
- uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: 1.64.0
toolchain: 1.77.2
override: true
- run: rustup component add rustfmt
- uses: actions-rs/cargo@v1
Expand All @@ -31,7 +31,7 @@ jobs:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: 1.64.0
toolchain: 1.77.2
components: clippy
override: true
- uses: actions-rs/clippy-check@v1
Expand Down Expand Up @@ -68,9 +68,9 @@ jobs:
matrix:
include:
- os: ubuntu-latest
rust: 1.64.0
rust: 1.77.2
- os: windows-latest
rust: 1.64.0
rust: 1.77.2
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@master
Expand Down
20 changes: 20 additions & 0 deletions benches/moar_links.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
Let's add some more links just for testing and benching:

these are some IPv6 links:

gopher://[::1]/
https://[::1]/سلام
https://[2345:0425:2CA1:0000:0000:0567:5673:23b5]/hello_world
https://[2345:425:2CA1:0:0:0567:5673:23b5]/hello_world

an IPvfuture link:
ftp://mrchickenkiller@[vA.A]/var/log/boot.log

some normal links:

https://www.ietf.org/rfc/rfc3987.txt
https://iamb.chat/messages/index.html
https://github.com/deltachat/message-parser/issues/67
https://far.chickenkiller.com
gopher://republic.circumlunar.space
https://far.chickenkiller.com/religion/a-god-who-does-not-care/
9 changes: 8 additions & 1 deletion benches/my_benchmark.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use deltachat_message_parser::parser::{parse_desktop_set, parse_markdown_text, parse_only_text};
use deltachat_message_parser::parser::{
parse_desktop_set, parse_markdown_text, parse_only_text, LinkDestination,
};

pub fn criterion_benchmark(c: &mut Criterion) {
let testdata = include_str!("testdata.md");
let lorem_ipsum_txt = include_str!("lorem_ipsum.txt");
let r10s_update_message = include_str!("r10s_update_message.txt");
let links = include_str!("moar_links.txt");

c.bench_function("only_text_lorem_ipsum.txt", |b| {
b.iter(|| parse_only_text(black_box(lorem_ipsum_txt)))
Expand Down Expand Up @@ -35,6 +38,10 @@ pub fn criterion_benchmark(c: &mut Criterion) {
c.bench_function("markdown_r10s_update_message.txt", |b| {
b.iter(|| parse_markdown_text(black_box(r10s_update_message)))
});

c.bench_function("parse_link_moar_links.txt", |b| {
b.iter(|| LinkDestination::parse(black_box(links)))
});
}

criterion_group!(benches, criterion_benchmark);
Expand Down
760 changes: 760 additions & 0 deletions benches/testdata.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion message_parser_wasm/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ pub fn parse_text(s: &str, enable_markdown: bool) -> JsValue {
serde_wasm_bindgen::to_value(&ast).expect("Element converts to JsValue")
}

/// parses text to json AST (text elements and labled links, to replicate current desktop implementation)
/// parses text to json AST (text elements and labeled links, to replicate current desktop implementation)
#[wasm_bindgen]
pub fn parse_desktop_set(s: &str) -> JsValue {
serde_wasm_bindgen::to_value(&deltachat_message_parser::parser::parse_desktop_set(s))
Expand Down
2 changes: 1 addition & 1 deletion rust-toolchain
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.64.0
1.77.2
1 change: 1 addition & 0 deletions spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ Make email addresses clickable, opens the chat with that contact and creates it
Make URLs clickable.

- detect all valid hyperlink URLs that have the `://` (protocol://host).
- according to [RFC3987](https://www.rfc-editor.org/rfc/rfc3987) and [RFC3988](https://www.rfc-editor.org/rfc/rfc3988)

- other links like `mailto:` (note there is just a single `:`, no `://`) will get separate parsing that includes a whitelisted protocol name, otherwise there will likely be unexpected behavior if user types `hello:world` - will be recognized as link.

Expand Down
2 changes: 1 addition & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
clippy::get_last_with_len,
clippy::get_unwrap,
clippy::get_unwrap,
clippy::integer_arithmetic,
clippy::arithmetic_side_effects,
clippy::match_on_vec_items,
clippy::match_wild_err_arm,
clippy::missing_panics_doc,
Expand Down
Loading
Loading