futures_unordered_driver.rs already borrowed: BorrowMutError (0.307.0)

[louis030195]

code is here https://github.com/mediar-ai/screenpipe/blob/pipe-logs/screenpipe-core/src/pipes.rs

https://github.com/mediar-ai/screenpipe/blob/4ff7d7d4b81a2f9ca54abc5f40b63f3169f8f770/screenpipe-server/src/bin/screenpipe-server.rs#L474

js code: https://github.com/mediar-ai/screenpipe/blob/pipe-logs/examples/typescript/pipe-stream-ocr-text/main.js

is that the right way to run multiple JS/TS file in parallel?

error is nondeterministic, sometimes it runs sometimes it crashes

not sure if i'm doing something wrong or it's a bug

[louis030195]

and https://github.com/mediar-ai/screenpipe/blob/pipe-logs/screenpipe-core/src/deno/runtime.js

i noticed if i comment the sendLogs it stops ocurring

https://github.com/mediar-ai/screenpipe/blob/4ff7d7d4b81a2f9ca54abc5f40b63f3169f8f770/screenpipe-core/src/deno/runtime.js#L34

as if there is some limit of promise or something or spamming too much deno?

[littledivy]

Can you provide the full backtrace with RUST_BACKTRACE=1? Also https://github.com/mediar-ai/screenpipe/blob/pipe-logs/examples/typescript/pipe-stream-ocr-text/main.js is 404

[louis030195]

2024-09-26T02:33:55.616924Z  INFO screenpipe_server::server: received search request: query='', content_type=OCR, limit=100, offset=0, start_time=Some(202
4-09-26T02:32:55.615Z), end_time=Some(2024-09-26T02:33:55.615Z), app_name=None, window_name=None, min_length=None, max_length=None
2024-09-26T02:33:59.436136Z  INFO screenpipe_server::video: Starting new video capture
2024-09-26T02:33:59.436163Z  INFO screenpipe_server::video: Started capture thread
2024-09-26T02:34:00.021492Z  INFO screenpipe_server::video: Starting FFmpeg process for file: /Users/louisbeaumont/.screenpipe/data/monitor_1_2024-09-26_0
2-34-00.mp4
2024-09-26T02:34:00.951694Z  WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="SELECT COUNT(*) FROM ocr_text …" db.statem
ent="\n\nSELECT\n  COUNT(*)\nFROM\n  ocr_text\n  JOIN frames ON ocr_text.frame_id = frames.id\nWHERE\n  (\n    ?1 = ''\n    OR ocr_text.text LIKE '%' || ?
1 || '%' COLLATE NOCASE\n  )\n  AND ocr_text.text != 'No text found'\n  AND (\n    ?2 IS NULL\n    OR frames.timestamp >= ?2\n  )\n  AND (\n    ?3 IS NULL
\n    OR frames.timestamp <= ?3\n  )\n  AND (\n    ?4 IS NULL\n    OR LENGTH(ocr_text.text) >= ?4\n  )\n  AND (\n    ?5 IS NULL\n    OR LENGTH(ocr_text.te
xt) <= ?5\n  )\n  AND (\n    ?6 IS NULL\n    OR ocr_text.app_name LIKE '%' || ?6 || '%' COLLATE NOCASE\n  )\n  AND (\n    ?7 IS NULL\n    OR ocr_text.wind
ow_name LIKE '%' || ?7 || '%' COLLATE NOCASE\n  )\n" rows_affected=0 rows_returned=1 elapsed=5.334521083s elapsed_secs=5.334521083 slow_threshold=1s
2024-09-26T02:34:00.951734Z  WARN sqlx::query: slow statement: execution time exceeded alert threshold summary="SELECT ocr_text.frame_id, ocr_text.text as
 …" db.statement="\n\nSELECT\n  ocr_text.frame_id,\n  ocr_text.text as ocr_text,\n  ocr_text.text_json,\n  frames.timestamp,\n  video_chunks.file_path,\n
 frames.offset_index,\n  ocr_text.app_name,\n  ocr_text.ocr_engine,\n  ocr_text.window_name,\n  GROUP_CONCAT(tags.name, ',') as tags\nFROM\n  ocr_text\n
JOIN frames ON ocr_text.frame_id = frames.id\n  JOIN video_chunks ON frames.video_chunk_id = video_chunks.id\n  LEFT JOIN vision_tags ON frames.id = visio
n_tags.vision_id\n  LEFT JOIN tags ON vision_tags.tag_id = tags.id\nWHERE\n  (\n    ?1 = ''\n    OR ocr_text.text LIKE '%' || ?1 || '%' COLLATE NOCASE\n
)\n  AND ocr_text.text != 'No text found'\n  AND (\n    ?2 IS NULL\n    OR frames.timestamp >= ?2\n  )\n  AND (\n    ?3 IS NULL\n    OR frames.timestamp <
= ?3\n  )\n  AND (\n    ?4 IS NULL\n    OR LENGTH(ocr_text.text) >= ?4\n  )\n  AND (\n    ?5 IS NULL\n    OR LENGTH(ocr_text.text) <= ?5\n  )\n  AND (\n
  ?6 IS NULL\n    OR ocr_text.app_name LIKE '%' || ?6 || '%' COLLATE NOCASE\n  )\n  AND (\n    ?7 IS NULL\n    OR ocr_text.window_name LIKE '%' || ?7 || '
%' COLLATE NOCASE\n  )\nGROUP BY\n  ocr_text.frame_id\nORDER BY\n  frames.timestamp DESC\nLIMIT\n  ?8 OFFSET ?9\n" rows_affected=0 rows_returned=0 elapsed
=5.334515458s elapsed_secs=5.334515458 slow_threshold=1s
2024-09-26T02:34:00.951897Z  INFO screenpipe_server::server: search completed: found 0 results
[js][info]: "attempt 1 failed, retrying in 1000ms..."
[js][info]: "attempt 2 failed, retrying in 1000ms..."
thread 'main' panicked at /Users/louisbeaumont/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_core-0.307.0/runtime/op_driver/futures_unordered_
driver.rs:307:22:
already borrowed: BorrowMutError
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::cell::panic_already_borrowed
   3: <deno_core::runtime::op_driver::futures_unordered_driver::FuturesUnorderedDriver<C> as deno_core::runtime::op_driver::OpDriver<C>>::submit_op_fallib
le
   4: screenpipe_core::pipes::pipes::op_set_timeout::op_set_timeout::v8_fn_ptr
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'main' panicked at library/core/src/panicking.rs:221:5:
panic in a function that cannot unwind
stack backtrace:
   0:        0x105a76b08 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h243268f17d714c7f
   1:        0x105a994d4 - core::fmt::write::hb3cfb8a30e72d7ff
   2:        0x105a72590 - std::io::Write::write_fmt::hfb2314975de9ecf1
   3:        0x105a77e7c - std::panicking::default_hook::{{closure}}::h14c7718ccf39d316
   4:        0x105a77aa0 - std::panicking::default_hook::hc62e60da3be2f352
   5:        0x105a78940 - std::panicking::rust_panic_with_hook::h09e8a656f11e82b2
   6:        0x105a78268 - std::panicking::begin_panic_handler::{{closure}}::h1230eb3cc91b241c
   7:        0x105a76f94 - std::sys::backtrace::__rust_end_short_backtrace::hc3491307aceda2c2
   8:        0x105a77f58 - _rust_begin_unwind
   9:        0x105b12248 - core::panicking::panic_nounwind_fmt::h91ee161184879b56
  10:        0x105b122c0 - core::panicking::panic_nounwind::heab7ebe7a6cd845c
  11:        0x105b123ac - core::panicking::panic_cannot_unwind::hedc43d82620205bf
  12:        0x103bee498 - screenpipe_core::pipes::pipes::op_set_timeout::op_set_timeout::v8_fn_ptr::h22a3c6621dbf1c64
thread caused non-unwinding panic. aborting.
Abort trap: 6
(base) louisbeaumont@louisbeemacbook:~/Documents/screen-pipe$

https://github.com/mediar-ai/screenpipe/blob/main/examples/typescript/pipe-email-daily-log/pipe.ts

@littledivy