Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serializing Large LRUs hangs/StackOverflows #45

Closed
jarbus opened this issue Jan 12, 2024 · 4 comments · Fixed by #46
Closed

Serializing Large LRUs hangs/StackOverflows #45

jarbus opened this issue Jan 12, 2024 · 4 comments · Fixed by #46

Comments

@jarbus
Copy link
Contributor

jarbus commented Jan 12, 2024

The following code hangs on my machine and performs a stackoverflow on another:

using LRUCache
using Serialization

# Create an LRU cache with a capacity of 100k elements
cache = LRU{Int, Int}(maxsize=100_000)

# Populate the cache with dummy data
for i in 1:100_000
    cache[i] = i
end

serialize("cache.jls", cache)

I think this is because when there are too many linked nodes, the stack overflows before all the nodes can be serialized. I'm going to be writing some custom serialization logic to handle large LRUs, was wondering if upstream might want the changes.

@Jutho
Copy link
Collaborator

Jutho commented Jan 12, 2024

I think the linked lists will indeed need custom serialisation, as otherwise probably the standard approach tries to traverse from every node down as if they were all infinitely deep trees of independent nodes.

@jarbus
Copy link
Contributor Author

jarbus commented Jan 12, 2024

@Jutho would you be interested in a PR?

@Jutho
Copy link
Collaborator

Jutho commented Jan 12, 2024

Sure, why not. We have so far not had the need to serialize LRUs, but other people might need it, and so might we at some point.

@jarbus
Copy link
Contributor Author

jarbus commented Jan 12, 2024

Submitted a PR, #46. Let me know what you think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants
@Jutho @jarbus and others