Skip to content

Commit

Permalink
reduce aggressiveness of flush thread sometimes
Browse files Browse the repository at this point in the history
if the delta between global pool and where we want to be is 100, and
there're 10 active flushable classes, we can end up flushing 1gb of
data. this tries to clamp that down.
  • Loading branch information
dormando committed Nov 18, 2024
1 parent e4cb5c5 commit 2ffb176
Showing 1 changed file with 13 additions and 1 deletion.
14 changes: 13 additions & 1 deletion storage.c
Original file line number Diff line number Diff line change
Expand Up @@ -617,12 +617,24 @@ static void *storage_write_thread(void *arg) {
int target_pages = 0;
if (global_pages < settings.ext_global_pool_min) {
target_pages = settings.ext_global_pool_min - global_pages;
// make sure there's room in each class but don't over-flush.
// ie: we're going to want to move pages from the class with the
// "oldest" items, but this thread has no way of getting that info
// up-to-date. So we just free some pages in all active classes
// and let the page mover free stuff up.
// However, if the target delta is large (100+ megs) we can end up
// flushing gigabytes of data across many classes.
// Thus, we clamp the target here.
if (target_pages > 5) {
target_pages = 5;
}
}
counter++;
if (to_sleep > settings.ext_max_sleep)
to_sleep = settings.ext_max_sleep;

for (int x = 0; x < MAX_NUMBER_OF_SLAB_CLASSES; x++) {
// the largest items have the least overhead from going to disk.
for (int x = MAX_NUMBER_OF_SLAB_CLASSES-1; x > 0; x--) {
bool did_move = false;
bool mem_limit_reached = false;
unsigned int chunks_free;
Expand Down

0 comments on commit 2ffb176

Please sign in to comment.