Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multiple reactive keys exist checker #2918

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

AnneMayor
Copy link
Contributor

  • You have read the Spring Data contribution guidelines.
  • You use the code formatters provided here and have them applied to your changes. Don’t submit any formatting related changes.
  • You submit test cases (unit or integration tests) that back your changes.
  • You added yourself as author in the headers of the classes you touched. Amend the date range in the Apache license header if needed. For new types, add the license header (copy from another file and set the current year only).

Resolved #2883

@@ -164,7 +184,7 @@ void renameShouldAlterKeyNameCorrectly() {
connection.keyCommands().rename(KEY_1_BBUFFER, KEY_2_BBUFFER).as(StepVerifier::create).expectNext(true)
.verifyComplete();
assertThat(nativeCommands.exists(KEY_2)).isEqualTo(1L);
assertThat(nativeCommands.exists(KEY_1)).isEqualTo(0L);
assertThat(nativeCommands.exists(KEY_1)).isZero();
Copy link
Contributor Author

@AnneMayor AnneMayor May 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just changed the command to make better readability according to sonarlint rules.

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label May 26, 2024
* @return {@link Mono} emitting {@literal true} if all keys exist.
* @see <a href="https://redis.io/commands/exists">Redis Documentation: EXISTS</a>
*/
default Mono<Boolean> exists(List<ByteBuffer> keys) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a validation method which is for checking if parameterized all keys exist in redis cache. If one of them does not exist, it returns false.
@kutlueren , please check if this function is what you meant to use.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @AnneMayor

Awesome to see that you added the function which carries out what I suggested. It is actually more straight forward than I initially thought, maybe I got confused in the codebase but nice! I shall try it out as soon as it gets merged.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation calls EXISTS for each key issuing a command for each element in the list.

Redis' EXISTS command accepts multiple arguments and returns how many of the keys exist. Our interface should use the existing functionality instead of putting another layer on top of commands with a worse performance behavior.

The issue we have here is however that exists(Publisher<KeyCommand> keys) and exists(Publisher<List<KeyCommand>> keys) or exists(Publisher<MultiKeyCommand>> keys) erase to the same type and we cannot simply introduce such an override.

I suggest to implement Mono<Long> exists(List<ByteBuffer> keys) directly instead of using default methods. Also, ReactiveRedisOperations should be extended with countExistingKeys(…) similar to RedisTemplate.countExistingKeys(…).

@AnneMayor AnneMayor force-pushed the add-multiple-reactive-keys-exist-checker branch from 0d0df34 to 78c6c7f Compare May 26, 2024 14:29
@kutlueren
Copy link

Hey @AnneMayor, I just wanted to follow up on the topic :) Is this being planned to be merged soon?

@AnneMayor
Copy link
Contributor Author

Hey @kutlueren ,
Unfortunately I have no feedback from our maintainer🥲
I am still waiting for their feedback.

@kutlueren
Copy link

Hey @kutlueren , Unfortunately I have no feedback from our maintainer🥲 I am still waiting for their feedback.

@mp911de would you mind taking a look at this one when you have time?

@AnneMayor
Copy link
Contributor Author

@mp911de ,
Please let me know if this PR has some issues🙏

Copy link
Member

@mp911de mp911de left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a review pass and left a comment regarding the design and implementation. Care to have a look?

* @return {@link Mono} emitting {@literal true} if all keys exist.
* @see <a href="https://redis.io/commands/exists">Redis Documentation: EXISTS</a>
*/
default Mono<Boolean> exists(List<ByteBuffer> keys) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation calls EXISTS for each key issuing a command for each element in the list.

Redis' EXISTS command accepts multiple arguments and returns how many of the keys exist. Our interface should use the existing functionality instead of putting another layer on top of commands with a worse performance behavior.

The issue we have here is however that exists(Publisher<KeyCommand> keys) and exists(Publisher<List<KeyCommand>> keys) or exists(Publisher<MultiKeyCommand>> keys) erase to the same type and we cannot simply introduce such an override.

I suggest to implement Mono<Long> exists(List<ByteBuffer> keys) directly instead of using default methods. Also, ReactiveRedisOperations should be extended with countExistingKeys(…) similar to RedisTemplate.countExistingKeys(…).

@AnneMayor
Copy link
Contributor Author

I did a review pass and left a comment regarding the design and implementation. Care to have a look?

Thanks a lot. I am going to apply your review within this weekend 👍

@dreamstar-enterprises
Copy link

dreamstar-enterprises commented Jan 15, 2025

Any luck solving this?

I have the same issue. Wanting to check the existence of multiple keys using exists in a single network call, without resorting to Lua, but using Springs own implementation

@dreamstar-enterprises
Copy link

dreamstar-enterprises commented Jan 15, 2025

My current workaround, since the spring API does not support exists with multiple keys yet.

/**
     * Executes expired session cleanup in Redis.
     *
     * Spring Session Redis uses multiple keys per session:
     * 1. Hash key (namespace:sessions:{id}) - Stores session data with 35 min TTL
     * 2. ZSet entry - Tracks session expiration times for batch cleanup
     *
     * Process flow:
     * 1. Query ZSet for sessions that should be expired
     * 2. Access their hash keys to trigger Redis expiration events
     *    - Redis only guarantees expiration events when keys are accessed
     *    - Spring Session listens for these events to cleanup related keys
     * 3. Remove processed entries from the ZSet
     *
     * Implementation details:
     * - Uses Lua script for efficient multi-key EXISTS checks
     * - Handles quoted session IDs in ZSet vs unquoted in hash keys
     * - Relies on Redis keyspace events for actual session deletion
     * - Bounded elastic scheduling for background processing
     *
     * @param context Operation boundaries and limits for ZSet query
     * @return Mono<Void> completing when cleanup process finishes
     * @throws RedisConnectionException if Redis operations fail
     *
     * @see "https://docs.spring.io/spring-session/reference/configuration/reactive-redis-indexed.html#how-spring-session-cleans-up-expired-sessions"
     */
    private fun cleanupExpiredSessions(context: CleanupContext): Mono<Void> =
        redisOperations.opsForZSet()
            .reverseRangeByScore(redisKeyExpirations, context.range, context.limit)
            .collectList()
            .flatMap { sessionIds ->
                if (sessionIds.isEmpty()) {
                    logger.debug("No expired sessions found")
                    Mono.empty()
                } else {
                    logger.debug("Found {} expired sessions to remove", sessionIds.size)

                    // Prepare session keys (remove quotes around session id in zset)
                    val sessionKeys = sessionIds.map { sessionId ->
                        "${sessionProperties.redis?.sessionNamespace}:sessions:${sessionId.trim('"')}"
                    }.also { keys ->
                        logger.debug("Checking Redis keys: {}", keys)
                    }

                    // First read the hash keys to trigger expiration event
                    val script = """
                       local keys = ARGV
                       local count = 0
                       for i, key in ipairs(keys) do
                           count = count + redis.call('EXISTS', key)
                       end
                       return count
                    """

                    redisOperations.execute(
                        RedisScript.of(script, Long::class.java),
                        listOf(),  // KEYS list is empty
                        sessionKeys
                    )
                    .doOnNext { count ->
                        logger.debug("Accessed {} session hash keys", count)
                    }
                    .flatMap { _ ->
                        redisOperations.opsForZSet()
                            .remove(redisKeyExpirations, *sessionIds.toTypedArray())
                            .doOnSuccess { logger.debug("Successfully removed {} expired sessions", sessionIds.size) }
                            .doOnError { e -> logger.error("Error during removal: ${e.message}") }
                    }
                    .then()
                }
            }
            .subscribeOn(Schedulers.boundedElastic())

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: waiting-for-triage An issue we've not yet triaged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ReactiveKeyCommands.Exists to check multiple key existence
5 participants