Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SQLStatementParserCacheHook to Provide Extension Point for Application SQL Parse Cache Warm-up , Referred to as Preheat #34156

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

youjie23
Copy link

Add SQLStatementParserCacheHook to Provide Extension Point for Application SQL Parse Cache Warm-up , Referred to as Preheat

Fixes #34155.

Changes proposed in this pull request:

  • Add SQLStatementParserCacheHook to Provide Extension Point for Application SQL Parse Cache Warm-up , Referred to as Preheat

Before committing this PR, I'm sure that I have checked the following options:

  • My code follows the code of conduct of this project.
  • I have self-reviewed the commit code.
  • I have (or in comment I request) added corresponding labels for the pull request.
  • I have passed maven check locally : ./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e.
  • I have made corresponding changes to the documentation.
  • I have added corresponding unit tests for my changes.
  • I have updated the Release Notes of the current development version. For more details, see Update Release Note

youjie_li and others added 3 commits December 26, 2024 12:08
Copy link
Member

@terrymanu terrymanu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SQL always changes, using SPI to provide a static java class maybe not good enough.

@youjie23
Copy link
Author

youjie23 commented Dec 29, 2024

The SQL always changes, using SPI to provide a static java class maybe not good enough.

As described in #34155
The SQLStatementParserCacheHook serves as an extension point, allowing business applications to capture cache-missed compiled SQL statements and store them in a designated location for later use . This mechanism is designed to enable application developers to use these captured SQL statements for parse cache warm-up before the actual user requests occur ,like right after restarting the application.

Objective: Facilitate application-level SQL parse cache warm-up to improve performance on initial queries.
Implementation: Introduce a hook that business logic can subscribe to, which triggers when there is a cache miss during SQL statement parsing. The hook should provide access to the missed SQL statement for external processing.
Benefit: Developers can preemptively compile and cache SQL statements in their applications, reducing the latency experienced by users on their first interaction with the system.

@terrymanu
Copy link
Member

I do not hope SQL input to be through Java's SPI. ShardingSphere is a database (or database protocol), and all SQL-related content should be provided through standard JDBC.
It is better to use SPI for feature extend only, not user API.

@youjie23
Copy link
Author

youjie23 commented Dec 29, 2024

I do not hope SQL input to be through Java's SPI. ShardingSphere is a database (or database protocol), and all SQL-related content should be provided through standard JDBC. It is better to use SPI for feature extend only, not user API.

Thank you for the review.
Therefore, the code presented below does not align with your expectations, is that right?

collect like:

public class CollectSQLStatementParserCacheHook implements SQLStatementParserCacheHook {

    public void finishSuccess(String sql, SQLStatement sqlStatement) {
        asyncSaveSqlToStorage(sql);
    }
}

warmup like:

public class SqlParseWarmupHandler {
    public void warmup() {
        SQLStatementParserEngine sqlParseEngine = SQLStatementParserEngineFactory.getSQLStatementParserEngine(new MySQLDatabaseType(), new CacheOption(), new CacheOption());
        List<String> sqlUsed = loadFromStorage();
        for (String sql: sqlUsed) {
            sqlParseEngine.parse(sql, true);
        }
    }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants