Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static Filter: support multiple positional parameters #81

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

yaauie
Copy link
Contributor

@yaauie yaauie commented Jul 16, 2021

When multiple prepared_parameters are passed to the local lookup, the Sequel library raises an exception with Mismatched number of placeholders even when the provided query has the correct number of placeholders. Our unit specs stub out the prepared statement details and therefore do not show off the issue.

There is a workaround listed at the end of this Issue.

This PR introduces a failing spec showing the behaviour when

  • the local lookup is configured with:
    • query => "SELECT * FROM servers WHERE ip = ? AND name = ?"
    • prepared_parameters => ["[host][ip]", "[host][name]"]
  • using an event with the structure:
    • "host" : { "ip" : "10.3.1.1", "name" : "mv-server-1"}, "message" : "ping"}
Failures:

  1) LogStash::Filters::JdbcStatic non scheduled operation under normal conditions with prepared statement with multiple positional parameters enhances an event
     Failure/Error: @db[statement, parameters].prepare(:select, @id)

     Sequel::Error:
       Mismatched number of placeholders (2) and placeholder arguments (1) when using placeholder string
     # ./lib/logstash/filters/jdbc/read_write_database.rb:32:in `prepare'
     # ./lib/logstash/filters/jdbc/lookup.rb:117:in `prepare'
     # ./lib/logstash/filters/jdbc/lookup_processor.rb:68:in `block in create_prepared_statements_for_lookups'
     # ./lib/logstash/filters/jdbc/lookup_processor.rb:66:in `create_prepared_statements_for_lookups'
     # ./lib/logstash/filters/jdbc/lookup_processor.rb:42:in `initialize'
     # ./lib/logstash/filters/jdbc_static.rb:191:in `prepare_runner'
     # ./lib/logstash/filters/jdbc_static.rb:155:in `register'
     # ./spec/filters/integration/jdbc_static_spec.rb:153:in `block in Filters'

Finished in 9.43 seconds (files took 9.28 seconds to load)
18 examples, 1 failure

Failed examples:

rspec ./spec/filters/integration/jdbc_static_spec.rb:152 # LogStash::Filters::JdbcStatic non scheduled operation under normal conditions with prepared statement with multiple positional parameters enhances an event

Workaround

Instead of using prepared statements and positional_parameters, it is safe to use a regular query with named placeholders and the parameters option, as demonstrated by a companion integration spec in this PR:

  • the local lookup is configured with:
    • query => "SELECT * FROM servers WHERE ip = :host_ip AND name = :host_name"
    • prepared_parameters => {"host_ip" => "[host][ip]" "host_name" => "[host][name]" }
  • using an event with the same structure as before:
    • "host" : { "ip" : "10.3.1.1", "name" : "mv-server-1"}, "message" : "ping"}

@yaauie yaauie added the bug Something isn't working label Jul 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants