Logstash-filter-jdbc_streaming fail: Could not load jdbc/mssql adapter

[yuliansen]

Logstash information:

1. Logstash version: 7.13
2. installed with RPM run on CentOS7
3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes)
4. How was the Logstash Plugin installed

JVM (e.g. java -version):

1. java -version : openjdk 11.0.11 2021-04-20 LTS
2. installed jdbc 6.0:"/home/jdbc/sqljdbc_6.0/enu/jre7/sqljdbc41.jar"

my .conf:

input {
  generator {
    lines => [
"CBN00000020121",
"CBN00000040121"
    ]
    count => 1
  }
}
filter{
csv{
   columns => ["noinvoice"]
   separator => ","
}
{
jdbc_streaming {
    jdbc_driver_library => "/home/jdbc/sqljdbc_6.0/enu/jre7/sqljdbc41.jar"
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_connection_string => "jdbc:mssql://10.64.2.250:1433/mydatabase"
    jdbc_user => "user"
    jdbc_password => "password"
    statement => "select EmailTime from TestInvoice.dbo.storing_ssis WHERE invoiceno= :noinvoice"
    parameters => {"noinvoice" => "noinvoice"}
    target => "EmailTime"
  }
}
output{
stdout{codec => rubydebug}
}

When I run by /usr/share/logstash/bin/logstash -f fjdbc.conf

Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2021-06-10 15:55:36.686 [main] runner - Starting Logstash {"logstash.version"=>"7.13.1", "jruby.version"=>"jruby 9.2.16.0 (2.5.7) 2021-03-03 f82228dc32 OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [linux-x86_64]"}
[WARN ] 2021-06-10 15:55:37.253 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2021-06-10 15:55:39.668 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2021-06-10 15:55:41.520 [Converge PipelineAction::Create<main>] Reflections - Reflections took 69 ms to scan 1 urls, producing 24 keys and 48 values
[ERROR] 2021-06-10 15:55:43.988 [[main]-pipeline-manager] javapipeline - Pipeline error {:pipeline_id=>"main", :exception=>#<Sequel::AdapterNotFound: Could not load jdbc/mssql adapter: adapter class not registered in ADAPTER_MAP>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.44.0/lib/sequel/database/connecting.rb:97:in `load_adapter'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.44.0/lib/sequel/adapters/jdbc.rb:380:in `adapter_initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.44.0/lib/sequel/database/misc.rb:154:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.44.0/lib/sequel/database/connecting.rb:57:in `connect'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/sequel-5.44.0/lib/sequel/core.rb:124:in `connect'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.7/lib/logstash/plugin_mixins/jdbc_streaming.rb:62:in `prepare_jdbc_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-jdbc-5.0.7/lib/logstash/filters/jdbc_streaming.rb:122:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:228:in `block in register_plugins'", "org/jruby/RubyArray.java:1809:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:227:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:586:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:240:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:185:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:137:in `block in start'"], "pipeline.sources"=>["/etc/logstash/fjdbc.conf"], :thread=>"#<Thread:0x67751750 run>"}
[INFO ] 2021-06-10 15:55:43.991 [[main]-pipeline-manager] javapipeline - Pipeline terminated {"pipeline.id"=>"main"}
[ERROR] 2021-06-10 15:55:44.015 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[INFO ] 2021-06-10 15:55:44.146 [LogStash::Runner] runner - Logstash shut down.

plugin is intalled /usr/share/logstash/bin/logstash-plugin list

logstash-integration-jdbc
 ├── logstash-input-jdbc
 ├── logstash-filter-jdbc_streaming
 └── logstash-filter-jdbc_static

update:
I realize jdk 11 doesn't suit with jdbc 6.0, so I download jdbc 7.2.2 and change the path into /root/jdbc2/sqljdbc_7.2/enu/mssql-jdbc-7.2.2.jre11.jar but it still returning the same error.

[andsel]

HI @yuliansen thanks for your report.

• Does it happen only with this driver or also with another (for example MySQL)?
• may you try to copy the jdbc driver jar into /usr/share/logstash/lib to verify it's a problem with the loading facility?

[gr790]

Hello @yuliansen ,

I guess you downloaded the JDBC jar from https://go.microsoft.com/fwlink/?linkid=2122435
Please check the sample code in Archive,
Connection string (jdbc_driver_class ) should be com.microsoft.sqlserver.jdbc.SQLServerDataSource instead of (jdbc_driver_class ) "com.microsoft.sqlserver.jdbc.SQLServerDriver"

Best Regards

[andsel]

Maybe related to #99 or #104