Send cookie to GET method from POST

[maksymsereda]

Trying to make a POST method to login and automatically send Cookie to GET method (re-authenticate periodically to fetch a new cookie and then use that in you second request), but smth is going wrong. To make a login I have to fill "login" and "password" poles which is in JSON like this

{
"login": "admin",
"password": "password"
}

and URL like this http://localhost:8080/api/ses/login
My logstash.conf is

input {
 http_poller {
   urls => {
     test2 => {
       url => "http://localhost:8080/api/dashboard/staging"
       method => "get"
       headers => {
         Accept => "application/json"
         Cookie => "JSESSIONID=5fad76f01bbed6f502dd68095e18"
       }
    }
   }
   request_timeout => 60
   # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
   schedule => { cron => "* * * * * UTC"}
   codec => "json"
   # A hash of request metadata info (timing, response headers, etc.) will be sent here
   metadata_target => "http_poller_metadata"
 }
}
output {
 stdout {
   codec => rubydebug
 }
}

Using CentOS 7, ELK Stack 6.3

[yaauie]

The HTTP Poller input doesn't support making a chain of requests that rely on each other, which doesn't make it particularly well-suited for cookie-based authentication described above.

Does the API you're connecting to support key-based authentication, or HTTP Basic Auth? If so, it would be much simpler to implement since a single request could authenticate the user and retrieve the data.

---

The Manticore client that this plugin uses is theoretically capable of holding onto offered cookies and injecting them into subsequent requests, but because we don't have ordering guarantees, we can't ensure that the "authorization" request to acquire the cookie is made before the "data" request.

A pipeline that did so would look something like the following. If the "data" request ran before the first "authentication" request, it would likely error out with 401 Unauthorized or 403 Forbidden, but on the next schedule run one minute later, a cookie should exist.

input {
  http_poller {
    urls => {
      authentication => {
        url => "http://localhost:8080/api/ses/login"
        method => "post"
        body => '{"login":"admin","password":"password"}'
        headers => {"Content-Type" => "application/json"}
      }
      data => {
        url => "http://localhost:8080/api/dashboard/staging"
        method => "get"
        # Cookie header should be injected if available
        headers => { "Accept" => "application/json" }
      }
    }
    request_timeout => 60
    schedule => { cron => "* * * * * UTC"}
    codec => json
  }
}
filter {
  # drop events originating from our authentication call
  if [@metadata][name] == "authentication" {
    drop {}
  }
}
output {
  # ...
}

[maksymsereda]

Thank you for answer. Unfortunatelly I don't know about this:

Does the API you're connecting to support key-based authentication, or HTTP Basic Auth?

I tried a code that you showed but it is not working: nothing showing in logstash. I am trying to do it with a python script. For now I have a such logstash.conf

input {
  exec {
        command => "python /home/maksym/PathFolder/pythonpractice/postsample.py"
        interval => 20
        #id => "plugin_id" 
        }
   request_timeout => 60
   schedule => { cron => "* * * * * UTC"}
   #codec => "json" 
}

output {
   stdout {
     codec => rubydebug 
   }
}   

and python script is:

import os
os.chdir("/home/maksym/PathFolder")
from cookielib import LWPCookieJar
import requests
s = requests.Session()
s.cookies = LWPCookieJar('cookiejar')
if not os.path.exists('cookiejar'):
    # Create a new cookies file and set our Session's cookies
    print('setting cookies')
    s.cookies.save()
    r = s.post('http://localhost:8080/api/session/login', json={'login': 'admin', 'password': 'pass'})
    print(r.text)
else:
    # Load saved cookies from the file and use them in a request
    print('loading saved cookies')
    s.cookies.load(ignore_discard=True)
    r = s.get('http://localhost:8080/api/dashboard/staging-space')
print(r.text)
# Save the session's cookies back to the file
s.cookies.save(ignore_discard=True)
print(r.status_code)
#print(r.json())
print(r.cookies)

If you have any idea or a tip I will listen attentively. Thanks

[yaauie]

I tried a code that you showed but it is not working: nothing showing in logstash.

What do the Logstash logs have to say? Do you have debug logging enabled (e.g., --log.level debug command-line flag)?

I am trying to do it with a python script

Debugging your python script is well out of scope of an issue on the HTTP Poller Input Plugin, and would be better suited as a topic in the Logstash Forums.

[sebastienbonami]

Hi, I'm in the exact same situation than @maksymsereda. I have to make a POST (like to /login) on a web app for getting a cookie and I want to keep this cookie for as long as it's valid and use it for GET requests (like to /health). As soon as there's a 401 error (so the cookie is not valid anymore), I want to to get a new cookie and use it from now on...

@yaauie Does the plugin now support making a chain of requests that rely on each other? If not, is it planned soon?

@maksymsereda Have you find a stable solution? I also made a python script printing a JSON text, but I'm now wondering how to parse the JSON for Logstash?

Thanks.

[kazizi-swe]

My logstash configuration solution.

input{
  http_poller {
    urls => {      
      authentication => {
        method => get
        user => "myEmailAddress"
        password => "myPassword"
        url => "https://api.balena-cloud.com/v4/release?$filter=belongs_to__application%20eq%20<APP ID>"
        headers => {
          "Content-Type" => "application/json"
          "Authorization" => "Bearer <AUTH_TOKEN>"
        }
      }
    }
    request_timeout => 60
    schedule => { every => "5s"}
    codec => "json"
  }
} 

output{
  stdout { 
    codec => rubydebug 
  }
}