Access control for browsers stops working when basic auth is enabled

[pkoretic]

• Elastic 6.2.3

Logstash config file

input {
	http {
		user => "some"
		password => "one"
		port => 38080
		codec => json
                response_headers => {
                    "Access-Control-Allow-Origin" => "*"
                    "Content-Type" => "text/plain"
                    "Access-Control-Allow-Headers" => "*"
                }
	}
}

This works as expected in non browser environment while in browser when POST request is made (xhr.open("POST", "http://localhost:38080", true, "some", "one")) this results in (Chrome 65.0.3325.181)

OPTIONS http://localhost:38080/ 401 (Unauthorized)
Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Removing user / password fields this works as expected. No difference with other options like Access-Control-Allow-Methods. It just seems to me like this is handled by plugin internally because this is what I see in wireshark:

OPTIONS / HTTP/1.1
Host: localhost:38080
Connection: keep-alive
Access-Control-Request-Method: POST
Origin: http://127.0.0.1:3333
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36
Access-Control-Request-Headers: authorization,content-type
Accept: /
DNT: 1
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,hr-HR;q=0.8,hr;q=0.7

HTTP/1.1 401 Unauthorized
Content-Type: text/plain
WWW-Authenticate: Basic realm=""
Content-Length: 0

This should return response_headers so that client can authenticate. I found same working example here, although this also uses ssl which I haven't tried yet (and only curl is used so I can't say if browsers are working):
https://discuss.elastic.co/t/http-input-with-ssl/115338

[pkoretic]

adding Rack::Cors middleware to the Rack::Builder.new seems to be on the right track although I don't have time to investigate more

[pkoretic]

CORS should definitely be handled by this plugin properly because this results in two events
for other codecs like msgpack or protobuf this results in errors being generated for OPTIONS request that has no body

[ryan-dyer-sp]

Any update on this? Also experiencing this issue with logstash 6.2.4

[antgel]

@pkoretic @ryan-dyer-sp I'd also like to know what the status is here. At the moment, it means that the http input plugin is useless for consuming events from webapps.

Cc'ing @yaauie @jsvd who were the most recent contributors to this plugin, maybe they can shed light on the subject?

I don't know the codebase well enough to chip in usefully, but I do have a budget to fund professional services if anyone can fix the issue.

[pkoretic]

@pkoretic @ryan-dyer-sp I'd also like to know what the status is here. At the moment, it means that the http input plugin is useless for consuming events from webapps.

Cc'ing @yaauie @jsvd who were the most recent contributors to this plugin, maybe they can shed light on the subject?

I don't know the codebase well enough to chip in usefully, but I do have a budget to fund professional services if anyone can fix the issue.

As for the issue mentioned here you can just skip empty events with filter plugin or even better, just check if message field is valid in output - don't have exact line here anymore but something like https://discuss.elastic.co/t/conditional-for-empty-string-in-field/77182/5.

However, we decided to develop our own HTTP/2 input directly for elasticsearch using vanilla Node.js that replaces HTTP+logstash which gave us all we wanted and 3x better throughout. No matter how we tried with this plugin it was too slow, used too much CPU and had issues like mentioned here.

I think that writing a new logstash HTTP/2 input from scratch would be the best direction and I wish we could contribute that but that was outside of our possibilities.

The elastic stack obviously supports custom development since Elasticsearch has REST api and both logstash and beats are using that so I think that is always a valid approach.

[antgel]

@pkoretic Perhaps I've missed the point - if OPTIONS fails with 401, Angular doesn't go ahead with the next request. Not sure what that has to do with empty events - I'd like to get as far as an event! ;)

[pkoretic]

@antgel ah, then you are here https://discuss.elastic.co/t/post-data-to-logstash-using-http-input/69166/7 :)

well actually, I wrote instructions in this post, so yes, probably didn't work in the end, which was another reason we did our own input

[antgel]

@pkoretic It's not the same issue, the issue I face is that the browser sends OPTIONS without an Authorization: header, as expected, and logstash doesn't cope with that when basic auth is configured. Have I missed something?

[pkoretic]

That is the issue that is reported here. It's not angular related since that is just a html framework.
If it works when you disable basic auth then I can only recommend to fork and create your own plugin and implement your logic for auth which we also did as part of investigation.
Basic auth is just base64 username:password so that one is very simple and can be implemented as a custom header or a query param instead. SSL is a must in all cases.

You can easily see the logic for this here: https://github.com/logstash-plugins/logstash-input-http/blob/master/lib/logstash/inputs/http.rb

You could also then implement some other auth logic like JWT or whichever you may already have for better security.

[antgel]

Unfortunately I don't have time to fork or dive into the logstash code. I'm currently testing nginx as a proxy in front of logstash. It offers far greater control over the incoming HTTP connection, and then the connection between nginx and logstash can be internal and trusted. I'll know in the next few days if it works well or not.

[pkoretic]

Yes, also valid approach, that will work for sure.

[wibbly13]

For what it's worth, my workaround for this is passing an 'auth_token' field in the request header with the value set to a token. The request header is checked in a filter and compared against the expected value. If it doesn't match the log is dropped.
FYI: The 'Authorization' request header doesn't get passed to the filter.
Beware of this gotcha if using environment variables in the filter.

[Emptyfruit]

This bug is still relevant..

[davift]

Yes, this issue is still relevant.

For preflight, browsers use the OPTIONS method without additional headers (like auth) or body.

How about adding a preflight flag to skip basic auth for the OPTIONS method?

After responding to the input http request, the body should be wiped or the request dropped to prevent unauthenticated payloads to be further parsed.

I am willing to help coding if the team agrees with the proposed solution.