Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify type 'keyword' for field 'geoip.postal_code', to avoid incorrectly autodetecting as type 'date' #788

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

smokris
Copy link

@smokris smokris commented Jul 5, 2018

Today my ELK started outputting a flood of warnings like this:

[2018-07-03T20:02:01,900][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-20180704", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"logstash-20180704", "_type"=>"doc", "_id"=>"jceYYmQBMUt-AEnBjPIk", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [geoip.postal_code]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"H3G\""}}}}}

The geoip.postal_code field had been automatically detected as type date because the first document indexed today had an IP address in Portugal, whose postal codes are indistinguishable from the year-week date format. Subsequently, many other countries' postal codes started failing because they can't be parsed as dates (such as H3G above).

I was able to work around the problem by specifying type keyword for the geoip.postal_code field, to avoid autodetection.

@smokris
Copy link
Author

smokris commented Jul 5, 2018

CLA — Commit author has not signed the CLA and is not a member of Elasticsearch

(I did complete the CLA though: Transaction ID CBJCHBCAABAA59rdjHjOigXuRcQOahDPwze8-1K7N-Ps and CBJCHBCAABAA-zv4EQp1vFbGSouaTC2wwwfmdbU0eWCI.)

@yaauie
Copy link
Contributor

yaauie commented Dec 20, 2018

@smokris it looks like you signed the CLA with a different e-mail address (s****@k*****) than is on either the commit or your github profile (smokris@s*****), so our CLA-checker can't verify the commit.

@smokris
Copy link
Author

smokris commented Dec 20, 2018

@yaauie Ah, thanks for clarifying. (I do have both email addresses associated with my GitHub account, but I made the commit using a different email address than the one on the CLA.) I just signed the CLA using my other email address, too.

…rd', to avoid incorrectly autodetecting as type 'date'.
@smokris
Copy link
Author

smokris commented Dec 20, 2018

Hmm, build failure 1290.12 doesn't seem to be related to this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants