Updating my ELK stack with GELF appender
Von Carsten
A couple of days ago I wrote about setting up an ELK stack with docker-compose. I did some small changes to the set up, so I thought it’s worth an update.
First change is in the docker-compose.yml to enable logstashs gelf input, move the Log4j socket to port 12202 and add an UDP port forwarding for docker on port 12201:
elasticsearch:
image: elasticsearch
ports:
- 9200:9200
logstash:
image: logstash:latest
links:
- elasticsearch:elasticsearch
ports:
- 12201:12201/udp
- 12202:12202
command: logstash agent -e 'input { gelf { port => "12201" } log4j { mode => "server" port => "12202"} } output { elasticsearch { hosts => ["elasticsearch"] } }'
kibana:
image: kibana
links:
- elasticsearch:elasticsearch
ports:
- 5601:5601
environment:
- ELASTICSEARCH_URL=http://elasticsearch:9200
This way, logstash will listen to ports 12201 with the gelf input and port 12202 with the Log4j socket input.
Now I modified the log4j.properties for my application and replaced the Log4j SocketAppender
with a biz.paluch.logging.gelf.log4j.GelfLogAppender
:
log4j.appender.gelf=biz.paluch.logging.gelf.log4j.GelfLogAppender
log4j.appender.gelf.Threshold=INFO
log4j.appender.gelf.Host=udp:127.0.0.1
log4j.appender.gelf.Port=12201
#log4j.appender.gelf.Version=1.1
#log4j.appender.gelf.Facility=java-test
log4j.appender.gelf.ExtractStackTrace=true
log4j.appender.gelf.FilterStackTrace=true
log4j.appender.gelf.MdcProfiling=true
log4j.appender.gelf.TimestampPattern=yyyy-MM-dd HH:mm:ss,SSSS
log4j.appender.gelf.MaximumMessageSize=8192
# This are static fields
log4j.appender.gelf.AdditionalFields=environment=local
The gelf appender is much more configurable and the search abilities in logstash are now a bit more useful to me. But I have to add another dependency to my application now:
<dependency>
<groupId>biz.paluch.logging</groupId>
<artifactId>logstash-gelf</artifactId>
<version>1.8.0</version>
</dependency>
You can find the documentation at https://github.com/mp911de/logstash-gelf.