How to configure logstash in docker compose?
19,686
You mounted a file as the name of a directory here:
volumes:
- ./logstash/logstash.conf:/etc/logstash/conf.d
Include the filename when mounting a single file:
volumes:
- ./logstash/logstash.conf:/etc/logstash/conf.d/logstash.conf:ro
I've also marked it read only since the container is likely not modifying this file.
Author by
rakeshh92
Updated on June 20, 2022Comments
-
rakeshh92 almost 2 years
I'm setting up a Docker container, but not able to configure the logstash to the elasticsearch which is running in AWS.
The folder structure is as below
. ├── logstash │ └── logstash.conf ├── docker-compose.yml ├── Dockerfile ├── file.jar
./logstash/logstash.conf file look like
(log folder in below input path will create automatically parallel to the jar file when we start the server using jar)
input { file { path => "/java_pro/log/*/*/*.log" type => "json" } } output { elasticsearch { hosts => [ "https://search-***********.es.amazonaws.com:443" ] index => "logback-%{+YYYY-MM-dd}" } }
Dockerfile
FROM java:8 RUN mkdir -p /java_pro WORKDIR /java_pro COPY . /java_pro EXPOSE 443 CMD java -jar file.jar
docker-compose.yml
version: "3" services: hub: build: . volumes: - .:/java_pro - static_volume:/java_pro networks: - rabbitmq-networks rabbitmq: image: "rabbitmq:3-management" ports: - "5672:5672" - "15672:15672" volumes: - "rabbitmq_data:/data" networks: - rabbitmq-networks environment: - RABBITMQ_DEFAULT_USER=**** - RABBITMQ_DEFAULT_PASS=**** logstash: image: docker.elastic.co/logstash/logstash:6.4.0 ports: - "9200:9200" command: logstash -f /etc/logstash/conf.d/logstash.conf volumes: - ./logstash/logstash.conf:/etc/logstash/conf.d environment: LS_JAVA_OPTS: "-Xmx256m -Xms256m" networks: - rabbitmq-networks networks: rabbitmq-networks: driver: bridge volumes: rabbitmq_data: static_volume:
Elasticsearch is up and running in AWS. Getting error while running using docker-compose up
No config files found in path {:path=>"/etc/logstash/conf.d/logstash.conf"} Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://elasticsearch:9200/, :path=>"/"} Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::ResolutionFailure] elasticsearch"}
How to resolve the above error and want to push all the logs from the application to elasticsearch using logstash.