Logstash not reading file input

Question

I have a strange problem with Logstash. I am providing a log file as input to logstash. The configuration is as follows:

input {
  file {
    type => "apache-access"
    path => ["C:\Users\spanguluri\Downloads\logstash\bin\test.log"]
  }
}
output {
  elasticsearch {
    protocol => "http"
    host => "10.35.143.93"
    port => "9200"
    index => "latestindex"
  }
}

I am running elasticsearch server already and verifying if the data is being received with curl queries. The problem is, no data is being received when the input is a file. However, if I change input to stdin { } as follows, it sends all input data smoothly:

input {
  stdin{ }
}
output {
  elasticsearch {
    protocol => "http"
    host => "10.35.143.93"
    port => "9200"
    index => "latestindex"
  }
}

I don't get where I am going wrong. Can someone please take a look at this?

Solution 1

You should set start_position under your file section:

start_position => "beginning"

It defaults to end and so won't read any existing lines in your file, only newly added ones:

start_position

Value can be any of: "beginning", "end"
Default value is "end"

Choose where Logstash starts initially reading files: at the beginning or at the end. The default behavior treats files like live streams and thus starts at the end. If you have old data you want to import, set this to ‘beginning’

This option only modifies “first contact” situations where a file is new and not seen before. If a file has already been seen before, this option has no effect.

Solution 2

In addition to the provided answer, I had to change the path from c:\my\path to c:/my/path in order for it to read the files.

Comments

[Sasanka Panguluri]

Thank you, can you answer my other question if possible? stackoverflow.com/questions/24025230/…

[LF00]

This works, and save me a lot of time. Thank you.