elasticsearch - Logstash does not parse json

Question

When i see results in Kibana, i see that there are no fields from JSON, more over, message field contains only "status" : "FAILED".

Is it possible to parse fields from json and to show them in Kibana? I have following config:

input {
  file {
    type => "json"
    path => "/home/logstash/test.json"
    codec => json
    sincedb_path => "/home/logstash/sincedb"
  }
} 

output {
  stdout {}
  elasticsearch {
    protocol => "http"
    codec => "json"
    host => "elasticsearch.dev"
    port => "9200"
  }
}

And following JSON file:

[{"uid":"441d1d1dd296fe60","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,"stop":1419621640491,"duration":17309},"severity":"NORMAL","status":"FAILED"},{"uid":"a88c89b377aca0c9","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,"stop":1419621640634,"duration":17452},"severity":"NORMAL","status":"FAILED"},{"uid":"32c3f8b52386c85c","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623185,"stop":1419621640826,"duration":17641},"severity":"NORMAL","status":"FAILED"}]

Answer 1

Yes. you need to add a filter to your config, something like this.

filter{
    json{
        source => "message"
    }
}

It's described pretty well in the docs here

EDIT The json codec doesn't seem to like having an array passed in. A single element works with this config:

Input:

{"uid":"441d1d1dd296fe60","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,       "stop":1419621640491,"duration":17309      },      "severity":"NORMAL",      "status":"FAILED"   }

Logstash Result:

{
      "message" => "{"uid":"441d1d1dd296fe60","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,       "stop":1419621640491,"duration":17309      },      "severity":"NORMAL",      "status":"FAILED"   }",
     "@version" => "1",
   "@timestamp" => "2015-02-26T23:25:12.011Z",
         "host" => "emmet.local",
          "uid" => "441d1d1dd296fe60",
         "name" => "test_buylinks",
        "title" => "Testbuylinks",
         "time" => {
          "start" => 1419621623182,
           "stop" => 1419621640491,
       "duration" => 17309
   },
     "severity" => "NORMAL",
       "status" => "FAILED"

}

Now with an array:

Input

[{"uid":"441d1d1dd296fe60","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,       "stop":1419621640491,"duration":17309      },      "severity":"NORMAL",      "status":"FAILED"   }, {"uid":"441d1d1dd296fe60","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,       "stop":1419621640491,"duration":17309      },      "severity":"NORMAL",      "status":"FAILED"   }]

Result:

Trouble parsing json {:source=>"message", :raw=>"[{"uid":"441d1d1dd296fe60","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,       "stop":1419621640491,"duration":17309      },      "severity":"NORMAL",      "status":"FAILED"   }, {"uid":"441d1d1dd296fe60","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,       "stop":1419621640491,"duration":17309      },      "severity":"NORMAL",      "status":"FAILED"   }]", :exception=>#<TypeError: can't convert Array into Hash>, :level=>:warn}
{
      "message" => "[{"uid":"441d1d1dd296fe60","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,       "stop":1419621640491,"duration":17309      },      "severity":"NORMAL",      "status":"FAILED"   }, {"uid":"441d1d1dd296fe60","name":"test_buylinks","title":"Testbuylinks","time":{"start":1419621623182,       "stop":1419621640491,"duration":17309      },      "severity":"NORMAL",      "status":"FAILED"   }]",
     "@version" => "1",
   "@timestamp" => "2015-02-26T23:28:21.195Z",
         "host" => "emmet.local",
         "tags" => [
       [0] "_jsonparsefailure"
   ]
}

This looks like a bug in the codec, can you change your messages to an object rather than an array?