Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
273 views
in Technique[技术] by (71.8m points)

Kafka Streams vs Logstash


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

Kafka Streams and Logstash are two completely different things

Kafka Streams is a client library that you can use to write an application to stream and process data stored in Kafka Brokers, you need to write your own application in Java.

Logstash is an ETL tool that you can use to extract/receive data from multiple sources, process this data using a wide range of filters and send it to different outputs, like elasticsearch, file, s3, kafka and many others.

It is very common to use Logstash and Kafka together, which Kafka working as a message queue for the messages that logstash will consume and process, you have shippers like Filebeat sending data to Kafka Brokers and then you use Logstash to consume this data.

You can build your own applications in Java using the Kafka Streams library to collect, process and ship the data to Elasticsearch, but this will be very complex in comparison with using the tools of the stack, Filebeat to collect logs, Logstash to receive/process, Elasticsearch to store.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...