EKK is the new ELK: Aggregating, Analyzing and Visualizing Logs

Level: 200

You've got servers. You've got applications. You've got AWS resources. That means, you've got logs. You can use Amazon Elasticsearch Service and your logs to source valuable insights like failure diagnosis, fraud detection, application performance, uptime, and other infrastructure monitoring. How do you move those logs into Amazon Elasticsearch Service? Amazon Kinesis Data Firehose (Firehose) is a robust delivery pipe to deliver data to Amazon Elasticsearch Service (ES). Firehose provides inline transformation via AWS Lambda callouts, retries with backoff, archival storage in S3, and error handling. You use Firehose's REST API or the Kinesis Agent to send your data to Amazon ES. You can use Kibana, an open-source, thin web client also provided by the service, to search and to build visualizations on your log data. In this tech talk, you'll learn about Firehose and Amazon ES, and how to connect them and how to use Kibana to build visualizations.