ELK Stack

The ELK stack, which stands for Elastic Search, Logstash, and Kibana, is a set of services used for log file aggregation and management. Recently, Amazon spun off forks of ElasticSearch and Kibana called OpenSearch and OpenSearch Dashboards after Elastic changed their licensing.

ElasticSearch (or OpenSearch) typically runs on port 9200 and provides full-text search capabilities by indexing documents. Logstash is a preprocessing pipeline that transforms data before storing it in ElasticSearch or OpenSearch. Kibana (or OpenSearch Dashboards) is a visualization layer that allows users to create dashboards and other visualizations of the data in ElasticSearch or OpenSearch.

ElasticSearch has a detailed API that can be found at here. Most of this documentation is directly compatible with OpenSearch.

Creating Custom Mappings in OpenSearch integrated with Graylog

When the auto index for ElasticSearch’s type on a field is incorrect, you can create custom mappings. In my case, the original mappings were created by Graylog (an alternative to log stash). You can created custom mappings according to this post. This post details more about how to get Graylog and ElasticSearch to pick up the custom mappings.

Links to this note