Configuring Logstash

You need to configure Logstash to ingest data from IMS Performance Analyzer.

The starter dashboards require a Logstash pipeline config with the following characteristics:

Maps the value of the start field in the incoming JSON Lines to the @timestamp field.
The starter dashboards use @timestamp as the event time stamp.
Matches Kibana index pattern imspa-%{id}
The indices are set to lowercase. The starter dashboards use indices such as imspa-as.
Creates data streams (create action) rather than time-based indices.

Here is a starter Logstash pipeline config:

input {
  tcp {
    id => "imspa_tcp_input"
    port => 5046
    codec => json_lines
  }
}
filter {
  date {
    match => ["start", "ISO8601"]
  }
  mutate {
    add_field => { "[@metadata][code_identifier]" => "%{id}" }
  }
  mutate {
    lowercase => [ "[@metadata][code_identifier]" ]
  }
}
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "imspa-%{[@metadata][code_identifier]}"
	  action => "create"
    manage_template => false
  }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

This starter config assumes that you have configured IMS Performance Analyzer to forward data over TCP in JSON Lines format.

This starter config assumes unsecure TCP: no Transport Layer Security (SSL/TLS).

In input.tcp.port, specify the port on which to listen for data from IMS Performance Analyzer.

TIP

If you deploy the Elastic Stack in Docker containers, then you need to understand the difference between port numbers exposed by the Docker host and port numbers used inside the Docker containers.

In output.elasticsearch.hosts, specify the host name of the computer that is running Elasticsearch.

One data stream per product code

The combination of the create action in this starter Logstash config and the data_stream object in the corresponding sample Elasticsearch index template cause Elasticsearch to store IMS Performance Analyzer data in data streamsopen in new window.

Each product code has its own data stream.

Single or multiple Logstash pipelines?

You need to know whether your instance of Logstash is for use only with these starter dashboards or is also used for other purposes, other inputs. Specifically, you need to know whether your use of Logstash involves a single pipeline or multiple pipelines.

If you have installed a new instance of Elastic Stack as a sandbox environment for testing these starter dashboards, then you can use a single Logstash pipeline.

However, if you are using these starter dashboards in an existing instance of Elastic Stack that already has other inputs, then it is more likely that you will need to use multiple pipelines.

Single pipeline

If your instance of Logstash is for use only with these starter dashboards, copy the starter config provided here to the file imspa-tcp-to-local-elasticsearch.conf and save it in the logstash bin folder and run the following command.

logstash -f imspa-tcp-to-local-elasticsearch.conf

Multiple pipelines

For information about configuring multiple pipelines, see the Logstash documentationopen in new window.

# Configuring Logstash

# One data stream per product code

# Single or multiple Logstash pipelines?

# Single pipeline

# Multiple pipelines

Configuring Logstash

One data stream per product code

Single or multiple Logstash pipelines?

Single pipeline

Multiple pipelines