Beats

Beats (e.g. Filebeat, Metricbeat, Packetbeat, Heartbeat) are great way to get data into an ElasticSearch cluster. Getting beats to work with ClusterNest managed ElasticSearch is quite simple.

Install

Since ClusterNest supports the OpenSource version of ElasticSearch (OSS), you must install the OSS version of your desired beat. We'll setup filebeat in this example but the process is very similar for other beats as well.

Run the following to install the OSS version of filebeat

ES_VERSION=7.10.2
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-${ES_VERSION}-amd64.deb
sudo dpkg -i filebeat-oss-${ES_VERSION}-amd64.deb

ES_VERSION=7.10.2
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-${ES_VERSION}-x86_64.rpm
sudo rpm -vi filebeat-oss-${ES_VERSION}-x86_64.rpm

ES_VERSION=7.10.2
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-${ES_VERSION}-linux-x86_64.tar.gz
tar xzvf filebeat-oss-${ES_VERSION}-linux-x86_64.tar.gz

ES_VERSION=7.10.2
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-${ES_VERSION}-darwin-x86_64.tar.gz
tar xzvf filebeat-oss-${ES_VERSION}-darwin-x86_64.tar.gz

Configure

Configure filebeat to work with ClusterNest managed ElasticSearch, change your filebeat.yml configuration file to match the following.

output.elasticsearch:
  hosts:
  - example-randomstring-es.clusternest.com:443
  protocol: "https"
  username: username
  password: password
setup.ilm.enabled: false
setup.kibana.host: "kibana-example-randomstring-es.clusternest.com:443"
setup.kibana.protocol: "https"

Replace the ElasticSearch and Kibana URIs with the ones displayed in the ClusterNest Console. Also make sure to create a user with the required privileges for filebeat.

Launch

Next you might want to enable some modules and their dashboards.

Get a list of available modules by running

filebeat modules list

Enable a module by running

filebeat modules enable module1 module2 module3

Setup the required dashboards, pipelines & index tempalate

filebeat setup

Finally, start filebeat

sudo service filebeat start

sudo service filebeat start

sudo chown root filebeat.yml 
sudo chown -R root modules.d/
sudo ./filebeat -e

sudo chown root filebeat.yml 
sudo chown -R root modules.d/
sudo ./filebeat -e

Your data should now be available in ClusterNest Managed ElasticSearch and you should be able to veiw it in Kibana.