How to install Elasticsearch and Kibana on Centos 7

Intro

Preamble

If you are new to Elasticsearch please read the blog post entitled “What is Elastic and Why Should You Care” it goes into further detail regarding what Elastic is, and how we will use it.

In this tutorial we will bring up a single node cluster for demonstration purposes, the setup we are doing now forms preparatory work to be able to test more advanced features of Elastic in later blog posts.

Don’t forget this is not intended for Production. Only as reference material.

This tutorial will focus on Linux as that is where I am most familiar with running Elastic. If there is sufficient demand, I will make a post on how to install and configure it on Windows.

If you don’t understand some of the terms on in this tutorial please see if it’s hyperlinked or in the blog post about Elasticsearch.

Prereqisites

Required Software
· OS (Centos 7 if you plan on running Elasticsearch on a VM be sure to add at least 8GB of RAM and 2 CPU cores)
· VirtualBox
· Vagrant
· Elasticsearch 7.15.0
· Kibana 7.15.0
· Vi/Vim/Emacs (or your favourite text editing program)
· Browser (Firefox, Chrome, etc)
· At least 8GB of RAM See Troubleshooting for advice

Conventions used in this blog post

Code will appear as code

Things to be cautious around are wrapped like this ! Warning ! Be careful and take heed.

Terminals are presented like this;
Windows PowerShell

example

Linux
Root user

example

Unprivileged user

example

There is a small copy button inside the code blocks it looks like this when moused over (It won’t copy hostname information so you can paste with ease)

Lab Diagram

If you open VirtualBox GUI and put it to one side you will get a better feel for how Vagrant is acting as an API to communicate and provision these VMs in VirtualBox. Also very useful for when things don’t go as planned and more advanced troubleshooting is necessary.

Building the Vagrant VM

This section can be skipped if you already have a Centos Linux VM to hand, or are not planning to use a VM. Skip to the configuration section.

If you are new to virtualization please see the blog post “What is Virtualization and Why Should You Care”.

We are going to use VirtualBox and Vagrant to provision a VM for our work. Vagrant provides a platform to provision the same VM again and again, useful for infrastructure as code.

To get Vagrant working you will need a VM provider, we will use VirtualBox but if you already have VMware Workstation then use that instead (you will need to replace the config.vm.provider section in the scripts). Oracle’s VirtualBox-6.1 can be downloaded here. Once it is installed on your host server you need to install Vagrant, download it from here. Vagrant provisions the guest VMs which run on the host server. Once both are installed and configured correctly, we can proceed.

If you are unfamiliar with Vagrant please read the documentation here.

OS

Windows

Provision Centos 7 VM

To provision the VM once VirtualBox and Vagrant have been installed run the following command

The code can be found on my Github here

Replace USERNAME with your username

mkdir -p C:\Users\USERNAME\Boxes\Elastic_Kibana

Next cd to that DIR

cd C:\Users\USERNAME\Boxes\Elastic_Kibana

To initialize the box run

vagrant.exe init bento/centos-7

Change the vagrantfile you just created to match this (if you have enough RAM on your host)

notepad.exe .\Vagrantfile

Copy and replace everything in the file with the below

Vagrant.configure("2") do |config|
  config.vm.box = "bento/centos-7"
  config.vm.network "forwarded_port", guest: 5601, host: 5601, host_ip: "127.0.0.1"
  config.vm.provider "virtualbox" do |vb|
    vb.cpus = "4"
    vb.memory = "8192"
  end
end

We are adding in a port_forwarding directive so we can get to the Kibana webpage from the host machine

Bring the Box up

vagrant.exe up

Log into the box to validate that everything worked thus far

vagrant.exe ssh

The expected result is

vagrant.exe ssh
This system is built by the Bento project by Chef Software
More information can be found at https://github.com/chef/bento

[vagrant@localhost ~]$

Linux

Provision Centos 7 VM

To provision the VM once VirtualBox and Vagrant have been installed run the following command

The code can be found on my Github here

Replace USERNAME with your username

mkdir -p /home/USERNAME/Boxes/Elastic_Kibana

Next cd to that DIR

cd /home/USERNAME/Boxes/Elastic_Kibana

To initialize the box run

vagrant init bento/centos-7

Change the vagrantfile you just created to match this (if you have enough RAM on your host)

vi Vagrantfile

Copy and replace everything in the file with the below

Vagrant.configure("2") do |config|
  config.vm.box = "bento/centos-7"
  config.vm.network "forwarded_port", guest: 5601, host: 5601, host_ip: "127.0.0.1"
  config.vm.provider "virtualbox" do |vb|
    vb.cpus = "4"
    vb.memory = "8192"
  end
end

We are adding in a port_forwarding directive so we can get to the Kibana webpage from the host machine

Bring the Box up

vagrant up

Log into the box to validate that everything worked thus far

vagrant ssh

The expected result is

vagrant ssh

This system is built by the Bento project by Chef Software
More information can be found at https://github.com/chef/bento
[vagrant@localhost ~]$

Install Elasticsearch and Kibana

Once you have logged into your VM we will now add the required repos for Elasticsearch and Kibana (Again this is a Centos Install Guide it won’t work in the current state with Deb based machines)

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Add the Elasticsearch repo to the repo directory (copy until after the last EOF and it can be pasted in one to the Linux terminal)

sudo bash -c 'cat > /etc/yum.repos.d/elasticsearch.repo' << EOF
[elasticsearch]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md
EOF

Add the Kibana Repo

sudo bash -c 'cat > /etc/yum.repos.d/kibana.repo' << EOF
[kibana-7.x]
name=Kibana repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

The way we get packages differs from the Github code.

Install Elasticsearch, Kibana, and Unzip

sudo yum install elasticsearch kibana unzip

Validate that the installs have worked

sudo head -n 3 /etc/elasticsearch/elasticsearch.yml

sudo head -n 3 /etc/kibana/kibana.yml

Expected result

If there is no text the installation failed please start from the beginning and/or check your distro (It is different for RedHat/Debian based Linux distributions, this install guide is only for Centos 7 which is RPM based)

Configure Elasticsearch and Kibana for Secure Settings

Now that Elastic and Kibana have been installed lets configure them for a secure single node cluster.
Change the below ip:[‘x.x.x.x’] to fit your environment (If you aren’t using the Vagrant VM make sure your IP address isn't a very dynamic DHCP address as you will have problems with SSL Certs later)
The certs will use the IP address as an attribute and if it changes you will need to generate new server certs (Can be done from the CA .crt and .key that will accompany the certs)
You have been warned.
Create the cert file. In this instance we are using a single node cluster so we only have two instances with the same IP.
We will generate two certificates, one for Elasticsearch and one for Kibana (and a Root CA cert)

Create the DIR

mkdir /tmp/certs

Make the instance.yml file

cat > /tmp/certs/instance.yml << EOF
instances:
  - name: 'elasticsearch'
    dns: [ 'elasticsearch.localdomain']
    ip: [ '10.0.2.15' ]
  - name: 'kibana' 
    dns: [ 'kibana.localdomain' ]
    ip: [ '10.0.2.15' ]
EOF

Generate the certificates

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert --keep-ca-key ca --pem --in /tmp/certs/instance.yml --out /tmp/certs/certs.zip

Expected results

Unzip the certs

sudo unzip /tmp/certs/certs.zip -d /tmp/certs/

Here we set a dir for ease of use

Make the dir for Elasticsearch and Kibana

sudo mkdir /etc/elasticsearch/certs/ /etc/kibana/certs/

copy the certs into place

! Rename "certs/elasticsearch and certs/kibana" to whatever you named your certs when you made them if you are using a different name !

sudo cp -r /tmp/certs/ca/ca.crt /tmp/certs/elasticsearch/* /etc/elasticsearch/certs

sudo cp -r /tmp/certs/ca/ca.crt /tmp/certs/kibana/* /etc/kibana/certs

Backup the existing certs to root home dir (or wherever else you'd like, you might need the CA files sooner or later)

sudo cp -r /tmp/certs/* /root/

Configure Elasticsearch

Backup existing file

sudo mv /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.bak

Paste new config into the .yml config file

! Rename "elasticsearch" in the http and ssl section to whatever you named your certs when you made them !

sudo bash -c 'cat > /etc/elasticsearch/elasticsearch.yml' << EOF
# ======================== Elasticsearch Configuration =========================
#
# ----------------------------------- Paths ------------------------------------
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
# ---------------------------------- Network -----------------------------------
network.host: 0.0.0.0
http.port: 9200
# --------------------------------- Discovery ----------------------------------
discovery.type: single-node
# ---------------------------------- Various -----------------------------------
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/elasticsearch.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/elasticsearch.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/certs/elasticsearch.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/elasticsearch.crt
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
EOF

Add firewall rules (N.B. Not needed if you are using the Vagrant Centos VM)

Add the port and make sure the ssh service is in the list so you don't lose connectivity

sudo firewall-cmd –permanent --add-port=9200/tcp

sudo firewall-cmd –permanent --add-port=5601/tcp

Now we test to see if Elasticsearch will start

sudo systemctl start elasticsearch

If the services fails to start make sure the certificates are ok and in the right place, if you get a service timeout you will need to edit the systemd service file.

Check the certificates

sudo openssl verify -CAfile /etc/elasticsearch/certs/ca.crt /etc/elasticsearch/certs/elasticsearch.crt

should report OK, if they aren’t found move them from where we made them in /tmp/

Increase the service start timeout

sudo sed -i 's/TimeoutStartSec=75/TimeoutStartSec=500/g' /lib/systemd/system/elasticsearch.service

Validate Elasticsearch

Now that it’s running we can issue a cURL command to get some info

curl -k -XGET https://10.0.2.15:9200/_cat/nodes?pretty

Expected result is

This is a good thing as now that we have setup the security settings Elasticsearch expects us to authenticate to it. We will now generate the required users.

Generate the users

This step uses the internal default Elastic user. You don’t have access to this users password and need to generate the users with the following step.

sudo bash -c '/usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto -b > /root/ESUsers.txt'

Check that the certs have been made

sudo cat /root/ESUsers.txt

Expected result

Test that you can now authenticate

curl --cacert /tmp/certs/ca/ca.crt -u elastic:$(sudo grep "elastic =" /root/ESUsers.txt | awk '{print $4}') -k -XGET https://10.0.2.15:9200/_cat/nodes?pretty

Expected result

The reason I am greping the password from the /root/ESUsers.txt file is two fold, firstly it’s very easy to automate now, secondly it doesn’t pollute your .bash_history with a bunch of plaintext passwords.

! The ESUsers.txt file should be copied down to your personal password store and deleted afterwards !

Configure Kibana

Add a new config file

! Change the “server.ssl.certificate” cert name if you used your hostname !

! In this example we are using the password stored in plain text in the kibana.yml file which is a no go for Prod !

! Use the Kibana keystore like it is done in the bootstrap.sh file !

sudo bash -c 'cat > /etc/kibana/kibana.yml' << EOF
# =========================== Kibana Configuration ============================
# -------------------------------- Network ------------------------------------
server.host: 0.0.0.0
server.port: 5601
# ------------------------------ Elasticsearch --------------------------------
elasticsearch.hosts: ["https://10.0.2.15:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "$(sudo grep "kibana_system =" /root/ESUsers.txt | awk '{print $4}')"
# ---------------------------------- Various -----------------------------------
server.ssl.enabled: true
server.ssl.certificate: "/etc/kibana/certs/kibana.crt"
server.ssl.key: "/etc/kibana/certs/kibana.key"
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/ca.crt" ]
# ---------------------------------- X-Pack ------------------------------------
xpack.security.encryptionKey: "$(tr -dc A-Za-z0-9 <∕dev∕urandom | head -c 32 ; echo '')"
xpack.encryptedSavedObjects.encryptionKey: "$(tr -dc A-Za-z0-9 <∕dev∕urandom | head -c 32 ; echo '')"
xpack.reporting.encryptionKey: "$(tr -dc A-Za-z0-9 <∕dev∕urandom | head -c 32 ; echo '')"

Don’t worry about the X-Pack settings now they will come into play in a later blog post.

The Vagrant bootstrap.sh script uses the Kibana keystore, this is beyond the scope of this post.

Validate the config

sudo cat /etc/kibana/kibana.yml

Expected result

Start Kibana

sudo systemctl start kibana

Validate it all comes together by visiting https://127.0.0.1:5601 in your favourite browser on your host machine (Not the headless Vagrant VM, but the machine you ran Vagrant from)

You should see the login page like below (After you added an exception for the Self-Signed cert)

To get the elastic users password input this command on the Vagrant VM

sudo grep "elastic =" /root/ESUsers.txt | awk '{print $4}'

If you have made a mistake and do not get the login page at this point you can issue a “vagrant destroy” command and start from the beginning and make sure you copied the Vagrantfile correctly if you followed along with the Vagrant section and have the configs copied correctly.

vagrant.exe destroy

vagrant destroy

Congratulations

Congratulations you now have a demo Elasticsearch + Kibana cluster, in the next blog post we will start to add data. In the mean time have a look around Kibana and it’s features.

You have now built a single node Elasticsearch + Kibana cluster. It won’t do much for you at the moment but watch this space for the next blog posts to see how we can make it work for us.

! Don’t forget this Vagrant setup is not for production !
! Once you have noted all the user/passes down from /root/ESUsers.txt DELETE IT !
! If a nefarious agent gets your Production clusters Elastic password it is GAME OVER !

Automation

If you would like to bring this single node cluster up automatically with all the configs already done please see my GitHub page
“WhY IsN’t It AnSiBlE?!?” The bootstrap.sh provision script is in bash for ease of use and understanding, I would like to make using Elastic as easy as possible. In a later guide we will deploy it with Ansible for “FULL AUTOMATION!”

The readme explains how to bring the VM up with Vagrant

Troubleshooting

Not enough RAM

In the Vagrantfile change this value
From
vb.memory = "8192"
to
vb.memory = "6144”

Once the change is made and the file saved reload the VMs with

vagrant.exe reload

vagrant reload

Not enough vCPU cores

You can reduce the CPU count for the Elastic VM in the Vagrantfile to 2 vCPUs (This might slowdown Elasticsearch and Kibana)
From
vb.cpus = "4"
to
vb.cpus = "2"

Once the change is made and the file saved reload the VMs with

vagrant.exe reload

vagrant reload

Links

SSL creation https://www.golinuxcloud.com/enable-https-ssl-secure-elasticsearch-linux/(very useful)

SSL creation https://www.elastic.co/guide/en/elasticsearch/reference/7.14/security-basic-setup-https.html#encrypt-kibana-elasticsearch(Kind of useful)

Nodes https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html(Useful)

Why we add the X-Pack settings in the Kibana Config https://www.elastic.co/guide/en/kibana/7.15/xpack-security-secure-saved-objects.html(Useful)