Kibana Tutorial : A Complete guide for the beginners

Kibana Tutorial : A Complete guide for the beginners

   December 26, 2019       Codez Tech

---

Kibana is an open-source data visualisation dashboard for Elasticsearch. It provides visualisation on top of the content indexed in an Elasticsearch cluster — users can build bar, line, and scatter plots, pie charts, and maps over large volumes of data.

Kibana

The below topics are covered in this blog -

1) Overview of Kibana
2) What is Kibana?
3) Installing Kibana
4) Indexing data into Kibana
5) GET, POST, PUT, DELETE from Kibana Dev Tools
6) Elasticsearch aggregation and projection
7) Elasticsearch pagination and scroll
8) Visualising Elasticsearch data in Kibana

1. Overview of Kibana

Kibana also includes a presentation tool called Canvas, which lets users build slide decks that pull live data directly from Elasticsearch. The combination of Elasticsearch, Kibana, and Logstash is known as the ELK Stack.

Developed by: Elastic NV.

2. What is Kibana?

Kibana is a data visualisation and management tool for Elasticsearch that provides real-time histograms, line graphs, pie charts, maps, and other diagrams.

3. Installing Kibana

Download Kibana from the official Elastic site. Always match the Kibana version to your Elasticsearch version:

https://www.elastic.co/downloads/kibana

4. Indexing Data into Kibana

When you index data into Elasticsearch, run Kibana alongside it so the data becomes available to both. Clone the project:

git clone https://bitbucket.org/atique1224/youtube_elasticsearch_indexing_tutorial.git

Reference tools used in this tutorial: Node.js, Visual Studio Code, MongoDB, and Robo 3T.

5. GET, POST, PUT, DELETE from Kibana Dev Tools

Kibana's Dev Tools console lets you run REST requests against Elasticsearch directly. Clone the project:

git clone https://bitbucket.org/atique1224/youtube_kibana_tutorial.git

6. Elasticsearch Aggregation and Projection

Aggregation lets you compute summaries over your data — sums, counts, and grouped buckets.

# Aggregation with a sum query
GET /students/_search
{
  "query": {
    "match_all": {}
  },
  "aggs": {
    "Casual_Leaves": {
      "sum": { "field": "leaves.CL" }
    }
  }
}

# Aggregation with a terms bucket (count of repeated values)
GET /students/_search
{
  "query": {
    "match_all": {}
  },
  "aggs": {
    "count": {
      "terms": {
        "field": "dept.keyword",
        "size": 100,
        "order": { "_key": "desc" }
      }
    }
  }
}

# Total count of records
GET /students/_count
{
}

Projection returns only the fields you ask for, using _source:

# Return only selected fields
GET /students/_search
{
  "_source": ["student_id", "skills"],
  "query": {
    "match_all": {}
  }
}

7. Elasticsearch Pagination and Scroll

Pagination. Suppose you have a huge index — ten million records or more. Returning them all to the frontend in one shot is not possible, so Elasticsearch offers two solutions. The first is pagination, using from and size to fetch one page at a time.

# Page through results: 10 at a time, starting at record 20
GET /students/_search
{
  "from": 20,
  "size": 10,
  "query": {
    "match_all": {}
  }
}

Scroll. The second solution is the scroll API. Each request creates a scroll ID and you set an expiry time, after which that scroll ID expires. It is designed for deep, sequential retrieval of large result sets rather than user-facing paging.

# Open a scroll that stays valid for 1 minute
GET /students/_search?scroll=1m
{
  "size": 1000,
  "query": {
    "match_all": {}
  }
}

# Fetch the next batch using the returned _scroll_id
GET /_search/scroll
{
  "scroll": "1m",
  "scroll_id": "PASTE_YOUR_SCROLL_ID_HERE"
}

Note: for deep pagination in current Elasticsearch, the search_after parameter with a point-in-time (PIT) is now the recommended approach over scroll for most use cases. Scroll still works but is considered legacy for real-time paging.

8. Visualising Elasticsearch Data in Kibana

To visualise Elasticsearch data, the data must first be present in Kibana, and you need to create an index pattern for the relevant indices. Good visualisations also depend on good data — the "4 Cs" of data quality:

• Correctness — validate data accuracy against an external reference.
• Currency — deliver new and updated content in a timely manner.
• Completeness — provide the right attributes and analysis so users have all the information they need to make decisions.
• Consistency — standardise identifiers and content across databases and products, so users get consistent information regardless of platform.

---

About the Author
Atique Ahmed — Principal AI Architect. 7x Microsoft MVP and Guinness World Record holder for Programming Excellence. Founder of Codez Tech.
Portfolio  |  LinkedIn  |  GitHub