Scribd
Upload
11 views5 pages

ELK Stack Training: Elasticsearch & Logstash

The document outlines the hardware requirements and prerequisites for understanding the Elastic ecosystem for log collection, aggregation, and visualization using the ELK stack. It provides a detailed curriculum for a three-day training program covering topics such as Elasticsearch architecture, Logstash installation, search techniques, and Kibana visualization. Additionally, it includes performance improvement strategies and monitoring cluster APIs for Elasticsearch.

Uploaded by

Suresh
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
11 views5 pages

ELK Stack Training: Elasticsearch & Logstash

The document outlines the hardware requirements and prerequisites for understanding the Elastic ecosystem for log collection, aggregation, and visualization using the ELK stack. It provides a detailed curriculum for a three-day training program covering topics such as Elasticsearch architecture, Logstash installation, search techniques, and Kibana visualization. Additionally, it includes performance improvement strategies and monitoring cluster APIs for Elasticsearch.

Uploaded by

Suresh
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Hardware Requirement:

1. Windows or Linux machine with minimum 8 GB


RAM
2. All the systems(Laptop) should be in the intranet
3. JDK1.8 should be installed and JAVA_HOME path
should be set.

Prerequisite:
1. Knowledge of Python OR Java
2. Understanding of SQL queries

Understanding Elastic eco system for log collection,


aggregation and visualization (ELK stack)
Making your content searchable is not about exposing content to Google for indexing; it's
about providing your own tailor made search, within your system to your users. Large
players like Twitter and LinkedIn have been using Lucene for years, and today is the day you
will see how easy it is to join their club by learning Elasticsearch from the ground up.

ELK Stack: Elastic Search, Log stash and Kibana

Table of Contents

Day 1
 Introduction
 Elastic Stack Overview
 Logs and Problems associated with it
 Role of Logstash for performing ETL
 Logstash Installation and Configuration
 Understanding and using Inputs, Filters, and Outputs
 How to apply Conditionals in filters
Elasticsearch
 Understanding Elastricsearch NoSQL architecture
 CAP theorm for elasticserch
 Comparing and contrasting Solr, Lucene and Elasticseach

JSON Structure
o Nested JSON
o Different Data Types

 Understanding Elastricsearch NoSQL architecture


 CAP theorm for elasticserch
 Comparing and contrasting Solr, Lucene and Elasticseach
 Understanding the Elasticsearch Architecture and different types of Nodes
 Different nodes and its uses
Master node
Data nodes
Ingest node
Tribe node

 Understanding the Elasticsearch Architecture and different types of Nodes
 Inside the shards , node and performing cluster installation
 Index Management :
 Multi Tenancy – Multiple indices, index aliases, cross index operations and introduction to
data flow
 Elasticsearch Index – Analysis, mappings, index operations, versioning, APIs and settings
 Create
Read
Update
Delete
Data Modelling
Mapping
Aliases

Day 2
 Performing different kinds of search : term, terms, match,multi_match,
fuzzy,regexp and wildcard
 Query DSL
Full Text Searches
Structured Searches
Proximity
Partial Matching
Analysis (Advance Topic)
Analyzers
Tokenizers
Token Filters
Char Filters

Aggregations : Performing different kinds of aggregation such as


• percentile_ranks
• cardinality
• significant_terms
• top hits
• scripted metric
• filters
• range
• geohash
• terms
• histogram
• date_histogram
• stats
• extended stats
• min / max
• sum

Suggestions (Advance Topic) :


• Term suggestion
• Phrase Suggestion
• Completion Suggestion
• Context Suggestion

Highlighting : Showing snippets of text matching the query


• highlighter
• FVH: fast-vector-highlighter
• Postings

 Writing JAVA programme for performing search and aggregation

 Understand relevancy (Advance Topic):


Boosting
Custom Scoring
• Boost
• Random
• Decay Function
• Script
• Field Value Factor
• Weight

 Index Template
 Index Alias
 Re-indexing from one index into another
 Plugin management
 Elasticseach backup and restore mechanism

How To Improve performances

 General recommendations
 Recipes
 Mixing exact search with stemming
 Getting consistent scoring
 Tune for indexing speed
 Tune for search speed
 Tune for disk usage

Monitoring clusters in Elasticsearch

Cluster APIs

 Cluster Health
 Cluster State
 Cluster Stats
 Pending cluster tasks
 Cluster Reroute
 Cluster Update Settings
 Cluster Get Settings
 Nodes Stats
 Nodes Info
 Nodes Feature Usage
 Remote Cluster Info
 Task Management API
 Nodes hot_threads
 Cluster Allocation Explain API

Day 3

Logstash :

 Installing Logstash
 Stashing Your First Event
 Structure of a Config File
 Accessing Event Data and Fields in the Configuration
 Using Environment Variables in the Configuration
 Logstash Configuration Examples
 Reloading the Config File
 Managing Multiline Events
 Glob Pattern Support
 Advanced Pipeline
 Multiple Output Plugins

FileBeat (Advance Topic)

 Installing Filebeat
 Configuring Filebeat
 Configuring Filebeat to Use Logstash
 Loading the Index Template in Elasticsearch
 Starting Filebeat
 Loading the Kibana Index Pattern

Kibana
 Using Kibana to visualize the ES data
 Settings
 Time Picker, Search, and Filters
 Kibana Discover, Visualization, and Dashboard Interfaces
 Lab
 Handling Back Pressure in ELK stack
 Deployment Architectures
 Hardware Best Practices
 Operation Checklist
 Debugging and Monitoring
 APM
 Stack Monitoring
 Saved Objects - importing / exporting
 Canvas
 Security Setting - roles and users
 REST APIs
 Spaces
 3rd Party plugin

How To Improve performances

 General recommendations
 Recipes
 Mixing exact search with stemming
 Getting consistent scoring
 Tune for indexing speed
 Tune for search speed
 Tune for disk usage

You might also like