Elasticsearch

Elasticsearch is a distributed search and analytics engine built on Apache Lucene. It excels at full-text search with relevance ranking, autocomplete, and fast aggregations over huge volumes of documents — the engine behind product search bars and log dashboards everywhere.

It's a complement to your primary database, not a replacement. Elasticsearch is where you index data for searching; your transactional database remains the source of truth. Treating it as the system of record is the classic mistake.

TL;DR

• Use it for search, relevance ranking, and analytics — not as your primary database.
• Define mappings intentionally; dynamic defaults can hurt.
• Distinguish text (full-text) from keyword (exact match/aggregations) fields.
• Plan operations: sharding, snapshots, index lifecycle, and cluster health.

Quick Example

Index a document, then run a full-text query that ranks by relevance:

Core Concepts

• Index — a collection of documents (loosely, a "table").
• Document — a JSON record.
• Mapping — the schema defining field types and how they're indexed.
• Analyzer — how text is tokenized and normalized for search.
• Shard / replica — how data is distributed and made redundant across the cluster.

Data Modeling

• Define explicit mappings for important fields — don't rely on dynamic mapping for everything.
• Use keyword fields for exact matches, filters, and aggregations.
• Use text fields (with analyzers) for full-text search.
• Denormalize for your search queries; Elasticsearch has no joins.

Operational Best Practices

• Monitor cluster health (green/yellow/red) and JVM heap/GC.
• Use Index Lifecycle Management (ILM) for rollover and retention (logs especially).
• Take snapshots for backup/restore.
• Keep an eye on shard count — too many small shards waste resources.

Common Mistakes

Letting dynamic mapping explode

Treating Elasticsearch as the source of truth

FAQ

When should I use Elasticsearch?

When you need rich full-text search (relevance ranking, fuzzy matching, autocomplete) or fast aggregations/analytics over large document or log volumes. For transactional, relational data, use a database like PostgreSQL — Elasticsearch sits alongside it for search.

Can Elasticsearch be my primary database?

It's not designed to be. It lacks the transactional guarantees and consistency model of a primary database, and its near-real-time indexing can briefly lag. Keep your source of truth in a transactional store and index into Elasticsearch.

What's the difference between text and keyword fields?

text fields are analyzed (tokenized, lowercased) for full-text search — great for matching within prose. keyword fields are stored verbatim for exact matches, filtering, sorting, and aggregations. Many fields are mapped as both.

Elasticsearch or OpenSearch?

OpenSearch is the Apache-2.0 fork created after Elasticsearch's license change. They're closely related and both production-grade; choose based on licensing preferences, managed-service availability (AWS offers OpenSearch), and the specific features you need.

Related Topics

• Databases — Where ES fits among data stores
• Logging — Log search and analytics (ELK)
• System Design — Search architecture
• PostgreSQL — Has built-in full-text search for simpler needs
• Caching — Complementary read acceleration

References

• Elasticsearch — Documentation
• Elasticsearch — Mapping
• Elasticsearch — Index lifecycle management
• OpenSearch