OpenSearchCon India 2026

Kubernetes platforms generate massive volumes of logs from microservices, infrastructure, and platform services. At scale, OpenSearch observability pipelines often struggle with shard explosion, indexing bottlenecks, JVM pressure, and slow queries.

This session presents a practical architecture for operating an OpenSearch observability platform handling 100M+ logs per day from Kubernetes environments. We will walk through the end-to-end pipeline—from log collection to ingestion and indexing—and share performance engineering techniques used to maintain cluster stability under heavy workloads.

Topics include shard and index design, JVM and thread-pool tuning, optimizing indexing throughput, and using lifecycle policies and hot-warm architectures to scale efficiently. Attendees will gain actionable strategies for building resilient OpenSearch observability platforms for cloud-native systems.

This session is about the recent productivity study we ran in our open source lab (OSL). I’ll walk through the end-to-end architecture: extracting device telemetry, transforming it into task sessions, and running association rule mining on top of OpenSearch to compute support and confidence for goal-relevant activity patterns. The entire setup runs locally, therefore no risk of activity data theft.

ActivityWatch is cross platform open-source time tracker that collects telemetry on how we spend time on devices. It comes with watchers that can do all the data collection from AFK to browser windows. In our setup, ActivityWatch runs on each device, and OpenSearch is self-hosted on our lab’s local LAN (I'm academic), and then we ingest logs into it every 10 seconds using API-based ingestion. While ActivityWatch runs, users tag their intended task (e.g., #learn, #java). We align tags with telemetry windows, sessionize events into transactions (items[]=apps/domains, duration), and mine rules with support/confidence/lift per tag. If the current window drifts from the active tag with high confidence for a short period, we send a nudge reminding they are distracted from the original goal.

Most search systems are designed primarily for English. But what happens when we want search to truly understand regional languages like Tamil?

Tamil is one of the world’s oldest living languages with a rich literary tradition, yet building effective search for it comes with unique challenges such as complex morphology, script handling, tokenization, and language-specific relevance tuning.

In this talk, we share how we built an intelligent Tamil search system using OpenSearch. Using Tirukkural, the classic Tamil text of 1,330 couplets, as a practical dataset, we explore how OpenSearch can be adapted to understand Tamil queries and return meaningful results.

We’ll walk through practical techniques such as custom analyzers, tokenization strategies, stemming approaches, and relevance tuning for Tamil search. We’ll also demonstrate how users can search Kurals using natural Tamil phrases and still discover the most relevant verses.

Beyond Tamil, these techniques can help developers build better search experiences for many regional Indian languages bridging ancient knowledge with modern search technology.

Traditional keyword search is no longer sufficient for modern SRE workflows because infrastructure error messages are often too generic to be actionable. This session explores a technical shift in observability: moving from standard indexing to high-dimensional vector embeddings for terminal traces and post-mortems using OpenSearch.

We will deep-dive into a privacy-first AI architecture, demonstrating how to integrate OpenSearch’s vector engine with local, private models (such as Llama) to ensure sensitive production logs never leave your VPC.

You will learn the mechanics of building a Retrieval-Augmented Generation (RAG) pipeline designed specifically for infrastructure telemetry. The session includes a live demo where a local agent "recalls" the exact historical fix for a messy, real-time stack trace, providing a direct link to the relevant PR or configuration change from the past.

This session explores building and maintaining highly available OpenSearch clusters with robust disaster recovery strategies. We'll examine critical components of resilient OpenSearch infrastructure, focusing on multi-node cluster design across availability zones and effective backup solutions. The presentation covers essential aspects of shard allocation, replica configuration, and cross-cluster replication for continuous operation during failures. Learn practical approaches to automated failover, snapshot management, and monitoring strategies for optimal cluster health. Through real-world examples and lessons learned from large-scale implementations, we'll discuss common pitfalls and best practices for both cloud and on-premises deployments. This talk is designed for OpenSearch administrators, DevOps engineers, and architects managing mission-critical search infrastructure.

Key Takeaways:

-Understanding HA architecture patterns and DR strategies
-Implementing effective backup and recovery solutions
-Monitoring and maintaining cluster health
-Real-world examples and practical demonstrations
-Best practices for different deployment scenarios

Databases are the backbone of every application, yet monitoring them effectively remains a challenge. In this session, we walk through monitoring production databases powering an e-commerce platform Valkey for caching and PostgreSQL for the catalog deployed on Kubernetes & Docker, using OpenSearch and Prometheus as the observability backbone.

We demonstrate a complete OpenTelemetry-based pipeline: instrumenting both databases with OTel SDKs, routing telemetry through Data Prepper to store time-series metrics in Prometheus and traces/logs in OpenSearch. With AI-assisted investigation using text-to-PPL and text-to-PromQL, engineers query live telemetry in natural language asking "show me slow queries in the catalog service in the last hour" without writing complex syntax.

We also show how to use AI to generate RCA visualizations and surface performance insights across your database fleet. Whether you manage a handful of databases or hundreds, you'll leave with practical patterns for building an OpenTelemetry-native database observability stack.

Search finds. Discovery reveals. At Shutterstock, serving one of the world's largest licensed creative content libraries at 2000+ requests per second, the difference is everything.
Generative Discovery changes the contract entirely. It treats every interaction as a signal of intent, builds context across modalities, and applies cognition to surface what users did not know to ask for.
This talk introduces a production architecture built around three principles:
Intent: multimodal signals including text, image, and behavioral context unified into rich intent representations driving OpenSearch k-NN and hybrid retrieval.
Context: session aware, editorially grounded RAG pipelines using OpenSearch as a dynamic retrieval backbone at scale.
Cognition: generative agents that think, orchestrate multi step retrieval, and resolve ambiguous intent, treating OpenSearch as an intelligent reasoning substrate rather than a passive index.
You will learn from real production decisions, evolving thinking, and lessons still being learned on the frontier of Generative Discovery.

Debugging production incidents in distributed systems often means manually searching through thousands of log lines to understand what went wrong. During high-impact outages, engineers must quickly correlate signals across services, identify error patterns, and reconstruct the sequence of events.

In this talk, we explore how AI investigation agents can assist engineers by using OpenSearch as the investigation backbone for incident analysis. Instead of simply retrieving logs, an agent can iteratively query OpenSearch, identify dominant error patterns, correlate events across services, and build a timeline of failures before producing a concise root-cause explanation.

We will demonstrate a lightweight investigation agent diagnosing a simulated microservices failure using logs stored in OpenSearch. Attendees will see how engineers can ask questions like “Why did the ingestion service fail?” and watch the agent autonomously investigate and explain the incident.

Participants will leave with a practical architecture for building AI-assisted debugging workflows on top of OpenSearch.

Search engines are widely used but rarely understood at a deeper level. What actually happens between the moment a user submits a query and the results appear on the screen?
This session provides a technical deep dive into the OpenSearch query execution pipeline, exploring how user queries are processed, distributed, and ranked across a cluster. The talk begins with how text is analyzed and indexed using Lucene, including tokenization, analyzers, and inverted indexes. It then walks through how queries are parsed, executed across shards, and scored using relevance algorithms such as BM25.
Participants will also learn how OpenSearch coordinates distributed search, merges shard results, and optimizes performance through caching and query planning. Finally, we will examine debugging tools and profiling techniques that help developers understand and tune search relevance and performance.
By the end of the session, attendees will have a deeper understanding of the internals of OpenSearch and how those internals influence search quality and system performance.