OpenSearchCon India 2026

Kubernetes platforms generate massive volumes of logs from microservices, infrastructure, and platform services. At scale, OpenSearch observability pipelines often struggle with shard explosion, indexing bottlenecks, JVM pressure, and slow queries.

This session presents a practical architecture for operating an OpenSearch observability platform handling 100M+ logs per day from Kubernetes environments. We will walk through the end-to-end pipeline—from log collection to ingestion and indexing—and share performance engineering techniques used to maintain cluster stability under heavy workloads.

Topics include shard and index design, JVM and thread-pool tuning, optimizing indexing throughput, and using lifecycle policies and hot-warm architectures to scale efficiently. Attendees will gain actionable strategies for building resilient OpenSearch observability platforms for cloud-native systems.

This session is about the recent productivity study we ran in our open source lab (OSL). I’ll walk through the end-to-end architecture: extracting device telemetry, transforming it into task sessions, and running association rule mining on top of OpenSearch to compute support and confidence for goal-relevant activity patterns. The entire setup runs locally, therefore no risk of activity data theft.

ActivityWatch is cross platform open-source time tracker that collects telemetry on how we spend time on devices. It comes with watchers that can do all the data collection from AFK to browser windows. In our setup, ActivityWatch runs on each device, and OpenSearch is self-hosted on our lab’s local LAN (I'm academic), and then we ingest logs into it every 10 seconds using API-based ingestion. While ActivityWatch runs, users tag their intended task (e.g., #learn, #java). We align tags with telemetry windows, sessionize events into transactions (items[]=apps/domains, duration), and mine rules with support/confidence/lift per tag. If the current window drifts from the active tag with high confidence for a short period, we send a nudge reminding they are distracted from the original goal.

Most search systems are designed primarily for English. But what happens when we want search to truly understand regional languages like Tamil?

Tamil is one of the world’s oldest living languages with a rich literary tradition, yet building effective search for it comes with unique challenges such as complex morphology, script handling, tokenization, and language-specific relevance tuning.

In this talk, we share how we built an intelligent Tamil search system using OpenSearch. Using Tirukkural, the classic Tamil text of 1,330 couplets, as a practical dataset, we explore how OpenSearch can be adapted to understand Tamil queries and return meaningful results.

We’ll walk through practical techniques such as custom analyzers, tokenization strategies, stemming approaches, and relevance tuning for Tamil search. We’ll also demonstrate how users can search Kurals using natural Tamil phrases and still discover the most relevant verses.

Beyond Tamil, these techniques can help developers build better search experiences for many regional Indian languages bridging ancient knowledge with modern search technology.

Traditional keyword search is no longer sufficient for modern SRE workflows because infrastructure error messages are often too generic to be actionable. This session explores a technical shift in observability: moving from standard indexing to high-dimensional vector embeddings for terminal traces and post-mortems using OpenSearch.

We will deep-dive into a privacy-first AI architecture, demonstrating how to integrate OpenSearch’s vector engine with local, private models (such as Llama) to ensure sensitive production logs never leave your VPC.

You will learn the mechanics of building a Retrieval-Augmented Generation (RAG) pipeline designed specifically for infrastructure telemetry. The session includes a live demo where a local agent "recalls" the exact historical fix for a messy, real-time stack trace, providing a direct link to the relevant PR or configuration change from the past.

This session explores building and maintaining highly available OpenSearch clusters with robust disaster recovery strategies. We'll examine critical components of resilient OpenSearch infrastructure, focusing on multi-node cluster design across availability zones and effective backup solutions. The presentation covers essential aspects of shard allocation, replica configuration, and cross-cluster replication for continuous operation during failures. Learn practical approaches to automated failover, snapshot management, and monitoring strategies for optimal cluster health. Through real-world examples and lessons learned from large-scale implementations, we'll discuss common pitfalls and best practices for both cloud and on-premises deployments. This talk is designed for OpenSearch administrators, DevOps engineers, and architects managing mission-critical search infrastructure.

Key Takeaways:

-Understanding HA architecture patterns and DR strategies
-Implementing effective backup and recovery solutions
-Monitoring and maintaining cluster health
-Real-world examples and practical demonstrations
-Best practices for different deployment scenarios

Databases are the backbone of every application, yet monitoring them effectively remains a challenge. In this session, we walk through monitoring production databases powering an e-commerce platform Valkey for caching and PostgreSQL for the catalog deployed on Kubernetes & Docker, using OpenSearch and Prometheus as the observability backbone.

We demonstrate a complete OpenTelemetry-based pipeline: instrumenting both databases with OTel SDKs, routing telemetry through Data Prepper to store time-series metrics in Prometheus and traces/logs in OpenSearch. With AI-assisted investigation using text-to-PPL and text-to-PromQL, engineers query live telemetry in natural language asking "show me slow queries in the catalog service in the last hour" without writing complex syntax.

We also show how to use AI to generate RCA visualizations and surface performance insights across your database fleet. Whether you manage a handful of databases or hundreds, you'll leave with practical patterns for building an OpenTelemetry-native database observability stack.

Search finds. Discovery reveals. At Shutterstock, serving one of the world's largest licensed creative content libraries at 2000+ requests per second, the difference is everything.
Generative Discovery changes the contract entirely. It treats every interaction as a signal of intent, builds context across modalities, and applies cognition to surface what users did not know to ask for.
This talk introduces a production architecture built around three principles:
Intent: multimodal signals including text, image, and behavioral context unified into rich intent representations driving OpenSearch k-NN and hybrid retrieval.
Context: session aware, editorially grounded RAG pipelines using OpenSearch as a dynamic retrieval backbone at scale.
Cognition: generative agents that think, orchestrate multi step retrieval, and resolve ambiguous intent, treating OpenSearch as an intelligent reasoning substrate rather than a passive index.
You will learn from real production decisions, evolving thinking, and lessons still being learned on the frontier of Generative Discovery.

Debugging production incidents in distributed systems often means manually searching through thousands of log lines to understand what went wrong. During high-impact outages, engineers must quickly correlate signals across services, identify error patterns, and reconstruct the sequence of events.

In this talk, we explore how AI investigation agents can assist engineers by using OpenSearch as the investigation backbone for incident analysis. Instead of simply retrieving logs, an agent can iteratively query OpenSearch, identify dominant error patterns, correlate events across services, and build a timeline of failures before producing a concise root-cause explanation.

We will demonstrate a lightweight investigation agent diagnosing a simulated microservices failure using logs stored in OpenSearch. Attendees will see how engineers can ask questions like “Why did the ingestion service fail?” and watch the agent autonomously investigate and explain the incident.

Participants will leave with a practical architecture for building AI-assisted debugging workflows on top of OpenSearch.

Search engines are widely used but rarely understood at a deeper level. What actually happens between the moment a user submits a query and the results appear on the screen?
This session provides a technical deep dive into the OpenSearch query execution pipeline, exploring how user queries are processed, distributed, and ranked across a cluster. The talk begins with how text is analyzed and indexed using Lucene, including tokenization, analyzers, and inverted indexes. It then walks through how queries are parsed, executed across shards, and scored using relevance algorithms such as BM25.
Participants will also learn how OpenSearch coordinates distributed search, merges shard results, and optimizes performance through caching and query planning. Finally, we will examine debugging tools and profiling techniques that help developers understand and tune search relevance and performance.
By the end of the session, attendees will have a deeper understanding of the internals of OpenSearch and how those internals influence search quality and system performance.

Organizations generate huge amounts of internal documentation across wikis, runbooks, architecture guides, and operational documents. Finding the right information quickly can be difficult when traditional keyword search cannot capture the intent behind a question.

In this talk, I will share how a Retrieval Augmented Generation assistant can be built using OpenSearch vector search to retrieve relevant knowledge from enterprise documentation. The system converts user queries into embeddings, retrieves the most relevant documents from OpenSearch, and provides them to a language model to generate grounded answers.

The session will walk through the architecture of the system, including document ingestion, embedding generation, vector indexing, and semantic retrieval. We will also discuss strategies for improving retrieval quality, managing latency in the query pipeline, and ensuring that enterprise data remains secure.

Attendees will learn how OpenSearch can power scalable semantic search systems and how vector search can be combined with language models to build reliable knowledge assistants for modern engineering organizations.

Modern distributed applications generate massive volumes of telemetry in the form of logs, metrics, and traces. Designing a scalable pipeline to collect, process, and analyze this data is essential for effective observability.

In this session, we explore how OpenTelemetry and OpenSearch can be combined to build a modern observability pipeline. We walk through the end-to-end telemetry flow - from instrumenting applications with OpenTelemetry SDKs to collecting and processing telemetry through the OpenTelemetry Collector and finally indexing and analyzing data in OpenSearch.

We also discuss key design considerations such as telemetry ingestion patterns, index strategies for observability workloads, and correlating logs, metrics, and traces. Attendees will see how OpenSearch can act as a scalable backend for storing, searching, and analyzing observability data.

Audience Takeaway:
Learn how to design a scalable observability pipeline using OpenTelemetry for telemetry collection and OpenSearch for storage and analysis, along with key considerations for ingesting and correlating logs, metrics, and traces.

Description
Join us to explore the latest performance improvements in OpenSearch, highlighting the integration of Lucene’s bulk collection API and enhancements to the gRPC Search API. This session will dive into how bulk collection optimizes aggregation execution, delivering measurable gains across analytical workloads by batching operations and reducing computational overhead. We will also cover the expanded features of the gRPC Search API in OpenSearch 3.4, which now supports new query types, improved bulk requests, and multiple document formats. To make these concepts clear and practical, we will showcase live demos illustrating performance boosts in aggregation workloads and expanded query handling through gRPC. Attendees will gain a clear understanding of how these advancements significantly enhance efficiency and broaden query capabilities in real‑world use cases. Key takeaways include understanding Lucene’s bulk collection benefits, learning about new gRPC query types, and discovering improvements in bulk request handling and document format support.

This session explores how to build a comprehensive code security platform using OpenSearch's powerful analytics capabilities enhanced with AI agentic workflows. I'll demonstrate how to implement a system that continuously monitors, analyzes, and remediates security vulnerabilities across your codebase. Attendees will learn how to leverage OpenSearch for storing and analyzing code security telemetry, while implementing agentic AI workflows that autonomously identify patterns, prioritize vulnerabilities, and suggest remediation strategies.

Key topics include:
• Configuring OpenSearch for efficient storage and querying of code security data
• Implementing vector search to identify similar vulnerability patterns across repositories
• Building AI agents that autonomously scan, analyze, and classify security issues
• Creating agentic workflows that coordinate specialized security analysis tasks
• Developing automated remediation suggestions using LLM-powered agents
• Visualizing security insights through OpenSearch Dashboards

This session provides a practical blueprint for security teams looking to scale their code security.

Large Language Models (LLMs) power some of today’s most advanced AI applications-from chatbots to intelligent copilots-yet monitoring their complex behavior remains a major challenge. This session demonstrates how OpenTelemetry’s auto-instrumentation capabilities, combined with OpenSearch’s powerful analytics, provide end-to-end observability for LLM-based systems.

You’ll learn how to automatically collect rich telemetry data-traces, metrics, and logs-from your LLM applications with minimal manual effort using OpenTelemetry and libraries like OpenLIT. See how this data flows into OpenSearch, enabling real-time analysis, anomaly detection, and performance monitoring tailored specifically for AI workloads.

By the end of this talk, you will get the tools and best practices to build scalable, AI-driven observability pipelines that keep your LLM applications reliable and performant.