OpenSearchCon India 2026

Operating OpenSearch at large scale with multi-tenant, hierarchical data introduces serious sharding challenges. Queries can fan out to hundreds of shards, hot shards reduce throughput, and rebalancing large clusters becomes operationally difficult.

This session introduces HSplit, a production-proven intelligent sharding system designed for OpenSearch deployments managing millions of folders, billions of documents, and petabytes of data. HSplit analyzes hierarchical structure, access-control patterns, and usage behavior to automatically determine optimal partition boundaries.

We’ll show how HSplit reduced per-query shard fanout from 200+ shards to single digits (95% reduction) and improved P95 latency from 2–5 seconds to 100–300 ms (10× faster) while maintaining stable performance as data grows. The session covers access-aware partitioning, composite scoring strategies, constraint-driven splitting, and stateless routing that supports thousands of queries per second.

Attendees will gain practical strategies, architectural patterns, and decision frameworks for designing efficient sharding strategies for large-scale, multi-tenant OpenSearch environments.