Log Search Performance Calculator
Estimate log search query time from index size and search throughput, and see the impact of query optimization techniques.
Inputs
Total size of the index (or shard set) being searched.
Effective scan throughput of the search cluster for the query pattern in question.
Fraction of the index actually scanned after applying filters, time-range narrowing and field selectivity (1 = full scan, 0.1 = only 10% scanned).
Estimated Search Time
250.00seconds
Speedup vs Full Scan
1.00×
Time Saved by Optimization
0.00seconds
Unoptimized Search Time
250.00seconds
Step by step
Effective scan volume: index × optimization factor
500 × 1
= 500.00 GB
Search time: effective scan ÷ throughput
500.00 ÷ 2
= 250.00s
Unoptimized (full scan) search time
500 ÷ 2
= 250.00s
How it works
Log search time is a function of how much data must actually be scanned relative to the search cluster's throughput: search_time = index_size ÷ search_throughput, but effective scan volume shrinks dramatically once filters (time range, indexed fields, term filters) narrow the search space: search_time = (index_size × optimization_factor) ÷ search_throughput. A tight time-range filter combined with selective field queries is usually the single biggest lever for search latency.
Formula
search_time_sec = data_scanned_GB / scan_throughput_GBps
- data_scanned_GB
- Volume of log data scanned for the query (GB)
- scan_throughput_GBps
- Search engine scan throughput (GB/sec)
Frequently Asked Questions
What's the single biggest lever for faster log search?
Narrowing the time range is usually the most effective optimization, since most log search backends partition data by time (daily/hourly indices) and can skip entire index segments outside the requested range without scanning them at all.
How does field selectivity affect the optimization factor?
Querying on low-cardinality, indexed fields (e.g. exact service name or log level) lets the search engine use inverted indices to jump directly to matching documents, versus full-text search across unindexed or highly variable fields which requires scanning much more data.
Does index size or shard count matter more for search speed?
Both matter — larger indices take proportionally longer to scan per this model, but poor shard sizing (too few large shards, or too many tiny shards) also hurts parallelism and can dominate actual query latency beyond what this simplified model captures.
Why is my actual search slower than this estimate?
Cluster resource contention (CPU, JVM heap pressure, concurrent queries), cold caches on infrequently-queried indices, and complex aggregations beyond a simple filtered scan can all add latency this linear throughput model doesn't capture.
You might also need
- Log Parsing Performance CalculatorCommonly used together
- Log Ingestion CalculatorCommonly used together
- OpenSearch Index Size CalculatorCommonly used together
- Observability Cost CalculatorCommonly used together
- Fluent Bit Throughput CalculatorAlso in Logging & Observability
- Fluentd Buffer CalculatorAlso in Logging & Observability