Calculate storage needed to retain logs for a given period at your ingestion rate.
Total log storage scales linearly with daily volume, retention window and replica count, and is reduced by whatever compression ratio your storage backend achieves at rest. Formula: storage = daily_volume × retention_days × replicas / compression_ratio. This is the core sizing calculation for provisioning Elasticsearch/OpenSearch data nodes, S3/object storage buckets, or any log pipeline's storage tier.
storage_GB = daily_volume_GB × retention_days × replicas / compression_ratio
Plain-text structured logs (JSON) typically compress 5-10× with standard algorithms (gzip, zstd); highly repetitive log formats can reach higher ratios, while already-compact binary formats compress less. Measure your own backend's actual ratio for accuracy.
Distributed stores like Elasticsearch/OpenSearch keep replica shards for durability and read scaling, and multi-region setups may keep full copies elsewhere — each replica consumes its own storage, so total capacity must account for all copies, not just the primary.
Often both — compliance requirements (e.g. security logs kept 1 year+) set a floor, but cost usually drives tiering: keep recent logs on fast/expensive storage and move older logs to cheaper cold storage or delete them once compliance windows pass.
No — this estimates the compressed log payload only. Search engines like Elasticsearch add index overhead (inverted indices, doc values) that can add 20-50%+ on top of the raw compressed size depending on mapping and field count.