Work out which Firehose buffer hint fires first, the resulting delivery latency and object size, and the 5 KB record rounding cost.
Firehose flushes when either hint is satisfied, whichever comes first, so on a busy stream the size hint dictates latency and on a quiet one the interval does. That single choice sets both your end-to-end freshness and your destination object size, and small objects are what make Athena and Spark queries slow. Billing rounds every record up to 5 KB, so 1 KB records cost five times what their raw volume suggests — confirm the current per-GB rate for your region. Buffer settings are the one Firehose knob that trades latency against query performance and cost simultaneously: a 1 MB buffer gives fresh data and thousands of tiny files, while a 128 MB buffer gives efficient Parquet objects and minutes of lag.
Firehose Buffer
time to fill = buffer size MB ÷ throughput MB/s; delivery latency = min(time to fill, buffer interval); average object = throughput × delivery latency.
5 KB record rounding
billed ingest GB = records/s × ceil(record KB ÷ 5) × 5 × seconds per month ÷ 1,048,576.
time to fill = buffer size MB ÷ throughput MB/s; delivery latency = min(time to fill, buffer interval); average object = throughput × delivery latency. Firehose flushes when either hint is satisfied, whichever comes first, so on a busy stream the size hint dictates latency and on a quiet one the interval does. That single choice sets both your end-to-end freshness and your destination object size, and small objects are what make Athena and Spark queries slow. Billing rounds every record up to 5 KB, so 1 KB records cost five times what their raw volume suggests — confirm the current per-GB rate for your region.
Buffer settings are the one Firehose knob that trades latency against query performance and cost simultaneously: a 1 MB buffer gives fresh data and thousands of tiny files, while a 128 MB buffer gives efficient Parquet objects and minutes of lag.
This calculator takes 7 inputs: Records per second, Average record size, Buffer size hint, Buffer interval hint, Compressed size as a share of raw, Firehose ingestion price per GB, Destination storage price per GB-month. The pre-filled defaults are a realistic starting point — replace them with figures from your own environment for a result you can act on.
For Athena, Spark or Redshift Spectrum, objects of roughly 128 MB or more read far more efficiently than many small files. If freshness forces a small buffer, compact the output later with a scheduled job rather than fighting the buffer hint.
Yes. A Lambda transformation applies its own buffering before delivery, and the billed ingest is measured on the data Firehose receives, not on the smaller output your transformation produces. Dropping records in a transform does not reduce the ingest charge.