S3 Access Logs: Why They Never Reach Your SIEM

S3 access logs record who read every object, but at terabyte scale, per-gigabyte SIEM ingestion makes full coverage a six-figure line item, so teams drop them. Here is why, and how to keep read visibility without the ingestion tax.

Is your SIEM budget choosing your blind spots?

Security teams do not skip S3 access logs because the reads are safe. They skip them because the logs are expensive to ingest. S3 server access logs at real scale run to terabytes a month, one line per request, and a SIEM prices ingestion by the gigabyte. At that volume, full coverage becomes a six-figure line item, so the logs get sampled, scoped to a handful of buckets, or dropped at the collector.

The one telemetry source that records who read which object never reaches the tool built to catch the breach. Not for lack of risk. For budget.

One security leader put it plainly: in practice, we accept the risk.

That is not negligence. It is economics. No team can justify paying potentially six figures to ingest logs they may open twice a year, after the fact, during an incident. But hope is not detection, and the gap stays invisible until the day it is not.

Why the SIEM never sees S3 object reads

Most detection pipelines were built around control-plane events and network telemetry. S3 object reads are neither. They are high-volume data-plane events, and they behave badly at every stage of a traditional SIEM pipeline.

• Volume. A single analytics job can fan out into millions of GETs against one prefix in minutes. Normal traffic reads to a SIEM like a flood.
• Cost model mismatch. SIEM pricing rewards low-volume, high-signal sources. S3 read logs are high-volume and low-signal per line, so the economics push teams to drop the source.
• Delivery lag. Server access logs are best-effort and can arrive hours late. Real-time rules do not fire on events that land three hours after the read.
• No baseline. Even when ingested, a rule needs a notion of normal. "A principal reads an object" is true billions of times a day. Without a per-prefix baseline of who normally touches which data, there is nothing to alert on.

The result is a source that is technically available, occasionally ingested, and almost never turned into a working detection.

What those logs are the only record of

The requester is already written on every read. A single server access log line records the principal, the operation, the object key, the timestamp, the status, the bytes moved, and the user-agent:

arn:aws:iam::4444-5555-6666:user/etl-batch  REST.GET.OBJECT  datalake/prod/events/year=2022/month=07/part-00381.parquet  200  4194304  "aws-sdk-java/1.12.470"

When an incident happens, that record is often the only evidence showing:

• Who accessed the object
• What was read or written
• When it happened
• Which application, SDK, service, or identifiable client made the request

Server access logs are free to generate; you pay only S3 storage for the log objects. The data exists. The requester field exists. The reason it never triggers an alert is that it never reaches a query.

What the requester field shows at scale

From the vantage point of more than 100 billion S3 requests a month, one pattern holds across almost every large bucket.

Most prefixes are read by three or fewer principals, and that set stays stable over time.

An ETL role, a service account, a query engine's execution role. The requester on a given prefix is remarkably consistent, and that consistency is what makes new access legible.

When something changes, it usually takes one of two shapes. New principal access: a prefix read by two identities is suddenly read by a third that has never appeared against it. Or an anomalous access pattern: cold data untouched for months receives a burst of GETs, or a principal that read one object a day starts reading thousands.

Neither shape tells you a cause. A new principal on cold data might be a new analyst, a misconfigured job, or something worth a page. The log records what happened, not intent. The value is that both are observable on metadata alone, without ever reading object contents.

A better architecture: process at the source

The fix is not to send every raw event into the SIEM. It is to process the logs where they land and forward only what matters.

• Read the server access logs at the source in S3. No ingestion tax.
• Baseline the requester per prefix, so a new principal on cold data surfaces on its own.
• Score the odd records and forward only those to the SIEM.
• Keep the terabytes in S3, where storage is cheap.

Your SOC keeps the correlation and case management it already runs. It just receives findings instead of a firehose, and the ingestion bill that forced the accept-the-risk call goes away.

What reCost automates

reCost does this continuously on S3 server access logs, plus S3 Inventory for object state. It learns the requester baseline for every bucket and prefix, watches for new principal access and anomalous access patterns, and forwards only the scored anomalies to your security stack.

It operates on metadata alone. It never reads object contents, never blocks a request, and never classifies data. It reports what the logs show: a new principal against a prefix, a burst against cold keys, a read against data idle for months. Detection and visibility, nothing more.

Because the logs never leave S3, there is no per-gigabyte ingestion decision to make. Coverage stops being a budget line.

FAQ

Why does my SIEM miss S3 object reads? The logs are usually dropped before ingestion. Per-gigabyte SIEM pricing makes terabytes of S3 read logs uneconomical, so the source is scoped down or never wired up. It exists; it just never reaches a query.

Do S3 server access logs record who read an object? Yes. Each record includes the requester, the account or IAM principal, the operation, the object key, the timestamp, and the user-agent. It is the field most detection pipelines never query.

Are S3 server access logs free? They are free to generate. You pay only S3 storage for the log objects. The cost that stops teams is SIEM ingestion, not generation.

Does this identify the cause of an anomaly? No. The logs show new principal access and anomalous access patterns. They record what happened, not intent. Investigation is still a human step.

Takeaway

The requester is already logged on every S3 read. The reason your SIEM never flags it is a budget line, not a blind spot in the data. Process the logs where they land, baseline the requester per prefix, and forward only the anomalies. The terabytes stay in S3. The signal reaches the SOC. Hope stops being the control.