Everything that touched your storage, and everything that stopped
The same access record that powers detection answers the operational questions. No instrumentation, no catalog access, no agents.
Illustrative data.
What reCost keeps queryable
Data flow
Every request mapped from identity to bucket to prefix to operation, with volumes and direction across the estate.
Identity and object trace
Give it an identity and see everything it touched. Give it an object key and see everything that touched it.
Pipeline health
Write cadence learned per prefix, so a writer that goes silent surfaces on its own.
Asset inventory
Every bucket and prefix with owners, access classes, coverage state, dormant share, and storage class distribution.
Error tracking
Denials and failures by bucket, prefix, operation, identity, and client, classified by cause rather than counted.
Data lake access
Iceberg, Delta, and Hudi resolved from the object layer, including readers that skipped the catalog.
Coverage map
Which buckets are logged, which are not, and where other tooling is being denied.
What it looks like in your account
Illustrative data.
Storage class distribution
Request profile
Identity relationships
| Identity | Class | Requests | First seen | State |
|---|---|---|---|---|
PE_Role Machine | Application | 99% ok · 72M req | ≥90 days | established |
svc:logdelivery… Service | AWS Service | 100% ok · 5.8M req | ≥90 days | established |
AmazonS3 Service | S3 Lifecycle | 100% ok · 2.7M req | ≥90 days | established |
snowflake-role Machine | Data Platform | 100% ok · 1.4M req | ≥90 days | established |
Daily ingestion
last 14 daysPipeline health
Job success does not mean data arrived. A Glue job reading an empty upstream partition completes cleanly and writes nothing. Firehose buffers records that never flush. reCost learns the write cadence of every prefix and pages when the silence exceeds it, with no instrumentation in the job.
Write cadence by prefix
| Prefix | Writer | Expected | Last write | |
|---|---|---|---|---|
| events-raw/dt=… | firehose-clickstream | every 5 min | 2 min ago | healthy |
| warehouse/orders/ | glue-orders-nightly | daily 02:00 | 4 h ago | healthy |
| exports/partner/ | msk-connect-s3 | hourly | 38 min ago | late |
| warehouse/billing/ | glue-billing-etl | daily 03:00 | 17 d ago | stale |
| ml-features/v3/ | spark-feature-build | every 6 h | 9 d ago | stale |
2 prefixes stale beyond their inferred SLO
Illustrative data.
Data lake access
Iceberg, Delta, and Hudi tables all resolve to objects. reCost reconstructs table health from S3 Inventory and access log patterns, so it works across all three without engine-specific instrumentation, and shows which readers came through the catalog and which went straight to the files.
Snapshot counts, manifest bloat, orphaned files, small file accumulation, compaction lag, and checkpoint age, per table.
Four pipelines stopped writing. No alarm ever fired.
Job success does not mean data arrived. Write cadence per prefix is a direct measure of pipeline liveness, and it is already in your access logs.
Signals that exist nowhere else
01Out-of-catalog reads. Which readers pulled Parquet or manifest files directly, bypassing the catalog entirely.
02External readers. Which identities outside your organization read from your buckets, what they read, and how much.
03Presigned URL consumers. The signing role appears in the log. The client that actually retrieved the object appears only in the user agent.
04Silent pipelines. Last write timestamp per prefix per writer identity, against a learned cadence.
05Broken integrations. Writes that authenticate but never land, so the pipeline reports success while the data does not exist.
06Coverage gaps. Which buckets have no access logging at all, and where your other tooling is being denied.
07Iceberg. Snapshot count per table, manifest bloat, and whether expire_snapshots has ever run.
08Delta Lake. Log depth, checkpoint age, orphaned file volume, and small file ratios.
09Hudi. Compaction lag, timeline file counts, and log-to-base-file ratio per partition.
Observability, answered
No. Table health is derived from S3 Inventory and access log patterns, which is why it works across Iceberg, Delta and Hudi without engine-specific instrumentation.
Start with proof,
not a pitch.
Scoped read-only role, 30-day lookback, results in 48 hours.