SOLUTIONS / DATA ENGINEERS

Stop debugging pipelines from query logs

See every table, pipeline, and partition at the object level. Know what's healthy, what's degrading, and what went silent, before downstream jobs or users feel it.

Book a Demo
THE PROBLEM

Data lake problems are invisible until it's too late

Pain point 1

Silent pipelines

A writer stops. Nothing alerts. Downstream consumers read stale data for hours or days before someone notices a wrong number in a dashboard.

Pain point 2

Metadata bloat

Iceberg and Delta tables accumulate thousands of snapshots and checkpoint files. Query performance degrades gradually. There's no view into the object layer to catch it early.

Pain point 3

No object-level visibility

CloudWatch shows bucket sizes. Query engines show query results. Nobody shows you what's actually happening at the file level across your entire lakehouse.

HOW RECOST HELPS

Object-level visibility into every table and pipeline

reCost reads your S3 access logs and object metadata without touching your data. It builds a live picture of every table, writer, and partition across your lakehouse, so you can catch problems before they surface in production.

  • Iceberg, Delta, and Hudi table health scored automatically
  • Silent writer detection with last-write timestamp and row-count alerts
  • Iceberg snapshot bloat and Delta _delta_log growth tracking
  • Small file detection: flags partitions exceeding 10,000 files under 128 MB
  • Per-table Athena and Spark query cost mapped to teams and pipelines
  • Query engine observability: Athena, Glue, Spark, Trino
  • Broken compaction and MOR compaction lag detection for Apache Hudi
Pipeline health feed
events.raw_clickstream
42,015 snapshots
Degraded
billing.transactions_v2
Compaction 6h ago
Healthy
users.profiles_iceberg
Silent 38h
Warning
analytics.sessions_delta
3 writers active
Healthy
Iceberg snapshot bloat alert
events.raw_clickstream
42,015 Iceberg snapshots detected. No expiry policy configured. Estimated 2.4TB of orphaned metadata files. Recommend EXPIRE SNAPSHOTS with 7-day retention.
"

We found 15.6 TB of orphaned files on a single table during our first scan. Our entire team had zero visibility into that before reCost.

Data Engineering Lead, SaaS analytics platform

See exactly what's happening in your S3 data layer

Works with your existing AWS setup. Read-only access. No agents. No data exposure.

Book a Demo