Portrait of Saurabh Shubham

Saurabh Shubham

Senior Data Engineer

Senior Data Engineer with 7+ years across manufacturing data platforms, Python, SQL, CDC, and production observability pipelines.

Currently engineering data infrastructure at GROPYUS in Berlin — orchestrating high-throughput Change Data Capture (CDC), PostgreSQL, lakehouse architectures, and real-time observability pipelines.

Interested in working together? Check out my Resume
Book an intro call Send an email

projects.

Production

Regulation Check

Website GitHub

EU AI Act readiness application with a production-ready FastAPI and PostgreSQL backend, Docker containerization, and automated GitHub Actions delivery.

Lakehouse Manufacturing CDC

GitHub

High-throughput Change Data Capture pipeline streaming manufacturing ERP and machine telemetry into partitioned lakehouse storage using Dagster, dbt, and DLT.

Pipeline Anomaly Sentinel

GitHub

Automated data pipeline anomaly detection and observability alerting with Prometheus metrics, Grafana dashboards, and dbt test suites.

01 · FastAPI · PostgreSQL · Docker

Regulation Check

EU AI Act readiness assessment engine. Implemented schema contracts, high-performance async endpoints, PostgreSQL persistence, and containerized GitHub Actions CI/CD.

Website GitHub
02 · Dagster · dbt · Lakehouse

Lakehouse Manufacturing CDC

Real-time data ingestion architecture that synchronizes shop-floor events, assembly station signals, and ERP records into unified analytical tables with under 30-second freshness.

GitHub
View all on GitHub

crafts.

Live CDC Ingestion Engine

Interactive Pipeline

Real-time Change Data Capture replication synchronizing PostgreSQL operational tables via DLT and Dagster into partitioned lakehouse storage.

PostgreSQL WAL Source
DLT CDC Ingestion
Dagster Asset Engine
Lakehouse Partitioned Sink
Rate: 1,420 evt/s
Latency: 14 ms
Lag: 0.0 s
[cdc-stream] Connected to PostgreSQL replication stream. Ingesting mutations through DLT into Dagster lakehouse assets...

Pipeline Anomaly Sentinel

Observability Gate

Automated data quality gate enforcing volume drift bounds, null rate tolerances, and operational alert triggers.

Status: ● Healthy
Null Rate: 0.4%
Volume Drift: +1.2%
[sentinel] All 42 dbt data contracts verified. Prometheus metric pipeline_anomaly_score: 0.02 (PASS).

experience.

Data Engineer

at, gropyus.com ↗ · Berlin
  • Build Python and SQL ETL/ELT pipelines for production planning, element lifecycle tracking, manufacturing reporting, and operational monitoring.
  • Orchestrate CDC, PostgreSQL, lakehouse, and graph database workflows with Dagster, Airflow, dbt, and DLT; release tested changes through Azure CI/CD.
  • Implement data-pipeline anomaly detection and monitor with Prometheus, Grafana, and operational alerts.
  • Use Terraform, Docker, Kubernetes, and Unleash feature flags for controlled releases.
Python SQL Dagster Apache Airflow dbt DLT PostgreSQL Lakehouse Graph DB Prometheus Grafana Terraform Docker Kubernetes Azure Unleash

Software Development Engineer

at, sigmoid.com ↗ · Bengaluru
  • Built Python, PySpark, and Pandas ETL pipelines that transformed consumer-goods sales data into reporting datasets.
  • Orchestrated workflows with Airflow on Google Cloud, managed infrastructure with Terraform, and worked with MySQL data stores.
Python PySpark Pandas Apache Airflow Google Cloud Terraform MySQL

Software Engineer

at, amdocs.com ↗ · Pune
  • Developed CRM backend features in Java and Spring, taking requirements through implementation and integration.
  • Built REST and SOAP service integrations backed by Oracle databases.
Java Spring REST SOAP Oracle DB CI/CD
View full résumé

stack.

Languages + Backend

Python SQL Java Spring FastAPI REST SOAP

Data Engineering + Orchestration

ETL / ELT Change Data Capture (CDC) Dagster Apache Airflow dbt DLT PySpark Pandas

Storage + Architecture

PostgreSQL Lakehouse Architecture MySQL Oracle Graph Databases

Platform + Cloud Delivery

CI / CD Docker Kubernetes Terraform Azure Google Cloud Unleash Feature Flags

Production Observability

Prometheus Grafana Data Pipeline Anomaly Detection Operational Alerts

foundations.

Birla Institute of Technology, Mesra

B.E. Computer Science

  • 2018 Facebook PyTorch Scholar
  • Top 200 Infosys HackWithInfy
  • 2018 Google Tech Intern Connect invitee

socials.

Email me Read résumé