Saurabh Shubham

senior data engineer · berlin, germany

summary

Senior Data Engineer with 7+ years of experience across manufacturing data platforms, sales-data pipelines, and backend services. Build Python and SQL ETL/ELT pipelines with Dagster, Airflow, dbt, and DLT. Work spans CDC, lakehouse and graph database systems, cloud delivery, infrastructure as code, feature flags, and production observability.

experience

GROPYUS berlin, germany

Senior Data Engineer aug 2022 — present

  • Build Python and SQL ETL/ELT pipelines for production planning, element lifecycle tracking, manufacturing reporting, and operational monitoring.
  • Orchestrate CDC, PostgreSQL, lakehouse, and graph database workflows with Dagster, Airflow, dbt, and DLT; release tested changes through Azure CI/CD.
  • Implement anomaly detection for data pipelines and monitor them with Prometheus, Grafana, and operational alerts, supporting improvements to robot and manufacturing efficiency.
  • Use Terraform, Docker, Kubernetes, and Unleash feature flags to support platform delivery and controlled releases.
  • Redesigned Python graph ingestion using PostgreSQL metadata and checksums to skip unchanged subgraphs, improving ingestion speed by approximately 50%.
  • Improved GraphDB ingestion, operations and monitoring, reducing cluster alerts from 300 to 5 per half-year.
  • Resolved increased run volume and restrictive Linkerd timeouts to PostgreSQL, reducing Dagster pipeline failures from 10% to below 1%. Connected graph updates to automatic Dagster runs.
  • Introduced dynamic buffers to select suitable inputs when data arrives late, keeping downstream outputs on schedule; added data lineage and custom alerts.
  • Developed task-sequencing and movement-path algorithms for insulation-blowing and screw-fastening robots, designed to minimise travel and mission time.
  • Prototyped natural-language querying and interpretation of building digital twins using LLMs. Use Claude Code and agentic coding workflows.
  • Work with Helm and Azure DevOps alongside the existing platform delivery stack.

Sigmoid bengaluru, india

Software Development Engineer jun 2021 — jul 2022

  • Built Python, PySpark, and Pandas ETL pipelines that transformed consumer-goods sales data into reporting datasets.
  • Orchestrated data workflows with Airflow on Google Cloud, managed infrastructure with Terraform, and worked with MySQL data stores.
  • Supported Colgate’s sales analysis and optimisation at Walmart. Implemented an MLflow pipeline to benchmark forecasting models against the previous algorithm and identify more accurate forecasts; received Sigmoid’s Bravo Award.
  • Provisioned Google Cloud and AWS infrastructure with Terraform.

Amdocs pune, india

Software Engineer jun 2019 — jun 2021

  • Developed CRM backend features in Java and Spring, taking requirements through implementation and integration.
  • Built REST and SOAP service integrations backed by Oracle databases.
  • Developed alternative-plan selection for Comcast’s customer-retention representatives handling cancellation requests; validated selection behaviour with test-driven development.
  • Worked from requirements and wireframes through development and integration; received the Amdocs Ninja Award for engineering performance.

selected project

regulation check

EU AI Act readiness application · independent project

  • Built a FastAPI and PostgreSQL application, packaged it with Docker, and automated testing and deployment with GitHub Actions.
  • Built AI-system inventories, rule-based EU AI Act readiness screening, auditable evidence and shareable reports using Supabase PostgreSQL with tenant isolation.
  • Used pinned Docker releases, deployment approvals, startup retries, health checks and automatic rollback to the last known good version.

technical skills

languages and frameworks
Python, SQL, Java, JavaScript, Spring, FastAPI
data engineering
ETL/ELT, CDC, Dagster, Airflow, dbt, dlt (DLT), Apache Spark / PySpark, Pandas, lakehouse
cloud and delivery
Microsoft Azure, AWS, Google Cloud, Terraform, Docker, Kubernetes, Helm, Azure DevOps, GitHub Actions, CI/CD, Git
observability and controls
Prometheus, Grafana, alerting, anomaly detection, data lineage, fault tolerance, Unleash feature flags
storage and interfaces
PostgreSQL / Supabase, MySQL, MongoDB, Oracle, GraphDB and graph databases, digital twins, REST APIs, SOAP APIs
ML and AI
MLflow, experiment tracking, model registry, model serving, drift checks, LLM prototyping, agent orchestration, validation gates, Claude Code, agentic coding workflows

education

recognition