Data Engineer | Scalable Data Systems | ETL & Orchestration | Trainer
B.Tech graduate experienced in scalable data engineering. Expert in building end-to-end ETL pipelines, orchestrating complex workflows, and deploying high-performance data solutions using PySpark, SQL, and the Databricks Lakehouse Platform.
// 01. About
I specialize in orchestrating complex workflows using Apache Airflow, with niche expertise in programmatic Dynamic DAG generation to automate scalable, multi-source ingestion patterns and optimized data logic.
Students Trained
Have specifically trained more than a thousand students and juniors on comprehensive understanding of Data Management.
Data Lakehouse
Azure Databricks, Medallion Architecture (Bronze/Silver/Gold), Delta Lake, Performance Tuning (Liquid Clustering).
Orchestration
Apache Airflow (Dynamic DAGs, Custom Operators, XComs), Databricks Jobs, dbt-core.
// Specializations
// 02. Tech Stack
// 03. Portfolio
DataCamp Awarded: A DAG Airflow framework for idempotent multi-regional flight telemetry ingestion.
Modernized 3,000+ lines of legacy code to Databricks Lakehouse with optimized workflow logic.
Developed exception identification for 52 distinct types using Dead Letter Queues (DLQ) to ensure 100% uptime.
// 04. Experience
Avasoft
Architected centralized Airflow layers, engineered production pipelines for 1TB+ datasets, and implemented robust data quality frameworks for BI and Product teams.