Results-driven Technical Data Engineering Manager with extensive experience in designing and leading large-scale data engineering projects. Proven expertise in building and managing end-to-end data pipelines using PySpark, Apache Airflow, AWS, and Airbyte for high-volume data ingestion from diverse advertising platforms including Snapchat, Pinterest, Facebook, and Twitter. A certified AWS Solutions Architect, GCP Professional Data Engineer, and Scrum Master, with a strong foundation in cloud-native architectures and agile methodologies.
As a dual-role Technical Lead and People Manager, I lead a team of 10 engineers, driving project execution from concept to delivery while fostering a high-performance, collaborative team culture. Skilled in stakeholder communication, cross-functional collaboration, and sprint management, I ensure alignment between business goals and technical implementation. Specialized in advertisement data, with a keen focus on performance optimization, data quality, and scalable architecture.Committed to mentorship and fostering collaboration, ensuring successful project outcomes.
Built various Pyspark Pipelines using AWS services like EMR, Glue.
Orchestrated using Airflow
Handled End-to-End pipelines which handled data upto 1 TB everyday.
Responsible for building Robust, scalable pipelines
Worked as a Data Engineer.
Implemented various pipelines to Ingest data from various Sources.
Used Python, Pyspark, Airflow to orchestrate pipelines.
Majorly worked on GCP and used various GCP services
Worked as ETL Developer.
Tools used like Informatica, SSIS,
Migration from SQL Server to AWS Redshift.
Project leadership
Agile methodology
Systems development lifecycle management
Technical analysis
Mentorship and coaching
Project management
Scrum framework
AWS: Solution Architect