Summary
Overview
Work History
Education
Skills
Certification
Awards
Timeline
Generic

Pratik Chafle

Data Engineer Manager
Pune

Summary

Project Manager with extensive 12 years experience in data engineering. Expertise in designing event-driven pipelines and enhancing data quality through DBT, Spark, and AWS, Apache NiFi. Demonstrated success in leading cross-functional teams to meet project objectives while improving performance and scalability. Background includes diverse domains such as Retail, Finance, Telecom, and Insurance.

Overview

12
12
years of professional experience
2
2
Certifications
3
3
Languages

Work History

Data Engineer Manager

Saksoft
Pune
12.2019 - Current
  • Designed and implemented event-driven ingestion pipelines using AWS SNS, SQS, and Lambda to process live streaming data in real time.
  • Built a daily auto-recovery framework to detect and re-ingest missing events, ensuring end-to-end data reliability and completeness.
  • Developed and maintained AWS Glue jobs with PySpark for large-scale data processing and transformation.
  • Created and managed Amazon QuickSight dashboards to monitor key business metrics, control errors, and track pipeline failures.
  • Built and deployed data transformation workflows using DBT, enhancing data quality and accelerating analytics delivery.
  • Led the architecture and development of real-time financial accounting workflows, including commission generation and advance payout processing frameworks for critical business functions.
  • Managed and mentored a 16-member cross-functional team, overseeing project delivery, task allocation, and performance tracking.

Senior Software Engineer

Accenture
Pune
08.2017 - 11.2019
  • Developed and optimized data ingestion workflows using Apache NiFi to migrate data from traditional databases to HDFS and Hive.
  • Designed and implemented Apache Airflow DAGs within Cloudera environment to orchestrate and automate Apache Spark batch processing pipelines.
  • Created and maintained ETL pipelines in Diyotta, Hive, and Spark, enabling scalable and efficient data processing based on business requirements.
  • Utilized Apache Spark for high-performance data transformation and processing, enhancing the speed and scalability of large data workloads.
  • Wrote Hive scripts to perform critical data validations, focusing on mandatory fields and ensuring data quality and integrity.
  • Optimized Diyotta workflows by improving logic and performance, resulting in faster job execution and better resource utilization.

Senior System Engineer

Infosys
Hyderabad
10.2013 - 07.2017
  • Designed and developed workflows to efficiently transfer data from RDBMS systems to the Hadoop ecosystem, ensuring scalable and reliable data ingestion.
  • Developed and scheduled complex data workflows using Apache Airflow on Cloudera to manage Spark jobs for large-scale ETL operations.
  • Built and maintained robust ETL processes using Hive, Spark SQL, and Scala based on specific business requirements and data modeling standards.
  • Identified and implemented optimization strategies for Hive queries, significantly improving query performance and resource utilization.
  • Established comprehensive data quality, monitoring, and validation frameworks to ensure end-to-end data accuracy, consistency, and integrity across the pipeline.
  • Conducted and coordinated thorough testing of data pipelines and outputs, ensuring alignment with defined business intelligence requirements and performance expectations.
  • Continuously evaluated and enhanced the ETL architecture, identifying opportunities for process improvement, automation, and scalability.

Education

BE - E&TC

D Y Patil College of Engineering, Akurdi
Pune
06.2013

Skills

PySpark

Python

Apache NiFi

Data Anlytics

AWS

SQL

undefined

Certification

AWS Certified Solutions Architect - Associate Completed on July-2025

Awards

03/2024 I-Core Customer Satisfaction Award Saksoft, 

12/2024 I-Core Respect Award Saksoft, 

03/2019 Ace Award Accenture, 

08/2016 Insta Award Infosys

Timeline

Data Engineer Manager

Saksoft
12.2019 - Current

Senior Software Engineer

Accenture
08.2017 - 11.2019

Senior System Engineer

Infosys
10.2013 - 07.2017

BE - E&TC

D Y Patil College of Engineering, Akurdi

Similar Profiles

CREATE PROFILE
Pratik ChafleData Engineer Manager