Prasath N
Senior Data Engineer
Chennai
Summary
Experienced Data Engineer with 12 years of IT experience including 8.5 years in Big Data ecosystem and 3.5 years in .NET development. Strong expertise in Spark, Scala, Hive, Control-M, Shell Scripting, Batch Scripting, and Tableau. Adept at building scalable data pipelines, optimizing performance in Hive and Spark, and automating workflows using Control-M. Experienced in Spark 2 to Spark 3 migration. Proficient in developing dashboards using Tableau and QlikView. Familiar with Git, ADO pipelines, and IntelliJ.
Overview
12
12
years of professional experience
7
7
years of post-secondary education
Work History
Senior Data Engineer
Standard Chartered
CHENNAI
09.2022 - Current
- Designed and developed data pipelines using Spark and Scala to transform and process large-scale financial data.
- Created and modified Control-M jobs to streamline scheduling and automation processes.
- Involved in Spark 2 to Spark 3 migration with code refactoring and validation.
- Wrote optimized Hive queries to handle large datasets with improved performance.
- Built automation workflows using shell scripting and Control-M to reduce manual interventions.
- Participated in performance tuning and incident resolution for business-critical data loads.
- Built dashboards using Tableau to visualize data and support stakeholder reporting.
- Used Git for version control and ADO pipelines for CI/CD.
Assistant Consultant
Tata Consultancy Services
CHENNAI
05.2021 - 08.2022
- Used Kafka for real-time data processing.
- Developed backend logic in Spark-Scala and Python.
- Managed job orchestration using Autosys.
- Stored batch processing results in Hive tables.
System Analyst
STG
CHENNAI
06.2019 - 05.2021
- Transferred RDBMS data to Hadoop using Sqoop and Attunity.
- Processed data in Spark using transformations and actions.
- Automated scripts using Python and utilized Pandas.
- Created ingestion pipelines via Attunity T1 and T2 tasks.
Lead Engineer
HCL Technologies
CHENNAI
04.2016 - 05.2019
- Developed analytics for medical data using Hive and Spark.
- Designed test cases and validated Big Data applications.
- Created Hive tables and optimized query performance.
Project Engineer
Wipro Technologies
Bangalore
04.2013 - 04.2016
- Built desktop tools and applications for courier and system verification.
- Created a Dropbox emulator using C# and .NET framework.
Education
B.E - E.C.E
RMD Engineering College
01.2008 - 01.2012
12th - undefined
St. Mary's Matriculation
01.2006 - 01.2008
10th - undefined
Vivekananda Vidhalaya
01.2005 - 01.2006
Skills
- Scala
- Python
- Java
- C#
- Spark
- Hive
- Sqoop
- HDFS
- Kafka
- Attunity
- Control-M
- Autosys
CI/CD practices
Timeline
Senior Data Engineer
Standard Chartered
09.2022 - Current
Assistant Consultant
Tata Consultancy Services
05.2021 - 08.2022
System Analyst
STG
06.2019 - 05.2021
Lead Engineer
HCL Technologies
04.2016 - 05.2019
Project Engineer
Wipro Technologies
04.2013 - 04.2016
B.E - E.C.E
RMD Engineering College
01.2008 - 01.2012
12th - undefined
St. Mary's Matriculation
01.2006 - 01.2008
10th - undefined
Vivekananda Vidhalaya
01.2005 - 01.2006
Similar Profiles
S Sakthi SabarishS Sakthi Sabarish
Client Care Executive - Multi Skill PSB at Standard CharteredClient Care Executive - Multi Skill PSB at Standard Chartered
Rekha JanarthananRekha Janarthanan
Associate Product Owner at Standard CharteredAssociate Product Owner at Standard Chartered
Kar Li LeeKar Li Lee
Team Leader at Standard CharteredTeam Leader at Standard Chartered
Sivasubramanian NatarajanSivasubramanian Natarajan
Associate Manager Service Delivery at DXC TechnologyAssociate Manager Service Delivery at DXC Technology
Dhinesh JDhinesh J
Associate Senior Software Engineer at Customer Analytics Pvt. Ltd.Associate Senior Software Engineer at Customer Analytics Pvt. Ltd.