Deepa Pamnani
Senior Consultant
Mumbai
Summary
Results-driven Data Engineering Manager with 10+ years of experience architecting and delivering scalable data solutions on AWS, Azure, and Databricks. Proven track record of leading and mentoring teams of 15+ members, managing full project lifecycles, and implementing modern data architectures. Skilled in cloud data engineering, big data technologies, and Agile methodologies. Experienced in RFP management and project estimation, ensuring solutions align with client requirements and business goals.
Overview
10
10
years of professional experience
2015
2015
years of post-secondary education
4
4
Certifications
4
4
Languages
Work History
Senior Consultant
Deloitte USI
10.2020 - Current
- Directed end-to-end automation of data integration between Anaqua and Counsel-Link, optimizing cross-system workflows and ensuring secure, reliable data exchange.
- Oversaw onboarding and processing of Empower data into AWS S3 and ElasticSearch, enabling robust API-driven data consumption for downstream business systems.
- Led migration and centralization of template data from NuGenesis/Signals into Aurora DB, empowering analytics and reporting via Power BI.
- Managed integration of stability and COA data for WuXi, facilitating seamless partner collaboration through secure API solutions.
- Guided consolidation of diverse eLN systems into a unified EDB platform, supporting advanced R&D experiment search and enhancing data-driven innovation.
- Partnered with stakeholders to migrate Nugenesis eLN data to the Enterprise Data Backbone, expanding user access and delivering significant cost savings by reducing reliance on legacy licenses.
- Tech-Skills: Cloud Data Engineering: AWS S3, AWS Glue, Lambda Functions, Aurora DB, ElasticSearch, API Gateway, Secret Manager
- Data Integration & Automation Strategy
- API Architecture & Governance
- Data Migration & Modernization Leadership
- Big Data Solutions & Analytics (Power BI)
- Agile Project Management
- Cross-functional Team Leadership
- Leadership Impact: Championed adoption of modern cloud data architectures, increasing operational efficiency and scalability.
- Fostered a culture of innovation and continuous improvement within cross-functional teams.
- Delivered measurable business value through cost optimization, enhanced data accessibility, and improved decision-making for business and R&D stakeholders.
Sr. Data Engineer
Larsen & Toubro Infotech LTD
09.2019 - 10.2020
- To process the raw files coming from different ERP sources to S3 and storing the processed data to redshift. Unloading the redshift data to S3 in CSV formats and generated reports using Python scripts and transferred that reports to a server using SFTP.
- Process and transform raw files coming from different ERP sources to s3, using AWS services and transfer the transformed data to Oracle Hyperion environment.
- To process raw data received in s3 buckets using python and SQL scripts and copy the processed data to Redshift.
- Projects Responsibilities: Requirements Analysis, Implementing the requirements using python scripts, Creating deployment documents and also involved in deployment on Production environments, Writing SQL queries for Unload data from redshift, Generated reports using Python scripts, Implementing the enhancements using python scripts, Creating deployment documents, Monitoring Jobs, Validating reports, To write Python and SQL scripts for data processing, To write Redshift queries for data validations.
Data Engineer
Tata Consultancy Services
08.2015 - 09.2019
- Implemented Data Lake on s3.
- Implemented Real time streaming using kinesis streams, also processed the data through lambda function and stored the results to Redshift.
- Worked on LEX and created Chabot’s for different use cases (using lambda functions).
- Implemented BI search on custom UI using LEX.
- Implemented services using Elastic Beanstalk.
- Data warehouse design and Implementation using redshift.
- Worked on user-based recommendation systems using python.
- Worked on AWS Sagemaker for predictive analysis.
- AWS Data warehouse: Build Data Lake on s3, and created Athena tables pointing to s3 locations. Build Metadata in glue Catalog using glue crawlers. Developed Glue jobs for transformation of data taking source from s3 and storing output to Redshift / s3. Developed data pipeline to copy data from s3 to Redshift. Written step function to automate Workflow of Data processing. Data warehouse design and implementation on Redshift.
- Digital Ad Ratings: Nielsen Digital Ad Ratings provides a comprehensive, next-day view of your ad’s online and mobile audience in a way comparable to the Nielsen TV ratings. Powered by the largest user databases and the highest quality panel in the world, Nielsen Digital Ad Ratings set a new industry standard for digital audience measurement. The Companies like (Samsung / Walmart / Colgate) posting Ads on Various Sites / Ad-Networks are also interested to know the demographics and other details of Audience visiting the site / advertisement. Client must register the campaign with Site and placement details with Nielsen. After registering the Client will be able to see the Demographic Details of Audience / Geography report / Number of Impression by Site and Placement / Educational / Financial / Number of Children of User and many more reports as per requirement.
- Projects Responsibilities: Done Automation of Data Quality Checks which were used to check the quality of Site Census and Facebook data (7 months project). Writing Unix Shell Scripts to generate control files which needs to be sent to Site Census and Facebook. Using Sqoop to Import the RDBMS data into Hadoop File System. Writing / Migrating the SQL Scripts from legacy system to Impala / Hive. Creating oozie workflow to execute Impala / hive / shell / java / map reduce task. Optimizing Impala / Hive Queries with Partitions. Created Metadata Schema in Oracle Database to store the metadata of Campaigns / Site / Placement. Analysing the real time processing problems / root cause analysis / finding – suggesting workaround in sev1 situations.
Education
Bachelor of Engineering - Electronics and Telecommunications
Sant Gadge Baba Amravati University
Amravati, Maharashtra01-2015
Skills
Cloud storage with AWS S3
Experience with AWS Glue
Experience with lambda functions
Experience with Aurora DB
Proficient in ElasticSearch implementation
API management
Secure secrets management
Data Integration & Automation Strategy
API design and management
Data Migration & Modernization Leadership
Data analytics solutions
Experience with Power BI
Agile project coordination
Cross-functional Team Leadership
Experienced in Python programming
Shell script development
PostgreSQL
MYSQL
Database management expertise
Certification
AWS Certified Solutions Architect – Professional
Accomplishments
- Received Multiple Outstanding Performance Awards
- Received Multiple Applause Awards
- Received Multiple Spot Awards
Timeline
Senior Consultant
Deloitte USI
10.2020 - Current
Sr. Data Engineer
Larsen & Toubro Infotech LTD
09.2019 - 10.2020
Data Engineer
Tata Consultancy Services
08.2015 - 09.2019
Bachelor of Engineering - Electronics and Telecommunications
Sant Gadge Baba Amravati University