Summary
Overview
Work History
Education
Skills
Projects
Certification
Timeline
Generic

SHINDE DNYANESHWAR

Specialist Data Engineer
Pune

Summary

Have Extensive Experience of 6+ yrs in IT industry on data warehouse projects, Hands on experience in migrating on premise ETLs to Google Cloud Platform (GCP) using cloud native tools such as BIG query, Dataflow, Google Cloud Storage, Composer,Cloud run.Experience working in agile development methodologies.Aiming to leverage my ability to successfully fill the data engineer role in team.

Overview

6
6
years of professional experience
2018
2018
years of post-secondary education
1
1
Certification
3
3
Languages

Work History

Specialist Data Engineer

LTIMindtree Ltd.
10.2024 - Current
  • Design,Implement data pipeline using Bigquery,Cloud run,workflow for large datasets,ensuring high performance and cost efficiency for data analytics workloads.
  • Creating Datasets,Sprocs ,Views,tables/external tables,UDF for use-case specific.
  • Developed and optimized sql,sprocs for data transformation and reporting .
  • Deployed containerized microservice using python's FastAPI to process the flat file data to bigquery tables and other business logic to convert diff file format to csv.
  • Creation of workflows to automate data movement and integration between multiple GCP services like invoke cloud run service,executing sprocs,sql scripts etc.
  • Utilized terraform IaC to deploy and manage GCP resources.Also implemented CICD pipeline to automate the deployment of infra and code.
  • Working with different team -Testing ,Ops team to insure the smooth transition of application before and after deployment.

Senior Software Engineer

Tech Mahindra Ltd.
12.2022 - 10.2024
  • Collaborate with project managers and stakeholders to understand project requirements, timelines, and deliverables.
  • Worked on end to end development of project like building data model design, developer design documentation, development and testing of data pipeline.
  • Creating tables, view using terraform repository.
  • Design and implement data models to support business requirements and analytics needs using Google BigQuery.
  • Implement efficient data transformation and processing logic using SQL queries.
  • Use orchestration tools like Apache Airflow(Google Cloud Composer) or equivalent to schedule, automate, and monitor data workflows.
  • Dag creation using different airflow operator.
  • Utilize GCP data storage solutions such as Cloud Storage to store the extracted data in different file format from BigQuery table for the business team.
  • Worked on Python pandas to work with files and loading data from and to files.
  • Worked on Cloud function (http) to implement API and send response to the third party app when url get triggered.
  • Use version control systems (Gitlab) to manage code changes and collaborate with team members.

SOFTWARE ENGINEER II

Datametica Solutions Pvt Ltd
10.2021 - 12.2022
  • Migrating an entire on-premise DW ETL solution on Google Cloud Platform
  • Worked on 20+ end to end data pipeline building and data validation
  • Build data pipelines using airflow on GCP for ETL related jobs using different airflow operators
  • Used cloud shell SDK in GCP to configure the services, Storage, BigQuery and for connecting to VMs
  • Experience in loading different file data to BigQuery and validation of data
  • Worked on command line utility Cloud CLI (gsutil, bq, gcloud)
  • Writing queries for ETL transformations and execute query in pipeline using airflow operator
  • Writing shell Script and python script for business logic in pipeline like reading file, archiving file in bucket etc
  • Created BigQuery authorized views, partitioned tables
  • Good understanding of BigQuery architecture and Query optimization technique
  • Experience in writing DAG and schedule it in airflow using different operators (BashOperator, DataflowOperator, PythonOperator)
  • Worked on dataflow to create jobs and schedule using apache airflow
  • Good knowledge of GCP Data transfer service, GCS, Cloud functions, BigQuery, Dataflow, Cloud Composer, Cloud SDK
  • Hands-on experience on Python
  • Good understanding of dataflow and apache beam framework.

SYSTEM ANALYST

Axis Bank Ltd
02.2019 - 10.2021
  • Interacting with business users, gathering and analyzing requirements and Involved in understanding the requirements of the end user
  • Writing complex SQL queries and PL/SQL procedures to extract data from various source tables of data warehouse
  • Understanding the business rule completely based on High-Level document specifications and implements the data transformations methodology
  • Created Shell Scripts for invoking SQL scripts and scheduled them using crontab
  • Working on data migration from different sources to staging environment
  • Worked on data validation part
  • Managing the DataMart for business team requirement.

Education

BACHELOR OF ENGINEERING IN IT - undefined

MGM College Of Engineering

PG_DAC IN IT -

Sunbeam Institute of Technology

Skills

Python

GCP (Composer,Dataflow,Storage,Cloud Function,CloudCLI,VMs,Cloud Run,Workflows)

BigQuery

ShellScript

Terraform

Projects

  • L'Oreal- ANZ
  • BT-GSIP
  • IBX- Data Migration
  • Axis Bank- STAR

Certification

Associate Google Cloud Engineer

Timeline

Specialist Data Engineer

LTIMindtree Ltd.
10.2024 - Current

Senior Software Engineer

Tech Mahindra Ltd.
12.2022 - 10.2024

SOFTWARE ENGINEER II

Datametica Solutions Pvt Ltd
10.2021 - 12.2022

SYSTEM ANALYST

Axis Bank Ltd
02.2019 - 10.2021

BACHELOR OF ENGINEERING IN IT - undefined

MGM College Of Engineering

PG_DAC IN IT -

Sunbeam Institute of Technology

Similar Profiles

CREATE PROFILE
SHINDE DNYANESHWAR Specialist Data Engineer