Debabrat Das

• Worked on a interesting project called SparkMatica , where we took Informatica mapping xmls as input and generated a Pyspark mapping and then executed them on Cloud Dataproc, which we built for a migration project from Oracle Cloud to Bigquery.
• Work on project where we built a tool called data sync, where we used airflow and Dataflow Flex templates to migrate data from Oracle Exacc to Bigquery.
• Worked on a project Jobnet where we replicated Informatica Jobnet structure to trigger jobs in Airflow
• Worked extensively on Airflow and Pyspark and created numerous tools and utilities for swift data migration.

• Worked on an python app using Flask called sementic search where combined the capabilities of AI embeddings and Solr to give users a seamless search experience
• Worked on a Loreal's Data Lake creation journey, and extensively worked on Bigquery and gcp worflows.

Project- P&G Cloud Data Migration-

Extracted Data from various sources such as XML files, Relational databases, Big Query, Segment personas etc.
Used Big query and GCP bucket SDK in Python, Pandas to transform the unstructured data acquired from various sources to a scalable schema and load them as new line delimited json into Big Query as structured data.