Siddhesh Dushi

• Developed and optimized complex SQL queries to extract data based on client specific requirements.
• Loaded data from production tables to non-production environment tables to support development and testing activities.
• Prepared SQL queries to perform daily volume checks on data ingested into the raw and gold tables.

• Designed ETL pipelines using AWS Glue for processing sales data and transforming raw JSON files into parquet format.
• Automated data ingestion from AWS S3 and implemented data transforming workflows using PySpark.
• Managed Redshift Data Warehouse, optimizing schema, distribution keys and queries for performance.
• Monitored data pipelines using AWS Cloud Watch and AWS SNS for failure alerts.
• Processed and validated large-scale datasets, ensuring scalability and reliability with AWS services.
• Partnered with BI teams to create custom queries for actionable insights and data-driven decision-making.

Retail data mart

• Designed data pipeline using AWS Glue to clean, transform and load raw JSON files from S3 into snowflake via Snowpipe.
• Performed data cleaning, null handling and aggregations using PySpark, ensuring high quality, analysis-ready for retail insights.
• Designed Snowflake tables in collaboration with analyst and optimized complex SQL queries for efficient data retrieval and reporting.
• Maintained clear documentation of PySpark logic, SQL queries and transformation rules to support cross functional collaboration and reproducibility.

API Data Extraction

• Orchestrated seamless TMDB API data retrieval using Python scripts and Terraform.
• Developing AWS lambda function to process and store data in S3.
• Optimizing performance and scalability via server less computing.
• Documenting integration architecture for streamlined knowledge transfer.
• Fostering data-driven decision making.