End-to-End Data Pipeline for NYC Green Taxi trip data using Azure Data Factory, Data Lake, and Databricks following the Medallion architecture
-
Updated
Dec 8, 2024 - Jupyter Notebook
End-to-End Data Pipeline for NYC Green Taxi trip data using Azure Data Factory, Data Lake, and Databricks following the Medallion architecture
This repository contains the NYC Taxi Data Engineering Pipeline project, which aims to build a comprehensive data engineering pipeline using NYC taxi data from the years 2022 and 2023. The pipeline involves extracting, transforming and loading (ETL) data into a Snowflake database, followed by creating a dashboard for visualisation.
Add a description, image, and links to the nyc-taxi-data topic page so that developers can more easily learn about it.
To associate your repository with the nyc-taxi-data topic, visit your repo's landing page and select "manage topics."