Gcp

Published on
July 13, 2024
Data Engineering Zoomcamp | Week 2.5 ETL, API to GCP using Parquet with Partitioning through PyArrow. Pipeline in Mage |
Mage GCP PyArrow Parquet Partitioning
load data to GCP. We will leverage PyArrow to output the NY_TAXI dataset partitioned by date using PyArrow.
Published on
July 12, 2024
Data Engineering Zoomcamp | Week 2.4 Configuring GCP for Mage |
Mage GCP Docker
set up Google Cloud components and configure Mage to read and write data using Google Cloud Storage and BigQuery.
Published on
July 5, 2024
Data Engineering Zoomcamp | Week 2.1. Mage Intro and Setup |
Guide Mage BigQuery GCP Python SQL Docker
High-level overview of Mage. How to configure Mage ready to start building pipelines.

Data Engineering Zoomcamp | Week 2.5 ETL, API to GCP using Parquet with Partitioning through PyArrow. Pipeline in Mage |