This repository has a collection of utilities for Glue Crawlers. These utilities come in the form of AWS CloudFormation templates or AWS CDK applications.
-
Updated
Dec 21, 2021 - Python
#
aws-glue-crawler
Here are 41 public repositories matching this topic...
This repository contains source code for the AWS Database Blog Post Reduce data archiving costs for compliance by automating RDS snapshot exports to Amazon S3
aws-lambda aws-kms aws-cloudformation amazon-rds amazon-sns amazon-s3 amazon-athena aws-backup aws-glue amazon-eventbridge aws-glue-crawler
-
Updated
Apr 26, 2023
Automation framework to catalog AWS data sources using Glue
aws typescript aws-s3 dynamodb glue python3 data-catalog rds gdpr pii data-governance aws-cdk aws-glue-workflow aws-glue-crawler aws-glue-data-catalog
-
Updated
Apr 10, 2025 - Python
ETL Data pipeline using aws services
-
Updated
Aug 23, 2024 - Python
Terraform configuration that creates several AWS services, uploads data in S3 and starts the Glue Crawler and Glue Job.
-
Updated
Feb 10, 2022 - Python
Smart City Realtime Data Engineering Project
python aws kafka aws-s3 pyspark spark-streaming aws-ec2 aws-athena aws-redshift aws-glue aws-quicksight aws-glue-crawler aws-glue-data-catalog
-
Updated
May 24, 2024 - Python
It is a project build using ETL(Extract, Transform, Load) pipeline using Spotify API on AWS.
python aws aws-lambda aws-s3 spotify-api data-engineering aws-athena data-engineering-pipeline spotipy-library aws-glue-crawler awscloudwatch aws-glue-data-catalog aws-trigger
-
Updated
Jan 22, 2025 - Jupyter Notebook
AWS Athena, Glue Database, Glue Crawler and S3 buckets deployment through AWS GUI console.
-
Updated
Dec 13, 2022
Analyzed a multicategory e-commerce store using big data techniques on a Kaggle dataset with the help of AWS EC2, AWS S3, PySpark, AWS Glue ETL, AWS Athena, AWS CloudFormation, AWS Lambda and Power BI!
aws big-data aws-lambda power-bi pyspark aws-ec2 aws-cloudformation aws-athena kaggle-dataset aws-services end-to-end-pipeline end-to-end-project aws-glue-crawler aws-s3-bucket
-
Updated
Sep 7, 2024 - Python
In this project I have used the Trending YouTube Video Statistics data from Kaggle to analyze and prepare it for usage.
-
Updated
Nov 7, 2022
This project automates the extraction, transformation, and loading (ETL) of Reddit data into a Redshift data warehouse using Airflow. Key technologies include Celery, PostgreSQL, S3, Glue, Athena, and Redshift, providing a complete data pipeline solution.
-
Updated
Apr 16, 2025
An end-to-end data pipeline built with AWS S3, Glue, Crawler, Athena, Tableau visulization
-
Updated
Mar 27, 2024 - Jupyter Notebook
Creating an audit table for a DynamoDB table using CloudTrail, Kinesis Data Stream, Lambda, S3, Glue and Athena and CloudFormation
aws-python-lambda aws-iam aws-cloudformation aws-cloudtrail aws-cloudwatch aws-athena aws-cloudwatch-logs aws-kinesis-stream aws-glue-crawler aws-iam-roles aws-iam-policies aws-s3-bucket aws-glue-data-catalog
-
Updated
Jul 6, 2023 - Python
Unlocking Spotify insights with an AWS data pipeline: S3 data lake, Glue ETL, Athena queries, and QuickSight dashboards.
python aws serverless athena etl analytics aws-s3 data-visualization pyspark data-engineering data-lake parquet data-pipeline aws-glue spotify-data quicksight-dashboard aws-glue-crawler
-
Updated
May 18, 2025 - Python
AWS Athena, Glue Database, Glue Crawler and S3 buckets deployment through CloudFormation stack on AWS console.
-
Updated
Dec 14, 2022
An end-to-end solution for managing and analyzing YouTube video data from Kaggle, leveraging AWS services and visualized through Quicksight and Tableau
aws marketing youtube aws-lambda aws-s3 youtube-api aws-iam tableau content-strategy aws-athena aws-lambda-python aws-glue quicksight aws-glue-crawler user-insights
-
Updated
Sep 23, 2023 - Python
Working with Glue Data Catalog and Running the Glue Crawler On Demand
-
Updated
May 11, 2023
Unveiling job market trends with Scrapy and AWS
-
Updated
Apr 5, 2024 - Python
Implementing data pipeline using AWS services for airlines data
python aws aws-s3 aws-sns aws-redshift step-functions aws-eventbridge aws-glue-workflow aws-glue-crawler
-
Updated
Oct 15, 2024 - Python
Working with Glue Data Catalog and running the using S3 Event Notification and creating the entire stack using AWS CloudFormation
-
Updated
May 8, 2023
Improve this page
Add a description, image, and links to the aws-glue-crawler topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the aws-glue-crawler topic, visit your repo's landing page and select "manage topics."