# รวบรวมโปรเจคทำ Data Engineering **URL:** https://discuss.dataengineercafe.io/t/data-engineering/625 **Category:** General Discussion เรื่อง Data Engineering **Tags:** data-engineering, portfolio **Created:** [September 14, 2023, 3:30am UTC](https://discuss.dataengineercafe.io/t/data-engineering/625 "2023-09-14T03:30:27Z") **Posts on this page:** 8 **Page:** 1
### Author: ![zkan](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.dataengineercafe.io/zkan/32/2_2.png) [@zkan](https://discuss.dataengineercafe.io/u/zkan) #### Post date: [September 14, 2023, 3:30am UTC](https://discuss.dataengineercafe.io/t/data-engineering/625/1 "2023-09-14T03:30:27Z")
พอดีเห็นโพสต์ของพี่ Marc 👉 [Marc Lamberti on LinkedIn: #dataengineering #dataengineer #airflow #apacheairflow | 24 comments](https://www.linkedin.com/posts/marclamberti_dataengineering-dataengineer-airflow-activity-7105928393455513600-8x_F/) แล้วมานึกดูว่าคำถามนี้ก็มีคนถามบ่อย ๆ เหมือนกันนะ ซึ่งโปรเจคแนว Data Engineering ก็หายากจริง 😂 ก็ขอสร้างโพสต์นี้รวบรวมไว้เลยละกันครับ ด้านล่างนี้ผมเอามาจากโพสต์ของพี่ Marc เอามาแปะไว้ 1️⃣ The FinnHub Streaming Data Pipeline 💬 The project is a streaming data pipeline based on [Finnhub.io](http://Finnhub.io) API/websocket real-time trading data. 💻 Kafka, Spark, Cassandra, Kubernetes, Grafana > **[GitHub - RSKriegs/finnhub-streaming-data-pipeline: Stream processing pipeline...](https://github.com/RSKriegs/finnhub-streaming-data-pipeline)** > > Stream processing pipeline from Finnhub websocket using Spark, Kafka, Kubernetes and more - GitHub - RSKriegs/finnhub-streaming-data-pipeline: Stream processing pipeline from Finnhub websocket usin... 2️⃣ Streamify 💬 The project will stream events generated from a fake music streaming service (like Spotify) and create a data pipeline that consumes the real-time data 💻 Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP > **[GitHub - ankurchavda/streamify: A data engineering project with Kafka, Spark...](https://github.com/ankurchavda/streamify)** > > A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more! - GitHub - ankurchavda/streamify: A data engineering project with Kafka, Spark Streaming,... 3️⃣ Reddit ETL Pipeline 💬 A data pipeline to extract Reddit data from r/dataengineering and provides a Google Data Studio report 💻 AWS S3/Redshift, dbt, Airflow, Docker, Terraform > **[GitHub - ABZ-Aaron/Reddit-API-Pipeline](https://github.com/ABZ-Aaron/Reddit-API-Pipeline)** > > Contribute to ABZ-Aaron/Reddit-API-Pipeline development by creating an account on GitHub. 4️⃣ Audiophile End-To-End ELT Pipeline 💬 Pipeline that extracts data from Crinacle’s Headphone and InEarMonitor databases and finalizes data for a Metabase Dashboard. 💻 AWS S3, Redshift, RDS, dbt, Airflow > **[GitHub - ris-tlp/audiophile-e2e-pipeline: Pipeline that extracts data from...](https://github.com/ris-tlp/audiophile-e2e-pipeline)** > > Pipeline that extracts data from Crinacle's Headphone and InEarMonitor databases and finalizes data for a Metabase Dashboard. - GitHub - ris-tlp/audiophile-e2e-pipeline: Pipeline that extracts ... 5️⃣ Surfline Dashboard 💬 The pipeline collects data from the surfline API and exports a csv file to S3. Then the most recent file in S3 is downloaded to be ingested into the Postgres datawarehouse. At the end, you obtain a beautiful dashboard showing the data 💻 AWS S3, Airflow, Pandas, Postgres, Ploty > **[GitHub - andrem8/surf\_dash](https://github.com/andrem8/surf_dash)** > > Contribute to andrem8/surf\_dash development by creating an account on GitHub. 6️⃣ The Data Retail Project 💬 The pipeline extracts data from a CSV file, stores the data in BigQuery, transforms the data with dbt, and runs data quality checks with Soda. At the end, you get a nice dashboard showing analytics with Metabase. [![](https://canada1.discourse-cdn.com/flex035/uploads/dataengineercafe/original/2X/7/72740dd1f281fe91d15c11688584a31373fb963a.jpeg "Data Engineer Project: An end-to-end Airflow data pipeline with BigQuery, dbt Soda, and more!") ](https://www.youtube.com/watch?v=DzxtCxi4YaA) ถ้าใครมีโปรเจคไหนน่าสนใจก็มาแชร์กันได้นะครับ ☺ ---
### Author: ![zkan](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.dataengineercafe.io/zkan/32/2_2.png) [@zkan](https://discuss.dataengineercafe.io/u/zkan) #### Post date: [September 14, 2023, 3:31am UTC](https://discuss.dataengineercafe.io/t/data-engineering/625/2 "2023-09-14T03:31:36Z")
ส่วนลิ้งค์ด้านล่างนี้เอามาจากคนที่มาคอมเมนต์ในโพสต์พี่ Marc ครับ > **[Building a Data Engineering Project in 20 Minutes](https://www.ssp.sh/blog/data-engineering-project-in-twenty-minutes/)** > > You'll learn web-scraping with real-estates, uploading them to S3, Spark and Delta Lake, adding Data Science with Jupyter, ingesting into Druid, visualising with Superset and managing everything with Dagster. > **[GitHub - SebasMBK/SunglassesHubETL: This project is for demonstrating...](https://github.com/SebasMBK/SunglassesHubETL)** > > This project is for demonstrating knowledge of Data Engineering tools and concepts and also learning in the process - GitHub - SebasMBK/SunglassesHubETL: This project is for demonstrating knowledge... > **[GitHub - SebasMBK/etl-houseslima-properati](https://github.com/SebasMBK/etl-houseslima-properati)** > > Contribute to SebasMBK/etl-houseslima-properati development by creating an account on GitHub. ---
### Author: ![zkan](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.dataengineercafe.io/zkan/32/2_2.png) [@zkan](https://discuss.dataengineercafe.io/u/zkan) #### Post date: [September 14, 2023, 3:33am UTC](https://discuss.dataengineercafe.io/t/data-engineering/625/3 "2023-09-14T03:33:58Z")
บทความด้านล่างนี้ก็น่าสนใจ ลองตามอ่านกันดูครับ > **[How to Create First Data Engineering Project? An Incremental Project Roadmap](https://blog.det.life/create-first-data-engineering-project-incremental-roadmap-6f0e66e7a5bf)** > > Build Data Engineering projects in this incremental approach for guaranteed success. Break tutorial hell and stop procrastinating.. > **[End-to-end data engineering project - batch edition](https://www.startdataengineering.com/post/data-engineering-project-e2e/)** > > Struggling to come up with a data engineering project idea? Overwhelmed by all the setup necessary to start building a data engineering project? Don't know where to get data for your side project? Then this post is for you. We will go over the key... > **[Data Engineering Project for Beginners - Batch edition](https://www.startdataengineering.com/post/data-engineering-project-for-beginners-batch-edition/)** > > Data engineering project for beginners, using AWS Redshift, Apache Spark in AWS EMR, Postgres and orchestrated by Apache Airflow. > **[Data Engineering Project: Stream Edition](https://www.startdataengineering.com/post/data-engineering-project-for-beginners-stream-edition/)** > > Stream processing differs from batch; one needs to be mindful of the system's memory, event order, and system recovery in case of failures. However, understanding the fundamental concepts of time attributes, cluster memory, time-bounded joins, and... ---
### Author: ![zkan](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.dataengineercafe.io/zkan/32/2_2.png) [@zkan](https://discuss.dataengineercafe.io/u/zkan) #### Post date: [September 14, 2023, 3:44am UTC](https://discuss.dataengineercafe.io/t/data-engineering/625/4 "2023-09-14T03:44:20Z")
หรือว่าจะค้นหาใน GitHub ก็ได้เช่นกันนะ ดูว่าชาวบ้านเค้าทำอะไรกัน มีเยอะอยู่ อิอิ 👉 [https://github.com/search?q=data%20engineering%20project&type=repositories](https://github.com/search?q=data%20engineering%20project&type=repositories) ---
### Author: ![zkan](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.dataengineercafe.io/zkan/32/2_2.png) [@zkan](https://discuss.dataengineercafe.io/u/zkan) #### Post date: [September 18, 2023, 5:38am UTC](https://discuss.dataengineercafe.io/t/data-engineering/625/5 "2023-09-18T05:38:10Z")
บทความของ อ. Aekanun ใช้ Apache Beam, Google Cloud Dataflow และ Terraform สร้าง Stream Processing Solution ครับ > **[Unlocking Real-Time Insights: Building a Stream Processing Solution with...](https://aekanunbigdata.medium.com/unlocking-real-time-insights-building-a-stream-processing-solution-with-apache-beam-google-cloud-1c060a0557cb)** > > 1. Introduction ---
### Author: ![zkan](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.dataengineercafe.io/zkan/32/2_2.png) [@zkan](https://discuss.dataengineercafe.io/u/zkan) #### Post date: [December 13, 2025, 11:26am UTC](https://discuss.dataengineercafe.io/t/data-engineering/625/6 "2025-12-13T11:26:31Z")
[From Scheduled Chaos to Intelligent Automation: Building Self-Managing Data Pipelines with Snowflake Dynamic Tables](https://medium.com/@vikneshwararb_99226/from-scheduled-chaos-to-intelligent-automation-building-self-managing-data-pipelines-with-718aba786f51) กับ Yelp Open Dataset ---
### Author: ![lif](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.dataengineercafe.io/lif/32/45_2.png) [@lif](https://discuss.dataengineercafe.io/u/lif) #### Post date: [December 16, 2025, 8:14am UTC](https://discuss.dataengineercafe.io/t/data-engineering/625/7 "2025-12-16T08:14:22Z")
พี่ marc เข้าเปิดเว็บนี้ขึ้นมา คล้ายๆ กับ kaggle เผื่อเรามี idea ทำแล้วอยากให้คนมารีวิว ก็ลองเว็บนี้ได้ > **[Data Project Hunt - Discover the Best Data Engineering Projects](https://www.dataprojecthunt.com/)** > > Explore and showcase exceptional data engineering projects. Find Apache Airflow workflows, ETL pipelines, data orchestration tools, and analytics platforms. ---
### Author: ![zkan](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.dataengineercafe.io/zkan/32/2_2.png) [@zkan](https://discuss.dataengineercafe.io/u/zkan) #### Post date: [September 26, 2026, 3:43pm UTC](https://discuss.dataengineercafe.io/t/data-engineering/625/8 "2026-09-26T15:43:59Z")
เพิ่มของ Simon Späti ด้วย มีเพียบเลย > **[Open-Source Data Engineering Projects (2022-2026)](https://www.ssp.sh/brain/open-source-data-engineering-projects/)** > > A curated list of many open-source data engineering projects collected over the years.