今天，我們將把物聯網與我們上一個系列【PySpark 大數據處理】完美結合，示範如何將雲端收集到的感測日誌匯出，打通從「硬體採集」到「大數據分析」的完整端到端 (End-to-End) 藍圖！
2. 對接 PySpark：用大數據引擎提煉價值
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, to_timestamp, date_format, round, avg
spark = SparkSession.builder.appName("IoT_Data_Analysis").getOrCreate()
df_raw = spark.read.csv("feeds.csv", header=True, inferSchema=True)
df_clean = df_raw.withColumnRenamed("field1", "Temperature") \
.withColumnRenamed("field2", "Humidity") \
.withColumn("Record_Time", to_timestamp(col("created_at"), "yyyy-MM-dd HH:mm:ss")) \
.withColumn("Date", date_format(col("Record_Time"), "yyyy-MM-dd")) \
.select("Date", "Record_Time", "Temperature", "Humidity") \
.dropna()
df_clean.createOrReplaceTempView("iot_sensor_data")
insight_sql = """
SELECT
Date,
ROUND(AVG(Temperature), 2) AS Avg_Temp,
ROUND(AVG(Humidity), 2) AS Avg_Humi,
MAX(Temperature) AS Max_Temp_Recorded
FROM iot_sensor_data
GROUP BY Date
ORDER BY Date ASC
"""
df_report = spark.sql(insight_sql)
print("=== 本週溫室環境每日監控報表 ===")
df_report.show()
3. 結語：萬物互聯，數據為王
【ESP32 物聯網實戰】10 篇連載在此正式完結！希望這套系列教學，能為資管系的學生或是渴望跨領域的初學者，點亮一盞通往全端資料工程師的明燈。我們，下個系列見！
