AI prompts for Data Engineering

Prompt Templates (10)

• Data Pipeline Architecture e.g. Design an ELT pipeline ingesting 50M daily events from Kafka into Snowflake with exactly-once semantics and sub-hour latency
• Data Warehouse Modeling e.g. Design a dimensional model for our e-commerce analytics warehouse covering orders, customers, products, and marketing attr…
• Data Platform Migration e.g. Plan migration from on-prem Hadoop cluster to Databricks Lakehouse, 200TB of data, 400 Spark jobs, zero-downtime requirement
• Real-Time Streaming Design e.g. Design a streaming architecture for real-time fraud detection processing 10K transactions per second with sub-second alerting
• Data Quality Framework e.g. Build a data quality framework for our Snowflake warehouse with automated testing, anomaly detection, and incident response
• Data Platform Cost Optimization e.g. Reduce our $45K monthly Snowflake bill by 30% without degrading query performance for 200 daily active analysts
• Pipeline Orchestration Design e.g. Design an Airflow orchestration layer for 300 dbt models, 50 ingestion jobs, and 20 ML pipelines with cross-DAG dependencies
• Data Contract Specification e.g. Define data contracts between our microservices and the analytics warehouse covering 12 event types and 5 entity tables
• Data Incident Response Plan e.g. Create an incident response plan for our data platform covering pipeline failures, data corruption, schema breaks, and SLA…
• Data Observability Design e.g. Design a data observability stack monitoring 500 dbt models, 100 Airflow DAGs, and 50 Snowflake warehouses with unified al…