DS614 - Big Data Engineering | DAU

Welcome to the course on Big Data Engineering.

---

Lectures

• Data Storage
• Prerequisites
  • System Prerequisites
  • Concurrency Prerequisites
  • Python Profiling (optional reading)
• Log based storage + Hash indexes
• SSTables + LSM Trees
• B-trees
• Columnar Storage
  • Overview
  • Pandas/Parquet
• Data Transformation
• Batch Processing
  • Introduction
  • Distributed Filesystem
  • Map reduce framework
  • Spark Intro
  • Transformation layer:
  • Querying
  • Modelling
  • Joins
• Stream Processing
  • Introduction
  • AWS Examples
• Data Ingestion
  • Introduction
  • Stream Ingestion
  • Data Orchestration
  • Data Encoding
• Distributed Systems
  • Replication
  • Replication(examples)
  • Partitioning
  • Distributed Systems Challenges

---