[Hindi] End to End Project | A pipeline with Medallion Architecture

About Course

Pre Requisites :

Python
SQL
PySpark
PySpark Optimizations
Big Data Foundations
Hadoop

Course Content

Project Flow | Data Ingestions | CDC

Comparison Between DE Project Architectures

38:05
Data Ingestion | RDBMS | CDC | Challenges | Performance

01:30:30
Data Ingestion | FTP to Bronze

36:07
Data Ingestion | RDBMS to Bronze | Hands On

00:00
Industry Practices for : Data Ingestion Using Glue

01:46:21
FTP to Bronze | Nifi Flow Industry Scenarios

01:41:32
NIFI Scenarios and Interview Questions

54:08

Kafka Essentials

Airflow Essentials

MongoDB Essentials

AWS for Data Engineering

Big Data With AWS || Demo Session

36:43
AWS IAM User, Group and Policies

01:26:35
Big Data With AWS | IAM Roles

01:23:47
AWS Cloud Infrastructure

31:39
S3 Session 1

01:40:25
S3 Session 2

01:00:49
S3 Session 3

01:09:45
AWS glue | Crawlers And Jobs

01:23:58
BDA | Glue Scenarios | Glue Workflows

01:24:01
Glue Scenarios

01:15:03
Glue Project : CDC

01:46:21
EMR basics EC2 introduction

23:36
EMR_basics_IAM_Role

24:26
Starting EMR cluster | Connecting to EMR Cluster

58:25
AWS EMR | Starting and Deploying Spark Application on EMR

58:04
Cluster Mode Deployment | Accessing Web UIs | Steps Introduction

53:48
Deploying Spark Application in Cluster Mode on EMR

11:15
Deploying Spark Application using Steps on AWS EMR

01:02:03
Athena Basics

01:23:48
Athena on Command Line

59:43
Using Athena through python code

41:27
Redshift And Data Warehousing Introduction

41:34
Redshift Clusters | Snapshots | S3 Copy

59:16
Creating redshift cluster and making it publicly accessible

13:58
Redshift connect using sql workbench and python script

38:55
dist keys and sort keys in redshift

24:30
DIST Keys hands on

30:38
Redshift Federated Queries

40:58
What is Streaming Data | Streaming Data Terminologies

23:24
Kinesis Data Streams | Kinesis Architecture and terminologies

40:05
Streaming Data using Console Producer | Python Producer and Python Consumer

39:48

Code Walkthrough

Development Phase

Deployment Phase

Project Installations and setups

Student Ratings & Reviews

No Review Yet

About Course

What Will You Learn?

Course Content

Project Flow | Data Ingestions | CDC

Comparison Between DE Project Architectures

Data Ingestion | RDBMS | CDC | Challenges | Performance

Data Ingestion | FTP to Bronze

Data Ingestion | RDBMS to Bronze | Hands On

Industry Practices for : Data Ingestion Using Glue

FTP to Bronze | Nifi Flow Industry Scenarios

NIFI Scenarios and Interview Questions

Kafka Essentials

Streaming Basics

Kafka session 1

Kafka session 2

kafka session 3

Kafka session 4

Kafka session 5

Airflow Essentials

Self Learning : Airflow Foundations

MongoDB Essentials

Introduction to NoSQL | Use Cases | Types

MongoDB Essential Elements | CRUD Operations

Indexing in MongoDB

Data Modeling in MongoDB

Sharding in MongoDB

AWS for Data Engineering

Big Data With AWS || Demo Session

AWS IAM User, Group and Policies

Big Data With AWS | IAM Roles

AWS Cloud Infrastructure

S3 Session 1

S3 Session 2

S3 Session 3

AWS glue | Crawlers And Jobs

BDA | Glue Scenarios | Glue Workflows

Glue Scenarios

Glue Project : CDC

EMR basics EC2 introduction

EMR_basics_IAM_Role

Starting EMR cluster | Connecting to EMR Cluster

AWS EMR | Starting and Deploying Spark Application on EMR

Cluster Mode Deployment | Accessing Web UIs | Steps Introduction

Deploying Spark Application in Cluster Mode on EMR

Deploying Spark Application using Steps on AWS EMR

Athena Basics

Athena on Command Line

Using Athena through python code

Redshift And Data Warehousing Introduction

Redshift Clusters | Snapshots | S3 Copy

Creating redshift cluster and making it publicly accessible

Redshift connect using sql workbench and python script

dist keys and sort keys in redshift

DIST Keys hands on

Redshift Federated Queries

What is Streaming Data | Streaming Data Terminologies

Kinesis Data Streams | Kinesis Architecture and terminologies

Streaming Data using Console Producer | Python Producer and Python Consumer

Code Walkthrough

End to End Spark App Overview

Understanding Code Flow | Bronze Layer Flow

Understanding Code Flow | Silver Layer Flow

Understanding Code Flow | Gold Layer Flow

Development Phase

Modularizing your code

logging, Validations, Environments, Scripts

Validations

Deployment Phase

Deployment Stage 1

Deploymemnt Stage 1 Internals

Deployment Stage Final

Project Installations and setups

Project | Kafka Installation

RDS MySQL Setup

Nifi Installation

Student Ratings & Reviews

Technological Geeks

Resources

Support