Skip to content

ibrahimhalilbayat/data_engineering_diary

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GitHub repo for Medium stories: -> https://medium.com/@ibrahimhalilbayat

Dairy for Data Engineering come on and learn with me.

Threads folder contains an example of threading:

* even_number_generator.py will generate an even number every 3 seconds 
* odd_number_generator.py will generate an odd number every 5 seconds
* thread_script.py will create two threads for the two scripts and execute both. 
* To see the example simply run the following code via shell: python thread_script.py

Sockets folder contains two examples of socket communication; TCP and UDP.

* tcp_server.py will listen to port 8000 in localhost for messages
* tcp_client.py will send messages from the user to port 8000 in localhost
* udp_server.py will listen to port 8002 in localhost for messages
* udp_server.py will send messages from the user to port 8002 in localhost

Kafka folder contains two Python scripts: kafka_producer.py and kafka_consumer.py

* kafka_consumer.py will listen to port 9092 and print the messages under the topic 'destroying-ring'
* kafka_producer.py will send the messages to the clients under the topic 'destroying-ring'

About

Data engineering with spark, SQL, and Kafka

Topics

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors