07-next-steps.md

Free online courses for Deep Learning

• Kadenze, creative applications of Deep Learning with Tensorflow
  https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow/info

• MIT, 6.S191: Introduction to Deep Learning
  http://introtodeeplearning.com/

• Fast.AI's Deep Learning
  https://www.fast.ai/

• Udacity, Deep Learning
  https://classroom.udacity.com/courses/ud730

• Udacity, Tensorflow (in progress)
  https://classroom.udacity.com/courses/ud187

Readings

• Deep Learning with Python by Francois Chollet | Dec 22, 2017
  https://smile.amazon.com/Deep-Learning-Python-Francois-Chollet/dp/1617294438

• Online books

• https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/
• http://neuralnetworksanddeeplearning.com/

Free online classes on Machine Learning

• Udacity, Intro to Machine Learning
  https://classroom.udacity.com/courses/ud120

• Coursera, Machine Learning
  https://www.coursera.org/learn/machine-learning

Computer Vision

• Stanford, cs231
  http://cs231n.stanford.edu

• Nancy's Brain Talks
  https://nancysbraintalks.mit.edu/

AI Projects

• Google's seedbank
  https://research.google.com/seedbank/
• Papers with code
  https://paperswithcode.com/

Kaggle

• Data sets https://www.kaggle.com/datasets

• Competitions https://www.kaggle.com/competitions

• Tutorials https://www.kaggle.com/learn/overview

• Coursera: Kaggle https://www.coursera.org/learn/competitive-data-science

Email newsletters

• https://medium.com/@founderstime/ai-newsletters-you-should-subscribe-to-part-2-3-individuals-9cf0d99006fa

Neural Networks

3brown1blue
https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi

Stephen Welch's Neural Networks

• Git for Jupyter notebooks: https://github.com/stephencwelch/Neural-Networks-Demystified

• http://www.welchlabs.com/blog/2015/1/16/neural-networks-demystified-part-1-data-and-architecture

• http://www.welchlabs.com/blog/2015/1/16/neural-networks-demystified-part-2-forward-propagation

• Jupyter for #2: https://nbviewer.jupyter.org/github/stephencwelch/Neural-Networks-Demysitifed/blob/master/Part%202%20Forward%20Propagation.ipynb

• http://www.welchlabs.com/blog/2015/1/16/neural-networks-demystified-part-3-gradient-descent

• http://www.welchlabs.com/blog/2015/1/16/neural-networks-demystified-part-4-backpropagation

• http://www.welchlabs.com/blog/2015/1/16/neural-networks-demystified-part-5-numerical-gradient-checking

• http://www.welchlabs.com/blog/2015/1/16/neural-networks-demystified-part-6

• http://www.welchlabs.com/blog/2015/1/16/neural-networks-demystified-part-7-overfitting-testing-and-regularization

Math

• Statistics
  https://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/

• Computational Algebra
  https://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/

• Eigenvectors and Eigenvalues
  https://www.udacity.com/course/eigenvectors-and-eigenvalues--ud104

Building custom GPU system?

• https://medium.com/the-mission/how-to-build-the-perfect-deep-learning-computer-and-save-thousands-of-dollars-9ec3b2eb4ce2

• https://pcpartpicker.com/b/B6LJ7P

• https://medium.com/@vitali.usau/install-cuda-10-0-cudnn-7-3-and-build-tensorflow-gpu-from-source-on-ubuntu-18-04-3daf720b83fe

• https://lambdalabs.com/deep-learning/workstations/4-gpu

• https://pcpartpicker.com/list/nwQnZ8 / https://www.gwern.net/Links

Tensorflow 2.0

• What’s coming in TensorFlow 2.0
  https://medium.com/tensorflow/whats-coming-in-tensorflow-2-0-d3663832e9b8

• Youtube: Tensorflow Dev 2019 Summit
  https://www.youtube.com/playlist?list=PLQY2H8rRoyvzoUYI26kHmKSJBedn3SQuB

• https://tfhub.dev

Hardware

GPU

• Graphics Processing Unit (vs CPU: Central Processing Unit)
• https://en.wikipedia.org/wiki/Graphics_processing_unit
• Nvidia, 1999, GeForce 256, first GPU
• Pixel shading, OpenGL, WebGL?
• Parallel GPU
• 2008, CUDA platform, GPU computing
• 2013, PS4, XBox One, GPUs from AMD's Radeon HD 7850 and 7790
• VR headsets (Oculus Rift, HTC Vive), GTX 970 and R9 290X
• 2016, Pascal architecture, GeForce 10 series (1080, 1080 Ti)
• 2018, RTX Turing GPU, Titan V. AMD's Vega GPUs
• GPUs for Deep Learning: can speed up 250 times faster than CPU
• TPU (Tensor Processing Unit) by Google
• eGPU (External GPU), outside the computer. Mac's don't have nvidia cards.

eGPU and Mac

- Only AMD GPU's are supported for Mac
- https://support.apple.com/en-us/HT208544
- https://9to5mac.com/guides/egpu/

Unofficial NVIDIA eGPU for Mac
- https://9to5mac.com/2018/05/05/nvidia-egpu-thunderbolt-macos-script-video/
- https://egpu.io/
-> Windows with Bootcamp
- https://9to5mac.com/2018/12/31/how-to-2018-mac-mini-nvidia-geforce-rtx-2080-egpu-windows-gaming-setup-video/

Multiple GPUs

• https://jhui.github.io/2017/03/07/TensorFlow-GPU/
• CIFAR 10 with multiple GPUs https://github.com/tensorflow/models/blob/master/tutorials/image/cifar10/cifar10_multi_gpu_train.py

TPU

Siraj Raval:

• https://www.youtube.com/watch?v=jgNwywYcH4w
• https://github.com/llSourcell/TPU_Machine_Learning/blob/master/TPU_MachineLearning%20(2).ipynb

Accelerates machine learning for Linux (like Raspberry Pi)

• https://coral.withgoogle.com/

• https://aiyprojects.withgoogle.com/edge-tpu

• http://linuxgizmos.com/googles-edge-tpu-ai-chip-will-run-on-a-usb-stick-computer/

• Google Coral Camera
  https://www.seeedstudio.com/Coral-Camera-Module-p-2914.html

Embedded Machine Learning

• https://heartbeat.fritz.ai/edge-tpu-google-coral-usb-accelerator-cf0d79c7ec56

• https://blog.hackster.io/say-hello-to-google-coral-cdbb49183864

• https://medium.com/@aallan/hands-on-with-the-coral-dev-board-adbcc317b6af

• https://medium.com/@aallan/hands-on-with-the-coral-usb-accelerator-a37fcb323553

• Google Coral
  https://coral.withgoogle.com/

• Intel Movidius
  https://www.amazon.com/Intel-NCSM2450-DK1-Movidius-Neural-Compute/dp/B076751BN8

• Nvidia Jetson TX2
  https://www.nvidia.com/en-us/autonomous-machines/embedded-systems-dev-kits-modules/

• https://blog.hackster.io/say-hello-to-google-coral-cdbb49183864