This repository contains material related to Udacity's Deep Learning v7 Nanodegree program. It consists of a bunch of tutorial notebooks for various deep learning topics. In most cases, the notebooks lead you through implementing models such as convolutional networks, recurrent networks, and GANs. There are other topics covered such as weight initialization and batch normalization.
There are also notebooks used as projects for the Nanodegree program. In the program itself, the projects are reviewed by real people (Udacity reviewers), but the starting code is available here, as well.
Table Of Contents
Tutorials
Introduction to Neural Networks
Introduction to Neural Networks: Learn how to implement gradient descent and apply it to predicting patterns in student admissions data.
Introduction to PyTorch: Learn how to build neural networks in PyTorch and use pre-trained networks for state-of-the-art image classifiers.
Convolutional Neural Networks
Convolutional Neural Networks: Visualize the output of layers that make up a CNN. Learn how to define and train a CNN for classifying MNIST data, a handwritten digit database that is notorious in the fields of machine and deep learning. Also, define and train a CNN for classifying images in the CIFAR10 dataset.
Transfer Learning. In practice, most people don't train their own networks on huge datasets; they use pre-trained networks such as VGGnet. Here you'll use VGGnet to help classify images of flowers without training an end-to-end network from scratch.
Weight Initialization: Explore how initializing network weights affects performance.
Autoencoders: Build models for image compression and de-noising, using feedforward and convolutional networks in PyTorch.
Style Transfer: Extract style and content features from images, using a pre-trained network. Implement style transfer according to the paper, Image Style Transfer Using Convolutional Neural Networks by Gatys et. al. Define appropriate losses for iteratively creating a target, style-transferred image of your own design!
Recurrent Neural Networks
Intro to Recurrent Networks (Time series & Character-level RNN): Recurrent neural networks are able to use information about the sequence of data, such as the sequence of characters in text; learn how to implement these in PyTorch for a variety of tasks.
Embeddings (Word2Vec): Implement the Word2Vec model to find semantic representations of words for use in natural language processing.
Sentiment Analysis RNN: Implement a recurrent neural network that can predict if the text of a moview review is positive or negative.
Attention: Implement attention and apply it to annotation vectors.
Batch Normalization: Learn how to improve training rates and network stability with batch normalizations.
Deep Convolutional GAN (DCGAN): Implement a DCGAN to generate new images based on the Street View House Numbers (SVHN) dataset.
CycleGAN: Implement a CycleGAN that is designed to learn from unpaired and unlabeled data; use trained generators to transform images from summer to winter and vice versa.
Deploying a Model (with AWS SageMaker)
All exercise and project notebooks for the lessons on model deployment can be found in the linked, Github repo. Learn to deploy pre-trained models using AWS SageMaker.
Conda is an open source package management system and environment management system
for installing multiple versions of software packages and their dependencies and
switching easily between them. It works on Linux, OS X and Windows, and was created
for Python programs but can package and distribute any software.
Overview
Using Anaconda consists of the following:
Install miniconda on your computer, by selecting the latest Python version for your operating system. If you already have conda or miniconda installed, you should be able to skip this step and move on to step 2.
For Windows users, these following commands need to be executed from the Anaconda prompt as opposed to a Windows terminal window. For Mac, a normal terminal window will work.
Git and version control
These instructions also assume you have git installed for working with Github from a terminal window, but if you do not, you can download that first with the command:
At this point your command line should look something like: (deep-learning) <User>:deep-learning-v2-pytorch <user>$. The (deep-learning) indicates that your environment has been activated, and you can proceed with further package installations.
Install PyTorch and torchvision; this should install the latest version of PyTorch.
Install a few required pip packages, which are specified in the requirements text file (including OpenCV).
pip install -r requirements.txt
That's it!
Now most of the deep-learning libraries are available to you. Very occasionally, you will see a repository with an addition requirements file, which exists should you want to use TensorFlow and Keras, for example. In this case, you're encouraged to install another library to your existing environment, or create a new environment for a specific project.
Now, assuming your deep-learning environment is still activated, you can navigate to the main repo and start looking at the notebooks:
cd
cd deep-learning-v2-pytorch
jupyter notebook
To exit the environment when you have completed your work session, simply close the terminal window.
请发表评论