Deep Learning for Visual Computing Course @ KAUST
Welcome to the CS 323 - Deep Learning for Visual Computing course! Throughout the course you will learn and implement various deep learning methods with their applications in visual computing. As mentioned in the prerequisites, we expect the students to be familiar with some basic machine learning concepts, linear algebra, multivariate calculus, probability and basic to intermediate programming skills.
To get yourselves started and/or refresh your concepts, you can refer to the following links:
Linear Algebra: Khan Academy’s linear algebra course provides a good resource to learn/revise the concepts online. Moreover, you can also follow the YouTube Channel 3Blue1Brown for amazing explanations of mathematical concepts including linear algebra.
Programming: You should be familiar with Python. For deep learning, mainly we would be using the PyTorch framework. We assume familiarity with Python, but would expect that students are able to study PyTorch on their own to implement the concepts discussed in class. We recommend you make yourselves familiar with the programming frameworks.
Here is a list of some other resources to help you learn about Python and the relevant libraries and packages commonly used in developing deep learning models.
numpywhich is the main library used for scientific computing in Python.
Please note that you need to make sure you have a higher end Nvidia GPU enabled device for this course. Training/running the code on the CPU becomes impractical with deep neural networks. You can talk to your academic advisor for the procurement/availability of such devices and/or refer to cloud services such as Google Colab.
The two disadvantages to using Colab are speed and storage. Colab sessions tend to be slower. Experiments need to be saved to the cloud or manually downloaded before the free 24 hours session is erased and all your local data will be lost.
condapackage and environment manager. For a quick introduction on how to install (and use)
condaplease read the following guide.
Once you have
conda installed, open the newly installed
Anaconda prompt application on Windows (or a simple terminal on Linux) and run the following
conda create --name pytorch pytorch torchvision cudatoolkit=10.0 jupyterlab nb_conda pillow matplotlib scikit-learn scikit-image h5py -c pytorch
For up-to-date information about the different PyTorch versions and suitable installation commands please take a look here.
You can then activate this new environment (called
pytorch) on Windows by running:
conda activate pytorch
on Linux. You can now start implementing deep learning models using PyTorch by simply running:
A browser window should automatically show JupyterLab. JupyterLab is a browser based IDE that allows you to create interactive Python
notebooks. Make sure to
select the right kernel type (e.g.
Python [conda env:conda-pytorch]).
To check the installation, run the following inside your notebooks:
import torch torch.cuda.is_available()
For a successful installation, it should return True.
Ibex is the name of the clusters at our campus. More information can be found here
A very concise introduction to the usage of Ibex can be found here
Here, we will introduce the basic functions of Ibex which we will use in this class. To connect to IBEX from a Windows machine, we highly recommend using MobaXterm.
ssh -X email@example.com
Change “username” to your portal ID.
module av AppName # view if the app is available module load AppName # load the app module purge AppName # unload the app
module load miniconda3/188.8.131.52 # load miniconda
salloc --cpus-per-task=2 --gres=gpu:gtx1080ti:4 --mem=16GB --time=8:00:00 --mail-type ALL
In the above command, we want the system to give us a GPU node with 4 GTX1080ti, 2 processors and 16GB memory. It allows us to use this node freely for a duration of 8 hours. You have to estimate the running time of your job before the allocation. Then you need to login to the allocated node,
srun --pty bash -i
The command starts an interactive shell which you can operate like in your local machine.
If it is successful, you can check the GPU by
srun --resv-ports=1 --pty bash -i jupyter notebook --no-browser --ip=0.0.0.0 --port=$SLURM_STEP_RESV_PORTS
More information can be found here