Artificial Neural Networks and Deep Learning
|Semester & Location:||
Summer 2023, Session 2 - DIS Copenhagen
|Type & Credits:||
Elective Course - 3 credits
Mathematics, Computer Science
Lucian Leahu, PhD email@example.com
Natalia Landázuri Sáenz, Ph.D
|Time & Place:||
Time: please see "Course Summary" below. Place: Classroom N7-B13
Artificial Neural Networks are programs that write themselves when given an objective, some data, and abundant computing power. Recently, these programs have brought about a wide array of impressive innovations, such as self-driving cars, face recognition, and human-like speech generators.
This course offers you an introduction to Artificial Neural Networks and Deep Learning. With focus on both theory and practice, we cover models for various applications, how they are trained and validated, and how they can be deployed in the wild.
Classes will be a mix of lectures, tutorials, hands-on problem solving, and programming work in groups.
- Introduction to machine learning and neural networks: supervised learning, logistic regression for classification, basic neural network structure, simple examples and motivation for deep networks
- Neural networks: forward propagation, cost functions, error backpropagation, training by gradient descent, bias/variance and under/overfitting, regularization
- 5 day study tour to London
- Convolutional Neural Networks
- Recurrent Neural Networks
- Special topics: Variational Autoencoders (VAEs) and generative adversarial networks (GANs)
- Final exam
- Python programming
- Supervised machine learning
- Logistic regression and neural network fundamentals
- Gradient descent and backpropagation
- Regularization and the vanishing gradient problem
- Classification (feed forward NNs)
- Image recognition (convolutional NNs)
- Sequence modelling (recurrent NNs)
- Manipulating data (auto encoders and adversarial NNs)
Upon successfully completing the course, the student will be able to:
- Use the backpropagation algorithm to calculate weight gradients in a feed forward neural network by hand
- Understand the motivation for different neural network architectures and select the appropriate architecture for a given problem
- Write a neural network from scratch in using PyTorch in Python, train it until convergence and test its performance given a dataset
- Understand how neural networks fit into the more general framework of machine learning, and what their limitations and advantages are in this context
Lucian Leahu, PhD in Computer Science from Cornell University 2012. ERCIM Postdoctoral Fellow at the Swedish Institute of Computer Science (2012-2013) and Project Leader in the Media Technology and Interaction Design Department at the Royal Institute of Technology (2014). Assistant professor at ITU Copenhagen (2015-2018). With DIS since 2019.
Most of the learning will be based on parts of the following books:
- Goodfellow et al., Deep Learning.
- Nielsen, Neural Networks and Deep Learning
Additional possible sources include blog posts, videos available online, and scientific papers.
Approach to Teaching
The course is designed around the principle of constructive alignment. The two major components in the course—the assignments and the final exam—implement this principle by stating clear outcome goals of every activity and the course as a whole.
Expectations of the Students
Students are expected to reach the preparation goal leading up to each session. Students who have little or no experience coding in Python should either follow a Python tutorial before the course starts, or prepare to invest some hours getting up to speed with the language once we start. Students should have a working laptop computer. It is advised that each machine has a least 4 GB of RAM and a reasonable processor.
During the course you will work in small groups on exercise sets during in-class programming sessions (labs). You will submit them on Canvas before the beginning of the following class.
The labs are group efforts; groups will be randomly generated on Canvas. During class time, you are allowed to consult freely with any of the other students in the course and the instructor. Needless to say, the right to consult does not include the right to copy — the code, papers, and/or presentations must be your own work.
Furthermore, you will take a final exam consisting of questions on topics covered in the course, including academic visits while on tour.
When assigning the final grades, your efforts will weigh as follows:
- Participation: 15% (includes class/exercise/project/field study/study trip behavior that is beneficial to the learning of others)
- Lab exercises: 42%
- Final project: 43%
Lab Exercises: There will be 6 lab sets consisting of exercises (including coding exercises). These will be introduced in class after each lecture. The students will work on them in small groups during the afternoon session and will turn them in before the beginning of class the following day. We encourage students to discuss the exercises and the solutions with other groups as needed; if you collaborate with other groups, you must acknowledge them in your submission. It is not allowed to copy others' solution.
Final exam: This is an individual, written exam consisting of a mixture of T/F, multiple choice, open answer questions, as well as code that you will have to interpret and/or complete/modify. The exam may include hypothetical project questions: given a particular data set and modeling question, how would you approach the task with neural networks?
Please make sure to read the Academic Regulations on the DIS website. There you will find regulations on:
DIS - Study Abroad in Scandinavia - www.DISabroad.org
The syllabus page shows a table-oriented view of the course schedule, and the basics of course grading. You can add any other comments, notes, or thoughts you have about the course structure, course policies or anything else.
To add some comments, click the "Edit" link at the top.