Course Syllabus

Jump to Today

Artificial Neural Networks and Deep Learning

 DIS Logo

stylized-image.png

Semester & Location:

Summer 2023, Session 2 - DIS Copenhagen
Type & Credits:

Elective Course - 3 credits
Study Tours:

London
Major Disciplines:

Mathematics, Computer Science
Prerequisite(s):

One year of computer science, a course in algorithms and data structures, one course in either probability theory, linear algebra, or statistics at university level. Knowledge of programming languages (e.g. in Python/Javascript/Java/C++/Matlab).
Faculty Member:

Lucian Leahu, PhD lucian.leahu@dis.dk
Program Director:

Natalia Landázuri Sáenz, Ph.D
Time & Place:

Time: please see "Course Summary" below.  Place: Classroom N7-B13

 

Course Description

Artificial Neural Networks are programs that write themselves when given an objective, some data, and abundant computing power. Recently, these programs have brought about a wide array of impressive innovations, such as self-driving cars, face recognition, and human-like speech generators.

This course offers you an introduction to Artificial Neural Networks and Deep Learning. With focus on both theory and practice, we cover models for various applications, how they are trained and validated, and how they can be deployed in the wild.

Classes will be a mix of lectures, tutorials, hands-on problem solving, and programming work in groups.

Course Overview: 

  • Introduction to machine learning and neural networks: supervised learning, logistic regression for classification, basic neural network structure, simple examples and motivation for deep networks
  • Neural networks: forward propagation, cost functions, error backpropagation, training by gradient descent, bias/variance and under/overfitting, regularization
  • 5 day study tour to London
  • Convolutional Neural Networks
  • Recurrent Neural Networks
  • Special topics: Variational Autoencoders (VAEs) and generative adversarial networks (GANs)
  • Final exam

Course Elements: 

  • Python programming
  • Supervised machine learning
  • Logistic regression and neural network fundamentals
  • Gradient descent and backpropagation
  • Regularization and the vanishing gradient problem
  • Classification (feed forward NNs)
  • Image recognition (convolutional NNs)
  • Sequence modelling (recurrent NNs)
  • Manipulating data (auto encoders and adversarial NNs)

 

Learning Objectives

Upon successfully completing the course, the student will be able to: 

  • Use the backpropagation algorithm to calculate weight gradients in a feed forward neural network by hand
  • Understand the motivation for different neural network architectures and select the appropriate  architecture for a given problem
  • Write a neural network from scratch in using PyTorch in Python, train it until convergence and test its performance given a dataset
  • Understand how neural networks fit into the more general framework of machine learning, and what their limitations and advantages are in this context

 

Faculty

Lucian Leahu, PhD in Computer Science from Cornell University 2012. ERCIM Postdoctoral Fellow at the Swedish Institute of Computer Science (2012-2013) and Project Leader in the Media Technology and Interaction Design Department at the Royal Institute of Technology (2014). Assistant professor at ITU Copenhagen (2015-2018). With DIS since 2019.

 

Readings

Most of the learning will be based on parts of the following books:

  • Goodfellow et al., Deep Learning.
  • Nielsen, Neural Networks and Deep Learning

Additional possible sources include blog posts, videos available online, and scientific papers.

Approach to Teaching

The course is designed around the principle of constructive alignment. The two major components in the course—the assignments and the final exam—implement this principle by stating clear outcome goals of every activity and the course as a whole.

Expectations of the Students

Students are expected to reach the preparation goal leading up to each session. Students who have little or no experience coding in Python should either follow a Python tutorial before the course starts, or prepare to invest some hours getting up to speed with the language once we start. Students should have a working laptop computer. It is advised that each machine has a least 4 GB of RAM and a reasonable processor.

Evaluation

During the course you will work in small groups on exercise sets during in-class programming sessions (labs). You will submit them on Canvas before the beginning of the following class.  

The labs are group efforts; groups will be randomly generated on Canvas. During class time, you are allowed to consult freely with any of the other students in the course and the instructor. Needless to say, the right to consult does not include the right to copy — the code, papers, and/or presentations must be your own work. 

Furthermore, you will take a final exam consisting of questions  on topics covered in the course, including academic visits while on tour.

Grading

When assigning the final grades, your efforts will weigh as follows: 

  • Participation: 15% (includes class/exercise/project/field study/study trip behavior that is beneficial to the learning of others)
  • Lab exercises: 42%
  • Final project: 43%

Lab Exercises: There will be 6 lab sets consisting of exercises (including coding exercises).  These will be introduced in class after each lecture.  The students will work on them in small groups during the afternoon session and will turn them in before the beginning of class the following day.  We encourage students to discuss the exercises and the solutions with other groups as needed; if you collaborate with other groups, you must acknowledge them in your submission.  It is not allowed to copy others' solution.

Final exam: This is an individual, written exam consisting of a mixture of T/F, multiple choice, open answer questions, as well as code that you will have to interpret and/or complete/modify.  The exam may include hypothetical project questions: given a particular data set and modeling question, how would you approach the task with neural networks?

Academic Regulations  

Please make sure to read the Academic Regulations on the DIS website. There you will find regulations on:

 

DIS - Study Abroad in Scandinavia - www.DISabroad.orgLinks to an external site.

 

Course Summary:

Date Details Due