Course Syllabus

Artificial Neural Networks and Deep Learning

DIS Logo

stylized-image.png

Semester & Location:

Summer 2022, Session 2 - DIS Copenhagen

Type & Credits:

Elective Course - 3 credits

Study Tours:

London

Major Disciplines:

Mathematics, Computer Science

Prerequisite(s):

One year of computer science, a course in algorithms and data structures, one course in either probability theory, linear algebra, or statistics at university level. Knowledge of programming languages (e.g. in Python/Javascript/Java/C++/Matlab).

Faculty Member:

Lucian Leahu, PhD lucian.leahu@dis.dk 

Program Director:

Natalia Landázuri Sáenz, Ph.D. 

Time & Place:

Classroom: V10-D11, Please see "Course Summary" below.

 

Course Description

Artificial Neural Networks are programs that write themselves when given an objective, some data, and abundant computing power. Recently, these programs have brought about a wide array of impressive innovations, such as self-driving cars, face recognition, and human-like speech generators.

This course offers you an introduction to Artificial Neural Networks and Deep Learning. With focus on both theory and practice, we cover models for various applications, how they are trained and validated, and how they can be deployed in the wild.

Classes will be a mix of short lectures and tutorials, hands-on problem solving, and project work in groups.

Course Overview: 

  • Introduction to machine learning and neural networks: supervised learning, logistic regression for classification, basic neural network structure, simple examples and motivation for deep networks
  • Neural networks: forward propagation, cost functions, error backpropagation, training by gradient descent, bias/variance and under/overfitting, regularization
  • 5 day study tour to London
  • Convolutional Neural Networks
  • Recurrent Neural Networks
  • Special topics: Reinforcement Learning, Variational Autoencoders (VAEs) and generative adversarial networks (GANs)
  • Final project

Course Elements: 

  • Python programming
  • Supervised machine learning
  • Logistic regression and neural network fundamentals
  • Gradient descent and backpropagation
  • Regularization and the vanishing gradient problem
  • Classification (feed forward NNs)
  • Image recognition (convolutional NNs)
  • Sequence modelling (recurrent NNs)
  • Manipulating data (auto encoders and adversarial NNs)

 

Learning Objectives

Upon successfully completing the course, the student will be able to: 

  • Use the backpropagation algorithm to calculate weight gradients in a feed forward neural network by hand
  • Understand the motivation for different neural network architectures and select the appropriate  architecture for a given problem
  • Write a neural network from scratch in using PyTorch in Python, train it until convergence and test its performance given a dataset
  • Understand how neural networks fit into the more general framework of machine learning, and what their limitations and advantages are in this context

 

Faculty

Lucien Leahu, PhD in Computer Science from Cornell University 2012. Assistant professor at ITU Copenhagen since 2015. ERCIM Postdoctoral Fellow at the Swedish Institute of Computer Science (2012-2013) and Project Leader in the Media Technology and Interaction Design Department at the Royal Institute of Technology (2014). With DIS since 2019. 

 

Daniel Svendsen, PhD in Electrical Engineering from the University of Valencia 2020. Research focused on the incorporation of physical knowledge in machine learning models. Machine learning consultant since 2020 (Pensure, Greenwood Engineering and GMP4Pharma). MSc in Mathematical modelling and computation from the Technical University of Denmark 2016. Teaching assistant in various courses 2015-2016. With DIS since 2021.

 

Readings

Most of the learning will be based on parts of the following books:

  • Goodfellow et al., Deep Learning.
  • Nielsen, Neural Networks and Deep Learning

Additional possible sources include blog posts, videos available online, and scientific papers.

Approach to Teaching

The course is designed around the principle of constructive alignment. The two major components in the course—the assignments and the final project—implement this principle by stating clear outcome goals of every activity and the course as a whole.

Another small but important component of the teaching approach is peer evaluation. Each student is tasked with reviewing 2 assignments after handing in their own (with or without a group). The reviewing process is anonymous. Using peer evaluations, each hand in gets a lot of varied feedback, and lets students reflect on their own work by reviewing how others solved the same problems. High quality feedback is incentivized by having each reviewee rate their received feedback such as to produce a feedback quality score for every reviewer which, by a small fraction, influences their final grade. 

Expectations of the Students

Students are expected to reach the preparation goal leading up to each session. Students who have little or no experience coding in Python should either follow a Python tutorial before the course starts, or prepare to invest some hours getting up to speed with the language once we start. Students should have a working laptop computer. It is advised that each machine has a least 4 GB of RAM and a reasonable processor.

Evaluation

During the course you will work on exercise sets during in-class programming sessions (labs). Furthermore, you will complete a larger project that uses tools which have been taught in the class. An acceptable project will cover e.g. data scraping and analysis. You will be allowed to define your own project, but you can also get assistance from the instructor.

Projects are group efforts. You are allowed to consult freely with any of the other students and the instructor. Contributions from other students, however, must be acknowledged with citations in your final report, as required by academic standards. Contributions to your presentations must similarly be acknowledged. Needless to say, the right to consult does not include the right to copy — programs, papers, and presentations must be your own original work. 

Grading

When assigning the final grades, your efforts will weigh as follows: 

  • Participation: 15% (includes class/exercise/project/field study/study trip behavior that is beneficial to the learning of others)
  • Lab exercises: 35%
  • Final project: 50% (10% proposal presentation, 30% project report + code repository and 10% presentation)

Lab Exercises: There will be 6 lab sets consisting of exercises (including coding exercises).  These will be introduced in class after each lecture.  The students will work on them individually during the afternoon session of the class and will turn them in before the beginning of class the following day.  Though submission is individual, we encourage students to discuss the exercises and the solutions with each other; if you collaborate with one or more colleagues, you must acknowledge them in your submission.  It is not ok to simply copy a colleagues solution.

Final project: From the beginning of the course the students are aware that an outcome of the course is a project that, if done well, can add value to their professional portfolio. The instructors will offer some topics which the students may pursue in their project.  The project can be a small study on some topic of students choosing, however this topic must be approved by the instructors. After they have completed their project students must communicate the results in the popular format of a blog post and deliver an in class presentation.  The final submission consists of links to the blog post and to their code repository.

Academic Regulations  

Please make sure to read the Academic Regulations on the DIS website. There you will find regulations on:

 

DIS - Study Abroad in Scandinavia - www.DISabroad.orgLinks to an external site.

 

Course Summary:

Date Details Due