Course Syllabus

Artificial Neural Networks and Deep Learning

Semester & Location:	Spring 2024 - DIS Copenhagen
Type & Credits:	Core Course - 3 credits
Major Disciplines:	Computer Science, Information Science, Mathematics
Prerequisite(s):	One year of computer science, a course in algorithms and data structures, one course in either probability theory, linear algebra, or statistics at university level. Knowledge of programming languages (e.g. in Python/Javascript/Java/C++/Matlab).
Faculty Members:	Lucian Leahu, PhD (current students please use the Canvas Inbox to contact faculty)
Program Director:	Natalia Landázuri Sáenz, PhD
Study Tours:	Denmark, Stockholm
Academic Support:	csc-engr@disstockholm.se
Time & Place:	Thursdays 8:30-11:25 Classroom: N7-A21

 

Course Description

Artificial Neural Networks are programs that write themselves when given an objective, some data, and abundant computing power. Recently, these programs have brought about a wide array of impressive innovations, such as self-driving cars, face recognition, and human-like speech generators.

This course offers you an introduction to Artificial Neural Networks and Deep Learning. With focus on both theory and practice, we cover models for various applications, how they are trained and validated, and how they can be deployed in the wild.

Classes will be a mix of lectures, tutorials, hands-on problem solving, and project work in groups.

Course elements

• Python programming
• Supervised machine learning
• Logistic regression and neural network fundamentals
• Gradient descent and backpropagation
• Regularization and the vanishing gradient problem
• Classification (feed forward NNs)
• Image recognition (convolutional NNs)
• Sequence modelling (recurrent NNs)
• Generative models (auto encoders and adversarial NNs)

Course overview

• Intro to machine learning and neural networks: supervised learning, logistic regression for classification, basic neural network structure, simple examples and motivation for deep networks.
• Neural networks: forward propagation, cost functions, error backpropagation, training by gradient descent, bias/variance and under/overfitting, regularization.
• 1 week travel (core course week).
• Convolutional Neural Networks.
• Transfer Learning.
• Recurrent Neural Networks.
• Autoencoders and adversarial networks.
• Final project

Learning Objectives

Upon successfully completing the course, the student will be able to: 

• Understand how neural networks fit into the more general framework of machine learning, and what their limitations and advantages are in this context.
• Use the backpropagation algorithm to calculate weight gradients in a feed forward neural network by hand.
• Understand the motivation for different neural network architectures and select the appropriate architecture for a given problem.
• Describe a neural network in Keras, train it until convergence and test its performance given a dataset.

Faculty 

Lucian Leahu.   PhD in Computer Science from Cornell University 2012. Assistant professor at ITU Copenhagen since 2015. ERCIM Postdoctoral Fellow at the Swedish Institute of Computer Science (2012-2013) and Project Leader in the Media Technology and Interaction Design Department at the Royal Institute of Technology (2014). With DIS since 2019.

Readings

Most of the learning will be based on parts of the following books:

• Goodfellow et al., Deep Learning.
• Nielsen, Neural Networks and Deep Learning

Additional possible sources include blog posts, videos available online, and scientific papers.

Approach to Teaching

The course is designed around the principle of constructive alignment. The three major components in the course—the assignments, the final and the final project—implement this principle by stating clear outcome goals of every activity and the course as a whole.

Assignments: Leading up to each session, students are given a "preparation goal" and a suggested list of materials they can use to reach it. Sessions start with a short lecture (less than 1 hour) that introduces the topic of the day, and then students work through a set of technical exercises. Each week, the students will be assigned to a random group of 2-3 students -- the groups will work through the technical exercises together, in class under the instructor's supervision, and afterwards on their own.  They have a week to hand-in their solutions.  Many groups complete the assignment in class. This gives the students a clear outcome goal for each session: "show up prepared and complete the exercises". It incentivises the students to prepare for class and be focused in class.

Final project: From the beginning of the course the students are aware that an outcome of the course is a project that, if done well, can add value to their professional portfolio. The project is a small study on some popular topic of their own choosing that they can investigate with data they have scraped or downloaded from the Internet. They submit the project in two parts: First, each team must compose a proposal presentation which demonstrates that they have made a plan for their project and are able to hypothesize about the outcomes. Second, after they have completed their project they must communicate the results in the popular format of a blog post. The proposal presentations serve as a platform for sharing ideas between groups and it gets the students to start with a very comprehensive idea of the outcome in mind.

Expectations of the Students

Students are expected to reach the preparation goal leading up to each session. Students who have little or no experience coding in Python should either follow a Python tutorial before the course starts, or prepare to invest some hours getting up to speed with the language once we start. Students should have a working laptop computer. It is advised that each machine has a least 4 GB of RAM and a reasonable processor (if it’s bought after 2019 you should be fine). The Unix operating system is preferred (OSX and Linux), but not a necessity.

Evaluation

During the course you will hand in seven weekly assignments containing exercises solved in class. Furthermore, you will complete a larger project that uses tools which have been taught in the class. An acceptable project will cover e.g. obtaining data, preparing the data for modelling, choosing an appropriate set of methods, visualizing results etc. Students pick their own group and the topic for their project, but they can also get assistance from the instructor.

Both project and assignments are group efforts.

During the programming project, you are allowed to consult freely with any of the other students and the instructor. Contributions from other students, however, must be acknowledged with citations in your final report, as required by academic standards. Contributions to your presentations must similarly be acknowledged. Needless to say, the right to consult does not include the right to copy — programs, papers, and presentations must be your own original work. 

The participation grade reflects a student's contributions to classes, exercises, comments on other students' questions on the Discussion boards, attendance and engagement with guest speakers and during field studies.  Inappropriate and/or unprofessional behaviour (e.g., sleeping during presentations, being rude towards our hosts during field studies) results in a score of 0 for participation for the entire semester.

Grading

When assigning the final grades, your efforts will weigh as follows: 

Participation: (includes attendance, engagement, attitudes that are beneficial to the learning of others)	15%
Assignments: 7 programming assignments	45%
Final Project:  (5% proposal presentation, 25% project report and presentation)	40%

 

Academic Regulations  

Please make sure to read the Academic Regulations on the DIS website. There you will find regulations on: 

• Course Enrollment and Grading  
• Academic Expectations and Honor Code