Course Syllabus

 

Note: We will be using Python in this course. If you have little or no experience coding in Python, you should either follow a Python tutorial before the course starts, or prepare to invest some hours getting up to speed with the language once we start.   

Course Description

Data is the fuel powering today’s digital world, from social media and streaming platforms to AI systems and smart cities. This course introduces students to the exciting field of Data Engineering, where raw data is transformed into reliable, scalable, and intelligent systems that drive modern applications. Students will learn how to collect, process, store, and analyze massive amounts of data using industry-standard tools and real-world workflows. Through hands-on projects, students will build data pipelines, work with cloud technologies, automate workflows, and explore big data systems used by companies such as Netflix, Spotify, and Uber. The course emphasizes practical experience with Python, SQL, distributed processing, and modern data architectures, giving students the foundational skills needed for careers in data engineering, analytics, AI infrastructure, and large-scale software systems.

The course will cover the following topics:

• Introduction to Data Engineering
• Data Collection and Ingestion
• Data Cleaning and Transformation
• Databases and SQL Fundamentals
• Advanced SQL and Data Modeling
• Data Warehousing
• Workflow Orchestration
• Big Data Fundamentals
• Distributed Processing with Spark
• Cloud Data Engineering
• Streaming and Real-Time Data
• Containerization
• Data Quality and Governance
• Building Modern Data Platforms

Learning Objectives

Knowledge/concepts that will be covered, discussed and acquired during the course:

• Understand what data engineering is and how data pipelines power modern apps
• Learn SQL for querying, managing, and modeling structured data
• Build data pipelines that collect, clean, and transform raw data
• Work with data warehouses and organized storage systems for analytics
• Use Python to automate and build data processing workflows
• Learn big data concepts for handling large-scale datasets efficiently
• Process distributed data using tools like Apache Spark
• Explore cloud-based data engineering systems and scalable infrastructure
• Ensure data quality, reliability, and governance in production systems

Skills that will be acquired during the course:

• SQL proficiency for querying, transforming, and modeling structured data
• Python programming for data engineering, including automation and data processing scripts
• Data pipeline development to move and transform data from raw sources to usable outputs
• Data cleaning and transformation skills to handle messy, real-world datasets
• Big data processing with distributed systems, including tools like Apache Spark
• Cloud data engineering skills for building and deploying scalable data systems
• Data quality and workflow orchestration to automate, monitor, and ensure reliable data pipelines

Faculty

TBA

Readings

Sections from the following books, in addition to online material that will be posted for each session.

1. Fundamentals of Data Engineering: Plan and Build Robust Data Systems, by Joe Reis

3. Python for Data Analysis, by Wes McKinney 

Field studies

Possible visits can include:

• Visit to universities and research institutes to learn about the latest research and development in the field of Data Engineering
• Visit to companies leading the development and implementation of Data Engineering

Approach to Teaching

The course is designed around the principle of constructive alignment. The major components in the course—the exams and the final project —implement this principle by stating clear goals and activities for every session and in the context of the course as a whole.

You are expected to engage actively in classroom discussions, presentations, exercises, and group work. In addition, you will participate in local field studies and extended course-integrated study tours within Sweden and in Amsterdam These visits give you the opportunity to learn first-hand from leaders in the field of data engineering, to speak with professionals about their cutting-edge work, and to better understand specific approaches to research, development, implementation and utilization of data engineering and data science principles.

Leading up to each session, you are given a "preparation goal" and a suggested list of materials you can use to reach it. Sessions start with theory that introduces the topic of the day. You then work through a set of technical exercises. This gives you a clear outcome goal for each session: "show up prepared and complete the exercises". It gives you incentive to be prepared and focus on the work.

Core Course Week and Study Tours

Core course week and study tours are integral parts of the core course. The classroom is “on the road” and theory presented in the classroom is applied in the field. You will travel with classmates and DIS faculty/staff on two study tours: a short study tour during the core course week and a long study tour to relevant European destinations. You are expected to

• participate in all activities
• show up in time for visits and activities
• engage in discussions, ask questions, and contribute to achieving the learning objectives
• be respectful to the destination/location, the speakers, DIS staff, and fellow classmates
• represent self, home university and DIS in a positive light

While on a program study tour, DIS will provide hostel/hotel accommodation, transportation to/from the destination(s), approx. 2 meals per day and entrances, guides, and visits relevant to your area of study or the destination. You will receive a more detailed itinerary prior to departure.

Travel policies: You are required to travel with your group to the destination. If you have to deviate from the group travel plans, you need approval from the program director and the study tours office. 

Expectations of the Students

If you have little or no experience coding in Python, you should either follow a Python tutorial before the course starts, or prepare to invest some hours getting up to speed with the language once we start. You should have a working laptop computer. 

• You are expected to reach the preparation goal leading up to each session.
• You should participate actively during lectures, discussions, group work, and exercises.
• Laptops may be used for note‐taking, fact‐checking, or assignments in the classroom, but only when indicated by the instructor. At all other times, laptops and electronic devices should be put away during class meetings.
• Readings must be done prior to the class session. 
• In addition to completing all assignments and exams, you need to be present, arrive on time, and actively participate in all classes and field studies to receive full credit. Your final grade will be affected, adversely, by unexcused absences and lack of participation. Your participation grade will be reduced by 10 points (over 100) for every unexcused absence. Remember to be in class on time!
• Classroom etiquette includes being respectful of other opinions, listening to others and entering a dialogue in a constructive manner.
• You are expected to ask relevant questions in regards to the material covered.
• Excuses for any absences must be given beforehand or as soon as possible. It is the responsibility of the student to make up any missed coursework.

Evaluation and Grading

When assigning the final grades, your efforts will weigh as follows: 

• Active participation: 20% (includes active participation and engagement in class and on study tours, exercises and project, being punctual to class and visits, and contributing to behavior beneficial to the learning of your peers)
• Weekly Assignments 40%
• Final project: 40% 

Active participation: It is mandatory to actively participate and engage in all scheduled sessions and activities in the course. You are expected to actively participate, engage in the topic and with the teacher, the guest lecturer, the hosts of the study visit etc. You are expected to show a behavior which is inducing to your peers' learning. In group exercises, you are expected to actively participate, engage, help your peers in their learning and contribute equally to the solution. Inappropriate, disrespectful and/or unprofessional behavior (e.g., sleeping during presentations, being rude towards our hosts during field studies) results in a score of 0 for participation for the entire semester.

Final project: From the beginning of the course, you are aware that an important outcome of the course is a project that could add value to your professional portfolio. To accomplish the project, you will work in teams. Each team selects a topic of their interest, and investigate it using data that needs to scraped or downloaded from the Internet. Student teams submit the project in two parts. First, each team must prepare a project proposal which demonstrates that they have a sound plan for their project and have clear hypotheses related to expected outcomes. The proposal presentation is a fun exercise that serves as a platform for sharing ideas between groups. We will view them all in class. In addition, it helps you start with a comprehensive idea of an outcome in mind. Second, once the project is finalised, student teams are required to communicate the results in the popular format of a presentation and github repository with the final solution. 

During the project, you are allowed to consult freely with any of the other students and the instructor. Contributions from other students, however, must be acknowledged with citations in your final report, as required by academic standards. Contributions to your presentations must similarly be acknowledged. Needless to say, the right to consult does not include the right to copy — programs, papers, and presentations must be your own original work. 

You are expected to respect all submission deadlines. If an submission is turned in after the due date, the grade of the assignment will either be reduced for each day the submission is late.

 

DIS Accommodations Statement 

Your learning experience in this class is important to me.  If you have approved academic accommodations with DIS, please make sure I receive your DIS accommodations letter within two weeks from the start of classes. If you can think of other ways I can support your learning, please don't hesitate to talk to me. If you have any further questions about your academic accommodations, contact Academic Support acadsupport@disstockholm.se.