Deep and Reinforcement Learning

Code:

M.IA003

Acronym:

ACPR

Keywords
Classification	Keyword
OFICIAL	Informatics Engineering
OFICIAL	Computer Science

Instance: 2024/2025 - 2S

Active?	Yes
Responsible unit:	Department of Informatics Engineering
Course/CS Responsible:	Master in Artificial Intelligence

Cycles of Study/Courses

Acronym	No. of Students	Study Plan	Curricular Years	Credits UCN	Credits ECTS	Contact hours	Total Time
M.IA	61	Syllabus	1	-	6	42	162

Teaching language

English

Objectives

- Understand the functioning of deep learning and reinforcement learning models

- Be able to select the most appropriate algorithms, model details, and learning techniques for various tasks

- Learn to design, test, and improve deep learning and reinforcement learning models for given tasks

Learning outcomes and competences

The students should be capable of:

1. Understand the fundamentals and main algorithms of Deep Learning (DL) and Reinforcement Learning (RL)


2. Identify DL and RL techniques that are suitable for different Machine Learning and Artificial Intelligence problems


3. Develop new methods of solving proposed problems


4. Apply the methods to concrete problems and evaluate the results

Working method

Presencial

Pre-requirements (prior knowledge) and co-requirements (common knowledge)

- Basic knowledge of machine learning and neural networks

Program

The course unit will be organized into interconnected modules, covering at least the following topics:

 

1 – Neural Networks

• Convolutional Neural Networks
• Recurrent Neural Networks
• Transformers
• Graph Neural Networks

 

2 – Generative Models

• Generative Adversarial Networks
• Variational Autoencoders
• Diffusion Models

 

3 – Advanced Methods

• Self-Supervised Learning
• Foundation Models
• Federated Learning

 

4 – Reinforcement Learning

• Policy- and Value-Based Algorithms
• Actor-Critic Algorithms
• Explainable Reinforcement Learning
• AutoRL

Mandatory literature

Chollet, F; Deep learning with Python, Simon and Schuster., 2021
Bilgin, E.; Mastering Reinforcement Learning with Python: Build Next-generation, Self-learning Models Using Reinforcement Learning Techniques and Best Practices, 2020
Lapan, M. ; Deep Reinforcement Learning Hands-On: Apply modern RL methods to practical problems of chatbots, robotics, discrete optimization, web automation, and more, Packt Publishing Ltd., 2020
Goodfellow, I., Bengio, Y., & Courville, A.; Deep learning, MIT Press., 2016

Teaching methods and learning activities

Partially expository classes (about 50%) with the presentation of concepts, algorithms and application examples. The remaining classes will be used for proposed practical exercises and for monitoring the development of projects throughout the UC. Projects will be completed outside of class. There will be an individual final exam.

Evaluation Type

Distributed evaluation with final exam

Assessment Components

Designation	Weight (%)
Trabalho prático ou de projeto	40,00
Exame	60,00
Total:	100,00

Amount of time allocated to each course unit

Designation	Time (hours)
Elaboração de projeto	60,00
Estudo autónomo	60,00
Frequência das aulas	42,00
Total:	162,00

Eligibility for exams

A minimum score of 6 out of 20 (2.4 out of 8) in the practical work.

Calculation formula of final grade

Final Grade = 60% * Exam + 40% * Practical Work

To pass, students must score at least 8 out of 20 (or 4.8 out of 12) in the final exam.

Classification improvement

It is only possible to improve the grade of the theoretical component. The exam takes place exclusively during the supplementary examination period.