Advanced topics on Optimization

Code:

M4026

Acronym:

M4026

Keywords
Classification	Keyword
OFICIAL	Mathematics

Instance: 2018/2019 - 2S

Active?	Yes
Responsible unit:	Department of Mathematics
Course/CS Responsible:	Master in Mathematical Engineering

Cycles of Study/Courses

Acronym	No. of Students	Study Plan	Curricular Years	Credits UCN	Credits ECTS	Contact hours	Total Time
M:ENM	21	Official Study Plan since 2013-2014	1	-	6	56	162
			2

Teaching language

Suitable for English-speaking students

Objectives

The course will focus on Markov decision processes and some generalizations. Markov decision processes, also referred to as stochastic dynamic programs or stochastic control problems, are models for sequential decision making when outcomes are uncertain. The Markov decision process model consists of decision epochs, states, actions, rewards, and transition probabilities. Choosing an action in a state generates a reward and determines the state at the next decision epoch through a transition probability function. Policies or strategies are prescriptions of which action to choose under any eventuality at every future decision epoch. Decision makers seek policies which are optimal in some sense. An analysis of this model includes

1. providing conditions under which there exist easily implementable optimal policies;
2. determining how to recognize these policies
3. developing and enhancing algorithms for computing them; and
4. establishing convergence of these algorithms.

Learning outcomes and competences

It is intended that the student formulate decision problems in various contexts of human activity, implement them algorithmically and computationally and analyze them from the point of view of coherence, adequacy, convergence and optimality.
Students should acquire autonomy and critical sense in the use of models and resources in the various applications of mathematics.

Working method

Presencial

Pre-requirements (prior knowledge) and co-requirements (common knowledge)

Students are expected to reveal consolidated knowledge in the various areas that are studied in a degree in Mathematics.

Program

1. A General Framework for Markov Decision Process


2. Algorithms


3. Linear Programming Formulations for Markov Decision Processes


4. Semi-Markov Decision Processes


5. Partially Observable and Adaptive Markov Decision Processes


6. Further Aspects of Markov Decision Processes


7. Some Markov Decision Process Problems, Formulations and Optimality Equations

Mandatory literature

Mine Hisashi; Markovian decision processes. ISBN: 0-444-00079-8
Howard Ronald A.; Dynamic programming and Markov processes. ISBN: 0-262-08009-5

Teaching methods and learning activities

Lectures of themes, accompanied by resolution of exercises or problems.

Evaluation Type

Distributed evaluation without final exam

Assessment Components

designation	Weight (%)
Defesa pública de dissertação, de relatório de projeto ou estágio, ou de tese	30,00
Participação presencial	30,00
Teste	40,00
Total:	100,00

Amount of time allocated to each course unit

designation	Time (hours)
Elaboração de projeto	20,00
Estudo autónomo	80,00
Frequência das aulas	52,00
Total:	152,00

Eligibility for exams

The student must attend at least 2/3 of the classes, must present a final project on a subject versed in the course, where it must reach a classification of at least 50% and must obtain the minimum classification of 50% in the final test.

Calculation formula of final grade

The final classification will be obtained according to the following weights:
Assiduity and active participation in classes (30%); presentation of a small project in one of the topics covered in the course (30%); final written test (40%).