Natural Language Processing

Code:

M.EIC022

Acronym:

PLN

Keywords
Classification	Keyword
OFICIAL	Artificial Intelligence

Instance: 2024/2025 - 2S

Active?	Yes
Responsible unit:	Department of Informatics Engineering
Course/CS Responsible:	Master in Informatics and Computing Engineering

Cycles of Study/Courses

Acronym	No. of Students	Study Plan	Curricular Years	Credits UCN	Credits ECTS	Contact hours	Total Time
M.EIC	33	Syllabus	1	-	6	39	162

Teaching language

English

Objectives

This course provides an introduction to the field of Natural Language Processing (NLP). By the end of the course, students should have acquired a comprehensive understanding of the field and its state-of-the-art, and recent research trends.

Learning outcomes and competences

Learning goals include:

1. Acquire the fundamental linguistic concepts that are relevant to processing natural language text.
2. Understand both basic and state-of-the-art algorithms and techniques for dealing with natural language text.
3. Familiarize with state-of-the-art NLP tools and linguistic resources.
4. Understand and employ evaluation metrics for different NLP tasks.
5. Be able to address an NLP problem and solve it with the appropriate techniques, algorithms, and tools.
6. Read and understand current research on natural language processing.

Working method

Presencial

Pre-requirements (prior knowledge) and co-requirements (common knowledge)

Knowledge of Python programming.
Basic knowledge of machine learning techniques.

Program

- Introduction to NLP: definitions, tasks, and applications
- Basic text processing: regular expressions, tokenization, normalization, lemmatization, stemming
- Language models: n-grams, perplexity, text generation
- Text classification: bag-of-words, n-grams, feature engineering; generative and discriminative classifiers
- Vectorized representations of words: lexical semantics, word embeddings
- Sequence models: POS-tagging, named entity recognition; hidden Markov models, conditional random fields
- Neural networks in NLP: neural language models, RNNs, encoder-decoder networks, attention
- Transformers: self-attention, positional encodings, Transformer-based architectures, pre-training, fine-tuning
- Machine Translation and Question Answering: data, sequence-to-sequence models, evaluation metrics
- Large Language Models: neural language models, decoding strategies, scaling laws, alignment, prompt engineering

Mandatory literature

Jurafsky, D. & Martin, J.H.; Speech and Language Processing: An Introduction to Natural Language Processing, 2024 (https://web.stanford.edu/~jurafsky/slp3/ )

Complementary Bibliography

Lewis Tunstall, Leandro von Werra, Thomas Wolf; Natural Language Processing with Transformers: Building Language Applications With Hugging Face, O'Reilly Media, 2022. ISBN: 1098136799 (https://transformersbook.com/)
Jacob Eisenstein; Introduction to natural language processing. ISBN: 978-0-262-04284-0
Steven Bird, Ewan Klein, and Edward Loper; Natural Language Processing with Python, O'Reilly, 2009. ISBN: 0596516495 (https://www.nltk.org/book/)

Teaching methods and learning activities

Course topics will be covered with motivating applications, and with source code examples, where applicable. The aim is to introduce the tools that are to be used in practical assignments as soon as possible. At the same time, pointers to related literature will be given as further reading opportunities.

Evaluation Type

Distributed evaluation with final exam

Assessment Components

Designation	Weight (%)
Exame	40,00
Trabalho laboratorial	60,00
Total:	100,00

Amount of time allocated to each course unit

Designation	Time (hours)
Estudo autónomo	50,00
Frequência das aulas	42,00
Elaboração de projeto	70,00
Total:	162,00

Eligibility for exams

An enrolled student obtains frequency if he/she attends theoretical-practical classes (maximum absences allowed corresponds to 25% of the scheduled classes) and submits the expected practical work.

Calculation formula of final grade

Evaluation will be composed of:
- 2 practical assignments (2x6/20)
- 1 final exam (8/20)

To obtain approval, the following minimum ratings apply:
- Assignment 1: 10.0 points out of 20
- Assignment 2: 10.0 points out of 20
- Exam: 7.0 points out of 20

Examinations or Special Assignments

Evaluation in special seasons consists of one practical Assignment and a written Exam, where each of these components weighs 50% on the final grade. Approval in the course requires a minimum score of 50% in the practical assignment, and a minimum of 35% in the written exam.

Special assessment (TE, DA, ...)

All assessment components are required for all students. Students enrolled using special frequency modes, without obligation to attend the classes, must arrange with teachers appropriate consultation and evaluation sessions.

Classification improvement

The improvement of classification in the distributed component (assignments and oral presentation) can only be obtained in the next edition of the course.