Parallel and Distributed Computing

Code:

L.EIC028

Acronym:

CPD

Keywords
Classification	Keyword
OFICIAL	Informatics Engineering and Computing

Instance: 2023/2024 - 2S

Active?	Yes
Responsible unit:	Department of Informatics Engineering
Course/CS Responsible:	Bachelor in Informatics and Computing Engineering

Cycles of Study/Courses

Acronym	No. of Students	Study Plan	Curricular Years	Credits UCN	Credits ECTS	Contact hours	Total Time
L.EIC	324	Syllabus	3	-	6	52	162

Teaching language

Suitable for English-speaking students

Objectives

Endow the students with:

• knowledge to develop parallel programs;


• solid foundations on algorithms parallelization, programming models, synchronization of processes, and performance measures by the development of programs;


• theoretical knowledge on distributed systems so they can make correct decisions when faced with the need to design/develop/administer such a system;


• programming experience so that they can develop distributed applications.

Learning outcomes and competences

Upon completing this course, the students should be able to:

• Analyze a problem and identify the adequate parallelization model;
• Write shared memory programs;
• Design parallel solutions for new problems;
• Use computational models to estimate applications computation time;
• Know how to share resources among concurrent processes.;
• Explain the implications of communication latency, partial failures, and asynchrony in a distributed application;
• Implement a distributed application using both thread-based and event-driven concurrency;
• Identify problems in which basic synchronization algorithms such as elections can be used, and explain why small changes may break them;
• Explain the workings of basic algorithms for fault tolerance and the guarantees they provide;
• Understand the purpose and challenges of data/service replication in distributed applications;
• Assess the trade-offs among performance, fault-tolerance, and consistency in replicated services.

Working method

Presencial

Pre-requirements (prior knowledge) and co-requirements (common knowledge)

Operating Systems

Computer Networks

Program

Introduction to parallel computing. Performance metrics.

Parallel architectures. Memory organization and the effect of cache management on processor performance.

Amdahl Law.

Ways of extracting parallelism: functional, data parallelism, and streaming. Methodologies for developing parallel programs.

Parallel programming models: shared memory and distributed model. Concurrency.

Multi-core programming with OpenMP. Computational models and performance measures.

Introduction to distributed systems and network computing. 

Communication paradigms: message passing, remote procedure call, and multicasting. 

Processing in distributed systems: state-full vs. stateless protocols, event-based and thread-based concurrency. 

Fault-tolerance: algorithms for leader election, two-phase commit, and primary-backup replication.

Basic techniques for improving scalability. Case study: Domain Name System (DNS)

Mandatory literature

M. van Steen and A. S. Tanenbaum; Distributed systems, 3rd Ed., 2017 (https://www.distributed-systems.net/index.php/books/ds3/))
Michael J. Quinn; Parallel programming in C with MPI and openMP. ISBN: 007-123265-6

Complementary Bibliography

George Coulouris; Distributed systems. ISBN: 0-201-61918-0
Andrew S. Tanenbaum; Distributed systems. ISBN: 0-13-613553-6
Ruud van der Pas, Eric Stotzer, Christian Terboven; Using OpenMP—The Next Step: Affinity, Accelerators, Tasking, and SIMD, MIT Press, 2017. ISBN: 9780262534789

Teaching methods and learning activities

All topics are covered in the lectures, which are mostly expository and usually use transparencies. To motivate the students, we use real-world examples. The course contents follow closely the mandatory bibliography. Whenever necessary, class notes and/or articles will be provided.

The students have to solve both paper-and-pencil problems and small programming problems, which can be completed in a few hours. In addition, they have to do two small projects.  Help for solving these problems and the project is given in the lab sessions.

Software

git
JDK
Go

Evaluation Type

Distributed evaluation with final exam

Assessment Components

Designation	Weight (%)
Exame	55,00
Trabalho prático ou de projeto	45,00
Total:	100,00

Amount of time allocated to each course unit

Designation	Time (hours)
Estudo autónomo	60,00
Frequência das aulas	52,00
Trabalho laboratorial	50,00
Total:	162,00

Eligibility for exams

For admission to the final exam the students must:
1) Attend at least 75% of the planned lab classes
2) Get a minimal score of 8 in each of the projects
3) Get a weighted average of 10 in the projects
4) Present/demo the projects
5) Fill the sel-evaluation/peer-evaluation forms of each project

Calculation formula of final grade

min( 0.45Pr + 0.55FE, 1.2FE)
where
Pr- projects weighted average grade
FE- final exam grade
where:
Pr = 0.4*PPP+0.6*DPP
where:
PPP- Parallel programming project grade
DPP- Distributed programming project grade

Projects are developed in groups, but the grades of the members of a given group may be different depending on their contribution.

Special assessment (TE, DA, ...)

Similar to the ordinary students.

Special evaluation students must contact the responsible of the course to schedule the checkpoints for the projects.

Classification improvement

The exam grade can be improved by submitting to another exam, to be scheduled by the Department.

The projects' grades can be improved only in the following instance of the course.

Observations

If there are registered students who do not speak Portuguese, the lectures may be taught in English.