Parallel Computing

Code:

PRODEI027

Acronym:

CP

Keywords
Classification	Keyword
OFICIAL	Science and Technology Programming

Instance: 2013/2014 - 2S (of 10-02-2014 to 06-06-2014)

Active?	Yes
E-learning page:	https://moodle.fe.up.pt/
Responsible unit:	Department of Informatics Engineering
Course/CS Responsible:	Doctoral Program in Informatics Engineering

Cycles of Study/Courses

Acronym	No. of Students	Study Plan	Curricular Years	Credits UCN	Credits ECTS	Contact hours	Total Time
PRODEI	4	Syllabus	1	-	6	54	162

Teaching language

English

Objectives

BACKGROUND

Parallel and distributed computing is becoming the computing paradigm as hardware tends to multi-processing units. The common desktop is today built with multicore processors that collectively have more processing power, than the single core processor, but cores are individually less powerful. Programmers will have to deal with multiprocessor architectures in order to use effectively the machines of today and of the future.

AIMS

Acquisition of useful knowledge to develop parallel programs. Construction of solid basis in parallel architectures, algorithms parallelization, programming models, synchronization of processes and performance measures by the development of programs.

Scientific component:50%

Technological component:50%

Learning outcomes and competences

Students should be able to: a) Analyze a problem and identify the adequate parallelization model (Knowledge and Understanding) b) Write message-passing and shared memory programs (Engineering Analysis, Engineering Practice) c) Design parallel solutions for new problems (Engineering Design) d) Use computational models to estimate applications computation time (Investigations) e) knowledge of process concurrency and best practices to implement resource sharing (Transferable skills).

Working method

Presencial

Pre-requirements (prior knowledge) and co-requirements (common knowledge)

Students should had completed successfully the programming subjects of the 1st and 2nd year.

Program

INTRODUCTION: - Introduction to Parallel Computing. Performance metrics: MIPS, FLOPS. Peak, Max and Sustained performance. Parallel machines: superscalar and vector processors, memory and network organization. Cache memory effect on processor performance - Data Locality.

PARALLEL PROGRAMMING FUNDAMENTALS: Amdahl Law. Ways of extracting parallelism: Functional Parallelism, Data Parallelism, Streaming. Steps to obtain a parallel version of an algorithm: Problem Division, Communication Patterns, Synchronization, Granularity of Parallelization, Staggering (distribution of work by the processors).

PARALLEL PROGRAMMING MODELS: Shared Memory model and Distributed Memory model. Race Condition, Critical sections, False sharing, Reduction operation.

MULTI-PROCESSOR AND MULTI-COMPUTER PROGRAMMING: - Message passing programming with MPI - Shared memory programming with OpenMP - Data Parallel programming with CUDA (GPUs).

CHARACTERIZATION OF PARALLEL COMPUTING: Execution Models, Computing Models, Efficiency and performance Measures, scalability (Isoefficiency Function).

INTRODUCTION TO DISTRIBUTED COMPUTING IN INTERNET ENVIRONMENT: P2P, GRID, CLOUD COMPUTING. Application characteristics.

Mandatory literature

Quinn, Michael J.; Parallel programming in C with MPI and openMP. ISBN: 007-123265-6
Calvin Lin, Lawrence Snyder; Principles of Parallel Programming , Pearson, Addison Wesley, 2009. ISBN: 0-321-48790-7
David B. Kirk, Wen-mei W. Hwu; Programming Massively Parallel Processors, Morgan Kaufmann, 2010. ISBN: 978-0-12-381472-2

Complementary Bibliography

Foster, Ian T.; Designing and building parallel programs. ISBN: 0-201-57594-9

Teaching methods and learning activities

Theoretical classes: exposition of the subject matter with presentation and discussion of examples. Theoretical-practical classes: problem solving and discussion, including the development of some programs.

Software

Visual Studio 2008 Professional (C#/C/C++)
MPI

keywords

Technological sciences > Technology > Computer technology > Software technology

Evaluation Type

Distributed evaluation without final exam

Assessment Components

Designation	Weight (%)
Participação presencial	10,00
Trabalho escrito	60,00
Trabalho laboratorial	30,00
Total:	100,00

Amount of time allocated to each course unit

Designation	Time (hours)
Elaboração de projeto	60,00
Estudo autónomo	40,00
Frequência das aulas	42,00
Trabalho laboratorial	20,00
Total:	162,00

Eligibility for exams

Concluding and presenting the programming assignments.

Calculation formula of final grade

Final Grade= 0.1*P + 0.3*CW + 0.6(0.5*R1 + 0.5*R2)

P - Attendance

CW - class work 

R1 - Report 1

R2 - Report 2

Special assessment (TE, DA, ...)

The programming assignments are developed during the classes but are optinal for special registration students.

Classification improvement

The programming assignments can only be improved in the course next instance.

Observations

Fluency in C/C++ programming is required.