Parallel Computing

Code:

EIC0089

Acronym:

CPAR

Keywords
Classification	Keyword
OFICIAL	Programming

Instance: 2014/2015 - 2S

Active?	Yes
Responsible unit:	Department of Informatics Engineering
Course/CS Responsible:	Master in Informatics and Computing Engineering

Cycles of Study/Courses

Acronym	No. of Students	Study Plan	Curricular Years	Credits UCN	Credits ECTS	Contact hours	Total Time
MIEIC	25	Syllabus since 2009/2010	4	-	6	56	162

Teaching language

Suitable for English-speaking students

Objectives

BACKGROUND

Parallel and distributed computing is becoming the computing paradigm as hardware tends to multi-processing units. The common desktop is today built with multicore processors that collectively have more processing power, than the single core processor, but cores are individually less powerful. Programmers will have to deal with multiprocessor architectures in order to use effectively the machines of today and of the future.

AIMS

Acquisition of useful knowledge to develop parallel programs. Construction of solid basis in parallel architectures, algorithms parallelization, programming models, synchronization of processes and performance measures by the development of programs.

Scientific component:50%

Technological component:50%

 

Learning outcomes and competences

Students should be able to: a) Analyze a problem and identify the adequate parallelization model (Knowledge and Understanding) b) Write message-passing and shared memory programs (Engineering Analysis, Engineering Practice) c) Design parallel solutions for new problems (Engineering Design) d) Use computational models to estimate applications computation time (Investigations) e) knowledge of process concurrency and best practices to implement resource sharing (Transferable skills).

Working method

Presencial

Pre-requirements (prior knowledge) and co-requirements (common knowledge)

Students should had completed successfully the programming subjects of the 1st and 2nd year.

Program

INTRODUCTION: - Introduction to Parallel Computing. Performance metrics: MIPS, FLOPS. Peak, Max and Sustained performance. Parallel machines: superscalar and vector processors, memory and network organization. Cache memory effect on processor performance - Data Locality.

PARALLEL PROGRAMMING FUNDAMENTALS: Amdahl Law. Ways of extracting parallelism: Functional Parallelism, Data Parallelism, Streaming. Steps to obtain a parallel version of an algorithm: Problem Division, Communication Patterns, Synchronization, Granularity of Parallelization, Staggering (distribution of work by the processors).

PARALLEL PROGRAMMING MODELS: Shared Memory model and Distributed Memory model. Race Condition, Critical sections, False sharing, Reduction operation.

MULTI-PROCESSOR AND MULTI-COMPUTER PROGRAMMING: - Message passing programming with MPI - Shared memory programming with OpenMP - Data Parallel programming with CUDA (GPUs).

CHARACTERIZATION OF PARALLEL COMPUTING: Execution Models, Computing Models, Efficiency and performance Measures, scalability (Isoefficiency Function).

INTRODUCTION TO DISTRIBUTED COMPUTING IN INTERNET ENVIRONMENT: P2P, GRID, CLOUD COMPUTING. Application characteristics.

Mandatory literature

Quinn, Michael J.; Parallel programming in C with MPI and openMP. ISBN: 007-123265-6
Calvin Lin, Lawrence Snyder; Principles of parallel programming, Pearson - Addison Wesley, 2009. ISBN: 0-321-48790-7
David B. Kirk, Wen-mei W. Hwu; Programming massively parallel processors, Morgan Kaufman, 2010. ISBN: 978-0-12-381472-2

Complementary Bibliography

Foster, Ian T.; Designing and building parallel programs. ISBN: 0-201-57594-9

Teaching methods and learning activities

Theoretical classes: exposition of the subject matter with presentation and discussion of examples. Theoretical-practical classes: problem solving and discussion, including the development of some programs.

Software

MPI
Visual Studio 2008 Professional (C#/C/C++)
g++

keywords

Technological sciences > Technology > Computer technology > Software technology

Evaluation Type

Distributed evaluation with final exam

Assessment Components

Designation	Weight (%)
Exame	50,00
Participação presencial	0,00
Trabalho laboratorial	50,00
Total:	100,00

Amount of time allocated to each course unit

Designation	Time (hours)
Estudo autónomo	60,00
Frequência das aulas	42,00
Trabalho laboratorial	60,00
Total:	162,00

Eligibility for exams

Not exceed the maximum number of absences to classes (25%) and deliver the course work.

Calculation formula of final grade

Final Grade= 0.5*Cont + 0.5*Ex

Cont – Programming assignments and class participation

Ex – Exam grade

Special assessment (TE, DA, ...)

The programming assignments are developed during the classes but are optinal for special registration students.

Classification improvement

The programming assignments can only be improved in the course next instance.

Observations

Experience in C/C++ programming is required.