Multimedia Systems

Code:

M.EEC057

Acronym:

SMUL

Keywords
Classification	Keyword
OFICIAL	Other Technical Areas

Instance: 2021/2022 - 1S

Active?	Yes
Responsible unit:	Department of Electrical and Computer Engineering
Course/CS Responsible:	Master in Electrical and Computer Engineering

Cycles of Study/Courses

Acronym	No. of Students	Study Plan	Curricular Years	Credits UCN	Credits ECTS	Contact hours	Total Time
M.EEC	20	Syllabus	2	-	6	39

Teaching language

Portuguese
Obs.: Suitable for English speaking students

Objectives

Provide students with skills to apply technologies and methodologies in the development of multimedia systems. Provide the basic elements for the design and development of multimedia applications for the Internet of the Future. Focus will be given to the audio component, addressing aspects of musical content analysis with a view to automatic identification and access to musical content.

With this objective in mind, it is intended that the UC enhances the development of the following professional skills and attitudes:

a) autonomy and initiative in the acquisition and integration of knowledge in the area of multimedia technologies;
b) management and documentation of multimedia projects;
c) Formal design of multimedia systems and applications, taking into account the needs of the target audience and introducing innovation using emerging technologies.

Learning outcomes and competences

At the end of the course, students will have acquired the following knowledge and skills:

a) knowledge:

- most current technologies for the distribution of media signals

- functioning of the human audiovisual system, its capabilities, and limitations in the perception of media signals (color, brightness, tonality, pitch, sonority, timbre)

- fundamental properties of media signals

- acoustic and psycho-acoustic characteristics of different types of sounds, through temporal and spectral analysis;

- Appropriate techniques for analyzing and retrieving musical information;

- requirements for multimedia applications and services

- tools for the design and development of multimedia services

- advanced human-machine interaction aspects

B) skills:

- use tools for analysis of media signals and interpretation of extracted characteristics

- explain the use of technologies and standards in the context of multimedia systems.

select tools and standards according to multimedia service requirements
use tools for designing and developing multimedia systems
design and test multimedia access and communication systems

Working method

Presencial

Pre-requirements (prior knowledge) and co-requirements (common knowledge)

Basic knowledge in digital signal processing and programming (Matlab/Python)

Program

Mechanisms of the human audiovisual system and characteristics of media signals

Introduction to Music Content Analysis: What is it; the different modes and representations, applications, research topics and tasks.

Sound and Music Analysis: DFT (Discrete Fourier Transform), FFT (Fast-Fourier Transform) and STFT (Short-Time Fourier Transform) in audio signal processing.

Description of musical content in terms of abstraction levels, temporal contexts and musical aspects; Computation of low-level descriptors.

MIR: Tone Description; Clustering and classification.

MIR: Pitch determination; Extraction of chroma, chords and tonality.

MIR: Novelty Detection; rhythm analysis and beat and downbeat detection.

User-centric design: user properties and context. multimedia interfaces

Conceptual modeling of multimedia applications;

Principles of prototyping and usability;

Formal specification in UML (use cases, collaboration diagrams, class diagrams);

Standards and protocols for creating multimedia systems (MPEG-2 TS, MPEG-M, DASH, HLS, DVB, IPTV, relevant TCP/IP protocol stack protocols);

Advanced human-machine interaction using computer vision and audio systems, such as voice recognition; face recognition; recognition of emotions; voice-text-to-speech conversion; “eye-tracking”; sensors and actuators.

Assessment in Music Content Analysis (MIR).

Scientific challenges in music content analysis (MIR).

Mandatory literature

François Fluckiger; Understanding networked multimedia. ISBN: 0-13-190992-4
Meinard Muller; Fundamentals of music processing. ISBN: 978-3-319-21945-5

Complementary Bibliography

Ed. Fernando Pereira; Comunicações Audiovisuais: Tecnologias, normas e aplicações,, IST Press
Curtis Roads; The computer music tutorial. ISBN: 978-0-262-68082-0

Teaching methods and learning activities

This curricular unit is based on theoretical and practical classes. It is intended that this course has a strong component of laboratory practice. Consequently, it is considered appropriate to adopt mostly the project-based teaching/learning method, supported by theoretical classes. These aim to make known the fundamental concepts and relevant technologies that provide the necessary bases for the conception and development of these projects. Tools for planning and developing applications will also be presented.


Theoretical classes consist of a theoretical presentation of topics in the area, using multimedia examples whenever appropriate.

Practical classes consist of a brief presentation of specific problems, followed by a practical activity to solve the problems exposed. Students must develop their work during practical classes, and participate in subsequent discussions. Three of these works will be carried out during the semester.

Additionally, it will be proposed to students (organized in groups of two or three students) the development of a multimedia project to be developed outside the class period. The development methodology of these projects is organized into three iterative phases: conception, design, and implementation, previously scheduled, encouraging the realization of validation and usability tests in the different phases.

Software

Jupyter Notebooks (Python)
Sonic Visualiser

Evaluation Type

Distributed evaluation without final exam

Assessment Components

Designation	Weight (%)
Participação presencial	10,00
Trabalho laboratorial	55,00
Trabalho prático ou de projeto	35,00
Total:	100,00

Amount of time allocated to each course unit

Designation	Time (hours)
Elaboração de projeto	39,00
Estudo autónomo	39,00
Frequência das aulas	42,00
Trabalho laboratorial	42,00
Total:	162,00

Eligibility for exams

Perform all work obtaining at least an overall average rating of 9.5.

Calculation formula of final grade

0,1 * engagement_in_classes + 0,55 * average_classification_laboratory_work + 0,35 * project_classification

Examinations or Special Assignments

N/A

Internship work/project

N/A

Special assessment (TE, DA, ...)

Student workers, despite being excused from classes, must carry out the same practical work as other students.

Classification improvement

Improvement of a maximum of two lab jobs. Students must submit the final report(s) no later than one week before the end of the exam period.

Observations

link Colibri/Zoom for theoretical class, friday 15h00 to 16h00:

https://videoconf-colibri.zoom.us/multsys