networked media - technion › isr-hun › magyar_gabor.pdf · networked, electronic media media...
TRANSCRIPT
![Page 1: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/1.jpg)
Budapest University of Technology and Economics, BME
Networked Media
in the Future nternet
![Page 2: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/2.jpg)
Israeli Hungarian Workshop on Future Internet Research
Motivation
• Future Internet --- Internet of media
• content delivery should rely on service-
centric networks
• New requirements in knowledge
representation and analysis
2
![Page 3: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/3.jpg)
Israeli Hungarian Workshop on Future Internet Research
Media Informatics Outline
Networked electronic media
Information and knowledge management
Human-computerinteraction
Media technologyContent delivery
Content productionweb 2.0 technologies
DatabasesKnowledge representation
Data- and text miningIR & CBIR
Personalized access to media
Natural language interfaces(Voice & visual ) Dialog systems
Cognitive interfacesContext identification
3
![Page 4: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/4.jpg)
Israeli Hungarian Workshop on Future Internet Research
Networked, electronic media
Media technologyContent delivery
Content productionweb 2.0 technologies
Digital audiovisual archiving
Content Management Systems
IPTV apps.
Mobile TV apps.
3Play apps.
Streaming media analysis and miningVideo based trackingVideo based traffic countAudio and video based in-home health care solutions
4
![Page 5: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/5.jpg)
Israeli Hungarian Workshop on Future Internet Research
Information and knowledge management
DatabasesKnowledge representation
Data- and text miningIR & CBIR
Personalized access to mediaText processing
Intelligent text retrieval
Domain specific search engines
Text categorization
Collaborative filtering
- in information filtering
- in recommender systems
- localization based services
- advertisement and content targeting
IndexingKnowledge retrieval by fuzzy
pseudo thesaurus Multi Criteria Decision Analysis
5
![Page 6: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/6.jpg)
Israeli Hungarian Workshop on Future Internet Research
Human-computerinteraction Natural language interfaces
(Voice & visual ) Dialog systemsCognitive interfaces
Context identificationContext identification in language processing
Question to SQL Query translationDeep Web Search EngineAnaphora resolutionGenitive phrase meaning discoveryText to ontology building
Inferring from natural language corpusPersonal rating (‘vox populi’)Context targeting and content recommendationsTerm extractionTextual corpus based decision support systems
Pattern recognition in images- recognition of pedestrians, signals- identification and classification in time series- photo based recognition- signature, face, handwriting identification- impression identification of imagesVoice, tone and feeling identification in audio
6
![Page 7: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/7.jpg)
Israeli Hungarian Workshop on Future Internet Research
Context recognition
7
Partneroldali vékonykliens
Szavak hálójában
Magas szintű modell
Nyelvi
elemzőKontextus-
felismerő
Mély háló
motor
Mediátor
rétegMély háló
adatbázis
Válasz-
elemzés
Kérdés
Formális mondat
CL kifejezés
Általános SQL
Eredmények
Rangsor
SQL
Abstract model
Context
recognizer
Morpho
analyzer
Deep web
engine
Answer
analyzer
Mediator
layerDeep web
DB
![Page 8: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/8.jpg)
Israeli Hungarian Workshop on Future Internet Research
Context recognizer
• Determines the context(s) of queries
• Identifies the focus and the schema of the query
• Determines the role of each informationgranule of the query and assigns them toappropriate attributes of the identifiedschema
• Creates so-called CL (Context Language) expressions that is the basis of SQL queries
8
![Page 9: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/9.jpg)
Israeli Hungarian Workshop on Future Internet Research
Multi Criteria Decision Analysis in engineering areas
• Info-communication networks
• Fault spreading problem
• Routing tasks
Mathematical modelling toolsAnalytic Hierarchy Process (AHP)
Analytic Network Process (ANP)
Bayesian Network
Method combining expert knowledge
and probabilistic theory to analyse fault
spreading problem
9
![Page 10: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/10.jpg)
Israeli Hungarian Workshop on Future Internet Research
• Task: extraction of feature vectors
characteristic to objects
• Referential comparison of different
categorization methods
![Page 11: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/11.jpg)
Israeli Hungarian Workshop on Future Internet Research
Method
• Originally used for text documents
• Hierarchical taxonomy tree
• Each node has a processing unit
• Each processing unit outputs a sigmoid
function of the linear combination of its
inputs
![Page 12: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/12.jpg)
Israeli Hungarian Workshop on Future Internet Research
![Page 13: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/13.jpg)
Israeli Hungarian Workshop on Future Internet Research
Data Mining Competitions
• ACM KDD Cup (classification of internet search query; automatic identification of pulmonary embolisms (PE) from 3D computed tomography data)
– First place in patient classification
– Runner-up in query categorization precision
– Runner-up in PE identification
– Runner-up in creativity
• ACM SIGKDD Challenge on Time Series Classification (general purpose time series classification task)
– 3rd place
• IEEE Conference on Data Mining – 5th in precision on high precision in-door localization
• Netflix Prize Competition– 2nd place
13
KDD Cup is the annual Data Mining and Knowledge Discovery competition organized by ACM, USA
![Page 14: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/14.jpg)
Israeli Hungarian Workshop on Future Internet Research
integrated 3D mouse
Integrated touch-based actuator
haptic vibration feedback
Audio (speach recognition and synthesis 3D input devices
Cognitive interfaces
14
![Page 15: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/15.jpg)
Israeli Hungarian Workshop on Future Internet Research
- RT middleware
- ICE based TRT
middle ware
-Platform independent
- 3D Internet
-Collaboration Arena
VirCA operation system - network
15
![Page 16: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/16.jpg)
Israeli Hungarian Workshop on Future Internet Research
Co-operation networks
• Emerging Information Technology in
Manufacturing (ITM) www.itm-lab.eu– Norwegian University of Science and Technology,
Narvik University College, BME, MTA-SZTAKI,
Budapest Tech (BMF), Széchenyi István University
• Intelligent Integrated Systems
Japanese-Hungarian Joint Laboratory
www.iisl-lab.org– University of Tokyo, BME
16
![Page 17: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/17.jpg)
Israeli Hungarian Workshop on Future Internet Research
7% - verbal communication; 38% - paraverbal communication; 55% - non-verbal signals, such
as gesticulation, posture
3D Motion capture
systemsSpeech recognition
17
![Page 18: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/18.jpg)
Israeli Hungarian Workshop on Future Internet Research
CAD
matching
Norway Hungary
18
![Page 19: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/19.jpg)
Israeli Hungarian Workshop on Future Internet Research
Two approaches:
Cell phone actuator in CAVE
CAVE-enhanced cell phone
Cell phone equipped with:
Motion sensors (integrated 3D mouse)
Touch sensors (integrated touch-based actuator)
Vibrating components (haptic feedback)
<===>
Data glove for position
capture and haptic
feedback
Wireless input device
Mobile 3D Internet
19
![Page 20: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/20.jpg)
Israeli Hungarian Workshop on Future Internet Research
Foundation of
Intelligent Space Network
• Dept. of Electrical and Computer Engineering North Carolina State
University
• Computer and Automation Research Institute of Hungarian Academy of
Sciences
• Eötvös University
• Budapest University of Technology and Economics
• National Institute of Advanced Industrial Science and Technology
• Hashimoto Laboratory, Institute of Industrial Science University of Tokyo
• Department of Electrical and Electronic Engineering Chuo University
• College of Information Science and Engineering Ritsumeikan University
• Department of Electrical Engineering Meiji University
• Department of Industrial Engineering, Narvik University College
• Department of Production and Quality Engineering Norwegian University
of Science and Technology
20
![Page 21: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/21.jpg)
Israeli Hungarian Workshop on Future Internet Research
Speech activities
• Multi-lingual artificial speech generation
• Speech recognition
• Speech databases
21
![Page 22: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/22.jpg)
Israeli Hungarian Workshop on Future Internet Research
• Multi-lingual artificial speech generation– limited vocabulary (e.g., numbers, date, address)
– multi-lingual TTS (Hungarian, German, Polish, Spanish)
– speech profiles (prosody, variability, individual features)
– expression/emotion presentation (user’s manual <-> news)
• Speech recognition– noise handling (telephone, in-car, …)
– dictation (good quality medical transcription, continuous)
– audio indexing (e.g. radio archives, broadcast news)
– speech segmentation
– emotion detection
– keyword spotting
• Speech databases
• Health, rehabilitation, e.g. Audio-visual speech-pronunciation teaching system
• Acoustic company image, eto-informatics
22
![Page 23: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/23.jpg)
Israeli Hungarian Workshop on Future Internet Research
ETO-comm
• Adaptive artificial character („personality”)
in man-machine communications
– adaptive to the mood, style, activity of the
human being
– Personal gadgets personal partners
• Analogy: the complex communication of
a dog and a human being
23
![Page 24: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/24.jpg)
Israeli Hungarian Workshop on Future Internet Research 24
![Page 25: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/25.jpg)
Israeli Hungarian Workshop on Future Internet Research
ETO-comm
• Adaptive artificial character („personality”)
in man-machine communications
– adaptive to the mood, style, activity of the
human being
– Personal gadgets personal partners
• Analogy: the complex communication of
a dog and a human being
• E.g. smartphone, robot (sound, vibration,
light, style of messages)
25
![Page 26: Networked Media - Technion › ISR-HUN › Magyar_Gabor.pdf · Networked, electronic media Media technology Content delivery Content production web 2.0 technologies Digital audiovisual](https://reader033.vdocument.in/reader033/viewer/2022052611/5f0496f57e708231d40eb893/html5/thumbnails/26.jpg)
Israeli Hungarian Workshop on Future Internet Research
Networked Media in TMIT
Media/Content Delivery PlatformsIPTV, mobile internet
User Centric Mediainformation-retrieval, community content,personalisation, integration of UGC and professional content,
Multimedia Searchtagged media, multimedia content analysis,automated annotation
3D Media3D internet, feeling of physical presence
26