1

Graduate Institute of Networking and Multimedia

College of Electrical Engineering and Computer Science

National Taiwan University

http://www.inm.ntu.edu.tw

2007

2

Ⅰ. Brief Introduction• Communications and Multimedia Lab. (CMLab.) was founded in Feb.1991 at D

epartment of Computer Science and Information Engineering (CSIE) National

Taiwan University – the first and the largest academic Lab., which focused on,

Multimedia and Networking researches in Taiwan. (Currently, CMLab. has 7 fa

culty members, 3 post Ph.D.s, 80 M.S./Ph.D. students, and 3 assistants.)

• CSIE department transferred from College of Engineering to College of Electric

al Engineering and Computer Science (EECS) in Aug. 2000.(CSIE is the first c

hoice of high-school students who attended the Joint Collegiate Entrance Exa

mination in the category of Computer Science and Engineering)

• Ministry of Education approved GINM to offer M.S. and Ph.D. programs startin

g at August 2004.

3

EECS College

EE Division CS Division

EE

Departm

ent & G

raduate Institute

Graduate Institute of P

hotonics and O

ptoelectronics. (GIP

O)

Graduate Institute of

Com

munication E

ngineering. (G

ICE

)

Graduate Institute of E

lectronics E

ngineering. (GIE

E)

CS

IE D

epartment &

Graduate

Institute

Graduate Institute of N

etworking

and Multim

edia. (GIN

M)

… …

One Undergraduate Department and Multiple Graduate Institutes Structure

Graduate Institute of B

iomedical

Electronics and B

ioinformatics.

4

Ⅱ.Missions

EducationNurturing first-class information technology professionals, specifically, in the area of Networking and Multimedia.

Research

Multiple-discipline researches among the faculties within the NTU or with other universities (Taiwan and other Nations) are encouraged to foster innovative, high-quality and advanced research.

University-Industry

Collaboration

The curriculum has been designed to minimize the gap between the industry and academia. Specifically, academic-industrial collective research projects are strongly encouraged.

GINM encourages the interaction with industries and the technology transfer to industries in hopes of advancing the research capability of industries.

5

Ⅲ.Current StatusA. FacultyFull-time: 44 (joint with GINM(9), CSIE(21), BEBI(2), EE(3), GICE(6), GIEE(2),

Depart IM(1))

Adjunct: 3

Ph.D. Degree: 47

TEL: +886-2-33664899

FAX: +886-2-33664898

http://www.inm.ntu.edu.tw

Director

Yi-Ping Hung ProfessorPh.D., Brown University, U.S.A.E-mail: hung@csie.ntu.edu.twAddress: No.1, Sec 4, Roosevelt Rd., Taipei, Taiwan 106Homepage: http://homepage.iis.sinica.edu.tw/~hung

6

GINM comprises 44 full-time faculties (including 21 faculties from Department of CSIE, 2 faculties from Graduate Institute of Biomedical Electronics and Bioinformatics, 3 faculties from Graduate Institute of Electrical Engineering, 6 faculties from Graduate Institute of Communication Engineering ,2 faculties from Graduate Institute of Electronics Engineering and 1 faculty from the Department of Information Management), 9 of them are dedicated to GINM. All the GINM faculties hold Ph.D. degrees. Full-Time Dedicated Faculty members:

Professors

Ja-Ling Wu Ph.D., Ta-Tung Institute of Technology

Ming Ouhyoung Ph.D., University of North Carolina, Chapel Hill

Yi-Ping Hung Ph.D., Brown University

Associate Professor

Ai-Chun Pang Ph.D., National Chiao-Tung University

Assistant Professors

Chi-Sheng Shih Ph.D., University of Illinois at Urbana-Champaign

Chih-Wen Hsueh Ph.D., University of California, Irvine.

Shou-De Lin Ph.D., University of Southern California

Ming-Sui Lee Ph.D., University of Southern California

Winston H. Hsu Ph.D., University of ColumbiaThree adjunct faculties have joined GINM since August 2005.

7

B. Facilities

The GINM maintains an array of powerful computers and software for instruction, as well as research laboratories maintained by individual faculty. The facilities are available to all students and faculties. They include SGI workstations, laptop computers, desktop computers, network traffic analyzers, V8 and Hi8 video recorders, digital cameras, digital video camcorders, scanners, 3D scanners, VHS/LD/DVD video player/recorders, video editors, audio mixers, VCD/MP3 testing platforms, and Sony PSII. In the coming years, GINM will continue expanding its facilities to serve its educational and research mission. The faculties in GINM lead five laboratories: Communications and Multimedia Laboratory, Embedded Systems and Wireless Networking Laboratory, Image and Vision Laboratory, Machine Discovery and Social Networking Mining Lab, and Multimedia Image and Graphics Lab.

8

C. Programs Requirements and Courses GINM offers graduate programs leading to the degrees of Master of Science (MS) and the Doctor of Philosophy (PhD). Students must complete course work or proficiency courses. All the courses in the GINM are divided into three categories: multi-media technology category, networking technology category, and systems and application category.

Master degreeThe MS requires 24 units of course work (one unit equals one semester hour) excluding seminar, individual study, and thesis.

Ph.D. degree

The PhD program requires 30 of semester hour units and the completion of a doctoral thesis under the direction of an approved thesis advisor and with the guidance of a doctoral committee. Students must complete at least 18 units of course work and the remainder are thesis research. PhD candidates must also satisfy a core course requirement, pass the qualifying and preliminary examinations and must successfully defend the doctoral thesis at the final examination. Candidates who do not complete an Master of Science in computer science as part of their graduate program or enter the GINM doctoral program without an approved MS in computer science or related area, are required to complete 12 additional units (42 units in total).

9

Course FlowBASIC KERNEL ADVANCE SEMINAR

Introduction To Digital Signal Processing

Computer GraphicsVirtual Reality Geometric

Modeling, Image-based Modeling and Rendering, Digital Image Synthesis, …

Patten Analysis and Classification

Digital Image Processing

Computer Vision

Digital Communications and Computing

Digital Visual Effects

Information Theory and Coding Techniques

Video Signal Processing

Digital Speech Processing

Digital Audio Processing

Computer NetworksAdvanced Computer

Networks

Channel Coding

Wireless Multimedia System Research, Network Data Security, Performance Modeling, ….

Complexity-based Cryptography

Network Data Security

Cryptography and Network Security

Personal Communications Services

Internet Telephony

Network System Management

Operating SystemsAdvanced Operating System

Compilers for Embedded SystemsHardware Software Codesign, Multiagent Systems,….

Networked SoC Embedded Software DesignReal-times Systems

Database Systems Information Retrieval and Extraction

Data Mining and Machine Learning Net arts, Bioinformatics Algorithms,….

Multim

ediaN

etworking

Sy

ste

ms

& A

pp

lica

tion

s

10

D. Students

Year M.S. Program Ph.D. Program Industry-specific

MS Program (joint with Depart. CSIE

2004 22 9 8

2005 33 10 6

2006 37 9 15

2007 39 13 -

11

• Prof. Ja-Ling Wu: Best Technical Full Paper Award, ACM Multimedia Conference, 2006.

• Prof. Su-Chu Hsu: The Second and Third Winners, 1st Digital Art Festival Taipei, 2006.

• Prof. Ja-Ling Wu: Excellent Award, Longterm Smile Competition, 2006.

• Prof. Ai-Chun Pang: Investigation Award, Pan Wen Yuan Foundation, 2006.

• Prof. Su-Chu Hsu: Technology Creating Award, 1st K.T. Originality Competition, 2006.

• Prof. Ai-Chun Pang: Excellent Teaching Award, National Taiwan University, 2005.

• Prof. Chi-Sheng Shih: Best Paper Award, IEEE Inte’l conf. on Embedded and Real Time

Systems and Applications, 2005.

• Prof. Yi-Ping Hung: Best Paper Award, CVGIP-Academic Society Taiwan, 2005.

• Prof.Ja-Ling Wu and Prof. Ming Ouhyoung: Fu Ssu-nien Award (for contributions to Indust

ry-University Cooperative Research), National Taiwan University, 2005.

• Prof. Ja-Ling Wu: Outstanding Faculty Award of Electrical Engineering, 2005.

• Prof. Chih-Wen Hsueh: Best Paper Award, IEEE Inte’l conf. on Embedded and Real Time

Systems and Applications, 2005.

E. Awards

12

Ⅳ.Research and ServicesA. Research 1. Research Areas

13

2. Related Research Labs.

Communications and Multimedia Lab.

(Profs. Ja-Ling Wu & Ming Ouhyoung & Winston H.Hsu)

Image and Vision Lab.

(Profs. Yi-Ping Hung)

Embedded Systems and Wireless Networking Lab.

(Profs. Ai-Chun Pang & Chi-Sheng Shih & Chih-Wen Hsueh)

Machine Discovery and Social Network Mining Lab.

(Prof. Shou-De Lin)

Multimedia Image and Graphics Lab.

(Prof. Ming-Sui Lee)

14

3. Large Scale ProjectsProgram for Promoting Academic Excellence of University (Phase )-Content Science for a Media-Rich Life.Ⅱ 卓越延續計畫 - 多媒體生活環境的數位內容科學NTU Center for Information and Electronics Technologies 資電中心（優勢重點領域拔尖計畫）The National Science & Technology Program for Telecommunications (NTP) 電信國家型計畫 - 數位家庭 : 網路 , 平台 , 與應用 Intimate Media Agent: Enabling Affective Content Experience from Media Context. 產學合作計畫 - 感人媒體精靈：結合媒體內涵創造感動人心之使用經驗

15

4. Selected Research Results:Image and Vision Lab (Prof. Yi-Ping Hung)

Multi-Resolution Displays Virtual Exhibition

Visual Surveillance Computer-Human Interaction

16

Nth Dwell (N+1)th Dwell

Embedded Systems and Wireless Networking Lab(Prof. Chi-Sheng Shih)

Digita

l Hom

e

Assisted Living

HW

/SW C

o-designMul

ti-r

esou

rce

Con

stra

int

sche

dulin

g

Security Gateway

Energy Efficiency

Real-Time Operating Systems

Embedded Systems

Performance and QoS

HW

(Platform)

17

Embedded Systems and Wireless Networking Lab

(Prof. Ai-Chun Pang)

18

SoCOS Architecture

HAL

SoCKernel

Tools

UserSpace

KernelSpace

KernelServices

Middlewares LibrariesDevice Drivers

UserTasks

microkernel

Ubiquitous Message Passing

SOC

…CPU DMANIC GPU UniCore DualCore MultiCore

SOCOS on SOA

Embedded Systems and Wireless Networking Lab.(Prof. Chih-Wen Hsueh)

19

Tiling Slideshow

J.-C. Chen, W.-T. Chu, J.-H. Kuo, C.-Y. Weng, and J.-L. Wu, “Tiling Slideshow,” Proceedings of ACM Multimedia Conference, pp. 25-34, 2006. (Best technical full paper award)

Music Beats

Time1 2 3 4 5 6 7 8

12 3 4

5

6

7

8

• A kind of new media that provides user elaborate photo browsing experience.• Photo filtering & organization• Music beat analysis• Spatial and temporal composition issues

20

Visual Region-of-Interest Determination

W.-H. Cheng, W.-T. Chu, and J.-L. Wu, “A Visual Attention Based Region-of-interest Determination Framework for Video Sequences,” IEICE Transactions on Information and Systems Journal, vol. E-88D, no. 7, pp. 1578-1586, July 2005.

• Video segmentation• Visual attention modeling• Camera motion registration• Media aesthetics theory• Feature fusion issues

21

Cubical Marching Squares: Adaptive Surface Extraction from Volume Data with Sharp Features and Better Topology

• Resolving topology ambiguity with sharp features and eliminating inter-cell dependency by sampling face sharp features. Best one for sculpting etc.

• Chien-Chang Ho, Pei-Lun Lee, Yung-Yu Chuang, Bing-Yu Chen, Ming Ouhyoung, Eurographics 2005,  Computer Graphics Forum,  Vol. 24, No. 4, pp. 537-545, 2005 , Ireland, August, 2005. (Best Student Paper Award)

Results

Comparison of methods

22

Hand Shadow Illusions and 3D DDR Based on Efficient Model Retrieval

• Real-time installation art and demo• First one from Taiwan• Got most discussion in Internet, and attracted hundreds of

participants for personal trial.

• J.S. Yeh, J.Y Chiang, T.H. Huan, L.F. Lin, Y.C Tsai, L.K Wang, D.Y.Chen, Y.Y Chuang, B.Y. Chen, M. Ouhyoung, "", Emerging Technology, SIGGRAPH2006, Boston, USA, July 2006.

23

Animating Picture

Yung-Yu Chuang et. Al. Animating Pictures with Stochastic Motion Textures. In ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2005), 24(3), Los Angeles, 2005.

We explore the problem of enhancing still pictures with subtly animated motions. The result is a looping video texture created from a single still image. We demonstrate the technique on a variety of photographs and paintings.

24

Real-Time Triple Product on GPU

Wan-Chun Ma et. al. Real-Time Triple Product Relighting Using Spherical Local-Frame Parameterization. In The Visual Computer (Proceedings of Pacific Graphics 2006), 22(9),, Taipei, 2006.

We implemented triple product on GPUs so that it can render the effectsof bump mapping, SBRDF, anisotropicBRDF, soft shadows and glossy materials in real time.

25

Feature fusion and selection from heterogeneous modalities Automatic discovery of semantic mid-level features Semantic threading across video sources Video search reranking through recurrent patterns

Large-Scale Video Structuring, Threading, and Retrieval

Communications & Multimedia Lab.(Prof. Winston Hsu)

low-level features

audioaudio

videovideo

texttext

multimedimultimedia contenta content

featurefeatureselectionselection

mid-levelfeatures

models &classifiers video unit video unit

structuringstructuring

documentsimilarity

topictopicthreadingthreading

videosearch

26

Goal – to promote progress in content-based video retrieval via open metric-based evaluation, supported by NIST, USA

Tasks including video story segmentation, high-level concept detection, rushes exploitation, and automatic video search

Achieving one of the best systems since 2003 Affiliating with Columbia University and IBM TJ Watson

TRECVID2003

24Participants

133 Hours of1998 ABC, CNN news& C-SPAN

TRECVID2004

38Participants

173 Hours of1998 ABC, CNN news& C-SPAN

Growing Participation

TRECVID2001

12Participants

11 Hours ofNIST video

TRECVID2002

17Participants

73 Hours ofvideo from Prelinger archives

TRECVID2005

65Participants

220 Hours of ARB, ENG,

& CHN news, BBC

stock shotsGrowing Data Sets

*courtesy of John Smith, IBM TJW

TREC Video Retrieval EvaluationCommunications & Multimedia Lab.(Prof. Winston Hsu)

27

Examples of topic “Pope sorry for his remarks on Muslim” in three languages, collected on 09/17/2006

English

EnglishArabic Chinese

Augmenting Image/Video Search with Multimodal Recurrences

Communications & Multimedia Lab.(Prof. Winston Hsu)

Improving text-based image/video search with multimodal similarities between documents across sources and domains

Formulating the solution as a random-walk framework Requiring no query expansion, search examples, or pre-trained models First work to consider recurrent patterns – improving text-based search up to 40%

28

(Prof. Shou-De Lin) Intelligent Information Process

Machine Learning

Artificial Intelligence

Knowledge Discovery and Data

Mining

Natural Language Processing

Social NetworkAnalysis

1. Electronic Holmes (Knowledge discovery in complex networks)

2. Interesting Instance Discovery (finding interesting things from data)

3: Explanation-based Knowledge Discovery

5: Information Retrieval and Information Extraction

4: Semantic Analysis (how machine can understand human language)

6: Ancient Script Decipherment

12

3

45

6

29

Image Mosaic and Super Resolution

MPEG-2, MPEG-4HD level (1280x720) (1920x1080)

MPEG Enc

MPEG Enc

MosaickingInteractive

cropper

SuperResolution

& Enhancement

SD levelasynchronous

MPEG-2, MPEG-4SD level (720x480)

zoom in

MPEG-2, MPEG-4HD levelHigh Visual Quality

Mosaicking

Temporal synchronizationFocal length compensation

RegistrationWhite balancingColor Matching

(Prof. Ming-Sui Lee)

30

B. ServicesProf. Yi-Ping Hung

• Editorial Board Member, International Journal of Computer Vision (IJCV), Kluwer Academic Publishers, 2004 – present

• Program Co-Chair, Multimedia Track, Third Taiwanese-French Conference on Information Technology (TFIT), Nancy, France, March 28-30, 2006

• Area Chair, Seventh Asian Conference on Computer Vision (ACCV’06), Hyderabad, India, January 22-25, 2006.

• Workshop Co-Chair, International Workshop on Task-Relevant Ubiquitous Media Access (TRUMA’05), Taipei, Dec. 19-20, 2005.

• Workshop Co-Chair, Second IEEE Workshop on Image and Video Registration (IVR’04), Washington DC, July 2, 2004.

• Workshop Co-Chair, International Conference on Computer Vision (ICCV’03), Nice, October 13-16, 2003.

• Registration Chair, International Conference on Robotics and Automation (ICRA’03), Taipei, September 14-19, 2003.

• Advisory Committee Member, Asian Conference on Computer Vision (ACCV’02), Melborne, January 2002.

• Program Committee Member: ICPR’06, ICCV’05, MVA’04, ICMI’04, ICPR’04, ACCV’04, ICCV’03, ICRA’03, ICPR’02, MVA’02, ICCV’01, ICDAR’01, MVA’00. ICPR’00, etc.

• Directorate, the Chinese Image Processing and Pattern Recognition Society, 1997 – present

31

Prof. Ja-Ling Wu• IEEE Consumer Electronic Society – Taipei Section: Chairman, 2004-2006.

• IEEE International Conference on Consumer Electronics (ICCE) 2005, 2006, Technical Program Committee: Chinese Representative.

• IEEE International Conference on Multimedia & Exposition (ICME) 2004, Organizing Committee: Demonstration Chair.

• International Workshop on Inter-media Semantic Extraction and Reasoning (ISERE) 2005: Organizing Committee.

• International Journal of Fuzzy Mathematics: Advisory Editor

• Journal of Visual Communications & Image Representation: Editor, Since 2004.

• Journal of Information Science and Engineering: Editor, since 2003.

• Reviewers for (1987 ~ present):

IEEE Transactions on Signal Processing, IEEE Transactions on Image Processing, IEEE Transactions on Multimedia, IEEE Transactions on Circuits and System for Video Technology, IEEE Transactions on Information Theory, IEEE Transactions on Communications, IEEE Journal on Selected Area on Communications, IEEE Signal Processing Letters, IEEE Conferences: ICIP, ISCAS, ICASSP, ICME, ICCE, and ISCE, IEE Proceedings part-I, IEE Proceedings part-F, IEE Electronics Letters, International Journal of Signal Processing, Signal Processing: Image Communications, Optical Engineering, Multimedia Systems, EURASIP Journal on Applied Signal Processing

32

Prof. Ming Ouhyoung

Guest Editor

• Special Issue on Search and retrieval of 3D content and associated knowledge extraction and propagation ,2006. EURASIP Journal on Applied Signal Processing (EURASIP JASP)

Program Co-Chair

• 2006 PacificGraphics Conference (PG2006),

Program Committee Member

• ACM Symposium on Interactive Computer Graphics and Games 2007

• ACM Virtual Reality Software and Technology (VRST), 2002, 2003, 2004

• PacificGraphics, 2000-2006.

• ChinaGraph, 2000-2006

General Co-Chair

• International Conference on Artificial Reality and Tele-existence (ICAT), 2000-2005/1/18

• International Conference on Computer Graphics and Imaging (CGIM), 2003-2005

33

Prof. Ai-Chun Pang

• Guest Editor: IEEE Wireless Communications Special Issue on Voice over Wireless Local Area Network

• Conference Committee:• IEEE Wireless Communications and Networking Conference (WCNC) 2007• International Conference on Mobile Computing, Communications, and

Applications (ICMOCCA) 2006• IEEE International Conference on Sensor Networks Ubiquitous and

Trustworthy Computing (SUTC) 2006• International Conference on Communications, Circuits and Systems

2005(ICCCAS),• The 11th International Conference on Parallel and Distributed Systems

(ICPADS-2005),• IADIS Web Applications and Research  (WARC2005) Conference• The Third IASTED International Conference on COMMUNICATIONS

AND COMPUTER NETWORKS(CCN 2005)• Mobile Computing Workshop 2004 and 2005

• Workshop Chair: The IEEE  International Workshop on Mobility, Agents, and Mobile Services

34

Prof. Chi-Sheng Shih•Publicity chair for RTAS 2005.•Advisory committee of OSSF.•Member of the Program Committee of ICPADS 2007.•Member of the Program Committee of TFIT 2007.•Member of the Program Committee of ICESS 2007.•Workshop Program Co-Chair of EUC 2007.•Member of the Program Committee of ATC 2006, 2007.•Member of the Program Committee of TENCON 2006.•Member of the Program Committee of WESE 2006.•Member of the Program Committee of RTCSA 2004 - 2007.•Member of the Program Committee of ICPADS 2005.

Prof. Chih-Wen Hsueh

•Program Committee member: RTCSA 2000, 2003, 2004, 2005, EEE04, 05, ADSN 2003

•Publication Chair: EEE04

•Registration Coordinator: RTAS 2001

•Program Co-chair: RTCSA 2006

35

Ⅴ.Future Developments

Education

The curriculum of GINM should be evolutionary enhanced so that the students graduate with most needed Networking and Multimedia technology skills, cultivated skills and hands-on experience.

Research

GINM seeks the cooperation with research institutes/universities in the country and all over the world. GINM also accepts the admission from foreign countries and welcomes the faculty member from foreign countries.

GINM should be recognized throughout the world as a leader in the research of networking and multimedia in the near future.

Industrial-academics collaboration

For cooperation with research institutes and industry in the country, GINM pursues the cooperation with other departments/institutes within NTU to promote the research results to the industry. Through the interaction with the industry, the research agenda in GINM will be highly related to the research and design needs of the industry so as to advance the technology development.