presenter: cindi dunn, principal evaluator office of educational innovation & evaluation college...

Presenter:

Cindi Dunn, Principal Evaluator

Office of Educational Innovation & Evaluation

College of Education, Kansas State University

June 13, 2013

EVALUATION AND ASSESSMENT

3RD PARTY EXTERNAL EVALUATION

NORTH GULF COASTAL HAZARDS COLLABORATORY TRACK II

“AREAS OF SUCCESS” METRICS

1. Cyberinfrastructure Improvement

(Cyberinfrastructure Development; Simulation

Experiments)

2. Research Production

3. Research Collaboration

4. Human Resource Development (Outreach

and Education Activities; Diversity)

5. Sustainability for Future Cyber-Enabled

Research and Innovation

DATA SOURCES

1. DRAFT Third Annual Report to NSF

2. Project Documents on Portal

3. NG-CHC Retreat held October 2012

4. NG-CHC Annual Conference June 2013

5. Collaboration Survey Spring 2014

6. Portal Analysis Spring 2014

7. Reports and other documentation



(Cyberinfrastructure Development;

Simulation Experiments)







Number of Researchers and Students Supported by NG-CHC

1. CYBERINFRASTRUCTURE AND EXPERIMENTS

Researchers Post-Doctoral Researchers

Graduate Students Undergraduate Students

0

10

20

30

40

50

38

6

15

1

48

6

20

4

34

7

27

8

Year 1 Year 2 Year 3


Metric Data Status Year 3

Number of high performance computing services and tools in NG-CHC

Operational Storm Surge Forecast System (MSU)

Data Grabber to Download Coastal Data (LSU)

Data Merging Tool (MMRI) Data Extraction Tool (MMRI) MODIS Daily Imagery Download Too (MMRI) ASGS-NHC Text Advisories to ATCF format ASGS-ADCIRC ASCII to NetCDF script Sulis Metadata Creator <beta> EnVis Spatial Watershed Erosion Tool FloodVix ADCIRC to ESRI Shape File Scripts Data to NetCDF Scripts Sulis Portal Viewer Sulis Severe Weather Viewer Sulis Informatics Services Portal Weather Underground Web Data Extraction

Tool (WUDEXT) Coastal Datafactory Multi-layer Data Infrastructure for System of

Systems


Need to define and develop measures

Metric Data Status

Functionality of each cyber-enabled tool and service

Year 1 = Stage 1 (organizing)Year 2 = Stage 2 (defined workflow)Year 3 = Stage 3 (product)

• What can users do now that they couldn’t do prior?

• What feedback have users provided?

Usability of each cyber-enabled tool and service

Functionality of the ASGS, SULIS, and Flood Risk Models

Usability of the ASGS, SULIS, and Flood Risk Models

Usability of Technologies developed from the ASGS, SULIS, and Flood Risk Models



Metric Data Status

Quantitative performance of each cyber-enabled tool and service

No data reported any year

Number of data analysis services developed and deployed in NG-CHC


Capacity and performance of distributed storage systems in NG-CHC

Developed by Mississippi in Year 1

Number of virtual organizations and communities established in NG-CHC

3 simulation experiments

Number of cyber-enabled applications supported by NG-CHC


Number of courses supported by NG-CHC

One reported in Year 1; no data reported in subsequent years




Experiments)







NSF No NSF NSF No NSF NSF No NSFYear 1 Year 2 Year 3

0

5

10

15

20

25

30

35

64

10

6

14

3

3

4

2

4

12

4

14

Alabama Louisiana Mississippi

Number of Publications in Major

Scientific Journals

2. RESEARCH PRODUCTION

Published or accepted

1

9 8

25

14

30

Presentation Presenters Presentation Presenters Presentation PresentersYear 1 Year 2 Year 3

0

5

10

15

20

25

30

35

40

45

50

4 4

17

610

3

9

13

15

3

7

3

13

4

19

4


2

Number of Presentations at Major

Peer-reviewed Conferences


20

9

43

12

44

10

Need to identify peer-reviewed

2

Number of Products and Other Types of Intellectual Property



Website Website Website

Portal Portal Portal

Structured Data Set via THREDDS

Server

Tools applicable?

Number and Types of Media Exposure



1

2

3

4

5

6

7

8

9

2

3

1

4

2

1

2 2

3

1

Radio TV News print Web

5

8

Awards Awardees Awards Awardees Awards AwardeesYear 1 Year 2 Year 3

0

2

4

6

8

10

12

14

16

18

9

4

9

24

3

3

1

4

1

11

4

4

4

1


Number and Types of Awards and Honors


12

5

17

79

5

Advisory Boards

Editorial Boards

Fellowships




Experiments)







Year 1 Year 2 Year 3 Total0

20

40

60

80

100

120

140

160

180

200

19 1642

27 2923

79

12 2231

65

Multi-State Multi-Institutional External

7

Number of Multi-State/Institution Research Projects (per year and total)

3. COLLABORATION

58 67 61

186

Number of Researchers from Multi-State/Institutions (per strategy* and total)

3. COLLABORATION

66

*Faculty could be in more than one strategy

38

86


10

20

30

40

50

60

70

80

90

100

10

30

1813

32

138

6

134

8

17

10

CI1 CI2 CI3 CI4 CI5

53

Quality of Collaboration (per strategy* and total Year 1 = 3.8; Year 2 = 4.1)

3. COLLABORATION

Governance , Administration, Mutuality (Y2H), Norms (Y1H)

CI5:E&O

CI4

CI3

CI2.4 River

CI2.3 ERFRR

CI2.2 SULIS

CI2.1ADCIRC

CI1: DDEM

0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.003.16

3.73

3.72

4.59

3.98

4.02

3.52

3.80

3.91

4.11

3.88

4.57

4.00

4.05

4.01

4.02

Year 2 Year 1

Overall Quality of Collaboration (per strategy* and total)

3. COLLABORATION

Satisfaction Commitment Efficiencies Productivity Overall Collaboration1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

3.944.38 4.18 4.09 4.15

3.72

4.444.17

3.87 4.04

Year 2 Year 1

Number of Publications Produced by Collaborative Efforts (NSF acknowledged)

3. COLLABORATION

Year 1 = 67%; Year 2 = 73%; Year 3 = 89%

No collaboration reported

Another department at the institution

Another institution in the state

Institution outside the state (in U.S)

Institution abroad0

1

2

3

4

5

6

7

8

9

10

322

6

2

5 5

2

9

5

3


Number of Collaborative Research Proposals Developed by NG-CHC

3. COLLABORATION


NG-CHC 1 Not evident Not evident

Multiple Partners 2 Not evident Not evident

External Partners 3 3 4





Experiments)







*Graduate Students in NG-CHC Research Program

*Number of Students Recruited for Project that Persist in STEM

4. HUMAN RESOURCES


NG-CHC Graduate Students

15 20 35

Earned Ph.D. 3 2 4

EarnedMasters 0 4 3

Earned Bachelors 0 2 0

Numbers of Minorities in Outreach and Education Activities

4. HUMAN RESOURCES


Education Activities

(UG/GRA)

3 women4 URM

8 women7 URM

8 women3 URM

OutreachSTEM Bowl

32 URM No report No report

Need values from Outreach Activities

*Number of Summer Workshops for Junior/Senior High Schools

4. HUMAN RESOURCES


• Beowulf (40)• Northern Gulf

Institute

• STEM Bowl Scholarship Competition (37)

• Lockheed Martin IT Day/STEM Bowl Scholarship Competition (1000)

• STEM Bowl Scholarship Competition

• Teachers Exploring Coastal Hazards and Resilience (20)

• Teachers Exploring Coastal Hazards and Resilience

• Dauphin Island Sea Lab (19 teachers)

# of participants and satisfactory levels not reported or limited.

*Number of Summer Workshops for Undergraduate/Graduate Students

4. HUMAN RESOURCES


• Evidence not explicitly stated

• Delft3D Tutorial

• Evidence not explicitly stated

• USGS National Wetlands Research Center Modeling Teams

• NG-CHC Student Poster Session (15)

# of participants and satisfactory levels not reported or limited.

*Number of faculty and staff trained in NG-CHC.

*Number of students trained in NC-CHC.

*Number of new courses.

4. HUMAN RESOURCES

Define and obtain existent data.




Experiments)







*Number of funded research clusters & other organizations established.

*Number of researchers and graduate students in each established center.

5. SUSTAINABILITY

Define and obtain existent data.


1

2

3

4

5

6

7

8

9

10

4

1

2

2

3

2

2


Number of Successful Awards from NSF and Other Agencies

5. SUSTAINABILITY

$2,604,374

$1,083,000$210,000

EPAFESD

IndustryNASA

NSFNOAA




Experiments)





