CAFE Faculty Council Meeting

March 2, 2018, 1:00 PM, Weldon Suite, E.S. Good Barn

Attending: Council Chair Ken Jones; Council members C. Agouridis, T. Chambers, R. Durham, M. Farman, M. Reed, and S. Wesley; Megan Lucy

Minutes

1. Dr. Ken Jones, FC chair, called the meeting to order at 1:01 PM. The Agenda is attached (p.4).

2. The minutes of the January 2018 FC meeting were approved; Dr. Agouridis so moved and Dr. Durham seconded.

3. Ms. Lucy gave an update on the FC elections currently in progress (handout attached, p.5): the voting window remains open until 11:59 PM. The latest update shows 86 votes had been recorded, or ~30% of CAFE faculty which is about average. She drew attention to the nominations for candidates at Assistant Professor and Lecturer levels; 12 Associate Professors had been asked to run but only 3 accepted. Assistant Professors do have a 5% Service DOE but may be steered away from FC service to focus on their other 95% of DOE. In 2017 there was criticism that females had been over-represented among the nominees, and this year the distribution is more even. After the voting is concluded, Ms. Lucy will send a breakdown of voting by department.

4. Digital Measures (DM): Ms. Lucy reported on a follow-up survey of junior faculty after they had received their annual performance review (APR) scores (handouts attached, p.6-13). These results are preliminary as there have been only 18 responses so far. Three respondents thought that DM had a negative effect on their APR outcome. Also one faculty member had appealed their APR score, but this was not one of the 3 respondents just mentioned and the outcome of the appeal did not hinge on DM. The last 3 pages of this set of handouts were comments made by survey respondents and Ms. Lucy’s responses. A second set of handouts (attached, p.14-27) are PowerPoint slides for a report on the first year of DM use attempting to identify areas where improvements may be made.

• Slide #7 (p.20) describes building Extension-specific screens into DM. Ms. Lucy can provide slides of the new screens to the FC; the intent is to have the new Extension screens fully vetted by Extension faculty in time for the Fall Semester APR round. Question—if screens change from the 2017 versions, will the Extension Assistant Professors have to re-enter their data? CAFE administration feels strongly the answer should be no, thus MS. Lucy’s office will do it for them if necessary. On that slide, Dr. Farman suggested adding an “Other” category to the screen list, for those cases where faculty do not see a match between the existing categories and what they do—an example being social media. Nobody currently knows how the different

categories will be weighed, and Ms. Lucy will seek feedback from the Deans and Chairs on what they want to see.

• Slide #8 (p.21) on Grants—this year the Assistant Dean’s office will get monthly uploads from OSPA so that this data can be vetted earlier and more frequently. the issue was raised about how to present subcontracts where the grant is to a PI who is not at UK. Drs. Agouridis and Farman stated that it is important that faculty get credit for the whole of such grants as otherwise they will be discouraged from engaging in those collaborations.

• Slide #9 (p.22), Research Publications—importation from Web of Science leads to disambiguation errors. Ms. Lucy will import the publications but faculty will need to review the output.

• Slide #10 (p.23)—how does the FC view the value of impact factors, which take a lot of time to find? Ms. Lucy stated that the Deans and Chairs will be looking for those numbers. Dr. Agouridis stated that if impact factors are going to be used for Promotion and Tenure evaluation, they should be kept in DM. Dr. Reed stated that several departments track impact factors making it unnecessary for individual faculty to track them. Ms. Lucy had already found 28 journals not listed in Web of Science. Publications in these may be imported using EndNote, Google Scholar, or other sources, but these sources may not provide the impact factor. Dr. Wesley asked what happens if the faculty member leaves the impact factor box empty? Ms. Lucy replied that CAFE administration knows that is a possibility. Also on this slide, does the FC agree with the Deans/Chairs’ choice that only actually published publications will be included? Dean Bennett is OK with including “accepted” publications. Drs Agouridis and Farman agreed with support for inclusion of “accepted.” Dr. Durham suggested that in such cases, in the following year that publication should be identified as having been accepted in the previous year. In conclusion, the sense of the FC was that papers “in press” is OK for DM.

• Slide #12 (p.25)—Faculty training for DM will be improved in 2018. There will be 3 trainers. Video and tutorials were not much used in 2017 so those will not be a focus for improvement. Dr. Wesley suggested that training should be offered during the course of which faculty can start their own DM records.

• Slide #14 (p.27)—Ms. Lucy suggests that faculty wait until the new modifications are finalized before starting their DM records. This is expected to be early April.

5. CAFE Faculty Awards (handout, p.28): in a previous meeting the FC had discussed taking on George E. Mitchell Award and the Master Teacher Award, to be presented at the annual Fall Faculty meeting. Also the winner of the Mack Whitaker Award, managed by KASEP, would be recognized at the Fall Faculty Meeting. For the Mitchell and Master Teacher awards the Assistant Dean’s office is thinking to reduce the number of letters of nomination from 2 to 1. Dr. Agouridis moved that the FC would take on these awards, and Dr. Farman seconded. All present were in favor, none opposed.

6. Old Business: none. 7. New Business: none. 8. Announcements (1): The next Lunch and Learn meeting will be March 7 at 12:00 PM,

tentatively in Room 426, Barnhart Building [subsequently changed to 249 Barnhart]. March’s Lunch & Learn guest speaker is Dr. Mike Reed, Director of International Programs for CAFE.

9. Announcements (2): The annual FC meeting with the Deans will be in N24 Ag Sci North, at 1:00 PM on April 4. [subsequently changed to 3:00 PM April 9, in the Culton Suite, E.S. Good Barn]. FC members that cannot attend should send a proxy to represent their department.

10. The next FC meeting will be May 4 at 8:30AM, in which outgoing FC Chair Dr. Jones will welcome the new FC members.

11. The meeting was adjourned at 2:16.

Drafted 3/20/2018, revised 3/23/2018 by Thomas Chambers, FC secretary.

Post APR Survey

March 2nd 2018, 12:30 pm MST

What is your department?

# Answer % Count 1 Agricultural Economics 10.00% 2 2 Animal and Food Sciences 10.00% 2 3 Biosystems and Agricultural Engineering 10.00% 2 4 Community and Leadership Development 5.00% 1 5 Dietetics and Human Nutrition 15.00% 3 6 Entomology 5.00% 1 7 Family Sciences 0.00% 0 8 Forestry and Natural Resources 5.00% 1 9 Horticulture 5.00% 1 10 Landscape Architecture 0.00% 0 11 Plant and Soil Sciences 10.00% 2 12 Plant Pathology 5.00% 1 13 Retailing and Tourism Management 10.00% 2 14 Veterinary Science 10.00% 2 Total 100% 20

What is your title series?

# Answer % Count

1 Clinical 0.00% 0

2 Extension 20.00% 4

3 Lecturer 20.00% 4

4 Regular 50.00% 10

5 Research 0.00% 0

6 Special 10.00% 2

Total 100% 20

To what extent do you feel the use of Digital Measures affected the outcome of your annual performance review?

# Answer % Count

1 Negatively affected 11.11% 2

2 Somewhat negatively affected 5.56% 1

3 Not affected 61.11% 11

4 Somewhat positively affected 16.67% 3

5 Positively affected 5.56% 1

Total 100% 18

To what extent do you feel your final CV from Digital Measures reflects your accomplishments in 2016-2017 in each of these mission areas?

# Question Not at all reflective Somewhat

unreflective Neither

reflective nor unreflective

Somewhat reflective Very

reflective Total

1 Research 5.00% 1 20.00% 4 15.00% 3 45.00% 9 15.00% 3 20

2 Teaching 5.00% 1 15.00% 3 20.00% 4 35.00% 7 25.00% 5 20

3 Extension 10.00% 2 10.00% 2 45.00% 9 20.00% 4 15.00% 3 20

4 Service 10.00% 2 10.00% 2 15.00% 3 40.00% 8 25.00% 5 20

5 Administration 20.00% 4 5.00% 1 40.00% 8 20.00% 4 15.00% 3 20

6 Professional Development 5.00% 1 15.00% 3 15.00% 3 55.00% 11 10.00% 2 20

Please list specific accomplishments that you feel were not effectively reflected in the CV generated using Digital Measures. This will help us find ways to better collect and report on those accomplishments next year.

Teaching

Implementation of novel teaching methodologies into courses, and meaningful 1:1 student contact. There is a section for new curriculum development, as well as a broad range of 1:1 student mentoring options that allow for descriptive text (for instance, undergraduate research). Longer narrative descriptions have always been better addressed in the teaching portfolio, though.

Teaching small classes in which TCE not reported This is not a Digital Measures issue. UK does not report TCEs for courses that don’t meet the threshold of enrollment or response numbers. This has been the case for many years. Lack of previous TCE for the course that I taught. This could either mean that the TCE was missing because of course enrollment or response rate, or that the faculty member wanted past semester TCEs displayed to put the current semester’s in context. If it is the latter, that has never been something asked for on our APR CVs, which are supposed to be limited to the 2-year evaluation period, but it is something that could be included in the Teaching Portfolio, at the faculty member’s discretion. It is difficult to remember but I think I would like to have more area for the student evaluation section. Maybe more student comments. I did include this in my teaching portfolio though.. This is what the teaching portfolio is for. We do not have the capacity to import student comments to DM. Curriculum development Curriculum development is a free text box, where the faculty member can write anything and as much as they want. It is found under “Program and Curriculum Development” under Teaching”

Research

Invited talks and others scholarly activities Guest lectures and invited presentations of a wide variety are able to be captured under either “Other Non-credit Teaching Activities,” or “Research- Presentations” depending on if the talk is best listed under the teaching or research heading. It is too unclear what this person means by “other scholarly activities” to do much with that statement.

grants with PIs from outside of UK (scope and size not reflected in amounts and UK only PIs) The Deans have told us multiple times that they only want the money coming to UK denoted, and consider this much more important than the overall size of the grant outside of UK. We can’t import non-uk PIs from OSPA, because they are not in the OSPA data, but faculty may add them into grants in Digital Measures themselves.

(Please note that I don't feel this negatively affected my APR, since my chair was aware of my situation): It was difficult to report research capacity building work We are trying to add a screen for field trials, and on-farm/lab/community research that may encompass some of this concern. In a broader sense, this would be a good thing to discuss in the narrative. publications in review We have heard many comments to this effect. The faculty were unhappy that they were only able to report published publications. This is not a DM limitation, it was a decision by the Deans and Chairs, which we will bring back to them this year for a second look. Can't show articles in peer review. See previous comment on publications. Only research articles that were accepted appeared on digital measures and the ones I put into digital measures that are "under review" did not appear on the CV. See previous comment on publications. The grant work and publications are the biggest problems for me. It is unclear what this is specifically referring to, but likely things addressed in the other comments about grants and publications.

Extension

Better Extension subcategories We are working on Extension specific screens that have been vetted by multiple Extension faculty to address this need in 2018.

Limits number of Extension presentations This is not actually true. As many Extension presentations may be reported as the faculty member wants. We were asked to create a screen for summarizing presentations, which we did, because faculty members told us they didn’t want to report every presentation, and the Chairs agreed that that level of reporting is unnecessary given the repetitive nature of many Extension Presentation. We asked for a suggested amount of 3-5 most representative presentations be entered in full bibliographic detail, but this was not meant to suggest that there was a limit of 5. We will better communicate this in training materials.

Service

Inadequate opportunity to document service to professional association and professional development Professional development activities are collected in a way that allows for as much descriptive text as the person wants. Professional Association service is collected under “Service to Profession” and include fields for “role” this is separate from just

“Professional Association Membership.” We could open up the “Description” field for “Service to Profession” too, to allow for more descriptive text.

General

As a person with a 100% service appointment, I felt that it was very difficult to express the extent and quality of my work. The most important part of my appointment was buried within the body of the report, and felt more like an afterthought. I needed to rely on the individuals evaluating me at the department level to know what I do and depend on the people above the level of the department to fully read my narrative statement. At the moment, there is too much potential for introducing errors and bugs in the report to rearrange the order of the activities based on DOE. It is hopeful that new developments from Digital Measures could change this in the future, but not for 2018. We will continue to communicate with the faculty and the departments that the order of the sections of the CV should not be taken to signify an order of importance.

Everything was captured, but the rigid structure doesn't allow for arranging things in context/order of effort. Yes, this is both by design, because it allows for predictability for reviewers in where to find information, and a limitation of DM. We cannot currently reorganize CVS based on DOE without the possibility of introducing many errors and bugs into the system. We will continue to look for improvements, but in the mean time the message needs to be communicated that order on the CV does not, in any way, reflect order of importance.

At times difficult to find/label certain accomplishments with the pre-generated categories We will be reworking the Data Entry guide based on the feedback we got this year to try to make categories easier to find.

The faculty expected a summary of accomplishments - the CV was long and didn't seem to emphasize major accomplishments or summarize them. I had to redo my digital measures CV for the faculty. We will work with chairs to better communicate expectations to their faculty committees. This person should not have been asked to re-do their CV. A high percentage of my DOE is service, this section is buried in the format as it is currently outlined. See previous comments about ordering the CV based on DOE.

Megan Lucy, Assistant Director for Faculty Resources Megan.lucy@uky.edu

Report on Digital Measures One Year In

Questions for Faculty Council, Department Chairs, Associate Deans What is your opinion on the value of Impact Factors, given that they

must be individually looked up and hand entered by the faculty members?

What is your opinion on including submitted, accepted, and in-progress articles on APR CVs?

What is your opinion on whether or not the “full” amount of grants for which the faculty member has a subcontract should be reported on the CV, given that this would have to be self-reported data?

Would you like the opportunity to fully review all of the required data

fields for 2018?

2017 Implementation • All non-tenured faculty used Digital Measures for Annual Performance

Reviews (75 total) • Megan led 17 training sessions and three open-lab events. She also did

one-on-one consultation with 5 faculty member in person and around a dozen over the phone.

• Faculty members were surveyed in December (after submitting APRs) and February (after receiving APR results).

• Chairs were surveyed from November- January.

• Continued comments are welcome.

Faculty Survey 2- February 2018 To what extent do you feel the use of Digital Measures affected the outcome of your annual performance review?

Note: Only 1 faculty member appealed a 2017 performance review, and it was not one of the two who said their APR scores were negatively affected in the survey.

Faculty Survey 1- December 2017 32 Respondents.

How many hours did you spend preparing your CV for review this year compared to last year?

Faculty Survey- December 2017 32 Respondents.

For your 2016-2017 CV in Digital Measures, how many hours did you spend on each of these sections?

Purple- primarily faculty entered, little opportunity for improvement Green- area of focus for improvement in 2018

Area of Focus- Extension In 2017, faculty members entered Extension activities on non-Extension specific screens, and marked the

items as “Extension.”

This system actually worked, there were very few errors, and the Extension portion of the CVs were correctly generated for the APRs. However, the message that faculty perceived was that Extension was not as important as the other mission areas.

For 2018, we are working on building an Extension heading in Digital Measures with these screens: Publications (i.e. numbered publications) Media Contributions (i.e. news interviews) Program and Curriculum Development Newsletters (generates table like the one used last year) Education and Training Programs Presentation Summary Field Work

Sent out to 15 Extension faculty members across all departments with Extension faculty, and including

Associate Dean Palmer, and Chairs Bastin and Stringer for review and comment (received comments from 4 faculty members)

Additional work is being done to learn from other land-grant universities how they address Extension in Digital Measures

Area of Focus- OSPA Grants In 2017, we did not receive an initial load of OSPA grant data until October 17. APR

CVs were due October 20. This led to late discovery of issues in the data, and having to extend the deadline.

In 2018 we have requested the following changes: Monthly uploads of OSPA grant data to allow for frequent vetting of the data

ahead of the APR deadline. Inclusion of scope account grants in the import

We can’t fix the issue of non-UK PIs not being imported because this information does

not exist in the dataset being imported.

Faculty members have complained that subcontracted grants only show the amount coming to their UK program, and not the full amount of the grant outside of UK. Research Office has advised that this is as it should be. Input from chairs is also requested. We can’t import the “full” grant amounts, because it is not in the original dataset.

Area of Focus- Research Publications In 2017, non-tenured faculty members were trained on how to import their own

publications from Web of Science (and upon request, EndNote or Google Scholar)

In 2018, Megan has done the Web of Science import of 2017 publications for tenured faculty, and will follow-up in the Fall to do the 2018 publications for all faculty.

A comparison of publications available through a Web of Science import against the 2016 Experiment Station Report resulted in a 30-60% success rate depending on department. Complicating factors are: Authors publishing under only first initials, not full names 28 journals in which our faculty published in 2016 are not collected by Web of

Science Disambiguation by staff (Megan) who are not the authors Disputed dates- Web of Science publication dates that don’t match self

reported dates.

The Assistant Dean’s Office can continue to import the publications, but faculty members will always have to review them and add missing publications.

A full report on the viability of Web of Science imports is available, should anyone want it.

Area of Focus- Research Publications Impact Factors cannot be imported from Web of Science, and must

be individually looked up and hand entered in to Digital Measures. This is a best guess as to why the faculty members reported such a surprising large amount of time spent on publications. Can we revisit the value of Impact Factors in light of this

information?

In 2017, the Deans and Department Chairs agreed that only published publications would appear on APR CVs. This is one of the top sources of complaint from the junior faculty members. They want their submitted, accepted, and in-progress papers to count towards their APRs. Should this decision be revisited for 2018?

Area of Focus- Training and Instructions

How helpful did you find each of these resources in working with Digital Measures?

Area of Focus- Training and Instructions We will have three trainers for 2018- Megan Lucy, Lisa Collins, and Susan

Ferguson-Wheeler We will schedule training sessions with specific focuses by title series, type of

activity, or department specific training sessions. We will increase communication about open-lab opportunities, especially in

September. We will focus more attention on improving the Data Entry guide and website

and less on the lesser used materials (video, written tutorials)

We encourage senior faculty to ask questions of the junior faculty who went through the trainings and process last year.

We encourage Chairs to attend the trainings with their faculty, and in some departments, staff may want to attend as well.

Questions for Faculty Council, Questions for Department Chairs, Associate Deans

What is your opinion on the value of Impact Factors, given that they must be individually looked up and hand entered by the faculty members?

What is your opinion on including submitted, accepted, and in-progress articles on APR CVs?

What is your opinion on whether or not the “full” amount of grants for which the faculty member has a subcontract should be reported on the CV, given that this would have to be self-reported data?

Would you like the opportunity to fully review all of the required data fields

for 2018?

Do you want more detailed information out of the Assistant Dean’s Office? Full survey reports? Report on Web of Science Import? Detailed Extension screen proposal?

Requests for Communication about Digital Measures Please discourage faculty members from “working ahead”

and trying to use Digital Measures before the new 2018 training materials are out. Working ahead will ultimately lead to more work and frustration.

Please emphasize that the order of items in the CVs generated in Digital Measures does not reflect order of importance, and the current lack of ability to reorder the CV is to protect from errors and bugs in the system.

Please communicate with your review committees about Digital Measures and what to expect on their colleagues CVs.

When someone has a question or complaint about Digital Measures, please direct them to Megan. Specific, detailed, information is key to improving the system for everyone.