Personal Statement on Research, Teaching and Service

Bradley Hemminger

1. INTRODUCTION My career started when, as an undergraduate, I connected a stationary bicycle to a virtual reality representation of the city streets of Boulder, Colorado so that the user could “bike” around and explore the city. The idea that I could take a physical experience like bicycling around the city, and reconstruct it in a digital environment so that it could be made available to anyone, anywhere, has always stuck with me. Later in graduate school I learned how to make photorealistic images from digital model data. This led me to explore possible research applications in medical imaging, where radiologists viewed multiple two-dimensional slices of anatomy on film using lightboxes. Working with digital representations of this same information, we visualized the images on computer monitors using the same paradigm with which the radiologists were familiar (tiled presentations of the 2D images showing slices through the human body), and were able to demonstrate comparable or improved performance by the radiologists. However, the breakthrough for me was in recognizing the power of combining highly interactive human computer interfaces with novel visualizations such as real-time three-dimensional displays. The result was SeeThru, a system I developed that allowed physicians to visualize the interior of a patient’s body for surgery planning and therapies. This allowed the surgeons to perform operations they could not have done previously [Hemminger, 2005]. The reward, as a researcher, of seeing patients’ lives saved was unforgettable. Since that time, my passion has been the investigation of novel combinations of information representations with dynamic information visualizations to achieve new breakthroughs. While working as a researcher in medical imaging, I was a leader in informatics and visualization research as I worked to embrace the changes made possible by digital representations of information objects such as medical images. As the scope of my research program broadened, I came to believe that the next big revolution would be in two areas: first, in dealing with the information explosion coming from genetics research, namely bioinformatics; second, from the explosion of scholarly literature that threatened to overload scientists who were already unable to process all the information relevant to their research. My research interests and my love of teaching led me to my current position as a SILS faculty member, half funded by the Carolina Center for Genome Sciences, with a focus on bioinformatics. In accepting the position, I realized that my goals were also my most significant challenges:

• Building my own research program in new areas: information representation and information visualization applied to bioinformatics and scholarly communications

• Building a strong bioinformatics program within SILS • Helping to build a nationally recognized campus-wide bioinformatics program at

UNC • Developing collaborations between SILS and other units on campus to support

research, teaching and service in informatics

One of my strengths is in building strong collaborative research and educational projects across different disciplines. I believed building a successful bioinformatics program at SILS would require not only developing a first rate research lab within SILS, but also forging strong relationships between SILS and other efforts on campus. Only by doing this could we develop UNC into a nationally recognized leader in bioinformatics, while at the same time developing individual units like SILS. My first years at SILS were spent building my new research program, and working to build the necessary infrastructure within SILS and on campus to support the development of a nationally prominent bioinformatics program. The rewards have come during the last two years. The research projects and educational programs that I have worked to develop have become very successful, generating quality research papers, attracting students and collaborators, and spinning off new projects. In the table below I summarize my major achievements, and then discuss them in more depth in the following sections.

Goal Progress

Build a strong research program at SILS

Established and continue to direct the Informatics and Visualization Laboratory (IVlab) at SILS. I currently lead 7 funded research projects.

Multiple publications in the top journals in my areas (JASIST, Bioinformatics, JAMIA).

Have received grant funding from NSF, NIH, NLM, foundations, and private companies. In the last 5 years have received 10 grants (total budget $9,628,000). As project leader on two large grants and PI on three smaller grants I am directly responsible for a budget of $1,166,000.

Developed a strong group of students working on my research projects, who have published, made presentations, and received fellowships and grants. Currently nine students are working with me on research projects On average I fund 7 students annually as research assistants, and supervise 4 master papers, 1-2 field experiences and several independent studies.

Build one of the top bioinformatics programs within schools of Information and Library Science

Established and continue to direct the Bioinformatics Research Group at SILS. Host seminars, workshops, lab, listserv, and web pages.

Established UNC Certificate of Specialization in Bioinformatics in conjunction with Masters degree at SILS.

Developed and continue to teach Bioinformatics Research Review course in SILS (INLS 706).

Established and funded three training programs for information and library scientists in bioinformatics (EPA, CCGS/HSL BIRSTP, BCB).

Raised the profile of SILS through guest editing the first JASIST special topics issue on bioinformatics.

Hosted an international conference, served as a conference chair and on program committees, wrote international standards, and made numerous presentations.

Developed and maintain the most extensive and widely used resource describing academic bioinformatics programs.

Help establish campus wide programs in my research areas to support research training, education and engagement.

Established and direct the Biomedical Informatics Research, Support, and Training Program in the Carolina Center for Genome Sciences (CCGS-BIRSTP), which does collaborative research, provides support to faculty on campus, and oversees training programs in bioinformatics for librarians.

Established and oversee the Center for Research and Development of Digital Libraries, including the seminar series.

Developed Electronic Theses and Dissertations programs for SILS (2004) and for UNC campus (2006).

One of four UNC faculty members who developed the Bioinformatics and Computational Biology (BCB) Curriculum and Training Program on campus. Have served on executive committee, and currently serve on admissions committee. Investigator on all three BCB training grants that fund approximately 8 new doctoral students each year.

Developed a new course (GNET 211, Information Science for Bioinformatics) and have taught every year to BCB students and campus students.

Develop collaborations between SILS and other academic units

I have established very strong connections to numerous programs on campus. This is evidenced by our joint participation in publications and presentations (over 60 co-authors from outside of SILS, including 13 other departments, and the libraries), research grants (over 50 faculty from over 20 different departments), and service (in my committee work, including chairing two campus committees in which there is involvement with faculty from many departments, the libraries and the graduate school).

2. RESEARCH Part of my motivation for studying scholarly communications is my underlying desire to empower individuals and societies by making the world’s knowledge freely and conveniently accessible to everyone. I have been influenced by earlier visionaries such as Vannevar Bush who wrote in 1945, “Consider a future device for individual use, which is a sort of mechanized private file and library. It needs a name, and to coin one at random, “memex” will do. A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.” [Bush, 1945]. Vannevar Bush was prescient in envisioning how technology would change the way we interact with information. We can, however, go beyond augmenting our individual memories. Think how much more powerful it would be to have all human knowledge available to help us anytime, anywhere—the materials in every library accessible to everyone. J.C.R. Licklider wrote in 1960, “It seems reasonable to envision, for a time 10 or 15 years hence, a 'thinking center' that will incorporate the functions of present-day libraries together with anticipated advances in information storage and retrieval.” [Licklider, 1960]. Today, we continue to evolve our thoughts about how these globally accessible stores of knowledge will be available, and how people will interact with them. Significant changes are already taking place. How many people visit the library physically in person to look up information today versus “Googling” it? Only a small minority do, even among academic scholars like research scientists at UNC [Hemminger 2005, Hemminger 2007]. How many high school students use traditional encyclopedias versus the novel approach taken by Wikipedia to capture knowledge as part of a freely available and freely editable community effort? What is exciting is that we are now capable of building the systems envisioned by Bush, Licklider and others. Even more importantly we are not limited to simple extensions of our capabilities by having access to larger digital collections of knowledge. Rather, the opportunity now exists to envision and build truly revolutionary systems that enable people not only to have access to all information, anywhere, anytime, but also to utilize it in ways never before imagined.

Since joining SILS, my research has focused on information representation and interactive information visualizations. I believe these two areas are critical to the development of new models that will transform our information use. Because of my passion for making human knowledge available freely to everyone, my focus has been on scholarly communications. Within scholarly communications I have concentrated on the particular domain areas of medical informatics and more recently bioinformatics. Scholarly content in medical informatics and bioinformatics has two features that make them especially significant to study. First, their scholarly content is comprised of many different data types ranging from simple text (scholarly journal articles, books, conference proceedings, etc.) to many types of structured data (genetic sequences, proteins, metabolomic data, laboratory experiment results, statistical analyses, images, etc.). Second, much of the research is interdisciplinary, generally involving researchers from multiple fields, including biology, genetics, computer science, biostatistics, math,

chemistry, and others. Developing systems to meet these needs addresses not only today’s problems but also tomorrow’s challenges.

2.1 Information Representation The primary focus of my research in information representation has been in understanding the kinds of information content scientists utilize, and how they use and share it, in order to design novel digital representations capable of improving scholarly communications.

Studying Information Use as Background for Information Representation I have approached the problem of understanding scientists’ information use and information workflows from two levels: studying individual scientists and studying the labs or research groups within which they work. Because scientists working in “bioinformatics” come from many departments in the life sciences and medical sciences, much of my work involves studying general faculty in these departments, as well as individuals whose primary research area might be considered “bioinformatics”. In my first years at SILS, I interviewed a number of bioinformatics laboratories and researchers informally, which led to pilot work developing a “common data model” database framework, and characterization of particular labs such as proteomics [MacMullen 2002]. Following on this work, in 2004 we received a large P20 planning grant from NIH, establishing the Carolina Center for Exploratory Genetic Analysis. I am the project leader of one of the three subprojects of this grant, which studies bioinformatics infrastructure. Our goal is to study the existing practice within the labs at UNC and to develop a common underlying database model that will enable integration of data from all clinical and research labs, while addressing issues of privacy, security and user acceptance. Following on this work, I am preparing an R01 grant to NIH with the primary aims of developing the model in conjunction with existing NIH and NLM standardization efforts, and to study in more detail some of the socio-political effects we discovered in the initial grant, which showed that most research labs do not take advantage of standards or common data models, even when storage is provided for free and their use is encouraged by their department or institution. I presented some of our initial findings in a panel session at ASIST 2005. I have two manuscripts in preparation, one on the summary of the labs’ data models and the resulting common data model, and the other on how laboratories make use of standards and shared database facilities, including factors affecting why they do and do not take advantage of such infrastructure.

The second effort is to understand individual scientist’s behavior and use of information content. I have two large grant funded studies underway in this area, the first a web-based survey of scientists, and the second conducting detailed interviews of scientists. The survey was of over 1,000 science faculty and doctoral students at UNC. I found that the information seeking behavior of scientists has been transformed significantly in the past 15 years, with most all work being done from their offices rather than visiting the library, and an increased use of electronic resources such as online databases and web pages. I presented initial results from this work at the ASIST conference in 2005, and

there was significant interest from other institutions around the US in using our methodology to survey their own faculty. This led us to develop a proposal for a national study of 20 universities based on our initial UNC survey study, which has just been funded. Our first two sites (Oklahoma and Florida) started enrollment in the winter of 2006. I have a number of publications resulting from the UNC survey including the main summary paper (JASIST, in press 2007), a detailed analysis looking at results grouped by departments and by characteristics of the scientists (in preparation), and analysis of open-ended comments (College and Research Libraries, submitted). Separately, the interview study has just finished the pilot phase, and final interviews will begin in early 2007.

Information Representations of Scholarly Communications

My background in information representation (especially databases) and open scholarly communications have influenced my thoughts about how digital scholarly objects should be represented. Because of this, my Neoref digital repository (URL) represents all objects as generic open digital content items, with any object capable of referencing another. This relational database approach provides much more power and flexibility than previous, usually hierarchical, constructs, which evolved from the need to define a single physical location for a resource such as printed journal or book. It also results in significantly faster database interactions, and the ability to expand gracefully to include new content (datatypes) as the technologies evolve (see ultrastructure methodology in next section, ref Jeff Long main paper). For the scholarly articles it also helps in being able to reference content items easily with respect to different contexts (multiple uses, different annotations, multiple versions of the item, etc.). I have proposed a number of changes that I believe are necessary for us to move to this new virtual digital world of resources (JCDL 2004, ASIST 2003 conference, ETD 2004, AMIA 2002, Neoref web pages, CRADLE presentation 2003).

To perform controlled quantitative research about scholarly communications requires an established collection of digital content accessible for research purposes that allows comparing different information interaction techniques. To that end I have invested significant effort in the development of large-scale resources at UNC that will allow our research group to compile detailed information about scholarly communications practices. In addition to development of the common data model repository for studying bioinformatics resource datasets, I have worked to develop three other types of resource collections: journal articles and annotations, institutional digital library repositories and wikis.

Much of my user interface design work is centered on understanding how scientists use information resources. We know the primary resource type used by scientists is still journal articles (Hemminger JASIST 2007). My long-range goal is helping to create simple and easy access to a universal collection of human knowledge. One of our research lab’s grand challenges is to implement the NeoRef digital library and populate it with all scholarly articles and their annotations (reviews), and to support different dynamic searching interfaces which will allow the results to be downloaded and stored into personal yet public shared digital libraries.

The work above deals with more general understandings of information seeking and use. In some cases where I am working on designing novel representations of objects or workflows, my work focuses on the details of particular elements. An example of this is our study of annotations in scholarly communications. My doctoral student John MacMullen is leading this work (supervised by Gary Marchionini and myself). As a new assistant professor focused on a small non-traditional area of information and library science (bioinformatics), I am fortunate to have so many outstanding students like John interested in working with me. John is my most senior doctoral student (going into his fifth year). He has 15 publications, was the recipient of a Microsoft fellowship last year, and this year was successful in obtaining an NLM grant. John’s dissertation work is focused on the contextual analysis of variation and quality in human curated Gene Ontology annotations. He is collaborating with two curation efforts, the Saccharomyces Genome Database (model organism database for yeast) and the Gene Ontology Consortium Meeting and Annotation Camp, and studying their curation records as well as observing the curation process first hand.

As part of our scholarly communications research work, we know the lack of complete and accurate metadata is a significant problem in the development of large digital repositories. I was motivated by discussions with my colleague Jane Greenberg, who has looked at the quality of author self-contributed metadata [Greenberg 2006]. Metadata is the data that describes the resource itself. One of the biggest challenges in developing freely accessible public repositories such as universities’ institutional repositories is having sufficiently good metadata descriptions to allow the content to be searched for and retrieved through search interfaces. I designed a system based on an existing public domain digital library application (MIT’s DSpace [DSpace 2007]), enhanced with the ability to determine the metadata automatically from certain types of documents (ETDs) through data mining, and combined this with an improved user interface design. The results from the initial analysis of our improved system showed that the time to submit was substantially faster and user satisfaction was much higher as compared to the unmodified digital library [Hemminger 2004a]. I have a larger study, based on the first four years of electronic theses submissions at SILS, which will be completed in the spring of 2008. It will analyze efficiency, user satisfaction with the system, error reports, and quality of author-contributed metadata compared to librarian assigned metadata.

I chose to study wikis because they epitomize dynamic, interactive scholarly communications involving groups and communities. To study wikis, starting about 3 years ago I made the decision to move all my research projects to wiki-based group collaborations and to record all our information on wikis [NeoRef Wiki]. I now have over 2 years of material compiled on 15 research projects, with the corpus continuing to grow rapidly. Additionally, I have developed and hosted wikis for over twenty other research groups, and for conferences including ASIST 2005 and ASIST 2006. I hosted a workshop discussing the effectiveness of wikis as tools for conference communications at the November ASIST 2006 conference, and presented our initial results analyzing usage statistics and user feedback from the ASIST and JCDL wikis (with a paper detailing studies of the two wikis currently in preparation for submission in spring 2007 to the

Journal of Biomedical Discovery and Collaboration). My research analyzes the usage of the wikis by looking qualitatively at individual use and experiences (especially collaborative), and quantitatively at patterns of use through examination of usage statistics by groups (users of the same type), frequency of access to individual pages or classes of pages, access over time, and security and trust issues.

Novel Information Representations Once we have an understanding of the information types used, and how scientists use them, the challenge is developing database models to support this. Perhaps the most challenging aspect of bioinformatics databases is that the underlying elements continually change. This is because the technology used to study samples evolves so rapidly that an experimental method used this year may be obsolete in two years, and all the information gathered between the new and old systems is incompatible. As I have begun to assimilate the information gained from my studies of scientists and labs, I have developed an understanding of what parts of the data model are consistent (generalizable) and which may change over time. Standard relational databases cannot support constant change to their underlying schemas gracefully. As a result I believe a completely different underlying system is required. My search for a solution led to discussions with my colleague Morgan Giddings in Biomedical Engineering and to our discovery of the ultrastructure theory proposed by Jeff Long in notational systems theory [Long 1995]. We believe this method, which represents the generalizations as “deep structure” and the changing elements as “surface structure” can allow constantly evolving information structures to be properly represented, such as those required in bioinformatics. We had difficulty getting these novel ideas funded in our initial grant submissions to NIH, but Morgan and I have continued to pursue this. We have been able to interest Jeff Long himself in collaborating with us to apply his theory to bioinformatics, and he now participates (remotely) in our lab’s work. We have completed an initial phase of constructing such a system, and incorporated a collection of initial biological knowledge as well as data from Morgan’s mass spectrometry lab [Maier 2006, Maier 2007]. With these results, we are developing a new grant proposal for submission in the spring of 2007. We are excited by our new results, and believe we have generated a truly novel approach that will be the first system capable of handling the constantly changing data elements and constructs involved in bioinformatics.

While my work in information representation is focused primarily on bioinformatics and scholarly communications, I often have novel ideas that generate interesting research in related fields. In these cases I have collaborated with others to help begin, or seed, novel research ideas in those communities. An example of this is my Virseum project (Virseum URL), in which I have proposed capturing all the content of museums and their exhibits digitally, so that they can be viewed on a computer in a virtual reality setting. What is novel about the approach I have suggested is that we create a virtual reality computer model of an entire exhibit space by using laser range-finder scanning for capturing spatially accurate geometric representations and combine this with high resolution digital camera images for capturing the color textures in the scene. Users can

then explore the museum exhibits in a virtual reality that is both spatially accurate and visually compelling. The result is that anyone with a computer has access anywhere, anytime to any recorded collection of any museum. I believe this has the potential to revolutionize access to museum material for both scholarly study and casual browsing. Together with the research staff at 3rdTech (technology transfer company operating the 3D digitization) and the Ackland Art Museum, we digitized Ackland’s 2005 Plum, Pine, and Bamboo: Seasonal and Spiritual Paths in Japanese Art exhibit, and made it available as the first spatially and visually accurate virtual museum exhibit in the world [Hemminger 2005c]. Evaluations by museum staff and conference attendees praised this exhibit and the methodology [Hemminger 2004b, SIGGRAPH 2005].

2.2 Interactive Information Visualization My research in this area is focused on dynamic, highly interactive interfaces that allow users to search through large datasets and quickly find what they are looking for, whether the content items are journal papers, educational resources, images, or medical records.

Dynamic User Interfaces for Search

I have designed a web based meta-search interface to the NeoRef digital library that allows interactive searches of the full-text of all the content items to be retrieved and displayed with highlighted terms at interactive rates (1/10th of a second). Once an article has been discovered, the challenge is how to find and review the most relevant content items, especially when the results may number in the thousands or millions (such as with search engines like Google Scholar). I am pursuing work on dynamic clustering attempting to develop real-time clustering (similar to Clusty [Clusty 2007] and Microsoft’s new Live Search interface [Live Search 2007]) in conjunction with dynamic filtering interfaces such as the relational browser work done by my colleague Gary Marchionini [Marchionini 2003].

I have evaluated the user interface developed as part of NeoRef for reviewing articles. As part of this work, I also conducted an extensive analysis comparing the utility of full-text searching (as possible in NeoRef or GoogleScholar) against traditional metadata (title, abstract) searching. Using the PubMed corpus and reviewing articles in two bioinformatics domains (schizophrenia genetics and Arabidopsis genetics) I found that full-text searching discovered over 10 times as many articles with nearly perfect recall. The precision of the full-text search results, however, was lower than the metadata search results, confirming the typical tradeoff between precision and recall. My study established a new result showing that by using the number of hits of the search term in the full-text to rank the importance of the article, the performance of full-text searching could be improved so that it outperformed metadata searching in recall while maintaining as good or better precision [Hemminger 2006a].

User Interfaces for Public Outreach and Translational Activities

Librarians and information scientists build resources to help others find information, and at the same time they themselves also serve as the resources. I have established two centers at UNC along these lines, in that we are developing information resources and also developing (by training) librarians who will be resources. The first is part of our recently awarded Carolina Environmental Bioinformatics Research Center (one of only two centers funded in US by EPA), for which I am director of Public Outreach and Translational Activities. In this project we are concerned about communicating environmental bioinformatics research information to many users in many different contexts (including scientists, policy makers, and the general public). In the second project I developed and direct the newly formed Biomedical Informatics Research Service and Training Program (BIRSTP) within the Carolina Genome Sciences Center. This project is in close collaboration with the UNC Health Sciences Library, and in particular the bioinformatics@HSL group including KT Vaughan and Barrie Hayes. Part of the reason for creating the BIRSTP program was because students and faculty interested in information science research, support and training did not have natural homes within the existing CCGS structure. With the creation of the BIRSTP project we have brought together a cross-disciplinary group focused on addressing the information science issues facing researchers working with bioinformatics: what information representations to use, how to manage and share data, and how to analyze and visualize results. We are developing both web-based resources and people resources to help provide this level of bioinformatics support to the campus.

For both of these centers I have established and funded training programs that support students working as interns in the programs (under the direction of EPA library director April Errickson, and under Barrie Hayes and KT Vaughan of HSL respectively). Once both of these resource centers are well established, we will conduct user surveys to gauge how successful these methods have been in improving user experiences.

User interfaces for navigating large images In my first two years after joining SILS, I was still completing a number of funded research projects in medical image visualization. Two of my research projects in that area have had a very significant impact, with almost all medical imaging equipment now adhering to the medical image communication and display standard I worked on [DICOM 1998-2004, AAPM 2003], and real-time 3D visualization is becoming standard practice for surgery planning [Hemminger 2005a]. I also completed the development and evaluation of the first computer based system to perform as well or better for mammography readings [Pisano 2002]. However, this system clearly still had some weaknesses, which led me to identify the several remaining challenges for real-time navigation and searching of large two dimensional images [Hemminger 2003]. I extended this work by conducting a comprehensive study to determine the best interfaces for supporting general navigational operations in very large images. The interface I proposed (using pointing devices for drag panning and the scrollwheel for zooming) outperformed all the standard interfaces methods available at the time [Hemminger 2005d]. This type of interface is now becoming standard in high-end applications such as

PhotoShop, and is beginning to trickle down to web-browser applications such as GoogleMaps.

User Interfaces for Medical Informatics A significant user interface challenge in medical informatics is how to present the information in a personal health record to end users, and how to integrate this information into the tools they use in their daily lives (such as calendars). As part of our Personal Health Record Usability grant, I developed a project to study the effectiveness of different user interface models for tracking medication history for elderly patients. We surveyed how users currently keep track of their information, interviewed them about their needs, and tested them on several different user interface paradigms. We found that the appropriate user interface depended significantly on the task, as well as other factors. This study has just concluded and has been submitted recently to JAMIA. Mary Roth of Pharmacy heard about our work, and we are now making plans to coordinate our research with her 5-year NIH grant so that together we can evaluate our interfaces on mobile devices, and conduct large scale evaluations in a clinical environment.

One of the most common requests I receive in my role as a campus bioinformatics resource is from researchers wanting to know how best to display their scientific results, and share them with other researchers. Most of the time I provide expert consultation to researchers on what to do. However, in some cases I have the opportunity to develop a new type of tool or resource that we believe will advance the field. An excellent example of this is the TAMAL project I developed with Pat Sullivan of the medical school. With Pat I developed a tool to help identify candidate SNPS based on computer analysis of multiple public resources, to help scientists choose the most fruitful SNPS for further experimental study [Hemminger 2006b]. We built this tool into a public web resource, which has been used over 50,000 times by scientists worldwide, and has become a valuable resource to the community.

3. TEACHING ACTIVITIES Teaching Philosophy

My goals as a teacher are:

1. To help students master the course material, while at the same time helping them to develop broader and more important skills such as critical thinking and communication.

2. To provide a safe and open environment where my students can be truly challenged, and honestly and fairly evaluated.

3. To continually improve my teaching skills and adapt them to meet the needs of my students.

I have found the third goal critical because students learn in different ways, and we have an increasing number of options for delivering the content to students (lecture, seminar, online, distance education, field experiences, etc.). For each of my classes I try to understand the dynamics of the class, and the needs of individual students in the class. Based on the class and the students, I will use different teaching styles, even sometimes within the same class, to connect with the students. I have tried to improve my teaching by acquiring a better understanding of teaching theory and practice, and through critical feedback. I have attended courses offered by the Center for Teaching and Learning (CTL), in particular related to active learning, which I favor as a teaching style. I have found that students learn best from an interactive environment where they are engaged and expected to perform. I want the students to see themselves as agents seeking knowledge, where the course materials and my knowledge are some of many guides that help them integrate information and knowledge from our course into their frame of reference.

I learn best when the environment is open, friendly, and without distractions. I try to provide this for my students. In the first class I carefully explain the course content, and my expectations for the students during the semester. I do set high standards for students, and have found that they tend to adjust themselves to my expectations. If I set challenging goals, and provide a supportive environment, they meet the goals, and we are both pleased. I let them know up front that this is a safe environment where I want everyone to be comfortable expressing themselves. Even more importantly, I work all semester at modeling this by how I treat students who ask and answer questions, especially when students come to the board to present their work and hear my comments and comments from others in the class. I reward alternative ways of thinking. In addition to the readings, labs, assignments and the final exam, I frequently give small homework exercises, which are to be handed in at the start of the next class, as a proactive measure to evaluate and then assist any student who is not staying on track with the class. I have found that a student’s commitment to a course can be diminished if they are penalized for what they perceive as poor instructions or assignment descriptions. I have made numerous small improvements to my in-class exercises and assignments, striving to keep the quality and clarity of my exercises and assignments very high.

Students constantly practice their thinking skills with our many exercises and assignments, and in their critical discussions of the readings and textbook. We practice

communication skills when students come to the board to work problems and explain their reasoning. They also practice communication in the group exercises and assignments, and when they serve as the group presenter. I prefer to teach skills rather than memorization of content, and I strive to do in my teaching what I advocate in my research, so I put all the class material (syllabus, schedule, readings, lecture notes, examples, and homework) online. Having all material and resources available via the web, combined with students being able to communicate via email and the listserv, greatly facilitates many of our students who work full-time or part-time, and have limited time at Manning Hall. Students are encouraged to annotate the lecture notes, allowing them to focus on our discussion. Much of the class time is spent working through problems, and reviewing assignments, as I find this the most effective way to teach critical thinking. Assignments and labs are tied to practical real life problems, to help motivate the students and to facilitate their transition to work environments. I try to involve students as much as possible in the class. I see my role as a guide and mentor, to help them understand the materials I present, and know where to go to learn more.

I have learned a great deal from student feedback, and from our faculty peer teaching reviews that occur each semester. Additionally, my responsibility to perform a peer review of another faculty member each semester (with a more senior faculty member) has provided me with the opportunity to observe many outstanding teachers, and to expose me to examples of other teaching styles. I have also attended seminars on mentoring students (offered by the Office of Post-Doctoral Students), which I have found very helpful in guiding my interactions with students doing doctoral research and independent studies. This feedback has led me to make many improvements to my courses, my teaching, and the environment, for example, adding a second whiteboard in 208, and video teleconferencing in 214. The improvement I am most excited about, however, is the incorporation of wikis into our classroom’s virtual space. For several years I had worked with Scott Adams, our IT director, investigating hardware and software solutions that would allow students to present their work from their computer work space to the rest of the class easily. Last year I had the idea to use the wikis I was using for our research groups, to allow the students to display and share their work with others. It has been a resounding success in our class this fall, with students placing their exercises on the wiki to share with the rest of the class, being able to modify work on the wiki during class, and having the ability to easily jump between different students’ solutions to contrast their work (http://bioivlab.ils.unc.edu/sandbox/INLS623Fa06/). Based on my experience and the students’ very positive feedback, I believe that wikis offer significant added value compared to existing collaborative group communication tools such as websites, listservs and chat. I am very excited about this technique, and plan to have discussions within SILS as well as with the Center for Teaching and Learning about it, in hopes of offering a workshop to other faculty on how to utilize wikis for classes.

I try to provide students with work that challenges them, and excites their minds. I try to expand their thinking and experiences, and tie our theory to the practical experiences in their real life. I have found that for the courses I teach, group learning and group projects offer particularly good opportunities for more of a “real world” experience, and I use them extensively for this purpose.

Courses Because my position is half funded by the Genome Sciences Center to develop research and teaching programs on campus in the area of bioinformatics, my SILS teaching load is one course per semester. Except for my first semester, I have always taught additional courses as an overload. The additional courses have been bioinformatics related, initially a special topics course in bioinformatics at SILS that evolved into the Bioinformatics Research Review Course (INLS 279). I offer this course every semester as part of our Masters Specialization in Bioinformatics at SILS. In addition I have developed an information science course as part of the Bioinformatics and Computational Biology (BCB) core curriculum, which I teach once a year in support of the Genome Sciences Center. I also guest teach modules on campus for other departments such as BME.

The course I teach within SILS has always been a core course from the database curriculum. Initially, I taught the Introduction to Databases course for five semesters. In 2004 I worked with other SILS faculty to revamp our database curriculum to better match our students’ needs after we added the undergraduate major. We expanded curriculum by one course (Database I, INLS 157). I developed this new course, and have taught it since its inception in fall 2004. Finally, I am finishing development of a new course on information visualization for SILS, which I hope to offer in the spring of 2008.

Introduction to Databases: INLS 256 (changed to INLS 157, renumbered to INLS 523)

While this is not a required course, most of our information science and many of our library science students take it. This mix of students presents an interesting challenge because of the diverse backgrounds of the students. When I first taught the course, I benefited from utilizing class notes from Stephanie Haas, and one of our doctoral students, Sheila Denn, who had previously taught this course. I felt much of the course material was well designed. I slowly evolved this course beginning in the third semester I taught it, primarily to update examples, to make all material available online, and to adapt the course to better suit an active learning environment.

I believe that this is an excellent course, based both on the positive student evaluations, and also because several other instructors (Lee, Lu, Blake) have taken my materials and used them successfully to teach the 256 course and later the 156 course (Introduction to Databases, the course that 256 evolved into after the database curriculum revision).

Database I: INLS 157 (renumbered to INLS 623)

As part of our database curriculum revision I created this course, expanding the database coverage of SQL queries, providing a complete coverage of indexing and tuning optimizations, adding non-text databases, and adding semi-structured formats, in particular XML, which is a topic in which many of the students have expressed an interest. After teaching it the first semester I met with Dr. Blake who was teaching Introduction to Databases and Kristin Chaffin who was teaching Databases II and Web Databases, to improve our coordination of content between the courses. This resulted in shifting material between the courses. Now all of indexing is covered in INLS 157, and

all of data warehousing is shifted to Databases II. This has resulted in improved continuity for the students when taking the Database sequence of courses. I will be meeting with Rob Capra at the end of this semester to continue this coordination, since he has taken over teaching some of Kristin’s former courses.

Special Topics: Bioinformatics Research Seminar: INLS 310-109

Bioinformatics Research Review: INLS 279 (renumbered to INLS 723)

These are essentially the same course. I created this course and offered it as a special topics course first. Then when I developed the masters certificate program in bioinformatics for SILS, I committed to offering it on a regular basis, and we assigned it a permanent course number (INLS 279).

When I came to SILS, I felt there were not enough one credit hour reading seminars, or project courses, where students could gain expertise selectively in specialized areas. Additionally, one of my goals was to build a bioinformatics community at SILS, as well as for the UNC campus. I have had very positive experiences in other departments, participating in journal club courses emphasizing critical thinking and communication skills, and I wanted to recreate this environment at SILS, as part of my effort to build a bioinformatics community. The course is structured so that students are expected to present 2-3 papers, which they carefully read, and research further. We often have a combination of undergraduate, masters, and doctoral students. They are expected to present the paper to the class, and lead us in a critical discussion of the paper. Supporting my research aims of digital libraries and open scholarly communications, all presentations are open to the public, and all the presentations and summaries are archived and publicly searchable and available on the web (http://ils.unc.edu/bioinfo). Through the classes’ efforts we are building an article review resource for information science papers on bioinformatics. One other change that I incorporated three semesters ago is to dedicate 1-2 sessions to “journal reviewing”. I educate the students about the process of journal reviewing, and then we use example journal review formats. We each perform a formal review of a paper, and then analyze our efforts. The students, especially those planning to continue in academics, have found this very helpful. My role in this course is to help guide the discussion, teach students how to evaluate papers critically, and to help them tie the individual papers into the larger literature on that subject.

Information Science for Bioinformatics: GNET 211 (renumbered to GNET 712)

I developed this course as part of the core curriculum for the campus’ Bioinformatics and Computational Biology program. I have taught it each semester since its inception. The course is designed to give BCB doctoral students a foundation in databases, and to introduce them to the main information science topics relevant to their studies (classification/indexing, information searching, information retrieval, digital libraries, metadata/ontologies). I also use the course to introduce the BCB students to SILS faculty and library and information science research interests, in particular by having Cathy Blake provide a lecture to them on text mining, and KT Vaughan provide an introduction to bioinformatics resources and searching. This is usually a small class and we work a

number of database exercises to give the students a foundation in the use of relational databases and common database environments. This is followed by discussions of current topics and issues in the other information science areas. While this course is successful as a standalone course, over the next two semesters I would like to integrate it more tightly with the other BCB courses so that the exercises and assignments could build on work the students do in the other BCB modules. This would support a long-term goal of our BCB curriculum committee, which is to integrate our core curriculum more closely.

I believe I am an effective and enthusiastic teacher. My student evaluations are positive (see teaching evaluations section). My students have expressed to me that they enjoy my courses, and the material presented is helpful to them after graduation. I have improved as a teacher since joining SILS. In my courses since spring 2003 generally over half of my students rank me as one of the best teachers they have had, and deserving of a teaching award. These words hold great value to me as a faculty member of our professional school, where the teaching standards are very high, and we have many truly outstanding teachers.

An old Chinese proverb says “a teacher for a day is like a parent for a lifetime”. My goal as a teacher is to help the students learn not just the course material, but critical thinking, reasoning and communication skills that they can apply their entire lives. I believe I am accomplishing this.

4. SERVICE ACTIVITIES Faculty members have a responsibility to serve at many different levels. My service activities include service within SILS, service within the university, service to the profession, and service to the local community. While many of my service commitments are to help manage and run organizations (SILS, professional societies, conferences), I strive to engage in activities that integrate service with my research and teaching activities.

Service within SILS Like my colleagues, I serve on SILS committees. Early on, I served on the Information and Technology Committee (helped provide initial survey of distance education technical requirements), the Undergraduate committee (serving during the transition when we added the undergraduate major), and the Ad-hoc Space committee. I was pleased to be appointed by the Dean to serve as chair of the Ad-hoc Space committee when the faculty had indicated that space was one of the highest priority issues facing SILS. As the chair, with significant support from Scott Adams and Rebecca Vargha, I oversaw our investigation of building renovation possibilities, and eventually the 2nd floor renovation in 2004. I used my grant funding to renovate space on the 4th floor stacks, 5th floor stacks, and in room 300, as well as to add video conferencing to our new conference room (214). Later, I served on the Masters Committee and the Personnel Committee, including playing a major role on the search committee for our Biomedical Research Imaging Center Position. I currently serve on the Information and Technology Committee, the New Building Committee, and the BRIC Search Committee.

Two of the more visible contributions I have made at SILS have been the Center for Research and Development of Digital Libraries (CRADLE, http://ils.unc.edu/cradle/), and the Bioinformatics Program. In order to help promote digital library efforts at SILS, within the University, and in the community at large, I spearheaded the effort to create a digital library center at SILS. We created CRADLE in 2003, with Paul Jones, Gary Marchionini, Helen Tibbo and I serving as the executive committee. I also serve as the organizer, and have overseen the seminar series, web pages, digital library and content management system, and recruitment of CRADLE fellows since the inception of the Center. CRADLE interactions have led to several exciting projects including the Virseum project (URL), which resulted from David Carr sharing his passion for museums with me, and the Minds of Carolina Project started by Helen Tibbo and Paul Jones, which is designed to capture and preserve the knowledge of UNC scholars (http://www.ibiblio.org/minds/). As part of our research into digital libraries, I built a digital library to house our masters papers, and make them available through search engines. This has been operational since 2003, and served as a test pilot for the development of the campus Electronic Theses and Dissertations program.

I have developed and nurtured our bioinformatics program at SILS. I established a seminar series the first year, and then evolved this into a fixed course (Bioinformatics Research Review, now INLS 706). I have also developed a certificate of specialization in

bioinformatics, awarded with the Masters degree, for LS and IS students. This was approved the end of 2003 by the graduate school. I established funding for fellowships for bioinformatics doctoral students, which has funded 3 SILS students (John MacMullen, Dihui Lu, and Nancy Baker). I founded and direct the Informatics and Visualization research laboratory, as well as being a member of the Interaction and Design Laboratory.

Service within the University I participate in committees in other units on campus including Genome Sciences, Radiology, Health Sciences Library, and the Academic Affairs Library. I have been very involved with the campus Program in Bioinformatics and Computational Biology (URL). I was one of the four executive members who developed the program, its curriculum, and established the current framework and committees. I continue to serve on the admissions committee, and also teach a course in support of this curriculum.

In 2003 I initiated efforts on campus to bring Electronic Theses and Dissertations (ETD) to the UNC campus. In December of 2003, I was appointed by the Provost and the Dean of the Graduate School to chair the committee to implement ETDs at UNC. In January 2006 we fulfilled our task, bringing online the ETD system at UNC, and had an astounding 80% of the dissertations submitted electronically in the first semester and over 96% the second semester.

I have worked hard to educate people and groups within the University about information science issues, in particular digital libraries, scholarly communication and electronic publishing. I have given talks on my research projects and current issues to Health Sciences Library staff, the UNC-Chapel Hill Scholarly Communication Working Group, Office of Arts and Science Information Services, the UNC Scholars for Tomorrow fellows, the Academic Affairs Library, and the Genome Sciences faculty.

In 2006 Sarah Michalak (University Librarian) and Provost Shelton appointed me to co-chair the University Scholarly Communications Committee, with Deborah Gerhardt of the Law School. As part of this effort we are working to educate faculty about their rights in the scholarly communications process, and how to retain control over their works. We have already brought about changes in copyright agreements (URL), and are making available resources that show faculty which publishers are supportive of their rights (JAR tool, http://neoref.ils.unc.edu/uncjournals/).

Service to the Profession I believe one of our most important contributions is to our profession, through professional societies, conferences, journals, reviewing, and publications and presentations. I am active in a number of professional societies including the American Society for Information Science and Technology (ASIST), American Medical Informatics Association (AMIA), American Association of Physicists in Medicine (AAPM), Medical Image Perception Society (MIPS), American College of Radiology (ACR), International Society for Optical Engineering (SPIE), Institute for Electrical and Electronics Engineers (IEEE), and the Association of Computing Machinery (ACM). I

am involved in special interest groups within ASIST and ACM. I serve as a reviewer for many journals including the Journal of American Society for Information Science and Technology, Bioinformatics, International Journal of Digital Libraries, Decision Support Systems Journal, IEEE Transactions on Medical Imaging, Academic Radiology, and Journal of Digital Imaging. I have served as grant reviewer for NSF and NIH. I have reviewed conference papers for the SPIE, JCDL, MIPS, ACM SIGGRAPH, and RSNA conferences.

I have been on the program committee for the SPIE, JCDL, and MIPS. I co-hosted the international Medical Image Perception Society X conference in 2003. I have hosted wikis for several conferences including ASIST and JCDL. After hosting the official ASIST 2005 conference wiki (their first) I was awarded their Certificate of Appreciation for Enhancing Communications Opportunities. I have given a number of invited talks, including at the FDA, DICOM workshops, several universities, and the Swedish Institute for Standards. In additional to my regular presentations and publications, I have contributed to panels, short talks, and mentoring sessions at SPIE, JASIST, JCDL, and RSNA. I developed a one-day course on Image Quality, which I have taught in conjunction with conferences (SPIE Medical Imaging, American Association of Physicists) and by invitation for Mayo Clinic. I was the guest editor of the special topics issue on bioinformatics in JASIST, which I think has helped to energize interest in bioinformatics in our field. I recently released the fourth edition of our survey of academic programs in bioinformatics in the US, and this year we extended the survey to include all informatics programs, making it the most comprehensive available. Finally, I have consulted for over a dozen companies ranging from product development and design, to general education about information science theory and practice.

Perhaps my single largest contribution has been to international standards development. I was initially asked to help lead, with Hartwig Blume, the development of a grayscale display function standard under the auspices of the Digital Image Communications in Medicine effort. I took over as chair of this group in the late 1990s and produced the three standards from that group, Grayscale Standard Display Function, the Grayscale Softcopy Presentation State Standard, and the Hanging Protocols Display Standard. Recognizing my work, the American College of Radiology (ACR) appointed me to represent them in DICOM Working Group 11, and extended this to appointment to WG 15 (Mammography and Computer Aided Detection) and WG 17 (3D Display). I was pleased to have been a significant contributor to eight supplements to the DICOM standard and also to the AAPM guideline on the Assessment of Display Performance for Medical Imaging Systems 2003. These standards have had an enormous positive effect on the industry, enabling the changeover to digital capture, storage and display of medical images in clinical practice. When I stepped down as chairman of WG 11 in 2004, the ACR awarded me their Distinguished Committee Service award.

Service to the Community

I volunteer my time at a number of local non-profit organizations. For example, I have helped with information technology issues at Rainbow Soccer, where I serve as a member of the Board of Directors. I have helped them develop a database system, online

registration, and update their financial and accounting system. I have also combined this with field experiences and independent studies for students. I often work with companies or organizations on campus to provide practical information science experiences for students, and for graduates. I give talks at elementary schools to schoolchildren on various science topics, and I have coached competitive youth soccer and basketball in Chapel Hill for over ten years.

5. SUMMARY I believe I have laid a very strong foundation for continued success in my endeavors at SILS and UNC. My SILS research program is going strong, with an increasing number students working with me on research projects, as well as masters papers, field experiences, and independent studies. The SILS bioinformatics certificate program continues to draw students to UNC, and we are now graduating two a year on average (up from one in the previous years). At SILS we have added Cathy Blake to the faculty, which significantly strengthens our research in informatics and especially text mining. This year we are adding a new faculty member in medical informatics and image databases who will be part of the Biomedical Research Imaging Center on campus. Combined with our other faculty with interests in medical and health informatics, this gives us one of the strongest bioinformatics faculties among ILS programs. The campus BCB program has expanded and we currently have an average of 8-10 new doctoral students per year under our BCB training grants. We are halfway through the process of establishing the BCB program as a standalone doctoral program at UNC, which is a significant step toward our university becoming a prominent bioinformatics research institution.

The majority of the research I have planned for the next 3 years are continuations of my existing research. The information seeking behavior study will continue nationally. This should lead to the most complete single description of academic scientists (20,000 subjects from 20 universities). I am investigating combining this with the excellent longitudinal studies done by Carol Tenopir and Donald King (who is now here at SILS) to develop more detailed definitions of the transformations that have occurred and are still occurring in academic scholarly communications. This will be complemented nicely by the in-depth interviews we will have from academic scientists working in bioinformatics. We plan to complete our work on the common data model for exchanging bioinformatics data, and work to incorporate our results into national efforts on this front, as well as a new CTSA grant (below). We will continue to develop our NeoRef digital library and further test dynamic search interfaces to this universal knowledge repository. Morgan Giddings and I remain very optimistic about the ultimate success of our work using Ultrastructure for the information representation of biological knowledge. We believe this will have significant payoffs over the next 5-10 years. I have several faculty members and students interested in extending the Virseum work, especially now that rendering speeds from web interfaces are approaching real-time, enabling us to study subjects’ online experiences in virtual reality museums.

I am pleased to have leadership roles in several projects, centers and committees on campus and I plan to use these opportunities to advance UNC as much as possible. I am excited by the chance to lead the Scholarly Communications Committee and believe I can help UNC become a leader on that front just as we did with ETDs. In addition to my leading the Public Outreach and Translational Project of the Carolina Environmental Bioinformatics Research Center, the Informatics Project of the P20 planning grant, and the CCGS Biomedical Informatics Research, Support and Training Program, I believe much of our continued success will come from the training programs that I have put in place in each of these settings. This will allow us to attract top quality students, provide excellent service, and develop the next generation of leaders in these areas. Outside of

UNC, I plan to become more involved at higher levels in the professional societies and conferences such as ASIST, AMIA and JCDL, as well as in editing journals.

We have several new research efforts under development. I am collaborating with David Threadgill on his Interdisciplinary Research Consortium Grant on the Genetics and Co-morbidity of Stress (U54), adding my expertise in visualization and public outreach and translational activities. One large effort that I am very excited about is the development of a campus-wide informatics educational and research program. This has been a dream of mine since joining SILS, and our new Dean, Jose-Marie Griffiths is spearheading this effort on campus. This has brought together faculty from a dozen different academic units on campus, all who are excited about the educational and research opportunities possible with a university-wide program focused on informatics. With my research and teaching experience in this area, and my experience in establishing the Bioinformatics and Computationally Biology program at UNC, I believe I can be a significant contributor to this program. Related to this, Dean Griffiths and I are participating in the campus’ Clinical and Translational Science Award submission to NIH (January 2007, 6 million per year, PI Paul Watkins) to bring about a campus-wide transformation of the academic, clinical, and translational research enterprises by integrating these activities. I am assisting Dean Griffiths who leads the biomedical informatics project, which will integrate biomedical informatics efforts across all research and clinical laboratories on campus, realizing the dream we started in the P20 bioinformatics planning grant.

In summary, SILS and the Carolina Center for Genome Sciences have provided me an excellent foundation from which to grow. I continue to be excited by my research opportunities, and especially about the projects and people with whom I am working and teaching. I look forward with great enthusiasm to my future endeavors at UNC.

6. REFERENCES American Association of Physicists in Medicine (AAPM) Task Group 18 Assessment of Display Performance for Medical Imaging Systems (2003). Bush V, “As We May Think,” The Atlantic Monthly; July (1945); Volume 176, No. 1; 101-108. Greenberg J, Pattuelli MC, Parsia B and Robertson WD, “Author-generated Dublin Core Metadata for Web Resources: A Baseline Study in an Organization”, Journal of Digital Information, Volume 2 Issue 2, Article No. 78, (2006). Clusty 2007, available at http://clusty.com/. DICOM 3.0 Grayscale Standard Display Function, 1998. DICOM 3.0 Grayscale Softcopy Presentation State Storage, 2000. DICOM 3.0 Mammography Computer Aided Detection, 2001. DICOM 3.0 Breast Imaging Report, 2003. DICOM 3.1 Hanging Protocols, 2004. DICOM 3.1 Color Softcopy Presentation State Storage, 2004. DSpace Digital Library Software, (2007), available at http://www.dspace.org/ Licklider JCR, Man-Computer Symbiosis, IRE Transactions on Human Factors in Electronics,volume HFE-1, pages 4-11, March (1960). Hemminger B.M. Softcopy Display Requirements for Digital Mammography. Journal of Digital Imaging, Sep;16(3):292-305, (2003). Hemminger BM, Fox J, Ni M, “Improving the ETD submission process through automated author self contribution using DSpace”, Electronic Theses and Dissertations 2004 conference, Lexington KY, published by NDLTD, http://hdl.handle.net/2340/213, (2004a). Hemminger BM, Gerald Bolas, David Carr, Paul Jones, Doug Schiff, Nick England, “Capturing Content for Virtual Museums: from Pieces to Exhibits”, Demostration, Joint Conference on Digital Libraries (2004b), Tucson AZ. Hemminger BM, Molina PL, Egan, TM, Detterbeck RC, Muller KE, Coffey C, Lee JKT, “Assessment of Real-Time 3D Visualization for Cardiothoracic Diagnostic Evaluation and Surgery Planning”, Journal of Digital Imaging, Vol 18, No 2, pp145-153, (2005a). Hemminger, BM, “Information Seeking Behavior of Scientists”, Panel Presentation at 2005 American Society of Information Science and Technology conference, Charlotte, NC, Nov (2005b). Hemminger BM, Gerald Bolas, Doug Schiff, “Capturing Content for Virtual Museums: from Pieces to Exhibits”, Journal of Digital Information, Vol 6, Issue 1, Article No. 313, p1-14, March (2005c). Hemminger BM, Bauers A, Yang J, “Comparison of Navigation Techniques for Large Digital Images”, ACM Transactions on Computer Human Interactions, submitted (Oct 2005d).

Hemminger BM, Saelim B, Sullivan PF, Vision TJ, “Comparison of Full-text Searching to Metadata Searching for Genes in Two Biomedical Literature Cohorts”, JASIST, submitted November 2006a. Hemminger BM, Saelim B, and Sullivan PF, “TAMAL: an integrated approach to choosing SNPs for genetic studies of human complex traits”, DOI 10.1093/bioinformatics/btk025, Bioinformatics 22: 626-627, 2006b. Hemminger BM, Lu D, Vaughan KTL, Adam SJ, “Information Seeking Behavior of Academic Scientists”, JASIST, in press, expected spring 2007. Long, J. G. and Denning, D. E. 1995. Ultra-structure: a design theory for complex systems and processes. Commun. ACM 38, 1 (Jan. 1995), 103-120. DOI= http://doi.acm.org/10.1145/204865.204892. MacMullen, W.J., Parmelee, M.C., Fenstermacher, D.A. & Hemminger, B.M. (2002). Defining an Open Metadata Framework for Proteomics: The PROMIS Project. In Proceedings of the 2002 American Medical Informatics Association (AMIA) Annual Symposium, p. 1093. Maier C. “CORE576: An Exploration of the Ultra-Structure Notational System for Systems Biology Research”, Masters paper, School of Information and Library Science, 2006. Maier C, Giddings M, Hemminger BM, “Application of Ultrastructure Theory to Biological Knowledge”, in preparation for submission to BMC Journal of Bioinformatics, 2007. Marchionini, G. and Brunk, B. (2003). Towards a General Relation Browser: A GUI for Information Architects. Journal of Digital Information, 4(1). MicroSoft Live Search, available at http://www.live.com/. NeoRef Wiki, 2007, Available at http://bioivlab.ils.unc.edu/wiki/index.php/Main_Page. Pisano ED, Cole EB, Kistner EO, Muller KE, Hemminger BM, Brown ML, Johnston RE, Kuzmiak CM, Braeuning MP, Freimanis RI, Soo MS, Baker JA, Walsh R., “Interpretation of Digital Mammograms: A Comparison of Speed and Accuracy of Soft-Copy Versus Printed-Film Display. Radiology, 2002; 223: 483-488. SIGGRAPH demonstration project, VIRSEUM, hosted by 3rdTech, SIGGRAPH 2004.