1
Biomedical Engineering Key Content Survey - Results from Round One of a Delphi Study
David W. Gatchell and Robert A. LinsenmeierVaNTH ERC for Bioengineering Educational Technologies and Northwestern University
Whitaker Foundation Biomedical Engineering Educational SummitMarch, 2005
Supported by NSF EEC 9876363
2
Why conduct a BME key content survey? Motivation and potential benefits
Motivation Establish an identity for undergraduate Biomedical
Engineers Improve communication between academic BME programs
and industry Academia – Inform industry of the knowledge, skills and
training of BMEs Industry – Inform academia of the knowledge, skills and
training expected
Benefits More industrial positions for BMEs Each graduate does not have to explain curriculum Recognition that BME degree is ideal preparation for at least
some industrial positions.
3
In General: An iterative process for collecting knowledge from, and disseminating
results to, a group of experts Four steps (repeat steps #2 and #3 to attempt to reach consensus)
1. Develop a set of questions on a topic. 2. Experts give opinions on topics; suggest new ideas that were missed3. Explore and evaluate inconsistencies uncovered in step 24. Disseminate findings, or revise questions and go back to 2
Key point is that experts remain anonymous
Our Study: A set of three surveys Round 0: Select concepts from VaNTH taxonomies; reviewed by domain
experts Round 1: Survey BME industrial representatives and faculty. Asked
participants to rate relevance of concepts important for ALL undergrads in BME, and make suggestions of concepts missed
Round 2: Refine and update list of concepts and resubmit to the above groups for further evaluation
Round 3: Question proficiencies expected (e.g., using Bloom’s Taxonomy)
Delphi study - Overview
4
Utilized an online survey tool to query ~274 concepts: Eleven bioengineering domains (including design) Physiology, cellular biology, molecular biology and genetics,
biochemistry Mathematical modeling, statistics, general engineering skills (e.g.,
computer programming) Survey divided in two parts, each with half the domains:
Total number of participants, n = 136 Part one: Academia – 42, Industry – 25 Part two: Academia – 35, Industry – 23
Participants were asked to: Provide demographic information
Employer, Job Title, Responsibilities, Years of Experience Self-assess level of expertise in each domain (e.g., Biomechanics) Rate the importance/relevance of each concept to a BME core
curriculum Suggest concepts that had not been included
Overview of the key content survey, round one
5
Concepts rated on 5 point Likert Scale 1- very unimportant for all BMEs 5 – very important for all BMEs
Mean ratings across concepts similar for industry and academia Academia (n=77) mean and SD rating: 3.71 ± 0.52 Industry (n=48) mean and SD rating: 3.75 ± 0.41
Domains Investigated: Bioinformatics, bioinstrumentation, biomaterials, biomechanics, biooptics, biosignals and systems, medical
imaging, thermodynamics, transport (fluid, heat, mass) Cell biology, biochemistry, molecular biology and genetics, physiology Statistics, general engineering
Overview of the key content survey
6
Some concepts included as “Ringers” -Expected to have low rating
Concept Rating (Academia)
Rating (Industry)
Statistical Physics (e.g., Bose-Einstein statistics; Fermi-Dirac statistics)
2.32 2.58
Statistical Physics (e.g., Partition function; statistical representation of entropy;
population of states)
2.82 2.58
Comparative Genomics (e.g., ortholog and paralog genes; gene fusion events)
2.50 2.94
Dynamical Instability and Chaos 2.59 3.11
Unsteady state mass diffusion equation (e.g. Fick’s second law; production and
consumption; boundary conditions; different geometries; multiple layers)
3.42 3.29
All except unsteady state mass diffusion equation met expectations
7
Some concepts included in more than one domain to check consistency of response
Two values shown are ratings when concepts are included in different domains
Generally good agreement, but rating sometimes depended on context
Concept Ratings (Academia)
Ratings (Industry)
Databases - Interfaces and Structures (e.g., MySQL, relational tables, simple queries, PERL,
CGI, DBI)
2.29/2.66 3.22/3.68
Signal Processing to Reduce Noise (e.g., signal-to-noise ratio; signal averaging)
4.24/3.83 4.17/4.14
Properties of Systems (e.g., boundary, surroundings, universe)
3.88/3.97 3.63/3.88
Electrochemical Potential, Nernst Potential, Fick's Law
4.09/4.23 3.54/4.00
8
Concept RatingHypothesis Testing (e.g., paired and un-paired t-tests; chi-square test) 4.69
Principles of Statics (e.g., forces; moments; couples; torques; free-body diagrams)
4.68
Descriptive Statistics (e.g., mean, median, variance, std deviation) 4.63
Circuit Elements (e.g., resistors, capacitors, sources, diodes, transistors, integrated circuits)
4.56
DC and AC circuit analyses (e.g., Ohm's and Kirchoff's laws) 4.56
Mathematical Descriptions of Physical Systems (e.g., functional relationships, logarithmic, exponential, power-law; ODEs; PDEs)
4.54
Strength of Materials (e.g., stress, strain; models of material behavior) 4.53
Pressure-Flow Relations in Tubes and Networks (e.g., flow rate = [change in pressure]/resistance; Poiseiulle relation; Starling resistor) 4.51
Measurement concepts (e.g. accuracy, precision, … 4.50
Regression analysis 4.49
Forces and pressures in fluids (e.g. shear, normal, surface tension… 4.49
Results: Highest rated eng’g concepts – AcademiaOrange concepts are from statistics and general engineering
9
Results: Highest rated eng’g concepts – IndustryOrange concepts are from statistics and general engineering
Concept RatingDescriptive Statistics (e.g., mean, median, variance, standard
deviation)4.76
Measurement Concepts (e.g., accuracy, precision, sensitivity; error analysis - sources, propagation of error)
4.71
Hypothesis Testing (e.g., paired and un-paired t-tests; chi-squared) 4.65
Probability Distributions (e.g., normal, Poisson, binomial) 4.62
Strength of Materials (e.g., stress, strain; models of material behavior)4.57
Fundamental Properties of Polymers, Metals and Ceramics 4.50
Product Specification (e.g., requirements, design, reliability, evolution/tracking of the product)
4.45
Principles of Statics (e.g., forces; moments; couples; torques; free-body diagrams)
4.43
Mechanical Properties of Biological Tissues (e.g., elastic; viscoelastic, hysteresis, creep, stress relaxation)
4.43
Data Acquisition (e.g., sampling rates and analog-digital conversion; Nyquist criterion; aliasing) 4.39
10
Results: lowest rated concepts some from “Ringers”
Academia
Databases - Interfaces and Structures (e.g., MySQL, relational tables, simple queries, PERL, CGI, DBI)
2.29
Statistical Physics (e.g., Bose-Einstein and Fermi-Dirac statistics) 2.32
Artificial Intelligence (e.g., artificial neural networks, fuzzy logic) 2.33
Analysis of Phylogenetic Trees, Molecular Evolution 2.47
Comparative Genomics (e.g., ortholog and paralog genes; gene fusion events)
2.50
Structural Prediction and Molecular Design 2.53
Industry
Statistical Physics (e.g., Partition function; statistical representation of entropy; population of states)
2.58
Statistical Physics (e.g., Bose-Einstein; Fermi-Dirac statistics) 2.58
Artificial Intelligence (e.g., artificial neural networks, fuzzy logic) 2.78
Storage Instruments and their properties (e.g., tape, disk, memory) 2.89
Comparative Genomics 2.94
Root Locus Plots (e.g., definition, properties, sketching) 2.95
11
Results: Industry - Academia agreementDistribution of mean ratings of all concepts
2.00
3.00
4.00
5.00
2.00 2.50 3.00 3.50 4.00 4.50 5.00
Academia
Ind
us
try
Concept Ratings
Most concepts rated highly. Few ringers in survey. All traditional domains had some highly rated concepts. Cutoff level for inclusion in recommended undergrad curriculum
still to be determined on basis of further analysis and round two.
12
Results: Industry – Academia AgreementDifferences in means (A-I)
-1.25
-1.00
-0.75
-0.50
-0.25
0.00
0.25
0.50
0.75
0 50 100 150 200 250Concept #
Dif
fere
nce
s in
Mea
n R
esp
on
ses
Design
13
Results: Discrepancies in design concepts
Rankings - BME Design Concepts (A comparison of opinions from Academia and Industry)
1.00 2.00 3.00 4.00 5.00
Human Factors Issues/FDA
Computer-Aided Design Considerations
Risk Analysis/Hazard Analysis
Software and Process Design Considerations
Software for Design and Project Management (e.g.,flowcharting; Gannt and PERT charts)
Design for Manufacturing and Assembly
Decision Matrix Approaches to Initial Design
Mean Ranking (all participants)
Industry
Academia
14
Results: A comparison of general engineering concepts
1.00 2.00 3.00 4.00 5.00
Measurement Concepts (e.g., accuracy, precision, sensitivity;error analysis - sources, propagation of error)
Estimation and Order of Magnitude Calculations
Competency with (at least) One Programming Environment(e.g., Matlab, Mathematica, C, C++, FORTRAN)
Generalized Ohm's Law (i.e., driving force-flow-resistanceconcept)
Numerical Differentiation and Integration
Scaling and Dimensional Analysis
Familiarity with Multiple Computing Platforms (e.g., Windows,Macintosh, LINUX, UNIX)
Artificial Intelligence (e.g., artificial neural networks, fuzzylogic, etc.)
Databases - Interfaces and Structures (e.g., MySQL, relationaltables, simple queries, PERL, CGI, DBI)
Mean Rating (all participants)
Industry
Academia
Ringer
Significant Deltas
15
2.5
3.0
3.5
4.0
4.5
5.0
2.5 3.0 3.5 4.0 4.5 5.0
Rating of Concept - Academia
Ra
tin
g o
f C
on
ce
pt
- In
du
str
yBiochemistry
Cell Biology
Molecular Biol.
Bioinformatics
Unity slope
Results: Biology Domains
Good agreement on the whole
All biology areas important, but industry sees molecular biology as being more important than academia
Bioinformatics generally scored low, but industry feels that it is more important than academia does
16
Concepts Academia Industry Academia - Industry
Flow of Genetic Information (i.e., DNA to RNA to Protein)
4.5 4.1 0.44
Methods for Determining Macromolecular Structure (e.g., NMR...)
3.5 4.2 -0.70
DNA Microarrays 3.4 3.8 -0.42
Biological Networks (e.g., genetic networks...)
3.2 3.7 -0.46
Structural Prediction and Molecular Design (e.g., homology modeling and
prediction of macromolecular structures and interactions)
2.5 3.3 -0.72
Results: Largest biology discrepancies
17
2.5
3.0
3.5
4.0
4.5
5.0
2.5 3.0 3.5 4.0 4.5 5.0
Rating of Concept - Academia
Rat
ing
of
Co
nce
pt
- In
du
stry
Physiology
Unity slope
Results: Physiology (82 concepts)
Very large span within domain
Generally good agreement
Cardiovascular, neural, cellular physiology concepts rated highly
Digestive, renal, parts of endocrine rated low
18
Results: Largest physiology discrepancies between academia and
industryConcepts Academia Industry Academia
- Industry
Cellular Anatomy (e.g...) 4.56 4.14 0.41
Membrane Dynamics (e.g....) 4.44 3.95 0.49
Processes of the Kidney (e.g....) 4.26 3.85 0.41
Renal Filtration (e.g...) 4.03 3.60 0.43
Homeostasis of Volume and Osmolarity
4.03 3.50 0.53
Water Balance and Urine Concentration 3.73 3.25 0.48
Platelets and Coagulation (e.g....) 3.21 3.75 -0.54
Sodium Balance and the Regulation of ECF Volume (e.g....)
3.76 3.15 0.61
19
Results: Should the following foundational courses be required?
Comparison of Responses - Industry and Academia
Calculus - Differential, Integral and Multivariate
Vector Calculus
Linear Algebra
Ordinary Differential Equations
Chemistry - General
Chemistry - Organic (Semester One)
Chemistry - Organic (Semester Two)
Physics - Mechanics
Physics - Electricity and Magnetism
Physics - Waves and Optics
Industry
Academia
"NO" "YES""UNSURE"
Agreement that second semester organic chemistry is not universally required; some uncertainty about one semester
20
Universities represented in round one of the survey
1. Arizona State University*2. Binghampton University3. Boston University*4. Columbia University5. Devry Institute of Tech6. Duke University*7. Florida International University8. IIT9. Johns Hopkins University*10. Marquette University*11. Milwaukee SOE*12. MIT13. NJIT14. NC State University*15. Northwestern University*16. RPI*17. RHIT18. Stanford University19. Syracuse University*
20. SUNY – Stony Brook
21. Tulane University*
22. University of Akron*
23. University of Cincinnati
24. University of Illinois – UC*
25. University of Iowa*
26. University of Memphis
27. University of Michigan
28. University of Minnesota*
29. University of Pittsburgh*
30. University of Rochester*
31. University of Texas – Austin*
32. University of Toledo*
33. Vanderbilt University*
34. VCU*
*ABET Accredited – 21 of 37 Accredited Programs Participated
21
Companies and industrial expertise represented in round one of the survey
Companies Represented Abbott Laboratories AstraZeneca Baxter Healthcare Boston Scientific Cardiodynamics Cleveland Medical Devices Datasciences, International Dentigenix, Inc. Depuy, a Johnson and Johnson Co. ESTECH Least Invasive Cardiac
Surgery GE Healthcare Intel, Corp. Materialise, Inc. Medtronic, Inc. Tyco Healthcare Underwriter Laboratories
Areas of Expertise Biomaterials Biomechanics Bioinformatics Bioinstrumentation BioMEMS Biotransport Cellular Biomechanics Computational Modeling Control Systems Engineering Fluid Mechanics Medical Devices Medical Imaging Medical Optics Signal Processing
22
Conclusions More analysis is required to:
Investigate variation of opinions for individual topics Correlate ratings with expertise levels Eliminate contextual bias Incorporate concepts omitted from first round
BUT, preliminary results have shown that: “Consistency checks” validate data Generally good agreement between industry and academia Industry and academia disagree on a significant number of
Design concepts Industry highly values knowledge of statistics and
probability Core biology should include all domains assessed
23
Conclusions
Remaining issues Determine level of significance for deciding what concepts
can be dropped from core curriculum Determine significance of differences between industry
and academia Launch second round – by summer
Full matrix of results by concept will be posted on www.vanth.org/curriculum