outcomes based education: enve continuous improvement environmental engineering study program...
TRANSCRIPT
OUTCOMES BASED EDUCATION:
EnvE
CONTINUOUS IMPROVEMENT
Environmental Engineering Study ProgramFaculty of Civil and Environmental Engineering
Institut Teknologi Bandung
Benno Rahardyan
Goals • Faculty efficiency and efficacy• Improved student learning
• Knowledge • Skills • Attitudes • Behavior
• Improved learning environments• Accountability • Accreditation
3
*EnvE
The Process of Continuous Improvement*4
*EnvE
Continuous Improvement Action Process
a. Scientific Expertiseb. Experimental skill
c. Design Abilities
d. Teamwork
e. Problem Solving
f. Professional Responsibilityg. Communication Skill
h. Understanding of Engineering Impact
i. Long life Learning
j. Contemporary Issues
k. Technical Know-how
0.00
2.00
4.00
Course Assess-mentCourse Ques-tionnaireExit Survey
Outcomes Course Assessment
Course Questionnair
e
Exit Survey
Employer Average
a. Scientific Expertise 2.36 3.30 2.92 3.35 2.98b. Experimental skill 2.62 3.28 3.06 3.18 3.17c. Design Abilities 2.28 3.36 3.02 3.09 2.94d. Teamwork 2.49 2.36 2.92 2.84 2.71e. Problem Solving 2.54 3.73 2.99 3.22 3.31f. Professional Responsibility 2.86 3.38 2.99 3.18 3.18g. Communication Skill 2.93 3.54 3.22 3.04 3.27h. Understanding of Engineering Impact 2.73 2.46 2.84 3.02 2.77i. Long life Learning 2.85 2.47 3.27 3.50 3.08j. Contemporary Issues 2.35 2.73 2.96 2.88 2.86k. Technical Know-how 2.55 2.05 2.99 3.11 2.72
Continuous Improvement Action Process
Student Outcome Performance Indicator
1-under expectation 2-below average 3-meet
expectation4-exceed
expectationAn ability to apply knowledge of mathematics, science, and engineering. Proficient in mathematics through differential equations, probability and statistics, calculus-based physics, general chemistry, relevant to the program study (earth science, biological science, fluid mechanics)
1. Describe mathematical, physical, chemical and biological relations for given facts/occurences in environmental engineering.
Student can not describe and can not see any relation
Student can describe but can not see any relation
Student can describe and can see the relation
Student can describe, see the relation and extend the potential impact of problem
2. Recognize basic engineering problems, identify the data, and classify the related knowledge for solution
Student can not recognize, identity and classify the related knowledge of solution
Student can recognize, but can not identity or classify the related knowledge of solution
Student can recognize, identity and classify the related knowledge of solution
Student can recognize, identity, classify the related knowledge of solution and also mention update invention.
3. Demonstrate application of differential equation, statistic, probability, calculus-based physics, general chemistry and other relevant science to solve the problem
Student can not demonstrate solution of given problem
Student can only mention several solutions without solving the given problem
Student can demonstrate application to solve the given problem
Student can demonstrate application to solve the given problem and also explain the interpretation of solution.
a. Scientific Expertiseb. Experimental skill
c. Design Abilities
d. Teamwork
e. Problem Solving
f. Professional Responsibilityg. Communication Skill
h. Understanding of Engineering Impact
i. Long life Learning
j. Contemporary Issues
k. Technical Know-how
0.00
2.00
4.00
Course Assessment
Course Ques-tionnaire
Exit Survey
Employer
Average 1 2 3 4
Out
com
e A
(EXAM
PLE)
7
Student Outcome Assessment results Comparison of Outcomes Assessment Results
Comparison of Outcomes MeasurementsAttainment standard of 3.00
Area of Improvement
Area of Improvement
Area of Improvement
Student Outcome Assessment results Comparison of Outcomes Assessment Results
Comparison of Outcomes Measurement Results (2012 Data)
OutcomesCourse
Assessment
Course Questionnaire
Exit Survey Employer Average
a. Scientific Expertise 2.36 3.30 2.92 3.35 2.98b. Experimental skill 2.62 3.28 3.06 3.18 3.17c. Design Abilities 2.28 3.36 3.02 3.09 2.94d. Teamwork 2.49 2.36 2.92 2.84 2.71e. Problem Solving 2.54 3.73 2.99 3.22 3.31f. Professional Responsibility 2.86 3.38 2.99 3.18 3.18g. Communication Skill 2.93 3.54 3.22 3.04 3.27h. Understanding of Engineering Impact 2.73
2.46 2.84 3.02 2.77
i. Long life Learning 2.85 2.47 3.27 3.50 3.08j. Contemporary Issues 2.35 2.73 2.96 2.88 2.86k. Technical Know-how 2.55 2.05 2.99 3.11 2.72
Closing the loop
•What do the findings tell us now? •Did our “treatment” improve learning? •What else do the findings show? •What’s the next step? •What have we learned about our assessment process? Infrastructure? What can be improved?
10
ITB:
• Exit survey (Directorate of Education)• Alumni survey (Career Center)• User survey (Career Center)
11
Outcomes 2013/2014 2014/2015 2015/2016 2016/2017 2017/2018d. Teamwork * *
2018CurriculumPreparation
g. Communication Skill * *j. Contemporary Issues * *a. Scientific Expertise * *h. Understanding of Engineering Impact * *k. Technical Know-How * *f. Professional Responsibility * *i. Life-long Learning * *b. Experimental Skill * *c. Design Abilities * *e. Problem Solving * *
Continuous Improvement
Outcomes Assessment and Improvement Plan*
Note: spesific observationa) scientific expertise: TL2101 Fluid Mechanics I, TL2104 Engineering Mathematics, TL2201 Fluid Mechanics
II, TL2202 Environmental Chemistry, TL2203 Environmental Microbiologyb) experimental skill: TL3103 Environmental Laboratory c) design abilities: TL3105 Water Supply Engineering,TL3202 Drainage & Sewerage,TL4101 Water Treatment
Plant Design, TL4102 Wastewater Treatment Plant Designd) Teamwork: TL4201 Environmental Impact Assessment, TL4202 Project Planninge) Problem Solving: TL3101 Unit Operation, TL3102 Unit Processes, TL3203 Water Managementf) Professional Responsibility: TL4098 Field Workg) Communication Skill: TL4098 Field Work, TL4099 Seminar and Final Taskh) Understanding of Engineering Impact: TL2105 Environmental Health, TL2205 Environmental Epidemiology
TL4201 Environmental Impact Assessmenti) Long life Learning: TL4099 Seminar and Final Taskj) Contemporary Issues: TL3104 Solid Waste Management, TL3201 Air Pollution, TL4103 Air Pollution
Controlk) Technical Know-how: TL2103 Engineering Drawing , TL3103 Environmental Laboratory
*EnvE
Optimizing assessment of student work
Daina Briedis, 2013
13
OUTCOMES IMPROVEMENT PLAN
ABET
Acc
redi
tatio
n
Program Outcomes
Courses Improvement
a. Scientific Expertise Action: maintain Quiz Assignment Book preparation Reading textbook material up dating Laboratory improvement
b. Experimental skill
Action : maintain Field trip Guest lecturer Laboratory improvement
c. Design Abilities
Better assignment Collaborative Learning Course Benchmarking Course Relevance Improvement Curriculum Development Evaluation General Indicators Faculty Capacity Building Improvement Students Project Improving Scientific Expertise Maintaining Class Attendance Maintaining Learning Process
Performance Training
OUTCOMES IMPROVEMENT PLAN
ABET
Acc
redi
tatio
n
Program Outcomes Courses Improvementi. Long life Learningii.
Environmental Epidemiology Air Pollution Control Seminar and Final Task
Action : maintain Note taking Problem based learning type
assignment Collaborative learning type
assignment Student centered learning collaborative learning
j. Contemporary Issues
Engineering Drawing Environmental Health Solid waste Management Air Pollution Control
Action : maintain More case studies of problem Content enrinchment
k. Technical Know-how
Engineering Drawing Engineering Mathematics Unit Process Solid waste Management Water Treatment Plant Design Air Pollution Control Air Pollution Monitoring
Action : maintain Audio visual content Supporting equipment Laboratory improvement Availability of facility.
No Code Course Credit(s) Program Outcomes
a b c d e f g h i j k
1 TL2102 Environmental Statistics 2 2.58 2.34
2.22 2.22
2 TL2105 Environmental Health 3 2.01 2.16
3 TL2201 Fluid Mechanics II 3 2.02 2.02
4 TL2203 Environmental Microbiology 3 2.29 2.73 2.38
5 TL2204 Hydrology 3 2.39 2.39
6 TL2205 Environmental Epidemiology 2 2.35 2.60 2.56 2.40 2.45 2.67 2.87
7 TL3101 Unit Operation 3
1.26 1.26 2.42
8 TL3102 Unit Process 3 2.18 2.60 1.58 2.79 2.61
9 TL3103 Environmental Laboratory 3 3.81 3.19
2.26
2.17
10 TL3104 Solid Waste Management 3 2.41 2.09 2.59 1.02
1.91 2.84
2.89 3.19
11 TL3202 Drainage and Sewerage 3
2.27 2.51 2.45
12 TL3203 Water Management 3 2.55 2.74
2.13 2.13
1.98
13 TL3205 Solidwaste Treatment 2 2.71 2.71 3.81
2.84
2.35 2.71
14 TL4098 Field Work 2
3.88 3.55
15 TL4102 Wastewater Treatment Plant
Design 3
1.92 1.92 3.12
16 TL4104 Management for
Environmental Engineering 3
2.06
2.07
3.00
17 TL4099 Seminar and Final Task 5
3.64
3.64
18 TL4201 Environmental Impact
Assessment 3
3.20
2.31
2.31 2.52 4.00
19 TL4202 Project Planning 2 2.43
2.80
2.78
2.52
20 TL4105 Plumbing and Pump 3
2.14 1.98 2.57
21 TL4106 Infrastructure and Sanitation 2 2.95 3.09 3.06
3.11 2.81 3.96 3.23 3.23 3.30 2.74
22 TL4108 Technical Aspect of Waste
Recycling 2
3.74
2.74
2.26
23 TL4109 Air Pollution Monitoring 2 2.94 3.46
3.25
3.19
24 TL4117 Environmental Conservation
Technology 2
2.71
1.95
2.00
25 TL4205 Waterworks 2 3.50 3.50
3.50 3.50
3.50 3.50
3.50
26 TL4211 Environmental Ecotechnology 2 2.36
1.89
Average 2.59 2.77 2.41 2.22 2.73 2.86 3.05 2.78 3.31 2.54 2.86
a) scientific expertise: TL2101 Fluid Mechanics I, TL2104 Engineering Mathematics, TL2201 Fluid Mechanics II, TL2202 Environmental Chemistry, TL2203 Environmental Microbiology
b) experimental skill: TL3103 Environmental Laboratory c) design abilities: TL3105 Water Supply Engineering,TL3202 Drainage &
Sewerage,TL4101 Water Treatment Plant Design, TL4102 Wastewater Treatment Plant Design
d) Teamwork: TL4201 Environmental Impact Assessment, TL4202 Project Planninge) Problem Solving: TL3101 Unit Operation, TL3102 Unit Processes, TL3203 Water
Managementf) Professional Responsibility: TL4098 Field Workg) Communication Skill: TL4098 Field Work, TL4099 Seminar and Final Taskh) Understanding of Engineering Impact: TL2105 Environmental Health, TL2205
Environmental Epidemiology TL4201 Environmental Impact Assessmenti) Long life Learning: TL4099 Seminar and Final Taskj) Contemporary Issues: TL3104 Solid Waste Management, TL3201 Air Pollution,
TL4103 Air Pollution Controlk) Technical Know-how: TL2103 Engineering Drawing , TL3103 Environmental
Laboratory
EVALUATION
CU
RREN
T ST
ATU
S
POSS
IBLE
IMPR
OVE
MEN
T
Outcome-related Courses
Priority/RepresentingCourses
Course Assessment Plans for Improving Outcomes
Course Assessment
IMPROVEMENT
Course Portfolio
CURRENT STATUS POSSIBLE IMPROVEMENT
Lab developmentC.2 Learning Process Improvement
C2.1 Maintaining Class AttendanceC2.2 Maintaining Learning Process Performance
C.3 Academic Atmosphere ImprovementC.4 Course Relevance ImprovementC.5 Advising ImprovementC.6 Teaching Method Improvement-Improving Soft Skills
C.6.1 Faculty Capacity BuildingC.6.2 Implementation of teaching method improvementC.6.3 Improvement of Communication Skill
C.7 Improving Scientific ExpertiseC.7.1 International Guest Lecture SeriesC.7.2 Environmental Engineering Seminars Series
+
• Portofolio online
• Metode Perkuliahan• Metode Evaluasi• Sistem Penilaian• Uraian terhadap Hasil Kuesioner
dan Statistik Kelas• Refleksi Pelaksanaan Perkuliahan
Rencana Tindak Lanjut
19
Continuous Improvement
Weakness Evaluation and Improvement action
20
Courses
Weakness andImprovement action
Continuous Improvement
The Result of Weakness Evaluation Related to Outcomes a) through d)
Outcome Weakness Evaluation (Improvement plan)
a. Scientific Expertise
Generally, students understanding about course material is less than 75%
Students still have difficulty to obtain literatures relevant with the course
The class room facilities cannot optimally support the learning atmosphere.
Method of teaching, improved supporting
facilities,
b. Experimental skill
Lack of modern laboratory equipment and very limited in number
Some of the laboratories are not clean and tidy enough
Students are less accustomed to perform experiments independently
Maintained
c. Design Abilities
The opportunity to practice about good design is still limited
Students are lack of knowledge about the design process
Maintained
d. Teamwork The students have not been able to communicate effectively
Increased opportunities in rehearse of team work, collaborative learning 21
Continuous Improvement
The Result of Weakness Evaluation Related to Outcomes e) through k)
Outcome Weakness Evaluation (Improvement plan)
e. Problem SolvingThe students do not understand the learning material integrally
Maintained
f. Professional ResponsibilityStudents having lack knowledge about environmental engineering professionalism
Maintained
g. Communication SkillStudents are not used to state their opinion well
Increased opportunities in group work presentations, collaborative learning
h. Understanding of Engineering Impact
Students’ knowledges on environmental engineering application are limited
Introduction of more Guest lectures in the courses, site visits
i. Long life Learning Maintained
j. Contemporary Issues
Students awareness about environmental related issues are low
Students too much focusing on learning environmental engineering academically
Guest lectures and site visits
k. Technical Know-how
Students’ knowledge and understanding about the engineering tools usage are low
Students do not understand the engineering tools application
More workshop sessions
22
Continuous Improvement
Activities of Continuous ImprovementLab developmentC.2 Learning Process ImprovementC2.1 Maintaining Class AttendanceC2.2 Maintaining Learning Process Performance
C.3 Academic Atmosphere ImprovementC.4 Course Relevance ImprovementC.5 Advising ImprovementC.6 Teaching Method Improvement-Improving Soft Skills
C.6.1 Faculty Capacity BuildingC.6.2 Implementation of teaching method improvementC.6.3 Improvement of Communication Skill
C.7 Improving Scientific ExpertiseC.7.1 International Guest Lecture SeriesC.7.2 Environmental Engineering Seminars Series
23
Continuous Improvement
Activities of Continuous ImprovementC.8 Improving Contemporary Issues and Access to Modern Tools
C.8.1 Invited SpeakersC.8.2 Field TripsC.8.3. Field Work Student ActivityC.8.4TrainingC.8.5 Laboratories Visit
C.9 Students CompetitionC.9.1. Internal Student CompetitionC.9.2. External Student Competition
C.10 Improvement of Evaluation MethodC.10.1 Improvement of Course PortfolioC.10.2 Course BenchmarkingC.10.3 Evaluation General Indicators
C.11Curriculum DevelopmentC.12 Improvement Students Project
24
Continuous Improvement
Activities of Continuous ImprovementLab developmentC.2 Learning Process ImprovementC2.1 Maintaining Class AttendanceC2.2 Maintaining Learning Process Performance
C.3 Academic Atmosphere ImprovementC.4 Course Relevance ImprovementC.5 Advising ImprovementC.6 Teaching Method Improvement-Improving Soft Skills
C.6.1 Faculty Capacity BuildingC.6.2 Implementation of teaching method improvementC.6.3 Improvement of Communication Skill
C.7 Improving Scientific ExpertiseC.7.1 International Guest Lecture SeriesC.7.2 Environmental Engineering Seminars Series
Course Z
Prev Year
Year-0
Year-1 ...
√ √ √ √ √ √
- √ √
- √ √
√ √ √ √ √ √
√ √ √ √ √ √
√ √ √ √ √ √
EXAMPLE
Continuous Improvement
Activities of Continuous ImprovementC.8 Improving Contemporary Issues and Access to Modern Tools
C.8.1 Invited SpeakersC.8.2 Field TripsC.8.3. Field Work Student ActivityC.8.4TrainingC.8.5 Laboratories Visit
C.9 Students CompetitionC.9.1. Internal Student CompetitionC.9.2. External Student Competition
C.10 Improvement of Evaluation MethodC.10.1 Improvement of Course PortfolioC.10.2 Course BenchmarkingC.10.3 Evaluation General Indicators
C.11Curriculum DevelopmentC.12 Improvement Students Project
Course Z
Prev Year
Year-0
Year-1 ...
√ √ √ √ √ √ - - √
- √ √
- √ √
Process-oriented Improvement
Improvement of Communication Skill
Teaching Method Improvement-Improving Soft SkillsContinuous Improvement
2007 2010 2011 2012
510
540
572
555
English Proficiency Score (predicting TOEFL score)
Year Writing skills Presentation skills Moderator Questions
Paper Summary Slide PosterOral
presentation-2005 Indonesian - Ind - Indonesian Indonesian Indonesian2005 Indonesian - Eng. - Indonesian Indonesian Indonesian2006 Indonesian Ind.&Eng Eng. Eng (optional) Indonesian Indonesian2007 Ind.&Eng Ind.&Eng Eng. Eng. Eng (optional) Eng (optional) Eng (optional)2008 Ind.&Eng Ind.&Eng Eng. Eng. Eng (obligatory) Eng (optional) Eng (optional)2009 Ind.&Eng Ind.&Eng Eng. Eng. Eng (obligatory) Eng (obligatory) Eng (optional)2010 Ind.&Eng Ind.&Eng Eng. Eng. Eng (obligatory) Eng (obligatory) Eng (obligatory)
Improvement in Final Project Seminar
Result/Output-oriented Improvement
EXAMPLE OUTCOME G
Recommendation summary
• General improvement scheme• Feedback information delivery• General continuous improvement schedule• Improvement online portfolio • Program development• General improvement action
28