economics - from theory to practice: field experimental ......from theory to practice: field...
TRANSCRIPT
From Theory to Practice: From Theory to Practice: From Theory to Practice: From Theory to Practice: Field Experimental Evidence on Early Exposure of Engineering Majors to Professional Work
Kevin Boudreau Kevin Boudreau Kevin Boudreau Kevin Boudreau (Northeastern & NBER)
& Matt Marx Matt Marx Matt Marx Matt Marx (Boston University)
Sept 2019
www.kevinboudreau.com
OverviewOverviewOverviewOverview
• Importance of STEM worker supplyImportance of STEM worker supplyImportance of STEM worker supplyImportance of STEM worker supply
• Predominant institutional approach: Predominant institutional approach: Predominant institutional approach: Predominant institutional approach: Higher Ed → professional work
• Possible tradeoffs—theory vs. practice?
• QuestionQuestionQuestionQuestion: Effect of : Effect of : Effect of : Effect of timing of first exposuretiming of first exposuretiming of first exposuretiming of first exposure to to to to professionalprofessionalprofessionalprofessional labor market + work experience?labor market + work experience?labor market + work experience?labor market + work experience?
• Research DesignResearch DesignResearch DesignResearch Design
• 1,787 Engineering majors, top-40 program, 6-month work terms
• Randomly assigned to earlier or later exposure
• Observe array of academicacademicacademicacademic + professionalprofessionalprofessionalprofessionaloutcomes
www.kevinboudreau.com
Overview of Effects of Overview of Effects of Overview of Effects of Overview of Effects of Earlier ExposureEarlier ExposureEarlier ExposureEarlier Exposure
1.1.1.1. Academic performance/choicesAcademic performance/choicesAcademic performance/choicesAcademic performance/choices
• LowerLowerLowerLower----incomeincomeincomeincome students get higher grades
• EveryoneEveryoneEveryoneEveryone switches to higher-paying majorsmore + more Engineering courses
• Weakest students Weakest students Weakest students Weakest students (slightly) lower grades (weak)
2.2.2.2. PostPostPostPost----graduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomes
• Stronger students Stronger students Stronger students Stronger students more likely to stay in Engineering (grad school)
• LowLowLowLow----income studeincome studeincome studeincome students get better jobs (wages +larger firms)
Interpretation:Interpretation:Interpretation:Interpretation:
• Greater exposure to field and income for lowlowlowlow----incomeincomeincomeincome
• Greater sorting, matching sorting, matching sorting, matching sorting, matching (majors, grad school)
• Little evidence of costly effects, in this context
www.kevinboudreau.com
AgendaAgendaAgendaAgenda
• Motivation & Background
• Research Design
• Results
• Discussion
www.kevinboudreau.com
STEM PipelineSTEM PipelineSTEM PipelineSTEM Pipeline• STEM workforce supply: employment, wages, firm founding, national STEM workforce supply: employment, wages, firm founding, national STEM workforce supply: employment, wages, firm founding, national STEM workforce supply: employment, wages, firm founding, national competitivenesscompetitivenesscompetitivenesscompetitiveness (Arrow and Capron, 1959; Murphy, et al. 1991; Rosenberg and Nelson, 1994; Pritchett 1997, Romer,
2000; Kerr, 2013; Comin & Ferrer 2013; Caicedo 2016)
• Esp. application of science to practical problem-solving
• Debate “Leaky Pipeline” ”Crisis” Debate “Leaky Pipeline” ”Crisis” Debate “Leaky Pipeline” ”Crisis” Debate “Leaky Pipeline” ”Crisis” (Atkinson, 2000; National Academy of Engineering, 2005; Freeman, 2006;
National Research Council, 2011; White House Press Release 2015, 2017, Economic Policy Institute, RAND, etc.)
• US: 14 % undergrads choose STEM; (G7 = 23% China = 39%)
• High attrition 1/3 work in STEM (½ new grads)
• Natives fill about ½ technical jobs in U.S. (Fayer/et.al’17)
• “At a time of high unemployment rates, we have 2MM high-skilled, high-wage jobs are unfilled” Arne Duncan, Secretary of Education
• Many STEM initiatives : interest/access/pedagogyMany STEM initiatives : interest/access/pedagogyMany STEM initiatives : interest/access/pedagogyMany STEM initiatives : interest/access/pedagogy
• Secondary, higher ed, employment “diversity”
�Here, we consider the Here, we consider the Here, we consider the Here, we consider the transitiontransitiontransitiontransition between between between between theory theory theory theory & & & & practicepracticepracticepractice
Motivation & Background
www.kevinboudreau.com
Tension Between Theory & PracticeTension Between Theory & PracticeTension Between Theory & PracticeTension Between Theory & Practice• Application of Application of Application of Application of scientific principles to practical problemscientific principles to practical problemscientific principles to practical problemscientific principles to practical problem----solvingsolvingsolvingsolving (Grayson, 1979)(Grayson, 1979)(Grayson, 1979)(Grayson, 1979)
• 18th Century: France 18th Century: France 18th Century: France 18th Century: France theoretical (incl. military) vs. UKUKUKUK apprenticeships, lyceums
• Early U.S. Early U.S. Early U.S. Early U.S. apprentice = practicing engineer
• PostPostPostPost----1812 expansion 1812 expansion 1812 expansion 1812 expansion of transport networks, mechanization, agriculture production
• Apprenticeship system cannot meet demand
• Dozen+ antebellum colleges Dozen+ antebellum colleges Dozen+ antebellum colleges Dozen+ antebellum colleges introduce some form of "practical" education(Engineering News, 1892; Mann, 1918; Caullery 1922; Cheit 1975; Grayson 1977)
• Military, Classic colleges, PolytechnicMilitary, Classic colleges, PolytechnicMilitary, Classic colleges, PolytechnicMilitary, Classic colleges, Polytechnic
• Morrill Act in 1862 landMorrill Act in 1862 landMorrill Act in 1862 landMorrill Act in 1862 land----grant colleges grant colleges grant colleges grant colleges (Kerr, 1987)... 85 Engineering programs by 1880s
• “To teach such branches of learning as are related to agriculture and the mechanic arts”
• Second industrial revolution and continuing advances...Second industrial revolution and continuing advances...Second industrial revolution and continuing advances...Second industrial revolution and continuing advances...
• Gradual evolution to formal 4Gradual evolution to formal 4Gradual evolution to formal 4Gradual evolution to formal 4----year degree programsyear degree programsyear degree programsyear degree programs
• Subset of “experiential education” programsSubset of “experiential education” programsSubset of “experiential education” programsSubset of “experiential education” programs: University of Akron, Carnegie Mellon University, the University of Cincinnati, Drexel University, Georgia Institute of Technology, Kettering University, Northeastern University, and Rochester Institute of Technology, .... grows to 100s
Motivation & Background
www.kevinboudreau.com
Tension Between Theory & PracticeTension Between Theory & PracticeTension Between Theory & PracticeTension Between Theory & Practice
• By 1980s, highly theoretical/mathematical degreeBy 1980s, highly theoretical/mathematical degreeBy 1980s, highly theoretical/mathematical degreeBy 1980s, highly theoretical/mathematical degree
• (Some) backlash.... (Some) backlash.... (Some) backlash.... (Some) backlash.... lab courses, term projects, prototypes
• Continuing advance & specialization
• Today, continuing 4Today, continuing 4Today, continuing 4Today, continuing 4----year degree predominates, year degree predominates, year degree predominates, year degree predominates, (largely) theory-then-practice...
• Also practical curriculum, part-time & summer jobs
• Are new grads ready to work? (Casner and Barrington, 2006; Association of American Colleges and Universities)
• Experiential programs identical in coursework, but work requirement
�... Inherent tension ... Inherent tension ... Inherent tension ... Inherent tension –––– manifest in institutionsmanifest in institutionsmanifest in institutionsmanifest in institutions
�TTTTradeoffs to theory-then-practice institutional design?
�Effect of earlier entry to professional labor marketearlier entry to professional labor marketearlier entry to professional labor marketearlier entry to professional labor market?
i.e., Time-shifting substantial full-time, professional work experience as a junior Engineer in U.S. to earlier time period
(cf. Griliches, 1981; Light, 1995; 2001; Acevedo, et al. 2017; Ashworth et al. 2017; Alfonsi, et al. 2017; Adhvaryu and Nyshadham 2018; Le Barbanchon, et al. 2019; and Lindenmeyer, 1967; Felder, et al., 1988; Crawley et al., 2007; Talgar et al., 2017; Blair et al. 2004; Van Gyn, 1997)
Motivation & Background
www.kevinboudreau.com
(Many) Possible Effects of Early Exposure(Many) Possible Effects of Early Exposure(Many) Possible Effects of Early Exposure(Many) Possible Effects of Early Exposure
• Positive, e.g.,Positive, e.g.,Positive, e.g.,Positive, e.g.,
• Bridge theory + applications + heuristic knowledge – learning, interest, understanding, motivation
• Learn professional norms, culture
• Direct feedback about career fit, prospects (Froyed et al., 2012; Manski, 1993; Arcidiacono, et al. 2016; Bleemer and Zafar, 2018; Conlon, 2019; Fricke, Grogger and Steinmayr, 2019)
• Income
• Professional networks
• Negative, e.g.,Negative, e.g.,Negative, e.g.,Negative, e.g.,
• Prolong degree + start with less training?
• Stress, hassle, distraction?
• Transient “Skills” rather than “knowledge” (Deming & Noray, 2019)
�Many possible effects across academic & later career outcomes
www.kevinboudreau.com
Research Design: Effects Research Design: Effects Research Design: Effects Research Design: Effects of Early Exposure?of Early Exposure?of Early Exposure?of Early Exposure?
• Ideal design:Ideal design:Ideal design:Ideal design:
• (i) identical groups of students…
• (ii) identical academic program…
• (iii) identical opportunities
…some with earlier exposure
• WithinWithinWithinWithin----institutioninstitutioninstitutioninstitution experimental design…experimental design…experimental design…experimental design…
• Many possible effects � observablesobservablesobservablesobservables
www.kevinboudreau.com
ContextContextContextContext
• Top 40, private U.S. engineering program
• ∼400 per year
• Mechanical, Civil, Chemical, Electrical, Computer, Biological
• Average SAT 719 math / 666 verbal, HS GPA 4.01
• 84% receive financial aid (avg = $17500)
• 23.4% women, 9.3% non-U.S.
• Also HS name/cohort, home address, dorm room, visa/citizenship, courses taken, semester GPA
• Data on 1,787 from 2009-2013 matriculants, over time
• Registrar, applications, work term admin, hand-collected Linkedin, D&B
www.kevinboudreau.com
Work TermsWork TermsWork TermsWork Terms
• At least one six-month (junior engineering) professional work term required (avg. 2.7)
• University-administrated labor market
• On-going stable relationships between university & firms
• ”Curated” relatively uniformly high-quality positions
• $19/hour wage
• 96% in engineering: GE, Amazon Robotics, Bose, etc.
• Junior Engineering responsibilities
www.kevinboudreau.com
Separation of Intake to 2 GroupsSeparation of Intake to 2 GroupsSeparation of Intake to 2 GroupsSeparation of Intake to 2 Groups
Year 1 Year 2 Year 3 Year 4 Year 5
Fall Spring Fall Spring Fall Spring Fall Spring Fall Spring
Early 1 2 3 Work 4 Work 5 Work 6 7
Late 1 2 3 4 Work 5 Work 6 Work 7
• Helps balance or smooth : supply/demand, teaching, operations
• Incremental difference; split students
• Students aware, but not viewed as consequential
www.kevinboudreau.com
Random AssignmentRandom AssignmentRandom AssignmentRandom Assignment
Year 1 Year 2 Year 3 Year 4 Year 5
Fall Spring Fall Spring Fall Spring Fall Spring Fall Spring
Early 1 2 3 Work 4 Work 5 Work 6 7
Late 1 2 3 4 Work 5 Work 6 Work 7
Declare major
within engineering
Informed of early/late assignment
Balance
checked
www.kevinboudreau.com
μ(early) μ(late) s.e. p <
Demographics
Age at matriculation 18.343 18.338 0.033 0.881
Female 0.233 0.212 0.018 0.221
International 0.060 0.085 0.011 0.023
Financial aid award $17,125 $17,440 $491 0.520
Academic performance (as of 3rd semester)
High-school GPA 3.997 3.997 0.015 0.993
Percentile on standardized tests 0.864 0.865 0.003 0.930
Cumulative GPA 3.364 3.371 0.019 0.734
Academic preferences (as of 3rd semester)
Major initially undeclared 0.306 0.308 0.019 0.928
# credits completed 53.215 53.291 0.140 0.592
Avg. # students in courses taken 53.724 54.289 0.645 0.381
% courses in Engineering 0.310 0.293 0.005 0.000
% courses w/ tenure-track instructors 0.809 0.788 0.008 0.007
Initial work term
Work term in U.S. 0.972 0.976 0.007 0.626
Hourly wage 18.790 18.691 0.254 0.698
Engineering job 0.956 0.967 0.006 0.033
Courses offered
Number of courses offered 1038.1 1084.3 23.539 0.070
% engineering courses offered 0.148 0.164 12.699 0.077
% courses taught by tenure-track professors 0.815 0.803 108.813 0.829
Enrollment per class 23.458 23.870 0.401 0.322
BalanceBalanceBalanceBalance
www.kevinboudreau.com
Analysis & ResultsAnalysis & ResultsAnalysis & ResultsAnalysis & Results
1.1.1.1. Academic performance/choicesAcademic performance/choicesAcademic performance/choicesAcademic performance/choices
• Academic performance
• Choice of majors
• Choice of classes
2.2.2.2. PostPostPostPost----graduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomes
• Choice of continuing in Engineering (work or grad school)
• Wages
• EstimatesEstimatesEstimatesEstimates::::
• Simple, x-sect OLS regressions of outcomes on early exposure treatment
• ... and covariates
• Academic: Major, cohort
• Demographic: Income, gender, in-state, age, foreign
• ...interactions
www.kevinboudreau.com
Analysis & ResultsAnalysis & ResultsAnalysis & ResultsAnalysis & Results
1.1.1.1. Academic performance/choicesAcademic performance/choicesAcademic performance/choicesAcademic performance/choices
• LowerLowerLowerLower----incomeincomeincomeincome students get higher grades
• EveryoneEveryoneEveryoneEveryone switches to higher-paying majors more + more Engineering courses
• Weakest students Weakest students Weakest students Weakest students (slightly) lower grades (weak)
2.2.2.2. PostPostPostPost----graduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomes
Dep. Var.:
Model: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)
Treatment Effects
Early Job Treatment -0.02 -0.24* -0.27* -0.25* -0.26* -0.24* -0.27* -0.27* -0.23 -0.84
(0.02) (0.14) (0.14) (0.14) (0.14) (0.14) (0.14) (0.14) (0.14) (0.85)
Early Job × GPA 0.07* 0.07* 0.07* 0.07* 0.07* 0.07* 0.07* 0.06 0.46
(0.04) (0.04) (0.04) (0.04) (0.04) (0.04) (0.04) (0.04) (0.51)
Early Job × GPA^2 -0.06
(0.08)
Early Job × Low Income 0.09* 0.09* 0.10**
(0.05) (0.05) (0.05)
Early Job × Female 0.02 0.02 0.02
(0.03) (0.03) (0.03)
Early Job × Foreign -0.12* -0.12* -0.11*
(0.07) (0.07) (0.07)
Early Job × In-State 0.03 0.02 0.02
(0.03) (0.03) (0.03)
Early Job × Older 0.00 0.01 0.01
(0.03) (0.03) (0.03)
Academic Characteristics
Pre-Treatment GPA 0.73*** 0.69*** 0.69*** 0.70*** 0.70*** 0.70*** 0.69*** 0.69*** 0.70*** -0.30
(0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.03) (0.02) (0.03) (0.30)
Pre-Treatment GPA^2 0.16***
(0.05)
Major & Cohort FEs Y Y Y Y Y Y Y Y
Demographic Characteristics
Low Income 0.02 -0.04 0.02 0.02 0.02 0.02 -0.04 -0.05
(0.03) (0.04) (0.03) (0.03) (0.03) (0.03) (0.04) (0.03)
Female 0.06*** 0.06*** 0.05** 0.06*** 0.06*** 0.06*** 0.05** 0.05**
(0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.02)
Foreign -0.08** -0.08** -0.08** -0.02 -0.09** -0.08** -0.03 -0.03
(0.03) (0.03) (0.03) (0.04) (0.03) (0.03) (0.04) (0.04)
In-State -0.00 -0.00 -0.00 -0.00 -0.02 -0.00 -0.01 -0.02
(0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.02)
Older -0.02 -0.02 -0.02 -0.02 -0.02 -0.02 -0.02 -0.02
(0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.02) (0.02)
Adjusted R-squared 0.50 0.50 0.52 0.52 0.52 0.52 0.52 0.52 0.52 0.52
Post-Treatment Grade Point Average (GPA)
www.kevinboudreau.com
• (Not much) evidence that early exposure has negative effect on weaker student’s GPAs
• LowLowLowLow----income students with early work income students with early work income students with early work income students with early work (highest 10% of financial aid)
• ...controlling for pre-treatment GPA
• ...increase GPA by 0.1 points ...increase GPA by 0.1 points ...increase GPA by 0.1 points ...increase GPA by 0.1 points more, out of 4-point scale
• s.e. = 0.05, when conditioning exhaustively on other variables
• InterpretationInterpretationInterpretationInterpretation
• WeakerWeakerWeakerWeaker students (slightly) more stressed
• LowLowLowLow----incomeincomeincomeincome
• Exposure Effect?Exposure Effect?Exposure Effect?Exposure Effect? Something they learn or acquire while working?(know-how, motivation, networks, career knowledge)
• Financial Effect?Financial Effect?Financial Effect?Financial Effect? Benefit from less (financial) stress or distraction
Post-Treatment Major Computer Chemical Civil
Electrical-
Computer Electrical Mechanical Total:
Computer 163 2 0 40 20 11 236
Chemical 0 303 0 0 0 2 305
Civil 0 2 266 0 1 0 269
Electrical-Computer 10 0 0 61 19 0 90
Electrical 6 3 2 17 134 3 165
Industrial 0 3 2 0 3 5 13
Mechanical 3 2 6 0 2 657 670
Outside Engineering 3 8 7 3 5 13 39
Total: 185 323 283 121 184 691 1,787
Pre-Treatment Major
Switching Majors
Dep. Var.:
Model: (1) (2) (3) (4) (5) (6) (7) (8)
Treatment Effects
Early Job Treatment 0.04*** 0.03* 0.02** 0.03* 0.00 -0.01 0.01 0.01
(0.01) (0.02) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01)
Early Job × Low Income -0.03 0.00 -0.05 -0.01
(0.07) (0.06) (0.04) (0.03)
Early Job × Female 0.02 -0.02 0.03 0.02
(0.03) (0.02) (0.02) (0.02)
Early Job × Foreign -0.01 -0.01 0.01 -0.00
(0.06) (0.04) (0.03) (0.02)
Early Job × In-State -0.01 -0.00 -0.00 -0.01
(0.03) (0.02) (0.02) (0.01)
Early Job × Older 0.01 -0.00 0.01 0.00
(0.03) (0.02) (0.02) (0.01)
Academic Characteristics
Pre-Treatment GPA -0.04** -0.07*** 0.01 -0.01 -0.01 -0.02 -0.04*** -0.04***
(0.02) (0.02) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01)
Major & Cohort FEs Y Y Y Y
Demographic Characteristics
Low Income 0.08 0.04 0.03 0.02
(0.05) (0.05) (0.04) (0.03)
Female -0.00 0.01 0.00 -0.01*
(0.02) (0.02) (0.01) (0.01)
Foreign -0.01 -0.00 -0.01 -0.01
(0.04) (0.03) (0.02) (0.02)
In-State Student 0.02 0.02 0.00 -0.00
(0.02) (0.02) (0.01) (0.01)
Older -0.01 0.01 0.00 -0.01
(0.02) (0.02) (0.01) (0.01)
Adjusted R-squared 0.01 0.18 0.00 0.15 -0.00 0.05 0.01 0.01
Switch Major
Higher-Wage Eng
Subfield Outside Eng
Lower-Wage Eng
Subfield
www.kevinboudreau.com
• Evidence that students with early workEvidence that students with early workEvidence that students with early workEvidence that students with early work
• ...switch 4 percentage points more often ...switch 4 percentage points more often ...switch 4 percentage points more often ...switch 4 percentage points more often (s.e. = 1)
• Only significant effect comes from switching to higher wage fieldsOnly significant effect comes from switching to higher wage fieldsOnly significant effect comes from switching to higher wage fieldsOnly significant effect comes from switching to higher wage fields
• Effect similar across subgroupsEffect similar across subgroupsEffect similar across subgroupsEffect similar across subgroups
• InterpretationInterpretationInterpretationInterpretation
• Learning about the work, the market, self, match
Dep. Var.:
Model: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)
Treatment Effects
Early Job Treatment 0.02*** -0.01 0.02*** 0.02*** 0.03*** 0.02*** 0.02*** 0.02*** 0.01* 0.28
(0.01) (0.05) (0.00) (0.00) (0.01) (0.01) (0.01) (0.01) (0.01) (0.23)
Early Job × GPA 0.01 -0.19
(0.01) (0.14)
Early Job × GPA^2 0.03
(0.02)
Early Job × Low Income 0.02 0.02 0.02
(0.03) (0.03) (0.03)
Early Job × Female -0.00 -0.00 -0.01
(0.01) (0.01) (0.01)
Early Job × Foreign 0.03 0.03 0.04*
(0.02) (0.02) (0.02)
Early Job × In-State 0.01 0.01 0.01
(0.01) (0.01) (0.01)
Early Job × Older 0.02 0.01 0.01
(0.01) (0.01) (0.01)
Academic Characteristics
Pre-Treatment GPA 0.00 -0.00 0.02*** 0.02*** 0.02*** 0.02*** 0.02*** 0.02*** 0.02*** 0.14
(0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.09)
Pre-Treatment GPA^2 -0.02
(0.01)
Major & Cohort FEs Y Y Y Y Y Y Y Y
Demographic Characteristics
Low Income -0.02 -0.03 -0.02 -0.02 -0.02 -0.02 -0.03 -0.03
(0.01) (0.02) (0.01) (0.01) (0.01) (0.01) (0.02) (0.02)
Female -0.00 -0.00 0.00 -0.00 -0.00 -0.00 0.00 0.00
(0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01)
Foreign 0.02** 0.02** 0.02** 0.01 0.02** 0.02** 0.01 0.01
(0.01) (0.01) (0.01) (0.02) (0.01) (0.01) (0.02) (0.02)
In-State -0.00 -0.00 -0.00 -0.00 -0.00 0.00 -0.01 -0.01
(0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01)
Older 0.02*** 0.02*** 0.02*** 0.02*** 0.02*** 0.01 0.01 0.01
(0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01) (0.01)
Adjusted R-squared 0.00 0.00 0.43 0.43 0.43 0.43 0.43 0.43 0.43 0.43
Post-Treatment Percent Courses in Engineering
www.kevinboudreau.com
• Evidence that students with early workEvidence that students with early workEvidence that students with early workEvidence that students with early work
• .... take 2 percent more Engineering courses (+ more tenure track).... take 2 percent more Engineering courses (+ more tenure track).... take 2 percent more Engineering courses (+ more tenure track).... take 2 percent more Engineering courses (+ more tenure track)
• Effect similar across subgroupsEffect similar across subgroupsEffect similar across subgroupsEffect similar across subgroups
• InterpretationInterpretationInterpretationInterpretation
• Greater engagement, interest
• Related to greater sorting/matching?
www.kevinboudreau.com
Analysis & ResultsAnalysis & ResultsAnalysis & ResultsAnalysis & Results
1.1.1.1. Academic performance/choicesAcademic performance/choicesAcademic performance/choicesAcademic performance/choices
2.2.2.2. PostPostPostPost----graduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomes
• Stronger students Stronger students Stronger students Stronger students more likely to stay in Engineering (grad school)
• LowLowLowLow----income studeincome studeincome studeincome students get better jobs (wages +larger firms)
Dep. Var.:
Model: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)
Treatment Effects
Early Job Treatment 0.01 -0.32*** 1.40*** -0.01 -0.01 0.14 -0.01 0.01 0.65 0.02 0.04 -2.10**
(0.01) (0.09) (0.44) (0.01) (0.01) (0.28) (0.03) (0.04) (1.17) (0.03) (0.04) (1.00)
Early Job × GPA 0.10*** -1.00*** -0.08 -0.43 1.34**
(0.03) (0.29) (0.17) (0.73) (0.63)
Early Job × GPA^2 0.17*** 0.01 0.07 -0.21**
(0.05) (0.03) (0.11) (0.10)
Early Job × Low Income -0.00 -0.02 -0.03 -0.03 0.03 0.01 -0.07 -0.06
(0.05) (0.05) (0.03) (0.03) (0.15) (0.15) (0.10) (0.10)
Early Job × Female -0.01 -0.01 0.02 0.02 0.04 0.03 -0.07 -0.07
(0.03) (0.03) (0.02) (0.02) (0.07) (0.07) (0.06) (0.06)
Early Job × Foreign 0.09 0.10 -0.04 -0.04 0.08 0.08 -0.06 -0.06
(0.09) (0.09) (0.02) (0.02) (0.15) (0.15) (0.11) (0.11)
Early Job × In-State -0.00 -0.00 -0.02 -0.02 -0.00 -0.00 -0.03 -0.03
(0.03) (0.03) (0.02) (0.02) (0.06) (0.06) (0.05) (0.05)
Early Job × Older -0.00 -0.00 0.03** 0.03** -0.07 -0.07 -0.02 -0.02
(0.03) (0.03) (0.01) (0.02) (0.06) (0.06) (0.05) (0.05)
Academic Characteristics
Pre-Treatment GPA 0.05*** 0.01 0.35* 0.01 0.01 0.09 0.01 0.03 0.68 -0.04 -0.11*** -1.09**
(0.01) (0.02) (0.20) (0.01) (0.01) (0.11) (0.03) (0.03) (0.55) (0.03) (0.03) (0.46)
Pre-Treatment GPA^2 -0.05* -0.01 -0.10 0.15**
(0.03) (0.02) (0.09) (0.07)
Major & Cohort FEs Y Y Y Y Y Y Y Y
Demographic Characteristics
Low Income 0.00 0.01 0.02 0.02 0.01 0.03 0.10 0.08
(0.04) (0.04) (0.03) (0.03) (0.12) (0.12) (0.08) (0.08)
Female 0.01 0.01 -0.01 -0.01 -0.10* -0.09* 0.08* 0.08*
(0.02) (0.02) (0.01) (0.01) (0.05) (0.05) (0.05) (0.05)
Foreign 0.10** 0.10** 0.01 0.01 -0.06 -0.07 0.08 0.08
(0.05) (0.05) (0.02) (0.02) (0.10) (0.10) (0.08) (0.08)
In-State 0.00 0.00 0.01 0.01 0.05 0.05 -0.02 -0.02
(0.02) (0.02) (0.01) (0.01) (0.04) (0.04) (0.04) (0.04)
Older 0.02 0.02 -0.03** -0.03*** 0.06 0.06 -0.01 -0.01
(0.02) (0.02) (0.01) (0.01) (0.04) (0.04) (0.04) (0.04)
Adjusted R-squared 0.01 0.07 0.07 -0.00 0.01 0.01 0.00 0.03 0.02 0.00 0.13 0.13
Pursue Engineering
Graduate School
Pursue Engineering
Employment
Pursue Non-Engineering
Employment
Pursue Non-Engineering
Graduate School
Dep. Var.:
Model: (1) (2) (3) (4) (5) (6) (7) (8)
Treatment Effects
Early Job Treatment 0.20 0.01 0.14 0.07 0.22 0.23 0.26 -6.98
(0.21) (0.20) (0.22) (0.20) (0.23) (0.23) (0.31) (8.68)
Early Job × GPA 4.92
(5.47)
Early Job × GPA^2 -0.81
(0.85)
Early Job × Low Income 1.81** 1.81** 1.92**
(0.80) (0.80) (0.80)
Early Job × Female -0.19 -0.37 -0.32
(0.45) (0.47) (0.48)
Early Job × Foreign 0.39 0.44 0.41
(0.94) (0.97) (0.98)
Early Job × In-State -0.38 -0.24 -0.23
(0.41) (0.42) (0.42)
Early Job × Older -0.46 -0.44 -0.44
(0.42) (0.44) (0.44)
Academic Characteristics
Pre-Treatment GPA 2.31*** 1.92*** 1.91*** 1.91*** 1.91*** 1.91*** 1.92*** -2.63
(0.25) (0.24) (0.23) (0.23) (0.23) (0.23) (0.23) (3.82)
Pre-Treatment GPA^2 0.73
(0.60)
Major & Cohort FEs Y Y Y Y Y Y Y
Demographic Characteristics
Low Income -0.98* 0.15 0.16 0.16 0.16 -0.97* -1.06**
(0.51) (0.46) (0.46) (0.46) (0.46) (0.51) (0.51)
Female 0.02 0.11 -0.00 -0.00 -0.01 0.22 0.19
(0.24) (0.36) (0.24) (0.25) (0.24) (0.37) (0.37)
Foreign -2.64***-2.64***-2.83***-2.63***-2.65***-2.84***-2.81***
(0.50) (0.50) (0.67) (0.50) (0.50) (0.69) (0.69)
In-State -0.18 -0.17 -0.17 0.04 -0.18 -0.06 -0.07
(0.21) (0.21) (0.21) (0.32) (0.21) (0.32) (0.32)
Older -0.10 -0.09 -0.09 -0.10 0.16 0.12 0.13
(0.22) (0.22) (0.22) (0.22) (0.33) (0.34) (0.34)
Adjusted R-squared 0.05 0.21 0.21 0.21 0.21 0.21 0.21 0.21
Wage
www.kevinboudreau.com
Summary: Effects of (Incrementally) Earlier ExposureSummary: Effects of (Incrementally) Earlier ExposureSummary: Effects of (Incrementally) Earlier ExposureSummary: Effects of (Incrementally) Earlier Exposure1.1.1.1. Academic performance/choicesAcademic performance/choicesAcademic performance/choicesAcademic performance/choices
• LowerLowerLowerLower----incomeincomeincomeincome students get higher grades
• EveryoneEveryoneEveryoneEveryone switches to higher-paying majors more + more Engineering courses
• Weakest students Weakest students Weakest students Weakest students (slightly) lower grades (weak)
2.2.2.2. PostPostPostPost----graduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomesgraduation & professional outcomes
• Stronger students Stronger students Stronger students Stronger students more likely to stay in Engineering (grad school)
• LowLowLowLow----income students income students income students income students get better jobs (wages +larger firms)
Interpretation:Interpretation:Interpretation:Interpretation:
• Greater exposure to field and income for lowlowlowlow----incomeincomeincomeincome
• Greater sorting, matching sorting, matching sorting, matching sorting, matching (majors, grad school)
• *Little evidence of costly effects in this context
• NB. No interactions with sex; womenwomenwomenwomen are better studentsbetter studentsbetter studentsbetter students, but less likely to stay in Engineering !!
www.kevinboudreau.com
ConclusionConclusionConclusionConclusion• Causal evidence from random assignment to earlier work experience
on multiple academic and professional outcomes
• Evidence consistent with greater engagement, productive
matching/sorting; benefits for low-income
• Fail to find much evidence of countervailing costs
• Results inconsistent with theory-then-practice (later exposure) being
necessarily optimal
• Scope for widening the funnel?
• Opportunities for institutional redesign?