Public Sector Structural Change in K-12 Education2012-13 Syllabus, FALL Semester

Session #1: September 5, 2012Introduction, Course Overview, and Expectations; ESEA Flexibility (Exercise)

Reading1. Lyndsey Layton, Obama to Issue No Child Left Behind Waivers to States, Washington Post,

Sept. 22, 20112. Kimberly Heffling & Ben Feller, No Child Left Behind: 10 States Receive Waivers from

Education Law’s Sweeping Requirements, AP, Feb. 9, 20123. Motoko Rich, States with Education Waivers Offer Varied Goals, N.Y. Times, July 26, 20124. Jeremy Ayers & Isabel Owen, No Child Left Behind Waivers: Promising Ideas from Second

Round Applications, Center for American Progress, July 27, 2012, at 4-7, Apps. A and B, website graphic

5. State of New Jersey Department of Education, ESEA Waiver Request (filed Nov. 14, 2011; approved by U.S. Dep’t of Ed. Feb. 9, 2012), at 15-19, 37-41, 53-58; App. 2

Discussion Questions1. How do the NCLB and ESEA Flexibility requirements differ? Why do you suppose the U.S.

Education Department offered ESEA Flexibility option? Is ESEA Flexibility, including as it will be implemented in New Jersey, a weakening or a strengthening of prior requirements? Do the differences between NCBL and ESEA Flexibility reflect divergent understandings of how government can work to influence behavior and solve public problems? How comfortable are you with the U.S. Education Department’s decision to offer this option instead of waiting for Congress to act?

2. New Jersey’s approved application for ESEA Flexibility says that its “central goal . . . is to ensure that all children, regardless of life circumstances, graduate from high school ready for college and career.” The application acknowledges that “[c]urrently, New Jersey is far from accomplishing this mission.” How likely is it that the steps New Jersey proposes in its application will succeed in this mission? As far as you can tell, what is the New Jersey Department of Education’s understanding of how government can work to influence behavior and solve public problems?

3. Seven field-based Regional Achievement Centers (RACs) are an important feature of New Jersey’s plan for turning around the state’s lowest performing schools, called “Priority Schools.” Suppose that you are a member of a cross-functional team of leaders and managers from the New Jersey Department of Education and from some of the RACs themselves, and that the team is charged, among other things, with developing a mission statement for the RACs’ interactions with Priority Schools. The mission statement is supposed to be congruent with the Department’s more general mission quoted above. The task is important, and New Jersey Commissioner of Education Christopher Cerf has asked to review the statement himself. Initial discussions have indicated that the task is not

1

straightforward. For example, various suggestions have been made to frame the statement around the following or cognate words, many of which appear in the approved request for ESEA Flexibility, but may reveal conflicting understandings of the tenor of the RACs’ work with Priority Schools: supervise, facilitate, collaborate, serve, direct, recommend, compromise, not compromise, negotiate, empower, achieve, teach (educate, instruct), learn, adults (principals, teachers), children. What process would you propose the team use to develop the mission statement, given the many directions it could take? Although the task at hand is to come up with a process for developing the statement, not to develop the statement itself, you might give a moment’s thought to which of the listed words, or which unlisted alternatives, you would use to describe the RACs’ mission as to Priority Schools.

Exercise in Class: New Jersey Regional Achievement Centers You are a member of a cross-functional team with leaders and managers from the New

Jersey Department of Education and from some of the state’s seven field-based Regional Achievement Centers (RACs).

Your team is charged, among other things, with developing a mission statement for the RACs’ interactions with failing schools, designated “Priority Schools.”

The mission statement is to be consistent with the Department’s general mission “to ensure that all children, regardless of life circumstances, graduate from high school ready for college and career.”

There seem to be divergent views about the RACs’ mission, as indicated by the following word cloud derived from the state’s ESEA Flexibility application and initial discussions about the RACs’ role vis-à-vis Priority Schools: supervise, facilitate, collaborate, serve, direct, recommend, compromise, not compromise, negotiate, empower, achieve, teach (educate, instruct), learn, adults (principals, teachers), children.

Your team has allocated 15 minutes to coming up with an agreed upon process for developing the mission statement? Please do so now.

Writing Assignment (after class):

In one page or less, please reflect on the New Jersey Regional Achievement Center exercise: What plan did your team come up with (bullets/summary)? What went well for your group? What could have gone better? What was your role in the team’s deliberations? How did your role contribute to the team’s accomplishment of its goals or any

challenges it encountered?

Due as a MS Word attachment emailed to CPRL@law.columbia.edu by Monday September 10 at 9:00 a.m.

Sessions #2: September 12, 2012When Bureaucracy (Technology-Based Regulation) Works and Fails, and What that Suggests about What Ails U.S. K-12 Public Education

2

Reading1. Council on Foreign Relations, U.S. Education Reform and National Security (Joel I. Klein

& Condoleeza Rice, Chairs 2012), at 14-352. Rudi R. Volti, An Introduction to the Sociology of Work and Occupations (2007), at 83 (1st

full paragraph: “Modern bureaucracy . . .”)-933. Cary Coglianese & David Lazer, Management-Based Regulation: Prescribing Private

Management to Achieve Public Goals, 37 L. & Soc. Rev. 691 (2003), at 691-706 (read through the preamble to Part III only; the rest of Part III is optional)

4. Examples from the New York City and other school systems, 1965-2012: a. Diane Ravitch, The Great School Wars: A History of the New York City Public Schools

(2d ed. 2000), at 329-37[Note: Professor Ravitch describes competing 1960s proposals for decentralizing the NYC schools. Shortly after the period shed discusses, the state legislature adopted a decentralization plan similar to the one supported by the UFT (see p. 335). A quarter century later, the following op-ed was published giving a contentiously stated but widely shared view of the result of the new arrangement.]

b. Johnny Ray Youngblood, Draining the School Swamp, N.Y. Daily News (1992) [Note: Doubts whether large public school systems have solved these problems in the ensuing decades are addressed or suggested by the next three readings.]

c. Eva Moskowitz, Charter-School Envy, N.Y. Post, July 23, 2012 d. Motoko Rich, Enrollment Off in Big Districts, Forcing Layoffs, N.Y. Times, July 24,

2012 e. Collective Bargaining Agreement between the New York City Dep’t of Education and

the United Federation of Teachers (UFT) (2007)5. James S. Liebman & Charles F. Sabel, A Public Laboratory Dewey Barely Imagined, 28

N.Y.U. Rev. L. & Soc. Change 183 (2003), at 184-89, 195-204 (excerpt #1)

Discussion Questions1. What is bureaucracy, understood as a form of organizing concerted activity? How does

bureaucracy manifest itself in public agencies devoted to delivering services? Agencies devoted to regulating private behavior?

2. What are some examples of bureaucracy working well? What are examples of bureaucracy working poorly? What accounts for the differences?

3. How would you characterize the current state of public education in the US? To what extent is bureaucracy responsible for the problems facing US public education? What other causes are important?

4. Focusing on the Ravitch excerpt, how was the decentralization of the New York City school system proposed in the 1960s expected to improve conditions for school children? Why else was decentralization proposed? How did the decentralization proposals differ, and what accounts for the differences?

5. Based on the Youngblood and later pieces, did decentralization in New York City schools alleviate bureaucracy and its harmful effects? Did decentralization benefit poor and minority children? Who did it benefit?

3

6. What relationship is there, if any, between bureaucracy and a. the New York City-UFT Collective Bargaining Agreement?b. difficulties big city systems are having today in addressing declining enrollments?c. failings of court-ordered school desegregation, finance equity and “adequacy”?

Session #3: September 19, 2012A Menu of Alternatives to Bureaucracy, Including “Experimentalist” or “Public-Learning” Arrangements

Reading1. Review from session 1 : Cary Coglianese & David Lazer, Management-Based

Regulation: Prescribing Private Management to Achieve Public Goals, supra 2. Charles F. Sabel & William H. Simon, Minimalism and Experimentalism in the

Administrative State, 100 Geo. L.J. 53 (2011) [Note: in this and other articles, footnotes are for your information only; you are not required to read them.]

3. Diane Ravitch, The Death and Life of the Great American School System (2010), Ch. 9 through the flag for Footnote 11

4. Jody Freeman, Extending Public Law Norms through Privatization, 116 Harv. L. Rev. 1285 (2003) (excerpts)

5. New York City’s Chancellor’s District:a. Randi Weingarten & Michael Mulgrew, Mayor Bloomberg: Stop Closing

Schools, There’s a Better Way, http://www.nydailynews.com/opinion/mayor-bloomberg-stop-closing-schools-a-better-article-1.1118928#ixzz21TBru12L

b. MassInsight, Meeting the Turnaround Challenge: New York City’s Chancellor’s District (2010), http://www.massinsight.org/publications/stg-resources/99/file/4/pubs/2010/04/20/III_District_Profile_NYC_CD.pdf

6. Optional : For a simplified version of the Sabel & Simon piece, see James S. Liebman, Public-Sector Reform through Learning Organizations and the Supporting Role of Universities, at 1-11 (through the 1st ¶ of Part V) – or read the whole thing if you’re interested in the overall logic of the Center for Public Research and Leadership of which this course is a part

Discussion Questions1. What variations on or alternatives to bureaucratic organization can you identify

being advocated, discussed or criticized in the readings?2. As to each variation or alternative, how if at all does it seek to mitigate common

problems with bureaucracy?3. In what ways are the variations and alternatives similar to each other, and in what

ways do they differ?4. With the caveats that many of the readings and excerpts only allude to solutions

and do not present full-fledged defenses of them, and that we will spend more time addressing each of them later on in the semester, what are your preliminary thoughts about --

4

a. how effectively each proposal avoids the difficulties with bureaucracy?b. other difficulties the proposals are likely to encounter?c. which seems most appealing to you and why?

Session #4: September 26, 2012Some Classic Learning Organizations in Action

Reading1. Steven J. Spear, The High Velocity Edge: How Market Leaders Leverage Operational

Excellence to Beat the Competition (2010), pp. 1-32, 36-55, 83-91 (skip 56-82 for now) (excerpts from Chapters 4-6 and 8: Alcoa, Nuclear Navy, Pratt-Whitney, Ave. A, Toyota)

Discussion Questions1. Based on Spear’s description of the organizations at the center of each of his examples,

what would you say is the organization’s “secret sauce?”2. In what ways are the organizations and their strategies that Spear describes similar? In

what ways do they differ?3. What attributes of the organizations and their strategies if any justify calling them

“experimentalist” or “learning organizations” in the senses introduced in Session #3? Are any of them more or less appropriately described as examples of that approach to organizing concerted action? Are any of them more or less useful as models for organizing concerted action in other contexts, including in public-sector contexts? Why?

4. As to each example Spear gives (including each of the different strategies described for some of the entities), how would you define the actual “organization” as issue that might be thought to be experimenting or learning? How is that organization embedded within others described; how do they all relate to and interact with each other? Assuming Spear’s examples they are relevant at all, do they reinforce your sense of the value of “learning” or “experimentalist” organizations as alternatives to bureaucracies, especially public-sector bureaucracies? Do they allay any of your concerns about other difficulties those kinds of organizations may encounter? Or, do they underscore or add new concerns about those organizations?

Session #5: October 3, 2012More Public-Learning Organizations in Action

Reading 1. Baltimore and Maryland/O’Malley : Tina Rosenberg, Armed with Data, Fighting More

than Crime, N.Y. Times Opinionator, May 2, 20122. New York City School System under Bloomberg/Klein :

a. Stacey Childress, et al., Managing for Results at the New York City Department of Education, in Jennifer O’Day, et al., Education Reform in New York City: Ambitious Change in the Nation’s Most Complex School System (2011), at pp. 1-15 of draft

b. Alliance for Excellent Education, New York City’s Strategy for Improving High Schools: An Overview (Jan. 2010), at pp. 1-11 (first half of p. 11 only)

5

c. Joel Klein, Yes, New York City Schools Are Improving, NY Daily News Opinion, Sept. 10, 2012

d. Optional : i. For more detailed reviews of the student results in New York City between 2003

and 2010, see James Kemple, Children First and Student Outcomes: 2003-2010, in O’Day, et

al., supra James S. Liebman & Jonah Rockoff, Moving Mountains in New York City: Joel

Klein’s Legacy by the Numbers, Ed. Week (Nov. 30, 2010)ii. For discussion of how the NYC system plays itself out for two very different

schools in the same building, see Daniel Lautzenheiser, A Tale of Two Schools: What New York Dep’t of Education is Getting Right, Huffington Post (July 24, 2012)

3. Case Study (one of the following three, as indicated) a. Students with last name beginning with A-G : Environmental Regulatory Innovations

i. Charles Sabel, Archon Fung & Bradley Karkkainen, Beyond Backyard Experimentalism, Boston Rev. Oct. 1999 (excerpts)

ii. Theordore Lowi, Frontyard Propoganda, A Response to Fung et al., Boston Rev. Oct. 1999

iii. Matt Wilson & Eric Weltman, Government’s Job, A Response to Fung et al., Boston Rev. Oct. 1999

b. Students with last names beginning with H-R: Texas School Reform i. Liebman & Sabel, A Laboratory Dewey Barely Imagined, supra, at 231-50 (except

#2)ii. Richard F. Elmore, Details, Details, Details, 28 N.Y.U. Rev. L. & Soc. Change

315 (2003) (excerpts)iii. Martha Minow, School Reform Outside Laboratory Conditions: A Response to

Liebman & Sabel, 28 N.Y.U. Rev. L. & Soc. Change 333 (2003) (excerpts)iv. James S. Liebman & Charles F. Sabel, The Fragile Promise of Provisionality: A

Reply to Responses, 28 N.Y.U. Rev. L. & Soc. Change 369 (2003) (excerpts)

c. Students with last names beginning with S-Z: Drug Courts i. Michael Dorf & Charles Sabel, Drug Treatment Courts and Emergent

Experimentalist Government 53 Vand. L. Rev. 831 (2000), at 1-5, 10 (1st full ¶) – 12 (run-over ¶ only), 16-33 (Part I), 52-56

ii. Martha Minow, School Reform Outside Laboratory Conditions: A Response to Liebman & Sabel, 28 N.Y.U. Rev. L. & Soc. Change 333 (2003) (excerpts) [NOTE: It is important for each group of students to consider criticisms of so-called “experimentalist” reforms. Although Dean Minow’s piece responds to an article about education reform, not drug courts, consider how her three criticisms (lack of capacity, resources and consensus) also apply to the drug court context.]

6

iii. James S. Liebman & Charles F. Sabel, The Fragile Promise of Provisionality: A Reply to Responses, 28 N.Y.U. Rev. L. & Soc. Change 369 (2003) (excerpts)

iv. Optional : Gov’t Accounting Office, ADULT DRUG COURTS: Studies Show Courts Reduce

Recidivism, but DOJ Could Enhance Future Performance Measure Revision Efforts (2011), http://www.gao.gov/assets/590/586793.pdf

Shelli B. Rossman et al., National Institute of Justice Study, The Multi-Site Adult Drug Court Evaluation (Dec. 2011), https://www.ncjrs.gov/pdffiles1/nij/grants/237108.pdf

Discussion Questions: All members of the class are assigned readings about Baltimore Mayor, now Maryland Governor, Martin O’Malley’s approach to organizing government’s response to social problems and about school reform in New York City ca. 2003-2010. Each of you also is assigned to read about one of three other sets of public-sector innovations in the 1990s and early 2000s: new forms of environmental regulation, Texas school reform, drug courts. For each of the domains about which you read, consider the following questions:

1. What public problem was the arrangement described designed to solve? 2. In what sense is each of the arrangements “experimentalist” or focused on “public

learning”? 3. In what ways is the arrangement similar to and how does it differ from other ways of

addressing public problems such as bureaucracy, performance-based regulation or managerialism, privatization, or craft-based activity?

4. How would you define the “organization(s)” that are said to have emerged from each of the arrangements? Who are the key actors? To what extent are those actors public or private? In what sense if any do they together constitute a real “organization,” as opposed to a fleeting confederation of people who happen to be focusing on the same problem at the same time and place?

5. What criticisms would you, or do the critics whom you read, apply to the arrangements as a whole? How serious are the criticisms? Are there convincing responses?

Writing Assignment: Write a ~2-page paper, due as a MS Word attachment emailed to CPRL@law.columbia.edu by Tuesday October 4, 2012 at 2:00 p.m., addressing any one of Discussion Questions 2-5 above. Feel free to focus on only a single: trait of the arrangement, alternative method of addressing problems, similarity or difference between the arrangement and one or more of the alternative methods of addressing problems, organizational or dis-organizational trait, criticism, etc.

Session #6: October 10, 2012A Conversation with Garth Harries, Assistant Superintendent, New Haven Public Schools, and Shael Polakow-Suransky, Chief Academic Officer and Senior Deputy Chancellor of the New York City Department of Education

Reading 1. Garth Harries Bio

7

2. New Haven School Change, PowerPoint Presentation3. Emily Bazelon, Can Public Schools Really Change? Why New Haven’s Ambitious New

Education Strategy Might Actually Succeed, Slate, Sept. 27, 2012, http://www.slate.com/articles/business/the_pivot/2012/09/public_schools_vs_charter_schools_has_new_haven_figured_out_a_school_reform_strategy_that_will_succeed_.html

4. Shael Polakow-Suransky Bio5. NY City Dep’t of Education, Citywide Instructional Expectations for 2011-126. Optional:

a. NY City Dep’t of Education, Citywide Instructional Expectations for 2012-13b. The Broad Center, Who’s the Boss Now? Shifting the Balance of Power in New

York City Schools (2011)

Discussion Questions: Please come to class with three or four questions you would like to pose to the speakers. Some of the questions should address issues you have encountered in, or concerns triggered by, our readings and discussions about the most effective ways to organize classrooms, schools, and school systems to accelerate student learning and improve student results and life chances.

Session #7: October 17, 2012Philosophical Pragmatism and Learning Organizations

Reading1. Christopher K. Ansell, Pragmatist Democracy: Evolutionary Learning as Public Philosophy

(2011), Chapters 1, 5 (pp. 3-19, 84-101)

Discussion Questions: 1. What is the relationship between bureaucracy and partisan politics? How do they

interact to undermine the capacity of public agencies to solve public problems? How does Ansell move from the conclusion that this interaction is responsible for the failure of “governance” to the solution of expanding the role of agencies and the breadth of public participation in them? How convincing is that solution? Does the structure of Ansell’s argument as laid out in Chapter 1’s summary of the rest of the book (pp.15-19) provide a convincing case for putting administrative agencies at the heart of solutions to paralysis in governance and obdurate partisanship?

2. How do you interpret and how useful is Ansell’s idea of “evolutionary learning,” which occurs when a “problem-driven perspective,” “reflexivity” and “deliberation” work together “recursively”?

3. Ansell distinguishes a form of evidence-based evaluation and decision making based on experimental and quasi-experimental research (which he associates with the work of Donald Campbell) from “experimentalism” or “evolutionary learning” (p.12). What is the difference between the two strategies? Do you agree with Ansell’s preference for the latter over the former – or, more accurately, his preference for treating the latter as encompassing but going beyond the former? What difficulties do so-called “wicked,”

8

“compound,” or “high-interaction” problems present for experimental and quasi-experimental research? Chapter 5, footnote 30 quotes a description of Dewey’s approach to inquiry as follows: “[T]he creative formation of conjectures in the absence of data is not the key to successful inquiry. Rather the interweaving of observation, conjecture, verification, and additional observation are the key ingredients of successful problem solving.” Can inquiry of this sort provide a substitute for classic experimental and quasi-experimental study methods in situations where too little is known, too much is changing too quickly, or the problems are too many and too ‘small’ to permit or justify classic study methods?

4. How enlightening did you find Ansell’s dualisms: “progressive conservatism,” “cosmopolitan localism,” “analytical holism,” and “processual structuralism”? How descriptive are they of experimentalism as you understand it, and how convincing are they as attributes of a “third way” to solving public problems that sits in between bureaucracy and interest-group politics on the one hand and market-based solutions and pure electoral politics on the other hand?

5. How well does Ansell’s description of “Pragmatist problem-solving” capture the processes you have seen at work in examples we have considered: Alcoa, Pratt & Whitney, the Nuclear Navy, Toyota, drug courts, 1990s Texas education reform, Habitat Conservation Plans, the Chesapeake Bay Plan, TRI/TURA regulation of toxic releases, the New York City and New Haven school reforms, etc.? What is missing from Ansell’s discussion to make it more descriptive of these examples? What is missing from the steps taken in these examples that keeps them from accomplishing the kind of problem solving that Ansell promotes? In what ways, if any, does Ansell understate the difficulty of his conception of public problem-solving?

Exercise in Preparation for ClassImagine a public agency with which you have some familiarity. Before class jot down some notes for use in class identifying (1) features of the operation or outcomes of that agency that strike you as harmful to the agency’s effectiveness or evidence of its ineffectiveness; (2) which of the McKinsey & Co. “7S’s” (see framework below) those features involve; (3) steps, consistent with the Pragmatist Learning organizations Ansell discusses, that you can imagine taking to improve, or even transform, the agency; and (4) difficulties you imagine those steps and that transformation would encounter

9

Sessions #8 and #9: October 24 and 31, 2012Addressing Core Difficulties of Government by Bureaucracy: Too Little Central Expertise, Too Much Street-Level Discretion; the Craft and Performance-Management Alternatives

Reading 1. Michael Lipsky, Street-Level Bureaucracy: Dilemmas of the Individual in Public Services

(1979), at 18-19, 48-532. James Q. Wilson, Varieties of Police Behavior (1968), at 1-11, 279-97 (end of paragraph

running over from p.296)a. Optional : To get a richer sense of the diversity of problems patrol officers and their

supervisors face when bureaucratic rules are used to govern street-level activity in the order-maintenance context, read or skim pp.16-82.

3. Diane Ravitch: Why Finland’s Schools Are Great (by Doing What We Don’t), http://www.washingtonpost.com/blogs/answer-sheet/post/ravitch-why-finlands-schools-are-great-by-doing-what-we-dont/2011/10/12/gIQAmTyLgL_blog.html

4. Liebman & Sabel, A Laboratory Dewey Barely Imagined, supra, at 213-31 (excerpt #3)5. Eric Hanushek, Teacher Deselection, in Creating a New Teaching Profession 165-78 (D.

Goldhaber & J. Hannaway eds. 2010)6. Emma Brown, 98 D.C. Teachers Fired for Poor Performance, Washington Post, Aug. 1,

20127. Simon Head, The Grim Threat to British Universities, N.Y. Rev. Books, Jan. 13, 2011 8. Rick Hess & Linda Darling-Hammond, How to Rescue Education Reform, N.Y. Times, Dec.

5, 2011 9. Arthur E. Wise & Michael D. Usdan, The Political Future of the Teaching Profession, Ed.

Week, March 13, 2013

Discussion Questions:1. In private- and public-sector organizations, why is shop-floor and street-level discretion

problematic?2. How do bureaucracies try to solve the problem of shop-floor or street-level discretion?

What problems does the bureaucratic solution pose for local- level employees, their supervisors, the organization as a whole? How (if at all) are those problems manifested in police departments, K-12 school systems, welfare agencies, environmental regulatory agencies, criminal courts, universities?

3. What solutions to the problem of shop-floor and street-level discretion are proposed or implied by James Q. Wilson, Diane Ravitch, the first and second iterations of the 1990s reforms in New York City’s Community School District (CSD) 2 (see Liebman & Sabel), Hanushek, Great Britain’s system for distributing public money to universities based on faculty productivity, Head’s critique of the British system, and Hess & Darling-Hammond? How is each solution similar to and different from the others – a question with particular salience when applied to the two phases of the New York City CSD 2 reforms? In what ways does each of the proposed solutions avoid problems otherwise encountered in regulating the actions of street-level government officials and in what

10

ways does each fail to solve those problems? What additional problems does each of the proposed solutions create?

4. Simon Head attributes the travails of British university academics to “command and control bureaucracy,” the “industrial” model, and “business school” ideas such as “Total Quality Management” and “benchmarking” – as if these are the same thing? Are they? How would you characterize the driving force behind the evaluation of the productivity of British academics? What do you think of the British system? Of Head’s critique of the system?

5. Rick Hess is a conservative advocate of privatization and performance-management approaches to public education. Linda Darling-Hammond is a progressive advocate of teacher autonomy and professionalization. They argue that their ability to agree on the wrongheadedness of current federal education policy is proof that it must certainly be wrongheaded. Are there other reasons why people with their bents on school reform issues might find common cause? What do you think of their critique of federal policy and proposed alternative?

Session #10: November 7, 2012Using Street-Level Public Learning and Problem Solving to Increase Organizational Expertise and Harness Street-Level Discretion

Reading1. Spear, High Velocity Edge, supra, at 56-71 and review pp. 88-91 (excerpts from Chapters

7 and 8 re: Toyota)2. Joan E. Talbert, Collaborative Inquiry to Expand Student Success in New York City

Schools, in in O’Day, et al., supra, at 1-6 (to 2 paragraphs), 12-17 & table 2 3. Elizabeth Chu et al., Getting Big to Go Small: Case Studies of Collaborative Inquiry Teams

in New York City, Nov. 2012, 2-10, 19-34 (DRAFT: do not circulate)4. Peg Tyre, The Writing Revolution, Atlantic Monthly (Oct. 2012),

http://www.theatlantic.com/magazine/print/2012/10/the-writing-revolution/309090/ 5. Collaborative Inquiry in Teacher Teams Data (NYC – PPT)6. Jennifer Morrison, Why Teachers Must be Data Experts, 66 Ed. Leadership No. 4 (2008-

2009) 7. Roxanna Elden, Data-Driven and Off Course, Ed. Next (Winter 2011) 8. Multidisciplinary Rounds How-to Guide. Cambridge, MA: Institute for Healthcare

Improvement; 2010, www.IHI.org 9. Optional:

a. Jim Frederickson, Are We Learning the Right Lessons from New Dorp, Atlantic blog, Sept. 28, 2012, www.theatlantic.com/national/archive/2012/09/are-we-learning-the-right-lessons-from-new-dorp-high-school/263010/]

b. Carnegie Foundation Network Initiativei. Carnegie Foundation for the Improvement of Teaching, Improvement Research,

http://www.carnegiefoundation.org/improvement-research/approach (last visited Sept. 2, 2012)

11

ii. Jal D. Mehta, Louis M. Gomez, & Anthony S. Bryk, Schooling as a Knowledge Profession, Educ. Week, March 30, 2011

Discussion Questions:1. A key feature of bureaucratic organization is the concentration of knowledge and

expertise at the center of the organization, using the central imposition of and local adherence to rules to diffuse that expertise to sites where it is applied to solve problems. What is the problem with this approach to the amassing and use of expertise? How does the New Dorp High School example bear on the question of the best likely sources of expertise, and do you agree with Tyre (or Frederickson, if you read it) on the answer to that question?

2. As described in the readings, how do Toyota problem-solving teams, inquiry and data teams, and multidisciplinary rounds in hospitals (and the Carnegie network initiative, if you read about it) generate and use knowledge and expertise? What similarities and differences do you see in these various approaches to the problem of knowledge? How do they differ from bureaucratic approaches to expertise and knowledge? (Relatedly, how do “multidisciplinary rounds” differ from traditional hospital rounds?”) How likely are these newer approaches to avoid the problems of knowledge and expertise encountered in bureaucracies?

3. What is the difference between “getting small in order to go big” (as described in Talbert’s discussion of New York City Inquiry Teams prior to 2010) and “Getting Big to Go Small” (as described in Chu et al.’s discussion of those teams since 2010)? How important is that difference? What accounts for it?

4. What is the relationship between these new approaches and “professionalism”? Do they make street-level actors more or less professional – a question, you may remember, that was addressed in last week’s Liebman & Sabel reading about the Community School District 2 experience in New York City in the 1990s? Or put differently, do the new approaches change the meaning of professionalism in any way? Or put yet another way, to what extent do these new approaches to the generation of knowledge and expertise help solve, or exacerbate, the problem of enabling while controlling the discretion of “street-level bureaucrats?”

5. What would you expect to be the biggest challenges to the success of these new approaches to generating expertise and knowledge, and how might organizations respond to those challenges? How do the new approaches affect the usual routines and allocation of time at the sites where they occur? How do they keep team-based activities from degenerating into diffuse “bull sessions?” How do they assure that learning that occurs at one site is usefully diffused to other sites – and how straightforward do you imagine the diffusion process to be?

Writing Assignment: Write a ~2-page paper, due as a MS Word attachment emailed to CPRL@law.columbia.edu by Tuesday November 6, 2012 at 2:00 p.m., addressing one of Discussion Questions 2-5 with regard to at least one (or in the case of question 3, with regard to both) of the knowledge-generating mechanisms referenced in the relevant question.

12

Inquiry Team Exercise: To be described in class

Session 11: November 14, 2012The Skilled Generalist: The Role and Skillset of Professionals in Learning Organizations

Reading1. Lane Wallace, Multicultural Critical Theory at B-School?, N.Y. Times, Jan. 10, 2010, at

Bu.1, http://www.nytimes.com/2010/01/10/business/10mba.html?pagewanted=all2. Karen Sloan, Reality’s Knocking: The Ivory Tower Gives Way to the Real World’s

Demands, Nat’l L.J., Sept. 7, 2009, http://www.law.com/jsp/nlj/PubArticleNLJ.jsp?id=1202433612463&Realitys_knocking&slreturn=1&hbxlogin=1

3. Howard B. Miller, Structural or Cyclical?, Calif. Bar J. Feb. 2010, at 8, http://www.calbarjournal.com/February2010/Opinion/FromthePresident.aspx

4. Lincoln Caplan, An Existential Crisis for Law Schools, N.Y. Times, July 15, 2012, http://www.nytimes.com/2012/07/15/opinion/sunday/an-existential-crisis-for-law-schools.html?pagewanted=print

5. Arne Duncan, Teacher Preparation: Reforming the Uncertain Profession—Remarks of Secretary Duncan at Teachers College, Columbia University, Oct. 22, 2009 (text, video)

6. Laura Pappano, Skills to Fix Failing Schools, N.Y. Times, Dec. 29, 2009, Educ. Life 1, http://www.nytimes.com/2010/01/03/education/edlife/03educ.html?_r=1

7. Skilled Generalist Handout8. James S. Liebman, Public-Sector Reform through Learning Organizations and the

Supporting Role of Universities, at 11-17 (or read the whole thing if you haven’t already and you are interested in the overall logic of the Center for Public Research and Leadership of which this course is a part)

9. Optional : a. Karen Sloan, Think of it as a Residency for Lawyers, Nat’l L.J., June 5, 2012b. Jamie Alter & Jane G. Cogshall, Teaching as a Clinical-Practice Profession:

Implications for Teacher Preparation and State Policy (2009), http://www.tqsource.org/publications/clinicalPractice.pdf

c. Nat’l Council for Accreditation of Teacher Education, Transforming Teacher Education Through Clinical Practice (2010), http://www.suny.edu/Files/sunynewsFiles/Pdf/NCATEBlueRibbonReport.pdf

d. NYU Revamps Third Year [Law] Curriculum, The Nat’l Jurist, Nov. 2, 2012, http://www.nationaljurist.com/content/nyu-revamps-third-year-curriculum

Discussion Questions:1. Each of you is in a school defined by a profession. What major changes if any do you

think are in store for that profession over the next several years or decades? What allied changes if any do you think are in store for professional schools of the sort you attend?

2. How does your professional school seem to understand the definition of a profession or a professional? How would you define a professional? If your and your school’s definitions differ, what accounts for your differing view?

13

3. Based on the answers to Questions 1 and 2 and also on the readings and discussion thus far in the semester, your project work, and your other professional experiences, what skills would you add to the preliminary list on the Skilled Generalist Handout, which you think reforming public-sector organizations require of their leaders and managers? Would you omit or revise any of the skills listed on that handout?

(Informal) Writing Assignment: Come to class with a list of skills you would add to the Skilled Generalist handout (as many or few as you think are appropriate). Your list may be handwritten or typed, but please have it in a form you can hand in or email in after class.

Session #12: November 21, 2012

Part 1 (55 minutes): Student Project Presentation (Louisiana “Course Choice” Project): What is the Project? To What Extent Does it or its Institutional Context Fit Public Learning or Another Model? What Are the Team’s Initial Thoughts on What the Project or its Context Suggest About the Possibility of Public Learning and Problem-Solving, and What Public Learning and Problem-Solving Suggest about the Project or its Context?Part 2 (55 minutes): The Finland (Public Elementary and Secondary Education) Rorschach

Reading for Part 1:2. Martha Minow, School Reform Outside Laboratory Conditions, 28 N.Y.U. Rev. L. Soc.

Change 333-38 (2003) (many of you have already read this; please review)3. Liebman & Sabel, A Laboratory Dewey Barely Imagined, supra, at 384 (starting TAN 102)

– 386 (ending TAN 108) (many of you have already read this; please review)4. Additional reading to be identified by the presenting team at least a week ahead of time

Reading for Part 2:1. Review: Diane Ravitch: Why Finland’s Schools Are Great (by Doing What We Don’t),

http://www.washingtonpost.com/blogs/answer-sheet/post/ravitch-why-finlands-schools-are-great-by-doing-what-we-dont/2011/10/12/gIQAmTyLgL_blog.html

2. Anu Partanen, What Americans Keep Ignoring About Finland’s School Success, The Atlantic, Dec. 29, 2011

3. Charles Sabel, AnnaLee Saxenian, Reijo Miettinen, Peer Hull Kristensen & Jarkko Hautamäki, Individualized Service Provision as the Key to the New Welfare State: Lessons from Special Education in Finland (Sitra Studies 62 Dec. 2011), at pp. 4-15, skim 26-30, 30 (“The next national core curriculum”)-53, 57-64

Discussion Questions for Part 1: 1. Does Louisiana Course Choice fit into an inquiry learning or experimentalist model? If so,

how? If not, why not?2. As you read about the Course Choice program and how it is expected to change public

education in Louisiana, what are some of the main management challenges you identify? Particularly, think about how this new program will affect the public school system and its operation going forward. What is likely to change about the delivery and

14

management of public education in the state of Louisiana? Does this program offer solutions for improving the Louisiana school system? Why or why not?

Discussion Questions for Part 2:1. What is Finland’s secret sauce vis-à-vis elementary and secondary education?2. How does your answer to Question 1 bear on the problem of street-level discretion? On

what it does or should mean for a teacher to be considered a professional? On the problem of building organizational knowledge and expertise?

3. How transferrable is the Finland success mechanism to the U.S.? What are the main challenges to such a transfer? What is the relevance to that question of differences between Finland and the U.S. in terms of social homogeneity, how much the population values public education, the commitment to “equality,” the extent to which responsibility for public education resides at the national or a more local level of government, the disposition to develop policy in this area top-down vs. bottom-up?

4. What lessons, if any, can Finland learn from current U.S. education reforms? 5. When two nations achieve different outcomes on international benchmark measures,

what is the best method of explaining the different outcomes?

Sessions #13 and 14: November 28 and December 5, 2012Student Project Presentations: What is the Project? To What Extent Does it or its Institutional Context Fit Public Learning or Another Model? What Are the Team’s Initial Thoughts on What the Project or its Context Suggest About the Possibility of Public Learning and Problem-Solving, and What Public Learning and Problem-Solving Suggest about the Project or its Context?Part 2 (55 minutes): The Finland (Public Elementary and Secondary Education) Rorschach

Order of Presentations:3. November 28, 2012, 1 st 55 minutes : Teacher Evaluation Pilot/Networks: 4. November 28, 2012, 2 nd 55 minutes : Teacher Evaluation Communication5. December 5, 2012, 1 st 55 minutes : Connecticut Dep’t of Education: Quality Review 6. December 5, 2012, 2 nd 55 minutes : Connecticut Reform Accelerator Accountability

Reading1. Readings for Teacher Evaluation Pilot/Networks (Nov. 28, 2012)

a. NYC Department of Education, Building Network Capacity around Teacher Effectiveness

b. School Book, A Last-Minute Deal on Teacher Evaluationsc. New York Times Schools for Tomorrow Conference d. School Book, Test Driving a Pilot Teacher Evaluation Systeme. Organization Chart for Network ABC

2. Reading for Teacher Evaluation Communication (Nov. 28, 2012)a. To be provided by student team

3. Readings for Connecticut Dep’t of Education: Quality Review (Dec. 5, 2012)a. To be provided by student team

3. Readings for Connecticut Reform Accelerator Accountability (Dec. 5, 2012)

15

a. To be provided by student team

Discussion Questions

1. Teacher Evaluation Pilot/Networks (Nov. 28, 2012)a. Is it possible to implement a large-scale system to evaluate and improve individual

teachers? What lessons can be learned from the Toyota model about successfully implementing a large-scale system for evaluation and improvement of front-line workers?

b. How does the Teacher Effectiveness project map to various alternative models of governance, such as learning organizations, professionalism, etc.? How does the Danielson Framework serve as a tool for implementing that model? Does teacher evaluation help or hinder front-line professionalism and discretion in classroom?

c. What does it mean to “build capacity?” Is the New York City building capacity in the right places for the right people?

d. What are the strengths and shortcomings of New York City’s approach to teacher evaluation?

2. Teacher Evaluation Communication (Nov. 28, 2012) a. How do the communication strategies and recommendations in the article compare

with your own experience of being evaluated in an educational or related context? b. Are the key lessons learned in Colorado regarding effective communication (detailed

on pages 2-3) relevant to broader discussions of learning organizations? How do they compare with the emphasis of other learning organizational models we have talked about?

c. How would you improve the usefulness of the Colorado report to make it more useful for other organizations seeking to learn from Colorado’s experience?

3. Connecticut Dep’t of Education: Quality Review (Dec. 5, 2012)a. To be provided by student team

4. Connecticut Reform Accelerator Accountability (Dec. 5, 2012)a. To be provided by student team

16

SPRING Semester

Session #15: January 28, 2013 Learning “Regimes” and the Elastic Boundaries of Private-Sector Learning “Organizations”: Dis-integrated Firms, Inter-firm Collaboration, and Private Associations

Reading1. Private-Sector Examples : Ronald J. Gilson et. al, Contracting for Innovation: Vertical

Disintegration and Interfirm Collaboration, 109 Colum. L. Rev. 431, 458-71 (2009) (Part III: pages 12-18 of the article print)

2. Joseph Rees, Hostages of Each Other: The Transformation of Nuclear Power Safety After Three Mile Island 1-7, 91-150 (1998)

3. Optional : National Commission on the BP Deepwater Horizon Oil Spill and Offshore Drilling, Deepwater: The Gulf Oil Disaster and the Future of Offshore Drilling 235-41 (Final Report, May 22, 2010), http://www.oilspillcommission.gov/sites/default/files/documents/FinalReportChapter8.pdf

Discussion Questions1. In what ways can organizations blur or reach beyond their boundaries with other

entities – and in what ways can public organizations blur or reach outside the public/private boundary – in search of better pathways to success? In this context, reconsider Ansell’s claim that institutions can be collectively controlled and revised but that, in the process, “institutional design must be closely attentive to ways that institutions are complexly ‘scaffolded’ by webs of related institutions” (14)

2. How would you define a “regime” of the sort(s) described in the reading? What is the value of thinking about organizational and governance arrangements as “regimes” of this sort?

3. In the examples in Gilson, et al.:a. Why did the companies involved vertically disintegrate? What did they gain and lose

by taking that step? b. What’s in it (or what’s lost) for the satellite organizations that operate within the

larger organization’s orbit?c. To what extent does “cooperation” as opposed to “cooptation” or “subjugation”

describe the interaction between the larger and satellite organizations?4. In your experience and based on prior readings, how do simulated markets attempt to

engage regulated entities in regulating themselves? How do “learning” regimes try to do the same? Which works better when and why?

5. In the INPO example offered by Rees:a. How much or little does the INPO arrangement for regulating the safety of nuclear

power plants add to or detract from your confidence in the ability to generate power safely through nuclear fission? What aspects of the arrangement influence your conclusion the most?

b. What role in the arrangement is played by the Nuclear Regulatory Agency?

17

c. What role is played by secrecy in the disclosure of results?d. What role is played by organizational or industry “culture” and what accounts for

changes in that culture?e. What else is the secret for success – assuming you see any evidence of success?

6. How promising is the INPO model for regulating other kinds of environmental harm, e.g., from deep-water oil drilling? Would technology-based regulation or market simulation work better?

Session #16: February 4, 2013Learning “Regimes” and the Elastic Boundaries of Public-Sector Learning “Organizations”: From Rotten Cantaloupes to Equality Directives to the Role of Charter Schools in Systematic Education Reform

Reading1. Public Regulatory Regimes :

a. Charles F. Sabel & William H. Simon, Contextualizing Regimes: Institutionalization as a Response to the Limits of Interpretation and Policy Engineering, 110 Mich. L. Rev. 1265, 1274-97 (2012)

b. Olatunde Johnson, Beyond the Private Attorney General: Equality Directives in American Law (2012) (excerpts) (full article, with footnotes, is available at http://lsr.nellco.org/cgi/viewcontent.cgi?article=1095&context=columbia_pllt)

c. Richard Thompson Ford, Moving Beyond Civil Rights, N.Y. Times, Oct. 27, 20112. Charter Schools – the Evidence Thus Far in New York City

a. Margaret Raymond, Multiple Choice: Charter School Study in 16 States (CREDO at Stanford University 2009), at 1-4 (Executive Summary excerpts), http://credo.stanford.edu/reports/MULTIPLE_CHOICE_CREDO.pdf

b. Margaret Raymond, Charter School Performance in New York City (CREDO at Stanford University 2010), at 2 (Summary), http://credo.stanford.edu/reports/NYC%202009%20_CREDO.pdf

c. Caroline M. Hoxby et al., How New York City’s Charter Schools Affect Achievement (Nat’l Bureau of Economic Research 2010), at v (first two ¶s), vii-viii (Executive Summary excerpts), http://users.nber.org/~schools/charterschoolseval/how_NYC_charter_schools_affect_achievement_sept2009.pdf

d. New York City Charter School Center, The State of the New York City Charter Sector 2012, http://c4258751.r51.cf2.rackcdn.com/state-of-the-sector-2012.pdf (pp.7-11, 13-37)

1. Charter Schools as Components of Regimes a. Paul Hill & Christine Campbell, Growing Number of Districts Seek Bold Change with

Portfolio Strategy (Center on Reinventing Public Education June 2011), at 1-3, http://www.crpe.org/sites/default/files/Brief_PSDP_SchoolDistricts_August2012.pdf

b. Neerav Kingsland, An Open Letter to Urban Superintendents in the United States of America, on Ed Week, Rick Hess Straight Up, Jan. 23-27, 2012

18

c. The School District of Philadelphia, A Blueprint for Transforming Philadelphia’s Public Schools (May 2012), at 1-20 [Note: Apart from a plan to close approximately one-third of the district’s schools as a money-saving strategy, the reforms subsequently proposed by the new Superintendent of Philadelphia Schools in January 2013 are less radical than those proposed in the May 2012 Blueprint. If you are interested in learning more, see The School District of Philadelphia, Action Plan v.1.0 (Jan. 7. 2013), http://www.philasd.org/announcements/actionplan/APv1.0.pdf ]

d. Eric Nadelstern, Schools Closed = Kids Saved, NY Daily News Opinion, Jan. 9, 2013 2. Criticism of Charter Schools

a. Naomi Klein, The Shock Doctrine (introduction- excerpts) (2010)b. Diane Ravitch, Privatizing Public Education in Philadelphia?, EdWeek Blogs (May 15,

2012) c. Adam B. Schaeffer, Tough Truths About Charters, N.Y. Daily News, Sept. 6, 2012,

http://www.cato.org/publications/commentary/tough-truths-about-charters

Discussion Questions1. Do the Hart & Sacks and other examples of legal, governance, or organizational

“regimes” described in the reading clarify, or confuse, that concept, or give you a different perspective on the value of thinking about organizational and governance arrangements as “regimes” of this sort?

2. As a consumer of leafy greens, meat, and other foodstuffs, do the changes in regulation in those contexts described by Sabel & Simon give you more or less confidence that what you eat is safe? What aspects of the old and new forms of regulation account for your answer?

3. What difficulties has enforcement of civil rights norms encountered over the last several decades? In what ways, if any, do the experimentalist approaches and/or the “equality directives” described in the readings change those dynamics? How, if at all, do Professors Sabel and Simon’s and Professor Johnson’s accounts of these new approaches overlap or differ?

4. Given the evidence in the reading about how charter schools work, what do they have to offer as solutions, or contributions to solving, the problems of public K-12 education? a. Do charter schools harm or improve student life chances? What accounts for the

different results in the two Raymond studies and the Hoxby study? b. Even if charter schools can improve the results of students who attend them, are

they bad for “school system(s)” they affect? In what ways? What do you make of suggestions in the reading that charter schools are a right wing conspiracy or a left wing conspiracy?

c. Can charter schools be scaled to reach 100% of urban school children as Kingsland suggests? How would you expect a “100% solution” to affect the results the Raymond and Hoxby studies reach?

d. As far as you can tell from the reading, is the role of charter schools the same or different in Cleveland, New Orleans, New York, the Philadelphia plan as originally conceived (May 2012), the “portfolio districts” that Hill & Campbell discuss?

19

i. To what extent do any of those roles qualify as “privatization”? To what extent do any of them qualify as components of a K-12 learning “regime” – and, if any do qualify, how do they contribute to such regimes?

ii. Apart from the answer to the question just above, in what ways could charter schools contribute to a K-12 learning regime?

iii. In what ways, if any, would you expect “privatizing” and/or “learning” uses of charter schools to maximize their positive impact on student outcomes?

Writing Assignment: Write a ~2-page paper, due as a MS Word attachment emailed to CPRL@law.columbia.edu by Monday February 4, 2013 at 8:00 a.m., on the most effective role, if any, for charter schools within a school-reform “regime.” In doing so, try to use the themes and concepts of the course to extract yourself from, or to find a better way to resolve or defuse, the standard debates about charter schools. For example: Are charter schools the reform itself or only a part of it, and if the latter, what part? Are they typically designed to be used (or, from a slightly different perspective, are they best used) as a tool for performance management, institutional learning, freeing up professionals to work their magic, or marketizing school system? From an institutional learning perspective, is a charter schools best thought of as the experimentalist institution itself, as a piece of a wider experimentalist structure or regime, or both—and how so? Or are charter schools anathema to institutional learning?

Session #17: February 11, 2013Tools Supporting Structured Public Learning: The “Balanced Scorecard” Approach to Strategy and Accountability

Reading1. Robert Kaplan & David Norton, The Balanced Scorecard: Translating Strategy into Action

1-19, 147-66 (1996)2. Jody Zall Kusek & Ray C. Rist, Ten Steps to a Results-Based Monitoring and Evaluation

System (World Bank 2004), http://www.oecd.org/dataoecd/23/27/35281194.pdf, at 11-27, 65-79

Discussion Questions1. What is the objective of the Balanced Score Card (“BSC”) and of allied measurement

strategies? What role do measurement and accountability play? What role do leading and lagging indicators play?

2. How are the Kaplan & Norton and Kuzek & Rist approaches to using measurement to improve success similar and different? Do you see any basis for preferring one over the other in general or in particular circumstances?

3. The title of this session suggests that there is an affinity between the BSC (and allied measurement strategies) and institutional learning. What affinities (or discontinuities) do you see? Why or why not might these approaches be equally or more clearly understood as management-based or performance-based tools? Market-mimicking tools? Bureaucratic oversight tools?

4. What critique of these tools might adherents of “craft” or “professionalism” make?

20

5. Imagine an institution with which you are familiar and how it might go about designing a BSC to advance its objectives—and exercise you will soon be asked to undertake for a state or local school system. What value do you see to the exercise? What difficulties? How would your answers differ if the institution in question had only recently been created or reorganized versus one that has been in existence and relatively unchanged for a long time?

Session #18: February 18, 2013The Role of Qualitative Review in Strategically Structured Public Learning: Using Leading Indicators to Build Expertise and Harness Street-Level Discretion

Reading1. John Braithwaite and Valerie Braithwaite, The Politics of Legalism: Rules versus

Standards in Nursing-Home Regulation, 4 Soc. & Leg. Stud. 307 (1995) 2. Kathleen Noonan, et al., Legal Accountability in the Service-Based Welfare System:

Lessons from Child Welfare Reform, 34 L. & Soc. Inq. 523, 523-26, 533-64 (2009)3. Daniel E. Ho, Fudging the Nudge: Information Disclosure and Restaurant Grading, 122

Yale L.J. 574 (2012) (excerpts)4. Anemona Hartocollis, New York City Ties Doctors’ Income to Quality of Care, N.Y. Times,

Jan. 11, 20135. 2013 Medicare Nursing Home evaluation rubric

Discussion Questions1. What differences are there in the qualitative review procedures used in support of:

nursing home regulation in the US as opposed to Australia; child welfare services in Utah and Alabama; restaurant health inspections in San Diego versus New York City; and quality of health care provided by doctors in public hospitals in New York?

2. To what extent do these differences map to differences in the governance model being implemented in each case, be it, for example, a model driven by professionalism, experimentalism, performance-management, or bureaucracy? To what extent do they map to the dichotomy between “rules” and “standards” that is discussed in the Noonan piece?

3. Is restaurant grading as practiced in San Diego and/or New York appropriately thought of as “qualitative” review at all? Why or why not? Does the use of a numerical scoring system automatically deny an evaluation scheme the advantages of being qualitative? What are those advantages?

4. Are the outcomes of these various kinds of inspections and reviews lagging or leading indicators—or something else entirely? Do the authors make any assumptions about which type of indicator each category of review or inspection should provide under the circumstances? What type of indicator do you believe each category or review or inspection should strive to be, and why?

5. In what ways do the different approaches to review and inspection serve or disserve the values and objectives promoted by Balanced Score Cards and allied measurement schemes?

21

6. A chief worry of doctors contemplating pay differences based on quality of care is that “they could be penalized for conditions they cannot control, including how clean the hospital floors are, the attentiveness of nurses and the availability of beds.” Are these conditions over which doctors have no control? Consider differences in background health conditions of the patients of hospitals in different neighborhoods. Might doctors in hospitals in impoverished neighborhoods where health conditions are generally worse fear that they will be penalized because they have chosen or been assigned to work with more challenging patients who are sicker when they are admitted and thus likely to have less favorable outcomes on average? Might this have the perverse effect of encouraging “better” doctors to prefer jobs in hospitals in more affluent neighborhoods, increasing inequities already faced by patients in poor neighborhoods? (This, of course, is a problem also faced in reviewing nursing homes, child welfare services, and schools, if perhaps not so clearly restaurants.) Are there any solutions for this problem other than not evaluating doctors based on quality?

7. Relatedly, continue, as you began to do in the last session, to imagine an institution with which you are familiar and for which you have been asked to designing a BSC. What model of qualitative review, if any, might you adopt?

Session #19: February 25, 2013 Qualitative Review of Schools and Educators

Reading1. Schools :

a. Helen F. Ladd, Education Inspectorate Systems in New Zealand and Netherlands: A Policy Note, 5 Ed. Fin. & Pol’y 378-92 (2010)

b. Quality Review Rubric 1: NYC Quality Review Rubric, Composite of 2007-2010c. Quality Review Rubric 2: NYC Quality Review Rubric, 2012-13

2. Educators :a. Robert Pianta & Hamre, Conceptualization, Measurement, and Improvement of

Classroom Processes: Standardized Observation Can Leverage Capacity, 38 Educational Researcher 109 (2009), at pp. 109 to very top of 112; very bottom of 113 to very top of 116

b. Charlotte Danielson, Evaluations that Help Teachers Learn, 68 The Effective Educator 35-39 (Dec. 2010/Jan. 2011)

c. Full Danielson Rubric, http://usny.nysed.gov/rttt/teachers-leaders/practicerubrics/Docs/Teachscape_Rubric.pdf (skim only)

d. NYC Abbreviated Danielson Rubric e. Teresa Watanabe & Stephen Ceasar, L.A.'s Revamped Teacher Evaluation System

Getting Mixed Grades, LA Times, Nov. 26, 2012

Discussion Questions1. What similarities and differences do you see when you compare:

a. Quality review of schools in New Zealand, the Netherlands, and New York

22

b. The two school quality review rubrics included in the readingc. The Pianta/Hamre and Danielson approaches to qualitative review of teachersd. The two Danielson rubrics included in the readinge. Quality review of schools and of teachers?

2. When you make the same comparisons, what links to different governance models (the usual: bureaucracy, performance management, professionalism, institutional learning, etc.) do you see?

3. Continuing the mental exercise from the last two classes, in thinking about building a balanced scorecard for an institution with which you are familiar, how valuable (or not) is each of these approaches to qualitative review as a model for a key component of your balanced scorecard? As a leading or lagging indicator?

4. Next week, we will discuss the empirical evidence on the relative value and correlation of test scores, student surveys, and observations of teachers as components of teacher-evaluation schemes. Based on what you already know, what uses, if any, would you make of qualitative review in the process of evaluating teachers?

5. How do the four rubrics in the reading fare on the criteria proposed by last week’s readings (Braithwaites, Noonan et al., Ho) for predicting the usefulness of qualitative review schemes?

6. “Validity” refers to whether measures used reflect the conditions or outcomes that are actually of interest. (E.g., thermometers provide a valid reflection of body temperature, but do not provide as valid a reflection of health; in the latter regard, that is, they are an imperfect “proxy” for health.) In your view, how “valid” (in this sense) are the measures proposed in the two school-review and two teacher-evaluation rubrics in the reading? What measures would be more valid? Can those measures be encapsulated in a quality review rubric for schools or teachers?

7. “Reliability” refers to how effectively a measurement scheme determines whether a particular condition (which condition may or may not be “valid” in the sense used above) is present. (E.g., thermometers may, in theory, provide a valid reflection of body temperature, but differences in the manufacture and use of thermometers may lead to unreliability –i.e., to different readings of the body temperature of the same person at the same time.) How “reliable” in this sense do you think the various methods of qualitative review described discussed are? This is the issue of “inter-rater reliability.” Is the likelihood of inter-rater unreliability” the death knell of qualitative review, or does it present some an opportunity of some sort for making particularly good use of such observations? How do the readings from last week (Braithwaites, Noonan et al., Ho) bear on this question?

Teacher-Observation Exercise: To be described in class. In preparation for class, please familiarize yourself with the NYC Abbreviated Danielson Rubric listed above.

Session #20: March 4, 2013Building a Better State or Local Balanced Scorecard for Public Education

Reading1. Example 1 : New York City’s Accountability System

23

a. Stacey Childress, et al., Managing for Results at the New York City Department of Education, in Jennifer O’Day, et al., Education Reform in New York City: Ambitious Change in the Nation’s Most Complex School System (2011), at pp. 7 (“Implementing Systems …”) - 15 of draft (reprise from Session 5)

b. Ledyard McFadden, New York’s Quest for Excellence, Educational Leadership (Oct. 2008)

c. New York City Accountability and Results, 2002-2012 d. NY Times Room for Debate, Making the Grade in NYC (Oct. 9, 2012)e. Optional : Educator Guide to New York City High School Progress Report (2011-2012)

2. Examples 2-5 : Differentiated Accountability and Support Systems included in ESEA Waiver applications for Louisiana, Massachusetts, New Mexico, and South Dakota, as described in Center for Education Policy, What Impact Will NCLB Waivers Have on the Consistency, Complexity and Transparency of State Accountability Systems? (2012), at 1-13, 15-18, 20-21, 24-27

3. Example 6 : The MET Project, Ensuring Fair and Reliable Measures of Effective Teaching: Culimating Findings from the MET Project’s Three-Year Study (Jan. 2013), http://metproject.org/downloads/MET_Ensuring_Fair_and_Reliable_Measures_Practitioner_Brief.pdf, at page-before-p. 1 (“About the MET Project”) and pp. 6-20

4. Other perspectives :a. The Pew Forum on Education Reform, Excerpts from “A Tribute to Al Shanker”

(reprinted from Ed Week, May 14, 1997), pp. 35-37 only (“Al Shanker Speaks …”) b. Eva L. Baker, Paul E. Barton, Linda Darling-Hammond, Edward Haertel, Helen F. Ladd,

Robert Linn, Diane Ravitch, Richard Rothstein, Richard Shavelson & Lorrie A. Sheppard, Problems with the Use of Student Test Scores to Evaluate Teachers (Economic Policy Institute 2010), at pp. 1-4

c. Deborah Kenney, Want to Ruin Teaching? Give Ratings, N.Y. Times, Oct. 14, 2012, http://www.nytimes.com/2012/10/15/opinion/want-to-ruin-teaching-give-ratings.html?_r=0

d. Motoko Rich, Beleaguered? Not Teachers, a Poll on ‘Well-Being’ Finds, NY Times, March 28, 2013

Writing Assignment As an MS Word or Excel attachment, prepare and email to CPRL@law.columbia.edu by Sunday March 3, 2013 at 1:00 p.m., an outline or sketch of a balanced scorecard for use by a state education department, school district, or school to hold school(s) and/or educators accountable for satisfying the relevant jurisdiction’s goals for promoting student learning and development. In your paper, please indicate:

a. The school or set of schools, and/or the set of educators to which your balanced scored card applies

b. The kinds of measures you consider to be “valid” for this purpose, as that term is defined in the Discussion Questions for Session 19.

Your outline or sketch need not address all of the following matters but should consider some of them, among other topics that you believe are important:

24

c. The appropriate mix of quantitative and qualitative (e.g., test score and other) measures, in any, and of leading and lagging indicators, if any

d. The tools you would use to determine how well schools and/or educators are performing on those measures

e. The weights to be accorded to different measuresf. Whether outcomes would or would not be publicly disclosedg. The stakes or consequences, if any, attached to particular outcomes

Discussion Questions:(Your writing assignment is not expected to address these questions explicitly, but you should be prepared to discuss them orally in class.)1. What considerations drove the design of your balanced score card?2. What role, if any, did different governance models play in your design?3. Did you follow any particular model or example (e.g., Kaplan & Norton; Kuzek & Rist; any of

those in the readings for this or earlier weeks), and why?4. How did you solve for the problem of student- and school-level differences that arise

because students and teachers are not randomly assigned to schools, leaving some schools and teachers with much more challenging student populations than others?

5. How do you imagine Albert Shanker would have gone about holding schools accountable? Which governance model(s) does Shanker appear to favor?

6. How would Baker et al. hold teachers accountable?7. To what extent do the recent findings of the MET Project answer the concerns of Baker et

al. and resolve the major issues in the long-running teacher-evaluation debate?

Session #21: March 11, 2013 A Conversation with Experts in the Mechanics of Teacher Evaluation

Reading1. Raj Chetty, John N. Friedman & Jonah Rockoff, The Long Term Impacts of Teachers,

Executive Summary and video (40 minutes): http://www.youtube.com/watch?v=vh22HBZMHvM

2. Teresa Wantanabe & Stephen Caesar, L.A. Times, Nov. 25, 2013, LA’s Revamped Teacher Evaluation System Getting Mixed Grades, http://articles.latimes.com/2012/nov/25/local/la-me-teacher-eval-20121125

3. Nicholas Kristof, The New Haven Experiment, N.Y. Times, February 15, 2012, http://www.nytimes.com/2012/02/16/opinion/kristof-the-new-haven-experiment.html?_r=0

4. Arne Duncan, The Tennessee Story, Huffington Post/Education (July 23, 2012), http://www.huffingtonpost.com/arne-duncan/the-tennessee-story_b_1695467.html

5. Dave Levin & Tim Daly, Better Teaching, Beyond Evaluations, NY Daily News Opinion, Jan. 4, 2013

6. Susan Keyock, Please Help Me Be a Better Teacher, NY Daily News Opinion, Jan. 2, 2013 7. Speaker BIOs

Assignment: This conversation will be partly about whether it is worthwhile, but mainly about how best, to implement a teacher-evaluation system that aims to benefit students and

25

teachers. Please come to class with three or four questions you would like to pose to experts on the choices they have made among the different approaches to teacher evaluation addressed in last week’s and this week’s reading, the outcomes they are seeing or at least are hoping to achieve the obstacles they face in implementing teacher-evaluation systems, and the solutions they advocate.

Spring Break: Week of March 18

Session #22-23: March 25, 2013 (extended session)Public Problem Solving and Democracy

Reading1. The case for a problem-solving democracy:

a. Christopher K. Ansell, Pragmatist Democracy, supra at 134-40, 166-83 b. Archon Fung, Deliberation and Social Conflict, in Empowered Participation:

Reinventing Urban Democracy pp. 173-97 (2004) 2. Doubts:

a. Jeffrey Henig et al., Parent and Community Engagement in NYC and the Sustainability Challenge for Urban Education Reform, in O’Day et al., supra, at 33-38, 43-45 (ending with first paragraph of “The Three Groups” section), 46, 48-54

b. Gordon Whitman, Making Accountability Work, 28 N.Y.U. Rev. L. & Soc. Change 361 (2003), at 361-67

3. Examples:a. Emma Brown, D.C. Parents Develop Alternatives to Chancellor’s School-Closure Plan,

Wash. Post, Jan. 1, 2013 b. Andrew Ujifusa, More States Consider 'Parent Trigger' Laws, Educ. Week, March 21,

2013, http://www.edweek.org/ew/articles/2013/03/21/26trigger_ep.h32.html?tkn=UOQFH9Hm5SOs00OBBWfJrDmX9j4UBzNGwxrM&cmp=clp-edweek

c. Bridgeport:i. Bridgeport Team’s 2011 Student Paper, The Theoretical Model pp. 42-53 ii. Michele Monar, Parents Take Lead in District’s Engagement Policy, Ed. Week

(Aug. 14, 2012) iii. Optional : CPRL, Parent Problem-Solving Groups Manual: A New Approach to

Parent Engagement (2012) c. Optional: Oliver C. Moles, Jr. & Arnold F. Fege, New Directions for Title 1 Family

Engagement: Lessons from the Past, Handbook on Family and Community Engagement, at 3 (title page), 5 (introductory § only), 8 (very end of page) – 12

Discussion Questions1. In a portion of Ansell we read earlier, he stated that, “Organizational transformation of

public agencies cannot easily occur without fundamental change in the relationship between agencies and democratic publics.” (p.17) Thus far, we have spent a lot of time talking about organizational transformation of public regimes, but not so much on how those regimes interact with their “democratic publics.” What kinds of changes in politics and democracy does Ansell have in mind? How do they differ from “normal politics”

26

that are familiar at the local, state, and federal levels in contexts such as public education, health care, deficit reduction, etc?

2. Ansell’s point suggests that for every governance model of how public agencies should be organized and administered, there is a corresponding approach to politics and democratic interaction. Taking that claim at face value for a moment, what form of politics and democracy would seem to fit best with bureaucracies? With performance management (New Public Management)? With professionalism/craft? With experimentalist regimes? In each case, what is the role of representation? Direct democracy? Interest or intermediary groups? Experts vs non-experts? Elections versus other forms of deliberation and participation?

3. Suppose, on the other hand, that, given the interplay of unevenly distributed resources, other economic realities, and divergent individual preferences, interest-group politics is the only stable or “natural” form of democratic interaction between the public and public actors and agencies. If that were the case—in other words, if “fundamental change in the relationship between agencies and democratic publics”—is not possible because one (the existing) form naturally predominates, would that mean that “organizational transformation of public agencies” of the sort Ansell contemplates is unlikely or impossible? Do you think interest-group politics are “natural” and immutable?

4. What form of politics and democracy do Professor Henig and colleagues suggest were necessary or best suited to the period during which the Bloomberg/Klein education reforms were taking in New York City (2003 to 2010)? What form of politics and democracy did Mayor Bloomberg and Chancellor Klein instead prefer? Do you agree with Henig et al. or Bloomberg-Klein about the best approach to engaging the public? Is there an alternative to those two competing approaches?

5. What forms of politics and democracy do you see being implemented in the “Traxton” example in Professor Fung’s article on community policing? In the Bridgeport, “parent working group” example? In the challenge issued by Washington, D.C. Schools Chancellor Kaya Henderson to parents opposing the closure of their schools? How are these approaches to politics and democracy similar or different? How promising are these approaches (and how effective were they in the particular contexts described) and what challenges and obstacles do the approaches face? What do these examples suggest about the possibility of overcoming the obstacles Whitman discusses to authentic stake-holder participation and problem-solving?

Sessions #24, 25 and 26 (first half): April 1, 8, and 15 (initial 50 minutes), 2013 Student Project Presentations: After providing the class with enough of a description of your project and its broader context to establish or reestablish a common understanding, please do one or both of the following:

1. Critically consider your project or its broader context from the perspective of experimentalism or, if you would prefer, from the perspective of one or more of the other governance models we've discussed. If you chose to focus on the institutional "context," not just the project, you may broaden it to include anything in between your

27

project and the outer boundary of the overall reform strategy your client NYCDOE, CSDE, or Zoom is pursuing. In what ways (if any) does the project or reform strategy proceed from, or is it grounded in, one or more of the governance model(s) we have discussed in class? How could the project or the broader reform strategy succeed more fully or efficiently if it were modified in some fashion along the lines suggested by one or another or a combination of the governance models? How does the project or the broader reform strategy deviate from the relevant governance model(s), and how do those deviations impede the project's or reform strategy's effectiveness? Overall, what could the leaders of your project or the broader reform strategy learn from your understanding of the theory underlying, and the best means of implementing, the relevant governance model(s)?

2. Critically consider experimentalism, or if you would prefer, one or more of the other governance models we've discussed from the perspective of your project or its broader context. Again, in what way does the project or strategy proceed from, or is it grounded in, one or more of those governance model(s)? What weaknesses, difficulties or challenges does your project or its broader context reveal about the theory of change and improvement that underlies the relevant governance model(s)? What light does the project or broader strategy shed on how easy or hard it is to implement the relevant governance model(s) effectively? How might those challenges lead you to modify the relevant governance model(s)? What do the challenges lead you to conclude about which model(s) would best facilitate the relevant reform strategy -- or about the validity and usefulness of one or more, or all, of the governance models in general? Overall, what could the proponents of the relevant governance model(s) learn from your project or its context?

Schedule:April 1 - NYC DOE teacher evaluation (first 50 minutes); communications (last 50 minutes)April 8th - CSDE quality review (first 55 minutes); school turnaround (last 45 minutes) April 15th - Zoom (first 50 minutes)

PLEASE LEAVE 20 MINUTES FOR Q&A

Readings and Discussion Questions; Electronic Copy of Presentation: 1. One week prior to your presentation, please email to Professor Liebman and Margaret

Symuleski a short set of background readings (no more than 20 pages) and a set of discussion questions for your presentation.

2. By 1 pm on the Sunday preceding your Monday session, please email to Professor Liebman an electronic version of your presentation.

Session #26: April 15, 2013 (final 55 minutes)Transitioning from Bureaucracy to the New Age Organization

28

Reading1. Ansell, supra, ch. 3, pp. 43-55, 61-622. Examples:

a. Aldine, Texas : Heather Zavadsky, Bringing School Reform to Scale (2009), Ch. 2, District Approach #1 (Aldine School District)

b. Norfolk, Virginia : Zavadsky, supra, Ch.6, District Approach #3 (Norfolk School District)c. New York City : Eric Nadelstern, The Evolution of School Support Networks in New

York City, Center on Reinventing Public Education Working Paper #2012-2 (2012), http://www.crpe.org/sites/default/files/pub_psdp_NYCnetworks_jun12.pdf [MAX, PLEASE LINK THIS ITEM TO OUR CITE]

i. Note from Prof. Liebman : I agree with Professor Nadelstern’s retrospective judgments about how the New York City reform could have been improved, but I would add one failing of the Networks that has kept them from becoming the key intermediary Nadelstern imagines: The networks had two somewhat competing objectives that had to be kept in balance, self-consciously, in a structured way: (1) support schools operationally (fending off the bureaucracy) and (2) promote and facilitate, without dictating steps schools were to take to assure, improvement of student achievement. Because the Networks were always palpably accountable for the former objective, because schools only opted into networks they trusted to serve them operationally, but Networks were not effectively held accountable for student improvement, the Networks took their service obligations more seriously than their achievement-facilitation role.

Writing Assignment: Write a ~2-page paper, due as a MS Word attachment emailed to CPRL@law.columbia.edu by Monday April 15, 2013 at 8:00 a.m., on your understanding of the “transformation strategy” (as defined below) that was implemented in Aldine, Norfolk, or New York City; what its strengths and weaknesses are (including compared to the transformation strategies used in the other cities); how you would have improved on that strategy; how useful you think the transformation strategy would be in different K-12 contexts (e.g., federal vs. state vs. local; large vs. small; urban vs. rural; collegial vs. conflictual; ones with strong or weak unions, etc.); how do Ansell’s insights bear on your judgments; and what does the example you focus on suggest about the validity of, or weaknesses in, Ansell’s analysis. By “transformation strategy,” I do not mean the governance or instructional strategy itself that each district adopted (e.g., “empowerment” in NYC), but instead the strategy the district adopted for moving the institution and its people to give up the old ways and embrace and adopt the new ways. For example, two districts that both decide to adopt a performance-management system focused on teacher-evaluation outcomes could make the transition to that common approach in very different ways, depending upon, for example, how abrupt or gradual the change is; how the change is introduced, explained and phased-in; who participated in deciding on a strategy and on the details of its implementation; etc. It is the latter set of choices that you should address in this paper.

Discussion Questions1. How would you characterize the transition strategy implemented in Aldine? Norfolk?

New York City? What is the appropriate scale for comparing transition strategies (e.g., evolution vs. revolution; top-down vs. bottom-up)?

2. Which transition strategy seems to be most promising or to have worked best? Why?

29

3. One claim that is sometimes made in support of strategies based on newly created charter schools or to justify more comprehensive (“start from scratch”), less incremental approaches to school turnaround is that it is easier to build learning institutions from scratch than to transform bureaucracies into them? Do you agree with that claim? What examples from the reading over the course of the year support or contradict this view? What is an example of a successful transformation of a previously bureaucratic institution?

4. Even if it is more difficult to transform bureaucracies than to replace them wholesale, there sometimes are no alternatives to transformation. In those cases, what steps are most likely to facilitate effective transformation? Are there kinds of organizations or organizational conditions that support or impede transformation?

5. How important are charismatic or determined leaders? Can existing organizations transform themselves, and sustain the changes, without charismatic leadership? What are the implications of your answer for the likely success of transformation efforts? Given example from the readings for this week and for the year that support your views on these questions.

Exercise: To be described in class.

Session #27: April 22, 2013Re-envisioning Schools and the Technology they Need: A Conversation with Sharren Bates, Chief Product Officer, inBloom; Larry Berger, Chief Executive Officer, Wireless Generation; Doug Jaffe, Senior Fellow, Regents Research Fund, New York State Education Department; and Jon Schnur, Executive Chairman, America Achieves

Reading:1. Panelist Bios

a. Sharren Batesb. Larry Bergerc. Doug Jaffed. Jon Schnur

2. Joel Rose, How to Break Free of Our 19 th -Century Factory-Model Education System , The Atlantic (May 9, 2012)

3. Jonathan Schorr & Deborah McGriff, Future Schools , Education Next, Summer 2011 4. Joanne Weiss, The Innovation Mismatch: “Smart Capital” and Education Innovation,

Harv. Bus. Rev. Blog, March 31, 2011, http://blogs.hbr.org/innovations-in-education/2011/03/the-innovation-mismatch-smart.html

5. Frank Catalano, How Will Student Data Be Used [in the Gates-Funded Shared Learning Infrasctuture], Mindshift, KQED, July 3, 2012, http://blogs.kqed.org/mindshift/2012/07/how-will-student-data-be-used/

6. Bill Tucker, The Country’s Most Ambitious Digital Learning Project , Education Next, Jan. 31, 2012

7. Alan Schwarz, Moorsville’s Shining Example (It’s Not Just About Laptops), N.Y. Times, Feb. 12, 2012

30

8. Matt Richtel, In Classroom of the Future, Stagnant Scores, N.Y. Times, September 3, 2011

9. Optional :a. Alison Bailey et al, “Unleashing the Potential of Technology in Education.” Boston

Consulting Group, August 2011, at 8-19b. Stacey Childress, Rethinking School, Harv. Bus. Rev., March 2012,

https://archive.harvardbusiness.org/cla/web/pl/product.seam?c=17835&i=17837&cs=60a41815cc7bdf2deee5c1c84002fdb3

c. Lawrence S. Bacow et al., Barriers to Adoption of Online Learning in U.S. Higher Education, Ithaka (May 2012), http://www.sr.ithaka.org/research-publications/barriers-adoption-online-learning-systems-us-higher-education

d. Cisco, Case Study, New York City School of One, http://www.cisco.com/web/strategy/docs/education/NewYorksSchoolofOne.pdf

e. Steve Rowley, Christensen’s Curve & The Digital Learning Revolution, Getting Smart, July 11, 2012, http://gettingsmart.com/blog/2012/07/christensen%E2%80%99s-curve-the-digital-learning-revolution/

Discussion Questions: Come to class with four or five questions for the panelists based on the readings and on your aspirations and concerns about changes in the design of schools, the role of educators, and the expected contributions of technology.

Session #28: April 29, 2013Summing Up; Looking Forward

Reading (tentative)1. http://opinionator.blogs.nytimes.com/2013/02/02/the-boys-at-the-back/?

src=me&ref=general (Boys at the Back)2. David Whitman, An Appeal to Authority, Education Next, Fall 20083. Reform School on the Bayou , Wall St. J., April 16, 20124. Catherine Gewertz, Districts Gird for Added Use of Nonfiction, Education Weekly, March

14, 20125. Review of “ All Together Now: Creating Middle-Class Schools through Public School

Choice” by Richard D. Kahlenberg. Harvard Educational Review6. Students First Policy

31