big-data-in-the-big-apple.pdf

Upload: pedro-henrique-almeida

Post on 05-Jul-2018

215 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    1/54

    BIG DATA

    IN THE BIGAPPLEThe lessons London canlearn from New York’sdata-driven approach tosmart cities

    Eddie Copeland

    Foreword by Mike Flowers

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    2/54

    AcknowledgementsTe au hor would like o hank Uni Par ner, Fuji su, for heir suppor and adviceon his piece of research.

    Par icular hanks are due o Mike Flowers, Chief Analy ics Officer for Enigma

    echnologies (htp://enigma.io/abou /) and founding direc or of he New YorkCi y Mayor’s Office of Da a Analy ics, for he suppor , ideas, ma erials and insighhe provided for his repor . Te remarkable model described in he following pagesis his. Special acknowledgemen is also due o Chris Corcoran, former Depu y Analy ics Officer wi h MODA, whose ma erials underpin much of he de ail in

    his s udy.Te au hor is also gra eful o he following people (and hose who prefer o

    remain anonymous) who kindly gave up heir ime o answer ques ions and shareheir perspec ives:

    Clark Vasey – Fuji su; Andrew Collinge, Paul Hodgson, Sara Kelly and

    Jamie Ra cliffe – Grea er London Au hori y; Nicholas O’Brien – New YorkCi y Mayor’s Office of Da a Analy ics; Dick Sorabji – London Councils; William Barker – Depar men for Communi ies and Local Governmen ;Ben Hawes – Depar men for Business, Innova ion and Skills; Chris Yiu –Scotish Council for Volun ary Organisa ions; Dr Andrew Hudson-Smi h,Cen re for Advanced Spa ial Analy ics UCL; Simon Reed – ranspor for London;Chris opher Gray – Accen ure; Geoff Marshall – Londonis ; Cameron Scot –Policy Exchange; Jamie urner.

    Te conclusions of his repor , along wi h any errors and omissions, remainhe au hor’s alone.

    About the authorEddie Copeland (@EddieACopeland) –Head of Technology PolicyEddie is responsible for leading research and crea ing policy recommenda ions

    on how echnology can deliver an innova ive digi al economy, a smar er publicsec or and a more connec ed socie y. Previously he has worked as Parliamen aryResearcher o Sir Alan Haselhurs , MP; Congressional in ern o Congressman

    om Pe ri and he Office of he Parliamen arians; Projec Manager of global Iinfras ruc ure projec s a Accen ure and Shell; Developmen Direc or of Te PerseSchool, Cambridge; and founder of web s ar -up, Orier Digi al. He blogs regularlyabou echnology policy issues a :policyby es.org.uk .

    http://enigma.io/abouthttp://localhost/var/www/apps/conversion/tmp/scratch_4/policybytes.org.ukhttp://localhost/var/www/apps/conversion/tmp/scratch_4/policybytes.org.ukhttp://enigma.io/about

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    3/54

    1Contents

    Contents

    Foreword 2

    Introduction: The data-driven city 4

    1 A lesson from Iraq 7

    2 The Mayor’s Ofce of Data Analytics 11

    3 The New York method 1 8

    4 New York’s lessons for London 2 6

    5 Why London needs a Mayor’s Ofce of Data Analytics 2 9

    6 How to create a London Mayor’s Ofce of Data Analytics 41

    Final thoughts 4 6

    Appendix 4 7

    Endnotes 4 9

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    4/54

    2 Big data in the big apple

    Foreword

    Do wha you can,Wi h wha you have,Where you are

    eddy Roosevel

    In 2009 I was given a s raigh orward mission by New York Ci y Mayor MichaelBloomberg: use da a o improve governmen services o he 8.5 millionNew Yorkers. His ollow up guidance was o do i inexpensively, wi h minimals aff, and make i impac ul and sus ainable. e cap ion a he op, scribbled ona pos -i no e on my governmen -issue compu er moni or on my rs day – pretymuch sums up he direc ion I was given, and i was enough.

    Ci ies are ooded wi h da a, bu da a by i sel is o litle value (a spreadsheeo raffic da a does no hing o ackle conges ion). o have impac i needs o be joined up. I requires people wi h he ime, skills and resources o in erpre andseek insigh s rom i . Above all, da a mus drive ac ion on ou comes ha reallymater o ci izens. a is why being da a-driven is no primarily a challengeo echnology; i is a challenge o direc ion and organiza ional leadership.

    New York Ci y ound such leadership in Mayor Bloomberg. Wi h he Mayor’s backing we used da a o improve cri ical services, empower ron line workers, and

    save no jus money bu lives. a work culmina ed in he crea ion o he Mayor’sOffice o Da a Analy ics (MODA) – he subjec o his repor . Almos six yearson, and under a new mayor wi h a very differen se o priori ies, MODA – andda a driven governmen in New York – is s ill going s rong. I has become a cen ralpar o Ci y Hall’s approach o governmen : enhancing areas as diverse as servicedelivery, emergency response imes, economic developmen , ax en orcemenand educa ion. In ac , i can be applied o help mee wha ever challenges matermos o a ci y.

    e only ci y ha rivals my affec ion or New York is London. Indeed, my wi eand I are so ond o i ha we named our rs born a er he a e Museum! I know

    he Bri ish capi al shares many o he same s reng hs and challenges o New YorkCi y. I know ha i oo wres les wi h he same impera ive o deliver more wi hless and o coordina e services across boroughs, depar men s and agencies. WhileI believe every ci y would bene rom puting i s da a o work, I believe Londonis a na ural place or i s own Mayor’s Office o Da a Analy ics.

    I have shared he many lessons we learned in New York Ci y o help in ormhis repor . Eddie Copeland and he Capi al Ci y Founda ion have provided

    a deep dive in o exac ly how he MODA model works and – mos impor an ly –how i could be adap ed or he specic con ex o London.

    is is no jus an idea or civic echnologis s or CIOs. I provides a playbookor how grea ci ies like New York and London can – indeed mus – be run

    in order o hrive and grow. I s success cri ically depends on he commi men

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    5/54

    3Foreword

    o ci y leaders o da a-driven principles – s ar ing wi h he Mayor, bu ex endingo every public official, civil servan and residen .

    People have he righ o expec and demand effec ive, ransparen governmen . We’ve shown ha , using da a, i is doable. e nex s ep is or leadership o decide

    o do i , and hen jus do i .

    Mike FlowersChie Analy ics Officer, Enigma echnologiesFounding Direc or, New York Ci y Mayor’s Office o Da a Analy ics

    June 2015

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    6/54

    Big data in the big apple4 Big data in the big apple

    IntroductionThe data-drivencity

    ‘I you can’ measure i , you can’ manage i ’ Michael Bloomberg

    Te age o he building.Te origin o he complain .Te value and size o he proper y.Whe her he building has a his ory o unpaid ax or mor gage liens.Whe her he Depar men o Buildings has received prior complain s abou

    he proper y.

    aken oge her – wi h some clever ma hs applied – hese are he predic iveindica ors or iden i ying some o he mos dangerous buildings in New York Ci y.

    Having made his or une providing da a-driven analy ics or he nancialsec or, as he 108 h Mayor o New York (2002–2014), Michael Bloomberg wan ed o prove ha he same echniques could bene ci ies, oo. One o his mossignican measures o ha end was o crea e he Mayor’s Office o Da a Analy ics(MODA). MODA is a small eam o analys s, based in Ci y Hall, who can combineand in erroga e da a rom numerous differen sources o increase he efficiencyand effec iveness o governmen opera ions and services.*

    MODA’s work on illegal conversions is illus ra ive o he impac i has had.New York Ci y’s Depar men o Buildings (DOB) responds o more han 18,000complain s o unlaw ul apar men conversions every year.1 ese are buildings ha

    have been illegally subdivided by rogue landlords. ey are over-crowded. eyare heal h hazards and re hazards. People ge ill in hem. Some imes hey even diein hem. In 2011, wo such buildings were he scenes o devas a ing res in whichve people, including young children, were killed.2

    I used o be ha suspec ed cases o illegal conversions were inves iga ed inhe same order as complain s came in via 311, he ho line number, websi e and

    app ha New Yorkers use o nd in orma ion and repor problems.3 Ou o all hecomplain s received, approxima ely 8% (1,400 per year) accura ely iden i y an illegalapar men where condi ions are so dangerous ha he Depar men o Buildings has

    o issue a vaca e order.** DOB asked MODA o crea e a model ha could analysehe complain s received via 311 and ag hose mos likely o iden i y hese highes

    risk proper ies, so hey could be inspec ed wi hin 48 hours.4 By analysing differenda ase s, MODA managed o iden i y predic ive indic ors o he mos dangerous

    * See Appendix or heExecu ive Order haes ablished he Mayor’sOffice o Da a Analy ics

    ** e reason or heseemingly low accuracyo complain s is haneighbours end o reporsuspec ed dangerous buildings based onex ernal evidence, suchas seeing large amoun s orubbish or builders working wi hou any obvious permi .

    is comes in con ras ocomplain s abou a brokens ree ligh , which end o be very accura e as i is clear

    o see whe her or no a lighis working

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    7/54

    5Introduction

    buildings (lis ed a he s ar o his chap er). ey were hen able o crea e a risk-pre-dic ion model ha enables DOB inspec ors o nd over 70% o he wors buildings by arge ing jus 30% o hem.5 A 233% improvemen ha saves no jus money bulives.

    Lessons for London?

    e example o he Mayor’s Office o Da a Analy ics has been highligh ed in severalrepor s and ar icles by Policy Exchange.6 However, hese have ou lined he modelonly in very high-level erms. e purpose o he presen repor is o provide adeep-dive in o New York’s pioneering da a me hods and o pu orward he case

    ha a similar approach could bene London, oo.e wo ci ies do, a er all, have much in common. ey have comparable

    popula ions wi h around eigh and a hal million residen s apiece. Bo h areleading nancial cen res and home o he headquar ers o many o he world’slarges en erprises. Bo h draw millions o visi ors rom around he world oexperience heir cul ural, ar is ic and hea rical deligh s. Bo h are es beds or

    new poli ical hinking and pioneering urban ini ia ives ( hink o New York Ci y’s‘broken windows’ approach o policing or London’s Oys er Cards and Conges ionCharge).7, 8, 9 ey appear o be he very embodimen o Harvard Pro essor EdwardGlaeser’s riumph o Te Ci y.10

    In spi e – or perhaps ra herbecause– o heir success, London and New York alsoace a number o similar policy challenges. Wi h heir expanding popula ions comes

    grea er pressure on housing, ranspor in ras ruc ure and public services. And whileheir respec ive economies have boomed, he wo me ropolises have seen signican

    reduc ions in he budge s available o heir ci y au hori ies.11 e resul on bo h sideso he A lan ic has been a pressing need o do much more wi h much less.

    Ye or all heir similari ies, applying he MODA model o he Bri ish capi al will no be a simple case o copy and pas e. In very signican ways, London isno New York. e mayoral y o he Big Apple is one o he mos power ul suchposi ions in he world, wi h con rol over hiring and ring he heads o he ci y’s keyagencies (such as police and schools), while also seting an annual budge in excesso £45 billion. In comparison, London’s mayor has rela ively ew execu ive powersand a budge o £14 billion.12 London has 33 boroughs o New York Ci y’s ve,and he ormer are more au onomous. Fewer direc personal and business axes arecollec ed and re ained locally in he U (London raises jus 26% o wha i spendscompared wi h 69% in New York).13 Differen rules and regula ions on da a sharingand da a pro ec ion apply. e lis goes on.

    ere ore, ra her han rying o impor he model wholesale, his repor aimso dis il he core elemen s o New York’s da a success and ou line a means or hemo be adap ed or he specic con ex o London. I describes he measures ha

    would need o be aken by cen ral governmen , he Mayor o London, LondonBorough Councils and he wider London public sec or o make his possible.

    e repor also highligh s how many o he lessons New York offers Londonrun coun er o common wisdom abou how o re orm public services or crea e a‘smar ci y’. As Chap er 4 explains, he New York MODA model:• Does no require ex ensive (and expensive) new echnology or placing sensors

    on every s ree , bu on making beter use o da a ha is already collec ed.• Does no involve undamen ally changing he na ure o ac ivi ies conduc ed by ron -line s aff, bu in elligen ly priori ising heir work.

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    8/54

    6 Big data in the big apple

    • Does no insis on da a puri y and open s andards (common orma sand schemas or recording da a), bu on da acomple eness.*

    • Does no en ail gambling on a radical new ‘smar ci y’ business model, bu on es ing and scaling ideas ha each provide a proven re urn oninves men (ROI).

    • Depends less on echnological exper ise and ar more on s rong poli ical

    leadership rom he mos senior gures in ci y and local governmen .• Is no abou preparing or some dis an vision o u ure urban in elligence, bu ins ead aking simple bu concre e s eps ha could s ar omorrow.

    Overall, his repor makes jus one – albei ar-reaching – recommenda ion: haLondon should es ablish i s own Mayor’s Office o Da a Analy ics in Ci y Hall osuppor he Mayor, he Grea er London Au hori y (GLA), London boroughs and

    he wider London public sec or in harnessing da a o: deliver beter public services;reduce he cos o local governmen ; and accelera e business grow h in he capi al.

    e ollowing chap ers explain wha his would en ail, why London should atempi , and – mos impor an ly – exac ly how i could be done.

    * ‘Da a comple eness’ meanshaving he ull se o da acollec ed abou a par icularissue, ra her han jus asample. e impor ance or big da a analysis o having

    he comple e da ase (some-imes re erred o as ‘n=all’)

    has been explored by Vik orMayer-Schönberger and

    enne h Cukier in Big Da a: A Revolu ion TaWill rans orm HowWe Live, Work, and Tink, John Murray, 2013

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    9/54

    7A lesson from Iraq

    A lessonfrom Iraq

    ‘People have a righ o expec heir governmen o be as well-managedas he mos modern organiza ions in he world.’ Mike Flowers quo ing Mayor Michael Bloomberg’s philosophy o governance.14

    Starting bit by bitI was in 2009 – he s ar o Mayor Michael Bloomberg’s hird erm in office –

    ha he s ory o he Mayor’s Office o Da a Analy ics begins. I was in ha yearha a 40-year-old lawyer called Mike Flowers joined Ci y Hall. Appoin ed by

    John Feinbat, New York Ci y’s hen Criminal Jus ice Co-ordina or, Flowers’ ini ial brie was o head up he ci y’s nancial crimes ask orce in he wake o he 2008economic mel down.15 He had previously spen wo years inves iga ing nancialcrimes or he US Sena e Permanen Subcommitee on Inves iga ions. Prior o ha ,he had been par o he legal eam ha handled he rial o Saddam Hussein.

    I was Flowers’ experience in Iraq ha persuaded him ha using da a couldmake a difference o his work or Ci y Hall. He had, repor S ephen Goldsmi h

    and Susan Craw ord in heir book,Te Responsive Ci y, been ‘inspired by he young econome ricians employed by he army’s Join Improvised ExplosiveDevice De ea Organisa ion (JIEDDO) who crunched da a on pas encoun ers wi h IEDs (Improvised Explosive Devices) in order o nd he sa es possiblerou e hrough Baghdad on any given day.’16 I da a could help save lives in hos ile

    erri ory, he hough , wha else could i be used or?

    Financial crimes and misdemeanours Wi h no exis ing job descrip ion or me hod o work rom, Flowers began his

    enure a Ci y Hall simply rying o unders and who did wha , and who had wha

    in orma ion. A er several mon hs he had walls covered in no es de ailing all hein orma ion he ci y held ha migh be relevan o nancial crime. By 2010 –and having used Craigslis o hire a young gradua e called Ben Dean as his rsda a analys – he realised one aspec o nancial crime hey could address wi hda a was how o beter arge mor gage raud inves iga ions. e revela ion came

    rom examining pas cases.

    ‘[Ben] Dean looked a he da a on abou 150 mor gage rauds… wi h one ques-ion in mind: “wha did he ci y know in i s proper y and building records a heime his raud happened ha could ell us ha his ransac ion needs scru iny?”’17

    eir research ound ha i was indeed possible o iden i y pieces o in orma ionabou nancial ransac ions which, when brough oge her, could predic hose

    01

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    10/54

    8 Big data in the big apple

    wi h he grea es likelihood o being raudulen . However, he work did noproceed much ur her. I urned ou ha banks were unwilling o prosecu e aileddeb ors due o ears over i s po en ial o undermine condence in mor gage-backedsecuri ies a a ime when condence was already in shor supply.18 Ye he effor s oFlowers and Dean in developing heir predic ion model were no was ed. ough

    he ini ial ocus on nancial crime may have come o an end, he MODA me hod-

    ology o using da a analy ics o address he ci y’s problems was rmly es ablished. And i s nex applica ion really would save lives.

    Fighting res before they starto unders and how a ci y really works, jus ask ron -line s aff, says Flowers. He

    and his eam spen mon hs shadowing ron -line workers o see how hey didheir jobs, observing he challenges hey aced and iden i ying he elemen s o

    knowledge ha in ormed heir work. eir ime wi h inspec ors in New York Ci y’sFire Depar men (FDNY) provides a good example. In Flowers’ own words:

    ‘Ve eran re gh ers know wha dangerous buildings look like. ey knowhow impor an i is or a building o have an operable sprinkler sys em,he impac ha he improved building and re codes have had over cen uries

    o cons ruc ion, and wha ype o business ac ivi y is mos requen ly corre-la ed wi h dangerous res. I you ask a ve eran o he re depar men , heirgu can give you a lis o cri eria or dangerous buildings nearly as effec ivelyas a s a is ical regression.’19

    Flowers’ challenge was o decipher wha hose cri eria were and o see i da a couldcomplemen and s reng hen re gh ers’ na ural in ui ion in iden i ying dangerous buildings. Could accessing da ase s held by organisa ions ou side he FireDepar men be use ul o heir work? Could he ac ors underpinning re gh ers’gu ins inc s ( he age o he building, he ype o business, e c.) be quan ied moreprecisely wi h beter da a? Where previous versions o he ci y’s re risk model had weigh ed he cri eria based on ocus group discussions wi h re gh ers, MODA

    es ed hem agains da a rom ac ual res o calcula e heir rela ive impor ance.Using his in orma ion, MODA crea ed a da a-driven model ha could

    predic which buildings were mos a risk o having serious res wi h ar grea eraccuracy. Figure 1A illus ra es he difference. On he le is a map showing heresul s o he original re predic ion model (based on ocus group discussions).

    e map in he cen re shows he predic ed loca ion o res according o MODA’smodel. On he ar righ -hand side is where pas res had ac ually occurred. econ ras is s riking. Whereas he old model ailed o iden i y high-risk zones inareas such as Harlem, Down own Manhatan and he Rockaways, he new model very closely reec ed reali y.20

    Deriving hese insigh s was ar rom being a mere academic exercise. eyprovided in elligence ha could be ac ed upon immedia ely. Every year, FDNYproac ively inspec s more han 25,000 buildings ha i believes may be a risko u ure res. Jus as wi h he visi s o suspec ed illegally conver ed apar men s,

    here is considerable bene o be gained i he mos dangerous buildings can be priori ised or inspec ion. As Figure 1B shows, prior o applying MODA’s

    da a-driven analysis, he rs 25% o FDNY inspec ions ypically resul ed in 21%o he mos severe viola ions being discovered. Using MODA’s predic ion model,

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    11/54

    9A lesson from Iraq

    he rs 25% o inspec ions now resul in more han 70% being discovered. oughhe o al number o inspec ions remains he same (FDNY is obliged o inves iga e

    every complain i receives), by going o he mos dangerous buildings rs , hedepar men is able o ake early ac ion o reduce he number o days ha New Yorkers are a serious risk.

    Figure 1A: Location of res as predicted before and after the useof MODA’s model

    Original model

    Original model over-predicted fires inDowntown Brooklyn,Park Slope andBay Ridge

    Updated model accuratelyreflects risks in WestBronx, Downtown,and Far Rockaways

    Observed fire frequency,2011 to present

    Updated model Actual fires

    Source: NYC – Mayor’s Ofce of Data Analytics, ‘Annual Report 2013’, December 2013, p.14

    Figure 1B: Percentage of dangerous buildings identied in rst 25%of inspections

    100

    0 100755025 0 100755025

    75

    50

    25

    0

    % o

    f s e v e r e v

    i o l a t i o n s

    f o u n

    d

    ROC curve pre-analysis

    First 25% of inspec-tions yield 21% ofsevere violations

    First 25% of inspec-tions yield 71% ofsevere violations

    ROC curve post-analysis

    100

    75

    50

    25

    0

    % o

    f s e v e r e v

    i o l a t i o n s

    f o u n

    d

    % of fire inspections conducted % of fire inspections conducted

    Source: NYC – Mayor’s Ofce of Data Analytics, ‘Annual Report 2013’, December 2013, p.15

    Crucial o he success o his ini ia ive, he improvemen s in alloca ing inspec ors’ime was achieved wi hou changing he work o ron -line s aff. Having provedheir re risk predic ion model worked, MODA was able o use echnology o

    au oma e he process o reviewing and priori ising 311 complain s. Each morning,re depar men building inspec ors would s ill receive a lis o proper ies oinves iga e ha day. e only difference was ha now ha lis was pre-priori ised

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    12/54

    10 Big data in the big apple

    o ocus on he mos dangerous buildings rs . As a resul , he work o MODA won he suppor o he Fire Depar men as i led o maximum improvemen in

    heir service wi h almos no disrup ion o day- o-day business. e ron -line s affliked i because i helped hem do heir jobs even beter han be ore. e Ci y andNew Yorkers liked i because i saved lives and made hem eel sa er.

    e MODA model was proven – nex i had o be ormalised.

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    13/54

    11The Mayor’s Ofce of Data Analytics

    The Mayor’sOfce of DataAnalytics

    ‘We believe his represen s a paradigma ic shi in how governmen works – one hais guided no only by da a, bu also he exper ise, experience, people and his orybehind he da a.’ Mike Flowers21

    Executive Order 306e work o Flowers and his eam became official in April 2013 when Mayor

    Michael Bloomberg signed Execu ive Order 306 (see Appendix) ormallyes ablishing he Mayor’s Office o Da a Analy ics (MODA). Mike Flowers becameCi y Hall’s rs Chie Analy ics Officer (CAO), a senior role repor ing direc ly o

    he Mayor. Along wi h he Ci y’s Chie In orma ion and Innova ion Officer (CIO)and he Chie Digi al Officer (CDO), he Chie Analy ics Officer serves on he

    Mayor’s echnology council, bringing da a-driven analy ical rigour in o all aspec so he ci y’s opera ions.22

    Figure 2A: NYC City Hall Technology Organisation

    Mayor

    ChiefInformation

    Officer

    ChiefAnalyticsOfficer

    ChiefDigitalOfficer

    Source: NYC – Mayor’s Ofce of Data Analytics, ‘Annual Report 2013’, December 2013, p.7

    02

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    14/54

    12 Big data in the big apple

    The teame MODA eam i sel is modes in size wi h jus nine people, including analys s

    and echnical and adminis ra ive suppor s aff. e analys s have a mix ure o s a is-ical, economic, and compu er science backgrounds.23 In 2013 he roles included:

    Chie Analy ics Officer and Chie Open Pla orm Officer; Depu y Direc or; Chieo S aff; Chie Analys ; Chie Programmer; Analys ; Special Advisor o he CAO;

    echnology Advisor o he CAO; and Senior Advisor o he CAO.24

    Far romcrea ing a signican addi ional layer o bureaucracy, he New York MODA modelis lean and highly efficien .

    What MODA does todayMODA’s remi has expanded over ime. Since 2013, he eam’s work can broadly be divided in o seven overlapping areas:25

    1 Helping New York City agencies improve the delivery of servicesMODA analyses da a o spo previously unknown paterns and rela ionships

    ha lead o beter decisions and help alloca e he ci y’s scarce resources moreeffec ively.26 A er pilo ing and es ing heir da a models o conrm ha heycan improve he delivery o a par icular service, MODA uses echnology(see nex sec ion) o au oma e he process so ha services can be enhancedon a permanen basis.

    In addi ion o i s work on illegally conver ed apar men s and assessing buildings’ re risk, a ur her example o his unc ion is how MODA helped

    he New York Ci y Depar men o Environmen al Pro ec ion (DEP), whichis responsible or main aining he ci y’s 6,000 miles o sewers.27 DEP wan ed

    o crack down on res auran s ha were illegally pouring cooking oil in o sewers,

    which is hough o be responsible or more han hal o New York’s cloggeddrains. MODA used da a rom he Business In egri y Commission, a ci yagency ha cer ies ha all local res auran s have paid or a service o legallydispose o heir grease. By comparing res auran s ha had no paid or sucha service wi h geo-spa ial da a on he sewers, MODA was able o hand DEPinspec ors a lis o s a is ically likely suspec s. e resul was a 95% successra e in racking down he offending res auran s.28

    2 Sharing data with NYC agencies and encouraging best practicein data analysis

    e da a ha MODA collec s and uses is also made available o s aff workingin 40 o her ci y agencies, enabling hem o combine i wi h heir owndepar men ’s da a o improve heir decision making.* Such da a sharing workson a s ric principle o reciproci y: ex ernal agencies can access da a collec ed by MODAon condi ion ha hey rs share heir own da a.

    MODA addi ionally provides raining o help NYC agencies developheir own da a analy ics capabili y. In he case o he re risk based inspec ion

    sys em (RBIS) ou lined in he previous chap er, MODA helped he FireDepar men se up and rain i s own da a analy ics eam. a eam has since

    aken on responsibili y or developing he RBIS model. e resul is ha , ra herhan keeping da a skills concen ra ed in one cen ral eam, MODA is ins ead

    a ca alys or promo ing and ex ending he use o da a analy ics hroughouhe ci y’s ins i u ions.

    * ose s aff may include:Business analys s – repor ingon he day- o-day opera ionso agencies; GIS (GeographicIn orma ion Sys ems) ana-lys s – ocusing on visualising

    he opera ions o he Ci y;Researchers – conduc ings udies on ci y issues andper ormance; and Compu erscience exper s – main aining

    he Ci y’s I in ras ruc ure.Source: NYC – Mayor’sOffice o Da a Analy ics,‘Annual Repor 2013’,December 2013, p.8–10

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    15/54

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    16/54

    14 Big data in the big apple

    6 Aiding disaster response and recoveryFollowing he devas a ion caused by Hurricane Sandy in la e Oc ober 2012,Ci y Hall realised here was no publicly-held map lis ing all o he ci y’s businesses. Consequen ly, i was ex remely challenging or officials o know which businesses were mos likely o have suffered rom problems such asooding, and here ore required suppor o ge back up and running. MODA brough oge her records rom six differen da abases o comple e he map.

    e resul s are highligh ed in Figure 2B.

    Figure 2B: Location of businesses in New York City

    Business map provided by3rd party data aggregator

    Original map + city data

    This heat map shows the density of commercial space in New York City. The map on theleft, formed with an initial set of third party commercial data, missed much of CentralBrooklyn and Eastern Queens. The updated map on the right, produced by MODA withadditional city data, presents a clearer picture of commercial activity.

    Source: NYC – Mayor’s Ofce of Data Analytics, ‘Annual Report 2013’, December 2013, p.19

    7 Providing open dataMODA leads New York Ci y’s effor s on providing open da a – da a ha is

    reely available o be used and reused or commercial or non-commercialpurposes by ci izens, businesses and hird sec or organisa ions.32 In New York,open da a is rea ed as a subse o he da a collec ed or use and analysis by he

    ci y’s depar men s and agencies. New York’s open da a por al can be viewed a :htps://da a.ci yo newyork.us.

    MODA’s approach to technologyReplica ing he ac ivi ies o he Mayor’s Office o Da a Analy ics does no dependon any specic ool, me hod or echnology pla orm.33 e New York eam began by using litle more han Excel spreadshee s and wha ever old da a hey could ge

    heir hands on. However, as heir work has grown in sophis ica ion (making inecessary o au oma e he process o da a collec ion and analysis), so oo have he

    ools hey use. A brie overview o he echnology ha enables MODA’s work helps

    explain how i unc ions. I s primary pieces are essen ially a da abase (Da aBridge)and a digi al ne work o exchange da a be ween agencies (DEEP).

    https://data.cityofnewyork.us/https://data.cityofnewyork.us/

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    17/54

    15The Mayor’s Ofce of Data Analytics

    Figure 2C: Data collected by agency, and agency system, and fedinto DataBridge

    Source: NYC – Mayor’s Ofce of Data Analytics, ‘Annual Report 2013’, December 2013, p.9

    MOEC

    CEO

    DOHMH

    DSNY

    NWS

    DOF

    DCP

    OMB

    FDNY

    DOT

    311

    DCA

    ACRIS and Richmond Co.NOV images and dispositionNYCServProperty charges

    Property assessmentOwnershipTax liens

    Service request (SCAN)DSNY service requests (TAG)Vehicle information

    Inspections (Stockpile, Transfer Stations)Work completeFleet management

    Permits/licenses (CAMIS, CCATS)Complaints (FSE, CCATS)Violations (FSE, CCATS)Inspections (FSE, CCATS)Service requests (PCS)

    BFIInspections (PPIMS, Child care, CBIDAS)

    Violations (PPIMS, Child care, CBIDAS)Permits (PPIMS)Cert. of fitnessComplaints (PPIMS, Child care, CBIDAS)

    Implemented

    Incoming

    Service requests (TERMS sidewalls)Service requests (TERMS streetlights)Service requests (MOSAICS)InspectionsPermits

    KPbs (PMA)Universal intake (CSMS)Enterprise Correspondence (CSMS)Inquiries (CSMS)NYC serviceService requests (CSMS)DOB appointments (CSMS)

    KPb

    CEQR (CEQR-View)

    STIMS

    ULURPZoningBINBBL

    Weather

    CAMIS

    Notice of violation

    DOB

    SBS

    DOITT

    ECB

    OPS

    HRO

    LPC

    NYCHA

    HPD

    DEP

    Permits (ARTS, CATS, WSPS)Violations (ARTS, CATS, WSPS)

    Complaints (Hansen)Inspections (ARTS, DIRS, Hansen)

    Water BillsService Requirements (Hansen)

    311 and service desk telephonyWeb analytics (WebTrends)

    IT service management (Remedy)Project management (Clarity)

    Fax (RightFax)Emails (MessageStats)

    NYC.gov portal

    CONEDFEMA

    Building assessmentHoteling (King)

    Rapid repair and statusLIPA

    Landmark status

    ComplaintsPermits

    InspectionsViolations

    SubsidyLocal Law 4 (LL4)OMO violations

    Service requests (Info)Building registration

    HMC Violations (HPD Info)HMC Inspections (HPD Info)

    Complaints and referrals (HPD Info)Portal log

    Placard

    Certificate of OccupancyLicenses

    Jobs, permits, Req. itemsComplaints

    Service requests (BIS)Violations/actionsInspections (PIPES)

    Forms

    Service requests (EIS)

    Licenses (Bex)

    Sanitation scorecard (TES)SCOUT (TES)CORE (TEST)

    DataBridge

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    18/54

    16 Big data in the big apple

    DataBridge As in London, da a in New York Ci y is held by dozens o differen organisa ions,using hundreds o differen I sys ems o varying ypes and ages. In order o beable o run i s da a models and share i s insigh s, MODA needs o be able o bring

    oge her da a rom around 40 differen ci y agencies in o a single da abase in amanner ha complies wi h da a privacy and legal obliga ions.

    o au oma ically collec da a rom hose differen agencies via secure APIs(Applica ion Programming In er aces), MODA used Da aShare – an exis ing da arans er sys em crea ed o rack prisoners hrough he criminal jus ice sys em. eeam hen harnessed he spare capaci y in 311’s da abase (Ci ywide Per ormance

    Repor ing), o cons ruc a power ul bu agile sys em called Da aBridge.34 MODAdescribes Da aBridge as:

    ‘a combina ion o echnologies, including da abase managemen and s a is icalanalysis ools. e ounda ion is an analy ics da a warehouse/reposi ory wi ha sui e o analy ic and da a usion ools, making he da a available no jus o

    MODA bu o analys s across he Ci y’.35

    Figure 2C shows how da a rom 40 differen agencies and heir various sys emsis brough oge her in Da aBridge.36

    DEEPI Da aBridge resembles a ‘hub and spoke’ sys em wi h differen agencies all con-nec ing o a cen ral da a warehouse, he Da a Elemen Exchange Program (DEEP)is more like a spider’s web: connec ing each depar men o each o her. Ins iga ed by MODA, DEEP enables ci y agencies o exchange in orma ion securely. I

    replaces ou da ed me hods o rans erring da a, such as email or ax, which are bo hime consuming and inefficien . NYC agencies using DEEP are able o send andreceive in orma ion in a consis en orma , on a regular, scheduled basis. One o

    he mos impor an ea ures ha DEEP allows is real- ime exchange o in orma ion.o da e, DEEP has implemen ed more han 200 au oma ed exchanges be ween

    30 ci y agencies, ex ernal vendors and o her governmen depar men s.37

    MODA’s approach to dataGeo-tagging (Geo-coding) A undamen al requiremen o MODA’s approach o da a is ha in orma ionsourced rom differen ci y organisa ions can be overlaid and ploted on hesame maps. is is made challenging by he ac ha agencies in New York Ci yuse several differen ways o geo- ag heir records (i.e. speci y heir loca ion).For example, some may record a piece o in orma ion agains a s ree address, whileo hers may use he block, grid re erence or ZIP code. MODA developed a sys em

    ha can link oge her records using differen geo- ags, so ha he in orma ion heldin Da aBridge is connec ed o a common loca ion iden ier. is allows da a rommul iple agencies o be easily merged and used oge her in analysis.38

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    19/54

    17The Mayor’s Ofce of Data Analytics

    Benets of MODABeyond i s ac ivi ies in op imising services, MODA’s work has delivered hreeaddi ional core bene s:

    1 Financial savings through data sharingUsing DEEP o au oma e he ow o in orma ion be ween 30 ci y agencieshas resul ed in angible improvemen s in ci y opera ions and cos savings.For example, a er each inspec ion by New York’s Fire Depar men , in orma ionabou specic viola ions has o be sen o he Environmen al Con rol Board(ECB).* Prior o he implemen a ion o DEEP his was a manual process

    ha ook up o a mon h and was prone o include errors. oday, he processhas been au oma ed, speeding i up o jus 1–2 days, elimina ing errors andresul ing in increased revenue collec ion o $1.2 million per year.

    2 Increased joined up working across departments and agenciesIn he pas , when agencies ried o address heir depar men al challenges anddesign services, hey were o en res ric ed o using he in orma ion held wi hin

    heir organisa ion. is made i hard o know wha was going on in he res ohe ci y ha could po en ially help solve a problem. Wi h access o Da aBridge,

    analys s in each agency can now use da a rom across he ci y o crea e a muchmore accura e pic ure o wha is going on. Beter in orma ion leads o beteranalysis, which in urn leads o beter decision making by agency leaders.39

    3 Spreading skills in data-driven management of public servicesMODA has recognised ha i canno exis as an island o da a exper ise in ano herwise da a-ignoran ci y. o ha end, MODA collabora ed wi h he Cen re

    or Urban Science and Progress (NYC CUSP –htp://cusp.nyu.edu ) o es ab-lish a series o raining workshops or ci y analys s – effec ively ini ia ing acourse in ‘Ci ywide Analy ics 101’. is eaches officials o use da a o improve

    heir day- o-day responsibili ies and s ra egic decision-making. For example,he Depar men o Finance (DOF) has used Da aBridge o beter unders andax raud. Similarly, he Sheriff ’s Office has used Da aBridge o rack illegal

    cigarete impor a ion rings, developing heir own in-house da a eam.40

    * e Environmen alCon rol Board (ECB)is a ype o cour called

    an adminis ra ive ribunal.I is like a cour , bu isno par o he s a e coursys em. A he Office o Adminis ra ive rials andHearings’ Environmen alCon rol Board, HearingOfficers hear cases onpo en ial viola ions o helaws ha pro ec he Ci y’squali y o li e. CommonECB viola ions include:dir y sidewalks, unleasheddogs, loi ering, noise, public

    indecency, rollerblading ormo orcycling in a orbiddenarea, sidewalk obs ruc ionand roden and pes con rol

    http://cusp.nyu.edu/http://cusp.nyu.edu/

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    20/54

    18 Big data in the big apple

    The New Yorkmethod

    ‘A ocus on ou comes is o en los in he discussion o big da a because i is so fequen ly an a er hough . We have a huge re hose o in orma ion, bu even a rehose is only valuable when i ’s poin ed a a re.’ Mike Flowers41

    Having laid ou he de ails o he organisa ion, responsibili ies, echnology and

    da a me hods o New York’s Mayor’s Office o Da a Analy ics, he nex impor ans ep is o unders and how hey go abou applying da a o address a specicchallenge. e ollowing wo chap ers dis il he key me hods (Chap er 3) and prin-ciples (Chap er 4) ha underlie MODA’s work so ha a new model can be crea ed

    or London (Chap ers 5 and 6).

    MODA’s 10 step model Arguably he mos impac ul aspec o MODA’s work has been he way i hassuppor ed service delivery eams (SD ), such as re gh ers, building inspec orsand environmen al pro ec ion officers, o bring da a-driven analysis o improve

    or priori ise heir ac ivi ies. e process hey use when approaching any newproblem – and which should orm he ramework or a London MODA eam –can be summarised in he ollowing 10 s eps (which are explored hrough ade ailed case s udy onpage 21):42

    1 Understand how day-to-day operations worke MODA eam spends ime shadowing ron -line SD s aff o unders and:

    a) he na ure o he service hey provide; b) how resources are alloca ed,scheduled and delivered; c) he ac ors ha go in o he priori isa ion o deliveryo he service; and d) how da a is recorded in he SD ’s I sys em(s). When

    he rs arrived a Ci y Hall, Mike Flowers spen six mon hs wi h ron -line s aff(which are explored hrough a de ailed case s udy onpage 8) o experience

    heir ac ivi ies or himsel and o unders and he da a hey used and recorded.

    2 Identify areas where data could helpMODA examines he da a ha is used and recorded during he process o deliv-ering a service. e eam hen considers how he service could be improved( or example, by being beter able o alloca e a scarce resource, such asinspec ors’ ime) and ries o iden i y wha in orma ion i would ake o achieve

    ha aim.

    03

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    21/54

    19The New York method

    Figure 3A: MODA’s 10 step process for improving a service withdata analytics

    Operationalchallenge

    Scopingdiscussion

    Projectplan

    Does MODAhave the

    data?

    Review andfeedback

    Identify dataand sample

    to MODA

    Identify dataand sample

    to MODA

    NONO

    YES

    MOUand data

    integration

    Client reviewand/or pilot

    Service Delivery team

    MODA

    3rd party data

    Analyticssolution

    Operationalimplementation

    Automatesolution

    Initialanalysis

    MOUand data

    integration

    Periodic

    calibration

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    22/54

    20 Big data in the big apple

    3 Form a project plan A projec plan is pu in place so ha he SD eam and MODA can agree:a) an approach ha works or each par y; b) he da a ha will be used;and c) he imeline or he projec .

    4 Understand data contexto unders and he value o he da a ha is used and collec ed during he

    course o providing a par icular service, MODA analys s need o unders andhow i is crea ed and wha i means in i s original con ex .

    5 Create a Memorandum of Understanding (MOU)Much like wri ing a con rac , MODA es ablishes a ormal writen agreemen wi h he SD ’s organisa ion (e.g. he Fire Depar men ). e MOU de ails

    he purpose o da a sharing and he privacy and da a pro ec ions ha will be applied by MODA and he SD . I also ensures ha here is ransparency

    and commi men abou wha is required rom each side.

    6 Integrate dataMODA se s up he echnical connec ion o ake he SD ’s da a so i can be s ored and analysed in Da aBridge (seepage 16). o combine records wi h o her da ase s, MODA insis s ha all records are geo- agged – in o her words, given a loca ion such as a s ree address, ZIP code or grid re erence.I is his ha allows differen da ase s o be mapped oge her so ha newcorrela ions can be iden ied.

    7 Test hypotheses Working wi h he SD , MODA crea es several hypo heses regarding whichpieces o in orma ion will be use ul in improving he service ou come. Forexample, when inves iga ing wha in orma ion could help predic illegal build-ing conversions, MODA discovered ha he mos likely sources o viola ionsare single amily homes ha are less han he average home value and smaller

    han 3,000 square ee . e homes wi hin ha subse ha have his ories o axdelinquency, mor gage liens and especially a his ory o building viola ions are

    he ones ha are mos likely o con ain illegal conversions. (See ull de ails incase s udy onpage 21.)

    8 Service delivery team reviewOnce MODA has run i s pilo da a model, he SD eam needs o check heanalysis o make sure ha MODA has in erpre ed he in orma ion correc ly.I needed, MODA upda es he model o correc any misunders andings.

    9 Automate the processo deliver sus ainable savings and improvemen s in per ormance, he processes

    designed by MODA mus no depend on human analys s (who ac as singlepoin s o ailure when hey are la e or work, sick or on holiday), bu should be au oma ed and in egra ed in o SD sys ems so ha hey become par o

    he normal workow.

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    23/54

    21The New York method

    10 Implement solutione nal s ep is or MODA o roll ou heir solu ion so ha i becomes

    a permanen x ure o he service.

    11 Delegate responsibility for the data model An eleven h s ep could be added: he model can be passed on o, and managed by, he depar men i sel , as per he example o he FDNY which ook over hedevelopmen o i s own re risk based inspec ion model.

    detailed case study: illegal building conversions in new york

    The following table outlines in detail the 10 step process used by MODA toapply data analytics to improve a service, including the key questions the teamasks at each stage. In the third column, each step is explained using the specicexample of how MODA helped the Department of Buildings (DOB) prioritisethe inspection of illegally converted apartments. (The table is adapted from

    ‘Memorandum on MODA Project Process Flow’ by Mike Flowers.)

    Steps Key questionsasked by MODA

    Case study: NYC illegal conversions

    Step OneSpend time inthe eld withfront-line staff tounderstand howtheir day-to-dayoperations work.

    What is theservice beingprovided?

    The Department of Buildings (DOB) inspects illegal conversioncomplaints to ensure that NYC residents are living in safe conditions.When conditions are not safe, orders (known as ‘violations’) are issued toproperty owners to remedy the apartment. In extreme conditions DOBwill vacate the living space.

    How is theservice allocated,scheduled anddelivered?

    In each of NYC’s ve boroughs, DOB has a Borough Command ofcewith a team of inspectors. When a new illegal conversion complaintcomes in (via 311), it is printed at the relevant Borough Command ofce.Typically, complaints are investigated in the same order they are received.DOB has a goal of inspecting every non-prioritised complaint within40 days.

    What factorsgo into theprioritisation ofdelivery?

    Complaints are inspected in the order they are received. However,priority is given to those that include phrases such as ‘no exit’ and‘exposed boiler’, which suggest higher risk.

    How is the deliv-ery recorded inthe organisation’sIT system(s)?

    DOB tracks the complaint number through the nal disposition in theBuilding Information System (BIS).The Environmental Control Board records and adjudicates DOBviolations.

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    24/54

    22 Big data in the big apple

    Steps Key questionsasked by MODA

    Case study: NYC illegal conversions

    Step Two Identify whatpart(s) of theservice can

    be improvedthrough dataanalysis.Check assump-tions with theteam(s) deliveringthe service.

    What type ofproblem is this?

    The main challenge is identifying which complaints to prioritise given thelimited number of inspectors.

    What data existsaround theoperation?

    The wording and details of the complaint in 311.BIS holds the inspection history of each property.ECB holds the violation history of each property.

    What other datawould be helpful(hypotheses)?

    Potentially:Department of Finance property records;Tax liens and lis pendens;The age of building.

    What is thedesired end goalof the data use?

    A priority ag that can added against the highest risk complaints on thelist printed out each morning at each DOB Borough Command ofce.

    What’s thecommitmentfrom the agencyand MODA?

    That DOB will provide expert guidance on how their service is delivered;review and pilot MODA’s data-driven prioritisation model; and then workwith MODA to automate the process.MODA will test and create a risk lter.

    Step Three Form a projectplan for thedelivery teamand MODAto agree an

    approach thatworks for eachparty.

    What data willbe used, andwhat new data isneeded?

    No new data required.

    What is the

    timeline for theproject?

    Three months: one month for analysis; two months for the pilot;

    one month for automating the IT processes (concurrent with the secondmonth of the pilot).

    What are thecheck pointsduring theproject?

    Two check points during development of the risk lter: one after twoweeks and the second after one month. End of month checks on pilotresults. Weekly checks on IT development once launched.

    Step Four Understand datacontext.To appreciate

    the value of thedata, MODAanalysts have tounderstand howit is created, andwhat it meansin its originalcontext.

    What are thedatasets andwhat are theymeasuring?

    Records of inspections and violations;Records of every visit to a property;Records of when access is granted and the inspection is completed;Records of violation notices, by type, which are found in inspections.

    How is the datagenerated? Whatroad bumpsshould MODAanticipate?

    Data is generated by inspectors or Borough Command staff whomanually enter records.In the case of DOB complaints and inspections, MODA learned that anynew complaint on an existing scheduled inspection is ‘administrativelyclosed’. This was important to understand why some complaints showedfast resolution but no history.

    How is the datainterpreted?

    The results of an inspection are recorded. Often the most seriousviolation is the violation that is written.

    How is the dataset stored?

    Inspections are stored in BIS. Violations are adjudicated throughthe Environmental Control Board (ECB).

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    25/54

    23The New York method

    Steps Key questionsasked by MODA

    Case study: NYC illegal conversions

    Step Five Creating aMemorandum ofUnderstanding

    (MOU).

    What is thepurpose of theproject?

    The purpose of the DOB project is to use DOB inspection and violationdata to perform an analysis of historical outcomes, nd common traits ofillegal apartments that are vacated, and use that data to risk-analyse newcomplaints to prioritise future inspections.

    What data secu-rity guarantees areprovided?

    In the case of the DOB, the information shared is available in the publicrecord, therefore no special care was needed for DOB records. However,while data from DOB BIS was not sensitive, MODA agreed not to use orshare the DOB data in other projects without notication to DOB.

    Step Six Data Integration.MODA sets upthe technologyrequired to takeagency data.Data must bematched togeo-locatedrecords inMODA’s system.

    What sort ofsystem recordsthe data?

    BIS records data in a mainframe system.

    What is the mostappropriatemethod fortransmitting thedata to MODA?

    A paging server sits on top of the BIS mainframe. Every day, the pagingservice automatically extracts data from BIS using A NiemXML transferprotocol. The data is transferred using DEEP to DataShare. DataSharethen pushes the data to MODA using an ETL workow. MODA loadsthe data into DataBridge using Informatica.

    Step Seven Testing hypothe-ses. Working withthe department,MODA choosesseveralhypotheses onwhich variableswill be useful inimproving theservice outcome.

    What variableswill we test?

    Property tax records; lien record; building age; building size; buildingvalue; violation history; neighbourhood conditions.

    What’s the mostappropriateanalyticaltechnique for thisanalysis?

    A decision tree was used to identify the relative value of variablesin predicting an illegal conversion.

    What do thepreliminaryresults show?

    Single family homes that are less than the average NYC home valueand smaller than 3,000 square feet are the most likely sources illegalconversions. The homes within that subset that have histories of taxdelinquency, mortgage liens and especially a history of DOB violationsare the ones that are most likely to now contain illegal conversions.

    How do wecommunicatethese resultsto the delivery

    team?

    A series of slides was used to graphically convey the value of thevariables in predicting the historical outcomes.

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    26/54

    24 Big data in the big apple

    Steps Key questionsasked by MODA

    Case study: NYC illegal conversions

    Step eight Client reviewor pilot. Thedelivery team

    needs to checkthe analysis tomake sure thatthe informationis interpretedcorrectly.MODA updatesits model ifrequired.

    Does any of theanalysis surprisethe deliveryteam? If so, why?

    The delivery team were surprised that the age of the building wasimportant. Initial MODA analysis had attempted to rank buildings’ riskby their age, with older buildings being more dangerous than newer ones.After discussion with DOB, it was apparent that age was binary: buildings

    constructed after the implementation of the 1938 Building Code aresignicantly safer than the buildings constructed prior to that year.

    What agencyprocedures couldaccount for datasurprises?

    The change in the New York City building code accounted for theimportance of pre- and post-1938 safety.

    How can theanalysis be alteredto produce amore accurateresult?

    Rather than apply a scale by age, MODA switched to a binary analysisthat gave more risk weighting to buildings constructed prior to 1938.

    How should theanalysis be testedin the eld (pilot)?

    MODA and DOB agreed on a 60-day eld pilot in Queens. MODAemailed a daily list (in Excel) of complaints prioritised by their dataanalysis. The Borough Command staff manually agged the precedingday’s complaints for inspection.

    How will we knowif the pilot issuccessful?

    After 30 days, and again after 60 days, MODA reviewed the history ofinspections and violations to determine if the pilot was achieving its goalof reducing time-to-vacate for dangerous structures. Reducing the timebetween complaint ling and vacate was important to measure.MODA also checked the number of vacates to make sure they were notsimply going up due to the greater exposure being given to the project

    by the pilot.MODA also observed the number of inspection attempts to make surethat improved results were not simply the outcome of increased effort.

    What systemsare necessaryto support thepilot and pilotmeasurement?

    MODA needed to manually run the script each morning and emailthe results to Queens Borough Command.Borough Command staff needed to manually annotate the prior day’scomplaints based on the MODA list.

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    27/54

    25The New York method

    Steps Key questionsasked by MODA

    Case study: NYC illegal conversions

    Step nine Automating theprocess.The processshould bereviewed at leasttwice a year toensure that thedata model cre-ated by MODAtakes account ofchanges in theeld.

    What systemneeds tobe changedand how?

    The DOB mainframe system could not use the MODA logic. Instead,MODA’s tech team developed a web-based service that caught thecomplaint upstream, between 311 and the delivery of the complaintto DOB.

    The web service would analyse the complaint (and the relevantinformation about the property in DataBridge) to determine whetherthe complaint met MODA’s risk priority. The web service provided apriority ag (high or normal) and forwarded the information to DOB BIS.BIS was updated to include a new eld for the ag.The DOB Borough Command prints lists of complaints with the priorityag included. Stop-gaps were built into the system to ensure that anysignicant downtime in the data model would not delay or disrupt thedelivery of 311 complaints to DOB Borough Commands.

    How often willthe solution be

    reviewed andcalibrated?

    Twice a year. In the case of the DOB lter, a community board inquiryled to an earlier review of the lter, however, detailed review revealed

    no need to change the logic.

    Are we condentthat this is notdisruptive tothe eld?

    DOB conrmed that no change in eld operations was required.

    How do wemaintain thesolution on anongoing basis?

    MODA’s tech team established a notication process with DOB to makesure that the system in place for the MODA lter is updated along withany underlying change to the DOB BIS technology.

    Step ten

    OperationalImplementation.The new auto-mated system islaunched.

    What education

    needs to beprovided to staffin the eld?

    MODA and DOB needed to explain the signicance of the ag on

    the complaint to Borough Commanders and building inspectors.

    How willsuccess bemeasured overtime?

    The MMR was changed to include a ‘time-to-vacate’ measurement toensure that the lter is leading to the desired policy outcome of reducingthe number of days that a dangerous apartment remains at risk.

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    28/54

    26 Big data in the big apple

    New York’slessons forLondon

    ‘Being a da a-driven ci y is really abou more efficien ly and effec ively deliveringhe core services o he ci y: smar er, risk-based resource alloca ion, beter sharing

    o in orma ion agency- o-agency o acili a e smar decision-making, and using heda a in a way ha in egra es in he es ablished day- o-day paterns o ci y agency fon line workers.’ Mike Flowers43

    is chap er de ails en key lessons ha can be derived rom he New Yorkexperience and which should orm he ounding principles o a London Mayor’sOffice o Da a Analy ics.

    1 Strong executive support is essentialModern echnology makes i much easier o access and analyse ci y da a, bui will only make a difference i here is he poli ical will o use i . As a resul ,

    he mos impor an lesson rom New York is ha he success o delivering ada a-driven ci y depends on having he comple e suppor o he mos seniorleadership gures, s ar ing wi h he Mayor. As S ephen Goldsmi h and SusanCraw ord have writen:

    ‘In New York Ci y, Michael Bloomberg ook office as a mayor a er long yearso experience in he use o da a, and he crea ed a me rics-drive mayoral y. Agencies agreed o coopera e o se up his proposed da a analy ics cen er ando her in eragency da a ini ia ives. Ye almos all o hem soon asser ed legal,

    echnical, and opera ional obs acles o ull par icipa ion. Budge exper s alsopushed back, worried abou cos s. Lawyers ci ied vas numbers o rules (mos

    rom he ederal governmen ) ha prohibi ed sharing o da a. Wi hin eachci y agency, i s chie in orma ion officer would explain why only he or shecould manage he complex legacy da abases o ha uni . Despi e his manda e,his commi men o da a and a ra o rs -ra e appoin ees, Bloomberg wouldno have succeeded in making New York Ci y a leader in da a-driven govern-men had he no pushed hard rom he op or change.’44

    As a January 2015 repor by Policy Exchange (‘Small Pieces Loosely Joined’)

    explained, similar cul ural, poli ical and (perceived) legal barriers exis be weendifferen public sec or bodies in he U and would need o be overcome wi hs rong poli ical leadership.45

    04

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    29/54

    27New York’s lessons for London

    2 Data models must be shaped by front-line experience and expertiseDa a is meaningless wi hou con ex . Crea ing he s a is ical models hasuccess ully predic illegal building conversions, re hazards and unlaw ulgrease disposal by res auran s required spending ime wi h ron -line s aff ounders and he ac ors impor an or delivering heir par icular service.

    3 Focus on outcomes that provide a proven return on investment While many repor s on smar ci y ini ia ives celebra e he process, echnologyand echniques o using da a o improve ci ies, MODA is clear ha wha materareou comes. According o i s model, da a ini ia ives do no proceed beyonda rial s age (s ep 8) unless here is clear evidence ha hey deliver angible bene s and improvemen s in service. e New York approach hereby avoidsgambling on a ‘build i and hey will come’ approach, where axpayer moneyruns a real risk o being was ed on unproven echnology-led ini ia ives. Ins ead,MODA’s philosophy is ha each and every ini ia ive mus have a clear re urnon inves men . In Mike Flowers’ words: ‘ ere can be no dead wood.’46

    4 Start small and with measures everyone can supportImplemen ing a MODA model can require a drama ic shi in ways o working

    ha is likely o mee wi h resis ance rom ci y officials, depar men heads,lawyers and ci y leaders (see quo e rom S ephen Goldsmi h and SusanCraw ord in poin 1). axpayers may likewise wonder whe her inves ing ina MODA eam represen s value or money. Mike Flowers was conscious o

    he need o win people over in New York. He here ore s ar ed by using hisda a-driven me hods on issues ha could receive universal poli ical and publicsuppor , such as preven ing res or s opping ra in es a ions. As he pu s i :‘Fires and ra s have no poli ical suppor ers.’47

    5 Do not try to change the work of front-line staffIn almos any organisa ion, he success o implemen ing a new way o workingcri ically depends on hose affec ed by he change embracing he new modelposi ively. One o he reasons he New York MODA eam was so success ulis ha hey collabora ed wi h ron -line s aff o make heir work even moreeffec ive, increasing employees’ job sa is ac ion. As Flowers’ has pu i :

    ‘Immedia ely, we [MODA] discoun any in erven ion ha changes he wayha he ron line works. New raining and processes are non-s ar ers because

    o he immense organiza ional difficul y in effec ively urning batleships andreorien ing hem around new processes. Even new orms are rowned upon,as hey ge in he way, or a leas change he way, he eldwork is done. Ourconcep is simple – a ligh oo prin means ha he solu ion mus be deliveredups ream o he ron line.’48

    6 Using data does not require vast numbers of specialised personnel ornew layers of bureaucracy

    e NYC MODA eam was led by a ormer lawyer who recrui ed a small eamo da a analys s via Craigslis . oday MODA is made up o jus nine people.Be ween 2011–2012 Flowers’ eam proved ha hey could save he ci y money

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    30/54

    28 Big data in the big apple

    and enhance he effec iveness o services be ore MODA became par o heofficial s ruc ure o governmen in 2013 (as described in Chap er 2).

    7 Using data does not require procuring high-end technologyMODA’s approach o echnology is a long way removed rom common no ionso he ‘smar ci y’ ha depend on complex new I sys ems, digi al ne works

    and he proli era ion o urban sensors. New York made use o he ci y’s exis ingda a, da abases and ne works wherever possible. e eam began wi h no hingmore han old spreadshee s and analysis conduc ed in Microso Excel.

    Addi ionally, Da aBridge and DEEP were crea ed o allow agencies okeep using heir own sys ems ra her han having o ins all expensive newI sys ems o comply. e de ailed case s udy in Chap er 3 highligh ed ha when he Depar men o Buildings’ own I sys em proved oo old o handle

    he risk-assessmen logic crea ed by MODA, he eam crea ed a web-basedool ha could work wi h DOB’s exis ing legacy sys em. ere is here ore no

    major echnical change required rom organisa ions wishing o bene rom

    MODA’s echniques. is ‘jus build some hing ha works’ approach reduceshe echnical obs acles ha could preven geting hings done. In cases wheremodes I changes are required by an organisa ion o connec o MODA, a

    ur her principle o he New York model is ha hose changes are paid or roma cen ral budge . is removes he nancial barriers o da a sharing.

    8 Any organisation that wants to access MODA’s data must rst sharetheir own

    o crea e he righ incen ive or organisa ions o open up heir da a (andcomba he cul ural resis ance o doing so), access o MODA’s exper ise and he

    da a i holds mus be condi ional on organisa ions rs sharing heir own da a.MODA also insis s ha while he da a provided by an organisa ion does nohave o be per ec , i mus be he en ire se .49

    9 All data must be geo-tagged (geo-coded)Some o he mos power ul da a analy ics processes depend on ploting da a

    rom differen sources on one map. ere is here ore a minimum requiremenha he records rom each organisa ion are geo- agged. While adop ing open

    s andards (i.e. common ways o recording in orma ion) can make his processmore s raigh orward, i is no a necessary condi ion or he MODA model

    o work. Organisa ions in New York Ci y use several differen me hods orgeo-coding and MODA does he hard work o ma ching hem.*

    10 No part of the data extraction or analytics process should requirehuman actionFor da a-driven analy ics o reliably improve a service, he processes crea ed by he MODA eam mus even ually be au oma ed. Requiring a person ocomple e a s ep in he chain be ore i reaches ron -line s aff crea es singlepoin s o ailure. As well as making simple errors, employees some imes needsick days, holidays or simply need o priori ise o her work. By au oma ing i s

    models, MODA is able o make da a analy ics a reliable and in egral – ra herhan a peripheral – par o delivering services.

    * For discussion onhe role o open s andardsor da a in he con ex

    o U local governmen ,see: Policy Exchange,‘Small Pieces Loosely Joined: How smar eruse o echnology andda a can deliver real re ormo local governmen ’, January 2015, p.29–30

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    31/54

    29Why London needs a Mayor’s Ofce of Data Analytics

    Why Londonneeds a Mayor’sOfce of DataAnalytics

    ‘Analy ics is no magic, and i ’s no necessarily complica ed. Analy ics really meansin elligence, and in elligence is beter in orma ion ha helps us make beter decisions.’ Mike Flowers50

    A some poin during he rs hree mon hs o 2015, London achieved a signicanmiles one: he popula ion nally caugh up o and surpassed i s 1939 peak o 8.6million people.51 is achievemen is indica ive o London’s success ul revival. When once here were ears ha he capi al was se or a long- erm decline

    ollowing an exodus o residen s in he 1970s and 80s, he dominan narra ive isnow rmly one o London as a magne or people, inves men , jobs and ideas.52

    Figure 5A: Total population of Greater London – GLA 2012round projections

    2001 2006 2011 2016 2021 2026 2031 2036 2041

    11.0

    10.5

    10.0

    9.5

    9.0

    8.5

    8.0

    7.5

    7.0

    6.5

    6.0

    GLA 2012 round SHLAA

    T o t a

    l p o p u

    l a t i o n

    ( m i l l i o n s )

    Year

    GLA 2012 round Trend

    Source: GLA Intelligence, ‘GLA 2012 Round Population Projections, Intelligence Update

    05–2013’, February 2013, p.3

    05

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    32/54

    30 Big data in the big apple

    Ye while London can airly be described as Bri ain’s economic engine (i wasresponsible or 22.8% o GVA in 2012), he inux o new residen s has no beenma ched wi h a commensura e rise in money available o ci y coffers.53 e periodo aus eri y ha has ollowed he nancial crash o 2008 has inevi ably placedconsiderable pressure on local au hori y budge s. is has coincided wi h he grow-ing popula ion placing unpreceden ed demand on London’s ageing in ras ruc ure

    and public services.54

    e experience rom New York Ci y sugges s ha es ablishing a Mayor’sOffice o Da a Analy ics in Ci y Hall could help London respond o hese nancialpressures and deliver services ha are no jus more efficien , bu undamen ally beter. rough he examples given in his repor , i has been shown ha a MODA

    eam can help arge a ci y’s scarce resources more efficien ly, increase he efficiencyo communica ions be ween differen public sec or organisa ions, pre-emp andaddress problems be ore hey become serious and expensive o resolve, and booslocal economic grow h hrough providing beter in orma ion o ci izens.

    Wha kinds o issues migh a London MODA eam be able o address?

    Potential applications of a London MODA teame very na ure o he MODA model means ha i is impossible o speci y romheory alone which ini ia ives would deni ely work or London. As Chap er 3

    made clear, he rs s ep or a MODA eam is o spend ime wi h ron -line s aff oobserve rs -hand he challenges hey ace so ha da a models can be buil rom

    he ground up. However, o give an indica ion o wha migh be possible or heBri ish capi al, below six hypo he ical scenarios are ou lined.*

    Scenario 1: Intelligently designing shared servicesLondon local au hori ies increasingly use heir own da a o crea e digi al mapsshowing he loca ion o hings like parks, buildings and parking spaces. ey canmap he addresses o individuals or amilies wi h par icular needs, rom educa ion

    o wel are. Bu – as was he case or many New York Ci y agencies be ore hecrea ion o MODA – hey o en have litle or no da a on hose same hings beyond

    heir boundaries.** e resul an endency can be or London boroughs o acas islands, a poin no ed by e Economis , which observed he densi y o new building developmen s and he provision o services in he cen re o boroughscompared wi h heir ringes.55 is makes litle sense given ha communi ies, areaso depriva ion, crime, litering and school ca chmen areas can (and requen ly do)cu across local au hori y borders.

    For London boroughs ha wish o achieve cos savings hrough sharing moreron -line services, his is a serious problem.*** Wi hou shared da a, i is ex remely

    hard or a council o know i a par icular problem hey are ackling, or service needhey are mee ing, represen s he ip o he iceberg or he mass below sea level.

    How ar does he area o urban depriva ion on he eas ern boundary con inuein o he neighbouring borough? Wha is he demand or library services in hecommuni y ha alls a he in ersec ion o hree councils’ areas? Lacking da a romo her boroughs also limi s councils’ abili y o learn bes prac ice rom, and work wi h, ‘s a is ical neighbours’ – non-neighbouring local au hori ies ha have similar

    ypes o area or challenges.†

    I each London borough shared i s own da a wi h a MODA eam based in Ci yHall, heir numerous differen da ase s could be combined and ploted on maps

    * e six examplesou lined are purelyillus ra ive. ey are noin ended o imply ha

    here is any deciencyin he curren servicesdirec ed a each area.

    ** A no able excep ion would be he ri-Borougho Wes mins er Ci yCouncil, Hammersmi hand Fulham BoroughCouncil, and ensing onand Chelsea BoroughCouncil

    *** Shared serviceshave a s rong rackrecord o saving money. According o he LGA,a leas 337 councils are

    already engaged in sharedservice arrangemen s,leading o savings o £165million in 2012, £278million in 2013 and £357million in 2014. Source:LGA, ‘Shared Services:cos s spared?’, Oc ober2014, p.5

    † An example o wherehis approach is already

    being used by he policecan be ound a : www.london.gov.uk/ webmaps/neighbour-hoodcondence ool/

    https://www.london.gov.uk/webmaps/neighbourhoodconfidencetoolhttps://www.london.gov.uk/webmaps/neighbourhoodconfidencetoolhttps://www.london.gov.uk/webmaps/neighbourhoodconfidencetoolhttps://www.london.gov.uk/webmaps/neighbourhoodconfidencetoolhttps://www.london.gov.uk/webmaps/neighbourhoodconfidencetoolhttps://www.london.gov.uk/webmaps/neighbourhoodconfidencetool

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    33/54

    31Why London needs a Mayor’s Ofce of Data Analytics

    so ha London boroughs could see he real size and shape o problems beyondheir jurisdic ions. Maps could be crea ed ha spanned he whole capi al, crea ing

    a comple e pic ure or any given issue, ra her han he individual jigsaw piecesLondon boroughs curren ly hold. Wi h ha kind o real- ime in orma ion, i would be easier or councils o make in ormed decisions abou whe her hey should join wi h a neighbouring borough o run or join ly und a service.

    I could be argued ha London boroughs could organise his kind o da asharing be ween hemselves (as some, such as he ri-borough o Wes mins erCi y Council, Hammersmi h and Fulham Borough Council, and ensing on andChelsea Borough Council already do). However, a London Mayor’s Office o Da a Analy ics would have a leas hree dis inc advan ages:

    • Translating between different data standardsFirs , MODA would handle he signican difficul ies around combining da a-se s ha are recorded in differen s yles (i.e. using differen le orma s, s andardsand conven ions). As in New York Ci y, London boroughs use a varie y o

    differen me hods or recording loca ion (pos code, s ree address, grid re erence,and so on) and he many o her ypes o in orma ion hey manage. Merginghose records o build a comple e map can here ore be he digi al equivalen

    o ma ching apples and oranges. A London MODA eam would have he ime,exper ise and echnical resources o ransla e each local au hori y’s records so

    hey could be joined oge her.* Wi hou ha capabili y, councils would s ruggleo make sense o heir dispara e da ase s un il hey agreed upon and implemen ed

    common da a s andards – a process ha could ake many years.**

    Figure 5B: Signicance of barriers to increasing collaboration betweenorganisations – BT survey of Local Authorities

    Incompatibility of IT systems

    Lack of time/resources

    Incompatibility of processes

    Data sharing barriers

    Leadership barriers

    Political barriers

    Cultural barriers

    Lack of understandingof other organisations

    Extreme barrier

    0% 20% 40% 60% 80% 100%

    Moderate barrier

    Somewhat of a barrier

    I don’t know

    Not a barrier

    Source: BT, ‘Public services: delivering the next generation of change’, April 2013, p.7,available at:http://connect.bt-comms.com/Dods-whitepaper.html

    * A London MODA would be able o buildon work already carriedou by organisa ionsincluding he CabineOffice, he Release oDa a Fund and he LocalGovernmen Associa ionon da a s andards

    ** Adop ing opens andards would ake imedue o he need o phaseou old, non-compliansys ems. e EuropeanCommission haspublished advice s a ing

    ha ‘ he change o as andards-based sys em…should… be carried ouon a long- erm basis(5 o 10 years), replacing

    hose sys ems harequire new procuremen wi h al erna ives haare s andards-based.’Source:htp://ec.europa.eu/digi al-agenda/en/open-s andards

    http://connect.bt-comms.com/Dods-whitepaper.htmlhttp://ec.europa.eu/digital-agenda/en/open-standardshttp://ec.europa.eu/digital-agenda/en/open-standardshttp://ec.europa.eu/digital-agenda/en/open-standardshttp://ec.europa.eu/digital-agenda/en/open-standardshttp://ec.europa.eu/digital-agenda/en/open-standardshttp://ec.europa.eu/digital-agenda/en/open-standardshttp://connect.bt-comms.com/Dods-whitepaper.html

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    34/54

    32 Big data in the big apple

    e difficul y o joining up records held in differen da abases washighligh ed in a comprehensive survey o local au hori ies conduc ed by B in2014. According o he survey, he single grea es barrier o increasing collabo-ra ion be ween differen organisa ions was ci ed as being ‘Incompa ibili y o Isys ems’, wi h 84% agreeing. In he same survey, 80% o responden s el ha‘Da a sharing barriers crea ed obs acles o collabora ion’.56

    • Speeding up implementation of data sharing Second, i each London borough ried o nego ia e individually wi h he32 o her councils o share heir da a, i would require seting up 528 one- o-oneconnec ions. By con ras , MODA could se up a single da a exchange wi h eachcouncil (33 in o al) o bring heir da a oge her in one secure loca ion ( heequivalen o New York’s Da aBridge). is would save a huge amoun o ime,money and effor . Added o his, MODA could combine local au hori y da a wi hda ase s rom sources such as he Me ropoli an Police, he London Fire Brigadeand o her London public sec or organisa ions o provide addi ional insigh s in

    a way ha would no be possible o achieve on a borough-by-borough basis.

    • Providing new insights for City Hallird, bringing London’s da a oge her in one place would be o huge beneo he Grea er London Au hori y and he Mayor’s Office. Remarkably, Ci y

    Hall curren ly does no sys ema ically collec any da a rom London boroughs,o her han ha required or s a u ory purposes, such as popula ion and schoolplace s a is ics.57 e in orma ion used o shape decisions affec ing he capi alis here ore largely based on da a collec ed by cen ral governmen depar men ssuch as he Depar men or Work and Pensions (DWP). As London seeks o

    address i s mos serious challenges a a London scale, i will need da a insigh sha cover he whole capi al.

    Scenario 2: Combating illegal conversionsNew York is no he only ci y o suffer rom dangerous buildings. Par s o Londonare bligh ed by ‘beds in sheds’ (a common erm or ren al accommoda ionprovided in illegal ou houses, normally buil wi hou planning permission behindconven ional houses), cos ing axpayers millions and making li e a misery oraffec ed communi ies.58 London is dispropor iona ely affec ed by he problem:eigh o he ci y’s boroughs are in he op nine wors affec ed local au hori ies in

    he U .59

    is migh be atribu ed o he capi al’s high cos o housing, he shor ageo supply, he heavy reliance on he priva e ren ed sec or among low-incomehouseholds, and he ac ha London is o en he s ar ing des ina ion or newmigran s. I is ypically new illegal migran s ha are housed by rogue landlordsin sheds and garages in condi ions ha are unsa e, unsani ary, and in which peopleare requen ly exploi ed.

    Addressing he problem is expensive. In 2012 he governmen awarded anex ra £2.3 million o he wors affec ed London boroughs o help hem comba

    he issue (see able below).60 ere are also addi ional indirec cos s. Illegaldwellings have been connec ed wi h increases in local crime and an i-social behaviour. ey pu addi ional pressure on u ili ies, such as was e and sewagedisposal. ey also increase illegal working prac ices and bene raud, wheremigran workers are par icularly arge ed.

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    35/54

    33Why London needs a Mayor’s Ofce of Data Analytics

    Table 5A: Funding granted by government to seven London boroughsworst affected by illegal conversions.

    Borough Tranche 1 grant Tranche 2 grant Total

    Ealing £280,000 £270,919 £550,919

    Hounslow £280,000 £0 £280,000

    Newham £280,000 £227,572 £507,575

    Brent £163,745 £0 £163,745

    Redbridge £163,000 £108,368 £271,368

    Southwark £163,000 £0 £163,000

    Hillingdon £150,000 £183,141 £333,141

    Total £1,479,745 £790,000 £2,269,745

    Compounding he problem is ha – according o he GLA – local au hori iesnd i ar rom s raigh orward o iden i y which proper ies con ain illegalconversions. Conduc ing s ree surveys and using hermal imaging ( o de ec

    he presence o people in ou buildings) have no been as effec ive as originallyhoped. In addi ion, some boroughs have ound ha while complain s received

    rom members o he public are help ul, some communi ies are re icen aboucoming orward. As Chap er 3 ou lined, hese same challenges were experiencedand addressed in New York, sugges ing ha a London MODA eam could helpiden i y where council inspec ors should arge heir resources. Relevan da ase smigh include:

    • Sewage owOne London borough knew ha official gures underrepresen ed i spopula ion, due o he high number o migran s being illegally housed in

    beds in sheds. ey jus did no know by how much. ey came up wi h anovel me hod or calcula ing he rue number o people: measuring he ou owrom he sewerage pipes ha serve he area. By calcula ing he ypical was e

    ‘ou pu ’ per person, hey es ima ed ha heir popula ion was up o 10% higherhan official gures sugges ed. While clearly a crude measure or loca ing he

    problem o illegal conversions, his da a could help iden i y he scale o heundocumen ed popula ion in cer ain areas.

    • Pest control and y-tippingDa a rom pes con rol ac ivi ies and inciden s o y- ipping could be indica ive

    o areas wi h insani ary condi ions caused by illegal housing. e experienceo New York Ci y was ha many complain s rom he public abou suspec ed

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    36/54

    34 Big data in the big apple

    illegally conver ed apar men s are riggered by neighbours no icing grea eramoun s o rubbish le ou side buildings.

    • Style of property and garden sizeBuildings rom cer ain decades or o cer ain s yles may be more liable orconversion han o hers. In par icular, garden sizes vary considerably hrough-

    ou he capi al. Conversions o ou buildings may be more likely in suburban boroughs where gardens end o be larger (see Figure 5C). A MODA eamcould use his same principle o analyse da a more closely on a ward-by-ward,or s ree -by-s ree basis using Ordnance Survey or Google maps.

    Figure 5C: The average size of a garden plot (m 2) in each of GreaterLondon’s boroughs

    C i t y o f L o n o d n

    C i t y o f W e s t m i n s t e r

    N e w h a m

    H a m m e r s m i t h & F u l h a m

    K i n g s t o n & C h e l s e a

    T o w e r H a m l e t s

    W a n d s w o r t h

    C a m d e n

    H a c k n e y

    W l t h a m F o r e s t

    S o u t h w a r k

    I s l i n g t o n

    M e r t o n

    L a m b e t h

    L e w i s h a m

    H a r i n g e y

    R i c h m o n u p o n T h a m e s

    B a r k i n g & D a g e n h a m

    S u t t o n

    G r e e n w i c h

    H o u n s l o w

    E a l i n g

    H a v e r i n g

    R e d b r i d g e

    B r e n t

    B e x l e y

    C r o y d o n

    E n f i e l d

    K i n g s t o n u p o n T h a m e s

    H i l l i n g d o n

    H a r r o w

    B a r n e t

    B r o m l e y

    200

    180

    160

    140

    120

    100

    80

    60

    40

    20

    0

    A v e r a g e s i z e o f a g a r d e n i n b o r o u g h ( m 2 )

    Source: Royal Borough of Kensington and Chelsea, ‘London: garden city? From green togrey; observed changes in garden vegetation structure in London, 1998–2008’

    Scenario 3: Identifying empty homes A similar approach migh be used o address he reverse problem: emp y homes.Since April 2013, councils have had he power o charge proper y owners an‘Emp y Homes Premium’ o 50% ex ra council ax i hey leave proper ies unoccu-pied or wo or more years.61 A Freedom o In orma ion reques conduc ed in July2014 by he BBC discovered ha here were more han 80,000 emp y homes in

    he capi al, bu ha seven councils did no apply he charge o a single proper y.*e BBC’s research ound ha o 80,489 emp y proper ies, only 4,399 had been

    subjec ed o he Emp y Homes Premium (albei no all o he proper ies had beenemp y or he wo years required o make a charge).

    Given ha he average weigh ed London council ax in 2014/15 is £1,296.44,62 ensuring ha he housands o o her quali ying emp y proper ies received he 50%

    * e councils were:ensing on and Chelsea,

    Wes mins er, Bromley,Havering, Hillingdon,

    ings on-upon- amesand Mer on. Some o

    hese councils, such as Wes mins er, have an explicipolicy no o make he charge

  • 8/15/2019 Big-Data-in-the-Big-Apple.pdf

    37/54

    35Why London needs a Mayor’s Ofce of Data Analytics

    surcharge could resul in subs an ial addi ional revenue or London boroughs.(I all remaining 76,090 emp y proper ies were le or wo years and charged hepremium, i would provide £49 million). ough i is unrealis ic o hink ha 100%o emp y proper ies could be iden ied, wi h he righ da a and he exper ise oa MODA eam, i may be possible o arge more han is curren ly he case.

    Scenario 4: Fighting tax and benets fraudEach year, housing enancy, bene and Council ax raud (such as alse en i le-men , illegal sub-leting, lease sell-on and unau horised succession) cos he U ’slocal au hori ies in excess o £1.3 billion. By bringing oge her and analysing da a

    rom pas cases and combining wi h hird-par y da a sources (e.g. he Elec oralRoll, he Pos Office and credi scoring agencies), i is possible o predic where

    u ure viola ions are mos likely o occur and direc inves iga ive eams o respondo hem rs .

    A rial using exac ly hese me hods conduc ed by Gravesham Borough Council, working in conjunc ion wi h Fuji su, iden ied 75 proper ies where he council madea range o in erven ions, including: eigh cases in which council proper y needed

    o be repossessed; our proper ies ha were under-occupied; and welve wherehere were illegal enancy successions. £108,000 o enancy raud was discovered.63

    Building on he work s ar ed in Gravesham, a London MODA eam could helpex end his da a-driven process o ackle raud across he res o he capi al.

    Scenario 5: Targeting food safety inspectionse average London borough has i s own eam o around 6 ull- ime-equivalenood sa e y inspec ors o conduc ood sa e y inspec ions and provide hygiene

    ra ings.64 e ypical borough has 1,838 premises o inspec (including res auran s,pubs, ca és, akeaways, ho els, supermarke s and o her ood shops).65 e

    requency wi h which businesses are inspec ed is based on heir risk ca egory. ehighes risk es ablishmen s (Ca egory A) will be inspec ed every six mon hs. O heres ablishmen s will be ar lower risk (down o Ca egory E) and only inspec edevery hree years.66 New businesses are ypically inspec ed wi hin heir rs 28 days. Addi ional inspec ions and in erven ions can be arranged based on complain sreceived by members o he public sen direc ly o heir local au hori y’s oods andards eams. MODA could po en ially use da a o help priori ise inspec ions wi h even grea er accuracy han is curren ly possible so ha eams could address

    he wors cases rs . e ollowing da ase s migh apply:

    • Data on past casesIn he same way ha MODA designed a Risk Based Inspec ion Sys em or

    he New York Fire Depar men (see Chap er 1), a London eam could analyseda a rom he 60,000+ inspec ions ha ake place across London each yearand see how accura ely he Food S andards Agency risk ca egories predic

    he wors cases.67 Ra her han use broad ca egories (A-E) ha apply ohundreds o businesses, MODA could use s a is ical analysis o weigh differenrisk cri eria wi h grea er granulari y.* is in orma ion could hen be used ore-priori ise scheduled inspec ions, as well as de ermining which complain sreceived rom he public are likely o represen he mos serious and urgen

    hrea s o public heal h.

    * For a ull lis o cri eriaha go in o he curren

    risk ca egory model, see:Food S andards Agency,‘Food Law Code o Prac ice: A5.6 Food s andardsscoring sys em’, availablea : www. ood.gov.uk/en orcemen /en orcework/

    ood-law/annex5- ood-es-ablish-in erven ion/

    a56- ood-s andards-scoring

    http://www.food.gov.uk/enforcement/enforcework/food-law/annex5-food-establish-intervention/a56-food-standards-scoringhttp://www.food.gov.uk/enforcem