w �Ge#ng,CleaningandVisualizingData
DongLei|SchoolofArch2017.06.12|Weibo:LEI_DONG
[email protected]:donglei.org
�
|Outline
0- w
1-
2- w
3- x
w |Preface
0- R cidlp
x
| x |
x
1- Op nodji
x
db Op nodjix
2- Qtno h
w
x
x
yy
|Chapter1
0-0 'Uct(
0-1 'Ucj(
0-2 'Fjr(
0-3 'Fjr*(
0-4 Rdkn
0-5
|Target
Qo k0 ,= Qo k1
,= Qo k2 ,= Qo k3 ,= Qo k4
|Target
Qo k0 ,= Qo k1
,= Qo k2 ,= Qo k3 ,= Qo k4
w w
| | x
7 441 -0. 4 / / 4.
|Target
Qo k0 ,= Qo k1
,= Qo k2 ,= Qo k3 ,= Qo k4
)) ))
Qo k2 Qo k3x
5,01 4//,0///
2 0// 4,0/
x
|Target
|SoOware
Qjaor m 9 Ntocji ,= A ijkt+ P ,= PQop_dj
w P w
w Ntocji-
N f b n9 Ntocji ' podapg Qjpk+ nq+ pmggd]1+ enji(:
P 'bbkgjo1+ bbh k+ h kn+ h k_ o (
| x
|BooksJ mi Ntocji oc F m_ U t 'cook9..g miktocjioc c m_r t-jmb.(
Ntocji FWPW Qop_dj
P di ? odji 'cook9..]jjf-_jp] i- jh.np]e o.5015220.(
Tdnp gdu Rcdn 'cook9..]jjf-_jp] i- jh.np]e o.4858344.(
|Bookscook9..]jjf-_jp] i- jh.np]e o.0/64/044.
|MOOC
Ajpmn m
B o Q d i
Y P Z+
Kj_ g Rcdifdib
Y Z+
K kn i_ E jnk od g P qjgpodji
Y CQPG
N iiQo o Z(
|Process v Uc m &n oc _ o >
0- w w w
9cook9.._ o -no on-bjq- i.w dedib Adot J ]
1- w w
2-
|Process
w w
x
0- r ] m rgdib
1- _ o g i 'nomp opm _ _ o qn- pinomp opm _ _ o (
2- b j j_dib
3- i gtndn
4- qdnp gdu
|Processb j j_dib, kd
E j j_dib ?NG x
cook9.._ q gjk m-] d_p- jh.h k.b j j_dib, kd-coh
x | 16
|gib9005-20874+g o928-848725 xenji
cook9.. kd-h k-] d_p- jh.b j j_ m> __m nn< %jpokpo<
%f t< % dot<
x |g o920-214041+gib9
01/-447846 | 207 x
cook9.. kd-h k-] d_p- jh.b j j_ m>gj odji< + %jpokpo<
%f t<
|Processb j j_dib, kd
0- dot x
1- w x
2- no opn MI
dot
x
3- gj odji ' g o gib(x
4- E j j_dib E j j_dib
x
5-
x' (
|Process U ]K m ojm1JjiJ o
Ejjbg K k+ Mk iQom oK k U ] K m ojm
x
" ,), j_dib9 poa,7 ,),
dhkjmo h oc
"
_ a K m ojm1JjiJ o's+ t(9
s < s.1//264/7-23)07/
t < t.1//264/7-23)07/
Jji < s
J o < 07/-/.h oc-kd)'1-/)h oc- o i'h oc- sk't)h oc-kd.07/-/((,h oc-kd.1-/(
m opmi Jji+ J o�
|Process
dhkjmo h oc
C moc[P _dpn < 5267 ) 0///
_ a m _'_(9
m opmi _ ) h oc-kd . 07/-/ " -/
_ a E oBdno i 'g o0+ gib0+ g o1+ gib1(9
< m _'g o0( , m _'g o1(
] < m _'gib0( , m _'gib1(
_dn < 1 ) C moc[P _dpn ) h oc- ndi'h oc-nlmo'h oc-kjr'h oc-ndi' .1(+1( *
h oc- jn'm _'g o0(() h oc- jn'm _'g o1(( ) h oc-kjr 'h oc-ndi'].1(+1(((
m opmi _dn
|Examples0-
|Examples0-
/( 0 1
Ajgjm m r m 2 ?G dggpnom ojm
x cook9..rrr- jijhdno- jh. jio io. cdi n [ lpdq g ion
cook9..tdcpd-i h . i.
1//6./8. cdi ,h k, o,kmjqdi ,g q g. P cook9..
jn-i h .1//8./6._m rdib, cdi ,h k,pndib,m.
|Examples1- bbh k 'P (
b j j_
b j j_ ' Acdi (
gji g o
0 0/3-0843 24-75055
x
h k_dno
h k_dno'&Rndibcp Sidq mndot&+ & dedib Sidq mndot&+ &r gfdib&(
amjh oj h fh hdg n n ji_n
0 Rndibcp Sidq mndot dedib Sidq mndot 2218 2-218 1-/57530 1347
hdipo n cjpmn
0 3/-85556 /-5716667
2-2 30 x-
|Examples1- bbh k 'P (
b o[h k
gd]m mt'bbh k(
gd]m mt'h kkmje(
h k ;, b o[h k'gj odji < & dedib&+ ujjh < 7(
" ujjh -
xEMMEJC EGQ
h kotk < zmj _h k
x
h k ;, b o[h k'gj odji < & dedib&+ ujjh < 0/+
h kotk < &mj _h k&( "
cook9.._jibg d-i h .1/02.0/.h k,_ o ,rdoc,m.
|Examples2- h kn 'P (
1//7 0/// 04,08 x
|Examples2- h kn 'P (
1//7 0/// 04,08 x
Ljo 9 AQT x
gd]m mt'h kn( " h kn
a modgdot ;, m _- nq' cook9..]jjf-agjrdib_ o - jh. c/7.kjdion. _jg,a modgdot- nq (
h k'&rjmg_&+ adgg < D?JQC+ jg < " ( "
nth]jgn'a modgdot#gjibdop_ + a modgdot#g odop_ +
dm g n < nlmo'a modgdot# _[a mo[m o (+ __ < RPSC+
di c n < /-0+ ]b < "82 a + ab < "aaaaaa ( "
|Examples3- pmggd]19 'Ntocji ( bbkgjo1+ bbh k 'P (
39.6
39.8
40.0
40.2
116.0 116.2 116.4 116.6 116.8lon
lat
House Price(RMB)0−20,00020,000−40,00040,000−60,00060,000−80,00080,000+
|Examples3- pmggd]19 'Ntocji ( bbkgjo1+ bbh k 'P (
|Examples3- pmggd]19 'Ntocji ( bbkgjo1+ bbh k 'P (
Qo k 0- E o B o
dhkjmo pmggd]1
dhkjmo odh
gdif[no mo < &cook9..] dedib- iepf - jh.i rh k.n m c1>hj_ g<1%jm_ m<ipgg%k<0%]jpi_n<&
gdif[ i_ < &%ujjh<03%] c<IU/[BU0[QV/[RB/[QD0[ON/[NV/[HF/[DW0[BODW[BR/%o no<z
m r_ o < &&
g o < 28-44
ajm d di m ib '2/(9
g o *< /-/14 g o < g o * /-/14
gib < 005-/4
ajm e di m ib '24(9
gib *< /-/14
|Examples3- pmggd]19 'Ntocji ( bbkgjo1+ bbh k 'P (
Qo k 0- E o B o
gdif < gdif[no mo * nom'g o( * &+& * nom'g o*/-0( * &+& * nom'gib( * &+& * nom'gib*/-0( * gdif[ i_
odh -ng k'2(
m nkjin < pmggd]1-pmgjk i'gdif(
cohg < m nkjin -m _'(
di_ s < cohg-adi_'&Z+ kmjkn&(
da di_ s = 0/9
htGo h < cohgY0/9di_ sZ * &+
m r_ o *< htGo h
o mb o < jk i'& io m-oso&+ &r&(
o mb o-ompi o '(
o mb o-rmdo 'm r_ o (
o mb o- gjn '(
|Examples3- pmggd]19 'Ntocji ( bbkgjo1+ bbh k 'P (
Qo k 0- E o B o
gd]m mt'bbkgjo1(
gd]m mt'bbh k(
kmd ;, m _- nq' Fjpn Nmd 1/02- nq (
kmd #GB ;, 'kmd #Nmd $.$ 1//// * 0(
kmd #GBYkmd #GB = 4Z ;, 4
kmd #GB ;, n-a ojm'kmd #GB(
kmd 0 ;, kmd Ykmd #Jib ; 005-7 % kmd #Jib = 005-/+Z
kmd 1 ;, kmd 0Ykmd 0#J o ; 3/-1 % kmd 0#J o = 28-5+Z
h k ;, b o[h k'gj odji < ] dedib + ujjh < 0/+ h nn bdib < D?JQC+ jgjm < ]r +
h kotk < &oji m&+ njpm < no h i (
|Examples3- pmggd]19 'Ntocji ( bbkgjo1+ bbh k 'P (
Qo k 0- E o B o
b ;, bbh k'h k(
k ;,b * b jh[kjdio'_ o <kmd 1+ n's<kmd 1#Jib+ t<kmd 1#J o+
jgjpm<kmd 1#GB(+ ndu <2+ gkc </-4(
k ;, k * n g [ jgjpm[h ip g'i h < Fjpn Nmd 'PK ( +
q gp n< ' "a0 a5 + "_6]4_7 + "_a54]/ + "__0 66 + "87//32 (+
g ] gn< ' /,1/+/// + 1/+///,3/+/// + 3/+///,5/+/// + 5/+///,7/+/// + 7/+///* ((
kmdio'k(
|Tips
• Q g
• K kkdib Rjo gn qn- K kkdib P o n
• Ajmm g odji qn- A pn odji
• Njkpg odji B k i_ i 9 pn h kn ja m r q gp n oj
gr tn cdbcgdbco c qdgt kjkpg o _ kg n
• nq ijo sgn+
• ?]jpo Ajgjm 'Ajgjm m r m(
• ?]jpo R so 'cook9..rrr-otk ]m r m-jmb.(
|TipsRjo gn qn- P o n
PhotoCredit:AnthonyC.Robinson�
|Tipspaa m Xji n+
PhotoCredit:AnthonyC.Robinson�
|TipsF o K kn
PhotoCredit:AnthonyC.Robinson�
|Reference
B o
Mk iQom oK k9 cook9..rrr-jk inom oh k-jmb.
Qj dj jijhd _ o i_ kkgd odjin io m 'QCB?A(9 cook9..
n _ - d ndi- jgph]d - _p._ o .n on.]mjrn
cook9..rrr-bjq-cf. i.oc h .knd._ o n on.
Sido _ L odjin cook9.._ o -pi-jmb.
cook9..rrr-_ o -bjq. cook9..rrr- inpn-bjq.
cook9..oc c ib gjb- jh.oc , dot,ja, cd bj,dn,ji,bdocp].
cook9.._ o -bjq-pf. ' cook9.._ o -gji_ji-bjq-pf.(
cook9..rrr-bjq_ o -_ .
cook9..rrr-_ o -bj-ek.
9 cook9..rrr-_ o -bjq.jk i_ o ndo n
' (
|Reference
Tdnp gdu odji
cook9..qdnp gdudib-jmb.
cook9..agjrdib_ o - jh
cook9..nk od g-gt.m.
cook9.. sdnh kn- jh.kjmoajgdj-kck
cook9..rrr-lbdn-jmb. i.ndo .
cook9.._ q gjk mn- moj_]- jh.opojmd gn-cohg
Aj_
cook9..rrr-ktocji c gg ib - jh.
cook9..rrr- j_ _ ht- jh.om fn.ktocji