Where's my monitoring at?
André Klapper <[email protected]>GUADEC 2007, Birmingham
Some random figures about a software project called GNOME(and a pretty untechy talk, compared to the rest here)
Monitoring
● Measurement & numbers for decision making support & a more optimal use of resources
● continuous process on various levels, can end up in defining and better documenting of processes and assigning of responsibilities
What is this about, dude?
● GNOME = not a company, but nevertheless professional?
● identifying problems with the help of some automatization – some examples what we have, and what we don't have
● end up in better overall quality and happier end customers by clearer internal processes?
„Where are we, where do we wanna be?“
GNOME: Structure
Here:● Release Team● Docs● GTP/I18N● Bugsquad● („Hacking“ only
w.r.t maintainers)
Source: https://twiki.softwarelivre.org/bin/viewfile/Main/ VicenteAguiar?rev=1;filename=OrganogramaGNOME2.4_.png
by Vicente Aguiar
Release Team (RT)
● Aim: Test & publish perfect(TM) releases [partially helped by GARNOME and the BuildBrigade (-> Thursday 19th, 16:00)]
● Tasks: Release Engineering, Schedule Planning, Nagging, module inclusion final decisions, assigning Release Notes
● Resources: 8 persons, see http://live.gnome.org/ReleasePlanning/Membership
● Stats: no, what for? :-)● Problems: response times, manpower/slacking.
RT: Break request amounts and response times
● Rules: http://live.gnome.org/ReleasePlanning/ RequestingFreezeBreaks
● ABI/API UI String Hardcode2.13 Amount 2 2 24 21
>1 >1, >3
2.17 Amount 1 1 5 10>1 >1 >1
ø Resp.Time (in days)
18× >1, 3× >2
ø Resp.Time (in days)
2× >1, 2× >2, 1× >4, 1× >9
RT: Wrong/missing announcements of maintainers
● miss of informing any of release-team, desktop-devel-list, gnome-i18n and gnome-doc-list (http://live.gnome.org/MaintainersCorner)
●
● Some modules do not have strings, therefore no announcements were sent to gnome-i18n@. Table results by searching for „branch“ in subject of the four mailing lists)
total correct missed...r-t i18n d-d-ldoc all
2005/09/05 – 2006/03/12 (2.13) 41 23 12 1 3 6 02006/03/13 – 2006/09/03 (2.15) 46 20 19 2 12 11 02006/09/04 – 2007/03/11 (2.17) 54 26 19 2 16 8 1
Maintainers' quick guide
ActionBranching % always inform inform inform informString Change trunk Announcement Period inform inform
trunk Freeze period request
stable inform informstable ...else: inform request
UI Change trunk Announcement Period informtrunk Freeze period request requeststable always request request
Code Change trunk Hardcode Freeze request
Code- base
Time http://live.gnome.org/Schedule
(accidentially forgotten to mark)
release-team@
gnome-i18n@
gnome-doc-list@
desktop-devel-list@
● Resources: pure luck?● Problems: last cycles: always new writers;
different lenghts & styles => no consistency● 2.18 release notes written in the last days, no
volunteers● 2.20: volunteers available, better planning and
embedded into the schedule (timeframe: august 6th – September 10th) => looks good ;-)
RT: Release notes
User Documentation
● http://live.gnome.org/DocumentationProject● Aim: Provide documentation for applications● Tasks: Write and update documentation● Resources, Stats: ?● Problems: not always up-to-date (only a few
weeks between freeze and release) => manpower (?); no way of tracking actuality
Documentation: How up to date?● Outdated if UI changes a lot => developers
have to inform the doc writers● Docs that haven't been updated for a long time:
svn log limi t 1 h ttp:/ /akla pper@ svn.g nome. org/ svn/ $modu le/tr unk/{d oc,he lp,do cs}/C /$mod ule.x ml
(excluded build fixes, migration to g-d-u)● => oldest docs: bug-buddy; gnome-keyring-manager
(2005-03-25), fast-user-switch-applet, zenity, gnome-applets\battstat (2005-12-11)
● http://l10n.gnome.org/module/: add „last change“ info?
I18N/GTP
● Aim: Translate UI and Documentation to other languages
● Resources: 2 GTP spokepersons (http://live.gnome.org/TranslationProject/SpokesPersons), >100 language teams (http://developer.gnome.org/projects/gtp/ teams.html), each one with one coordinator and an unknown number of translators
I18N/GTP
● Stats: damned-lies at http://l10n.gnome.org per language, module, team, and release set. Powerful.
● Tasks of GTP: String Freeze Break Approvals, Language Team Maintainership changes, new languages approval: gnome-18n@ for entry at l10n.gnome.org, bugzilla team for creation of L10N component, and later on accounts-team for svn permissions (only 3 1/2 people on accounts team)
I18N Teams: Maintenance
● http://l10n.gnome.org/languages/ lists 133 langs●
● (note: partially buggy stats due to http://bugzilla.gnome.org/416009)
Strings: 2.14 2.18UI ~35000 36916Docs ~18100 24094
I18N Teams: UI Translation changes within the last 12 months
Positive (>10%) Negative (>-7%)Lang 2.14 2.18 diff Lang 2.14 2.18 diffBengali Indian 9 80 71 Nepali 97 77 -20Malayalam 19 76 57 Albanian 98 84 -14Slovenian 40 89 49 Indonesian 96 84 -12Oriya 18 61 43 Czech 98 87 -11Marathi 9 52 43 Croatian 65 54 -11Malagasy 8 50 42 Romanian 93 83 -10Arabic 57 99 42 Norwegian Nynorsk 80 70 -10Tamil 67 85 18 Serbian Latvian 98 89 -9
Serbian 98 89 -9Welsh 97 88 -9Canadian English 96 87 -9Latvian 97 89 -8Georgian 66 58 -8Slovak 61 54 -7
GNOME 2.18: 63 langs with UI translation >50%
Source: Stats as per http://l10n.gnome.org/releases/gnome-2-1* on 2007-07-10, 02:00 UTC
I18N Teams: Docu Translation changes within the last 12 months
Docs with >10% status ComparisonLang 2.14 2.18 diff Lang UI DocsSpanish 71 88 17 Arabic 99 0French 34 85 51 Dzongkha 99 0Swedish 24 70 46 Macedonian 99 0Italian 41 44 3 Hungarian 99 2Russian 14 40 26 Catalan 99 2British English 0 34 34 Finnish 99 2Ukrainian 33 32 -1 Lithuanian 99 0Brazilian Portuguese25 27 2 Danish 99 0Punjabi 5 18 13 Japanese 99 1Portuguese 0 16 16 Vietnamese 99 0Korean 0 14 14German 0 12 12Bulgarian 11 12 1 63 langs with UI >50%, but
13 langs with Docs >10%?
Source: Stats as per http://l10n.gnome.org/releases/gnome-2-1* on 2007-07-10, 02:00 UTC
I18N Teams: Maintainer changes and other non-“freeze requests“
2.13 9 1 82.15 12 6 72.17 9 0 6
Maintainership change or drop actively by maintainer himself, or after a complaint with positive reaction of maintainer to change
Complaint about unresponsiveness, and NO response by the team leader => maintainership change
Granting for SVN Access; webpage or email address change, or wrong maintainer in b.g.o
● data taken from gnome-18n@ mailinglist
● response times?
I18N Teams: Maintainer changes etc: Response times of GTP
●
● some requests perhaps answered off-list, or really unanswered...
Version2.13 17 16 1 7,062.15 23 20 2,5 7,752.17 13 8 0 2,38
Number of requests
„Trackable“ requests
Median [in days]
Average [in days]
GTP: Announcement amounts
●
● not significant
2.14 (6 weeks) 2.18 (8 weeks)Total 163 69Top 3 evolution (58) gedit (8)
gnome-utils (13) eog (6)gnome-panel (12) evolution (6)
Bugsquad
● Aim: Quality Assurance (http://live.gnome.org/Bugsquad)
● Tasks: Keeping track of bugs, make major bugs not go unnoticed, user help and feedback.
● Resources: Technical bugmasters and varying number of triagers (2-8 „hardcore“ triagers, plus maintainers and some „normal“ triagers)
● Existing stats: b.g.o/weekly-bug-summary.cgi, b.g.o/duplicates.cgi, Patches: b.g.o/reports/patch-diligence-report.cgi, b.g.o/reports/patch-report.cgi (general)
Bugsquad: Workload
2005 2006 2007Amount of Days: 365 365 193
18888 26942 30691Opened in that period: 37845 67543 64952
+78%+81%
Closed: 34196 59006 62413
3211 8371 6788
Current open reports, excludes enhance. req.:
#Bugs closed by top bug closer
All-time: 15842
Sources: <http://developer.gnome.org/projects/bugsquad/statistics/2005/> and /2006/>, <http://bugzilla.gnome.org/reports/weekly-bug-summary.cgi?days=194&products=0>, <http://mail.gnome.org/archives/desktop-devel-list/2006-January/msg00004.html>, Olav
Top 10 Bug closers in 2007
Bugs without a responseProductEvolution 4259 1045 24,54%nautilus 2057 665 32,33%gtk+ 2015 406 20,15%doxygen 650 344 52,92%rhythmbox 818 100 12,22%gnome-vfs 456 120 26,32%epiphany 612 23 3,76%GnuCash 514 38 7,39%gnome-panel 436 69 15,83%gnome-applets 377 124 32,89%gimmie 296 233 78,72%Gnumeric 486 55 11,32%Evolution Exchange 288 117 40,63%GtkHtml 319 65 20,38%Gstreamer 446 56 12,56%
Open bugs (incl. enhanc.)
Bugs without response
Source: top 15 modules of the last 365 days (http://bugzilla.gnome.org/reports/weekly-bug-summary.cgi?days=365&products=30) as per 2007/07/13 UTC 02:45
Graphical Reports?
Source: http://bugzilla.ximian.com/reports.cgi?product=Evolution& output=show_chart&datasets=NEW%3A&banner=1&quip=0
Bugsquad
● Problems: Amount of untriaged bugs, response rate and time; understaffed products (bugzilla mail ignored by maintainers and only up to triagers), quality of stacktraces and installing debug packages, hacked up Auto-reject feature, manpower, Patch nagging (-> Patchsquad BoF on Saturday 21st, 10:30)
● 2.20: Breakpad Support (see Fernando Herrera's „Managing bugs and crashes in GNOME“, Sa 21st July, 14:00)
● => Major aim: reduce workloud
Bugsquad: Flood me, baby!
Bugsquad: Auto-reject
● http://live.gnome.org/Bugsquad/AutoReject●
Bugsquad: Auto-reject
Date 2007/05/08 2007/06/10 2007/07/10Nautilus 8925 9347 9603Evolution 10604 11190 11524TOTAL 26236 27571 28619
Dead modules in SVN
● Problems: Identify dead modules (bit rott) => translators waste time; or new maintainership needed?
● "svn log --limit 5" to find modules without recent changes, then send emails to maintainers: http://live.gnome.org/ThomasAndersen/CvsCleanup, but only average response rate (note to myself: nag again)
● waiting for bkor to set up an SVN archive, should happen within thre next months(TM)
Other stuff
● Bugzilla: Response times (confirmed -> new; first comment of !reporter) and rates
● Accounts Team (http://live.gnome.org/AccountsTeam) needs more people (complaints about waiting weeks to get SVN access at gnome-i18n@)
Summary
● We have some nice snapshot stats, but we miss stats showing changes of a period of time => easier to identify dangerous developments
● sometimes communication fails (even if a process is well-documented)
● We could probably be more successful and happier with better organization... conclusions?
Thanks for your interest!