driving fit & finish in ux quality: desirability is in the details

70
Driving Fit & Finish in UX Driving Fit & Finish in UX Quality: Desirability is in Quality: Desirability is in the Details the Details Yingzhao Liu, Program Manager, WEX User Interface Yingzhao Liu, Program Manager, WEX User Interface Platform Platform Donna Wallace, Senior UX Manager, DDE User Experience Donna Wallace, Senior UX Manager, DDE User Experience Kynan Antos, UX Designer, Windows Home Server Kynan Antos, UX Designer, Windows Home Server Dan Harman, Program Manager, Dynamic Systems Dan Harman, Program Manager, Dynamic Systems Foundation Foundation August 2007 August 2007

Upload: robbin

Post on 11-Jan-2016

25 views

Category:

Documents


1 download

DESCRIPTION

Driving Fit & Finish in UX Quality: Desirability is in the Details. Yingzhao Liu, Program Manager, WEX User Interface Platform Donna Wallace, Senior UX Manager, DDE User Experience Kynan Antos, UX Designer, Windows Home Server Dan Harman, Program Manager, Dynamic Systems Foundation August 2007. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Driving Fit & Finish in UX Quality: Desirability is in the Details

Driving Fit & Finish in UX Driving Fit & Finish in UX Quality: Desirability is in the Quality: Desirability is in the Details Details

Yingzhao Liu, Program Manager, WEX User Interface PlatformYingzhao Liu, Program Manager, WEX User Interface PlatformDonna Wallace, Senior UX Manager, DDE User ExperienceDonna Wallace, Senior UX Manager, DDE User ExperienceKynan Antos, UX Designer, Windows Home ServerKynan Antos, UX Designer, Windows Home Server

Dan Harman, Program Manager, Dynamic Systems FoundationDan Harman, Program Manager, Dynamic Systems FoundationAugust 2007August 2007

Page 2: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

About this presentationAbout this presentation

Introduction Introduction What is Fit & FinishWhat is Fit & FinishDriving Fit & Finish for Windows VistaDriving Fit & Finish for Windows Vista

Fit & Finish everywhereFit & Finish everywhereVisual StudioVisual StudioWindows Home ServerWindows Home ServerServer ManagerServer Manager

Best practicesBest practicesTimelineTimelineCulture ChangeCulture ChangeIngredients for a successful processIngredients for a successful processWarning signsWarning signs

Take-awaysTake-aways

Page 3: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

What is Fit & FinishWhat is Fit & Finish

A term in the manufacturing industry in general and A term in the manufacturing industry in general and typically refers to typically refers to the process of doing the final the process of doing the final assemblyassembly:  putting the pieces together perfectly (Fit) :  putting the pieces together perfectly (Fit) and adding the final layer of paint, polish, removing and adding the final layer of paint, polish, removing fingerprints, etc. (Finish).fingerprints, etc. (Finish).

In software as in manufacturing, it can not wait to In software as in manufacturing, it can not wait to the end. the end.

Page 4: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Vista: requires high Fit & FinishVista: requires high Fit & Finish

Page 5: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Some examples of Fit & Finish Some examples of Fit & Finish bugsbugs

Page 6: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Some feedback on Release Some feedback on Release CandidateCandidate

““There are still too many pre-Vista icons. There are still too many pre-Vista icons.   Take a Take a look inside the Windows folder, for example.  The look inside the Windows folder, for example.  The “Offline web pages” icon is 95-style, and the “Offline web pages” icon is 95-style, and the “Downloaded program files” icon is 2000-style. “Downloaded program files” icon is 2000-style.  These have got to be updated!  There is ZERO  These have got to be updated!  There is ZERO excuse for not doing it…excuse for not doing it…

““There are too many slightly-different shades of There are too many slightly-different shades of blue/green throughout the product. blue/green throughout the product.   

““Each of the following apps have different Each of the following apps have different back/forward buttons: Windows Explorer, IE, Help & back/forward buttons: Windows Explorer, IE, Help & Support (*quite* different), Windows Media Support (*quite* different), Windows Media Player/Photo Gallery (these two are the same—yay!), Player/Photo Gallery (these two are the same—yay!), Windows Defender…”Windows Defender…”

Page 7: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Fit & Finish bugsFit & Finish bugs

10,000 FnF bugs were entered; 8,000 unique; 4400 10,000 FnF bugs were entered; 8,000 unique; 4400 fixed. fixed.

Bug Categories:Bug Categories:

Visual glitchesVisual glitches

Incorrect text or graphicsIncorrect text or graphics

Layout errorsLayout errors

Animation errorsAnimation errors

Visual clarity problemsVisual clarity problems

Windows Vista UX Guidelines violations Windows Vista UX Guidelines violations (http://uxguide)(http://uxguide)

UI implementation errorsUI implementation errors

Page 8: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Driving Fit & Finish for VistaDriving Fit & Finish for Vista

Got VP approvalGot VP approval

Focused on areas of the UI with the highest impactFocused on areas of the UI with the highest impact

Conducted review meetings, enforced UX Guide and Conducted review meetings, enforced UX Guide and exit criteriaexit criteria

Made it easy for anyone to enter visual bugsMade it easy for anyone to enter visual bugs

Staffed with our own testers and developersStaffed with our own testers and developers

Partnered with Basics: usability and accessibilityPartnered with Basics: usability and accessibility

Attended central shiproomAttended central shiproom

Page 9: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

About this presentationAbout this presentation

IntroductionIntroduction What is Fit & FinishWhat is Fit & FinishDriving Fit & Finish for Windows VistaDriving Fit & Finish for Windows Vista

Fit & Finish everywhereFit & Finish everywhereVisual StudioVisual StudioWindows Home ServerWindows Home ServerServer ManagerServer Manager

Best practicesBest practicesTimelineTimelineCulture ChangeCulture ChangeIngredients for a successful processIngredients for a successful processWarning signsWarning signs

Take-awaysTake-aways

Page 10: Driving Fit & Finish in UX Quality: Desirability is in the Details

Visual Studio WhidbeyVisual Studio Whidbey

Donna Wallace, Senior UX Manager, DDE User ExperienceDonna Wallace, Senior UX Manager, DDE User Experience

Page 11: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Visual Studio Whidbey (v7)Visual Studio Whidbey (v7)““One big piece of feedback from [our recent VP One big piece of feedback from [our recent VP reviews] that I agree with is that we need to focus reviews] that I agree with is that we need to focus more on end-to-end UI consistency across these more on end-to-end UI consistency across these products…. products….

……When you look at a product that is completely When you look at a product that is completely consistent in its UI model versus one that isn’t, I consistent in its UI model versus one that isn’t, I personally jump to all sorts of conclusions about the personally jump to all sorts of conclusions about the consistent one beingconsistent one being faster, more productive, faster, more productive, more stable, more compellingmore stable, more compelling, etc, etc. all because of . all because of that one attribute… in the end, the that one attribute… in the end, the consistency has consistency has a multiplier effect that helps sell the overall a multiplier effect that helps sell the overall valuevalue.”.”

— — Craig Symonds, GM Craig Symonds, GM

Page 12: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Whidbey consistency initiatives Whidbey consistency initiatives

Promote designing for consistencyPromote designing for consistencyVia design guidelinesVia design guidelines

Target high profile inconsistenciesTarget high profile inconsistenciesVia core scenario walkthroughsVia core scenario walkthroughs

Address “death by 1000 paper cuts”Address “death by 1000 paper cuts”Finding and fixing the right issuesFinding and fixing the right issues

Invest in engineering for consistencyInvest in engineering for consistencyVia greater implementation sharingVia greater implementation sharing

Page 13: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Whidbey consistency initiatives Whidbey consistency initiatives

Promote designing for consistencyPromote designing for consistencyVia design guidelinesVia design guidelines

Target high profile inconsistenciesTarget high profile inconsistenciesVia core scenario walkthroughsVia core scenario walkthroughs

Address “death by 1000 paper cuts”Address “death by 1000 paper cuts”Finding and fixing the right issuesFinding and fixing the right issues

Invest in engineering for consistencyInvest in engineering for consistencyVia greater implementation sharingVia greater implementation sharing

Page 14: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Components of UX issue Components of UX issue managementmanagement1.1. Identify and prioritize all UI elementsIdentify and prioritize all UI elements

2.2. Determine prioritization schemeDetermine prioritization schemeSystematic method, communicated broadlySystematic method, communicated broadly

3.3. Find the right issues / bugsFind the right issues / bugs

4.4. Fix the right issues / bugsFix the right issues / bugsDefine success: set quality targetsDefine success: set quality targetsTriage team keeps teams honest and ensures Triage team keeps teams honest and ensures consistent severity ratings across the divisionconsistent severity ratings across the division

5.5. Track and reportTrack and report

Page 15: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Identify UIIdentify UIUX : track UX reviews, Fit & Finish progressUX : track UX reviews, Fit & Finish progress

UE, Loc: track UI work and changesUE, Loc: track UI work and changes

Today some of the list is generated, most is manualToday some of the list is generated, most is manual

Dialogs Tool windows Editors

Total 630 76 50

# New ? 25

24

Reviewed ? 43 9

No review planned

236 14 11

Whidbey UIComponents.xls Whidbey UIComponents.xls

Page 16: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

UI Element Priority: EP ratingUI Element Priority: EP rating

Upgrade priority based on Upgrade priority based on Part of main scenario: EP1Part of main scenario: EP1Market focus or affecting user SATMarket focus or affecting user SAT

EP1EP1

EP2EP2

EP2EP2

EP3EP3

Page 17: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Severity definitionsSeverity definitionsSev 1 • Can’t use the feature

• Product appears unstable• Inadvertent user action causes data loss

Sev 2 •Feature is inconsistent with other elements • Customers are frustrated•Hard to use •Product appears poorly engineered

Sev 3 • Customers annoyed by UI inconsistencies• UI appears sloppy

Sev 4 • Little or no user impact

Page 18: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Example: Severity 1 UI BugsExample: Severity 1 UI Bugs

Breadth:Breadth: 90% of customers will regularly see 90% of customers will regularly see issueissue

Depth:Depth: Prevents usage Prevents usage

Impact:Impact: Same impact as any Sev 1 technical Same impact as any Sev 1 technical bugbug

Page 19: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Example: Severity 2 UI BugsExample: Severity 2 UI Bugs

Breadth:Breadth: > 50% of customers will regularly see > 50% of customers will regularly see issueissue

Depth:Depth: Significantly impacts usage Significantly impacts usage

Impact:Impact: Customer questions overall stability Customer questions overall stability

Page 20: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Example: Severity 3 UI BugsExample: Severity 3 UI Bugs

Breadth:Breadth: ~ 25% of customers will see paper ~ 25% of customers will see paper cutcut

Depth:Depth: Annoying but doesn’t significantly Annoying but doesn’t significantly impact usageimpact usage

Impact:Impact: Multiple bugs portray sloppiness & lack Multiple bugs portray sloppiness & lack of professionalismof professionalism

Page 21: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Example: Severity 4 UI BugsExample: Severity 4 UI Bugs

Breadth:Breadth: Only the trained eye will notice paper Only the trained eye will notice paper cutcut

Depth:Depth: Doesn’t impact daily usage Doesn’t impact daily usage

Impact:Impact: Customers won’t question specific bug, Customers won’t question specific bug, but multiple bugs make UI appear unfinishedbut multiple bugs make UI appear unfinished

Page 22: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Prioritization matrix defines Prioritization matrix defines goalsgoals

EP1 EP2 EP3Sev 1

Pri 1 Pri 1 Pri 2

Sev 2

Pri 1 Pri 2 Pri 3

Sev 3

Pri 2 Pri 3 Pri 4

Sev 4

Pri 3 Pri 4 Pri 4

Page 23: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Conducted by UI experts and feature teams Conducted by UI experts and feature teams together at each milestone start or end together at each milestone start or end

Evaluate both task flow and UI detailsEvaluate both task flow and UI details

Avoid surprises: Disclose upcoming feature workAvoid surprises: Disclose upcoming feature work

Get a “B” in coverage: Get first look at low Get a “B” in coverage: Get first look at low priority areaspriority areas

Gain consensus, set expectations accordinglyGain consensus, set expectations accordingly

Provide the management team a milestone Provide the management team a milestone checkpoint on the “Overall Product UI Health” checkpoint on the “Overall Product UI Health” scorecardscorecard

Many issues can be fixed without logging bugsMany issues can be fixed without logging bugs

Bug finding: UI reviewsBug finding: UI reviews

Page 24: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

High profile inconsistenciesHigh profile inconsistencies

Smart TagsSmart Tags

Early designs: similar needs, Early designs: similar needs, but multiple but multiple implementations, very implementations, very different appearance and different appearance and behaviorbehavior

Page 25: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

High profile inconsistenciesHigh profile inconsistencies

Final designs: fewer implementations Final designs: fewer implementations with a common look and feel. with a common look and feel.

Page 26: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Bug finding: UI testingBug finding: UI testing

Ad hoc UI bugs (from UI Guidelines posted for teams)Ad hoc UI bugs (from UI Guidelines posted for teams)

Systematic approach: UI test casesSystematic approach: UI test cases

Derive objective test cases from guidelinesDerive objective test cases from guidelines

Format test cases to fit QA toolsFormat test cases to fit QA tools

Teach QA teams how to review UITeach QA teams how to review UI

Categories of bugs:Categories of bugs:CommunicationCommunicationEase and efficiencyEase and efficiencyError preventionError preventionVisual designVisual designConsistencyConsistencyEngineeringEngineering

Page 27: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Detail design Fit & Finish: Setup Detail design Fit & Finish: Setup

Before:Before:Pseudo-web Pseudo-web appearance appearance and navigationand navigation

Brand Brand imagery imagery not not consisteconsistentnt

Nonstandard Nonstandard themed themed controlscontrols

Nonstandard Nonstandard font face & font face & colorcolor

nonstandard nonstandard colorscolors

Use of links to Use of links to perform perform actionsactions

nonstandard nonstandard grid grid

Page 28: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Detail design Fit & Finish: Setup Detail design Fit & Finish: Setup

Updated use of Updated use of brand imagerybrand imagery

Standard Standard themed themed controlscontrols

Standard Standard foreground foreground color & color & fontsfonts

Standard colorsStandard colors

Appropriate use Appropriate use of linksof links

Use of buttons for action Use of buttons for action commandscommands

Standard grid Standard grid

After:After:

Page 29: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

ReportingReporting

 

All Active Fixed Won’t Fix/Postponed

All UX Bugs 4303 342 1952 706

Sev 1 / Sev 2 1993 136 922 239

XP Theme Bugs

105 2 55 9 (2 violate exit criteria)

Key Guidelines Violations

120 3 56 0

• Due to the security push, seeing lower fix rates over those weeksDue to the security push, seeing lower fix rates over those weeks• 102 UX bugs added since 11/12102 UX bugs added since 11/12• 73 UX bugs resolved as Fixed since 11/1273 UX bugs resolved as Fixed since 11/12• 13 UX bugs closed as Won’t fix or Postponed since 11/1213 UX bugs closed as Won’t fix or Postponed since 11/12

Page 30: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

In summaryIn summary

Things that worked wellThings that worked wellKey: had clear directive from executivesKey: had clear directive from executives

Built reviews into divisional schedule and set expectationsBuilt reviews into divisional schedule and set expectations

Systematic approach resonated well with engineering practicesSystematic approach resonated well with engineering practices

Ensured flexible process to tailor fit individual teamsEnsured flexible process to tailor fit individual teams

Had adequate reporting and communication both up and downHad adequate reporting and communication both up and down

Things that we’re still working onThings that we’re still working onUnderstanding how to better integrate into Feature Crew model Understanding how to better integrate into Feature Crew model

Tuning quality gates and exit criteriaTuning quality gates and exit criteria

Measuring success beyond using only bug counts (find or fix Measuring success beyond using only bug counts (find or fix

rates)rates)

Defining a method to better identify “dirtiness factor”Defining a method to better identify “dirtiness factor”

Page 31: Driving Fit & Finish in UX Quality: Desirability is in the Details

Windows Home ServerWindows Home Server

Kynan Antos, UX Designer, Windows Home ServerKynan Antos, UX Designer, Windows Home Server

Page 32: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Fit & Finish is about qualityFit & Finish is about quality

“From what we’ve seen so far, we’d rate Windows Home Server as one of Microsoft’s most polished and most impressive 1.0 releases to date.”

— APC Magazine

Page 33: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Examples of FnF issues we Examples of FnF issues we encountered…encountered…

Improperly rendered transparency…Improperly rendered transparency…

Page 34: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

colors didn’t match…colors didn’t match…

Background mismatchBackground mismatchAliased edges: lack of Aliased edges: lack of transparencytransparency

Page 35: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

placeholders were still placeholders were still placeholding…placeholding…Placeholder icons…Placeholder icons…

Page 36: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

deep investigation led to big deep investigation led to big issues…issues…

Collision…Collision…

Page 37: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

poor implementation created poor implementation created issues…issues…

Artifacts from Artifacts from stretching…stretching…

Page 38: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

creative solutions are often creative solutions are often necessarynecessary

btn_end_lft.pngbtn_end_lft.png btn_bg.pngbtn_bg.png btn_end_rt.pngbtn_end_rt.png

Page 39: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

sometimes things were all sometimes things were all wrong…wrong…Coordinates and spacing…Coordinates and spacing…

Page 40: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

all of these issues add up fast…all of these issues add up fast…

Page 41: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

it isn’t risky (or difficult) to fix it isn’t risky (or difficult) to fix them…them…

Fit & Finish Bugs Filed

Bugs Resolved Regressions

114 74 1

Page 42: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

...fixing these issues has a big ...fixing these issues has a big impact.impact.“We don’t want to award anyone before the

final product is out, but we’ll say that we came out of the closed beta very impressed.”

— CNET

“I found Home Server to be intuitive to use... Its simplicity and automation will make it a winner with many home users.”

— Computer World Magazine

Page 43: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

What workedWhat worked

Adoption of the Vista Fit & Finish Adoption of the Vista Fit & Finish

Involve team in prioritizing feature areasInvolve team in prioritizing feature areas

Use soft skills, build relationshipsUse soft skills, build relationships

Get out of your officeGet out of your officeWork directly with developers to fix issuesWork directly with developers to fix issues

Write good bugs, always use screenshotsWrite good bugs, always use screenshots

Focus on quality in the product, not on bug countFocus on quality in the product, not on bug count

Page 44: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

What didn’t workWhat didn’t work

Lack of early executive buy-inLack of early executive buy-in

Started too late in the scheduleStarted too late in the schedule

Feature slip was prioritized over Fit & Finish scheduleFeature slip was prioritized over Fit & Finish schedule

No time to file, triage, and manage bug overheadNo time to file, triage, and manage bug overhead

No ship criteria with teethNo ship criteria with teeth

No incentives for building qualityNo incentives for building quality

Page 45: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

What we are doing next timeWhat we are doing next time

Getting early executive buy-inGetting early executive buy-inGetting Fit & Finish on the schedule earlyGetting Fit & Finish on the schedule earlyScheduling Fit & Finish within each milestone after ZBBScheduling Fit & Finish within each milestone after ZBB

Using Product Studio for tracking Fit & Finish issuesUsing Product Studio for tracking Fit & Finish issues

Developing and enforcing Fit & Finish ship criteria Developing and enforcing Fit & Finish ship criteria

Giving UX the authority to get resultsGiving UX the authority to get results

Sharing the Fit & Finish processSharing the Fit & Finish process

Page 46: Driving Fit & Finish in UX Quality: Desirability is in the Details

Server ManagerServer Manager

Dan Harman, Program Manager, Dynamic Systems FoundationDan Harman, Program Manager, Dynamic Systems Foundation

Presented by Yingzhao LiuPresented by Yingzhao Liu

Page 47: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Server ManagerServer Manager

Page 48: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Review Process for Server Review Process for Server ManagerManager““We were able to fix numerous issues that We were able to fix numerous issues that

would not have been fixed otherwise, and would not have been fixed otherwise, and collectively, the impact on the UX has been collectively, the impact on the UX has been significant. Combined with the changes to significant. Combined with the changes to our team culture after having witnessed the our team culture after having witnessed the cost versus benefit, we’re impressed with the cost versus benefit, we’re impressed with the results.”results.”

—Dan Harman, PM

Page 49: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Review Process for Server Review Process for Server ManagerManager• Used Vista Fit & Finish review process as a baselineUsed Vista Fit & Finish review process as a baseline• Conducted two 2-hour review sessions in March 2007Conducted two 2-hour review sessions in March 2007• Focused on high priority areas:Focused on high priority areas:• Core console elementsCore console elements• Core wizard framework elementsCore wizard framework elements• Custom controlsCustom controls• Dynamics in the UIDynamics in the UI• Key experiencesKey experiences

• Involved participants from 4 disciplines: PM, design, Involved participants from 4 disciplines: PM, design, dev, and testdev, and test• PM drove process, took notes, and filed bugsPM drove process, took notes, and filed bugs• Design contributed to reviews and assigned severity to issuesDesign contributed to reviews and assigned severity to issues• Test contributed to reviews and provided test environmentsTest contributed to reviews and provided test environments• Dev triaged, costed, and fixed bugsDev triaged, costed, and fixed bugs

Page 50: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Filing BugsFiling Bugs

• Used same bug categories as Vista Fit & Finish Used same bug categories as Vista Fit & Finish reviewsreviews• Visual glitchesVisual glitches• Incorrect text or graphicsIncorrect text or graphics• Layout errorsLayout errors• Animation errorsAnimation errors• Visual clarity problemsVisual clarity problems• UX Guide violationsUX Guide violations• UI implementation errors UI implementation errors

• Followed Vista Fit & Finish process for filing bugsFollowed Vista Fit & Finish process for filing bugs• Filed bugs for all issues foundFiled bugs for all issues found• Assigned Priority 2, Severity 2 for most bugsAssigned Priority 2, Severity 2 for most bugs• Added “RmtVisual” keyword for tracking purposesAdded “RmtVisual” keyword for tracking purposes

Page 51: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

ResultsResults

• 41 issues identified:41 issues identified:• Fixed: 27Fixed: 27• Pushed: 7Pushed: 7• External: 4External: 4• By Design: 3By Design: 3

• Fixed top 80% of bugs with highest impact/cost ratio Fixed top 80% of bugs with highest impact/cost ratio (27/34)(27/34)

• Examples of bugs fixed:Examples of bugs fixed:• Default OOBE layout should not scrollDefault OOBE layout should not scroll• Painting issues on initial console loadPainting issues on initial console load• Role captions not aligned properlyRole captions not aligned properly• Padding around all tiles should be consistentPadding around all tiles should be consistent

• Examples of bugs pushed to Windows 7:Examples of bugs pushed to Windows 7:• Visual inconsistency in NotificationsVisual inconsistency in Notifications• OOBE status column font size slightly too large in 120 DPIOOBE status column font size slightly too large in 120 DPI

Page 52: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Culture ShiftCulture Shift

• Dev fixed all Fit & Finish bugs from reviews in 2 Dev fixed all Fit & Finish bugs from reviews in 2 daysdays

• Team realized visual impact of fixing these bugsTeam realized visual impact of fixing these bugs• Given good results and low cost, team embraced Given good results and low cost, team embraced

Fit & Finish reviews as a worthwhile investmentFit & Finish reviews as a worthwhile investment• Success has lowered the barrier Success has lowered the barrier to fixing other to fixing other

Fit & Finish bugs since the reviewsFit & Finish bugs since the reviews• Moving forward, plan to make Fit & Finish a Moving forward, plan to make Fit & Finish a

more formal part of development process for more formal part of development process for future releasesfuture releases

Page 53: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

About this presentationAbout this presentation

IntroductionIntroduction What is Fit & FinishWhat is Fit & FinishDriving Fit & Finish for Windows VistaDriving Fit & Finish for Windows Vista

Fit & Finish everywhereFit & Finish everywhereVisual StudioVisual StudioWindows Home ServerWindows Home ServerServer ManagerServer Manager

Best practicesBest practicesTimelineTimelineCulture ChangeCulture ChangeIngredients for a successful processIngredients for a successful processWarning signsWarning signs

Take-awaysTake-aways

Page 54: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

TimelineTimeline

Get Fit & Finish on divisional scheduleGet Fit & Finish on divisional schedule

During planningDuring planning

Conduct spec review, verify UX Guide complianceConduct spec review, verify UX Guide compliance

At key points during product cycle At key points during product cycle (e.g. milestones, beta, (e.g. milestones, beta, RC)RC)

Conduct Fit & Finish review and triage bugsConduct Fit & Finish review and triage bugs

Encourage customer bugs and feedback on UXEncourage customer bugs and feedback on UX

Conduct Fit & Finish bug bashConduct Fit & Finish bug bash

Before visual freezeBefore visual freeze

Coordinate focused Fit & Finish dev pushCoordinate focused Fit & Finish dev push

Work with localization on late graphical resource Work with localization on late graphical resource changeschanges

Page 55: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Culture changeCulture change

In our competitive markets, quality experience is In our competitive markets, quality experience is criticalcritical

Get executive buy-in—in the schedule and funded Get executive buy-in—in the schedule and funded

Clarify what we mean by shipping qualityClarify what we mean by shipping quality

Set bug Severity and Priority correctly and Set bug Severity and Priority correctly and consistentlyconsistently

Be clear that low quality work isn’t acceptableBe clear that low quality work isn’t acceptable

Be helpful, open, and respectfulBe helpful, open, and respectful

Reward high quality UX Reward high quality UX

Page 56: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Ingredients for a successful Ingredients for a successful processprocess

Use http://uxguide as baseline guidelinesUse http://uxguide as baseline guidelines

Define what successful FnF looks likeDefine what successful FnF looks like

Equip testers with specific, testable set of guidelinesEquip testers with specific, testable set of guidelines

Circulate best practices for UI codingCirculate best practices for UI coding

““Conference room testing” to call out UX issuesConference room testing” to call out UX issues

Triage and drive Fit & Finish bugsTriage and drive Fit & Finish bugs

Coordinate focused UX bug fix pushesCoordinate focused UX bug fix pushes

Communicate and report, let people know of your Communicate and report, let people know of your impactimpact

Page 57: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Warning signsWarning signs

““We are really serious about FnF, but there’s no We are really serious about FnF, but there’s no budget, nor time in the schedule.”budget, nor time in the schedule.”

  You and customers never know when to file FnF You and customers never know when to file FnF bugs: “Now is too early.” “Now is too late.”bugs: “Now is too early.” “Now is too late.”

““We will have to change documentation if we fix this We will have to change documentation if we fix this bug.” bug.”

““These FnF bugs are just in ______ mode, most These FnF bugs are just in ______ mode, most customers won’t see it.”customers won’t see it.”

““FnF is nice to have, but we have to resolve our FnF is nice to have, but we have to resolve our realreal bugs first.” bugs first.”

Page 58: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

About this presentationAbout this presentation

IntroductionIntroduction What is Fit & FinishWhat is Fit & FinishDriving Fit & Finish for Windows VistaDriving Fit & Finish for Windows Vista

Fit & Finish everywhereFit & Finish everywhereVisual StudioVisual StudioWindows Home ServerWindows Home ServerServer ManagerServer Manager

Best practicesBest practicesTimelineTimelineCulture ChangeCulture ChangeIngredients for a successful processIngredients for a successful processWarning signsWarning signs

Take-awaysTake-aways

Page 59: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

The value of Fit & FinishThe value of Fit & Finish

Conducting Fit & FinishConducting Fit & FinishPut it in the schedulePut it in the schedule(Culture) change is inevitable(Culture) change is inevitablePlant ingredients for successPlant ingredients for successKnow the warning signsKnow the warning signs

Everyone can make an impact with Fit & FinishEveryone can make an impact with Fit & Finish

Take-awaysTake-aways

Page 60: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

ResourcesResources

Windows Vista User Experience GuidelinesWindows Vista User Experience Guidelineshttp://uxguidehttp://uxguide (internal) (internal)

http://msdn.microsoft.com/windowsvista/uxguidehttp://msdn.microsoft.com/windowsvista/uxguide (external) (external)

Guidelines-based User Experience DesignGuidelines-based User Experience Design online course online course

Vista Fit & Finish ProcessVista Fit & Finish ProcessProcess document by Dave VronayProcess document by Dave Vronay

Talk by Everett McKayTalk by Everett McKay

Page 61: Driving Fit & Finish in UX Quality: Desirability is in the Details
Page 62: Driving Fit & Finish in UX Quality: Desirability is in the Details

AppendixAppendix

Page 63: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

FnF ReviewsFnF Reviews

Page 64: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Page 65: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Page 66: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

FnF Bugs Root Cause Analysis FnF Bugs Root Cause Analysis (RCA)(RCA)

Page 67: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Analysis of Apple’s development Analysis of Apple’s development process:process:

From Russ Albright’s Analysis of Apple’s Development From Russ Albright’s Analysis of Apple’s Development Process (FTE’s only)Process (FTE’s only)User focused culture (p. 16, 49)User focused culture (p. 16, 49)Smaller code base (15 vs. 50 mloc), leverage open Smaller code base (15 vs. 50 mloc), leverage open source code, focus on code reuse (p. 18)source code, focus on code reuse (p. 18)Better UI development frameworks and toolsBetter UI development frameworks and toolsFnF bugs are P2/P3 instead of P3/P4 (p. 52)FnF bugs are P2/P3 instead of P3/P4 (p. 52)Ready to release when 85% of bugs are resolved (p. Ready to release when 85% of bugs are resolved (p. 54)54)Have a Senior HCI Director responsible for cross Have a Senior HCI Director responsible for cross product consistency (p. 56)product consistency (p. 56)All designs must conform to their guidelines, managers All designs must conform to their guidelines, managers are responsible for consistency of their products (p. are responsible for consistency of their products (p. 58)58). .

Page 68: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Connection Dialog: refreshConnection Dialog: refreshGenerally, Generally,

provider tab rarely provider tab rarely used after the first used after the first

timetime

Generally, Generally, provider tab rarely provider tab rarely used after the first used after the first

timetime

98% of the time 98% of the time the same the same

connection page is connection page is used for all used for all connectionsconnections

98% of the time 98% of the time the same the same

connection page is connection page is used for all used for all connectionsconnections

““All” is there only All” is there only to expose more to expose more

advanced settings advanced settings in a gridin a grid

““All” is there only All” is there only to expose more to expose more

advanced settings advanced settings in a gridin a grid

Page 69: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

settings for the current provider settings for the current provider are “wrapped” by the frameare “wrapped” by the frame

settings for the current provider settings for the current provider are “wrapped” by the frameare “wrapped” by the frame

Provider is now Provider is now changed by launching changed by launching

secondary dialogsecondary dialog

Provider is now Provider is now changed by launching changed by launching

secondary dialogsecondary dialog

Advanced settings now available under an Advanced settings now available under an advanced… buttonadvanced… button

Advanced settings now available under an Advanced settings now available under an advanced… buttonadvanced… button

Page 70: Driving Fit & Finish in UX Quality: Desirability is in the Details

Engineering Excellence • Trustworthy ComputingMicrosoft Confidential

Method Communi-cation

Consistency

Ease & Efficienc

y

Engineering

ErrorPreven

t.

VisualDesign

AdHoc 1.33% 48% 9.33% 2.66% 0 38.66%

QA Guideline Test Case

30.66% 53.33% 4% 1.33% 0 10.66%

Observational Field Research

33.33% 18.66% 33.33% 5.33% 2.66% 6.66%

Milestone UI Walkthroughs

1.33% 36% 6.66% 1.33% 1.33% 53.33%

Usability Studies

50.66% 12% 26.66% 0 4% 6.66%

User Correspondence

12% 28% 36% 0 4% 20%

Cognitive Walkthrough

40% 29.33% 5.33% 9.33% 2.66% 13.33%