Plugin Quality Check

Penny Wyatt

JIRA QA Team Lead

Atlassian

Quality Assistance

• Help devs to deliver value to customers

• Testing and testing advice

• Bug prevention

Plugin Quality Process

• Make it work

• Make it lustworthy

• Make it maintainable

• Make it safe

Setting the Quality Bar

• Tradeoff between time, scope and quality

• Mismatched quality expectations - unhappy customers

Make it work

“But I’ve already tested it, it works!”

in the real world

Happy Path Fallacy

• “We’ll only worry about the happy path, for now”

• Only test the actions a normal, reasonable user would perform.

Happy Path Fallacy

• Quality bar defines what you fix, not what you test.

• OK not to support everything.

• Set customer expectations!

Valid Scenarios

• Sample Plugin

• Purpose-built but not contrived

• Completely unrelated to similar plugins on PAC!

• Only a short list, more detail on CAC

Valid Scenarios

• Different types of data

• Special characters

• Large data sets

• Long strings

Valid Scenarios

• Different browsers

• Layout issues

• Broken functionality

• Behaviour in unsupported browsers

Valid Scenarios

• Different use cases

• Session timeouts

• Deletions/Moves/Edits

• Anonymous access

Make it Lustworthy

User experience

• Guide users towards the actions they should do.

• Avoid empty boxes - use appropriate controls.

• Help them to avoid mistakes.

• Reading documentation should not be required.

Administration experience

• Clear flow for configuration

• Provide helpful error information

• Don’t show stack traces

• Use logging sparingly

Make it Maintainable

Manual and Automated Testing

• Manual testing

• Fast, effective, broad.

• Only tests the current state.

• Automated testing

• Scalable, sustainable.

• Takes time, limited assertions.

Automated Testing

• Good for long-term regression tests

• Optimise for:

• Scenarios most likely to break

• Integration with code out of your control

• Use page objects for UI tests for maintainability

Version Numbering

• Versions are constant

• A version number refers to one state of the code only.

• Never re-release a plugin with the same version number.

25

26

v2.5

v2.5v2.5

Latest?

Yes!

v2.5

Help!

v2.5

???

v2.5

Argh!

!!!!!!!!!

27

28

Version Numbering

• One version number means one binary.

• Always bump up the version number.

29

Make it Safe

How plugins break apps

• XSS

• Inserting user-supplied data into HTML without HTML-encoding it.

• Allows an attacker to gain control of the victim’s browser.

How plugins break apps

• Insufficient permission checking

• Exposing data to anonymous users

• Not respecting permission schemes

How plugins break apps

• Performance

• Synchronous external requests

• Unbounded memory allocation

How plugins break apps

• Lack of CSS scoping

• Scope every item in the plugin CSS

• Avoid overriding built-in styles

• Lack of JavaScript scoping

(function () {

// code goes here

})();

Final check

Made it work

Made it lustworthy

Made it maintainable

Made it safe

Ship it!

#atlascamp

TAKE-AWAYS

Penny Wyatt

pwyatt@atlassian.com