testing tools in alfresco. about me fran Álvarez zaizi iberia director keywords: ecm, information...

Testing Tools in Alfresco

About me

• Fran Álvarez• Zaizi Iberia Director• Keywords: ECM, Information retrieval, testing, semantic

technologies

Twitter: @fran7alvarezEmail: falvarez@zaizi.comSkype: fjalvarezb

WARNINGThis presentation may contain technical content

Manager discretion is advised

Index

• Testing concept

• Zaizi testing stack

• Challenges

• Ecosystem

• Continous Testing

• Advice

• Evolution

• TDD in Alfresco

• Conclusions

What is ‘Test’

A test is an assessment intended to measure a test-taker’s knowledge, skill, aptitude, physical fitness or classification in many other topics.

Test in Software

Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test.

Our measurement (aka Metrics)

• How well our code works

• How ‘well coded’ our code is

• How good our tests are

• How well the requirements are met

• How well our features responds to massively requests

•

Our assessment

•Different ways to measure:• Border Line

• 100%-0%

•Acceptance criteria definition• Example:

• 100% UT, 95% FT, 70% CC

Our test-taker…

…knowledge

The importance of testing

• Testing means: • In architecture:

• The building where I live is not going to fall down

• In aeronautics:• The plane where I’m traveling is not going to fall down

• In food industries:• The food you are having is not going to cause you pain

Have I convinced you yet?

The difficulty of testing

• Other tested subjects know their ‘environment’

The difficulty of testing (II)

• In Software Engineering we may have:• Code run in different servers

• Race conditions• Performance issues

• Code is integrated with 3rd parties• 3rd parties changed• 3rd parties might not be available

• Requirements change after placed in production

…

The difficulty of testing (III)

• It was never easy to properly test

Don’t waste your time testing! We need to get more

profit from the project

Scenario

Scenario

• Lots of features• Long duration project• Several components:

• API• Behaviours• Out-of-the-box customizations• Huge permission complexity• 3rd party integrations • Etc

The ZAIZI Testing Ecosystem

Coding does not end within the code

The ZAIZI Testing Ecosystem

The ZAIZI Testing Ecosystem

• Perform unit tests for each piece of code

• Mock 3rd parties or subsystems

• Create reports for code covering and code health

• Create harnesses for APIs

• Mimic user operations for functional tests

• Create report for multiuser and multithread testing

• And more…

The ZAIZI Testing Ecosystem : Maven

• The foundations

• Full lifecycle management

• Deliver Alfresco code in any format• War, AMP, Jar, RPM

• Integrate some useful plugins• Surefire• Cobertura• Sonar

The ZAIZI Testing Ecosystem : jUnit

• Unit tests

• Maven surefire integration

• Virtual Alfresco context

• Clean run every time

• Database wiping

• Does not generate the package if some test

fails

The ZAIZI Testing Ecosystem : Mockito

• Mimic components, 3rd parties

• Very useful to isolate your code

• Just test what’s yours

The ZAIZI Testing Ecosystem : Sonar

• Open Source code quality platform

• We used for:

• Check how complex the code is

• Cyclomatic complexity

• Check the style of the code

• Variable/Methods names

• Code coverage• An alternative for cobertura

The ZAIZI Testing Ecosystem : Cobertura (II)

•Line coverage acceptable

•Branch covering not enough

The ZAIZI Testing Ecosystem :JMeter

• Central axis

• Running services automatically

• HTTP: Webscripts

• SMTP: Emails

• Etc

• Used for creating test suites:

• API harnesses

• Emulate users

• Massive requests

The ZAIZI Testing Ecosystem : JMeter (III)

• Test harness purposes:

• Catalog of operations

• Practical examples• Inputs, outputs

• Check unexpected changes visually

DEMO

The ZAIZI Testing Ecosystem : JMeter (IV)

• Functional test purposes

• Collecting use cases

• Emulating them

• Checking the response is the one expected

The ZAIZI Testing Ecosystem : JMeter (V)

• Performance test purposes

• Basic operations executed several times

• Several threads, several users

• Reports for diagnosis

• Combined with Appdynamics

The ZAIZI Testing Ecosystem : Confluence

• Used for uploading reports

The ZAIZI Testing Ecosystem : Confluence

• Checks every single jMeter operation

The ZAIZI Testing Ecosystem : AppDynamics

• Used for checking and montoring the

performance of the applications in the

environments

• Detect slow transactions, memory leaks, etc

• Huge amount of options and configurations

• Could be talking about it for hours

The ZAIZI Testing Ecosystem : AppDynamics

Automated Continous Testing

•Testing without human involvement

• Dedicated server for testing purpose

•Continous deployment purpose

•Configurations close to production

•Repeatable, robust, auditable

Again, thanks to the customer!!

Automated Continous Testing

•Not anymore!!

Automated Continous Testing - Algorithm

Don’t become a test-addict

• Don’t ‘all-in’ if you don’t need it• Meet your requirements, do your testing

Leverage your code with testing, not the opposite!

Evolutioning the ecosystem

• Making functional tests more functional

• Detecting some behaviors within the log files

Ecosystem 2.0: Splunk

• Monitor Alfresco logs with ‘live’ tests• Detect unexpected outputs• Alert unexpected behaviour

“Would you like to know more?”

Ecosystem 2.0: Selenium

• Limitations with JMeter in Share

• Asynchronous calls are not catched properly

• Selenium is perfect to emulate user&browser

interaction

TDD in Alfresco

• Test Driven Development• Simple software development technique

• Short development cycles

• Focused on deliver what is required, nothing else

TDD in Alfresco

TDD in Alfresco

• Possible thanks to Alfresco and Maven-Surefire integration

• Virtual Database and Alfresco context

TDD in Alfresco

• Benefits• You focus on delivering, not coding for coding• Coding behaviors, not just code• Tests are really part of documentation• Debug actual code from tests• Usually less number of defects

• Inconvenients• Tests are slow – Alfresco context starting up• Not prepared to test UI

Cost Savings

•Time saving• Validation could take more than 3 hours

• Create, check, delete…• Project duration might be 2+ years

• Do you need a calculator?• Reducing development cycle

•Moral savings• Testing the same everyday is frustrating• You don’t test the very same twice

•Cost effective testing• You only test when something went wrong

Conclusions

Conclusions

• Testing is a good QA indicator

• Not conform yourself with Unit tests

• Build a stack to fully test your apps• You will save time and frustations

• Don’t become crazy by testing

Credits (or thanks, thanks, thanks)

• Trafigura• Michael Phan – Technical Architect

• James Bennett – Head of QA

• Ian Carr – Project Manager

Thank you!