automation process qaac

8/12/2019 Automation Process QAAC

1/23

Test Automation Success:

Choosing the Right People & Process

Kiran Pyneni, Automation Manager

Aetna, Inc.


2/23

Agenda

Automation Lifecycle

Automation Team

Metrics Of Our Success

Questions

Automation Coverage, Efforts Automated, Utilization & Cost Avoidance

Company Background

Strategy Team

Delivery Team

Benefits of Teams Strategy

Benefits of Process Integration

Our Journey

Automation Objectives


3/23

Aetna Background

Aetna is one of the nation's leading diversified health care benefits

companies, serving members with information and resources to help themmake better informed decisions about their health care.

Products and management Programs:

Aetna offers a broad range of insurance and employee benefits products.

The first national, full-service health insurer to offer a consumer-directed healthplan, Aetna continues to lead the way with its Aetna HealthFund2 line of products,including HSA, HRA and RRA options.

Aetna offers a wide array of programs and services that help control risingemployee benefits costs while striving to improve the quality of health care, such ascase management; disease and patient safety programs; integrated medical,dental, pharmaceutical, behavioral health and disability information.

Aetna provides members with access to convenient tools and easy-to-understandinformation that can help them make better-informed decisions about their healthand financial wellbeing.

3


4/23

Automation Team

Improv es eff ic iencies and qual i ty at our o rganizat ion by automating

consis tent and repeatable manual tasks us ing industry s tandard

techno logies / tools / framewo rks.

Automation

TeamAssessments &

DesignsReports & Metrics

Standards &

Best Practices

Automation

Development &

Maintenance


5/23

Automation Objectives

Automation

Objectives

Automating Right Tasks

Sustaining and Growing

Automation Suite

Utilization of existing

automation


6/23

Our Journey

Benefits Easy identification of

automation opportunities

Higher collaboration betweenapplication & automation teams

Challenges Automation development has

become expensive

Usage of different tools &solutions

Standards & Best practices arenot shared across applications

Benefits Standardization of automation

tool set at enterprise level

Sharing of coding best practices,templates and standards

Cost of automation developmentis lowered

Challenges Slower automation coverage

progression

Lower automated scriptsutilization

Funding Challenges

Decentralized Approach Centralized Approach


7/23

Our Journey Contd.

Challenges & Solutions

Challenges

Identification of automation opportunities

Prioritization of automation projects

based on ROI

Cost of automation

Implementation of optimized designs

Solution

Structuring automation as strategy &

delivery team

Challenges

Early completion of maintenancefor

automation scripts

Funding for automation projects

Collaboration between automation &business teams

Solution

Create automation life cycle process and

integrate with company's overall software

development life cycle


8/23

Automation Team Structure

Automation Team

Automation

Strategy Team

Automation

Delivery Team


9/23

Automation Strategy Team

Definition: To provide automation strategic direction and identify

potential new automation opportunities

Characteristics: Project management experience

Various automation frameworks/tools knowledge

Ability to manage multiple projects in parallel

Very strong written and verbal communication skills

Very strong leadership and analytical skills

Activities: Process & Best Practices

Assessment & Strategies

Design Solutions & Techniques

Estimations & Planning

ROI Cost/Benefit Analysis

Reports & Metrics

Tool Technical Support & Analysis


10/23

Automation Delivery Team

Definition: To develop, deliver, and maintain automation solutions

Characteristics: Understanding of programming principles and best

practices

Knowledge of various automation tools

Knowledge of multiple programming languages

Strong communication skills

Activities: Scripts Development

Scripts Maintenance

Coding Standards

Scripts Planning & Execution

Day to Day interactions w/End User


11/23

Benefits of Teams Strategy

Benefits

Easy identification of automation opportunities

Prioritization of automation projects based on ROI

Improved automation design

Optimized the cost of automation

Clearly defined roles & responsibilities

Better communication process

Optimized automation project planning


12/23

Preliminary Estimations

Automation Test Strategy

Business Case - ROI

Selected automation tool

Detailed Estimations

Go, No Go

Automation Project Plan

Framework documentation

Re-usable components,

Portability, test data grid

design, etc.

Business Process Test

(BPT) components

Automation Scripts

Test Data

Tested automation

scripts

Automated test

execution results

Lessons Learned

Sign Off

G1 G2 G3

Automation Evaluation

Understand functionality of

the project / application

Understand testing

processes

Understand target

environments

Understand test data

details

Verify automation tool

compatibility

Identify Automation

opportunities

Phases

Ac

tivities

Deliverables

Automation Preparation

Identify the resourcesneeded.

Understand the

requirements with the helpapplication demos.

Create automation

framework / design

Identify timelines and

milestones

Training required for

automation tools

Tool installation if required

Access & Environment

Setup

Automation Generation

Create Business Process

Test (BPT) components

Parameterize test data in

BPT components

Review components for

quality and correctness

String the BPT

components together to

create a test script

Develop automation scripts

for BPT components

Identify test data to

conduct unit & acceptance

testing

Unit testing of

automation

Deliver automation to

application team

Execute automation by

application team

Validation of ROI

Retrospection

Sign off

Execution & Acceptance

Automation Lifecycle


13/23

Team Collaboration

AutomationEvaluation

Automation

Preparation

Automation

Generation

Execution

& Acceptance

Automation

Strategy Team

Automation

Delivery Team


14/23

Automation ROI Calculator


15/23


I t ti f A t ti Lif l &


16/23

Integration of Automation Lifecycle &

Software Development Lifecycle

Automation

Evaluation Feasibility/Analysis

Automation

Preparation Design

AutomationGeneration Implementation

Execution &

Acceptance

Testing &

Maintenance


17/23

Benefits of Process Integration

Benefits

Improved understanding on automation process across

the entire organization

Better collaboration between business and automationteams

Funding challenges are removed

Improved on-time delivery of automation scripts

Helped to promote the automation across entireorganization

Formal escalation process developed

Initialized the creation of automation reports & metrics

Automation Coverage


18/23

Automation Coverage

Regression Testing

42%Q1-2009

45%Q2-2009

47%Q3-2009

52%Q4-2009

35%Year-2008
http://www.gnurf.net/v3/wp-content/uploads/2009/04/041-vintage-racer.pnghttp://www.gnurf.net/v3/wp-content/uploads/2009/04/041-vintage-racer.pnghttp://www.gnurf.net/v3/wp-content/uploads/2009/04/041-vintage-racer.pnghttp://www.gnurf.net/v3/wp-content/uploads/2009/04/041-vintage-racer.pnghttp://www.gnurf.net/v3/wp-content/uploads/2009/04/041-vintage-racer.png


19/23

Manual Testing Efforts Automated

7,047 hrs

8,570 hrs

10,200 hrs

11,750 hrs

13,803 hrs

Year As of 2008 As of 2009 2009 only efforts

Total Efforts Automated 7047 hrs 13803 hrs 6756 hrs

Year-2008 Q1-2009 Q2-2009 Q3-2009 Q4-2009


20/23

Utilization & Cost Avoidance

Q1-2009 Q2-2009 Q3-2009 Q4-2009

Cost Avoidance

7,200 hrs

10,810 hrs

9,618 hrs 9,867 hrs

Q1-2009 Q2-2009 Q3-2009 Q4-2009

Utilization

45%

Year-2008

87%

150% 134% 130%


21/23

Savings Vs. Cost Avoidance

Cost Savings

Efforts difference between manual and automation testing

Manual (Not automation) execution frequency

Produced by automation strategy team after assessment


Cost Avoidance

Test coverage difference between manual & automation testing

Efforts difference between manual and automation testing

Automation execution frequency

Generated from Quality Center after release implementation

Helped to improve the credibility of automation metrics significantly


22/23

Automation MetricsBest Practices

Automation is a way of Testing and process followed for manual testing

can be applied

Best Practices followed in Test Management ToolQuality Center

Define automation work flow Regression Testing identifier

Automation Assessment status

Automation Development status Store automation inventory

Manage automated test scripts execution


23/23

Questions

automation process qaac

Documents