reporting and build statistics using business intelligence by naga sowjanya karumuri build team,...
TRANSCRIPT
Reporting and Build Statistics Using Business Intelligence
By
Naga Sowjanya Karumuri Build Team,
VMware,Cambridge
Summer Internship 2008
1. Problem Background
• The Release Engineering management team needs meaningful views into build-related data in order to make informed tactical and strategic decisions.
• For this purpose, they started generating reports using Perl/python scripts as shown below.
• Two basic problems with this method:
- Script-generated output presents values but not the trends; as a result decisions cannot be made based in this data.
- Manually generating the reports is a tedious process.
2. Manual Method
Database Perl/Python Scripts
Spreadsheet / Word Document
Querying Database
Manually Copying Script Output
3. Solution
• This project sets a Reporting system using Pentaho Business Intelligence Suite, an open source equivalent of Crystal Objects.
• Business Intelligence (BI) refers to technologies, applications and practices for the collection, integration, analysis, and presentation of business information to support better business decision making.
• Why this solution ?
- Reports present the data in tables and graphs in a way that improves understanding of the data.
- Reduces labor of manager in writing the scripts.
4. Pentaho BI Suite
Report Designer• Helps us set the layout of the report. • Connects the reports to a result set of a query.• Displays data in the result set using text and number fields.• Display data in the form of a chart.• Display date and page numbers.• Publish the report to the server
Design Studio• Assign author name, description and version number to the report.• Helps us set the output type of the report.• Allows user to give input parameters to the report.• Helps to set the cron string so as to run the report automatically at regular intervals.
5. Architecture of the Project
Backend is a reporting database which is a collective representation of BuildDB, Bugzilla, ContinuousBuild data files. This is generated by extracting data from different data sources and transforming them to tables and loading them into the reporting database.
Middle Layer generates requested reports using Report Designer and Design Studio.
Front End is a web portal that presents the list of reports that the user can generate with a single click.
6. Database and Report Designer
Source data Store Report Template from
Report Designer
Continuous Build Data files
Publish to server
Perl/python scripts
7. Adding User Interaction
Design Studio Web Portal
Add title
Add author name
Setup recurring schedule
Add User interaction to the webpage
Report
Click to view the output
8. Reports Generated
Product Statistics : The success/failure rates presented indicates which products needs more attention.
Build Summary : Presents a detailed view of the reasons for build failures, e.g. infrastructure, merge, dependencies, during the current reporting period tells us if an area requires immediate intervention.
Build Summary for 1 yr : Comparing failure trends with other trends over a similar period, to spot causal relationships.
Branch Deltas : Tracks the trend over the number of branches built upon.
9. Reports Generated: Description (cntd..)
Official Build Server Utilization : Graph will help with the prediction of when new servers need to be purchased due to low server availability. The graph can also indicate the greater need for capacity between Linux and Windows servers.
Branch Summary : Shows the trend in branches that can be used to compare to other resources over the same period.
Official Stats Summary : Provides a snapshot into the current resource usage to answer the question, “Do we have sufficient resources applied to the official build subsystem?”.
10. Official Stats Summary for 1 Year
The results of this report are shown below. From this we can learn :
• Trends among various build resources. For example, did an increase in active branches result in increased build failures.
• Trends in Success Rate of Official Builds. Abnormalities will indicate other issues (like inadequacy of pre-checkin tests) to be investigated.
• The current resource usage to answer the question, “Do we have sufficient resources applied to the official build subsystem?”.
11. Report Generated
12. “Wins”
Proof-of-Concept
Meaningful output can be produced with minimal expense and a reasonable amount of effort.
Preliminary Reports Completed
The build team can make decisions based on the first set of reports
Automation
Reports are automatically generated and posted.
Extensible
The groundwork has been laid so that new data sources and new reports can be easily added into the reporting system.