BIM Strategy June 2013

Risk Based Validation
January 2014
© 2013 IBM Corporation
 Do you wish to be able to stay current with the latest releases of Maximo?
 Do you wish to save costs in validation and upgrade?
If you answered yes to either or both of these questions….
stay tuned, we are here to help in achieving these objectives!
© 2013 IBM Corporation
 Risk-Based Validation Program – Overview
 Approach
 Analysis Questions
 Supportive Documentation
 History and 2013 Hi-lights
 Developing a Risk Based Validation Strategy – Best Practices
 Discussion Questions
© 2013 IBM Corporation
Risk-Based Validation
 For many years, the FDA has promoted a shift to a risk-based approach to validation
– Software validation should be focused on “a justified and documented risk
assessment and a determination of the potential of the system to affect product
quality and safety, and record integrity.”
 Practical motivations behind risk-based validation:
– Companies can more rigorously test the areas of the software system that pose
the greatest risk to product quality and patient safety based on intended use
– Overall validation costs are reduced and efficiency is increased
– An industry-wide shift to risk-based validation would lead to faster adoption of
innovation in manufacturing and new technological advances
 While the risk-based approach is taking hold in regulatory industries, it can and
should be used by any and all industries
© 2013 IBM Corporation
Risk Based Validation Program - Overview
 This unique program supports customers’ implementation of a risk-based approach to Maximo
validation for all releases (e.g. new installs, upgrades, fix packs)
– Enables moving to latest fix pack/versions
– Enables reduction of costs and efforts
– Shifts $ to other areas as needed
• Customer validation is “mapped” to IBM validation; eliminate duplications
• IBM provides supportive evidence from detailed test documentation
• Risk-based testing is an effective way to prioritize risks for building validation centered around
high risk areas in your business
• Early risk assessment results in more effective testing
• IBM provides documentation and assistance, including:
• Automated & manual test execution reports
Summary report of test strategy
• Pareto analysis of defects
Traceability matrices
• Mapping analysis for test coverage
 Other benefits :
– Develop best regression testing practices, automation practices
– Directly influence IBM test efforts for Maximo by providing scenarios to include in IBM testing
IBM Confidential
© 2013 IBM Corporation
1. Gather business requirements
2. Categorize requirements
Criticality/Development Method
3. Perform risk based testing approach
• Analyze your requirements/test cases against IBM test cases, proposed
reductions where duplications exist
4. If needed, perform audit IBM quality management system, to support acceptance of
IBM validation and verification data
5. Provide supporting documentation to support elimination of tests
6. Result - develop consistent validation strategy to be used for all release updates
© 2013 IBM Corporation
History and 2013 Hi-lights
Focus Areas:
Maximo and Calibration; Maximo 7.5 upgrade and later versions of 7.1.x
Wide range of business requirements (200 to 2000)
Benefits stem mostly from OOTB areas (critical and/or non critical)
Concentration in Life Sciences/regulatory sectors
Customer references available on request
Reductions in validation costs range from 20-82%
Provided and helped companies develop ongoing validation best practices
2013 Updates:
Doubled number of customers as compared to 2012, reductions hold same
Based on customer input, updated Fix Pack read-me, to allow required impact
Looking to work with business partners for expansion
Interest with Nuclear and Oil and Gas customers
Published QPP content on the ISM Library
© 2013 IBM Corporation
Business requirements mapped to IBM results
© 2013 IBM Corporation
Business Requirement mapped to IBM test case
Example client business requirements mapped to IBM Test cases
© 2013 IBM Corporation
Sample Data – Business Requirement
Business requirement traced to automated test result : System shall allow work order to be
auto generated from a Preventive Maintenance record
IBM Confidential
© 2013 IBM Corporation
Mapping Document
© 2013 IBM Corporation
Enhanced Fix pack Documentation
© 2013 IBM Corporation
Updated Fix pack readme updated to provide list of APARs provided by module vs. by APAR
prior to changes
© 2013 IBM Corporation
Fix pack documentation example
Fix pack test status tracking spreadsheet
© 2013 IBM Corporation
Fix Pack Documentation – List of Class File Changes
List of file changes per fix pack
© 2013 IBM Corporation
Fix pack documentation – Quality plan summary
Quality plan summary information
© 2013 IBM Corporation
Test Results
© 2013 IBM Corporation
Sample Data – Automated BVT Test Results
IBM Confidential
© 2013 IBM Corporation
Sample Data – Test Execution Summary Report
Test Execution summary reports
IBM Confidential
© 2013 IBM Corporation
Sample Data – Look up of specific test case
IBM Confidential
© 2013 IBM Corporation
Developing a Risk Based Validation Strategy – Best Practices
 Determine regression “strategy” to be used for all releases
– Articulate exactly what’s been tested/not tested, giving accountability to testing efforts
 Areas to focus:
– Analyzing the most important/critical functionalities: Priority Indicator
– Analyzing the areas where the probability of failure is high: Defect Indicator
– Analyzing the impact of failure of a particular functionality: Severity Indicator
 Analyze fix packs thoughtfully
– Review areas that are impacted; assess risk
– Often can reduce testing
 Include user “exploratory” testing
– All testing in the world does not replace having your own users test
– Include in test planning, set of users testing the release for x man-days
 Include testing for newly configured or customized areas
 Consider Automation
– Speed up validation and more easily trace results to initial risk assessment
– Best used for areas that are repeated in testing
© 2013 IBM Corporation
Discussion Questions
 What are roadblocks to getting onto the latest release cycle?
 Are there challenges in your validation strategy?
 What percentage of your application is OOTB?
 How can you streamline your regression testing?
– What validation takes up the most time?
– Are there areas that are there duplications? Do groups duplicate testing?
 Are there quality risks that you have had in past?
 Do you really need to test all Maximo areas, for every release? Where are the risk areas?
What can you do about the risk areas?
 Automation?
 Performance?
© 2013 IBM Corporation

similar documents