Quality Gates - National Statistical Service

A tool for managing the
quality of statistical
Quality Gates
Paul Schubert
Head, Statistical Services Branch
November 2011 National Statistical Service
Seminar Series
What are Quality Gates
Attributes of Quality Gates
Six Components of Quality Gates
Outcomes of Quality Gates
Further information
Credibility and Integrity
Statistical Risk
• The chance or likelihood that something
could go wrong in the statistical process
that impacts on the quality or integrity of
the statistical outputs produced.
The statistical cycle
Budgets have decreased
Demand for statistics have increased
Getting it ‘right’
What are quality gates?
"checkpoints or decision points at various
strategic places in a statistical process at
which the quality of the process (and data) at
or up to that point is explicitly assessed"
Attributes of Quality Gates
• Planned in advance of issues occurring
• Knowledge Management (e.g. create a
store of corporate knowledge)
• Constantly reviewed and revised
Attributes of Quality Gates
• Collaborative – provide a model of
accountability and responsibility for
collection processes
• Communicate openly the
progress/issues associated with
collections, throughout the cycle
• Proactively managing risk – can save
time and $ in the long run
Six components of quality gates
Quality measures
1. Placement
Assigning priorities
Related to risk profile of collection cycle
Want to detect errors 'upstream‘
What can go wrong?
When can this problem occur?
What impact can it have?
Placement of quality gates
Areas of risk
changes to processes, systems,
data transformations
overlap / coordination / integration with
other areas
knowledge management
2. Quality measures
Choose good indicators of potential
May need to drill down
Time series useful
How would we know if this problem occurred?
3. Roles
Owner area
Operational person / Gate Keeper
Gate definition
Gate assessment / sign-off
Who is responsible?
Who will this affect?
4. Tolerance
Driven by user requirements
Consider size of natural variation, sampling
errors, level of detail, importance of outputs
Need to form expectations
Historical data may help
What is an acceptable level of quality?
5. Actions
reflect extent of the problem
reflect consequences
"traffic lights" concept
What will we do (if there is a problem)?
Who needs to be informed?
6. Evaluation
evaluation of processes
evaluation of quality gates
What has this information told us about our
How can we improve in the future?
Example of a Quality Gate
• Information paper 1540.0 Quality Gate
Outcomes from Quality Gates
Quality, Business processes and
responsibilities well-defined and known by
Statistical risk explicitly identified &
Explicit sign-off occurs at Quality Gates
Outcomes from Quality Gates
Quality Measures clearly defined and
• Errors identified earlier (and fixed)
• Improved analysis of data throughout the
process in relation to fitness for purpose
of outputs
Outcomes from Quality Gates
• Efficiencies long term
– Errors identified earlier
– Processes evaluated in terms of quality
management (improvements identified)
• Possible time reductions in when ‘things’ are done
• Cost savings due to changes in ‘how’ things are
– Reduction in ‘wasted’ resources because of
not identifying errors until too late
Outcomes from Quality Gates
• Continuous improvement
– Quality and issues identified during process
can be used to help declare the quality of the
statistical outputs
Further information
• Quality Management of Statistical
Processes Using Quality Gates, Dec
2010, (cat.no. 1540.0) on the ABS
• http://www.abs.gov.au/ausstats/abs@.nsf/mf/1540.0
• Quality Page on ABS website – December 22nd, 2011

similar documents