Report

ISO/TS 16949:2009(E) and AIAG MSA 4th edn. (2010) Martin Gibson CStat, CSci, MSc, MBB AQUIST Consulting gg1000@waitrose.com © M G Gibson 2010 RSS Destructive Testing MSA 1 Making sense of MSA Do you know how accurate and precise your measurement and test equipment are? Do you suspect that good work is sometimes condemned as bad simply because of uncertainty in the measurement system; is bad work ever released as good? Do you know the cost of non-capable measurement systems? Do you realise how important it is to understand measurement systems uncertainty? Does your auditor share your understanding of measurement systems? What can you do about it? © M G Gibson 2014 RSS Destructive Testing MSA 2 ISO/TS 16949:2009(E) 7.6.1 Measurement System Analysis Statistical studies shall be conducted to analyse the variation present in the results of each type of measuring and test equipment system. ... applies to measurement systems in the control plan. ... analytical methods and acceptance criteria used shall conform to those in customer reference manuals on MSA. Other analytical methods and acceptance criteria may be used if approved by the customer. Questions: What is the operational definition of statistical studies? Do organisations, auditors, quality mangers understand statistical studies? Why do auditors ask?, “Can you show me GR&R studies for each type of measuring and test equipment system referenced in the control plan?” © M G Gibson 2014 RSS Destructive Testing MSA 3 ISO/TS 16949 Scheme Update IF SMMT Webinar, 5 Nov. 2013 Common problems found in ISO/TS16949 audits Calibration and MSA (7.6 and 7.6.1) Definition of Laboratory scope Control of external laboratories Traceability to national or international standards MSA not done for all types of measuring systems MSA only considering gauge R and R Questions: 1. Why is MSA regarded as GR&R? © M G Gibson 2014 RSS Destructive Testing MSA 4 Ford Motor Company MSA requirements (2009) 4.35 (ISO/TS 16949 cl. 7.6.1) All gauges used for checking Ford components/parts per the control plan shall have a gauge R&R performed in accordance with the appropriate methods described by the latest AIAG MSA to determine measurement capability. Variable gauge studies should utilize 10 parts, 3 operators & 3 trials Attribute gauge studies should utilize 50 parts, 3 operators & 3 trials Questions: 1. Are some Customers leading the thinking? 2. Why just limited to products? 3. What are your Customer expectations? © M G Gibson 2014 RSS Destructive Testing MSA 5 Measurement System Variation Observed Variation = Process Variation + Measurement System Variation Bias Accuracy Linearity Measurement System Variation Calibration Stability Repeatability Precision Gauge R&R Reproducibility © M G Gibson 2014 RSS Destructive Testing MSA 6 AIAG MSA 4th edn. (2010) Accuracy, Bias, Stability, Linearity, Precision, Repeatability, Reproducibility, GR&R Attributes, Variables, & non-replicable data considered Variables GR&R study 10 parts, 3 operators, 3 measurements Parts chosen from 80% of tolerance Destructive testing requires 90 parts from a homogeneous batch Three analytical methods: Range – basic analysis, no estimates of R&R 2. Average & Range – provides estimates of R&R 3. ANOVA – preferred, estimates of parts, appraisers, parts*operators interaction, replication error due to gauge Question: 1. Do organisations, auditors, quality mangers understand MSA? 1. © M G Gibson 2014 RSS Destructive Testing MSA 7 AIAG ANOVA Models Crossed vs. Nested Y ijk = μ + Operator i + Part j + (Operator*Part) ij + ε k(ij) Y ijk = + Operator i + Part j(i) + (ij)k Crossed vs. Nested? See Barrentine, Moen, Nolan & Provost, Bower, Burdick, Skrivanek Fixed vs. mixed effects models? Software? MTB V16+ includes fixed, mixed effects, enhanced models, pooled standard deviation approach not included. SPC for Excel – fixed effects Other software packages? Question: Do organisations, auditors, quality mangers understand ANOVA? © M G Gibson 2014 RSS Destructive Testing MSA 8 GR&R Variables Data Acceptance Criteria % Contribution Measurement System Variation as a percentage of Total Observed Process Variation using variances (additive) % Study Variation Measurement System Standard Deviation as a percentage of Total observed process standard deviation (not additive) % Tolerance Measurement Error as a percentage of Tolerance Number of Distinct Categories (ndc) Measures the resolution of the scale % Contribution % Study Variation % Tolerance ndc > 9% Unacceptable > 30% Unacceptable > 30% Unacceptable < 5 Unacceptable 2-9% Acceptable 11-30% Acceptable 11-30% Acceptable 5-10 Acceptable < 1% Good < 10% Good < 10% Good > 10 Good Do organisations, auditors, quality mangers understand the metrics? 9 Non-replicable GR&R case study (Anon, 2002) Ensure that all the conditions surrounding the measurement testing atmosphere are: defined, standardized and controlled appraisers should be similarly qualified and trained lighting should be adequate and consistently controlled work instructions should be detailed and operationally defined environmental conditions should be controlled to an adequate degree equipment should be properly maintained and calibrated, failure modes understood, etc. 10 Non-replicable GR&R case study (Anon, 2002) If the overall process appears to be stable & capable, and all the surrounding pre-requisites have been met, it may not make sense to spend the effort to do a non-replicable study since the overall capability includes measurement error – if the total product variation and location is OK, the measurement system may be considered acceptable. Ironically high Cp / Cpk gives poor ndc! Question: 1. Do organisations, auditors, quality mangers understand this concept? AIAG FAQs response: If your process is stable and capable, the spread of this acceptable process distribution includes your measurement error. There may be no need to study your measurement error from a purely "acceptability" viewpoint.’ 11 Questions for Making sense of MSA What is the operational definition of statistical studies? Do organisations, auditors, quality mangers understand statistical studies? Why do auditors ask for GR&R studies? Why is MSA regarded as GR&R? Are (some) Customers leading the thinking? Why is MSA limited to products? Do you know your Customer expectations? Do organisations, auditors, quality mangers understand? MSA, ANOVA, crossed vs. nested, fixed vs. mixed models, metrics, high Cp/Cpk gives low ndc? Is MSA seen just as a QMS requirement or a true part of continuous improvement? © M G Gibson 2014 RSS Destructive Testing MSA 12 References AIAG Measurement System Analysis, 4th edn., (2010) Anon. Non-replicable GR&R case study, (circa 2002) Barrentine, Concepts for R&R Studies, 2nd edn., ASQ, (2003) Bower, A Comment on MSA with Destructive Testing, (2004) ; see also keithbower.com Gorman & Bower, Measurement Systems Analysis and Destructive Testing, ASQ Six Sigma Forum Magazine, (August 2002, Vol. 1, No. 4) Burdick, Borror & Montgomery, A review of methods for measurement systems capability analysis; JQT, 35(4): 342-354, (2003) Burdick, Borror & Montgomery, Design & Analysis of Gauge R&R Studies, SIAM, ASA, (2005) Moen, Nolan & Provost, “using a Nested Design for quantifying a destructive test” in Improving Quality Through Planned Experimentation, McGraw-Hill; 1st edn., (1991) Skrivanek, How to conduct an MSA when the part is destroyed during measurement, moresteam.com/whitepapers/nested-gage-rr.pdf 13