Presentation Title Will Appear Here

Report
Registration Management Committee (RMC)
Use of PEAR’s
Auditing Application of AS9101
Boston, MA
July 21-22, 2011
Henry Gamboa
UL DQS Aerospace Program Manager
Michelle Barton
UL DQS Senior Lead Assessor/Reviewer
Company Confidential
Auditor Workshop
Boston, MA
July 21-22, 2011
1
Registration Management Committee (RMC)
PEAR Definition
• A PEAR is a document stating results and providing
evidence of determination on the effectiveness of a
process.
Company Confidential
Registration Management Committee (RMC)
PEAR Intent/Purpose
• The PEAR was introduced to provide a tool that
allows the auditor to systematically evaluate the
effectiveness of a process based on the organizations
method for monitoring and measuring their
processes.
Company Confidential
Registration Management Committee (RMC)
What is Effectiveness?
• It is the extent to which planned activities are realized
and planned results are achieved
• (Reference ISO 9000 Quality management systems – Fundamentals
and Vocabulary)
Company Confidential
Registration Management Committee (RMC)
What is Effectiveness?
In other words, is the process effective in:
• Fulfilling customer requirements?
• Delivering quality products, on time?
• Improving continually?
• Meeting established goals and objectives?
More than just a conformity assessment!
Company Confidential
Registration Management Committee (RMC)
• What will this Mean to the Auditor?
• Review and examine an organization’s
“Effectiveness” from:
– The customer viewpoint; satisfaction and performance.
– Organizational leadership goals and objectives.
– Alignment of process performance measures with
quality objectives and customer requirements.
– Criteria used for deciding what data to analyze for
determining process effectiveness.
Company Confidential
Registration Management Committee (RMC)
General rules for use/application of a
PEAR:
• At a minimum, the results of each audited product
realization process, as determined by the organization
(reference 9100-series clauses 4.1 and 7.1), shall be
recorded on a PEAR.
• Each assessed process shall be recorded on a
separate PEAR.
• There is no preset minimal number of PEAR’s required
from an audit
Company Confidential
Registration Management Committee (RMC)
To what level of detail does a process have to
be to require a PEAR?
• Consider what defines a process – inputs, outputs, process owners,
etc.
– If they are the same, then a higher level PEAR might be
appropriate.
– If they are not the same, then 2 separate PEARs should
be created.
• In determining the appropriate level reference the Form Instructions
for the the QMS Process Matrix Report (Appendix D) – it shows that
Product Realization includes: Manufacturing-Machining,
Manufacturing-Paint, Manufacturing – Assembly, etc.
Company Confidential
Registration Management Committee (RMC)
When does a PEAR have to be done?
• Reference quality manual & other QMS
documentation for description and sequence of
interaction of processes
• At a minimum, the results of each audited product
realization process, as determined by the organization
(reference 9100-series clauses 4.1 and 7.1), shall be
recorded on a PEAR
• Will discuss more during group activity
Company Confidential
Registration Management Committee (RMC)
New Focus on Audit Planning in AS9101D
• The AS9101D now requires substantial review of
information/data in preparing for audits.
• Included in this data is the identification of processes
including their interactions and sequencing.
• The identification of these processes will then be used in
preparing the audit plan/agenda and to identify the PEAR
processes.
Company Confidential
Registration Management Committee (RMC)
Process Based Audit Tool is an Input into the
PEAR
• The information that is recorded on the process
based tool will then be an input for the completing the
PEAR later.
Company Confidential
Registration Management Committee (RMC)
• Process Based Audit Tool is an Input into the
PEAR (cont.)
Collection of Data (4.3.2.2 of AS9101D) NOTE 1: The processes can be depicted in various ways [e.g., process
maps, turtle diagrams, SIPOC method (breakdown of supplier,
inputs, process steps/tasks, outputs, customer), octopus].
AATT training Whatever audit tool is used, it should include process control
methods for monitoring and measuring process effectiveness.
Company Confidential
Electronic Assembly Turtle
Registration Management Committee (RMC)
Aerospace Process Turtle
Process name:
Electronic Assembly
Critical success factors:
Maintain schedule and zero failures
Process
owner:
Start event:
End event:
Process input
Kitted Assembly with
Traveler
BOM
Pick List
Instructions
MRP Need Date
Supplier for process /
Previous process
Stockroom
Interfaces for input
Production Control
Maintenance
Calibration
Engineering
Planning
Quality
SWOT Analysis
Process map
Company Confidential
Resources / material
(with what?)
Process personnel
(with whom?)
Process output
Material noted in BOM
Test Equipment, wire bonders
Prod Equipment/Testing
Techniciians trained to J-Std and
IPC-610, Mfg Self Inspection,
Secret Clearance, ESD Training
Completed assemblies
Completed records
Process:
Process steps / main activities:
iiiiiiiiiiiiiiiiijjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj
Assemble per drawing –
Tune & align unit – pre-cap
inspection – laser weld lid
– environmental test – final
test – paint & label – final
inspect
Process indicators
(how many?)
Process objectives /
Purpose
RMA levels, OTD, FPY
1% RMA or less, 90% OTD,
98%FPY
Reference documents (how?)
Customer of process /
Next process
Interfaces for output
Next level assembly
Production control
Process Improvement
Notes
Registration Management Committee (RMC)
UL-DQS Inc.
Aerospace Process Effectiveness
Assessment Report
Organization:
PEAR Number:
Site:
OIN:
Audit Report Number:
Issue Date:
Process Name:
Process details, including associated process interfaces:
Process Details/Interfaces
Applicable 9100/9110/9120 clause(s):
Organization’s method for determining process effectiveness:
Auditor observations and comments supporting process effectiveness determination:
Statement of Effectiveness Level:
The process is:
1. Not implemented; planned results are not achieved.
2. Implemented; planned results are not achieved and appropriate actions not taken.
3. Implemented; planned results are not achieved, but appropriate actions being taken.
4. Implemented; planned results are achieved.
Company Confidential
Auditor Name(s):
Auditee Representative Acknowledgement
Name:
Signature(s):
Signature(s):
Registration Management Committee (RMC)
Process Details/Interfaces
• Form Completion Instructions from AS9101D - Summarize
the process activities, inputs, and outputs; including the
identification of associated process interfaces.
Company Confidential
Registration Management Committee (RMC)
Process Details/Interfaces –
Example for Electronics Assembly Process
Process Details/Interfaces –
• Inputs into the process include traveler, BOM, pick list, work instructions,
kitted assembly & MRP need date. Process flow is as follows: “Assemble
per drawing – Tune & align unit – pre-cap inspection – laser weld lid –
environmental test – final test – paint & label – final inspect.” The outputs
include completed assemblies and completed assembly records. Process
interfaces include production control, planning and engineering.
Question: Is there something that could be added to this description of
process details/interfaces?
Answer: The specific details of the process flow sequence alone does not
highlight potential risk factors in meeting customer requirements and
measureable objectives of the process.
Company Confidential
Registration Management Committee (RMC)
Same Example, Different Write up
Process Details/Interfaces –
• Inputs into the process include kitted assembly and build paperwork & MRP
need date. Process flow is as follows: “Assemble per drawing – Tune &
align unit – pre-cap inspection – laser weld lid – environmental test – final
test – paint & label – final inspect.” The nature of the product developed and
manufactured in the organization requires highly specialized and state of
the art microwave/hybrid processes. The product itself can be very complex
and on occasion requires frequent design and development changes. The
outputs includes completed assembly and assembly records. Process
interfaces include production control, planning and engineering.
Question: Why is this process description better than the previous?
Answer: Because it highlights potential risk factors (product complexity,
frequent design changes) in meeting customer requirements and
measurable objectives of the process.
Company Confidential
Registration Management Committee (RMC)
UL-DQS Inc.
Aerospace Process Effectiveness
Assessment Report
Organization:
PEAR Number:
Site:
OIN:
Audit Report Number:
Issue Date:
Process Name:
Process details, including associated process interfaces:
Applicable 9100/9110/9120 clause(s):
Organizations method for determining
process effectiveness
Organization’s method for determining process effectiveness:
Auditor observations and comments supporting process effectiveness determination:
Statement of Effectiveness Level:
The process is:
1. Not implemented; planned results are not achieved.
2. Implemented; planned results are not achieved and appropriate actions not taken.
3. Implemented; planned results are not achieved, but appropriate actions being taken.
4. Implemented; planned results are achieved.
Company Confidential
Auditor Name(s):
Auditee Representative Acknowledgement
Name:
Signature(s):
Signature(s):
Registration Management Committee (RMC)
Organizations method for determining
process effectiveness
• Form Completion Instructions from AS9101D - Describe
the method used by the organization to determine process
effectiveness [e.g., identification of Key Performance
Indicators (KPIs) and associated targets, process
capability data].
Company Confidential
Registration Management Committee (RMC)
Organizations method for determining
process effectiveness –
Example for Electronic Assembly Process
Organizations method for determining process effectiveness • Quality and on time delivery is tracked periodically.
Question: What is wrong with this description?
Answer: Description is too brief and does not tell how quality is tracked
(e.g., 1st pass yield, RMA levels, scrap, etc.). It also does not tell how
often quality and on time delivery is tracked and by whom.
Company Confidential
Registration Management Committee (RMC)
Organizations method for determining process
effectiveness
Same Example, Different Write up
• 1st pass yields and on-time delivery is tracked daily and discussed during
daily production meetings with cross functional teams. This information is
summarized and reported monthly by the quality manager to the general
manager. The general manager will go over this information on a quarterly
basis with all employees during state of the company meetings.
Question – Why is this example better than the previous?
Answer – Because description is more thorough and describes how quality is
tracked (1st pass yields). It also mentions how often quality and on time
delivery is tracked (daily production meetings, monthly reporting to GM and
quarterly state of the company meetings)
Company Confidential
Registration Management Committee (RMC)
UL-DQS Inc.
Aerospace Process Effectiveness
Assessment Report
Organization:
PEAR Number:
Site:
OIN:
Audit Report Number:
Issue Date:
Process Name:
Process details, including associated process interfaces:
Applicable 9100/9110/9120 clause(s):
Organization’s method for determining process effectiveness:
Auditor observations and comments supporting process effectiveness determination:
Auditor observations & comments supporting
process effectiveness determination
Statement of Effectiveness Level:
The process is:
1. Not implemented; planned results are not achieved.
2. Implemented; planned results are not achieved and appropriate actions not taken.
3. Implemented; planned results are not achieved, but appropriate actions being taken.
4. Implemented; planned results are achieved.
Company Confidential
Auditor Name(s):
Auditee Representative Acknowledgement
Name:
Signature(s):
Signature(s):
Registration Management Committee (RMC)
Auditor observations & comments supporting
process effectiveness determination
• Form Completion Instructions from AS9101D - Annotate
relevant objective evidence, observed conditions, data,
information, comments, etc. to support the auditor's
statement of effectiveness or ineffectiveness
Company Confidential
Registration Management Committee (RMC)
Auditor observations & comments supporting
process effectiveness determination
Electronic Assembly Process Example
• Assessed 5 items through the assembly process. Work instructions were
available for the operator at stations in each of cells. Talked with 6 operators
all who knew the quality policy. Records were available for review. All
gauges had calibration stickers on them. Verified the training records for
each of the employees interviewed and all had records of competence for
activity observed performing.
• Question: What is wrong with this description?
• Answer: This description gives the evidence of compliance that was
assessed. It does not address the effectiveness of the process. Based on
this information the reader of the PEAR would not be able to determine what
Level the process is performing to.
Company Confidential
Registration Management Committee (RMC)
• Auditor observations & comments supporting
process effectiveness determination
Same Example, Different Write up
• On time delivery target of 95% has not been met for the past 12 months.
Average for past 3 months has been slightly below 75%. Factory is running
at capacity according to Manufacturing Manager. Actions taken include
hiring 2 new swing shift technicians and 7 new day shift technicians. Also,
training of technicians of highly specialized processes is on-going. In
addition, action is planned to automate some of the existing tuning and
testing processes which should help improve delivery. Also, weekly
material status meetings are having a positive impact to schedule according
to area manager.
• Question: Why is this better than the previous response?
• Answer: This description tells of the performance of the process that was
observed during the audit rather than identifying the evidence of
conformance.
Company Confidential
Registration Management Committee (RMC)
UL-DQS Inc.
Aerospace Process Effectiveness
Assessment Report
Organization:
PEAR Number:
Site:
OIN:
Audit Report Number:
Issue Date:
Process Name:
Process details, including associated process interfaces:
Applicable 9100/9110/9120 clause(s):
Organization’s method for determining process effectiveness:
Auditor observations and comments supporting process effectiveness determination:
Determining the appropriate
level of effectiveness
Statement of Effectiveness Level:
The process is:
1. Not implemented; planned results are not achieved.
2. Implemented; planned results are not achieved and appropriate actions not taken.
3. Implemented; planned results are not achieved, but appropriate actions being taken.
4. Implemented; planned results are achieved.
Company Confidential
Auditor Name(s):
Auditee Representative Acknowledgement
Name:
Signature(s):
Signature(s):
Registration Management Committee (RMC)
New Levels of Effectiveness introduced
in PEAR documents.
1) Process not implemented; planned results are not
achieved?
2) Process implemented; planned results are not
achieved and appropriate actions not taken?
3) Process implemented; planned results are not
achieved, but appropriate actions being taken?
4) Process implemented; planned results are achieved?
Company Confidential
Registration Management Committee (RMC)
Determining the appropriate level of effectiveness
Form Instructions –
Indicate to what extent the audited process was determined effective.
Effectiveness: extent to which planned activities are realized and
planned results achieved (reference ISO 9000).
NOTE 1: A major (ma) NCR shall be issued when the effectiveness
level of the process is rated a "1".
NOTE 2: A major (ma) or minor (mi) NCR shall be issued when the
effectiveness level of the process is rated a "2" (see nonconformity
definitions in clause 3).
Company Confidential
Registration Management Committee (RMC)
Determining the appropriate level of effectiveness
• What is written in the previous 2 sections should
support this determination.
– In essence, the reader of the PEAR should already
know what level the process is performing at before
looking at the checked box.
Company Confidential
Registration Management Committee (RMC)
Example of Determining Effectiveness Level
Electronic Assembly Process
• On time delivery target for electronic assembly process of 95%
has not been met for the past 12 months. Average for past 3
months has been slightly below 75%. According to the Assembly
Manager, the factory is running at capacity but he has not been
able to find new qualified technicians for hire required for some
of its highly specialized processes. Performance data posted in
the assembly area shows that electronic assembly has been
running on over time for the past 9 months and 1st pass yield
targets have not been met. RMA levels during this time frame
have exceeded the goal of 1% or less for product shipped and is
currently at 3%.
Company Confidential
Registration Management Committee (RMC)
What level of effectiveness would you
assign?
1) Process not implemented; planned results are not
achieved?
2) Process implemented; planned results are not
achieved and appropriate actions not taken?
3) Process implemented; planned results are not
achieved, but appropriate actions being taken?
4) Process implemented; planned results are achieved?
Company Confidential
Registration Management Committee (RMC)
Example of Determining Effectiveness Level
Electronic Assembly Process
• On time delivery target of 95% has not been met for
the past 12 months. Average for past 3 months has
been slightly below 75%. Factory is running at
capacity according to Manufacturing Manager.
Actions taken include hiring 2 new swing shift
technicians and 7 new day shift technicians. Also,
training of technicians of highly specialized
processes is on-going. In addition, action is planned
to automate some of the existing tuning and testing
processes which should help improve delivery. Also,
weekly material status meetings are having a positive
impact to schedule according to area manager.
Company Confidential
Registration Management Committee (RMC)
What level of effectiveness would you
assign?
1) Process not implemented; planned results are not
achieved?
2) Process implemented; planned results are not
achieved and appropriate actions not taken?
3) Process implemented; planned results are not
achieved, but appropriate actions being taken?
4) Process implemented; planned results are achieved?
Company Confidential
Registration Management Committee (RMC)
Example of Determining Effectiveness
Level
Electronic Assembly Process
• On time delivery target of 95% has not been met for the past 12
months. Average for past 3 months has been slightly below 55%.
Several key personnel have left the organization since the last
surveillance due to retirement and other reasons. Replacements
have not been trained and there are no plans to have them
trained until the next big contract is awarded. Furthermore, many
of the assembly processes have not been documented and there
seems to be a dependence on tribal knowledge on how to get the
job done. Several production travelers sampled contained
operations that were not signed off and had inspection
operations that were by-passed. Customer complaints and
RMA’s have tripled in the past 9 months.
Company Confidential
Registration Management Committee (RMC)
What level of effectiveness would you
assign?
1) Process not implemented; planned results are not
achieved?
2) Process implemented; planned results are not
achieved and appropriate actions not taken?
3) Process implemented; planned results are not
achieved, but appropriate actions being taken?
4) Process implemented; planned results are achieved?
Company Confidential
Registration Management Committee (RMC)
Group Activity
Company Confidential
Registration Management Committee (RMC)
Credits & Acknowledgements
• IAQG AS9100 Team
Company Confidential
Registration Management Committee (RMC)
Questions?
Company Confidential

similar documents