Cart is empty
Retrieve a Cart
Friday, July 30, 2010
Software Tools
•
QuART
•
REPERTOIRE
•
217
Plus
•
RASTER
Knowledge Sources
•
The RIAC
•
Reliability Toolkits
•
Reliability Ques
•
Mil Stds/Hdbks
•
RMQ Reports
•
RMQ Training
FREE Resources
•
QuART Demo
•
Mil Stds/Hdbks
•
MIL-HDBK-217
•
FAA-HDBK-006A
•
Lunchtime Learning
•
Recommended Texts
•
RMQ Links
•
Reliability Self-assessment
•
"QuICKER" Online Training
Quanterion
•
How We Can Help
•
Contract Vehicles
•
Experience
•
Doing Business
•
How To Find Us
•
Contact
811 Court Street
Utica NY 13502-4096
Phone: 315.732.0097
Toll free: 877.808.0097
FAX: 315.732.3261
qinfo@quanterion.com
RASTER
Page:
Introduction
Area 1 - Defining a Reliability Program
Area 2 - Developing Reliability Requirements
Area 3 - Designing for Reliability
Area 4 - Assessing Reliability Progress
Area 5 - Measuring Product Reliability
Area 6 - Ensuring Reliable Performance
Enter Email
Recommendations
Page Score:
Total Score:
Introduction
This tool allows the user to assess the level of maturity of its own organization's, or another organization's, approach to reliability by answering a series of questions about the approach. The answers are used to "score" the maturity of the approach as a means of indicating areas of possible improvement. The absolute value obtained from the scoring is a coarse approximation of the robustness of the organization's approach. An in-person interview is always more effective in any kind of assessment because more detailed follow-up questions can be used. The areas of assessment are:
- Defining Reliability Program
- Developing Reliability Requirements
- Designing for Reliability
- Assessing Reliability Progress
- Measuring Product Reliability
- Ensuring Reliable Performance
Areas, Elements, And Questions
The assessment is divided into areas, elements, and questions. A series of questions makes up each element, and a series of elements make up each area. The questions are the top-level questions that must be answered to complete the assessment. Weights are assigned to each question, element and area behind the scenes in this tool, with the Designing for Reliability considered most important.
Recommendations For Improvement
The last page of this tool indicates a series of recommendations for any of the six assessment areas with ratings below average (less than 3 out of 5).
Determining Customer Satisfaction
1. How do you determine customer satisfaction? Select all that apply.
a.
Customer complaints used
b.
Customer surveys / interviews used
c.
Industry trends analyzed
d.
Competitors are benchmarked
e.
Best practices are benchmarked (business processes, support services, internal customer satisfaction)
2. How is policy on reliability formed, documented, and implemented? Select all that apply.
a.
Reliability activities done as needed
b.
Reliability is part of strategic plans / policies
c.
Formal reliability policies, practices are in place
d.
Follow-up used to ensure that policies are implemented
e.
Management reviews progress and adjusts accordingly
3. Describe the distribution of reliability responsibilities and authority in the organization.
- Select the one that best fits -
a. Reliability activities have no real focal point
b. There is a focal point but without significant influence
c. There is a focal point with reasonable authority
i.
Reliability responsibilities are clearly defined and monitored
ii.
All organizational entities have a plan for continuously improving their reliability efforts
4. What training is provided in reliability?
- Select the one that best fits -
a. Reactive training
b. Training of persons with reliability responsibilities
i.
Training of management in reliability strategies and benefits
ii.
Training of designers in "design for reliability"
iii.
Periodic training updates (and symposia participation) for latest approaches
5. How does the reliability effort interface with other assurance and design activities?
- Select the one that best fits -
a. Little or no interaction
b. Informal sharing of data, analyses, and results
c. Structured sharing of progress, analyses, and results
i.
Routine integrated working arrangement of reliability with designers
ii.
Company-wide network including reliability data / information and ongoing initiatives for continuous improvement
6. How does the Organization know what level of reliability it's delivering?
- Select the one that best fits -
a. Doesn't really know or has no basis for opinion
b. Subjective based on customer feedback
c. Quantitative based on warranty or other field failure data
d. Quantitative based on warranty or field failure data and failure analysis of failed products
e. Quantitative based on data, failure analysis, updated reliability assessment, and trend analysis
Tailoring Program or Product Reliability Efforts
1. How do you determine what reliability tasks are needed for a given program / product development?
- Select the one that best fits -
a. No reliability activities unless mandated by customer
b. Boilerplate reliability program
c. Reliability activities implemented based on undocumented tailoring procedure
d. Documented tailoring procedure for reliability activities based on program / product factors
e. Answer d. including consideration of strategic goals, warranty tradeoffs, life cycle costs, etc.
2. How does management get visibility into the reliability program?
- Select the one that best fits -
a. Crisis reporting only
b. Informal review procedures
c. Formalized review procedures
d. Answer c. plus inclusion in strategic planning considerations
e. Answer d. plus constant improvement efforts
3. Is there a reliability strategy for products developed?
- Select the one that best fits -
a. No strategy
b. Non-documented strategy
c. Documented strategy
d. Documented procedure applying strategy to a specific program/product
e. Documented strategy that is continuously updated to reflect new methods, technologies, etc.
Reliability Figures of Merit
1. How are figures of merit (measures of reliability) determined?
- Select the one that best fits -
a. No Figures of Merit are used
b. Determined informally from customer mandates or previous programs
c. Answer b. except with a standardized process
d. Formal analysis of customer needs used
Consideration of differences between factory and field, mission and logistics figures of merit and / or considering transcendent relationships such as availability and effectiveness.
Quantitative Requirements and Goals
1. How are quantitative requirements / goals determined?
- Select the one that best fits -
a. Solely as required by customers
b. Based on analysis of previous programs to comply with customer needs/requirements
c. Answer b. plus use of environment data
d. Answer c. plus use of trade-off studies (e.g., warranty costs vs. profit)
e. Answer d. plus consideration of impact of use of new technologies
Integration
1. How is reliability integrated into the design effort?
- Select the one that best fits -
a. No established procedures
b. Reliability specialists provide guidance to designers
c. Use of integrated product design team
d. Answer c. plus customized design rules and constraints developed for each case
e. Answer d. plus automated reliability analyses integrated into CAD/CAM
Critical Items
1. How do you identify critical items?
- Select the one that best fits -
a. Critical items not always identified
b. Informal procedure for identification
c. Standardized procedure for identification
d. Answer c. plus management attention to critical items
e. Answer d. plus procedures are continuously improved
2. How do you handle critical items? Select all that apply.
a.
By specifications to suppliers
b.
By periodic review of status
c.
By tailored analysis and tests
d.
Periodic management review
e.
Organization-supplier partnership, back-up plans, or mandatory milestones to be met before release
Use Environment
1. How do you determine the use environment?
- Select the one that best fits -
a. Not concerned about actual use environment
b. Consoles use Environment
i.
Use of past experience of exposure conditions
ii.
Use of standardized tables of environments and stresses
iii.
Use of customer data for expected exposure
iv.
Special measurement of the environmental exposure
Parts Control
1. How do you control parts selection?
- Select the one that best fits -
a. No established method for control of parts selection
b. Use of preferred supplier list
c. Formal parts selection procedure
d. Answer c. plus consideration of parts obsolescence
e. Answer d. plus formal review of selection process periodically
Supplier Control
1. How do you select suppliers?
- Select the one that best fits -
a. By price alone
b. Past experience used informally
c. By formal evaluation
d. Answer c. plus their willingness to share information is a prerequisite
e. Answer d. plus commitment to partnership for improvement is a prerequisite
2. How do you work with suppliers? Select all that apply.
a.
By providing specification
b.
By testing of products
c.
By auditing of processes
d.
Through dialogue on requirements and processes
e.
By joint improvement efforts
3. How do you establish supplier reliability requirements?
- Select the one that best fits -
a. By accepting whatever is offered or available
b. By using past results from supplier
c. By allocating from product requirements
d. Answer c. plus by deriving them from dialogue on needs and capabilities
e. Answer d. plus by deriving them from joint commitment to stretch the state of the art
Software Reliability
Software reliability not measured by the SEI CMM (Software Engineering Institute Capability Maturity Model)
Software reliability measured by the SEI CMM (Software Engineering Institute Capability Maturity Model)
- Select the one that best fits -
Program Risks
1. How do you identify program risks and do design reviews address reliability issues?
- Select the one that best fits -
a. Issues addressed reactively as they arise
b. Previous experience used informally to address risks
c. Formal risk analysis performed
d. Answer c. plus periodic update of analysis
e. Answer d. plus periodic review of analysis criteria
Reliability Tools
1. How are reliability tools (e.g., modeling, derating, FMECA, FTA, etc.) selected for use on a program?
- Select the one that best fits -
a. Used as reactive response to problems or customer specified tools
b. Used as informal repetition of past efforts
c. Use standardized programs
d. Tailoring of standardized program as appropriate
e. Answer d. plus periodic review of tailoring guide/constant study of new tools for use
2. How do you ensure a robust design?
- Select the one that best fits -
a. Leave it to the designer with no formal guidance
b. Consider significant stresses effects on critical parts only
c. Use detailed analyses (e.g., worst case analysis) of impacts
d. Answer c. plus process capability tracking
e. Answer d. plus continuous improvement efforts (e.g., Design of Experiments)
Methodology
1. What data do you use to assess progress? Select all that apply.
a.
Customer complaints
b.
Program manager assessments and FRACAS
c.
Standardized analyses and tests
d.
Tracking “cost of quality” metrics
e.
Continual evaluation of new methods
Timing
1. When do you assess progress in reliability?
- Select the one that best fits -
a. After product is delivered using customer data
b. During production using factory tests of end product
c. During development tests, compiling, and using all test data (parts, assemblies, software)
d. During design using reliability analyses of proposed designs and parts in addition to test data
e. Answer d. plus during development of processes using process analysis (e.g., DOE)
Methodology
1. How do you measure reliability? Select all that apply.
a.
Customer required tests
b.
Use of factory test data
c.
Collection and use of customer data
d.
Tests tailored to product or program (or both)
e.
Continual evaluation of methodology
2. What conditions are used to measure reliability?
- Select the one that best fits -
a. Measurement through analyses
b. Whatever conditions apply to available data
c. Testing at average conditions
d. Testing using expected environmental profiles
e. Answer d. plus duty cycling for expected operation
Scope
1. To what level do you measure reliability?
- Select the one that best fits -
a. End item only
b. All critical items
c. All purchased items
d. Answer c. plus documented tailoring procedure based on cost effectiveness
e. Answer d. plus periodic updates of tailoring procedures
Defects
1. How do you control defects?
- Select the one that best fits -
a. Ad hoc crisis control
b. Ad hoc screening or testing
c. Standardized screening or testing
d. Screening or testing based on root cause analysis
e. Answer d. plus process improvement program
2. How do you handle defective material?
- Select the one that best fits -
a. Scrap or rework only
b. Use of "top 10" problem solving (product problems)
c. Routine root cause analysis (product problems)
d. Routine root cause analysis of product and process
e. Answer d. plus constant improvement of manufacturing processes
3. How do you address variability?
- Select the one that best fits -
a. Crisis control only
b. Use of manufacturing engineers' experience
c. Use of statistical methods to monitor and control process performance (e.g., use of SPC, capability indexes, etc.)
d. Answer c. plus manufacturing presence on design team
e. Answer d. plus continuous process improvement (e.g., use of DOE)
Support Services
1. How do you consider handling, packaging, and shipping?
- Select the one that best fits -
a. Ad hoc effort at time to ship
b. Consideration of item to be shipped and mode of shipment
c. Shipment and transportation considered in design process
d. Answer c. plus consideration given to storage conditions
e. Answer d. plus continuous improvements ongoing
2. How do you address business processes?
- Select the one that best fits -
a. Crisis control only
b. Use a defined process
i.
Reaction to customer complaints
ii.
Standardization
iii.
Statistical analysis / benchmarking
iv.
Continual improvement efforts
Results
1. How do you ensure that you provide reliability services? Select all that apply.
- Select the one that best fits -
a. No effort to measure results beyond program / product requirements
b. Use various methods
i.
Compilation of program / product reliability results
ii.
Use of field data from customers
iii.
Comparison against competitors
iv.
Comparison against industry leaders
2. What trends do you see and what do you mean?
- Select the one that best fits -
a. No trend data compiled
b. Data shows wide variation in results
c. Data shows consistency but no trend
d. Performance that shows consistent improvement
e. Data showing industry leadership
3. What actions do you take as a result of detecting trends?
- Select the one that best fits -
a. None
b. Actions taken to address negative trends but without adequate problem investigation ("shotgun" approach)
c. Actions taken to address negative trends based on analysis to determine underlying root causes
d. Actions taken address negative trends / learn from positive trends
e. Actions taken as in d and to continuously improve design processes and tools
4. How do you assure that you provide reliable services?
- Select the one that best fits -
a. Reaction to customer complaints
b. Tracking service reliability with appropriate metrics
c. Tracking vs. established goals
d. Benchmarking competitors and best in class
e. Constant improvement efforts
You've already entered your email address. Thank you!
View as Printable PDF
Interpreting the Resulting Score
The scores for the six areas of assessment are indicated below out of a five point system (five is best). It should be stressed that the process of developing reliable products cannot be so structured that a single numerical metric can represent its overall effectiveness. Nevertheless, the rating system presented can be valuable as a gross indicator, capable of identifying areas of weakness.
Areas and Elements Requiring Remedial Action
For areas and elements that are given ratings below average, remedial action is strongly recommended. The initial steps for remedial action should be documented in an improvement plan that represents the strategy and commitment to improve. Clues regarding the areas of improvement should be evident from the questions posed that could not be answered affirmatively, and for processes/ procedures that are implemented on an ad hoc basis, or not at all. Other common problems include lack of communication and integration regarding reliability activities. Improvement might include additional training; real-time access by engineers to the organization's data collection, analysis, and corrective action system; refinements in testing; and more thorough root case analyses. For each step, a detailed schedule showing the necessary activities should be developed, responsibility for each activity assigned, and budgets allocated.
Area 1: Defining a Reliability Program
Score:
Area 2: Developing Reliability Requirements
Score:
Area 3: Designing for Reliability
Score:
Area 4: Assessing Reliability Progress
Score:
Area 5: Measuring Product Reliability
Score:
Area 6: Ensuring Reliable Performance
Score:
