FAA’s AC poses new statistical test for ONLY two types of PMAs; WHAT’S up???

JDA Aviation Technology Solutions

As Mr. Broderick ably reports in his below article, the FAA has issued for comment a draft AC 33-13 entitled “Sample Size Considerations for Comparative Test and Analysis for Turbine Aircraft Engine and Auxiliary Power Unit (APU), Parts Manufacturer Approval (PMA), and Third-Party Repair Parts.”

The AC states that this policy will be applied specifically to Turbine Engines and APUs (ONLY? All ACs are “one way” of achieving the desired result). Paragraphs 4 to 5 address technical statistical considerations in generic terms and do not hint why only two classes of PMA applications are at issue in this AC.

For decades the FAA has recognized that skilled engineers can replicate complex parts to be sold as Parts Manufacturer Approval (PMA). The statement of the test is found at 14 CFR § 21.303(a)(4) and it is SURPRISINGLY SIMPLISTIC for defining proof that the parts are the same:

Test reports and computations necessary to show that the design of the article meets the airworthiness requirements of this subchapter. The test reports and computations must be applicable to the product on which the article is to be installed, unless the applicant shows that the design of the ARTICLE IS IDENTICAL to the design of an article that is covered under a type certificate. If the design of the article was obtained by a licensing agreement, the applicant must provide evidence of that agreement.

Not surprisingly, the regulation has been supplemented by Order 8110.42D, Change 2 (44 pages) and a long compendium of policies and relevant regulations.

Safety gains come from innovation and to incentivize research/development of new technologies some protection is merited. Intellectual property is normally the source of some exclusive rights for such inventions.

To add to the complexity of supporting safety enhancements, the OEMs do not (cannot?) include the full cost of the expensive engineering in the sales price of any one part/system/instrument/etc. The expectation is that that substantial investment of capital can be recovered over the life of the product.

Reverse engineering can define the precise measurements of a part which might be submitted for a PMA. Laser measurements now increase the accuracy of the “identical” design.

According to ARSA and MARPA (see below) FAA Advisory Circular (AC) 33-13 appears to add to the burden of the 14 CFR § 21.303(a)(4) applicants and they aver that the new statistical method of proving “identicality” is unjustified:

ARSA:

“The guidance’s implication that such analysis is necessary for all applications would impose an unnecessary burden on the industry. Statistical comparison is inappropriate where features can be directly measured, such as direct dimensional analysis – it may be a tool available to support equivalency during an application but need not be imposed as one.”

MARPA:

“This draft seems to once again imply that every ‘comparative test and analysis’ application needs to use statistical analysis and thus every such application needs to meet a minimum sample size.”

This unusually obtuse document creates more beautiful policy issues than it eliminates, LIKE:

• Is this AC meant to be an initial foray into statistical justification (not requirement; it’s ADVISORY) AND THEN using this statistical approach to other products under PMA?
• Why start with Engines and APUs first? Turbines are highly reliable, but they have had 65 failures between 2020 and today. Uncontained failures and flames exhaust from an engine have commanded major media attention; why add to the frenzy?

• MOST PROVOCATIVELY AND WITHOUT ANY FOUNDATION, MIGHT THIS “OPTIONAL” METHODOLOGY PROVIDE ADDITIONAL PROTECTION FOR THE OEM’S INTELLECTUAL PROPERTY RIGHTS?
• 

As Mr. Broderick mentions, ACs are not supposed to be THE METHOD of proof, but they have been known to become the unarticulated measure by field staff. PMA organizations other than Engine and APU manufacturers may not feel the need to comment on this AC; however, criticism here may deter further expansion of the concept to other parts!!!

The essence of this draft is complex statistical issues. An appendix at the end of this post attempts to illuminate what’s at stake.

 

Draft AC Imposes Statistical Burden on PMA Applications

August 07, 2025

On Aug. 6, ARSA learned the FAA has circulated for public comment its Draft Advisory Circular (AC) 33-13, “Sample Size Considerations for Comparative Test and Analysis for Turbine Aircraft Engine and APU, PMA, and Third-Party Repair Parts.”

The AC attempts to provide “practical approaches to define sample sizes” necessary for statistical analysis demonstrating equivalency by a PMA applicant to an approved article. The guidance’s implication that such analysis is necessary for all applications would impose an unnecessary burden on the industry. Statistical comparison is inappropriate where features can be directly measured, such as direct dimensional analysis – it may be a tool available to support equivalency during an application but need not be imposed as one.

In an example calculation utilizing one of the AC’s methods, a PMA applicant would need to obtain 44 approved parts to compare against the same number of the applicant’s replacement article. The sample of test articles is to be drawn from “more than one lot, melt, or other relevant production [factor]” to establish variation. The quantity and scope requirements implied by the analytical guidance stretch beyond necessity, especially for directly measurable features such as size or hardness.

Comments on the draft AC are due on Sept. 12. ARSA is working with its allies to analyze and respond, encouraging interested members to contact the association with feedback on the draft.

To read the complete draft, click here.

 

 

FAA Proposes PMA Applicant ‘Sample Size’ Methods

Sean Broderick August 19, 2025

The FAA is seeking public COMMENTS on proposed guidance that would define sample sizes for parts manufacturer approval (PMA) applicants seeking certification for engine and auxiliary power unit (APU) parts.

“The FAA has observed that PMA applicants use a variety of statistical principles and methods in their comparative test and analyses” to demonstrate a proposed part is “AT LEAST” equivalent to a type certificate holder’s part, the agency said in draft advisory circular (AC) 33-13. “However, the data often provided is based on statistical methods that are designed to detect statistical differences, not to show equivalency.”

 

The draft AC proposes TWO METHODS: a “practical” approach and a “statistical” approach. Early industry feedback suggests that while sample sizes are an acceptable way to demonstrate a part’s equivalency, they have their limits.

“This draft seems to once again imply that every ‘comparative test and analysis’ application needs to use statistical analysis and thus every such application needs to meet a minimum sample size,” Modification and Replacement Parts Association (MARPA) President Jason Dickstein wrote in a blog post discussing the AC.

This is the same approach the FAA took in a previous draft AC on the issue that prompted significant industry pushback, Dickstein noted. The latest draft appears headed in the same direction.

“Statistical comparison is inappropriate where features can be directly measured, such as direct dimensional analysis—it may be a tool available to support equivalency during an application but need not be imposed as one,” the Aeronautical Repair Station Association noted. “The quantity and scope requirements implied by the analytical guidance stretch beyond necessity, especially for directly measurable features such as size or hardness.”

While ACs are by definition not mandatory, they often become de facto means of compliance.

The draft AC is available at https://www.faa.gov/aircraft/draft_docs/ac. The agency is accepting comments on the AC through Sept. 12.

 

Appendix Copilot’s AI explanation of the Statistics

✈️ Purpose

• • • Establishes two practical approaches to define minimum sample sizes for comparative testing and analysis.
    • Supports compliance with 14 CFR § 21.303(a)(4), Part 33, and Technical Standard Order (TSO) C77.
    • Aims to ensure repaired or replacement parts maintain airworthiness standards equivalent to original parts.

Key Concepts

• • • Comparative Test and Analysis: Used to demonstrate equivalency between PMA or repaired parts and approved articles.
    • Sample Size Determination: Based on statistical principles to ensure sufficient variation and reliability.
    • Multiple Lots Required: Samples must be drawn from more than one production lot, melt, or other relevant factor to capture variability.

️ Applicability

• Directed at applicants seeking FAA approval for:
  • • PMA
    • TC/STC
    • Major repairs or alterations under Part 43
• Also relevant for FAA engine certification engineers and designees.

Regulatory Context

• • Reinforces that maintenance must restore parts to a condition “equal to original or properly altered” with respect to:
    • Aerodynamic function
    • Structural strength
    • Vibration resistance
    • Deterioration
    • Overall airworthiness

⚠️ Industry Feedback

• • Some industry groups, like ARSA, have raised concerns that the statistical burden may be excessive for features that can be directly measured (e.g., dimensions or hardness).
  • Example: One method in the AC suggests comparing 44 approved parts against 44 replacement parts to establish equivalency.

Approach 1: Statistical Sampling Based on Population Variability

This method is rooted in classical statistics and is used when the variability of the population is known or can be estimated.

Key Features:

• • Objective: Demonstrate equivalency between the PMA or repaired part and the approved part with statistical confidence.
  • Sample Size Formula: Based on standard deviation, confidence level (typically 95%), and acceptable margin of error.
  • Typical Sample Size: Often requires 30–50 samples per group (approved vs. replacement) to achieve meaningful statistical power.
  • Multiple Lots Required: Samples must come from at least two different production lots to account for manufacturing variability.

Use Case:

Ideal when comparing material properties, dimensional tolerances, or performance metrics that exhibit measurable variation.

Approach 2: Engineering Judgment with Justification

This method allows for smaller sample sizes when the feature being compared is non-variable, directly measurable, or well-understood.

Key Features:

• • Objective: Use engineering rationale to justify a reduced sample size.
  • Documentation Required: Must include detailed explanation of why fewer samples are sufficient.
  • Examples:
    • Hardness tests on homogeneous materials
    • Dimensional checks on machined parts with tight tolerances
  • FAA Acceptance: Depends on the strength of the justification and supporting data.

Use Case:

Useful for non-critical features, low-variability characteristics, or when prior data supports equivalency.

⚖️ Comparison Table

Feature	Approach 1: Statistical Sampling	Approach 2: Engineering Judgment
Sample Size	Larger (30–50+)	Smaller (as justified)
Basis	Statistical confidence	Technical rationale
Variability Consideration	High	Low or negligible
Regulatory Scrutiny	Standardized	Case-by-case
Common Use	Performance, fatigue, materials	Dimensions, hardness, coatings