The Pope and the Office of Inspector General are both requested to examine difficult questions. The Holy Father, when speaking ex cathedra on matters of faith and morals, is infallible. The OIG, while intimating that its analyses are without flaw, is known to be fallible. This study was at the urging of Rep. Peter DeFazio (D-OR), ranking minority on the House T&I Aviation Subcommittee. He “requested that we review FAA’s staffing and oversight of the ODA program’s delegated organizations. In particular, Representative DeFazio expressed concerns as to whether FAA has the resources and risk-based tools in place to provide effective oversight of the ODA program. Accordingly, our audit objectives were to assess FAA’s (1) process for determining staffing levels needed to conduct ODA oversight and (2) oversight of delegated organizations’ program controls.”
It is clear that the OIG analysis of the FAA’s staffing model for ODA failed examine all of the relevant facts relevant to this question. Here is what the report concluded:
“…FAA lacks a comprehensive process for determining staffing levels needed to provide ODA oversight. While the Agency uses a staffing model to aid in identifying overall staffing needs, the model does not include detailed ODA data on important workload drivers, such as a company’s size and location, type of work performed, and project complexity. In addition, FAA’s oversight of ODA program controls is not fully systems- and risk-based, as recommended by an aviation rulemaking committee. This is largely because FAA inspectors and engineers lack adequate guidance and risk-based tools and do not conduct robust analyses of ODA data. Furthermore, FAA does not conduct sufficient oversight of ODA personnel conducting certification work at companies that supply components to other manufacturers.”
The OIG Report ignored the existence of the FAA’s SMS and SASO systems, which are designed specifically to identify precise issues for the inspectors to address. The document does devote some attention to the development of the ODA concept and its value to the certification process:
While that history is useful and relevant, its description of the processes involved in oversight of ODAs uses the antiquated terminology associated with surveillance and inspections, such as:
“…the company’s size and location, type of work performed, past performance, and project complexity and volume rather than the staffing model results because the model does not include these factors. FAA managers at two of six offices expressed concerns that there are not enough labor distribution codes to adequately reflect all of their specific workload drivers, such as time spent on other certification activities versus ODA oversight.”
Safety Management System (SMS) and System Approach for Safety Oversight Program (SASO) are risked based models for managing FAA’s safety mandate, using data to flag potential problems and assigning the right resources for each identified task. SMS is the responsibility of the carrier; the certificate holder uses masses of data and a comprehensive internal process to prioritize the issues which it faces. That airline specific information is input into the FAA’s SASO program, where again data and internal review point out what projects of all air carriers deserve the highest and most immediate attention. These recommendations will also carry with them some indications of the inspector skills which should best be used. Under this SMS/SASO scenario, the selection of the right resources for the specified task will be exercised by the Flight Standards organization. The nature of the task will dictate the people and time needed to complete the work which the SMS/SASO merits.
The past ODA practice would periodically send one or more inspectors to surveil the facility. The inspectors would be engaged in a general audit of the organization– reviewing documents, examining the work process of the engineers of that company, assessing the capabilities of the ODA staff, even visiting the application of the ODA approvals to specific aircraft, etc. There is some randomness to these inspections; details, which were not highlighted prior to the investigation, are found in the course of the review. From that inception point, the trail may follow a series of clues. At the beginning of such an assignment, the length of the time required would not be known; so preexisting metrics would be of great value. In contrast, the job task analysis created by SMS/SASO will be considerably more finite and predictable.
Before making judgments about the need for OPM like guidelines, OIG should do a 360° review (an SMS term) of the FAA’s existing systems and incorporate the value of those advances in its castigation of the inadequacy of staffing in responding to Rep. De Fazio’s request. This failure in its oversight reminds the world that the OIG is fallible.