Home | Login | Schedule | Pilot Store | 7-Day IFR | IFR Adventure | Trip Reports | Blog | Weather | Articles | Links | Helicopter | Download | Bio

Abstract

This study conducts wings level stall tests on a Piper PA-38 aircraft to determine nose down pitch moment at stall. Recent government test results conflict with previous findings and are a source of controversy. A 20-year history of PA-38 stall/spin problems is recounted and includes review of a recent accident report fueling the controversy. Testing methodology mirrored Federal Aviation Administration recommended procedures. Findings conclude that the aircraft does not exhibit a nose down pitch moment at stall. Additionally, aircraft stall/spin accident rate data is analyzed to determine whether the PA-38 Tomahawk has a statistically higher stall/spin accident rate vs. similar trainers. Accident rate analysis is performed via the chi-square statistic. Statistical analysis concludes that PA-38 accident occurrences are greater than expected at the .005 level of significance. Aggregate findings imply a need for further stall/spin testing.

Piper PA-38-112 Wings Level Stall Testing and Stall/Spin Accident Rate Analysis

Introduction

The aerodynamic stall/spin characteristics of the Piper PA-38 Tomahawk aircraft have been a general aviation subject of interest for years. Shortly after Piper’s introduction of the aircraft in 1977, the National Transportation Safety Board (NTSB) and the Federal Aviation Administration (FAA) expressed concern with the aircraft’s stall characteristics. In 1981 a stall/spin accident investigation in Gadsden, Alabama, resulted in a NTSB recommendation that culminated in the issuance of a mandatory Airworthiness Directive (AD). This directive required the installation of additional stall strips on the leading edges of the PA-38’s wings. Recent NTSB findings involving another PA-38 stall/spin accident have rekindled concerns that the stall/spin characteristics of the aircraft are responsible for the aircraft’s high stall/spin accident rate. Findings of this recent investigation cast doubt on the aircraft’s original flight certification tests. As a result the NTSB has recommended FAA re-certification of PA-38 stall/spin characteristics under Federal Aviation Regulation (FAR) Part 23.

To date the FAA has re-tested wings level stall characteristics and determined that the test aircraft did exhibit a nose down pitch moment at stall. The test involved one aircraft, N9246T. These FAA test findings are incongruent with findings from Swedish tests that concluded the PA-38 exhibited no pitching moment at stall. The adequacy of a single test aircraft has been questioned. Consequently, the main purpose of this study was to replicate Swiss or FAA findings.

The NTSB and FAA are at odds over whether or not the spin characteristics of the aircraft should be reinvestigated. The FAA stated that a review of records indicated that the aircraft has already been adequately spin tested. Additionally, the FAA has officially stated that there is no need for further spin testing. The FAA’s stance is that the matter is considered closed. The NTSB has classified this FAA response to the spin testing recommendation to be an "Open, Unacceptable Response" (Federal Aviation Administration [FAA], 1998, P.2).

Background. The now defunct Piper Aircraft Company introduced the Tomahawk in 1977 and produced a total of 2,484 aircraft. The PA-38’s stall/spin characteristics became notorious almost immediately after introduction to the market. In one of the earliest articles on the aircraft, Berl Brechner reported what was to become a common and recurring theme:

• A new pilot of a Tomahawk, and particularly a flight instructor teaching in one, should be aware of the craft’s unique flight characteristics…. In light, the Tomahawk’s stall characteristics bear utterly no resemblance to those of the Cherokee. The Tomahawk will really stall. It will even give you a fast-breaking secondary stall if recovery from the first one is timid or tenuous….A break to the left was characteristic of the craft used for this check flight. However, during one lax recovery, a secondary stall resulted in a break to the right (as cited in Landberg, 1996, p. 4-2).

Concern over the aircraft’s stall characteristics resulted in official government action in 1981. That year the Flight Test Section Chief of the FAA’s Eastern Region drafted a letter to the FAA’s Engineering and Manufacturing Branch calling for a certification review of PA-38 stall/spin characteristics. The chief’s impetus for drafting the letter included an input from Australia’s Department of Transport stating that the PA-38 did not comply with stalling requirements of FAR Part 23 unless installation of two additional stall strips was mandated by Airworthiness Directive (Lowery, 1998, p.14). The manufacturer was already busy addressing this issue and in 1982 the FAA issued AD 83-14-08 mandating that Tomahawks be retrofitted with two additional stall strips (Stowell, 1997, p.3).

Aerodynamic effect of the mandated stall strip configuration was reported by one writer several years after the AD came out as follows:

The Inman, Kansas, Accident Investigation. The proverbial straw that broke the camel’s back was a 1994 Inman, Kansas, stall/spin accident involving two fatalities. The NTSB’s investigation, spearheaded by Frank Gattolin, was exceptionally probing. The 300 plus page document was released in 1997 along with several recommendations to the FAA. The factual portion of the report indicates the aircraft entered an inadvertent spin and impacted the ground. The pilot in command was a high time flight instructor administering a biennial flight review. Aircraft mechanical deficiencies and aircrew errors were not found to be contributing factors in the accident. The report involved a lengthy inquiry concerning whether or not production aircraft structure varied from that of the original certification aircraft. This concern arose from interviews with engineers who were involved in PA-38 development.

The accident report contains over 250 pages of transcripts from interviews with design engineers and test pilots involved in the development of the PA-38. This extensive interview research appeared aimed at uncovering whether or not production aircraft conformed to type certified drawings. The report questioned whether production aircraft performance in stall and spin flight regimes mirrored the performance documented in original certification tests. Some interview responses indicated that certification tests may have been performed in prototype aircraft structurally different from production aircraft. At least one engineer noted that certification tests occurred at one facility while production occurred at another facility hundred of miles away.

Interviews with the original Tomahawk designer and other engineers cast doubt on the validity of original flight tests. The investigation report quoted a production engineer as follows, "The production Tomahawk’s I eventually became airborne in, only as part of my job, were, to a plane totally unpredictable, one never knew which direction they would roll-off, or to what degree as a result of a stall" (National Transportation Safety Board [NTSB], 1997, p. 1f). The FAA has recently assured the NTSB and the flying public that there is no structural difference between production aircraft and the original certification aircraft (FAA, 1998, p.2). During the investigative interviews a PA-38 design engineer stated, "One of the problems we had in the very beginning was stall characteristics" (NTSB, 1997, Item 17, p.15). This engineer reported that a wing root glove configuration significantly tamed stall characteristics but was not utilized in the final production models (NTSB, 1997, Item 17, p. 16).

Another record of interview with a former design engineer on the PA-38 project noted that wing structure rigidity is critical to stall performance of an aircraft equipped with a wing design like the PA-38. An aerodynamics engineer familiar with the Tomahawk’s wing stated that the wing would be very sensitive to oil-canning, a condition at stall where due to insufficient support structure, the aircraft skin distorts in response to oscillating pressures resulting in dynamic variances in lift (NTSB, 1997, Item 24, p. 2). Still another aeronautical engineer familiar with the PA-38 wing stated that a flexible Tomahawk wing could create stability problems during the stall phase of flight. The accident report included a letter from one engineer that reported that wing stiffness was a problem on the Tomahawk. This engineer stated:

NTSB Recommendations and Status. As mentioned, recommendations of the NTSB resulting from the Inman accident investigation included re-certification of the PA-38 aircraft’s stall/spin performance. Disagreement and conflict between the NTSB and the FAA continues today. NTSB Inman accident investigation recommendations and their current status are listed in Appendix A. So far, the NTSB has refused to withdraw its recommendations, partly because the accident report contains considerable data from the aircraft’s original designer, Piper’s key developmental engineer, a Lock Haven flight test engineer and Piper’s former chief test pilot, all of whom go on record against the aircraft’s flight characteristics (Lowery, 1998, p.14).

Stall/Spin Accident Rate. Statistics on PA-38 accidents reveal a higher stall/spin accident rate than that of other general aviation trainers. The first page of the NTSB Inman report notes, "Although the PA-38 has four non-adjustable stall strips, intended to improve its lateral-directional characteristics, it’s stall/spin accident rate is significantly higher than comparable trainer type airplanes" (NTSB, 1997, p. 1). AOPA’s safety review of the PA-38 notes a stall spin accident rate for the PA-38 that is more than twice that of comparable trainers. The review states, "The Piper PA-38 is involved in roughly double the number of stall/spin accidents per 100 aircraft as the Cessna 150/152 or the Beech 77 (Landsberg, 1996, p. 1-7). A total of 1388 active PA-38’s, 16373 active Cessna 150/152’s, and 225 active Beech 77’s were involved in the AOPA analysis. This longitudinal sample spanned ten years of flight operations. Aviation Safety magazine reported that the "NTSB has found that during a recent 10 year period, the Tomahawk had a fatal stall/spin accident rate almost five times higher than that of the Cessna 150/152" (Lowery, 1997a, p. 9). Others have concluded that the PA-38 stall/spin accident record is more than 20 times greater than comparable trainers (Lowery, 1997b, p. 12).

The NTSB uses a particular method in deriving figures for statistical analysis. In a letter to acting FAA Administrator Barry Valentine dated July 10,1997, NTSB Chairman Jim Hall explains the statistical calculations underlying a conclusion that the PA-38 exhibits a alarmingly large stall/spin accident rate. The letter states:

Airworthiness is defined as the capability of an aircraft to perform in a satisfactory manner in operations for which it was designed in terms of flight crew workload, flight handling characteristics, performance within the design envelope, safety margins and welfare of occupants (Stinton, 1993, p. 4). A goal of flight test is to uncover any adverse handling characteristics prior to allowing the public to operate the aircraft at will. Because critical takeoff and landing phases of flight are performed at relatively slow speed, slow flight characteristics receive considerable attention during flight test.

Testing Stall Characteristics. Slow flight is performed up to and beyond an aircraft’s maximum angle of attack in an effort to define stall characteristics. "The purpose of stall-spin testing is to ensure that your airplane has adequate stall warning and that it can be recovered from a stall or spin" (Askue, 1992, p.126). Additional objectives of stall tests are to ensure that the average pilot will be able to achieve safe recovery, that adequate stall warning exists, and to identify speeds at which the stall occurs. (Stinton, 1993, p. 281). Defining the aircraft’s aerodynamic behavior at stall in various configurations and at various power settings is routine in any flight testing procedure. Uncovering peculiar or dangerous tendencies at stall is a particular interest and concern of stall testing.

One experienced test pilot and engineer asserted that in principle the stall must be detectable. The aircraft must pitch nose-down when it occurs; and the pilot must be able to use the ailerons and rudder effectively. The safest stalls are those in which loss of lift is equal on either side of the plane of symmetry of an airplane, accompanied by a lady-like pitching down of the nose, with little or no tendency to drop a wing (Stinton, 1993, p. 281-282). Stinton’s rules for good stall quality include the following:

Swiss Findings. Following a fatal stall/spin accident in a PA-38 in Sweden, the National Aeronautics Board Investigation Commission of Sweden decided to conduct its own PA-38 flight test program. Swedish tests addressed the stall/spin characteristics of the aircraft. Swiss testing found that the aircraft did not exhibit conventional nose down pitch at stall. "Instead, stalls were characterized by a roll disturbance without pitch change. After more than 60 stalls Swedish authorities found that PA-38 stall characteristics ‘did not meet 14CFR23 certification requirements for wings level stall characteristics" (Lowery, 1998, p12). As previously mentioned, these findings differ from recent FAA findings.

Confused Pilots. While the NTSB and FAA attempt to address their differences on this PA-38 public safety issue, hundreds of flight instructors, pilots and student pilots continue to perform stall and spin training in the aircraft. On one hand, pilots hear the FAA standing behind the original aircraft certification tests. On the other hand, they hear of contrasting stall test findings, conflicting testimony from their fellow pilots, and the unwillingness of the NTSB to retract its recommendations to re-certify the aircraft.

Independent analysis of PA-38 stall/spin characteristics would be of use to the flying public as well as the NTSB and the FAA. Incongruent findings regarding pitch moment during wings level stall, as well as reports of wide variation in individual aircraft stall characteristics, merit further investigation. Additionally, further statistical analysis of stall/spin accident data would be useful in determining if PA-38 pilots are exposing themselves to significantly greater risk than pilots flying other type trainers.

Purpose. The purpose of experiment 1 is to determine wings level stall characteristics of one PA-38 aircraft, serial number 38-80A0107, U.S. registry N25287. Stall testing is to be conducted in accordance with FAR Part 23.201 and AC 23-8A Change 1. Whether or not the pitch down at stall is required for the PA-38-112 to meet certification requirements is not the purpose of this study. The purpose is only to establish whether or not pitch down occurs when performing wings level stalls in the aircraft. Lack of downward pitching moment at stall is the anticipated finding based on previous personal experience stalling the PA-38.

The purpose of experiment 2 is to determine if the PA-38’s stall/spin accident rate is statistically higher than comparable aircraft by examining data available from AOPA’s Safety Foundation. It is anticipated that a chi-square proportional analysis will uncover a statistically significant high stall/spin accident rate for the PA-38 compared to other training aircraft.

The study may provide stimulus for further private or public inquiry. Findings will likely affirm either Swedish or FAA findings regarding pitch moment at stall. Additionally, results may verify previous reports that the PA-38 does not exhibit consistent stall characteristics across individual aircraft. Findings in contrast to the FAA’s recent tests would support NTSB concerns that production aircraft stall characteristics vary widely from aircraft to aircraft. Chi-square statistical verification of a high stall/spin rate can serve to validate or negate concerns over the training aircraft’s safety.
 
 

Experiment 1

Method

Overview Expert opinion was the method utilized to evaluate PA-38-112 pitch moment during wings level stall. The method involved evaluation of aircraft stall characteristics by the author, a Certificated Flight Instructor (CFI) experienced in the PA-38 aircraft. Wings level stall testing was conducted and evaluated in accordance with the recommended procedure of AC 23-8A Change 1. Appendix B contains specifics of AC 23-8A Change 1 pertinent to these tests. Pitch moment at stall was observed and recorded for a total of 60 stalls conducted in 10 varying flap and power configurations.

Participants The Author, a FAA CFI, conducted the stall evaluations. The author is an active flight instructor in a FAA part 141 flight school and has accumulated a total of approximately 3200 flight hours and 100 flight hours instructing in the PA-38 Tomahawk.

Apparatus and Materials Piper aircraft, type PA-38-112, Serial Number 38-80A0107, U.S. registration Number N25287, was used for the stall tests. An annual inspection was conducted approximately one month prior to stall testing. Aircraft weight and balance information and the Pilots Operating Handbook (POH) were critical documents referenced in order to manipulate aircraft center of gravity (CG) close to the full aft limit during stall testing. Four 25-pound bags of lead shot were used as ballast to bring aircraft center of gravity near aft limits and aircraft gross weight to maximum allowed by the POH.

A data-recording sheet was developed and formatted for ease of recording in the flight environment. The sheet was designed so that recording flap and power configuration, as well as stall type, would require only a simple circling of the appropriate term. Similarly, the data sheet allowed for circling of one of three possible aircraft pitch responses to stall; pitch up, pitch down, or no pitch. A separate sheet was used for recording the pitching moment of each series of ten similar stalls (similar in configuration and power setting). Six data recording cards were used to record a total of 60 pitch moment at stall evaluations. The card format is depicted in Appendix C.
 
 

Procedure

Stall characteristic familiarization. Despite recent experience flying the PA-38, greater experience stalling the aircraft prior to testing wings level stall was considered appropriate. Development of stall experience at more docile aircraft weight and balance configurations is recommended in AC-23-8A and is prudent prior to performing stall tests at critical CG configurations.

A series of 180 stalls, conducted over two flights, was performed to obtain greater familiarization with aircraft stall characteristics prior to aft CG testing. These stalls included wings level and turning flight stalls, both power on and off, in nine varying flap and power configurations. Aircraft weight was near maximum during these test stalls, and CG was in the mid range of the weight and balance envelope. These familiarization stalls were not conducted strictly in accordance with AC-23-8A Change 1, but were representative of stalls routinely performed in an instructional environment. Specifically, the AC 23-201 recommendation to decelerate at one knot per second or less and to recover initially without adding power were not strictly observed during familiarization stall training. Aircraft pitch and roll response during these familiarization stalls was recorded. The results, while not considered part of the wings level stall results, are available for review in Appendix D.

In addition to practicing actual stalls, thorough study of the POH, with particular attention to recommended stall and spin recovery techniques, was undertaken. Experience recovering from spins in the PA-38 was considered a necessary prerequisite to the testing.

Preflight preparation. A review of terrain elevation, spin recovery techniques, and the recommended FAA procedures for testing wings level stall performance was conducted in addition to routine preflight procedures. Aspects of weather conditions that were of particular interest were visibility and winds aloft. Exceptional visibility was desired in order to ensure that a distinct horizon was available for noticing even the slightest nose down pitch at stall. Light winds were desired to minimize any effect on stall characteristics.

Weight and balance. The flight was conducted with full fuel and 120 pounds of ballast to bring the aircraft to a maximum allowable gross weight of 1670. 100 pounds of ballast was used to manipulate the center of gravity close to the aft limit and 20 pounds of ballast was permanently left in the passenger seat secured with the safety harness. Operating CG was computed to be 78.1 inches aft of datum, within .4 inches of the maximum allowable figure of 78.5 inches. A flight duration of 1.6 hours resulted in a 60 pound fuel burn and subsequent reduction in gross weight with a negligible shift in CG. A safety note for anyone wishing to replicate the study is that the lead ballast was placed in the right seat for both takeoff and landing and was not placed aft until the aircraft was at a safe altitude and flying speed. Additionally, provisions were devised to rapidly shift the ballast forward in the event of difficulty recovering from a stall. Four lanyards, tied together and readily accessible to the pilot, were secured to each of the ballast bags. Shifting CG forward would require an aggressive pull on one or all of the lanyards. Prior to takeoff a point aft of the reference datum was marked with masking tape for ease of ballast placement in flight.

Conduct of test flight. The test flight date was chosen due to near ideal weather conditions. The flight was conducted on October 12, 1998 from Offutt AFB Nebraska at 4 P.M. local time. The flight proceeded 15 miles south southwest of Offutt AFB in the vicinity of the Plattsmouth, Nebraska. A series of wings level stalls was performed and aircraft pitch moment at stall was evaluated. Aircraft pitch reaction to stall was recorded as either up, down or none. No magnitude of pitch or pitch rate data was recorded. A total of 60 wings level stalls were performed in six varying aircraft configurations as noted in table 1.
 
 

Table 1. Stall Testing Configurations and Trial Numbers

Configuration Number of events/observations

Power off/ no flap 10

Power off/ partial flap 10

Power off/ full flap 10

Power on/ no flap 10

Power on/ partial flap 10

Power on/ full flap 10
 
 

The stalls were conducted in accordance with guidelines of AC-23-8A Change 1. As previously noted, details for conducting stalls in accordance with these guidelines are outlined in Appendix B. In addition to these guidelines, the aircraft was flown into the stalled condition in coordinated flight (centered ball) for all tests. All stalls were initiated from a pressure altitude of 4300 feet. Outside air temperature at test altitude varied from 50 to 52 degrees. Weather was clear with unrestricted visibility. A distinct horizon was available for reference throughout the stall testing. Winds at 3000 feet were forecast to be from 320 degrees at 8 knots and 6000 foot winds were forecast from 340 degrees at 19 knots. Actual winds during the test are estimated to have been from 330 to 350 degrees at 10 knots. The stall tests were conducted on headings within ten degrees of 340 and 160 degrees to minimize any wind effect at stall.
 
 

Results

The aircraft did not exhibit consistent nose down pitch moment at stall. A total of 13 of 60 stalls (21.6%) exhibited nose down pitch moment at stall. An abrupt aircraft roll response during certain stalls was a finding not targeted in the original experiment. Results of each stall series are detailed in Appendix E.
 
 

Discussion

The aircraft exhibited inconsistent pitch moment at stall. The majority of test stalls (78.4%) did not exhibit nose down pitch at stall. The findings are inconsistent with recent FAA findings. This inconsistency, in light of similar Swiss findings, lends credence to concerns of widely varying stall performance in individual aircraft. The findings support test pilot testimony that PA-38 stall performance was unpredictable and inconsistent across production aircraft. Abrupt roll off encountered during some of the tests is consistent with previous pilot reports and testimony from Inman investigation interviews.

Although analysis of roll off at stall was not targeted in the wings level stall tests, analysis of roll response recorded in the 180 familiarization stalls provides insight into the aircraft’s roll reaction at stall. One fourth (25%) of the 180 familiarization stalls exhibited moderate or greater roll off at stall. The worst roll off was obtained performing a power off, flaps up, 30 degree angle of bank left turning stall. This configuration is fairly common when flying a standard traffic pattern for landing. The data indicates that the use of flaps tended to mitigate roll off tendencies during the familiarization stalls. Oil canning, the tendency for the wing skin to flex in and out, was observed during familiarization and wings levels stall tests.

Limitations. All findings are based on the wings level stall characteristics of one aircraft. This aircraft may not be representative of the population of Piper PA-38 aircraft. Although this aircraft underwent an annual inspection within 30 days of the test flights, there was no formal type conformity inspection conducted on the test aircraft. Another limitation was that pitch moment at stall was evaluated only as "up", "down", or "none". Those interested in analyzing data obtained in the familiarization stall tests are cautioned against comparing flight instructor performed stall results and student performed stall results. Variation in stall recognition and initiation of recovery between the author and the student pilot negates validity in any comparison of results and meaningful insights are unlikely.
 
 

Experiment 2

Method

A comparison of PA-38 stall/spin accident rate data with Cessna 150/152 and Beech 77 stall/spin data was performed to determine if the PA-38 exhibited a statistically significant higher stall/spin accident rate. The analysis involved a comparison of expected and observed proportions using Pearson’s chi-square statistic. Data available in the AOPA Safety Foundation’s publication, Safety Review Piper PA-38-112 ,was used to perform the analysis. This publication lists data obtained from FAA general aviation activity surveys. The data covered a 10-year period from 1982 to 1993. Two chi-square analyses were performed. One examined reported stall/spin accident rates in relation to numbers of active aircraft and one examined reported stall/spin accident rate in relation to numbers of flight hours flown.

Participants. The author of this study conducted the chi-square analyses.

Apparatus and Materials. A computer based chi-square statistical analysis, available on the internet, was utilized to conduct the two tests (Lowry, 1998).

Procedure. The null hypotheses were that stall/spin accident frequencies would mirror proportions of active airframes and flight hours flown. Expected cell frequencies reflecting these null hypotheses were computed as depicted in tables 2 and 3. Expected frequencies were contrasted with observed frequencies. Both chi-square analyses were run with the C-150/152 and the BE-77 combined into one category to allay concerns over low BE-77 cell counts. As a result, no significant change occurred in chi-square values and a decision was made to keep the categories of aircraft separate.
 
 

Table 2. Expected Number of Stall/Spin Accidents as a Proportion of Active Aircraft.

A/C Type -- Active Aircraft -- Prop. of total A/C -- Expected N of 313 Stall/Spin Accidents

C-150/152 -- 16,373 -- .910 -- 284.83

BE-77 -- 225 -- .013 -- 4.07

PA-38 -- 1388 -- .077 -- 24.10

TOTALS -- 17986 -- 1.0 -- 313
 
 

Table 3. Expected Number of Stall/Spin Accidents as a Proportion of Flight Hours by Type A/C.

A/C Type -- Hours Flown -- Prop. of Hours Flown -- Expected N of 313 Stall/Spin Accidents

C-150/152 -- 3,300,609 -- .8829 -- 276.35

BE-77 -- 59,745 -- .0160 -- 5.00

PA-38 -- 377,970 -- .1011 -- 31.65

TOTALS -- 3,738,324 -- 1.0 -- 313
 
 

Results

An alpha level of .05 was used for all statistical tests. Both chi-square tests were considered one tailed because a higher PA-38 stall/spin accident rate was the only possibility given the available data. Chi-square analysis of number of stall/spin accidents per active PA-38, Cessna 150/152 and the Beech 77 aircraft resulted in a chi-square of 30.18, which was significant at less than the .0005 level. Effect size was moderate with the PA-38 proportion of active aircraft (7.7%) accounting for 15.97 % of stall/spin accidents.

Analysis of cells, by active aircraft, revealed that the PA-38 was involved in twice as many stall/spin accidents as predicted by the null hypothesis. Expected cell count for the PA-38 was slightly less than 25 (24.154554) accidents. Actual stall/spin accidents numbered 50 for the Tomahawk. The residual number of 25 in the Tomahawk’s chi-square cell was equal to a residual of —25 for the Cessna 150/152 series aircraft. The Beech 77 was the only aircraft with equal expected and actual cell counts (4 and 3.91). Table 4 summarizes expected and observed results of the chi-square test by active aircraft.
 
 

Table 4. Expected and observed Stall/Spin Accidents by Active Aircraft with residuals.

A/C Type -- Expected N -- Observed N of 313 Accidents -- Residuals

C-150/152 -- 284.83 -- 259 -- - 25.83

BE-77 -- 4.07 -- 4 -- - .07

PA-38 -- 24.10 -- 50 -- + 25.90 .

Chi-Square: 30.18 df: 2 p < .0005, two-tailed
 
 

Chi-square analysis of stall/spin accidents per flight hours flown resulted in a chi-square of 11.93, which was significant at the .005 level. The chi-square value by flight hours was less than the chi-square by number of active aircraft yet still indicated strong statistical significance. The expected cell count for the PA-38-112 was slightly over thirty-one, resulting in a residual of 19. The Cessna 150 residual was 17.35 and the BE-77 had a residual of 1. Table 5 summarizes expected and observed for the chi-square by flight hours. Effect size was slightly less than moderate with PA-38 proportion of flight hours (10.11%) accounting for 15.97 % of accidents.
 
 

Table 5. Expected and Observed Stall/Spin Accidents by Flight Hours with Residuals.

A/C Type -- Expected N -- Observed N of 313 Accidents -- Residuals

C-150/152 -- 276.35 -- 259 -- - 17.35

BE-77 -- 4.07 -- 4 -- - 1.0

PA-38 -- 31.65 -- 50 -- + 18.35 .

Chi-Square: 11.93 df = 2 p < .005, two tailed
 
 

Discussion

The results of the chi-square analyses, by number of active aircraft and by flight hours flown, indicate that the PA-38 is statistically more likely to be involved in a stall spin accident than either the Cessna 150/152 or the Beech 77 Skipper. The observed significance levels of .0005 and .005 for the two tests are very statistically significant. Observed cell counts approximately twice the size of expected incidents, coupled with moderate effect sizes, portend substantive significance as well.

Large sample sizes in both active aircraft and hours flown enhance the validity of the findings. Sample size coupled with the fact the data spans 10 years of flight operations makes it likely that the data is representative of the population. The data indicates that for every active PA-38 aircraft nearly 12 Cessna 150/152 aircraft were active. These C-150/152 aircraft were exposed to nearly nine times as many flight hours as the PA-38 aircraft. Based on exposure to the environment (in numbers of aircraft and hours flown) the Cessna aircraft had a considerably greater "opportunity" to be involved in a stall/spin accident. On the other hand, the PA-38 had relatively small exposure to the environment necessary to become involved in a stall/spin accident yet exhibited a much larger stall/spin accident rate. Analysis of sample sizes together with very statistically significant chi-square values is concerning. A logical independent variable influencing the phenomenon of a high stall/spin accident rate for the PA-38 is the aircraft itself.

Limitations. Data used for analysis was taken from a secondary source of compiled NTSB and FAA data. The 313 stall/spin accidents analyzed were coded as stall/spin accidents by the NTSB. NTSB accident coding was accepted and no analysis of individual accidents was performed. Many variables exist in these accidents. The only variable analyzed in this study was aircraft type.
 
 

Summary discussion

The wings level stall findings indicate lack of a definitive nose down pitch at stall. This lack of nose down pitch is coupled with varying degrees of roll off, at times, in an unpredictable direction. The divergent findings of the Swiss and the FAA, and now these findings, create serious doubt that the PA-38 exhibits consistent flight characteristics at stall. This lack of consistent aircraft reaction to stall may be a factor influencing the higher stall/spin accident rate of the PA-38 compared to other trainers.

Authors Note

Substantive findings. The author’s opinion is that the results of this study are substantive. Large numbers of C-150 aircraft and flight hours limit effect size estimates to moderate and understate the significance of the PA-38 stall/spin accident rate. In aggregate, the wings level stall test results of this study, the statistical significance of high chi-square values derived from large PA-38 stall/spin accident numbers together with a long history of problems with the aircraft’s stall/spin performance amount to a convincing indictment against PA-38 safety. The FAA needs to act on all of the NTSB recommendations stemming from the Inman investigation.

Allegations of a lack of PA-38 nose down pitch at stall were answered in two ways by the FAA. First, a single aircraft was wings level stall tested and found to pitch nose down at stall. Secondly, the FAA noted that a nose down pitch at stall was not necessarily required by regulations.

The tests conducted by this author did not specifically target recording of nose down pitch magnitude or rate. However, it is worth noting that of those stalls exhibiting nose down pitch (21.6%) most exhibited very little nose down pitch, almost always less than 5 degrees. Many test stalls were recorded as nose down pitch only by intently watching the horizon and discerning what was often a very small nose down pitch in rate and degree. None of the test stalls that exhibited nose down pitch exhibited the distinct and obvious pitch down typical of Cessna and Piper PA-28 aircraft.

The issue of whether an FAA requirement to pitch down at stall exists is aside from the fact that nearly all authoritative sources on stall characteristics consider pitch down at stall to be highly desirable from a safety standpoint. Many general aviation sources consider the phenomenon of pitch down at stall to be a given. In Paul A. Craig’s book, Stalls and Spins, he notes, "As the full stall takes place, the nose will pitch down….When the nose pitches down in the stall, the job of lowering the nose to below the critical angle is done for you" (1993, p. 58-59). In describing approach stalls, Michael C. Love, in Spin Management and recovery, notes, "When the plane stalls, the nose will probably drop perceptibly" (1996, p.36). Even the FAA’s Flight Training Handbook, a major source of training information, states, "The full stall will be evidenced by such clues as full up-elevator, high sink rate, uncontrollable nose-down pitching and possible buffeting" (FAA, 1980, p. 147).

AC 23-8A, the flight test guide for certification of Part 23 airplanes, Section 23.141, provides an explanation of what a test pilot should be looking for during the certification process and provides what may be considered the spirit of Part 23 flight testing:

"The regulation of Aviation Safety rests primarily in the hands of the Federal Aviation Administration of the United States Government. It is complete and encompasses all aspects of aviation safety…" ( Kane, 1996, p. 8-1). In terms of meeting its responsibilities regarding the issue of PA-38 safety, the FAA has fallen tragically short.

Acknowledgements. Many thanks to Dr. Brent Bowen of the University of Nebraska at Omaha-Aviation institute, his graduate assistant Karisa Kane and my wife Karen. Their proof reading and guidance in reporting of findings and formatting of the paper was invaluable. An additional thank you goes out to Jason Schneweis for his assistance in performing familiarization stall training.
 
 

References

Askue, Vaughan (1992). Flight testing homebuilt aircraft. Ames, IA: Iowa State University.

Craig, Paul A. (1993). Stalls & Spins. Blue Ridge Summit,PA: Tab books.

Federal aviation administration (1993). Federal aviation regulation 23.201. FAA [On-line]. Available: http/www.faa.gov.

Federal aviation administration (1998). NTSB recommendations to faa and faa response report. [On-line]. Available: http://nasdac.faa.gov/asp/fw_searchus.asp.

Kane, Robert M. (1996). Air transportation (12^th Ed), Dubuque, IA: Kendall/Hunt.

Landsberg, Bruce (1996). Safety review, piper tomahawk PA-38-112, AOPA Air Safety Foundation. Frederick, MD.

Love, Michael C. (1996). Spin management and recovery. New York: McGraw-Hill.

Lowery, John. (1997b, June). Case not closed on PA-38. Aviation safety. 11.

Lowery, John (1997a, September). NTSB takes action on tomahawk. Aviation safety. 9.

Lowery, John (1998, September). The tomahawk chop. Aviation Safety. 12.

Lowry, Richard (1998). VassarStats. [On-line]. Available: http://faculty.vassar.edu~lowry/csfit.html.

NTSB (1997). Brief of accident CHI 94 FA 097, file no. 1793. Washington, DC:National Transportation Safety Board.

Stinton, Darrol (1996). Flying qualities and flight testing of the airplane. Oxford, UK: Blackwell Scientific Publications.

Stowell, Rich (1997). Special safety report. [On-line]. Available: http://www.west.net/~rstowell/prices.html.

U.S. Department of Transportation. (1980). Flight training handbook (AC 61-21A). Washington, DC: U.S. Government Printing Office.
 
 

Appendix A NTSB Inmann, Kansas Accident Investigation Recommendations

Recommendation A-97-41: Perform certification flight tests with a minimum of two test airplanes and document any changes required to bring these test airplanes into conformance with the type certificate.

Status: A-97-41. Both the FAA and New Piper performed certification flight tests in accordance with the recommendations. Stalls were performed to fill a few areas of missing data in the original tests. FAA findings were that the production PA-38-112 met FAR Part 23 stall certification requirements.

Recommendation A-97-42: Perform evaluation of wings-level stalls to ascertain if the stall is defined by a downward pitching motion of the aircraft.

Status: A-97-42. The FAA reported to the NTSB in a letter dated march 31,1998," the FAA conducted wings level stall tests in accordance with 14 CFR 23.201, Amendment 23-14. The flight tests were conducted using the Piper PA-38-112, S/N 38-78A0294, Registration Number N9246T. The airplane demonstrated satisfactory stall characteristics….The stalls for the Piper PA-38-112 were accompanied by the classic downward pitching motion…" (NTSB on-line).

Recommendation A-97-43: Perform stall warning tests in accordance with FAR Section 23.207 to ensure the stall warning horn activates at least 5 knots before stall.

Status: A-97-43. The FAA agreed to perform stall warning horn tests as part of their wings level stall re-certification of the PA-38-112. In a March 31,1998 letter to the NTDSB, the FAA reported that "Stall warning tests were conducted satisfactorily in accordance with 14 CFR 23.207.

Recommendation A-97-44: Perform spin certification tests to ensure it is impossible to obtain an unrecoverable spin by any use of flight controls or engine controls. Verify that the results obtained in the original certification tests can be duplicated in production airplanes. Use at least two airplanes in these tests.

Status: A-97-44. ). In March of 1998 the FAA responded as follows, "The FAA determined that there was adequate certification data available to demonstrate compliance of the Piper PA-38-112 with the spin test requirements and to show conformity of the aircraft. However, the original flight test report did not properly document turning flight and accelerated stalls per 14 CFR 23.203, Amendment 23-14. Although the FAA and Piper believe these tests were conducted as part of the original certification, they were not documented. Consequently, these tests were repeated in the recent certification tests. The flight tests found that the PA-38-112 complied with certification requirements. The FAA plans no spin certification testing of the PA-38-112.

Recommendation A-97-45: Issue a mandatory requirement that all slow flight and stall training be conducted at or above the minimum altitude currently specified in the PA-38-112 pilot operating handbook for spin training and inform pilots of alternate methods of recovery from an inadvertent spin. This recommendation was deemed "Urgent".

Status: A-97-45. In a letter dated September 25, 1997 The FAA agreed with calling for stall and slow flight training at or above the POH’s recommended minimum spin altitude. The FAA considered action complete on this recommendation through issuance of a letter on August 18,1997, to all flight standards managers requesting that they inform operators of the PA-38-112 of the procedures contained in the POH for spin training. The FAA initially disagreed with informing pilots of alternate recovery methods prior to completion of any spin testing and told the NTSB that they considered action complete on all of recommendation A-97-45. The NTSB in January of 98 went on record as considering the status of Safety Recommendation A-97-45 to be classified ‘Open Unacceptable Response’.

Source: Federal aviation administration (1998). "NTSB recommendations to faa and faa response report". [On-line]. Available: http://nasdac.faa.gov/asp/fw_searchus.asp.
 
 

Appendix B FAA Advisory Circular 23-8A Change 1 Wings Level Stall Test Procedures (Pertinent sections only)

a. Explanation.

(1) Stall. Section 23.201 (c) defines when the airplane can be considered stalled, for airplane certification purposes. When either of two conditions occurs, whichever occurs first, the airplane is stalled. The conditions are:

(i) Uncontrollable downward pitching motion; or

(ii) The control reaches the stop

(2) Related Sections. The stalled condition is a flight condition that comes within the scope of 23.49, 23.141, 23,143 (b), 23.171, and 23.173 (a). Section 23.143 (b) requires that it be possible to effect a "smooth transition" from a flying condition up to the stalled flight condition and return without requiring an exceptional degree of skill, alertness, ore strength. Any need for anticipated or rapid control inputs exceeding that associated with average piloting skill, is considered unacceptable.

(3) Recovery. The flight tests include a determination that the airplane can be stalled and flight control recovered , with normal use of the controls.

(4) Power. The propeller condition for the "power-off" tests prescribed should be the same as the "throttles closed" condition prescribed for the stalling speed tests of 23.49, that is , propellers in the takeoff position, engine idling with throttles closed.

(5) Altitude Loss. Altitude loss in excess of 100 feet and nose-down pitch in excess of 30 degrees will be entered in the performance information section of the Aircraft Flight Manual. The power used to regain level flight may not be applied until flying control is regained. This is considered to mean not before a speed of 1.2 Vs1 is attained in the recovery dive.

(6) Configurations. Stall characteristics should be evaluated:

(i) At maximum to minimum weights at aft c.g.

(ii) With the elevator up stop set to the maximum allowable deflection.

(iii) With maximum allowable fuel imbalance.

(iv) At or near maximum approved altitude.

b. Procedures.

(3) Pilot Determination. During the entry and recovery , the test pilot should determine:

(i) That the stick force curve remains positive up to the stall.

(ii) That it is possible to produce and correct roll and yaw by unreversed use of the rolling and directional control.

(iii) The altitude loss.

(iv) The pitch attitude below level.

(v) the amount of roll or yaw encountered during the recovery.

(4) Speed Reduction Rate. /section 23.201 requires the rate of speed reduction for entry not exceed one knot per second.

Source: Federal Aviation Administration (1993). Advisory Circular 23-8A Change 1, Flight test guide for certification of part 23 airplanes. Washington, DC: U.S. Government Printing Office.

Appendix C Stall Data Recording Card

Configuration:

No flap Power on / off

Partial flap Power on / off

Full flap Power on / off

1. Pitch Up Pitch Down No Pitch
2. Pitch Up Pitch Down No Pitch
3. Pitch Up Pitch Down No Pitch
4. Pitch Up Pitch Down No Pitch
3. Pitch Up Pitch Down No Pitch
4. Pitch Up Pitch Down No Pitch
4. Pitch Up Pitch Down No Pitch
5. Pitch Up Pitch Down No Pitch
6. Pitch Up Pitch Down No Pitch
7. Pitch Up Pitch Down No Pitch

Appendix D Results of Familiarization Stall Training

This appendix contains the results of 180 familiarization stall tests. These tests were conducted as part of training in preparation for conducting wings level stall tests. These stalls were not strictly conducted in accordance with AC 23-8A standards however, were representative of stalls normally performed in a primary flight training syllabus.

Pitch and roll was evaluated during these stalls. Pitch at stall was evaluated as either "up", "down" or "none" and roll was evaluated in direction left or right and classified as either "less than slight", "slight", "moderate", or "great". "Less than slight" was up to 5 degrees of roll off. "Slight" was considered to be from 5 to 15 degrees of roll off before the pilot could correct to wings level and recover. "Moderate" was considered to be 15-30 degrees of roll off before the pilot could correct to wings level and recover. "Great" was considered to be more than 30 degrees of roll off before the pilot could reverse the roll off and recover the aircraft. The pilots initiated recovery immediately after stall in all cases.

This data was gathered during a total of 180 stalls in 9 varying aircraft configurations and flight parameters. The stalls were performed by two test pilots. One was the author and the other was a student pilot who was within days of taking his private pilot certification check ride. All evaluations of pitch and roll were performed by the Certificated flight Instructor. Both pilots attempted to keep the aircraft in coordinated flight during the tests. Results for each are summarized as follows:

Certificated Flight Instructor performed stalls

Power on wings level with flaps up

10 stalls performed

8 exhibited no pitch at stall

2 exhibited slight nose down pitch

5 exhibited slight right roll off at stall

1 exhibited great right roll off at stall

3 exhibited slight left roll off at stall

1 exhibited moderate left roll off at stall

SUMMARY: No significant nose down pitch at stall, controllable roll off in an unpredictable direction.
 
 

Power on in a left hand turn with flaps up

10 stalls performed

10 exhibited no pitch at stall

3 exhibited great right roll off at stall

5 exhibited moderate right roll off at stall

2 exhibited slight right roll off at stall

SUMMARY: aircraft exhibited no pitch down and moderate roll off in a predictable direction
 
 

Power on in a right hand turn with flaps up

10 stalls were performed

10 exhibited no pitch down at stall

3 exhibited less than slight left roll off at stall

3 exhibited moderate left roll off at stall

2 exhibited great left roll off at stall

2 exhibited slight left roll off at stall

SUMMARY: No nose down pitch, predictable roll off at stall
 
 

Power off, wings level, full flap

10 stalls were performed

9 exhibited no pitch at stall

1 exhibited nose down pitch at stall

2 exhibited slight left roll off at stall

1 exhibited moderate left roll off at stall

7 exhibited less than slight roll off either left or right

SUMMARY: No nose down pitch at stall with a tendency to roll off left
 
 

Power off in a left hand turn with full flaps

10 stalls were performed

6 exhibited no nose down pitch

4 exhibited slight nose down pitch at stall

10 exhibited slight right roll off at stall

SUMMARY: No significant nose down pitch at stall, no significant roll off.
 
 

Power off in a right turn with full flaps

10 stalls were performed

10 exhibited no pitch moment at stall

1 exhibited moderate right roll off a stall

9 exhibited less than slight right roll off at stall

SUMMARY: No pitch moment at stall and no significant roll off at stall
 
 

Power off, wings level flaps up

10 stalls were performed

0 exhibited nose down pitch at stall

3 exhibited moderate right roll off

4 exhibited slight right roll off

1 exhibited slight left roll off

2 exhibited less than slight left and right roll off

SUMMARY: No nose down pitch at stall with unpredictable slight to moderate roll off
 
 

Power off, 30 degree left hand turn, flaps up

10 stalls were performed

0 exhibited nose down pitch at stall

3 exhibited great right roll off at stall

3 exhibited moderate right roll off at stall

4 exhibited no roll off at stall

SUMMARY: No nose down pitch at stall with tendency to roll off 1 abruptly to the right
 
 

Power off, 30-degree right hand turn, flaps up

10 stalls were performed

0 exhibited nose down pitch at stall

3 exhibited slight right roll off at stall

7 exhibited slight left roll off

SUMMARY: No nose down pitch at stall with unpredictable slight roll off
 
 

Student pilot performed stalls

Power on wings level with flaps up

10 stalls were performed

10 exhibited no pitch at stall

3 exhibited less than slight right roll off at stall

1 exhibited moderate right roll off at stall

1 exhibited slight left roll off at stall

5 exhibited slight right roll off at stall

SUMMARY: No significant nose down pitch, predictable right roll off.
 
 

Power on in a left hand turn with flaps up

10 stalls performed

10 exhibited no pitch at stall

3 exhibited slight right roll off at stall

3 exhibited moderate right roll off at stall

4 exhibited slight right roll off at stall

SUMMARY: no pitch down, predictable moderate right roll off

Power on in a right hand turn with flaps up

10 stalls were performed

10 exhibited no pitch down at stall

3 exhibited less than slight roll off at stall

3 exhibited moderate left roll off at stall

2 exhibited great left roll off at stall

2 exhibited slight left roll off at stall

SUMMARY: No nose down pitch, predominant left roll off at stall
 
 

Power off, wings level, full flap

10 stalls were performed

10 exhibited no pitch at stall

3 exhibited slight right roll off at stall

3 exhibited moderate right roll off at stall

4 exhibited less than slight right roll off at stall

SUMMARY: No nose down pitch at stall with a tendency to roll off right
 
 

Power off in a left hand turn with full flaps

10 stalls were performed

9 exhibited no nose down pitch

1 exhibited slight right roll off at stall

1 exhibited slight nose up pitch

1 exhibited slight left roll off at stall

6 exhibited less than slight left roll off at stall

2 exhibited no roll off at stall

SUMMARY: No significant nose down pitch at stall, unpredictable significant roll off.

Power off in a right turn with full flaps

10 stalls were performed

0 exhibited nose down pitch at stall

10 exhibited no roll off at stall

SUMMARY: No pitch moment at stall and no significant roll off at stall
 
 

Power off, wings level flaps up

10 stalls were performed

0 exhibited nose down pitch at stall

2 exhibited moderate right roll off

6 exhibited slight right roll off

2 exhibited slight left roll off

SUMMARY: No nose down pitch at stall with unpredictable slight to moderate roll off
 
 

Power off, 30 degree left hand turn, flaps up

10 stalls were performed

0 exhibited nose down pitch at stall

3 exhibited great right roll off at stall

2 exhibited moderate right roll off at stall

1 exhibited no roll off at stall

2 exhibited less than slight left roll off at stall

2 exhibited slight left roll off at stall

SUMMARY: No nose down pitch at stall with unpredictable slight to great roll off.
 
 

Power off, 30-degree right hand turn, flaps up

10 stalls were performed

0 exhibited nose down pitch at stall

8 exhibited slight left roll off

5 exhibited less than slight right roll off at stall

SUMMARY: No nose down pitch with unpredictable slight roll off.
 
 

Appendix E Wings Level Stall Test Findings

Power off Stalls (30 Performed)
 
 

No Flap: (10 performed)

3 exhibited nose up pitch at stall

2 exhibited nose down pitch at stall

5 exhibited no pitch at stall

Summary: 20% of stalls exhibited nose down pitch

Additional comment: Aircraft exhibited somewhat abrupt roll off to the left at stall.
 
 

Partial Flap: (10 performed)

6 exhibited nose down pitch at stall

4 exhibited no pitch at stall

Summary: 60 percent of stalls exhibited nose down pitch
 
 

Full Flap: (10 performed)

4 exhibited nose down pitch at stall

6 exhibited no pitch at stall

40 percent of stall exhibited nose down pitch
 
 

Power on Stalls (30 Performed)
 
 

No Flap: (10 performed)

10 exhibited no pitch at stall.

Summary: 0% exhibited pitch down at stall
 
 

Partial Flap: (10 performed)

1 exhibited pitch down at stall

1 exhibited pitch up at stall

8 exhibited no pitch at stall

Summary: 10 % exhibited pitch down at stall

Additional Note: Hard right roll off at stall
 
 

Full Flap: (10 performed)

10 exhibited no pitch at stall

Summary: 0% exhibited pitch down at stall

Additional note: uncomfortable roll during stall. Near 60-degree wing drops at stall.