Week 2 – Statistical Significance vs Clinical Significance
Please review the article An Overview of Statistical and Clinical Signficance in Nursing Research
(listed in week 2) as well a view this week’s recording. Pair up with another student(s) based on instructor instructions, being assigned a group number. Create a brief PowerPoint (or other approved media) presentation explaining:
Slide 1. The definition of Statistical Significance and Clinical Significance
Slide 2. Explain how they are individually important when looking at research.
Slide 3. Present an example, using referenced literature, of each.
Slide 4. Explain how you will utilize this difference as you review literature.
Please create a link (YouTube, Vimeo, etc) and post this recorded presentation link in the discussion board. Each person from your group needs to post in the discussion board, but the post should be titled “Group#_Significance.”
This discussion board lasts one week. Each student is expected to participating incrafting the initial post in a new thread that refers to relevant course readings, this week’s highlighted article and draws from at least one additional external reference.. Discussion board posts may incorporate personal experiences in addition to course content.
You must have these components covered to earn all points:
1. APA formatting is required
2. At least 3 References.

overviewofstatisticalandclinicalsignificanceinnursingresearchstudymaterial.pdf
JUNE 1997. VOL 65, NO 6
R E S E A R C H C O R N E R
An overview of statistical significance in nursing
Editof’s note: This column in /he Journal highlights reseorch issues related to peri operative nursing practice. The outbors, AORNs codirectors of perioperative reseorch, provide practical advice for reading, conducting, and using reseorch in perioperative nursing pmctice.
esearchers talk about sturisri cul signrficuizce, and clini R cians talk about clirzical sig
rz$cutzce. Both terms often are used in research reports, and it is important for perioperative nurses to understand how these terms are similar and different. Although these terms sound alike, they are not interchangeable concepts.
QUANTITATIVE RESEARCH DATA
researchers collect data that are numerical (eg, physiologic mea sures, test scores, ratings). The people who provide this informa tion are the subjects, and the group of study subjects makes up the study sample. The sample is a subset of the population of inter est (ie, all possible subjects who meet the study criteria). After the researchers collect the data, they analyze them using a variety of statistical procedures (eg, t tests, analysis of variance, multiple regression). The researchers use the data analysis results to make inferences (ie, decisions) about the population of interest based on the results obtained from the study sample.
The basic premise of statisti cal testing is that the study sam ple is representative (ie, is a theo retical distribution) of the popula
In a quantitative study.
tion of interest. The researchers may make decision errors if the sample does not reflect the popu lation of interest. Statistical tests are only as good as the data that are analyzed. If the data are flawed, the statistical test results also may be suspect.
During the planning phase of a study, the researchers must make two important decisions. The first decision is to formulate the null and research hypotheses. and the second is to establish the level of statistical significance (ie, the alpha [a] level). Both of these decisions are important for the process known as hypothesis test ing, which is the basis of quanti tative statistical analysis.
Hypothesis formulation. In research that is based on hypothe sis testing, nothing is ever proven. This often is a point of confusion for novice researchers or new readers of research. In a quantitative study, the researchers formulate a null hypothesis and a research (ie, alternative) hypothe sis. The null hypothesis states that there is no relationship between the variables of interest in the study, whereas the research hypothesis states that there is a relationship between these vari ables. The research hypothesis is
SUZANNE C. BEYEA, RN, CS, PHD. is AORN codirec tor ofperioperutiv reseorrli.
LESLIE H. NICOLL, RN, MBA, PHD, is AORN codirector of’perinperutii7e research.
and clinical research based on previous research, sci entific principles, and the researchers’ knowledge and expertise. Based on the statistical test results, the researchers either accept or reject the null hypothe sis (ie, if the results demonstrate that there is no difference between variables, the null hypothesis is accepted). If the null hypothesis is rejected, the researchers surmise that some thing else is true (eg, the relation ship between variables that is stated in the research hypothesis actually exists). Just because the null hypothesis is rejected, how ever, does not mean that the research hypothesis is proven.
level of significance. The level of significance establishes the risk that the researchers are willing to take when testing the null hypothesis (ie, the probabili ty of rejecting a null hypothesis that really is true). Researchers traditionally set the a level at .05, although some researchers use more stringent (ie, .Ol ) or more relaxed (ie, .lo) a levels. When setting the level of significance, the researchers are determining how crucial it is to have accurate results. When researchers set the a level at .05, it means that they are willing to be wrong five times out of 100 when rejecting the null hypothesis. If researchers set the a level at .01, it means that they are willing to be wrong only one time out of 100.
It is important to remember that all statistical testing is based on the concept of probability.
1128 AORY JOL’RhUAL
JUNE 1997, VOL 65, NO 6
When researchers set an a level at .05 and the study results indicate that the null hypothesis should be rejected, the researchers make the following conclusion: There is a 95% chance that these results do in fact reflect what is happening in the population of interest, and there is a 5% chance that these results occurred by chance alone and the null hypothesis actually is true.
Types of errors. In hypothesis testing, researchers can make two types of errors (Table 1). A type I error occurs when researchers reject a null hypothesis that actual ly is true. A type II error is just the opposite (ie, researchers accept a null hypothesis that actually is false). Researchers are most con cerned with making type I errors (ie, concluding that a difference exists when there is none).
STATISTICAL SIGNIFICANCE During the process of hypoth
esis testing, researchers perform the statistical test and calculate a probability ( P value), and com pare that value to the a level. If the P value is less than the a level, the researchers reject the null hypothesis. If the P value is greater than the a level, the researchers accept the null hypothesis. Researchers often write, “The probability was less than .05 (P < .05),” to describe a
Table 1
result in which the null hypothe sis was rejected.
Researchers also may calcu late an actual probability, which can be reported; however, they still compare this figure to the a for decisionmaking purposes. In the research article “Effect of sur gical hand scrub time on subse quent bacterial growth” that appears in this issue of the Jour nal, the researchers calculated that the actual probability was .02, which is less than the level of .05 that they chose; thus, they rejected the null hypothesis.’
CLINICAL SIGNIFICANCE When researchers conclude
that statistical significance exists, this means that there is a very low probability that the findings occurred by chance alone. In sig nificance testing, the word signif cant does not mean that the results are important or that they have clinical or practical use. A study with nonsignificant statisti cal results still may be helpful in understanding the lack of a rela tionship between a group of vari ables. Statistically nonsignificant results also may help researchers discover study design or measure ment flaws. Regardless of what researchers conclude about the statistical tests used in a study, readers and consumers of research always must consider the question
POTENTIAL OUTCOMES OF STATISTICAL DECISION MAKING
“What is the practical or clinical Significance of this research?”
There are no statistical tests to help readers and consumers of research determine the clinical significance of study results. We must use our critical thinking skills and clinical expertise to study and interpret research results, and then we need to decide whether the statistical results have any relevance to our clinical practice settings.
Whereas statistical signifi cance relates to group differences or effects, clinical significance is more important at the individual patient level. For example, researchers might design a study to evaluate a weightloss inter vention. If subjects in the inter vention group lost an average of 20 lb (9 kg) and subjects in the nonintervention group lost an average of 15 lb (6.8 kg), the 5lb (2.27kg) difference between the two groups in this hypothetical study would not be statistically significant. The intervention, however, might be clinically sig nificant to individual patients if the subjects in the intervention group reported an improved sense of wellbeing and quality of life.
In contrast, other researchers might study oxygen saturation differences related to patients’ body positions (ie, supine, sit
ting). Mean oxygen satura tion in the supine posi tion might be
The actual situation in the population is that the null hypothesis is: g6%, com true false pared to 99%
in the sitting position. When tested statistically,
The researcher concludes that the null hypothesis is:
true (accepted) correct decision lype I 1 error
false (rejected) lype I error correct decision this difference
1129 AORN JOURNAL
JUNE 1997. VOL 65. NO 6
might be statistically significant ( P < .05), but the difference might not be clinically significant for individual patients.
INTERPRETING RESEARCH When reading and interpreting
research reports, it is important to assess the relevance of the find ings, regardless of whether they are statistically significant. Nurs es can decide what constitutes clinically important research results by considering what dif ference the findings make to indi vidual patients and also the costs involved in achieving the out comes. If the costs are high and the benefits are minimal, statisti cally significant interventions may not be costeffective or clini cally significant.
The research article in this issue of the Jouriial illustrates some of the challenges in inter preting statistical and clinical sig nificance. The authors state,
Although the mean hacteri al count differed signifi cantly (P = .02) between the twominute and three minute surgical hand scrub times, it fell below 0.5 log, which is the threshold for practical and cliriicol significance.?
Some practitioners may be tempted to adopt this study’s findings in clinical practice. The authors, however, recommend
N O T E S
that future studies be conducted to test the clinical importance of bacterial reductions achieved with various surgical hand scrub times, using the 0.5 log reduction benchmark.’ Changes in surgical hand scrub times should not be
When reading and interpreting
research, it is important to assess the relevance of the
findings.
made based on one study. Although this study’s results
are statistically significant, their clinical significance is a complex issue. The study was conducted in one institution with a relative ly small sample. The authors include evidence (ie, power analysis) that the sample is repre sentative of the population of interest; however, the population in this case was the perioperative RNs and surgical technologists (STs) at the authors’ institution, not the entire US population of perioperative RNs and STs. The results, therefore, cannot be gen eralized to another institution unless the study sample is similar
2. Ibid.
to the population of RNs and STs in the other institution. The authors suggest two practice changes, but they note that these changes should occur only if additional studies using other surgical hand scrub agents sup port their finding^.^ These authors clearly are aware of the importance of differentiating between statistical and clinical significance.
This study presents new find ings and raises interesting ques tions. Statistical significance should never be the sole reason to change clinical practice; clini cal significance also must be considered. Readers of research always should ask the question “So what?” For example, if there is a statistically significant differ ence between patients’ oxygen saturations in the supine and sit ting positions, so what? If this difference is only 3%, is it clini cally significant? The same ques tion applies to the surgical hand scrub time study results de scribed previously. The mean bacterial count differed signifi cantly between the two scrub time groups, but the clinical sig nificance of this difference is not clear. Such issues require further study to clarify the clinical sig nificance of the findings. A Readers who have questions and ideas about perioperative nursing research are encouraged to caN AORN’s codirectors of perioperative research ot (800) 755 2676 x 8277
1, S M Wheelock. S Lookinland, “Effect of surgical 3 . Ihid. 4. h i d . hand scrub time on subsequent bacterial growth,” AORN
Journal 65 (June 1997) 10871098.
1130 AORN JOURNAL
 An overview of statistical significance in nursing and clinical research
 QUANTITATIVE RESEARCH DATA
 Hypothesis formulation.
 level of significance.
 Types of errors.
 STATISTICAL SIGNIFICANCE
 CLINICAL SIGNIFICANCE
 INTERPRETING RESEARCH
 NOTES