When using these tools, don’t forget to try data transformations, especially when the sample variances increase as the sample means increase. Keep in mind that the p-values you see may be underestimated if the variances are not equal. If a t-test or ANOVA are used and the reported p-value is marginally significant, then the actual p-value may be marginally insignificant. When using ANY statistical tool, one should ALWAYS consider the practical significance of the result as well as the statistical significance of the result before passing final judgement.
In the Improve Phase, the Black Belt will often use designed experiments to make dramatic improvements in the performance of the CTQ. A designed experiment is a procedure for simultaneously altering all of the leverage variables discovered in the Analyze Phase and observing what effects these changes have on the CTQ. The Black Belt must determine exactly which leverage variables are critical to improving the performance of the CTQ, and establish settings for those critical variables.
In order to determine whether an effect from a leverage variable, or an interaction between 2 or more leverage variables, is statistically significant, the Black Belt will often utilize an ANOVA table, or a Pareto chart of effects, or a normal plot of effects. The results from all of these are based upon the estimate of the error variance, which is affected when the sample variances are not equal. Fortunately, most designed experiments are balanced, in other words, the sample sizes are all equal. Thus, the equal variances assumption can be relaxed for balanced experiments.