User Guide
Statistical test function
A statistical test functions in the following way:
- A hypothesis is considered null when there is no difference between the samples.
- We calculate the probability of falling into the same configuration as that obtained through previously observed samples by following the nul hypothesis. This probability is called "Alpha risk" or "p-value".
- If the alpha risk is <5% then obtaining such a configuration under the nul hypothesis is considered unlikely. The null hypothesis is therefore rejected and we consider the difference between the samples to be significant.
This is why all the results from the statistical tests offered by Ellistat will be associated with an alpha risk value according to the following scale:
The number entered at the bottom of the scale is equal to alpha risk of the given test:
- If the alpha risk < 0.01 then the difference is considered to be very significant.
- If the alpha risk < 0.05 then the difference is considered to be significant.
- If the alpha risk < 0.01 then the difference is considered to be limited. (It cannot be confirmed that the difference is significant but the hypothesis is interesting)
- If the alpha risk > 0.1 then the difference is considered to be insignificant.
The Ellistat alpha risk limit can be modified by going to "Home/Settings"
Example
Let us use the following example to show how a statistical test functions. Suppose that we want to detect the fact that a coin is loaded when flipping it. Assume that the coin always comes up tails.
After the first toss the coin comes up heads, can we deduce from this that the coin is loaded?
It would be daring to say that a coin is loaded in this case because this result could occur with any standard coin.
In this example the null hypothesis is: the coin is not loaded and therefore has a one in two chance of coming up heads or tails. The probability of a non-loaded coin coming up tails is 50%.
Consequently, the probability of getting heads on the first toss of a non-loaded coin is 50% and we would say that the alpha risk of the test is:
I.e. there is a 50% chance of getting the same result in the following null hypothesis.
After the second throw the coin again comes up tails. The alpha risk becomes:
Can we then deduce that the coin is loaded? The question therefore is: at what alpha risk can we say that the coin is loaded?
The general rule in the industry is a minimum alpha risk of 5%. I.e.:
If the alpha risk is < 5% then we reject the null hypothesis and we can assume that the coin is loaded.
If the alpha risk is > 5% then we cannot confirm that the coin is loaded. This does not mean that the coin is necessarily not loaded, this depends on the number of tosses made.
Continuing with our example:
In this case, after the 5th consecutive toss where the coin comes up tails, we can confirm that the coin is loaded as it has a risk below 5%.