Different Hypothesis Testing Using R

What is Hypothesis Testing

It is a type of inferential statistics that involves extrapolating
results from a sample (random) to the entire population. It is used to
make decisions based on statistical tests and models that use the
p-value, also known as the Type I error or alpha error.

Type I Error : When we reject true null hypothesis then it is called
type I error

Type II Error : When we do not reject false null hypothesis then it
is called type II error.

It can be done using parametric or non-parametric methods/models.

Parametric : They have certain assumptions about the data (model)
and/or errors that must be validated before the results can be accepted.

Non-parametric : They are non-parametric because they make no
assumptions about the data distribution (model) or mistakes.

Why to use parametric test?

Because they are based on the mean, standard deviation, and normal
distribution, parametric tests are regarded “more powerful” than non
parametric tests/models. Non-parametric tests are based on median, IQR,
and non-normal distributions, non-parametric tests are deemed “less
powerful” than parametric tests/models.

Two Statistical Hypothesis

Null Hypothesis : It is also known as hypothesis of no difference
Alternative Hypothesis : It is complementary to the null hypothesis
also known as research hypotheis.

When to accept Null or Alternative Hypothesis

Accept (fail to reject) null hypothesis from parametric or
non-parametric tests requires a P-value > 0.05. (Goodness-of-fit tests)

To accept it from parametric or non-parametric testing (Research
hypothesis tests! ), the P-value must be less than 0.05.

Some Commonly Used Parametric Test Using R

One Sample Z Test On Mtcars Data

In this blog I am only going to explain how to test one sample z test
using R without explain what is z-test, how it work because I already
explained it in my past blog.

# we need to define parameter
muO <- 20
sigma <- 6
xbar <- mean(mtcars$mpg)
n <- length(mtcars$mpg)
z <-sqrt(n)*(xbar-muO)/sigma
p_value<-2*pnorm(-abs(z))

Let’s check z value and p value,

z
## [1] 0.08544207

Hence, we found value of z is 0.08544207

p_value
## [1] 0.9319099

We found p-vale 0.9319099 which is > 0.05 hence we accpet null
hypothesis, i.e means of sample and population are equal.

Why there is no one sample z-test in base R package?

Because the t-distribution behaves like the z-distribution for n>=30,
the T-test can be employed for both small and big samples. Thus, we
don’t need one-sample z-test in R!

One Sample t-test: We can work for small sample as well as for large sample

t.test(mtcars$mpg, mu =20)
## 
##  One Sample t-test
## 
## data:  mtcars$mpg
## t = 0.08506, df = 31, p-value = 0.9328
## alternative hypothesis: true mean is not equal to 20
## 95 percent confidence interval:
##  17.91768 22.26357
## sample estimates:
## mean of x 
##  20.09062

Hence we obtained p-valued 0.9328 it means we do not reject null
hypothesis.

Two Sample T-test

It is used to compare the means of a dependent variable with two
categories of grouped independent variables. For instance, we can
compare exam score (dependent variable) between male and female groups
of students!

Assumptions

  • For each category, the dependent variable must follow the normal
    distribution (Test of normality-GOF)

  • The variance is homogeneous (i.e. equal) across independent variable
    categories (Test of equal variance-GOF)

What to do if variance across independent variable categories not equal

In this case we used Welch test.

Assumption

  • For each category, the dependent variable must follow the normal
    distribution (Test of normality-GOF)

  • Variance across independent variable categories are not homogenous
    i.e; not equal.

Let’s do narmality test on mtcars data

with(mtcars, shapiro.test(mpg[am == 0]))
## 
##  Shapiro-Wilk normality test
## 
## data:  mpg[am == 0]
## W = 0.97677, p-value = 0.8987

Here, p-value is 0.8987. Hence, we do not reject null hypothesis that
means it follows normal distribution.

with(mtcars, shapiro.test(mpg[am == 1]))
## 
##  Shapiro-Wilk normality test
## 
## data:  mpg[am == 1]
## W = 0.9458, p-value = 0.5363

It also follows normal distribution. Hence first condition is satisfied
i.e; dependent variable mpg follows normal distribution.

Variance Check

var.test(mpg ~ am, data = mtcars)
## 
##  F test to compare two variances
## 
## data:  mpg by am
## F = 0.38656, num df = 18, denom df = 12, p-value = 0.06691
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
##  0.1243721 1.0703429
## sample estimates:
## ratio of variances 
##          0.3865615

We can see p-value is 0.06691 which is grater than 0.05. Hence we can
say variance across independent variable categories are same. Now we can
use two sample student t test.

t.test(mpg ~ am, var.equal= T, data = mtcars)
## 
##  Two Sample t-test
## 
## data:  mpg by am
## t = -4.1061, df = 30, p-value = 0.000285
## alternative hypothesis: true difference in means between group 0 and group 1 is not equal to 0
## 95 percent confidence interval:
##  -10.84837  -3.64151
## sample estimates:
## mean in group 0 mean in group 1 
##        17.14737        24.39231

Here we saw p-value 0.000285 which is less then 0.05. Hence we reject ho
that means milage (mpg) is statistically different among cars with
automatic and manual transmission system.

Let’s check two sample student t-test result with simple linear regression model

summary(lm(mpg ~ am, data = mtcars))
## 
## Call:
## lm(formula = mpg ~ am, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -9.3923 -3.0923 -0.2974  3.2439  9.5077 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   17.147      1.125  15.247 1.13e-15 ***
## am             7.245      1.764   4.106 0.000285 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 4.902 on 30 degrees of freedom
## Multiple R-squared:  0.3598, Adjusted R-squared:  0.3385 
## F-statistic: 16.86 on 1 and 30 DF,  p-value: 0.000285

This difference is statistically significant and the p-value is same as
given by the two-samples t-test.

What test should we used if we have to compare mean of more than two samples

If we need to compare mean of more than two samples we used 1-way ANOVA
test.

Assumption

  • Dependent variable must be “normally distributed”

  • Variance across categories must be same

1-way ANOVA assumptions checks

Normality by categories

with(mtcars, shapiro.test(mpg[gear == 3]))
## 
##  Shapiro-Wilk normality test
## 
## data:  mpg[gear == 3]
## W = 0.95833, p-value = 0.6634

Category 3 follows normal distribution.

with(mtcars, shapiro.test(mpg[gear == 4]))
## 
##  Shapiro-Wilk normality test
## 
## data:  mpg[gear == 4]
## W = 0.90908, p-value = 0.2076

Category 4 also follows normal distribution.

with(mtcars, shapiro.test(mpg[gear == 5]))
## 
##  Shapiro-Wilk normality test
## 
## data:  mpg[gear == 5]
## W = 0.90897, p-value = 0.4614

So, dependent variable follows normal distribution.

Let’s do variance test

In case of more than two samples case we do not use
var.test(). For this we usedleveneTest() avilable in car packages.
Let’s check. Before doing this we need to change our independent
variable into factor.

library(car)
## Loading required package: carData
leveneTest(mpg ~ as.factor(gear), data=mtcars)
## Levene's Test for Homogeneity of Variance (center = median)
##       Df F value Pr(>F)
## group  2  1.4886 0.2424
##       29

Here, we find p-value grater than 0.2424. Hence variance across
categories is same. So, we can now used one way calssical ANOVA test.

1-Way Classical ANOVA test

summary(aov(mpg ~ gear, data = mtcars))
##             Df Sum Sq Mean Sq F value Pr(>F)   
## gear         1  259.7  259.75   8.995 0.0054 **
## Residuals   30  866.3   28.88                  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

We find p-value less than 0.05. Hence we reject null hypothesis that
means sample means are not equal. This means, post-hoc test or pairwise
comparison is required. If alternative hypothesis is accepted we need to
do post-hoc test. For classical 1-way ANOVA TukeyHSD post-hoc test is
best. Let’s used it.

TukeyHSD(aov(mpg ~ as.factor(gear), data = mtcars))
##   Tukey multiple comparisons of means
##     95% family-wise confidence level
## 
## Fit: aov(formula = mpg ~ as.factor(gear), data = mtcars)
## 
## $as.factor(gear)
##          diff        lwr       upr     p adj
## 4-3  8.426667  3.9234704 12.929863 0.0002088
## 5-3  5.273333 -0.7309284 11.277595 0.0937176
## 5-4 -3.153333 -9.3423846  3.035718 0.4295874

Let’s check this result with simple linear model

summary(lm(mpg ~ gear, data = mtcars))
## 
## Call:
## lm(formula = mpg ~ gear, data = mtcars)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -10.240  -2.793  -0.205   2.126  12.583 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)   
## (Intercept)    5.623      4.916   1.144   0.2618   
## gear           3.923      1.308   2.999   0.0054 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 5.374 on 30 degrees of freedom
## Multiple R-squared:  0.2307, Adjusted R-squared:  0.205 
## F-statistic: 8.995 on 1 and 30 DF,  p-value: 0.005401
pairwise.t.test(mtcars$mpg, mtcars$gear, p.adj= "none")
## 
##  Pairwise comparisons using t tests with pooled SD 
## 
## data:  mtcars$mpg and mtcars$gear 
## 
##   3       4    
## 4 7.3e-05 -    
## 5 0.038   0.218
## 
## P value adjustment method: none

gear = 3 category is omitted from the result because R automatically
creates 3 dummy variables for 3 categories of gear variable i.e. 3, 4
and 5 and uses only last two of them in the model and takes the first
one as reference.

Leave a Reply

Scroll to top
Subscribe to our Newsletter

Hello surfer, thank you for being here. We are as excited as you are to share what we know about data. Please subscribe to our newsletter for weekly data blogs and many more. If you’ve already done it, please close this popup.



No, thank you. I do not want.
100% secure.