The stepwise method is a modification of the forward selection method. The stepwise
method is differerent because the variables that are already in the model do not
necessarily stay there. As in the forward selection method, variables are added one by one
to the model, and the F statistic for a variable to be added must be significant at the
Significance level to add an effect to the model value.
After a variable is added, the stepwise method looks at all the variables already included in
the model and deletes any variable that does not produce an F statistic significant at the
Significance level to remove an effect from the model value. Only after this check is
made and the necessary deletions are accomplished can another variable be added to the
The stepwise process ends under either of these conditions:
when no variable outside the model has an F statistic significant at the Significance
level to add an effect to the model value and every variable in the model is significant
at the Significance level to remove an effect from the model value.
when the variable to be added to the model is the variable that was just deleted from it.
Minimum R square improvement
The minimum R-square improvement method closely resembles the maximum R-square
improvement method, but the variables that are chosen produce the smallest increase in R
. For a given number of variables in the model, the maximum R-square and minimum R-
square methods usually produce the same "best" model, but the minimum R-square
method considers more models of each size.
Maximum R square improvement
The maximum R-square improvement method does not settle on a single model. Instead, it
tries to find the "best" one-variable model, the "best" two-variable model, and so on,
although it is not guaranteed to find the model with the largest R
for each size.
This method begins by finding the one-variable model that produces the highest R
another variable, the one that yields the greatest increase in R
, is added. After the two-
variable model is obtained, each variable in the model is compared to each variable not in
the model. For each comparison, this method determines whether removing one variable
and replacing it with the other variable increases R
. After comparing all possible switches,
this method makes the switch that produces the largest increase in R
. Comparisons begin
again, and the process continues until this method finds that no further switch could
. Thus, the resulting two-variable model is considered the "best" two-variable
model that the method can find. Another variable is then added to the model, and the
comparing-and-switching process is repeated to find the "best" three-variable model, and
The difference between the stepwise selection method and the maximum R
method is that in the maximum R
method, all switches are evaluated before any switch is
made. In the stepwise selection method, the "worst" variable might be removed without
considering what adding the "best" remaining variable might accomplish.
Linear Regression Task