Chapter 2 - Note

Introduction to machine learning - third edition, Ethem Alpaydin

 

Chapter 2. 

p. 26

In the problem classifying training points to two classes(1,-1), we train the ML model. 

Hypothesis class , H includes all of the possible fitted models. It contains all of the possible models that are solutions of ML problem

The most specific model that correctly classifies the data is called S, while the most general model that correctly classifies the data is called G. The model that has E(h|X)=0 is called C. We need to make sure that H is flexible enough to fit the model until it learns C. 

 

p.28

VC(Vapnik-Chervonenkis) dimsension: maximum number of points that can be shattered by H. 

Although rectangle in 2D cannot separate any four points in 2D space, in application perspective, points are mapped in the space restrictively. So low VC dimension model can also be a good model.