Understanding Machine Learning: From Theory to Algorithms
10.5 Summary 141 Figure5:ThefirstandsecondfeaturesselectedbyAdaBoost.Thetwofeaturesareshowninthetoprow andthenoverlayedonatypica ...
142 Boosting His weakly learnable usingB. For example, Klivans & Sherstov (2006) have shown that PAC learning of the class o ...
10.7 Exercises 143 Show that the error ofhtw.r.t. the distributionD(t+1)is exactly 1/2. That is, show that for everyt∈[T] ∑m i=1 ...
11 Model Selection and Validation In the previous chapter we have described the AdaBoost algorithm and have shown how the parame ...
11.1 Model Selection Using SRM 145 In this chapter we will present two approaches for model selection. The first approach is bas ...
146 Model Selection and Validation and a complexity term that depends ond. The SRM rule will search fordand h∈Hdthat minimize th ...
11.2 Validation 147 with respect to a hypothesis class of VC-dimensiond, over a training set ofm examples. Then, from the fundam ...
148 Model Selection and Validation This theorem tells us that the error on the validation set approximates the true error as lon ...
11.2 Validation 149 2 4 6 8 10 0 0. 1 0. 2 0. 3 0. 4 d error train validation As can be shown, the training error is monotonical ...
150 Model Selection and Validation estimate of the true error. The special casek=m, wheremis the number of examples, is calledle ...
11.3 What to Do If Learning Fails 151 11.3 What to Do If Learning Fails Consider the following scenario: You were given a learni ...
152 Model Selection and Validation Instead, we give a different error decomposition, one that can be estimated from the train an ...
11.3 What to Do If Learning Fails 153 m error train error validation error m error train error validation error Figure 11.1Examp ...
154 Model Selection and Validation validation error is starting to decrease then the best solution is to increase the number of ...
11.5 Exercises 155 Divide themexamples into a training set of size (1−α)mand a validation set of sizeαm, for someα∈(0,1). Then, ...
12 Convex Learning Problems In this chapter we introduceconvex learning problems. Convex learning comprises an important family ...
12.1 Convexity, Lipschitzness, and Smoothness 157 non-convex convex Givenα∈[0,1], the combination,αu+ (1−α)vof the pointsu,vis c ...
158 Convex Learning Problems x f(x) An important property of convex functions is that every local minimum of the function is als ...
12.1 Convexity, Lipschitzness, and Smoothness 159 f(w) f(u) w u f( w) + 〈u −w ,∇ f(w )〉 Iffis a scalar differentiable function, ...
160 Convex Learning Problems Given somex ∈Rdandy ∈R, letf :Rd →Rbe defined asf(w) = (〈w,x〉−y)^2. Then,fis a composition of the ...
«
3
4
5
6
7
8
9
10
11
12
»
Free download pdf