Report

Questions and Topics Review Nov. 30, 2010 1. Give an example of a problem that might benefit from feature creation 2. How does DENCLUE form clusters? Why does DENCLUE use grid-cells? What are the main differences between DENCLUE and DBSCAN? 3. Compute the Silhouette of the following clustering that consists of 2 clusters: {(0,0), (0,1), (2,2)} {(3,2), (3,3)}. 4. Compare Decision Trees, Support Vector Machines, and K-NN with respect to the number of decision boundary each approach uses! 5. K-NN is a lazy approach; what does it mean? What are the disadvantages of K-NN’s lazy approach? Do you see any advantages in using K-NN’s lazy approach. 6. Why do some support vector machine approaches map examples from a lower dimensional space to a higher dimensional space? 7. What is the role of slack variables in the Linear/SVM/Non-separable approach (textbook pages 266-270)—what do they measure? What properties of hyperplanes are maximized by the objective function f(w) (on page 268) in the approach? • Silhouette: For an individual point, i – Calculate a = average distance of i to the points in its cluster – Calculate b = min (average distance of i to points in another cluster) – The silhouette coefficient for a point is then given by: s = (b-a)/max(a,b) Christoph F. Eick Support Vector Machines • What if the problem is not linearly separable? Christoph F. Eick Linear SVM for Non-linearly Separable Problems What if the problem is not linearly separable? Parameter – Introduce slack variables 2 N || w || k Need to minimize: L( w) C i 2 i 1 Inverse size of margin between hyperplanes Slack variable Subject to (i=1,..,N): (1) ( 2) Measures testing error yi * (w x i b) 1 - i 0 i allows constraint violation to a certain degree C is chosen using a validation set trying to keep the margins wide while keeping the training error low. Tan, Steinbach, Kumar, Eick: NN-Classifiers and Support Vector Machines Questions and Topics Review Nov. 30, 2010 1. Discussion of Problem1/2of Assignment4 2. Give an example of a problem that might benefit from feature creation 3. How does DENCLUE form clusters? Why does DENCLUE use grid-cells? What are the main differences between DENCLUE and DBSCAN? 4. Compute the Silhouette of the following clustering that consists of 2 clusters: {(0,0), (0.1), (2,2)} {(3,2), (3,3)}. 6. Compare Decision Trees, Support Vector Machines, and K-NN with respect to the number of decision boundary each approach uses! DT: many, rectangular for numerical attributes K-NN: many, convex polygons (Voronoi cells), SVM: one, hyperplane 6. K-NN is a lazy approach; what does it mean? What are the disadvantages of K-NN’s lazy approach? Do you see any advantages in using K-NN’s lazy approach. … advantages: for quickly changing streaming data learning the model might be a waste of time and a lazy approach might be better… 7. Why do some support vector machine approaches map examples from a lower dimensional space to a higher dimensional space? To make them linearly separable. 7. What is the role of slack variables in the Linear/SVM/Non-separable approach (textbook pages 266-270)—what do they measure? What properties of hyperplanes are maximized by the objective function f(w) (on page 268) in the approach? Christoph F. Eick