Classifiers

MUMT621

Introduction

• Preprocessing
  • e.g.: segmentation (by bar, by salami slice), windowing
• Feature extraction
  • low-level: e.g.: FFT, MFCC (mel-frequency cepstrum coefficients)
  • higher-level: e.g.: centroid, area, mean
• Feature selection, which is a special case of feature weighting
• Classification
  • Training: ground-truth (separated into: training, validation, and testing datasets)
  • Validation
    • holdout method (save subset for testing)
    • k-fold cross validation and nested cross vaidation (pdf), leave-one-out
    • Bootstrapping: resampling with replacement
  • Ensemble training
    • Bagging (Boostrsap Aggregating): Parallel training
    • Boosting: Sequential training (favour training with wrongly classified samples)

Classifiers (supervised): use labelled training data

• Bayes classifier
• Deceision Tree
• Support Vector Machines
• Hidden Markov Models
• Ensemble learning
  • Random Forest (bagging)
  • Gradient Boosting Machines
• Non-parametric density estimation (distribution-free)
  • k-nearest neighbour (non-greedy, lazy)
  • Neural networks (greedy)

Clustering (unsupervised): use unlablled training data

• Hierachical methods
• k-means (demo)
  • Gaussian mixture
• Self Organizing Maps (demo)

Resources

• General
  • Pattern Recognition (Toussaint)
  • Weka (Docs) (Python version)
  • Michie, D., D, J, Spiegelhalter, and C. C. Taylor. 1994. Machine Learning, Neural and Statistical Classification.
  • Statistical Data Mining Tutorials (Andrew Moore)
  • scikit-learn (Python) Tutorials
  • TensorFlow (Google) Tutorials
  • PyTorch (Meta)
  • Kaggle: 27 Best Resources to Study Machine Learning
• Bayes classifiers
  • Naive Bayes Classifier
• Support Vector Machine
  • Introduction to Support Vector Machines
  • Support Vector Machine Light (source code)
  • SVM Tutorial (Kowalczyk)
  • SVM video
  • SVM / Kernel video
• Neural networks
  • Neural Networks and Deep Learning (Nielsen)
  • FAQ
  • A Beginner's Guide to Neural Networks and Deep Learning
  • AI and Deep Learning in 2017: A Year in Review
  • A two-layer demo
  • Backpropagation neural network video
  • Gradient decent video
• Hidden Markov Model
  • Tutorial (Kanungo)
  • Tutotrial slides (Moore)
  • HTK software
• Ensemble Learning
  • Random Forest (Sruthi)
  • XGBoost (Brownlee)
  • LightGBM (Brownlee)
  • CatBoost (Brownlee)
  • Gradient Boosted Tree video
• Cross validation
  • A Gentle Introduction to k-fold Cross-Validation (Brownlee)
  • Nested Cross-Validation for Machine Learning with Python (Brownlee)
• Genetic Algorithms (for optimization)
  • “Was Darwin a Great Computer Scientist?”(Nicolle)
  • “How evolution taught us the genetic algorithm ?” (Barbier) [same as above]
  • GA Tutotrial (tutorialspoint)
  • PCAI
  • Genetic Cars 2