Classifiers

MUMT621

Introduction

• Preprocessing
  • e.g.: segmentation, windowing, FFT, MFCC (mel-frequency cepstrum coefficients)
• Feature extraction
  • e.g.: centroid, area, mean
• Feature selection (feature weighting)
• Classification
  • Training: ground-truth (separated into: training, validation, and testing datasets)
  • Validation
    • holdout method
    • k-fold cross-validation, leave-one-out (pdf)
    • Bootstrapping: resampling with replacement
  • Ensemble training
    • Bagging (Boostrsap Aggregating): Parallel training
    • Boosting: Sequential training (favour training withwrongly classified samples)

Classifiers (supervised)

• Bayes classifier
• Support Vector Machines
• Boosting algorithms
• Hidden Markov models
• Non-parametric density estimation (distribution-free)
  • k-nearest neighbour (non-greedy, lazy)
  • Neural networks (greedy)

Clustering (unsupervised)

• Hierachical methods
• k-means (demo)
  • Gaussian mixture
• Self Organizing Maps (demo)

Resources

• General
  • Pattern Recognition (Toussaint)
  • Weka (readme)
  • Michie, D., Spiegelhalter, D.J. and Taylor, C.C. 1994. Machine Learning, Neural and Statistical Classification. (online book)
  • Statistical Data Mining Tutorials (Andrew Moore)
• Bayes classifiers
  • Naive Bayes Classifier
• Neural networks
  • Neural Networks (Stergou & Signos)
  • Neural Networks and Deep Learning (Nielsen)
  • Introduction (StatSoft)
  • FAQ
  • AI and Deep Learning in 2017: A Year in Review
• Support Vector Machine
  • SVM Tutorial (Ghose)
  • Introduction to Support Vector Machines
  • SVM Tutorials
  • Support Vector Machine Light (source code)
  • SVM applets
  • SVM Tutorial (Kowalczyk)
• Hidden Markov Model
  • Tutorial (Kanungo)
  • Tutotrial slides (Moore)
  • Links
  • HTK software
• Genetic Algorithms (for optimization)
  • "Was Darwin a Great Computer Scientist?" (Nicolle)
  • GA Tutotrial (tutorialspoint)
  • PCAI
  • BoxCar 2D
  • Genetic Cars 2