Overview

This panel allows users to view and edit preference settings. This panel can be seen in Figure 1.

Training Preferences

• Fraction of Recordings to use for Training: The fraction of candidate recordings to use for testing during Randomly Partitioned Recs training or cross-validation. Must be greater than 0 and less than 1.
• Train With Overide Features: Whether training should be performed with the features marked as Selection Candidates in the Features Panel as candidates for feature selection (unchecked) or with the features marked as having Overide Status as candidates for feature selection (checked).
• Train With One-D Features: Whether training is to be performed with classifier ensembles that include a KNN classifier for all selected one-dimensional features.
• Train With Multi-D Features: Whether training is to be performed with classifier ensembles that include a neural net classifier for each selected multi-dimensional feature.
• Find 1-D Features Selection: Whether or not feature selection should be performed on the one-dimensional features, if any, during training. A value of unchecked means that all one-dimensional features will automatically be selected.
• Find 1-D Feature Weightings: Whether or not feature weighting should be found for the selected one-dimensional features, if any, during training. A value of unchecked means that all one-dimensional features are given equal weighting.
• Find Classifier Selection: Whether or not classifier selection should be performed on all classifiers in classifier ensembles during training. A value of unchecked means that all available classifiers will automatically be selected.
• Find Classifier Weightings: Whether or not classifier weighting should be performed on all selected classifiers in classifier ensembles during training. A value of unchecked means that half of the weighting is given to the KNN classifier and the remaining weighting is divided among all available neural networks.
• Selection/Weighting Training Fraction: The fraction of training samples that will actually be used for training when feature selection and weighting are calculated for one-dimensional features. The remaining recordings are used for testing. Only has an effect of the feature selection/weighting process, and nothing else. This value must be between 0 and 1.
• Classify Into Root Categories: Whether or not a classifier ensemble should be trained that classifies all recordings into none or more root categories. Note that this option should not be enabled if one wishes to classify recordings using classifiers that were not trained with this option, but it may be disabled even if the classifiers were trained with this option.
• Classify Into Direct Descendants of Parents: Whether or not hierarchal classifier ensembles should be trained, which is to say an ensemble for each category with children that classifies recordings into none or more of its children. Note that this option cannot be enabled if one wishes to classify recordings using classifiers that were not trained with this option, but it may be disabled even if the classifiers were trained with this option.
• Classify Into All Leaf Categories: Whether or not a classifier ensemble should be trained that classifies all recordings into none or more leaf categories. Note that this option cannot be enabled if one wishes to classify recordings using classifiers that were not trained with this option, but it may be disabled even if the classifiers were trained with this option.
• Classify Using Round Robin Ensembles: Whether or not round robins classifier ensembles should be trained, which is to say an ensemble for every possible pair of leaf categories that classifies recordings into none or more of the two categories. Note that this option cannot be enabled if one wishes to classify recordings using classifiers that were not trained with this option, but it may be disabled even if the classifiers were trained with this option.

Training Termination Preferences

• Auto End GA Training: Whether or not genetic algorithm training should be terminated early (i.e. before the Max Number GA Epochs has been reached) if the best fitness has not changed beyond a certain point for more than a certain number of epochs.
• Max Number GA Epochs: The maximum number of epochs that should be iterated through for feature (but not classifier) selections and weightings. Must be 1 or greater.
• Classifier vs. Feature Iter. Multiplier: The multiplier applied to the Max Number GA Epochs in order to get the corresponding value for classifier selections and weightings. Is also used to multiply the population. Must be 1 or greater.
• Max Change in GA Error for Training Abort: The maximum acceptable change in fitness that will make it possible to end genetic algorithm evolution before the Max Number GA Epochs has been reached. Must be greater than 0.
• Min Consec GA Epochs for Training Abort: The minimum number of epochs that must go by in genetic algorithm evolution where the fitness is below the Max Change in GA Error for Training Abort in order for training to end before the Max Number GA Epochs is reached. Must be 0 or greater.
• Auto End NN Training: Whether or not neural network training should be terminated early (i.e. before the Max Number NN Iterations have been reached) if the sum of squares error has not changed beyond a certain point for more than a certain number of epochs.
• Max Number NN Iterations: The maximum number of neural network epochs that should be iterated through. Must be 1 or greater.
• Max Change in NN Error for Training Abort: The maximum acceptable change in sum of squares error that will make it possible to end neural network training before the Max Number NN Iterations has been reached. Must be greater than 0.
• Min Consec NN Iters for Training Abort: The minimum number of epochs that must go by in neural network training where the sum of squares error is below the Max Change in NN Error for Training Abort in order for training to end before the Max Number NN Iterations is reached. Must be 0 or greater.

Classification Preferences

• Hierarchal Relative Weighting: The relative weighting to be given to the hierarchal classifier ensembles when the results of multiple classifier ensembles are combined. This weighting is relative to the All Leaf Category Relative Weighting and Round Robin Relative Weightings weightings. Must be 0 or greater.
• All Leaf Category Relative Weighting: The relative weighting to be given to the flat all leaf classifier ensemble when the results of multiple classifier ensembles are combined. This weighting is relative to the Hierarchal Relative Weighting andRound Robin Relative Weightings weightings. Must be 0 or greater.
• Round Robin Relative Weightings: The relative weighting to be given to the hierarchal classifier ensembles when the results of multiple classifier ensembles are combined. This weighting is relative to the All Leaf Category Relative Weighting and Hierarchal Relative Weighting weightings. Must be 0 or greater.
• Round Robin Score Divisor: The divisor that is used to divide all round robin scores during classification. A higher number makes it less likely that the round robin classifiers will select too many categories. Must be 1 or greater.
• Should Use Winner Only For Round Robin: Whether or not all but the highest round robin score should be set to 0 for each recording.
• Min Score for Auto 1st Choice: The minimum score that a category must have to automatically be
  counted as a first choice during classification, regardless of the scores of other categories. Must be between 0 and 1.
• Min Score to be 1st Choice: The minimum score that a recording must have to qualify as a first choice during classification. Must be between 0 and 1.
• % Below Highest to be 1st Choice: The percentage below the score of the highest category that sets the minimum for a category in order for it to be classified as a first choice (unless it has a score over Min Score for Auto 1st Choice). Must be between 0 and 100.
• Min Score to be 2nd Choice: The minimum score that a recording must have to qualify as a second choice during classification. Must be between 0 and 1.
• % Below Highest to be 2nd Choice: The percentage below the score of the highest category that sets the minimum for a category in order for it to be classified as a second choice. Must be between 0 and 100.
• Only Choose One Winner in Combined Classification: Whether or not only one winning category is permitted per recording

Reporting Preferences

• Seconds Between Progress Bar Polls: How often, in seconds, that the training progress bar (see the Classification Panel) updates its status. Must be 0 or greater.
• Should Display Training Times in Raw Seconds: Whether training times should be displayed as seconds (checked) or as hours, minutes and seconds, as appropriate (unchecked).
• Report Classifier Settings: Whether or not classifier settings should be included in classification reports.
• Report Ensemble Success Rates: Whether or not classification reports should include overall classification statistics for the combined and individual classifiers for each ensemble of classifiers. This option is only available when recordings have been reserved for testing during training (see the Classification Panel).
• Report Root Category Success Rates: Whether or not classification reports should include how often the root categories of recordings were successfully found. This option is only available when recordings have been reserved for testing during training (see the Classification Panel).
• Report Confusion Matrix: Whether or not classification reports should include a confusion matrix. This option is only available when recordings have been reserved for testing during training (see the Classification Panel).
• Report Ensemble Eligible Categories: Whether or not classification reports should include a list of the eligible categories into which recordings could be classified by each ensemble of classifiers.
• Report Combined Ensemble Classifications: Whether or not classification reports should include the combined classification results (i.e. name of final categories) for each recording by each ensemble of classifiers.
• Report Individual Classifier Classifications: Whether or not classification reports should include separately, for each classifier, the classification results (i.e. name of final categories) for each recording for an ensemble of classifiers.
• Include Classification Scores w/ Category Names: Whether or not classification reports should include the associated scores along with the names of the winning category(ies) for each recording.
• Include Secondary Classification Results: Whether or not secondary classifications should be included in classification reports alongside primary classifications.
• Include Model Results With Classifications: Whether or not classification reports should include the names of model classifications, if available. This option is only available when recordings have been reserved for testing during training (see the Classification Panel).
• Report Combined Ensemble Category Scores: Whether or not classification reports should include the combined classification scores for each category of each recording for each ensemble of classifiers.
• Report Individual Ensemble Category Scores: Whether or not classification reports should include, separately for each classifier, the classification scores for each category of each recording for each ensemble of classifiers.
• Report Model Results in Separate Table: Whether or not classification reports should include, as a separate table, a list of the eligible classification categories for each ensemble of classifiers, along with their code numbers.
• Lines to Skip When Reporting Training Errors: The number of epochs that are skipped between reportings of fitness or sum of squares error in training reports. Must be 0 or greater.
• Report Recordings Used to Train Ensemble: Whether or not training reports should include the names and model leaf categories of the recordings used in training.
• Report Ensemble 1-D Feature Training Error: Whether or not training reports should include the one-dimensional feature selection and/or weighting training progress for ensembles of classifiers.
• Report Ensemble Multi-D Training Error: Whether or not training reports should include training progress for the multi-dimensional classifiers in ensemble of classifiers.
• Report Ensemble Classifier Selection Error: Whether or not training reports should include classifier selection training progress in ensembles of classifiers.
• Report Ensemble Classifier Weighting Error: Whether or not training reports should include classifier weighting training progress for ensembles of classifiers.
• Report Ensemble 1-D Features Selected: Whether or not selection/weighting reports should include the one-dimensional features selected for each ensemble of classifiers.
• Report Ensemble 1-D Feature Weightings: Whether or not selection/weighting reports should include the one-dimensional feature weightings for each ensemble of classifiers.
• Report Ensemble Classifiers Selected: Whether or not selection/weighting reports should include the classifiers selected by each ensemble of classifiers.
• Report Ensemble Classifier Weightings: Whether or not selection/weighting reports should include the classifier weightings for each ensemble of classifiers.

Buttons

• Neural Net Preferences: Brings up the Feedforward Neural Network Initial Settings Dialog Box.
• Genetic Algorithm Preferences: Brings up the Genetic Algorithm Initial Settings Diaolog Box.
• Load Preferences: Loads the preferences contained in a preferences_file XML file on disk into memory and displays them. Allows the user to choose the file using a file chooser dialog box.
• Save Preferences: Saves the current preferences into a preferences_file XML file on disk. If a path is given in the Preferences file field of the Configure File Locations Dialog Box, then automatically saves the file to that path, overwriting any existing file. If this field is blank, then allows the user to choose the file location using a file chooser dialog box.
• Save Preferences As: Saves the current preferences into a preferences_file XML file on disk. Allows the user to choose the file location using a file chooser dialog box.

Status Bar

The status bar has no function on this panel.

Warnings

• Training preferences only affect behaviour at beginning of training, and subsequent changes will have no effect on classifications performed using existing trained classifiers. Exceptions are the Classify Into Root Categories, Classify Into Direct Descendants of Parents, Classify Into All Leaf Categories and Classify Using Round Robin Ensembles preferences, which play a roll during classification as well as training. See the descriptions of these preferences above for details.

Screen Shots

Figure 1: Preferences panel.