The specific goal of this project was to locate the pixels belonging to red
score lines in digitized colour photographs of Medieval and Renaissance scores.
This software was designed in order to have more general applicability, however,
and is able to potentially detect horizontal lines of any given colour and height
within a digital image.
A prototype with a full GUI was implemented in Java. The techniques developed
here are intended to eventually be integrated into the Gamera
software framework.
Functionality
The software is able to perform the following tasks:
Load image files encoded in any standard format.
Save image files in JPEG format.
Rotate images a specified amount in order to make images that were improperly
scanned at an angle take on an appropriate orientation.
Filter an image in order to select only those pixels which meet the following
criteria:
Have absolute red, green and blue channel values within specified ranges.
Have the red, green and blue channels each making up fractions within
specified ranges of the total content of each pixel.
Have a brightness falling within a specified range.
Have a vertical run length with other pixels meeting one or more of
the above criteria that is below a specified threshold.
Have a horizontal run length with other pixels meeting one or more of
the above criteria that is above a specified threshold.
Take those pixels meeting one or more of the above criteria and apply a
blur filter with surrounding pixels that do not meet the above criteria in
order to eliminate the lines from the image.
The user can manually select how many and which of the above methods to use
in order to select candidate pixels that are likely to belong to a horizontal
line of the desired type. The user may also set the specific thresholds for
each of these methods. Alternatively, the user may use the hard-coded presets
for detecting red score lines.
Screen Shots
Algorithm
The following algorithm was used to select pixels as candidates for belonging
to horizontal lines of the specified colour:
Obtain a bitmap of the image, with a value for each of the red, green and
blue channels for each pixel.
Create a 2-D boolean array named selected with each entry corresponding
to a pixel in the bitmap. Each entry indicates whether a pixel has passed
all filters so far. Set all entries to true as the default.
If the absolute RGB value filtering strategy is selected for use, then set
all entries of selected that correspond to pixels that do not fall
within the specified value ranges for the red, green and blue channels to
false.
If the fractional RGB value filtering strategy is selected for use, then
set all entries of selected that correspond to pixels that do not
have the fractions of total colour content for the red, green and blue channels
within the specified ranges to false.
If the brightness filtering strategy is selected for use, then set all entries
of selected that correspond to pixels that do not have brightnesses
(average of the three RGB channels) within the specified range to false.
If the maximum vertical run length filtering strategy is selected for use,
then set all entries of selected that correspond to pixels that are
iteratively adjacent in the direct vertical dimension (i.e. have a vertical
run length) to more than the set threshold of pixels that are marked as true
in selected to false.
If the minimum horizontal run length filtering strategy is selected for
use, then set all entries of selected that correspond to pixels that
are iteratively adjacent in the direct horizontal dimension (i.e. have a horizontal
run length) to less than the set threshold of pixels that are marked as true
in selected to false.
Any pixels that correspond to entries that are still marked as true in selected
are considered as belonging to a horizontal line with the specified brightness/colour
properties.
Once this task has been completed, the horizontal lines with the specified
brightness/colour properties can be removed from the image using the following
algorithm:
Take in a 2-D boolean array named selected with each entry corresponding
to a pixel in a corresponding bitmap. Each entry of selected is marked
as true if the corresponding pixel is considered to belong to a horizontal
line with the specified brightness/colour properties, and as false otherwise.
Go through all pixels one by one. If a pixel has a corresponding entry of
true in selected:
If the pixel has one or more directly surrounding pixels marked false
in selected, then replace each of its RGB channel values with
the averages of the values of the directly surrounding pixels that have
corresponding entries of false in selected. Change the value
of the corresponding entry of the newly replaced pixel in selected
to false.
If none of the directly surrounding pixels have corresponding entries
of false in selected, then skip the pixel for now.
Repeat until no more entries in selected are marked true.
Results
The following examples show the results of the first algorithm detailed above
on excerpts from three different scores. The default thresholds for red score
line detection were used in all processing, with no tailouring. The areas that
are marked with bright green indicate where the software detected the score
lines as being.
Example 1a: original
Example 1b: after standard red staff line detection
Example 2a: original
Example 2b: after standard red staff line detection
Example 3a: original
Example 3b: after standard red staff line detection
Example 3c: after automated red staff line removal
It can be seen from the above examples that the staff line detection worked
very well in Examples 1 and 3, but poorly in Example 2. These types of results
were typical of a number of other scores that were processed, and demonstrate
the generally good performance with occasional failures that was obtained. The
performance is highly dependant on the pre-set thresholds, of course. Those
willing to accept some manual intervention to tailour thresholds for problematic
scores such as Example 2 will be able to drastically improve results, but such
tweaking is undesirable in an automatic system. The system as it is is therefore
successful as an automatic system in many but not all cases, and successful
as a manual system in almost all cases.
The last graphic for Example 3 shows that there was some success with removing
the score lines without degrading the rest of the score, but there is certainly
room for improvement. The problems are largely with the blur filter used, and
not with score line detection.
Possible Improvements
The horizontal line detection system could potentially be improved in the following
ways:
Implement a contrast filter.
Implement further gradient techniques related to local changes.
Automatically connect horizontal line segments with similar vertical coordinates
that are separated by a short distance.
Statistically study the colour properties of the red lines in different
scores in order to improve the default thresholds.
Use histograms for each of the RGB channels to automatically modify thresholds.
The line removal system following line detection could potentially be improved
in the following ways:
Increase the size of the blur filter. In particular, make it taller.
Incorporate a bias against reddish pixels in the averaging, even if they
were not selected as part of a line.
Downloads
The software is available in a self-contained runnable Jar file here.
The source code and JavaDocs are available in a zipped file here.
