Read article here, or Download it as Word doc.

How Accurate are Measurements in Metron?

Introduction

It will always be true (with any system) that the more care you take in getting good images, the better the results are likely to be.  The same is true with the Metron system.  With any given scheme, it is important to ask if the system is “well-conditioned”.    If relatively small errors made in taking the image, or in picking key points in the image lead to relatively large errors in the results, then we have a poorly-conditioned scheme.  It is our belief (and we’ll back it up with some math below) that the Metron system of analyzing images  is a well-conditioned one.  For example, in one case we’ll present below, a 33% error made in data entry only causes a 4% error in a computed value.  This is a good situation, and means that if the images are taken with some care, you will get good results.  Probably to get the last 5% of accuracy out of the system, you need to take images with “great care”, but even this is probably possible.

The biggest source of errors probably comes from improper alignment of the camera and the hoof, or in the case of radiographs, the relative alignment of the X-ray machine, leg, and film.  Metron always analyzes a 2-D image which is a projection of some 3-D reality.  The concept is based on making sure that the 2-D projection is one taken along a certain axis – generally perpendicular to some aspect of the foot, or perpendicular to the “bone column”.  By “bone column” we mean, somewhat abstractly, the approximate cylinder of bone that constitutes the leg of the horse.  Actually, use of Metron itself can make this definition more precise, but roughly speaking, people seem to understand what it means to try to position the pointing direction of the X-ray machine “perpendicular to the bone column”.

Another general thing to say about accuracy of values computed by Metron has to do with relative versus absolute accuracy.  It is very often the case that what is important is a comparison between sets of parameters.  For example, we are often more interested in how the hoof angle changes over a few months, than on the absolute numerical value of the hoof angle.  This means that as long as each user of Metron follows repeatable methods for taking images and picking the markers in Metron, they can get very good results for use in comparisons.   That is, relative accuracy is achievable as long as repeatable methods are used by the user of the Metron system.

If  it is desired to compare values obtained by one user with values obtained from another user, then it becomes more important that both users do exactly the same things, make the same assumptions, etc.  That is, we need to ask to what extent the Metron system possesses absolute accuracy.   We have attempted to carefully design the “markers” that users are asked to pick so that they are “well defined” and that hopefully various users would accomplish very similar picks.  It is for this reason that some features which may indeed be of high interest are nonetheless not included in the standard markers and parameters because we felt that reliably picking the markers could not be well-defined (e.g. the absolute rear-support provided by the back of the frog in a solar image is hard to determine – so we pick only the rear heel support points).

These other items which may be of great importance, and yet are left out of our standard markers and parameters should be measured by interested parties using the “free mark-up” feature of Metron.

It should be understood that although this paper is focused on numerical accuracy of measurements made in images using the Metron system, that is not to say that we believe that all things of importance to the health and function of the equine hoof can be so measured.   Indeed, conformation of the hoof and bones is not related one-to-one in a simple way with lameness or performance.   Lame horses can have great looking photos and radiographs, and horses with horrible looking photos and radiographs can nevertheless be sound.  But, these are probably special cases.  We do believe that there is a correlation between conformation and performance.

In the following sections, we look at a few different sources of error that can influence the values computed by Metron.

The Solar Photo

Error Due to Resolution of the Digital Image

Let’s consider a typical example.   Lets say that a digital camera is used to take a photo that is 640X480 pixels.  This is actually quite a low resolution, and easily available with inexpensive digital cameras.  If the image is well-framed in the field of view, then the image of the horse’s foot will take up about 75% of the image.  That is to say,  if a horse’s foot (viewed from the sole) is about 6 inches long and across, then the dimensions of the field of view of the camera would be about 8 inches by 10 inches.  With 640x480 pixels this gives us approximately 60 pixels per inch.   Roughly speaking, this means that any pick made on the image later (in Metron) could be in error by 1/60 – th of an inch, which is approximately 0.0167 inch. Note that with state of the art digital cameras, it is perfectly feasible to take an image which is perhaps 1600 X 1200 pixels, which translates to about 160 pixels per inch, which would lower the potential error due to resolution to 1/160 which is 0.00625 inch.  Note that an image from a conventional (film) camera can be scanned with a flatbed scanner at a resolution of 300 dpi with no problems, which would actually yield even a smaller error due to digital resolution (but potentially could introduce other effects).

Error Due to Camera Misalignment

In taking the Solar Photo, one should try to align the camera so that the “pointing direction” of the lens is perpendicular to the plane of the underside of the foot.   Most all of the features we are interested in do lie approximately in a plane (or vary in and out of the plane by a half an inch or less).  For the moment, lets consider that all features of interest lie exactly in a plane.  Now, whereas the angle this plane makes with the pointing direction of the lens should be 90 degrees, lets say that instead it is “q”.  If the distance from the camera to the foot is “D”, and if the length of the foot is “L”,  then an expression for the maximum error per linear inch of measurement in the image is:

E = ( L Cos(q) ) / ( D + L/2 Cos(q) )

Qualitatively, we see that errors are bigger the more q deviates from 90 degrees; errors are smaller when the camera is distant (D is large) from the foot, and we see that errors are potentially bigger the bigger the size of the foot.   So, its best to be far back from the foot (using the maximum zoom available in the camera lens) and to try to keep the lens pointing direction perpendicular to the foot.

Lets put some reasonable values into the equation above to see what this means.  Lets say the camera is about 30 inches away from the foot ( D = 30.0 ).  The foot is about 6 inches in length ( L = 6.0 ).   In pointing the camera, the photographer is off by 10 degrees ( q = 80.0 ).   Finally, lets say we are attempting to measure a feature whose length is 2 inches.  With these values, the error comes out to about 0.068 inch.                                                         

Note that our use of the location of the “bulbs” in the Solar Photo is subject to greater errors because the bulbs lie significantly out of the plane of the foot.  That is to say, if the camera is not pointed exactly perpendicular to the plane of the foot, the measurements concerning the bulbs will suffer most in accuracy.

Error Due to Combinations of Points to Compute one Parameter

In Metron, several picks are sometimes combined to compute a single parameter value.  A complete analysis would require examination of each of the parameters and how they are computed.   Instead, lets just consider one case which may be one of the worst case examples.  Consider computing the angle between two line segments, each about 3 inches long, where each of the end-points was picked and subject to the resolution errors mentioned above for the 640X480 pixel image.  This situation is similar to what happens in computing some of the parameters in the Solar Photo in Metron. 

If each of the four picks was picked with one pixel error in exactly the wrong direction,  then each of the line segments could be off by an agle given by the arcsin of 1/60 over 1.5.  For the case mentioned, the angle measured between the two line segments could be in error by 1.28 degrees.

The Lateral and Frontal Photos

The analysis is largely the same as the case of the Solar Photo.   In these photos, we desire the pointing direction of the lens to be perpendicular to the foot, and to be aimed at a point just at the top of the block the horse is standing on. 

 

Radiographs in Metron-PX

In radiographs,  there is a magnification factor which depends upon the “Film Focal Distance” (FFD) and the “Object Film Distance” (OFD).    The scale factor, or magnification, due to a point-source of radiation and these two distances is given by:

           S = FFD / ( FFD – OFD )

Standard practice is to place the film as close as possible to the limb, for example, for a lateral digit radiograph,  OFD is probably less than 3 inches.  Most veterinarians use a set value for FFD (the distance from the X-Ray machine to the film) which is often specified by the manufacturer.  Typical values might range from 20 inches to 40 inches.

Let's use some reasonable values to compute how much of an error is introduced when these values are inaccurately specified to Metron.  Lets say that the true values used to take the radiograph were OFD = 3 inches, and FFD = 24 inches.  But suppose the Metron user didn’t know the value for FFD, and entered

FFD’ = 32 inches, a value which is 33% bigger than the true value.  In this case, using the formula above, the true scale factor is S = 24/(24-3) = 1.1429, but due to the misinformation given to Metron, a scale value of S’ = 32/(32-3) = 1.1034 would be used.   For a feature measurement which is actually 2 inches long, the error in length would be 2(1.1429 – 1.1034) = 0.079 inch, which is an error of less than 4%.

Hence, in this example, an error of 33% in knowledge of the parameter FFD results in less than 4% error in a measurement derived from the image.  This shows that measurements are not very “sensitive” to errors in these parameters – if you have even approximate values for FFD and OFD, then Metron results will be pretty accurate.

In fact, it’s a simple matter to try an experiment in Metron to check this.  For example, with a radiograph taken with OFD = 2.5 inches and with FFD = 32 inches, we compute a “P3 Dist. to Toe” of 1.27 inches.

This is an important dimension that the farrier may use in trimming the toe.  How much error is in this number if the FFD value was not correct?  By going back to the calibration panel in Metron and entering alternate values for FFD, and then returning to the parameters panel, we can see exactly how this value changes as we change FFD.  In our example, with FFD changed to 24 inches, the parameter value changes to 1.24 inches; and with FFD changed to 44 inches, the parameter changes to 1.30 inches.  Hence, we see that this important parameter value changes by less than 3% even when errors in knowledge of FFD are quite large.

Other error sources for radiographs in Metron are similar to effects discussed above for photos: pixel resolution and alignment issues. 

Repeatability of the Method

It can be difficult to achieve consistency in radiographs of live horses for a number of reasons.  In addition, a user of our method introduces an additional source of non-repeatability when picking the key points in the radiograph.

By repeatability we mean the ability of the technique to derive the same parameter values from different radiographs of the same horse, as well as the ability for different practitioners to derive the same parameter values from the same radiographic image.  Hence two tests were performed: in the first test, the technique was used by a single practitioner to measure a certain biomechanical parameter in eight radiographs of the same equine digit, taken over several years by various independent radiographers.   Variation in the measured parameter indicate that we can derive highly repeatable parameter values in such a situation.  In the second test, the same radiographic image was analyzed using our technique by thirteen different practitioners working independently.  Variation in measured parameters indicate good repeatability of results in this situation.

In the first test, we estimate the repeatability of our method for length measurements as follows.  Eight radiographs of the right fore digit of the same horse, taken over a ten year period (1991 – 2001), were parameterized using our technique.   The horse’s age was 7 in 1991, and so could be considered fully mature when the first radiograph in the study was taken.   The 8 radiographs were taken by 4 different practitioners using different equipment.  We chose a particular parameter (the “Length of P2” as shown in figure 4) which should be a constant value in a mature horse.  This particular parameter was chosen for this test because it is not affected by how the hoof is trimmed, nor by the horse’s stance at the moment the radiograph is taken.   The variations observed in the measured values give an estimate of the repeatability of our method.   The mean value measured was 4.96 cm with a minimum value of 4.88 cm and a maximum of 5.10 cm.  Hence, all values are within 1.46 mm (or 2.94%) of the mean.  We could also say that by these repeated measurements, we have determined the 95% confidence interval bounds on the mean value to be +/- 0.76 mm.

In the second test, we asked thirteen practitioners to make measurements following our technique in the same image.  Since the same image was used by all practitioners, this test was designed to measure the repeatability of the technique as regards variations introduced by the practitioner following the procedure.   For the purposes of this test, we looked at the repeatability of the parameters shown in figure 3, namely the coffin joint angle and the pastern joint angle.  The mean value determined for the coffin joint angle was 14.70 degrees, with a minimum of 13.23 degrees and a maximum of 16.02 degrees.  The mean value found for the pastern joint angle was 7.95 degrees, with a minimum of 6.45 degrees and a maximum of 9.90 degrees.  Hence, all measured coffin-joint values were within 1.5 degrees of their means, and all measured pastern-joint angles were within 2.0 degree of their means.  Combining all thirteen measurements results in a 95% confidence interval of  +/- 0.50 degrees for the coffin-joint, and +/- 0.68 degrees for the pastern-joint.

Summary

It is impossible to give a simple, concrete answer for the question “How Accurate is Metron?”.  Rather, each parameter computed for each type of image would have to be considered carefully to say exactly the accuracy for that particular parameter.   However, as we have shown above, it appears that the errors introduced from several common causes are not large.  Hence we feel that the results from Metron are quite accurate if care is taken in acquiring the images.