| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
* University of Aarhus, Faculty of Agricultural Sciences, Research Centre Foulum, PO Box 50, 8830 Tjele, Denmark
Lattec I/S, Slangerupgade 69, 3400 Hillerød, Denmark
1 Corresponding author: n.friggens{at}agrsci.dk
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGEMENTSREFERENCES |
|---|
Key Words: mastitis • detection • time-series • dairy cow
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGEMENTSREFERENCES |
|---|
The system on which this study focuses is designed to use repeated measures of an indicator of mastitis; that is, rather than detecting mastitis from one isolated measurement, it claims to be able to track the development of mastitis through time. Implicit in this approach is the concept of a degree of mastitis, or risk of mastitis, that can evolve through time. Concepts have an intrinsic scientific value but when they are incorporated into a model that is designed for application, it is important to evaluate their worth in terms of the practical advantages they may provide the end-user. What are the advantages for mastitis detection of tracking the development of a continuous measure of mastitis through time? This question is difficult to address with specificity/sensitivity testing, in which it is assumed that mastitis state is a binary variable (healthy or sick), and the time dimension is reduced to an aggregate value within a fixed time-window. It is possible to overcome these constraints, at least in part, by performing a series of specificity/sensitivity calculations that track through a sequence of time-points or work through a series of different thresholds for classifying cows as sick or healthy; for example, using receiver operating characteristic methodology (Lasko et al., 2005; Gardner and Greiner, 2006). This approach, however, largely ignores the correlations inherent in the progression of a time-series of measurements. Further, the limited uptake of these methodologies indicates that they tend to produce results that are intuitively difficult for nonspecialists to interpret. Thus, the aim of this study was to test a dynamic model for mastitis detection using a rationale that allows explicit examination of the time-related changes and a progressive scale of predicted mastitis state.
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGEMENTSREFERENCES |
|---|
The model has many features that warrant testing, especially in relation to the ARF. In the present study we have chosen to focus on the overall risk of acute mastitis, because this is the major output and the one by which the usefulness of the model in commercial application will be primarily judged. For the purposes of testing, no constraints were placed on the values of overall risk of acute mastitis. Although the model was designed to produce values in the range from 0 and 1, values can be outside this range because, for example, a negative slope of LDH amount generates a negative component to the IBR.
Structure of Test Data
A test data set was created by a period of intense monitoring and milk sampling on one farm, the Danish Cattle Research Centre (Tjele, Denmark). All cows were milked with an automatic milking system (3 units, on average 2.3 ± 0.83 milkings/cow per d) in which the milk yield was automatically recorded. The data used were collected in the period between April 2003 and September 2006. The cows were of 3 breeds: Danish Holstein (n = 123), Danish Red (n = 130), and Jersey (n = 79), and were loose-housed in one barn. After data editing (see below), the total number of lactations in first, second, and greater parities were 246, 179, and 177, respectively. Proportional samples of composite milk (all quarters combined) were collected automatically from each cow, at each milking. The automatic milk sampling system was emptied in the morning and in the afternoon. Milk samples were kept at 4°C until laboratory analysis, which was done within 24 h of sampling.
Milk samples were analyzed for LDH and SCC. Samples were analyzed for LDH activity (µmol/min per L) using a fluorometer (Fluostar, BMG Labtechnologies, Offenburg, Germany) as described by Larsen (2005). Somatic cell count was measured using Fossomatic 5000 automatic equipment (Foss Electric, Hillerød, Denmark). Additionally, foremilk quarter samples were taken each time a veterinary treatment was initiated and cultured, and the resulting proportions of the different pathogen types were typical of Danish dairy farms (36% Staphylococcus aureus, 25% CNS, 10% Streptococcus uberis, 5% Escherichia coli, 5% Streptococcus dysgalactiae, and 13% no culture).
Test Rationale
The rationale for testing was according to the following stepwise progression, each step being conditional on the outcome of the previous step.
First, 2 subsets of cows were identified. These were cows that could be classified, with reasonable certainty, as either having mastitis or being completely free of mastitis (herein called healthy controls) within specific time-windows. The health definitions involved and the procedure for selecting control cows are described below. The profiles of overall risk of acute mastitis (subsequently referred to as mastitis risk, MR) within the time-window of these 2 groups were then examined to see if they were significantly different. The test hypothesis was that there would be a significant difference between mastitic and healthy controls at and around the time of veterinary treatment, and that this difference would decrease with increasing time (before and after) from treatment.
Second, given a clear description of the time profile of a typical acute mastitis case, the ability of the model to provide early identification of mastitis cases was explored. This was done in terms of both the time lag between model identification and veterinary treatment and the degree of uncertainty associated with different levels of model-generated MR. The latter provides a measure of the reliability of the model based on the time windows.
Third, the reliability of the model throughout all stages of lactation and degrees of infection was examined. To do this, a continuous reference measure of degree of mastitis is required, and it was assumed here that SCC provides such a measure (the pros and cons of this assumption are considered in the Discussion). Thus, the reliability of the model was quantified relative to SCC.
Health Definitions
The basis for the health definitions was veterinary treatment records. In Denmark, only veterinarians are permitted to initiate treatments and they are legally required to record treatment data. Cows were selected for veterinary examination by the farm staff based on cows showing physical signs of mastitis and from weekly visual inspection of milk. A mastitic cow was one that had a recorded veterinary treatment of mastitis with the day of treatment being designated days from mastitis = 0. From the start of lactation until 30 d before the first recorded treatment of mastitis, cows were classified as healthy and were available for selection as control cows.
Because of the possibility of mastitis cases being overlooked or misdiagnosed, a refinement of the basic health definitions was made. This was based on patterns of SCC through time. When a positive systematic deviation in a cow’s time-series of log(SCC) greater than 2.25 occurred the cow was assumed to have had a mastitis and to not be healthy in the period –10 to +10 d relative to the SCC peak. (The method for calculating the deviations is described below). The threshold of 2.25 was chosen because the average increase in logSCC for cows identified as having had mastitis on the basis of this threshold was the same as the average increase in log(SCC) for cows identified by the veterinarian as having had mastitis. If one considers the veterinary treatment records relative to the SCC peaks in terms of specificity and sensitivity, the basic health definitions can be subdivided within the classes mastitic and healthy controls as follows. If there was a veterinary treatment record and an SCC peak (true positive), then the case was termed "true mastitic"; if there was a veterinary treatment record but no SCC peak (false positive), then the case was termed "false mastitic"; if a healthy control had no SCC peak during the control period or previously in that lactation, it was termed "true healthy"; and finally, if a healthy control had an SCC peak during the control period or previously in that lactation, it was termed "false healthy". The choice of an SCC peak of 2.25 resulted in 76% of the cases treated by the veterinarian being classified as true mastitic and 35% of the healthy controls being classified as true healthy.
Data Editing
The raw data consisted of 496,014 milking records, which included 101,046 records with LDH measurements. There were 749 cow lactations with at least 1 LDH measurement. Data were edited for abnormal frequency of sampling. If the gap between 2 consecutive LDH amount records was greater than 5 d, then the milk records in that period were coded as being in a gap. The same applied between calving and the first LDH record, and between the last LDH record and the last record in a lactation. Further, limited runs of valid LDH data (i.e., <10 d between 2 gaps) were deemed to be of little use and also coded as being in a gap; in total, 6,922 LDH records occurred in gaps. Using the gap code as a basis, lactations with few, spread out LDH measurements were excluded. This was done by calculating for each run of nongap data, the frequency of LDH measurements and the number of LDH measurements so that the exclusion was based on the nongap windows. If (and in this order) the total number of nongap LDH records for the whole lactation was less than 21, then that lactation was excluded. If, additionally, the maximum frequency of LDH measures in any nongap (for a given cow-lactation) was less than 0.35 LDH measures per milking, then that lactation was excluded. If, additionally, the maximum number of LDH measures in any nongap was less than 21, then that lactation was excluded. The choice of gap lengths was based on consideration of the typical time-course of a mastitis case combined with visual inspection of the data. The resulting data set contained 602 cow lactations, 90,477 LDH measurements, and 253 mastitis treatment records. In the selection of mastitis cases, if there was a gap in LDH during the period from –30 to +30 d from the day of veterinary treatment, then that case was excluded.
If 2 records of veterinary treatment of mastitis occurred within 8 d, they were treated as a single case (IDF, 1997). For each mastitis case, days from treatment was calculated. Days from treatment was positive and increased with time after the mastitis case, whereas it was negative and more negative with time before the mastitis case. Between 2 mastitis cases, the change from positive days from treatment (after the previous case) to negative days from treatment (before the next case) occurs midway between the 2 cases. In this study, a mastitis case window was defined as being from –30 to +30 d from treatment. For each mastitis case, matching control periods were chosen from the pool of cows of the same breed and lactation number class (1, 2, 3+) that had no veterinary treatment of mastitis in that lactation until at least 30 d after the DIM of the mastitis case being matched. The chosen control period spanned the same DIM of the case being matched. Mastitis cases occurring after 350 DIM were excluded. The process of finding healthy controls was carried out in 3 rounds, a round consisted of finding 1 healthy control for each mastitis case, where possible. The second round repeated the process to identify a second healthy control for each mastitis case, where possible, and likewise for the third round. Thus, a maximum of 3 healthy controls were found per mastitis case. A given cow lactation was allowed to provide more than 1 control period (each to different mastitis cases) but if a cow lactation was used in 1 round of selection, then it was excluded from subsequent rounds of selection. Within cow lactation, control periods were not permitted to overlap. The final number of mastitis cases was 185 (from 146 cow lactations). The final number of healthy controls was 426 (from 319 cow lactations). When comparing true mastitic vs. true healthy cows, to maintain balance in the distribution of cases relative to DIM, breed, and parity, it was decided to exclude whole sets of mastitic cows and their controls when one or other was not classified as "true" (mastitic or healthy). For the "true" comparisons, this resulted in a data set containing 58 true mastitic cases and 71 true healthy cases.
The following procedure was used to identify peaks in the time-series of SCC. For each time-point t, the sum of all the individual changes in log SCC between t-25 milkings and t was calculated. This rolling cumulative sum is relatively insensitive to random noise but increases when there is a consistent increase in log SCC over a number of measurements. Each individual change in log SCC was calculated as
 |
The breed-parity expected change; that is, the normal lactation curve in log SCC, was calculated by fitting an exponential + linear effect to the average log SCC vs. DIM for healthy cows as follows. For each breed-parity combination independently, the daily average log SCC was regressed on DIM using data from 50 to 350 d to obtain the linear trend through lactation and adjust for it. For DIM 0 to 50, the rapid decline in log SCC was modeled as an exponential decline estimated by regressing loge(adjusted average daily log SCC values)on DIM. The resulting exponential + linear equations provided the breed-parity expected change in logSCC(t –t-1). In practice, these expected changes were very small relative to the changes in logSCC related to known mastitis cases. The same rolling cumulative sum procedure was used to identify peaks in the time-series of MR (although in this case the expected change was assumed to be zero). These were then subsequently used when time-aligning the mastitis cases according to the time of peak mastitis risk in the window –10 to +5 d from treatment.
Statistical Analysis
The time-points at which the MR profile of the true mastitic cows was significantly different from that of the true healthy cows were analyzed using a mixed model (SAS version 8.e; SAS Inst. Inc., Cary, NC) with breed, parity, days from treatment, and the interaction between days from treatment and case-type (healthy vs. mastitic) as fixed effects. The model used was
 |
where MastRisk is the MR of cow m (m = 1...110) of breed i (i = Danish Red, Danish Holstein, Jersey), lactation class j (j = 1, 2, 3+), case-type k (k = healthy, mastitic) on day from treatment l (l = –30,...,30); a is the intercept, Aijkmn is the random cow effect, and 
ijklm is the residual. It was assumed that Aijkmn 
N(0, 
2A), ijklm N(0, 2), and all are independent. To account for repeated measures and unequal variances in the healthy and mastitic profiles at different time-points, cow within parity was included as a random effect within combinations of case-type k and period n with 6 levels defined as the intervals [–30, –20], [–20, –10], [–10, 0], [0, 10], [10, 20], [20, 30] days from treatment. The reduction of days from treatment into these 6 periods in the random effect was necessary to obtain convergence of the estimation procedure. All effects were included as factors (i.e., not as continuous variables), and Satterthwaite’s method for estimating degrees of freedom was used. Not including case-type as a main effect allowed direct estimation, for each day, of the difference between healthy and mastitic cows and the significance of this difference.
The reliability of the model for distinguishing true mastitic from true healthy cows within the time window was estimated using the mean (µ) and the standard deviation () of MR from the true healthy cows to calculate the likelihood of a given MR (x) belonging to the normal standard distribution of the true healthy cows, thus:
 |
where F is the distribution function for the standard normal distribution.
The mixed model procedure was also used to derive the residual MR values from the whole data set (not just the windows around mastitis treatments) after adjustment for the corresponding SCC values. Because there is no a priori linear relationship between MR and log SCC, the second and third powers of log SCC were included in the regression, as were milk yield and log SCC x milk yield to allow for the fact that MR is based on LDH amount (LDH concentration x milk yield), whereas log SCC is not:
 |
where MastRisk is the MR of cow i on day of lactation j; logSCC is the log of SCC; MY is milk yield at that milking (kg); a is the intercept; b,c,d,e,f are the regression coefficients; and ij is the residual. In this regression, logSCC, (logSCC)2, (logSCC)3, milk yield, and the interaction between milk yield and logSCC were included as continuous explanatory variables for MR. Cow within parity was included as a random effect.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGEMENTSREFERENCES |
|---|
shows a typical example of a time profile for 1 cow through lactation of the LDH amount and the model-derived MR. In this example, the cow received veterinary treatment for mastitis on d 73 and 90 after calving. The corresponding profile of SCC and milk yield is also shown. To make valid comparisons between healthy control cows and mastitic cows, these time profiles were aligned based on days from veterinary treatment.
|
. These are based solely on the presence or absence of veterinary treatments. The profile for the health controls is flat throughout the 60-d window with an average median MR of 0.02. In contrast, the profile of the mastitic cows increased above the control cows’ baseline from about –8 d from treatment, increasing to a MR value of 0.12 at –4 d from treatment and remaining at that level until d 0. After treatment, the MR declined rapidly to a level below that of the healthy controls, from which it gradually increased to the control cow level. The corresponding 25 and 75% quartiles of the distribution of MR are shown in Figure 2B.
|
that there were systematic differences between the time profiles of median MR of mastitic and healthy cows and that these differences were related to days from treatment. Thus, the model can detect clinical mastitis. The peak of the time profile of median MR for mastitic cows was, however, only 0.12 and at no time was there separation between the 25 and 75% quartile ranges of the mastitic and healthy cows. A possible reason for this could be that the correspondence between veterinary treatments and true mastitis is not 100%. Therefore, as described in Materials and Methods, peaks in SCC were used to differentiate veterinary-treated cows into true mastitic and false mastitic (141 and 44 cases, respectively). Likewise, control cows were differentiated into true healthy and false healthy (147 and 279 cases, respectively). The median time profiles of true healthy and true mastitic cows are shown in Figure 3. The overall patterns of these time profiles are similar to those based solely on veterinary treatments but the peak for true mastitic cows is better defined, increasing to a higher peak of 0.2 on d 0. There were significant differences between true mastitic and true healthy cows from –4 to +2 d from veterinary treatment (Figure 4). The average MR for true healthy cows across the time window was 0.044, which was not (quite) significantly different from 0 (P = 0.056). There were no days on which the average MR of the true healthy cows differed significantly from the average of 0.044. The MR of Jersey cows was, on average, 0.028 less than the other breeds (P < 0.05). There was no significant effect of parity on MR. Despite a significant difference between mastitic and healthy cows from –4 to +2 d from veterinary treatment, the peak for mastitic cows was low (Figure 4) when considering that the intended scale for MR was from 0 for healthy cows increasing to 1 for clinical mastitis.
|
|
The peak-aligned time profiles for true mastitic and true healthy cows are shown in Figure 5. As expected, the effect of aligning to peak MR was to produce a much sharper peak to the time profile of true mastitic cows. The median MR at peak was 0.62 and the average was 0.80. The time profile of MR was largely symmetrical around the time of peak although after peak it tended to drop below the level of the healthy controls (Figures 4 and 5). This reflects the way in which the MR was calculated from the indicator, LDH amount. The IBR has 2 additive components, one based on the slope and one based on the level of the LDH time-series [see Chagunda et al. (2006) for details]. The steeper the slope in LDH, the greater the slope-based risk. When the slope is negative, so is the slope-based risk; this allows the model to differentiate between 2 cases with the same level of LDH but where one is prepeak and rising and the other is postpeak and falling. Because the MR is designed to capture acute mastitis cases, the level-based risk uses not the absolute LDH level but the current level minus a baseline for that cow. This baseline is a rolling average of the previous 7 d of values of LDH. Thus, once the LDH level has returned to baseline postmastitis, the level-based risk is zero, and the IBR can be negative if there is still a downward slope in LDH. This is why the median MR for mastitic cows goes below that of the healthy controls postpeak. In practice, this may not be sensible, especially from the point of view of presentation to the end-user, and setting bounds (0 to 1) for MR would be one way to deal with this. In the present study, however, it is desirable to not truncate the MR values when evaluating the underlying equations. Had the MR values been truncated, then the estimates of the variability associated with the median profile would have been biased, which would have affected the quantification of the reliability of using any given value of MR to indicate mastitis.
|
gives the likelihood of a given risk value being part of the healthy population. For example, a MR value of 0.6 has a <1% likelihood of actually coming from a healthy control, and a MR value of 0.3 has a 15% likelihood of coming from a healthy control (Figure 5). Likewise, when the MR value was close to 0 and similar to that of the control cows, the likelihood of this value belonging to the healthy population was 50%; that is, there is an equal chance of a MR value of 0 belonging to healthy cows or to cows that will subsequently have mastitis. This likelihood provides a measure of the uncertainty of diagnosis should one use a particular MR value as a threshold for taking action; for example, calling the veterinarian or starting a treatment.
Figure 6 shows the MR peak-aligned time profiles for SCC, milk yield, and the 2 components of the MR: the IBR and the ARF. Figure 6 shows clearly that the MR profiles for true mastitic cows corresponded closely to the expected changes in SCC and milk yield. Further, it can be seen that the basis of the MR profile comes from the IBR, which is the rate of change and level of LDH amount. The ARF shows no clear time trend relative to mastitis peak.
|
and 6) indicates that SCC can be used as a simple surrogate for degree of mastitis. Accepting SCC as a continuous estimate of degree of mastitis allows the performance of the model to be evaluated on data from all time-points in lactation and not just the windows around mastitis treatments. Therefore, the reliability of the model was evaluated by examining the residuals from a regression of MR on log SCC (including adjustments for milk yield). These residuals can be interpreted as being the size of the error in MR (the assumptions involved are explored in the Discussion). The residual MR is shown in Figure 7.
|
). The likelihood of a MR of 0.7 belonging to the healthy population of 1.7% implies that in a population of healthy cows (with the mean and SD of MR given above), 1.7% of the MR values generated by the model would be >0.7. Assuming that the true degree of mastitis of the healthy population is zero, the model will have generated erroneous values above 0.7 for 1.7% of the healthy population. Similarly, the size of the residuals (model-generated MR – MR estimated from SCC) from the full data set can be interpreted under the assumption that SCC provides an error-free measure of true degree of mastitis (this assumption is examined in the Discussion) and that the regression model did not introduce any bias. Given this assumption, the proportion of residuals greater than +0.7 gives the proportion of occurrences of MR wrongly inflating the true MR by 0.7. Assuming that this proportion, found to be 2.1%, is the same across the range of true degrees of MR, then for healthy cows (i.e., true MR = 0), this is the proportion of records with erroneous model-generated MR values >0.7. Using the assumptions given above, the percentage of residuals >0.7 provides an estimate of the proportion of healthy observations that would be misclassified as mastitis when using a classification threshold of 0.7. This equates to a specificity (the ability to avoid misclassifying healthy observations as mastitis) of (100 – 2.1) = 97.9%. Conversely, the percentage of residuals less than –0.3 was found to be 7.2%. This provides an estimate of the proportion of true mastitic observations; that is, the true degree of mastitis = 1, that would erroneously be assigned a model-generated degree of mastitis less than 0.7 and thus be misclassified as healthy. Assuming that mastitic cows have a MR of 1, this is equivalent to a sensitivity (the ability to avoid misclassifying mastitis observations as healthy) of (100 – 7.2) = 92.8%. Both these estimates of specificity and sensitivity are for a classifying threshold of 0.7.
Because the residuals are generated from all stages of lactation and degrees of mastitis, as indexed by SCC, they provide further information about the performance of the model being tested. The standard deviation of the residuals was greatest in early lactation (SD = 0.34 for 1 to 50 DIM) and declined as lactation progressed (SD = 0.20 for 100 to 150 DIM; Figure 7). There was also a tendency for the residuals to be greater for greater values of average MR, so the increased standard deviation of residuals in early lactation could have been because the frequency of mastitis occurrence was greater in early lactation. Therefore, the residuals from true healthy cows were examined relative to stage of lactation. There remained a greater error in early lactation: the percentages of errors >0.7 for the first, second, third, and fourth 50-d periods in lactation were 4.5, 1.9, 1.3, and 1.0, respectively.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGEMENTSREFERENCES |
|---|
A highly significant difference in model-derived MR was found between true mastitic (i.e., received veterinary treatment for mastitis and had an SCC peak) and true healthy cows (i.e., no veterinary treatment and no positive systematic deviation in SCC to that point in lactation). The differences between true mastitic and true healthy cows were significant from –4 to +2 d realtive to veterinary treatment, indicating that one advantage of having a time-series of measurements is that mastitis can reliably be detected early. This would be a benefit of an automated, inline mastitis detection system based on an indicator such as LDH.
The second aim of this study was to use a test logic that allowed explicit examination of the time-related changes and a progressive scale of predicted mastitis state. We have previously reported favorable specificity/ sensitivity values for this model (Chagunda et al., 2006). These values are, however, based on the use of a threshold to classify MR into positive and negative indications of mastitis in a fixed time window. In the present study, we have instead chosen to retain the time-profile of MR; that is, to retain the continuous nature of the model outputs. This is because we believe that there is valuable information in the time profiles that is lost when these are condensed into single specificity/sensitivity estimates. Using time profiles raises the issue of the appropriate reference time-point by which to align the profiles. We started with the time of veterinary treatment as the reference point, which resulted in a median profile of MR that had a relatively broad peak that, although significantly different from true healthy controls, was relatively low (0.2; Figure 3). This timescale is relevant for considering the benefits of these time profiles relative to time of veterinary treatment (see above) but it does not reflect the true shape of the MR profiles of mastitic cows. There was a substantial variation in the time between peak MR and time of veterinary treatment (SD ± 2.8 d) and thus, a flattening of the median time profile when expressed relative to time of treatment. When the profiles were aligned relative to the peak MR, a much higher and narrower peak was found in the median profile. This reflects the shape of the individual MR peaks in this study. It has been documented that different mastitis pathogens develop into clinical mastitis at different rates and thus it is expected that different pathogens would generate different MR profiles (de Haas et al., 2002). In the present study, there were insufficient cases to allow meaningful results from splitting the cases into different pathogen types. This aspect remains to be explored.
As can be seen from Figure 5, the reliability of using a given value of MR to indicate mastitis decreases with decreasing MR value. This reliability is given as the likelihood of that value belonging to the distribution of healthy values, where healthy is defined as never having had a veterinary treatment of mastitis and never having had a systematic increase in logSCC above 2.25. The relationship between the likelihood of a MR belonging to the healthy population and the observed MR was readily approximated by the following exponential function (coefficients derived by log-linear regression): likelihood = 0.55 x [exp(–5 x MR)]. Thus, for example, the likelihood of a MR of 0.7 belonging to the healthy population was estimated by this function to be 1.7%. Another way to assess the reliability of the model, by quantifying the deviations in MR values relative to the degree of mastitis, is discussed below.
When testing models, it should be remembered that the test is only as good as the reference values used. Clearly, if the reference value is wrong (e.g., a misdiagnosis of mastitis), then this will be counted as a failure of the model. The extent to which poor reference data compromises the test of a model increases the closer the true model performance is to the performance of the reference method. If the model is better than the reference method, then the "polarity" of the test is effectively reversed such that it more valid to view the test as being a test of the reference method relative to the model, not vice versa.
Although a number of statistical techniques exist for dealing with variable quality reference data (Wang et al., 2006), the ways in which the quality of the reference data and the test method used compromise the evaluation of a given model can be difficult to ascertain. In the present study, by using a graphically based, stepwise approach, it became clear that the quality of the reference data was affecting the test results. Supplementing the veterinary treatment records with SCC data increased the difference in MR between mastitic and healthy cows. Likewise, considering the time-related aspects; that is, recognizing that the time lag between peak infection and treatment is subject to random noise affected the results. Adjusting for this further increased the difference in MR between mastitic and healthy cows. Finally, the robustness of the test is affected by the assumed nature of the reference data. The shape of the time profiles of MR and SCC (Figures 5 and 6) provided good evidence that mastitis is not a binary condition but rather involves a continuous development from healthy to clinical mastitis through intermediate and increasing degrees of mastitis. Others have come to the same point of view (Green et al., 2004; Detilleux et al., 2006) and therefore we felt justified in using a continuous reference measure of degree of mastitis (or risk of developing clinical mastitis).
If it is accepted that mastitis is best described by degree of mastitis, then the robustness of any test based on specificity/sensitivity calculations is limited. To calculate specificity and sensitivity it is necessary to exclude a portion of the data if the underlying condition is continuous. As can be seen in Figure 8, the specificity (and sensitivity) of a detection method will always be poor when the true degree of mastitis is close to the classification threshold. Typically, the midrange values (the so-called subclinical cases) are excluded from calculations of specificity and sensitivity when the underlying condition is approximately continuous. Thus, a substantial and highly selective data reduction has to be undertaken to obtain useful specificity/sensitivity estimates. In the present study there were 58 true mastitic and 71 true healthy cases; these could provide the basis for a "sensible" specificity/sensitivity calculation but represented less than 1% of full data set. If more than 95% of your test data are excluded, then what is the worth of that test? Further, specificity/sensitivity calculations require that a threshold MR be defined for classifying cows as healthy or mastitic, and the resulting values of specificity and sensitivity will vary according to the threshold chosen (Norberg et al., 2004; Wang et al., 2006; Cavero et al., 2007). If instead, it can be demonstrated that a suitable measure is available to provide a continuous reference measure of degree of mastitis, then the model can be tested on the whole data set. In this situation, we suggest that it is more informative to describe the ability of the model to esti mate degree of mastitis (in this study, MR) in terms of the residuals relative to the assumed true degree of mastitis. The estimates of reliability derived from the full data set using SCC as the degree of mastitis reference agreed well with those derived from quantifying the differences between true mastitic and true healthy cases, and are comparable with the highest reported values from other mastitis detection studies (Hillerton, 2000; de Mol and Ouweltjes, 2001; Cavero et al., 2007).
|
By combining these with data filtering and smoothing techniques, it should be possible to evaluate which combination of mastitis measures best indexes degree of mastitis while at the same time dealing with random noise in these measures. We consider this the logical next step in the development of the test rationale for explicit examination of time-related changes in degree of mastitis, and thus an issue worth pursuing.
|
CONCLUSIONS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGEMENTSREFERENCES |
|---|
|
ACKNOWLEDGEMENTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGEMENTSREFERENCES |
|---|
|
FOOTNOTES |
|---|
Received for publication February 27, 2007. Accepted for publication August 17, 2007.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGEMENTSREFERENCES |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
; right y-axis) relative to weeks in milk. Peaks in MR (e.g., at wk 5.5, 10.5, 12.3, and 12.8) are generated by rapid increases in the model indicator, smoothed lactate dehydrogenase (LDH) amount (µmol/ min; left y-axis), as shown by the solid line with no symbols. The raw, unsmoothed, values of the inline indicator LDH amount are shown as solid circles. This cow received veterinary treatment for mastitis on DIM 73 and 90 (indicated by arrow,
) but not on DIM 39 (wk 5.5) when both MR and SCC showed a peak. The profile of SCC is shown on the left y-axis (+, count x 10–5); milk yield (
, kg/d) is also shown on the left y-axis.
