|
|
||||||||
Department of Large Animal Sciences, The Royal Veterinary and Agricultural University, Grønnegårdsvej 8, DK-1870 Frederiksberg C, Denmark
1 Corresponding author: ssn{at}kvl.dk
| ABSTRACT |
|---|
|
|
|---|
Key Words: ELISA paratuberculosis immune response bacterial shedding
| INTRODUCTION |
|---|
|
|
|---|
The diagnostic accuracy of a test is usually described by its diagnostic sensitivity (the probability that a test is positive given that infection is present) and the diagnostic specificity (the probability that a test is negative given that infection is not present). Underlying the sensitivity and specificity measure is an implicit assumption of an infection that is either present or absent in each animal at a level of interest for particular purposes and resulting decisions. The "infection" condition must reflect the purpose of testing as defined by the decision maker. This requires an understanding of the progressive stages of infection and accuracies for various testing schemes so that appropriate actions can be initiated subsequent to a positive test result.
In general, purpose-specific objectives could be 1) certification, 2) confirmation of clinical disease, 3) detection of infected animals, 4) detection of animals that are about to become an economic burden, and 5) detection of infectious animals that are shedding large amounts of bacteria to the environment, thus representing a risk to susceptible animals. Decisions subsequent to a test-positive result could be culling, treating the test-positive animals as high-risk animals, or confirmatory testing to provide further information.
For this study, it is assumed that paratuberculosis develops in 3 stages: 1) infection, which generally affects calves; 2) infectious (i.e., shedding infective doses of Map in feces or milk), which follows infection after some variable period of time; and 3) end-stage disease with production losses. Obviously, the strategy of the decision maker influences the optimal choice of which of these 3 stages constitutes a truly positive or infected animal. To establish freedom from disease, information at stage 1 is necessary, but to control the infection to minimize losses, stage 2 might be adequate. Hence, it is necessary to define the purpose of the testing to establish the underlying condition that the test is intended to identify; only then may the diagnostic properties of the test be evaluated.
Two commonly used diagnostic techniques are 1) culture of Map from fecal samples or fecal culture (FC), and 2) detection of antibodies using ELISA. These tests are imperfect for detection of infection, primarily because of a long, and probably variable, incubation period. However, it is uncommon to assess the diagnostic information both relative to the purpose of testing and the chronicity of infection. Furthermore, it is known that other covariates, such as age, influence the test response, especially for the ELISA. Thus, as age of the animal is often readily available, such information should be used when evaluating a diagnostic test.
Although all infected animals must be assumed to be infectious to some degree, some animals may be more infectious than others. During the cell-mediated immune responses, some control of infection is still maintained (Stabel, 2000), and infectiousness is expectedly kept relatively low. During the subsequent humoral immune responses, the infectiousness is expected to be higher than during the cell-mediated immune responses. Thus, it can be assumed that the appearance of antibodies is a predictor of infectiousness. Therefore, an antibody (AB) test, such as an indirect ELISA, would make a good choice given that it can predict highly infectious animals.
The objective of this study was to describe 2 tests, a FC test and an indirect milk ELISA, for their ability to predict 2 conditions: "infection" and "infectious," when the tests are used as screening tests adjusted for age as a covariate. The primary focus was describing the probability of detecting the conditions (sensitivity) and secondarily to describe the probability of absence of the conditions (specificity).
| MATERIALS AND METHODS |
|---|
|
|
|---|
A summary of information regarding milk production and herd structure is given in Table 1
. Information on milk production, breed, and age was obtained from the Danish Cattle Database. Date of birth was missing from 10 cows that were excluded from the study. The herds consisted of 6 different breeds, including crossbred cows. Breeds represented by few animals including 11 Red Danish, 1 Finnish Ayrshire, and 1 Old Danish were excluded from the study. The distribution of the breeds of the remaining cows was 1,430 Danish Holsteins, 435 Danish Jerseys, and 120 crossbreds. These 1,985 cows contributed a total of 23,219 milk samples and 8,832 fecal samples.
|
The distribution of milk samples per cow was as follows: minimum = 1 sample per cow, median = 16 samples per cow, and maximum = 31 samples per cow. The distribution of fecal samples per cow was as follows: minimum = 1 sample per cow, median = 4 samples per cow, and maximum = 13 samples per cow. The distribution of milk samples per year of age is summarized in Table 2
, and the distribution of fecal samples per year of age is summarized in Table 3
.
|
|
Fecal samples were processed as follows: 1 to 2 g of feces was decontaminated with 10 mL of NaOH solution followed by centrifugation at 1,300 x g. The supernatant was discarded, and the remaining material was dissolved in 5% oxalic acid with 0.1% malachite green. After a further incubation step, the material was centrifuged, and the supernatant was discarded. Neomycin sulfate and amphotericin B were added to the solution and incubated. After incubation and mixing, 3 to 4 drops of solution were applied on each of 4 tubes of medium. The fecal samples were cultured on Löwenstein-Jensen medium before July 2002 and on Herrolds egg yolk medium from August 2002 to the end of the testing period. Samples collected in July and August 2002 were tested in both media, and Herrolds egg yolk medium was more sensitive. Hence, it was decided to change and use the more sensitive medium. All positive cultures were confirmed for presence of the IS900 sequence in PCR. Further descriptions of the methods and the comparison of the 2 media are given in Nielsen et al. (2004).
Data Preparation
Data control was done by comparing the cow identity of a sampled cow with the information in the Danish Cattle Database to determine whether the cow had actually been in the herd at testing.
Figure 1
is a schematic representation of the statistical analyses. "Objective 1" refers to the investigations of ELISA as the diagnostic test, and "Objective 2" refers to FC as the diagnostic tests. The FC and ELISA were evaluated against the conditions: "Map infection" and "Map infectious." Because there is no antemortem reference test that can correctly determine the true status of an animal, the following definitions were used:
|
For objective 2 (evaluation of the FC) the condition "infection" was considered present in any cow whose moving average of 2 consecutive ODC values was > 0.3. These animals were also referred to as AB positive. The moving average at a given age was calculated by taking the average of the present ODC value and the ODC value on the previous test of the cow. "Infectious" was defined as the age at which a cow obtained the "infection" status (AB+). She could be classified as "infectious" as follows: never AB+; AB+ in third year of life; AB+ in fourth year of life; AB+ in fifth year of life; or AB+ in sixth year of life; or AB+ in seventh year of life or later.
Statistical Analyses
Cross-tabulations showing the distributions of test-positive samples (FC+ or AB+) obtained by age group and by infectious group (age group at which an animal became test-positive) contributed to the basic descriptive statistics. For descriptive purposes, mean ODC values were calculated for the various groups.
In the following analyses, cows contributed samples in the age group in which they were found FC+/AB+ and 2 age groups later, as shown in Table 2
(for analyses of the ELISA response) and Table 3
(for the analyses of the FC).
Predictions of the AB response as a function of age were estimated by nonparametric regression of ODC values as a function of age, using cubic-smoothing splines (Hastie and Tibshirani, 1991) using the GAM procedure in SAS (Version 8.2, SAS Inst., Inc., Cary, NC). The predictions were performed for all observations in 2 groups (FC+ and FC) in which the underlying condition was "infection" and were divided into groups depending on which age they turned FC+ or when the underlying condition was "infectious." The model used for each of these groups was as follows:
![]() | [1] |
where ODC was the ODC value from the ELISA, ß0 was the baseline value of the ODC, and S(age) was the smoothing function of age in year for each of the FC groups.
To assess the specificity of the ELISA, an additional analysis was performed. In this analysis, the noninfected cows were defined as cows with
8 negative FC obtained over a 2-yr period. Samples from the 2-yr period were excluded. Only samples from about the first year were included.
Subsequent to analysis of the ODC response on a continuous scale, the ODC were dichotomized at a cut- off of 0.3, which is the recommended laboratory cut-off (where ODC > 0.3 are positive). Data were analyzed with the following model:
![]() | [2] |
where P(ODC > 0.3) was the probability of testing positive in the ELISA at cut-off 0.3, ß0 was the baseline probability of testing positive in ELISA, and S(age) was the smoothing function of age in years in each FC group.
As with the ODC on a continuous scale, a separate analysis was done with cows that had
8 negative FC over a 2-yr period. Samples from the 2-yr period were excluded. Only samples from about the first year were included.
A similar model was used for predictions of the probability of testing positive at any time in FC. The model was
![]() | [3] |
where P(FC+) is the probability of testing positive in FC, ß0 was the baseline probability of testing positive in FC, and S(age) was the smoothing function of age in years in each AB group.
The predictions for FC+ were done for all observations divided into 2 groups: AB+ and AB with the underlying condition being "infection" and also when divided in groups depending on the age at which they turned "infectious."
The effect of change in FC method was assessed by an extension of Equation 3, where culture method was included as a parametric term, which could be either the old or the new method.
| RESULTS |
|---|
|
|
|---|
In Figure 2a
, the predicted ODC values are shown for 2 groups of cows, one group never becoming FC+ and the other group becoming FC+ at some time during 2 to 6 yr of age. These are the predictions of the ELISA response for "infection." In Figure 2b
, the latter group was further divided into 5 groups, one for each age group in which cows became FC+. These are predictions of the ELISA-response for being "infectious." The estimated probabilities of testing ELISA-positive are shown for the same groups in Figures 3a and b
, subsequent to the dichotomization of the ELISA response.
|
|
In Figures 3
and 4
, the sensitivity and specificity estimates obtained in the present study can be read on the vertical axis, although they are referred to there as probabilities given a condition. The sensitivity of the test for a given age is read directly as the probability for the positive groups, whereas the specificity given age is read as 1 the probability for the group in which the condition was never found. This condition may be interpreted as infection for Figures 3a
and 4a
and infectious for Figures 3b
and 4b
.
|
The effect of culture method was insignificant averaged over all groups of AB+ (P = 0.22). Hence, for ease of interpretation, the results given were averages of both methods.
| DISCUSSION |
|---|
|
|
|---|
Disease Definition, Sensitivity, and Specificity
Given a unique disease definition (the underlying condition), the ability of a test to detect this condition can easily be characterized. However, for paratuberculosis, a suitable, unique disease definition does not exist. For decision makers seeking to eradicate Map from a herd or region, the appropriate definition of paratuberculosis varies in that decision makers may merely be wishing to reduce economic losses or they may be hopeful of using the test results for appropriate management of infectious cows.
A frequent approach for the evaluation of tests for detection of Map infection is the selection of a sample population, for which infection status is determined postmortem. Such a population is often older than the population that is to be evaluated by the test subsequently. It may also be selected based on criteria, which favor agent-detecting methods (such as FC), because bacteria detected are deemed more indicative of infection than the finding of an immune response. Selection biases are difficult to avoid, because the test scheme chosen as the reference or gold standard essentially defines what constitutes absence and presence of infection (Nielsen and Toft, 2002).
Latent-class methods, which do not presume the definitive status of an animal, can be used, as exemplified in Nielsen and Toft (2002). Currently, these methods do not take into account the purpose of testing, which still need to be addressed. However, latent class methods may provide a promising alternative for the analyses carried out in the present study.
In this study, the repeated testing of animals by FC was used as a reference for the ELISA test, and the repeated testing of milk samples using ELISA was used as a reference for the FC. These are not perfect reference methods, but will underestimate the true sensitivity of each method because the low sensitivity of any reference method will falsely classify some test-positive as false-positive even though they are actually true positives that are not detected by the reference method if the reference method has a low sensitivity. However, to simplify the discussion, we will assume that when used for classification into "infected" or "infectious" the relevant reference method is perfect. Thus, for evaluation of ELISA, repeated FC is a perfect reference method and vice versa. The conditions "infected" and "infectious" were assumed to be the same irrespective of reference method.
Sensitivity and Specificity Given Disease Definition: Infection
Figures 2a
, 3a
, and 4a
illustrate the test responses as a function of age when the underlying condition is infection. Given the aim is to detect an infected animal, these figures show which test responses can be expected with increasing age. For ELISA, the sensitivity increases almost linearly from 0.06 at 2 yr of age to 0.50 at 5 yr of age (Figure 3a
), whereas the specificity decreases from 0.997 to 0.93 as the age of the tested animal increases. This apparent drop in specificity is likely to be an artifact caused by the low sensitivity of FC (Figure 4a
), although some of this potential artifact can be removed by testing more frequently with FC. The average sensitivity of FC increases from 0.05 at 2 yr of age to 0.21 at 5 yr of age. The specificity is high (0.964 to 0.984). A true false-positive is a positive test response from an animal that is not infected. It is possible, although unlikely, that a bacterium detected is not Map, as the cultured bacteria were confirmed with IS900 PCR. However, false-positives may be caused by pass-through, in which cows consume Map without becoming infected and lead to a possible false-positive test reaction. Another possibility is that ELISA has not detected cows that were shedding bacteria subsequent to Map infection. The conclusion at this stage must be that the ELISA is more sensitive for detection of Map infection than FC, but at an early age, it is still low. There may be a risk of obtaining many false-positives, but it is unlikely that they are all attributable to false-positive reactions of the ELISA. Many could be due to the low sensitivity of FC.
Sensitivity and Specificity Given Disease Definition: Infectious
The results shown in Figures 2b
, 3b
, and 4b
are the test responses when the underlying condition is infectious. First, the ELISA response, on average, for a cow that is detected FC+ in her third year of life will also test positive within that year (Figures 2b
and 3b
), although initially, the sensitivity is only 0.06 at 2 yr of age, but at 3 yr of age, it has increased to 0.58 (Figure 3b
). The pattern is similar for cows found positive in FC in their fourth and fifth year of life: initially, the sensitivity is low, but steeply increases within the following year. The increases in sensitivity are almost parallel for the different age groups. This indicates that a positive ODC is coherent with a positive FC and vice versa. Thus, the cow may transfer to the infectious stage any time, regardless of age of the cow (although still in the interval 2 to 5 yr of age). For FC, the sensitivity is also increasing from 0.06 at 2 yr of age to 0.35 at 3 yr of age for cows that are AB+ in their third year of life. These results indicate that there is a good relationship between being infectious and AB+ and that both the FC and the ELISA may detect some of these animals. However, the ELISA has a greater sensitivity than the FC, although to some degree it may be at the expense of specificity, particularly for older animals. The apparent loss in specificity is likely caused by the generally low sensitivity of the FC method. This can partly be deduced from Figure 2b
. A cow testing FC+ in her fifth year of life does not have AB before 3 to 3.5 yr of age, whereas the average of the FC cows is higher. These would be expected to be similar if the FC cows were all correctly classified. Given that some FC animals are actually infected, the sensitivity of ELISA is underestimated. The magnitude of this underestimation could easily be 0.05 to 0.10, corresponding to the apparent drop in specificity.
If the ODC is used as an approximation of AB production, it is observed that AB are generally present in low levels at 2 to 3 yr of age, whereas higher levels are not reached before the animals are 4 to 5 yr of age. This is consistent with previous findings (Nielsen et al., 2002a). Detected fecal shedding is maximum around 4 to 5 yr of age, with a maximum P(FC+|AB+) = 0.17. The maximum at 4 to 5 yr of age is consistent with the findings by Kalis et al. (1999). If AB was used as the gold standard in a test evaluation, the P(FC+|AB+) would have corresponded to a diagnostic test sensitivity of 0.17. This is consistent with what has been obtained using methods in the absence of a gold standard (Nielsen and Toft, 2002; Nielsen et al., 2002c), in which sensitivity estimates in the range 0.2 to 0.4 were obtained. The lower sensitivity found here reflects the misclassification of false-positives introduced because of the lack of specificity of the ELISA.
Cows that are never AB+ have a probability of 0.02 to 0.04 of testing FC+. Hence, most FC+ cows will, at some point in time, test positive in ELISA. Those that do not become AB+ are potentially passive carriers of Map without ever being infected, as described by Sweeney et al. (1992). Another explanation is that they are unable to produce AB, perhaps because they were infected in uteri while the immune system was developing. Hence, Map could produce persistent infections similar to those occurring with bovine virus diarrhea infections. A last possible explanation is censoring, i.e., FC+ cows have not been kept long enough after first shedding has been detected, and therefore, production of AB simply has not begun. The finding suggests that, in most cases, the ELISA will at some time detect infection, but not necessarily before the cow becomes infectious to other cattle.
Concluding Remarks
Discerning between "infection" and "infectious" can be highly relevant for a decision maker, as the infected cow may prohibit a declaration of freedom from infection, but this cow may never become an economic burden. Hence, if a farmer has no intention of declaring freedom from infection, he may only want to focus on detecting infectious animals to reduce spread of infection. Infectious animals can be detected with ELISA with a fair sensitivity in all age groups even with a single testing. Repeated tests can be required to further characterize the infection state, e.g., whether or not the cow will experience production losses. Infected animals cannot be detected at younger ages, but this is less relevant if the objective only is to detect infectious animals. Thus, a clear aim of the testing strategy should be defined before testing. Given this strategy, an advisor may provide more specific estimates of the probability of detecting the condition, whether this specific condition is infection or infectiousness.
The nature of this study was descriptive; thus, to simplify the presentation of the results, some elements have been given less attention. Consequently, some of the results should be interpreted carefully. For some groups, the number of observations is low, and generally, the right ends of the curves are more uncertain that the rest because of the censoring of old cows. To simplify the results and because of the computational complexity, uncertainty estimates have generally been excluded from the presentation. However, 95% confidence limits are given in Figure 2a
, 3a
, and 4a
. It should be noted that correlation between samples from the same cows has not been addressed, and single observations are not independent. Finally, the predictions presented are average values, which do not take into account the variation in the response to infection of individual cows as previously described (Nielsen et al., 2002b). This variation may be reflected in variation among herds. Variation attributable to herd could not be assessed because of the lack of statistical power. However, there were indications that cows of Herds 2 and 7 did not, on average, yield similar responses as cows in the other herds.
To handle some of the issues raised here, other techniques, such as time-to-event analyses, would be more appropriate, with the events being AB+ and FC+. Also assessing the relations between the 2 events would be important. However, the purpose of the current studies was primarily descriptive and should be considered as such.
Diagnostic sensitivity and diagnostic specificity are test properties normally characterizing diagnostic tests in control and eradication programs. With a clear disease definition and consensus on actions following diagnosis, sensitivity and specificity are adequate and informative characterizations of a test. However, some decision makers require tests for conditions other than those optimal in eradication programs, for example. The chronic nature of paratuberculosis and the lack of perfect diagnostic tests is a continuous challenge in the interpretation of the test results obtained in any testing. Chronicity affects the sensitivity in that progression of infection increases the sensitivity. In a given population, progression may not only be determined by a fixed incubation time. It may be influenced by management factors (e.g., feeding strategy) that lead to variation in incubation times. Yet, the fixed incubation time approach can provide us with some information, e.g., by inclusion of age as factor in interpretation of the test results. Another factor is the definition of actions that are made subsequent to a diagnosis, when potential production losses may require one action whereas transmission of Map may call for another, with different requirements at different prevalences.
| CONCLUSIONS |
|---|
|
|
|---|
| ACKNOWLEDGEMENTS |
|---|
|
|
|---|
Received for publication July 10, 2005. Accepted for publication September 30, 2005.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
A. B. Kudahl, S. S. Nielsen, and S. Ostergaard Economy, Efficacy, and Feasibility of a Risk-Based Control Program Against Paratuberculosis J Dairy Sci, December 1, 2008; 91(12): 4599 - 4609. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. S. Nielsen, H. Bjerre, and N. Toft Colostrum and Milk as Risk Factors for Infection with Mycobacterium avium subspecies paratuberculosis in Dairy Cattle J Dairy Sci, December 1, 2008; 91(12): 4610 - 4615. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. M. Baptista, S. S. Nielsen, and N. Toft Association Between the Presence of Antibodies to Mycobacterium avium subspecies paratuberculosis and Somatic Cell Count J Dairy Sci, January 1, 2008; 91(1): 109 - 118. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. S. Nielsen and A. K. Ersboll Age at Occurrence of Mycobacterium avium Subspecies paratuberculosis in Naturally Infected Dairy Cows J Dairy Sci, December 1, 2006; 89(12): 4557 - 4566. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Cho and M. T. Collins Comparison of the Proteosomes and Antigenicities of Secreted and Cellular Proteins Produced by Mycobacterium paratuberculosis Clin. Vaccine Immunol., October 1, 2006; 13(10): 1155 - 1161. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. J. Wells, M. T. Collins, K. S. Faaberg, C. Wees, S. Tavornpanich, K. R. Petrini, J. E. Collins, N. Cernicchiaro, and R. H. Whitlock Evaluation of a Rapid Fecal PCR Test for Detection of Mycobacterium avium subsp. paratuberculosis in Dairy Cattle Clin. Vaccine Immunol., October 1, 2006; 13(10): 1125 - 1130. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Wang, B. W. Turnbull, Y. T. Grohn, and S. S. Nielsen Estimating receiver operating characteristic curves with covariates when there is no perfect reference test for diagnosis of Johne's disease. J Dairy Sci, August 1, 2006; 89(8): 3038 - 3046. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |