JDS
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

J. Dairy Sci. 2007. 90:4612-4622. doi:10.3168/jds.2007-0276
© 2007 American Dairy Science Association ®
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Interpretive Summary
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Silva, E.
Right arrow Articles by Fricke, P. M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Silva, E.
Right arrow Articles by Fricke, P. M.

Accuracy of a Pregnancy-Associated Glycoprotein ELISA to Determine Pregnancy Status of Lactating Dairy Cows Twenty-Seven Days After Timed Artificial Insemination

E. Silva*, R. A. Sterry*, D. Kolb{dagger}, N. Mathialagan{ddagger}, M. F. McGrath{ddagger}, J. M. Ballam{ddagger} and P. M. Fricke*,1

* Department of Dairy Science, University of Wisconsin, Madison 53706
{dagger} Lodi Veterinary Clinic, Lodi, WI 53555
{ddagger} Monsanto Agricultural Company, St. Louis, MO 63167

1 Corresponding author: pmfricke{at}wisc.edu


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 ACKNOWLEDGEMENTS
 REFERENCES
 
To determine the accuracy of a pregnancy-associated glycoprotein (PAG) ELISA in identifying pregnancy status 27 d after timed artificial insemination (TAI), blood samples were collected from lactating Holstein cows (n = 1,079) 27 d after their first, second, and third postpartum TAI services. Pregnancy diagnosis by transrectal ultrasonography (TU) was performed immediately after blood sample collection, and pregnancy outcomes by TU served as a standard to test the accuracy of the PAG ELISA. Pregnancy outcomes based on the PAG ELISA and TU that agreed were considered correct, whereas the pregnancy status of cows in which pregnancy outcomes between PAG and TU disagreed were reassessed by TU 5 d later. The accuracy of pregnancy diagnosis was less than expected when using TU 27 d after TAI (93.7 to 97.8%), especially when pregnancy outcomes were based on visualization of chorioallantoic fluid and a corpus luteum but when an embryo was not visualized. The accuracy of PAG ELISA outcomes 27 d after TAI was 93.7, 95.4, and 96.2% for first, second, and third postpartum TAI services, respectively. Statistical agreement (kappa) between TU and the PAG ELISA 27 d after TAI was 0.87 to 0.90. Pregnancy outcomes based on the PAG ELISA had a high negative predictive value, indicating that the probability of incorrectly administering PGF2{alpha} to pregnant cows would be low if this test were implemented on a commercial dairy.

Key Words: transrectal ultrasonography • pregnancy-associated glycoprotein • pregnancy diagnosis


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 ACKNOWLEDGEMENTS
 REFERENCES
 
Efficient and aggressive reproductive management of lactating dairy cows can be achieved if an accurate early nonpregnant diagnosis is combined with a resynchronization protocol, resulting in acceptable fertility. Palpation per rectum is routinely used to determine pregnancy status in dairy cattle; however, with this method it can be difficult to diagnose pregnancy status accurately earlier than 30 to 35 d after AI (Momont, 1990; Youngquist, 2007). Use of transrectal ultrasonography (TU) is gaining popularity among bovine practitioners (Fricke, 2002), but accuracy of pregnancy diagnosis by TU decreases at a gestational age of less than 33 d (Pieterse et al., 1990; Badtram et al., 1991; Romano et al., 2006).

Laboratory assays for detecting proteins originating from binucleate cells of the embryonic trophoblast have been developed to determine pregnancy status in cattle (Sasser et al., 1986; Zoli et al., 1992; Green et al., 2005). Pregnancy-specific protein-B was the first pregnancy-specific protein identified in cattle (Butler et al., 1982) and was later found to have the same N-terminal AA sequence as pregnancy-associated glycoprotein (PAG; Xie et al., 1991; Lynch et al., 1992). Both pregnancy-specific protein-B and PAG have subsequently been re-classified as boPAG-1, and an ELISA was developed to detect PAG as a method of early pregnancy diagnosis in cattle (Green et al., 2005). Mean PAG concentrations in cattle increase from 15 to 35 d in gestation; however, variation in serum PAG levels among cows precludes PAG as a reliable indicator of pregnancy until about 26 to 30 d in gestation (Zoli et al., 1992; Humblot, 2001). Coupling an early nonpregnant diagnosis with a management strategy to rapidly reinitiate AI can improve reproductive efficiency by decreasing the interval between AI services, thereby improving the AI service rate (Fricke, 2002).

The objective of this study was to compare the accuracy of a plasma PAG ELISA with TU for determining the pregnancy status of lactating dairy cows 27 d after timed AI (TAI). When a new test is evaluated, it is necessary to analyze the accuracy and feasibility of the new test compared with existing methods (Bossuyt et al., 2006). A "gold standard" is a quality control that provides the basis for determining the value of a diagnostic test. The sensitivity of TU (the gold standard) to detect pregnant cows and heifers from 23 to 33 d after AI varies from 61.5 to 97.7%, whereas the specificity of TU to detect nonpregnant animals varies from 76.6 to 87.8% (Pieterse et al., 1990; Badtram et al., 1991). Nonetheless, TU is the most reliable method of determining pregnancy status 27 d after TAI under farm conditions and can be performed concurrently with blood sample collection for the PAG ELISA 27 d after TAI.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS AND DISCUSSION
 CONCLUSIONS
 ACKNOWLEDGEMENTS
 REFERENCES
 
Animals and Reproductive Management
Lactating Holstein cows (n = 1,079) on a commercial dairy with approximately 1,600 lactating cows near DeForest, Wisconsin, were enrolled into the study from December 2004 to August 2005. Cows were housed in free-stall barns and were fed a TMR with ad libitum access to feed and water. Cows were milked 3 times daily, and all cows received recombinant bST (Posilac, 500 mg, Monsanto Co, St. Louis, MO) beginning 57 to 70 d postpartum and continuing every 14 d throughout the study. Initiation of the first postpartum bST treatment coincided with the Presynch + Ovsynch protocol as described by Moreira et al. (2000).

Lists for scheduled injections and pregnancy examinations for individual cows were generated weekly by using a commercial on-farm computer software program (Dairy Comp 305, Valley Agricultural Software, Tulare, CA). This program also was used to track the cows enrolled in the study and to track reproductive outcomes and events for individual cows. Data from "cowfile" archives were exported into a computer spreadsheet program (Microsoft Excel 2002, Microsoft Corporation, Redmond, WA) for organization before statistical analysis by SAS (SAS Institute, 2003).

Submission of Cows for the PAG ELISA Analysis
Lactating Holstein cows were allocated weekly to breeding groups based on their date of calving, and cows were managed in groups to receive hormonal injections on 2 preselected days of the week (Tuesdays and Thursday), with TAI conducted on Friday mornings. All cows received a Presynch + Ovsynch protocol by intramuscular injection of 25 mg of PGF2{alpha} (5 mL of Lutalyse, Pfizer Animal Health, New York, NY) 39 ± 3 and 53 ± 3 d after parturition. Twelve days later, Ovsynch was initiated by administering 100 µg of GnRH (Cystorelin, Merial Ltd., Duluth, GA) and 25 mg of PGF2{alpha} (5 mL of Lutalyse, Pfizer Animal Health). The Ovsynch protocol for first postpartum TAI was GnRH (d 65 ± 3), PGF2{alpha} (d 72 ± 3), and GnRH 54 h after PGF2{alpha}, followed by TAI ~16 h later (d 75 ± 3 postpartum). Cows were enrolled in the study at the second PGF2{alpha} injection of Presynch, and cows with a BCS of ≤2.0 (Wildman et al., 1982) were not enrolled in the study based on criteria established by the herd manager.

The first pregnancy diagnosis was conducted 27 d after first TAI; thus, a minimum of 100 d elapsed between calving and the first pregnancy diagnoses with the PAG ELISA. A second pregnancy diagnosis was conducted 39 d after TAI by using TU, and cows diagnosed as pregnant at this diagnosis completed the study. Cows failing to conceive to first postpartum TAI were resynchronized by Ovsynch as described for first postpartum TAI, with the first GnRH injection administered either 25 or 32 d after TAI. Cows failing to conceive to second postpartum TAI were resynchronized by Ovsynch for a third postpartum TAI service. Cows remained in the study until they were diagnosed as pregnant or until they completed the pregnancy diagnosis conducted 39 d after their third postpartum TAI service. The herd veterinarian, who had 8 yr of experience with TU, conducted all pregnancy examinations 27 d after TAI by using a high-quality portable scanner equipped with a 5- to 10-MHz linear-array transducer (Sonosite VET 180plus, SonoSite Inc., Bothell, WA). Pregnancy examination 39 d after TAI was performed by a second veterinarian with a portable scanner equipped with a 5- to 10-MHz linear-array transducer (Easi-scan, BCF Technology Ltd., Livingston, UK).

Blood Sample Collection for the PAG ELISA
Blood samples used for the PAG ELISA were collected 27 d (Thursday) after TAI throughout the trial into 3-mL K3 EDTA evacuated tubes (BD Vacutainer, Franklin Lakes, NJ). Blood samples were collected via venipuncture of the median caudal vein or artery. Immediately after collection, samples were transported to the laboratory, placed on ice, and shipped as whole blood from the University of Wisconsin-Madison to the Monsanto Company by a commercial overnight shipping courier (FedEx Corporation, Memphis, TN).

Blood samples were analyzed in a laboratory located at the Monsanto Company for PAG concentration by using ELISA (Harlow and Lane, 1998) as described by Green et al. (2005), with slight modifications. Briefly, 96-well ELISA plates were coated with rabbit anti-PAG polyclonal antibodies in coating buffer (0.1 M Na2CO3 buffer, pH 9.35) and allowed to incubate overnight at 4°C. The plates were then washed 4 times (200 µL/well for each wash) with wash buffer (PBS, pH 7.4, containing 0.05% Tween 20) by an automatic 96-well plate washer (ELx405, BioTek, Winooski, VT). Blocking solution (200 µL/well) was then added to each well and the plates were incubated for 1 h at 37°C. After the 1-h incubation, the blocking solution was removed and the wells were washed 4 times with 300 µL of wash buffer by the plate washer. After the final wash, either 100 µL of plasma collected from a study animal or a prediluted PAG standard in blocking buffer was added to duplicate wells. Blocking buffer also was used as the blank. The plates were then incubated at 37°C for 1 h. After this incubation, plates were washed 4 times with 300 µL of wash buffer by the plate washer. Biotin-labeled PAG antibody (100 µL/well) diluted in blocking buffer was added to each well and incubated for 1 h at 37°C. After incubation, the plates were again washed 4 times with 300 µL of wash buffer. Streptavidin-horse-radish peroxidase (100 µL/well, diluted in blocking buffer) was added to each well and incubated for 1 h at 37°C, and the plates were again washed 4 times with 300 µL of wash buffer. After washing, horseradish peroxidase substrate solution (100 µL/well) was added to each well and incubated at room temperature (~25°C, 15 min with shaking) to allow color development. Color development was stopped by adding 1 M hydrochloric acid (100 µL/well). A SpectraMax Plus Microplate Reader (multidimensional scaling) was used to measure the absorbance. A SoftMax Pro instrument (multidimensional scaling) was used to estimate PAG concentration in each well by using the standard curve and a linear regression plot. A standard curve was included on every ELISA plate.

Plasma samples from study cows with a PAG concentration greater than a preestablished cutoff value were identified as pregnant, whereas plasma samples from cows with a PAG concentration less than the preestab-lished lower cutoff value were identified as nonpregnant. The preestablished cutoff value was determined by Monsanto laboratory personnel and remained unknown to laboratory personnel at the University of Wisconsin-Madison throughout the trial. Pregnancy outcomes based on TU were unknown to Monsanto laboratory personnel until they delivered PAG ELISA outcomes. Pregnancy outcomes based on the PAG ELISA were delivered to the laboratory personnel at the University of Wisconsin-Madison via e-mail and were subsequently conveyed to the farm manager. Overall turnaround time from blood sample collection to the return of the pregnancy outcomes to the farm was 36 h.

Exclusion of PAG ELISA Outcomes
A total of 1,079 cows were initially enrolled in the study; however, 37 cows were excluded from further analysis before their first pregnancy diagnoses because they were either sold, died, or failed to complete the correct hormonal injection sequence during the Presynch + Ovsynch protocol. Thus, a total of 1,042 cows were available for first pregnancy diagnosis. Some outcomes were not included in the PAG ELISA accuracy analysis for the reasons summarized in Table 1Go. Briefly, reasons for exclusion included 1) cows missing a blood sample for the PAG ELISA (n = 97) or missing a TU evaluation (n = 4); 2) a malfunction of the 96-well plate washer (110 pregnancy outcomes were excluded because of this occurrence); 3) an inconclusive PAG result provided by the Monsanto laboratory (12 outcomes were excluded); 4) d 27 TU results that were inconclusive and did not meet protocol-defined category criteria (28 outcomes were excluded that occurred during the first month of the study, and this was corrected); 5) data for cows in which pregnancy outcomes between the PAG ELISA and TU disagreed if pregnancy reevaluations were missing.


View this table:
[in this window]
[in a new window]

  		Table 1. Number of cows and pregnancy outcomes excluded for the pregnancy-associated glycoprotein (PAG) ELISA analysis by timed AI (TAI) number
 
For the analysis of first postpartum TAI, 160 total outcomes were excluded from the initial 1,042 cows, resulting in 882 outcomes in the PAG ELISA accuracy analysis at first postpartum TAI. After first postpartum TAI, a total of 399 cows were determined pregnant 39 d after TAI and did not have a second TAI; another 91 cows did not continue resynchronization in the interval between first and second postpartum TAI, resulting in 552 cows available for the analysis at second postpartum TAI.

For the analysis of second postpartum TAI, a total of 74 outcomes were excluded, resulting in 478 outcomes included in the PAG accuracy assessment. A total of 169 cows were pregnant after second postpartum TAI and another 34 cows were excluded before the third pregnancy diagnosis because they did not continue resynchronization. Thus, 349 cows remained in the study for analysis of third postpartum TAI, and 313 outcomes were included in the analysis after exclusion of 36 outcomes.

In summary, a total of 1,042, 552, and 349 cows were available for the PAG ELISA analysis at first, second, and third postpartum TAI, respectively. After exclusions (summarized in Table 1Go), a total of 882, 478, and 313 outcomes were available for analysis of the PAG ELISA at first, second, and third postpartum TAI, respectively.

Assessment of PAG ELISA Accuracy
This experiment was designed to test the accuracy of pregnancy outcomes based on a PAG ELISA of blood samples collected 27 d after TAI by comparing these outcomes with those based on TU conducted 27 d after TAI. With this experimental design, pregnancy outcomes by TU served as a gold standard with which to test the accuracy of the PAG ELISA. Throughout the experiment, pregnancy outcomes between the PAG ELISA and TU were compared, and if a given cow had a missed PAG ELISA or TU outcome, the cow was not included in the analysis of PAG ELISA accuracy. When pregnancy outcomes based on the PAG ELISA and TU agreed, the outcome was considered to be correct. By contrast, when pregnancy outcomes based on the PAG ELISA and TU 27 d after TAI disagreed for a given cow, pregnancy status of that cow was reevaluated by TU 5 d later, 32 d after TAI. Cows in which pregnancy outcomes based on TU were incorrect 27 d after TAI resulted in an incorrect gold standard outcome. Thus, incorrect TU outcomes 27 d after TAI were adjusted to the pregnancy outcome based on the pregnancy recheck by TU 32 d after TAI to avoid inaccuracy in the evaluation of the PAG ELISA. After the appropriate adjustments were made by using the TU outcomes 32 d after TAI, the actual gold standard was based on TU outcomes 27 d after TAI that agreed with the PAG ELISA or the result of the TU outcomes 32 d after TAI if the outcomes between the 2 diagnostic methodologies 27 d after TAI did not agree. This new gold standard was used to estimate the accuracy of both TU and the PAG ELISA 27 d after TAI.

During the initial month of the study, TU outcomes were classified into each of the following 3 categories: 1) pregnant (PG; embryo visualized); 2) questionable pregnant (QP; chorioallantoic fluid visualized; embryo not visualized); or 3) nonpregnant (NP). Because some pregnancy outcomes based on TU 27 d after TAI were found to be incorrect according to the pregnancy recheck conducted 32 d after TAI, pregnancy outcomes based on TU were subclassified into each of the 5 categories defined in Table 2Go. Therefore, outcomes during the first month of the study were excluded from the analysis of the accuracy of the PAG ELISA. Agreement between TU and the PAG ELISA was determined by considering cows in the PG, QP1 (pregnant based on the presence of a normal amount of chorioallantoic fluid and a corpus luteum), or QP2 (pregnant based on the presence of abnormally low amount of chorioallantoic fluid and a corpus luteum) categories as pregnant and by considering cows in the pregnancy loss (PL) or NP categories as nonpregnant (Table 2Go).


View this table:
[in this window]
[in a new window]

  		Table 2. Category definitions used to classify pregnancy outcomes based on transrectal ultrasonography (TU) examinations conducted 27 d after timed AI
 
Statistical Analyses
The kappa statistic in PROC FREQ of SAS (SAS Institute, 2003) was used to analyze agreement between the pregnancy outcomes of TU and the PAG ELISA. A kappa value of 1 indicates perfect agreement and a value of 0 indicates no agreement beyond chance (Martin et al., 1987; Noordhuizen et al., 2001). In comparing tests, a kappa value of 0.4 to 0.5 indicates a moderate level of agreement, 0.5 to 0.6 indicates good agreement, and >0.6 indicates a high level of agreement (Martin et al., 1987).

McNemar’s test was used to compare proportions for data from matched pairs. A matched pair occurs when each observation in the first group has a corresponding observation in the second group (Pagano and Gauvreau, 2000). In the present study, paired data included the PAG ELISA and TU pregnancy outcomes for each cow and time period. The chi-square test statistic in PROC FREQ of SAS was used to analyze the nonpaired data of incorrect TU outcomes in each category. Fisher’s exact test was used when the cell frequency was low and the chi-squared test was not appropriate (Table 3Go).


View this table:
[in this window]
[in a new window]

  		Table 3. Frequency of pregnancy outcomes based on transrectal ultrasonography (TU) categories 27 d after timed AI (TAI) and the frequency of incorrect TU outcomes based on pregnancy status reevaluation by TU 32 d after TAI
 
The sensitivity of the assay was expressed as the proportion of pregnant cows with a positive PAG ELISA result [number of true positive results/(number of true positive results + number of false negative results)]. By contrast, test specificity was calculated as the proportion of nonpregnant cows with a negative test result [number of true negative results/(number of true negative results + number of false positive results)]. The positive predictive value (PPV) was calculated as the proportion of cows testing pregnant that were truly pregnant [number of true positive results/(number of true positive results + number of false positive results)], whereas the negative predictive value (NPV) was calculated as the proportion of cows testing negative that were not truly pregnant [number of true negative results/(number of true negative results + number of false negative results)]. Test accuracy was defined as the proportion of pregnant and nonpregnant cows correctly identified by the test [(number of true positive results + number of true negative results)/(number of true positive results + number of true negative results + number of false positive results + number of false negative results); Martin et al., 1987; Smith, 1991; Noordhuizen et al., 2001]. Two additional calculations included determination of the rate of false positive and false negative results. The rate of false positive results is the likelihood of a positive result in cows known not to be pregnant, and this rate is related to the test specificity (rate of false positive = 1 – specificity). By contrast, the rate of false negative results is the likelihood of a negative result in cows known to be pregnant, and this rate is related to the test sensitivity (rate of false negative results = 1 – sensitivity).


   RESULTS AND DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
CONCLUSIONS
 ACKNOWLEDGEMENTS
 REFERENCES
 
Accuracy of Pregnancy Outcomes by TU
The frequency distribution of pregnancy outcomes for each TU category is summarized in Table 3Go. The percentage of cows assigned to each category was 17.4, 19.6, 3.4, 0.7, and 58.9% for the categories PG, QP1, QP2, PL, and NP, respectively. The percentage of pregnant (PG, QP1, and QP2) and nonpregnant cows (PL and NP) was 40.5 and 59.5%, respectively. The percentage of cows diagnosed as pregnant based on visualization of an embryo by TU (PG, 17.4%) was similar to the number of cows diagnosed as pregnant based solely on the presence of chorioallantoic fluid and a corpus luteum but without visualizing an embryo (QP1, 19.6%). For nonlactating heifers evaluated under optimal experimental conditions and with no time constraints, the embryonic vesicle was first detectable at about 12 d of gestation, and detection of the embryo proper was possible by 20 d (Curran et al., 1986). Circular nonechogenic structures within the uterus were common in non-inseminated and pregnant heifers 10 to 14 d after ovulation, and by 16 d there were more elongated nonechogenic structures in pregnant heifers (Kastelic et al., 1991a). Although pregnancy status can be determined early during gestation, the accuracy of TU is compromised because of the location of the uterus far cranial to the pelvic inlet, early pregnancy loss, age of the cows, and accumulation of intrauterine estrual fluid in non-pregnant cows (Hughes and Davies, 1989; Kastelic et al., 1991a; Szenci et al., 1995; Szenci et al., 1998a). For the TU outcomes that disagreed with the PAG ELISA, the percentage of incorrect TU outcomes was less for nonpregnant than for pregnant outcomes (26.5%, n = 83 vs. 70.4%, n = 98), mainly because of the incorrect TU outcomes classified as either QP1 or QP2. Important considerations when interpreting data from the present study are that TU outcomes were made by only one bovine practitioner, and all of the pregnancy outcomes were conducted on a single farm. Variation among farms in the rate of embryonic loss as well as variation in the skill at TU among practitioners could result in different outcomes among farms and practitioners.

One caveat of this study is that we assumed that when both diagnostic methodologies agreed, the resulting pregnancy outcomes were correct. Although it is possible that both tests could be wrong simultaneously, the frequency of such an occurrence would be expected to be low and would have a minimal impact on the data. Furthermore, restraining all cows enrolled into the study at both 27 and 32 d after TAI to determine pregnancy status was not possible because it would have disrupted cow flow on the farm and because cows would have been restrained in feedline headlocks for longer than the farm manager allowed. By contrast, when disagreement between pregnancy outcomes based on TU and the PAG ELISA occurred, pregnancy outcomes by TU 5 d later were used to determine the correct diagnosis. Overall, 200 pregnancy outcomes disagreed between the PAG ELISA and TU 27 d after TAI, resulting in 181 reevaluations based on TU conducted 32 d after TAI. A total of 19 cows that required pregnancy rechecks were missed, and these data were excluded from the analysis. After exclusion of these outcomes, a total of 1,673 pregnancy outcomes were evaluated for the accuracy of the PAG ELISA for first, second, and third postpartum TAI services.

After reevaluation of outcomes that disagreed, the overall percentage of incorrect TU outcomes 27 d after TAI was greater for cows diagnosed as pregnant based on the presence of abnormal uterine fluid and a corpus luteum (i.e., QP2, 57.4%) than when a normal amount of fluid (i.e., QP1, 9.5%) or an embryo was visualized (i.e., PG, 2.4%). In a previous study in which a pregnancy diagnosis between 26 and 58 d after AI was evaluated by TU, more false positive diagnoses were made when visualization of chorioallantoic fluid alone was the determining criterion compared with when the embryo proper was visualized (Szenci et al., 1998b).

A portion of cows in the QP1 and QP2 categories misdiagnosed as pregnant may have been undergoing pregnancy loss at the time of the TU examination. Differences in these categories based on the amount of chorioallantoic fluid detected by TU may be explained by differences in the timing of embryonic death. When embryonic death (spontaneous or induced) in heifers preceded luteal regression, the conceptus fluid and embryonic tissue were retained longer in the uterus than when luteolysis was induced (Kastelic and Ginther, 1989; Kastelic et al., 1991b). This delay in expulsion of the conceptus from the uterus may have produced false positive results when using TU in the present study. The number of cows diagnosed as PL (n = 11, 0.7%) was low because cows that initiated pregnancy loss before TU examination were probably categorized as either QP1 or QP2. Furthermore, embryonic death is usually diagnosed based on visualization of the embryo proper by TU, and cows in the present study were classified into the QP1 and QP2 categories based on visualization of chorioallantoic fluid alone. Overall, less than half (43.1%, 295/685) of the pregnant outcomes were based on visualization of an embryo (i.e., PG) probably because of the small mass of the embryo 27 d after TAI and the time constraints for individual cow diagnoses imposed by the cow flow on the commercial dairy. Four cows classified as PL by TU disagreed with the PAG ELISA outcome (pregnant), and 2 of these cows were found to be pregnant 5 d later based on the recheck by using TU.

Sensitivity, specificity, PPV, and NPV of TU 27 d after TAI were calculated based on the assumption that the outcomes that agreed with the PAG ELISA were correct, whereas the outcomes that disagreed were readjusted to the correct outcome based on the TU reevaluation 32 d after TAI (Table 4Go). A 2 x 2 contingency table was constructed to analyze the data (Table 5Go) and calculate sensitivity, specificity, PPV, NPV, and accuracy of the TU (Table 6Go). In this study, sensitivity ranged from 94.2 to 98.9% and specificity ranged from 91.7 to 97.3%. Pieterse et al. (1990) reported a sensitivity and specificity of pregnancy diagnosis by ultrasound of 44.8 and 82.3%, respectively, when conducted between 21 and 25 d after AI and 97.7 and 87.7%, respectively, when conducted between 26 and 33 d after AI. In a second field study with TU 27 d after AI to determine pregnancy status in cows, sensitivity and specificity were 93.8 and 96.2%, respectively (Romano et al., 2006), similar to results from the present study for pregnancy outcomes after first, second, and third postpartum TAI.


View this table:
[in this window]
[in a new window]

  		Table 4. Pregnancy outcomes based on transrectal ultrasonography (TU) 27 d after timed AI (TAI), and adjusted outcomes for TU 27 and 32 d after TAI
 

View this table:
[in this window]
[in a new window]

  		Table 5. Contingency table for evaluation of sensitivity,1 specificity,2 positive predictive value,3 negative predictive value,4 and accuracy5 of transrectal ultrasonography (TU) for determining pregnancy status 27 d after timed AI
 

View this table:
[in this window]
[in a new window]

  		Table 6. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of transrectal ultrasonography (TU) for determining pregnancy status 27 d after timed AI (TAI) by TAI number
 
Because some errors occurred with pregnancy outcomes 27 d after TAI based on TU (i.e., our gold standard), a different approach was taken and incorrect TU outcomes were not included in the analysis in an attempt to minimize bias in determining the accuracy of the PAG ELISA due to errors in the gold standard (Enøe et al., 2000). The use of pregnancy outcomes based on TU 32 d after TAI for outcomes that disagreed at the initial examination avoided underestimation of the accuracy of the PAG test due to incorrect TU outcomes. A fair criticism of this methodology is that pregnancy reevaluations conducted 5 d after the initial evaluation did not account for pregnancy loss occurring during the 5-d interval between pregnancy evaluations and therefore may have introduced some error in determining the correct outcome at the initial pregnancy examination. Although this is a possibility, the occurrence of pregnancy loss was only 3.7% (13/349) from 33 to 40 d after TAI (Sterry et al., 2006). Furthermore, this methodology assumed that when the diagnoses between the methodologies agreed, the outcomes were correct. Finally, the exclusion of data when the d 27 TU and PAG ELISA results disagreed but no reevaluation was conducted (n = 19) may have resulted in a slight overestimation of the agreement between the 2 diagnostic methodologies.

Results from the present study and those of others support the notion that pregnancy outcomes based on TU before about 30 d after TAI can lead to errors, which may substantially reduce the benefit of early pregnancy diagnosis. From a practical perspective, although there is an advantage of the PAG ELISA over TU with regard to the false positive results at 27 d after TAI associated with embryonic loss, the 2-d delay from the time of blood collection to the establishment of pregnancy diagnosis based on the PAG ELISA has a negative impact on the reproductive management program of a dairy implementing a systematic synchronization and resynchronization program. With TU, cows treated with GnRH 7 d before pregnancy diagnosis to initiate Resynch can be diagnosed as nonpregnant and be immediately treated with PGF2{alpha} during the same cow-handling period (Fricke et al., 2003; Sterry et al., 2006). By contrast, an additional cow-handling period is required during Resynch to collect the blood sample for the PAG ELISA at least 2 d before the scheduled PGF2{alpha} injection. Development of an on-farm or cow-side form of this PAG assay would improve the management aspects of adopting this technology on a dairy. Furthermore, results of studies evaluating the timing of initiation of Resynch indicate that the most aggressive strategies, in which Resynch is initiated 19 or 26 d after a previous TAI, result in lower fertility compared with initiation of Resynch 32 or 33 d after TAI (Fricke et al., 2003; Sterry et al., 2006). Thus, both the efficacy of and the need for determining pregnancy status as early as 26 d after a previous TAI need to be questioned when deciding when to position a pregnancy diagnosis within a reproductive management strategy that uses a systematic synchronization and resynchronization approach.

Assessment of PAG ELISA Accuracy
A 2 x 2 contingency table was constructed to calculate overall sensitivity, specificity, PPV, and NPV of the PAG ELISA 27 d after first, second, and third postpartum TAI (Table 7Go). Kappa values for the agreement in pregnancy diagnosis between PAG ELISA and TU were similar among the first 3 postpartum TAI services. Kappa values in this study exceeded 0.85, indicating a high level of agreement between PAG ELISA and adjusted TU pregnancy outcomes (Martin et al., 1987).


View this table:
[in this window]
[in a new window]

  		Table 7. Contingency table for evaluation of sensitivity,1 specificity,2 positive predictive value,3 negative predictive value,4 and accuracy5 of pregnancy-associated glycoprotein (PAG) ELISA for determining pregnancy status 27 d after timed AI, considering transrectal ultrasonography (TU) the standard test
 
Values for sensitivity, specificity, PPV, NPV, and accuracy of the PAG ELISA are summarized in Table 8Go. In this study, sensitivity ranged from 93.5 to 96.3% and specificity ranged from 91.7 to 96.8%. Zoli et al. (1992) reported an accuracy similar to the present study for determining pregnancy status 35 d after AI in cows carrying transferred embryos (94.7%, 407/430) based on a PAG RIA, which was compared with rectal palpation 45 d after AI. When comparing PAG ELISA and TU, the rate of false negative results for the PAG ELISA for second postpartum TAI (6.5%) was similar to the rate of false negative results by TU 27 d after TAI was performed on cows (6.2%, Romano et al. 2006), but the rate of false negative results was lower for first and third postpartum TAI (3.7 and 5.4%, respectively). Moreover, the rate of false positive results was greater for the PAG ELISA for first postpartum TAI than for TU in the study by Romano et al. (2006; 8.3 vs. 3.8%), but was similar for second and third postpartum TAI (3.7 and 3.2%, respectively). Thus, both the PAG ELISA and TU had similar sensitivity and specificity 27 d after TAI.


View this table:
[in this window]
[in a new window]

  		Table 8. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of pregnancy-associated glycoprotein (PAG) ELISA 27 d after timed AI (TAI) by TAI number
 
The occurrence of pregnancy loss may account for some of the false positive results of the PAG ELISA. The half-life of PAG in maternal circulation was 4.3 d during the postcalving period (Green et al., 2005) and 2.7 to 7.0 d after induction of embryonic mortality (Semambo et al., 1992; Szenci et al., 2003). During the period of decreasing PAG concentration in maternal circulation after embryonic death to values lower than the cutoff point for pregnancy, PAG could be detected by the ELISA, leading to a false positive diagnosis, whereas TU can be used to visualize embryonic death and reduce the frequency of false positive results. By contrast, if PAG concentrations decrease rapidly after embryonic death, the PAG ELISA may detect losses earlier than TU because of the lag between embryonic death and expulsion of the conceptus (Kastelic and Ginther, 1989; Kastelic et al., 1991b). In the present study, a direct comparison between the PAG ELISA and TU was confounded because the calculation of sensitivity and specificity for one method was based on the outcomes from the other method.

A second possibility that may account for some of the false positive outcomes of the PAG ELISA is detection of circulating PAG originating from the previous gestation. Zoli et al. (1992) developed a specific RIA to characterize PAG concentrations during pregnancy and after calving in dairy and beef cows. Serum PAG concentrations were 0.38 ± 0.13 ng/mL at 22 d of gestation and increased continuously as pregnancy advanced, until 220 d of gestation. After parturition, PAG concentrations decreased steadily to 499.60 ± 267.20 ng/mL at 14 d, 131.70 ± 77.90 ng/mL at 30 d, and 10.10 ± 7.80 ng/mL at 60 d, with undetectable levels achieved only by 100 ± 20 d postpartum (Zoli et al., 1992). Szenci et al. (1998b) reported lower specificity to detect nonpregnant cows 26 to 27 d after AI by using 2 RIA methods (85.1 and 56.7%). The reason for low specificity was likely due to sampling within 70 d after calving, which can increase the rate of false positive outcomes caused by detection of circulating PAG originating from the previous gestation, and thereby reducing the specificity and positive predictive value of the test. In the present study, cows exceeded 100 d after calving at the first pregnancy diagnosis. In addition, the "early" PAG ELISA used in this study does not detect residual PAG beyond 40 d postpartum. In addition, Zoli et al. (1992) reported detection of PAG-like immunoreactivity in 7 of 30 noninseminated heifers (0.3 ± 0.09 to 0.5 ± 0.17 ng/mL) and 3 of 20 bulls (3.01 ± 1.73 to 4.75 ± 1.42 ng/mL). Sasser et al. (1986) and Green et al. (2005) detected PAG 15 d after AI in 3 (n = 21) and 5 (n = 42) animals, respectively, but the source of PAG does not appear to be from a conceptus because placental attachment has not yet occurred at this time. These data suggest a possible cross-reaction with another protein that may lead to false positive results and a reduced specificity of the test.

There was a greater frequency of disagreements between the PAG ELISA and adjusted TU outcomes when the PAG ELISA outcome was a pregnant diagnosis compared with a nonpregnant diagnosis (61 vs. 29, P < 0.01 by McNemar’s test). Of the 29 cows incorrectly classified as nonpregnant based on the PAG ELISA 27 d after TAI, 10 cows continued the resynchronization protocol by receiving an injection of PGF2{alpha}. Among the remaining 19 cows, 4 were diagnosed as nonpregnant 39 d after TAI, whereas 15 remained pregnant until 62 d after TAI. The false-negative results were probably the consequence of low PAG concentration 27 d after TAI and variation in PAG levels among cows (Zoli et al., 1992). When conception occurs, PAG concentration in the maternal circulation is detected as early as 22 to 24 d after AI and increases steadily throughout gestation, peaking before parturition (Sasser et al., 1986; Zoli et al., 1992; Green et al. 2005). Therefore, sensitivity is expected to increase with gestational age, and consequently, the rate of false negative outcomes is expected to decrease. Serum bovine PAG concentrations were 0.38 ± 0.13 ng/mL 22 d after AI and had an average of 8.75 ± 3.04 ng/mL by 28 d after AI (Zoli et al., 1992; Green et al., 2005). Szenci et al. (1998b) reported lower sensitivity to detect pregnant cows before 29 d after AI (75 and 81.2%) than later in gestation when using a PAG RIA, and sensitivity reached nearly 100% by 37 d after AI. The increase in sensitivity is related to the increase in the concentration of PAG in maternal circulation.


   CONCLUSIONS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
ACKNOWLEDGEMENTS
 REFERENCES
 
The PAG ELISA used for determination of PAG concentration in cows had an accuracy of 93.7 to 96.2% 27 d after TAI and is similar to the accuracy of the TU method (93.7 to 97.8%). Results from this study support the view that pregnancy diagnosis by TU 27 d after TAI, based on the presence of chorioallantoic fluid in the uterine horn and a corpus luteum alone, leads to more false positive results than when an embryo is visualized.

Determination of pregnancy status based on plasma PAG concentration 27 d after TAI resulted in acceptable sensitivity and specificity. The negative predictive value of the PAG ELISA was high (96.9 to 97.7%), indicating that few cows would be subjected to induced pregnancy loss because of administration of PGF2{alpha} during the resynchronization protocol. Although the PAG ELISA had an accuracy similar to TU in determining pregnancy status, a direct comparison between methods in this study was confounded because accuracy of TU was based on the PAG ELISA outcome.


   ACKNOWLEDGEMENTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 ACKNOWLEDGEMENTS
REFERENCES
 
The authors thank Blue Star Dairy (De Forest, WI) and their staff for the use of their cows and facilities for this study, and Jerry Guenther and Diego Calderon for their technical assistance and expertise. This research was supported by funding provided by the Monsanto Company, with which P.M.F. has had consulting arrangements.

Received for publication April 11, 2007. Accepted for publication June 21, 2007.


   REFERENCES
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 ACKNOWLEDGEMENTS
 REFERENCES
 


Badtram, G. A., J. D. Gaines, C. B. Thomas, and W. T. K. Bosu. 1991. Factors influencing the accuracy of early pregnancy detection in cattle by real-time ultrasound scanning of the uterus. Theriogenology 35:1153–1167.[CrossRef]

Bossuyt, P. M., L. Irwig, J. Craig, and P. Glasziou. 2006. Comparative accuracy: Assessing new tests against existing diagnostic pathways. Br. Med. J. 332:1089–1092.[Free Full Text]

Butler, J. E., W. C. Hamilton, R. G. Sasser, C. A. Ruder, G. M. Hass, and R. J. Williams. 1982. Detection and partial characterization of two bovine pregnancy-specific proteins. Biol. Reprod. 26:925–933.[Abstract]

Curran, S., R. A. Pierson, and O. J. Ginther. 1986. Ultrasonographic appearance of the bovine conceptus from days 10 through 20. J. Am. Vet. Med. Asssoc. 189:1289–1294.

Enøe, C., M. P. Georgiadis, and W. O. Johnson. 2000. Estimation of sensitivity and specificity of diagnostic test and disease prevalence when the true disease state is unknown. Prev. Vet. Med. 45:61–81.[CrossRef][Medline]

Fricke, P. M. 2002. Scanning the future—Ultrasonography as a reproductive management tool for dairy cattle. J. Dairy Sci. 85:1918–1926.[Abstract/Free Full Text]

Fricke, P. M., D. Z. Caraviello, K. A. Weigel, and M. L. Welle. 2003. Fertility of dairy cows after resynchronization of ovulation at three intervals after first timed insemination. J. Dairy Sci. 86:3941–3950.[Abstract/Free Full Text]

Green, J. A., T. E. Parks, M. P. Avalle, B. P. Telugu, A. L. McLain, A. J. Peterson, W. McMillan, N. Mathialagan, R. R. Hook, S. Xie, and R. M. Roberts. 2005. The establishment of an ELISA for the detection of pregnancy-associated glycoproteins (PAGs) in the serum of pregnant cows and heifers. Theriogenology 63:1481–1503.[CrossRef][Medline]

Harlow, E., and D. Lane. 1998. Immunoassays. Pages 553–612 in Antibodies: A Laboratory Manual. Cold Spring Harbor Laboratory, Cold Spring Harbor, NY.

Hughes, E. A., and D. A. Davies. 1989. Practical uses of ultrasound in early pregnancy in cattle. Vet. Rec. 124:456–458.[Abstract]

Humblot, P. 2001. Use of pregnancy specific proteins and progesterone assays to monitor pregnancy and determine the timing, frequencies and sources of embryonic mortality in ruminants. Theriogenology 56:1417–1433.[CrossRef][Medline]

Kastelic, J. P., D. R. Bergfelt, and O. J. Ginther. 1991a. Ultrasonic detection of the conceptus and characterization of intrauterine fluid on days 10 to 22 in heifers. Theriogenology 35:569–581.[CrossRef][Medline]

Kastelic, J. P., and O. J. Ginther. 1989. Fate of conceptus and corpus luteum after induced embryonic loss in heifers. J. Am. Vet. Med. Assoc. 194:922–928.[Medline]

Kastelic, J. P., D. L. Northey, and O. J. Ginther. 1991b. Spontaneous embryonic death on days 20 to 40 in heifers. Theriogenology 35:351–363.[CrossRef][Medline]

Lynch, R. A., B. M. Alexander, and R. G. Sasser. 1992. The cloning and expression of the pregnancy-specific protein B. Biol. Reprod. 46(Suppl. 1):72. (Abstr.)[CrossRef]

Martin, S. W., A. H. Meek, and P. Willeberg. 1987. Measurement of disease frequency and production. Pages 62–76 in Veterinary Epidemiology: Principles and Methods. 1st ed. Iowa State Univ. Press, Ames.

Momont, H. 1990. Rectal palpation: Safety issues. Bovine Pract. 25:122–123.

Moreira, F., C. A. Risco, M. F. A. Pires, J. D. Ambrose, M. Drost, and W. W. Thatcher. 2000. Use of bovine somatotropin in lactating dairy cows receiving timed artificial insemination. J. Dairy Sci. 83:1237–1247.[Abstract]

Noordhuizen, J. P. T. M., K. Frankena, M. V. Thrusfield, and E. A. M. Graat. 2001. Measurement of disease frequency. Pages 63–82 in Application of Quantitative Methods in Veterinary Epidemiology. 2nd ed. Wageningen Pers, Wageningen, the Netherlands.

Pagano, M., and K. Gauvreau. 2000. Contingency tables. Pages 349–352 in Principles of Biostatistics. 2nd ed. Duxbury, Pacific Grove, CA.

Pieterse, M. C., O. Szenci, A. H. Willemse, C. S. A. Bajcsy, S. J. Dieleman, and M. A. M. Taverne. 1990. Early pregnancy diagnosis in cattle by means of linear-array real-time ultrasound scanning of the uterus and a qualitative and quantitative milk progesterone test. Theriogenology 33:697–707.[CrossRef][Medline]

Romano, J. E., J. A. Thompson, D. W. Forrest, M. E. Westhusin, M. A. Tomaszweski, and D. C. Kraemer. 2006. Early pregnancy diagnosis by transrectal ultrasonography in dairy cattle. Theriogenology 66:1034–1041.[CrossRef][Medline]

SAS Institute. 2003. SAS 9.1, Version 3. SAS Institute Inc., Cary, NC.

Sasser, R. G., C. A. Ruder, K. A. Ivani, J. E. Butler, and W. C. Hamilton. 1986. Detection of pregnancy by radioimmunoassay of a novel pregnancy-specific protein in serum of cows and a profile of serum concentrations during gestation. Biol. Reprod. 35:936–942.[Abstract]

Semambo, D. K. N., P. D. Eckersall, R. G. Sasser, and T. R. Ayliffe. 1992. Pregnancy-specific protein B and progesterone in monitoring viability of the embryo in early pregnancy in the cow after experimental infection with Actinomyces pyogenes. Theriogenology 37:741–748.[CrossRef][Medline]

Smith, R. D. 1991. Evaluation of diagnostic tests. Pages 29–34 in Veterinary Clinical Epidemiology: A Problem-Oriented Approach. Butterworth-Heinemann, Stoneham, MA.

Sterry, R. A., M. L. Welle, and P. M. Fricke. 2006. Effect of interval from timed artificial insemination to initiation of resynchronization of ovulation on fertility of lactating dairy cows. J. Dairy Sci. 89:2099–2109.[Abstract/Free Full Text]

Szenci, O., J. F. Beckers, P. Humblot, J. Sulon, G. Sasser, M. A. M. Taverne, J. Varga, R. Baltusen, and G. Scheckk. 1998b. Comparison of ultrasonography, bovine pregnancy-specific protein B, and bovine pregnancy-associated glycoprotein 1 tests for pregnancy detection in dairy cows. Theriogenology 50:77–88.[CrossRef][Medline]

Szenci, O., J. F. Beckers, J. Sulon, M. M. Bevers, L. Börzsönyi, L. Fodor, F. Kovács, and M. A. M. Taverne. 2003. Effect of induction of late embryonic mortality on plasma profiles of pregnancy associated glycoprotein 1 in heifers. Vet. J. 165:307–313.[CrossRef][Medline]

Szenci, O., Gy. Gyulai, P. Nagy, L. Kovács, J. Varga, and M. A. M. Taverne. 1995. Effect of uterus position relative to the pelvic inlet on the accuracy of early bovine pregnancy diagnosis by means of ultrasonography. Vet. Q. 17:37–39.[Medline]

Szenci, O., M. A. M. Taverne, J. F. Beckers, J. Sulon, J. Varga, L. Börzsönyi, Ch. Hanzen, and Gy. Schekk. 1998a. Evaluation of false ultrasonographic diagnoses in cows by measuring plasma levels of bovine pregnancy-associated glycoprotein 1. Vet. Rec. 142:304–306.[Abstract/Free Full Text]

Wildman, E. E., G. M. Jones, P. E. Wagner, R. L. Boman, H. F. Troutt, and T. N. Lesch. 1982. A dairy cow body condition scoring system and its relationship to selected production characteristics. J. Dairy Sci. 65:495–501.[Abstract/Free Full Text]

Xie, S., B. G. Low, R. J. Nagel, K. K. Kramer, R. V. Anthony, A. P. Zoli, J. F. Beckers, and R. M. Roberts. 1991. Identification of the major pregnancy-specific antigens of cattle and sheep as inactive members of the aspartic proteinase family. Proc. Natl. Acad. Sci. USA 88:10247–10251.[Abstract/Free Full Text]

Youngquist, R. S. 2007. Pregnancy diagnosis. Pages 294–303 in Current Therapy in Large Animal Theriogenology. 2nd ed. R. S. Youngquist and W. R. Threlfall, ed. Elsevier Saunders, Philadelphia, PA.

Zoli, A. P., L. A. Guilbault, P. Delahaut, W. B. Ortiz, and J. F. Beckers. 1992. Radioimmunoassay of a bovine pregnancy-associated glycoprotein in serum: Its application for pregnancy diagnosis. Biol. Reprod. 46:83–92.[Abstract]


This article has been cited by other articles:


Home page
 J DAIRY SCIHome page
R. L. Bamber, G. E. Shook, M. C. Wiltbank, J. E. P. Santos, and P. M. Fricke
Genetic parameters for anovulation and pregnancy loss in dairy cattle
J Dairy Sci, November 1, 2009; 92(11): 5739 - 5753.
[Abstract] [Full Text] [PDF]

Home page
 J DAIRY SCIHome page
E. Silva, R. A. Sterry, D. Kolb, N. Mathialagan, M. F. McGrath, J. M. Ballam, and P. M. Fricke
Effect of interval to resynchronization of ovulation on fertility of lactating Holstein cows when using transrectal ultrasonography or a pregnancy-associated glycoprotein enzyme-linked immunosorbent assay to diagnose pregnancy status
J Dairy Sci, August 1, 2009; 92(8): 3643 - 3650.
[Abstract] [Full Text] [PDF]

Home page
 J DAIRY SCIHome page
J. C. Green, D. H. Volkmann, S. E. Poock, M. F. McGrath, M. Ehrhardt, A. E. Moseley, and M. C. Lucy
Technical note: A rapid enzyme-linked immunosorbent assay blood test for pregnancy in dairy and beef cattle
J Dairy Sci, August 1, 2009; 92(8): 3819 - 3824.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Interpretive Summary
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Silva, E.
Right arrow Articles by Fricke, P. M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Silva, E.
Right arrow Articles by Fricke, P. M.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS