JDS
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Interpretive Summary
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Mark, T.
Right arrow Articles by Sullivan, P. G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Mark, T.
Right arrow Articles by Sullivan, P. G.
J. Dairy Sci. 89:4874-4885
© American Dairy Science Association, 2006.

Multiple-Trait Multiple-Country Genetic Evaluations for Udder Health Traits

T. Mark*,{dagger},1 and P. G. Sullivan{ddagger},§

* Interbull Centre, Department of Animal Breeding and Genetics, Swedish University of Agricultural Sciences, Box 7023, 750 07 Uppsala, Sweden
{dagger} Department of Large Animal Sciences, The Royal Veterinarian and Agricultural University, Grønnegårdsvej 8, 1870 Frederiksberg C, Denmark
{ddagger} Beef Improvement Ontario, 660 Speedvale Ave. W., Suite 102, Guelph, Ontario N1K 1E5, Canada
§ Canadian Dairy Network, 150 Research Lane, Guelph, Ontario N1G 4T2, Canada

1 Corresponding author: thm{at}kvl.dk


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 APPENDIX 1
 APPENDIX 2
 ACKNOWLEDGEMENTS
 REFERENCES
 
International genetic bull evaluations of somatic cell counts (SCC) from 8 different Holstein populations and clinical mastitis from 3 of these populations were inferred simultaneously using a multiple-trait–multiple-country evaluation (MT-MACE) model. This model considered effective independent weighting factors and multivariately deregressed national genetic evaluations for countries with multiple-trait national models. Predictions of genetic merit from MT-MACE and their reliabilities were compared with the corresponding results from 2 separate single-trait–multiple-country evaluations (ST-MACE) for different groups of bulls. The assumed heritabilities for clinical mastitis (h2 = 0.02 to 0.05) were substantially lower than the heritabilities for SCC (h2 = 0.08 to 0.27). The predictive ability of MT-MACE was essentially equal to or better than the predictive ability of ST-MACE for all country-trait combinations, but both methods yielded effectively unbiased and consistent consecutive predictions (correlation > 0.93). Both sets of predictions also agreed well with future national genetic evaluations for bulls receiving additional daughter information (correlation > 0.96), except for evaluations for which within-country correlations were utilized internationally, but not nationally (correlation = 0.86 to 0.97). The reliabilities for MT-MACE were essentially equal to or higher than reliabilities for ST-MACE, depending on the trait and group of bulls in question. Reliabilities increased most for young bulls, and for clinical mastitis in countries that did not use the within-country correlations with SCC in the national evaluation (up to a 23% increase in average reliability).

Key Words: multiple trait • international genetic evaluation • clinical mastitis • somatic cell concentration


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 APPENDIX 1
 APPENDIX 2
 ACKNOWLEDGEMENTS
 REFERENCES
 
Udder health is one of the most important traits economically in dairy cattle production. Resistance to clinical mastitis (CM) is the trait of main interest regarding udder health, but resistance to subclinical mastitis is also important. Clinical mastitis is recorded in only a few countries, whereas SCC, usually in the form of SCS, is widely used as an indicator of both clinical and subclinical mastitis. In addition to SCC, some countries also consider correlated information from conformation traits and milking speed in their udder health indexes. Thus, several sources of information need to be considered simultaneously for an optimal evaluation of udder health.

International genetic evaluations for udder health currently comprise 2 separate evaluations because of limitations in the methodology used and lack of CM recording in most countries—one including SCC from all participating countries and the other including CM from the Nordic countries and SCC from the non-Nordic countries (Mark et al., 2002a). This practice does not use all available information in an optimal way. It is therefore of interest to extend the current single-trait–multiple-country evaluation method (ST-MACE; Schaeffer, 1994) to allow more than one trait per country.

The theory of different multiple-trait–multiple-country evaluation (MT-MACE) models has been described (e.g., Schaeffer, 2001; Liu et al., 2004; Mark et al., 2005) and 2 different MT-MACE strategies have been compared by simulation (Sullivan et al., 2005). The simulation study by Sullivan et al. confirmed the theoretical expectation that MT-MACE methods should be preferred over methods that allow only one trait per country, when assuming true genetic parameters. There was no clear preference between the 2 MT-MACE strategies investigated based on mean square error of prediction, but the strategy using effective independent weighting factors (Sullivan and Wilton, 2001) is simpler to implement compared with the strategy using multivariate weighting factor blocks (Schaeffer, 2001).

Investigations on field data are of interest before MT-MACE is applied in practice. It is of interest to compare ST-MACE and MT-MACE using estimated parameters and to consider more complex scenarios, such as a mixture of multiple-trait and single-trait genetic evaluations among the countries, when heritabilities differ considerably across traits and countries, and when the distribution of records is uneven across traits within country. The diversity of national genetic evaluation scenarios highlights a fundamental feature of using MACE for international genetic evaluation. Such diversity is available in the udder health data used for routine international genetic evaluations. A similar diversity is also expected for other trait complexes such as fertility and, to a lesser extent, for production traits.

The aim of the present study was to apply MT-MACE to field data for udder health, to quantify the benefits in terms of reliability and ability to predict international genetic merit of MT-MACE compared with ST-MACE, and to discuss the practical implementation of MT-MACE.


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 APPENDIX 1
 APPENDIX 2
 ACKNOWLEDGEMENTS
 REFERENCES
 
National genetic evaluation results for Holstein bulls for CM and SCC from Denmark (DNK), Finland (FIN), and Sweden (SWE) as well as SCC from Canada (CAN), Austria–Germany (DEA), Estonia (EST), France (FRA), and the United States (USA) were considered. A subset of the data was used in the February 2004 Interbull routine evaluation for udder health traits and included populations with different recorded traits, numbers of animals, progeny group sizes, degrees of genetic exchange with other populations, and national genetic evaluation methods (Interbull, 2004a,c; Table 1Go). The SWE and FIN national genetic evaluations were single-trait, whereas a multiple-trait model including both SCC and CM was used in DNK (Nielsen et al., 2000). In the DNK national genetic evaluation, a residual correlation of 0.11 and a genetic correlation of 0.509 were assumed between CM and SCC.


View this table:
[in this window]
[in a new window]
 
Table 1. Number of national breeding values (n), average weighing factor (AVGEDC), standard deviation of weighting factor (SDEDC), and heritabilities (h2) per trait1
 
In addition to the data described in Table 1Go, similar data as used in the November 2003 Interbull routine evaluation from the same countries were used to assess the consistency of consecutive national and international evaluations. There were no changes in the national genetic evaluation models between November 2003 and February 2004, but for all countries, additional daughter records were included. There were 1,616 new bull records across all 11 traits in February 2004 compared with November 2003, and the average effective daughter contributions (EDC; Fikse and Banos, 2001) and number of daughters of the bulls considered for both evaluations increased by 13.9 and 12.1, respectively. Although the amount of available information increased for all countries, the relative increase in the number of bull records differed between countries (e.g., the increase ranged from 0.5% in CAN to 5.4% in EST). The relative increase in the number of qualifying bull records also differed for traits within DNK (2.7 and 1.6% increase for CM and SCC, respectively) and FIN (2.9 and 3.3% increase for CM and SCC, respectively), but not substantially within SWE (1.9 and 1.8% increase for CM and SCC, respectively).

The data edits used in this study were identical to those applied for Interbull routine and test evaluations. That is, 2 main steps were performed in which different edits were imposed: 1) deregression and correlation estimation, and 2) deregression, sire variance estimation, and prediction of international genetic merit. In both of these steps, bulls were required to have daughters in at least 10 herds and to be sampled through an official progeny-testing scheme. For CM, bulls were also required to have at least 50 daughters for the corresponding country.

In the first step, a well-connected subset of bulls having evaluations for more than one country-trait combination (common bulls) as well as bulls that belonged to three-quarters-sib groups that have members with evaluations for more than one country-trait combination were considered. In the second step, only bulls born since 1986 were considered, and imported bulls were required to have at least 150 daughters and daughters in at least 50 herds for the country-trait combination considered.

The data were standardized within country and trait so that each set of national genetic evaluations had equal means (10) and standard deviations (1) and so that all breeding values were expressed in the same direction. This did not change the traits genetically, but facilitated interpretation of results and had some potential computational advantageous.

The ST-MACE was essentially as applied for Interbull routine evaluations (Mark et al., 2002a). The MT-MACE was similar to the ST-MACE, except that effective independent weighting factors (EDCMT; Appendix 1; Sullivan and Wilton, 2001) were used, and observations were based on a multivariate deregression procedure (Schaeffer, 2001). This allowed the MT-MACE to analyze both CM and SCC from the Nordic countries together with SCC from non-Nordic countries, whereas 2 separate analyses were required for the ST-MACE, in which only one trait was considered per country.

MACE Model
The statistical model used in all analyses was based on Schaeffer (1994) except that multiple traits (j) per country (i) were accommodated for in MT-MACE:


Formula

where yij is the vector of within-country univariately or multivariately deregressed national evaluations adjusted for residual correlations (Appendix 1); µij is a fixed effect of the country-trait mean; gij is a vector of random genetic group effects; sij is a vector of random sire effects; eij is a vector of random residuals; Zij is a matrix assigning observations to sire effects; Q is a matrix assigning sires in s to group effects in g. The (co)variance of the random variables was


Formula

where A is the additive genetic relationship matrix relating bulls with their sires and maternal grandsires, I is an identity matrix, G0 is the genetic (co)variance matrix between traits, and Rij is the (co)variance among elements of eij. In this implementation, Rij is a diagonal matrix with diagonal elements equal to {sigma}2e(i,j)/EDCMT(i,j,k) for bull k. The EDCMT are weighting factors (Appendix 1) and {sigma}2e(i,j) are the residual variances. The residual variances were assumed to be equal to [4{sigma}2 sire(i,j)/hij2] – {sigma}2sire(i,j), where {sigma}2sire(i,j) is the sire variance and hij2 is the heritability. The ST-MACE model was identical to the MT-MACE model, except that only one trait was considered per country in ST-MACE. Hence, the original weighting factors and single-trait deregressed national breeding values were used in ST-MACE, so the ST-MACE results did consider within-country correlated information for DNK, although in a suboptimal way.

MT-MACE Computation Steps
Genetic correlations were obtained as follows.

Step 1. Create Effective Independent Weighting Factors for the Correlation Estimation.
The original weighting factors provided by the countries (EDC) were transformed (Appendix 1) to effective independent weighting factors (EDCMT). This transformation allowed residual correlations to be zero in the successive steps. The national heritabilities and national within-country residual and genetic correlations were used for the transformations. This step was necessary only for countries with multiple-trait national models (i.e., DNK), and the original EDC was used for all other countries.

Step 2. Run a Multivariate Deregression of National Genetic Evaluations for the Correlation Estimation.
National genetic evaluations were deregressed across traits and within country according to Schaeffer (2001), but using the weighting factors from step 1. National heritabilities and national genetic correlations were used. This step ensured that genetic information within a country, from related individuals and from correlated traits, was not double-counted between the national and international evaluations. The deregression was only multivariate for countries with multiple-trait national models (i.e., DNK).

Step 3. Estimate Genetic Correlations.
Genetic correlations were estimated based on weighting factors and deregressed national genetic evaluations from steps 1 and 2 using an expectation–maximization REML algorithm applied to a reduced set of MACE equations (Klei and Weigel, 1998). Subsequently, the estimated within-country genetic correlation between CM and SCC for DNK (i.e., the country with a multiple-trait national genetic evaluation) was replaced by the correlation used in the national evaluation and the blended correlation matrix was bent (Jorjani et al., 2004) to ensure positive definiteness. In the bending procedure, the diagonal elements and the within-country correlation for DNK were not allowed to change, whereas the allowed changes for the remaining genetic correlations were inverse-proportional to the number of common bulls.

International breeding values were obtained as follows.

Step 1.
Create effective independent weighting factors for prediction of breeding values, as before, but now for the bulls considered in the prediction of breeding values.

Step 2.
Run a multivariate deregression of national genetic evaluations to predict breeding values, as before, but now for the bulls considered in the prediction of breeding values.

Step 3.
Estimate sire variances using a single-trait expectation–maximization REML algorithm (Sullivan, 1999) and weighting factors and dependent variables from steps 1 and 2.

Step 4.
Predict international genetic merits based on national heritabilities and the parameters, weighting factors, and dependent variables from steps 1 to 3. Obtain solutions by Lower Diagonal Lower (LDL) decomposition (FSPAK90; Misztal and Perez-Enciso, 1998).

Reliabilities for international genetic evaluations were approximated using a multivariate approach to combine information sources (Appendix 2). This approach was used for both ST-MACE and MT-MACE.

Genetic groups were treated as random effects throughout this study, and a minimum group size of 30 was required for the across-country data set. The same genetic group definitions were used in the deregression as in both the ST-MACE and MT-MACE.

Analyses and Comparisons
Evaluations (ST04) and approximate reliabilities from 2 ST-MACE analyses, each involving 8 traits, were combined to resemble current Interbull practice. The first ST-MACE considered SCC evaluations from all 8 countries. The second ST-MACE considered CM evaluations from the 3 Nordic countries (DNK, FIN, and SWE) and SCC data from the other 5 countries. Both ST-MACE analyses were based on univariately deregressed national evaluations and the original weighting factors provided by the countries. The international genetic evaluations for SCC in all countries from the first ST-MACE, and for CM in the Nordic countries from the second ST-MACE, were compared with evaluations (MT04) from the MT-MACE applied to all 11 traits simultaneously. Similar evaluations, based on the November 2003 data, were also conducted for ST-MACE (ST03) and MT-MACE (MT03), respectively. The cross-reference and pedigree files from the February 2004 Interbull routine evaluation, and the genetic correlations estimated for MT04, were used in all analyses. However, within-country sire variances were reestimated for each data set.

Three different groups of bulls were studied: 1) young bulls, that is, bulls that were born in 1997 or later and had daughters in only one country (studied for both the domestic country and the foreign countries where the bulls had no daughters); 2) export bulls, that is, bulls with daughters in at least 2 countries and most daughters in the given country; 3) import bulls, that is, bulls with daughters in the given country, but most daughters in a country other than the given country. Thus, a single bull could be labeled as an export bull in only one country while at the same time being labeled an import bull in 1 to 7 countries. The number of export bulls ranged from one in FIN to 584 in USA. For most countries, the number of import bulls was greater than the number of export bulls, and ranged from 37 for CM in FIN to 777 in USA. Bulls with both ST-MACE and MT-MACE evaluations, for a given trait, were considered for comparisons of the 2 methods.


    RESULTS AND DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 APPENDIX 1
 APPENDIX 2
 ACKNOWLEDGEMENTS
 REFERENCES
 
Estimated Genetic Correlations
The genetic correlations between SCC in one country and SCC in another country ranged from 0.80 to 0.96; the across-country genetic correlation between SCC and CM ranged from 0.51 to 0.73, and the genetic correlations between CM in one country and CM in another country ranged from 0.71 to 0.86 (Table 2Go). The estimated genetic correlation between CM and SCC in DNK was 0.65, but the correlation used in MT-MACE was the correlation used in the national evaluation (i.e., 0.51). The discrepancy between the national and estimated correlations may have been due to the use of different models, data, and pedigrees at the national and international levels, as well as bias in residual MACE (co)variances because of imperfect national weighting factors and heritabilities, or imperfect adjustment for residual correlations. The national correlation was used to ensure that, for bulls with local information only, local evaluations and reliabilities from MT-MACE would align with the corresponding national values. Aligning these results was considered more important in practice than possible improvements in the predictive ability of MT-MACE with the use of new parameters within a country.


View this table:
[in this window]
[in a new window]
 
Table 2. Genetic correlations used for the multiple-country evaluation runs1
 
The estimated within-country genetic correlations for Finland and Sweden may have been affected by residual covariances not accounted for in the respective national genetic evaluations. Thus, they can be interpreted as aggregate correlations due to both genetic and residual coincidences. The estimated within-country genetic correlation for SWE was identical to the genetic correlation between SCC and CM estimated with an animal model applied to raw performance data for SWE first-parity Holstein cows (Carlén et al., 2004). The estimated correlation for FIN was within the range of estimated genetic correlations (0.37 to 0.67) for each of the first 3 parities of FIN Ayrshire cows (Pösö and Mäntysaari, 1996), but it was 0.1 units higher than the genetic correlation estimated by Luttinen and Juga (1997) for the 3 first parities of FIN Friesian cows. However, the models and data used in the 3 studies referred to differed substantially from the respective national evaluations, especially for FIN. Thus, the genetic correlations estimated for MT-MACE should not necessarily be equal to the estimates reported in the literature.

It was necessary to bend the correlation matrix in instances in which one estimated genetic correlation was replaced by the national genetic correlation, but the impacts were small. All changes between the correlation before and after bending were 0.01 or lower, except for the correlation between EST and FIN, which decreased by 0.03 units, and the correlation between CAN and FIN, which increased by 0.02 units. Both of these estimated correlations were based on relatively few common bulls ( < 18).

Estimates of genetic correlation with CM in DNK increased from 0.84 to 0.86 units for CM in SWE and decreased from 0.03 to 0.07 units for SCC in the non-Nordic countries when the correlation estimation was done for MT-MACE instead of ST-MACE. These differences are likely due to the addition of, or the techniques used to add, SCC in DNK to the MT-MACE model. The within-country correlated information was deregressed for the DNK traits in MT-MACE, which explains why the multivariately deregressed national evaluations (e.g., for CM in DNK) were more highly correlated with the deregressed national evaluations of similar traits in other countries (e.g., CM in SWE) compared with the univariately deregressed DNK evaluations, which did contain information due to correlated traits. The multivariate deregression separated the information due to each trait and thereby ensured comparisons of "cleaner" traits at the correlation estimation and MT-MACE level. Although different parameters for ST-MACE relative to MT-MACE may be the most suitable, the correlations used for ST-MACE in this study were the correlations used in MT-MACE to remove the impacts of different genetic correlations from the comparisons.

International Genetic Merits
Impact of ST-MACE vs. MT-MACE.
Predicted international genetic merits did not differ much, regardless of whether ST-MACE or MT-MACE was used (Table 3Go). This result was especially true for the SCC traits, whereas the predictions differed more for CM than for SCC traits between ST-MACE and MT-MACE. The impact of the prediction method was largest for the FIN and especially the SWE CM (results not shown). For example, the correlation between ST04 and MT04 for all bulls was 0.990 and 0.989 for the FIN and SWE CM, respectively. In comparison, the correlation was 0.993 for the DNK CM and 0.997 or higher for the SCC traits.


View this table:
[in this window]
[in a new window]
 
Table 3. Pearson product-moment correlations between international genetic evaluations from single-trait– and multiple-trait–multiple-country evaluations for clinical mastitis (CM) and milk SCC, respectively1
 
For young domestic bulls in each country, respectively, the impact was noticeable only for the SWE and FIN traits. That is, the correlations between ST04 and MT04 were 0.870, 0.959, 0.989, and 0.995 for CM in SWE and FIN and for SCC in SWE and FIN, respectively, whereas it was > 0.999 for all other country-traits. The MT-MACE was essentially changing the national single-trait evaluations to multiple-trait evaluations for SWE and FIN, and this did not influence the evaluations for young domestic bulls in other countries. However, all country-traits were affected for other groups of bulls. The correlation between ST04 and MT04 for young foreign bulls was, for example, lowest for CM in DNK (0.991), followed by CM in SWE and FIN (0.993). Similarly, the correlation for young foreign bulls was 0.999 for SCC in SWE and FIN, whereas it ranged between 0.996 and 0.999 for the other SCC traits.

The ST-MACE and MT-MACE predictions for the traits measured in the Nordic countries were, on average, more consistent for young foreign bulls than young domestic bulls. This was because the vast majority of bulls had daughters only in non-Nordic countries (i.e., only SCC records), and such bulls benefit less from MT-MACE relative to bulls that have both SCC and CM daughter records. When young bulls with daughters only in SWE were considered, correlations between ST04 and MT04 in foreign countries were 0.83 for CM in FIN and 0.86 for CM in DNK, and ranged between 0.98 and 0.99 for the SCC traits. The mean and standard deviation of predicted international genetic merits did not change for different methods of prediction when all bulls were considered.

Spearman rank-correlations between ST04 and MT04 were high and ranged from 0.989 to 0.992 for the CM traits, and from 0.997 to 0.998 for the SCC traits when all bulls were considered. The number of common bulls in the top 100 ST-MACE and MT-MACE rankings (i.e., most favorable breeding values) were 77 bulls for FIN and SWE CM and 89 to 95 bulls for the other traits. Hence, the choice of prediction method is important for the selection of top bulls.

Predictive Ability.
International genetic merits were consistent between consecutive evaluations (correlations > 0.93) and small or no biases were detected (regression coefficients = 0.95 to 1.04) for both ST-MACE and MT-MACE (Table 4Go). This was the case for all country-traits and for both domestic and foreign bulls. The predictive ability of MT-MACE was always essentially equal to or better than the predictive ability of ST-MACE for all country-traits. The MT-MACE method yielded more consistent consecutive evaluations than did ST-MACE for young domestic bulls on the Finnish CM and especially the Swedish CM. The consistency of consecutive evaluations for the non-Nordic SCC were the same for young domestic bulls regardless of the method of prediction, but tended to be slightly higher for MT-MACE for young foreign bulls.


View this table:
[in this window]
[in a new window]
 
Table 4. Regression coefficients from regressions of predicted international genetic merit of February 2004 data on predicted international genetic merit of November 2003 data (b04|03) and correlations between predicted genetic merit in February and November data (r04,03) for young1 bulls for the domestic (d) and foreign (f) countries
 
The predictive advantage of MT-MACE relative to ST-MACE for foreign bulls was more noticeable when young bulls with daughters only in SWE were considered. Then rf04|03 for MT-MACE were 0.990 and 0.989 for CM in DNK and FIN, respectively, compared with 0.985 and 0.984 for ST-MACE. Similarly, rf04|03 were 0.01 units higher for MT-MACE than ST-MACE for all SCC traits.

The comparisons in Table 4Go were based on bulls that had an evaluation for both ST-MACE and MT-MACE in both November 2003 and February 2004 to compare the 2 methods fairly. However, additional predictions were available for MT-MACE that were not available for the applied ST-MACE; for example, bulls with a national CM breeding value, but no SCC breeding value, received MT-MACE predictions for SCC for all countries. With ST-MACE, such bulls did not receive breeding values for the Nordic SCC traits and the SCC breeding values for the non-Nordic countries would, in practice, be disregarded (as they were in this study).

A time span longer than 3 mo between consecutive national evaluations would allow detection of larger differences in predictive ability. However, we were interested in investigating the pure impact of additional daughters for different models, and it was therefore not possible to use data from before November 2003 because modified national genetic evaluation methods and procedures were introduced in November 2003. The standard errors of the regression coefficients presented in Table 4Go were all less than 2.2 x 10–3 for young domestic bulls (highest for EST SCC) and less than 5.0 x 10–6 for young foreign bulls (highest for USA SCC), indicating that the small differences in predictive ability between ST-MACE and MT-MACE (Table 4Go) were significant.

Reliabilities
Reliabilities were generally greater for MT-MACE than ST-MACE (Table 5Go), but overall impacts were small. The impact of ST-MACE vs. MT-MACE differed depending on the group of bulls studied, and the impact was largest for young bulls, because they typically had less information. The relative increase in reliability between consecutive evaluations was slightly higher for ST-MACE than MT-MACE for young bulls, because bulls obtain a given reliability earlier in life with MT-MACE. For each group of bulls, and for young bulls in particular, average reliabilities from either data set were greater with MT-MACE.


View this table:
[in this window]
[in a new window]
 
Table 5. Number of reliabilities and relative increase (%) in average1 reliability2 from ST-MACE to MT-MACE, for different groups of bulls3 and selected country-traits
 
Switching from ST-MACE to MT-MACE had the largest impact on international reliabilities for the SWE CM (1.94% increase on average for all bulls), followed by the FIN CM (1.58% increase, on average, for all bulls). This can be explained by much lower heritabilities for CM than for SCC and that neither SWE nor FIN used correlated information from SCC in their national CM evaluations. The average increase was lowest for the DNK SCC (0.05%), because CM information from DNK was incorporated into the national DNK SCC evaluations that were used in ST-MACE. The largest increase for a non-Nordic country was 0.96% for the USA SCC, which can be explained by a combination of a relatively low heritability for SCC in the USA and the relatively high genetic correlation (0.65) between SCC in the USA and CM in SWE. The bulls with most daughters in SWE had much higher increases in average reliability when going from ST-MACE to MT-MACE compared with other bulls, that is, between a 16 and 23% increase for the 3 CM traits and between an 0.5 and 3.8% increase for the 8 SCC traits (Table 5Go).

General Discussion
Our results, based on analyses of field data, support and supplement the conclusions from the simulation study by Sullivan et al. (2005). In this simulation study, MT-MACE predictions were closer to true breeding values than were ST-MACE predictions. The true parameters were assumed in both MACE strategies, whereas genetic correlations were estimated in our analyses. We showed that the MT-MACE model generalizes well to new data and that statistics comparing consecutive predictions agree well with expectations derived from basic genetic theory.

The data considered in this study were not the best to illustrate the potential benefits of MT-MACE over ST-MACE, because the majority of bulls had daughters only in non-Nordic countries (i.e., only SCC records). However, MT-MACE opens up many other possibilities than were considered here. For example, multiple lactation-specific traits per country could be considered, and reliabilities of predicted genetic merits could be improved by considering udder conformation traits (e.g., udder depth) and milking speed evaluations simultaneously with CM and SCC. The MT-MACE method can generally be used for both national and international purposes, when it is of interest to combine different genetically and residually correlated evaluations.

Weighting Factors.
In this application, MT-MACE was implemented using the modified weighting factor approach of Sullivan and Wilton (2001). Although EDCMT can always be derived, there may be situations in which the MT-MACE model using EDCMT does not produce the same reliabilities as a MT-MACE model using multivariate weighting factor blocks, which include information about residual correlations in off-diagonal elements of the weighting factor matrix (Schaeffer, 2001). This is because of a required constraint to avoid negative EDCMT (Appendix 1) and hence negative definite weighting factor blocks per animal, which could occur because of problems with weighting factors provided by the countries.

An MT-MACE model that explicitly accounts for residual correlations with multivariate weighting factor blocks may have theoretical advantages, especially when countries do not use the residual and genetic correlations in the national evaluations. However, the necessary weighting factor blocks may be difficult to calculate, especially for those countries that do not utilize the covariances among traits in the national models. For such countries, the necessary information to build appropriate weighting factor blocks for MT-MACE may not be available. In principle, the coefficient matrix of a multiple-trait national model, with exactly the same effects as specified in the single-trait models, may be required to derive the correct weighting factor blocks for MT-MACE. However, the effects, incidence matrices, and animals with observations often differ among national models for traits that are evaluated separately. For example, in the present study, FIN used a random regression animal model to evaluate SCC, whereas they used a simpler sire model to evaluate CM. The weighting factor blocks would need to account for these differences between the national models.

For countries with a national multiple-trait genetic evaluation model (e.g., DNK), the weighting factor blocks or approximations thereof would need to be derived from the coefficient matrix of the national model, which could be relatively easy if flexible routines to compute reliabilities are already in place.

In either case, the necessary weighting blocks were not available for this study, and it could be difficult for individual countries to compute the required weighting factor blocks for such a MT-MACE strategy, or even to approximate them in an appropriate and harmonized way. It is therefore more convenient to account for the residual correlations in transforming the weighting factors to effective independent weights, as suggested by Sullivan and Wilton (2001), and it appears to be better to encourage countries to develop multiple-trait national evaluations rather than requiring them to submit weighting factor blocks with off-diagonal elements that may differ from zero and are specific for each bull.

Correlations.
The estimated within-country genetic correlations for FIN and SWE may be biased because of residual correlations not accounted for. Hence, the estimated genetic correlations may be viewed as an "aggregate" residual and genetic correlation. Combining the within-country residual and genetic correlation into the genetic correlation may yield essentially equivalent MT-MACE predictions as if the 2 correlations were estimated and treated separately in MT-MACE, if a similar design is assumed for the 2 traits within country for the deregressed national evaluations of each bull. The aggregate residual and genetic correlation was considered the best possible correlation to use for FIN and SWE in the current implementation of MT-MACE, that is, for a model that assumes residual correlations to be zero. Furthermore, estimation of genetic parameters based on pseudo observations (i.e., evaluations adjusted for fixed effects) has been shown to be robust to ignoring nonzero residual correlations (true re = 0.4) when a sire model is used for the meta-analysis (Tarres et al., 2006). The MACE predictions are also robust to varying genetic correlations (Mark et al., 2005). However, reliabilities for MT-MACE are sensitive to variations in residual and genetic correlations, but we believe that using the aggregate correlations for SWE and FIN yields the most appropriate reliability approximations as well as evaluations.

The estimated genetic correlation between CM and SCC in DNK may be inflated by the positive residual correlation between CM and SCC and the fact that many daughters of bulls had observations for both traits (coincidence). The transformation of the original weighting factors to EDCMT aligns national and international reliabilities for any correlation estimate, reducing the adverse effects of suboptimal correlations on reliabilities. However, by forcing the national genetic correlations on the MT-MACE model, all residual covariances were ignored, instead of ignoring only the variation in residual covariances among deregressed national evaluations. Iterating between parameter estimation and calculation of effective independent weights may circumvent this.

Estimating Correlations and Adjusting Weighting Factors Iteratively.
In such a scheme, estimated correlations would be used to retransform weighting factors, and the new weighting factors and estimated correlations would subsequently be used in a new multivariate deregression, and finally the correlation estimation would be repeated, until convergence.

The final correlation estimates should then be used in MT-MACE instead of the correlations from the national model to ensure that residual correlations are properly accounted for. However, this iterative scheme could dramatically increase computational requirements because the correlation estimation needs to be run repeatedly. Relatively few repetitions may be necessary for this data because only 2 of the considered traits were analyzed simultaneously at the national level. However, convergence may require more iterations for other applications. More efficient correlation estimation algorithms or methods such as AI-REML (Madsen et al., 2000) or pseudo REML may be considered to overcome the increased computational burden. The convergence properties of this strategy should be carefully examined before such a strategy is considered in practice.

Computational Demands.
For very large applications, where memory is a limitation, a Gauss–Seidel iteration may be a viable alternative to LDL decomposition (i.e., exact solutions for breeding values). We compared both approaches. The correlations between international genetic evaluations from the 2 solution methods were consistently 0.9999 or greater for all traits, but the total runtime required for solving the MACE equations was much less with LDL decomposition. Preconditioned conjugate gradient iteration (Strandén and Lidauer, 1999) could also be considered to decrease the time needed to solve the iterative system.

The MT-MACE method is computationally more demanding than a single ST-MACE run, but can be less demanding than multiple ST-MACE runs. For udder health traits, only one simultaneous analysis is needed with MT-MACE instead of the 2 separate ST-MACE runs currently used for international genetic routine evaluations.

The main practical limitation for MT-MACE is the total number of traits that can be accommodated, and hence the number of correlations to be estimated. However, at least 3 traits per country appear to be feasible with the current correlation estimation procedure (Interbull, 2004b), in which several country subsets are analyzed separately and the estimates are subsequently combined (Mark et al., 2005) and bent (Jorjani et al., 2004). Rank reduction of very large correlation matrices may be necessary, however, to avoid near-singular correlation matrices and to improve convergence if iteration is used to predict international genetic merits.

Reduced-rank MACE may reduce problems with unexpected negative partial regressions for bulls with a certain distribution of daughters across countries (van der Beek, 1999), reduce computational requirements, and potentially increase precision of inferences by using data more efficiently (e.g., Kirkpatrick and Meyer, 2004; Mäntysaari, 2004; Leclerc et al., 2005). For MT-MACE, the dimensions of the genetic correlation matrix will increase compared with ST-MACE; hence, it may become more advantageous to reduce the rank of the correlation matrix used in MT-MACE. In the present study, the MT-MACE covariance matrix (Table 2Go) could be reduced from rank 11 to rank 9 and still explain more than 99% of the total additive genetic variance. Rank reduction of the correlation matrix used in MACE may reduce the alignment of national and international predictions for countries with multiple traits because it can be difficult to maintain the within-country correlation structure while taking advantage of the benefits of rank reduction.

Pedigrees and Parameters Used for Deregression Relative to MACE.
That international genetic evaluations and reliabilities are equal to the corresponding national values for bulls with only local information has practical advantages. That is, the deregression is reversible through MACE. In this study, the same genetic group definitions were used for both deregression and MACE. This practice is different from current Interbull practice, in which group definitions for foreign bulls are denser in MACE than in the deregression (Mark et al., 2002b). Although using exactly the same group definitions in both steps ensures reversibility of national and international predictions for bulls with only local information, it may still be advantageous to define groups differently in these 2 steps to reduce model differences between the deregression and the national evaluation. The final MACE step should take advantage of having more information about foreign phantom parents by specifying denser groups, which better account for selection among unknown ancestors. Having several uninformative groups in the within-country deregression means that these group effects will be regressed toward zero, because genetic group effects were random.

For the present data, we compared the ST-MACE results with corresponding results based on a grouping strategy that resembled current Interbull practice (i.e., less dense groups for foreign bulls in the deregression; a minimum of 30 unknown parents required per group). The effect of the different group definitions was negligible for the present data. That is, correlations between ST-MACE predictions (ST04) using the 2 different grouping strategies were > 0.999 for all trait–country combinations. However, this may not be the case for traits that are subject to stronger selection, such as production traits.

The same dilemma applies to the known pedigree information and parameters used in various steps of the international genetic evaluation procedure. The settings of the present study were consistent with current Interbull practice with respect to this. That is, national settings were always used in the deregression, except for the pedigree information. Pedigree files based on information supplied by all countries were used in both deregression and MACE. To be consistent with the grouping strategy currently applied by Interbull, it may be advantageous to use exactly the same pedigree information as provided by the countries in the deregression, while using only the complete information compiled from all countries in the final MACE step. Otherwise, information not accounted for in the national evaluation will be deregressed. Although this information is regressed again in MACE, the result of both steps is that MACE predictions do not take full advantage of pedigrees known in foreign countries, but not in the given country. Although this ensures consistent national and international predictions, the strategy using national pedigree information in the deregression might yield better international predictions for a few bulls.


    CONCLUSIONS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 APPENDIX 1
 APPENDIX 2
 ACKNOWLEDGEMENTS
 REFERENCES
 
This study illustrates that MT-MACE is a flexible method that can be used to simultaneously analyze a complex scenario of different genetic evaluations within and across countries. The predictive ability of MT-MACE was equally good or slightly better than the predictive ability of ST-MACE for all traits, but both methods yielded essentially unbiased and consistent consecutive predictions. The predictive advantage of MT-MACE relative to ST-MACE was more noticeable when only young bulls with clinical mastitis records were considered. The reliabilities for MT-MACE were generally higher compared with reliabilities for ST-MACE, but the difference greatly depended on the trait and group of bulls in question. The most notable increases in reliabilities were for SWE and FIN CM and especially for young bulls with most daughters in these 2 countries. The MT-MACE method is recommended for international genetic udder health evaluations. Further research could focus on developing better correlation estimation procedures and how to best use these estimates in the different MT-MACE steps.


    APPENDIX 1
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 APPENDIX 1
 APPENDIX 2
 ACKNOWLEDGEMENTS
 REFERENCES
 
Effective Independent Weighting Factors
This procedure was first described by Sullivan and Wilton (2001). The objective is to find appropriate weighting factors (EDCMT) for a MT-MACE model that assumes zero residual correlations and nonzero genetic correlations. The use of these weighting factors should yield prediction error variances (PEV) equivalent to those from a MT-MACE model assuming multivariate weighting factor blocks (Schaeffer, 2001). Furthermore, these weighting factors should be generated from the single-trait weighting factors (EDC; Fikse and Banos, 2001) that are used in ST-MACE.

For each animal and each country, let n = diag(EDC), m = diag(EDCMT), r = diag(R), B = [mr–1 + G–1] –1, g = diag(G), and b = diag(B). The vectors EDC and EDCMT, and each of the remaining matrices, have dimensions equal to the number of traits in the given country. Matrix b provides, for each trait, PEV associated with m. The objective is to find m for MT-MACE such that the PEV are equivalent to the PEV associated with n for ST-MACE. That is, b = [nr–1 + g–1] –1 = rg[ng + r] –1 or n = r(gb)(gb) –1.

The problem can be solved by iteration as follows:

  1. i = 0, m(i) = n.
  2. Compute B(i) = [m(i)r–1 + G–1] –1 and extract b(i) = diag[B(i)].
  3. error(i) = n(i) r[gb(i)][gb(i)] –1.
  4. m(i+1) = m(i) + error(i) x relax. 5. i = i + 1.
  5. Until {max[error(i–1)] < {varepsilon} for all traits}, go to step 2.

where {varepsilon} is a low number (e.g., 0.001), 0 < relax < 1 (e.g., 0.5), and EDCMT ≥ 0 for all traits. The latter constraint ensures that bulls do not have a negative number of progeny records observed and guarantees that the MT-MACE equations remain positive definite. In the present study this constraint was not necessary, but it may be required in other situations. If the EDC provided by a country are such that unconstrained EDCMT would be negative for one or more traits, then the resulting reliabilities from MT-MACE based on constrained EDCMT would be larger than would be the case from ST-MACE based on EDC.


    APPENDIX 2
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 APPENDIX 1
 APPENDIX 2
 ACKNOWLEDGEMENTS
 REFERENCES
 
Multivariate Approximation of Reliabilities
The reliability approximation procedure presented here was developed by Sullivan (1999), and it follows the 4 basic steps of the method of Harris and Johnson (1998a,b):

  1. Calculate reliabilities based on progeny records within country.
  2. Incorporate multiple generations of progeny records within country.
  3. Combine within-country reliabilities across countries.
  4. Incorporate parent information by country-trait.

Harris and Johnson used single-trait selection index equations for steps 1, 2, and 4. However, the single-trait selection index equations to include progeny information in reliabilities of ancestors (step 1 and 2) are not easily extended to multiple traits. They were therefore replaced by an equivalent mixed-model approach in which effective record contributions from an animal with p observations are approximated by absorbing row 1 of the following 2 matrices for sire and maternal grandsire (mgs), respectively:


Formula

where k is equal to (4 – h2)/h2. The resulting effective records, which can be accumulated for multiple progeny, are


Formula

Reliabilities (R) for each ancestor, after combining the progeny contribution with their own observations (s and m), are


Formula

Formula

Substituting s = kRs/(1 – Rs), m = kRm/(1 – Rm), and p = kRp/(1 – Rp), into the above equations, and canceling the common terms from the numerator and denominator, k2/(1 – Rs)(1 – Rp) for the sire and k2/(1 – Rm)(1 – Rp) for the mgs, yields


Formula

Formula

These are identical to the corresponding equations of Harris and Johnson (1998b). The procedure was extended to multiple traits by replacing p with Zp'MZp and k with G–1 in the above steps. M is equal to R–1R–11(1'R–11) 1'R–1, G is the (co)variance matrix for the structure among elements of the sire vector s, and Zp denotes the elements of Z pertaining to progeny. Progeny contribution matrices were derived by absorbing progeny equations. These were added to the information matrices of sire (Zs'MZs) and mgs (Zm'MZm) observations, respectively, to form combined information matrices (E). The information from all r trait groups was combined as follows:


Formula

Then for trait t, the reliability is as follows:


Formula

The last step of the Harris and Johnson method was applied for multiple traits without modification, because only minor improvements are expected from extending the parent absorption step to multiple traits. Finally, the software was tested for both ST- and MT-MACE, and approximate reliabilities for ST-MACE were essentially equal to the corresponding Harris and Johnson reliabilities, as expected. The average reliability was 0.05 percentage units higher compared with the corresponding Harris and Johnson reliabilities, the maximum difference between the 2 sets of reliabilities was 1 percentage unit, and the correlation between the 2 sets of reliabilities was 0.9999. Approximations from both methods should be compared with exact reliabilities for a 2-country MACE model.


    ACKNOWLEDGEMENTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 APPENDIX 1
 APPENDIX 2
 ACKNOWLEDGEMENTS
 REFERENCES
 
The participating countries are acknowledged for making data and within-country parameters available. We appreciate our fruitful discussions with several colleagues, especially Zengting Liu, Per Madsen, Larry Schaeffer, Freddy Fikse, Ulf Emanuelson, and Jan Philipsson. Jan, Freddy, and the reviewers are also acknowledged for their comments on the manuscript.

Received for publication August 11, 2005. Accepted for publication June 7, 2006.


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS AND DISCUSSION
 CONCLUSIONS
 APPENDIX 1
 APPENDIX 2
 ACKNOWLEDGEMENTS
 REFERENCES
 


Carlén, E., E. Strandberg, and A. Roth. 2004. Genetic parameters for clinical mastitis, somatic cell score, and production in the first three lactations of Swedish Holstein cows. J. Dairy Sci. 87:3062–3070.[Abstract/Free Full Text]

Fikse, W. F., and G. Banos. 2001. Weighting factors of sire daughter information in international genetic evaluations. J. Dairy Sci. 84:1759–1767.[Abstract]

Harris, B., and D. Johnson. 1998a. Approximate reliability of genetic evaluations under an animal model. J. Dairy Sci. 81:2723–2728.[Abstract]

Harris, B., and D. Johnson. 1998b. Information source reliability method applied to MACE. Interbull Bull. 17:31–36.

Interbull. 2004a. Evaluation summary for udder health traits February 2004. Interbull Centre, Uppsala, Sweden. www.interbull.org Accessed Arpil 20, 2004.

Interbull. 2004b. Correlation estimation procedure. Interbull Centre, Uppsala, Sweden. www.interbull.org/documents/genetic_correlation_estimation_procedure_042t.pdf Accessed July 6, 2005.

Interbull. 2004c. Description of GES as applied in member countries. Interbull Centre, Uppsala, Sweden. www.interbull.org Accessed April 20, 2004.

Jorjani, H., L. Klei, and U. Emanuelson. 2004. A simple method for weighted bending of genetic (co)variance matrices. J. Dairy Sci. 86:677–679.

Kirkpatrick, M., and K. Meyer. 2004. Direct estimation of genetic principal components: Simplified analysis of complex phenotypes. Genetics 168:2295–2306.[Abstract/Free Full Text]

Klei, B., and K. A. Weigel. 1998. A method to estimate correlations among traits in different countries using data on all bulls. Proc. 1998 Interbull Mtg., Rotorua, New Zealand. Interbull Bull. 17:8–14.

Leclerc, H., W. F. Fikse, and V. Ducrocq. 2005. Principal components and factorial approaches for estimating genetic correlations in international dairy sire evaluation. J. Dairy Sci. 88:3306–3315.[Abstract/Free Full Text]

Liu, Z., F. Reinhardt, and R. Reents. 2004. A multi-trait MACE model for international bull comparison based on daughter yield deviations. Interbull Bull. 32:46–52.

Luttinen, A., and J. Juga. 1997. Genetic relationships between milk yield, somatic cell count, mastitis, milkability and leakage in Finnish dairy cattle population. Interbull Bull. 15:78–83.

Madsen, P., J. Jensen, and T. Mark. 2000. Reduced rank estimation of (co)variance components for international evaluation using AI-REML. Proc. 2000 Interbull meeting, Bled, Slovenia. Interbull Bull. 25:46–50.

Mäntysaari, E. A. 2004. Multiple-trait across-country evaluations using singular (co)variance matrix and random regression model. Interbull Bull. 32:70–74.

Mark, T., W. F. Fikse, U. Emanuelson, and J. Philipsson. 2002a. International genetic evaluations of Holstein sires for milk somatic cell and clinical mastitis. J. Dairy Sci. 85:2384–2392.[Abstract/Free Full Text]

Mark, T., W. F. Fikse, U. Emanuelson, and J. Philipsson. 2002b. Short communication: Effect of phantom parent grouping and properties of deregression for a low heritability trait. J. Dairy Sci. 85:2393–2395.[Abstract/Free Full Text]

Mark, T., P. Madsen, J. Jensen, and W. F. Fikse. 2005. Prior (co)variances can improve multiple-trait across-country evaluations of weakly linked bull populations. J. Dairy Sci. 88:3290–3302.[Abstract/Free Full Text]

Misztal, I., and M. Perez-Enciso. 1998. A Fortran 90 interface to sparse matrix package FSPAK with dynamic memory allocation and sparse matrix structure. Proc. 6th World Cong. Genet. Appl. Livest. Prod. (Armidale, Australia) 22:77–78.

Nielsen, U. S., G. P. Aamand, and T. Mark. 2000. National genetic evaluation of udder health and other health traits in Denmark. Interbull Bull. 25:143–150.

Pösö, J., and E. A. Mäntysaari. 1996. Relationships between clinical mastitis, somatic cell score, and production for the first three lactations of Finnish Ayrshire. J. Dairy Sci. 79:1284–1291.[Abstract]

Schaeffer, L. R. 1994. Multiple-country comparisons of dairy sires. J. Dairy Sci. 77:2671–2678.[Abstract]

Schaeffer, L. R. 2001. Multiple trait international bull comparisons. Livest. Prod. Sci. 69:145–153.[Medline]

Strandén, I., and M. Lidauer. 1999. Solving large mixed linear models using preconditioned conjugate gradient iteration. J. Dairy Sci. 82:2779–2787.[Abstract]

Sullivan, P. G. 1999. Appendix: REML estimation of heterogeneous sire (co)variances for MACE. Interbull Bull. 22:146–148.

Sullivan, P., and J. W. Wilton. 2001. Multiple-trait MACE with a variable number of traits per country. Interbull Bull. 27:68–72.

Sullivan, P. G., J. W. Wilton, L. R. Schaeffer, G. J. Jansen, J. A. B. Robinson, and O. B. Allen. 2005. Genetic evaluation strategies for multiple traits and countries. Livest. Prod. Sci. 92:195–205.

Tarres, J., J. Piedrafita, and V. Ducrocq. 2006. Validation of an approximate approach to compute genetic correlations between longevity and linear traits. Genet. Sel. Evol. 38:65–83.[Medline]

van der Beek. 1999. Exploring the (inverse of the) international genetic correlation matrix. Interbull Bull. 22:14–20.


This article has been cited by other articles:


Home page
J DAIRY SCIHome page
R. L. Powell, A. H. Sanders, and H. D. Norman
Investigation of Country Bias in International Genetic Evaluations Using Full-Brother Information
J Dairy Sci, July 1, 2008; 91(7): 2885 - 2892.
[Abstract] [Full Text] [PDF]


Home page
J DAIRY SCIHome page
T. Mark, W. F. Fikse, P. G. Sullivan, and P. M. VanRaden
Prediction of Genetic Correlations and International Breeding Values for Missing Traits
J Dairy Sci, October 1, 2007; 90(10): 4805 - 4813.
[Abstract] [Full Text] [PDF]


Home page
J DAIRY SCIHome page
J. Tarres, Z. Liu, V. Ducrocq, F. Reinhardt, and R. Reents
Validation of an Approximate REML Algorithm for Parameter Estimation in a Multitrait, Multiple Across-Country Evaluation Model: A Simulation Study
J Dairy Sci, October 1, 2007; 90(10): 4846 - 4855.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Interpretive Summary
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Mark, T.
Right arrow Articles by Sullivan, P. G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Mark, T.
Right arrow Articles by Sullivan, P. G.


HOME HELP