J. Dairy Sci. 88:2624-2631
© American Dairy Science Association, 2005.
Accuracy and Stability of National and International Somatic Cell Score Evaluations
R. L. Powell,
A. H. Sanders and
H. D. Norman
Animal Improvement Programs Laboratory, Agricultural Research Service, USDA, Beltsville, MD 20705-2350
Corresponding author: Rex L. Powell: e-mail: rpowell{at}aipl.arsusda.gov.
 |
ABSTRACT
|
|---|
Somatic cell score (SCS) evaluations have been published in the United States since 1994 and international evaluations have been available through Interbull since May 2001. The accumulated data provides an opportunity to investigate the accuracy and stability of SCS evaluations. United States domestic evaluations from January 1995 through August 2004, for 21,500 Holstein bulls were considered, over time and sequentially within bull, for changes to the November 2004 evaluation. On average, predicted transmitting ability (PTA) SCS increased (worsened) by 0.002 from earlier evaluations to November 2004. Although bias was small, PTA changes were more than expected based on change in reliability. When looked at sequentially, bulls earlier evaluations were generally lower (i.e., merit was overestimated) relative to November 2004. Differences were small, and PTA SCS increased steadily with the addition of second-crop daughters. All 524,081 evaluations were considered pairwise providing over 8,000,000 pairs of bulls evaluations for analysis of PTA differences relative to change in reliability. Agreement of observed and expected SD improved for larger changes in reliability. The November 2004 US and Interbull PTA were matched with US and Interbull PTA from May 2001 (US04, IB04, US01, and IB01, respectively) for 14,652 Holstein bulls. For bulls having only US daughters in IB01, correlations were similar for US01 and IB01 with US04, and IB01 with IB04. Corresponding regressions were all nearly 1.00. For bulls also having nonUS daughters in IB01, correlations with yield deviations calculated for later daughters (used as source of independent data) were higher (0.747 vs. 0.714) for IB01 than for US01. For bulls with added US daughters, correlation with US04 was also higher for IB01 than US01, showing that inclusion of foreign data improved predictive value of SCS evaluations.
Key Words: genetic evaluation Interbull evaluation accuracy somatic cell score
Abbreviation key: DDE = DE from daughters, DE = daughter equivalents, DYD = daughter yield deviation, REL = reliability.
 |
INTRODUCTION
|
|---|
The accuracy and stability of bulls genetic evaluations are central to their usefulness in achieving genetic progress. Accordingly, attention has been given to these qualities for evaluations of yield traits (milk, fat, and protein). More recently developed evaluations for traits having lower heritabilities and less available data have received less attention. Somatic cell score was first evaluated in the United States in January 1994 (Schutz, 1994), thus providing 10 yr of data for analysis of US evaluations. Over that time, US evaluation procedures have remained unchanged except for the adoption of best-prediction procedures for incomplete records in 1999 (VanRaden et al., 1999). Interbull evaluation of SCS began in May 2001 (Mark et al., 2002) and those evaluations can now be considered in light of 3 yr of additional data. Traits other than yield have gained economic recognition as reflected in increasing emphasis in the US net merit index with SCS having the highest relative value (9%) aside from yield and productive life (VanRaden and Seykora, 2003).
Over birth years 1990 to 2001, the phenotypic mean SCS for Holsteins decreased (improved) by 0.21 (from 3.19 to 2.98), while the cow EBV increased (worsened) by 0.06 (AIPL, 2004). The genetic decline is attributed to the positive (unfavorable) genetic and phenotypic correlations between SCS and yield; during the same period, mean milk yield increased 2006 kg and cow EBV for milk increased 1186 kg. Thus, management has been the key factor in improvement (i.e., reduction) of SCS.
Presentation of genetic evaluations for SCS is far from standard across countries. VanRaden (2004) reported that for 20 countries, few used the same presentation scheme. For half of the countries, higher numbers represented improvement for udder health (i.e., fewer somatic cells) and for the other half, lower numbers represented fewer cells. Several transformations and standardization schemes are used. In the US, SCS = log2 (SCC/100,000) + 3; the sign is conserved, thus higher PTA are undesirable and published PTA are computed with a genetic base of zero plus the breed mean (beginning in 2005, a standard constant of 3.0 replaced the breed means).
Powell et al. (2004b) reported declines in yield PTA over time for Holstein bulls that were, or had been, in active AI service. A number of studies have shown that bulls sampled outside the programs of the major AI organizations were initially overevaluated for yield (Cassell et al., 1992; Powell et al., 1994; Powell et al., 2004b). To provide information on how bulls were sampled, the National Association of Animal Breeders (NAAB) added sampling status to their cross-reference program (Sattler, 1990). Bulls reported by 3 yr of age as having semen distributed to at least 40 herds are assigned sampling status "S", and other bulls are assigned sampling status "O". Powell et al. (2004b) reported that O-code bulls tended to be overevaluated relative to S-code bulls for yield, but to a lesser extent than found in previous years (Powell et al., 1994). Conclusions from yield traits should not be extrapolated to SCS. VanRaden et al. (1997) found that the regression of later SCS PTA on corresponding earlier PTA was 1.16 (expected regression is 1.00), and evaluations changed 31% more than expected.
Addition of data from other countries to domestic data has been shown to improve prediction of future domestic evaluations for yield (Powell et al., 2000). Even foreign data alone were a good predictor of the later domestic yield evaluation (Powell et al., 2004a). For the limited data on SCS in that study, the correlations between Interbull evaluations from foreign data and US evaluations from domestic data were higher than expected based on the reliabilities (REL).
The objective of this study was to investigate the stability, accuracy, and predictive value of SCS evaluations for Holstein bulls from 2 perspectives: 1) by the examination of changes over a decade of US bull evaluations and 2) by comparison of earlier (May 2001) US and Interbull evaluations with corresponding current (November 2004) US and Interbull evaluations.
 |
MATERIALS AND METHODS
|
|---|
Data
United States domestic SCS genetic evaluations for Holstein bulls from the period January 1995 through November 2004 were examined in study 1. Earlier evaluations were converted to the current genetic base. An NAAB-assigned sampling status code was required for all bulls, and any bull whose REL decreased more than 1% (typically the result of pedigree correction) in consecutive runs was discarded. Evaluations from 21,500 bulls were included in the analysis. In study 2, US and Interbull SCS PTA from May 2001 and November 2004 (US01, IB01, US04, and IB04, respectively) for Holstein bulls having all 4 evaluations were examined. Of these bulls, the IB01 evaluations included only US daughters for 13,624 (US-only bulls), and 1028 had additional foreign daughters (multicountry bulls). Country of origin was not a factor in either study, and a large majority of bulls in all categories were identified as US bulls (first registered in the United States). Of the bulls in study 1, 97% were US bulls and nearly all others were identified as Canadian. In study 2, 99% of the US-only bulls were identified as US bulls. Even among the multicountry bulls, 82.1% were identified as US bulls, and 17.5% were Canadian; only France and the Netherlands also had bulls included in this group.
Methods
For study 1, which used only domestic US evaluations, mean differences and SD of differences between November 2004 PTA and corresponding earlier PTA were computed for each of the 35 evaluation runs from January 1995 through August 2004. The November 2004 evaluations were used as the standard for assessing accuracy of earlier evaluations because, as the latest available, they were assumed the most accurate. Changes in PTA were also investigated relative to the sequence of evaluation for individual bulls, rather than calendar time, to see if any pattern emerged, particularly as bulls added second-crop daughters. Evaluations for bulls first evaluated for SCS between January 1995 and February 1999 were numbered sequentially within bull such that each increment represented 3 mo; in 1995 and 1996, evaluations were calculated only twice per year, thus they were assigned only odd sequence numbers.
Mean differences and SD of differences for all 8,001,717 pairs of corresponding bull evaluations from January 1995 through November 2004 were computed according to the change (increase) in REL. Expected SD of change were computed as
after Powell and Norman (1981), where SDsire was the sire genetic SD (0.220) for US Holstein bulls as calculated by the Interbull Centre (Interbull, 2004a). Expected correlations between earlier evaluations (A) and corresponding subsequent evaluations (B) were computed as
. Statistics were also compared separately for bull groups of each sampling status.
For study 2, mean differences, SD of differences, correlations, and regressions were computed for corresponding US01, US04, IB01, and IB04 evaluations paired across time. For multicountry bulls, correlations and mean changes from US01 and IB01 to US04 were compared to determine if the added foreign data aided in prediction of the later US evaluations. The US04 evaluations were used as the standard for assessing accuracy of the earlier evaluations. However, bulls adding little or no data (new daughters or added records on earlier daughters) would have a high correlation between earlier and later evaluations by being based on the same data, not necessarily by being accurate. To investigate and partially account for this dependency, statistics were also computed according to the percentage increase in US daughters (25, 50, 100, and 200%).
Because the 2001 US data comprises a major part of the 2004 US data, correlation of these evaluations is not an ideal indicator of the accuracy of the earlier evaluation. To provide an independent source of the most current data, independent daughter yield deviations (DYD) for daughters added between May 2001 and November 2004 were computed. Reliability is a function of heritability and the amount of information as reflected by daughter equivalents (DE) (VanRaden and Wiggans, 1991), and can be computed as REL = DE/(DE + k) where k = (4 2h2)/h2. Thus, for SCS (h2 = 0.10), k = 38 and DE = 38REL/(1 REL). By applying this to the REL of both the PTA and the parent average, the difference (DEPTA DEPA) provides DE from daughters (DDE). Using the DDE calculated for 2001 and for 2004, and the DYD from both evaluations,
can be transformed as
where DDEadded is DDE2004 DDE2001 and DYDInd is the independent DYD. This provided an alternative measure of genetic merit for assessing the IB01 and US01 evaluations without the part-whole impediment. Correlations were computed for bulls having DDEadded
30 and bulls having DDEadded
100 in the US data.
 |
RESULTS AND DISCUSSION
|
|---|
In study 1, a total of 524,081 genetic evaluations for the 21,500 bulls were included (e.g., a bull with 10 evaluations before November 2004 contributed 10 observations). On average, November 2004 US domestic SCS evaluations were higher (i.e., worse) by 0.002 than corresponding earlier evaluations, with a mean absolute change of 0.030. Mean changes from selected evaluations to November 2004 and the SD of those changes is shown in Table 1
. The trend for intervening runs is the same as that shown by the July (1995 to 1996) or August (1997 to 2004) evaluations. As expected, greater variation in changes were seen from the earlier run dates as more time for new daughter data to accumulate had passed. For each run date, mean change to November 2004 was positive, meaning that the earlier PTA were consistently more favorable. However, the bias was very small; the largest (0.007 for February 1998) being only 3% of the sire SD and many being essentially zero. Mean and median REL of November 2004 evaluations was 71% and mean REL of corresponding earlier evaluations increased from 61% for January 1995 to 70% for August 2004. For all runs investigated, SD of change in evaluation was more than expected based on the change in REL.
View this table:
[in this window]
[in a new window]
|
Table 1. Numbers of Holstein bulls born in 1985 or later, with at least 10 daughters included in SCS evaluations, mean absolute changes, means and SD of changes, and expected SD of changes from bulls earlier evaluations to their November 2004 evaluations, by evaluation date.
|
|
With no change in REL, our expected SD of change is zero. In many cases, small differences in REL are masked when rounded to a percentage. Even if REL stays exactly the same, data for daughters management group mates may change, impacting the bulls PTA or the group of contributing daughters may change due to corrected sire ID. Also contributing to larger SD of change and not considered in the formula for expected change are differences across daughters lactations. The US model assumes a single trait across lactations, but bulls may differ in daughter maturity rate (merit over time). Estimated genetic correlations among lactations, shown in Table 2
for 7 countries, are considerably different from unity. A model accounting for differing maturity rates should improve evaluation stability.
Table 3
contains the mean differences and SD of differences relative to November 2004 for the first 16 evaluation runs for bulls having their first evaluation between January 1995 and February 1999. That allowed run 16 to be at least 2 yr before May 2004. A requirement for the addition of 200 daughters from run 7 to run 16 was applied to select bulls with second-crop daughters included by run 16. First SCS evaluation was later than first yield evaluation for 5% of bulls because not all yield daughters contribute to SCS evaluations (Wiggans, 1997). Thus, second-crop daughters may have arrived sooner relative to SCS evaluation (i.e., may be included in lower-numbered evaluations) for some bulls. Nevertheless, the median numbers of daughters indicate that for most bulls, the addition of second-crop daughters was
3.5 yr after first SCS evaluation. Early merit for SCS was generally estimated too favorably (i.e., PTA was lower than in November 2004), but bias decreased steadily with the addition of second-crop daughters. The mean absolute difference (not shown) from November 2004 and the SD of the difference also decreased over time. The SD of difference typically was 0.02 higher than expected from the change in REL.
View this table:
[in this window]
[in a new window]
|
Table 3. Mean changes to the November 2004 evaluation from corresponding earlier evaluations, SD and expected SD of changes, and median numbers of daughters for Holstein bulls first evaluated for SCS between January 1995 and February 1999, that added at least 200 daughters between evaluations numbered 7 and 16, by evaluation number.1
|
|
Change in evaluations should be related to the amount of additional information as represented by REL. All 8,001,717 evaluation pairs were examined for changes in evaluation relative to the change in REL. Table 4
presents numbers of S-code and O-code bulls, mean observed PTA changes and their SD, expected SD of PTA changes, and mean absolute PTA changes by the concurrent increase in REL. Mean PTA change increased steadily with REL change for both groups of bulls but the amounts were not large. For smaller changes in REL, the O-code bulls unexpectedly tended to have slightly smaller changes than S-code bulls (difference < 0.001). Although REL is a function of number of daughters, the highest reported value is 99%. Thus, when a bull reaches this ceiling, the change in REL (e.g., from 98 to 99%) may be disproportionately small relative to the number of daughters added in the same period. Rounding can compound the effect of this ceiling; for example, a 1% change in reported REL from 98 to 99% could actually reflect a change of up to 2.4% (97.5 to 99.9%) if available to one additional decimal place. The median number of added daughters for small changes in REL (
5%) was 15% higher for S-code bulls than for O-code bulls and these changes were more often bulls reaching REL of 99% (data not shown). For small changes in REL, the SD of change was larger than expected for both "S" and "O" bulls. For larger changes in REL, the difference in PTA change between S-code and O-code bulls was either essentially zero or S-code bulls had slightly less change in PTA (difference <0.02); SD of change were nearly as expected.
View this table:
[in this window]
[in a new window]
|
Table 4. Means and SD of changes, and expected SD of changes between bulls SCS evaluations at different times by the concurrent change in reliability for bulls with S or O sampling status.
|
|
From study 2, the correlations between IB01 and IB04, US01 and US04, and IB01 and US04 were very similar (Table 5
) for US-only bulls. With increasing requirements for added US daughters between 2001 and 2004, the correlations declined. This was expected with the increasing addition of independent data.
View this table:
[in this window]
[in a new window]
|
Table 5. Correlations of 2001 Interbull (IB01) or US (US01) with 2004 Interbull (IB04) or US (US04) evaluations for bulls having only US daughters in IB01 (US-only) or with daughters from an additional country in IB01 (multicountry), by increase in US daughters between May 2001 and November 2004.
|
|
The correlations for multicountry bulls provide an indication of whether the foreign data in IB01 improved the prediction of the later US PTA. Overall, these bulls had an average of 1235 daughters in US01 and 3013 daughters in IB01. Without a requirement of additional US daughters, US01 had a higher correlation than IB01 with US04, which is expected because a greater proportion of 2004 data would be the same as that contributing to the 2001 evaluation. With the addition of more US daughters, correlations of both US01 and IB01 with US04 decreased, but the difference between the correlations increased (i.e., superiority of the IB evaluations for prediction became more apparent due to reduced influence of common US daughters). Bulls adding at least 25% more US daughters between 2001 and 2004 averaged 1800 US daughters in US01 and 4090 in US04, thus these later evaluations included a substantial amount of independent information. Although correlations were not as high with the independent DYD, which were from as few as 30 DE, the predictive advantage of IB01 over US01 was more clearly shown (correlation with DYD of 0.747 vs. 0.714).
The regression of later on earlier PTA is expected to be 1.0. That is, the difference between bulls, in PTA units, should be the same in 2004 as it was in 2001. For US-only bulls, regressions were all essentially 1.00, although when based on the large total number of bulls (with no requirement for addition of daughters), the regressions (0.99) were significantly different (P < 0.01) from expectation (Table 6
). For multicountry bulls, regressions were lower and were significantly different (P < 0.01) from expectation with no requirement for addition of US daughters. For categories of multicountry bulls having additional US daughters, numbers of bulls were limited and regressions generally were not significantly different from expectation; however, the trend suggests that early PTA for these bulls were not realized in later national evaluations. For bulls with at least 30 additional DE between 2001 and 2004, the regressions of independent DYD on earlier evaluations for multicountry bulls (Table 7
) were significantly less than 1.0 for US01 (0.88) but not for IB01 (0.92). Corresponding regressions for US-only bulls were both 0.94 and significantly less than 1.0. Requiring 100 additional DE improved correlations, and regressions were all near unity.
View this table:
[in this window]
[in a new window]
|
Table 6. Regressions of 2004 Interbull (IB04) or US (US04) on 2001 Interbull (IB01) or US (US01) evaluations for bulls having only US daughters in IB01 (US-only) or with daughters from an additional country in IB01 (multicountry), by increase in US daughters between May 2001 and May 2004.
|
|
View this table:
[in this window]
[in a new window]
|
Table 7. Correlations and regressions of independent daughter yield deviations (DYD)1 with and on 2001 Interbull (IB01) or US (US01) evaluations for bulls having only US daughters in IB01 (US-only) or with daughters from an additional country in IB01 (multicounty), by increase in daughter equivalents (DE) between May 2001 and May 2004.
|
|
 |
CONCLUSIONS
|
|---|
Mean SCS PTA increased from each of 35 evaluation runs to the November 2004 run. Thus, the earlier evaluations overestimated bull merit for SCS reduction but the bias was so small it should not cause concern. For all runs, the SD of PTA change to November 2004 was larger than expected but the formula for computing the expected SD does not consider all possible contributions to change. When considered by increase in REL, bulls with larger changes in REL showed larger increases in mean PTA. For practical purposes, SCS evaluations of bulls with either "O" or "S" sampling code appeared equally accurate and stable. This is in contrast to yield evaluations, where, on average, O-code bulls have been overevaluated with first-crop daughters. Apparently, the processes that produce bias in yield evaluations for O-code bulls (e.g., preferential treatment of daughters) are not manifest in SCS evaluations.
For bulls returned to service, as for the overall group, early evaluations overpredicted later merit. Mean change for each sequential run to November 2004 was unfavorable and, in early evaluations, bias was about 20% of sire SD. However, this bias decreased steadily, being only 10% of sire SD with the addition of second-crop information. Still, the reason for the general declines in estimated merit from the more favorable early results remains unknown. A possibility is that bulls are returned to service based largely on yield information from the initial sample of daughters. A multitrait approach considering the negative correlation between yield and udder health might improve accuracy of prediction for SCS by lowering the expected merit of these bulls.
For bulls with only US daughters, the US and Interbull evaluations from May 2001 had similar correlations with the November 2004 US evaluations. These correlations decreased with increasing proportions of new data, as expected. For multicountry bulls adding US data, the correlations with 2004 US evaluations were higher for Interbull 2001 evaluations than for those from the US, and the difference increased with the addition of more new US daughters. Thus, as has been previously reported for yield traits, the Interbull evaluation including foreign data was a superior predictor of the later US SCS evaluation. This conclusion was supported even more clearly through the correlation with the calculated independent DYD. The systems for estimating genetic merit for SCS nationally and internationally are generally operating as they should.
Received for publication December 30, 2004.
Accepted for publication March 28, 2005.
 |
REFERENCES
|
|---|
Animal Improvement Programs Laboratory (AIPL). 2004. Genetic and phenotypic trends. http://aipl.arsusda.gov/dynamic/trend/current/trndx.html. Accessed Dec. 1, 2004.
Banos, G., and G. E. Shook. 1990. Genotype by environment interaction and genetic correlations among parities for somatic cell count and milk yield. J. Dairy Sci. 73:25632573.[Abstract]
Carlén, E., E. Strandberg, and A. Roth. 2004. Genetic parameters for clinical mastitis, somatic cell score, and production in the first three lactations of Swedish Holstein cows. J. Dairy Sci. 87:30623070.[Abstract/Free Full Text]
Cassell, B. G., R. E. Pearson, M. L. McGilliard, and T. R. Meinert. 1992. Genetic merit and usage patterns of bulls from different sampling programs. J. Dairy Sci. 75:572579.[Abstract]
de Roos, A. P. W., A. G. F. Harbers, and G. de Jong. 2003. Genetic parameters of test-day somatic cell score estimated with a random regression model. Proc. 2003 Interbull Mtg., Rome, Italy. Interbull Bull. 31:97101.
Harris, B. L., and A. M. Winkelman. 2004. Test-day model for national genetic evaluation of somatic cell count in New Zealand. Proc. 2004 Interbull Mtg., Sousse, Tunisia. Interbull Bull. 32:101104.
International Bull Evaluation Service. (Interbull). 2004a. Interbull routine genetic evaluation for udder health traits, May 2004. http://www.interbull.slu.se/eval/framesida-prod.htm. Accessed June 21, 2004.
International Bull Evaluation Service. (Interbull). 2004b. Description of national genetic evaluation systems for dairy cattle traits as applied in different Interbull member countries. http://www-interbull.slu.se/national_ges_info2/framesida-ges.htm. Accessed Nov. 30, 2004.
Jamrozik, J., L. R. Schaeffer, and F. Grignola. 1998. Genetic parameters for production traits and somatic cell score of Canadian Holsteins with multiple trait random regression model. Proc. 6th World Congr. Genet. Appl. Livest. Prod. Armidale, Australia. 23:303306.
Mark, T., W. F. Fikse, U. Emanuelson, and J. Philipsson. 2002. International genetic evaluations of Holstein sires for milk somatic cell and clinical mastitis. J. Dairy Sci. 85:23842392.[Abstract/Free Full Text]
Liu, Z., A. Reinhardt, J. Bünger, J. Jaitner, and R. Reents. 2003. Genetic evaluation of somatic cell scores using a random regression test-day model for a very large dairy cattle population. Proc. 2003 Interbull Mtg., Rome, Italy. Interbull Bull. 31:8891.
Mrode, R. A., and G. J. T. Swanson. 2003. Estimation of genetic parameters for somatic cell count in the first three lactations using random regression. Livest. Prod. Sci. 79:239247.
Powell, R. L., and H. D. Norman. 1981. Changes in predicted difference milk with increased reliability. J. Dairy Sci. 63:19721977.
Powell, R. L., H. D. Norman, and G. Banos. 2000. Improving prediction of national evaluations by use of data from other countries. J. Dairy Sci. 83(Feb.). Online. Available: http://www.adsa.org/jds/papers/2000/online/jds9123.htm.
Powell, R. L., A. H. Sanders, and H. D. Norman. 2004a. Accuracy of foreign dairy bull evaluations in predicting US evaluations for yield. J. Dairy Sci. 87:26212626.[Abstract/Free Full Text]
Powell, R. L., A. H. Sanders, and H. D. Norman. 2004b. Stability of evaluations for active AI bulls. J. Dairy Sci. 87:26142620.[Abstract/Free Full Text]
Powell, R. L., G. R. Wiggans, and H. D. Norman. 1994. Effect of sampling status and adjustment for heterogeneous variance on bias in bull evaluations. J. Dairy Sci. 77:883890.[Abstract]
Sattler, C. G. 1990. A. I. bulls will be labeled by sampling status. Hoards Dairyman 135:600.
Schutz, M. M. 1994. Genetic evaluation of somatic cell scores for United States dairy cattle. J. Dairy Sci. 77:21132129.[Abstract]
VanRaden, P. M. 2004. Choices of scales for delivery of genetic evaluations to the public. Proc. 2004 Interbull Mtg., Sousse, Tunisia. Interbull Bull. 32:118121.
VanRaden, P. M., and A. J. Seykora. 2003. Net merit as a measure of lifetime profit: 2003 revision. USDA Agricultural Research Service AIPL research report NM$2. Online. Available: http://aipl.arsusda.gov/reference/nmcalc.htm. Accessed Nov. 6, 2004.
VanRaden, P. M., R. J. Starkenburg, and T. J. Lawlor. 1997. Stability of genetic evaluations for productive life, somatic cell score, and yield over time. J. Dairy Sci. 80(Suppl. 1):253.
VanRaden, P. M., and G. R. Wiggans. 1991. Derivation, calculation, and use of national animal model information. J. Dairy Sci. 74:27372746.[Abstract]
VanRaden, P. M., G. R. Wiggans, and C. P. Van Tassell. 1999. Changes in USDA-DHIA genetic evaluations (February 1999). USDA Agricultural Research Service AIPL research report CH13 (299). Online. Available: http://aipl.arsusda.gov/reference/changes/chng9902.html. Accessed Nov. 6, 2004.
Wiggans, G. R. 1997. NCDHIP Participation as of January 1, 1997. http://aipl.arsusda.gov/publish/dhi/dhi97/97k1.html. Accessed Nov. 6, 2004.