| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
* Department of Farm Animal Health and Reproduction, Utrecht University, PO Box 80151, 3584 CN Utrecht, the Netherlands
Department of Production Animal Clinical Sciences, Norwegian School of Veterinary Science, PO Box 8146 Dep., N-0033 Oslo, Norway
1 Corresponding author: t.h.halasa{at}uu.nl
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGMENTSAPPENDIXREFERENCES |
|---|
Key Words: mastitis • dairy cattle • dry period • meta-analysis
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGMENTSAPPENDIXREFERENCES |
|---|
An alternative to DCT is the use of an internal or external teat sealant (TS), which is meant to prevent pathogen access to the mammary glands (Meaney, 1976). Teat sealant application has been recommended by the National Mastitis Council, together with DCT in some cases to provide higher protection against new IMI (Godden et al., 2003).
The importance and the protective effect of DP management on new IMI are well recognized (Bradley and Green, 2004). However, large variations in the protective effect and the risk of new IMI can be encountered when comparing the different studies (Bradley and Green, 2004). Moreover, different interventions are expected to provide different degrees of protection. For calculations of the economic effects and subsequent support of decisions, it is essential to quantify an estimate of the risk of new IMI by using the different DP interventions. Only one study quantified summary risks of new IMI related to the DP intervention (Robert et al., 2006). However, this study focused only on quantifying the effect of DCT with and without teat sealants. The protective effect postcalving was not estimated, which might be important because some long-acting antibiotics are expected to limit new IMI around calving and early lactation (Pearson and Wright, 1969). More important, the potential publication bias in the studies reviewed was not assessed. Because studies that result in large and interesting treatment effects are more likely to be published than studies that show relatively small or no treatment effects, the outcome could be a biased body of research (Rothstein et al., 2005). Therefore, it is important to address and discuss this potential bias to draw proper conclusions on the preventive effect of DCT and other interventions. Meta-analysis is a statistical analysis of a large collection of analytical results from individual studies for the purpose of integrating the findings, which would facilitate drawing conclusions based on the available information (Dohoo et al., 2003). The technique is powerful in the sense that it considers study attributes, such as study precision, and weight those properly, but in some cases, the diversity among studies demands caution while drawing conclusions (Dohoo et al., 2003). The objectives of the present study were to 1) provide a summary quantification of the protective effect of different DP interventions on the risk of new IMI during the DP up to 21 d postcalving, based on meta-analysis of existing peer-reviewed literature, and 2) address and discuss the potential bias in the existing peer-reviewed literature.
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGMENTSAPPENDIXREFERENCES |
|---|
Papers included in the meta-analysis 1) had to be original research papers published in peer-reviewed journals; 2) had to report the number or the rate of new IMI at the quarter level in at least 2 groups (treated and control group) and the total number of quarters in each group; 3) could include only papers that had both pre-dry off and postcalving milk sampling at the quarter level, as by definition these were the only studies that could provide the new IMI data; and 4) had to report the outcome of a new data set or new protocol. When several studies were published based on the same data, the most detailed study was used. Further details of the studies are provided in subsequent sections.
A total of 33 studies fitted the above criteria for inclusion in the meta-analyses. When studies reported the outcome of one or several protocols, Roman numerals were added to distinguish protocols. Both negative control (i.e., those that included placebo or untreated cows or quarters) and positive control designs were included in the analyses.
Two formats for randomization and treatment were observed in the selected studies: 1) the whole udder was assigned to either the treatment or control group (between-cow comparison), or 2) one or more quarters of the udder were assigned to the treatment group and the other quarters were assigned to the control group (within-cow comparison).
Because different studies differed in design and in the observational units, discrepancies were expected that could influence the validity of the meta-analysis. Data related to the study design, level of analysis, and dry cow management were recorded from the original studies and are presented in Table 1, in Appendix Tables A1, A2, and A3, and in the descriptive results.
|
Studies that compared cloxacillin with other DCT products and that reported the number or incidence of new quarter IMI during the DP were included in the analysis. In those studies, sampling at a single time point just after calving was used to define the presence of a new IMI. Cloxacillin was included as the treatment group and the other DCT product was included as the control group. There was not enough data to conduct comparisons between other antibiotics; hence, only cloxacillin was used for comparison with other DCT products.
The SDCT comparison was based on studies that measured the rate or number of new quarter IMI in the SDCT compared with no DCT during DP up to 21 d postcalving as a cumulative incidence. The analysis was carried out separately for quarter- or cow-level treatment and was reported as a combined overall effect. A separate meta-analysis was carried out for studies that compared SDCT with BDCT as a positive control group.
Studies that investigated the protective effect of TS using negative or positive control groups were analyzed together. The studies reported the number or incidence of new quarter IMI during DP up to 21 d postcalving as a cumulative incidence in the treatment and control groups.
A comparison was also conducted on supplementations that enhanced the immune system to better protect against new IMI during the DP. These supplementations were vitamins, minerals, and J5 vaccine. Another comparison was also conducted on teat dipping during the DP because it could protect cows from new IMI during the DP. These 2 analyses were conducted to complete the message of the current study, in which all interventions that could protect against new IMI during the DP were reviewed in meta-analyses. The papers in these 2 analyses are not summarized or referred to, but the outcomes and the reasons that impeded complete presentation of results are discussed briefly.
Milk and Secretion Samples for Bacterial Culturing.
In most studies, milk samples were collected at dry off, and at calving or within few days after calving to diagnose IMI during the DP. Some studies collected 1 extra milk sample (up to 21 d after the calving sample) to examine the effects postcalving (Table 1). In 21 studies, 2 or more consecutive single samples were collected to diagnose IMI, while a duplicate sample was collected in the other studies (Table 1).
Definition of a New Quarter IMI in the Meta-Analysis
A quarter was considered as having a new IMI when a pathogen was isolated in the calving or postcalving samples from a quarter that was free of pathogen at the previous sampling or that had a different pathogen (or pathogens) in the dry-off sample. The majority of the studies defined a new IMI at calving or postcalving only when a pathogen was isolated from a previously healthy quarter (Table 1).
Recalculating the Incidence of New Quarter IMI in the Meta-Analysis
The majority of studies defined a new IMI in healthy quarters at dry off. Hence, the incidence of new quarter IMI was recalculated for all studies based on the number of healthy quarters at dry off being the number of quarters at risk for new IMI (Table A3).
Meta-Analysis Procedure
Outcome Parameters.
The relative risk (RR) of new IMI was calculated as the incidence of new quarter IMI in the treatment group divided by the incidence of new quarter IMI in the control group per intervention and were pooled over studies in separate meta-analyses using a commercial analytical package (Comprehensive Meta-Analysis, 2008): The pooled RR was calculated per intervention as an overall effect (all pathogens together), and per pathogen group (Staphylococcus, Streptococcus, and coliforms) separately. In studies that included more than one protocol, a combined effect was calculated per study (Comprehensive Meta-Analysis, 2008). Because studies were conducted by different people, in different areas, and at different times, which create a heterogeneous population of studies, a random effect model was used to estimate the pooled RR. The pooled risk differences were similarly estimated. Forest plots were used to provide illustration of the calculated RR per study as well as the overall pooled effect of all studies in the last line of the plot. The forest plot is a graphical presentation of the results that displays the point estimate and confidence interval of the effect observed in each study, along with the summary estimate and its confidence interval (Dohoo et al., 2003).
Meta-Regression.
A weighted meta-regression was conducted in an attempt to explain the variation among studies. Factors were selected for inclusion in the model based on expert assessment of likely factors, which were the factors presented in Table 1 (except sample size). Additionally, a factor gram was defined to discriminate between studies that isolated only gram-positive bacteria and studies that isolated gram-positive and gram-negative bacteria. All the above factors were regressed against the RR results of each study and weighted by the inverse variance (Dohoo et al., 2003).
The variables were first tested in univariable models and then combined in one multivariable model by using a backward stepwise regression method. A liberal P < 0.3 was chosen for the variable to be included in a combined multivariable model. In cases of a significant association between the explanatory variable and the dependant variable (RR per study) with P < 0.05 in the combined multivariable model, the variable was believed to explain the heterogeneity significantly. A meta-analysis was carried out only when at least 4 studies were available (Robert et al., 2006).
Publication Bias.
The publication bias was assessed using funnel plots. A funnel plot is a plot of a measure of study size (standard error) on the vertical axis as a function of effect size on the horizontal axis. Large studies appear toward the top of the graph, and tend to cluster near the mean effect value. Smaller studies appear toward the bottom of the graph, and will be dispersed across a range of values. Methods included the fill and trim method of Duval and Tweedie (2000), the rank correlation test of Begg and Mazumdar (1994), and the regression test of Egger et al. (1997). When significant publication bias and change on the estimated pooled RR were detected, the number of studies necessary to reverse the overall pooled effect was calculated using the fail-safe N method of Orwin (1983). The study influence was also examined using the one study removed method (Dohoo et al., 2003). When significant publication bias was deemed to exist, the pooled RR was presented based on the fill and trim method (Duval and Tweedie, 2000) estimation after correcting for the bias. The interpretation of each of the above-mentioned tests is provided in the Results and the Discussion sections. It is important to mention that these methods are hypothetical and based on statistical theory. They do not necessarily prove existence of bias, but they do indicate the potential to bias existence based on a statistical technique that mainly relate the effect size to the study size (Thornton and Lee, 2000). Further information about the strong and weak points of the publication bias methods are mentioned in the Discussion section and are discussed in detail by Thornton and Lee (2000).
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGMENTSAPPENDIXREFERENCES |
|---|
Management of Dry and Lactating Cows.
In nearly all studies, dry cows were separated from lactating cows. In some studies, dry cows were kept in separate stalls from a few days before calving until calving or for a few days postcalving and then moved to the dairy herd. Generally, dry cows were housed on pasture, in free stalls, or in closed barns, and lactating cows were housed in free stalls or on pasture. The bedding mentioned was frequently sand, sawdust, or straw.
Experimental Design.
All studies involved in this analysis were randomized field trials (28 studies) or experiments (5 studies; Table 1). Although the inclusion criteria varied among studies, the infection status at dry off was frequently used to include or exclude cows (Table A1). The majority of the studies had a negative control group, but some had positive controls (Table A2), and the majority of studies applied a between-cow comparison.
Bacteriological Culturing Procedure.
The procedure of the National Mastitis Council was followed in most studies (Hogan et al., 1999). Other studies used modified procedures based on this source. Nevertheless, in all studies, sample handling from the farm until processing was similar, especially storage, freezing, and processing of the samples. Milk samples were obtained by proper cleaning of the teat and after discarding the first 3 to 4 squirts of milk. In most studies, samples were stored at –20°C until processed.
Intervention
BDCT vs. No DCT (n = 18 Studies).
The treatment was based on intramammary infusion of the antibiotic being tested in all quarters and all studies except 2 (Soback et al., 1990; Tarabla and Canavesio, 2003), in which systemic DCT was applied intramuscularly.
Cloxacillin vs. Other DCT Product (n = 5 Studies).
In the studies involved in the analysis, cloxacillin and the other product were applied by the intramammary route.
SDCT vs. No DCT and SDCT vs. BDCT (n = 5 Studies in Each).
Treatment was carried out in the SDCT group based on the unit of selection. Cow-level selection treatment resulted in treating all quarters of the whole udder. At the quarter level, only quarters that had elevated SCC or IMI were treated. Other quarters within the same udder were left untreated. When the control group was no DCT, no antibiotic was infused (negative control), but when BDCT was the control group (positive control), all quarters of the udder were infused with the same antibiotic that was used in the SDCT group at the quarter level only.
TS vs. No TS and TS + BDCT vs. BDCT (n = 4 Studies).
Studies used internal TS, which was applied after proper hygiene of the teat and after DCT in the case of positive control group studies.
Meta-Analysis Results
BDCT vs. No DCT.
In the separate univariable models, gram (gram positive vs. gram positive and negative), year of publication (because of suspicions of bacterial antibiotic resistance over time), and the continent where the study originated were selected to be included in the multivariable model (P < 0.3). None of the variables found to be associated at the univariate level was found to be associated in the final (multivariable) model.
Overall, BDCT was protective against new IMI compared with untreated controls (Table 2). However, the protection varied among pathogens (Table 2). When Staphylococcus spp. were involved, and before adjusting for publication bias, DCT quarters had 0.62 (0.47 to 0.83) times less risk of new IMI than untreated quarters. Applying the one study removed method showed that removing any of the studies did not alter the random pooled RR significantly (Figure 1). The Begg and Mazumdar rank correlation test suggested that there was no correlation between the study size and effect (P = 0.33). However, the regression test of Egger et al. contradicted the Begg and Mazumdar rank correlation test, suggesting a significant association between study size and effect size (intercept = –1.73 with 16 df, one-tailed P = 0.01). The fill and trim method of Duval and Tweedie indicated no missing studies on the left-hand side of the funnel plot, but 7 studies (black spots) were missing on the right-hand side to reach complete symmetry (Figure 2). This figure indicates that if publication bias did not exist and complete symmetry was reached by including the missing studies, the preventive effect would shift to the null effect. Because significant publication bias was indicated (Figure 2), the number of studies necessary to move the pooled RR above 1 was calculated using fail-safe N method of Orwin. According to the Orwin fail-safe N method, and when the mean RR in the missing studies was assumed to be 1.10, the number of necessary studies was 17, but when the mean RR in the missing studies was assumed to be 1.20, the number of necessary studies was only 8. This number of studies is quite low, suggesting that the protective effect of DCT against new Staphylococcus spp. IMI during the DP up to 21 d postcalving might be truly insignificant.
|
|
|
Cloxacillin vs. Other DCT Products.
In general, other DCT showed similar protective effects from new IMI during DP compared with cloxacillin (Table 2). Removing one study altered the effect, which is expected when a small number of studies are used (results not shown). The Duval and Tweedie fill and trim method suggested that 2 studies were missing from the right-hand side, which would support the finding that cloxacillin provides similar protection compared with other DCT (results not shown). The regression test of Egger et al. supported the presence of publication bias (P = 0.01). Correcting for publication bias would only confirm the nonsignificant difference between cloxacillin and other DCT in protection against new IMI during the DP.
There was no significant difference between cloxacillin and other DCT against new Staphylococcus spp. quarter IMI (Table 2). All publication bias tests indicated an absence of significant publication bias. The analysis was not carried out for other pathogen groups because the number of studies available was less than 4.
SDCT vs. No DCT or SDCT vs. BDCT.
Selective DCT provided significant protection against new quarter IMI, and protection was slightly higher when selection and treatment were carried out at the cow level (Figure 3). When SDCT was compared with BDCT, BDCT showed higher protection than SDCT (Figure 4). Nonetheless, there was no significant difference in protection from new quarter IMI between SDCT and BDCT when the selection unit was the cow, with the whole udder treated (Figure 4). When the SDCT selection unit was the quarter, BDCT provided better protection from new quarter IMI (on a population level) than SDCT (Figure 4).
|
|
TS vs. No TS or TS + BDCT vs. BDCT.
The only study with the positive control group gave similar protection compared with the other studies (P >0.05); therefore, a combined RR based on all studies is presented. In general, TS-injected quarters had 0.39 (0.18 to 0.82) times less risk of new IMI than non-TS-injected quarters (Table 2). None of the publication bias tests indicated the existence of bias (results not shown). Analysis was not conducted per pathogen group because of the limited number of studies.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGMENTSAPPENDIXREFERENCES |
|---|
Dry cow therapy was the largest group of studies, but results were heterogeneous, indicating unexplained risk factors for the efficacy of DCT and that perhaps a production system-dependent decision about the efficacy of DCT may be important to consider. Several attempts were made to restrict the heterogeneity. Inclusion only of studies with a negative control group, or only of studies that reported new quarter IMI, or only of studies conducted on dairy cows (excluding primiparous and beef cattle), or only randomized studies did not improve the homogeneity. Including only peer-reviewed papers in the English language could have been a source of heterogeneity. Language might be a source of bias, because non-English-speaking researchers might publish their positive results in English-language journals to gain more publicity, whereas they might publish their negative results in their native language (Gregoire et al., 1995). Although attempts were made to use meta-regression with several potential variables to explain the heterogeneity, none of them was able to explain it significantly. A possible source of heterogeneity could be the length of the DP such that the longer the DP, the higher the chance of new IMI because of the lower concentration of antibiotics around calving (Rindsig et al., 1978). However, because this information was not available in most studies, and because the length of the DP could differ considerably among cows within a study, it was not possible to test the effect of length of the DP on heterogeneity in the meta-regression.
The protective effect of DCT against new coliform IMI was not significantly higher than for untreated quarters (Table 2). Publication bias did not influence this result. In a previous meta-analysis (Robert et al., 2006), DCT also did not significantly protect cows from new coliform IMI during the DP. This was explained by the fact that new coliform IMI occurs late in the DP, when DCT might not provide protection against new IMI any longer because of the low concentration (Robert et al., 2006). Another explanation of the apparent lack of efficacy is that the spectrum of activity of many of the DCT products did not cover gram-negative pathogens (Bradley and Green, 2001).
On the basis of a random effect model, initially BDCT provided significant protection against new quarter Staphylococcus spp. IMI. However, publication bias tests indicated the presence of significant bias. Adding 7 studies to the right-hand side of the funnel plot (Figure 2), led to nullifying the effect and suggested that DCT does not protect significantly against new Staphylococcus spp. IMI. Although large and small studies are present on both sides of the plot, 3 large studies, including the largest study, suggested that DCT does not protect against new Staphylococcus spp. IMI (Figure 2). The fact that the inclusion criterion of many studies that showed significant protection against new Staphylococcus spp. IMI was not mentioned (Table A1) suggested a possible lower quality study design in those studies. Moreover, the 7 missing studies might have been conducted but not published in peer-reviewed journals because they showed no protective effect. The fact that 2 of these studies were small and another 2 studies relatively small (Figure 2) would support this speculation regarding why they were not published (Dohoo et al., 2003). The reason for not publishing the data could be that those studies were sponsored by commercial companies and that it would not be commercially beneficial to publish such data (Thornton and Lee, 2000). Another reason could be that the results might not have been interesting enough for publication. Many peer-reviewed journals do not publish results if they are not striking or interesting enough or if they fail to show significant differences between treatment effects (Ferguson, 2007). Moreover, researchers themselves might not attempt to publish their research when the results contradict previous expectations (Ferguson, 2007). The regression test of Egger et al. (1997) was reported to be powerful in detecting publication bias for heterogeneous data such as ours (Peters et al., 2006). The fail-safe N method of Orwin (1983) showed that adding a few studies would completely reverse the pooled RR and move it above 1. The disadvantage of the fail-safe N method is that it calculates the number of studies that are necessary to alter the effect, but does not calculate the number of studies that are needed to reach a null effect, as indicated by Rothstein (2008). Therefore, in this case, fewer studies are expected to make the difference insignificant, depending on the average RR in those studies.
In a previous meta-analysis (Robert et al., 2006), DCT quarters had an RR of 0.75 (0.61 to 0.91) for new coagulase-positive Staphylococcus spp. IMI during the DP, but DCT quarters did not have significantly less risk in the case of coagulase-negative Staphylococcus. However, the potential publication bias was not investigated further in that study. A separate analysis for coagulase-positive Staphylococcus in the current study showed the same trends as for the Staphylococcus group as a whole, indicating that asymmetry existed because of the missing studies (results not shown). Moreover, because subtherapeutic doses were tested in 4 studies (Table A2), which might have affected the symmetry around the pooled RR, a separate subset was analyzed by removing protocols with subtherapeutic doses. Nevertheless, the fill and trim method of Duval and Tweedie (2000) suggested adding 5 studies, mainly small studies, to the right-hand side of the plot to reach complete symmetry, which would lead to nullifying the protective effect against new Staphylococcus spp. IMI during the DP up to 21 d postcalving (results not shown).
The procedure followed in the meta-analysis of Robert et al. (2006) to calculate the pooled RR apparently assumed one true pooled RR (fixed effect model). Because of the heterogeneous nature of the studies selected, this assumption was violated. Therefore, the pooled RR should have been calculated by assuming a distribution of true RR (random effect model), in which weighting would be applied differently, as indicated by Borenstein et al. (2007) and as applied in recent research (Duffield et al., 2008). Because different assumptions are involved for the different models of calculating the pooled RR, the pooled RR would not be precise, and in some cases, it might actually fall out of the confidence limit of the true pooled RR (Borenstein et al., 2007). In the current study, the nature of the studies selected was taken into account and proper models and assumptions were considered.
Several studies found that many new Staphylococcus spp. IMI occur late in the DP and in early lactation (Soback et al., 1990; Hassan et al., 1999). Soback et al. (1990) indicated that the failure of DCT to prevent new Staph. aureus IMI during the DP could have occurred because of the failure of the DCT to cure existing infections at dry off. Hogan et al. (1994) showed that DCT was not successful in preventing new Staph. aureus IMI during the DP. They suggested that because Staph. aureus is part of the normal flora of the teat skin, in the late DP, the teat canal would open and the bacteria would have access, causing new IMI. Rindsig et al. (1978) observed that the longer the DP, the greater the chance of new Staphylococcus spp. IMI, and reasoned that the decreased DCT concentration provided a less prophylactic effect. Analysis of data from a recent large field trial in Norway indicated that neither SDCT nor BDCT protected cows significantly from new quarter Staph. aureus IMI during the DP and early lactation (A. C. Whist, TINE Norwegian Dairies, Norwegian Cattle Health Services, Ås, Norway; unpublished data). When the publication bias tests indicate a potential bias, it might not ultimately be true. However, the methods are useful, because they indicate a potential bias, and hence enhance thinking of possible explanations that should be based on rational and biological reasoning. In our case and for all the rational and biological explanations mentioned above, it was deemed that publication bias could truly have existed in the peer-reviewed literature, hence making it difficult to conclude on the protective effect of BDCT against new quarter Staphylococcus spp. IMI during the DP up to 21 d postcalving. On the other hand, the pooled RR was <1, which economically might be good enough to pay off the cost of DCT, but economic analysis has still to be conducted.
The preventive effect of SDCT was significantly higher than not treating quarters (Table 2). When SDCT was compared with BDCT, and when selection was carried out at the cow level for SDCT, there was no significant difference between SDCT and BDCT. However, BDCT showed higher protection when SDCT selection was carried out at the quarter level (Figure 4). This might be explained because an infected quarter in a cow could have a higher chance of infecting other healthy quarters in the same cow during the DP than infecting healthy quarters in other cows in the DP (Buddle et al., 1987). Treatment at the cow level could or could not cure this infected quarter, but at least would prevent the infection of other healthy quarters. Another possible reason is a misclassification error, whereby some glands within a cow are incorrectly left untreated, for example, because of intermittent shedding of bacteria. Nonetheless, the small number of studies per stratum precluded meta-analysis per stratum; therefore, the results are presented as a stratified overall effect.
Generally, internal TS provided significant protection against new IMI (Table 2). Although large studies do exist, only 4 studies were found to fit our criteria. Therefore, more studies could be necessary to draw conclusions properly on the efficacy of TS.
A meta-analysis was conducted on studies that challenged the effect of external supplementation to enhance the immune system to protect against new IMI during the DP. The supplementations did not provide significant protection against new IMI during the DP. Similarly, teat dipping did not provide protection against new IMI during the DP. The number of studies per comparison was low and the diversity among studies was high, which precluded proper comparison and hence conclusions on the protective effect of these interventions against new IMI during the DP.
|
CONCLUSIONS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGMENTSAPPENDIXREFERENCES |
|---|
|
ACKNOWLEDGMENTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGMENTSAPPENDIXREFERENCES |
|---|
|
APPENDIX |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGMENTSAPPENDIXREFERENCES |
|---|
|
|
|
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONCONCLUSIONSACKNOWLEDGMENTSAPPENDIXREFERENCES |
|---|
Related articles in JDS:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
