The study identified all published articles labeled as “protocols” in the Health behavior, health promotion and society section of BMC Public Health between 2011 and 2015. We limited our review to these years to ensure a reasonable time interval between protocol publication, trial completion, and publication of trial results. For each study protocol, we recorded the date of receipt by BMC Public Health-a time point that precedes the actual publication date of the journal protocol. In light of evidence showing poor adherence to registration among observational studies, we further restricted protocols to described registered randomized controlled trials (RRCTs).
record data abstraction
For each RRCT described, we reviewed associated online registration requests from 13 different trial registration sites and recorded the following data: trial registration number (if available), date of registration. ‘essay (the earliest date of essay submission for registration, which may precede the official date of publication online registration); trial start date (meaning the date participants first enrolled in the trial or the date data was first collected, whichever comes first); trial end date (i.e. the date of final data collection); the primary outcome(s) to be measured in the trial; and whether the primary outcome(s) changed after the trial start date (where this information was available). Given the different fields/language used for each of the 13 registration sites, we assessed each site and standardized fields to our study definitions of trial registration date, trial start date, date end of trial, and primary outcome(s) (Supplementary Table 1). Full details of all data collected can be found in Supplementary Table 2.
Our first analysis was limited to RRCTs that were prospectively registered (ie trials that were registered before the trial start date). One registration site (clinicaltrials.gov) only reports the month/year of the trial start date. To mitigate potential bias effects that could arise from trials enrolling late in the same month as their trial start date, we deemed prospective any trial registered at clinicaltrials.gov within the first 10 days of the same month that the trial is reported to have begun. If a trial registered at clinicaltrials.gov was registered and started in the same month, and the registration was after the tenth day of the month, it was considered retrospective.
Identification of published test results
After finalizing the analytical cohort of RRCTs, we reviewed the trial register for references to subsequent publications of the presentation of results regarding the study of outcomes. Next, a systematic search of publications describing primary outcome results was performed for each RRCT using the following databases: MEDLINE (Pubmed); APA PsycINFO; and Web of Science. A defined set of search terms was used for each database, and included: full name of principal investigator; trial registration number; unique assay name (if applicable); and keywords relevant to individual RRCTs. The searches were carried out in June and July, 2020; a detailed account of the search terms used in this study can be found in Supplementary Table 3.
Once publications reporting results for each RRCT’s primary outcomes were identified, up to three letters were emailed to each of the principal investigators or primary contact of prospectively registered trials asking them to review the list. publications identified. If no publications were identified by the database search, the principal investigator or lead contact was asked to confirm that there were no subsequent publications from the study.
The two investigators independently reviewed the publication of the outcome(s) for the RRCTs included in the analytical cohort and compared the primary outcomes reported in these outcomes to the primary outcomes recorded. The two investigators met and discussed their assessments of the concordance between the primary recorded results and those reported in subsequent publications. The review and assessment of each study focused on describing the main outcomes and how these were measured, as well as the timing of the assessment and follow-up point(s). For registers, we used the primary outcome and the secondary outcome categories included in these outcomes to differentiate primary and secondary. For publications, we used as primary or secondary categorization of author variables when they were specified; when they are not, and the authors reported outcome variables, we considered them to be all primary outcomes.
The specific objective of the review was (a) the introduction of unrecorded primary results in the publication, (b) the promotion of secondary results listed as primary results in the publication, failure (c) the report on a primary outcome listed in the publication, (d) relegation of a primary outcome recorded to a secondary outcome in the publication, and (e) failure to follow the follow-up assessment protocol recorded in the publication. Additionally, trial registration applications frequently contained vague descriptions of the two primary outcomes and the timing of follow-up assessments and often failed to specify how the primary outcomes would be assessed/measured. These vague registry descriptions of primary outcomes, measurements and evaluation periods made it difficult to verify whether these were consistent with those reported in the publications; Indeed, a vague description without specified measurement could be in agreement with a large number of results and measurements described in a document. Accordingly, studies were also assessed in terms of (f) whether the Registry description of primary variables was vague, (g) whether measurements of primary variables were specified, and (h) whether the assessment protocol was tracking was well -defined.
Recorded protocol changes
This study was registered in the AsPredicted registry before data collection began. The Registry entry, which is found in Supplementary Materials, proposed two phases: Phase 1 (Objective 1)—to examine whether consistency between information in published protocols pertaining to social and behavioral science research public health and subsequent publications of these research projects; Phase 2 (Objective 2)—to distinguish two groups of published manuscripts identified in Phase 1: (a) those that followed their explicitly associated protocols and (b) those that did not. This document relates to Objective 1. We have offered for review in the registry 51 protocols published in BMC Public Health in 2011 and 2012. As shown in the 10 results section below, we discontinued the use of the BMC Public Health protocols as a source of results since so little pre-established were published before the start time of the studies they described. Instead, we used the records cited in these protocols as the source of the previously defined results. However, since half of the registrations were also retrospective, we had to extend the period from 2011-2012 to 2011-2015 in order to identify a sufficient number of protocols that describe studies that were registered prospectively.
In the registry, we proposed to consider the following discrepancies between protocols and publications: (a) a primary outcome in the protocol is reported as a secondary outcome in the published document; (B) a secondary outcome in the protocol is reported as a primary endpoint in the published article; (C) a primary outcome described in the protocol is omitted from the published document; (D) a secondary outcome described in the protocol is omitted from the published document; (E) a primary outcome not listed in the protocol is indicated in the published document; (F) A secondary outcome that is not described in the protocol is indicated in the published document. Given the additional time and labor required to collect data from registers to conduct the study, we limited the analysis to the main outcomes reported in these areas and therefore did not assess agreement between registers and publications. on the secondary results.