Randomized trials are often presented at medical conferences and published simultaneously or later. Predictors of simultaneous publication and its consequences are undetermined. Our aim was to characterize the practice of simultaneous publication, identify its predictors, and evaluate its impact.
MethodsIn this cross-sectional study, we included randomized trials presented at late-breaking science sessions of major cardiovascular conferences from 2015 to 2021. The association of trial characteristics with the timing of publication was analyzed. The impact of simultaneous vs nonsimultaneous publication was investigated on the number of 1-year citations and 1-month mentions, and the total citations and mentions at the longest observation follow-up.
ResultsOf 478 trials included in the analysis, 48.7% were published simultaneously. Simultaneous publications were more likely to be presented in the main conference room (OR, 6.09; 95%CI, 1.34-36.92; P=.029) and were characterized by a shorter review time (OR, 0.95; 95%CI, 0.91-0.96; P<.001). Simultaneous publications were associated with higher 1-year citations (R2, 43.81; 95%CI, 23.89-63.73; P<.001), 1-month mentions (R2, 132.32; 95%CI, 85.42-179.22; P<.001) and total citations (R2, 222.89; 95%CI, 127.98-317.80; P<.001).
ConclusionsRandomized trials presented in the main conference room and with shorter review time were more likely to be published simultaneously. Simultaneous publications were associated with more citations and mentions than nonsimultaneous publications.
Keywords
The results of randomized trials are commonly presented at major medical congresses in cardiovascular medicine during late-breaking trial sessions.1–4 Trials highlighted in these sessions are typically relevant for practicing clinicians and often inform practice guidelines following their simultaneous or nonsimultaneous publication in top tier peer reviewed medical journals.5
The simultaneous or parallel publication of a late-breaking trial (eg, when the release of the full paper is concomitant to the congress presentation of the study) enhances the dissemination of the results at a time when the visibility and the impact of the study are at a critical moment owing to news outlets and press releases.6 The rapid access to a reference of peer reviewed data that expand those included in the congress presentation has the potential to increase the impact of the study as it relates to citations, social media metrics, and implementation in clinical practice.7 In addition, a simultaneous publication can bring value to multiple stakeholders, including authors, journals, medical societies, congresses, study sponsors, and readers themselves.8 Many journals and conferences have now developed guidelines for simultaneous publications and make public calls for these submissions.9 However, the process of developing simultaneous publication can pose multiple challenges, including shortened timelines, reviewer fatigue, manuscript rejection, and tight management of processes and timelines.10
In the case of nonsimultaneous publication (eg, when there is a certain time lag between the presentation of a study and its publication), the reporting of the study results may not benefit from the same degree of dissemination, and the authors may miss the opportunity to gain additional impact in the short term.11 However, delaying publication also has potential benefits. For example, following conference presentation, the authors benefit from the comments received, which may lead to improving the manuscript before publication. On the other hand, some readers interpret delays that are too long as a sign of publication bias.12
To our knowledge, the practice of simultaneous publication has not been thoroughly investigated so far. Academic cardiovascular medicine is a highly competitive field with several conferences and publications, making it an ideal area to investigate this phenomenon. In this cross-sectional study, we aimed to compare the characteristics, predictors, and impact of simultaneous vs nonsimultaneous publication in the field of cardiovascular medicine.
METHODSStudy eligibilityLate-breaking trials presented at the American College of Cardiology (ACC), American Heart Association (AHA), European Society of Cardiology (ESC) and Transcatheter Cardiovascular Therapeutics (TCT) conferences were scrutinized from January 2015 to December 2021. Two investigators independently searched PubMed, systematically reviewed, and agreed on the eligibility of peer reviewed publications corresponding to such trials. The studies were included if they were: a) randomized trials of devices, drugs, or strategies; b) active- or placebo- controlled; c) powered for superiority, noninferiority, or equivalence; d) published in a peer reviewed medical journal. Substudies and subanalyses were excluded. Studies encompassing 2 coprimary endpoints were included as a single entity, as they correspond to a presentation and its corresponding publication. Follow-up extensions were included only if they were prespecified by the original trial statistical plan. Studies selected for inclusion were grouped based on their time of publication (ie, simultaneous with the conference presentation, or nonsimultaneous otherwise).
Data collection and outcomesFor each study included in the analysis, several baseline characteristics were collected, including details of the study (eg, focus, purpose, conduct, design, sample size, power, number of centers and countries, follow-up, sponsor, authorship), conference (eg, name, room of presentation, date of presentation), publication (eg, date of submission, date of publication) and journal (eg, name, impact factor, publisher). The impact of each study was quantified using Altmetric Explorer and “dimensions” service, following a research authorization. Total, 1-year and 2-year citations (eg, in the published literature), and total and 1-month mentions (eg, in social media and news outlets) were collected. Because Altmetric Explorer monitor article citations by calendar year, when citations spanned 2 calendar years, we employed a weighted aggregation method to ensure methodological consistency while accommodating temporal overlaps.
Statistical analysisContinuous variables are reported as the means and standard deviation or the median with interquartile range [IQR] based on data distribution and were compared with the Student t test or Mann-Whitney tests. Categorical variables are reported as frequencies and percentages and were compared with the chi square or Fisher exact tests or ANOVA, as appropriate. Counts for missing baseline characteristics are reported alongside the results. The temporal trend of simultaneous publications was analyzed using the Box-Ljung test.
Univariable logistic regression analyses were conducted to explore the association of simultaneous publication with baseline study, conference, publication, or journal characteristics. To identify independent predictors, the variables showing a significant association in the univariate analysis were entered into 2 multivariable models, using simultaneous publication as the outcome variable. One model included all the characteristics found to be significant in the previous step, while the other omitted ‘review time,’ a variable known to be shorter for manuscripts undergoing simultaneous publication. The Akaike's information criterion was used to measure the goodness-of-fit of each model.
We used linear regression analyses to explore the association of simultaneous publication with total citations, 1- and 2-year citations, total mentions and 1-month mentions. Additionally, to account for the observation lag variable (ie, time elapsed between study publication and data collection), which is inevitably associated with the number of citations and mentions, multivariable linear regression analyses were performed using observation time and simultaneous publication status as exploratory variables.
All the tests were conducted at a 2-sided 5% significance level and were performed with R, version 4.0.5 (R Foundation for Statistical Computing, Vienna, Austria). The study is reported based on STROBE guidelines for reporting of observational research (table 1 of the supplementary data).
RESULTSStudy populationThe study flow chart is provided in figure 1 of the supplementary data. After screening of the selected medical conferences, 955 studies were identified, and 478 matched the eligibility criteria. Of these, 233 (48.7%) were published simultaneously. The proportion of simultaneous publications was generally consistent across the conferences examined and over time (P=.652 for trend); however, year 2018 registered a higher number of presentations (figure 1).
Annual trends in simultaneous and nonsimultaneous publication of randomized trials. The bar chart displays the annual trends in simultaneous and nonsimultaneous publication of randomized trials in cardiovascular medicine from 2015 to 2021. The y-axis shows the cumulative percentage of publications, with simultaneous and nonsimultaneous bars represented in red and sky blue, respectively. The x-axis shows the years, with each year represented by a separate bar. The number of trials published each year is shown at the top of each column.
Key characteristics of studies with simultaneous and nonsimultaneous publication are reported in table 1 and 2 of the supplementary data. Studies with simultaneous publication were more commonly multicenter trials, selected by the organizers for presentation in the main conference room, and published in a journal in the first quartile by impact factor. Additionally, these studies more frequently had a sponsor involved in the study design, were more likely to have a characterizing trial name or acronym, a larger sample size, and a longer follow-up, were more frequently powered for hard clinical endpoints (eg, mortality or myocardial infarction), and had a shorter total review time from journal submission to acceptance (23 [IQR 17-35] vs 107 [51-168] days; P<.001).
Studies with nonsimultaneous publication had a median time to publication of 239 [131.00-346.00] days. Compared with those published simultaneously, these studies more frequently reported neutral results (42.0% vs 31.8%; P=.026) and were more frequently presented during the final day of the conference (29.0% vs 18.9%; P=.022). The New England Journal of Medicine had the largest number of simultaneous publications (47.6%), while the Journal of the American Medical Association had the largest number of nonsimultaneous publications (15.1%) (figure 2). Simultaneous publications were more frequently accompanied by editorials than nonsimultaneous publications (61.8% vs 51.8%; P=.039). This calculation included editorials published at the time of ahead of print publications and editorials published later in the publication process (eg, when the ahead of print article was allocated to an issue of the journal) (table 1). However, simultaneous publications were less likely to have a simultaneous editorial (42.3% vs 66.1%; P=.027) (table 2 of the supplementary data).
Journals publishing randomized trials with simultaneous or nonsimultaneous publication. The figure presents a breakdown of peer reviewed journals publishing randomized trials simultaneously or nonsimultaneously after presentation at major cardiovascular conferences between 2015 and 2021. The journals are arranged based on the frequency of simultaneous publication. EHJ, European Heart Journal; JACC, Journal of the American College of Cardiology; JAMA, Journal of the American Medical Association; NEJM, New England Journal of Medicine. *The category “Others” refers to cumulative values for journals publishing<10 simultaneously presented manuscript over the observed timeframe.
Baseline characteristics of randomized trials presented at major cardiovascular congresses with simultaneous or nonsimultaneous publication
Simultaneous (n=233) | Nonsimultaneous (n=245) | P | |
---|---|---|---|
Presentation characteristics | |||
Congress | .865 | ||
ACC | 66/233 (28.3) | 66/245 (26.9) | |
AHA | 52/233 (22.3) | 55/245 (22.4) | |
ESC | 76/233 (32.6) | 79/245 (32.2) | |
TCT | 39/233 (16.7) | 45/245 (18.4) | |
Day of presentation | .022 | ||
First day | 54/233 (23.2) | 58/245 (23.7) | |
Last day | 44/233 (18.9) | 71/245 (29.0) | |
Middle days | 135/233 (57.9) | 116/245 (47.3) | |
Main conference room | 91/111 (81.9) | 85/135 (62.9) | |
COVID-19 period | 61/233 (26.1) | 64/245 (26.1) | .981 |
Study characteristics | |||
Sample size | 233/233; 1384 [511-4465] | 245/245; 590 [248-1919] | <.001 |
Study design | |||
Superiority | 182/233 (78.1) | 191/245 (78.0) | 1.000 |
Noninferiority | 63/233 (27.0) | 53/245 (21.6) | .204 |
Hard primary endpoint | 169/233 (72.5) | 146/245 (59.5) | .003 |
Blinding | .451 | ||
Open label | 128/233 (54.9) | 145/245 (59.4) | |
Single | 26/233 (11.2) | 24/245 (9.8) | |
Double | 75/233 (32.2) | 75/245 (30.6) | |
Triple | 4/233 (1.7) | 1/245 (0.4) | |
Multicenter | 223/233 (95.7) | 216/245 (88.2) | .004 |
Intercontinental trialists | 103/233 (42.2) | 97/245 (39.5) | .348 |
Sponsor role | |||
Industry funded | 171/233 (73.4) | 172/245 (70.2) | .502 |
Involved in trial design | 101/229 (44.1) | 79/237 (33.3) | .032 |
Involved in trial analysis | 69/229 (30.1) | 57/237 (24.5) | .187 |
Study topic | .292 | ||
Drugs | 112/233 (48.1) | 109/245 (44.5) | |
Devices | 71/233 (30.5) | 65/245 (26.5) | |
Drugs and Devices | 1/233 (0.4) | 2/245 (0.8) | |
Strategy | 49/233 (21.0) | 69/245 (28.2) | |
Study analysis | |||
Statistical power >90% | 82/221 (37.1) | 66/227 (29.1) | .120 |
Intention to treat | 205/225 (91.1) | 207/224 (92.4) | .055 |
Study conduct | |||
Study stopped prematurely | 12/233 (5.2) | 16/245 (6.5) | .654 |
Follow-up, wk | 233/233; 52 [26-126] | 245/245; 52 [16-80] | .005 |
Neutral result | 74/233 (31.8) | 104/245 (42.4) | .026 |
Trial name adoption | 155/233 (66.5) | 105/245 (42.9) | <.001 |
Established corporate authorship | 24/233 (10.3) | 22/245 (9.0) | .738 |
Publication timing, days | |||
Registration to publication* | 233/233; 1709 [1212-2332] | 245/245;1782 [1240-2430] | .287 |
Review | 55/233; 23 [17-35] | 110/245; 107 [51-168] | <.001 |
Presentation to publication | - | 245/245; 239 [131-346] | <.001 |
Journal characteristics | |||
Journal Q1 | 232/233 (99.5) | 227/245 (92.6) | <.001 |
Accompanying editorial | 144/233 (61.8) | 127/245 (51.8) | .039 |
ACC, American Congress of Cardiology; AHA, American Heart Association; COVID-19, coronavirus disease 2019; ESC, European Society of Cardiology; Q1, first quartile; TCT, transcatheter cardiovascular therapeutics.
Data are expressed as n/N (%), or median [interquartile range].
Studies with positive results (ie, those meeting the primary endpoint) had a lower median time from protocol registration to publication (1671 [IQR 1153-2264] vs 1873 [IQR 1334-2616]; P=.001), and similar proportions of ahead of print publication (75.1% vs 82.8%; P=.10) compared with studies not meeting the primary endpoint (ie, neutral). Additionally, studies with positive results gained a higher median number of total citations than studies with neutral results (127 [IQR 52.5-329] vs 123 [48-322.5]; P=.002). A similar proportion of studies with positive and neutral results were industry funded (35.0% vs 43.0%; P=.10).
Predictors of simultaneous publicationOn univariate analysis, several candidate predictors of simultaneous publication were identified (table 3 of the supplementary data) and entered into the 2 multivariable models (table 2). In model 1 (with review time as a continuous variable), presentation in the main conference room (odds ratio [OR], 6.09; 95% confidence interval [95%CI], 1.34-36.92; P=.029) and review time (OR, 0.95; 95%CI, 0.91-0.96; P<.001) emerged as independent predictors. In model 2, independent predictors were presentation in the main conference room (OR, 2.55; 95%CI, 1.35-4.96; P=.048) and use of a shortened title identifier, including trial acronyms or characterizing names (OR, 2.50; 95%CI, 1.41-4.50; P=.002). Model 1 showed a better goodness-of-fit (Akaike's information criterion=79).
Multivariable regression analyses among selected study characteristics and simultaneous publication
Variable | Odds Ratio | (95%CI) | P |
---|---|---|---|
Model 1 (AIC: 77) | |||
Presentation characteristics | |||
Main conference room | 7.75 | (1.24-74.75) | .045 |
Study characteristics | |||
Neutral results | 0.32 | (0.06-1.46) | .156 |
Follow-up duration, wk | 1.00 | (1.00-1.01) | .161 |
Multicenter | 1.36 | (0.13- 14.61) | .794 |
Sponsor involved in trial design | 3.25 | (0.53-24.28) | .218 |
Trial name adoption | 2.18 | (0.42-12.62) | .361 |
Hard primary endpoint | 0.40 | (0.07-1.80) | .245 |
Publication characteristics | |||
Journal Q1 | >99 | (0.00-∞) | .992 |
Review time | 0.94 | (0.90-0.97) | <.001 |
Accompanying editorial | 0.17 | (0.13-2.81) | .529 |
Simultaneous accompanying editorial | 0.61 | (0.03-0.71) | .023 |
Model 2 (AIC: 290) | |||
Presentation characteristics | |||
Main conference room | 2.72 | (1.40-5.45) | .004 |
Study characteristics | |||
Neutral results | 0.74 | (0.40-1.33) | .316 |
Follow-up duration, wk | 1.00 | (0.99-1.00) | .672 |
Multicenter | 0.96 | (0.29- 3.47) | .947 |
Sponsor involved in trial design | 1.64 | (0.88-3.10) | .123 |
Trial name adoption | 2.24 | (1.22-4.14) | .010 |
Hard primary endpoint | 1.53 | (0.81-2.91) | .192 |
Publication characteristics | |||
Journal Q1 | >99 | (0.00-∞) | .987 |
Accompanying editorial | 1.48 | (0.81-2.71) | .199 |
Simultaneous accompanying editorial | 0.66 | (0.35-1.24) | .195 |
95%CI, 95% confidence interval; AIC, Akaike's information criterion; Q1, first quartile.
In the univariable linear regression analyses, simultaneous publication was significantly associated with the number of citations at 1 year (R2, 47.24; 95%CI, 27.31-67.16; P<.001), citations at 2 years (R2, 90.43; 95%CI, 41.40-139.46; P<.001), total citations (R2, 240.35; 95%CI, 145.41-335.28, P<.001), and mentions at 1 month (R2, 140.96; 95%CI, 94.04-187.87; P<.001) (table 4 of the supplementary data).
These results were confirmed by the multivariable linear regression model created using observation lag (ie, time elapsed between study publication and data collection) and simultaneous publication status as exploratory variables. In this model, simultaneous publication was significantly associated with higher total citations (R2, 222.89; 95%CI, 127.98-317.80; P<.001), 1-year citations (R2, 43.81; 95%CI, 23.89-63.73; P<.001) and 1-month mentions (R2, 132.32; 95%CI, 85.42-179.22; P<.001). In addition, observation lag time was strongly associated with the total number citations (R2, 0.78; 95%CI, 0.24-1.22; P=.004), but this association was less consistent for 1-year citations and 1-month mentions (table 5 of the supplementary data).
DISCUSSIONThis study provides valuable insights into the phenomenon of simultaneous publication of randomized clinical trials in cardiovascular medicine (figure 3). Our results suggest that trials selected for presentation in the main conference room and those with shorter review times are more likely to be published at the time of their presentation. Additionally, we found that simultaneous publications are associated with higher citation and mention metrics compared with nonsimultaneous publications.
Central illustration. Illustrates the main aspects of the study design and results. On the left, the study flow is outlined, showcasing the aims and methods used. On the right, findings are detailed, highlighting both the predictors and the impact of simultaneous publication. 95%CI, 95% confidence interval; ACC, American Congress of Cardiology; AHA, American Heart Association; ESC, European Society of Cardiology; OR, odds ratio; TCT, transcatheter cardiovascular therapeutics.
The decision by authors to publish a trial simultaneously may be influenced by several factors, such as the need for rapid dissemination of study findings and the desire for early impact and visibility. However, the appeal of the characteristics of the study to conference scientific committees and journals also plays a significant role. Other factors that may contribute to variation in the number of studies published simultaneously across different journals are editorial policies, reviewers’ preferences, and the specific field of study. Several challenges may hinder authors from achieving their goal of simultaneous publication. Various factors, including delays in final manuscript submission, lengthy peer review processes, initial rejections, or unsuccessful attempts to publish in higher impact journals, have diverse implications. Conversely, some authors may choose to respond to calls for science from major conferences even before completing their analyses and crafting a concise key message. These authors leverage potential acceptance as late breakers to secure a place in prestigious medical journals. This scenario emphasizes the need for rigorous assessment of proposed research and underscores the crucial roles of reviewers and editors, even when working under tight timelines. These aspects collectively highlight the intricate nature of decision-making processes in research dissemination.
The growing interest in simultaneous publication among scholars and journals has led some journals to encourage submissions for rigorous but short peer review to increase visibility and dissemination of findings.6,8,13 Nevertheless, no study has provided evidence supporting the practice, or investigated the predictors and features of simultaneous publication, making our study significant in emphasizing its importance and exploring its characteristics. We found similar proportions of studies published simultaneously and nonsimultaneously, and the proportion of studies with simultaneous publication (approximately 1 out of 2) remained constant over time. While authors, conferences, and journals have become more accustomed to simultaneous publication practices over time, the consistent release of such studies can be attributed to persistent underlying factors that may limit substantial growth. These factors encompass the unique characteristics of individual studies and broader trends in scientific knowledge, as well as the ability of the study to address gaps in the existing body of knowledge.
The observation that studies published simultaneously were more commonly presented in the main conference room, reflecting the interest of the scientific committees of the conference in ensuring maximum exposure to its results, highlights the prominent role of the committees responsible for abstract selection, and possibly the indirect influence of their choice on the journals’ publication priorities. In fact, the selection of late-breaking trials usually, but not necessarily, precedes the submission of the manuscript to a top tier scientific Journal, and journal editors may be more inclined to accept studies that are predicted to have wider dissemination and resonance.14
Perhaps not surprisingly, a short review time was also significantly associated with a higher chance of simultaneous publication, indicating the challenge faced by journal reviewers and authors in preparing a peer reviewed manuscript in time for the presentation. This is possibly a spurious association because several top tier journals offer fast track review options to authors who consider submitting a late-breaking trial for potential publication.9 However, this finding also highlights the potential threat associated with a shorter review process. Under such time constraints, reviewers and editors may face heightened pressure to expedite their evaluations, which can potentially lead to reduced accuracy. Reviewers may also find themselves more inclined to accept manuscripts without all comments to the first round being fully addressed, and their comments and decisions may lack the usual level of precision. Similarly, authors, who have to prepare manuscripts and revisions within tight timelines, may be more susceptible to errors or to delivering work that is not as thoroughly prepared. Our study cannot establish the quality of the review process for simultaneous vs nonsimultaneous publications but suggests implications that may warrant further investigation.
When the review time was excluded from the model for statistical adjustment, one more independent predictor emerged. Trials with a characterizing collective name (eg, an acronym) were more likely to be published simultaneously than trials with no name. The reasons for this finding are difficult to identify but could be related to the implicit bias that tends to favor trials with a branded identity. Typically, the design of these trials is known years in advance, anticipated in major conferences, and recorded in repositories such as clinicaltrials.gov.
Consistently, we found that studies published simultaneously were more likely to be cited in the literature and mentioned on social media and news outlets. Interestingly, for total citation number, every additional week of observation was found to be associated with an increase of 0.73 citations; however, this increment was less consistent for 1-month mentions and 1-year citations (table 5 of the supplementary data). Other factors may also contribute to the increased visibility and impact of studies published simultaneously, and correlation does not imply causation. However, these findings may suggest that simultaneous publication can increase the immediate impact of research findings and their impact over time. This effect may ultimately result in an increase in the impact factor of journals accepting late-breaking trials, representing a compelling implicit reason for journals to accept these articles.
LimitationsOur study has some limitations. First, we only included trials presented at major cardiovascular conferences, which may not be representative of trials presented at smaller or more specialized meetings. Second, although we assumed that a larger number of citations as a proxy for broader acceptance of the study results by the scientific community, we did not investigate the impact of simultaneous vs nonsimultaneous publication on the implementation of trial results (eg, into practice guidelines). Third, although we demonstrated that the timing of publication may have a significant impact on dissemination and impact, this is likely not the only factor at play. We cannot determine whether and to what extent other unidentified characteristics that also contribute to the success of a study may have influenced our findings. Fourth, while we observed an association between variables such as presentation in the main conference room and review time, it is important to note that this does not imply causation and the observed relationships may be influenced by various factors and dynamics within the research and publication process. Ultimately, our study only focused on a timeframe of 7 years due to the absence of good quality data before 2015.
CONCLUSIONSOur study revealed that trials presented in the main conference room and with shorter review times were more likely to be published simultaneously. Simultaneous publication was found to be associated with higher short- and long-term citation rates, as well as more short-term mentions. These results shed light on the practice of simultaneous publication and suggest that simultaneous publication is a key component of a successful dissemination strategy.
The predictors and impact of publishing randomized clinical trials simultaneously with their presentation at a medical conference vs their nonsimultaneous publication at a later time are unknown.
WHAT DOES THIS STUDY ADD?Analysis of 478 randomized clinical trials revealed that almost half of them were published simultaneously with their presentation. Trials selected for presentation in the main conference room and with shorter review times were more likely to be published simultaneously in medical journals. Furthermore, simultaneous publication was associated with a higher number of citations and mentions than nonsimultaneous publication. Simultaneous publication is a key component of an effective dissemination strategy and understanding the predictors of simultaneous publication could help researchers and organizers improve the impact of their trials.
No funding was received for the study design, conduct, or analysis.
ETHICAL CONSIDERATIONSThis study, concentrating on the metrics of publications, did not require approval from an ethics committee nor did it involve direct patient data, negating the need for informed consent. Furthermore, the SAGER guidelines for sex and gender reporting were not applicable to our study since our primary subjects were publications, not individual patients. Moreover, we did not collect any variables related to the authors of these publications. The study is reported based on STROBE guidelines for reporting of observational research (table 1 of the supplementary data).
STATEMENT ON THE USE OF ARTIFICIAL INTELLIGENCENo artificial intelligence methodologies or tools were used in the formulation or analysis of this manuscript.
AUTHORS’ CONTRIBUTIONSM. Spagnolo and A. Greco contributed to data curation, formal analysis, methodology, writing, and visualization. D. Capodanno contributed to conceptualization, review and editing, supervision, and validation. All other authors contributed to data curation and review.
CONFLICTS OF INTERESTNone.