Advertisement

Published trials of TACE for HCC are often not registered and subject to outcome reporting bias

  • Jules Grégory
    Correspondence
    Corresponding author. Address: Beaujon Hospital, 100 Boulevard du Général Leclerc, 92110 Clichy, France. Tel.: +33 140874170.
    Affiliations
    INSERM, UMR1153, Epidemiology and Biostatistics Sorbonne Paris Cité Center (CRESS), METHODS team, Hôpital Hôtel Dieu, Paris, France

    Université de Paris, Paris, France

    Department of Radiology, Beaujon Hospital, Paris Nord Val de Seine Hospitals, Assistance Publique des Hôpitaux de Paris, Clichy, France
    Search for articles by this author
  • Perrine Créquit
    Affiliations
    INSERM, UMR1153, Epidemiology and Biostatistics Sorbonne Paris Cité Center (CRESS), METHODS team, Hôpital Hôtel Dieu, Paris, France

    Direction of Clinical Research, Foch Hospital, Suresnes, France
    Search for articles by this author
  • Valérie Vilgrain
    Affiliations
    Université de Paris, Paris, France

    Department of Radiology, Beaujon Hospital, Paris Nord Val de Seine Hospitals, Assistance Publique des Hôpitaux de Paris, Clichy, France

    Laboratory of Imaging Biomarkers, INSERM U1149, Centre for Research on Inflammation, Paris, France
    Search for articles by this author
  • Isabelle Boutron
    Affiliations
    INSERM, UMR1153, Epidemiology and Biostatistics Sorbonne Paris Cité Center (CRESS), METHODS team, Hôpital Hôtel Dieu, Paris, France

    Université de Paris, Paris, France
    Search for articles by this author
  • Maxime Ronot
    Affiliations
    Université de Paris, Paris, France

    Department of Radiology, Beaujon Hospital, Paris Nord Val de Seine Hospitals, Assistance Publique des Hôpitaux de Paris, Clichy, France

    Laboratory of Imaging Biomarkers, INSERM U1149, Centre for Research on Inflammation, Paris, France
    Search for articles by this author
Open AccessPublished:October 16, 2020DOI:https://doi.org/10.1016/j.jhepr.2020.100196

      Highlights

      • Trial registration is fundamental to our understanding and interpretation of results.
      • Half of published randomised controlled trials evaluating transarterial chemoembolisation for HCC were not registered.
      • When registered, one-third had major discrepancies between the registered and published primary outcomes.
      • Selective outcome reporting favouring the trial results was observed in 7% of published reports.

      Background & Aims

      In 2005, the registration of all randomised controlled trials (RCTs) before enrolment of participants became a condition for publication by the International Committee of Medical Journal Editors to increase transparency in trial reporting. Among RCTs on transarterial chemoembolisation (TACE) for the treatment of hepatocellular carcinoma (HCC) published after 2007, we assess the proportion that were registered and compare registered primary outcomes (PO) with those reported in publications to determine whether primary outcome reporting bias favoured significant outcomes.

      Methods

      We searched MEDLINE and EMBASE for reports of RCTs evaluating TACE for HCC treatment between 1 September 2007 and 31 March 2018. Registration and publication information for each included RCT was compared using a standardised data extraction form.

      Results

      Thirteen out of 53 (25%) included RCTs were correctly registered (i.e. before the starting date of the RCT), 14 (26%) were registered after the RCT starting date, and 26 (49%) were not registered. Six out of 14 of the retrospectively registered RCTs (43%) were registered after their completion date. The PO was clearly reported in the published article of all registered RCTs, whereas the report was not clear in 8/26 (31%) of the non-registered RCTs (p = 0.01). Among registered RCTs, 8/27 (30%) had major discrepancies between registered and published PO. The influence of these discrepancies could be assessed in 6 of them and was shown to statistically favour significant results in 2.

      Conclusions

      Registration and outcome reporting in RCTs on TACE for HCC are often inadequate. Registration should be reinforced because it is a key to transparency.

      Lay summary

      Trial registration is fundamental to our understanding and interpretation of results, as it provides information on all relevant clinical trials (to place the results in a broader context), and on the details of their associated protocols (to ensure that the scientific plan is followed). Once a randomised controlled trial (RCT) is completed, the trial results are usually publicly shared via scientific articles that are expected to thoroughly and objectively report them. This study shows that half of the RCTs evaluating transarterial chemoembolisation for hepatocellular carcinoma were not registered, and identified major discrepancies between registered and published primary outcome favouring significant results.

      Graphical abstract

      Keywords

      Abbreviations:

      CENTRAL (Cochrane Central Register of Controlled Trials), ChiCTR (Chinese Clinical Trial Registry), HCC (hepatocellular carcinoma), ICMJE (International Committee of Medical Journal Editors), ICTRP (International Clinical Trials Registry Platform), PO (primary outcome), RCTs (randomised controlled trials), TACE (transarterial chemoembolisation), UMIN-CTR (University Hospital Medical Information Network Clinical Trials registry)

      Introduction

      Liver cancer, which is mainly hepatocellular carcinoma (HCC), is the third leading cause of cancer-related death worldwide, and its incidence is expected to increase in the future.
      • Ferlay J.E.M.
      • Lam F.
      • Colombet M.
      • Mery L.
      • Piñeros M.
      • Znaor A.
      • et al.
      Global Cancer Observatory: Cancer today.
      In 2002, in the studies by Llovet et al.
      • Llovet J.M.
      • Real M.I.
      • Montana X.
      • Planas R.
      • Coll S.
      • Aponte J.
      • et al.
      Arterial embolisation or chemoembolisation versus symptomatic treatment in patients with unresectable hepatocellular carcinoma: a randomised controlled trial.
      and Lo et al.
      • Lo C.M.
      • Ngan H.
      • Tso W.K.
      • Liu C.L.
      • Lam C.M.
      • Poon R.T.P.
      • et al.
      Randomized controlled trial of transarterial lipiodol chemoembolization for unresectable hepatocellular carcinoma.
      transarterial chemoembolisation (TACE) was shown to have a significant benefit to survival compared with the best supportive care for the treatment of HCC. Almost 2 decades later, TACE continues to be the first-line treatment for most HCC patients.
      European Association for the Study of the Liver
      EASL clinical practice guidelines: management of hepatocellular carcinoma.
      ,
      • Heimbach J.K.
      • Kulik L.M.
      • Finn R.S.
      • Sirlin C.B.
      • Abecassis M.M.
      • Roberts L.R.
      • et al.
      AASLD guidelines for the treatment of hepatocellular carcinoma.
      At the same time, research on the efficacy and safety of TACE remains active, either by innovating TACE treatment using emerging technologies, or by evaluating new treatment strategies and combinations of TACE with other treatments.
      In 2005, the International Committee of Medical Journal Editors (ICMJE) initiated a registration policy for randomised controlled trials (RCTs) to increase transparency in research. All RCTs that started recruiting on or after September 2005 were required to be registered prospectively on a publicly accessible web-based registry before participant enrolment as a prerequisite for publication in member journals.
      • DeAngelis C.D.
      • Drazen J.M.
      • Frizelle F.A.
      • Haug C.
      • Hoey J.
      • Horton R.
      • et al.
      Clinical trial registration: a statement from the international committee of medical journal Editors.
      Indeed, trial registration is fundamental to our understanding and interpretation of results, as it provides information on all relevant clinical trials (to place the results in a broader context), and on the details of their associated protocols (to ensure that the scientific plan is followed).
      • Schulz K.F.
      • Altman D.G.
      • Moher D.
      CONSORT Group
      CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials.
      ,
      • Zarin D.A.
      • Tse T.
      • Williams R.J.
      • Rajakannan T.
      Update on trial registration 11 years after the ICMJE policy was established.
      One of the main objectives of registration is to improve the transparency of study results.
      • Tannock I.F.
      False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons.
      Once an RCT is completed, the trial results are usually publicly shared via scientific articles that are expected to thoroughly and objectively report them. However, there may be significant differences between trial registration and published results. The term ‘outcome reporting bias’ refers to the selective reporting of results in trial publications, depending on the nature and direction of the results.
      • Chan A.-W.
      • Hróbjartsson A.
      • Haahr M.T.
      • Gøtzsche P.C.
      • Altman D.G.
      Empirical evidence for selective reporting of outcomes in randomized trials comparison of protocols to published articles.
      Although a study may be published in full, pre-specified outcomes may be omitted or misrepresented to report results more favourably.
      The registration and selective reporting of studies on TACE for HCC have not been examined. Thus, our study had three objectives: (1) to evaluate the proportion of registered RCTs on TACE treatment for HCC among trials published after 2007, (2) to compare registered primary outcomes (POs) with those actually reported in publications, and finally (3) to determine whether primary outcome reporting bias favoured significant outcomes.

      Materials and methods

      We performed a methodological 2-step review of RCTs evaluating the efficacy and/or safety of TACE for the treatment of HCC. First, we identified all RCTs published after September 2007 and systematically searched for registration on online registries. Then, for each registered RCT, we assessed possible selective outcome reporting.

      Search strategy for published RCTs

      We systematically searched MEDLINE (PubMed interface) and EMBASE (Elsevier interface) on April 2019 to identify eligible published RCTs from 1 September 2007 to 31 March 2018 (see Supplementary material for search terms and equations). We chose September 2007 to leave a reasonable delay after the prospective registration requirement date (i.e. September 2005) by the International Committee of Medical Journal Editors.
      • DeAngelis C.D.
      • Drazen J.M.
      • Frizelle F.A.
      • Haug C.
      • Hoey J.
      • Horton R.
      • et al.
      Clinical trial registration: a statement from the international committee of medical journal Editors.
      We also screened reference lists of included studies, and relevant systematic reviews and meta-analyses.
      • Llovet J.M.
      • Bruix J.
      Systematic review of randomized trials for unresectable hepatocellular carcinoma: chemoembolization improves survival.
      • Reidy D.L.
      • Schwartz J.D.
      Therapy for unresectable hepatocellular carcinoma: review of the randomized clinical trials – I: Hepatic arterial embolization and embolization-based therapies in unresectable hepatocellular carcinoma.
      • Cheng X.
      • Sun P.
      • Hu Q.G.
      • Song Z.F.
      • Xiong J.
      • Zheng Q.C.
      Transarterial (chemo)embolization for curative resection of hepatocellular carcinoma: a systematic review and meta-analyses.
      • Chen Q.W.
      • Ying H.F.
      • Gao S.
      • Shen Y.H.
      • Meng Z.Q.
      • Chen H.
      • et al.
      Radiofrequency ablation plus chemoembolization versus radiofrequency ablation alone for hepatocellular carcinoma: a systematic review and meta-analysis.
      • Facciorusso A.
      • Di Maso M.
      • Muscatiello N.
      Drug-eluting beads versus conventional chemoembolization for the treatment of unresectable hepatocellular carcinoma: a meta-analysis.
      • Hyun M.H.
      • Lee Y.S.
      • Kim J.H.
      • Lee C.U.
      • Jung Y.K.
      • Seo Y.S.
      • et al.
      Hepatic resection compared to chemoembolization in intermediate- to advanced-stage hepatocellular carcinoma: a meta-analysis of high-quality studies.
      All the records from these searches were imported via EndNote into Rayyan (a free web application for screening abstracts) to proceed with the screening.

      Identification of relevant published RCTs

      We identified all RCTs that assessed the efficacy and/or safety of TACE for HCC, regardless of the phase, from the retrieved records. To focus on trials in which TACE was the primary treatment and was not an associated treatment, we excluded RCTs if the same TACE procedure was combined with another treatment and was therefore present in all groups or arms. We also excluded systematic reviews or meta-analyses, diagnostic studies, methodological publications, editorial style reviews, abstracts and posters, conference papers, case reports, studies not involving human participants, or studies that were not in English. One reviewer (JG) examined each title, keywords, and abstract and then selected full-text articles according to the pre-specified eligibility criteria.

      Data extraction of published RCTs

      One reviewer (JG) extracted all data from the original reports and supplementary files according to a standardised data collection form (Supplementary material). For quality assurance a second reviewer (PC) collected data from a random sample of 10% of the RCTs. Disagreement was resolved by discussion.
      The following characteristics were systematically recorded from the published article: study location (defined as Eastern-Asia, Western-Europe, North America, and North Africa; location of international RCTs was determined according to the primary investigator); number of involved centres (either single or multicentre); presence of a university-affiliated centre; phase of the RCT when stated; funding source (profit, non-profit, or mixed), sample size (defined as total number of randomised patients); date of journal publication; RCT start and completion dates; type of journal (i.e. radiological or clinical), median journal impact factor in the 2 years before publication; and we additionally collected the interventions to which TACE was compared in each RCT.
      We also assessed the methodological quality of the included RCTs using selected items from the Cochrane risk of bias tool,
      • Higgins J.P.
      • Altman D.G.
      • Gotzsche P.C.
      • Juni P.
      • Moher D.
      • Oxman A.D.
      • et al.
      The Cochrane Collaboration's tool for assessing risk of bias in randomised trials.
      in particular, random sequence generation and allocation concealment. In addition, we investigated whether the delay between randomisation and the TACE procedure, the reasons for and percentages of withdrawal or patient dropouts, and a flow chart were reported in the published article. Finally, we evaluated whether the intention-to-treat analysis was correctly performed.

      RCT registration

      One reviewer (JG) systematically looked to see if each published RCT was registered.
      In particular we checked if the authors mentioned the registration of their trial in the published article and whether a registration number was provided.
      When no registration number was reported in the published article, we searched for a corresponding registration using a combination of keywords, including a description of the interventions and population, location, responsible party, and primary outcome name. We first searched meta-platforms including the Cochrane Central Register of Controlled Trials (CENTRAL) and the International Clinical Trials Registry Platform (ICTRP). We then searched the following primary registries: Clinicaltrials.gov, Clinicaltrialsregister.eu, the Chinese Clinical Trial Registry (ChiCTR), and the University Hospital Medical Information Network Clinical Trials registry (UMIN-CTR).
      RCT registrations were classified according to the ICMJE statement: (1) prospectively registered if the initial registration date was before or within 1 month after the trial start date as stated in the registry (to allow for incomplete start dates and processing delays in the registry); (2) retrospectively registered; and (3) not registered.

      Selective outcome reporting

      Finally, we evaluated the consistency between the primary outcomes as specified in the registered RCT (when possible in the version before patient inclusion began, including amendments, and if not in the most recent available version), and those defined in the published articles. We also recorded whether the primary outcome (PO) was clearly identified in the article (defined and graded as primary and secondary upfront), and if multiple primary outcomes were investigated in the same RCT.
      We defined major discrepancies according to the classification by Chan et al.,
      • Chan A.-W.
      • Hróbjartsson A.
      • Haahr M.T.
      • Gøtzsche P.C.
      • Altman D.G.
      Empirical evidence for selective reporting of outcomes in randomized trials comparison of protocols to published articles.
      as those in which (1) a pre-specified primary outcome in the RCT registration protocol was later reported as a secondary outcome or was not labelled as either in the final published article; (2) a pre-specified primary outcome was omitted from the published articles; (3) the published primary outcome was described as a secondary outcome in the registry; and (4) a new primary outcome was introduced in the published articles.

      Results of the registered and published primary outcomes

      We extracted the p values from the full text article for all the registered primary outcomes and for all the outcomes reported in the article. We quoted results according to statistical significance: results significantly supporting or refuting the study intervention (i.e. p <0.05) and results that did not reach statistical significance (i.e. p ≥0.05).
      A discrepancy was said to favour statistically significant results if a new statistically significant primary outcome was introduced or if a non-significant primary outcome was defined as non-primary in the published article. We also judged the discrepancy as positive when a new statistically non-significant safety primary outcome was introduced in the published article. The influence of some discrepancies could not be assessed because the published text contained no results concerning the registered primary outcome. Thus, the positivity or negativity of the discrepancy was considered to be impossible to decide, and the study was considered impossible to assess.

      Statistical analysis

      Quantitative variables are presented as medians (quartiles 1–3, Q1–Q3) and qualitative variables as n (%). Logistic regression was used to identify RCT characteristics associated with RCT registration status. Two-sided p <0.05 was considered to be statistically significant. All data were analysed with R version 4.2 (https://www.r-project.org, the R Foundation for Statistical Computing, Vienna, Austria), (CTAT Table, Supplementary material).

      Data sharing

      The final data spreadsheet, including the list of eligible RCTs, is available in the Supplementary material.

      Results

      The search retrieved 6,104 references. A total of 349 reports were selected after screening titles, abstracts, and keywords. A total of 75 published RCTs were eligible and 53 were included for analysis. The flow chart is reported in Fig. 1.
      Figure thumbnail gr1
      Fig. 1Selection of randomised controlled trials.
      HCC, hepatocellular carcinoma; TACE, transarterial chemoembolisation.

      Registration of included RCTs

      The description of included articles is presented in Table 1. Twenty-six (49%) RCTs published after September 2007 (i.e. 2 years after the ICMJE statement for prospective registration of RCTs) were not registered. Registration of 14 of the 27 registered RCTs was retrospective (52%). Six of these retrospectively registered RCTs (43%) were registered after their completion date. RCTs that were retrospectively registered before their completion date were more recent (median [Q1–Q3], 2011 [2010–2013]) than those registered after their completion date (2007 [2004–2009], p = 0.02).
      Table 1General characteristics of included randomised controlled trials by registration status.
      CharacteristicsRegistered

      n = 27
      Not registered

      n = 26
      p value
      Prospectively

      n = 13
      Retrospectively

      n = 14
      Study location0.63
       Eastern-Asia (n = 39)8 (21)11 (28)20 (51)
       Western-Europe (n = 9)2 (22)3 (33)4 (45)
       North America (n = 4)3 (75)0 (–)1 (25)
       North Africa (n = 1)0 (–)0 (–)1 (100)
      Number of centres0.20
       Multicentre (n = 12)4 (33)4 (33)4 (33)
       Single-centre (n = 53)9 (22)10 (24)22 (54)
      Type of centre0.21
       University affiliated (n = 43)11 (26)12 (28)20 (46)
       Non-university affiliated (n = 3)0 (–)0 (–)3 (100)
       Both (n = 7)2 (29)2 (29)3 (42)
      Funding source0.08
       Non-profit (n = 50)11 (22)13 (26)26 (52)
       Profit (n = 3)2 (66)1 (33)0 (–)
       Mixed (n = 0)0 (–)0 (–)0 (–)
      Phase0.13
       II (n = 8)3 (37)2 (25)3 (37)
       III (n = 10)6 (60)2 (20)2 (20)
       n.a. (n = 35)4 (11)10 (29)21 (60)
      Median sample size (min–max)189 (92–247)131 (93–238)97 (52–125)<0.01
      Type of journal0.27
       Radiological (n = 14)3 (21)2 (14)9 (65)
       Clinical (n = 39)10 (25)12 (31)17 (44)
      Median date of publication (Q1–Q3)2016 (2013–2017)2014 (2012–2016)2011 (2009–2013)<0.01
      Median date of first inclusion (Q1–Q3)2010 (2009–2013)2010 (2008–2012)2008 (2004–2011)0.47
      Median journal impact factor (min–max)15 (7.6–20.9)5.6 (3.4–7.2)2.3 (1.5–2.9)0.02
      Unless otherwise noted, data are presented as n (%). Percentages are calculated per row. Univariate logistic regression analyses were performed to compare the variables.
      Information on RCT registration was reported in the text of the article in 85% of the registered RCTs (13/13 of the prospectively registered and 10/14 of the retrospectively registered studies, p = 0.05).
      Prospectively registered RCTs were more recently published (median [Q1–Q3], 2016 [2013–2017]) than retrospectively registered (2014 [2012–2016]) and non-registered (2011 [2009–2013], p <0.01) studies. The median (Q1–Q3) journal impact factor was 15 (7.6–20.9) for prospectively registered RCTs, 5.6 for retrospectively registered RCTs and 2.3 for non-registered RCTs (p = 0.02).
      Registration rate by type of intervention TACE is compared with is presented in Fig. 2.
      Figure thumbnail gr2
      Fig. 2Network graph of included randomised controlled trials showing the registration rate by type of intervention compared with TACE.
      Each node represents a treatment and each edge a randomised comparison of two treatments. Thickness of each edge increases with the number of randomised comparisons. Interventions are pooled according to main treatment category. For each group of intervention, number and percentage of registered trials is presented. oTACE, other type of TACE; PA, percutaneous ablation; TAC, transarterial chemotherapy; TACE, transarterial chemoembolisation; TAE, transarterial embolisation; SBRT, stereotactic body radiation therapy; SIRT, selective internal radiotherapy; Surg, surgery.

      Methodological characteristics of RCTs according to registration status

      A flow chart was reported in 13/13 of the prospectively registered RCTs, 12/14 retrospectively registered RCTs (86%), and 8/26 of the non-registered RCTs (31%), (p <0.01). None of the prospectively registered RCTs reported a high-risk/unclear randomisation sequence or allocation concealment compared with 2/14 (14%) and 2/14 (14%) of the retrospectively registered RCTs, and 14/26 (54%) and 17/26 (65%) of the non-registered RCTs (p <0.01), respectively. All prospectively registered RCTs (13/13) performed an intention-to-treat analysis, compared with 13/14 (93%) of the retrospectively registered RCTs, and 15/26 (58%) of the non-registered RCTs (p <0.01) (Table 2).
      Table 2Methodological characteristics of included randomised controlled trials by registration status.
      CharacteristicsRegistered

      n = 27
      Not registered

      n = 26
      p value
      Prospectively

      n = 13
      Retrospectively

      n = 14
      Sequence generation
      According to the Cochrane risk of bias tool.
      (high/unclear)
      02 (14)4 (54)<0.01
      Allocation concealment
      According to the Cochrane risk of bias tool.
      (high/unclear)
      02 (14)17 (65)<0.01
      Reported delay between randomisation and transarterial chemoembolisation6 (46)8 (57)7 (27)0.28
      Dropouts or withdrawal reported13 (100)13 (93)23 (88)0.20
      Reason of dropouts reported13 (100)13 (93)20 (77)0.13
      Intention to treat performed13 (100)13 (93)15 (58)<0.01
      Presence of a flow chart13 (100)12 (86)8 (31)<0.01
      Data are presented as n (%). Percentages are calculated per column. Univariate logistic regression analyses were performed to compare the variables.
      According to the Cochrane risk of bias tool.

      Primary outcome of RCTs according to registration status

      The primary outcome was clearly defined in the published article of all registered RCTs (27/27), whereas this was not reported clearly in 8/26 non-registered RCTs (31%) (p <0.01). Multiple primary outcomes were reported in 1/27 registered RCT publications (4%), and in 8/26 non-registered RCTs (31%) (p = 0.01). Among registered RCTs, multiple primary outcomes were initially registered for 0/13 prospectively registered RCTs and 4/14 retrospectively registered RCTs (29%) (p = 0.01).

      Selective outcome reporting for registered RCTs

      Eight of the 27 registered RCTs (30%) had major discrepancies between the registered PO and those reported in the publication article (Table 3). The frequency of discrepancies was higher in retrospectively registered RCTs (7/14, 50%) than in prospectively registered RCTs (1/13, 8%) (p = 0.01). The list of major discrepancies is shown in Box 1. Only 1 of the 4 registries that initially reported multiple POs reported the same POs in the published article; the 3 others reported a single PO.
      Table 3Proportion of RCTs with major discrepancies in the specification of primary outcomes when comparing registries and published articles.
      Discrepancy between RCT report and registration (n = 27)RCTs with discrepancies for primary outcome
      Primary outcomes specified in protocols
      Categories are not mutually exclusive: 1 RCT could have more than 1 type of discrepancy for different primary outcomes.
       Any change to registration-defined primary outcome8 (30)
       Reported as secondary in published articles6 (22)
       Omitted from published articles3 (11)
      Primary outcomes specified in published articles
       Any new primary outcome defined in published articles5 (19)
       Changed from secondary in registration to primary in published articles2 (7)
       Not mentioned in protocol2 (7)
       Any discrepancy in primary outcome
      Primary outcomes defined in either registration or published articles. RCT, randomised controlled trial.
      8 (30)
      Data are presented as n (%).
      Categories are not mutually exclusive: 1 RCT could have more than 1 type of discrepancy for different primary outcomes.
      Primary outcomes defined in either registration or published articles. RCT, randomised controlled trial.
      Box 1List of major discrepancies between RCT registry and published article in the specification of primary outcomes.
      RCT, randomised controlled trial.
      The influence of the discrepancy could be assessed in 6/8 (75%) of the RCTs with a PO discrepancy between the registry and the published article. The registered PO was omitted from the published article in the two remaining RCTs. Two of the 6 RCTs in which the discrepancy could be assessed, had a discrepancy that favoured statistically significant results (1 had a discrepancy that favoured statistically significant secondary outcomes and 2 non-significant primary outcomes; the other study had a new statistically non-significant safety primary outcome introduced in the published article).

      Discussion

      In a previous study, we showed that despite the ethical commitments and public expectations for the disclosure of results, the availability of RCT results evaluating TACE for the treatment of HCC is very limited.
      • Grégory J.
      • Créquit P.
      • Vilgrain V.
      • Ronot M.
      • Boutron I.
      Results of trials assessing transarterial chemoembolization for treating hepatocellular carcinoma are critically underreported.
      In this study we focused on RCTs on the same topic with results published after 2007, and showed that only half were registered. It is important to note that only 25% were prospectively registered as recommended by the ICJME guidelines. We also found that one-third of registered RCTs had major discrepancies between the registered and published primary outcomes, with 7% presenting with selective outcome reporting favouring the trial results.
      Registration rates have already been studied in other specialties also highlighting the low proportion of correctly registered RCTs, although with some discrepancies. Rosenthal and Dawn
      • Rosenthal R.
      • Dwan K.
      Comparison of randomized controlled trial registry entries and content of reports in surgery journals.
      focused on registered RCTs published in 2010 in 3 leading general surgical journals. This study identified 55 RCTs including 4 (8%) which were not registered, 46 (84%) which were retrospectively registered, and only 5 (9%) which were prospectively registered.
      • Rosenthal R.
      • Dwan K.
      Comparison of randomized controlled trial registry entries and content of reports in surgery journals.
      Bonnot et al.
      • Bonnot B.
      • Yavchitz A.
      • Mantz J.
      • Paugam-Burtz C.
      • Boutron I.
      Selective primary outcome reporting in high-impact journals of anaesthesia and pain.
      investigated the registration of 183 RCTs published in 2013 in 12 high-impact anaesthesia journals. These authors showed that trial registration was lacking in 70 published reports (38%), and that 60 (53%) and 53 (29%) were retrospectively and prospectively registered, respectively. Finally, Mathieu et al.
      • Mathieu S.
      • Boutron I.
      • Moher D.
      • Altman D.G.
      • Ravaud P.
      Comparison of registered and published primary outcomes in randomized controlled trials.
      evaluated 323 RCTs published in 2008 in a panel of medical specialties in high-impact journals, and reported that trial registration was lacking in 89 (28%) published reports, whereas 45 (15%) had been retrospectively registered and 147 (46%) had been prospectively registered. The fact that we did not focus on high-impact journals only probably explains the higher rate of unregistered trials in our study. Despite this difference, the proportion of prospectively registered RCTs observed in our study appears to be similar to that reported by Bonnot et al.
      • Bonnot B.
      • Yavchitz A.
      • Mantz J.
      • Paugam-Burtz C.
      • Boutron I.
      Selective primary outcome reporting in high-impact journals of anaesthesia and pain.
      and Mathieu et al.,
      • Mathieu S.
      • Boutron I.
      • Moher D.
      • Altman D.G.
      • Ravaud P.
      Comparison of registered and published primary outcomes in randomized controlled trials.
      but higher than that of surgical RCTs published in high-impact journals. Although we covered a longer and later time period than Rosenthal and Dwan,
      • Rosenthal R.
      • Dwan K.
      Comparison of randomized controlled trial registry entries and content of reports in surgery journals.
      the comparison seems valid because interventional oncology is often compared with surgery owing to similarities between the 2 disciplines, with particular difficulty in designing RCTs, and complex interventions that are difficult to evaluate.
      The 2005 ICJME initiative for trial registration is a significant step towards more transparent and accountable research. However, the timing of registration must still be improved. Indeed, only 25% of RCTs were registered prospectively. This is also in line with a previous report.
      • Chan A.-W.
      • Pello A.
      • Kitchen J.
      • Axentiev A.
      • Virtanen J.I.
      • Liu A.
      • et al.
      Association of trial registration with reporting of primary outcomes in protocols and publications.
      One possible explanation is that researchers need time to adjust to new regulations and recommendations. We chose to analyse the results of RCTs published after 2007 to take this into consideration, but integrating new habits into research protocols may take more time. The fact that prospectively registered RCTs were published more recently than retrospectively registered or unregistered studies seems to support this hypothesis. Furthermore, the involvement of institutional boards and trial funders is likely to help enforce prospective trial registration.
      • Chan A.-W.
      • Hróbjartsson A.
      • Haahr M.T.
      • Gøtzsche P.C.
      • Altman D.G.
      Empirical evidence for selective reporting of outcomes in randomized trials comparison of protocols to published articles.
      ,
      • Chan A.-W.
      • Altman D.G.
      Epidemiology and reporting of randomised trials published in PubMed journals.
      It is important to note that registration and especially prospective registration was associated with higher methodological standards (e.g. less allocation concealment, more intention-to-treat analysis, more frequent flow chart, etc.). This is probably because of the higher standards of the journals publishing the results of registered RCTs, as confirmed by their higher impact factors. Indeed, clear journal submission guidelines are associated with better adherence to ICJME standards,
      • Mathieu S.
      • Boutron I.
      • Moher D.
      • Altman D.G.
      • Ravaud P.
      Comparison of registered and published primary outcomes in randomized controlled trials.
      which are mainly endorsed by journals with a high impact factor.
      • Fleming P.S.
      • Koletsi D.
      • Dwan K.
      • Pandis N.
      Outcome discrepancies and selective reporting: impacting the leading journals?.
      Altogether, trial registration and publication standards create a virtuous circle that is beneficial to the scientific community and, more importantly, to patients. Nevertheless, the positive effect of trial registration on the quality of research reporting cannot prevent possible selective reporting of outcomes.
      Publications have shown that discrepancies between registries and publications are frequent.
      • Chan A.-W.
      • Pello A.
      • Kitchen J.
      • Axentiev A.
      • Virtanen J.I.
      • Liu A.
      • et al.
      Association of trial registration with reporting of primary outcomes in protocols and publications.
      ,
      • Dechartres A.
      • Ravaud P.
      • Atal I.
      • Riveros C.
      • Boutron I.
      Association between trial registration and treatment effect estimates: a meta-epidemiological study.
      ,
      • Vera-Badillo F.E.
      • Napoleone M.
      • Krzyzanowska M.K.
      • Alibhai S.M.
      • Chan A.-W.
      • Ocana A.
      • et al.
      Bias in reporting of randomised clinical trials in oncology.
      A systematic review comparing registered and published outcomes in RCTs, including 27 studies from a panel of medical and surgical specialties, reported a median proportion of discrepancy of 31%, which is similar to our results.
      • Jones C.W.
      • Keil L.G.
      • Holland W.C.
      • Caughey M.C.
      • Platts-Mills T.F.
      Comparison of registered and published outcomes in randomized controlled trials: a systematic review.
      However, we may have underestimated the selective reporting of outcomes associated with statistical significance because we could only access registered protocols.
      • Chan A.-W.
      • Altman D.G.
      Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors.
      Indeed, the extent of selective reporting in the results of unregistered RCTs remains unknown. Of course, the decision to omit outcomes from publications is not necessarily a sign of dishonest practices. It may be related to a combination of space constraints, a lack of clinical importance, and statistical results.
      • Chan A.-W.
      • Pello A.
      • Kitchen J.
      • Axentiev A.
      • Virtanen J.I.
      • Liu A.
      • et al.
      Association of trial registration with reporting of primary outcomes in protocols and publications.
      ,
      • Chen T.
      • Li C.
      • Qin R.
      • Wang Y.
      • Yu D.
      • Dodd J.
      • et al.
      Comparison of clinical trial changes in primary outcome and reported intervention effect size between trial registration and publication.
      Nevertheless, selective reporting may result in a bias of favourably reporting efficacy which could cause physicians to perform interventions with no evidence of benefit. Similarly, under-reporting toxicity could result in patients being exposed to treatments that may not be optimal, even with proven benefit.
      Interventional radiologists, like other physicians, are generally not well-trained to identify outcome reporting bias. However, they should be aware that study reports may be biased, despite ICJME guidelines. One might argue that it is the role of publishers and peer reviewers to detect these types of bias to reduce their frequency in published results.
      • Mathieu S.
      • Chan A.-W.
      • Ravaud P.
      Use of trial register information during the peer review process.
      Unfortunately, our results suggest that they do not sufficiently take advantage of the benefits of the transparency provided by registration. Indeed, we found some evidence of selective outcome reporting in one-third of published results. Therefore, as the consistency of outcomes between the registered protocol and published reports are available in online registries, readers should make a habit of consulting this information when a study can have a significant impact on clinical practices. Moreover, we believe that tracking registration status and selective outcome reporting are fundamental steps in the conduct of a meta-analysis as well as in the development of recommendations, as their omission could impact subsequent conclusions and statements.
      Our study has limitations. First, it only focused on a specific treatment (TACE for treating HCC). Nevertheless, TACE remains the most frequent first line treatment in patients with HCC.
      • Park J.W.
      • Chen M.
      • Colombo M.
      • Roberts L.R.
      • Schwartz M.
      • Chen P.J.
      • et al.
      Global patterns of hepatocellular carcinoma management from diagnosis to death: the BRIDGE Study.
      We did not include RCTs with combinations of TACE with other treatments because it would have been methodologically difficult to analyse the heterogenous data. Also, we did not contact the authors of unregistered RCTs to identify unreported POs and determine whether they did not reach statistical significance. Finally, we did not address the issue of selective reporting of non-primary outcomes, which may be more common. The strength of this study is that it is the first to show that under-registration and selective outcome reporting are common in one of the most extensively studied and performed interventional oncology procedures.

      Conclusion

      Trial registration improves the transparency of research and is a safeguard against reporting bias or other deviations from the planned study. The registration and timing of registration in RCTs on TACE for HCC must still be improved, and outcome reporting is too often inadequate in the results of registered trials. The registration of RCTs in interventional oncology of the liver should be reinforced.

      Financial support

      JG received a PhD grant (DOC20180507331) from the Fondation ARC pour la Recherche sur le Cancer.

      Authors' contributions

      Study design: JG, MR, IB.
      Data collection: JG, PC.
      Data analysis: JG, PC, MR, IB.
      Manuscript drafting: JG.
      Manuscript revision: all authors.
      Manuscript final approval: all authors.

      Data availability

      The authors declare that data supporting the findings of this study are available within the article and its Supplementary material.

      Conflict of interest

      The authors declare no conflict of interest than pertain to this work.
      Please refer to the accompanying ICMJE disclosure forms for further details.

      Supplementary data

      References

        • Ferlay J.E.M.
        • Lam F.
        • Colombet M.
        • Mery L.
        • Piñeros M.
        • Znaor A.
        • et al.
        Global Cancer Observatory: Cancer today.
        International Agency for Research on Cancer, Lyon, France2018 (Available at:)
        https://gco.iarc.fr/today
        Date accessed: September 2, 2019
        • Llovet J.M.
        • Real M.I.
        • Montana X.
        • Planas R.
        • Coll S.
        • Aponte J.
        • et al.
        Arterial embolisation or chemoembolisation versus symptomatic treatment in patients with unresectable hepatocellular carcinoma: a randomised controlled trial.
        Lancet. 2002; 359: 1734-1739
        • Lo C.M.
        • Ngan H.
        • Tso W.K.
        • Liu C.L.
        • Lam C.M.
        • Poon R.T.P.
        • et al.
        Randomized controlled trial of transarterial lipiodol chemoembolization for unresectable hepatocellular carcinoma.
        Hepatology. 2002; 35: 1164-1171
        • European Association for the Study of the Liver
        EASL clinical practice guidelines: management of hepatocellular carcinoma.
        J Hepatol. 2018; 69: 182-236
        • Heimbach J.K.
        • Kulik L.M.
        • Finn R.S.
        • Sirlin C.B.
        • Abecassis M.M.
        • Roberts L.R.
        • et al.
        AASLD guidelines for the treatment of hepatocellular carcinoma.
        Hepatology. 2018; 67: 358-380
        • DeAngelis C.D.
        • Drazen J.M.
        • Frizelle F.A.
        • Haug C.
        • Hoey J.
        • Horton R.
        • et al.
        Clinical trial registration: a statement from the international committee of medical journal Editors.
        JAMA. 2004; 292: 1363-1364
        • Schulz K.F.
        • Altman D.G.
        • Moher D.
        • CONSORT Group
        CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials.
        BMJ. 2010; 340: c332
        • Zarin D.A.
        • Tse T.
        • Williams R.J.
        • Rajakannan T.
        Update on trial registration 11 years after the ICMJE policy was established.
        N Engl J Med. 2017; 376: 383-391
        • Tannock I.F.
        False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons.
        J Natl Cancer Inst. 1996; 88: 206-207
        • Chan A.-W.
        • Hróbjartsson A.
        • Haahr M.T.
        • Gøtzsche P.C.
        • Altman D.G.
        Empirical evidence for selective reporting of outcomes in randomized trials comparison of protocols to published articles.
        JAMA. 2004; 291: 2457-2465
        • Llovet J.M.
        • Bruix J.
        Systematic review of randomized trials for unresectable hepatocellular carcinoma: chemoembolization improves survival.
        Hepatology. 2003; 37: 429-442
        • Reidy D.L.
        • Schwartz J.D.
        Therapy for unresectable hepatocellular carcinoma: review of the randomized clinical trials – I: Hepatic arterial embolization and embolization-based therapies in unresectable hepatocellular carcinoma.
        Anticancer Drugs. 2004; 15: 427-437
        • Cheng X.
        • Sun P.
        • Hu Q.G.
        • Song Z.F.
        • Xiong J.
        • Zheng Q.C.
        Transarterial (chemo)embolization for curative resection of hepatocellular carcinoma: a systematic review and meta-analyses.
        J Cancer Res Clin Oncol. 2014; 140: 1159-1170
        • Chen Q.W.
        • Ying H.F.
        • Gao S.
        • Shen Y.H.
        • Meng Z.Q.
        • Chen H.
        • et al.
        Radiofrequency ablation plus chemoembolization versus radiofrequency ablation alone for hepatocellular carcinoma: a systematic review and meta-analysis.
        Clin Res Hepatol Gastroenterol. 2016; 40: 309-314
        • Facciorusso A.
        • Di Maso M.
        • Muscatiello N.
        Drug-eluting beads versus conventional chemoembolization for the treatment of unresectable hepatocellular carcinoma: a meta-analysis.
        Dig Liver Dis. 2016; 48: 571-577
        • Hyun M.H.
        • Lee Y.S.
        • Kim J.H.
        • Lee C.U.
        • Jung Y.K.
        • Seo Y.S.
        • et al.
        Hepatic resection compared to chemoembolization in intermediate- to advanced-stage hepatocellular carcinoma: a meta-analysis of high-quality studies.
        Hepatology. 2018; 68: 977-993
        • Higgins J.P.
        • Altman D.G.
        • Gotzsche P.C.
        • Juni P.
        • Moher D.
        • Oxman A.D.
        • et al.
        The Cochrane Collaboration's tool for assessing risk of bias in randomised trials.
        BMJ. 2011; 343: d5928
        • Grégory J.
        • Créquit P.
        • Vilgrain V.
        • Ronot M.
        • Boutron I.
        Results of trials assessing transarterial chemoembolization for treating hepatocellular carcinoma are critically underreported.
        Eur Radiol. 2020; 30: 5633-5640
        • Rosenthal R.
        • Dwan K.
        Comparison of randomized controlled trial registry entries and content of reports in surgery journals.
        Ann Surg. 2013; 257: 1007-1015
        • Bonnot B.
        • Yavchitz A.
        • Mantz J.
        • Paugam-Burtz C.
        • Boutron I.
        Selective primary outcome reporting in high-impact journals of anaesthesia and pain.
        Br J Anaesth. 2016; 117: 542-543
        • Mathieu S.
        • Boutron I.
        • Moher D.
        • Altman D.G.
        • Ravaud P.
        Comparison of registered and published primary outcomes in randomized controlled trials.
        JAMA. 2009; 302: 977-984
        • Chan A.-W.
        • Pello A.
        • Kitchen J.
        • Axentiev A.
        • Virtanen J.I.
        • Liu A.
        • et al.
        Association of trial registration with reporting of primary outcomes in protocols and publications.
        JAMA. 2017; 318: 1709-1711
        • Chan A.-W.
        • Altman D.G.
        Epidemiology and reporting of randomised trials published in PubMed journals.
        Lancet. 2005; 365: 1159-1162
        • Fleming P.S.
        • Koletsi D.
        • Dwan K.
        • Pandis N.
        Outcome discrepancies and selective reporting: impacting the leading journals?.
        PLoS One. 2015; 10: e0127495
        • Dechartres A.
        • Ravaud P.
        • Atal I.
        • Riveros C.
        • Boutron I.
        Association between trial registration and treatment effect estimates: a meta-epidemiological study.
        BMC Med. 2016; 14: 100
        • Vera-Badillo F.E.
        • Napoleone M.
        • Krzyzanowska M.K.
        • Alibhai S.M.
        • Chan A.-W.
        • Ocana A.
        • et al.
        Bias in reporting of randomised clinical trials in oncology.
        Eur J Cancer. 2016; 61: 29-35
        • Jones C.W.
        • Keil L.G.
        • Holland W.C.
        • Caughey M.C.
        • Platts-Mills T.F.
        Comparison of registered and published outcomes in randomized controlled trials: a systematic review.
        BMC Med. 2015; 13: 282
        • Chan A.-W.
        • Altman D.G.
        Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors.
        BMJ. 2005; 330: 753
        • Chen T.
        • Li C.
        • Qin R.
        • Wang Y.
        • Yu D.
        • Dodd J.
        • et al.
        Comparison of clinical trial changes in primary outcome and reported intervention effect size between trial registration and publication.
        JAMA Netw Open. 2019; 2: e197242
        • Mathieu S.
        • Chan A.-W.
        • Ravaud P.
        Use of trial register information during the peer review process.
        PLoS One. 2013; 8: e59910
        • Park J.W.
        • Chen M.
        • Colombo M.
        • Roberts L.R.
        • Schwartz M.
        • Chen P.J.
        • et al.
        Global patterns of hepatocellular carcinoma management from diagnosis to death: the BRIDGE Study.
        Liver Int. 2015; 35: 2155-2166