Cole Phelps Mr.¹, Jessica Hardin Ms.¹, Conner Howard Mr.¹, J. Michael Anderson Dr.², Brian Chalkin Dr.², Reece M. Anderson Mr.¹, Thuc K. Vu Mr.³, Matt Vassar PhD¹, Jake Checketts Dr.^2
1Oklahoma State University Center for Health Sciences – College of Osteopathic Medicine
²Oklahoma State University Medical Center – Department of Orthopaedic
³Kansas City University College of Osteopathic Medicine

Abstract

Background
Randomized controlled trials (RCTs) related to the management of carpal tunnel syndrome (CTS) impact the recommendations seen within clinical practice guidelines (CPGs). RCTs are often required to follow the CONSORT checklist. Since the enactment of CONSORT, the CONSORT Extension for Harms was implemented to assure harms are reported thoroughly. The extension has failed to adequately improve harms reporting among RCTs in several specialties. We evaluated harms data reporting among RCTs cited for recommendations in CTS management.

Methods
We included RCTs cited as evidence supporting recommendations in the AAOS Management of Carpal Tunnel Syndrome CPG. Screening and data extraction were performed in a blinded duplicate manner. Individual CONSORT Harms items were evaluated in each RCT. Frequencies, percentages, and 95% confidence intervals were used to summarize overall adherence. Additionally, we conducted an interrupted time-series analysis to evaluate harms reporting by comparing overall percent adherence to trials published before and after the release of the extension.

Results
Sixty-eight RCTs were included in our study. None of the included trials reported all 18 checklist items. Five RCTs (7.35%) reported at least 12 of the 18 checklist items. Forty-four RCTs (54.7%) reported less than 6 of the 18 checklist items. Twenty-five were published prior to the release of the Extension for Harms (2004) and forty-three were published after. 

Conclusions
There is insufficient reporting of harms in the RCTs supporting recommendations in the AAOS Management of Carpal Tunnel Syndrome CPG along with no advancement of harms reporting following publication of the CONSORT Extension on Harms.

Keywords: CONSORT, CONSORT Harms, Carpal Tunnel Syndrome, Randomized Control Trials, Clinical Practice Guidelines, Outcomes

Introduction
Carpal Tunnel Syndrome (CTS) is a mononeuropathic condition affecting 1-8% of the general population (1-3). This syndrome can arise from trauma and inflammation around the median nerve due to sprain, fracture, endocrine, and/or rheumatologic abnormalities (1). CTS is the most expensive upper extremity musculoskeletal injury in the United States with an estimated annual cost of $2 billion in the U.S. (4).  Given the prevalence of CTS and the substantial costs for both operative and non-operative treatment, high quality and properly reported randomized controlled trials (RCTs) are imperative for CTS management as they often drive clinical decision making and patient care. 

Randomized controlled trials provide the foundation of data for Clinical Practice Guidelines (CPGs). Clinical practice guidelines provide recommendations used to help inform clinical decision making for organizations like the American Academy of Orthopaedic Surgeons (AAOS) (5). The AAOS  has created such a CPG specifically for the treatment of CTS (6,7). These guidelines assist both patients and physicians in the treatment plan process so that both parties fully understand management options. Therefore, RCTs must report both the benefits and harms from their studies with accuracy and transparency in order to strengthen CPG recommendations and ensure public trust (8).

The Consolidated Standards for Reporting (CONSORT) checklist was developed to uphold the quality of RCT reporting (9). Currently the standard CONSORT is endorsed by over 600 core medical journals including the International Committee of Medical Journal Editors (ICMJE) and the World Association of Medical Editors (WAME) (10). In the years since the conception of the CONSORT checklist, many extensions have been added, one being the CONSORT Harms Extension (10). This extension was released in 2004 and is an 18 item checklist aimed at addressing the inconsistent reporting on adverse events (harms, complications) in treatment modality outcomes (11-14). The intent of this extension was to reduce instances of reporting on positive outcomes with minimal to no mention of harms resulting from trial interventions in RCTs. It has been demonstrated that even with the CONSORT Harms Extension Checklist, RCTs in medicine have poorly reported harms (14). For instance, a 2022 study in the Journal of Arthroplasty investigating the quality of harms-related reporting in a sample of osteoarthritis clinical trials concluded that harms-related data is infrequently quantified or reported (14). To determine if a similar trend is seen in orthopaedic studies, we evaluated the reporting of information regarding harms is seen in RCTs cited under the AAOS CPGs for CTS.

Materials and Methods
No human subjects were involved in the current cross-sectional study, therefore there was no Institutional Review Board oversight (15). The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed during this study (16). Our primary goal was to analyze the completeness of harms and adverse events reporting among RCTs cited as evidence for the AAOS Management of Carpal Tunnel Syndrome CPG. 

Identification of CPG and RCTs
The AAOS CPG was obtained from Orthoguidelines.org (17). The Management of Carpal Tunnel Syndrome CPG was then screened to identify RCTs. Trials were included if: 1) the trial met the International Committee of Medical Journal Editors’ (ICMJE) definition of an RCT (18); 2) the RCT was attainable in the English language; and 3) the RCT was cited as supporting evidence for at least one recommendation within the CPG. Of the 79 RCTs identified in the CPG, 68 (86.1%) met inclusion criteria. The 68 included RCTs were extracted from the bibliography of the CPG into a Microsoft Excel (Microsoft Office, Seattle, WA, USA) spreadsheet for analysis. 

Data Collection
Two authors extracted specific characteristics from the 68 included RCTs in a blind and duplicate fashion using a pilot-tested Google form, similar to previous studies (19-21). Information extracted from each RCT included: fulfillment of the ICMJE definition of an RCT, RCT title, PMID, year of publication, journal RCT was published in, RCT funding source, country RCT was conducted in, sample, blinding, number of participating centers, and mention of CONSORT adherence. In addition to the RCT characteristics described, the same two authors also assessed each trials’ adherence to the CONSORT Extension for Harms checklist. Each of the 18 checklist items received a “yes” (1 point) or “no” (0 points) according to its adherence to the specific item’s criteria. Following data collection, authors’ responses were compared with one another and differences were reconciled, with senior authors available, if needed. Supplemental Table 1 provides details for each of the CONSORT Extension for Harms items, along with criteria which guided individual responses. 

Statistical Analysis
Descriptive statistics, including frequencies, percentages, and 95% confidence intervals, were calculated to summarize CONSORT Harms adherence. Individual checklist items were evaluated to determine levels of reporting and are represented using a histogram. An interrupted time-series analysis was conducted to determine the influence of the CONSORT Extension for Harms on adverse event reporting. Specifically, the CONSORT Harms checklist was scored as a percentage of completion for each individual RCT, similar to previous studies that have analyzed the effect of guidelines on the completeness of reporting (22,23). This analysis requires one data point per period, which led us to calculate the mean adherence of all RCTs published in the same month and year. The Newey-West method and Stata’s ITSA command was used to estimate standard errors (24).

In order to assess whether the publication of the Harms extension for CONSORT influenced the completeness of reporting, we conducted an interrupted time series analysis (ITSA). We scored adherence to CONSORT-Harms items as a percentage of completeness according to the checklist. The ITSA required only one datapoint per time period and we calculated the average completeness percentage for months with multiple published trials. We used Stata 17.0 to conduct a pre- and post-trend analysis for completeness of reporting according to CONSORT-Harms checklist criteria; Stata’s “.itsa” command and the Newey-West method were used to estimate standard errors (25).

Results
A total of 79 references were identified in the bibliography of the AAOS Management of Carpal Tunnel Syndrome CPG. Of the 79 references, eight RCTs did not underpin a recommendation and three were inaccessible. Therefore, 68 RCTs were identified and fit inclusion criteria for this study (Figure 1). Of the 68 included trials, 25 were published prior to the release of the CONSORT Extension for Harms (1989-2004) and 43 were published after its release (2005-2016). 

Sample Characteristics
Table 1 includes the sample characteristics for the 68 included RCTs. The majority of included trials were conducted at a single center (41 of 68; 60.3%). No funding statement was provided for 42 of the 68 trials (61.8%) and no funding was received for 14 trials (20.6%). Among studies that were funded, and reported funding appropriately, private funding was the most common (6 of 68; 8.82%). Blinding methods varied among the included RCTs. A single blind method was used by 19 RCTs (27.9%), a double blind method was used by 21 RCTs (30.9%), and 16 RCTs (23.5%) were unblinded. Sample s varied among the trials, with a total of 5,609 study participants across the 68 included RCTs.

Adherence to CONSORT Extension for Harms 
The 68 included RCTs appropriately reported 4.56 of the 18 CONSORT Extension for Harms checklist items (25.3%). None of the included trials reported all 18 checklist items and 15 reported zero of the 18 items. Five RCTs (7.35%) reported at least 12 of the 18 checklist items (i.e., ⅔ of items). Forty-four RCTs (54.7%) of RCTs reported less than 6 of the 18 checklist items (i.e., ⅓ of items). Only item 7a (denominators for AEs) had at least 50% adherence among the 68 included trials. Items with less than 20% adherence included items 3a (lists or defines AEs), 3b (distinguishes between expected and unexpected AEs), 3c (measures AE severity), 4c (attribution methods or ascertainment), 4d (harms monitoring and stopping), 5 (analysis of harms), 7b (intention to treat analysis), and 9 (subgroup or exploratory analyses on harms) (Table 2, Figure 2). 

Interrupted Time Series Analysis
Of the 68 included RCTs cited under recommendations in the AAOS Management of Carpal Tunnel Syndrome CPG, 25 were published prior to the release of the CONSORT Extension for Harms (2004) and 43 were published after its release (2005-2014). Adherence to checklist items 3c, 4c, 5, 6a, 6b, 7a, and 7b improved by at least 10% after the release of the CONSORT Extension for Harms. However, adherence to items 1, 2, and 8a, regressed by at least 10%. Trials published from 1990-2004 adequately reported on average 4.40 of 18 checklist items (24.4%; SD = 4.2 items) and trials published after 2004 adequately reported 4.65 of 18 items (25.8%; SD = 3.8 items) (Table 2, Figure 3). The interrupted time series analysis, shown in Figure 3, demonstrates no significant difference in the completeness of Harms reporting in RCTs cited in the AAOS guidelines for management of Carpal Tunnel syndrome since the publication of the CONSORT-Harms checklist in 2004 (p = 0.7604; 95% CI: -0.94 – 2.63).

Discussion
The purpose of this study was to examine harms reporting using the CONSORT Extension for Harms among RCTs cited as supporting evidence for recommendations in the AAOS Management of Carpal Tunnel Syndrome CPG. Of the 68 included RCTs, overall percent adherence to the 18 CONSORT Harms items was 25.3%, suggesting substantial underreporting of adverse events. This finding (e.g., inadequate harms reporting among CTS RCTs) is not an isolated occurrence, as similar studies frequently exhibit poor overall adherence, as well (11,21,26-30). A recent study investigating harms reporting among RCTs underpinning recommendations in three osteoarthritis CPGs found that an average of 44.7% of CONSORT Harms checklist items were adequately reported among the included trials. (20) Among trials cited as supporting evidence for recommendations in a CPG on rotator cuff injuries, a 31.7% adherence to CONSORT Harms items was observed. (19) In our study, the most frequently omitted CONSORT Harms checklist items were: (1) item 3b – distinguishing between expected and unexpected adverse events, (2) item 4d – harms-associated monitoring and stopping, and (3) item 5 –  analysis of harms data. Similarly, these checklist items (e.g., 3b, 4d, 5) were among the most frequently omitted in RCTs underpinning clinical recommendations in OA and rotator cuff injury CPGs, as well (19,20).

The intent of publishing the CONSORT Extension for Harms in 2004 was to alleviate the issue of RCTs underreporting harms, yet the interrupted time-series analysis in our study demonstrated no statistically significant increase in harms reporting over time. The 43 RCTs published after the CONSORT Extension for Harms (2005-2016) demonstrated an average adherence rate of 25.8%, while trials published before 2005 would have had an average adherence rate of 24.4% (if the CONSORT Harms Extension was available). This slight increase in harms reporting over time suggests the mere existence of the CONSORT Harms Extension is insufficient to assure adequate reporting. Furthermore, evidence suggests journals may not require authors to abide by the CONSORT Extension for Harms to the same extent that the standard CONSORT guidelines are followed. In a review of 116 health journals, 41 provided online instructions to authors (31). Nearly half (46%) of the journals that had online instructions mentioned the standard CONSORT guideline, yet none of them mentioned the CONSORT Extension for Harms (31). The lack of journal requirements to follow the CONSORT Harms checklist may be contributing to the low adherence rate found throughout literature, potentially due to poor communication between journals and researchers. The frequent omission of the CONSORT Extension for Harms within journals’ submission requirements and the poor quality of harms reporting among RCTs creates an opportunity for misjudgement and misinterpretation of interventions’ safety profiles. Consequently, poor data representation and incomplete reporting hinders proper evaluations of interventions and potentially alters clinical-decision making. In an attempt to alleviate the common finding of poor harms reporting among RCTs, we offer recommendations to improve this issue. 

First, we recommend that journals and editorial boards empha the CONSORT Extension for Harms in their submission guidelines. As an alternative, it may be more feasible for the CONSORT authors to incorporate the Harms Extension into the main CONSORT guideline, along with any other CONSORT extensions. As a result, it would be expected for harm-related data reporting to improve (32). An active role of the CONSORT group to advertise their checklist may also lead to more optimal levels of harms reporting. Additionally, it has been demonstrated that RCTs funded by pharmaceutical companies have higher harms reporting (13,14,27,33,34). Therefore, funding may also be a target to ensure recipients are upholding high quality reporting of all trial outcomes. To avoid complete withdrawal of funding from RCTs, implementing funding tiers based on the quality of reporting may be more practical. For example, full funding could be provided to trials with at least 75% adherence to the main CONSORT guidelines and the CONSORT Extension for Harms, with a step-wise decrease in the funding amount rewarded as adherence decreases. Finally, we suggest that RCTs define what they consider for an adverse event and report “no adverse events” if nothing significant was found according to their definition. It is our hope that these actions will improve the quality of orthopaedic surgery RCTs, subsequent CPGs, and most importantly, improve the transparency and completeness of reporting related to CTS management for the safety of patient care. Since RCTs are crucial to CPGs, it is imperative that adverse events are adequately reported, so that clinicians and patients are then best equipped to carry out a proper risk and benefit analysis. 

Funding
No funding to report.

Figure 1 | Figure 2 | Figure 3 | Supplemental Table 1 | Table 1 | Table 2

Figures Legend
Figure 1. Flow diagram of inclusion criteria
Figure 2. Percentage of Included RCTs Adequately Reporting Each CONSORT Harms Recommendation.
Figure 3. Interval Time Series Analysis Series Regression before and after the implementation of the CONSORT Extension on Harms.

Required Disclosures and Declaration

Copyright Information: No copyright information added
IRB Approval Information: Not applicable
Disclosure Information: No known conflicts of interest

References

1. (ninds) NI of NDAS, National Institute of Neurological Disorders and Stroke (NINDS). Carpal Tunnel Syndrome. Definitions. Published online 2019. doi:10.32388/640746
2. Atroshi I, Gummesson C, Johnsson R, Ornstein E, Ranstam J, Rosén I. Prevalence of carpal tunnel syndrome in a general population. JAMA. 1999;282(2):153-158.
3. Dale, Ann Marie et al. “Prevalence and incidence of carpal tunnel syndrome in US working populations: pooled analysis of six prospective studies.” Scandinavian journal of work, environment & health vol. 39,5 (2013): 495-505. doi:10.5271/sjweh.3351
4. Stapleton MJ. Occupation and carpal tunnel syndrome. ANZ J Surg. 2006;76(6):494-496.
5. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E. Summary. National Academies Press (US); 2011. 
6. Classic Home. AAOS AUC. Accessed March 26, 2022. https://www.orthoguidelines.org/topic?id=1020&tab=all_guidelines.
7. American Academy of Orthopaedic Surgeon Management of Carpal Tunnel Syndrome Evidence-Based Clinical Practice Guideline. https://www.aaos.org/globalassets/quality-and-practice-resources/carpal-tunnel/cts-cpg_4-25-19.pdf
8. Akobeng AK. Understanding randomised controlled trials. Arch Dis Child. 2005;90(8):840-844.
9. Moher D, Schulz KF, Altman D, CONSORT Group. The CONSORT Statement: revised recommendations for improving the quality of reports of parallel-group randomized trials 2001. Explore. 2005;1(1):40-45.
10. Harms. Accessed March 26, 2022. http://www.consort-statement.org/extensions/overview/harms
11. Breau RH, Gaboury I, Scales CD Jr, Fesperman SF, Watterson JD, Dahm P. Reporting of harm in randomized controlled trials published in the urological literature. J Urol. 2010;183(5):1693-1697.
12. Cuervo LG, Clarke M. Balancing benefits and harms in health care. BMJ. 2003;327(7406):65-66.
13. Haddad C, Sigha OB, Lebrun-Vignes B, Chosidow O, Fardet L. Reporting of harm and safety results in randomized controlled trials published in 5 dermatology journals. J Am Acad Dermatol. 2017;77(1):98-104.e1.
14. Haidich AB, Birtsou C, Dardavessis T, Tirodimos I, Arvanitidou M. The quality of safety reporting in trials is still suboptimal: survey of major general medical journals. J Clin Epidemiol. 2011;64(2):124-135. 
15. Electronic Code of Federal Regulations- US Department of Health and Human Services’ Code of Federal Regulation 45 CFR 46.102(d). https://www.ecfr.gov/cgi-bin/retrieveECFR?gp=&SID=83cd09e1c0f5c6937cd9d7513160fc3f&pitd=20180719&n=pt45.1.46&r=PART&ty=HTML#se45.1.46_1102 in effect July 19, 2018
16. Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med. 2017;22(4):139-142.
17. OrthoGuidelines. OrthoGuidelines. Accessed December 30, 2019. http://www.orthoguidelines.org/guidelines
18. ICMJE. Accessed June 3, 2021. http://icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html
19. Anderson JM, Howard C, Hardin J, et al. Harms-Related Data are Poorly Reported Among Randomized Controlled Trials Underpinning the AAOS Clinical Practice Guideline Recommendations for Rotator Cuff Injuries. J Shoulder Elbow Surg. Published online August 9, 2022. doi:10.1016/j.jse.2022.06.021
20. Anderson JM, Howard C, Staggs J, et al. Inadequate Reporting of Complications in Randomized Controlled Trials Cited as Supporting Evidence Underpinning AAOS CPG Recommendations for Hip and Knee Osteoarthritis: Application of the CONSORT Harms Checklist. J Arthroplasty. Published online June 13, 2022. doi:10.1016/j.arth.2022.06.004
21. Xu ZY, Azuara-Blanco A, Kadonosono K, et al. Reporting of Complications in Retinal Detachment Surgical Trials: A Systematic Review Using the CONSORT Extension for Harms. JAMA Ophthalmol. Published online June 17, 2021. doi:10.1001/jamaophthalmol.2021.1836 
22. Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ. 2012;344:e4178.
23. Anderson JM, Stafford A, Jellison S, Vassar M. Intervention Reporting of Published Trials Is Insufficient in Orthopaedic Surgery Journals: Application of the Template for Intervention Description and Replication Checklist. Sports Med Arthrosc Rehabil Ther Technol. 2021;3(3):e619-e627.
24. Linden A. Conducting Interrupted Time-series Analysis for Single- and Multiple-group Comparisons. The Stata Journal: Promoting communications on statistics and Stata. 2015;15(2):480-500. doi:10.1177/1536867×1501500208 
25. Linden A. Conducting Interrupted Time-series Analysis for Single- and Multiple-group Comparisons. Stata J. 2015;15(2):480-500.
26. Shukralla AA, Tudur-Smith C, Powell GA, Williamson PR, Marson AG. Reporting of adverse events in randomised controlled trials of antiepileptic drugs using the CONSORT criteria for reporting harms. Epilepsy Res. 2011;97(1-2):20-29.
27. Maggi CB, Griebeler IH, Dal Pizzol T da S. Information on adverse events in randomised clinical trials assessing drug interventions published in four medical journals with high impact factors. Int J Risk Saf Med. 2014;26(1):9-22.
28. Hadi MA, McHugh GA, Conaghan PG. Quality of reporting of harms in randomised controlled trials of pharmacological interventions for rheumatoid arthritis: a systematic review. Evid Based Med. 2017;22(5):170-177.
29. Hum SW, Golder S, Shaikh N. Inadequate harms reporting in randomized control trials of antibiotics for pediatric acute otitis media: a systematic review. Drug Saf. 2018;41(10):933-938.
30. Stubenrouch FE, Cohen ES, Bossuyt PMM, Koelemay MJW, van der Vet PCR, Ubbink DT. Systematic review of reporting benefits and harms of surgical interventions in randomized clinical trials. BJS Open. 2020;4(2):171-181. 
31. Hirst A, Altman DG. Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PLoS One. 2012;7(4):e35621. 
32. Turner L, Shamseer L, Altman DG, Schulz KF, Moher D. Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev. 2012;1:60.
33. Péron J, Maillet D, Gan HK, Chen EX, You B. Adherence to CONSORT adverse event reporting guidelines in randomized clinical trials evaluating systemic cancer therapy: a systematic review. J Clin Oncol. 2013;31(31):3957-3963.
34. Williams MR, McKeown A, Pressman Z, et al. Adverse Event Reporting in Clinical Trials of Intravenous and Invasive Pain Treatments: An ACTTION Systematic Review. J Pain. 2016;17(11):1137-1149.