Authors

1. Joseph Patrick, Medical Student – Heritage College of Osteopathic Medicine – Ohio University
2. Brian Handal, Orthopaedic Surgery Resident – Department of Orthopaedic Surgery – Kettering Health
3. Julieanne Sees, Pediatric Orthopaedic Surgeon – Department of Orthopaedic Surgery, Nemours/Alfred I. duPont Hospital for Children
4. Adam Dann, Orthopaedic Hand Surgeon – Department of Orthopaedic Surgery – Kettering Health
5. Safet Hatic, Orthopaedic Foot & Ankle Surgeon – Department of Orthopaedic Surgery – Kettering Health

Abstract

OBJECTIVES: Research expectations in orthopedic surgery residency programs vary greatly from program to program. The purpose of our study was to investigate whether the availability of research funding to orthopaedic surgery residents affects publication productivity.

METHODS: This cross-sectional study was conducted using telephone and email surveys with anonymous participants including program directors and coordinators of Accreditation Council for Graduate Medical Education (ACGME)-accredited orthopedic surgery residency training programs.

RESULTS: Several factors impact the quantity and quality of research that orthopedic surgery residents investigate. These factors may include funding, program standards, graduation requirements, protected research time, faculty involvement, and many other variables.

CONCLUSION: 69 individual programs completed the survey, and our findings indicate that resident research productivity is associated with higher annual research budgets and dedicated research assistants.

Keywords: Orthopedics, Residency, ACGME, Funding, Research, Publication

Introduction
As more journals adopt a pay to play model, the publication space available for orthopaedic residents is becoming increasingly difficult to afford. Many journals offer tiered costs based on one’s training level or type of research one submits. The costs typically cover website maintenance, reviewer costs, hardcopy publications, Pubmed listing, advertising, and several others. Publication productivity has been correlated to many factors in residency-based programs^1-8; however, this is the first research sought to investigate orthopaedic residency programs accessibility to funding to subsidize costs.

The benefits of residents conducting research are copious for both residency programs and residents. Scholarly activity is a key tenet of the ACGME for all accredited orthopaedic residencies, and as Robbins et. al. noted, ACGME adequacies including medical knowledge, practice-based learning and improvement, and professionalism are met through increased resident research involvement.⁷ Furthermore, productive resident research programs boost the reputation of residency programs⁹ and increase grant funding⁷ fostering residency program environments that will attract more competitive applicants and provide more opportunities for residents’ graduate medical education (GME). Additionally, residents who have completed research projects have reported higher satisfaction rates during residency¹⁰ and has helped many residents earn competitive fellowship positions.⁹ In this study we investigate the barriers to funding for orthopaedic residency programs as well as the value and opportunities accessible research funding provides for orthopaedic surgery residents.

Methods
A list of all ACGME-accredited orthopaedic surgery residency training programs was compiled. We then visited all the orthopaedic residency program websites and searched for contact information for program coordinators and program directors. The contact information collected included names, email addresses and telephone numbers. Additional information was gathered from program websites including the number of residents (program ), training length, optional research tracks, type of program (majority allopathic or osteopathic residents), city, and state.

Table 1 lists the questions which were included in our survey. The 8-question survey was conducted over the telephone and by email with ACGME-accredited orthopaedic surgery residency training program affiliates. During the data collection period, programs were first contacted by the phone numbers listed on the respective program websites. When phone numbers were not available on these websites or when the program representatives could not be reached, the programs were contacted via email instead. The survey remained open for 6 months total from July to December 2021.

Results
Of the 203 surveys sent, 75 surveys were completed where 69 individual programs participated, resulting in a response rate of about 34%. It is notable that 4 programs answered the survey more than once and only their most recent response was included in data analysis. Of the participating programs, 17 were primarily osteopathic while 52 were primarily allopathic (determined by majority of residents’ medical education within each program).  

The estimated average annual research budget was answered by 42 of the 69 programs. The study showed that 29% of programs’ research budgets were between $0 – $999, 17% of budgets fell between $1000 – $9,999, 36% were between $10,000 – $99,999, and 19% of budgets were greater than or equal to $100,000 (Chart 1). The responses for average annual research budgets ranged from $0 to $5,000,000. Of the respondents that noted an annual per resident budget (n=5), an average of $1800 was found. Similarly, of the respondents that noted poster presentation and travel fee budgets (n=5), an average of $1840 was recorded.

Of the 56 programs that responded to the estimated number of publications in the last year, 27 programs indicated 10 or less publications, 23 programs indicated between 11 – 25 publications, 2 programs noted between 26 – 50 publications, and 4 programs noted greater than 50 publications (Chart 2). The number of publications per program over the previous year ranged from a minimum of 0 publications to a maximum of 145 publications (Chart 3). Of the 55 programs that addressed the estimated number of publications over the last 5 years, 3 programs noted 10 or less publications, 13 programs indicated 11 – 25 publications, 19 programs noted 26 – 50 publications, and 20 programs indicated greater than 50 publications (Chart 2). The number of publications per program over the last 5 years ranged from a minimum of an estimated 17.5 publications to a maximum of 598 publications (Chart 3). Chart 4 displays data which accounts for the number of residents per program and displays the estimated number of publications per resident published for the past year and the last 5 years. The average number of publications per resident for the past year was 0.69655 publications (invalid responses excluded). The average number of publications for residents for the last 5 years was 3.18756 publications (invalid responses excluded).  

The current study found that 41 of 65 programs indicated that 75-100% of their residents’ publications are peer reviewed or Pubmed indexed (Chart 5). This is in comparison to 15 of 65 programs which noted <75-50% of their publications are peer reviewed or Pubmed indexed (Chart 5). 4 of 65 programs estimated between <50-25% while 5 of 65 programs estimated <25% of their publications are peer reviewed or Pubmed indexed (Chart 5).

Journals where the participating programs publish in are shown in Chart 6. The top three most common journals in which these 69 participating programs typically publish are JBJS, Arthroscopy, and JAAOS respectively. Numerous other journals were listed additionally by programs such as JJSE, IJSCR, AJSCR, Pediatric Emergency Care, Neurosurgery, JAMA, Global Spine, Journal of Wrist Surgery, Journal of Spine Surgery, Journal of Clinical Neuroscience, Journal of Knee Surgery, POSNA, Scoliosis, and several others.

40 of 68 programs indicated their department had a dedicated research assistant as opposed to 28 of 68 programs which indicated their department did not have a dedicated research assistant. In regards to dedicated research time for residents, 55 of 69 programs indicated their orthopaedic surgery residents did have protected research time, while 14 of 69 programs noted no such dedicated time. Of the programs which indicated yes to dedicated time, dedicated research varied greatly between programs in terms of when the time is primarily divided in residents’ training and the amount of time provided to residents in general. 4 of the 69 programs indicated offering an optional 1-year dedicated research year to selected residents. Many programs noted maintaining compliance with ACGME requirements for research while several other programs offered greater amounts of dedicated research time and commitment. Notably, many programs reported protected research time of 2-3 months during PGY3, while many other programs offered continuous research time in smaller increments throughout the entire duration of their residents’ training.

Discussion
This study revealed that annual research budgets vary greatly between orthopaedic residency programs around the United States. The majority of programs (36%) estimated their budgets to be between the $10,000 – $99,999 range. This is a drastic comparison to 29% of programs which fall within the $0 – $999 budget range. Furthermore, 19% of programs budgets were estimated to be above $100,000, including a handful of programs’ budgets reaching greater than $1,000,000. This vast difference in budgets tend to reflect the research productivity of programs seen in this study. For example, of the five most productive programs as seen in Charts 3 and 4 (anonymous programs denoted as~ 14, 24, 25, 54, and 58) in terms of the number research publications per resident at both the 1 year and 5 year intervals, two of these program’s budgets were greater than $100,000, two budgets were within the $10,000 – $99,999 range, and one program did not provide a clear budget. Between these top four productive programs, which did provide budgets, they had an average of 28 residents. This compares to the 21.058 residents per program average of all 69 participating programs. Importantly, these five most productive programs all had a dedicated research assistant, but interestingly only three of these programs had dedicated research time for residents. Our study provides evidence that the more funding and support programs have, the more productive their residents’ research will likely be. Notably, there are select programs that are exceptionally productive with relatively average number of residents and mid-level budgets between the $10,000-$99,999 range while maintaining much higher than average publish rates per resident with 75-100% peer reviewed or Pubmed indexed works. Of course, quantity can sometimes come at a cost to quality as evidence shown by the program with the highest 5-year publication rate (639 publications) where less than 25% of their residents’ publications are estimated to be peer reviewed or Pubmed indexed.

The variables which allow for programs to support residents’ research endeavors may be difficult to determine but some may include recruiting residents who are interested in research initially, providing ample protected research time, having dedicated research staff and assistants, utilizing medical and graduate students, and having faculty mentors who are actively conducting research. It is notable that one study examined three orthopaedic residency programs with varying degrees of dedicated research time, and found that programs with dedicated research time did not produce significantly higher amounts (p > 0.05) or higher quality peer-review publications than programs with no dedicated time.¹¹ This contrasts with findings from a comprehensive study by Williams BR et al., which investigated 125 ACGME-accredited orthopaedic residency programs as to whether protected research time versus no protected research time for residents impacted research productivity.¹² The study which included 1,690 residents, found there was a significant difference (p=0.02) in the average number of publications per resident in programs with protected time (1.1 ± 1.2 publications) versus programs without protected time (0.6 ± 0.5 publications).¹² Regardless, there are likely many factors which may influence the quality and quantity of publications produced by orthopaedic residents each year.  

It is important to note that data collection occurred during the COVID-19 pandemic and some orthopaedic surgery residency program affiliates may not have been as accessible by phone due to virtual working. It is also notable that this peri-pandemic study may not reflect typical publication output from orthopaedic residents due to potentially increased numbers of cancelled surgeries and diverted clinical efforts. Response bias is potential factor that may be affecting our study’s findings as some respondents may have falsely estimated values. Also, non-response bias could be a potential factor as well due to the large percentage of programs which did not participate in the survey and therefore could have impacted the study’s data.

Conclusion
According to the 69 ACGME-accredited orthopaedic residency programs which participated in the present study, our findings suggest that resident research productivity regarding the quality and quantity of published works is associated with higher annual research budgets and a dedicated research assistant. Our findings also indicate there may a limit where quantity compromises quality in terms of peer-reviewed or pubmed indexed works.

References

1. Osborn PM, Ames SE, Turner NS, et al. An Analysis of Research Quality and Productivity at Six Academic Orthopaedic Residencies. J Surg Educ. 2018;75(6):1635-1642. doi:10.1016/j.jsurg.2018.04.022.
2. Torres D, Gugala Z, Lindsey RW. A dedicated research program increases the quantity and quality of orthopaedic resident publications. Clin Orthop Relat Res. 2015;473(4):1515-1521. doi:10.1007/s11999-014-4080-1.
3. Chang CW, Mills JC. Effects of a reward system on resident research productivity. JAMA Otolaryngol Head Neck Surg. 2013;139(12):1285-1290. doi:10.1001/jamaoto.2013.5303.
4. Scarlat MM, Mauffrey C, Mavrogenis A. Equal access to orthopaedic research funding, databases and scientific publications. Int Orthop. 2019;43(10):2205-2207. doi:10.1007/s00264-019-04413-y.
5. Stevenson MD, Smigielski EM, Naifeh MM, Abramson EL, Todd C, Li ST. Increasing Scholarly Activity Productivity During Residency: A Systematic Review. Acad Med. 2017;92(2):250-266. doi:10.1097/ACM.0000000000001169.
6. Zimmerman R, Alweis R, Short A, Wasser T, Donato A. Interventions to increase research publications in graduate medical education trainees: a systematic review. Arch Med Sci. 2019;15(1):1-11. doi:10.5114/aoms.2018.81033.
7. Robbins L, Bostrom M, Marx R, Roberts T, Sculco TP. Restructuring the orthopedic resident research curriculum to increase scholarly activity. J Grad Med Educ. 2013;5(4):646-651. doi:10.4300/JGME-D-12-00303.1.
8. Chan RK, Lockyer J, Hutchison C. Block to succeed: the Canadian orthopedic resident research experience. Can J Surg. 2009;52(3):187-195.
9. Rothberg MB, Kleppel R, Friderici JL, Hinchey K. Implementing a resident research program to overcome barriers to resident research. Acad Med. 2014;89(8):1133-1139. doi:10.1097/ACM.0000000000000281.
10. Takahashi O, Ohde S, Jacobs JL, Tokuda Y, Omata F, Fukui T. Residents’ experience of scholarly activities is associated with higher satisfaction with residency training. J Gen Intern Med. 2009;24:716–720.
11. Krueger CA, Hoffman JD, Balazs GC, Johnson AE, Potter BK, Belmont PJ Jr. Protected Resident Research Time Does Not Increase the Quantity or Quality of Residency Program Research Publications: A Comparison of 3 Orthopedic Residencies. J Surg Educ. 2017;74(2):264-270. doi:10.1016/j.jsurg.2016.08.008.
12. Williams BR, Agel JA, Van Heest AE. Protected Time for Research During Orthopaedic Residency Correlates with an Increased Number of Resident Publications. J Bone Joint Surg Am. 2017;99(13):e73. doi:10.2106/JBJS.16.00983.