Volume 2, November 2002

www.psljournal.com/archives/newsedit/douglas.cfm

 

 

What Role Should Rules, Guidelines, and Education Play in the Responsible Conduct of Research?  A National Conference Addresses the Issue

 

Jennifer Douglas-Vidas, M.A. and Marsha E. Reichman, Ph.D.*

 

*Northrop Grumman Information Technology Health Solutions and Services (formerly ROW Sciences, Inc.)

 

 

On September 23-24, 2002, the Office of Research Integrity (ORI), Department of Health and Human Services (DHHS), convened a national conference of approximately 100 individuals to discuss the usefulness of research guidelines in preventing misconduct and promoting the responsible conduct of research (RCR).  Conference participants included representatives from academia, such as professors and research integrity officers, as well as hospital administrators and members of institutional associations.  The conference was a follow-up to the report Analysis of Guidelines for the Conduct of Research Adopted by Medical Schools or Their Components, prepared by ORI contractor Northrop Grumman Information Technology (NGIT) Health Solutions and Services (formerly ROW Sciences, Inc.).  The report led to the development of a resource document for use in creating and implementing effective guidelines.  Conference participants were asked to provide feedback on the content, layout, and overall utility of this document.  

 

The meeting began with introductory remarks from Lawrence J. Rhoades, Ph.D., Director of the Division of Policy and Education at ORI.  Dr. Rhoades reviewed various arguments for organizational guideline development, citing reports from the Institute of Medicine (2002, 1989) and the National Academy of Sciences (1992).  He also discussed extant negative attitudes toward guidelines of this sort, as well as potential positive results from developing them.  Lastly, Dr. Rhoades delineated the purpose of the assigned small group breakout sessions, scheduled to occur after each of the plenary sessions.  The small groups were given the task of developing issues and questions relating to each session topic to be summarized in a series of five-minute presentations. 

 

Next, Jennifer Douglas-Vidas, MA, Research Analyst and Project Manager at NGIT Health Solutions and Services, presented results from the 2001 ORI-commissioned study Analysis of Guidelines for the Conduct of Research Adopted by Medical Schools or Their Components.  This study examined the number of accredited U.S. medical schools that had guidelines relating to the conduct of research, the organizational level at which the guidelines were developed, and the topics addressed and specific behaviors recommended by the guidelines.  The analysis revealed that although an increasing number of medical schools have some sort of research guidelines (e.g., 78% in 2000 compared to 13% in 1990), most focused on a narrow range of topics, such as legal issues like intellectual property, and did not provide a comprehensive, well-rounded perspective on the different aspects of responsible research conduct.  In addition, guidelines were frequently found in several different documents rather than in a centralized location. 

 

Marsha Reichman, Ph.D., Director of Epidemiology and Survey Research at NGIT Health Solutions and Services, led a discussion relating to the resource document, Creating Effective Research Guidelines, that was based, in part, on the results from the 2001 study.  The document contains specific examples selected from the content analysis of the medical school guidelines, and ORI intends this document to be used as a resource tool by any institution interested in developing or updating research guidelines – universities (including but not limited to medical schools), professional societies, and institutional associations.  Conference participants offered specific suggestions on document layout, terminology, and potential topics to add.  The importance of explicitly delineating the difference between policy, guidelines, and rules was emphasized, as was the desirability of clarifying that the purpose of the document is to provide assistance and guidance, not to dictate specific behavior.

 

The first plenary session addressed the issue of guideline content.  Speakers were Margaret L. Dale, JD, Associate Dean for Faculty Affairs and Director of the Office for Research Issues at Harvard Medical School, and Francis L. Macrina, Ph.D., Edward Myers Professor and Director of the Philips Institute at Virginia Commonwealth University.  Ms. Dale discussed specific topics that guidelines should address, and differentiated between topics that stem from regulatory requirements (e.g., conflict of interest, scientific misconduct), topics of emerging interest (e.g., data sharing), and topics of concern institutionally (e.g., authorship).  She emphasized that “every guideline tells a story,” and that guidelines are not created in a vacuum; they arise from the realization that changing circumstances require codification of agreed-upon standards in an increasingly complex environment. 

 

Francis Macrina provided a historical perspective on the development of research guidelines, and expanded on the point that guideline development typically has some sort of “trigger.”  For instance, atrocities such as the 1932 study of untreated syphilis and 1940s Nazi experimentation resulted in codes regulating the use of human subjects in research such as the 1947 Nuremberg Code and the 1964 Declaration of Helsinki.  More recently, the increase in highly publicized cases of scientific misconduct in the 1970s and 80s helped lead to Health Research Extension Act Section 493: U.S. DHHS Public Health Service (PHS) awardees must review reports of scientific fraud and inform HHS.  Dr. Macrina traced the development of RCR guideline topics, noting that guidelines on issues such as data management, publication practices, authorship, peer review and mentoring were typically developed in the 1980s; guidelines involving conflict of interest, intellectual property, human subjects, animal subjects, and research misconduct were developed in the 1990s; and by 2002, guidelines were developed on the role of principal investigator and laboratory safety.  One challenge to guideline development, he pointed out, is the diversity of disciplines and cultures at academic institutions, which can complicate the creation of universally applicable – and useful – guidelines. 

 

The second plenary session focused on the issue of utility of guidelines or rules.  Speaking were Linda Wilcox, M.A., Ombudsperson of the Harvard Medical Area, and Anna Mastroianni, J.D., MPH, of the University of Washington School of Law and Institute for Public Health Genetics.  Ms. Wilcox traced the development and implementation of authorship guidelines at Harvard, and noted that the guidelines resulted in more awareness, more willingness to discuss concerns with supervisors, and more people comparing their views and decisions with those of the guidelines.  In her view, guidelines promote the use of objective measures for decision-making, and encourage people to discuss authorship in advance, thus averting disastrous misunderstandings later.  The advantage of “guidelines,” as opposed to set-in-stone “rules,” is the fact that they promote discussion, and can be adapted to individual situations.  As a result, fewer disputes reach a “boiling point,” and there are fewer personal agendas and power plays in decision-making.  Ongoing challenges include the fact that many people continue to ignore the guidelines until a crisis arises, and that while students and postdocs are made aware of the guidelines through ethics classes, programs for faculty are not required. 

 

Anna Mastroianni then discussed Federal RCR training mandates, and the fact that NIH-funded Institutional National Research Award (T32) research grantees are frequently the only ones trained in RCR.  Typically, training includes segments on authorship, human/animal subjects, data management, misconduct, and conflict of interest.  Ms. Mastroianni reiterated that having some sort of crisis or scandal is the primary way to get attention focused on research guidelines, and that the “stick” approach is often more effective than the “carrot.”

 

The third plenary session focused on the issue of educating staff about research guidelines.  Speakers were Michael Kalichman, Ph.D., Adjunct Professor of Pathology and Director of the Research Ethics Program at University of California, San Diego, and Beth Fischer, M.Ed., co-director of the Survival Skills and Ethics Program at the University of Pittsburgh.  Dr. Kalichman described the different methods and formats available for RCR instruction, and provided conference participants with numerous resources on which to draw when designing an educational program.[1]  He emphasized that teaching RCR should ideally focus on three components: knowledge, skills, and community.  Knowledge, including RCR-relevant policies, guidelines, and rules, is most helpful when a problem or dilemma has a clear, definitive answer.  However, when a given dilemma has no clear solution, skills such as people management, conflict resolution, and critical thinking, as well as shared community standards, become especially crucial. 

 

Beth Fischer expanded on this sentiment by pointing out that research guidelines are not always sufficient to promote RCR; researchers need to learn ethical reasoning.  Mechanisms for learning ethical reasoning have typically been inadequate, which is why programs such as Pittsburgh’s Survival Skills and Ethics have been developed.  Ms. Fischer distinguished between “misdemeanors” in research (e.g., failure to publish, “honorary” authorship, misleading graphics) and “high crimes” (e.g., fabrication, plagiarism), and explained that misdemeanors are highlighted in training because they arise more in day-to-day life, and involve an ethical “slippery slope” that can lead to more serious violations.  Ms. Fischer emphasized the importance of faculty researchers providing much of the instruction – faculty can learn by teaching, and thus improve their own ethical reasoning skills. 

 

The fourth plenary session focused on the issue of research guideline implementation.  Speakers were David Wright, Ph.D., Assistant Vice President for Research Ethics and Standards and University Intellectual Integrity Officer at Michigan State University, and Randall Reed, Ph.D., Professor of Molecular Biology and Genetics and Chair of the Conflict of Interest Policy Review Committee at Johns Hopkins University.  Using a case study approach, Dr. Wright discussed the implementation of authorship and data management guidelines at Michigan State, and reported mixed success as to their effectiveness in dispute resolution.  Acknowledging that information is anecdotal and sample size small, Dr. Wright argued that the guidelines have in some cases been useful in resolving incipient disputes through arbitration before they could lead to misconduct allegations.  However, other cases have been disappointing failures.  It is unclear whether a more formal, enforceable policy, as opposed to the voluntary guidelines currently in place, would have been more effective.  Dr. Wright asserted that empirical studies on this issue, while difficult to design, are crucial in order to answer this question. 

 

Randall Reed described the process of revising the Johns Hopkins University School of Medicine conflict of interest policy.  The previous policy was largely restricted to faculty, did not address human subjects research, and was not consistent with AAMC guidelines.  The revised policy broadens coverage to anyone participating in research, makes reporting requirements explicit, and ensures transparency of procedures, review, and management.  It strengthens human subjects protection through greater involvement of IRBs in evaluation, and sets a threshold for presumptive prohibition of concurrent financial interests and participation in related human subjects research.  Non-compliance may trigger review under procedures for dealing with allegations of misconduct.  The new policy, including frequently asked questions, is published on the school Web site and a Web-based training module is currently under development and will be required for anyone participating in research. 

 

After a dinner break, a representative from each of the five breakout groups presented a brief summary of important issues to consider in research guideline development and implementation.  Common themes included the importance of dean, department, and faculty participation and buy-in when creating guidelines; flexibility when applying guidelines to individual departments or situations; avoiding overstepping existing policies; preserving academic freedom; using multiple education strategies and media; creating a culture of “concern” as opposed to “compliance,” reasonable expectations; and assessment through outcome studies.  Also emphasized were the importance of modifying guidelines for each institution, and the difference between training and education. 

 

On the second morning of the conference, the fifth plenary session focused on assessing the impact of research guidelines.  The speaker was Richard McGee, Ph.D., Associate Dean for Student Affairs at the Mayo Graduate School.  Dr. McGee provided an overview of basic principles of assessment and evaluation.  He then described how one might assess the impact of some of the guideline examples from Creating Effective Research Guidelines.  He pointed out that while qualitative assessment can provide much more information regarding complex issues in guideline effectiveness, this type of evaluation can be more difficult due to factors such as the extra effort and cost involved in conducting focus groups and structured interviews.  Success of any given guidelines, he argued, will often depend on the extent to which the recommended behavior is normative and well-accepted.  Assessing underlying assumptions, values, or norms reflected in guidelines is critical in understanding why a guideline may or may not be adopted. 

 

The sixth plenary session focused on climate and environmental factors that affect the development of research guidelines.  Speakers were Paul Friedman, M.D., Professor Emeritus at University of California at San Diego, and Jeffrey Kahn, Ph.D., MPH, Director of the Center for Bioethics and Professor of Medicine at University of Minnesota.  Dr. Friedman provided a historical perspective, summarizing aspects of research climates that may not be conducive to RCR, including pressure to publish and/or get drug company support, and the fact that researchers often have considerable autonomy but have unsuitable role models on which to base their behavior.  The responsible conduct of research, he argued, involves a great deal of effort by the individual, who must resist institutional pressures, conflicts of interest, and time pressures.  The key to honest research is an open environment that promotes admitting uncertainty, receiving constructive criticism, and helping others. 

 

Jeffrey Kahn discussed his experiences at University of Minnesota, where he arrived in the wake of a serious scandal that resulted in the university’s placement on NIH’s “Exceptional Status” list.  This meant that the university was proven to have poor business practices, and that NIH would consequently monitor all grant management very closely.  At that time, the university made a commitment to RCR education and training to help prevent future difficulties.  Dr. Kahn argued that the importance of leadership from the top, including strong public commitments from the highest levels of administration, cooperation from the deans, and significant commitment of resources, cannot be overstated.  Commitment to RCR should be faculty-led, and it should be recognized that while different schools and departments have different needs, everyone should understand the basics, while subsets of researchers should have more in-depth training in specific areas, such as human and animal subjects issues.  The “fear of God” helps this process move forward, Kahn argues, but “carrots” or rewards, such as a reception at the President’s residence, can also help.  (Note:  University of Minnesota was removed from the NIH Exceptional Status list in February 2001). 

 

The conference emphasized an interactive focus, and conference participants enthusiastically contributed energy and insight to the proceedings.  Each plenary session, except for the last, included a fifteen-minute question and answer session, and ended with a forty-five minute breakout session.  The last session concluded with an extended question and answer period, and closed with adjourning remarks from Dr. Rhoades.  ORI will use feedback from conference participants when revising the resource document Creating Effective Research Guidelines.  The ultimate goal will be to produce a useful resource on which institutions can draw when creating and/or revising research guidelines.