ENTRAR            

SPECIAL REPORT

Nursing Research and Peer Review
Jordi Piqué Angordans,¹ Ramón Camaño Puig^2
1Catedrático de Escuela Universitaria, ²Profesor Titular de Escuela Universitaria. Escola Universitària d'Infermeria, Universitat de València, Valencia, Spain

Mail delivery: Jordi Piqué Angordans. Escola Universitària d'Infermeria. C/ Jaume Roig s/n 46010 Valencia, España

Manuscript received by 15.02.2008
Manuscript accepted by 19.05.2008

Index de Enfermería [Index Enferm] 2008; 17(3):222-226

How to cite this document

Abstract

Through peer review a journal edeavours to guarantee the quality of published articles. The aim of this paper is to analyze the difficult assessment task in the Spanish nursing area. We also present some suggestions on how to develop this task, based on data gathered from Spanish nursing journals, and thus contribute to enhancing nursing publications. We also give the results of a survey of relevant Spanish journals, and offer recommendations for nursing professionals.
Key-words: nursing, peer review, Spanish nursing journals, publication quality.

Introduction

    Peer review has been taking place for over 200 years and today is used by most professional journals. However, until the decade of the 40s it did not become an institutionalized practice (Rennie, 1999: 4); since its beginnings it aimed at guaranteeing the article's quality and its passing through the editorial process constituted the guarantee of such quality.
    In nursing publications, however, the tradition has not been accepted worldwide, and much less so among Spanish nursing professionals. In fact, such publications are commonly influenced by medical professionals whose decisions operate on matters pertaining to structure, quotations and references, as well as the revision of proposed publications. In Spain, the schools of nursing were incorporated into the university structure in 1977, but it was not until recently that the possibility of pursuing studies beyond the university diploma has become a reality; thus, the research and publication in journals by nursing professionals has been lagging behind in the last few decades.
    The first Spanish nursing journal, Rol de Enfermería (Barcelona), appeared in 1978, and during the decade of the 80s a few more were published, such as the now extinct Enfermería Científica (1982), Nursing in its Spanish edition (1983), and Enfermería Intensiva (1989). During the 1990s, some of the main journals today appeared, such as Enfermería Clínica (1991), Index de Enfermería (1992), Metas de Enfermería (1998), among others (Camaño Puig, 2002: 218).
    In this paper we will analyze the state of the art of peer review in the nursing area, a discipline with an enormous human potential, while at the same time we offer some aspects of the process showing how to carry it out and thus contribute to enhancing, in the best way possible, publications in this area.

Peer review and quality publications

    The development of the review process in nursing journals can only be understood in comparison with other disciplines, mainly with medicine. In the last few decades numerous publications have analyzed and questioned the peer review system, claiming the need to make it more democratic and less biased towards private interests (Godlee & Jefferson, 1999). The literature has repeatedly deliberated over the process, whether due to the excessive subjectivity of some reviews as a "human activity" (Rennie, 1999: 8), or their lack of greater reliability and validity (Crandall, 1986). In turn, Bower (1991) suggested that the process, as "science's gatekeeping system", called for "an overhaul", and Hagan (2003), while accepting the problems entailed by the revision process, informed that an urgent reform was necessarty since it was being subjected to a constant survaillance and scrutiny.
   Swanson et al. (1991) analyzed the possibilities of nurses publishing in US journals, while Fondiller (1994) enquired as to what extent peer review favored the publication of nursing papers. Regarding the ethics underlying peer review, Freda and Kearney (2005) claimed that it should be revised, while questioning the reliability of blind revisions, proposing better preparation of reviewers for the task of criticizing manuscripts and writing reviews. Hoyt and Proehl (2007: 264), in turn, recognized the benefits derived from peer review since they show the actual implication and commitment to the profession.
    Up till now, medical journals and their editors have shown a major concern over peer review, particularly since 1989 when the American Medical Association organized in Chicago the first International Conference on Peer Review (the sixth has been announced to be held in Vancouver next year). Produced after the first conference, the publications dwelled on the problems related to open, blind or double blind peer review, its positive and negative aspects, and on the fact that a similar reaction was produced by Spanish scientists, such as Campanario (1993, 1998), from his perspective as a physicist, as well as Silva and Campillo-Artero (1991) and García et al. (2002), in relation to medical articles.
    Nevertheless, peer review has not received the necessary attention from Spanish nursing professionals. This is because their attention has basically focused on the global bibliometric analysis of publications or on mitigating the problems derived from the impact factor (see, for instance, the papers by Pardo et al., 2001; Galvez Toro et al., 2005, and others).

What is a peer review?

    The peer review process can be defined as "a critical assessment by knowledgeable scholars of the quality of a scholarly article submitted for publication to a scholarly journal" (Davison et al., 2005: 969). This assessment is generally carried out by two or more anonymous experts in a specific area. The idea is that the members of a journal's editorial board do not have the full comprehension of every aspect of research in their field; this would prevent them from an adequate evaluation of the quality of each submitted article, particularly in areas such as nursing in which research can be so complex and diversified. In addition, the anonymity of the process, both of experts and authors during the review process, is the main characteristic that guarantees and keeps tha quality of the publications, through which, according to King et al. (1997), less biased publications may be obtained.

Characteristics of reviewers and their reports

    It is easy to give advice, but it is more difficult to know who and what a good reviewer of manuscripts should be like. In their study of 420 manuscripts submitted to the British Medical Journal, Black et al. (1998: 232-233) gave the main characteristics a good reviewer should have, drawn from both editors' and author's assessments. Although it is a British journal and one of the oldest, the results carry a significant American bias, particularly from the editors' point of view: the ideal reviewer should be young, between 30 and 40 years of age, preferably a United States resident, with preparation in epidemiology and statistics, a member of a research group, but not a member of the editorial team, and the report should be a product of at least three hours of work (ibid., p. 232). Godlee et al. (1998: 238) coincide in pointing to this epidemiological and statistical knowledge, but they also add that, in order to be able to identify the manuscript's weaknesses, publishing experience is also recommended. Finally, editorialyzing on Black et al. (1998), Yucha (2002: 71) suggests the following features for the nursing reviewer: "expertise in the specific field, recent publication record, knowledge of one's own limitations, and the ability to complete the review in the required time frame." Davison et al. (2005: 974) add reviewer qualities, such as open-mindedness, freedom from biases and prejudices; reviewers should also be ethical and diligent in their work. When writing their reviews, the reviewers' comments should be polite and tactfully presented, avoiding confrontations with authors, grose and unkind language. In other words, their attitude "should be constructive [...] rather than destructive" (Davison et al., 2005: 971).
    Seals and Tanaka (2000), however, adopt a didactic approach for students and novel reviewers; they first propose the analysis of previous reports on the same topic, which is usually the basis and justification of the research, with the aim of finding out if a manuscript is not simply "more of the same" (p. 54), or if, on the contrary, the study provides a new and "more effective approach that could results in the resolution of currently controversial and/or equivocal findings" (p. 54). In other words, the adequacy of the manuscript to the journal is detected when we analyze the experimental question or hypothesis: if it is not sufficiently significant, the rest is superfluous and, therefore, it is not relevant.

The review process is not all that rewarding and requires a hefty dosis of work and, above all, idealism. Molassiotis and Richardson (2004) editorialize, in the European Oncology Nursing Society, and quote Wager et al.'s (2002) three basic questions a reviewer must ask about a manuscript in order to produce a good review and how to survive peer review:
    a) Do I understand it? (clear research questions and methods);
    b) Do I believe it? (conclusions justified by the data and use of valid methods) and
    c) Do I care? (important and interesting research questions or the "so what" (Molassiotis & Richardson, 2004: 360).

    In similar terms, Hooper (2003: 222) explains that the review process consists in asking experts if a manuscript is relevant, if it is a quality work and if it can be improved. She claims that the term "experts" is fundamental in the review process, since they should provide a scientific opinion of the manuscript's content. In fact, King et al. (1997: 163) contend that "content expertise remains the single most defining qualification"; thus, they would claim more specialization from reviewers. Nevertheless, they also suggest that an excessive specialization may also imply conflict of interests due to the bias opinion in an area in which they are highly qualified.

The author and peer review

    For a novel author, manuscript rejection is not much less than an academic trauma. The rejection stems from the technical supposition that the review, as an integral part of the evaluation process, is the result of a critical, independent and fair assessment, with no preconceived ideas, biases or conflicts of interests that may devalue it. The process, then, should help the journal's editor to decide if a given manuscript is adequate for publication.
    Reviewers are instructed simply over what should be assessed; however, for their work to be effective, manuscripts should be filtered by the editorial team before reaching the reviewer's hands for the review. Lack of preparation on the reviewer's part may lead to the publishing of fraudulant articles due to the fact that errors contained therein have not been previously detected (Campanario, 1998: 279-280). The literature, however, has shown (Campanario & Acedo, 2007: 734) that manuscript rejection is not always due to purely objective reasons; thus, an obvious need of reviewer preparation exists, since reviewers are responsible of assessing if a manuscript is scientifically valid, and the success of the process "hinges on the skill, discernment, dedication, and fair-mindedness" of expert reviewers (Kliewer et al., 2005: 1731).
    Although the problem affects every author who has suffered a manuscript rejection, it is understandably more important for the nonnative author who tries to write in a language which is not his or her own. The authors of this article have encountered this situation: one of the reviewers considered our manuscript inadequate for publication simply and only because it contained two terms perfectly adequate in the British variety of English and less in the American variety. The second reviewer, in turn, simply suggested a change of the two terms in question and requested additional information on some data provided. This, we understand, is an example of a destructive review (the first one) versus a constructive one (the second review), as Davison et al. (2005: 971) pointed out.
    Even though the review system is intended to be double blind, its result should not be affected by the possible identification of the author; it "should be based on the scientific merit of the material reviewed" (Molassiotis & Richardson, 2004: 361). It is obvious that external factors exist that are beyond the reviewer's control; in such case, they should be referred to the editorial board, but in any case it is the author who is affected by the rejection. Westergren (2006), an experienced author and journal editor, recalls how his first rejection as author affected him; he offers some advice both for the preparation of manuscripts and for the time span that elapses between submission and an editor's response and, finally, how to react when faced with the decision and reviewers' comments. When a review is negative, it is difficult not to accept it as scomething personal but, as he says, quoting Kahnweiler (1997: 7): "Deal with it, accept it, and move on" (Westergren, 2006). The fact is, there are a lot of journals that may be interested in the manuscript, and it not being published in the end might be an indication that the manuscript contains significant defficiencies.

Spanish nursing journals and peer review

    We have mentioned how complicated and difficult it is to choose good reviewers, but there is another important aspect which we must mention in reference to Spanish journals. Specifically, if a journal claims that a given anonymous and independent editorial revision process is followed, it would ostensibly mean that it really follows that process. It is not just a question of a lack of quality, frequently quite evident in some publications, but that these publications are also a reflection of the research category of the editorial team and its reviewers who are responsible for the examination of the manuscripts' quality. Elsewhere we have commented (Piqué-Angordans & Camaño, 2008) on the topic of whether or not nursing professionals should adhere to the norms of the International Committee of Medical Journal Editors (ICMJE, 2007) for their scientific publications. The fact is that many journals do announce that they closely follow the ICMJE requirements; in fact, under the subtitle "Peer Review" it is said that the journal adhering to such a system must specify the review process for the public interest and editorial transparency. Nevertheless, this adherence is often limited to following little more than the style sheet and instructions to authors. The result is a proliferation of published papers that are partial repetitions of manuals on nursing procedures, while few of them are original articles that make nursing science advance. Gálvez Toro et al. (2005) have already exposed the ambiguity of criteria in the evaluation of the quality of publications. Nursing needs quality publications in order to properly establish itself in the world of science. We believe that the well-known "publish or perish" aphorism frequenly contributes to the proliferation of mediocre works; this is why the revision of experts is of utmost importance when assessing manuscripts and obtaining quality publications. Although the current peer review process is not perfect, it is currently the best method available and our efforts must be directed at contributing to its efficacy.
    Nevertheless, the question is whether our professionals are surrounded by the adequate publishing environment for this quality production. In our Spanish schools of nursing, the staff is made up of both medical and nursing professionals. On the one hand, however, few of these medical professionals produce articles related to nursing; what is more, they strive for publication in the best medical journals with the highest impact range possible. Nursing professionals, on the other, resort to nursing journals included in the Thompson ISI list, usually having a low impact factor (according to the Journal Citation Records, during the 2002-2004 period, Nursing Research had the highest index among nursing journals with 1.313). However, not one single Spanish nursing journal was included in that list and only one originally not publishing in English, the Brazilian Revista Latinoamericana de Enfermagem, has been recently added. Furthermore, none of the three most cited Spanish nursing journals, Enfermería Clínica, Revista Rol de Enfermería and Enfermería Científica (Pardo et al., 2001: 940), are included in the ISI list, even though their ranking is relatively high in the CUIDEN Rch index of historical repercussion with 3.31, 3.10 and 1.50, respectively (Gálvez Toro et al., 2004).
    From a survey we have carried out by personal email communication with the editorial teams of Spanish nursing journals we have found that, in the first place, the journals follow the recommendations of the ICMJE (2007), in which the anonymity of the reviews is underscored. They also recommend that they be carried out by "experienced and independent authors" (Enfermería Clínica). In the reviewer selection, they generally suggest that reviewers be experienced researchers and active in clinical practice (Metas de Enfermería and Gerokomos), and also be from both nursing and medical areas (Rol de Enfermería); in addition, they should be knowledgeable in specific areas, and their curricula should include a number of publications, with information regarding the type publication it is and its impact factor (Index de Enermería, Rol de Enfermería). In one instance, the reviewers were related to the private firm that own and publish the journal (Híades), which obviously implies a certain bias in the acceptance or rejection of submitted manuscripts. As far as the type of revisión that was being carried out, most journals provided a standardized questionnaire for reviewers in which the quality of the paper and its relatedness to the journal's editorial line was assessed, in addition to other aspects such as justification of aim, clarity of exposition, bibliographic documentation, contribution of new knowledgte, amond others (Metas de Enfermería, Híades, Index de Enfermería).
    The literature usually attributes a greater refusal of manuscripts from journals ranking high in the impact lists. However, we cannot confirm this fact from the data we were able to collect. It is significant that the percentage of rejection of submitted articles ranges from a 40% for Metas de Enfermería to a 70% for Index de Enfermería. Furthermore, of the accepted papers for publication, 90% of them are published once corrections and changes have been incorporated by the authors (Metas de Enfermería), while the rest (10%) are published as submitted. Finally, the journal editors commented that most differences of opinion with authors were resolved by speaking directly with them and explaining the criteria followed by the reviewers (Rol de Enfermería). At times, some of the more tactless comments by reviewers are reworded by the editorial staff of the journal (Rol de Enfermería). Although it is difficult to tell an author that an article is not publishable, the communication is usually done in a positive manner indicating that if no reply has been received in a given period of time, the author can submit the manuscript to another publication (Rol de Enfermería). Seldom de they receive reactions of outrage from authors whose manuscripts have been rejected; no specific case was reported by the respondents. In most cases, authors accept the rejection or the request for extensive changes by simply withdrawing their papers from the editorial process.

Possible alternative

    Campanario (2002: 175) mentioned the various problems related to peer review and, at the same time, the lack of solutions offered. One of these problems has to do with the position of power stemming from the possible connections between authors, editors and reviewers, or the "invisible college" as it has been called, which frequently leads to favoritism during the proces of publication. BioMed Central has proposed that authors themselves suggest four reviewers as they submit their manuscripts for publication (Wager et al., 2002); however, they disqualify the idea since the reviews received by suggested reviewers by authors produce similar quality reviews (p. 64).
    Campanario (2002) offers a computer-based solution to minimize the problems of the current system. It would consist in the establishment of a "Central Facility", or Metajournal, organized by disciplines and open to any scientist who wishes to communicate relevant research results. Editorial teams would name a journal scout, who would be responsible for locating innovative results, and offer authors of the best papers the possibility of publication; the author, in turn, could freely choose among the best "offers" (ibid., pp. 178-179). Aside from the logical flaws any new system might have, specified by Campanario (p. 180) himself, it would save review time and responsibility on the reviewers' part, solving the more problematic issues of the current system, notwithstanding the fact that any scientist could be self-eligible as a free-lance reviewer.

Final observations

    As members of our incipient academia in the nursing area, we are respectfully asked to publish and to publish quality papers to cope with the pressure of building up our curricula. It is obvious that the critical mass in the nursing area is insufficient, given its short academic history and the fact that some journals are forced to look for reviewers in areas other than their own, or perhaps having to limit their reviewers exclussively from the clinical areas. Here is where Spanish nursing journal editors must hone their methods for choosing the members of their editorial boards so that the review may help produce the desired quality effect in nursing publications.
    Obviously, until some years have passed, this hope for repercution will not take place in the area of nursing as a result of the issuance of masters and doctorate degrees in nursing alone, the programs for which are currently being developed in Spain. Thus, we expect that peer reviews will produce more quality articles in the near future and thus obtain the corresponding acknowledgement in the scientific world.

References

    Black N, Rooyen S. van, Godlee F, Smith R, Evans S (1998). What makes a good reviewer and a good review for a general medical journal? JAMA 280: 231-233.
    Bower B (1991). Peer review under fire: science's gatekeeping system may need an overhaul. Science News (June 22).
    Camaño Puig R (2002). The state of the nursing science in Spain. Revista Latino-americana de Enfermagem 10,2: 214-220.
    Campanario JM (1993). Consolation for the scientist: Sometimes it is hard to publish papers that are later highly cited. Social Studies of Science 23: 342-362.
    Campanario JM (1998). Peer review for journals as it stands today-Part 1 & 2. Science Communication 19: 181-211, 277-306.
    Campanario JM (2002). El sistema de revisión por expertos (peer review): muchos problemas y pocas soluciones. Revista Española de Documentación Científica 25: 166-184.
    Campanario JM, Acedo E (2007). Rejecting highly cited papers: the views of scientists who encounter resistance to their discoveries from other scientists. Journal of the American Society for Information Science and Technology 58: 734-743.
    Crandall R (1986). Peer review: improving editorial procedures. BioScience 36,9: 607-609.
    Davison R, Vreede GJ de & Briggs RO (2005). On peer review standards for the information systems literature. Communications of the Association for Information Systems 16: 967-980.
    Fondiller SH (1994). Editorial Peer Review: Is Nursing at Risk? Nursing and Health Care 15,3: 142-148.
    Freda MC, Kearney MH (2005). Ethical issues faced by nursing editors. Western Journal of Nursing Research 27: 487-499.
    Gálvez Toro A, Amezcua M, Hueso Montoro C (2005). CUIDEN Citación y la valoración de las publicaciones científicas enfermeras. Index de Enfermería 51: 7-9.
    Gálvez Toro A, Hueso Montoro C, Amezcua M (2004). Indicadores CUIDEN de repercusión de las revistas de enfermería del área lingüística del español y del portugués. Index de Enfermería 13,46: 76-80.
    García AM, Plasència A, Fernández E (2002). La calidad de las evaluaciones de manuscritos en Gaceta Sanitaria. Gaceta Sanitaria 16,3: 244-249.
    Godlee F, Gale CR, Martyn CN (1998). Effect on the quality of peer review of blinding reviewers and asking them to sign their reports. JAMA 280: 237-240.
    Godlee F, Jefferson T (eds.) (1999). Peer Review in Health Sciences. London: BMJ Books.
    Hagan P (2003). Peer review under scrutiny. The Scientist 4,1 (February 3).
    Hooper VD (2003). Peer review: is it for you? Journal of PeriAnesthesia Nursing 18,3: 222-223.
    Hoyt KS, Proehl JA (2007). Peer review for professional publications. Advanced Emergency Nursing Journal 29: 260-264.
    King CR, McGuire DB, Longman AJ & Carroll-Johnson RM (1997). Peer review, authorship, ethics, and conflict of interest. Image: Journal of Nursing Scholarship 29: 163-167.
    Kliewer MA, Freed KS, DeLong DM, Pickhardt PJ, Provenzale JM (2005). Reviewing the reviewers: Comparison of review quality and reviewer characteristics at the American Journal of Roentgenology. American Journal of Roentgenology 184: 1731-1735.
    Molassiotis A, Richardson A (2004). The peer review process in an academic journal. European Oncology Nursing Society 8: 359-362.
    Pardo C, Reolid M, Delicado M-V, Mallebrera E, García-Meseguer M-J (2001). Nursing research in Spain: bibliometrics of referentes of research papers in the decade 1985-1994. Journal of Advanced Nursing 35: 933-943.
    Piqué-Angordans J, Camaño R (2008). Enfermería e investigación: To Vancouver or not to Vancouver. Revista Latinoamericana de Enfermagem (en prensa).
    Rennie D (1999). Editorial peer review: its development and rationale, in Godlee & Jefferson (eds.), Peer Review in Health Sciences. London: BMJ Books. pp. 1-13.
    Seals DR, Tanaka H (2000). Manuscript peer review: a helpful checklist for students and novice referees. Advances in Physiology Education 23: 52-58.
    Silva A, Campillo-Artero C (1991). Cómo se deben evaluar los artículos científicos propuestos para publicación. Medicina Clínica 97: 744-748.
    Swanson E, McCloskey J, Bodensteiner A (1991). Publishing opportunities for nurses: a comparison of 92 U.S. journals. Image-Journal of Nursing Scholarship 23: 35-38.
    Wager G, Goodlee F, Jefferson T (2002). How to Survive Peer Review. London: BMJ Books.
    Westergren A (2006). What makes a good peer review? An author's perspective. Nurse Author & Editor 16,3. Disponible en:
<https://www.nurseauthoreditor.com> [accedido 2 de enero 2008].
    Yucha CB (2002). The peer review process (aka Peer Reviewology). Biological Research for Nursing 4: 71-72.

RECURSOS CUIDEN

RECURSOS CIBERINDEX

FUNDACION INDEX

GRUPOS DE INVESTIGACION

¿Quiénes somos?
RICO Red de Centros Colaboradores
Convenios
Casa de Mágina
MINERVA Jóvenes investigadores
Publicaciones
Consultoría