How much can I trust information from various types of publications?
Definitions and implications for reliability of information
Not all scientific and technical information is created equal – there is considerable variation in reliability, validity, and credibility. The nature of scientific investigation is such that the validity commonly changes with time as knowledge improves. The best scientific publications contain undisputed facts that can be verified by scientists other than the authors. Then, such publications interpret the information in an objective way, but interpretations are not facts. The definitions below are an attempt to guide those unfamiliar with different types of publications. Bear in mind that the reliability, validity, and credibility of information can still vary within the categories of publications listed below.
Summary of a written work; briefly describes the substance of the work. Most peer-reviewed journal articles and many other publications have them at the beginning of the publication.
In some cases, abstracts are the only written record of oral or poster presentations at conferences or symposia. Abstracts of such presentations are commonly, but not exclusively, preliminary works that are later formalized as peer-reviewed articles, which usually undergo more rigorous review.
An official record of a conference meeting. These proceedings can include a collection of documents that correspond to presentations and information given at the conference. A publication in a proceedings might be an abstract, an extended abstract (3-4 pages), or a more formal conference paper.
The level of review varies with the organization sponsoring the conference. Abstracts and papers in some proceedings have not received as much professional scrutiny as others, and they commonly receive less scrutiny than a peer-reviewed journal article. Professionals in subdisciplines often know what level of review is employed for particular conferences. There is no easy way for the general public to evaluate the level of review, and thus the reliability, of a proceedings publication.
A publication (which could be called a report, an investigation, a code, and many other names) by a branch of government, which can include local, state or federal government. The level of peer review and scientific or technical rigor varies widely.
Peer Reviewed (Journal/Article/Refereed):
Written material that has been reviewed by experts in the field for legitimacy and validity before being published. Commonly one of the most reliable, and technical, of sources. Authors may submit manuscript drafts to colleagues informally, then send revisions to a scientific or technical journal. The journal editor usually sends the manuscript for review by two or three individuals who should be very knowledgeable. These individuals ideally evaluate the manuscript, figures, and references very carefully and objectively. They then recommend changes, and either acceptance or rejection of the manuscript. The editor returns the manuscript and reviews to the original authors. If permitted by the editor, the authors then revise the manuscript. Depending on the nature of the changes required, the editor may then publish the revised manuscript or require a complete re-review of the manuscript.
Scientists and engineers commonly recognize that the quality of all journals is not the same. Some journals have very wide readership (and therefore a high “impact factor”). The journals Science and Nature have high impact factors and cover a wide range of scientific topics; articles in these journals are highly regarded. Other more specialized journals have lower impact factors, and cater more to specialists. Within the specialized journals there are upper-tier journals that are more highly regarded than lower-tier journals. Professionals in subdisciplines often are familiar with journal reputations, but the general public is not likely to know the level of review that articles have undergone.
Information organized in a narrative, or presented in graphics. Reports can be prepared on an as-needed basis depending on the topic, or may be re-occurring on a specific time interval basis. Reports may be produced by individuals or teams in consulting firms working for industry, for government agencies, or for environmental groups. Other reports are produced by industry groups, individual companies, by government agencies, or by environmental organizations. Rigorous peer review is not the rule, but some governmental reports do undergo rigorous peer review; an example is the rigorous review that most US Geological Survey publications undergo.
A presentation of information in a format emphasizing brevity, key points of interest or concern and a general desire to convey the most relevant points in the least amount of space. The information in fact sheet can be manipulated by the author by the inclusion or exclusion of information. The term “fact” may have a looser meaning than “observation” as used by scientists or engineers.