One of the significant differences between a systematic and non-systematic (i.e. traditional) literature review is that during a systematic review process information is searched also within grey literature that has not been peer-reviewed. In practice this means looking for studies using internet search engines, searching websites of specialist organizations, and asking experts of the research topic if they know any potential studies. Including grey literature into a review enhances its comprehensiveness and reduces the possibility of publication bias (Collaboration for Environmental Evidence 2018).
Variety is an asset
The research question and the geographical scope of the review determine which sources of grey literature are relevant. However, when scoping potential sources, it is important not to allow other factors limit the search too much. For example, researchers should not restrict search purely to organizations that are already familiar to the research group or only to organizations of certain type, for example non-governmental organizations (NGO) or government agencies. Thus, it is worth thinking what kind of different sources can provide data for the review. For example, there are various NGOs, and governmental agencies exist at many administrative levels. In addition to these, also different research institutes, educational institutions, societies, clubs, foundations and companies may have relevant studies.
Making use of the snowball effect
The internet is full of forest-related information, and after finding one relevant organization it is usually easy to find more. Often, websites of international organizations help to identify other potential organizations. In the case of forest research these organizations include, for example, the European Forest Institute (EFI) and the International Union of Forest Research Organizations (IUFRO). Websites of these organizations often include links to their collaborative organizations, and nowadays also internet search engines can suggest sites comparable to the ones already found. As a consequence of this snowball effect, more and more new potential organizations encounter.
How to implement into practice?
The scoping phase of our planned review “Effects of forest management on biodiversity” provides an example of the diversity of grey literature sources. During scoping, 109 organizations were evaluated as potentially relevant. After a more detailed assessment 67 organizations were included in the research protocol. In addition to the websites of potentially relevant organizations, dozens of other organizational websites were visited but immediately excluded as irrelevant. So, the problem isn’t a lack of possible grey literature sources but instead, how to find only the organizations relevant to the review.
Reference:
Collaboration for Environmental Evidence. 2018. Guidelines and Standards for Evidence synthesis in Environmental Management. Version 5.0 (AS Pullin, GK Frampton, B Livoreil & G Petrokofsky, Eds) www.environmentalevidence.org/information-for-authors.
