Who actually did this research review for government? And does it matter?

In October 2021, during a consultation with the Classification Policy group of the Dept of Communications, CMA was invited to comment on a recently commissioned research review of the impact of violent videogames. We did so, but were not impressed with the outcomes. Neither the review’s coverage of the extensive range of research and reports in the field, nor the conclusions drawn, seemed adequate. 

Given that such reviews can influence government policies, CMA has been asking for the reviewers’ names, qualifications and experience in the field of violent video game research. Such information is always available for published scientific research reviews. 

However, all of our 18 months of requests have been refused.

This never-ending story started in late November 2021, when CMA informed the Assistant Secretary Classification, Department of Infrastructure, Transport, Regional Development and Communications that we found its review of the research on the impacts of violent video games seriously deficient.

We included these preliminary findings:

• It purports to provide an Australian perspective, and yet only mentions one article from Wayne Warburton (and then only as co-author). Assoc Prof Warburton is the pre-eminent researcher in this field in Australia. [Full disclosure: Assoc Prof Warburton is the Vice-President of CMA.]
• A search of Google Scholar for articles by Warburton, between 2009 and 2021, found at least a dozen with relevance to this review.
• There is ongoing debate in this field, but the review’s bibliography suggests an over-representation of researchers who usually find no effects.
• For example there are 17 articles which have Chris Ferguson as lead author, and a further 4 as a co-author. Respected researchers with long track records in the field, such as Craig Anderson, Doug Gentile, Brad Bushman, Barbara Krahe, Tobias Greitemeyer have respectively 4, 2, 2, 0, 2 each. (We note that the only author cited in Appendix B on bias is Ferguson!)
• Those who have worked extensively in the field and whose research achievements have received wide recognition seem not to have been adequately cited.
• In such reports, it is usual practice to name the authors even if a company were the commissioned agent. No names are cited here.

CMA asked for:

• the qualifications and relevant experience of the people within the Behavioural Insights Team (BIT) who did the work.
• What was the actual search strategy adopted? A list of terms used is not sufficient to judge the adequacy of the search.

The Department responded that:

The original impetus for the BIT review was to understand the links between violent and very violent video games and physical violence committed by adults, including against women. The review sought to take a measured, transparent and objective approach to evaluating the evidence that has been undertaken over the last 12 years around the link between violent video games and aggression and whether international research (which we knew would be far more extensive) can be generalised to an Australian context. It is important to note that the BIT review, despite its findings regarding the present evidence base, does not entirely close the door on further research on the impacts to be explored in this area.

Further correspondence ensued, with CMA asking again for the names and qualifications of the researchers within BIT who conducted the review. After all, the Department had claimed the process was transparent, as well as measured and objective. CMA also provided its comprehensive review of the report, which included the opinions of several overseas researchers with extensive credentials in the field.

These comments included:

This report contradicts many other reports on the topic.

This was clearly an incredibly biased and selective review of the literature – and the results of some of the studies noted are not even conveyed accurately.

I noticed that the work of several leading researchers in the field (those whose work suggests the opposite effect of the review) was barely mentioned or not at all (for example, no mention of work by Barbara Krahe or Karen Dill (Shackleford), or of early work by Funk and colleagues … ). Valid reviews examine all views of a controversial issue and then weigh the evidence according to various established criteria. This alone casts doubt on the value and validity of the review. As a frequent reviewer I would never recommend such a review for publication in any journal or other reputable outlet.

Having again received a refusal to engage in discussion on whether the review was deficient, and to provide the names of the researchers, CMA lodged an FOI request in June 2022 requesting access to documents in the possession of the Department of Infrastructure, Transport, Cities and Regional Development (the department) in accordance with the Freedom of Information Act 1982 (the FOI Act).

Access was granted to numerous documents in September 2022, but with the names and qualifications of the researchers redacted on grounds of sections 47C, 47F and 47G of the FOI Act.

Questions on notice related to these issues were put to the Department during Senate Estimates in November 2022, with a further refusal issued in February 2023. One answer included this defence:

Neither iteration of the Behavioural Insights Team literature review commissioned by the department contained original research, as the aim was to review existing Australian and international academic research. The authors of the original studies cited in the review are clearly referenced.

When a research review is published in a scientific journal, it is considered research and the authors are named. No reason is given for expecting any different of a review commissioned, and presumably to be relied on, by government.

CMA’s aim in challenging this review was to support the development of a classification system that is well-grounded in credible research, and that enables informed choice.

However, the Classification Branch has shown an unwillingness to engage with the evidence CMA provided regarding the deficiencies of the review. In the course of doing so, it has raised questions as to whether key decision-makers even understand the nature of research.

For now the story is ended.