Quality of Systematic Reviews of the Foods with Function Claims in Japan: Comparative Before- and After-Evaluation of Verification Reports by the Consumer Affairs Agency

Background: In Japan, a new type of foods with health claims, called Foods with Function Claims (FFC), was introduced in April 2015 in order to make more products available that are clearly labeled with certain health functions. Regarding substantiating product effectiveness, scientific evidence for the proposed function claims must be explained by systematic reviews (SRs), but the quality of SRs was not clear. The objectives of this review were to assess the quality of SRs based on the FFC registered on the Consumer Affairs Agency (CAA) website in Japan, and to determine whether the CAA’s verification report in 2016 was associated with improvement in the quality of SRs. Methods: We evaluated the reporting quality of each SR by the AMSTAR checklist on methodological quality. We searched the database from 1 April to 31 October 2015 as the before-SR and from 1 July 2017 to 31 January 2018 as the after-SR. Results: Among the 104 SRs reviewed, 96 final products were included: 51 (53.1%) were supplements, 42 (43.8%) were processed foods without supplements, and 3 (3.1%) were fresh foods. Of the 104 SRs, 92 (88.5%) were qualitative reviews (i.e., without meta-analysis) and 12 (11.5%) performed a meta-analysis. The average quality score of before-SRs and after-SRs was 6.2 ± 1.8 and 5.0 ± 1.9, respectively, a statistically significant decrease (p < 0.001). Conclusion: Overall, the methodology and reporting quality of after-SRs based on the FFC were poorer than those of before-SRs. In particular, there were very poor descriptions and/or implementations of study selection and data extraction, search strategy, evaluation methods for risk of bias, assessment of publication bias, and formulating conclusions based on methodological rigor and scientific quality of the included studies.