Carrying out a GenAI-assisted scoping review - key lessons learnt

This work presents our findings from a recent work where we compared human decisions with those made by GenAI at different stages within a scoping review. The chosen area of the scoping review was related to teamwork and regulation of learning. The process of carrying out a scoping review and indeed a systematic review is usually quite involved. It often uses a team of researchers to enhance the reliability of the work. This can be time consuming and repetitive, which makes this ideal workflow to be ethically and responsibly offloaded to GenAI. We decided to help build trust in GenAI within this area by first comparing the decisions made by humans and GenAI objectively. Once we were satisfied with the reliability of the tool, we then chose to use GenAI to augment our work with the speed GenAI can bring into the workflow. We will present the results from the reliability testing to compare general GenAI use vs human interrater reliability.