Jasper Wijnen: LLM Participation in Online Research: How Worried Should We Be?

Large language models (LLMs) are no longer just tools used by researchers; they are increasingly “present” on the participant side of our online studies. Participants may outsource entire surveys to LLMs or use them more subtly for translation, paraphrasing, or generating open‑ended responses. This raises fundamental questions for the validity, reproducibility, and credibility of our findings, especially in fields that rely heavily on online data collection.

In this talk I will first sketch the emerging risk profile of LLM participation in online research, using a recent incident in our FMG subject pool as a concrete example. I will distinguish between full LLM delegation (synthetic respondents completing studies end‑to‑end) and LLM‑assisted participation (human respondents whose answers are partially generated or shaped by AI), and discuss why both forms are problematic in different ways.

The main goal of the session, however, is not to provide a finished solution, but to learn from your practice:

• How often do you suspect LLM involvement in your own studies?
• Which signals or checks do you already use?
• How serious do you judge the risk for your type of research?

We will collect experiences, map disciplinary differences in risk perception, and identify promising mitigation strategies. Finally, I will outline possible follow‑up steps (e.g. a faculty working group and shared guidelines) and invite interested researchers to become involved in shaping our collective response.