Learning with AIEducation quality
Feedback Companion and the AI Act: Not high risk, but great opportunities
With the arrival of the European AI Act, questions are being raised about the impact of this legislation on AI systems in education. Innovative tools such as the Feedback Companion, designed to strengthen learning with targeted feedback, raise the question: does this tool fall into the “high-risk” category? And what does this mean for schools and institutions? In this blog I explain why the Feedback Companion is not considered a high-risk application, and I reassure education professionals with facts and evidence.
What are high-risk AI systems under the AI Act?
The European AI Act takes a risk-based approach to AI systems, subjecting high-risk applications to strict obligations. In education, for example, the following systems fall into this category:
- Admission systems: Systems that determine who is admitted to programmes.
- Assessment systems: Systems that automatically assess learning outcomes.
- Adaptive learning tools: Systems that dynamically adapt to a learner's needs and performance (European Commission, 2024).
These applications are considered high-risk because of their direct influence on decisions that affect students' learning process.
The Feedback Companion, however, works differently. It is a supporting tool that helps teachers and students generate feedback and feedforward. It makes no decisions about assessments and has no autonomous influence on learning outcomes. This difference is crucial for its classification under the AI Act.
Why the Feedback Companion is not a high-risk application
The Feedback Companion does not fall within the definition of a high-risk application, for the following reasons:
- A supporting function, not decision-making
The Feedback Companion offers suggestions for feedback and feedforward based on predetermined learning objectives or submitted work. These suggestions are always validated and personalised by the teacher. There is no autonomous decision-making that directly affects a student's learning path or assessment (Hattie & Timperley, 2007). - Human control remains leading
Under the AI Act, human oversight must be guaranteed for AI applications in sensitive sectors such as education (European Commission, 2024). The Feedback Companion functions as a tool that supports the teacher, not one that replaces them. This means that final responsibility for assessments and feedback lies with the teacher. - No dynamic adaptation of learning paths
Unlike adaptive learning tools, the Feedback Companion does not automatically adjust a student's learning pathway. It only provides additional insights based on static criteria, without autonomously influencing the learning process.
What does this mean for schools?
Although the Feedback Companion is not a high-risk application, schools still have responsibilities:
- Privacy protection: Data processing must comply with the GDPR, including strict safeguards for data security and transparency about data use. The Feedback Companion runs on European servers (for example Azure) and complies with GDPR guidelines.
- Training and guidance: Teachers and students need to understand how the tool works and what its role is in the learning process. This falls within schools' normal training obligations under the AI Act.
- Regular evaluation: It is essential to check regularly whether the tool is functioning as expected and whether users are satisfied with the results (Sluijsmans & Segers, 2018).
Handling AI carefully and with confidence
Implementing AI in education raises understandable questions, but it is important to distinguish between systems that actually carry risks and tools such as the Feedback Companion. Because of the tool's supporting nature, its focus on human control and its compliance with privacy rules, the Feedback Companion is not a high-risk application.
Schools can continue to innovate and benefit from AI tools with peace of mind, as long as they use them in a transparent and responsible way. With the Feedback Companion, the strength lies in the collaboration between people and technology, exactly as the AI Act intended.
Curious how the data flows work?
Then check out this knowledge clip:
Sources
- European Commission. (2024). AI Act: Comprehensive framework for trustworthy AI. Retrieved from https://eur-lex.europa.eu.
- Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112. https://doi.org/10.3102/003465430298487.
- Sluijsmans, D., & Segers, M. (2018). Toetsrevolutie: Naar een feedbackcultuur in het hoger onderwijs. Culemborg: Phronese.
- EU AI Act Compliance Checker. (n.d.). EU Artificial Intelligence Act. Retrieved from https://artificialintelligenceact.eu/assessment/eu-ai-act-compliance-checker/
