Feedback Prize by The Learning Agency Lab
The Problem
- Most high school graduates are not prepared for the writing demands of college or the workplace.114
- Difficulties in evaluating writing are reinforced by the fact that teachers aren’t given enough resources and time to provide the feedback necessary to help students become strong writers. Moreover, they are overburdened by routine duties such as grading essays.
- For students, the lack of opportunity for practice with useful, timely, ongoing feedback tied to instruction leads to unskilled and painful production, procrastination, anxiety and fear.
CZI Support
In 2019, The Learning Agency Lab partnered with Georgia State University and Vanderbilt University to launch The Feedback Prize. The project leveraged artificial intelligence to develop free tools to help teachers supplement their feedback to students struggling with writing by providing more frequent and automated feedback, in addition to that given by teachers directly. Working with teachers, students and researchers, the team gained a detailed understanding of the challenges teachers and students face with writing. They collected data and launched an open competition to develop tools that could help provide high-quality feedback from educators reliably and without bias. The team also worked to create tools that will help English language learners receive more accurate feedback on their language development and expedite the grading cycle for teachers. The new tools developed through these projects will be free and open source.
The Impact
- A case study of the Feedback Prize showed that teachers must be part of the development of assisted writing feedback tools — from data collection to evaluation of performance — to ensure writing tools are successful.115
- While technology alone is not enough and these innovations are best viewed as tools to support teachers rather than a cure-all, they can be leveraged to help teachers have an even more significant impact on students.
- The team gathered 26,000 argumentative essays submitted by students, which were annotated for writing elements. This is the largest, publicly available annotated dataset of argumentative writing. Given the dataset’s size — and the inclusion of related demographic information — this dataset will greatly advance the field for academics, practitioners, educators and students. Specifically, this data will be used to improve automated assessment tools for writing to ensure that these tools are more accurate, less biased and student-centered.
- The project team hosted three Kaggle machine learning competitions that produced open-source algorithms that are readily transferable to educational ecosystems. The winning models achieved an accuracy rate of 75% — the same accuracy as teachers who reviewed and scored those materials.
- Finally, the project team published five academic publications, conducted eight academic presentations, and had more than 12 popular articles published about this work. 116
Founded:
2019
Website:
Mission:
The Learning Agency Lab is a nonprofit that seeks to improve the effectiveness of K-12 and post-secondary education — especially for historically marginalized populations — by creating free or at-cost resources that are aligned to the sciences of learning.
“With the very generous support of CZI, our team in collaboration with professor Scott Crossley has been attempting to create more effective, accessible, and equitable assisted writing feedback tools.”
