EdTech impacts on learning can be broad and long-term. Here’s how you can measure those impacts over time.
Education has been forever changed by the introduction of technology. There is no question that students are learning in entirely new ways, but the question most often asked is: how can we measure the impact of technology? In this post, we’ll talk about measuring the correlation between edtech and learning as well as other important outcomes to consider, from digital citizenship and inclusiveness to important real-world skill development. We collectively call all of these outcomes a Return on Education (RoE).
Before we talk outcomes, first we must begin with a vision; simply deploying devices or apps will not guarantee positive outcomes. Pedagogy is the foundation of delivering a solid RoE for any edtech investment. Districts require clear vision and leadership on how teaching and learning are supposed to change, and how best to support that vision at both the IT and classroom level with the right technology and training.
Among the top reasons edtech projects fail are:
- Device purchases fail to meet curriculum needs
- Infrastructure limitations (bandwidth, Bluetooth, home connectivity)
- Deployment and management issues
- Lack of leadership on educational vision
- Stakeholders are not engaged beyond the initial launch of new devices
- Devices are used passively within the classroom (a failure to shift to new pedagogy models)
Once we’ve established a clear vision and plan to support new technology, we can then talk outcomes.
The Impact of Digital Learning on Achievement
Test scores. This is where most parents and school districts want to see mobility as a result of edtech purchases. Are these expensive devices, gadgets, or applications improving test scores? First, as a district or school, you must decide what quantitative or formative metrics matter the most and what percentage improvement would constitute success.
For many schools, measurements will include Measures of Academic Progress (MAP) assessments or newer assessments such as Edulastic – typically focusing on math and reading / language skills. Assessments may include a pilot evaluation stage or a comparing the results of students from one year (pre investment) to the next, noting that such evaluations may include other socioeconomic variables.
Although achievement is important, the impact of technology must be more broadly understood and measured. In many causes, these quantitative gains will be seen in the short-term and will level over time. As such, it’s important to look at other qualitative outcomes and consider the long-term outcomes associated with edtech investments.
EdTech Adoption
A critical metric of success is its actual adoption: are teachers and students using the purchase? Has there been a functional shift in pedagogy? After all, the expected use of a tool may not align with its use in the classroom, so soliciting qualitative feedback from teachers and students can help assess if a tool is useful, relevant, and engaging. Such feedback may determine if technology should be reconsidered or if there are areas of improvement to increase the RoE.
In measuring edtech adoption, look to feedback from all areas: administrators, teachers, students, and IT. We should look to the cost of edtech – purchase, maintenance, and end of life – and the experiences of those on the ground. We can ask if the technology has changed how we teach or learn, if students are engaged, if students are showing initiative or critical thinking skills, if using the product is easy, or if expectations are somehow being unmet.
Student Engagement
There are other quantitative measurements you can consider beyond math or reading scores such as number of assignments completed on time, number of books read (if using e-reader or reading apps), graduation rates, participation rates, or attendance. For example, one high school district in California used mobile technology to track student whereabouts, leading to a 15% increase in attendance and a 31% GPA increase. In another instance, technology-enabled gamification has helped engage and motivate students, reducing truancy rates.
There are many innovative uses of technology to boost student engagement. Even at the most basic level, laptops and mobile devices have demonstrated improvements in engagement and output for many students. Combined with innovative applications, in-class polling, new technologies such as VR, or independent STEM tools, today’s digital learning environment is offering more ways to engage students and keep tabs on learning than ever before.
Personalized Learning and Skill Development
Technology offers the opportunity to boost engagement and to support personalized learning, promoting greater agency, critical thinking skills, and creativity. The goal of personalized learning is to challenge and support each child at his or her own level, which can be difficult in a standard pedagogy model that often ends up “teaching to the middle.” Using a variety of digital tools, educators can deliver instructions in multiple formats, supported by hands-on activities and ongoing feedback to allow teachers to assign lessons according to student needs.
If we look beyond math and literacy, we can evaluate the development of other soft skills that are essential for success in tomorrow’s workforce. At the top level, this includes the use of technology or cutting edge applications in virtual reality or augmented reality. But on a deeper level, technology has shifted how we learn and how we think, moving from fact retention to hands-on experiential learning.
“The power of edtech is not the promise of increased efficiencies or about squeezing marginal outcomes for students, the real opportunity for districts is the potential for a very positive transformation for teachers and students – one the enables lasting change, one where the connections between edtech, pedagogy, and outcomes are clear.” – Ben Politzer, Education Elements
Can you measure the short or long term benefits to these changes? There’s no question that skill development will impact academic performance, but the long-term outcome of critical thinking? Some outcomes are best aimed for, not measured.
Digital Citizenship and Equity
The most recent guidelines for education leaders highlights the International Society for Technology in Education (ISTE) focus on equity and digital citizenship. The standards challenge education leaders to ensure all students have skilled teachers who actively use technology to meet student needs, to ensure all students have access to the technology and connectivity necessary to participate, to model digital citizenship to contribute to positive social change, and to cultivate responsible online behavior.
Digital equity is a complex issue that needs to be addressed at many levels, but is nonetheless an important outcome to consider with edtech investments. While you can measure who has access, and whether that connectivity is being addressed at home to reduce the homework gap, how can you measure digital citizenship? The lack of measurement does not negate the importance of aiming toward this outcome.
For those districts focusing on these “soft” skill developments or non-cognitive outcomes, there is little way to measure the success of these edtech investments. As noted by Alexandra Resch for EdSurge, data may be self-reported by teacher and students on surveys or include looking at attendance or disciplinary incidents. There are new tools coming on the marking, such as the Edtech Rapid Cycle Evaluation (RCE) Coach, a free, online tool that helps school districts choose the best tools for their needs, helping define desired outcomes, design effective pilots, and collect and analyze the results to create a truly informed decision-making process.
Overall, a theme emerges around edtech about the importance of technology, but also about vision and implementation, supported by the necessary tools and training that will result in positive outcomes. As outlined in this article, infrastructure, deployment, and management needs can result in a failure of edtech projects to realize their potential. If you would like help supporting your edtech initiative, we can help.
Contact us to find out whether FileWave is a fit for your team. Request your 30-day free trial now.
