Assessment Testing: In Their Hands

Assessment in K-12 education is nothing new. Summative assessments-benchmark evaluations conducted at the end of an academic year-have been around for years in the form of big, scary exams. On the other hand, formative assessments, tests that monitor student performance throughout a portion of the curriculum are relatively new to this decade. In the late 1990s, academic publishers introduced computer-based assessment tests to be administered multiple times during a school year. Since then, interesting trends in elementary assessment have developed: Grades 4-12 have emphasized summative but not so much formative assessments, while grades K-3 have focused more on formative.

Today, the bleeding edge of assessment is written for personal digital assistant (PDA) operating systems and is administered on handheld technology: various devices from Palm, Hewlett-Packard, and others. According to Naomi Hupert, senior research associate with the Education Development Center in Newton, MA, the approach monitors students on elements they need to master in order to move on to the next level of a particular subject. Hupert says formative assessment is a perfect way for teachers to catch and address gaps in a student’s knowledge or learning, early on. To this end, she adds that the technology provides teachers with the opportunity to teach more effectively.

“If there’s a content area where there is some underlying knowledge, students need to know it and have it available to them,” says Hupert, who also serves as a Reading First evaluator in New Mexico. (Reading First is the US Department of Education’s nationwide effort to enable all students to become successful early readers.) “The process of formatively assessing those students can become a valuable tool for teachers so they can be sure that the baseline of knowledge is being used appropriately by students.”

As Hupert suggests, handheld formative assessment systems don’t only assess student progress; the tests also can provide subtle evaluations of curriculum effectiveness. Here, teachers can use the assessment technology to gauge how well they’re teaching to student needs over time. If, for instance, an October assessment reveals a particular student is at risk of failing to grasp a particular subject, a teacher can reassess that student in December to see if changes in the curriculum have helped. If the student shows progress, the teacher can note what kind of approach to learning works best. If the student demonstrates little to no progress, the teacher knows it’s time to try something new.

Still, handheld-based formative assessment systems are not without a little controversy. For starters, some teachers say that the technology is disruptive, cutting into valuable classroom time that teachers could spend teaching. What’s more, because the latest formative assessments are generating precise and transparent data, many school districts see the tools as panaceas for assessment needs as a whole, and treat the formative tools as if they are more summative than they really are.

“At the end of the day, the truth is that formative assessment is larger than merely assessing more frequently,” says Rick Stiggins, CEO of the Assessment Training Institute in Portland, OR.

The trailblazer hands-down, Wireless Generation is the market leader in handheld-based formative assessment systems. The company was founded in 2000 with the goal of putting formative assessment into the hands of teachers and getting them away from more cumbersome materials such as desktops and paper. Today, the firm’s mCLASS (mobile classroom assessment) technology is used to assess 10 percent of the K-3 population in 48 states. Half of the Reading First students across the country are being assessed using its software. The company’s products constitute the official Reading First solution in 16 states.

Wireless Generation uses fixed, or preset, assessments to monitor student progress in two main subjects: reading and math. Reading is by far the larger of the two markets, with products specifically designed to assess pre-kindergarten literacy as well as K-3 student performance. This is accomplished via early-reading standards such as Dynamic Indicators of Basic Early Literacy Skills (DIBELS); Phonological Awareness Literacy Screening (PALS); Texas Primary Reading Inventory (TPRI); and El Inventario de Lectura en Espanol de Tejas (Tejas LEE ), which establishes baselines for reading comprehension in Spanish. Math products, set to be released later this year, will cover early numeracy, operations, and all the basics of algebra.

Whatever subject an assessment is evaluating, assessment systems from the predominant vendors work in pretty much the same way. First, a teacher administers a paper-based test and assesses student performance on the test using a handheld computer. When the assessment is complete, the teacher uses the device to look back at student performance as a whole. Next, the teacher puts the handheld device in its cradle and “syncs” all applicable data to the Internet. Finally, the teacher, coaches, and relevant administrators can log in through secure, password-protected access to view a variety of data reports and analyses and use instructional planning tools on a website.

“When you get data to teachers in real time, a bunch of things change,” says Larry Berger, CEO of Wireless Generation. “The entire learning process becomes more responsive in a way that motivates kids to get to the next goal.”

This is exactly what has happened at the Orange County PSD, where, at the start of the 2003 school year, district officials signed up for the DIBELS assessment software from Wireless Generation. The product, dubbed mCLASS: DIBELS software, offers teachers a specific K-6 assessment with progress-monitoring questions that meets requirements established by 45 Reading First states. Baldwin, the district’s director of assessment, says teachers in most of Orange County’s Reading First schools are required to utilize the technology four times a year, while those in other schools must use it three times. So far, he adds, reading scores have improved across the board.

Products from Texas-centric Tango Software have yielded similarly encouraging results for the Rio Grande City Certified Independent School District (TX). There, Technology and Instruction Director Vilma Garza says the district has used Tango products since 2002 to facilitate huge jumps in reading ability. During that time, Garza says the district went from having 33 percent of third-graders pass the TPRI reading exam, to a 90 percent pass rate. Rio Grande City CISD teachers have embraced the technology so completely that Garza notes that the tools have changed the fabric of evaluations themselves, making tests something students actually look forward to.

“What once took weeks of manual tabulation [is] now available to us as soon as teachers tap ‘finish’ on the handhelds,” she explains. “Our students love being able to use technology to take their assessment. Can you imagine students being excited about testing?”

Liberty Source, the firm that makes Tango, offers two distinct solutions: Tango RX and Tango Suite. Both of these solutions can be preloaded with benchmarks to assess student compliance with TPRI and Tejas LEE, to name two. The big difference between Tango’s tools and Wireless Generation’s offerings is approach: While Wireless Generation focuses on improving the process of giving fixed, proven assessments and helping teachers and administrators to understand and use the data, Tango emphasizes flexibility by enabling teachers to load any other assessment whatsoever. Edward Barerra, Liberty’s president, says that neither of the Tango solutions differentiates between formative and summative assessment because the company wants to enable teachers to collect any kind of information they want.

This open-ended approach enables teachers to personalize the assessment experience in any way they see fit. Many of them, including Garza at the Rio Grande City CISD, author items for their day-to-day instruction, reword questions, or upload entirely different types of assessments-even summative ones, if they so desire. According to Barrera, the idea behind all of this is to maximize flexibility. Barrera says that by providing educators with these kinds of options within a set of basic parameters, Tango improves upon the likelihood that teachers actually will do something with assessment data once they administer the tests.

“Assessment technology needs to align with a teacher’s personal style of instruction,” he says. “In some instances, that’s not the case, and we need to make sure we help them incorporate our products into the way each and every educator runs [his or her] classroom.”

While both of these formative assessment models work well, experts say that K-12 school districts must overcome some critical challenges surrounding the technology before it enters the mainstream. First, of course, is teaching teachers how to use it. In many districts, teachers are just getting accustomed to having desktop computers in the classrooms, and teaching them how to administer assessments on handheld devices is an entirely different ballgame. In Rio Grande, teachers are required to take workshops on the new technology; at Orange County PSD, the training is ongoing, as 25 teachers run refresher courses in the technology throughout the year.

Once teachers learn how to use the handheld technology, perhaps the biggest challenge is getting the educators to actually incorporate it into their curriculum. Stiggins, the CEO at Assessment Training Institute in Portland, says the issue here is contextual communication — finding formative assessment systems that communicate data in context so that teachers know how to use it. Stiggins says the key to ensuring this critical step is a worthwhile reporting system that involves both teachers and students in a continuous process.

“If a formative assessment system generates information about student achievement, it needs to include within the system effective ways to communicate results,” he says. “The best technology and most accurate assessments in the world are wasted if the results aren’t communicated in a way that can be used to improve learning as a whole.”

Wireless Generation’s Berger calls this the “now what?” syndrome, explaining that once teachers have identified what formative assessment is, and why it’s important, they must learn how to contextualize the data it provides. Berger lauds the use of established and validated assessments that enable educators to make “apples to apples” comparisons of data over time, and says that with these tools teachers can visualize student progress, customize instruction, and help group students according to needs. He adds that his firm currently is working with publishers of instructional programs to integrate its mCLASS assessments products with other curriculum and intervention offerings.

Hupert, the researcher at the Education Development Center, likes this strategy, going so far as to say that no assessment is worthwhile without a valid plan. Educators who prefer a heavy phonics approach to literacy may choose an assessment that emphasizes knowledge of basic phonics in early reading; those educators who prefer to focus on comprehension skills may choose an assessment that includes more emphasis on students reading from texts. Hupert says the bottom line is that a district’s formative assessments shouldn’t be plucked out of a pile and plugged into a program, but rather selected and administered carefully to meet specific achievement goals throughout the year.

“No matter which technology you choose — handhelds, desktops, laptops, whatever — all assessments should reflect student need,” she says. “We need to rethink the way we utilize testing across the educational system, understand that different students require different kinds of teaching, and come to accept that formative assessment is the only way to make the learning process reflect how students really learn.”