Photo credit: SVETA/STOCK.ADOBE.COM.

A mere half-century ago, America’s public schools were widely regarded as a successful strategy for inducting the nation’s youth into adulthood. Back then, indeed, our public schools were thought to be one of America’s crowning societal accomplishments—providing students with pathways to attain their personal incarnations of the American dream. Today, in contrast, many observers of the nation’s public schools register substantially less enthusiastic appraisals of our schools.

It is possible, of course, to identify many possible culprits contributing to this perceived drop-off in school quality. Fortunately, however, we also can trot out several tactics for rectifying such shortcomings. In the following analysis, however, only one reason for reduced school quality will be identified—and only one strategy will be recommended to wrestle that reason to the mat. It will first be argued that our schools’ success is lowered because of uninformed decisions based on students’ test performances. It will then be contended that an increase in assessment literacy represents our most cost-effective way to improve American schooling.

Defining assessment literacy

Before considering a strategy to promote increased assessment know-how, it is important that we agree on a serviceable definition of the phrase “assessment literacy.” At this moment, there is no single, widely accepted conception of what this label represents. Although most users of the assessment literacy label believe that assessment literacy refers to one’s knowledge about educational testing, the depth, breadth, and sophistication embodied by such knowledge can vary substantially.

To illustrate, I have employed the following definition in my own writing on this topic during the last two decades: Assessment literacy consists of an individual’s understanding of the fundamental assessment concepts and procedures deemed likely to influence educational decisions.

As you can see, my take on assessment literacy calls for it to represent someone’s grasp of those basic measurement understandings that are apt to influence actual educational decisions. And, because I believe that those understandings are relatively few, my conception of the definition’s decision-influencing “fundamental assessment concepts and procedures” is closer to a half-dozen than to a half-hundred. (My preferred set of six assessment-related understandings are presented in my 2018 book, Assessment Literacy for Educators in a Hurry.) Yes, I would prefer genuine mastery of a modest number of assessment understandings so that, for example, a small collection of fully comprehended and internalized assessment understandings will have an impact on real-world educational decisions.

Who, then, should be the recipients of this modest number of fundamental assessment understandings? In other words, who needs to become assessment literate? Well, we would surely want to see assessment literacy possessed by as many potential influencers of education decisions as possible. At the top of any target list, we usually find the teachers and administrators who make a flock of daily educational decisions—many of which are determined by students’ performances on educational tests.

Next would be such educational policymakers as the school board members and legislators whose most important educational choices often hinge on students’ test scores. Then, too, citizens in general—and particularly the parents or guardians of school-age children—have a right to know how well their tax-supported schools are performing. And, finally, we come to the students themselves whose lives are frequently buffeted by their performances on significant educational tests. All these pivotal groups are apt to make more defensible decisions after receiving a serious splash of assessment literacy.

Cost-effective assessment literacy

Expanding assessment literacy, of course, is not the only way to improve U.S. schools. To illustrate, we could dramatically boost teachers’ salaries—hence attract even greater numbers of high-talent teachers to our classrooms. Clearly, a larger pool of stellar teachers would almost certainly trigger improved learning in our schools. Similarly, we could install substantially reduced pupil-teacher ratios in our classrooms and thus allow teachers in smaller classes to engage in more individualized, hence more effective, instruction.

Yet, both of those interventions call for nontrivial financial expenditures. Accordingly, it comes as no surprise that recent decades have witnessed few large-scale efforts to boost teachers’ salaries or to reduce class sizes. At least for the present, society regards such remedies as too costly.

Happily, however, if we were able to enhance the assessment literacy of those who most need it, we could bring about school improvement at a definitely affordable cost—a pittance in contrast to the funds needed for the aforementioned salary-boosts or smaller classes. More assessment literacy, you see, would almost certainly contribute to our making substantially fewer evaluative and instructional mistakes. The learning dividends derived from reduced instructional and evaluative mistakes could be colossal.

Currently, for example, many educational policymakers now incorrectly judge the effectiveness of teachers’ instructional efforts according to their students’ scores on standardized achievement tests unaccompanied by evidence indicating that those tests are appropriate for such an evaluative mission. Many teachers, therefore, end up being mistakenly directed by assessment-unknowledgeable administrators to alter their classroom teaching procedures primarily because of students’ insufficiently improved scores on the wrong tests.

Yet, not all standardized achievement tests can accurately distinguish between effectively and ineffectively taught students. Assessment-literate educational leaders would less often direct the teachers of low-scoring students to abandon what those teachers regard as effective instructional tactics. Assessment-literate educational leaders would realize that determinations about teachers’ skills were being based on data from evaluatively unsuitable tests.

When teachers are urged to relinquish what they regard as appropriate teaching practices because of their students’ scores on the wrong kinds of evaluative tests, this constitutes a serious educational mistake. Students are most likely missing out on their teacher’s best teaching tactics because of someone’s shoddy understanding of the evaluative merits of the tests being used.

As another example, many teachers are currently being asked to arrive at diagnostic judgments about their students’ distinctive strengths and weaknesses based on far too few items measuring those strengths and weaknesses. Assessment-literate teachers would make fewer instructional mistakes after they recognized which tests do and don’t provide them with instructionally actionable results.

If students’ test results are to supply useful guidance for teachers’ next-step teaching moves, the knowledge and skills on the test must be measured by sufficient numbers of items so that teachers can make a reasonably accurate determination about a student’s mastery level of what’s being measured. The necessary number of items on a given test, of course, depends on the specific nature of what’s assessed. Yet, skills or knowledge being measured by only one item (or by a tiny handful of items) rarely provide a teacher with sufficiently accurate results to beget suitable next-step teaching choices.

Ideally, the subset of items deemed adequate to provide action-worthy instructional interpretations for teachers should be conceptualized at grain-sizes conducive to teachers’ routine instructional choices. Yes, this means that experienced educators, those with instructional chops, should play a significant role in carving out the tests intended to assist teachers.

If the purpose of a test—classroom or large-scale standardized—is to help teachers teach better, then every student’s results must be reasonably straightforward for teachers to interpret and use. Ideally, tests aimed at improved instruction by teachers also should be accompanied by at least a minimal dollop of guidance for teachers about possible next-step instructional moves.

Thus far it has been suggested that varied education-related target groups—if in possession of greater assessment literacy—could significantly lessen the number and impact of wrong-headed decisions based on educational tests. How, then, should we go about increasing the degree to which members of such key groups become more assessment literate?

Needed: a national assessment literacy collaborative

Although, during the past decade or so, we have seen scattered efforts to improve assessment literacy—particularly focused on teachers as targets—those initiatives have usually lacked the prestige, punch, breadth, or funding to be genuinely successful. Such well-intentioned attempts to boost the measurement moxie of chosen recipients have, therefore, often fallen short of expectations. It is clearly time for assessment literacy proponents to haul out heavier weaponry.

One promising approach could be centered on a carefully selected set of stewards who are committed to forming and operating a national collaborative of associations and agencies promoting increased assessment literacy among members of the target group(s) on which the collaborative chose to focus.

For example, the leaders of such a national assessment literacy collaborative might promote increased assessment literacy exclusively for teachers, for school board members, or even for members of the public. Funding for such a collaborative could come from either governmental resources or, more likely, from a collection of charitable foundations.

The permutations of possibilities in which such a collaborative might be structured and operate are too numerous to enumerate here. What’s most important is that the organization must be conceptualized so it embodies sufficient clout—namely, perceived eminence, representativeness, competence, and funding to engender a positive response from members of the audiences on which it focuses.

Put simply, the organization being suggested here must be regarded as a potent, properly constituted group that’s capable of accomplishing an important outcome intended to benefit educators and, accordingly, the children our schools serve.

W. James Popham (wpopham@g.ucla.edu) is an emeritus professor at the University of California, Los Angeles.