QUESTION:
Why are federal officials allowing states to “get by” when implementing NCLB? Aren’t a number of states being permitted to dodge this federal law’s impact by employing some fairly slippery data analysis procedures?

ANSWER:
    When I first realized that federal officials were permitting state officials to employ some fairly suspect analyses and reporting procedures, I concluded that those federal officials had been hoodwinked. As time goes by, however, I find myself wondering if states are being allowed to dodge many NCLB bullets simply because U.S. Department of Education leaders don’t want too many U.S. public schools to fail. I really don’t know what’s happening when federal personnel are tough as nails on some aspects of the law and tough as Silly Putty on others. (04/04)

QUESTION:
What are “confidence intervals” and why is our state using them to determine if a school has made adequate yearly progress?

ANSWER:
   A confidence interval is a statistical technique that allows us to estimate the accuracy with which a sample-based statistic (such as a mean) represents what would be found if the mean for an entire population had been computed. The mean of the population, incidentally is called a “parameter.” Confidence intervals are frequently seen on television news reports when a sample-based poll of citizens’ opinions indicate that 54 percent of registered voters prefer Candidate Jones—with a plus or minus error margin of, say, 3%.
   A 95% confidence interval, for example, defines a range above and below the mean of a sample to indicate that if another sample were to be drawn from the very same population, there is a 95% likelihood that the second sample’s mean would fall within the plus-or-minus confidence interval surrounding the first sample’s mean.
   Many states, however, are employing confidence intervals in an obvious attempt to reduce the number of schools who are failing to make adequate yearly progress (AYP) because of low test scores. Their argument goes something like this: “Because of the substantial imprecision of sampling (and educational measurement in general), we are going to allow our otherwise AYP-failing schools to be considered to have satisfied AYP if their students’ test scores are not so low that those scores fall below our state-set confidence intervals.”
   And, because many states are establishing 99% confidence intervals—very wide intervals, indeed—a great many schools that would other wise be regarded as AYP-losers are allowed to escape that label. The problem with this sample-based statistical procedure is that there is no sample involved! When this year’s students in P.S.104 score poorly on their state’s NCLB tests, those students aren’t a sample of any group. They are that year’s P.S.104 population. Although the proponents of confidence intervals for NCLB analyses claim that there is, in fact, sampling involved, I just can’t buy their argument.
   To me, the use of confidence intervals is an obvious tactic intended to allow schools whose students score poorly on state NCLB tests to escape AYP-failure. It is, in my view, based on the specious application of a sensible statistical concept in situations where it doesn’t belong. (04/04)

QUESTION:
Aren’t the recent relaxations in some NCLB requirements indicative of the U. S. Department of Education’s willingness to be flexible regarding this law’s implementation?

ANSWER:
   Flexibility is one of those positively loaded words that invariably evokes applause. However, in the recent NCLB changes that we see being made by Secretary Paige and his associates, it seems that the feds are fussing at the fringes of the law, and not confronting its root problem. As the law now stands, the possibility of most schools attaining their “adequate yearly progress” (AYP) goals for the next few years is almost zero. The current demands for regular jumps---staggering jumps---in the number of students earning proficient-level test scores are altogether unrealistic. That’s the NCLB shortcoming that most shrieks out for a remedy. Flexibility is fine, but if it only deals with the edges of the law, then NCLB’s core doesn’t get fixed. (04/04)

QUESTION:
Isn’t it likely that those teachers who most want to work with special populations of students will, because of the NCLB attention to the need for AYP on the part of such youngsters, end up being punished when those special students cause a school to be labeled as a failing school?

ANSWER:
   Regrettably, that’s what is likely to happen in many settings. For me, special education teachers tackle an education task that I personally couldn’t pull off. Accordingly I have immense admiration for them. But, because their students are often going to push a school into the NCLB loser-column, those very teachers are going to be viewed negatively. (04/04)

QUESTION:
Isn’t the No Child Left Behind Act a thinly disguised attempt to make public schools look bad so that, in time, our nation’s public schools will simply disappear?

ANSWER:
   I’ve heard that sort of speculation expressed by a number of folks whose judgment I respect. However, I just don’t buy into the “NCLB-Conspiracy” theory. Think back to when this law was first passed almost 40 years ago. The intent of the Elementary and Secondary Education Act of 1965 was to improve the caliber of public schooling being delivered to historically underserved groups of students. It represented a heavily funded federal attempt to help the nation’s public schools, not harm them.
    I grant that the altogether unrealistic requirements of NCLB for substantial across-the-board jumps in students’ test scores each year might lead one to the view that the law was designed to destroy our public schools. Yet, I continue to believe that the architects of this most recent reauthorization of ESEA, passed with immense bipartisan support, were well intentioned. They were, unfortunately, way off base when they established their expectations for adequate yearly progress (AYP). Indeed, I am confident that in a few years those AYP requirements will be made more realistic. America’s citizens will not accept the idea that half or more of our public schools are doing a rotten instructional job. At that point, it will become apparent that the framers of NCLB missed the mark when they laid out their AYP demands. That part of NCLB, I am confident, will be modified. (11/03)

QUESTION:
Why has our state approved such an enormous number of content standards and benchmarks? Most of our state’s elementary teachers, when one considers all the subjects they must teach, are supposed to get their students to master more than 50 separate benchmarks each year!

ANSWER:
   The profusion of content standards and benchmarks currently found in many states represents a serious shortcoming in the way we are trying to educate our students. Too many curricular aims, because they almost literally overwhelm teachers, are downright dysfunctional.
     Regarding the reason for this curricular calamity, I suspect my answer may surprise you. I believe the excessive numbers of content standards and benchmarks we find almost everywhere are attributable to a very human motivation, namely, the drive for self-concept enhancement.
     You see, in every state that I know about, the state’s official content standards or benchmarks (the knowledge and skills we want students to learn) have been chosen by subject matter specialists in a particular field. For instance, state authorities select 20-30 of the state’s strongest mathematics teachers and curriculum experts, then ask those individuals to identify a set of curricular aims that must be approved by the state’s legislature or the state school board. Now these subject-matter specialists are just that---specialists. Because such specialists invariably regard their subject as important, they often recommend that the state’s students master just about every skill or chunk of knowledge contained in that subject. The net effect is that in many states there now exists an unwieldy array of far too many skills and much too much knowledge---either to be taught or to be tested.
     Human beings want to be regarded positively. Just about all the people I know try to enhance their own self-esteem whenever an opportunity to do so presents itself. So when subject-matter specialists aggrandize their own subject, the one with which they are personally identified, they are also, very subtly, aggrandizing themselves. We can understand, therefore, why many states have endorsed hundreds and hundreds of curricular aims. But that doesn’t make it right. (11/03)

QUESTION:
In our school district there is so much pressure on teachers to raise their students’ test scores that many teachers, in desperation, resort to drill, drill, and more drill. Shouldn’t we be concerned that such excessive drilling will drive out the genuine joy children should experience when they learn?

ANSWER:
    Your question implies that our frenzy to boost children’s test scores may, in fact, be promoting negative attitudes toward learning on the part of our students. Unfortunately, you couldn’t be more right. If our public schools turn out a generation of students who abhor learning, how tragic that would be. And yet, if things keep going the way they are currently seem to be, that’s just what might happen.
     This problem can be addressed, at least in part, by routinely assessing students’ attitude, specifically, students’ attitudes toward school or toward the act of learning. The measurement of student affect is a low-cost, non-invasive activity and can, if carried out properly, yield some hefty instructional dividends. I try to describe in most of my recent books how student affect ought to be assessed.
     One school district in New York, Bethlehem Central Schools, has recently been attempting to enhance its instructional program by routinely assessing several important student affective variables on a first-of-year and end-of-year basis. If educators in the Bethlehem schools discover that students’ attitudes toward learning are not sufficiently positive, then those educators can initiate activities to foster more positive attitudes.
     I wish more educators were doing something similar. In this test-dominated era, attention to students’ affect is needed more than ever. (11/03)

QUESTION:
Most of the teachers in my school don’t know the first thing about educational measurement. However, our professional competence is now being judged chiefly on the basis of students’ test scores. What can we do to get more knowledgeable regarding educational testing?

ANSWER:
    You accurately identify a huge gap in most teachers’ professional knowledge. (I could have said a “huge lacuna,” because I rarely get an opportunity to use that ritzy word---nonetheless, I resisted the urge---almost.) How can members of a profession remain ignorant of the fundamental means by which their professional competence is being judged? Yet, only a dozen or so states require teachers-in-training to complete a formal course in educational assessment. So the absence of much measurement moxie among educators shouldn’t surprise us.
     Now, however, we need to alter that situation. If your district’s or state’s officials aren’t promoting an effective program in assessment literacy for teachers (and administrators), then maybe you need to take the initiative---personally. Fortunately, there are a good many books and videos out there now that will help. Some are listed on this web site. Others can be found on the internet by using any reasonably effective search engine. Given the right materials and a big chunk of motivation, you really can change the assessment-literacy situation in your school. You really should. (11/03)

QUESTION:
Our state is developing new NCLB tests, but the tests will be built by an external contractor, one whose dominant test-development experience has centered on the construction of nationally standardized achievement tests. Won’t they simply build traditional, same-old, same-old norm-referenced tests for us?

ANSWER:
    Realistically, given the enormity of the nation’s current need for new NCLB achievement tests, it is certain that the bulk of those tests will be built by commercial test-development firms. It is also true that those firms will have a tendency to tackle test-development in the ways that they have historically built achievement tests. However, that need not be the case. Commercial test-development companies have the capability to develop more instructionally supportive NCLB tests (or, if not, can hire the sorts of instructionally savvy people whom they need). The way to make new test development work is to issue a request for proposals that calls for more suitable NCLB tests, and then select an external vendor who promises to build such tests. After that, of course, state authorities need to be particularly vigilant to make sure that what finally emerges will not be same-old, same-old achievement tests. (11/03)

QUESTION:
What sort of NCLB tests would give our state’s educators even a remote chance of satisfying our annual adequate yearly progress requirements?

ANSWER:
    NCLB tests that would allow a state’s educators to actually make measurable adequate yearly progress, at least for the next several years, have three essential attributes. First, they must measure students’ mastery of only a modest number of particularly significant skills or bodies of knowledge (so as not to overwhelm a state’s educators). Second, whatever is to be measured must be described in clear, teacher-palatable language so that teachers know where to aim their instruction. Third, results of the tests must be reported so that a teacher can determine which curricular (assessment) targets have and haven’t been mastered by individual students (thereby allowing the teacher to modify any ineffective instructional approaches, while retaining those approaches that have worked well). Such NCLB tests can properly be described as instructionally supportive accountability tests because, in fact, they are. (11/03)

QUESTION:
What can we do if our state’s NCLB tests are not instructionally sensitive, that is, if those tests are not likely to show instructional improvement even if our teachers are doing a better job in their classrooms?

ANSWER:
    If you teach in a state where the tests chosen for NCLB are instructionally insensitive, you are facing a genuinely untenable set of circumstances. About the only two courses of action that I can recommend are (1) try to replace those insensitive tests with better ones and (2) mount a meaningful public information effort to let the world know that teachers’ educational success is being judged with the wrong measuring sticks. I wrote an article for school leaders dealing specifically with this sort of situation. The article is in the December 2003 issue of The School Administrator. I thought it was a pretty good analysis, so I recommend it. However, I don’t think I’ve ever written an article that I didn’t regard as pretty good, so you probably should discount my appraisal of the article.(11/03)