Monday, February 06, 2006

Teaching students to think outside the standardized test box

Read this article linked here, then tell me if you think this type of instruction is more likely or less likely under NCLB. One could argue that this type of instruction was hard to find before NCLB and you might be right. However, teachers tell us you are even less likely to find such creative, enthusiastic, open-ended activities like this when teachers are overly pressed to "teach to the test."

America, once the world's leader in developing creative entrepreneurs, is rapidly joining the rest of the world in standardized test mania. With the move to spring ISTEP in Indiana we will move from 5 weeks of test review to 5 months of test review. Our kids are so enthused they can't wait!


At Tuesday, February 07, 2006, Anonymous Anonymous said...

Thank you for the article. I will read it shortly --

Two comments/thoughts. I had an opportunity to talk with our school board about our ENDs statements (policy governance - what you want the child to accomplish by the time they graduate). Our ENDs statements lead me to believe our district's goal is to produce good citizens and this is measured by the number of students who register to vote. Big whoop.

My challenge to the board was where are the entrepreneurs, the next generation of leaders, etc. Isn't that what our schools are to produce? Another person has since added the statement of providing students the opportunity to learn to their highest potential.

I do not understand why my district chooses to issue its "knowledge" test at the end of the school year rather than four weeks into the new school year. By testing in early April that leaves 6 weeks for real teaching and real learning to take place, if it does. However, if the test were given at the first part of the year when you are doing heavy review from the prior year anyway, you could get the test out of the way and hopefully real teaching and learning would take place throughout the rest of the school year.

What am I missing?

Thanks for letting me post. I read your blog every day.


At Tuesday, February 07, 2006, Blogger Indiana Public School Superintendent said...


Your question is a good one. Unfortunately it is not often discussed as a philosphical issue but rather in sparse partisanship terms these days. Republican legislators speak of the switch to Spring as more "logical" and "makes sense" which tells the public nothing about the purposes. Democrats are against it because the Republicans are for it. Again, taking sides isn't all that helpful in the debate.

Spring testing emphasizes accountability for students and staff. Fall testing emphasizes diagnosis and remediation of student weaknesses. I am sure someone will plan on proposing a teacher "merit pay" bill using test scores on evaluations next year if their spring ISTEP bill passes.

Do we want to hammer students or help them? Society can't decide. We had a spring test. We switched to fall. We have a fall test. We switch to spring. There is no consensus among educators - mainly because there are reasonable people that understand they both have their good points and bad points.

The argument that the fall test wastes 5 weeks of instruction preparing for the test is somehat silly. The spring test will result in 5 MONTHS reviewing for the test.

If the test somehow reflected the entrepreneurial aims you described to your board, that might be interesting to "teach toward." Spending 5 months preparing for some huge culminating entrepreneurial project integrating math science and soicial studies might be worth wpending time on. I am afraid of what schools will feel they have to give up spending all year preparing for a bubble sheet answer test.

Unfortunately, the only aspects of the current ISTEP test that are worth spending massive amounts of time preparing for are the open-ended applied skills sections that emphasize writing skills.

At least if kids are spending time writing that is a life skill as opposed to practice bubbling little circles.

With the current administrations claims that the Spring test will not cost any (or very much) new money can only be achieved if they drop the applied skills sections and go to a simple on-line or bubble sheet multiple choice format.

Elizabeth, I don't think you missed anything. You have picked up on one of the most critical questions in this debate no one will talk about.

What will our students give up to be successful in preparing them for the spring testing?

We may climb the ladder of NCLB somewhat succesfully only to find it is leaning against the wrong wall.

At Tuesday, February 07, 2006, Anonymous Anonymous said...

Where to begin. Oh, where to begin?

1) How is it that so many people in Indiana tout our state standards, but those same people bash a test of those standards? Schools SHOULD be teaching to our standards. That's why we have them. And they should do that all year, not cram for 5-6 weeks preceding the test.

2) You cannot foster creativity, entrepreneurialship, etc. if you do not FIRST have a good grounding in the basics. Such skills do not appear in a vacuum. They are fostered and built upon a strong base of established knowledge. Unfortunately, there are some people out there who seem to think that fostering creativity will somehow help our current students to discover the next medical cure, even without a solid grounding in chemistry or math. But don't stop there, the examples of such lunacy are endless under this thought process.

3) Super is right in suggesting that this issue has become a political one. Every school adminstrator group in the state has had a position in the past favoring spring testing. They opposed the change when it occured in the mid-90's and they have advocated for a return to spring testing since then -- until recently. So what changed? For one, Gov Daniels, who has also been critical of school adminstrators for spending too much money outside of the classroom, supports spring testing. So, is it purely coincidence that several of these administrator groups have changed their tune from their long-standing previous position?

4) Actually, there IS consensus among the vast majority of educators that spring testing is better than fall. In direct polls of teachers, the percent favoring spring ranges from 70-80 percent. Unfortunately, the outspoken "leaders" of our education system do not reflect that clear and simple fact.

5) It's not the 5-6 weeks of cramming that are most problematic with the current fall schedule. That is bad, yes. But even worse is that the cramming is focused on LAST YEAR's material -- because that's what the fall test covers.

6) It's not the "current adminstration" that claims the test will not cost any additional money -- it's the testing companies, who have provided signed letters attesting to that fact. I guess there are some people who would rather believe Suellen Reed, who initially said it would cost $17 million, then said something closer to $10 million and now says $45 million. Hmmm...

7) No proposal has suggested that the applied skills be skipped in a new test. BUT, according to studies commissioned by Reed, even the current test does not do a good job of testing such skills. Unfortunatley, too many people equate open-ended and essay questions with applied or higher end skills, while suggesting that multiple choice can't do such things. Today, we have a test with both formats that only tests basic skills.

Elizabeth did ask good questions. It's just unfortunate that Super did not have the answers.

At Tuesday, February 07, 2006, Anonymous Anonymous said...

In response the Anonymous, "Where to begin. Oh, where to begin?"

"How is it that so many people in Indiana tout our state standards, but those same people bash a test of those standards? Schools SHOULD be teaching to our standards. That's why we have them."
Yes, let's talk about those Indiana Academic Standards, especially the English/Language Arts standards and the state test. Out of the 7 Standards, three of them deal with writing and the writing process; and one deals with listening and speaking. With the most recent change in the legislation to move the spring test, “for school years beginning after June 30, 2007… during the first two (2) weeks that end in May of the school year.” The bill than requires that: “Test scoring completed before June 16 (and) test results reported to teachers and parents before July 1.” There is no way the writing standards can be assessed (we have never assessed Standard 7 dealing with listening and speaking). That means only the three standards dealing with reading will be assessed. Are you crazy enough to believe that the writing and listening/speaking standards will be emphasized as much as the reading standards if they are not assessed on the state test - that teachers won't feel the pressure to teach to the test?! You and "our (yep, unfortunately, I did vote for him - not in the primaries, however!) man Mitch" need to do a little research on the importance of writing as far as improving students' language arts skills.

"It's the testing companies, who have provided signed letters attesting to that fact" (that the test will not cost any additional money).
Produce those signed letters! Share them with the public!! Newspapers have reported that a testing corporation employee told legislators that he could not guarantee that the test would not cost any additional money. Prove to your public that you have the signed letters stating that. Those of us who have been involved in the testing process have a hard time believing that. The state just finished setting cut scores for Grade 7 science where teachers were involved from throughout the state - working in Indianapolis for two days - cost was involved there! Won't that all have to be redone for all tests with changing the testing date and the test?! The public really doesn't have enough information, and I truly believe the governor doesn't really know what is involved!!! There have been enough other instances where he has shown he really doesn't have a clue about what happens in education/ in our public schools.

Suellen Reed is right, she needs to be allowed to do her job - to do what she is paid to do and elected to do (she did receive more votes than our governor). Governor Daniels needs to listen to her and those in the Indiana Department of Education who work with testing and know what is involved!

At Tuesday, February 07, 2006, Anonymous Anonymous said...

I just opened the standards -- and it took about 30 seconds to find a whole bunch of "writing" standards that could easily be assessed on a test with quick turnaround time. I have pasted the first that I saw -- again, in 30 seconds -- at the bottom of this message. So, frankly, I am finding absolutely no point in your message.

The letters from testing companies have been distributed publicly already. If you have a friend or a representative who works in the State House, then you should ask them for them. If they tell you that the letters are not available, then frankly, they are lying to you. (Unfortunately, there is a lot of that coming from the opponents side.)

Finally, which Suellen Reed should "be allowed to do her job?" The one who announced publicly in a joing press conference with the governor that she SUPPORTS spring testing? Or the one who holds a surprise press conference on the day of a vote? The one who says the transition will cost $17 million? Or the one who says it will cost $10 million? Or the one who says, at a last-minute press conference on the day of a vote, $41 million?

Here's a quick snapshot of the writing standards, which you have interpretted, somehow, as not being assessable on a quickly-returned test. (Do you think these things might be tested on NWEA or a host of other tests that states use annually? Hmmm??):

• 4.6.4 Identify and use in writing regular (live/lived, shout/shouted) and irregular verbs (swim/swam, ride/rode, hit/hit), adverbs (constantly, quickly), and prepositions (through, beyond, between).
• 4.6.5 Use parentheses to explain something that is not considered of primary importance to the sentence, commas in direct quotations (He said, “I’d be happy to go.”), apostrophes to show possession (Jim’s shoes, the dog’s food), and apostrophes in contractions (can’t, didn’t, won’t).
• 4.6.6 Use underlining, quotation marks, or italics to identify titles of documents.
• When writing by hand or by computer, use quotation marks to identify the titles of articles, short stories, poems, or chapters of books.
• When writing on a computer italicize the following, when writing by hand underline them: the titles of books, names of newspapers and magazines, works of art, and musical compositions.
• 4.6.7 Capitalize names of magazines, newspapers, works of art, musical compositions, organizations, and the first word in quotations, when appropriate.

At Tuesday, February 07, 2006, Anonymous Kindergarten Dropout said...

According to the governor, changing the time of Istep will result in "minimal cost." Of course, he said the same thing about the cost of the Iraq war when he was the White House budget director.

Suellen Reed has been on record for some time in favor of keeping the Istep testing in the fall. I believe that she said she supports additional testing during the spring.

If the issue is going to hinge on whether Mitch Daniels or Suellen Reed have the most integrity (which it shouldn't, but Anonymous seems to want to go there), Suellen Reed will win that contest every time.

Unfortunately, when the Istep is given shouldn't have anything to do with which elected official is more popular or trustworthy. But it seems that in a Republican controlled country, everything is personal. I would have never guessed it, but these are not my father's Republicans (nor mine, anymore.)

At Tuesday, February 07, 2006, Blogger Indiana Public School Superintendent said...

Anonymous one said:

Unfortunately, there are some people out there who seem to think that fostering creativity will somehow help our current students to discover the next medical cure, even without a solid grounding in chemistry or math.

Boy, I can't think of any teachers or administrators in my school district who fit into this category. I suppose there are some but they won't admit it in front of me.

Just because we question the way standardized testing is overemphasized doesn't mean teachers and adminstrators don't believe in the basics. Cut us a break.

Where we differ is that policy makers just don't understand the side effects of heavy accountability on standardized tests.

The GOOD side of it is that we are more focused and work harder with those who struggle to keep up. Schools feel much more accountable today then they were 10 years ago. The BAD side is that with limited time and resources people will focus on the TEST not the standards.


What most people fail to realize is that very few of Indiana's standards are actually tested.

The applied skills and writing sections of ISTEP are probably the best parts of the test because writing and problem-solving are "leveraged" skills that provide benefits in ALL content areas. Plus - the writing and problem-solving are very useful in pushing educators to change methodology. Teaching towards writing and problem-solving skills makes sense.

Unfortunately these skills are the hardest to score consistently and the most expensive to assess.

Hence - Indiana is most likely to abandon the parts that have actually had the most beneficial impact on instruction in the trenches if money indeed DOES become a problem.

The standards you pulled down are indeed easy to assess. But - a student could use all those skills and still be a terrible writer. Assessing how well a student frames an argument and writes a persuasive letter using information is more difficult to assess. When educators talk about writing they usually mean more than only grammar and punctuation.

At Tuesday, February 07, 2006, Anonymous Anonymous said...

No argument with Super's last point. But the earlier post argued that there were a bunch of Indiana's writing standards that can't be measured on a test with quick turn-around time. That's just not true.

If you like applied skills, then you DEFINITELY should not like ISTEP in its current format. According to ACHIEVE, a group brought in by Reed herself to evaluate ISTEP, Indiana is not using its extended response questions to measure anything that could not be measured more easily with a multiple choice test. TO BE CLEAR, ACHIEVE LIKES EXTENDED RESPONSE QUESTIONS BECAUSE THEY CAN MEASURE SOME THINGS THAT MULTIPLE CHOICE CANNOT. But ISTEP in its current form does not measure those things. And ACHIEVE did also note, by the way, that multiple choice CAN measure a LOT of applied and higher order thinking skills.

Super is also right about few of Indiana's standards being tested. One analyst has suggested that just 25% are covered. But that is precisely one of the reasons that moving the test from fall to spring does not take much adjustment. We're not measuring every standard up to the last day now; so if we move from fall (which is designed to measure the previous year's standards) to late spring (designed to measure the same set of standards), how much needs to change? NOT MUCH. In fact, Indiana students will probably do better on the SAME content because they will be fresher on it, warmed up to learning (as opposed to coming off of summer vacation), and will have had more time to study the material (since they will not have wasted 6 weeks at the beginning of the year reviewing material from the previous test).

Okay folks. I've answered your questions very directly. So how about answering some of mine?

1) Why is that administrators in this state still think the testing company letters do not exist? Are your representatives being honest with you?
2) How do we get to $45 million and why have there been two other grossly different estimates from the same person? Is one of them a lie? A distortion, if you prefer? Or just incompetence that keeps her from giving a consistent answer?
3) Why have the administrator groups changed their position on spring/fall? They used to be the leaders in calling for spring testing -- and they were very good as communicating all the reasons that spring is better. Could it be politics?

I guess I'll stop there since there really is no hope that even these three will get answered.

At Tuesday, February 07, 2006, Anonymous kindergarten dropout said...

Anonymous, I'm sure you have read the Star article in which Reed provides a (very general) breakdown of the cost of the change. Do you not agree that a Major part of the cost of that Move is due to the fact that whenever the switch is made, the test will have to be administered twice in order to satisfy the feds?
Since I haven't seen the "letter" and you obviously have, is the double up on the testing covered in the letter?

And can you provide documentation of the times and places when Dr. Reed allegedly lied/distorted or whatever when throwing out the $10 million or the $17 million figures? I'm sure the articles are available, but I couldn't find them. It should be no problem for you to provide a link.

As far as the answers to your other questions, you will have to ask someone else. You can't be an "administrator" if you are a
kindergarten dropout.

At Tuesday, February 07, 2006, Anonymous Anonymous said...

Here's the breakdown of Reed's $45 million estimate:

$30 million -- double-counting of the same test. No double-up is needed. Even through the transition, we will only test the progress of students once per academic year. In other words, this "additional cost" does not exist. Indeed, after more than a year of debating the transition costs, this is the first that Reed has ever mentioned it. Do you really believe that she is so incompetent that she missed this expense all this time? Or do you think she is playing games as her opposition becomes more desperate?

$10 million -- pilot of the new test. This is lower than the $17 million that her representatives have estimated in State House testimony, and less than what she apparently told the legislature, according the 1240 fiscal note. But it is still more than the estimates that the testing companies have provided. $10 million more!

$5 million -- addition of new 9th & 10th grade tests. These are tests that are contemplated in 1240, along with a higher-level GQE; but ALL of this is based on changes that Reed has ALREADY committed to doing REGARDLESS of whether 1240 passes. It's all part of the "American Diploma Project."

So, you see how all this stuff gets distorted? Frankly, I don't blame the local admins (too much) for their reverse position on this issue, because I have seen the distorted reports that are put out by Reed and by the administrator groups in the State House. It's a lot like all the scare tactics that the admin lobbyists did on school funding last year -- and which still is cause for a lot of the tension with Daniels. Remember all the cries that schools would get funding cuts? Remember the big press conference in which the admin reps claimed that 10% of ALL teachers in Indiana would be laid off if the budget is passed? Well, the budget did pass -- and 1) education was one of the few parts of state government that got an increase and 2) we now have MORE teachers in the state than we did last year. So, when information like all of this is SO wrong, is that a distortion, a lie or ignorance? You be the judge.

In the meantime, my respectful suggestion to administrators on this list is: stop asking these questions of "anonymous" people on a weblog and start INSISTING on some better answers from your representatives in the State House. It's pretty darned clear from what you are writing that you are NOT getting good information.

At Wednesday, February 08, 2006, Anonymous kindergarten dropout said...

Anonymous, thanks for the analysis of the Star article I referenced. I was hoping you could provide links to articles where Dr. Reed suggested different costs to switch the Istep. It is nice of you to state your opinion, but I think most of us already know what that is. It would be even nicer if you could provide the links or at least some reference where the statements were made.


At Wednesday, February 08, 2006, Blogger Indiana Public School Superintendent said...

I am not sure these are "answers" - so I'll call them "responses." :-)

1) Why is that administrators in this state still think the testing company letters do not exist? Are your representatives being honest with you?

I have no knowledge about "testing letters." I have served in statewide capacities with the individuals involved and have never found them to be anything but forthright and passionate about children. However, I have had teachers report to me who have participated in the cut score setting process and other aspects of the ISTEP testing process and some of what it involves. I do have some experience in assessment and data collection statewide and I find comments like "it won't cost a thing or not very much" to be hard to swallow. The complexities of creating rigorous, reliable,valid, discrimination-free statewide assessments then moving them from one year to another is not as simple as moving your spelling test from Monday to Friday.

The governors acknowledged penchant for "ready, fire, aim" has led administrators in the trenches to wonder if his advisors have "missed the mark" again. That's all.

2) How do we get to $45 million and why have there been two other grossly different estimates from the same person? Is one of them a lie? A distortion, if you prefer? Or just incompetence that keeps her from giving a consistent answer?

No one can give a consistent answer until more is known. The range in cost COULD be huge -or it could be nothing.

It could be ZERO additional costs if we use existing multiple-choice data banks, shorten the test, and only test grammar and punctuation
- but even then some renorming of cut score changes will have to occur from moving the test.

If we keep all the same testing priorities we have now it is not hard for most of us to believe there will be additional costs.

3) Why have the administrator groups changed their position on spring/fall? They used to be the leaders in calling for spring testing -- and they were very good as communicating all the reasons that spring is better. Could it be politics?

Not politics with me. I personally know that over 1,000 schools have used current ISTEP baseline data as part of their "pretest" on their school improvement efforts. They are supposed to use "post test" data 6 years from the baseline year to calculate effect sizes as an internal evaluation of their improvement efforts. Moving the test violates their protocol rendering the ISTEP data source unusable in their accreditation documentation. Schools are just tired of being jerked around. They adjust, make a plan - then someone out of their control changes things because "spring just makes more sense."

It only makes more sense when people don't know enough about it. This debate will go for ever because there are good points and bad points to spring and fall. Educators are just tired of people jerking it around.

For the first time I have been in the business I have seen tremendous efforts on the part of schools to be more assessment driven and to broaden their horizons on assessment methodology.

Changing the test right now stops our trend gathering and creates entirely new baselines in the middle of documentaion cycles. Not productive for schools given the small return on changing it. The only real "return" is for the policy makers who have their own reasons for wanting it that have little to do with the reasons I have just noted.

My .o2

I can't speak for all superintendents on why they might oppose the switch.

At Wednesday, February 08, 2006, Anonymous Anonymous said...

You can't seriously be suggesting that your acceditation will be a risk if ISTEP moves from fall to spring. You certainly are not talking about Indiana state accreditation, because test scores are nearly irrelevant to that process. (A sad fact, but true.)

At Thursday, February 09, 2006, Blogger Indiana Public School Superintendent said...

No, I don't mean that accreditation itself is at risk. I just mean that the law requires schools to belong to an accepted accrediting association or state accepted accrediting process. Over 1,000 Indiana schools are members of NCA, a volunteer organization providing school improvement support in 19 states and all Department of Defense Schools. NCA accreditation is one of the few that uses somewhat complex accountability measures as just ONE factor in accreditation.

Schools are required to set goals and develop intervention strategies to improve student achievement. Then use statistical methods to do program evaluation.

Schools must develop an assessment plan spread over their 6 year accreditation cycle and calculate effect sizes between their baseline years and their documentation years. These effect sizes show the "magnitude of change" between those years.

The schools use a combination of "hard data" and "soft data" to calculate effect sizes over the time period to determine if their interventions with students make a measurable difference.

Switching the tests does not endanger accreditation but it fouls up the use of ISTEP itself because the baseline test year and the post testing years may no longer be "psychometrically equivlent."

When this happens schools can only use their other data sources for calculating their effect sizes, taking away one of their "hard data" sources.

At Thursday, February 09, 2006, Anonymous Anonymous said...

I completely understand your concern; but it is one that you need not have.

Unbeknownst to the public (and also to schools, judging by your comments), ISTEP has already been changing EVERY YEAR. A couple years ago, it changed dramatically when it was rewritten to reflect our new standards. Since then, it has been changing as the DOE responds to an ACHIEVE report, which indicated that much of ISTEP was not testing the right grade-level content (ex: 30% of the 3rd grade English exam was based on FIRST grade standards). And recently, the state's testing administrator testified before the State Board of Education that the questions on ISTEP change every year and, as a result, the results and cut scores have to be recalibrated each year.

In each of these situations -- even during the big changes resulting from new standards and adjustments to actually reflect our standards -- the test administrators have been able to maintain "psychometrically valid" results. A change from fall to spring -- which actually involves a smaller change in content than has occured in these other recent changes -- would not threaten this issue.

At Thursday, February 09, 2006, Blogger Indiana Public School Superintendent said...

It doesn't threaten the issue but here is why it complicates it even more.

I KNOW the test changes every year. That is not necessarily a "validity" problem provided CTB reallly knows what they are doing. You are still looking at it like a policymaker though. It meets YOUR needs, but switching the testing time period and grade levels does not necessarily make the test "pschometrically equivalent" for program evaluation purposes over a period of years.

Psychometric equivalency is not the same thing as validity.

Just the fact that they change items every year creates some havoc with "equivalency" at the local level. The only "equivalency" we get is from the scale scores or the % Passsing score. When they float the cut scores every year by changing items it makes the IPI trends unusable. (Example: Say the school has a goal of improving Reading Comprehension. Say the mean IPI for RC is 62 in 2003 and the cut score is 60. The IPI goes to 65 in 2006 with a cut score of 66. See why equivalency is important? I can't run effect sizes on the change from 62 to 65 because they aren't eqivalent.The scores went up BUT the student body is actually performing lower in RC than before. The only score we can use is the % passing which only provides two categories - Pass and Did Not Pass. Not real precise. Tremendous gains can be made by lower performing students but they still might not quite pass. THis is important information for people trying to decide if one methodology or program is making a big difference in a standard such as Reading Comprehension. Conversely, top students might still pass the ISTEP but have been ignored completely and they are losing ground - this might not show up in Pass rates.

Switching tests solves some problems and creates others.

At Thursday, February 09, 2006, Anonymous Anonymous said...

No disagreement on anything you've said; but everything you've said is as much of a problem now as it would be under a change from fall to spring.

Moreover, your completely valid concerns about a rough pass/no pass division point to yet another problem with ISTEP. Not all tests are done this way. Thus, another reason why changing the test will be better.

Spring-fall is driving this debate. But the greater issue is that ISTEP fails, as an effective assessment system, in many, many ways. And Dr. Reed is not just fighting the spring-fall issue; she is fighting an effort that could fix ALL of these problems.

At Sunday, February 12, 2006, Anonymous Anonymous said...

One would think that "creative, enthusiastic, open-ended activities" would be one of the most effective ways of teaching any subject matter.

So I think a more appropriate question to ask is this:

If this type of instruction were commonplace in our schools, would NCLB and Indiana's PL221 even be necessary or perceived to be so?

Also, if teachers do not believe they can teach the standards within a creative and enthusiastic environment, is that a problem with the standards or the teachers?

Or is this complaint just one more attempt by teachers to undermine the public's support for academic standards and accountability?

Unfortunately, the skeptic in me suspects the latter.


Post a Comment

<< Home