Monday, January 02, 2006

Are the Indy Star's graduation stats just as inaccurate?

Many high school principals have privately complained in years past about there not being a consistent way of reporting graduation and dropout statistics.

This fall the Indianapolis Star ran a series on graduation rates and dropouts in an effort to bring public attention to the issue.

Unfortunately, after a number of pretty decent editorials, they go too far with this one.

The Star's version of graduation rates is just as twisted as the old high school rates.

Here are a few quotes from the Star:

IPS, with a graduation rate of 39 percent in 2005, remains home to the region's worst dropout factories.


The failure of state and local educators to report realistic graduation rates, however, conceals such dismal performance. Says (Stan )Jones: "They think their schools have nice facilities and things are fine, when in fact things are not."


Inflated graduation numbers have lulled the public into believing that dropping out is rare. It's not.

Read through this list of Indy area schools and then look at the Star's definition of "graduation."

The Star's editorial board's calculations were made by taking the number of seniors graduating in 2005 and comparing it to the number of 8th graders in 2001. Couldn't get it more simple...and couldn't get it more wrong. Just as wrong as many of the "bloated" high school rates they are taking to task.

The Star claims that the Indianapolis Public Schools has a graduation rate of 39%. However, compare this 39% statistic to the Census Data ( note: click on the county for which you want stats - wait for the red arrow and click when the hand appears) which shows the percentage of adults over the age of 25 in Marion County with high school diplomas is 81.6%. Is there a disconnect? I guess there could be. Maybe 42.6% of them all leave the county or finish up high school before they turn 25. Or maybe this is a new version of the famous economic "brain drain" problem in Indiana? :-)

Except where do they go? Here are all the counties surrounding Marion county and the percentages of adults over 25 with high school diplomas.
Marion County 81.6%
Hamilton County 94.2%
Boone County 88.3 %
Hendricks County 88.5%
Morgan County 80.7%
Johnson County 85.7%
Shelby County 79.8%
Hancock County 87.8%

According to Census Data placed on the Department of Education website for the Indianapolis Public Schools, the percentage of adults WITHOUT a high school diploma in the IPS area has been reduced from 41.3% in 1980 to 28.3 % in 2000. This improvement trend was the same in almost every single school corporation on the Star's list.

I suppose the true graduation rates are closer to the census data numbers than they are the old "glorified" graduation rates the high schools reported or the Star's pessimistic view.

22 Comments:

At Wednesday, January 04, 2006, Anonymous Anonymous said...

HUGE FLAW IN YOUR REASONING: IPS is not the only district in Marion County. There are TEN other districts, in addition to IPS. And there is also a large number of private high schools.

Additionally, the Census Data, I believe, includes students who have dropped out but went back for a GED. Some people think that the original districts of those students ought to get credit for those GEDs; but most people recognize a GED as inferior to a standard diploma -- and it's attainment does not negate the fact that those students originally dropped out.

Sorry, Super. But you are really reaching on this one. And why? This issue is quickly getting settled in states and cities throughout the country -- and organizations like the Star are helping to get us there. Your persistent resistence to reality is both disappointing -- and downright silly.

 
At Thursday, January 05, 2006, Blogger Indiana Public School Superintendent said...

I am not sure what part of reality you think I am persistently resisting. :-) I was pointing out that neither the schools' nor the Star's statistics appear to be close to reality.

Counting graduation rates using the Star's methods is certainly simple. Unfortunately it is simply wrong or simply disingenuous take your pick.

My point was simply that the census data numbers for the various counties seems to be closer to a realistic view of how many students are actaully graduating.

That was all.

 
At Thursday, January 05, 2006, Anonymous Anonymous said...

Since we all know that everything used to be better than it is now, I would like someone to take a stand on exactly at what point in time we should go back to in terms of adults age 20-29 who have completed high school.

How about the 1940's when between 30% and 40% of adults 20-29 had completed high school?

How about the 1960's when that number rose to about 70%?

The point is that the number of adults age 20-29 who have completed high school has never been higher in the history of our country. So the graduation issue we are talking about is how to get the remaining relatively few young people to graduate from high school and become productive members of society. By definition, that group is going to be the most problematic. It seems to me that that problem will best be solved when concerned, reasonable people work together. The easy simple solutions have already been tried. Simply taking potshots at public education seems to be counter productive, since that same system seems to work very well for most students.

It seems to me that people who really care about this issue will propose serious, thoughtful solutions. Those with political agendas will provide criticism and solutions that serve their agenda, but don't really help kids.

 
At Thursday, January 05, 2006, Anonymous Anonymous said...

Sorry, Anonymous, but you just don't have your facts right. See charts here: http://www.edroundtable.state.in.us/pdf/David%20Shane%20Grad%20Rates%20for%20RT%209%2028%2005.ppt

Using NINE different calculation methods that do not depend on reported dropout counts, which have been widely documented as grossly flawed, Indiana's graduation rate in the late 1980's was likely 80-85%. Using those same calculations today, the rate is somewhere between 70-75%. This is a genuine DECLINE, not a mythical one as you suggest.

Is everything in education worse today than it was in "the past?" Of course not. In fact, the only people I hear suggesting this are those who use that idea to criticize business leaders and others who are critical of education in other, substantive ways. People like you love to suggest that critics of public education are saying such things, but you are never willing to actually attribute such comments to real people.

Nonetheless, there are SOME indicators -- including grad rates -- that really ARE declining. And given that higher achievement levels are necessary for EVERY generation (as has been true for CENTURIES!), a decline in any such measure ought to be alarming. It's bad enough that there are people out there, like yourself, suggesting that all we need is to do is perform the same or better than the past. But it is a recipe for disaster to ignore those indicators that really are declining.

 
At Thursday, January 05, 2006, Anonymous Anonymous said...

To super: Your "persisent resistence" is to the facts about graduation rates. You have NOT presented evidence to prove that the Indy Star calculations are flawed. As noted earlier, your own assumptions were GROSSLY flawed.

Frankly, most people researching and writing on this subject have presented clear reasons why the Star calculation method (grads over 8th graders five years earlier) produces an INFLATED graduation rate, not a deflated one. As is shown in the charts referenced in the previous post, there are several other widely-accepted calculation methods that show lower graduation rates. Yes, the Star method is overly simple -- but the result of that simplicity is an inflated grad number, not a deflated one.

 
At Thursday, January 05, 2006, Anonymous Anonymous said...

Anonymous:
1. Sorry, but they're not "my facts." Those facts belong to the U.S. Dept of Ed., Digest of Education Statistics 1998. If you have a problem, take it up with them.

2. When I look at Mr. Shane's ppt., it would appear that the sky is falling. However, when I look at the website he cites for some of his data (National Center for Public Policy and Higher Education) it appears that they report a graduation rate for Indiana at 82% with an additional 7% with a G.E.D. I'm hoping someone can explain the discrepancy.

3. It is also interesting to see how much more of their family income it costs the average Hoosier family to send their child to college when they graduate compared to other states. It's too bad there doesn't seem to be much of an outcry at the Star about that.

 
At Thursday, January 05, 2006, Anonymous Anonymous said...

BTW, Anonymous, no one is "suggesting" the status quo. Quite the opposite. Everyone involved needs to do differently and better than they have in the past to address this issue. However, having said that, my opinion is that the dropout/graduation issue is more broad based than simply an educational issue. Therefore, any workable solution needs to be more broad based as well. It will also help to leave political agendas at the door.

 
At Thursday, January 05, 2006, Anonymous Anonymous said...

So you are saying that the ACTUAL graduation rates reflected by the Stars simplistic methods are actually INFLATED?

So the 59% Pike Township number quoted by the Star on the link provided is in actuality even lower? These numbers just don't square with other traditionally accepted data found on NCES or Census data. Whether you use status dropout numbers from ages 16-24 from NCES or the % adults above the age of 25 from Census data who've graduated(which is what the Super was quoting) in either case the Stars numbers don't square up to the broader question of how educated are our young people today?


Mainly because the Star's simple methods miss the complexities of the problem. They do not take into consideration any student who graduates earlier than normal or who graduates later than normal. It doesn't consider GED's which we can argue about but I personally know of a student who just scored high enough on their GED to land a full ride to a state university. Too bad he's just another dropout failure/public school disaster statistic instead of a success. (I'll let him know what the Star thinks of HIM!) :-) It doesn't consider my friend's high school age adopted deaf child from a foreign country that is taking an extra year to complete high school on purpose. She too is another failure statistic on the Star's radar screen. It doesn't count my cousin who returned to high school at the age of 21 and completed it as a model student after dropping out at 16. It doesn't consider the 100+ students in our Alternative program who are taking an extra semester or two but most of whom WILL graduate (if they choose to of course).
It doesn't factor in local complexities like a corporation redistricting plan or a major demographic shift in the district. It doesn't consider immigration into the community when new job opportunities arise or major factors like the two year period we lost a major business with 600+ jobs in our small community. All those families with students between 8th grade and 12th grade who left to find work just contributed to our "dropout" rate using the Star's methods.

Back to what seems to be the Super's only point. The Chicken Little "the sky is falling" view the Star often displays doesn't square with even the conservative Cato Institutes admission that Americans today are better educated than at any time in the history of the world.

Whether or not this is good enough for the next 50 years is a different question.

 
At Thursday, January 05, 2006, Anonymous Anonymous said...

The last several responses have mixed so many issues -- some directly related to this strand, some only tangentially related and some not related at all -- that it is nearly impossible to respond. Nonethess, here is a brief attempt at a few of the points:

1) Super's misuse of NCES statistics (using Marion Co data to counter analyses about IPS) is NOT something to "take up" with NCES. Does NCES have information police that will come arrest Super for misuse, or what?

2) As best I can tell, the source of 82+7 on the National Center for Public Policy & Higher Ed is not explained. The nine calculation methods used in Shane's ppt DO have explanations. So, if want an explanation, they are all available -- except for the highest numbers, which have not been included in the graphs.

3) You say that solutions to the dropout issue need to be "broad based." Fine. You'll get plenty of agreement here. But to really get people engaged to help with the problem, you must first admit that a problem exists. You can't seem to make a decision on this point. First you call the statistics inflated and both you and Super suggest that we are the best we've ever been (not really a problem, eh?), then you say that everyone needs to help with this problem. What problem? I thought we were the best we've ever been!

4) Yes, if the 59% calculation for Pike is based on grads over 8th graders, then it is likely inflated. It ignores growth in the high school (Pike is a growing district) and it ignores the students who enter move from private school settings between the 8th & 9th grades. Both of these facts inflate this simple calculation.

5) Yes, this calculation is a simple one. And for that reason, it is only an estimate. But while you offer several reasons that the real number should be higher, there are also reasons (like the two above) that suggest the number should be lower.

6) All of your examples of late graduates would actually INFLATE the number, not deflate it. That's because those grads WOULD be counted in the numerator of the calculation (grads), even though they were not in the denominator (8th graders). And besides, if there is any consistency to this pattern, then the numbers would wash out. If there is not consistency, then the calculations would appear eratic (which they do not).

7) To paraphrase one of Cato's hero's, "there you go again." If you are going to defend against those who supposedly claim that all of education is getting worse, then you should at least point out who is saying what that results in your defense. As I said before, the only people I hear making comments about the "disaster" of public education are those who claim, erroneously, that others are saying such things.

8) All of the degrading comments about your friends who got GEDs are your own words, no one elses. They are not mine; and they are not the Star's. What I did say, which I have never heard anyone dispute, is that 1) GEDs are not as rigorous as standard diplomas and 2) those who earn GEDs are typically still dropouts, as far as their home district is concerned.

You said more which is mostly tangential or irrelevant; but sorry, I have already wasted too much time on someone who seems far too unwilling consider facts and far too quick to cast false statements onto others.

 
At Friday, January 06, 2006, Blogger Indiana Public School Superintendent said...

This is what I started with...."Many high school principals have privately complained in years past about there not being a consistent way of reporting graduation and dropout statistics." This argument sure demonstrates that.

This is what I concluded with..."I suppose the true graduation rates are closer to the census data numbers than they are the old "glorified" graduation rates the high schools reported or the Star's pessimistic view."

All the debate over whether the Stars figures are inflated or deflated or whether the schools numbers are inflated or deflated continues to drive the point home that getting clean data is difficult.

However, the census data show consistently that the percent of adults who are educated is at the highest levels it has ever been. My point is that those numbers seem to provide a better 'bird's eye view" of the educational level of society and seem a little more in line with what I think represents a truer picture of graduates today then EITHER the Star's numbers or high school's numbers.

 
At Friday, January 06, 2006, Anonymous Anonymous said...

Anonymous,
1. You used David Shane's ppt. as your source.
2. David Shane used the National Center for Public Policy and Higher Education (NCPPHE) as his source.
3. If there is any explanation necessary, that would be your responsibility, since it is your source.

 
At Friday, January 06, 2006, Anonymous Anonymous said...

Wrong. The National Center for Public Policy and Higher Education was the source of the last graph in Shane's ppt. I referenced only the preceding graphs, which did NOT come from NCPPHE. Nonetheless, after your question, I did try to research the NCPPHE info to understand its data. As I noted earlier, I could not find any explanatory info on the NCPPHE data and, therefore, I am neither referencing it nor trying to defend it. Surely you would agree that if I am not referencing that data then I have no responsibily to explain it. Right? You, on the other hand, have referenced it as a counter to the data that are explained. So please: explain away!!

Besides, this question about explaining data really came up regarding Super's use of Marion Co NCES and Census data to counter other information that was focused on IPS. As I noted, those are two different populations. Your flippant response to that was to "take it up" with NCES. Again, if Super misused the data, then that definitely is not my responsibility to defend.

Having said all this, I do agree with Super that there are some interesting questions to pursue on this issue. Even though they are aggregated, the Census data do present a different picture. That is a question worth exploring. But there really are not any blatant holes in the dropout data to cause an automatic dismissal of them. Since the Census data cover a different population (different age group and different geographic areas), it is quite possible that both sets of data are absolutely accurate. For example, populations shift, the Census data include private school grads that are not in the district numbers and/or the GED attainment could be higher than is realized. Also, given that the Census is based on personal surveys, there could be some error there (indeed, we KNOW that there is SOME degree of error): 1) survey respondents have a tendency to report personal information that is more positive than reality (education attainment, income, etc), and 2) there is some documented concern that lower income populations are not accurately represented in the census, which could also mean that poorly educated populations (stong correlation here with low income) are also under-represented.

All interesting questions -- but they simply do not counter the grad data that have been presented.

 
At Friday, January 06, 2006, Anonymous historian said...

Couldn't these issues be resolved by keeping more complete records of the destinations of students? I would think that, while schools certainly are asked to do a lot already, it would be relatively simple to use a database application in order to record the status and result of a child's education.

Let us say, for example, you start out with 200 5th graders. Through middle and high school, 10 transfer out, 5 transfer in, 3 die, and 175 graduate. That means that if you record what happened to the best of your ability for those 200 students, you know that you had the opportunity to graduate 192 students (200 beginning students - (10 tranfers - 5 inbound transfers) - 3 deaths). That would yield a graduation rate of 87.5%.

Certainly the real world is immensely more complicated than that, but if each parent is required to fill out paperwork when a student leaves/moves on, etc., then that status could be entered into the database. Obviously, there will be times when a parent can't/won't fill out the paperwork, and it sounds like that would be added to the list of those who failed to graduate.

Perhaps the intermediary to those processes is just for everyone to use the same procedure. While both the census method, and the comparison of 8th graders to graduates are both flawed, as long as everyone uses the same flawed method from year to year, the results would be (relatively) consistent.

 
At Friday, January 06, 2006, Anonymous Anonymous said...

All the stats, theories and debates are really stimulating to this sheltered stay at home mom. But I can't help but think one simple but serious question.... where are the parents to these high school drop outs? Whether the non-graduation rate is 10% or 50%, we hold the schools responsible. I agree, yes, we need to take a hard look at this issue, the real issue, not the inflated to drive my point home one. Why do we not hold parents accountable?

I have several kids. If one did not graduate, who would I blame first? My spouse and I. Period. If I had troubles with a school not doing it's job, well... I would have had 12 years to work on this. My kids are relatively young but they know they are expected to graduate with grades that reflect their best effort. They also know college is the next step. No if, ands, or buts.

It's easy to blame public education for all these failures. It's much more difficult to understand we have a serious lack of parenting going on. Schools will never be able to make up for absentee parents. I doubt any politician will call this one out... doesn't lead to a lot of votes.

concerned parent

a concerned parent

 
At Friday, January 06, 2006, Anonymous Anonymous said...

It is interesting that as long as David Shane could use the NCPPHE data without scrutiny, it was just fine. Now that someone is questioning the data, all of the sudden it is not worth defending. All I know is that Mr. Shane's ppt. shows one thing and the NCPPHE website (which he references in his ppt.) shows something different. What is the explanation for the discrepancy?

The concerned parent nails it though. Whatever happened to individual and parental responsibility?

I appreciate the fact that Anonymous recognizes that more study of this issue is warranted. He goes on to say that just because more study is needed doesn't mean we should automatically dismiss the Star's data. It seems to me that that is also a good reason to not automatically accept it at face value either.

 
At Friday, January 06, 2006, Anonymous Anonymous said...

I'm sorry, but the reasoning displayed above is a sad but all-too-typical example of the irrational, thoughtless response of far too may "education leaders."

YOU are the one who relied on the NCPPHE data to COUNTER my reference to OTHER data which just happened to be nicely summarized in Shane's OTHER graphs. I made NO reference whatsoever to the NCPPHE data. Yet, you continue to criticize my reference to OTHER data because I will not defend the NCPPHE data. YOU referenced the NCPPHE data, so YOU explain it.

And on the latter point, you are actually suggesting that data for a DIFFERENT population ought to be sufficient evidence to cast doubt on 9 other calculations, each of which show similar results and whose rationale you are incapable of inpugning. ISN"T THAT CONVENIENT!? You can't present a rational argument why nine different calculation methods are wrong, so you throw out a different set of data, which clearly are not the same population and which have other problems that have been documented, and that is sufficient evidence to cast doubt on 9 other data sets for which you cannot present a rational criticism?!?!?

Frankly, I have had an easier time explaining such rationale to my PRESCHOOLER! NO JOKE!!

 
At Friday, January 06, 2006, Anonymous Anonymous said...

To concerned parent,

I know it runs contrary to common public perception, but there is actually a lot of research out there now that says pretty clearly that teachers have the greatest impact on education achievement of their students, regardless of their socio-economic or other family backgrounds. None of these studies dismisses the importance of parenting; but when all the factors are thrown into the analysis, teacher quality matters more. For more on this, look first to the Education Trust. But there is a lot more out there for those who are truly interested. (Reprinted from a previous post.)

 
At Friday, January 06, 2006, Anonymous Anonymous said...

No study will ever convince me of that. And this is why.... As an involved parent I have chosen a quality school district. As an involved parent I have chosen quality teachers that fit my child's personality. As an involved parent I have modeled that education is pivatol, required, and nonegotiable. My child wouldn't be having exceptional teaching if I weren't first involved. Perhaps they would luck into it from time to time but not as consistently if I weren't their advocate.

In the years when the teaching isn't so great... I supplement at home and look at ways to make up for it next year. How can any study reasonably identify high quality parenting? economic status, a parenting questionaire? I have also seen many studies that contradict this one. One if fact in The School Board Journal publication last year. A researcher studied numerous findings from other researchers on this very topic and found that parents were the number one identifier for school success.

Anyhoo.... I was talking about drop out rates, not teaching vs. parenting effects on education. How many kids would drop out if they knew there would severe and immediate consequences at home? For me, it simply isn't an option if my kids want shelter, food, clothing and support. I'm not a whiz at the law, but don't they need parental consent if under 17?

Sorry, not buying it. And no study will make me change my fundamental belief system. We have really good public schools for the most part. Yes, there are areas of concern, room for improvement. And, all schools are not created equal. But as I walk through the halls of my children's schools I see incredible hard work going on. Much harder than when I was in school. The standards are higher. The curriculum has expanded. And the opportunities have increased. The kids who are sent to school ready to learn will do so with the necessary supports and services. I will believe my own two eyes and my own personal experience.

concerned parent

 
At Friday, January 06, 2006, Anonymous Anonymous said...

Anonymous, sorry to be causing you to have a rough day, although I'm glad you and your pre-schooler are getting some quality time together.

Anyway, I'm not trying to impugn this or present a rationale for that. I'm just asking a simple question. Can someone explain the discrepancy between what Mr. Shane shows in his ppt. and the data shown on the NCPPHE website. And furthermore, if I am not mistaken, the data used on the NCPPHE website is taken from the exact same population that Mr. Shane uses on the last slide of his ppt.

I asked this question yesterday and no one has answered it yet. But I understand these things take time. I will check back now and then. I'm sure someone will come up with something.

Oh, and BTW, whatever reasoning you used to come to the conclusion that I am an "education leader"; you might want to discuss that with your pre-schooler. Since I am not an educator, it would be difficult to be an education leader. Just picture someone sitting around in their pajamas. Asking questions. Waiting for explanations.

 
At Friday, January 06, 2006, Anonymous Anonymous said...

Wow. You really do have too much time to be so dense in your understanding. For the... I don't know... 5th time maybe: I know absolutely nothing about the NCPPHE data. That's why I have neither referenced it nor defended it. I'll happily leave that to those of you who have used the NCPPHE data to make a point.

No matter how many times you ask, the ball is still in your court. Check back as many times as you like. But if you want explanations about your own points, you'll have to find them yourself. Apparently, you have the time for it!

 
At Friday, January 06, 2006, Blogger annie said...

"Just picture someone sitting around in their pajamas. Asking questions. Waiting for explanations."

cool...do you get paid for it or is it a casual day at the office.

 
At Saturday, January 07, 2006, Anonymous Anonymous said...

Let's see, I believe that would be Mr. Shane who initially referenced the NCPPHE data. And you (Anonymous) were the one who referenced Mr. Shane's powerpoint. Therefore, since you won't defend Mr. Shane, I guess someone else will have to do it. No matter who does it, my question remains. Can someone, anyone, explain the discrepancy between the graduation rates shown on Mr. Shane's ppt. and the graduation rates shown by the NCPPHE? I'm sure that there is someone out there smart enough to explain it for me. It will probably be the same person who can explain the discrepancy between the Star's data and the U.S. Census data.

Still in my pajamas. Still asking questions. Still waiting for explanations. Still dense in my understanding. Still have too much time to be so dense in my understanding. Still casual day at the office. Oh, and Annie, still cool.

 

Post a Comment

<< Home