I Grade a Report Card: Or, I’m Naming My First Kid O’Hanlon

I Grade a Report Card: Or, I’m Naming My First Kid O’Hanlon

Hi. I'm Mike. I loves the maths. Can I haz calculator? (via Brookings Institution)

I admit that I don’t know a whole lot about Michael E. O’Hanlon or Ian Livingston, but they wrote a piece for the New York Times: basically, things in Afghanistan are going just fine (By the way, Stephen Saideman did a short post on this. He raises some interesting questions.):

Here is what we know: Afghans are wealthier, healthier and better educated than ever before. Unquestionably, Afghan security forces are bigger and better. Despite the occasional spectacular attack, Kabul is relatively safe, accounting for less than 1 percent of violent episodes nationwide, thanks largely to the efforts of these troops. The security situation in the more dangerous south is also much improved, after two years of efforts by foreign and Afghan forces. The north and west are at least no longer deteriorating and collectively account for less than 10 percent of violence nationwide.

And now I know all I need to know about O’Hanlon/Livingston.

Oh, for those of you following along? This post is the one I talked about yesterday.

Allow me to retort, and I’m only going to limit myself to one line in that paragraph, as much as it physically pains me to do so.

Unquestionably, Afghan security forces are bigger and better.

That’s a great word: unquestionably. That means you have “facts” that are likely “irrefutable” which is another big word for “we are experts,” and can therefore “do math.”

That last shot across the bow will make sense shortly.

I’m not going to debate the quantity of ANSF. The force is definitely bigger: every year, there are more of them.

Better? No.

And since I’m not entirely happy with making unfounded pronouncements and therefore would never be able to work for the Times, let’s kick it off with a quote from the last 1230 report in October of 2011 :

The overall operational effectiveness of the ANA continues to improve. In August 2011, the total number of reporting ANA units in the field increased to 204, and the number of units achieving an operational effectiveness rating of “Effective with Assistance” or higher was sustained at 147; alternatively, 37 units (18 percent) of fielded ANA units are in the lowest assessment categories, “Developing” or “Established,” due to an inability to perform their mission or the immaturity of a newly-fielded unit. Even the ANA’s highest-rated kandak, 2nd kandak, 2nd Brigade, 205th Corps, which achieved the rating of “Independent,” remains dependent on ISAF for combat support and combat enablers. In locations without a large ISAF footprint, the ANA has exhibited little improvement and there is little reporting on their operational strengths and weaknesses. These units are typically located in the west and far northeast regions.

Usually “independent” means: “not dependent; not depending or contingent upon something else for existence, operation, etc.” But reading the paragraph above, it means they’re still very dependent for key tasks necessary to conduct operations.

In ISAF Land, the word used to mean what it means to the rest of us as of March 2011, when this is how ISAF defined “independent”:

“Independent” denotes a unit that is capable of the full spectrum of its missions without assistance from Coalition Forces.

Unfortunately, the result of that definition was this pesky problem:

State of the ANA, February 2011

That’s right: after nearly 10 years of ISAF intervention, and nearly two years of concerted effort by NATO Training Mission – Afghanistan (NTM-A) personnel, no unit at any level had achieved an “independent rating.”

So they changed the definition.

From page 43 of the 1230 report in October of 2011:

Prior to the spring campaign, IJC reviewed the definition of an Independent unit and concluded that the definition was too restrictive and would be difficult for any ANSF element to attain. As a result, IJC rewrote the definition of an Independent unit to reflect the reality that most ANSF force enablers will likely require long-term coalition assistance.

Which, is how you get this:

State of the ANA, November 2011

Need an “independent” unit? Don’t actually make it independent: just change the meaning of the word.

Now remember that “204″ number previously? That’s counting all units (to include headquarters, etc.) that are considered “effective.”

For purposes of effectiveness, though, your best unit of measurement would be the kandak: this is the equivalent of a battalion, and the unit that’s the best indicator of the effectiveness of a fighting force writ large. This is because a battalion is the smallest military unit that is able to conduct independent military operations. (I know, some militaries are organized differently, but this is generally true.)

So your number of kandaks rated as “effective” would be…important. And if you’re truly getting better, then you would see more kandaks moved into that category.

In this case, what I did was total up all kandaks (not kandak equivalents) that were rated as “Effective with Assistance” and higher.

April 2011: 116

October 2011: 115

Must be one of those public school education “fuzzy math” kind of things, because that seems…lower.

In a reporting period (of about six months) when they fielded an additional five kandaks or “kandak equivalent” unit (the GSU), ANA end strength went from 116 kandak units rated as at least “effective,” to 115 units with that same rating.

When one fields more units of a particular size, yet experiences a net loss in effectiveness in those units, one’s army isn’t really getting “better.”

Favoring quantity over quality, O’Hanlon and Livingston seemed to not have dug too deeply into the numbers supporting the premise that the ANSF is “unquestionably” better. However, I could be completely wrong on this, since the article cites no actual sources for their research. I, on the other hand, just had to go with publicly available reports on NATO progress generated by NATO itself.

In a war that offers relatively few metrics by which to measure success, being run by an organization that shifts those metrics randomly to fit their message, it’s unusual to find solid numbers to demonstrate anything. In this case, it’s simple math.

The interwebs is hard.

I’m off to break the news to my wife: in honor of the genius that walks among us mere mortals, we’re naming our first child “O’Hanlon.” And he shall be great. And able to do the maths.

Share This
  • Djsmock22

    On the bright side at least the name sounds Irish :) what a joke! !

    • http://findingmytribe.wordpress.com El Snarkistani

      Yeah I had to keep checking my math. Then checking it again. Even got the big caculator out.

  • Ben

    From your article on the Kabul attacks:

    “Which may seem to be a contradiction of the previous statement, but what needs to be understood here is that the ANSF, while achieving greater levels of capability, are not yet ready to conduct any operations completely independently of ISAF support.”

    “If the message is: “We can get you anywhere,” message sent. Just, once they get there, they don’t tend to accomplish much. This speaks to increasing leves of proficiency by the ANSF in their response to the situation, but also to the lack of quality intel/planning to make sure these kinds of events do not happen again.”

    Isn’t saying that the ANSF have higher levels of capability and are more proficient the same as saying they have a higher quality level?

    On the specific number of effective kandaks, what is the number today? Would O’hanlon possibly have newer figures?

    • http://findingmytribe.wordpress.com El Snarkistani

      If you need someone else to provide all of your fire support, you aren’t ready to do it on your own. ANSF were able to assault, kill bad guys, etc., but they still required ISAF mentorship, and were unable to conduct any fire support on their own. That was entirely run and executed by ISAF.

      They respond better than they would have a few years ago, but their inability to support the assault, as well as intel failures warning them of the attack, speaks volumes about how far they still need to go. This is the gem in the ISAF transition crown, and I’m still only giving them a C-.

      • Ben

        My point was that you attacked O’Hanlon for saying the ANSF quality has increased when only a few days ago you said exactly the same thing.

        ANSF will need support for a while yet. It takes a lot longer to setup artillery and air support units than it does infantry units.

        • http://findingmytribe.wordpress.com El Snarkistani

          Quality in the ability to respond is not the same as quality when it comes to planning, supporting, and executing complex military operations. At this point, there is not a single completely independent unit in the ANSF. The ANSF in Kabul were mostly able to respond because they were in the area. Granted, few years ago they probably wouldn’t have been there, or would have run if that had happened in the first place.

          The problem is this: definitions. Since “independent” means “independent with advisors,” is ISAF going to develop a “for real independent” category?

  • Ben

    Yep he does have updated figures:

    “NOTE: In January 2011, there were 155 ratable Army Kandaks, 101 were rated in the top-3 tiers of readiness. By early 2012, there were
    168 Army Kandaks with 127 rated in the top-3 tiers and 11 rated independent.”

    http://www.brookings.edu/~/media/Files/Programs/FP/afghanistan%20index/index.pdf

    pg 8

    • Ben

      Considering his source for the 2011 figures is General Allen, I’d say he has access to fresh data. Which I assume is where he got the 137 kandaks rated in the top 3 tier figure in the NYT article from.

      I’ll guess we will know for sure when the latest 1230 report is released in the next few weeks.

      Maybe hold off on the “bad at maths” jabs until then, or simply not use 8 month old data to criticise an article published barely a week ago ;)

      • http://findingmytribe.wordpress.com El Snarkistani

        You’re right…he must have access to more data. More up to date data, yes?

        Except that when it comes to hard, actually reportable data, I’m working with newer information than O’Hanlon and Livingston, with the exception of GEN Allen’s speech to the SASC.

        Where I was wrong was in not looking at kandak strength, vs. kandaks + GSUs. If I do that, there’s actually a drop of 3 kandaks over the course of the same reporting period. Granted, one can not fight without one’s support units, but to actually have fewer capable units at the end of the last reporting period is unsettling.

        Also, I’m actually digging into a report. O’Hanlon and Livingston’s footnote on the Brookings document (thanks for the post, by the way) references ISAF briefing slides vs. the 1230 reports to Congress. And, they last looked at those slides in June of 2011. I’ve pasted their footnote for reference:

        12 ISAF, “Metrics Brief September 2010”, unclassified briefing slides. ISAF, “Metrics Brief June 2011”, unclassified briefing slides. General John R. Allen, “Hearing of the Senate Armed Services Committee Subject: Situation in Afghanistan”, Senate Armed Services Committee, March 22, 2012.

        The 1230 report I used actually goes through August of 2011.

        Their next “data” point is Gen. Allen’s SASC testimony, and unless there are graphics I missed (always possible), he just gave a summary, vs. showing the breakdown of kandak/GSU units.

        I see your point re: waiting, but what’s a little troubling about their NY Times piece is that they don’t provide any supporting data, just colors on a chart. While you obviously found their reporting source, not showing that up front is disingenuous, and expects us as readers to just take their word for it rather than giving us visibility on their references.

        Final thought: using a general’s speech as a source for metric data is about as intellectually lazy as one can be. LTG Caldwell, for example, in the fall of 2011 reported a nearly 6% increase (to around 10%) in southern Pashtun recruits into the ANA. What he didn’t mention in his speeches made on the topic was the fact that this increase had come about by simply re-classifying a tribal group in Nangarhar as being “southern” Pashtun vs. just Pashtun.

        What was deceptive about that was the fact that this group hadn’t been in the south as a group for several decades.

        So using a general’s speech as any kind of reporting point is pointless.

      • Ben

        What exactly is the difference between a table produced by the military and a report by General Allen to the Senate Armed Services Comitee in term of it being “hard, actually reportable data”?

        They both come from the very same source. The military. Sure be skeptical about data from them but don’t treat one number as dramtically more reliable because someone put into a table for a written report. I’m sure General Allen is using the numbers in the soon to be released April 2012 1230 report. He would be a fool to use numbers that differ from that.

        Not also that the general’s numbers is very much in line with past data. Such increases are not unusual.

        the numbers seem very reliable, and your childish “math” jibes at O’Hanlon were baseless. You should correct the post and apologise to him imo.

        As for not showing their source up front in the article. It’s an article in the NYT. How many newspaper opinion pieces give detailed footnotes?

        Very few. I’d like it to be different, but it wasn’t hard to find O’hanlons sources was it? You only had to look for a couple of minutes like I did and you obviously didn’t;t before writing this baseless hit piece.

        • http://findingmytribe.wordpress.com El Snarkistani

          You’re right. A speech made by a general wherein he provides summary figures that wouldn’t allow for someone to go through the data to determine what’s really behind those rollup of numbers makes a dandy research tool. If you read my piece, between the two reporting periods they showed an increase in overall effective units, but when you start parsing it out, they actually lost units from the effective categories. They didn’t move up, they moved down. So just taking a summary number is lazy.

          What’s interesting about the op-ed piece is this: there’s no link to their base data. And, just as an FYI, I sent an email to the Brookings Institution asking them where they got their info from for that piece.

          I still haven’t heard back. That was nearly a week ago.

          You’re right: it’s an op-ed, but it’s an op-ed that purports to be reporting “facts.” In that case, a) show your sources, and b) don’t ever use words like “unquestionably.” That’s arrogant, and shows a certain lack of intellectual openness for debate. There’s no such thing as an “unquestionable” statement about things like improvement in troop quality. Far too many variables are at play: I’m just looking at basic numbers.

          I’m working on a follow up that will (hopefully, once they release it) include the latest 1230 reports.

          I still say the numbers aren’t reliable until that report comes out. Then we’ll see.

          As for the “baseless hit piece” — if I’m wrong, it’s because I worked from different sources. “Baseless” means that I had no basis for my critique. I did. I still stand by that on the basis of my source data. My assessment based on that data is 100% accurate.

  • http://diplopundit.net DS

    O’Hanlon El Snarkistani? Nooooo, the kid will never forgive you!

    • http://findingmytribe.wordpress.com El Snarkistani

      I’m saving for therapy now.

  • Pingback: Astute Reader Is Helpful, or O’Hanlon, Round 2 | It's Always Sunny in Kabul

  • Pingback: Afghanistan SmackDown: El Snarkistani v. O’Hanlon, Unfuzzy Math and Creative No More Problemo | Diplopundit

  • Pingback: Follow Friday: The Kabul Cable | It's Always Sunny in Kabul

  • Pingback: New Feature: Whack-a-Wonk Wednesdays | It's Always Sunny in Kabul

  • Pingback: Three Things the NY Times Ignored In That One Op Ed | It's Always Sunny in Kabul