The Mexican drug war has spawned a statistical battle -- whether or not Mexico's official government crime rates are real, and what should be done about the drug war itself in light of the statistical evidence, or lack thereof.  The James A. Baker III Institute for Public Policy at Rice University has a recent five-part October 2013 series on these topics.

President Enrique Peña Nieto claimed "he would bring a new strategy to [Mexico's] struggle with organized crime -- one that de-emphasized the targeting of drug kingpins and focused on reducing homicides, extortion cases and kidnappings."  Peña Nieto assumed office on December 1, 2012.  Thus, his administration has been in power for less than one year.  Quite simply, even if the Mexican crime statistics are valid, it is still far too early to assess either way whether Peña Nieto's policies have had any meaningful impact on drug war-related violence.

But as Molly Molloy, Sylvia Longmire, Robert Bunker, and John Sullivan have observed, we need to seriously question the Mexican government's official crime statistics.  Mexico is an extremely corrupt country, and the statistics-based policy train travels in all directions.  Of course, the desire to justify policies can lead to data distortion and fabrication.  Even in free and open Western democracies with minimal levels of government corruption, the gathering and interpretation of crime statistics can be fraught with errors and omissions.  But in nations like Mexico, we likely have no idea what the actual crime rates are and have been, where they are headed and why, and whether various policies are positively or negatively impacting crime rates.

We must also focus on rates, not absolute numbers, as some authors have done.  Crime data must, at a minimum, always be normalized to population to decipher any trends.  Mexico's population has been consistently growing at about 1.2% per year for the past decade, down substantially from the 1.8% growth rate seen in 1997, when many of the widely used Mexican government's crime statistics datasets begin.  Just looking at total numbers of a particular crime over a certain period of time tells us little about trends and their causes.  Changes in crime rates are also linked to more than just law enforcement policies.  Demographic and other socio-economic changes over time also exert a significant influence on crime rates, but these are very difficult to control for.

On their face, what do the official Mexican crime statistics from the Executive Secretariat of Public Security (SESNSP) show?  The rising crime rates in the mid- to late 2000s from the drug war are clear, but we may also be seeing a reduction in some crime rates over the past couple years.  This presumes data validity, which -- as mentioned already -- is effectively impossible to clearly support or refute.  Furthermore, attempting to correct the Mexican crime data is a fool's errand.  Reconstructions for socio-economic datasets can often introduce more errors than they fix, getting us farther from reality.  While evidence-based policymaking is a generally desirable goal, in the absence of solid data, we are often forced to rely more heavily on a values-based policy prescription approach.

Just looking at the raw data raises substantial quality concerns.  After having 788 homicides in 1997, the number of homicides in the southwestern state of Oaxaca dropped by more than 98% to only 14 in 1998, and then increased almost 6,000% the following year (1999) to 823.  So how real is the data point for 1998?  Looking at Oaxaca's homicide rate between 1997 and 2012, 1998 is at 0.39 per 100,000 population, while the remainder of the years range between 12.0 and 42.3.  It is almost a statistical certainty that the 1998 data point is junk.  It sticks out like a blazing neon sign in what is a generally small, but critically important, dataset for arguably the most important crime statistic out there.  So what other junk data points are sitting in this dataset?

As well, after ranging between 163 and 352 homicides per year between 1999 and 2006, the number of homicides in the state of Tlaxcala dropped to zero in 2007 before rebounding somewhat to range between 42 and 71 from 2008 through 2012.  The reported homicide rate for Tlaxcala between 1997 and 2012 is shown below.

Somehow, after increasing by a factor of 7 over only three years to what equates to an extremely high homicide rate of 34 per 100,000 population, the homicide rate drops to zero from 32 in only two years...and remains at a much lower level for the next five years?  While homicide rates can have significant, albeit modest, inter-annual variability, extreme datasets like this make me reach for my tinfoil hat.  Yet, apparently, the BBC and The Economist have no problems with Tlaxcala's homicide data.  Some independent bloggers, at least, appear to be thinking far more critically about the underlying quality of Mexican crime data than the mainstream media is.

What about numbers for motor vehicle thefts with violence? This data approaches the absurd in some regions. For example, after having zero motor vehicle thefts with violence during 1997, the state of Baja California saw this rise to 29,143 in a single year by 1998, to remain between 22,736 and 29,532 for the next five years, only to drop to zero again in 2004 and remain exactly at zero until 2008, after which there was a small increase up to 300 by 2012. Perhaps the post-2003 data was rigged to avoid scaring off tourists bringing their vehicles in from the USA? But how does this explain the equally anomalously low 1997 data point? Overall, there is little hope of explaining such bizarre trends, thereby precluding any reliable analysis of the motor vehicle thefts with violence dataset.

With these potential problems in mind, and using the official government data, the following figures show the geographical and temporal trends in homicide, extortion, and kidnapping rates between 1997 and 2012.

A superficial examination of the patterns suggests a potential reduction from the 2009-2011 peaks in some states, and some evidence that the crime wave is diffusing across the country.  But, to return to the key point that Molloy and others have correctly raised: how real is this data?  There is no point in wasting advanced statistical analyses on suspect data.  Garbage in, garbage out (GIGO) applies.

The policy implications of trends in Mexico's crime statistics are profound.  Those who favor an end to the war on drugs and pursuit of drug legalization will argue that drug-related crime in Mexico since 2006 has been underestimated, that many more innocent civilians than generally thought have been caught up as collateral damage, and that there is little -- if any -- evidence that government activities have reduced crime rates.  The other side, favoring a continuation or escalation of a government war on drugs, will be looking for evidence of reduced crime rates that corresponds with crackdowns by law enforcement agencies and a minimization of innocent casualties.

Neither side can rigorously work with the data.  The datasets appear to be too contaminated by political influence and corruption -- as well as errors and other forms of either intentional or unintentional under-/over-reporting, leading to an unknown degree and direction of systematic bias -- to prove any points.  Whether we like it or not, the policies surrounding Mexico's drug war are likely going to have to proceed based on subjective values and untested hypotheses, rather than a path paved with objective evidence.