Problems with the crime data
Unfortunately, the data released each month from the 43 Police Forces is not perfect and in some cases, far from it. The data contains notable omissions, data in the wrong field and some very incoherent entries. This requires considerable investment in time and resource for cleaning before we can safely put it into our database and website. If you spot something, please let us know – we can't pay you but we will gladly credit you with your observation, should you want that.
Here is a running list of broad points of what's wrong, questionable or inexplicable with the data, in no particular order:
1. Population data is based on last census in 2001
People come and go, are born and pass away but by and large the population is substantially bigger than the last census of 2001. According to the Office for National Statistics, the population of England and Wales was 54,450,000 in 2008. www.statistics.gov.uk and the next census will presumably show an increase on that. The total population on our database for England and Wales is 52,830,760. The official census figure is 52,041,916 . So there is a difference of 800,000 which we can't account for.
How does this affect www.ukcrimestats.com?
Population data is used to create crime rate statistics. Crime rate statistics are based on crimes per 1,000 residents in a given area over a selected period of time. So these figures are not an entirely accurate reflection of the risk of crime in a given area, because there are in many cases more and different people living in a given area in the present than in the past. And very broadly speaking, often crimes are not committed by criminals in their own area. When the 2011 census data is added in, then this may well show a substantial drop in crime in many areas.
|
|
|
2. Sussex Police did not provided complete crime data for Dec 10 and Jan 11
We don't know why but Sussex Police was the only police force that did not provide neighbourhood crime data in Dec 2010 or Jan 2011. They did start providing it in Feb 2011.
How does this affect ukcrimestats.com?
When police forces do not report crime it can skew results quite substantially and portray an inaccurate crime rate. For example, if we measure trends from Dec 2010 to Feb 2011, Sussex neighbourhoods could show high on reports that show the highest jump in crime from one month to another (becausee they started with zero crime!). Hopefully they, and other police forces, will continue to provide timely and accurate crime numbers so that our reports can be accurate and represent the true crime trends.
3. Crime rates measure static resident populations
It's by no means a perfect measure. One of the early findings of this website is that a lot of crime tends to happen where people are not resident. What really counts is the total throughput of population in a given area as to put it crudely, this creates more opportunities for crime. So for example, John Lennon Liverpool Airport has a total resident population of 1 but has 11 crimes in 1 month (December 2010). That creates a crime rate of 11,000 per 1,000 residents making it the top crime rate neighbourhood in the UK. Whereas actually, if you were able to factor in (and you can't) the total amount of people moving through a given area like John Lennon Airport, you'd probably find that this was one of the lowest crime rate areas in the UK. So crime rates can be distortive.
How does this affect www.ukcrimestats.com?
In order to overcome this distortion, we have suppressed the 221 neighbourhoods (out of a total of 5,526) with a population of 1 or 0 when crime rates are asked for (total crimes are still shown). After that, the next population size for a neighbourhood is 131.
4. Police Ranks
We had to do a lot of cleaning of the Police Rank data. A bit like the different branches of the Armed Forces, for reasons of evolution and history, the 43 Police Forces do not have an identical rank system. However, some really strange data was put in like 'Mr', '1628', 'Mrs', 'Other', 'Default Contract' and then in some cases, a description of the role rather than the rank like 'Neighbourhood Watch Co-Ordinator' or 'Travel Community Support Officer'. In over 2,600 cases no rank was provided.
Different Police Forces can refer to the same rank differently. For example, Special Constable ('SC') is known as 'MSC' in the London Metropolitan. These have been accounted for where known. There is still confusion around the data where the Police Forces have used the acronym of the rank/role e.g. 'PS'. We don't know if this refers to Police Sergeant, or, judging by the large number of these, more likely 'Police Staff' which we've seen used, which probably equates to a Police Constable.
How does this affect www.ukcrimestats.com?
We have narrowed down the ranks to the following; Chief Inspector, Inspector, Sergeant, Police Constable, and PCSO. This is what you will see on our Officers page when you click on the By Rank section. However, we have kept some of the others (e.g. where role is described rather than rank) and kept all of them on as registered officers.
5. Linking Crimes by Street to its Neighbourhood
We can't. Unfortunately the data for the boundaries of each neighbourhood has not been made available. So whilst we have Easting and Northing data for individual street crimes which we can translate into Longitude and Latitude, the data did not include at the same time, which administrative unit, i.e. Neighbourhood it was located in. For now, Street crime and Neighbourhood crime are disconnected data fields and this is beyond our control.
How does this affect www.ukcrimestats.com?
This is a data relationship issue and does not affect the general usage of ukcrimestats.com. However, it prevents us from easily asking questions about street crime data from the direction of the neighbourhood. We must always start with the individual street crime, then fetch the neighbourhood/police force information directly from Police.UK's own API using the latitude/longitude of the street crime. We could, in theory, collect this data ourselves but there are over 321,000 separate streets that have had at least 1 crime reported since we began collecting data, which we'd need to collect API data on. To date we have refrained from hitting the Police.UK API that many times in order to collect this data!
6. Missing data from neighbourhood profiles
2 neighbourhoods are either not showing valid Latitude/Longitude coordinates for their central point, or we cannot reach the profiles due to recent restructuring of neighbourhoods:
| Police Force | Neighbourhood | Code |
| Cleveland Police | Headland and Harbour | Headland and Harbour |
| Cleveland Police | Manor House | Manor House |
How does this affect www.ukcrimestats.com?
Restructuring neighbourhoods and altering their unique identifiers causes considerable problems when reporting data. For example, if 2 separate neighbourhoods merge to form a new neighbourhood, with different boundaries and population, then this would mean the new neighbourhood would have no crime data available prior to the change.
|
|
|
7. Some neighbourhoods have a listed population of 1 or 0
Some neighbourhoods are not reporting their true population size. Some of these population figures may be correct, such as East Midlands Airport with a population of zero. Another explanation is that these neighbourhoods are obsolete and perhaps a result of that Police forces current restructuring of neighbourhood names and boundaries.
If any neighbourhoods are reporting population sizes of 0 or 1 they will be listed below.
| Police Force | Neighbourhood | Code | Population | |
| 1 | Cleveland Police | Headland and Harbour | Headland and Harbour | 0 |
| 2 | Cleveland Police | Manor House | Manor House | 0 |
| 3 | Nottinghamshire Police | Balderton and Fernwood | 31-258 | 0 |
| 4 | Nottinghamshire Police | Farndon & Elston | 31-18 | 0 |
| 5 | Nottinghamshire Police | Hillside | 31-134 | 0 |
| 6 | Nottinghamshire Police | Kirkby In Ashfield Town Centre | 31-65 | 0 |
| 7 | Nottinghamshire Police | Sutton In Ashfield Town Centre | 31-80 | 0 |
| 8 | Sussex Police | Regency Seafront | CC1NH3 | 1 |
How does this affect www.ukcrimestats.com?
Zero neighbourhood population levels create zero neighbourhood crime rates. Populations of just 1 lead to massive crime rates. So both 1 and 0 are a massive distortion when calculating crime rates. We have therefore suppressed the following Neighbourhoods each with a population of one or zero for the display of crime rates, as they will always appear at the top or bottom of the rankings and are not a true and fair representation. The next largest population for a neighbourhood is 131.
8. When crimes merge
There are only 12 types of crime stratified by the data; anti-social behaviour, burglary, other crime (consisting of shoplifting, drugs, criminal damage and arson, public disorder and weapons, other theft), robbery, vehicle crime, and violent crime. But what happens when there is a violent burglary – is this one crime or two? We now have it on good authority that there is an order of hierarchy according to the length of sentence per crime committed. So in this example, if the burglary involved homicide, then this would be registered as a violent crime, not a burglary. But if there was a relatively small and ineffectual violent blow landed by the perpetrator on the victim but a great deal of valuable items were burgled, then it is registered as a burglary. So a combination of different offences are recorded as one crime according to the hierarchy of potential sentencing length.