About data.police.uk

Contents

General Information

Data Download

The latest complete copy of all the data described on this page can be downloaded in CSV format from the archive page of this site or from data.police.uk/data/archive/latest.zip.

URLs are structured consistently in the following format so you can download newer versions programmatically each month.

http://data.police.uk/data/archive/[year]-[month].zip

Because the data file is over 19 million rows, we also provide Custom CSV Download and JSON API helper interfaces so you can easily access subsets of the data. Generally, we recommend working with one of these interfaces instead of the full data dump.

Data Provenance

The data on this site is published by the Home Office, and is provided to us by the 43 geographic police forces in England and Wales, the British Transport Police, the Police Service of Northern Ireland and the Ministry of Justice.

Data Flow

Every month each police force generates a Crime and ASB file and a Police Outcomes file in a set format. The forces upload these to a private server managed by the Home Office in the Government network, where the files undergo quality assurance.

Copies of the data from police forces is then sent to the Ministry of Justice (MoJ), where they try to match the crimes with any court results contained in their own records. The MoJ send any matching court results back to the Home Office, where they are integrated with the existing data.

All data is then anonymised before being published to this site.

Police Force Systems and Data Extraction

To fully understand the provenance of the data, it's helpful to explain the source IT systems typically used by police forces, and how they relate to the data we publish here.

System Description Typical Data Inputters Provides Data For
Incident Management/Command and Control Used for recording of all incidents handled by the force, including road traffic collisions, ASB and crime reports, and public enquiries. Call-handling and Dispatch staff. Crime and ASB file, Court Result matching
Crime Management If an incident is confirmed as a crime, a record is created on the Crime Management system. This is then used to maintain a record of the subsequent investigation. Crime Recording Bureau staff and investigating Officers. Crime and ASB file, Police Outcomes file, Court Result matching
Custody Used to record the details of any arrests in relation to a crime. Custody staff and investigation Officers. Police Outcomes file, Court Result matching
Case Management If someone is charged for a crime, a case file is prepared ready for the court hearing. This system helps track the preparation of the case and, in some forces, the court result. Criminal Justice Unit staff. Court Result matching

In some police forces these four systems integrate tightly with each other. In other forces they operate as distinct, siloed systems, with limited or no connectivity between them.

When the systems are siloed, there is typically more work required for forces to produce the data and a higher risk of quality issues due to manual double-keying on the source systems.

Because of the differences in IT systems throughout the police forces, the exact process for creating the two files varies. Generally speaking though, a force statistician will either:

Crime and ASB File

This file contains a record of all the Crime and ASB incidents in the previous month. The data typically comes from a mixture of Crime Management systems and Command and Control systems.

The file that forces upload to the Home Office contains the following fields:

It also contains a variety of other reference numbers, which are used as part of the internal Court Result Matching process, but which are not published.

The data then undergoes quality assurance and anonymisation.

Police Outcomes File

This file contains status updates for crimes where the status update happened in the previous month. It's common for this file to contain updates for crimes that originally occurred many months in the past.

It contains the following fields:

It also contains a variety of other reference numbers, which are used as part of the internal Court Result Matching process, but which are not published.

After upload, the outcomes are joined up with the original crime based on the Offence Reference field.

The data then undergoes quality assurance and anonymisation.

Court Result Matching

A copy of all the data relating to crimes where someone has been charged and sent to court is then transferred securely to the Ministry of Justice (MoJ).

The MoJ try to match the crime details against all their court result records to find any relevant court outcomes. The matching process is described in the Understanding the Justice Outcome Data on the police.uk website PDF document provided by the MoJ. Match rates are available to download.

For about information and answers to frequently asked questions about Court Results, please see police.uk/about-this-site/faqs/

Privacy and Anonymisation

Trying to find a balance between providing granular crime data and protecting the privacy of victims has been one of the biggest challenges involved in releasing this data.

We consulted heavily with the Information Commissioner's Office and Data Protection specialists in the Home Office in the run up to releasing this data, and worked within their guidance to create an anonymisation process which adequately minimises privacy risks whilst still meeting our transparency goals and being useful to the public.

We continue to work with the ICO and data users to improve the anonymisation processes and make sure that we have the balance right. You can see some of the improvements that we've made over the previous years on the changelog page.

Crime and ASB Anonymisation

The data in the Crime and ASB file uploaded by forces each month contains sensitive personal data and has to be anonymised before publication. The table below summarises the anonymisation process.

Field Raw Example Anonymisation Method Anonymised Example Published in CSVs Published in API
Offence Reference C2/0123/13 Replaced with a unique one-way hash of the record. 7512cb691a6f9c6c​47a2cdfdcd0a1f87​35a0870d3a2bde21​d8311bd74f17eeeb Crime ID persistent_id
Date 2013-07-23 Truncated to show the year and month only. 2013-07 Month date
Home Office Offence Code 104/25 Assigned into one of 15 categories. A complete mapping between Home Office Offence Codes and Categories can be downloaded here Violence and Sexual Offences Crime Type category
Easting 519500 Anonymised in line with the location anonymisation process. Converted to WGS84 latitude. 52.58019 Latitude latitude
Northing 299500 Anonymised in line with the location anonymisation process. Converted to WGS84 longitude. -0.23782 Longitude longitude
Context This was part of the recent public disorder events in the town centre. None This was part of the recent public disorder events in the town centre. Context

Police Outcome Anonymisation

The data in the Police Outcomes file uploaded by forces each month undergoes a small amount of anonymisation before publication.

Field Raw Example Anonymisation Method Anonymised Example Published in CSVs Published in API
Offence Reference C2/0123/13 Not published Not published
Date 2013-07-23 Truncated to show the year and month only. 2013-17 Month date
Category Caution None Caution Outcome type category

Location Anonymisation

The latitude and longitude locations of Crime and ASB incidents published on this site always represent the approximate location of a crime — not the exact place that it happened.

How are crime locations anonymised?

We maintain a master list of over 750,000 'anonymous' map points. Each map point is specifically chosen so that it:

When crime data is uploaded by police forces, the exact location of each crime is compared against this master list to find the nearest map point. The co-ordinates of the actual crime are then replaced with the co-ordinates of the map point. If the nearest map point is more than 20km away, the co-ordinates are zeroed out. No other filtering or rules are applied.

How was the master list of map points created?

In summary, to create the master list of anonymous points, we:

Type of map point Count Count (before privacy filtering)
Street 679089 833913
Sports/Recreation Area 24510 34237
Parking Area 17797 29591
Park/Open Space 14051 20418
Supermarket 5703 7518
Petrol Station 5501 7296
Pedestrian Subway 4570 6173
Shopping Area 3024 4232
Further/Higher Educational Building 1347 2095
Police Station 1083 1605
Hospital 982 1721
Nightclub 824 1109
Bus/Coach Station 816 1141
Theatre/Concert Hall 733 997
Conference/Exhibition Centre 524 781
Airport/Airfield 369 564
Added by Police Force 350 577
Ferry Terminal 203 320
Theme/Adventure Park 106 370
Prison 102 171
Race Track 100 170
Motorway Service Area 79 147

Data Quality

Quality Assurance

The data that police forces provide to the Home Office and Ministry of Justice goes through a rigorous quality control process, involving format validation, automated testing, and manual verification and approval by two separate people.

If a problem with the data is spotted, the team at the Home Office will ask the force to fix and re-upload the data before it is published.

Format Validation

At the point of upload the following checks are carried out on the Crime & ASB and Police Outcomes files. The upload is rejected if any of the checks fail.

Manual Verification

After the upload has been completed, a summary of the data and potential issues are provided to the force and to the Home Office team. Both parties must review the data and approve it as suitable for publication.

This manual check includes assessing:

Automated Testing

After the automated tests have been completed, a final round of automated testing is carried out. If any anomalies are spotted, these are flagged to the team at the Home Office to investigate.

The automated tests run include such checks as:

Known Issues

There are some major, difficult to fix, issues with the data published on this site.

Sometimes there are other specific known issues that arise. For example, a force might be in the middle of upgrading their Incident Management system, making them unable to extract and provide ASB data for the latest month. In most cases, these known issues are temporary and fixed in the following month's publication. Any current known issues will be documented on the download page.

Reporting Issues

If you find any issues or anomalies with the data, please report it to us via our contact form, selecting 'A problem with the data' as the subject. Alternatively, you can email us at contact@support.police.uk.

Fixing Issues

When a problem with the data has made it into production, we ask the relevant force to fix it by re-uploading and overwriting the erroneous data.

Most of the time these fixes are made in the next monthly data publication. If the force doesn't have the relevant people available to fix it or the root cause of the problem lies in the source IT system, it may take longer to rectify.

You can view all data changes and fixes on the changelog.

Data Verification

All the zip files published on the archive page have a 32-character MD5 hash displayed next to them (for example: a0f2a3c1dcd5b1cac71bf0c03f2ff1bd).

You can verify that the zip file has downloaded correctly by calculating the MD5 hash of your downloaded file and comparing it to the one shown on the site.

On Mac OS X, Linux and other Unix-based systems you can use md5sum from your terminal.

On Windows, you can use the built-in File Checksum Integrity Verifier from your command line or a third-party tool such as the md5summer software.

The Home Office does not provide support for these utilities. You use these utilities at your own risk.

Tools and Libraries

You may find the following software and libraries useful when working with the data made available on this site. All are available at no cost and are open-source.

The Home Office does not provide support for these utilities. You use these utilities at your own risk.

Working with the API

Working with the CSV files

Working with the KML files

Contact Details

If you have and questions about the data, suggestions for improvements, concerns about the disclosure or personal details, or have noticed any errors in the data, please get in touch with us via the contact form or by emailing contact@support.police.uk.

If you would like to stay up-to-date with news, updates and any data corrections, please subscribe to our mailing list.