The SpotCrime Open Crime Standard (SOCS)

There are already methods for agencies to share information among each other like N-Dex, NEIM, GJXDM. But what about sharing information openly with the public?

Currently, Baltimore shares data differently than Baltimore County who shares it differently than Annapolis, MD who shares it differently than Arlington, VA who shares it differently than Dallas, TX who shares it differently than LA who shares it differently than Vancouver, BC who shares it differently than London, England. Until now.

We’ve created the SpotCrime Open Crime Standard (SOCS) to encourage police agencies throughout the world to keep crime public data uniform, simple, and ubiquitous. We’ve also created the standard to set guidelines for making public crime information open and accessible to the public.

Similar to LIVES with health inspection data or GTFS with transit data, the SpotCrime Open Crime Standard (SOCS) will allow agencies to report the same kind of information to the public openly, causing less confusion from agency to agency.

‘Open’ data means the information is available to anyone in machine readable format without restriction on ability to use, consume, or share.

We believe applying this standard will not only help agencies collaborate, but will help close the gap between desire and ability for developers and residents. It will also allow for local crime data to be easier understood at any level whether it’s locally, countywide, regionally, statewide, nationally, or internationally.

Some additional benefits of standardizing and releasing data openly includes saving on costs associated with mundanely responding to public records requests and inquires by interested residents or the media. It will also eliminate costs associated with hiring a vendor to display information to the public. And, most importantly, it will allow developers to write apps consuming the data in a ‘pretty’ format (think crime maps!) - for free! Philadelphia is a great example of a city that releases crime data and has received multiple free crime mapping apps in return.

We think this is a work in progress, but feel that the process needed to be initiated.

Check out the standard below. Let us know what you think.

What Data Should a Police Agency Release?

An agency should release anything that would appear on a Daily Incident Log or Call Log (public information only). Most departments pull something from their RMS. SpotCrime offers Catapult to any agency who doesn’t have the ability to create a public file from their RMS.

We typically ask for Part I and II crime information since most agencies already compile this information for their UCR or NIBRS reporting. Our motto is the more information (as long as it’s public), the better. Our system can decipher and define an array of crime types, as well as sort through what information is important and what information is useless. Information typically not released is victim information, juvenile information, or information pertaining to an arrestee’s personal information.

The Data Set should include:

Data type	Required	Permitted Values	Examples	Details
Area	Yes	Text	Baltimore County, MD	Defines the city/county/jurisdiction and state
Date	Yes	ISO 8601; YYYY-MM-DD	2014-03-10	Date occurred. Allows easy identifier and separation of data. Date displayed in mm/dd/yyyy on SpotCrime.
Time	Yes	ISO 8601; UTC and 24 hour	23:11	Time occurred. Allows easy identifier and separation of data. Displayed in 12 hour format on SpotCrime.
Incident Type	Yes	Text	Aggravated assault	Allows icons to be assigned and decipher between crime types. SpotCrime dictionary categorizes each incident into 1 of 9 icons (listed below).
Location	Yes	Street number and street name.	322 York Road	Full and block addresses are accepted. Needed for geo-coding accuracy. All full addresses are masked to the block level on SpotCrime.
City	Yes	Text	Towson, MD	Needed for geo-coding accuracy.
Narrative	Yes	Text	A victim was stabbed in the abdomen by an unknown subject after a verbal altercation	Provides more description and information on incident type (commercial burglary or residential burglary). Displayed in the details section on SpotCrime.
Latitude	Yes	Geographic coordinate system	39.399262	Geo-coding accuracy. Not displayed by SpotCrime.
Longitude	Yes	Geographic coordinate system	-76.602990	Geo-coding accuracy. Not displayed by SpotCrime.
Case Number	Yes	Text and number accepted	2014-031023111	Helps identify specific incidents. Displayed in the details section on SpotCrime.
Incident Number (if different from case number)	Optional, highly recommended	Text and number accepted	2014-00032908	Helps identify specific incidents. Displayed in the details section on SpotCrime.
Beat/ Sector	Optional	Text and number accepted	6A	Displayed in the details section on SpotCrime.
District/ Precinct	Optional	Text and number accepted	Towson	Displayed in the details section on SpotCrime.
ZIP Code	Optional	US and international postal codes accepted	21204	Helps with geo-coding accuracy. Displayed in the details section on SpotCrime.

File type:

A lot of police agencies already generate a public file so as long as it holds the data required above and is in a machine readable format (meaning it can be easily processed by a computer). We also recommend that the data should be downloadable in bulk and publicly displayed on the agency’s or city’s own website. For example, Catapult creates a CSV file.

Machine readable formats that fall under the Open Data definition and are accepted under SOCS:		Formats that do not fall under the Open Data definition and are not accepted under SOCS:
XML		PDF
RSS feed		HTML
CSV		DOC(X)
RDF		Anything Scanned
JSON		Anything Faxed
TXT		Anything typed in an email
XLS(X)
KML

Frequency:

Daily.
Expectations should be set high. Frequency of weekly or longer defeats the purpose of open data.

Open Crime Data XML:

<OpenCrimeData

 Version="0.1"  # OpenCrimeData standard version.

 Language ="en" # Language data is published in.

 ReportingAgency="NYPD" # Reporting agency.

 ReportingAgencyURL="http://www.nyc.gov/html/nypd" # Reporting agency URL.

 ReportingTimeOffset="-5" # Indicates local time zone offset of the reporting agency.

 PubDate="Date in ISO 8601 format"

>

<CrimeData>

 <Crime>

   <CaseNumber></CaseNumber>

   <IncidentNumber></IncidentNumber>

   <Date>Crime date in ISO 8601 format</IncidentDate>

   <Time>UTC Time in ISO 8601 format</IncidentTime>

   <Type>Crime Type (not necessarily Spotcrime type).</Type>

   <Description>Crime description</Description>

   <Location>

     <Address>street address</Address>

     <Locality>city</Locality>

     <Region1>state</Region1>

     <Region2>{might be used in some countries}</Region2>

     <PostalCode>Zip</PostalCode>

     <Country>Country, ISO 3166-1 alpha-2</Country>

   </Location>

   <GeoLocation>

     <Latitude></Latitude>

     <Longitude></Longitude>

   </GeoLocation>

 </Crime>

 <Crime>

   ...

 </Crime>

</CrimeData>

</OpenCrimeData>

SpotCrime Classifications:

SpotCrime has classified crime incidents into 9 categories and what we believe to be the simplest and most straightforward fashion. Agencies are encouraged to provide all of their crime types and let our dictionary take care of classification.

Classification	Definitions	Examples of crime types covered
Assault	A physical attempt or threat to use violence with the intent to do harm to another.	Simple, Aggravated, Rape, Murder, Homicide, Sexual, Domestic, Intimidation, Kidnapping
Shooting	The act of firing a weapon in order to hit, wound, or kill someone or something.	Discharged firearm, shooting
Robbery	Using force or intimidation to take property away from another person in the presence of that person.	Armed, Carjacking, Commercial, Individual
Arson	The crime of intentionally setting fire to a building or property of another or the burning of one's own property to collect insurance.	Arson
Burglary	The criminal offense of breaking and entering a building illegally for the purpose of committing a crime in that building.	Business, Residential, Breaking and Entering
Theft	The act in which property belonging to another is taken without that person's consent.	Larceny, Motor vehicle, Shoplifting, Forgery, Counterfeiting, Embezzlement, Fraud, Stolen Property
Vandalism	The intentional destruction of or damage to the property of another.	Destruction of Property, Criminal Mischief
Arrest	The seizure of an alleged or suspected offender to answer for a crime.	DUI, Drug Violations, Prostitution
Other	Any incident that does not fit within the other definitions	Public Drunkenness, Runaways, Loitering, Weapons Offense, Missing Person

Examples of feeds already in existence:

City	Description of Dataset	Link
Philly	Crime incidents from the Philadelphia Police Department. Part I crimes include violent offenses such as aggravated assault, rape, arson, among others. Part II crimes include simple assault, prostitution, gambling, fraud, and other non-violent offenses. This dataset previously had separate endpoints for various years and types of incidents. These have since been consolidated into a single dataset. Trouble downloading or have questions about this City dataset? Visit the OpenDataPhilly Discussion Group	https://www.opendataphilly.org/dataset/crime-incidents
Denver	This dataset includes criminal offenses in the City and County of Denver for the previous five calendar years plus the current year to date. The data is based on the National Incident Based Reporting System (NIBRS) which includes all victims of person crimes and all crimes within an incident. The data is dynamic, which allows for additions, deletions and/or modifications at any time, resulting in more accurate information in the database. Due to continuous data entry, the number of records in subsequent extractions are subject to change. Crime data is updated Monday through Friday.	http://data.denvergov.org/ dataset/city-and-county-of-denver-crime
San Francisco	Incidents reported via the SFPD CABLE System, from 1/1/2003 to present. In KML format (past 90 days) In Shapefile format (by year) In CSV format (all: 800,000 plus records).	https://data.sfgov.org/Public-Safety/Police-Department-Incident-Reports-2018-to-Present/wg3w-h783
Chicago	This dataset reflects reported incidents of crime (with the exception of murders where data exists for each victim) that occurred in the City of Chicago from 2001 to present, minus the most recent seven days. Data is extracted from the Chicago Police Department's CLEAR (Citizen Law Enforcement Analysis and Reporting) system. In order to protect the privacy of crime victims, addresses are shown at the block level only and specific locations are not identified. Should you have questions about this dataset, you may contact the Research & Development Division of the Chicago Police Department at 312.745.6071 or RandD@chicagopolice.org.	https://data.cityofchicago.org/ Public-Safety/Crimes-2001-to-present/ ijzp-q8t2
Albuquerque	http://data.cabq.gov/publicsafety/policeincidents/ MetaData.pdf/view	http://www.cabq.gov/abq-data/

This post was edited on 5/29/15. To read about the edits we made, click here.