The SpotCrime Open Crime Standard (SOCS)

There are already methods for agencies to share information among each other like N-Dex, NEIM, GJXDM. But what about sharing information openly with the public?

Currently, Baltimore shares data differently than Baltimore County who shares it differently than Annapolis, MD who shares it differently than Arlington, VA who shares it differently than Dallas, TX who shares it differently than LA who shares it differently than Vancouver, BC who shares it differently than London, England. Until now.

We’ve created the SpotCrime Open Crime Standard (SOCS) to encourage police agencies throughout the world to keep crime public data uniform, simple, and ubiquitous. We’ve also created the standard to set guidelines for making public crime information open and accessible to the public.

Similar to LIVES with health inspection data or GTFS with transit data, the SpotCrime Open Crime Standard (SOCS) will allow agencies to report the same kind of information to the public openly, causing less confusion from agency to agency.

‘Open’ data means the information is available to anyone in machine readable format without restriction on ability to use, consume, or share.

We believe applying this standard will not only help agencies collaborate, but will help close the gap between desire and ability for developers and residents. It will also allow for local crime data to be easier understood at any level whether it’s locally, countywide, regionally, statewide, nationally, or internationally.

Some additional benefits of standardizing and releasing data openly includes saving on costs associated with mundanely responding to public records requests and inquires by interested residents or the media. It will also eliminate costs associated with hiring a vendor to display information to the public. And, most importantly, it will allow developers to write apps consuming the data in a ‘pretty’ format (think crime maps!) - for free! Philadelphia is a great example of a city that releases crime data and has received multiple free crime mapping apps in return.

We think this is a work in progress, but feel that the process needed to be initiated.

Check out the standard below. Let us know what you think.

What Data Should a Police Agency Release?
An agency should release anything that would appear on a Daily Incident Log or Call Log (public information only). Most departments pull something from their RMS. SpotCrime offers Catapult to any agency who doesn’t have the ability to create a public file from their RMS.

We typically ask for Part I and II crime information since most agencies already compile this information for their UCR or NIBRS reporting. Our motto is the more information (as long as it’s public), the better. Our system can decipher and define an array of crime types, as well as sort through what information is important and what information is useless. Information typically not released is victim information, juvenile information, or information pertaining to an arrestee’s personal information.

The Data Set should include:
Data typeRequiredPermitted ValuesExamplesDetails
AreaYesTextBaltimore County, MDDefines the city/county/jurisdiction and state
DateYesISO 8601; YYYY-MM-DD2014-03-10Date occurred. Allows easy identifier and separation of data. Date displayed in mm/dd/yyyy on SpotCrime.
TimeYesISO 8601; UTC and 24 hour23:11Time occurred. Allows easy identifier and separation of data. Displayed in 12 hour format on SpotCrime.
Incident TypeYesTextAggravated assaultAllows icons to be assigned and decipher between crime types. SpotCrime dictionary categorizes each incident into 1 of 9 icons (listed below).
LocationYesStreet number and street name.322 York RoadFull and block addresses are accepted. Needed for geo-coding accuracy. All full addresses are masked to the block level on SpotCrime.
CityYesTextTowson, MDNeeded for geo-coding accuracy.
NarrativeYesTextA victim was stabbed in the abdomen by an unknown subject after a verbal altercationProvides more description and information on incident type (commercial burglary or residential burglary). Displayed in the details section on SpotCrime.
LatitudeYesGeographic coordinate system39.399262Geo-coding accuracy. Not displayed by SpotCrime.
LongitudeYesGeographic coordinate system-76.602990Geo-coding accuracy. Not displayed by SpotCrime.
Case NumberYesText and number accepted2014-031023111Helps identify specific incidents. Displayed in the details section on SpotCrime.
Incident Number (if different from case number)Optional, highly recommendedText and number accepted2014-00032908Helps identify specific incidents. Displayed in the details section on SpotCrime.
Beat/ SectorOptionalText and number accepted6ADisplayed in the details section on SpotCrime.
District/ PrecinctOptionalText and number acceptedTowsonDisplayed in the details section on SpotCrime.
ZIP CodeOptionalUS and international postal codes accepted21204Helps with geo-coding accuracy. Displayed in the details section on SpotCrime.

File type:
A lot of police agencies already generate a public file so as long as it holds the data required above and is in a machine readable format (meaning it can be easily processed by a computer). We also recommend that the data should be downloadable in bulk and publicly displayed on the agency’s or city’s own website. For example, Catapult creates a CSV file.
Machine readable formats that fall under the Open Data definition and are accepted under SOCS:
Formats that do not fall under the Open Data definition and are not accepted under SOCS:
XML
PDF
RSS feed
HTML
CSV
DOC(X)
RDF
Anything Scanned
JSON
Anything Faxed
TXT
Anything typed in an email
XLS(X)

KML


Frequency:
Daily.
Expectations should be set high. Frequency of weekly or longer defeats the purpose of open data.

Open Crime Data XML:
<OpenCrimeData
 Version="0.1"  # OpenCrimeData standard version.
 Language ="en" # Language data is published in.
 ReportingAgency="NYPD" # Reporting agency.
 ReportingAgencyURL="http://www.nyc.gov/html/nypd" # Reporting agency URL.
 ReportingTimeOffset="-5" # Indicates local time zone offset of the reporting agency.
 PubDate="Date in ISO 8601 format"
>
<CrimeData>
 <Crime>
   <CaseNumber></CaseNumber>
   <IncidentNumber></IncidentNumber>
   <Date>Crime date in ISO 8601 format</IncidentDate>
   <Time>UTC Time in ISO 8601 format</IncidentTime>
   <Type>Crime Type (not necessarily Spotcrime type).</Type>
   <Description>Crime description</Description>
   <Location>
     <Address>street address</Address>
     <Locality>city</Locality>
     <Region1>state</Region1>
     <Region2>{might be used in some countries}</Region2>
     <PostalCode>Zip</PostalCode>
     <Country>Country, ISO 3166-1 alpha-2</Country>
   </Location>
   <GeoLocation>
     <Latitude></Latitude>
     <Longitude></Longitude>
   </GeoLocation>
 </Crime>
 <Crime>
   ...
 </Crime>
</CrimeData>
</OpenCrimeData>

SpotCrime Classifications:
SpotCrime has classified crime incidents into 9 categories and what we believe to be the simplest and most straightforward fashion. Agencies are encouraged to provide all of their crime types and let our dictionary take care of classification.
ClassificationDefinitionsExamples of crime types covered
AssaultA physical attempt or threat to use violence with the intent to do harm to another.Simple, Aggravated, Rape, Murder, Homicide, Sexual, Domestic, Intimidation, Kidnapping
ShootingThe act of firing a weapon in order to hit, wound, or kill someone or something.Discharged firearm, shooting
RobberyUsing force or intimidation to take property away from another person in the presence of that person.Armed, Carjacking, Commercial, Individual
Arson



The crime of intentionally setting fire to a building or property of another or the burning of one's own property to collect insurance.Arson
BurglaryThe criminal offense of breaking and entering a building illegally for the purpose of committing a crime in that building.Business, Residential, Breaking and Entering
TheftThe act in which property belonging to another is taken without that person's consent.Larceny, Motor vehicle, Shoplifting, Forgery, Counterfeiting, Embezzlement, Fraud, Stolen Property
VandalismThe intentional destruction of or damage to the property of another.Destruction of Property, Criminal Mischief
ArrestThe seizure of an alleged or suspected offender to answer for a crime.DUI, Drug Violations, Prostitution
OtherAny incident that does not fit within the other definitionsPublic Drunkenness, Runaways, Loitering, Weapons Offense, Missing Person

Examples of feeds already in existence:
CityDescription of DatasetLink
PhillyCrime incidents from the Philadelphia Police Department. Part I crimes include violent offenses such as aggravated assault, rape, arson, among others. Part II crimes include simple assault, prostitution, gambling, fraud, and other non-violent offenses.

This dataset previously had separate endpoints for various years and types of incidents. These have since been consolidated into a single dataset.

Trouble downloading or have questions about this City dataset? Visit the OpenDataPhilly Discussion Group

https://www.opendataphilly.org/dataset/crime-incidents

DenverThis dataset includes criminal offenses in the City and County of Denver for the previous five calendar years plus the current year to date. The data is based on the National Incident Based Reporting System (NIBRS) which includes all victims of person crimes and all crimes within an incident. The data is dynamic, which allows for additions, deletions and/or modifications at any time, resulting in more accurate information in the database. Due to continuous data entry, the number of records in subsequent extractions are subject to change. Crime data is updated Monday through Friday.http://data.denvergov.org/
dataset/city-and-county-of-denver-crime
San FranciscoIncidents reported via the SFPD CABLE System, from 1/1/2003 to present. In KML format (past 90 days) In Shapefile format (by year) In CSV format (all: 800,000 plus records).https://data.sfgov.org/Public-Safety/Police-Department-Incident-Reports-2018-to-Present/wg3w-h783
ChicagoThis dataset reflects reported incidents of crime (with the exception of murders where data exists for each victim) that occurred in the City of Chicago from 2001 to present, minus the most recent seven days. Data is extracted from the Chicago Police Department's CLEAR (Citizen Law Enforcement Analysis and Reporting) system. In order to protect the privacy of crime victims, addresses are shown at the block level only and specific locations are not identified. Should you have questions about this dataset, you may contact the Research & Development Division of the Chicago Police Department at 312.745.6071 or RandD@chicagopolice.org. https://data.cityofchicago.org/
Public-Safety/Crimes-2001-to-present/
ijzp-q8t2
Albuquerquehttp://data.cabq.gov/publicsafety/policeincidents/
MetaData.pdf/view
http://www.cabq.gov/abq-data/

This post was edited on 5/29/15. To read about the edits we made, click here.

Comments

Popular posts from this blog

SpotCrime Weekly Reads: Violent crime, AI tech, transparency

SpotCrime Weekly Reads: AI, police conduct, transparency

SpotCrime Weekly Reads: Body cam, AI, gun violence