Arrested for bigamy, broken headlight and inadequate windshield wipers. Does the NYPD know this data is public?

TLDR: On https://nyponline.org you can change the url parameters and receive things like disceplenary entries, arrests made with dates for entire career and maybe more. I made a twitter handle https://twitter.com/NYPDdata to tweet out the NYPD arrests made.

Reminder, NYPD officers are not your neighbors, thats a problem.

Intro

I was reading the ProPublic article on the NYPD (https://projects.propublica.org/nypd-disciplinary-records/) and saw this link to https://nypdonline.org/link/1026 a nypd website.

Its pretty cool, NYPD is providing a good amount of info on officers like rank, command history, training summary and other items.

You can see that for each tab the query is a api endpoint Request URL: https://oip.nypdonline.org/api/reports/13/datasource/list

I iterated through all the filter lists, here are some interesting items that don’t show up in the UI. It’s unclear if the NYPD knows that this data is also public.

Since 2018 NYPD has been publishing arrest data quarterly but that data does not include information on the arresting officer https://data.cityofnewyork.us/Public-Safety/NYPD-Arrest-Data-Year-to-Date-/uip8-fykc so there seems to be some new data here.

Examples for a officer LINK TO EXAMPLE FOLDER

Filters:

LIST_OF_FILTERS = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,1027,1028,1029,1030,1031,1032,1033,1034,1035,1036,1037,2041,2042]

  1. Basic Profile Info, Ethnicity, Shield No,
  2. Some kind of summary of command value and unspecified value.
  3. List of arrests per year
  4. Data for some kind of visualization involving arrests and misdemeanor information
  5. Data for some kind of visualization, unclear
  6. Another list of arrests per year
  7. Some unknown value for command, PERMANENT is included in each json value
  8. Says ArrestYear but a single value
  9. List of all arrests the officer has ever made, just date and type of arrest
  10. Some kind of history of a value for the officer, the values start with a PROB like 22 MONTHS PROB and then end with ANNUAL
  11. Another visualization of ArrestYear
  12. Unknown value of NO or YES with a date
  13. Awards for EXCELLENT POLICE DUTY
  14. Similar to 13
  15. Another @ArrestYear but with different values. These seem to be some search of different items on Arrest per year. Perhaps type of arrest or characteristics of arrest.
  16. Similar to above
  17. Similar to above, all with different values. Must be some kind of filter on Arrests
  18. Static image
  19. Just name of officer and image
  20. Value PERMANENT with date
  21. When they were promoted to current command
  22. Some record of officer discipline perhaps? Items like INTERNAL INVESTIGATION, CHRONIC SICK-A, DEPT. AUTO ACCIDENT,
  23. Promotion history
  24. Really long list of trainings: CUSTOMER SERVICE PROGRAM: ROLL CALL MODULE 1, INTRODUCTION TO PRECINCT GREETER PROGRAM , INVESTIGATIVE ENCOUNTERS-VIDEO #4 DOCUMENTATION AND SUPERVISION-UNIT TRAINING JULY 2016, COBRA REFRESHER TRAINING (THREAT)
  25. List of events or trainings attended:POLICING A DEMOCRACY JULY 03. JUNE 07
  26. List of arrest types but all with the same value, not sure what this is
  27. “Search Instructions go here”, “Interactions”: [], “RelatedItems”:
  28. More detailed courses:DETECTIVE BUREAU RETRIEVING BODY WORN CAMERA DATA, COUNTERTERRORISM U.A.S. (UNMANNED AIRCRAFT SYSTEMS) DRONE AWARENESS,IPHONE AND DAS MOBILE OPERATION, LOCATING APPLE DEVICES USING YOUR DEPARTMENT SMARTPHONE
  29. Another “@ArrestYear” with different values
  30. Unfilled Event details
  31. Related to FOIL requests? List of values “FOIL 4234” with dates
  32. FOIL & Discipline Documents is the key but no values
  33. Some graph that is common across all officers: Lists things like FEMALE (6,191), MALE (26,539), NON-BI (1)
  34. Some graph based on ethnicity same across all officers: Lists things like Asian 3055, BLACK 4933, NATIVE AMERICAN 23
  35. empty
  36. Active Disciplinary Cases = 100, same across all officers
  37. VIOLATION, MISDEMEANOR, FELONY with values

CSV of Arrests

I parsed through all officers and saved all raw requests for each filter. You can see the data HERE . I also parsed all the arrest data into a single CSV which you can see HERE ( caution 5 Million rows, you will have to re-append the csv files together)

For CSV, this is the headers. There is data from way back in 1982.

Epoch_time_of_arrest,string_date_of_arrest,day_of_week,holiday,arrest_type,officer_name_without_comma,officer_tax_id,officer_race,officer_appointment,
officer_tenure,officer_assignment,officer_command,officer_rank,officer_shield,officer_substantiated_complaints,officer_unsubstantiated_complaints

Example data

378709200, 1/1/1982 12:00:00 AM, Friday, New Year’s Day, ROB-1ST:FORC THEFT/DEADLY WEAP, SOHAIB RAFI, 942750, ASIAN, 7/10/2006 12:00:00 AM, 14, 2/18/2021 12:00:00 AM, UNIFORMED OPERATIONS UNIT, DETECTIVE 2ND GRADE, 1044, 0, 0

378709200, 1/1/1982 12:00:00 AM, Friday, New Year’s Day, ROB-1ST:FORC THEFT/DEADLY WEAP, SOHAIB RAFI, 942750, ASIAN, 7/10/2006 12:00:00 AM, 14, 2/18/2021 12:00:00 AM, UNIFORMED OPERATIONS UNIT, DETECTIVE 2ND GRADE, 1044, 0, 0

Complaints are from the CCRB data.

Arrest Types

Some interesting items were the list of arrests by type and frequency. See HERE . There are 2100 things people were arrested for!!

The top

395723 CRIM POSS MARIHUANA-5TH:PUBLIC
381779 ASLT W/INT CAUSES PHYS INJURY
287489 INTENT/FRAUD OBT TRANS W/O PAY
279440 CRIM POSS CONTRL SUBST-7TH
246144 PETIT LARCENY
200643 VIOL OF LOCAL LAW VIOL
149816 CRIM POSSESSION STOLN PROP-5TH
145830 AGGRAVATED UNLIC OPER/MV-3RD
131656 AGGRAVATED UNLIC OPER VEH-3RD
109170 CPCS-3RD:NARC DRUG INT/SELL

WOW. Marijuana. Jumping turnstiles!!!

Jumping the turnstile is the third most common.

Ones I found interesting

13676 TRADEMARK COUNTERFEITING 3RD
8458 US CODE UNCLASSIFIED
7214 PROMOTING GAMBLING-2ND
6379 UNAUTHORIZED SALE OF FARECARD
6125 POSS/SELL UNSTAMPED CIGARETTES
5504 PUBLIC LEWDNESS
5222 MV LICENSE VIOL:NO LICENSE
5097 UNLAWFUL ASSEMBLY
2637 FORGERY-3RD
2252 UNLICENSED BOTTLE CLUB
2146 URINATE IN PUBLIC
1644 AUTO STRIPPING-3RD
1531 EQUIP VIO:WINDSHIELD TINT VIOL
1487 CPCS-4TH:HALLUCINOGEN
1435 UNAUTH PRACT PROFESSION
999 RIOT-2ND
864 FAILURE TO COMPLY WITH SIGN
840 POS GAMB DEVICE:SLOT MACHINE
689 INADEQUATE OR NO STOP LAMPS
643 INCITING TO RIOT
558 REFUSAL TO TAKE BREATH TEST
558 EQUIP VIO:HEADLIGHTS
495 JOSTLING HAND NEAR POCKET
400 BICYCLE OPER ON SIDEWALK
330 LOITERING
240 SEAT BELT VIOLATION-DRIVER
235 BICYCLE:FAIL TO KEEP RIGHT
202 UNAUTH RECORDING PERFORMCE-2ND
158 MFG DANGEROUS INSTRUMENT
85 SELL TIX W/O PRINTED PRICE
80 BRIBERY PUBLIC SERVANT-2ND
78 DISRUPT REL SER/FUNRL/MEMORIAL
64 BRIBING A WITNESS
48 ILL SIGNAL:LESS THAN 100 FEET
35 EAVESDROPPING
29 UNAUTH BOTTLING OF ALC BEV
29 CEMETERY DESECRATION-1ST DEG
24 TRANSPORT UNAUTH RECORDING-2ND
14 FORTUNE TELLING
10 LOG BOOK VIOLATIONS
10 INADEQUATE WINDSHIELD WIPERS
8 OPER PRIV MV W/O US MAIL SIGN
8 CONSPIRACY TO FORM MONOPOLY
7 TAKING BRIBE FOR PUBLIC OFFICE
7 ON PREMS RECORDKEEPING VIOL
6 LITTER RAILROAD RIGHT-OF-WAY
6 ABORTION 2ND
5 ILL OP SNOMOB
4 OWNING/HARBORING UNLICEN DOG
3 RADIO/TV/MOVIES PROCEEDINGS
2 TRUSTEE VOTE
2 FAIL TO CHANGE ADDRESS ON LIC
2 EQUIP VIO:SPLASH GUARDS
2 EQUIP VIO:NO DOOR HANDLES
2 ABORTION-1ST
1 UNLAW SALE BICYCLES
1 SNOWMOBILE - IMPRUDENT SPEED
1 KEEPING UNVACCINATED DOG
1 FAIL TO MAINTAIN PROPER FILES
1 BIGAMY
1 ANARCHY-PUB/DIST DOCUMENT

These are listed as arrests. But does this really mean someone was arrested for not having door handles? Or not wearing a seatbelt? Why are these arrests and not tickets? There are millions of tickets given each year in NYC, so these cant be tickets.

I tried cross referencing with the posted arrest data, for instance on 3/16/2021 in this data there is a arrest for Windshield Tint Violation. In the official NYPD quarterly releases, there is a 3/16/2021 arrest for TRAFFIC,UNCLASSIFIED INFRACTIO. ‘TRAFFIC,UNCLASSIFIED INFRACTIO’ never appears in the nypdonline dataset. So is someone doing some data ‘cleaning’? Why remove the data about tinting?

Twitter Account

Posts everyday https://twitter.com/NYPDdata the arrests from 2 weeks ago. I could move it to arrests from yesterday, but I think sometimes arrests are added later.

I could add more info like about how it compares historically etc.

Some Interesting Graphs


Arrests over time from Currently Employed NYPD Officers



Just last 10 years of arrests for current employed



Last 4 years for currently employed



Last 1 year of arrests over time



Arrests by day of week. I was surprised by this. Without any real knowledge I would assume that arrests were much higher on weekends.



Arrests by holiday


Graphs about arrest type


Marijuana arrests going down over time



Drug arrests



Controlled Substance



The number of arrests for Petit Larceny shot up in mid 2013. I’m really curious why!



Arrests for bicycle related things









Weird daily spikes for graffitti arrests


What about arrests by weekday for type of arrest








So basically, people going to work/school during the week are arrested for jumping turnstile.




What about individual officers arrests?


How often do officers make more than 1 arrest per day. Who is arresting 200 people in a day? Data issue?


Diving in, says that on 04/25/2018, tax id 929600 (https://oip.nypdonline.org/view/1/@TAXID=929600/) arrested 196 people. Seems like this officer is part of the special fraud squad and arrested them for GRAND LARCENY 3RD DEGREE.


Arrests per year in 2019



Log graph of above



Occurrences of arrests per day 1 or greater



How many days make an arrest in 2019 per officer



Percentage of officers who make arrests on X days for value Police Officer



Percentage of officers who make arrests on X days for value Detective












Tenure and arrest frequency







Next Ideas

Code

None polished. All hacked together with bugs. Github

comments powered by Disqus