Chapter 9 08-311

9.1 Intro

This report explores the NYC 311 dataset. New York City’s 311 system provides residents with a recourse to report quality-of-life issues, making it possible to understand neighborhood needs, and public concerns to address across New York City.

In this project, we explore:

Which boroughs have the most 311 complaints
What the most common complaint types are
How these patterns differ by borough
The reasons behind common complaints

👉 Go to App

cache_file <- "data/nyc311_10000.csv"

get_311_api <- function(limit = 10000) {
  endpoint <- "https://data.cityofnewyork.us/resource/erm2-nwe9.json"
  resp <- httr::GET(endpoint, query = list("$limit" = limit))
  if (httr::status_code(resp) != 200) stop("NYC Open Data request failed.")
  
  jsonlite::fromJSON(httr::content(resp, as = "text"), flatten = TRUE) %>%
    as_tibble() %>%
    dplyr::select(borough, complaint_type, descriptor) %>%
    dplyr::mutate(
      borough = ifelse(is.na(borough) | borough == "", "Unspecified", borough)
    )
}

data_311 <- tryCatch(
  {
    df <- get_311_api(10000)
    dir.create("data", showWarnings = FALSE)
    readr::write_csv(df, cache_file)
    df
  },
  error = function(e) {
    if (file.exists(cache_file)) {
      readr::read_csv(cache_file, show_col_types = FALSE)
    } else {
      stop("Could not download NYC 311 data and no cached file exists at: ", cache_file)
    }
  }
)

9.2 Below is the first 20 rows of the dataset:

9.3 Here is a summary of total complaints per borough

borough_counts <- data_311 %>% 
  count(borough, sort = TRUE)

knitr::kable(
  borough_counts,
  caption = "Number of NYC 311 complaints by borough in a sample of 10,000 requests."
)

Table 9.1: Number of NYC 311 complaints by borough in a sample of 10,000 requests.
borough	n
BRONX	3455
BROOKLYN	2494
QUEENS	1945
MANHATTAN	1617
STATEN ISLAND	480
Unspecified	9

Figure 9.1: Total number of NYC 311 complaints by borough in a sample of 10,000 requests.

9.4 Interactive Shiny App

9.5 Based on the most recent 10,000 NYC 311 complaints:

Bronx (2,541 total complaints) has the most complaints about noise.
Brooklyn (2,681) has the most complaints about illegal parking.
Manhattan (2,077) has the most complaints about noise.
Queens (2,412) has most complaints about illegal parking.
Staten Island (284) has most complaints about illegal parking.
Unspecified (5) is mostly taxi complaints.

Overall, Brooklyn had the highest number of complaints compared to the other boroughs.

Most complaint types have more specific reasons stored in the descriptor column. It is critical to know the exact reason why complaints are being made so that New York City can address the issues properly and serve our people.

Here, we see the top 3 complaint types overall (heat/hot water, noise and parking violations), and then find the most common reasons associated with them.

top_types <- data_311 %>% 
  count(complaint_type, sort = TRUE) %>% 
  slice_head(n = 3) %>% 
  pull(complaint_type)

knitr::kable(
  data.frame(Top_Complaint_Types = top_types),
  caption = "Top three NYC 311 complaint types in the 10,000 request sample."
)

Table 9.2: Top three NYC 311 complaint types in the 10,000 request sample.
Top_Complaint_Types
Noise - Residential
HEAT/HOT WATER
Illegal Parking

top_reasons <- data_311 %>%
  filter(complaint_type %in% top_types) %>%
  count(complaint_type, descriptor, sort = TRUE) %>%
  group_by(complaint_type) %>%
  slice_max(n, n = 5) %>%
  ungroup()

knitr::kable(
  top_reasons,
  caption = "Most common descriptors associated with the top three NYC 311 complaint types."
)

Table 9.2: Most common descriptors associated with the top three NYC 311 complaint types.
complaint_type	descriptor	n
HEAT/HOT WATER	ENTIRE BUILDING	1390
HEAT/HOT WATER	APARTMENT ONLY	743
Illegal Parking	Blocked Hydrant	350
Illegal Parking	Blocked Sidewalk	205
Illegal Parking	Posted Parking Sign Violation	139
Illegal Parking	Commercial Overnight Parking	65
Illegal Parking	Double Parked Blocking Traffic	63
Noise - Residential	Loud Music/Party	2042
Noise - Residential	Banging/Pounding	425
Noise - Residential	Loud Talking	86
Noise - Residential	Loud Television	18

top_reasons %>% 
  ggplot(aes(x = n, y = fct_reorder(descriptor, n))) +
  geom_col(fill = "steelblue") +
  facet_wrap(~ complaint_type, scales = "free_y") +
  labs(
    title = "Top Reasons for Common Complaint Types",
    x = "Number of Complaints",
    y = "Descriptor"
  ) +
  coord_flip()

Figure 9.2: Most common reasons associated with the three most frequent NYC 311 complaint types.

We can see that:

For heat and hot water complaints, the majority of issues were being reported for an entire building, rather than a single residence, showing widespread service outages need to be adressed.
For illegal parking complaints, most were related to blocked fire hydrants.
For noise complaints, most were residential complaints reporting loud music from parties.

This not only show how many complaints there are across New York City, but starts to answer the questions: What are people complaining about, and how can we address these issues?

This knowledge can help us put programs into place to help provide NYC residents with the help they need to improve quality of life.