Day: October 12, 2023

Show Only ...
Maps - Photos - Videos

Geocoding Using NY Geocoder and Finding NY Political Districts Using R

Geocoding Using NY Geocoder and Finding NY Political Districts Using R

I re-wrote my Python script for coding political districts in NY using R and the ArcGIS Feature Services containing the political districts. This is a much more elegant approach then using Python, as it is self-contained and geocodes 750 records at a time, so large files that contain more then 1,000 records can be used with this script to be geocoded. You could just modify this script for purposes of getting coordinates, and skip the section that figures out what district persons live in.

001library(tidyverse)
002library(httr)
003library(sf)
004 
005rm(list=ls())
006 
007# get district lines from nys gis
011 
012# read file
013df <- openxlsx::read.xlsx('~/Downloads/addresses.xlsx') %>%
014  janitor::clean_names()
015 
016# clean up data, remove " and return characters which will break the
017# json query and choke geocoder, convert to single line
018df <- df %>%
019  mutate(
020    'OBJECTID' = row_number(),
021    'SINGLELINE' = paste(str_replace(address, '-\\d+ ', ' ') %>% str_replace_all("[\r\n]",'') %>% str_replace_all('"', '\\"') %>% str_trim(),
022                         str_replace(municipality,'\\(.*?\\)',''), state, zip_code, sep = ', '),
023  )
024 
025# temporary tibble for storing geocoded addresses
026res_df <- tibble()
027 
028# geocode 750 records at a time
029for (i in seq(1, nrow(df), 750)) {
030   
031  # keep from over-running the dataframe
032  j <- i + 749
033  j <- ifelse(j > nrow(df), nrow(df), j)
034 
035  # build json for query
036  query <- str_c(
037                 '{ "attributes": { "OBJECTID": ', df[i:j, 'OBJECTID'],
038                 ', "SINGLELINE": "', df[i:j, 'SINGLELINE'],
039                 '" } },', collapse = '\n')
040   
041  query <- str_c('{"records": [', query, ']}', sep='\n')
042   
043  post <- list( 'f' = 'pjson', 'outSR' = 4326, 'addresses' =  query )
045   
046  # query state geocoder
047  res <- POST(url, body = post, encode = "multipart")
048 
049  Sys.sleep(5)
050   
051  # extract from json the coordinates
052  res_df <- bind_rows(res_df,
053                      jsonlite::fromJSON(content(res, as='text')) %>%
054                        as.data.frame() %>%
055                        as_tibble() %>%
056                      janitor::clean_names() %>%
057                      unnest(cols = c(locations_location, locations_attributes)) %>%
058                      select(OBJECTID = ResultID, matched_address = Match_addr,
059                             score = locations_score, lng = x, lat = y)
060            )
061 
062}
063 
064# join the found coordinates from state geocoder to df
065df <- df %>% left_join(res_df, by='OBJECTID')
066 
067# spatial join against cd, sd, ad
068df <- df %>%
069  mutate(lat = ifelse(is.na(lat), 0, lat),
070         lng = ifelse(is.na(lng), 0, lng)) %>%
071  st_as_sf(coords = c("lng", "lat"),
072                crs=4326) %>%
073  st_intersection(cd) %>%
074  st_drop_geometry() %>%
075  select(OBJECTID, CD.2022 = DISTRICT, CD.2023.MEMBER = NAME) %>%
076  left_join(df, ., by='OBJECTID')
077 
078 
079df <- df %>%
080  mutate(lat = ifelse(is.na(lat), 0, lat),
081         lng = ifelse(is.na(lng), 0, lng)) %>%
082  st_as_sf(coords = c("lng", "lat"),
083           crs=4326) %>%
084  st_intersection(sd) %>%
085  st_drop_geometry() %>%
086  select(OBJECTID, SD.2022 = DISTRICT, SD.2023.MEMBER = NAME) %>%
087  left_join(df, ., by='OBJECTID')
088 
089df <- df %>%
090  mutate(lat = ifelse(is.na(lat), 0, lat),
091         lng = ifelse(is.na(lng), 0, lng)) %>%
092  st_as_sf(coords = c("lng", "lat"),
093           crs=4326) %>%
094  st_intersection(ad) %>%
095  st_drop_geometry() %>%
096  select(OBJECTID, AD.2022 = DISTRICT, AD.2023.MEMBER = NAME) %>%
097  left_join(df, ., by='OBJECTID')
098 
099# output the excel spreadsheet
100df %>%
101  select(addresses:zip_code, CD.2022:AD.2023.MEMBER, score, lat, lng, matched_address) %>%
102  openxlsx::write.xlsx('/tmp/geocoded_addresses.xlsx')

Jill Stein Part II

Some people are surprised that I voted for Jill Stein on the Green line for President. I voted for Working Families candidates down the rest of the line. Most of the races weren’t particularly competitive or interesting, but I wanted to sending a message that I want candidates in office that are working for average people.

The last three Presidential Elections I’ve voted for have been for Democrats β€” John Kerry, then Barack Obama’s two terms β€” all on the Working Families Party line. But I certainly don’t always vote for Democrats. In retrospect, I think I would have voted for George Bush on Conservative line in 2004, as I thought he showed a lot of courage during the start of the Great Recession, and I approving his 2007 energy bill did much to move our nation’s energy policies forward, including the strong new fuel economy standards, lighting energy standards and other efficiency standards that will save millions of megawatts and billions of gallons of fuel.

But I’ve always been somebody to split tickets and vote on third party lines …

On the state level In 2002, I voted for Tom Golisano for Governor who had the Independence line. In 2006, I voted for Elliot Spitzer, a Democrat for Governor then in 2010 I voted for Carl Paladino on the conservative line and also in 2014, I voted for Rob Astorino on conservative line. I’m not a real big conservative, but I do like the idea of limiting the role of government and checking the power of governor. I’ve never liked a certain governor, ever since he served in lower levels of government, nor his poorly thought out, SAFE Act. Down ballot, I tend to vote overwhelmingly Democratic, as I think we want local politicians that are more environmentally sensitive β€” especially on the planning board level. Some of the local Republicans and Conservatives, where they exist, are just awful. Most are just nuts.

I’m not a big fan of voting for candidates on the major party lines. I want candidates to have ideas and an ideology. The Democratic and Republican Parties do good job supporting their members, but they are hardly idea factories. The minor parties force the candidates to take a stand and help express a view point. Thanks to New York’s unique fusion law, I try to vote with the minor parties whenever they run a candidate I like. And I don’t I feel like I have to be tied to any one party or ideology when it comes down to voting.

I also like a divided government. I think public policy decisions should be taken cautiously and in general the status quo isn’t that bad. Divided government protects different regions and groups of people in our country, it ultimately protects liberty. A Republican US Senate would be an important check on Hillary’s power. Many years I do split my ticket, and while I didn’t do that this year, I am sure in future years I will keep an open mind, and maybe even vote for some Republicans like I have on state or local level in years past.

While I wouldn’t and I’m glad I didn’t vote for Hillary Clinton, I think it’s good news that she is the likely next president. She seems like a responsible individual, and it will be good to have a women president β€” and that our country is becoming more pluralistic and open to different groups and people. While I have reservations on some of her policy positions, I think she will do a good job. Jill Stein may be a better choice β€” and I’m happy having voted for her β€” but Hillary won’t be a bad second choice.

That’s my thoughts on the election. You are free to have your thoughts.