Search Results for: map ny senate districts by town

Site Map

🌲🌲 Our Public Lands 🌳🌲

Interactive maps with backcountry and roadside camping: New York, Pennsylvania, West Virginia & Vermont.
List of NYS DEC Lean-Tos with map coordinates. List of NYS DEC Firetowers with map coordinates and more information.
Google Spreadsheet with Roadside, Primitive and Pay Campsites

Explore the Finger Lakes Trail, Long Path, Northville-Placid Trail and Long Trail/Appalachian in Vermont.
Catskill Park Mountain Peaks, Hudson Valley & Long Island Peaks, Peaks Over 3000 ft Elevation, Highest Peaks in Adirondacks, Interactive Map of All Named Summits in NYS, Blaze Colors in Catskill Park, Trailhead Parking Coordinates and Addresses in the Catskills.

Browse USGS Topo Quads as PDF 🆕 by State Lands or County. You can Bulk Download New & Old USGS Topograpic Maps.

Links to various NY State Land Websites 🆕. Get latest GIS Data from state Web Services.

⛺🌲 Camp 🌲🏕

Moose River PlainsCampsite Listing, Maps and photos of state’s largest free camping area.
Piseco-Powley RoadCampsite Listing, Maps and photos of 15 mile dirt road with camping.
Catskill Park Primitive CampsitesAn overview of free camping locations in Catskill Park.
Burnt-Rossman Forest, Cattaraugus County, East Branch Sacandaga River, Finger Lakes National Forest, Madison County, Pennsylvania, Vermont and West Virigina.

Campsite Coordinates for Bog River Flow / Lows Lake, Hudson River SMA (Buttermilk Falls), Lake Lila, Oswegathie River, Nine-Corner Lake, Pharaoh Lake Wilderness, Saranac River Campsites, Stillwater Lake, Schoharie County, and Sugar Hill State Forest.

Overview of Camping Areas in the Catskills, Green Mountains, Southern Adirondacks, Central Adirondacks, Northern Adirondacks, Allegheny National Forest and Penna. DCNR Motorized Campsites and the Monongahela National Forest West Virginia.

Free Campsite Overview Maps: Adirondack – North Country, Catskills, Central NY, Finger Lakes, Western NY. Interactive Map.

Places I camped in 2023, 2022, 2021 and 2020.

🏞 🛹 Bicycle Trails and “Blackie” My Mountain Bike 🚲 🚶

Finally bought a mountain bike, after chewing over a mountain vs commuter bike. Really enjoying riding my bike to work and when it rains there is always a bike rack to safely take it back home. One way to get to adventures at Thacher Park is the Nature Bus.

Empire Trail – KMZ and Interactive Map. Parking along it.

More Trailways with KMZ files including the Albany County Rail Trail, Black Diamond Trail, Catharine Valley Trail, Catskill Scenic Trail, Fonda, Johnstown & Gloversville Rail Trail, Genesee Valley Trail, Link Trail.

🦌🌲 Hunt 🦃🐿

Wildlife Management Units (Deer)KMZ Map shows the WMU boundaries.

Summer 2019 Aerial Photographs of WMUs

KMZ Maps of Deer Harvest Density by Town: 2019, 2018, 2017, 2016. By WMU 2017, 2016, 2015.

KMZ Maps of Buck Harvest Density by Town: 2019, 2018, 2017, 2016. By WMU 2017, 2016

2016 -2019 Deer and Buck Harvest by TownKMZ Spreadsheet with FIPS codes for making your own calculations.

🎣🐡 Fish 🐟🐠

Parking and Access to Trout StreamsAn interactive, downloadable KMZ Map.
Lakes with DEC Contour MapsA KMZ Map links to Contour Maps for Fishing.

🌨🏔 Sled & Wheel 🚙❄

State Truck Trails Over A Half MileDirt roads to explore in the backcountry.
NYS Statewide Snowmobile Trail SystemState trails on public and private lands.

📉📊 Learn 💵📈

Interactive Maps of NY CensusExplore and download KML files.
Charts and Interactive DiagramsFrom population to pollution control.
Andy Arthur GitHubGit my R and Python scripts used to make maps and diagrams.
Use ArcPullR to Get Geospatial DataSuper easy way to connect to get GIS data in R from government servers.
GDAL Opens E00 FilesMost open source programs nowadays can open common geospatial formats.
NY Building FootprintsWhere to find on the internet for making maps.
WMS and ArcMap ServicesDownloadable CSV file listing services used on the blog.
2022 US Census Population EstimatesRed states, south continue to gain population.
2020 Cartogram of State Population

💳 🏛 Property Taxes 🏠💸

Properties in Albany Pine Bush Study Area, Excel Files: Various Tax Rolls, Find coordinates and political districts, Look Up State Tax Records and a Script for Processing RPTL 1520 PDFs. Match NY SWIS Codes to FIPS Codes and GEOID. How to Find MapServer Data and Load Into QGIS.

🚗🚗 Big Red 🚗🚗

Big RedPhotos and Videos of my lifted truck with its camper shell. Big Red’s Dual Battery Setup for Camp Power, Video Tour and Diagram. Big Red is getting old. What is next? I’ve thought about going carless for a while to save money and reduce pollution. Or maybe going bigger? Or smaller? Five dollar gas sucks.

🔥🌲 Off-Grid Living 🏠🤠

I am seriously thinking about building an off-grid house. I have a first draft. I need to learn CAD! I have a road map towards buying land and building. I concede might have to live with long commute and give up traveling and camping. I need to be strong.

Why off grid? Well, I’m not into contemporary society. I want to own land, but not be called a landowner, and a cabin, not hooked to electrical grid, farm, raise pigs for food and burn my own trash. I’m saving for a better tomorrow, hoping to make the leap to another freer state. Having acreage is important. Cornfields aren’t bad neighbors. Maybe though my vision has grown smaller and more local. More on off-grid living.

I am 16 years into my career and have made some significant progress in my life. I love my job. But I do wonder on all the things I’m missing out but saving sure makes me high. Maybe it will be different when I own my own land — the end of goal of all this saving.

2020 into 2021 during the pandemic was a year of remote work. It was a struggle not having internet at home, worked a lot out of my truck. But I worked remotely from Horseshoe Lake which was super cool.

Generally I like the idea of owning land in a red state, particularly Idaho, Iowa, Pennsylvania, West Virginia, Wisconsin — and Midwest more generally. But I may settle for New York – it’s all about the f-ing money!

💻👨‍💻 Open Source 🗺️📍

I use open source software and public sources of data for the blog. Quantum GIS (QGIS), GDAL/ogr2ogr, PyQGIS, GeoPANDAS, R Studio and Leaflet for map making, Arduino and ESP32 microprocessors, Ubuntu Linux and XFCE Window Manager. I’ve recently gotten interested in machine learning.

I avoid using commercial software like Microsoft Windows and do not have home internet or television. If you don’t use commercial software and use your brain, fears of computer viruses are overblown. I deleted most of my social media accounts.

Creating Digital Surface Models using LiDAR Point Clouds.

📊🗺 R Statistical Programming 📜👨‍🏫

The R programming language and RStudio are powerful tools for statistical analysis, making maps and charts. Many of the blog posts and analysis I do are in R, ggplot not only makes great charts but also maps using tidycensus. Generally, R is better then Python for geospatial work.

Use IDW Interpolation to fill in missing Census data, Zonal Histograms for land cover, load WMS Aerial Photography in R, find mountain peaks, save Census shapefiles using tigris quickly, pull NY Election Night Results using Selenium. Fast reverse Geocoding in PostGIS. Working with PDFs in R. Fix a common error starting rselenium/wdman. Make data-filled calendars. R is wonderful and weird, learn it!

🐼🔢 Python and Pandas 💻🐍

Querying state property database, political enrollments, PL 94-171 Census files, calculating population statistics, what address is a district in, converting old districts to new districts, Shapefiles missing Projection information in QGIS.

Learn to code for free modern HTML, Javascript, Python and SQL at freeCodeCamp and web development at the Odin Project.

🐴 🐘 Politics 🦁 🐍

Crunched Election Results with Turnout for Albany County: November 2023, 2022, 2021, 2020, 2019 and Primaries June 2019, Pres/June 2020, June 2021, June 2022, Aug 2022, June 2023.

Albany County Races converted to the new 2023 EDs using Super EDs and Code: 2022, 2021, 2020, 2019 and Primaries June 2019, Pres/June 2020, June 2021, June 2022, Aug 2022.

Above Election Results as zipped Excel files.

Albany County Legislature Districts 2024 Maps

Maps Comparing 2017 and 2023 Albany County Election Districts and a Crosswalk Table Showing the Proposition of Voting Age Population in New and Old EDs

Maps of 2022 NYC Assembly Races, NYS Assembly Races, NY Senate Races, Governor’s Race in Erie County and Statewide. Partisan shift in governor race between 2010 and 2018.

A comparison of Democratic Performance 2022 Assembly Districts to those proposed in 2023 by the IRC. Here is latest 4/20/23 IRC Maps, showing ADP and how they change from existing Assembly districts. Most towns upstate, outside of cities, are quite red. Using LATFOR data with R to calculate Average Democratic Performance.

You can scrape employee salary data from SeeThroughNY using R. Other useful investigative resources.

I often think politics is for losers. I’m into the politics of statistical analysis and reading history books.

I believe strongly in the first amendment, second amendment, oppose gun restrictions and I support de-funding the police in favor of lower-cost technology and civilian employees. Maybe use red flag laws for voting to stop dangerous voters? And the media should stop promoting mass-shootings, even if it’s super profitable for all involved. They should tax the media when it promotes violence. I think some people are much too paranoid in politics. How elections are rigged under law to benefit incumbents. But vote, it’s the best option and inexpensive.

Yeah for the third parties! I voted for Larry Sharpe for Governor and Jo Jergenson for President but my views are complicated and often vote for Democrats, after voting Jill Stein Green Party in 2016.

Some thoughts on Joe Biden not seeking re-election. Times change, but Biden was been a good change over DJT and glad the Trump era is over and are glad prosecutors and grand jurors are holding him responsible by indicting him for many serious felonies. I don’t think Trump can win in 2024, as nothing has changed politically from 2020.

I think rural people should be left alone and not worship government workers or have parades for them. I am no fan of Donald Trump, his speeches are bad, I don’t like Trump’s embrace of radical environmentalists, but do admire the homemade roadside monuments to DJT.

🌲🌳The Earth 🌎 🐸

Why I oppose wilderness areas and parks. It’s trendy to be green these days, but is eco-marketing good for the planet? I visited the Mount Storm Coal Plant and Corridor H.

I worry about a lot about overly-aggressive Climate Change Action, and Undermining Environment Laws for Climate Action. I think we should all admit we are Addicted to Fossil Fuels. These days, urban recycling has become a joke, when it’s still an option at all. It’s better to just buy less shit and avoid the alure of Costcos. I really don’t like how aging radicals have become industrial solar salespeople.

Big bucks are coming to state-designated disadvantaged communities under the CLCP. Which counties and political districts are in line for the the most pork? Interactive map.

I’m a big of farmers who are essentially Living Off the Earth and think Rednecks are Noble Savages. Dairy Farming are key to our rural landscape. I’d trust a farmer or a hunter in a pile of guts he’s butchered over any ivory-tower scientist.

🌎🔆 Industrial Solar 🌞 🏭

Hundreds of multi-acre industrial solar farms are being built in our state. How bad is solar for the environment? We should ask tough questions. Interactive of recently built solar farms, proposed facilities. List of proposed industrial solar facilities. See how the Greenville Solar Farm changed the landscape.

💳 💸Saving Money 💰 💷

I am not a fan of ESG Investing as it’s not well diversified. I prefer index-funds and other tax-advantaged ways of saving. Why I am concerned about saving enough for retirement, even though I’m in my late 30s. We as a nation should save more, consume less. I like the idea of carbon tax to replace capital gains taxes to discourage consumption.

🥦 🍎Mission Fifty, Smoking Grass & Being Healthier 🏠🧠

I am now officially in my 40s! I am building to a better life in my 50s, which means getting up early, walking a lot, saying no to cake and yes to more fruit. In many ways, the forties are an awesome time to be alive.

And eating healthy for less without losing sleep over arsenic. And I don’t think we should subsidize unhealthy habits. How I got started in eating healther. Meals are too focused on meat and carbs due to how we describe them, maybe I eat too many bananas in the office, what to eat while camping, worry more about salt then GMOs, eat more beans. Do spend extra for farmers market peaches, especially doughnut peaches and plums. Consider ethnic supermarkets. Thinking about how to make a healthier macaroni and cheese, spinach-mackeral-pasta salad, quick-cook biscuits and whole-wheat bread. That said, too many recipes are junk food crap. Okay in moderation is not okay. The fact that I’m thinner is not a sign I’m dying.

A few years back I decided to explore my mental illness with therapy, thinking about why I have so much anxiety and how many of my values are rational or just thinking too much rednecks’ burn barrels and how much of a throwaway society we live in. Do I want to change?

I’ve learned to care less about the world, and focus more on myself. Maybe I am happier as I am now, saving and investing a lot towards owning my own land, where I don’t have to deal with all the bullshit of modern life.

And smoke more grass, now that’s legal. Smoking pot is fascinating. Enjoying the rich colors. A map of licensed pot retailers in NYS. Yes, that stuff we used to think of as the smell of crime. Thoughts on Stoner Culture. Not that the blog advertisers like it.

Mission Fifty: Getting to the point where I own my own land. 🚜
Healthy Eating 🍎 / Growing My Wealth 💰
Healthy Thoughts 💭 / Enjoying Life 😃

Questions, comments? Feel free to email me at andy@andyarthur.org.

You do your thing, I’ll do mine.
I smoke pot and drink cheap beer, up in the wilderness. Yes it’s my right !

I use GNU open source software.
Plus I like buck goats,
because they’re real macho men
spraying their beards with goat urine.

I own guns, but not nearly enough. Some day when I live in a pro-2A state …
I can’t make everybody happy, so I’ll just be myself.” – Andy Arthur

Return to the Top

Laugh a bit and learn how your data is used on this blog: Privacy Policy

This blog is © Copyright 1997-2023 Andy Arthur.org, please share using the Creative Commons Attribution 3.0 License.

Categories:

How To Make Maps from Redistricting Block Lists

When a city council, county legislature, or state legislature redistricts itself to reflect changing population, they usually release data in two formats:

  1. Census Block Equivalency – A list of census blocks in each district, generated by the commerical GIS program (such as Mapitude) used for redistricting.
  2. Metes and Bounds – A legal description of each district, used in resolving court disputes over district boundaries, and assisting board of elections on where to put voters whose property might be crossed by a Census block

If you planning on making a map, Metes and Bounds won’t be particularly useful. Computers don’t understand english very well, they need numbers and lists. In contrast, the Census Block Equivalency is very useful for mapping things.

Every year, the Census Bureau puts out series of ERSI Shapefiles known as TIGER/Line. You can download TIGER/Line for any state and county in the United States from their website. They provide many different shapefiles and layers such as a Highway, Faces, Edges, and County Subdivision layers, however the one you will be most interested for making district maps is the Tabulation Block (tabblock) layer.

You can use these files in the free program known as Quantum GIS or QGIS. While this tutorial will not explain the ins and outs of QGIS, this should get you started on making redistricting maps.

The Tabulation Block Layer is the file containing all of the Census Blocks for a particular county. A Census Block is the smallest unit of population gathered by Census Block, and consists of all bordering features (bounds) — roads, rivers, shorelines, along with all imaginary lines (metes) — town lines, village lines, other lines drawn for statisitical purposes.

Each Census Block has a number, that is a subdivision of the Census Block, County ID, and State ID that it resides within. For example, the Governor’s Mansion in Albany is located in Census Block 2000 in Census Track 23.00 (zero padded to 002300) in Albany County (Federal Information Processing Standard — FIPS ID: 01) which is in NY State (FIPS ID: 36). County subdivisions are not applied to Census Tract Numbers, as they may in some cases cross county subdivisions, as is the case of smaller districts.

You put those numbers together to get the GEOID — which is the key used for redistricting block lists and most other block-level census data. The Governor’s Mansion is located at a block with a GEOID 360010023002000.

36 001 002300 2000
State ID County ID Zero Padded Census Tract Number Census Block Number

The block list you get from a redistricting commission typically is in Database Exchange Format (.DBF) or Comma Deliminated Format (.CSV) which are both openable by common spreadsheet applications like Microsoft Excel or OpenOffice Spreadsheet and GIS programs like ArcGIS or Quantum GIS.

This is taken from the LATFOR State Senate Proposed Districts (January 2012) DBF file. It shows you that the Governor resides in Proposed Senate District 44. Across the river in Census Block 4010, Census Tract 524.03, in Rensselear County (FIPS ID 83), NY State (FIPS ID: 36) is located in Proposed Senate District 43.

360010023002000 SD44
360010021002008 SD44
360010021002004 SD44
360010021002001 SD44
360830524034017 SD43
360830524034010 SD43

Download the TIGER/line “Tabulation Block” Shpaefile file for the district you are interested in. You will want the 2010 version. You can download a state-wide tabulation block file, however that is not recommended as the next step will be impossibly slow on most computers. You may also want to open the .CSV or .DBF file in your spreadsheet program and cut out the county you want to speed things up.

You will then want to open up the file in Quantum GIS. You will get a nice map of the county you downloaded, showing all of the Census Blocks.

  • From there, go to the Vector -> Join Attributes submenu.
  • Make sure that the Target vector layer matches the Tabulation Block Shapefile you wish to join against, then set Target join field to GEOID10 .
  • Select click Join dbf table and select the DBF or CSV file you wish to join.
  • Change the Join field to BLOCK or whatever the GEOID is titled in your redistricting block file.
  • Enter in a location to save the Output Shapefile
  • Click okay.

Then wait. A typical county will take 10-40 minutes to join on my 5 year old laptop; your computer may be quicker. If you have a dual processor machine, go on to doing other work in other programs. You will end up with a map that looks like this (stylized for your enjoyment). Each block will be assigned a Senate District (in this example).

Halfway there. Now you need to “dissolve” each Census Block into it’s larger political district. Go to Vector -> Geoprocessing Tools -> Dissolve . Set the Input vector layer to the file you previously joined. Then set the Dissolve field to the field containing the district number — such as DISTRICTID or whatever it is named. Enter a name to safe the file. Click Dissolve.

Outputed will be a Shapefile containing all the political districts in the county you joined and dissolved. This will take 5-20 minutes on my laptop. Other data may exist in that file, such as Census Block number, however at this point that data will be invalid, as only the district number is accurately preserved in such a join. All other data will be picked at random, so delete those columns.

I hope this is helpful. If you just want the Proposed State Senate or State Assembly Districts you can download them from Center for Urban Research. These are the same data, joined using the above process by somebody with a much faster computer. I have also made up a Shapefile containing the Albany County Legislative Districts using this process.

2010 Statewide Elections in Maps

2010 Statewide Elections in Maps A look back at the last election cycle and how things played out.

Here is a map of the Average Democratic Preformance for all Statewide Candidates in 2010.

Here is the Gubernatorial Race. Notice how Andrew Cuomo won most rural and urban communities, with the exception of the most conservative towns in the Southern Adirondacks, Catskill Mountains, and also Western NY, where hometown favorite Carl Paladino snapped up many votes.

In Competitive State Senate Elections (which there many in 2010), Democrats won over many small towns in Upstate New York. That said, the votes that Democrats won, often where not enough to offset the more populated areas where Republicans won. Votes on third party lines (not included) also helped win Republicans over in certain districts.

The same can be said with the State Assembly. Despite winning far fewer towns, they kept a strong majority, in part thanks to their strong New York City base, and fushion candidates, running on multiple lines not shown on this map.

Neat Things You Can Do With Public Data

Andy Arthur GitHubGit my R and Python scripts used to make maps and diagrams.

Mapping and Political Data 🐴 🐘

Crunched Election Results with Turnout for Albany County: November 2023, 2022, 2021, 2020, 2019 and Primaries June 2019, Pres/June 2020, June 2021, June 2022, Aug 2022, June 2023.

Albany County Races converted to the new 2023 EDs using Super EDs and Code: 2022, 2021, 2020, 2019 and Primaries June 2019, Pres/June 2020, June 2021, June 2022, Aug 2022.

Above Election Results as zipped Excel files.

Albany County Legislature Districts 2024 Maps

Maps Comparing 2017 and 2023 Albany County Election Districts and a Crosswalk Table Showing the Proposition of Voting Age Population in New and Old EDs

Maps of 2022 NYC Assembly Races, NYS Assembly Races, NY Senate Races, Governor’s Race in Erie County and Statewide. Partisan shift in governor race between 2010 and 2018.

A comparison of Democratic Performance 2022 Assembly Districts to those proposed in 2023 by the IRC. Here is latest 4/20/23 IRC Maps, showing ADP and how they change from existing Assembly districts. Most towns upstate, outside of cities, are quite red. Using LATFOR data with R to calculate Average Democratic Performance.

State Worker Data👱👷

You can scrape state and local employee salary data from SeeThroughNY using R.

Other useful investigative resources using public data!

Property Taxes 🏠💸

Properties in Albany Pine Bush Study Area, Excel Files: Various Tax Rolls

Find coordinates and political districts

Look Up State Tax Records

Script for Processing RPTL 1520 PDFs.

Match NY SWIS Codes to FIPS Codes and GEOID.

How to Find MapServer Data and Load Into QGIS.

Making Maps and Processing Data 📉📊

Use ArcPullR to Get Geospatial DataSuper easy way to connect to get GIS data in R from government servers.
GDAL Opens E00 FilesMost open source programs nowadays can open common geospatial formats.
NY Building FootprintsWhere to find on the internet for making maps.

WMS and ArcMap ServicesDownloadable CSV file listing services used on the blog.

2022 US Census Population EstimatesRed states, south continue to gain population.
2020 Cartogram of State Population

Interactive Maps of NY CensusExplore and download KML files.
Charts and Interactive DiagramsFrom population to pollution control.

Categories:

PHP Script for Converting RPTL 1590 Reports into Excel Files

Real Property Tax Law 1590 requires that municipalities post their tax rolls, within 10 days of the proposed and final rolls being approved. The rolls are generally searchable PDF files, but that isn't that helpful if you are trying to search and compare multiple properties or want to use the North-East Coordinate data to make a map.

This script -- which uses the Linux program pdfttext and other common Linux commands to convert the PDF to a text file, then processes it into a .CSV file that can be opened with a GIS program such as Quantum GIS or a spreadsheet like Microsoft Excel or OpenOffice Calc.

Real Property Tax Law 1590 requires that municipalities post their tax rolls, within 10 days of the proposed and final rolls being approved. Below is an PHP script that will extract the reports into a CSV file for importing into Microsoft Excel or a GIS program. It extracts the text from the PDF using pdftotext from the poppler-util.

If you do not want to install poppler-util, I would encourage to check out the simpler and better maintained R Script for for Converting RPTL 1590 Reports that I also wrote. Both versions can also be found on my GitHub.

001<?php
002 
003// this program requires pdftotext (a linux program) and PHP version 7.2
004 
005// first convert PDF to text
006$pdfdir = "input-pdf";
007$textdir = "output-txt";
008 
009// delete old input-text
010if (isset($argv[1]) && $argv[1] == 'delete') {
011    echo "Deleting old conversions ...\n";
012    system("rm $textdir/*");
013}
014 
015foreach (scandir($pdfdir) as $file) {
016    if (substr($file, -4) !== '.pdf') {
017        continue;
018    }
019     
020    $textfile = substr($file, 0, -4).".txt";
021    $town = substr($file, 0, -4);
022     
023    echo ("#### START $town #### \n");
024     
025    if (file_exists("$textdir/$textfile")) {
026        echo "Text file exists, not converting PDF again (arg[1] == delete to override).\n";
027    }
028    else {
029        echo "Converting to text file ...";
030        system('pdftotext -layout '.escapeshellarg("$pdfdir/$file").' '.escapeshellarg("$textdir/$textfile"));
031        echo " DONE\n";
032    }
033     
034    $text = file("$textdir/$textfile");
035    $town = substr($file, 0, -4);
036 
037    $taxroll = array();
038    $payerId = 0;
039     
040    $output = "";
041     
042    $townId = "";
043    $swisId = "";
044    $countyId = "";
045    $villageId = "";
046     
047    for ($i = 0; $i < count($text); $i++) {
048        if ($i % 100 == 0) echo "#";
049         
050        // capture county - town - swis
051        if (preg_match('/COUNTY\s*?- (.*?)\s{2}/', $text[$i], $matches)) $countyId = $matches[1];
052        if (preg_match('/CITY\s*?- (.*?)\s{2}/', $text[$i], $matches)) $townId = $matches[1];
053        if (preg_match('/TOWN\s*?- (.*?)\s{2}/', $text[$i], $matches)) $townId = $matches[1];
054        if (preg_match('/VILLAGE\s*?- (.*?)\s{2}/', $text[$i], $matches)) $villageId = $matches[1];
055        if (preg_match('/SWIS\s*?- (.*?)\s{2}/', $text[$i], $matches)) $swisId = $matches[1];
056 
057         
058        // first line = tax id
059        $pattern = '/\*{3,} ((\d|\-|\.){4,}) \*{3,}/';
060        preg_match($pattern, $text[$i], $matches);
061 
062        // we've found the start of a new tax record!
063        if (isset($matches[1])) {
064            $i++;
065             
066            $taxpayer = array();
067            $j = 0;
068            // output each part onto the line
069            while (isset($text[$i]) && !preg_match('/\*{3,}/', $text[$i])) {
070                $split = preg_split('/\s{2,}/', $text[$i]);
071                 
072                $taxpayer[$j] = $split;
073                $i++; $j++;
074            }
075             
076             
077            $taxpayer[$j] = array('location',$countyId, $townId, $villageId, $swisId);
078                         
079            $taxroll[$payerId++] = $taxpayer;
080            $i--;
081        }
082    }
083 
084    // export unprocess tax rolls for debug
085    file_put_contents("output-debug/$town.txt", print_r($taxroll,true));
086     
087    // next scan for all special district types in file
088    $specialDistType = array();
089     
090    foreach ($taxroll as $taxpayer) {
091        for ($i = 0; $i < count($taxpayer); $i++) {
092                for ($j = 0; $j < count($taxpayer[$i]); $j++) {
093                    if (preg_match('/^([A-Z]{2})(\d\d\d) (.*?)( TO|$|\d{2,})/', $taxpayer[$i][$j],$matches)) {
094                        $specialDistType[$matches[1]] = $matches[1];                       
095                    }
096                }  
097        }
098    }
099    ksort($specialDistType);
100 
101    // then process into a nice field
102    $formTax = array();
103 
104    foreach ($taxroll as $taxpayer) {
105        $formPayer = array();
106         
107        $formPayer[0] = $taxpayer[1][0]; // tax id
108         
109        if (isset($taxpayer[0][1]) && preg_match('/^(\d.*?) (.*?)$/',$taxpayer[0][1], $address)) {
110            $formPayer[1] = $address[1]; // street number
111            $formPayer[2] = ucwords(strtolower($address[2])); // street name
112        }
113        elseif (isset($taxpayer[0][1]))  {
114            $formPayer[1] = '';
115            $formPayer[2] = ucwords(strtolower($taxpayer[0][1])); // street name
116        }
117         
118        if (isset($formPayer[1])) $formPayer[23] = ltrim($formPayer[1].' '.$formPayer[2]); // full street
119        else if (isset($formPayer[1])) $formPayer[23] = ltrim($formPayer[2]);
120         
121        $formPayer[3] = ucwords(strtolower($taxpayer[2][0])); // owner 1
122         
123        // next five lines are either are owner or address info
124        for ($i = 3; $i < 8; $i++) {
125             
126            if (!isset($taxpayer[$i][0])) continue;
127             
128            // if a taxpayer name
129            if (preg_match('/^[A-Z]/',$taxpayer[$i][0]) && !preg_match('/^PO/',$taxpayer[$i][0]) && !preg_match('/^(.*?), (\w\w) (.*?)$/',$taxpayer[$i][0]))    {
130                 
131                if (!isset($formPayer[4])) $formPayer[4] = ucwords(strtolower($taxpayer[$i][0]));
132                else if (!isset($formPayer[5])) $formPayer[5] = ucwords(strtolower($taxpayer[$i][0]));
133                else if (!isset($formPayer[6])) $formPayer[6] = ucwords(strtolower($taxpayer[$i][0]));
134            }
135             
136            // if a city - state - zip
137            else if (preg_match('/^(.*?), (\w\w) (.*?)$/',$taxpayer[$i][0], $address)) {
138                $formPayer[10] = ucwords(strtolower($address[1]));
139                $formPayer[11] = strtoupper($address[2]);
140                $formPayer[12] = ucwords(strtolower($address[3]));
141            }
142             
143            // if an address (pad to this field)
144            else if (preg_match('/^\d/',$taxpayer[$i][0]) || preg_match('/^PO/',$taxpayer[$i][0])) {
145                if (!isset($formPayer[7])) $formPayer[7] =  ucwords(strtolower($taxpayer[$i][0]));
146                else if (!isset($formPayer[8])) $formPayer[8] =  ucwords(strtolower($taxpayer[$i][0]));
147                else if (!isset($formPayer[9])) $formPayer[9] =  ucwords(strtolower($taxpayer[$i][0]));
148            }
149         
150        $formPayer[13] = $taxpayer[1][1];
151    }
152         
153        // extract coordinates by searching through array
154        for ($i = 0; $i < count($taxpayer); $i++) {
155            for ($j = 0; $j < count($taxpayer[$i]); $j++) {
156                if (preg_match('/EAST-(\d*) NRTH-(\d*)/', $taxpayer[$i][$j], $coord)) {
157                    $formPayer[14] = $coord[1];
158                    $formPayer[15] = $coord[2];    
159                }
160            }
161        }
162         
163        // extract acres
164         
165            for ($i = 0; $i < count($taxpayer); $i++) {
166            for ($j = 0; $j < count($taxpayer[$i]); $j++) {
167                if (preg_match('/ACRES *?(\d+)/', $taxpayer[$i][$j],$acres)) {
168                    $formPayer[16] = $acres[1];
169                }
170                else if (preg_match('/ACRES/', $taxpayer[$i][$j])) {
171                    if (preg_match('/^([0-9.]+)/', $taxpayer[$i][$j+1], $acres)) $formPayer[16] = $acres[1];
172                }
173            }
174        }
175 
176    // extract full market value
177 
178            for ($i = 0; $i < count($taxpayer); $i++) {
179            for ($j = 0; $j < count($taxpayer[$i]); $j++) {
180                if (preg_match('/FULL MARKET VALUE *?(\d+)/', $taxpayer[$i][$j],$value)) {
181                    $formPayer[17] = str_replace(',','',$value[1]);
182                }
183                else if (preg_match('/FULL MARKET VALUE/', $taxpayer[$i][$j])) {
184                    if (preg_match('/^([0-9,]+)/', $taxpayer[$i][$j+1], $value)) $formPayer[17] = str_replace(',','',$value[1]);
185                }
186            }
187        }
188         
189        // extract deed book info
190            for ($i = 0; $i < count($taxpayer); $i++) {
191                for ($j = 0; $j < count($taxpayer[$i]); $j++) {
192                     
193                                         
194                    if (preg_match('/DEED BOOK *?(\d+) *?PG-(\d+)/', $taxpayer[$i][$j],$value)) {
195                        $formPayer[18] = $value[1];
196                        $formPayer[19] = $value[2];
197                    }
198                    else if (preg_match('/DEED BOOK *?(\d+)/', $taxpayer[$i][$j],$value)) {
199                        $formPayer[18] = $value[1];
200                        if (isset($taxpayer[$i][$j+1]) && preg_match('/^PG-(\d+)/', $taxpayer[$i][$j+1], $value)) $formPayer[19] = $value[1];
201                    }
202                }
203            }
204                 
205            // county taxable amount
206            for ($i = 0; $i < count($taxpayer); $i++) {
207                for ($j = 0; $j < count($taxpayer[$i]); $j++) {
208                    if (preg_match('/COUNTY TAXABLE VALUE/', $taxpayer[$i][$j])) $formPayer[20] = chop(str_replace(',','',$taxpayer[$i][$j+1]));
209                }
210            }
211 
212        // school taxable amount
213            for ($i = 0; $i < count($taxpayer); $i++) {
214                for ($j = 0; $j < count($taxpayer[$i]); $j++) {
215                    if (preg_match('/SCHOOL TAXABLE VALUE/', $taxpayer[$i][$j])) $formPayer[21] = chop(str_replace(',','',$taxpayer[$i][$j+1]));
216                }
217            }  
218        // city taxable amount
219            for ($i = 0; $i < count($taxpayer); $i++) {
220                for ($j = 0; $j < count($taxpayer[$i]); $j++) {
221                    if (isset($taxpayer[$i][$j]) && preg_match('/^(CITY|TOWN)/', $taxpayer[$i][$j])) {
222                        if (isset($taxpayer[$i][$j+1]) && preg_match('/^TAXABLE VALUE/', $taxpayer[$i][$j+1])) $formPayer[22] =  chop(str_replace(',','',$taxpayer[$i][$j+2]));
223                         
224                    }
225                }  
226            }
227     
228         
229        // field relating to solar power (for munis that have such laws)
230        $formPayer[24] = '';
231        for ($i = 0; $i < count($taxpayer); $i++) {
232            for ($j = 0; $j < count($taxpayer[$i]); $j++) {
233                if (preg_match('/solar/i', $taxpayer[$i][$j])) {
234                    $formPayer[24] .= "{$taxpayer[$i][$j]},";
235                }
236            }  
237        }  
238         
239        // STAR
240        $formPayer[25] = '';
241        for ($i = 0; $i < count($taxpayer); $i++) {
242            for ($j = 0; $j < count($taxpayer[$i]); $j++) {
243                if (preg_match('/ STAR/', $taxpayer[$i][$j])) {
244                    $formPayer[25] .= "{$taxpayer[$i][$j]},";
245                }
246            }  
247        }
248         
249        // STAR
250        $formPayer[26] = '';
251        for ($i = 0; $i < count($taxpayer); $i++) {
252            for ($j = 0; $j < count($taxpayer[$i]); $j++) {
253                if (preg_match('/(VET WAR|CW_15_VET|VETWAR|VETDIS|VETERANS)/', $taxpayer[$i][$j])) {
254                    $formPayer[26] .= "{$taxpayer[$i][$j]},";
255                }
256            }  
257        }  
258         
259        // SCHOOL
260        $formPayer[27] = $taxpayer[2][1];  
261         
262        // columns 28+ are special districts
263        $l = 28;
264         
265        foreach ($specialDistType as $type) {  
266            $formPayer[$l] = '';
267                 
268            for ($i = 0; $i < count($taxpayer); $i++) {
269                for ($j = 0; $j < count($taxpayer[$i]); $j++) {
270                    if (isset($taxpayer[$i][$j]) && preg_match('/^(\w\w)(\d\d\d) (.*?)( TO|$|\d{2,})/', $taxpayer[$i][$j],$matches)) {
271                        if ($matches[1] == $type) $formPayer[$l] .= "{$matches[1]}{$matches[2]} {$matches[3]} ";
272                    }
273                }  
274            }
275             
276            $l++;
277        }
278         
279         
280        // sort and add missing keys
281        for ($i = 0; $i < count($formPayer); $i++) {
282            if (!isset($formPayer[$i])) $formPayer[$i] = '';
283        }
284         
285         
286        ksort($formPayer);
287         
288                // shift onto the rolls county, town, village, swis
289        for ($i = 0; $i < count($taxpayer); $i++) {
290                 
291                if ($taxpayer[$i][0] != 'location') continue;
292                 
293                // add array to line               
294                for ($j = count($taxpayer[$i])-1; $j > 0; $j--) array_unshift($formPayer, $taxpayer[$i][$j]);
295                 
296        }
297         
298         
299        $formTax[] = $formPayer;
300         
301        }
302 
303 
304        // lastly sort form by street and number
305         
306        $addNum = array();
307        $addSt = array();
308        $own1 = array();
309        for ($i = 0; $i < count($formTax); $i++) {
310          $addSt[] = $formTax[$i][6];
311          $addNum[] = $formTax[$i][5];
312          $own1[] =  $formTax[$i][7];
313        }
314 
315        // now apply sort
316        array_multisort($addSt, SORT_ASC,
317                $addNum, SORT_NUMERIC, SORT_ASC,
318                $own1, SORT_ASC,
319                $formTax);
320                 
321                 
322    //print_r($formTax);
323 
324    echo "\nWriting to CSV ...";
325 
326    // print out form
327    $output .=  '"Tax Roll","County","Town","Village","SWIS","Tax ID","Street Number","Street Name","Owner 1","Owner 2","Owner 3","Owner 4",'
328                .'"Mail Address 1","Mail Address 2","Mail Address 3","Mail City","Mail State","Mail Zip",'
329                .'"Property Type","East","North","Acres","Full Market Value","Deed Book","Deed Pg",'
330                .'"County Value","School Value","Town Value","Full Street",'
331                .'"Solar","STAR","VETS","School",';
332                 
333    foreach ($specialDistType as $type) {
334        $output .= "\"$type\",";
335    }
336                 
337    $output .=  "\n";
338 
339    foreach ($formTax as $line) {
340        $output .=  '"'.$town.'",';
341        foreach ($line as $item) {
342            $output .=  '"'.$item.'",';
343        }
344         
345        $output .=  "\n";
346    }
347     
348    // save output to file
349    file_put_contents("output-csv/$town.csv", $output);
350     
351    echo " DONE\n";
352}
353 
354// last, create a great big file
355//system("cat output-csv/*.csv > all-property.csv");
356 
357system("zip output-csv.zip output-csv/*");