Housing Data Analysis

This repository contains scripts and data for analyzing affordable housing production in New York City, with a focus on identifying new construction projects and correlating HPD (Housing Preservation and Development) financing data with DOB (Department of Buildings) permit filings.

Files

Data Files

• Affordable_Housing_Production_by_Building.csv - HPD data on affordable housing buildings
• Affordable_Housing_Production_by_Project.csv - HPD data on affordable housing projects
• new_construction_bins_dob_filings.csv - DOB job application filings for new construction (NB/New Building type)

Scripts

query_dob_filings.py

Queries NYC Open Data APIs to find DOB job filings for new construction projects.

• Searches DOB Job Application Filings API for job type "NB"
• Searches DOB NOW Job Applications API for job type "New Building"
• Matches filings to BINs from the housing data

Usage:

python query_dob_filings.py new_construction_bins.txt

HPD_DOB_Join_On_BIN.py

Creates a timeline combining HPD financing data with DOB filing/approval dates.

• Extracts HPD financing start and completion dates
• Extracts DOB/DOB NOW application submission and approval dates
• Creates a unified timeline CSV sorted by BIN and date

Usage:

python HPD_DOB_Join_On_BIN.py Affordable_Housing_Production_by_Building.csv new_construction_bins_dob_filings.csv

Output:

• Affordable_Housing_Production_by_Building_timeline.csv - Timeline of all events by BIN

create_timeline_chart.py

Generates a Gantt-style timeline chart showing DOB application and HPD financing timelines for each BIN.

Usage:

python create_timeline_chart.py Affordable_Housing_Production_by_Building_timeline.csv

Output:

• Affordable_Housing_Production_by_Building_timeline_timeline_chart.png - Visualization chart
• Affordable_Housing_Production_by_Building_timeline_timeline_data.csv - Extracted timeline data

Data Sources

• HPD Data: Affordable Housing Production data from NYC HPD
• DOB Job Applications: NYC Open Data - DOB Job Application Filings
• DOB NOW Job Applications: NYC Open Data - DOB NOW Job Applications

Requirements

pip install pandas matplotlib requests beautifulsoup4

Workflow

1. Extract BINs for new construction projects from Affordable_Housing_Production_by_Building.csv
2. Query DOB APIs for NB/New Building filings using query_dob_filings.py
3. Join HPD and DOB data by BIN using HPD_DOB_Join_On_BIN.py
4. Visualize timelines using create_timeline_chart.py

Timeline Event Types

• HPD financing submitted - Project start date from HPD
• HPD financing completed - Project completion date from HPD
• DOB NB Application submitted - First DOB filing for new building
• DOB NB Application approved - First DOB approval for new building
• DOB NOW New Building Application submitted - First DOB NOW filing
• DOB NOW New Building Application approved - First DOB NOW approval