# Load required packages (hint: you need tidycensus, tidyverse, and knitr)
library(tidyverse)
library(tidycensus)
library(knitr)
# Set your Census API key
census_api_key("fe841b7ef0aa73d9579f0517bd1c8f26d33c789b")
# Choose your state for analysis - assign it to a variable called my_stateAssignment 1: Census Data Quality for Policy Decisions
Evaluating Data Reliability for Algorithmic Decision-Making
Assignment Overview
Scenario
You are a data analyst for the New York Department of Human Services. The department is considering implementing an algorithmic system to identify communities that should receive priority for social service funding and outreach programs. Your supervisor has asked you to evaluate the quality and reliability of available census data to inform this decision.
Drawing on our Week 2 discussion of algorithmic bias, you need to assess not just what the data shows, but how reliable it is and what communities might be affected by data quality issues.
Learning Objectives
- Apply dplyr functions to real census data for policy analysis
- Evaluate data quality using margins of error
- Connect technical analysis to algorithmic decision-making
- Identify potential equity implications of data reliability issues
- Create professional documentation for policy stakeholders
Submission Instructions
Submit by posting your updated portfolio link on Canvas. Your assignment should be accessible at your-portfolio-url/assignments/assignment_1/
Make sure to update your _quarto.yml navigation to include this assignment under an “Assignments” menu.
Part 1: Portfolio Integration
Create this assignment in your portfolio repository under an assignments/assignment_1/ folder structure. Update your navigation menu to include:
- text: Assignments
menu:
- href: assignments/assignment_1/your_file_name.qmd
text: "Assignment 1: Census Data Exploration"If there is a special character like comma, you need use double quote mark so that the quarto can identify this as text
Setup
State Selection: I have chosen New York State for this analysis because I will be working in New York City post-graduation.
Part 2: County-Level Resource Assessment
2.1 Data Retrieval
Your Task: Use get_acs() to retrieve county-level data for your chosen state.
Requirements: - Geography: county level - Variables: median household income (B19013_001) and total population (B01003_001)
- Year: 2022 - Survey: acs5 - Output format: wide
Hint: Remember to give your variables descriptive names using the variables = c(name = "code") syntax.
# Write your get_acs() code here
ny_data <- get_acs(
geography = "county",
variables = c(
total_pop = "B01003_001",
median_income = "B19013_001"
),
state = "NY",
year = 2022,
survey = "acs5",
output = "wide"
)
# Clean the county names to remove state name and "County"
# Hint: use mutate() with str_remove()
ny_clean <- ny_data |>
mutate(
# Remove state name from county names
county_name = str_remove(NAME, ", New York"),
# Remove "County" word
county_name = str_remove(county_name, " County")
)
# Display the first few rows
head(ny_clean)# A tibble: 6 × 7
GEOID NAME total_popE total_popM median_incomeE median_incomeM county_name
<chr> <chr> <dbl> <dbl> <dbl> <dbl> <chr>
1 36001 Albany … 315041 NA 78829 2049 Albany
2 36003 Allegan… 47222 NA 58725 1965 Allegany
3 36005 Bronx C… 1443229 NA 47036 890 Bronx
4 36007 Broome … 198365 NA 58317 1761 Broome
5 36009 Cattara… 77000 NA 56889 1778 Cattaraugus
6 36011 Cayuga … 76171 NA 63227 2736 Cayuga 2.2 Data Quality Assessment
Your Task: Calculate margin of error percentages and create reliability categories.
Requirements:
- Calculate MOE percentage: (margin of error / estimate) * 100
- Create reliability categories:
- High Confidence: MOE < 5%
- Moderate Confidence: MOE 5-10%
- Low Confidence: MOE > 10%
- Create a flag for unreliable estimates (MOE > 10%)
Hint: Use mutate() with case_when() for the categories.
# Calculate MOE percentage and reliability categories using mutate()
ny_reliability <- ny_clean |>
mutate(
moe_percent = round((median_incomeM / median_incomeE) * 100, 2),
# Create reliability categories
reliability = case_when(
moe_percent < 5 ~ "High",
moe_percent >= 5 & moe_percent <= 10 ~ "Moderate",
moe_percent > 10 ~ "Low"
)
)
ny_reliability# A tibble: 62 × 9
GEOID NAME total_popE total_popM median_incomeE median_incomeM county_name
<chr> <chr> <dbl> <dbl> <dbl> <dbl> <chr>
1 36001 Albany… 315041 NA 78829 2049 Albany
2 36003 Allega… 47222 NA 58725 1965 Allegany
3 36005 Bronx … 1443229 NA 47036 890 Bronx
4 36007 Broome… 198365 NA 58317 1761 Broome
5 36009 Cattar… 77000 NA 56889 1778 Cattaraugus
6 36011 Cayuga… 76171 NA 63227 2736 Cayuga
7 36013 Chauta… 127440 NA 54625 1754 Chautauqua
8 36015 Chemun… 83584 NA 61358 2475 Chemung
9 36017 Chenan… 47096 NA 61741 2526 Chenango
10 36019 Clinto… 79839 NA 67097 2802 Clinton
# ℹ 52 more rows
# ℹ 2 more variables: moe_percent <dbl>, reliability <chr># Create a summary showing count of counties in each reliability category
# Hint: use count() and mutate() to add percentages
count(ny_reliability, reliability)# A tibble: 3 × 2
reliability n
<chr> <int>
1 High 56
2 Low 1
3 Moderate 52.3 High Uncertainty Counties
Your Task: Identify the 5 counties with the highest MOE percentages.
Requirements: - Sort by MOE percentage (highest first) - Select the top 5 counties - Display: county name, median income, margin of error, MOE percentage, reliability category - Format as a professional table using kable()
Hint: Use arrange(), slice(), and select() functions.
# Create table of top 5 counties by MOE percentage
high_uncertainty <- ny_reliability |>
arrange(desc(moe_percent)) |>
slice(1:5) |>
select(county_name, median_incomeE, moe_percent, reliability)
high_uncertainty# A tibble: 5 × 4
county_name median_incomeE moe_percent reliability
<chr> <dbl> <dbl> <chr>
1 Hamilton 66891 11.4 Low
2 Schuyler 61316 9.49 Moderate
3 Greene 70294 6.18 Moderate
4 Yates 63974 5.84 Moderate
5 Essex 68090 5.27 Moderate # Format as table with kable() - include appropriate column names and caption
kable(high_uncertainty,
col.names = c("County", "Median Imcome", "MOE %", "Reliability"),
caption = "NY Counties with Greatest Income Data Uncertainty",
format.args = list(big.mark = ",")) #large nums have 000s separators| County | Median Imcome | MOE % | Reliability |
|---|---|---|---|
| Hamilton | 66,891 | 11.39 | Low |
| Schuyler | 61,316 | 9.49 | Moderate |
| Greene | 70,294 | 6.18 | Moderate |
| Yates | 63,974 | 5.84 | Moderate |
| Essex | 68,090 | 5.27 | Moderate |
Data Quality Commentary: Algorithmic decision-making at the New York state-level cannot occur with one broad brush. While 56 out of 62 NY counties are classified as having highly reliable income data, 5 counties are classified as having moderate reliability and 1 county is classified as having low reliability. As shown in the table below, a possible reason for these 6 counties having low/moderate reliability could be their relatively smaller populations compared to other counties. Hence, special attention must be paid to these 6 counties, with algorithmic tools to identify communities that should receive priority for social service funding and outreach programs being complemented by qualitative on-the-ground outreach and interviews with local residents.
high_uncertainty_pop <- ny_reliability |>
arrange(desc(moe_percent)) |>
slice(1:5) |>
select(county_name, total_popE, median_incomeE, moe_percent, reliability)
high_uncertainty_pop# A tibble: 5 × 5
county_name total_popE median_incomeE moe_percent reliability
<chr> <dbl> <dbl> <dbl> <chr>
1 Hamilton 5090 66891 11.4 Low
2 Schuyler 17855 61316 9.49 Moderate
3 Greene 48067 70294 6.18 Moderate
4 Yates 24713 63974 5.84 Moderate
5 Essex 37314 68090 5.27 Moderate Part 3: Neighborhood-Level Analysis
3.1 Focus Area Selection
Your Task: Select 2-3 counties from your reliability analysis for detailed tract-level study.
Strategy: Choose counties that represent different reliability levels (e.g., 1 high confidence, 1 moderate, 1 low confidence) to compare how data quality varies.
# Use filter() to select 2-3 counties from your county_reliability data
# Store the selected counties in a variable called selected_counties
selected_counties <- ny_reliability |>
slice_min(moe_percent, n = 1, by = reliability)
selected_counties# A tibble: 3 × 9
GEOID NAME total_popE total_popM median_incomeE median_incomeM county_name
<chr> <chr> <dbl> <dbl> <dbl> <dbl> <chr>
1 36081 Queens … 2360826 NA 82431 873 Queens
2 36099 Seneca … 33651 NA 64050 3355 Seneca
3 36041 Hamilto… 5090 NA 66891 7622 Hamilton
# ℹ 2 more variables: moe_percent <dbl>, reliability <chr># Display the selected counties with their key characteristics
# Show: county name, median income, MOE percentage, reliability category
select(selected_counties, county_name, median_incomeE, moe_percent, reliability)# A tibble: 3 × 4
county_name median_incomeE moe_percent reliability
<chr> <dbl> <dbl> <chr>
1 Queens 82431 1.06 High
2 Seneca 64050 5.24 Moderate
3 Hamilton 66891 11.4 Low Comment on the output: There is a significant difference between the margin of error for different counties. For example, Queens County has a margin of error percentage of only 1.06%, whereas Hamilton County has a margin of error percentage of 11.39%.
3.2 Tract-Level Demographics
Your Task: Get demographic data for census tracts in your selected counties.
Requirements:
- Geography: tract level
- Variables: white alone (B03002_003), Black/African American (B03002_004), Hispanic/Latino (B03002_012), total population (B03002_001)
- Use the same state and year as before
- Output format: wide
Challenge: You’ll need county codes, not names. Look at the GEOID patterns in your county data for hints.
# Define your race/ethnicity variables with descriptive names
# Use get_acs() to retrieve tract-level data
# Hint: You may need to specify county codes in the county parameter
# Calculate percentage of each group using mutate()
# Create percentages for white, Black, and Hispanic populations
# Add readable tract and county name columns using str_extract() or similar3.3 Demographic Analysis
Your Task: Analyze the demographic patterns in your selected areas.
# Find the tract with the highest percentage of Hispanic/Latino residents
# Hint: use arrange() and slice() to get the top tract
# Calculate average demographics by county using group_by() and summarize()
# Show: number of tracts, average percentage for each racial/ethnic group
# Create a nicely formatted table of your results using kable()Part 4: Comprehensive Data Quality Evaluation
4.1 MOE Analysis for Demographic Variables
Your Task: Examine margins of error for demographic variables to see if some communities have less reliable data.
Requirements:
- Calculate MOE percentages for each demographic variable
- Flag tracts where any demographic variable has MOE > 15%
- Create Summary Statistics
# Calculate MOE percentages for white, Black, and Hispanic variables
# Hint: use the same formula as before (margin/estimate * 100)
# Create a flag for tracts with high MOE on any demographic variable
# Use logical operators (| for OR) in an ifelse() statement
# Create summary statistics showing how many tracts have data quality issues4.2 Pattern Analysis
Your Task: Investigate whether data quality problems are randomly distributed or concentrated in certain types of communities.
# Group tracts by whether they have high MOE issues
# Calculate average characteristics for each group:
# - population size, demographic percentages
# Use group_by() and summarize() to create this comparison
# Create a professional table showing the patternsPattern Analysis: [Describe any patterns you observe. Do certain types of communities have less reliable data? What might explain this?]
Part 5: Policy Recommendations
5.1 Analysis Integration and Professional Summary
Your Task: Write an executive summary that integrates findings from all four analyses.
Executive Summary Requirements:
1. Overall Pattern Identification: What are the systematic patterns across all your analyses?
2. Equity Assessment: Which communities face the greatest risk of algorithmic bias based on your findings?
3. Root Cause Analysis: What underlying factors drive both data quality issues and bias risk?
4. Strategic Recommendations: What should the Department implement to address these systematic issues?
Executive Summary:
[Your integrated 4-paragraph summary here]
6.3 Specific Recommendations
Your Task: Create a decision framework for algorithm implementation.
# Create a summary table using your county reliability data
# Include: county name, median income, MOE percentage, reliability category
# Add a new column with algorithm recommendations using case_when():
# - High Confidence: "Safe for algorithmic decisions"
# - Moderate Confidence: "Use with caution - monitor outcomes"
# - Low Confidence: "Requires manual review or additional data"
# Format as a professional table with kable()Key Recommendations:
Your Task: Use your analysis results to provide specific guidance to the department.
Counties suitable for immediate algorithmic implementation: [List counties with high confidence data and explain why they’re appropriate]
Counties requiring additional oversight: [List counties with moderate confidence data and describe what kind of monitoring would be needed]
Counties needing alternative approaches: [List counties with low confidence data and suggest specific alternatives - manual review, additional surveys, etc.]
Questions for Further Investigation
[List 2-3 questions that your analysis raised that you’d like to explore further in future assignments. Consider questions about spatial patterns, time trends, or other demographic factors.]
Technical Notes
Data Sources: - U.S. Census Bureau, American Community Survey 2018-2022 5-Year Estimates - Retrieved via tidycensus R package on September 17, 2025
Reproducibility: - All analysis conducted in R version 4.5.1 - Census API key required for replication - Complete code and documentation available at: https://musa-5080-fall-2025.github.io/portfolio-setup-jedchewjm/
Methodology Notes: [Describe any decisions you made about data processing, county selection, or analytical choices that might affect reproducibility]
Limitations: [Note any limitations in your analysis - sample size issues, geographic scope, temporal factors, etc.]
Submission Checklist
Before submitting your portfolio link on Canvas:
Remember: Submit your portfolio URL on Canvas, not the file itself. Your assignment should be accessible at your-portfolio-url/assignments/assignment_1/your_file_name.html