Future City Lab: Neighborhood Stats

Chi-Squared Statistics

Grade Level: 11-12 (AP level)
Keywords: null hypothesis, bias, statistical significance, chi-squared, p-value, social science
Thumbnail
Source: www.flickr.com/franciscodaum

Objectives

Students will

  • Derive and use chi-squared tests  

  • Explain the use of this test in the context of New York City demographics 

  • Compare and contrast social sciences with so-called “hard science” 

  • Consider to what extent statistics are useful in the study of humans 

Materials

  • One bag of 30 beads (or Skittles) per pair of students. There should be five different colors. 

  • Handouts (two student activity handouts, one data sheet, and one teacher key)

  • Calculators 

Standards: 

  • CCSS.ELA-LITERACY.RST.9-10.7: Translate quantitative or technical information expressed in words in a text into visual form (e.g., a table or chart) and translate information expressed visually or mathematically (e.g., in an equation) into words. 

  • CCSS.MATH.CONTENT.HSS.MD.A.4: Develop a probability distribution for a random variable defined for a sample space in which probabilities are assigned empirically; find the expected value.  

Guiding Questions

  1. How can you test whether a group is randomly distributed? 

  2. What are the major differences between the “hard sciences” and “social sciences” and what are the data limitations in each? 

  3. How can we use statistics from today to help us understand the past? 

    Procedures

    Chi-squared tables are tools to help scientists understand whether data is significantly different from a randomized outcome – in other words, whether there is some external factor at play that needs to be explained. They are often illustrated in the classroom using inanimate objects or nonhuman actors (beads, genetic outcomes). This lesson considers whether statistical methods like chi-squares can be useful to evaluate human behavior and whether it might be a tool to consider in the history classroom. A helpful tutorial for the non-science classroom teacher can be found here: http://www.ling.upenn.edu/~clight/chisquared.htm . 

    In the first part of the lesson, students derive chi-squared values by testing the distribution of beads. Then, students are presented with a more difficult problem: Are different ethnic groups in New York distributed randomly? Students can determine this by using calculators. (All New York City neighborhoods that they will be given do have significantly high chi-squared values.) The teacher then will lead the students through a discussion of whether these types of tools can be meaningful in analyzing human behavior, and a discussion of the differences between social science and “hard science” more generally. Finally, the lesson ends with a discussion of whether these tools are useful in analyzing history and points towards resources that explore historical factors that influence why people in New York City are grouped non-randomly. 

    Download the Neighborhood Stats Bead Activity Worksheet

    Download the Neighborhood Stats Demographic Handout

    Download the Neighborhood Stats Data Table

    Download the Neighborhood Stats Teacher Key

  1. Hook
  2. Make an outlandish claim to students, something like “People who wear button-down shirts can jump higher than those who wear t-shirts.” Find a student to disagree with you. Ask the class: Who has the burden of proof? Which should we assume with no evidence? Why? Have students share responses. 

    Explain the goal of all experiments is to test the null hypothesis. Hypotheses are testable explanations for phenomena. The null hypothesis is the hypothesis that explains any differences as the result of randomness/chance. Until sufficient evidence is discovered, we cannot adopt new (alternate) hypotheses.  

    The specific alternative hypothesis the teacher proposed (Halt) was: The differences between jumping heights of people can be predicted by the type of shirt they wear. 

    The null hypothesis (H0) for the t-shirt example would be: The differences between jumping heights of people is not related to the shirt they wear. 

    How can we figure out which is correct? This is where chi-square analysis comes in. 

  3. Practice
  4. Explain to students that you’ve filled each bag with 30 beads of five different colors. Their job is to determine if you did so randomly or with a bias. Check students: Which explanation is the null hypothesis? (Answer: The null hypothesis is that they were distributed randomly, with no bias.) 

    Ask: if the null hypothesis (no bias) is true, what would you expect the distribution of the beads to be? (Answer: I’d expect with 30 beads and 5 colors that there would be 6 of each.)  

    How might a scientist/statistician determine how far away the actual (observed) numbers are from the expected? Allow students time to brainstorm potential mathematical solutions. They should be able to come up with subtraction as the way of measuring difference. 

    The equation for chi-squared is: 

    x2 = [SUM] (o – e)/ e

    o = observed (this would be how many of each category there actually are) 

    e = expected (this is the amount predicted by random chance) 

    The difference represents how far each value is from the prediction of the H0

    The differences are squared to eliminate negative numbers. Then they are divided by the expected value. This is like taking an average. 

    The resulting values are all added up (denoted by the sum, or sigma on handout). The larger the differences, the larger the chi-squared value. Note that there is no consideration of sample size here, so a larger sample will in general mean a larger chi-squared value. 

    Explain: Chi-squared is a value that measures how far the observed is from the expected. If it is far enough away, we can reject the null hypothesis. In other words, we can say in this case that you, the teacher, did not act randomly. Instead, for some reason, you selected which beads to give to each group and therefore acted with bias. Naturally, this test cannot ascertain where this bias comes from. Ask students to come up with their own alternative hypotheses. Examples may include: 

    1.) The teacher prefers to use a certain color bead over other color beads. 

    2.) The teacher choice was random, but the original source of beads did not have equal numbers of each color.  

    Have students follow the directions on the handout to test their own bag. The will need to calculate degrees of freedom (number of colors – 1) and be taught to use the p-value chart. All tables and charts are provided on the worksheet.  

  5. Summary
  6. Students share out results as group, come to group decision as to whether there was bias in the distribution of beads. 

  7. Apply
  8. Students are presented with real demographic data of neighborhoods in New York City from the Department of Education. The first task is to determine what the null hypothesis is: If the U.S. census tells us that 29% of New Yorkers identify as Hispanic, what would the null hypothesis state about the population of people in a given neighborhood of the city? (Answer: We’d expect 29% of the people in each neighborhood to be of Hispanic origin.) 

    Students choose a neighborhood from the provided data handout to test. Since chi-squared only works well with smaller sample sizes, they shouldn’t use the raw data. They can use the percentages as the sample, as if each neighborhood had only 100 people in it. This work can be done on the handout with calculators. 

  9. Analyze the Data
  10. Ask students to share their results. Why did every neighborhood have a significant chi-squared value? What does that mean, statistically? (Every neighborhood has a significant chi-squared value. That means we can reject the null hypothesis that people are distributed randomly.) 

    Reinforce: the data show that populations are not distributed randomly in these New York City neighborhoods – but the data also cannot tell us why. The Chi-square analysis simply lets us know that bias of some form is at play, and that the difference between the actual outcomes and the expected outcomes is statistically significant. In this scenario, determining why this might be the case is traditionally the realm of the social sciences – not statistics. 

  11. Discussion: the Nature of Social Sciences
  12. Use these questions to guide an in-depth discussion and/or writing activity on the potential applications of the skills the students have just learned. 

    Explain to students that chi-squares are traditionally used with populations that don’t exert choice (as in the case of the beads or skittles in the first activity). Analyzing human behavior has generally been the realm of the so-called “social sciences” (e.g. psychology, economics, sociology, political science, and history).  

    Ask: What do the “hard sciences” and “social sciences” have in common?  

    Ask: What challenges are unique to social science? Why is it more difficult to draw conclusions from a study of humans than one of plants or even model animals like mice? Is it useful to quantify human behavior? What potential use are demographics? What are the limitations of statistical studies like this one? 

    Ask: Why does it make sense that the we could reject the null hypothesis in all the neighborhoods we analyzed? (Expect answers along the lines of: people choose where they live, or people are limited by what they can afford.)  

    On the topic of the choices of immigrants: Why might groups of people choose to live near each other? 

    On the topic of other external factors: What other factors might limit the choice of immigrants in terms of where they can live? 

  13. Discussion: Introduction to Historical Factors
  14. Throughout New York City’s history, certain groups of people have been coerced or forced to live in certain areas – or, at the very least, have been kept from living in the most “desirable” parts of the city. In the middle of the 20th century, redlining – in which African Americans were denied mortgages within traditionally Black neighborhoods – prevented many African Americans from owning homes (and thus joining the middle class), and helped concentrate African Americans within certain sections of the city. On the other hand, many African Americans moved to traditionally Black neighborhoods like Harlem, which served as a safe space amidst the violence of Jim Crow America and allowed the freedom of creation, thought, and speech that fomented the Harlem Renaissance.  

    With the push-pull of history at play and multiple historical factors and contexts informing the choices (or lack of choice) of human actors, can we use statistical analysis as a meaningful tool in history? If you were writing a history paper, would you consider including chi-squared analysis to support your argument? 

Additional Resources 

Use this website to teach about the history of redlining: how 1940s policies resulted in de facto segregation in cities: https://dsl.richmond.edu/panorama/redlining/ 

For a comprehensive history of redlining, see Richard Rothstein’s The Color of Law: A Forgotten History of How Our Government Segregated America (Liveright, 2017). 

Fieldtrips: This content is inspired by the World City, 1898-2012 and Future City Lab galleries in the Museum’s flagship exhibition, New York at Its Core. If possible, consider bringing your students on a fieldtrip! Visit http://mcny.org/education/field-trips to find out more. 

Acknowledgements

This series of lesson plans for New York at Its Core was developed in conjunction with a focus group of New York City public school teachers: Joy Canning, Max Chomet, Vassili Frantzis, Jessica Lam, Patty Ng, and Patricia Schultz.

This project was made possible in part by the Institute of Museum and Library Services.

The views, findings, conclusions or recommendations expressed in these lessons do not necessarily represent those of the Institute of Museum and Library Services.