Note: This blog entry is, in part, a response to a news report of my comments/questions about the 2011 New York Community Study. It’s a brief summary of my key questions and also reflects conversations with the research team, lead by Prof. Steven M. Cohen.
Leonard Saxe, director of CMJS and SSRI
In June, UJA-Federation of New York released its long-anticipated 2011 population study. For some, the headline was stunning: The Jewish population of New York had grown more than 8% during the last decade, pushed by high birthrates among Orthodox women. Along with the increase in the Orthodox population, there was also a dramatic increase in poverty rates and evidence of a continuing downward slide in the Jewish engagement of non-Orthodox Jews. The results were riveting and suggested the need for a major reorientation of UJA funding priorities. The study used state-of-the-art methods, but is the narrative of a changing New York Jewish community, as told by the survey, accurate? How do the outcomes reported in the study compare to non-survey data and what we know from other sources about New York Jewry?
The overall finding of a population increase of more than 8% (from 1.41 million in 2002 to 1.54 million in 2011) matches our own findings of an overall increase in the U.S. Jewish population and an increase, in the New York area in particular, of about 9%. The proportion of the total Jewish population that is in New York also comports well with our sense of how the population is distributed. The New York Community study’s estimate of 1.5 million is 23% of the 6.5 million we estimate to be the 2010 U.S. Jewish population.
The overall population numbers can, thus, be validated, but there is a secondary question concerning whether the study accurately describes the characteristics of New York households and individuals. Were some in the community more likely to be identified and participate in the study than others? And did this yield a different picture of the community than is actually the case? Given the finding of growth of the Orthodox population and decline in Jewish engagement by the least identified, did the study capture the right proportions of each group?
Although we do not have a direct measure of the size of the Orthodox population, several highly reliable sources of related data are available. The most readily available and comparable data are enrollment figures for New York’s day schools which, according to UJA’s study, serves a student body that is more than 90% Orthodox. According to the report, 145,000 children (5 to 17-years-old) — 66% of all Jewish school-age children in New York — were in day schools during the 2010-2011 school year (p. 185). Per the findings of the study, 10% of children in Reform households and 32% of children in Conservative households attend a day school.
The day school estimates can be compared to a host of official surveys and reports, including The AVI CHAI Foundation’s day school censuses conducted by Marvin Schick. These non-survey data suggest a very different picture. Per analysis of these sources by researchers at the Cohen Center/Steinhardt Institute’s JData project, the census of New York area day schools yields only 96,000 students during 2010-2011. JData researchers not only verified data from individual schools, but compared it to official lists and published reports of enrollment (in particular, the findings to the 2009 day school census produced by AVI CHAI/Marvin Schick).
The New York report appears to have found 50% more day school students than actual enrollment figures indicate. The discrepancy is far greater than could be attributed to sampling error. Is it possible that official statistics on day school enrollment are inaccurate and underreport the number of students? Anything is possible, although it seems unimaginable that there are 50,000 day school students in the New York area who have been missed by multiple censuses and are excluded from official statistics (see Schick, 2009). Some students may be sent from their homes to day schools outside of New York, but the net effect (if one subtracted those who come to New York for elementary or high school) seems unlikely to be tens of thousands of students.
The apparent overestimate raises broader questions about the reliability of the community survey data. Does it, for example, suggest that the report overestimates the number of Orthodox families or the size of these families? The “extra” 50,000 students who were estimated to be in day schools potentially translate to an overestimate of 100,000 or more in the Orthodox population. And what about the number of children from non-Orthodox families in day schools or part-time schools? Are the numbers inaccurate and/or are the proportions incorrect?
There are other seeming anomalies in the results, as described in the report of the UJA-Federation study. One concerns young adults and, in particular, Taglit-Birthright Israel alumni. The study acknowledges that Taglit alumni seem to be engaged with the Jewish community, but suggests that there are not enough of them to challenge the tide of assimilation. But it’s not clear whether they reached alumni in proportion to their numbers in the population. The study seems to indicate that there are 11,000 non-Orthodox Taglit alumni (see p. 180), but the authors explained (private correspondence) that this is only an estimate of respondents (not all household members), and that there are 45,000 alumni (Orthodox and non-Orthodox).
Taglit’s registration system and evaluation data indicate a somewhat higher number (closer to 50,000 as of the time of the study) and, given the mobile nature of this population, it’s difficult to know exactly how many alumni there are. We do, however, know that New York is a magnet community and that many young adults migrate to New York after college. The question, however, is why there appear to be so few alumni among respondents.
Because Taglit, which began in December 1999, serves 18 to 26-year-olds, the alumni are concentrated in the age group that was 26-32 at the time of the survey. This is a population that is extremely difficult to reach. They rely on cellphones rather than landlines and, if they are not native New Yorkers, are unlikely to have a New York area code. Thus, along with the difficulties of contacting cellphone users, many were outside the sample frame. One indication of an issue is that the estimated population size of those 25-34, as found by UJA’s researchers, is dramatically lower than the size of the population of those 18-24. The 18-24 group includes an average of about 23,000 individuals in each age cohort, while for those 25-34, cohorts average 13,000 per each age cohort.
As with the day school data, the discrepancy with the young adult population is potentially important and affects the study’s conclusions regarding the relative size of the Orthodox population. It also affects our understanding of the impact of a host of Jewish identity initiatives, including Taglit. Until the study dataset is released, however, it won’t be possible to assess alternative explanations and the implications of making different assumptions about the sample.
Unlike election and other social surveys, socio-demographic surveys of the Jewish population cannot be adjusted or compared to census data about religious identification. This means that conducting Jewish socio-demographic research is inherently difficult. New York, however, because of its size and the wealth of data available about it, provides an opportunity to compare survey data to known information and validate the findings. My present effort to do so has raised a host of questions. There appears to be a lack of concordance between the findings of the survey and known population parameters. As the community tries to draw out the implications of its changing character, these anomalies call out for understanding. Further analysis is part of the process of using science to advance a field and will, as well, redound to the benefit of the community.