The Neighborhood Vitality Index is a tool to track progress and advocate for resources to make changes in Detroit’s neighborhoods. The index measures things Detroiters actually care about—including health, well-being, feelings of safety, and more—by including data from multiple sources, especially from Detroit residents. Community Development Advocates of Detroit is partnering with JFM Consulting Group and Data Driven Detroit to manage the Neighborhood Vitality Index as part of a larger effort to strengthen the community development ecosystem in Detroit. The goal is to strengthen all of Detroit’s neighborhoods by creating a well-coordinated, effective, and equitable system for community development work in Detroit. The Neighborhood Vitality Index was created because we need regular access to data that can track changes in the system and changes in outcomes for people over time.
There have been several past attempts to create indicator and index projects for Detroit and the region (One D Scorecard, Southeast Michigan Regional Opportunity Index, Turning the Corner, Detroit Drilldown, etc.) Most, if not all, were launched, and pretty quickly thereafter, shelved. We identified barriers to adoption based on these past efforts, and designed our approach to the NVI with the aim of surmounting these barriers.
| Past Barriers | NVI Approach |
|---|---|
| Designed and developed by a small group of people, often researchers | Designed by local stakeholders representing a variety of sectors, with input from hundreds of people over several years |
| Created as a standalone project with little or no consideration of other existing local resources | Conducted a landscape scan of other local resources, created alignment where possible, and leveraged an existing data infrastructure |
| Developed without an audience in mind | Identified primary and secondary audiences for a variety of products using the same data infrastructure |
| Paid for by a single philanthropic funder as a one-time, high cost project | Funded by eight philanthropic partners who anticipate using the data as both a baseline and a measure of progress over time for their other investments in Detroit |
| Scoped as a fully conceived project, designed from start to finish, and implemented accordingly | Used a phased approach to conduct smaller, low-cost tests along the way to learn at a small scale and build from there, while also documenting and publishing lessons learned along the way |
| Used nationally-available data for convenience and to ensure future availability | Combined widely available secondary data with primary data about residents’ lived experiences |
| Presented data at geographic levels that aren’t useful to a broad array of stakeholders | Designed “Neighborhood Zones ” with input from local stakeholders representing a variety of sectors |
| Little forethought for how the data may be weaponized | Designed products with multi-sector input on how the data may be perceived and utilized |
| Published without a plan for activating the data once released | Created a framework for mapping stakeholder interest and establishing action tables |
The roots for the Neighborhood Vitality Index began back in 2016 with the launch of a new initiative called Building the Engine of Community Development in Detroit (BECDD). BECDD was a citywide initiative to strengthen all of our neighborhoods by building a coordinated, equitable system for community development work in Detroit. In the words of the initiative’s champion Maggie DeSantis, “our first goal was to understand why community development was not working like it could work for our neighborhoods. To develop this analysis, we convened a series of eight Kitchen Cabinets, meetings where like stakeholders could candidly speak about the challenges of community development in Detroit. Next, we formed temporary subcommittees, mixing the stakeholders together, to review that analysis and lay a roadmap for the process in the ensuing years. By the end of 2016, we completed a Detroit-led analysis of the problem, a set of 7 System Elements fundamental to ensuring successful community development work in Detroit, and a beginning sense of what the solutions might be.”
In 2016, the BECDD initiative created a Design Subcommittee on Research and Development. Through this Subcommittee, an early-stage idea of a measurement and data system began to emerge. The group was focused in particular on the idea of social cohesion, and explored ways of measuring neighborhood progress that weren’t only rooted in traditional demographic and socioeconomic indicators, which always seemed to fall short of measuring what residents actually care about. With this framing, it became clear that using only traditional, secondary data sources was not going to meet the needs of our community, so the committee began discussing potential primary indicators that could complement traditional secondary data sources. We prepared a list of these potential indicators for the 2016 BECDD summit, and shared the list with other members of the initiative for review and discussion. Out of this discussion came the Consensus Framework for Detroit Neighborhood Success Measures .
In early 2017, a group of researchers at Lawrence Technological University conducted a literature review focused on the idea of social cohesion that became an important foundational document for stakeholders moving forward. In particular, the review noted that social cohesion is a “necessary but insufficient” condition for strong neighborhoods, laying the ultimate foundation for the notion of “neighborhood vitality” to become a cornerstone of the developing data system element. Equally important to this work, the report detailed the following findings, all of which were considered in the design and development of the NVI:
"The literature stresses that approval of indicators from the community is a key to the success in data evaluation process. Other community-based approaches such as setting neighborhood boundaries that make logical sense to community organizations (as opposed to census tracts, for example), and overlaying economic data like the unemployment rate, etc., are likely to encourage the community to have a sense of ownership about the community progress assessment and actively participate in the evaluation process (Parenteau et al. 2008). All of this, as the literature suggests, should be done with open and committed sharing of data by the city government (e.g., that of Milwaukee), even when they know it has the potential to be used to challenge their own policies (Ghose, 2003). As to organizational assessment, the risks of diversifying into service areas beyond traditional expertise can lead to failed ventures, which can create distrust and further lack of financial support. Being honest with support communities about any financial issues is also listed as a key to organizational success in the literature (Bratt & Rohe 2005)".
Around the same time, Tom Burns was engaged by BECDD to support the initiative. Tom's unique expertise flowed from his work with foundations, ground-level community development organizations, and local/national intermediaries. Because he worked in multiple cities, and with multiple types of organizations, brought a unique perspective that was both global and local. He offered insightful coaching and strategic advice based on an understanding on how these different kinds of organizations operate - and the challenges they face - in the community development arena. He also felt that our work in Detroit, unlike the other cities he had long studied, was the first effort he'd seen to "formalize" a community development system - to intentionally and thoughtfully study the elements of such a system.
BECDD decided to engage him directly, through his consulting firm Urban Ventures Group, to look at five cities with"mature" community development systems . His conclusion from that engagement was that there wasn't going to be a "silver bullet" model we could copy, but that there would be lessons we could take from each example. He identified community development systems that weren't necessarily "models", but were examples of community development principles and practices that were in play for long periods of time in these cities, and had experienced ebbs-flows, problems and solutions, and could offer learnings.
Tom passed away after a brief illness in August 2018 in the midst of our work together. We miss him, and we are so grateful for the wisdom he contributed to this work in the last few years of his life. His legacy reaches far and wide, and we hope this work does him proud.
By 2017, a Research, Data, and Evaluation Planning Team was established to carry forward the work of the 2016 Design Subcommittee. The group continued to flesh out the idea of social cohesion and how it plays into the potential measurement of success in neighborhoods. This involved research and a number of facilitated discussions around several options for “naming” the new framework. The process also included a survey of other BECDD stakeholders as well, to solicit input from the larger group. This led to the selection of “Neighborhood Vitality” as the descriptive moniker for the new framework. Neighborhood Vitality met the criteria established by the Research, Data and Evaluation planning team in that it would be comprehensive enough to capture a wide range of indicators; resonate with a cross-section of stakeholder groups; align with the indicators under consideration; and it would be actionable, allowing CDOs, the city, or other stakeholders to use the framework to measure neighborhood success.
Having identified Neighborhood Vitality as the overarching framework, the work of developing a geographic framework got underway. The process of defining neighborhoods for purposes of collecting data was not going to be easy. The City of Detroit, for example, had identified more than 200 neighborhoods, an unmanageable number of neighborhoods from a data collection standpoint. Data Driven Detroit would play a key role in identifying neighborhood boundaries, but the bulk of this work would take place in 2018. All of the planning activities described here culminated in the 2017 BECDD Annual Summit, where the broad array of participants in attendance gathered to provide feedback and recommendations .
Beginning in 2018, the Success Framework Task Force took on the task of determining the indicators that would comprise the Neighborhood Vitality Index. Beginning with research on indicators that other similar projects around the country have used, the indicators were then developed, debated, and refined. Four distinct categories of indicators emerged: Community Capacity, Resident Opportunity, Neighborhood Conditions, and Quality of Life. Thought was given to aligning the indicators with the Mayor’s dashboard indicators where appropriate to help drive buy-in from the public sector. In addition, using the definition of equity developed by another task force, the group flagged indicators that had some relationship to the BECDD’s equitable development framing. This process resulted in the development of an extensive list of potential indicators. As a result, the Success Framework Task Force engaged in additional discussions and participated in polls to identify a feasible number of high priority indicators to include in the Index. Further, the Task Force engaged in discussions that resulted in the development of clear definitions for each indicator. Some early product design came out of this group as well, including discussions of strategies to help ensure that the data would not be “weaponized” or used in ways that would harm communities.
Also, in 2018, the Defining Neighborhoods Task Force engaged in a series of robust discussions around identifying a geographic framework for the NVI. Building on the work that began in 2017, Data Driven Detroit developed maps that were built from City of Detroit neighborhoods to align with City Council Districts. Council Districts were divided into smaller Neighborhood Zones, where every effort was made to ensure that the zones generally did not cross census tract boundaries, and that, if they did, the differences would be so small as to be negligible from a reporting perspective. The Neighborhood Zones would provide the geographic boundaries for the purposes of collecting data. The Neighborhoods Task Force ultimately created a final Neighborhood Zones map with 23 Zones in total,comprised of between 3-4 zones per Council District. Similar to 2017, additional feedback and questions from the broader community were gathered at the 2018 BECDD Annual Summit.
In 2019, JFM and D3 were designated as the “implementation team” for the NVI. Realizing that this was going to be a large undertaking that had no precedent in Detroit, JFM and D3 decided it would be best to use a phased approach to develop the NVI. We would do smaller scale pilot tests of each component of the work, and re-tool for each subsequent phase based on lessons learned along the way. From 2019 to 2020, D3 and JFM engaged the Success Framework Task Force to provide input on the pre-pilot stakeholder engagement and develop consensus on next steps, including transitioning responsibility for the work to an expanded group of stakeholders: the Neighborhood Vitality Index Task Force. In advance of convening this expanded group, JFM and D3 developed a more user-friendly overview of the NVI Framework, conducted one-on-one interviews with City officials, funders, and others to solicit input on their priorities, and gathered feedback from community development organizations through focus groups on the following topics: geographies; structure and content of the survey tool; priorities and end products; data collection process. In the 2019 BECDD Annual Summit, participants shared their thoughts on priorities and gaps in the framework at the time.
Meanwhile, in parallel with this phase of work, D3 and JFM also worked with Doug Stewart from the Max M. and Marjorie S. Fisher Foundation to explore the concept of “Gross Domestic Opportunity.” The concept was designed as a high-level, more impact-focused measure that could complement (or even supplant) Gross Domestic Product as a key indicator of national progress. This process led to us better understand the role that NVI could play in the broader data ecosystem: if a measure of Gross Domestic Opportunity could capture the hearts of actors across the country, perhaps the Neighborhood Vitality Index could complement as a locally-informed and locally-relevant dataset to engage the minds of stakeholders. With community-rooted data driving informed decision-making on the ground, perhaps we could see collective action move the needle at the local level and, if the model were replicated in other places, we could eventually see progress at a national scale as well.
The first phase of the NVI pilot was focused mainly on testing the survey instrument and gathering feedback on potential ways of consuming the information after it was gathered. The original plan was to launch the survey in two Neighborhood Zones, testing different ways of contacting and incentivizing residents to see which methods garnered the best response rates. Then COVID struck and in-person data collection was no longer possible. The implementation team pivoted to a snowball survey strategy, reaching out to residents mainly through CDOs to collect primary data from anyone in the community willing to participate. The team then received feedback from residents to understand which questions worked and which didn’t. Using these responses and the available secondarydata from the list of NVI indicators, D3 and JFM created a basic methodology for calculating the NVI. During this phase, D3 also created a preliminary wireframe for the first planned data product in order to gather feedback from the Task Force and other stakeholders.
The D3 and JFM teams also spent time conducting interviews with other local groups involved in surveys and indicator projects to determine whether and how the NVI would add value to the local ecosystem. Once established that the NVI was bringing something unique to the system, we sought to understand where we could better align with these efforts and document how other efforts were distinct in their focus or intent. This is all documented in the alignment matrix we shared back with stakeholders in 2021.
During this phase we ended up adding two additional sections to the NVI framework, in partnership with various stakeholder groups. The emergence of the pandemic and the racial reckoning of summer 2020 had illuminated a lot of equity issues that the NVI framework was not well-equipped to address; this led to the creation of the Equity and Impact section of the NVI framework. In contrast to other sections of the framework, Equity and Impact is intended to evolve over time to enable NVI to be responsive to pressing issues affecting Detroit communities. During this phase, we also aligned NVI with a Youth Perceptions of Hope and Opportunity Survey that JFM was conducting for The Skillman Foundation, which led to the creation of the initial Youth Quality of Life section of indicators. Lastly, at the conclusion of the survey, focus groups were conducted with residents who participated in the survey in an effort to identify potential issues with the instrument and data collection process and to make informed course corrections, as needed.
In the second phase of the pilot, JFM and D3 revised the indicators and survey instrument based on feedback gathered during Phase 1. The team then piloted a formal, statistically rigorous primary data collection effort in Council District 7, which resulted in 313 responses. D3 interviewed three organizations from the National Neighborhood Indicators Partnership with similar indicator projects to understand how they steward data, ensure relevancy over time, and identify the long-term outcomes and lessons learned from each project. D3 began the stakeholder mapping process in this phase to prepare for a more action-oriented first year of full implementation. A draft map was created to demonstrate the potential, and further work on this will take place in 2024. Similar to Phase 1, JFM solicited feedback from residents on the survey and the process to inform improvements as the pilot moved into Phase 3. Also during this phase, the first interactive wireframe was created based on feedback from stakeholders.
During the third phase of the pilot, JFM and D3 further refined the indicators and survey instrument, and expanded the scope of the survey to cover the entire City of Detroit. Beginning in February 2023, the team sent out the survey to 4,400 Detroit households using a random sampling strategy. After a few months, with a very low response rate, it became evident that random sampling was not going to give us a sufficient sample size within our project budget and we pivoted to a convenience sample approach that launched in June 2023. The convenience sample strategy allowed for much greater engagement with local partners: canvassers attended community events with paper surveys and flyers, our community development partners encouraged their residents to fill out the survey, and local media partners got the word out to even more residents. This change in tactics resulted in the collection of over 2,500 responses, enough for the online tool that we plan to release in 2024 to present results at the Detroit City Council District level.
Surveys in Phase 3 integrated some elements of the Center for Disease Control's (CDC) Behavioral Risk Factor Surveillance System. Annually, the CDC administers a variety of health-related surveys to the American public to understand how people’s behaviors, interactions, and access to services might impact their mental and physical well-being. Some segments of the population are less likely to respond to these surveys due to factors such as language barriers, housing challenges, and lack of trust in the system. The Robert Wood Johnson Foundation funded a project with the CDC Foundation to work with local data intermediaries like Data Driven Detroit to identify potential method improvements that leverage local relationships to improve those data collection methods. In Phase 3, the NVI survey incorporated pilot modules from the CDC’s Behavioral Risk Factor Surveillance System on reactions to racism and social determinants of health for respondents through mid-July. In addition, three questions from the Reactions to Race module were integrated into the Equity and Impact section of the NVI framework.
Being able to validate the NVI survey data with the nationally developed BRFSS survey, we noticed two important details from over 400 NVI survey respondents who completed the two additional modules (Reactions to Racism and Social Determinants of Health):
See History for a description of the process to decide which indicators would be part of the Neighborhood Vitality Index.
There are two distinct types of indicators included in the Neighborhood Vitality Index: primary (data collected through resident surveys) and secondary (data from a variety of existing administrative sources). The data presented in the current tool is from Phase 3 of the pilot process,with primary data from 2,463 Detroit residents. Due to the number of responses, we are sharing data at the City Council District-level, and since there are only seven Council Districts the overall index was not calculated for the current tool. After a year of full implementation (beginning in 2024), we expect to gather enough resident survey responses to present data at the Neighborhood Zone-level with indexing.
Primary data collection in Phase 3 consisted of a survey of Detroit residents launched in February, 2023. Eligibility criteria were based on age (18 years of age or older) and residence (primary residence in the city of Detroit; some exceptions were made in this phase for residents of Hamtramck and Highland Park though responses outside Detroit are not included in this tool). Eligible residents who completed the survey were provided $25 Visa gift cards to compensate them for their time and sharing information. Gift cards were limited to one per person, and duplicate surveys from residents who had already completed the survey once were disqualified.
The Phase 3 survey was largely the same as the survey tested in previous phases, with minor additions. Spanish and Arabic translations of the survey were made available online or on paper if requested. The median time to complete the survey was between 16 and 17 minutes.
The Phase 3 survey was implemented in two phases. Phase 3A would continue the random sampling design of Phase 2 at a city-wide scale, but to save resources and allow for changes to plans later, only a portion of the required sample size was initially invited. More than 4,500 Detroit households were sampled and invited via mail to participate in the survey. Though the sample was intended to exclude vacant properties, a small number of invitation packets were returned by USPS.
The mailed invitation packet was packaged in an attractive 9x12” envelope containing a letter from CDAD with instructions to participate online, a paper version of the survey , and a self-addressed, stamped envelope to return the paper survey. Household identification numbers were included to allow the survey team to track online responses from each household. The letter also provided Spanish and Arabic translations of the survey instructions. A phone number and email address were provided for questions or requests to take the survey over the phone. The households that did not immediately respond to the survey and for whom mail was not returned were sent reminder postcards a few weeks later. Phase 3A remained open until June, during which time 408 complete surveys were collected for a response rateof approximately 8%.
Given the pace of survey responses in 3A and projections of the resources and time that would be required to meet the sample goal, the survey team pivoted to a new strategy based on lessons learned through Phase 1 and Phase 2. In Phase 3B, the survey was opened to all residents of Detroit 18 years or older, regardless of where they lived in the city and without a personal invitation. The representativeness of the resulting sample would be determined through data analysis after the survey closed. By opening the survey to the whole city,outreach became simpler and was able to leverage the relationships with the CDOs that the NVI team developed over the last 7 years. Now the message would pertain to all residents of Detroit 18 or over, as opposed to “watch for an invitation in your mailbox.” Phase 3B outreach efforts included traditional media, social media, text message campaigns, and community engagement. More than 50,000 flyers were distributed to community members at local events, libraries, local businesses, and through the Rocket Community Fund’s Neighbor to Neighbor canvassing.
A drawback of the 3B approach with an online survey was an influx of responses from bots or spammers around the world. The survey team needed to verify that each complete survey response was eligible by contacting respondents. After verifying thousands of responses, the cumulative sample across 3A and 3B included 2,526 eligible complete surveys.
Survey respondents were asked to provide a valid current address in the City of Detroit. These addresses were geocoded and resulted in 2,463 matches within the City of Detroit. Responses were then tagged with Council Districts and indicators were aggregated based on the criteria in the data dictionary.
Secondary data processing derived from two main sources: the American Community Survey (ACS) and Data Driven Detroit’s Integrated Property Data System (IPDS). ACS data was downloaded from data.census.gov at either the census block group or census tract level, and aggregated to City Council Districts. IPDS data was extracted at the parcel level and aggregated to City Council Districts. Indicator details are included in the data dictionary.
Many thanks to the Neighborhood Vitality Task Force, the Resident Advisory Panel, our partners, and our funders for their help in creating this body of work.
Community Development Advocates of Detroit (CDAD). As a membership centered organization with more than 100 dues-paying members, CDAD advocates for public policies and resources that advance the work of nonprofit, community-based organizations and resident-led groups in Detroit neighborhoods who are engaged in physical development, land use planning, community organizing, and other activities designed to stabilize and revitalize the quality of life in Detroit.
Data Driven Detroit (D3). D3 is Metro Detroit’s community data hub with a commitment to driving informed decision-making through the provision of accessible, high-quality information and analysis. D3 helps to foster positive change, improve lives, and strengthen communities by offering data, analysis, and technical assistance to various entities, including nonprofits, foundations, governments, and others.
JFM Consulting Group, Inc. JFM is a Detroit-based strategy, planning and evaluation consulting firm that works in partnership with a wide range of public, nonprofit and philanthropic organizations to find solutions to the challenges facing urban communities. JFM strives to promote positive social change and increase the effectiveness of the nonprofit community through planning and evaluation services and technical assistance.
Michigan Nonprofit Association (MNA). MNA is a statewide membership organization that serves the diverse nonprofit sector through advocacy, inclusive services, programming, and resources. MNA serves as fiduciary for several grants funding this work.
