Yay! Signs a Seattle Homelessness Program is Working (But Can we Believe the Report?)

Earlier this week, Seattle City Councilmember Andrew Lewis held a press conference in downtown Seattle, the heart of his district, to highlight the first visible progress made in reducing the number of homeless encampments in the area. Lewis gave all the credit to the JustCARE program. He also highlighted the release of a report by a research team from the UW Department of Sociology that lavishes praise upon JustCARE. But is that praise deserved? Turns out, we don’t know.

The report is the work of UW sociology professor Katherine Beckett and her team of grad students. Beckett is a well-known, well respected researcher on the criminal justice system; in 2018 her research on racial bias in Washington’s application of the death penalty led to a unanimous opinion by the state Supreme Court ending its use. Beckett also did the first study on the LEAD arrest-diversion program, back in 2014.

Here is how the JustCARE report describes itself:

“This report is intended to provide policymakers, practitioners, and other interested parties with a detailed overview of what JustCARE is, how it emerged, the work it accomplishes, and lessons it has learned. It also offers policy recommendations aimed at enhancing JustCARE’s impact and recommendations for data collection that will facilitate ongoing evaluation.

“The analysis draws on several different types of data, all of which were collected from April 2020 through March of 2021. These data include interviews with:
• Forty-two JustCARE participants, many of whom were interviewed multiple times over a period of months.
• Twelve outreach responders and other program staff.
• Ten leaders of participating organizations.
• Eleven community members representing organizations of people who live and/or work in impacted areas.
• Seven stakeholders involved with alternative crisis response models across the United States.
The data also include observation of select JustCARE planning and operations meetings, review of administrative documents, and analysis of administrative data provided by REACH, PDA, ACRS, and Chief Seattle Club.“

It’s worth examining the evolutionary path from LEAD to JustCARE. LEAD began as an arrest-diversion program, focused on low-level drug possession and prostitution offenders, wherein police officers referred individuals to the LEAD program in lieu of arresting them. LEAD also provided case management and services to help the individuals find a new path and exit the revolving-door of the criminal justice system. From its early days as a small, privately-funded pilot program, LEAD grew slowly, attracting public funding along the way. It also evolved from a program that mainly took referrals from SPD to one that mostly took referrals from community members; in fact, recently the program was changed to eliminate SPD from the approval process for new referrals.

Last year the COVID shutdown curtailed some of LEAD’s regular activities, but also created a new opportunity when King County Jail began to “de-densify” by releasing low-level offenders, but without any real plan for their re-integration into society. Thus was born CoLEAD, a new version of LEAD targeted at those being released to prevent them from ending up back on the street and ultimately back in the criminal justice system. CoLEAD took a “housing first” model, placing its clients in hotel rooms where possible.

Then in late 2020, as problematic encampments began to overwhelm the streets of downtown Seattle, a coalition of human services organizations banded together to create JustCARE, another twist on the LEAD/CoLEAD model but focused on homeless individuals downtown who are creating problems for the neighborhood — many of whom would otherwise be LEAD clients if the program were to scale up to handle them all.

LEAD, CoLEAD, and JustCARE are all coalition-based service programs, with multiple organizations providing part of the portfolio of services offered to the programs’ clients. Across all of them, the central coordinating organizations have been the Public Defenders Association and Evergreen Treatment Services’ REACH outreach program.

Beckett and her team were funded by several progressive nonprofits, including Blue Meridian Partners and UW’s West Coast Poverty Center, to conduct an evaluation of the first six months of JustCARE’s work. This week they issued their glowing, wholly uncritical, report on the program.

Beckett was a friendly pick for the evaluation. She has a long history of research findings matching the underlying philosophy of the LEAD/CoLEAD/JustCARE programs. But there’s a problem with the report’s purported independence. As documented in Beckett’s curriculum vitae posted on her UW web site, her research team is partially funded by the Public Defenders Association through a three-year grant stretching from 2020 to 2022. The grant topic is “Exploring Alternative Approaches to Behavioral Health and Extreme Poverty: Promise and Challenges” — overlapping significantly with LEAD and JustCARE. Beckett has a seeming financial conflict of interest in evaluating a program run by an organization that is funding her research.

Worse, Beckett did not disclose in the report issued this week that she receives funding from the Public Defenders Association. Such disclosures are required for this kind of work, and omitting it raises a significant ethical issue. Simply having the conflict of interest would cause us to view her report with a great deal of skepticism (especially for a report so glowing), but not disclosing the conflict raises much larger questions about the trustworthiness of the report — and the researchers who worked on it.  The report contains literally zero criticisms of JustCARE, though it does opine on several ways in which Seattle and King County should do more to help JustCARE, including giving it more funding. Calling it an evaluation is a stretch; “advertisement” is closer to the truth. Sadly, we now must decide whether it’s a paid advertisement.

If you’re wondering what a more robust evaluation of JustCARE would look like, it turns out we have an example. In 2019, the City of Seattle paid a consultant firm, Bennett Midland, to conduct an evaluation of the LEAD program. The report was delivered last year, including an update for CoLEAD. The Mayor’s Office buried the report, declaring that “as a draft it’s still under the deliberative process,” a reference to an exemption that governments may use to avoid producing documents under the Public Records Act.  Fortunately a persistent SCC Insight reader eventually managed to obtain it through a Public Document Request, and provided it to us. And now we can share it with you.

Now it must be pointed out that LEAD and JustCARE are not the same; JustCARE is an evolution of LEAD, with a different mix of providers, clients, and outcomes. Every criticism you read in the Bennett Midland report (which is very much worth reading) isn’t necessarily a valid criticism of JustCARE — or isn’t necessarily anymore, as some of the issues may have been addressed in the intervening time. But reading the Bennett Midland report gives us a much clearer idea of the kinds of questions that should be asked about JustCARE, and simply weren’t by Beckett and her team.

For instance, it raises questions about the extraordinary amount of discretion for decisions that is delegated down to LEAD front-line workers. In many cases that delegation of authority may be appropriate in order to provide flexible case management and services that are client-centered, but it leaves us knowing little about what does and doesn’t work. That is especially true in light of the scant documentation that LEAD generates about its clients, its case management, and the program in general, making it difficult to track outcomes (and to tie those outcomes back to the type of services provided). That also means that there is little understanding of which clients LEAD is best positioned to serve.

Many of these critiques apply equally to JustCARE. To be sure, workers use creativity and discretion with the intent of benefiting the program’s clients; the Beckett report provides examples of using gummy bears to calm someone down, and providing “tinker boxes” with items that clients using methamphetamine can tear apart as an alternative to disassembling the TV in their hotel room. But the report also gives the more questionable example of a JustCARE worker giving a client on a “meth binge” a cannabis edible to help them to sleep:

“One night, we were really struggling with a guy that has meth use but also mental health issues. He was coming and going throughout the day. The hotel was really upset with him. He was stealing things, and we really thought he was on a meth binge, and maybe on day four or five. So, we put together a food box, and I added a cannabis edible to it, and then got it all ready to go. And then, by the time we left, he still hadn’t come back. So, the safety team, we asked them to post up there. They were there when he got back. They were able to say, ‘Hey, man. How’s it going? CoLEAD left all this stuff for you. We’d like to just get you settled into room and take a load off, maybe get some sleep. We’ve got this to help you sleep.’ And it kept a whole situation, that I know would have gone down with hotel staff, from happening.”

(A note about this incident: JustCARE and Beckett state clearly that no federal or state funds were used to pay for the cannabis product given to the client.)

The Beckett report unflinchingly praises all of this, and fails to even raise the meta-questions about who is making these decisions, whether they have the qualifications and experience to do so, and whether the decisions are being documented in a manner that allows for analysis, pattern identification, learning, and accountability.

But the biggest issue with the Beckett report, in declaring JustCARE to be a success, is that it implicitly defines success to be “keeping the clients housed in hotel rooms and out of trouble.” According to the report, of the 225 clients who have participated in the program, 21 have exited to permanent housing, 30 were removed from the program, one is deceased, and the remainder are still living in JustCARE-managed hotel rooms.

To be sure, there is a critical lack of permanent affordable housing and permanent supportive housing in Seattle to place them into, which is not JustCARE’s fault. But that doesn’t make the program successful just because the clients are still enrolled. Rather, it means that we don’t yet know whether JustCARE will be successful. The goal of JustCARE can’t be just to shelter and support chronically homeless individuals in hotels indefinitely. That result is far too expensive, the hotel rooms won’t be available forever, and there are better, more cost-effective alternatives (such as tiny homes). Hotelling homeless individuals is a great short-term measure, but a poor long-term one — even though it helps with other issues such as the public safety concerns raised by large encampments on downtown streets and in public parks.

And that points to one other issue with the JustCARE program that the Beckett report declares as a feature of the program: it tries to be everything to everyone. It’s a “housing first” intervention. It’s a health intervention. It’s a public safety/”quality of life” initiative. It’s a decarcerative strategy. First problem: there aren’t good metrics in place to measure how well JustCARE is doing at each of these, let alone all of them. Second, there will be natural and inevitable tensions between those goals, and there is little process for deciding how to manage those tensions — and the different stakeholders they affect.

The Beckett report does admit that it provides only “initial insights regarding the evolution and impact of JustCARE in its first year of operations,” and then it offers a long list of the data required to do “a more robust and ongoing assessment of JustCARE’s impact at the participant, neighborhood, and system-levels.” More than anything else, that list of missing data highlights just how shallow and data-poor the current evaluation is:

Professor Beckett sent a letter to the City Council in response to this article, posted earlier on my blog, and provided a copy of it to me. Here it is, in its entirety.

It’s important to remember that the absence of a robust, trustworthy evaluation does not mean that the thing being evaluated isn’t good. It simply means that we don’t know. Contrary to Councilmember Lewis’s public pronouncements, we don’t yet know whether JustCARE is a valuable investment. We know that it’s moved many homeless individuals off 3rd Avenue into hotel rooms (to the benefit of downtown businesses and residents), and it seems to have found some success in helping them to remain there. We also know that the current situation is not sustainable indefinitely, nor can it scale up without a path for the current hotel occupants in the program to move to permanent housing. What we don’t know is what happens next to the set of people currently enrolled in the program, though that will ultimately be the determinant of JustCARE’s success.