BetaGov Interviews

Embracing Change

Date:   July 17, 2017

Sandra Kimmons is Director of Economic and Employment Services in the Kansas Department for Children and Families (DCF). Her team completed a trial to test alternative versions of the Low Income Energy Assistance Program (LIEAP) application form, to improve the rate of form completion. 

What inspired your trial idea?

We were having a discussion about staffing and resources, and someone mentioned there seemed to be a high number of applications returned to clients that would delay their eligibility. We dug a little deeper, and it became obvious that faxed applications were often missing the required signature. After some brainstorming, it was mentioned that the last page wasn't always being faxed. The application was two-sided, with the signature line on the last page. This led to the idea that perhaps we should change where the signature was located on the form. We were able to reformat the form so the signature was on an odd-numbered page. We changed the signature line from page 6 to page 5.

We were interested in BetaGov because we had heard of some private-sector experiments that made an impact, so we thought we would like to try this and get the actual data, and not rely on anecdotal evidence.

Did you think it would be feasible to test?

Our team was unsure how we could measure the outcomes, so we had to determine if it was feasible to test. We were really unsure of the next steps, and had concerns on how to make sure it was random as well as figuring out how to measure results because we receive tens of thousands of applications across the state. BetaGov provided guidance on how we could form the idea into a study and measure it.

What were some of the challenges you encountered with your trial, and how did you overcome them? 

Setting up the control group was not clear to our staff. BetaGov provided technical assistance along with a plan with specific instructions on how to ensure the trial was random. We selected two offices in different geographical areas that were similar in size and to be participants in our study. We had two different versions of the form—one with the signature form on an even page, and one in the original format. We had to have a way to keep track of who was getting which form. The randomization was even more fascinating to us.

Tell us about the results and any changes you made after the trial.

Receiving an incomplete application delays processing by two or three weeks, because we make a hard copy and send a letter requesting the applicant's signature. Because it's a manual process, it also uses additional resources.

On every application we received by fax in the intervention group, the signature page was received. The number of faxed applications we received was minimal because of other factors, so it wasn't hundreds of applications, but we were able to process 100 percent of them, so we considered that successful and obviously adopted the new format statewide.

Since then, the number of faxes has stayed the same, but we rarely receive applications with missing signature pages. What surprised us is that the idea hadn't been tried earlier. A relatively minor change created substantial results.

What impact has your trial had on your agency? 

The trial has led us to think more creatively in our work and embrace change. Staff have pursued other situations and asked to complete BetaGov trials with their ideas. Whenever we discuss it or present the findings, it does lead to staff to think more critically and more solution-based. We have started some other RCTs; some have not led to what we thought they would, but that's okay. There is no right or wrong answer. I find that philosophy helpful; we like the openness of it. The biggest change is we now feel that a trial doesn't have to be long term, and the analysis of results so lengthy that the opportunity to improve has passed before a trial has been completed.

Anybody can pull off SCF in their facility. You just have to try it. It can't hurt, that's for sure.