Thirteen years after they began, researchers in a landmark study of asset building in Oklahoma are poised to receive new information gathered despite obstacles posed by COVID-19, which struck just as the new phase of research was getting underway.
The experiment, SEED for Oklahoma Kids (SEED OK), began in 2007 with interviews of over 2,700 mothers of Oklahoma newborns. Next, half of the newborns automatically received $1,000 in a state-owned Oklahoma 529 plan account as part of this long-term study of the benefits of Child Development Accounts. Early analyses show promising results, some especially for financially vulnerable families, and also provide lessons from the Great Recession for the current COVID crisis.
In early 2020, a third round of surveys—Wave 3—began in order to assess the impact of the accounts as the children in the study turned age 12.
“As far as we know, SEED OK is the longest running social experiment in the country,” said Michael Sherraden, Director of the Center for Social Development (CSD) at Washington University in St. Louis, which is leading the study. “Research methods are robust, and results are informing policy innovations.” Deposits to the state 529 plan were funded by the Ford Foundation and SEED OK is administered by the Oklahoma State Treasurer’s office. As the study progresses, survey data from families and account and savings data from the Oklahoma 529 plan provide key information on outcomes, such as the growth of the accounts; the social development of children; and the mental health, parenting practices and educational expectations of mothers.
But the advent of COVID-19 made traditional information-gathering methods, such as call-center interviewing and in-person contacting, impossible, so major adjustments were required. An on-line data collection system was created, and the research timeframe was extended, with a continued focus on high-quality data and a high response rate.
“The very good news is that we are now on track to receive the SEED OK Wave 3 data set by mid-August,” said Margaret Clancy, CSD’s Policy Director who has been leading the study.
“The response rate will be over 66% of the initial sample, which is very respectable after 13 years. CSD has reviewed preliminary data files and we are well prepared to begin analyzing the data.”
A benefit of the delay is that CSD researchers will be able to test the impact of the virus, with survey data both before and after its arrival. CSD researchers also added some new survey questions about COVID-19 for study participants responding after its onset.
The success was achieved through intensive planning with RTI, the firm CSD contracted with to collect the data, and by adopting the following changes:
- Creating a virtual call center. CSD originally contracted with RTI to perform all surveys by telephone with interviewers operating in a supervised call center. Due to the virus, RTI closed its call center and transitioned to a virtual call center model, with interviewers working from their homes. The virtual call center allowed interviewers and supervisors to access the same systems and achieve the same quality controls as before.
- Cancelling in-person field visits. RTI cancelled in-person field visits due to concerns about the health of the workers and social distancing requirements. Those visits, to take place for 8 weeks, were designed to secure telephone interviews with participants who could not previously be reached.
- Creating a web-based survey. CSD worked closely with RTI to revise the telephone survey, update programming specifications, review procedures and implement a new, web-based survey. CSD crafted and reviewed new communications with participants (letters, postcards, email and texts with messages) about the revised web survey invitation.
- Increasing financial incentives for respondents. Greater incentives were offered much earlier in the data-collection process. Due to the cancellation of field work, incentives were increased for participants who completed telephone or web-based surveys. In the final weeks, those incentives were increased again.
- Adding a second, preliminary data set transfer. With adjustments for the virus, CSD received another data file after RTI completed the first 50 web surveys. This enabled CSD to check if interviewer-led telephone survey responses differed from the web mode and for any undetected programming issues.
“We know from long experience that achieving high-quality data positions us to inform policy innovations going forward,” Clancy said. “We believe the SEED OK Wave 3 data will help us achieve this goal.”