For the environmental scan, we will conduct a review of recent published and unpublished studies, RSA monitoring reports, and agencies’ annual stage plans. State plans include data on the agency’s involvement with state departments of education and local schools, programs specific to youth with disabilities, and transition staffing. Since 2011, the monitoring reports have included a section on transition service and employment outcomes for youth, based on RSA visits to VR agencies that occur every six years to assess performance and collect information on areas of special interest. From this review, we will identify agencies with multiple programs for transition-age youth and select five to seven agencies for follow-up interviews. We will design the interviews to confirm the list of policies and programs that we developed as part of the environmental scan and ask about evidence the agency may collect on program efficacy, its current use of data analytics, and its interest and ability to partner with us in the second stage.
For the DAS, as we proceed with the first stage, we will contact the agencies we have identified as having an array of services and supports for youth, as well as the information technology staff and software capability needed to support a DAS. We will choose one agency and negotiate an agreement under which our team will provide technical assistance to design and implement a DAS. Together we will (1) develop a low-cost system that the VR agency can use with minimal external technical assistance, and (2) demonstrate that this system works well and has high value relative to its cost.
We will use two data analytic tools: (1) predictive analysis (PA), and (2) rapid-cycle evaluation (RCE). PA refers to statistical models to predict outcomes for individuals based on information available. VR outcome examples include measures of client engagement, cooperation, and follow-up; employment outcomes; and resource use and expenditures. Predictors would come from the VR client’s application and any information from the current engagement to date. Predictions of the models are used to target resources more efficiently to current clients. For example, if a model’s prediction indicates that a client is likely to not stay engaged, the client’s counselor can take steps to ensure continued engagement—steps that would waste effort if used for those very likely to stay engaged on their own. RCE refers to modest, rapid experiments, usually involving promising but unproven innovations in processes or services, most often designed to address specific challenges. We will work with the agency to identify a known challenge (for example, lack of engagement after applying for VR services), develop one or more innovations to address the challenge, then test the efficacy of the solution by implementing it for targeted clients.
Read less >