eScholar manages a lot of data! We manage billions of educational experience records for millions of individuals from birth through higher education. Several years ago while working with one of our large educational agency partners, our data science team set out to answer a seemingly simple question, “Is it possible to analyze the anonymous records of students that have previously achieved a goal and identify pathways to show similarly situated students, the “experiences” that correlate most strongly with that success?” Simple, right?
The work was remarkably challenging. It pushed our data science work to new limits and required that we grapple with much more than just statistics and regressions. It forced us to incorporate artificial intelligence and econometric principles as well. But the struggle and effort was well worth it—we have developed the data science based system to identify and measure the likelihood of success for the future goal based on the current trajectory. Ultimately and most importantly, we can show and quantify future experiences and pathways that a particular student could take that correlate to his/her increased likelihood of success. The objective of this system is to offer students information about where they stand versus their goals and it is information they can use immediately to guide decisions about future actions.
Another key benefit of the eScholar data science project, “Project Enterprise” is its ability to look at the current standing of an individual and compare completed experiences to different future goals those experiences support. Doing this allows students to explore possible future careers and study goals. For example, career goals may not be obvious to them at the moment, but correlate well to the experiences they have already mastered. This capability will offer students career options that they might not have been aware of previously and help them chose a pathway to successfully reaching that career.
For many years now, Texas had been one of eScholar’s key educational agency partners. Recently, we’ve embarked on a new collaboration together—working on our data science initiative together under the University of Texas Educational Research Center (ERC). After producing promising results in another state, Texas was intrigued and wanted to determine if our approach yielded valid, actionable results in Texas. To determine this, the Education Research Center has provided eScholar with access to nearly 1 billion anonymized educational experience records for tens of millions of individuals spanning pre-kindergarten through post-secondary and covering a 20-year period. This is profoundly big data. The sheer size and diversity of the data available for this study will allow us to measure the efficacy of our system to an unprecedented degree. For example, the fundamental nature of the system means that accuracy should increase as the specificity of goals, such as majors’ increases. In our earlier work with only millions of records, we often did not have a wide variety of majors that also had sufficient membership to produce valid results. This dynamic will be dramatically improved in Texas.
We are very excited about this work because it is giving voice to the data we have all been working so hard to collect and manage. With the eScholar data science initiative, we are literally using data to help students succeed. What could be better?
