XPRIZE: Back to the Future

   GAIL SPANGENBERG – President, National Council for Adult Learning  —- The recently announced Barbara Bush Foundation XPRIZE for Adult Literacy has been met with a mixture of reactions and NCAL has heard from many colleagues in Adult Education about it.  Nearly all are pleased and hopeful about the initiative.  To succeed, however, the initiative will have to be well informed in a number of substantive areas –– e.g., different instructional design(s) for both students and facilitators… appropriate instructional design(s) for different kinds of low-level learners…various assessment and ongoing evaluation tools, in addition to what CASAS will provide…effective strategies for student recruitment and retention…findings from research and practice (so that the effort shows awareness of what others are doing as well as the new context for adult education and workforce skills development reflected in current state and federal policy)…strategies for partnering and collaboration…and perhaps a few other areas.

To help provide constructive input to XPRIZE thinkers and doers, and to give Adult Education planners and practitioners a forum in which to express their views, NCAL invited a small group of state and national leaders to contribute to this Blog by speaking to their hopes and concerns in the above substantive areas or another area of their choosing. They were also encouraged to offer any specific advice they might want to give. We are pleased to share their perspectives here.

Leadership groups in Adult Education and Workforce Skills, especially those involved in the reauthorization of WIA and implementation of WIOA, share a common vision of the future for low-skilled adults. It includes the deployment of technology in all its forms.  It also includes a dedication to serving the lowest skilled adults, though we know from experience how hard that can be. The target populations for the XPRIZE are a small but highly important part of the spectrum of adults in need of services.  It is in everyone’s best interest for the program to succeed.  If it does, it could potentially help move many of our fellow citizens from the lowest skills levels toward higher level skills acquisition and paths to further education and jobs.

NCAL hopes the observations and advice offered below will be helpful to the XPRIZE team and others as they plan, take part in, and carry out their bold new program.

[Note: For an overview of the XPRIZE program, please see NCAL’s E-News issue of June 18 at http://www.ncalamerica.org/NCALENews3June182015.pdf and/or http://adultliteracy.XPRIZE.org].

~ ~ ~ ~ ~

   DIANNA BAYCICH – Literacy Projects Coordinator, Ohio Literacy Resource Center, Kent State University  —- The XPRIZE is a great idea, but many things need to be understood and worked out if this is to succeed.  For example, I am concerned that someone with such low literacy skills does not or cannot own a smart phone, or for that matter may be unable to take a computer-based assessment. In addition, I hope the software programs that XPRIZE will develop will address all the components of reading.  I also have two cautions. One is that the word “illiterate” has fallen out of use for good reason and I hope XPRIZE will avoid using it. Another is that high test scores will not necessarily be proof of high quality software but may be due to other factors that their program will need to control for. Finally, some people with whom I’ve spoken are concerned about the lack of upfront money that many knowledgable software design competitors would need to be able to participate.

   FORREST CHISMAN – Former Vice President, Council for Advancement of Adult Literacy, and former President of the Southport Institute for Policy Analysis —- Here are a few quick thoughts: (1) A large part of the non-ESL population at this low skills level almost certainly has serious learning disabilities and they are unlikely to be addressed by a generic mobile application. (2) A “third grade reading” level and “basic literacy” are concepts on two different scales. (3) We know from years of experience that the problems of low skilled ESL students are almost wholly different from those of native speakers and can’t be addressed by the same tools. 4) It seems unrealistic to expect “functional illiterates,” a term that no longer has meaning, to download an app they can’t read, and I wonder if these adults even own or can afford a smart phone. 5) I am skeptical about successfully finding pilot test groups of 1000!

   JOHN COMINGS – Senior Technical Consultant at World Education and Adjunct Faculty at the Center for International Education, University of Massachusetts (Amherst) —- NCSALL’s 1997 Longitudinal Study of Adult Learners (LSAL) found that adults use episodes of program participation and self-study to reach their learning goals. XPRIZE could spur the development of instructional tools that support self-study. Findings from the 2011 PIAAC assessment found that as many as 11.5 million Below Level 1 adults in the U.S. could benefit from the XPRIZE ELL app and up to 23 million Level 1 adults could benefit from the ELS app.

My concern is that the field-test approach proposed by XPRIZE may not produce a measurable gain or at best only a small gain, which could cause app developers to give up on these two populations. LSAL found that episodes of program participation and self-study had a significant impact on scores from a test similar to PIAAC, but skill development was slow and steady over many years. In addition, the CASAS test could under-report learning gains, since it is not focused on measuring the gains made by using the specific apps.

To address this concern, the field test could make these three changes:

1.  Collect data from four samples of adults from each of the two XPRIZE target groups (ELL and ELS). The four samples would be adults who are (1) NOT in programs with access to the apps, (2) NOT in programs without access to the apps, (3) IN programs with access to the apps, and (4) IN programs without access to the apps. This would provide data that might show whether or not access to a program enhances the use of the apps. Right now, the app evaluation is set up with a control group and a treatment group, both of which are NOT in programs. I’m suggesting that there be a second control group and second treatment group for both ELL and ELS that are IN programs.

2.  Along with a CASAS pre-test and post-test, all of the samples from both groups could be assessed with tests that measure the components of reading (real word reading and oral reading fluency, for example, might be good choices) and the specific skills taught by the apps (a vocabulary test or a comprehension test using the vocabulary employed by the app, for example). These two additional tests would ensure that learning gains that result from the use of the apps can be captured, even if the impact on the CASAS score were small.

3.  Since the field test would be of short duration, the possibility that learning gains would grow over time should be part of the final report, and XPRIZE or the winning app group should extend the data collection to assess the learning of all the sample groups 12 to 18 months after the initial assessment.

   MARY ANN CORLEY — Recently retired Principal Researcher and Director of Teaching Excellence in Adult Literacy (TEAL) and founding director of the California Adult Literacy Professional Development Project (CALPRO), American Institutes for Research; former director of the National Adult Literacy and Learning Disabilities Center; currently Principal Associate, Adult Learning Partners —- Adult literacy programs, whose mission is to serve the entire range of literacy learners, often find that they attract predominantly higher level learners, rather than those “most in need.” Reasons include the obvious challenges of identifying and recruiting non- or low-level readers, the smaller numbers of low-level readers in the general population, and the programs’ ability to more readily demonstrate gains in functioning levels for higher level learners over one year’s time. It is commendable, then, that the XPRIZE program is focused specifically on adult learners at the lowest literacy levels.

The proposed use of technology (mobile devices) will eliminate some of the barriers that learners face in accessing traditional brick-and-mortar literacy programs, e.g., childcare, transportation, and non-negotiable meeting times and places. However, we know that the lowest level learners are unlikely to have or to use technology. XPRIZE competitors will want to include in their planning strategies for getting the devices into learners’ hands and helping them become comfortable with their use.

Recruiting learners could pose yet another problem: How will these learners be identified and recruited from among the general population? What incentives would there be to encourage them to participate? What supports will be in place to help participants with the technology?

More significant, perhaps, is the issue of the instructional design of the software programs. One or more literacy specialists should be part of the design teams to offer advice on content and implementation questions such as the following:

1.  Are all the components of reading included in the design, not just alphabetics (e.g., phonics and word attack skills)? Other critical skill areas include fluency, vocabulary, and comprehension, as well as the all-important integration of reading and writing skills.

2.  Does the software incorporate principles of universal design for learning (UDL) to ensure that the content is accessible to adults with disabilities (both physical and learning disabilities) and includes multiple options for accessing it, such as text-to-speech options, alternate font sizes and styles and colors, etc.?

3.  To prevent or reduce the possibility of drop-outs/non-completers, will supports be put in place such as monitoring, tutoring, and counseling to help learners when they “get stuck”? For this program to be effective, learners need to meet with success every step of the way.

4.  Is there a way to incorporate a hybrid design (face-to-face plus online learning) that includes a collaborative approach? For example, a small group of students working together in an online setting and meeting every so often with an instructor might keep students motivated to continue with the program.

5.  Finally, in the second phase of the XPRIZE program, can measures be developed to ensure that the reports of downloads are not inflated and that downloads are indeed performed by adults who comprise the lowest literacy levels?

   BRENDA DANN-MESSIER – Education and Workforce Consultant & former Assistant Secretary for Office of Career, Technical, and Adult Education, U.S. Department of Education —- I applaud the Barbara Bush Foundation, and their XPRIZE challenge, for helping to advance the field of adult education and more importantly, for focusing on individuals with the lowest literacy and language levels. This is a group of adult learners often overlooked and underserved. It takes significant resources, time, and intensive case management support for these students to complete their educational journey and improve their economic prospects.

The Survey of Adult Skills released by OECD in 2013 reports that our nation has large numbers of low-skilled adults.  Unfortunately, the current system’s capacity to serve millions of adults is severely limited.  That is why the challenge put forth by the Foundation could potentially be a game changer, allowing us to more effectively and efficiently serve thousands more low-skilled adults.  This is just the type of innovation our field needs.

Those of us who have worked with adults with very low literacy, numeracy, and language skills know there are no quick fixes or solutions.  Students with limited skills need programs and services of greater intensity and duration. Seeing results take time, and organizations find it difficult without additional funding and staffing to serve these students. But the hardest to serve have been left behind for too long. The X PRIZE challenge will spur innovative and effective strategies to accelerate student progress, allow adults to learn 24-7 and develop products that could open up new opportunities for low skilled adults to move along a career pathway.  Effective programs will lead to self-sufficient adults able to support themselves and their families.

I know many policymakers and practitioners who are committed to serving adult learners with multiple barriers.  They are ready to work with the Barbara Bush Foundation to help design, promote, and eventually implement the XPRIZE program. The field and our students will benefit from this exciting new opportunity.  Thank you to the Barbara Bush Foundation for its new commitment to adult learners and for expanding educational opportunities by developing and implementing this challenge.

   JOHN FLEISCHMAN — Director, USA Learns; Director and Manager of Technology Services, Sacramento (CA) County Office of Education, including the Outreach and Technical Assistance Network (OTAN) and Cybersteps, a national distance learning project; Columnist, Ed Tech Digest —- I am absolutely jazzed about the Adult Literacy XPRIZE competition and offer sincere appreciation to the Barbara Bush and Dollar General Literacy Foundations. However, I have some anxiety about the challenges and efficacy of delivering comprehensive literacy and language instruction via smart phones and phablets as explained below.

A recent Tyton Partners report, Learning for Life: The Opportunity for Technology to Transform Adult Education, indicates that between “55% to 75% of 4.1 million adult education students in programs today own smart phones.” An adult learner technology survey by California’s Outreach and Technical Assistance Network, representing over 34,000 enrolled students, has found that 75% of adult learners actually have a smart phone. Clearly, this represents a huge opportunity for delivering instructional content beyond traditional centers of adult skills learning.

My concerns about using mobile devices as a primary delivery system have to do with screen size, cost for wireless data, and learner persistence:

1.  As an instructional designer (creator of USA Learns), I know that creating comprehensive content for 4” or even 6” screens presents significant challenges, especially because learners can view only limited amounts of text or multimedia on a single screen. It will be very important for the XPRIZE teams to strategically “chunk” the learning material.

2.  Many adult learners at the lower end of the socio-economic spectrum don’t have unlimited data plans for their smart phones.  If they don’t have access to a Wi-Fi connection, and are using their own mobile device, they will certainly want to keep personal costs to a minimum.

3.  How many times and for how long do you look at your smart phone each day? From following users of USA Learns, we have validated that most people tend to use smart phones in small increments of time.  Thus, developers will need to consider mobile usability heuristics to be sure they are providing a positive learning experience that increases persistence for learning.

These are large challenges, so I believe it is important for instruction with phablets and smart phones to be coupled with more traditional approaches. The last thing we need is for this population to fail again.  Instead of attempting to create an entire course for a small screen, I would urge the XPRIZE teams to consider m-Learning courses that can be used in a blended fashion.  For example, components of the learning could be delivered before or after face-to-face instruction, or the m-Learning component could be used to reinforce what is being taught in the classroom.

   GARRETT MURPHY — Senior Advisor to National Council for Adult Learning and its predecessor Council for Advancement of Adult Literacy; national policy analyst; former NYS ABE Director —- When an old-timer has been in the field for as long as I have, over 50 years, though I have not been particularly active for the last seven, it is certainly appropriate for that person to recognize his limitations and realize that things may be happening in the field with which he is unfamiliar.  In spite of this caution, when I was invited to offer my thoughts about programming for adults falling into the zero to four test level, I jumped at the chance, for these reasons:

(a) My local adult education program in Albany, New York had a large enrollment in 0-4 for the entire fifteen (15) years that I was in charge.  (b) When I took over the reins of the State program in 1980, Paul Delker and his federal staff insisted that we set aside adequate funding for the “least educated and most in need” which translated as 0-4.
(c) After I left the Albany program my successor hired my wife to continue work at the
0-4 level, so I could begin this piece by interviewing her.

I’m glad for the opportunity to make a few observations and recommendations.

1.  At the lowest skill levels there will be many different types of potential learners – from angry bitter adults to eager, enthusiastic ones.  Efforts that aim to serve them should keep in mind that many have impressive accomplishments despite their skills handicaps–such as driving across the country despite the inability to read road signs, or marching a family across half a continent to reach a refugee camp, or being able to recite whole sections of the Koran.  They deserve our respect.

2.  Adults at this level tend to need assistance from, or are subject to procedures of, a number of government and non-governmental agencies.  A strong counseling program will be needed.  Moreover, counselors who work with clients attending night classes must have paid time during the day to deal with agencies that are only operational during the day.

3.  Special care must be taken to use the correct level test to measure gain.  Some provider agencies give everyone the test designed for mid-range students, but it may be necessary to develop a locator test or process to assist providers in making the correct choice of entry or progress testing.

4.  In circumstances where several adults test at different levels within the 0-4 category, it will be very difficult to adhere to the WIOA mandate with respect to integrating workforce training.  One way to address that difficulty is to offer discussion of “integration with workforce preparation” in a group or class context.

   JAMES PARKER – Senior Vice President, National Council for Adult Learning; formerly with Division of Adult Education & Literacy, U.S. Department of Education —-  Barbara Bush has said: “I am inspired and hopeful that this Adult Literacy XPRIZE will bring about a radical breakthrough to end the cycle of low literacy in America.”  Many in the Adult Education field had hoped that the reauthorization of WIA (subsequently WIOA) might have had that radical effect also. However, high school graduation or college readiness, after one year of instruction, is not a realistic goal for most lower-skilled adults. The XPRIZE Project has the potential to make an important contribution to adult education R&D and program improvement. This type of project – funded by private organizations – is especially needed since Congress decided not to include in WIOA the provisions of the Adult Education and Economic Growth Act for a comprehensive technology program and an adult education research center. [Note: See https://www.congress.gov/bill/113th-congress/senate-bill/1400/text]

OCTAE and LINCS have recently explored the use of digital badges for adult education, and the Department of Education has gone further in testing the use of competency-based measures in higher education programs. It would be forward-thinking for the XPRIZE Program to also include the alternative of digital badges or other competency-based measures in the design. This is a “natural” since the project would design learning-to-read software for mobile learning devices. And because students involved in field testing may not be involved in any other literacy programs for the duration of their involvement in the XPRIZE initiative, the sanctions that NRS applies to WIOA Title II programs would not be in play here. Moreover, CASAS could be of assistance in designing CBE options, having pioneered Competency-Based Adult Education since the 1980s. Their Employability Competency and Workforce Skills Certification Systems are examples of high-quality competency-based measures.

Can XPRIZE help Adult Education get back to its roots, where grade levels don’t matter, but skills and competence do? Stay tuned.

   DAVID ROSEN – President, Newsome Associates —- I am enthusiastic about the Adult Literacy XPRIZE: the literacy app solutions could be useful to adults who want to acquire or improve reading skills. The attention the prize may give to the need for adult literacy instruction could raise public awareness and increase policy makers’ interest in addressing the need with increased program funding. And, focusing developers’ attention on adult literacy could result in the future in creative and engaging software programs that could meet adult basic education needs of other kinds and at other levels, and that could be useful to both learners and programs.

However, I am also concerned about several things that could easily be fixed. I agree that a focus on basic literacy, defined as an initial score on the CASAS test of 210 or below, is very important. But there are a great many more adults whose skills are at an intermediate level, roughly grades 3-8, represented by a score on the CASAS test of 211 to 235. Extending the design specifications through this intermediate level could produce apps that would reach a larger number of adults, many more of whom would already have smart phones, which is a requirement of participating in the field test.

Further, the prize guidelines make clear that field test participants cannot be enrolled in another adult literacy program. However, this should not preclude recruiting field test participants from those on waiting lists for these programs, which could be a great help to the organization hired to recruit the participants. If the XPRIZE Foundation sees the waiting lists as an opportunity for participant recruitment, it would be important to make that clear now, and to build relationships with national adult literacy education organizations that could be helpful in identifying programs with waiting lists, and encouraging programs’ help in letting those on waiting lists know about the opportunity to participate in the field test.

Recruiting 5,000 field test participants, especially native speakers of English at the lowest reading levels, will be a difficult challenge, for which a good deal of planning will be needed. It will also be difficult to keep field test participants engaged in the learning for twelve months without any face-to-face instruction or support, but that’s why the XPRIZE foundation says its prizes push the limits of what’s possible!

   REECIE STAGNOLIA – Vice President for Adult Education, Kentucky Council on Postsecondary Education —-  Adult education leaders should champion any organization or foundation effort to support the challenges of adult literacy. I heartily applaud the steadfast support provided to adult education by both the Barbara Bush Foundation and the Dollar General Literacy Foundation. These foundations have consistently provided financial, awareness, and advocacy support to our field. We have accomplished a good deal over the years as a result, although much work remains. The PIAAC findings have certainly illuminated the literacy and numeracy deficiencies in the United States, particularly as we strive to compete in the global economy. The sobering reality is that improving our nation’s educational attainment level is a matter of economic and national security.

I have two suggestions to offer relative to the XPRIZE Initiative.

First, states are quite diverse in the selection of assessment instruments to pre-test and post-test our students. So XPRIZE should give consideration to developing the mobile APP to support ALL approved National Reporting assessments of the U.S. Department of Education’s Office of Career Technical & Adult Education’s (OCTAE), not only CASAS.

Second, focusing only on those individuals reading below a 3^rd grade level seems too narrow in scope. Adult educators need access to tools that will support students at all levels, especially since data indicates most enroll above the 3^rd grade level.

For example, in Kentucky, nearly 40% of our students enroll at a functioning level below the 6^th grade. As a result we are committed to serving those who are most educationally and economically disadvantaged.

Some analysts suggest that the new Workforce Innovation & Opportunity Act could be interpreted as having a focus on students only at the higher levels, but others say that WIOA intends to include adults at all proficiency levels.   But nearly everyone agrees on the high importance to our nation of serving the lowest skilled adults. We must continue to reach back and serve those at the lowest levels. I’m excited by the potential of the XPRIZE to help us do that.

    JOHAN UVIN — Acting Assistant Secretary,  Office of Career, Technical, & Adult Education, U.S. Department of Education —- It does not happen very often that the challenges that adult learners with poor foundation skills face are the center of attention. Since the release of the results of PIAAC (also known as the Survey of Adult Skills), that has changed.  The data were clear: 36 million adults in America face considerable literacy challenges – 24 million of whom are working one or more jobs – and 48 million American adults struggle with basic math. The release of Vice President Biden’s Ready to Work report last summer, the simultaneous enactment of the Workforce Innovation and Opportunity Act, the launch of the UpSkill America initiative, the establishment of an administration-wide Skills Work Group that focuses on career pathways, upskilling, apprenticeship, and other key areas of work to address our skills challenges have moved this issue of foundational skills from the margins into the center of public policy making in our country.

It happens even less often that the needs of those at the lowest rungs of the skills distribution are the center of an effort. That is why the Barbara Bush Foundation’s XPRIZE for Adult Literacy is significant. The prize focuses on a subpopulation for which good solutions are far and few between: adults with skills at or below the third-grade equivalent. By leveraging the best and creative minds across fields and sectors, I am confident that solutions can be created moving forward. This challenge is only the beginning.

The prize appropriately challenges the market place to leverage the best tech innovators, adult learning researchers, and practitioners to create solutions to get these lower skilled individuals to a level of reading that can allow them to get ahead. This focus and approach have never been used. I believe the XPRIZE for Adult Literacy is the beginning of a different quest for solutions. We all know there are no or only partial proven solutions for this population today.

By leveraging the best and creative minds across fields and sectors, I am confident that solutions can be created. Adult learning professionals will play an important role in this effort but it is the cross-sectoral partnership between tech innovators, researchers, practitioners, and the adult learners themselves that has the potential to create breakthrough solutions for the eight million adults who scored Below Level 1 on the Survey of Adult Skills and the additional eight million who did not provide a literacy-related response. The XPRIZE creates the incentives to “invent” a new and overdue solution. We owe it to these millions of fellow Americans to find one. They have no time to waste.

   HEIDE SPRUCK WRIGLEY — Founder and Director, Literacywork International —- The X Prize calls for the development of software to increase the literacy skills of both native speakers of English and speakers of other languages whose English is still a work in progress. Developing materials and creating learning experiences that meet the needs of both groups will be an exciting challenge.

A number of factors that distinguish English learners from native speakers need to be taken into account if both groups are to increase their skills.

1.  Adult English learners who have difficulty reading English tend to have a much broader range of educational backgrounds than adults who grew up speaking English. English learners who fall into the intermediate levels on the NRS could include any of the following: (a) Adults who never went to school or have only a primary education; (b) Adults who have professional certification or degrees; and (c ) Adults whose schooling was interrupted due to war or civil strife or because they moved from country to country.  Clearly, materials will need to be developed so that those with lower levels of education receive appropriate support and those with higher levels are challenged with materials that take advantage of their background knowledge.

2.  English learners face additional challenges that make them distinct from native speakers.  By definition, English speaking adults speak English.  So the meaning of the sentence “I’ve been fired” may be clear to a native speaker but not necessarily to someone who is still trying to sort out what “I’ve” might mean and where the fire might be.

3.  Trying  to read and understand even everyday materials can be a challenge for English language learners.  While native speakers can transfer their knowledge of English to print, non-native speakers may get stymied by an everyday expression such as booting up a computer, backing up a file, or wrapping up a session. These gaps can significantly affect their comprehension.

4.  Not having sufficient knowledge of the culture can impede reading comprehension as well. These include references to sports (scoring a touchdown), government structures (Supreme Court), or cultural references to schooling (lunch lady, prom, detention).

In short, English learners who face reading difficulties stand to benefit greatly from software that offers learning opportunities customized to their needs – as does the entire field of Adult English Language Acquisition.