A skilled workforce is critical in high-quality out-of-school time (OST) programs (Smith, Devaney, Akiva & Sugar, 2009). However, the workshops commonly used to train OST staff are not adequately preparing practitioners to deliver quality programs that can benefit youth(Durlak & Weissberg, 2007; Smith et al, 2009).
DATE:
TEAM MEMBERS:
Femi VanceEmily SalvaterraJocelyn Atkins MichelsenCorey Newhouse
According to the Harvard Family Research Project (2010), schools need collaborative partners to help children and youth thrive. For over a decade, afterschool programs have been positioning themselves as viable partners. After all, afterschool programs challenge students’ thinking, teach collaboration, and help children and youth find their passion.
As part of an overall strategy to enhance learning within informal environments, the Innovations at the Nexus of Food, Energy, and Water Systems (INFEWS) and Advancing Informal STEM Learning (AISL) programs partnered to support innovative models poised to catalyze well-integrated interdisciplinary research and development efforts within informal contexts that transform scientific understanding of the food, energy, and water systems (FEWS) nexus in order to improve system function and management, address system stress, increase resilience, and ensure sustainability. This project addresses this aim by using systems thinking and interdisciplinary integration approaches to develop a novel immersive educational simulation game and associated materials designed to highlight the role and importance of corn-water-ethanol-beef (CWEB) systems in supporting the ever increasing demands for food, energy, and water in the United States. The focus on FEWS and sustainable energy aligns well with both the INFEWS program and the sizable sustainability-related projects in the AISL program portfolio. The development and broad dissemination of a multiuser game specific to CWEB systems are particularly innovative contributions and advance for both program portfolios and their requisite fields of study. An additional unique feature of the game is the embedding of varying degrees of economic principles and decision-making along with the nuisances of cultural context as salient variables that influence systems thinking. Of note, a team of computer science, management and engineering undergraduate students at the University of Nebraska - Lincoln will be responsible for the engineering, development, and deployment of the game as their university capstone projects. If successful, this game will have a significant reach and impact on youth in informal programs (i.e., 4-H clubs), high school teachers and students in agriculture vocational education courses, college students, and the public. The impact could extend well beyond Nebraska and the targeted Midwestern region. In conjunction with the game development, mixed-methods formative and summative evaluations will be conducted by an external evaluator. The formative evaluation of the game will focus on usability testing, interest and engagement with a select sample of youth at local 4-H clubs and youth day camps. Data will be collected from embedded in-game survey questionnaires, rating scales, observations and focus groups conducted with evaluation sample. These data and feedback will be used to inform the design and refinement of the game. The summative evaluation will focus on the overall impacts of the game. Changes in agricultural systems knowledge, attitudes toward agricultural systems, interest in pursuing careers in agricultural systems, and decision making will be aligned with the Nebraska State Science Standards and tracked using the National Agricultural Literacy Outcomes (NALOs) assessment, game analytics and pre/post-test measures administered to the evaluation study sample pre/post exposure to the game.
DATE:
-
TEAM MEMBERS:
Jeyamkondan SubbiahEric ThompsonDeepak KeshwaniRichard KoelschDavid Rosenbaum
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at the University of Colorado. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at DePaul University. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
Summary
Girlstart’s mission is to increase girls’ interest and engagement in STEM through innovative, nationally-recognized informal STEM education programs. Girlstart examines in this report how STEM education directed toward elementary school girls influences long-term readiness and participation to math and science learning. This report compares Girlstart After School participants’ academic performance to nonparticipant performance. Specifically, it examines how Girlstart After School influences science STAAR performance and course enrollment in subsequent elementary and early middle
This project by California State University San Marcos and their collaborators will expand and continue to innovate on a pilot Mobile Making program with the goal of developing a sustainable, regional model for serving underserved, middle-school aged youth in twelve after-school programs in the San Diego region. Evaluation of the current Mobile Making program has documented positive impacts on participants' interest, self-efficacy, and perception of the relevance of Making/STEM in everyday life, and led to a model for engaging underserved youth in Making. The work will focus on implementing the program model sustainably at greater capacity by increasing the number of undergraduate activity leaders, after-school sites, and level of community engagement. The expanded Mobile Making program is expected to engage ~1800 middle school youth at 12 local school sites, with activities facilitated by ~1020 undergraduate CSU-SM STEM majors. The sites are in ethnically diverse and economically disadvantaged neighborhoods, with as many as 90% of students at some sites qualifying for free or reduced price lunch. The undergraduate facilitators are drawn from CSU-SM's diverse student body, which includes 44% underrepresented minorities. Outcomes are expected to include increases in the youth participants' interest, self-efficacy, and perception of the relevance of Making/STEM in everyday life. Positive impacts on the undergraduate facilitators will include broadened technical skills, increased leadership and 21st century skills, and increased lifelong interest in STEM outreach/informal science education. The program is designed to achieve sustainability through innovative means such as involving undergraduate facilitators via Community Service Learning (rather than paid positions), and increased community engagement via development and support of a community of practice including local after-school providers, teachers, Makers, and University members. Evaluation of the program outcomes and lessons learned are expected to result in a comprehensive model for a sustainable, university-based after-school Making program with regional impact in underserved communities. Dissemination to other regions will be leveraged via CSU-SM's membership in the California State University (CSU) system, yielding a potential statewide impact. The support of the CSU Chancellor's Office and input from a CSU implementation group will ensure the applicability of the model to other regional university settings, identify common structural barriers and solutions, and increase the probability of secondary implementations. This work is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.
Researchers at Georgia Tech and staff at the Center for Puppetry Arts in Atlanta will address the following problem: The growth of computing calls for increased teaching of basic hardware skills, but engaging students beyond those who are already interested in STEM fields is difficult. Emphasizing cultural context through personal expression has shown to attract new audiences such as women and minorities; however, balancing accessibility and educational effectiveness remains problematic. Current educational tools often use "black boxing" to simplify access, to protect the underlying functionality, and/or to stay commercially competitive. Innovations are needed where the basic skills of design and making, along with new educational models, can support students' exploration of everyday electronics and critical thinking. The team will test an after-school model that could be applied in cultural institutions and other venues that would provide students in 4 - 8 grades with multiple computer/electronics-driven puppet design and building workshops. These workshops will provide opportunities for basic hardware and electronics prototyping, personal cultural expression, and performance.
The exploratory research and development method involves three steps: the students develop a story-line (narrative) that will incorporate puppets; they learn and apply basic electronics and mechanical crafts to design, build and test their puppets; and they perform their shows to other participants and their families. Over the course of the project, there will be five workshops: two pilot tests (to test feasibility and design), one mid-project evaluation workshop (to evaluate and re-iterate), and two final workshops conducted by educators, where the research team only observes (to test for transferability). Data collection throughout this development cycle will include video and sound recording, still photography, field notes, pre- and post-questionnaires, and the production of puppet "artifacts" during the workshops. In addition to the data collected in the workshops, the project will produce design documents and educational guidelines, as well as other media (such as instructional video clips), which will be disseminated to educators and researchers.
The project has been successfully concluded. It included additional pilot workshops with puppetry experts and educators in addition to the scheduled development and final evaluation workshops. Overall, the project has led to publications in both educational as well as performance/ game- and interface-related events.
The project also published material on the puppet designs and their possible adaptation to educational settings on the project web site, which also include video documentation. More information is available at: http://dwig.lmc.gatech.edu/projects/prototypingpuppets/
As a part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds research and innovative resources for use in a variety of settings. This Broad Implementation project would scale up the CryptoClub Project, an afterschool and online program designed to engage middle school youth in mathematics and cryptography. The project builds on previous successful work and evaluation that is ready for scale up using a train-the-trainer model implemented through a partnership with the National Girls Collaborative. The project will train 160 new CryptoClub leaders who will then train 800 new leaders at 20 hub sites reaching 9600 students. In addition, professional development modules and webinars will continue to refresh leader skills. Other project components include an online multiplayer cryptography game, weekly challenges through social media, and digital cryptology badges for students.
The research uses a think-aloud method with students as they actually attempt to solve the cryptology problems using mathematical thinking. Three think-aloud studies will be performed during the Project. The research team will code transcripts of the interviews for evidence of the mathematical thinking intended to be addressed by each activity, as well as capturing unexpected kinds of thinking. Tasks will also be rated according to the type of knowledge elicited. A written report will include statistical analyses of the think-aloud and interview responses, interpreted in light of the overall CryptoClub goals. The findings will contribute to both future research efforts and practice. The evaluation by EDC uses a quasi-experimental design, which assesses project outcomes for trainers, leaders, students, and Internet users. EDC will also investigate the fidelity to the CryptoClub model as it is scaled up. These studies have strong potential for informing numerous other projects that are at a stage where scale up is under consideration.
Youth environmental education (EE) programs often serve as gateway experiences in which diverse audiences engage in informal science learning. While there is evidence that these programs can have positive impacts on participants, little empirical research has been conducted to determine what makes one program more successful than another. To be able to conduct such research, this Exploratory Pathways study will (1) develop and statistically validate ways to measure meaningful outcomes for participants across a variety of programs and (2) test observational methods that will enable research that can determine which elements of program delivery most powerfully influence participant engagement and learning outcomes in different contexts. These efforts will include consultations with diverse subject matter experts from the National Park Service, nature centers, and academia; survey research with participants in afterschool and free-choice EE programs; and observations of EE programs designed to fine tune the measurement of program delivery elements and student engagement. Developing valid and reliable outcomes measures and observational protocols will enable a larger investigation that will specifically address the following research question: What program characteristics lead to the best learning outcomes for program participants in different contexts? This research will result in empirically tested guidelines that will enable educators to design and deliver more effective programs for a wide range of audiences in a wide range of contexts. It is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This effort will refine methods necessary to undertake an unprecedented study (and future AISL Research in Service to Practice proposal) to examine the linkages between pedagogical approaches, participant engagement, and learning outcomes in informal STEM-focused youth EE programs. The larger study will involve systematically observing a large number of programs to assess the use of different approaches and to link those approaches to engagement and learning outcomes through both observation and survey research. In this current study the team will develop and refine crosscutting outcome measures to ensure validity, reliability, and sensitivity by drawing upon the literature and consultation with key stakeholders to develop suites of indicators for subsequent psychometric testing and revision. They will also refine observational techniques for assessing pedagogical approaches through extensive testing of inter-rater reliability. Finally, techniques for measuring participant engagement, incorporating both observational techniques and retrospective participant surveys will be refined. The work will be conducted by researchers at Clemson University and Virginia Tech, in partnership with the U.S. National Park Service, the North American Association for Environmental Education, and the American Association of Nature Center Administrators. This work represents the first step in a longer research process to determine the "best practices" most responsible for achieving outcomes in a wide range of contexts.
In this chapter we present the ways in which institutional cultural differences impact the development and implementation of learning activities in informal settings. Five university-based centers for the study of chemistry worked with informal learning professionals to re-envision educational and public outreach activities about science. The projects were part of a broader effort to catalyze new thinking and innovation in informal education and chemistry centers. The set of projects illustrates the broad possibilities for informal learning settings, with projects targeting diverse audiences
As part of its overall strategy to enhance learning in informal environments, the Advancing Informal STEM Learning (AISL) program funds innovative resources for use in a variety of settings. This Research in Service to Practice project will examine how a wide range of pre-college out-of-school-time activities facilitate or hinder females' participation in STEM fields in terms of interest, identity, and career choices. The study will address the ongoing problem that, despite females' persistence to degree once declaring a major in college, initially fewer females than males choose a STEM career path. To uncover what these factors might be, this study will look at the extent to which college freshmen's pre-college involvement in informal activities (e.g., science clubs, internships, shadowing of STEM professionals, museum-going, engineering competitions, citizen science pursuits, summer camps, and hobbies) is associated with their career aspirations and avocational STEM interests and pursuits. While deep-seated factors, originating in culture and gender socialization, sometimes lower females' interest in STEM throughout schooling, this study will examine the degree to which out-of-school-time involvement ameliorates the subtle messages females encounter about women and science that can interfere with their aspiration to a STEM careers.
The Social Cognitive Career Theory will serve as the theoretical framework to connect the development of interest in STEM with students' later career choices. An epidemiological approach will be used to test a wide range of hypotheses garnered from a review of relevant literature, face-to-face or telephone interviews with stakeholders, and retrospective online surveys of students. These hypotheses, as well as questions about the students' demographic background and in-school experiences, will be incorporated into the main empirical instrument, which will be a comprehensive paper-and-pencil survey to be administered in classes, such as English Composition, that are compulsory for both students with STEM interests and those without by 6500 students entering 40 large and small institutions of higher learning. Data analysis will proceed from descriptive statistics, such as contingency tables and correlation matrices, to multiple regression and hierarchical modeling that will link out-of-school-time experiences to STEM interest, identity, and career aspirations. Factor analysis will be used to combine individual out-of-school activities into indices. Propensity score weighting will be used to estimate causal effects in cases where out-of-school-time activities may be confounded with other factors. The expected products will be scholarly publications and presentations. Results will be disseminated to out-of-school-time providers and stakeholders, educators, and educational researchers through appropriate-level journals and national meetings and conferences. In addition, the Public Affairs and Information Office of the Harvard-Smithsonian Center for Astrophysics will assist with communicating results through mainstream media. Press releases will be available through academic outlets and Op-Ed pieces for newspapers. The expected outcome will be research-based evidence about which types of out-of-school STEM experiences may be effective in increasing young females' STEM interests. This information will be crucial to educators, service providers, as well as policy makers who work toward broadening the participation of females in STEM.
DATE:
-
TEAM MEMBERS:
Roy GouldPhilip SadlerGerhard Sonnert