This funder reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It describes the importance of evaluation to funders, and provides some prompts and questions for thinking about evaluation with relation to funding.
This leadership reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It provides an organizational leadership perspective on evaluation.
This practitioner reflection was provided to stimulate conversation at the June 20-21, 2013 CAISE Evaluation Convening. It discusses improving practice through reflection.
Although major growth in engineering and computing jobs is expected in the next 10 years, students are not majoring in sufficient numbers to meet this demand. These impending workforce demands cannot be met without developing the skills of racial and ethnic minorities: however, Hispanics and Black/African Americans make up only a small percentage of doctoral students in the United States. The goal of the Consortium of Minority Doctoral Scholars (CMDS) Design and Development Launch Pilot is to broaden the participation of minorities in these fields. This pilot project will create a data portal that will allow the research team to study and understand the efficacy of various mentoring strategies that might be piloted across institutions and minority doctoral scholars programs.
Part II
The Consortium of Minority Doctoral Scholars (CMDS) will unite three of the nation's oldest and most prominent minority doctoral scholars programs (GEM, SREB and McKnight); organizations with a long history of impact in increasing the numbers of minorities obtaining advanced degrees. The CMDS Design and Development Launch Pilot will conduct extensive studies using data from these three programs. The research team will conduct a mixed method analysis of the data to discover commonalities and distinctions about the three programs' mentoring efforts as compared to students not involved in the three programs. This will result in a data-driven strategy for researching the efficacy of mentoring programs that can be applied across the three CMSD member and other minority doctoral scholars programs. By utilizing data from successful programs to pinpoint effective mentoring strategies, the project will create opportunities for larger numbers of minorities to be successful. This approach has implications not only with respect to equity and access, but also the development of a workforce that will drive future advances.
DATE:
-
TEAM MEMBERS:
Juan GilbertShaundra DailyJerlando Jackson
National Science Foundation (NSF) awarded an Informal Science Education (ISE) grant, since renamed Advancing Informal STEM Learning (AISL) to a group of institutions led by two of the University of California, Davis’s centers: the Tahoe Environmental Research Center (TERC) and the W.M. Keck Center for Active Visualization in Earth Sciences (KeckCAVES). Additional partner institutions were the ECHO Lake Aquarium and Science Center (ECHO), Lawrence Hall of Science (LHS) at the University of California, Berkeley, and Audience Viewpoints Consulting (AVC). The summative evaluation study was
The mixed methods randomized experimental study assessed a model of engagement and education that examined the contribution of SciGirls multimedia to fifth grade girls’ experience of citizen science. The treatment group (n = 49) experienced 2 hours of SciGirls videos and games at home followed by a 2.5 hour FrogWatch USA citizen science session. The control group (n = 49) experienced the citizen science session without prior exposure to SciGirls. Data from post surveys and interviews revealed that treatment girls, compared to control girls, demonstrated significantly greater interest in their
Through the NSF Innovation Corps for Learning Program, (I-Corps L), this project will develop ways to enable the SciStarter program to extend the promise of citizen science by connecting millions of citizen scientists with scientists in need of their help through formal and informal research projects. Citizen science is a fast growing field that engages the public in scientific inquiry through data collection projects and environmental monitoring using sensors, mini spectrometers, water testing kits and other tools. A challenge for the citizen science community has been access to the tools required to collect the types of data needed in citizen science projects. SciStarter facilitates broader participation in citizen science by reducing the barrier for volunteers to identify, acquire, and use the right scientific tools and instruments for each project. This I-Corps for Learning project will develop approaches to enable SciStarter to provide a larger number of citizen scientists with easier access to required and recommended instruments needed for meaningful participation in citizen science projects.
SciStarter aims to provide a holistic solution to the needs of citizen scientists that includes projects, support, and products such as training materials and consulting. SciStarter can be a catalyst in citizen science by connecting people to opportunities to engage and in lowering barriers to public participation in scientific research while creating a hybrid academic-consumer sustainability model. A central focus of this current effort will be establishing a sustainable and scalable means of enabling citizen scientists to obtain equipment and instruments in an efficient and cost-effective manner. The project will make use of elements already in place to expand the engagement of citizen scientists in new or multiple projects, to empower citizens in the process of citizen science, and to provide a useful, scalable and sustainable solution for scientists leading citizen science research projects. The extension of SciStarter will set the stage for greater inclusion of previously marginalized groups in citizen science activities and will extend to all forms of public engagement in science.
As part of an overall strategy to enhance learning within informal environments, the Innovations at the Nexus of Food, Energy, and Water Systems (INFEWS) and Advancing Informal STEM Learning (AISL) programs partnered to support innovative models poised to catalyze well-integrated interdisciplinary research and development efforts within informal contexts that transform scientific understanding of the food, energy, and water systems (FEWS) nexus in order to improve system function and management, address system stress, increase resilience, and ensure sustainability. This project addresses this aim by using systems thinking and interdisciplinary integration approaches to develop a novel immersive educational simulation game and associated materials designed to highlight the role and importance of corn-water-ethanol-beef (CWEB) systems in supporting the ever increasing demands for food, energy, and water in the United States. The focus on FEWS and sustainable energy aligns well with both the INFEWS program and the sizable sustainability-related projects in the AISL program portfolio. The development and broad dissemination of a multiuser game specific to CWEB systems are particularly innovative contributions and advance for both program portfolios and their requisite fields of study. An additional unique feature of the game is the embedding of varying degrees of economic principles and decision-making along with the nuisances of cultural context as salient variables that influence systems thinking. Of note, a team of computer science, management and engineering undergraduate students at the University of Nebraska - Lincoln will be responsible for the engineering, development, and deployment of the game as their university capstone projects. If successful, this game will have a significant reach and impact on youth in informal programs (i.e., 4-H clubs), high school teachers and students in agriculture vocational education courses, college students, and the public. The impact could extend well beyond Nebraska and the targeted Midwestern region. In conjunction with the game development, mixed-methods formative and summative evaluations will be conducted by an external evaluator. The formative evaluation of the game will focus on usability testing, interest and engagement with a select sample of youth at local 4-H clubs and youth day camps. Data will be collected from embedded in-game survey questionnaires, rating scales, observations and focus groups conducted with evaluation sample. These data and feedback will be used to inform the design and refinement of the game. The summative evaluation will focus on the overall impacts of the game. Changes in agricultural systems knowledge, attitudes toward agricultural systems, interest in pursuing careers in agricultural systems, and decision making will be aligned with the Nebraska State Science Standards and tracked using the National Agricultural Literacy Outcomes (NALOs) assessment, game analytics and pre/post-test measures administered to the evaluation study sample pre/post exposure to the game.
DATE:
-
TEAM MEMBERS:
Jeyamkondan SubbiahEric ThompsonDeepak KeshwaniRichard KoelschDavid Rosenbaum
Changes in household-level actions in the U.S. have the potential to reduce rates of greenhouse gas (GHG) emissions and climate change by reducing consumption of food, energy and water (FEW). This project will identify potential interventions for reducing household FEW consumption, test options in participating households in two communities, and collect data to develop new environmental impact models. It will also identify household consumption behavior and cost-effective interventions to reduce FEW resource use. Research insights can be applied to increase the well-being of individuals at the household level, improve FEW resource security, reduce climate-related risks, and increase economic competitiveness of the U.S. The project will recruit, train, and graduate more than 20 students and early-career scientists from underrepresented groups. Students will be eligible to participate in exchanges to conduct interdisciplinary research with collaborators in the Netherlands, a highly industrialized nation that uses 20% less energy and water per person than the U.S.
This study uses an interdisciplinary approach to investigate methods for reducing household FEW consumption and associated direct and indirect environmental impacts, including GHG emissions and water resources depletion. The approach includes: 1) interactive role-playing activities and qualitative interviews with homeowners; 2) a survey of households to examine existing attitudes and behaviors related to FEW consumption, as well as possible approaches and barriers to reduce consumption; and 3) experimental research in residential households in two case-study communities, selected to be representative of U.S. suburban households and appropriate for comparative experiments. These studies will iteratively examine approaches for reducing household FEW consumption, test possible intervention strategies, and provide data for developing systems models to quantify impacts of household FEW resource flows and emissions. A FEW consumption-based life cycle assessment (LCA) model will be developed to provide accurate information for household decision making and design of intervention strategies. The LCA model will include the first known farm-to-fork representation of household food consumption impacts, spatially explicit inventories of food waste and water withdrawals, and a model of multi-level price responsiveness in the electricity sector. By translating FEW consumption impacts, results will identify "hot spots" and cost-effective household interventions for reducing ecological footprints. Applying a set of climate and technology scenarios in the LCA model will provide additional insights on potential benefits of technology adoption for informing policymaking. The environmental impact models, household consumption tracking tool, and role-playing software developed in this research will be general purpose and publicly available at the end of the project to inform future education, research and outreach activities.
DATE:
-
TEAM MEMBERS:
David WatkinsBuyung AgusdinataChelsea SchellyRachael ShwomJenni-Louise Evans
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at the University of Colorado. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
This is an Early-concept Grant for Exploratory Research supporting research in Smart and Connected Communities. The research supported by the award is collaborative with research at DePaul University. The researchers are studying the use of technologies to enable communities to connect youth and youth organizations to effectively support diverse learning pathways for all students. These communities, the youth, the youth organizations, formal and informal education organizations, and civic organizations form a learning ecology. The DePaul University researchers will design and implement a smart community infrastructure in the City of Chicago to track real-time student participation in community STEM activities and to develop mobile applications for both students and adults. The smart community infrastructure will bring together information from a variety of sources that affect students' participation in community activities. These include geographic information (e.g., where the student lives, where the activities take place, the student transportation options, the school the student attends), student related information (e.g., the education and experience background of the student, the economic status of the student, students' schedules), and activity information (e.g., location of activity, requirements for participation). The University of Colorado researchers will take the lead on analyzing these data in terms of a community learning ecologies framework and will explore computational approaches (i.e., recommender systems, visualizations of learning opportunities) to improve youth exploration and uptake of interests and programs. These smart technologies are then used to reduce the friction in the learning connection infrastructure (called L3 for informal, formal, and virtual learning) to enable the student to access opportunities for participation in STEM activities that are most feasible and most appropriate for the student. Such a flexible computational approach is needed to support the necessary diversity of potential recommendations: new interests for youth to explore; specific programs based on interests, friends' activities, or geographic accessibility; or programs needed to "level-up" (develop deeper skills) and complete skills to enhance youths' learning portfolios. Although this information was always available, it was never integrated so it could be used to serve the community of both learners and the providers and to provide measurable student learning and participation outcomes. The learning ecologies theoretical framework and supporting computational methods are a contribution to the state of the art in studying afterschool learning opportunities. While the concept of learning ecologies is not new, to date, no one has offered such a systematic and theoretically-grounded portfolio of measures for characterizing the health and resilience of STEM learning ecologies at multiple scales. The theoretical frameworks and concepts draw together multiple research and application domains: computer science, sociology of education, complexity science, and urban planning. The L3 Connects infrastructure itself represents an unprecedented opportunities for conducting "living lab" experiments to improve stakeholder experience of linking providers to a single network and linking youth to more expanded and varied opportunities. The University of Colorado team will employ three methods: mapping, modeling, and linking youth to STEM learning opportunities in school and out of school settings in a large urban city (Chicago). The recommender system will be embedded into youth and parent facing mobile apps, enabling the team to characterize the degree to which content-based, collaborative filtering, or constraint based recommendations influence youth actions. The project will result in two measurable outcomes of importance to key L3 stakeholder groups: a 10% increase in the number of providers (programs that are part of the infrastructure) in target neighborhoods and a 20% increase in the number of youth participating in programs.
InformalScience.org is an online collection of resources designed to serve a broad community of professionals whose work relates to informal education in science, technology, engineering, and math (STEM). Funded by the National Science Foundation and managed by the Center for Advancement of Informal Science Education (CAISE), InformalScience hosts a variety of usercontributed resources, including a wiki of evidence-based information on the impacts of informal STEM and a database of over 7,000 reports, articles, project descriptions, and other items uploaded by CAISE and website members.
The