This website is set up to provide easy to use tips to design, implement and assess the quality of evaluations on programs and projects to improve the quality, quantity and diversity of the STEM workforce. However, most of the tips and other information can be of use to a broad range of researchers and evaluators and those who fund them.
DATE:
TEAM MEMBERS:
Science Museum of MinnesotaEric JollyPatricia CampbellTom Kibler
Produced by National Geographic Kids and Cricket Moon Media with support from the National Science Foundation, Marine Missions is a free iPad touch screen application for preschoolers. The app is hosted by Jacques, a hermit crab character who guides pre-readers through six ocean missions and the building of a fantasy sea creature. Players steer Jacques’ boat to three green markers to clean up polluted spots in the ocean and three orange markers to complete different water current challenges in which players rescue Jacques’ tools from a whirlpool, run past blowholes, and surf on tidal bores
Rockman et al (REA) conducted an independent summative evaluation of the Eutelelost Tree of Life: Web Module. The Euteleost Tree of Life (EToL) is a collaborative National Science Foundation research project between eight institutions, including Kansas University, to resolve evolutionary relationships within the euteleost fishes. The main aim of the interactive Web module is for users to explore the evolutionary history of fish and how they are related to one another. REA's summative evaluation was designed to (1) gather users' feedback about the website, as well as (2) assess the impact of
Across the globe, citizen science projects are becoming increasingly poised to address social and environmental challenges and answer broad scientific questions. Although rapidly increasing in number, these projects need easy-to-use software tools for data management, analysis, and visualization to be successful. This project transforms how citizen science projects unfold locally, regionally, and globally by creating software that supports the full spectrum of project activities. It empowers projects to ask and answer their own local questions while contributing data critical to larger-scale issues. These tools will allow projects to announce training events; track volunteers; create datasheets; enter, review, analyze, and visualize data; publish reports; discover resources; integrate data; and ensure that data are contributed to repositories (e.g., DataONE, NEON, GBIF, HydroShare, and EOL). Tools will be made available to citizen science projects and will be delivered as reusable software elements for use in existing websites; as website features on CitSci.org; and as Application Programming Interface (API) services and mobile applications. The tools will expand the national reach, local appeal, computational abilities, visualization techniques, statistical analysis capabilities, and interoperability of the nations' cyber-infrastructure. Using participatory design and agile methods, the project will: (1) develop reusable software elements that citizen science organizations can embed into their own websites, (2) harden and expand the functionality and capabilities of CitSci.org through new website features, and (3) extend the APIs of CitSci.org and develop associated mobile applications to increase system and tool interoperability. The target user communities will include citizen science project coordinators. It will deliver customizable tools and services related to all project activities and engage projects across a wide array of disciplines. Project coordinators will be able to customize all tools developed to suit their specific project needs. Adoption and use of the tools developed will create a cyber-ready workforce capable of collecting, contributing, and applying high quality ecological, geophysical, social, and human health related observations to solve real-world problems. These broader impacts will help the citizen science community better understand effective models of public engagement to ensure more impactful application of citizen science to societal challenges.
DATE:
-
TEAM MEMBERS:
Gregory NewmanStacy LynnMelinda Laituri
This project from the University of Florida proposes to derive and develop a network and community of practice (CoP) among amateur and professional paleontologists across the country. Should this project be successful, it would put together 40 professional Paleontologist with 23 amateur Paleontology clubs across the country. The advantage of this organization would be to facilitate sharing of specimens (digitally on the web), educating each other, and most important, making the public outreach from each club more effective. While each club has specimens, this network would provide access to over 100 million digitized samples. The web-base for this collection will be managed at the University of Florida under the direction of the PI. The research portion of this project will determine what the essential elements necessary are for effective learning between professionals and amateurs and how the CoP enhances amateur learning and outreach efficacy. The project plan includes a centralized organization to initially form the community of practice, call general meetings, publish newsletters and organized a large meeting at the University of Florida in the coming year. Further, the project team will conduct evaluation on how the project is helping members develop and how the organization can be improved. Educating individuals in the field of paleontology is generally a positive experience. This project will facilitate knowledge building among the individual members of the clubs, which will enhance their perspective and enable them to reach out to members of their communities. The project will be evaluated at every level to ensure that the existing clubs are incorporated into the project and new clubs are welcomed and engaged as well.
This full-scale project addresses the need for more youth, especially girls, to pursue an interest in engineering and eventually fill a critical workforce need. The project leverages museum-based exhibits, girls' activity groups, and social media to enhance participants' engineering-related interests and identities. The project includes the following bilingual deliverables: (1) Creative Solutions programming will engage girls in group oriented engineering activities at partner community-based organizations, where the activities highlight altruistic, personally relevant, and social aspects of engineering. Existing community groups will use the activities in their regular meeting structure. Visits to the museum exhibits, titled Design Your World will reinforce messages; (2) Design Your World Exhibits will serve as a community hub at two ISE institutions (Oregon Museum of Science and Industry and the Hatfield Marine Science Center). They will leverage existing NSF-funded Engineer It! (DRL-9803989) exhibits redesigned to attract, engage, and mobilize a more diverse population by showcasing altruistic, personally relevant, and social aspects of engineering; (3) Digital engagement through targeted use of social media will complement program and exhibit content and be an online portal for groups engaged in the project; (4) A community action group (CAG) will provide professional development opportunities to stakeholders interested in girls' STEM identity (e.g. parents, STEM-based business professionals) to promote effective engineering messaging throughout the community and engage them in supporting project participants; and (5) Longitudinal research will explore how girls construct and negotiate engineering-related identities through discourse across the project activities and over time.
COASSTal Communities of Science is a citizen-science project whose goal is to increase the scientific and educational reach of a highly successful, action-oriented and rigorous citizen science program - the Coastal Observation And Seabird Survey Team (COASST), by adding a new data module on marine debris that will feature innovative technological approaches including mobile apps and web-based crowdsourcing. The marine debris module will complement an existing module on beached birds, allowing COASST to more completely assess coastal environmental health. For instance, marine debris data, focused particularly on issues of invasive species, harm to wildlife, and debris sourcing, will be immediately useful in marine science and resource management. Once designed and vetted by professional scientists and science educators, the new module will be implemented by citizen scientists in over 100 in-community trainings conducted throughout the COASST geographic range, from northern California to the coast of Alaska, including remote coastal communities with limited access to scientific information. Over 1,000 new participants will join the program, bringing the total number of active volunteers to 2,000 within the 4 years of the project. A complementary social science research component will advance the field of informal STEM learning by focusing on the factors facilitating recruitment and especially retention in citizen science projects, using COASST as a model. The current models of science learning in informal contexts will be extended by bringing them together with conceptual approaches to the development of interest, communities of practice, and activity theory. Research will specifically focus on differences in individual motivation to join COASST; follow participants as they enter the program and eventually become central members of the COASST community of practice; and assess the degree to which individual, programmatic and socio-cultural factors contribute to participant retention. A linked independent evaluation will assess the depth of learning individuals experience as a function the training and materials they receive, and amount and type of data they collect. Both research and evaluation components will utilize pre/post surveys, interviews, and longitudinal journaling.
The Badges for College Credit project designs and researches: (1) a digital badge system that leads to college credit as the context for investigating how to integrate badges with learning programs; (2) how to assess learning associated with badges; and (3) how badges facilitate learning pathways and contribute to science identity formation. The project is one of the first efforts to develop a system to associate informal science learning with college credit. The project will partner with three regional informal science institutions, the Pacific Science Center, the Future of Flight, and the Seattle of Aquarium, that will facilitate activities for participants that are linked to informal science learning and earning badges. The project uses the iRemix platform, a social learning platform, as a delivery system to direct participants to materials, resources, and activities that support the learning goals of the project. Badges earned within the system can be exported to the Mozilla Open Badges platform. Participants can earn three types of badges, automatic (based on participation), community (based on contributions to building the online community), and skill (based on mastery of science and communication) badges. Using a learning ecologies framework, the project will investigate multiple influences on how and why youth participate in science learning and making decisions. Project research uses a qualitative and quantitative approach, including observations, interviews, case studies, surveys, and learning analytics data, and data analytics. Project evaluation will focus on the nature and function of the collaboration, and on the scale-up aspects of the innovation and expansion, by: (1) analyzing and documenting effective procedures,and optimal contexts for the dissemination of the model and (2) by analyzing the collaboration between informal science organizations and higher education.
DATE:
-
TEAM MEMBERS:
Carrie TzouKaren LennonAmanda GoertzGray Kochlar-Lindgren
The Global Soundscapes! Big Data, Big Screens, Open Ears Project uses the new science of soundscape ecology to design a variety of informal science learning experiences that engage participants through acoustic discovery Soundscape ecology is an interdisciplinary science that studies how humans relate to place through sound and how humans influence the environment through the alteration of natural sound composition. The project includes: (1) an interface to the NSF-funded Global Sustainable Soundscapes Network, which includes 12 universities around the world; (2) sound-based learning experiences targeting middle-school students (grades 5-8), visually impaired and urban students, and the general public; and (3) professional development for informal science educators. Project educational components include: the first interactive, sound-based digital theater experience; hands-on Your Ecosystem Listening Labs (YELLS), a 1-2 day program for school classes and out-of school groups; a soundscape database that will assist researchers in developing a soundscape Big Database; and iListen, a virtual online portal for learning and discovery about soundscape. The project team includes Purdue-based researchers involved in soundscape and other ecological research; Foxfire Interactive, an award-winning educational media company; science museum partners with digital theaters; the National Audubon Society and its national network of field stations; the Perkins School for the Blind; and Multimedia Research (as the external evaluator).
This full scale/collaborative research project targets underserved youth who will collaborate with STEM professionals to co-create community-relevant technology and media content creating a potentially scalable model. App Lab DIY is a collaboration between Youth Radio, the MIT Media Lab's Center for Mobile Learning, and the Mozilla Foundation. There are multiple and well integrated project strategies. Youth working with professionals will design, create, and disseminate original apps that engage users around community issues (e.g. urban wildlife mapping; Community DNA). They will work directly with the MIT Media Lab/Mobile Learning Center to improve App Inventor, an open-source tool. Youth participants will create media content for national on-air and online outlets including NPR. They will design digital badges to validate and document what they've learned creating media and apps. Curricular resources and tools for educators will be created and disseminated to advance STEM learning among underserved youth. Deliverables include: 1) creation of three apps (media-rich, STEM-relevant, community valued) that engage communities; 2) improvements in App Inventor, MIT's open-source tool that democratizes app develop for and by all; 3) radio stories that are integrated with the apps; 4) a multimedia toolkit for informal and formal educators to adapt in their programs; 5) a pilot badging system to mark mastery of skills associated with youth-driven media production and technological innovation; and 6) a research study focusing on the use of app development to drive STEM learning and engagement among underserved youth. Original research will be conducted by an embedded Scholar-in-Residence to inform and advance the field about driving STEM engagement via media/tech innovation. The Scholar-in-Residence will use Participant-Action-Research to inform and improve practice and study the big general questions about how the program works and why. The evaluation will leverage the instruments and methods established in Youth Radio's past NSF grants to study the impact on the youth's skills, knowledge and attitudes. They will also study impacts on two additional audiences: educators using the multimedia toolkit and users of the mobile apps. Approximate 1000 Youth Radio students will participate in the project over the three years through contributions to app development, radio production, and digital badges creation. The multimedia toolkit will be nationally disseminated to STEM educators with a large potential reach.
This research and development project explores the mechanisms that initiate and support innovation in early childhood education, especially in combining informal learning via public media and technology with teacher and family interactions to maximize children's math learning. Deliverables include 27 episodes of Peg+Cat, an animated math adventure series on PBS, 8 related online games and apps, summer math institutes and school-year training for preschool/Head Start teachers, and complementary activities and resources to support parent's knowledge and practice and student's engagement, interest, and learning of foundational math concepts. The research agenda will test several hypotheses regarding the strategies to extend teacher's math content knowledge and pedagogy and parent/ caregiver's understanding and valuing of math. A key question will focus on how coupled learning opportunities (professional development for teachers, transmedia, and support for families) enable and sustain children's engagement and learning in math. Formative evaluation of the media components will use focus groups of 3-5 year olds to assure develerables are engaging and accessible. The summative evaluation by Rockman et al will focus on how well the project met its overall goals including the project implementation, and impacts on Head Start teachers, parents/caregivers, and preschoolers. The project's transmedia deliverables will reach millions of preschoolers through daily PBS broadcasts and online games and apps. Fifty Head Start teachers will participate in the two-year professional development program and will be using new content knowledge and pedagogy to teach 550 Head Start children in southwest Pennsylvania and engage their parents/caregivers. The Head Start infrastructure will provide further dissemination of the project's findings and resources.
SciGirls and Citizen Science: Real Data, Real Kids, Real Discoveries SciGirls is showcasing Citizen Science! From their own backyards to a NASA research center, the bright, relatable, real girls featured on the groundbreaking PBS series are seriously into science, technology, engineering and math, or STEM. And Season Three of SciGirls finds these STEM adventurers tracking toads, counting clouds and much more, all in the name of citizen science. The brand-new season of the Emmy-winning show, featuring six stand-out episodes, debuted April 2015 on PBS KIDS (check local listings) and online at http://pbskids.org/scigirls. Citizen science is the newest STEM frontier that engages the general public –and kids – in real science. Scientists worldwide invite ordinary people—like the SciGirls—to observe and record data about everything from birds to beaches, monarch butterflies to maple trees. The data is then shared with scientists, who use it to generate new scientific knowledge. In six exciting new episodes, middle school girls and their female STEM professional mentors hit the great outdoors, cataloging frog calls, tracking the changing seasons, verifying satellite imagery of clouds, monitoring fragile butterfly populations, improving urban bird habitats, and advocating for healthy oceans. In addition, animated characters Izzie and Jake are back and finding themselves in sticky situations that can only be solved by STEM—and the SciGirls. When the SciGirls share their data with professional scientists, they save the day for Izzie and Jake and help save the environment! The new mobile-friendly website at http://pbskids.org/scigirls lets kids play new games, watch episodes and videos, and connect with fellow STEM explorers anywhere, anytime. “Collaboration is the key to successful citizen science,” said SciGirls executive producer Richard Hudson. “Since SciGirls’ beginning, working together—making discoveries, mistakes and friends—is one of the important research-based methods we use to engage girls around STEM. This new season underscores the importance of collaboration within the scientific research community and workforce. SciGirls is fortunate to have powerful partners advising us about citizen science, including the Cornell Lab of Ornithology, NASA and SciStarter.” The SciGirls creative team is headed by Twin Cities Public Television’s Director of Science Content Richard Hudson, Executive Producer of the long-running PBS children’s science series Newton's Apple and creator of DragonflyTV and the SciGirls initiative. Animation is created by Soup2Nuts, producers of PBS’ WordGirl. Strategic partners for the new series are the Cornell Lab of Ornithology, Rick Bonney co-PI, and the National Girls Collaborative Project, co-PI Karen Peterson. SciGirls is made possible by a major grant from the National Science Foundation. Additional funding is provided by INFOR, Northrop Grumman Foundation, and PPG Industries Foundation.