In this paper, the Franklin Institute's Ann Mintz discusses the managerial challenges associated with evaluation projects. Mintz explains how evaluators teeter on a continuum serving as both as artists and educators throughout the evaluation process. She cites evidence from an ongoing project at the Franklin Institute called the The Franklin Institute Computer Network that serves seven categories of museum visitors.
This paper analyzes the effectiveness of real people (actors) as communicators of messages in museums. It includes findings from an evaluation of professional actors, who assume the roles of fictitious and real characters from the history of science, technology, and medicine at the Science Museum in London. The study attempted to understand more fully how visitors react to such live interpretations.
This report presents front-end evaluation findings at the North Carolina Transportation Museum. The study was conducted to assist with the formidable task of programming the 110,000-square-foot roadhouse and to provide data for researchers and exhibit designers. Front-end evaluation goals included: (1) determine if pre-existing or natural traffic flow patterns were present in the roundhouse; (2) evaluate visitor interest in proposed research topics; and (3) obtain visitor reactions, insights, attitudes and expectations concerning proposed roundhouse programming.
In this paper, researchers at the Brookfield Zoo present a case study in evaluating a technology project involving partnerships between museums and formal education. THe focus is on the multiple-method design, which was required in order to work with all participants, from funders to educators, to teachers and students. A set of tools, from traditional surveys through teacher-led performance assessments, was used to measure student learning, teacher satisfaction, and effective implementation of technology and museum content into quality Web pages. The authors share their experiences to help
DATE:
TEAM MEMBERS:
H. Elizabeth Stuart PerryCarol D. Saunders
This is a summary of A.W. Melton's 1936 paper, "Distribution of Attention in Galleries in a Museum of Science and Industry." Melton studied the role of movement in attracting visitors by evaluating a gear-shaper located in the machine tool section of the New York Museum of Science and Industry. Melton determined that while any kind of movement attracts visitors, the overall consequences of this movement on the exhibit may not be be positive.
In this article, Annette Noschka-Roos discusses a study of a computer-supported information system (CIS) touch-screen interactive in the "New Energy Techniques" gallery at the Deutsches Museum. The objective of the study was to gather systematic data on how the medium is used by visitors. Noschka-Roos provides key findings from the study.
DATE:
TEAM MEMBERS:
Annette Noschka-RoosVisitor Studies Association
Project LIFTOFF works with local, regional, and national partners to engineer statewide systems for Informal Science Education that inspire: YOUTH to pursue STEM education and careers through increased opportunities for quality, hands-on STEM learning. AFTERSCHOOL STAFF to facilitate STEM learning experiences that contribute to the overall STEM education and aspirations of youth in their programs. PROGRAM ADMINISTRATORS to encourage and support staff in the integration of STEM enrichment into the daily programming. STATE LEADERS to sustain and expand afterschool learning opportunities so that all students have access to engaging STEM experiences outside of the regular school day. Project LIFTOFF is dedicated to the development of the following essential elements of statewide systems for informal science education:
Access to appropriate STEM Curriculum for youth of all ages, abilities, and socio-cultural backgrounds that meets the needs and interests of individual community programs
Systematic STEM Professional Development that matches individual skills in positive youth development with abilities to facilitate discovery and science learning
A diverse Cadres of Trainers who will deliver the professional development, technical assistance and curriculum dissemination in their local communities
Authentic Evaluation of informal science efforts that determine the impacts on youth aspirations and the capacity of youth programs to provide quality STEM experiences
Local STEM education leadership to identify the ways in which collaborative education efforts can advance the development of 21st Century Skills and the preparedness for STEM workforce and higher education
Partnerships in support of youth development and informal science education that convene local, regional, and statewide organizations and stakeholders
To advance national initiatives and states' sySTEM engineering efforts, LIFTOFF coordinates an annual convening, the Midwest Afterschool Science Academy, that brings together national informal science experts, system leaders and youth development professionals to elevate the levels of science after school. The 5th MASA will be in the spring of 2014 in Kansas City, MO
DATE:
TEAM MEMBERS:
Missouri AfterSchool NetworkJeff Buehler
Museums continue to invest in and experiment with internet technologies and increasingly with social software environments (i.e., social networking). These technologies have the potential to lead to a number of important intellectual and social outcomes such as learning, community building, and greater public understanding of, in our case, science. It is the possibility of supporting learning in digital environments that is the focus of this research project. In our previous work, online facilitation has emerged as a big deal and perhaps determines successful online museum environments from unsuccessful environments. To study facilitation, we seek to understand facilitation styles and their outcomes in two distinct but representative museum environments. The first, Science Buzz at Science Museum of Minnesota, is a popular website identified by the field to be exemplary because of its educational value and its use of Web 2.0 functionality. The second case is the more distributed use of social software at the North Carolina Museum of Life and Science (MLS). Instead of creating learning platforms that are hosted internally, MLS is experimenting with building learning communities where people are already gathering on the web like Flickr, Twitter, and YouTube. We anticipate being able to identify clear, replicable facilitation styles and to identify outcomes associated with those styles.
This MSP-Start Partnership, led by Widener University, in partnership with Bryn Mawr College, Delaware County Community College, Philadelphia University, Lincoln University, and Haverford Township School District, is developing the Greater Philadelphia Environment, Energy, and Sustainability Science (ES)2 Teacher Leader Institute. Additional partners include the Center for Social and Economic Research at West Chester University, Delaware Valley Industrial Resource Center, Energy Coordinating Agency, US EPA Region 3 Office of Innovation, National Center for Science and Civic Engagement and its SENCER program, Pennsylvania Campus Compact, Philadelphia Higher Education Network for Neighborhood Development, Project Kaleidoscope, Sustainable Business Network of Greater Philadelphia, and the 21st Century Partnership for STEM Education. Building on a base of relationships developed over the past five years by many partners in the Math Science Partnership of Greater Philadelphia, the project brings together faculty and resources from multiple institutions (a "Mega-University" model) to develop a coherent, innovative, and content-rich, multi-year curriculum in environment, energy, and sustainability science for an Institute that leads to a newly developed Master's degree. Teachers participating in the Institute (A) improve their STEM content knowledge in areas critical to human environmental sustainability, (B) improve their use of project based/service learning and scientific teaching pedagogies in their teaching, (C) engage in real-world sustainability problem solving in an externship with a local business, non-profit or government organization that is active in the newly emerging green economy, and (D) develop important leadership skills as change agents in their schools to improve student interest, learning, and engagement in STEM education. The Institute aims to serve as a regional hub, connecting educational, business, non-profit and government organizations to strengthen the STEM education and workforce development pipelines in the region and simultaneously support positive social change toward environmental sustainability and citizenship. The project's "Mega-University" and "Institute as a regional connector-hub" approaches are powerful models of collaboration that could have widespread and significant national applicability as organizations and systems adjust to the new challenges of our global economy and to the needed transition to sustainability.
DATE:
-
TEAM MEMBERS:
Stephen MadigoskyWilliam KeilbaughVictor DonnayBruce GrantThomas Schrand
Realizing the power of CyberLearning to transform education will require vision, strategy, and an engaged, talented community. Activities are needed to energize the community, refine and sharpen the path forward, and provide a more active and ongoing forum for clarifying the big ideas and challenging questions. In response to this need, SRI International, together with the Lawrence Hall of Science and with key support from the National Geographic Society, will organize a set of activities to advance a shared vision of the future of learning, encompassing the systems, people, and technology dimensions mutually necessary for any scalable and lasting advances in education. The innovative format for these activities is inspired by the TED talks, Wikipedia, and social networking. As in TED, a small set of leading researchers will be selected to give very short, very high quality, stimulating talks. These CyberLearning Talks will be featured at a 1-day summit meeting in Washington, DC, streamed so that local cyberlearning research communities may participate at a distance, and posted on a website. As in Wikipedia, CyberLearning Pages will be created, each page featuring a synopsis of a big idea in CyberLearning and the relevant research challenges. The 1-day conference will be followed by a small 1-day workshop focusing on how to evaluate cyberlearning efforts, identify progress, and identify important new directions. Finally, to disseminate and stimulate conversation about both the video talks and Wikipedia entries, a presence for the community will be created on social networking sites. The target outcomes of the effort will be (i) a cyberlearning research community with participants from across the many current constituent communities, and fostered awareness and appreciation of the broad range of expertise and interests across that wider community; (ii) foundations for sustained discussion of big ideas, insights, and challenges to help this new community define a more engaged, crisper vision of its own future, (iii) a community resource that can become a site for interconnecting stakeholders in the CyberLearning community and supporting investigators in improving field-generated proposals, and (iv) an emerging sense of direction for CyberLearning among a wider audience of leaders. Such community building and awareness is expected to foster collaborations that will lead to innovative and research-grounded ways of using technology to transform education -- formal and informal and across a lifetime.
The Learning and Youth Research and Evaluation Center (LYREC) is a collaboration of the Exploratorium, Harvard University, Kings College London, SRI International and UC Santa Cruz. LYREC provides technical assistance to NSF AYS projects, collects and synthesizes their impact data, and oversees dissemination of progress and results. This center builds on the Center for Informal Learning in Schools (CILS) that has developed a theoretical approach that takes into account the particular strengths and affordances of both Out of School Teaching (OST) and school environments. This foundation will permit strengthening the potential of the NSF AYS projects to develop strong local models that can generate valid and reliable data that can guide future investment, design and research aimed at creating coherence across OST and school settings. The overarching questions for the work are: 1. How can OST programs support K-8 engagement and learning in science, and in particular how can they contribute to student engagement with K-8 school science and beyond? 2. What is the range of science learning outcomes OST programs can promote, particularly when in collaboration with schools, IHE's, businesses, and other community partners? 3. How can classroom teachers and schools build on children's OST experiences to strengthen children's participation and achievement in K-12 school science Additionally, the data analysis will reveal: 1. How OST programs may be positioned to support, in particular, high-poverty, female and/or minority children traditionally excluded from STEM academic and career paths; and 2. The structural/organizational challenges and constraints that exist to complicate or confound efforts to provide OST experiences that support school science engagement, and conversely, the new possibilities which are created by collaboration across organizational fields. Data will be gathered from surveys, interviews, focus groups, evaluation reports, and classroom and school data.
Researchers at Michigan State University, University of Washington, Science Museum of Minnesota, and Museum of Life and Science found that there are clear indicators of learning in Science Buzz (www.sciencebuzz.org), the online museum environment studied as part of the Take 2 project. People who participate in conversations through the Buzz blog demonstrate an interest in science, and they leverage their own experiences and identities in order to share science knowledge with others. Researchers utilized indicators of learning as identified in the National Academies report on Learning Science in Informal Environments. Aspects of learning that were particularly important for an online environment like Science Buzz were interest in science, participating in science through the use of language, and identifying as someone who knows about or uses science. Researchers found that Science Buzz participants had a strong interest in scientific issues, utilized argumentation strategies--an important scientific practice--and identified with the importance of science in their lives. In particular: (1) Interest in scientific issues, caring about scientific issues, identifying personally with scientific issues were commonly evident in Science Buzz; (2) There is widespread use of argumentation in relation to scientific issues, an important scientific practice, although the quality of the scientific reasoning associated with these argumentation practices varies; (3) The co-construction of identity between online participants and the host museum is a potentially powerful outcome, as it suggests that online learning environments can facilitate longer-term relationships; (4) The analytical tools developed by this project advance our ability to understand learning in online environments; (5) While some indicators of learning are present, others, such as reflecting on science or co-constructing science knowledge with others, are not present. For museums, encouraging museum staff to engage digital tools and online participants is relatively easy. However, measuring online activity with regard to complex outcomes like learning is extremely difficult. Perhaps the most useful outcome of the Take 2 project, therefore, is a tool that will enable museums to make sense of online activity in relation to powerful outcomes like learning.