Evaluation Report: Framework for the Assessment of Environmental Literacy

Leaders of the only two large-scale assessments of environmental literacy used in the U.S. to date (Programme for International Student Assessment [PISA] and the National Environmental Literacy Assessment [NELA]), led by an experienced PI representing the North American Association for Environmental Education, to develop a new, comprehensive, research-based framework for assessing environmental literacy. By bringing together, for the first time, experts in research, assessment, and evaluation from the fields of science education, environmental education, and related social science fields - professional communities that do not traditionally share knowledge - this project was intended to access and build on the existing literature and the insights of many disciplines. The project had three overarching goals for the conference grant: 1. Synthesize the knowledge and research base for environmental literacy, including its dimensions and precursors, and draft a comprehensive environmental literacy framework that identifies competencies, dispositions and behaviors. To achieve this, the project intended to: 2. Produce and publish an Environmental Literacy Framework that had been critiqued by experts in science education, environmental education, and other related fields involved in assessing environmental literacy. 3. Disseminate the Environmental Literacy Framework to a wide audience of assessment specialists, agencies making policy decisions about and funding work on assessments, and organizations working to achieve environmental literacy. The evaluation was driven by four overarching evaluative questions. The first three served as the framework for the formative evaluation and relate directly to the two phases of the project; the fourth frames the accountability or summative evaluation. The questions asked: 1. In what ways did the planning process affect the understandings and beliefs of the Organizing Committee? 2. What is the value of the literature review by those most in positions to use it? 3. In what ways did the Workshop, through the work of building the framework, change the participants? 4. How do experts describe the effectiveness of the Workshop in providing a useful framework for the field? The primary outputs for this project were the workshop, the literature review, and the framework for assessment of environmental literacy. In terms of producing those outputs, the project clearly was effective. The quality of the products and the quality of the process leading to these outputs does, however, have qualitative components and thus to understand the outcomes of the project, more specific process and use questions are necessary. Thus, the summative evaluation was framed on four large questions: 1. In what ways did the planning process affect the understandings and beliefs of the Organizing Committee? The desired outcome of organizers being aware of positive change as a result of engaging exceeded its impact level: organizers described many more than two areas of growth and change each. The organizing committee emerged through this project and process as a team that honored each other, honored and valued differing perspectives and disciplines, and treated (and continue) to treat each other with respect and compassion. The depth of learning by the team regarding the framework, the components of literacy and environmental literacy, and the complexity of attempting to frame such a complex construct is tremendous. The products resulting from their work are seen as useful and of value to those in the field. 2. What is the value of the products by those most in positions to use it? The desired outcome of having participants serve as champions for the framework was met. All participants reported and described areas of growth and change. Although initial thoughts were that the literature review would be used for university classes, the real use of the literature review is by professionals (academics and practitioners) in their own work and in sharing with colleagues. The use of the framework, likewise, is seeing its potential use in individual consideration of practice, and as of yet is not serving for assessment construction, though that will likely occur. Participants in the workshop reported fairly consistent use of products of the project, some using the products routinely in their work. 3. In what ways did the Workshop, through the work of building the framework, change the participants? The desired outcome of participants in the workshop being champions for the framework was met with 100% of participants reporting and describing areas of growth and change was met. Participants in the workshop were very supportive of, and identified a commitment to being champions of the assessment framework. Further, the workshop facilitated participants gaining new insights into cross-disciplinary understanding and dialogue. Participants felt the workshop was important, useful, and productive. Many entered the workshop not knowing what to expect or with expectations of a different type of experience. All left the workshop with a positive orientation toward the project, the work of the project, and the products. These orientations remained strongly positive a year out from the workshop.

If you would like to edit a resource, please email us to submit your request.