What Can you Find in the BISE Database?

This post was co-written by Amy Grack Nelson and Denise Deng of the Science Museum of Minnesota.

As described in our previous blog posts, the Building Informal Science Education (BISE) project created a coding framework to code and synthesize evaluation reports on the previous version of InformalScience.org, which included all reports voluntarily posted through May 2013. In total, we ended up coding 521 evaluation reports and will soon share the BISE database and related resources with the field. So, what will you find in the BISE database? What is available for you to dig into to help inform your own work? This blog post presents a snapshot of what you will soon have access to through these new resources.

Evaluands

The BISE database contains evaluations of a wide variety of evaluands, where “evaluand” refers to the object(s) being evaluated. Evaluations of exhibitions are the most common in the database, followed by broadcast media, and website or software. A number of reports included evaluations of multiple evaluands.

Table 1. Number of evaluations of each type of evaluand

Year Reports Were Written

Of the reports included in the BISE database, the earliest was written in 1990. The number of reports written each year increases dramatically after 2001, with the peak year in 2010.

Figure 1. Number of reports written each year (n=521)

Evaluation Types

A majority of the reports (66%) in the BISE database are summative evaluations. The database also includes formative, front-end, and remedial evaluations, as well as audience studies. Some reports included multiple types of evaluation studies.

Figure 2. Number of each type of evaluation included in the BISE database (n = 521)

Data Collection Methods

Evaluators use a wide variety of data collection methods to evaluate informal learning projects. Interviews and surveys are the most commonly used methods in the evaluations included in the BISE database, with many reports using multiple methods.

Figure 3. Number of evaluation reports citing each data collection method (n = 521)

Language Translation

One of the BISE coding categories looked at language translation as part of the project deliverable and/or data collection. For example, an evaluation studying an exhibition produced in multiple languages or an evaluation that collects data by conducting a focus group in Spanish. Of the 521 reports, 38 (7%) dealt with language translation. Listed below are the various language translations included in the evaluations, with Spanish being the most common.

1. Spanish (33 reports)
2. American Sign Language (3 reports)
3. Chinese (3 reports)
4. Yucatec (Mayan)
5. Vietnamese
6. Tohono O'odham
7. Braille
8. Bengali
9. French

Accessibility

Reports were coded to indicate if the evaluation dealt with issues of accessibility. This meant the evaluator gathered feedback from individuals with cognitive and/or physical disabilities. Only 15 reports (3%) in the BISE database address accessibility issues.

Next Steps: Using the Database

While the BISE database is not representative of all evaluations in the informal science education field, the data from these reports offer a window into how evaluations are conducted in the field and can be a useful resource to evaluators. How would you use the BISE database? How do you see this as being a useful resource for the field? Our upcoming blog posts will feature examples of how the database can be used, but we’d love to hear from you, too. Please post your ideas in the Comments section below!