A Start To Developing a Logic Model

In the informal science, technology, engineering, and math (STEM) education (ISE) community, we might have different levels of awareness or knowledge of logic models, and that is okay. There are those who might not have heard of logic models, or have heard of them in bits of conversation with respect to proposal writing. I first heard of logic models when taking a course in graduate school. At the time I wasn’t quite sure what to make of it, especially in a theoretical use in the classroom. Now, after some years and seeing more examples, I recognize the uses and usefulness as part of program and project planning. This blog article provides resources, tips, and real examples that may help you start developing a logic model for your informal STEM education program or project. 
*Links to examples of logic models from NSF-AISL projects appear towards the end of this blog article. 
 
First, let’s begin with defining logic models, and how and where they are used. Logic models are, broadly speaking, intended to be a tool and a process to visually understand and clarify the relationships between actions and expected results for a program (Knowlton & Phillips, 2013 The Logic Model Guidebook). Logic models are used in nonprofits and government agencies for program planning, connecting a program to evaluation, and to explicitly show the steps that connect the activities with a program’s intended outcomes and impact. Finally, logic models are dynamic and can be updated during a program’s lifetime to reflect changes, such as new approaches because of lessons learned during the program’s implementation, or new information gained such as unintended effects and influences of a program. 

The question remains, however, why go through the process of writing a logic model for your ISE program or project? There are lots of good reasons, and some are summarized here from a range of resources including The Logic Model Guidebook by Knowlton & Phillips, and Renger and Titcomb’s 2002 article in the American Journal of Evaluation, “A Three-Step Approach to Teaching Logic Models.”
 
Benefits of a Logic Model (in no particular order)

1. Show the rationale, and help make an implicit understanding of the program explicit.
2. Connect the program rationale to the needed resources and activities.
3. Establish links to evaluating the program impacts, and outcomes.
4. Lead to improved design, planning, and management.
5. Develop a common language among stakeholders (staff, funders, the organization) and help stakeholders be on the same page.
6. Communicate effectively with funders and decision makers. (Logic models are required for some grant proposal submissions including National Institutes of Health Science Education Partnership Awards.)

 
As Tina Phillips and Rick Bonney, authors of Chapter Five of the Principal Investigator’s Guide To Managing Evaluation in Informal STEM Education Projects, succinctly wrote about logic models, “You have to know exactly what you want your project to accomplish before you can decide what you'll do to accomplish it.” They recommend developing logic models as a group from the start between the team and the evaluator in order to create a shared vision and a common understanding of project outcomes and how those outcomes will occur.  

The following 5 steps, adapted and summarized from The Logic Model Guidebook, are to help you start building a program logic model. It is important to remember to think “if-then” at each step to connect the different elements of a logic model together; and be aware of your own and your team’s assumptions at each step. 

STEP 1: Think of the Program’s Intended Results

Once you have your team and evaluator(s) together, the first step is to think of the bigger picture, or the intended results (outcomes & impacts). Impacts are thought of as the ultimate intended change and a change that will happen over time. Impacts can occur near in time, or take years to realize. Something as complex as a program influencing an individual’s identity can take years and might be measured at the end of a program, or years after a program has concluded. Thinking of a program’s overall timeline can be useful at this step.

Note: Though the traditional template of the logic model is a giant rectangle with columns labeled inputs, activities, outputs, outcomes, and impacts (impacts are sometimes included), don’t let the order on the template lead you to believe that you must begin with input and go in linear order from left to right. Evaluators and resources recommend to start with the outcomes and impacts. And, a variety of visual depictions are used to show and develop a logic model; so if the traditional template feels too linear, experiment with an alternative depiction.
 
Traditional Template of a Logic Model: 

Alternative Visual Depiction of a Logic Model: 

 
STEP 2: Describe the Outcomes that Show Progress Toward Impact

For outcomes, think of changes the program will make on participants. A program can have multiple desired outcomes that can happen in a few months, in 1-3 years (short term), or much later in 7-10 years (long term). Describing the changes that will lead to an intended impact starts to make any implicit understandings clearer as outcomes are articulated aloud and written out by team members.   

STEP 3: Link Activities to Outcomes

Activities are defined as the specific actions of a program. In ISE this can be an event, a technology (ex. mobile application, or website), media, workshops, or other. Though very different, activities are purposeful and designed to yield changes or results. In other fields, activities are also known as interventions. Linking activities to desired changes on participants can make the path towards the ultimate impact much clearer, and can answer questions as to why a particular activity was chosen. 

STEP 4: Define the Inputs (Resources) For the Activities

Inputs, or resources, can be funds to support the program, staff time, expertise of team members, audiences intended to be reached, the community where the program is based, the organization’s infrastructure, and more. Listing and defining the resources available can show if all the pieces are in place. The team can also think about the program’s feasibility in terms of time and scale, and if needed resources are not currently in hand.

STEP 5: Identify Outputs That Would Show What The Activities Have Achieved

Outputs are the direct products of activities and can often be quantified or qualified in some way. Some examples of outputs are the number of participants during a special event, the number of research briefs written, the number of views to a blog post, the number of visitors who submit comments, the amount of audience participation, the audiences who return to an activity, the types of community audiences reached, etc. Outcomes differ from outputs because outcomes are about an intended change on participants (such as skills attained, or increased awareness) and can be understood by analyzing a program’s activities and resulting outputs. 

What is Theory of Change?

In a nutshell, theory of change is the reasoning or explanation behind the steps that connect activities with a program’s intended outcomes and impact. The reasoning can be informed by previous research in the area of interest, an understanding of assumptions, knowledge of influential factors, and more. It is an essential part of articulating the purpose of a program and showing a causal pathway. Perhaps more simply put, a theory of change explains how a problem or issue is being addressed, while a logic model is a map of the program. In Chapter Five of the Principal Investigator’s Guide To Managing Evaluation in Informal STEM Education Projects, Phillips and Bonney describe theory of change in more detail, and provide various examples and a graphical representation.

Examples of Logic Models from NSF-funded AISL Projects

It can be hard to think of writing a logic model de novo, and looking at examples of how others have connected the dots can help. The following are logic models that have been submitted as part of project evaluation reports on InformalScience.org, and we’ve included links to some from other STEM programs that are freely available to download from the internet. We offer these examples as a way to learn from colleagues in the field, and not as perfect examples. In the spirit of collegiality, use the InformalScience.org Member Directory to get advice from the ISE community.

Final Report: Pathways to Brighter Futures Through STEM Careers
The logic model was submitted with the final report and is available on pages 57-60. The project focused on exploring strategies through which at-risk and incarcerated Hispanic youth could be engaged in STEM careers, and think a STEM career was possible for them. 

ScienceMakers: African Americans and Scientific Innovation
The logic model was submitted with the summative evaluation and is available on page 40. The overall project goals were to increase the general public’s awareness of the contributions of African American scientists, and lead to increased youth participation in STEM-related careers.

Investigating the Implementation of the Be a Scientist! Project in New York City and Los Angeles
The logic model was submitted with the project’s first year formative report and is available on page 5. The goal of this five-year afterschool family science program was to provide quality science and engineering courses to underserved families in New York City and Los Angeles, and for families to develop STEM knowledge and transfer STEM learning. 

STAR_Net: A Hands-on Learning Program for Libraries and Their Communities 
The logic model is available in Appendix A of the summative evaluation report. STAR_Net brought inquiry-based STEM learning experiences to public libraries. A number of activities and resources were developed to help librarians and community partners offer a wide variety of STEM programs for their patrons.

Examples of Logic Models from STEM-related Education Programs

Through a Google search on logic models for STEM education programs, the following examples of federally or state funded programs were found.

AccessSTEM: Increasing the Participation of People with Disabilities in Science, Technology, Engineering, and Mathematics Fields
This is a Research in Disabilities Education (RDE) NSF-funded project through the University of Washington. AccessSTEM helps students with disabilities succeed in STEM and reach critical junctures on paths toward college and careers in STEM. Download and print the logic model labeled, “ALTS Logic Model.”

Increasing Student Interest in Science, Technology, Engineering,and Math (STEM): Massachusetts STEM Pipeline Fund, Programs Using Promising Practices
The report describes programs in the state of Massachusetts that employed promising practices to increase student interest in STEM. The logic model can be found on page 15. 

Virginia 4-H: Science, Technology. Engineering and Math Program Logic Model
The logic model shows how Virginia 4-H Programs affect youth knowledge, awareness, and skills in STEM.  
 
Logic Model for Agricultural Literacy Programming
From the National Institute of Food and Agriculture (NIFA), United States Department of Agriculture (USDA), this logic model shows how programs about agriculture can increase youth and adult understanding of agriculture or how agriculture impacts their lives. The programs aim to help youth and adults become informed consumers, advocates, and policy makers.

More Resources on InformalScience.org

For more information on InformalScience.org about logic models such as how-to guides and templates, the following list of resources might be helpful.

Principal Investigator's Guide, Chapter 5: Planning for Success: Supporting the Development of an Evaluation Plan
This section of the PI Guide covers logic models and theory of change. For information on choosing an evaluator and collaboration, look for Chapter 3 and Chapter 4.

Assessing the Impacts of STEM Learning Ecosystems: Logic Model Template & Recommendations for Next Steps
This paper describes the impact of cross-sector partnerships on youth, and includes a logic model template for communities so they may further develop the attributes, strategies, and measures of progress that enable them to advance opportunities for youth to succeed. 

Important External Resources 

W.K. Kellogg Foundation Logic Model Development Guide
The logic model guide from the Kellogg Foundation is free and the go-to resource for evaluators and program managers across sectors in nonprofit work.

Logic Models: Getting Them Right and Using Them Well
The presentation and materials from this recent webinar (8/17/2016) was created by EvaluATE, the evaluation resource center for the National Science Foundation’s Advanced Technological Education program. The presentation provides an overview, plus real world examples and advice on where to add a logic model to a NSF grant proposal. 

The Logic Model Guidebook 
Written by Lisa Wyatt Knowlton and Cynthia C. Phillips this is a helpful and clearly written book for anyone interested in logic models. The book offers examples, checklists, and exercises to help you and your team develop and think through logic models. 

Share Your Work 

If your program or project team feels comfortable, share your logic models with the rest of the  field, and let’s learn from each other. You can submit your logic model to become a part of the growing number of resources available on InformalScience.org by clicking on the orange “Share Your Work” button at the top of the page, or by emailing Grace Troxel, CAISE Digital Librarian, at GTroxel@astc.org.