What is Output-itis? I clearly made it up. I haven’t been to dictionary.com, but I’m pretty sure you won’t find it there. But like Stephen Colbert’s “truthiness” it may be a useful addition to the language – or not.
Ouput-itis is the urge to deliver products or programs with no clear outcomes or impacts in mind. We brainstorm, design and deliver media of all kinds, personal and non-personal, without any specific expectation of results. We hope it does good and assure supervisors it will and has in the past. This is how interpretation has been done for decades.
Nowadays most organizations want RESULTS, measurable RESULTS. Many foundations require a Logic Model of measurable results in the application for a foundation grant. They have limited funds (especially after the recent stock market slide into the magma) and they want to know what RESULTS they’ll get for their investment.
As we developed the professional certification courses at NAI, we embraced the idea that interpretation is purposeful – it’s management. We encouraged planning with outcomes in mind, but our terminology in the early years was not consistent with other terminology about results-oriented programming and management.
Conversations with Dr. Tom Marcinkowski of Florida Institute of Technology and NAI’s own Jim Covel led us to the emerging use of logic models. In 2004 Tom wrote a monograph on the use of a logic model to review and analyze an environmental education program and it’s available from the North American Association for Environmental Education (NAAEE). Jim told us of the importance placed on logic models by the David and Lucile Packard Foundation. The Kellogg Foundation helps teach about Logic Models through their website.
Logic models vary some so we adopted the language being used in the environmental education community for three kinds of objectives – outputs, outcomes and impacts. Output objectives are what we do, like “conduct 4 new programs each month, complete three new exhibits, etc.” Outcomes demonstrate changed behaviors by our audiences such as ” ten guests will become volunteers, book sales on the Civil War will increase 20%, guests will donate $,1000 per month to our scholarships for children, etc.” Impacts are the changes for the organization or the resource such as “conservation easements will increase by 5,000 acres, vandalism of the historic site will decrease by 50% per year, etc.). These objectives have to be very measurable and something we care about.
It’s tempting to measure knowledge or cognition in outcome objectives, “The guest will know the three major landforms in the valley.” Since people tend to not remember facts from interpretive experiences, why would we measure their recollection of them unless it has some importance to the management of the site. “The guest will be able to distinguish brook trout from rainbow trout.” This measure of knowledge is useful to the guest and management if its legal to take home the rainbow trout to eat but not the brook trout. Cognitive objectives can be useful outcomes but usually they are not.
You can also think of these tiered objectives as “if then” statements. If we do our “sea turtles can’t digest plastic bags program” four times a month (output), then ten volunteers will stay after each program and help clean up the turtle beach (outcome), and that results in no dead sea turtles on this beach this year (impact).
The use of logic models in interpretive planning adds a whole new dimension to it. We identify evaluation of our interpretive experiences within the plan and then manage the programming and/or products to get the RESULTS we want.
Output-itis is easy and that’s the problem. It’s too easy to deliver interpretive experiences with no accountability to anyone. But then comes the economic crunch and our programs go away because they lack value for management. Logic models used well are a cure for output-itis and they include their own rewards. It’s comforting to know that what we planned has delivered desirable outcomes and impacts for the resource and organization. It’s great to get the RESULTS we wanted.
- Tim Merriman
P.S. Lisa Brochu’s new BLOG on Interpretive Planning has begun at http://www.interpretiveplans.com. Check it out!
3 Responses
July 10th, 2009 at 12:30 pm
Thanks for your interesting contribution, it is very clear, enlightening and useful.
However I have a comment: Once I had to do an interpretive planning in Chile, and I was asked to adjust to a logical model (“logical matrix”). But we (the team) could not adjust because we could not know the results BEFORE planning. We propose as a solution the design of some pilot products (signals).
How can we solve this? How can we say what the results will be without analyzing the real possibilities during the planning process?
Greetings from Spain,
Jorge
July 10th, 2009 at 2:46 pm
Gracias, Jorge, for a challenging question. I immediately asked Lisa Brochu to collaborate on this response for she dealt with these kinds of planning challenges many times in her 30 years as a planner/consultant.
Logic models are both a planning and evaluation tool. At the front end of the planning process you look at the organization mission and goals and talk about the RESULTS the client or manager wants, what you hope to achieve by undertaking a planning process and the projects that result.
You still can use analysis tools during the plan to study the options you have but identifying the IMPACT or RESULTS you want is a way of making the product and programming choices the right ones. If your in-stream analysis tells you that your desired results are inappropriate in any way, you can always tweak your objectives during the process. Having identified desirable results allows you to think about what types of products will help you achieve those specific results as you plan and ensure that your objectives are reasonable. It also allows you to check every part of your plan against the test of whether or not what you’re planning will get you further down the road towards achieving those results. Putting up pilot products to test your ideas is a great thing to do during the planning and design process, because you can then see if you are getting the OUTCOMES (reactions) that you hope for before spending the entire budget on production.
There’s an old saying, “If you don’t know where you’re going, any road will get you there.” Identifying the results in the interpretive plan as part of a logic model allows you to use the logic model as a planning tool, an evaluation or testing tool during the process, and after the plan is put into effect. If the desired RESULT or IMPACT is not being realized, then you must modify the OUTPUTS until you begin getting the desired OUTCOMES that will get the desired RESULTS (or change your objective – maybe the result you desired wasn’t reasonable). This all assumes that there is a manager or governance group who expects some VALUE or RESULTS from the interpretive efforts. Too often we don’t work long enough with clients/managers to get a clear idea of what they will consider success.
Forgive a long answer, but it’s an important question. Thanks for commenting, Jorge.
Tim Merriman & Lisa Brochu
July 10th, 2009 at 7:38 pm
Great timing Tim – I am offering a concurrent session (if it’s accepted) at the NAI Region 10 Workshop this October in the Columbia Gorge (Washington/Oregon) on using Logic Models. I find them very helpful in organizing information in one location to prepare for planning or for an evaluation process. A participant at another session I offered also mentioned using Logic Models to organize information for grant applications
Leave a Comment