Outcomes and Assessment?

In developing a presentation on out-of-school outcomes and assessment, I was amazed at the number of tools out there to measure program quality or lead to other management outcomes. These tools are too numerous to get into an in-depth discussion here, but the larger issue raised is a good one: If we are measuring, who has the authority to determine what is measured? Another follow up question may be, is that authority final, or subject to change in two, three, four years and every two or more years after?

Damned Lies and Statistics by Joel Best is a well-known work poking a few holes in our use of numbers in shaping social programs or solving societal issues. The first case tackled in the book is that of a statistic used in 1994 that claimed that every year the number of "American children who have been gunned down has doubled." Taking aside that "children" by 1994 had expanded to include 21 years old (today "youth violence" seems to include 27 years old) the number of children "gunned down" would have by that year been 35 trillion since the source of that "fact" was from 1950 and even if one child had been gunned down, that number would have grown exponentially.

So, in our popular media we cannot trust 90% of the statistics %100 of the time and for many of us, this is not a new part of life. What is related, however, to our work and the use of numbers, is how were the numbers gathered in the first place and what tools shaped which outcomes or "facts" are elevated? These assessments and the outcomes they highlight are not just exercises, they are increasingly seen as a way to merit (read fund) a program. Take for example the 21st Century Learning Center process. Programs rushed to the initial funding, altered how they structured their programs and even how they interacted with children, and then when the reapplication process came due, how many met the criteria of "exemplary"? Fewer than the initial cohort of sites, and of those achieving "exemplary" status, it is no secret that funding ends in 2010 and that this program is perhaps a ghost of the past administration.

And, these 21st Century Learning Centers were but one example of a move for increased "outcomes" and more "standards" that applied across communities and programmatic forms. Who created those assessments, and what do they hold up as examples of a quality program. It appears that in this current "free market" of ideas, we still have competing interests and attitudes vying for the Nihil Obstat of funders, the Out-of-School field, and the formal educational community that impacts and shapes an increasing amount of childrens' time.

In researching the topic of outcomes and assessment in Out-of-School Time for a future BOSTnet Roundtable, there are many questions that are being raised. Do these current tools lead to quality? What are we measuring as "quality" and does this need to reflect individual programmatic goals or are there indeed larger "systems" in place?

We will see and in the mean time, caveat emptor.