Outcomes and Assessment?

In developing a presentation on out-of-school outcomes and assessment, I was amazed at the number of tools out there to measure program quality or lead to other management outcomes. These tools are too numerous to get into an in-depth discussion here, but the larger issue raised is a good one: If we are measuring, who has the authority to determine what is measured? Another follow up question may be, is that authority final, or subject to change in two, three, four years and every two or more years after?

Damned Lies and Statistics by Joel Best is a well-known work poking a few holes in our use of numbers in shaping social programs or solving societal issues. The first case tackled in the book is that of a statistic used in 1994 that claimed that every year the number of "American children who have been gunned down has doubled." Taking aside that "children" by 1994 had expanded to include 21 years old (today "youth violence" seems to include 27 years old) the number of children "gunned down" would have by that year been 35 trillion since the source of that "fact" was from 1950 and even if one child had been gunned down, that number would have grown exponentially.

So, in our popular media we cannot trust 90% of the statistics %100 of the time and for many of us, this is not a new part of life. What is related, however, to our work and the use of numbers, is how were the numbers gathered in the first place and what tools shaped which outcomes or "facts" are elevated? These assessments and the outcomes they highlight are not just exercises, they are increasingly seen as a way to merit (read fund) a program. Take for example the 21st Century Learning Center process. Programs rushed to the initial funding, altered how they structured their programs and even how they interacted with children, and then when the reapplication process came due, how many met the criteria of "exemplary"? Fewer than the initial cohort of sites, and of those achieving "exemplary" status, it is no secret that funding ends in 2010 and that this program is perhaps a ghost of the past administration.

And, these 21st Century Learning Centers were but one example of a move for increased "outcomes" and more "standards" that applied across communities and programmatic forms. Who created those assessments, and what do they hold up as examples of a quality program. It appears that in this current "free market" of ideas, we still have competing interests and attitudes vying for the Nihil Obstat of funders, the Out-of-School field, and the formal educational community that impacts and shapes an increasing amount of childrens' time.

In researching the topic of outcomes and assessment in Out-of-School Time for a future BOSTnet Roundtable, there are many questions that are being raised. Do these current tools lead to quality? What are we measuring as "quality" and does this need to reflect individual programmatic goals or are there indeed larger "systems" in place?

We will see and in the mean time, caveat emptor.

Is it time for an Economic Youth Development Summit?

In recent years the Out-of-School field has grown to include diverse programming from urban after-school programs for school age children to theater, arts, and community service for all ages. However, fragmentation of funding, a host of competing forces to standardize diverse programmatic approaches and a push to formalize a work force that has traditionally been permeable and reliant on local talent all threaten the existence of an increasing number of programs that work with children and youth but do so in different ways that the established school system.

This shift - currently compounded by the economic climate - has created a high-level of stress in a field that is also experiencing an identity problem. Recent cuts to state budgets has strained public funding and private foundations are seeing their endowments shrink as more nonprofits turn to them for investments. OST programs are asked to form partnerships that may not honor their particular approach to youth development. They also do not ensure the sustainability of community-based organizations as increasing attention is given to school-based solutions and a "pipeline" mentality to education and learning.

The Out-of-School Time field is increasingly asked to adopt new school-based models as it is increasingly framed against the needs of the school system rather than looking at the potential of using these diverse programs to address social needs that are today and have traditionally been outside the keen of the governmental school system. There is a wider issue of youth development that may need to supersede the uncertain identity of "after-school" and "out-of-school." Many practitioners are looking toward new research on the importance of youth development and a more integrated "whole child" approach but are locked into language that places them in competition with or in a very unequal relationship with the current educational system.

BOSTnet is proposing an Economics of Youth Development Summit to bring together the diverse field as it stands today and look ahead to how these programs and organizations can survive in the future. The field of Youth Development must create dialogue as well as lead to new ways of elevating the work many organizations have developed after a century of practice.

Perhaps the outcome of this could be a stronger platform and an energized support network that honors social-emotional work, informal learning, and community development.

Doing More With Less?

Our programs are increasingly asked to do more with less. Again, after the Governor of MA announced the "9C cuts" (post budget cuts that are part of the state office's executive privilege), already lean programs are asked to "be creative." This is a strange request - but then, what have we come to take as normal? We put bumper stickers on our cars that exclaim, imagination a world where education has limitless funding and the Air Force has to have a bake sale to buy an F15! We tell our friends we wish for that to happen - but then we snicker people who try to make that happen saying they don't know how the world works. Perhaps deep down we don't want that to happen. We seem to ensure that for every 42.3 cents that go to the military that only 4.4 cents go to education, training, and social services not just one year, but each year. Do we not control over our priorities? Are these things the way that are because people actually want them that way? Perhaps we need to spend a moment and really reflect on the world that we really want not by our words alone, but by our actions and those of our extended networks. With our programs vying for those 4 cents (shared by schools, training, and social services, what are our proper investments and how does OST play a role in making those investments rather than responding time and again to cuts?

Investing in Out-of-School programs is usually cast as charity money - that old worn social services mantel that the inner cities are limitless slums and the people within them desperate and only able to provide their children with chaos and uncertainty for which OST programs are solutions to crisis after crisis. We need to rethink OST's role as neither an add-on to school nor only about intervention in cycles of abuse and poverty, but as investment in local economies. This investment is for today, and not for a distant future. This investment produces meaningful employment for young adults and integrated learning for children, the kind found less and less in Public Schools and increasingly the privilege of a private (voucher, charter, parochial, home) school.

"Children are our future" is a cliche we traditionally sell for investment in these programs. Today, we need less "future" outcomes and more delivered to our economy right now through meaningful jobs for our young adults, community foundations for working families, and quality learning environments where children get the privilege of get to have a childhood that is not tested or drilled.

Out of School Time programs can provide:
Meaningful employment for young adults
Learning experiences that move beyond homework and synthetic seamless days
Serving as micro economic redevelopment of a block or street
Making a low income area more attractive to families
Responding to community needs
Being able to influence school and community relationships

These are some potential ways programs can position themselves is they so choose to. The 9C cuts may be only the beginning of our economic hardships. The OST field must look now and really consider, what world do we want to belong to? We are already being asked by our policy makers to be more creative and true, many programs can survive or limp along using tried and true methods of funding and description of services. However, are we taking some of our own advice and "reaching for the stars"?

Perhaps it is time we answered that call with some actual creativity.

New Directions in Youth Policy

At the invitation of PPV, BOSTnet was able to attend the New Directions in Youth Policy event October 7th, 2008 in New York City. Held at the Ford Foundation (supporter of PPV), a large building reminiscent of the now-distant era of Cold War philanthropy, the agenda was dominated by the discussion of the economy. Not as you would expect the economy at hand - the financial meltdown that is occurring now and which may last for years - but the economy of youth as workers. This loss in employed youth is not a new trend, the result of video games and youtube, but a hollowing out of our economy over the past thirty years. It was proposed that young people who work at 17 are more likely to work at 18 and so on. They develop habits of mind as well as need to work in situations that get them in contact with people different from themselves - and the back room of McDonald's does not offer that socio-economic as well as skill set diversity.

Andrew Sum, Director of the Center for Labor Studies at Northeastern University and a professor at the same institution painted a bleak picture of the landscape of young people in this country. According to the data on youth, young people are struggling to achieve economic and social stability. They are failing to sustain long-term personal relationships beneficial to the economic well being of children. They are failing to learn common job skills. Life expectancy and fertility are increasingly correlated to income level in ways that haven't been since in this country.

Andrew Sum proposed that this was the result of several administrations not taking youth employment seriously and allowing the industrial economy to be replaced with, what Wilbur Toss, an older businessman interviewed on NPR's Marketplace recently said out service economy was a sham. That we cannot build the sort of level of prosperity we are accustomed to by "flipping burgers, selling scraps of papers, or suing each other." This loss of employment with low barriers to entry (not requiring degrees and certifications) that connected young people to a world of work was not the inevitability of "globalization" but a lack of policies that protected our workforce and nurtured our native economy.

This was also not an issue with recessions apparently, since the only time where the numbers of employed youth increased was for a few years during the late 1990s. Youth in the labor market have been facing trouble whether the economy is robust or recession and the past eight years have been dismal. One issue is a loss of job usually held by young people to undocumented workers. Another issue is that adult workers are increasingly needing to fill jobs once held by teens. (Not spoken about but of note is the expansion of elderly in the workforce - such as at Walmarts and the like). For teens and young adults who still want to work, few of these jobs expose young workers to skilled people, train them in marketable skills, or pay them a wage they can live on. This, Andrew Sum argues, has led to an across the board decline in living standard that is not left behind when this generation grows up but leads to a lifetime of underemployment of unemployment. Sum noted a rise in unstable single parent homes, increasing numbers of children born to proportional to the lack of income, and lower life expectancies - especially for native-born minorities.

The presentation was compelling, as it was dismal news, especially to a room of people who had worked in philanthropy for thirty years or more. However, within this great problem there is great opportunity for Out-of-School programs. More than reaching out to teach more children in need and to compensate for the shortcomings of young parents unable to provide for their off spring, Out-of-School programs can serve as a place of meaningful employment for young people and job creation so that these young workers can learn meaningful skills as they provide needed labor to programs that cannot afford to pay high wages and yet need qualified and quality people.

What are the costs of Out-of-School programs? What are some ways to maximize their benefit to communities, especially low income communities? There is usually a coordinator/director and then "direct-service" or line staff. Those Direct Service staff are paid perhaps $8.25 - $11 per hour and in some areas or programs as much as $20 per hour. The major and constant issue with staffing is employee turn over and (at least prior to the melt down) many programs have constant trouble attracting staff at all. Staff come in, work a few months and leave. In New York State these after-school employees were considered "migrant workers" the same as farm help. Very telling. This is a labor issue, and an issue of compensation but also an issue of who is the appropriate workforce. If compensation is raised too much, it may put the programs out of business since overhead will outstrip resources. If a constant drum beat is on professionalization with its certificates, degrees, and career ladders, will that actually prevent the creation of a viable workforce? Young people need meaningful employment. Out-of-School programs need low-cost quality staff for direct service. Looking at a particular segment of the population as these workers would allow for more refined approach to training and development.

Out-of-School programs, if intentionally done, can learn to market to that workforce outreach, training and technical assistance, and provide dual services - one to children and youth development, the other to economic development of communities and the youth who need to learn how to work as they learn how to do that work. (The US military looks towards a particular age group and skill level, why not OST?). Youth learn marketable skills working at an out-of-school program they may not folding shirts or waiting for the buzzer of the fry-o-later to sound. They learn critical thinking skills, problem solving, and perhaps project management if their program has a project-based learning focus. Many young people may themselves want to go into youth work as a career. Many will see adults with skills they can learn from. Trainings are no longer cattle calls trying to reach out to diverse skills and competencies (how many OST trainings include teachers of 10 years and 16 year old high school students?) but can focus on a certain level and develop and refine a language to speak to that level. Staff turn over need not be seen as a bad thing, if the staff last for the academic year. In this way, the turn over is build into the system rather than fought against. A battle that cannot be won by increasing moral, raising a low wage a few cents, printed certificates, or top heavy college degree programs.

Perhaps after-school and OST programs can serve as that job that made a difference as the programs are to make a difference to children. We need to use this crises in youth employment and see the opportunities OST can provide. Perhaps rather than another set of competencies we need come up with who we see actually doing this work now, and in the years to come.