115-200 billion tons moving daily. Not tons of data, but water – in and out of the Bay of Fundy (left) alternating up to 50 ft between low and high tide. Just in case we’re tempted to take the concept of “rising tide” lightly, it’s good to revisit the roots of metaphors with which we may have a bit too much familiarity and appreciate their immenseness. In this case, Michele Jolin, Managing Partner of America Achieves, tracks the rising tide of evidence-based solutions in the social sector and the political alignment happening to drive it on a meaningful scale. Talk about syzygy! (We won’t.That’s pushing the tide thing a bit too far. But,um, we could.)

…

Recently, Results for America joined with The Hamilton Project to host a wide-ranging discussion on the importance of using evidence and data to inform policy and funding decisions. We brought together those who are doing the hard work in communities with those who are in charge of crafting policies at the local and federal levels. At the event, leaders from both parties, and from both ends of Pennsylvania Avenue, called for greater use of evidence in crafting social policies and improving outcomes for young people, families and their communities.

Too often, issues are divided by politics, but during our discussions in April, partisanship gave way to agreement on the importance of working together to confront the challenges facing our country.

Over the next decade, America will face enormous social and economic shifts driven by global competition, an aging population and budget constraints at all levels of government. Young people will be vulnerable to these pressures at a time when we need to be preparing them for lifelong success.

Now more than ever, government funds need to be invested in a way that will produce better outcomes for young people and their families. At Results for America, an initiative of America Achieves, we are committed to directing public resources toward solutions that harness evidence and data to continuously improve and expand the impact of those programs that work in communities across the country.

By collecting data, continuously evaluating programs and directing government funds toward those that are succeeding and away from those that are falling short, we can achieve better results and can get a better return on taxpayer investment. If we can make tax dollars go farther, and have greater impact in communities nationwide, we can begin to restore trust in a government that has proven to be less efficient than the workforce it represents. Something Alan Krueger, the Chair of President Obama’s Council of Economic Advisers, highlighted at our recent event.

President Obama’s Fiscal Year 2014 budget request, released in early April, takes significant steps toward that goal, building on progress he made during his first term. His budget includes an unprecedented focus on improving access to data and increasing the use of evidence in making funding decisions. He expands federal innovation funds and Pay for Success initiatives, increases resources committed to fund evaluations, takes first steps toward using evidence in directing formula grant dollars and begins to create “what works” clearinghouses in federal agencies. The President’s budget also boosts some of the most successful results-based programs created to date to evaluate new ideas and scale up the most effective approaches for improving schools, student achievement and job training, to name just a few. His budget proves that he meant what he said during his 2013 State of the Union address, to “do what works” and “make sure none of our children start the race of life already behind.”

Other opinion leaders and public officials – including New York Times columnist David Brooks, Harvard economist Jeff Liebman, U.S. Senators Rob Portman (R-OH), Mark Warner (D-VA) Mary Landrieu (D-LA) – have touted the demonstrated success and future promise of results-oriented programs. And, thanks to an amendment by Senators Ron Wyden (D-OR) and Debbie Stabenow (D-MI), the Senate Democratic budget calls for greater use of data and evaluation in making funding decisions.

In cities and states across the country, mayors and governors are already adopting an evidence-based approach and using data to improve their communities. The private sector has also stepped up. In order to really improve outcomes, our government cannot be afraid to partner with companies, foundations and communities on what works.

At our recent event, Senator Portman said that government decisions need to be informed by more evidence. He said what many of us know to be true: that in this time of limited budget resources, we have to make sure every dollar is spent as efficiently and effectively as possible, because we cannot afford to spend federal dollars in ways that are ineffective.

The next step will be to build on this growing momentum. Communities large and small are having tremendous success implementing innovative solutions that are improving lives every day. We need to harness their spirit and encourage our lawmakers to drive more public dollars to what works for young people and their families.

I think discussions such as those in the clips here below (excerpted from the recent Global Philanthropy Forum) not only keep our journey human, first and foremost (not getting lost in the technical), but also help us orient ourselves to discover new questions and angles of thought along the way. Many thanks to Sasha Dichter, Chief Innovation Officer, Acumen; Jacob Harold, President and Chief Executive Officer, Guidestar.

Thanks,

Eric

…

Today, President Obama issued an Executive Order — Making Open and Machine Readable the New Default for Government Information

Our take at Markets For Good? It was big news when the President’s FY 2014 Budget was released with a proposal for expanding mandatory e-filing to all tax-exempt organizations. With this Executive Order, it’s great to see the pace continuing en route to fully open data. The importance is not only the accessibility of data, but, rather what we are increasingly being enabled to DO. The sum of all of these new data sets is a living platform for deeper analysis, experimentation and discovery as we attempt to solve tough social problems.

…

More information can be found on these links.

http://www.whitehouse.gov/blog/2013/05/09/landmark-steps-liberate-open-data

http://www.whitehouse.gov/photos-and-video/video/2013/05/09/clock-open-data-executive-order

http://www.whitehouse.gov/sites/default/files/omb/memoranda/2013/m-13-13.pdf

http://www.whitehouse.gov/the-press-office/2013/05/09/executive-order-making-open-and-machine-readable-new-default-government-

Yes. That’s a corn field… but, you could call it a dugout for our purposes today. Daniel Stid, of Bridgespan Group, takes this Field of Dreams cue to ask a few tough questions to you, and to us at Markets For Good. We’re always excited to dig in to real discussion and debate to sort through conversation for ways forward.  There are a lot of ideas buzzing about data that look mighty good on whiteboards and sound great spoken from your wireless mic. And, beyond that (we’re not pessimists), there’s a lot of actual “doing” in the sector as organizations are finding ways to make data work for them and their missions. But will these ideas and work change people’s lives? That’s why we’re here, not “ultimately,” but at the outset. Daniel Stid asks what happens after we build this upgraded information infrastructure to clue us in on how to think – and how to LEAD -  while we work to that end.

…

In its initial concept paper, Markets for Good has put forward an inspiring vision, one of “a social sector powered by information, where capital flows efficiently to the organizations that are having the greatest impact, programs and interventions are more effective and responsive, beneficiaries have a voice, and there is a dynamic culture of continuous learning, development, and innovation.”

Markets for Good proposes that the key to realizing this vision is building out the sector’s information infrastructure, i.e., “the architecture that can help to connect, organize, and structure information so that it can be supplied and used more easily.”

As we undertake this construction, however, let’s ask ourselves whether, if we build it, the putative buyers and sellers in the envisioned marketplace—the philanthropists and nonprofits spending and soliciting money within it—will use it as planned. In short, will better information change their behavior?

A premise of Markets for Good is that we don’t really have such markets yet because the available data is “limited, unstructured, and unused.” But as Friedrich A. Hayek noted in “The Use of Knowledge in Society,” it is precisely in the dim light of these circumstances that markets serve as the best means for allocating scarce resources and in which they flourish. The social sector’s challenge may not be to establish market dynamics, but to temper those already at work.

On Giving and Data

Consider the motives and behaviors of individual philanthropists in this market. Indeed, take out your tax return or checkbook and assess what prompted your own contributions in 2012. Was evidence of impact always the driving concern? Speaking for our household, the other motives included maintaining ties to alma maters, fulfilling religious obligations (and thereby avoiding guilt!), supporting leaders we admire, and giving to causes at the request of friends as they have done in response to our requests. Was your charitable giving prompted by a similar range of motives? How much of it would be influenced by better evidence of impact?

What about the presumably more rational grant-making of institutional philanthropy? I’d like to think that foundations recognize the importance of evidence and thus support promising grantees in establishing the data systems, infrastructure, and evaluations they need to assemble it. I’d also like to think that foundations relentlessly seek out evidence to test their own theories of change and refine them when they encounter countervailing data. In my experience, though, foundations that behave in this way are exceptional relative to a somewhat dismal norm.

Defending Our Positions

The motives and behaviors of nonprofits seeking funding may be just as impervious to better information as those providing it, albeit for different reasons. We shouldn’t be surprised when nonprofits seek to share information about their work in ways that maximize the contributions they receive. Nor should we be surprised when funders’ external demands for evidence of impact leads nonprofits to assume a defensive stance in which they seek to present the best possible case for their performance. Alas, this defensiveness too often prevents organizations from doing the deeper work of measurement, reflection, and learning that could actually improve their results. Will the drive for more transparency pull nonprofits out of their defensive positions, or will it push them to hunker down even more?

In this same light, we could question whether nonprofits seeking funding will readily conform to more standardized definitions of their work and outcomes as envisioned by Markets for Good. More robust markets tend to accelerate the quest for defensible positioning and breed more differentiation, not less—as anyone who has stood bewildered in front of drug store shelves trying to choose among myriad brands, prices, sizes, and features of pain reliever or tooth paste can attest.

The Way Forward

All this is not to say that we shouldn’t improve the social sector’s information infrastructure. As we do so, though, let’s bear in mind that the information produced by it will be used by individuals and institutions driven by a wide range of motives, of which maximizing social impact is but one (and not always preeminent). Individuals and institutions will adapt and use this improved information (or not) to suit their own purposes. If we want more money to go to better solutions and continuous improvement in social impact, we will also continue to need sustained leadership and moral suasion geared toward these ends.

Yumi Sera, Operations Director for the Disability Rights Fund shares the blog post with Mariane Arsenault, an evaluation consultant for the Universalia Management Group, a Canadian consulting firm specializing in monitoring and evaluation, organizational assessment and strategic management for bilateral and multilateral agencies, private sector companies, and NGOs.  In addition to heading up the recent evaluation for the Disability Rights Fund, Mariane has worked on several other evaluation assignments focusing on international development interventions.

…

From the outset, the most important aim was that the M&E System be an extension of our commitment to a rights-based approach and aligned to the principles of the UN Convention on the Rights of Persons with Disabilities (CRPD).

DRF believes that enhancing the participation of persons with disabilities in the realization of their rights is essential to reducing poverty amongst persons with disabilities.

We understand that achieving across the board improved quality of life for persons with disabilities in the developing world is far in the future. DRF’s role is to make grants to change national and local frameworks and attitudes that affect the most marginalized groups in the disability community.

Despite the challenges of M&E in the advocacy realm, we knew that we required a system to collect reliable and valid data about the effects of our grants and to track our collective achievements, as well as to learn from the obstacles we face.

The M&E system we created includes tools and most critically, a logframe with measurable outcome and outputs and SMART indicators. Clear year-to-year milestones guide the work of our Program Officers who provide oversight to grantees through annual Grantee Convenings and frequent contact throughout the life of the projects. Country strategy development and annual assessments provide an in-depth analysis of trends, obstacles, and opportunities for movement building in advancing the rights outlined in the CRPD. Periodic evaluations allow us to reflect on achievements and gaps and change course, as needed.

In late 2012, we tested this system through an independent evaluation conducted by Universalia. This evaluation was one of the first to be commissioned by a grantmaker focusing on supporting disability rights advocacy in the developing world.

A number of challenges for the evaluation were to be expected given the nature of DRF’s work – a grantmaker making modest advocacy grants to disabled persons organizations across numerous developing countries. Data on disability in the countries where we work is scarce, and strong evaluations rely on the availability of quality data. Frameworks for measurement of advocacy achievements are new and yet untested. With DRF’s M&E system and logframe, the evaluation at least had a solid baseline and articulated objectives to measure results.

Traditional evaluation concepts — such as relevance, efficiency and effectiveness – were applied to evaluate hard-to-grasp concepts such as advocacy efforts and movement building. Applying these criteria for DRF’s participatory approach was not impossible, but required adaptation, reflection, and multiple approaches to data collection.

Along with a review of documents, the evaluators conducted focus group discussions with grantees in Uganda and Bangladesh and stakeholder interviews (approximately 90 individuals were consulted). Frameworks based on existing literature were also developed to assess the human rights based approach as well as advocacy and movement building efforts.

The evaluators went into this evaluation with the expectation that they would find very small-scale results, given the fact that many DRF grantees are very small and marginalized grassroots organizations. However, they were surprised by the magnitude of the results achieved.

We have treated the evaluation as an important learning experience – both for the Fund and for the evaluators – and are using the findings for learning conversations among key stakeholders. Here, we share 10 learnings coming out of the experience:

Stick to your principles and values

Being a mission-driven organization, we strive in all that we do to align our work with the principles of the Convention on the Rights of Persons with Disabilities. An evaluation should be aligned with your values.

Seek to empower the people you are working with

We believe that meaningful change will happen only when persons with disabilities are empowered to take decisions that affect their own lives. Thus, it is crucial to involve and listen to the people your intervention seeks to benefit.

Commit your organization to robust monitoring and evaluation

Our governing body, global advisors, and staff were involved in and committed to measuring the results of our work and learning from the process. Creating an organizational culture of learning can help you and your evaluators tread into new territory.

Be accountable to your donors and to the public

The evaluation helped us to report our progress to our donors and to the public. To be accountable for public funds, you should have an independent entity validate assumptions and interventions and point out new areas to explore.

Engage stakeholders — learn from them and educate them

We network with stakeholders in global fora and meetings. Collaborating with others creates a rich environment for movement building that is essential to work for social change. Sharing lessons helps build a larger and stronger movement.

Adapt and refine monitoring and evaluation tools

To some, traditional data collection and measurement may seem too linear or quantitative for the complexity of the social sector, but by articulating a theory of change and logframe, we have strengthened our tactics. Combining quantitative measurements with stories can be a powerful portrayal of your mission.

Be rigorous in your analysis

Collecting data about each grant seems like a micro-managed process, but aggregating and analyzing data illustrate how we are (or are not) having an impact. Developing a shared meaning of concepts and terms across your organization can help you to standardize your monitoring and reporting, especially across diverse populations and geographic areas.

Review and evaluate processes, in addition to results

One of our biggest learnings for our evaluation was that the process, especially in advocacy work, is as important as the intended results. How you succeed or fail can be more of a learning than the attainment of a goal. A theory of change helps articulate and make these processes visible.

Apply the learning to your work

Whether or not we agree with the findings, we know that the evaluation has contributed to a healthy dialogue in our organization about our work. You can use the opportunity of an evaluation to test and question your assumptions and become a better organization.

Contribute to the field of social change

Sharing what we are doing with a wider audience can raise awareness of the rights of persons with disabilities. We also hope that our approach to monitoring and evaluation will contribute to the exchange of ideas about evaluation in the social sector and among grantmakers.

See http://www.disabilityrightsfund.org/evaluation for our evaluation and our recent report on our first years, “One in Seven” http://www.disabilityrightsfund.org/oneinseven.

…

Many of you were with us for the Live Twitter Q&A. Others have followed our announcements and conversations on Markets For Good and Twitter (#MFGChallenge) over the past few weeks. We’ve been listening closely to your questions and answering them along the route. As a summary, we’re glad to send along these two FAQ resources to help guide you to the submission finish line.

…

[1]

Review the General Guidelines and “Grand Challenges | Explorations” FAQs here: GCE FAQ’s

[2]

More FAQs, Culled From Our Conversations With You

Q:  What about a project which is started and funded partially by other grant? Can we apply?

A: You can apply with a project in process –  proofs of concept, pilots, prototypes, and  full solutions welcome.

Q: Is the Challenge limited to health-related projects? (I noticed the header “Grand Challenges in Global Health”on this page: http://www.grandchallenges.org/Explorations/Topics/Pages/SocialDataInteroperability_Round11.aspx)

A: No. Your data interoperability project submission is not limited to health-related projects. On that same page, in the section labeled “Opportunity”, note that the Challenge is open for “solutions that span issue areas (e.g. education, health, etc.) and/or geographies to the widest extent possible.” Grand Challenges | Explorations was begun as part of the Grand Challenges in Global Health efforts, but it has been and is now being used to solicit ideas for a wider range of ideas beyond health – including data interoperability for the social sector.

Q: Do we need to specify exactly what we are going to achieve in the $100K in a separate budget sheet?

A: The application asks for more information on how the funds will be used, but does not request a separate budget sheet.

Q: Another funder will give us funds if we are selected.  Should we mention this in the application?

A:  Applications can include information about additional sources of funding.

Q:  The overall Rules & Guidelines note: “Applications are sent to reviewers without personal or organizational information. Do not include this information in the body of your proposal. Proposals that include personal or organizational information in the body of the proposal are at risk of being automatically removed from the review.” The separate document detailing the “Increasing Interoperability of Social Good Data” challenge says “Successful Proposals Will Include…Explanation of why the organization or group submitting the proposal has the capacity to achieve success.” 

How do I describe capacity without including identifying information?

A: Applicants should describe the capacity of their organizations – strengths, etc. But do not include any specific, identifying information. Do not provide the name of your organization or institution or your individual name.  Applicants can describe their services and programs, share impact data (and/or constituent feedback!), revenue model and sources, staff and management teams, organizational history, governance, etc.

Q:  Can I name my partners, intended partners, hoped for customers/clients, etc?

A: According to the Rule (above) regarding exclusion of personal or organizational information, please do not name official partners in your application.  You can describe the type of organization your partner/partners are.

Q:  Do you encourage early-stage startups (in alpha or beta phase) to apply?

A: Start-ups are welcome.

Thank you!

 Dr. Denise Raquel Dunning, Program Director of the Adolescent Girls’ Advocacy & Leadership Initiative (AGALI) at the Public Health Institute, advocates for a new evaluation model to maximize the impact of social sector initiatives.

…

It is clear that evaluation is a valuable tool to assess the impact of social sector initiatives. Less apparent but equally important is the correlate – evaluation directly influences social sector impact. And not just in the ways that we often think – where strong evaluations result in replication, expansion, and increased funding for programs that demonstrate results, and weak evaluations lead us to adapt our models or try new ones.

Beyond the obvious ways in which evaluation findings shape the social sector, the very process of evaluation also has the potential to strengthen and sharpen our impact, if we use it wisely. Contrary to widely-held views, evaluation is not a neutral force that objectively assesses impact without influencing the initiative in question. Quite the opposite, evaluation is one of the social sector’s most important forms of intervention. Through interviews, surveys, and measurement of social sector initiatives, we are changing the very people, communities, and landscape we seek to ‘objectively’ evaluate.

Take, for example, the external evaluation commissioned by the UN Foundation and the Public Health Institute to examine the results and lessons learned from the Adolescent Girls’ Advocacy & Leadership Initiative (AGALI). Since 2009, AGALI has improved adolescent girls’ rights, health, education, and livelihoods in Africa and Latin America by empowering leaders and organizations to advocate for girl-friendly laws, policies, programs, and funding. The impact evaluation examines AGALI’s national advocacy and policy results, explores the differences AGALI has made in the lives of adolescent girls, and analyzes how the AGALI model has built the advocacy capacity of leaders, organizations, and networks.

While the principal focus of the AGALI evaluation is to examine the program’s impacts, we are realizing that the actual process of evaluation is reinforcing and strengthening the very results we set out to measure. In analyzing AGALI’s national policy outcomes, the evaluation process is amplifying the program’s original advocacy results.

For example, the evaluation is using contribution analysis to explore how Miguel Angel Lopez, a 2009 AGALI Fellow from Guatemala, advocated successfully for passage of a national policy to ensure specialized care and treatment for adolescent girl survivors of sexual violence. Contribution analysis uses a six-step process to assess cause and effect by developing and testing a theory of change. So in addition to interviewing Lopez and other national leaders who developed and advocated for the policy, the evaluation team will interview the political decision-makers who adopted the policy. During these interviews, the evaluation team will ask about the advocacy process, the need for the sexual violence policy, and its impact on the lives and health of adolescent girls.

Research demonstrates that participating in an evaluation process often leads interview subjects – in this case, the political decision-makers who passed Guatemala’s national sexual violence policy – to reflect on and reaffirm their initial commitment to the initiative in question. Therefore, this evaluation is not only a mechanism to assess the impacts of AGALI’s advocacy, but also an opportunity to deepen policymakers’ commitment to adolescent girls’ needs by ensuring the successful implementation, funding, and monitoring of the new sexual violence policy. For more details about AGALI’s advocacy in Guatemala, click here to read a full case study.

In Liberia, the evaluation uses the Most Significant Change methodology, an inductive approach to analyze AGALI’s direct impacts on adolescent girls and identify unexpected changes resulting from the program. The evaluation in Liberia focuses on AGALI Fellows’ successful advocacy for passage of a national Children’s Law that guarantees comprehensive protection for the rights, health, and education of adolescent girls, including outlawing child marriage and female genital mutilation. In addition to interviewing AGALI Fellows and political decision-makers, the evaluation team will interview adolescent girl leaders of the National Children’s Parliament who played a key role in the Children’s Law advocacy campaign.

As part of the process, the evaluation team will ask the young Parliamentarians to reflect on their experiences advocating for the needs of adolescent girls and becoming leaders in Liberia’s Children’s Law movement. Through that process of introspection, the young women interviewed will reflect on how their own burgeoning leadership was integral to the success of the national advocacy campaign. Yet again, evaluation serves not merely as a strategy to assess impact, but also provides an opportunity for learning, reflection, and growth for both the program and participants. For more details about AGALI’s advocacy in Liberia, click here to watch a three-minute video or click here to read a full case study.

Evaluation is an iterative feedback loop that can continually reinforce and deepen advancements in the social sector. Evaluation has the potential to enrich and reinforce our programmatic goals – but done badly, it may also erode important achievements and progress underway. To that end, we must not only structure evaluations to accurately capture impacts, but also to ensure that the process itself further strengthens and builds upon our existing outcomes.

Social sector investment in evaluation is crucial to ensuring that we maximize our impact – and yet, in a zero-sum funding environment, we often face the seemingly difficult trade-off of choosing whether to invest our limited dollars in interventions or evaluations. But only by recognizing this false dichotomy and understanding that evaluation is intervention will we develop initiatives that are as targeted, effective, and sustainable as they can be. Donors also play a key role in this conundrum – by incentivizing risk-taking, learning, and innovation through long-term investments that explicitly fund evaluation, foundations have the potential to transform the social sector’s resistance to evaluation.

What are we really asking for when we require nonprofits to produce data on performance, effectiveness and impact? While the surface logic is clear – we need to know this information – the full context and set of assumptions surrounding the request bears closer examination. Laura Quinn, founder of Idealware, breaks it down to reveal the barriers to generating good data and, further, calls out a few ways that the sector can and should support any request for more and better data.

…

Few would dispute that there is a lack of transparency in the nonprofit sector, but the blame for that transparency falls all too often—and far too easily—on nonprofits themselves. This refrain goes, “If only they would collect better data and better show their impact, it would be clear to funders and donors where to best spend their money.” While this type of thinking is hard to refute in theory, in practice it’s almost impossible to live up to.

To illustrate, let’s play through a hypothetical scenario: Let’s say you’re the data and program evaluation manager for a mid-sized human service nonprofit that provides counseling to victims of domestic violence in the large city of Springfield, with about 35 social workers in the field. It’s your job to help oversee the data systems, analyze data to identify how programs are going and how to improve them, report to funders and foundations on what they want to know, and to think strategically about how you’re measuring and evaluating in general.

With all the recent interest in data and measurement, you have substantial buy-in from your executive team to try to think strategically about you can best use data—after all, they hired you, and the very existence of your position speaks to their commitment. You also have the luxury of solid data systems that allow staff to enter data from any browser and see case data for their own clients, and that let you pull high level numbers and reports on a number of important metrics.

Sounds like you’re in good shape, right? Compared to a lot of nonprofits, you are—but you still have a lot to juggle. What are your biggest headaches likely to be?

• Data Quality. Your social workers are generally on board with the idea of systematically entering data, but they’re already overworked and underpaid—should they stop to enter data if it means putting a woman’s life at risk? Entering data sometimes falls off the bottom of their critical priorities list, leaving the data out-of-date. You’re thinking through options: Would giving them mobile devices to enter data from the field help—and can you find funding for that? How about simply being stricter about data being part of their job—would that help, or would it damage morale for critical client-facing staff? What about trying to find the budget to hire someone just to help with the data entry? There are no easy solutions.

• Providing Data to Funders. Let’s say your organization receives funding from two different state programs and three foundations, which is not at all unusual. There’s no standard set of metrics, so each foundation asks for its own, often requesting similar metrics with meaningful differences in definitions—so, for instance, one asks for detailed data on children vs. adults served and defines children as under 16, while another asks for similar data but defines children as dependents living in a parent’s household. What’s more, two funders ask for client-level data so they can do their own analysis. For one, you can download the appropriate data from your system and upload it to their system, but the other won’t accept an upload, so you need to one at a time copy and paste from your database all the data about the constituents you’ve served under their grant, field by field. (This may sound agonizing, but it’s not rare. A number of funders—especially government entities—require detailed data but don’t accept any form of upload or automatic data transfer, apparently expecting that nonprofits will not have any data systems of their own.)

• Meeting Changing Data Requirements. It’s complicated enough providing all the metrics funders want, but every year about a third of your funders change their data requirements. What’s more, you’re not likely to be reporting to all your funders at the same time each year, so several times per year you’ll need to update your reports, your processes, and maybe even your systems to account for new requirements

• Defining How Best to Measure for Improvement. A huge part of your job is making sure you have the right data to report to funders, but is that data actually useful to your organization? Does it help you understand what’s working and improve what isn’t? At best, funders are likely asking for a lot of disparate data, requiring some strategy to figure out how best to use it to improve your own programs. More likely, some of what would be truly useful to internal improvement requires additional reporting and analysis, so you need to make time to work with executive management to define precisely what should be measured and how, and to make that happen.

• Trying to Measure Impact. These days, everyone wants information on actual impact. Many people will tell you it’s not enough to know how many people you’re serving or what happened to them after you served them—it’s also critical to understand the long-term impact of your services on the community. There’s just one problem: This type of measurement generally requires extensive, university-level research—often with control groups, enormous budgets and large spans of time. If someone had already done research relevant to your services, you could use that to define your impact based on more easily gathered data, but unfortunately, nothing exists. (In fact, it’s rare to be working in a program area where this kind of research does exist). Funders don’t seem interested in funding this type of research for the good of all the organizations doing this type of work, but seem to expect your organization to be able to produce it on your own with your very limited data and evaluation budget.

• Fending off Bad Research. With so many demands for data that isn’t really “knowable,” it’s tempting to take on research projects that might appear to address them but don’t provide any real value to your organization. Which means you spend a lot of time trying to dissuade the powers-that-be from taking on foolish research projects that can’t possibly provide useful data on your limited budget.

• Proving Your Value. Even as you think through all this, you’re often called upon to prove that the money the funders are spending on you makes sense—after all, your salary isn’t directly going to help the enormous amount of women who need help, and who’s to say all your work isn’t just a waste of money? You’re asked on a monthly basis to show how you’re saving the organization money or helping with fundraising, and there’s always the lurking danger that the executive team will no longer prioritize data and evaluation and you’ll be out of work.

Not an easy job, right? Some might say it’s nearly impossible.

But for many, if not most, small to midsized nonprofits, the reality is even worse. Remember, this example assumed that you had the money and buy-in to get up and running with solid data systems, which is probably not an accurate assumption for the vast majority of nonprofits. It also assumed that there was actually a person in the organization able to put any strategic thought to using data effectively on top of all things needed just to keep the doors open—again, not a likely assumption.

The point of this hypothetical exercise is, primarily, to show that we can’t assume nonprofits have the resources to provide high quality data about their own effectiveness. While that might seem like an easy and obvious thing for them to be able to do, it’s not—not in the least.

How do we get them to a point where that’s possible? It would take more than just a little training or a second look at their priorities. They’d need sizable investments in a number of areas. They’d need help with technology, and to understand how to best make use of data and metrics on a limited budget. They’d need a rationalized set of metrics and indicators that they’re expected to report on, standardized as much as possible per sector with a standard way to provide them to those who need them.

Funders need to understand what is and isn’t feasible, and to redirect the focus of their desire for community impact evaluations from small nonprofits to the university and research world so the nonprofits they support can be unencumbered to work toward a better world.

We all need to understand that if we as a sector lean on nonprofits to provide data they simply don’t have the infrastructure to provide, what we’ll get is not better data—in fact, we may data that’s worse. Organizations pushed to provide impact data to get funding will provide something, but it’s not likely to be the high quality data or strategic metrics that would actually help them improve, or that would help the sector learn anything about the effectiveness of the services they provide.

These organizations rely on funders to help them meet their missions, but sometimes the burdens put on them by the reporting requirements that come with that funding can make it more difficult for them to carry out their work.

In this dispatch of Markets For Good | Notes From The Field, our guest contributor, Ambassador Curtis S. Chin, approaches the information infrastructure topic from a policy perspective. Using developing, high-growth economies as a backdrop, he focuses on three levers that business, governments and the social sector need to manage well for equitable economic growth. Ignoring them, he says, can undermine the efforts to drive both growth and “good,” together. Drawing from senior roles in government, business and civil society, Ambassador Chin issues a self-assessment challenge to each sector, starting witn a new conceptualization of “brics.”

…

Markets For Good focuses on how the social sector can better use and share information to improve outcomes and change lives. I’ve spent the past two decades in and out of many sectors, including many years in multi-national business and not quite four years as a diplomat and U.S. Treasury official in the Obama and Bush Administrations. As U.S. Ambassador to the Asian Development Bank (ADB), I sought to drive change at that international financial institution, which focuses on spurring economic growth and reducing poverty in the Asia and Pacific region through more than US$10 billion per annum in loans, grants and technical assistance to governments.

While the intertwined nature of business with economic policy and quality of life may be obvious to many, the implications for the more narrow topic of data and information infrastructure may appear less obvious. But they aren’t for me.

A critical and recurring mission of mine has been the broad advocacy for greater partnership across sectors and an explicit advocacy for high-quality, transparent flows of data and the sharing of the information in ways that break down bureaucratic barriers. The ultimate goal is to change economies and lives.

Building with “brics” (lower case)

In 2001, an economist at Goldman Sachs named Jim O’Neill was credited for coining the acronym BRIC, or “B-R-I-C,” for Brazil, Russia, India and China, in the report, “Building Better Global Economic BRICs.”

We know what happened after that: The BRIC acronym caught on and recently expanded, with South Africa formally taking its place as a member of the now five-nation grouping.

“Who’s next?” is a constant question I receive.

My response is often a little different from what people expect. I tell them that I spend much less time worrying about which nation will next join the BRIC, and I spend more time thinking about the challenges to growth posed by what I describe as a new lower-case “bric.” That is a small letter “b-r-i-c” for bureaucracy, regulation, interventionism and corruption.

(For now, we’ll leave aside the “s” I have added to the “bric” for increasing and re-emergent “sectarianism” as seen in recent years, whether in Iraq, Nepal, India or Burma – also known as Myanmar.)

Overcoming these constraints to growth is not easy anywhere, particularly given vested interests and “how things are done in our sector.” But one critical assist will be the ability of all sectors of society, including the non-governmental organization community, to work together and share information. Creative partnerships – whether on data collection or service delivery – among stakeholders who do not necessarily always see eye-to-eye need to be explored.

Yet, despite all the growing talk, including within and with the development community, of the importance of public-private partnerships, I find there often remains a general lack of understanding all around – business, government and the social sector – as to what motivates or incentivizes the other. This deficit, or “understanding gap,” suggests three areas for self-assessment.

Transparency

How transparent are you? Even something as simple as a lack of understanding of one another’s budget cycles and calendars can become a major stumbling block. How transparent is government in its dealings with others, including with other government departments? How transparent also are business and civil society? With due respect to confidential strategies, mission-sensitive intelligence and proprietary information, I find that data and analyses can indeed be shared more freely. Cost savings and greater effectiveness and impact can be among the benefits. Something as simple as posting information on a user-friendly website, whether contract awards or contact information, can be an easy and important step forward.

The World Bank’s “Doing Business” reports and Transparency International’s Corruption Perception Index all make clear the challenges of dealing with government, particularly in the developing world.

The extent to which each of us, regardless of sector, understands and works transparently with and within a bureaucracy, a business or a NGO in a given market will vastly improve one’s reputation in that and other geographies. Reputation travels.

During my time on the Board of Directors of the Asian Development Bank, it was an ongoing struggle to push that institution to be more transparent in areas ranging from public disclosure of all contract awards to allowing the voluntary release of formal statements delivered by members of the Board fully explaining their votes for or against a proposed loan, program or policy. The reality though is that one stakeholder’s concept of “transparency” may well be simply seen as more costs and more trouble by another.

Still, let’s not give up hope. Sure, many NGOs and development agencies still insist on their own institution-specific reporting requirements, despite efforts and accords to speed harmonization and sharing of data. Yet, progress is being made.

After a “speedy” two years’ process, I was able to help push through the first ever major partnership agreement between UNICEF and ADB. The Memorandum of Understanding (MOU) provided a framework for the two organizations to build on existing cooperation and engage in joint technical assistance, projects and program work in selected countries and sectors, to the benefit of disadvantaged children across the region. This was to include promotion of research and analysis and the development of “knowledge products” linked to their common activities. Now more than two years since the signing of that MOU, the question remains as to what has happened since then? Both organizations will want to transparently report back on progress to date.

You should also ask yourself a question: Are you clearly communicating how regulations and policies that you advocate for are to the benefit of people, whether on a direct level or with respect to a nation’s economy, its industry and its jobs?

Credibility

How credible are you? No individual sector has the monopoly on socially responsible behavior. Or on trust. Data – and widely spread anecdotes – tell of how both government and civil society representatives have acted in ways not necessarily in the interest of the populations that they are meant to serve. In some countries, government leaders are trusted less than business executives.

If you are from the private sector and seeking to partner with government, to what degree have you shown that you understand a government’s own challenges and its priorities for the near and mid-term? In many an emerging market, governments particularly in Asia, have taken to five-year plans, setting forth critical priorities. To what degree have you understood these plans, or even helped shape them?

As an example, every recipient of assistance from the likes of the World Bank and the ADB has a negotiated and agreed upon roadmap forward, or “country partnership strategy.” This publicly available document lays out priorities for the next few years. In the policy area, this could include spelling out which sectors will receive a so-called policy loan – for example, to shape the regulations related to the energy sector or financial sector.

With the right knowledge and insights, private and social sector players can be important and credible sources of information for developers of strategies, or for the actual providers of technical assistance or other support given to a government under a policy loan.

All of these efforts speak to helping define oneself as a credible, responsible partner.

At the end of the day though, I said then and I say now, that corporate social responsibility is not about what you do with your money. It is about how you make your money. Being a member of the public or social sectors also does not give one an automatic “pass” when it comes to “responsible conduct.”

This question applies equally to business and social investors. For civil society, variations of the same questions apply. How did you finance or raise the donor dollars for you efforts? And how well do you treat your own employees, above and beyond what you are able to pay them.

To be a credible partner, you will need to have and be seen as having conducted your affairs in an honest, transparent and fair manner.

Commitment

How committed are you? Do you have the long-term staying power and the commitment to succeed in a given market? Everyone says they are committed to stay, or to help. But what are your metrics – from staff retention to re-investment dollars – and does the data prove this out? It’s not just “show me the money,” but show me the commitment. This is a particularly important point in emerging markets, where foreign businesses, government aid programs and NGO offices come and go. But it also applies at home to your visibility among community members at the neighborhood level.

Do you have the capacity and will for long-term engagement necessary for slow, if not always steady, progress? Will your people on the ground now be there long enough to build the proper relationships to collaborate on policy? That is, do you have the long-term commitment – and critical Board and management support – to raise the necessary budget, develop the design systems, and create the incentives for long-term progress in a fast-changing world that often overemphasizes short-term results?

It is necessary to demonstrate commitment at both the top and the bottom and to build relationships. This means walking the ground with people who live in the targeted communities, not just visiting as an “expert” or as a fly-in-and-fly-out consultant. The organizations that succeed are the ones that do this the best.

When it comes to Credibility and Commitment, there is no better measure or sign of both than having people in country with local knowledge and local language skills. For all sectors, it is people on the ground, after all, reading local media, socializing with others in the industry, and meeting and engaging with local counterparts, who are best positioned to anticipate changes down the road.

It is time to come together to help spur innovation and entrepreneurship, to increase the flows of investment and re-investment dollars, to focus on people and on what one does best, and to be true partners for progress.

How now to improve perceptions, and the reality, of your own engagement in and across different sectors? Demonstrated transparency, credibility, and commitment will help get you there. These three levers are also the fundamental drivers for more effective information sharing.