Research

An Essay About My Intellectual Project

I have had the extraordinary good fortune to have spent my professional life on a project I chose early and stayed with throughout my career. Throughout, the work was made possible by a position Harvard’s Kennedy School of Government. I was one of the original student and faculty recruits to that enterprise. Its mission was to develop good public policy ideas and develop individuals to become effective public leaders whether as policy advocates, politicians, political executives, agency managers, or experts of one kind or another. 

A major part of that work was intellectual in the sense that it required one to produce both methods and objects of thought that could be used by others to do their professional jobs. That work could be put down on paper and seen and examined by those who could read, think, and apply what was suggested. 

But the main purpose of the writing was not to prove beyond doubt an empirical claim about how a particular piece of the world worked (though it often depended a great deal on having a great deal of empirical knowledge and insight about the particular piece of the world that was under examination. The main purpose of the writing was to suggest a method for figuring out what could or should be done about some individual or social conditions that were viewed as bad, or unjust, or simply problematic, and what particular actions a particular individual or team or organization could take to collectively authorize and implement an agreed upon strategy for improvement. This was intellectual work focused on developing methods of policy design, leadership, and management, and applying them to particular substantive problems or particular organizations that existed in the real world. We were a professional school; not a social science department. 

And, as a professional school, we faced not only the challenge of developing the methods that would be useful in the practice of policy analysis and design, the mobilization of legitimacy and support for taking actions that promised improvement, and deploying the mobilized resources through a process of production that was spread across several different organizations, but of developing a pedagogy that would not only teach our students these methods, but build their commitment and capacity to use them in real circumstances. It was this unique situation that has guided and distributed my work over my career.

Crime Policy Analysis and Design 

I began my career primarily as a policy analyst and designer, trying to work out solid, plausibly effective solutions to real complex policy problems. 

To do that, I had to learn not only a set of analytic tools for gathering data about the world, describing how it seemed to work, investigating and imagining alternative ideas about how to push the world towards a better state, and set up processes for learning as we searched for a better approach to an old problem, or a thoughtful approach to an emergent one.  

As it turned out, while the tools I had learned were sufficiently abstract and general to be used in dealing with many different substantive problems, for reasons that are still mysterious to me, I ended up working on the problems of drug abuse and crime.  That pushed me into several decades of work on drug and alcohol policy, violence prevention and control, youth gangs and juvenile justice, domestic violence and child protection, weakening the power of organized crime, and wondering how we could reduce the violence, fear, and despair that seemed to be undermining our society, including our effort to improve race relations across the society. 

Strategic Management of Public Agencies and the Concept of Public Value

Soon enough, as the HKS continued its development as a professional school of government, we realized that it we were not going to be able to achieve our ultimate goals if we trained only those individuals who worked with the tools of social science, statistics, operations research to design or evaluate new government policies or programs; we had to think about developing knowledge, methods, and skills that could enable those in more political, operational, and accountable positions to lead and innovate in government agencies that had too often become internally focused, bureaucratic, and staid rather than goal oriented, adaptive, and creative in achieving important social goals. After all, we existed to improve the performance of government, not just have good ideas that were unused. And to do that, we had to bring not only the intellectual tools of social science, but also the humanistic perspectives of philosophy and history, and the practical perspectives and methods that had been developed over the years by those who faced the concrete challenges of public leadership and management. 

Somewhat against my better judgment, I was recruited into a sustained, intense effort to develop the field of “public management” as an alternative to the established field of “public administration.” The emphasis of emerging public policy schools on the creation and implementation of new public policies seemed to overshadow the existing schools of public administration because the public policy schools seemed to prepare folks for the exciting part of government: the part were the government stared at an important social condition or problem, and decided whether and how it would take that problem on. This was the world of policy making – of ends, not just means, of new ideas rather than old, and perhaps of politics and leadership not just organizational administration. 

It was not as though the challenges of public administration – the capacity to manage an organization to be incorruptible, deal with individual cases consistently and reliably, to develop and deploy a professional work force that could operate efficiently, effectively, and lawfully in pursuing its mission – had disappeared. It was just that it seemed that we could take that for granted, and that the important way to improve government was to design, adopt, and implement new policies and procedures, 

To pursue the issue of public management, a particular crucible was constructed that consisted of the following elements. 

First, a small group of senior faculty from the Kennedy School who were not only distinguished academics, but also had had extensive experience with government, was combined with similarly situated senior faculty in the Law School and the Business School, and a small group of more junior faculty at the HKS in a group that meant for several hours each week to work out the content of a curriculum that would teach our MPP students what they needed to know about how government made decisions and implemented those decisions to achieve intended results. Everyone had a basic academic background in positive and normative political theory, in the structures and processes of government policy making and implementation, in public law and administration, in organizational theory, and in social and political movements that could emerge out of nowhere to profoundly alter the potential for government action.  

Second, the HKS decided to make a commitment to “executive education.” Since many of the HKS faculty members were drawn from the Faculty of Arts and Sciences, we did not have much experience with the design or execution or evaluation of such programs. So, we agreed to do this in partnership with the HBS, and the members of their faculty who had joined our intense discussions about “public management” and whether it was similar to or different from private sector management. 

Third, we began writing cases that described the actual conditions that public managers faced in practice, how they seemed to diagnose those situations, what they did, and what the results were. To a degree, this was empirical data on the thinking and practices of existing skilled practitioners. But it actually turned out to be a great deal more than that. The cases became the basis for conversation within the small faculty group, and out of that conversation, various analytic frameworks began to emerge that seemed capable of revealing opportunities, and providing information for judgment about what should be done not only in that case, but in other “similar” cases. Moreover, the more we worked on the cases, the more “similar” they began to seem. It seemed as though it might be possible to write down some general methods or analytic frameworks that could be used by practitioners to diagnose their environments, and using information from that diagnosis, imagine and test a path forward that had not bee visible at the outset. 

Eventually, these ideas were written up in many different books and articles. My own attempt at this contained in Creating Public Value: Strategic Management in Government. That book was based heavily on a decade or more of experience not only in conversation with my colleagues at the HKS, but also in a constant dialogue with the practitioners who came to the HKS to learn about “public management”, and eventually public leadership. At the outset, I think I was learning much more about managing in the public sector than I was teaching; but over time as I absorbed the knowledge and experience of the practitioners as they were brought to bear on many particular cases, and as I argued with, and stole shamelessly from what I was learning from my colleagues, we began to develop something like a “school of thought” about strategic management in the public sector. 

What made this school of thought different was that it chose to confront directly two key features of managing in the public sector that had been “papered over” in the past. The first was that one could make a firm distinction between politics and policy on one hand (the province of elected officials), and administration and implementation on the other (the domain of professional experts in subject matter areas such as public health, civil engineering, national defense, etc. and in the tools of administration such as budgeting, human resource management systems, management control systems, etc.) 

The distinction between policy and operations was never very clear at a theoretical level; and it became much muddier as politically elected or appointed government executives struggled for control over what particular objectives the government was to pursue, and how resources could best be deployed to pursue those goals. The politicians complained bitterly about the intransigent bureaucracy (latter, the deep state) that was pursing its own interests or its own idiosyncratic politics ignoring the purposes of those elected to define government’s purposes, scope, and general methods. The professional experts and managers in government complained about the micro-management of their operations by individuals who had little substantive knowledge of the work, and who seemed all too inclined to allow personal or partisan interests to influence their decisions about the ends and means of government.  

The second misplaced assumption was that the world in which public managers – both political executives and career public officials -- worked was reasonably stable, and that relatively durable, coherent policy mandates could be constructed by the politicians as the framework within which government could work.  What this ignored was the simple fact that environment in which public managers worked was extremely chaotic and uncertain. The strategic approach to public sector management, like the strategic approach to private sector management, made a big deal about paying close attention to the environment in which an organization was working to create something of value or utility. This was essential in the competitive market environments faced by commercial managers who had to consistently re-position their organizations through investment and disinvestment to ensure that they could sustain profitability over time. 

The public sector, in contrast, seemed to focus a great deal on the internal processes of existing agencies – trying to ensure that they could meet stringent requirements for accountability in the use of assets entrusted to them, fairness in their encounters with citizens, and ideally “efficient and effective” in achieving collectively desired results. 

But in order to determine whether the agencies were efficient and effective (as well as accountable and fair), one had to look outside the boundaries of the organization to the environments within which government organizations were operating. When one did, one did see a stable, coherent environment; one saw a dynamic, complex, and contested environment. 

The political environment – which was supposed to supply managers with stable, coherent mandates that would provide the consistent frame to use in planning and evaluation – was simply not up to this task across the government as a whole. Instead of being simple (in the sense that the political environment wanted only one particular thing from a government organization), it was complex (in the sense that different competing political forces were insistent on performance along several different dimensions of value). Instead of being stable, the political environment was dynamic, with new dimensions of value being nominated as important values for the organizations to advance or reflect in its operations, or with the relative importance of one value to another being constantly shifted. 

Similarly, the social conditions that government was assigned to improve – what we might call the organization’s task environment – was equally complex and dynamic. The operational challenges that organizations faced in pursuit of their mission were rarely the same thing over and over again; the particular problems they faced were varied, and required different responses. Moreover, the problems changed over time with some new conditions arising that needed a response, or an old, previously unimportant problem suddenly taking on new urgency. 

The fact that the political and task environments of public agencies were dynamic and fluid meant that government organizations needed to keep adapting and fitting things together in a sufficiently coherent form that they could get on with their work. This meant that the relative importance of policy development and organizational innovation increased. But often policy development and innovation raised important questions about ends as well as means, and created risks as well as clear opportunities for success. This meant that the tangle between the politicians who were supposed to confront the new and manage the risks (but were actually quite risk averse!), and the professionals who could see the problems and the risks, but not always the solution, got ever more heated and often de-generated into a process of defensive blaming rather than making adaptive responses to the problems the society faced. 

The intellectual challenge for the HKS thus became the question of how one could manage to keep working to define and create public value in a hotly contested, value laden, and technically challenging world – a set of challenges that looked quite a bit different than learning how to use to tools of administrative influence to keep an organization operating like clockwork. 

The Voluntary and Nonprofit Sector

After working for about three decades on criminal justice policy and the management of government agencies (including but far from limited to criminal justice agencies such as police, prosecutors, public defenders, courts, prisons and jails, and juvenile justice), I was drafted by the HKS to establish a newly endowed Research Center – the Hauser Center on the Non-Profit, Voluntary Sector. The HKS was motivated to establish this Center for the simple reason that many important public sector functions were being underwritten and complemented by voluntary sector organizations. It was also true that many students who sought opportunities to exercise public leadership were taking up or returning to positions in the voluntary sector. 

In many ways, I was an unlikely choice to lead this initiative. I had not been a scholar of the voluntary sector. But I had been immersed in the processes of democratic governance – including not only the management of government agencies acting alone, but also in the complex processes that were described generally as the “privatization” of the public sector – which included but was not limited to the idea of “contracting out” services currently provided by government agencies to non-profit or commercial suppliers. I had also been drawn fairly deeply into the worlds of electoral politics, policy advocacy, public policy making processes, and public deliberation since it was these processes that defined the purposes to be pursued by public agencies as valued dimensions of public value, and provide the legitimacy and support that public managers needed to achieve the desired objectives. Finally, I became very interested in the work of my colleague Robert Putnam who was developing the idea of “social capital” and demonstrating its impact on the quality of both individual and collective, civic and political life. 

Once the scope of our concerns at the HKS shifted from the narrow frame of government and public policy making and implementation to the much wider frame of governance; the creation, mobilization and direction of “public spirit”; and “three sector approaches to the solution of public problems”, the role of the structures and processes of the voluntary sector became much more central to our concerns. We needed to understand more than we then knew about the different functions that the voluntary sector played not only in producing expanded, more innovative approaches to dealing with existing and emergent social problems, but also in “calling a public into existence” to act directly on social and public problems or give guidance to government as it sought to recognize and respond justly and effectively to problematic social conditions.  And, we also needed to learn more about the special practices that were associated with leading and managing the variety of voluntary sector organizations ranging from and through:

  • large civic service organizations such as Doctors without Borders, or the American Red Cross,

  • local community-based organizations such as the Bronx Desperadoes or the Harlem Children’s Zone, 

  • cultural and religious organizations such as the NAACP, or the emergence of Christian Mega-Churches

  • political advocacy organizations such as the National Rifle Association, or the Children’s Defense Fund

  • political parties, and 

  • the “philanthropists” and volunteers who made both large and small donations of money and time to sustain these organizations and their efforts to pursue public purposes through direct action or through social and political influence. 

Working on the voluntary sector as an important part of society that shapes the quality of individual and social life alongside the commercial sector and government forced me to take a much broader view of the structures and processes that democratic societies relied on to learn about and act on problematic social conditions. The society did not consist only of a commercial sector and a government sector; it had a crucially important “third sector.” But that third sector wasn’t important just to handle the problems that the commercial sector and government sector could not, or chose not to manage. It was to cultivate and allow for the expression of individually held ideas about what individuals should do to help one another, and what a good and just society would expect and demand of its citizens in the pursuit of prosperity, civility, and justice. The voluntary sector emerges from the intimate and powerful influence of individuals, families, neighborhoods, churches, synagogues, temples, in search of a social as well as a solitary life, and a life of relationships and meaning as well as material goods and services. From that base, the voluntary sector produces some important independent effects on individual and social life; but it is even more important as it influences the character, purposes, and performance of the commercial and governmental sector. It is the ground from which these other more visibly collective structures and processes emerge. 

Problem Solving Societies: Networked Government, Social Innovators and Social Learning

As we were developing the Hauser Center on the NonProfit and Voluntary Sector, two compelling new ideas began to emerge that had an important impact on how the HKS should be thinking about improving individual and social conditions. One of them has already been mentioned: namely the idea that many of the most important social problems facing society could be solved not by government, but by collaborative partnerships that could be developed between government and the commercial sector, or among the three sectors of liberal societies: government, voluntary, and commercial.  

The second, was that efforts to improve social conditions might benefit from the kind of entrepreneurship and innovation that drove productivity changes in the commercial sector, but had not been encouraged by governments who often enjoyed a monopoly on the supply of public goods and services, and had succumbed to the monopolist’s natural tendency towards dull repetition rather than creative invention and a sustained search for the “production possibility frontier” in particular industries. 

The best way to investigate the first claim, and to help our students exploit the opportunities for social and public value creation that might come from cross sector collaborations was to observe various collaborations that sprang up, and see how they performed. 

At the outset, this seemed particular promising – particularly when we understood that collaborative efforts might be needed across organizations and levels within government, as well as across organizations from different sectors.  After all, the structures we relied on within government had been created to deal with problems from the past. As social conditions changed – reducing the importance of some old problems and increasing the importance of emergent problems; or as social and political aspirations changed; the structures, allocation of resources, and operational methods had been slow to change. Moreover, it seemed that many of the new problems – environmental threats, pandemics, financial risks – were spread across different government organizations and sectors. 

Indeed, a little reflection suggested that we might have made a big mistake in thinking that there were some social conditions or problems that were purely (and exclusively) private, and some that were purely (and exclusively) public. 

Consider for a minute that conditions were think of as being purely private. 

Surely, raising children would be one important such function. The right and responsibility to form a family and care for any children seems to be at the core of private life. 

Yet, a little reflection suggests that this work of raising children is heavily subsidized by public funds. If a child does not have a parent, the state will supply one at public expense. The right to a public education is guaranteed in most liberal societies (along with the duty to become educated in some other way if one does not wish to attend the publicly provided schools). Parental leave is often legally mandated, and child care is often exempt from taxation.

Similarly, child-care is highly regulated. Institutions that assume responsibility for caring for children are often tightly regulated to protect health and safety. If parents divorce, a public court steps in to insure that any children will be well cared for. And, all parents are obligated to avoid “abusing or neglecting” their children on pain of civil or criminal penalty if they fail to meet their duties. 

Turning to the other end of the spectrum, surely the administration of justice, and national defense would be purely public. 

Yet, again, a little reflection suggests that neither of these sectors is purely public. There are many more privately paid security guards than public police in liberal societies. Most individuals have to pay for private attorneys to represent them in court even when they are facing criminal charges.  Increasingly, important disputes among private companies go to private arbitration rather than public adjudication. Most of weapons used by our military for national defense are developed and produced by commercial enterprises. And, increasingly, we are using private companies for different kinds of security operations overseas. One could even say that our choice to go with an all volunteer army that is paid to fight represents a private sector turn in our defense sector.  

This suggests that we have always been wrong to think that there is a clear dividing line between the private and public sector – that the reality of the way in which we live our lives individually and collectively is much more complicated than this. That, in turn suggests that we might need individuals who can help us see the reality of how our private and public institutions, operating at different levels of society as well as in different sectors, actually function to nominate and begin to take collective action to deal with individually and collectively experiences problems. 

One simple way to think about this challenge is that we need not a more complex vision of how a liberal society decides what issues to take on, but also individuals who can use that vision to organize the collaborative arrangements that might be beneficial. In effect, we need “three sector athletes” who can work happily across sectors forging value creating partnerships rather than public administrators, non-profit managers, or profit motivated executives who lacked the vision, the incentives, or the authorization to step forward. 

The second claim – that we needed to overcome the lethargy of the public sector and its low rate of productivity gains by creating a new form of public leadership focused on social innovation and entrepreneurship – was also worth investigating. Indeed, this idea gathered a head of steam rather quickly as a consequence of the success of tech entrepreneurs, and their turn to philanthropy. 

To many tech entrepreneurs, it seemed that it ought to be possible to produce the same sustained increases in productivity that we could observe in the commercial sector in social and public sectors. All that we needed to do was to change the structures and processes of governance to allow for more effective collaboration with government and across sectors, and to create room for innovative entrepreneurs with good ideas to enter more easily into the social and public sector production systems, and show what imagination and a creative enterprising system could do. Since many such entrepreneurs had accumulated a significant amount of personal wealth, it was within their power to create significant capital funding for innovative approaches to public sector problems proposed by “social entrepreneurs” who did not want to subject themselves to the political accountability and bureaucratic red tape that made it hard to innovate within government agencies.

Educational System Change, Policy, and Management

The idea that the solution to many important social and public problems lay outside the boundaries of government organizations, and could be solved or ameliorated by cross sector approaches with the help of social innovators was appealing to many students and donors to the Kennedy School who were hoping to engage fresh energy and creative thinking into collective efforts to solve social problems. I confess that I was a bit skeptical of this idea, but it was worth trying to find out more. 

At the time, the social/public sectors that was under the most pressure to improve, and to make use of these new ideas was the educational sector. Long dominated by government, and seeming to fail consistently in its efforts to educate students, and create the upward mobility that would reduce social inequalities and demonstrate that we remained a society of equal opportunity, the pressure was on to search for and find both better methods of teaching, and better ways of organizing and guiding the educational sector as whole towards sustained productivity gains. 

One important response to this challenge was the creation of a new, professional education program at Harvard – an educational program designed to train individuals who were committed and able to take advantage of the changes that were occurring the educational sector and push the sector towards rapid productivity gains. The program offered something unusual: a three year program involving extensive classroom work focused on professional knowledge and skills, but also an intensive year-long placement in an educational supplier within which the student would be expected not simply to observe, but to try to improve educational performance at the site. The reward was a professional rather than academic doctoral degree – a Doctorate in Educational Leadership. 

Since my entire career had been devoted to professional education and this program represented a significant departure from what most of Harvard’s professional schools were doing; and since the Ed Sector was one of those three sector social production systems; it seemed an ideal opportunity to develop and test both the wider ideas about how to improve the performance of such systems, and how to develop individuals who could carry this load. I joined the Ed School faculty as a half time member, and went to work on how the nation’s educational systems (including but not limited to public schools) could be improved. This brought me back to the substantive challenge I had taken on early in my career in the criminal justice domain where I had sought to improve the quality of community security by helping public police departments understand that their performance depended critically on the quality of the help they got from their local communities.

A Reflection on the Whole: Creating Policy Ideas and Public Leaders

Over the course of my career, each of the big hunks described above attracted and absorbed my intellectual attention and effort. Any one of them could have made a career. And I am painfully aware of how shallow my understanding of these subjects might have been. I am also painfully aware that I left some of these fields with promises unkept – to colleagues, to the practitioners I wanted to help and to myself, and promising initiatives wasted. 

But I am also aware that as I went through my life, I left none of these subjects entirely behind. Each one shaped the way I approached the next and offered a somewhat distinctive angle on that new subject. 

It is perhaps not surprising, then, that I ended at the core of where I began: with the question of what could individuals do to improve society, and how might society construct and manage itself to get the most out of those individuals who wanted to improve material conditions for all (while not exhausting the world), and who wanted to create right relationships and justice in our economic, social, and political realms. 

This last question – what could a professional school, designed to help social change agents develop and pursue initiatives that could improve our individual and collective life together do to accomplish that demanding goal  – has been my abiding passion. It is where I began, where I finish, and what consumed my attention throughout my career. Much of my work has sought to answer that question either directly, or by example. This subject defines the last body of work on display in this site: writings on what it takes to create an excellent school of public policy, leadership, and management. 

I have put this body of work together in hopes that others might find it easy to access some of my thinking so they can put it to good use directly, or better still, improve it. There’s got to be a pony in there somewhere!!!!