Scholarship with Impact

Abstract

This essay offers a personal and therefore partial view of the possibilities of scholarship with impact. The challenge of scholarly impact is framed in the duality of scholarly and policy/practice impact. This is an aspiration that not all social scientists interested in the field of management may aspire to. However, at a time when the British Academy of Management (BAM) is celebrating its 25th anniversary, it may be appropriate for some members of our local and international communities to raise their aspirations and deliver forms and processes of knowledge which meet the double hurdle of scholarly quality and policy/practice impact (Pettigrew, 1997, 2001). The paper sets out to accomplish this task in the following way. The first section outlines some of the current instrumentalities facing UK scholars of management in and around the 'impact agenda'. Of particular interest here are the recently published Decisions on Assessing Research Impact, published by Higher Education Funding Council of England (HEFCE, 2011) for the Research Excellence Framework (REF) 2014, and the work on impact and assessment measurement carried out by the Economic and Social Research Council Evaluation Committee and published in 2009 and 2011 (ESRC, 2009, 2011). The following section then seeks to contextualize the present instrumentalities by locating the impact agenda in the much wider, deeper and historically embedded debates on the purposes and practices of social science and management research. Reference is made here to some of the influences on current UK management research and the way our fields of knowledge are themselves a product of wider pressures to change the character of knowledge and how it is produced. The third section deepens our treatment of impact by examining the progressive influence of the ESRC Commission on Management Research (ESRC, 1994); the British Journal of Management Special Issue (Hodgkinson, 2001); the work of the ESRC (2009, 2011); and pressure from scholars on both sides of the Atlantic, e.g. Hambrick (1994), Pettigrew (1997, 2001), Starkey and Madan (2001) and Van de Ven (2007). In the fourth section some of the determinants of research impact in the sphere of policy and practice are discussed. The fifth and final section then offers a conclusion. The recent announcement by HEFCE (2011) that 'there will be an explicit element to assess the impact arising from excellent research, alongside the outputs and environmental elements' is a notable step forward in national policy and a clear statement of intent for the future. The weighting of 20% for impact alongside the 65% for outputs and 15% for research environment is a compromise for those who fought hard to resist the idea of assessing impact. But the change has now been made and there is a clear signal that the 20% will be increased in subsequent exercises. The REF and its predecessor the Research Assessment Exercise (RAE) perform multiple functions. They are at once a resource allocation mechanism to reward high research performers and admonish the lesser performers; a quality assessment mechanism; and crucially a potential mechanism of culture change for all the academic communities in the UK higher education system. We as scholars are fond of our scholarly routines and those routines take much shifting even with pressure from HEFCE. In my view, successive RAE assessments right back to the mid-1980s have changed individual and institutional behaviour. Scholars are now much more attuned to where, when and how they publish. The spotlight of performance has become an uncomfortable zone for individual, group and institutional behaviour. But the stringency of the audit society (Power, 1999) has had positive and negative effects. The RAE has added to other career, professional and self-generated pressures and may have created a displacement of goals. For some publishing may have become an end in itself. Including the impact of research in the social, economic and cultural spheres beyond academia is an important corrective to this displacement of goals. In shifting the focus from the intermediate good of published outputs to the final good of scholarly impact (in the outputs criteria) and policy/practice impact in the impact criteria, there may indeed be important behavioural and cultural changes to our practice as scholars. Enhancing scholarly quality remains essential but in time this opening up of the impact agenda can create greater legitimacy for a portfolio approach to publishing for those scholars deadened by the retreat to defining scholarship just in terms of publication in A-rated scholarly journals. Of course, our position in business and management schools means we have many possible links out into stakeholders in the economic, industrial and public policy spheres of society. We should not be threatened by the impact agenda but colleagues in the arts and humanities most certainly have been. Stefan Collini writing in The Times Literary Supplement, 13 November 2009, commented at the end of his forthright article that unless the REF guidelines are changed 'Scholars in British Universities … will become "door to door" salesmen for vulgarised versions of their increasingly market orientated products'. In a similar vein, James Ladyman in the Oxford Magazine No. 294 observed that 'It is astonishing that the ostensibly intelligent and knowledgeable people who are responsible for higher education policy should evince the combination of philistinism and ignorance responsible for the ridiculous and deplorable ideology of impact now being foisted upon us' (2009, p. 4). As we all know, one person's evolutionary change is another's revolutionary change. The impact agenda, which, of course, is not just confined to the REF but also now shapes the applications to and evaluations of Research Council grants and awards, will remain controversial even for some living within professional schools. But all changes have their antecedents and it is to those germane to our management research communities that I now turn. Interest in the impact of social science and management knowledge is as old as the fields of study themselves. Current UK instrumentalities are but a moment in a long process whose pathway of development is unpredictable and shaped by many stakeholder interests. For most of the 20th century, social scientists have debated 'knowledge for what purpose' (Lynd, 1939) and, indeed, for whose benefit? Current discussions of impact are embedded in the thesis of a change in the character of knowledge and assumptions about a new process for producing knowledge. This is a not a place to review such development but the interested reader may wish to explore Jasanoff (1990, 2006), British Academy (2004) and Nowotny, Scott and Gibbons (2001). Elsewhere (Pettigrew, 2001), I have argued that the elements of the change are many, but the most often debated features include a more porous boundary between science and society; a resultant loss of research autonomy; a breakdown of assumptions about unitary views of science and linear notions of the scientific process; a greater range of participants in the knowledge development process and greater pluralism of research practice; a greater recognition of the localized (in time and space) character of research practice and outcomes; and a recognition of the complex interactions between multiple stakeholders in the research process and a more contested landscape for evaluating the quality and relevance of research processes, outputs and outcomes. These hypothesized and real changes have influenced and are influencing us as a management research community. Colleagues recruited into business and management schools in the last 18 years conceivably may have missed the report of the ESRC Commission on Management Research which was published in 1994 (ESRC, 1994). This Commission was chaired by George Bain, then Principal of London Business School, and included visible BAM members such as Arthur Francis, Andrew Thomson, Robin Wensley and myself. The title of the Commission's report Building Partnerships: Enhancing the Quality of Management Research well signals its core theme. Published as it was seven years after the creation of BAM, the report's findings provide a barometer of the perceived state of management research in the UK at that time. The report noted the rise and rise of business and management schools in the UK higher education system (a trend that has been sustained), but also acknowledged that the expansion had brought its problems as well as its successes. 'Rising concern about the character, quality, funding and relevance of management research led to a series of meetings between the ESRC and the management research community and, in particular, with the British Academy of Management' (ESRC, 1994, p. 1). Looking back, it is amazing how prophetic and measured the report was. Although in the contemporary sense the language of impact did not permeate the document, the lack of impact of management research was a key part of the diagnosis. 'One of our themes is that management research still does not make enough impact on users and thus on management practice. But its capacity to do so is not in doubt' (ESRC, 1994, p. 5). Notwithstanding stereotypes about the pervasiveness of implementation gaps in public policy, many of the recommendations of the report did in time happen. Thus we find that the ESRC, BAM and the Foundation for Management Education (FME) have supported investments in fellowships to develop scholars; the same bodies have greatly supported research networks in the management research community; and the ESRC has included relevance to users in the grant application and evaluation processes. Crucially, the £30m ESRC/Engineering and Physical Sciences Research Council management research initiative (Advanced Institute of Management (AIM) Research) has happened and has benefited the development of many UK management scholars. It is also noticeable that recommendation 9 of the report anticipates aspects of the new REF, although 'relevance' was preferred at that stage to 'impact' as the criterion of worth. 'We recommend to the Higher Education Funding Councils that they incorporate the assessment of research relevance alongside their periodic assessments of research quality' (ESRC, 1994, p. viii). The only important recommendation of the ESRC Commission which did not happen was the creation of a Management Research Forum whose task was to have been to facilitate agenda building, knowledge exchange and constructive partnerships between researcher and user communities. It seems the ESRC and other interested parties could find neither a very senior industrial figure to chair this body or the right calibre of executives and policy-makers to populate it. So our community problem of the management of indifference in the market place of potential users was real enough in 1994. I wonder if that situation has changed much in the last 20 years? It seems we remain challenged by the diffuse nature of the communities who could be our potential collaborators and users. There seems to be no natural focused community for our management research and the challenge remains a local one as we engage project by project (Adler, Shani and Styhre, 2004; Beech, MacIntosh and MacLean, 2010; Pettigrew, 2003). The ESRC Commission on Management Research was foresighted and delivered within its remit. Subsequently BAM and the FME have played their part in encouraging intellectual debate about the character of management research in the UK, with Tranfield and Starkey (1998), Starkey and Madan (2001) and the Special Issue of the British Journal of Management (Hodgkinson, 2001), which offered a UK and US academic commentary on the Starkey and Madan report. The report itself was a valuable contribution to an issue where there are no easy answers. The British Journal of Managment Special Issue revealed disparate views within the UK and between the UK and US scholars about the nature and purposes of business schools and universities and the varieties of knowledge and knowledge processes open to us, and helped to unpack easily used but complex terms such as relevance, user, collaborative research and Mode 1 and 2 research. But in 2001 the impact word had not yet captured the academic and political space. This language capture began to happen in the UK around the period 2005 onwards as the ESRC Evaluation Committee started to pick up the issue of assessing the non-academic impact of social science research. One of the material effects of this work has been the inclusion of non-academic impact assessment in the 2014 REF and, indeed, the form and process of assessment in that REF. Over the last several years the ESRC's Research Evaluation Committee (now Evaluation Committee) has been at the forefront of developing impact evaluation practice. Those scholars interested in clear statements of the progressive development of such work may wish to read two papers published on the ESRC Society Today website. The papers were respectively published in 2009 and 2011 and are titled 'Taking Stock' and 'Branching Out'. Margaret Macadam, one of the officers of the ESRC, played a key role in preparing these papers. Important related work includes Davies, Nutley and Walter (2005), Nutley, Walter and Davies (2007) and Meagher, Lyall and Nutley (2008). The first symposium organized by the ESRC in this stream of work was in May 2005. The report of this symposium by Davies, Nutley and Walter (2005) was an important influence shaping the principles behind subsequent developments. Key messages from this symposium included the recognition that different types of research produce different types of knowledge; non-academic impacts are diverse and can be stimulated by a number of different mechanisms; research contexts can be facilitators and inhibitors of research impact; and the assessment of research impact has to be multidimensional and draw upon both qualitative and quantitative techniques. There was also the clear message that at that time the RAE was a major barrier to non-academic research impact. In subsequent thinking the term non-academic impact has been substituted by policy/practice impact and the framing of policy/practice impact has always been addressed within the double hurdle of scholarly and policy impact. Earlier work on knowledge creep by Weiss (1980) reinforced later messages about the time lag and attribution problems in connecting research processes and outcomes with policy and practice impacts. A further progressive step was the recognition that the fate of knowledge was not just tied to receptive and non-receptive contexts (Pettigrew, Ferlie and McKee, 1992), but also to the tripartite relationship between the content of research, its context and the knowledge production process itself (Pettigrew, 1987). The explicit recognition that processes of knowledge production might matter made it much easier to incorporate recent thinking about the co-production of knowledge (Jasanoff, 2006). With this came the working hypothesis that early and continuing engagement between knowledge producers inside and outside universities increases the probability of policy/practice impact. The ESRC impact evaluation work has not just stopped with developing impact evaluation methods, including the possibility of identifying the economic benefits of social science research. The Evaluation Committee work has also explored the key drivers or determinants of impact and thereby how impact may be achieved. For this work it was even more important to be able to define conceptually and operationally what research impact on policy and practice is. Here unpublished work by Laura Meagher of the Technology Development Group turned out to be valuable (2009). Meagher identified five categories of impact: instrumental conceptual capacity building cultural change, and enduring connectivity impacts Instrumental impacts were defined as tangible products or services taken up by companies, policy-makers and practitioners. Conceptual impacts included generating new understanding or raising awareness among potential users of research findings. In my experience, conceptual impacts can also include reframing policy issues in order that new policy options and pathways to implementation open up or close down in a policy domain. This closing down of policy options through intellectual challenge can be a very important scholarly contribution, especially when robustly evidence based. Capacity building impacts include training and/or developing collaborative activities and cultural changes, and enduring connectivity impacts can shape both willingness to engage in knowledge exchange activities and the establishment of sustainable relationships between knowledge producers in and outside universities. As we all know, the manifest payoff from this ESRC Evaluation Committee work will be real understanding of why and how policy/practice impacts occur when they do. Here there are early signs of illumination which will help us to shape a practical theory to inform the development of the capabilities, practices and processes to increase the range of scholars doing high impact research. Research in any field is a probabilistic activity. There are few predetermined processes and outcomes in creative activities such as scholarly research. Aspirations that researchers may have to influence policy and practice outcomes are themselves complex influence and change processes and we know from the established research literature on change that such changes are much shaped by the context, content and process of the changes themselves (Pettigrew, 1985, 1987, 2002). Encouraged by the report from their first symposium on non-academic impact (Davies, Nutley and Walter, 2005), the ESRC funded a series of case studies to examine why and how policy and practice impact occurs. Subsequent analysis (ESRC, 2009) has explored the determinants of impact by using the framework which explores the interdependent effects of research context, content and process. Thus the ESRC's impact evaluation work (ESRC, 2009) identified a set of key factors that appeared vital for impact generation. These include established relationships and networks with user communities involving users at all stages of research well-planned user engagement and knowledge exchange strategies portfolios of sustained research activities that build reputations with research users good research infrastructure and management support for user and knowledge exchanges, and where appropriate, the involvement of intermediaries and knowledge brokers as translators, amplifiers and network providers. These determinants of policy and practice impact are both contextually and temporally embedded. The history of knowledge production in the natural, medical and social sciences is full of examples where societal, religious and policy contexts have presented non-receptive settings for new ideas and those ideas have had to wait for the emergence of more receptive contexts for more developed recognition. But social scientists do not need just to wait until more receptive contexts emerge. If they have pre-existing relationships and credibility in policy arenas it may be possible to shape the climate for new knowledge to find acceptance. Serendipity can also act as a determinant of impact. Research findings may have the good fortune to coincide with the rise of policy initiatives. So one enabling feature of impact is the extent to which the content, originality and timing of ideas and new knowledge fits the rise and fall of issues in the policy context. On the process side, the ESRC impact case studies demonstrate that 'the most important factor contributing to the generation of policy and practice impact was the pre-existence of networks and relationships with research users' (ESRC, 2009). Suc

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 122 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.