Improving the Measurement of Innovation: Supporting International Comparisons

Organization For Economic Cooperation & Development, Paris Cedex 16 ILE-DE-FRANCE

Investigators

Abstract

Business leaders and public policymakers worldwide continue to have high interest in identifying the conditions, investments, infrastructure, and policies likely to promote and hasten the pace of innovation by businesses and other economic actors. Increasingly, surveys of innovation are being developed and used to gauge the incidence of innovation across national economies and to identify the driving variables and relationships. The European Union has run its Community Innovation Survey (CIS) among member countries regularly since 1992. CIS-based surveys have also been implemented in many of the member countries of the Organisation for Economic Co-operation and Development (OECD) and, more recently, in China and some African countries. In the United States, the National Science Foundation?s new Business R&D and Innovation Survey (BRDIS), first fielded in 2009, includes questions on business innovation drawn from the CIS approach. This is an opportune time to review the CIS approach from several perspectives: what are the CIS questions actually measuring; how well can these measures, transferred to the U.S and other national economies, yield robust statistical results and produce indicators that can be compared over time and with those of other countries using the CIS model; and what might also usefully be measured were new questions to be added. Project Description The proposal outlines a project to conduct an in-depth review of the questions in the CIS and other existing innovation surveys and to develop new and improved questions. The project would be conducted over two years, as part of an ongoing OECD taskforce on business R&D and innovation survey re-design. The work program would include cognitive testing of existing questions in English and selected languages, as well as the development and testing of new questions in areas of growing interest in the study of innovation, such as the measurement of intangible assets for innovation. The result would be a set of carefully tested questions and guidelines for their use. These results could be used to extend the coverage of existing surveys, or for new surveys, to produce internationally and inter-temporally comparable indicators of innovation. The project would be managed by the Secretariat of the OECD Working Party of National Experts on Science and Technology Indicators (NESTI) -- which includes the European Union and the United States and has observers from the African Union, Brazil, China, and the Russian Federation, the Ibero-American Network on Science and Technology Indicators (RICYT) and the UNESCO Institute of Statistics (UIS). Participants in this proposed activity would be invited to bring together present work on question testing and methodology, as applied to questions in innovation surveys, and to contribute to an agenda for new work. Importance and Broader Significance The intellectual merit of this work arises from the use of leading edge cognitive testing and questionnaire development methods and the application of such methods in the domain of innovation survey questions. Survey design techniques will also benefit from work on a survey instrument that is used in a wide variety of economies and cultures. The work will give rise to a community of practice that will continue well beyond the end of the project -- as part of the NESTI membership, supported by the OECD. The impact of the project would be considerable. A better understanding of innovation and its impacts will provide input to policy development directed at fostering business innovation and entrepreneurship. The project?s final report will include detailed assessments of existing questions and will offer new or improved questions to advance the capture of information on innovation. For NSF?s BRDIS, adding rigorously tested questions and implementing a reviewed structure will contribute a better understanding of business survey methods. And with wide dissemination, survey results will support public debate of the role of innovation in the U.S. economy and inform the development of better designed policies. The report will be also available for use internationally, including for survey programs in developing countries.