While normalized bibliometric indicators are expected to resolve the subject-field differences between organizations in research evaluations, the identification of reference organizations working on similar researchtopics is still of importance. Research organizations, policymakers and research funders tend to use benchmark units as points of comparison for a certain research unit in order to understand and monitor its development and performance. In addition, benchmarkorganizations can also be used to pinpoint potential collaboration partners or competitors. Therefore, methods for identifying benchmark research units are of practical significance. Even so, few studies have further explored this problem. This study aims to propose a bibliometric approach for the identification of benchmark units. We define an appropriate benchmark as a well-connected research environment, in which researchers investigate similar topics and publish a similar number of publications compared to a given research organization during the same period. Four essential attributes for the evaluation of benchmarks are research topics, output, connectedness, and scientific impact. We apply this strategy to two research organizations in Sweden and examine the effectiveness of the proposed method. Identified benchmark units are evaluated by examining the research similarity and the robustness of various measures of connectivity.

While normalized bibliometric indicators are expected to resolve the subject-field differences between organizations in research evaluations, size still matters. Furthermore, research organizations, policymakers and research funding providers tend to use benchmark units as points of comparison for a given research center in order to understand and monitor its development and performance. In addition to monitoring and evaluations, the identification of comparable benchmark organizations can also be used to pinpoint potential collaboration partners or competitors. Therefore, methods to identify benchmark research units are of practical significance. However, few studies have investigated this problem. This study aims to propose a bibliometric method to identify benchmarks. We define a benchmark as a well-connected research environment, in which researchers work on similar topics and publish a similar number of publications compared to a given research center during the same period. Three essential attributes for the evaluation of benchmarks are research topics, output, and coherence. We apply this strategy to a Swedish research center, and examine the effectiveness of the method.

Measuring interdisciplinarity is a pertinent but challenging issue in quantitative studies of science. There seems to be a consensus in the literature that the concept of interdisciplinarity is multifaceted and ambiguous. Unsurprisingly, various different measures of interdisciplinarity have been proposed. However, few studies have thoroughly examined the validity and relations between these measures. In this study, we present a systematic review of these interdisciplinarity measures and explore their inherent relations. We examine these measures in relation to the Web of Science journal subject categories. Our results corroborate recent claims that the current measurements of interdisciplinarity in science studies are both confusing and unsatisfying. We find surprisingly deviant results when comparing measures that supposedly should capture similar features or dimensions of the concept of interdisciplinarity. We therefore argue that the current measurements of interdisciplinarity should be interpreted with much caution in science and evaluation studies, or in relation to science policies. We also question the validity of current measures and argue that we do not need more of the same, but rather something different in order to be able to measure the multidimensional and complex construct of interdisciplinarity.

Detecting emerging research topics is essential, not only for research agencies but also for individual researchers. Previous studies have created various bibliographic indicators for the identification of emerging research topics. However, as indicated by Rotolo et al. (Research Policy 44, 1827-1843, ), the most serious problems are the lack of an acknowledged definition of emergence and incomplete elaboration of the linkages between the definitions that are used and the indicators that are created. With these issues in mind, this study first adjusts the definition of an emerging technology that Rotolo et al. (2015) have proposed to accommodate the analysis. Next, a set of criteria for the identification of emerging topics is proposed according to the adjusted definition and attributes of emergence. Using two sets of parameter values, several emerging research topics are identified. Finally, evaluation tests are conducted by demonstration of the proposed approach and comparison with previous studies. The strength of the present methodology lies in the fact that it is fully transparent, straightforward, and flexible.

Journal classification systems play an important role in bibliometric analyses. The two most important bibliographic databases, Web of Science and Scopus, each provide a journal classification system. However, no study has systematically investigated the accuracy of these classification systems. To examine and compare the accuracy of journal classification systems, we define two criteria on the basis of direct citation relations between journals and categories. We use Criterion I to select journals that have weak connections with their assigned categories, and we use Criterion II to identify journals that are not assigned to categories with which they have strong connections. If a journal satisfies either of the two criteria, we conclude that its assignment to categories may be questionable. Accordingly, we identify all journals with questionable classifications in Web of Science and Scopus. Furthermore, we perform a more in-depth analysis for the field of Library and Information Science to assess whether our proposed criteria are appropriate and whether they yield meaningful results. It turns out that according to our citation-based criteria Web of Science performs significantly better than Scopus in terms of the accuracy of its journal classification system.

The dynamic nature of science is embodied in the growth of knowledge in magnitude and the transformation of knowledge in structure. More specifically, the growth in magnitude is indicated by a sharp increase in the number of scientific publications in recent decades. The transformation of knowledge occurs as the boundaries of scientific disciplines become increasingly less distinct, resulting in a complicated situation wherein disciplines and interdisciplinary research topics coexist and co-evolve. Knowledge production in such a context creates challenges for the measurement of science.This thesisaims to develop more flexible bibliometric methodologies in order to address some of the challenges to measuring science effectively. To be specific, this thesis1) proposes a new approach for identifying emerging research topics; 2) measuresthe interdisciplinarity of research topics; 3) explores the accuracy of the journal classification systems of the Web of Science and Scopus; 4) examines the role of cognitive distance in grant decisions; and 5) investigates the effect of cognitive distance between collaborators on their research output. The data used in this thesisaremainly from the in-house Web of Science and Scopus databases of the Centre for Science and Technology Studies (CWTS) at Leiden University. Quantitativeanalyses, in particular bibliometric analyses,are the main research methodologies employed in this thesis. Furthermore, this thesis primarily offers methodological contributions, proposing a series of approaches designed to tackle the challenges created by the dynamics of science. While the major contribution of this dissertation lies in the improvement of certain bibliometric approaches, it also enhances the understanding of the current system of science. In particular, the approaches and research findings presented here have implications for various stakeholders, including publishing organizations, bibliographic database producers, research policy makers, and research funding agencies. Indeed, these approaches could be built into a software tool and thereby be made available to researchers beyond the field of bibliometric studies.

The aim of this paper is twofold: (1) to provide a methodology formeasurement ofcognitive distance between researchers and (2) to explorethe role of cognitive distance on the results of peer review processes. Citedreferencesand the contentof articlesare used to represent their respective scientific knowledge bases. Based on the two different approaches—Author-Bibliographic Coupling analysis andAuthor-Topic analysis—we apply the methodologyon a recent competition for grants from the Swedish Strategic Foundation.Results indicate thatcognitive distances between applicants and reviewersmight influence peer review results, but that the impact is to some extent at the unexpectedend. The main contribution of this paper is the elaborationon the relevance of the concept of cognitive distance to the issue of research evaluationingeneral, and especially in relation to peer review as a model used in grant decisions.

Identifying interdisciplinary research topics is an essential subject, not only for research policy but also research funding agencies. Previous research was constructed on measuring interdisciplinarity mainly at the macro level of research, such as Web of Science subject category and journal. However, these studies lack analysis at the micro level of the current science system. It means few studies have analyzed interdisciplinarity at the level of publications. To cover this gap, we introduce an approach for measuring interdisciplinarity at the level of micro research topics. The research topics are clustered by direct citation relations in a large scale database. According to the characteristics of boundary-crossing research, we provide an alternative approach to measure interdisciplinarity. Comparing with the widely used Rao-Stirring indicator (Integration score), we found that the results obtained by two indicators of interdisciplinarity have a strong correlation, thus we believe that this approach could effectively identify boundary-crossing research topics.