KStats Lab › ScholarGauge

Methods and references

How each index is defined, where the idea comes from, and how to read it. Version 1.1.0. Notation: O = observed count, E = expected count, pi = share of label i.

1. Novelty (gap) index, 0–100

Question it answers. Two ideas A and B are each studied in a context C (a field, or the whole of OpenAlex). Are papers that combine A and B rarer than the popularity of A and B would lead us to expect?

Counts

SymbolMeaning (OpenAlex search counts)
n(C)works matching the context C; all works in OpenAlex when no context is given
nA = n(A ∧ C)works matching idea A and the context
nB = n(B ∧ C)works matching idea B and the context
O = n(A ∧ B ∧ C)works matching both ideas and the context (observed)

Alternative spellings typed with ; are joined by OR inside each idea, e.g. ("drone" OR "UAV").

Expected count under independence

If a work in C mentioned A with probability pA = nA/n(C) and, independently, B with probability pB = nB/n(C), the number of works mentioning both would be Binomial(n(C), pApB) with mean

E=n(C)pApB = nAnBn(C) .

The ratio O/E is the classical observed-to-expected measure of co-occurrence: it is the "association strength" of co-word analysis[1],[25] and, for journal pairs, the "commonness" used to score the novelty of papers[9], commonness = Nijt Nt / (Nit Njt).

The index

Because pApB is small, the binomial count is very close to Poisson(E): by Le Cam's inequality[8] the total-variation distance is below n(C)(pApB)2 = E2/n(C). With X ~ Poisson(E),

G=100×P(X>O) =100 [1− ∑k=0O e−EEkk! ] =100(1−plow)

where plow = P(X ≤ O) is the exact one-sided p-value for H0: "A and B are combined at the independence rate" against H1: "they are combined less often". So G ≥ 95 means plow ≤ 0.05 and G ≥ 90 means plow ≤ 0.10.

How to read it

GReading shown by ScholarGauge
≥ 90studied much less than its parts suggest: a likely gap (with O = 0: "not done, although its parts are common")
60–90somewhat less studied than expected
40–60about as studied as its parts suggest
10–40somewhat more studied than expected
≤ 10crowded: studied more than its parts suggest

Counts are whole numbers, so when O equals E the index sits a little below 50 (O = E = 12 gives 42.4). When E < 1 and O = 0, G = 100(1 − e−E) < 64: a missing combination of two rare ideas is weak evidence of a gap, and ScholarGauge says so.

Worked example. Idea A "deep learning", idea B "Mughal painting", context "art history": n(C) = 400, nA = 40, nB = 40, O = 3.
E = 40 × 40 / 400 = 4.00; P(X ≤ 3) = e−4(1 + 4 + 8 + 10.667) = 0.4335; G = 100(1 − 0.4335) = 56.7 — about as expected.
Had no paper combined them (O = 0): G = 100(1 − e−4) = 98.2 — a clear gap. With E = 10 and O = 4, G = 97.1.

In "Has it been done?" for reliability designs

A = the chosen distribution, B = the chosen censoring scheme, each searched together with the chosen model; C = papers on the model (all of OpenAlex if no model is chosen); O = papers with all three. Each dictionary term is searched with its usual spellings joined by OR.

Where the idea comes from

Comparing how often two things occur together with how often they would by chance is the basis of co-word analysis (Callon et al., 1983)[1] and of normalised co-occurrence measures (van Eck and Waltman, 2009)[25]. Novelty studies apply it to pairs of journals cited together: Uzzi et al. (2013)[24] compare observed pair frequencies with randomised citation networks (z-scores), Lee, Walsh and Wang (2015)[9] use the observed-to-expected ratio ("commonness"), and Wang, Veugelers and Stephan (2017)[28] score first-time pairs. The notion of a gap between two literatures that are each well studied but rarely linked goes back to Swanson's "undiscovered public knowledge" (1986)[23].

What is ScholarGauge's own: turning the observed-versus-expected comparison for two ideas into a Poisson tail probability on a 0–100 scale, computed from OpenAlex search counts. It is not a published standard index, so in a paper describe it by its formula (see How to cite).

Limits. (i) Counts are OpenAlex full-text keyword matches (title, abstract and full text where available), so a paper that only mentions a term is counted; O and E share that error. (ii) Ideas are rarely independent: related ideas co-occur more than chance, so a low G does not mean "uninteresting", and a high G may reflect different vocabularies. (iii) The result depends on the context C you choose. (iv) A gap is a lead to read further, not proof of novelty; OpenAlex misses some regional journals and many abstracts.

2. Gaps on the topic map

The topic map reads the N most relevant OpenAlex papers for your query (100 or 200). For two keywords r and c among them, O = papers carrying both, nr and nc = papers carrying each, N = papers read that have keywords, and

E=nrncN

with the same index G. A cell is outlined as a gap when G ≥ 90 and E ≥ 1. The reliability view does the same for distribution × censoring scheme among papers whose design is recognised.

Many cells, many tests. 14 keywords give 91 pairs. Under independence each pair reaches G ≥ 90 with probability at most 0.10, so a few outlined cells can arise by chance; treat them as leads and check each with "Has it been done?", which uses the whole of OpenAlex rather than the papers read.

3. Variety (diversity) indices on a profile

The profile table measures how varied a researcher's papers are along one dimension at a time: OpenAlex topics, subfields, fields and domains; OpenAlex keywords; journals; co-authors; co-author countries; UN SDGs; and, when the reliability dictionary recognises them, distributions, censoring schemes, models and methods. The same indices are used in ecology for species and in scientometrics for disciplines[22].

Fractional counting

Paper j carries a set Lj of labels in the dimension (one primary topic, but possibly several keywords, co-authors or SDGs). Its weight 1 is split equally among its labels, so a paper with many labels does not count more than one with a single label[27]:

wi=∑j 1{i∈Lj}|Lj| , n=∑iwi , pi=win .

n (column "Papers") is the number of papers with at least one label in that dimension.

The columns

ColumnFormulaMeaningSource
KindsS = #{i : wi > 0}richness: number of different labels–
Effective kindsN1 = eH, H = −Σ pi ln pi Shannon entropy H turned into a number of kinds: the number of equally common labels that would give the same H Shannon (1948)[20]; Hill (1973)[5]; Jost (2006)[7]
SimpsonD = 1 − Σ pi2 Gini–Simpson index: chance that two papers drawn at random (with replacement) have different labels Simpson (1949)[21]
EvennessJ = H / ln S (S ≥ 2)Pielou evenness, 0–1; 1 when all labels are equally common Pielou (1966)[14]
Most commonarg max pithe commonest label and its share; ties are reported–

Richness, effective kinds and the inverse Simpson index are the Hill numbers of order 0, 1 and 2,

Dq =(∑ipiq)1/(1−q) : D0=S, D1=eH, D2=1/∑pi2

(1D is the limit as q → 1), so S ≥ eH ≥ 1/Σpi2 always, with equality only when all labels are equally common (Hill, 1973; Jost, 2006)[5],[7]. Effective numbers are easier to compare than entropies: doubling eH means twice as many equally common kinds.

Simpson's own estimator uses ni(ni − 1)/[n(n − 1)] to remove small-sample bias. With fractional counts that correction is not defined, so ScholarGauge uses the plug-in form; for a profile with few papers all indices are descriptive, not estimates of a population value.

Worked example (fractional counting). Three papers: paper 1 has keywords {a, b}, paper 2 has {a}, paper 3 has {c}. Weights: wa = ½ + 1 = 1.5, wb = 0.5, wc = 1; n = 3; p = (0.500, 0.167, 0.333).
S = 3; H = 1.0114, eH = 2.75; D = 1 − (0.25 + 0.028 + 0.111) = 0.61; J = 1.0114 / ln 3 = 0.92.
Two topics shared 2 : 1 by three papers give eH = 1.89, D = 0.44, J = 0.92.

4. Interdisciplinarity (Rao–Stirling diversity)

Variety ignores how different the labels are: two topics in statistics count as much as statistics and history. Rao–Stirling diversity weights each pair of topics by their distance dij:

Δ= ∑i∑j≠i dijpipj

pi = share of the profile's papers whose primary OpenAlex topic is i. This is Rao's quadratic entropy[19] and Stirling's general diversity heuristic Σi≠j dijα(pipj)β with α = β = 1[22], the form used to measure interdisciplinarity in bibliometrics[16],[18],[10].

Distance between two topics

ScholarGauge measures distance on OpenAlex's four-level classification (4 domains › 26 fields › 252 subfields › 4,516 topics)[13], in equal steps up the tree, as in taxonomic distinctness in ecology (Warwick and Clarke, 1995)[29]:

The two topics sharedij
the same subfield0.25
the same field, different subfields0.50
the same domain, different fields0.75
different domains1.00

Bibliometric studies often use 1 − cosine similarity of citation profiles instead (Porter and Rafols, 2009); the tree distance needs no citation data and is the same for every field, which keeps profiles from different disciplines comparable.

Range. Δ = 0 when all papers share one topic. Because dij ≤ 1, Δ ≤ 1 − Σpi2 (the Gini–Simpson index), with equality when all topics lie in different domains; so Δ < 1. Some authors sum over i < j only, which gives half this value; ScholarGauge sums over all ordered pairs, as in Rao (1982).

Worked example. Three papers: two in a statistics topic (field Mathematics) and one in a reliability-engineering topic (field Engineering); both fields belong to the domain Physical Sciences, so d = 0.75. p = (2/3, 1/3) and Δ = (2/3)(1/3)(0.75) + (1/3)(2/3)(0.75) = 0.33. Had the third paper been in a history topic (domain Social Sciences), d = 1 and Δ = 0.44 = the Gini–Simpson index.

5. Exploration

The share of a researcher's later papers that open a topic new to them. Papers are grouped by year; the topics of the first year form the starting set T. For each later year y, in order, every paper of year y whose primary topic is not in T counts as new; then the topics of year y are added to T.

Exploration= number of papers after the first year with a topic new to the authornumber of papers after the first year

Papers of the same year do not count each other as earlier work. 0% means every later paper stayed in topics already worked on; 100% means every later paper opened a new topic. It is a simple descriptive rate in the spirit of studies of topic switching (Zeng et al., 2019)[30] and of tradition versus innovation in research strategies (Foster, Rzhetsky and Evans, 2015)[3], not the measure of either paper.

Example. Topics by year: 2015 {T1}, 2023 {T1}, 2025 {T2}. Later papers: 2 (2023, 2025); new topic: 1 (T2 in 2025). Exploration = 1/2 = 50%.

6. Impact indices, author role and open access on a profile

The third row of cards on a profile. All of them are computed from the papers ScholarGauge has read on the page, so load all papers ("Show more" on Scholar) before analysing; a profile with papers left unloaded gives smaller values than Scholar's own.

Notation

c(1) ≥ c(2) ≥ … ≥ c(n) are the citation counts of the profile's n papers in decreasing order (Google Scholar's counts on a Scholar profile, OpenAlex's in the Android app and on OpenAlex author profiles). Nj is the number of authors of paper j (OpenAlex), and the academic age is a = Y − y0 + 1, where Y is the current year and y0 the year of the first paper.

IndexDefinitionMeaningSource
h-index h=max{k:c(k)≥k} h papers have at least h citations each Hirsch (2005)[6]
g-index g=max{k≤n:∑i=1kc(i)≥k2} the top g papers together have at least g² citations; g ≥ h, and g gives credit to very highly cited papers Egghe (2006)[2]
m-quotient m=h/a h gained per year of academic age; lets researchers at different career stages be compared Hirsch (2005)[6]
hI,norm hI,norm=max{k:c~(k)≥k},c~j=cj/Nj the h-index after sharing each paper's citations equally among its authors; corrects for field differences in team size Harzing, Alakangas and Adams (2014)[4]
hIa hIa=hI,norm/a individual annual h-index: hI,norm per year of academic age; corrects for both team size and career length Harzing, Alakangas and Adams (2014)[4]

The g-index is capped at the number of papers n (Egghe's paper also discusses padding with uncited "fictitious" papers; ScholarGauge does not). hI,norm and hIa use only the papers matched to OpenAlex, where the number of authors is known; the card says how many. Scholar's own h-index and i10-index are shown unchanged in the citations chart below the cards.

Worked example. Five papers with 10, 6, 5, 2 and 1 citations, 2, 1, 5, 3 and 1 authors, the first in 2015; in 2026 the academic age is a = 12.
h: c(3) = 5 ≥ 3 but c(4) = 2 < 4, so h = 3. g: cumulative sums 10, 16, 21, 23, 24 against k² = 1, 4, 9, 16, 25, so g = 4. m = 3/12 = 0.25.
Shared citations 5, 6, 1, 0.67, 1, in order 6, 5, 1, 1, 0.67: c̃(2) = 5 ≥ 2 but c̃(3) = 1 < 3, so hI,norm = 2 and hIa = 2/12 = 0.17.

Citations per year

On a Google Scholar profile the chart and the totals (all years and "since" year) are read from Scholar's own sidebar, unchanged. Scholar draws no bar for a year without citations; ScholarGauge places each bar under the nearest year label, so such a year shows 0.

Author role

For each paper matched to OpenAlex, the profile owner's position in OpenAlex's author list (byline order) is classed as sole (one author), first, last or middle. The owner is recognised by the OpenAlex author ids that match the profile name, or by the name itself. The card gives the share of first-authored papers among papers where the owner was found, and the median number of authors per paper.

Author order does not mean the same everywhere. Alphabetical order is rare in science as a whole (under 4% of papers in 2011) but most common in mathematics, economics (including finance) and high-energy physics (Waltman, 2012)[26]; there the first and last positions say little about who did most of the work.

Open access

OpenAlex gives each work an open-access status[11], following the categories of Piwowar et al. (2018)[15] with "diamond" added:

StatusMeaning
diamondin a fully open-access journal that charges authors no fee
goldin a fully open-access journal
hybridfree under an open licence in a subscription journal
greensubscription only on the publisher's site, but a free copy is in a repository
bronzefree to read on the publisher's site, without an identifiable open licence
closedeverything else

The card shows the share of matched papers that are not closed, and the count of each kind.

7. Data behind the indices

8. How to cite these indices in a paper

Report the formula, the data source and the date of the counts, and cite the method papers. Suggested wording:

"Novelty was measured with the Poisson gap index G = 100·P(X > O), X ~ Poisson(E), where O is the number of OpenAlex works matching both ideas within the context and E = n(A∧C)·n(B∧C)/n(C) is the number expected under independence (cf. van Eck and Waltman, 2009; Lee, Walsh and Wang, 2015), computed with ScholarGauge 1.1.0 (KStats Lab, 2026) from OpenAlex counts retrieved on <date>."

"Research variety was summarised by the effective number of topics eH (Hill, 1973; Jost, 2006), the Gini–Simpson index (Simpson, 1949) and Pielou's evenness (Pielou, 1966) over OpenAlex topics with fractional counting, and interdisciplinarity by Rao–Stirling diversity (Rao, 1982; Stirling, 2007) with taxonomic distances on the OpenAlex topic hierarchy."

"Citation impact was summarised by the h-index (Hirsch, 2005), the g-index (Egghe, 2006), the m-quotient h/a and the author-normalised hI,norm and hIa (Harzing, Alakangas and Adams, 2014), with academic age a counted from the year of the first paper, computed with ScholarGauge 1.1.0 from Google Scholar citation counts retrieved on <date>."

9. References

  1. Callon, M., Courtial, J.-P., Turner, W. A., and Bauin, S. (1983). From translations to problematic networks: An introduction to co-word analysis. Social Science Information, 22(2), 191–235. doi:10.1177/053901883022002003
  2. Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152. doi:10.1007/s11192-006-0144-7
  3. Foster, J. G., Rzhetsky, A., and Evans, J. A. (2015). Tradition and innovation in scientists' research strategies. American Sociological Review, 80(5), 875–908. doi:10.1177/0003122415601618
  4. Harzing, A.-W., Alakangas, S., and Adams, D. (2014). hIa: An individual annual h-index to accommodate disciplinary and career length differences. Scientometrics, 99(3), 811–821. doi:10.1007/s11192-013-1208-0
  5. Hill, M. O. (1973). Diversity and evenness: A unifying notation and its consequences. Ecology, 54(2), 427–432. doi:10.2307/1934352
  6. Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569–16572. doi:10.1073/pnas.0507655102
  7. Jost, L. (2006). Entropy and diversity. Oikos, 113(2), 363–375. doi:10.1111/j.2006.0030-1299.14714.x
  8. Le Cam, L. (1960). An approximation theorem for the Poisson binomial distribution. Pacific Journal of Mathematics, 10(4), 1181–1197. doi:10.2140/pjm.1960.10.1181
  9. Lee, Y.-N., Walsh, J. P., and Wang, J. (2015). Creativity in scientific teams: Unpacking novelty and impact. Research Policy, 44(3), 684–697. doi:10.1016/j.respol.2014.10.007
  10. Leydesdorff, L., Wagner, C. S., and Bornmann, L. (2019). Interdisciplinarity as diversity in citation patterns among journals: Rao-Stirling diversity, relative variety, and the Gini coefficient. Journal of Informetrics, 13(1), 255–269. doi:10.1016/j.joi.2018.12.006
  11. OpenAlex (2026). Open access. OpenAlex help pages. help.openalex.org/data/works/open-access (accessed 13 November 2026).
  12. OpenAlex (2026). Sustainable Development Goals. OpenAlex help pages. help.openalex.org/data/sdgs (accessed 28 October 2026).
  13. OpenAlex (2026). Topics. OpenAlex help pages. help.openalex.org/data/topics (accessed 28 October 2026).
  14. Pielou, E. C. (1966). The measurement of diversity in different types of biological collections. Journal of Theoretical Biology, 13, 131–144. doi:10.1016/0022-5193(66)90013-0
  15. Piwowar, H., Priem, J., Larivière, V., Alperin, J. P., Matthias, L., Norlander, B., Farley, A., West, J., and Haustein, S. (2018). The state of OA: A large-scale analysis of the prevalence and impact of Open Access articles. PeerJ, 6, e4375. doi:10.7717/peerj.4375
  16. Porter, A. L., and Rafols, I. (2009). Is science becoming more interdisciplinary? Measuring and mapping six research fields over time. Scientometrics, 81(3), 719–745. doi:10.1007/s11192-008-2197-2
  17. Priem, J., Piwowar, H., and Orr, R. (2022). OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. arXiv preprint. arXiv:2205.01833
  18. Rafols, I., and Meyer, M. (2010). Diversity and network coherence as indicators of interdisciplinarity: Case studies in bionanoscience. Scientometrics, 82(2), 263–287. doi:10.1007/s11192-009-0041-y
  19. Rao, C. R. (1982). Diversity and dissimilarity coefficients: A unified approach. Theoretical Population Biology, 21(1), 24–43. doi:10.1016/0040-5809(82)90004-1
  20. Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379–423. doi:10.1002/j.1538-7305.1948.tb01338.x
  21. Simpson, E. H. (1949). Measurement of diversity. Nature, 163(4148), 688. doi:10.1038/163688a0
  22. Stirling, A. (2007). A general framework for analysing diversity in science, technology and society. Journal of the Royal Society Interface, 4(15), 707–719. doi:10.1098/rsif.2007.0213
  23. Swanson, D. R. (1986). Fish oil, Raynaud's syndrome, and undiscovered public knowledge. Perspectives in Biology and Medicine, 30(1), 7–18. doi:10.1353/pbm.1986.0087
  24. Uzzi, B., Mukherjee, S., Stringer, M., and Jones, B. (2013). Atypical combinations and scientific impact. Science, 342(6157), 468–472. doi:10.1126/science.1240474
  25. van Eck, N. J., and Waltman, L. (2009). How to normalize cooccurrence data? An analysis of some well-known similarity measures. Journal of the American Society for Information Science and Technology, 60(8), 1635–1651. doi:10.1002/asi.21075
  26. Waltman, L. (2012). An empirical analysis of the use of alphabetical authorship in scientific publishing. Journal of Informetrics, 6(4), 700–711. doi:10.1016/j.joi.2012.07.008
  27. Waltman, L., and van Eck, N. J. (2015). Field-normalized citation impact indicators and the choice of an appropriate counting method. Journal of Informetrics, 9(4), 872–894. doi:10.1016/j.joi.2015.08.001
  28. Wang, J., Veugelers, R., and Stephan, P. (2017). Bias against novelty in science: A cautionary tale for users of bibliometric indicators. Research Policy, 46(8), 1416–1436. doi:10.1016/j.respol.2017.06.006
  29. Warwick, R. M., and Clarke, K. R. (1995). New 'biodiversity' measures reveal a decrease in taxonomic distinctness with increasing stress. Marine Ecology Progress Series, 129, 301–305. doi:10.3354/meps129301
  30. Zeng, A., Shen, Z., Zhou, J., Fan, Y., Di, Z., Wang, Y., Stanley, H. E., and Havlin, S. (2019). Increasing trend of scientists to switch between topics. Nature Communications, 10, 3439. doi:10.1038/s41467-019-11401-8

Every reference was checked against its Crossref record (DOI) or, for web pages and the arXiv preprint, against the page itself: on 28 October 2026, and the references added in version 1.1 on 13 November 2026.