How to identify similar scientists based on published abstracts

Question

[Shicheng Guo]

Hi All,

Here is the idea. I want to identify a group of scientists based on their published abstracts. Any R packages?

Thanks

[Clément Frainay]

If I'm getting this right, you want to represent a bunch of authors using the content of their abstracts, then compute similarities between their representation and perform some clustering to get meaningful subgroups? If so , there is probably a way to do that using the PubMed API and some text mining library like gensim and nltk (in python, don't know for R), but I don't know any packages doing the whole pipeline. For example representing authors by the word frequency vectors of their abstract or even embeddings using something like doc2vec, and compute cosine similarities between the vectors, but it will require a bit of work...

[GenoMax]

Why not use PubMed or Google scholar?

I want to identify a group of scientists based on their published abstracts

What is the starting point? Abstracts (keywords within?) or names of people?

[Joe]

Are abstracts really the best way to do this? Surely just getting the list of a given authors frequent coauthors from google scholar or something would be simpler and probably fulfil 90% of the task?

Answer 1

[Mensur Dlakic]

There used to be a service called eTBLAST to do that, but it is discontinued. I have a feeling this type of search is becoming a big business as it is invaluable in identifying reviewers and experts in general.

Don't think that this service is quite as good as eTBLAST, but it should do the trick for simple queries. It will give you experts on the side as part of the text search.

I doubt that there will be a free package, but would be very happy to know that I am wrong.

[Mensur Dlakic]

This could be useful as well.