Scanning for ultra-hot topics in science via pubmed, or using other tools
Entering edit mode
10 weeks ago
LauferVA 3.8k

A search in pubmed for chatGpt returns:

  • 0 articles in 2021
  • 4 articles in 2022
  • 1238 articles (as of Sept 17) in 2023

I am not aware of a more rapid growth in scholarship relating to any scientific idea or advance whatsoever, though there may be one...

My question is, how might one search for such a thing? No restriction on tools that could be used, could use EntrezUtils and BioPython, Google and webscripting, anything. Thoughts?

growth pubmed • 382 views
Entering edit mode
10 weeks ago
GenoMax 136k

Because you are limited to 1000 results via EntrezDirect searches a live option is not going to work. And that may apply to most other technologies you mention above.

You can download the XML PubMed database files from (up to end of of 2022). Or to keep things simpler just 2023 updates here

Parsing the XML files for tiles/abstracts locally should allow you to use a "word cloud" like app to find the information you need.

How ridiculous the live search can be is illustrated by this search for one (or two) days of publications.

$ esearch -db pubmed -query "("2023/01/09"[Date - Publication] : "2023/01/10"[Date - Publication])"

results in 10K+ results.

Looking at the title of I think just first 1K results you can get the most oft appearing words in the titles.

$ esearch -db pubmed -query "("2023/01/09"[Date - Publication] : "2023/01/10"[Date - Publication])" | efetch -format docsum | xtract -pattern DocumentSummary -element Title |  tr ' ' '\n' | sort | uniq -c | sort -k1,1nr > search_res
Entering edit mode

GenoMax , thank you. this is a really great start.

i will likely use a python based approach, but other than that, i think ill proceed exactly as suggested.

with kindness, and thanks,



Login before adding your answer.

Traffic: 1240 users visited in the last hour
Help About
Access RSS

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6