How can I get the abstract pubmed articles along with associated mesh terms
0
0
Entering edit mode
11 months ago

Hi guys,

Sorry I am new here and would appreciate it if someone can help me. For my research, I need to retrieve the abstract of papers with keyword Covid-19, coronavirus, ... and save the abstract along with the corresponding mesh terms.

What I have done so far I am able to retrieve the abstracts and KEYWORDS using the below piece of codes. However, later I got to know that the keywords cannot help me because for instance in the keyword I have "covid19", "Covid-19", "learning in pandemic", "learning tools for pandemic events" ,... As you see "covid19", "Covid-19" refer to one thing ---> covid-19 and "learning in pandemic", "learning tools for pandemic events" refers to the same thing---> pandemic and learning.

But as I need to retrieve a lot of data I cannot go through each keyword individually to map them to the same concept. Thats why I thought instead of retrivieing kewords I can use mesh terms (with the hope that it does the same).

But I am not able to get it from pubmed data base!!!

I tried different ways none of them was successful:

1.  Here the mesh are blank

    fetch = PubMedFetcher()

    pmids = fetch.pmids_for_query('covid-19', retmax=10)

    def retrieve():
        path = '.\DeepLearning_Lesson1\pubmedfile'
        while True:
            for pmid in pmids:
                    # abstracts[pmid] = fetch.article_by_pmid(pmid).mesh
                    if fetch.article_by_pmid(pmid):
                        abstract = fetch.article_by_pmid(pmid).abstract
                        mesh = fetch.article_by_pmid(pmid).mesh
                    time.sleep(1)

    t = threading.Thread(target=retrieve)
    t.start()

And then I have tried this:

2. 

    pubmed = PubMed( email="myemail.com")
    results = pubmed.query("coronavirus", max_results=1)

    path = '.\DeepLearning_Lesson1\pubmedfile2'
    import pandas as pd
    def retrieve():
        mydic = {}   
        for article in results:

            js = article.toJSON()

            jso = json.loads(js)
            if jso['abstract'] != None and jso['abstract'] != '' and len(str(jso['pubmed_id'])) <= 10 and 'keywords' in jso:
                if len(jso['keywords']) != 0:
                    with open(os.path.join(path, str(jso['pubmed_id'])+'.txt'), 'w+', encoding='utf-8') as fp:
                        fp.write(jso['abstract'])

                    mydic['pubid'] = str(jso['pubmed_id'])
                    mydic['keywords'] = jso['keywords']

This is the scenario that we can retrieve the keywords associated with the abstracts but as I explained above the keyword cannot help me.

3. I have also tried this, although could get the abstract but the mesh is empty or raising error if I change the code:

    from Bio import Entrez
    Entrez.email = "sgnbx@mail.umkc.edu"     # Always tell NCBI who you are
    pmids = [12654674 ,29058482 ,28991880 ,28984387,28551248]
    handle = Entrez.efetch(db="pubmed", id=','.join(map(str, pmids)),
                       rettype="xml", retmode="text")
    record = Entrez.read(handle)
    abstracts = [pubmed_article['MedlineCitation']['Article']['Abstract']['AbstractText'][0]
                 if 'Abstract' in pubmed_article['MedlineCitation']['Article'].keys()
                 else pubmed_article['MedlineCitation']['Article']['ArticleTitle']
                 for pubmed_article in record['PubmedArticle']]

    mesh = [pubmed_article['MedlineCitation']['MeshHeadingList']['MeshHeading']['DescriptorName'][0]
                 if 'MeshHeading' in pubmed_article['MedlineCitation']['MeshHeadingList'].keys()
                 else None
                 for pubmed_article in record['PubmedArticle']]

Any help is really appreciated.

pubmed mesh terms • 382 views
ADD COMMENT

Login before adding your answer.

Traffic: 1423 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6