How to get information from the UHGG website by python

Question

[xzhang55]

https://www.ebi.ac.uk/metagenomics/genomes/MGYG000000001#overview

Hi Guys,

I want to get the Genome statistics parts from the webpage and I don't know how to do it by python. I would appreciate it if anyone could help me. I tried by this but the output is nothing.

import requests
from bs4 import BeautifulSoup 
html = requests.get("https://www.ebi.ac.uk/metagenomics/genomes/MGYG000000001#overview")
soup = BeautifulSoup(html.text,"lxml")
# print(soup.prettify())
# print(soup.title)
# print(soup.head)
soup1 = soup.select("div.vf-grid.vf-grid__col-2")
print(soup1)

Answer 1

[Pierre Lindenbaum]

why scrapping when there is a REST API ?

$ wget -q -O - "https://www.ebi.ac.uk/metagenomics/api/v1/genomes?accession=MGYG000000001" | python -m json.tool
{
    "data": [
        {
            "attributes": {
                "accession": "MGYG000000001",
                "completeness": 98.59,
                "contamination": 0.7,
                "eggnog-coverage": 93.78,
                "ena-genome-accession": null,
                "ena-sample-accession": "ERS370061",
                "ena-study-accession": "ERP105624",
                "first-created": "2021-12-07T18:06:34.762463",
                "gc-content": 28.26,
                "genome-id": 4888,
                "geographic-origin": "Europe",
                "geographic-range": [
(...)

[xzhang55]

Hi, I'm not very familiar with the API, could you please give me more details? I would like to download the "data" part as a csv file.

[Pierre Lindenbaum]

https://www.ebi.ac.uk/metagenomics/api/v1/