How to scrape BioMart data from https://sorfs.ugent.be/ website
1
0
Entering edit mode
20 days ago
QX ▴ 20

Hi all,

I would like to download database from this website: https://sorfs.ugent.be/database but it seems that they do not maintain the database anymore, only the table available.

Is there anyway to pull the table from this website as a csv file?

Best,

BioMart • 466 views
ADD COMMENT
0
Entering edit mode

did you ask the authors to restore their server ?

ADD REPLY
0
Entering edit mode

yes but they did not reply

ADD REPLY
2
Entering edit mode
20 days ago

something like:

seq 1 100 4377380 | while read F ; do wget   --no-check-certificate  -O - "https://sorfs.ugent.be/database/micropep?sEcho=4&iColumns=18&sColumns=%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C&iDisplayStart=${F}&iDisplayLength=100&mDataProp_0=0&sSearch_0=&bRegex_0=false&bSearchable_0=true&bSortable_0=true&mDataProp_1=1&sSearch_1=&bRegex_1=false&bSearchable_1=true&bSortable_1=true&mDataProp_2=2&sSearch_2=&bRegex_2=false&bSearchable_2=true&bSortable_2=true&mDataProp_3=3&sSearch_3=&bRegex_3=false&bSearchable_3=true&bSortable_3=true&mDataProp_4=4&sSearch_4=&bRegex_4=false&bSearchable_4=true&bSortable_4=true&mDataProp_5=5&sSearch_5=&bRegex_5=false&bSearchable_5=true&bSortable_5=true&mDataProp_6=6&sSearch_6=&bRegex_6=false&bSearchable_6=true&bSortable_6=true&mDataProp_7=7&sSearch_7=&bRegex_7=false&bSearchable_7=true&bSortable_7=true&mDataProp_8=8&sSearch_8=&bRegex_8=false&bSearchable_8=true&bSortable_8=true&mDataProp_9=9&sSearch_9=&bRegex_9=false&bSearchable_9=true&bSortable_9=true&mDataProp_10=10&sSearch_10=&bRegex_10=false&bSearchable_10=true&bSortable_10=true&mDataProp_11=11&sSearch_11=&bRegex_11=false&bSearchable_11=false&bSortable_11=true&mDataProp_12=12&sSearch_12=&bRegex_12=false&bSearchable_12=false&bSortable_12=true&mDataProp_13=13&sSearch_13=&bRegex_13=false&bSearchable_13=false&bSortable_13=true&mDataProp_14=14&sSearch_14=&bRegex_14=false&bSearchable_14=false&bSortable_14=true&mDataProp_15=15&sSearch_15=&bRegex_15=false&bSearchable_15=false&bSortable_15=true&mDataProp_16=16&sSearch_16=&bRegex_16=false&bSearchable_16=false&bSortable_16=true&mDataProp_17=17&sSearch_17=&bRegex_17=false&bSearchable_17=false&bSortable_17=true&sSearch=&bRegex=true&iSortCol_0=0&sSortDir_0=asc&iSortingCols=1&_=1716563877578" | python -m json.tool ; done
{
    "aaData": [
        [
            "<a href = 'database/andreev_2015:100170' >andreev_2015:100170</a>",
            "<a href='http://www.ensembl.org/Multi/Search/Results?q=ENST00000405356;y=0;site=ensembl_all;x=0;page=1;facet_feature_type=Gene'>ENST00000405356</a>"
,
            "human",
            "andreev_2015",
            "10",
            102157437,
            102157610,
            58,
            "exonic",
            "protein_coding",
            "yes",
            "0.4749",
            "1.5047",
            "MPSESVCLLGRLQPKHQRVAAVKSPVMMMMRRTKRNSLSRFAALGRKRGLRIRKEET*",
            "CTGCCAAGCGAGTCGGTCTGCCTCCTGGGAAGGCTGCAGCCAAAGCATCAGAGAGTAGCAGCAGTGAAGAGTCCAGTGATGATGATGATGAGGAGGACCAAAAGAAACAGCCTGTCCAGGTTTGCAGCTTTGGGAAGAAAAAGGGGTT
TAAGGATTAGAAAGGAAGAAACCTAA",
            "0.357142857143",
            "-9.23678239084",
            "ENSG00000166197"
        ],
        [
            "<a href = 'database/andreev_2015:100182' >andreev_2015:100182</a>",
            "<a href='http://www.ensembl.org/Multi/Search/Results?q=ENST00000405356;y=0;site=ensembl_all;x=0;page=1;facet_feature_type=Gene'>ENST00000405356</a>"
,
            "human",
            "andreev_2015",
            "10",
            102158107,
            102158169,
            21,
            "exonic",
            "protein_coding",
            "yes",
            "0.8899",
            "2.4239",
            "MILILTQAPRMSHQRTRSQR*",
            "CTGATTCTGATTCTGACTCAAGCTCCGAGGATGAGCCACCAAAGAACCAGAAGCCAAAGATAA",
            "0.368421052632",
            "-7.55004712725",
ADD COMMENT
0
Entering edit mode

thank you so much!

ADD REPLY
1
Entering edit mode

Don't forget to follow up on your threads. If an answer was helpful, you should upvote it; if the answer resolved your question, you should mark it as accepted. You can accept more than one answer if they all work. If an answer was not really helpful or did not work, provide detailed feedback so others know not to use that answer.

Upvote|Bookmark|Accept

ADD REPLY

Login before adding your answer.

Traffic: 1887 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6