ncbi geoprofiles web parser

Question

[biokaervas]

I'm doing a little script for return results for this web: http://www.ncbi.nlm.nih.gov/geoprofiles?LinkName=geoprofiles_geoprofiles_prof&from_uid=58016926

My problem is that i only can get 20 results of 200. I know that i can change the items showed per page in the form "display settings". Where there are placed some options:

• Format
• Items per page
• sort by

With different values.

When you click on a POST method this options are sent to the server. I'm trying to figure out what are the parameters sent by this form.

For example, I'm able to change the display settings of "format" adding this post parameter to the URL: http://www.ncbi.nlm.nih.gov/geoprofiles?LinkName=geoprofiles_geoprofiles_prof&from_uid=58016926&format=text

I would like to know the parameter sent by post to change the option 'Items per page'

I have tried with: http://www.ncbi.nlm.nih.gov/geoprofiles?LinkName=geoprofiles_geoprofiles_prof&from_uid=58016926&format=text&items=50 without result.

My question is, how I can know the parameter sent by post that change the "items per page"?

Answer 1

[Neilfws]

Suggestion: don't try to scrape web pages, use the Entrez Programming Utilities.

The elink URI for your query is:

http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=geoprofiles&db=geoprofiles&id=58016926

This returns XML which you can parse. Part of it looks like this:

                <LinkSetDb>
                        <DbTo>geoprofiles</DbTo>
                        <LinkName>geoprofiles_geoprofiles_prof</LinkName>
                        <Link>
                                <Id>58016926</Id>
                        </Link>
                        <Link>
                                <Id>57998590</Id>
                        </Link>

and it contains all the Ids.