Question: Anyone Using "Biomart + Java Web Services" ?
10
gravatar for Pierre Lindenbaum
8.9 years ago by
France/Nantes/Institut du Thorax - INSERM UMR1087
Pierre Lindenbaum116k wrote:

Hi all, Thanks to Neil, I've discovered that Biomart is not just a HTML front-end but it can also be invoked as a web service.

So, I played with java and the following webservice: http://www.ensembl.org/biomart/martwsdl .

${JAVA_HOME}/bin/wsimport http://www.ensembl.org/biomart/martwsdl

was called to generate the java sources from the WSDL file and I also wrote the following client:

import org.ensembl._80.martservicesoap.*;

public class BioMartClient
 {
 public static void  main(String args[]) throws Exception
  {
  BioMartSoapService service=new BioMartSoapService();
  MartServiceSoap martsoap= service.getBioMartSoapPort();
  for(Mart mart: martsoap.getRegistry())
   {
   System.out.println(mart.getName());
   }
  }
 }

but the program throwed the following exception when it invokes getRegistry():

Exception in thread "main" com.sun.xml.internal.ws.server.UnsupportedMediaException: Unsupported Content-Type: text/html; charset=utf-8 Supported ones are: [text/xml]
        at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.decode(StreamSOAPCodec.java:284)
(...)
        at $Proxy30.getRegistry(Unknown Source)
        at BioMartClient.main(BioMartClient.java:9)

I also tested a few more BioMart servers: Sometimes getRegistry just returns an empty list (http://www.biomart.org/biomart/martwsdl ), sometimes the path martwsdl does not exist ( http://www.wormbase.org/biomart/martwsdl ), etc...

In the end I didn't find any server containing some data that could be processed via the SOAP protocol.

Is it a known issue ? Am I missing something ? Is there a functional BioMart+SOAP server anywhere ?

Thanks,

Update: I've tested it again today (16 jan 2012) http://www.biomart.org/biomart/martwsdl : there was no error but the response for getRegistry is empty.

java webservice biomart • 4.9k views
ADD COMMENTlink modified 5 months ago by RamRS20k • written 8.9 years ago by Pierre Lindenbaum116k

Actually, I see we are using different marts, could you try the 'real' mart at http://www.biomart.org/biomart/martwsdl ?

ADD REPLYlink written 8.9 years ago by Michael Dondrup45k

Just sent my message to mart-dev, let's see what comes out of it.

ADD REPLYlink written 8.9 years ago by Michael Dondrup45k

I wanted to use a telephone, but was told to use a walkie-talkie instead... For now, no chance for the SOAP service, have to try the REST service instead. The developers say, they are working on porting BioMart to Java, which I appreciate.

ADD REPLYlink written 8.9 years ago by Michael Dondrup45k

I saw your messages on the mart-dev mailing Michael. Thanks for that. Yes, some answers were weird "your java thing cannot understand our service ? solution is: use REST..." :-/

ADD REPLYlink written 8.9 years ago by Pierre Lindenbaum116k
4
gravatar for Michael Dondrup
8.9 years ago by
Bergen, Norway
Michael Dondrup45k wrote:

Edit: Conclusion: It does not work and is unlikely to work without changing the service.

In the meantime, I would like to ask everyone who has similar experiences with non-functional SOAP/REST web-services, non-parsable WSDLS, non-validating XML-messages or other problems with service interoperability not to give up, but to report to the providers, to challenge them to improve the interoperability!

Those experiencing similar problems with BioMart can for example report to: mart-dev@ebi.ac.uk

Testing bioinformatics web-services can sometimes be a frustrating experience. It looks as if all the specs of interoperability are not worth the (white-) papers they are printed on. I actually have had very little success in connecting to perl, SOAP-lite based web-services like BioMart, Kegg, etc.

Actually, I tried to use a generated Axis2 client and got an exception. Then, I tried to validate the response message in SoapUI from BioMart, and it does not validate!

line 4: Expected element 'mart' instead of 'mart@http://www.biomart.org:80/MartServiceSoap' here in element getRegistryResponse@http://www.biomart.org:80/MartServiceSoap

So it's very unlikely that this is going to work at all, except with perl and SOAP::Lite. There are too many tiny glitches involved. To clean up this mess I intend to contact the Bimart developers and ask them to change their interface. In fact, it seems that nobody has tested that yet with Java, otherwise somebody would have noticed that it does not work.

Maybe, you wish to join me in this effort?


Hi Pierre,

actually it would surprise me if there were no problems. This reminds me of some experiences I encountered when I tried to use the highly appraised standardized SOAP web-services world (actually I wanted to ask a question about user's experiences with bioinfo services but didn't get that far). So, it is a bit like the famous radio yerevan, "In principle it's standardized, but everyone has different standards."

However, and not to just bash the providers, I can remember that we had successfully accessed biomart SOAP web-services with SoapUI. That tool is great for testing and building messages. Alternatively try a different SOAP stack like Axis2 to generate the binding.

I might give it a try and generate a simple Axis2 client, then I will edit and provide the java code. So long there could be something wrong with the way you are trying to use the library. Also, Biomart is sensitive to the exact workflow of service calls.

This is what I just tried in soapui: with SoapUI load the wsdl at: http://www.biomart.org/biomart/martwsdl With other tools (eg curl, see comments), send the xml-message below to the service endpoint address: http://www.biomart.org/biomart/martsoap

then call "getRegistry" as you did with the following message:

<soapenv:Envelope xmlns:soapenv="&lt;a href=" http:="" schemas.xmlsoap.org="" soap="" envelope="" "="" rel="nofollow">http://schemas.xmlsoap.org/soap/envelope/" xmlns:mar="http://www.biomart.org:80/MartServiceSoap">

   <soapenv:Header/>

   <soapenv:Body>

      <mar:getRegistry/>
   </soapenv:Body>
</soapenv:Envelope>

Then you will get this response, which looks very much ok.

<soap:Envelope soap:encodingStyle="&lt;a href=" http:="" schemas.xmlsoap.org="" soap="" encoding="" "="" rel="nofollow">http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <getRegistryResponse xmlns="&lt;a href=" http:="" www.biomart.org:80="" MartServiceSoap"="" rel="nofollow">http://www.biomart.org:80/MartServiceSoap">
         <mart>
            <name xsi:type="xsd:string">ensembl</name>
            <displayName xsi:type="xsd:string">ENSEMBL GENES 57 (SANGER UK)</displayName>
            <database xsi:type="xsd:string">ensembl_mart_57</database>
            <host xsi:type="xsd:string">www.biomart.org</host>
            /biomart/martservice
            <port xsi:type="xsd:string">80</port>
            <visible xsi:type="xsd:int">1</visible>
            <default xsi:type="xsd:int">1</default>
            <serverVirtualSchema xsi:type="xsd:string">default</serverVirtualSchema>
            <includeDatasets xsi:type="xsd:string"/>
            <martUser xsi:type="xsd:string"/>
            <redirect xsi:nil="true" xsi:type="xsd:int"/>
         </mart>
ADD COMMENTlink modified 5 months ago by RamRS20k • written 8.9 years ago by Michael Dondrup45k

many thanks Michael, I will try with another tool. wsimport works fine with the services created with ${JAVA_HOME}/bin/wsgen . For example I tested it with EBI/Intact http://plindenbaum.blogspot.com/2008/10/ebiintact-web-service-api-my-notebook.html

ADD REPLYlink written 8.9 years ago by Pierre Lindenbaum116k

Michael, a small typo; your URL should be http://www.biomart.org/biomart/martsoap to martwsdl

ADD REPLYlink written 8.9 years ago by Pierre Lindenbaum116k

Hum, I tried CXF/wsdl2java : same (empty) result

ADD REPLYlink written 8.9 years ago by Pierre Lindenbaum116k

Michael, a small typo; your URL should be biomart.org/biomart/martsoap , not martwsdl

ADD REPLYlink written 8.9 years ago by Pierre Lindenbaum116k

but, as you said

curl --header 'Content-Type: text/xml' --data '[?]
[?]
   [?]
      [?]
   [?]
[?]
' http://www.biomart.org/biomart/martsoap

is OK :-/

ADD REPLYlink modified 5 months ago by RamRS20k • written 8.9 years ago by Pierre Lindenbaum116k

correct: biomart.org/biomart/martsoap is the service endpoint, while /martwsdl is the wsdl location. Whith soapui, you just put in the wsdl and the rest is automatic. Good luck!

ADD REPLYlink written 8.9 years ago by Michael Dondrup45k

i asked for help on SO : http://stackoverflow.com/questions/2479069

ADD REPLYlink written 8.9 years ago by Pierre Lindenbaum116k

I got the anwser from the mart mailing list: http://www.mail-archive.com/mart-dev@ebi.ac.uk/msg02406.html ... hum....

ADD REPLYlink written 8.9 years ago by Pierre Lindenbaum116k

I saw it, looks as if they didn't get what the problem is. I will write to the mart_dev with an in-depth description of the problems in the service and wsdl. That might need a little more time.

ADD REPLYlink written 8.9 years ago by Michael Dondrup45k
2
gravatar for Michael Dondrup
8.9 years ago by
Bergen, Norway
Michael Dondrup45k wrote:

Hi again, I was finally able to edit the received message such that it can be validated. The biomart reponse semed to screw up namespace definitions and they use XML-schema instance. :o) The schema instance definitions had no obvious use and did not validate. So I just removed them. If you look at that w3c site you see that W3C says this should never be used. So compare the valid edited reponse massage with the real (but shortened) output. The next step is to make the maintainers have their service send valid XML Here is the edited and schema-valid response, only the first mart given:

<soap:Envelope soap:encodingStyle="&lt;a href=" http:="" schemas.xmlsoap.org="" soap="" encoding="" "="" rel="nofollow">http://schemas.xmlsoap.org/soap/encoding/"

xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" 
xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
xmlns:tns="http://www.biomart.org:80/MartServiceSoap"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <tns:getRegistryResponse>
         <mart>
            <name>ensembl</name>
            <displayName>ENSEMBL GENES 57 (SANGER UK)</displayName>
            <database>ensembl_mart_57</database>
            <host>www.biomart.org</host>
            /biomart/martservice
            <port>80</port>
            <visible>1</visible>
            <default>1</default>
            <serverVirtualSchema>default</serverVirtualSchema>
            <includeDatasets/>
            <martUser/>
            <redirect>1</redirect>
         </mart>

      </tns:getRegistryResponse>
   </soap:Body>
</soap:Envelope>

And compare this with the original response:

<soap:Envelope soap:encodingStyle="&lt;a href=" http:="" schemas.xmlsoap.org="" soap="" encoding="" "="" rel="nofollow">http://schemas.xmlsoap.org/soap/encoding/"     
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" 
xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Body>
      <getRegistryResponse xmlns="&lt;a href=" http:="" www.biomart.org:80="" MartServiceSoap"="" rel="nofollow">http://www.biomart.org:80/MartServiceSoap">
         <mart>
            <name xsi:type="xsd:string">ensembl</name>
            <displayName xsi:type="xsd:string">ENSEMBL GENES 57 (SANGER UK)</displayName>
            <database xsi:type="xsd:string">ensembl_mart_57</database>
            <host xsi:type="xsd:string">www.biomart.org</host>
            /biomart/martservice
            <port xsi:type="xsd:string">80</port>
            <visible xsi:type="xsd:int">1</visible>
            <default xsi:type="xsd:int">1</default>
            <serverVirtualSchema xsi:type="xsd:string">default</serverVirtualSchema>
            <includeDatasets xsi:type="xsd:string"/>
            <martUser xsi:type="xsd:string"/>
            <redirect xsi:nil="true" xsi:type="xsd:int"/>
         </mart>
      </getRegistryResponse>
   </soap:Body>
</soap:Envelope>
ADD COMMENTlink modified 5 months ago by RamRS20k • written 8.9 years ago by Michael Dondrup45k

Thanks Michael !

ADD REPLYlink written 8.9 years ago by Pierre Lindenbaum116k
1
gravatar for Manu Prestat
7.1 years ago by
Manu Prestat3.9k
Marseille, France
Manu Prestat3.9k wrote:

The R/BioConductor library biomaRt is also pretty useful for R lovers.

ADD COMMENTlink written 7.1 years ago by Manu Prestat3.9k
1
gravatar for Gorka
6.3 years ago by
Gorka10
Spain
Gorka10 wrote:

Hi all,

I have problems too using the WSDL at http://www.biomart.org/biomart/martwsdl. In my case I get an empty List<ResultsRow> in the code after MartServiceSoap.query(), despite the results are actually returned by the server as I have confirmed using Wireshark.

Finally, I've been able to use the service using the WSDL at http://central.biomart.org/martsoap?wsdl. With this WSDL there is a "raw" PortalServiceImpl.getResults() method that accepts the query as XML and returns all the results as a string.

Here there is the automatically generated code for consuming the WSDL:
https://code.google.com/p/s-chpp/source/browse/trunk/java/src/main/java/#java%2Forg%2Fbiomart%2Fapi%2Fsoap

And here how I have managed to get the results:
https://code.google.com/p/s-chpp/source/browse/trunk/java/src/main/java/org/schpp/db/Ensembl.java

I hope that this service support is continued since it is very useful!

ADD COMMENTlink written 6.3 years ago by Gorka10
0
gravatar for Steve Moss
7.1 years ago by
Steve Moss2.2k
United Kingdom
Steve Moss2.2k wrote:

The MartService is pretty cool, I discovered it while doing research for my placement at the EBI (see below).

I am/have been working on an EnsEMBL RESTful web service with the EnsEMBL software development team. See these posts from my blog:

I implemented it using Perl, Catalyst and JSON, which I think is the way to go?

There is currently no time-frame for completion/public release, but I'll keep things updated on my blog and on here as much as I can. It is something that I am doing around my PhD, and as such my time is limited.

Any recommendations, or ideas on the best way to provide such as service should be appreciated. The prototype seemed to work well, but I need to look at security, caching and other performance issues etc.

ADD COMMENTlink modified 7.1 years ago • written 7.1 years ago by Steve Moss2.2k

thanks, but I was more interested by the WSDL/SOAP API (which doesn't work)

ADD REPLYlink written 7.1 years ago by Pierre Lindenbaum116k

Cool, no problem!

ADD REPLYlink written 7.1 years ago by Steve Moss2.2k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 849 users visited in the last hour