Genscan Server Query Through Java

Question

[Chami]

Hi , I am trying to access the GenScan server @ MIT to get gene predictions automated through java. Here is the code I managed to come up with. ( lot of it is cut & paste and me modifying )

public class POST_GenScan {

/**
 * @param args the command line arguments
 */
public static void getDATA() throws FileNotFoundException, UnsupportedEncodingException, IOException {


    String charset = "utf-8";

    String param = "value";
    //String boundary = "-----------------------------7db271286025a"; // Just generate some unique random value.
    String CRLF = "\r\n"; // Line separator required by multipart/form-data.
    String url = "http://genes.mit.edu/cgi-bin/genscanw_py.cgi";



    String boundary = Long.toHexString(System.currentTimeMillis());
    boundary = "--" + boundary;

    URLConnection connection = null;
    try {
        connection = new URL(url).openConnection();
    } catch (IOException ex) {
        Logger.getLogger(POST_GenScan.class.getName()).log(Level.SEVERE, null, ex);
    }
    connection.setDoOutput(true);
    connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
    connection.setRequestProperty("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.3");
    connection.setRequestProperty("Accept-Language", "en-US,en;q=0.8");


    PrintWriter writer = null;
    try {
        OutputStream output = null;
        try {
            output = connection.getOutputStream();
        } catch (IOException ex) {
            Logger.getLogger(POST_GenScan.class.getName()).log(Level.SEVERE, null, ex);
        }
        try {
            writer = new PrintWriter(new OutputStreamWriter(output, charset), true); // true = autoFlush, important!
        } catch (UnsupportedEncodingException ex) {
            Logger.getLogger(POST_GenScan.class.getName()).log(Level.SEVERE, null, ex);
        }

        // Send normal param.
        writer.append(boundary).append(CRLF);
        writer.append("Content-Disposition: form-data; name=\"-o\"").append(CRLF);
        writer.append(CRLF);
        writer.append("Vertebrate").append(CRLF);
        writer.append(boundary).append(CRLF).flush();

        writer.append("Content-Disposition: form-data; name=\"-e\"").append(CRLF);
        writer.append(CRLF);
        writer.append("1.00").append(CRLF);
        writer.append(boundary).append(CRLF).flush();

        writer.append("Content-Disposition: form-data; name=\"-n\"").append(CRLF);
        writer.append(CRLF);
        writer.append(CRLF);
        writer.append(boundary).append(CRLF).flush();

        writer.append("Content-Disposition: form-data; name=\"-p\"").append(CRLF);
        writer.append(CRLF);
        writer.append("Predicted peptides only").append(CRLF);
        writer.append(boundary).append(CRLF).flush();

        writer.append("--" + boundary).append(CRLF);
        writer.append("Content-Disposition: form-data; name=\"textFile\"; filename=\"\"").append(CRLF);
        writer.append("Content-Type: application/octet-stream").append(CRLF);
        writer.append(CRLF).flush();

        writer.append("Content-Disposition: form-data; name=\"-s\"").append(CRLF);
        writer.append(CRLF);
        writer.append("ATGACGATAAAGGCACGGCCTCCAACGAGACCTGTGGGCACGGCCATGTTGGGGGCGGGGCTTCCGGTCA").append(CRLF);
        writer.append(boundary).append(CRLF).flush();


        writer.append(boundary + "--").append(CRLF).flush();


        try {
            Thread.sleep(1000); // do nothing for 1000 miliseconds (1 second)
        } catch (InterruptedException e) {
            e.printStackTrace();
        }


        // Get the response
        BufferedReader rd = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String line;
        while ((line = rd.readLine()) != null) {
            // Process line...

            System.out.println(line);

        }

        rd.close();

    } catch (Exception e) {
        System.out.println(e);
    }
}

}

This gives a error something like this;

The above is a description of an error in a Python program, formatted for a Web browser because the 'cgitb' module was enabled. In case you are not reading this in a Web browser, here is the original traceback:

Traceback (most recent call last): File "/home/website/cgi-bin/genscanw_py.cgi", line 16, in <module> organism = form['-o'].value File "/usr/lib64/python2.5/cgi.py", line 567, in getitem raise KeyError, key KeyError: '-o'

I tried with a HTTP analyzer and the rawdata sent to the server is identical. Can anyone please see where I am going wrong. Thank,

[Jeremy Leipzig]

This question is incomprehensible. Is it mit's genscanw_py.cgi that is throwing the error or yours? You will need to provide reproducible code for anyone to help you.

[Fred]

I think it is genscanw_py.cgi that is throwing the error. The http post is not well formed, and then the server cannot access the variable "-o" for example.

Answer 1

[Fred]

Hi, I tried with a modified version of your code, and it seems to work. Basically, I modified the boundary String, and the "--" before and after.

Hope this helps.

`

        String boundary = "-----------------------------" + Long.toHexString(System.currentTimeMillis());
        //boundary = "--" + boundary;

        ...

        // Send normal param.
        writer.append("--"+boundary).append(CRLF);
        writer.append("Content-Disposition: form-data; name=\"-o\"").append(CRLF);
        writer.append(CRLF);
        writer.append("Vertebrate").append(CRLF);
        writer.append("--"+boundary).append(CRLF).flush();

        writer.append("Content-Disposition: form-data; name=\"-e\"").append(CRLF);
        writer.append(CRLF);
        writer.append("1.00").append(CRLF);
        writer.append("--"+boundary).append(CRLF).flush();

        writer.append("Content-Disposition: form-data; name=\"-n\"").append(CRLF);
        writer.append(CRLF);
        writer.append(CRLF);
        writer.append("--"+boundary).append(CRLF).flush();

        writer.append("Content-Disposition: form-data; name=\"-p\"").append(CRLF);
        writer.append(CRLF);
        writer.append("Predicted peptides only").append(CRLF);
        //writer.append(boundary).append(CRLF).flush();

        writer.append("--"+boundary).append(CRLF);
        writer.append("Content-Disposition: form-data; name=\"textFile\"; filename=\"\"").append(CRLF);
        writer.append("Content-Type: application/octet-stream").append(CRLF);
        writer.append(CRLF).flush();

        writer.append("--"+boundary).append(CRLF);
        writer.append("Content-Disposition: form-data; name=\"-s\"").append(CRLF);
        writer.append(CRLF);
        writer.append("ATGACGATAAAGGCACGGCCTCCAACGAGACCTGTGGGCACGGCCATGTTGGGGGCGGGGCTTCCGGTCA").append(CRLF);
        //writer.append(boundary).append("--").append(CRLF).flush();

        writer.append(boundary + "--").append(CRLF).flush();

`

[Chami]

Thanks work like charm !