Question: Crossbow Final Step Failing On Emr
1
gravatar for bashwin.u
6.1 years ago by
bashwin.u40
bashwin.u40 wrote:

Hello,

I am trying to run Crossbow via EMR command line. I managed to complete all the crossbow steps- Alignment with Bowtie, Calling SNPS and Postprocess. I am getting an error in the final step Get Counters. Can anyone please help me fix this?

controller

2013-10-08T03:39:40.661Z INFO Fetching jar file.
2013-10-08T03:39:42.169Z INFO Working dir /mnt/var/lib/hadoop/steps/5
2013-10-08T03:39:42.169Z INFO Executing /usr/lib/jvm/java-6-sun/bin/java -cp /home/hadoop/conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/hadoop:/home/hadoop/hadoop-tools.jar:/home/hadoop/hadoop-core.jar:/home/hadoop/hadoop-core-0.20.205.jar:/home/hadoop/hadoop-tools-0.20.205.jar:/home/hadoop/lib/*:/home/hadoop/lib/jetty-ext/* -Xmx1000m -Dhadoop.log.dir=/mnt/var/log/hadoop/steps/5 -Dhadoop.log.file=syslog -Dhadoop.home.dir=/home/hadoop -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,DRFA -Djava.io.tmpdir=/mnt/var/lib/hadoop/steps/5/tmp -Djava.library.path=/home/hadoop/native/Linux-amd64-64 org.apache.hadoop.util.RunJar /home/hadoop/contrib/streaming/hadoop-streaming-0.20.205.jar -D mapred.reduce.tasks=1 -input s3n://crossbow-emr/dummy-input -output s3n://ashwin-test/crossbow-emr-cli_crossbow_counters/ignoreme1 -mapper cat -reducer s3n://crossbow-emr/1.2.1/Counters.pl  --output=S3N://ashwin-test/crossbow-emr-cli_crossbow_counters -cacheFile s3n://crossbow-emr/1.2.1/Get.pm#Get.pm -cacheFile s3n://crossbow-emr/1.2.1/Counters.pm#Counters.pm -cacheFile s3n://crossbow-emr/1.2.1/Util.pm#Util.pm -cacheFile s3n://crossbow-emr/1.2.1/Tools.pm#Tools.pm -cacheFile s3n://crossbow-emr/1.2.1/AWS.pm#AWS.pm
2013-10-08T03:39:45.175Z INFO Execution ended with ret val 1
2013-10-08T03:39:45.176Z WARN Step failed with bad retval
2013-10-08T03:39:46.681Z INFO Step created jobs:

stderr

Exception in thread "main" Status Code: 403, AWS Request ID: 2977B25629DD5007, AWS Error Code: null, AWS Error Message: Forbidden, S3 Extended Request ID: OcPQrMLKUHBKHfdh4ICR5BgEWNzDtUEzc8H2km55h0nCL92RKph4rFXSCEY9y6vq
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:544)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:284)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:169)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2619)
at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:708)
at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:688)
at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:100)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at org.apache.hadoop.fs.s3native.$Proxy3.retrieveMetadata(Unknown Source)
at org.apache.hadoop.fs.s3native.NativeS3FileSystem.listStatus(NativeS3FileSystem.java:730)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:783)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:808)
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:185)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:1026)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1018)
at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:172)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:934)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:887)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:887)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:861)
at org.apache.hadoop.streaming.StreamJob.submitAndMonitorJob(StreamJob.java:1010)
at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:127)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

stdout

packageJobJar: [/mnt/var/lib/hadoop/tmp/hadoop-unjar9002137556695792672/] [] /mnt/var/lib/hadoop/steps/5/tmp/streamjob4081705531014015666.jar tmpDir=null

syslog

    2013-10-08 03:39:42,458 WARN org.apache.hadoop.conf.Configuration (main): DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
2013-10-08 03:39:43,393 INFO org.apache.hadoop.mapred.JobClient (main): Default number of map tasks: null
2013-10-08 03:39:43,393 INFO org.apache.hadoop.mapred.JobClient (main): Setting default number of map tasks based on cluster size to : 56
2013-10-08 03:39:43,393 INFO org.apache.hadoop.mapred.JobClient (main): Default number of reduce tasks: 1
2013-10-08 03:39:44,940 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader (main): Loaded native gpl library
2013-10-08 03:39:44,943 WARN com.hadoop.compression.lzo.LzoCodec (main): Could not find build properties file with revision hash
2013-10-08 03:39:44,943 INFO com.hadoop.compression.lzo.LzoCodec (main): Successfully loaded & initialized native-lzo library [hadoop-lzo rev UNKNOWN]
2013-10-08 03:39:44,950 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library is available
2013-10-08 03:39:44,951 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library loaded
2013-10-08 03:39:45,047 INFO org.apache.hadoop.mapred.JobClient (main): Cleaning up the staging area hdfs://10.159.25.174:9000/mnt/var/lib/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310080306_0004

Thanks

genome paired-end bowtie • 2.4k views
ADD COMMENTlink modified 5.9 years ago by gecgooden10 • written 6.1 years ago by bashwin.u40
1
gravatar for gecgooden
5.9 years ago by
gecgooden10
gecgooden10 wrote:

Hi,
I was having the same issue as you, and I was able to resolve it by doing the steps I described here: https://github.com/BenLangmead/crossbow/issues/6#issuecomment-30985968.
I have also informed the developers of this fix and hopefully it will get resolved properly soon.

Thanks,
George

ADD COMMENTlink modified 5.9 years ago • written 5.9 years ago by gecgooden10
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 1665 users visited in the last hour