Below is an example of downloading a single sample.
parallel-fastq-dump --tmpdir . --threads 8 --gzip --readids --split-files --sra-id SRR13528085
There are a total of 12 samples (6 biological samples and 2 technical replicates
Order by: Date
I am currently working on a ChIP-seq experiment with a collaborator and I'm running into problems that may be due to the overall experimental design.
So for an overall background, I was brought on as a bioinformatics analyst after the experiment was ordered and sequenced. From what I know so far there were four experimental groups and two replicates for each experimental group. The lab did not …
updated 15 days ago • dmj6ab
Good morning,
I have multiple fastq.gz file generated using **ONT** and belonging to **one sample**. Each fastq.gz file is composed of many reads, and each read could belong to gene 1 or gene 2, or gene 3.
I have the reference sequence for each gene.
**I want to classify each read and determining if it belongs to gene 1, gene 2, or gene 3.**
The goal is to generate **three fastq** files: o…
updated 27 days ago • maevalefeuvre
zcat PAW*.fastq.gz > fld.fastq.gz, I encountered an error indicating that the files are not in gzip format. However, the files were basecalled by the ONT machine and have the .gz extension.
How can I resolve this issue and...ensure that the files are in the correct gzip format?
I tried to unzip them using gunzip but again error says the same
bioconductor.org/packages/3.17/bioc/src/contrib/densvis_1.10.3.tar.gz'
Content type 'application/gzip' length 1731202 bytes (1.7 MB)
==================================================
downloaded 1.7 MB
trying URL 'https://bioconductor.org/packages/3.17/bioc/src/contrib/scater_1.28.0.tar.gz...Content type 'application/gzip' length 4089325 bytes (3.9 MB)
===========================================…
I'm having an odd problem downloading files from SRA archive - I have tried running fastq-dump --gzip --split-3 SRR11819912, but despite being clearly labelled as paired end data on the SRA website, I am only getting one fastq
prev_header=$0} END {print header; print max_seq; getline; getline; print; getline; print}' | gzip > reads_max_length.fastq.gz
Also, when I align with minimap2, I don't get any results. Maybe I did something wrong? Thanks
updated 8 weeks ago • marco.barr
bcftools view Chester_Genome_variants.bcf -O v -o Chester_Genome_variants.vcf
gzip Chester_Genome_variants.vcf
bcftools index Chester_Genome_variants.vcf.gz #This step instead of using tabix
updated 10 weeks ago • sainavyav22
looping through a fastq file using the method stated in [this stackoverflow post][1].
```py
import gzip
from Bio import SeqIO
sampleid = 'G4G3811_S77'
with gzip.open('filename.fastq.gz', 'rt') as forward_fastq:
for forward_record
updated 11 weeks ago • joreamayarom
SRR18187893) I used this commands:
```
fasterq-dump --outdir "$output_directory" "$SRR_ID"
gzip -c
<file> > <new_name>.gz
```
I renamed my files to match the requirements:
```
OSCC11_S1_L001_R1_001.fastq.gz OSCC11_S1_L001_R2_001.fastq.gz
updated 12 weeks ago • Elinor
split-spot \
--skip-technical {wildcards.sra} \
--stdout 2>{log} \
| gzip -c > {output}
"""
```
can you please help me to parallelize this
13:19:26] Found vcffirstheader - /home/ubuntu-pc/miniconda3/bin/vcffirstheader
[13:19:26] Found gzip - /usr/bin/gzip
[13:19:26] Found vt - /home/ubuntu-pc/miniconda3/bin/vt
[13:19:26] Found snippy-vcf_to_tab - /home/ubuntu-pc/miniconda3
updated 3 months ago • blur
Hey everyone,
I am having some issues updating some packages. I am using `Bioconductor 3.18` and sometimes, when I want to update my packages using `BiocManager::install()` (or, if I install a specific package), I am asked, if I want to update some packages. So far so normal and I mostly select all then. Most of the packages are updated, but some simply aren't. I don't get a warning or error for…
updated 3 months ago • gernophil
sequence length for both reads before a sequence pair gets removed: 20 bp
Output file will be GZIP compressed
This is cutadapt 4.4 with Python 3.11.4
Command line parameters: -j 8 -e 0.1 -q 20 -O 1 -a AGATCGGAAGAGC Sample02.R1.fastq.gz...sequence length for both reads before a sequence pair gets removed: 20 bp
Output file will be GZIP compressed
This is cutadapt 4.4 with Python 3.11.4
Command l…
updated 3 months ago • leonmcswain
You can also save adata object
```py
adata.write_h5ad('adata_batch.h5ad',compression='gzip')
#adata=ov.read('adata_batch.h5ad')
```
**Compare the dataset before and after correction**
We specify three different colours
updated 3 months ago • Julia Ma
for the next use
```py
aml_tcga.adata.write_h5ad('data/TCGA_AML/aml_tcga_raw.h5ad',compression='gzip')
```
Note: Each time we read the anndata file, we need to initialize the TCGA object using three paths so that the subsequent...your result.
```py
aml_tcga.adata.write_h5ad('data/TCGA_AML/aml_tcga_survial_all.h5ad',compression='gzip')
```
[1]: /media/images/0fb09d0a-5aec-41a6-8f38-d5a96bb0
updated 3 months ago • Julia Ma
command I am using (with for loop and in small number of RUNS):
> fastq-dump-orig.3.0.9 --split-3 --gzip ERR1343843
The majority of downloaded files show errors/warnings (enlisted below). I used older version 3.0.0. and newer...Example 1 (with error and resulted in truncated file - the truncated files are reported as OK with gzip -t command):
>2024-01-16T22:13:51 fastq-dump-orig.3.…
Part-1 here: https://www.biostars.org/p/9584982/
Part-2 here: https://www.biostars.org/p/9584985/
Full article lifted from: https://omicverse.readthedocs.io/en/latest/Tutorials-single/t_single_batch/
scVI
----
An important task of single-cell analysis is the integration of several samples, which we can perform with scVI. For integration, scVI treats the data as unlabelled. When our dataset is…
updated 3 months ago • Julia Ma
save the pre-processed data.
```py
adata.write_h5ad('neurips2021_batch_normlog.h5ad',compression='gzip')
```
Similarly, we calculated PCA for HVGs and visualised potential batch effects in the data using pymde. pymde is GPU-accelerated
updated 3 months ago • Julia Ma
enter image description here][4]
```py
#adata.write_h5ad('adata_mde.h5ad',compression='gzip')
#adata=sc.read('adata_mde.h5ad')
```
MeteTiME model init
-------------------
Next, let's load the pre-computed MetaTiME MetaComponents (MeCs), and
can save the pre-processed data.
adata.write_h5ad('neurips2021_batch_normlog.h5ad',compression='gzip')
Similarly, we calculated PCA for HVGs and visualised potential batch effects in the data using pymde. pymde is GPU-accelerated
updated 3 months ago • Julia Ma
above command I prepared mpileup files through samtools command:
**> samtools mpileup file.bam | gzip -c > file.mp.gz**
However I am not sure how to make this refpanel-phased vcf file for wheat.
Looking forward to your valuable
updated 3 months ago • analyst
Hello,
what are your experiences towards efficiently storing large amounts (2-3 digit Terabytes) of flow cytometry (FCS) and (multiplex) imaging (folders with thousands of TIFs) data?
It must be lossless of course and should notably reduce uncompressed file size.
FCS folders seem to compress relatively well with standard `zip`. For the TIF folders `zip` / `gzip` basically do no reduction i…
updated 3 months ago • ATpoint
for aligning some rna-seq fastq.
First thing I did was download hg38.fa.align.gz from UCSC
Then I:
gzip -d hg38.fa.align.gz
sudo apt-get install bwa
Here comes the problem. BWA instructions reccomend bwtsw algorithm, but
fastq.gz files were formed by SRA toolkit using fastq-dump.
fastq-dump -I --split-files --gzip
When I try to run the two files for analysis by fastp I am unable to specify the two paired end files inside my shell script..._trimmed.fastq.gz -O ${file2}_trimmed.fastq.gz
done
```
but it gives an error:
Error to read gzip file ............... the file doesn't exist.
I don't know where …
NT_113901.1 unplaced-scaffold) using the following command:
zcat freq.vcf.gz | grep -E '^#|^NC_' | gzip > freq_only_NC.vcf.gz
Next, I attempted to use `bcftools annotate --rename-chrs` to change the encoding to the standard
prefetch $LINE && /media/huk/PirLab/sratoolkit.2.11.0-ubuntu64/bin/fastq-dump --split-files --gzip ./$LINE
I have 4 Fastq files for some samples and 3 Fastq files for others.
For example the SRR12273020 (https://trace.ncbi.nlm.nih.gov
using the following command:
zcat ISample1-mRNA_R2.fastq.gz | tail -1
and got the following output:
gzip: Sample1-mRNA_R2.fastq.gz: unexpected end of file
AGGCGTATCTCACTGACTTCCTGTGTCAGTTTGCACAGCAGCCCTGCTATGCCATGTTTTCAGACCATCTCAATGAGAATGAAAAGCGAGTGCTGCAGGCCATTGGCAT
in there somewhere.
Here is the Python script I used:
# imports
import operator, sys, time, gzip, re
# String searching function:
import pickle
def string_find(usage_term):
for idx, elem in enumerate(sys.argv):
this_elem
updated 5 months ago • eamaunders
URL 'https://cloud.r-project.org/src/contrib/magick_2.8.1.tar.gz'
Content type 'application/x-gzip' length 4601205 bytes (4.4 MB)
==================================================
downloaded 4.4 MB
* installing *source* package ‘magick’ ...
** package ‘magick’ successfully unpacked
updated 5 months ago • applepie
bioconductor.org/packages/3.18/bioc/src/contrib/DESeq2_1.42.0.tar.gz'
Content type 'application/x-gzip' length 2097442 bytes (2.0 MB)
==================================================
downloaded 2.0 MB
installing source package ‘DESeq2’ ...
** using staged installation
** libs
using
updated 5 months ago • dantuluri
can not be indexed directly because the file is not `BGZIP` compressed.
The file is gzipped with `x-gzip` aplication. Therefore, I tried to unzip it using `gzip -dc` command in order to perform `BGZIP` compression but it tells me that...the file not in gzip format. From some explanations of diffrence between `gzip` and `x-gzip` formats seems that they are equivalent.
Any suggestions
updated 5 months ago • Katsiaryna
Edit 24 Oct 2023: I use nextflow in projects. This is just a curiosity and should be treated as just experimentation. Take a look at great advice from experienced users below.
This is a ~~tutorial~~ about creating a pipeline for sequence analysis in a single line.
It is made for capture/amplicon short read sequencing in mind for human DNA and tested with reference exome sequencing data [describe…
updated 6 months ago • barslmn
I have this code:
samtools fastq -T $bwarefgenomepath ART03_FINAL.cram > BACKUP.fastq
gzip BACKUP.fastq
The BACKUP.fastq.gz is created but it is much smaller than the original
103M Oct 17 20:21 ART03_S8_L001_R1_001.fastq.gz
updated 6 months ago • ManuelDB
t MDA-MB-231/SRR1021654_recall_reads.bam | sequenza-utils seqz_binning -w 50 -s - | gzip > tumor_small.seqz.gz
```
I am quite new to this kind of analyses, so I am sorry if it is a very simple issue to solve.
Thanks
updated 8 months ago • Camilo Andres
using prefetch and the using the following command on the terminal:
fastq-dump --outdir fastq --gzip --skip-technical --readids --read-filter pass --dumpbase --split-2 --clip SRR7898910
My issue is that this creates a single fastq
updated 8 months ago • V
GEO, unpack the .tar file, and read it into scanpy.
!tar -xzf GSE159115_RAW.tar
Traceback:
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
[1]: https://www.ncbi.nlm.nih.gov
gt; xxaa.sai
*Suppose the unsorted BWA alignment results are xx??.sam.gz. Generate rawMask with:*
gzip -dc xx??.sam.gz | gen_raw_mask.pl > rawMask_35.fa
*Generate the final mask:*
gen_mask -l 35 -r 0.5 rawMask_35.fa > mask_35_50.fa...samtools merge merged.sam ${allsamfiles}
and then processed steps 3 and 4 with that merged sam file:
gzip -dc merged.sam | gen_raw_…
updated 8 months ago • biostars
and their original file types look like this:
File: MG430_L4_2.fq.gz, Type: application/x-gzip File: MG431_L4_1.fq.gz, Type: application/x-gzip File:MG431_L4_2.fq.gz, Type: application/x-gzip
I ran fastqc, trimmomatic
original single cell data from GEO (SRR13403717), and converted it: `fastq-dump --split-files --gzip SRR13403717`, two files `SRR13403717_1.fastq.gz` (40G) and `SRR13403717_1.fastq.gz` (38G), both files are 150bp, I don't know
100%
INFO Downloaded file ref_GRCh37.p13_top_level.gff3.gz looks like a valid gzip'ed file
INFO Downloading https://ftp.ncbi.nlm.nih.gov/genomes/archive/old_refseq/H_sapiens/ARCHIVE/ANNOTATION_RELEASE.105...100%
INFO Downloaded file rna.fa.gz looks like a valid gzip'ed file
INFO Downloading http://hgdownload.soe.ucsc.edu/goldenPath/hg19/database/chromInfo.txt.gz
[========…
updated 9 months ago • curious
refseq/H_sapiens/annotation/GRCh38_latest/refseq_identifiers/GRCh38_latest_rna.fna.gz
gzip -d GRCh38_latest_genomic.fna.gz
```
3. Make an index bd with emboss
```
dbifasta -dbname GRCh38_latest_genomic.fna -idformat
updated 9 months ago • Ana
100GB on the server, so I do not think that this is the issue.
Also, the format of the index is `gzip compressed data` but somehow, after the download, it is saved with that strange name.
Thank you very much for the help. I'll
updated 9 months ago • dylannicoembros
paste - - - - \
| awk -v FS="\t" -v OFS="\n" '$2 ~ "TGGACTAC" {print $1, $2, $3, $4}' \
| gzip > Root-R01_AGA_L001_R2_001.filtered.fq.gz
Can someone help me in this?
Many thanks in advance
by [Heng Li][1] that adds support for several common biological data formats, including optionally gzip'ed BED, GFF, SAM, VCF, FASTA/Q as well as generic TAB-delimited formats with the column names.
**Code**
The source code can be found
updated 10 months ago • Istvan Albert
v2.2. This includes fixes for an `` unstarch `` row cutoff bug, restoration of `` starchcat ``'s `` gzip ``-backing support, as well as a reversion to the C-based `` wig2bed `` conversion utility to restore performance lost with the
updated 10 months ago • Alex Reynolds
This Python module allows fast random access to files compressed with [bgzip][1] and indexed by [tabix][1]. It includes a C extension with code from [klib][2]. The bgzip and tabix programs are available [here][3].
## Installation
https://github.com/slowkow/pytabix
$ pip install --user pytabix
or
$ wget https://pypi.python.org/packages/source/p/pytabix/pytabix-0.1.tar.gz
$ tar xf …
download a few RNAseq data from NCBI using SRA toolkit with the following command:
fastq-dump --gzip SRR924395
and I'm getting the following error:
fastq-dump.2.6.3 err: function unsupported while opening file within...that I can download one specific RNAseq data (SRR924399) with the following command:
fastq-dump --gzip SRR924399
but for all the other data that I tried, I got t…
updated 10 months ago • BCArg
The Bioinformatics Databank Manager System (*BeeDeeM*) is capable of installing on a local computing system major sequence databank file formats: Genbank, Refseq, Embl, Genpept, Swissprot, TrEmbl, Fasta, Silva and BOLD. In addition to sequence data files, BeeDeeM can install major biological classifications (ontologies), such as NCBI Taxonomy, Gene Ontology, Enzyme Commission and Intepro domains.…
Recent Votes
Recent Locations •
All
France, just now
Sweden, just now
Sweden, just now
Italy, 1 minute ago
Thailand, 3 minutes ago
India, 3 minutes ago
Belgium, Brussels, 5 minutes ago
Recent Awards •
All
Popular Question to
kuttibiotech2009
▴
30
Popular Question to
analyst
▴
30
Popular Question to
Shicheng Guo
★
9.4k
Popular Question to
Sasha
▴
840
Popular Question to
Buffo
★
2.4k
Popular Question to
yvonnehhe
•
0
Popular Question to
curious
▴
750
Recent Replies
Comment: how to split multi-fasta file into single fasta file named by header
by
rsieber
▴
10
This is great also for multiline fastas
Comment: how to split multi-fasta file into single fasta file named by header
by
rsieber
▴
10
This only works for the first line of sequences.
Answer: What purposes can TPM values be used for?
by
b.contreras.moreira
▴
180
In case it helps: https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units
Comment: What analysis suitable to identify similarly expressed genes between two samples
by
alifafiq1
•
0
Thank you @AT point. I was planning to use DESeq2 and use the alternate hypothesis (lessAbs) too considering that is the only package that …
Comment: Does comparing two different groups to a common third group introduce bias in th
by
ATpoint
82k
Note that by basic algebra A vs C and B vs C is the same as A vs B, so you might rethink your analysis to answer your question in a much si…
Comment: RNAseq one control two conditions, shared and exclusive genes
by
Mohamed Abderrahmane
▴
20
Yes, why not? If it works for you like that, bingo!
Answer: Does comparing two different groups to a common third group introduce bias in th
by
christopher medway
▴
450
You are testing groups A and B to a common baseline (C) - I don't see that there is anything wrong with doing that. It is a fairly common a…
Comment: HOMER on AWS
by
clairechung112
•
0
Hi. I guess it is solved by now, but as I did not find the answer immediately online, here is the answer I posted on a relevant question: h…
Answer: homer not configured properly
by
clairechung112
•
0
A late reply, but I just solved exactly the same error upon a request in the team. Please see if it helps in case anyone meets the same err…
Comment: Spike-in control found in raw reads (16S amplicon seq) but not picked up by DADA
by
Chris Dean
▴
390
Was the spike-in a commercial product, e.g., from Zymo? Can you provide more information about what cells or DNA was spiked into your samp…
Comment: Bedtools merge minimum overlap?
by
Alex Reynolds
35k
Note that, in place of `sort-bed - | uniq -`, you can pass the `--unique` option to `sort-bed`.
The input is not strictly a BED6 file, but…
Comment: Post-imputation plot
by
LChart
3.9k
Imputation takes what? A day or two of compute time? Just re-do it.
Comment: How to access TCGA samples that were treated with a specific drug?
by
Qroid
▴
40
Thanks. Really appreciate your help with this. This site is such a great resource.
The methods aren't totally clear on how they're gettin…
Comment: How to access TCGA samples that were treated with a specific drug?
by
GenoMax
142k
It is possible that these drugs may not have been directly used in TCGA. Authors could have looked for mutations known to be acted on by th…
Comment: Does comparing two different groups to a common third group introduce bias in th
by
Qroid
▴
40
Are you thinking about a specific scenario? If so, can you provide more info? I think the basic answer is that, yes, your choice of group C…
Traffic: 2238 users visited in the last hour
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by the
version 2.3.6
version 2.3.6