Question: How to import huge .csv files in R studio?
0
gravatar for krushnach80
2.0 years ago by
krushnach80420
krushnach80420 wrote:

I want to import an expression .csv file in R studio ,but the size limit is 5 mb max my file is 30 mb how do I do that?

The file is mrna seq data from TCGA which I want to extract.How can I do that is there any way to import the huge file and see ?

rna-seq R • 7.9k views
ADD COMMENTlink modified 2.0 years ago by Ron820 • written 2.0 years ago by krushnach80420

I think you may use ff lib

library(ff)

your_data <- read.csv.ffdf(file = 'your_file.csv', header = T)

other solution use bigmemory

the authors successfully load a CSV with size as large as 11GB

more details and other solution could be found here

ADD REPLYlink modified 2.0 years ago • written 2.0 years ago by Medhat8.0k
7
gravatar for Santosh Anand
2.0 years ago by
Santosh Anand4.1k
Santosh Anand4.1k wrote:

fread() from data.table package is blazing fast for reading large files. It tries to guess the delimiter and header automatically. It will give you an object of class "data.table", which is very similar to data.frame though quirky sometimes. You can easily convert it to your familiar data.frame by using as.data.frame(x) or by using data.table=FALSE in the argument of fread().

ADD COMMENTlink modified 8 months ago • written 2.0 years ago by Santosh Anand4.1k
6
gravatar for Devon Ryan
2.0 years ago by
Devon Ryan86k
Freiburg, Germany
Devon Ryan86k wrote:

Try using read.csv() rather than clicking on files. The limitation is there to prevent you from trying to edit large files, which will degrade performance significantly.

ADD COMMENTlink written 2.0 years ago by Devon Ryan86k
4
gravatar for sm30
2.0 years ago by
sm3040
sm3040 wrote:

CRAN pakage TCGA2STAT can be used to directly import TCGA data into R.

ADD COMMENTlink written 2.0 years ago by sm3040

yes finally I used that library

ADD REPLYlink written 24 months ago by krushnach80420
4
gravatar for Ron
2.0 years ago by
Ron820
United States
Ron820 wrote:

http://stackoverflow.com/questions/1727772/quickly-reading-very-large-tables-as-dataframes-in-r

read.csv.sql() from https://cran.r-project.org/web/packages/sqldf/sqldf.pdf

ADD COMMENTlink written 2.0 years ago by Ron820
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 750 users visited in the last hour