Question: (Closed) Basic computer question: why does my computer slow AFTER running a script?
0
gravatar for QVINTVS_FABIVS_MAXIMVS
6.8 years ago by
USA SoCal
QVINTVS_FABIVS_MAXIMVS2.4k wrote:

Hi All,

 

This is a basic computer question that could help enlighten a biologist turned data analyst.

 

I have a script that forks into 6 processes, my computer has 8 cores and 8GB of RAM.

 

After running the script and quitting my terminal, my computer is extremely slow. What's the reason behind this? How can I fix it?

 

I know this is probably a stupid question, but I'm curious.

 

Thanks

memory procs ram computer • 3.3k views
ADD COMMENTlink modified 6.8 years ago by Sukhi Singh10k • written 6.8 years ago by QVINTVS_FABIVS_MAXIMVS2.4k
1

Hi,

Like you, I am a biologist who has started doing some data analysis, and I too use a forking perl script that reads data from external files. I like stupid questions, I ask them often myself.

Do you close the files after you have read them? It is not essential to close files, the script will still work, but I guess the open file will use up some computer resources.

Are you using Parallel::ForkManager perl module from cpan? Avoid forking a forked process. Try forking 4 processes rather than 6

How big are the files that you are reading from? are they compressed, and how big are the files that you write too and subsequently delete?

It may be useful to run your script and have a second terminal window and type htop to see how much memory is being used, and which cores are being used.

ADD REPLYlink modified 14 months ago by Ram32k • written 6.8 years ago by rbagnall1.7k

I agree. Check memory usage, ensure you are closing all handles, check usage guidelines of external program, do not explicitly split into multiple threads because sometimes, a single thread is better or the splitting algorithm you use might not gel well with the way the program works.

ADD REPLYlink modified 14 months ago • written 6.8 years ago by Ram32k
  1. Yes I close my files.
  2. Yes I used ForkManager. I might use 6 but I'm writing this script for a computer with 16 processors and will be expanding to 12 or 14
  3. Around 250MB are the .txt files, they are not compressed
  4. Interesting. But it's still slow after 5 or 10 minutes the process has completed running.
ADD REPLYlink modified 14 months ago by Ram32k • written 6.8 years ago by QVINTVS_FABIVS_MAXIMVS2.4k

Hi,

I think we'd need more data on the problem, like if any external resources are being used (files, databases etc). Also, what does this script do?

ADD REPLYlink written 6.8 years ago by Ram32k

Sure,

The perl script pulls certain binary files from a parent directory and then forks into 6 processes .Next, the binary files are converted to .txt files using an outside script which it calls using system().

.txt files are opened and read, parsed out for data, and then deleted while the parsed out data is printed to a file.

I assumed this is a common problem with forking, I never had this issue before I started writing scripts that fork processes.

ADD REPLYlink modified 14 months ago by Ram32k • written 6.8 years ago by QVINTVS_FABIVS_MAXIMVS2.4k

A trivial answer apart form what others have already said: perhaps your CPU is heated up!!

ADD REPLYlink written 6.8 years ago by Bharat Iyengar300

I was thinking that it could be a disk read/write thing, especially if OP doesn't have a SSD.

ADD REPLYlink written 6.8 years ago by 5heikki9.3k

Assuming you're running Linux, what happens if you try the following (after a script runs to completion and your computer runs slowly):

$ sudo su -
...
# sync; echo 3 > /proc/sys/vm/drop_caches
ADD REPLYlink written 6.8 years ago by Alex Reynolds31k

Hello QVINTVS_FABIVS_MAXIMVS!

We believe that this post is does not fit the main topic of this site.

Whilst you may well be doing bioinformatics with your scripts, this is not technically a bioinformatics question and would probably be better off posted on the SuperUsers StackExchange site.

For this reason we have closed your question. This allows us to keep the site focused on the topics that the community can help with.

If you disagree please tell us why in a reply below, we'll be happy to talk about it.

Cheers!

ADD REPLYlink written 6.8 years ago by User 5913k
3
gravatar for Sukhi Singh
6.8 years ago by
Sukhi Singh10k
Netherlands
Sukhi Singh10k wrote:

This might answer it, I asked last 2 years back on SuperUser.

http://superuser.com/questions/511470/why-system-still-hangs-slows-after-process-is-finished

ADD COMMENTlink written 6.8 years ago by Sukhi Singh10k
1

8 GB RAM and 8 cores. I'm pretty sure OP's not running into swap. A htop should shine some light in the situation for us.

ADD REPLYlink written 6.8 years ago by Ram32k

Interesting. I think you're right about this. Thanks!

ADD REPLYlink modified 14 months ago by Ram32k • written 6.8 years ago by QVINTVS_FABIVS_MAXIMVS2.4k
Please log in to add an answer.
The thread is closed. No new answers may be added.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.3.0
Traffic: 2009 users visited in the last hour
_