Memory leak, possibly in shared pool

Posted on 2006-09-14 20:45:31-07 by ssinyagin

hi all,

I'm building the threads support into Torrus software (torrus.org)

The Collector process runs constantly as a daemon. In threaded version, the process memory footprint grows slowly but constantly, and in threading-disabled mode it doesn't encounter such problem.

The code in question is here:

Line 71 initializes the background thread. It will read from the queue and store the data in RRDtool files.

Line 474 adds a new job entry to the queue: it creates an empty shared array, then pushes the data into it, and then adds its pointer to the queue.

Line 499 reads the jobs from the queue, stores the data, and then releases the array reference. I suspect it's at this point that the memory is not actually freed up.

Current Perl version is 5.8.8 running on Solaris (binary packages available from www.blastwave.org). The same problem occurs with the original threads module that came with perl 5.8.8, and also with the latest threads version 1.41.

The same problem is noticed at another installation under Linux and Perl 5.8.5, confirmed with the original threads module. I don't have yet information about the latest module behaviour.

regards,

Stanislav Sinyagin ssinyagin@yahoo.com

Direct Responses: 3046 | 3047 | Write a response

Posted on 2006-09-15 18:02:19-07 by jdhedden in response to 3033

Re: Memory leak, possibly in shared pool

I think I have traced the problem to the use of semaphores. I'm still researching the problem, but here's a sample program that exhibits the bug:

use threads; use Thread::Semaphore; my $sema = Thread::Semaphore->new(); for (1..100000) { $sema->down(); $sema->up(); } sleep(15);

As an aside, I noted you're using way too much code in dealing with shared variables. For instance, the following at line 474:

my $cmdlist; &threads::shared::share( \$cmdlist ); $cmdlist = &threads::shared::share([]); push( @{$cmdlist}, $filename, @cmd ); $thrUpdateQueue->enqueue( $cmdlist ); $cmdlist = undef;

Can be simplified to:

my @cmdlist :shared; push(@cmdlist, $filename, @cmd); $thrUpdateQueue->enqueue(\@cmdlist);

Similarly, you can simplify rrdUpdateThread() (at line 499):

# A background thread that updates RRD files sub rrdUpdateThread { &Torrus::Log::setTID(threads->tid()); while (my $cmdlist = $thrUpdateQueue->dequeue()) { if (isDebug) { Debug("Updating RRD: " . join(' ', @{$cmdlist})); } $rrdtoolSemaphore->down(); RRDs::update(@{$cmdlist}); my $err = RRDs::error(); $rrdtoolSemaphore->up(); if ($err) { Error('ERROR updating' . $cmdlist->[0] . ': ' . $err); $thrErrorsQueue->enqueue($cmdlist->[0]); } } }

The above has the added value that you can terminate the thread (if desired) simply by enqueuing an 'undef'.

Direct Responses: Write a response

Posted on 2006-09-15 19:13:39-07 by jdhedden in response to 3033

Re: Memory leak, possibly in shared pool

I think I found the memory leak. I have posted threads::shared v1.03 to CPAN. It should fix this problem.

Direct Responses: 3060 | Write a response

Posted on 2006-09-16 10:07:11-07 by ssinyagin in response to 3047

Re: Memory leak, possibly in shared pool

Jerry, thanks a lot for the fix, I'll test it and report the results.

As for the code simplification, this form:

my @cmdlist :shared;

will not work in this module, because I load Threads module via require() after checking that threads are enabled in the Torrus configuration. I checked this before, and ":shared" declaration works only with "use threads", but not with "require threads".

thanks,

stan

Direct Responses: 3078 | Write a response

Posted on 2006-09-18 16:06:21-07 by ssinyagin in response to 3060

Re: Memory leak, possibly in shared pool

Jerry, with the new threads::shared module, the process runs since last Saturday and no memory growth noted. thanks

Direct Responses: Write a response