Thursday, 8 August 2013

Huge Memory and CPU usage with Multi-threading

Huge Memory and CPU usage with Multi-threading

I'm trying to create a media player that has a playlist option. When a
load 10-20 songs there is no problem. So I tried something more demanding:
I tried to load 2048 songs (I took several songs and copied them a lot of
times). Trying to load them in my media player, my CPU and Ram memory had
grown at over 95 % (loading just the first 250 songs) and one time my
computer even restarted. So I tried to slow the operation by using
something that doesn't let the application to take over the computer : I
stop loading new songs if the CPU load is over 85 % and the Memory load
over 90 %. It somehow worked at the beginning permitting me to load almost
600 songs and then:
A first chance exception of type 'System.InvalidOperationException'
occurred in mscorlib.dll
A first chance exception of type 'System.InvalidOperationException'
occurred in mscorlib.dll
A first chance exception of type 'System.InvalidOperationException'
occurred in mscorlib.dll
A first chance exception of type 'System.InvalidOperationException'
occurred in mscorlib.dll
A first chance exception of type 'System.InvalidOperationException'
occurred in mscorlib.dll
The thread 'vshost.NotifyLoad' (0x1d0c) has exited with code 0 (0x0).
The thread 'vshost.LoadReference' (0x1e48) has exited with code 0 (0x0).
A first chance exception of type 'System.OutOfMemoryException' occurred in
mscorlib.dll
A first chance exception of type
'Microsoft.VisualStudio.Debugger.Runtime.CrossThreadMessagingException'
occurred in Microsoft.VisualStudio.Debugger.Runtime.dll
Finally the application stoped at "An unhandled exception of type
'System.OutOfMemoryException' occurred in mscorlib.dll".
Now to explain what does "loading a song" means in my application:
a thread that iterates through every song loaded from the OpenFileDialog
and checks if the extension of the file is known and if it is (and in this
case it is: mp3) it merges the path of the file at the end of a Queue.
another thread verifies if there are any element in the Queue.
if there are, it extracts the first element and if the CpuLoad and the
MemoryLoad (calculated by another thread) are not too high, it starts a
new thread that makes some operations (presented at 4)
the thread that makes operations loads the song in a
System.Windows.Media.MediaPlayer class and verifies the next things : the
TimeSpan of the file, if the file has audio and if the file has video and
remembers those 3 variables along with the path of the file in a List.
there is also another thread that verifies if there are threads that have
finished their job and have added the media file to the List and if there
are, then it removes the reference to them so that the garbage collector
will take care of them.
The "An unhandled exception of type 'System.OutOfMemoryException' occurred
in mscorlib.dll" appears to the next line :
MediaCreator[idx].CreatorThread.Start();
That would be the line that starts the threads that process the songs. So
I made the next thing: before the line posted above, I added
Thread.Sleep(100);. It worked (doing this actually resulted in loading all
the 2048 files), except the fact that (according to a Stopwatch I added)
it took 3 minutes and 28 seconds to load all the songs. Also, I know that
Thread.Sleep usually it's not the recommended method and I know that same
people even believe to be a prove of weak programming skills (and I
somehow agree with them). I don't want to use this method either because
it obviously takes a long time and it's untrustworthy to work on every
computer/cpu/hdd/ram. To prove the untrustworthiness of this I tested with
Sleep(10) with which it fails quickly, and with Sleep(20) with which it
loads almost 1000 songs before it fails again. I also tried reducing the
CPU load to as little as 15 % and the memory load to 80 %, but no more
than 1300 songs were loaded (this also proved inefficiently because there
were short peaks of 60% for the CPU load).
I also want to mention that Winamp loaded all the files in just a bit over
30 seconds using around 11 % of my CPU (from 5% to 16%) and less than 40
MB of memory.
So my question is : how should I proceed ? I could limit the number of
threads so that no more than X threads to be running at the same type, but
this also seems to me a prove of weak programming skills as not every CPU
could hold the same amount of running threads. So what should I do ? I
really need to take those details from the songs - as long as possible
with as little resources as possible (to know how long they are, and if
they are audio or video files : here I should mention that my application
also plays movie, it's just that I don't think that anyone needs to load
thousands of movies at once in application and if I resolve the problem
for audio, it will be resolve for videos also because the songs are
differentiated by movies only before they are stored in the list - so
nothing that would conflict with the solution of my problem). I really
need help in getting this sorted out.

No comments:

Post a Comment