Controlling number of active threads

[Neil Croft]

This is probably trivial and I'll look at it tomorrow, slap my forehead and wonder why I needed to go on the forum to ask. Still, brain mush time today so,

I have a program that runs on a variety of uni and multi processor and uni and multi core machines "laying around" here. To save having to run multiple copies of the program on each machine that supports it, I've re-written it to use threads as the task splits nicely in a number (44 to be precise) of parallel tasks however the 44 tasks aren't of equal length.

If I just do

FOR f&=1 TO 44
      THREAD CREATE MyThread(f) TO idThread(f)
NEXT

it works and the OS sorts out all the thread stuff but it seems to spend a lot more time doing the work so I presume it's spending it's life context switching the threads instead of working.

I tried messing around with SLEEP but then the box just sits around sleeping.

I can enumerate the CPUs/cores with

    LOCAL SI AS SYSTEM_INFO
    GetSystemInfo SI
    cp&=SI.dwNumberOfProcessors

so I could just launch cp& worker threads at a time by doing something like using a STEP in the loop but that's not smart enough to spot when a thread's done and spawn the next one as they may not finish in sequence.

The logic of what I need/want to do is launch cp& worker threads and as any one finishes, spawn the next so that cp& worker threads are running.

Note the loop counter (ie 1 to 44) is a parameter each thread needs to know which part of the overall task it's doing.

Or, launch all 44 threads (memory resource isn't an issue), sleep 44-cp& of them and just wake one up every time one completes until they're all done.

Currently, once spawned, the following code

        DO
            SLEEP 5000
            FOR f&=1 TO 44
                THREAD STATUS idThread(f&) TO e&
                IF e&<>-1 THEN EXIT FOR
            NEXT
            i$=INKEY$
            IF i$="q" THEN EXIT FUNCTION
        LOOP WHILE e&<>-1

lifted almost verbatim from the samples waits for all 44 to finish and also monitors for the user (me ) quitting the program, before continuing.

[Michael Mattias]

Use a semaphore object with a max count = max number of additional threads of execution you want to permit in your process.

WaitForsingleObject (or multiple objects if you want to handle an abort event) on the semaphore handle before launching thread. You'll need a "launch" TOE to make this work.

Windows will handle all the queueing for you.

NOTE: You are NOT guaranteed "FIFO" release of objects queued on WFSO/WFMO, only that the thread releases will be "fair" whatever that means. That does not appear to be an issue here, but you should be aware of it.

You can kind of look at Terminate Worker Threads Using Windows Events (and Using Thread Local Storage) Demo Dec 23 2005 as a starter. That's not exactly what you want but it should give you an idea of how you can wait on things.

There's also GUI + Worker Thread + Abort Demo 11-24-07. That works with a GUI and the mechanics are not as complex as what you have to do, but I think in that one getting the 'big picture' regarding control of multiple synchronization objects is probably a bit clearer than in the first example.

(One of these days I am going to have to create the semaphore demo I think I promised a while ago).


MCM

[Michael Mattias]

Or, just search Source Code Forum thread titles for "QUEUE" ...there's a number of examples shown from multiple contributors.

[Paul Dixon]

I presume it's spending it's life context switching the threads instead of working.

That seems very unlikely. If you don't use SLEEP in the threads (or certain graphic commands which cause a SLEEP when you might not expect it) then the context will switch at the end of each timeslice which is usually 16ms but may be as low as 1ms.
A context switch only takes about 1us so you only lose 0.1% max. of your CPU time switching threads.

If your therads each take a long time to complete (long meaning many timeslices) then just creating 44 threads and letting the OS sort it out should work well but e.g. if you have 2 cpus and 44 theads with 16ms timeslices then each cpu with run about 22 threads so each thread will only get a timeslice every 22x16=352ms which may appear to make that thread respond slowly even though the total amount of processing completes sooner.

Have you tried ctl-alt-del while the program is running and see if the task manager shows all your CPUs running near 100%

Paul.

[Neil Croft]

Lots of stuff to look up there Michael. Thanks. Yes, a semaphore demo would be good.

Paul. Yes, the thread count jumps from 1 to 45 (44 slave + 1 master) as it launches the threads then drops 44,43, etc. as the shorter tasks complete and the CPU is pegged at 100% where it belongs.

I was just concerned that I'd be wasting those oh so valuable clock cycles throwing stuff in and out of the processor. All being well, Santa's bringing me a dual quad core box and letting the OS sort it out for me will always be easier given the wide variety of CPUs here.

Thanks guys. Plenty of food for thought. I shall return later after reading up on queues.

[Michael Mattias]

I started on a semaphore demo and have run into some issues..

Do you need/want to control the number of threads? Or the number of NOT-SUSPENDED threads?

That is, in your example, is it OK to have 44 threads of execution, but only "n" threads are permitted to be actually competing for CPU time and other resources? FWIW, this would make it pretty easy to add something to "un-queue all TOEs which have not actually started doing something".. that is, your program would allow the 'n' permitted TOEs to complete what they are doing, but no other TOEs could be started after you issue this "No more" command.

If you want to limit the actual number of TOEs.... I'm having trouble doing that using a semaphore , although I could do it quite nicely using a different synchronization object. That is, I think the semaphore oject was not a good idea if the absolute number of TOEs must be limited.

I really don't know how much impact either plan would affect system resources, even though I am sure there must be 'some' cost for creating a suspended (waiting for a slot) thread of execution; and I'm sure the cost for an 'executing' TOE is greater than the cost for a 'waiting' TOE.

MCM

[Neil Croft]

Either. In my particular application, each thread consumes very little memory so having 42 from 44 idle is no great shakes however I can imagine scenarios where this wouldn't be the case. In that instance, launching the first x number of threads then watching those and launching another each time one completes until they're all done would be the preferred route.

Whilst for my application, just waking 2/4/whatever from the full set is fine, it seems that number 1 takes nearly as long as the other 43 together, number 2 nearly as long as the other 42 etc. so in this particular instance it is preferable to run the lower numbered threads (or certainly thread 1) first. Again, other people's applications might vary.

[Michael Mattias]

I decided I'd do it both ways. It's been a while since I last provided a demo, so I figger it's only fair this be a nice one.

I have the scenario where all 44 threads start but only %MAX_WORKER_THREADS are actually running working. (needs cleanup).

I found another one of my demos I can adapt for scenario two. I can make it work now but it would require modifying the worker thread function..... so I'm trying something so that the user won't have to touch one line of the worker thread function....that is, I want this to be so easy to use there is no excuse not to.

MCM