Threads and Mutex

[Logman]

Hello:

As a new EBasic user, I am updating and porting over some VB.net/PowerBasic code I wrote a couple years back. The code includes an AI engine that does background file and web searches using separate threads.

I haven't discovered a built-in EBasic thread capability; or maybe I just missed it.

Currently, I am targeting dual and quad core systems and need to do many routines in the background that support the user with data without the user having to wait while the program searches for specific information. Is there a way in EBasic to perform background functions or do I need to use Win32 API calls?

Logman

[Ionic Wind Support Team]

I just use the API myself. 

If you need an example let me know, it's only a a couple of calls.

Paul.

[Logman]

Hello Paul:

An example would be great. I mostly perform file and web searches as well as perform some calls to statistical analysis routines using separate threads. Threads were inherent in the .net languages so I've never actually thought about how to use the Win32 API.

If you could show me how to setup a thread from EBasic, that would help me tremendously.

Logman

[Ionic Wind Support Team]

Just a quickie snippet, from the Emergence Designer, modified for brevity:

Code Select Expand


int nTemp
uint hThread = CreateThread(NULL,0,&Process_Listener,&t,CREATE_SUSPENDED,ntemp)
if(hThread = NULL)
    err = GetLastError()
    print "Thread could not be created"
else
    ResumeThread(hThread)
endif


Basically the CreateThread API call, creates a new thread of execution in your process.  For everyday use the only parameters you are worried about is the address of the thread subroutine and it's parameter.  The thread subroutine is just a SUB that expects one pointer parameter and returns an INT

Code Select Expand


sub Process_Listener(param as pointer),int
...do something in the thread here.
return 0
Endsub


In the call to CreateThread &t is the address of a UDT I am using to pass information to the thread function.  The thread ends when the thread function returns.  I create the thread as suspended so I can check the return value and act appropriately.  Then resume the thread to make it go.

for the API declares use Sapero's includes, or the windows.inc file.

Simple as that.  Gotta run with the wife, when I get back I'll show you how to use a mutex to share access with a global variable between threads.

Paul.

[Logman]

Paul:

Got it, thanks. It would have taken me a couple of days to research and work up a solution and possibly come back on the forum with all kinds of bugs. This gets me back on track. The more I dig into EBasic, the more I discover the small ingenious ideas behind some of the coding approaches, the more I am enjoying EBasic.

Logman

[Logman]

Aha!

Paul, your explanation made the Win API documentation make sense to me because at first I thought it was going to be too daunting a task. To follow through for others who may be interested, here is the Win API setup:

HANDLE CreateThread(
    LPSECURITY_ATTRIBUTES ipThreadAttributes,    ' Pointer to thread security attributes (NULL means handle cannot be inherited)
    DWORD dwStackSize,                                       ' Initial new thread stack size (0 makes t default to the same size as the primary thread process)
    LPTHREAD_START_ROUTINE ipStartAddress        ' Pointer to thread function (32-bit address pointer of the function declare)
    LPVOID lpParameter,                                        ' Argument for new thread (single 32-bit parameter value passed to the thread)
    DWORD dwCreationFlags,                                  ' Creation flags (specifies additional flags - CREATE_SUSPENDED flag starts thread in suspended mode)
    LPDWORD lpThreadId                                       ' Pointer of returned thread ID (32-bit variable that receives the thread identifier)
);

uint hThread = CreateThread(NULL,0,&Process_Listener,&t,CREATE_SUSPENDED,ntemp)

Return Values: If function succeeds, the return value is a handle to the new thread. If the function fails, the return value is NULL.

Really nice Paul. Thanks for the help. Now I'll match this up with a mutex call to ensure my database record and/or file isn't opened by more than one person at a time and I'll be all set.

Logman

[Ionic Wind Support Team]

Mutex's are just as easy once you break it down.  Basically you create a mutex "object", which is really a 32 bit handle, wait for that object to be available, and release it when your done.

Code Select Expand


'create an unnamed mutex for sharing global variables between threads
UINT SharedGlobalLock
SharedGlobalLock = CreateMutex(NULL,FALSE,NULL)


When you want to synchronize access between threads to X, whatever X might be you ask the system for control of the mutex:

Code Select Expand


'shared access for writing a global variable when multiple threads are in play.
if WaitForSingleObject(SharedGlobalLock,100) = 0
SomeGlobalVariable = 999
ReleaseMutex(SharedGlobalLock)
endif


And when your program is done just give the mutex back to the system:

Code Select Expand


CloseHandle( SharedGlobalLock )


The last step isn't technically necessary since Windows will automatically close the handle when the main process ends.

Again it's simple.  The thing to keep in mind is the mutex really has nothing to do with the shared resource (whether it is a global variable, or hardware or whatever) it is just an object for stopping your code until the mutex is signaled as "released".  In the above example WaitForSingleObject waits a maximum of 100ms and then gives up, you would need to adjust that for whatever fits your purpose.  It is common to see it as:

Code Select Expand


'shared access for writing a global variable when multiple threads are in play.
if WaitForSingleObject(SharedGlobalLock, -1) = 0
SomeGlobalVariable = 999
ReleaseMutex(SharedGlobalLock)
endif


The -1 specifies infinity, which means that code will stop the thread and wait indefinitely for the mutex to be released by the other thread.  Something to keep in mind if the other thread never calls ReleaseMutex.

Paul.

[Logman]

Paul:

I was just deciphering Microsoft's Win API reference and you beat me to it. You make it look pretty straightforward compared to MS' explanation. The last version of basic I used handed mutex and thread functions to me on a platter, but I had no control over their details, which also made me set my API reference aside.

I like the control I gain by setting up critical API calls in EBasic. It has also made me once again pick up my API reference guide and rediscover why I enjoyed programming in the first place. I had forgotten about all the little details I was losing out on by using the "built-in" functions in some other basics.

Thanks, Logman.

[Logman]

Hello there:

How interesting. I always thought of using mutex's for file-sharing and database record-sharing. I never thought about locking and unlocking global variables that may be shared among users.

Logman

[Ionic Wind Support Team]

Logman,
Part of the reason there isn't a built in function for creating threads is the API really only requires two, one if you don't create the thread suspended.  And I am against thin wrapping of API calls just because I can.  I get bothered by the lack of efficiency when doing something like this:

sub EBCreateThread(uint sub,pointer data),uint
int nTemp
return CreateThread(NULL,0,sub,data,0,ntemp)
endsub

The overhead of two function calls, instead of just one.  And what if the user wants to create it suspended?, or wants to supply a different stack size?  Then the command starts to get silly:

sub EBCreateThread(uint sub,pointer data,int bSuspend, opt int Stacksize = 0),uint
int nTemp
return CreateThread(NULL,stacksize,sub,data,bSuspend * CREATE_SUSPENDED,ntemp)
endsub

I look at code like that and think; What is the benefit?  Is it really saving the user any time?  and when both answers are no then it doesn't become a built in command.

Mutex's on the other hand might benefit from a little Emergence behind the scenes magic.  Still it is only three API calls.   And for more complex uses a simple command set doesn't work.  I like to use a mutex as part of a class where a Get/Set methods automatically waits for the mutex.  But others might not prefer that method.

I also use named mutex's in a class that does everything in one step using the constructor:

Code Select Expand


class MutexLocker
    declare MutexLocker()
    declare _MutexLocker()
    uint m_hMutex
endclass

sub MutexLocker::MutexLocker()
    m_hMutex = CreateMutex(NULL,FALSE,"SOMERANDOMSTRING")
    while WaitForSingleObject(m_hMutex, -1) <> 0
    endwhile
endsub

sub MutexLocker::_MutexLocker()
    ReleaseMutex(m_hMutex)
endsub


So in a thread subroutine I can ensure thread safe access just by declaring a variable:

sub somethread(pointer lparam),int
MutexLocker lock
....
endsub

Which works because the destructor releases the mutex.  So any other thread that creates a MutexLocker variable will wait until this thread is done.  Only works for multiple threads of course, if your sharing variables with the main process then you'll need to use the first method I presented.

Paul.

[Logman]

Paul:

I saw an article on mutex classes and was intrigued. I'm going to look at it, but not being familiar as I should with declaring/importing API calls as I should, I think I am having a problem in this area.

I put together some quick and dirty code to make sure I'm integrating threads and mutex use correctly. I get the threads to work, but can't seem to get the WaitForSingleObject call to work correctly.

I think I have two problems: first in the declare (type identification matching EBASIC types with API variable types), and second synchronizing the code correctly so that mutex's actually work.

Here's the code:
'----------------------------------------------------

$INCLUDE "windows.inc"

'Thread declares
DECLARE IMPORT,CreateThread(lpSecAttributes as UINT,dwStackSize as UINT,lpStartAddress as UINT,lpParam as UINT,dwFlags as UINT,lpThreadId as UINT),UINT
DECLARE IMPORT,ResumeThread(hThread as UINT),UINT
DECLARE IMPORT,GetLastError(),UINT

'Mutex declares
DECLARE IMPORT,CreateMutex ALIAS CreateMutexA(lpMutexAttributes as UINT,bInitialOwner as UINT,lpName as POINTER),UINT
DECLARE IMPORT,ReleaseMutex(hMutex as UINT)
DECLARE IMPORT,CloseHandle(hObject as UINT)
DECLARE IMPORT,WaitForSingleObject(hHandle as UINT,dwMilliseconds as UINT)

'Thread variables
DEF hThread_1,hThread_2 as UINT         'Thread handles
DEF ThreadId_1,ThreadId_2 as INT              'Thread IDs
DEF t as INT                     '32-bit parameter used by threads
DEF err as UINT                     'Return error codes

'Mutex variables
DEF GlobalVariable=333 as UINT                 'Global variable shared by all processes
UINT SharedGlobalLock               'Mutex handle
SharedGlobalLock = CreateMutex(NULL,FALSE,NULL)

OPENCONSOLE

hThread_1=CreateThread(NULL,0,&Process_A,&t,CREATE_SUSPENDED,ThreadId_1)
   IF(hThread_1 = NULL)
      err = GetLastError()
      PRINT "Thread 1 could not be created"
   ELSE
      ResumeThread(hThread_1)
   ENDIF

hThread_2=CreateThread(NULL,0,&Process_B,&t,CREATE_SUSPENDED,ThreadId_2)
   IF(hThread_2 = NULL)
      err = GetLastError()
      PRINT "Thread 2 could not be created"
   ELSE
      ResumeThread(hThread_2)
   ENDIF

SUB Process_A(param as POINTER),INT
PRINT "Start Thread 1 >"
   FOR i = 1 to 5
      PRINT "  Process A-",i
   NEXT i
PRINT "End Thread 1 >"
   'IF WaitForSingleObject(SharedGlobalLock,100)=0    'Commented out as it doesn't seem to work
      'GlobalVariable = 111
      'ReleaseMutex(SharedGlobalLock)
   'ENDIF
PRINT "Process A Global Variable: ",GlobalVariable
RETURN 0
ENDSUB

SUB Process_B(param as POINTER),INT
PRINT "Start Thread 2 >"
   FOR i = 1 to 5
      PRINT "  Process B-",i
   NEXT i
PRINT "End Thread 2 >"
   'IF WaitForSingleObject(SharedGlobalLock,100)=0    'Commented out as it doesn't seem to work
      'GlobalVariable = 222
      'ReleaseMutex(SharedGlobalLock)
   'ENDIF
PRINT "Process B Global Variable: ",GlobalVariable
RETURN 0
ENDSUB

PRINT "Global Variable: ",GlobalVariable

DO
UNTIL INKEY$<>""
CloseHandle(SharedGlobalLock)
PRINT "Global Variable: ",GlobalVariable    'See what GlobalVariable contains
DO
UNTIL INKEY$<>""
CLOSECONSOLE
END

'---------------------------------------------------

Logman

[Ionic Wind Support Team]

Your declares are wrong.  Missing the return type on WaitForSingleObject for one.  Kind of hard for the compiler to compare the return of the function with 0 when it doesn't know the type.

Your already including windows.inc so you can just use _WaitForSingleObject

windows.inc declares all of the API functions prepended with an underscore, think I mentioned that

Paul.

[Logman]

Paul:

Can't believe I totally missed that. Thanks for the re-enforcement--think I spent too much time in the .net environment.

Logman

[Logman]

Paul:

RE: Your declares are wrong.  Missing the return type on WaitForSingleObject for one.  Kind of hard for the compiler to compare the return of the function with 0 when it doesn't know the type.

You're already including windows.inc so you can just use _WaitForSingleObject

windows.inc declares all of the API functions prepended with an underscore, think I mentioned that Wink

--------------------------------------------------------------

How simple and elegant. However, I must still be doing something wrong as the following code gives me an error message in line 9 (UINT hThread = _CreateThread(NULL...) that says: (9) no appropriate conversion exists. I think I have structued or defined 't' incorrectly.

Can you help me with this? What am I still missing?

$INCLUDE "windows.inc"

OPENCONSOLE

INT nTemp
INT t
INT err

UINT hThread = _CreateThread(NULL,0,&Process_Listener,&t,CREATE_SUSPENDED,nTemp)    'This is line 9
    IF (hThread = NULL)
   err = _GetLastError()
   PRINT "Thread could not be created!"
    ELSE
   _ResumeThread(hThread)
    ENDIF

SUB Process_Listener(param AS POINTER),INT
PRINT "Start Thread >"
    FOR i = 1 to 5
     PRINT "  Process-",i
    NEXT i
PRINT "End Thread >"
RETURN 0
ENDSUB

PRINT "\nPress any key to quit"
DO
until inkey$ <> ""
CLOSECONSOLE
END

[Ionic Wind Support Team]

When all else fails, actually look at the declare that's in windows.inc.   It's wrong as the first parameter should be a pointer, but didn't convert correctly.  Go back to using your own declare as you obviously had that line compiling before

Paul.

[Logman]

Paul:

Thanks for your help; your advice paid off.

I did a search of some of your earlier work and found a console program you wrote, which defined the CreateThread routine.  I noticed that the first parameter in the CreateThread routine (i.e., lpThreadAttributes) was set as a pointer type. In the "windows.inc" file it is set thusly: lpThreadAttributes AS SECURITY_ATTRIBUTES. Not able to satisfactorily replicate the SECURITY_ATTRIBUTES structure, I simply went into the "windows.inc" file and at line 2178 changed the type from SECURITY_ATTRIBUTES to POINTER. After that, the program ran perfectly using the underline prefix i.e., _CreateThread(...).

I also got threads working by simply using DECLARE IMPORT, CreateThread(lpThreadAttributes as POINTER,dwStackSize as UINT,lpStartAddress as UINT,lpParameter as POINTER,dwCreationFlags as UINT,lpThreadId as POINTER),UINT from your earlier work and removing the underline prefix just as you suggested.

Anyway, the following program snippet runs like a champ. 

Code Select Expand



$INCLUDE "windows.inc"

DECLARE IMPORT,CreateThread(lpThreadAttributes as POINTER,dwStackSize as UINT,lpStartAddress as UINT,lpParameter as POINTER,dwCreationFlags as UINT,lpThreadId as POINTER),UINT

OPENCONSOLE

'Thread variables
DEF Data_1 = 1 as INT 'Thread 1 data
DEF Data_2 = 2 as INT 'Thread 2 data
DEF Data_3 = 3 as INT 'Thread 3 data

DEF nTemp_1 as INT
DEF nTemp_2 as INT
DEF nTemp_3 as INT

DEF hThread_1 as INT 'Thread 1 handle
DEF hThread_2 as INT 'Thread 2 handle
DEF hThread_3 as INT 'Thread 3 handle

DEF Array_of_Thread_Handles[3] as INT 'Array to store thread handles
DEF bResult,err as INT

hThread_1 = CreateThread(NULL,0,&Process_1,&Data_1,CREATE_SUSPENDED,&nTemp_1)
IF (hThread_1 = NULL)
err = _GetLastError()
PRINT "Thread 1 could not be created!\n"
ELSE
_ResumeThread(hThread_1)
ENDIF

hThread_2 = CreateThread(NULL,0,&Process_2,&Data_2,CREATE_SUSPENDED,&nTemp_2)
IF (hThread_2 = NULL)
err = _GetLastError()
PRINT "Thread 2 could not be created!\n"
ELSE
_ResumeThread(hThread_2)
ENDIF

hThread_3 = CreateThread(NULL,0,&Process_3,&Data_3,CREATE_SUSPENDED,&nTemp_3)
IF (hThread_3 = NULL)
err = _GetLastError()
PRINT "Thread 3 could not be created!\n"
ELSE
_ResumeThread(hThread_3)
ENDIF

SUB Process_1(param AS POINTER),INT
PRINT "Start Process A >"
FOR i = 1 to 6
PRINT "  Process Loop A-",i
NEXT i
PRINT "End Process A >"

RETURN 0
ENDSUB

SUB Process_2(param AS POINTER),INT
PRINT "Start Process B >"
FOR i = 1 to 6
PRINT "  Process Loop B-",i
NEXT i
PRINT "End Process B >"
RETURN 0
ENDSUB

SUB Process_3(param AS POINTER),INT
PRINT "Start Process C >"
FOR i = 1 to 6
PRINT "  Process Loop C-",i
NEXT i
PRINT "End Process C >"
RETURN 0
ENDSUB

'Close all threads
Array_of_Thread_Handles[0] = hThread_1
Array_of_Thread_Handles[1] = hThread_2
Array_of_Thread_Handles[2] = hThread_3

bResult = _WaitForMultipleObjects(3,&Array_of_Thread_Handles,TRUE,INFINITE)
IF bResult <> NULL
err = _GetLastError()
PRINT "Could not close all threads!\n"
ELSE
_CloseHandle(hThread_1)
_CloseHandle(hThread_2)
_CloseHandle(hThread_3)
ENDIF

DO
UNTIL INKEY$ <> ""
CLOSECONSOLE
END



I also got the mutex's working properly following your examples.

Logman