Porting of Win32 API WaitFor to Solaris Platform

Following is a list of requirements for the solution:

1. More than one thread could be waiting at any one time on the same handle.
2. Spurious signals may not be sent to threads.
   Note: For example, if thread 1 is waiting on two handles, A and B, and thread 2 is waiting on two handles A and C, and handle B is set, a signal may not be sent to thread 2 just because it is waiting on handle A common to threads 1 and 2.
3. No polling.

The solution detailed in this paper makes use of the subscription model, that is, the interested thread subscribes itself to the interested wait object and goes to sleep until signaled.

The handle is a composite object containing a list of mutex_cond objects. Each mutex_cond object contains a condition variable and a mutex.

Back to Top

The handle structure will look like this:

Handle{
  List mutex_cond;
  Boolean is_signaled;

}

mutex_cond{
  Mutex mutex;
  Cond_var cond_var;
}

Instead of having two lists, one of mutex and the other of cond_var, the mutex_cond object is created so that the action of adding the pair of mutex and condition variables to the handle is atomic.

The model proposes that each thread that issues or executes on a WaitFor call on any one or more handles subscribes itself to those handles by adding a pair of mutex/cond_var to that handle list and then waits on that mutex/cond_var pair. When a handle is set, all threads waiting on that handle are signaled. Thus, if two threads wait on a handle, there will be two elements in the list of that handle. And when the handle is set, both of the threads are signaled.

ALL or logical AND

In the case of ALL or logical AND, the thread needs to wait for all handles to be set, or for timeout, before exiting from the WaitFor call. One mutex_cond object containing a condition variable and a mutex is created for every handle and added to that handle's list.

Back to Top

The code will look like this:

Boolean WaitForMultipleObjects(int Count, void 
**Handle, boolean fWaitall, long msTimeOut)
{
 mutex_cond_t mutex_cond[Count];

 for(i=0; i < COUNT; I++){ 
   INITIALIZE THE MUTEX AND CONDITION VARIABLE 
   CREATE A NEW MUTEX_COND OBJECT MUTEX_COND[I]
   ADD THE MUTEX_COND OBJECT TO THE LIST OF THE 
   HANDLE[I]
 }

   FOR(I=0; I < COUNT; I++){ 
   LOCK MUTEX_COND[I]->mutex 
   while (! Handle[i]->is_signaled) {
     Wait until timeout on mutex_cond[i]->cond_var
   }
   UnLock mutex_cond[i]->mutex
 }

   for(i=0; i < COUNT; I++){ 
   DELETE MUTEX_COND[I] FROM HANDLE LIST
 }
}

ANY or logical OR

In the case of ANY or logical OR, the preceding example illustrates that instead of the thread creating a new mutex_cond object for each handle, there is a single common mutex_cond object for all the handles specified, and the thread wait on this common mutex_cond object for all the handles. In other words, the thread subscribes to each handle with the common mutex_cond object, and waits on it. So, if any handle gets set, the thread returns from the WaitFor method as expected. This prevents the need for any kind of polling, which can be highly CPU intensive.

Back to Top

The code will look like this:

Boolean WaitForMultipleObjects(int Count, 
  void **Handle, 
boolean fWaitall, long TimeOut)
{
 Create one common mutex_cond object: 
   common_mutex_cond
 Initialize the mutex and condition variable 

 for(i=0; i < COUNT; I++){
   ADD THE COMMON_MUTEX_COND OBJECT TO THE LIST OF THE
   HANDLE[I] 
   }

   LOCK COMMON_MUTEX_COND->mutex

   for(i=0; i < COUNT; I++){ 
   CHECK FOR EACH HANDLE IF IT HAS BEEN SET, IF 
   SET EXIT
   }

   IF (NONE OF THE HANDLES HAVE BEEN SET) {
   WAIT UNTIL TIMEOUT ON COMMON_MUTEX_COND->cond_var,
   common_mutex_cond->mutex 
   }

 UnLock common_mutex_cond->mutex

 for(i=0; i < COUNT; I++){ 
   DELETE COMMON_MUTEX_COND ELEMENT FROM LIST OF
   HANDLE[I]
 }
}

Back to Top

SIGNALING

When a handle is set, a signal is sent to all threads waiting on that handle.

void HandleSet(void *Handle ){
 Iterate through the Handle List {
   Lock Handle->List_element->mutex
 }

 Iterate through the Handle List {
   Signal Handle->List_element->cond_var
 }

 Iterate through the Handle List {
   Unlock Handle->List_element->mutex
 }
}

The complete code for the case of an event handle and a test driver is available in the Appendix. The design is generic and this code can be modified to use a mix of one or more handle types. The code uses the POSIX thread library.

The same subscription model can be used for the case of WaitForSingleObject if the Count parameter in the case of WaitForMultipleObjects is set to 1.

For example:

Boolean WaitForSingleObjects(void **Handle, long 
  msTimeOut)
{
  WaitForMultipleObjects(1, **Handle, AND, 
    msTimeOut);
}

Back to Top

LIMITATION

1. Synchronized objects states are not modified. In other words, they are not non-signaled.
2. Named handles require the handles to reside in shared space, and could be acted upon by other processes. This requires that the mutex and condition variables be created with system scope (the threads are already created with system scope). The shared space could be mmaped on startup and shared by different processes.
3. Handles can be closed multiple times and need to be freed when the reference count goes to zero.
4. There is no check for duplicate handles.
5. Handle type is not checked.

CONCLUSIONS

The port of WaitForMultipleObjects() Win32 API has been successfully done and can be used by other groups for porting Win32 applications from NT to the Solaris operating environment. Much more than that, this implementation of the handle structure forms the basic building block, making use of the Solaris synchronization variables, on top of which complex synchronization objects can be built.

REFERENCES

1. Kleinman, R. (1995) Emulation of Server-side Win32 Functionality under Solaris.
2. Ruediger R. Asche (1994) Compound Win32 Synchronization Objects

Back to Top

APPENDIX

The source code includes a file called WaitForMultipleObjects.c, a file called include_type.h and a file called driver.c. driver.c is a test driver that can be used to test the ported API. include_type.h defines the mutex_cond and event objects. WaitForMultipleObjects.c is the source code for the port of the API. This code uses the POSIX library. (Source code tar file and README.)

The source code can be compiled on Solaris 8 as:

cc-o driver driver.c 
WaitForMultipleObjects.c-DREENTRANT-D_POSIX_C_SOURCE=
199506L-lrt-lpthread


/***************************************************/


/* File include_type.h */


#define AND 1
#define OR 0
#define      NULLP ((void *)0)


/* Type of the mutex_cond object */

typedef struct mutex_cond{
        pthread_mutex_t i_mutex;
        pthread_cond_t i_cv;
}mutex_cond_t;

/* Type of the List element object */

typedef struct List_element{
        struct List_element *next;
        struct List_element *prev;
        struct mutex_cond *i_mutex_cond;
}List_element_t;

/* Type of the List object, Any other 
 *  implementation of List can be used
*/

typedef struct List{
        struct List_element *start_list;
        struct List_element *end_list;
        /* This mutex is used to protect the List
        *critical section 
        */
        pthread_mutex_t i_mutex;        

}List_t;

/* Type of the Event Handle object */

typedef struct win32_event {
        boolean_t i_signalled;
        struct List *i_list;
}solaris_win32_event_t;

/* End include_type.h */ 
/************************************************/
/* File WaitForMultipleObjects.c */ 

#include < STDIO.H >
#include < STDLIB.H >
#include < PTHREAD.H >
#include < TIME.H >
#include < ERRNO.H >
#include "include_type.h"

void AddToList();
void DeleteFromList();

/* This method gets the actual system time at
 * which the timeout must occur, based on the 
 * timeout parameter
 */

static boolean_t ClockGetTime(struct timespec *tp, 
long msTimeOut)
{
  long seconds, nseconds; /*for time calculations*/

  if (clock_gettime(CLOCK_REALTIME, tp) != 0)
     return(_B_FALSE);


   /* since timeOutPeriod is in milliseconds and  
    * tv_nsec is nanoseconds,and both are long  
    * integers, we'd rather extract the second   
    * andnanoseconds from the timeOutPeriod and add  
    * them separately, making sure we're not over  
    * 1,000,000,000 nanoseconds.
    */
       seconds = msTimeOut / 1000;
       nseconds = (msTimeOut - seconds*1000)*1000000;

       tp->tv_sec += seconds;
       tp->tv_nsec += nseconds;
       if (tp->tv_nsec >= 1000000000)
       {
          tp->tv_nsec -= 1000000000;
          tp->tv_sec += 1;
       }

       return(_B_TRUE);
}


/* Method for setting the Event object. Signal
 * is sent to all threads waiting on that event
 * to be set.
 */

void
EventSet(void *Event)
{
  solaris_win32_event_t *event = 
  (solaris_win32_event_t *)Event;
  struct List_element *start;
  struct List_element *p;

  event->i_signalled = _B_TRUE;

  /*
   * At this point iterate through the list
   */

  start = event->i_list->start_list;

  if ( start == NULLP )
  {
     fprintf(stderr, " Nothing to iterate...no 
     elements in the list!n");
     return;
  }
  /*
   * All the mutexes on the Event Handle List are
   * first locked, then all condition variables are
   * signalled and then all mutexes are unlocked. 
   * This is done to prevent any race condition.
   */

     p = start;
     do {
          pthread_mutex_lock( 
          &p->i_mutex_cond->i_mutex );
          p = p->next;
 
      } while ( p != NULLP );

      p = start;
      do {
           pthread_cond_signal( 
           &p->i_mutex_cond->i_cv ); 
           p = p->next;

      } while ( p != NULLP );

      p = start;
      do {
           pthread_mutex_unlock( 
           &p->i_mutex_cond->i_mutex ); 
           p = p->next;

      } while ( p != NULLP );

} /* EventSet */

boolean_t 
WaitForMultipleObjects(int Count, void 
**Events, boolean_t fWaitall, long msTimeOut)
{
  solaris_win32_event_t *event;
  boolean_t Result;
  int i;
  struct timespec to_time; /* time value */
  struct timespec delta_time; /* time value */
  mutex_cond_t *p, *common_mutex_cond;
  List_element_t *q;
  /*
   * Local array of mutex_cond objects and List 
   * is maintained so that a search through the 
   * Event list is not required later when these 
   * objects are needed.
   */

   mutex_cond_t **mutex_cond;
   List_element_t **list_element;
   mutex_cond = (mutex_cond_t **)malloc(sizeof
   (mutex_cond_t *)* Count);
   list_element = (List_element_t **)malloc(sizeof
   (List_element_t *)* Count);

   for (i = 0; i < COUNT; I++)
   {
     EVENT = (SOLARIS_WIN32_EVENT_T*)
     EVENTS[I];
     IF (EVENT == NULL) RETURN(_B_FALSE);
   }

   IF (MSTIMEOUT != -1)
   {   
      IF (CLOCKGETTIME(&TO_TIME, MSTIMEOUT) == 
      _B_FALSE)
      {
        RETURN(_B_FALSE);
      }
   }    


   SWITCH (FWAITALL){

   CASE AND:
        RESULT = _B_TRUE;
        /*
         * MUTEX_COND OBJECTS ARE CREATED AND 
         * INITIALIZED, LIST OBJECT ELEMENTS ARE 
         * CREATED, AND MUTEX_COND OBJECT IS
	 * IS ADDED TO EACH LIST ELEMENT.
         */

         FOR(I=0; I < COUNT; I++){
           SOLARIS_WIN32_EVENT_T *EVENT = 
           (SOLARIS_WIN32_EVENT_T *)
           EVENTS[I];P = (MUTEX_COND_T *)  
           MALLOC(SIZEOF(MUTEX_COND_T));
           PTHREAD_MUTEX_INIT ( &P->i_mutex, 
           NULL); pthread_cond_init 
           ( &p->i_cv, NULL); q = 
           (List_element_t *) malloc(sizeof
           (List_element_t));mutex_cond[i] 
           = p; q->i_mutex_cond = p;
           list_element[i]=q;

           /*
            * At this point we need to 
            * add the malloced object to
	    * the list
            */

            AddToList( event, q);
          }
          /* Wait on each event to be set or 
	   * timedout 
           */
          for(i=0; i < COUNT; I++){
            SOLARIS_WIN32_EVENT_T *EVENT = 
            (SOLARIS_WIN32_EVENT_T *)
            EVENTS[I]; PTHREAD_MUTEX_LOCK
            (&(MUTEX_COND[I]->i_mutex));
            while(!event->i_signalled){
              if (msTimeOut != -1) {
                if (pthread_cond_timedwait
                (&mutex_cond[i]->i_cv, 
                &mutex_cond[i]->i_mutex, &to_time)
                == ETIMEDOUT)

                {
                  Result = _B_FALSE;
                  break;
                }
              }
              else 
              {
              pthread_cond_wait(&mutex_cond[i]->i_cv,
              &mutex_cond[i]->i_mutex);
              }
            }
            pthread_mutex_unlock
            (&mutex_cond[i]->i_mutex);
            if (Result == _B_FALSE)
            {          
              break;
            }
          }
          /*               
           * Delete all the List elements created
           * by this thread for each event.
           */

           for(i=0; i < COUNT; I++){
             SOLARIS_WIN32_EVENT_T *EVENT = 
             (SOLARIS_WIN32_EVENT_T *)EVENTS[I];
             DELETEFROMLIST(EVENT, LIST_ELEMENT[I]);
           }
           BREAK;

case OR:
           Result = _B_FALSE;
           /*
            * A common mutex_cond object is created 
            * and initialized, List object elements 
            * are created, mutex_cond and same   
	    * common object is added to each List 
	    * element.
            */

            common_mutex_cond = (mutex_cond_t*)
            malloc(sizeof(mutex_cond_t));
            pthread_mutex_init(
            &common_mutex_cond->i_mutex, NULL);
            pthread_cond_init(
            &common_mutex_cond->i_cv, NULL);
            for(i=0; i < COUNT; I++){
                SOLARIS_WIN32_EVENT_T *EVENT = 
                (SOLARIS_WIN32_EVENT_T*)EVENTS[I];
                Q = (LIST_ELEMENT_T *) MALLOC(SIZEOF(
                LIST_ELEMENT_T));
                Q->i_mutex_cond = common_mutex_cond;
                list_element[i]=q;
               /*
                * At this point the malloced List 
                * element is added to the Event 
	        * Handle.
                */
                   AddToList(event, q);
              }
               /*
                * It is first tested if any of the
                * event has already been set, if yes,
                * the function returns as expected.
                * Or else, wait on the common mutex
                * for any event tobe set or time out.
                */

                pthread_mutex_lock(
                &common_mutex_cond->i_mutex);
                for(i=0; i < COUNT; I++){
                  SOLARIS_WIN32_EVENT_T *EVENT =    
                  (SOLARIS_WIN32_EVENT_T*)EVENTS[I];
                  IF(EVENT->i_signalled){
                    Result = _B_TRUE;
                    break;
                  }
                }
                if(Result == _B_FALSE){
                  if (msTimeOut != -1) {
                    if (pthread_cond_timedwait(
                    &common_mutex_cond->i_cv, 
                      &common_mutex_cond->i_mutex,
                      &to_time) != ETIMEDOUT)
                    {
                      Result = _B_TRUE;
                    }
                  }
                  else
                  {
                    pthread_cond_wait
                    (&common_mutex_cond->i_cv,
                    &common_mutex_cond->i_mutex);
                    Result = _B_TRUE;
                  }
                }
                pthread_mutex_unlock
                (&common_mutex_cond->i_mutex);
                /*
                 * Delete all the List elements 
                 * created by this thread for
                 * each event.
                 */
                 for(i=0; i < COUNT; I++){
                   SOLARIS_WIN32_EVENT_T *EVENT = 
                   (SOLARIS_WIN32_EVENT_T *)
                     EVENTS[I];
                   DELETEFROMLIST(EVENT, 
                   LIST_ELEMENT[I]);
                 }
                 BREAK;
               DEFAULT: 
                 BREAK;
               }
               FREE(MUTEX_COND);
               FREE(LIST_ELEMENT);
               RETURN RESULT;

} /* WaitForMultipleObjects */

void
AddToList( solaris_win32_event_t *event, 
List_element_t *new_element)
{
  struct List_element *start, *end;
  struct List_element *p = (struct List_element*)
  new_element ;

  pthread_mutex_lock(&event->i_list->i_mutex);

  start = event->i_list->start_list;
  end = event->i_list->end_list;

  if ( start == NULLP ) {
    /* First element of the list */
    start = p;
    p->next = NULLP;
    p->prev = NULLP;
    end = p;
    event->i_list->start_list = p;
    event->i_list->end_list = p;
  }
  else {

       /* 
        * Already some elements in the list...
        * add this element at the end 
        */
        end->next = p;
        p->prev = end;
        p->next = NULLP;
        end = p;
        event->i_list->end_list = p;
  }
  pthread_mutex_unlock(&event->i_list->i_mutex);

} /* AddToList */

/*
 * Delete element from the list of mutex_cond_t...
 */
void
DeleteFromList( solaris_win32_event_t *event, 
List_element_t *element )
{
  struct List_element *start, *end, *q;
  struct List_element *p = (struct List_element *)
  element ;

  pthread_mutex_lock(&event->i_list->i_mutex);

  start = event->i_list->start_list;
  end = event->i_list->end_list;

  if ( start == p && end == p )
  {
    /* only element in the list */ 
    start = end = NULLP;
    /*
     * Call the appropriate function for freeing
     * the memory
     */
     free(p);
     event->i_list->start_list = NULLP;
     event->i_list->end_list = NULLP;
  }
  else if ( start == p ) {
    q = p->next;
    start = q;
    q->prev = NULLP;	
    free(p);
    event->i_list->start_list = start;
  }
  else if ( end == p ) {
    end = p->prev;	
    end->next = NULLP;
    free(p);
    event->i_list->end_list = end;
  }
  else {
        /*
         * Element is somewhere in the middle of the
         * list 
         */
         (p->prev)->next = p->next;
	 (p->next)->prev = p->prev;
         free(p);
   }
   pthread_mutex_unlock(&event->i_list->i_mutex);
} /* DeleteFromList */

/* End WaitForMultipleObjects.c */
/************************************************/
/* 
 * File driver.c
 * This is the driver file for testing the 
 * ported API
 */

#include 
#include 
#include 
#include "include_type.h"


#define NUM_THREADS 5 /* Number of threads created */
#define EVENT_COUNT 3 /* Actual number of events 
                   /* passed to the WaitFor object */

void  *or_wait_thread(void *);
void  *and_wait_thread(void *);
void  *signal_thread(void *);
solaris_win32_event_t *event[EVENT_COUNT];

pthread_t thread[NUM_THREADS];
pthread_attr_t attr;

main() {
	int i;
	for(i=0; i < EVENT_COUNT; I++){
	  EVENT[I]= (SOLARIS_WIN32_EVENT_T*)
          MALLOC(SIZEOF(SOLARIS_WIN32_EVENT_T));
          EVENT[I]->i_signalled = _B_FALSE;
	  event[i]->i_list = (struct List *)
          malloc(sizeof(struct List));
          pthread_mutex_init(&(
          event[i]->i_list->i_mutex), NULL);
          event[i]->i_list->start_list = NULLP;
	  event[i]->i_list->end_list = NULLP;

	}
	pthread_attr_init(&attr);
	pthread_attr_setscope(&attr, 
        PTHREAD_SCOPE_SYSTEM);
	pthread_attr_setdetachstate(&attr, 
        PTHREAD_CREATE_DETACHED);
	for(i=0; i < NUM_THREADS-3; I++){
	  PTHREAD_CREATE(&THREAD[I], &ATTR, 
          OR_WAIT_THREAD, (VOID *)NULL);
	}
	PTHREAD_CREATE(&THREAD[NUM_THREADS-3],
        &ATTR, SIGNAL_THREAD, (VOID *)NULL);
	PTHREAD_CREATE(&THREAD[NUM_THREADS-2],
        &ATTR, AND_WAIT_THREAD, (VOID *)NULL);
	PTHREAD_CREATE(&THREAD[NUM_THREADS-1],
        &ATTR, AND_WAIT_THREAD, (VOID *)NULL);


	PTHREAD_EXIT(0);

}

VOID *OR_WAIT_THREAD(VOID *ARG){
        WAITFORMULTIPLEOBJECTS(EVENT_COUNT, EVENT,
        OR, 100000); 
}


VOID *AND_WAIT_THREAD(VOID *ARG){
        WAITFORMULTIPLEOBJECTS(EVENT_COUNT, EVENT,
        AND, 100000);
}

VOID *SIGNAL_THREAD(VOID *ARG){
	SLEEP(5);
	EVENTSET(EVENT[2]);
	SLEEP(5);
	EVENTSET(EVENT[1]);
	SLEEP(5);
	EVENTSET(EVENT[0]);
}

/* END OF DRIVER.C */
/*************************************************/

September 2000, updated January 2001

Back to Top