OpenSSL and Application Complexity

Whew.  My schedule is now back to normal and caught up with my share of Zs.  During my rush, I ran into a possible bug in OpenSSL that caused its random number generator to take more than a minute to initialize when the application heap is overly complex.  The code also crashed in the same 'complex' application environment when executing in a separate thread.

A collegue of mine told me that it might be due to a 'controversial' code within OpenSSL so I am trying to disable that code this morning to see if that makes a difference.  It's silly how supposedly stable code like OpenSSL can become unstable under 'normal' desktop application environment.  Using the delay as a measure of complexity, Netscape 4 takes a few seconds, IE takes tens of seconds, and Acrobat 6 takes a minute.

Update #1: Debugging inside OpenSSL code confirmed that the problem was with Win32 implementation of RAND_poll (rand_win.c) which walks through all the heaps to harvest 8 bits of entropy per block.  There was an upper bound on the number of blocks (50 in the version I had, 80 in latest), but no bound on the number of heaps.  Usually applications use small number of heaps, but Acrobat 6 had 26 heaps.  More heaps and blocks you check, more chance of contention.  Ouch.

My solution was to put a reasonable limit to number of heaps checked.  Maximum of 5 heaps capped the PRNG init time to about 7 seconds instead of 60 seconds or more.  Phew.  I hate messing in crypto crap.