Homeby Nikolai Sklobovsky,
Senior System Analyst,
Retail Technologies International, Inc.
May 9 2000: Published in June issue of Delphi Informant, namely here.
Multi-threaded programming has never been an entry-level subject. Even with the proper wrapper classes, which modern development tools provide so generously, it is not always an easy task to create an elegant, smoothly working multi-threaded application. Two major problems that a novice programmer is apt to encounter when dealing with multiple threads are synchronization and visualization. Debugging multiple threads also can hardly be called a pleasure. Even if your favorite IDE provides thread-debugging support, the extremely volatile nature of threads-based processes often prevents you from using IDE and makes you go back to the good old "debug print" technique. And sometimes even this does not help...
Several years ago I reached a point where multi-threaded programming ceased to be just a buzzword and became the sine qua non condition for my projects. After having had enough painful "fun" with my first threads, especially on multi-processor machines, I developed a technique for dealing with them. Since that time neither my colleagues (who gladly employed this technique in a variety of projects) nor I have had a single problem with multiple threads. Multi-threaded programming has now become routine, bringing with it many perks and, if properly used, no pain. During these years this technique has been refined, revised and thoroughly tested. We now have many mission-critical applications that use it in a 24x7 manner.
The purpose of this article is to share this robust, reliable and relatively simple method with a broad community of fellow programmers. If you are already familiar with the subject and already have enough multi-threaded experience under your belt, you can jump right to the implementation section.
As you will notice, only a few code excerpts are included in the article's text. The rest of the code is provided separately. You can download a zip file by clicking here.
While the technique itself is non language-specific and can be implemented in different languages in various ways, all the examples here are written in Delphi. You can basically use any 32-bit version of the compiler, although the latest possible version is preferable. You should also be familiar with basic Delphi and threads-related terminology. For this terminology, refer to the Delphi manuals and its online help system or to various books, such as "Delphi Developer's Guide" by Teixeira & Pacheco, Charlie Calvert's "Delphi Unleashed" and "Mastering Delphi" by Marco Cantù.
back to the topOne of the most important questions for a developer to ask about any new technology is "Why should I use it?" or "How would I (or my application) benefit from it?". This question arises naturally because most, if not all, of these technologies promise a lot in some foggy future while the suffering starts almost immediately. Unfortunately - or quite fortunately for us IT professionals - in programming, as in geometry, there is no king's way to the bright shiny tops of today's "cutting edge". Usually the learning curve is very steep, which means you must acquire tons of new information before your application even starts working at all, let alone producing any benefits. The standard rule is: if your app already works the way you want it, don't touch it. This simple policy works pretty well until that sad moment when you realize that you've already used 100% of your existing arsenal to make an app work - and still it doesn't. This is the moment of truth. Now you'd better be familiar with the new technology that you're about to use. Otherwise you might make an erroneous choice with disastrous results.
MS Windows began providing support for multiple threads with its first 32-bit version. The benefits were very attractive. After a mess of 16-bit event-driven-only cooperative multi-tasking, when any extra millisecond in a tight loop could easily result in a system-wide "freeze", programmers were finally given a nice opportunity to separate hard working - and mostly sequential - data crunching business processes from relatively slow, random, unpredictable event-based GUI. It also opened a door for smooth background processing. The only thing you seemed to need to do was to isolate those processes, put them into separate threads - ét voilá! You can forget about taking complex and sophisticated measures allowing the other parts of your app and the system itself to breathe while you are busy digging your data. Everything became simple and straightforward, almost like in the old DOS world :-). Each part was simply doing its job while the operating system provided each with its fair share of the CPU cycles. Simpler implementations meant fewer bugs and a shorter path to market. Long live multiple threads! (Oh, by the way, what exactly is it that I must do to use this wonderful technology?)
Delphi has offered thread support starting with its first 32-bit version, namely Delphi 2, which hit the market soon after Windows'95. Although Delphi provided a wrapper class conveniently named TThread, still, as is always the case with Delphi, you could use bare-metal Win32 API thread routines instead. Examples and online help articles provided some decent information on the subject, so hundreds and hundreds of programmers rushed to use - and to fight - this fascinating innovative technology
And many were slain.
back to the topAlmost all thread-specific problems arise from a single paradigm change: you no longer live in a synchronous world. You can no longer assume that the next line of code will be reached right after the current one has been executed. To make things worse, you cannot even assume that any given single line of your high-level Delphi code will be executed as a whole. Instead, a thread or process switch may happen at any moment - sometimes even during a simple assignment operation.
In the old sequential world you could think of yourself as a mighty lone warrior. Your errands were tough, but at least you knew your exact position at any particular moment. Now, suddenly, you are leading a commando squad. Although your troops still accept your commands instantaneously, it takes an unpredictable amount of time for them to be implemented.. You immediately experience a certain lack of control. Each of your troopers being properly trained for a specific job, they can carry it out very effectively, much more effectively than you used to do. But this also means that they no longer report their every step or ask your permission to move. Radio silence. Once they have accepted the task they disappear into the jungles of CPU. The only way to get them back is to wait until they are done or to kill them.
Under the new conditions you need to radically revise your strategy. Rather than continuing to think about the whole campaign every single moment, you have to design the campaign in advance, discuss each part with the relevant participant, provide a reliable communications link between them when joint efforts are required, and then sit back and relax, watching their progress and making small corrections, as needed. Sounds easy, eh? Yes, it really is, provided you've managed to complete your part, i.e. to design the campaign, establish the communications, and monitor the action.
back to the topMany people have recognized that multi-threaded applications typically require much more planning than conventional single-threaded ones. Basically, threads usage is very similar to units' usage in any strategic computer game. The overall success depends not only on the strength and training of each unit, but also on how well the plan was designed and thought out. The best troops can lose to the weakest enemy if not properly deployed; the best plan can fail if the tools are not good enough.
The benefits and desirability of good design were generally recognized in conventional programming. In the world of multi-threading, good design is pure necessity. Proceeding here without good design makes failure inevitable.
back to the topCommunication between the deployed units and HQ is required for total victory. You need to know, for instance, that the bridge ahead is secured before you advance your main forces to the river. For multiple threads this essential communication is called synchronization. Unfortunately, this part of Delphi's implementation of thread support is very unsafe and unclear.
Let's consider the OnTerminate event. At first sight, this appears to be a perfectly convenient way for a thread to say: "hey, I'm done, you can proceed safely". Unfortunately, in real life it's no so simple. The problem is that by default this event is fired (via Synchronize method) while the thread is still alive. As the result, all that you can do at this point is to set up a flag or send yourself an asynchronous message. You definitely cannot proceed. If you try, your app will be caught in a deadlock or you will encounter some nasty random AV effects when you try to close it. Of course, you might think that you could override the virtual DoTerminate method. This would seem to be a great idea until you actually start implementing it. The guys at Borland didn't leave it this way just because they didn't care even though there was an easy way out. There simply wasn't.
Another easy trap to fall into concerns the similarly named couple Terminated/Terminate.
You might think that once you've called Terminate method for a thread,
that thread will magically terminate. You may be even so naive as to expect it to stop working immediately.
It ain't so. Terminate is a static procedure that simply sets the protected read-only
Terminated property to TRUE, and that is all it does! Basically, Delphi provides a programmer
with a free Boolean thread-located variable and a decent way to set its value to TRUE.
All further responsibility for checking this variable value falls to the application programmer,
making him insert endless conditional operators in loops, thus fogging what formerly was a
crystal clear, simple algorithm.
As you can see, Delphi's support for "polite" thread termination is quite limited. Besides, if you use native thread API rather than TThread object you cannot use this support at all.
back to the topAnother important aspect of multi-threaded applications is feedback - visual or otherwise - from the working threads. With VCL itself being officially non-thread-safe, visual feedback draws especially hot attention from the programmers. Delphi developers have suggested a simple solution to this problem, namely the Synchronize method mentioned above. This method acquires VCL-global critical section, effectively making VCL thread-safe for this particular thread for the duration of each call. You can think of this as two highways that merge at some point, run together for a while, and then finally part and go their own separate ways until the next merge somewhere in the future.
This simple solution works surprisingly well until you decide to check for your end-user reaction during this synchronize-based call. Then all of a sudden, your calling (working) thread stops and waits along with you. Quite often this is exactly the opposite of your intentions. You had hoped that the thread would work steadily regardless of innocent end-user actions. And even if you didn't launch any message boxes, simple mouse moving (and what else is a poor user supposed to do while looking at your progress bar but play with his/her mouse?) can slow your "synchronized" application manifold, thus immediately losing one of the most important benefits: performance.
Another problem may also arise if - for some good reason - you decide to not use TThread object and switch to native Win32 API calls instead. You've already lost Terminate group, and now there is no Synchronize for you either! But this is unfair! You didn't say that you were not going to use VCL at all; you just didn't want to use its poorly developed thread-support.
back to the topA cure for these problems can be found when you realize that all your multi-threaded designs have a lot in common (at least mine do:-).
Let's consider the simplest case, a two-level multi-threaded application. The main thread, which always exists, will be responsible for GUI and user interaction. One or more secondary threads will perform some useful data-processing work, such as sorting thousands of rows or copying multi-megabyte files.
At this point the design seems to be nice and solid, leaving us with only two things remaining to be worked out: communication (synchronization) and feedback (visualization). We would like for each thread's progress to be smoothly displayed with its own labeled progress bar and for each thread to have a humble "Cancel" button just in case something goes wrong. By "smooth display" of the thread's progress we mean that a working thread should be free to inform its host about its own status whenever it's most suitable for the thread, not necessarily for the host. At the same time, we also expect the host to report the current status in a uniform way, say, two times per second, regardless of the thread's capability to speak. Implementing a "Cancel" button means that any secondary thread - TThread-based or not - can be easily and almost instantly "terminated" at any moment. "Almost" is good enough (a thread may actually need some time for closing files, releasing resources, etc.) provided our user can have immediate feedback that the thread has accepted the command and is carrying it out as soon as possible.
We also want all our secondary threads to terminate peacefully in the event that the user decides to close the application or shutdown the whole system. And, of course, we would like to have a chance to get rid of all the thread-associated GUI components once the corresponding thread has finished its job. It goes without saying that during all this data processing your main window should remain as sensitive to the slightest user action as if it were doing nothing else at all.
In other words, we want our working secondary thread to be free to do its primary job and
to diligently report its progress without any severe loss of performance.
At the same time we want our primary GUI thread to be able to display this
information and to
have ultimate control over the working thread's life and death.
This means that a certain amount of work needs to be done by somebody to accomplish our wishes.
Which means - we need an agent.
Actually, we need two. One agent should be available to the working processes. It needs to be small and simple and, theoretically speaking, could even be absent altogether. Our troopers cannot carry a home theater on a mission. Likewise, they certainly should not slow down if their radio goes dead. On the contrary, they should simply throw the whole set away and move faster.
The other agent needs to be more bulky. This will be a command center, a central headquarters. It should take care of all the incoming signals from multiple field devices (which may reside in different threads), serialize them and provide a convenient way for the GUI to retrieve this valuable information.
back to the topThe primary roles of the field agent are to provide a working thread with an easy way to inform HQ about its progress and to allow HQ to send it a self-destruction signal, if necessary. This agent should be very lightweight and its absence should not prevent a working thread from doing its job.
That last part may sound tricky for a novice Delphi user, but actually there is a very simple solution for this kind of problem. Delphi itself uses it in the ubiquitous Free method. The only thing you need to do is to make sure that all the methods you are about to publish are static (not virtual or dynamic) and that the first line of all these methods looks like:
if self = nil then EXIT;Simple enough! Now we can develop our business algorithms without bothering to check whether our agent is present or not - its code will handle a nil situation automagically.
The public portion of our field agent - let's call it Client - may appear as follows:
function Start(const csCaption: string;
bProgressable: boolean = FALSE;
pUserData: pointer = nil
): integer;
procedure Finish(pid: integer = pidCurrent); // always accepted if pid is ok
procedure Report(const csText: string = '';
bImportant: boolean = FALSE;
pid: integer = pidCurrent); overload;
procedure Report(Index, Count: integer;
const csText: string = '';
pid: integer = pidCurrent); overload;
procedure ReportAlways(const csText: string;
ErrorClass: ExceptClass = nil;
pid: integer = pidCurrent); // always accepted if pid is ok
Here the Start and Finish methods serve to denote the entry and exit point of some logical process,
while the Report methods obviously do a humble servant's job in providing HQ with valuable progress data.
Thus, if the working method of the original business process looked like this:
Procedure DoTheJob;
var
i, iCount: integer;
begin
iCount := 10000;
for i := 0 to pred(iCount) do
begin
DoSomething(i);
end;
DoSomethingElse;
end;
Then our agent-aware version would look like this:
Procedure DoTheJob(Client: TOurCustomFieldAgent);
var
i, iCount: integer;
begin
Client.Start('Doing something', TRUE);
try
iCount := 10000;
for i := 0 to pred(iCount) do
begin
Client.Report(i, iCount);
DoSomething(i);
end;
Client.Report('Doing something else', TRUE);
DoSomethingElse;
finally
Client.Finish;
end;
end;
Fair enough. If this thing really works this way, you could possibly buy it.
At this point you might have a couple of questions, such as why do we need to insert a
try-finally pair, and what are those suspicious comments about "acceptance" all about?
And, by the way, how are we supposed to "terminate" this process in case we need to?
We'll get to the answers to these and other questions right around the corner, in the next section.Ok, folks, now it's time to raise the curtain, or should I say - the "exception"? Of course, you've got it! Our agents utilize this powerful technique to maximize both the robustness and the safety of the application.
The idea is very simple. The business process does not monitor our agent, it just calls its methods. However, it must be prepared in case an agent raises an exception in response to one of these calls. This, in turn, would result in immediate quitting from all the loops and call stacks, through all the except and finally parts, of course. Thus, we can solve both problems by having the thread use the same line to both report its status and receive an "abort" feedback via exception.
Exceptions are very powerful - and this means they should not be abused. Our agents are smart enough to NOT raise an exception in response to a Finish method or to a special method called ReportAlways (sometimes you need to send some information to HQ even if you know that the mission has been aborted, such as when you know who the mole is. :-)
Let's briefly scan over other features of our Client before we switch to its more complicated HQ partner.
back to the topEach time you're about to start some logically related group of actions,
it is wise to give this group a human-readable name and call the Start method
for the available Client. If you know that this action can be theoretically associated
with some progress-bar-like UI component, you can set its second argument to TRUE.
This will give you an opportunity to conveniently report on its progress.
Start method always returns an integer: Process ID, or PID. Note that this is not a Windows' process ID, but rather is our own logical ID, which we can use to identify each of our actions. In the vast majority of cases you can simply ignore it - all other methods by default accept the so called "current ID", which is supported by our HQ on your behalf. This current ID is defined by a constant named pidCurrent. Think of PID as of a radio frequency. You generally don't need to know its exact value to be able to say a few words. However, if you want to have ultimate control, here you are.
There is another special ID available. If you don't feel like using the Start/Finish pair but still need to report something, you can use any of the Report methods with a constant pidMain. This "process" always exists and everybody is aware of its presence. You can treat it as a common "open" frequency.
As described above, Finish will never raise an exception (except for the weird case when you try to finish a process with unknown ID).
back to the topClient allows us to send three kinds of reports back to HQ. First, and most commonly used, is a text report. This text information can be either important or unimportant. Importance guarantees delivery. Imagine a situation where you are scanning several drives. You would probably want to notify the end-user about each drive, which could be conveniently handled within the time required to scan each one. However, it would not be important to display all the file names. All important messages are queued and eventually delivered. All unimportant ones use a single storage place, each new message thus overwriting the previous one.
A special kind of report is a progress report. If you declared your process to be "progressable" you can report its progress in a very convenient way. Naturally, this kind of report is treated as unimportant.
Last but not least is the ReportAlways method. Like Finish, it does not raise an exception even if the process or the whole mission has been cancelled. You can use it to send some "famous last words". Its usual place is in the except part of the thread's Execute method.
back to the topThis part of our work is essentially more complex. Agency is responsible for all its field agents. It should provide radio channels (IDs), separate important information from the unimportant, inform the GUI, and solve many other problems "just to keep things moving". Here's how it works.
We have a class (component), which usually resides in a main application thread. Let's name it Log. Internally it creates a critical section. When initialized, it creates an upper-level "activity process" with pidMain ID. All secondary "activity processes" are created during Start calls. For each process there is a queue for important messages, as well as some other data slots, e.g. time of start, current progress, error status, etc. Also, Log starts an internal timer. This timer defines how often GUI will be notified about the new information.
Whenever a working thread calls one of the client's methods, client acquires Log's critical section and then performs all the actions needed. This guarantees data consistency. These notifications are very fast. Hence they do not affect overall performance and the threads do not usually have to wait for each other.
When Log's timer ticks, it first checks whether there was
any new data since the last tick. If there was, Log generates a series of events.
Here is the list:
There are some other events you may find useful. Here they are:
As you can well guess, there is no need to use Synchronize
anymore. Anything you do inside Log's event handler is perfectly thread-safe. Log takes care
of it for you. Another bonus: there is no need to use an OnTerminate event.
Whenever Log detects that a thread's master activity process has finished,
it waits until the thread is actually terminated (via WaitForSingleObject API call)
before firing an OnProcessFinish event.
This suggested approach provides an easy-to-use way for multi-threaded programming. All you need to do is to transfer the data processing algorithms into secondary threads, supply them with the Log's client, and then drop the Log component into your progress dialog form and hook up a few event handlers.
Happy multi-threading!
Nikolai Sklobovsky,
threadmare@sklobovsky.com
Much more practical client-side article is now available here: meet Children of Threadmare.
 You are Visitor No:
|