9.4. A Multi-threaded Chat Server — Network Programming Study Guide Navigation index modules | next | previous | Network Programming Study Guide » 9. Topic 6 - Developing Network Server Programs » 9.4. A Multi-threaded Chat Server¶ You may have noticed that after the threaded echo server was discussed in the The Text Book, the author suggests that as an exercise, the reader try writing a multi-threaded chat server. It was nice to see that he left some fun projects for his readers. Well, I thought that sounded like an excellent idea and did just that. It took just a little thought up front to decide how to go about doing it, but once I got the basic design figured out, it wasn’t too bad and was actually a very fun program to write. The chat server developed is a multi-party chat server, sometimes called a chat room, similar to IRC (Internet Relay Chat). It uses straight TCP sockets with it’s own small application layer protocol of a few commands. As with IRC, all the commands begin with slash (/) character. For example, the command: /nick Bob sets the user’s nick name to Bob. Actually there are three programming strategies for developing server programs. In addition to purely threaded and purely asynchronous, a hybrid solution is also possible for many problems. For this problem, the hybrid solution may actually be preferred solution. However, at this point, my chat server is purely based on threads. Following are design considerations for each alternative. 9.4.1. Asynchronous Servers¶ See also Asynchronous Network Server Programming The asynchronous approach works well when any action by the server must be initiated by the server receiving a message from one of the clients being served. This is exactly the situation with a chat server. The main advantage of asynchronous technology is that since there is only one thread, then by definition, the server only does one thing at a time. So, all of the issues associated with synchronizing access to shared data between multiple threads disappears. A disadvantage of this approach is that managing the state and data of each client becomes a juggling exercise. Asynchronous programs can also be fairly tricky to get right and for some problems, just can not be used at all. The main challenge that asynchronous servers face is that no blocking system call can be made. If, in serving one client, the server does not return immediately from a blocking system call, such as to access a data base, acquire a lock, or even to load a file, then the whole program is held up and the other clients are not served. The Text Book shows a chat server that uses Twisted in chapter 22 and I found one on the Internet that uses the select.select() system call. [PILLAI07] (See chatSelectServer.py) Both of these servers keep a list of client sockets and when a message is received, they loop through all the client sockets sending the message to each. This is probably the simplest way to develop a chat server and should have fair performance. These solutions; however, do not keep a record of any old message and thus are not able to show a new user any of the discussion from before they joined the session. Note It should be noted that this is a fairly typical distinction between multi-threaded and asynchronous servers. Multi-threaded servers tend to be more modular and tend to do a better job of keeping track of the clients and associated data. Thus, it can be easier to implement advanced features with a multi-threaded design. 9.4.2. Multi-threaded Servers¶ See also Multi-threaded Network Server Programming As the name implies, a multi-treaded server is one that uses a separate thread of execution for each client. The primary advantage of using multiple threads is that the code for each thread need only worry about one client. If a blocking system call is needed to serve the client, no problem is created because it only affects one client. [1] The code for threaded servers tends to be fairly easy to follow. The most obvious complexity of the multi-threaded approach comes from dealing with data that is shared between multiple threads. This data must be protected with locks and associated synchronization algorithms. These synchronization algorithms can be more complex than one would hope, but there are known solutions to most any synchronization problem. The most difficult chore for the programmer is just to determine which of the generic problems with known solutions can be used to solve the task at hand. In general, once the synchronization issues are resolved, multi-threaded techniques have better potential than asynchronous attempts to scale to larger, more complex problems. Another challenge of multi-threaded network programming stems from the fact that, as was previously mentioned, the default behavior of the socket.recv() function is to block (not return) until either data or a disconnect is received on the socket. For applications, such as web servers where the clients do not interact with each other, this is not a problem at all. When a connection is received, a child thread is started and it waits on the socket.recv() call until data is received and then it collects the response needed, perhaps via a data base query, and returns the results to the client. However, in a multi-party chat server, the thread must not only watch the socket for new data, but must also pay watch for new messages from other users that need to be sent to the client. Thus, the socket.recv() calls can not block indefinitely. In a pure threads based solution, the only way to watch a socket and other data at more or less the same time is to set a timeout on the socket such that after a short time with no data received, a socket.timeout event will be raised. The main disadvantage of this approach is that users are delayed waiting for the timeout before being sent a new message. However, if the timeout value, is kept fairly small, this delay will not generally be noticed by users. However, the performance is perhaps not as good as we would like because of repeated checking for messages after each timeout. Socket timeouts can also be used with client applications. In this case, the application must be ready to both receive data from the server and to send data, which the local user entered. Since the client only manages one socket, the use of timeouts to facilitate the multi-tasking is a good solution. Since the objective of this activity was to develop a multi-threaded chat server, the approach of using timeouts was also adopted for the initial implementation of the server. I decided that a pretty clean approach would be to hold a global message queue that all threads can write to and read from per the synchronization mechanisms of the readers and writers problem. So grabbing an operating systems text book, [nutt] which I knew had a solution to readers and writers, I had a synchronization solution. When I first opened an operating systems text book and saw all of the locks, (actually, they used semaphores, but in this case, those semaphores can be switched out with Python’s simple Lock object) I was a little overwhelmed for a minute and wondered if I really needed so many locks. Then I remembered that readers and writers has been worked on by lots of really smart people and that it has proven itself to be safe and free of deadlocks. 9.4.3. Combined Asynchronous and Threaded Servers¶ Another approach is to combine synchronous and threaded technologies. This approach may be slightly more complex, but in the case of a multi-party chat server, may offer the best combination of performance and flexibility for adding additional features. In the chat server, activity is initiated by receiving a message from one of the clients. So once a message has been received, we can go about updating any data and sending the message to each client socket. The asynchronous select.select() function can do a nice job of monitoring all of the sockets for incoming data. While a thread for each client makes sense so that each client can have it’s own data, fairly simple code that only worries about one client and is not restricted from using blocking system calls – to read or write to the message queue. The author gives a suggestion for developing the multi-threaded chat server. ... You’ll want to make sure that the client threads do the transmission, not the main thread. One way to accomplish that is to have a semaphore and a queue for each client thread. You could maintain a list of these objects, adding and removing entries from the list as threads come and go. When data is to be sent, it’s added to the appropriate queues and the semaphore is signaled. This suggestion is right on target with an excellent plan of attack. [2] Unfortunately, he never mentions using a combined asynchronous / threaded approach. When I first read his suggestion, it did not make sense to me at all because I was thinking of a pure threaded solution. Thus, his suggestion was not followed in my first implementation. But a combined solution is sure to come in a future revision of the server. [3] The next section (Programming Assignment 5 – Add BRB feature to Multi-threaded Chat Server) describes the chat server programming project. Footnotes [1] We are fairly tolerent of small delays caused by waiting on a blocking system call, or a socket timeout, when it only affects one client. (We can always blame delays on the slow network, right?) But any delay that affects all of the clients, is unacceptable because when the server has a large number of client connections, those short delays can add-up to long delays. [2] The Queue.Queue class from the standard library could be used to keep the client threads on task and remove the need to signal them with a semaphore. The asynchronous thread could use the queue to alert a waiting thread of a message received on its socket. Perhaps another thread could be used just to tell the children that they should check the message board for messages. This thread could be signaled with a threading.Condition (monitor) whenever a writer exits the message queue. Because the message queue can block, only the children thread should ever act as reader or writer to the message board queue. [3] Although the purely threaded solution with timeouts may use a few more CPU cycles by checking for messages when none are there, it still seems to be a reasonable solution. On a heavily loaded server, it might read multiple messages from the queue at a time; whereas, the hybrid solution would tend to alert each child after each message, thus the hybrid solution might loose some of its performance advantage when the server is most loaded. – Hmm, this will merit some testing. Table Of Contents 9.4. A Multi-threaded Chat Server 9.4.1. Asynchronous Servers 9.4.2. Multi-threaded Servers 9.4.3. Combined Asynchronous and Threaded Servers Previous topic 9.3. Asynchronous Network Server Programming Next topic 9.5. Programming Assignment 5 – Add BRB feature to Multi-threaded Chat Server This Page Show Source Quick search Navigation index modules | next | previous | Network Programming Study Guide » 9. Topic 6 - Developing Network Server Programs » © Copyright 2009, Tim Bower. Created using Sphinx 1.4.4.
