pythonnetexamples Python Socket Programming Examples Let’s do socket-level programming in Python. Unit Goals To gain proficiency in writing client-server applications in Python at the socket level. Overview We will look at four network applications, all running over TCP using sockets, written completely from scratch in Python, presented in order of increasing complexity: A trivial date server and client, illustrating simple one-way communication. The server sends data to the client only. A capitalize server and client, illustrating two-way communication, and server-side threads to more efficiently handle multiple connections simultaneously. A two-player tic tac toe game, illustrating a server that needs to keep track of the state of a game, and inform each client of it, so they can each update their own displays. A multi-user chat application, in which a server must broadcast messages to all of its clients. A Trivial Sequential Server This is perhaps the simplest possible server. It listens on port 59090. When a client connects, the server sends the current datetime to the client. The with statement ensures that the connection is automatically closed at the end of the block. After closing the connection, the server goes back to waiting for the next client. date_server.py # A simple TCP server. When a client connects, it sends the client the current
# datetime, then closes the connection. This is arguably the simplest server
# you can write. Beware though that a client has to be completely served its
# date before the server will be able to handle another client.
import socketserver
from datetime import datetime
class DateHandler(socketserver.StreamRequestHandler):
def handle(self):
self.wfile.write(f'{datetime.now().isoformat()}\n'.encode('utf-8'))
with socketserver.TCPServer(('', 59090), DateHandler) as server:
print('The date server is running...')
server.serve_forever()
Discussion: This code is just for illustration; you are unlikely to ever write anything so simple. This does not handle multiple clients well; each client must wait until the previous client is completely served before it even gets accepted. As in virtually all socket programs, a server socket just listens, and a different, “plain” socket communicates with the client. However, Python has powerful abstractions that actually hide the actual sockets! The TCPServer object abstracts the server socket: You pass the server a Handler class, in our case, a subclass StreamRequestHandler. You have to override handle; you can also override setup and finish (but call the base class versions if you do). The instance fields of all handlers are server, client_address, and request. For StreamRequestHandlers, you get instance variables rfile and wfile. When the handle method ends, wfile will be automatically flushed. You cannot see any explicit blocking calls. The abstractions arrange for a new handler object to be created when a client connects. WOW. (So you can hold state in a handler.) Socket communication is always with bytes, so if you want characters you will do a lot of explicit encoding and decoding. After sending the datetime to the client, the handle method ends and the communication socket is closed, so in this case, closing the connection is initiated by the server.
$ python3 date_server.py
The date server is running...
A quick test with nc:
$ nc localhost 59090
2019-02-17T22:26:21.629324
Here’s how to do the client in Python. It connects, prints the datetime it gets from the server, then exits. date_client.py # A command line client for the date server.
import sys
import socket
if len(sys.argv) != 2:
print('Pass the server IP as the sole command line argument')
else:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.connect((sys.argv[1], 59090))
print(sock.recv(1024).decode('utf-8'))
Discussion: AF_INET means the networking layer underneath is IPv4. Other possible values are AF_UNIX, AF_INET6, AF_BLUETOOTH, and others. SOCK_STREAM is for TCP; there is also SOCK_DGRAM for UDP among others. To get file-like reading and writing from a socket, we could use socket.makefile; however, for now it’s fine to use the lower-level socket calls. We know the date will fit in 1024 bytes, so recv will be fine. (We’ll do the file thing later.)
$ python3 date_client.py localhost
2019-02-17T22:28:14.988791
A Simple Threaded Server The previous example was pretty trivial: it did not read any data from the client, and worse, it served only one client at a time. This next server receives lines of text from a client and sends back the lines uppercased. It efficiently handles multiple clients at once: When a client connects, the server spawns a thread, dedicated to just that client, to read, uppercase, and reply. The server can listen for and serve other clients at the same time, so we have true concurrency. We make use of Python’s ThreadingMixIn to do this thread spawning automatically: capitalize_server.py # A server program which accepts requests from clients to capitalize strings. When
# clients connect, a new thread is started to handle a client. The receiving of the
# client data, the capitalizing, and the sending back of the data is handled on the
# worker thread, allowing much greater throughput because more clients can be handled
# concurrently.
import socketserver
import threading
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
daemon_threads = True
allow_reuse_address = True
class CapitalizeHandler(socketserver.StreamRequestHandler):
def handle(self):
client = f'{self.client_address} on {threading.currentThread().getName()}'
print(f'Connected: {client}')
while True:
data = self.rfile.readline()
if not data:
break
self.wfile.write(data.decode('utf-8').upper().encode('utf-8'))
print(f'Closed: {client}')
with ThreadedTCPServer(('', 59898), CapitalizeHandler) as server:
print(f'The capitalization server is running...')
server.serve_forever()
Discussion: The server we are using here is a class extending both TCPServer and ThreadingMixIn. This means that every time a client connects, the handler for it will run on a new thread. We’ve echoed the client address+port and the thread on which the client runs, both when the client connects and also when it goes away The handler repeatedly reads lines of text from the client until there is no more data from the client. Each line from the client is capitalized and then written back to the client. Note that the StreamRequestHandler’s wfile attribute is an unbuffered stream! Therefore, no flush is necessary after writing the capitalized string. The way we know there is no more data is when rfile.readLine() returns the empty string. Note that if the client sends a blank line, it will arrive at the server as a one-character string containing a newline. Only when the client closed its end will readline return the empty string (with no new line). At this point, the loop exits and the handle method returns, closing the server-side socket. We can test with nc. Since the handler repeatedly reads lines from the client until the end of standard input, you need to terminate your nc session with Ctrl+D or Ctrl+C:
$ nc localhost 59898
you’re not me!
YOU’RE NOT ME!
Say fellas, did somebody mention the Door to Darkness?
SAY FELLAS, DID SOMEBODY MENTION THE DOOR TO DARKNESS?
e d g e l o r d
E D G E L O R D
Of course, we can write our own simple command line client: capitalize_client.py import sys
import socket
if len(sys.argv) != 2:
print('Pass the server IP as the sole command line argument')
else:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.connect((sys.argv[1], 59898))
print('Enter lines of text then Ctrl+D or Ctrl+C to quit')
while True:
line = sys.stdin.readline()
if not line:
# End of standard input, exit this entire script
break
sock.sendall(f'{line}'.encode('utf-8'))
while True:
data = sock.recv(128)
print(data.decode("utf-8"), end='')
if len(data) < 128:
# No more of this message, go back to waiting for next message
break
Discussion: I’m not terribly happy about the while True loops but they will do for now. The outer loop says keep going until standard input has no more data. We know there is no more data when readLine() returns the empty string. Blank lines from standard input are read in as strings with a single newline character in them. The inner loop is there because we are using the low-level recv method, which requires us to specify a buffer size. If the server message is larger than that size, we have to repeatedly read. We know we have reached the end of the message when the buffer is not completely full. The client can be used interactively, or we can redirect standard input:
$ python3 capitalize_client.py localhost < hello.py
Enter lines of text then Ctrl+D or Ctrl+C to quit
PRINT('HELLO, WORLD')
Classwork Experimentation time. Replace the loop in the client that keeps reading from the receive buffer until we get an empty or partially full buffer with code that just reads a fixed amount of bytes. Arrange for the server to send too much data. How the client behave in this case? A Network Tic-Tac-Toe Game Here is the server for a two-player game. It listens for two clients to connect, and spawns a thread for each: the first is Player X and the second is Player O. The client and server send simple string messages back and forth to each other; messages correspond to the Tic Tac Toe protocol, which I made up for this example. tic_tac_toe_server.py # A server for a multi-player tic tac toe game. Loosely based on an example in
# Deitel and Deitel’s “Java How to Program” book. For this project I created a
# new application-level protocol called TTTP (for Tic Tac Toe Protocol), which
# is entirely plain text. The messages of TTTP are:
#
# Client -> Server
# MOVE
# QUIT
#
# Server -> Client
# WELCOME
# VALID_MOVE
# OTHER_PLAYER_MOVED
# OTHER_PLAYER_LEFT
# VICTORY
# DEFEAT
# TIE
# MESSAGE
import socketserver
import threading
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
daemon_threads = True
allow_reuse_address = True
class PlayerHandler(socketserver.StreamRequestHandler):
def handle(self):
self.opponent = None
client = f'{self.client_address} on {threading.currentThread().getName()}'
print(f'Connected: {client}')
try:
self.initialize()
self.process_commands()
except Exception as e:
print(e)
finally:
try:
self.opponent.send('OTHER_PLAYER_LEFT')
except:
# Hack for when the game ends, not happy about this
pass
print(f'Closed: {client}')
def send(self, message):
self.wfile.write(f'{message}\n'.encode('utf-8'))
def initialize(self):
Game.join(self)
self.send('WELCOME ' + self.mark)
if self.mark == 'X':
self.game.current_player = self
self.send('MESSAGE Waiting for opponent to connect')
else:
self.opponent = self.game.current_player
self.opponent.opponent = self
self.opponent.send('MESSAGE Your move')
def process_commands(self):
while True:
command = self.rfile.readline()
if not command:
break
command = command.decode('utf-8')
if command.startswith('QUIT'):
return
elif command.startswith('MOVE'):
self.process_move_command(int(command[5:]))
def process_move_command(self, location):
try:
self.game.move(location, self)
self.send('VALID_MOVE')
self.opponent.send(f'OPPONENT_MOVED {location}')
if self.game.has_winner():
self.send('VICTORY')
self.opponent.send('DEFEAT')
elif self.game.board_filled_up():
self.send('TIE')
self.opponent.send('TIE')
except Exception as e:
self.send('MESSAGE ' + str(e))
class Game:
next_game = None
game_selection_lock = threading.Lock()
def __init__(self):
self.board = [None] * 9
self.current_player = None
self.lock = threading.Lock()
def has_winner(self):
b = self.board
return ((b[0] is not None and b[0] == b[1] and b[0] == b[2])
or (b[3] is not None and b[3] == b[4] and b[3] == b[5])
or (b[6] is not None and b[6] == b[7] and b[6] == b[8])
or (b[0] is not None and b[0] == b[3] and b[0] == b[6])
or (b[1] is not None and b[1] == b[4] and b[1] == b[7])
or (b[2] is not None and b[2] == b[5] and b[2] == b[8])
or (b[0] is not None and b[0] == b[4] and b[0] == b[8])
or (b[2] is not None and b[2] == b[4] and b[2] == b[6]))
def board_filled_up(self):
return all(cell is not None for cell in self.board)
def move(self, location, player):
with self.lock:
if player != self.current_player:
raise ValueError('Not your turn')
elif player.opponent is None:
raise ValueError('You don’t have an opponent yet')
elif self.board[location] is not None:
raise ValueError('Cell already occupied')
self.board[location] = self.current_player
self.current_player = self.current_player.opponent
@classmethod
def join(cls, player):
with cls.game_selection_lock:
if cls.next_game is None:
cls.next_game = Game()
player.game = cls.next_game
player.mark = 'X'
else:
player.mark = 'O'
player.game = cls.next_game
cls.next_game = None
with ThreadedTCPServer(('', 58901), PlayerHandler) as server:
print(f'The Tic Tac Toe server is running...')
server.serve_forever()
We can skip the client. We can just use a graphical Java client on this page; after all, it’s cool to say you have a game written in multiple programming languages. If you feel like being really old school, you can play the game using nc. A Multi-User Chat Application Sorry, this is a homework assignment. Summary We’ve covered: A lot of little details about Python networking. One-way communication between client and server Two-way communication between client and server Using threads on the server Keeping track of state in a network-based multiplayer game.
