The socket module in Python: More than a footgun
Previously: Beginner Problems With TCP & The socket Module in Python, Things To Think About Before You Implement Online Multiplayer
I think I have previously undersold how terrible the socket module in Python is. It's worse. Or rather, what's worse is that often enough, the socket module is all that is available with the "batteries included" with Python 3.X. For anything more, you need to look on PyPI, and there are far too many modules on there that sort of do what you want, but are unmaintained. That is not good for networking, especially on the Internet!
The socket module has a lot of functionality you don't need for Internet applications, and is missing a lot of functionality you do need. Some of that functionality is present with asyncio, but not with socket. This weird feature imparity leads to beginner programmers using asyncio without understanding the use case for asyncio, or without understanding what async does, mixing blocking functions with asyncio.
Even experienced programmers won't be happy with socket. They can see and sidestep some of the pitfalls, but they can't really make productive use of socket on its own. There is no pure-Python way to get your own IP address, network interfaces, or your public IP address. There is no interface for path MTU discovery, and no abstraction layer for handling mixed IPv6/IPv4 connections.
The socket module is not just a footgun. It's little helper gnomes opening your gun safe at light, reloading the footguns you thought safe while you sleep. Even an experienced programmer must treat the socket module as if it can go off at any time!
Solution: A modern Internet-oriented networking module
We need a high-level Internet-oriented networking module as part of Python. Let's call it "networking". It doesn't need any OS-specific networking features, and it doesn't need IPC. It should just have an easy way to opening a socket and communicating over the Internet.
You should be able to query things like IPv4/IPv6 support, and stuff like mobile IP or multi-path TCP, but by default, there should just be a simple interface that takes a DNS name, IPv4 address or IPv6 address, and lets you connect.
There should probably be a "do-what-i-mean multicast" option for UDP, with functions to send and receive broadcast and multicast packets, instead of re-implementing the wrangling of IP addresses and subnet masks every time.
There should probably be a way to extend this, by implementing different message-oriented, datagram-oriented, or stream-oriented protocols, such as SSL and ENet as separate modules.
Most importantly, this module would have its equivalent of Java's BufferedReader. I know that this is super easy to implement, but the lack of a BufferedReader is a major stumbling block for beginners, and it pushes those beginners to use asyncio without understanding what that is or how it works. The "networking" module could have async versions of everything in a separate networking.async namespace. I don't think having awaitable and blocking operations available on the same socket/connection object is sensible, it would be another footgun, so this API duplication seems to me to be the safer option.
We really need full blocking/async parity. I can't stress this enough. Otherwise you end up in bizarre situations where library users write async code inside sync functions and spin up the event loop for every function call, but then call blocking functions in their async code, because to them, async is just a cumbersome way to call blocking functions.
Solution: Interface/Network/Service Discovery
We need a network discovery module as part of Python, maintained and developed in tandem with the old socket and the new networking. It should allow the user/developer to easily enumerate all available network interfaces, their IP addresses, MTU, IPv4/IPv6 status, connectivity, whether they are metered, and whether their networks are connected to the Internet.
Path MTU discovery and ping might also be useful.
Maybe service discovery protocols like Bonjour could live in this module too, or they could be their own module.
Solution: Dealing with NAT
In an ideal world, we would all be using IPv6, and we would somehow know the worldwide unique IP address of our friend's PC - the device we are trying to communicate with. Our router would know to route packets to that IP to our friend's router, and all would be well.
In the real world we are using NAT and VPNs. Some devices have an IPv4 address, others don't. IPv6 usually doesn't help us connect to a friend's PC to play a game.
We need a module (I propose the name "unfirewall" or "traversal") to allow the user/developer to connect or send packets to a computer behind NAT. This module must set up a connection that is either stream-based or datagram-based, and present the same API as raw sockets created with the networking module, or SSL sockets built on top of it.
Whatever system you are using to connect your stream or datagram socket, whether you go through a SOCKS proxy or a TURN server, or you manage to do NAT hole punching via STUN, it should return a networking object that is mostly indistinguishable from one that was connected through your local LAN.
Solution: Common Data Types
Python needs data types for networking addresses, such as IPv4 addresses, IPv6 addresses, DNS names, IDNA, and so on. The above mentioned modules should accept parameters with those types, in addition to old-fashioned 32-bit unsigned integers for IPv4 addresses.
Solution: Network Reliability Simulation
Once we have an ecosystem of modern networking APIs, it becomes easier to write your own network un-reliability simulator. Dropped connections, uneven transmission speeds, partial reads and writes leading to TCP stream read() not lining up with send() from the other end, dropped UDP packets, a whole second of latency, low bandwidth, all those could be simulated on localhost.
Those changes would turn Python networking from a footgun for beginners into something that still causes cursing and frozen GUI widgets when your WLAN connection gets choppy. There is no software fix for radio signal quality, or for an excavator cutting the fiber optic connection between your town and the rest of the world.
