I think OpenSSL needs a documentation project. My first week of GSoC coding was dedicated to transport-tls, so I started with establishing a TLS connection and accessing different parts of the X.509 certificates to check them. I would have thought these are basic tasks for every TLS-enabled application and yet I found this unexpectedly difficult.
When I came to work on Syslog one of the most disturbing texts I came across was Rainer’s observation “On the (un)reliability of plain tcp syslog…“. The problem is that a sendmsg() system call is nearly always successful — it only indicates local errors (like a full send queue), but no network errors. So even after the other side initiated a connection shutdown one can happily write into the local buffer and only get an error on the second write.
The structured data (SD) in syslog-protocol offers some interesting possibilities when it comes to modifying a message in transit. What would be necessary to enable rewriting in transit while still be able to authenticate by syslog-sign?
Today the participants in Google’s Summer of Code 2008 were announced. – And my project was chosen. :-)
So now I will work for NetBSD and implement the new IETF syslog protocols.
Skimming through last year’s Informatik Spektrum I found an interesting master thesis by Steffen Tambach (german) examining the reliability of UDP transport. The analysis showed that no data was lost on the Fast Ethernet, but all lost packets were dropped by the receiver sowewhere on the network stack between NIC and socket API.
The following recommendations are given to prevent UDP package loss in a local network (besides the obvious ‘have enough bandwidth for your data’ and ‘be able to store/process it fast enough’):
This is a little known technological fact about the Internet:
the Internet is actually made up of words and enthusiasm.
One of the most important issues in a logging system (i.e. syslog) is reliability. But sometimes I get the impression most people asking for reliability do not really want it. IMO a reliability requirement has to be tested against the following extreme case: suppose the logging system fails and you enter a command — should the command be executed although it cannot be logged? If the answer is “Yes” then you do not really want reliable logging (at least for the command you thought about).
I admit that this constructs a dichotomy that might not be necessary. But so far I have not heard of some “weak” or “semi-reliability”, thus the binary distinction only reflects the state of discourse on the subject. I also admit that most related real-life problems were solved if the syslog-world would eventually throw out UDP in favor of TCP or TLS. So this whole reasoning is not about pressing needs but rather abstract and mainly a response to people claiming “just implement a rate-limit, then everything is solved”.