Helpful Books to Understand the Architecture of Snort
Every now and then the Snort mailing lists get “homework questions” like how to learn about the program’s architecture. IMHO the most helpful books for a more detailed insight are:
Because after all Snort is open source and a reasonably small application, so Use the source. Compile everything, record a small tcpdump, and start gdb. Just be aware of modularity and do not get lost in config parsing, the content rule optimizations, or in one of the heavy-weight preprocessors (like frag and stream).
Once you have reached the ProcessPacket() and Preprocess() functions then you should see how all other subsystems (decoding, calling preprocessors, rule evaluation, output) are called from there for every packet.
My next best book advice (on an abstraction layer somewhere between the source and the Snort Manual) is Snort: IDS and IPS Toolkit from Syngress (they used to publish a new Snort book every few years, but the 2007 edition seems to be the last one). – For me this was a good guidebook to find things in the source code. The inherent problem with books is that they become outdated as the software develops.
Finally use Google Scholar and read. [shameless plug] My own thesis has a dozen pages on Snort architecture overview (obviously with emphasis on plugins). Since Snort is quite popular there are many similar theses and papers, each one with a slightly different focus depending on whether they write and describe new decoders, preprocessors, or output modules.