There're only two protocols - UCI and Winboard. Winboard is an old protocol and not really being used nowadays. Crafty is the only major engine still supporting the Winboard protocol, but it's only because the engine is also very old. UCI is a newer protocol developed by Shredder, and is used everywhere - Windows, Macs, Linux, Android, iOS etc. UCI is really the only protocol you should consider.
- UCI is supported everywhere
- Every chess GUI other than xboard supports UCI, such as, Chessbase, Scid, Arena etc. Even xboard can connect to a UCI engine by something known as Polyglot.
- UCI documentation is at http://wbec-ridderkerk.nl/html/UCIProtocol.html
Winboard has more or less obsoleted because the protocol is more complicated than the cleaner UCI. Winboard is a stated protocol, it means the engine state depends on the previous iterations, whereas UCI is easier to deal with as you only have to supply with your current FEN or a move-list.