Stream data buffering
I am trying to create a simple IRC bot. I first want to come up with the right project for this project. One of the things I'm thinking about right now is the read engine. I am developing this bot on a Linux system. (Fedora 12) To read from a socket, I use the "read ()" system call. I am planning to use the read functions as follows (the code is just an example. Not something from the final product):
while (uBytesRead = read(iServerSocket, caBuffer, MAX_MESSAGE_SIZE))
{
//1. Parse the buffer and place it into a Message structure.
//2. Add the message structure to a linked list that will act as a queue of message that are to be processed.
}
This code will run its own thread in it. I opt for this option because I wanted to have as little delay between reads as possible. (records will be implemented the same way). This is all a bit based on assumptions that I would like to clarify. My question is, what if you are receiving so much data at such a fast rate that reading and processing the data (in this case, just parsing it) is slower than the rate at which the data arrives. I made the assumption that this data is buffered by the system. is this a correct assumption? And if yes:
- How big is this buffer?
- What happens to the incoming data when this buffer fills up?
- To protect the application from spam, how can I deal with it?
I hope I have explained my problem in sufficient detail.
Thanks in advance.
a source to share
IRC uses TCP sockets to connect to the network. Linux / Posix TCP sockets have a data buffer to send and another one to receive. You can change the size of the buffers with setsockopt () and SO_SNDBUF / SO_RCVBUF.
TCP has flow control, so when the receive buffer fills up, the OS sends a congestion notification. Packets received that do not fit in the buffer will not be acknowledged by the receiver and will eventually be transmitted by the sender.
In order not to worry. What matters is what the sender program does when the socket's send buffer fills up. Some programs will close the socket, others will simply discard the written data and try again, while others may linger inside.
a source to share