One bidirectional tcp connector OR two unidirectional? (linux, high volume, low latency)
I need to periodically send (exchange) a large amount of data with the lowest possible latency between the two machines. The network is pretty fast (like 1Gbps or even 2G +). Os - linux. Is it faster using 1 tcp socket (for send and recv) or using 2 unidirectional tcp sockets?
The test for this problem is very similar to the NetPIPE network test - it measures latency and throughput for sizes from 2 ^ 1 to 2 ^ 13 bytes, each size is sent and received at least 3 times (in the teal problem, the number of sends is greater. Both processes will send and receive e.g. ping pong).
The advantage of two unidirectional connections comes from linux:
http://lxr.linux.no/linux+v2.6.18/net/ipv4/tcp_input.c#L3847
3847/*
3848 * TCP receive function for the ESTABLISHED state.
3849 *
3850 * It is split into a fast path and a slow path. The fast path is
3851 * disabled when:
...
3859 * - Data is sent in both directions. Fast path only supports pure senders
3860 * or pure receivers (this means either the sequence number or the ack
3861 * value must stay constant)
...
3863 *
3864 * When these conditions are not satisfied it drops into a standard
3865 * receive procedure patterned after RFC793 to handle all cases.
3866 * The first three cases are guaranteed by proper pred_flags setting,
3867 * the rest is checked inline. Fast processing is turned on in
3868 * tcp_data_queue when everything is OK.
All other conditions for disabling the fast track are false. And only a non-unidirected socket stops the kernel from fastpath on receiving
a source to share
There are too many variables for one answer, which always runs here. If you have very fast network connectivity - perhaps 1 GBit / sec on modern hardware - the fastpath / slowpath stuff you are linked to probably doesn't matter.
Just in case, you can write your program to work anyway. Just keep the readsocket and writeocket and during connect () you can assign them to the same socket or two different sockets. Then you can just try both ways and see which is faster.
You probably won't notice any difference between the two.
a source to share
I know this doesn't directly answer your question, but I would suggest looking at something like ZeroMQ . It has an introductory article: 0MQ: A New Approach to Messaging .
I haven't tried it yet, but I've read it a bit and it looks like this might be what you are looking for. Why reinvent the wheel?
a source to share