network engine

[patrick]

Of course it is too much to create a finished well working network implementation within one semester. There is a lot of things that are needed by any network implementation, things that are very handy and important, things that may help and things that are just nice to have.
Find out which of your problems you defined correspond to which category (compression, diff images, motion extrapolation, scalable network topology, etc.). And also determine which things that you may implement later will affect modules you created earlier.

Therefore I really support the suggestion from Beni: create a basic framework that will support the most important features you want. You don't need to implement all these features now but your framework needs to be adaptable and extendable to support it in the future.

And don't think about the network delay too much. If you create a nice extendable framework you can always care bout delays later. You can't do any more magic than to follow the tips that you are given by the papers I posted earlier. The bottleneck anyways will be at a place where you don't expect it to be

I like the questions you ask and the problems you confront the community with because they show, that you are thinking about the right things. If you get stuck, never hesitate to ask us, because networking really is no trivial task.

[greenman]

Well, after a lot of talking today, we decided to start without caring about delay. Dumeni will now take care of the PacketEncoder and -Decoder and I will take care of the (low level) network talk. I will implement a class which will use either several processes or several threads (one for each client) to handle incoming connections. In order to get the communication between processes and threads working I read a page about IPC (good overview): IPC

What would you say is more suited to use:
a) multiple processes using one of the above IPC-methods
b) multiple threads using "don't know what yet" to communicate

[Dude]

I thought about that too. We should consider creating just one thread that listens to all clients, Someone told me, that's possible with UDP..
An advantage of using threads would be, that threads use the same memory whereas each process has its own, what would solve the problem of how two processes can communicate.
Another argument for threads is that communication between processes might be slow.

[silvan]

I would definitely use several threads.
The main reason is that Data exchange between parallel processes generates more overhead and is harder to implement (IPC). I don't have any experience with multi processing but I imagine that it is more difficult to implement a cross platform multi process application than a multi threaded one.

There are several good cross platform thread libraries out there on the net. For example boost.thread boost.thread provides solid and very basic thread support.

For inter-thread communication you could use something like this. It is basically a wrapper around std::vector which provides mutual exclusion. This is called monitor pattern hence the filename monitor.cpp . Of course you should ommit the "do_sender" and "do_receiver" methods in your implementation

[beni]

Great! Thanks for your input, Silvan. I guess with this we can code using multi threading easily. I just hope that we don't get too many debugging problems.

[greenman]

Well great, thank you
I think that is exactly what we need

As soon as I understand this code entirely I will adapt it to our needs

[greenman]

At the moment I'm implementing a class managing the clients (connection) by using multiple threads. First part is now a packetbuffer which is multithreadable. Because I had problems with the templates of the stl-queue I implemented my own (very basic) fifo-class which is locked if someone writes to or reads from it.
Because I don't know yet which datatypes dumeni takes for the packets this buffer only works for int's so far but adjusting this shouldn't be a problem at all...

edit: BTW, does anyone know wheter usleep also works on other OSes or if there's a wrapper for this OS-specific function?

[beni]

usleep() is sleep() in Windows and I think in Mac as well..

What about a cross platform thread library?

[greenman]

Hm... yes well have to take care about that too...

Do we have some log/debug-handling ??

[beni]

No, not at all at the moment. I'm really sorry.

[x3n]

which is locked if someone writes to or reads from it.

Uh, oh, be very careful with such statements. Unless you used semaphores (or something comparable) I'm almost sure it isn't thread-safe. There are possible implementations without semaphores, but they only work for a known number of threads (usually 2).

Look at "Versuch 1-6" (pages 5-10) in this pdf:
http://svn.orxonox.net/webdev/pps/hs07/ ... anager.pdf

[greenman]

Why shouldn't it be thread-safe ?


readfunction:
if(!locked){
locked=true;
read.....
locked=false;
} else return false;

writefunction:
if(!locked){
locked=true;
write....
locked=false;
} else return false;


Both threads use the same object and locked is a variable of this object which should be the same for both thread, or am I wrong?

[x3n]

problem:

Code: Select all

locked = false;

thread1 starts, locked is false:

Code: Select all

readfunction: 
if(!locked){

switch to thread 2, locked is still false:

Code: Select all

writefunction: 
if(!locked){

and now are both in the critical section.


(that was, by the way, "versuch 1" in the pdf )

[silvan]

To implement such a thread safe queue you should really use locks and condition variables provided by a threading library like boost or the OS directly if you want to do it the hard way ( portability!), otherwise it is almost impossible to do the checking and locking of the critical section in an atomic way so that it works for more than 2 threads.

For some background on how to implement a thread safe queue using the lock and condition variable concept have a look at this paper:
http://portal.acm.org/citation.cfm?id=3 ... N=69312656

It should be accessible from within the eth (or via vpn). In chapter 3.4 "Monitors" you will find an example in pseudo code. Looking at it, you will find that it is very close to the exampe I posted previously.
Good luck!

[beni]

Found another multi-threading, platform independent library called zthread. Check it out.