network engine

Introduce your project and write about your work.

Moderator: PPS-Leaders

User avatar
patrick
Baron Vladimir Harkonnen
Posts: 350
Joined: Mon Oct 02, 2006 6:03 pm
Location: Bern

Post by patrick » Tue Oct 30, 2007 11:06 pm

Of course it is too much to create a finished well working network implementation within one semester. There is a lot of things that are needed by any network implementation, things that are very handy and important, things that may help and things that are just nice to have.
Find out which of your problems you defined correspond to which category (compression, diff images, motion extrapolation, scalable network topology, etc.). And also determine which things that you may implement later will affect modules you created earlier.

Therefore I really support the suggestion from Beni: create a basic framework that will support the most important features you want. You don't need to implement all these features now but your framework needs to be adaptable and extendable to support it in the future.

And don't think about the network delay too much. If you create a nice extendable framework you can always care bout delays later. You can't do any more magic than to follow the tips that you are given by the papers I posted earlier. The bottleneck anyways will be at a place where you don't expect it to be :D

I like the questions you ask and the problems you confront the community with because they show, that you are thinking about the right things. If you get stuck, never hesitate to ask us, because networking really is no trivial task.

User avatar
greenman
Baron Vladimir Harkonnen
Posts: 360
Joined: Wed Oct 03, 2007 2:53 pm
Contact:

Post by greenman » Wed Oct 31, 2007 8:55 pm

Well, after a lot of talking today, we decided to start without caring about delay. Dumeni will now take care of the PacketEncoder and -Decoder and I will take care of the (low level) network talk. I will implement a class which will use either several processes or several threads (one for each client) to handle incoming connections. In order to get the communication between processes and threads working I read a page about IPC (good overview): IPC

What would you say is more suited to use:
a) multiple processes using one of the above IPC-methods
b) multiple threads using "don't know what yet" to communicate

Dude
Human Space Navy Private
Posts: 2
Joined: Mon Oct 08, 2007 1:32 pm

Post by Dude » Thu Nov 01, 2007 7:05 pm

I thought about that too. We should consider creating just one thread that listens to all clients, Someone told me, that's possible with UDP..
An advantage of using threads would be, that threads use the same memory whereas each process has its own, what would solve the problem of how two processes can communicate.
Another argument for threads is that communication between processes might be slow.

silvan
Noxonian Brolghormeg
Posts: 37
Joined: Wed Oct 11, 2006 7:43 am

Post by silvan » Thu Nov 01, 2007 7:19 pm

I would definitely use several threads.
The main reason is that Data exchange between parallel processes generates more overhead and is harder to implement (IPC). I don't have any experience with multi processing but I imagine that it is more difficult to implement a cross platform multi process application than a multi threaded one.

There are several good cross platform thread libraries out there on the net. For example boost.thread boost.thread provides solid and very basic thread support.

For inter-thread communication you could use something like this. It is basically a wrapper around std::vector which provides mutual exclusion. This is called monitor pattern hence the filename monitor.cpp . Of course you should ommit the "do_sender" and "do_receiver" methods in your implementation :-)
Last edited by silvan on Thu Nov 01, 2007 7:26 pm, edited 1 time in total.
The light on the end of the tunnel is a train.

User avatar
beni
Baron Vladimir Harkonnen
Posts: 949
Joined: Tue Oct 03, 2006 9:15 am
Location: Zurich
Contact:

Post by beni » Thu Nov 01, 2007 7:25 pm

Great! Thanks for your input, Silvan. I guess with this we can code using multi threading easily. I just hope that we don't get too many debugging problems.
"I'm Commander Shepard and this is my favorite forum on the internet."

User avatar
greenman
Baron Vladimir Harkonnen
Posts: 360
Joined: Wed Oct 03, 2007 2:53 pm
Contact:

Post by greenman » Fri Nov 02, 2007 10:40 pm

Well great, thank you :beerchug:
I think that is exactly what we need

As soon as I understand this code entirely I will adapt it to our needs ;)

User avatar
greenman
Baron Vladimir Harkonnen
Posts: 360
Joined: Wed Oct 03, 2007 2:53 pm
Contact:

Post by greenman » Mon Nov 05, 2007 12:25 pm

At the moment I'm implementing a class managing the clients (connection) by using multiple threads. First part is now a packetbuffer which is multithreadable. Because I had problems with the templates of the stl-queue I implemented my own (very basic) fifo-class which is locked if someone writes to or reads from it.
Because I don't know yet which datatypes dumeni takes for the packets this buffer only works for int's so far ;) but adjusting this shouldn't be a problem at all...

edit: BTW, does anyone know wheter usleep also works on other OSes or if there's a wrapper for this OS-specific function?

User avatar
beni
Baron Vladimir Harkonnen
Posts: 949
Joined: Tue Oct 03, 2006 9:15 am
Location: Zurich
Contact:

Post by beni » Mon Nov 05, 2007 3:32 pm

usleep() is sleep() in Windows and I think in Mac as well..

What about a cross platform thread library?
"I'm Commander Shepard and this is my favorite forum on the internet."

User avatar
greenman
Baron Vladimir Harkonnen
Posts: 360
Joined: Wed Oct 03, 2007 2:53 pm
Contact:

Post by greenman » Mon Nov 05, 2007 4:01 pm

Hm... yes well have to take care about that too...

Do we have some log/debug-handling ??

User avatar
beni
Baron Vladimir Harkonnen
Posts: 949
Joined: Tue Oct 03, 2006 9:15 am
Location: Zurich
Contact:

Post by beni » Mon Nov 05, 2007 4:04 pm

No, not at all at the moment. I'm really sorry.
"I'm Commander Shepard and this is my favorite forum on the internet."

User avatar
x3n
Baron Vladimir Harkonnen
Posts: 810
Joined: Mon Oct 30, 2006 5:40 pm
Contact:

Post by x3n » Mon Nov 05, 2007 4:39 pm

greenman wrote:which is locked if someone writes to or reads from it.
Uh, oh, be very careful with such statements. Unless you used semaphores (or something comparable) I'm almost sure it isn't thread-safe. There are possible implementations without semaphores, but they only work for a known number of threads (usually 2).

Look at "Versuch 1-6" (pages 5-10) in this pdf:
http://svn.orxonox.net/webdev/pps/hs07/ ... anager.pdf

User avatar
greenman
Baron Vladimir Harkonnen
Posts: 360
Joined: Wed Oct 03, 2007 2:53 pm
Contact:

Post by greenman » Mon Nov 05, 2007 8:25 pm

Why shouldn't it be thread-safe ?


readfunction:
if(!locked){
locked=true;
read.....
locked=false;
} else return false;

writefunction:
if(!locked){
locked=true;
write....
locked=false;
} else return false;


Both threads use the same object and locked is a variable of this object which should be the same for both thread, or am I wrong?

User avatar
x3n
Baron Vladimir Harkonnen
Posts: 810
Joined: Mon Oct 30, 2006 5:40 pm
Contact:

Post by x3n » Mon Nov 05, 2007 8:43 pm

problem:

Code: Select all

locked = false;
thread1 starts, locked is false:

Code: Select all

readfunction: 
if(!locked){
switch to thread 2, locked is still false:

Code: Select all

writefunction: 
if(!locked){
and now are both in the critical section.


(that was, by the way, "versuch 1" in the pdf ;))

silvan
Noxonian Brolghormeg
Posts: 37
Joined: Wed Oct 11, 2006 7:43 am

Post by silvan » Tue Nov 06, 2007 8:09 am

To implement such a thread safe queue you should really use locks and condition variables provided by a threading library like boost or the OS directly if you want to do it the hard way ( portability!), otherwise it is almost impossible to do the checking and locking of the critical section in an atomic way so that it works for more than 2 threads.

For some background on how to implement a thread safe queue using the lock and condition variable concept have a look at this paper:
http://portal.acm.org/citation.cfm?id=3 ... N=69312656

It should be accessible from within the eth (or via vpn). In chapter 3.4 "Monitors" you will find an example in pseudo code. Looking at it, you will find that it is very close to the exampe I posted previously.
Good luck!
The light on the end of the tunnel is a train.

User avatar
beni
Baron Vladimir Harkonnen
Posts: 949
Joined: Tue Oct 03, 2006 9:15 am
Location: Zurich
Contact:

Post by beni » Tue Nov 06, 2007 12:44 pm

Found another multi-threading, platform independent library called zthread. Check it out.
"I'm Commander Shepard and this is my favorite forum on the internet."

Post Reply

Who is online

Users browsing this forum: No registered users and 17 guests