Network - the story goes on

[greenman]

Hi all

It's been quite a while since we've updated our status....

We're now (or at least we hope so) quite close to get the network working.
But I'm encountering a really persistent problem with stl::lists:
When I first start a server and then a client which connects to the server, the server sends the client a gamestate. The client tries to process this gamestate but fails accessing a stl-list (syncList in src/network/Synchronisable.cc/h).

After a lot of debugging I found out the following:
- When the Synchronisable gets created and the list gets filled up everything is fine (can access elements, can use syncList.size() )
- When the function updateData() gets called, everythings messed up (syncList._M_start is 0x0) and most calls (except the ones that return an iterator) to the list (i.e. syncList.size()) ends up with a segfault, which is obvious if the start-pointer is NULL
- I also replaced the list by a vector (which is externally quite the same) with exactly the same result -> segfault
- I tried to set some watchpoints to the list in order to see, when the pointers get messed up, but with no success

I already spent a whole lot of time for this bug and I don't know what else I should try.

- A possible explanation for this would be, if something would write to the synchronisable and that way destroy the list

Has someone any idea why this could happen ?

I hope someone can help us here, because I'm really stuck.


Thanks and greets


PS: This is all about the network branch.

[beni]

Has this been solved in r1083?

[greenman]

Sorry for the late answer.

Yes, it has been solved, thanks to x3n and 1337

[greenman]

For those of you who do not read commit messages:
we have made quite a lot of progress in the last maybe 2 weeks:
- first thing was: simple synchronisation of the universe and the spaceship with spaceship movement works
- afterwards I made some more objects as models (i.e. asteroids), NPCs, bullets synchronisable
- then we optimized our memory usage (it's quite good now)
- then we managed to get the diff algorithm working
- then we managed to get the compress function working (~70-80% less transfer volume, ~50% less if we disable diff algorithm)
-> it works quite stable and with server and client on same pc it's not buggy at all

I guess these are quite good news

We will try out server/client on different PCs the next time we meet in ETL

If someone wants to try it out here are the steps required:

- checkout the network3 branch
- cmake and stuff
- configure (media datapath, console output, ...)
- run 2 instances: 1. "./run --mode server" ; 2. "./run --mode client"
-> this will work on 1 pc if you just let the client screen be in the background and move the server screen a bit aside to view the client window
otherwise you can start the client with ./run --mode client --ip [IP-Address] and use a different PC as the client

Have fun!

[x3n]

Wow, cool
Is it already possible to connect multiple clients?

[greenman]

Yes it should be, but i haven't tested it so far
but it'll be just another clone of the 1st client

Maybe that's why I haven't tested it yet ...

[beni]

Great news. I'm eager to test it out and I'm looking forward to the next time we'll meet.

[patrick]

Congratulations, looks like serious work you did there!

[greenman]

after a long time here's the news:

- multiple clients should work fine now
- we found the reason of our memory corruptions: enet is not threadsafe (we caught that now)
-> great enhancement in stability
- some memory improvements (speed and memory size)

next steps will be:

- replace enet to make network part more efficient (utopistic)
- put classes PacketGenerator/Decoder into a single packet class to provide a more objectoriented way
- do the same with gamestates (put structs into one single class)

[beni]

Sounds good, especially what you established after finding the reasons for the stability problems.

You write that there is improvement in the memory size. I guess when writing network code, it would be great to be able to keep track of the throughput. On the presentation you said quite a big number of data size you have to transmit. I don't know how much this improved, but it would be good to know how much it is and how or if we can improve this. Also for comparison some actual numbers would be great.

Besides that: Great work there.

[greenman]

what I meant with memory optimimsation wasn't the traffic, but the ram usage ...

@presentation: yeah I know that statement was a bit shocking ^^

at the moment we trimmed the network frequency to 30 t/s (ticks per second) and have the following resulting traffic:

- static asteroids: 30 * 0.29 = 8.7 kB/s (compressed/diffed from 720kB/s)
- rotating asteroids: 30 * 4 = 120 kB/s (compressed/diffed from the same size as above)
these values are with velocity and rotationrate of all objects (including asteroids) transmitted (client-side interpolation)

without velocity and rotationrate:
- static: 30 * 0.18 = 5.7 kB/s ( from 500kB/s )
- rotating: 30 * 3.8 = 114 kB/s

I think these differences are really impressive. with rotating asteroids we still got 120kB/s against 8.7. so we probably have to think about something to manage these asteroids more efficient ...

what do you think about that ?