Page 1 of 2

Diary: Masterserver

Posted: Wed Oct 06, 2010 1:49 pm
by smerkli
I'll post general problems and development notes here while working on the masterserver.
Right now I'm the planning phase, having developed a basic draft for the protocol.

scheusso said I should use TCP for the implementation, and I am now looking at the server code of orxonox itself.
I'm planning to copy large parts of it so I don't have to re-implement anything that's already been done. I'll then add what's needed as I go.

I'm splitting the master server coding project into the following parts:

Communication over network:
- Communication MS - GS
- Communication MS - CS

Storage of data:
- Storage structure
- Modifications to storage structures

Main routine
- Listen for connections and trigger respones
- Update serverlist:
-- Ping connected GS
-- Adjust stored data to the mutations

(MS := Master server, GS := Game server, CC := Game client)


In addition to the master server, the game client and the game server also need some extensions:

Game client:
- Server browser
- Server queries and response parser

Game server:
- Register/Unregister at master server
- Respond to requests and pings
- Send gamestate updates

Re: Diary: Masterserver

Posted: Thu Oct 07, 2010 2:33 pm
by greenman
if you will reuse parts of the orxonox network code it's probably the best to use enet (and thus udp) and send the information you require via a reliable packet (this would be samehow similar to tcp).
you could probably create one/several new packet(s) and use orxonox::packet::Packet as base class
i will probably soon move things like packet compression into this class, so this will provide some features you maybe want to use

Re: Diary: Masterserver

Posted: Thu Oct 07, 2010 5:05 pm
by 1337
Just a note: ENet can use IP v6 since Adi put in some serious work.

Re: Diary: Masterserver

Posted: Sun Oct 10, 2010 6:48 pm
by smerkli
I'll go for ENet, then. I'll start out going over the details of the networking implementation next wednesday. I'll be back with questions, that's for sure :D
I appreciate your help, guys. Thanks a lot.

UPDATE: I've been poking around the internet for material about the ENet library, and found this:

http://enet.bespin.org/Tutorial.html

This looks like a quite efficient way to get me started with the ENet library. (Don't worry, I'm not new to network programming)

Is this the library you guys were talking about, or do you keep some modified version of it around?

Also, I found this:
http://www.orxonox.net/wiki/Network

I guess it wouldn't be a bad idea to reuse that networking concept for my purposes?

cheers

Re: Diary: Masterserver

Posted: Wed Oct 20, 2010 11:34 am
by smerkli
Hey guys, I just started out implementing the main routine of the master server and asked myself where I was to put it in the Orxonox source code context. We thought about making it a module, but since it does hardly use any code belonging to the other parts of Orxonox, a standalone binary with its own main routine would also make sense to me. I do, however, require the Enet library and some other source code for it to work, so it will definitely have to reside inside the Orxonox code tree and be built with the Orxonox build system.

Any ideas?

Re: Diary: Masterserver

Posted: Wed Oct 20, 2010 12:54 pm
by 1337
The current executable is located directly in src/, so I would simply make a folder src/masterserver or something like that.

However I would rather recommend to create a new mode triggered by a command line argument. That way we don't have to deal with multiple executables. Furthermore you will have the IOConsole running. That might come in very handy.

Re: Diary: Masterserver

Posted: Wed Oct 20, 2010 6:40 pm
by greenman
i think having a seperate executable would be much better:
- this way you could still use any code of orxonox (even the IOConsole)
- you could also use our enet bindings or some specific network classes
- it would be possible to create a light orxonox package or src distribution with only the master server in it

so if it's not too much work with two executables then i would recommend going that way

Re: Diary: Masterserver

Posted: Wed Oct 20, 2010 8:49 pm
by 1337
It is possible with a few modifications so that certain stuff doesn't get installed for the masterserver (just the libs and no data).

If I find the time, I can help you with that. Just no right now. In a week or say things look a lot better.

Re: Diary: Masterserver

Posted: Wed Oct 20, 2010 9:12 pm
by smerkli
I'll be focusing on the implementation of the networking and list for the next week or so anyway, so we've got plenty of time to think about the specifics of the packaging. No worries :D

Re: Diary: Masterserver

Posted: Sat Oct 23, 2010 1:35 pm
by x3n
Well I don't think it matters wheter it's a separate executable or a command-line option for the orxonox main executable. Both is possible and I think both can be placed in src/modules/masterserver.

Also note that we won't need a masterserver package, because there's only one masterserver running in the world (which will probably reside on our server). There's no need to distribute it.


Now let's talk about more interesting stuff, the protocol. Is there already a concept? I see 3 necessary communication lines:

1) Gameserver -> Masterserver: A gameserver must register itself at the masterserver and probably also respond to ping queries to detect timeouts

2) Client -> Masterserver: Get a list of all gameservers, maybe with some search criterias (like "only capture the flag servers" or something, but that's not yet possible)

3) Client -> Gameserver: Get detailed information about a gameserver (like the current level, ping, number of players, etc.)

Note that you could also go with only 2 communication lines (if the client gets all information from the masterserver), but I think that's pretty pointless, because it's way more work for the masterserver and also less flexible. The communication (3) allows us to have a list of favorite servers and display their state without querying the masterserver.

The protocols for all 3 communication lines also need a closer look. We're completely free to implement (1), because there's no 3rd party software involved. (2) is also pretty open, except maybe for some tools like ASE or HLSW although they may be using their own masterserver.

(3) however is quite important, because there are many open gameserver-query-tools and -protocols (I have already discussed with oli about this topic). We should definitely support the common server query tools like qstat (for example used in irc channel bots) or gameQ (used on websites), xFire, etc.

Thus we would have to use an open protocol, for example Gamespy (1 or 2), or also older (but still very popular) protocols like quake3 or halflife server queries. Please refer to this site for an overview and more details about the mentioned protocols.


Now you will probably not implement all this in a few weeks, but it can't hurt to think about it as early as possible. ;)

Re: Diary: Masterserver

Posted: Wed Oct 27, 2010 2:30 pm
by smerkli
I actually made a coarse protocol concept already, located here:

http://www.orxonox.net/wiki/Masterserver

I've now realized the master server part as a module with a singleton, some more fine-tuning will be required though. I'll be testing and debugging the networking primitives for next week and check out some of those protocols. I'm not sure though if they also use UDP? It'd be pretty pointless to implement a protocol if it actually uses TCP in all other implementations.

cheers

Re: Diary: Masterserver

Posted: Wed Oct 27, 2010 3:17 pm
by greenman
hm... key is that they probably don't use enet. so if we wan't to be compatible to these protocols we cannot use enet for this communication part (i think this only concerns client->masterserver, right?) because enet also puts some stuff into the network packets ...

Re: Diary: Masterserver

Posted: Wed Oct 27, 2010 3:33 pm
by x3n
I don't know anything about these protocols, but I guess they use TCP. Speed doesn't really matter for server stats, but reliability does.

And yes, these protocols require raw packets, no enet data allowed. Except maybe if there's a key-byte for the start of the data or something. But does enet really add its own stuff to a packet? I see some need for this if you use UDP, but TCP should work without additional information (TCP includes a sequence number, UDP doesn't).

Re: Diary: Masterserver

Posted: Wed Oct 27, 2010 6:38 pm
by greenman
afaik enet cannot use tcp

Re: Diary: Masterserver

Posted: Wed Oct 27, 2010 8:18 pm
by x3n
what about using boost asio?