XMLFile loading

A place to discuss everything about the Orxonox framework.

Moderator: PPS-Leaders

Post Reply
User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

XMLFile loading

Post by 1337 » Mon Jul 20, 2009 11:24 am

While looking through the Ogre Resource Management I realised something quite interesting:
The actual resource you can create (for instance with MaterialManager::create(...)) is of course not always the file itself but rather part of a content of a file. One *.material file may of course hold several materials.
The scripts get compiled upon initialiseAllResourceGroups(). However the resources are not being loaded there.

All that made me think: We could do the same with our XML files by parsing them at startup (or per reload button so you don't have to restart orxonox when editing level files). I'm not sure whether this works with the XML Templates (core class) though. But we could of course simply store temporary XML code (without the lua code) if that's easier.
The point I'm trying to get to is simple: Instead of giving Loader::load() a file and receiving nothing, we could give the LevelManager the name of the level and receive a orxonox::Level pointer.

I would implement this by deriving the LevelManager from Ogre::ResourceManager and write some code that can identify Level-XML-Nodes without actually loading them.

That idea could be extended for OverlayGroups or whatever we need to load as a whole. So the whole thing is more of an advanced orxonox::Template class...

What do you think?
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

User avatar
beni
Baron Vladimir Harkonnen
Posts: 949
Joined: Tue Oct 03, 2006 9:15 am
Location: Zurich
Contact:

Re: XMLFile loading

Post by beni » Mon Jul 20, 2009 11:45 am

I think that is great. Finally we get some insight in the more sophisticated loading process of the Ogre engine. I had a quick look at the system some years ago, but Orxonox was nothing then at the moment and I was just happy to be able to load a some meshes.
Your approach sounds fine I guess. Since the lua code only generates correct XML we should store that generated code temporarily, but rerun the lua code when reloading.

However reloading the level isn't really something for the user, but it could be a first step into the direction of a level editor. So this reloading function would be useful in the future anyway, maybe in another context then.
"I'm Commander Shepard and this is my favorite forum on the internet."

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Re: XMLFile loading

Post by 1337 » Mon Jul 20, 2009 11:53 am

Reloading is probably best interfaced with a console command. It's just a detail but we should keep that in mind in case my suggestion were to be found valid.
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

User avatar
x3n
Baron Vladimir Harkonnen
Posts: 810
Joined: Mon Oct 30, 2006 5:40 pm
Contact:

Re: XMLFile loading

Post by x3n » Mon Jul 20, 2009 4:33 pm

I thought about that too when we had all the include problems for overlay and template classes in XML files (you have to include the corresponding overlay file if your levelfile gives the player spaceship with a HUD). Of course it's "correct" if you always have to include the needed stuff, but there are some cases where it's a problem: We have Gametype specific HUDs (for example the Pong-HUD or the Health-Bar of the destroyer in the UnderAttack gametype). All those HUDs are currently defined in default.oxo (which also includes the default HUD with the chat overlay, because default.oxo is the only file which is loaded "hardcoded" - yes it's a hack). And this sucks - we really need a dynamic load-by-name function for this.

But: I'm pretty sure parsing all XML files at startup (and remember - somewhen we'll have a fucking great universe with thousands of spacesations and planets and of course hundreds of user created levels) will be a serious performance issue. It just takes ages to run each file through the Lua parser and after this through the XML parser just to get the name of the root-node (or, if we're using "incorrect" XML-syntax for a collection of classes, the name of all root-nodes).

And you also noticed the need for a reload button - yet another no-go in game design (and usability in general).

That's why I think we need a different approch. One of the simplest of course is to use the filename itself as the name of the root node. We could then use some sort of paths (maybe XPath if we want to start high - http://en.wikipedia.org/wiki/XPath) to point to the classes. So if we want to load the Pong gametype HUD which is still in default.oxo, then we simply load "default.pongHUD". The same notation could be used within other XML files to load templates (templates of Spaceships are needed for spawnpoints), so for example <SpawnPoint ... spawnclass="pirates.HeavyBomber" /> which would then load the whole "pirates.*" file and search for a class named <Template name="HeavyBomber"> ... </Template>

And levels are of course still loaded by name. Returning a Level* pointer makes sense if we agree to downgrade the Loader (it was initially built to load any type of XML files and to create all classes inside. and not just whole levels, but also parts of levels like for example a group of pirates. but since we're using the more generic xml templates and dynamic lua scripts, there's not much need for this feature anymore).
Fabian 'x3n' Landau, Orxonox developer

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Re: XMLFile loading

Post by 1337 » Mon Jul 20, 2009 10:52 pm

Not much of a reply today (it's late and I should really go to bed after an interesting match of night speedminton ;)):

Performance might be a problem, Ogre::initialiseAllResourceGroups already takes quite a few seconds (it parses all the scripts). On the other hand when you compile a source file that includes boost/thread.hpp then it only takes maybe four seconds but pareses an enormous amount of code (in an optimised environment of course...).

We could of course bind the file extension to the type of root class encountered and force it with loading exceptions. A little enumeration (in whatever form) might help there.

That's it for tonight and yeah, I haven't really thought about your answer yet x3n :P
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

User avatar
beni
Baron Vladimir Harkonnen
Posts: 949
Joined: Tue Oct 03, 2006 9:15 am
Location: Zurich
Contact:

Re: XMLFile loading

Post by beni » Tue Jul 21, 2009 8:31 am

Concerning the performance: Their are other functions beside initializeAllResourceGroups. Loading resources in Ogre has many steps and you can initialize, load, unload and stuff for each resource group separately. So this is possible to achieve with a bit more effort in splitting up our media into sensible resource groups and handle them as we need to.
"I'm Commander Shepard and this is my favorite forum on the internet."

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Re: XMLFile loading

Post by 1337 » Tue Jul 21, 2009 8:55 am

Exactly. You can even use those resource groups dynamically. So the group "Level" might change when you load a level. This way you don't have to deal with hundreds of different resource groups.
My idea was somehow provide an easy way to determine the resources associated with a level, but I have no idea how yet.
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

User avatar
greenman
Baron Vladimir Harkonnen
Posts: 360
Joined: Wed Oct 03, 2007 2:53 pm
Contact:

Re: XMLFile loading

Post by greenman » Wed Jul 22, 2009 10:12 am

as ogre provides some form of multithreaded/background resources loading, would it be possible to do other tasks (like class hierarchy creation, input initialization, ...) while loading/parsing the files in background?
There are only 10 types of people in the world: Those who understand binary, and those who don't.

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Re: XMLFile loading

Post by 1337 » Wed Jul 22, 2009 10:19 am

Unfortunately Ogre doesn't really support MT, or would you like to create MT packages for Ubuntu, Debian, etc.?
The current packages haven't been compiled with MT support.
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

User avatar
beni
Baron Vladimir Harkonnen
Posts: 949
Joined: Tue Oct 03, 2006 9:15 am
Location: Zurich
Contact:

Re: XMLFile loading

Post by beni » Wed Jul 22, 2009 3:56 pm

Well the OGRE forum is spewing from stuff about multithreading. However mostly about how and if one does do multithreading during runtime inside the engine or the efficiency to do multithreading with parts like AI and physics. OgreBullet for instance is multithreaded.

About loading resources I didn't find too much, but I imagine that the OGRE forums are worth a loow.
"I'm Commander Shepard and this is my favorite forum on the internet."

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Re: XMLFile loading

Post by 1337 » Wed Jul 22, 2009 4:37 pm

You can load any kind of resource in the background if Ogre was compiled accordingly. No doubt about that (there is information about that in the API documentation as well as in the book).
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

User avatar
greenman
Baron Vladimir Harkonnen
Posts: 360
Joined: Wed Oct 03, 2007 2:53 pm
Contact:

Re: XMLFile loading

Post by greenman » Thu Jul 23, 2009 3:17 pm

i think we should just use this whether it is supported (today) in the binary packages or not. probably if this feature is more stable it will become default for the various distribution packages.

my point is: if we now rethink/restructure our loading stuff we should definately support this, because it will probably be much easier than change everything later on when we need it.
There are only 10 types of people in the world: Those who understand binary, and those who don't.

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Re: XMLFile loading

Post by 1337 » Thu Jul 23, 2009 5:57 pm

On that point I totally agree of course. I've already had a few thoughts about this, but just a few..
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

Post Reply

Who is online

Users browsing this forum: No registered users and 8 guests