cmake: out of source builds

Found a bug? Report it here.

Moderator: PPS-Leaders

Post Reply
adi
Noxonian Warkmrghon
Posts: 18
Joined: Wed Sep 24, 2008 1:12 pm

cmake: out of source builds

Post by adi » Sun Sep 28, 2008 9:27 pm

When trying to build orxonox outside the source tree, the following files are still created inside the source tree:

bin/lib/libnetwork.so
bin/lib/libceguilua_orxonox.so
bin/lib/libaudio.so
bin/lib/libcore.so
bin/lib/libtinyxml_orxonox.so
bin/lib/libutil.so
bin/lib/libtolualib_orxonox.so
bin/lib/libcpptcl_orxonox.so
bin/lib/libois_orxonox.so
bin/lib/liblua_orxonox.so
bin/orxonox
bin/tolua_orxonox
bin/toluagen_orxonox
src/core/tolua/tolua_bind.cc
src/core/tolua/tolua_bind.h
src/orxonox/tolua/tolua_bind.cc
src/orxonox/tolua/tolua_bind.h
src/tolua/toluabind_orxonox.c
src/tolua/toluabind_orxonox.h

User avatar
beni
Baron Vladimir Harkonnen
Posts: 949
Joined: Tue Oct 03, 2006 9:15 am
Location: Zurich
Contact:

Post by beni » Sun Sep 28, 2008 9:51 pm

This is why Orxonox is not yet fully distributable. Unfortunately you cannot where to put the orxonox binary to.

the tolua prefixed files, however, have to be created in the src-directory because they are necessary to compile the binary, but have to be created automatically. Of course those files can be created beforehand but they need to be changed when certain code gets altered. This is why the creation of those files are done before compiling, thus integrated into cmake.

I think this should be solved when there is going to be a release in form of a deb-package and other ways of actually releasing this code.
"I'm Commander Shepard and this is my favorite forum on the internet."

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Post by 1337 » Sun Sep 28, 2008 9:52 pm

Actually that is intended for the moment. The files in bin/ have to be there, that is clear. However the other ones are not that good. Today I modified the CMakeLists.txt to include all of them when applying 'make clean'.

The problem is the following: we use tolua, which generates source files according to other source files. But these files that get generated have to be linked into the right executable.
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

adi
Noxonian Warkmrghon
Posts: 18
Joined: Wed Sep 24, 2008 1:12 pm

Post by adi » Mon Sep 29, 2008 1:52 pm

There is no problem in creating any of these files while compiling, but they should be put into the build directory instead of the source directory (which are actually the same while doing an in source build).
An untouched source directory will allow different builds with different compiler options, different compilers, different configuration, different architectures and so on, without the need to have multiple source trees. Furthermore cleaning up an out of source build is just removing the build directory and it would ease automated packaging.

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Post by 1337 » Mon Sep 29, 2008 2:05 pm

Actually I totally agree with that. Which is why I have realised out-of-source builds in the first place...

Unfortunately, there is not enough time to fix things soon... 2 Weeks at least until I can take care of that. I would also like to fix the visual studio generator related parts in our CMake files. That inevitably means taking care of release/debug builds (I am currently working with my self managed visual studio files for obvious reasons).
It really is on my todo list ;)

EDIT: About the tolua bind files. They're not exactly dependent on debug/release or any other compiler option. However, we might want to add some more code when in debug mode. CEGUI had the same problem and they solved it by inserting preprocessor code while automatically generating these source files. It would also add a certain level of inconsistency if some source files are different (regarding build options) and others are not.
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

User avatar
greenman
Baron Vladimir Harkonnen
Posts: 360
Joined: Wed Oct 03, 2007 2:53 pm
Contact:

Post by greenman » Mon Sep 29, 2008 5:36 pm

i think the tolua_bind files are perfectly fine, where they are placed during pre-compilation

afaik we could also add them to the repo (because they're platform independant), but we would always have to change them, when something with the tolua_bind changes

I'm not really familiar with this, but thats how I understood it so far
There are only 10 types of people in the world: Those who understand binary, and those who don't.

User avatar
beni
Baron Vladimir Harkonnen
Posts: 949
Joined: Tue Oct 03, 2006 9:15 am
Location: Zurich
Contact:

Post by beni » Mon Sep 29, 2008 7:46 pm

That's correct. The only files which should be considered to create outside of the source tree is the bin folder. The tolua files are fine, since they are independent of everything.
"I'm Commander Shepard and this is my favorite forum on the internet."

adi
Noxonian Warkmrghon
Posts: 18
Joined: Wed Sep 24, 2008 1:12 pm

Post by adi » Mon Sep 29, 2008 10:05 pm

greenman wrote:i think the tolua_bind files are perfectly fine, where they are placed during pre-compilation
No. The reason is, that you should get a pristine source tree with only deleting the build directory. See CMake FAQ.

Having pregeneretad files in a version control system neither is a good idea.

I've started working on this issue.

CU, Adi

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Post by 1337 » Tue Sep 30, 2008 5:31 am

adi wrote:
greenman wrote:i think the tolua_bind files are perfectly fine, where they are placed during pre-compilation
No. The reason is, that you should get a pristine source tree with only deleting the build directory. See CMake FAQ.

Having pregeneretad files in a version control system neither is a good idea.

I've started working on this issue.

CU, Adi
As I said, I do agree on some points with you, especially on the clean source tree. If would like to work on this, please make a branch to have a clean suggestion. I would like to see things being in build/src/core/tolua, etc. to keep structure. The problem is referencing it in the CMake files will look quite ugly with that long paths, but should be manageable.

EDIT: Since you sound intereseted in our build system, I took the liberty of creating a PPS ticket. If you like, your'e welcome to accept that as a first job because it probably won't take too long. If have no interest, I am most probably going to do it in collaboration with the CMake Visual Studio generator. But thats going to take 2 or 3 weeks..
http://www.orxonox.net/ticket/278

EDIT2: I thought about it a little more. We still haven't modified tolua enough to avoid path confusions. Currently, you will have to run tolua in src/bin/lib. But the executable is in src/bin. That means you have to reference all used header files in the tolua.pkg files in respect to src/bin/lib, which we do now.
It's not really a problem, but you certainly will have to consider it. I also suggest removing the tolua folders in each library (and yes, we need a tolua bind source file for each library of our own because of dependencies) and putting tolua.pkg into the library folder (e.g. src/core, src/orxonox) directly.
If someone has the time and knows lua a little bit, he could modify tolua to treat the paths in a better way (relative to the pkg-file would be nice).
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

User avatar
1337
Baron Vladimir Harkonnen
Posts: 521
Joined: Wed Oct 10, 2007 7:59 am

Post by 1337 » Fri Oct 10, 2008 10:51 pm

Today I've been working on better support for the Visual Studio project files. And I have come across a major problem, that has yet been ignored:
The files in bin/

As far as Oli told me, this is an issue in linux anyway. But I neither can nor do care about the problem under linux. I'll let the Adi or Oli handle this issue.
Anyway, the problem still exists under Windows: Even if you still have all your config files in bin, you cannot build debug and release at the same time. For now, I have dealt with it by postfixing "_d" to every file. But that's not exactly a very good solution. So I've decided to make folders for every msvc solution (I'm starting to port things to VS08) and every configuration (only debug and release).
And of course I want to make this Orxonox SDK easily distributable, so I have to deliver the plugins.cfg from ogre and I also need to set the dataPath differently than the default value (hard coded).
In order to solve this by not adding four more files to bin/ (name plugins.cfg-init2_d or something even worse), I took the liberty of creating a new root directory in the trunk: "init"
It contains a folder named "common" where all those files lie, that are needed by Orxonox to run and are platform independent (they also need to be exactly in the same folder as the executable). There is another folder, "orxonox_vc8", with two subfolders, "debug" and "release", which contain plugins.cfg and orxonox.ini (specifically designed that the Visual Studio SDK works in the first place).

Now whenever the build process is complete, these files are all copied to the output directory (for instance bin/orxonox_vc8/release) if not yet existing.

I honestly have very little idea how that is handled under linux (make install?), so however this is dealt with, I am willing to adapt to it as long as I can apply the scheme described above in some way (for visual studio of course).
Furthermore I talked to Oli and he wasn't too pleased about the name "init". "config" was his suggestion. As I said I'm open for suggestions and I don't really care how it is solved as long as the Visual Studio stuff can be made to work properly like now.

The changes with the folders were applied to the trunk, so Adi, If this is part of your upcoming work, I suggest to merge back the buildsystem branch as soon as one more linux user, fabian (mingw) and me have tested it. But it's only a suggestion. Again, I don't feel too well informed to judge the issue like that...

EDIT: Perhaps something I would like to add:
We are currently quite creative when it comes to distributing plugins.cfg:
  • On standard linux, the run-script copies plugins.cfg-init to plugins.cfg

    On Windows with MinGW, we simply use Plugins.cfg

    On an eth tardis box, again, the run-script copies plugins.cfg-init, but also edits the PluginsFolder.

    On Windows with Visual Studio, contents from the init folder are copied to the output directory, including plugins.cfg.
Btw: There is an SVN patch on my box that would effectively remove plugins.cfg and put its contents to orxonox.ini, if that is desired. --> One config file less, but plugins.cfg would probably be easier to configure by CMake.
http://www.xkcd.com/
A webcomic of romance, sarcasm, math, and language.

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests