[Pipmak-Devel] The Pipmak Make setup for Linux

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[Pipmak-Devel] The Pipmak Make setup for Linux

Aidan Gauland
Hello,

  Ok, I have.. more than "tweaked" the Pipmak Makefile for building on Linux,
and here's how it works...

  Note: Skip step 1 if you're going to use your system-wide shared libraries,
but it still links to Lua statically so you still have to compile Lua.

  Step 1: Compile libogg, libvorbis, physfs, and
openal with the --disable-static option passed to the configure script.

  Step 2: Compile Lua.  (Its make system is different from most libraries,
and doesn't have the option to disable shared libraries.)

  Step 3: Move or symolically link each of those libraries to
pipmak/build-linux, truncating the version number from the directory name.
Example:
ln -s /path/to/libvorbis-1.2.0 build-linux/libvorbis
OR
mv /path/to/libvorbis-1.2.0 build-linux/libvorbis

  Step 4: Now if you're building for the official tarball release, just type
make version=x.y.z release
as before.  But if you want to build Pipmak from source for yourself, using
your system-wide shared libraries, then you will need to comment and uncomment
a few lines in the Makefile.  The lines are clearly marked.  After that, run
make, and it will build Pipmak against your system-wide libraries (assuming
you have installed them correctly).

  What actually happens when you run "make version=x.y.z release" is (in
addition to what it did with the original Makefile) is copies the shared
object files for the libraries it needs to pipmak-x.y.z/libs/ and copies a
wrapper script to pipmak-x.y.z/pipmak and the actual Pipmak binary is named
pipmak-bin.  The wrapper script (just a shell script) sets the LD_LIBRARY_PATH
environment variable to look in the libs folder that was created earlier, and
executes pipmak-bin.

  However, I still left it so that it links to Lua statically, not because I
think it an uncommon library, but because we will want Pipmak to be using a
properly configured build of Lua (right?).  For example I found that it
doesn't enable loadlib by default in the configuration file.

  So why did I do it this way instead of linking to all of these libraries
statically?  Because (at least on my system, I haven't done extensive testing)
if the system already has the libraries installed system-wide, then those are
used instead of the .so files packaged with Pipmak, so a user, at their
option, can remove the .so files packaged with Pipmak, or replace them with
their own.  And after talking to some other Linux users, I understand that
when you try to link to several libraries statically, especially when some
libraries depend on others, you get into what some call dependency hell.
That's when you have to keep trying to resolve dependencies of dependencies,
until it's nearly impossible to figure out what it is that you're missing.
So the answer is, in short, because it's simpler.

  But what do all you other Pipmak users/developers think of this?

  -Aidan



Reply | Threaded
Open this post in threaded view
|

Re: [Pipmak-Devel] The Pipmak Make setup for Linux

cwalther
Administrator
Aidan Gauland wrote:
>   Ok, I have.. more than "tweaked" the Pipmak Makefile for building on Linux,
> and here's how it works...
>
> ...
>
>   But what do all you other Pipmak users/developers think of this?

I don't see a Makefile here, so I can't comment in detail...

Generally, I don't really have an opinion either way. I lack the
experience with Linux to tell whether that is the approach that leads to
the best compatibility with different distributions. As long as either
a) it does not get much more difficult for me to build the binary
releases or b) you take over building of the binary releases, I'm open
to trying this. (Not for the 0.2.7 release though, which I am planning
on getting done this weekend.)

> The wrapper script (just a shell script) sets the LD_LIBRARY_PATH
> environment variable to look in the libs folder that was created earlier, and
> executes pipmak-bin.

That seems to have been the usual approach for closed-source games.
However, I seem to remember Gabriele Greco posting that he had some
problems with that because some newer distributions used that variable
for system purposes, or something. Have you happened on this in your
research?

Thanks for your work!

  -Christian



Reply | Threaded
Open this post in threaded view
|

Re: [Pipmak-Devel] The Pipmak Make setup for Linux

Aidan Gauland
Christian Walther wrote:
> Aidan Gauland wrote:
>>   Ok, I have.. more than "tweaked" the Pipmak Makefile for building on Linux,
>> and here's how it works...
>>
>> ...
>>
>>   But what do all you other Pipmak users/developers think of this?
>
> I don't see a Makefile here, so I can't comment in detail...

I'll send that now. :-D  And the wrapper script.

>
> Generally, I don't really have an opinion either way. I lack the
> experience with Linux to tell whether that is the approach that leads to
> the best compatibility with different distributions. As long as either
> a) it does not get much more difficult for me to build the binary
> releases or b) you take over building of the binary releases, I'm open
> to trying this. (Not for the 0.2.7 release though, which I am planning
> on getting done this weekend.)

I'll take over building the Linux binary release, if that would make things
easier.

>
>> The wrapper script (just a shell script) sets the LD_LIBRARY_PATH
>> environment variable to look in the libs folder that was created earlier, and
>> executes pipmak-bin.
>
> That seems to have been the usual approach for closed-source games.
> However, I seem to remember Gabriele Greco posting that he had some
> problems with that because some newer distributions used that variable
> for system purposes, or something. Have you happened on this in your
> research?
I've not heard of that.  I thought Linux systems used something else.  But
even if that is a problem, I can easily avoid that, by appending to that
variable, rather than overwriting it.

And by the way, I tried searching the SDL mailing list archives, but it wasn't
working for me.

makeMod-r186.tar.gz (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Pipmak-Devel] The Pipmak Make setup for Linux

cwalther
Administrator
Aidan Gauland wrote:
> I'll take over building the Linux binary release, if that would make things
> easier.

That would be very welcome, let's see how that goes. One of the main
reasons why I release so rarely is the effort it takes to build and test
on all platforms. Distributing that burden onto more shoulders should
make things faster. (It also means more management duties for me,
getting away a bit from my hermit programmer attitude. We'll see how I
cope with that. :) )

If you tell me your SourceForge user name (make one at
<http://sourceforge.net/account/registration/> if you don't already have
one), I'll give you write access to the SVN repository.

>> I don't see a Makefile here, so I can't comment in detail...
>
> I'll send that now.

I haven't tested it, but it looks OK. If you commit it, don't forget to
update Building.txt. One thing I wonder, wouldn't it be more elegant to
do the uncomment-this-comment-out-that sections using ifs and a variable
set on the command line (like 'version' and 'dependencies')?

> And the wrapper script.

That looks like it only runs if the current working directory is the
directory that contains pipmak-bin and libs. Is that acceptable?

>> That seems to have been the usual approach for closed-source games.
>> However, I seem to remember Gabriele Greco posting that he had some
>> problems with that because some newer distributions used that variable
>> for system purposes, or something. Have you happened on this in your
>> research?
>
> I've not heard of that.  I thought Linux systems used something else.  But
> even if that is a problem, I can easily avoid that, by appending to that
> variable, rather than overwriting it.

For reference, since I just have it in front of me, that post is here:
<http://thread.gmane.org/gmane.comp.lib.sdl/35714/focus=35778>

> And by the way, I tried searching the SDL mailing list archives, but it
> wasn't working for me.

What archive were you using? Gmane works fine for me:
<http://search.gmane.org/?query=&author=Gabriele+Greco&group=gmane.comp.lib.sdl&sort=date&DEFAULTOP=and>
(I hope Gabriele doesn't mind me using him as a generic search term, but
it seems that he reliably has at least one post in every thread relevant
to this topic :) )

  -Christian



Reply | Threaded
Open this post in threaded view
|

Re: [Pipmak-Devel] The Pipmak Make setup for Linux

Aidan Gauland
Christian Walther wrote:

> Aidan Gauland wrote:
>> I'll take over building the Linux binary release, if that would make things
>> easier.
>
> That would be very welcome, let's see how that goes. One of the main
> reasons why I release so rarely is the effort it takes to build and test
> on all platforms. Distributing that burden onto more shoulders should
> make things faster. (It also means more management duties for me,
> getting away a bit from my hermit programmer attitude. We'll see how I
> cope with that. :) )
>
> If you tell me your SourceForge user name (make one at
> <http://sourceforge.net/account/registration/> if you don't already have
> one), I'll give you write access to the SVN repository.

My SourceForge username is wgsilkie, but hang on a minute, when I type "make"
in "pipmak/documentation/" it seems to run something that looks like it's
dealing with LATEX in an infinite loop, but when I run "pdflatex
Reference.tex" it seems to generate Reference.pdf, Reference.aux,
Reference.idx, Reference.log, Reference.out, and Reference.toc just fine.  Is
this okay for putting in the Pipmak releases?

>
>>> I don't see a Makefile here, so I can't comment in detail...
>> I'll send that now.
>
> I haven't tested it, but it looks OK. If you commit it, don't forget to
> update Building.txt. One thing I wonder, wouldn't it be more elegant to
> do the uncomment-this-comment-out-that sections using ifs and a variable
> set on the command line (like 'version' and 'dependencies')?
>
>> And the wrapper script.
>
> That looks like it only runs if the current working directory is the
> directory that contains pipmak-bin and libs. Is that acceptable?

Hmmm, maybe not.  I could put pipmak-bin and libs in a directory called
something like "data", and leave the wrapper script "pipmak" where it is.  Or,
then again, maybe that would be needlessly redundant.  I guess the best way to
find out works best is to let Linux users try it out, and make changes based
on feedback (on the Pipmak users mailing list).

>
>>> That seems to have been the usual approach for closed-source games.
>>> However, I seem to remember Gabriele Greco posting that he had some
>>> problems with that because some newer distributions used that variable
>>> for system purposes, or something. Have you happened on this in your
>>> research?
>> I've not heard of that.  I thought Linux systems used something else.  But
>> even if that is a problem, I can easily avoid that, by appending to that
>> variable, rather than overwriting it.
>
> For reference, since I just have it in front of me, that post is here:
> <http://thread.gmane.org/gmane.comp.lib.sdl/35714/focus=35778>
>
>> And by the way, I tried searching the SDL mailing list archives, but it
>> wasn't working for me.

I was just using the archive that the SDL website lead me to, but now that
I've tried Gmane, I'm having no trouble.  Thanks.

>
> What archive were you using? Gmane works fine for me:
> <http://search.gmane.org/?query=&author=Gabriele+Greco&group=gmane.comp.lib.sdl&sort=date&DEFAULTOP=and>
> (I hope Gabriele doesn't mind me using him as a generic search term, but
> it seems that he reliably has at least one post in every thread relevant
> to this topic :) )
>
>   -Christian
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Pipmak-Devel mailing list
> [hidden email]
> news://news.gmane.org/gmane.games.devel.pipmak.devel
> https://lists.sourceforge.net/lists/listinfo/pipmak-devel
>




Reply | Threaded
Open this post in threaded view
|

Re: [Pipmak-Devel] The Pipmak Make setup for Linux

cwalther
Administrator
Aidan Gauland wrote:
> when I type "make" in "pipmak/documentation/" it seems to run
> something that looks like it's dealing with LATEX in an infinite
> loop

The loop isn't supposed to be infinite. It's supposed to loop until the
.toc, .aux, and .ind files stop changing. Two or three times through the
loop should be sufficient for that. The files depend on each other in a
roundabout way: the first pdflatex run creates Reference.toc,
Reference.idx, and an unfinished version of Reference.pdf. Then
makeindex generates Reference.ind from Reference.idx. The second run of
pdflatex integrates Reference.toc and Reference.ind to generate an
updated version of Reference.pdf. Or something like that. I don't
remember what the .aux file is for, perhaps section links.

> but when I run "pdflatex Reference.tex" it seems to generate
> Reference.pdf, Reference.aux, Reference.idx, Reference.log,
> Reference.out, and Reference.toc just fine.

But I bet that Reference.pdf has incomplete or missing table of
contents, index, or section links.

> Is this okay for putting in the Pipmak releases?

If you don't want to fiddle with the LaTeX stuff, you could package the
Reference.pdf generated by me. We can talk about how to best organize that.

>> That looks like it only runs if the current working directory is the
>> directory that contains pipmak-bin and libs. Is that acceptable?
>
> Hmmm, maybe not.  I could put pipmak-bin and libs in a directory called
> something like "data", and leave the wrapper script "pipmak" where it is.

How would that solve the problem?

> I guess the best way to find out works best is to let Linux users try
> it out, and make changes based on feedback (on the Pipmak users
> mailing list).

OK. I'm still trying to find out how we would publish such test
packages. SourceForge doesn't want us to release files on the web site,
and using the file release systems where each release will be kept
around forever doesn't sound right either. I somehow seem to remember
that SourceForge had a specific facility for releasing nightly builds,
but I can't find any trace of it now.

  -Christian



Reply | Threaded
Open this post in threaded view
|

Re: [Pipmak-Devel] The Pipmak Make setup for Linux

Aidan Gauland
Christian Walther wrote:

> Aidan Gauland wrote:
>> when I type "make" in "pipmak/documentation/" it seems to run
>> something that looks like it's dealing with LATEX in an infinite
>> loop
>
> The loop isn't supposed to be infinite. It's supposed to loop until the
> .toc, .aux, and .ind files stop changing. Two or three times through the
> loop should be sufficient for that. The files depend on each other in a
> roundabout way: the first pdflatex run creates Reference.toc,
> Reference.idx, and an unfinished version of Reference.pdf. Then
> makeindex generates Reference.ind from Reference.idx. The second run of
> pdflatex integrates Reference.toc and Reference.ind to generate an
> updated version of Reference.pdf. Or something like that. I don't
> remember what the .aux file is for, perhaps section links.

Ok, I figured out what the problem was: By default, make uses /bin/sh as the
shell, and that is very often, on Unix systems, a link to the bash binary, but
on some Linux configurations, that is instead a link to (or is the binary) a
more stripped down, POSIX compliant shell that isn't compatable with all bash
scripts.  So if I add the line...

export SHELL=/bin/bash

...to the top of the Makefile, it runs, and actually finishes.

Since the Makefile for the Pipmak documentation uses some bash specific
features (i.e. is not 100% POSIX compliant), should this line be added to the
documentation Makefile?

>
>> but when I run "pdflatex Reference.tex" it seems to generate
>> Reference.pdf, Reference.aux, Reference.idx, Reference.log,
>> Reference.out, and Reference.toc just fine.
>
> But I bet that Reference.pdf has incomplete or missing table of
> contents, index, or section links.
>
>> Is this okay for putting in the Pipmak releases?
>
> If you don't want to fiddle with the LaTeX stuff, you could package the
> Reference.pdf generated by me. We can talk about how to best organize that.

I think that might be best, so that the EXACT SAME file is packaged with each
OS build of Pipmak.  I don't know if there would be any slight differences if
there were different people generating Reference.pdf on different systems, but
it would probably best to try to avoid that for the official release.

>
>>> That looks like it only runs if the current working directory is the
>>> directory that contains pipmak-bin and libs. Is that acceptable?
>> Hmmm, maybe not.  I could put pipmak-bin and libs in a directory called
>> something like "data", and leave the wrapper script "pipmak" where it is.
>
> How would that solve the problem?
>
>> I guess the best way to find out works best is to let Linux users try
>> it out, and make changes based on feedback (on the Pipmak users
>> mailing list).
>
> OK. I'm still trying to find out how we would publish such test
> packages. SourceForge doesn't want us to release files on the web site,
> and using the file release systems where each release will be kept
> around forever doesn't sound right either. I somehow seem to remember
> that SourceForge had a specific facility for releasing nightly builds,
> but I can't find any trace of it now.
>
>   -Christian
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Pipmak-Devel mailing list
> [hidden email]
> news://news.gmane.org/gmane.games.devel.pipmak.devel
> https://lists.sourceforge.net/lists/listinfo/pipmak-devel
>




Reply | Threaded
Open this post in threaded view
|

Re: [Pipmak-Devel] The Pipmak Make setup for Linux

cwalther
Administrator
Aidan Gauland wrote:
> So if I add the line...
>
> export SHELL=/bin/bash
>
> ....to the top of the Makefile, it runs, and actually finishes.
>
> Since the Makefile for the Pipmak documentation uses some bash specific
> features (i.e. is not 100% POSIX compliant), should this line be added to the
> documentation Makefile?

I would suggest that instead the Makefile be changed so that it runs on
any POSIX shell.

> I think that might be best, so that the EXACT SAME file is packaged with each
> OS build of Pipmak.  I don't know if there would be any slight differences if
> there were different people generating Reference.pdf on different systems, but
> it would probably best to try to avoid that for the official release.

Agreed.

  -Christian



Reply | Threaded
Open this post in threaded view
|

Re: [Pipmak-Devel] The Pipmak Make setup for Linux

cwalther
Administrator
In reply to this post by Aidan Gauland
Aidan Gauland wrote:
> # A wrapper script for Pipmak on UNIX systems.  It makes sure Pipmak gets the
> # full path to project files given to it (if any).

Of course, the better solution would be to fix Pipmak's inability to
deal with relative paths on the command line. Thanks for bringing that
bug to my attention! I wonder why I never noticed that...

  -Christian