TeX Live build procedure

Building the executable programs included in TeX Live involves usual steps of downloading the TL sources, configuring, compiling, and installing. But there are some peculiarities, described here.

TeX Live, unlike many other projects, ships precompiled binaries to TeX users. This web page is especially for the volunteers who contribute those binaries, but anyone who wants to build TeX Live from source may find it useful.

The tlbuild manual has full details about the source organization (overview), how to build, etc.

We have a mailing list specifically for discussion of build issues: tlbuild@tug.org. Feel free to join or peruse the archives if you are interested.

On this page: downloading - building - deployment/testing - xz & wget - xindy - biber - asymptote - volunteers.

Downloading the sources

A variety of ways to get the development sources are available. In short, if all you want to do is compile the sources (as opposed to having a complete runnable TeX system), you can use rsync -a --delete --exclude=.svn tug.org::tldevsrc/Build/source/ /your/dir/

All the components of the TeX Live release, including the program sources, are also available as a tarball.

Overview of build system.

Building

The prerequisites for a default build are not too taxing, but you may want to check through them.

The simplest way is to configure, compile, and install in one step using the top-level Build script. Use Build --debug to compile without optimization. (The unstripped binaries will be left in the build directory, namely ./Work, where you can remake normally after source changes.) Other options are passed to configure.

If you want to exclude certain programs from the build, use --disable-progname. Run configure --help to see the list.

Most everything that Build does can be overridden with environment variables; take a look. Alternatively, what Build boils down to is configure && make world, where make world is the same as make install strip. Adapt as desired. Run make check for to run the available tests of what gets built. More info on configure options.

When making the TL-distributed executables, please use Build or otherwise arrange for the binaries to be stripped. (The targets world and strip in the top-level Makefile will do this.)

When making executables for an independent distro or other use, we request/encourage adding a suffix to the banner, via adding something like this to the configure arguments: --with-banner-add='/MyDistro'. When using the Build script, that would be TL_CONFIGURE_ARGS=--with-banner-add=....

More build info.

Deployment and testing

The result of the TL make install is a directory such as Build/source/inst/bin/i686-pc-linux-gnu full of executables, but they can't be directly run from there. (The data files created by make install under inst/ aren't directly useful for anything. Ignore them.)

The recommended way to test newly-built binaries for a platform already supported by TL is to copy/move that bindir to the existing Master/bin/archname in a full TL checkout, where archname is the TL platform name, such as i386-linux. (This name is different than the directory name resulting from the make install.)

If your binaries are for a platform not already supported by TeX Live, using the TL installer with the -custom-bin option is the easiest method.

Once you have a tree including both all the support files (gigabytes of stuff) and your new binaries, then run, for instance, latex small2e; the latex.fmt file should be automatically built and you should end up with a small2e.dvi file that you can, for example, view with xdvi or convert with dvips. Similarly for pdflatex and all the other engines.

For the TL-distributed executables, when you are happy with the results, either post a tarball of the bindir (preferably with no top level directory, and including asy, biber, and xindy if they've changed) on the web somewhere and email karl@tug.org, or commit them to the TL repository yourself. If you like to do it youself, you may find the tl-update-bindir script useful. (It takes care of Subversion housekeeping chores, mostly related to symbolic links. It also avoids deleting the special cases of binaries we distribute that are not created by make install, xindy, biber, and asymptote, all of which are discussed further below.)

For deploying and testing your own builds, you may find the TeX Live tarballs convenient. For instance, if you have run configure with --disable-multiplatform --prefix=/somewhere/texlive, then unpack at least the texmf and texmf-dist tarballs under /somewhere.

Special considerations for xz & wget

xz is a compression format (essentially zip with improved parameters), which compresses both better and faster than bzip2. As of TeX Live 2009, we use it in the infrastructure (intentionally not provided as user-level executables). (We used xz's predecessor, lzma, in 2008.)

Please download and compile xz from the original distribution. What we want to end up with are the stripped xz and xzdec binaries (only) in Master/tlpkg/installer/xz, with the TL platform name as a suffix, e.g., xz.i386-linux.

So, for the TL-distributed executables, something like this:

  1. cd /some/directory/xz-*
  2. ./configure --disable-nls --disable-shared --disable-threads && make
  3. strip src/xz/xz src/xzdec/xzdec
  4. cp src/xz/xz xz.`tlmgr -print-arch`
  5. cp src/xzdec/xzdec xzdec.`tlmgr -print-arch`
  6. Either email a url to those binaries to karl@tug.org or commit them yourself in the location tlpkg/installer/xz.

The `tlmgr -print-arch` is just a generic short cut to get the platform name as known to TeX Live—the names of the directories under Master/bin/ in the source repository. You can also just type in the string.

wget: on systems which do not provide GNU wget as part of the normal installation, it is good to download, compile, and install it under tlpkg/installer/wget, as with xz (mutatis mutandi). Configure options to remove most library dependencies:
--enable-ipv6 --disable-iri --disable-nls --disable-ntlm --disable-pcre --without-libiconv-prefix --without-libintl-prefix --without-libpsl --without-libuuid --without-ssl --without-zlib

Special considerations for biber

TeX Live includes biber (a BibTeX replacement for BibLaTeX users) executables. These executables bundle the many Perl modules which biber requires, using the Par::Packer mechanism. This process is described in the biber documentation; it isn't feasible to incorporate it in the standard TL build.

So, TL builders who wish to provide it should make and upload the binaries as part of the upstream Biber project. These binaries are then distributed to CTAN (requires a manual request to the CTAN maintainers, usually done by the Biber maintainers), and TL imports from CTAN as usual.

Special considerations for xindy

TeX Live includes Joachim Schrod's xindy indexing program. However, xindy requires GNU CLISP, a highly nontrivial dependency; therefore, it is disabled by default in the Build script.

For the TL-distributed executables, to enable xindy, first compile clisp, as explained in the instructions for setting up clisp. (The relevant info is at the end of the linked-to file; you can ignore the notes about the other programs.) There is no need to install clisp anywhere. Then, specify the path to the clisp binary built in your clisp source tree to Build or configure, as in --enable-xindy CLISP=/path/to/clisp/binary.

The reason for building our own clisp to use for the TL compilation is to minimize shared library dependencies. Although some systems come with clisp, using the system version will probably pull in many other libraries, resulting in an xindy executable that's unlikely to work for many TL users.

Testing xindy: you can run texindy idxhello.idx, where idxhello.idx (and the LaTeX file that creates it) is in the TL build tree. The output should be a sorted two-item index. Unfortunately you must temporarily put the new files texindy, tex2xindy, xindy, xindy.mem, xindy.run (from Build/inst/bin/*/) in the Master/bin/*/ subdirectory and run it from there, as the xindy modules are not otherwise available. Be sure to revert them afterwards.

Special considerations for asymptote

We include the asymptote 3d graphics program in TeX Live. However, due to the complexity of its build process, it is not enabled in the Build script by default. Instead, you have to do it by hand:

  1. cd /your/texlive/checkout/source/utils/asymptote
  2. svn update
  3. ./configure --prefix=/tmp/asyinst --enable-texlive-build \
    CXXFLAGS=-ansi --disable-gsl --disable-fftw # --disable-gl
  4. make # possibly relink with static readline
  5. make check # if you want
  6. strip asy

You can test the various libraries with the asytestlibs.asy file available in the source tree.

Then, send karl@tug.org a url to the resulting stripped asy executable. Or, if you commit it yourself, also commit a symlink named xasy, pointing to ../../texmf-dist/asymptote/GUI/xasy.py.

Explanations:

Implementation

This web page isn't the place to write in depth about how TeX and friends are compiled in TeX Live. But in brief, Knuth's original sources are implemented as literate programs using Pascal, and TL's web2c utility converts them to C for compilation. The build infrastructure is based entirely on GNU Automake.

One tip: if you need to debug what symbols are getting #defined where, and you're using gcc, you can try:
gcc ... -E -dD -o /tmp/out
It's usually best to copy the exact invocation from the make output and replace the usual -c -o foo.o with the above. Other letters besides D can be used with -d (documentation in the GCC manual).

Another tip, also for gcc:
gcc -E -dM - </dev/null | awk '{$1="";print}' | sort
will show the symbols predefined by the compiler. Nelson Beebe has a cc-defs script to do the same thing with many other compilers.

Volunteer builders

Here is a list of platforms and their respective builders, to the best of our knowledge. Volunteers for other systems are most welcome.

Please send corrections to the tlbuild list.


$Date: 2018/04/02 22:31:02 $; TeX Live; [an error occurred while processing this directive]