This is Info file kpathsea.info, produced by Makeinfo-1.64 from the input file kpathsea.texi. START-INFO-DIR-ENTRY * Kpathsea: (kpathsea). File lookup along search paths. END-INFO-DIR-ENTRY This file documents the Kpathsea library for path searching. Copyright (C) 1993, 94, 95 Karl Berry. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the sections entitled "Freedom" and "GNU General Public License" are included exactly as in the original, and provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that the sections entitled "Freedom" and "GNU General Public License" may be included in a translation approved by the Free Software Foundation instead of in the original English.  File: kpathsea.info, Node: Top, Next: Introduction, Up: (dir) Kpathsea library **************** This manual documents how to install and use the Kpathsea library for filename lookup. It corresponds to version 2.6 (released in January 1995). * Menu: * Introduction:: Overview. * Installation:: Compilation, installation, and bug reporting. * Debugging:: Analyzing runtime problems. * Path searching:: How filename lookups work. * TeX searching:: Special support for TeX lookups. * TeX directory structure:: Ways to manage the many input files. * Programming:: How to use the library in your program. * Copying:: Conditions for copying, modifying and sharing. * Freedom:: Regain your programming freedom. * Index:: General index.  File: kpathsea.info, Node: Introduction, Next: Installation, Prev: Top, Up: Top Introduction ************ This manual corresponds to version 2.6 of the Kpathsea library, released in January 1995. The library's fundamental purpose is to look up a file in a list of directories specified by the user, similar to what shells do when looking up program names to execute. The following software, all of which I maintain, uses this library: * Dviljk * Dvipsk (*note Introduction: (dvipsk)Top.) * GNU font utilities (*note Introduction: (fontu)Top.) * Web2c (*note Introduction: (web2c)Top.) * Xdvik The library is still under development (and probably always will be, despite my hopes). I do not promise to keep the interface unchanged. If you have comments or suggestions, please send them to me (*note Reporting bugs::.). Currently, I distribute the library under the GNU General Public License (*note Copying::.). In short, this means if you write a program using the library, you must (offer to) distribute the source, and allow anyone to modify the source and distribute their modifications. If you have a problem with this, contact me. I would consider putting the library under the GNU Library General Public License, which would permit you to distribute the source only to the library, not to your program using it. But I will only do this if someone actually says they will not use the library under the GPL conditions, and would use it under the LGPL. If you know enough about TeX to be reading this manual, then you (or perhaps your institution) should consider joining the TeX Users Group (if you're already a member, great!). TUG produces a periodical called `TUGboat', sponsors an annual meeting (the proceedings of which are published in `TUGboat'), and arranges courses on TeX for all levels of users. Given sufficient funding (which your joining will help) TUG could sponsor more projects that will benefit the TeX community, such as a successor to TeX pi . Anyway, here is the address: TeX Users Group P.O. Box 869 Santa Barbara, CA 93102 USA phone: (805) 899-4673 email: `tug@tug.org' * Menu: * History::  File: kpathsea.info, Node: History, Up: Introduction History ======= (This section is for those people who are curious about how this came about.) (If you like to read historical accounts of software, I urge you to seek out the GNU Autoconf manual and, even more fun, the "Errors of TeX" paper that Don Knuth published in `Software--Practice and Experience'.) My first ChangeLog entry for Web2c seems to be February 1990, but I may have done some stuff before then. In any case, Tim Morgan and I were sort of jointly maintaining it for a time. (I should say that Tim had made Web2c into a real distribution long before I had ever used it or even heard of it, and Tom Rokicki did the original implementation.) It must have been later in 1990 and 1991 that I started working on `TeX for the Impatient' and Dvips, Xdvi, Web2c, and the GNU fontutils (which I was also writing at the time) using different environment variables, and, even more importantly, having different bugs in their path searching became extremely painful. I also desperately wanted to implement subdirectory searching, since I couldn't stand putting everything in one big directory, and also couldn't stand having to explicitly specify `pandora', `cm' in a path. In the first incarnation, I just hacked separately on each program-- that was the original subdirectory searching code in both Xdvi and Dvips, though I think Paul Vojta has completely rewritten Xdvi's support by now. That is, I tried to go with the flow in each program, rather than changing the program's calling sequences to conform to common routines. Then, as bugs inevitably appeared, I found I was fixing the same thing in each of three (Web2c and fontutils were always sharing code, since I maintained those--there was no Dvipsk or Xdvik or Dviljk at this point). After a while, I finally started sharing source files. They weren't a library, though. I just kept things up to date with shell scripts. (I was developing on a 386 running ISC 2.2 at the time, and so didn't have symbolic links. An awful experience.) Things kept on like this for quite a while. The `ChangeLog's for Xdvik and Dvipsk record initial releases of those distributions in May and June 1992. I think it was because I was tired of the different configuration strategies of each program, not so much because of the path searching. (Autoconf was being developed by David MacKenzie and others, and I was adapting it to TeX and friends.) I starting to make it a separate library that other programs could link with on my birthday in April 1993, according to the ChangeLog. I don't remember exactly why I finally took the time to make it a separate library; I think it was a conversation with david zuhn that led to doing it. Just seemed like it was time. Dviljk got started in March 1994 after I bought a Laserjet 4. (Kpathsea work got suspended while Norm Walsh and I, with Gustaf Neumann's help, implemented a way for TeX to get at all those neat builtin LJ4 fonts ... such a treat to have something to typeset in besides Palatino!) At this point (October 1994), I've implemented just about all the path-searching features in Kpathsea that I ever intended to (and some I didn't intend ...). After the next stable release of Web2c, I figure I'll be able to stop development, and turn most of my attention back to making fonts for GNU. (Always assuming Microsoft hasn't completely obliterated Unix by then, or that software patents haven't stopped software development by anybody smaller than a company with a million-dollar-a-year legal budget. Which is actually what I think is likely to happen, but that's another story...)  File: kpathsea.info, Node: Installation, Next: Debugging, Prev: Introduction, Up: Top Installation ************ Here are the basic steps for configuration and installation: 1. Edit the file `make/paths.make' if you want to make changes to the installation directories or paths that will have effect across different runs of `configure'. Alternatively, override the Make variables on the command line when you run Make. Exception: to reliably change the top-level `prefix', you must give `configure' the option `-prefix=PREFIX', instead of changing the value in `paths.make'. 2. Edit `kpathsea/texmf.cnf.in' to change the local paths to match your local setup. *Note Default paths: (kpathsea)Default paths, for more details on changing the paths. A copy is in `kpathsea/INSTALL'. See `kpathsea/HIER' for an explanation of the default setup. If the paths do not match where the files actually are, the programs will probably start up Very, Very, Slowly, and/or not be able to find the fonts or other input files. 3. `sh configure' (in the top-level directory). This makes system-dependent `#define's' in `*/c-auto.h' (from the corresponding `c-auto.h.in') and creates a `Makefile' (from the corresponding `Makefile.in', by doing `@VAR@' and `ac_include' substitutions). Perhaps the most common desire is to compile with optimization instead of or as well as debugging. You can change the options passed to the compiler by changing `CFLAGS', either for `configure' or `make'. For example: prompt$ env CFLAGS="-g -O" configure prompt$ make or prompt$ configure prompt$ make CFLAGS="-g -O" *Note Running `configure' scripts: (autoconf)Invoking configure, for detailed `configure' options. (A copy is in `kpathsea/CONFIGURE'.) 4. `make' (still in the top-level directory). Barring configuration and compiler bugs, this will compile all the programs. *Note Common problems: (kpathsea)Common problems, for system-dependent problems (this section is also in `kpathsea/INSTALL'). This also creates the `texmf.cnf' and `paths.h' files that define the default search paths. 5. `make install'. This installs the library, header files, and documentation. Or `make install-data' to just install the architecture-independent files. Or `make install-exec' to just install the (binary) archive library file. Since I only distribute Kpathsea as part of another package, you will probably be doing the above in a top-level directory that contains a `Makefile', `kpathsea', and the other package. But you can do the installation in `kpathsea' itself, if you only want to install the library, not the other package. 6. The first time you install any manual in Info, you have to add a line (you choose where) to the `dir' file in your `$(infodir)' directory. A sample line to add is given near the top of the Texinfo source files (`kpathsea/kpathsea.texi' and `dvipsk/dvips.texi'). 7. `make distclean'. This removes all files created by the build. *Note Filename database::, for a description of an externally-generated database that can help speed searches. *Note Debugging::, for runtime debugging support that may help track down problems. Do not attempt to use any version of Kpathsea with any program except the version that the program came with, unless you are a glutton for punishment. * Menu: * Default paths:: Changing default installation directories and paths. * Common problems:: When things go wrong. * Shared library:: Making Kpathsea a shared library. * Reporting bugs:: Where and how to report bugs.  File: kpathsea.info, Node: Default paths, Next: Common problems, Up: Installation Default paths ============= To summarize the chain of events that go into defining the default paths: 1. `configure' creates a `Makefile' from each `Makefile.in'. 2. When Make runs in the `kpathsea' directory, it creates a file `texmf.sed' that substitutes the Make value of `$(var)' for a string `@var@'. The variables in question are the one that define the installation directories. 3. `texmf.sed' (and a little extra magic--see `kpathsea/Makefile') is applied to `texmf.cnf.in' to generate `texmf.cnf'. This is the file that will eventually be installed and used by the programs to look up programs. 4. The definitions in `texmf.cnf' are changed into the form of C `#define''s, producing `paths.h'. These values will be the compile-time defaults; they are not used unless no `texmf.cnf' file can be found at runtime. (That's a partial lie: the compile-time defaults are what extra `:''s in `texmf.cnf' expand into; but the paths as distributed have no extra `:''s, and there's no particular reason for them to.) The purpose of this elaborate sequence is to avoid having the same information in more than one place. If you change the installation directories or top-level prefix before running `configure', those changes will propagate through the whole sequence. If you change the default paths in `texmf.cnf.in', those changes are propagated to the compile-time defaults. Alternatively, you can ignore the whole mess and edit `texmf.cnf' after it is installed. Maybe even copying it into place beforehand so you can complete the installation, if TeX or Metafont is having trouble finding their input files. Unfortunately, editing `Makefile.in' *does not work* in one common case--changing the `prefix' or `exec_prefix' variables. For these, you must use the `-prefix' or `-exec-prefix' options to `configure'. *Note Running `configure' scripts: (autoconf)Running configure Scripts. (That's another partial lie: editing does work, as long as a program named `tex' is not in your `PATH'.) *Note TeX directory structure: TeX directory structure, for a description of some ways to arrange the TeX library files, and some features of the distributed paths that may not be obvious. The file `kpathsea/HIER' is a copy of that section. The Make definitions are all repeated in several `Makefile''s; but changing the top-level `Makefile' should suffice, as it passes down all the variable definitions, thus overriding the submakes. (The definitions are repeated so you can potentially run Make in the subdirectories.)  File: kpathsea.info, Node: Common problems, Next: Shared library, Prev: Default paths, Up: Installation Common problems =============== Some common problems with compilation, linking, or execution are described below. * Menu: * Unable to find files:: If your program can't find fonts or anything else. * Slow path searching:: If it takes forever to find anything. * XtInherit:: For XtInherit link problems on OSF/1 1.x. * wchar_t:: For wchar_t difficulties. * ShellWidgetClass:: For dynamic linking with Sun's openwin libraries. * Pointer combination warnings:: For old compilers that don't grok char *.  File: kpathsea.info, Node: Unable to find files, Next: Slow path searching, Up: Common problems Unable to find files -------------------- If a program complains it cannot find fonts (or other input files), any of several things might be wrong: * You don't have the fonts (or whatever) installed. Nothing will automatically generate TFM files or TeX and Metafont sources for you (by default). *Note Obtaining Web2c: (web2c)Obtaining Web2c. You can, however, configure TeX and Metafont to run a script to generate these input files, if you have (or write) such scripts. *Note `MakeTeX'... invocation: (web2c)MakeTeX... invocation. * You have (perhaps unknowingly) told Kpathsea to use search paths that don't reflect where the files actually are. One common cause is having environment variables set, thus overriding what you carefully set in `texmf.cnf'. *Note TeX environment variables: TeX environment variables. * Your files reside in a directory that is only pointed to via a symbolic link, in a leaf directory. Unfortunately, Kpathsea's subdirectory searching has a (congenital) deficiency: If a directory D being searched for subdirectories contains plain files and symbolic links to other directories, but no true subdirectories, D will be considered a leaf directory, i.e., the symbolic links will not be followed. *Note Subdirectory expansion::, for an explanation of why this happens. You can work around this problem by creating an empty dummy subdirectory in D. Then D will no longer be a leaf, and the symlinks will be followed. The directory immediately followed by the `//' in the path specification, however, is always searched for subdirectories, even if it is a leaf. This is since presumably you would not have asked for the directory to be searched for subdirectories if you didn't want it to be. * There is a bug in the library. *Note Reporting bugs::. In any case, you may find the debugging options helpful in determining precisely where the fonts (or whatever) are being looked for. See the program's documentation for its debugging options, and also *note Debugging::..  File: kpathsea.info, Node: Slow path searching, Next: XtInherit, Prev: Unable to find files, Up: Common problems Slow path searching ------------------- If your program takes an excessively long time to find fonts or other input files, but does eventually succeed, here are some possible culprits: * Most likely, you just have a lot of directories to search, and that takes a noticeable time. The solution is to create and maintain a separate `ls-R' file that lists all the files in your main TeX hierarchy. *Note Filename database::. (Kpathsea always uses `ls-R' if it's present; there's no need to recompile or reinstall any of the programs.) * Your recursively-searched directories (e.g., `/usr/local/lib/tex/fonts//'), contain a mixture of files and directories. This prevents Kpathsea from using a useful optimization (*note Subdirectory expansion::.). It is best to have only directories (and perhaps a `README') in the upper levels of the directory structure, and it's very important to have *only* files, and no subdirectories, in the directories where the dozens of TFM, PK, or whatever files reside. * Finally, one simple-to-fix (but unlikely) cause: If you recursively search `$HOME' or `~', and you are running as `root', you will search every directory on the system. This typically takes quite some time! In any case, you may find the debugging options helpful in determining precisely when the disk or network is being pounded. *Note Debugging::, and also see the program's documentation.  File: kpathsea.info, Node: XtInherit, Next: wchar_t, Prev: Slow path searching, Up: Common problems `XtInherit' ----------- On DEC OSF/1 1.x systems, the loader has a bug that manifests itself in the following error (all on one line, but for the sake of the paper width it's broken here): xdvik/xdvi: /sbin/loader: Fatal Error: search_for_undefineds: symbol _XtInherit should not have any relocation entry According to Michael Rickabaugh `': This is a bug fixed in DEC OSF/1 2.0. If you know how, installing `/sbin/loader' from a 2.0 system onto a 1.3 system will work. Make sure that `/usr' is *not* mounted when you do this. (If you forget about umounting `/usr', it is possible most of your filesystems will become corrupted.) Otherwise, I suggest getting a hold of a 2.0 CD and running `/usr/sbin/installupdate'. Alternatively, you may be able to use the freely available X11 libraries that come with the MIT distribution (on `ftp.x.org', for example). Linking statically, perhaps only with some of the X libraries, may also work. (if you find the definitive workaround, please let me know.)  File: kpathsea.info, Node: wchar_t, Next: ShellWidgetClass, Prev: XtInherit, Up: Common problems `wchar_t' --------- The upshot of all the following is that if you get error messages regarding `wchar_t', try defining `NO_FOIL_X_WCHAR_T' (for Web2c) or `FOIL_X_WCHAR_T' (for everything else). `wchar_t' has caused infinite trouble. None of my code ever uses `wchar_t'; all I want to do is include X header files and various system header files, possibly compiling with GCC. This seems an impossible task! The X11 header `' and GCC's `' have conflicting definitions for wchar_t. The particulars: `' from MIT X11R5 defines `wchar_t' if `X_WCHAR' is defined, which is defined if `X_NOT_STDC_ENV' is defined, and we define *that* if `STDC_HEADERS' is not defined (`configure' decides if STDC_HEADERS gets defined). But when compiling with gcc on SunOS 4.1.x, `STDC_HEADERS' is not defined (`string.h' doesn't declare the `mem'* functions), so we do get X's `wchar_t'--and we also get gcc's `wchar_t' from its `'. Conflict. On the other hand, SunOS 4.1.1 with some other X configurations actually needs GCC to define `wchar_t', and fails otherwise. My current theory is to define `wchar_t' to a nonsense symbol before the X include files are read; that way its definition (if any) will be ignored by other system include files. Going along with that, define `X_WCHAR' to tell X not to use `', that we've already included, but instead to make its own definition. But this is not the end of the story. The X11 include files distributed with DG/UX 5.4.2 for the Aviion have been modified to include `<_int_wchar_t.h>' if `X_WCHAR', so our `#define' will not have any typedef to change--but the uses of `wchar_t' in the X include files will be changed to reference this undefined symbol. So there's nothing to foil in this case. I don't know how to detect this automatically, so it's up to you to define `NO_FOIL_X_WCHAR_T' yourself.  File: kpathsea.info, Node: ShellWidgetClass, Next: Pointer combination warnings, Prev: wchar_t, Up: Common problems `ShellWidgetClass' ------------------ This section is adapted from question 47 from the `comp.sys.sun.admin' FAQ. If you are linking with Sun's OpenWindows libraries in SunOS 4.1.x, you may get undefined symbols `_get_wmShellWidgetClass' and `_get_applicationShellWidgetClass'. This problem does not arise with the standard MIT libraries under SunOS. The cause is bugs in the `Xmu' shared library as shipped from Sun. There are several fixes: * Get the Openwindows patches that apply to this problem. * Statically link the `Xmu' library into the executable. * Avoid using `Xmu' at all. For this last, if you are compiling Metafont, *note Online Metafont graphics: (Web2c)Online Metafont graphics.. If you are compiling Xdvi, see the `-DNOTOOL' option in `xdvik/INSTALL'. * Ignore the errors. The binary runs fine regardless. Here is the information for getting the two patches: Patch ID: 100512-02 Bug ID's: 1086793, 1086912, 1074766 Description: 4.1.x OpenWindows 3.0 `libXt' jumbo patch Patch ID: 100573-03 Bug ID: 1087332 Description: 4.1.x OpenWindows 3.0 undefined symbols when using shared `libXmu'. The way to statically link with `libXmu' depends on whether you are using a Sun compiler (e.g., `cc') or `gcc'. If the format, alter the `x_libs' make variable to include -Bstatic -lXmu -Bdynamic If you are using `gcc', include `-static' in `LDFLAGS'; this will link all libraries statically. If you want to link only `Xmu' statically and everything else dynamically, you have to do it by hand: run `gcc -v', grab the `ld' line, and add the `-B''s given above around `-lXmu'. The reason is that gcc moves all linker options to the front of the `ld' command line. So you can't specify different options for different libraries. When I reported this to the GCC maintainers, the reply was that they would happily merge in the changes, but they didn't want to take the time to do it themselves.  File: kpathsea.info, Node: Pointer combination warnings, Prev: ShellWidgetClass, Up: Common problems Pointer combination warnings ---------------------------- When compiling with old C compilers, you may get some warnings about "illegal pointer combinations". These are spurious; just ignore them. I decline to clutter up the source with casts to get rid of them. In general, if you have trouble with a system C compiler, I advise trying the GNU C compiler. (And vice versa, unfortunately; but in that case I also recommend reporting a bug to the GCC bug list.)  File: kpathsea.info, Node: Shared library, Next: Reporting bugs, Prev: Common problems, Up: Installation Shared library ============== You can compile Kpathsea as a shared library. The advantage in doing this is that the different executables can then share the code, decreasing memory usage. (The other advantage in general of shared libraries is that it's possible to update the library and programs independently. But since the Kpathsea interface is not and can not be frozen, that doesn't apply here.) Under Solaris, use `-K pic -xstrconst' if you compile with a Sun compiler, `-fpic' if you use GCC. Also add `-L$(LIBDIR) -R$(LIBDIR)' to `LDFLAGS' when you link the binaries, so that the library can be found, and users do not have set `LD_LIBRARY_PATH'. (If you know how to make Kpathsea shared on other systems, please send a message to the bug address in the next section.)  File: kpathsea.info, Node: Reporting bugs, Prev: Shared library, Up: Installation Reporting bugs ============== If you encounter problems, please report them to `tex-k@cs.umb.edu'. Include the version number of the library, the system you are using, and enough information to reproduce the bug in your report. To get on this mailing list yourself, email `tex-k-request@cs.umb.edu' with a message whose body contains a line subscribe YOU@YOUR.PREFERRED.ADDRESS To avoid wasted effort and time (both mine and yours), I strongly advise applying the principles given in the GNU C manual (*note Reporting Bugs: (gcc)Bugs.) to your bug reports. Please also report bugs in this documentation--not only factual errors, but unclear explanations, typos, wrong fonts, ...  File: kpathsea.info, Node: Debugging, Next: Path searching, Prev: Installation, Up: Top Debugging ********* Kpathsea provides a number of runtime debugging options, detailed below by their names (and corresponding numeric values). You can set these with some runtime argument (e.g., `-d') to the program; in that case, you should use the numeric values described in the program's documentation (which, except for Dviljk, are different from those below). You can also set the environment variable `KPATHSEA_DEBUG'. In this case, you should use the numbers below. Also use the numbers below if you run the program under a debugger and set the the variable `kpathsea_debug' yourself. In any case, you can *not* use the *names* below; you must always use somebody's numbers. (Sorry.) And to set more than option, just sum the corresponding numbers. `KPSE_DEBUG_STAT' (1). Reports `stat'(2) calls. This is useful for verifying that your directory structure is not forcing Kpathsea to do many additional file tests (*note Slow path searching::. and *note Subdirectory expansion::.). If you are using an up-to-date `ls-R' database (*note Filename database::.), this should produce no output unless a nonexistent file is searched for. `KPSE_DEBUG_HASH' (2). Reports lookups in all hash tables, including `ls-R' (*note Filename database::.), font aliases (*note Fontmap::.), and config file values (*note Config files::.). Useful when expected values are not being found, e.g.., file searches are looking at the disk instead of using `ls-R'. `KPSE_DEBUG_FOPEN' (4). Reports file openings and closings. Especially useful when your system's file table is full, for seeing if some files have been opened but never closed. In case you want to set breakpoints: this works by redefining `fopen' (`fclose') to be `kpse_fopen_trace' (`kpse_fclose_trace'). `KPSE_DEBUG_PATHS' (8). Reports general path information for each file type Kpathsea is asked to search. This is useful when you are trying to track down how a particular path got defined--from `texmf.cnf', `config.ps', the compile-time default, an environment variable, etc. This is the contents of a structure defined in `tex-file.h'. `KPSE_DEBUG_EXPAND' (16). Reports the directory list corresponding to each path element Kpathsea searches in. This is only relevant when Kpathsea is searching the disk, since `ls-R' searches don't look through directory lists in this way (they go straight to the file using the hash table). `KPSE_DEBUG_SEARCH' (32). Reports on each file search Kpathsea attempts: the name of the file searched for, the path searched in, whether or not the file must exist (when drivers search for `cmr10.vf', it need not exist), and whether or not we are collecting all occurrences of the file in the path (as with, e.g., `texmf.cnf' and `texfonts.map'), or just the first (as with most lookups). This can help you correlate what Kpathsea is doing with what is in your input file. Debugging output from Kpathsea is always written to standard error, and begins with `kdebug:'. (Except for hash table buckets, which just start with the number.) * Menu: * Logging:: Recording successful searches.  File: kpathsea.info, Node: Logging, Up: Debugging Logging ======= Kpathsea can record the time and filename found for each successful search. This may be useful in finding good candidates for deletion when your disk is full. To do this, define the environment or config file variable `TEXMFLOG'. The value is the name of the file to append the information to. The file is created if it doesn't exist. Each successful search turns into one line in the log file, with two words separated by a space. The first word is the time of the search, as the integer number of seconds since "the epoch", i.e., UTC midnight 1 January 1970 (more precisely, the result of the `time' system call). The second word is the filename. For example, after `setenv TEXMFLOG /tmp/log', running Dvips on `story.dvi' appends the following lines: 774455887 /usr/local/lib/texmf/dvips/config.ps 774455887 /usr/local/lib/texmf/dvips/psfonts.map 774455888 /usr/local/lib/texmf/dvips/texc.pro 774455888 /usr/local/lib/texmf/fonts/public/cm/pk/ljfour/cmbx10.600pk 774455889 /usr/local/lib/texmf/fonts/public/cm/pk/ljfour/cmsl10.600pk 774455889 /usr/local/lib/texmf/fonts/public/cm/pk/ljfour/cmr10.600pk 774455889 /usr/local/lib/texmf/dvips/texc.pro Only filenames that are absolute are recorded, to preserve some semblance of privacy.  File: kpathsea.info, Node: Path searching, Next: TeX searching, Prev: Debugging, Up: Top Path searching ************** This chapter describes the generic path searching mechanism Kpathsea provides. For information about searching for particular file types (e.g., TeX fonts), see the next chapter. * Menu: * Searching overview:: Basic scheme for searching. * Path sources:: Constructing the search path. * Default expansion:: a: or :a or a::b expands to a default. * Variable expansion:: $foo and ${foo} expand to environment values. * Tilde expansion:: ~ and ~user expand to home directories. * Subdirectory expansion:: a// and a//b recursively expand to subdirs. * Filename database:: Using an externally-built list to search.  File: kpathsea.info, Node: Searching overview, Next: Path sources, Up: Path searching Searching overview ================== A "search path" is a colon-separated list of path elements, which are directory names with some extra frills. A search path can come from (a combination of) many sources; see below. To look up a file `foo' along a path `.:/dir', Kpathsea checks each element of the path in turn: first `./foo', then `/dir/foo', (typically) returning the first one that exists. The "colon" and "slash" mentioned here aren't necessarily `:' and `/' on non-Unix systems. Kpathsea tries to adapt to other operating systems' conventions. To check a path element E, Kpathsea first sees if a prebuilt database (see below) applies to E, i.e., if the database is in a directory that is a prefix of E. If so, the path specification is matched against the contents of the database. If the database does not exist, or does not apply to this path element, contains no matches, the filesystem is searched. Kpathsea constructs the list of directories that correspond to this path element, and then checks in them for the file being searched for. (To help speed future lookups of files in the same directory, the directory in which a file is found is floated to the top of the directory list.) Each path element is checked in turn: first the database, then the disk. Once a match is found, the searching stops and the result is returned. This avoids possibly-expensive processing of path specifications that are never needed on a particular run. Although the simplest and most common path element is a directory name, Kpathsea supports additional features in search paths: layers of default values, environment variable names, config file values, users' home directories, and recursive subdirectory searching. Thus, we say that Kpathsea "expands" a path element, meaning getting rid of all the magic specifications and getting down to the basic directory name or names. This process is described in the sections below. It happens in the same order as the sections. Exception to the above: If the filename being searched for is absolute or explicitly relative, i.e., starts with `/' or `./' or `../', Kpathsea simply checks if that file exists; it is not looked for along any paths.  File: kpathsea.info, Node: Path sources, Next: Default expansion, Prev: Searching overview, Up: Path searching Path sources ============ A search path can come from many sources. In priority order (meaning Kpathsea will use whichever it finds first): 1. A user-set environment variable, e.g., `TEXINPUTS'. 2. A program-specific configuration file, e.g., an `S /a:/b' line in Dvips' `config.ps'. 3. A line in a Kpathsea configuration file `texmf.cnf', e.g., `TEXINPUTS=/c:/d'. See section below. 4. The compile-time default (specified in `kpathsea/paths.h'). In any case, once the path specification to use is determined, its evaluation is independent of its source. These sources may also be combined via default expansion. See the next section. You can see each of these values for a given search path by using the debugging options of Kpathsea or your program. *Note Debugging::. * Menu: * Config files:: Kpathsea's runtime config files (texmf.cnf).  File: kpathsea.info, Node: Config files, Up: Path sources Config files ------------ As mentioned above, Kpathsea reads "runtime configuration files" named `texmf.cnf' for search path definitions. The path used to search for them is constructed in the usual way, as described above (except that configuration files cannot be used to define the path, naturally; also, an `ls-R' database is not used to search for them, for technical reasons). The environment variable used is `TEXMFCNF'. Kpathsea reads *all* `texmf.cnf' files in the search path, not just the first one found; it uses the first definition of each variable encountered. Thus, with the (default) search path of `.:$TEXMF', values from `./texmf.cnf' override those from `$TEXMF/texmf.cnf'. Here is the format for `texmf.cnf' files: * Anything after a `%' or `#' is ignored; this is for comments. * Blank lines are ignored. * Each remaining nonblank line must look like VARIABLE [. PROGNAME] [=] VALUE where the `=' and surrounding whitespace is optional. * The VARIABLE name may contain any characters except whitespace, `=', or `.' characters, but sticking to `A-Za-z_' is safest. * If the `.PROGNAME' is present, the definition only applies if the program that is running is named (i.e., the last component of `argv[0]') PROGNAME. This allows (for example) different flavors of TeX to have different search paths. * The VALUE may contain any characters except whitespace, `%', and `@'. (These restrictions are necessary because of the various `sed' and other processing done on `texmf.cnf' at build time.) * All definitions are read before anything is expanded, so you can use variables before they are defined (like `make', unlike most everything else). Here is the fragment from the distributed file illustrating most of these points: % TeX input files -- i.e., anything to be found by \input or \openin [...] latex209_inputs = .:$TEXMF/tex/latex209//:$TEXMF/tex// latex2e_inputs = .:$TEXMF/tex/latex2e//:$TEXMF/tex// TEXINPUTS = .:$TEXMF/tex// TEXINPUTS.latex209 = $latex209_inputs TEXINPUTS.latex2e = $latex2e_inputs TEXINPUTS.latex = $latex2e_inputs Although this format has obvious similarities to Bourne shell scripts--change the comment character to `#', disallow spaces around the `=', and get rid of the `.PROGRAM' convention, and it could be run through the shell. But there seemed little advantage to doing this, since all the information would have to passed back (with `echo''s, presumably) to Kpathsea and parsed there anyway, since the `sh' process couldn't affect its parent's environment. The implementation of all this is in `kpathsea/cnf.c'.  File: kpathsea.info, Node: Default expansion, Next: Variable expansion, Prev: Path sources, Up: Path searching Default expansion ================= If the highest-priority search path (in the list in the previous section) contains an "extra colon" (i.e., leading, trailing, or doubled), Kpathsea inserts the next-highest-priority search path that is set at that point. If that search path has an extra colon, the same happens with the next-highest. (An extra colon in the compile-time default value has unpredictable results, and may cause the program to crash, so installers beware.) For example, given setenv TEXINPUTS /home/karl: and a `TEXINPUTS' value from `texmf.cnf' of .:$TEXMF//tex then the final value used for searching will be: /home/karl:.:$TEXMF//tex You can trace this by debugging "paths" (*note Debugging::.). Minor technical point: Since it would be useless to insert the default value in more than one place, Kpathsea changes only one extra `:' and leaves any others in place (where they will eventually be effectively equivalent to `.', i.e., the current directory). It checks first for a leading `:', then a trailing `:', then a doubled `:'.  File: kpathsea.info, Node: Variable expansion, Next: Tilde expansion, Prev: Default expansion, Up: Path searching Variable expansion ================== `$foo' or `${foo}' in a path element is replaced by (1) the value of an environment variable `foo' (if it is set); (2) the value of `foo' from `texmf.cnf' (if any such exists); (3) the empty string. If the character after the `$' is alphanumeric or `_', the variable name consists of all consecutive such characters. If the character after the `$' is a `{', the variable name consists of everything up to the next `}' (braces are not balanced!). Otherwise, Kpathsea gives a warning and ignores the `$' and its following character. Remember to quote the `$''s and braces as necessary for your shell. *Shell* variable values cannot be seen by Kpathsea. For example, given setenv TEXMF /home/tex setenv TEXINPUTS .:$TEXMF:${TEXMF}new the final `TEXINPUTS' path is the three directories: .:/home/tex:/home/texnew You can trace this by debugging "paths" (*note Debugging::.).  File: kpathsea.info, Node: Tilde expansion, Next: Subdirectory expansion, Prev: Variable expansion, Up: Path searching Tilde expansion =============== A leading `~' or `~USER' in a path element is replaced by the current or USER's home directory, respectively. If USER is invalid, or the home directory cannot be determined, Kpathsea uses `.' instead. For example, setenv TEXINPUTS ~/mymacros: will prepend a directory `mymacros' in your home directory to the default path.  File: kpathsea.info, Node: Subdirectory expansion, Next: Filename database, Prev: Tilde expansion, Up: Path searching Subdirectory expansion ====================== A `//' in a path element following a directory D is replaced by all subdirectories of D: first those subdirectories directly under D, then the subsubdirectories under those, and so on. At each level, the order in which the directories are searched is unspecified. (It's "directory order", and definitely not alphabetical.) If you specify any filename components after the `//', only subdirectories which contain those components are included. For example, `/a//b' would expand into directories `/a/1/b', `/a/2/b', `/a/1/1/b', and so on, but not `/a/b/c' or `/a/1'. I should mention one related implementation trick, which I stole from GNU find. Matthew Farwell `' suggested it, and David MacKenzie `' implemented it (as far as I know). The trick is that in every real Unix implementation (as opposed to the POSIX specification), a directory which contains no subdirectories will have exactly two links (namely, one for `.' and one for `..'). That is to say, the `st_nlink' field in the `stat' structure will be two. Thus, we don't have to stat everything in the bottom-level (leaf) directories--we can just check `st_nlink', notice it's two, and do no more work. But if you have a directory that contains *one* subdirectory and five hundred files, `st_nlink' will be 3, and Kpathsea has to stat every one of those 501 entries. Therein lies slowness. You can disable the trick by undefining `UNIX_ST_LINK' in `kpathsea/config.h'. (It is undefined by default except under Unix.) Unfortunately, in some cases files in leaf directories are `stat''d: if the path specification is, say, `$TEXMF/fonts//pk//', then files in a subdirectory `.../pk', even if it is a leaf, are checked. The reason cannot be explained without reference to the implementation, so read `kpathsea/elt-dirs.c' (search for `may descend') if you are curious. (And if you can find a way to *solve* the problem, please let me know.)  File: kpathsea.info, Node: Filename database, Prev: Subdirectory expansion, Up: Path searching Filename database (`ls-R') ========================== Kpathsea goes to some lengths to minimize disk accesses for searches (*note Subdirectory expansion::.). Nevertheless, at installations with enough directories, doing a linear search of each possible directory for a given file can take an excessively long time ("excessive" depending on the speed of the disk, whether it's NFS-mounted, how patient you are, etc.). In practice, the union of font directories from the Dvips(k) and Dviljk distributions is large enough for searching to be noticeably slow on typical machines these days. Therefore, Kpathsea can use an externally-built "database" that maps files to directories, thus avoiding the need to exhaustively search the disk. By fiat, you must name the file `ls-R', and put it at the root of the TeX installation hierarchy (`$TEXMF' by default). Kpathsea does variable expansion on the `$TEXMF', naturally, so you can use different `ls-R''s for different trees, if you are testing new ones. However, one and only one `ls-R' is read; it is not searched for along any paths. You can build `ls-R' with the command ls -R /YOUR/ROOT/DIR >ls-R if your `ls' produces the right output format (see the section below). GNU `ls', for example, outputs in this format. It is probably best to do this via `cron', so changes in the installed files will be automatically reflected (albeit with some delay) in the database. If your system uses symbolic links, the command `ls -LR' will be more reliable than plain `ls -R'. The former follows the symbolic links to the real files, which is what Kpathsea needs. Kpathsea warns you if it finds an `ls-R' file, but the file does not contain any usable entries. The usual culprit is using just `ls -R' to generate the `ls-R' file instead of `ls -R /YOUR/DIR'. Kpathsea looks for lines starting with `/', to improve reliability with unusual filenames (specifically, those ending with a `:'). Because the database may be out-of-date for a particular run (e.g., if a font was just built with `MakeTeXPK'), if a file is not found in the database, by default Kpathsea goes ahead and searches the disk. If a particular path element begins with `!!', however, *only* the database will be searched for that element, never the disk. If the database does not exist, nothing will be searched. Because this can greatly surprise users ("I see the font `foo.tfm' when I do an `ls'; why can't Dvips find it?"), I do not recommend using this feature. * Menu: * Database format:: Syntax details of the database file.  File: kpathsea.info, Node: Database format, Up: Filename database Database format --------------- The "database" read by Kpathsea is a line-oriented file of plain text. The format is that generated by GNU (and perhaps other) `ls' programs given the `-R' option, as follows. * Blank lines are ignored. * If a line begins with `/' and ends with a colon, it's the name of a directory. * All other lines name entries in the most recently seen directory. `/''s in such lines will produce possibly-strange results. * Files with no preceding directory line are ignored. For example, here's the first few lines of `ls-R' on my system: bibtex dvips fonts ini ls-R mf tex /usr/local/lib/texmf/bibtex: bib bst doc /usr/local/lib/texmf/bibtex/bib: asi.bib bibshare btxdoc.bib On my system, `ls-R' is about 30K bytes.  File: kpathsea.info, Node: TeX searching, Next: TeX directory structure, Prev: Path searching, Up: Top TeX searching ************* Although the basic features in Kpathsea can be used for any type of path searching, it came about (like all libraries) with a specific application in mind: I wrote Kpathsea specifically for TeX system programs. I had been struggling with the programs I was using (Dvips, Xdvi, and TeX itself) having slightly different notions of how to specify paths; and debugging was painful, since no code was shared. Therefore, Kpathsea provides some TeX-specific features. Indeed, many of the supposedly generic path searching features were provided because they seemed useful in that conTeXt (font lookup, particularly). * Menu: * Envvars: TeX environment variables. Overriding compiled-in paths. * Glyph lookup:: Searching for bitmap fonts.  File: kpathsea.info, Node: TeX environment variables, Next: Glyph lookup, Up: TeX searching TeX environment variables ========================= Kpathsea defines a sequence of environment variables to search for each file type it supports. This makes it easy for different programs to check the same environment variables, in the same order. The following table lists the environment variables searched for each file type in the order they are searched (and a brief description of the file type). That is, only if the first variable is unset is the second variable checked, and so on. If none are set, various other things are checked; *note Path sources::.. `.base' (Metafont memory dump) `MFBASES' `.bib' (BibTeX bibliography source) `BIBINPUTS' `.bst' (BibTeX style file) `BSTINPUTS', `TEXINPUTS' `.cnf' (Kpathsea runtime configuration files) `TEXMFCNF' `.eps' (Encapsulated PostScript figures) `TEXPICTS', `TEXINPUTS' `.fmt' (TeX memory dump) `TEXFORMATS' `gf' (generic font bitmap) `PROGRAMFONTS', `GFFONTS', `GLYPHFONTS', `TEXFONTS' `.mf' (Metafont source) `MFINPUTS' `mf.pool' (Metafont program strings) `MFPOOL' `.pict' (Other kinds of figures) Same as `.eps'. `pk' (packed bitmap font) `PROGRAMFONTS', `PKFONTS', `TEXPKS', `GLYPHFONTS', `TEXFONTS' `.tex' (TeX source) `TEXINPUTS' `tex.pool' (TeX program strings) `TEXPOOL' `.tfm' (TeX font metrics) `TFMFONTS', `TEXFONTS' `.vf' (virtual font) `VFFONTS', `TEXFONTS' For the font variables, the intent is that: 1. `TEXFONTS' is the default for everything. 2. `GLYPHFONTS' is the default for bitmap (or, more precisely, non-metric) files. 3. Each format has its own variable. 4. Each program can and should have its own font override path as well; e.g., `DVIPSFONTS' for Dvipsk. Again, this is for bitmaps, not metrics. If these environment variables are set, the corresponding `texmf.cnf' definition won't be looked at (unless, as usual, the environment variable has an extra `:'). *Note Default expansion::.  File: kpathsea.info, Node: Glyph lookup, Prev: TeX environment variables, Up: TeX searching Glyph lookup ============ Kpathsea provides a routine (`kpse_find_glyph_format' in `kpathsea/tex-glyph.c') which searches for a bitmap font in GF or PK format (or either) given a font name (e.g., `cmr10') and a resolution (e.g., 300). The search is based solely on filenames, not file contents--if a PK file is named `cmr10.300gf', it will be found as a GF file. Here is an outline of the search strategy (details in the sections below) for a file NAME at resolution DPI. The search stops at the first successful lookup. 1. Look for an existing file NAME.DPI in the specified format(s). 2. If NAME is an alias for a file F in the fontmap file `texfonts.map', look for F.DPI. 3. Run an external script (typically named `MakeTeXPK') to generate the font. 4. Look for FALLBACK.DPI, where FALLBACK is some last-resort font (typically `cmr10'). * Menu: * Basic glyph lookup:: Features common to all glyph lookups. * Fontmap:: Aliases for fonts. * MakeTeX... scripts:: Creating files on the fly. * Fallback font:: Resolutions and fonts of last resort.