LaTeXTools: a new, customizable build system

Edited 3/13/14: updated name of configuration file. Also, this code is now in the master branch! More details in a later post.

One of the requests I receive most frequently, especially from LaTeX power user, is the ability to customize the build process. In principle, this is possible. However, it requires modifying the LaTeX.sublime-build file. This is inconvenient for a number of reasons. First, the file must be copied to the User directory, or it will be clobbered by subsequent updates of the plugin. Second, the file also contains internal settings that cannot and must not be customized. Third, the Sublime Text build system is really designed to launch a single, command-line program, or else a “make”-like utility. Indeed, LaTeXTools uses it to invoke either texify or latexmk, which are essentially TeX-specific “make” utilities. In principle, one can write one’s own custom build script and use that in lieu of the default build command; however, most users will not want to do that. Also, one is somewhat limited by the fact that external scripts do not “know” about Sublime Text, and hence cannot meaningfully communicate with it.

Enter the new “master builder” system; you will find it in the mbuilder branch on GitHub. Here’s what’s new:

1. All user settings are now in the LaTeXTools.sublime-settings file. This includes the path to tex & friends, and all the settings discussed below. With the new code, LaTeX.sublime-build becomes effectively just an internal file that users need not and, indeed, must not modify. NOTE: if you want to play with the mbuilder branch, make sure to delete any stray LaTeX.sublime-build files you may have in the User directory. Also, as for any Sublime Text settings file, LaTeXTools.sublime-settings should be placed in User directory and customized there.
2. The build system is now split into two components. One is the “master builder,” which takes care of handling the Cmd/Ctrl-B command and manages the (delicate) threading infrastructure required to run external commands such as pdflatex or latexmk; this is the code in the file makePDF.py. The other is the actual “builder” or “build engine,”  which examines the tex root file, decides how to process it, and tells the master builder which external commands to run. Indeed, there are currently two build engines (called “traditional” and “simple”), with a third (“script”) coming soon: you will find the code in the builders directory.
3. The “traditional” build engine does exactly what the current code in the master branch does: it calls latexmk or texify (and also allows for the selection of a tex engine). The “simple” build engine, on the other hand, requires no external tools, and can be useful, for instance, if you don’t want to install latexmk on OS X or Linux. It runs pdflatex, then bibtex if needed, then pdflatex again, as needed, until references / citations are either resolved or just missing. Thus, the “simple” engine is an example of a (simple!) “make” tool for tex & friends that runs entirely in Python, within Sublime Text. I hope one day to write (or, better yet, receive as a user contribution…) an actual, robust replacement for texify / latexmk. For now, the “simple” engine serves as an example to aspiring contributors… Finally, the “script” engine, when ready, will simply invoke a user-specified external command.
4. All parameters of the build engines are configurable in LaTeXTools.sublime-settings. Again, you can forget about LaTeX.build-settings. In particular, the “builder” option selects your preferred build engine. When the “script” engine is ready, this is where you will specify which external script to use, and whether to set any environment variables, for instance.
5. Techie note: the beauty of this system is that build engines can access the Sublime Text API, as well as the whole Python library, but they do not need to know anything about threading. Instead, builders use the powerful Python mechanism of generator functions. Each time a build engine wants to run an external command (pdflatex etc.), it uses the Python yield command to send it to the master builder. The master builder runs the external command, captures its output, and returns it to the build engine.

The code is still beta quality; I’m running it daily myself, but I’m interested in feedback from adventurous users. When I’m reasonably confident that things are working, I’ll merge it into the master branch.