Directory support/tenkana
tenkana, a spider that jumps from the checkpoints of your TeX document
----------------------------------------------------------------------
/\ ______ /\
/ \ / . . \ / \
_/\___/ \__|o(O)(O)o|__/ \___/\_
/ \ __ | '' | __ / \
/ \/ \ \______/ / \/ \
tenkana is free software that makes repeated LaTeX builds faster. Its command
is `tenkana'. It needs no root access and changes neither the TeX engine nor
your document.
While TeX runs, tenkana keeps frozen copies ("snapshots") of the running
engine: one just before \begin{document} and one before each \include.
After an edit, tenkana continues from the last snapshot before the change, so
the preamble, its packages and fonts, and all unchanged chapters before
the edit are not processed again.
You give tenkana the TeX command you would type without it, as with env or
time; tenkana's own options come before it:
tenkana lualatex thesis draft build (default): fast,
references not updated
tenkana --final lualatex thesis final build: references, citations
and table of contents up to date
tenkana -v pdflatex thesis also show tenkana's details
(passes, snapshots, memory)
tenkana pdflatex -synctex=1 thesis TeX's options go to TeX unchanged
tenkana -q thesis short for `tenkana lualatex thesis'
`man tenkana' has the full manual.
Contents
--------
1. What you see
2. Installation
3. How it works
4. Snapshots on disk
5. Measured results
6. Building without snapshots
7. Options and environment
8. Limitations
9. Files in this package
10. Copyright and license
1. What you see
---------------
TeX's own terminal output, followed by tenkana's messages:
Output written on talk.pdf (14 pages, 74150 bytes).
Transcript written on talk.log.
tenkana: jumped from ~5.6s for a plain run to 1.8s (68% less time).
Each message is one line and starts with `tenkana:', as with other Unix
commands; error messages go to stderr. --silent (or TENKANA_SILENT) leaves out
all but error messages. The comparison uses the time of the last full run of
the document. When a build saves no time, tenkana says so and gives the
reason. All texts live in src/tenkana/messages.py; tools/catalogue.py lists
every message tenkana can print, with examples, for review.
tenkana runs your command as you typed it, in the current folder, and adds
only -recorder right after the command name: it needs the list of files
TeX reads, which TeX writes to <job>.fls. (XeTeX also gets -no-pdf, see
section 3.) TeX's questions, after an error or from \typein, appear
on your terminal and take your answers, and Ctrl-C reaches TeX, which
answers with its "Interruption" prompt; this holds for resumed builds too.
On a resumed build, tenkana first prints what TeX printed up to the snapshot
(saved with it), then the live output, so the terminal transcript is the
same as a plain run's with the same options and -recorder. So is the
.log file, except for two memory-statistics lines at the very end (tenkana
defines one extra macro). Errors, warnings and SyncTeX refer to your
real file names and line numbers.
2. Installation
---------------
tenkana itself needs only Python 3.9 or newer. It installs and runs without
TeX, but to turn a document into a PDF it calls a TeX engine (lualatex,
pdflatex, xelatex, ...), so for that you need a TeX distribution such as
TeX Live, with LaTeX 2020-10 or newer.
For snapshots, tenkana also needs:
* GNU/Linux (tenkana uses LD_PRELOAD and /proc; it is not tested
elsewhere);
* a C compiler (cc, gcc or clang) for the snapshot library, which tenkana
compiles from src/tenkana/tenkanasnap.c on first use and caches in
~/.cache/tenkana/ (in /tmp/tenkana-<uid>/ if that path contains a space or
a colon, which LD_PRELOAD cannot handle);
* an x86-64 machine.
Without a C compiler or on another processor, tenkana builds with plain runs
(section 6).
To install tenkana from a copy of the git repository or of the CTAN package
(no root access needed):
./install.sh installs ~/.local/bin/tenkana and the man
page (~/.local/share/man/man1/tenkana.1)
./install.sh --system installs into /usr/local (uses sudo)
./install.sh --prefix DIR installs into DIR/bin and DIR/share/man/man1
(uses sudo if DIR is not yours)
./install.sh --uninstall removes tenkana again; with --prefix DIR,
from DIR
If ~/.local/bin is not on your PATH, the installer says what to add.
Building. The `tenkana' command is a single executable file, tenkana.pyz (a
Python zip application), which ./install.sh makes with ./build.sh. The
package holds only sources; tenkana.pyz is always built from them. To build
it without installing, for example to package tenkana for a distribution:
./build.sh writes tenkana.pyz next to build.sh
./build.sh PATH writes it to PATH instead (a file, or a
folder to put tenkana.pyz in)
./build.sh clean removes build/ and tenkana.pyz
build.sh works in the folder build/, which it empties first, and copies the
file out only after checking that it runs and reports the right version. The
file is the same byte for byte whatever Python 3.9 or newer builds it. TeX
Live, for example, keeps it as texmf-dist/scripts/tenkana/tenkana.pyz:
./build.sh /path/to/Master/texmf-dist/scripts/tenkana/tenkana.pyz
The man page needs no building: it is tenkana.1, as it is. build/ and
tenkana.pyz are not kept in git or in the CTAN package.
3. How it works
---------------
The core idea of process checkpointing is to capture a running process and
later continue from that exact point. Tools such as CRIU do this from
outside the process, which needs root or special privileges. tenkana lets the
engine copy itself instead, with fork(), which any program may do.
* A small library, src/tenkana/tenkanasnap.c, is loaded into the unmodified
engine with LD_PRELOAD. The engine is started with your command, as
in section 1 (`lualatex -recorder talk').
* Pause points. When the engine opens your main file, the library hands
it the same text with two hooks added at the start of line 1, so line
numbers do not change. TeX still prints and records your file's real
name. The hooks make TeX open a special file just before
\begin{document} and before every \include that TeX actually runs
(not those skipped by \includeonly or a false conditional). When the
library sees that file opened, it notes the position of every open
file and forks:
- the copy freezes and becomes a snapshot. It waits for commands
and shows up in ps as `tenkana-snap-<n>'. It marks itself as the first
process the kernel's out-of-memory killer should stop, so if
memory runs out, the kernel ends snapshots before your other
programs (other memory managers, such as systemd-oomd or
earlyoom, may choose differently);
- the original reports to tenkana, waits while tenkana saves copies of
the output files (.pdf, .aux, .log, ...), and carries on typesetting.
* Resuming. On the next run, tenkana puts the output files back as they
were at the snapshot and tells that frozen copy to fork again. The
new copy restores every file position and continues typesetting from
the pause point; for the main file it also discards text that had been
read ahead, so an edited body is read fresh. It reads its terminal
input from the `tenkana' command that resumed it and gets back the
engine's own signal handlers, so TeX can ask questions and Ctrl-C
works as without tenkana. The frozen copy stays frozen (holding no
terminal) and can be resumed again and again.
* Memory. A snapshot holds what the engine holds at that point, mostly
fonts and packages. fork() is copy-on-write, so further snapshots of
the same run only cost the memory pages changed in between: all 13
snapshots of the 220-page test book took 330 to 400MB in our tests.
`tenkana -v' shows the figure for your document.
* Lifetime. Snapshots are ordinary processes, so they survive between
runs of tenkana and the closing of the terminal. They end:
- at a reboot, like every process;
- at a logout only if the system ends a user's processes then (with
systemd, when KillUserProcesses is on in logind.conf; distributions
differ);
- 8 hours after the document's last build (TENKANA_TTL; any build
counts, even one that resumes no snapshot);
- with `tenkana --clean', or within half a minute after tenkana's cache
folder for the document is deleted;
- if the kernel runs out of memory (see above).
tenkana does not watch for any of this. At the next build it finds that
the snapshots are gone, and restores the preamble snapshot from disk
(section 4) or makes a full run.
* Which engine. tenkana finds it from the program your command finally
runs: pdflatex and pdflatex-dev run pdfTeX, lualatex runs LuaHBTeX,
xelatex runs XeTeX. Any command that runs pdfTeX, LuaTeX (also
LuaHBTeX, LuaJITTeX) or XeTeX works.
* Which file. TeX's first line is what follows the options: usually the
file name, as in `lualatex thesis'. It may also be TeX code, as in
`pdflatex '\nonstopmode\input paper.ltx''; tenkana then takes the file of
the first \input as the main file. Without an \input, tenkana runs the
command as it is, without snapshots, and says so. The file is looked
for as TeX does: in the current folder, then on TeX's search path (as
kpsewhich reports it), so `tenkana pdflatex sample2e' works. If TeX
would not find the file either, tenkana runs the command as it is and
TeX reports the problem.
* Where tenkana keeps its data. Everything goes to ~/.cache/tenkana/
($XDG_CACHE_HOME/tenkana/ if that is set), one folder per document,
readable only by you. Nothing is written next to the document except
TeX's own output (including <job>.fls), so a folder synchronised
between computers (Nextcloud, Syncthing, ...) carries no snapshots;
each computer keeps its own. One build of a document runs at a time:
a second one (from an editor that builds on every save, say) waits
for the first.
* Draft and final builds. LaTeX reads the previous run's .aux (and
.toc, ...) at \begin{document} and writes new ones during the run. A
normal build always does one pass, so a new or moved label shows as
?? or with an old number until you run `tenkana --final'. That reruns
from the preamble snapshot (taken before the .aux is read) until the
.aux, .toc and similar files stop changing, like latexmk (at most 4
passes). In drafts, references keep the values read by the last
build that started at \begin{document} (a full run, a final build, or
an edit that resumed from the preamble snapshot); a draft that resumes
from a chapter snapshot uses what that snapshot read. The text and
page layout are always current.
* Change tracking. Every file the engine reads is recorded (-recorder)
and assigned to the part of the run that first read it: the preamble,
the main file's body, or the stretch after one of the pauses. The
parts follow what TeX actually did, so \includeonly, conditionals and
\include inside an \input file are handled correctly. A change to a
file read in the preamble needs a full run; any change to the main
file's body resumes from the preamble snapshot; any other change
resumes from the snapshot just before the part that first read the
file. The library also records every file TeX looked for but did not
find in the document's folder, so a new chapter, image, .bbl (after
biber) or .ind (after makeindex) is noticed too. Files that other
programs write for TeX are also watched by name: .bbl, .ind, .gls,
.acr, .nls, .glstex, and glossary types declared in the .aux. More
can be added with TENKANA_WATCH.
* Edits during a build. A file saved while a build is running may or
may not have been read by TeX already. tenkana records each input as it
was when the build started, so the next build takes such an edit as a
change and none is lost.
* Errors. Snapshots taken before a LaTeX error stay valid, so after
you fix the error tenkana resumes just before the chapter that failed.
* Interrupted builds. If tenkana is ended while it builds (kill, a closed
terminal), it ends the TeX engine too. The output files are then
incomplete, and tenkana knows: the next build rebuilds them even if no
input changed.
* XeTeX runs with -no-pdf, so snapshots never involve its separate PDF
converter (a second process, which a snapshot cannot hold); tenkana runs
it once at the end, as XeTeX would (xdvipdfmx -q -E). If you give
-no-pdf yourself, tenkana leaves the .xdv file as it is.
* SyncTeX. TeX writes <job>.synctex(busy) during the run and renames
it to <job>.synctex.gz at the end. A snapshot holds the busy file
open, so the snapshot library keeps that file in tenkana's cache (it
catches the file however the engine opens it, also in engines with
zlib built in, as TeX Live's own binaries are), and the renaming
becomes a copy to <job>.synctex.gz. So snapshots work with
SyncTeX on. For pdfTeX and LuaTeX the .synctex.gz file is the same as
a plain run's; XeTeX's names the .xdv file as its output (it runs with
-no-pdf), and viewers find the same positions in it.
4. Snapshots on disk
--------------------
After a build that made a new preamble snapshot, a copy of that snapshot
writes itself to tenkana's cache (section 3) in the background, unless it
would be larger than TENKANA_SAVE_LIMIT, so it survives a reboot. Only what
is needed to rebuild it is stored:
* memory backed by files (the engine, its libraries, mapped fonts) is
mapped again from the files; only pages that differ are stored;
* empty pages are skipped;
* the rest is compressed with zstd (zlib if zstd is not installed).
Saving happens in a copy of the frozen engine, which reads the memory of a
pristine child of its own (a parent may read its child's memory without
root), so no build waits for it.
Restoring needs no root either. tenkana starts the engine exactly as it
started the original: same command, recorded environment, and
address-space randomisation switched off for the engine (allowed for any
user), so the binary, libraries, stack and vDSO land at the same
addresses. Before the engine's main() runs, the snapshot library
replaces the process's memory with the image, reopens its files at their
positions, restores signal handlers, working directory and thread pointer,
and continues from the frozen point.
An image is used only on the same kernel, with the same engine binary,
libraries and mapped files; otherwise tenkana makes a full run and says why.
TENKANA_SAVE_LIMIT (in MB, default 20) sets the largest image kept; 0 turns
saving off. Images contain the engine's memory, including its environment
variables, which is why tenkana's cache is readable only by you.
Measured as an ordinary user:
document engine memory on disk restore
article, pdfLaTeX 28MB 6.7MB 0.1s
article, XeLaTeX 113MB 8.4MB 0.2s
article, LuaLaTeX 96MB 13.5MB 0.2s
220-page book, LuaLaTeX 90MB 13.3MB 0.2s
talk with a Japanese font 719MB 55.5MB 1.1s
(over the default limit)
5. Measured results
-------------------
Test files are in testbook/ (run `python3 make_chapters.py' there first).
Machine: TeX Live 2023, LuaLaTeX, as an ordinary user.
book.tex: 220 pages, 12 \include'd chapters with TikZ, pgfplots, siunitx,
hyperref and cleveref. A plain lualatex run takes 8.5s. Medians of three
draft builds after an edit:
edit in tenkana compared with a plain run
chapter 12 0.77s 11 times as fast
chapter 11 1.45s 5.8 times as fast
chapter 9 2.63s 3.2 times as fast
chapter 6 4.54s 1.9 times as fast
chapter 1 7.54s 1.1 times as fast
preamble 9.09s 7% slower (a full run)
nothing changed 0.08s build skipped
article.tex (the same preamble, one chapter, 20 pages): 2.07s plain,
0.76s with tenkana after an edit (63% less time).
Correctness: after `tenkana --final', the book's PDF was compared with plain
lualatex run on the same sources; all 220 pages were identical, also after
an edit that changes references across chapters. The same held for
pdfLaTeX and XeLaTeX, and for builds restored from disk.
6. Building without snapshots
-----------------------------
When snapshots are not possible (no C compiler, not an x86-64 machine), or
with --no-snapshots, tenkana builds with plain runs:
* pdfTeX, and XeTeX when the preamble loads no system fonts: the
preamble is cached once in a private format (mylatexformat) and
reused until it changes, unless TeX's first line is TeX code;
* LuaTeX, and XeTeX with system fonts: these fonts cannot be stored in
a format file, so the preamble is processed normally;
* all engines: reruns until references settle with --final, and no run
at all when no input changed.
7. Options and environment
--------------------------
tenkana [tenkana options] TeX-command [TeX options] file
tenkana [tenkana options] -q file
tenkana's options come first; the first word that is not one of them starts
the TeX command (`--' ends tenkana's options explicitly).
-q, --quick the file only, with the default engine:
`tenkana -q thesis' is `tenkana lualatex thesis'
-F, --final rerun until references, citations and table of
contents are up to date
-d, --draft one pass; references not updated (the default)
-v, --verbose also show passes, changed files, snapshots and
their memory
--silent show only TeX's output and tenkana's error messages
--no-snapshots build with plain runs instead of snapshots
-f, --force stop snapshots, discard cached state, build from
scratch
--clean stop a document's snapshots, delete its cache,
and exit. Name the document as for a build
(`tenkana --clean lualatex thesis') or just by its
name (`tenkana --clean thesis'; thesis.tex and
thesis.ltx mean the same). The file itself need
not exist any more.
-V, --version show version, copyright and license
-h, --help show a short help text
TeX's options go to TeX unchanged (-shell-escape, -synctex=1,
-interaction=batchmode, -file-line-error, -jobname=NAME, ...). tenkana does
not support these yet, and says so: -ini, -fmt (and &format),
-output-directory, -aux-directory, -output-format and -output-driver. A
change of the TeX command makes the next build a full run.
TENKANA_ENGINE the command for -q instead of lualatex; it may
have options ("pdflatex -shell-escape")
TENKANA_SILENT if set, as --silent
TENKANA_TTL seconds snapshots live after the document's last
build (default 28800, i.e. 8 hours)
TENKANA_SAVE_LIMIT largest preamble snapshot kept on disk, in MB
(default 20; 0 = never)
TENKANA_WATCH more generated-file extensions to watch
TENKANA_SHIM use this snapshot library instead of building one
TENKANA_DEBUG if set, explain why a snapshot could not be used
8. Limitations
--------------
* Snapshots use memory while they live, roughly what the engine itself
needs for your fonts and packages. Use `tenkana --clean' when done, or a
lower TENKANA_TTL.
* Snapshots need \begin{document} in the main file itself. Other
documents -- plain TeX, or a main file that \inputs another with
\begin{document} -- are built with plain runs (section 6); -v says
so.
* Chapter snapshots happen only at \include; documents that use only
\input get the preamble snapshot. With \includeonly, only the
chapters it includes get snapshots.
* The TeX options -ini, -fmt, -output-directory, -aux-directory,
-output-format and -output-driver are not supported yet.
* Snapshots keep the output files (.pdf, .aux, ...) open. If another
program replaces one of them (some synchronisation tools do), the
snapshots that hold it cannot be used, and tenkana starts from an earlier
one or makes a full run.
* Only the preamble snapshot is saved on disk; after a reboot the first
build resumes from the preamble and makes the chapter snapshots again.
* Disk images need the same kernel and TeX installation as when they were
saved (checked; otherwise a full run). Where processes may not read
their children's memory (Yama ptrace scope 2 or 3), images cannot be
saved; snapshots in memory still work.
* The engine runs with address-space randomisation switched off, which
restoring images requires. This affects only the engine tenkana starts.
* tenkana's hooks are added to the start of line 1 of the copy of the main
file that the engine reads, so a %&format line or a byte-order mark at
the very start of the file is not recognised.
* The pause uses read stream 15 (\openin15) for an instant; it would
clash only with a document that keeps stream 15 open across
\begin{document} or an \include.
* \today, time stamps and \write18 side effects come from the run that
made the snapshot.
* The engine must be dynamically linked (true for TeX Live and the
usual distribution packages), because the library is loaded with
LD_PRELOAD.
9. Files in this package
------------------------
README this file
COPYING the GNU General Public License, version 3
AUTHORS who wrote tenkana
NEWS changes between versions
tenkana.1 the man page
install.sh installs tenkana (building it with build.sh)
build.sh builds the single-file `tenkana', tenkana.pyz
pyproject.toml alternative installation with pip or pipx
src/tenkana/cli.py command line
src/tenkana/messages.py every text tenkana prints
src/tenkana/snapshot.py builds with snapshots
src/tenkana/tenkanasnap.c the snapshot library (LD_PRELOAD)
src/tenkana/shim.py compiles and locates the snapshot library
src/tenkana/fallback.py builds without snapshots
src/tenkana/common.py shared helpers
tools/catalogue.py lists every message, the help page and the man
page, for review
testbook/ test documents (book.tex, article.tex,
talk.tex, make_chapters.py)
10. Copyright and license
-------------------------
Copyright © 2026 निरंजन (Niranjan)
tenkana is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation, either version 3 of the License, or (at your
option) any later version.
tenkana is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
for more details.
You should have received a copy of the GNU General Public License
along with tenkana. If not, see <https://www.gnu.org/licenses/>.
tenkana was written by Claude, a large language model (LLM) made by Anthropic.
Courts are divided on whether material generated by an LLM can be protected by
copyright. Should it be found that the copyright claimant cannot hold
copyright in any part of tenkana, that part is in the public domain. Wherever
copyright does hold, tenkana is distributed under the terms of the GNU General
Public License, version 3 or (at your option) any later version, as stated
above.
Download the contents of this package in one zip archive (97.2k).
Tenkana – Faster repeated LaTeX builds by resuming from engine snapshots
tenkana (named after a fast jumping spider in India) makes repeated LaTeX builds faster. While TeX runs, it keeps frozen copies (snapshots) of the running engine, just before \begin{document} and before each \include. After an edit, tenkana continues from the last snapshot before the change, so the preamble, its packages and fonts, and all unchanged chapters before the edit are not processed again. It needs no root access and changes neither the TeX engine nor the document.
tenkana is given the TeX command line one would type without it, e.g. tenkana lualatex thesis; any command that runs pdfTeX, LuaTeX or XeTeX works. Builds are drafts by default; --final reruns until cross-references, citations and the table of contents are up to date. Unlike build tools such as latexmk or arara, which decide which programs to run, tenkana speeds up the TeX runs themselves.
Snapshots need GNU/Linux on x86-64 and a C compiler; elsewhere tenkana builds with plain runs. The program was written with Claude, a large language model by Anthropic.
| Package | Tenkana |
| Bug tracker | |
| Announcements | |
| Repository | |
| Version | 0.2.0 |
| Licenses | GNU General Public License, version 3 or newer |
| Copyright | 2026 निरंजन (Niranjan) |
| Maintainer | निरंजन |
| Topics | Compilation |