CTAN has a new package: tex2tok

A new package has been installed at tug.ctan.org and should soon make its way to your favorite mirror. Thanks again, Jim ====================================================================== The following information was provided by the package's contributor. Name of contribution: tex2tok Author's name: Jonathan Fine Location on CTAN: /support/tex2tok Summary description: tex2tok: convert a TeX source file into token License type: gpl Announcement text given by the package's contributor: - - ---------------------------------------------------------------------- tex2tok: convert a TeX source file into tokens usage: $ tex '&tex2tok' '\LEX{story}' This will create a file story.tok. This file will contain one line for each token. - - ---------------------------------------------------------------------- You can have a look at the package at http://tug.ctan.org/tex-archive/support/tex2tok although you may get a better network connection by visiting a mirror of CTAN that is near to you; see http://www.ctan.org/tex-archive/README.mirrors .