{"id":3915,"date":"2016-12-19T18:20:52","date_gmt":"2016-12-19T18:20:52","guid":{"rendered":"http:\/\/www.readytext.co.uk\/?p=3915"},"modified":"2019-02-03T11:16:20","modified_gmt":"2019-02-03T11:16:20","slug":"luatex-token-library-simple-example-of-scanners-to-mix-tex-and-metapost","status":"publish","type":"post","link":"https:\/\/www.readytext.co.uk\/?p=3915","title":{"rendered":"LuaTeX token library: simple example of scanners to mix TeX and MetaPost"},"content":{"rendered":"<h1>Introduction<\/h1>\n<p>In this short article I share the results of using LuaTeX\u2019s token library as <em>one way<\/em> to intermix TeX and MetaPost code. I used LuaTeX 1.0.1 which I compiled from the source code in the experimental branch of the LuaTeX SVN repository but, I believe, it should also work with LuaTeX 1.0 (may also work with some earlier versions too). In addition, I used <code>luamplib<\/code> version 2016\/03\/31 v2.11.3 (I downloaded my copy from <a href=\"https:\/\/github.com\/lualatex\/luamplib\">github<\/a>). Note that I do not have a \u201cstandard\u201d TeX installation\u2014I prefer to build and maintain my own custom setup (very small and compact).<\/p>\n<h2>The LuaTeX token library<\/h2>\n<p>I will not try to explain the underlying technical details but simply point you to read <a href=\"https:\/\/www.tug.org\/TUGboat\/tb36-1\/tb112hagen-scan.pdf\">this article by Hans Hagen<\/a> and the relevant section in the <a href=\"http:\/\/www.luatex.org\/svn\/trunk\/manual\/luatex.pdf\">LuaTeX Reference Manual<\/a> (section 9.6: The token library). Here, I\u2019ll just provide an example of using the token library\u2014it is not a sophisticated or \u201cclever\u201d example but one simply designed to demonstrate the idea.<\/p>\n<h1>Objective<\/h1>\n<p>The goal is to have TeX macros that contain a mixture of TeX and MetaPost code and find a way to expand those macros into a string of MetaPost code which can be passed to LuaTeX\u2019s in-built MetaPost interpreter: mplib.&nbsp; Suppose, just by way of a simple example, that we defined the following simple TeX macros:<\/p>\n<pre>\\def\\mpbf#1{beginfig(#1);\\space}\n\\def\\fc{fullcircle\\space}\n\\def\\pp#1#2{pickup pencircle xscaled #1mm yscaled #2mm;\\space}\n\\def\\mpef{endfig; end}\n\\def\\draw#1{draw \\fc scaled #1;\\space}<\/pre>\n<p>and we\u2019d like to use them to write TeX code that can build MetaPost graphics. Note that the definition of <code>\\draw<\/code> also contains the command <code>\\fc<\/code><\/p>\n<h1>The scanner<\/h1>\n<p>The following code uses LuaTeX\u2019s token library function <code>scan_string()<\/code> to generate a string from a series of incoming TeX macros\u2014by <em>expanding<\/em> them. It stores the resulting string into a toks register so that we can later use and obtain the result.<\/p>\n<pre>\\newtoks\\mpcode\n\\def\\scanit{%\n\\directlua{\nlocal p = token.scan_string()\ntex.toks[\"mpcode\"]=p\n}}\n<\/pre>\n<p>We could use this macro like this:<\/p>\n<pre> \\scanit{...text and TeX macro in braces...}\n<\/pre>\n<p>but instead we\u2019ll add just a little more functionality. Suppose we further define another TeX macro<\/p>\n<pre>\\def\\codetest{\\mpbf{1} \\pp{0.3}{0.75} \\draw{12}\\mpef }\n\n<\/pre>\n<p>which contains a sequence of commands that, once expanded, will generate MetaPost program to produce our graphic. To expand our TeX macro (<code>\\codetest<\/code>) we can do the following:<\/p>\n<pre>\\scanit{\\codetest} and the output is \\the\\mpcode\n\n<\/pre>\n<p>Here, the braces &#8220;{&#8221; and &#8220;}&#8221; are needed (I think&#8230;) in order to make <code>token.scan_string()<\/code> work correctly (I maybe wrong about that so please run your own tests). Anyway, the <code>\\codetest<\/code> macro is expanded and (due to <code>\\scanit<\/code>) the resulting MetaPost code is stored in the toks register called <code>mpcode<\/code>. We can see what the toks register contains simply by typesetting the result: doing <code>\\the\\mpcode<\/code>\u2014note you may get strange typesetting results, or an error, if your MetaPost contains characters with catcode oddities. You can use <code>\\directlua{print(tex.toks[\"mpcode\"])}<\/code> to dump the content of the <code>mpcode<\/code> toks register to the console (rather than typesetting it). What I see in my tests is that the <code>mpcode<\/code> toks register contains the following fully expanded MetaPost code:<\/p>\n<pre>beginfig(1); pickup pencircle xscaled 0.3mm yscaled 0.75mm; draw fullcircle scaled 12; endfig; end\n\n<\/pre>\n<p>And this is now ready for passing to MetaPost (via mplib). But how? Well, one option is to use the package <code>luamplib<\/code>. If you are using LaTeX (i.e., luaLaTeX format) then you can use the following environment provided by <code>luamplib<\/code>:<\/p>\n<pre>\\begin{mplibcode}\n...your MetaPost code goes here\n\\end{mplibcode}\n\n<\/pre>\n<p>However, our MetaPost code is currently contained in the <code>mpcode<\/code> toks register. So here\u2019s my (rather ugly hack) that uses <code>\\expandafter<\/code> to get the text out of <code>mpcode<\/code> and sandwiched between <code>\\begin{mplibcode}<\/code> and <code>\\end{mplibcode}<\/code>. I am sure that real TeX programmers can program something far more elegant!<\/p>\n<pre>\\def\\sa{\\begin{mplibcode}}\n\\def\\mcodex{\\expandafter\\sa\\the\\mpcode\\relax\\end{mplibcode}}\n\n<\/pre>\n<p>Consequently, we&nbsp; just need to issue the command <code>\\mcodex<\/code> to draw our MetaPost graphic. I hope this is an interesting suggestion\/solution that others might find useful.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction In this short article I share the results of using LuaTeX\u2019s token library as one way to intermix TeX and MetaPost code. I used LuaTeX 1.0.1 which I compiled from the source code in the experimental branch of the LuaTeX SVN repository but, I believe, it should also work with LuaTeX 1.0 (may also [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3915","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/3915","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3915"}],"version-history":[{"count":15,"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/3915\/revisions"}],"predecessor-version":[{"id":3973,"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/3915\/revisions\/3973"}],"wp:attachment":[{"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3915"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3915"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.readytext.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3915"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}