Using LuaTeX to create SVG of typeset formulae
Introduction
This is a current work-in-progress so I'll keep it brief and outline the ideas.
There are, of course, a number of tools available to generate SVG from TeX or, more correctly, SVG from DVI. Indeed, I wrote one such tool myself some 9 years ago: as an event-driven COM object which fired events to a Perl backend. For sure, DVI to SVG works but with LuaTeX you can do it differently and, in my opinion, in a much more natural and integrated way. The key is the node structures which result from typeset maths. By parsing the node tree you can create the data to construct the layout and generate SVG (or whatever you like).
Math node structures
Let's take some plain TeX math and store it in a box:
\setbox101=\hbox{$\displaystyle\eqalign{\left| {1 \over \zeta - z - h} -
{1 \over \zeta - z} \right| & = \left|{(\zeta - z) - (\zeta - z - h) \over (\zeta - z - h)(\zeta - z)}
\right| \cr & =\left| {h \over (\zeta - z - h)(\zeta - z)} \right| \cr
& \leq {2 |h| \over |\zeta - z|^2}.\cr}$}
What does the internal node structure, resulting from this math, actually look like? Well, it looks pretty complicated but in reality it's quite easy to parse with recursive functions, visiting each node in turn and exporting the data contained in each node. Note that you must take care to preserve context by "opening" and "closing" the node data for each hlist or vlist as you return from each level of recursion.
The idea is that you pass box101 to Lua function which starts at the root node of the box and works its way through and down the node tree. One such function might look like this:
<<snip lots of code>>
function listnodes(head)
while head do
local id = head.id
if id==0 then
mnodes.nodedispatch[id](head, hdepth+1)
elseif id==1 then
mnodes.nodedispatch[id](head, vdepth+1)
else
mnodes.nodedispatch[id](head)
end
if id == node.id('hlist') or id == node.id('vlist') then
--print("enter recursing", depth)
if id==0 then
hdepth=hdepth+1
elseif id==1 then
vdepth=vdepth+1
else
end
--mnodes.open(id, depth,head)
listnodes(head.list)
if id==0 then
mnodes.close(id, hdepth)
hdepth=hdepth-1
elseif id==1 then
mnodes.close(id, vdepth)
vdepth=vdepth-1
else
end
--print("return recursing", depth)
end
head = head.next
end
end
What you do with the data in each node depends on your objectives. My preference (current thinking) is to generate a "Lua program" which is a set of Lua functions that you can run to do the conversion. The function definitions are dictated by the conversion you want to perform. For example, the result of parsing the node tree could be something like this (lots of lines omitted):
HLISTopen(1,13295174,0,2445314,2772995,0,0,0) MATH(0,0) HLISTopen(2,0,0,0,0,0,0,0) HLISTclose(2) GLUE(skip,109224,0,0,0,0) VLISTopen(1,13076726,0,2445314,2772995,0,0,0) HLISTopen(2,13076726,0,622600,950280,0,0,0) GLUE(tabskip,0,0,0,0,0) HLISTopen(3,5670377,0,622600,950280,0,0,0) RULE(0,557056,229376) GLUE(skip,0,65536,0,2,0) MATH(0,0) HLISTopen(4,5670377,0,622600,950280,0,0,0) HLISTopen(5,5670377,0,622600,950280,0,0,0) VLISTopen(2,218453,-950280,1572880,0,0,0,0) HLISTopen(6,218453,0,393220,0,0,0,0) GLYPH(12,19,218453,0,393220) HLISTclose(6) KERN(0,0) HLISTopen(6,218453,0,393220,0,0,0,0) GLYPH(12,19,218453,0,393220) HLISTclose(6) KERN(0,0) HLISTopen(6,218453,0,393220,0,0,0,0) GLYPH(12,19,218453,0,393220) HLISTclose(6) KERN(0,0) HLISTopen(6,218453,0,393220,0,0,0,0) GLYPH(12,19,218453,0,393220) HLISTclose(6) VLISTclose(2) HLISTopen(6,2805531,0,576976,865699,0,0,0) HLISTopen(7,2805531,0,576976,865699,0,0,0) HLISTopen(8,78643,-163840,0,0,0,0,0) HLISTclose(8) VLISTopen(2,2648245,0,576976,865699,0,0,0) HLISTopen(8,2648245,0,0,422343,2,1,17) GLUE(skip,0,65536,65536,2,2) GLYPH(49,1,327681,422343,0) GLUE(skip,0,65536,65536,2,2) HLISTclose(8) KERN(266409,0) RULE(-1073741824,26214,0) KERN(145167,0) HLISTopen(8,2648245,0,127431,455111,0,0,0) GLYPH(16,7,286721,455111,127431) KERN(48355,0) GLUE(medmuskip,145632,72816,145632,0,0) GLYPH(0,13,509726,382293,54613) GLUE(medmuskip,145632,72816,145632,0,0) GLYPH(122,7,304775,282168,0) KERN(28823,0) GLUE(medmuskip,145632,72816,145632,0,0) GLYPH(0,13,509726,382293,54613) GLUE(medmuskip,145632,72816,145632,0,0) GLYPH(104,7,377591,455111,0) HLISTclose(8) VLISTclose(2) HLISTopen(8,78643,-163840,0,0,0,0,0) HLISTclose(8) HLISTclose(7) HLISTclose(6) GLUE(medmuskip,145632,72816,145632,0,0) GLYPH(0,13,509726,382293,54613) GLUE(medmuskip,145632,72816,145632,0,0) HLISTopen(6,1626950,0,576976,865699,0,0,0) HLISTopen(7,1626950,0,576976,865699,0,0,0) <<snip loads of lines>> GLYPH(58,7,182045,69176,0) HLISTclose(4) MATH(1,0) GLUE(skip,0,65536,0,2,0) HLISTclose(3) GLUE(tabskip,0,0,0,0,0) HLISTclose(2) VLISTclose(1) GLUE(skip,109224,0,0,0,0) MATH(1,0) HLISTclose(1)
At each node you can emit a "function" such as GLUE(skip,0,65536,65536,2,2) or GLYPH(49,1,327681,422343,0) which contain the node data as arguments of the function call. Each of these "functions" can then be "run" by providing a suitable function body: perhaps one for SVG, HTML5 canvas and JavaScript, or EPS file or whatever. The point is you can create whatever you like simply by emitting the appropriate data from each node.
What about glyph outlines?
Fortunately, there is a truly wonderful C library called FreeType which has everything you need to generate the spline data, even with TeX's wonderfully arcane font encodings for the 8-bit world of Type 1 fonts. Of course, to plug FreeType into the overall solution you will need to write a linkable library: I use DLLs on Windows. FreeType is a really nice library, easy to use and made even more enjoyable by Lua's C API.
Summary
Even though I have omitted a vast amount of detail, and the work is not yet finished, I hope you can see that LuaTeX offers great potential for new and powerful solutions.
LuaTeX: nodes, graphics and PDF coordinate transformation matrices
Note: if you want to zoom in on the matrices, right-click over an equation and set the MathJax parameters to your preferred values:
Introduction
In this post I'll introduce a nice matrix-manipulation library called lua-matrix. It is written in pure Lua and so should be usable with any LuaTeX installation. You can download the code from GitHub. You can use lua-matrix as a convenient method to create PDF transformation matrices.
Note: where to install Lua code modules
The texmf.cnf variable you need to check is LUAINPUTS. See this newsgroup post for more details.
Tip: a Git tool for Windows users
Like many open source projects lua-matrix is hosted on GitHub. If you are a Windows user you may need to install some utilities that let you "check out" a copy of the code on repositories such as GitHub or others based on SVN. For SVN repositories there is the excellent TortoiseSVN but for Git repos I use a free tool called Git for Windows.
Very brief summary of matrices
Quoting from the PDF standard:
"PDF represents coordinates in a two-dimensional space. The point (x, y) in such a space can be expressed in vector form as [x y 1]. The constant third element of this vector (1) is needed so that the vector can be used with 3-by-3 matrices. The transformation between two coordinate systems is represented by a 3-by-3 transformation matrix written as
$$\displaystyle \left( \matrix{ a & b & 0 \cr c & d & 0 \cr e & f & 1 \cr} \right)$$
Because a transformation matrix has only six elements that can be changed, it is usually specified in PDF as the six-element array [a b c d e f].
Note: This is method of representing coordinates is referred to as homogeneous coordinates.
Rotation
The matrix for rotation by an angle \(\theta\) counter clockwise about the origin:
$$\displaystyle \left( \matrix{ \cos (\theta) & \sin (\theta) & 0 \cr -\sin (\theta) & \cos (\theta) & 0 \cr 0 & 0 & 1 \cr} \right)$$
This is expressed in PDF code as \(\cos(\theta)\ \sin(\theta)\ -\hskip-2pt\sin(\theta)\ \cos(\theta)\ 0\ 0\) cm
Translation
The matrix for translation by \((t_x, t_y)\) relative to the origin:
$$\displaystyle \left( \matrix{ 1 & 0 & 0 \cr 0 & 1 & 0 \cr t_x & t_y & 1 \cr} \right)$$
This is expressed in PDF code as \(1\ 0\ 0\ 1\ t_x\ t_y\) cm
Scale
The matrix for scaling by \(s_x\) in the horizonal direction and \(s_y\) in the vertical direction is:
$$\displaystyle \left( \matrix{ s_x & 0 & 0 \cr 0 & s_y & 0 \cr 0 & 0 & 1 \cr} \right)$$
This is expressed in PDF code as \(s_x\ 0\ 0\ s_y\ 0\ 0\) cm
Demonstration graphic
The following simple graphic (shown as inline SVG) will be used to explore transformations. The equivalent PDF graphic (in hand-coded PDF data) is shown below.
Equivalent PDF data
The following PDF code will draw a very similar graphic:
q % save graphics state 1 j % set the line join style 1 J % set the line cap style 10 M % set the miter limit %Set the stroking color space to DeviceRGB 0 0 0 RG % black % draw the axes 0 0 m 0.5 w 0 30 l 0 0 m 30 0 l S % stroke % draw the red arrow head on x axis q % save graphics state %Set the non-stroking color space to DeviceRGB 1 0 0 rg % red % translate to end of line on x-axis 1 0 0 1 30 0 cm % draw an arrowhead 0 0 m % move to the origin 0 1.5 l 2.5 0 l 0 -1.5 l h % close the current subpath B % fill and stroke Q % restore graphics state % draw the blue arrow head on y axis q % save graphics state %Set the non-stroking color space to DeviceRGB 0 0 1 rg % blue % translate to end of line on y-axis 1 0 0 1 0 30 cm % draw an arrowhead 0 0 m % move to the origin -1.5 0 l 0 2.5 l 1.5 0 l h % close the current subpath B % fill and stroke % restore graphics state Q Q
Creating a graphic with LuaTeX nodes
As usual, a simple plain TeX setup.
\pdfoutput=1
\pdfcompresslevel=0
\hoffset-1in
\voffset-1in
\pdfpageheight=50mm
\pdfpagewidth=50mm
\vsize=50mm
\hsize=50mm
\topskip=0pt
\maxdepth=0pt
\nopagenumbers
\directlua{
grafik = node.new("whatsit","pdf_literal")
grafik.mode=0
grafik.data=" q % save graphics state
1 j % set the line join style
1 J % set the line cap style
10 M % set the miter limit
%Set the stroking color space to DeviceRGB
0 0 0 RG % black
% draw the axes
0 0 m
0.5 w
0 30 l
0 0 m
30 0 l
S % stroke
% draw the red arrow head on x axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
1 0 0 rg % red
% translate to end of line on x-axis
1 0 0 1 30 0 cm
% draw an arrowhead
0 0 m % move to the origin
0 1.5 l
2.5 0 l
0 -1.5 l
h % close the current subpath
B % fill and stroke
Q % restore graphics state
% draw the blue arrow head on y axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
0 0 1 rg % blue
% translate to end of line on y-axis
1 0 0 1 0 30 cm
% draw an arrowhead
0 0 m % move to the origin
-1.5 0 l
0 2.5 l
1.5 0 l
h % close the current subpath
B % fill and stroke
% restore graphics state
Q
Q "
tex.box[1000]=node.hpack(grafik)
}
Here is out new graphic \vskip 35mm
\noindent\hskip 15mm \copy1000
\bye
Notes about the PDF graphic
In the code above we have not assigned any size to the box containing the graphic, hence I needed to add \vskip 35mm \noindent\hskip 15mm to push the graphic into a space where it will be seen. To give the graphic some dimensions, we'll need to add code such as
tex.box[1000].width = width value in sp
tex.box[1000].height = height value in sp
tex.box[1000].depth = depth value in sp
where the values assigned are in sp (special points). You may recall that 65536sp = 1 TeX point, where 72.27 TeX points = 1 inch = 72 PostScript points (same as default in PDF).
As far as the LuaTeX engine is concerned, the box containing the graphic has zero size, we have to tell LuaTeX how big we want it to be. In addition, the line widths, based on the above code, will be affected by any scaling but it is not too difficult to fix that.
The Lua code
The idea is that we create a number of functions based on the lua-matrix library and save those functions into a Lua module called "mymatrix.lua". Within "mymatrix.lua" we import the lua-matrix code via its module called "matrix" which we load with:
local matrix=require("matrix")
Our simple API
Here are the functions defined within our "mymatrix.lua" module:
- rotate(angle): returns a 3 x 3 rotation matrix (angle positive for counter clockwise)
- translate(tx,ty): returns a 3 x 3 translation matrix (tx, ty are translations in x, y directions)
- scale(sx,sy): returns a 3 x 3 scaling matrix (sx, sy are scaling in x,y directions)
- dump(mtx): simple debugging function (mtx = 3 x 3 matrix)
- toinlinetex(mtx): returns inline TeX code so we can typeset the matrix (mtx = 3 x 3 matrix)
- todisplaytex(mtx): returns display TeX code so we can typeset the matrix (mtx = 3 x 3 matrix)
- topdf(mtx): returns matrix in PDF code format (mtx = 3 x 3 matrix)
Here's the source code for mymodule.lua. One huge advantage of putting Lua code into Lua modules is that it greatly simplifies dealing with \catcode issues. Note that within code saved in Lua files you use the regular Lua comment "--" mechanism and not the TeX comment "%" mechanism. You can use "%" when the code is embedded in a \directlua{...} call.
module("mymatrix",package.seeall)
local matrix=require("matrix")
local rad=math.rad
local sin=math.sin
local cos=math.cos
-- Function to generate PDF transformation (rotation) matrices.
function rotate(angle)
local rot = matrix {{cos(rad(angle)),sin(rad(angle)),0},{-sin(rad(angle)),cos(rad(angle)),0},{0,0,1}}
return rot
end
-- Function to generate PDF transformation (translation) matrices.
function translate(tx,ty)
local tran = matrix {{1,0,0},{0,1,0},{tx,ty,1}}
return tran
end
-- Function to generate PDF transformation (scale) matrices.
function scale(sx,sy)
local scale = matrix {{sx,0,0},{0,sy,0},{0,0,1}}
return scale
end
function dump(mtx)
for i=1,3 do
for j=1,3 do
print("(i,j)=("..i..","..j..")="..mtx[i][j])
end
end
end
function todisplaytex(mtx)
texcode=string.format([[$$\left(\matrix {%3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr } \right)$$]],
mtx[1][1], mtx[1][2], mtx[1][3],mtx[2][1], mtx[2][2], mtx[2][3],
mtx[3][1], mtx[3][2], mtx[3][3])
return texcode
end
function toinlinetex(mtx)
texcode=string.format([[$\left(\matrix {%3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr } \right)$]],
mtx[1][1], mtx[1][2], mtx[1][3],mtx[2][1], mtx[2][2], mtx[2][3],
mtx[3][1], mtx[3][2], mtx[3][3])
return texcode
end
function topdf(mtx)
pdftext = string.format("%3.3f %3.3f %3.3f %3.3f %3.3f %3.3f cm",
mtx[1][1], mtx[1][2],mtx[2][1], mtx[2][2], mtx[3][1], mtx[3][2])
return pdftext
end
Full example
\pdfoutput=1
\pdfcompresslevel=0
\hoffset-1in
\voffset-1in
\pdfpageheight=350mm
\pdfpagewidth=150mm
\vsize=350mm
\hsize=150mm
\topskip=0pt
\maxdepth=0pt
\nopagenumbers
\directlua{require("mymatrix")}
\directlua{
% We'll create an non-transformed graphic and store it in box 1000
grafik = node.new("whatsit","pdf_literal")
grafik.mode=0
grafik.data="
q % save graphics state
1 j % set the line join style
1 J % set the line cap style
10 M % set the miter limit
%Set the stroking color space to DeviceRGB
0 0 0 RG % black
% draw the axes
0 0 m
0.5 w
0 30 l
0 0 m
30 0 l
S % stroke
% draw the red arrow head on x axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
1 0 0 rg % red
% translate to end of line on x-axis
1 0 0 1 30 0 cm
% draw an arrowhead
0 0 m % move to the origin
0 1.5 l
2.5 0 l
0 -1.5 l
h % close the current subpath
B % fill and stroke
Q % restore graphics state
% draw the blue arrow head on y axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
0 0 1 rg % blue
% translate to end of line on y-axis
1 0 0 1 0 30 cm
% draw an arrowhead
0 0 m % move to the origin
-1.5 0 l
0 2.5 l
1.5 0 l
h % close the current subpath
B % fill and stroke
% restore graphics state
Q
Q "
% store graphic in box 1000
tex.box[1000]=node.hpack(grafik)
% create some transformation matrices
mtx1 = mymatrix.rotate(45)
mtx2 = mymatrix.scale(1.5,2)
mtx3 = mymatrix.translate(15,15)
% Now we copy our untransformed node
% and add the PDF transformation matrices to
% rotate, scale etc.
% We'll do a rotation and store in box 1001
grafik2 = node.copy(grafik)
grafik2.mode=0
cm = mymatrix.topdf(mtx1)
% copy the PDF data from the untransformed
% graphic and add the PDF rotation matrix
grafik2.data="q ".. cm ..grafik.data.." Q "
% store graphic in box 1001
tex.box[1001]=node.hpack(grafik2)
% We'll do a scale and store in box 1002
grafik3 = node.copy(grafik)
grafik3.mode=0
cm = mymatrix.topdf(mtx2)
% copy the PDF data from the untransformed
% graphic and add the PDF rotation matrix
grafik3.data="q ".. cm ..grafik.data.." Q "
% store graphic in box 1002
tex.box[1002]=node.hpack(grafik3)
% Now multiply the scale and rotation matrices
% --- experiment with different combinations
combo=mtx1*mtx2
grafik4 = node.copy(grafik)
grafik4.mode=0
cm = mymatrix.topdf(combo)
% copy the PDF data from the untransformed
% graphic and add the result of multiplication
grafik4.data="q ".. cm ..grafik.data.." Q "
% store graphic in box 1002
tex.box[1003]=node.hpack(grafik4)
% Now multiply the scale, rotation and translation
% matrices --- experiment with different combinations
combo2=mtx1*(mtx2*mtx3)
grafik5 = node.copy(grafik)
grafik5.mode=0
cm = mymatrix.topdf(combo2)
% copy the PDF data from the untransformed
% graphic and add the result of multiplication
grafik5.data="q ".. cm ..grafik.data.." Q "
% store graphic in box 1002
tex.box[1004]=node.hpack(grafik5)
}
Here are our graphics \vskip 35mm
\noindent\hskip 15mm \copy1000 Default graphic
\vskip 35mm
\noindent\hskip 15mm \copy1001 Rotated graphic = \directlua{tex.sprint(mymatrix.toinlinetex(mtx1))}
\vskip 35mm
\noindent\hskip 15mm \copy1002 Non-uniformly scaled graphic = \directlua{tex.sprint(mymatrix.toinlinetex(mtx2))}
\vskip 35mm
\noindent\hskip 15mm \copy1003 Rotation $\times$ scale = \directlua{tex.sprint(mymatrix.toinlinetex(combo))}
\vskip 35mm
\noindent\hskip 15mm \copy1004 Rotation $\times$ scale $\times$ translate =\directlua{tex.sprint(mymatrix.toinlinetex(combo2))}
\bye
Resulting PDF
As usual, through the Google Docs viewer or download PDF here.
Downloads
Quick and dirty method for creating spot colours in PDFs
Introduction
Just a 10-minute hack to explore putting spot colours into a PDF via pdf_colorstack nodes. I don't have access to Acrobat Professional at the moment to check the separations properly, so treat this as an "alpha" method (i.e., not fully tested...). The colour defined below is lifted straight from an early PDF specification and implemented via LuaTeX nodes. As it says "on the tin": a quick and dirty method
.
\pdfoutput=1
\hoffset-1in
\voffset-1in
\nopagenumbers
\pdfcompresslevel=0
\directlua {
n = pdf.immediateobj("stream", "{ dup 0.84 mul
exch 0.00 exch dup 0.44 mul
exch 0.21 mul
}", "/FunctionType 4
/Domain [0.0 1.0]
/Range [0.0 1.0 0.0 1.0 0.0 1.0 0.0 1.0] ")
o = pdf.immediateobj("[ /Separation
/LogoGreen
/DeviceCMYK".." "..n.." 0 R]")
pdf.pageresources = " /ColorSpace << /LogoGreen "..o.." 0 R >> "
pdf_colstart = node.new("whatsit","pdf_colorstack")
pdf_end = node.new("whatsit","pdf_colorstack")
pdf_colstart.data="/LogoGreen CS /LogoGreen cs 1 SC 1 sc "
pdf_colstart.cmd=1
pdf_end.data= " "
pdf_end.cmd = 2
tex.box[1999]= node.hpack(pdf_colstart)
tex.box[2000]= node.hpack(pdf_end)
}
\def\makeitgreen#1{\copy1999\relax#1\copy2000\relax}
There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything \makeitgreen{embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to} repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looks reasonable. The generated Lorem Ipsum is therefore always free from repetition, injected humour, or non-characteristic words etc.
\bye
Resulting PDF
As usual, through the Google Docs viewer or download here.

