STM publishing: tools, technologies and change A WordPress site for STM Publishing

14Dec/11Off

LuaTeX: nodes, graphics and PDF coordinate transformation matrices

Posted by Graham Douglas

Powered by MathJax

Note: if you want to zoom in on the matrices, right-click over an equation and set the MathJax parameters to your preferred values:

Zooming MathJax

Introduction

In this post I'll introduce a nice matrix-manipulation library called lua-matrix. It is written in pure Lua and so should be usable with any LuaTeX installation. You can download the code from GitHub. You can use lua-matrix as a convenient method to create PDF transformation matrices.

Note: where to install Lua code modules
The texmf.cnf variable you need to check is LUAINPUTS. See this newsgroup post for more details.

Tip: a Git tool for Windows users
Like many open source projects lua-matrix is hosted on GitHub. If you are a Windows user you may need to install some utilities that let you "check out" a copy of the code on repositories such as GitHub or others based on SVN. For SVN repositories there is the excellent TortoiseSVN but for Git repos I use a free tool called Git for Windows.

Very brief summary of matrices

Quoting from the PDF standard:

"PDF represents coordinates in a two-dimensional space. The point (x, y) in such a space can be expressed in vector form as [x y 1]. The constant third element of this vector (1) is needed so that the vector can be used with 3-by-3 matrices. The transformation between two coordinate systems is represented by a 3-by-3 transformation matrix written as

$$\displaystyle \left( \matrix{ a & b & 0 \cr c & d & 0 \cr e & f & 1 \cr} \right)$$

Because a transformation matrix has only six elements that can be changed, it is usually specified in PDF as the six-element array [a b c d e f].

Note: This is method of representing coordinates is referred to as homogeneous coordinates.

Rotation

The matrix for rotation by an angle \[\theta\] counter clockwise about the origin:
$$\displaystyle \left( \matrix{ \cos (\theta) & \sin (\theta) & 0 \cr -\sin (\theta) & \cos (\theta) & 0 \cr 0 & 0 & 1 \cr} \right)$$

This is expressed in PDF code as \[\cos(\theta)\ \sin(\theta)\ -\hskip-2pt\sin(\theta)\ \cos(\theta)\ 0\ 0\] cm

Translation

The matrix for translation by \[(t_x, t_y)\] relative to the origin:

$$\displaystyle \left( \matrix{ 1 & 0 & t_x \cr 0 & 1 & t_y \cr 0 & 0 & 1 \cr} \right)$$

This is expressed in PDF code as \[1\ 0\ 0\ 1\ t_x\ t_y\] cm

Scale

The matrix for scaling by \[s_x\] in the horizonal direction and \[s_y\] in the vertical direction is:

$$\displaystyle \left( \matrix{ s_x & 0 & 0 \cr 0 & s_y & 0 \cr 0 & 0 & 1 \cr} \right)$$

This is expressed in PDF code as \[s_x\ 0\ 0\ s_y\ 0\ 0\] cm

Demonstration graphic

The following simple graphic (shown as inline SVG) will be used to explore transformations. The equivalent PDF graphic (in hand-coded PDF data) is shown below.

Equivalent PDF data

The following PDF code will draw a very similar graphic:

q % save graphics state
1 j % set the line join style
1 J % set the line cap style
10 M % set the miter limit
%Set the stroking color space to DeviceRGB
0 0 0 RG % black
% draw the axes
0 0 m 
0.5 w
0 30 l 
0 0 m 
30 0 l 
S % stroke
% draw the red arrow head on x axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
1 0 0 rg % red
% translate to end of line on x-axis
1 0 0 1 30 0 cm
% draw an arrowhead
0 0 m % move to the origin
0 1.5 l
2.5 0 l
0 -1.5 l 
h % close the current subpath
B % fill and stroke
Q % restore graphics state
% draw the blue arrow head on y axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
0 0 1 rg % blue
% translate to end of line on y-axis
1 0 0 1 0 30 cm
% draw an arrowhead
0 0 m % move to the origin
-1.5 0 l
 0 2.5 l
1.5 0 l 
h % close the current subpath
B % fill and stroke
% restore graphics state
Q
Q

Creating a graphic with LuaTeX nodes

As usual, a simple plain TeX setup.

\pdfoutput=1
\pdfcompresslevel=0
\hoffset-1in
\voffset-1in
\pdfpageheight=50mm
\pdfpagewidth=50mm
\vsize=50mm
\hsize=50mm
\topskip=0pt
\maxdepth=0pt
\nopagenumbers

\directlua{
grafik = node.new("whatsit","pdf_literal")
grafik.mode=0
grafik.data="  q % save graphics state
1 j % set the line join style
1 J % set the line cap style
10 M % set the miter limit
%Set the stroking color space to DeviceRGB
0 0 0 RG % black
% draw the axes
0 0 m 
0.5 w
0 30 l 
0 0 m 
30 0 l 
S % stroke
% draw the red arrow head on x axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
1 0 0 rg % red
% translate to end of line on x-axis
1 0 0 1 30 0 cm
% draw an arrowhead
0 0 m % move to the origin
0 1.5 l
2.5 0 l
0 -1.5 l 
h % close the current subpath
B % fill and stroke
Q % restore graphics state
% draw the blue arrow head on y axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
0 0 1 rg % blue
% translate to end of line on y-axis
1 0 0 1 0 30 cm
% draw an arrowhead
0 0 m % move to the origin
-1.5 0 l
 0 2.5 l
1.5 0 l 
h % close the current subpath
B % fill and stroke
% restore graphics state
Q
Q "

tex.box[1000]=node.hpack(grafik)
}

Here is out new graphic \vskip 35mm
\noindent\hskip 15mm \copy1000
\bye

Notes about the PDF graphic

In the code above we have not assigned any size to the box containing the graphic, hence I needed to add \vskip 35mm \noindent\hskip 15mm to push the graphic into a space where it will be seen. To give the graphic some dimensions, we'll need to add code such as

tex.box[1000].width = width value in sp
tex.box[1000].height = height value in sp
tex.box[1000].depth = depth value in sp

where the values assigned are in sp (special points). You may recall that 65536sp = 1 TeX point, where 72.27 TeX points = 1 inch = 72 PostScript points (same as default in PDF).

As far as the LuaTeX engine is concerned, the box containing the graphic has zero size, we have to tell LuaTeX how big we want it to be. In addition, the line widths, based on the above code, will be affected by any scaling but it is not too difficult to fix that.

The Lua code

The idea is that we create a number of functions based on the lua-matrix library and save those functions into a Lua module called "mymatrix.lua". Within "mymatrix.lua" we import the lua-matrix code via its module called "matrix" which we load with:

local matrix=require("matrix")

Our simple API

Here are the functions defined within our "mymatrix.lua" module:

  • rotate(angle): returns a 3 x 3 rotation matrix (angle positive for counter clockwise)
  • translate(tx,ty): returns a 3 x 3 translation matrix (tx, ty are translations in x, y directions)
  • scale(sx,sy): returns a 3 x 3 scaling matrix (sx, sy are scaling in x,y directions)
  • dump(mtx): simple debugging function (mtx = 3 x 3 matrix)
  • toinlinetex(mtx): returns inline TeX code so we can typeset the matrix (mtx = 3 x 3 matrix)
  • todisplaytex(mtx): returns display TeX code so we can typeset the matrix (mtx = 3 x 3 matrix)
  • topdf(mtx): returns matrix in PDF code format (mtx = 3 x 3 matrix)

Here's the source code for mymodule.lua. One huge advantage of putting Lua code into Lua modules is that it greatly simplifies dealing with \catcode issues. Note that within code saved in Lua files you use the regular Lua comment "--" mechanism and not the TeX comment "%" mechanism. You can use "%" when the code is embedded in a \directlua{...} call.

module("mymatrix",package.seeall)
local matrix=require("matrix")
local rad=math.rad
local sin=math.sin
local cos=math.cos

-- Function to generate PDF transformation (rotation) matrices.
function rotate(angle)
local rot = matrix {{cos(rad(angle)),sin(rad(angle)),0},{-sin(rad(angle)),cos(rad(angle)),0},{0,0,1}}
return rot
end

-- Function to generate PDF transformation (translation) matrices.
function translate(tx,ty)
local tran = matrix {{1,0,0},{0,1,0},{tx,ty,1}}
return tran
end

-- Function to generate PDF transformation (scale) matrices.
function scale(sx,sy)
local scale = matrix {{sx,0,0},{0,sy,0},{0,0,1}}
return scale
end

function dump(mtx)
for i=1,3 do
	for j=1,3 do
		print("(i,j)=("..i..","..j..")="..mtx[i][j])
end
end
end

function todisplaytex(mtx)
texcode=string.format([[$$\left(\matrix {%3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr } \right)$$]], 
	mtx[1][1], mtx[1][2], mtx[1][3],mtx[2][1], mtx[2][2], mtx[2][3],
	mtx[3][1], mtx[3][2], mtx[3][3])
	return texcode
end

function toinlinetex(mtx)
texcode=string.format([[$\left(\matrix {%3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr } \right)$]], 
	mtx[1][1], mtx[1][2], mtx[1][3],mtx[2][1], mtx[2][2], mtx[2][3],
	mtx[3][1], mtx[3][2], mtx[3][3])
	return texcode
end

function topdf(mtx)
pdftext = string.format("%3.3f %3.3f %3.3f %3.3f %3.3f %3.3f  cm", 
	mtx[1][1], mtx[1][2],mtx[2][1], mtx[2][2], mtx[3][1], mtx[3][2])
	return pdftext
end

Full example

\pdfoutput=1
\pdfcompresslevel=0
\hoffset-1in
\voffset-1in
\pdfpageheight=350mm
\pdfpagewidth=150mm
\vsize=350mm
\hsize=150mm
\topskip=0pt
\maxdepth=0pt
\nopagenumbers
\directlua{require("mymatrix")}

\directlua{
% We'll create an non-transformed graphic and store it in box 1000
grafik = node.new("whatsit","pdf_literal")
grafik.mode=0
grafik.data="  
q % save graphics state
1 j % set the line join style
1 J % set the line cap style
10 M % set the miter limit
%Set the stroking color space to DeviceRGB
0 0 0 RG % black
% draw the axes
0 0 m 
0.5 w
0 30 l 
0 0 m 
30 0 l 
S % stroke
% draw the red arrow head on x axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
1 0 0 rg % red
% translate to end of line on x-axis
1 0 0 1 30 0 cm
% draw an arrowhead
0 0 m % move to the origin
0 1.5 l
2.5 0 l
0 -1.5 l 
h % close the current subpath
B % fill and stroke
Q % restore graphics state
% draw the blue arrow head on y axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
0 0 1 rg % blue
% translate to end of line on y-axis
1 0 0 1 0 30 cm
% draw an arrowhead
0 0 m % move to the origin
-1.5 0 l
 0 2.5 l
1.5 0 l 
h % close the current subpath
B % fill and stroke
% restore graphics state
Q
Q "
% store graphic in box 1000
tex.box[1000]=node.hpack(grafik)

% create some transformation matrices
mtx1 = mymatrix.rotate(45)
mtx2 = mymatrix.scale(1.5,2)
mtx3 = mymatrix.translate(15,15)

% Now we copy our untransformed node 
% and add the PDF transformation matrices to
% rotate, scale etc.

% We'll do a rotation and store in box 1001
grafik2 = node.copy(grafik)
grafik2.mode=0
cm = mymatrix.topdf(mtx1)
% copy the PDF data from the untransformed 
% graphic and add the PDF rotation matrix
grafik2.data="q ".. cm ..grafik.data.." Q "
% store graphic in box 1001
tex.box[1001]=node.hpack(grafik2)

% We'll do a scale and store in box 1002
grafik3 = node.copy(grafik)
grafik3.mode=0
cm = mymatrix.topdf(mtx2)
% copy the PDF data from the untransformed 
% graphic and add the PDF rotation matrix
grafik3.data="q ".. cm ..grafik.data.." Q "
% store graphic in box 1002
tex.box[1002]=node.hpack(grafik3)

% Now multiply the scale and rotation matrices
% --- experiment with different combinations
combo=mtx1*mtx2

grafik4 = node.copy(grafik)
grafik4.mode=0
cm = mymatrix.topdf(combo)
% copy the PDF data from the untransformed 
% graphic and add the result of multiplication
grafik4.data="q ".. cm ..grafik.data.." Q "

% store graphic in box 1002
tex.box[1003]=node.hpack(grafik4)

% Now multiply the scale, rotation and translation 
% matrices --- experiment with different combinations
combo2=mtx1*(mtx2*mtx3)

grafik5 = node.copy(grafik)
grafik5.mode=0
cm = mymatrix.topdf(combo2)
% copy the PDF data from the untransformed 
% graphic and add the result of multiplication
grafik5.data="q ".. cm ..grafik.data.." Q "

% store graphic in box 1002
tex.box[1004]=node.hpack(grafik5)


}

Here are our graphics \vskip 35mm
\noindent\hskip 15mm \copy1000 Default graphic
\vskip 35mm
\noindent\hskip 15mm \copy1001 Rotated graphic =  \directlua{tex.sprint(mymatrix.toinlinetex(mtx1))}
\vskip 35mm
\noindent\hskip 15mm \copy1002 Non-uniformly scaled graphic = \directlua{tex.sprint(mymatrix.toinlinetex(mtx2))}
\vskip 35mm
\noindent\hskip 15mm \copy1003 Rotation $\times$ scale = \directlua{tex.sprint(mymatrix.toinlinetex(combo))}
\vskip 35mm
\noindent\hskip 15mm \copy1004 Rotation $\times$ scale $\times$ translate =\directlua{tex.sprint(mymatrix.toinlinetex(combo2))}
\bye

Resulting PDF

As usual, through the Google Docs viewer or download PDF here.

Downloads

Download all code.

5Dec/11Off

Quick and dirty method for creating spot colours in PDFs

Posted by Graham Douglas

Introduction

Just a 10-minute hack to explore putting spot colours into a PDF via pdf_colorstack nodes. I don't have access to Acrobat Professional at the moment to check the separations properly, so treat this as an "alpha" method (i.e., not fully tested...). The colour defined below is lifted straight from an early PDF specification and implemented via LuaTeX nodes. As it says "on the tin": a quick and dirty method :-).

\pdfoutput=1
\hoffset-1in
\voffset-1in
\nopagenumbers
\pdfcompresslevel=0

\directlua {

n = pdf.immediateobj("stream", "{ dup 0.84 mul
exch 0.00 exch dup 0.44 mul
exch 0.21 mul
}", "/FunctionType 4
/Domain [0.0 1.0]
/Range [0.0 1.0 0.0 1.0 0.0 1.0 0.0 1.0] ")

o = pdf.immediateobj("[ /Separation
/LogoGreen
/DeviceCMYK".." "..n.." 0 R]")

pdf.pageresources =  " /ColorSpace << /LogoGreen "..o.." 0 R >> "

pdf_colstart = node.new("whatsit","pdf_colorstack")
pdf_end = node.new("whatsit","pdf_colorstack")

pdf_colstart.data="/LogoGreen  CS  /LogoGreen cs 1  SC  1 sc "
pdf_colstart.cmd=1

pdf_end.data= " "
pdf_end.cmd = 2

tex.box[1999]= node.hpack(pdf_colstart) 
tex.box[2000]= node.hpack(pdf_end) 

}

\def\makeitgreen#1{\copy1999\relax#1\copy2000\relax}

There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything \makeitgreen{embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to}  repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looks reasonable. The generated Lorem Ipsum is therefore always free from repetition, injected humour, or non-characteristic words etc.

\bye

Resulting PDF

As usual, through the Google Docs viewer or download here.

4Dec/11Off

LuaTeX: visualizing interword glue calculations

Posted by Graham Douglas

Introduction

In this example we will use the "post_linebreak_filter" to iterate over all the lines in a typeset paragraph and use the data provided by LuaTeX to insert pdf_literal nodes which draw a box to show the size of the glue between words. It is based on a nice piece of code on the LuaTeX wiki by (I believe) Patrick Gundlach.

Note: The code does not recurse into nested lists; so if you have further boxes within the lines of your paragraph you should note that this code is not designed to handle them. An exercise for the reader ;-).

If you extend the code to deal with nested boxes, other glue types, such as \baselineskip etc, and account for glyph widths, heights and depths, what this leads to is the ability to calculate precise (x,y) positions within many types of node lists.

Code and explanations

Here, we are placing a paragraph into a \vbox{There are many variations of ....} and hooking into the post_linebreak_filter to access the node list after TeX has broken the paragraph into lines. Each line in the paragraph is accessible from LuaTeX as an hlist which is traversed using the LuaTeX API call node.traverse_id(...). We then scan through the nodes contained in each hlist, looking for glue. As mentioned, the code does not recurse so we are just choosing the individual paragraph lines and not exploring anything below that level.

Each hlist contains its glue setting parameters which let us calculate by how much the individual glue items are stretched or shrunk within each line of the paragraph. From those values you can calculate the size of the set glue and draw an appropriately sized box with a pdf_literal, whose zero size does not affect the positioning of the text. The pdf_literal is inserted into the hlist node tree just before each glue node.

For further reading on TeX's glue calculations see page 77 of The TeXBook.

\pdfoutput=1
\hoffset-1in
\voffset-1in
\nopagenumbers
\directlua{

function scanskips(hlist)

local size=0
% Some debug output
print(string.char(10), "glue_order = "..hlist.glue_order, "glue_set = "..hlist.glue_set, "glue_sign = "..hlist.glue_sign)

id = node.id("glue")

for n in node.traverse_id(id, hlist.head) do 
% Some more debug output
          print("subtype="..n.subtype, "width: ", n.spec.width,  "stretch: ", n.spec.stretch,  "shrink: ", 
                  n.spec.shrink, "stretch order: ", n.spec.stretch_order, 
                  "shrink order: ", n.spec.shrink_order, string.char(10)) 

          order=hlist.glue_order
          set=hlist.glue_set
          sign=hlist.glue_sign

          if sign ==2 then 
                size=n.spec.width - n.spec.shrink*set
          else
                 size=n.spec.width + n.spec.stretch*set
          end

          bp=(size/65536)*(72.0/72.27)

          local pdf = node.new("whatsit","pdf_literal")
          pdf.mode = 0
          pdf.data = "q 0 0 "..bp.." 12 re 0.3 w S Q"
          local prev = n.prev
          prev.next=pdf
          pdf.next = n

      end %for loop
end % function
}

\directlua{
          linelist=function(head)
                    for line in node.traverse_id('hlist',head) do
                              scanskips(line)
                    end
          return head
          end

          callback.register("post_linebreak_filter",linelist)
}
 
\setbox1000=\vbox{\hsize=50mm There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looks reasonable. The generated Lorem Ipsum is therefore always free from repetition, injected humour, or non-characteristic words etc.}
\pdfpagewidth=\wd1000
\vsize=\ht1000
\advance\vsize by 65536 sp
\pdfpageheight=\vsize
\box1000
\bye

Resulting PDF

As usual, through the Google Docs viewer or download here.

Filed under: Examples, LuaTeX Comments Off
2Dec/11Off

Producing printers crop marks with MetaPost and LuaTeX nodes

Posted by Graham Douglas

Introduction

In this article I'll show a technique for producing crop marks using the MetaPost library (MPlib) which is built into LuaTeX. There is a lot of ground to cover so I'll try to focus on/summarise the most important/useful areas to prevent this article becoming far too long! It has already taken a few days to write it and prepare the graphics... I'm not going to attempt a tutorial on the MetaPost language because there are many excellent articles already written by people who are best qualified to produce that material. A great starting point is the TeX Users Group page on MetaPost.

SVG-enabled browser needed: This post uses inline SVG and SVG via in iframe, which may not work in all browsers.

Citing various sources: a few thanks are in order

Much of the material in this post is derived from existing work by members of the LuaTeX community, so I'd like to acknowledge those sources. Firstly, a huge thank you to Hans Hagen for creating the Lua code which converts the MPlib data to raw PDF code. It is a really excellent and hugely useful piece of code which is part of the ConTeXt distribution. In addition, I learnt a lot by reading the source code of luamplib by Hans Hagen, Taco Hoekwater, Elie Roux and Manuel Pégourié-Gonnard. I also discovered some code by Dohyun Kim which makes some very helpful additions to luamplib through which you can support the traditional btex ... etex construct for including TeX within MetaPost code processed by MPlib (more of that in another post).

Pieces of the process

The key elements of the technique I'll describe are:

  1. Embedding MetaPost code (to draw a crop mark) within your LuaTeX document.
  2. Converting the MetaPost output to PDF code (using the code from Hans Hagen's ConTeXt).
  3. Generating pdf_literal nodes representing the crop marks (modification to luamplib).
  4. Placing the crop marks in the appropriate locations through an output routine.

What are crop marks?

Crop marks, also referred to as "printers marks'', "cut marks'' or "trim marks'', are small graphics placed at the corners of a page to indicate the physical size of a final printed document pages. They are used during commercial printing activities, such as page imposition, colour separation, folding and trimming. The physical appearance of crop marks will vary depending on the application used to generate the pages but, of course, with LuaTeX and other TeX engines you are free to create your own designs. Advanced readers will be aware that for multi-colour separations (spot and 4 colour CMYK) the crop marks must appear on all plates but I'll not cover that topic here. It would be fairly easy to do through injecting appropriate PDF code to set the colour space for the crop marks or using the Printer’s Mark Annotations feature of PDF.

The following graphic (produced with Inkscape) shows the general idea.

The following inline SVG graphic (produced directly by exporting from MPlib) shows the design of the crop mark we will be producing via MetaPost code.


The MetaPost code

I'll readily admit that I'm an amateur when it comes to programming with MetaPost so due apologies to any experts reading the code :-). The idea here is that the MetaPost code is inline in the LuaTeX document and I'm using the process_input_buffer callback (see this post) to store the MetaPost code into a buffer which will processed via MPlib. The \startbuffer and \stopbuffer commands were described in an earlier article.

\def\startbuffer{\directlua{callback.register("process_input_buffer",dobuffer)}}
\def\stopbuffer{\directlua{callback.register("process_input_buffer",nil)}}
\startbuffer
beginfig(1);
numeric n,g,tl, alpha; 
pair dl,zc,zl,zr,zt,zb,db;
path c, t;
alpha:=0.75;
n:=2;
g:=72*(3/25.4);
tl:=(n+1)*g;
pickup pensquare scaled 0.5bp;
draw (0,g)--(0,tl);
draw (-g,0)--(-tl,0);
rd:=0.5*tl;
rc:=0.5*rd;
dl=(-(1+alpha)*rc,0);
db=(0,-(1+alpha)*rc);
zc=(-rd,rd);
zl=zc+dl;
zr=zc-dl;
zt=zc+db;
zb=zc-db;
t=zc+(0.9*rc,0)--zc-(0.9*rc,0);
c = fullcircle scaled 2rc shifted(-rd,rd);
pickup pensquare scaled 0.5bp;
draw zl--zr;
draw zt--zb;
fill c withcolor black;
pickup pensquare scaled 1bp;
draw t withcolor white;
draw t  rotatedabout(zc, 90) withcolor white;
endfig;
end;
\stopbuffer%

Very brief introduction to MPlib

As mentioned, MPlib is the library version on the MetaPost interpreter built into LuaTeX. Instead of using MetaPost as a standalone executable (e.g., mpost.exe) you access it through an API provided by the LuaTeX engine. The value of the integration of MetaPost with LuaTeX is well demonstrated by the truly stunning results achieved by Hans Hagen and the ConTeXt distribution. The ambitions of this article are rather more modest.

In outline the steps are as follows.

  1. You need to create a "finder" function which MPlib will call to locate any files it needs.
  2. Provide the finder function as one of the arguments to the API call mplib.new() which
    is responsible for creating an instance of the interpreter.
  3. Load the "format file" containing the macro package you want to use (e.g., plain.mp).
  4. Use the interpreter instance returned by mplib.new() to execure your code.
  5. Process the figure objects generated and returned by MPlib (if your code worked without error).

Here is some sample code for a finder function and creating an instance of the MetaPost interpreter.

-- finder function for MPlib
function finder(name, mode, ftype)
        local found 
                if mode=="w" then found = name else 
                        found = kpse.find_file(name,ftype) 
                end 
                if found then
                        print("MPlib finder: " .. name .. " -> " .. found) 
                end
        return found 
end 

-- create new interpreter instance
function newmp (memname)
        local preamble = "let dump = endinput ; input \%s ;"
        local mp = mplib.new {
            ini_version = true,
            find_file = finder,
        }
        mp:execute(string.format(preamble, memname))
        return mp
 end

MetaPost to PDF or SVG

One nice feature of MPlib is that it will automatically generate an SVG representation of the graphic produced from the MetaPost code, and that is how the inline SVG crop mark (above) was produced. The MPlib library will also PostScript code and an "object representation" of the graphic. The "object representation" can be parsed to convert the graphic into other data formats and this is how the PDF data is generated by the ConTeXt Lua code contained in luamplib. It runs over the collection of objects and converts them to the equivalent representation in PDF data/structures. I really admire that code and it's great to have it available.

Re-using code and ideas in luamplib

Within the luamplib distribution is the core Lua code from ConTeXt which does the "heavy lifting" to convert MPlib data structures to PDF code. For the purposes of the work described in this article I re-used that Lua code and placed it into Lua file (mpnodes.lua) which can be downloaded here. To use it you'll need to load it as a Lua module:

\directlua{require("mpnodes.lua")}

Using mpnodes.lua

The mpnodes.lua module contains functions which create a new instance of the MetaPost interpreter (via MPlib), execute MetaPost code and use the functions from ConTeXt to generate the PDF data. During the process of generating PDF data, the Lua code makes a number of calls to defined TeX macros (originally in luamplib.sty) which, in effect, pass the PDF data to the (Lua)TeX engine. The TeX macros of interest here are:

  • \def\mplibstarttoPDF#1#2#3#4{....}
  • \def\mplibtoPDF#1{...}
  • \def\mplibstoptoPDF{...}

As part of this implementation those macros were redefined to work with LuaTeX nodes. Examples will be provided later in the article.

Note: the mpnodes.lua module also contains other functions which I won't describe: including work based on the code from Dohyun Kim which implements the btex ... etex functionality of the standalone MetaPost interpreter. The MPlib version of MetaPost does not directly support the btex ... etex construct; other methods have to be employed to include TeX code within MetaPost graphics processed by MPlib. Again, those methods are based on the pioneering work of the ConTeXt distribution. I'll write about this in a future post.

Creating nodes to store crop marks

If you look at the figure above, which shows 4 crop marks on a page, you can see that you only need to create 1 crop mark graphic and then rotate it in increments of 90 degrees as you place it at the 4 corners. In the approach, described below, the PDF data generated from the MetaPost graphic is stored as LuaTeX pdf_literal nodes which are then drawn at each corner via the \output routine described below.

Enough description, here's the code

Here's the Plain-TeX-based code with inline comments.

\pdfoutput=1
\pdfcompresslevel=0
\hoffset-1in
\voffset-1in
\pdfpageheight=200mm
\pdfpagewidth=300mm
\vsize=100mm
\hsize=200mm
\topskip=0pt
\maxdepth=0pt

% This output routine centres \box255 horizontally and vertically
% on the PDF page and ships out 4 boxes (2000--2003) onto every page.
% These boxes contain the crop mark graphic, suitably rotated.
% The \output routine is described in the article text.

\output={\shipout\vbox to \pdfpageheight{%
\vfill%
\vbox to \vsize{\offinterlineskip%
\vfill%
\hbox to\pdfpagewidth{\hfill\hbox to \hsize{\copy2000\hfill\copy2001}\hfill}%
\hbox to \pdfpagewidth{\hfill\box255 \hfill}%
\hbox to \pdfpagewidth{\hfill\hbox to \hsize{\copy2003\hfill\copy2002}\hfill}%
\vfill}%
\vfill%
}%pdfpageheight
}%output

% Load the mpnodes.lua module

\directlua{require("mpnodes")}

% Here we redefine the macros in luamplib to 
% work with nodes. 


% \mplibstarttoPDF simply stores the coordinates of the bounding box of the MetaPost
% graphic in 4 TeX tokens, but they are not used in the code below.

\def\mplibstarttoPDF#1#2#3#4{%
\directlua{
		tex.toks[500]=#1
		tex.toks[501]=#2
		tex.toks[502]=#3
		tex.toks[503]=#4		 
%tex.print(tex.toks[500],tex.toks[501],tex.toks[502],tex.toks[503])
}}

% \mplibtoPDF stores each line of PDF data generated from Lua
% and builds it up into one long string. Note I am adding a carriage
% return to each line (string.char(10))

\def\mplibtoPDF#1{%
\directlua{
		buffy= buffy or ""
		buffy=buffy..string.char(10).."#1"
}}

% Here's the main work. Once the PDF data describing the crop mark has been collected 
% the final macro call made by the Lua code is to \mplibstoptoPDF. In this macro 
% we create 4 pdf_literal nodes which only differ by adding extra PDF data to rotate the  
% crop mark in increments of 90 degrees. The PDF data has been collected by \mplibtoPDF
% in a text string called "buffy".

\def\mplibstoptoPDF{%
\directlua{
	rad=math.rad
	sin=math.sin
	cos=math.cos

% Function to generate PDF transformation (rotation) matrices.
function rotate(angle)
	local d=string.format("q 0 0 m \%3.3f  \%3.3f  \%3.3f \%3.3f  0 0 cm ", 
	cos(rad(angle)), 
	sin(rad(angle)),
	-sin(rad(angle)), 
	cos(rad(angle)))
	return d
end

% The names of the pdf_literal nodes reflect their position on the page.
% Note: PDF rotations are counter-clockwise hence negative angles used
% in the code below.

pdftopleft = node.new("whatsit","pdf_literal")
pdftopleft.mode=0
pdftopleft.data=buffy

pdftopright = node.new("whatsit","pdf_literal")
pdftopright.mode=0

rot90=rotate(-90)
print(rot90)%..buffy.." Q")
pdftopright.data= rot90..buffy.." Q"

pdfbottomright = node.new("whatsit","pdf_literal") 
pdfbottomright.mode=0
rot180=rotate(180)
pdfbottomright.data=rot180..buffy.." Q"

pdfbottomleft = node.new("whatsit","pdf_literal") 
pdfbottomleft.mode=0
rot270=rotate(-270)
pdfbottomleft.data=rot270..buffy.." Q"

% We have our nodes now pack them into boxes for shipping
% out via the \output routine as \copy2000....\copy2003

tex.box[2000]= node.hpack(pdftopleft)
tex.box[2001]= node.hpack(pdftopright)
tex.box[2002]= node.hpack(pdfbottomright)
tex.box[2003]= node.hpack(pdfbottomleft)

}}

% Macro to create an instance of the MPlib interpreter and execute
% the MetaPost code collected by the \startbuffer and \stopbuffer macros.
% Lua code is stored in the mpnodes.lua module hence you prefix the functions
% with "mpnodes" to call them: mpnodes.function_name(...)

\def\runmpcode{%
\directlua{
%print(buffer)

%Error checking could be improved here...
mp=mpnodes.newmpx("plain")
res,err=mp:execute(buffer)
%print(res,err)
if res then 
	mpnodes.outputpdf(res) %core function to produce PDF data
		else
		print("No figures")
	end
}}

% Code to implement buffering the inline MetaPost code
\directlua{
function addline(line)
	line=line..string.char(10)
	%print("called with "..line)
	buffer = buffer..line
end}

% Code to implement buffering the inline MetaPost code
\directlua{
buffer=""
function dobuffer(line)
if string.match(line,"stopbuffer") then
	callback.register("process_input_buffer",nil)
	return ""
end
	addline(line)
%	print(line)
	return ""
end
}

% Code to implement buffering the inline MetaPost code through 
% process_input_buffer callback 
\def\startbuffer{\directlua{callback.register("process_input_buffer",dobuffer)}}
\def\stopbuffer{\directlua{callback.register("process_input_buffer",nil)}}
% Here's the inline MetaPost code
\startbuffer
beginfig(1);
numeric n,g,tl, alpha; 
pair dl,zc,zl,zr,zt,zb,db;
path c, t;
alpha:=0.75;
n:=2;
g:=72*(3/25.4);
tl:=(n+1)*g;
pickup pensquare scaled 0.5bp;
draw (0,g)--(0,tl);
draw (-g,0)--(-tl,0);
rd:=0.5*tl;
rc:=0.5*rd;
dl=(-(1+alpha)*rc,0);
db=(0,-(1+alpha)*rc);
zc=(-rd,rd);
zl=zc+dl;
zr=zc-dl;
zt=zc+db;
zb=zc-db;
t=zc+(0.9*rc,0)--zc-(0.9*rc,0);
c = fullcircle scaled 2rc shifted(-rd,rd);
pickup pensquare scaled 0.5bp;
draw zl--zr;
draw zt--zb;
fill c withcolor black;
pickup pensquare scaled 1bp;
draw t withcolor white;
draw t  rotatedabout(zc, 90) withcolor white;
endfig;
end;
\stopbuffer%
\runmpcode
\def\apar{Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, discovered the undoubtable source. Lorem Ipsum comes from sections 1.10.32 and 1.10.33 of "de Finibus Bonorum et Malorum" (The Extremes of Good and Evil) by Cicero, written in 45 BC. This book is a treatise on the theory of ethics, very popular during the Renaissance. The first line of Lorem Ipsum, "Lorem ipsum dolor sit amet.."\par}%

\apar\apar\apar\apar\apar\apar\apar\apar\apar\apar\apar\apar
\bye

Sample PDF

Here's an example PDF produced by the code above.

Have you spotted the flaw?

Readers who have a background in print production may have spotted a problem with the positioning of the crop marks and the text area defined by \box255: there is no white space between the text area and the crop marks, they are too close to the live text material. Can this be fixed? Yes, very easily. What you need to do is insert some vertical kerns to move the crop marks vertically, and offset the crop marks horizontally by adjusting the width of the horizontal boxes containing the crop marks. These actions will create offsets between the crops and the text area. Of course, these will have to be factored into any calculations for your page design but that should not be difficult. What this will mean is that the page size defined by the crop marks would then be:

  • width of page = \hsize + 2*(horizontal offset)
  • height of page = \vsize + 2*(vertical kern offset)

We'll look at this below.

The output routine

A crucial part of the process is to assemble everything onto the final PDF page with the crop marks placed as required. In the following I'll discuss a basic \output routine, that ignores many complications, such as inserts, but which could form the basis for your own experiments. It could, for example, be used as a starting point for with simpler documents such as business cards. I'll also give an example of fixing the "crop marks offset" problem mentioned above.

So where do we start?

The \output routine is responsible for assembling the page components to achieve your desired page design and the way that the following output routine works is by wrapping \box255 in a series of \vboxes and \hboxes with flexible glues to centre \box255 on the page. The overall structure is an outer \vbox to the same size as the height of the PDF page followed by another \vbox to the value of \vsize. Very flexible (stretchy) vertical glue is used to fill the "space" above and below the inner \vbox, i.e., the space that needs to be filled due to the different heights of the two \vboxes (\pdfpageheight versus \vsize). This glue is responsible for vertical centring the inner \vbox.

Inside the second \vbox (the one to \vsize) we place a series of horizontal boxes (to a width of \pdfpagewidth) to contain the crop marks above and below \box255, plus the actual typeset content of \box255 itself. We use horizontal glue to centre everything... horizontally.

\output={\shipout\vbox to \pdfpageheight{%
\vfill%
\vbox to \vsize{\offinterlineskip%
\vfill%
\hbox to\pdfpagewidth{\hfill\hbox to \hsize{\copy2000\hfill\copy2001}\hfill}%
\hbox to \pdfpagewidth{\hfill\box255 \hfill}%
\hbox to \pdfpagewidth{\hfill\hbox to \hsize{\copy2003\hfill\copy2002}\hfill}%
\vfill}%
\vfill%
}%pdfpageheight
}%output

Placing the crop marks

Two of the three "\hboxes to \pdfpagewidth" contain yet another \hbox (of width \hsize) and the purpose of those is to place the crop marks above and below \box255.

\hbox to\pdfpagewidth{\hfill\hbox to \hsize{\copy2000\hfill\copy2001}\hfill}%
\hbox to \pdfpagewidth{\hfill\box255 \hfill}%
\hbox to \pdfpagewidth{\hfill\hbox to \hsize{\copy2003\hfill\copy2002}\hfill}%

Let's take a look at the first one:

\hbox to\pdfpagewidth{\hfill\hbox to \hsize{\copy2000\hfill\copy2001}\hfill}%

The inner \hbox to \hsize{\copy2000\hfill\copy2001} contains an infinitely stretchable \hfill glue which streches to fill the \hbox. The key point is that the pdf_literal nodes from which the boxes 2000 and 2001 have zero width and so the \hfill glue pushes them to the far left and right of the containing \hbox.

If you look back at the Lua code which created the pdf_literal nodes:

pdftopleft = node.new("whatsit","pdf_literal")
pdftopleft.mode=0
pdftopleft.data=buffy

you'll see that the "mode" is set to 0. This defines the origin for drawing them as the point on the page where they appear, which is just above and below \box255 thanks to the actions of the glue.

And finally, offsetting the crop marks

  • Vertically: one solution is simply to absorb some of the space which is occupied by the strechable glues, thus preventing the crop marks being pushed up against \box255.
  • Horizonally: adjust the size of the \hboxes containing the crop marks (make them wider).

Here's one very quick example where we add 20pt offset vertically and increase the \hboxes containing the crop marks to 1.25 x \hsize. A proper solution would of course paramaterise everything.

\output={\shipout\vbox to \pdfpageheight{%
\vfill%
\vbox to \vsize{\offinterlineskip%
\vfill%
\hbox to\pdfpagewidth{\hfill\hbox to 1.25\hsize{\copy2000\hfill\copy2001}\hfill}%
\kern20pt%
\hbox to \pdfpagewidth{\hfill\box255 \hfill}%
\kern20pt%
\hbox to \pdfpagewidth{\hfill\hbox to 1.25\hsize{\copy2003\hfill\copy2002}\hfill}%
\vfill}%
\vfill%
}%pdfpageheight
}%output

And the resulting PDF:

Download mpnode.lua

You can download it here.

Filed under: Examples, LuaTeX Comments Off
1Dec/11Off

Creating PDF pattern fills with LuaTeX nodes

Posted by Graham Douglas

Introduction

This is a short example to introduce two very useful LuaTeX API functions which let you work with low-level PDF objects. Here we'll use them to explore a PDF feature called pattern fills which (from the PDF specification) are a way to

"... apply “paint” that consists of a repeating graphical figure or a smoothly varying color gradient instead of a simple color. Such a repeating figure or smooth gradient is called a pattern. Patterns are quite general, and have many uses; for example, they can be used to create various graphical textures, such as weaves, brick walls, sunbursts, and similar geometrical and chromatic effects"

I'm not going to attempt any explanation of pattern fills because the PDF specification explains them, and their many options, in great detail. Hopefully, these small code examples will be helpful should you want to explore using them in your work with LuaTeX.

LuaTeX API

Here are the functions we'll be using:

pdf.immediateobj(...)
Quoting from The LuaTeX Reference Manual: This function creates a pdf object and immediately writes it to the pdf file. It is modelled after pdfTEX's \immediate\pdfobj primitives. All function variants return the object number of the newly generated object.

  • n = pdf.immediateobj( objtext)
  • n = pdf.immediateobj("file", filename)
  • n = pdf.immediateobj("stream", streamtext, attrtext)

pdf.pageresources =...
This lets you add named resources to the page /Resources dictionary so that they can be used within page content streams.

In outline...

There are two parts to the approach:

  1. Writing the appropriate pattern fill objects and data to the PDF file.
  2. Creating a pdf_literal node to use our newly defined pattern fills.

The code

Here's the \directlua code with in-line comments.

\directlua{
% If you want to quickly view the LuaTeX PDF data in a text editor you should 
% set the compression level to 0. 
tex.pdfobjcompresslevel=0

% We'll use the object number o to store the object reference in the page /Resources dictionary.
o = pdf.immediateobj("[/Pattern /DeviceRGB] ")

% Here we use pdf.immediateobj(...) write the data which actually defines the pattern fill. 
% We'll use the object number n to store the object reference in the page /Resources dictionary.
 
n = pdf.immediateobj("stream", "1 J
.5 w
1 1 4 4 re 5 5 3 3 re f", "/Type /Pattern
/PatternType 1
/PaintType 2
/TilingType 1
/BBox [0 0 10 10]
/XStep 5
/YStep 5
/Resources << >>")

% Here we add the appropriate named resources (for the pattern  fill) to the page /Resources dictionary so
% that we can use the pattern fill in content streams within any page.

pdf.pageresources =  "/Pattern << /P1 "..n.." 0 R >> /ColorSpace << /Cs12 "..o.." 0 R >> "

% Here we create a pdf_literal node which simply draws a box (0 0 12 12  re)
% and fills it using our pattern. Note that if you omit the q ... Q construct to save and 
% restore then the pattern fill will affect other content on your page, with very strange
% results...

pdfdata = node.new("whatsit","pdf_literal")
pdfdata.mode=0
pdfdata.data=" q 0 0 12 12  re  /Cs12 cs 0.77 0.20 0.00 /P1 scn f Q"

% Package our pdf_literal into a box so that we can use it with TeX
% code such as \copy999

tex.box[999]= node.vpack(pdfdata)
}

Full example minus comments

\pdfoutput=1
\pdfcompresslevel=0
\hoffset-1in
\voffset-1in
\pdfpageheight=200mm
\pdfpagewidth=300mm
\vsize=100mm
\hsize=200mm

\directlua{
tex.pdfobjcompresslevel=0

o = pdf.immediateobj("[/Pattern /DeviceRGB] ")

n = pdf.immediateobj("stream", "1 J
.5 w
1 1 4 4 re 5 5 3 3 re f", "/Type /Pattern
/PatternType 1
/PaintType 2
/TilingType 1
/BBox [0 0 10 10]
/XStep 5
/YStep 5
/Resources << >>")

pdf.pageresources =  "/Pattern << /P1 "..n.." 0 R >> /ColorSpace << /Cs12 "..o.." 0 R >> "

pdfdata = node.new("whatsit","pdf_literal")
pdfdata.mode=0
pdfdata.data=" q 0 0 12 12  re  /Cs12 cs 0.77 0.20 0.00 /P1 scn f Q"

tex.box[999]= node.vpack(pdfdata)
}

Hip \copy999 Hip Hooray \copy999

\bye

Resulting PDF

You can download the PDF output from the above example. It displays OK with my version of Adobe Reader (8.2.1 for Windows) and Evince (2.28 for Windows).

28Nov/11Off

Basic example of LuaTeX’s process_input_buffer callback

Posted by Graham Douglas

Introduction

As mentioned in a previous post, LuaTeX provides a facility called callbacks in which you write a Lua function (your callback) and register it with LuaTeX through the callback.register() API function.

callback.register() takes two parameters:

  • A predefined callback name: as defined by LuaTeX. This defines the action or purpose of your of function and is the "hook" into LuaTeX telling it where/why/when your function should be called.
  • Your actual callback function: either a named or anonymous Lua function.

In this post I'll give an example/framework for one simple way to use the process_input_buffer callback. This callback allows you to hook into LuaTeX's input buffer before LuaTeX actually starts looking at it. With this callback you can, for example, completely re-process the input and return the processed results back to LuaTeX, or filter out entire sections of the input–perhaps to store it elsewhere for later use or processing.

Code example and explanation

We will register a callback function which strips out the input sandwiched between two macros called \startbuffer{..} and \stopbuffer. Again, this is a simple example and there are undoubtedly many much more sophisticated ways to do this.

The basic idea here is that the macro

\def\startbuffer#1{%
\directlua{
     dosomething(#1)
     callback.register("process_input_buffer",dobuffer)}
}

switches on intercepting the input, and a dummy macro (\stopbuffer{}) is "intercepted" to switch off processing the input.

There's a very important point here in that after we make a call to \startbuffer{...} all our input bypasses LuaTeX processing from that point on and is passed to the function we have registered: "dobuffer". To detect the end of the buffer, and switch off the callback, dobuffer must scan the input for the string "stopbuffer" and "unregister" the callback so that LuaTeX reverts back to its standard processing of the input. For sure, this approach could fail if the string stopbuffer occurs elsewhere within the input stream. I did say it was a simple example ;-).

In the example you can see a string of characters " ^ & # _ $$ % \" whose \catcode settings could, without their \catcode being reset temporarily, cause problems if input and processed directly by LuaTeX.

\startbuffer{figure1}
Let's have some characters with various catcodes:
^
&
#
_
$$
%
\
\stopbuffer

However, in the example these characters are being intercepted long before LuaTeX starts to process them and so the idea of catcodes, at this point, has no meaning. Of course, they can still trip you up later in your processing.

Explanation

After the call to \startbuffer{...}, every line in the input is passed to the function dobuffer() which in our example does very little except print it out to the screen. Because the dobuffer() function returns the empty string "" the entire input between \startbuffer and \stopbuffer is removed from LuaTeX's input: it never reaches the typesetting engine. You could use this as a simple way to implement long comment sections in your document.

You could implement the dobuffer function to perform all sorts of tasks, such as save a copy of the filtered input and store it into, say, a table perhaps indexed by the value passed into \startbuffer{.....}. The options and possibilities are endless!

\pdfoutput=1
\hoffset-1in
\voffset-1in
\pdfpagewidth=100mm
\pdfpageheight=100mm

\directlua{

function addline(line)
     print("I was called with "..line)
end}

\directlua{

function dobuffer(line)
   if string.match(line,"stopbuffer") then
	callback.register("process_input_buffer",nil)
	return ""
   end
	addline(line)
	return "" % this removes line from LuaTeX's input buffer
end
}

\directlua{
	function dosomething(str)
	% do something useful with the parameter to \startbuffer{.....}
	end
}

\def\startbuffer#1{\directlua{dosomething(#1);callback.register("process_input_buffer",dobuffer)}}
\def\stopbuffer{}

\startbuffer{figure1}
Let's have some characters with various catcodes:
^
&
#
_ 
$$ 
% 
\
\stopbuffer

Hello

\bye
Filed under: Examples, LuaTeX Comments Off