STM publishing: tools, technologies and change A WordPress site for STM Publishing

29Aug/13Off

Building LuaTeX using Microsoft Visual Studio… Why?

Posted by Graham Douglas

Having successfully built Knuthian TeX from WEB source code (using Visual Studio), and a couple of other TeX engines, I finally feel sufficiently "brave" (or perhaps masochistic...) to attempt "the big one": LuaTeX. You may well ask "What's the point when you have MinGW/MSYS to build LuaTeX on Windows for you, and so easily?" Good question! Anyone who has looked at the C code produced by the Web2C process will understand that it's practically impossible to single-step through it with a debugger and understand what's going on: it's machine-generated C code for compilers, not humans! Having built CXTeX (which dates from 2004), it's a joy to step through CXTeX's clean C code in an attempt to understand the innards of TeX in more detail. However, CXTeX is nearly 10 years old so it would be nice to have a modern TeX-based engine to explore, hence building LuaTeX, with its CWEB-based C code base. Well, some 6 hours in I've managed to build about half the components as Windows .lib files – including Cairo, Kpathsea, FontForge, Poppler and a few others. I can't say it's been easy but it's an interesting challenge to occupy the evenings over the next few days. No doubt I'm in for some interesting challenges as I try to assemble the parts into a working LuaTeX executable. Can't wait to have LuaTeX running in Visual Studio's debugging IDE!

13Jul/13Off

Example: Importing a Cairo PDF into XeTeX

Posted by Graham Douglas

A quick example...

Just a quick example of importing the Cairo graphics clock into the latest XeTeX engine (using XeLaTeX). Crop marks were added using a raw \special{pdf: bop ...} command.

Download PDF

13Jul/13Off

Creating a clock with Arabic digits using the Cairo graphics library

Posted by Graham Douglas

Cairo graphics

Cairo is an excellent graphics library, albeit a little tricky to build on Windows. After successfully compiling it as a static library (.lib) I wanted to explore using it to create PDFs containing Arabic. Cairo is a graphics engine, not a text layout engine, so with complex scripts like Arabic you need to take care of the shaping and text placement yourself. Naturally, this is pretty fiddly but it's certainly quite possible. So, here are a couple of clocks as examples – note that the positioning of the numbers is not quite perfect so I have a little tweaking to do on that. Additionally, the resulting PDF imports nicely into the latest XeTeX engine. For the digits I used the font ScheherazadeRegOT, available from SIL.

Download PDF

Download PDF

14Dec/11Off

LuaTeX: nodes, graphics and PDF coordinate transformation matrices

Posted by Graham Douglas

Powered by MathJax

Note: if you want to zoom in on the matrices, right-click over an equation and set the MathJax parameters to your preferred values:

Zooming MathJax

Introduction

In this post I'll introduce a nice matrix-manipulation library called lua-matrix. It is written in pure Lua and so should be usable with any LuaTeX installation. You can download the code from GitHub. You can use lua-matrix as a convenient method to create PDF transformation matrices.

Note: where to install Lua code modules
The texmf.cnf variable you need to check is LUAINPUTS. See this newsgroup post for more details.

Tip: a Git tool for Windows users
Like many open source projects lua-matrix is hosted on GitHub. If you are a Windows user you may need to install some utilities that let you "check out" a copy of the code on repositories such as GitHub or others based on SVN. For SVN repositories there is the excellent TortoiseSVN but for Git repos I use a free tool called Git for Windows.

Very brief summary of matrices

Quoting from the PDF standard:

"PDF represents coordinates in a two-dimensional space. The point (x, y) in such a space can be expressed in vector form as [x y 1]. The constant third element of this vector (1) is needed so that the vector can be used with 3-by-3 matrices. The transformation between two coordinate systems is represented by a 3-by-3 transformation matrix written as

$$\displaystyle \left( \matrix{ a & b & 0 \cr c & d & 0 \cr e & f & 1 \cr} \right)$$

Because a transformation matrix has only six elements that can be changed, it is usually specified in PDF as the six-element array [a b c d e f].

Note: This is method of representing coordinates is referred to as homogeneous coordinates.

Rotation

The matrix for rotation by an angle \(\theta\) counter clockwise about the origin:
$$\displaystyle \left( \matrix{ \cos (\theta) & \sin (\theta) & 0 \cr -\sin (\theta) & \cos (\theta) & 0 \cr 0 & 0 & 1 \cr} \right)$$

This is expressed in PDF code as \(\cos(\theta)\ \sin(\theta)\ -\hskip-2pt\sin(\theta)\ \cos(\theta)\ 0\ 0\) cm

Translation

The matrix for translation by \((t_x, t_y)\) relative to the origin:

$$\displaystyle \left( \matrix{ 1 & 0 & 0 \cr 0 & 1 & 0 \cr t_x & t_y & 1 \cr} \right)$$

This is expressed in PDF code as \(1\ 0\ 0\ 1\ t_x\ t_y\) cm

Scale

The matrix for scaling by \(s_x\) in the horizonal direction and \(s_y\) in the vertical direction is:

$$\displaystyle \left( \matrix{ s_x & 0 & 0 \cr 0 & s_y & 0 \cr 0 & 0 & 1 \cr} \right)$$

This is expressed in PDF code as \(s_x\ 0\ 0\ s_y\ 0\ 0\) cm

Demonstration graphic

The following simple graphic (shown as inline SVG) will be used to explore transformations. The equivalent PDF graphic (in hand-coded PDF data) is shown below.

Equivalent PDF data

The following PDF code will draw a very similar graphic:

q % save graphics state
1 j % set the line join style
1 J % set the line cap style
10 M % set the miter limit
%Set the stroking color space to DeviceRGB
0 0 0 RG % black
% draw the axes
0 0 m
0.5 w
0 30 l
0 0 m
30 0 l
S % stroke
% draw the red arrow head on x axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
1 0 0 rg % red
% translate to end of line on x-axis
1 0 0 1 30 0 cm
% draw an arrowhead
0 0 m % move to the origin
0 1.5 l
2.5 0 l
0 -1.5 l
h % close the current subpath
B % fill and stroke
Q % restore graphics state
% draw the blue arrow head on y axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
0 0 1 rg % blue
% translate to end of line on y-axis
1 0 0 1 0 30 cm
% draw an arrowhead
0 0 m % move to the origin
-1.5 0 l
 0 2.5 l
1.5 0 l
h % close the current subpath
B % fill and stroke
% restore graphics state
Q
Q

Creating a graphic with LuaTeX nodes

As usual, a simple plain TeX setup.

\pdfoutput=1
\pdfcompresslevel=0
\hoffset-1in
\voffset-1in
\pdfpageheight=50mm
\pdfpagewidth=50mm
\vsize=50mm
\hsize=50mm
\topskip=0pt
\maxdepth=0pt
\nopagenumbers

\directlua{
grafik = node.new("whatsit","pdf_literal")
grafik.mode=0
grafik.data="  q % save graphics state
1 j % set the line join style
1 J % set the line cap style
10 M % set the miter limit
%Set the stroking color space to DeviceRGB
0 0 0 RG % black
% draw the axes
0 0 m
0.5 w
0 30 l
0 0 m
30 0 l
S % stroke
% draw the red arrow head on x axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
1 0 0 rg % red
% translate to end of line on x-axis
1 0 0 1 30 0 cm
% draw an arrowhead
0 0 m % move to the origin
0 1.5 l
2.5 0 l
0 -1.5 l
h % close the current subpath
B % fill and stroke
Q % restore graphics state
% draw the blue arrow head on y axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
0 0 1 rg % blue
% translate to end of line on y-axis
1 0 0 1 0 30 cm
% draw an arrowhead
0 0 m % move to the origin
-1.5 0 l
 0 2.5 l
1.5 0 l
h % close the current subpath
B % fill and stroke
% restore graphics state
Q
Q "

tex.box[1000]=node.hpack(grafik)
}

Here is out new graphic \vskip 35mm
\noindent\hskip 15mm \copy1000
\bye

Notes about the PDF graphic

In the code above we have not assigned any size to the box containing the graphic, hence I needed to add \vskip 35mm \noindent\hskip 15mm to push the graphic into a space where it will be seen. To give the graphic some dimensions, we'll need to add code such as

tex.box[1000].width = width value in sp
tex.box[1000].height = height value in sp
tex.box[1000].depth = depth value in sp

where the values assigned are in sp (special points). You may recall that 65536sp = 1 TeX point, where 72.27 TeX points = 1 inch = 72 PostScript points (same as default in PDF).

As far as the LuaTeX engine is concerned, the box containing the graphic has zero size, we have to tell LuaTeX how big we want it to be. In addition, the line widths, based on the above code, will be affected by any scaling but it is not too difficult to fix that.

The Lua code

The idea is that we create a number of functions based on the lua-matrix library and save those functions into a Lua module called "mymatrix.lua". Within "mymatrix.lua" we import the lua-matrix code via its module called "matrix" which we load with:

local matrix=require("matrix")

Our simple API

Here are the functions defined within our "mymatrix.lua" module:

  • rotate(angle): returns a 3 x 3 rotation matrix (angle positive for counter clockwise)
  • translate(tx,ty): returns a 3 x 3 translation matrix (tx, ty are translations in x, y directions)
  • scale(sx,sy): returns a 3 x 3 scaling matrix (sx, sy are scaling in x,y directions)
  • dump(mtx): simple debugging function (mtx = 3 x 3 matrix)
  • toinlinetex(mtx): returns inline TeX code so we can typeset the matrix (mtx = 3 x 3 matrix)
  • todisplaytex(mtx): returns display TeX code so we can typeset the matrix (mtx = 3 x 3 matrix)
  • topdf(mtx): returns matrix in PDF code format (mtx = 3 x 3 matrix)

Here's the source code for mymodule.lua. One huge advantage of putting Lua code into Lua modules is that it greatly simplifies dealing with \catcode issues. Note that within code saved in Lua files you use the regular Lua comment "--" mechanism and not the TeX comment "%" mechanism. You can use "%" when the code is embedded in a \directlua{...} call.

module("mymatrix",package.seeall)
local matrix=require("matrix")
local rad=math.rad
local sin=math.sin
local cos=math.cos

-- Function to generate PDF transformation (rotation) matrices.
function rotate(angle)
local rot = matrix {{cos(rad(angle)),sin(rad(angle)),0},{-sin(rad(angle)),cos(rad(angle)),0},{0,0,1}}
return rot
end

-- Function to generate PDF transformation (translation) matrices.
function translate(tx,ty)
local tran = matrix {{1,0,0},{0,1,0},{tx,ty,1}}
return tran
end

-- Function to generate PDF transformation (scale) matrices.
function scale(sx,sy)
local scale = matrix {{sx,0,0},{0,sy,0},{0,0,1}}
return scale
end

function dump(mtx)
for i=1,3 do
	for j=1,3 do
		print("(i,j)=("..i..","..j..")="..mtx[i][j])
end
end
end

function todisplaytex(mtx)
texcode=string.format([[$$\left(\matrix {%3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr } \right)$$]],
	mtx[1][1], mtx[1][2], mtx[1][3],mtx[2][1], mtx[2][2], mtx[2][3],
	mtx[3][1], mtx[3][2], mtx[3][3])
	return texcode
end

function toinlinetex(mtx)
texcode=string.format([[$\left(\matrix {%3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr %3.3f & %3.3f & %3.3f \cr } \right)$]],
	mtx[1][1], mtx[1][2], mtx[1][3],mtx[2][1], mtx[2][2], mtx[2][3],
	mtx[3][1], mtx[3][2], mtx[3][3])
	return texcode
end

function topdf(mtx)
pdftext = string.format("%3.3f %3.3f %3.3f %3.3f %3.3f %3.3f  cm",
	mtx[1][1], mtx[1][2],mtx[2][1], mtx[2][2], mtx[3][1], mtx[3][2])
	return pdftext
end

Full example

\pdfoutput=1
\pdfcompresslevel=0
\hoffset-1in
\voffset-1in
\pdfpageheight=350mm
\pdfpagewidth=150mm
\vsize=350mm
\hsize=150mm
\topskip=0pt
\maxdepth=0pt
\nopagenumbers
\directlua{require("mymatrix")}

\directlua{
% We'll create an non-transformed graphic and store it in box 1000
grafik = node.new("whatsit","pdf_literal")
grafik.mode=0
grafik.data="
q % save graphics state
1 j % set the line join style
1 J % set the line cap style
10 M % set the miter limit
%Set the stroking color space to DeviceRGB
0 0 0 RG % black
% draw the axes
0 0 m
0.5 w
0 30 l
0 0 m
30 0 l
S % stroke
% draw the red arrow head on x axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
1 0 0 rg % red
% translate to end of line on x-axis
1 0 0 1 30 0 cm
% draw an arrowhead
0 0 m % move to the origin
0 1.5 l
2.5 0 l
0 -1.5 l
h % close the current subpath
B % fill and stroke
Q % restore graphics state
% draw the blue arrow head on y axis
q % save graphics state
%Set the non-stroking color space to DeviceRGB
0 0 1 rg % blue
% translate to end of line on y-axis
1 0 0 1 0 30 cm
% draw an arrowhead
0 0 m % move to the origin
-1.5 0 l
 0 2.5 l
1.5 0 l
h % close the current subpath
B % fill and stroke
% restore graphics state
Q
Q "
% store graphic in box 1000
tex.box[1000]=node.hpack(grafik)

% create some transformation matrices
mtx1 = mymatrix.rotate(45)
mtx2 = mymatrix.scale(1.5,2)
mtx3 = mymatrix.translate(15,15)

% Now we copy our untransformed node
% and add the PDF transformation matrices to
% rotate, scale etc.

% We'll do a rotation and store in box 1001
grafik2 = node.copy(grafik)
grafik2.mode=0
cm = mymatrix.topdf(mtx1)
% copy the PDF data from the untransformed
% graphic and add the PDF rotation matrix
grafik2.data="q ".. cm ..grafik.data.." Q "
% store graphic in box 1001
tex.box[1001]=node.hpack(grafik2)

% We'll do a scale and store in box 1002
grafik3 = node.copy(grafik)
grafik3.mode=0
cm = mymatrix.topdf(mtx2)
% copy the PDF data from the untransformed
% graphic and add the PDF rotation matrix
grafik3.data="q ".. cm ..grafik.data.." Q "
% store graphic in box 1002
tex.box[1002]=node.hpack(grafik3)

% Now multiply the scale and rotation matrices
% --- experiment with different combinations
combo=mtx1*mtx2

grafik4 = node.copy(grafik)
grafik4.mode=0
cm = mymatrix.topdf(combo)
% copy the PDF data from the untransformed
% graphic and add the result of multiplication
grafik4.data="q ".. cm ..grafik.data.." Q "

% store graphic in box 1002
tex.box[1003]=node.hpack(grafik4)

% Now multiply the scale, rotation and translation
% matrices --- experiment with different combinations
combo2=mtx1*(mtx2*mtx3)

grafik5 = node.copy(grafik)
grafik5.mode=0
cm = mymatrix.topdf(combo2)
% copy the PDF data from the untransformed
% graphic and add the result of multiplication
grafik5.data="q ".. cm ..grafik.data.." Q "

% store graphic in box 1002
tex.box[1004]=node.hpack(grafik5)

}

Here are our graphics \vskip 35mm
\noindent\hskip 15mm \copy1000 Default graphic
\vskip 35mm
\noindent\hskip 15mm \copy1001 Rotated graphic =  \directlua{tex.sprint(mymatrix.toinlinetex(mtx1))}
\vskip 35mm
\noindent\hskip 15mm \copy1002 Non-uniformly scaled graphic = \directlua{tex.sprint(mymatrix.toinlinetex(mtx2))}
\vskip 35mm
\noindent\hskip 15mm \copy1003 Rotation $\times$ scale = \directlua{tex.sprint(mymatrix.toinlinetex(combo))}
\vskip 35mm
\noindent\hskip 15mm \copy1004 Rotation $\times$ scale $\times$ translate =\directlua{tex.sprint(mymatrix.toinlinetex(combo2))}
\bye

Resulting PDF

As usual, through the Google Docs viewer or download PDF here.

Downloads

Download all code.

5Dec/11Off

Quick and dirty method for creating spot colours in PDFs

Posted by Graham Douglas

Introduction

Just a 10-minute hack to explore putting spot colours into a PDF via pdf_colorstack nodes. I don't have access to Acrobat Professional at the moment to check the separations properly, so treat this as an "alpha" method (i.e., not fully tested...). The colour defined below is lifted straight from an early PDF specification and implemented via LuaTeX nodes. As it says "on the tin": a quick and dirty method :-) .

\pdfoutput=1
\hoffset-1in
\voffset-1in
\nopagenumbers
\pdfcompresslevel=0

\directlua {

n = pdf.immediateobj("stream", "{ dup 0.84 mul
exch 0.00 exch dup 0.44 mul
exch 0.21 mul
}", "/FunctionType 4
/Domain [0.0 1.0]
/Range [0.0 1.0 0.0 1.0 0.0 1.0 0.0 1.0] ")

o = pdf.immediateobj("[ /Separation
/LogoGreen
/DeviceCMYK".." "..n.." 0 R]")

pdf.pageresources =  " /ColorSpace << /LogoGreen "..o.." 0 R >> "

pdf_colstart = node.new("whatsit","pdf_colorstack")
pdf_end = node.new("whatsit","pdf_colorstack")

pdf_colstart.data="/LogoGreen  CS  /LogoGreen cs 1  SC  1 sc "
pdf_colstart.cmd=1

pdf_end.data= " "
pdf_end.cmd = 2

tex.box[1999]= node.hpack(pdf_colstart)
tex.box[2000]= node.hpack(pdf_end) 

}

\def\makeitgreen#1{\copy1999\relax#1\copy2000\relax}

There are many variations of passages of Lorem Ipsum available, but the majority have suffered alteration in some form, by injected humour, or randomised words which don't look even slightly believable. If you are going to use a passage of Lorem Ipsum, you need to be sure there isn't anything \makeitgreen{embarrassing hidden in the middle of text. All the Lorem Ipsum generators on the Internet tend to}  repeat predefined chunks as necessary, making this the first true generator on the Internet. It uses a dictionary of over 200 Latin words, combined with a handful of model sentence structures, to generate Lorem Ipsum which looks reasonable. The generated Lorem Ipsum is therefore always free from repetition, injected humour, or non-characteristic words etc.

\bye

Resulting PDF

As usual, through the Google Docs viewer or download here.

1Dec/11Off

Creating PDF pattern fills with LuaTeX nodes

Posted by Graham Douglas

Introduction

This is a short example to introduce two very useful LuaTeX API functions which let you work with low-level PDF objects. Here we'll use them to explore a PDF feature called pattern fills which (from the PDF specification) are a way to

"... apply “paint” that consists of a repeating graphical figure or a smoothly varying color gradient instead of a simple color. Such a repeating figure or smooth gradient is called a pattern. Patterns are quite general, and have many uses; for example, they can be used to create various graphical textures, such as weaves, brick walls, sunbursts, and similar geometrical and chromatic effects"

I'm not going to attempt any explanation of pattern fills because the PDF specification explains them, and their many options, in great detail. Hopefully, these small code examples will be helpful should you want to explore using them in your work with LuaTeX.

LuaTeX API

Here are the functions we'll be using:

pdf.immediateobj(...)
Quoting from The LuaTeX Reference Manual: This function creates a pdf object and immediately writes it to the pdf file. It is modelled after pdfTEX's \immediate\pdfobj primitives. All function variants return the object number of the newly generated object.

  • n = pdf.immediateobj( objtext)
  • n = pdf.immediateobj("file", filename)
  • n = pdf.immediateobj("stream", streamtext, attrtext)

pdf.pageresources =...
This lets you add named resources to the page /Resources dictionary so that they can be used within page content streams.

In outline...

There are two parts to the approach:

  1. Writing the appropriate pattern fill objects and data to the PDF file.
  2. Creating a pdf_literal node to use our newly defined pattern fills.

The code

Here's the \directlua code with in-line comments.

\directlua{
% If you want to quickly view the LuaTeX PDF data in a text editor you should
% set the compression level to 0.
tex.pdfobjcompresslevel=0

% We'll use the object number o to store the object reference in the page /Resources dictionary.
o = pdf.immediateobj("[/Pattern /DeviceRGB] ")

% Here we use pdf.immediateobj(...) write the data which actually defines the pattern fill.
% We'll use the object number n to store the object reference in the page /Resources dictionary.

n = pdf.immediateobj("stream", "1 J
.5 w
1 1 4 4 re 5 5 3 3 re f", "/Type /Pattern
/PatternType 1
/PaintType 2
/TilingType 1
/BBox [0 0 10 10]
/XStep 5
/YStep 5
/Resources << >>")

% Here we add the appropriate named resources (for the pattern  fill) to the page /Resources dictionary so
% that we can use the pattern fill in content streams within any page.

pdf.pageresources =  "/Pattern << /P1 "..n.." 0 R >> /ColorSpace << /Cs12 "..o.." 0 R >> "

% Here we create a pdf_literal node which simply draws a box (0 0 12 12  re)
% and fills it using our pattern. Note that if you omit the q ... Q construct to save and
% restore then the pattern fill will affect other content on your page, with very strange
% results...

pdfdata = node.new("whatsit","pdf_literal")
pdfdata.mode=0
pdfdata.data=" q 0 0 12 12  re  /Cs12 cs 0.77 0.20 0.00 /P1 scn f Q"

% Package our pdf_literal into a box so that we can use it with TeX
% code such as \copy999

tex.box[999]= node.vpack(pdfdata)
}

Full example minus comments

\pdfoutput=1
\pdfcompresslevel=0
\hoffset-1in
\voffset-1in
\pdfpageheight=200mm
\pdfpagewidth=300mm
\vsize=100mm
\hsize=200mm

\directlua{
tex.pdfobjcompresslevel=0

o = pdf.immediateobj("[/Pattern /DeviceRGB] ")

n = pdf.immediateobj("stream", "1 J
.5 w
1 1 4 4 re 5 5 3 3 re f", "/Type /Pattern
/PatternType 1
/PaintType 2
/TilingType 1
/BBox [0 0 10 10]
/XStep 5
/YStep 5
/Resources << >>")

pdf.pageresources =  "/Pattern << /P1 "..n.." 0 R >> /ColorSpace << /Cs12 "..o.." 0 R >> "

pdfdata = node.new("whatsit","pdf_literal")
pdfdata.mode=0
pdfdata.data=" q 0 0 12 12  re  /Cs12 cs 0.77 0.20 0.00 /P1 scn f Q"

tex.box[999]= node.vpack(pdfdata)
}

Hip \copy999 Hip Hooray \copy999

\bye

Resulting PDF

You can download the PDF output from the above example. It displays OK with my version of Adobe Reader (8.2.1 for Windows) and Evince (2.28 for Windows).