Scripting DOM: Changes and Additions

It is uneasy to trace the changes made to the Scripting DOM with the advent of MathML in InDesign. I used GitHub Desktop features to reveal these recent transformations.

Application (app)

METHOD	TYPE	COMMENT
getContextMathMLDescription()	→ str	(Internal.) Used by UXP mathjax-interface.
getPathToExportMml2svg()	→ str	(Internal.) Used by UXP mathjax-interface.
handleMathMLMessage(resync)	→ und	(Internal.) Used by MathExpr panel.

All MathML methods exposed to app are for “internal use only”. This does indicate that the Math Expressions panel and the nested WebView component (UXP plug-in) make heavy use of InDesign's scripting API. We will see that this has significant consequences on execution time.

Document

METHOD / PROPERTY	TYPE	COMMENT
mathObjects	MathObjects	Collection of available Math objects.
appliedMathMLFontSize	number	(R/W) Font size (pt) for Math objects.
appliedMathMLSwatch	Swatch/str	(R/W) Swatch for Math object color.
appliedMathMLRgbColor	number[3]	(R/W) RGB Color applied on MO, 0-255 range.
tintValue	0..100	(R/W) “Percent of base color.”
createFromMathML(mml,pge,lyr,pos)	→ MathObject	Creates and places a new MO from a MathML description.
handleMathMLMessage(resync)	→ und	(Internal.) Used by MathExpr panel.

Note. - The weird tintValue property (added to Document instances) actually controls the default tint percentage in Math Expressions ⏵ Fill (popup).

myDoc.createFromMathML(...) expects four mandatory arguments: a MathML description (string), a Page and a Layer instance, then a [x,y] position in Ruler coordinates. This position determines the top-left corner of the inner MathObject (MO) instance, which seems to behave as a SVG object but would be better described as a “SVG proxy” (see below.)

Rectangle

PROPERTY	TYPE	COMMENT
mathObjects	MathObjects	Collection of available Math objects.

Rectangle is the only PageItem subclass that supports the new .mathObjects collection, implying that the parent of a MO is necessarily a rectangle. The fun fact is that InDesign users could well transform that container into any polygon through Object ⏵ Convert Shape... In which circumstances you can no longer access the underlying MO from its container. (Also, the MO might export incorrectly!)

As reported by my Japanese colleague ホーム (Ten—A) on his excellent blog (JP), the mathObjects collection of the Rectangle class does not actually allow you to insert a MO inside any existing rectangle. When you try the below snippet, you get a new parent rectangle on the first page of the document.

var mml ='''<math>
<mrow>
  <mi>x</mi>
  <mo>=</mo>
  <mfrac>
    <mrow>
      <mrow>
        <mo>-</mo>
        <mi>b</mi>
      </mrow>
      <mo>&PlusMinus;</mo>
      <msqrt>
        <mrow>
          <msup>
            <mi>b</mi>
            <mn>2</mn>
          </msup>
          <mo>-</mo>
          <mrow>
            <mn>4</mn>
            <mo>&InvisibleTimes;</mo>
            <mi>a</mi>
            <mo>&InvisibleTimes;</mo>
            <mi>c</mi>
          </mrow>
        </mrow>
      </msqrt>
    </mrow>
    <mrow>
      <mn>2</mn>
      <mo>&InvisibleTimes;</mo>
      <mi>a</mi>
    </mrow>
  </mfrac>
</mrow>
</math>''';
 
// Having a Rectangle selected:
var rec = app.selection[0];
 
var mob = rec.mathObjects.add(mml);
// ⚠ The MathObject is NOT created inside the rectangle :-(
 

This is a serious inconsistency. The best you can do is to provide additional arguments to get the new MathObject placed at the rectangle location, e.g.

// . . .
var mob = rec.mathObjects.add(
  mml,
  rec.parentPage,
  rec.itemLayer,
  rec.geometricBounds.slice(0,2).reverse()
);
 

but you still won't get the mob.parent===rec relationship you were hoping for.

Graphic

PROPERTY	TYPE	COMMENT
parent	various	Can now include Spread.

Note. - In principle, a Graphic refers to an “imported graphic in any graphic file format (including vector, metafile, and bitmap formats).”

Making the Spread object a possible parent of Graphic seems quite insignificant, but everything suggests that this option coincides with the insertion of MathObject into the InDesign graphics space. I have not yet found the exact reason for this amendment though.

SVG (subclass of Graphic)

METHOD / PROPERTY	TYPE	COMMENT
isMathMLObject	bool	“Is the SVG Object a MathML object.”
parent	various	Can now include Spread.
resync(data)	→ und	Internal use.

Surprisingly, MathObject instances are not reported in regular SVGS collection. So MathObject and SVG entities are clearly distinct from the DOM standpoint. Even worse, MOs are not registered in myDoc.allGraphics, so you definitely need to explore the special myDoc.mathObjects collection. Yet the SVG class exposes an obscure isMathMLObject boolean property now, which suggests that both classes may inherit similar features from Graphic. In fact, they don't.

My working hypothesis is that MathObject is intended to (drastically) restrict the standard SVG prototype, including disallowing access to any internal link (i.e. the itemLink key of regular Graphic classes) and entirely removing transform features. This explains why the new MathObject entity, though semantically assimilated to a placeable Graphic, only provides the following, minimalistic interface:

MathObject

METHOD / PROPERTY	TYPE	COMMENT
isMathMLObject	bool	“Is the SVG Object a MathML object.”
id	uint	Unique ID of the MathObject.
isValid	bool	Whether its specifier is valid.
parent	Rectangle	⚠ The parent of the MO should be a Rectangle.
index	uint	Index of the MO within its containing object.
events	Events	Collection of events.
eventListeners	EventListeners	Collection of event listeners.
appliedMathMLFontSize	number	(R/W) Font size in pt.
appliedMathMLSwatch	Swatch/str	(R/W) Swatch for Math object color.
appliedMathMLRgbColor	number[3]	(R/W) RGB Color applied on MO, 0-255 range.
tintValue	0..100	(R/W) Percent of base color.
mathmlDescription	str	(R/W) “MathML description of the SVG Object. Returns empty string if not a MathML Object.”
label	str	(R/W) Property that can be set to any string.
name	str	Name of the MathObject; “this is an alias to the MathObject's label property”.
properties	obj	(R/W) Property that allows setting of several properties at the same time.
insertLabel(key,val)	→ und	Sets the label to the value associated with the specified key.
extractLabel(key)	→ str	Gets the label value associated with the specified key.
toSource()	→ str	Generates a string which, if executed, will return the MathObject.
getElements()	→ MathObject[]	Resolves the object specifier, creating an array of object references.
toSpecifier()	→ str	Retrieves the object specifier as a string.
addEventListener(...)	→ EventListener	Adds an event listener.
removeEventListener(...)	→ bool	Removes an event listener.

Note that the informative property isMathMLObject is the only one that belongs specifically to both SVG and MathObject classes. But given a Rectangle with a MathObject, the contentType of the container remains UNASSIGNED and it doesn't appear that you can access the internal MO other than through the myRec.mathObjects collection. So it is difficult to elucidate what isMathMLObject could be used for!

• A MathObjects collection was created too, in correspondence with the MathObject structure; it provides the common Collection interface — everyItem(), itemByID(), etc — including the add(...) method which works as specified in Document.createFromMathML(...) and without regard to the collection container.

• An interesting feature of MathObject instances is that they are shown in the Layer panel within their container (Rectangle), so the GUI allows the user to hide them separately. (The same treatment can only be applied to Graphics via a script.)

• If the user rescales/distorts a MO from the GUI, you won't be able to access its transform state via rotationAngle, transformValuesOf(), etc.

More on the SVG Mystery

For the record, SVG Support dates back to InDesign 2020 and the SVG object is visible in the DOM since version 15.0 or thereabouts. The case of MathObject raises the question of access to the XML stream constituting the SVG file.

As we have just seen, MathObject as a placeable-like entity should expose Graphic properties, but it does not. Its container, a Rectangle, has its svgs.length==0 although a “ghost SVG” is actually involved.

Everything happens in Math Expressions, a UXP panel that takes a very complicated path to generate a hidden SVG file through MathJax. You need to inspect com.adobe.indesign.mathexprpanel in InDesign's Resources/UXP directory to reconstruct the process. Additionally, there is a separate mathjax resource (which in turn contains the entire MathJax distribution) that provides a mathjax-interface.idjs script. The latter calls the Scripting DOM and is very likely our best entry point to the secret SVG file:

// In InDesign/Resources/mathjax/mathjax-interface.idjs
 
// . . .
let sample = app.getContextMathMLDescription();
let pathToExport = app.getPathToExportMml2svg();
 
// . . .
//  Create DOM adaptor and register it for HTML documents
const adaptor = liteAdaptor();
const handler = RegisterHTMLHandler(adaptor);
 
//  Create input and output jax...
const mml = new MathML();
const svg = new SVG({fontCache: 'none'});
const html = mathjax.document('', {InputJax:mml, OutputJax:svg});
const node = html.convert(sample);
let innerhtml = adaptor.innerHTML(node);
 
let resultStr = {
  width:node.children[0].attributes.width,
  height:node.children[0].attributes.height,
  svg_str:innerhtml
};
 
fs.writeFileSync(pathToExport, JSON.stringify(resultStr));
 

The last line of this snippet shows that a temporary file is created at the location returned by app.getPathToExportMml2svg(). (Obviously, this method only returns a edible path in the so-called “internal usage” context.)

WebView Detour

Regarding the Math Expressions panel, it essentially relies on a WebView, that is, a browser window displayed inside the UXP plugin. “WebViews can be used to display complex web pages inside the UXP plugins. You can use it to access external web services, to create custom UI and to isolate the web content from the rest of the plugin.”

   → developer.adobe.com/indesign/uxp/reference

“WebView support has been introduced in UXP quite recently, adds Davide Barranca, first in modal dialogues only, then, reluctantly, in modeless panels as well. While developers were requesting the WebView sometimes as a way to work around UXP limitations (in other words, trying to re-create a browser CEP-like environment in disguise), Adobe did express concerns about system resources consumption and loading time, which were the leading causes for the CEP's demise.”

   → Professional Photoshop UXP, p. 160.

The most important thing to remember is this sentence from Kerri Shotts: “Webviews do incur a fairly heavy memory and performance hit.”

   → forums.creativeclouddeveloper.com

When you setup a communication channel between WebView and some plugin, the data is sent via the postMessage() method and the whole processing is done, in both directions, from event listeners.

This is what you will find if you browse the structure of com.adobe.indesign.mathexprpanel:

• manifest.json sets the webview permissions: enableMessageBridge: "localAndRemote"

• index.html includes index.js and declares the <webview> component, setting the essential uxpAllowInspector="true" attribute.

• index.js contains the basic UXP stuff and installs the event listeners of the communication channel:

// Skeleton of
// Resources/UXP/com.adobe.indesign.mathexprpanel/index.js
 
const uxp = require('uxp');
const fs = require('fs');
const { app } = require("indesign");
function showAlert(str){...}
 
// TO WebView
document.body.addEventListener('uxpcommand', event =>
{
   var data = event.data;
   let webView = document.getElementById("panelWebview");
   // . . .
   webView.postMessage(event.data);
})
 
// FROM WebView
window.addEventListener("message", (e) =>
{
   // . . .
   // Typical call of InDesign DOM API:
   app.handleMathMLMessage(msg);
   // . . .
});
 

• test.html (as its name does not indicate!) integrates the WebView itself, that is, a kind of encapsulated, local HTML page that can talk to MathJax:

// In Resources/UXP/com.adobe.indesign.mathexprpanel/test.html
 
<script src="polyfill.min.js" />
<script id="MathJax-script" async src="./mathjax/es5/mml-svg.js" />
<script src="Events.js" />
<script>MathJax={ loader:{load:['[mml]/mml3']}, mml:{forceReparse:true} };</script>
 
// etc
 

Note. - The test.html name is another good reason to think MathObject is not finished! We really feel that the InDesign team was content to illustrate a PoC.

• Finally, Events.js handles all internal events, syntax errors, formatting of MathML descriptors, and so on. It sends messages back to the UXP host using window.uxpHost.postMessage(...)

These few insights at least allow you to measure the extraordinary interweaving of technologies to achieve the goal, i.e. produce some SVG stream from a MathML syntax. Should we encounter bugs or discomfort in the Math Expressions panel, we now know where to hack it!

Taking a shortcut to MathJax!

The most interesting code pattern is probably the mathjax-interface.idjs script I already mentioned. It is likely that one could set breakpoints there and extract the temporary location of the SVG file to, for example, copy and transform it into a real Graphic itemlink.

Better, you can invoke MathJax for your own needs from an idjs script. Well, this still involves copying the library into your project in order to make the require mantra work. The following code shows how to generate a “flesh and blood” SVG file from a MathML expression.

let { app } = require("indesign");
 
const fs = require("uxp").storage.localFileSystem;
 
// Assuming your project contains the [mathjax] distrib.
const path = './mathjax/mathjax_modules/';
const {mathjax} = require(path + 'mathjax-full/js/mathjax.js');
const {MathML} = require(path + 'mathjax-full/js/input/mathml.js');
const {SVG} = require(path + 'mathjax-full/js/output/svg.js');
const {liteAdaptor} = require(path + 'mathjax-full/js/adaptors/liteAdaptor.js');
const {RegisterHTMLHandler} = require(path + 'mathjax-full/js/handlers/html.js');
 
function showAlert(str)
{
  const dialog = app.dialogs.add();
  const col = dialog.dialogColumns.add();
  const colText = col.staticTexts.add();
  colText.staticLabel = ''+str;
 
  dialog.canCancel = false;
  dialog.show();
  dialog.destroy();
  return;
}
 
try
{
const sample = `<math>
  <mrow>
    <mi> x </mi>
    <mo> + </mo>
    <mrow>
      <mi> a </mi>
      <mo> / </mo>
      <mi> b </mi>
    </mrow>
  </mrow>
</math>`;
 
// Create DOM adaptor and register it for HTML docs.
const adaptor = liteAdaptor();
const handler = RegisterHTMLHandler(adaptor);
 
// Create input and output structures.
const mml = new MathML();
const svg = new SVG({fontCache: 'none'});
const html = mathjax.document('', {InputJax: mml, OutputJax: svg});
 
const node = html.convert(sample);
let svgStr = adaptor.innerHTML(node); // `<svg...>...</svg>`
 
// const nodeAtt = node.children[0].attributes;
// If needed => { width:'...', height:'...', viewBox:'...', ... }
 
const fd = await fs.getTemporaryFolder();
const ff = await fd.createFile("test.svg", {overwrite:true});
ff.write(svgStr);
 
} catch(e) { showAlert(e) }
 

You'll have no trouble imagining what's possible next. Note, by the way, that MathJax offers much more extensive functionality than MathML-to-SVG conversion — think LaTeX and AsciiMath.

But rather than digress, let's return to the crucial limitation that you should never overlook: the InDesign API for accessing MathJax and creating MathObjects is excessively slow. This is due to the complexity of the dataflow, the interconnection between DOM and UXP layers, and the event-driven implementation:

If you plan to automate the insertion of equations into a large InDesign document, expect commands like
   myDoc.mathObjects.everyItem().appliedMathMLFontSize = 16;
to create fatal issues. We are still far from having a robust and reliable workflow for quality scientific publishing.

GO BACK TO PART 1

Useful links (UXP):
• About UXP WebView
• WebViews and manifest.json
• More on WebViewPermission
• Discussion about “Embedded WebView”
• WebView and postMessage
• On the Logics of postMessage (MDN)

See also:
• MathJax Documentation
• MathJax Source Code
• MathJax: Non-Component-Based Examples
• “Professional Photoshop UXP” (review)
• Adobe XD Platform: storage