The pipeline

by Duncan Wilcox, Jun 16, 2016

This post is of the more technical kind. The opportunity is to explain the inner workings of the pipeline that translates your canvas layout into a set of files and assets, with stunning speed in preview and accuracy plus precision in publishing.

Let’s start with the blazingly fast preview. If you have tried just about any other website builder you’ll have noticed that the preview step is invariably sluggish, so much so that sometimes you’re deterred from trying.

Contrast this with Sparkle. You tweak an element in the canvas and the preview is instant, it updates immediately. It’s so fast you don’t really need to think about it, it just happens. You can leave the preview open all the time. In fact you can leave it open in multiple browser, say one in a secondary Mac screen, one on an iPad and one an on iPhone, and check them all in real time.

How is this possible? How can Sparkle produce the web page so quickly? What sort of magic?

First, Sparkle has a built-in web server, so the preview browsers connect to that. As an example of a bit of a hack, I used to show the preview server, and it works fine.

The benefit of a built-in web server is we can play tricks. So what is the problem to solve to make previews lightning fast?

Here’s how you can think about it. The speed of the Mac CPU is a bottleneck, some tasks are easier and some are harder. In general resizing images is the most time consuming task, that slows down preview generation, so let’s refer to image resizing for simplicity.

If every time Sparkle updated the preview it processed all the images, there would be no way for it to be nearly as quick, the images just can’t go through the bottleneck at the required speed. Additionally there could be a large number number of images to be resized, for example 100 images in a gallery, resized once for each pixel density (@2x/@3x) and each one resized for each device.

The trick is to do none of the hard work. Easy! Done!

Ok, you might have guessed that can’t be true. Let’s say it’s a partial truth. Producing the content, layout and interaction files is actually pretty quick, it’s a lot of text and isn’t too complicated, so that’s done every time. But producing images is hard so at a first pass it is skipped, only a reference to the original image is sent to the preview browser, and everything zip right through the bottleneck.

Sparkle holds on to the image settings, so that when the preview browser requests the image, it will be resized on the fly.

The beauty of this approach is there are a number of instances where the preview browser will not request the images at all. For example:

  • images that have a different pixel density than the preview browser’s screen
  • images that are for different devices
  • images that are lazily loaded and aren’t in view
  • images in a gallery that aren’t being shown (because galleries load the images lazily, when shown)

Most importantly, browsers won’t request images they have cached, just like images from a website, so unless you change an image in a way that requires it to be re-rendered, the browser won’t actually ask Sparkle’s web server for the image, and it won’t be re-generated.

This results in a very snappy preview service, with nearly no unnecessary resizing of images. Because Sparkle’s preview server is multithreaded, the incoming requests are parallelized and Sparkle takes less time to generate all the content and images, resulting in shorter “full throttle” CPU usage and ultimately in lower power consumption.

Here I played around with live preview loaded with, showing four concurrent live previews in four iframes, each with its own device size. You’ll note that switching to the mobile layout, only the mobile preview is re-generated.

When publishing to a website or exporting to disk the process is very similar, except every asset is generated every time.

Copyright © Crinon SRL 


This website makes use of cookies. Please see our privacy policy for details.