How We Wrote a Desktop Quality App in JavaScript That Avoids Framework Hell

http://negativespace.co/photos/computer-stock-photo-3/We recently announced SmartDraw Cloud, a browser-based version of our native Windows diagramming application SmartDraw 2016. SmartDraw Cloud, built entirely in JavaScript using the HTML 5 stack, was more than three years in the making and has the full feature set of our Windows Business Edition.

Ask most people how they feel about browser-based apps and they’ll tell you that there’s a tradeoff between the convenience the cloud offers, and the power they get from the desktop versions of the same programs. Even the browser versions of Microsoft Office applications do not have the full feature sets of Microsoft’s desktop versions of the same. When we designed SmartDraw Cloud our goal was to eliminate this tradeoff and we’ve succeeded.

We’ve shown that it’s entirely possible to create a desktop-quality experience with app written in JavaScript and running in a web browser. You just have to go about it in the right way.

How Did We Do It?

We started with three design goals:

  1. SmartDraw Cloud had to have the full feature set and be file compatible withSmartDraw for Windows.
  2. It had to be fast enough to be fun to use.
  3. It had to be written in JavaScript in a manner which was easy to understand, maintainable, robust and fast.

Frankly we didn’t know if it was even possible. Our team consists of very senior and experienced application developers – but with little to no experience in JavaScript or browser-based apps.

Our initial approach was to do what everyone else seemed to be doing – research existing JavaScript frameworks and libraries, choose the right ones, piece them together, write some glue, and then build an app. After a few months, and a few false starts, we realized that this approach was not going to work for us.

Off-the shelf frameworks and components can save time and effort if you want to develop a quick in-house line of business app, but they are absolutely not the way to go if your goal is to develop a reliable, scalable, maintainable commercial application. In fact, this may be the reason why so many cloud apps have a “lightweight” reputation.

So we wrote some code! In fact, a lot of code. We built our own object libraries which are:

  • lightweight,
  • easy to maintain,
  • under our control, and
  • quick and scalable.

It took more time but SmartDraw is our bread and butter. Quick and dirty was not going to cut it.

This post is the first in a series that explains our approach, reviews some of the problems we encountered, and discusses why we made decisions we made. Later posts in this series will cover:

  1. Why we ditched Angular.js and wrote our own lightweight UI Library
  2. Why we used SVG for the graphics engine and not Canvas, and why we wrote own graphics library instead of using Raphael.
  3. How we used a combination of sockets and our memory manager to save incremental changes.
  4. And why, even with something as seemingly ubiquitous as hosting, we decided to create our own solution and avoid the pitfalls of AWS and similar services.

Writing a commercial app is the same in any language

No matter what language or platform you choose in writing a commercial application, the basic steps are the same.

  1. Decide what your app will do and what interface you want.This was easy for us since we had our Windows desktop application as a template.
  2. Propose an internal design to achieve this. Break it down into functional components that will become the building blocks of the app.This was also pretty easy because we had the Windows product as a guide. We would need a memory block manager to handle undo and redo effortlessly, a graphics interface, a way of presenting and interacting with a user interface, a file format, a text editor, a graphic editor, a business logic manager and so on.
  3. Propose the software architectureThis is where we specify the object design, object hierarchy and data that are maintained.
  4. Look to see if there are existing libraries that can be used for one or more of these building blocks. If so, carefully evaluate them and if they do the job by all means use them. Otherwise write your own.
  5. Write code in a disciplined and maintainable way.Code has to be readable and understandable by everyone on the team – including anyone that might join the team in the future. Code must also follow standards that the team agrees to.

The SmartDraw Cloud Architecture

The diagram below shows a simplified view of the SmartDraw Cloud architecture.

We made the some important design decisions early on:

  1. We would use SVG as our graphics engine because of its performance and superior quality when compared to Canvas. SVG is also quite scalable for printing and export.
  2. We would design a memory block manager that manages all of the objects that make up the composition of a document as discrete blocks, and also manages undo and the writing of the file to the server in a very efficient way.
  3. We would use Web Sockets to save the data and to communicate with the server.
  4. Our file format would be the very compact binary (but xml-like) format we use for our Windows product.

The List Manager manages the shapes on a page. The Business Manager applies the business logic that controls the behavior of specific diagrams like flowcharts or floor plans.

We also made some decisions after we got started and learned some of the pitfalls of traditional JavaScript development for this complex of a project. Those decisions included deciding to build many of our own components and frameworks.

Rolling Our Own Components

The user interface

The most common approach to developing a JavaScript app is to decide on a “framework” that isolates you from the common tasks of developing a UI, getting user input from it and showing the user the state of their document.

This is the sort of thing a framework is supposed to do for you: create and handle a simple control like this for the text font. The control shows the font of the currently selected shape or text, and allows you to change it.

sdcloud_ui_ex1

We looked at a wide range of such frameworks, including Dojo and React before setting on Angular.js. As we began development we quickly realized three things:

  1. Angular wouldn’t scale to the extent our sophisticated app required, and it obfuscated what was going on.
  2. The whole approach of wrapping the UI in a code-based framework made it impossible to separate the UI from the code. This is an important principle to us. The UI in SmartDraw is very rich and it needs to be coded by experts in HTML and CSS. The calls made to the app from the UI need to be specified in the UI HTML in manner easy enough for a non-JavaScript programmer to handle. We needed to be able to add commands by merely updating the HTML. Angular (and the other frameworks) just don’t work this way.
  3. Once you pick a framework, you become dependent on it. It becomes integral to your code. If support for it wanes, or your want to move to the latest “cool” framework, you have to pull your app apart. Our code base for Windows has been around for 20 years. We expect our JavaScript code base to last for many years too. We wanted no part of this “framework hell.”

Our solution was to build a lightweight JavaScript library, called “Bantm” to handle this.

The Graphics Interface

Another choice we made was to use SVG to display graphics instead of Canvas. From a performance point of view this was an easy choice. SVG is a vector graphic format that gives you resolution independent output. However it does present some difficulties:

  1. There is no open source rich text editor for SVG. We would have to write one.
  2. We weren’t satisfied with the existing graphics libraries for SVG. Raphael for example was designed to use either Canvas or SVG, and to also work with older browsers. This gave it a “lowest common denominator” feature set. We wanted the advanced effects of a pure SVG library and so we wrote our own.

Memory Management

Our “Block Manager” lies at the core of the SmartDraw Cloud app. It allows us to store JavaScript objects as discrete blocks. We get references to objects by getting their block. When we get a block we can tell the block manager whether we are going to modify it or not. If we modify it, the block manager makes a backup of just that block. When a user operation is complete, we commit the changed blocks to become the new state of the document. This is a lot like a commit operation in a database.

If an error occurs during an operation, the previous good state is restored, maintaining the integrity of the model.

An undo operation exchanges only the modified blocks from the previous state with the current state. Redo does the reverse. Undo states maintain only the changes to the document model. This is very efficient both in speed and memory usage.

The Block Manager also makes it possible to write only the modified objects to the server after each operation. A change will often only write a few bytes back to the server. This combined with the use of web sockets to send the data makes saving the document after each change very fast.

Libraries we did use.

After much prototyping and research we used just a handful of third party components. These include:

  • jQuery – to get references to UI elements,
  • Fileparser.js – to read and write the binary file format,
  • Hammer.js – to handle both mouse and touch events, and
  • Svg.js – to provide the lowest level interface to SVG markup.

We used these sparingly, often with our own modifications to make them more robust. We load them all from our own site so that our app is not dependent someone else’s site being available, or on updates which we don’t control. (Relying on external loading of components can lead to serious problems, such as the recent NPM Kik module debacle.)

It took us a year or more to write these libraries. With these in place we set about writing the app.

Writing JavaScript like C++

We all have long resumes using C, C++ and C# to write apps. So it was natural for us to treat JavaScript the same way – with well-defined objects and disciplined coding techniques.

A lot of the JavaScript we saw looked indescribably sloppy to us. We noted practices such as:

  • Adding members to an object on the fly,
  • Never actually formally defining an object, and
  • Assigning variables as hard coded constants (like align=16 instead of align=SDJS.ListManager.TextFlags.AlignLeft).

These are all recipes for unreliable code that is impossible to maintain three years out when the author has moved on or forgotten what he or she did.

Here are just a few of the constructs we established to write disciplined code:

  1. Created a consistent name space for all objects. For example, all of the ListManager objects began with SDJS.ListManager…. The UI objects began with SDUI. And so on.
  2. Defined constructors for all of the objects we used. For example:
// arrowhead record
SDJS.ListManager.ArrowheadRecord = function ()
{
	"use strict";
	this.StartArrowID = 0; //starting arrowhead
	this.StartArrowDisp = false; //starting arrowhead displacement
	this.EndArrowID = 0; //ending arrowhead
	this.EndArrowDisp = false; //ending arrowhead displacement
	this.ArrowSizeIndex = 1; //arrowhead size
};

If we needed to add a new member to the object, we added it to the definition, never to an instance on the fly. We also froze objects so this wasn’t possible. Constants were also formally defined:

SDJS.ListManager.ActionArrow = {
	"UP": 1,
	"LEFT": 2,
	"DOWN": 3,
	"RIGHT": 4,
	"SLOP": 5
};
Object.freeze(SDJS.ListManager.ActionArrow);
  1. Created an inherited object model for shapes.The list manager manages a list of shapes that together make up the drawing. Shapes are all inherited versions of a basic drawing object. For example:
SDJS.ListManager.BaseDrawingObject = function (attributes) {
	"use strict";

	// Set the stored object type
	this.Type = SDJS.Globals.StoredObjectType.BASE_LM_DRAWING_OBJECT;

	// Positional attributes
	this.Frame = attributes.Frame || {
		x: 0,
		y: 0,
		width: 0,
		height: 0
	};
};

Etc….
The base object has prototypes for ALL methods used by the derived shapes, but implements very few of them. There are derived objects for shape, simple line, and polygon line and so on. For example:

SDJS.ListManager.BaseLine.prototype = new SDJS.ListManager.BaseDrawingObject();
SDJS.ListManager.BaseLine.prototype.constructor = SDJS.ListManager.BaseLine;

/**
List Manager SmartDraw Base Line Object

Base line object class

@class BaseLine
@namespace ListManager
@constructor
@static
**/
SDJS.ListManager.BaseLine = function (attributes) {
	attributes.DrawingObjectBaseClass = SDJS.ListManager.DrawingObjectBaseClass.LINE;

These objects either implement their own methods, or inherit them from their base class. Again all methods appear in the

BaseDrawingObject 

class, even if they do nothing, so that it represents the union of all methods for all objects.

The emergence of TypeScript and Ecmascript 6 make it easier to write this kind of code. Writing a serious application requires disciplined coding.

Performance

We made many optimizations to ensure snappy performance including simple things like:
Pre-calculating array lengths so we never write:

For (i=0; i < array.length; i++)
{
 …
}

Instead we write :

len= array.length;
for (i=0; i < len; i++)
{
 …
}

Getting the length of a long array inside a loop takes time.

Looking up constants:

If a comparison is made in a loop with SDJS.ListManager.TextFlags.AlignLeft, it takes time within each loop to find this value. Instead we assign:

var AlignLeft= SDJS.ListManager.TextFlags.AlignLeft;

before the loop so this is done just once.

Adding Visio Compatibility

One of our design goals was to import Visio files in an editable form. To do this we had to extend the model of SmartDraw for Cloud and Windows to accommodate the Visio file format. These included multiple pages per document and support for new types of curves including NURBs, Splines and more.

The inherited object structure we implemented for the list manager objects we described earlier made this much simpler to implement. We just added or overrode the methods we already had for other polygon lines, and so on. This applied to our 20-year-old windows model too.

Conclusions

The HTML 5 stack can support desktop quality apps. What’s important is applying the same techniques that make desktop apps work well to the web platform. The language in which the app is written is much less important than applying the design and discipline a large app requires.

It’s also important to use well-designed components that you control and understand, even if it means writing them yourself. Slapping together a grab bag of open source code will make your app as reliable as the worst component you use. It can also make it much larger and significantly slower than it need be. And if you do use a third party library – load a stable version from our own site!

I am confident that as more JavaScript developers acquire the skills to develop commercial-quality applications, and more traditional app developers try their hand at JavaScript, the universe of truly powerful cloud apps will increase dramatically and the trade-off between desktop and cloud will fade away.

Editors Note: This is part 1 of a technical series about the making of SmartDraw Cloud. You can read part 2 here.