Building Desktop Applications using Web Technologies
with Electron

Jason Ronallo
Head, Digital Library Initiatives
NCSU Libraries
@ronallo
http://ronallo.com/presentations/c4l16

Hi. I'm Jason Ronallo, Head of Digital Library Initiatives at NCSU Libraries. And I'll be talking about building desktop applications.

No Skipping Ahead, Please!

Outline

Why build desktop applications anyways?

Why in the world would anyone build a desktop application these days. Everything is now on the Web now, right? A brief inaccurate history of the relationship between desktop applications and the web...

Survey

OK. Quick survey to make sure I know my audience well enough.

Who here has some role in creating Web applications?

Raise your hand!

Who here has some role in creating desktop applications?

Raise your hand!

The Web Has Won!

(well, except on mobile)

Lots of complex applications that were once developed for the desktop have now gone to the browser.

Yay!(?)

yay?

Too Many Tabs


We've traded too many windows for too many tabs.

Tab Management Tools

Things are so bad with tabs that there's a whole class of plugins and extensions and "top 5 ways" lists to help you manage tab overload.

Bookmarks

You can use bookmarks to organize things, but I basically use Pinboard to close tabs and forget about them so I don't feel guilty about all the unread tabs I have open and still feel like I have good intentions of getting back to them.

Tab Bankruptcy

And then sometimes I just declare tab bankruptcy and hope I didn't close anything really important.

Just Open a New Window, Right?

So to find things you just open up a new window right? Who has had so many browser windows open that they can't find the window that contains the tab that you know you've opened up? Or is playing audio?

Then It Will Be Easier to Find

And now I have 5 windows with N tabs all with the same pages in there somewhere.

Why Desktop Applications?

So why desktop applications again?

Stand Out

You might want to develop a desktop application if you want to stand out from all the tabs and browser windows. If you want to integrate with the desktop and stay in the task bar to make it easy to find.

Focus

Sometimes you just want an application to let you have focus on one thing. You can see why companies would want to develop desktop applications. You can understand why a library might want staff at the desk for instance to be able to quickly focus.

it is really hard to focus when that Gmail tab is there beconing you.

Why Not Desktop Applications?

So why don't we build more desktop applications?

GUI Toolkits and
Different Skillsets

Any time I've considered building a desktop application I've always been thwarted by difficult to use and ugly GUI toolkits.

You Already Know
How to Create a GUI

But if you know how to make web pages with HTML and CSS you already know how to make a user interface.

Creating Desktop Apps
with Web Technologies

It is now possible to create desktop apps with Web technologies.

Platforms for Building Desktop Apps with Web Technologies

There are a couple platforms for doing this. I'll be talking about Electron.

What Can You Build with Electron?

What can you make with Electron. I'd like to show you some interesting use cases for creating desktop applications that might also apply to libraries.

Companies that Use Electron

Lots of companies and open source projects are beginning to use Electron. As I was developing these slides and going back to the home page they just kept adding more and more companies to this list. There's a growing list of open source applications as well.

Chat & Communications

Slack
Slack
Discord
Discord
Wire
Wire

Speak
Speak
Rocket Chat
Rocket Chat

Tweetman
Tweetman
Sync
Sync
N1 (email client)
N1 (email client)
Lots of applications like Slack and Discord for chat and real-time communications.

Collaborative Tools

Avocode
Avocode
Pixate
Pixate

Flow
Flow
Avocode let's you collaborate on design. Flow does team task management.

Why Chat and Collaborative Tools?

Code Editors

Atom
Atom
Nuclide
Nuclide

Visual Studio Code
Visual Studio Code
Ionic Lab
Ionic Lab
Lots of code editors are using electron. Atom from Github. Visual Studio code from Microsoft. Nuclude from Facebook.

Editors

Wordpress (desktop)
Wordpress (desktop)
Yhat
Yhat
GitBook
GitBook

MdNote
MdNote
Caret
Caret
Abricotine
Abricotine
And lots of other editors. The new Wordpress desktop application is written using Electron. Yhat Rodeo is an IDE for data science. Markdown editors in 2nd column.

Why Code and Text Editors?

Developer Tools

GitKraken (git client)
GitKraken (git client)
gitify (desktop notifications)
gitify (desktop notifications)
Kitematic (now part of Docker)
Kitematic (now part of Docker)

Wagon (SQL editor)
Wagon (SQL editor)
Fireball (HTML5 game dev)
Fireball (HTML5 game dev)
Postman
Postman
And a whole assortment of developer tools. From working with git to an SQL editor to tools for HTML5 game development.

Hardware Configuration

Particle
Particle
Steelseries Engine 3
Steelseries Engine 3

Jibo (social robot)
Jibo (social robot)
Airtame
Airtame
One of the most interesting uses I found was for hardware configuration. Particle dev (IDE for IoT; Photon development board). Steelseries Engine is used to configure gaming keyboards and mice.

Jibo social robot

This is Jibo the social robot.

Jibo fan

Why Hardware Configuration?

Misc

Kakapo (ambient sound mixer)
Kakapo (ambient sound mixer)
Zoommy
Zoommy

GIF MAKER
GIF MAKER
Microstockr
Microstockr
OK. By this point I hope you have plenty of ideas for what is possible to build with Electron.
Zoomy stock photo search

OK, so what is Electron actually?

Chromium + Node.js

An Electron application is just a

Access to All Web APIs

Since you're using Chromium for the UI you have access to all the web APIs that Chromium supports.

Plus Access to All of Node

https://github.com/maxogden/art-of-node

I'll show you in a bit what all that means for you.

Atom
Electron formerly Atom Shell

imdone-atom

task board for Atom

https://atom.io/packages/imdone-atom

And then I found imdone-atom which is a plugin to Atom that gives you a kanban board right in your editor. So I can write code or take notes and easily find and prioritize all my tasks.

Demo Application

OK. I'm going to briefly show a demo application I developed that's a bit closer to libraries. I don't have the time to go through and show you every feature of Electron, but I wanted to show you a demo application I built that shows off some of the features that I think are really cool and make creating desktop applications compelling again.

Who uses JPEG2000 images?

Who has memorized all of their JPEG2000 conversion
command line options?

Example with Kakadu

kdu_compress -rate \
  2.4,1.48331273,.91673033,.56657224,.35016049,.21641118,.13374944,.08266171
  -precise \
  Clevels=6 Cblk=\{64,64\} \
  -jp2_space sRGB \
  Cuse_sop=yes Cuse_eph=yes Corder=RLCP \
  ORGgen_plt=yes ORGtparts=R \
  Stiles=\{1024,1024\} \
  -double_buffering 10 \
  -num_threads 4 \
  Creversible=no \
  -no_weights \
  -i /path/to/input.tif -o /path/to/output.jp2

Thanks, Jon Stroop!

Here's what I'm currently using based on Jon Stroop's recipe.

JPEG2000 Converter

I made a little GUI application called JPEG2000 Converter. That takes sources images and creates JP2s. Just makes things a bit simpler and graphical.

Comparison

Handbrake is to ffmpeg

as JPEG2000 Converter is to OpenJPEG and Kakadu

JPEG2000 Converter Window

Here's what the main window looks like.

Desktop Integration

One thing to notice right way is how the desktop integration makes it easy to find. Makes alt-tabbing through to the application easy and natural.
You can select some files. It creates thumbnails so you see what you're doing. And then you can create the JP2s.

Converting to JPEG2000

  1. Ensures the images are converted to an RGBA TIF to work with Kakadu
  2. Converts to JPEG2000
  3. Cleans up temporary files
  4. Sends a desktop notification
It strings together some other steps like making sure the TIF is RGBa.

Script Tag

<script src="../javascripts/convert.js"></script>
The conversion all happens with code in a script tag.

HTML page
HTML page
The code you are about to see is running from the embedded browser.

Trigger image conversion

$ = require('jquery')
$(document).ready ->
  $("#convert-jpeg2000-btn").on 'click', () ->
    # Hide some stuff from the view...
    async.eachSeries(
      $('.file-row') # take each row
      (file_row, callback) ->
        convert_image(file_row, callback)
      # When all done this callback is triggered
      (err) ->
        new Notification("Images done!")
        # Change the view some more...

convert.coffee

Here's how we trigger the image conversion. We use jquery to hide some stuff from the view and begin processing the images in series.

Convert an Image

child_process = require('child_process')
fs = require('fs')

kdu_compress =
  (tif_tmp_rgba, out_file, async_callback) ->
    cmd = kdu_command tif_tmp_rgba, out_file
    child_process.exec cmd, (stdout2, stderr2) ->
      console.log "jp2: #{out_file}"
      fs.unlinkSync(tif_tmp_rgba)
      async_callback()

convert.coffee

To do the actual image conversion we can call child_process here to execute a command in the shell and delete a file. And we can also console.log to the browser window.

But Wait!

This is code running in a browser from a script tag?

child_process & fs

# List out contents of /usr directory
child_process = require('child_process')
ls = spawn 'ls', ['-lh', '/usr']
ls.stdout.on 'data', (data) ->
  console.log "stdout: #{data}"

# Delete a file!
fs = require('fs')
fs.unlink '/path/to/a/file.png', (err) ->
  console.log err

This code is running the in browser!

Yes, the child_process and filesystem modules from node do things like launch shell commands and delete files from the file system.

Direct Access to the File System in the Web Browser Context!!!!!!!1

This is what's really mind blowing about Electron is the way that you can work in two modes at once.

Electron Allows an App To:

Opening an Image Viewer in a New Window to See the Image Conversion Results

Inter-Process Communication

(Window Communication)

I want to show you some code for how easy it is to open up new windows and communicate between them.

Open New Browser Window

ipc_renderer = require('electron').ipcRenderer
$(document).ready ->
  # Click a link or button
  $('#launch-pan-zoom').on 'click', ->
    jp2_image_path = get_jp2_image_path(this)
    # Send "open-jp2" message to main process
    ipc_renderer.send('open-jp2', jp2_image_path)

index.coffee (simplified)

Here we just use some jQuery to open a file input when we click on a button. And at the bottom we see how we send an "open-jp2" message.

Main Process

electron = require 'electron'
# Open IPC (inter-process communication)
# message channel
ipc_main = electron.ipcMain
# Listen for the "open-jp2" message
ipc_main.on 'open-jp2', (event, jp2_image_path) ->
  # Create a new window
  jp2_window = new (electron.BrowserWindow)()
  # Load the OpenSeadragon viewer in our window
  url = get_openseadragon_url(jp2_image_path)
  jp2_window.loadURL(url)

index.coffee (simplified)

Then in the main process we set a listener for that message and open up a new browser window.

OpenSeadragon Viewer

Here you see that code in action. You can even click on the magnifying glass to open up a file browser and then open up any JP2 file at all.

Full Power of Node.js

And since we have the full power of node here.

Embedded IIIF Image Server

express = require('express')
express_app = express()
# Route for IIIF Image Information Requests
express_app.get '*info.json', (req, res) ->
  # Pretend we have this synchronous function
  info_json = create_info_json(req)
  # Deliver a IIIF Image Information Request
  res.send info_json

express_app.get '*.(jpg|png)', (req, res) ->
  # Extract the requested image from the JP2...
  res.send image

express.coffee (simplified)

I've embedded a IIIF-compatible image server within our Electron application.

Yak Shaving

https://www.npmjs.com/package/iiif-image

Once I realized this it lead to a bit of yak shaving. I created

Issues with Electron

Yes, it is cross-platform, but...

packaging and building applications for distribution could certainly be much simpler and well-documented. Apps can be packaged for the Mac App store. There are some tools that can help you do some of this and more tools are being developed.

Resources:
Awesome Electron

I could provide you with a lot of links, but just look up awesome electron for lots of great links.

Thanks!

Jason Ronallo
@ronallo

http://ronallo.com/presentations/c4l16

Settings