MacKuba

Back in spring I built a website that lets you browse charts of coronavirus cases for each country separately, or to compare any chosen countries or regions together on one chart. I spent about a month of time working on it, but I mostly stopped around early May, since I ran out of feature ideas and the pandemic situation was getting better (at least in Europe). The traffic that was huge at the beginning (over 10k visits daily at first) gradually fell to something around 1-1.5k over a few months, and I was only checking the page myself now and then. So it seemed like it wouldn’t be needed for much longer…

“Oh, my sweet summer child”, I kinda want to tell the June me 😬

So now that autumn is here and winter is coming, I suddenly found new motivation to work on the charts site again. But instead of adding a bunch of new features right away, I figured that maybe some refactoring would make sense first. I initially built this page as a sort of hackathon-style prototype (“let’s see if I can build this in a day”), but it grew much more complex since then, to reach around 2k lines of plain JavaScript – all in one file and on one level.

I started thinking about how I can make this easier to manage, and somehow I got the idea to try TypeScript.

Why add static typing?

I used to believe that static typing was just unnecessary complication.

My first programming experiments back in school were in Pascal and very bad C++. At the university, pretty much everything was either plain C or Java (or C#, depending on which group you picked). It was only near the end of the studies that I suddenly discovered Python and later Ruby, and it was like a breath of fresh air. I also read Paul Graham’s book “Hackers and Painters”, which made a big impression on me, steered me away from big corpos towards the startup world (for which I’m forever grateful), and also showed me how much better dynamically typed languages (specifically Lisp) were than statically typed ones.

I spent the next few years writing mostly Ruby and some JavaScript for the frontend, and I loved it. I still love Ruby and to this day I use it for all my scripting and server-side code.

However, at some point I also started building Mac and iOS apps, first in ObjC, and then in Swift. ObjC just felt like so much unnecessary boilerplate, and it really was. Then Swift came and simplified everything, but it exchanged the boilerplate for a very strict type system, much stricter than anything I’ve seen before. It was annoying to have to explain the compiler which property can be nil and when and what to do with it, or what to do if this JSON array does not contain what I think it does.

But after using Swift for a few years, I really appreciate the feeling of safety it gives you. You have to put in more work up front, but once you do, and once it compiles, you can be sure that whole categories of possible errors have already been eliminated before you even run the app. You have to do fewer build – run – find an error – fix the error cycles while building new features, and it’s also great while refactoring. And most importantly, it makes it harder to break one part of the code while changing another – we don’t always test every part and every single path in the app after every change, so such accidentally introduced errors can make their way to production and to users before they’re discovered.

Long story short, I miss this feeling a bit when working with Ruby and JavaScript now. It would be nice to have someone or something look over my code as I’m working on it, and not only the parts that are currently executed.

Learning TypeScript

TypeScript is not a difficult language, it’s not even a completely new language – it’s like JavaScript with some extra features and a compiler. So if you know JavaScript, you just need to learn the syntax for adding types, and the type declarations is the only thing you need to change in your code.

The official TypeScript site has a great docs section, and you can learn everything you need there. Start with the TypeScript for JS programmers intro and then go through the whole handbook and possibly the reference part if you want more, and that’s it.

Setting up the editor

A normal person would just download VS Code… however, that’s not me. I just refuse to run any IDE or editor without a fully native UI look & feel, so that leaves me with very little choice for those moments when I’m not using Xcode. For Ruby and JavaScript, I use TextMate, which I’ve been using non stop since 2008 (now the version 2 for the last few years).

There is a TextMate bundle for TypeScript, however, it only provides code highlighting and formatting. To simplify running the compiler while I’m in the editor, I manually added two actions using the bundle editor so that I don’t need to switch to the terminal to run npx tsc every time:

1) Compile to first error (shows output in a tooltip):

#!/usr/bin/env ruby18
require ENV['TM_SUPPORT_PATH'] + '/lib/textmate'

tsc = File.expand_path(File.join(ENV['TM_PROJECT_DIRECTORY'], 'node_modules', '.bin', 'tsc'))

ENV['PATH'] += ':/usr/local/bin'

result = `#{tsc} --noEmit`
first_line = result.each_line.first

if first_line.to_s.strip.length > 0
  puts first_line

  if first_line =~ /\((\d+)\,(\d+)\)\:/
    TextMate.go_to :line => $1.to_i
  end
else
  puts "Build OK"
end

2) Compile file (shows output in a special new window):

#!/usr/bin/env ruby18
require ENV['TM_SUPPORT_PATH'] + '/lib/textmate'
require ENV["TM_SUPPORT_PATH"] + "/lib/tm/executor"

tsc = File.expand_path(File.join(ENV['TM_PROJECT_DIRECTORY'], 'node_modules', '.bin', 'tsc'))

ENV['PATH'] += ':/usr/local/bin'

TextMate::Executor.run(tsc, '--noEmit')

This is a bit hacky, but it works for me. It only supports a scenario with one TypeScript file for now – there’s apparently no way to make tsc both use the tsconfig.json config to configure compiler options, but also specify a specific filename on the command line. And I’d love to have real autocompletion too, but you can’t have everything…

(If you know a good programmer’s editor with a native Mac UI, please let me know!)

Setting up the compiler & build

Once you download the TypeScript compiler node module, you can run npx tsc --init to create a tsconfig.json file at the root of your project. In this file you can specify where to look for .ts files, how modern JavaScript it should output (I chose es2018 since I don’t need to support any old browsers), and turn specific checks on and off. I’ve experimented a lot with the compiler options, and eventually I’ve left everything on except strict, strictNullChecks and strictPropertyInitialization (which requires strictNullChecks). The null checks add a ton of additional errors everywhere, and would require me to unwrap everything with ! like in Swift on every step (e.g. every call to querySelector, querySelectorAll, parentNode and such things), and I decided it’s just not worth the effort. It could possibly make sense if I was using some framework that was abstracting all interaction with DOM like React.

If you use any external libraries, like Chart.js in my case, you will also want to download their .d.ts definition files before you start working on your code – otherwise the compiler will keep telling you “I have no idea what this chart thing is and whether it has such property”.

As for running the build, npx tsc does everything once you configure it in tsconfig.json – the problem is how to make it run when it needs to, and what to do with what it outputs…

Again, a normal person would just set it up in some Gulp, Grunt, Webpack or whatever it is that people use in JavaScript land this month 😛 In my case however, I have a Ruby project built on top of Sinatra that has no JavaScript build system (since I have very little JavaScript in general outside of the Corona Charts page) and even no proper asset pipeline configured. So I could either set up some new build system just for this, or write some kind of hack. You can guess what I picked.

Since I only have one TypeScript file for now, and it’s only used on one page, I realized I can just manually check the timestamp in the controller action and rebuild if the .ts file is newer:

get '/corona/' do
  # ...
  unless production?
    original = 'public/javascripts/corona.ts'
    compiled = 'public/javascripts/corona.js'

    if File.mtime(original) > File.mtime(compiled)
      puts "Compiling #{original}..."
      `npx tsc`
      puts "Done"
    end
  end

  erb :corona, layout: false
end

That’s for development – for production, I simply run it as one of the deployment phases in my Capistrano script:

after 'deploy:update_code', 'deploy:install_node_packages', 'deploy:compile_typescript'

task :install_node_packages do
  run "cd #{release_path}; npm install"
end

task :compile_typescript do
  run "cd #{release_path}; ./node_modules/.bin/tsc"
end

Updating the code

It took me a good day or two to update all the code to silence all TypeScript errors. The good thing is that even though you get errors, the TypeScript compiler still outputs proper JavaScript that looks like what you had before (as long as it makes any sense at all), it just can’t promise it will work, so you could possibly deploy it as is, treat the errors like warnings in Xcode and get rid of them gradually. But ideally you want to not have any errors at all, since just like with warnings in Xcode, once you have too many of them, you stop noticing the important ones.

The changes I had to make can be grouped in a few categories:

• deleting some unused code, variables and parameters (that I haven’t realized were unused)

• creating type definitions for all informal data structures used in the code – e.g. declaring that a DataSeries is a hash mapping a string to an array of exactly 4 numbers, what the structure of the downloaded JSON is, or that the valueMode parameter can only be “confirmed”, “deaths” or “active” (it’s so cool that you can have such specific types!):

  type ChartMode = "total" | "daily";
  type ValueMode = "confirmed" | "deaths" | "active";
  type ValueIndex = 0 | 1 | 2 | 3;
  
  type RankingItem = [number, Place];
  type DataPoint = [number, number, number, number]
  type DataSeries = Record<string, DataPoint>;
  
  interface DataItem {
    place: Place;
    data: DataSeries;
    ...
  }
  

• defining all global variables as explicitly typed properties on Window:

  interface Window {
    colorSet: string[];
    coronaData: DataItem[];
    chart: Chart;
    ...
  }
  

• declaring all custom properties I set on built-in objects like DOM elements, or method extensions added to built-in types like Object or Array:

  interface HTMLAnchorElement {
    country: string;
    region: string;
    place: Place;
  }
  

• declaring class variables and their types:

  class Place {
    country?: string;
    region?: string;
    title?: string;
    ...
  }
  

• declaring the type of all function parameters (you don’t usually need to specify return types, those are inferred):

  function datasetsForSingleCountry(place: Place, dates: string[], json: DataSeries) {
    ...
  }
  

• casting some DOM objects to a more specific type like HTMLInputElement when I want to use the value:

  let checkbox = document.getElementById('show_trend') as HTMLInputElement;
  window.showTrend = checkbox.checked;
  

• providing a type for local vars initialized with [] or {}:

  let ranking: RankingItem[] = [];
  let autocompleteList: string[] = [];
  

It’s a lot of changes in total when you look at the diff, but most of it is things I needed to write once somewhere at the top. When I write a new function now, I usually just need to add types to the parameters in the function header.

Was it worth it?

So far – I’d say, absolutely. Like I wrote above, when adding new functions I usually just need to declare parameter types, unless I start adding completely new types, but most of the time I operate on the ones I already have. Like in most modern languages, you usually don’t need to define a type for a local variable like let thing = getThing(), because the compiler knows that it’s of type Thing. And if you return it, it knows this function always returns a Thing when it’s called elsewhere.

So it doesn’t add much overhead for new code, but it does give me this nice feeling that someone is checking what I write. I’ve done one refactoring since then, modifying the structure of the JSON file to make it smaller, since it naturally got way larger over the last few months (1.2 MB uncompressed, although I managed to compress it to 190 KB now using Brotli compression set at max level).

I changed the declaration of window.coronaData at the top to be an object instead of an array, and the DataSeries to be an array instead of a hash. And the compiler immediately showed me every single place in the code that was using these objects and had to be updated to the new format. I didn’t have to use the editor search to hunt down every single place of use, and worry that I might have missed one that I’ll only discover after some thorough testing (or a complaint from a user). Once it compiled, it was basically done and it worked from the first run.

So am I going to use TypeScript now for every 1-page-long piece of JS that adds some animations to a blog? Of course not. But does it make sense to use it in a webapp with dozens of features that builds and manages the whole UI in JavaScript? I think it does.