Last week I wrote a tool for reverse engineering protobuf data called protofudger. I wanted to show it off to a few people, but didn’t want them to have to download it and run an unsigned binary on their system, or set up a whole go toolchain just to build one program. If only there were a better way!
Enter GopherJS. If you haven’t heard of it, here’s an excellent description right from their home page:
First task: get the program to compile nicely.
This was trivial. I ran
gopherjs build in my project directory, and it gave
protofudger.js file, along with a source map. GopherJS scores major
points for being pain-free. Five stars, easy transaction, would download
Second task: refactor the program to operate more like a library.
I had a typical go command line thing going on, using the
flag package for
fmt package to write stuff to the terminal, and everything
happening from start to end in a
main function. This… Did not transfer so
well to the web. You know, obviously. Makes sense in hindsight.
The meat of the matter was in refactoring all the
Printf statements out of the
parsing functions, and instead returning strings to be printed by the
function. This was pretty straightforward, so I won’t go into too much detail.
Take a look at this commit
if you’d like to see the actual changes. I also had to make sure the parsing
functions weren’t reading stuff from the environment (e.g. command line args),
instead receiving them as parameters.
Third task: somehow bind the program to the UI.
I decided that I wanted the UI to remain largely unchanged. This means text output, some very limited binary options, and an input file.
On the command line, the most obvious way to do the “input file” bit was to
read from standard input, e.g.
./protofudger < file.dat. I figured the best
conversion to the web would be to drop a file onto the web page. This meant
that I was going to have to get the data from a
FileReader to look like a
byte for the go code. Again, most of the heavy lifting here is done by
GopherJS. If you define a go function as
func(byte), you can call it from
Uint8Array and it just works as you’d expect. In my case,
this looked like
The next thing would be the options. There are currently only two options for
protofudger - one to display all possible interpretations of numeric fields, and
one to display byte offsets alongside field numbers. These mapped pretty well to
checkboxes. It’s pretty easy to get the state of a checkbox, so I just do this:
parse(new Uint8Array(reader.result), showAll.checked, showOffsets.checked).
Output was easy. That
parse function returns an array of strings that I can
join together and shove into a
<pre> element for that fixed-width terminal
Fourth task: build a bundle that I can deploy.
First I tried to build the
protofudger go code as a module that I could
import from the main program. This actually works, but it’s not what I
settled on. This would be a boring post if I ended it with “yep, the first
thing I tried was perfect,” wouldn’t it?
GopherJS actually provides hooks to build CommonJS modules - in addition to a
global handle, it provides a
module handle to go programs. This allows you
to do something like
js.Module.Set("exports", ...), and whatever you expose
will be available via
import './file.js'. This is
awesome, and it’s something that I think would be totally manageable if it
weren’t for some limitations in webpack and Babel. What it would work very
well for would be Node.js programs. With GopherJS you could very easily build
a wrapper around your go program for use in Node.
So, obvious solution: don’t run Babel over the GopherJS output. Yep! That worked. It built correctly, and I had a bundle that I could put into a page. But sourcemaps were broken!
Fifth task: get source maps working.
Well, yeah. There kind of is. There’s a package called source-map-loader, but it hasn’t seen any love for a while. There were a couple of bugs in it that I thought prevented it from operating at all (see issue 18), but even once I fixed them it still didn’t give me great results.
I really wanted to have working source maps, so I decided to go back to basics. When I included the GopherJS output directly in the page, they worked fine. The actual source code doesn’t show up, because GopherJS doesn’t include it in the source map, but all the line numbers and filenames were correct. I figured it wasn’t too horrible to have two files if it meant I had working source maps.
Conclusion: JS like it’s 2004.
What I ended up doing was treating the GopherJS output like a third party
library. I switched from setting
module.exports to setting
window.protofudger in the GopherJS entry file, then I access that explicitly
within the UI code. This has upsides and downsides. One downside is that there
are multiple files now. This makes deployment a tiny bit more complex, as I
need to put two
<script> tags in the HTML file. One upside is that there are
multiple files now (lol). The browser can download the UI file at the same
time as the GopherJS code, which has (anecdotally) resulted in slightly faster
load times. The UI code makes sure that
window.protofudger exists before it
tries to use it, so I could even load the GopherJS output asynchronously if I
wanted to optimise things further.
I hope this is helpful for someone out there, and I hope I’ve littered this post with enough keywords to catch people googling for “gopherjs webpack source maps,” to warn them off the path I tried to walk!