393 82 4MB
English Pages 231 Year 2019
Early praise for Programming WebAssembly with Rust Concise and well-paced, this book quickly dives into the details of WebAssembly, letting readers get their hands dirty building interesting Wasm applications. It’s loaded with great examples and touches on many different aspects of programming while paving the trail for WebAssembly development. ➤ Sean Boyle Senior Software Engineer, Cerner Programming WebAssembly with Rust is a great resource for learning a low-level language (WebAssembly) and showing how its power can be harnessed with Rust. It is perfect for people who like to understand how things work. ➤ Jason Pike Software Development Coach, Sigao Studios Starting with a detailed look at WebAssembly internals and the WAST language before moving on to solving fun gaming problems with Rust and Wasm, Programming WebAssembly with Rust ensures readers gain a foundational knowledge of WebAssembly and have fun doing so. ➤ Balaji Sivaraman Senior Technology Consultant, ThoughtWorks
I read Programming WebAssembly with Rust hardly knowing anything about either. I came away planning to make some time to build a WebAssembly module and publish to an npm registry—and with a clear idea of how to do so. An enjoyable read which suggests some mind-bending possibilities for the future of the web. ➤ Stephen Wolff Director, Max Gate Digital Ltd.
Programming WebAssembly with Rust Unified Development for Web, Mobile, and Embedded Applications
Kevin Hoffman
The Pragmatic Bookshelf Raleigh, North Carolina
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and The Pragmatic Programmers, LLC was aware of a trademark claim, the designations have been printed in initial capital letters or in all capitals. The Pragmatic Starter Kit, The Pragmatic Programmer, Pragmatic Programming, Pragmatic Bookshelf, PragProg and the linking g device are trademarks of The Pragmatic Programmers, LLC. Every precaution was taken in the preparation of this book. However, the publisher assumes no responsibility for errors or omissions, or for damages that may result from the use of information (including program listings) contained herein. Our Pragmatic books, screencasts, and audio books can help you and your team create better software and have more fun. Visit us at https://pragprog.com. The team that produced this book includes: Publisher: Andy Hunt VP of Operations: Janet Furlow Managing Editor: Susan Conant Development Editor: Andrea Stewart Copy Editor: Jasmine Kwityn Indexing: Potomac Indexing, LLC Layout: Gilson Graphics For sales, volume licensing, and support, please contact [email protected]. For international rights, please contact [email protected].
Copyright © 2019 The Pragmatic Programmers, LLC. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior consent of the publisher. ISBN-13: 978-1-68050-636-5 Book version: P1.0—March 2019
For my grandfather—Walter K. MacAdam— inventor, tinkerer, and IEEE president. He quietly supported my exposure to computers and programming throughout my childhood, often in ways I didn’t know until after his death. I always wanted to grow up to be like him, and I only wish he could’ve seen this book.
Contents Acknowledgments Introduction .
. .
. .
. .
. .
. .
. .
. .
. .
. .
. .
.
xi xiii
.
3 3 8 12 17
.
Part I — Building a Foundation 1.
WebAssembly Fundamentals . . . . Introducing WebAssembly Understanding WebAssembly Architecture Building a WebAssembly Application Wrapping Up
2.
Building WebAssembly Checkers . . Playing Checkers, the Board Game Coping with Data Structure Constraints Implementing Game Rules Moving Players Testing Wasm Checkers Wrapping Up
.
.
.
.
.
.
.
.
.
19 19 20 34 35 38 41
.
.
.
45 45 47 47 50 60 64 65
Part II — Interacting with JavaScript 3.
Wading into WebAssembly with Rust . . . . Introducing Rust Installing Rust Building Hello WebAssembly in Rust Creating Rusty Checkers Coding the Rusty Checkers WebAssembly Interface Playing Rusty Checkers in JavaScript Wrapping Up
Contents
• viii
4.
Integrating WebAssembly with JavaScript Creating a Better “Hello, World” Building the Rogue WebAssembly Game Experimenting Further Wrapping Up
.
.
.
.
.
67 67 73 87 88
5.
Advanced JavaScript Integration with Yew Getting Started with Yew Building a Live Chat Application Wrapping Up
.
.
.
.
.
89 89 99 112
Part III — Working with Non-Web Hosts 6.
Hosting Modules Outside the Browser . . How to Be a Good Host Interpreting WebAssembly Modules with Rust Building a Console Host Checkers Player Wrapping Up
.
.
.
.
115 115 117 120 129
7.
Exploring the Internet of WebAssembly Things . Overview of the Generic Indicator Module Creating Indicator Modules Building Rust Applications for ARM Devices Hosting Indicator Modules on a Raspberry Pi Hardware Shopping List Endless Possibilities Wrapping Up
.
.
.
131 132 135 140 141 150 152 154
8.
Building WARoS—The WebAssembly Robot System . An Homage to Crobots Designing the WARoS API Building the WARoS Match Engine Creating WebAssembly Robots Robots in the Cloud Wrapping Up Conclusion
.
.
155 156 157 162 186 191 191 192
.
.
193 193 194 194
A1. WebAssembly and Serverless . . . . Serverless 101 Intersection of WebAssembly and Serverless WebAssembly in the Cloud
.
.
Contents
Serverless WebAssembly in the Wild Integration with OpenFaaS
195 197
A2. Securing WebAssembly Modules . . . . General Security Concerns Browser-Based Attack Vectors Signing and Encrypting WebAssembly Modules Index
.
.
.
.
.
.
.
• ix
.
.
.
.
.
.
.
199 199 199 200
.
.
.
205
Acknowledgments This book would not have been possible without the infinite patience and support of my wife, who has far more faith in me than I do. I would also like to thank all of the technical reviewers for keeping me honest and accurate: Vijay Raghavan Aravamudhan, Jacob Chae, Nick Fitzgerald, Peter Perlepes, Jason Pike, Sean Boyle, Martjin Reuvers, Balaji Sivaraman, and Stephen Wolff.
report erratum • discuss
Introduction I’m old enough to have lived through quite a few seismic changes in the way developers build software and the kinds of products we can build. I was just starting my career when DPMI gave us native access to 32-bit integers, allowed unfettered access to a heap greater than 640k, and enabled the creation of ground-breaking games like DOOM. I remember the potential behind Java’s promise of write once, run anywhere. I was there when small, local communities built around dial-up bulletin board systems (BBSes) faded as the world became a single, digital community riding the wave of the Internet’s surge toward ubiquity. I experienced the shift in solution design from client/server to fat server to fat client and back again, today landing on cloud native applications, microservices, and independent functions where everything including our infrastructure is a service. I remember the web’s growth from a billion archipelagos of text (often blinking!) and Under Construction signs where the coolest places were the ones with the most intricate full-page background images, to the vast, sprawling engine of commerce, communication, lifestyle, and social connection that it is today. The web has gone from a place where only an elite few dabbled in that strange new world to a place where millions of people spend their days coding some of the most powerful and complex applications of the modern era. I firmly believe that we stand on the precipice of another seismic change— WebAssembly. This new technology holds within it the potential to radically change how developers build applications for the web. Moreover, as you’ll see throughout this book, WebAssembly is more than just a new pebble thrown into the ocean of web technologies. It’s a tsunami that can change not only how consumers interact with and how developers build applications but also fundamentally alter the kinds of applications we can create. It may even transform our core definition of the word application.
report erratum • discuss
Introduction
• xiv
Today’s Web Technology Today’s web is a veritable playground for developers. You have the luxury of easy access to broadband speeds (with exceptions). Browsers are faster and more powerful than they’ve ever been, and the workstations people use to run those browsers have oodles of RAM, storage, and multiple cores—even the mobile devices. Today’s JavaScript is nothing like the primordial 1995 JavaScript that drew so much ire from the developer community. It dominates the web development landscape so much that its own ubiquity has become something of a joke or a meme. Competition between browser vendors (they didn’t call it the browser wars for nothing) has spurred years of refining the way their products execute JavaScript, which will be a key discussion point when you get into the details of browser-hosted WebAssembly. Modern browsers have virtual machines responsible for JavaScript execution. Internally they optimize and produce a form of bytecode from processed JavaScript. This, coupled with more memory and processing power, means that JavaScript is actually fast. Not just a little bit fast, but it’s so fast you can play full, grade-A video games in the browser. Applications can perform complex calculations, run machine learning models, process vast amounts of data, and otherwise treat the browser like an operating system. Frameworks like React, Angular, Backbone, VueJS, and countless others have made a dramatic impact on how web applications are built. Modern web applications can render incredibly dense user interfaces like what you see on Facebook or YouTube, all while receiving real-time events published from servers in the cloud to provide a level of interaction that’s now such a ubiquitous feature that web sites that don’t provide this new level of real-time interaction are often publicly shamed and doomed to fail. Nothing in the rest of this book that extols the virtues of WebAssembly should take away from the fact that the modern, programmable web is a giant virtual toy store, ripe for the plundering by eager developers. For any avid learner of technology, it’s a great time to be alive (and probably learn some JavaScript).
The Tech of Tomorrow WebAssembly is currently a 1.0 product, having just reached its first MVP (Minimum Viable Product). As with most 1.0 products, it’s bound to experience
report erratum • discuss
Who This Book Is For
• xv
some growing pains and points of friction, and we’ll go over those in depth as they come up in this book. As you look at the current state of WebAssembly and its limitations, you might get discouraged and feel the urge to give up and wait for things to get more mature. But I think the time is right to start learning and developing with this incredible new technology, and there are already many WebAssembly 1.0 products deployed and running in the wild and more appearing every day. In the span between two edits of this chapter, someone released a virtual machine built in WebAssembly that runs Windows 95 in a browser. The good news is that the experience will only improve over time. The tooling will get better, the interface between the browser and WebAssembly modules will get better, support for non-browser hosts will get better, and the number of tested and proven use cases will grow. In short, as time goes on, every aspect of the development of WebAssembly modules will improve. I am convinced that WebAssembly is at the tip of the next wave of truly paradigm-shifting changes in the programmable Internet. I did indeed mean to say Internet and not just web. The distinction might seem subtle, but I’m also thoroughly convinced that the browser as a host for WebAssembly modules is just the tip of the iceberg. WebAssembly is going to join the long line of game-changing innovations in the history of the Internet and fundamentally alter our concept of applications.
Who This Book Is For This book is for anyone who wants to build web applications. Whether you have had just a little bit of JavaScript exposure or whether you are a seasoned professional with dozens of React and Redux applications under your belt, WebAssembly has the potential to change the way that you build apps and the power of those applications in a way that few technologies before ever have. Whether you consider yourself a front-end, back-end, embedded, or any other kind of developer—this book is for you. Compiling other languages to WebAssembly means you get to use familiar development life cycles and toolchains and build and test strongly-typed, powerful code. Finally, if you think that there is more to this WebAssembly thing than just the web applications, then you will enjoy this book as well as we build WebAssembly interpreters in Rust and run them on Raspberry Pis to control
report erratum • discuss
Introduction
• xvi
hardware via GPIO. WebAssembly holds a lot of promise for many different types of developers, including the promise of unifying back- and front-end coding experiences.
Why Rust? Rust is a systems language that compiles to native binaries on any number of operating systems and hardware architectures. It is fast, its binaries take up very little space and have a small memory footprint, and is designed from the ground up to avoid accidental mutation, null referencing, and data races. In fact, the compiler will check your code and prevent you from making those mistakes. But I chose Rust for this book for reasons beyond just the language syntax and its powerful compiler. What excited me about Rust was how quickly it embraced WebAssembly. While other languages right now can compile code to WebAssembly, the sheer number of libraries and tools available within the Rust community for WebAssembly is staggering. It is the enthusiasm, support, and rapid pace of advancement in the Rust WebAssembly community that influenced my decision to use Rust for this book.
What You’ll Learn This book is divided up into three main parts: Building a Foundation As you build a foundation, you’ll learn the fundamentals and the core architecture of WebAssembly, including what it can and cannot do and how you can develop basic applications. By the time you reach the end of this section, you’ll be able to create a Checkers engine written entirely in raw WebAssembly. Interacting with JavaScript Building on your solid WebAssembly foundation, you’ll move on to using Rust to create your WebAssembly modules. You’ll start with the basics of creating a Rust version of your Checkers engine, and then you’ll move on to using code generation, advanced tooling, and macros to build powerful web applications that interact with JavaScript. By the end of this section, you’ll be able to write a multiuser, real-time chat application in Rust that compiles to WebAssembly. Working with Non-Web Hosts Once you’ve had your first taste of the power of WebAssembly, it’s time to take it to the next level and start working with non-web hosts.
report erratum • discuss
What You’ll Learn
• xvii
WebAssembly is about far more than just building things for the web, and you’ll see this firsthand as you create modules that control LED patterns for lights attached to a Raspberry Pi and, as your final project, you create a fully multiplayer arena battle game that lets developers pit their WebAssembly code against each other in a battle to the death. Now it’s time to get coding!
report erratum • discuss
Part I
Building a Foundation
Let’s get started by exploring WebAssembly fundamentals and learn how to write raw WebAssembly code.
CHAPTER 1
WebAssembly Fundamentals With WebAssembly, there is a symbiotic relationship between the compiled WebAssembly binary (called a module) and the host responsible for interpreting it. This relationship is at the heart of everything that you can do with this new technology, and understanding where the boundaries are between module and host is key to being able to build effective WebAssembly applications. WebAssembly can be viewed at two different levels—the raw, foundational level and at the higher level of other programming languages using WebAssembly as a target. Before you can understand and appreciate what languages like Rust are doing when they produce WebAssembly modules, you’ll need to know what WebAssembly can do, what it can’t, and how to use language-independent tools. This chapter gets you started at the foundation level, giving you an overview of what WebAssembly is, how it works, and how other features can be built upon this foundation. By the end of this chapter, you’ll be able to create and build your own WebAssembly modules using cross-platform language tools and your favorite code editor. While what you learn in these first few chapters may not be things you do on a daily basis, the context they provide will be invaluable as you build real applications with WebAssembly.
Introducing WebAssembly If the modern programmable web is merely a toy store, then WebAssembly is a toy warehouse filled with toys as far as the eye can see. Developers who spend most of their time working on the front end often long for some of the features, testability, and constraints prevalent in the world of servers and services. Likewise, folks who spend most of their time toiling behind the counter, away from the customer but carefully assembling all the parts of their order, often long for the fluid, expressive, blank canvas world the front
report erratum • discuss
Chapter 1. WebAssembly Fundamentals
•4
end represents. People who work in both worlds are keenly aware of the paradigm differences between the two and why the grass isn’t always greener on the other side. What if “sides” or “front” or “back” didn’t matter anymore? What if there was a new way of doing things, where you could write loosely coupled business logic that flows between servers, services, clients, and browsers without any shenanigans? What if you could have the best parts of the front- and backend worlds and still choose the most appropriate language for your problems? By the time you reach the end of this book, you’ll have learned enough about WebAssembly development that these propositions won’t sound like they came from a snake-oil salesman. They’ll ring true and hopefully inspire you to start building amazing new WebAssembly applications.
What Is WebAssembly? The WebAssembly home page1 says that it is a binary instruction format for a stack-based virtual machine. Wasm (a contraction, not an acronym, for WebAssembly) is designed to be portable (capable of running on different OSes, architectures, and environments without modification), and used as a compilation target for higher-level languages like C++, Rust, Go, and many others. The website also claims that Wasm enables deployment on the web for client and server applications alike. Let’s pick this definition apart a bit, because it’s rather dense. First, and most importantly, WebAssembly is a portable binary instruction format. This is very similar to the original intent behind Java’s bytecode and, if you’re familiar with the .NET Framework, you may recognize this concept as implemented in ILASM, the low-level instruction set supporting the Common Language Runtime. You’ll see this in depth in the next chapter, but for now it should suffice to know that the operations encoded in a WebAssembly module are not tightly coupled to any one hardware architecture or operating system, and these operations are just codes that a parser knows how to interpret. Next, there’s the phrase stack-based virtual machine. We’ll go over this in detail soon, but the short explanation is that this stack machine simultaneously contributes to WebAssembly’s tremendous speed, power, and several of its limitations.
1.
webassembly.org
report erratum • discuss
Introducing WebAssembly
•5
Finally, there’s a spot in the definition on which I fundamentally disagree. The phrase “deployment on the web” might limit your thinking and your imagination. This is a portable format that can run anywhere you can build a host, which you’ll also be learning about later. Limiting WebAssembly’s scope to the web (despite its name) does it a disservice.
What WebAssembly Is Not The first question I get asked once I get on my WebAssembly soapbox is, “Isn’t Wasm just another transpile target for JS?” Transpiling is translating from one high-level source language to another high-level source language. This is in contrast to the usual compiling, which takes a high-level source language and translates it into a low-level machine code. For example, converting TypeScript or React JSX into browser-executable JavaScript is done through transpiling. WebAssembly is not a JavaScript transpile target (though you can actually compile TypeScript into a Wasm module if you’re into that sort of thing). WebAssembly is also not meant to replace JavaScript. This is somewhat of a controversial opinion, as a large group of WebAssembly devotees online are convinced that it represents the death knell of JavaScript. While it might signal the beginning of a new era in which you write significantly less manual JavaScript, you still need JS to host WebAssembly 1.0 in the browser. It’s also not intended as a mere replacement for (or successor to) Flash, Silverlight, Adobe AIR, or Java Applets. As you’ll discover, WebAssembly isn’t run as a process outside the browser. How seamlessly it integrates with the user experience is entirely up to the developer and the tools they use. Another important thing to remember is that WebAssembly, on its own, isn’t a programming language. While there is both a binary and a text format, writing it by hand for anything beyond a few samples would take far too long and be too difficult to test and troubleshoot. Knowing how to write it by hand, however, will help you make the right decisions as you learn to build WebAssembly applications with Rust. WebAssembly doesn’t stand on its own. Like a game cartridge without a console or a BluRay disc without a player, it’s incomplete in isolation. Much like a symbiote that needs to feed off of its host to survive, WebAssembly can’t interact with anything outside the bounds of its own sandbox unless the host allows it. All I/O and other interactions are done entirely at the behest of the host such as a browser or a console application. While this might sound like
report erratum • discuss
Chapter 1. WebAssembly Fundamentals
•6
an unfair limitation, you’ll see a number of times throughout this book why this is actually a good thing.
Try It Out The best way to start learning something is to jump right in. Imagine one of those ball pits kids get to play in but, sadly, us adults are usually forbidden. As learners, our first exposure to new material is like jumping into this pit, surrounded by a dizzying array of colors and buried up to our necks in confusion. As we struggle to make our way to the far end of the ball pit, we gradually get our footing, the colors become more familiar, the shape and landscape of the ball pit gets smaller, and we’re able to reason about it. After a while, we crawl out of the pit, having learned enough that we’re eager to jump back in and discover more. For WebAssembly, you’re going to jump into the ball pit by using an online tool called WebAssembly Studio. In December 2017, Mozilla started working on this project as a way to provide a low-friction introduction to WebAssembly. It’s a combination of some of their other WebAssembly tools like WasmFiddle.2 This tool is entirely online and lets you create new projects in C, Rust, or AssemblyScript, a tool for compiling TypeScript to WebAssembly. It has gone through bursts of community activity and contribution since its creation. Open up your browser (any up-to-date version of Firefox, Edge, Chrome, or Safari should work) and point it at webassembly.studio. You’ll come to a screen that presents the following options for creating a new project: • • • • •
Empty C Project Empty Rust Project Empty AssemblyScript Project Hello World in C Hello World Rust Project
Choose “Empty Rust Project” and then click the Create button. This will create a new project within WebAssembly Studio, including a README.md file, a src directory, and the file you want to explore: main.rs. You’ll see the following code: #[no_mangle] pub extern "C" fn add_one(x: i32) -> i32 { x + 1 }
2.
wasdk.github.io/WasmFiddle/
report erratum • discuss
Introducing WebAssembly
•7
The WebAssembly Studio site, with its almost entirely black color scheme, doesn’t make for print-friendly screenshots so none have been included here. If you’re familiar with Rust, then this should be self-explanatory. If you’re new to Rust, don’t worry—you’ll spend quite a bit of time working through Rust’s syntax throughout the book. This code listing defines a function that adds 1 to a 32-bit signed integer, returning that value in a signed 32-bit integer. Believe it or not, this code can produce a WebAssembly module. Using Rust’s no_mangle macro tells the compiler not to change the signature of the function during compilation. As you’ll see in upcoming chapters, there are some aspects of executing WebAssembly that require some naming conventions. Click the Build and Run button and you’ll see the blue bar on the bottom of the editor flash. Then, the white square canvas in the bottom right-hand corner will display the number 42. Congratulations, you’ve just written, compiled, and executed your first WebAssembly module. Where did the number 42 come from? Take a look at the main.js file by clicking on it in your browser: fetch('../out/main.wasm').then(response => response.arrayBuffer() ).then(bytes => WebAssembly.instantiate(bytes)).then(results => { instance = results.instance; ➤ document.getElementById("container").innerText = instance.exports.add_one(41); }).catch(console.error);
Don’t worry if some of this JavaScript looks unfamiliar to you—it’ll be second nature by the time we’re done. The indicated line of code invokes the add_one function in the WebAssembly module and places the result value inside the container DOM element. And now you’ve built a WebAssembly module that adds 1 to any number. It might not be all that impressive, but you’ve taken that first leap into WebAssembly. The pattern of writing Rust code, compiling to WebAssembly, and then running the module in a browser is one that you’ll repeat many times throughout this book. For now, though, let’s take a step back from this code and take a look at what makes WebAssembly tick so you can have a better idea of what’s happening in the preceding example.
report erratum • discuss
Chapter 1. WebAssembly Fundamentals
•8
Understanding WebAssembly Architecture In this section, you’ll get a good look inside the engine that makes WebAssembly work. Its unique architecture makes it incredibly powerful, portable, and efficient—though this power comes with some limitations.
Stack Machines The type of computer that you’re using right now is likely a Register Machine. Laptops, desktops, mobile devices, virtual machines, even microcontrollers and embedded devices are register machines. A register machine is a machine (physical or virtual) where the processor instructions explicitly refer to certain registers, or data storage locations, on the processor. Accessing these registers is fast and efficient because the data is available directly within the CPU. For example, if you want to add two numbers together, you’d use the ADD instruction and you’d pass it the names of two registers as parameters, as shown in this bit of x86 assembly: ADD al, ah
In the preceding code, the values contained in ah and al will be added together, with the result stored in al. WebAssembly is a stack machine. In a stack machine, most of the instructions assume that the operands are sitting on the stack, rather than stored in specified registers. The WebAssembly stack is a LIFO (Last In, First Out) stack. If you’re unfamiliar with the concept of a stack: it is as its name implies— values are piled (stacked) on top of each other, and unlike arrays where you can access any data regardless of location in the pile, stacks only allow you to pop data off or push data onto the top. To add two numbers in a stack machine, you push those numbers onto the top of the stack. Then you push the ADD instruction onto the stack. The two operands and the instruction are then popped off the top and the result of the addition is pushed on in their place. There are a number of advantages to a stack machine that made it an appealing choice for WebAssembly: their small binary size, efficient instruction coding, and ease of portability just to name a few. There are some fairly well-known stack machines, including the Java Virtual Machine (JVM) and the bytecode executor for the .NET Common Language Runtime. In the case of those virtual machines, developers are spared the
report erratum • discuss
Understanding WebAssembly Architecture
•9
effort of writing assembly or thinking in prefix or Polish3 (where the operator comes first) notation because of the intermediate steps and code generation happening behind the scenes.
Data Types Admit it—you’ve been spoiled. Modern programming languages with hashes, lists, arrays, sets, extra-large numbers, and tuples have spoiled you. These languages also probably let you create your own types through structs or classes. Some of them even let you overload operators, and some of those overloads can even work on custom types. The world is your oyster and you have few limits. That is not the world of WebAssembly. As their name should imply, assembly languages are designed to be made up of primitives that can be used as building blocks by higher level languages. WebAssembly 1.0 has exactly four data types: Type
Description
i32
32-Bit Integer
i64
64-Bit Integer
f32
32-Bit Floating-Point Number
f64
64-Bit Floating-Point Number
One aspect of this relatively limited set of data types is that WebAssembly doesn’t assign any intrinsic signed-ness to numbers as they’re stored. The assumption of whether a number is signed or unsigned is only performed at the time of an operation. For example, while there’s only one i32 data type, there are signed and unsigned versions of that type’s arithmetic operators, e.g. i32.add and i32.add_u. When you’re using a high-level language that compiles to WebAssembly on your behalf, you shouldn’t have to worry about this subtlety. But when you’re writing raw Wasm in the text format by hand, it could trip you up in unexpected ways.
Control Flow WebAssembly’s handling of control flow is a little different than other, less portable assembly languages. WebAssembly goes to great lengths to ensure that its control flow can’t invalidate type safety, and can’t be hijacked by attackers even with a “heap corruption”4-style attack in linear memory. For 3. 4.
en.wikipedia.org/wiki/Polish_notation https://pdfs.semanticscholar.org/14f1/4b032235c345dfb3b3ecc8a879bbe4072407.pdf
report erratum • discuss
Chapter 1. WebAssembly Fundamentals
• 10
example, many assembly languages allow easily exploited blind jump instructions, whereas you’ll discover that WebAssembly does not. This additional layer of safety pairs well with the safety-first philosophy of Rust. Wasm control flow is accomplished the same way everything else is within a stack machine—by pushing things onto, and popping things off of, the stack. For example, with an if instruction, if whatever is at the top of the stack evaluates as true (non-zero), then the if branch will be executed. Take a look at an example of the if statement in action: (if (i32.eq (call $getHealth) (i32.const 0)) (then (call $doDeath)) (else (call $stillAlive)) )
In this code, if our hypothetical player’s health has reached 0, then we’ll call the doDeath function, otherwise we’ll call the stillAlive function. All those seemingly extra parentheses will make sense later in the chapter. WebAssembly has the following control flow instructions available: Instruction
Description
if
Marks the beginning of an if branching instruction.
else
Marks the else block of an if instruction
loop
A labeled block used to create loops
block
A sequence of instructions, often used within expressions
br
Branch to the given label in a containing instruction or block
br_if
Identical to a branch, but with a prerequisite condition
br_table
Branches, but instead of to a label it jumps to a function index in a table
return
Returns a value from the instruction (1.0 only supports one return value)
end
Marks the end of a block, loop, if, or a function
nop
No self-respecting assembly language is without an operation that does nothing
Linear Memory As you work with linear memory, you’ll truly begin to appreciate the extent to which modern high-level languages have spoiled you. With most languages, you can quickly and easily create a new instance of something on the heap with an operator like new.
report erratum • discuss
Understanding WebAssembly Architecture
• 11
Internally, the compiler knows the size of this thing (or has some trick to compensate for not knowing). When you pass an instance of something to a function, the compiler knows whether you’re passing a pointer or a value and how to arrange that value on your stack or heap in order to make the data available to a function. WebAssembly doesn’t have a heap in the traditional sense. There’s no concept of a new operator. In fact, you don’t allocate memory at the object level because there are no objects. There’s also no garbage collection (at least not in the 1.0 MVP). Instead, WebAssembly has linear memory. This is a contiguous block of bytes that can be declared internally within the module, exported out of a module, or imported from the host. Think of it as though the code you’re writing is restricted to using a single variable that is a byte array. Your WebAssembly module can grow the linear memory block in increments called pages of 64KB if it needs more space. Sadly, determining if you need more space is entirely up to you and your code—there’s no runtime to do this for you. This image with variables and byte offsets illustrates just one way to store data in a block of linear memory (how you choose to use and fill linear memory is entirely up to you and your code): var1 [0...39]
var2 [40...79]
var3 [80...119]
unused
In addition to the efficiency of direct memory access, there’s another reason why it’s ideal for WebAssembly: security. While the host can read and write any linear memory given to a Wasm module at any time, the Wasm module can never access any of the host’s memory.
Direct DOM Access Is an Illusion If you’ve seen WebAssembly demos that look like they’re directly accessing the browser DOM from inside the module—that’s an illusion. The host and module are sharing a block of linear memory, and the host is choosing to execute bespoke JavaScript to translate the contents of that shared memory area into updates to the DOM, just like you saw at the beginning of this chapter. This may change in future versions of WebAssembly, but for now, this remains little more than smoke and mirrors.
As you’ll see in the coming chapters, linear memory is crucial to being able to create powerful applications with WebAssembly. Before using high-level languages like Rust, you should learn how to manipulate linear memory
report erratum • discuss
Chapter 1. WebAssembly Fundamentals
• 12
manually so you can appreciate the extent of the work done on your behalf by tools and code generation and understand the impact of your designs.
Building a WebAssembly Application At the beginning of this chapter, you built and ran a simple Rust-based WebAssembly application using WebAssembly studio. In this section, you’ll install some tools on your machine that will allow you to compile and interpret WebAssembly modules.
Installing the WebAssembly Binary Toolkit The WebAssembly Binary Toolkit (pronounced “wabbit”) is a general-purpose set of command-line tools you’ll use for building, examining, and troubleshooting WebAssembly modules. Whether you’re on Windows, Mac, or Linux, the first thing you’re going to need to install is CMake.5 Installing CMake varies widely across operating systems, so you’ll need to check the instructions specific to your platform. Come back and continue with the wabt installation once you’ve verified that CMake is up and running locally. Next, follow the instructions on the wabt6 GitHub repository README to complete the installation. You should be able to get the binaries from a release, but if you want to build the toolkit yourself, go ahead and simply run make install and then, after a fairly lengthy compilation process, you’ll see a bunch of executables in the bin directory beneath wherever you checked out the repository, including some or all of the following: • • • • • • • • •
wasm2c wasm2wat wasm-interp wasm-objdump wasm-opcodecnt wasm-validate wast2json wat2wasm wat-desugar
The exact list of files may have changed since this book was published, but you’ll at the very least need wat2wasm and wasm-objdump for the next section where you will be writing real WebAssembly code. Make sure that you can
5. 6.
cmake.org github.com/WebAssembly/wabt
report erratum • discuss
Building a WebAssembly Application
• 13
execute both of these commands in a terminal or a shell and get the help text before continuing on. On my machine, make install installed all of the compiled binaries in /usr/local/bin: -------------
Install configuration: "Debug" Installing: /usr/local/bin/wat2wasm Installing: /usr/local/bin/wast2json Installing: /usr/local/bin/wasm2wat Installing: /usr/local/bin/wasm2c Installing: /usr/local/bin/wasm-opcodecnt Installing: /usr/local/bin/wasm-objdump Installing: /usr/local/bin/wasm-interp Installing: /usr/local/bin/spectest-interp Installing: /usr/local/bin/wat-desugar Installing: /usr/local/bin/wasm-validate Installing: /usr/local/bin/wabt-unittests
Coding in the WebAssembly Text Format Before I launch into an arguably dense and detailed section of fairly low-level coding, I want to answer the question of why. Why should you spend the effort learning how to write raw wast when you’ve got modern compilers that can do it for you? I don’t like magic. I don’t like black boxes that do things I don’t understand. I am put off when I have to use a third-party system when I don’t understand its internal workings, or at least the motivations behind the decisions made in the building of the thing. You could go forward and build powerful WebAssembly applications and live your life without knowing how it all works inside. However, I contend that this chapter and its contents provide the foundation on which a solid WebAssembly development practice should be built. You could, in theory, skip this chapter. But, as Morpheus told Neo in The Matrix, “You take the red pill—you stay in Wonderland and I show you how deep the rabbit-hole goes.” When we get into the chapters on building alternative, non-browser hosts for WebAssembly modules, you will absolutely benefit from the knowledge and experience gained in this chapter. The physical process of writing code in the WebAssembly Text Format (.wat files) is pretty easy—just open up your favorite text editor (VSCode, Atom, and others all have syntax highlighters for WebAssembly) and start typing. As you now know, WebAssembly doesn’t have a string data type. This makes the canonical “Hello, World” sample a little difficult. In fact, the code required
report erratum • discuss
Chapter 1. WebAssembly Fundamentals
• 14
to actually produce that output in a browser is pretty complicated. If you try to go this route as your first exposure to writing wat, you’re likely to get discouraged. For a simpler example that takes advantage of WebAssembly’s simple data types, let’s try creating a module with a single function that adds two numbers together and returns the result. Open up your text editor and create the add1.wat file with the following contents: fundamentals/add1.wat (module (func $add (param $lhs i32) (param $rhs i32) (result i32) get_local $lhs get_local $rhs i32.add) (export "add" (func $add)) )
The first expression at the top of every module is the module declaration. Note that there’s no module name, package name, or namespace. There are a number of other things that can go below this declaration, but in this case, you’re just creating a single function, marked by the func keyword. Each parameter to a function is indicated with an S-expression7 (a parenthesized syntax for representing data or code as nested trees, first created for Lisp) in the following form: (param $parametername datatype)
In this case, there are two 32-bit integer parameters: $lhs and $rhs and the result will be a 32-bit integer. The call to get_local retrieves a function-scoped value and places it on the Wasm execution stack. In this function, you’re calling get_local twice, putting the two parameters on the stack, and then calling i32.add. This adds the two values on the stack, pops them off, and puts the sum in their place. The value left on the stack at the end of the function is the default return value. The order these instructions appear in the code can seem awkward. This is pure postfix (operator-last) notation and makes for some difficult reading. You don’t necessarily have to do it this way. There’s an alternative, pure S-expression prefix notation where you put the operation first and operands second, as you see in this code:
7.
en.wikipedia.org/wiki/S-expression
report erratum • discuss
Building a WebAssembly Application
• 15
fundamentals/add2.wat (module (func $add (param $lhs i32) (param $rhs i32) (result i32) (i32.add (get_local $lhs) (get_local $rhs) ) ) (export "add" (func $add)) )
In the second form, there are more parentheses, but the code is easier to read, especially as you get into more complicated patterns like those in the next chapter. I’ll be using the second form from now on. To invoke this function in WebAssembly, you use the call keyword, as shown in this code that adds 9 and 5: (call $add (i32.const 5) (i32.const 9))
Using the Binary Toolkit Now that you’ve got some source code, it’s time to compile it and do some exploration with the “wabbit” tools. Go to the directory where you created add1.wat and run the following commands (wat2wasm should be in your path): $ wat2wasm add1.wat -o add.wasm $ wat2wasm add2.wat -o add_sexpr.wasm
If everything goes according to plan, you’ll receive the “no news is good news” response—nothing. Next, use wasm-objdump to get a look at what actually made it into your compiled add.wasm file. The -x option adds extra detail to the output: $ wasm-objdump add.wasm -x add.wasm:
file format wasm 0x1
Section Details: Type: - type[0] (i32, i32) -> i32 Function: - func[0] sig=0 Export: - func[0] -> "add"
There’s some pretty good information here. You can tell that there’s a function type declared that accepts two integers and returns an integer. There’s a function at index 0 called add. Finally, there’s an export called add.
report erratum • discuss
Chapter 1. WebAssembly Fundamentals
• 16
It is important to keep in mind that you can create functions in a WebAssembly module and not export them. In fact, functions are all private to the module and remain unexported until you explicitly write an export statement. If you run the same object dump command on add_sexpr.wasm, you’ll see the exact same type and exports. There’s one last thing worth exploring—take a look at what the code looks like after a round trip when you convert the binary add_sexpr.wasm file to its corresponding text format: $ wasm2wat add_sexpr.wasm -o roundtrip.wat fundamentals/roundtrip.wat (module (type (;0;) (func (param i32 i32) (result i32))) (func (;0;) (type 0) (param i32 i32) (result i32) get_local 0 get_local 1 i32.add) (export "add" (func 0)))
Here you can see that there’s a type created, a function of that data type, and then the instructions have reverted to the inverted stack notation rather than the instruction-first version. Finally, you can see that the export now just refers to the function at index 0 rather than a name. The separation of the symbol name and the function index is part of what ensures this kind of round-trip compilation is possible. The reason this round trip is important is to show what WebAssembly might look like when produced by other languages, and, just in case you had any doubts, to prove that your code is truly portable and can be opened, disassembled, and re-generated at will. This also means anyone with access to these tools can see everything inside your WebAssembly module—a point to take note of when it comes to design and security in the future.
WebAssembly Source Maps If a web browser has access to both the binary format and the original source code, then you can expose a source map through the developer tools. Even at this early stage in the technology’s life, you can already set breakpoints within WebAssembly modules written in Rust running in Firefox. When the respective line of code comes up, the browser renders the original (non-wat) code via the source map and not the minified code that you saw in the round-trip file.
report erratum • discuss
Wrapping Up
• 17
Wrapping Up In this chapter, you got your first real exposure to writing WebAssembly code. You got the “wabbit” toolkit installed and you used it to compile, examine, and disassemble WebAssembly modules. You’ve received a preliminary exposure to the syntax and specifications for WebAssembly and you should be able to experiment with creating your own very basic modules. In the next chapter, you’ll build on what you’ve learned so far to create a fully functioning game with just the basic WebAssembly tools.
report erratum • discuss
...the imagination is unleashed by constraints. You break out of the box by stepping into shackles.
➤ Jonah Lehrer
CHAPTER 2
Building WebAssembly Checkers Sample applications small enough to fit into a quick blog post are very good at giving you a quick, painless introduction to some new piece of syntax. They show you how to print to the console, they show you how many parentheses you might normally need for a given line of code, and you get to see working code so long as you don’t mind lacking the context of the surrounding application not shown in the post. Even when full applications are created, the nature of the easily consumed medium means that these illustrative applications often look nothing like realworld applications, and they bear little to no resemblance to anything you might deploy to production. In this chapter, you won’t be creating a contrived “Hello, World” WebAssembly module. Instead, you’ll be creating a module that can be used to run a game of checkers (also called draughts depending on which part of the world you are from). You’ll build this module by creating a series of small functions that, once complete, will work together to provide the fundamentals of a working checkers game. There’s always a trade-off between the complexity of a real application and the need to keep an example simple enough to be used as a learning tool, so we cut a few corners on evaluating some game rules and edge cases, but the code will be playable when you’re done.
Playing Checkers, the Board Game If you’ve played checkers, then you can probably skip this section. If you need a refresher, then this will still be a quick read.
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 20
Checkers is a fairly simple game played on an 8×8 game board. The board’s squares are typically alternating colors (one of the most common in the US is a black and red board). Each player then positions 12 pieces on the board in fixed squares spaced evenly one square apart from each other. One player controls the black pieces and another controls the white (or red) pieces. The player controlling the black pieces makes the first move. The simplest move is where a piece is allowed to slide diagonally on the board if there is no other player occupying that spot. If there is, you might be able to jump and capture that player’s piece. Players take turns moving or jumping (which can include multiple jumps per turn if pieces are positioned right) until one player reaches the opponent’s home row. This row is called the kings row or crownhead. Once in the opponent’s kings row, the player’s piece is crowned and gains the ability to move either backward or forward. The game is over when one player has captured all of their opponent’s pieces, or left their opponent with no more legal moves. When neither side can force a win, the game ends in a draw.
Coping with Data Structure Constraints If you were to build this game using a high-level programming language like Java, Rust, Python, or even C++, then you would likely have a very different approach to setting up the data structures needed for the game than what you’ll use in WebAssembly. From the rules described in the previous section, you’re going to need to hold the state of an 8×8 game board. The positions on this game board can be empty, or they can hold black or white pieces. In this section, you’ll learn how to manage this kind of state in WebAssembly using nothing but the limited data structures available to the language. If you’re playing along with the home game, then you’ll be creating functions throughout the chapter as you’re exposed to more fundamental WebAssembly language primitives and concepts. To get started, let’s create a directory for our project. I called mine wasmcheckers but you can choose anything. One advantage of writing code at this low level is we’re just going to end up with a single text file—no project files, no Makefiles, just text. Create a checkers.wat file in your project directory and define an empty module:
report erratum • discuss
Coping with Data Structure Constraints
• 21
(module (memory $mem 1) )
The 1 in the memory declaration indicates that the memory named $mem must have at least one 64KB page of memory allocated to it. Memory can grow at the request of either the Wasm module or the host. You can compile this with wat2wasm and then examine the contents with wasm-objdump before moving on to the details of state management. You should see some output that looks like the following: $ wasm-objdump checkers.wasm -x checkers.wasm:
file format wasm 0x1
Section Details: Memory: - memory[0] pages: initial=1
As you continue through this chapter, if you get curious about things you can always compile and then use the tools to examine what ended up in the compiled module.
Managing Game Board State The first thing to tackle when it comes to managing the state of a game board is, obviously, the board itself. As mentioned, a checkerboard is an 8×8 grid. This probably triggers the part of your programmer brain that wants to declare a two-dimensional array. In Rust, that might look something like this: let mut checkerboard: [[GamePiece; 8]; 8];
This is how most developers tend to visualize this problem, differences in syntax aside. But here’s the rub: WebAssembly doesn’t have arrays—singledimension or otherwise. It also doesn’t have complex types, so you can’t create a struct or a tuple or even a hash map called GamePiece. One thing that WebAssembly does have is linear memory. As we discussed in the preceding chapter, WebAssembly can have named, contiguous blocks of memory that it can write to, read from, import, or export. So if you’re going to use a linear memory block, how do you represent a two-dimensional array in that space? The solution is to linearize the two-dimensional array. Many of your favorite programming languages most likely already do this linearization for efficiency
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 22
without you noticing. The trick to linearization is figuring out the math behind converting an (x,y) coordinate pair into a memory offset. In this figure, you can see some blocks that represent values in individual positions in memory. Above some of these are the corresponding (x,y) coordinate on the game board and below you see the unit offset of that piece of memory. For example, the coordinate (0,0) on the board is at unit offset 0. The coordinate (7,0) on the game board is at unit offset 7, coordinate (7,1) is at unit offset 15, and so on: (0,0)
(7,0)
02020202 0
7
(0,1)
(7,1)
20202020 8
15
(0,2)
(7,2)
02020202 16
23
You may have detected a pattern, and the equation for that pattern is offset = (x + y*8), where 8 is the number of squares in a row. This would be fine if you were just linearizing a two-dimensional array into a one-dimensional space indexed as an array, but WebAssembly’s memory isn’t indexed like an array. It’s indexed by byte. This means that if you’re going to store a 32-bit integer (4 bytes) in each spot on the game board, you need to adjust the equation to offset = (x + y*8) * 4 where 4 is the number of bytes per unit offset. Thinking about data storage this way activated a part of my brain I had long since thought dead and covered in cobwebs. It takes a little getting used to, and it may feel like a harsh constraint compared to what you’re used to, but
report erratum • discuss
Coping with Data Structure Constraints
• 23
if you stick with this until the end of the chapter, you’ll see how operating within these constraints pays off. The foundation of the entire checkers game will be a function that determines the byte offset for a given X and Y coordinate, as this will be something your code is going to need to do every time it updates the board (this is inside the module declaration, after the memory declaration): wasmcheckers/checkers.wat (func $indexForPosition (param $x i32) (param $y i32) (result i32) (i32.add (i32.mul (i32.const 8) (get_local $y) ) (get_local $x) ) ) ;; Offset = ( x + y * 8 ) * 4 (func $offsetForPosition (param $x i32) (param $y i32) (result i32) (i32.mul (call $indexForPosition (get_local $x) (get_local $y)) (i32.const 4) ) )
The math being performed here boils down to the following: offsetForPosition(1, 2) = (1 + 2 * 8) * 4 = 68
The nesting of the multiplication and addition operators is a little tough to read, but it’s still manageable. Armed with the ability to determine where to put your game state, the next step is to figure out how to represent that state.
Fun with Bit Flags In each of the locations in linear memory, you have access to 32 bits (or 4 bytes). You don’t have the luxury of a complex structure or any language primitives for storing lists, fields, or tuples. You’ve just got raw bits, so how are you going to represent game state? This dusts off another technique from the patterns vault called bit flags. Bit flags is a technique for assigning meaning to individual bits within a number. When more than one consecutive bit are combined to store some other meaning, that’s typically called bit masking. Most of what we’re going to do is bit flags, though we’ll touch on masking briefly.
report erratum • discuss
Chapter 2. Building WebAssembly Checkers Unlocked Cave of Nightmares
Elven King Slain
Tutorial Completed
• 24
Is Player?
00101101 Game Master
Access to Fire Magic
Healer Discovered
Illuminated
You can see the 8-bit number 00101101, but rather than simply meaning the numerical constant 45, this is actually a set of boolean values. As the previous image indicates, the value 45 carries all of the following meaning within the context of a fictional video game: • • • • • • • •
This game object is a player Game object is not illuminated Tutorial is completed Healer has been discovered The Elven king has not been slain The player has access to fire magic The player has not yet unlocked the cave of nightmares The player is not a game master
When you describe certain data structures this way, you end up passing around simple, raw numeric values to represent players and game objects rather than high-level constructs like structs or enums. Given strategies like bit flags or masking, it’s kind of surprising just how much information you can pack into a single number. Underneath all of your higher level languages, many structs are getting densely packed into numbers just like this. Tooling and compilers doing all this work on your behalf is a recurring theme, and you’re looking at the individual gears within your familiar machine now. You can use bitwise logical operators on numbers to query and manipulate these bit values. For example, a bitwise AND operation compares each bit of one number with the corresponding bit of a second number, and if both bits in the same relative position are 1, then the bit in the resulting number will
report erratum • discuss
Coping with Data Structure Constraints
• 25
be 1. Otherwise, the output bit will be 0. A bitwise OR produces a 1 when either of the two compared bits are 1, and a bitwise XOR performs an exclusive or on the compared bits. The following quick reference table will come in super handy when working with bit masked or “packed” values: Logical Operation
WebAssembly
AND
i32.and
Query the value of a bit
Bitmask Action
OR
i32.or
Sets a bit to true (1)
XOR
i32.xor
Toggles the value of a bit
So far, you’ve written a function that lets you convert a Cartesian checkerboard coordinate into a memory address, and you’ve seen how you can pack lots of data into simple integers for storage and retrieval. Now you can assign meaning to the bits of an integer on a checkerboard: Binary Value
Decimal Value
Game Meaning
[unused 24 bits]...00000000
0
Unoccupied Square
[unused 24 bits]...00000001
1
Black Piece
[unused 24 bits]...00000010
2
White Piece
[unused 24 bits]...00000100
4
Crowned Piece
Using these bit flags, you know that a crowned black piece will have a value of 5 (crowned + black), a crowned white piece will have a value of 6 (crowned + white), and all empty spaces on the board will be represented by 0s. If you have a keen eye, you may have noticed that this bit-packed data structure technically allows for a piece to be both white and black at the same time (a value of 3). Keeping the piece colors mutually exclusive will have to be something you enforce with code, though you could also use different bit flag meanings to avoid this situation if you wanted.
Bitwise versus Regular Math You might have noticed that to activate multiple boolean flags within a bit masked value, you can simply add the two flags together, as in the case of Black (1) and Crowned (4) equals 5. This is a bad idea because this operation isn’t idempotent. If you add the Crown flag to a number twice, you’ve basically corrupted your state. When you activate a flag with i32.or, that’s idempotent, and you can do it over and over again without corrupting the rest of the number.
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 26
Once you decide what each bit means, you can write some functions to query, set, and remove these values from a piece. And in this case, a “piece” is literally nothing more than a 4-byte integer value: wasmcheckers/checkers.wat ;; Determine if a piece has been crowned (func $isCrowned (param $piece i32) (result i32) (i32.eq (i32.and (get_local $piece) (get_global $CROWN)) (get_global $CROWN) ) ) ;; Determine if a piece is white (func $isWhite (param $piece i32) (result i32) (i32.eq (i32.and (get_local $piece) (get_global $WHITE)) (get_global $WHITE) ) ) ;; Determine if a piece is black (func $isBlack (param $piece i32) (result i32) (i32.eq (i32.and (get_local $piece) (get_global $BLACK)) (get_global $BLACK) ) ) ;; Adds a crown to a given piece (no mutation) (func $withCrown (param $piece i32) (result i32) (i32.or (get_local $piece) (get_global $CROWN)) ) ;; Removes a crown from a given piece (no mutation) (func $withoutCrown (param $piece i32) (result i32) (i32.and (get_local $piece) (i32.const 3)) )
This code relies on immutable global values like $CROWN, $BLACK, and $WHITE that act like constants would in other languages. These are defined at the top of the module like so: (global $WHITE (global $BLACK (global $CROWN
i32 (i32.const 2)) i32 (i32.const 1)) i32 (i32.const 4))
The withoutCrown function is a bit tricky. We can’t use XOR with the crown bit (4) to force it to be zero. That will instead toggle it, setting the bit for a piece that isn’t already crowned. The intent of this function is to compare two pieces
report erratum • discuss
Coping with Data Structure Constraints
• 27
regardless of their crown status. The safe way to do this is to use AND with a mask that only returns the black and white bits and ignores all else. For me, as someone who doesn’t think in binary and bitwise operations all day, this was difficult to visualize. Take a look at the following truth table that illustrates how this works: Value
Operation
Mask
Result
0101
Crowned Black
Meaning
&
0011
0001 (B)
0001
Uncrowned Black
&
0011
0001 (B)
0110
Crowned White
&
0011
0010 (W)
$withCrown works on the same bitmasking principal, just with a different
operator. This code invokes i32.or on the bits in the $piece variable and the bits in the 32-bit constant 4. Referring to our handy bitmasking reference chart, we know that the OR bitwise operation is used to set a bit. So, the bit in the third spot from the right (22) will be set to 1 and the new value is returned. It’s important to remember here that you’re not (yet) affecting stored state, you’re just playing with the bits inside numbers. At this point you’ve created some functions that perform bit masking operations on simple integers to give them contextual meaning within the checkers game. Before we move on to writing more code, it would be nice to be able to test and experiment with the functions we’re writing. Since you’re using raw wast syntax, there’s no easy unit testing framework or code generation available. The easiest thing to do is just export all the functions we’re working on so we can invoke them from JavaScript until we’re satisfied they work as intended. First, put $offsetForPosition, $indexForPosition, and all of the bitmasking functions into a test module called func_test.wat: wasmcheckers/func_test.wat (module (memory $mem 1) (global $WHITE i32 (i32.const 2)) (global $BLACK i32 (i32.const 1)) (global $CROWN i32 (i32.const 4)) (func $indexForPosition (param $x i32) (param $y i32) (result i32) (i32.add (i32.mul (i32.const 8) (get_local $y) ) (get_local $x) ) )
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 28
;; Offset = ( x + y * 8 ) * 4 (func $offsetForPosition (param $x i32) (param $y i32) (result i32) (i32.mul (call $indexForPosition (get_local $x) (get_local $y)) (i32.const 4) ) ) ;; Determine if a piece has been crowned (func $isCrowned (param $piece i32) (result i32) (i32.eq (i32.and (get_local $piece) (get_global $CROWN)) (get_global $CROWN) ) ) ;; Determine if a piece is white (func $isWhite (param $piece i32) (result i32) (i32.eq (i32.and (get_local $piece) (get_global $WHITE)) (get_global $WHITE) ) ) ;; Determine if a piece is black (func $isBlack (param $piece i32) (result i32) (i32.eq (i32.and (get_local $piece) (get_global $BLACK)) (get_global $BLACK) ) ) ;; Adds a crown to a given piece (no mutation) (func $withCrown (param $piece i32) (result i32) (i32.or (get_local $piece) (get_global $CROWN)) ) ;; Removes a crown from a given piece (no mutation) (func $withoutCrown (param $piece i32) (result i32) (i32.and (get_local $piece) (i32.const 3)) ) (export (export (export (export (export (export
"offsetForPosition" (func $offsetForPosition)) "isCrowned" (func $isCrowned)) "isWhite" (func $isWhite)) "isBlack" (func $isBlack)) "withCrown" (func $withCrown)) "withoutCrown" (func $withoutCrown))
)
Compile that into func_test.wasm with wat2wasm, then create a JavaScript wrapper that will load the WebAssembly module and execute the exported functions so we can make sure they’re working as we expect:
report erratum • discuss
Coping with Data Structure Constraints
• 29
wasmcheckers/func_test.js fetch('./func_test.wasm').then(response => response.arrayBuffer() ).then(bytes => WebAssembly.instantiate(bytes)).then(results => { console.log("Loaded wasm module"); instance = results.instance; console.log("instance", instance); var var var var
white = 2; black = 1; crowned_white = 6; crowned_black = 5;
console.log("Calling offset"); var offset = instance.exports.offsetForPosition(3,4); console.log("Offset for 3,4 is ",offset); console.debug("White is white?", instance.exports.isWhite(white)); console.debug("Black is black?", instance.exports.isBlack(black)); console.debug("Black is white?", instance.exports.isWhite(black)); console.debug("Uncrowned white", instance.exports.isWhite(instance.exports.withoutCrown(crowned_white))); console.debug("Uncrowned black", instance.exports.isBlack(instance.exports.withoutCrown(crowned_black))); console.debug("Crowned is crowned", instance.exports.isCrowned(crowned_black)); console.debug("Crowned is crowned (b)", instance.exports.isCrowned(crowned_white)); });
Finally, we can create a little index.html file that starts the func_test.js script: wasmcheckers/func_test.html
Because of cross-site scripting rules in the browser, it might not let you open a file from the file system without having an HTTP server (my Firefox on
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 30
Ubuntu won’t, but apparently Firefox on Arch Linux does), so spin up your favorite one or you can just use Python to launch this lightweight one in the same directory as your HTML file: $ python3 -m http.server Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
Now open your browser to localhost:8000/func_test.html. Open up the JavaScript debug console and you should see something like the following output, verifying that we’re getting the values we expect: Loaded wasm module instance WebAssembly.Instance { exports: {…} } Calling offset Offset for 3,4 is 140 White is white 1 Black is black 1 Black is not white 0 Uncrowned white 1 Uncrowned black 1 Crowned is crowned 1 Crowned is crowned (b) 1
While reassuring (our code works), this is also a boring and dry set of console output. As we progress further in the book, you’ll be able to produce real, userfacing web visualizations of state, game boards, and other module internals. As you progress through this chapter, you can just copy and paste whatever function you like into this harness, make sure it’s exported, and test it with JavaScript. In the next section, you’ll build on what you’ve coded in checkers.wat so far to start reading and writing game state to manage the checkerboard.
Manipulating the Board The next couple of functions you’re going to write involve storing and retrieving values from the game board grid in linear memory and performing some validation checks on parameters and data before allowing state mutation: wasmcheckers/checkers.wat ;; Sets a piece on the board. (func $setPiece (param $x i32) (param $y i32) (param $piece i32) (i32.store (call $offsetForPosition (get_local $x) (get_local $y) ) (get_local $piece) ) )
report erratum • discuss
Coping with Data Structure Constraints
• 31
;; Gets a piece from the board. Out of range causes a trap (func $getPiece (param $x i32) (param $y i32) (result i32) (if (result i32) (block (result i32) (i32.and (call $inRange (i32.const 0) (i32.const 7) (get_local $x) ) (call $inRange (i32.const 0) (i32.const 7) (get_local $y) ) ) ) (then (i32.load (call $offsetForPosition (get_local $x) (get_local $y)) ) ) (else (unreachable) ) ) ) ;; Detect if values are within range (inclusive high and low) (func $inRange (param $low i32) (param $high i32) (param $value i32) (result i32) (i32.and (i32.ge_s (get_local $value) (get_local $low)) (i32.le_s (get_local $value) (get_local $high)) ) )
The $setPiece function has some new syntax, and this is the first place where your code uses the return value from one function as a parameter to another function. $setPiece calls i32.store, which stores a 32-bit integer in a memory address. The memory address will be the value returned by calling $offsetForPosition, and the value stored will be the parameter $piece. For example, to set a white piece at grid position (5,5), you could call (i32.store 200 2). $getPiece illustrates the first use of an if block with a function call as a predicate.
You need to specify the type of the value that’ll be returned from a predicate if it executes a function. The block keyword simply wraps one or more statements
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 32
and indicates the return type of that block, kind of like an anonymous function. The general structure for an if/then/else block that also executes code in the predicate clause looks like this: (if (result i32) (block (result i32) ...) (then ... ) (else ... ) )
Finally, the $inRange function is used to prevent querying beyond the edge of the game board. Most high-level programming languages have built-in features that will prevent situations like this or will throw an out-of-bounds runtime exception. Without those luxuries, in WebAssembly you’ve got to manually ensure you never access memory beyond the bounds of a given data structure. It’s worth pointing out, however, that even an index out-of-bounds error in WebAssembly can never reach beyond the limits of its linear memory. This keeps the chance of heap corruption pretty low.
Bounds Checking Blunders Throughout computing history, failing to properly check bounds at the assembly level has resulted in some catastrophic bugs, malicious viruses, and some amazing video game exploits. One of my favorites is this one from the original Famicom version of The Legend of Zelda,a which shows how reading beyond implied bounds can convert raw data into executable instructions to teleport straight to the end of the game.
a.
youtu.be/fj9u00PMkYU
Keeping Track of the Current Turn The game board itself isn’t the only bit of state that needs to be maintained in a checkers game. You’ll need to keep track of who has the next move. For obvious reasons, the checkers engine needs to prevent a player from taking two turns in a row or moving pieces that don’t belong to them. You’ve already defined two different values to indicate piece color: 1 for black and 2 for white. In a regular programming language you might simply create a global variable called currentTurn and set that to an enum like White or Black. With WebAssembly, we can create global variables that are either private to the module or can be shared with the host. Let’s take a look at the global declaration near the top of the module: (memory $mem 1) (global $currentTurn (mut i32) (i32.const 0))
report erratum • discuss
Coping with Data Structure Constraints
• 33
When we declare global variables in wast (raw text WebAssembly syntax), we can specify their mutability (immutable globals can be treated like constants in other languages), and we need to specify an initial value. The $currentTurn global will be set to 1 when it is black’s turn and 2 when it is white’s turn. The following code is responsible for querying and updating the current turn state: wasmcheckers/checkers.wat ;; Gets the current turn owner (white or black) (func $getTurnOwner (result i32) (get_global $currentTurn) ) ;; At the end of a turn, switch turn owner to the other player (func $toggleTurnOwner (if (i32.eq (call $getTurnOwner) (i32.const 1)) (then (call $setTurnOwner (i32.const 2))) (else (call $setTurnOwner (i32.const 1))) ) ) ;; Sets the turn owner (func $setTurnOwner (param $piece i32) (set_global $currentTurn (get_local $piece)) ) ;; Determine if it's a player's turn (func $isPlayersTurn (param $player i32) (result i32) (i32.gt_s (i32.and (get_local $player) (call $getTurnOwner)) (i32.const 0) ) )
To check whether it is a current player’s turn, we might be tempted to check if the value of $player is the same as the value returned by $isPlayersTurn. The problem is that $player can be holding a value indicating a crowned piece, which would mess up our equality check. To compensate for this, when the $isPlayersTurn function checks if the player value passed is the current player, it will perform an AND (remember, this queries only the bits in the mask) of the value and the player, and if that is greater than zero, that player is the color currently stored in the $currentTurn global. This is kind of like comparing player.color with currenturn.color in other, higher-level programming languages. Here, think of the concept of “color” as a bitmask.
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 34
Implementing Game Rules From a rules standpoint, checkers is not all that complex of a game. All 12 of your pieces start out with the same movement capabilities, and the only special pieces ever on the board are those that have been crowned. The goal of this chapter isn’t to produce a game of checkers that might be ready to plug into a WebGL site so you can ship it tomorrow. You’re building a foundation for such a thing, but the nitty-gritty details of checkers edgecase rules are beyond the scope of this chapter. This checkers game implements some basic rules like crowning a piece and moving. It doesn’t implement jumps, multi-jumps, or detect a winner. If you would like to do that as a reader exercise, I would love to see your game on GitHub! The first function in the following code listing, $shouldCrown, determines if a piece should be crowned. A piece is eligible for upgrade if it resides in its opponent’s kings row. The $crownPiece function uses a few functions you’ve already written to add a crown to a piece and then store it in the gameboard: wasmcheckers/checkers.wat ;; Should this piece get crowned? ;; We crown black pieces in row 0, white pieces in row 7 (func $shouldCrown (param $pieceY i32) (param $piece i32) (result i32) (i32.or (i32.and (i32.eq (get_local $pieceY) (i32.const 0) ) (call $isBlack (get_local $piece)) ) (i32.and (i32.eq (get_local $pieceY) (i32.const 7) ) (call $isWhite (get_local $piece)) ) ) ) ;; Converts a piece into a crowned piece and invokes ;; a host notifier (func $crownPiece (param $x i32) (param $y i32) (local $piece i32) (set_local $piece (call $getPiece (get_local $x)(get_local $y)))
report erratum • discuss
Moving Players
• 35
(call $setPiece (get_local $x) (get_local $y) (call $withCrown (get_local $piece))) (call $notify_piececrowned (get_local $x)(get_local $y)) ) (func $distance (param $x i32)(param $y i32)(result i32) (i32.sub (get_local $x) (get_local $y)) )
The local keyword declares a temporary variable that will only be visible within its enclosing scope. In the case of this code listing, all the locals are scoped to their containing function. The set_local instruction sets the value of a local variable. Sadly, you can’t declare and set a local in the same line of code. High-level languages that allow this sort of thing are just generating the separate declaration and assignment operations on your behalf. Another thing worth pointing out in the preceding code listing is the call to $notify_piececrowned. This function will be imported by this checkers module and implemented by the host. Whenever a piece is crowned, the module calls this to let the host react to the change. This could include modifying the sprite on display for the newly crowned piece, playing a victorious sound, or both. The $distance function is just a simple subtraction. In the next section on movement, you’ll see how that value is used to determine if that distance is valid. Nearly all of the (naive) fundamentals of checkers are done at this point: the only thing left is to code player movement.
Moving Players Before a player can move, you need to ensure that it is a valid move. In the case of this naive implementation of checkers, a valid move is: • The “move distance” from current to target (single axis) is valid • The target space is unoccupied • The piece being moved belongs to the current player Let’s take a look at the code to determine if a move is valid: wasmcheckers/checkers.wat ;; Determine if the move is valid (func $isValidMove (param $fromX i32) (param $fromY i32) (param $toX i32) (param $toY i32) (result i32) (local $player i32) (local $target i32) (set_local $player (call $getPiece (get_local $fromX) (get_local $fromY))) (set_local $target (call $getPiece (get_local $toX) (get_local $toY)))
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 36
(if (result i32) (block (result i32) (i32.and (call $validJumpDistance (get_local $fromY) (get_local $toY)) (i32.and (call $isPlayersTurn (get_local $player)) ;; target must be unoccupied (i32.eq (get_local $target) (i32.const 0)) ) ) ) (then (i32.const 1) ) (else (i32.const 0) ) ) ) ;; Ensures travel is 1 or 2 squares (func $validJumpDistance (param $from i32) (param $to i32) (result i32) (local $d i32) (set_local $d (if (result i32) (i32.gt_s (get_local $to) (get_local $from)) (then (call $distance (get_local $to) (get_local $from)) ) (else (call $distance (get_local $from) (get_local $to)) )) ) (i32.le_u (get_local $d) (i32.const 2) ) )
The code for $validJumpDistance might look a little strange. Since WebAssembly’s integers don’t support the abs() function, we’re basically deciding on the order in which we supply arguments to the $distance function based on whether or not it will result in a negative number. Also remember that we’re not building a rules-compliant version of checkers—this is just enough code to exercise some basic rules in service of learning WebAssembly. In the following code listing, the $move function will eventually be exported (typically done at the end of the module code) function so the host can call it. Since
report erratum • discuss
Moving Players
• 37
this makes it a public API method, it has to guard against bad data and invalid moves. $move returns 1 for a successful move and 0 for a failed one: wasmcheckers/checkers.wat ;; Exported move function to be called by the game host (func $move (param $fromX i32) (param $fromY i32) (param $toX i32) (param $toY i32) (result i32) (if (result i32) (block (result i32) (call $isValidMove (get_local $fromX) (get_local $fromY) (get_local $toX) (get_local $toY)) ) (then (call $do_move (get_local $fromX) (get_local $fromY) (get_local $toX) (get_local $toY)) ) (else (i32.const 0) ) ) )
Notice that the $move function defers some functionality to the $do_move function. This separates the conditional guard checking code from the actual state changes in $do_move. After a move toggles the current turn owner, the state change sets the piece at the destination location, and wipes out the piece at the original location: wasmcheckers/checkers.wat ;; Internal move function, performs actual move post-validation of target. ;; Currently not handled: ;; - removing opponent piece during a jump ;; - detecting win condition (func $do_move (param $fromX i32) (param $fromY i32) (param $toX i32) (param $toY i32) (result i32) (local $curpiece i32) (set_local $curpiece (call $getPiece (get_local $fromX)(get_local $fromY))) (call $toggleTurnOwner) (call $setPiece (get_local $toX) (get_local $toY) (get_local $curpiece)) (call $setPiece (get_local $fromX) (get_local $fromY) (i32.const 0)) (if (call $shouldCrown (get_local $toY) (get_local $curpiece)) (then (call $crownPiece (get_local $toX) (get_local $toY)))) (call $notify_piecemoved (get_local $fromX) (get_local $fromY) (get_local $toX) (get_local $toY)) (i32.const 1) )
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 38
As a convenience, the $do_move function invokes a function called $notify_piecemoved that lets the host know when a piece has finished moving. This function will be declared as an import at the beginning of the module. This function is called so the host can react to player movement rather than repeatedly polling for changes to the game board state. Finally, with all the basic functions in place, it’s time to connect this module to a host and test it.
Testing Wasm Checkers Before you can start testing or playing checkers, you need just one more function. You’ve written functions to maintain the game board state, to move player pieces, and even to notify the host when important events occur. What’s missing is the initial set up—placing all of the players’ pieces on the board. $initBoard is a simple, brute-force function that just calls $setPiece over and over
to place the white and black pieces, finally setting the current turn to black: ;; Manually place each piece on the board to initialize the game (func $initBoard ;; Place the white pieces at the top of the board (call $setPiece (i32.const 1) (i32.const 0) (i32.const 2)) (call $setPiece (i32.const 3) (i32.const 0) (i32.const 2)) (call $setPiece (i32.const 5) (i32.const 0) (i32.const 2)) (call $setPiece (i32.const 7) (i32.const 0) (i32.const 2))
«many,
many more calls to $setPiece»
(call $setTurnOwner (i32.const 1)) ;; Black goes first )
The next step is to include all the imports at the top of the checkers.wat module: wasmcheckers/checkers.wat (import "events" "piecemoved" (func $notify_piecemoved (param $fromX i32) (param $fromY i32) (param $toX i32) (param $toY i32))) (import "events" "piececrowned" (func $notify_piececrowned (param $pieceX i32) (param $pieceY i32)))
And declare the exports at the bottom of the module: wasmcheckers/checkers.wat (export "getPiece" (func $getPiece)) (export "isCrowned" (func $isCrowned)) (export "initBoard" (func $initBoard)) (export "getTurnOwner" (func $getTurnOwner)) (export "move" (func $move)) (export "memory" (memory $mem))
report erratum • discuss
Testing Wasm Checkers
• 39
It is important to notice that not all of the functions are exported. The host is limited to invoking only the public (exported) API. For example, you do not export $setPiece, which could possibly let the host code cheat or allow a bug to corrupt the game board. This is a standard defensive programming practice. With the checkers.wat file complete, you should be able to build it using wat2wasm: $ /path/to/wabt/bin/wat2wasm checkers.wat -o checkers.wasm
Next, create an index.js file to load the module and run through some sample moves: wasmcheckers/index.js fetch('./checkers.wasm').then(response => response.arrayBuffer() ).then(bytes => WebAssembly.instantiate(bytes, { events: { piecemoved: (fX, fY, tX, tY) => { console.log("A piece moved from (" + fX + "," + fY + ") to (" + tX + "," + tY + ")"); }, piececrowned: (x, y) => { console.log("A piece was crowned at (" + x + "," + y + ")"); } }, } )).then(results => { instance = results.instance; instance.exports.initBoard(); console.log("At start, turn owner is " + instance.exports.getTurnOwner()); instance.exports.move(0, 5, 0, 4); instance.exports.move(1, 0, 1, 1); instance.exports.move(0, 4, 0, 3); instance.exports.move(1, 1, 1, 0); instance.exports.move(0, 3, 0, 2); instance.exports.move(1, 0, 1, 1); instance.exports.move(0, 2, 0, 0); instance.exports.move(1, 1, 1, 0); // B - move the crowned piece out let res = instance.exports.move(0,
// // // // // // // //
B W B W B W B - this will get a crown W
0, 0, 2);
document.getElementById("container").innerText = res; console.log("At end, turn owner is " + instance.exports.getTurnOwner()); }).catch(console.error);
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 40
The moves in this test are just exercising the naive game rules. A real game wouldn’t allow movement like this, but this at least helps us verify that the code we did write works as intended. Now you can create an index.html file that will run this script: wasmcheckers/index.html
As you did earlier in the chapter, you can host this file behind a simple Python web server: $ python3 -m http.server Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
You should now be able to point your browser at localhost:8000, open the developer tools, and see the following in your JavaScript console: At start, turn owner is 1 A piece moved from (0,5) to (0,4) A piece moved from (1,0) to (1,1) A piece moved from (0,4) to (0,3) A piece moved from (1,1) to (1,0) A piece moved from (0,3) to (0,2) A piece moved from (1,0) to (1,1) A piece was crowned at (0,0) A piece moved from (0,2) to (0,0) A piece moved from (1,1) to (1,0) A piece moved from (0,0) to (0,2)
The module works, so we’ve got that going for us. Take a step back and bask in the glow of knowing that you’ve written WebAssembly to handle in-memory game board state, turn management, game rule validation, and even notify the host when important events occur. Before you rush off to make millions
report erratum • discuss
Wrapping Up
• 41
on your new checkers game, there’s one more thing you really need to see— take a look at the file size of your compiled wasm module: $ ls -l total 48 -rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--rw-r--r--
1 1 1 1 1 1 1 1
kevin kevin kevin kevin kevin kevin kevin kevin
kevin kevin kevin kevin kevin kevin kevin kevin
853 9952 243 1063 184 1405 238 1072
Oct Oct Oct Oct Oct Oct Jul Jul
28 28 28 28 28 28 7 15
13:08 11:04 10:38 11:13 10:57 10:57 15:16 10:57
checkers.wasm checkers.wat func_test.html func_test.js func_test.wasm func_test.wat index.html index.js
Your eyes are not deceiving you—the compiled size of the checkers module is a mere 853 bytes! It might vary a bit depending on your OS and file system, but it will still be tiny. You also know, because you wrote all your own state management code, that the checkers module will never use more than a single 64KB page of memory. In fact, your module only uses a grand total of 256 bytes of linear memory. Just think about how many simultaneous, in-memory copies of a live checkers game a server could hold. You’ll see in subsequent chapters that as you add tooling and code generation, this will dramatically affect the size of your wasm modules. Some of the tools you’ve seen so far can also be used to trim out some of the excess fat from these modules, but you’ll never be able to create modules this small unless you’re hand-coding your wat.
Wrapping Up There was more to this chapter than just learning how to write WebAssembly “the hard way.” Even now, in the early days of the WebAssembly ecosystem, multiple layers of tooling and code generation are available that can easily hide the inner workings from you. Because you’ve written more than just a simple module that adds two numbers together, you know what’s going on at the bottom levels of abstraction. More importantly, you can appreciate what tools and code generators will do for you. This gives you a deeper understanding of the consequences of the Rust code you’ll write and the design decisions you’ll make when building WebAssembly solutions. Further, when you get to the chapter on hosting WebAssembly with Rust, you will already have an in-depth knowledge of the components of a WebAssembly module and so the parsing, loading, and executing tasks will all make more sense.
report erratum • discuss
Chapter 2. Building WebAssembly Checkers
• 42
In the next chapter, you’ll start to write at higher levels of abstraction, using Rust to target WebAssembly. As you explore that chapter, keep in mind what you’ve learned so far, as keeping a mental map between high-level code and its raw WebAssembly equivalent will help you squeeze every ounce of power, performance, and elegance from WebAssembly.
report erratum • discuss
Part II
Interacting with JavaScript
In this part, we’ll expand on our foundation by learning how to build WebAssembly modules with Rust and integrate them with JavaScript and the browser.
CHAPTER 3
Wading into WebAssembly with Rust The first part of this book introduced you to the world of WebAssembly. You learned about its internals and its architecture, what stack machines are, and where WebAssembly fits within the larger world of web applications. You even built a mostly functioning checkers module entirely in wast, the text representation of WebAssembly instructions. In this part of the book, you will focus not only on increasing your ability to interweave WebAssembly and JavaScript functionality, but you will also go through an introduction to Rust and you will see how you can use it to add strong types, memory safety, and elegant, expressive code to your WebAssembly modules. In this chapter, you’ll get an introduction to Rust and get your workstation tooling set up to target WebAssembly from Rust. By the end of this chapter, you’ll build a new version of the checkers module entirely in Rust. The Rust language has a longer learning curve than other languages like Go. As such, you might want to start skimming through the official Rust book1 to get familiar with some of the syntax coming up in the book. It’s not required, but consulting multiple reference sources is never a bad idea.
Introducing Rust If you’re already well versed in Rust, then feel free to skip to the next section. If you’re new to it or you’d like a refresher on the core concepts of the language, then you may find this section beneficial. Rust started in 2006 as Graydon Hoare’s personal project while he was working for Mozilla. Mozilla sponsored it in 2009 and announced it in 2010. In the history of each programming language, its genesis moment is when a 1.
doc.rust-lang.org/book/
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 46
compiler written in that language is able to compile that language—a compiler inception,2 if you will. For Rust, this came in 2011 and the first stable 1.0 release came in 2015. Rust has been gaining in popularity exponentially ever since. In addition to powering Mozilla’s new browser layout engine and showing up in a number of high-visibility systems and networking open source projects, it won the “Most Loved Programming Language” award in 2016, 2017, and 2018,3 surpassing Kotlin, Python, TypeScript, and even Go. This is Rust’s elevator pitch from the official website: Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.
Some of the main characteristics of the Rust language that make it appealing for building applications with WebAssembly include: Safety Rust doesn’t have the concept of null. Instead, any data that can be missing is represented as an Option type. This gets rid of an entire class of errors that plagues other systems languages like C and C++ and even many higher-level languages. In addition to lack of nulls, Rust will prevent your code from compiling if it could potentially create a data race, free already deallocated resources, or access something that has gone out of scope. Expressivity Despite being labeled as a “systems language,” Rust has a remarkably expressive syntax and includes many features that people typically laud when talking about functional programming: pattern matching, destructuring, streams, iterators, and much more. Combined with the concepts of traits and generics, Rust (most of the time) manages to facilitate highly readable and maintainable code. Performance Rust binaries are standalone, native binaries. They consume relatively little disk space, have a fairly small memory footprint, and the code generally performs extremely well. Despite all of the additional safety constraints in the language, Rust doesn’t need a garbage collector, which can often produce raw, C-like performance.
2. 3.
en.wikipedia.org/wiki/Bootstrapping_(compilers) insights.stackoverflow.com/survey/2018/#most-loved-dreaded-and-wanted
report erratum • discuss
Installing Rust
• 47
Installing Rust It’s time to start writing some Rust. The first thing you’ll need to do is follow the installation instructions from the official Rust4 website. On that site, you’ll find a curl script you can run that will install rustup, the main entry point into the Rust toolchain. Once you’ve completed the installation appropriate for your system, you should be able to interrogate the version of the Rust compiler: $ rustc --version rustc 1.30.1 (1433507eb 2018-11-07)
Rust is on a six-week release cycle for the stable toolchain, so your version will very likely be newer than the one in this book. You need to be able to successfully run rustc and rustup before continuing to the next section.
Building Hello WebAssembly in Rust Rust’s tools intrinsically support the notion of multiple targets. You can compile for different targets without having to run the compilation on that target machine. For example, you can target an ARM architecture (e.g., Raspberry Pi) from a regular workstation. WebAssembly manifests as a target in Rust’s toolchain. In addition to the regular six-week releases, Rust also has a nightly build that enables newer language features that haven’t yet stabilized. As of Rust version 1.30, most of the WebAssembly tools and libraries you’ll encounter in this book should build just fine on the stable toolchain.
Installing the WebAssembly Target If you issue the command, rustup toolchain list, you’ll see a toolchain that looks something like stable-x86_64-unknown-linux-gnu (default) (this will vary by operating system). Next, you can add the WebAssembly target by typing rustup target add wasm32-unknown-unknown. This may also take some time and will get everything set up for that target. If you’re curious about all of the targets Rust supports, you can type rustup target list. This list also shows which targets are installed and which one is the default.
4.
www.rust-lang.org/en-US/install.html
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 48
Creating a WebAssembly Project You can use Cargo, Rust’s project build tool, to create new empty projects. To create one for your first WebAssembly project, type the following: $ cargo new --lib rustwasmhello Created library `rustwasmhello` project
This creates a Cargo.toml file in the rustwasmhello directory. Edit this file so that it looks like this: rustwasmhello/Cargo.toml [package] name = "rustwasmhello" version = "0.1.0" authors = ["Your Name "] [lib] crate-type = ["cdylib"] [dependencies]
The only lines in this file that differ from what you get by default from cargo is the indicator that this project will expose a C-style dynamic library, which is then used by other linkers to produce a WebAssembly module: [lib] crate-type = ["cdylib"]
Now edit the lib.rs file that cargo created and set its contents to the following: rustwasmhello/src/lib.rs #[no_mangle] pub extern "C" fn add_one(x: i32) -> i32 { x + 1 }
This code should look familiar. It’s the same code that you saw in the first chapter when you created a WebAssembly Studio project online. This project should now be ready to compile into a WebAssembly module. The first step is to tell cargo to build it (make sure you’re in the rustwasmhello directory): $ cargo build --target wasm32-unknown-unknown Compiling rustwasmhello v0.1.0 (file:///home/kevin/Code/Rust/wasmbook_text/khrust/Book/code/rustwasmhello) Finished dev [unoptimized + debuginfo] target(s) in 0.24s
This produced a rustwasmhello.wasm file under the target/wasm32-unknown-unknown/debug folder. To see that your release build will produce a Wasm file very similar to something that you’d get building it by hand, first compile in release mode:
report erratum • discuss
Building Hello WebAssembly in Rust
• 49
$ cargo build --release --target=wasm32-unknown-unknown Compiling rustwasmhello v0.1.0 (/home/kevin/Code/Rust/wasmbook/khrust/Book/code/rustwasmhello) Finished release [optimized] target(s) in 0.12s
Now you can verify that the output contains an exported function called add_one by using the wasm-objdump tool: $ wasm-objdump -x target/wasm32-unknown-unknown/release/rustwasmhello.wasm rustwasmhello.wasm:
file format wasm 0x1
Section Details: Type: - type[0] () -> nil - type[1] (i32) -> i32 Function: - func[0] sig=0 - func[1] sig=1 Table: - table[0] type=anyfunc initial=1 max=1 Memory: - memory[0] pages: initial=16 Global: - global[0] i32 mutable=1 - init i32=1048576 - global[1] i32 mutable=0 - init i32=1048576 - global[2] i32 mutable=0 - init i32=1048576 Export: - memory[0] -> "memory" - table[0] -> "__indirect_function_table" - global[1] -> "__heap_base" - global[2] -> "__data_end" - func[1] -> "add_one" Custom: - name: ".debug_info" Custom: - name: ".debug_macinfo" Custom: - name: ".debug_pubtypes" Custom: - name: ".debug_ranges" Custom: - name: ".debug_abbrev" Custom: - name: ".debug_line" Custom: - name: ".debug_str" Custom: - name: ".debug_pubnames"
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 50
Custom: - name: "name" - func[0] - func[1]
You can see that this module comes with a single function table, a default 16-page (16 * 64KB) block of linear memory, some globals, and the add_one() function that we wrote. Congratulations, you’ve compiled your first Rust WebAssembly module. Now it’s time to learn some cool Rust features while you build out a new checkers module.
Creating Rusty Checkers This section has a very specific goal: build a new checkers module entirely in Rust WebAssembly that conforms as closely as possible to the interface of the hand-written one from the previous chapter. Diving into Rust is no small feat, so to keep things from getting too overwhelming, you’re going to be building something you’ve already built. Hopefully, the powerful and expressive syntax of Rust will make the game easier to reason about, and you might even be able to add features and rules that weren’t in the last game. This new version of checkers will expose the same kinds of functions to the host (JavaScript), but you’ll also be writing some code that determines the list of valid moves, something that was just too excruciatingly painful in raw wast code to write in the previous chapter. The Rust version will also have the same concept of turns so that it can be played from a console or virtually any other kind of client. To get started, find an empty directory and type the following: $ cargo new --lib rustycheckers Created library `rustycheckers` project
Edit the Cargo.toml file in the project’s root directory as you did for the last sample to tell the build system that this is a dynamic library. Once you’ve got that set up, you should be ready to start coding. If you’re using a version of Rust version 1.31.0 or newer, then your default Cargo.toml will come with a line in the package section: edition = "2018". This indicates the use of the “2018 edition”5 which allows for some newer (and often simpler) syntax, as well as many new features. 5.
rust-lang-nursery.github.io/edition-guide/rust-2018/index.html
report erratum • discuss
Creating Rusty Checkers
• 51
Make sure you remove this line from your Cargo.toml. For now, I want to focus on the core parts of Rust required to interact with WebAssembly, and will defer the use of the 2018 edition until the end of the book after you’ve had time to get used to the “regular” Rust syntax. Unless otherwise stated, all Rust samples in this book are designed to run on 1.31.0 or newer, but use the “2015 edition” syntax.
Setting Up the Board Open up the rustycheckers/src/lib.rs file and empty it. At the bottom of the file, add the following line: mod board;
Now add a file called rustycheckers/src/board.rs. Rust’s module system is hierarchical. Wherever you add mod [module_name]; in your code, think of that module as being injected at exactly that spot in the hierarchy. This allows you to nest modules, and it fosters some really good isolation and encapsulation practices, but it can be hard to get used to for people who come from languages with implicit or directory-first module hierarchies. Each Rust library has a single root module, the name of which is specified in the Cargo.toml file (the name setting under the [package] section). By convention, the code for the root module is found in the lib.rs file, so by including mod board; in the top of lib.rs, we’re declaring a submodule named board that exists directly under the root. Elements in the board submodule can be referenced from anywhere using the rustycheckers::board prefix. The Rust 2018 edition6 (not used until the end of this book) has more flexible rules around referring to modules. To start writing the code to manage the game board, the first thing you’ll want to do is write some code to manage a GamePiece. In the previous chapter, game pieces were just 32-bit integers that could be bitmasked for information. Now that you’ve got a little more power to play with, you can create an enum for piece color and a struct (Rust doesn’t officially have classes, though its structs can often behave like classes) to represent a single piece: rustycheckers/src/board.rs #[derive(Debug, Copy, Clone, PartialEq)] pub enum PieceColor { White, Black, }
6.
rust-lang-nursery.github.io/edition-guide/rust-2018/module-system/index.html
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 52
#[derive(Debug, Clone, Copy, PartialEq)] pub struct GamePiece { pub color: PieceColor, pub crowned: bool, } impl GamePiece { pub fn new(color: PieceColor) -> GamePiece { GamePiece { color, crowned: false, } } pub fn crowned(p: GamePiece) -> GamePiece { GamePiece { color: p.color, crowned: true, } } }
Don’t worry about the derive macros for now. They just auto-generate some boilerplate code to deal with common things most data structures in Rust need, like the ability to be compared, copied, cloned, and printed out for debug purposes. This game piece has two functions: new() and crowned(). The first creates a new game piece of a given color, while the second creates a new piece of a given color with a crown on top. The latter function will come in handy when you want to crown a piece on the board. Next, you can create the concept of a Coordinate. In raw WebAssembly, you didn’t have anything to represent this other than just numbers. Creating a struct for a grid coordinate here should help simplify the code, reduce the number of parameters that get passed around, and generally make things more readable. Also, as you’ll see, it’s a great place to stick some preliminary logic to support game rules: rustycheckers/src/board.rs #[derive(Debug, Clone, PartialEq, Copy)] pub struct Coordinate(pub usize, pub usize); impl Coordinate {
❶
pub fn on_board(self) -> bool { let Coordinate(x, y) = self; x = 2 { jumps.push(Coordinate(x + 2, y - 2)); } jumps.push(Coordinate(x + 2, y + 2)); if x >= 2 && y >= 2 { jumps.push(Coordinate(x - 2, y - 2)); } if x >= 2 { jumps.push(Coordinate(x - 2, y + 2)); } jumps.into_iter() }
❸ pub fn move_targets_from(&self) -> impl Iterator { let mut moves = Vec::new(); let Coordinate(x, y) = *self; if x >= 1 { moves.push(Coordinate(x - 1, y + 1)); } moves.push(Coordinate(x + 1, y + 1)); if y >= 1 { moves.push(Coordinate(x + 1, y - 1)); } if x >= 1 && y >= 1 { moves.push(Coordinate(x - 1, y - 1)); } moves.into_iter() } }
❶ An example of destructuring to pull the x and y values out of a Coordinate structure. ❷ Produce an iterator of Coordinates of all possible jumps from the given coordinate. ❸ Produce an iterator of Coordinates of all possible moves from the given coordinate. The jump_targets_from() function returns a list of potential jump targets from the given coordinate. The move_targets_from() function returns a list of potential move (adjacent space) targets given the current coordinate location. These functions will be used to calculate a list of valid moves for each turn. Note
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 54
that in the raw WebAssembly version of checkers, we just did a quick, naive check to verify that a move was valid. We can up the complexity a little here. The code pub struct Coordinate(pub usize, pub usize) defines what’s called a tuple struct. Rather than this struct having named fields, it instead represents a stronglytyped tuple with public fields. You can access those fields with tuple accessors .0 and .1, but most of the code in this chapter uses destructuring and pattern matching to get at the x and y values in a more human-friendly fashion. In this case, each of the two anonymous fields in this tuple struct are of type usize, which is the data type Rust uses for vector indexes, allowing code to be more safe and portable across 32-bit and 64-bit architectures. Rust’s standard library is replete with usages of this data type. The list of potential target squares for a checker piece to move is pretty small. The preceding code creates iterators that expose these potential squares to callers. As you’ll see shortly, iterators can be chained with functions like map(), zip(), and filter() to make for some fairly expressive syntax to define game rules. Finish out the board.rs module by creating a struct that represents a game move. Here you can see one place where it was simpler to access the tuple values as fields rather than using a pattern match: rustycheckers/src/board.rs #[derive(Debug, Clone, PartialEq, Copy)] pub struct Move { pub from: Coordinate, pub to: Coordinate, } impl Move { pub fn new(from: (usize, usize), to: (usize, usize)) -> Move { Move { from: Coordinate(from.0, from.1), to: Coordinate(to.0, to.1), } } }
One more thing before getting into the details of writing the game rules: references. In the coordinate code, &self is a reference to the struct. In Rust, anything prefixed with & is a reference. Any assignment that isn’t a reference is a move. Unless you explicitly treat something as a reference, ownership will be transferred during an assignment, meaning the value in the “old” variable will no longer be there (and you’ll get a compile error if you try to access it). This “move by default” pattern takes a bit of getting used to when learning Rust, and if the difference between move and reference is a little murky, you
report erratum • discuss
Creating Rusty Checkers
• 55
might want to take a minute and do a little background research on Rust’s concept of ownership.7
Writing the Engine Rules Now for the fun part... open up rustycheckers/src/lib.rs and add the following line of code to the bottom: mod game;
Create an empty file called rustycheckers/src/game.rs. This will give you a nice clean space to work on the game rules, and it illustrates a pretty common pattern in idiomatic Rust: modularization. I am partial to small, purposeful files that are easy to read and understand, and I feel like I spend more time thinking about modular structure in Rust than I have in other languages I’ve used (which is a good thing). The first thing you’ll want is a struct to anchor all of the game engine functionality to the same place, and to give you a spot to maintain game state. If you hadn’t noticed, thus far you haven’t done anything remotely related to WebAssembly, and that’s deliberate. You’ll want to keep the WebAssembly border clear of debris and make sure that you are free to build a game on this side of the Wasm/Rust border with very little crossover or coupling. Patterns like this where we actively avoid tightly coupling two separate concerns allow our code to be easier to read, easier to maintain, and easier to add more features and use for unintended consumers in the future without modification. This kind of tactic is often referred to as a facade or an AntiCorruption Layer: rustycheckers/src/game.rs use super::board::{Coordinate, GamePiece, Move, PieceColor}; pub struct GameEngine { board: [[Option; 8]; 8], current_turn: PieceColor, move_count: u32, } pub struct MoveResult { pub mv: Move, pub crowned: bool, }
In the previous chapter, you managed game state by manipulating bytes with direct memory access. Here, not only do you have a traditional two-dimensional 7.
doc.rust-lang.org/book/ch04-00-understanding-ownership.html
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 56
array, but the type of the piece is Option. It will make your code eminently more readable when an empty square is represented by the matchfriendly keyword None rather than just a 0. Remembering that the functionality for a struct resides in an impl block, start writing the code to initialize the game engine and its state inside the impl GameEngine block: rustycheckers/src/game.rs impl GameEngine { pub fn new() -> GameEngine { let mut engine = GameEngine { board: [[None; 8]; 8], current_turn: PieceColor::Black, move_count: 0, }; engine.initialize_pieces(); engine } pub fn initialize_pieces(&mut self) { [1, 3, 5, 7, 0, 2, 4, 6, 1, 3, 5, 7] .iter() .zip([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2].iter()) .map(|(a, b)| (*a as usize, *b as usize)) .for_each(|(x, y)| { self.board[x][y] = Some(GamePiece::new(PieceColor::White)); }); [0, 2, 4, 6, 1, 3, 5, 7, .iter() .zip([5, 5, 5, 5, 6, .map(|(a, b)| (*a as .for_each(|(x, y)| { self.board[x][y] });
0, 2, 4, 6] 6, 6, 6, 7, 7, 7, 7].iter()) usize, *b as usize)) = Some(GamePiece::new(PieceColor::Black));
}
The &mut self parameter to initialize_pieces() indicates that it can only be used by a mutable reference to a game engine. Further, it’s a sign to any developer that this function mutates state. In the engine’s constructor, we create a mutable instance of the GameEngine struct, and then call initialize_pieces() to set the board up for play. This function might seem confusing if you’re not used to Rust syntax. In the previous chapter, you initialized the board by manually placing every single piece with a different line of code. In the real world, we have tools like iterators and filters that can help us with these tasks.
report erratum • discuss
Creating Rusty Checkers
• 57
The iter() function converts an array of known x- or y-coordinate positions into an iterator, which can then be zipped (a function that merges two iterators into an iterator of tuples) with another iterator via the zip() function. Next, the map() function converts the coordinates from i32 to usize (the data type for array indexes in Rust—remember Rust doesn’t have implicit conversions). For each one of the newly zipped (x,y) coordinates, you can set the appropriate black or white piece using Rust’s anonymous function notation. The next bit of code takes a top-down approach and contains the logic to move pieces on the board. If you enter this by hand right now, it won’t compile because you’re missing a bunch of helper functions. But I think it’s easier to understand what’s happening by looking at this logic first: rustycheckers/src/game.rs pub fn move_piece(&mut self, mv: &Move) -> Result { let legal_moves = self.legal_moves(); if !legal_moves.contains(mv) { return Err(()); } let Coordinate(fx, fy) = mv.from; let Coordinate(tx, ty) = mv.to; let piece = self.board[fx][fy].unwrap(); let midpiece_coordinate = self.midpiece_coordinate(fx, fy, tx, ty); if let Some(Coordinate(x, y)) = midpiece_coordinate { self.board[x][y] = None; // remove the jumped piece } // Move piece from source to dest self.board[tx][ty] = Some(piece); self.board[fx][fy] = None; let crowned = if self.should_crown(piece, mv.to) { self.crown_piece(mv.to); true } else { false }; self.advance_turn(); Ok(MoveResult { mv: mv.clone(), crowned: crowned, }) }
The first thing move_piece() does is compute the list of legal moves based on whose turn it is and the state of the game board. If the intended move isn’t
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 58
in the list of legal moves (this is why you need the derive macros to generate equality tests for the structs), then the function quits early. There’s a line of code in this function that uses the unwrap() function. Ordinarily we shy away from this, but since we know that accessing that one piece will work because of validation, we can get away with it. In a production application and adhering to defensive coding practices, we would probably validate this instead of calling unwrap(). Next, the engine checks to see if there’s a game piece to jump on the way from the source to the destination coordinate. If there is, the jumped piece is removed from the board. The engine then performs the piece move by setting the old location to None. Finally, the function finishes by checking if it should crown the piece after it moved. Can you imagine how hard it would be to read or write the jump logic code in raw wast? The move_piece() function returns a Result. Similar to an Option, a Result can have two values: either Ok(...) or Err(...). Using a pattern match on a result is a clean way to handle and propagate errors back up the call stack. Pattern matching against optional values or result types is something that you might be familiar with if you’ve developed with functional programming languages or used functional programming styles.
Computing Legal Moves Computing the list of legal moves is the most complicated thing this Rust code does. In the following code, legal_moves() loops through every space on the board and then computes a list of valid moves from that position. This way, this function returns a list of every valid move that the current player can make (rusty checkers doesn’t support multi-jumps to try to keep the codebase legible and book-friendly). rustycheckers/src/game.rs fn legal_moves(&self) -> Vec { let mut moves: Vec = Vec::new(); for col in 0..8 { for row in 0..8 { if let Some(piece) = self.board[col][row] { if piece.color == self.current_turn { let loc = Coordinate(col, row); let mut vmoves = self.valid_moves_from(loc); moves.append(&mut vmoves); } } } }
report erratum • discuss
Creating Rusty Checkers
• 59
moves } fn valid_moves_from(&self, loc: Coordinate) -> Vec { let Coordinate(x, y) = loc; if let Some(p) = self.board[x][y] { let mut jumps = loc .jump_targets_from() .filter(|t| self.valid_jump(&p, &loc, &t)) .map(|ref t| Move { from: loc.clone(), to: t.clone(), }).collect::(); let mut moves = loc .move_targets_from() .filter(|t| self.valid_move(&p, &loc, &t)) .map(|ref t| Move { from: loc.clone(), to: t.clone(), }).collect::(); jumps.append(&mut moves); jumps } else { Vec::new() } }
There’s an interesting pattern used in the valid_moves_from() function that uses the iterator functions created earlier and chains those results through filter(), map(), and collect(): let mut moves = loc .move_targets_from() .filter(|t| self.valid_move(&p, &loc, &t)) .map(|ref t| Move { from: loc.clone(), to: t.clone(), }).collect::();
This takes all of the potential move targets and filters them based on whether that target is a valid move for a given game piece, at a given location, for a given target. Then, for each of the valid moves, we convert the coordinate target (indicated by the ref t in the lambda, showing that we’re taking the lambda’s sole parameter by reference) into a Move struct. Finally, the resulting iterator is converted into a vector via collect() and the “turbofish”8 syntax collect::().
8.
doc.rust-lang.org/book/2018-edition/appendix-02-operators.html?highlight=turbo#non-operator-symbols
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 60
The valid_moves_from() function produces a list of Move instances that are valid from the given coordinate. In formal checkers, if a player has both a valid jump and a valid move, they must perform the jump. This method is structured to place valid jumps first so that you can add that kind of strict rule checking if you want. You could also add a new property to the Move struct to tag a move as a jump, which might help in adding multi-jump capabilities to the game. You still haven’t added anything to the code that deals with WebAssembly. In the interest of saving a few trees (or bytes), the full listings for all of the various helper functions like valid_move() aren’t in the book. If you want the full source for board.rs and game.rs, you can grab it from the book’s full example code. You’ll also see several unit tests in the full version of the code that shows how you can test this code without crossing the WebAssembly boundary.
Coding the Rusty Checkers WebAssembly Interface So far you’ve been coding in pure Rust without any indication that the game will eventually be available as a WebAssembly module. This is actually a good practice to adopt, and I’m a very big fan of keeping the boundary-crossing code at the edges, leaving the domain-specific code (in our case, checkers) in its own separate module to test in isolation. You’ll need to do two things to make this code work with a JavaScript host. First, you’ll need to export functions that can be called by the host. Second, you’ll need to import functions that you want to be called by the WebAssembly code on the host. This follows the same pattern as the previous chapter, but with stricter boundaries. First, add the following two dependencies to your Cargo.toml file: [dependencies] mut_static = "5.0.0" lazy_static = "1.0.2"
Rust is ruthless about its control over shared mutability, and here arises a conflict in philosophies. The previous raw WebAssembly module had global, module-wide state. How can you accomplish the same thing in Rust without violating all of its rules about sharing and mutability? You’re going to use something called a lazy static to create a globally available instance of the GameEngine struct. This instance will then be used by all of the functions you’re exporting out of the WebAssembly module. The following code creates a lazy static game engine. Put it at the top of lib.rs:
report erratum • discuss
Coding the Rusty Checkers WebAssembly Interface
• 61
rustycheckers/src/lib.rs #[macro_use] extern crate lazy_static; use board::{Coordinate, GamePiece, Move, PieceColor}; use game::GameEngine; use mut_static::MutStatic; lazy_static! { pub static ref GAME_ENGINE: MutStatic = { MutStatic::from(GameEngine::new()) }; }
In nearly every single case when writing Rust code, I would vehemently and aggressively push against the use of global mutable state. It goes against everything that Rust stands for, is bad for the environment, and makes puppies cry. However, in our edge case, like providing a mutable state store for a WebAssembly module, it is pretty much the only way to make the module work. If you recall from the previous chapter, you exported a couple of functions out of your wat file to allow the host to query the state of the game: get_piece() and get_current_turn(). You’ll rewrite those now: rustycheckers/src/lib.rs #[no_mangle] pub extern "C" fn get_piece(x: i32, y: i32) -> i32 { let engine = GAME_ENGINE.read().unwrap(); let piece = engine.get_piece(Coordinate(x as usize, y as usize)); match piece { Ok(Some(p)) => p.into(), Ok(None) => -1, Err(_) => -1, } } #[no_mangle] pub extern "C" fn get_current_turn() -> i32 { let engine = GAME_ENGINE.read().unwrap(); GamePiece::new(engine.current_turn()).into() }
To maintain the safety of the globally mutable game engine, you have to acquire a read or write lock with the read() or write() functions. Since there’s a chance that acquiring one of these locks can fail, those functions return Results. You have to handle results one way or another, and here calling unwrap() either grabs the value within Ok(...) or crashes the program upon failure. A tip learned from painful experience: in production-grade
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 62
applications, seek out and destroy all unwrap()s with very few exceptions. Being unable to acquire a read or write lock on your game’s sole source of state is, however, something that is exceptional and should produce a trap for the host. The raw version of the checkers module you wrote returned the current piece and the current turn owners both as i32 values. As mentioned at the beginning of the chapter, you’ll adhere to this contract wherever possible. If you look at both of these functions, you may notice that they both make use of a function called into(). The into() function shows one of my favorite aspects of Rust in action. Anything that implements the generic trait called Into for a given type can be converted into that type. While Rust doesn’t have implicit conversions, its explicit conversions are quite powerful. To convert a game piece into a 32-bit integer, you just need to implement Into: rustycheckers/src/lib.rs const PIECEFLAG_BLACK: u8 = 1; const PIECEFLAG_WHITE: u8 = 2; const PIECEFLAG_CROWN: u8 = 4; impl Into for GamePiece { fn into(self) -> i32 { let mut val: u8 = 0; if self.color == PieceColor::Black { val += PIECEFLAG_BLACK; } else if self.color == PieceColor::White { val += PIECEFLAG_WHITE; } if self.crowned { val += PIECEFLAG_CROWN; } val as i32 } }
The fact that the self variable here is not a reference means that the GamePiece will be consumed when converted into an integer. In other words, if you try to access a game piece after you convert it into an integer, you’ll get a compilation error with a message like “move after use,” and an arrow pointing to the spot where you used it and where it moved. Rust’s compiler errors are some of the most verbose and helpful I’ve ever seen. There are some cool Rust libraries that help you do bit flags, but aren’t really needed for this sample. Next, add a function to let players move:
report erratum • discuss
Coding the Rusty Checkers WebAssembly Interface
• 63
rustycheckers/src/lib.rs #[no_mangle] pub extern "C" fn move_piece(fx: i32, fy: i32, tx: i32, ty: i32) -> i32 { let mut engine = GAME_ENGINE.write().unwrap(); let mv = Move::new((fx as usize, fy as usize), (tx as usize, ty as usize)); let res = engine.move_piece(&mv); match res { Ok(mr) => { unsafe { notify_piecemoved(fx, fy, tx, ty); } if mr.crowned { unsafe { notify_piececrowned(tx, ty); } } 1 } Err(_) => 0, } }
The move_piece() function (it can’t be called move() because that’s a reserved word in Rust) just forwards the call to the game engine and examines the result. It does, however, call the two notification functions that need to be imported from the host: notify_piecemoved() and notify_piececrowned(). These functions are both wrapped inside an unsafe block because Rust can’t guarantee that the code on the other side of the host-module barrier meets Rust’s standards for safety. Any extern function must be wrapped in an unsafe block to invoke it. It’s a best practice to keep unsafe blocks as small as possible, wrapping only the thing you need to invoke. Unsafe here shouldn’t instill a sense of fear in you. This kind of unsafe is just an indicator the code you’re invoking is unknown to the Rust compiler, so it can’t give you the safety guarantees that you get from regular compiled Rust code. Importing functions from a host is done by including their signatures in an extern block: rustycheckers/src/lib.rs extern "C" { fn notify_piecemoved(fromX: i32, fromY: i32, toX: i32, toY: i32); fn notify_piececrowned(x: i32, y: i32); }
report erratum • discuss
Chapter 3. Wading into WebAssembly with Rust
• 64
Finally, you can compile this code for the WebAssembly target and create the stripped binary file suitable for use with JavaScript. The following commands will build your WebAssembly module and place the trimmed-down, release version in a demo directory (you’ll need to create this directory yourself): $ cargo build --release --target wasm32-unknown-unknown Finished release [optimized] target(s) in 0.01s $ cp target/wasm32-unknown-unknown/release/rustycheckers.wasm demo/
At this point, you’ve made a stack (definitely not a heap) of progress. You’ve got a Rust library that compiles to WebAssembly, imports notification functions, and exports functions that will let a host play a game of checkers while maintaining internal state. Next, you’ll see how that works by testing out the game in a JavaScript host.
Playing Rusty Checkers in JavaScript With just a few minor changes, your new Rust-built checkers WebAssembly module should work the same way as it did before. To get started, copy the index.html from the previous chapter into the demo directory. Next, create the demo/index.js file and edit its contents to match the following: rustycheckers/demo/index.js fetch('./rustycheckers.wasm').then(response => response.arrayBuffer() ).then(bytes => WebAssembly.instantiate(bytes, { ➤ env: { notify_piecemoved: (fX, fY, tX, tY) => { console.log("A piece moved from (" + fX + "," + fY + ") to (" + tX + "," + tY + ")"); }, notify_piececrowned: (x, y) => { console.log("A piece was crowned at (" + x + "," + y + ")"); } }, } )).then(results => { instance = results.instance; console.log("At start, current turn is " + instance.exports.get_current_turn()); let piece = instance.exports.get_piece(0, 7); console.log("Piece at 0,7 is " + piece); let res = instance.exports.move_piece(0, 5, 1, 4); // B console.log("First move result: " + res); console.log("Turn after move: " + instance.exports.get_current_turn());
report erratum • discuss
Wrapping Up
• 65
let bad = instance.exports.move_piece(1, 4, 2, 3); // illegal move console.log("Illegal move result: " + bad); console.log("Turn after illegal move: " + instance.exports.get_current_turn()); }).catch(console.error);
The first real difference between this JavaScript and the code from the previous chapter is that the object name wrapping the functions to be imported by the checkers module is env. In the previous chapter, you had control over the namespace name of the imports and called it events. Rust code built for the wasm32-unknown-unknown target will default to putting function imports in the env namespace. It’s not that big of an inconvenience, but being aware of subtleties like this can save you headaches in the future. The rest of the code should look nearly identical, except that now when you attempt to make a truly illegal move, the new and improved code will catch it and return 0. Just for fun, see if you can set up a series of moves that allows a single jump so you can verify that a jump really does remove a piece from the board.
Wrapping Up This chapter was fairly dense and covered a lot of material. You installed Rust, set up the wasm32-unknown-unknown target, and built a functioning checkers module with Rust. Now that you’ve experienced first-hand building the checkers module from scratch using nothing but wast syntax, you can compare and contrast that with building the same functionality with the additional benefits of Rust’s strong type system, safety, and expressiveness. So far, you’ve been spending nearly all of your time inside the WebAssembly module and little to no time working with the browser host. In the coming chapters, that’s going to change as you unlock more features with tooling and code generation designed to bridge the gap between WebAssembly modules and their hosts. You’ll start building real web applications with seamless user experiences.
report erratum • discuss
CHAPTER 4
Integrating WebAssembly with JavaScript So far, everything you’ve done in this book has been isolated almost entirely within the realm of WebAssembly. You’ve written a checkers engine in raw wast, then you wrote an upgraded one in Rust and compiled it to the wasm32unknown-unknown target. Even though both of these projects had places where JavaScript could attach its tentacles, you can’t really refer to either of those samples as tightly integrated. In this chapter, you’ll take a look at the Rust WebAssembly ecosystem, including the tooling and libraries available to help bridge the gap between WebAssembly and JavaScript. You’ll start off by creating a new “Hello, World” template that illustrates a new way of communicating between JavaScript and Rust. Next, you’ll explore tools like wasm-bindgen that allow Rust to see JavaScript classes (and JavaScript to use Rust structures), expose and invoke callbacks in either language, send strings as function parameters, and return complex values, all while maintaining Rust’s strict sharing rules. By the end of the chapter, you’ll not only know the mechanics of how to interoperate with JavaScript, but you’ll have seen patterns and examples of when and where you should divide your logic up between JavaScript and Rust WebAssembly modules by building an interactive, browser-based game.
Creating a Better “Hello, World” Most of the information in the book thus far can be applied universally to all kinds of WebAssembly applications written in all kinds of languages. That path diverges in this chapter. From here on out, everything in the book will be specific to Rust. More importantly, everything you do after this point will rely on tools or libraries created by the Rust community. As interest in WebAssembly with
report erratum • discuss
Chapter 4. Integrating WebAssembly with JavaScript
• 68
Rust grows, we can only expect this community to grow, and the power and usefulness of its tools and libraries to grow along with it. Even as young as WebAssembly is, you already have quite a few Rust tools at your disposal. While you have a degree of choice when it comes to JavaScript bindings, the one that you’ll be using in this chapter is wasm-bindgen1. This is a combination of a set of crates (Rust’s name for shared libraries, though there’s more nuance to them than that) that support bindings between JavaScript and Rust. At its core, wasm-bindgen injects a bunch of metadata into your compiled WebAssembly module. Then, a separate command-line tool reads that metadata, strips it out, and uses that information to generate an appropriate JavaScript “wrapper bridge” containing the kinds of functions, classes, and other primitives that the developer wants bound to Rust.
Installing the New Tools wasm-bindgen uses procedural macros and a few other features that at one point
were only available in the nightly build of Rust. Thankfully, during the course of writing this book, Rust’s support for those features is now stable. Now that you’re going to be writing a bit more JavaScript, you’ll be using features that often call for the use of npm. Refer to the instructions2 for your operating system to install Node and the Node Package Manager (npm). Finally, you’ll need to install the wasm-bindgen command-line tool. To do that, you’ll use cargo, Rust’s build tool: $ cargo install wasm-bindgen-cli
If through previous experiments you’ve already installed wasm-bindgen and you want to force the installation of the latest version, add --force to the end of the cargo install command. This might take quite a while as there are a large number of dependencies and each one gets compiled after the source is downloaded.
Creating a New Rust WebAssembly Project In this section you’ll be creating a new WebAssembly module that makes use of wasm-bindgen bindings and its CLI, as well as webpack, npm, and a few other tools. As a self-identified “back end” developer, I get a little nervous when people mention all of these JavaScript build tools. Don’t worry, though, we only need a few, and most of that is just to get a basic web server running. Also as a non-authority on JavaScript, JavaScript developers may find far 1. 2.
rustwasm.github.io/wasm-bindgen www.npmjs.com/get-npm
report erratum • discuss
Creating a Better “Hello, World”
• 69
more optimal ways of accomplishing some of the tasks in this chapter than the way I’ve outlined. You’ll go through the process of setting up this “Hello, World” piece by piece, and when you’re done, you’ll have a nice template that you can use as scaffolding to build future projects (which will come in handy in the second half of this chapter). To start, create a new Rust project called bindgenhello (this is in the jsint_bindgenhello directory in the book’s code samples) in a clean root directory: $ cargo new bindgenhello --lib
This should look familiar. As with all the other Rust WebAssembly projects, you need to change its library type to cdylib in Cargo.toml. Also, add a reference to wasmbindgen (and make sure you delete the “2018 edition” line if you have it): jsint_bindgenhello/Cargo.toml [package] name = "bindgenhello" version = "0.1.0" authors = ["Your Name "] [lib] crate-type = ["cdylib"] [dependencies] wasm-bindgen = "0.2"
The last time you went through this exercise, you created a simple function that performed addition. This time, you’ll clean out the lib.rs file and replace it with the following: jsint_bindgenhello/src/lib.rs extern crate wasm_bindgen; use wasm_bindgen::prelude::*; // Import 'window.alert' #[wasm_bindgen] extern "C" { fn alert(s: &str); } // Export a 'hello' function #[wasm_bindgen] pub fn hello(name: &str) { alert(&format!("Hello, {}!", name)); }
Decorating Rust code with #[wasm_bindgen] triggers the invocation of a compiletime Rust macro. Each time the compiler encounters this macro, it generates
report erratum • discuss
Chapter 4. Integrating WebAssembly with JavaScript
• 70
some code on your behalf. Some of it will be code that winds up in your .wasm module, but some of it will be metadata used to help generate the corresponding JavaScript output produced by the wasm-bindgen command-line tool. In this lib.rs file, there are two bindings. The first binds the alert() function to the alert() JavaScript function. With this binding in place, any Rust code that invokes the alert() function will be converted into a bunch of code that invokes the JavaScript alert() function from inside a WebAssembly module. Attaching the right window context and making all of the JavaScript pieces work properly is all done for us by wasm-bindgen. The second binding exposes the hello() function. You’ve seen how this kind of function can be exposed before. However, in this case, it takes a reference to a string as a parameter. We know that string parameters aren’t possible in pure WebAssembly, so the generated wrapper code produces the necessary plumbing (in both wast instructions and JavaScript boilerplate) to allow complex data to flow seamlessly between boundaries. Behind the scenes, memory allocation and disposal functions are created that operate on the module’s linear memory (remember the good old days when you had to do that by hand?). Then, each time wasm-bindgen encounters the need for string allocation, the generated JavaScript invokes those functions. In short, all of the hard work you’ve been doing in the past couple of chapters is now done automatically on your behalf. These wrappers are convenient, of course, but I still firmly believe that you are better off for having learned how things were done “the hard way.” Go ahead and build this to make sure that you get a valid WebAssembly module: $ cargo build --target wasm32-unknown-unknown
There’s one final piece to this compilation that you need to complete when you’re using wasm-bindgen—invoke the CLI to produce a new WebAssembly module and a JavaScript wrapper file: $ wasm-bindgen target/wasm32-unknown-unknown/debug/bindgenhello.wasm \ --out-dir .
This drops a new file, bindgenhello_bg.wasm, in the project directory. It also generates the wrapper JavaScript file, bindgenhello.js, and a TypeScript definition (bindgenhello.d.ts) in that directory. Since these are all generated, you might want to exclude them from your version control system, though checking generated code into VCS is actually a pretty nuanced subject, so your mileage may vary. I’ve included a build.sh script in the code samples that produces the .wasm file and then calls wasm-bindgen.
report erratum • discuss
Creating a Better “Hello, World”
• 71
Cargo Update Occasionally, when you go to build a project that built successfully the week before (or when this book’s samples were generated, for instance), you may see weird errors in libraries that you didn’t write. This happens sometimes with conflicts between locally cached builds and libraries pulled from the internet. Most of the time you should be able to resolve this by running cargo update and then attempting the build again. You may also be prompted to update if the versions of wasm-bindgen used in the CLI and in your .wasm module don’t match.
The following function is auto-generated, but it’s worth looking at the wrapper function for hello(): export function hello(arg0) { const ptr0 = passStringToWasm(arg0); const len0 = WASM_VECTOR_LEN; try { return wasm.hello(ptr0, len0); } finally { wasm.__wbindgen_free(ptr0, len0 * 1); } }
The memory offset and length of the allocated string are returned from a function called passStringToWasm(). This function invokes the generated allocation function inside the WebAssembly module, placing the encoded string in the module’s linear memory and then doing the relevant pointer arithmetic. Having written your own wast code, you should be able to appreciate how great it is to have this code generated on your behalf. After the hello() function is done, the code will free the previously allocated memory via the __wbindgen_free() function that wasm-bindgen stuffed into the WebAssembly module for us. With the WebAssembly side of this “Hello, World” done, it’s time to move on to the JavaScript side of the house.
Integrating with JavaScript and npm In order to run this sample in a browser like you did before, you’ll need a web server and some way of serving up your script content that invokes the wasm module. In addition, for this sample, you’re going to set up a webpack configuration. Follow the appropriate instructions to ensure you’re using the latest version of webpack. If you’re a JavaScript pro (unlike myself), then feel free to use whatever tooling feels most comfortable to you.
report erratum • discuss
Chapter 4. Integrating WebAssembly with JavaScript
• 72
There are some handy shortcuts you can use with webpack to do things like automatically generate the index.html. To keep things simple and easier to understand, I’m deliberately not optimizing certain things in this book. You could also choose not to use webpack at all. The real work happens in the index.js file. This is where it really pays to use additional tools. You can see that it looks like simple, clean, idiomatic JavaScript, even though it’s going through a bridge to integrate with a WebAssembly module. It’s truly the best of both worlds—the Rust code targeting WebAssembly looks like idiomatic Rust, and the JavaScript looks like standard, unmodified JavaScript: jsint_bindgenhello/index.js const wasm = import('./bindgenhello'); wasm .then(h => h.hello("world!")) .catch(console.error);
Next, set up a web pack configuration so that you can use it to manage your JavaScript bundles. Note that the entry point is index.js. Previous versions of this book required some shoe-horning and other shenanigans to get this working, but the WebAssembly ecosystem is always improving, and things are getting simpler every day: jsint_bindgenhello/webpack.config.js const path = require('path'); const HtmlWebpackPlugin = require('html-webpack-plugin'); const webpack = require('webpack'); module.exports = { entry: './index.js', output: { path: path.resolve(__dirname, 'dist'), filename: 'index.js', }, plugins: [ new HtmlWebpackPlugin(), // Have this example work in Edge which doesn't ship `TextEncoder` or // `TextDecoder` at this time. new webpack.ProvidePlugin({ TextDecoder: ['text-encoding', 'TextDecoder'], TextEncoder: ['text-encoding', 'TextEncoder'] }) ], mode: 'development' };
report erratum • discuss
Building the Rogue WebAssembly Game
• 73
Finally, create a package.json file. In addition to setting up your webpack development web server, you could also use this as a place to automate your build process by calling the build.sh shell script or something similar: jsint_bindgenhello/package.json { "scripts": { "build": "webpack", "serve": "webpack-dev-server" }, "devDependencies": { "text-encoding": "^0.7.0", "html-webpack-plugin": "^3.2.0", "webpack": "^4.11.1", "webpack-cli": "^3.1.1", "webpack-dev-server": "^3.1.0" } }
Now you should be able to execute a build script to produce the WebAssembly module and the generated JavaScript files, then npm run serve to start the webpack server. Pointing your WebAssembly-enabled browser at your local host on port 8080 should then pop up a JavaScript alert dialog box. Try this out on your own and bask in the glow of autogenerated JavaScript interop goodness. With this “Hello, World” project template in hand, it’s time to move on to building something a little more powerful with the help of wasm-bindgen.
Building the Rogue WebAssembly Game Back in the days before the Internet filled up with pictures of kittens, animated memes, and ubiquitous social networking, university students had to walk three miles uphill in the snow to computer labs so they could play games on monochrome monitors tethered to Unix servers. One of these games was an incredible creation called Rogue.3 Remember when I suggested that constraints are often good for innovation? Rogue is a fantastic example of that. Fed up with text games that you could only play once, Rogue’s creators managed to let players hack and slash their way through procedurally generated dungeons in a simple 80 column by 24 row terminal. For this code sample, you’ll be taking a JavaScript library for creating Roguelike games and building a game with it. The point of this exercise isn’t to build a game (though that is a fun side effect), but rather to illustrate some possible 3.
www.gamasutra.com/view/feature/4013/the_history_of_rogue_have__you_.php?print=1
report erratum • discuss
Chapter 4. Integrating WebAssembly with JavaScript
• 74
ways for WebAssembly to interact with your JavaScript code and third-party JavaScript libraries. We want to examine strategies for spreading code and logic across the boundary between Rust (WebAssembly) and JavaScript. It’s important to prepare for our decision about where to draw those boundaries and how to spread the code to be wrong. We will make an attempt, see how it plays, see what the code looks like, and then decide how to refactor it from there. Perfect software isn’t created, it comes from iteration and the deliberate choice of an imperfect starting point. In the Rogue WebAssembly game, your objective is to find and open all of the treasure chests. Inside one of these chests is a WebAssembly module. You will have to find this module before the dreaded Rust Borrow Checker captures you!
Getting Started with Rot.js Before getting started, you might want to take a few minutes to familiarize yourself with the Rot.js4 library. You don’t need to become an expert. Just take a look at some of the basic documentation. In short, Rot.js injects a virtual 80x24 (you can resize it) terminal window into an HTML canvas. With that in place, you can use methods like draw() to place characters on the map, and hook up a turn-based actor system. This library has far more functionality than you’ll use for this simple example (but plenty of goodies to play with if you want to add on later). You can get started by making a new copy of the better “Hello, World” from the previous section. I called my new directory roguewasm, but you can call yours whatever you like. Make sure you’ve got a build script that builds the wasm module and invokes the wasm-bindgen CLI. You can do this with a shell script or with changes to package.json. You won’t need to add a reference to Rot.js for npm. Instead, add that reference to the index.html file, where it will be clear that this is a traditional, client-side JavaScript dependency. There are other ways to rely on these kinds of dependencies, but this isn’t a JavaScript book so I will steer clear of them. The game screen consists of the canvas area managed by Rot.js, a header for the game title, and a sidebar that will serve as an area to display the player’s statistics. Here’s a very simple index.html file that uses CSS flex grid to create those regions (it may be difficult to tell, but I am not a designer):
4.
ondras.github.io/rot.js/manual/#intro
report erratum • discuss
Building the Rogue WebAssembly Game
• 75
jsint_roguewasm/index.html
Rogue WebAssembly
Rogue WebAssembly
Stats
HitPoints: 0 / 0
Moves: 0