Working with JSON and PNaCl using jsoncpp

When I started developing the PHP Ninja Manual I’d hoped that the IndexedDB API for client-side storage would be fast enough to search through 10,000 terms. Well, it is — but there’s a catch. It’s only speedy if it makes use of indices. For example, it’s quick if you look for items that start with your search term. However, if you also wanted to search for items that contain your search term — or make something similar to “fuzzy search” — it’s very slow (about two to three seconds) since you’d have to iterate over all stored items. To remedy that, I made a small Native Client (NaCl) app in C++ that iterates entire 10k array and returns all items that start with or contain your search term. With that app, search time dropped to about 20ms.

In this tutorial, I’ll show you a vital mechanic required to build such a solution: how to exchange data between Portable Native Client (PNaCl) and JavaScript using the jsonccp library. I assume nobody wants to send simple strings to/from NaCl, but rather some data structure that can be converted into a string like JSON. We’ll make a simple PNaCl app that takes an array of numbers and returns its sum.

There are plenty of C or C++ JSON libraries, but they likely require some customization to compile for architectures supported by NaCl. In addition, there are many libraries already prepared for you on naclports but for us, there’s a precompiled jsoncpp that comes with the NaCl SDK.

Before we start

You’ll need the following in order to install everything.

Follow the instructions on this page and Install NaCl SDK. Be sure to grab the latest stable released, dubbed pepper.
Go through the C++ tutorial: Getting Started. If you finish this tutorial, you’ll have all necessary components to compile and run NaCl modules. We’ll use this demo source code as a template for our app.

Actually, we’re not going to make an NaCl app, per se, but rather PNaCl app, which is architecture independent. For us it means that we can build a single binary instead of 4 for each platform (X86-32, X86-64, MIPS32 and ARM). My PHP Ninja Manual uses standard NaCl because at the time I was developing it PNaCl didn’t exist and, to be honest, compiling NaCl was quite painful.

Right now, PNaCl is recommended instead of NaCl. If you can spare a minute, read up on PNaCl and NaCl here:

NaCl and PNaCl – These are probably the most important things you need to know. Pay attention to the section “When to use NaCl”.
Introduction to Portable Native Client
Google Native Client on Wikipedia
NaCl/PNaCl Application Structure – Interesting reading about PNaCl/NaCl app structure and handy C/C++ methods.
Building NaCl/PNaCl app – More in-depth information about PNaCl/NaCl compiler.

Please, make sure that your environment variable NACL_SDK_ROOT exists and is set to the latest pepper_# directory. If not, create it with:

export NACL_SDK_ROOT=/home/you/nacl_sdk/pepper_#

Although, you can compile “C++ tutorial: Getting Started” without NACL_SDK_ROOT, I think it’s better to set it because you can avoid dealing with weird messages about missing files and libraries.

Making a simple PNaCl + jsoncpp app

It might seem redundant that I’m renaming all files from hello_tutorial.* to json_tutorial.* but I want to highlight what files you have to edit (and where they are!) in case you wanted to start making your own PNaCl app.

1. Make a copy of `getting_started/part1` from `pepper_#` directory.

It’s much easier to start from this than writing everything from scratch.

cp -r pepper_35/getting_started/part1/ ~/pnacl_jsoncpp

Of course, you can choose whatever directory you want. Now, rename hello_tutorial.cc and hello_tutorial.nmf to json_tutorial.cc and json_tutorial.nmf, respectively.

2. Open `Makefile`

Change all occurrences of hello_tutorial to json_tutorial.

Run make in your ~/pnacl_jsoncpp directory. This is just to make sure that you renamed everything properly. If it prints some errors make sure you have your NACL_SDK_ROOT set and double check Makefile.

Also, find the line that starts with LDFLAGS (in pepper_35 it’s line 34). These are linker flags and lppapi_cpp and -lppapi are libraries ppapi_cpp and ppapi. Add -ljsoncpp at the end (-lis parameter and whatever follows it is the name of linked library). The PNaCl compiler knows where to find the library already because jsoncpp comes with pepper out of the box. LDFLAGS definition should look like:

LDFLAGS := -L$(NACL_SDK_ROOT)/lib/pnacl/Release -lppapi_cpp -lppapi -ljsoncpp

3. Open `json_tutorial.cc`

Rename all occurrences of HelloTutorialInstance and HelloTutorialModule to JsonTutorialInstance and JsonTutorialModule, respectively.

Add the following top of the file:

#include "json/json.h"
#include <sstream>

Again, the PNaCl compiler knows where to look for header files (it’s pepper_#/include). We’re including sstream in order to use std::stringstream later.

Next, we’re going to define a helper method that sends a JSON response back to JavaScript and wraps it with some basic structure. You can put this method right after HandleMessage in JsonTutorialInstance class.

void send_json(const char *status, Json::Value response_data) {
// Json::Value is a wrapper structure around any data (object, array, number, string)
// we're passing to its constructor Json::objectValue which basically creates
// a JavaScript object or associative array if you want
Json::Value response = Json::Value(Json::objectValue);
response["status"] = status; // like in any other language, this creates key and assign in value
response["response"] = response_data;
// object that converts Json::Value into it's JSON string representation
Json::FastWriter writer;
std::string json_output = writer.write(response);
// wrap C++ string with Var object used by ppapi and send it to JavaScript
// https://developer.chrome.com/native-client/pepper_stable/cpp/classpp_1_1_var
PostMessage(pp::Var(json_output));
}

Now let’s keep it very easy and implement HandleMessage in the most simple way:

virtual void HandleMessage(const pp::Var& var_message) {
send_json("ok", Json::Value(42));
}

This will respond with simple JSON to all messages that come from JavaScript. For more information about jsoncpp see the jsoncpp documentation. I couldn’t figure out what version of jsoncpp is included in the latest pepper_35 but I think it’s 0.5.0, but the version on sourceforge.net is 0.6.0-rc2. Unfortunately, if you want to see documentation for 0.5.0 you have to clone their git repository, checkout to 0.5.0 and build the documentation on your own.

Now go to terminal, make sure you’re in ~/pnacl_jsoncpp directory and run:

make clean
make

4. Open `index.html`

Rename all occurrences of HelloTutorialModule to JsonTutorialModule
Find <embed id=“hello_tutorial” … /> and change its id and src attributes to json_tutorial and hello_tutorial.nmf respectively.
Find document.getElementById('hello_tutorial'); and change it to document.getElementById(‘json_tutorial');.

We’ll append a message call to JsonTutorialModule which is a DOM element representing our PNaCl app so replace moduleDidLoad() with:

function moduleDidLoad() {
JsonTutorialModule = document.getElementById('json_tutorial');
updateStatus('SUCCESS');
JsonTutorialModule.postMessage('hello');
}

Then replace handleMessage() in order to print parsed JSON into console:

function handleMessage(message_event) {
alert(message_event.data);
console.log(JSON.parse(message_event.data));
}

NaCl SDK comes with a simple Python web server that you already used in “C++ tutorial: Getting Started” and we’ll use it to test our plugin. Note, that you can’t just double click index.html because it will refuse to open json_tutorial.nmf. I guess this is due to the same-origin policy.

Make sure you’re in ~/pnacl_jsoncpp directory and run:

python $NACL_SDK_ROOT/tools/httpd.py --no-dir-check

By default, this runs a web server on port 5103 so open http://localhost:5103/ in Chrome and you should see:

If you see this alert window, it means PNaCl loaded properly, received our message and responded. This means it’s alive!

5. Add more logic and parse received JSON string in our PNaCl app

So far, it’s pretty simple. Now we want our app to parse strings we send it from JavaScript into Json::Value, which we can use further. It’s time to rewrite the method HandleMessage():

virtual void HandleMessage(const pp::Var& var_message) {
Json::Value root; // Will contain the root value after parsing.
Json::Reader reader;
// try to parse message
if (!reader.parse(var_message.AsString(), root)) {
// Report to the user that it failed and where it failed.
std::stringstream error_message_stream("Failed to parse JSON: ");
error_message_stream << reader.getFormatedErrorMessages();
send_json("error", error_message_stream.str());
return;
}
// Our root has to be an object (like a JavaScript
// object or associative array in other languages).
if (!root.isObject()) {
send_json("error", "invalid data structure");
return;
}
// second parameter is default value
const std::string action = root.get("action", "unknown").asString();
Json::Value data = root["data"];
if (action == "sum" && data.isArray()) {
int sum = 0;
for (uint32_t i=0; i < data.size(); i++) {
sum += data[i].asInt();
}
send_json("ok", Json::Int(sum));
} else {
send_json("error", "unknown action");
}
}

Finally update moduleDidLoad() for the last time:

var msg = {
'action': 'sum',
'data': [2,4,11,13,23]
}
function moduleDidLoad() {
JsonTutorialModule = document.getElementById('json_tutorial');
updateStatus('SUCCESS');
JsonTutorialModule.postMessage(JSON.stringify(msg));
}

Now, when you refresh http://localhost:5103/ it should return 53 as an integer (not inside double quotes).

If you’re not keen on using copy and paste to recreate our app, you can find the source code on GitHub.

A few notes

NaCl/PNaCl supports debugging, although I’ve never managed to set it up properly. You can read more about debugging here. There’s also a Visual Studio plugin.
Probably the easiest way to see what’s going on inside is by using PostMessage(pp::Var(var));
I tried to develop NaCl/PNaCl apps in XCode 5, although there’s no dedicated plugin for that. I haven’t managed to setup a debug environment, but you can at least use custom build targets that use your makefile (you have to define NACL_SDK_ROOT in build options) and use code completion if you provide your IDE with ppapi/cpp/*.h or json/json.h.

Author: Martin Sikora

Working with JSON and PNaCl using jsoncpp

Before we start

Making a simple PNaCl + jsoncpp app

1. Make a copy of getting_started/part1 from pepper_# directory.

2. Open Makefile

3. Open json_tutorial.cc