Monitoring the blockchain

Monitoring the blockchain is possibly an important part of your application on NEM. The most obvious example being monitoring activities involving your account. Two approaches are possible.

The first is polling, meaning that regularly you retrieve the last block and handle it. If your polling interval is significantly smaller than the average block generation time, you can be sure you will handle all blocks. And if your polling interval is bigger and multiple blocks have been generated between two polling, you can still request all new blocks in your handler. This approach has downsides, notably that the client is regularly sending queries, even if activity of interest to the poller occured on the blockchain. This means that the scalability of this approach is limited. But its development is very easy and might be sufficient in most cases.

The second way is subscribing to notifications from the NIS instance you connect to. This is done with websockets. This is the way it is done in the NanoWallet. This is a cleaner approach, but is a bit more complex.

Active monitoring: the polling approach

The pollign approach is very easy to explain, and it translates immediately in runnable code in any language, expecially if it provides REST and JSON libraries. Here are the steps to do in a loop:

The number of seconds x we wait (the polling interval) will determine if we will handle all blocks. If it is smaller than the average generation time of a block, we can be sure that all blocks will be retrieved at least once by this code. We call this rapid polling in opposition of slow polling. An interval of 15 seconds seems safe for rapid polling.

Rapid polling

It is quite straight-forward to translate that to code, even in a bash script using httpie and jq. Let’s see how this can be done!

First we initialise 2 variables: the monitored address, and the last block seen by the script which is set to 0 at the start of the script:

As we will continually monitor the blockchain, our code will be wrapped in an infinite while loop:

In the loop, we retrieve the last block by sending a GET query to /chain/last-block, and save the result in variable block:

This will retrieve the last block in a json document. Here is an example from the mainnet chain showing the format of the document retrieved.

Parsing that with jq is easy. The height of the block retrieved is available in the top level property height, extract with jq using the filter .height. As the JSON describing the last block is saved in variable $block, we can simply pipe it to jq

The transactions are stored in the top level property named transactions as an array value, that can be empty if no transaction is included in that block. This corresponds to a jq filter '.transactions[]?, the question mark indicating that it can be empty. But we are only interested in the transactions to our address. This means that the property recipient of the transaction should hold a value equal to our $observed_address. This corresponds to a filte select(.recipient=="'$observed_address'"). Assigning the transaction select by our filter to a variable found_recipient results in this code:

That variable will be empty if no transaction involving the monitored address was found. We can use the bash empty string test [[ -n $variable ]] to determine if the block holds a transaction to our monitored address:

Putting all this in a script results in this. You can select another tab to see the same implementation in that language:

#!/bin/bash

observed_address="TA6XFSJYZYAIYP7FL7X2RL63647FRMB65YC6CO3G"
last_block_analysed=0
while true; do
    # get last block
    block=$(http :7890/chain/last-block)
    echo "$block"
    chain_height=$(echo "$block" | jq '.height')
    if [[ $last_block_analysed -eq $chain_height ]]; then
        echo "Waiting at $chain_height"
        sleep 15
    else
        last_block_analysed=$chain_height
        found_recipient=$(echo "$block" | jq -r '.transactions[]?|select(.recipient=="'$observed_address'")')
        if [[ -n $found_recipient ]] ; then
          echo "Our monitored address received a transaction!"
        else
          echo "Not our observed recipient"
        fi
    fi
done

require 'rest-client'
require 'json'

observed_address="TC6LCF7ZCL5WL4HTE64JPUU2UPVKNZB2LMVUNELV"
last_block_analysed=0
while true
  r=RestClient.get 'localhost:7890/chain/height'
  chain_height=JSON.parse(r.body)["height"]
  if last_block_analysed==chain_height
    puts "sleeping"
    sleep 15
  else
    r=RestClient.get('localhost:7890/chain/last-block')
    b=JSON.parse(r.body)
    last_block_analysed=b["height"]
    b["transactions"].select{|t| t["recipient"]=="observed_address"}.each do |recipient|
      puts "We received a transaction!"
    end
  end
end

Slow polling

If the polling interval is too big, the code has to be prepared to query blocks that were generated during the polling interval.

As this requires a bit more logic than the previous example, we will structure our code a bit better. For each operation we will define a function. The first one is to retrieve the current blockchain height, which issues a get request to /chain/height.

We will also need a function to retrieve a block at a specific height. This is done by passing a JSON payload specifying the height to a POST request:

This code sends a POST request to /block/at/public, with a JSON payload specifying the height of the block to return, and sets HTTP headers regarging the content type.

We will also need to check if a block as a transaction to the observed address, and this is done with this function:

What this code is first collect all recipients found in transaction, and then check if this array contains our recipient of interes.

With these helper functions in place, putting the pieces together is a nearly literal translation of the steps highlighted above. We require the needed libraries, define the helper functions, initialise the variables holding the address observed, and the height of the last block analysed (in our case set to the current height), resulting in this code:

Passive monitoring: the subscription approach

While in the previous sections our program was actively contacting our NIS instance to check if action should be taken because a new block was generated, we can use a passive approach in which our program subscribes to notification of specific events. This is done over a websocket connection.

The NEM Infrastructure Server listens for websocket connections on port 7778. It expects STOMP formatted messages. If your language provides libraries to communicate with STOMP over websockets, it should be easy to interoperate with NIS. Just note that Websocksets require a handshake, which is visible at the stomp client level. If you are interested in more details, a good explanation is available on the web.

Typescript and nem-library

nem-library provides a great abstraction layer, and makes this passive monitoring really easy. You just initialise an addres; a, UnconfirmedTransactionListener passing it the NIS to connect to, and you are reasy to go! Here’s the code, from the nem-library documentation.

Javascript and nem-sdk

We will work in javascript, for which the library stompjs covers our needs. These can be installed with

You might find examples using SockJS, and this guide started by using it. However, due to problems with SockJS (such as code working one day but not the day after), we switched to a solution using only stompjs.

The implementation is quite straightforward when you know how and to which messages you need to subscribe. We will work in the nodejs REPL, which you can start with node, or if you use the containers of this guide, with ./ndev repl.js.

The URL you need to connect to for websockets is http://$NIS:7778/w/messages/websocket (replace $NIS with the IP or hostname of the NIS you want to connect to). If you run in the tools container, you can use localhost.

To be notified of new blocks, you subscribe to /blocks/new. The body of the notification is simply the height of the new block.

Now that we have all information, we can take a closer look at the implementation. First we require the libraries:

Note that if you use the repljs script with ndev repl.js, these are already available and this step is not required.

As mentioned above, websockets require a handshake, and we need to initiate it from the STOMP client. When the connection is established, it will trigger a callback passed as second argument to the connect function:

The callback is a function taking as argument the frame received from the server. In this case it will be the frame confirming the connection. In the callback you can subscribe to the messages of interest. In this example we will subscribe to the new block notification, and simply log the height of the new block in the console:

This will notify your program when a new block is available, but you would still need to go and retrieve the block to check if it includes transactions involving your account. But we can do better! We can ask to be notified of new transactions involving a specific account. This is done by subscribing to the channel /w/api/account/subscribe with the payload "{'account':'$ADDRESS'}" (replace $ADDRESS by the account’s address, all uppercase and without hyphen). Here is an example illustrating the format of the frame you receive:

With this information, it is easy to write our program that will log to the console the amount in microXEMs of transactions involving our account:

You might want to check if this is an incoming our outgoing transaction. But that’s left as an exercice for the reader ;-)