David - Musings of an SRE

A Python RethinkDB walkthrough

RethinkDB is a modern NoSQL document-based database that baked in push/pull features into its database core which allows developers to build realtime applications without having to add a websocket/messaging layer ontop of their existing stack.

If you’re familar with MongoDB and its schemaless document structure, RethinkDB works in a similar manner with flexible schemas.

With RethinkDB, you can write a client in any language (Python, JS, Ruby etc…) to listen to specific database queries and perform operations when there’s a change in the database.

In this quick walkthrough, as an example, I’m going to show you how to write a python client that can monitor a database for changes.

This is not meant to be a serious tutorial but just a quick and dirty way to show you what you can do with RethinkDB.

Setting up RethinkDB

I’d recommend the best way to get started is to use the RethinkDB docker image. Its a quick way to setup an instance without having to install all sorts of dependencies.

If you have Docker installed, you can quickly get things rolling with just:

$ docker run -d -p 8080:8080 -p 28015:28015 -p 29015:29015 dockerfile/rethinkdb

If you prefer not to use docker, you can follow the installation instructions at the official RethinkDB page

Note: Your RethinkDB actual is listening on port 28015 and its webadmin panel is at 8080. You can check out the really neat admin panel by hitting http://localhost:8080 in your browser.

Setup the Python Client

You’re going to need the rethinkdb python client library.

Lets do a

$ pip install rethinkdb

Schema

As an example, lets say we’re building a server monitoring tool that records a server’s hostname and its alert status in the database.

Lets bring up the python console to create the database

$ python
>>>> import rethinkdb as r
>>>> r.connect("localhost", 28015).repl()
>>>> r.create_db('pingdom').run()
>>>> r.db('pingdom').table_create('servers').run()

This would create a database called pingdom and a table servers which we will use to store information about the servers that we would like to monitor.

Inserting Data into RethinkDB

Let’s now add a record of a server that the system is supposed to monitor into our table.

$ python
>>>> r.table('servers').insert([{"host": '127.0.0.1', "status": 0])

Monitoring Data Changes

With the server information added, lets say our alerting system detects a failure in one of the servers and updates the alert attribute of the specific entry in the database table to 1, to signify that an alert should be done, how can we monitor that change?

This is where RethinkDB shines. All you need is a simple chained .changes() function and you can write in a quick script that returns you both the old value and new value of any database entry changes.

# client.py

import rethinkdb as r

connection = r.connect("localhost", 28015)
r.db('pingdom').run(connection)
changes = r.table('servers').changes().run(connection)
for document in changes:
  print(document)

When a change is made in the above example, document would return something like:

{
  "new_val": {
    "id": "1d854219-85c6-4e6c-8259-dbda0ab386d4",
    "host": "127.0.0.1",
    "alert": 1},
  "old_val": {
    "id": "1d854219-85c6-4e6c-8259-dbda0ab386d4",
    "name": "127.0.0.1",
    "alert": 0
  }
}

All this happening in real-time without having to build a pub/sub layer in between.

Cool right?

References