Models

Warning

Walrus models should not be considered production-grade code and I strongly advise against anyone actually using it for anything other than experimenting or for inspiration/learning.

My advice: just use hashes for your structured data. If you need ad-hoc queries, then use a relational database.

Walrus provides a lightweight Model class for storing structured data and executing queries using secondary indexes.

>>> from walrus import *
>>> db = Database()

Let’s create a simple data model to store some users.

>>> class User(Model):
...     __database__ = db
...     name = TextField(primary_key=True)
...     dob = DateField(index=True)

Note

As of 0.4.0, the Model.database attribute has been renamed to Model.__database__. Similarly, Model.namespace is now Model.__namespace__.

Creating, Updating and Deleting

To add objects to a collection, you can use Model.create():

>>> User.create(name='Charlie', dob=datetime.date(1983, 1, 1))
<User: Charlie>

>>> names_dobs = [
...     ('Huey', datetime.date(2011, 6, 1)),
...     ('Zaizee', datetime.date(2012, 5, 1)),
...     ('Mickey', datetime.date(2007, 8, 1)),

>>> for name, dob in names_dobs:
...     User.create(name=name, dob=dob)

We can retrieve objects by primary key (name in this case). Objects can be modified or deleted after they have been created.

>>> zaizee = User.load('Zaizee')  # Get object by primary key.
>>> zaizee.name
'Zaizee'
>>> zaizee.dob
datetime.date(2012, 5, 1)

>>> zaizee.dob = datetime.date(2012, 4, 1)
>>> zaizee.save()

>>> nobody = User.create(name='nobody', dob=datetime.date(1990, 1, 1))
>>> nobody.delete()

Retrieving all records in a collection

We can retrieve all objects in the collection by calling Model.all(), which returns an iterator that successively yields model instances:

>>> for user in User.all():
...     print(user.name)
Huey
Zaizee
Charlie
Mickey

Note

The objects from all() are returned in an undefined order. This is because the index containing all primary keys is implemented as an unordered Set.

Sorting records

To get the objects in order, we can use Model.query():

>>> for user in User.query(order_by=User.name):
...     print(user.name)
Charlie
Huey
Mickey
Zaizee

>>> for user in User.query(order_by=User.dob.desc()):
...     print(user.dob)
2012-04-01
2011-06-01
2007-08-01
1983-01-01

Filtering records

Walrus supports basic filtering. The filtering options available vary by field type, so that TextField, UUIDField and similar non-scalar types support only equality and inequality tests. Scalar values, on the other hand, like integers, floats or dates, support range operations.

Warning

You must specify index=True to be able to use a field for filtering.

Let’s see how this works by filtering on name and dob. The query() method returns zero or more objects, while the get() method requires that there be exactly one result:

>>> for user in User.query(User.dob <= datetime.date(2009, 1, 1)):
...     print(user.dob)
2007-08-01
1983-01-01

>>> charlie = User.get(User.name == 'Charlie')
>>> User.get(User.name = 'missing')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/charles/pypath/walrus.py", line 1662, in get
    raise ValueError('Got %s results, expected 1.' % len(result))
ValueError: Got 0 results, expected 1.

We can combine multiple filters using bitwise and and or:

>>> low = datetime.date(2006, 1, 1)
>>> high = datetime.date(2012, 1, 1)
>>> query = User.query(
...     (User.dob >= low) &
...     (User.dob <= high))

>>> for user in query:
...     print(user.dob)

2011-06-01
2007-08-01

>>> query = User.query(User.dob.between(low, high))  # Equivalent to above.
>>> for user in query:
...     print(user.dob)

2011-06-01
2007-08-01

>>> query = User.query(
...     (User.dob <= low) |
...     (User.dob >= high))

>>> for user in query:
...     print(user.dob)
2012-04-01
1983-01-01

You can combine filters with ordering:

>>> expr = (User.name == 'Charlie') | (User.name == 'Zaizee')
>>> for user in User.query(expr, order_by=User.name):
...     print(user.name)
Charlie
Zaizee

>>> for user in User.query(User.name != 'Charlie', order_by=User.name.desc()):
...     print(user.name)
Zaizee
Mickey
Huey

Container Fields

Up until now the fields we’ve used have been simple key/value pairs that are stored directly in the hash of model data. In this section we’ll look at a group of special fields that correspond to Redis container types.

Let’s create a model for storing personal notes. The notes will have a text field for the content and a timestamp, and as an interesting flourish we’ll add a SetField to store a collection of tags.

class Note(Model):
    __database__ = db
    text = TextField()
    timestamp = DateTimeField(
        default=datetime.datetime.now,
        index=True)
    tags = SetField()

Note

Container fields cannot be used as a secondary index, nor can they be used as the primary key for a model. Finally, they do not accept a default value.

Warning

Due to the implementation, it is necessary that the model instance have a primary key value before you can access the container field. This is because the key identifying the container field needs to be associated with the instance, and the way we do that is with the primary key.

Here is how we might use the new note model:

>>> note = Note.create(content='my first note')
>>> note.tags
<Set "note:container.tags.note:id.3": 0 items>
>>> note.tags.add('testing', 'walrus')

>>> Note.load(note._id).tags
<Set "note:container.tags.note:id.3": 0 items>

In addition to SetField, there is also HashField, ListField, ZSetField.

Need more power?

walrus’ querying capabilities are extremely basic. If you want more sophisticated querying, check out StdNet. StdNet makes extensive use of Lua scripts to provide some really neat querying/filtering options.