Getting values with the right type in Redis

Question

I'm using redis in my python application to store simple values like counters and time stamp lists, but trying to get a counter and comparing it with a number I came across a problem.

If I do:

import redis
...
myserver = redis.Redis("localhost")
myserver.set('counter', 5)

and then try to get that value like this:

if myserver.get('counter') < 10:
     myserver.incr('counter')

then I get a type error in the if statement because I'm comparing '5' < 10, which means I'm storing an integer value and getting a string one (which can be considered as a different value).

My question is: is this supposed to work like that? I mean its a very basic type, I understand if I have to parse objects but an int? Seems that I'm doing something wrong.

Is there any configuration I'm missing?

Is there any way to make redis return the right type and not always a string? I say this because its the same for lists and datetimes or even floating point values.

Could this be a problem with the redis-py client I'm using and not redis itself?

Solution 1

Technically speaking you need to take care of that on your own.

However, have a look at this link, especially at the part of their README that refers to parsers and response callbacks, maybe that's something you can use. Question would be whether this is an overkill for you or not.

Solution 2

As @favoretti said, response callbacks will do the trick. It's not complicate at all, just one line and all will be taken care of.

In [2]: import redis
In [3]: r = redis.Redis()
In [10]: r.set_response_callback('HGET', float)
In [11]: r.hget('myhash', 'field0')
Out[11]: 4.6

for hmget, it returns a list of strings, not one single string, so you need to construct a little more comprehensive callback function:

In [12]: r.set_response_callback('HMGET', lambda l: [float(i) for i in l])

In [13]: r.hmget('myhash', 'field0')
Out[13]: [4.6]

same for hgetall.

Solution 3

Though leveraging set_response_callback is fine for simple data types, if you're wondering about the fastest-and-easiest route to storing things like dictionaries, list, tuples – and preserving the python-native datatypes they may-or-may-not contain – I recommend using python's built-in pickle library:

# Imports and simplified client setup
>>> import pickle
>>> import redis
>>> client = redis.Redis()
# Store a dictionary
>>> to_store = {'a': 1, 'b': 'A string!', 'c': [1, True, False, 14.4]}
>>> client.set('TestKey', pickle.dumps(to_store))
True
# Retrieve the dictionary you just stored.
>>> retrieved = pickle.loads(client.get('TestKey'))
{'a': 1, 'b': 'A string!', 'c': [1, True, False, 14.4]}

Here's a simple client that will reduce the pickle boilerplate in the above example and provide you with a clean interface for storing and retrieving native python datatypes to and from Redis:

"""Redis cache."""
import pickle
import redis

redis_host = redis.Redis()


class PythonNativeRedisClient(object):
    """A simple redis client for storing and retrieving native python datatypes."""

    def __init__(self, redis_host=redis_host):
        """Initialize client."""
        self.client = redis_host

    def set(self, key, value, **kwargs):
        """Store a value in Redis."""
        return self.client.set(key, pickle.dumps(value), **kwargs)

    def get(self, key):
        """Retrieve a value from Redis."""
        val = self.client.get(key)
        if val:
            return pickle.loads(val)
        return None

redis_client = PythonNativeRedisClient()

Usage:

>>> from some_module import redis_client
>>> to_store = {'a': 1, 'b': 'A string!', 'c': [1, True, False, 14.4]}
>>> redis_client.set('TestKey', to_store)
True
>>> retrieve = redis_client.get('TestKey')
{'a': 1, 'b': 'A string!', 'c': [1, True, False, 14.4]}

Solution 4

You can set decode_respone like True

redis.StrictRedis(host="localhost", port=6379, db=0, decode_responses=True)

Solution 5

Looks like that is how redis stores its data:

redis 127.0.0.1:6379> set counter 5
OK
redis 127.0.0.1:6379> type counter
string
redis 127.0.0.1:6379> incr counter
(integer) 6
redis 127.0.0.1:6379> type counter
string

If you really want to, you could probably monkeypatch the redis-py client to infer data types.

View more solutions