Errata for Redis in Action

booktitle

Last updated December 14, 2015

You can view the most recent version of this errata here

Known errata

Recent updates

Chapter 2

Section 2.5, Page 35, Listing 2.10 (2014/05/04)

There is a bug on the third line of the rescale_viewed() function definition.

The full function definition reads:

def rescale_viewed(conn):
    while not QUIT:
        conn.zremrangebyrank('viewed:', 20000, -1)
        conn.zinterstore('viewed:', {'viewed:': .5})
        time.sleep(300)

The third line, updated inline, should read:

def rescale_viewed(conn):
    while not QUIT:
        conn.zremrangebyrank('viewed:', 0, -20001)
        conn.zinterstore('viewed:', {'viewed:': .5})
        time.sleep(300)

You can see the updated code in-context by visiting: http://goo.gl/1hlS3m

Chapter 5

Section 5.1.2, Page 93, Listing 5.2 (2015/10/10)

There is a bug caused by a missing elif clause inside the try block of the log_common() function definition.

The original full function definition reads:

def log_common(conn, name, message, severity=logging.INFO, timeout=5):
    severity = str(SEVERITY.get(severity, severity)).lower()
    destination = 'common:%s:%s'%(name, severity)
    start_key = destination + ':start'
    pipe = conn.pipeline()
    end = time.time() + timeout
    while time.time() < end:
        try:
            pipe.watch(start_key)
            now = datetime.utcnow().timetuple()
            hour_start = datetime(*now[:4]).isoformat()

            existing = pipe.get(start_key)
            pipe.multi()
            if existing and existing < hour_start:
                pipe.rename(destination, destination + ':last')
                pipe.rename(start_key, destination + ':pstart')
                pipe.set(start_key, hour_start)

            pipe.zincrby(destination, message)
            log_recent(pipe, name, message, severity, pipe)
            return
        except redis.exceptions.WatchError:
            continue

And with the added elif block (prefixed by a comment line below), the function should read:

def log_common(conn, name, message, severity=logging.INFO, timeout=5):
    severity = str(SEVERITY.get(severity, severity)).lower()
    destination = 'common:%s:%s'%(name, severity)
    start_key = destination + ':start'
    pipe = conn.pipeline()
    end = time.time() + timeout
    while time.time() < end:
        try:
            pipe.watch(start_key)
            now = datetime.utcnow().timetuple()
            hour_start = datetime(*now[:4]).isoformat()

            existing = pipe.get(start_key)
            pipe.multi()
            if existing and existing < hour_start:
                pipe.rename(destination, destination + ':last')
                pipe.rename(start_key, destination + ':pstart')
                pipe.set(start_key, hour_start)
            # add the following two lines
            elif not existing:
                pipe.set(start_key, hour_start)

            pipe.zincrby(destination, message)
            log_recent(pipe, name, message, severity, pipe)
            return
        except redis.exceptions.WatchError:
            continue

You can see the change in-context by visiting: https://goo.gl/UN5kMw

Chapter 6

Section 6.2.3, Page 121, Listing 6.9 (2013/08/24, 2014/02/16, 2014/04/22)

There is one printing errata (wrong in the printed version, correct in the source code), three other bugs, and three cleanups in this listing for purchase_item_with_lock().

Printing errata: there is a missing line between the two lines that read:

locked = acquire_lock(conn, market)
    return False

With the missing line replaced, it should read:

locked = acquire_lock(conn, market)
if not locked:
    return False

You can see the code in-context by visiting: http://goo.gl/QpcbuC

Bug: there is an extra pipe.watch(buyer) call that breaks the remaining behavior, which can be removed. The lines that read:

try:
    pipe.watch(buyer)
    pipe.zscore("market:", item)
    pipe.hget(buyer, 'funds')

Should instead read:

try:
    pipe.zscore("market:", item)
    pipe.hget(buyer, 'funds')

Further, there are missing 'funds' arguments to the pipe.hincrby() calls later, *and* a misnamed argument. The lines that read:

pipe.hincrby(seller, int(price))
pipe.hincrby(buyerid, int(-price))

Should instead read:

pipe.hincrby(seller, 'funds', int(price))
pipe.hincrby(buyer, 'funds', int(-price))

Note the addition of the 'funds' arguments and the renaming of the buyerid argument to buyer.

You can see the change in-context by visiting: http://goo.gl/kQltLd

Bug: the calls to acquire_lock() and release_lock() reference the variable market when they should instead pass the string 'market:'.

There are also several additional (but unnecessary) cleanups that can be done to this listing. These changes include: 1) removing the while loop, 2) removing the try/except clause, 3) removing the unnecessary pipe.unwatch() call.

You can see the change for these cleanups and the fixed market -> 'market:' reference inline by visiting: http://goo.gl/LxGvV8

Section 6.5.2, Page 145, Listing 6.28 (2015/04/10)

There are two bugs on the last line of the leave_chat() function. The first bug is where the oldest argument to the conn.zremrangebyscore() call should really be oldest[0][1] (this was discovered on 2015-04-10). The second bug is what reads as 'chat:' should read 'msgs:' (this was discovered on 2015-12-14). The lines that read:

        'chat:' + chat_id, 0, 0, withscores=True)
        conn.zremrangebyscore('chat:' + chat_id, 0, oldest)

Should instead read:

        'chat:' + chat_id, 0, 0, withscores=True)
        conn.zremrangebyscore('msgs:' + chat_id, 0, oldest[0][1])

You can see the code in-context by visiting: https://goo.gl/T8zCp2

Chapter 7

Section 7.3.4, Page 177, Listing 7.15 (2014/04/22)

There is a printing errata and bug in the record_click() function.

Printing errata and bug: in the printed book, there is a missing else: line between the two pipeline.incr() calls below, and the first pipeline.incr() call is missing a '%s' for the string templating to work. So lines 13-15 in the book:

    if action and type == 'cpa':
        pipeline.incr('type:cpa:actions:' % type)
        pipeline.incr('type:%s:clicks:' % type)

Should instead read:

    if action and type == 'cpa':
        pipeline.incr('type:%s:actions:' % type)
    else:
        pipeline.incr('type:%s:clicks:' % type)

You can see the change in-context with the else: clause by visiting: http://goo.gl/XXiNTD

Chapter 8

Section 8.1.1, Page 187, Listing 8.1 (2015/04/13)

There is a bug in the create_user() between lines 7 and 8, where there is a missing release_lock() call prior to return. In practice, this may cause a 1 second delay in deleting a status message for a user if someone were trying to create a new account with the same user name.

Lines 7-8 of create_user() originally read:

    if conn.hget('users:', llogin):
        return None

Lines 7-8 should be replaced with these 3 lines:

    if conn.hget('users:', llogin):
        release_lock(conn, 'user:' + llogin, lock)
        return None

You can see the code in-context by visiting: http://goo.gl/quX7ee

Section 8.3, Page 190-191, Listing 8.4 (2015/04/13)

There is a race condition in the follow_user() function, caused by directly setting the size of the following/follower lists instead of incrementally changing them. This is fixed as part of other updates to follow_user() in section 10.3.3, listing 10.12, and we are just bringing a couple of these changes back to chapter 8.

Lines 12-21 originally read:

    pipeline.zadd(fkey1, other_uid, now)
    pipeline.zadd(fkey2, uid, now)
    pipeline.zcard(fkey1)
    pipeline.zcard(fkey2)
    pipeline.zrevrange('profile:%s'%other_uid,
        0, HOME_TIMELINE_SIZE-1, withscores=True)
    following, followers, status_and_score = pipeline.execute()[-3:]

    pipeline.hset('user:%s'%uid, 'following', following)
    pipeline.hset('user:%s'%other_uid, 'followers', followers)

With 2 lines deleted and 2 lines changed, lines 12-19 should now read:

    pipeline.zadd(fkey1, other_uid, now)
    pipeline.zadd(fkey2, uid, now)
    pipeline.zrevrange('profile:%s'%other_uid,
        0, HOME_TIMELINE_SIZE-1, withscores=True)
    following, followers, status_and_score = pipeline.execute()[-3:]

    pipeline.hincrby('user:%s'%uid, 'following', int(following))
    pipeline.hincrby('user:%s'%other_uid, 'followers', int(followers))

You can see the code in-context by visiting: http://goo.gl/mvmxwV

Section 8.3, Page 191-192, Listing 8.5 (2015/04/13)

There is a race condition in the unfollow_user() function, caused by directly setting the size of the following/follower lists instead of incrementally changing them. This is the exact same bug that occurs in the follow_user() function, and has the same fix.

Lines 12-21 originally read:

    pipeline.zrem(fkey1, other_uid, now)
    pipeline.zrem(fkey2, uid, now)
    pipeline.zcard(fkey1)
    pipeline.zcard(fkey2)
    pipeline.zrevrange('profile:%s'%other_uid,
        0, HOME_TIMELINE_SIZE-1, withscores=True)
    following, followers, status_and_score = pipeline.execute()[-3:]

    pipeline.hset('user:%s'%uid, 'following', following)
    pipeline.hset('user:%s'%other_uid, 'followers', followers)

With 2 lines deleted and 2 lines changed, lines 12-19 should now read:

    pipeline.zrem(fkey1, other_uid, now)
    pipeline.zrem(fkey2, uid, now)
    pipeline.zrevrange('profile:%s'%other_uid,
        0, HOME_TIMELINE_SIZE-1, withscores=True)
    following, followers, status_and_score = pipeline.execute()[-3:]

    pipeline.hincrby('user:%s'%uid, 'following', int(following))
    pipeline.hincrby('user:%s'%other_uid, 'followers', int(followers))

You can see the code in-context by visiting: http://goo.gl/mxcUqZ

Section 8.4, Page 194, Listing 8.8 (2015/04/13)

There is a bug in the delete_status() function between lines 7 and 8, where there is a missing release_lock() call prior to return. In practice, this may cause a 1 second delay in deleting a status message for a user if someone tried to delete a status message they didn't own.

Lines 7-8 of delete_status(conn, uid, status_id) originally read:

    if conn.hget(key, 'uid') != str(uid):
        return None

Lines 7-8 should be replaced with these 3 lines:

    if conn.hget(key, 'uid') != str(uid):
        release_lock(conn, key, lock)
        return None

You can see the code in-context by visiting: http://goo.gl/gL6dPG

Section 8.5.3, Page 201-202, Listing 8.14 (2015/04/13)

There is a bug in the streaming delete_status() function between lines 7 and 8, where there is a missing release_lock() call prior to return. In practice, this may cause a 1 second delay in deleting a status message for a user if someone tried to delete a status message they didn't own.

Lines 7-8 of delete_status(conn, uid, status_id) originally read:

    if conn.hget(key, 'uid') != str(uid):
        return None

Lines 7-8 should be replaced with these 3 lines:

    if conn.hget(key, 'uid') != str(uid):
        release_lock(conn, key, lock)
        return None

You can see the code in-context by visiting: http://goo.gl/DzweRD

Section 8.5.3, Page 205, Listing 8.19 (2014/07/30)

There is a bug in the FollowFilter() function on lines 2, 4, and 10 of the code listing, where the argument names is overridden by an empty set, which prevents the FollowFilter() call from actually following anyone.

The function originally read:

def FollowFilter(names):
    names = set()
    for name in names:
        names.add('@' + name.lower().lstrip('@'))

    def check(status):
        message_words = set(status['message'].lower().split())
        message_words.add('@' + status['login'].lower())

        return message_words & names
    return check

The function (with comments here to show the changes) now reads:

def FollowFilter(names):
    nset = set() # first fix here
    for name in names:
        nset.add('@' + name.lower().lstrip('@')) # second fix here

    def check(status):
        message_words = set(status['message'].lower().split())
        message_words.add('@' + status['login'].lower())

        return message_words & nset # third fix here
    return check

The diff (which might be easier to read) can be found: http://goo.gl/Opbp13

And the code in-context can be seen: http://goo.gl/GkqXp5

Chapter 11

Section 11.3.2, Page 260-261, Listing 11.12 (2015/04/22)

This code listing is a copy of the listing from Chapter 6, Section 6.2.3, Page 121, Listing 6.9, and has all of the same issues except for the printing errata and the market variable reference bug.