1. home •
  2. blog •
  3. picky •
  4. phd •
  5. phony •
  6. view models

Searching with Picky: Redis

ruby / picky / gems / redis

This is a post in the Picky series on its workings. If you haven’t tried it yet, do so in the Getting Started section. It’s quick and painless :)

This post will be a very short introduction on Redis index backends and Picky, and how to configure your indexes to run on Redis.

I intended to do a massive writeup, but since all you do is change 6 characters Memory into 5 different characters Redis it just seemed like a massive overkill.

I admit though that many massive writeups have been done on even smaller changes, like “1.8” → “1.9” ;)

Ok, so what am I talking about?

tl;dr

  1. Redis can now be used in Picky as an index backend.
  2. In your config, do Index::Memory.new → Index::Redis.new and you’re set :)
  3. Memory and Redis indexes cannot (yet) be mixed and matched.
  4. In 1.5.0, Picky uses Redis database 15.

What is Redis?

Redis is – taken from the website – an “open source, advanced key-value store”. But this is not all. It also is a “data structure server”. Check it out on its very nicely done website.

“But we already have the massively fast in-memory backend. Why Redis?”, you scream, indignantly.

Why Redis?

Granted, in-memory indexes in Picky are really fast. But they have a few drawbacks:

  1. Relatively slow search engine startup, as the JSON index files need to be loaded into memory. This is especially noticeable if the index is around 12 GB.
  2. To restart Unicorn without a hitch you need double the space the in-memory index needs, since Unicorn starts up a second master in parallel to the old one.
  3. They need to be reloaded to be updated (see last blog post).

I haven’t had any problems with that, but I can see the problem. Hence, Redis.

How do you use Redis indexes?

Looking at the configuration that the scaffolding generates, you see that it uses an Index::Memory called books:

books_index = Index::Memory.new :books, Sources::CSV.new(:title, :author, file: 'app/library.csv')

If you’d like to use the Redis backend instead, you’ll have to change Memory into Redis.

books_index = Index::Redis.new :books, Sources::CSV.new(:title, :author, file: 'app/library.csv')

I know. Picky is hard on the typing hand ;)

Uh. That’s already it. Welcome Redis. Bye bye, Memory.

What you have to do now is re-index and start Picky:

$ rake index
... indexing output ...
$ rake start

Or, start Picky, re-index and search while it is indexing, to get some added fun value.

What is the impact of Redis indexes?

Compared to the in-memory index, what are the advantages and disadvantages?

Advantages:

Drawbacks:

Caveats / Next Versions

The Redis backend implementation in Picky is not yet customizable. This means that:

  1. It uses Redis database 15.
  2. Returned entry ids are always strings, even when they were integers going in. You’ll have to convert them back.
  3. Redis and Memory indexes cannot (yet) be mixed and matched. So this isn’t possible: Query::Full.new(redis_index, memory_index). Picky will notify you if you try to do so, so no worries.

I am focusing on these points in the upcoming 1.5.* versions.

Outlook

One of the next blog posts will look at the performance differences between the Redis backend and the memory backend.

I can already reveal that the memory backend will be faster. Surprise! ;) The question is: Is Redis so much slower as to be unbearable?

Music, pregnant with suspense, fills the room: Dun dun DUNNN.

Conclusion

So we’ve seen

  1. what Redis is.
  2. that Picky offers two different index backends: In-Memory and Redis.
  3. how you use/implement the Redis index backend in your search.

Hope you learnt something new :)

Next Searching with Picky: Rake Tasks

Share


Previous

Comments?