1. home •
  2. blog •
  3. picky •
  4. phd •
  5. phony •
  6. view models

Searching with Picky: Live reloading indexes

ruby / picky / gems

This is a post in the Picky series on its workings. If you haven’t tried it yet, do so in the Getting Started section. It’s quick and painless :)

This post is on reloading indexes by way of signals. So, first let’s talk a little about signals. Then, in the second half, I talk about reloading the memory index in Picky.

Warp 9?

En-gage.

Signals in Ruby

Signals are way of sending instructions to a running process. Here’s a list of signals.

In Ruby you handle these signals by giving the Signal#trap method a handler block:

What if I give it two? Let’s try it.

Signal.trap('USR1') { p "hello" }
Signal.trap('USR1') { p "world" }

# Print out the process PID such that it is easier
# to enter "kill -USR1 the_printed_process_pid"
#
puts Process.pid

# You have sixty seconds to defuse … err try this example.
#
sleep 60

Then, enter kill -USR1 <the printed process pid> and see what happens.

What happens is that the second block that prints “world” replaces the first one. So we see:

type here> ruby signals.rb 
77306
"world"

Ruby throws the old block away. What if I don’t want this?

Still calling the old trap handler

So, for example, in Unicorn, sending the USR1 signal handler reopens all logs. What if I want to do something else? If I just do

Signal.trap('USR1') { something_else }

the old handler will be gone.

So, my assumption was that Ruby gives me the old handler when calling

old_handler = Signal.trap('USR1')

Nope. Hurting the POLS a little here. It only gives me the old handler when installing a new one.

So what can you do? Use this “trick”:

old_handler = Signal.trap('USR1') {}
Signal.trap('USR1') { something_else; old_handler.call }

So I install a bogus handler to get the old handler, then throw the bogus handler away, right away, and call the old handler in the new handler.

Reloading the indexes

Currently, Picky does not support realtime indexes. It also runs with memory-only indexes (a Redis index backend is in the works). So, while the Picky server is running, it does not by itself pick up the new indexes, even if you generate new index files by running rake index.

Btw, did you ever try to call rake -T while in your Picky server project?

How can we reload the indexes?

Quite easy, actually. Reloading the memory indexes is done by calling

Indexes.reload

That’s it.

How do we get the Picky server process to call Indexes.reload?

Now talking about all that signal handling pays off! :)

… in a non-forking web server.

When running Picky in a non-forking web server, in e.g. thin, in the file app/application.rb we’d call

Signal.trap('USR1') { Indexes.reload }

and then in the Terminal, we run

type here> rake index
... (Picky indexes and writes new index files. Afterwards you tell the server to reload the indexes.)
type here> kill -USR1 your_picky_server_process_id

You should see some output that the server has reloaded the indexes.

… in a forking web server.

Unicorn, for example. Picky’s current web server of choice.

Since Unicorn already defines USR1, we use the trick we’ve talked about above to not replace the unicorn handler (if you need it):

old_handler = Signal.trap('USR1') {}
Signal.trap('USR1') { Indexes.reload; old_handler.call }

(Doesn’t have to be USR1, btw)

After indexing and sending the USR1 signal to the Unicorn master, we aren’t finished. Since the indexes have only been reloaded in the master, while the children are still happily using the old indexes.

Check out this very helpful page about signals in Unicorn. If preload_app is set to false in the unicorn.ru, you can just send a HUP signal to the master. It will then kill all children, and fork then. Finished.

When using Unicorn, you may of course also use the way Unicorn does it. See the instructions under “Procedure to replace a running unicorn executable”.

Good stuff! Although this procedure uses around double the memory the Picky server uses normally, while the index reloading uses around 1.5 times the size of the largest sub-index (in a nutshell, a lot less than the Unicorn replacement technique).

… periodically.

What about reloading the indexes periodically?

You could, of course, try to use a Thread, trying to reload the indexes every X time units and monkey around with it (tell me if you are successful :) ). I wouldn’t.

I recommend to externally trigger rake index, and then trigger reloads from outside using the mentioned signals.

Btw, a fun thing with signals you should not do

Back when Ruby was mostly foxes and bacons, I happened to type this:

begin
  p Process.pid
  looong_running_method
rescue Exception => e
  p "Oh deary me!"
  retry
end

Note: I did not actually type looong_running_method and "Oh deary me!", but you get the idea ;)

The idea was that if the long running method fails, it’d just retry running it.

Sounds good, right? Try running it, and stop it with Ctrl-C. The problem is the line rescue Exception => e.

Why? I soon found out that catching all Exceptions is not a good idea if you’d like stopping your program by way of Ctrl-C, since SignalException inherits from Exception:

p SignalException.ancestors # => [SignalException, Exception, Object, Kernel, BasicObject]

Ctrl-C sends a SIGINT, an INT signal to your process. Internally, a SignalException is raised, which is then caught by the rescue.

A kill -9 sends this process to Walhalla. The place where all programs go that have incurred a major learning experience on their writers.

Conclusion

So we’ve seen

  1. how signals work
  2. that reloading indexes in a running Picky server is easy
  3. how you use signals to reload the server
  4. how reloading works in different web servers
  5. that reloading the indexes isn’t without problems
  6. that you need to be careful when catching exceptions

Hope you learnt something new!

Next Searching with Picky: Redis

Share


Previous

Comments?