A better Rubygems search

ruby / picky / gems

Some time ago, Kaspar mentioned to me that it would be nice to have a gem dependency search, i.e. where you could search in which gems a gem is used.

I thought so too, so I wrote one :) (and added some more features in the process)

Take a look: http://gemsearch.heroku.com/

(Note, it might take Heroku some time to ramp up the server)

Current state

While the current search isn’t bad, it is missing the possibility of searching for an author, where a gem is used, or which version it has. Or any combination thereof, for that matter.

Building the search

I happened to have a fast & clever search engine lying around ;) so this is what I used.

How do you go about building or configuring a search engine?

1. Look at what your goals are.

My goals seemed simple enough.

Each gem should be findable under:

I leave out the description for now, as it requires quite a bit of thinking and tinkering.

With the goals defined…

2. Look at the data.

I downloaded the Marshal file, extracted the relevant data and restructured it into a readable CSV.

Two potential problems I noticed:

Those were problematic. What does one do? Try to find an optimal solution.

3. Marry the goals and the data.

I decided not to tackle the display issues of the second point, encodings, but just the indexing issues. What I do is use character substitution, to make “Nicolás” findable under “nicolas”. This I do by saving the name as “nicolas” in the index, and also perform this character substitution on each search. Try it with án áccent.

Deciding on what to with the gem names was a little harder. What is the problem?

The problem is manyfold. For one, searchers should not need to know whether a gem was spaced with an underscore or a hyphen. Actually, I thought it best they be able to find it using a space. So the picky-live gem should be findable by typing “picky live” (Try it).

However, if you then look for “sinatra”, the actual sinatra gem is not the first in the list. This is because I opted to go for an alphabetical ordering.

However, if I need the user to enter the full name (like “anthonymoralez-apn_on_rails”), they might not find it at all.

So, the way I did it now is have the user be able to use spaces when searching and trust people to depend on Picky’s combinatorial nature. For example, if you look for sinatra and know that one of the owners is called Tomayko, you’ll get to your answer directly: Search for ‘sinatra tomayko’.

Generally, the more you can help Picky, the more it will help you right back.

4. Have users try it and get feedback.

This is where you come in :)

Check it out, if you haven’t yet and tell me what you think @hanke! Do you have ideas for improvement? (If yes, tell me which so I can improve it)

How about we use this search on rubygems.org? :)

A few technical Picky specifics.

A few Picky specifics for insiders:

Yes, currently I break the web with hashtags – I’m rewriting it to use pushState.


Many thanks to Heroku for providing the infrastructure!


So we’ve seen

  1. that there’s a better way to search Rubygems
  2. where you can go to try it
  3. how you could go about creating a search

Hope you learnt something new :)

Next Searching with Picky: Live reloading indexes