1. home •
  2. blog •
  3. picky •
  4. phd •
  5. phony •
  6. view models

Picky: Designing an ORM Integration 1

ruby / picky / orm

This is a post in the Picky series on its workings.

In this post, I want you to peek over my shoulder as I go through some of my thoughts regarding Picky ORM integration.

tl;dr

Picky needs to be more accessible. How can we do this? We provide a simple API to be used in an ActiveModel which provides indexing and searching.

The result: A possible Picky API.

Intro

Now Picky is cool, sports quite a few features, and is written in Ruby so you can easily extend it. I also think it fills a feature gap that “Generic Search Engine X” and “Hyperfast Russian Text Looker-Througher” (I write this lovingly) do not address. Etc etc, yadda yadda.

So what is the problem I’m addressing?

El problemo: Picky is not as accessible as other search engines.

What do I mean by accessible?

Accessibility?

One example for accessibility is Karel Minařik’s Tire frontend for ElasticSearch.

He did a great job in making it accessible through this script. The gist installs Rails & ElasticSearch in one fell swoop. Let’s call this kind of accessibility the “Boom” factor.

Remember Steve Jobs? “Boom” this and “Boom” that. Magique!

Now, sure, Picky does have a Getting Started that does exactly that in 5 minutes, including an in-site manual. And to be fair, it also generates the views including a full search interface.

But still. The question remains: If I have an existing Rails app, how does this work? Can’t I just add Picky to my model and have a search?

class Person
  pickify
end

and then

Person.indexes(:mi5, :cia, :kgb).offset(30).search 'bond, james'

Not yet. I do have my reservations about this approach (see last post), but I see its appeal: People have a nice starting point to get into the finer details of searching (which is exactly what I want people to do – build better searches!).

In short: Picky needs to up its Boom Factor!

The Boom Factor

Between us and going to Boom Factor 11 stands a lot of code.

But before the code, a lot of thinking of how the code is supposed to look.

And before we can even begin to think, we should know what we want, and what information we need in the API.

What do we want?

A few things:

That is what we want. What information do we need?

What do we need?

We need different things for searching and for indexing.

For searching, we need to be able to tell Picky:

Quite a bit of information!

For indexing, we need to be able to tell Picky:

Not bad either…

Let’s try a few variations!

API Designs

All this goes into a special gem called picky-activemodel.

Let’s say we start with the obvious, telling the class that it can be pickified.

class Person
  include Picky
end

This is snappy and short. Maybe too short? Let’s take a look at indexing.

Indexing

Since Picky does not yet offer incremental indexing (most people don’t need it even if they think so), we’d have to provide an explicit index! method of sorts.

Person.index!

But how would we define the indexing? In Picky you can define index text preparation for all indexes, for each index separately, even for each category separately.

Let’s see. (Using just split_on in the example)

class Person
  include Picky

  index.split_on /[\s]/

  index do
    split_on /\W/

    category :first_name do
      split_on /\s/
      partial :substring, 1
    end
    category :name do
      from :last_name
    end
  end

  index :advertisements do
    split_on /\s/
    category :last_name do
      qualifiers [:ad_name, :an]
    end
  end
end

Person.index!

That means that generally, index text is split on /\s/. Then, make an index with the implicitly pluralized name "persons", which splits on /\W/. It indexes two categories, the first name which is specially split, and indexed for partial searching.

category :first_name do
  split_on /\s/
  partial :substring, 1
end

There’s an interesting question there: Should it be

partial :substring, 1

using a weak symbol/number parameter based config or a more powerful

partial Picky::Partial::Substring.new(1)

with the problem that we now need the Substring class defined not only in Picky, but also in the picky-activemodel gem.

Not too easy indeed. I’m not a big fan of String definitions. It’s just so incredibly weak.

Anyway, back to the example.

category :name do
  from :last_name
end

What does this mean? It means that the data for category :name is taken from the attribute :last_name.

Further down, we have another index definition, :advertisements, which is explicitly named.

index :advertisements do

Last but not least, we index explicitly using 

Person.index!

Searching

Searching is quite interesting.

On the one hand, we could have a fluent interface for which indexes to search, and with what parameters. Let’s look at it:

Person.search.indexes(:advertisements).offset(30).ids(20).with("Bond, James")

to search with text “Bond, James” in index :advertisements, getting 20 result ids starting after the first 30.

The short form

Person.search("Bond, James")

would be much more crisp, searching in the default, unnamed index with offset 0 and 20 result ids.

This would not return an array of ids, but the Picky result hash, which contains weights, categories, totals, search duration.

An alternative would be

Person.search do
  indexes :advertisements
  offset  30
  ids     20
  with    "Bond, James"
end

or any combination thereof. I’m inclined to allow both, or a combination of all.

This was the easy part. But where do I tell Picky how to prepare the search text? (How to split and so on?)

One idea is to put this in the model as well.

class Person
  include Picky

  searching do
    split_on /\s/
  end

end

Sound good, but is the way we prepare the search text really model-specific?

Not really. Let’s try the search request:

Person.search("Bond, James") do
  split_on /\s/
end

Not too sexy either. Perhaps also chained?

Person.search.split_on(/\s/).with("Bond, James")

Could work but is too wordy.

How about we use a simple method?

class Person
  def self.simple_splitting_search
    @simple_splitting_search ||= search.split_on(/\s/).removes_characters(/[\&\-]/)
  end
end

Person.simple_splitting_search.with("Bond, James")

Now this would be Ruby-esque! Methods and stuff. Who needs scopes? :)

Also, the truly dynamic part would be exposed, the semi-fixed part would be summarized in the method name. Also one could decide to memoize it, as above.

I think we can work with something like that.

But the case where we just index a Person is the easy case. What if we also want to index its addresses, which are saved as a separate model, together in a single index?

Indexing relations

The best way in my humble opinion would be to define a very specific model, just for searching – to avoid cluttering the normal model, obey the SRP.

But probably this is not what many people would want.

So let’s give it a go with the abovementioned addresses relation:

class Person
  include Picky

  index do
    category :first_name do
      # ...
    end
    category :street do
      from { addresses.map(&:street).join(" ") }
    end
  end

end

Yep. I wouldn’t conjure up a complicated DSL, but use the trusty from method, and then just give it a block which is evaluated in each model instance, just taking the data the block returns.

Possible problems

The search and index methods could already have been installed by other libraries. So what could we do in this case?

The Picky way of doing things would be to play nice:

class Person
  include Picky

  picky.index do
    category :first_name do
      split_on /\s/
    end
  end

end

So if the index, index! or search method was already installed, it would just install a – presumably yet uninstalled method named picky that acts as a proxy.

Also in searching,

Person.picky.search("Bond, James")

reads quite ok.

One idea might be to call it picky_search, but not too partial to that.

So yeah, hope you enjoyed looking over my shoulder. There’s a lot to do still, but this looks like a hopeful start. I’d give it a Boom Factor of 10 :)

If you find any problems or have ideas, let me know in the comments!

Conclusion

So we’ve seen

  1. how you might go about designing an API.

Hope you learnt something new!

Next James

Share


Previous

Comments?