<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>ruby on rails postgres text search on random thoughts</title><link>https://awesomeprogrammer.com/categories/ruby-on-rails-postgres-text-search/</link><description>Recent content in ruby on rails postgres text search on random thoughts</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><lastBuildDate>Tue, 11 Sep 2012 00:00:00 +0000</lastBuildDate><atom:link href="https://awesomeprogrammer.com/categories/ruby-on-rails-postgres-text-search/index.xml" rel="self" type="application/rss+xml"/><item><title> Efficient use of PostgreSQL full text search with highlighting using pg_search</title><link>https://awesomeprogrammer.com/blog/2012/09/11/efficient-use-of-postgresql-full-text-search-with-highlighting-using-pg-search/</link><pubDate>Tue, 11 Sep 2012 00:00:00 +0000</pubDate><guid>https://awesomeprogrammer.com/blog/2012/09/11/efficient-use-of-postgresql-full-text-search-with-highlighting-using-pg-search/</guid><description><![CDATA[<p>For some time I was struggling with efficient of <a href="http://www.postgresql.org/docs/9.1/static/textsearch-controls.html">ts_highligh</a> function of postgres using Rails and <a href="https://github.com/Casecommons/pg_search">pg_search</a>. I had bunch of really long articles and I wanted to highlight search results in the content as well in the title with the power of tsquery and polish dictionary that we feed our database.</p>
<p>In the end solutions was quite simple really, maybe not the most elegant one but does the job just right and in the end your will end up still with ActiveRecord::Relation, so you can easily paginate your results with will_paginate or kaminari gem.</p>
<p>So, let&rsquo;s break it down.</p>
<p>First - I was using custom polish <a href="http://www.postgresql.org/docs/current/static/textsearch-dictionaries.html">dictionary</a>, and I already had a trigger in database that recreated my tsvector column every update/insert. So in that case you can declare PgSearch scope like that:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-ruby" data-lang="ruby"><span style="display:flex;"><span>pg_search_scope <span style="color:#e6db74">:search_full_text</span>,
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">:against</span> <span style="color:#f92672">=&gt;</span> <span style="color:#e6db74">:body</span>,
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">:using</span>  <span style="color:#f92672">=&gt;</span> { <span style="color:#e6db74">:tsearch</span> <span style="color:#f92672">=&gt;</span>
</span></span><span style="display:flex;"><span>              {
</span></span><span style="display:flex;"><span>                <span style="color:#e6db74">:dictionary</span>       <span style="color:#f92672">=&gt;</span> <span style="color:#e6db74">&#34;polish&#34;</span>,
</span></span><span style="display:flex;"><span>                <span style="color:#e6db74">:tsvector_column</span>  <span style="color:#f92672">=&gt;</span> <span style="color:#e6db74">&#39;fts&#39;</span>,
</span></span><span style="display:flex;"><span>                <span style="color:#e6db74">:prefix</span>           <span style="color:#f92672">=&gt;</span> <span style="color:#66d9ef">true</span>
</span></span><span style="display:flex;"><span>              }
</span></span><span style="display:flex;"><span>            }
</span></span></code></pre></div><p>And basically that works great, except when your try to select ts_highligh with can be painfully slow. So what I did there:</p>
<ul>
<li>
<p>first I monkey patched pg_search for my application - I didn&rsquo;t wanted to select all the columns, just few ones. So in <code>pg_search/lib/pg_search/scope_options.rb</code> I customized <code>apply</code> method</p>
</li>
<li>
<p>then I just used select from select in a simple manner, take a look at example below</p>
</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-ruby" data-lang="ruby"><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">self</span><span style="color:#f92672">.</span><span style="color:#a6e22e">search</span>(search_query, page)
</span></span><span style="display:flex;"><span>  body <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;ts_headline(&#39;polish&#39;, body, plainto_tsquery(&#39;polish&#39;,&#39;</span><span style="color:#e6db74">#{</span>search_query<span style="color:#e6db74">}</span><span style="color:#e6db74">&#39;), &#39;startsel=&#39;&#39;&lt;em class=</span><span style="color:#ae81ff">\&#34;</span><span style="color:#e6db74">headline</span><span style="color:#ae81ff">\&#34;</span><span style="color:#e6db74">&gt;&#39;&#39; stopsel=&lt;/em&gt;, MaxFragments=2, maxwords=15, minwords=5&#39;) as body&#34;</span>
</span></span><span style="display:flex;"><span>  self<span style="color:#f92672">.</span>select<span style="color:#f92672">[</span><span style="color:#e6db74">:id</span>, body<span style="color:#f92672">].</span>from(<span style="color:#e6db74">&#34;(</span><span style="color:#e6db74">#{</span>self<span style="color:#f92672">.</span>search_full_text(search_query)<span style="color:#f92672">.</span>paginate(<span style="color:#e6db74">:page</span> <span style="color:#f92672">=&gt;</span> page)<span style="color:#f92672">.</span>to_sql<span style="color:#e6db74">}</span><span style="color:#e6db74">) as result&#34;</span>)
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">end</span>
</span></span></code></pre></div><p>As you see I&rsquo;m selecting id column and my highlighted body from paginated results (using will_paginate here). In the view you can display pagination over <code>search_full_text(search_query).paginate(:page =&gt; page)</code>. Refactor it as you please. Right now it seems like the best solution I could think of - if you have better idea let me know!</p>]]></description><content:encoded><![CDATA[<p>For some time I was struggling with efficient of <a href="http://www.postgresql.org/docs/9.1/static/textsearch-controls.html">ts_highligh</a> function of postgres using Rails and <a href="https://github.com/Casecommons/pg_search">pg_search</a>. I had bunch of really long articles and I wanted to highlight search results in the content as well in the title with the power of tsquery and polish dictionary that we feed our database.</p>
<p>In the end solutions was quite simple really, maybe not the most elegant one but does the job just right and in the end your will end up still with ActiveRecord::Relation, so you can easily paginate your results with will_paginate or kaminari gem.</p>
<p>So, let&rsquo;s break it down.</p>
<p>First - I was using custom polish <a href="http://www.postgresql.org/docs/current/static/textsearch-dictionaries.html">dictionary</a>, and I already had a trigger in database that recreated my tsvector column every update/insert. So in that case you can declare PgSearch scope like that:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-ruby" data-lang="ruby"><span style="display:flex;"><span>pg_search_scope <span style="color:#e6db74">:search_full_text</span>,
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">:against</span> <span style="color:#f92672">=&gt;</span> <span style="color:#e6db74">:body</span>,
</span></span><span style="display:flex;"><span> <span style="color:#e6db74">:using</span>  <span style="color:#f92672">=&gt;</span> { <span style="color:#e6db74">:tsearch</span> <span style="color:#f92672">=&gt;</span>
</span></span><span style="display:flex;"><span>              {
</span></span><span style="display:flex;"><span>                <span style="color:#e6db74">:dictionary</span>       <span style="color:#f92672">=&gt;</span> <span style="color:#e6db74">&#34;polish&#34;</span>,
</span></span><span style="display:flex;"><span>                <span style="color:#e6db74">:tsvector_column</span>  <span style="color:#f92672">=&gt;</span> <span style="color:#e6db74">&#39;fts&#39;</span>,
</span></span><span style="display:flex;"><span>                <span style="color:#e6db74">:prefix</span>           <span style="color:#f92672">=&gt;</span> <span style="color:#66d9ef">true</span>
</span></span><span style="display:flex;"><span>              }
</span></span><span style="display:flex;"><span>            }
</span></span></code></pre></div><p>And basically that works great, except when your try to select ts_highligh with can be painfully slow. So what I did there:</p>
<ul>
<li>
<p>first I monkey patched pg_search for my application - I didn&rsquo;t wanted to select all the columns, just few ones. So in <code>pg_search/lib/pg_search/scope_options.rb</code> I customized <code>apply</code> method</p>
</li>
<li>
<p>then I just used select from select in a simple manner, take a look at example below</p>
</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-ruby" data-lang="ruby"><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">self</span><span style="color:#f92672">.</span><span style="color:#a6e22e">search</span>(search_query, page)
</span></span><span style="display:flex;"><span>  body <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;ts_headline(&#39;polish&#39;, body, plainto_tsquery(&#39;polish&#39;,&#39;</span><span style="color:#e6db74">#{</span>search_query<span style="color:#e6db74">}</span><span style="color:#e6db74">&#39;), &#39;startsel=&#39;&#39;&lt;em class=</span><span style="color:#ae81ff">\&#34;</span><span style="color:#e6db74">headline</span><span style="color:#ae81ff">\&#34;</span><span style="color:#e6db74">&gt;&#39;&#39; stopsel=&lt;/em&gt;, MaxFragments=2, maxwords=15, minwords=5&#39;) as body&#34;</span>
</span></span><span style="display:flex;"><span>  self<span style="color:#f92672">.</span>select<span style="color:#f92672">[</span><span style="color:#e6db74">:id</span>, body<span style="color:#f92672">].</span>from(<span style="color:#e6db74">&#34;(</span><span style="color:#e6db74">#{</span>self<span style="color:#f92672">.</span>search_full_text(search_query)<span style="color:#f92672">.</span>paginate(<span style="color:#e6db74">:page</span> <span style="color:#f92672">=&gt;</span> page)<span style="color:#f92672">.</span>to_sql<span style="color:#e6db74">}</span><span style="color:#e6db74">) as result&#34;</span>)
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">end</span>
</span></span></code></pre></div><p>As you see I&rsquo;m selecting id column and my highlighted body from paginated results (using will_paginate here). In the view you can display pagination over <code>search_full_text(search_query).paginate(:page =&gt; page)</code>. Refactor it as you please. Right now it seems like the best solution I could think of - if you have better idea let me know!</p>]]></content:encoded></item></channel></rss>