#255 ✓resolved
Michael Ragalie

paginate very slow on large queries with :group

Reported by Michael Ragalie | December 10th, 2008 @ 03:37 AM

In finder.rb, wp_count() passes off to count() in order to figure out how many total results exist. When :group is specified, count() uses execute_grouped_calculation() in ActiveRecord::Calculations, which calls ActiveSupport::OrderedHash[]= . ActiveSupport::OrderedHash[]= calls assoc() on the its internal Array, which compares the value passed to assoc() to the first value of each array within its internal Array. As a result, for my application, which has a total of 2,895 matching rows, a total of 4,189,065 == comparisons are made in the course of determining the number of rows.

This is obviously very slow. Is there be a faster way to calculate the number of rows for queries with :group?

Comments and changes to this ticket

  • Mislav

    Mislav December 11th, 2008 @ 11:45 PM

    • State changed from “new” to “open”

    That's rather unfortunate.

    Does this help?

    
    Model.paginate :page => 1, :per_page => 10,
      :group => 'foo', :count => { :group => nil }
    

    That will cause the count to be performed like the :group option wasn't even there.

  • Michael Ragalie

    Michael Ragalie December 12th, 2008 @ 12:57 AM

    That worked like a charm. All I did was use

    
    Model.paginate :page => 1, :per_page => 10, :group => 'foo.id', :count => {:group => nil, :select => "DISTINCT foo.id"}
    

    and got the expected behavior.

    Thanks for your help!

  • Mislav

    Mislav December 12th, 2008 @ 02:22 AM

    • State changed from “open” to “resolved”

    No prob ;)

  • Sebastian

    Sebastian February 6th, 2009 @ 12:12 AM

    I*m sorry. But I think this is absolutly wrong, and will return wrong results.

    When you group something by a specific value, and then count without the group option as described above you'll get the count of all ungrouped results.

    I don't know about a better workaround to speed things up.

  • Mislav

    Mislav February 6th, 2009 @ 01:32 AM

    You may be right.

    But still, little can be done from will_paginate. Maybe you guys should open a Rails ticket? On the other hand, the internal implementation of OrderedHash was rewritten and now the problem may be gone (better check to be sure)

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

Everyone's favorite Ruby library for pagination of practically anything!

People watching this ticket

Pages