#257 ✓invalid
ReggieB

order by not working with joined tables

Reported by ReggieB | December 22nd, 2008 @ 02:24 PM

I'm currently using Rails 2.1 and was using will_paginate 2.2.2. I've upgraded to mislav-will_paginate 2.3.6, but the problem persists. MS SQL 2005 database.

The problem

I am listing information from a CRM system. The listing pulls information from three tables. To maximise performance I'm joining the tables. This works fine, until I try to order the results. will_paginate seems to grab the order text and then escapes out the period used to identify which table the sort element belongs to. So: @@@ORDER BY c.company_name


becomes
@@@ORDER BY c\.company_name

Which leads to a SQL error.

As I am using MS SQL I also use square brackets in my code and these are also escaped out: @@@ORDER BY [c].[company_name]



It is not the core SQL query where this occurs, but rather the encapsulating tmp1 and tmp2 entries. 

Here is my code

@@@      @opportunities = WillPaginate::Collection.create(page, per_page) do |pager|
        result = Opportunity.find(:all, 
               :select => "
[o].*,
 [c].[Comp_Name] AS company_name,
 [p].[Pers_Salutation] AS person_salutation,
 [p].[Pers_FirstName] AS person_first_name,
 [p].[Pers_LastName] AS person_last_name,
 [u].[User_FirstName] AS user_first_name,
 [u].[User_LastName] AS user_last_name
",
               :conditions => conditions.join(" AND "),
               :joins => "
AS [o]
LEFT JOIN [Company] AS [c]
  ON [o].[Oppo_PrimaryCompanyId] = [c].[Comp_CompanyId]
LEFT JOIN [Person] AS [p]
  ON [o].[Oppo_PrimaryPersonId] = [p].[Pers_PersonId]
LEFT JOIN [Users] AS [u]
  ON [o].[Oppo_AssignedUserId] = [u].[User_UserId]
",
               :order => 'c.comp_name',
               :limit => pager.per_page,
               :offset => pager.offset)
        # inject the result array into the paginated collection:
        pager.replace(result)

        unless pager.total_entries
          # the pager didn't manage to guess the total count, do it manually
          pager.total_entries = Opportunity.count(:all, :conditions => conditions.join(" AND "))
        end
      end

This works fine if you comment out the :order line, but fails as it is above. The SQL error this generates is here:

@@@DBI::DatabaseError: Execute

OLE error code:80040E14 in Microsoft OLE DB Provider for SQL Server
  Incorrect syntax near '\.'.
HRESULT error code:0x80020009
  Exception occurred.: SELECT * FROM (SELECT TOP 20 * FROM (SELECT TOP 20  

[o].*, [c].[Comp_Name] AS company_name, [p].[Pers_Salutation] AS person_salutation, [p].[Pers_FirstName] AS person_first_name, [p].[Pers_LastName] AS person_last_name, [u].[User_FirstName] AS user_first_name, [u].[User_LastName] AS user_last_name FROM Opportunity
AS [o] LEFT JOIN [Company] AS [c] ON [o].[Oppo_PrimaryCompanyId] = [c].[Comp_CompanyId] LEFT JOIN [Person] AS [p] ON [o].[Oppo_PrimaryPersonId] = [p].[Pers_PersonId] LEFT JOIN [Users] AS [u] ON [o].[Oppo_AssignedUserId] = [u].[User_UserId] WHERE (Oppo_CreatedDate < '2008-12-01 00:00:00' AND Oppo_CreatedDate > '2008-11-01 00:00:00') ORDER BY c.comp_name) AS tmp1 ORDER BY c.[comp_name] DESC) AS tmp2 ORDER BY c.[comp_name]



As you can see, the problem is shown right at the end of the error message

Comments and changes to this ticket

  • ReggieB

    ReggieB December 22nd, 2008 @ 02:29 PM

    Sorry - didn't get the format syntax right.

    My problem is that:

    
    ORDER BY c.comp_name
    

    becomes

    
    ORDER BY c\.comp_name
    

    My code is

    
          @opportunities = WillPaginate::Collection.create(page, per_page) do |pager|
            result = Opportunity.find(:all, 
                   :select => "
    [o].*,
     [c].[Comp_Name] AS company_name,
     [p].[Pers_Salutation] AS person_salutation,
     [p].[Pers_FirstName] AS person_first_name,
     [p].[Pers_LastName] AS person_last_name,
     [u].[User_FirstName] AS user_first_name,
     [u].[User_LastName] AS user_last_name
    ",
                   :conditions => conditions.join(" AND "),
                   :joins => "
    AS [o]
    LEFT JOIN [Company] AS [c]
      ON [o].[Oppo_PrimaryCompanyId] = [c].[Comp_CompanyId]
    LEFT JOIN [Person] AS [p]
      ON [o].[Oppo_PrimaryPersonId] = [p].[Pers_PersonId]
    LEFT JOIN [Users] AS [u]
      ON [o].[Oppo_AssignedUserId] = [u].[User_UserId]
    ",
                   :order => 'c.comp_name',
                   :limit => pager.per_page,
                   :offset => pager.offset)
            # inject the result array into the paginated collection:
            pager.replace(result)
    
            unless pager.total_entries
              # the pager didn't manage to guess the total count, do it manually
              pager.total_entries = Opportunity.count(:all, :conditions => conditions.join(" AND "))
            end
          end
    

    And the error message is:

    
    DBI::DatabaseError: Execute
        OLE error code:80040E14 in Microsoft OLE DB Provider for SQL Server
          Incorrect syntax near '\.'.
        HRESULT error code:0x80020009
          Exception occurred.: SELECT * FROM (SELECT TOP 20 * FROM (SELECT TOP 20  
    [o].*,
     [c].[Comp_Name] AS company_name,
     [p].[Pers_Salutation] AS person_salutation,
     [p].[Pers_FirstName] AS person_first_name,
     [p].[Pers_LastName] AS person_last_name,
     [u].[User_FirstName] AS user_first_name,
     [u].[User_LastName] AS user_last_name
     FROM Opportunity  
    AS [o]
    LEFT JOIN [Company] AS [c]
      ON [o].[Oppo_PrimaryCompanyId] = [c].[Comp_CompanyId]
    LEFT JOIN [Person] AS [p]
      ON [o].[Oppo_PrimaryPersonId] = [p].[Pers_PersonId]
    LEFT JOIN [Users] AS [u]
      ON [o].[Oppo_AssignedUserId] = [u].[User_UserId]
     WHERE (Oppo_CreatedDate < '2008-12-01 00:00:00' AND Oppo_CreatedDate > '2008-11-01 00:00:00')  ORDER BY c.comp_name) AS tmp1 ORDER BY c\.\[comp_name\] DESC) AS tmp2 ORDER BY c\.\[comp_name\]
    
  • ReggieB

    ReggieB December 22nd, 2008 @ 02:33 PM

    I've also tried using paginate_by_sql:

    
          sql = <<EOF
    SELECT [o].*,
      [c].[Comp_Name] AS company_name,
      [p].[Pers_Salutation] AS person_salutation,
      [p].[Pers_FirstName] AS person_first_name,
      [p].[Pers_LastName] AS person_last_name,
      [u].[User_FirstName] AS user_first_name,
      [u].[User_LastName] AS user_last_name
    FROM [Opportunity] AS [o]
      LEFT JOIN [Company] AS [c]
        ON [o].[Oppo_PrimaryCompanyId] = [c].[Comp_CompanyId]
      LEFT JOIN [Person] AS [p]
        ON [o].[Oppo_PrimaryPersonId] = [p].[Pers_PersonId]
      LEFT JOIN [Users] AS [u]
        ON [o].[Oppo_AssignedUserId] = [u].[User_UserId]
    WHERE #{conditions.join(" AND ")}
    ORDER BY [c].[comp_name]
    EOF
    
          @opportunities = Opportunity.paginate_by_sql(sql, :page => params[:page], :per_page => 3)
    

    But this results in another error:

    
    DBI::DatabaseError: Execute
        OLE error code:80040E14 in Microsoft OLE DB Provider for SQL Server
          The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP or FOR XML is also specified.
        HRESULT error code:0x80020009
          Exception occurred.: SELECT COUNT(*) FROM (SELECT [o].*,
      [c].[Comp_Name] AS company_name,
      [p].[Pers_Salutation] AS person_salutation,
      [p].[Pers_FirstName] AS person_first_name,
      [p].[Pers_LastName] AS person_last_name,
      [u].[User_FirstName] AS user_first_name,
      [u].[User_LastName] AS user_last_name
    FROM [Opportunity] AS [o]
      LEFT JOIN [Company] AS [c]
        ON [o].[Oppo_PrimaryCompanyId] = [c].[Comp_CompanyId]
      LEFT JOIN [Person] AS [p]
        ON [o].[Oppo_PrimaryPersonId] = [p].[Pers_PersonId]
      LEFT JOIN [Users] AS [u]
        ON [o].[Oppo_AssignedUserId] = [u].[User_UserId]
    WHERE Oppo_CreatedDate < '2008-12-01 00:00:00' AND Oppo_CreatedDate > '2008-11-01 00:00:00'
    ORDER BY [c].[comp_name]
    ) AS count_table
    
  • ReggieB

    ReggieB December 24th, 2008 @ 11:46 AM

    I've investigated this further and it is not a will_paginate bug. It's an active record bug. The same thing happens with a find :

    
    class Opportunity
      def self.find_test
        find(:all,
                   :select => "
    [o].*,
     [c].[Comp_Name] AS company_name,
     [p].[Pers_Salutation] AS person_salutation,
     [p].[Pers_FirstName] AS person_first_name,
     [p].[Pers_LastName] AS person_last_name,
     [u].[User_FirstName] AS user_first_name,
     [u].[User_LastName] AS user_last_name
    ",
                   :joins => "
    AS [o]
    LEFT JOIN [Company] AS [c]
      ON [o].[Oppo_PrimaryCompanyId] = [c].[Comp_CompanyId]
    LEFT JOIN [Person] AS [p]
      ON [o].[Oppo_PrimaryPersonId] = [p].[Pers_PersonId]
    LEFT JOIN [Users] AS [u]
      ON [o].[Oppo_AssignedUserId] = [u].[User_UserId]
    ",
                   :order => 'c.comp_name',
                   :limit => 30,
                   :offset => 30)
      end
    end
    
    Opportunity.find_test
    

    Gives the same error. Can you close this ticket please, and I'll see if I can get the problem sorted elsewhere.

  • ReggieB

    ReggieB December 24th, 2008 @ 12:23 PM

    Working on the problem further, its the square brackets causing the problem. They are commonly used in MS SQL code, but ActiveRecord seems to behave differently if it finds them in and offset or limit option. Take out the square brackets and I started getting somewhere. This version of the above test works:

    
    class Opportunity << Base
      def self.find_test
        find(:all,
                   :select => "
    o.*,
     c.Comp_Name AS company_name,
     p.Pers_Salutation AS person_salutation,
     p.Pers_FirstName AS person_first_name,
     p.Pers_LastName AS person_last_name,
     u.User_FirstName AS user_first_name,
     u.User_LastName AS user_last_name
    ",
                   :joins => "
    AS o
    LEFT JOIN Company AS c
      ON o.Oppo_PrimaryCompanyId = c.Comp_CompanyId
    LEFT JOIN Person AS p
      ON o.Oppo_PrimaryPersonId = p.Pers_PersonId
    LEFT JOIN Users AS u
      ON o.Oppo_AssignedUserId = u.User_UserId
    ",
                   :order => 'company_name',
                   :limit => 30,
                   :offset => 30)
      end 
    end
    

    Note: I also has to tweak the :order statement as the limit and offset are used in a SELECT statement within another SELECT statement, and the preceding table name gets stripped off. Therefore you need to use a column name that is consistent in each of the SELECT statements.

    So definitely close this ticket now please.

  • Mislav

    Mislav January 8th, 2009 @ 03:33 PM

    • State changed from “new” to “invalid”

    Glad you solved your problem :) Closing

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

Everyone's favorite Ruby library for pagination of practically anything!

People watching this ticket

Tags

Pages