I'm aware of numerous issues raised on the EF7 Github repo, and these still appear to be present with the latest RC2 nightly builds.
The issue appears to be when using OrderBy and Take() when you're trying to Include relations within your query. I believe, according to previous issues raised on Github that the SQL generated within the join is incorrect and does not take into account the OrderBy. After reading replies, people have suggested using Skip(0).Take(x) as a workaround, but unfortunately in my scenario, this didn't work.
In my specific scenario, I'm querying a model based on an ancestor PK. So, in order to work around this, instead of passing the PK straight into the query, I've added the ancestor model to a List<T> and used the following code in the query instead:
Before (not working when Including nested relations): p => p.examplePK == id
After (appears to be working):
p => myList.Select(c => c.myId).Contains(p.examplePK)
I'm not entirely sure why the 2nd example works and I would be grateful of any info that can be given - could there potentially be some client side evaluation going on here instead? From what I gather, the 2nd example will be performing a SQL IN statement, will it not?
Thanks in advance!
I want to display the SQL used in a count. However, Model.count.to_sql will not work because count returns a FixNum that doesn't have a to_sql method. I think the simplest solution is to do this:
Model.where(nil).to_sql.sub(/SELECT.*FROM/, "SELECT COUNT(*) FROM")
This creates the same SQL as is used in Model.count, but is it going to cause a problem further down the line? For example, if I add a complicated where clause and some joins.
Is there a better way of doing this?
You can try
Model.select("count(*) as model_count").to_sql
You may want to dip into Arel:
Model.select(Arel.star.count).to_sql
ASIDE:
I find I often want to find sub counts, so I embed the count(*) into another query:
child_counts = ChildModel.select(Arel.star.count)
.where(Model.arel_attribute(:id).eq(
ChildModel.arel_attribute(:model_id)))
Model.select(Arel.star).select(child_counts.as("child_count"))
.order(:id).limit(10).to_sql
which then gives you all the child counts for each of the models:
SELECT *,
(
SELECT COUNT(*)
FROM "child_models"
WHERE "models"."id" = "child_models"."model_id"
) child_count
FROM "models"
ORDER BY "models"."id" ASC
LIMIT 10
Best of luck
UPDATE:
Not sure if you are trying to solve this in a generic way or not. Also not sure what kind of scopes you are using on your Model.
We do have a method that automatically calls a count for a query that is put into the ui layer. I found using count(:all) is more stable than the simple count, but sounds like that does not overlap your use case. Maybe you can improve your solution using the except clause that we use:
scope.except(:select, :includes, :references, :offset, :limit, :order)
.count(:all)
The where clause and the joins necessary for the where clause work just fine for us. We tend to want to keep the joins and where clause since that needs to be part of the count. While you definitely want to remove the includes (which should be removed by rails automatically in my opinion), but the references (much trickier especially in the case where it references a has_many and requires a distinct) that starts to throw a wrench in there. If you need to use references, you may be able to convert these over to a left_join.
You may want to double check the parameters that these "join" methods take. Some of them take table names and others take relation names. Later rails version have gotten better and take relation names - be sure you are looking at the docs for the right version of rails.
Also, in our case, we spend more time trying to get sub selects with more complicated relationships, we have to do some munging. Looks like we are not dealing with where clauses as much.
ref2
I have tested and done quite some research online, but still no luck. Did anyone ever encounter this problem ?
Say, I have a doctrine query set up like:
$q = Doctrine_Query::create()
->update('PckFolder')
->set('id_path', "CONCAT(?, RIGHT(id_path, LENGTH(id_path)-?))", array($newPath, $lenOld))
->where("id_path like '$oldPath%'");
// and I print the query out
$qstr = $q->getSqlQuery(array($newPath, $lenOld));
Instead of giving me:
UPDATE pck_folder SET id_path = CONCAT(?, RIGHT(id_path, LENGTH(id_path)-?)) WHERE (id_path like '1/2//%')
Doctrine gave me:
UPDATE pck_folder SET id_path = CONCAT(?, RIGHT(id_path, LENGTH(id_path-?))) WHERE (id_path like '1/2//%')
please note this part RIGHT(id_path, LENGTH(id_path)-?)
(Note: I'm assuming you're using Doctrine 1.2. I haven't used Doctrine 2.0 yet.)
I had not encountered that specific bug before, but I have found numerous problems with the implementation of update() in Doctrine_Query. Essentially anything but the most very straightforward update queries will cause the parser to generate wrong or invalid queries. For example, it can't handle sub-selects within an update.
Try writing a Raw SQL query, or else use a less efficient but fully-functional workaround: Select the records you want to update using Doctrine_Query, then iterate over them and set the field in PHP, then call save() on each one.
By the way, there's a big GOTCHA inherent with use of UPDATE queries and Doctrine that sort of forces you to use that workaround in many cases anyway. That is, if you or your plugins have made use of the nifty Doctrine hook methods within your models but you execute a SQL-level update that affects those records, the hooks will get silently circumvented. Depending on your application, that may wreck your business logic processing.
I'm trying to reproduce the HqlQuery style 'select new ObjectToProjectOut' functionality. i.e. take a list of columns returned from a query and return as a list of ObjectToProjectOut types that are instantiated using a Constructor with as many parameters as the columns in the query.
This is in effect what 'select new ObjectToProjectOut' achieves in Hql.... but clearly that's not available in SqlQuery. I think I need to set a result transform and use either PassThroughResultTransformer, DistinctRootEntityResultTransformer etc to get it to work.
Anyone know what I should use ?
ok.... after looking at the NHibernate code it seems that I was looking for AliasToBeanConstructorResultTransformer.... of course!
However I may have found an nHibernate bug. If you have the same column name returned twice from two different tables (market.name and account.name, say) then by the time nHibernate returns the array from the db to the transformer, the first occurance of 'Name' will be used for both. Nasty.
Work around is to uniquely alias. With Hql, the sql generated is heavily aliased, so this is only a bug with SqlQuery.
Grrrr. Today must be my day, also found another nHibernate bug/issue I've posted to StackOverflow for comment.
You could use the AddEntity method to fill entities from a SQL query.
Here are two examples from the NHibernate docs:
sess.CreateSQLQuery("SELECT * FROM CATS")
.AddEntity(typeof(Cat));
sess.CreateSQLQuery("SELECT ID, NAME, BIRTHDATE FROM CATS")
.AddEntity(typeof(Cat));
I need to get a list of all documents in a site collection, which I believe I can do with either the alldocs table or the alluserdata table (MOSS 2007 SP1) but do not see how I can get the author information for the document. I do not need the contents of the document (e.g. AllDocStreams content)
Something like this:
SELECT tp_DirName, tp_LeafName, tp_Version, tp_Modified, tp_Created
FROM AllUserData
WHERE (tp_ContentType = 'Document')
AND (tp_LeafName NOT LIKE '%.css')
AND (tp_LeafName NOT LIKE '%.jpg')
AND (tp_LeafName NOT LIKE '%.png')
AND (tp_LeafName NOT LIKE '%.wmf')
AND (tp_LeafName NOT LIKE '%.gif')
AND (tp_DirName NOT LIKE '%Template%')
AND (tp_IsCurrentVersion = 1)
AND (tp_LeafName NOT LIKE '%.xsl')
ORDER BY tp_SiteId, tp_ListId, tp_DirName, tp_LeafName, tp_IsCurrentVersion DESC
Is there a better way to go about this?
People that claim that you cannot query SharePoint databases because it is not supported are wrong. From reading the documentation, it is fine to query the database as long as you use the 'With(NoLock)' clause. It is clearly not supported to update, delete, or insert records.
The below query is supported:
Select *
From your_content_database.dbo.AllDocs With (NoLock)
I will post a query that provides the desired result in a few minutes.
Why not use the sharepoint object model rather then using the raw database approach? I know that the object model approach does have a performance penalty compared to the database, but MS could change the db schema with the next path. On the other hand the likelyhood of MS breaking their own object model is far less, and as far as I know the recommended way is to use either the object model or the web services.
Don't ever query the SharePoint database directly. This is completely unsupported and can get you into trouble moving forward (for instance, if a service-pack or hotfix modifies schema, then you app is broken).
The below would return the top 100 largest documents that were added in the last 24 hours to the content database.
Select Top 100
W.FullUrl,
W.Title,
L.tp_Title as ListTitle,
A.tp_DirName,
A.tp_LeafName,
A.tp_id ,
DS.Content ,
DS.Size,
D.DocLibRowID,
D.TimeCreated,
D.Size,
D.MetaInfoTimeLastModified,
D.ExtensionForFile
From your_content_database.dbo.AllLists L With (NoLock)
join your_content_database.dbo.AllUserData A With (NoLock)
On L.tp_ID=tp_ListId
join your_content_database.dbo.AllDocs D With (NoLock)
On A.tp_ListID=D.ListID
And A.tp_SiteID=D.SiteID
And A.tp_DirName=D.DirName
And A.tp_LeafName=D.LeafName
join your_content_database.dbo.AllDocStreams DS With (NoLock)
On DS.SiteID=A.tp_SiteID
And DS.ParentID=D.ParentID
And DS.ID=D.ID
join your_content_database.dbo.Webs W With (NoLock)
On W.ID=D.WebID
And W.ID=L.Tp_WebID
And W.SiteID=A.tp_SiteID
Where DS.DeleteTransactionID=0x
And D.DeleteTransactionID=0x
And D.IsCurrentVersion=1
And A.tp_DeleteTransactionID=0x
And A.tp_IsCurrentVersion=1
And D.HasStream=1
And L.tp_DeleteTransactionId=0x
And ExtensionForFile not in('webpart','dwp','aspx','xsn','master','rules','xoml')
And D.MetaInfoTimeLastModified>DateAdd(d,-1,GetDate())
Order by DS.Size desc
I recommend that you have a look at the Camelot .NET Connector which allows you to query SharePoint 2007/2010 using standard SQL queries. Its a ADO.NET driver that can also be exposed through a simple WCF service and by that available through any programming language. Lets say one would like to select from "shared documents", you would write something like:
select * from `shared documents`
or with certain columns:
select id, title, filetype, filesize, created, createdby from `shared documents`
or with where statement:
select id, title, filetype, filesize, created, createdby from `shared documents` where filetype = '.gif'
Why don't you use a Content Query web part?
Why don't you use a search object to query the same? This would be my preferred solution. Search has most properties already and you can add more if you need them. Search is probably a lot quicker than querying content database(s).
Whether it is supported or not, it is still bad form to query the Content Database directly and any developer who would suggest this as a solution should get a lecture ;). For instance, what happens if an admin creates a second content database to your webapp? If you query goes across site collections it will not return the desired results until you provide for this in code.
MOSS provides many webservices out of the box which make life a little easier. They are always worth exploring.
For this particular instance, I think the article, Getting a list of files from a MOSS document library using a SharePoint web service, will be of assistance. If this isn't your exact scenario, it will get you on the right track.
If the Document service doesn't help you, the Search service will I'm sure. Check the documentation for usage.
You can get some of the information from the UserInfo table by joining AllUserData.tp_Author to UserInfo.tp_ID, but messing around in these tables is not recommended and can be very fragile, and also your queries are not guaranteed to work after applying any patches or service packs to SharePoint. I would use either webservices or the SharePoint object model to access the data.