This is an extremely common situation, so I'm expecting a good solution. Basically we need to update counters in our tables. As an example a web page visit:
Web_Page
--------
Id
Url
Visit_Count
So in hibernate, we might have this code:
webPage.setVisitCount(webPage.getVisitCount()+1);
The problem there is reads in mysql by default don't pay attention to transactions. So a highly trafficked webpage will have inaccurate counts.
The way I'm used to doing this type of thing is simply call:
update Web_Page set Visit_Count=Visit_Count+1 where Id=12345;
I guess my question is, how do I do that in Hibernate? And secondly, how can I do an update like this in Hibernate which is a bit more complex?
update Web_Page wp set wp.Visit_Count=(select stats.Visits from Statistics stats where stats.Web_Page_Id=wp.Id) + 1 where Id=12345;
The problem there is reads in mysql by default don't pay attention to transactions. So a highly trafficked webpage will have inaccurate counts.
Indeed. I would use a DML style operation here (see chapter 13.4. DML-style operations):
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
String hqlUpdate = "update webPage wp set wp.visitCount = wp.visitCount + 1 where wp.id = :id";
int updatedEntities = s.createQuery( hqlUpdate )
.setLong( "newName", 1234l )
.executeUpdate();
tx.commit();
session.close();
Which should result in
update Web_Page set Visit_Count=Visit_Count+1 where Id=12345;
And secondly, how can I do an update like this in Hibernate which is a bit more complex?
Hmm... I'm tempted to say "you're screwed"... need to think more about this.
A stored procedure offers several benefits:
In case the schema changes, the code need not change if it were call increment($id)
Concurrency issues can be localized.
Faster execution in many cases.
A possible implementation is:
create procedure increment (IN id integer)
begin
update web_page
set visit_count = visit_count + 1
where `id` = id;
end
Related
I have recently implemented SQL rowversion to prevent concurrency issues in my system. I use rowversion in where clause when updating single rows in the tables. So far I have tested and seems like a good solution. Now I'm looking for an easy way to implement this feature in my system. Here is SP that runs when user wants to update the record:
CREATE PROCEDURE [dbo].[UpdateBuilding]
#Status BIT = NULL,
#Name VARCHAR(50) = NULL,
#Code CHAR(2) = NULL,
#OriginalRowVersion ROWVERSION
AS
SET NOCOUNT ON
SET XACT_ABORT ON
BEGIN
UPDATE dbo.Building
SET Status = #Status,
Name = #Name,
Code = #Code,
ActionDt = CURRENT_TIMESTAMP
WHERE RowVersion = #OriginalRowVersion
IF ##ROWCOUNT = 0
BEGIN
RAISERROR('Buildingwith code %s was modified or deleted by another user.', 16, 1, #Code);
END;
END;
If I want to execute SP above I would need to pass required parameters. This is how I call SP in SQL Management Studio:
EXEC UpdateBuilding
#Status = 1,
#Name = "Rockefeller Center",
#Code = 436,
#OriginalRowVersion = 0x0000000000006955;
Now I started looking how to implement this in my system where I use ColdFusion to communicate with Datatbase. Here is example on how this procedure will be executed with CF 2016:
<cfstoredproc procedure="UpdateBuilding" datasource="#dsn#">
<cfprocparam dbvarname="#Status" value="#trim(arguments.status)#" cfsqltype="cf_sql_bit" />
<cfprocparam dbvarname="#Code" value="#trim(arguments.code)#" cfsqltype="cf_sql_char" maxlength="2" null="#!len(trim(arguments.code))#" />
<cfprocparam dbvarname="#Name" value="#trim(arguments.name)#" cfsqltype="cf_sql_varchar" maxlength="50" null="#!len(trim(arguments.name))#" />
<cfprocresult name="Result"/>
</cfstoredproc>
You can see that all values are passed with arguments that user submits in the form. However, updating based on PK value (Code column in my case) was pretty simple. Now I have binary value and that makes everything more complicated. First I use JSON to send the data to client side. Sending rowversion in JSON object would require converting that value to binary and then converting back when user submits the form. I'm wondering if there is better way to achieve this? Ideally I would not even send rowversion value to the user side. I woul keep that on the back end and once user submits the form pull row version value based on PK then call stored procedure. If anyone knows good way to handle this kind of situations please let me know. I have not used rowversion before and this is new to me.
I use a similar approach where I have a column named version of type int. I pass that to the client in any read operation. If the client updates a record, it must send back the version of the record being updating, and the update will increment the version number. However, my approach sets a ColdFusion lock instead of a DB lock. Here's some simplified logic:
function updateRecord (
required numeric recordID,
required struct updateData,
required numeric version) {
lock name="#arguments.recordID#" type="exclusive" timeout="1" throwontimeout=true {
qRecord = queryExecute() // get the record
if (qRecord.recordCount != 1) {
throw();
}
if (qRecord.version != arguments.version) {
throw();
}
// do the update using arguments.updateData
}
}
One issue with this is that other nodes in a cluster would not be aware of the named lock. You would have to come up with another way of locking that section of code to other requests across the cluster. There are ways to do it, as described in this thread:
https://dev.lucee.org/t/distributed-lock-management/1004
And if this is an issue, I'm sure there are other solutions available.
I am using Postgres as main DB and REDIS for caching. I am working on caching mechanism for one db query which takes to much time (It's about 5-6 JOINS + nested SELECTS). For now I am caching results of this query using SET 'some key' JSON.stringify(query.result). This works fine, however I have one column that cannot be cached - it is called commentsCount. It has to be always up to date. As a temporary solution, I am querying db just for this one particular field like this:
app.get('/post/getBySlug/:slug',function(req,res,next){
var cacheKey = req.params.slug+'|'+req.params.language; // "my-post-slug|en-us" for example
cache.get(cacheKey, function(err, post){
throw err if err;
if(post) {
db.getPostCommentsCount({ where: { id: post.id }}).done(function(err,commentsCount){
throw err if err;
post.commentsCount = commentsCount;
res.json(post);
next()
})
} else {
db.getFullPostBySlug(req.params.slug, req.params.language).done(function(err, post){
throw err if err;
cache.set(cacheKey, post);
res.json(post);
next();
})
}
})
})
But it is still now what I want, because main DB is still queried. Is there any standard/good practise on storing counters in REDIS? My comment insert function looks like this:
START TRANSACTION
INSERT INTO "Comments" VALUES (...) // insert comments
UPDATE "Posts" SET "commentsCount" = "commentsCount" + 1 WHERE "Posts"."id" = 123456 // update counter on post
COMMIT TRANSACTION
I am using transaction because I dont want comment to be inserted without incrementing comments count. As a "side" question - is it better to make 2 sql queries in transaction or write a trigger to handle incrementing counter??
According to my query (I posted link to gist in comments):
We dont plan more than 2 languages (though it is possible)
I made those counters because I have to keep counters separate per language, be able to order by those separate counters and also be able to order by sum of the counters (total for all languages) - I found it hard to make query that would order by sum of columns from separate rows while still returning those rows... (At the begining counters were stored in language translations).
Generally this query looks for post where exists translation with specific 'slug' and 'language' (slug+language on post translation is unique index). Morover post has to be published (isPublished = boolean) and post.status has to be 'published' (status = enum) or post.iscomingSoon has to be true (isComingSoon = boolean). Do you have idea what index/ordering I could add to this query? Or should I just remove limit?
In every translation table I keep language as TEXT. It can be for example en-us or zh-cn etc. Do you think I should make it enum or maybe I should make another table to store languages and just keep language_id in translations?
Author actually can be null :)
I want to retrieve all the files from a cabinet (called 'Wombat Insurance Co'). Currently I am using this DQL query:
select r_object_id, object_name from dm_document(all)
where folder('/Wombat Insurance Co', descend);
This is ok except it only returns a maximum of 100 results. If there are 5000 files in the cabinet I want to get all 5000 results. Is there a way to use pagination to get all the results?
I have tried this query:
select r_object_id, object_name from dm_document(all)
where folder('/Wombat Insurance Co', descend)
ENABLE (RETURN_RANGE 0 100 'r_object_id DESC');
with the intention of getting results in 100 file increments, but this query gives me an error when I try to execute it. The error says this:
com.emc.documentum.fs.services.core.CoreServiceException: "QUERY" action failed.
java.lang.Exception: [DM_QUERY2_E_UNRECOGNIZED_HINT]error:
"RETURN_RANGE is an unknown hint or is being used incorrectly."
I think I am using the RETURN_RANGE hint correctly, but maybe I'm not. Any help would be appreciated!
I have also tried using the hint ENABLE(FETCH_ALL_RESULTS 0) but this still only returns a maximum of 100 results.
To clarify, my question is: how can I get all the files from a cabinet?
You have already accepted an answer which is using DFS.
Since your are playing with DFC, these information might help you.
DFS:
If you are using DFS, you have to aware about the number of concurrent sessions that you can consume with DFS.
I think it is 100 or 150.
DFC:
Actually there is a limit that you can fetch via DFC (I'm not sure with DFS).
Go to your DFC application(webtop or da or anything) and check the dfc.properties file.
# Maximum number of results to retrieve by a query search.
# min value: 1, max value: 10000000
#
dfc.search.max_results = 100
# Maximum number of results to retrieve per source by a query search.
# min value: 1, max value: 10000000
#
dfc.search.max_results_per_source = 400
dfc.properties.full or similar file is there and you can verify these values according to your system.
And I'm talking about the ContentServer side, not the client side dfc.properties file.
If you use ENABLE (RETURN_TOP) hint with DFC, there are 2 ways to fetch the results from the ContentServer.
Object based
Row based
You have to configure this by using the parameter return_top_results_row_based in the server.ini file.
All of these changes for the documentum server side, not for your DFC/DQL client.
Aha, I've figured it out. Using DFS with Java (an abstraction layer on top of DFC) you can set the starting index for query results:
String queryStr = "select r_object_id, object_name from dm_document(all)
where folder('/Wombat Insurance Co', descend);"
PassthroughQuery query = new PassthroughQuery();
query.setQueryString(queryStr);
query.addRepository(repositoryStr);
QueryExecution queryEx = new QueryExecution();
queryEx.setCacheStrategyType(CacheStrategyType.DEFAULT_CACHE_STRATEGY);
queryEx.setStartingIndex(currentIndex); // set start index here
OperationOptions operationOptions = null;
// will return 100 results starting from currentIndex
QueryResult queryResult = queryService.execute(query, queryEx, operationOptions);
You can just increment the currentIndex variable to get all results.
Well, the hint is being used incorrectly. Start with 1, not 0.
There is no built-in limit in DQL itself. All results are returned by default. The reason you get only 100 results must have something to do with the way you're using DFC (or whichever other client you are using). Using IDfCollection in the following way will surely return everything:
IDfQuery query = new DfQuery("SELECT r_object_id, object_name "
+ "FROM dm_document(all) WHERE FOLDER('/System', DESCEND)");
IDfCollection coll = query.execute(session, IDfQuery.DF_READ_QUERY);
int i = 0;
while (coll.next()) i++;
System.out.println("Number of results: " + i);
In a test environment (CS 6.7 SP1 x64, MS SQL), this outputs:
Number of results: 37162
Now, there's proof. Using paging is however a good idea if you want to improve the overall performance in your application. As mentioned, start counting with the number 1:
ENABLE(RETURN_RANGE 1 100 'r_object_id DESC')
This way of paging requires that sorting be specified in the hint rather than as a DQL statement. If all you want is the first 100 records, try this hint instead:
ENABLE(RETURN_TOP 100)
In this case sorting with ORDER BY will work as you'd expect.
Lastly, note that adding (all) will not only find all documents matching the specified qualification, but all versions of every document. If this was your intention, that's fine.
I've worked with DFC API (with Java) for a while but I don't remember any default limit on queries, IIRC we've always got all of the documents, there weren't any limit. Actually (according to my notes) we have to set the limit explicitly with, for example, enable (return_top 2000). (As far I know the syntax might be depend on the DBMS behind EMC Documentum.)
Just a guess: check your dfc.properties file.
I have a question about how Hibernate generates SQL when used with the Criteria API. I have an #Entity called Mission. Each mission is linked to a Client (which the mission has been created for) and a ressource (which is the person assigned to this mission). When I query the list of all missions like this :
entityManager.getTransaction().begin();
CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();
CriteriaQuery<Mission> criteriaQuery = criteriaBuilder.createQuery(Mission.class);
Root<Mission> mission = criteriaQuery.from(Mission.class);
...
criteriaQuery.select(mission);
TypedQuery<Mission> query = entityManager.createQuery(criteriaQuery);
missions = query.setFirstResult(firstResult).setMaxResults(maxResults).getResultList();
entityManager.getTransaction().commit();
It seems that Hibernate generates maxResults queries for each of the mission.
Hibernate:
select
missionsc0_.ID_CLIENT as ID10_7_1_,
missionsc0_.ID_MISSION as ID1_1_,
missionsc0_.ID_MISSION as ID1_11_0_,
missionsc0_.active as active11_0_,
missionsc0_.ID_CLIENT as ID10_11_0_,
missionsc0_.CODE_ARTICLE as CODE3_11_0_,
missionsc0_.HAS_PERIODE as HAS4_11_0_,
missionsc0_.DESCRIPTION as DESCRIPT5_11_0_,
missionsc0_.END_DATE as END6_11_0_,
missionsc0_.LIBELLE as LIBELLE11_0_,
missionsc0_.ID_RESSOURCE as ID11_11_0_,
missionsc0_.START_DATE as START8_11_0_,
missionsc0_.VERSION as VERSION11_0_
from
MISSION missionsc0_
where
missionsc0_.ID_CLIENT=?
So I have maxResults times this query. I thought it would only fire ONE TIME with something like
SELECT * FROM Mission WHERE ... LIMIT 0, MaxResults
Can somebody explain to me why it's generating all those queries ? I suspect it to have an impact on general performances.
Thanks a lot,
What is the underlying database? Hibernate does what it can, but if the database does not implement limit, it'll have to do the legwork itself (SQL Server for example).
Sorry if this has been asked before, I did a search but couldn't find anything.
Is it possible to execute inline sql in nHibernate? I have something like this which I would like to run against the dB:
_session.CreateSQLQuery(
#"update things
set defaultThing = 0 where parentId = :parentId AND thingId <> :thingId")
.SetInt32("parentId ", parent.Id)
.SetInt32("thingId", thing.Id)
;
I suppose I could loop through a bunch of 'things' and set the defaultThing setting to false then call _session.Update(thing), but if I can do it how I outlined above, that would be great.
Yes, just use ExecuteUpdate() on that query. It's the equivalent of IDbCommand.ExecuteNonQuery().
As kay mentioned, you can use HQL too. Check 12.3. DML-style operations