I want to poll for new documents in my Raven DB. What is the recommended way of doing this? Can I use the IndexTimestamp or can I rely on the order of the documents?
I guess I want to either do it in two steps:
1. Check if there is anything new, if so:
1.1. Get the latest X documents.
Or in one step: Get the latest X documents and have it return those or tell me that there's nothing new according some argument I sent.
FYI: I have no corresponding CLR objects to the documents.
I would not poll for it, but I would use the Changes API included with RavenDB to just get the continuous stream of documents from RavenDB.
Check out the Changes API here http://ravendb.net/docs/2.0/client-api/changes-api
I personally would use the Changes API with some kind of Message Bus (RabbitMQ) to make sure every change is processed and resilient.
If you still want to poll, just create an index with your date time and sort in descending order.
var result = session.Query<Orders>()
.OrderByDescending(x => x.Created)
.Take(10)
.ToList();
If you need to process every document, you might want to create marker documents that include the id of the document you get and make sure they have not been processed.
To do that:
marker id : polling/processed/order/1
Index:
from o in orders
let processed = LoadDocument("polling/processed/" + o.Id)
select new {
WasProcessed = processed != null,
Created = o.Created
}
A few options for you, hope that helps :)
Related
I'm a newbie in NetSuite Scripting and was recently asked to apply the value from a date field (custbody_expiration_date) on item receipt transaction body to the expiration date field in the inventory details of all items when the item receipt is created.
Since there is no way to create a workflow on inventory details, I've managed to work out below codes however I'm keeping getting all sorts of different error message. Below is one of them after I click on save on item receipt.
Notice (SuiteScript)
org.mozilla.javascript.EcmaError: TypeError: Cannot find function getCurrentLineItemValue in object standard record. (/SuiteScripts/ARROW/Expiration_date_apply_to_all (1).js#27)
I am very confused on the difference between dynamic and standard mode, which functions should be used in which mode? Also, I am a bit hesitated on whether user event script is the correct way to go?
/**
*#NApiVersion 2.0
*#NScriptType UserEventScript
*#NModuleScope Public
*/
define(['N/record','N/search'], function (record, search) {
function beforeSubmit(context) {
var IRrecord = context.newRecord;
var numberOfLineItems = IRrecord.getLineCount({
sublistId: 'item'
});
var expirationdate = IRrecord.getValue({
fieldId: 'custbody_expiration_date'
});
for (var i=1; i<=numberOfLineItems; i++){
IRrecord.setSublistValue({
sublistId: 'item',
fieldId: 'item',
line: i,
value: true
});
//First get Lot Number and Quantity
var lotNumber = IRrecord.getCurrentLineItemValue('item', 'receiptinventorynumber');
var quantity = IRrecord.getCurrentLineItemValue('item', 'quantity');
var inventoryDetail = IRrecord.createCurrentLineItemSubrecord('item','inventorydetail');
inventoryDetail.selectNewLineItem('inventoryassignment');
inventoryDetail.setCurrentLineItemValue('inventoryassignment', 'issueinventorynumber', lotNumber);
inventoryDetail.setCurrentLineItemValue('inventoryassignment', 'quantity', quantity);
inventoryDetail.setCurrentLineItemValue('inventoryassignment', 'expirationdate', expirationdate);
inventoryDetail.commitLineItem('inventoryassignment');
inventoryDetail.commit();
IRrecord.commitLineItem('item');
}
nlapiSubmitRecord(IRrecord);
}
return {
beforeSubmit: beforeSubmit
}
});
Dynamic records are the kind you see client-side (as a rule) - modify a field value and some other field becomes refreshed and updated in real time. Forms sometimes need to have their fields filled in a particular order to prevent form completion errors triggering or field sourcing to work. For example, when entering a sales order, selecting the customer then defaults the sales tax when items are added to the order. Errors may be thrown at any point before the record save because a field is triggering dynamic sourcing (updating other fields), based on what has been entered.
Standard mode is - less dynamic. You populate the fields of the record in any order you choose, and when the save is performed, you choose whether sourcing (updating other fields from the data available) is triggered. Any errors in data entry are reported when the save is performed. I think it also has a lower client-side load as there are fewer AJAX queries being triggered.
Both are available in client-side and server-side javascript, but some record types cannot be updated client-side and must be done server-side using workflow actions, User Event, Restlet, Suitelets, or scheduled scripts. To the best of my knowledge, inventory subrecords on fulfillments, receipts and the like are one such type.
The way lines are updated changes between dynamic and standard mode. In dynamic mode, lines are selected, updated then committed and the methods used would be :
selectLine
setCurrentLineItemValue
commitLine (only do this if actually changing the line)
For standard mode, the way of changing lines is only to use setSublistValue and include the line number in the parameters.
Workflow action scripts will load the record in dynamic mode, but the load method can be investigated using the isDynamic() method on the record.
The other thing is, in SuiteScript 2, sublist lines are indexed from 0, not from 1 as your script is using. What's confusing is, in Suitescript 1, indexing was from 1. The code is using a mix of v1 & v2. nlapiSubmitRecord is v1, IRrecord.save is v2.
And for more information, see SuiteAnswer 79715 which explains how to set a value on the inventory detail on an item receipt. The example reloads the record in standard mode and updates the inventoryStatus field. SuiteAnswer 45372 explains the Record object and the difference between standard and dynamic modes. Take a look at SuiteAnswer 67605 which explains the basics of SuiteScript v2. SuiteAnswers is an amazing resource and the search is surprisingly good. I can also recommend Eric T Grubaugh's site (#erictgrubaugh) which has some great videos including comparisons between v1 & v2.
I am using TaxonomyManager gettree(path) method to get a particular tree hierarchy in my c# code but it is taking more than 3 min to get the result, due to this the website is taking long time to load. How to reduce the time to load the website, is there any other way i can use to get the hierarchy from Ektron.
We had this exact same issue and actually got on with Ektron support to help resolve it.
Now, whenever we work with taxonomies we cache them on the server-side to avoid the performance hit. Something like
string cacheKey = "Something unique for your situation";
TaxonomyData taxonomyData;
if (Ektron.Cms.Context.HttpContext.Cache[cacheKey] == null)
{
// Pull taxonomy data and store in cache.
Ektron.Cms.Context.HttpContext.Cache.Insert(cacheKey, taxonomyData);
}
else
{
taxonomyData = (TaxonomyData)Ektron.Cms.Context.HttpContext.Cache[cacheKey];
}
Since you already know how to pull the TaxonomyData I left that out. We don't store the taxonomy data, instead we store the object we create with the taxonomy data, so just cache whatever you need to and then you can avoid the performance hit 'most' of the time.
I don't remember where the ektron cache time is set, whether it's in the web.config or within the WorkArea. Ektron support said to use the Ektron cache, not sure how much of a difference it would make to use the regular cache instead.
I have a bunch of existing sagas in various states of a long running process.
Recently we decided to make one of the properties on our IContainSagaData implementation unique by using the Saga.UniqueAttribute (about which more here http://docs.particular.net/nservicebus/nservicebus-sagas-and-concurrency).
After deploying the change, we realized that all our old saga instances were not being found, and after further digging (thanks Charlie!) discovered that by adding the unique attribute, we were required to data fix all our existing sagas in Raven.
Now, this is pretty poor, kind of like adding a index to a database column and then finding that all the table data no longer select-able, but being what it is, we decided to create a tool for doing this.
So after creating and running this tool we've now patched up the old sagas so that they now resemble the new sagas (sagas created since we went live with the change).
However, despite all the data now looking right we're still not able to find old instances of the saga!
The tool we wrote does two things. For each existing saga, the tool:
Adds a new RavenJToken called "NServiceBus-UniqueValue" to the saga metadata, setting the value to the same value as our unique property for that saga, and
Creates a new document of type NServiceBus.Persistence.Raven.SagaPersister.SagaUniqueIdentity, setting the SagaId, SagaDocId, and UniqueValue fields accordingly.
My questions are:
Is it sufficient to simply make the data look correct or is there something else we need to do?
Another option we have is to revert the change which added the unique attribute. However in this scenario, would those new sagas which have been created since the change went in be OK with this?
Code for adding metadata token:
var policyKey = RavenJToken.FromObject(saga.PolicyKey); // This is the unique field
sagaDataMetadata.Add("NServiceBus-UniqueValue", policyKey);
Code for adding new doc:
var policyKeySagaUniqueId = new SagaUniqueIdentity
{
Id = "Matlock.Renewals.RenewalSaga.RenewalSagaData/PolicyKey/" + Guid.NewGuid().ToString(),
SagaId = saga.Id,
UniqueValue = saga.PolicyKey,
SagaDocId = "RenewalSaga/" + saga.Id.ToString()
};
session.Store(policyKeySagaUniqueId);
Any help much appreciated.
EDIT
Thanks to David's help on this we have fixed our problem - the key difference was we used the SagaUniqueIdentity.FormatId() to generate our document IDs rather than a new guid - this was trivial tio do since we were already referencing the NServiceBus and NServiceBus.Core assemblies.
The short answer is that it is not enough to make the data resemble the new identity documents. Where you are using Guid.NewGuid().ToString(), that data is important! That's why your solution isn't working right now. I spoke about the concept of identity documents (specifically about the NServiceBus use case) during the last quarter of my talk at RavenConf 2014 - here are the slides and video.
So here is the long answer:
In RavenDB, the only ACID guarantees are on the Load/Store by Id operations. So if two threads are acting on the same Saga concurrently, and one stores the Saga data, the second thread can only expect to get back the correct saga data if it is also loading a document by its Id.
To guarantee this, the Raven Saga Persister uses an identity document like the one you showed. It contains the SagaId, the UniqueValue (mostly for human comprehension and debugging, the database doesn't technically need it), and the SagaDocId (which is a little duplication as its only the {SagaTypeName}/{SagaId} where we already have the SagaId.
With the SagaDocId, we can use the Include feature of RavenDB to do a query like this (which is from memory, probably wrong, and should only serve to illustrate the concept as pseudocode)...
var identityDocId = // some value based on incoming message
var idDoc = RavenSession
// Look at the identity doc's SagaDocId and pull back that document too!
.Include<SagaIdentity>(identityDoc => identityDoc.SagaDocId)
.Load(identityDocId);
var sagaData = RavenSession
.Load(idDoc.SagaDocId); // Already in-memory, no 2nd round-trip to database!
So then the identityDocId is very important because it describes the uniqueness of the value coming from the message, not just any old Guid will do. So what we really need to know is how to calculate that.
For that, the NServiceBus saga persister code is instructive:
void StoreUniqueProperty(IContainSagaData saga)
{
var uniqueProperty = UniqueAttribute.GetUniqueProperty(saga);
if (!uniqueProperty.HasValue) return;
var id = SagaUniqueIdentity.FormatId(saga.GetType(), uniqueProperty.Value);
var sagaDocId = sessionFactory.Store.Conventions.FindFullDocumentKeyFromNonStringIdentifier(saga.Id, saga.GetType(), false);
Session.Store(new SagaUniqueIdentity
{
Id = id,
SagaId = saga.Id,
UniqueValue = uniqueProperty.Value.Value,
SagaDocId = sagaDocId
});
SetUniqueValueMetadata(saga, uniqueProperty.Value);
}
The important part is the SagaUniqueIdentity.FormatId method from the same file.
public static string FormatId(Type sagaType, KeyValuePair<string, object> uniqueProperty)
{
if (uniqueProperty.Value == null)
{
throw new ArgumentNullException("uniqueProperty", string.Format("Property {0} is marked with the [Unique] attribute on {1} but contains a null value. Please make sure that all unique properties are set on your SagaData and/or that you have marked the correct properties as unique.", uniqueProperty.Key, sagaType.Name));
}
var value = Utils.DeterministicGuid.Create(uniqueProperty.Value.ToString());
var id = string.Format("{0}/{1}/{2}", sagaType.FullName.Replace('+', '-'), uniqueProperty.Key, value);
// raven has a size limit of 255 bytes == 127 unicode chars
if (id.Length > 127)
{
// generate a guid from the hash:
var key = Utils.DeterministicGuid.Create(sagaType.FullName, uniqueProperty.Key);
id = string.Format("MoreThan127/{0}/{1}", key, value);
}
return id;
}
This relies on Utils.DeterministicGuid.Create(params object[] data) which creates a Guid out of an MD5 hash. (MD5 sucks for actual security but we are only looking for likely uniqueness.)
static class DeterministicGuid
{
public static Guid Create(params object[] data)
{
// use MD5 hash to get a 16-byte hash of the string
using (var provider = new MD5CryptoServiceProvider())
{
var inputBytes = Encoding.Default.GetBytes(String.Concat(data));
var hashBytes = provider.ComputeHash(inputBytes);
// generate a guid from the hash:
return new Guid(hashBytes);
}
}
}
That's what you need to replicate to get your utility to work properly.
What's really interesting is that this code made it all the way to production - I'm surprised you didn't run into trouble before this, with messages creating new saga instances when they really shouldn't because they couldn't find the existing Saga data.
I almost think it might be a good idea if NServiceBus would raise a warning any time you tried to find Saga Data by anything other than a [Unique] marked property, because it's an easy thing to forget to do. I filed this issue on GitHub and submitted this pull request to do just that.
I want to be able to use the AppReceiptId from the result of CurrentApp.GetAppReceiptAsync() and tie it to a username in my backend service, to verify that the user has actually purchased the app.
I know I'm supposed to use CurrentAppSimulator in place of CurrentApp, but CurrentAppSimulator.GetAppReceiptAsync() always returns a different, random value for AppReceiptId. This makes it difficult to test with my service.
Is there a way to make it always return the same value, other than just using a hardcoded one? I'm worried that when I replace CurrentAppSimulator with CurrentApp and submit it to the store, it won't behave the way I expect it to. In the real world, the AppReceiptId won't ever change, right?
The Code I use to get AppReceiptId:
var receiptString = await CurrentAppSimulator.GetAppReceiptAsync();
XmlDocument doc = new XmlDocument();
doc.LoadXml(receiptString);
var ReceiptNode = (from s in doc.ChildNodes
where s.NodeName == "Receipt"
select s).Single();
var AppReceiptNode = (from s in ReceiptNode.ChildNodes
where s.NodeName == "AppReceipt"
select s).Single();
var idNode = (from s in AppReceiptNode.Attributes
where s.NodeName == "Id"
select s).Single();
string id = idNode.NodeValue.ToString();
id will always be some random Guid.
CurrentApp.GetAppReceiptAsync().Id is a unique ID for the actual purchase. Although it does technically represent a unique purchase made by a single Windows ID, it doesn't represent the user themselves and I don't think there's any guarantee on the durability of that ID.
Would you be better suited using the Windows Live SDK to track the actual user identity across devices?
At any rate, to answer your original question, no I don't believe there's any way to make it return the same ID all the time. The only logical place for that functionality would be in the WindowsStoreProxy.xml file, and I don't see anything in the schema that would allow you to specify this information.
I want to add property to existing document (using clues form http://ravendb.net/docs/client-api/partial-document-updates). But before adding want to check if that property already exists in my database.
Is any "special,proper ravendB way" to achieve that?
Or just load document and check if this property is null or not?
You can do this using a set based database update. You carry it out using JavaScript, which fortunately is similar enough to C# to make it a pretty painless process for anybody. Here's an example of an update I just ran.
Note: You have to be very careful doing this because errors in your script may have undesired results. For example, in my code CustomId contains something like '1234-1'. In my first iteration of writing the script, I had:
product.Order = parseInt(product.CustomId.split('-'));
Notice I forgot the indexer after split. The result? An error, right? Nope. Order had the value of 12341! It is supposed to be 1. So be careful and be sure to test it thoroughly.
Example:
Job has a Products property (a collection) and I'm adding the new Order property to existing Products.
ravenSession.Advanced.DocumentStore.DatabaseCommands.UpdateByIndex(
"Raven/DocumentsByEntityName",
new IndexQuery { Query = "Tag:Jobs" },
new ScriptedPatchRequest { Script =
#"
this.Products.Map(function(product) {
if(product.Order == undefined)
{
product.Order = parseInt(product.CustomId.split('-')[1]);
}
return product;
});"
}
);
I referenced these pages to build it:
set based ops
partial document updates (in particular the Map section)