Validating XML documents and working with numbers - xml-validation

Suppose I have an XML document that lists all of the sales made for some items. Amongst a list of price values, there are also "total" elements that is the sum of some set of items. The XML document is generated from somewhere, and I would like to validate the XML to verify that
The form is correct
The content is correct. For example, the total of a set of child elements is equal to the sum of all of the values associated with each child.
The XML documents are used as the standard medium between multiple programs, so it is important that the data is accurate. Currently there are checks done by the XML exporter, but I am wondering whether it is possible to validate the XML itself at the end to ensure that it is correct? And only if it is correct would it be valid?

XSD is useful for validating the XML is in a proper format and may help with "1. The form is correct". For example, XSD could be used to validate someone hasn't entered an item SKU where a price was needed.
However, XSD not a good tool for validating business logic as in "2. The content is correct". For that, you will probably need to custom validate the logic using your programming language of choice, perhaps with the help of something like XPath to extract the elements.
-Larry

Related

How do I store data in an SQL database if the database columns can be different?

I'm creating an ASP.NET web application which allows users to digitize paper forms. The user will import their own forms which will be converted into HTML with placeholders inserted to accept values from an input form.
Using the fields on this imported paper form, the website will create an input form based on what information is required. "Templates" can also be created for a specific form which allows a user to auto-fill any data that doesn't normally change with each form fill. A user can also save a form they haven't finished for completing/reviewing later.
My question is: how do I store this data? I can't really use a traditional database table because Form X could look nothing like Form Y and require completely different data. I have a SQL database to store the data in (I need this for other aspects of the site too), but I can't simply store all form data in one table or even have separate tables for each form as this will be impractical on a larger scale.
My initial thoughts were using JSON but I have absolutely no idea where to start with this. Can I put JSON data into a regular SQL database column? Can this be used to generate code to build a web form to allow a user to easily fill out their forms using any device (as per my design requirements)?
I think your problem would be very well served using a document DB like mongoDB or Arangodb. Reality nowadays is that applications can , and sometimes should, use more than one DB.
Having said that, if I had to use a relational DB, I would convert your forms into 3 tables. The first top level form would just store something like:
form name ,
form id ,
etc...
The second table would capture the form fields and would look be something like:
FormID
field Id
field name
fiel type (int, varchar, etc...)
sort no.
etc ...
The third table would capture the information entered by the user:
user id
form id
field id
value
creation date time
last modification date time
etc...
Note that by storing the fields in rows instead of columns, it does not matter that you have different types of forms.
I should mention that the above table definitions are not meant to be complete by any means, they are there to give you an idea on how to get started.
Finally, note that many relational DB allow you to store JSON nowadays directly in the DB as you suggested, but that may not be a very good option depending on which DB you are using. Here is an example of storing JSON in mysql just for your reference.

Is it possible to update Lucene document field from custom token filter?

This is what I want to do.
I have some calculation for each token/term/word in the document.
After calculating each token/term/word, I would like to sum (for example) this number (described in #1) and store it as an attribute of the document.
As I understand, for #1, I can use custom token filter which will help me to perform calculation on each word. After all the words in a document are processed, I will sum up those number for a given document.
My question is, How can I save this aggregate number I calculated? I would like to save it as an attribute or field of a document. As I understand, I can not access field in token filter. Is there any other strategy to achieve it?
Please guide.

How can I incorporate entitlements with over 50,000 data points in SSRS Report?

In my SSRS report I am attempting to incorporate entitlements. If a user puts a specific person ID in the parameter, I'd like to be able to compare it against the up to 50,000 ids they are entitled to use and to put up an error message if not.
The available values feature only allows me to use a dropdown, which is incovenient for so many ids. What are my options?
A drop down would be very inconvenient for that many choices. Is it possible to break them down by a category or something like that? You could then create cascading parameters to make the list easier to deal with.
Otherwise it would have to be a free text field. Depending on the complexity of the report contents you could handle it a couple of ways.
If it is a very simple report with just a tablix you can set the no data row value to your error message.
If it is more complex you can display the error message using two rectangles. The first one would simply contain a text box with the error message in it. Then put the actual report comments in the second box. Control the visibility of these rectangles based on an expression that checks the number of rows returned by the dataset or some other indication that the input was invalid. You may even need to create a second dataset to check for input validity if there isn't a reliable way to use your normal query.

How to handle search for custom fields in form for FROM and TO fields?

I have just started implementing search module in a project, where I have a form with fixed fields consisting of combo box, text box, radio button etc (around 200 fields in multiple tabs), and later client should be able to add extra fields too. Once user fills the fields which he wants to search, that search criteria also he should be able to save. For all these reasons, for each field I am associating metadata in the following format.
"EntityName.attributeName": attributeValue
Once the user fills the form fields to search, I will validate form data and and only non empty fields metadata I am sending to server in JSON format. Everything is fine till now. But I am facing an issues now.
Using the metadata of each field I will create a new criteria for each field. but if there are fields where one field metadata depends on other field metadata I am struck.
In the form I have few special category fields in following format : for example DOB,
FROM DATE (meta data: entity1.dob)
TO DATE (meta data: entity1.dob)
both fields belongs to same entity and same column only field name in the UI is different
Like this I have around 20 fields which asks for FROM and TO to query the range (it need not be on date, for example no of bed rooms..it can be on integer, string etc)
My query formation should be in the following way depending on user search criteria. If user entered only FROM field of number of bed rooms then I have to query using EQUAL to operator in sql and if both mentioned then MORETHANEQUAL to for FROM field and LESSTHANEQUAL to for to field. So how I can handle this special case ?
like if he entered number of fields as 4 in TO field of number of bed rooms, then I have to query for houses having number of bed rooms equal to 4. but if in FROM he entered 3 and in To if he entered 7 then I have to query for houses having greater than or equal to 3 bed rooms and less than or equal to 7 bed rooms.
Since I have same metadata for these category fields also I am unable to proceed, to achieve this, what kind of metadata I need to prepare ?
How I can generalize this process to handle all the cases ?
my technology stack: ExtJs, Eclipse Link, spring.
and what are the best practices to follow to support custom fields adding feature in Forms in enterprise applications ?
Off of the top of my head, I would create wizards for these particular cases. So for example, have a custom wizard that allows the user to define a "from" field, a "to" field, and then the comparison operator in one action. This wizard could also be responsible for adding custom properties to the generated fields that could be used by your validation routine. So based on the combo of from, to, and operator, you could create a flexible validation mechanism for ensuring that correct values are entered, ranges are correct, whatever.
You might consider this "wizard" approach for all custom fields, in fact. I could see you predefining all the possible custom field types that could be used and create classes that can be used for those. The classes could be responsible not only for the field creation, but also for providing any custom validation, pre-submit transformation, etc. This approach would make adding new custom field types incredibly simple since all you'd have to do is follow the same implementation as the others that already exist, extend an existing one, etc.

#DBColumn in Lotus Notes

I've been tasked with learning Lotus Domino Designer - not sure what I did in a previous life, but it must have been pretty bad... - and was wondering how to do a lookup on a database to get some values for selections. As this information could potentially be used in a lot of the applications, I'd prefer it only to be in the one place.
I gather I can use #DBColumn, but what happens if an entry in that lookup changes? If the unique value of the lookup is the text, then the relationship would be broken, wouldn't it? Is there any way of mimicing the idea of relational lookups?
I'm assuming I'm looking at Lotus development from the wrong angle, as this seems to be a real limitation of look ups.
I haven't found any decent learning material on the interwebs, so would appreciate any help.
Ta
You would want to store a unique ID along with the textual value in the source database (not unlike what you would do in an RDBMS). Then, only store that ID in any referencing documents, and use a computed-for-display field to lookup the display value. (There is a performance consideration here - and you could "de-normalize" the data and store the ID and text value in the referencing documents, and do some asynchronous work to keep the values in sync - eg: using a scheduled agent that runs every night or every week).
If DB1 has the key values and DB2 has the documents which will reference these values, then in the form in DB2, you would still do a #DbColumn to lookup your value list. In the lookup view in DB1, concat the text value and ID with a pipe separator (textField + "|" + ID) in the first column. That will tell Notes to store only the ID value (what follows the pipe is the "alias" and is what will be stored).
Note: I would avoid using #DocumentUniqueID as the unique ID for these values, as the Document Unique ID will change if the documents are copied and pasted, or the entire database is copied, etc. You can use the #unique formula function in a computed-when-composed field to generate something close to a unique ID (almost like an identity column in sql).
If you need relational properties, look for non-Notes solutions. It is possible to get some relational behavior using document UNIDs and update agents, but it will be harder than with a proper relational backend.
Your specific problem with referencing to a piece of text that might change can to some extent be resolved by using aliases in the choice fields. If a dialog list contains values on the form...
Foo|id1
Bar|id2
...the form will display Foo but the back-end document will store the value id1 - (and this is what you will be able to show in standard views - although xpages could solve that). Using the #DocumentUniqueID for alias can be a good idea under some circumstances.
It depends on where your using the data. The #DBLookup or #DBColumn will work in Lotus Notes fields if the fields are set to be computed for display. That way they always get the most up to date information when you open the form etc.
If you make it so the data is saved on to the document then you will have to write some update code when you need to refresh the values.
The Lotus Notes help files for designer are pretty good, have a look at that.
SM
You could use a key or alias to store the relationship to your lookup value so if the value itself changes, the connection remains because the alias is intact. For example, if your lookup values were being stored as a collection of documents, I'd have the #DBColumn retrieve Document UNID|lookup value pairs. When in display mode, you could then retrive the value using #GetDocField. If the lookup values are in a different database, then you'd have to retrieve them for display using #DBLookup and construct a view that is keyed off of the UNID or whatever key you decide to use.The only drawback to this technique is that you wouldn't be able to display the field value in views as the actual value isn't stored in the document, just a reference to it. Using XPages, though, you COULD map the relationship into a dynamic datatable just like you would in a truly relational system.
It's tricky, but using LEI, you could also use Notes to front-end a relational backend system, also giving you the dynamic relationship you desire in your lookups.
Hope this helps!
The content of the lookup can change freely. A problem only arises (as it would on any other platform in the same circumstances) if the lookup key changes. You need to use a key that won't change. Human-readable text is an advantage, but if you want to be able to change your key description from, say, "Divisions" to "Business Units" and still have lookups work, you need to use an alias of some kind, which will presumably be mapped to your text description and only used internally. #Unique is pretty good for this, and gives a shortish key, if that is important to you. #DocumentUniqueID is most reliable, but as Ed pointed out, will change (must change - it's a new document) if you copy/paste or make a non-replica copy. This is easy to get around, though. Create a Computed-when-composed field (called, say, "LookupRef") on the form you are using for your reference document with the formula "#DocumentUniqueID". That will capture the ID at the time of creation, and it will not change on copy/paste etc. Use that as your key.