I Have a Entry field "Batch name" linked to sql data base . The batch name is entered manually by me for each batch through scada system . With this batch name I will sort sql database table and retrieve data pertaining to this batch name only.
Now I am looking for vbscript that will not allow any previously entered batch name to be entered again and will pop a message "Batch name already exist".
In new ScadaLTS, we may this in several ways but we use java-script (not VBScript):
Create virtual data source with virtual data point Alphanumeric (Series) settable.
Create meta data source with the point and when value change then validate (in java-script) unique name.
Another way:
Create virtual data source with virtual data point Alphanumeric (Series).
And save data use java-script with REST who the previous data checks whether or not the value added.
Another way:
Contact the Scada LTS programmers and request for the implemented data type set. (To which by default you will not be able to add duplicates).
Related
I'm creating an ASP.NET web application which allows users to digitize paper forms. The user will import their own forms which will be converted into HTML with placeholders inserted to accept values from an input form.
Using the fields on this imported paper form, the website will create an input form based on what information is required. "Templates" can also be created for a specific form which allows a user to auto-fill any data that doesn't normally change with each form fill. A user can also save a form they haven't finished for completing/reviewing later.
My question is: how do I store this data? I can't really use a traditional database table because Form X could look nothing like Form Y and require completely different data. I have a SQL database to store the data in (I need this for other aspects of the site too), but I can't simply store all form data in one table or even have separate tables for each form as this will be impractical on a larger scale.
My initial thoughts were using JSON but I have absolutely no idea where to start with this. Can I put JSON data into a regular SQL database column? Can this be used to generate code to build a web form to allow a user to easily fill out their forms using any device (as per my design requirements)?
I think your problem would be very well served using a document DB like mongoDB or Arangodb. Reality nowadays is that applications can , and sometimes should, use more than one DB.
Having said that, if I had to use a relational DB, I would convert your forms into 3 tables. The first top level form would just store something like:
form name ,
form id ,
etc...
The second table would capture the form fields and would look be something like:
FormID
field Id
field name
fiel type (int, varchar, etc...)
sort no.
etc ...
The third table would capture the information entered by the user:
user id
form id
field id
value
creation date time
last modification date time
etc...
Note that by storing the fields in rows instead of columns, it does not matter that you have different types of forms.
I should mention that the above table definitions are not meant to be complete by any means, they are there to give you an idea on how to get started.
Finally, note that many relational DB allow you to store JSON nowadays directly in the DB as you suggested, but that may not be a very good option depending on which DB you are using. Here is an example of storing JSON in mysql just for your reference.
I'm new to PDI and Kettle, and what I thought was a simple experiment to teach myself some basics has turned into a lot of frustration.
I want to check a database to see if a particular record exists (i.e. vendor). I would like to get the name of the vendor from reading a flat file (.CSV).
My first hurdle selecting only the vendor name from 8 fields in the CSV
The second hurdle is how to use that vendor name as a variable in a database query.
My third issue is what type of step to use for the database lookup.
I tried a dynamic SQL query, but I couldn't determine how to build the query using a variable, then how to pass the desired value to the variable.
The database table (VendorRatings) has 30 fields, one of which is vendor. The CSV also has 8 fields, one of which is also vendor.
My best effort was to use a dynamic query using:
SELECT * FROM VENDORRATINGS WHERE VENDOR = ?
How do I programmatically assign the desired value to "?" in the query? Specifically, how do I link the output of a specific field from Text File Input to the "vendor = ?" SQL query?
The best practice is a Stream lookup. For each record in the main flow (VendorRating) lookup in the reference file (the CSV) for the vendor details (lookup fields), based on its identifier (possibly its number or name or firstname+lastname).
First "hurdle" : Once the path of the csv file defined, press the Get field button.
It will take the first line as header to know the field names and explore the first 100 (customizable) record to determine the field types.
If the name is not on the first line, uncheck the Header row present, press the Get field button, and then change the name on the panel.
If there is more than one header row or other complexities, use the Text file input.
The same is valid for the lookup step: use the Get lookup field button and delete the fields you do not need.
Due to the fact that
There is at most one vendorrating per vendor.
You have to do something if there is no match.
I suggest the following flow:
Read the CSV and for each row look up in the table (i.e.: the lookup table is the SQL table rather that the CSV file). And put default upon not matching. I suggest something really visible like "--- NO MATCH ---".
Then, in case of no match, the filter redirect the flow to the alternative action (here: insert into the SQL table). Then the two flows and merged into the downstream flow.
I need guidance regarding the most approriate approach to perform a index function using pentaho Data integration ( kettle )
my situation is as following :
using the GLOBAL voip system report, I stored all data in a Mysql Database, which gives me several id number + name and lastname but whithout the departement name.
each departement name has it's own excel reports that can be identified by the group file name, which is not available in the Global file.
what i am trying to achieve is a lookup for each identification number to identify the departement where he belongs using the report filename and store it on the approriate column.
Any help will be appreciated.
Assuming you're using the Excel File Input step, there is an option on the Additional Output Fields tab that will allow you to specify the Full Filename Field. You can name this whatever you want, and it will add an additional column to your incoming Excel data that has the name of the file as one of the columns. You may need to do some regex cleanup on that fields since it's the full file path, not just the filename.
As far as doing the lookup, there are many lookup options to merge streams in the Lookup category of the design tab. I think the Stream Lookup is the step you'll want.
As far as I understood your need, you have to first build a "mapping table" of two columns: the department (aka the start of the xls filename) and the employee (aka its ID).
This table does not need to be materialized and may stay in a step of the the PDI. So
Read all the xls files with a Microsoft Excel File. In case you do not know how to do it: Browse to any of these file, press the Add button, then in the Selected files table, remove the filename to keep only its directory path and write .*\.xls in the Regex wildcard. Check you select the appropriates files with the Show filename button.
In the same step, define the Sheet to be "Fiche technique" (assuming they are all the same). Define the field to be "A" with type String (an empty column) and "ID" also with type String (otherwise you'll have a un-trappable error on "Agent ID" and "Total". Also follow #eicherjc suggestion and keep the filename, although I suggest you keep the Short file name and call it filename.
You should get a two column stream: ID and filename, which need some bit of data massage before to be used. The ID contains non-integer fields and the file name contains extra characters.
The simplest way to do this is with a Modified Javascript Value. I may suggest the code:
var ID = Number(ID);
var regex = filename.match(/(.*)__\d+\.xls/);
if(regex) filename = regex[1];
and do not forget specify the the ID has now a type Integer and to put a "Y" in the Replace value in field of the Fields`` table at the bottom.
The first line will convert any number in its value, and non-number in a 0, which is an ID that does not exists.
The next lines will extract the department from the filename with a Regex. If you do not like regex, you may use a filename = filename.substr(0, filename.indexOf('__')), or any formula that will do the job.
Now you have a stream ready to be used, except that some employees may, right or wrong, be in more than one department. If it does not matter which one, then leave it like that. Otherwise you have to provide some logic to filter the correct department.
You can now use a Lookup Stream to read the department of each employee. The Lookup step is the Modified Javascript value (or whatever name you gave to this step). The field to lookup is the field of the ID in your mySql. The Lookup field is the ID (or whatever name you gave to the column B of your xls files). And the field to retrieveenter code here is the filename (or more precisely, the department name extracted from the filename).
I am currently building a CMBD in ServiceNow and have come across a problem I cannot seem to figure out. In my database instance table, I would like to be able to add a field to the table that pulls from all of the names of applications I have in my applications table, and then i would like to display both the instance name and the application name in the label for my dependency view like this:
Database Instance, Application Name
or
Database Instance (Application Name)
I have managed to set up the application name field by creating a reference in the database instance table but I cannot figure out how to make the label for the database instance objects to display both the instance and the application name. I know it probably has some scripting behind it but I do not know where to start.
The label shown in the dependency field is the field where "Display" is set to true. On Configuration Items, when no display field is set, it takes the name field by default.
If you want to keep the name field for the Application's name only, you will have to do the following
Create an additional field for the display name, u_display_name for example
Make the field read-only and check it as Display field
Create an onBefore Business Rule that triggers on the change of the Application or Instance name fields
Let the Business Rule set the display field as required
See the below screenshot for an example
I am new in Pentaho project. Need to know about some validation rules whether these can be handled by PDI or not.
My input data is CSV format and target is ORACLE table. Suppose total number of column is 50.
Validation Rules :
if any less number of columns coming from input file (like 48) we
should inform via email
if any more number of columns coming from input file (like 54) we
should inform via email
if there are any duplicate columns coming in input file we should
inform via email
if there a missing order in input file with respect to target table
layout we should inform via email
if there any columns started with a number we should inform via email
if there any columns more than 30 characters in length we should
inform via email
Any help is appreciated.
If anything required from me kindly reply.
After lots of R&D it can be done by adding "Metadata Structure" under Utility in Transformation and load all column list into a text file.
Then create a Job and compare this text file with lookup text file (containing all target column list) by "File Compare" under File Management. Make a check option if true then invoke the final Transformation or if False send email notification.
Yes. There are steps that will allow you to identify all of these things.
More generally, if you want to do something in Kettle and don't see a step available that makes it easy to do, you can always write custom steps or steps with custom Java or JavaScript code in them to do it.