Is there a way to generate UUID in pentaho step using input fields?
Example:
Input: Name, Address.
Output: UUID = UUID(Name + Address)
You can add a user defined java class and use a code similar to this:
String input = "Some name" + "Some address";
byte[] serialized = input.getBytes("UTF8");
UUID yourId = UUID.nameUUIDFromBytes(serialized);
This will generate a deterministic UUID based on the given input you have.
you can use Add checkup step of pentaho data integration, it will create a unique code for combination of fields.
The UUID.nameUUIDFromBytes() generates MD5 UUIDs. SHA1 is preferred over MD5. You can create SHA1 UUIDs with UuidCreator.getNameBasedSha1().
In this example, the variables name and address are concatenated to generate a SHA1 UUID:
// Create a name based UUID
String name = "localhost";
String address = "127.0.0.1";
UUID uuid = UuidCreator.getNameBasedSha1(name + address);
In this other example, a custom name space called "network" is used along with name and address:
// Create a custom namespace called 'network'
UUID namespace = UuidCreator.getNameBasedSha1("network");
// Create a name based UUID inside the 'network'
String name = "localhost";
String address = "127.0.0.1";
UUID uuid = UuidCreator.getNameBasedSha1(namespace, name + address);
Project page: https://github.com/f4b6a3/uuid-creator
Related
I am trying to upload json data to one of the table created under the dataset in Bigquery but fails with "Google.GoogleApiException: 'Google.Apis.Requests.RequestError
Not found: Table currency-342912:sampleDataset.currencyTable [404]"
Service account is created with roles BigQuery.Admin/DataEditor/DataOwner/DataViewer.
The roles are also applied to the table also.
Below is the snippet
public static void LoadTableGcsJson(string projectId = "currency-342912", string datasetId = "sampleDataset", string tableId= "currencyTable ")
{
//Read the Serviceaccount key json file
string dir = Directory.GetParent(Directory.GetCurrentDirectory()).Parent.Parent.FullName + "\\" + "currency-342912-ae9b22f23a36.json";
GoogleCredential credential = GoogleCredential.FromFile(dir);
string toFileName = Directory.GetParent(Directory.GetCurrentDirectory()).Parent.Parent.FullName + "\\" + "sample.json";
BigQueryClient client = BigQueryClient.Create(projectId,credential);
var dataset = client.GetDataset(datasetId);
using (FileStream stream = File.Open(toFileName, FileMode.Open))
{
// Create and run job
BigQueryJob loadJob = client.UploadJson(datasetId, tableId, null, stream); //This throws error
loadJob.PollUntilCompleted();
}
}
Permissions for the table, using the service account "sampleservicenew" from the screenshot
Any leads on this , much appreciated
Your issue might reside in your user credentials. Please follow this steps to check your code:
Please check if the user you are using to execute your application have access to the table you want to insert data.
If your json tags matchs your table columns.
If you json inputs are correct ( table name, dataset name ).
Use a dummy table to perform a quick test of your credentials and data integrity.
These steps will help you identifying what could be missing on your side. I perform the following operations to reproduce your case:
I created a table on BigQuery based on the values of your json data:
create or replace table `projectid.datasetid.tableid` (
IsFee BOOL,
BlockDateTime timestamp,
Address STRING,
BlockHeight INT64,
Type STRING,
Value INT64
);
Created a .json file with your test data
{"IsFee":false,"BlockDateTime":"2018-09-11T00:12:14Z","Address":"tz3UoffC7FG7zfpmvmjUmUeAaHvzdcUvAj6r","BlockHeight":98304,"Type":"OUT","Value":1}
{"IsFee":false,"BlockDateTime":"2018-09-11T00:12:14Z","Address":"tz2KuCcKSyMzs8wRJXzjqoHgojPkSUem8ZBS","BlockHeight":98304,"Type":"IN","Value":18}
Build & Run below code.
using System;
using Google.Cloud.BigQuery.V2;
using Google.Apis.Auth.OAuth2;
using System.IO;
namespace stackoverflow
{
class Program
{
static void Main(string[] args)
{
String projectid = "projectid";
String datasetid = "datasetid";
String tableid = "tableid";
String safilepath ="credentials.json";
var credentials = GoogleCredential.FromFile(safilepath);
BigQueryClient client = BigQueryClient.Create(projectid,credentials);
using (FileStream stream = File.Open("data.json", FileMode.Open))
{
BigQueryJob loadJob = client.UploadJson(datasetid, tableid, null, stream);
loadJob.PollUntilCompleted();
}
}
}
}
output
Row
IsFee
BlockDateTime
Address
BlockHeight
Type
value
1
false
2018-09-11 00:12:14 UTC
tz3UoffC7FG7zfpmvmjUmUeAaHvzdcUvAj6r
98304
OUT
1
2
false
2018-09-11 00:12:14 UTC
tz2KuCcKSyMzs8wRJXzjqoHgojPkSUem8ZBS
98304
IN
18
Note: You can use above code to perform your quick tests of your credentials and the integrity of the data to insert.
I also make use of the following documentation:
Load Credentials from a file
Google.Cloud.BigQuery.V2
Load Json data into a new table
This is my hash map, it is not adding the actual generated id
val user = hashMapOf(
"deviceToker" to " ",
"dob" to date.text.toString(),
"email" to email_reg.text.toString(),
"id" to db.collection("patients").document().id,
"name" to name.text.toString()
)
If you want to add the randomly generated ID of a document to the contents of the document itself, first just call document() with no parameters to get a reference to the document that doesn't exist yet:
val ref = db.collection("patients").document()
Then use that reference to both add the ID to the document contents:
val user = hashMapOf(
"deviceToker" to " ",
"dob" to date.text.toString(),
"email" to email_reg.text.toString(),
"id" to ref.id, // note the use of ref.id here to get the random id
"name" to name.text.toString()
)
Now add the document using the same referece:
ref.set(user) // be sure to check for errors
As you can see, the ID is generated on the client, not on the server.
I think the procedure you are using isn't right. From the docs:
// Add a new document with a generated id.
val newCityRef = db.collection("cities").doc();
// later...
newCityRef.set(data);
So the way I go about it:
Create the ref to the doc first.
Then set the data into the doc ref.
I'm starting to use karate for testing. I need to validate one json response.
JSON Schema Design:
response{
id* Integer Not null
Name* String can be null
}
now i need to verified id and name with below constraints,
id should be integer and should not be null.
Name can either in string or can be null.
what equation we can use in Karate.
Thanks in Advances
def jsonValidate = {name: '#integer',Name: '#present'}
so if i use Present here ,it means Name can be null or can have value of any data type. but i need to check Name can be either String or Null Value only
Read the docs, and try this: https://github.com/intuit/karate#optional-fields
* def jsonValidate = { id: '#number', name: '##string' }
I use golang as application server. I do user-auth and I search storing system.
I have next model:
{
email string // it should be index
passwordhash string // it should be index too
token string
}
I tried to use key-value storage leveldb with 2 databases:
(key = "email", value = "passwordhash") for login user by passowrd
(key = "email", value = "token") for storing user's auth info
But I'm not sure that double email is good idea. Could you recommend me solution for storing auth info for golang?
Using email as a unique identifier is fine; you can just append a relevant string to your key to differentiate the key value, such as
(key = "email", value = "passwordhash") for login user by password
(key = "email:token", value = "token") for storing user's auth info
How to set Custom Menu Field values in Rightnow API of Oracle ?
I have a Custom field of data type Menu like :
Custom field Name : user type
Data Type : Menu
Value can be : Free, Paid or Premium
Can any one send me the java code by solving this problem?
Thanks in Advance
The following link is from the Oracle Service Cloud developer documentation. It has an example of setting a contact custom field using Java and Axis2, which would likely give you most of the information that you need in order to set your custom field.
At a high level, you must create an Incident object and specific the ID of the incident that you want to update. Then, you must create the custom field object structure using generic objects (because each site can have its own unique custom fields). Ultimately, your SOAP envelope will contain the node structure that you build through your java code. Since you're trying to set a menu, the end result is that your custom field is a NamedID object. You'll set the lookup name of the menu to one of the three values that you give above.
I'm a C# guy myself, so my example is in C#, but it should be easy to port to Java using the link above as an example too.
public static void SetMenuTest()
{
Incident incident = new Incident();
incident.ID = new ID();
incident.ID.id = 1234;
incident.ID.idSpecified = true;
GenericField customField = new GenericField();
customField.name = "user_type";
customField.dataType = DataTypeEnum.NAMED_ID;
customField.dataTypeSpecified = true;
customField.DataValue = new DataValue();
customField.DataValue.Items = new object[1];
customField.DataValue.ItemsElementName = new ItemsChoiceType[18]; //18 is a named ID value. Inspect ItemChoiceTypes for values.
customField.DataValue.Items[0] = "Free"; //Or Paid, or Premium
customField.DataValue.ItemsElementName[0] = ItemsChoiceType.NamedIDValue;
GenericObject customFieldsc = new GenericObject();
customFieldsc.GenericFields = new GenericField[1];
customFieldsc.GenericFields[0] = customField;
customFieldsc.ObjectType = new RNObjectType();
customFieldsc.ObjectType.TypeName = "IncidentCustomFieldsc";
GenericField cField = new GenericField();
cField.name = "c";
cField.dataType = DataTypeEnum.OBJECT;
cField.dataTypeSpecified = true;
cField.DataValue = new DataValue();
cField.DataValue.Items = new object[1];
cField.DataValue.Items[0] = customFieldsc;
cField.DataValue.ItemsElementName = new ItemsChoiceType[1];
cField.DataValue.ItemsElementName[0] = ItemsChoiceType.ObjectValue;
incident.CustomFields = new GenericObject();
incident.CustomFields.GenericFields = new GenericField[1];
incident.CustomFields.GenericFields[0] = cField;
incident.CustomFields.ObjectType = new RNObjectType();
incident.CustomFields.ObjectType.TypeName = "IncidentCustomFields";
}