I want the search with a field which match with any of the values in
look up table.
For now, I have used below where in query. But, I still want to query with Look up table instead of manually putting all those values in double quotes using the in clause.
|where in(search,"abcd","bcda","efsg","zyca");
First, you need to create a lookup field in the Splunk Lookup manager. Here you can specify a CSV file or KMZ file as the lookup. You will name the lookup definition here too. Be sure to share this lookup definition with the applications that will use it.
Once you have a lookup definition created, you can use it in a query with the Lookup Command. Say you named your lookup definition "my_lookup_csv", and your lookup column in your search is "event_column", and your csv column names are "column1", "column2", etc. Your search query will now end in:
| lookup my_lookup_csv column1 as event_column
Related
I have a some file in an azure data lake 2 and I want to load them as a column value nvarchar(max) in AzureSQLDW. The table in AzureSQLDW is heap. I couldn't find any way to do it? All I see is column delimited when load them into multiple rows instead of one row in single column. How I achieve this?
I don't guarantee this will work, but try using COPY INTO and define non-present values for row and column delimiters. Make your target a single column table.
I would create a Source Dataset with a single column. You do this by specifying "No delimiter":
Next, go to the "Schema" tab and Import the schema, which should create a single column called "Prop_0":
Now the data should come through as a single string instead of delimited columns.
I'm new to PDI and Kettle, and what I thought was a simple experiment to teach myself some basics has turned into a lot of frustration.
I want to check a database to see if a particular record exists (i.e. vendor). I would like to get the name of the vendor from reading a flat file (.CSV).
My first hurdle selecting only the vendor name from 8 fields in the CSV
The second hurdle is how to use that vendor name as a variable in a database query.
My third issue is what type of step to use for the database lookup.
I tried a dynamic SQL query, but I couldn't determine how to build the query using a variable, then how to pass the desired value to the variable.
The database table (VendorRatings) has 30 fields, one of which is vendor. The CSV also has 8 fields, one of which is also vendor.
My best effort was to use a dynamic query using:
SELECT * FROM VENDORRATINGS WHERE VENDOR = ?
How do I programmatically assign the desired value to "?" in the query? Specifically, how do I link the output of a specific field from Text File Input to the "vendor = ?" SQL query?
The best practice is a Stream lookup. For each record in the main flow (VendorRating) lookup in the reference file (the CSV) for the vendor details (lookup fields), based on its identifier (possibly its number or name or firstname+lastname).
First "hurdle" : Once the path of the csv file defined, press the Get field button.
It will take the first line as header to know the field names and explore the first 100 (customizable) record to determine the field types.
If the name is not on the first line, uncheck the Header row present, press the Get field button, and then change the name on the panel.
If there is more than one header row or other complexities, use the Text file input.
The same is valid for the lookup step: use the Get lookup field button and delete the fields you do not need.
Due to the fact that
There is at most one vendorrating per vendor.
You have to do something if there is no match.
I suggest the following flow:
Read the CSV and for each row look up in the table (i.e.: the lookup table is the SQL table rather that the CSV file). And put default upon not matching. I suggest something really visible like "--- NO MATCH ---".
Then, in case of no match, the filter redirect the flow to the alternative action (here: insert into the SQL table). Then the two flows and merged into the downstream flow.
hive table that has around 50 columns. Finding out a specific using Describe command or running select command is becoming tedious.
Is there a way we can search for existence of column in Hive table?
And also, can we use substring in the column name instead of complete name that will much more useful.
Thanks in Advance.
I believe there is no command to check this directly ,but you have below options.
1) Query column name , if you get NOT FOUND exception , the column doesn't exist.
2) Describe first , iterate through results and search for the column you are looking for.
3) Use Hive meta store client and get HiveMetastoreClient.getFields ,this will give list of columns ,iterate and search for your column
Is there a way to retrieve the field names from an actual query, similar to how you can retrieve field names from a table using the INFORMATION_SCHEMA? In essence, I'm wanting to accept an entire query as a parameter, and be able to build an empty table with the field names from said query.
Right now, I'm using a STUFF() function to replace the SELECT with SELECT TOP 1 and replacing the FROM with a INTO tablename FROM, and then truncating tablename. It works, but I need this to be able to handle complex queries where the first occurrence of FROM might not be the one I need (in case the user is using a subquery for a field, for example).
I was wondering if there was a way to search an entire SQLite database for one specific word. I do not know the column that it is in or even the table that it is in.
The table and row/column that contains this specific word also contains the other entries that i need to edit.
In-short:
Need to find a specific word
Can't query (i don't think i can atleast) since i don't know the table or column name that its located in.
I need to know where this specific word is referenced. In what table and row so I can access the others that are along side it.
Basically, is there a CTRL+F functionality of SQlite that searches the entirety of the SQLite file?
I have mac/windows/linux machines. I am not limited by software if that is a solution.
Any such functionality would essentially be running queries that check every column of every table. You can do that via a script that runs the following SQL:
1) Get a list of all the tables:
select name from sqlite_master where type = 'table'
2) For each table, get all of its columns (column name is available in the name field)
pragma table_info(cows)
3) Then for each table, generate a query that checks every field and run it:
select
*
from cows
where name like '%Daisy%'
or owner like '%Daisy%'
or farm like '%Daisy%'