I'm looking to return the id or better yet, all information that was inserted, using a raw query with TypeORM and NestJS. Example as follows:
await connection.manager.query(`INSERT INTO...`)
When assigning the query to a constant and console logging it below, it does not yield any helpful information:
OkPacket {
fieldCount: 0,
affectedRows: 1,
insertId: 0,
serverStatus: 2,
warningCount: 1,
message: '',
protocol41: true,
changedRows: 0
}
As you can see, it returns no pertinent information, the insertId above is obviously incorrect, and it returns this every time, regardless of the actual parameters of the query.
I know with more typical TypeORM queries you can use .return(['name_of_column_you_want_returned']).execute()
and it will return the relevant information just fine. Is there any way to do this with a raw query? Thank you!
tl;dr You're getting the raw mariadb driver response (OkPacket) from the INSERT command, and you'd need a new SELECT query to see the data.
You're using the TypeORM EntityManager, and the docs don't mention a return value. Looking at the source code for query, the return type is any. Since it's a raw query, it probably returns an object based on the type of database you're using rather than having a standard format.
In this case, you're using MariaDb, which returned an OkPacket. Here's the documentation:
https://mariadb.com/kb/en/ok_packet/
I am trying to do a bulk insert where I have to reference a variable in the payload.
<db:sql >#['INSERT INTO public."PERSON_TBL" (FIRST_NAME,LAST_NAME, ADDRESSES)
VALUES
(:firstnameParam,
:lastnameParam,
' ++ "'$(someVariable)'" ++ ')']
</db:sql>
In the example above, "ADDRESSES" is a DB column of type 'text[]'. Because of this I have to pass my array as a string something that looks like '{"Address1", "Address2"}'. The problem comes when this is a bulk insert and "someVariable" is just one of many within the payload. The question is: How do I set the exact "someVariable" for the current step. How can I get a reference to that? Is it something like
$(payload[currentIndex].someVariable)
Please help with this.
I'm working on a personal project and very new (learning as I go) to JSON, NiFi, SQL, etc., so forgive any confusing language used here or a potentially really obvious solution. I can clarify as needed.
I need to take the JSON output from a website's API call and insert it into a table in my MariaDB local server that I've set up. The issue is that the JSON data is nested, and two of the key pieces of data that I need to insert are used as variable key objects rather than values, so I don't know how to extract it and put it in the database table. Essentially, I think I need to identify different pieces of the JSON expression and insert them as values, but I'm clueless how to do so.
I've played around with the EvaluateJSON, SplitJSON, and FlattenJSON processors in particular, but I can't make it work. All I can ever do is get the result of the whole expression, rather than each piece of it.
{"5381":{"wind_speed":4.0,"tm_st_snp":26.0,"tm_off_snp":74.0,"tm_def_snp":63.0,"temperature":58.0,"st_snp":8.0,"punts":4.0,"punt_yds":178.0,"punt_lng":55.0,"punt_in_20":1.0,"punt_avg":44.5,"humidity":47.0,"gp":1.0,"gms_active":1.0},
"1023":{"wind_speed":4.0,"tm_st_snp":26.0,"tm_off_snp":82.0,"tm_def_snp":56.0,"temperature":74.0,"off_snp":82.0,"humidity":66.0,"gs":1.0,"gp":1.0,"gms_active":1.0},
"5300":{"wind_speed":17.0,"tm_st_snp":27.0,"tm_off_snp":80.0,"tm_def_snp":64.0,"temperature":64.0,"st_snp":21.0,"pts_std":9.0,"pts_ppr":9.0,"pts_half_ppr":9.0,"idp_tkl_solo":4.0,"idp_tkl_loss":1.0,"idp_tkl":4.0,"idp_sack":1.0,"idp_qb_hit":2.0,"humidity":100.0,"gp":1.0,"gms_active":1.0,"def_snp":23.0},
"608":{"wind_speed":6.0,"tm_st_snp":20.0,"tm_off_snp":53.0,"tm_def_snp":79.0,"temperature":88.0,"st_snp":4.0,"pts_std":5.5,"pts_ppr":5.5,"pts_half_ppr":5.5,"idp_tkl_solo":4.0,"idp_tkl_loss":1.0,"idp_tkl_ast":1.0,"idp_tkl":5.0,"humidity":78.0,"gs":1.0,"gp":1.0,"gms_active":1.0,"def_snp":56.0},
"3396":{"wind_speed":6.0,"tm_st_snp":20.0,"tm_off_snp":60.0,"tm_def_snp":70.0,"temperature":63.0,"st_snp":19.0,"off_snp":13.0,"humidity":100.0,"gp":1.0,"gms_active":1.0}}
This is a snapshot of an output with a couple thousand lines. Each of the numeric keys that you see above (5381, 1023, 5300, etc) are player IDs for the following stats. I have a table set up with three columns: Player ID, Stat ID, and Stat Value. For example, I need that first snippet to be inserted into my table as such:
Player ID Stat ID Stat Value
5381 wind_speed 4.0
5381 tm_st_snp 26.0
5381 tm_off_snp 74.0
And so on, for each piece of data. But I don't know how to have NiFi select the right pieces of data to insert in the right columns.
I believe that it's possible to use jolt to transform your json into a format:
[
{"playerId":"5381", "statId":"wind_speed", "statValue": 0.123},
{"playerId":"5381", "statId":"tm_st_snp", "statValue": 0.456},
...
]
then use PutDatabaseRecord with json reader.
Another approach is to use ExecuteGroovyScript processor.
Add new parameter to it with name SQL.mydb and link it to your DBCP controller service
And use the following script as Script Body parameter:
import groovy.json.JsonSlurper
import groovy.json.JsonBuilder
def ff=session.get()
if(!ff)return
//read flow file content and parse it
def body = ff.read().withReader("UTF-8"){reader->
new JsonSlurper().parse(reader)
}
def results = []
//use defined sql connection to create a batch
SQL.mydb.withTransaction{
def cmd = 'insert into mytable(playerId, statId, statValue) values(?,?,?)'
results = SQL.mydb.withBatch(100, cmd){statement->
//run through all keys/subkeys in flow file body
body.each{pid,keys->
keys.each{k,v->
statement.addBatch(pid,k,v)
}
}
}
}
//write results as a new flow file content
ff.write("UTF-8"){writer->
new JsonBuilder(results).writeTo(writer)
}
//transfer to success
REL_SUCCESS << ff
i have saved one json using Rejson against a key,now i would like to filter/query out data using ReJson.
Please let me know how can i do it ...python prefered .
print("Abount to execute coomnad")
response=redisClient.execute_command('JSON.SET', 'object', '.', json.dumps(data))
print(response)
reply = json.loads(redisClient.execute_command('JSON.GET', 'object'))
print(reply)
using the above code i was able to set data using ReJson .now lets suppose i want to filer data .
my test json is :
data = {
'foo': 'bar',
'ans': 42
}
How can you filter say json in which foo has value as bar
Redis in general, and ReJSON specifically, do not provide search-by-value functionality. For that, you'll have to either index the values yourself (see https://redis.io/topics/indexes) or use RediSearch.
I have a PG table with a field of type char(10)[].
I need to update a record in the table with values from a Mule flow.
So, i did something like this:
flowVars.test=['aaa', 'bbb',ccc'];
Then, I'm trying to submit an update statement like this:
update tab1 set fld1=#[flowVars.test]
it's failing with the error:
Cannot cast an instance of java.util.ArrayList to type Types.ARRAY
My understanding is that SQL array should be used in this scenario but I can't figure out how to get an instance of such an array in a flow and how to work with it in MEL.
Can someone please advise?
Thank you,
There are many sources that suggest to use the Connection#createArrayOf(). But I don't know how to use it in the Database connector.
However, for this purpose I will do this solution:
Convert the ArrayList to a String. It should be formed as: {value1, value2, ...}
Change the Database Query Type from Parameterized into Dynamic
Update the SQL Query become: update tab1 set fld1 = '#[flowVars.test]'. The additional single quote is required for this query type.
Finally, by using the following configuration I can update field of type character(10)[]:
<expression-transformer expression="#[flowVars.test = ['aaa', 'bbb', 'ccc'].toString().replace('[', '{').replace(']', '}')]" doc:name="Expression"/>
<db:update config-ref="Postgre_Database_Configuration" doc:name="Database">
<db:dynamic-query><![CDATA[update tab1 set fld1 = '#[flowVars.test]']]></db:dynamic-query>
</db:update>
Ok, I've found an answer in MuleSoft doc.
Starting from version 3.6 DB connector supports custom types and allows defining mapping between SQL arrays and structures and custom user classes.
It's documented here .