I know mysql and realm are different structures but,
Is there a way to quickly insert 1 million records like LOAD DATA INFILE as in mysql for realm on react native?
The api returning 1 million records and it takes 25 minutes to write schema.
My method:
Realm.open({ schema: [EnvanterSchema] }).then(realm => {
realm.write(() => {
response.data.forEach(obj => {
realm.create("Inventory", obj);
});
});
data = [{
barcode: "25836693",
Item: "Men Shoes",
Brand: "brand",
Qty: "x",
Size: "L"
},....]
react native: 0.57.0
realm: 2.16.0
Related
I have been stuck with this and similar dilemmas for a while now and can't seem to find an acceptable solution. Prisma currently doesn't allow for upsertMany in it's queries, or in my case a nested upsertMany. I have found some accepted solutions and the most popular one is using $transaction from prisma to write a loop of upserts which doesn't work for me as each query contains up to 200 items to upsert (currently testing 100 items takes roughly 10sec to run so it really isn't an acceptable solution). Prisma offers the ability to use $queryRaw to use a raw SQL query which may offer a more acceptable solution. I've supplied the code to their currently accepted solution as well as the query (that isn't currently possible) that I'm hoping someone could help me turn into a raw sql query or a better solution.
Their accepted solution:
prisma.$transaction(data.map((d) =>
prisma.MODEL.upsert({
where:{
id: d.id
},
update:{
foo:d.foo,
},
create:{
foo: d.foo,
bar: d.bar
},
}),
),
);
How I would need to implement it as I would like to create one MODEL with a related array of MODELS and another array of related MODELS:
let transactionOne = prisma.$transaction(
arrayOne.map((d)=>
prisma.MODEL.upsert({
where:{
id: d.id,
},
update:{
foo: d.foo,
},
create:{
foo: d.foo,
bar: d.bar,
},
}),
)
);
let transactionTwo = prisma.$transaction(
arrayTwo.map((d)=>
prisma.MODEL.upsert({
where:{
id: d.id,
},
update:{
foo: d.foo,
},
create:{
foo: d.foo,
bar: d.bar,
},
}),
)
);
let data = await prisma.MODEL.create({
data:{
foo: foo,
bar: bar,
example:{
connect: transactionOne.map((a)=>({id: a.id})),
},
exampleTwo:{
connect: transactionTwo.map((b)=>({id: b.id})),
},
},
});
My example above like mentioned before takes just over 10 seconds with 100 items and may possibly require 200 items so 20 seconds per call. Is there an easier way to make this faster or like I said a way to create a raw SQL query with the same results?
Thanks for any help!
I'm trying to write a typeORM query which includes multiple where clauses. I am able to achieve this via where option as follows:
const categories = [
{ name: 'master', categoryTypeId: 2, parentId: 1, locationId: null },
{ name: 'food', categoryTypeId: 3, parentId: null, locationId: null }];
const rows = await Repo.find({
where: categories.map((category) => ({
name: category.name,
categoryTypeId: category.categoryTypeId,
locationId: category.locationId
})),
});
I would want to maintain the mapping b/w the input array and the rows returned. For example, I know that the second category doesn't exist in the DB. I would want to have an empty object in the rows variable so that I know which categories didn't yield any result.
Upon research I have found that we can do something with SQL as mentioned here. But, I'm not sure how do I translate into typeORM if I can.
I'm working with on nodejs with knex.js and sql as a database I have a record of 4000 data in array of object I have to update it into a table, which is currently taking more than 10 minutes. I wanted to reduce the time taken.
this is the records
const records = [
{ ID: "45159", UnitCost: "1", country: "GB" },
{ ID: "45209", UnitCost: "2", country: "GB" },
];
currently I'm mapping through each record and writing this condition
const response = await baseQueryBuilder
.clone()
.where("country", countryCode)
.where({ID:id})
.update({UnitCost:UnitCost});
is there any method that we can do batch or bulk upload so that the time can be reduced. Thanks.
How do i push my results into a data object please
I would like to push the :key to the id field and the :data to map across to my other data fields
my data object is called orgs: []
I have the following data returned from my get query
[{…}]
0:
data: {person: 'John', orgName: 'test', due: 'this is due'}
key: "11ed25ccf7548160bff2ab4c1d56ee40"
I have tried
this.$db.collection('orgs').get({ keys: true })
.then
(response => {
console.log('orgs: ', response) <--- this displays the results
this.orgs.forEach(response => {
const data = {
'person': response.data.person,
'orgName': response.data.orgName,
'due': response.data.due,
'id': response.key
}
this.orgs.push(data)
})
});
I get no errors but my this.orgs data item remains empty so assume I am not updating the array for some reason
Thanks
I seem to be able to populate it in a simple way as follows
this.$db.collection('orgs').get({ keys: true })
.then
(response => {
this.orgs = response
console.log('response:', this.orgs)
})
Is there any issue with this approach for just getting my results or should I be using push?
I have followed the doc for the ReferenceArrayInput (https://marmelab.com/react-admin/Inputs.html#common-input-props) but it does not seem to be working with relationship fields.
For example, I have this many-to-many relation for my Users (serialized version) :
Coming from (raw response from my API):
I have setup the ReferenceArrayInput as followed :
<ReferenceArrayInput source="profiles" reference="profiles" >
<SelectArrayInput optionText="label" />
</ReferenceArrayInput>
I think it's making the appropriate calls :
But here is my result :
Any idea what I'm doing wrong ?
Thanks in advance for your help !
On docs, ReferenceArrayInput is said to expect a source prop pointing to an array os ids, array of primitive types, and not array of objects with id. Looks like you are already transforming your raw response from api, so if you could transform a bit more, mapping [{id}] to [id], it could work.
If other parts of your app expects profiles to be an array of objects, just create a new object entry like profilesIds or _profiles.
As gstvg said, ReferenceArrayInput expects an array of primitive type, not array of objects.
If your current record is like below:
{
"id": 1,
"tags": [
{ id: 'programming', name: 'Programming' },
{ id: 'lifestyle', name: 'Lifestyle' }
]
}
And you have a resource /tags, which returns all tags like:
[
{ id: 'programming', name: 'Programming' },
{ id: 'lifestyle', name: 'Lifestyle' },
{ id: 'photography', name: 'Photography' }
]
Then you can do something like this (it will select the tags of current record)
<ReferenceArrayInput
reference="tags"
source="tags"
parse={(value) => value && value.map((v) => ({ id: v }))}
format={(value) => value && value.map((v) => v.id)}
>
<AutocompleteArrayInput />
</ReferenceArrayInput>