database writes need to be verified in an integration test? - testing

I basically have an api that writes to 2 different tables, and am using supertest to make api calls to write a certain data to the database e.g. POST /userCertificates writes his address in the address table, education in status table like so:
I basically do this as one of the integration tests:
payload = {user_id: 1, name: abc, address: {a: x, b: y}, education: {a: x, b: y} ...}
it('Post to the database'), ()=>{
request.post(/userCertificate).send(payload).expect(200)
.then(res =>
expect(payload.userAddress).toEqual(res.body.userAddress)
// more expects to verify the response received)
}
The /userCertificates api basically does this:
creates a new user with name and user_id
saves data of address to the address table and establishes FK relation.
saves data of education to the education table and establishes FK
relation.
But, the real confusion is, do I also need to use a dao layer that directly talks with the database to absolutely verify that the data has been written to the 2 tables? E.g.
for my case above(in the same describe block):
NOTE: user_id is a foreign key in the address and education table
it('Verify data written to the address table', ()=> {
const data = AddressDao.find({user_id: 1})
if(data.length <= 0) {
throw 'Data not written to the database
}
}
it('Verify data written to the education table', ()=> {
const data = EducationDao.find({user_id: 1})
if(data.length <= 0) {
throw 'Data not written to the database
}
}
But, not just in the case of writing to multiple tables, is such verification of data
written to the database required?

Related

FaunaDB: how to fetch a custom column

I'm just learning FaunaDB and FQL and having some trouble (mainly because I come from MySQL). I can successfully query a table (eg: users) and fetch a specific user. This user has a property users.expiry_date which is a faunadb Time() type.
What I would like to do is know if this date has expired by using the function LT(Now(), users.expiry_date), but I don't know how to create this query. Do I have to create an Index first?
So in short, just fetching one of the users documents gets me this:
{
id: 1,
username: 'test',
expiry_date: Time("2022-01-10T16:01:47.394Z")
}
But I would like to get this:
{
id: 1,
username: 'test',
expiry_date: Time("2022-01-10T16:01:47.394Z"),
has_expired: true,
}
I have this FQL query now (ignore oauthInfo):
Query(
Let(
{
oauthInfo: Select(['data'], Get(Ref(Collection('user_oauth_info'), refId))),
user: Select(['data'], Get(Select(['user_id'], Var('oauthInfo'))))
},
Merge({ oauthInfo: Var('oauthInfo') }, { user: Var('user') })
)
)
How would I do the equivalent of the mySQL query SELECT users.*, IF(users.expiry_date < NOW(), 1, 0) as is_expired FROM users in FQL?
Your use of Let and Merge show that you are thinking about FQL in a good way. These are functions that can go a long way to making your queries more organized and readable!
I will start with some notes, but they will be relevant to the final answer, so please stick with me.
The Query function
https://docs.fauna.com/fauna/current/api/fql/functions/query
First, you should not need to wrap anything in the Query function, here. Query is necessary for defining functions in FQL that will be run later, for example, in the User-Defined Function body. You will always see it as Query(Lambda(...)).
Fauna IDs
https://docs.fauna.com/fauna/current/learn/understanding/documents
Remember that Fauna assigns unique IDs for every Document for you. When I see fields named id, that is a bit of a red flag, so I want to highlight that. There are plenty of reasons that you might store some business-ID in a Document, but be sure that you need it.
Getting an ID
A Document in Fauna is shaped like:
{
ref: Ref(Collection("users"), "101"), // <-- "id" is 101
ts: 1641508095450000,
data: { /* ... */ }
}
In the JS driver you can use this id by using documentResult.ref.id (other drivers can do this in similar ways)
You can access the ID directly in FQL as well. You use the Select function.
Let(
{
user: Get(Select(['user_id'], Var('oauthInfo')))
id: Select(["ref", "id"], Var("user"))
},
Var("id")
)
More about the Select function.
https://docs.fauna.com/fauna/current/api/fql/functions/select
You are already using Select and that's the function you are looking for. It's what you use to grab any piece of an object or array.
Here's a contrived example that gets the zip code for the 3rd user in the Collection:
Let(
{
page: Paginate(Documents(Collection("user")),
},
Select(["data", 2, "data", "address", "zip"], Var("user"))
)
Bring it together
That said, your Let function is a great start. Let's break things down into smaller steps.
Let(
{
oauthInfo_ref: Ref(Collection('user_oauth_info'), refId)
oauthInfo_doc: Get(Var("oathInfoRef")),
// make sure that user_oath_info.user_id is a full Ref, not just a number
user_ref: Select(["data", "user_id"], Var("oauthInfo_doc"))
user_doc: Get(Var("user_ref")),
user_id: Select("id", Var("user_ref")),
// calculate expired
expiry_date: Select(["data", "expiry_date"], Var("user_doc")),
has_expired: LT(Now(), Var("expiry_date"))
},
// if the data does not overlap, Merge is not required.
// you can build plain objects in FQL
{
oauthInfo: Var("oauthInfo_doc"), // entire Document
user: Var("user_doc"), // entire Document
has_expired: Var("has_expired") // an extra field
}
)
Instead of returning the auth info and user as separate points if you do want to Merge them and/or add additional fields, then feel free to do that
// ...
Merge(
Select("data", Var("user_doc")), // just the data
{
user_id: Var("user_id"), // added field
has_expired: Var("has_expired") // added field
}
)
)

TypeORM cannot delete row with ManyToOne / OneToMany relation

I have this problem right now that I don't know how to fix honestly. I spent hours on this already and cannot find the solution. I am using MS-SQL on Azure.
The way I have set up my entities is the following:
Customer and Visits: OneToMany (Primary)
Visits and Customers: ManyToOne (Inverse)
I am soft-deleting my customers, so that the information for the visits can be retrieved regardless of whether or not the user wants to see the customer data specifically. The data is still getting resolved correctly using the relationship. That's also why I don't want to use "Cascade DELETE" here.
However, since I want to delete visits completely (not soft-deleting like the customers) I am facing issues probably regarding foreign key constraints (not sure, because I don't get any error output from TypeORM). The DeleteResult.affected property however returns 0, which is what I see in my DataGrip queries as well, where I check the actual table data.
Whats important as well is that I am able to manually delete the row using a simple SQL statement like the following:
DELETE FROM visits
WHERE uuid = 'f0ea300d-...-656a'
My entities are set up like this (left unimportant information out):
#Entity({ name: 'customers' })
export class Customer {
#PrimaryColumn()
uuid: string
#OneToMany(() => Visit, (visit) => visit.customer)
visits?: Visit[]
}
#Entity({ name: 'visits' })
export class Visit {
#PrimaryColumn()
uuid: string
#ManyToOne(() => Customer, (customer) => customer.visits)
customer: Customer
}
My GraphQL resolver:
#Mutation(() => Boolean)
async deleteVisitsByUuid(
#Arg('uuid') uuid: string,
#Ctx() { conn }: AppContext,
): Promise<boolean> {
const repo = conn.getRepository(Customer)
const result = await repo.delete(uuid)
const affected = result.affected
if (affected === undefined || affected == null) {
return false
} else {
return affected > 0
}
}
The problem was conn.getRepository(Customer). I have replaced it with conn.getRepository(Visit).

Duplicate console.log output of gun map when using gundb

Does this function duplicate results as a bug or am I causing this? The output always has 1 or more records duplicated. In this example, Bank of China is always listed twice in output.
gun.get('savings_accounts').map(function (name, ID) {
console.log( name.name, ID );
}, true)
My code:
localStorage.clear();
var gun = Gun();
////////////////////////////////////////////////////// create record
var acc1 = gun.put({
name: "Bank of America",
accType: "Savings",
last4: "4123",
favorite: true,
status: true,
created: "some date created"
});
var acc2 = gun.put({
name: "Bank of China",
accType: "Savings",
last4: "5123",
favorite: true,
status: true,
created: "some date created"
});
gun.get('savings_accounts').map(function (name, ID) {
console.log( name.name, ID );
}, true)
From the author of GunDB, Mark Nadal
1) gun.get('savings_accounts').map().val(cb) is what you want for normal / procedural / easy things.
HOWEVER...
2) gun is actually functional/reactive (FRP), or also known as streaming/event oriented. The data might/will get called multiple times (if you don't use .val) because A) in-memory replies, B) your browser's localStorage replies, C) the server will reply, D) server will relay to other browser peers which each might reply with data.
^ that is the "realtime" part of gun.
.val only fires once (well per item on the chain, so if you do map().val(cb) the val will get fired multiple times but only once from each item in the list).
use .val(cb) if you are doing procedural things.
Use .on(cb) (which is what .map(cb) uses internally. Most API methods internally use .on) if you want to subscribe to realtime updates of the data.
you'll slowly find that the realtime/FRP/event/streaming as being a much cleaner way to write your apps.

RavenDB Get document count after BulkInsertOperations

I am using RavenDB to bulk load some documents. Is there a way to get the count of documents loaded into the database?
For insert operations I am doing:
BulkInsertOperation _bulk = docStore.BulkInsert(null,
new BulkInsertOptions{ CheckForUpdates = true});
foreach(MyDocument myDoc in docCollection)
_bulk.Store(myDoc);
_bulk.Dispose();
And right after that I call the following:
session.Query<MyDocument>().Count();
but I always get a number which is less than the count I see in raven studio.
By default, the query you are doing limits to a sane number of results, part of RavenDB's promise to be safe by default and not stream back millions of records.
In order to get the number of a specific type of document in yoru database, you need a special map-reduce index whose job it is to track the counts for each document type. Because this type of index deals directly with document metadata, it's easier to define this in Raven Studio instead of trying to create it with code.
The source for that index is in this question but I'll copy it here:
// Index Name: Raven/DocumentCollections
// Map Query
from doc in docs
let Name = doc["#metadata"]["Raven-Entity-Name"]
where Name != null
select new { Name , Count = 1}
// Reduce Query
from result in results
group result by result.Name into g
select new { Name = g.Key, Count = g.Sum(x=>x.Count) }
Then to access it in your code you would need a class that mimics the structure of the anonymous type created by both the Map and Reduce queries:
public class Collection
{
public string Name { get; set; }
public int Count { get; set; }
}
Then, as Ayende notes in the answer to the previously linked question, you can get results from the index like this:
session.Query<Collection>("Raven/DocumentCollections")
.Where(x => x.Name == "MyDocument")
.FirstOrDefault();
Keep in mind, however, that indexes are updated asynchronously so after bulk-inserting a bunch of documents, the index may be stale. You can force it to wait by adding .Customize(x => x.WaitForNonStaleResults()) right after the .Query(...).
Raven Studio actually gets this data from the index Raven/DocumentsByEntityName which exists for every database, by sidestepping normal queries and getting metadata on the index. You can emulate that like this:
QueryResult result = docStore.DatabaseCommands.Query("Raven/DocumentsByEntityName",
new Raven.Abstractions.Data.IndexQuery
{
Query = "Tag:MyDocument",
PageSize = 0
},
includes: null,
metadataOnly: true);
var totalDocsOfType = result.TotalResults;
That QueryResult contains a lot of useful data:
{
Results: [ ],
Includes: [ ],
IsStale: false,
IndexTimestamp: "2013-11-08T15:51:25.6463491Z",
TotalResults: 3,
SkippedResults: 0,
IndexName: "Raven/DocumentsByEntityName",
IndexEtag: "01000000-0000-0040-0000-00000000000B",
ResultEtag: "BA222B85-627A-FABE-DC7C-3CBC968124DE",
Highlightings: { },
NonAuthoritativeInformation: false,
LastQueryTime: "2014-02-06T18:12:56.1990451Z",
DurationMilliseconds: 1
}
A lot of that is the same data you get on any query if you request statistics, like this:
RavenQueryStatistics stats;
Session.Query<Course>()
.Statistics(out stats)
// Rest of query

PouchDB Query like sql

with CouchDB is possible do queries "like" SQL. http://guide.couchdb.org/draft/cookbook.html says that
How you would do this in SQL:
SELECT field FROM table WHERE value="searchterm"
How you can do this in CouchDB:
Use case: get a result (which can be a record or set of records) associated with a key ("searchterm").
To look something up quickly, regardless of the storage mechanism, an index is needed. An index is a data structure optimized for quick search and retrieval. CouchDB’s map result is stored in such an index, which happens to be a B+ tree.
To look up a value by "searchterm", we need to put all values into the key of a view. All we need is a simple map function:
function(doc) {
if(doc.value) {
emit(doc.value, null);
}
}
This creates a list of documents that have a value field sorted by the data in the value field. To find all the records that match "searchterm", we query the view and specify the search term as a query parameter:
/database/_design/application/_view/viewname?key="searchterm"
how can I do this with PouchDB? the API provide methods to create temp view, but how I can personalize the get request with key="searchterm"?
You just add your attribute settings to the options object:
var searchterm = "boop";
db.query({map: function(doc) {
if(doc.value) {
emit(doc.value, null);
}
}, { key: searchterm }, function(err, res) { ... });
see http://pouchdb.com/api.html#query_database for more info
using regex
import PouchDB from 'pouchdb';
import PouchDBFind from 'pouchdb-find';
...
PouchDB.plugin(PouchDBFind)
const db = new PouchDB(dbName);
db.createIndex({index: {fields: ['description']}})
....
const {docs, warning} = await db.find({selector: { description: { $regex: /OVO/}}})