Creating a model in nodejs without an ORM or ODM - sql

I am surprised why this is so hard to find.
I am trying to convert my monolith to a 3 layer architecture for my node express app with a propriety sql.
Repository
BEFORE architecture change
1 file entry.js
1 endpoint with business logic
Functions with raw sql that are called after validation of res.body objects
AFTER
📁 backend
📁 src
📁 services
📄 service1
📁 routes
📄 service1Route [ Route handling /service1 from entry.js ]
📁 models
-service1Model [ Contains sql functions, not schema ]
📄 entry.js [ Main express app]
Tech used
- Using .js not .ts
- nodejs
- express
-
Constraints
- I cannot use postgres or sequlize as they don't support the propriety db that I am using.
Assumptions
postgres or mongoose have popular ORMs and ODMs without which you cannot use them hence developers are ( as a good practice ) forced to create models.
Now I want to create my own models / schema with such validations.
Ask
How do I create models without ORM or ODM.
Is there a difference between schema and model ?
Writing sql functions in model folder : is that the right way to use this architectural pattern.
If schema/model is created in 📁 models folder then where do the sql queries reside ?
What I have tried?
For validating objects with required keys from res.body object

I'll have a go at answering. I'm a database engineer and have written node.js apps that connect directly to the database without using an ORM.
1: So long as you know the structure of the data that you wish to store as it resides in the database you can write a javascript class that has methods to do all of your updating and class instantiation etc. For instance, you can have a user class with a constructor and a number of methods for creating a new user in the database, retrieving a user from the database, updating a user etc. You can read more about javascript classes here and here.
So here your Model is just a class that knows how to interact with the database.
You could store all of your SQL here, but I would advise against that in favour of using Stored Procedures in the database. That way if you ever need to tune your query, or make changes, you can change just the stored procedure without having to create a whole release of your application. Your DBAs can also tinker round with your SPs for you as well.
2: Depends what you're referring to here. A schema in the database is a container for functionality, business areas, or whole applications. Like within a business you could have a Sales schema and a Marketing schema, or you could store all of your application logic in the MySalesApp schema.
A javascript model is one particular piece of functionality and its interactions with the database. Like a user, or a person, or an animal etc. In the database all of these objects (there would probably be a person/user/animal table, with a number of stored procedures for updating, creating, etc) would be stored inside a schema. In MySQL a schema can also be a whole database.
3: You could store your SQL there. But for the reasons mentioned before I'd do this with Stored Procedures. Basically because you can tune them for efficiency without having to redeploy your whole application.
4: This also answered by using Stored Procedures. So instead of having a bunch of code that creates a user, you have an SP that lives in the database and handles everything for you. Then you just end up with a call to a stored procedure that looks like this:
var query = "BB.sp_CreateUser";
var params = [
parseInt(user.userId),
user.firstName,
user.surname,
user.userInitials,
user.telephone,
user.username,
user.password
];
let result = await database.asyncQueryDynamic(query, params);
I've got a bit of abstraction going on here because I'm using connection pools etc. But you can read more about using Stored Procedures here.

Related

Is it possible to access an SQL database migrated using Django with a JavaScript ORM instead of raw SQL for a microservice architecture?

Suppose that I have an app where the models have been created using Django ORM and Django acts like an API for authentication and gives control to the relation model User using its ORM. At the same time, we want to use Express JS to continuously update a field in one the User model for some feature that requires performance. Is it possible to use a JavaScript ORM with Express JS to update such field? If yes, then how?
In the following tutorial tutorial for Golang, the database is created using SQL and a table called go_test_model is created. Then he uses the struct called GoTestModel to create a row in the table go_test_model.
This basically means that if we create an app called api in Django and add in it a model called Example, then to handle that model in Golang we just create a struct called ApiExample and from there we can have CRUD access to the same table, there might be some conflicts in the fields datatypes between GORM and Django ORM but integrity is still applied in the database itself.
So this particular example solves my problem with Golang and can be replicated using Node JS.
You can use Sequelize for using native functions for performing queries instead of writing raw queries.
Also, please refer to Models section to define models.

Can I populate database from Terraform script (for GCP)

I am a newby in terraform and working with GCP project.
I have following terraform code that creates a postgres database for an existing database instance
resource "google_sql_database" "db" {
name = "test_db1"
instance = "already_existing_instance"
project = var.project_id
}
And I would like to understand how (if possible) to run some SQL initial script for that newly created database with terraform script. SQL script should create a table and insert few records.
Terraform is not really designed for schema management, so in the long run you might be better to seek out a tool more specialized for that task (for example, so that it can potentially modify the schema later as your needs change).
If you do still want to take these actions with Terraform, you'd need to seek out a provider which supports the protocol for whichever database engine you've selected. For example, if you're using the Postgres database engine then you could potentially use the cyrilgdn/postgresql provider. (I've not used this provider myself, so I'm just giving this as an example and I can't vouch for it's suitability.)
However, providers like these typically go only as far as creating the object which contains tables (in Postgres terms, the "schema" object). To go more granular than that, creating tables and possibly data, you may not be able to achieve your goals with Terraform alone (unless you write your own provider), and may need to resort to using external software for that final setup step.

Can you or How can you make custom functions MongoDB like db.mydbname.customFunction()?

I'd like to know if you could write custom functions in MongoDB similar to a stored procedure for queries?
There aren't exactly stored procedures in MongoDB, but you can write Javascript functions that get stored in the DB (in the system.js collection).
Have a look at this article to get started.
There is a special system collection named system.js that can store JavaScript functions for reuse.
NOTE:
Do not store application logic in the database. There are performance
limitations to running JavaScript inside of MongoDB. Application code
also is typically most effective when it shares version control with
the application itself.
See The Reference.

Is there is something like stored procedures in NOSQL databases?

I am new to NOSQL world and still comparing between nosql and sql databases,
I Just tried making few samples using mongodb.
I am asking about stored procedures when we send few parameters to one stored procedure and this procedure execute number of other stored procedures in the database, will get data from stored procedures and send data to others.
In other words, will make the logic happen on the database side using sequence of functions and stored procedures.
Is that behavior or something the same already exist on NOSQL databases, or its completely different and i am thinking in the wrong way?
Mongo uses stored Javascript in a few places including Map/Reduce, db.eval and where clauses. Checkout this blog post for a survey:
Working With Stored JavaScript in MongoDB
The key to storing your functions on the server and making them available in these three contexts is db.system.js.save:
db.system.js.save( { _id : "foo" , value : function( x , y ){ return x + y; } } );
More details in the Mongo docs
Depends whether you define "NOSQL" as "No SQL" or "Not Only SQL."
OpenLink Virtuoso [1] (produced by my employer) is the latter, and has stored procedures, and SPARQL-in-SQL, and SQL-in-SPARQL, among other neat tricks; including extensibility via in-process runtime hosting (Perl, PHP, Python, Ruby, JSP, and others), external libraries or helper apps, and more.
Other NoSQL DBs have other ways of handling this, such as Mongo's extensibility via JavaScript, described in that other answer.
[1] http://virtuoso.openlinksw.com/

What is the best way to create an enhanced Data Dictionary?

Without going into specifics...I have a large SQL Server 2005 database with umpteen stored-procedures.
I have multiple applications from WinForm apps to WebServices all of which use this DB.
My simple objective now is to create a meta-database...a prospective data-dictionary where I can maintain details of which specific app. file uses which SP.
For example, My application Alpha which has a file Beta.aspx...uses 3 SPs which are physically configured for usage in BetaDAL.cs
You might have inferred by now,it will make life easier for me later when there is a migration or deprecation....where I can just query this DB SP-wise to get all Apps/Files that use the DB or vice-versa.
I can establish this as a single de-normalized table..or structure it in a better way.
Does some schema already exist for this purpose?
SQL Server supports what are called extended properties, basically a key-value dictionary attached to every object in the catalog. You can add whatever custom information about the catalog (comments on tables, columns, stored procedures, ...) you wish to store as extended properties and query them along with the normal catalog views.
Here's one overview (written for SQL Server 2005, but roughly the same techniques should apply for 2000 or 2008).