Related
I've been spending a considerable amount of time learning MongoDB as part of the MEAN stack (I'm new to the MEAN stack), and I feel like I'm just barely starting to get the hang of it. Recently, however, my supervisor at work (who's an experienced programmer) suggested I learn relational databases (it could be that it's used by more companies), and I have taken his suggestion to heart since I'm only a junior developer, and would like to move up soon. My personal goal/project is to build a social networking site where one group of users have the ability to search for another group of users (and most likely charge a membership fee). Would the database matter for a project like this? I would love to kill two birds with one stone by learning something (and learning it really well) that could be used in my personal project, and in a professional job.
I'm certainly open to learning more than one language, but I need a starting point, so I need something that will help me accomplish my personal goal. Since I've spent a considerable amount of time with JavaScript (as oppose to, say, Python), having to learn relational databases concerns me. Are there any downsides to just sticking to MongoDB?
Thanks in advance!
The first thing you need is to assess what type of data you are going to deal with (will it be structured, easily represented by tables, or it will be more of the non-structured type).
If structured, RDBMS is the way to go.
Incidentally, as far as I know, the first versions of Facebook were implemented using MySQL (the type of DBMS your boss was referring to).
Farid, as you're a junior programmer, I'd suggest you to learn as much as possible of both approach.
The knowledge acquired on one of them will also help you getting the most from the other (as they both concern data, how to structure it and how to query it).
Also, in the day by day job (unless you specialize only in large sites where scalability is the first concern) you might encounter RDBMS more frequently than NoSQL choices.
Both technologies have pro and cons and, unless you know both sides (at least documenting yourself, if not by direct experience) you might go for one solution when the other could have been preferable.
Well, anyone have concern about learn SQL (and use JOIN), but:
One day, you don't know how, you will use it (or understand it), so perhaps is better take a moment for SQL (it's the most widespread)
MongoDB is good if you development a REST service, but if you want make a social network like project, perhaps is better look at Graph database.
Perhaps that video can help, is an overview of SQL vs NoSQL.
I'm looking at the documentation for the Play framework and it appears as though SQL is being written in the framework.
Coming from Rails, I know this as a bad practice. Mainly because developers require the need to switch databases as they scale.
What are the practices in Play to allow for developers to implement conventions and to work with databases without having to hard code SQL?
One of the feature/defects (depending on who you ask) of Play is the there is no ORM. (An ORM is an object-relational mapper/mapping; it is the part of Rails, Django, etc. that writes your SQL for you.)
Pro-ORM: You don't have to write any SQL.
Ease-of-use: Some developers unused to SQL will find this easier.
Code resuse: Tables are usually based on your classes; there is less duplication of code.
Portability: ORMs try smooth over any differences between DMBS vendors.
No-ORM: You get to write your own SQL, and not rely on unseen ORM (black)magic.
Performance: I work for a company which produces high-traffic web applications. With millions of visitors, you need to know exactly what queries you are running, and exactly what indicies you are using. Otherwise, the code that worked so well in dev will crash production.
Flexibility: ORMs often do not have the full range of expression that a domain-specific language like SQL does. Some more complex sub-selects and aggregation queries will be difficult/impossible to write with ORMs.
While you may think "developers require the need to switch databases as they scale", if you scale enough to change your DBMS, changing query syntax will be the least of your scalability issues. Often, the queries themselves will have to be rewritten to use sharding, etc., at which point the ORM is dead.
It is a tradeoff; one that in my experience has often favored no ORM.
See the Anorm page for Play's justification of this decision:
You don’t need another DSL to access relational databases
SQL is already the best DSL for accessing relational databases. We don’t need to invent something new.
...
Play developers will typically write their own SQL (much the same way they will write in other languages, like HTML), use Anorm, and follow common SQL conventions.
If portability is a requirement, use only ANSI SQL (no vendor-specific featues). It is generally well supported.
EDIT: Or if you are really open-minded, you might have a look at NoSQL databases, like Mongo. They are inherently object-based, so object-oriented Ruby/Python/Scala can be used as the API naturally.
In addition to Paul Draper's excellent answer, this post is meant to tell you about what Play developers usually do in practice.
TL;DR: use Slick
Play is less opinionated than Rails and gives the user many more choices for their data storage backend. Many people use Play as a web layer for very complex existing backend systems. Many people use Play with a NoSQL storage backend (e.g. MongoDB). Then there's also people using Play for traditional web-service-with-SQL applications. Generalizing a bit too much, one can recognize two people using Play with relational databases.
"Traditional web developers"
They are used to standard Java technologies or are part of an organization that uses them. The Java Persistence API and its implementations (Hibernate, EclipseLink, etc...) are their ORM. You can do so too. There are also appear to be Scala ORMs, but I'm less familiar with those.
Note that Java/Scala ORMs are still different ORMs in style when compared to Rails' ActiveRecord. Ruby is a dynamic language that allows/promotes loads of monkey patching and method_missing stuff, so there is MyObject.finder_that_I_didnt_necessarily_have_to_write_myself(). This style of ORM is called the active record pattern. This style is impossible to accomplish in pure Java and discouraged in Scala (as it violates type safety), so you have to get used to writing a more traditional style using service layers and data access objects.
"Scala web developers"
Many Scala people think that ORMs are a bad abstraction for database access. They also agree that using raw SQL is a bad idea, and that an abstraction is still in order. Luckily Scala is an expressive compiled language, and people have found a way to abstract database access that does not rely on the object oriented language paradigm as ORMs do, but mostly on the functional language paradigm. This is quite a similar idea to the LINQ query language Microsoft has made for their .NET framework languages. The core idea is that you don't have an ORM to perform query magic, nor write queries in SQL, but write them in Scala itself. The advantages of this approach are twofold:
You get a more fine grained control over what your queries actually execute when compared to ORMs.
Queries are checked for validity by the Scala compiler, so you don't get runtime errors for invalid SQL you wrote yourself. If it is valid Scala, it is translated to a valid SQL statement for you.
Two major libraries exist for accomplishing this. The first is Squeryl. The second is Slick. Slick appears to be the most popular one, and there are some examples floating around the web that show how you are supposed to make it work with Play. Also check out this video that serves as an introduction to Slick and which compares it to the ORM approach.
FWIW, I wrote a short summary of the state of database access in Scala some months ago. I personally found that using Slick in combination with its plain SQL querying capabilities to be quite convenient.
I am starting to delve into the realm of ORMs, particularly NHibernate in developing .NET data-aware applications. I must say that the learning curve is pretty steep and that a lot of things should be noted. Apparently, it actually changes the way you do data-aware applications, manner of coding, development and just about everything.
Anyway, I want to ask if you do set some parameters when deciding to USE or NOT TO USE ORMs in your applications? How do you decide then the approach that one needs to make it valuable to your organization?
The organization which I work for now apparently has made a lot of SQL and Data Access thing running through back end and I must say that these class/methods/procedures have successfully performed their tasks of providing the data which is needed and when it is needed. I think it would be a tremendous effort just to map some of this into ORM and derive the same business value that the company has for the last few years.
Nevertheless, I know that ORM paves the way for applications to talk with database servers, if properly implemented. I must admit that I am at a learning stage and that I would possibly need all the help, resources and the guidance to make this transition. I was also thinking of buying the book from Manning but I feel that with so much changes to NHibernate, the book may be a bit outdated. Perhaps waiting for the Packt book on NHibernate (release on May 2010??) would help me better get up and running.
Kindly share your thoughts. By the way, if you could also point me in a small sample web app which uses NHibernate + Visual Web Developer 2008 Express and SQL Server, that would be highly appreciated.
Thanks.
For me, the short of it is the following:
If you don't use an established ORM, and you develop correctly (meaning you refactor out duplication and look to simplify where you can), you'll wind up building your own ORM through the evolution of your data access layer.
The question then becomes:
"Do I want my developers spending time learning the idiosyncrasies of my home-grown ORM or learning those of a well-documented and well-tested ORM?"
Furthermore:
"If I'm hiring a new developer, wouldn't it be nicer to bring in a developer that knows the established ORM tool we're using rather than having to train someone up on this thing I built?"
I use NHibernate, particularly Fluent - and it's great; if given the choice, I wouldn't develop on an RDBMS any other way.
To be successful with an ORM you must make sure to normalize correctly, and use the database for it's designed purpose, storing data.
I don't use an orm when:
I don't use a relation database (Relational databases are not the best choice of database for every application)
The database is has a very small amount of tables. (I might need less code without an orm)
I use a very simple database that can map to code with simple naming
conventions. (Mapping to dumb DTO classes and all queries like select * from tablename where id=#id)
Learning a good orm is worth the time and effort, it will save you writing a lot of code when you use relational databases a lot.
You can find example apps/tutorials/video's about NHibernate on with stackoverflow search. There is another book in progress by manning, maybe it's possible to read it with the early access program.
Time and again, I've seen people here and everywhere else advocating avoidance of nonportable extensions to the SQL language, this being the latest example. I recall only one article stating what I'm about to say, and I don't have that link anymore.
Have you actually benefited from writing portable SQL and dismissing your dialect's proprietary tools/syntax?
I've never seen a case of someone taking pains to build a complex application on mysql and then saying You know what would be just peachy? Let's switch to (PostGreSQL|Oracle|SQL Server)!
Common libraries in -say- PHP do abstract the intricacies of SQL, but at what cost? You end up unable to use efficient constructs and functions, for a presumed glimmer of portability you most likely will never use. This sounds like textbook YAGNI to me.
EDIT: Maybe the example I mentioned is too snarky, but I think the point remains: if you are planning a move from one DBMS to another, you are likely redesigning the app anyway, or you wouldn't be doing it at all.
Software vendors who deal with large enterprises may have no choice (indeed that's my world) - their customers may have policies of using only one database vendor's products. To miss out on major customers is commercially difficult.
When you work within an enterprise you may be able to benefit from the knowledge of the platform.
Generally speaking the DB layer should be well encapsulated, so even if you had to port to a new database the change should not be pervasive. I think it's reasonable to take a YAGNI approach to porting unless you have a specific requriement for immediate multi-vendor support. Make it work with your current target database, but structure the code carefully to enable future portability.
The problem with extensions is that you need to update them when you're updating the database system itself. Developers often think their code will last forever but most code will need to be rewritten within 5 to 10 years. Databases tend to survive longer than most applications since administrators are smart enough to not fix things that aren't broken so they often don't upgrade their systems with every new version.Still, it's a real pain when you upgrade your database to a newer version yet the extensions aren't compatible with that one and thus won't work. It makes the upgrade much more complex and demands more code to be rewritten.When you pick a database system, you're often stuck with that decision for years.When you pick a database and a few extensions, you're stuck with that decision for much, much longer!
The only case where I can see it necessary is when you are creating software the client will buy and use on their own systems. By far the majority of programming does not fall into this category. To refuse to use vendor specific code is to ensure that you have a porrly performing database as the vendor specific code is usually written to improve the performance of certain tasks over ANSII Standard SQL and it written to take advatage of the specific architecture of that database. I've worked with datbases for over 30 years and never yet have I seen a company change their backend database without a complete application rewrite as well. Avoiding vendor-specific code in this case means that you are harming your performance for no reason whatsoever most of the time.
I have also used a lot of different commercial products with database backends through the years. Without exception, every one of them was written to support multiple backends and, without exception, every one of them was a miserable, slow dog of a program to actually use on a daily basis.
In the vast majority of applications I would wager there is little to no benefit and even a negative effect of trying to write portable sql; however, in some cases there is a real use case. Let's assume you are building a Time Tracking Web Application. And you'd like to offer a self hosted solution.
In this case your clients will need to have a DB Server. You have some options here. You could force them into using a specific version which could limit your client base. If you can support multiple DBMS then you have a wider potential client that can use your web application.
If you're corporate, then you use the platform you are given
If you're a vendor, you have to plan for multiple platforms
Longevity for corporate:
You'll probably rewrite the client code before you migrate DBMS
The DBMS will probably outlive your client code (Java or c# against '80 mainframe)
Remember:
SQL within a platform is usually backward compatible, but client libraries are not. You are forced to migrate if the OS can not support an old library, or security environment, or driver architecture, or 16 bit library etc
So, assume you had an app on SQL Server 6.5. It still runs with a few tweaks on SQL Server 2008. I bet you're not using the sane client code...
There are always some benefits and some costs to using the "lowest common denominator" dialect of a language in order to safeguard portability. I think the dangers of lock-in to a particular DBMS are low, when compared to the similar dangers for programming languges, object and function libraries, report writers, and the like.
Here's what I would recommend as the primary way of safeguarding future portability. Make a logical model of the schema that includes tables, columns, constraints and domains. Make this as DBMS independent as you can, within the context of SQL databases. About the only thing that will be dialect dependent is the datatype and size for a few domains. Some older dialects lack domain support, but you should make your logical model in terms of domains anyway. The fact that two columns are drawn from the same domain, and don't just share a common datatype and size, is of crucial importance in logical modelling.
If you don't understand the distinction between logical modeling and physical modeling, learn it.
Make as much of the index structure portable as you can. While each DBMS has its own special index features, the relationship between indexes, tables, and columns is just about DBMS independent.
In terms of CRUD SQL processing within the application, use DBMS specific constructs whenever necessary, but try to keep them documented. As an example, I don't hesitate to use Oracle's "CONNECT BY" construct whenever I think it will do me some good. If your logical modeling has been DBMS independent, much of your CRUD SQL will also be DBMS independent even without much effort on your part.
When it comes time to move, expect some obstacles, but expect to overcome them in a systematic way.
(The word "you" in the above is to whom it may concern, and not to the OP in particular.)
Closed. This question needs to be more focused. It is not currently accepting answers.
Closed 1 year ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I've never had much need for programming with databases. Since their use is so widespread it seems like a good thing for me to learn. SQL seems like the place to start, possibly SQLite and maybe the Python bindings. What would you recommend for someone new to this? Libraries, tools and project ideas are all welcome.
Structure Query Language (SQL) is the language used to talk to database management systems (DBMS). While it's a good thing to learn, it's probably best to do it with a project in mind that you'd like to do. It's funny you say you've never had a need, because I'm the opposite, almost every program I've ever written has used a database of some sort. The vast majority (mostly web-based) revolve around using a database.
Learn about relations and database architecture. This means how to structure your tables, make foreign keys and relations.
For example, you might have a movies database. In it, you store information about the Movies, Studios that released the movies, and the Actors in the movies. Each of these becomes a table. Each Movie is released by one Studio. Since you don't want to store duplicate the studio information (address, etc) in each Movie entry, you store a relation to it, so each Movie item contains a reference to a Studio item. This is called a one-to-many relationship (one studio has many movies). Likewise, you don't want to store Actor information for each Movie. But one Actor can be in many Movies, so this is stored as a many-to-many relationship.
Learn SQL itself. SQLCourse is a good place to get started, but there are many other books and resources. SQL is a standard, but each RDBMS has its own vendor-specific ways of doing certain things and other limitations (for example, some systems don't support sub-queries, there are several different syntaxes for limiting the number of rows returned, etc). It's important to learn the syntax for the one you're using (eg, don't learn Oracle syntax and then try and use it in MySQL) but they are similar enough that the concepts are the same.
Tools depend on the DBMS you use. MySQL is a pretty popular database, lots of tools are available, and lots of books. SQLite and Postgresql are also quite popular, and also free/open-source.
If you can, you really want to find someone who knows how to use it, and pick their brains. That's because there are a lot of important principles (eg 3rd normal form) which will are a lot easier to learn through discussion rather than from books.
If you want to teach yourself, you should learn the syntax for doing basic selects, joins, updates, deletes, and group by queries. You should also learn the "Swiss army knife" of selects, the CASE statement. Too many people don't. Many of the tutorials recommended in this thread will do that. Then you need to try to solve SQL problems. I'm sure that Joe Celko's SQL Puzzles and Answers is a good source of them, though it may be a little advanced.
This will let you actually write SQL. But you still need to learn how to organize a database. Which for most purposes means that you really need to learn what 3rd normal form looks like. You don't have to be able to give a formal definition of it, just recognize it when you see it, and know how to adjust something to be in that format.
Lots of references will explain it, but you won't know if you're reading them correctly. This is where it really, really helps to have access to someone who can look at a table layout and tell you, "That's right" vs "That's wrong, here's what needs to be changed." Failing all else, you could post a question here with a proposed layout. But a back and forth discussion with a live person would still be preferable IMO.
Try Wikipedia, http://www.w3schools.com/sql/default.asp and http://www.sql-tutorial.net/ Also check YouTube for SQL Videos.
You are correct, SQLite is a great place to start because it is free, lightweight, and available on many platforms. This is only a start though. SQLite is very liberal on SQL syntax and lacks an intneral programming language like DBMS systems have. Still, if you want to start and learn with minimal overhead, SQLite is the way to go.
SQLite is nice and they have really nice documentation, however you should be aware that it is not a full featured SQL database like MySQL, Postgres or the commercial variants. SQLite's API relies on callbacks which is a fine model, but not all database APIs work that way. If you are familiar with Perl, then DBI is another nice way to explore SQL.
/Allan
"A Gentle Introduction to SQL" - You can even practice "live queries" right on this tutorial website.
http://sqlzoo.net/
I always recommend The Practical SQL Handbook for a good starting point for beginners - especially those who have seen SQL but never understood how to build up a query them selves or how they work.
All Celko's books are great.
Hernandez's Mere Mortals book is good.
Ken Henderson's books are also excellent.
Reading up a bit on relational algebra is a good way to understand the underlying concepts of relational databases.
Jeo Clecko's SQL for smarties is excellent.
I recommend the exercises at this site: sql-ex.ru
You can even get a certificate if you do the right.
Start with Ideone and try queries on line just with a web browser.
If you program using the .NET framework, then learning LINQ might be a good place to start. The LINQ "engine" will handle the back end communication with the database (or objects, or entities, or XML, etc.) for you. If you want to dig deeper, you can explore the SQL generated by the LINQ that you write.
If you already know a thing or two about web applications, then that would be a good place to start. Nearly every serious webapp uses an SQL database as its backend.
The folks at Head First have come out with a book. Going by how good their other books are, I'd recommend this one. Haven't read it yet though. (LINK)
You may want to consider starting with MySQL as it is widely documented and very easy to get started with. You can download the Community Edition and then add the GUI Tools and you'll both GUI and command line interfaces.
Read a book to start learning about SQL. I read Beginning SQL Queries from Apress not long ago, and found it clear and logically written for a beginner (I reviewed it for a colleague).
I learnt 90% of what I know about SQL from here. In 1997.
I think it still stands up.
Hey although not complete it's great to get hands on with SQLite as mentioned above, Google 'Learn SQL the hard way' and there is an online book which you can work through which uses SQLite.
Google is great for downloading pdf's for free 'Cough Cough' but try http://www.sqlfiddle.com/
It's an online platform which is free! No log in required just go to their page, create your database in whichever language you want (That's the best bit I choose T-SQL as I'm learning that), and then you can query it as much as you like. I'm learning with a pdf file which has opensource SQL files you can download to follow along, and SQLFiddle has been great to learn vendor specific SQL.