sometimes when working in rails, I work on several things at once using git branches
sometimes, I'd like to test new ideas by implementing them and testing how and if they work accordingly. This involves sometimes adding models and migrations.
When switching branches, however, the migrations were already migrated to the DB and they stay, causing problems later on..
Is there a way to work with several branches and each to have different migration files, and before starting to work on a branch to "soft reset" the db only to the current migration files without losing data?
Normally, in development, I need some sample data that I keep in seed.rb which enables me to recreate the db, its structure and the sample data with a rake task.
Another thing I did was to keep more than one database. I would then just manually change the entry in database.yml according to the current git branch.
Related
I am using Django as my web framework with Django REST API. Time and time again, when I try to migrate the table on production, I get a litany of errors. I believe my migrations on development are out of sync with production, and as a result, chaos. Thus each time I attempt major migrations on production I end up needing to use the nuclear option - delete all migrations, and if that fails, nuke the database. (Are migrations even supposed to be committed?)
This time however, I have too much data to lose. I would like to preserve the data. I would like to construct a new database with the new schema, and then manually transfer the old database to the new one. I am not exactly sure how to go about this. Does anyone have any suggestions? Additionally, how can I prevent this from occurring in the future?
From what you're saying, it sounds like you have migration files that are out of wack and you're constantly running into issues relating to database migrations. I would recommend you just remove all of your migration files and start with a new initial migration after you make all the necessary model changes and restructuring of the schema.
When it comes time to make the migration on your production server, it might make the most sense to --fake-initial and manually making the database changes outside of Django so it matches your schema.
I might get a lot of backlash about this and obviously use your best judgement, but from my experience it was much easier to go about this problem this way and not wasting time making custom migration files that try to fix all of your problems.
Addressing your other questions
Time and time again, when I try to migrate the table on production, I get a litany of errors.
I highly recommend you take the time to get acquainted with how to make migrations by reading the official Django docs, you will save yourself a LOT of headache.
... each time I attempt major migrations on production I end up needing to use the nuclear option - delete all migrations
You shouldn't be deleting your migration files every time there's an issue.
Are migrations even supposed to be committed?
You should definitely be committing your migrations. If you're working on a team, they would be using the migration files you created to make the necessary changes on their local DB as well as any dev/prod server you may have.
I have the following problem:
I am working on two different branches: dev, feature1
Both branches work on the same database schema.
Imagine I have changelog-v1.0 on my dev-branch und my database fits to that version.
Now if I have changelog-v1.1 on my feature1-branch and start my app from there liquibase will apply thhis changes to my database.
From this moment on my app on dev-branch will not work anymore, because even though there is no changelog-v1.1 on the dev-branch, the database is already migrated to that version
and if I start my app on the dev-branch liquibase will not roll back the changes of changelog-v1.1.
Working on many branches with different changelog but just one database.
How do I manage this?
You can have a master changelog file pointing to children changelog files. But need to make sure to have same files in all branches that have to be merged from dev to feature branch.
Also, note that you have rollback code and tagging in the liquibase changesets in order to successfully rollback changes.
How to properly work with migrations during development when the model is not in its final form or there are still some inconsistencies in relations?
Should migrations be treated more as commits in version control system like git and represent even the smallest change, or what?
Should I add new migration for each small change in my model, or should I just re-create the database and initial migration until I'm satisfied with how my model looks? I can still remove all those migrations in the end and create initial one to clean those small migrations up.
How to properly name migrations? Most often I can't come up with a good name and I give them some meaningless names like X1, X2, X3
Similar questions:
How to deal with database changes during development with EF Core?
1: I personally would keep it as a history. Of course you could always delete all migrations and create one that contains everything but imagine that migration after you have added 100++ tables (entity-types) and you cannot make sure your production database for instance is being updated, when you only have one migration with same name that you just always recreate.
2: Yes, you should indeed make small migrations. You can undo a migration by updating your database to a specific migration and then removing all the others before step by step. This at least works with the package-manager-console tool (maybe also with dotnet tool).
For instance you have already added a migration with models that have changed you go back to the old migration by using this command:
Update-Database -Migration MyMigrationBeforeBadModelMigration
Be aware that this might drop tables if some were added in the migration that you want to undo.
Then remove bad migrations step by step
Remove-Migration // will always remove the latest migration so repeat that if you have many to remove
Then just create the new and proper migration and update your db.
3: Yes, give them proper names. For instance CustomerEntityAdded or CustomerUniqueNameIndexAdded.
I have the following scenario:
I'm starting development of a long project (around 6 months) and I need to have some information on the database in order to test my features. The problem is that right now, I don't have the forms to insert this information (I will in the future) but I need the information loaded on the DB, what's the best way to handle this? Specially considering that once the app is complete, I won't need this process anymore.
As an example, lets say I have tasks that need to be categorized. I've begun working on the tasks, but I need to have some categories loaded on my db already.
I'm working with Rails 3.1 btw.
Thanks in advance!
Edit
About seeds:I've been told that seeds are not the way to go if your data may vary a bit, since you'd have to delete all information and reinsert it again. Say.. I want to change or add categories, then I'd have to edit the seeds.rb file, do my modifications and then delete and reload all data...., is there another way? Or are seeds the defenitely best way to solve this problem?
So it sounds like you'll possibly be adding, changing, or deleting data along the way that will be intermingled amongst other data. So seeds.rb is out. What you need to use are migrations. That way you can search for and identify the data you want to change through a sequential process, which migrations are exactly designed for. Otherwise I think your best bet is to change the data manually through the rails console.
EDIT: A good example would be as follows.
You're using Capistrano to handle your deployment. You want to add a new Category, Toys, to your system. In a migration file then you would add Category.create(:name => "Toys") or something similar in your migration function (I forget what they call it now in Rails 3.1, I know there's only a single method though), run rake db:migrate locally, test your changes, commit them, then if it's acceptable deploy it using cap:deploy and that will run the new migration against your production database, insert the new category, and make it available for use in the deployed application.
That example aside, it really depends on your workflow. If you think that adding new data via migrations won't hose your application, then go for it. I will say that DHH (David Heinemeier Hansson) is not a fan of it, as he uses it strictly for changing the structure of the database over time. If you didn't know DHH is the creator of Rails.
EDIT 2:
A thought I just had, which would let you skip the notion of using migrations if you weren't comfortable with it. You could 100% rely on your db/seeds.rb file. When you think of "seeds.rb" you think of creating information, but this doesn't necessarily have to be the case. Rather than just blindly creating data, you can check to see if the pertinent data already exists, and if it does then modify and save it, but if it doesn't exist then just create a new record plain and simple.
db/seeds.rb
toys = Category.find_by_name("Toys")
if toys then
toys.name = "More Toys"
toys.save
else
Category.create(:name => "More Toys")
end
Run rake db:seeds and that code will run. You just need to consistently update the seeds.rb file every time you change your data, so that 1) it's searching for the right data value and 2) it's updating the correct attributes.
In the end there's no right or wrong way to do this, it's just whatever works for you and your workflow.
The place to load development data is db/seeds.rb. Since you can write arbitrary Ruby code there, you can even load your dev data from external files, for instance.
there is a file called db/seeds.rb
you can instantiate records using it
user1=User.create(:email=>"user#test.com",
:first_name=>"user",
:last_name=>"name",
:bio=>"User bio...",
:website=>"http://www.website.com",
:occupation=>"WebDeveloper",
:password=>"changeme",
:password_confirmation=>"changeme",
:avatar => File.open(File.join(Rails.root, '/app/assets/images/profiles/image.png'))
)
user2=User.create(:email=>"user2#test.com",
:first_name=>"user2",
:last_name=>"name2",
:bio=>"User2 bio...",
:website=>"http://www.website.com",
:occupation=>"WebDeveloper",
:password=>"changeme",
:password_confirmation=>"changeme",
:avatar => File.open(File.join(Rails.root, '/app/assets/images/profiles/image.png'))
)
Just run rake db:seed from command line to get it into the db
My app is in beta, and I've been doing limited testing of a feature that involves a new model. After a fair amount of testing I had to make a structural change that makes the old data non-functional.
What I need to do is just drop and recreate one table. I know that I could do this in a migration, but that seems like such a hack. In a local dev copy I would just use db:reset, but in the beta app I don't want to lose data in any tables except this one.
Is this a simple way to instruct a production app to drop and recreate a single table. In my case, I'm deploying with Heroku, in case that affects how you would solve this issue.
To empty a table on Heroku without changing the schema, in your application's directory:
$ heroku run console
Ruby console for myap.heroku.com
>> ModelName.delete_all
>> exit
I know that I could do this in a migration, but that seems like such a hack.
It's not a hack. It's precisely what migrations are designed to do.
You need to rerun the migration for that table to make structural changes. I haven't used ActiveRecord before, but I'd also delete the data in the table using ModelName.delete_all from heroku console.
heroku run console
irb(main):001:0> ModelName.delete_all
And you are done.