{{http://alexle.net/wp-content/uploads/2007/06/rails.thumbnail.png }}Working on Wars of Earth by myself, I run into several situations when I’d like to move migration files around to get them executed in a particular order. The trick is simple: just rename the sequence of the migration files to get them to be executed in the order you want.
Besides sloppy schema design (hey, I’m trying to be agile here - fix it as you go), an example of a situation when you want to move migration files around: table A has a foreign key to table B. Migration file for A is at 005, while that of B is at 010. If you try to put in the FORIEGN KEY constraint in A migration file after the table creation, migration will fail since table B hasn’t been created yet, not until migration #010. You can add another migration file just to add the FORIEGN KEY constraint from A to B, say at #020. But then you will have fragmented migration code all over the place. For production environment, this is the only solid and proven way to perform database changes. However, during development, you have the luxury to drop and recreate the entire database from scratch, it’s just a lot more convenient and makes more sense to be able to move migration files around. Ideally, you just need to run migration for B first, then you can run migration for A, which also execute the SQL to add in the FOREIGN KEY constraint properly.
Until now, you have to it manually. Imagine if you have 20 migration files in between, from 5 to 25 (like what I have), you will probably have to renumber 20 migration files. And if your project is in SVN, it would be more time-consuming and error-prone. You can either do “svn rename”, or just rename in the shell, then deleted the previous migration files and added the newly-named ones back into the repository. But what would happen if you yawn for a second and misnumber a file at #6, so you have two 006 migration files. Oops.
Worry no more, here comes **numergrate** utility to the rescue. In short, numergrate is “to numerate migration files” (or to renumber migration files)
What you need is to drop in the numergrate script (download below) inside your Rails’ script folder and you are ready to go. With this utility, you can execute at the root of the Rails project
ruby script/numergrate 5 before 25
to (test) move 005 to position 024 instead.
or
ruby script/numergrate 10 after 2 --svn
to move 010 before 002 and also rename the files in the Subversion repository.
I also took the opportunity to integrate the utility with Subversion in case your migration files are in a repository. There’s no easy way for CVS so I didn’t implement it. However, implementing renaming mechanism for other SCM should be straight forward if you know their command line renaming tool.
I hope this comes handy for you.
==== Supported tools ====
* {{http://alexle.net/wp-content/uploads/2007/06/subversion_logo.thumbnail.jpg}} Subversion (renaming files in repository)
* {{http://alexle.net/wp-content/uploads/2007/06/utilities-terminal.png}} Shell (just like renaming manually)
==== Script ====
#!/usr/bin/env ruby
# (c) 2007 Alex Le
# www.alexle.net - nworld3d@yahoo.com
# This script is developed for www.warsofearth.com. (shameless self-promotion)
# Released under the same license as Ruby
# Disclaimer: The author is not responsible for any incorrect results from running
# the script. Use it at your own risk.
#
# USAGE:
# ------
# This script is used to re-number the migration files into the desired sequence
# In development, this comes handy as you can organize migration files into logical groups
# by running a simple command line utility instead of manually renaming the filenames.
#
# numergration = numerate migration files
#
# INSTALL
# -------
# Copy numergration into script/ folder of your Rails application.
# On linux system you may have to chmod the script to be executable (+x)
#
# HOW TO RUN
# ----------
# At your Rails application root, run:
# On Windows:
# > ruby script/numergrate
# On Linux: users can just run the script without calling the ruby executable since
# there’s a #! on top, provided that you set the permission correctly. (chmod to +x)
# $ script/numergrate
#
# The [mode] options are
# –test Default mode to test the result before you run
# with –shell or –svn
#
# –shell Renaming file as you would do manually in the shell
#
# –svn Integrate with Subversion by executing `svn rename` on each file
# This option alters your working copy so please be extra careful.
# (a.k.a. use it at your own risk)”
#
# Example:
# ——–
# a. Move Migration file 50 to position 3, hence shifting migration files
# from 3 to 49 to the right by 1.
# > ruby script/numergrate 50 before 3
# (the above will just execute with the –test default option)
#
# Or to actually rename the files,
# > ruby script/numergrate 50 before 3 –shell (or –svn)
#
# Or you can even run
# > ruby script/numergrate 50 after 3
# to put migration file 50 after migration file 3 (shifting migration
# file 4 to 49 to the right by 1)
#
# TODO:
# —-
# 1. Better sequence handling. Currently it’s default to 000 for the
# sequence series. However, there can be potentially a lot migration files.
# (more than 999 files). The solution is to find the max sequence and
# use that as the series template
# 2. Better SVN integration.
# a. Do some checking to see if the svn client exists before running.
# Otherwise throw an error
# 3. Better sequence checking. Currently it doesn’t check for input
# range so we can have “index out of bound” errors.
#
require ‘fileutils’
include FileUtils
# which folder we would skip while iterate through
SKIPPED_FILES = [’.', ‘..’,’.svn’]
# check for arguments
unless ARGV.size == 3 or ARGV.size == 4
puts ‘invalid syntax’
exit
end
# this class hold the information about the migration file
class MigrationFile
attr_accessor :sequence, :name, :new_sequence
def to_s(options={})
if @new_sequence != @sequence
s = sprintf(”%03d”,@new_sequence) << "_#{@name}"
else
s = sprintf("%03d",@sequence) << "_#{@name}"
end
end
def initialize(sequence, name)
@sequence = sequence
@name = name
@new_sequence = @sequence
end
def shift_left()
@new_sequence -= 1
end
def shift_right()
@new_sequence += 1
end
def is_changed?
return @new_sequence != @sequence
end
def old_name
s = sprintf("%03d",@sequence) << "_#{@name}"
end
def new_name
s = sprintf("%03d",@new_sequence) << "_#{@name}"
end
end
# 123 after 234 --test
src, task, dest, mode = [ARGV[0].to_i, ARGV[1], ARGV[2].to_i, ARGV[3]] # got to explicitly convert to number for comparision
# exit if don't have to move
exit if src == dest
# default mode to --test
mode ||= "--test"
#grab the migration files
files = []
Dir.entries("db/migrate").each { |file|
unless SKIPPED_FILES.include?file
files << MigrationFile.new(file.to_i, file.match(/.+?_(.*)/)[1])
end
}
# now perform shifting
files.each{ |file|
if src > dest
if file.sequence == src
if task == “before”
file.new_sequence = dest
elsif task == “after”
file.new_sequence = dest + 1
end
else
# shift the innner range files
if file.sequence >= dest && file.sequence < src
if task == "before"
file.shift_right
else
file.shift_right unless file.sequence == dest # if insert after, we don't need to shift the dest
end
end
end
elsif src < dest
if file.sequence == src
if task == "before"
file.new_sequence = dest - 1
elsif task == "after"
file.new_sequence = dest
end
else
# shift the innner range files
if file.sequence <= dest && file.sequence > src
if task == “before”
file.shift_left unless file.sequence == dest # if insert before, we don’t need to shift the dest
else
file.shift_left
end
end
end
end # if src > dest
}
#files.each{ |f| puts f if f.new_sequence != f.sequence }
# now issue
puts “”
puts ” Execute using #{mode} option”
puts “”
puts ” You can execute with these options: ”
puts “”
puts ” –test Default mode to test the result before you run”
puts ” with –shell or –svn”
puts “”
puts ” –shell Renaming file as you would do manually in the shell”
puts “”
puts ” –svn Integrate with Subversion by executing `svn rename` on each file”
puts ” This option alters your working copy so please be extra careful.”
puts ” (a.k.a. use it at your own risk)”
puts “”
files.each{ |file|
if file.is_changed?
if mode == “–shell”
puts ” rename ” << "db/migrate/" << file.old_name
puts " to " << "db/migrate/" << file.new_name
cp("db/migrate/" << file.old_name, "db/migrate/" << file.new_name )
rm("db/migrate/" << file.old_name)
elsif mode.downcase == "--svn"
#puts "executing svn commmand here"
puts " svn rename " << "db/migrate/" << file.old_name
puts " to " << "db/migrate/" << file.new_name
system 'svn rename --force db/migrate/' << file.old_name << " db/migrate/" << file.new_name
elsif mode.downcase == "--test"
puts " [TEST] rename " << "db/migrate/" << file.old_name
puts " to " << "db/migrate/" << file.new_name
end
end
}
==== Download ====
* To download, please click here: [[http://alexle.net/wp-content/uploads/2007/06/numergrate.zip|numergrate]] (version 1.0, 06/17/2007)
==== Note ====
* Moving migration files around can do serious damage to your database. Please be careful. This comes extremely dangerous if you are working with other people on the same repository. Please be smart about it. Use it at your own rick.
* Comments and suggestions are very welcome.
* Support me at [[http://www.warsofearth.com|Wars of Earth]] if you can. Another shameless self-promotion.
You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
Subscribe in a reader. Enjoy reading my blog? Subscribe to my RSS to read new post whenever I publish something new.
It turns out that `svn rename` doesn’t actually rename the file. It’s an ADD followed by a DELETE of the original file. So performing a `svn rename` on a large migration files move-around can result in devastating consequence to your team.
You’ve been warned.
Once you move migration using –svn, you have to explicitly commit it to the repository. Otherwise you will get complaints about the file doesn’t exist in the repository or something.
Another prerequisite to run numergrate with –svn option is you have to have Subversion client installed (svn.exe on Windows, included in the full Subversion installation, not the TortoiseSVN installation). Otherwise nothing will happen, even though you will see messages printed out to the screen.