1 Comment →
So my wife laughs at me quite a bit, but I’m a little obsessed with efficiency and automation. I hate wasting my time on non-engaging tasks. Which is, as it happens, most of life. So I try to find ways to automate things. The less I have to invest my time in tasks with a clearly defined process the more time I can spend working on things I actually care about. At work I put this to the test constantly. Working as a software engineer there is just a pile of repetitive tasks that are ripe for automation. Fortunately a number of tools have entered the battlefield against wasting our time on processes that can be automated.
I love the growth and ever increasing adoption of continuous integration. At my new job I was thrilled to see how much they had already begun to move in that direction. It’s not perfect, that’s for sure, but we’ve got a solid start and we always keep our eyes out for ways to improve that process. That said, there’s still a lot of room for grabbing the low hanging fruit and Python/IronPython (two tools I’ve mentioned a number of times) come in to save the day for me.
One place I decided to tackle recently was our revision control. We use Hg which is a Python based tool itself, but they expressly discourage people from importing their libraries and using them, simply because they don’t guarantee they won’t change and therefore break everything. I had a short internal debate about if I thought it was worth it and eventually I opted against it, simply because it was just too easy to work around that issue (for my needs at least).
The problem came up that we have a pile of repositories that we all work in frequently. Keeping these repositories up to date was kind of a pain. That said the more familiar I get with the command line for Hg the easier, however I opted to write something with python to handle the task at hand. I came across this script which really accomplished what I needed, but I just wasn’t in love with the implementation. I wanted something a bit more flexible, I had in mind that I could use some of this for some tasks with our automated build too.
I started off by writing a simple “repo” class.
import os PENDING_STATUS = ['A','M'] ERRORS = ['access denied', 'abort'] PULL = 'hg pull' UPDATE = 'hg update' STATUS = 'hg status' TIP = 'hg tip' NUDGE = 'hg push --rev .' class repo: def __init__(self, path=''): self.path = path self.validdir = False if path: self.__validate_dir__() def __validate_dir__(self): if not os.path.exists(os.path.join(self.path,'.hg')): print self.path, 'is not an hg directory.' self.validdir = False return self.validdir = True def __haspending__(self, show=False): if not self.__prep__(): return False out = os.popen(STATUS) changes = [ln for ln in out if ln in PENDING_STATUS] if show: for chg in changes: print chg return len(changes) > 0 def __indir__(self): return os.getcwd() == self.path def __gotorepo__(self): if not self.validdir: return False if not self.__indir__(): os.chdir(self.path) return True def __prep__(self): if not self.validdir: return False return self.__gotorepo__() def __checkerror__(self, output): for l in output: err = [e for e in ERRORS if e in l] if err: print err return True return False def setpath(self, path): self.path = path self.__validate_dir__() def issafe(self): if not self.__prep__(): return False if self.__haspending__(True): print 'Will not update. Pending changes found.' return False print 'No pending changes found... safe to update.' return True def getchg(self): if not self.validdir: return False changeset = [ln for ln in os.popen(TIP) if 'changeset:' in ln] return ln.split(':') def anypending(self): return __safe_to_update__() def pull(self): if not self.__prep__(): return out = [ln for ln in os.popen(PULL)] if self.__checkerror__(out): return for ln in out: if 'no changes found' in ln: print ln return False print 'pulled latest' return True def update(self, show=True): if not self.__prep__(): return out = [ln for ln in os.popen(UPDATE)] if self.__checkerror__(out): return for ln in out: if show: print ln def sync(self, update_if_safe = True): if not self.__prep__(): return print 'Getting latest for ' + os.getcwd() #only try to update if there were changes... if self.pull(): if update_if_safe and self.issafe(): print 'Updating...' self.update(False) print 'done' def nudge(self, show=True): if not self.__prep__(): return out = [ln for ln in os.popen(NUDGE)] if self.__checkerror__(out): return for ln in out: if show: print ln
It’s not the prettiest, in fact I’ll change it soon to have a ‘verbose’ property or something so I can ditch the “show=True” params from a number of those functions. The idea for the “nudge” function came from an hgtip.com post where they address what is honestly an annoying part of mercurial to me.
Once I had this repo class then I just needed a simple script to wrap around it. This is a bit closer to the script I mentioned earlier, but it’s got some options for command line params.
import os, hgutil, config, sys def main(argv=None): if argv == None: argv = sys.argv arg_path = None # start at 1, 0 is the command for i in range(1, len(argv)): if argv[i] == '-p': arg_path = argv[i+1] if argv[i] == '-h': show_help() return True path = get_path(arg_path) if has_connection(): for f in os.listdir(path): p = os.path.join(path,f) if os.path.isdir(p): repo = hgutil.repo(p) repo.sync() else: print 'No connection to hg found. Please ensure pageant is running.' return False return True def has_connection(): return len([ln for ln in os.popen('tasklist') if 'pageant' in ln]) > 0 def get_path(path=None): if path != None and os.path.exists(path): print 'starting from path:', path return path ##default print 'No valid path found... using default.' return config.get_home() def show_help(): print '--- sync.py ---' print 'Attempts to sync all hg repositories under a path.' print 'Returns errors if pageant is not running.' print 'Args:' print '-p : path to work from' print 'Example: sync.py -p c:\dev' print 'Done.' if __name__ == "__main__": sys.exit(main())
Since we use pageant for our connections to mercurial it’s nice to have this be able to check and let me know in case I’ve forgotten to enable it.
Now that I had this full script, I create a windows scheduled task to run it behind the scenes for me every 10 mins. Now if I’m not actually working with a specific repository it just gets updated for me, and if I am, I pull the changes, but don’t update until I’m ready. This needs more work before it’s terribly useful on my automated build server, but the ground work is there and I’ve been really happy with it so far. It might not seem like it but this script saves me a solid 10-15 mins a day manually doing pulls and updates on my repos and a lot of help where I run into issues from running an older version of some code.
Now, it is possible that I’ll be working on something I don’t want automatically updated. Then I’ll just change it over to default to not update. The automated pull is still useful.Tags: automation, continuous integration, hg, mercurial, Python, source control