This Week in Bazaar

Welcome back to the terrarium of the Bazaar distributed version control system. Written by co-authors John Arbash Meinel, one of the primary developers on Bazaar, and Paul Hummer, who works on integrating Bazaar into Launchpad as he refines his plans for world domination from his shiny new lair.

Bazaar 1.6b3 released

The next beta release of Bazaar has just been cut, and is available at your local PPA:
https://launchpad.net/~bzr/+archive

The Windows installers should be available later today. This release provides lots of the shiny things that we've been talking about, like Stacked Branches, Real Weave Merge, more hooks for server-side operation, and lots of bug fixes and general polishing. The full UI for using stacked branches still needs a little bit of polishing, so the feature is not enabled by default. The functionality is all there, and if you are interested, we'd love to hear from you (kudos and complaints are equally welcome).


New updates to Gnome Bazaar Playground

Coming back from a very productive trip to Guadec, Tim Penhey has been overseeing some customizations to the Bazaar Playground for Gnome. All of the branches created at the local server in Turkey for Guadec have been added to the public playground. The Loggerhead installation has received some TLC by way of customizations to the UI. Accerciser's playground page is a good demonstration af the UI changes that have been made. The playground is actively being used by applications such as Brasero, jhbuild, Metacity and more.

One of the fun results of meeting with people at Guadec, is that it showed ways to improve Loggerhead when dealing with lots of projects and lots of branches. Work is continuing to make customizing Loggerhead's look-and-feel easier, and providing better tools for creating these "Bazaar Playgrounds" to use in evaluating Bazaar. The Bazaar developers are committed with making tools easier to use, and making the process as simple and powerful as possible.

Up and Coming Repository Format Updates

Robert Collins has been hard at work to refine how Bazaar stores its history information. We all like to have deep context, but we don't like to have to pay the penalty of downloading all of that context. Because Bazaar has a flexible repository structure, Robert has been able to play with changing the on-disk structure without major surgery to the rest of the code.

First is a change to how indexes are written, switching from a bisectable list to a btree structure. This paged structure allows us to compress the indexes, making them smaller, and faster to process remotely. It also reduces the number of lookups to find a key. (On average, a bisect search is log₂N, while the btree is closer to log₁₀₀N.) At the moment, he is testing this with a shared repository containing all of the projects available from in the Ubuntu apt repositories. This weighs in at around 13k branches, and somewhere around 20GB of disk space used.

Second is an update to how texts are stored. At the moment we use a simple format which places fulltexts periodically, and then stores deltas against those fulltexts. It has served us rather well, but can be improved upon. With his Group compress work, we can see a savings of as much as 2x-3x. Further, the data is stored such that you can do simple linear reads to get the base fulltext and all deltas necessary to generate a given fulltext. This reduces the pressure on indices, as you don't have to search for base texts. (Instead you just store a pointer to the start, and give the total length that needs to be read.)

These are still in development phase, but a format that uses them will likely appear in the next release (bzr 1.7).

Community Agile

Ian Clatworthy has recently released a wonderful document describing the workflow we (generally) use at Canonical. It describes how basic practices are similar to, and different from, other systems like Agile. The biggest (IMO) being a recognition that the community surrounding your project is one of the strongest and most important pieces. This has always been true in software development, but it has traditionally been somewhat hidden. Open Source has exposed just how powerful the community can be. For people interested in how software can be developed, rather than just what, I certainly recommend it.

This Week in Bazaar

Here we are again, bringing you the gossip and dirty secrets in the development world of the Bazaar distributed version control system. In this, the 10th week, the series is now under new management, with co-authors John Arbash Meinel, one of the primary developers on Bazaar, and Paul Hummer, who works on integrating Bazaar into Launchpad.


Bundle Buggy

Aaron Bentley has once again been improving his wonderful Bundle Buggy. He just introduced support for multiple projects using a single instance of Bundle Buggy. There are now 5 Bazaar projects using the main bundle buggy instance. (Bazaar, bzr-gtk, Bundle Buggy itself, Bzrtools, and PQM.) Of course, Daniel Watkins has made excellent use of his time, and has managed to crank out lots of updates for PQM. At this point it is code clean up, reducing the dependencies making it easier to set up and install.

Bazaar playground for Gnome

Originally, John Carr set up Bazaar mirrors of all the Gnome modules, which people could then use as a starting point for publishing code and collaborating. This week, the Bazaar playground for gnome was created so that any Gnome developers could be involved in pushing, branching, and sharing code through bazaar. This new server runs Loggerhead for viewing the code committed to these Bazaar branches. Damned Lies is also set up on the playground. This server was also reproduced locally at GUADEC because of the flaky internet connection at the conference, and all those local branches will be moved to the playground shortly.


Weave merging and handling "interesting" history

One of the great things about having a large project like MySQL using your software is that they push and stretch you in ways that you haven't necessarily encountered before. Specifically, their branch workflow looks a bit like a pile of spaghetti. With several long-term maintenance branches, team branches based off of that, and individual developer branches based off of that. Patches have a tendency to travel in unexpected ways (you may go user => team => release 1 => release 2, or you might go release 1 => team => team-2 => release 2, etc). They also are very fond of 'null merging' patches that aren't relevant to the next release. They merge the change and revert the text changes and commit.

Bazaar supports all of this, but it exposes weaknesses in simple 3-way merge logic. Because patches don't flow in anything considered orderly, you don't have the opportunity to select a "clean" base very often. Bazaar has long had an option for doing a "--weave" merge. It didn't receive much attention for a while, and had become rather slow. It turned out to be a good fit for MySQL's workflow, so John has spent a bit of time recently to make the functionality efficient and correct in some specific edge cases. Expect the improvements to show up in the next release.

This Week in Bazaar

This is the eighth (wow, 2 whole months of solid updates, yipee!) in a series of posts about current topics in the development world of the Bazaar distributed version control system. The series is co-authored by John Arbash Meinel, one of the primary developers on Bazaar, and Elliot Murphy, who drinks the rain. This week we are joined by Martin Albisetti, talking about Loggerhead, and dreaming of a cold pint.

bzr-search, loggerhead, gnome, and you

Robert Collins recently published his awesome bzr-search plugin, and John Carr has been doing a lot of work on setting up a bzr mirror of Gnome. A neat search module and a bunch of source trees is just begging to be combined in some sort of web interface!

There are a few web front ends for Bazaar at the moment, such as Loggerhead, webserve, viewbzr, and bzrweb. Today we are going to be focusing on Loggerhead (you can also go to its Launchpad project page to watch the development activity). It is probably the one with the most active development at the moment. An installation of the latest stuff in action is available at the bzr mirror of Gnome. Loggerhead shows side-by-side diffs, has RSS feeds, and lets you download specific changes, just like you would expect.

You can get the latest version of it yourself by doing:
bzr branch lp:loggerhead
You'll need python-simpletal and python-paste. Then by running "serve-branches.py" in the directory where you're branches live, you should be up and running with your own web interface. Eventually serve-branches.py is to expected to become a bzr plugin which will let you easily serve your branches with a single bzr command.

We hinted at it above; recent versions have started integrating with bzr-search. So for branches that you've run "bzr index" on, it can give hints in the search dialog, and quickly find revisions that match your search terms. You can try it yourself by just typing a few letters into the search dialog.

In the coming weeks, Loggerhead will be getting a bit of a face lift with a new theme to make its externals as shiny and new as its internals.

So give it a poke, and send any feedback to either bazaar@lists.canonical.com, or https://bugs.launchpad.net/loggerhead.