Ubuntu users should use the following command to install:

$ sudo python setup.py install –install-layout=deb

All other users should do:

$ sudo python setup.py install

Thanks,Adam

The pygtk world used to be quite simple and nice, with good documentation and well known best practices.  These were “static” gtk bindings that someone had to set up by hand (including me for nautilus-python).  These bindings were a pain in the butt to maintain over time but they worked pretty well and were predictable (good for coding against them).  However, a couple years ago the pygtk maintainers decided to abandon these bindings and start from scratch with “dynamic” bindings that would execute the original C code of the gtk library, which would have the effect of dramatically reducing the amount of hand-written binding code the pygtk developers would have to maintain.  These new dynamic bindings have been released for a while but are still pretty new to the world, but for the most part they work pretty well.  As you might be able to tell, the original Nautilus extension used the static pygtk bindings to run, and the new Nautilus 3 extension uses the new dynamic bindings to run.
Running Nautilus 2 with the static bindings works great, and running Nautilus 3 with the dynamic bindings works great as well, but problems start to occur when you try to run Nautilus (2 or 3) with both the static and dynamic bindings at the same time (this is possible).  Unfortunately this is what is happening for many of you right now.  Some extensions are written with the expectation that they can run with a mix of static and dynamic bindings, and if Nautilus 3 loads extensions that try to use the static and dynamic versions of a set of bindings at the same time, it will crash.

I’m not entirely sure what the solution to this is, though it will become less of a problem over time, as people switch completely over to the dynamic bindings.  However, for now, things will be a bit messy and you will have to decide which extensions you want to run.  Right now, the Nautilus 3 RabbitVCS extension uses dynamic bindings.  If you see the following error:

** (nautilus:2259): DEBUG: Syncdaemon not running, waiting for it to start in NameOwnerChanged
/usr/lib/python2.7/dist-packages/gobject/constants.py:24: Warning: g_boxed_type_register_static: assertion `g_type_from_name (name) == 0′ failed
import gobject._gobject
/usr/lib/python2.7/dist-packages/gtk-2.0/gtk/__init__.py:40: Warning: specified class size for type `PyGtkGenericCellRenderer’ is smaller than the parent type’s `GtkCellRenderer’ class size
from gtk import _gtk
/usr/lib/python2.7/dist-packages/gtk-2.0/gtk/__init__.py:40: Warning: g_type_get_qdata: assertion `node != NULL’ failed
from gtk import _gtk
Segmentation fault

Then it means you have another extension running that is trying to use the static bindings.  If you see this error (check in ~/.config/rabbitvcs/RabbitVCS.log) and your Nautilus isn’t able to run, first disable all nautilus-python extensions, then try to run RabbitVCS again.  Then one-by-one, enable your other extensions until Nautilus crashes again.  You should then report the issue either to myself or to the maintainer of said extension.

Hopefully this will resolve itself within a year or so, but until then it will be a somewhat bumpy ride.

Adam

Well, some of you may have heard about GNOME 3, which is scheduled to be released within a month or two. Unfortunately, the current versions of both nautilus-python and RabbitVCS are completely broken in GNOME 3 because PyGTK has not and will not be ported to GTK+3, and nautilus-python is what RabbitVCS uses to interface with the Nautilus extension framework.  The good news is that not only am I the maintainer for RabbitVCS, I’m the nautilus-python maintainer as well!  I’ve just about gotten the new bindings to work correctly on GNOME 3 and will put out a release as soon as possible.  They will require Nautilus 3 and the next bug-fix release of PyGObject 2.28 (I uncovered a bug in the current version).

The result of all of this is that I will need to port RabbitVCS’s Nautilus client to GTK+3, and possibly the rest of RabbitVCS as well.  It’s going to get a little hairy for a while as we figure out what to do.  So if anyone has any ideas, I’m all ears.

I’m also the maintainer of thunarx-python, which I started originally as a project on github.com, and I recently did my first release as an officially hosted Xfce project!  I just released v0.2.3 last weekend and the project is stable and ready to be packaged by distributions.

Once I get this nautilus-python release out the door, I will start focusing on RabbitVCS bugs once again and maybe make some more progress on advancing towards v0.15.

Happy St. Patricks Day!

Aside from my personal update, here are the latest development updates to whet your appetites.  First of all, git support is coming along slowly but surely, and it will be available for v0.14.  Since git adds so many new concepts to the UI and to our code, we’re going to have a fairly long beta period where I hope people will be interested enough to try it out, suggest improvements, and maybe even be energized enough to get involved in some coding.  I can’t make any guarantees, but I think this might happen before Christmas.  The git support for v0.14 will be somewhat rudimentary, but it should allow you to do most of the things you need for git version control.

I also spent some time this past weekend and worked on making the RabbitVCS nautilus extension truly non-blocking, and I succeeded!  The trick of it is that while attaching the correct emblems to files has been non-blocking for a while, the parts of the extension that ask for context menu items has not been.  What this meant is that when you were loading a working copy, everything would be fine until you actually started clicking on stuff.  Once you did that, Nautilus would freeze for some amount of time until the menu was generated.  Unfortunately, until recently, there was no way around this due to limitations in the nautilus-python bindings.  I say “until recently” because in January of this year I took over development of the nautilus-python bindings and removed the limitation.  So with the release of nautilus-python 0.7.0 in May, we can finally populate context menus in a non-blocking way.  The catch is that this will only work if you have nautilus-python 0.7.0 or greater installed.  I just checked the package databases for Fedora and Ubuntu, and v0.7.0 is available for Fedora but not for Ubuntu (I just emailed the nautilus-python maintainer for Debian, so maybe it’ll be available in the next release).  However, we can probably set up 0.7 in a PPA for folks that want the newest shiny ;)

In summary, things are moving in the right direction, but just quite slowly.  There are definitely opportunities available for people who are willing to jump in and lend a hand (programming, packaging, whatever) so don’t be shy.

I’ve wanted to write something about our performance problems for a while now. It’s a long standing issue, and something that is a real showstopper for some users. I’ve written this for two reasons: firstly, to tell our users why we haven’t “just fixed it” yet; secondly, so that other/future developers can get an idea of the WHY of the architecture and not just the HOW.

One of the oldest issues on the RabbitVCS issue tracker is issue number 4 — “Inadequate (sub-par) performance.” Hurtful, hurtful words, but ultimately true. The basic problem is that for small working copies, our extension works great. Once you get above about 50,000 items though, things really…

…Slow

…Down.

When you try to open up a folder with many version controlled children, Nautilus locks up. It is completely unrepsonsive until almost all the emblems are populated and — as much as I hate to say it — this renders RabbitVCS a little bit useless. Basically, we’re taking the I out of GUI.

Worst of all, it makes our “clever play on the word Tortoise” into a terrible, terrible irony.

What Performance Problems?

One of the first things  to understand is that performance is not actually the issue here. This may sound a little strange, but think about it: when our users say they want better performance, what they really seem to mean is that they want Nautilus to stay responsive. To put it another way, given the choice between:

1. Entering a versioned directory and waiting 1 minute for all the emblems to display correctly BUT have Nautilus lock up for the whole time; and
2. Entering a versioned directory and waiting 1 minute 20 seconds for all the emblems to display, but still being able to use Nautilus

…most users really seem to want (2), even though techincally (1) is the “better performance” option. Of course, if it were 20 minutes instead of one then yes, we’d have a problem. But that’s not what we’re experiencing at all. Status checks take the same amount of time, no matter what we do, no matter what language we use, no matter what the phase of the moon is. The call to vsc_client.status(path) will always always be our performance floor.

Now this might seem like minor semantic bickering — performance, responsiveness, who cares what you call it? But how you state a problem has a huge influence on how you deal with it (and how patient people are with your efforts…). Calling it a performance problem immediately implies a certain kind of solution, and causes well-meaning users to suggest things like JIT compilers and optimisers (bah), or rewriting in C (go on then). But if you stop obsessing over performance for five seconds, the solution becomes a little clearer. As Bruce and Jason (Field) realised, as soon as a Nautilus extension becomes non-trivial you need to start looking at how to do things asynchronously.

The simple story is:

1. Nautilus calls all of its Python extensions. They are all called in the same thread, and no other threading is permitted.
2. The Python extensions do their thing and return.
3. Nautilus displays the results.

So if step #2 takes minutes to complete, Nautilus is frozen waiting for a function call to return. That’s our problem.

Doing things in the background isn’t all that trivial though. For example, some users have suggested using threads to do things asynchronously. But threading is not permitted in Nautilus Python extensions, and so either the module will hang or just fail to run (depending on the direction of the local magnetic field). The next logical step is to use a separate process for the heavy lifting, so we have something like:

1. Nautilus calls all of its Python extensions.
2. The RabbitVCS extension calls a status checker service running in the background.
3. The status cache service should return immediately with either the real status or some indication that it will calculate the status (those cute little clock emblems).
4. The status is calculated.
5. The background service tells our extension when the status is calculated.
6. The extension tells Nautilus to invalidate the information for that path. This triggers another check, but now we have the status!
7. Profit.

This is what we’ve achieved so far for doing the emblems. We use DBUS to communicate with a background service that is automatically started on demand — we could have simple used a self-contained subprocess, but a DBUS service means that the other parts of RabbitVCS can access it too.

Diversions

The transition to DBUS was not as easy as it sounds. When we first moved things out to a DBUS controlled “subprocess,” we didn’t notice any improvement. This was because the Subversion bindings we use also prevent threading, and we were trying to intersperse status checks with status requests. This lead to me implementing a hideously complicated system of another layer of subprocesses to solve this. Really, we should have been using DBUS’ in-built asynchronous calling system to simplify things, which I finally got around to figuring out last week.

Another upshot of all this was what I learned about profiling. One of the most frustrating things for me was just how difficult it was to get a decent picture of what was taking so long in the first place. Profiling (in Python or otherwise) is an arcane art — unless you have a single process with a single thread, you’re going to get some mysterious results. The things that cause the most problems are thread synchronisation mechanisms, and if you’re lucky they’ll show up as huuuuuuge blocks of time spent waiting on a locking function. If you’re unlucky, you might as well throw darts at a stack trace. (Protip: print it out first.)

This isn’t a fundamental flaw with Python. But I was frustrated because the Python profiling tools aren’t designed to work on individual sections of the code. The most infuriating comment from #python regarding this was “which part of all this is cProfile.run(‘foo()’) ?” It’s a freaking embedded module inside a C extension API, not a shell script! Eventually I found a way to get a return value from cProfile so that I can profile the functions that Nautilus calls under normal use. (Of course, the good thing about subjecting yourself to #python is that the inhabitants do help you live up to the general Python philosophy of “if it seems harder than it should be, you’re doing it wrong, or possibly you’re an idiot with unsalvageable code; but let’s work on the other thing first.” I should have used DBUS properly from the start, and I should have remembered that it’s very rare for Python/C bindings to allow threading.)

So Close…

We’re now at the point where status emblems have almost no noticeable impact on Nautilus’ behaviour. Fantastic! So if we open up a test repository (in my case, sections 0-9 and A-D of the official Debian repository), Nautilus should just sit back and relax while waiting for everything to haaaaaappeeeeeeeeeoh the humanity it’s still freaking slow.

Here’s the problem: I forgot about the menus. Whenever Nautilus enters a new directory, it requests the menu items for the directory (“get_background_items”). This requires a status check, which yes indeed has to happen in the extension code. Nuts.

So why don’t we just do the same thing? Load up a menu that says “Please wait, loading…” and then use a callback to populate the menu? It’s a great idea, and the Nautilus extension API even has such a function: “nautilus_menu_provider_emit_items_updated_signal” which does exactly what it says on the box. Alas, the problem is with the Nautilus/Python bindings, which don’t allow us to use this method! The Nautilus/Python bindings have some real quirks in them, and it’s no trivial matter to simply pass Python objects and C interfaces around and expect things to work.

On the bright side, the Nautilus/Python bindings are now in the hands of Adam himself. It’s not always the case that cross-language bindings are in the hands of someone who uses them heavily enough to come up against constraints like this, so we’re at an advantage there. Adam has been hacking away to get the bindings to cover what we need, and while it’s not an easy task, it is certainly doable and should have some pretty good benefits even besides the menu signal. For example, once the proper Nautilus “update_file_info” callbacks are available, we can clean up a lot of hacky code and workarounds in the extension (possibly fixing a mysterious memory leak too).

I don’t want to promise anything too soon, but I do want to emphasise the fact that we are working on it, with an actual plan and everything! Keep your eye on issue #4, and if you’re a developer (or feeling brave), there will be a related branch in the repository soon. When we’ve got the code sufficiently functional, we’ll also release some packages for testing. So if you’re one of our users who has had to drop RabbitVCS because of performance problems, please hang in there — we are working on it, and it won’t be long now before we’ll be asking for all the real-world testing we can get.

I’m in the process of getting thunarx-python hosted on Xfce.org, but it is slow going.  I’ll update everyone when this does happen, though.

Get it here: thunarx-python-0.2.0.tar.gz

SHASUM: e5838c597612d90f262add39ceae56dc59ad798b
MD5SUM: 6b6555157888e7dd91a15732b4970642

Git Repository: http://github.com/adamplumb/thunarx-python
Bug Tracker: http://github.com/adamplumb/thunarx-python/issues

With RabbitVCS running on Thunar, we’re a small step closer to one of our big goals, which is to become file manager agnostic.

You can read my post on our mailing list about my trials and tribulations getting this all to work, but RabbitVCS on Thunar is a mixed bag.  On the one hand, I was able to get sub-menus working without having to patch Thunar, but on the other hand, there will be no emblem support for the foreseeable future.  However, one thing that Thunar has that Nautilus does not is a get_dnd_actions call, which asks plugins for menu items for when the user drags and drops an item into a folder. I haven’t implemented it yet, but in the near future this will allow you to move files around in your SVN repository via drag ‘n drop in Thunar.  Also not implemented yet is a property page that reports file status and ownership, etc.  All of that will be ready for the v0.13 release.

For now, I leave you with a short video of RabbitVCS running Thunar.

Get thunarx-python here: http://github.com/adamplumb/thunarx-python.  I’ve made a release, and you can get that in the Downloads section in github.  Currently, the only way to install the Thunar extension for RabbitVCS is to check out the latest code from SVN and run the development install.  Instructions for installing a thunarx-python plugin is available in the thunarx-python README file.  If you’re willing to brave the install, I’d love to hear your feedback!

Here is a quick ‘n dirty video I made showing some basic functionality.  Hopefully we’ll have an alpha or beta ready for v0.13 and more people can test the new stuff out.