Update to Release Announcement for RabbitVCS v0.13.1

Posted in Miscellaneous on March 27th, 2010 by Jason Heeris – 7 Comments

Sincere apologies to everyone! The package we uploaded still had some mixups in it from some restructuring we did. I’ve replaced the download on the Google Code project site, but beware that many web browsers can be rather aggressive in their caching of large files, so please check the file hash if you have any problems.

I’m working on the debs too :)

VN:F [1.9.22_1171]
Rating: 5.0/5 (2 votes cast)

Respite for Fedora 64 bit users

Posted in Miscellaneous on March 1st, 2010 by Adam Plumb – 10 Comments

A long standing bug in nautilus-python has caused problems for Fedora 64 bit users (and others) wanting to use RabbitVCS.  When nautilus-python is loaded, it loads the python shared object library that is in /usr/lib.  However, some 64 bit distributions put their python library in /usr/lib64, and nautilus-python isn’t smart enough to deal with this correctly.

This was actually fixed in the recently released nautilus-python 0.6 series, but existing distributions will generally not upgrade to major new releases (0.5 to 0.6), so the fix would only apply to new distribution releases.  Fortunately, there has just been a new nautilus-python release in the 0.5 series (0.5.2) which fixes this bug and a few others, and I believe Fedora will upgrade to it, making it available for all users.

Once the nautilus-python Fedora package is upgraded, we will release a 64 bit RPM for RabbitVCS.

VN:F [1.9.22_1171]
Rating: 3.8/5 (4 votes cast)

Would anyone care for a Gedit plugin?

Posted in Community, Miscellaneous on January 25th, 2010 by Adam Plumb – 11 Comments

With some time to kill before our big v0.13 release, I wanted to see what it would be like to create a RabbitVCS plugin for Gedit.  It turns out, it actually isn’t too terribly hard.  In fact, all I needed to do was implement the Gedit plugin parts and write a new class to generate menus that work with Gedit.  Currently, the plugin has the narrow focus of providing menu items for RabbitVCS actions, but in the future it could be expanded to providing a sidebar panel, or more.

Here is a quick demo video showing RabbitVCS-on-Gedit in action!

If you’re interested in running the Gedit plugin, you’ll need to be running up-to-date code from our SVN repository.  The Gedit bits are here.  If you’d like to wait for a release, it will be available for the beta release (coming next week).

I’d love it if someone would like to take over this plugin and really make it something special, as I really just wanted to get the ball rolling and show that it is possible to take RabbitVCS outside of the file manager domain.  If you’re interested in contributing to or even taking over development for the Gedit plugin, send an email to our developer’s mailing list.

VN:F [1.9.22_1171]
Rating: 5.0/5 (7 votes cast)

A preview of new log and compare functionality in v0.13

Posted in Miscellaneous, Status Update on October 30th, 2009 by Adam Plumb – 14 Comments

For me, and I think for a lot of other people, one of the key missing pieces of functionality for RabbitVCS v0.12 is robust support for comparing files, folders, and revisions; so this is the first thing I started working on for v0.13.  So far, I’ve updated the log viewer and created a new dialog devoted to comparisons, but I’ll be improving context menu functionality throughout all dialogs.  Here is a video preview of the new log view functionality and the compare dialog.

VN:F [1.9.22_1171]
Rating: 4.8/5 (6 votes cast)

The RabbitVCS Project as seen by CodeSwarm

Posted in Miscellaneous on October 17th, 2009 by Adam Plumb – Be the first to comment

Today, I happened upon the code_swarm project, which allows you visualize your project’s development over time.  One of the neatest things about it does everything from your repository’s log.  Here is what code_swarm spit out for RabbitVCS.  Pretty cool…

VN:F [1.9.22_1171]
Rating: 4.0/5 (4 votes cast)

See RabbitVCS v0.12 in action

Posted in Miscellaneous on October 7th, 2009 by Adam Plumb – 1 Comment

I’ve posted some videos of RabbitVCS in action on Youtube.  Here is the first of the series, checking out a simple working copy from a repository.

VN:F [1.9.22_1171]
Rating: 4.0/5 (2 votes cast)

The envisioned overall architecture

Posted in Miscellaneous on May 8th, 2009 by Bruce van der Kooij – 1 Comment

I just created this diagram of the envisioned overall architecture of NautilusSvn and thought it would be useful to share. It doesn’t completely represent the currently implemented architecture though it is quite close.

Overall architecture of NautilusSvn

The D-Bus service consists out of a status monitor which uses inotify to monitor the filesystem and a status cache to speed up status requests etc. The status monitor will notify all registered clients of any interesting actions, so information is pushed to clients. Note that it may be possible for clients to use the VCS abstraction layer directly and not have to pass through the D-Bus service if they so prefer.

VN:F [1.9.22_1171]
Rating: 4.7/5 (6 votes cast)

File access times and caching

Posted in Miscellaneous on May 8th, 2009 by Bruce van der Kooij – 1 Comment

I’ve been hanging around the Nautilus IRC channel for quite a few years already and have gotten quite familiar with the devs. Every single one of them is a great individual and always willing to lend a helping hand.

I thought I’d ask Alexander Larsson, a Red Hat employee and the lead developer for Nautilus and other projects including GVFS, for his insights as to why initial status checks take so much longer than consecutive ones. I already had a general idea as to why (mostly the idea of caching) but was not familiar with the details and quite interested in the opinion of somebody I consider a true expert.

The resulting answer was so informative I just had to take the time to share it with all of you. Here’s how Alex explained it:

Well, you know about caching I suppose? At this point we’re talking about the kernel using spare RAM to keep information about whats on the disk.

Say you start with a blank slate, i.e. you have not accessed the filesystem at all. Now say you run stat(“/some/dir/file”). First the kernel has to find the file, which in technical terms is called the inode. It starts by looking in the filesystem superblock, which stores the inode of the root directory. Then it opens the root directory, finds “some”, opens that, finds “dir”, etc. eventually finding the inode for file.

However, on a second access to /some/dir/file it uses the “dcache” (directory cache) which keeps around a set of recently accessed paths like /some, /some/dir, and /some/dir/file. So, it can now find the inode without any disk I/O.

Then you have to actually read the inode data. After first read this is also cached in RAM. So, a read only has to happen once.

I interupted Alex for a moment to ask him how long entries are maintained in the cache.

It depends on what else the system is doing. If you actually start reading the file data, that is also cached (unless you hint the kernel not to do it). In general Linux tries to use all memory that is not otherwise allocated for cache and has a form of least recently used policy for what to throw out. So, if there is any form of memory pressure, the oldest cache info is thrown out. So, reading lots of data is a good way to invalidate caches. Which is why there are ways to hint the kernel that the data read will not be reused.

He continued:

Now, if you look at a HD performance sheet you see pretty impressive performance figures, maybe a disk can read 10MB/s, which surely sounds a lot more than what some itty bitty svn info is. I mean, if your svn status took 1s does that mean it had to read 10 meg of data? The problem is that the read rates is when you read consecutive data from the disk.

Think of the HD like an old record player, once you’re in the right place with the needle you can keep reading stuff fast as it rotates. However, once you need to move to a different place, called “seeking” you’re doing something very different. You need to physically move the arm, then wait for the platter to spin until the right place is under the needle. This kind of physical motion is inherently slow so seek times for disks are pretty long.

So, when do we seek? It depends on the filesystem layout of course. Filesystems try to store files consecutively as to increase read performance, and they generally also try to store inodes for a single directory near each other but it all depends on things like when the files are written, filesystem fragmentation, etc. So, in the worst case, each stat of a file will cause a seek and then each open of the file will cause a second seek. So, thats why things take such a long time when nothing is cached.

Some filesystems are better than others, defragmentation might help. You can do some things in apps. For instance, GIO sorts the received inodes from readdir() before stating them hoping that the inode number has some sort of relation to disk order (it generally has) thus minimizing random seeks back and forth.

One important thing is to design your data storage and apps to minimize seeking. For instance, this is why Nautilus reading /usr/bin is slow, because the files in there generally have no extension we need to do magic sniffing for each. So, we need to open each file => one seek per file => slooooow. Another example is apps that store information in lots of small files, like gconf used to do, also a bad idea. Anyway, in practice I don’t think there is much you can do except try to hide the latencies.

I also asked Alex why Thunar was so fast at loading the same directories, however a few moments later it occurred to me that the reason I thought Thunar was fast might have been because I was opening /usr/bin in it after I had already previously opened it in Nautilus, therefor the seeking and caching Alex talked about had already occurred. Alex responded with a resonant “aha!”. This may well be the reason why so many people say Nautilus is slow compared to X, they’re probably doing the same thing.

He then explained to me how to clear the cache, it turns out all you have to do is execute the following command as root:

sync; echo 3 > /proc/sys/vm/drop_caches

After doing so you’ll see that Thunar will also take quite a while to load up a directory such as /usr/bin, because of the same reason Nautilus does. Another interesting tidbit Alex pointed out was timing gvfs-ls directly. He told me to try out the following commands and stated “you’ll be surprised”.

sync; echo 3 > /proc/sys/vm/drop_caches
time gvfs-ls -a "standard::content-type" /usr/bin/ > /dev/null
sync; echo 3 > /proc/sys/vm/drop_caches
time gvfs-ls  /usr/bin/ > /dev/null

Note that the first gvfs-ls command took 16 seconds and the latter took 1.5 seconds. He explained that the only difference is that the first one reads the first 2k of each file. With the answer he provided earlier it should be quite obvious why there’s such a difference between the two commands.

Alex ended with the following note:

The real fix for this whole dilemma is to move away from rotating media. I hear the intel SSDs are teh shit. Linus swears by them.

I hope you all found this information as interesting as I did.

VN:F [1.9.22_1171]
Rating: 5.0/5 (7 votes cast)

Minor update released for beta

Posted in Miscellaneous on March 16th, 2009 by Bruce van der Kooij – Be the first to comment

There were some packaging problems I felt warranted an update/rebuild. Here’s what got fixed:

  • Updated version dependencies in control file so the package can be installed under Ubuntu 8.04. See Issue #100.
  • Make sure that gtk-update-icon-cache is executed after install. See Issue #87.
  • Have to prepend emblem- to all emblem files for them to be picked up by Nautilus <= 2.22.3. See Issue #87.

Looks like we still have to work out our version numbering scheme. :-)

Next, performance issues! \o/

That’s all folks.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Change in strategy

Posted in Miscellaneous on March 11th, 2009 by Bruce van der Kooij – 3 Comments

Adam and I have come to the conclusion that the new status monitor that is needed to keep emblems up-to-date can not be included in the upcoming release, it’s simply not yet ready for production usage (among other things see Issues 4, 8, 89 and 90).

As we don’t see how we can significantly improve things with a few tweaks here or there we’ve decided to instead work on removing the status monitor completely for the time being and only have a status checker (similar to the situation in v0.11, but more elegant). We’ll also get the rest of the code up to production quality.

Regretfully, without the new status monitor we are not able to detect changes to files that are not directly visible in the Nautilus window. So if you’re in the directory /foo but the file /foo/bar/baz (baz) is added the emblems for foo and bar will not change to modified (assuming the status before was normal). This also means you cannot use the command-line svn client and expect emblems to be kept up-to-date.

UPDATE: actually after implementing the new status checker it turns out the above isn’t entirely correct. If you do alll VCS actions through NautilusSvn and don’t use the command-line all emblems will remain up-to-date. There seem to be two situations where we can’t keep emblems up-to-date:

  • If you’re in /foo and /foo/bar/baz is modified the status for /foo and /foo/bar will be incorrect. Nautilus doesn’t inform us about these modifications (which makes sense for normal extensions). The new status monitor is required to be able to do this.
  • If you’re in /foo and /foo/bar is “normal” and you add /foo/bar/baz by selecting NautilusSvn -> Add from the context menu of /foo. We’re only rechecking the paths you originally selected and any item you saw that was either “added”,  “deleted”, “replaced”, “modified”, “missing”, “unversioned”, or “incomplete”. There might be something I can think up here that might not impact performance all too badly.

Hopefully we will be able to get the status monitor ready for v0.13.

VN:F [1.9.22_1171]
Rating: 3.0/5 (1 vote cast)