Tech Notes And Miscellaneous Thoughts
 

why is bzr so slow?

from the whinge of the day dept:

I started a bzr branch of calibre about 2.5 hours ago because I wanted to see how difficult it would be to understand the code and make a few changes. The calibre Get Involved page warns that it can take about an hour….which is excessive to begin with and, worse, a huge understatement.

$ date ; ps u -Cbzr
Sat Apr 14 15:10:00 EST 2012
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
cas      14279  3.0  1.3 286836 215976 pts/3   S+   12:39   4:36 /usr/bin/python /usr/bin/bzr branch lp:calibre

It's still far from finished.

cas@ganesh:/usr/local/src/calibre$ bzr branch lp:calibre
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data.  See "bzr help launchpad-login".
1395260kB    65kB/s - Fetching revisions:Inserting stream:Estimate 178815/203940

a git clone of a much larger project would have been finished in mere minutes. WTF is bzr so horrendously slow?

It's not my connection, I'm on an otherwise idle ADSL2 connection, that syncs about 14Mbps.  i.e. capable of downloads of up 1.4 megabytes per second, not the 50-70kB/s that bzr is dawdling along at.  It's not my CPU, an AMD 1090T hex-core overclocked to 3.7GHz (and almost completely idle)…and since I have 16GB RAM, it's not lack of RAM either.

Is there anything good about bzr that makes people actually want to use it?  or is it just the association with ununtu and launchpad?

maybe the purpose is to actively discourage casual involvement….you have to make a massive life commitment and run the gauntlet of tediously long waits before you can even look at the code.

No wonder github is so popular.

 

17 Comments

  1. bzr branch lp:calibre took 7:20 min
    git clone https://github.com/sss/calibre.git took 5:10 min
    I understand that git is faster, but honestly, I wasted the two remaining minutes by checking if I actually checked something out with git, and what, and why my filemanager can not tell me if I have files in the checkout etc.  
    I think that version management should just work, because I find the topic extremely boring und I don't want to wast my time with it. a tool like bzr is just closer to not having to learn how to use it, but just beaing able to start. (I'm using both tools in different projects, but I really prefer bzr.)

    1. cas

      there's something seriously screwed up here – your 'bzr branch lp:calibre' took 7 minutes 20 seconds.

      mine took three and a half HOURS.

      I wouldn't care if it was only 7 mins.  or 10 mins. or thereabouts.  but 3.5 hours to clone a repo?

      thanks for the tip about the github calibre repo.  That only took 4m42s

       

      $ time git clone https://github.com/sss/calibre.git
      Cloning into 'calibre'...
      remote: Counting objects: 155864, done.
      remote: Compressing objects: 100% (30156/30156), done.
      remote: Total 155864 (delta 125006), reused 154484 (delta 123626)
      Receiving objects: 100% (155864/155864), 115.00 MiB | 378 KiB/s, done.
      Resolving deltas: 100% (125006/125006), done.
      
      real 4m42.175s	user 0m38.942s	sys 0m6.120s

      download speeds fluctuated quite a bit…at one point, it was peaking at 1.48MiB/s

      1. I really do not know why your bzr version takes so long. I just deleted everything and tried again:
        time bzr branch lp:calibre
        Branched 11903 revisions.                                                     

        real    6m9.980s
        user    4m38.633s
        sys    0m7.120s
         
        I don't think that this is a bzr or launchpad problem. More than 10 minutes should definitely not happen!

        1. AFAIU if you’ve got a Launchpad account, uploaded your SSH public key, and used bzr lp-login, bzr will use it’s smart protocol over ssh, which will be much faster than the dumb HTTP protocol it uses otherwise with Launchpad.

          There’s an open bug somewhere about making Launchpad support the smart bzr protocol for anonymous repository access too.

          1. cas

            thanks for the tip about using an ssh key and bzr launchpad-login. it made a significant difference, but bzr was still absurdly slow:

            $ time bzr branch lp:calibre
            The authenticity of host 'bazaar.launchpad.net (91.189.90.11)' can't be established.
            RSA key fingerprint is 9d:38:3a:63:b1:d5:6f:c4:44:67:53:49:2e:ee:fc:89.
            Are you sure you want to continue connecting (yes/no)? yes
            Warning: Permanently added 'bazaar.launchpad.net,91.189.90.11' (RSA) to the list of known hosts.
            Branched 11904 revisions.                                      
            
            real 43m33.937s	user 3m53.827s	sys 0m3.052s

            43 mins is a lot better than 3.5 hours. but nowhere near as a good as <5 mins for git.  I did (and still do) have another non-ssh bzr branch running in the background (started 17:49, it’s now 19:11), but that’s using maybe 60KB/sec of my 1.4MB/sec bandwidth.

             

  2. bzr at least manages to import svn repos from legacy projects on which git fails… (on the other hand, Launchpad regularily fails at importing my CVS live repos, so it may just be that LP sucks a bit more than bzr alone).

    As for the communities – both are helpful and fast.

    1. Ah, another thing: bzr appears to just access storage on the server, whereas git actually spawns a process on the server that creates a “pack” for you.

      Which may take down the entire server VM if it only has 1 GiB RAM and ½ GiB swap, and someone just committed a 380 MiB BLOB to the git repository (and a few others), letting it grow from ~280 MiB to about one GiB in a single commit.

      Took me a while to find out how to revert that… hell, even svn never was that ressource intensive, although it’s the slowest VCS I know of and very heavy on the client.

  3. jimis

    Which version of bzr did you have installed? My experience is that anything less than 2.4 can be awfully slow, so try using 2.4 or 2.5. Keep in mind that you need python >= 2.6, could be a problem if you are stuck in old enterprise distro.

    1. cas

      bzr version was whatever was in debian sid at the time.

      according to /var/log/dpkg.log, that would have been 2.6.0~bzr6520-1, upgraded a few days before this post, on April 11:

      2012-04-11 11:41:49 upgrade bzr:all 2.6.0~bzr6511-2 2.6.0~bzr6520-1
      
      1. jimis

        Strange, Bazaar 2.6.0 is still in beta today, 2.5.1 is the latest stable. Anyway, I guess that’s why “sid” is also called unstable. :-)

        Other than checking your ~/.bzr.log for strange output, I guess I’m out of ideas. Maybe it’s related to your location (high ping times?), maybe file a bug report…

  4. L.Arnold

    1-26-13. Working on a OpenERP install. Yesterday did a “slow” but by no means terrible Apt-Get and all came quick.

    BZR went well on the first batc (opneobject -server 7.0) but is CRAWLING on openobject -addons 7.0 branch.

    Console says it is cruising, 500KB sec, but takes about 10 times that log to actually show some progress. I expect I am 4 hours in w/ 72000KB. Machine has plenty of HD and Ram still available.

    I did not login to LaunchPad. Looks like I should have.

  5. Brian Minton

    bminton@bminton:~/src$ bzr lp-login
    bjmgeek
    bminton@bminton:~/src$ time bzr branch lp:calibre
    Branched 15111 revisions.

    real 86m28.960s
    user 9m14.156s
    sys 0m35.669s

  6. Brian Minton

    bzr –version
    Bazaar (bzr) 2.6.0dev3
    Python interpreter: /usr/bin/python 2.7.5
    Python standard library: /usr/lib/python2.7
    Platform: Linux-3.9.0-x86_64-with-debian-jessie-sid
    bzrlib: /usr/lib/python2.7/dist-packages/bzrlib
    Bazaar configuration: /home/bminton/.bazaar
    Bazaar log file: /home/bminton/.bzr.log

    Copyright 2005-2012 Canonical Ltd.

Comments are closed.