Using Git with BitTorrent Sync

The various cloud sync services provide a good way to backup or remote a Git repository for ‘single developer’ situations. The main advantages are that the cost is minimal and no ‘server’ is required. In this post we use something quite different, the new BitTorrent Sync (BTSync) system.

Use Cases
1. Single developer on multiple devices, PC, laptop, mobile. Also see Single Developer Git Workflow, .
2. Easy, fast, and lightweight code sharing setup. See also Sneakernet with Git
3. Ad Hoc Version Control: How to do Ad Hoc Version Control, Ad Hoc Version Control With Git

BTSync is a serverless folder syncing system. Instead of using a remote server storage system, it creates a fast private peer-to-peer file sync system using a P2P protocol. Note it is not necessarily a replacement for a server, backup system, or even other services such as DropBox, more like a welcome addition that covers some limits that others may have, such as file size limitations, speed, and privacy.

Using this type of service is very easy. I took the easy way out and “forked” a very well written blog post by Sergei Shvetsov that did the same thing, only using DropBox. Using Git with Dropbox. In this post, however, I use BTSync and I am running on Windows (most blogs show examples on *nix). Of course, experienced Git users may approach this very differently. The following uses a console shell UI.

1. Create a local repo
2. Create a “bare” repo that lives in the Synced folders
3. Add the bare repo as the origin of the local repo.

Now on another system, the synced folder, which contains the bare repo is available as if it was created locally. During development or other uses, since we are using the working repo and only occasionally the ‘bare’ or origin repo in the synced folder, the synced folder is not constantly transferring data over the network to any other synced locations (there can be many).

This approach is diagrammed below. (On mobile device this ASCII diagram using <PRE> tag looks horrible.) Two systems with the two Git repos:

+-----------------+                     +----------------+
|     Local       |                     |     Local      |
|     Repo        |                     |     Repo       |
+-----------------+                     +----------------+
      ^ +                                    +   ^
      | |                                    |   |
      | | push/pull                          |   |  push/pull
      | |                                    |   |
      | |                                    +   |
      + v                                    v   +
+-----------------+         BT Sync     +----------------+
|      Bare       | <-----------------+ |      Bare      |
|      repo       | +------------------>|      repo      |
+-----------------+                     +----------------+

Of course whenever possible direct access to remote repos from a clone is preferred, or via a server. For private use and 5 users, Bitbucket offers free code repositories.

Below is a walkthrough of this process using a Windows cmd shell.

Create a local repository

C:\temp\git-on-BTSync>git init new-project
Initialized empty Git repository in C:/temp/git-on-BTSync/new-project/.git/

C:\temp\git-on-BTSync>cd new-project
C:\temp\git-on-BTSync\new-project>echo "" > README.txt

git add .
C:\temp\git-on-BTSync\new-project>git commit -m "Initial Commit"
[master (root-commit) dcb3f2b] Initial Commit
 1 file changed, 1 insertion(+)
 create mode 100644 README.txt

Create a new ‘bare’ repo inside of local BTSynced folder

new-project>mkdir C:\Users\jbetancourt\BTSync\git
C:\new-project>git init --bare C:\Users\jbetancourt\BTSync\git\new-project.git
Initialized empty Git repository in C:/Users/jbetancourt/BTSync/git/new-project.git/

Add this new bare repo as upstream remote to the local repo

new-project>git remote add dropbox C:\Users\jbetancourt\BTSync\git\new-project.git

Push local changes to the bare repo

C:\temp\git-on-BTSync\new-project>git push -u dropbox master
Counting objects: 3, done.
Writing objects: 100% (3/3), 223 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To C:\Users\jbetancourt\BTSync\git\new-project.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from dropbox.

Use another system
On another system that is running BTSync, like a laptop, the bare repository folder is already synced. Now we can clone the repo.

cd \temp
mkdir remote-workspace
cd remote-workspace
git clone -o dropbox \Users\jbetancourt\BTSync\git\new-project.git
Cloning into 'new-project' ...
cd new-project
11/29/2013  10:48 AM      5 README.txt

Now make some changes
On the laptop we modify a file and commit it.

echo "Hello world" > README.txt
C:\temp\remote-workspace\new-project>type README.txt
"Hello world"
C:\temp\remote-workspace\new-project>git add README.txt

C:\temp\remote-workspace\new-project>git commit -m "changed readme"
[master 452b02e] changed readme
1 file changed, 1 insertion(+), 1 deletion(-)

Now push the changes to the local bare repo

C:\temp\remote-workspace\new-project>git push
Counting objects: 5, done.
Writing objects: 100% (3/3), 260 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To \Users\jbetancourt\BTSync\git\new-project.git
   dcb3f2b..452b02e  master -> master

Back to the original repo on the PC
We pull the changes that were automatically synced via BTSync into
the bare repo.

type README.txt
C:\temp\git-on-BTSync\new-project&gt;git pull
remote: Counting objects: 5, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From C:\Users\jbetancourt\BTSync\git\new-project
   dcb3f2b..452b02e  master     -> dropbox/master
Updating dcb3f2b..452b02e
 README.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

C:\temp\git-on-BTSync\new-project>type README.txt
"Hello world"

That was easy!

Issues with syncing git repos with BTSync?

  • BitTorrentSync has been updated and now the process of syncing folders is either easier or harder, depending on you viewpoint. There are also some limitation for the free product, like ten folder limit.
  • The machine having a source sync must be on to allow syncing, of course. Not true with a server based sync solution like Dropbox. This is only required while the local and remote folders are syncing of course. In many BTSync articles and blog posts the wrong impression is given that this is a continuous requirement. In fact, as soon as you see the sync complete, if you have the BTSync app visible you can shut down the source machine.
  • Damage is also synced: If one of the synced repos gets damaged, that damage is reproduced in all correlated syncs. This can be prevented by using BTsync’s read only share feature. This would introduce some limitations or other complexities.
  • Repository ignored files are synced
  • There was a discussion on whether the .git folder should be synced. Not sure I follow the rational.
  • I don’t know if there are any issues with BitTorrent Sync for long term work with a Git repo. People have complained of such issues with Dropbox. See the link for Mercurial use on DropBox below. In the comments of that blog post, robhamilton posts: “… found that it would break the Mercurial repo. Mercurial locks files and creates temp journal files which get sync’d by the dropbox daemon. My advice is to stop dropbox, perform your push/commit, then restart dropbox. Pulls and clones are readonly.” Is this an issue with Git? I don’t think so since we are using the bare repo approach.


    Mobile Git

  • Dec 24, 2013: I did not investigate the mobile Git use with BTSync as shown above. BTSync has a mobile app that allows the sync to mobile devices. On that device a mobile Git client can access the synced bare repo to clone into a mobile local repo. There are now a few mobile Git clients, for example, SGit.
  • June 26, 2015: Bittorrent Sync has an API: BitTorrent Gives Developers A Cloud-Free Alternative.

Some links

6 thoughts on “Using Git with BitTorrent Sync”

  1. Hello T.C. Mits, glad you liked my post.

    Came across your blog by following a pingback, and with it discovered BitTorrent Sync. Looks like an nice and secure alternative to Dropbox. This could be a good way to auto-sync photos from a mobile phone.

    Started reading your other posts and enjoying them so far.

    Best regards,

  2. Hi Sergei. Glad you stopped by. Yes, I liked your post and the others you write.

    Regarding BTSync. I use it all the time. Have had no issues. My next task with it is to be able to fill a folder on my home PC with music I want to play while I’m at work. By the time I’m on the road it will sync to phone and there will be today’s personal eclectic music mix to play. Just have to figure out how to get mobile “Google Play Music” app to see the folder, or change the location of the BTSync folder.

    Regards ….

  3. I’m going to have to chime in here. In the beginning, I sync’d SVN and GIT repos with Robocopy (yep, that long ago). Then Windows Live Mesh, that got killed. Windows Live Sync and Skydrive exclude too many files, so I moved to Dropbox. Had various locking file issues and corrupted files. Google Drive was never working with these repos.

    What I think the issue was with me, and with other commenters int he posts you linked to, is that we were attempting to sync our “working” directory. But even syncing the base repos, and push/pulling to them locally, still created some sync issues in the long term.

    What I ended up doing is exactly what you mentioned above: using Bitbucket private repos for everything not on Github. But still, this creates a “must remember to push local changes up” kind of thing, instead of the automatic sync.

    I’ve now ran across BtSync, which to me seems just like the Bitbucket sync pattern I am already using. Only difference is with bitbucket I have to tether on the commuter trains to create checkpoints, whereas with btsync I could push locally and let it sync when it gets online later on. That may be a benefit.

    But, I do want to stress one serious advantage I’ve had with Dropbox… I continue to use it for all of my non-Git files to sync on my Linux machines (Sublime user packages and settings, private ssh keys and scripts, etc). That one advantage, over Google Drive and OneDrive and any other file syncing service I’ve ever used:

    Dropbox Events

    This golden nugget has saved my ass multiple times and is the ONLY reason I use Dropbox. Events can be considered Event Streams, like in CQS development patterns such as Event Sourcing.

    Everything that happens is captured in an “Event.” Did you corrupt an install of something? Go into Dropbox and select that one event, and restore it to the previous state!

Leave a Reply

Your email address will not be published. Required fields are marked *