Category: git

Using Git with BitTorrent Sync

The various cloud sync services provide a good way to backup or remote a Git repository for ‘single developer’ situations. The main advantages are that the cost is minimal and no ‘server’ is required. In this post we use something quite different, the new BitTorrent Sync (BTSync) system.

Use Cases
1. Single developer on multiple devices, PC, laptop, mobile. Also see Single Developer Git Workflow, .
2. Easy, fast, and lightweight code sharing setup. See also Sneakernet with Git
3. Ad Hoc Version Control: How to do Ad Hoc Version Control, Ad Hoc Version Control With Git

BTSync is a serverless folder syncing system. Instead of using a remote server storage system, it creates a fast private peer-to-peer file sync system using a P2P protocol. Note it is not necessarily a replacement for a server, backup system, or even other services such as DropBox, more like a welcome addition that covers some limits that others may have, such as file size limitations, speed, and privacy.

Using this type of service is very easy. I took the easy way out and “forked” a very well written blog post by Sergei Shvetsov that did the same thing, only using DropBox. Using Git with Dropbox. In this post, however, I use BTSync and I am running on Windows (most blogs show examples on *nix). Of course, experienced Git users may approach this very differently. The following uses a console shell UI.

Process
1. Create a local repo
2. Create a “bare” repo that lives in the Synced folders
3. Add the bare repo as the origin of the local repo.

Result
Now on another system, the synced folder, which contains the bare repo is available as if it was created locally. During development or other uses, since we are using the working repo and only occasionally the ‘bare’ or origin repo in the synced folder, the synced folder is not constantly transferring data over the network to any other synced locations (there can be many).

This approach is diagrammed below. (On mobile device this ASCII diagram using <PRE> tag looks horrible.) Two systems with the two Git repos:

+-----------------+                     +----------------+
|     Local       |                     |     Local      |
|     Repo        |                     |     Repo       |
+-----------------+                     +----------------+
      ^ +                                    +   ^
      | |                                    |   |
      | | push/pull                          |   |  push/pull
      | |                                    |   |
      | |                                    +   |
      + v                                    v   +
+-----------------+         BT Sync     +----------------+
|      Bare       | <-----------------+ |      Bare      |
|      repo       | +------------------>|      repo      |
+-----------------+                     +----------------+

Alternatives
Of course whenever possible direct access to remote repos from a clone is preferred, or via a server. For private use and 5 users, Bitbucket offers free code repositories.

Example
Below is a walkthrough of this process using a Windows cmd shell.

Create a local repository

C:\temp\git-on-BTSync>git init new-project
Initialized empty Git repository in C:/temp/git-on-BTSync/new-project/.git/

C:\temp\git-on-BTSync>cd new-project
C:\temp\git-on-BTSync\new-project>echo "" > README.txt

git add .
C:\temp\git-on-BTSync\new-project>git commit -m "Initial Commit"
[master (root-commit) dcb3f2b] Initial Commit
 1 file changed, 1 insertion(+)
 create mode 100644 README.txt

Create a new ‘bare’ repo inside of local BTSynced folder

new-project>mkdir C:\Users\jbetancourt\BTSync\git
C:\new-project>git init --bare C:\Users\jbetancourt\BTSync\git\new-project.git
Initialized empty Git repository in C:/Users/jbetancourt/BTSync/git/new-project.git/

Add this new bare repo as upstream remote to the local repo

new-project>git remote add dropbox C:\Users\jbetancourt\BTSync\git\new-project.git

Push local changes to the bare repo

C:\temp\git-on-BTSync\new-project>git push -u dropbox master
Counting objects: 3, done.
Writing objects: 100% (3/3), 223 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To C:\Users\jbetancourt\BTSync\git\new-project.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from dropbox.

Use another system
On another system that is running BTSync, like a laptop, the bare repository folder is already synced. Now we can clone the repo.

cd \temp
mkdir remote-workspace
cd remote-workspace
git clone -o dropbox \Users\jbetancourt\BTSync\git\new-project.git
Cloning into 'new-project' ...
done.
cd new-project
dir
11/29/2013  10:48 AM      5 README.txt

Now make some changes
On the laptop we modify a file and commit it.

echo "Hello world" > README.txt
C:\temp\remote-workspace\new-project>type README.txt
"Hello world"
C:\temp\remote-workspace\new-project>git add README.txt

C:\temp\remote-workspace\new-project>git commit -m "changed readme"
[master 452b02e] changed readme
1 file changed, 1 insertion(+), 1 deletion(-)

Now push the changes to the local bare repo

C:\temp\remote-workspace\new-project>git push
Counting objects: 5, done.
Writing objects: 100% (3/3), 260 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To \Users\jbetancourt\BTSync\git\new-project.git
   dcb3f2b..452b02e  master -> master

Back to the original repo on the PC
We pull the changes that were automatically synced via BTSync into
the bare repo.

type README.txt
""
C:\temp\git-on-BTSync\new-project&gt;git pull
remote: Counting objects: 5, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From C:\Users\jbetancourt\BTSync\git\new-project
   dcb3f2b..452b02e  master     -> dropbox/master
Updating dcb3f2b..452b02e
Fast-forward
 README.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

C:\temp\git-on-BTSync\new-project>type README.txt
"Hello world"

That was easy!

Issues with syncing git repos with BTSync?

  • BitTorrentSync has been updated and now the process of syncing folders is either easier or harder, depending on you viewpoint. There are also some limitation for the free product, like ten folder limit.
  • The machine having a source sync must be on to allow syncing, of course. Not true with a server based sync solution like Dropbox. This is only required while the local and remote folders are syncing of course. In many BTSync articles and blog posts the wrong impression is given that this is a continuous requirement. In fact, as soon as you see the sync complete, if you have the BTSync app visible you can shut down the source machine.
  • Damage is also synced: If one of the synced repos gets damaged, that damage is reproduced in all correlated syncs. This can be prevented by using BTsync’s read only share feature. This would introduce some limitations or other complexities.
  • Repository ignored files are synced
  • There was a discussion on whether the .git folder should be synced. Not sure I follow the rational.
  • I don’t know if there are any issues with BitTorrent Sync for long term work with a Git repo. People have complained of such issues with Dropbox. See the link for Mercurial use on DropBox below. In the comments of that blog post, robhamilton posts: “… found that it would break the Mercurial repo. Mercurial locks files and creates temp journal files which get sync’d by the dropbox daemon. My advice is to stop dropbox, perform your push/commit, then restart dropbox. Pulls and clones are readonly.” Is this an issue with Git? I don’t think so since we are using the bare repo approach.

Updates

    Mobile Git

  • Dec 24, 2013: I did not investigate the mobile Git use with BTSync as shown above. BTSync has a mobile app that allows the sync to mobile devices. On that device a mobile Git client can access the synced bare repo to clone into a mobile local repo. There are now a few mobile Git clients, for example, SGit.
  • June 26, 2015: Bittorrent Sync has an API: BitTorrent Gives Developers A Cloud-Free Alternative.

Some links

Ant hooks using Groovy, continued

The Ant Hook scripts using Groovy implementation is extended with the ability to also skip target execution. Thus, an “around” advice AOP like or filter.

In the last post, Ant Hooks using Groovy Script via Scriptdef, we used the Ant BuildListener interface to add a hooks feature that invokes a Groovy script mapped to build events. Now we implement a true “around advice”. Admittingly kludgy, but ….

Approach
Since the hook script has the Ant Event object, we have access to the Target object currently about to be executed. Target has a convenient method to allow target skip:

public void setUnless(java.lang.String property):  
 Sets the "unless" condition to test on execution. 
 if the property is set, the task will not execute.

Demo
The demo Ant build script is modified with two new targets, ‘deploy’ and ‘build’. Also, the default target is changed to ‘build’.

<project name="demo1" default="build" basedir=".">
    ............ see original code .....
     
    <target name="compile">
        <echo>Hello compile world!</echo>  
    </target> 

    <target name="deploy">
        <echo>Deploying ... ${skipSet}</echo>
    </target>
    
    <target name="build" depends="compile,deploy">
        <echo>Building ... </echo>
    </target>

</project>

A new hook script to filter the ‘deploy’ target and skip it is added.
demo1/deploy_targetStarted.groovy

println "  hook: {project=${event.project.name},target=${event.target.name},when=pre,event=$event}"
hook.skipTarget(event)

Listener implementation
The actual listener is slightly modified to test for the skip property. If found the ‘unLess’ method is used on the target and the target is skipped.

/**
 * Ant build listener that invokes groovy hook scripts.
 *  
 * @author josef betancourt
 *
 */
class HookListener implements SubBuildListener {
    
    ...... see prior post ......

    /** Invoke the 'started' or 'finished' root or target hook script */
    def evokeTargetHook(BuildEvent event, When when){
        def b = new Binding()
        b.event=event
        b.hook=this

        def shell = new GroovyShell(b)
        
        def hookName = "${event.target.name}_${TARGETHOOK}${when.name}"
        
        def hooked = false
        
        def stored = projectHooks[hookName]        
        if(stored){
            hooked = true
            shell.evaluate(stored)
        }else{ 
            def hook = rootHooks[hookName]            
            if(hook){
                hooked = true
                shell.evaluate(hook)
            }
        }        
        
        if( hooked && (when == When.STARTED) ){
            def skipSet = event.project.getProperty(
                 "${event.target.name}_skipTarget")

            if(skipSet){
                event.project.setProperty(skipSet, "true")
                event.target.setUnless(skipSet)
            }            
        }
    }    

    /** invoked from a hook script */
    static void skipTarget(BuildEvent event) {
        def name = event.target.name
        def project = event.project
        project.setProperty(name + "_skipTarget", 'set')
    }

    ...... see prior post ......
}

    ...... see prior post ......

Output

Buildfile: C:\Users\jbetancourt\workspace-4.3\AntAroundAdvice\build.xml
compile:
  hook: {project=demo1,target=compile,when=pre,event=org.apache.tools.ant.BuildEvent}
     [echo] Hello compile world!
 hook: root,{target=compile,when=post,event=org.apache.tools.ant.BuildEvent}
deploy:
  hook: {project=demo1,target=deploy,when=pre,event=org.apache.tools.ant.BuildEvent}
build:
     [echo] Building ... 
BUILD SUCCESSFUL

Further reading

JGit Examples?

On a StackOverflow question asking about example source code, one answer pointed to a very good resource: jgit-cookbook.

JGit is a lightweight pure Java library implementing the Git version control system. It is used by the EGit team provider in Eclipse among other places.

References

Git does subdirectory globs

Finally more tools are starting to use this convention. I first saw its use in the Ant build framework. I even mentioned it to some Linux nerd on how to improve his utility and got an angry rebuff. Oh well.

From the Git v1.8.2 release notes:

* The patterns in .gitignore and .gitattributes files can have **/, as a pattern that matches 0 or more levels of subdirectory. E.g. “foo/**/bar” matches “bar” in “foo” itself or in a subdirectory of “foo”.
 

Links

Git status not showing changes or new files?

In a git repository created using the –separate-git-dir option executing “git status” is not showing any changes.

In a git repository created using the –separate-git-dir option executing “git status” is not showing any changes. This was working correctly for weeks.

Details
I created a new file in the top level folder, it doesn’t show as untracked. Tried “git status -u -v”. Nothing. Removed the .gitignore file. Nothing. Strange.

The separate directory is on a shared drive in Windows with a connected drive “k:”.
“git fsck” shows no issue, etc.

Work around
Anyway, what I did. I deleted the .git file. Then I reinited:

git init –separate-git-dir K:/where/it/is/at.git

Now it works.

Details
OS: Windows 7, 64bit.
Git: 1.7.9.msysgit.0

Current Git repo branch name using Groovy

As some light research I was looking at how to access a Git repository’s current branch name in a Groovy script. Could use a process invocation or a library like JGit

As some light research I was looking at how to determine a Git repository’s current branch name in a Groovy script.

In the Git CLI this would be shown with:

C:\temp\git\SdWf\project>git branch
  d12345-validation-fails
  d54321-login-box
  f52123-error-report
* master

The current branch is shown with a leading asterisk.

Invoke Git
One method is to just invoke the Git executable:

gitExe = "serversGitbingit.exe"
currentBranch = ''

matcher = ~/* (.*)s/

process = "${gitExe} branch".execute()
process.in.eachLine{ 
     line -&gt; println line
     m = line =~ /*s+(.*)s?/
     if(m){ 	
          currentBranch = m[0][1] 
          return
     }
}

println "current: '$currentBranch'"

Use JGit library
Another is to use a Java based Git library like JGit:

@GrabResolver(
     name='jgit-repository', 
     root='http://download.eclipse.org/jgit/maven'
)
@Grab(
     group='org.eclipse.jgit',
     module='org.eclipse.jgit',
     version='1.1.0.201109151100-r'
)

import org.eclipse.jgit.*
import org.eclipse.jgit.lib.*
import org.eclipse.jgit.util.*

directory = new File(args[0])

Repository repository =
  RepositoryCache.open(
       RepositoryCache.FileKey.lenient(directory,FS.DETECTED), 
       true)
	
println(repository.getFullBranch())

This would be run as:
groovy test\Git.groovy temp\git\SdWf\depot\project.git
refs/heads/master

One problem with the JGit approach is that, afaik, this won’t work if the repo is represented by a ‘pointer’ file, i.e., created with the –separate-git-dir option. Also, unlike the Git branch command, you must open the target git repo at it’s root level, not a nested subdirectory of the repo.

Environment
— Windows 7 64bit Professional
— git version 1.7.6.msysgit.0
— Groovy Version: 1.8.2 JVM: 1.6.0_25

References

Single Developer Git Workflow

I give an example of the simple workflow I’m currently using in a single developer single location Git workflow.

I’m finding Git to be very useful. Since I use it in conjunction with the “official” VCS at my job, I classify this as a SDGWF situation. Below I give an example of the simple workflow I’m currently experimenting with. The following is based on a Windows development workstation. Usually any blog posts involving Git are *nix based.

What is shown?

  1. Use of a bare repository
  2. Branching
  3. Merging

Note: This is an old post, and there are hopefully much better examples on web about single-developer workflows. The Simplified Git Process blog post is very good.

log in gui
graphic log

Two local repos
Creating the two repositories:

cd \temp\git\SdWf
C:\temp\git\SdWf>mkdir project
C:\temp\git\SdWf>mkdir depot
C:\temp\git\SdWf>cd depot

C:\temp\git\SdWfdepot>git init --bare project.git
Initialized empty Git repository in C:/temp/gitSdWf/depot/project.git/

C:\temp\git\SdWfdepot>cd ..\project
C:\temp\git\SdWf\project>git init --separate-git-dir ..\depot\project.git
Reinitialized existing Git repository in C:/temp/gitSdWf/depot/project.git/
C:\temp\git\SdWf\project>echo hello world! > hello.txt
C:\temp\git\SdWf\project>git add .
C:\temp\git\SdWf\project>git commit -m "initial commit"
[master (root-commit) e935e02] initial commit
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 hello.txt

C:\temp\git\SdWf\project>git log
commit e935e0220ab90945f6160db00eee6bfd13173b6d
Author: josef betancourt 
Date:   Mon Oct 3 19:07:29 2011 -0400
    initial commit

Note that the first repository created is “bare“, i.e., it does not have any work files. It is not created in the project work folder. At the project folder, a new Git repository is created, but with the –separate-git-dir option the first repository is used as the actual project repository.

Instead of a .git subdirectory, a .git file is created at the project folder, and this file points to the actual Git repo.

Why use this setup?
One reason is to allow tools to transparently work with the sanctioned project folder. For example, if we use the ‘normal’ repo setup, to zip the project, one would have to filter out the
.git subfolder (which could get large). Another reason is to allow backups of the working repo; one can delete a project and inadvertently delete the local repo. The last two reasons could be avoided by not creating the local repo within the project folder, of course.

Usage examples
Next day we are assigned some tasks at work. For this example, we’ll make up a naming approach:
defect: dNNNNN-name
feature: fNNNNN-name

Also, for this example, we’ll just have two defects and one feature:
d12345-validation-fails
d54321-login-box
f52123-error-report

Rather then waiting to create the branches when the branch is needed, we just create them now:

git branch d12345-validation-fails
git branch d54321-login-box
git branch f52123-error-report

Our repository now has four branches:

C:\temp\git\SdWf\project>git branch | sort
  d12345-validation-fails
  d54321-login-box
  f52123-error-report
* master

Let’s work on the first defect.

git checkout b12345-validation-fails

C:\temp\git\SdWf\project>git branch | sort
  d54321-login-box
  f52123-error-report
  master
* d12345-validation-fails

Make this branch is up to date against the master branch:

C:\temp\git\SdWf\project>git merge master
Already up-to-date.

C:\temp\git\SdWf\project>git commit -a -m "worked on defect"
[d12345-validation-fails 0aa5930] worked on defect
 1 files changed, 1 insertions(+), 1 deletions(-)

C:\temp\git\SdWf\project>git diff master

diff --git a/hello.txt b/hello.txt
index d4f3cf2..9f2b679 100644
--- a/hello.txt
+++ b/hello.txt
@@ -1 +1 @@
-hello world!
+initial hello ^M

Work on the other defect:

C:\temp\git\SdWf\project>git checkout d54321-login-box
Switched to branch 'd54321-login-box'

C:\temp\git\SdWf\project>git merge master
Already up-to-date.

C:\temp\git\SdWf\project>git checkout d54321-login-box
Switched to branch 'd54321-login-box'

C:\temp\git\SdWf\project>git merge master
Already up-to-date.

C:\temp\git\SdWf\project>echo Have to go. >> hello.txt

C:\temp\git\SdWf\project>git commit -a -m "made some fixes"
[d54321-login-box db17a9e] made some fixes
 1 files changed, 1 insertions(+), 0 deletions(-)

C:\temp\git\SdWf\project>git diff master

diff --git a/hello.txt b/hello.txt
index d4f3cf2..7fe577f 100644
--- a/hello.txt
+++ b/hello.txt
@@ -1 +1,2 @@
 hello world!
+Have to go. ^M

Now lets switch to the feature branch …


C:\temp\git\SdWf\project>git checkout f52123-error-report
Switched to branch 'f52123-error-report'

C:\temp\git\SdWf\project>git merge master
Already up-to-date.

C:\temp\git\SdWf\project>echo Created new feature >> hello.txt

C:\temp\git\SdWf\project>git diff master
diff --git a/hello.txt b/hello.txt
index d4f3cf2..17244a7 100644
--- a/hello.txt
+++ b/hello.txt
@@ -1 +1,2 @@
 hello world!
+Created new feature ^M

C:\temp\git\SdWf\project>git commit -a -m "finished the feature"
[f52123-error-report aef7f49] finished the feature
 1 files changed, 1 insertions(+), 0 deletions(-)



C:\temp\git\SdWf\project>git log --graph
* commit e935e0220ab90945f6160db00eee6bfd13173b6d
  Author: josef betancourt 
  Date:   Mon Oct 3 19:07:29 2011 -0400

      initial commit

Lets log all the branches ….


C:\temp\git\SdWf\project>git log --graph --branches
* commit aef7f4990b63dbb8da4f81558ac492609afa3522
| Author: josef betancourt 
| Date:   Mon Oct 3 19:33:09 2011 -0400
|
|     finished the feature
|
| * commit db17a9eb9e0672b0e6d2b9a9bef645a5d3a6e1de
|/  Author: josef betancourt 
|   Date:   Mon Oct 3 19:29:41 2011 -0400
|
|       made some fixes
|
| * commit 0aa5930c88d619b58ce14908ba8795736988d46c
|/  Author: josef betancourt 
|   Date:   Mon Oct 3 19:26:05 2011 -0400
|
|       worked on defect
|
* commit e935e0220ab90945f6160db00eee6bfd13173b6d
  Author: josef betancourt 
  Date:   Mon Oct 3 19:07:29 2011 -0400

      initial commit

Show the detailed branch structure …

C:\temp\git\SdWf\project>git show-branch
! [d12345-validation-fails] worked on defect
 ! [d54321-login-box] made some fixes
  ! [f52123-error-report] finished the feature
   * [master] initial commit
----
  +  [f52123-error-report] finished the feature
 +   [d54321-login-box] made some fixes
+    [d12345-validation-fails] worked on defect
+++* [master] initial commit

Now check out master branch and merge the first defect …

git checkout master
C:\temp\git\SdWf\project>git merge --no-ff d12345-validation-fails
Merge made by recursive.
 hello.txt |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

C:\temp\git\SdWf\project>git merge --no-ff d54321-login-box
Auto-merging hello.txt
CONFLICT (content): Merge conflict in hello.txt
Automatic merge failed; fix conflicts and then commit the result.

Yuck, a merge conflict. What was the conflict?


C:\temp\git\SdWf\project>type hello.txt
<<<<<<>>>>>> d54321-login-box

C:\temp\git\SdWf\project>npp hello.txt

C:\temp\git\SdWf\project>"C:Program Files (x86)Notepad++notepad++.exe" hello.txt

Edit the file and accept the changes, then merge …


C:\temp\git\SdWf\project>type hello.txt
initial hello
hello world!
Have to go.

C:\temp\git\SdWf\project>git status
# On branch master
# Unmerged paths:
#   (use "git add/rm ..." as appropriate to mark resolution)
#
#       both modified:      hello.txt
#
no changes added to commit (use "git add" and/or "git commit -a")

C:\temp\git\SdWf\project>git commit -a -m "merged in 2nd defect"
[master cc7ebf6] merged in 2nd defect

Now merge the feature branch into master …


C:\temp\git\SdWf\project>git merge --no-ff f52123-error-report
Auto-merging hello.txt
CONFLICT (content): Merge conflict in hello.txt
Automatic merge failed; fix conflicts and then commit the result.

C:\temp\git\SdWf\project>type hello.txt
initial hello
hello world!
<<<<<<>>>>>> f52123-error-report

Edit the file, then commit the merge …


C:\temp\git\SdWf\project>npp hello.txt

C:\temp\git\SdWf\project>"C:Program Files (x86)Notepad++notepad++.exe" hello.txt

C:\temp\git\SdWf\project>type hello.txt
initial hello
hello world!
Have to go.
Created new feature
C:\temp\git\SdWf\project>git commit -a -m "merged in feature"
[master dc58815] merged in feature

Show the final results using a log with graphics ….


*   commit dc58815ae250ee089e9e83db3b6e301a5a4552a9
|  Merge: cc7ebf6 aef7f49
| | Author: josef betancourt 
| | Date:   Mon Oct 3 19:57:14 2011 -0400
| |
| |     merged in feature
| |
| * commit aef7f4990b63dbb8da4f81558ac492609afa3522
| | Author: josef betancourt 
| | Date:   Mon Oct 3 19:33:09 2011 -0400
| |
| |     finished the feature
| |
* |   commit cc7ebf6c79fc0abb82b2b0d1fd5e53bcbf4fb94c
|   Merge: 8e47975 db17a9e
| | | Author: josef betancourt 
| | | Date:   Mon Oct 3 19:55:25 2011 -0400
| | |
| | |     merged in 2nd defect
| | |
| * | commit db17a9eb9e0672b0e6d2b9a9bef645a5d3a6e1de
| |/  Author: josef betancourt 
| |   Date:   Mon Oct 3 19:29:41 2011 -0400
| |
| |       made some fixes
| |
* |   commit 8e479756f7db2b926259c9774d5fd8cb38d14935
|   Merge: e935e02 0aa5930
| |/  Author: josef betancourt 
|/|   Date:   Mon Oct 3 19:51:30 2011 -0400
| |
| |       Merge branch 'd12345-validation-fails'
| |
| * commit 0aa5930c88d619b58ce14908ba8795736988d46c
|/  Author: josef betancourt 
|   Date:   Mon Oct 3 19:26:05 2011 -0400
|
|       worked on defect
|
* commit e935e0220ab90945f6160db00eee6bfd13173b6d
  Author: josef betancourt 
  Date:   Mon Oct 3 19:07:29 2011 -0400

      initial commit

That’s a simple example. I’m sure I missed some steps during the text captures.

Currently I’m just using the “master” branch as the development branch. I only use branches for short term coding. I may go back to using more branches as I get better with Git.

Further reading

Debugging: copying .git folder hangs Windows Explorer

Windows Explorer started hanging when using copy & paste of folders. I noticed that it only involved folders that had a “.git” subfolder.

Windows Explorer started hanging when using copy & paste of folders. I noticed that it only involved folders that had a “.git” subfolder.

Symptoms

  • Copy a folder and explorer does not refresh; you have to hit F5.
  • After the manual refresh, click on a folder and the explorer gets an hour glass.
  • The explorer.exe CPU usage reaches 50% and higher.
  • Only happens when a subfolder was named “.git”.
  • .git folder could even be empty.

Easy approach unsuccessful
I uninstalled Git and TortoiseGit then restarted the system. Still happening. Hmmm, that should have removed everything. Nope, I looked in the Registry and TortiseGit was still hanging around. I removed all traces and then even used ShellExView tool and could not find any shell extension that could be doing this. Using that tool I disabled all non-Windows shell extensions. Still happening.

More tools to help
This happened a month ago so I don’t remember all the steps I took. I used some of the SysInternals tools but they didn’t help much; you need a lot of internal Windows details to really use some of them.

However, I think it was using the Process Manager and looking only at file activity that all the info finally gave me a clue.

Progress

5:23:14.4666723 PM	explorer.exe	5096	ReadFile	C:Documents and Settingst16205Start MenuLocal Disk (C)target.lnk	SUCCESS	Offset: 0, Length: 299
5:23:14.4668257 PM	explorer.exe	5096	QueryInformationVolume	C:Documents and Settingst16205Start MenuLocal Disk (C)target.lnk	SUCCESS	VolumeCreationTime: 9/15/2009 8:47:39 AM, VolumeSerialNumber: 64B6-6315, SupportsObjects: True, VolumeLabel:

It seemed to be hanging at that point!

Solution
I looked in the Windows Start Menu list and saw that I had placed a link of the C: drive there. Not a junction or anything fancy, just a drag and drop of a shortcut (I think). I removed that link and the problem was solved.

Back to Git
Reinstalled Git and all is fine, knock on wood.

No explanation yet
Not really debugging, just floundering with a purpose. But, why would this only effect folders named “.git”, when even Git is not even installed. Strange.

OS
Windows XP Professional

Git
TortoiseGit
mysysGit

Tools
WhatIsHang v 1.10:
ShellExView v 1.66
Process Explorer v15.04
Process Monitor v 2.96
brain

Sneakernet with Git?

So your Git repo is on a USB stick. How do you merge it back into the real repo?

When remote connection is not possible, you can put a Git repo on a stick. But how do you merge without getting so many conflicts?

Why would you get so many conflicts? Well, since they are the same files, and you are changing them in local and remote, you will probably change the same lines. Or my present Git understanding is missing something, like a force option?

If you don’t care about branching and all that, just use file synchronization tools.

Here is an example session where the USB stick contains a Git repo:

git fetch  path-to-repo-on-stick
git merge -s recursive -Xtheirs --no-commit FETCH_HEAD

... now inspect, test, etc. ...

git commit

Explanation
1. Get fetch will get the new stuff from the “remote” repo that is on the USB stick.

2. Now we want to merge this into the local repo.
-s recursive is the strategy.
-X gives the option to the strategy
–no-commit we want to inspect and test before we actually merge.
FETCH_HEAD is where the new stuff gets placed (the commit object?)

4. We visually inspect and run tests here. Nothing broke?

6. Commit the changes in the working set, that was created at 2.

Warning
Posted by a Git newbie, so “be careful out there”.

Updates
The git-bundle command is the standard way of doing this. Find out how to use it: Git’s Little Bundle of Joy.

Does using bundling remove the excess conflicts?

Further Reading

Remove CVS folders from Git version control

Open a bash shell and just do:
$ find . -type d -name CVS -exec git rm -r –cached ‘{}’ ;
Is there an easier or more direct method using just the Git commands?

Open a bash shell and just do:
$ find . -type d -name CVS -exec git rm -r –cached ‘{}’ ;

Is there an easier or more direct method just using Git commands?

Note the –cached option (there are two dashes there). This will ensure we don’t remove the CVS folders in the work area, only those in the index.

Note:
If your on Windows, the msysGit or Cygwin installs include a Bash shell. Git, though usable on Windoze, still has a lot of *nixisms.

But
before doing the above, make a backup or a backup clone of the project. Maybe even do a: git fsck

Check
if everything was done correctly. Do a git status and it should print a bunch of:
# deleted: some/path/to/CVS/cvs track file

and, a reminder to add the .gitignore and .cvsignore files that you finally created, right?

Then to make sure the folders and their content are in the work area do:
$ find -type d -name CVS -print

In a Windows shell this would be: dir /S CVS
but, I didn’t test that.

Update ignores
; the .gitignore file should contain a line: CVS/
Make sure there is no trailing space at the end of the line.

I did a: git add -n . | grep CVS
to make sure nothing was going to be added.

Or instead, add this line to the .git/info/exclude file.

Don’t forget to modify the .cvsignore file to ignore the .git folder.

Now do
git commit

But wait …

WARNING:
CVS keeps file tracking version information in CVS subfolders (Subversion uses SVN). These are in each subdirectory of your working files.

It is very important that only CVS manage these. Thus, using Git (or Mercurial, btw) should have valid ignore files set up.

Just the existence of these subfolders is a good reason to abandon these systems. They make the use of external tools, even search, even more complicated. You have to use filters for everything. To be fair, good tools already have these filters in place.

Why
would someone want to do all this? Well, if you forget to create a .gitignore file before you put all your CVS officially managed project files into Git. Of course, no one forgets to create the ignore file first.

Why
would you ever use Git parallel with CVS? Well, a centralized repository discourages checking in of code. You have to have perfect code before you commit stuff. Don’t even think about branching off a branch (twigs).

An aspect of a centralized VCS is:

When you check new code in, everybody else gets it.

Since all new code that you write has bugs, you have a choice.

You can check in buggy code and drive everyone else crazy, or
You can avoid checking it in until it’s fully debugged.

Subversion always gives you this horrible dilemma. Either the repository is full of bugs because it includes new code that was just written, or new code that was just written is not in the repository.

As Subversion users, we are so used to this dilemma that it’s hard to imagine it not existing.

— from HgInit

So, if you can’t use the VCS to check in your work what do you do? Make manual backups? If you say use the IDE to manage it, you can’t. For example, an IDE (in my experience) cannot do “undo” and “redo” across separate source files, thus the ‘undo’ can only get you so far. For small changes not really an issue. But, if your working on a subsystem or module, or a small team, this can be an issue.

An alternative is
to just add a DVCS to the Integrated Development Environment (IDE) to do true local versioning (maybe embedded Git?). That way I can just indicate to the IDE that I want to mark my current work as a milestone, for example, and it will know what to do. Now I can proceed and do all the fancy little versioning stuff, like branch, log, etc. It is just local.

I don’t even need to know about the arcana of the embedded VCS, it is just the IDE doing its thing. Further, higher-order functions like tasks, bug tracking, could also relate to the IDE’s repo, not only the centralized repository.

When it is finally debugged brilliant code, as usual, I just push to the ‘real’ local version control system, which could just be the same DVCS or a centralized one like SVN or CVS, and then commit as usual.

I bet some IDE already does this. Oh well.

Updates
5Sep11: Took a look at the IntelliJ site. Their IDE does allow directory history and ability to set a version labels. Nice.

System
Git: git version 1.7.3.1.msysgit.0
OS: Windows 7 64bit Professional
PC: AMD quad-core, 8GB ram.

Further Reading