Shell script to check git for changes and then loop through changed files?

Question

I am trying write a shell script that does the following:

1. Checks the remote git repository for any changes to pull.
2. If there are changes in the remote git repository pull those changes.
3. Loops through the files that are new or have been modified.

Through my research I have found some of the necessary commands to do these things but I haven't been able to get them to work together in a shell script.

Here is a script with some of the commands that I have:

#!/bin/sh

#Check if there are any changed files
git --git-dir="/dir/.git" fetch origin

if git --git-dir="/dir/.git" log HEAD..origin/master --oneline
then

#Output the modified files from the last pull
        git --git-dir="/dir/.git" diff --name-status ORIG_HEAD..

fi

The things that I have not been able to get working with the commands in this script are:

1. The if statement to check if there are changes or not always is true. I have tried if statements with other git commands and they also are always true. It seems that git does not work like normal shell commands where you get a 0 or 1 response. How can I get a git command like this or other git commands to return the right responses in an if statement?
2. How can I assign the variables output from the command to see the changed files to an array so that I can loop through them using a for?

If the commands in my script won't really work in this case what is a better way to do this?

Edit: Sorry should have been more clear when I loop through the changed files I need to pull the changes from the remote repository before I loop through the files so that when I work with these files I have the latest changes.

Solution 1

For your first question, you can use git diff --quiet (or git diff --exit-code, but generally when you're using it for its exit code you want it not to print output anyhow, and git diff --quiet implies --exit-code) to determine if there have been any changes. That will give you a 1 value if there are changes, and a 0 if there are not. So if you want to have code that will run only if there are changes:

if ! git --git-dir="/dir/.git" diff --quiet
then
    # do stuff...
fi

For your second question, I'd recommend a while read ... loop to read lines from git diff-tree:

git --git-dir="/dir/.git" diff-tree ORIG_HEAD.. | \
    while read srcmode dstmode srcsha dstsha status srcfile dstfile
    do
        # do something with $srcfile and $dstfile
    done

Note that $srcmode will have an extra : at the beginning, and $dstfile will only have a value if the file was renamed. If you don't want to worry about renames, pass in --no-renames, and instead of seeing renames you'll see just the adds and deletes.

Solution 2

You can simply use git diff --name-only to list all filenames of files changed.

For example, here a simple script to list all PHP files edited and test the syntax.

#!/bin/bash
files=`git diff --name-only | grep -E '.php$' `
for file in $files; do
  php -l $file
done

Solution 3

You're right, git does not normally exit with nonzero status unless there's an error. The easiest way I've found to detect uncommitted changes is something like this:

 let changes=0
 while read status filename; do
    let changes=1
    # do something with this filename/status
 done < <(git status --porcelain)
 if (( ! changes )); then
    echo "No changes."
 fi

In general, if you're going to try to drive git from code, you should use --porcelain on those subcommands which support it, to make it act in a more automation-friendly fashion. You may also want to investigate libraries for other languages to interact with git without constructing your own shell commands; for instance, in Ruby there's grit.

Now, in your case, you want to detect what changed upstream. For that, you probably want to use git diff-tree. Similar logic to the above:

 git fetch   # not pull
 while read filename; do
   # do something with filename
 done < <(git diff-tree --name-only origin/master master)  # or whatever branch you're using

Solution 4

If you're scripting and want to make sure the repo is in a committed state (e.g. no new, modified, or staged files), try this:

# Get to code.  Exit if unsaved changes in repo
if `git status | grep -q "nothing to commit"`; then
  git checkout --quiet $BRANCH || exit 1
else
  echo "ERROR: repo has unsaved changes"
  exit 1
fi

This is the simplest thing I could find that checked for new, changed, and staged files.

View more solutions

Comments

[startupsmith]

Hi Mark. Thanks for your reply. I tried running your script but I am getting the following error: syntax error near unexpected token <' done < <(git status --porcelain)'. Any ideas what the correct syntax should be?

[Mark Reed]

You sure you're using bash?

[startupsmith]

Hi Mark. You are right I wasn't using bash... it works now. I need to look into this further but from what I can see this script will loop through the changed files before they have been pulled from the remote repository?

[Mark Reed]

sorry, that does the wrong thing - it loops through the files that have changes in the working copy that have not yet been committed. You're looking for changes from upstream, so git diff-tree is probably what you want. See my edited answer and @BrianCampbell's.

[startupsmith]

Hi Mark. iirc the problem with git fetch is that it does not merge the changes from the remote to update the local files. When I loop through the files I need the files to be up to date... I have tried using git pull however it does not seem to want to merge when I execute it from within the script. Should git pull be possible in the script for what I am trying to do or will I need to do something else?

[Mark Reed]

You should be able to git pull from the script, sure. Once you have merged the changes in, there won't be any differences to report between the two master branches, so you'd have to diff HEAD^ and HEAD instead.

[Tal Jacob - Sir Jacques]

For me worked: git diff HEAD --name-only