Need help with SFTP in a bash script

Charles · 31st December 2007, 09:16 AM

I am trying to transfer files to a remote server using SFTP in a bash
script.
I have ran into a problem though.
Part of the script cd to different directories on the remote server
but if it cd to a
directory that doesn't exist then the entire script gets terminated.
I know that I can
use scp which I am already doing but I also want to be able to use SFTP.

Here is my script:

Code:

#################### Begin Local Settings ####################
##### Backup Script Settings Begin #####
homeDIR="/home/xxxxx"
BUScriptLocation="/backupScript"
BULogName="backupLog"
##### Backup Script Settings End #####

##### Logs Backup Settings Begin #####
BUFolder3="/var/log/httpd" # Leave blank to not backup
logsBackupLocation="backupFolder"
encryptLogs="no" # yes or no
##### Logs Backup Settings End #####

#################### End Local Settings ####################

#################### Begin Remote Settings ####################
transferFiles="yes" # yes or no
transferMethod="sftp" # scp, sftp, or ftp
remoteUsername="xxxxxxxxxxxx" # UserName for remote Server
remoteServer="xxxxxxxx.xxxxxx" # Remote Server
remoteFolder="backups" # Remote Folder to save backups to
miscSetting1="/remote/home" # Misc Settings

tempcmds="tempcmdsFile" # File for temp. cmds when using SFTP or FTP
#################### End Remote Settings ####################

myDate=`date +%Y_%a`
dayOfWeek=`date +%w` # 0-6 0 is Sunday
weekDay=`date +%a` # Mon, Tues, Wed, etc...
month=`date +%b` # Jan, Feb, Mar, etc...
day=`date +%d` # 01-31
year=`date +%Y` # 2007
time=`date +%H:%M:%S` # Hour:Minute:Second
time2=`date +%T` # Hour:Minute:Second
time3=`date +%X` # Hour:Minute:Second
timeZone=`date +%Z` # UTC, CST, CDT, etc..
backupDate=`date +%Y-%b-%d-%X-%Z`

#perl $homeDIR/$BUScriptLocation/backup.script $homeDIR/ $logsBackupLocation/ $BUFolder3

if [ "$transferMethod" = "sftp" ];
then
echo "cd $remoteFolder" >> $homeDIR/$tempcmds
echo "mkdir $month" >> $homeDIR/$tempcmds
echo "cd $month" >> $homeDIR/$tempcmds
echo "mkdir $day" >> $homeDIR/$tempcmds
echo "cd $day" >> $homeDIR/$tempcmds
echo "mkdir Logs" >> $homeDIR/$tempcmds
#	 echo "put $homeDIR/$logsBackupLocation/* $remoteUsername@ 
$remoteServer:$remoteFolder/$month/$day/Logs" >> $homeDIR/$tempcmds

echo "quit" >> $homeDIR/$tempcmds
sftp -b $homeDIR/$tempcmds $remoteUsername@$remoteServer
#	 rm -f $homeDIR/$tempcmds
fi

Well I figured out how to keep the bash script from quiting when errors come from sftp using this:

Code:

sftp $remoteUsername@$remoteServer <<**
if [ -d /usr/home3/6134248/backups_business/script/Dec ]; 
then
cd /usr/home3/6134248/backups_business/script/Dec
else
mkdir /usr/home3/6134248/backups_business/script/Dec
cd /usr/home3/6134248/backups_business/script/Dec
fi
if [ -d /usr/home3/6134248/backups_business/script/Dec/testing ];
then
cd /usr/home3/6134248/backups_business/script/Dec/testing
else
mkdir /usr/home3/6134248/backups_business/script/Dec/testing
cd /usr/home3/6134248/backups_business/script/Dec/testing
fi
put $homeDIR/$logsBackupLocation/folder_files/*.*
exit
**

But now when I run the script I run into the problem where the if, then, else, fi are not
being honored and it is saying Invalid Command.
I am going off of the an example I found here:
CrunchBang ~ Bash Script: MySQL Backup

Any suggestions?

a-j · 2nd January 2008, 12:57 PM

You want to just create directories if they don't exist.

Unfortunately sftp doesn't have any flow control, and you are trying to use bash 'if...then' syntax in sftp, which won't work.

The simplest fix would be to invoke sftp repeatedly with just one 'mkdir' command to try to create each directory you need. If the directory already exists sftp will just terminate. Something like this should work (at least it does with my sftp client/server):

#!/bin/bash

# be sure not to include another sftp command after 'mkdir $1' or
# else the session will hang if the directory already exists!
remote_mkdir() {
sftp $HOST<<END
<put your authentication dialogue here>
mkdir $1
END
}

DIRS="dir1 dir2 dir2/subdir"

for d in $DIRS ; do
remote_mkdir $d
done

... then continue with the script you had before.

If you want a fancier approach you could investigate 'expect' but jut for this it's not worth the effort.

Happy New Year,
Tony