Archive

Archive for the ‘Expect/TCL’ Category

self defined timeout for telnet on Linux

December 26th, 2013 Comments off

telnet's default timeout value is relative high, so you may want to change timeout value to lower value such as 5 seconds. Here's the way that we can fulfill this:

#!/bin/bash

timeout()
{
waitfor=5
command=$*
$command &
commandpid=$!
( sleep $waitfor ; kill -9 $commandpid > /dev/null 2>&1 ) &
watchdog=$!
sleeppid=$PPID
wait $commandpid > /dev/null 2>&1
kill $sleeppid > /dev/null 2>&1
}

timeout telnet slcc29-scan1.us.oracle.com 1521 >> $output

Also, we can use expect and set timeout for expect. When telnet is integrated with expect, we can fulfill timeout for telnet through using expect's timeout value:

#!/usr/bin/expect

set timeout 30

send "<put telnet command here>\r"

remove duplicate images using fdupes and expect in linux

December 13th, 2013 Comments off

I've got several thousands of pictures, but most of them had several exact copies of themselves. So I had to remove duplicate ones by hand firstly.

Later, I thought of that in linux we had md5sum which will give the same string for files with exact same contents. Then I tried to write some program, and that toke me some while.

I searched google and found that in linux, we had fdupes which can do the job very well. fdupes will calculate duplicate files based on file size/md5 value, and will prompt you to reserve one copy or all copies of the duplicates and remove others if you gave -d parameter to it. You can read more about fdupes here http://linux.die.net/man/1/fdupes

As all the pictures were on a windows machine, so I installed cygwin and installed fdupes and expect. Later I wrote a small script to reserve only one copy of the duplicate pictures for me(you will have to enter your option either reserving one copy or all copies by hand if you do not use expect, as there's no option for reserve one copy by the author of fdupes). Here's my program:

$ cat fdupes.expect
#!/usr/bin/expect
set timeout 1000000
spawn /home/andy/fdupes.sh
expect "preserve files" {
send "1\r";exp_continue
}

$ cat /home/andy/fdupes.sh
fdupes.exe -d /cygdrive/d/pictures #yup, my pictures are all on this directory on windows, i.e. d:\pictures

After this, you can just run fdupes.expect, and it will reserve only one copy and remove other duplicates for you.

PS: Here's man page of fdupes https://github.com/adrianlopezroche/fdupes

bash & expect tips

August 19th, 2013 Comments off


/u02/0-sync.sh

#!/usr/bin/expect
set timeout 10800
spawn /u02/sync.sh
expect "*assword*" {send "test\r";exp_continue
expect "*#*" {close}
}

/u02/sync.sh

#! /bin/sh
logpathroot=/u02/logs
synclist=/u02/sync.list

num=1
allnum=`cat $synclist | wc -l`
while [ $num -le $allnum ];
do
line=`awk '{if(NR=='$num') print}' $synclist`
component=`echo $line | awk '{print $2}'`
logtype=`echo $line | awk '{print $3}'`
host=`echo $line | awk '{print $4}'`
source=`echo $line | awk '{print $5}'`
env=`echo $line | awk '{print $1}'`
if [[ "$env" =~ "#" ]]; then
env=`echo $env | sed "s/#//"`
lnenv=`ls /u01/logs | grep -i "$env"`
exist=`ls "/u01/logs/${lnenv}/${component}_${logtype}-$host"`
if [ "$exist" != "" ]; then
rm -f /u01/logs/${lnenv}/${component}_${logtype}-$host
fi
else
lnenv=`ls /u01/logs | grep -i "$env"`
/bin/mkdir -p "${logpathroot}/${env}/${component}/${logtype}-$host"

#rsync -avz --exclude "aime/work/CLOUDTOP/*" /scratch/ /scratch_new/
/usr/bin/rsync -ave ssh --exclude '.zfs' --delete-excluded logs@${host}:${source}/ ${logpathroot}/${env}/${component}/${logtype}-$host
#chmod 755 -R "${logpathroot}/${env}/${component}/"
exist=`ls "/u01/logs/${lnenv}/${component}_${logtype}-$host"`
if [ "$exist" = "" ]; then
# rm -f /u01/logs/${lnenv}/${component}_${logtype}-$host
component1=`echo $component | tr "[:upper:]" "[:lower:]"`
logtype1=`echo $logtype | sed "s/-/_/"`
ln -sf ${logpathroot}/${env}/${component}/${logtype}-$host/ /u01/logs/${lnenv}/${component1}_${host}_${logtype1}_log
fi
fi
num=`expr $num + 1`
done

rm -f /u02/sync_at_*
echo > /u02/sync_at_"`date +%Y-%m-%d` `date +%H:%M` (UTC)"

/u02/sync.list

DC1 apex AdminServer_adr test27vmf4053 /u01/local/config/m_domains/base_domain/servers/AdminServer/adr
DC1 emgc EMGC_ADMINSERVER_adr test27cn05 /u01/local/exalogic/Middleware/gc_inst/user_projects/domains/GCDomain/servers/EMGC_ADMINSERVER/adr
DC1 emgc EMGC_OMS1_adr test27cn05 /u01/local/exalogic/Middleware/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/adr
DC1 emgc EMGC_OMS2_adr test27cn06 /u01/local/exalogic/Middleware/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS2/adr
#DC1 emgc EMGC_OMS3_adr test27cn07 /u01/local/exalogic/Middleware/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS3/adr

PS:

  • If you want to match empty space in expect, you can use the ascii code of space which is 040, and expect -re "\040" would work.
  • About special characters in tcl/expect/shell:

Some characters have special meanings in Tcl or shells (e.g., sh, tcsh, etc.). In particular, these characters may provide security holes for shells.
The characters listed here are escaped by preceeding each one with a backslash (\).
The following characters are dissallowed or have special meanings in Tcl and so are escaped: &;`'"|*?~<>^()[]{}$\-
The following characters are dissallowed or have special meanings in command shells and so are escaped: &;`'"|*?~<>^()[]{}$\\n
The only difference between the two tables is the addition of \n to the shell escape table.

  • To execute multiple commands in one run, you can do below:
[root@vm1 ~]# cat conf.andy
sudo su -
echo $UID >/tmp/e
for i in {1..10}
do
touch /tmp/$i.txt
done
ls /tmp/

[root@vm1 ~]# cat /bin/autoexcute.andy
#!/usr/bin/expect --
set timeout 60
set user [lindex $argv 0]
set ip [lindex $argv 1]
set password [lindex $argv 2]
set com [lindex $argv 3]

spawn ssh -t -o "StrictHostKeyChecking no" $user@$ip
expect {
  "assword:" {
    send "$password\r"
    expect -re "(\\\$|\\\$\040)$" {
        send "/usr/bin/sudo su -\r"
        sleep 2
        expect -re "(\->\040|#|#\040)$" {
            send "$com\r"
            sleep 1
            expect -re "(\->\040|#|#\040)$" {close}
        }
    }
  }
}

[root@vm1 ~]# /bin/autoexcute.andy root testvm password1 "`cat conf.andy`"
spawn ssh -t -o StrictHostKeyChecking no root@testvm
root@testvm's password:
Last login: Tue Sep 15 19:06:49 2015 from vm1.us.oracle.com
-bash-4.1$ /usr/bin/sudo su -
[root@testvm-luis ~]# sudo su -
[root@testvm-luis ~]# echo $UID >/tmp/e
[root@testvm-luis ~]# for i in {1..10}
> do
> touch /tmp/$i.txt
> done
[root@testvm-luis ~]# ls /tmp/
1.txt  
10.txt 
2.txt  
3.txt  
4.txt  
5.txt  
6.txt  
7.txt  
8.txt  
9.txt  
  • Here's something about expect_out in tcp/expect:

expect_feedback_01

expect_feedback_02

resolved – how to remove whitespaces in expect – tcl language

November 15th, 2012 Comments off

Here's the way to remove whitespaces automatically when using expect/tcl language:

If $line is the string variable that you wanna remove whitespaces, just replace $line with the following in your programs context:

[string trim $line]

That's it, simple, Ehh..

autossh sshldap

March 25th, 2012 Comments off

1. create /root/autossh.sh

#!/usr/bin/expect --
#!/bin/sh
#
# autossh.sh
# SimpleSSHProxy
#
# Created by ivan on 10-9-10.
# http://twitter.com/ivan_wl
#
# This is another wheel of autossh built using expect.
# If you have to use ssh password authorizing,
# use this instead of autossh.
#
# WARNING: This script is NOT SAFE for your password,
# DON'T USE THIS IF YOU HAVE SECURITY CONCERNS!
#
# Usage: autossh.sh [foo] [foo] [bindingIP:port] [foo] [username] [remoteHost] [password]
# get arguments
set username [lrange $argv 0 0]
set remoteHost [lrange $argv 1 1]
set password [lrange $argv 2 2]

while (1) {
set connectedFlag 0;
spawn /usr/bin/ssh $username@$remoteHost;
match_max 100000;
set timeout 60;
expect {
"?sh: Error*"
{ puts "CONNECTION_ERROR"; exit; }
"*yes/no*"
{ send "yes\r"; exp_continue; }
"*?assword:*" {
send "$password\r"; set timeout 4;
expect "*?assword:*" { send "$password\r"; set timeout 4; }
set connectedFlag 1;
}
# if no password
"*~*"
{ send "echo hello\r"; set connectedFlag 1; }
}
if { $connectedFlag == 0 } {
close;
puts "SSH server unavailable, retrying...";
continue;
}

while (1) {
set conAliveFlag 0;
interact {
# time interval for checking connection
timeout 600 {
set timeout 30;
send "echo hello\r";
expect "*hello*" { set conAliveFlag 1; }
if { $conAliveFlag == 1 } {
# connection is alive
continue;
} else { break; }
}
}
}

close;
puts "SSH connection failed, restarting...";
}

2. create autoldap.sh:

#!/bin/bash
/root/autossh.sh <replace with your SSH username>$1 <replace with your SSH password> 2>/dev/null

3. run autoldap.sh <SSH hostname>

general expect tips

March 23rd, 2012 Comments off

#!/bin/bash
PATH=$PATH:/opt/csw/bin/
export PATH
CMD="uname -a && finishthecoreadm"
#....ksh: finishthecoreadm: not found,......,......
USERNAME="root"
PASSWORD="test"
for i in `cat /home/liandy/servers_list`
do
echo ''>/root/.ssh/known_hosts
expect <<EOF
spawn ssh -l${USERNAME} -p22 $i $CMD
set timeout 20
expect "*yes*"
send "yes\r"
expect "assword:"
send "${PASSWORD}\r"
expect eof
EOF
done

================
spawn ssh 172.2.0.79 ifconfig
expect {
“*yes*” { send “yes\r” }
“password:” { send “yytt22\r” }
}
expect eof

#interact
spawn ssh 172.2.0.79
expect {
“*yes*” { send “yes\r” }
“password:” { send “yytt22\r” }
}
interact
expect “hi\n”
send “youyouyou”

expect “hi”
send “you typed $expect_out(buffer)”
send “but I only expected $expect_out(0,string)”

#spawn_id
spawn bc
set bc_id $spawn_id

spawn /bin/sh
set sh_id $spawn_id

set spawn_id $bc_id
send “scale=50\r”

#send -i & expect -i
set spawn_id $ftp_id
send “get $file1\r”;expect “220*ftp> ”
send -i $write_id “success get file” #another spawn_id
send “get $file2\r”;expect “220*ftp>” #return to current spawn_id

expect {
-i $ftp_id
“220*ftp> ” action1
“550*ftp ” action2
}

same as:

expect {
-i $ftp “220*ftp> ” action
“550*ftp ” action2 #return current spawn_id
}

#proc
proc login(id) { #spawn as parameter
set spawn $id
expect “login:”
send “$name\r”

expect “password:”
send “$password\r”

expect “$prompt”
}

#!/bin/bash
expect -c ”
set send_human {.1 .3 1 .05 2}

send -h \”I’m hungry. Let’s do lunch.\”

Using expect to auto login and execute commands(relieve you from tedious login and typing)

November 19th, 2010 Comments off

Another imagination:

You have 100 servers.Now you've got a task.The boss needs to create a directory under /root and put some files under it.

What would you do next?Just be honestly and sincerely and start log in ssh then use mkdir to create directory,then use sftp or even ftp to put the files under the directory?

That would tire you so much,sour you hands and mind.Now let one line bash shell script to help you.

Here goes the general idea about the solution:

1.Write a bash script with multiple parameters.Each two of the parameters match the ip address and password to your ssh root account.

2.Now the script runs.It auto get the two parameters as a pair,and use expect to auto answer to interactive output of ssh command('yes' to 'Are you sure you want to continue connecting (yes/no)?',and your password parameter to 'root@xxx.xxx.xxx.xxx's password:').

3.After log in the remote server using ssh,the script download a script from url you designate.Like http://yourblog.com/mkdir_and_put_files_in_it.sh.After the download,use 'sh ./mkdir_and_put_files_in_it.sh' to run it which has command 'mkdir' and ftp get in it.

Above is the skeleton of the design.Now let's go to the code area:
#!/bin/bash
while [ $# -ne 0];
do
sshhost=$1
password=$2
shift 2
CMD='curl -o nginx_log.sh http://yourblog.com/mkdir_and_put_files_in_it.sh && sh ./mkdir_and_put_files_in_it.sh'
echo ''>/root/.ssh/known_hosts #first,delete known_hosts to let the expect "*yes*" work
/usr/bin/expect <<EOF
spawn ssh -lroot -p22 $sshhost $CMD
set timeout 60 #1 minute,command will be executed on the remote server even though 60s
expect "*yes*"
send "yes\r"
expect "*password*"
send "$password\r"
#interact
expect eof
EOF
done
Ok.Name the above shell script auto_sh.sh,grant it with execute privilege.

Then,create your mkdir and ftp get files(You can refer to here to auto-ftp get files).

Now use ./auto_sh.sh first_server_ip first_server_ip_password second_server_ip second_server_ip_password third_server_ip third_server_ip_password forth_server_ip forth_server_ip_password.You can assign as many pairs ip/password to the parameter position as you want.Just be relieved,your work will be perfectly done as you wish!