Archive

Archive for November, 2010

Using php pear_mail in kohana php framework

November 23rd, 2010 4 comments

You may have many sites that scatter over several servers.Each site need service mail account to receive and sent letters to your visitors.You may set up mail server on each server one by one,but that’s not very efficiency and have weakness on availability & manageability.For some day,you may imagine,one of your server crash and then all mail accounts on it are destroyed. As a substitution,using dedicated mail servers is a good idea.Even though your httpd servers crash,your mail accounts are still there,with nothing loss.(You may extend this idea to dedicated DB servers,dedicated cache servers,etc.)

But when you come to know that php mail() function do not support selecting a MTA.It just uses the localhost’s mail server.That may be a problem to you somehow,for after separating httpd & mail servers,you no longer have mail server on each single httpd server.Then,don’t worry about that! Pear has a package named ‘Mail’.It implements RFC822 and helps you to select one MTA from your dedicated mail servers.Here,I’m going to show the usage of it under kohana php framework. First,download pear Mail package on http://pear.php.net/package/Mail.Unpack it and then follow the steps below. Make directory follow the architecture:KOHANA_PATH/modules/PearMail/libraries/.Then put directories and files you got from the Mail package.Rename directories & filenames & class names in them(which I’ve done that for you.Check that in the following package I supplied). Then,you come to the most important step,which is easy too.Load PearMail module by modifying KOHANA_PATH/application/config.php:
$config['modules'] = array
(
MODPATH.’PearMail’,        //PearMail by doxer.org
);
And,create a controller named pearmail.php:
<?php defined(‘SYSPATH’) OR die(‘No direct access allowed.’);
class Pearmail_Controller extends Controller
{
public function index(){
$mail=new PearMail();
$smtp_arr=array(
array(‘xxx.xxx.xxx.xxx’,’[email protected]’,’PnC9GvfjJYSRk’),        //your first dedicated mail server
array(‘xxx.xxx.xxx.xxx’,’[email protected]’,’PnC9GvfjJYSRk’)   //your second dedicated mail server
);
$host_key=array_rand($smtp_arr);//select a mail server at random to send letter

/**—————————–Modify following area——————————————–**/

$site_name=’yoursite.tld’;
$headers['from']=’service@’.$site_name;//mail from
$to=’[email protected]’;
$headers['subject']=’This is the subject_lizhy’;//mail subject
$headers['Content-type']=’text/html; charset=utf-8′;//mail content encoding
$content=”<html><body>Just a test<br /><h1>This is H1</h1>”.$smtp_arr[$host_key][0].$smtp_arr[$host_key][1].$smtp_arr[$host_key][2].”</body></html>”;//mail content

/**—————————–Modify above area——————————————–**/

$params=array(
‘host’=>$smtp_arr[$host_key][0],
‘port’=>’25′,
‘auth’=>true,
‘username’=>$smtp_arr[$host_key][1],//your account on #1 dedicated servers
‘password’=>$smtp_arr[$host_key][2] //your account on #2 dedicated servers
);
$smtp=$mail->factory(‘smtp’,$params);
$result=$smtp->send($to,$headers,$content); //OK.Send the mail!
var_dump($result); //test
var_dump($smtp_arr[$host_key]); //test
}
}
Ok.Now it’s time to check if that works now!Type in your browser the url to the pearmail controller and wait for email-receiving alert from your MUA like Outlook etc! Download:PearMail_doxer.org.tar.gz

Auto-monitor your server status and restart or reboot server(bash shell script)

November 22nd, 2010 2 comments

I’m now in charge of about 70 vps.Every vps has www web-server & proftpd ftp server & mysql db server on it.Now I want to know every server’s status.There’s of course method that notify you waking up at mid-night and handling with the simple reboot of apache & mysql & proftpd,yes,that’s true,for your boss won’t be happy enough to see all sites offline on servers that are within your duty.This is called a ‘pull’ working style.Ok.Now why don’t we try the ‘push’ working style now?Just write the following script on your server,and add it to the cron schedule at a proper execution rate.

The bash script auto-detect and monitor status of your service on a server,and if it find that the software has stopped working,the script will actively start it.And,if the number of  httpd or mysqld exceeds the point you set in the script,the server will restart.All of this primarily should be your working scope,and now it’s all the bash script’s duty!

Here goes the script:
#!/bin/bash
#check apache,mysql thread and auto reboot system
#Powered by ipaddr(aspbiz)
#Modified by doxer.org
which os is the system,debian or centos;others exit 1
ip_address=’xxx.xxx.xxx.xxx’
mail_doxer()
{
mail -s “Abnormal Message from VPS $ip_address” [email protected]</tmp/result.txt
}
echo ”>/tmp/result.txt
os=””;
apachename=””;
grep “centos” /etc/issue -i -q
if [ $? = '0' ];then
os=’centos’
apachename=”httpd”
fi

grep “debian” /etc/issue -i -q
if [ $? = '0' ];then
os=’debian’
apachename=”apache”
fi

if [ $os = "" ];then
echo “not a valid system os”
exit 1
fi

ApacheThread=`ps -A|grep ${apachename}|wc -l`
MysqldThread=`ps -A|grep mysql|wc -l`
ProftpThread=`ps -A|grep proftpd|wc -l`

if [ $ApacheThread -eq 0 ]
then
echo “Apache Server has stoped, it will start at $(date +”%y-%m-%d %H:%M:%S”)”>>/tmp/result.txt
mail_doxer
/etc/init.d/apache2 start
fi

if [ $MysqldThread -eq 0 ]
then
echo “MySQL Server has stoped, it will start at $(date +”%y-%m-%d %H:%M:%S”)”>>/tmp/result.txt
mail_doxer
/etc/init.d/mysql start
fi

if [ $ProftpThread -eq 0 ]
then
echo “ProftpThread Server has stoped, it will start at $(date +”%y-%m-%d %H:%M:%S”)”>>/tmp/result.txt
mail_doxer
/etc/init.d/proftpd start
fi

#process forks too much,needs reboot the system
MaxApacheThread=30
MaxMysqlThread=250

NeedReboot=0

if [ $ApacheThread -gt $MaxApacheThread ]
then
NeedReboot=1
fi

if [ $MysqldThread -gt $MaxMysqlThread ]
then
NeedReboot=1
fi

if [ $NeedReboot -eq 1 ]
then
date_reboot=$(date +”%y-%m-%d %H:%M:%S”)

echo “System is busy,reboot\nApache:$ApacheThread;Mysql:$MysqldThread;Vsftp:$ProftpThread.”>>/tmp/result.txt
mail_doxer
reboot
fi

Ok.Now add the script to your cron schedule by crontab -e and type in the following(change it to meet your needs):
5/* * * * * /usr/local/apache_mysql_ftp_check_restart.sh #every 5 minutes

Categories: Linux Tags: ,

Using expect to auto login and execute commands(relieve you from tedious login and typing)

November 19th, 2010 No comments

Another imagination:

You have 100 servers.Now you’ve got a task.The boss needs to create a directory under /root and put some files under it.

What would you do next?Just be honestly and sincerely and start log in ssh then use mkdir to create directory,then use sftp or even ftp to put the files under the directory?

That would tire you so much,sour you hands and mind.Now let one line bash shell script to help you.

Here goes the general idea about the solution:

1.Write a bash script with multiple parameters.Each two of the parameters match the ip address and password to your ssh root account.

2.Now the script runs.It auto get the two parameters as a pair,and use expect to auto answer to interactive output of ssh command(‘yes’ to ‘Are you sure you want to continue connecting (yes/no)?’,and your password parameter to ‘[email protected]’s password:’).

3.After log in the remote server using ssh,the script download a script from url you designate.Like http://yourblog.com/mkdir_and_put_files_in_it.sh.After the download,use ‘sh ./mkdir_and_put_files_in_it.sh’ to run it which has command ‘mkdir’ and ftp get in it.

Above is the skeleton of the design.Now let’s go to the code area:
#!/bin/bash
while [ $# -ne 0];
do
sshhost=$1
password=$2
shift 2
CMD=’curl -o nginx_log.sh http://yourblog.com/mkdir_and_put_files_in_it.sh && sh ./mkdir_and_put_files_in_it.sh’
echo ”>/root/.ssh/known_hosts #first,delete known_hosts to let the expect “*yes*” work
/usr/bin/expect <<EOF
spawn ssh -lroot -p22 $sshhost $CMD
set timeout 60 #1 minute,command will be executed on the remote server even though 60s
expect “*yes*”
send “yes\r”
expect “*password*”
send “$password\r”
#interact
expect eof
EOF
done
Ok.Name the above shell script auto_sh.sh,grant it with execute privilege.

Then,create your mkdir and ftp get files(You can refer to here to auto-ftp get files).

Now use ./auto_sh.sh first_server_ip first_server_ip_password second_server_ip second_server_ip_password third_server_ip third_server_ip_password forth_server_ip forth_server_ip_password.You can assign as many pairs ip/password to the parameter position as you want.Just be relieved,your work will be perfectly done as you wish!

Categories: Linux Tags: ,

bash shell mark & simulating multi-processing feature

November 13th, 2010 2 comments

The so called multi-processing,means at the very same moment,multi processes are handled by the cpu.In bash shell and other shells(ksh,csh,etc.),mark & has a special simulation of multi-processing except for the widely known running background feature.The below shell snippets is an example for a explanation purpose.It’s very funny,try it!(file name as if.sh and use sh if.sh to run it)
#!/bin/bash

for ((i=0;i<5;i++));do
{
sleep 3;echo 1>>aa && echo “done!”
#you may of course use other more complicated operations here
} &
done
wait
cat aa|wc -l

Ok.Now let’s use time command to test the execution time of the script:
time ./if.sh
done!
done!
done!
done!
done!
5

real    0m3.108s #here,5 loop of `sleep 3` costs 3 and some little more time
user    0m0.056s
sys    0m0.092s

What about we ignore the & ?Let’s just try it!
#!/bin/bash #single processing

for ((i=0;i<5;i++));do
{
sleep 3;echo 1>>aa && echo “done!”
}
done
wait
cat aa|wc -l

now ,use time bash if.sh to see the result:
done!
done!
done!
done!
done!
5

real    0m15.132s
user    0m0.028s
sys    0m0.096s

Now,you can see,0m15.132s shows in the real column of the output,which we can conclude that 5 loop of `sleep 3` costs more than 15 seconds.It’s then called the single processing.

using logrotate to rotates log files(per site log,by day,reserve last 5 days)

November 8th, 2010 1 comment

The needs as follows:
1.rotate the apache and nginx log files(access log,error log),by day and compress the log file;
2.keep last 5 days’ log files,the others throw away automatically;
3.using the date as the suffix of the log file.
The implementation as follows:
In apache:
#/bin/bash
#by doxer.org
echo -E ‘/usr/local/httpd/logs/*.log {
daily
rotate 5
compress
notifempty
dateext
lastaction
/usr/local/httpd/bin/apachectl -k graceful
endscript
}’ > /etc/logrotate.d/httpd_log
/etc/init.d/crond restart

In nginx:

#/bin/bash
echo -E ‘/usr/local/nginx/logs/*.log {
daily
rotate 5
compress
notifempty
dateext
lastaction
/usr/local/nginx/sbin/nginx -s reload
endscript
}’ > /etc/logrotate.d/nginx_log
/etc/init.d/crond restart

PS:logrotate most commonly used options
logrotate -d /etc/logrotate.conf #to debug
logrotate -f /etc/logrotate.conf #to debug,force running logrotate by now.

The you may think what’s the mechanism of logrotate?Maybe it has something related to crond!We can see in /etc/cron.daily has the following line:(in file /etc/cron.daily/logrotate)
#!/bin/sh
/usr/sbin/logrotate /etc/logrotate.conf
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
/usr/bin/logger -t logrotate “ALERT exited abnormally with [$EXITVALUE]”
fi
exit 0

In debian:
#!/bin/sh
test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

This proves the assumption is right.logrotate does implement it’s functionality by crond.