Blog.Avijit.Info

Do:

iptables -I INPUT ! -i lo; iptables -I OUTPUT ! -o lo

Then setup a cronjob that runs once a month when your host checks your bandwidth that does:

iptables -R INPUT 1 ! -i lo; iptables -R OUTPUT 1 ! -o lo

to reset it. You could do iptables -Z INPUT instead, but if you ever add any more rules to INPUT it would reset their counters too.

Next setup a cron job that runs regularly (I have it run every hour) that does:

iptables-save -c > /etc/iptables

That saves every table, all the chains in those tables, and all of the rules in those chains with their counters to the file /etc/iptables (feel free to pick a different name, that’s just what I use).

After that put:

iptables-restore -c < /etc/iptables

in your rc.local. This restores those rules and their counters on boot.

Now just do iptables -L -v to see your current bandwidth usage. Look at those rules’ counters, not the chains’ counters.

Lets say we want to get the files out of a .iso image from the most recent DC distro.

Download the iso to the system and mount as below to copy the data from /mnt/tmp

# mkdir /mnt/tmp
# mount -t iso9660 -o loop=/dev/loop0 shrike-i386-disc1.iso /mnt/tmp
# cp -af /mnt/tmp/* /var/ftp/pub/redhat/fc6/
# umount /mnt/tmp

You are done copying the files for a ftp installation server. Inform your mates that they can use your box to install the latest FC6. Just burn the first bootable disk for them so that they can start the network ftp installation.

Note : This document was originally written by Troy Johnson in the  http://www.jdmz.net/ssh/ and this has been very very helpfull to me for my regular  backups. I  appologize for my failure earlier to acknowledge his original work.

I had to backup some of servers impotant data,mails and users data accros the internet between the hosts. I used the power of rsync and sshd to do that.

This document covers using cron, ssh, and rsync to backup files over a local network or the Internet. Part of my goal is to ensure no user intervention is required when the computer is restarted (for passwords, keys, or key managers).

You’ll need these packages installed:

* rsync
* openssh
* cron (or vixie-cron)

First, I’ll define some variables. In my explanation, I will be synchronizing files (copying only new or changed files) one way, and I will be starting this process from the host I want to copy things to. In other words, I will be syncing files from /remote/dir/ on remotehost, as remoteuser, to /this/dir/ on thishost, as thisuser.
I want to make sure that ‘rsync’ over ’ssh’ works at all before I begin to automate the process, so I test it first as thisuser:

$ rsync -avz -e ssh remoteuser@remotehost:/remote/dir /this/dir/

and type in remoteuser@remotehost’s password when prompted. I do need to make sure that remoteuser has read permissions to /remote/dir/ on remotehost, and that thisuser has write permissions to /this/dir/ on thishost. Also, ‘rsync’ and ’ssh’ should be in thisuser’s path (use “which ssh” and “which rsync”), ‘rsync’ should be in remoteuser’s path, and ’sshd’ should be running on remotehost.

Configuring thishost

If that all worked out, or I eventually made it work, I am ready for the next step. I need to generate a private/public pair of keys to allow a ’ssh’ connection without asking for a password. This may sound dangerous, and it is, but it is better than storing a user password (or key password) as clear text in the script. And I can put some limitations on what connections made with this key can do. Anyway, I generate the key I will use on thishost (as thisuser):

$ ssh-keygen -t dsa -b 2048 -f /home/thisuser/cron/thishost-rsync-key
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase): [press enter here]
Enter same passphrase again: [press enter here]
Your identification has been saved in /home/thisuser/cron/thishost-rsync-key.
Your public key has been saved in /home/thisuser/cron/thishost-rsync-key.pub.
The key fingerprint is:
2e:28:d9:ec:85:21:e7:ff:73:df:2e:07:78:f0:d0:a0 thisuser@thishost

and now we have a key with no password in the two files mentioned above. Make sure that no other unauthorized user can read the private key file (the one without the ‘.pub’ extension).
This key serves no purpose until we put the public portion into the ‘authorized_keys’ file on remotehost, specifically the one for remoteuser:

/home/remoteuser/.ssh/authorized_keys

I use scp to get the file over to remotehost:

$ scp /home/thisuser/cron/thishost-rsync-key.pub remoteuser@remotehost:/home/remoteuser/

and then I can prepare things on remotehost.
Configuring remotehost

I ’ssh’ over to remotehost:

$ ssh remoteuser@remotehost remoteuser@remotehost’s password: [type correct password here] $ echo I am now $USER at $HOSTNAME I am now remoteuser at remotehost

to do some work.
I need to make sure I have the directory and files I need to authorize connections with this key :

$ if [ ! -d .ssh ]; then mkdir .ssh ; chmod 700 .ssh ; fi
$ mv thishost-rsync-key.pub .ssh/
$ cd .ssh/
$ if [ ! -f authorized_keys ]; then touch authorized_keys ; chmod 600 authorized_keys ; fi
$ cat thishost-rsync-key.pub >> authorized_keys

Now the key can be used to make connections to this host, but these connections can be from anywhere (that the ssh daemon on remotehost allows connections from) and they can do anything (that remoteuser can do), and I don’t want that. I edit the ‘authorized_keys’ file (with vi) and modify the line with ‘thishost-rsync-key.pub’ information on it. I will only be adding a few things in front of what is already there, changing the line from this:

ssh-dss AAAAB3NzaC1kc3MAAAEBAKYJenaYvMG3nHwWxKwlWLjHb77CT2hXwmC8Ap+fG8wjlaY/9t4u
A+2qx9JNorgdrWKhHSKHokFFlWRj+qk3q+lGHS+hsXuvta44W0yD0y0sW62wrEVegz+JVmntxeYc0nDz
5tVGfZe6ydlgomzj1bhfdpYe+BAwop8L+EMqKLS4iSacNjoPlHsmqHMnbibn3tBqJEq2QJjEPaiYj1iP
5IaCuYBhuTKQGa+oyH3mXEif5CKdsIKBj46B0tCy0/GC7oWcUN92QdLrUyTeRJZsTWsxKpRbMliD2pBh
4oyX/aXEf8+HZBrO5vQjDBCfTFQA+35Xrd3eTVEjkGkncI0SAeUAAAAVAMZSASmQ9Pi38mdm6oiVXD55
Kk2rAAABAE/bA402VuCsOLg9YS0NKxugT+o4UuIjyl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK
1/ZIvtl92DLlMhci5c4tBjCODey4yjLhApjWgvX9D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV
5KLUl7FTL2KZ583KrcWkUA0Id4ptUa9CAkcqn/gWkHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw
46+ucWxwTJttCHLzUmNN7w1cIv0w/OHh5IGh+wWjV9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKr
Y+aJz7myu4Unn9de4cYsuXoAB6FQ5I8AAAEBAJSmDndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5
GjfBCRvHNo2DF4YW9MKdUQiv+ILMY8OISduTeu32nyA7dwx7z5M8b+DtasRAa1U03EfpvRQps6ovu79m
bt1OE8LS9ql8trx8qyIpYmJxmzIdBQ+kzkY+9ZlaXsaU0Ssuda7xPrX4405CbnKcpvM6q6okMP86Ejjn
75Cfzhv65hJkCjbiF7FZxosCRIuYbhEEKu2Z9Dgh+ZbsZ+9FETZVzKBs4fySA6dIw6zmGINd+KY6umMW
yJNej2Sia70fu3XLHj2yBgN5cy8arlZ80q1Mcy763RjYGkR/FkLJ611HWIA= thisuser@thishost

to this :

from=”10.1.1.1″,command=”/home/remoteuser/cron/validate-rsync” ssh-dss AAAAB3Nza
C1kc3MAAAEBAKYJenaYvMG3nHwWxKwlWLjHb77CT2hXwmC8Ap+fG8wjlaY/9t4uA+2qx9JNorgdrWKhH
SKHokFFlWRj+qk3q+lGHS+hsXuvta44W0yD0y0sW62wrEVegz+JVmntxeYc0nDz5tVGfZe6ydlgomzj1
bhfdpYe+BAwop8L+EMqKLS4iSacNjoPlHsmqHMnbibn3tBqJEq2QJjEPaiYj1iP5IaCuYBhuTKQGa+oy
H3mXEif5CKdsIKBj46B0tCy0/GC7oWcUN92QdLrUyTeRJZsTWsxKpRbMliD2pBh4oyX/aXEf8+HZBrO5
vQjDBCfTFQA+35Xrd3eTVEjkGkncI0SAeUAAAAVAMZSASmQ9Pi38mdm6oiVXD55Kk2rAAABAE/bA402V
uCsOLg9YS0NKxugT+o4UuIjyl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK1/ZIvtl92DLlMhci5
c4tBjCODey4yjLhApjWgvX9D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV5KLUl7FTL2KZ583Kr
cWkUA0Id4ptUa9CAkcqn/gWkHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw46+ucWxwTJttCHLzU
mNN7w1cIv0w/OHh5IGh+wWjV9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKrY+aJz7myu4Unn9de4
cYsuXoAB6FQ5I8AAAEBAJSmDndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5GjfBCRvHNo2DF4YW9
MKdUQiv+ILMY8OISduTeu32nyA7dwx7z5M8b+DtasRAa1U03EfpvRQps6ovu79mbt1OE8LS9ql8trx8q
yIpYmJxmzIdBQ+kzkY+9ZlaXsaU0Ssuda7xPrX4405CbnKcpvM6q6okMP86Ejjn75Cfzhv65hJkCjbiF
7FZxosCRIuYbhEEKu2Z9Dgh+ZbsZ+9FETZVzKBs4fySA6dIw6zmGINd+KY6umMWyJNej2Sia70fu3XLH
j2yBgN5cy8arlZ80q1Mcy763RjYGkR/FkLJ611HWIA= thisuser@thishost

where “10.1.1.1″ is the IP (version 4) address of thishost, and “/home/remoteuser/cron/validate-rsync” is a script that looks something like this:

#!/bin/sh

case “$SSH_ORIGINAL_COMMAND” in
*\&*)
echo “Rejected”
;;
*\(*)
echo “Rejected”
;;
*\{*)
echo “Rejected”
;;
*\;*)
echo “Rejected”
;;
*\<*)
echo “Rejected”
;;
*\`*)
echo “Rejected”
;;
rsync\ –server*)
$SSH_ORIGINAL_COMMAND
;;
*)
echo “Rejected”
;;
esac

If thishost has a variable address, or shares it’s address (via NAT or something similar) with hosts you do not trust, omit the ‘from=”10.1.1.1″,’ part of the line (including the comma), but leave the ‘command’ portion. This way, only the ‘rsync’ will be possible from connections using this key. Make certain that the ‘validate-rsync’ script is executable by remoteuser on remotehost and test it.
PLEASE NOTE: The private key, though now somewhat limited in what it can do (and hopefully where it can be done from), allows the possessor to copy any file from remotehost that remoteuser has access to. This is dangerous, and I should take whatever precautions I deem necessary to maintain the security and secrecy of this key. Some possibilities would be ensuring proper file permissions are assigned, consider using a key caching daemon, and consider if I really need this process automated verses the risk.
ALSO NOTE: Another security detail to consider is the SSH daemon configuration on remotehost. This example focuses on a user (remoteuser) who is not root. I recommend not using root as the remote user because root has access to every file on remotehost. That capability alone is very dangerous, and the penalties for a mistake or misconfiguration can be far steeper than those for a ‘normal’ user. If you do not use root as your remote user (ever), and you make security decisions for remotehost, I recommend either:

PermitRootLogin no

or

PermitRootLogin forced-commands-only

be included in the ‘/etc/ssh/sshd_config’ file on remotehost. These are global settings, not just related to this connection, so be sure you do not need the capability these configuration options prohibit.

Troubleshooting

Now that I have the key with no password in place and configured, I need to test it out before putting it in a cron job (which has it’s own small set of baggage). I exit from the ssh session to remotehost and try:

$ rsync -avz -e “ssh -i /home/thisuser/cron/thishost-rsync-key” remoteuser@remotehost:/remote/dir /this/dir/

f this doesn’t work, I will take off the “command” restriction on the key and try again. If it asks for a password, I will check permissions on the private key file (on thishost) and on ‘authorized_keys’ and (on remotehost). If some cryptic ‘rsync’ protocol error occurs mentioning the ‘validate-rsync’ script, I will make sure the permissions on ‘validate-rsync’ (on remotehost) allow remoteuser to read and execute it. Hopefully, it will always just work flawlessly so I never have to extend the troubleshooting information listed here.

Cron Job Setup

The last step is the cron script. I use something like this:

#!/bin/sh

RSYNC=/usr/bin/rsync
SSH=/usr/bin/ssh
KEY=/home/thisuser/cron/thishost-rsync-key
RUSER=remoteuser
RHOST=remotehost
RPATH=/remote/dir
LPATH=/this/dir/

$RSYNC -az -e “$SSH -i $KEY” $RUSER@$RHOST:$RPATH $LPATH

Because it is easy to modify the bits and pieces of the command line for different hosts and paths. I will usually call it something like ‘rsync-remotehost-backups’ if it contains backups. I test the script too, just in case I carefully inserted an error somewhere.
When I get the script running successfully, I use ‘crontab -e’ to insert a line for this new cron job:

0 5 * * * /home/thisuser/cron/rsync-remotehost-backups

for a daily 5 AM sync, or:

0 5 * * 5 /home/thisuser/cron/rsync-remotehost-backups

for a weekly (5 AM on Fridays). Monthly and yearly ones are rarer for me, so look at “man crontab”.

Alright! Except for the everyday “keeping up with patches” thing, the insidious “hidden configuration flaws” part, and the unforgettable “total failure of human logic” set of problems, my work here is done. Enjoy!

dd. Yes, dd. Take or generate a stream of bytes, and do something with it. With this nifty little utility, you can create disk images from scratch and format them. You can create disk images from existing devices (and then mount those images, mount -o loop filename.img!), write disk images to devices, send output from one device to another, or any number of interesting things. Say I want to create an image of a driver floppy for later, just in case I need it, and stick it in an archive somewhere:

dd if=/dev/fd0 of=driverfloppy.img

(if and of being “inputfile” and outputfile” respectively)

Or say I want to create a file exactly one gigabyte in size, composed of nothing but null bytes.

dd if=/dev/zero of=zerofile bs=1024 count=1000000

(bs being “block size” and count being the number of blocks to write)

Note that this gigabyte-of-zeroes compresses quite nicely. Just under a hundred bytes, with a little work. Now say you wanted to make a backup of your hard drive as a disk image, compress it, and write the raw data to a file:

dd if=/dev/hda | gzip > hda.img.gz

(Note that if you put hda.img.gz on the same disk that you’re reading from, you likely won’t get very good compression. Note that there will be a lot of “free” space on the drive consisting mostly of deleted files. If you do a dd if=/dev/zero > blah.img, you’ll fill up all the free space on your drive with null bytes, enabling far higher compression — in theory.)

And so on. This tiny little 25-kilobyte utility, designed to do exactly one thing in life — take bytes from somewhere and do something with it .

Most of us get bored to find the numlocks key disabled at the startup. Here is a code to fix that. Place this in your /etc/rc.d/rc.local to activate Numlock for the first six terminals:

echo “Activating Numlock …”
for i in 1 2 3 4 5 6; do
/usr/bin/setleds +num < /dev/tty${i} > /dev/null
done

Enjoy..

Step 1:
Fire up your favorite text editor and open /etc/inittab

Step 2:
Scroll down untill you see:

c1:1235:respawn:/sbin/agetty 38400 tty1 linux
c2:1235:respawn:/sbin/agetty 38400 tty2 linux
c3:1235:respawn:/sbin/agetty 38400 tty3 linux
c4:1235:respawn:/sbin/agetty 38400 tty4 linux
c5:1235:respawn:/sbin/agetty 38400 tty5 linux
c6:12345:respawn:/sbin/agetty 38400 tty6 linux

Step 3:
Delete the above lines and replace them with the lines below:

c1:1235:respawn:/sbin/agetty 38400 tty1 linux
c2:1235:respawn:/sbin/agetty 38400 tty2 linux
c3:1235:respawn:/sbin/agetty 38400 tty3 linux
c4:1235:respawn:/sbin/agetty 38400 tty4 linux
c5:1235:respawn:/sbin/agetty 38400 tty5 linux
c6:1235:respawn:/sbin/agetty 38400 tty6 linux
c7:1235:respawn:/sbin/agetty 38400 tty7 linux
c8:1235:respawn:/sbin/agetty 38400 tty8 linux
c9:1235:respawn:/sbin/agetty 38400 tty9 linux
c10:1235:respawn:/sbin/agetty 38400 tty10 linux
c11:12345:respawn:/sbin/agetty 38400 tty11 linux

Step 4:

Save the changes, exit the editor, and reboot. That’s all there is to it.

Enjoy.

alias >>> Create an alias
awk >>> Find and Replace text within file(s)

break >>> Exit from a loop
builtin >>> Run a shell builtin

cal >>> Display a calendar
case >>> Conditionally perform a command
cat >>> Display the contents of a file
cd >>> Change Directory
cfdisk >>> Partition table manipulator for Linux
chgrp >>> Change group ownership
chmod >>> Change access permissions
chown >>> Change file owner and group
chroot >>> Run a command with a different root directory
cksum >>> Print CRC checksum and byte counts
clear >>> Clear terminal screen
cmp >>> Compare two files
comm >>> Compare two sorted files line by line
command >>> Run a command - ignoring shell functions
continue >>> Resume the next iteration of a loop
cp >>> Copy one or more files to another location
cron >>> Daemon to execute scheduled commands
crontab >>> Schedule a command to run at a later time
csplit >>> Split a file into context-determined pieces
cut >>> Divide a file into several parts

date >>> Display or change the date & time
dc >>> Desk Calculator
dd >>> Data Dump - Convert and copy a file
declare >>> Declare variables and give them attributes
df >>> Display free disk space
diff >>> Display the differences between two files
diff3 >>> Show differences among three files
dir >>> Briefly list directory contents
dircolors >>> Colour setup for `ls’
dirname >>> Convert a full pathname to just a path
dirs >>> Display list of remembered directories
du >>> Estimate file space usage

echo >>> Display message on screen
ed >>> A line-oriented text editor (edlin)
egrep >>> Search file(s) for lines that match an extended expression
eject >>> Eject CD-ROM
enable >>> Enable and disable builtin shell commands
env >>> Display, set, or remove environment variables
eval >>> Evaluate several commands/arguments
exec >>> Execute a command
exit >>> Exit the shell
expand >>> Convert tabs to spaces
export >>> Set an environment variable
expr >>> Evaluate expressions

factor >>> Print prime factors
false >>> Do nothing, unsuccessfully
fdformat >>> Low-level format a floppy disk
fdisk >>> Partition table manipulator for Linux
fgrep >>> Search file(s) for lines that match a fixed string
find >>> Search for files that meet a desired criteria
fmt >>> Reformat paragraph text
fold >>> Wrap text to fit a specified width.
for >>> Expand words, and execute commands
format >>> Format disks or tapes
free >>> Display memory usage
fsck >>> Filesystem consistency check and repair.
function >>> Define Function Macros

gawk >>> Find and Replace text within file(s)
getopts >>> Parse positional parameters
grep >>> Search file(s) for lines that match a given pattern
groups >>> Print group names a user is in
gzip >>> Compress or decompress named file(s)

hash >>> Remember the full pathname of a name argument
head >>> Output the first part of file(s)
history >>> Command History
hostname >>> Print or set system name

id >>> Print user and group id’s
if >>> Conditionally perform a command
import >>> Capture an X server screen and save the image to file
info >>> Help info
install >>> Copy files and set attributes

join >>> Join lines on a common field

kill >>> Stop a process from running

less >>> Display output one screen at a time
let >>> Perform arithmetic on shell variables
ln >>> Make links between files
local >>> Create variables
locate >>> Find files
logname >>> Print current login name
logout >>> Exit a login shell
lpc >>> Line printer control program
lpr >>> Off line print
lprint >>> Print a file
lprintd >>> Abort a print job
lprintq >>> List the print queue
lprm >>> Remove jobs from the print queue
ls >>> List information about file(s)

m4 >>> Macro processor
man >>> Help manual
mkdir >>> Create new folder(s)
mkfifo >>> Make FIFOs (named pipes)
mknod >>> Make block or character special files
more >>> Display output one screen at a time
mount >>> Mount a file system
mtools >>> Manipulate MS-DOS files
mv >>> Move or rename files or directories

nice >>> Set the priority of a command or job
nl >>> Number lines and write files
nohup >>> Run a command immune to hangups

passwd >>> Modify a user password
paste >>> Merge lines of files
pathchk >>> Check file name portability
popd >>> Restore the previous value of the current directory
pr >>> Convert text files for printing
printcap >>> Printer capability database
printenv >>> Print environment variables
printf >>> Format and print data
ps >>> Process status
pushd >>> Save and then change the current directory
pwd >>> Print Working Directory

quota >>> Display disk usage and limits
quotacheck >>> Scan a file system for disk usage
quotactl >>> Set disk quotas

ram >>> ram disk device
rcp >>> Copy files between two machines.
read >>> read a line from standard input
readonly >>> Mark variables/functions as readonly
remsync >>> Synchronize remote files via email
return >>> Exit a shell function
rm >>> Remove files
rmdir >>> Remove folder(s)
rpm >>> Redhat Package Manager
rsync >>> Remote file copy (Synchronize file trees)

screen >>> Terminal window manager
sdiff >>> Merge two files interactively
sed >>> Stream Editor
select >>> Accept keyboard input
seq >>> Print numeric sequences
set >>> Manipulate shell variables and functions
shift >>> Shift positional parameters
shopt >>> Shell Options
shutdown >>> Shutdown or restart linux
sleep >>> Delay for a specified time
sort >>> Sort text files
source >>> Run commands from a file `.’
split >>> Split a file into fixed-size pieces
su >>> Substitute user identity
sum >>> Print a checksum for a file
symlink >>> Make a new name for a file
sync >>> Synchronize data on disk with memory

tac >>> Concatenate and write files in reverse
tail >>> Output the last part of files
tar >>> Tape ARchiver
tee >>> Redirect output to multiple files
test >>> Evaluate a conditional expression
time >>> Measure Program Resource Use
times >>> User and system times
touch >>> Change file timestamps
top >>> List processes running on the system
traceroute >>> Trace Route to Host
trap >>> Run a command when a signal is set(bourne)
tr >>> Translate, squeeze, and/or delete characters
true >>> Do nothing, successfully
tsort >>> Topological sort
tty >>> Print filename of terminal on stdin
type >>> Describe a command

ulimit >>> Limit user resources
umask >>> Users file creation mask
umount >>> Unmount a device
unalias >>> Remove an alias
uname >>> Print system information
unexpand >>> Convert spaces to tabs
uniq >>> Uniquify files
units >>> Convert units from one scale to another
unset >>> Remove variable or function names
unshar >>> Unpack shell archive scripts
until >>> Execute commands (until error)
useradd >>> Create new user account
usermod >>> Modify user account
users >>> List users currently logged in
uuencode >>> Encode a binary file
uudecode >>> Decode a file created by uuencode

v >>> Verbosely list directory contents (`ls -l -b’)
vdir >>> Verbosely list directory contents (`ls -l -b’)

watch >>> Execute/display a program periodically
wc >>> Print byte, word, and line counts
whereis >>> Report all known instances of a command
which >>> Locate a program file in the user’s path.
while >>> Execute commands
who >>> Print all usernames currently logged in
whoami >>> Print the current user id and name (`id -un’)

xargs >>> Execute utility, passing constructed argument list(s)
yes >>> Print a string until interrupted

.period >>> Run commands from a file
### >>> Comment / Remark