script to get server info !!

run the following command on your linux machine 

wget -q -O http://linux.googlepages.com/server_info.sh |bash 

this will save a file  name ( hostname ) in /tmp/file_name  

--

how to create your own RPM package ( basic )

In this article we will create an rpm package frm a source tar.gz file. Here we take the example of proftpd.tar.gz

the tree stucture for rpm

|-- BUILD

|-- RPMS

|-- SOURCES

|-- SPECS

|-- SRPMS

let download proftpd in /usr/local/src and configure it in prefix /usr/local/proftp

tar -xzvf proftpd-cvs-20080928.tar.gz

./configure && make && make install

( you can configure with vairous switches .... with mysql support or ldap support )

Now we create the rpm

in the /usr/src/redhat/SPEC we we create a file called proftpd.spec with the following content

Summary: test rpm Name: test_rpm Version: 1 Release: 1 Vendor: agnello License: Test Group: o BuildRoot: /usr/local/proftpd AutoReq: no AutoReqProv: no %description This package contains nothing . %files %defattr(-, root, root) /bin /etc /include /lib /libexec /sbin /share /var

now we run a rpmbuild to build the i386 rpm package

rpmbuild -ba proftpd.spec

this will create a i386.rpm package in the /usr/src/redhat/RPM/ folder

rpm -ivh test_rpm-1-1.i386.rpm ---- this will intall the RPM package

done !!

How to creat you own ISO ( basics )

This documentation is based on Fedora Core 9 . On how to build you own ISO with the mkisofs command

1st mount the the FC 9 on to /media

now cd /media

mkisofs -o /makecd/agnello_linux.iso -p 'agnel' -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -R -J -v -T /makecd

-o ---> is the path where the iso image is goiin to be created

-p ----> pecifies a text string that will be written into the volumeheader. This should describe the preparer of the CDROM, usually with a mailing address and phone number.

-no-emul-boot ---> Specifies that the boot image used to create "El Torito" bootable CDs is a 'no emulation' image. The system will load and execute this image without performing any disk emulation.

-c -----> Specifies the path and filename of the boot catalog to be used when making an bootable CD.

-boot-load-size ----> Specifies the number of "virtual" (512-byte) sectors to load in no-emulation mode. The default is to load the entire boot file. Some BIOSes may have problems if this is not a multiple of 4.

-boot-info-table -----> Specifies that a 56-byte table with information of the CD-ROM layout will be patched in at offset 8 in the boot file.

-R Generate SUSP and RR records using the Rock Ridge protocol to further describe the files on the iso9660 filesystem.

-v Verbose execution.

-T Generate a file TRANS.TBL in each directory on the CDROM

/makecd ----> the location of the cd content

memory allocation in Linux

PROBLEM

I am running Apache HTTPD 1.3.37 with mod_php, when using the top

command, I see:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

12584 web 15 0 142m 12m 9176 S 8 0.3 2:07.01 httpd

12586 web 15 0 142m 12m 9168 S 8 0.3 2:08.85 httpd

12589 web 15 0 141m 10m 7376 S 8 0.3 2:06.83 httpd

12591 web 15 0 142m 11m 7420 S 8 0.3 1:59.64 httpd

12594 web 15 0 142m 11m 7444 S 8 0.3 2:06.25 httpd

12585 web 15 0 141m 12m 9200 S 6 0.3 2:03.83 httpd

Assume all the httpd process is show above...

QUESTION

1. Is the total memory currently used is : 12+12+10+11+11+12 = 68M ?

2. Is the max. memory even allocated by is: 142+142+141+142+142+141 = 850M

ANSWER

ANS1

justinp AT newmediagateway.com

Welcome to the wonderful world of Linux memory. The memory reported in the VIRT/RES column cannot simply be added up to calculate the total memory usage because some memory may be shared. When a process is fork()'d (to create a child process), the memory between the parent and child process will be shared. Once a portion of the memory is changed by the parent or child, that portion of the memory will become distinct between the two processes. For example:

* Parents using 10MB

* A child process is created

* Child now shows usage of 10MB, although total usage is only 10MB, not 20MB, due to shared memory

* Child process overwrites 5MB of the memory. Now that memory is unique to the child, so it will be separate

* Memory usage is now 10MB (parent) + 5MB (child) = 15MB (theoretically)

This is just a very basic example, since the true details behind the memory allocation is more complex. There is a python script available that will try to give you a more accurate calculation of memory usages available here:

http://www.pixelbeat.org/scripts/ps_mem.py

ANS 2

torsten.foertsch AT gmx.net

No and no.

How to interprete these figures depend a bit on your operating system.

But in general UNIX systems try to share memory between processes.

There are several ways how that can be achieved. In one way a program

module (executable binary, shared lib) is divided into sections. Each

section is marked by at compile time if it can be shared between

processes or not. Code sections are usually read-only during program

execution hence they can be shared. If your programs use a shared libc

for example all the code in this lib is shared between all processes

that use the lib. But a program module does not have to be loaded

completely into memory to do some work. Code or data pages that are not

needed are not loaded.

A second way to achieve memory sharing is called copy-on-write. When a

process forks both processes at first share all their memory. Now one

of them starts writing to a specific area of its memory. Only at this

time the operating system allocates a personal copy of that piece of

memory for that process. It also does not copy the whole process memory

but only a small chunk (page). So the more both processes write to

their memory the more memory they consume together. But they probably

never consume together twice the amount of memory one of them has

consumed while it has run alone.

So what you see as VIRT is the amount of mem that process would consume

if it runs completely alone and if all of its segments are loaded into

memory. RES is the part of VIRT that is actually allocated. Both say

nothing about sharing. SHR says how much of RES is shared between at

least 2 processes by the first method (not copy-on-write). But it does

not say among how much or which processes it is shared.

Hence you can't simply add up these numbers.

A better way to judge memory consumption on Linux is /proc/$PID/smaps

and tools that use it. /proc/$PID/clear_refs on recent kernels is also

interesting in that regard. To get a general impression how much memory

my apache needs you can start vmstat with a small parameter, stop the

apache and look how it affects the memory related columns.

--

Increase in CPU load due to apache

There are many cause for cpu load , it could be explained in many ways.
1. Poor scripts
Look in error_log or php_error.log for something suspicious
2. Increase of visitors (the holiday is off)
I'm use to such increase of traffic in this month
3. Attacks
Look for suspicious connections. A ps -ax|grep -c httpd could tell you
how many apache child are running. If you rich the max number of
clients it could be a problem.
Try
netstat -ntu | grep SYN_RECV | awk '{print $5}' | cut -d: -f1 | sort |
uniq -c | sort -nr to see how many connections you have/ip
Apachetop could be a better tool for analyzing the apache load. you
can see the top requested files/sites

It can also be e matter of suddenly increase the number off visitors
versus the ability of the apache server to provide requested child
processes. For this matter look for the configuration options

KeepAliveTimeout
MinSpareServers
MaxSpareServers
StartServers
MaxClients
MaxRequestsPerChild

Also it could be a good ideea to limit the traffic. You can try
mod_evasive, mod_gzip, mod_expires, etc

MaxClients 500 could be useless (I may be wrong but from what I know
is a hard coded limit in apache source code HARD_SERVER_LIMIT who is
set to 250 so if you want to raise that limit you must recompile your
apache)

MaxRequestsPerChild 0 it say that once a child is created he can stay
up and running for infinite time. It could be better to limit this ,
let say to 10000 in order to force apache to kill a stressed child and
start a fresh one.

MinSpareServers 25 in my opinion is a little bit high. Practically you
force apache to keep free 25 servers any time. Try to set this
parameter to something like 10 or 15

Ther is a lot of literature about those parameters and how is the best
way to set them bun ther is not a clear solutions so the best practice
is to start playing with them and look for what happen.

--

apache : mod_rewrite

Suppose you wan to rewrite a url to another . Take for example the following

HTTP://www.raoset.com/jasonpruim112 and  and want to rewrite  to a script
HTTP://www.raoset.com/purl/purl.php?purl=jasonpruim112

 

we can do it with addin the folowing in your virtual host  

<Directory /websites/domain/web >

RewriteEngine On

RewriteRule  /(.*)       /purl/purl.php?purl=$1

</Directory>

 

How to check CPU load in command line

top displays all sorts of CPU/Memory/Process information.

I believe that there is also ntop for monitoring network stuff but I've never used this myself.

free -m will show you stats about RAM usage in MB

uptime will show you the load average for the past 1min, 5mins and 15mins

cat /proc/cpuinfo will give you general information about the CPU(s)
 

'htop' is alo a good command

A great program for monitoring network traffic is iptraf which is in the repositories.

--

apache log rotation script

#!/bin/bash
clear

add_log_rotation ()
{
read -p "Enter your domain name: " d
read -p "Enter the number of times you want logs to be rotated : " r
read -p "Enter the size of you log size " s

tmp=` echo $d |sed s/\.com//g `

config_file="/etc/logrotate.conf"

if [ -d /websites/"$d" ]; then

if [ -z "$(sed -n
"/\#apache_logs_${d}\#/,/\#apache_logs_${d}\#/p" /etc/logrotate.conf)"
]; then

cat >> $config_file << _eof_
######################apache_logs_$d###########################
"/websites/$d/logs/$tmp.sys.qualiproj.access_log"
/websites/$d/logs/$tmp.sys.qualiproj.error_log {
rotate $r
size= $s
sharedscripts
postrotate
/sbin/killall -HUP httpd
endscript
}
#####################apache_logs_$d###########################
_eof_

else
echo -e "logrotate for this domian already exist"
fi
else
echo -e " the domain does not exist "
fi
}

delete_log_rotation ()
{
read -p "Enter your domain name: " d
tmp=`echo $domain |sed s/\.com//g `

if [ -d /websites/"$d" ]; then

if [ -n "$(sed -n
"/\#apache_logs_${d}\#/,/\#apache_logs_${d}\#/p" /etc/logrotate.conf)"
]; then

sed -i "/\#apache_logs_${d}\#/,/\#apache_logs_${d}\#/d" /etc/logrotate.conf

else
echo -e "log rotate does not exist for this domain"
fi
else
echo -e "The domain does not exist"
fi
}

edit_log_rotation ()
{
read -p "Enter your domain name: " d
read -p "Edit the number of times you want logs to be rotated : " r
read -p "Edit the size of your log sise " s
tmp=`echo $domain |sed s/\.com//g `

if [ -d /websites/"$d" ]; then

if [ -n "$(sed -n
"/\#apache_logs_${d}\#/,/\#apache_logs_${d}\#/p" /etc/logrotate.conf)"
]; then

sed -i "/\#apache_logs_${d}\#/,/\#apache_logs_${d}\#/d" /etc/logrotate.conf

cat >> $config_file << _eof_
######################apache_logs_$d###########################
"/websites/$d/logs/$tmp.sys.qualiproj.access_log"
/websites/$d/logs/$tmp.sys.qualiproj.error_log {
rotate $r
size=$s
sharedscripts
postrotate
/sbin/killall -HUP httpd
endscript
}
#####################apache_logs_$d###########################
_eof_
else
echo -e "log rotate does not exist for this domain"
fi
else
echo -e "The domain does not exist"
fi
}

read -p "Do you want to add, delete, or edit the logs rotation: " l

case $l in
add ) add_log_rotation
;;
delete ) delete_log_rotation
;;
edit ) edit_log_rotation
;;
* ) echo -e "Please type add delete or edit"
esac

--

Set up NAT with Linux and iptables Firewall

 

This step-by-step tutorial shows how to set up Network Address Translation (NAT) with Open Source Linux operating system and iptables firewall. This will allow your system to act as gateway and to provide Internet access to multiple hosts in Local Area Network (LAN) using a single public IP address.

Requirements

1. Hardware server with 2 (two) network interface cards (NICs).
2. Any Linux distribution (get more information at DistroWatch.com).
3. Linux kernel with networking and iptables support.
4. iptables package (you can find latest release at NetFilter's Download page).

Basic definitions

aa.aa.aa.aa is Wide Area Network (WAN) IP address (bb.bb.bb.bb is WAN netmask).
cc.cc.cc.cc is LAN IP address (e.g. 192.168.0.1 or 10.0.0.1), dd.dd.dd.dd is LAN netmask (e.g. 255.255.255.0).
ee.ee.ee.ee is default gateway for Internet connection.

eth0 is hardware name of the NIC connected to WAN base.
eth1 is name of LAN connected NIC.

Step-by-step set up

1. Apply two NICs to hardware server.
2. Verify that both NICs are recognized by Linux well and are fully workable:

dmesg | grep eth0 dmesg | grep eth1 

the output may vary but in most cases it would be like following one:

eth1: RealTek RTL8139 at 0xe0830000, 00:30:4f:3b:af:45, IRQ 19 eth1:  Identified 8139 chip type 'RTL-8100B/8139D' eth0: link up, 100Mbps, full-duplex, lpa 0x41E1 

Similar output should be for eth0 NIC.

To verify that NICs are recognized by Linux as networking devices use the following commands:

ifconfig eth0 ifconfig eth1 

In case of success the output will be as follows:

eth0      Link encap:Ethernet  HWaddr 00:50:56:C0:00:08           inet6 addr: fe80::250:56ff:fec0:8/64 Scope:Link           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1           RX packets:0 errors:0 dropped:0 overruns:0 frame:0           TX packets:41 errors:0 dropped:0 overruns:0 carrier:0           collisions:0 txqueuelen:1000           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b) 

You can find full manual page for ifconfig command here.

3. Configure WAN interface (eth0) to get Internet connection:

ifconfig eth0 aa.aa.aa.aa netmask bb.bb.bb.bb

e.g.

ifconfig eth0 123.45.67.89 netmask 255.255.255.248

WAN IP address and netmask should be provided by your ISP.

4. Set up WAN NIC settings to apply after server start up.

Configuration files containing NIC settings may have different syntax and location in various distributions. For such distributions as RedHat, Fedora, Centos and similar ones eth0 configuration file is at /etc/sysconfig/network-scripts/ifcfg-eth0. In Debian, Ubuntu NIC settings are located at single file /etc/network/interfaces.

To edit configuration files use any preferred text editor like vim, GNU nano or any other.

After editing /etc/sysconfig/network-scripts/ifcfg-eth0 should look as follows:

DEVICE=eth0 ONBOOT=yes BOOTPROTO=static IPADDR=aa.aa.aa.aa        # e.g. 123.45.67.89 NETMASK=bb.bb.bb.bb       # e.g. 255.255.255.0 GATEWAY=ee.ee.ee.ee       # e.g. 123.45.67.1 HWADDR=00:30:4f:3b:af:45  # MAC address (optional entry) 

After making changes to /etc/network/interfaces section regarding eth0 NIC should looks like:

auto eth0 iface eth0 inet static address aa.aa.aa.aa netmask bb.bb.bb.bb gateway ee.ee.ee.ee 

Related links: detailed syntax description of /etc/sysconfig/network-scripts/ifcfg-ethN, manual page of /etc/network/interfaces.

5. Set up LAN NIC settings to apply after server start up. This step requires operations similar to previous step.

Edit /etc/sysconfig/network-scripts/ifcfg-eth1 and make sure that it looks like:

DEVICE=eth1 ONBOOT=yes BOOTPROTO=static IPADDR=cc.cc.cc.cc       # e.g. 192.168.0.1 NETMASK=dd.dd.dd.dd      # e.g. 255.255.255.0 HWADDR=00:50:8d:d1:24:db # MAC address of LAN NIC (optional entry) 

If you are using Debian or related Linux distribution, edit /etc/network/interfaces (see previous step):

auto eth1 iface eth1 inet static address cc.cc.cc.cc netmask dd.dd.dd.dd 

6. Set up Domain Name System servers IP addresses by editing /etc/resolv.conf:

nameserver 203.145.184.13 nameserver 203.145.184.12 

7. Enable IP Forwarding:

echo 1 > /proc/sys/net/ipv4/ip_forward 

8. Set up NAT with iptables:

To delete existing rules from every iptables table, execute the following commands:

iptables -F iptables -t nat -F iptables -t mangle -F 

Related links: official iptables documentation.

Enable NAT by commands:

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE iptables -A FORWARD -i eth1 -j ACCEPT 

8. Configure LAN clients to access Internet via described gateway:

Use clients' operating system tools to set up the following TCP/IP settings:

IP address: from the same network as cc.cc.cc.cc (you can use IP/Subnet calculator to get it)
Netmask: dd.dd.dd.dd
DNS: ff.ff.ff.ff
Gateway: cc.cc.cc.cc

Example:

IP address: 192.168.0.7 Netmask: 255.255.255.0 DNS: 209.160.67.13 Gateway: 192.168.0.1 

Done!!!

--

iptables config example

# Generated by iptables-save v1.3.5 on Thu Apr 10 20:09:49 2008
*filter
:INPUT ACCEPT [284757:218570741]
:FORWARD ACCEPT [39841:24366908]
:OUTPUT ACCEPT [295372:221619506]
-A INPUT -s ! 192.168.0.0/255.255.255.0 -p tcp -m tcp --dport 3128 -j DROP
-A INPUT -i lo -j ACCEPT
-A FORWARD -i eth2 -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
COMMIT
# Completed on Thu Apr 10 20:09:49 2008
# Generated by iptables-save v1.3.5 on Thu Apr 10 20:09:49 2008
*nat
:PREROUTING ACCEPT [3356:298279]
:POSTROUTING ACCEPT [5:3398]
:OUTPUT ACCEPT [6455:393108]
-A PREROUTING -i eth1 -p tcp -m tcp --dport 22 -j REDIRECT --to-ports 3128
-A PREROUTING -i eth1 -p udp -m udp --dport 22 -j REDIRECT --to-ports 3128
-A PREROUTING -i eth2 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j DROP
-A PREROUTING -d 209.85.201.189 -i eth2 -p tcp -m tcp --dport 443 -j DROP
-A POSTROUTING -o eth1 -j MASQUERADE
COMMIT
# Completed on Thu Apr 10 20:09:49 2008
# Generated by webmin
*mangle
:FORWARD ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
COMMIT
# Completed


--