reiserfs file system info
http://linuxtweaks.wordpress.com/tag/reiserfs-reiser3/
http://www.funtoo.org/en/articles/linux/ffg/2/
http://www.webhostingtalk.com/archive/index.php/t-410205.html
pxeboot installations how to
wget http://192.168.10.105/pxebootCentos54_64bit.tgz
- untar pxebootCentos54_64bit.tgz to your /boot
- add the following to /etc/grub.conf
title CentOS 5.4 Installation
root (hd0,0)
kernel /pxeboot/vmlinuz ro root=LABEL=/ ip=192.168.50.110 netmask=255.255.255.0 gateway=192.168.2.234
dns=192.168.2.234 hostname=centos44 text ks=http://202.162.242.200/ks.cfg ksdevice=eth0
initrd /pxeboot/initrd.img
this is the ks.fcg file http://192.168.10.105/ks.cfg
auth --useshadow --enablemd5
install
url --url=http://192.168.10.105/CentOS54X
lang en_US
keyboard us
network --bootproto=static --device=eth0 --ip=206.183.108.166 --netmask=255.255.255.0 --gateway=206.183.108.1 --nameserver=209.120.245.58 --hostname=dell860-535.rapidns.com --onboot=on
zerombr
rootpw --iscrypted $1$4kHiDPby$etAhqn01O83WXVXju0K/S/
firewall --disable
firstboot --disable
authconfig --enableshadow --enablemd5
selinux --disable
timezone Asia/Calcutta
bootloader --location=mbr --append="rhgb quiet"
zerombr yes
skipx
clearpart --all
part --ondrive=sda / --fstype ext3 --size=1 --grow
part --ondrive=sda /boot --fstype ext3 --size=200 --asprimary
part --ondrive=sda swap --size=7024
reboot
%packages
@base
@editors
@core
@development-libs
@development-tools
@sql-server
kernel-devel
emacs
- make the above added kernel image to be default image to boot and reboot the server.
- After reboot Centos 5.4 installation will begin.
Script removes all old files from particular directories
#!/bin/bash
#
# This Script removes all old files from particular directories
#
set -x
remFiles(){
path=`echo $1` # This is inefficient but We dont expect too many files
days=$2
dirs=$3
for dir in $path;do
if [ -z "$dirs" ];then
find "$dir" -mtime +$days -type f -exec rm -vf {} \;
else
find "$dir" -mtime +$days -type d -exec rm -vfr {} \;
fi
done
}
remFiles "/net/serverbackup/backup/*" "2"
#
# This Script removes all old files from particular directories
#
set -x
remFiles(){
path=`echo $1` # This is inefficient but We dont expect too many files
days=$2
dirs=$3
for dir in $path;do
if [ -z "$dirs" ];then
find "$dir" -mtime +$days -type f -exec rm -vf {} \;
else
find "$dir" -mtime +$days -type d -exec rm -vfr {} \;
fi
done
}
remFiles "/net/serverbackup/backup/*" "2"
Important things to know in apache
ServerAdmin
ServerAdmin webmaster@domain.comSets the email address for the server administrator - this will be used if you have setup the server to contact you on errors. It is also shown in the ServerSignature (if set to 'Email' - see below)
Domain Name
ServerName and ServerAlias
ServerName domain.com
ServerAlias www.domain.comSets the domain name for the virtual host. You can have as many aliases as required. For example, you can have domain.com and domain.net point to the same content.
Note this is not a rewrite rule (we'll look at those later) but the domains defined here will serve the same content (assuming you have set the DNS to point to your Slice IP).
Index Files
DirectoryIndex
DirectoryIndex index.htmlDefines the index file (the 'home' page that is shown on entering the domain address). Useful if you have want the user to be directed to an alternate page or to a non-standard home page.
Do note this is not a good way of redirecting users as they may go directly to a non specified page such as domain.com/index.php whilst the DirectoryIndex will only work for those entering domain.com.
Documents
DocumentRoot
DocumentRoot /home/demo/public_html/domain.com/publicThe location of the domain's public files. Use an absolute path name.
Log Files
ErrorLog and CustomLog
LogLevel warn
ErrorLog /home/demo/public_html/domain.com/log/error.log
CustomLog /home/demo/public_html/domain.com/log/access.log combinedSet the Log levels and the location for the Virtual Hosts log files. Very useful for easy analysis of the domain statistics.
Error Documents
ErrorDocument
ErrorDocument 404 /errors/404.html
ErrorDocument 403 /errors/403.htmlUsed for all the standard error messages.
In these examples I have an 'errors' folder in my public directory. I created each error document and place them in the 'errors' folder. The paths shown are relative to the DocumentRoot folder defined above.
If not defined, Apache will generated its own error pages. Custom error pages are more user friendly and can be customised as much, or as little, as you want.
Apache Footers
ServerSignature
ServerSignature OnSets whether the server details are displayed in any server generated error pages or index lists. Options are On, Off and Email.
Note the level of detail in the signature is configured via ServerTokens which cannot be set in the Virtual Hosts file - only in the main apache2.conf. See the Apache configuration #2 article for more details.
If set to Email, the ServerAdmin email will be displayed.
cgi-bin
ScriptAlias
ScriptAlias /cgi-bin/ /home/demo/public_html/domain.com/cgi-bin/
<Location /cgi-bin>
Options +ExecCGI
</Location>Enables the cgi-bin location as defined by the custom virtual hosts layout. You can, of course, leave the cgi-bin in the DocumentRoot location if you so wish.
Directory
<Directory xxx/xxx>
<Directory /home/demo/public_html/domain.com/public>
Options FollowSymLinks
</Directory>Set the Options for the specified directory - the example shown allows the Option FollowSymLinks to be enable for the public directory of domain.com
Listed below are further Options that can be set:
Directory Browsing
Options
Options -IndexesTo turn off directory browsing use '-Indexes' or 'None'. To turn them on, use '+Indexes'.
SSI
Options
Options -IncludesThis Option disables Server Side Inlcudes.
Symlinks
Options
Options -FollowSymLinksEnable or disable the option to follow symlinks. Be careful with this option as it can lead to security risks (inadvertently linking to configuration folders).
Dejay Clayton made a good suggestion in using SymLinksIfOwnerMatch instead of FollowSymLinks.
The SymLinksIfOwnerMatch allows symbolic links to be followed only if the owner of the link is identical to the owner of the target file or directory. Thus preventing many of the security risks than a simple FollowSymlinks can create.
.htaccess
AllowOverride
AllowOverride NoneSetting AllowOverride to none disables .htaccess support. Set to All to allow them.
You can also specify which .htaccess features to enable such as:
AllowOverride AuthConfig IndexesThe Apache AllowOverride docs has more information on the different features.
Remember to specifically protect your .htaccess file. This can be done in two ways:
Firstly rename it to something obscure and, secondly, deny access to the file from external sources:
AccessFileName .myobscurefilename
<Files ~ "^\.my">
Order allow,deny
Deny from all
Satisfy All
</Files>No Options
Options
Options NoneThis will turn off all the available options.
Hierarchy
Remember that the Options directives can be set per directory like this:
<Directory />
AllowOverride None
Options None
</Directory>
<Directory /home/demo/public_html/domain.com/public>
AllowOverride All
</directory>This will turn of all Options and disable .htaccess support for all directories.
However, the second Directory setting will override the first and allow .htaccess support for the domain.com/public directory.
Summary
The Virtual Hosts file is at once an easy tool to use and a very powerful one. My advice is to enter one setting and test it. Then enter the next setting and so on.
Once familiar you will see you have fine control over all of your web folders and files.
how to create linux ISO
BUILD Machines
Current Build machines are
192.168.40.100 <i386>
192.168.50.165 <x86_64>
Chose OS
Currently the build is done on centos5
Build Directories
On 192.168.40.100 <i386>
distros name
On 192.168.50.165 <x86_64>
distros name
Getting Started
eg to create a product TEST ( Test can be any of name you would like to give you distros )
<arch> ---- i386 or x86_64
Create directory /test of desired <arch> machine.
$ cd /test
$ mkdir <arch>
Copy the centos <arch> cd to /test/<arch>
$ mount /dev/cdrom /media
$ cd /media
$ cp -ar * /test/<arch>/
$ cp .discinfo /test/<arch>/
NOTE : .discinfo file os imp without wich it will give a error of not valid centos
Editing stage2
Editing Default CentOS 5.2 stage2.img
Install squashfs rpm
$ rpm -ivh squashfs-tools-3.0-4.x86_64.rpm
Copy the stage2.img file from the images folder of the cd
$ cp stage2.img /opt/
Make Directory
$ mkdir -p /opt/stage2
Mount the copied stage2.img
$ mount -t squashfs stage2.img /media/ -o loop
Copy Contents
$ cd /media
$ cp -arp * /opt/stage2
$ cp .buildstamp /opt/stage2
$ cd /opt/stage2
$ ls -a
. .. .buildstamp etc lib lib64 modules proc usr var
Now to start editing the stage2
All the python files are stored under /opt/stage2/usr/lib/anaconda
$ vi kickstart.py
Hash the lines this is so that kickstart will ask for network if not specified in ks file
if len(filter(lambda nd: nd.bootProto == "query", self.ksdata.network)) == 0:
dispatch.skipStep("network")
Hash this line so as to ask for the welcome screen
dispatch.skipStep("welcome")
Copy following image files (stage2) into /opt/stage2/usr/share/anaconda/pixmaps
anaconda_header.png
first-lowres.png
first.png
progress_first-375.png
progress_first-lowres.png
progress_first.png
splash1.png
splash.png
syslinux-splash.png
Copy /opt/stage2/usr/share/anaconda/pixmaps/rnotes after deleting all the files already present
03-centos5-yum.png
06-centos5-support.png
08-centos5-wiki.png
05-centos5-centosplus.png
01-centos5-welcome.png
02-centos5-donate.png
09-centos5-virtualization.png
07-centos5-docs.png
04-centos5-repos.png
now ur nearly done all thats left is a nice new stage2.img file
$ cd /opt/stage2
$ mksquashfs . ../stage2.img.new
$ mv stage2.img.new stage2.img
Copy the new stage2.img into /test/<arch>/images/
Creating the kickstart file for auto install
$ cd /test/<arch>
$ vi legacy-mpart.cfg
auth --useshadow --enablemd5
install
cdrom
lang en_US
keyboard us
zerombr
firstboot --disable
firewall --disabled
authconfig --enableshadow --enablemd5
selinux --disabled
bootloader --location=mbr --driveorder=hda
skipx
reboot
%packages
@ msx_packages
Include for auto partition under skipx
clearpart --all
part /boot --fstype="ext3" --size=200
part swap --recommended
part / --fstype="ext3" --grow --size=1
Including Scripts
Copy the scripts dir into /test/
$ cd /test/scripts
Make changes in each script according to arch...
Creating Make file
Create a Makefile in /test/ dir
vi Makefile
MKISOFS=/usr/bin/mkisofs
DATE:=$(shell date +"%d-%m-%Y")
CDDIR=/test
ARCH=x86_64
REPODIR=$(CDDIR)/$(ARCH)
RPMDIR=$(CDDIR)/$(ARCH)/CentOS
INSTDIR=$(CDDIR)/roots/installer/usr/local/net/_install
VER=$(shell scripts/version.sh)
#BUILDRPM=/usr/src/redhat/RPMS/$(ARCH)
BUILDRPM=/usr/src/redhat/RPMS/noarch
test: repoclean repo iso
repoclean:
rm -rf $(REPODIR)/repodata/filelists.xml.gz $(REPODIR)/repodata/other.xml.gz $(REPODIR)/repodata/primary.xml.gz $(REPODIR)/repodata/repomd.xml
svnco:
$(CDDIR)/scripts/svnco.sh $(CDDIR)
$(CDDIR)/scripts/premake $(CDDIR)
initrd:
$(CDDIR)/scripts/makeinitrd.sh $(CDDIR)
repo:
scripts/makecomps.sh $(REPODIR)
iso:
@$(MKISOFS) -o $(CDDIR)/images/MailServ_x64-$(DATE).iso -V MailServ-$(VER) -p 'EMS' -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4\
-boot-info-table -R -J -v -T $(REPODIR)
rpmtest:
$(CDDIR)/scripts/rpmtest.sh $(CDDIR)
$(CDDIR)/scripts/dupshow.pl $(RPMDIR)
dupshow:
$(CDDIR)/scripts/dupshow.pl $(RPMDIR)
isocheck:
$(CDDIR)/scripts/isocheck $(CDDIR)
Generating repos
$ cd /test/<arch>/CentOS/
Keep only the desired packages and delete the rest
Making Initrd
The Makefile includes initrd:
Please run
$ make initrd
This will pick up the kickstart file (legacy.cfg and include it into the initrd)
Isolinux edit
cd /test/<arch>/isolinux
$ vi isolinux.cfg
default linux
prompt 1
timeout 600
display boot.msg
F1 boot.msg
F2 options.msg
F3 general.msg
F4 param.msg
F5 rescue.msg
label linux
kernel vmlinuz
append ks=file:/legacy.cfg initrd=initrd.img
label mpart
kernel vmlinuz
append ks=file:/legacy-mpart.cfg initrd=initrd.img
label text
kernel vmlinuz
append initrd=initrd.img text
label ks
kernel vmlinuz
append ks initrd=initrd.img
label local
localboot 1
label memtest86
kernel memtest
append -
This is to prevent errors of cannot find kickstart file since it is in the initrd
Making CD
$ mkdir /test/images
make test
how load balancer works ( ldirector )
cat /etc/ha.d/ldirectord.cf
________
| |
| client |
|________|
CIP=SGW=192.168.2.62 (eth0)
|
|
__________ |
| | | (Virtual IP=192.168.2.110, eth0:110) listening on port 25
| director |---|
| (suse) | |
|__________| | Director IP=192.168.2.24 (eth0)
|
|
-----------------------------------
| |
| |
Real IP=192.168.2.214(eth0) Real IP=192.168.2.230(eth0)
(VIP=192.168.2.110, lo:0) (VIP=192.168.2.110, lo:0)
______|______ ______|______
| | | |
| realserver | | realserver |
|____________| |____________|
on the LVS server
quiescent=yes
checktimeout=10
checkinterval=30
autoreload=yes
logfile="/var/log/ldirectord.log"
virtual=192.168.2.110:25
real=192.168.2.214:25 gate 9
real=192.168.2.230:25 gate 8
service=smtp
protocol=tcp
scheduler=wrr
THE APACHE SERVER STATUS & ERROR CODES
| Successful Client Requests | ||
| 200 | OK | |
| 201 | Created | |
| 202 | Accepted | |
| 203 | Non-Authorative Information | |
| 204 | No Content | |
| 205 | Reset Content | |
| 206 | Partial Content | |
| Client Request Redirected | ||
| 300 | Multiple Choices | |
| 301 | Moved Permanently | |
| 302 | Moved Temporarily | |
| 303 | See Other | |
| 304 | Not Modified | |
| 305 | Use Proxy | |
| Client Request Errors | ||
| 400 | Bad Request | |
| 401 | Authorization Required | |
| 402 | Payment Required (not used yet) | |
| 403 | Forbidden | |
| 404 | Not Found | |
| 405 | Method Not Allowed | |
| 406 | Not Acceptable (encoding) | |
| 407 | Proxy Authentication Required | |
| 408 | Request Timed Out | |
| 409 | Conflicting Request | |
| 410 | Gone | |
| 411 | Content Length Required | |
| 412 | Precondition Failed | |
| 413 | Request Entity Too Long | |
| 414 | Request URI Too Long | |
| 415 | Unsupported Media Type | |
| Server Errors | ||
| 500 | Internal Server Error | |
| 501 | Not Implemented | |
| 502 | Bad Gateway | |
| 503 | Service Unavailable | |
| 504 | Gateway Timeout | |
| 505 | HTTP Version Not Supported | |
mysql questions
- How do you start and stop MySQL on Windows? - net start MySQL, net stop MySQL
- How do you start MySQL on Linux? - /etc/init.d/mysql start
- Explain the difference between mysql and mysqli interfaces in PHP? - mysqli is the object-oriented version of mysql library functions.
- What's the default port for MySQL Server? - 3306
- What does tee command do in MySQL? - tee followed by a filename turns on MySQL logging to a specified file. It can be stopped by
- command notee.
- Can you save your connection settings to a conf file? - Yes, and name it ~/.my.conf. You might want to change the permissions on the file to 600, so that it's not readable by others.
- How do you change a password for an existing user via mysqladmin? - mysqladmin -u root -p password "newpassword"
- Use mysqldump to create a copy of the database? - mysqldump -h mysqlhost -u username -p mydatabasename > dbdump.sql
- Have you ever used MySQL Administrator and MySQL Query Browser? Describe the tasks you accomplished with these tools.
- What are some good ideas regarding user security in MySQL? - There is no user without a password. There is no user without a user name. There is no user whose Host column contains % (which here indicates that the user can log in from anywhere in the network or the Internet). There are as few users as possible (in the ideal case only root) who have unrestricted access.
- Explain the difference between MyISAM Static and MyISAM Dynamic. - In MyISAM static all the fields have fixed width. The Dynamic MyISAM table would include fields such as TEXT, BLOB, etc. to accommodate the data types with various lengths. MyISAM Static would be easier to restore in case of corruption, since even though you might lose some data, you know exactly where to look for the beginning of the next record.
- What does myisamchk do? - It compressed the MyISAM tables, which reduces their disk usage.
- Explain advantages of InnoDB over MyISAM? - Row-level locking, transactions, foreign key constraints and crash recovery.
- Explain advantages of MyISAM over InnoDB? - Much more conservative approach to disk space management - each MyISAM table is stored in a separate file, which could be compressed then with myisamchk if needed. With InnoDB the tables are stored in tablespace, and not much further optimization is possible. All data except for TEXT and BLOB can occupy 8,000 bytes at most. No full text indexing is available for InnoDB. TRhe COUNT(*)s execute slower than in MyISAM due to tablespace complexity.
- What are HEAP tables in MySQL? - HEAP tables are in-memory. They are usually used for high-speed temporary storage. No TEXT or BLOB fields are allowed within HEAP tables. You can only use the comparison operators = and <=>. HEAP tables do not support AUTO_INCREMENT. Indexes must be NOT NULL.
- How do you control the max size of a HEAP table? - MySQL config variable max_heap_table_size.
- What are CSV tables? - Those are the special tables, data for which is saved into comma-separated values files. They cannot be indexed.
- Explain federated tables. - Introduced in MySQL 5.0, federated tables allow access to the tables located on other databases on other servers.
- What is SERIAL data type in MySQL? - BIGINT NOT NULL PRIMARY KEY AUTO_INCREMENT
- What happens when the column is set to AUTO INCREMENT and you reach the maximum value for that table? - It stops incrementing. It does not overflow to 0 to prevent data losses, but further inserts are going to produce an error, since the key has been used already.
- Explain the difference between BOOL, TINYINT and BIT. - Prior to MySQL 5.0.3: those are all synonyms. After MySQL 5.0.3: BIT data type can store 8 bytes of data and should be used for binary data.
- Explain the difference between FLOAT, DOUBLE and REAL. - FLOATs store floating point numbers with 8 place accuracy and take up 4 bytes. DOUBLEs store floating point numbers with 16 place accuracy and take up 8 bytes. REAL is a synonym of FLOAT for now.
- If you specify the data type as DECIMAL (5,2), what's the range of values that can go in this table? - 999.99 to -99.99. Note that with the negative number the minus sign is considered one of the digits.
- What happens if a table has one column defined as TIMESTAMP? - That field gets the current timestamp whenever the row gets altered.
- But what if you really want to store the timestamp data, such as the publication date of the article? - Create two columns of type TIMESTAMP and use the second one for your real data.
- Explain data type TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP - The column exhibits the same behavior as a single timestamp column in a table with no other timestamp columns.
- What does TIMESTAMP ON UPDATE CURRENT_TIMESTAMP data type do? - On initialization places a zero in that column, on future updates puts the current value of the timestamp in.
- Explain TIMESTAMP DEFAULT '2006:09:02 17:38:44′ ON UPDATE CURRENT_TIMESTAMP. - A default value is used on initialization, a current timestamp is inserted on update of the row.
- If I created a column with data type VARCHAR(3), what would I expect to see in MySQL table? - CHAR(3), since MySQL automatically adjusted the data type.
Difference between ip virtual hosting and name virtual hosting
IP-based virtual hosts use the IP address of the connection to determine the correct virtual host to serve. Therefore you need to have a separate IP address for each host. With name-based virtual hosting, the server relies on the client to report the hostname as part of the HTTP headers. Using this technique, many different hosts can share the same IP address.
Name-based virtual hosting is usually simpler, since you need only configure your DNS server to map each hostname to the correct IP address and then configure the Apache HTTP Server to recognize the different hostnames. Name-based virtual hosting also eases the demand for scarce IP addresses. Therefore you should use name-based virtual hosting unless there is a specific reason to choose IP-based virtual hosting. Some reasons why you might consider using IP-based virtual hosting:
Name-based virtual hosting is usually simpler, since you need only configure your DNS server to map each hostname to the correct IP address and then configure the Apache HTTP Server to recognize the different hostnames. Name-based virtual hosting also eases the demand for scarce IP addresses. Therefore you should use name-based virtual hosting unless there is a specific reason to choose IP-based virtual hosting. Some reasons why you might consider using IP-based virtual hosting:
- Some ancient clients are not compatible with name-based virtual hosting. For name-based virtual hosting to work, the client must send the HTTP Host header. This is required by HTTP/1.1, and is implemented by all modern HTTP/1.0 browsers as an extension. If you need to support obsolete clients and still use name-based virtual hosting, a possible technique is discussed at the end of this document.
- Name-based virtual hosting cannot be used with SSL secure servers because of the nature of the SSL protocol.
- Some operating systems
and network equipment implement bandwidth management techniques that cannot differentiate between hosts unless they are on separate IP addresses.
Important linux tips ( to read )
1) If you specify both deny from all and allow from all, what will be the default action of Apache?
deny will be taken first.
order allow, deny # connection will be denied
order deny, allow # connection will be allowed
2) what does ./configure , make and make in stall do
http://www.codecoffee.com/tipsforlinux/articles/27.html
3) what are shared libaries abs ldconfig or ld.so
http://www.linux.com/archive/feature/114007
4) what is openssl
SSL stands for Secure Sockets Layer. SSL's most common job, in the real world, is to encrypt the contents of web forms. This greatly decreases the chances that your credit card number, when entered into an online store's web form, ends up in somewhere else, other than the store's database. OpenSSL is the open source version of this and is available under the Apache-style license for free commercial and non-commercial use.
Since you're installing Apache, that means that you'll be on the receiving end of sensitive information. To help OpenSSL process this information, we'll need the help of a package called 'mm'. This is available at: http://www.ossp.org/pkg/lib/mm/. We'll need to compile and install this first. Get the tarball for 'mm' and untar it in our aforementioned apache_install directory. Now we're ready to configure, compile and install it. Do the following as a normal user:
./configure --disable-shared
Then:
make
and finally, as root.
make install
This will install the 'mm' libraries that OpenSSL can use to work more efficiently. Now we'll do the same for OpenSSL. You can pick up the source for OpenSSL from http://www.openssl.org/source/ Pick up the source tarball and untar it in our apache_install directory.
5) Fine tuning my.cnf
http://www.linuxweblog.com/tune-my.cnf
6) what is stiky bit
http://osr507doc.sco.com/en/OSAdminG/ssC.stickydirs.html
6) what is server type directive in apache ?
The ServerType directive sets how the server is executed by the system. Type is one of
inetd
The server will be run from the system process inetd; the command to start the server is added to /etc/inetd.conf
standalone
The server will run as a daemon process; the command to start the server is added to the system startup scripts. (/etc/rc.local or /etc/rc3.d/....)
Inetd is the lesser used of the two options. For each http connection received, a new copy of the server is started from scratch; after the connection is complete, this program exits. There is a high price to pay per connection, but for security reasons, some admins prefer this option. Inetd mode is no longer recommended and does not always work properly. Avoid it if at all possible.
Standalone is the most common setting for ServerType since it is far more efficient. The server is started once, and services all subsequent connections. If you intend running Apache to serve a busy site, standalone will probably be your only option.
Database Replication with MySQL
A typical way of backing up data in a database server is retrieving a dump of the database(s). Depending on the update frequency and the criticality of the data, this kind of backup can be done, once a week, once a day or even once every hour! When you conclude that the database is to be backed up once an hour or less than an hour, then retrieving a database dump (using tools like mysqldump in case of MySQL) is not an efficient way. This kind of requirement calls for a database replication setup. With replication, you can have two database servers, running on different machines having the same data at any point in time. The main database server, where applications and users connect, issue queries, add/update or delete data is called the master. The second database where the master database is mirrored is called the slave database. We are calling it mirroring because, under ideal circumstances, the slave database will always be an exact replica of the master database.
Replication makes the slave database an exact replica of the master. This means if some incorrect data makes its way into the master database, it will also get replicated on the slave. So replication is not a solution if you are looking for database snapshots with the goal of recovering the data from an earlier snapshot in case the latest data gets erroneous.
Let's get started with a replication setup for MySQL. For this article we assume that you are running MySQL on PCQLinux 2005 for both master and the slave databases. That is, we set up two machines with PCQLinux 2005, one will run the master database server and the other will be running the slave.
After installing PCQLinux 2005 on both the machines, install and set up MySQL on them as per the instructions in the section 'Connect to Mysql' in the article, JDBC Drivers, page 76, March 2005, PCQuest. For our setup, the master database server was running on a machine with the IP address 192.168.1.1 and the slave was running at 192.168.1.2.
Setting up the master
On the master machine, open the file, my.cnf, found in the directory /etc. Add the following under the [mysqld] section:
log-bin=/var/log/mysqlbin.log
server-id=1
Here the server-id must be a unique ID for the MySQL server. Later we'll give a server ID of 2 to the slave database server. Issue this
mysql -u root -p
|
When prompted, supply the password for the MySQL root user, specified during the setup mentioned in the article, JDBC Drivers. Subsequently, you will be dropped into a mysql> prompt. At this prompt issue:
grant replication slave on *.* to repl@"192.168.1.2" identified by 'secret123';
Substitute secret123 with your preferred password. We'll use the username 'repl' and password 'secret123' while setting up the slave in the section 'On the slave' later.
Take master's snapshot
Before starting the replication process, we need to import the latest snapshot or backup of the master's databases to the slave. Subsequently, once the replication is started, the slave will automatically sync with the master. We must ensure that the no updates happen on the master while taking the snapshot. For this we'll lock all the tables, allowing only 'read only' access. At the mysql> prompt, issue:
flush tables with read lock;
If your database is live and being used by a Web application(s) or any other application, you may like to put up a 'website under maintenance' page or inform users about it. Note that queries which only read the database like the select queries, will still work. Next, quit from the mysql> prompt by typing 'quit'.
Issue the following to take a database snapshot.
mysqldump -u root p --all-databases --master-data > backup.sql
When prompted, supply the password for MySQL's root user. Copy the file backup.sql to 192.168.1.2 (say using SSH or Secure SHell). Before proceeding with the slave configuration, release the read only lock. For this, get into the mysql> prompt and issue:
unlock tables;
This is also the right time to restart the database for the settings, made in the previous section, to get effective. Restart MySQL as:
/etc/init.d/mysql stop
/etc/init.d/mysql start
Your master database is back online and you may inform users to get started and remove the maintenance message from your site.
On the slave
On the slave machine open the file, my.cnf, found in /etc directory and add the following under the [mysqld] section.
server-id = 2
master-host = 192.168.1.1
master-user = repl
master-password = secret123
replicate-ignore-db=mysql
Note that we have set the server-id to 2, which must be different from the master database's server ID (which is 1 in our case). Also note that we have set the master-host to the IP address of the master machine and the master-user and master-password to the username and password that we had set up in the section 'Setting up the master'. With replicate-ignore-db we have specified not to replicate the database named mysql for the reasons mentioned in the following paragraph.
Next, we will import the data from backup.sql file. The backup.sql file will also contain the table definition and data for the database named mysql. You may like to exclude this database because it contains the database users and access privileges for the master. You may like to have a different set of users and access privileges on the slave. To exclude this database, open the file backup.sql on the slave machine in a text editor. Scroll down to the line that says, Current Database: `mysql`
Delete all the lines till you find the next Current Database line or upto the file's end. Delete all the SQL statements pertaining
to the mysql database. Now import backup.sql to the slave database as:
mysql -u root -p < backup.sql
When prompted, supply the password you had set up for root MySQL user on the slave database. Restart the slave database.
Get, set, go
To start the replication process, issue the start slave command from the mysql> prompt. Next, issue:
show slave status
Check whether the output shows any errors. Issue the above intermittently to check for any errors. If you don't notice any
error, it means the replication has been set up properly. To test, make changes (update or insert) on one of databases on the master machine. The changes will also get reflected on the slave machine too.
Not to forget that here you have not only configured a real-time back-up setup, but also a fault-tolerant setup. If the master machine goes down for some reason, point your application to the slave machine to resume your operations.
Shell perl scripts one liners
An equivalent of the other find-replace, except it's a one-liner that generates no temp files, and is more flexible:
perl -pi -e 's/find/replace/g' *.txt
Or, to change matching files in a hierarchy:
find . -name '*.txt' |xargs perl -pi -e 's/find/replace/g'
Find the full name of a user
If you want to find out the full name for a user name you can use one of these one-liners to do the job:
ypmatch matkin passwd | cut -d: -f5 | cut -d, -f1
grep "^matkin:" /etc/passwd | cut -d: -f5 | cut -d, -f1
Remove processes matching some regular expression
If you have a number of processes that you want to kill, one of the following one-liners might be useful:
kill `ps xww | grep "sleep" | cut -c1-5` 2>/dev/null
ps xww | grep "sleep" | cut -c1-5 | xargs kill 2>/dev/null
This will kill any processes that has the word "sleep" in the calling command. If your kill does not handle multiple pids' you can either use the one-liner
ps xww | grep "sleep" | cut -c1-5 | xargs -i kill {} 2>/dev/null
or use a for-loop:
for x in `ps xww | grep "sleep" | cut -c1-5`
do
kill $x 2>/dev/null
To list all files in the `/usr/local' directory tree that are greater than 10,000 kilobytes in size, type:
$ find /usr/local -size +10000k [RET]
To list all files in your home directory tree less than 300 bytes in size, type:
$ find ~ -size -300b [RET]
To list all files on the system whose size is exactly 42 512-byte blocks, type:
$ find / -size 42 [RET]
Use the `-empty' option to find empty files -- files whose size is 0 bytes. This is useful for finding files that you might not need, and can remove.
To find all empty files in your home directory tree, type:
$ find ~ -empty [RET]
perl -pi -e 's/find/replace/g' *.txt
Or, to change matching files in a hierarchy:
find . -name '*.txt' |xargs perl -pi -e 's/find/replace/g'
Find the full name of a user
If you want to find out the full name for a user name you can use one of these one-liners to do the job:
ypmatch matkin passwd | cut -d: -f5 | cut -d, -f1
grep "^matkin:" /etc/passwd | cut -d: -f5 | cut -d, -f1
Remove processes matching some regular expression
If you have a number of processes that you want to kill, one of the following one-liners might be useful:
kill `ps xww | grep "sleep" | cut -c1-5` 2>/dev/null
ps xww | grep "sleep" | cut -c1-5 | xargs kill 2>/dev/null
This will kill any processes that has the word "sleep" in the calling command. If your kill does not handle multiple pids' you can either use the one-liner
ps xww | grep "sleep" | cut -c1-5 | xargs -i kill {} 2>/dev/null
or use a for-loop:
for x in `ps xww | grep "sleep" | cut -c1-5`
do
kill $x 2>/dev/null
To list all files in the `/usr/local' directory tree that are greater than 10,000 kilobytes in size, type:
$ find /usr/local -size +10000k [RET]
To list all files in your home directory tree less than 300 bytes in size, type:
$ find ~ -size -300b [RET]
To list all files on the system whose size is exactly 42 512-byte blocks, type:
$ find / -size 42 [RET]
Use the `-empty' option to find empty files -- files whose size is 0 bytes. This is useful for finding files that you might not need, and can remove.
To find all empty files in your home directory tree, type:
$ find ~ -empty [RET]
Apache important questions
If you specify both deny from all and allow from all, what
will be the default action of Apache?
It also depends on Order directive
order allow, deny # connection will be denied
order deny, allow # connection will be allowed
what is apachectl graceful
Note that you will either need to be running as root or use the "sudo" command in order to run this command.
If Apache is not already running it will be started. If it is already running then it will reload with the new changes but will not abort active connections, meaning that anyone who is in the middle of downloading something will continue to be able to download it.
Before restarting the Apache service a check will be done on the configuration files to ensure they are valid. If there is an error in them the error will be displayed and the Apache service will continue running using the old settings. You need to correct your settings before attempting to restart again.
How do you check for the httpd.conf consistency and any errors in it? -
apachectl configtest
When I do ps -aux, why do I have one copy of httpd running as root and the rest as nouser?
You need to be a root to attach yourself to any Unix port below 1024, and we need 80.
running apache as a root is a security risk? -
No, That one root process opens port 80, but never listens to it, so no user will actually enter the site with root rights. If you kill the root process, you will see the other kids disappear as well.
What is ServerType directive? -
It defines whether Apache should spawn itself as a child process (standalone) or keep everything in a single process (inetd). Keeping it inetd conserves resources. This is deprecated, however.
perl script to check duplicate files
use strict;
use warnings;
use File::Find;
use Digest::MD5;
use Data::Dumper ;
my @dup = fnd_dup(@ARGV) ;
#print Dumper ([@dup]);
foreach my $cur_cup (@dup) {
foreach my $cur_fil (@$cur_cup) {
print "my duplicate file is $cur_fil\n";
}
}
sub fnd_dup (@){
my (@dir_list) = @_;
if ( $#dir_list < 0 ) {
return (undef) ;
}
my %file ;
find( sub { -f && push @{$file{(stat($_))[7]}}, $File::Find::name }, @dir_list );
#print Dumper ([\%file]);
#'2040' => [
# '/opt/dkim-milter-2.8.3/obj.Linux.2.6.18-92.el5.x86_64/libsm/fpurge.o',
# '/opt/dkim-milter-2.8.3/obj.Linux.2.6.18-128.el5.x86_64/libsm/fpurge.o'
# ]
my (@resulth);
my %md ;
foreach my $size ( keys %file ) {
if ( $#{$file{$size}} < 1 ) {
next ;
}
foreach my $curr_file ( @{$file{$size}}) {
open (FILE, $curr_file ) or next ;
binmode(FILE) ;
push @{$md{Digest::MD5->new->addfile(*FILE)->hexdigest}}, $curr_file ;
close(FILE);
}
foreach my $hash (keys %md ) {
if ($#{$md{$hash}} >= 1 ) {
push (@resulth, [@{$md{$hash}}]);
}
}
}
return (@resulth)
Basic Shell scripting
shell scripting
1. How do you find out what's your shell? - echo $SHELL- What's the command to find out today's date? - date
- What's the command to find out users on the system? - who
- How do you find out the current directory you're in? - pwd
- How do you remove a file? - rm
- How do you remove a - rm -rf
- How do you find out your own username? - whoami
- How do you send a mail message to somebody? - mail somebody@techinterviews.com -s 'Your subject' -c 'cc@techinterviews.com'
- How do you count words, lines and characters in a file? - wc
- How do you search for a string inside a given file? - grep string filename
- How do you search for a string inside a directory? - grep string *
- How do you search for a string in a directory with the subdirectories recursed? - grep -r string *
- What are PIDs? - They are process IDs given to processes. A PID can vary from 0 to 65535.
- How do you list currently running process? - ps
- How do you stop a process? - kill pid
- How do you find out about all running processes? - ps -ag
- How do you stop all the processes, except the shell window? - kill 0
- How do you fire a process in the background? - ./process-name &
- How do you refer to the arguments passed to a shell script? - $1, $2 and so on. $0 is your script name.
- What's the conditional statement in shell scripting? - if {condition} then … fi
- How do you do number comparison in shell scripts? - -eq, -ne, -lt, -le, -gt, -ge
- How do you test for file properties in shell scripts? - -s filename tells you if the file is not empty, -f filename tells you whether the argument is a file, and not a directory, -d filename tests if the argument is a directory, and not a file, -w filename tests for writeability, -r filename tests for readability, -x filename tests for executability
- How do you do Boolean logic operators in shell scripting? - ! tests for logical not, -a tests for logical and, and -o tests for logical or.
- How do you find out the number of arguments passed to the shell script? - $#
- What's a way to do multilevel if-else's in shell scripting? - if {condition} then {statement} elif {condition} {statement} fi
- How do you write a for loop in shell? - for {variable name} in {list} do {statement} done
- How do you write a while loop in shell? - while {condition} do {statement} done
- How does a case statement look in shell scripts? - case {variable} in {possible-value-1}) {statement};; {possible-value-2}) {statement};; esac
- How do you read keyboard input in shell scripts? - read {variable-name}
- How do you define a function in a shell script? - function-name() { #some code here return }
- How does getopts command work? - The parameters to your script can be passed as -n 15 -x 20. Inside the script, you can iterate through the getopts array as while getopts n:x option, and the variable $option contains the value of the entered option.
Subscribe to:
Posts (Atom)