python video tutorial

No comments:

50 ebooks on python

No comments:

interview questions

indepth understainding of LVS -- how the webserver knows how to communicated with loadbalancer.
difference between nginx and apache
how replication working in mysql
how to install 100 servers automatically
how to install an application on 100 of servers .

post 8080 is not able to telnet --- hint wait state
content of grub
your website is not working  -- wat are the steps
wat is pingattack
load on the systems what do you check . what tools do you use
how does apache work . -- the flow
wat is rewrite rules -- why is it used
jboss , weblogic , apache-tomcat
what is DOS attack
how do you know dns is not working .




No comments:

Linux guru


* Hardcore System Administrator, with vast experience in Linux Guru.
* Security and Uptime freak.
* A master of regular expressions and nifty one liner in sed, awk. Perl etc
* Ability to understand and solve complex problems, and to make quick decisions
* Ability to manage large server farms including cloud remotely and in a automated fashion.
* Strong customer service and computer skills.
* Able to manage multiple projects & prioritize diverse tasks
* Ability to solve a problem quickly & completely.
* A solid understanding of a LINUX-based operating system; understands paging and swapping, inter-process communication, devices and what device drivers do, file system concepts. Linux internal hacking.
* A solid understanding of networking/distributed computing environment concepts; understands principles of routing, client/server programming, the design of consistent network-wide file system layouts.
* A deep understanding of networking and networking technologies Switches, Routers, IP routing, load balancers, failure modes, diagnostics,...).
* Strong DNS Concepts.
* In-depth Knowledge of Apache web server and related modules like PHP, mod_security, Tomcat, SSL etc
* - Reverse proxies/HTTP accelerators.
* Ability to setup and maintain High Availability Clusters and Load Balanced environment
* In depth knowledge about Load balancing & High Availability.
* Firewalls (Iptables, BSD packet filter, other commercial etc)
* Understanding of Root kits, IDS, Firewall concepts
* SSH, SSL, Cyphers, PGP etc
* In-depth Knowledge of MRTG, Nagios, selinux.
* Knowledge about Linux system performance monitoring & tuning using tools like sysstat, dstat, oprofile, Frysk, SystemTap.
* Knowledge of MySQL, databases & in-depth fine-tuning procedures & advanced troubleshooting.
* Good knowledge of SQL & Databases (relational and non-relational,the ability to think in sets and graphs, performance tuning).
* Knowledge about RAID & LVM.
* Deep understanding of file systems - ext{234}, ntfs, XFS and recovery techniques.
* Knowledge of replicated & network file systems like NFS, DRBD.
* Knowledge of LDAP & Active Directory.
* Good program debugging skills.
* Scalability (Partitioning, Coupling, Caching etc) & capacity planning concepts.

Key Responsibilities:
* Monitoring of the Network, IT infrastructure and various applications to check for any new and outstanding alarms or issues.
* Troubleshooting, Resolving all Technical issues related to our software/networks and the range of Products and Services
* Design, build & implement infrastructures for large scalable web2.0 applications & networked systems.
* Creating high level architectural design of products & systems.
* Set up Monitoring, Configuration management, Administration & Operation practices for such applications/systems.
* Planning large migrations & executing it.
* Developing better monitoring and systems management practices to maximize uptime.
* Develop/Adopt new & efficient strategies to minimize manual/repetitive work, operational costs, deployment times, response times etc.
* Troubleshooting complex systems/network issues.
* Good understanding of statistics.
* Capacity planning
* Training & managing teams.
* Plan team todo's/goals, product upgrades/releases.
* Documenting processes, troubleshooting information, problems and their resolutions.
* Evaluate software/hardware products (free/commercial) as per needs.
* Prepare & present proposals with respect to systems/software infrastructure.
* Research & build specifications for new products & tools
* Acting as liaisons between junior system administrators, application administrators and software developers
* Converting software package requirements into configuration input for the configuration management system.
* High level log analysis, performance reports, trend analysis.
* Quick in decision making and execution.
* Assist operations teams in daily operations when required
* Assist developers in architectural and design problems.
No comments:

LVM understanding




--------------------------------------------------------------------------------------------------------------
we use fdisk to format the raw hard disk 

fdisk /dev/sdb

 /dev/sdb1               1          14      112423+  83  Linux
/dev/sdb2              15          28      112455   83  Linux
/dev/sdb3              29          42      112455   83  Linux
partprobe
-------------------------------------------------------------------------------------------------------------
now we create a physical volume
pvcreate /dev/sdb1 /dev/sdb2 /dev/sdb3 
pvdisplay
-------------------------------------------------------------------------------------------------------------

 We create the volume group vg1

 vgcreate vg1 /dev/sdb1 /dev/sdb2 
vgdisplay

 ------------------------------------------------

 We now create the logical volume

lvcreate -L 200M vg1 -n lv1
lvdisplay 
mkfs.ext4 /dev/vg1/lv1
mount    /dev/vg1/lv1   /mnt/
-------------------------------------------------------------------------------------------------------------
to extend new hdd :
pvcreate  /dev/sdb1
vgextend vg1 /dev/sdb1
lvresize -L +100M /dev/vg1/lv1
resize2fs /dev/vg1/lv1
lvdisplay /dev/vg1/lv1

 -------------------------------------------------------------------------------------------------------------
the following procedure, in this example we shrink a partition from 10G to 9G:
First, we unmount.
# umount /dev/vg_blah/lv_blah
Check the file system.
# e2fsck -f /dev/vg_blah/lv_blah
Resize the file system.
# resize2fs -p /dev/vg_blah/lv_blah 9G
Reduce the size of the logical volume.
# lvreduce -L -1G /dev/vg_blah/lv_blah
Shrink the volume group if desired.
# vgreduce vg_blah /dev/sdxy


No comments:

pfsense useful links

No comments:

linux documentation

No comments:

the success indicator



No comments:

set up of selenium server running in x virtual framebuffer

yum install  xorg-x11-server-Xvfb
yum install xorg-x11-fonts*
yum install libXfont

run in background

 DISPLAY=:1 xvfb-run java -jar  /usr/local/bin/selenium-server-standalone-2.31.0.jar  &

# netstat -tupln |grep :::4444
tcp        0      0 :::4444                     :::*                        LISTEN      17369/java  



No comments:

jenkins -- bigbucket - git - apache-tomcat -- sonar intergration doc





step 1


Installation of jenkins :  ( with out  apache tomcat , stand alone instance )
( this steps are done on test machine )


yum install java-1.6.0-openjdk
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
Sudo yum install jenkins





All config details :
/etc/sysconfig/jenkins
/var/lib/jenkins/



Installation of jenkins :  ( with   apache tomcat. Tomcat will inturn run the jeninks.war file  )


( these steps are done on Actual machine ) 



    Step 1: Install the JDK\




[root@station1 Downloads]# rpm -ivh jdk-7u7-linux-x64.rpm
[root@station1 Downloads]# JAVA_HOME=/usr/java/jdk1.7.0_
07
[root@station1 Downloads]# export JAVA_HOME
[root@station1 Downloads]# PATH=$JAVA_HOME/bin:$PATH
[root@station1 Downloads]# export PATH
[root@station1 Downloads]# vim /root/.bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
       . ~/.bashrc
fi
# User specific environment and startup programs
JAVA_HOME=/usr/java/jdk1.7.0_07/
export JAVA_HOME
PATH=$JAVA_HOME/bin:$PATH
export PATH
#PATH=$PATH:$HOME/bin
#export PATH
Restart or logout and login


[root@station1 Downloads]# echo $JAVA_HOME
/usr/java/jdk1.7.0_07/



      Step 2: Download and Unpack apache-tomcat-7.0.30 (or latest)


To download latest  apache-tomcat

[root@station1 Downloads]# tar -xzvf apache-tomcat-7.0.30.tar.gz
[root@station1 Downloads]# mv apache-tomcat-7.0.30 /usr/share/
[root@station1 Downloads]# vim /etc/init.d/tomcat
   #!/bin/bash
    # description: Tomcat Start Stop Restart
    # processname: tomcat
   # chkconfig: 234 20 80
    JAVA_HOME=/usr/java/jdk1.7.0_07
    export JAVA_HOME
    PATH=$JAVA_HOME/bin:$PATH
    export PATH
    CATALINA_HOME=/usr/share/apache-tomcat-7.0.30
    case $1 in
    start)
    sh $CATALINA_HOME/bin/startup.sh
    ;;
    stop)
    sh $CATALINA_HOME/bin/shutdown.sh
    ;;
    restart)
    sh $CATALINA_HOME/bin/shutdown.sh
    sh $CATALINA_HOME/bin/startup.sh
    ;;
    esac
    exit 0
[root@station1 ~]# chmod 755 /etc/init.d/tomcat
[root@station1 ~]# chkconfig --add tomcat
[root@station1 ~]# chkconfig --level 234 tomcat on
[root@station1 ~]# chkconfig --list tomcat
tomcat             0:off    1:off    2:on    3:on    4:on    5:off    6:off
[root@station1 ~]# service tomcat stop
Using CATALINA_BASE:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_HOME:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_TMPDIR: /usr/share/apache-tomcat-7.0.30/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_07
Using CLASSPATH:       /usr/share/apache-tomcat-7.0.30/bin/bootstrap.jar:/usr/share/apache-tomcat-7.0.30/bin/tomcat-juli.jar
[root@station1 ~]# service tomcat start
Using CATALINA_BASE:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_HOME:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_TMPDIR: /usr/share/apache-tomcat-7.0.30/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_07
Using CLASSPATH:       /usr/share/apache-tomcat-7.0.30/bin/bootstrap.jar:/usr/share/apache-tomcat-7.0.30/bin/tomcat-juli.jar
[root@station1 ~]# service tomcat restart
Using CATALINA_BASE:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_HOME:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_TMPDIR: /usr/share/apache-tomcat-7.0.30/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_07
Using CLASSPATH:       /usr/share/apache-tomcat-7.0.30/bin/bootstrap.jar:/usr/share/apache-tomcat-7.0.30/bin/tomcat-juli.jar
Using CATALINA_BASE:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_HOME:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_TMPDIR: /usr/share/apache-tomcat-7.0.30/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_07
Using CLASSPATH:       /usr/share/apache-tomcat-7.0.30/bin/bootstrap.jar:/usr/share/apache-tomcat-7.0.30/bin/tomcat-juli.jar


For logs


[root@station1 ~]# less /usr/share/apache-tomcat-7.0.30/logs/catalina.out
We can now access the Tomcat Manager page at
[root@station1 ~]# firefox http://station1.com:8080


copy the jenkins.war file to the tomcats


Jenkins home dir will be /root/.jenkins


We can access jenkins






step 2


-- Configure jenkins for git -- need to add plugins from --> manage plugins


-- create new jobs for each product  ( eg:somedomain.com )



step 3


Create Bit Bucket accounts


-- Add accounts in bit bucket at product level eg: somedomain1, somedomain2, etc
       


-- Separate email ids need to be created for each product eg: project1@domain.com,porject2@domain.com
    -- cretate product level repo :
    -- Upto 5 users per account can be added at no cost as per bit bucket pricing terms
       -- emails will be sent to each team members


   


-- install git in  local machine , follow inrustions "start from stratch "  ( this could be done a git-gui client )
     -- updated .git/config file
      [user]
            name = agnello
            email = agnello@domain.com


     -- steps to be follwoed 1st time on local machine  by TL
           mkdir /path/to/your/project
        cd /path/to/your/project
        git init
           git remote add origin https://<<the https URL from BB>>



     -- step to be done on Developer machines
         -- install git-core
         -- access BB URL from the email he would recv .
         -- Once he logs into BB -- systems team will auth him to grant access to repo.
         -- on the linux/ubuntu  Konsole he creats a clone repo
Cloning into cs442-spring12-jdoe...
Username: jdoe
Password: *********
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
$ ls
cs442-spring12-jdoe/


Adding files and commiting to the local repository
$ cd cs442-spring12-jdoe
$ git status
# On branch master
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       MyNewProject/
nothing added to commit but untracked files present (use "git add" to track)
$ git add MyNewProject


$ git config user.name "John Doe"
$ git config user.email "jdoe@hawk.iit.edu"

$ git commit -m "Initial commit for MyNewProject"
[master 453a4f8] Initial commit for MyNewProject
7 files changed, 416 insertions(+), 0 deletions(-)
create mode 100644 MyNewProject/MyNewProject.xcodeproj/project.pbxproj
create mode 100644 MyNewProject/MyNewProject/AppDelegate.h
create mode 100644 MyNewProject/MyNewProject/AppDelegate.m
create mode 100644 MyNewProject/MyNewProject/MyNewProject-Info.plist
create mode 100644 MyNewProject/MyNewProject/MyNewProject-Prefix.pch
create mode 100644 MyNewProject/MyNewProject/en.lproj/InfoPlist.strings
create mode 100644 MyNewProject/MyNewProject/main.m
$ git log
commit 048c71c22f1d38b1a4c8cb73be55ce351fea641b
Author: John Doe <jdoe@hawk.iit.edu>
Date:   Wed Feb 15 22:01:34 2012 -0600
    Initial commit for MyNewProject


commit 3caa74862b8f17f356841bfa2d380a817f371ffe
Author: Michael Lee <lee@soi2.org>
Date:   Thu Jan 12 16:48:59 2012 -0600
    Adding initial README


commit 12cb858e8ecb869d9dc1489283741c9897a3b7f9
Author: Michael Lee <lee@soi2.org>
Date:   Thu Jan 12 16:37:00 2012 -0600
    Adding Xcode gitignore






step 4


-- On Jenkins server , Login as Jenkins, run ss-keygen
-- copy the  .ssh/id_rsa.pub to bit bucket ( with account project1@domain.com ) this is done so that jenkins can connet to BB with our password
-- Add at the New Job level  --> configuration -- > source code managt --> <<  ssh URl from BB >>



step 5


-- In "Manage Jenkins" --> E-mail Notification -->


smtp server  -- smtp.gmail.com
Default user e-mail suffix  --  username@domain.com
-- check "Use SMTP Authentication"
user name  -- jenkins@domain.com
password --- !jenkins@345
use ssl --
smtp port -- 465
Reply-To Address  -- noreply@domain.com




Step 6


installing ant
yum install ant


installing maven


cd /tmp
tar zxvf apache-maven-3.0.5-bin.tar.gz
mv apache-maven-3.0.5 /usr/local/
vi /etc/profile.d/maven.sh
export PATH=/usr/local/apache-maven-
3.0.5/bin:$PATH










installing plugins:  installed the following jenkins plugin


phing
checkstyle
cloverphp
dry
htmlpublisher
jdepend
plot
pmd
violations
xunit



installing all the pear modules .


pecl install xdebug
pear channel-discover pear.pdepend.org
pear install pdepend/PHP_Depend
pear channel-discover pear.phpmd.org
pear install phpmd/PHP_PMD
pear channel-discover components.ez.no
pear channel-discover pear.symfony-project.com
pear install phpunit/phploc
pear install PHPDocumentor
pear install PHP_CodeSniffer
pear install phpunit/PHP_CodeBrowser
pear install phpunit/PHPUnit
pear install phpunit/ppw
pear install Zend_Framework
pear channel-discover pear.zfcampus.org
pear install zfcampus/ZF
pear channel-discover pear.phing.info
pear install phing/phing



step 7


Update On Jenkins


Understanding of git - branches


there are two branches " master " and " development "



-------------------------------------------------------------
These step are used to check into master 1st time


git init
git remote add origin ssh://git@bitbucket.org/agnellodsouza/somedomain1.git
git add .
git commit -m "First commit"
git push -u origin master


--------------------------------------------------------------


On the developers desktop


git clone -b development ssh://git@bitbucket.org/agnellodsouza/somedomain1.git
git branch development
git checkout development
<<< developer makes some changes to the file README >>>
git push -u origin development
git add .
git commit -m "made changes to README "
git push -u origin development



<< again developer pulls from developement branch >>
git branch
git pull origin development



Once developer checks into the Developer branch jenkins will check if any commits have been made ever 5 min . If it see change in commit it will start the build


-- poll from development branch
-- invoke ant -- using the build.xml
-- run various test  ( currently no php unit test for someproject.com)
-- if build successful
-- Using the Git Publisher plugin , will commit to the master branch and create a separate  TAG branch.





Here are few install plugins in for Jenkins :




------------------------------------------------------------------------------------------------


The content of build.xml file that ant need to invoke


<?xml version="1.0" encoding="UTF-8"?>


<project name="Sample project" default="build">
<target name="build"    depends="prepare,lint,perltestcases,phploc,pdepend,phpmd-ci,phpcs-ci,phpcpd,phpcb,phpdoc"/>


<target name="build-parallel"
  depends="prepare,lint,tools-parallel,phpcb"/>


<target name="tools-parallel"
        description="Run tools in parallel">
 <parallel threadCount="2">
  <sequential>
    <antcall target="pdepend"/>
    <antcall target="phpmd-ci"/>
  </sequential>
  <antcall target="phpcpd"/>
  <antcall target="phpcs-ci"/>
  <antcall target="phploc"/>
 </parallel>
</target>


<target name="clean" description="Cleanup build artifacts">
 <delete dir="${basedir}/build/api"/>
 <delete dir="${basedir}/build/code-browser"/>
 <delete dir="${basedir}/build/coverage"/>
 <delete dir="${basedir}/build/logs"/>
 <delete dir="${basedir}/build/pdepend"/>
 <delete dir="${basedir}/build/phpdox"/>
 <delete dir="${basedir}/build/code-browser"/>
</target>


<target name="prepare" depends="clean"
        description="Prepare for build">
 <mkdir dir="${basedir}/build/api"/>
 <mkdir dir="${basedir}/build/code-browser"/>
 <mkdir dir="${basedir}/build/coverage"/>
 <mkdir dir="${basedir}/build/logs"/>
 <mkdir dir="${basedir}/build/pdepend"/>
 <mkdir dir="${basedir}/build/phpdox"/>
 <mkdir dir="${basedir}/build/code-browser"/>
</target>




<target name="lint">
 <apply executable="php" failonerror="true">
  <arg value="-l" />


  <fileset dir="${basedir}/src">
    <include name="**/*.php" />
    <modified />
  </fileset>


 </apply>
</target>



<target name="perltestcases">
<property name="perl" location="/usr/bin/perl"/>
<property name="wrapper" location="${basedir}/wrapper2.pl"/>
<exec executable="${perl}" spawn="false">
<arg value="${wrapper}"/>
<arg value="${jobid}"/>
</exec>
</target>


<target name="phploc" description="phploc can be used to track project size metrics, such as Lines of Code (LOC)">
 <exec executable="phploc">
  <arg value="--log-csv" />
  <arg value="${basedir}/build/logs/phploc.csv" />
  <arg path="${basedir}/src" />
 </exec>
</target>


<target name="pdepend"
        description="Calculate software metrics using PHP_Depend">
 <exec executable="pdepend">
  <arg value="--jdepend-xml=${basedir}/build/logs/jdepend.xml" />
  <arg value="--jdepend-chart=${basedir}/build/pdepend/dependencies.svg" />
  <arg value="--overview-pyramid=${basedir}/build/pdepend/overview-pyramid.svg" />
  <arg path="${basedir}/src" />
 </exec>
</target>


<target name="phpmd"
        description="Perform project mess detection using PHPMD">
 <exec executable="phpmd">
  <arg path="${basedir}/src" />
 <mkdir dir="${basedir}/build/phpdox"/>
 <mkdir dir="${basedir}/build/phpdox"/>
  <arg value="text" />
  <arg value="${basedir}/build/phpmd.xml" />
 </exec>
</target>


<target name="phpmd-ci"
        description="Perform project mess detection using PHPMD">
 <exec executable="phpmd">
  <arg path="${basedir}/src" />
  <arg value="xml" />
  <arg value="${basedir}/build/phpmd.xml" />
  <arg value="--reportfile" />
  <arg value="${basedir}/build/logs/pmd.xml" />
 </exec>
</target>


<target name="phpcs"
        description="Find coding standard violations using PHP_CodeSniffer">
 <exec executable="phpcs">
  <arg value="--standard=${basedir}/build/phpcs.xml" />
  <arg path="${basedir}/src" />
 </exec>
</target>


<target name="phpcs-ci"
        description="Find coding standard violations using PHP_CodeSniffer">
 <exec executable="phpcs" output="/dev/null">
  <arg value="--report=checkstyle" />
  <arg value="--report-file=${basedir}/build/logs/checkstyle.xml" />
  <arg value="--standard=${basedir}/build/phpcs.xml" />
  <arg path="${basedir}/src" />
 </exec>
</target>


<target name="phpcpd" description="Find duplicate code using PHPCPD">
 <exec executable="phpcpd">
  <arg value="--log-pmd" />
  <arg value="${basedir}/build/logs/pmd-cpd.xml" />
  <arg path="${basedir}/src" />
 </exec>
</target>


<target name="phpcb"
        description="Aggregate tool output with PHP_CodeBrowser">
 <exec executable="phpcb">
  <arg value="--log" />
  <arg path="${basedir}/build/logs" />
  <arg value="--source" />
  <arg path="${basedir}/src" />
  <arg value="--output" />
  <arg path="${basedir}/build/code-browser" />
 </exec>
</target>


<target name="phpdoc"
       description="Generate API documentation using PHPDocumentor">
  <exec executable="phpdoc">
  <arg value="--directory" />
  <arg path="${basedir}/src" />
  <arg value="--target" />
  <arg path="${basedir}/build/api" />
  </exec>
</target>


</project>


--------------------------------------------------------------------------------






Sonar intergration with jenkins using maven


sonar + maven + jenkins


we leave the sonar conf file as default
/usr/local/bin/sonar-3.5.1/conf/sonar.properties


setting for maven
/usr/local/bin/apache-maven-3.0.5/conf/settings.xml


<profile>
<id>sonar-profile</id>
<properties>
<sonar.language>php</sonar.language>
</properties>
</profile>


<activeProfiles>
    <activeProfile>sonar-profile</activeProfile>
</activeProfiles>





setting the pom.xml for maven ( this is to be set a individual project level )



<project>
<modelVersion>4.0.0</modelVersion>
<groupId>arsapi</groupId>
<artifactId>arsapi</artifactId>
<name>ars</name>
<version>1.0</version>
<packaging>pom</packaging>
<build>
<directory>build</directory>
<sourceDirectory>src</sourceDirectory>
<testSourceDirectory>test</testSourceDirectory>
</build>


<properties>
<sonar.language>php</sonar.language>
<sonar.dynamicAnalysis>true</sonar.dynamicAnalysis>
<sonar.phpDepend.shouldRun>true</sonar.phpDepend.shouldRun>
<sonar.phpDepend.exclude>vendor/, app/, bin/, web/, src/*/Bundle/*Bundle/Tests</sonar.phpDepend.exclude>
<sonar.phpPmd.shouldRun>true</sonar.phpPmd.shouldRun>
<sonar.phpPmd.analyzeOnly>false</sonar.phpPmd.analyzeOnly>
<sonar.phpCodesniffer.shouldRun>true</sonar.phpCodesniffer.shouldRun>
<sonar.phpcpd.shouldRun>true</sonar.phpcpd.shouldRun>


<sonar.phpUnit.coverage.skip>true</sonar.phpUnit.coverage.skip>
<sonar.phpUnit.skip>true</sonar.phpUnit.skip>


</properties>


</project>



in Jenkins go to the manage configuration and set the maven path .




admin / password



Here are few more imp links :

































very imp docs













No comments:
Subscribe to: Posts (Atom)

Other Articles

Enter your email address: