Thursday, August 25, 2016

DNSMASQ as internal DNS server with a PHP mangement page (GUI)


  • Apache as the web server. (In my server apache set to run in port 9000)
          PHP and SQLite DB files located at "/var/www/html/dnsmasq" and the same path set as the root directory.

         service accessed via http://server1/dnsmasq/login.php
  • login.php
  • admin.php - add/remove/modify host entries and update the DB
  • admin.db - user logins
  • db.php - DB connections
  • hosts.db - host entry DB
  • - updates the local hosts file when invoked from the PHP file
  • dnsmasq.conf - replace dnsmasq initial file with this
* make sure to grant dnsmasq service reload access to apache user in visudo
   # visudo
   -- add below entry
    apache  ALL = NOPASSWD: /etc/init.d/dnsmasq reload

* current user in admin.db is "admin", password is "password"
   to change the login or to add a user use below sql commands;
   change admin user password ---  sqlite3 admin.db "update logins set password='123456' where username='admin'"

   add a user --- sqlite3 admin.db "insert into logins (username,password) values ('admin','password')"

Thursday, February 25, 2016

Configure HornetQ in JBoss EAP 6 to work with NAT (Eucalyptus Cloud network environment)

You need to apply below changes in "server\default\deploy\hornetq\hornetq-configuration.xml" file.

      <connector name="netty">
         <param key="host"  value=" – this is the NATTED IP"/>
         <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
      <connector name="netty-throughput">
         <param key="host"  value=" – this is the NATTED IP "/>
         <param key="port"  value="${hornetq.remoting.netty.batch.port:5455}"/>
         <param key="batch-delay" value="50"/>

      <connector name="in-vm">
         <param key="server-id" value="${hornetq.server-id:0}"/>


      <acceptor name="netty">
         <param key="host"  value=" – this is the server private IP"/>
         <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
      <acceptor name="netty-throughput">
         <param key="host"  value=" – this is the server private IP "/>
         <param key="port"  value="${hornetq.remoting.netty.batch.port:5455}"/>
         <param key="batch-delay" value="50"/>
         <param key="direct-deliver" value="false"/>

      <acceptor name="in-vm">
        <param key="server-id" value="0"/>


Change the "run.conf" file as below;

-Djboss.bind.address=  -Djava.rmi.server.hostname=

Monday, November 16, 2015

Few important Windows Resource Kit tools

Following are some tools from the Windows Resource Kit, which I think are very important.

You can download the Windows Resource Kit from here.

These will work at any location of the command prompt when copied to "c:\windows\system32"
  • consume
          From this you can run a simple stress test on your PC's memory, CPU and disks.

          This is similar to the Linux "stress" command.

          CAUTION !!! use with care if you are going run this in a production server.

          C:\Users\lahiru>consume /?
          Universal Resource Consumer - Just an innocent stress program, v 0.1.0
          Copyright (c) 1998, 1999, Microsoft Corporation

          consume RESOURCE [-time SECONDS]

          RESOURCE can be one of the following:


  • createfil
          Use this command if you need to create a big dummy file, which is equal to Linux "dd" command.

          C:\Users\lahiru>creatfil /?

          Usage: creatfil FileName [FileSize]

          -? :  This message
          -FileName -- name of the new file
          -FileSize -- size of file in KBytes, default is 1024 KBytes

  • tail
        Simple tail command which is similar to Linux "tail" command.

          C:\Users\lahiru>tail /?

          usage: TAIL [switches] [filename]*
          switches: [-?] display this message
          [-n] display last n lines of each file (default 10)
          [-f filename] keep checking filename for new lines 

  • robocopy
        This is more like Linux "rsync'.

          Refer "robocopy /?" or "robocopy.doc" for command usage/parameters.

  • qgrep
          This command is useful if you need to find a string from set of files.

  • PortQry 
        portqry.exe does not come in resource kit. You can download it from here.
        portqry is useful if you want to test a port status and use it in a batch file. Because telnet can't be used to get port status to a script.


Thursday, November 5, 2015

LVS DR mode - Loopback interface creation in real servers


Linux servers

           -- create a loopback adaptor with the relevant VIP

        vi /etc/sysconfig/network-scripts/ifcfg-lo:vip1
        enter relevant values as below;


          -- disable ARP responses on loopback

        vi /etc/sysctl.conf
        append below lines to the file

        net.ipv4.conf.lo.arp_ignore = 1
        net.ipv4.conf.lo.arp_announce = 2
        net.ipv4.conf.all.arp_ignore = 1
        net.ipv4.conf.all.arp_announce = 2

         -- update sysctl values

        sysctl –p

         -- start the loopback adaptor

        ifup lo:vip1

    Windows servers

            -- add a new loopback adaptor via add remove hardware wizard

         -- first disable the newly created adapter and configure the VIP with the
            netmask temporary. We need to change the netmask to
   which can’t be done via interface properties. We need 
            to  update the registry for this.



            -- configure the relevant VIP on loopback interface

                   Now before enabling the loopback device, go to registry and modify
                   the subnet mask.


                     Change the highlighted SubnetMask value to
                     Close the registry and enable the loopback interface.


Wednesday, November 4, 2015

Tomcat + Redis high availability with Keepalived

Physical setup view :

Tomcat + Redis high availability with Keepalived

For above setup we will use IPs as below;

Server 1 :
Server 2 :
VIP        :

  • Nginx stays at the top and it load balances the connections to tomcats using private IPs (,
  • VIP configured using Keepalived service and it runs in both servers.
  • Tomcat runs in both servers. In Tomcat, Redis host IP configured to be the VIP (
Initially, VIP will be owned by the server which starts first or the server where keepalived service starts first. We will call this the master server.

  1. When the master goes down, VIP will be taken over by the secondary server. When this happens, we use keepalived's notify script to make the redis as master and the failed server's redis will become the slave.
  2. If Nginx service or the Redis service goes down, then also the VIP will be moved to the secondary server. For this, we use keepalived's tracking script to float the VIP based on service status.
  3. Redis service init script also changed to check the VIP status at startup. It will check whether the specific server owns the VIP. If it has the VIP, Redis will change it's role to master.

Configuration of each service as a summary

Tomcat session replication with redis :

  • To enable tomcat-redis session replication, you need to add redis session manager jar files to tomcat’s lib directory.

Note : copy “tomcat-redis-session-manager-1.1.jar” if java6 is used. Copy “tomcat-redis-session-manager-1.2-tomcat-7-java-7.jar” if java7 is used. Other two libs are same for both versions.

make sure to add commons-pool-1.6.jar and jedis-2.0.0.jar also to the lib directory.

  • Edit “/$CATALINA_HOME/conf/context.xml” and add following under context

<Valve className="com.radiadesign.catalina.session.RedisSessionHandlerValve" />
<Manager className="com.radiadesign.catalina.session.RedisSessionManager"
 maxInactiveInterval="60" />

* make sure redis listens on the server IP instead of lookback IP.

Sample nginx upstream configuration :

upstream tomcat {
        check interval=3000 rise=2 fall=5 timeout=3000;
        keepalive 16;
* Above will monitor the service port only. You can use an URI also to check the service status.

Keepalived configuration :

vrrp_script chk_myscript {
  script       "/etc/keepalived/scripts/"
  interval 5
  fall 2
  rise 2

vrrp_instance vip_testweb {
    state BACKUP
    interface eth0
    virtual_router_id 61
    priority 101
    advert_int 1
    notify "/etc/keepalived/scripts/"

        track_script {

        virtual_ipaddress {

-------------------------------------------------------------------------------------------------------------------------- -- this would trigger the VIP float based on nginx and redis service status.

#### This script will check the redis by adding a key with current time as value. And will check the nginx service status also.
#### If both are running prpoerly script should return "0" as the result code.

ngxstat=`/usr/bin/pgrep nginx | wc -l`
time=`date +%T`
redstat=`/usr/bin/redis-cli set health_check $time`

cmdstat=`echo $?`
if [ $cmdstat -ne 0 ];

echo  redstat $redstat
echo ngxstat $ngxstat

if [ $redstat == "OK" ]  && [ $ngxstat != 0 ];
echo services OK
exit 0
echo services ERROR
exit 123

------------------------------------------------------------------------------------------------------------------------------------------------------ -- this will trigger the script with the keepalived status (Master/Backup/Fault) as parameter. Based on keepalived VRRP instance status, redis role will be changed. In the script put the proper peer IP and port. 



case $STATE in
        "MASTER") /usr/bin/redis-cli slaveof no one
                  exit 0
        "BACKUP") /usr/bin/redis-cli slaveof $PEER_IP $PEER_PORT
                  exit 0
        "FAULT") /usr/bin/redis-cli slaveof no one
                  exit 0
        *)       echo "unknown state"
                 exit 1


Redis init script modification :

Add below lines to redis init script. This way, when redis starts it will decide to become master or slave. In the script put the proper peer IP and port.


vipcheck() {

       vip=`ip add sh | grep $VIP | wc -l`
        if [ $vip -eq 1 ]; then
        echo $"VRRP is MASTER"
        /usr/bin/redis-cli slaveof no one
        echo $"VRRP is BACKUP"
        /usr/bin/redis-cli slaveof $PEER_IP $PEER_PORT

  • Call the vipcheck() function at start()

start() {
    [ -x $redis ] || exit 5
    [ -f $REDIS_CONF_FILE ] || exit 6
    echo -n $"Starting $prog: "
    daemon $redis $REDIS_CONF_FILE
    [ $retval -eq 0 ] && touch $lockfile
    return $retval


After applying above configuration in both servers, you can test the failover by stopping keepalived service or nginx service or even redis service.

 All keepalived events will be logged at "/var/log/messages"

VIP can be seen by typing the command, "ip add sh"

Even though configurations and steps are not in detail, I hope this would help you to understand how I have achieved redis level high availability.