Abstract:

The problem of managing multiple servers that do basically similar stuff and are used to host in house applications. Avoiding configuration time hogs, and automation of server configuration allowing fast deployments with no scaling limit.

The problem:

The problem with MultiAll is the fact of having all the servers on sync when they are in different “Farms or Grids” and the fact that one server can provide services to more than one porpuse. This is why we invent the “Pool system” where all the servers respond or act for a defined pool.

If a server responds to multiple pools the aggregation of every pool is the server.

Understanding how we can make it work:

Under our research its possible to merge more than one server to work for a common goal, doing some modifications to the normal database driven apps to have a cache table, and when that server is unable to communicate to others under the same pool they save the changes they have to make to their brother servers and later on when they are back online deploy the changes.

On the file system side, every server is responsible of keeping others from the same pool up to date, bi partitioning the problem as every server is responsible for others and their self.  We’re doing similar to how Rsync works, and we would only push differences between servers under a secure connection IE: (VPN).

By this moment we would have a DB Layer abstraction for multiple servers responding to a DB and file system.

Next is the Domain Name Server problem, that we don’t want clients reaching servers that are offline due to maintenance or problems, for this we’re developing on BIND 9 an abstraction layer that every server in a pool must be “Network Aware” of others of his kind, and if hes unable to reach others, then he must change the DNS registry to reflect the changes.

How MultiAll solves the problem:

Basically MultiAll will work as a service provider inside the server, as a global abstraction layer, and once the application is provided with a bridge or connection layer to it, it would be able to take advantage of the system. MultiAll has as key features:

1. Topology-Aware Neighboring System (TNS): provides a topology discovery service for the servers (or “nodes”), which may eventually “recalculate” the topology in case of unexpected downtime (such as network, power or hardware failures, among others) or in cases of planned downtime (such as server maintenance, hardware upgrades or migrations). The different synchronization agents of the “Multi All” depends on information learned through TNS, therefore it is considered a critical component of the “Multi All” system.
2. System Baseline Monitor (SBM, formerly known as “checker”): provides a 24/7 server health monitoring, locally on each server (or “node”) using “Baseline Rules”. These rules are defined on a “per pool” or “per server” basis, allowing the configuration to be as granular as needed, making exceptions if the need arises. The health status is published via TNS using standardized codes known as “SBM Statuses”. It is also considered a critical component of the “Multi All” system.
3. File System Synchronization Agent (FS-SA): lets you define structures inside your filesystem to keep synchronized across a pool of servers. FS-SA, used in combination with the rest of the “Multi All’s” Synchronization Agents (SA’s) provides you with high availability, data redundancy and server load balancing on your pool of servers.
4. Software and Libraries Synchronization Agent (SL-SA): especially useful for large-scale unattended deployments, the SL-SA keeps all your software packages, services and libraries consistent across the pool, raising awareness to the system administrators when possible conflicts, incompatibilities or other issues arise.
5. Database Synchronization Agent (DB-SA): keeps the different database servers of the pool synchronized. Depending on the needs of the underlying applications, the DB-SA may either work 24/7 to keep the DBs in perfect sync, or you could define your own database synchronization policies.
6. Domain Name System Synchronization Agent (DNS-SA): keeps the DNS zones up to date with the pool’s topology either via a pull-push synchronization mechanism (handled by the FS-SA) or by rebuilding the DNS zone according to the topology discovered by the TNS.

The project has gotten a bit more ambitious… so here is how its changed!

Okay so some research has been going on, and the project has been growing quite a bit. Among the changes that have been happening around, it has gotten bigger, now all the things stated before, are part of a much larger system now.
TakeConnector will now be the daemon that will keep our infrastructure.

Overview: After having a high temperature in one of our servers and not being able to notice it as a ‘heat wave’ came in… I thought about adding to our monitoring stack a simple script to monitor the temperature and notify us in case something goes wrong.

Basically, the script will run SENSORS and parse the output from it to let us know if something goes wrong, and leave a simple logline to check the average temperatures.

So packages you’ll need : sensors.

To install install sensors and be able to run this script:

1. Install the lm-sensors package.
2. Run sudo sensors-detect and choose YES to all YES/no questions.
3. At the end of sensors-detect, a list of modules that needs to be loaded will displayed. Type “yes” to have sensors-detect insert those modules into /etc/modules, or edit /etc/modules yourself.
4. Next, run sudo service module-init-tools restart This will read the changes you made to /etc/modules in step 3, and insert the new modules into the kernel.
5. Copy this script to ~ (root folder), add permissions to execute chmod u+x (scriptname).sh
6. crontab -e add the line “@reboot /root/(scriptname).sh

#!/bin/bash
######################################################################################
# Derek Demuro, this script is given as is, CopyLEFT                                 #
######################################################################################
######################################################################################
# README                              LEAME                                          #
######################################################################################
# This script will run and check the temperature, in case of being high, MAIL!
#
############################To Set Up#################################################
#
# To set this script up, you'll need to add it to a cronjob to run on boot
# Most linux distros will allow a param @reboot /(path)/servermon.sh
# So... crontab -e
# At the bottom add: @reboot /(path)/servermon.sh
# REMEMBER TO ADD EXECUTABLE BIT TO THE FILE (777 Permissions)
######################################################################################
# SCRIPT CONFIGURATION                                                               #
######################################################################################
### Where should the log be saved?
readonly LOGNAME='servertemp.log'
### Who should we mail on error
readonly MAILTO='<a href="mailto:mail@derekdemuro.me" class="mailto">mail@derekdemuro.me<span class="mailto"><span class="element-invisible"> (link sends e-mail)</span></span></a>'
### How much time between check's?
readonly SLEEPTIME=30
###Alert if temp Above
readonly MAXTEMP=75
###ServName
readonly SERVNAME='UYMF1DEB'
##How long to sleep after message sent
readonly SLEEPERROR=216000
### How many records to keep
readonly CLEARLOGTMS=1000
######################################################################################
#################################FUNCTIONS START################################
 
###Function to clear the log
function clearLog() {
  echo 'Log cleared' > $LOGNAME
  echo 'Script will run ' $1 ' times then will clear itself'>> $LOGNAME
  return 0
}
 
#################################FUNCTIONS FINISH################################
#################################MAIN SCRIPT FUNC################################
while [ TRUE ]; do
    #add 1 to times
    times=`expr $times + 1`
    ##CLEAR THE LOG
    if [ $times -eq $CLEARLOGTMS ]; then
        clearlog
        $times=0
    fi
    ##Run script for every line in serverList
    currentTemp=`sensors|grep Core|awk '{print $3}'|cut -b2,3,3|tail -1`
    if [ $currentTemp -gt  $MAXTEMP ]; then
        echo -e "\e[00;31m`date +'%Y-%m-%d %H:%M:%S'` ALERT: Current temperature: $currentTemp, at server: $SERVNAME \e[39m"
        mail -s "ALERT: Temperature above umbral" $MAILTO -a "Reply-To: " <<< "`date +'%Y-%m-%d %H:%M:%S'` ALERT: Current temperature: $currentTemp, at server: $SERVNAME"
        sleep $SLEEPERROR
    else
        echo -e "\033[38;5;148m `date +'%Y-%m-%d %H:%M:%S'` All good: $currentTemp at server: $SERVNAME \033[39m"
    fi
  ##We sleep till new run
  sleep $SLEEPTIME
done

So there you have the script, now if you want a list of DNS’s to check your domains against….

8.8.8.8
8.8.4.4
156.154.70.1
156.154.71.1
208.67.222.222
208.67.220.220
198.153.192.1
198.153.194.1
4.2.2.1
4.2.2.2
4.2.2.3
4.2.2.4
4.2.2.5
4.2.2.6
67.138.54.100
207.225.209.66
85.88.19.10
85.88.19.11
87.118.100.175
94.75.228.28
62.141.58.13
85.25.251.254
85.214.73.63
212.82.225.7
212.82.226.212
213.73.91.35
58.6.115.42
58.6.115.43
119.31.230.42
200.252.98.162
217.79.186.148
82.229.244.191
216.87.84.211
66.244.95.20
204.152.184.76
194.150.168.168
80.237.196.2
194.95.202.198
88.198.130.211
78.46.89.147
129.206.100.126
79.99.234.56
208.67.220.220
208.67.222.222
156.154.70.22
156.154.71.22
85.25.149.144
87.106.37.196
8.8.8.8
8.8.4.4
88.198.24.111
58.6.115.42
202.83.95.227
119.31.230.42
217.79.186.148
178.63.26.173
178.63.26.174
27.110.120.30
89.16.173.11
210.80.60.1
210.80.60.2
199.166.24.253
199.166.27.253
199.166.28.10
199.166.29.3
199.166.31.3
195.117.6.25
204.57.55.100
4.2.2.1
4.2.2.2
4.2.2.3
4.2.2.4
4.2.2.5
4.2.2.6
64.129.67.101
64.129.67.102
64.129.67.103
151.197.0.38
151.197.0.39
151.202.0.84
151.202.0.85
151.202.0.85
151.203.0.84
151.203.0.85
199.45.32.37
199.45.32.38
199.45.32.40
199.45.32.43
192.76.85.133
206.124.64.1
67.138.54.100
220.233.167.31
199.166.31.3
66.93.87.2
216.231.41.2
216.254.95.2
64.81.45.2
64.81.111.2
64.81.127.2
64.81.79.2
64.81.159.2
66.92.64.2
66.92.224.2
66.92.159.2
64.81.79.2
64.81.159.2
64.81.127.2
64.81.45.2
216.27.175.2
66.92.159.2
66.93.87.2
199.2.252.10
204.97.212.10
204.117.214.10
64.102.255.44
128.107.241.185

#!/bin/bash
######################################################################################
# Derek Demuro, this script is given as is, CopyLEFT                                 #
######################################################################################
######################################################################################
# README                              LEAME                                          #
######################################################################################
# This script will run and ping each server in the list domain or ip, and if error 
# mail will be sent to address.
#
############################To Set Up#################################################
#
# To set this script up, you'll need to add it to a cronjob to run on boot
# Most linux distros will allow a param <a href="https://twitter.com/reboot" class="twitter-atreply ext">@reboot<span class="ext"><span class="element-invisible"> (link is external)</span></span></a> /(path)/servermon.sh
# So... crontab -e
# At the bottom add: <a href="https://twitter.com/reboot" class="twitter-atreply ext">@reboot<span class="ext"><span class="element-invisible"> (link is external)</span></span></a> /(path)/servermon.sh
# REMEMBER TO ADD EXECUTABLE BIT TO THE FILE (777 Permissions)
######################################################################################
# SCRIPT CONFIGURATION                                                               #
######################################################################################
### List of servers to check connection with
readonly SERVERLIST='serverList.txt'
### Where should the log be saved?
readonly LOGNAME='servermonitor.log'
### Who should we mail?
readonly MAILTO='mail@gmail.com'
### How many pings before we call it dead
readonly PINGCOUNT=2
### How much time between check's?
readonly SLEEPTIME=5
### How many records to keep
readonly CLEARLOGTMS=1000
######################################################################################
#################################FUNCTIONS START################################
 
###Function to clear the log
function clearLog() {
  echo 'Log cleared' > $LOGNAME
  echo 'Script will run ' $1 ' times then will clear itself'>> $LOGNAME
  return 0
}
 
#################################FUNCTIONS FINISH################################
#################################MAIN SCRIPT FUNC################################
while [ TRUE ]; do
    #add 1 to times
    times=`expr $times + 1`
    ##CLEAR THE LOG
    if [ $times -eq $CLEARLOGTMS ]; then
    clearlog
    $times=0
    fi
     
    ##Run script for every line in serverList
    cat $SERVERLIST | while read line
    do
        # check if there are no blank lines
        if [ ! -z $line ]; then
            ping -c 2 $line > /dev/null
            if [ "$?" -ne 0 ]; then
                echo -e "\e[00;31m`date +'%Y-%m-%d %H:%M:%S'` Something wrong with the server: $line\e[39m"
                echo "`date +'%Y-%m-%d %H:%M:%S'` Server has gone DOWN!" | mail -s "Server $line is DOWN `date +'%Y-%m-%d %H:%M:%S'`" $MAILTO
                # Or do send out mail
            else
                echo -e "\033[38;5;148m`date +'%Y-%m-%d %H:%M:%S'` All good: $line\033[39m"
            fi
        fi
    done
  ##We sleep till new run
  sleep $SLEEPTIME
done

#!/bin/sh
###############################################
#  Derek Demuro - OpenSUSE Packages Export    #
###############################################
 
# Absolute path to de zypper repositories lists.
readonly reposPath='/etc/zypp/repos.d/'
 
echo 'Welcome to Package Automation V1.1'
 
echo 'Enter 1 to do a full package export with package details'
echo 'Enter 2 to do a exportable package export, to install in another system'
echo 'Enter 3 to install a exported package list in another system'
echo 'Enter 4 to export your repos'
echo 'Enter 5 to import your repos'
echo 'Enter 6 to differ two sources files'
echo 'Enter 7 to quit'
 
PS3='Please enter your choice: '
options=("1" "2" "3" "4" "5" "6" "7")
select opt in "${options[@]}"
do
    case $opt in
        "1")
        echo "You chose 1, we'll do a full package export with description"
        echo 'What name would you like us to use to save the output?'
        read fname
        #Package name: Version : Release : Version Installed
        #Package Ej: yast2-trans-zh_CN:2.22.0:8.7.1:1346256812:YaST2 - Simplified Chinese Translations
        rpm -qa --qf '%{NAME}:%{VERSION}:%{RELEASE}:%{INSTALLTID}:%{SUMMARY}\n' | sort >> $fname
            ;;
        "2")
        echo "You chose 2, we'll do an installable version"
        echo 'What name would you like us to use to save the output?'
        read name
        #For installable version
        rpm -qa --qf '%{NAME}\n' | sort >> $name
            ;;
        "3")
        echo "You chose 3, we'll install all the packages in the file."
        echo 'Select 0 to install everything without confirmation, 1 for confirmation'
 
        read conf
 
        if [ $conf -eq 0 ]
        then
            conf=0
        else
            conf=1
        fi
 
        #Do you want to update your repository list
        echo "If you want to update your package list press 1"
        read update
        if [ $update -eq 1 ]
        then
            zypper ref
        fi
 
        echo 'Please type the name of the file that contains the package list'
        read FILENAME
         
        rpm -qa --qf '%{NAME}\n' | sort > old.txt
         
        diff -Naur old.txt $FILENAME | grep '^+' | sed s/+//g | tail -n +2 > differencesToInstall.txt
         
        FILENAME=differencesToInstall.txt
         
            #Loop untill file provided is okay.
            until [  -f $FILENAME ]; do
                if [ -f $FILENAME ]
                then
                    echo 'File seems to be fine'
                else
                    echo 'Whoops file doesnt seem to exists, please introduce it again'
                    echo 'Please type the name of the file that contains the package list'
                    read FILENAME
                fi
            done
 
        count=0
        #Count how many lines, and start installing
        cat $FILENAME | while read LINE
        do
            let count++
            echo "$count $LINE"
            #Zypper -With no output
            if [ $conf = 0 ]
            then
                #NO CONFIRMATION OR MESSAGES                
                zypper --non-interactive install $LINE
            else
                #NORMAL INSTALL
                zypper install $LINE
            fi
        done
 
        echo -e "\nTotal $count Lines read"
        rm $FILENAME;
            ;;
        "4")
        echo "You chose 4, we'll backup your sources list located at /etc/zypp/repos.d/."
        echo 'Input the full path of the folder where you want them copied'
        read dirtree
        cp /etc/zypp/repos.d/* $dirtree
         
        repoammount=`ls /etc/zypp/repos.d | wc -w`
        copiedamm=`ls $dirtree | wc -w`
        echo 'You had: ' $repoammount 'sources, and we copied: ' $copiedamm
        if [ $repoammount -eq $copiedamm ]
        then
            echo 'Everything seems to be copied'
        else
            echo 'Whoops, seems something is missing'
        fi
    ;;
     
    "5")    
        ########################################################################################
        # IMPORT THE SOURCES LIST                                                              #
        ########################################################################################
        cpCount=0
     
        echo 'Enter the absolute path to the directory containing the .repo files you wish to import.'
        read srcDir
         
        # Make sure there isn't a trailing slash on the path variable.
        echo $srcDir | egrep '.+\/$' >/dev/null
         
        if [ $? -eq 0 ]
        then
        # Remove the trailing slash.
        srcDir=`echo "${srcDir%/}"`
        fi
 
        srcFilesCount=`ls "$srcDir"/*.repo | wc -w`
        srcFilesList=`echo "$srcDir"/*.repo`
         
        for i in $srcFilesList
        do
        cp $i $reposPath
        if [ $? -ne 0 ]
        then
            echo "Something went wrong while copying the following file: $i :("
            break
        fi
        $(( $cpCount++ ))
        done       
     
        echo "We had $srcFilesCount files to copy, and we were able to copy $cpCount files."       
        ;;
    "6")
        ########################################################################################
        # COMPRARE TWO SOURCES FILES                                                           #
        ########################################################################################
        echo 'Type sources file 1'
        read sources1
        echo 'Type sources file 2'
        read sources2
        echo 'Type output file name'
        read filename
        # diff -u $sources1 $sources2 > $filename
        diff -Naur $sources1 $sources2 | grep '^+' | sed s/+//g | tail -n +2 > $filename
        ;;
             
        "7")
            break
            ;;
        *)
            echo 'Invalid option.'
            ;;
    esac
done

Using this, you can open a modal dialog in your view to show what you need.

#!/bin/bash
######################################################################################
# Derek Demuro, this script is given as is, CopyLEFT                                 #
######################################################################################
######################################################################################
# README                              LEAME                                          #
######################################################################################
# This script is done to attempt reboot on fallen server,
# After reboot message, so if reboot failed, it will send message.
# The reboot only works with local server, as ssh and autonegociation isnt done yet
############################To Set Up#################################################
#
# To set this script up, you'll need to add it to a cronjob to run on boot
# Most linux distros will allow a param <a href="https://twitter.com/reboot" class="twitter-atreply ext">@reboot<span class="ext"><span class="element-invisible"> (link is external)</span></span></a> /(path)/servermon.sh
# So... crontab -e
# At the bottom add: <a href="https://twitter.com/reboot" class="twitter-atreply ext">@reboot<span class="ext"><span class="element-invisible"> (link is external)</span></span></a> /(path)/servermon.sh >> (logname.log)
# REMEMBER TO ADD EXECUTABLE BIT TO THE FILE (777 Permissions)
######################################################################################
# SCRIPT CONFIGURATION                                                               #
######################################################################################
### List of servers to check connection with
readonly SERVERLIST='serverList.txt'
### Where should the log be saved?
readonly LOGNAME='servermonitor.log'
### Who should we mail?
readonly MAILTO='ddemuro@gmail.com'
### How many pings before we call it dead
readonly PINGCOUNT=2
### What is your local ip?
readonly LOCALIP='192.64.81.19'
### How much time between check's?
readonly SLEEPTIME=5
### How many records to keep
readonly CLEARLOGTMS=1000
#Critical load to stop everything and restart
readonly HIGH_LOAD=5
#Wait till load gets to this load
readonly LOW_LOAD=2
#Wait for highload
readonly HIGH=4
#Wait for midload
readonly MID=3
#Wait more for lowload
readonly LOW=1
# Enter the services you wish to start/stop when starting/stopping
readonly SERVICES='apache2 mysql postfix dovecot proftpd'
#####ENABLE SYSTEM RESTART IN CASE MACHINE LOOSES INTERNET?#######
readonly HARD_RESTART_FIX=1
######################################################################################
#################################FUNCTIONS START################################
function checkIfSpecial() {
  if [ $LOCALIP==$1 ]; then
    ###Add special actions here
     
    if [ $HARD_RESTART_FIX -eq 0 ]; then
      shutdown -r now
      mailx -s "Server $line is seems to be down" < /dev/null $MAILTO
    fi
  fi
  return 0
}
 
###Function to START a service
function startService() {
    echo "Starting $1"
    messages=`service $1 start`
    if [ $? -eq "0" ]
    then
        # Service failed to start
        echo $messages
        return 1
    else
        echo "$1 started succesfully"
        return 0
    fi
}
 
###Function to STOP a service
function stopService() {
    echo "Stopping $1"
    messages=`service $1 stop`
    if [ $? -eq "0" ]
    then
        # Service failed to stop
        echo $messages
        return 1
    else
        echo "$1 stopped succesfully"
        return 0
    fi
}
 
#####Check server load, if high stop!
function checkServerLoad() {
    #check = load at the moment
    check=`cat /proc/loadavg | sed 's/\./ /' | awk '{print $1}'`
    #Print of the time the cron started.
    echo "Load at the moment $check"
    #If load > 10
    if [ $check -gt $HIGH_LOAD ]
    then
    ###Shutting down! high load!
    changeServicesStatus 1
        #Moment of shutdown
        date=`date`
        echo 'Highload autorestart did run and load average was: '
        echo 'At: ' $date 'With load: ' $check 'services shutted down successfoully'
        #Sleep while HIGH LOAD
            while [ $check -gt $LOW_LOAD ]
            do
                if [ $check -gt $HIGH ]
                then
                    sleep $HIGH
                else
                    if [ $check -gt $MID ]
                    then
                        sleep $MID
                    else
                        if [ $check -gt $LOW ]
                        then
                            sleep $LOW
                        else
                            sleep 3                     
                        fi                
                    fi
                fi            
                #New check
                check=`cat /proc/loadavg | sed 's/\./ /' | awk '{print $1}'`
            done
        #Continue
        echo 'Restarting on load: ' $check
        ###Start Services
        changeServicesStatus 0
     fi
     return 0
}
 
###Function to clear the log
function changeServicesStatus() {
  if [ $1 -eq 0 ]
  then
    for service in `echo $SERVICES`
    do
      startService $service
    done
  else
    for service in `echo $SERVICES`
    do
      stopService $service
    done
  fi
}
 
###Function to clear the log
function clearLog() {
  echo 'Log cleared' > $LOGNAME
  echo 'Script will run ' $1 ' times then will clear itself'>> $LOGNAME
  return 0
}
 
#################################FUNCTIONS FINISH################################
#################################MAIN SCRIPT FUNC################################
while [ TRUE ]; do
    #add 1 to times
    times=`expr $times + 1`
    ##CLEAR THE LOG
    if [ $times -eq $CLEARLOGTMS ]; then
    clearlog
    $times=0
    fi
     
    checkServerLoad 
     
    ##Run script for every line in serverList
    cat $SERVERLIST | while read line
    do
        # check if there are no blank lines
        if [ ! -z $line ]; then
            ping -c 2 $line > /dev/null
            if [ "$?" -ne 0 ]; then
             
                echo "Something wrong with the server: $line"
             
                checkIfSpecial $line
                # Or do send out mail
            else
                echo "All good: $line"
            fi
        fi
    done
  ##We sleep till new run
  sleep $SLEEPTIME
done

Abstract:

The problem of managing multiple servers that do similar stuff and are used to host in house applications. Avoiding configuration time hogs and automation of server configuration, allowing fast deployments with no scaling limit.

The problem:

The problem with MultiAll is having all the servers on sync when they are indifferent “Farms or Grids” and the fact that one server can provide services to more than one purpose. This is why we invent the “Pool system” where all the servers respond or act for a defined pool.

If a server responds to multiple pools the aggregation of every pool is the server.

Understanding how we can make it work:

It’s possible to merge more than one server to work for a common goal under our research, making some modifications to the typical database-driven apps to have a cache table. When that server is unable to communicate to others under the same pool, they save the changes they have to make to their brother servers, and later on, when they are back online, deploy the changes.

On the file system side, every server is responsible for keeping others from the same pool up to date, bi partitioning the problem as every server is responsible for others and themselves.  We’re doing similar to how Rsync works, and we would only push differences between servers under a secure connection IE: (VPN).

By this moment we would have a DB Layer abstraction for multiple servers responding to a DB and file system.

Next is the Domain Name Server problem. We don’t want clients to reach servers offline due to maintenance or issues; for this, we’re developing on BIND 9 an abstraction layer that every server in a pool must be “Network-Aware” of others of his kind. If he’s unable to reach others, he must change the DNS registry to reflect the changes.

How MultiAll solves the problem:

MultiAll will work as a service provider inside the server, as a global abstraction layer. Once the application is provided with a bridge or connection layer to it, it would take advantage of the system. MultiAll has as key features:

1. Topology-Aware Neighboring System (TNS): provides a topology discovery service for the servers (or “nodes”), which may eventually “recalculate” the topology in case of unexpected downtime (such as network, power or hardware failures, among others) or instances of planned downtime (such as server maintenance, hardware upgrades or migrations). The different synchronization agents of the “Multi All” depend on information learned through TNS; therefore, it is considered a critical component of the “Multi All” system.
2. System Baseline Monitor (SBM, formerly known as “checker”): provides a 24/7 server health monitoring, locally on each server (or “node”) using “Baseline Rules.” These rules are defined on a “per pool” or “per server” basis, allowing the configuration to be as granular as needed, making exceptions if the need arises. The health status is published via TNS using standardized codes known as “SBM Statuses.” It is also considered a critical component of the “Multi All” system.
3. File System Synchronization Agent (FS-SA): lets you define structures inside your filesystem to keep synchronized across a pool of servers. FS-SA, used in combination with the rest of the “Multi All’s” Synchronization Agents (SA’s), provides you with high availability, data redundancy, and server load balancing on your pool of servers.
4. Software and Libraries Synchronization Agent (SL-SA): especially useful for large-scale unattended deployments, the SL-SA keeps all your software packages, services, and libraries consistent across the pool, raising awareness to the system administrators when possible conflicts, incompatibilities or other issues arise.
5. Database Synchronization Agent (DB-SA): keeps the different database servers of the pool synchronized. Depending on the needs of the underlying applications, the DB-SA may keep the DBs in perfect sync, or you could define your database synchronization policies.
6. Domain Name System Synchronization Agent (DNS-SA): keeps the DNS zones up to date with the pool’s topology either via a pull-push synchronization mechanism (handled by the FS-SA) or by rebuilding the DNS zone according to the topology discovered by the TNS.

The project has gotten a bit more ambitious… so here is how its changed!

Okay, so some research has been going on, and the project has been growing quite a bit. Among the changes that have been happening around, it has gotten bigger, now all the things stated before, are part of a much larger system.
TakeConnector will now be the daemon that will keep our infrastructure.

Abstract:

Using automation processes and technical skills, the idea is to develop a modularized, configurable, and environment aware trading system that can adapt to different currencies, news, and market trends without or with little human interaction.

The problem:

Automated trading is the same as manual trading, leaving the feelings out of the equation. Followed by intuition, computers don’t have feelings or intuition; that is the whole idea’s biggest downside. So, to begin with, we must break the problem into pieces, thinking about the best way to get accurate signals from the market, running them by our own “automated market entry proofing,” avoiding wrong signals and leaving only the safe ones.

Understanding how we can make it work:

Using our technical skills in math, programming, and tests, we can produce a system aware of news, wrong long and short market entries. Taking advantage of those brokers who widen the spread during news time, avoiding entering the market on those moments.

We are using trading strategies as BEP (Break-Even Point), which will allow us to step back on our initial entry or lot size. And according to the trades, we have opened running them through the system, deciding if the stop loss and take profits are well fitted and if the sudden market change happened being able to step away without losses.

As you may or may not know, in Forex, you always lose. Every time you win, someone else is losing. Vice versa, so we have to try to do a great job on money management, and later on, trying to show or teach the computer the best well-known methods to step in or out of market-defining possible conditions, or giving the system a way to understand either good or bad moments.

How this expert solves the problem:

We are making this expert with all the knowledge we have on Forex, plus technical and mathematical skills.  

Dividing the problem on the following steps:

1. Shutdown (System that will make the expert stop trading and once all trades are close, shutdown completely)
2. Emergency Shut Down (Close everything assuming losses).
3. Money Management (According to how much money you’re investing, calculating your risk choose the best lot sizes).
4. Lot Adjustment: Adjust lots manually if you change from one broker or another or don’t want money management.
5. Entry Signals (Configure the way according to the pair to be traded what signals work best or are more accurate).
6. Exit Trades: (Configure how opened trades will or should exit, setting stop losses, take profits, monitoring, etc.)
7. Trade Monitor: (Configure what will be monitored from opened trades).
8. Display: (Configure what is displayed on the screen when on automated trading).
9. Alerts: (Configure when a trader should be alerted).