Controlling
This page is under construction until the implementation of the
controller is done. In the meanwhile, you may enjoy the draft
document.
Looky Controller
===============
load
^
|
load.max --|---------------------------[SL] load.max
|
load.opt --|-------------------[HL] load.opt
|
|
|
| [M]
|
+---------------------|-------|-----------> # users
have.opt have.max
(1) For each resource the following characteristics must be given
(by configuration on start)
- a high-load watermark
The high-load watermark is a tuple (have.opt, load.opt). These
are mandatory parameters and must be set in the configuration file
for each resource.
The high-load watermark defines the optimum of #users and #load
that has to be controlled by Looky. The high-load watermark
must be computed externally.
- a stress-load watermark
The stress-load watermark is a tuple (have.max, load.max). The
tuple is optional. If not set, these values are same as
high-load watermark.
For each pool the following characteristics must be given as
configuration:
- pool-forecast-factor
The load-factor is used to estimate the forecast of user
development. Actually it is a measure how many consumer may ask
for a new resource during a period on a statistic level.
If this value is not set, the pool-forecast-factor is set to 1.
- pool-increaser
The increase-pool is used to level Controllers behaviour in case
of overloaded pools. An overloaded pool is a pool where no
further resources exist, but the load of the resources used for
this pool still signals there is further capacity on a specific resource.
In this case, the controller will try to increase the pool
resources according to the capacity of the resources in the
pool. Looky will try to modify the pool.resource.ratio to get at
least pool-increaser free resources for this pool.
If this value is not set, the pool-increaser value is set to 100.
(2) The Looky-Controller must master the Looky-Server. If this fails,
a fatal error occurs and the Looky-Controller terminates.
Once Looky-Controller is started, Looky-Controller tries to
contact the Looky-Server.
* If this was successfully, the controlled Looky-Server is
mastered. If the attempt to master the Looky-Server fails,
the Looky-Controller server terminates immediately.
* However, if the Looky server is not running, the
Looky-Controller server ignores that fact: Instead of mastering
(see (6)), the Looky-Controller server does reconnecting.
The reconnecting procedure is quite simple: the Looky-Controller
tries to contact and master the Looky-Server. If there is no contact,
the reconnecting procedure will be redone in the next
period. If no mastering is possible (maybe an other Controller process
is active) the Looky-Controller will terminate. If the
Looky-Server is accepting this Controller as master, the
reconnecting procedure is replaced by a mastering procedure in
the next period.
If the Looky-Controller loses connection during mastering, the
mastering procedure is replaced with the reconnecting procedure in
the next period.
(3) During Looky-Controllers life-time, the server accepts the
following control data per resource
PRESENCE (active)
An presence pdu passes a Boolean value 'active'. If this
value is false, the corresponding resource is assumed as not
being present for consumers. If a resource is not present, the
resource is set to "locked" by the Looky-Server immediately.
METRIC (have.opt, load.opt)
METRIC (have.opt, load.opt, have.max, load.max)
An metric pdu passes the high-load/stress-load metric points
of the resource. have.max and load.max are optional. If not
set, the .opt values are used as defaults.
A metric pdu dramatically changes the relationship between
quantity and have.opt value.
* By start up, the looky controller sets the resource in so called
'fundamental mode': here the have.opt value is calculated by
quantity:
- If the resource is shared, the have.opt value is the sum of
all quantities set for this resource in different pools.
- If the resource is coupled, the quantity value of this
resource is the same in all pools. Here the have.opt value is
equal to this quantity value.
* If a M-data pda is received, the controller changes the resource
into so called 'measured mode'. Now the quantity values are
calculated by given have.opt value:
- If the resource is shared, the have.opt value is the
sum of all quantities (sum.quantities) of this
resource. If sum.quantities is different to have.opt, all
quantities are leveled by the formula have.opt/sum.quantities.
- If the resource is coupled, the have.opt value sets all
quantities in all pools the resource is used in.
A presence pdu changes the availability of a resource: the
resource is rated as 'inactive' or 'active':
* An 'active' resource is unlocked on Looky-Server. The resource
is recognised during mastering process! (See (6))
* An 'inactive' resource is locked on Looky-Server. The resource
is not recognised during mastering process! (See (6))
(4) For each resource a probe may be passed in regular or irregular
time periods to the Looky Controller. The Looky Controller accepts
the following information per resource:
PROBE (got.used)
PROBE (got.used, got.load)
PROBE (got.used, [got.load.pool, ...])
PROBE ([got.used.pool, ...])
PROBE ([got.used.pool, ...], got.load)
PROBE ([got.used.pool, ...], [got.load.pool, ...])
A probe pdu is a tuple of current load got.load and current
number of user got.used.
The load value may be omitted. In this case the load value is
estimated using the formula
got.load = (resource.load.opt/resource.have.opt) * got.used
The got.used and got.load values are seen as the sum of the
use of the pools. The individual got.used.pool or got.load.pool
value is computed with the following formula
got.used.pool = got.used/#{resourceInPoolList.resource}
resp.
got.load.pool = got.load/#{resourceInPoolList.resource}
Instead of got.used or got.load values, individual
got.load.pool resp. got.used.pool values may be passed with
the probe pdu. In this case the corresponding got.used and
got.load values are computed as sum of the individual pool
values.
(5) The Looky-Controller process works in so called 'periods'. Each
period has a time-point (timet()) where this period is started ---
duration is given by configuration.
During each period the Looky-Controller process waits for probe
pdus. The data is collected until the period ends.
If the period is over, all collected data is rated and passed back
to the mastered Looky server.
period
,-----------------.
| |
[0]---------------[*]---------------[*]------- ...
| collecting | collecting |
| | |
mastering(0) mastering mastering
| | |
v v v
(6) The mastering process is done per resource with respects to the pools.
* First of all the mastering process reads-out the current
looky-data values. Here we get three values for each resource
in each pool: a looky.pool.resource.have, a looky.pool.resource.used
and a looky.pool.resource.ratio value.
* Each looky.pool.resource.have value is compared with the
corresponding pool.quantity.resource value. If a
looky.pool.resource.have value differs, the
looky.pool.resource.have value is overwritten by
pool.quantity.resource value.
[DONE,TESTED]
* After this, all resources are rated if got.load or got.used
values were updated since last period.
+ If an update was done, the resource is rated as 'confirmed'.
For each confirmed resource, the controller must check the
looky.pool.resource.used value:
- if the resource is 'shared' the looky.pool.resource.used
value is set to the got.used.pool value
- if the resource is 'coupled', tje looky.pool.resource.used
value is set to the got.used value.
+ If no update was done, the resource is rated as 'unendorsed'.
For 'unendorsed' resource, the Controller process believes in
the values which were sent by Looky server. There are no
looky.pool.resource.used changes done for this resources.
[DONE,TESTED]
* After this two steps, the looky.pool.resource.used and
looky.pool.resource.have values are updated according to the current
data collected by the Controller process. The first step is to rate
the looky.pool.resource.ratio value. There are three conditions
that have to be taken into account: once a condition matches, the
action is performed and this step is done.
Load
^ high max load-status:
|xxxxxxxxxxxxxxx|xxxxxxx| ,' x ~ overheated
|xxxxxxxxxxxxxxx|xxxxxxx|,' o ~ stressed
+---------------|------,#----- max d ~ hyperactive
|ooooooooooooooo|oooo,'o|
|ooooooooooooooo|oo,'ooo|
|ooooooooooooooo|,'ooooo|
+--------------,#-------|----- high
|dddddddddddd,' |rrrrrrr|
|dddddddddd,' |rrrrrrr|
|dddddddd,' |rrrrrrr|
|dddddd,' |rrrrrrr|
|dddd,' |rrrrrrr|
|dd,' |rrrrrrr|
|,' |rrrrrrr|
+---------------+-------+------> Used
(a) The resource is inactive or resource.have.opt is 0. In this
case the Controller does nothing. The resource is rated as
'ignored'.
(b) The got.load value is higher than it should be: in this case
the looky.pool.resource.ratio must be set to a value smaller
than 1 to reduce the load that might be given by future
consumers (those are counted by 'used' values)
First of all the got.load value is compared to the load.max
value. If the got.load value is higher than the load.max
value (region marked up by 'x'es in picture above) the
looky.pool.resource.ratio is set to a value that stops
the usage of this resource in all pools immediately. The
following equation is used.
looky.pool.resource.ratio =
min(1, looky.pool.resource.used / looky.pool.resource.have)
[DONE]
If the got.load value is smaller than the load.max but
bigger than the load.opt value (region marked up by 'o's in
picture above) the following is computed for a coupled
resource:
^
|
|
load.max |--------------x------------#---
| /| /|
| m/ | m/ |
got.load |...........O | / |
| : | / |
load.opt |--------------|-------#----+---
| : | | |
| : | | |
| : | | |
+-----------+--+-------+----+---->
| | | |
got.used | | have.max
| have.opt
have.load.max.forecast
with:
m = (load.max - load.opt) / (have.max - have.opt)
f(x) = m ( x - got.used ) + got.load
f(x) <=> load.max
x <=> have.load.max.forecast
we get
have.load.max.forecast = got.used + (load.max - got.load) / m
looky.pool.resource.ratio =
min(1, have.load.max.forecast / have.max)
If the resource is shared, the got.used and got.load values
are replaced by a got.used.pool resp. got.load.pool in the
equations above.
[DONE]
If the got.load value is smaller than the load.opt value,
the got.load value is compared to the expected.load
value. The expected.load provides the load that is expected
by the got.used number of average users, the expected.load
is calculated as follows:
expected.load = (load.opt * got.used) / have.opt
If the got.load value is bigger than the the expected.load
value (marked up by 'd's in picture above), the
looky.pool.resource.ratio value is calculated as follows for
a coupled resource:
^
|
load.max -+-----------x1-------#
| /| /|
| m/ | m/ |
| / | / |
load.opt -+-------x0-------#---+---
| n,' | ,'| |
got.load -+----O |,' | |
| | ,' | |
| | n,' | | |
| | ,' | | |
| |' | | |
| ,'| | | |
|,' | | | |
+----+------+----+---+----------->
| | | |
got.used | | have.max
| have.opt
have.load.max.forecast
with
n = load.opt / have.opt
f(x0) = n ( x0 - got.used ) + got.load
f(x0) <=> load.opt
x0 = got.used + (load.opt - got.load) / n
m = (load.max - load.opt) / (have.max - have.opt)
g(x1) = m ( x1 - x0 ) + f(x0)
g(x1) <=> load.max
x1 <=> have.load.max.forecast
we get:
have.load.max.forecast = got.used +
(load.opt - got.load) / n +
(load.max - load.opt) / m
looky.pool.resource.ratio =
min(1, have.load.max.forecast / have.max)
If the resource is shared, the got.used and got.load values
are replaced by a got.used.pool resp. got.load.pool in the
formulas above.
[DONE]
If the got.load value is smaller that the expected.load
value, the looky.pool.resource.ratio is set to 1.
[DONE]
(c) A pool does not provide any further resources (maybe some
resources are inactive), but the got.load value of at least
one resource in this pool is lower than expected. In this
case the looky.pool.resource.ratio may be set to a value bigger
than 1 to allow, for this specific case, an 'overloaded'
number of users.
However, the looky.pool.resource.ratio must not violate the
given resource.have.max value and it must not violate the
given resource.load.max value.
First of all, the number of available resources must be
calculated per pool and the resources of a pool must be
rated as 'underloaded', 'overloaded', 'available' or
'not-available'.
- A resource is 'not-available', if the resource is inactive
or frozen (as done by administrator).
- A resource is 'overloaded', if the resource isn't
'not-available' and the ratio value is smaller than 1.
- A resource is 'available', if the resource is neither
'not-available' nor 'overloaded' and the computed
looky.pool.resource.used * pool.forecast.factor value is
smaller than the looky.pool.resource.have value.
- All other resources are 'underloaded'.
If at least one resource is rated as available, there is
no need for the Looky.Controller to continue this procedure.
The pool may accept new customers and the procedure stops
here.
If no available resources are located in a pool, the pool is
rated as 'full'. For each full pool, the Looky Controller checks
if the pool is expandable. To do this, the number of
underloaded resources in the pool are calculated as
pool.ursize
pool.ursize = sum(underloaded)(1)
A pool is 'underloaded' if at least one underloaded resource
exists in this pool (or pool.ursize > 0). If a pool is not
underloaded, Looky Controller cannot do anything and the
procedure stops here.
For an underloaded pool, the Looky Controller checks if
there is a chance to extend the resources used in the
pool. For this, the pool.sum.capacity and the pool.sum.used
values are computed as follows:
pool.sum.capacity = sum(underloaded)[have.max]
pool.sum.used =
sum(underloaded)[looky.pool.resource.used]
A pool is 'extendable' if the pool.sum.capacity value is
bigger than the pool.sum.used value. If the pool is not
extendable, the Looky.Controller can't do anything. The
procedure stops.
If the pool is extendable, we must think about a good algo
sometimes later ... [CCC]
(7) Adding, Deleting, Assigned or Dropping Resource
very very painfull! [CCC]
|
|
controlling