Cube Builder

Cube Builder is a .NET Client/Server application divided in three different modules which aim to perform the following:
– Register Several OLAP Clients in a farm, split by a dividing criteria (user created Groups, also called Affinity Groups)
– Clone OLAP cubes using the SSAS Services by Microsoft
– Distribute evenly the work among the Clients in either “free automatic fashion” or the mentioned user criteria
– Provide fault tolerance by these rules:
– Any Client can promote to Server in case the Server dies
– Eliminating a Client in case it becomes unnaccessible and re-distributing the work among those who remain alive
– Should a group of Clients die, the pending jobs are transferred to the catch-all Group (more on this below)

Distributed architecture overview:
“The work” consist of a List of Jobs with a priority which can be “normal” or “high”. Both priorities are queued in a circular priority queue and the piece that dispatches “the work” takes care of starvation by periodically examining the queue for nearly timed out Jobs. This queue is in reality divided in two, one per priority
There is also a Group of Non-Grouped (NGG) Clients which works as a kind of catch-all Group. This Group is mandatory

As you may already have guessed these three modules are:
1) The Server
2) The Client
3) The GUI

The Server is a Windows service basically divided in three parts:
1) The Client Dispatcher:
Takes care of a per-Client (FIFO) list of Jobs
Should take notice of an unnaccessible Client it will re-arrange the List of pending Jobs for that particular Client in this way:
– If the Client is the last in its Group the pending Jobs are queued in
– If the Client is not the last, it will take the freemost Client in the Group
– If all Clients in that Group are being equally used, the list of pending Jobs for that Client is sent to the NGG
Takes note of the Client’s Availability Index

2) The Job Scheduler
Receives new Lists of Jobs and attempts to pass them to the Client Dispatcher in a timely manner
Takes care of the work queue, Items starvation and priority and maintains the Affinity Groups
Each Item in the Work Queue is composed of List of Jobs. The Jobs in that List are executed FIFO
The Job Scheduler examines the Queue an removes Items in this way:
– If there’s a free Client in the Item’s Affinity Group the Item is sent to that Client
– If NO free Client in the Item’s Affinity Group the Item’s priority is increased (until eventually reaching ‘topmost’) and the Item is queued back
– If the Item in the Queue is nearly starving its priority becomes ‘topmost’
– If Item priority = ‘topmost’:
– If free Client in the Affinity Group, the Item takes precedence over the rest. If more than one Item is ‘topmost’ they execute FIFO
– If NO free Client in the Affinity Group the Item is sent to the NGG. If more than one Item is ‘topmost’ they execute FIFO
– If everything didn’t work, the List of Jobs is queued back and waits for another opportunity later
Optionally, the end user can specify that Jobs should keep within their bounds (so they don’t get routed to the NGG). In such case the rules are the same except that the NGG is not considered. Starvation is prevented by the priority queue.

3) The keepalive framework
– Makes sure the fault tolerance is met
– Promotes Clients to Servers
– Removes unnaccessible Clients
– Tells the Client Dispatcher about changing the Affinity Groups if all the Clients in a Group died
– Maintains the list of Clients so each Client and the Server are aware of changes. If a Client is promoted it gets removed from the List of Clients. If the Client is the last in its Group all the requests to that Group are re-routed to the NGG, no matter if the end user specified the contrary
– Allows a Client to promote. Promotion occurs in FIRST-WIN fashion:
– Client A realizes the Servers is not responding the keepalives
– Broadcasts a CLIENTATTEMPTPROMOTION message
– Waits a configurable amount of seconds for an answer
– If the broadcast message is not answered it promotes to Server and removes itself from the List of Clients. A Server cannot work as a Client
– If the broadcast message is answered by Client B, Client A compares its ID with Client B’s. The lowest ID wins. If Client A wins it will answer Client B with a PREVENTCLIENTPROMOTION message. Since Client B was expecting an answer to its own CLIENTATTEMPTPROMOTION, promotion gets aborted

The Client is a Windows service that sends items in a List of Jobs to the OLAP servers defined in its list of servers and computes the Availability Index. Each Client belongs to an Affinity Group (even those that belong to the NGG)
Items in the List of Jobs are executed FIFO
All Clients share Client Information with the Server and with each other so if the Server dies, one Client can promote
Clients also send status information to the Server

The GUI allows the end user to launch jobs, configure the application and add/remove Clients. It also contains business related information such as Cube models etc.

Among the business related capabilities, the application allows to prune portions of a cube, create cube templates and clone cubes based upon those templates. It also allows certain functionality in OLAP 2000 cubes, limited only by the limitations of that version. Its natural environment is SSAS 2008 and/or SSAS 2012.

Technologies used:
C#, C++, .NET, SQL Server, SSAS, AMO, DSO, TCP/IP, Winsock