Configure High Availability

High availability architectures provide the fault tolerance required today.

Access Gateway high availability consists of:

  • A single administration instance of Access Gateway. The Administration instance, typically referred to as the admin node, is used to maintain and propagate configuration changes to worker nodes. In addition the admin node can be used a normal Access Gateway instance.
  • One or more worker instances, bound to the admin node, which service requests.
  • A customer provided load balancer, routing requests to the Access Gateway High Availability cluster.

Access Gateway high availability instance life-cycle overview:

  • An instance of Access Gateway is provisioned. This instance, referred to as the Admin node, is configured normally, including defining protected applications. Note that is not a requirement to configure applications or IDP support before configuring high availability.
  • Second and subsequent instances of Access Gateway are provisioned. These instances, referred to as Worker nodes, are not configured with applications, but rather obtain all configuration from the Admin node.
  • Worker nodes are then specifically configured to use the Admin node for all configuration. Once configured, worker nodes no longer expose the admin UI but can only be accessed using the command line interface.
Access Gateway High Availability architecture
Important Note

Important

The Admin UI is only available on Admin nodes and those nodes which have yet to be configured as worker nodes. Once a instance is configured as a worker, it no longer can be accessed using the Admin UI. All administration of a an Access Gateway cluster must occur via the Admin node and its admin ui.

Overview

Configuring high availability includes the following overall process:

  1. Configuring an Admin node.
    During this step the Administration node is configured normally.
  2. Configuring a Worker node.
    During this step worker nodes are configured without any applications.
  3. Preparing the Admin node.
    During this step the command line interface is used on the Admin node to alert or prepare the admin node for the addition of one or more worker nodes.
  4. Preparing the worker node.
    During this step the command line interface is used on the Worker not to prepare the worker to become part of the Access Gateway cluster.
  5. Worker integration into cluster.
    After the prior two steps, the worker is automatically integrated into the cluster. During this phase the worker Admin UI is disabled and worker exchanges keys and is provided configuration from the Admin node.
Access Gateway High Availability add worker node sequence diagram

 

Operations

To configure High Availability:

  1. Reset the key associated with an Access Gateway Node - Reset the keys in both the Admin instance and the worker instance.
  2. Add a worker node to an Access Gateway Cluster - Add one or more worker nodes to the cluster.
  3. Check Cluster Configuration - Review the final configuration.

 

Reset the key associated with an Access Gateway Node

Access Gateway nodes use various keys to intercommunicate.  Keys must be regenerated if an instance is to be used as part of a Access Gateway High Availability cluster.
Each instance need regenerate its keys only once.

To reset a nodes keys:

  1. Connect to the Access Gateway Management console.
    ssh oag-mgmt@[admin or worker]
  2. Select 5 - System
  3. Select 8- High Availability Configuration
    The High Availability Configuration menu will display.
  4. Select 1 - Reset Key node. Confirm the key reset will be displayed.
    This will reset keys being used by high availability sync process.
    If high availability is already setup, this option should not be used.
    Proceed with reset? [y/N]: 
  5. Enter y to reset or N to abort the reset process.
    The reset of either abort, or display a success message.
  6. Enter x to exit or any other menu item to continue.

 

Add a worker node to an Access Gateway Cluster

Info

Note

When adding a worker node, both the Administration node and the worker node:

To add a worker node to a cluster:

  1. On the Admin node

    1. Connect to the instance Access Gateway Management console.
      ssh oag-mgmt@[admin.tld]
    2. Select 5 - System
    3. Select 8- High Availability Configuration
      The High Availability Configuration menu will display.
      Access Gateway High Availability Setup (role)
      1 - Reset Key
      2 - Prepare Admin
      3 - Prepare Worker
      4 - List Nodes
      5 - Remove Node
      6 - Check Status
      
      X- Exit
      Choice: 

      Info

      Note

      The High Availability menu displays the current role of a Access Gateway node.
      Roles can be:

      • Single - The node has not yet been configured as either a Worker or an Admin.
      • Admin - The node has been configured as an administrator for High Availability.
      • Worker - The node has been configured as a worker for High Availability).
    4. Select 2 - Prepare Admin
      Info

      Note

      The first time the Admin Node is configured for high availability option 1 - Reset Keys should be executed to reset the instance's SSH keys.
      Reset Keys need only be executed a single time per instance. See Command Line Management Console reference for complete details.

      Important Note

      Important

      Access Gateway Replication users the hostname setting from the command line console.
      Please be sure to update the hostname for both Admin and Worker nodes using the command line console System (5) > Change Hostname (1).

      . . .

    5. The Admin node generate and display an authorization code which must be provided to the Worker node.
      Copy the authorization code shown in bold italic below.
      Authorization token required to initiate setup from worker nodes is given
      below. Copy the text below this line and paste it on worker node when prompted.
      oag.okta.com:8ba1c123-715d-4b70-ab5d-0e41493bef73
      Worker nodes available so far:
      . . . 								
      The Admin node will then wait for worker nodes.
    Important Note

    Important

    At this point the admin node is waiting for connections from worker nodes. Leave the window open until all workers nodes have been added. Entering X prematurely will cause the Admin node to assume the process is complete and stop listing for worker node additions.
    Enter X only after adding all worker nodes

    1. Return to the command prompt on the instance which is being attached as a worker node.
  2. On each Worker node
    1. Connect to the instance Access Gateway Management console.
      ssh oag-mgmt@[worker.tld]
    2. Select 5 - System
    3. Select 8- High Availability Configuration
      The HA Configuration menu will display.
    4. Select 3 - Prepare Worker
      Info

      Note

      The first time the worker Node is configured for high availability option 1 - Reset Keys should be executed to reset the instance's SSH keys.
      Reset Keys need only be executed a single time per instance. See Command Line Management Console reference for complete details.

    5. The worker will then display
      Checking HA readiness for host worker. . .   
      
      NOTE: Please ensure that admin node is ready for setup and you have the
      authorization token displayed on admin node.
      
      Enter the authorization token displayed on admin node: admin.. . . .com:927da506-7efb-4520-bd32-dd03b86f2a9b
      

      Once entered the Worker node will then connect to the Admin node and exchange authorization and confirmation information resembling:
      Requesting admin  node admin. . . .com to allow connection  
      Node worker. . .com successfully added on admin node
      Synchronizing current configuration  
       
      Press enter to continue ....
      
    6. When prompted press any key to continue.
    7. Enter X to exit or any other menu item to continue.

    The worker instance is now configured and ready for use.

On the Admin node:

  1. Return to the Admin instance Access Gateway Management console.
  2. Examine results of adding the new worker node which should resemble:
    Authorization token required to initiate setup from worker nodes is given
    below. Copy the text below this line and paste it on worker node when prompted.
    
    admin. . .com:927da506-7efb-4520-bd32-dd03b86f2a9b
    Worker nodes available so far:
    worker1. . .com
    worker2. . .com
    worker3. . .com

    Note the addition of the new worker node.
  3. Enter X to exit or any other menu item to continue.

 

Check Cluster Configuration

To review or check the status of an Access Gateway High Availability Cluster:

  1. Connect to the Access Gateway Management console.
    ssh oag-mgmt@[admin or worker]
  2. Select 5 - System
  3. Select 8- High Availability Configuration
    The High Availability Configuration menu will display.
  4. Select 6 -Check Status .
  5. A list of cluster instances is displayed with their associated status.
    Use the Up/Down/Page Up/Page Down/Home/End keys to scroll through the list. Which will resemble:
     
    HA Synch Status    Up/Down/Page Up/Page Down/Home/End - Scroll    x-exit
    worker1.yourdoman.tld:    Pass
    worker2.yourdoman.tld:    Pass
    . . . 
    workern.yourdoman.tld:    Fail										
    					
    Where:
    • Pass: Reachable functioning worker node.
    • Fail: Non-functional worker. See node log for details.
  6. Enter x to exit.