Task: Distribute Analysis Work Over Specific Machines

• Overview
• Permissions Needed
• Set Up and Run a Distributed Analysis
• What Next?
• Links

Overview

To set up an analysis that is distributed across a specific set of machines you will first ensure that CodeSonar launch daemons are running on all machines in the set, then run an analysis that is configured to request slaves from all those launch daemons.

Most of the work in this task is concerned with establishing bidirectional relationships between the analysis and the launch daemons that will contribute slaves.

• Launch daemons must specify that the analysis may make use of them.
  Specifically, the hub user who is authorizing the analysis must have permission LAUNCHD_START_SLAVE for every launch daemon they wish to make use of.
  A straightforward way to ensure this is to use only launch daemons for which you are the owner, since the owner of a launch daemon L is always treated as if it has all LAUNCHD_* permissions for L. This is the approach we will take in this task.
• The analysis must specify which launch daemons may be used.
  To do this, use configuration file parameter REMOTE_ANALYSIS_SLAVES_LAUNCHDS.

For the sake of this task, we suppose the following.

• Your hub user account has username alex, and uses password-based authentication.
• Your system user name on all machines you will be using is sysalex.
  • For many users, system username and hub user account username will be the same. We distinguish them here for clarity.
• The software project to be analyzed has build command make projX.
  We will call the corresponding CodeSonar project "ProjectX", and let it be a child of the root project tree.
• The hub location is hubmach:7340.
• You are distributing the analysis across machines machineA, machineB, and machineC.

Permissions Needed

You will need the following permissions.

• On the root launchd group:
  • LAUNCHDGROUP_ADD_CHILD so that you can create new launch daemons if necessary.
  • LAUNCHDGROUP_EXISTS and LAUNCHDGROUP_READ so that you can access all the relevant functionality on the Analysis Cloud page.
• If project ProjectX already exists on the hub:
  • PROJECT_ADD_CHILD on ProjectX, so that you can perform an analysis.
  • PROJECT_EXISTS and PROJECT_READ on ProjectX, so that you can view project information in the CodeSonar web GUI.
  If it does NOT already exist on the hub, you will need the following permissions on some project tree. (This task assumes that this is the root project tree.)
  • PTREE_ADD_CHILD so that you can create a new project.
  • PTREE_READ so that you can access all the relevant functionality on the Home page.
  • PROJECT_ADD_CHILD, so that ProjectX inherits it, allowing you to perform an analysis of ProjectX.
  • PROJECT_EXISTS and PROJECT_READ, so that ProjectX inherits them, allowing you to view ProjectX project information in the CodeSonar web GUI.
• Global permissions G_SIGN_IN and G_SIGN_IN_PASSWORD.

Additional permissions

These permissions are not required to set up and run the distributed analysis of ProjectX, but provide access to analysis information and functionality. You will therefore need these permissions if you want to browse analysis results.

• The following ANALYSIS_* permissions for ProjectX (either directly or through resource inheritance from an ancestor project tree): they will be resource-inherited by all of its analyses.
  • ANALYSIS_DELETE
  • ANALYSIS_EXISTS
  • ANALYSIS_READ
  • ANALYSIS_WARNING_READ
  • ANALYSIS_WARNING_EXISTS
  • ANALYSIS_WRITE
• Global permissions G_LIST_USERS and G_LIST_PROPERTIES.

Set Up and Run a Distributed Analysis

The procedure for setting up and running a distributed analysis can be divided into four stages.

• A. Select the machines that will contribute to the analysis
• B. Start launch daemons on the selected machines
• C. Set up and run the analysis

A. Select the machines that will contribute to the analysis

The first step is to choose which machines you will distribute the analysis across. Note that there must be at least one machine, or else the analysis will not be able to make any progress.

• The machines must all be running the same version of CodeSonar as the machine on which you will start the analysis.
• The machines must all be able to access the hub.
• The machines must all be able to connect to the machine running the analysis master (that is, the machine on which you start the analysis). Specifically, they must be able to connect to the ANALYSIS_MASTER_LISTEN_INTERFACE specified for the analysis.

Once you have identified the machines you will use, go on to Stage B.

B. Start launch daemons

Next, you will start a launch daemon on each machine that you selected in stage A.

1. First, check to see if you are already running launch daemons on any of your selected machines.
   1. If the Web GUI is not already open, open it now.
   2. Sign in as a user with the required permissions.
   3. Navigate to the Analysis Cloud page:
      1. Click the Settings link at the top right of the page.
         The Settings page will open.
      2. Change to the Other Links tab.
      3. Click the Analysis Cloud link.
         The Analysis Cloud page will open.
   4. Inspect the table of launch daemons. For each machinename that you selected in Stage A, look for a table entry for which:
      • Machine is machinename
      • Hub User is alex
      • User is your system user name on machinename
      • Status is Online
   For each machinename that does not have such a table entry, you will start a new launch daemon in the next step.
2. On each machinename on which a suitable launch daemon is not already running:
   1. Sign in as sysalex.
   2. Execute the following codesonar install-launchd command.
      codesonar install-launchd -auth password -hubuser alex hubmach:7340 -max-processes auto
      • Enter the password for the alex hub user account when prompted.
      • [Windows only] If the command seems to be taking a long time to complete, it is probably waiting for service authentication: the service authentication dialog may be hidden under other windows.
3. Reload the Analysis Cloud page and inspect the table of launch daemons.
   • There should be one entry for each of the launch daemons in your selected set.
   • The launch daemons should all be "Online".
   • The Max Processes for each launch daemon in your set should be either auto or a positive integer.

     If any launch daemon in the set has 0 in this column, enter a new value and click Save.

4. Add the Path column to the table:
   1. Click on any column heading to open the column management menu.
   2. Select Path from the Show> submenu.
   The table will update to include the Path column.
5. Keep the Analysis Cloud page open. You will need the Path value for the launch daemons later, and you can read them out of the table.

C. Set up and run the analysis

On the machine where you want to run the analysis master, configure and run the analysis as follows.

1. Change to the directory where you will be running the analysis.
   cd /myprojects/projX
2. Create a general project configuration file for ProjectX if one does not already exist.
   codesonar create-conf ProjectX
   CodeSonar will create a file called ProjectX.conf in the current working directory.
3. Edit ProjectX.conf as follows.
   1. Add the following rules.

      REQUEST_REMOTE_ANALYSIS_SLAVES=Yes
      
      ANALYSIS_SLAVES=Auto
      

   2. For each launch daemon in your selected set, a rule of the form REMOTE_ANALYSIS_SLAVES_LAUNCHDS+=launchd_path, where launchd_path is the launch daemon Path as shown in the Analysis Cloud page.

      For example:

      REMOTE_ANALYSIS_SLAVES_LAUNCHDS += /sysalex@machineA
      REMOTE_ANALYSIS_SLAVES_LAUNCHDS += /sysalex@machineB
      REMOTE_ANALYSIS_SLAVES_LAUNCHDS += /sysalex@machineC
      

   3. If necessary, add a rule of the form ANALYSIS_MASTER_LISTEN_INTERFACE+=accessible_interface, where accessible_interface is an interface on the analysis master that all the machines in your selected set can access.
      Note that this is not always necessary: the default behavior when REQUEST_REMOTE_ANALYSIS_SLAVES=Yes and ANALYSIS_MASTER_LISTEN_INTERFACE is not specified is to listen on all interfaces on a port allocated by the operating system. You therefore only need to set ANALYSIS_MASTER_LISTEN_INTERFACE in the following cases.
      • A configuration file used by the analysis (such as the general template configuration file) has been customized to specify a restrictive interface that some of your selected machines cannot access, so you need to override it with a more permissive setting.
      • The default behavior is too permissive for your security needs, so you need to override it with a more restrictive setting, while still ensuring that the specified interface can be accessed by all your selected machines.
      For example:

      ANALYSIS_MASTER_LISTEN_INTERFACE=[::]
      # or
      ANALYSIS_MASTER_LISTEN_INTERFACE=[::]:1234
      

   (See the documentation for REQUEST_REMOTE_ANALYSIS_SLAVES, REMOTE_ANALYSIS_SLAVES_LAUNCHDS, and ANALYSIS_SLAVES , and ANALYSIS_MASTER_LISTEN_INTERFACE for more information.)
4. Execute the analysis command.
   codesonar analyze ProjectX -auth password -hubuser alex hubmach:7340 make projX
   • Enter the password for the alex hub user account when prompted.
   • [Windows only] If the command seems to be taking a long time to complete, it is probably waiting for service authentication: the service authentication dialog may be hidden under other windows.

The analysis master will build and analyze the project, requesting slaves from the hub as required and specifying (due to your REMOTE_ANALYSIS_SLAVES_LAUNCHDS setting) that those slaves should be from the launch daemons you selected.

What Next?

• View the results of an analysis
• View statistics for an analysis

Links

• Task: Set Up and Perform a Remote-Managed Analysis
• Typical CodeSonar Tasks
• CodeSonar GUI Reference