Writing and Debugging CodeSonar Plug-Ins

• Overview
• Warnings
• Metrics
• Visitors
• Parallelizing and Distributing
• Testing and Debugging

Overview

This page provides general information about writing CodeSonar plug-ins. We also provide sections with language-specific details: C++, Python, C.

In general, writing a CodeSonar plug-in will involve the following steps.

1. Create a source file for the plug-in.
2. If your plug-in defines a new warning class or new metric class, create it.
3. Add one or more visitors to perform the main work of the plug-in. This may include handling warnings, metric values, or both.
4. If you intend to use the plug-in with parallel-mode analyses, perform any necessary parallelization work.
5. [C and C++ plug-ins only] Compile the plug-in into a library with a suitable name.
6. Install the plug-in.
7. Test and debug the plug-in.

Warnings

Plug-ins can:

• define new warning classes,
• issue warnings of any class,
• retract warnings,
• obtain information about warning classes.

New Warning Classes

Plug-ins can define new CodeSonar warning classes.

Issuing Warnings

Plug-ins can issue warnings of any class.

Depending on the warning-issuing function used, warnings can be associated with...

• ... the analysis as a whole (that is, no particular file, procedure, or code location).
• ... a specific source file instance.
• ... a code span.
• ... a code span in a specified procedure.
• ... a source line.
• ... a source line in a specified procedure.
• ... a list of code locations.
• ... a list of code locations in a specified procedure.
• ... a program path.
• ... a point (PDG_VERTEX)
• ... a step path.

Warnings issued by a CodeSonar plug-in will be issued by one of the following functions.

Retracting Warnings

For incremental analyses, warning retraction is handled during the drop traversal.

The primary mechanism for warning retraction is based on the retraction info parameter specified when the warning was issued. This parameter specifies PDGs (procedures) and compilation units such that if any of the listed PDGs or compilation units change between incremental analyses, the warning instance will be retracted. Each warning-issuing function also provides a default handling option: see function documentation for details.

If specifying retraction information at reporting time is insufficient for some reason, you can also manually retract warnings.

For more information about how and when your custom checkers should retract warnings, see Custom Checks: Accounting for Incrementality.

Obtaining Information About Warning Classes

The API provides functions for checking properties of warning classes.

• Numerical ID
• Name
• Are WARNING_FILTER settings such that instances of this warning class will always be ignored?

It also provides functionality for retrieving warning classes by ID or name.

Metrics

Plug-ins can:

• define new metric classes,
• report metric values,
• retract metric values,
• obtain information about metric classes and values.

New Metric Classes

Plug-ins can define new metric classes.

Reporting Metric Values

The Plug-In API provides a number of options for reporting metric values. In particular, each metric class defined in the plug-in will have two important properties specified by flag settings:

• automatic vs manual.
  • Automatically-reported metrics are computed and reported automatically by the CodeSonar analysis.
  • Manually-reported metrics must be reported explicitly by the plug-in.

    API Language	Explicitly report metric values with
    C++	Metric class report() method (details).
    Python	Metric class report() method (details).
    C	Granularity-specific csonar_report_metric_*() function (details).

• analysis-phase vs post-analysis-phase.
  • Analysis-phase metrics are computed and reported before the transition to the bottom-up analysis phase.
    Manually-reported analysis-phase metrics must therefore be reported in one of the following visitor types.
    • serial depth-first phase visitors
    • parallel depth-first phase visitors
    • program bottom-up visitors: analysis::add_project_bottom_up_visitor() | @project_bottom_up_visitor | csonar_add_program_bottom_up_visitor()
      (but not other kinds of bottom-up phase visitors)
  • Post-analysis-phase metrics are computed and reported after the transition to the bottom-up analysis phase.
    Manually-reported post-analysis-phase metrics must therefore be reported in bottom-up phase visitors or program bottom-up finish visitors.

Retracting Metric Values

For incremental analyses, metric retraction is handled during the drop traversal.

Both automatic and manual retraction can take place.

• All analysis-granularity metrics are automatically retracted.
• For each compilation unit that has changed (or been removed) between incremental analyses, CodeSonar will automatically retract:
  • all compilation-unit-granularity metric values for the compilation unit,
  • all file-granularity metric values for files in the compilation unit,
  • all procedure-granularity metric values for procedures in the compilation unit.
• Metrics whose computation crosses compilation unit boundaries should be manually retracted: they may not be identified for automatic retraction if only some of the compilation units involved are modified. The retraction functions are shown in the following table.

  API language	Explicitly retract metric values with
  C++	Metric class retract() method (details) .
  Python	Metric class retract() method (details).
  C	Granularity-specific csonar_metric_retract_*() function (details).

Obtaining Information About Metric Classes and Values

The CodeSonar plug-in API provides additional functionality for metric values and metric classes.

• Get the granularity of a metric class.
• Get the tag (short identifier) of a metric class.
• Get the description (longer, human readable description) of a metric class.
• Get the flags associated with a metric class.
• Look up a metric class.
• Get the value of a specified metric for a specified program element.
• List or iterate over all metric classes with a specified granularity.

Visitors

The bulk of plug-in functionality will be applied through one or more visitors, each of which specifies:

• a set of actions, to carry out on
• a specific element type within the CodeSonar internal representation (IR), at
• a specific stage of the CodeSonar build/analysis.

Detailed information is provided in section Visitors. The following tables summarize the key points.

Visitors other than step visitors are added by defining a visitor function and then instructing CodeSonar to add it as a particular kind of visitor. Step visitors involve a slightly different mechanism.

All visitor types except step visitors

Step Visitors

Note: Step visitors will generally not produce useful information when applied to C# and Java analyses, because internal representation for points is not generated for those analyses.

Parallelizing and Distributing

A plug-in requires explicit parallelization work if:

• it maintains global state information (whether through global or static variables, or by utilizing input/output), and
• it defines visitors that are applied during the parallel phase (ie bottom-up visitors or bottom-up finish visitors with PDG or finer granularity), and
• one or more of those visitors modifies the global state information, and
• you intend to use the plug-in with parallel-mode analyses.

In such cases, the plug-in must incorporate inter-process communication mechanisms to ensure that the global state information is correctly shared between multiple slave processes, unless the global state is is only used within a single PDG and its components (and not, for example, to communicate between PDGs or accumulate information across multiple PDGs).

When the analysis is running in parallel mode, the build/analysis traversals (and therefore plug-in processing) proceed as follows.

1. [Incremental build/analyses only] The analysis master performs the drop traversal and calls all drop visitors.
2. The analysis master calls all program setup visitors, then all program visitors.
3. The analysis master performs the serial depth-first traversal, calling serial depth-first phase visitors.
4. The analysis master calls all program finish visitors, then all program parallel visitors.
5. The analysis master performs the parallel depth-first traversal, calling parallel depth-first phase visitors.
6. The analysis master calls all program parallel finish visitors.
7. The analysis master performs the pointer analysis traversal. No visitors are associated with this traversal.
8. The analysis master calls all program bottom-up visitors.
9. [parallel phase] The analysis master manages slave processes, while the slave processes perform the remainder of the bottom up traversal.

   Typically, the unit of work carried out by a slave corresponds to the analysis of a single PDG (procedure). This entails calling all PDG bottom-up visitors, PDG_VERTEX (point) bottom-up visitors, step visitors, and PDG finish visitors on the appropriate elements of the PDG.

10. The analysis master calls all program bottom-up finish visitors.

The plug-in API provides access to the multiprocess mode for the CodeSonar process in which the plug-in is running:

Distributed Analysis Note

A remote-requesting analysis master M will only accept an analysis slave S if every CodeSonar plug-in in the set of plug-ins for M is also in the set of plug-ins for S. (Note that it is acceptable for S to have additional plug-ins beyond this set.) For full details, see Distributed Analysis: Plug-In Note.

Testing and Debugging Plug-Ins

When your plug-in is ready for live testing/debugging, install it as directed in the appropriate language-specific section (C++, Python, C) and analyze a sample project. Note that the plug-in mechanism provides close interaction with the CodeSonar analysis, so bugs in plug-ins can cause the entire analysis to crash. Similarly, terminating plug-in execution will terminate the entire analysis. The following hints may be useful.

• For fast feedback, analyze a small project. It may be useful to create an artificial project that illustrates the issue your plug-in is designed to address.
• Always debug with FOREGROUND=Yes in the general project configuration file.
  • The CodeSonar analysis will run inside the process launched by codesonar analyze, rather than in a separate background process. If you run the analyze command inside a debugger, the full lifetime of the analysis process will be covered.
  • The plug-in stderr and stdout will correspond to the analyze command's stderr/stdout, eliminating the need to check the Analysis Log (where these streams would otherwise be directed).
• To debug parallelized plug-ins that run in analysis slaves, set ANALYSIS_SLAVES=0 in the general project configuration file. When you analyze the project, the analysis master will start, then wait for a manually started slave to connect. At this point you can start an analysis slave from inside your debugger.

  By default, the master will retry any failed unit of work (usually the analysis of a single procedure) three times before giving up and moving on to the next unit of work. This often means that bugs can be reproduced instantly and repeatedly without needing to restart the master: the slave will attempt the unit of work that previously failed immediately upon starting. If you want more retries, change the setting of UNIT_OF_WORK_RETRIES. In particular, you may wish to set UNIT_OF_WORK_RETRIES=-1 so that crashes can be reproduced as many times as you want. The plug-in can even be modified in between retries in order to fix the issue while the analysis is live.

• Your plug-in may need to terminate immediately in some situations. We recommend using cs_fatal() rather than abort() in these cases, because it includes a mechanism for specifying an error message.
• Many C API functions return a cs_result, which in turn is often useful for debugging. Use cs_resolve_error() to obtain a descriptive string corresponding to a particular cs_result value.

More on Plug-Ins

General Information:

Specific API Language: