Custom Checks: Accounting for Incrementality

• Overview
• Determining Retraction Conditions
• Selecting a Retraction Strategy
• Default Automated Retraction
• Customized Automated Retraction
• Manual Retraction
• Accounting for Incrementality when Reporting Warnings
• More on Plug-Ins

Overview

If you haven't already read the manual page on incrementality it may be useful to do so before proceeding: it provides context for the material on this page.

When you implement a custom warning class, an important goal is for all of the following to be true after an incremental analysis.

• Previously-existing warnings that no longer apply are no longer reported.
  All warnings that should be retracted are retracted.
• There are no duplicate warnings.
  There aren't any cases where a warning is kept from the previous analysis (rather than retracted) and re-reported by the incremental analysis.
• There are no missing warnings.
  There aren't any cases where the incremental analysis should report a warning but does not do so.

To satisfy this goal, the warning class plug-in must do two things.

• Utilize one of the available mechanisms to retract warnings that an incremental build may have caused to become invalid.
• Ensure that the warning class implementation is capable of reissuing any warning that was retracted but whose conditions are still satisfied in the incremental project, and also capable of issuing any new warnings that have arisen.

The remainder of this page discusses the decisions and requirements involved in implementing a custom check that behaves as desired in incremental analyses. While it is generally most straightforward to consider these issues when initially designing the check, the same principles can be applied to modify existing plug-ins. The page is organized as follows.

• Determining Retraction Conditions outlines the general principles involved in deciding when warnings of your custom class should be retracted.
• Selecting a retraction strategy provides a checklist for determining which retraction strategy is best for your custom class.
• Each available retraction strategy is described in its own section.
  • Default Automated Retraction
  • Customized Automated Retraction
  • Manual Retraction
• Accounting for Incrementality when Reporting Warnings discusses the ways in which incremental considerations can affect warning reporting.

Determining Retraction Conditions

When you are designing a check for a custom warning class, think about which incremental program changes should cause a warning of this class to be retracted. There are three important factors to bear in mind.

• Retraction conditions are those that could cause the warning to become invalid. In particular, they do not necessarily always cause a warning to become invalid. Provided that retraction conditions correctly CodeSonar will re-analyze the modified program elements after the retraction takes place, issuing new instances of previously-seen warnings that are still present along with any new warnings that have manifested.

  For example, suppose your custom check issues a warning for every source file instance whose line count exceeds some limit. Any incremental program change in which a source file is modified could cause an existing warning to become invalid, because the modification may have removed enough lines that the file length is now inside the limit. It is therefore appropriate to retract a warning of this class if the corresponding file has changed, while making sure that your checker will issue a new warning in the analysis phase if the file length is still outside the limit.

• There is no one-size-fits-all answer. Retraction conditions may be straightforward and easily described or they may be relatively complex, depending on the details of your checker and the information it depends on. In general, the factors used to determine warning retraction will parallel those used to issue the warning.
• Neither overapproximating nor underapproximating the conditions for warning retraction is advisable.
  • Overapproximating can lead to lost warnings.
    For example, if warnings of your custom class are issued by a procedure visitor and depend only on conditions within a single procedure, the most appropriate retraction condition for a warning issued in procedure P is a change to P. It is not advisable to overapproximate by retracting the warning every time there is an incremental analysis: if there are no changes in the compilation unit containing P then the incremental analysis will not apply procedure visitors to P and so the warning will not be reissued.
  • Underapproximating can lead to duplicate warnings.
    For example, suppose you have a custom warning class where warnings depend on conditions within multiple procedures. The most appropriate retraction condition for a warning of this class is a change to any of the relevant procedures. It is not advisable to underapproximate by retracting the warning only if the procedure P containing the warning endpoint changes. To understand why, consider the case where P is unchanged in an incremental analysis but one of the other relevant procedures changes in such a way that the warning is issued again. Because P was not changed, the previous warning was not retracted, so now the analysis has duplicate instances of the warning: one instance corresponding to unretracted warning from the parent analysis, plus one instance issued by the current analysis.
  The information about visitors in incremental analyses may be useful.

Selecting a Retraction Strategy

Once you have determined the conditions under which a warning should be retracted, work through the following checklist to determine how to proceed.

1. Is this a "global warning" (that is, one that applies to the analysis as a whole rather than to a specific file, procedure, or other program location)?
   • YES: Global warnings are always retracted in incremental analyses. There are no retraction considerations or requirements in this case.
   • NO: Go on to the next step.
2. Check the default automated retraction behavior for the warning reporting function you are using. Is this behavior suitable for your warnings?
   • YES: Your plug-in should use the appropriate mechanism to specify default automated retraction behavior every time a warning is reported.
   • NO: Go on to the next step.
3. If the default automated retraction behavior isn't appropriate, determine whether you can fully describe the required retraction behavior with a retraction info value.
   • YES: Your plug-in should construct a suitable retraction info value and pass it to the warning reporting function every time a warning is reported in order to take advantage of customized automated retraction behavior.
   • NO: Go on to the next step.
4. If neither form of automated retraction behavior is suitable for your retraction conditions, your plug-in will need to manually retract warnings.

Note: If warnings of your custom class are issued under complicated conditions, the considerations involved in implementing suitable retraction behavior may lead to new insights about the analysis phase behavior required of your checker. See below for an example.

Default Automated Retraction

The CodeSonar API provides a wide range of warning reporting functionality covering the various kinds of structural information that might be associated with a warning. For example:

• If you create a custom check that issues a warning if a source file contains too many lines, report each warning in association with the source file as a whole.
• If you create a custom check that issues a warning for every occurrence of a forbidden operation, report each warning in association with a single source location (the location at which the operation occurs).

In the C++ and Python APIs, this functionality range is provided by overloading the warning class report() and report_return_warning() methods. In the C API, it is provided by a set of functions with names of the form csonar_report*_warning(). For more information, see Writing and Debugging CodeSonar Plug-Ins: Issuing Warnings.

Each warning reporting function (or overload) has a default retraction behavior, and provides a mechanism for specifying that this default should be applied. The default behavior depends on the individual function or overload: consult the function documentation for details. There is one exception: warnings associated with the analysis as a whole ("global warnings") are always retracted in incremental analyses.

For example, consider the functions that report a warning at a single program point P. The default warning retraction behavior for a warning issued with one of these functions is to retract it if and only if the procedure containing P is modified between incremental analyses. This is often going to be exactly what you want, because if the warning were affected by parts of the program beyond the immediately-containing procedure you would probably be using a different reporting function: one that allows you to describe all the relevant program locations.

Using Default Automated Retraction

If the default behavior for the function you are using is appropriate for your check, you don't need to construct explicit retraction conditions when you report a warning. Instead, use the appropriate mechanism for specifying that the default retraction behavior should be applied.

• C++ and Python: use the overload variant without an explicit warning_retraction_info parameter
• C: all warning reporting functions take a parameter of type cs_warning_retraction_info_t*. To indicate that default retraction behavior should be used, specify value NULL for this parameter.

Customized Automated Retraction

All implementations of the CodeSonar API provide a retraction info data type, such that a value of this type characterizes the conditions for automatically retracting a single warning instance.

Considered abstractly, retraction info always consists of three sets bu_procs, changed_procs, changed_compunits, such that CodeSonar will automatically retract the corresponding warning instance in an incremental analysis A (with parent analysis Aparent) if any of the following are true.

• Any procedure in bu_procs is visited during the bottom-up traversal phase in A. Note that retraction always occurs in the drop phase: the bottom-up phase has not yet occurred at this point, but the set of procedures to be visited in the bottom-up phase is known.
• Any procedure in changed_procs has been modified between Aparent and A.
• Any compilation unit in changed_compunits has been modified between Aparent and A.

To determine which of bu_procs and changed_procs is the appropriate set to use in specifying relevant procedures for retraction, the most important factor will be the kind of visitor that issues the warning.

• If the warning is issued in a bottom-up visitor, bu_procs is generally the correct set.
• Otherwise, changed_procs is generally the correct set.

Before opting to use customized automated retraction, make sure that it cannot lead to false negatives for your check. These can arise if your specified retraction info values cause a warning to be retracted but, due to the incremental nature of the analysis, the warning is not subsequently reissued even though its conditions are met. (For more details, see Accounting for Incrementality when Reporting Warnings, below.) If you can't coordinate the conditions for automated retraction and those for reissuing the warning if it's still present, use manual retraction instead.

Using Customized Automated Retraction

In order to take advantage of customized automated retraction, your plug-in must do the following.

• Report warnings with functions that take a retraction info parameter.
  • C++ and Python: if the plug-in is not already using the overload variant with an explicit warning_retraction_info parameter, change the implementation so that it does.
  • C: all warning reporting functions take a parameter of type cs_warning_retraction_info_t*, so you will not need to switch to calling a different function.
• Provide a suitable retraction info value every time it reports a warning. This will generally entail accumulating values into a retraction info object as the check progresses, then passing the object as an argument to the warning reporting function if all conditions are satisfied and a warning is reported.

Manual Retraction

If neither default automated retraction nor customized automated retraction is sufficient to express the circumstances under which a warning of your custom class should be retracted, you will need to define a drop phase visitor that checks the necessary retraction conditions for each warning of that class that is present in the parent analysis, explicitly retracting those for which the conditions are met.

This mechanism has two important consequences.

• For each analysis, your checker must store information about the warnings it has issued.
  At minimum you will need the warning object itself so that it can be retracted if necessary. See Storing Retraction Information for details.
• Depending on the complexity of the retraction conditions and the type of visitor you use to perform retraction, you may also need to adjust the

Storing Retraction Information

Any plug-in that performs explicit warning retraction will need to store information about each warning that may be retracted.

• You will always need the warning object itself (in the C API, the cs_warning_id plays this role) in order to invoke the retraction function.
• The nature of any additional required information will depend on the retraction conditions to be checked. For example, you might need to save the list of procedures to check for changes.

This information must be stored using some mechanism that persists between analyses. In most cases, it will be sufficient to use a dedicated file, as follows.

• A drop visitor in the plug-in looks for the dedicated file.
  • If the file exists, the visitor reads the information from the file, checks the required retraction conditions, and explicitly retracts any warnings for which conditions are met. The visitor then removes from the file any entries describing warnings that have been retracted. If these entries include information that is also used to ensure the relevant parts of the program are re-checked, the visitor records this information for the checker to use during the analysis phase.
  • If the file does not exist there is nothing to do in the drop phase.
• During the analysis phase, the plug-in adds all relevant information to the file each time it issues a warning.
  Note that the file is extended rather than rewritten: information about unretracted warnings from the parent analysis should stay in the file in case a future analysis needs to retract any of them.

The following API functionality is useful for storing warning information between analyses.

Performing Manual Retraction

To set up your plug-in to perform manual retraction, do the following.

1. Ensure that the plug-in obtains and stores all necessary information every time it reports a warning.
2. Add a drop visitor to the plug-in and implement the retraction logic. This will involve three phases, as follows.
   1. Retrieve the information that the plug-in stored for the parent analysis.
   2. For each warning with information recorded, check whether the conditions for retraction are satisfied.
      Note that the subset of API functions available in drop visitors is severely restricted.
   3. For each warning W to be retracted, call the warning retraction function.

      C++	W.retract()
      Python	W.retract()
      C	csonar_retract_warning(W)

Accounting for Incrementality when Reporting Warnings

As discussed above, determining retraction behavior for your custom warning class will always involve considering your warning reporting conditions and operations, and will sometimes also involve extending the part of your plug-in that reports warnings. At minimum, implementing a retraction strategy will involve the following.

If warnings of your custom class are issued under complicated conditions, the considerations involved in implementing suitable retraction behavior may also lead to new insights about the analysis phase behavior required of your checker.

For example, suppose we have implemented a customer checker prototype using a procedure visitor. Each warning is issued in a particular procedure, under conditions that depend on one or more additional procedures that are not necessarily in the same compilation unit.

Imagine a project with procedures procX and procY in compilation units compunitA and compunitB, respectively, such that:

• A warning warningW is issued in procY due to conditions involving both procX and procY.
• No warning is issued in procX.

Because this warning depends on both procX and procY, we determine that it should be retracted if either of these procedures changes. This condition can be expressed in a warning retraction info data object, so we can take advantage of customized automated retraction.

However, we also consider the conditions under which the plug-in will be able to reissue warningW in an incremental analysis.

• If both procX and procY are modified then the incremental analysis will apply all procedure visitors to both procX and procY and the warning will be reissued as in the base analysis as depicted above.
• If only procY changes, then only procY is revisited in the incremental analysis: warningW is reissued if procX and procY still meet the required conditions.

  

• If only procX changes, then only procX is revisited in the incremental analysis: warningW is not reissued, even if procX and procY still meet the required conditions.

  

In considering retraction and reanalysis behavior, we have identified a problem with using a procedure visitor to implement the check: it can lead to false negatives.

It remains to determine how the check should be adjusted: in particular, what kind of visitor it will need to use. In many complicated cases, the best option is to apply the check through a program visitor. These are the only visitors that are guaranteed to always be applied in an incremental analysis, independent of exactly which parts of the program were modified. The general approach is as follows.

1. Add a program visitor to the plug-in.
2. Add code to the program visitor so that it checks the analysis mode and applies your custom check to every element of the relevant type (for example, every procedure) if it is a base analysis or nonincremental analysis:

   C++	analysis::get_mode returns analysis_mode::NORMAL
   Python	analysis.get_mode returns analysis_mode.NORMAL
   C	csonar_get_analysis_mode() returns csam_normal

3. Determine which elements of the relevant type (for example, procedures) your program visitor will need to check in incremental analyses.
   Will it be possible for the visitor to identify the relevant subset of elements for incremental checking?
   • YES: The incremental checks can be applied to this relevant subset only.
     1. Add an else clause to the analysis mode test you added in step 2.
     2. Inside the else clause, apply the necessary test or tests to identify the relevant subset of elements for incremental checking.
     3. [Still in inside the else clause] For each element in the identified subset, apply your custom check.
   • NO: The incremental checks will need to cover every element in the program.
     1. Remove the analysis mode test from the code you added in step 2 (keep all the other added code).
        The visitor will check every element of the appropriate type in every analysis, regardless of whether or not the analysis is incremental.

More on Plug-Ins

General Information:

Specific API Language: