Example/Tutorial: Custom Checking with the Extension API

• Goal
• Files
• Iterator Operations
• Checks
• Incorporating the Check
• Building and Analyzing the Project
• Extension Exercise

Goal

We have an iterator structure and functions, and we want to make sure that programs that use the iterator always do so correctly. In particular, we don't want programs to call my_iterator_GetVal(iter) unless they have checked the validity of iterator iter with my_iterator_IsValid(iter) and iter has not changed since the check.

For example:

if (my_iterator_Next(my_it) && my_iterator_IsValid(my_it))
    x = my_iterator_GetVal(my_it); /* OK: called my_iterator_IsValid() */

if (my_iterator_Next(my_it))
    x = my_iterator_GetVal(my_it); /* BAD: no call to my_iterator_IsValid() */

We use the CodeSonar extension functions and macros and attributes to record for each iterator whether its validity has been checked since it was last changed, and examine this record every time my_iterator_GetVal() is called.

Note

Files

The following files are used.

• Save copies of the tutorial files to a working directory.

The Iterator Operations

Functions representing the standard operations for an iterator are declared in my_iterator.h and implemented in my_iterator.c.

The CodeSonar Checks

File check_iterator.c is reproduced below.

/*
 *      Copyright (c) 2023, an unpublished work by CodeSecure, Inc.
 *                      ALL RIGHTS RESERVED
 *
 *      Copyright (c) 2005-2023, an unpublished work by GrammaTech, Inc.
 *                      ALL RIGHTS RESERVED
 *
 *      This software is furnished under a license and may be used and
 *      copied only in accordance with the terms of such license and the
 *      inclusion of the above copyright notice.  This software or any
 *      other copies thereof may not be provided or otherwise made
 *      available to any other person.  Title to and ownership of the
 *      software is retained by CodeSecure, Inc.
 */

#ifdef __CODESONAR__


/* All files that use the Extension API must include csonar.h and
 * stdlib.h. */
#include <stdlib.h>
#include "csonar.h"
#include "my_iterator.h"


/* Define a new attribute called validitychecked.
 * We will use this to keep track of whether the my_iterator_IsValid
 * check has been performed since the last time the iterator was moved
 * (or initialized).*/

CSONAR_DEFINE_ATTRIBUTE(validitychecked);

it *my_iterator_New(void);

#define __CSURF_MARKER_LIBRARY_FUNCTION__ 1

/* The specially-named csonar_replace_my_iterator_New() will intercept
 * all calls to my_iterator_New (and similarly for all other
 * procedures with names of the form csonar_replace_functionname).
 *
 * Note that the symbol __CSURF_MARKER_LIBRARY_FUNCTION__ is defined
 * (as 1) before the definitions of the csonar_replace_*
 * functions. This marks them as LIBRARY FUNCTIONS: any warnings
 * triggered inside a replacement function will be reported at the
 * appropriate call site to that function, not inside the function
 * itself.*/

it *csonar_replace_my_iterator_New(void){

    /* The intercepting function should call the real my_iterator_New
     * so that the iterator is allocated exactly as it would have been
     * if the original call were not intercepted. Note that calls to
     * my_iterator_New from within the body of
     * csonar_replace_my_iterator_New are not intercepted, unlike
     * calls from anywhere else.*/
    it *i =  my_iterator_New();

    /* Since space for i was only just allocated, we have not yet had the
     * opportunity to call my_iterator_IsValid(i). Therefore, we set
     * the value of the validitychecked attribute of i to 0.*/
    csonar_set_validitychecked(i,0);
    return i;
}

void my_iterator_Init(it *i);
void csonar_replace_my_iterator_Init(it *i){
  csonar_set_validitychecked(i,0);
  my_iterator_Init(i);
}

void my_iterator_First(it *i);
void csonar_replace_my_iterator_First(it *i){
   csonar_set_validitychecked(i,0);
   my_iterator_First(i);
}

int my_iterator_Next(it *i);
int csonar_replace_my_iterator_Next(it *i){
   csonar_set_validitychecked(i,0);
   return my_iterator_Next(i);
}

int my_iterator_IsValid(it *i);
int csonar_replace_my_iterator_IsValid(it *i){
    /* my_iterator_IsValid(i) has been called, so we set the value of
     * the validitychecked attribute of i to 1.*/
    csonar_set_validitychecked(i,1);
    return my_iterator_IsValid(i);
}

int my_iterator_GetVal(it *i);
int csonar_replace_my_iterator_GetVal(it *i){
    /* Triggers a "validity not checked before GetVal" warning if the
     * current value of the validitychecked attribute of i is not 1,
     * since that would mean that i has not been checked with
     * my_iterator_IsValid since the last time the iterator was moved
     * (or initialized).*/
    csonar_trigger_m( csonar_get_validitychecked(i), "==", 0, 
                    "validity not checked before GetVal", "CI:1;GT_TESTING", csws_reliability );
    return my_iterator_GetVal(i);
}

void my_iterator_SetVal(it *i, int val);
void csonar_replace_my_iterator_SetVal(it *i, int val){
   my_iterator_SetVal(i,val);
}

#endif

Incorporating the Check

We want to compile check_iterator.c and include it in the CodeSonar project.

1. Identify the hub you will use; start the hub if it is not already running.
2. Determine the appropriate values for the following elements, which will be required to customize your build commands.

   host:port	the hub location.
   libmodels_path	the full path to the codesonar/libmodels subdirectory of your CodeSonar installation.

3. From a command line in your working directory, use codesonar build to observe the compilation of check_iterator.c and add a corresponding CodeSonar representation to the test_iterator project.

   With gcc:	codesonar build test_iterator host:port \ gcc "-Ilibmodels_path" -c check_iterator.c
   With cl:	codesonar build test_iterator host:port ^ cl "/Ilibmodels_path" /c check_iterator.c
   Otherwise:	Use a command of the form: codesonar build test_iterator host:port \ compiler_command include_libmodels options check_iterator.c where: host:port is the hub location. compiler_command is the command for invoking your compiler. include_libmodels is the compiler option setting required to include the codesonar/libmodels subdirectory of your CodeSonar installation. options are any other compilation options that might be required.

   For example, with gcc, a hub located at alex:7340, and CodeSonar installed at /programs/CodeSecure/CodeSonar:
   codesonar build test_iterator alex:7340 \
   gcc -I/programs/CodeSecure/CodeSonar/codesonar/libmodels -c check_iterator.c
   For more example command lines, see Implementing and Including Custom Checks (if you use any of those examples directly, remember to change the name of the compiled source file to check_iterator.c).

Building and Analyzing the Project

If you are not already familiar with the CodeSonar build and analysis steps, you may wish to refer the following sections.

• Building and Analyzing a CodeSonar Project
• The CodeSonar Hub.

Now build the project and run the CodeSonar analysis.

1. Use codesonar analyze to build the remainder of the project and then analyze the entire project.

   With make:	codesonar analyze test_iterator \ [host:port] make test_iterator If you are using CodeSonar SaaS, specify -remote "/saas/*": codesonar analyze test_iterator \ -remote "/saas/*" [host:port] make test_iterator
   With nmake:	codesonar analyze test_iterator ^ [host:port] nmake /Ftest_iterator.mak If you are using CodeSonar SaaS, specify -remote "/saas/*": codesonar analyze test_iterator ^ -remote "/saas/*" [host:port] nmake /Ftest_iterator.mak
   With Microsoft Visual Studio:	Open test_iterator.sln. Your Visual Studio installation will display messages if it does not have the SDK (or "target platform") version or platform toolset specified in the example project. If you don't encounter such messages, go on to setting up the CodeSonar build/analysis in the Windows build wizard. Otherwise, use the Visual Studio GUI to modify these settings before proceeding. Right click on the test_iterator project in the Solution Explorer, and select Properties from the menu that pops up. Navigate to Configuration Properties > General. Make sure the following are set to suitable values. The Visual Studio GUI will provide a menu of available settings. Target Platform Version / Windows SDK Version (name will depend on your Visual Studio installation) Platform Toolset Start up the CodeSonar build wizard and work through the steps to build a CodeSonar project based on the Visual Studio build of test_iterator.
   Otherwise:	Build and analyze a CodeSonar project from test_iterator.c and check_iterator.c, using your normal build environment and the instructions for building a CodeSonar project under UNIX/Linux or Windows.

2. View the analysis results. There should be a warning of class "validity not checked before GetVal".
3. Examine the warning reports. Notice that the warning class name and problem message come from strings specified in check_iterator.c.

Extension Exercise

In addition to the validity check, an initialization check would be useful. Programmers shouldn't call the following functions unless the iterator has been initialized with my_iterator_Init:

• my_iterator_First
• my_iterator_Next
• my_iterator_IsValid
• imy_iterator_GetVal
• my_iterator_SetVal

CodeSonar will report an Uninitialized Variable warning if a program tries to access a variable that has not been initialized, but we actually want a stronger check. The Uninitialized Variable warning will not be issued if a program initializes iterator fields directly instead of calling my_iterator_Init, but this is still undesirable behavior that we want to report.

Add a check for calls to my_iterator_Init to check_iterator.c and test it:

1. Define and name values for a new attribute called (something like) iter_init. (Don't call it initialized: CodeSonar already uses that attribute name for its own checks.)
2. Decide which csonar_replace_functionname function(s) should set the value of iter_init, and add new lines to check_iterator.c accordingly.
3. Decide which csonar_replace_functionname function(s) should trigger a warning based on the value of iter_init, and add new lines to check_iterator.c accordingly.
4. Build the new version of check_iterator.c into the project, using the same codesonar build command line that you used previously.
5. Add new code to test_iterator.c to test your new check. For example, you could try replacing the call to my_iterator_Init(my_it); with the lines my_it->ptr = 0; my_it->used = -1;
6. Re-analyze the CodeSonar project.

While attributes provide a powerful mechanism for keeping track of possible warning conditions, they do have limitations. In particular, because attribute att is not defined for memory location p unless csonar_set_att(p,_) has been called at least once, it can be very difficult to check whether a particular procedure has been called unless we know of an earlier procedure that must already have been called. We made use of this in our check for my_iterator_IsValid by setting validitychecked to 0 in my_iterator_Init. You will have done something similar in your check for correct initialization. But how do we check that the iterator was allocated with my_iterator_New rather than a direct call to malloc (for example)? The answer is that we can't, and this is an important limitation to keep in mind.