Authoring Compiler Models

• Do you need a custom compiler model?
• Overview
• A. Determine Behaviors
• B. Implement Model
• C. Build and Install
• D. Configuration Templates
• E. Test

Do you need a custom compiler model?

In most cases you will not need a custom compiler model. Before reading the rest of this section, check the following to see it that there is a simpler solution for your build.

You do not need a custom compiler model if:

CodeSonar recognizes your compiler and builds the project correctly.

In this case you don't need to do anything special.

Your compiler is based on an existing model but is named differently:

Specify which of the existing models CodeSonar should use, as described in section Mapping Compiler Names to Models.

Even if your compiler is not directly based on any of the existing models, it may be similar enough to an existing model to work with no modification. Therefore, you should try mapping your compiler name to each model in turn to see if the build works before resorting to a custom model. Try the gcc model first: it is the most flexible, versatile, and tolerant.

Your compiler is based on an existing model but you need to prepend or append flags to the command line.

Specify the flags using CFLAGS_PREPEND and CFLAGS_APPEND in the configuration file.

You need a custom compiler model if:

There is no existing compiler model for your compiler.

The available compiler models are listed in section Compiler Models.

Your build system has requirements that are not covered by the compiler defaults.

(And cannot be accommodated by appending command line flags.)

If your project compiles correctly under your normal build system but you get errors when you run the CodeSonar build system, you may need to modify the compiler model.

Overview

The compiler model is responsible for obtaining all relevant information from a native compiler invocation and translating this information to individual compilation units for the CodeSonar front end.

Typical tasks for a compiler model include the following.

• Translating command arguments.
  For example, the cl compiler model translates native compiler argument /J into front end arguments -D_CHAR_UNSIGNED --unsigned_chars.
  

  Some native command arguments have no effect on the compilation and so will not influence the CodeSonar front end command constructed by the compiler model. For example, the argument that controls warning message verbosity can be simply ignored by the compiler model.

• Obtaining preprocessor macros.
  The model is responsible for determining the set of predefined macros for each compilation and passing them to the front end with -D. The contents of this set may be affected by a number of factors, including explicit define/undefine command arguments, the source file language, the compiler version, and so forth.
• Adjusting include paths.
  The model is responsible for determining the set of include paths for each compilation and passing them to the front end with -I. The contents of this set may be affected by a number of factors, including command arguments and the native compiler installation path.

The process for creating a new compiler model can be divided into five steps.

• A. Determine the behaviors that need to be modeled.
• B. Implement the model.
• C. Build and install.
• D. Create configuration templates.
• E. Test the model.

The remainder of this page describes these steps, with links to the sections about authoring compiler models in C++ and authoring compiler models in Python when the details are language-specific.

Note

A. Determine Behaviors

Compiler models need to account for the following. The compiler should have documentation that provides the necssary information.

• The arguments that the compiler takes from the command line.
• The preprocessor macros that it defines. These are usually influenced by the command arguments
• What source files it can compile.
  • C files, C++, or both?
  • What source file extensions does it recognize?
• Language standards it supports, and support for non-standard features.
  (For example, does it support C++11? Variable length arrays in C++?)

You will also need to choose a name for your compiler model. This is the name that you will use in COMPILER_MODELS rules for mapping compiler executables to this model. The model name can be the same as the compiler executable name (without file extension), but does not have to be.

B. Implement the Model

You can implement the model in either C++ or Python. Detailed implementation information is provided in separate sections:

In both cases, compiler models are implemented as classes that must must...

• ... subclass a general compiler model class.
• ... have a class name that is unique among compiler model class names. A useful convention is to use class names of the form CmnameModel, where Cmname is the capitalized form of the compiler model name.
• ... implement the following.
  • name()
  • virtual std::string verbose_name()
  • A method or operator that takes information about a compiler invocation and converts it into one or more invocations of the CodeSonar front end.
    • C++: operator()()
    • Python:__call__()
  • get_prepend_flags()
  • get_append_flags()
• ... be registered.
  • C++: by invoking register_cm_plugin() on the compiler model class name.
  • Python: by invoking register_compiler_model() on an instance of the compiler model class.

Illustrated example compiler models are provided:

• C++ Example Model
• Python Example Model

C. Build (if necessary) and Install

Compiler models implemented in C++ must be built into shared libraries before installation; models implemented in Python are installed directly. In both cases, you will also need to set up one or more COMPILER_MODELS rules to map the relevant native compiler executables to your model.

D. Create Configuration Templates

When you use your new model to build a CodeSonar project, CodeSonar will load a corresponding project-compiler configuration file to control aspects of the process, creating it if necessary.

• If you have provided a corresponding compiler template configuration file for your model, it will be copied to create the project-compiler configuration file.
• Otherwise, the project-compiler configuration file will be copied from the corresponding default template: $CSONAR/csurf/compiler_confs/unknown.L.S.conf or $CSONAR/csurf/compiler_confs/unknown.A.L.S.conf.

The process for creating a new compiler model therefore includes establishing whether or not you need to provide specific compiler template configuration files, and implementing them if necessary.

For a new compiler model called cmname:

1. Evaluate the configuration file needs for cmname.
   1. What languages L can be compiled with the model?
   2. What are the possible target sizes S?
   3. [If applicable] Will different ABI keys A correspond to different configuration needs?
   4. With respect to your answers to the previous questions, what compiler template configuration files cmname[.A].L.S will you need to provide?
   For example, suppose cmname only compiles C++ (not C), and can target both 32-bit and 64-bit architectures, and that you don't intend to use ABI keys to designate specific configuration needs. Then we need two compiler template configuration files.

   C++	cmname.c++.32.conf and cmname.c++.64.conf.
   Python	pya_cmname.c++.32.conf and pya_cmname.c++.64.conf.

2. Once you have determined which language/size/ABI key combinations you need to address, create initial compiler template configuration files by copying the corresponding project-compiler defaults.
   For example:

   C++	cp $CSONAR/csurf/compiler_confs/unknown.c++.32.conf $CSONAR/csurf/compiler_confs/cmname.c++.32.conf cp $CSONAR/csurf/compiler_confs/unknown.c++.64.conf $CSONAR/csurf/compiler_confs/cmname.c++.64.conf
   Python	cp $CSONAR/csurf/compiler_confs/unknown.c++.32.conf $CSONAR/csurf/compiler_confs/pya_cmname.c++.32.conf cp $CSONAR/csurf/compiler_confs/unknown.c++.64.conf $CSONAR/csurf/compiler_confs/pya_cmname.c++.64.conf

3. Inspect your new templates and make any required changes and additions.
   Note that you may need to modify the templates further if testing reveals any issues with your model. In particular, you may wish to specify one or more of the following.
   • front end options (EDG_FRONTEND_OPTIONS_PREPEND, EDG_FRONTEND_OPTIONS_APPEND)
   • source modifications ( SOURCE_PATCH_DIRECTORIES, SOURCE_REPLACE_COMMAND, SOURCE_PATTERN_REPLACEMENT)

E. Test the Model

Suppose you have created a model mycompilermodel for native compiler mycompiler, where the path to the native compiler executable is path/to/compilers/mycomp/bin/mycompiler. To test the model, proceed as follows.

1. Create a sample source file testcm.cpp for testing. The source file should:
   • Have a file extension that is recognized by mycompiler (if .cpp is not recognized, use a different extension).
   • Include one or more system includes, such as <stdio.h>.
   • Use the non-standard features supported by mycompiler.
2. Choose a set of command arguments to test. We will refer to this set as <testflags> in the next step. Note that there may be some flags that you need to include every time. For example, if your compiler uses a flag (such as -c) to indicate a file for compilation, you will always need to include that flag.
3. On a command line, run
   rm -rf CMT*
   then
   codesonar compile -TCMT -CCMT -invoke-compiler no -transparent yes
   -compiler-location /path/to/compilers/mycomp/bin/mycompiler mycompilermodel
   -conf-file myproject.CSURF_COMPILER_CONF_MARKER <testflags> testcm.cpp
4. Examine the output from the codesonar compile command.

   If there are parse errors, then the compiler model is doing something wrong. Use the error messages to determine what constructs are being handled incorrectly, and amend your model (or configuration templates) to address the problems.

5. Repeat from step 2) with a new set of <testflags>, continuing until you are satisfied that you have tested the model thoroughly.