JavaScript is not currently enabled, but is required for full CodeSonar manual search and browse functionality.

If you are viewing this file in your hub's Web GUI, enable JavaScript in your browser: you will also need it for GUI functionality.

If you opened this file directly from disk, your browser may be directly suppressing JavaScript functionality: certain browsers perform this suppression on local files (but not files delivered by web servers) for security reasons.

CodeSonar® 9.2p0 CONFIDENTIAL CodeSecure Inc
C and C++


Authoring CodeSonar Compiler Models in C++

You can author custom compiler models in C++ using the headers provided with CodeSonar.



Overview

You can author custom compiler models in C++ using the headers provided with CodeSonar.

These custom compiler models are implemented as C++ classes that subclass one of the provided compiler model classes. The remainder of this page describes the requirements for implementing, registering, building, and installing a compiler model authored in C++. It also includes an illustrated example model.

Before you start

Before you start implementing a compiler model:

  1. Make sure you have read the Authoring Compiler Models section. In particular, you should have...
  2. Choose a name for your compiler model. The remainder of these instructions will refer to this name as cmname. The model name can be the same as the compiler executable name (without file extension), but does not have to be.

B. Implement the Model

Compiler models are implemented as C++ classes. A custom C/C++ compiler model class must...

An illustrated example compiler model is shown below; you may also wish to examine the source code for the built-in models.

Compiler Model Headers

Various cscm_*.hpp headers are provided in $CSONAR/csurf/include. These fall into two categories.

C. Build and Install

Once you have your model source code, you will compile it into a binary and make it available to the CodeSonar installation.

The following requirements apply.

Build and Installation Steps

The build and installation steps are as follows.

  1. Compile your model source file, observing the requirements described above.
    For example, for a compiler model named cmname implemented in cmname_src.cpp, you could use one of the following.
    Windows
    with cl
    cl /c /Focname.obj cname_src.cpp /DWIN32 /MD /nologo /Z7 /EHsc /EHsc
    /I %CSONAR%\csurf\include /I <boost_inst_path>

    link /nologo /MANIFEST /MANIFESTUAC:NO /INCREMENTAL:NO /dll /out:libcname.dll
    /implib:libcname.lib /LIBPATH:%CSONAR%\csurf\lib
    <boost_regex_path> <boost_regex_flag> compmodel_util.lib /PDB:libcname.pdb /DEBUG cname.obj
    Other systems
    with gcc
    gcc -shared -fPIC cname_src.cpp -o lib$cname.so -lcompmodel_util
    <boost_regex_flag> -static-libgcc -L $CSONAR/csurf/lib
    -L <boost_regex_path> -I $CSONAR/csurf/include
    -I <boost_inst_path> -pthread -fno-exceptions
    Where
  2. Copy the resulting .so or .dll file to $CSONAR/csurf/lib/:
    Windows
    cp libcmname.dll %CSONAR%\csurf\lib\
    Otherwise
    cp libcmname.so $CSONAR/csurf/lib/
    where $CSONAR is the CodeSonar installation directory.

    This naming scheme and location is sufficient to have CodeSonar recognize that the file contains a compiler model named cmname.

  3. Set up one or more COMPILER_MODELS rules to map the relevant native compiler executables to cmname.
    For example: COMPILER_MODELS += mycompiler -> cmname

Configuration Templates and Testing

Follow the instructions in section Authoring Compiler Models to:

  1. Create configuration templates (if necessary).
    The configuration template names will be of the form cmname.L.S.conf or cmname.A.L.S.conf for language L, target size S, ABI key A.
  2. Test your compiler model.

Example Model

Suppose we wish to write a model for the (fictitious) sc2000 compiler, and that we have determined that the following properties and behaviors need to be accounted for in the model.

sc2000...

We choose samplecm as the compiler model name.

Example Files

The example uses the following files.

samplecm.cpp The compiler model source file.
makefile A makefile for building samplecm.cpp (use with make).
samplecm_build.bat A Windows batch file for building samplecm.cpp.

Save copies of these files to a working directory.

samplecm.cpp


/* samplecm.cpp
 *
 * This file implements an CodeSonar compiler model for the
 * (fictional) sc2000 compiler.
 *
 * For more information, see the "Authoring Compiler Models" manual section.
 */

/* Defines class cscm::C_CompilerModel. */
#include "cscm_libcm.hpp"

/* Defines some utility functions for argument matching. */ 
#include "cscm_argument_matcher.hpp"

#include <map>
#include <fstream>


namespace cscm{


  /* All compiler models must subclass cscm::CompilerModel.
   * The name of the class is not important, except that it must be
   * used as the argument to  register_cm_plugin() later.
   */
class SampleModel : public C_CompilerModel {

private:
    /* The list of options that will always be passed to the front
     * end.
     */
    strings fecmds_always; 

    /* The list of options that will be passed to the front end for
     * all compilations in C++ mode.
     */
    strings fecmds_always_cxx;   

    /* The list of options that will be passed to the front end for
     * all compilations in C mode.
     */
    strings fecmds_always_c;     
    
    /* A list of flags that the compiler recognizes as enabling all warnings
     * (IE, -Wall in gcc). If this vector is empty, the compiler is assumed to
     * enable all warnings by default. */
    strings wall_flags = {"-Wall"};

    /* A list of flags that the compiler recognizes as treating warnings as errors
     * (IE, -Werror in gcc). */
    strings werror_flags = {"-Werror"};

    /* A list of all flags seen on the command line that disable reporting of warnings */
    strings seen_wall_disable_flags = {};

    /* A list of all flags seen on the command line that disable reporting of errors or
     * downgrade errors to warnings. */
    strings seen_werror_disable_flags {};

    boost::regex dotc;      /* Will match C source file names. */ 
    boost::regex dotcxx;    /* Will match C++ source file names. */ 
    
    //The name of the compiler model
    std::string m_name;

public:
    /* See the CompilerModel class for more information about requirements.
     */

    SampleModel(std::string name){
        
        /* Macros always predefined: ALPHA, BETA, GAMMA */
        fecmds_always.push_back("-DALPHA");
        fecmds_always.push_back("-DBETA");
        fecmds_always.push_back("-DGAMMA");
        
        /* Additional macro predefined in C++ mode: BARATHRUM */
        fecmds_always_cxx.push_back("-DBARATHRUM");

        /* Additional macro predefined in C mode: RUBICK */
        fecmds_always_c.push_back("-DRUBICK");

        /* The compiler recognizes file extensions as follows:
         * - .c or .C indicates a C source file.
         * - (case-insensitive).cpp or .cxx indicates a C++ source file.
         */
        dotc = boost::regex("\\.(c|C)$");
        dotcxx = boost::regex("\\.[cC]([pP][pP]|[xX][xX])$");
        
        //Store the name of the compiler model
        m_name = name;
    }

    /* All compiler model classes must implement name(). The returned
       name must be unique in the set of compiler model names.
     * The name of this model is "samplecm".
     */
    virtual std::string name() {
        return m_name;
    }

    /* All compiler model classes must implement verbose_name().
     */
    virtual std::string verbose_name() { 
        return "CodeSecure's Sample Compiler Model Plugin."; 
    }

    /* All compiler model classes must implement this overloaded form
     * of operator().
     *
     * Given information about a single invocation of the sc2000
     * compiler, it creates a CodeSonar front end invocation
     * (or sequence of invocations) that models the sc2000 compilation.
     *
     * arguments is the list of command arguments passed to sc2000 for a
     * particular invocation.
     *
     * config is a ParseConfiguration object that describes various
     * properties of the native sc2000 compiler. Most of these are not
     * directly interesting to the compiler model; exceptions are
     * config.compiler_name, config.compiler_path, and
     * config.verbose.
     * The model will use this object in three locations:
     * - To determine the compiler path, and thus the system include
     *   directory.
     * - As an argument to add_version_predefined_macros(), to help
     *   determine the compiler version, and thus whether macro
     *   __INVISIBLE_TREE__ should be defined.
     * - As an argument to cs_and_fe_options,
     *   to determine basic CodeSonar-specific front end command
     *   components.
     */
    virtual CompilerModelResult operator()(const strings &arguments, 
                                           const ParseConfiguration &config)
    {
        strings args = arguments;
        std::vector<source_file> source_files;
        strings commands;
        strings native_flags,
            native_cmd_line,
            native_all_warnings_flags,
            native_warning_suppression_flags,
            native_warnings_as_errors_flags,
            native_error_suppression_flags;

        /* Compilers may read options from the environment.
           Use cu::get_environment instead of getenv() for reading
           environment variables. Here is an example where sc2000
           compiler doesnt compile anything if DISABLE_SC2000 is set.
        */
        std::string disable_sc2000 = cu::get_environment("DISABLE_SC2000");
        if(disable_sc2000 == "1"){
            return CompilerModelResult();
        }

        /* A typical use case for the config argument: extract the
         * path to the native compiler and use it to construct one or
         * more system include directory names.
         */
        cu::path sincdir = config.get_compiler_path();
        sincdir = sincdir.parent_path() / ".." / "include";

        /*Results from the argument matcher are returned as a pair.
        The first element is a match_result enum, which has one of these values:
            SUCCESS - argument was found
            SUCCESS_USED_ARG2 - argument was found and the next argument was consumed
            MATCH_FAILED - argument was not found
        The second element is a std::string containing the value of that argument. */
        std::pair<match_result, std::string> res;
        
        /* Loop over all arguments for the sc2000 invocation and
         * translate them one by one. For convenience, we use function
         * short_mand() from argument_matcher.hpp to match arguments
         * with mandatory values.
         */
        for(size_t ai=0;ai < args.size();++ai){
            std::string value;
            bool valid_arg2 = args.size() > ai+1;
            bool used_arg2 = false;
            const std::string& arg1 = args[ai];
            const std::string& arg2 = valid_arg2 ? args[ai+1] : std::string("dummy");

            /* Skip the next argument if it was already consumed
             * (because it was the "value" component of a flag with a
             * mandatory value).
             */
            if(res.first == SUCCESS_USED_ARG2) {
                res.first = MATCH_FAILED;
                continue;
            }
            
            /* sc2000 -D, -I, and -U flags behave the same as the
             * front end flags of the same names, so "translation"
             * consists merely of pushing the same flag and value onto
             * the list of front end arguments.
             */
            res = short_mand("-D", arg1, arg2);
            if(res.first != MATCH_FAILED){
                commands.push_back("-D"+res.second);
                native_flags.push_back("-D"+res.second);
                continue;
            }
            res = short_mand("-I",arg1,arg2);
            if(res.first != MATCH_FAILED){
                commands.push_back("-I"+res.second);
                native_flags.push_back("-I"+res.second);
                continue;
            }
            res = short_mand("-U",arg1,arg2);
            if(res.first != MATCH_FAILED){
                commands.push_back("-U"+res.second);
                native_flags.push_back("-U"+res.second);
                continue;
            }

            /* --ENABLE-RTTI is directly equivalent to the front end
             * --rtti flag.
             */
            if(arg1 == "--ENABLE-RTTI"){
                fecmds_always_cxx.push_back("--rtti");
                native_flags.push_back(arg1);
                continue;
            }

            /*  -O causes macro __OPTIMIZE__ to be defined, so
             * translate it into a -D argument.
             */
            if(arg1 == "-O"){
                commands.push_back("-D__OPTIMIZE__");
                native_flags.push_back(arg1);
                continue;
            }

            /* -c and -help have no effect on the compilation.
             */
            if(arg1 == "-c" || arg1 == "-help"){
                //do nothing
                continue;
            }
            
            if(arg1 == "-Wno-warnings"){
                seen_wall_disable_flags.push_back(arg1);
                native_flags.push_back(arg1);
                continue;
            }
            
            if(arg1 == "-Wno-errors"){
                seen_werror_disable_flags.push_back(arg1);
                native_flags.push_back(arg1);
                continue;
            }

            /* If @fname is specified for nonempty fname, read
             * additional arguments out of file fname and append to
             * the argument list for processing.
             */
            if(arg1.size() > 1 && arg1[0] == '@'){
                strings options_from_file = cu::read_response_file(arg1.substr(1),
                                                            false);
                args.insert(args.end(),
                            options_from_file.begin(),
                            options_from_file.end());
                continue;
            }

            /* If an argument matches the "C source file name"
             * pattern, append it to the list of source files and note
             * that it's a C file.
             */
            if(boost::regex_search(arg1,dotc)){
                source_files.push_back(source_file(arg1,csl_edgcp_c));
                continue;
            }
            /* ... and correspondingly for C++ source file names. */
            if( boost::regex_search(arg1,dotcxx)){
                source_files.push_back(source_file(arg1,csl_edgcp_cxx));
                continue;
            }

        }

        /* Some front end arguments are OS-specific. */
        if(OS_WINDOWS){
            commands.push_back("--cs_cygwin_path_translation");
        }

        /* Make sure the standard EDG macros are defined.
           This is typically used if the real compiler is EDG based.
        */
        commands.push_back("--cs_define_edg_macros");
        
        /* Call add_version_predefined_macros() (defined below) to
         * update commands with any macro definitions that depend on
         * the native compiler version.
         */
        add_version_predefined_macros(commands, config);

        /* The bit width is always 32 for this compiler.
         * If you are modeling a compiler for which bit width can vary
         * depending on (for example) certain command arguments, you
         * will need to take that into account.
         */
        const csuint32 bit_width = 32;

        /* Command arguments and strings may be shared across multiple
         * FrontEndCommand objects, so avoid making redundant copies
         * by storing them as CmdArgumentVector objects.
         */

        /* Basic front end arguments, which depend on source language (C or
         * C++), bit width, and ParseConfiguration.
         */
        CmdArgumentVector edg_fe_cmds_c(EDGFrontEnd::cs_and_fe_options(csl_edgcp_c,bit_width,config));
        CmdArgumentVector edg_fe_cmds_cxx(EDGFrontEnd::cs_and_fe_options(csl_edgcp_cxx,bit_width,config));

        /* The model's standard front end argument sets.
         */
        CmdArgumentVector vfecmds_always(fecmds_always);
        CmdArgumentVector vfecmds_always_cxx(fecmds_always_cxx);
        CmdArgumentVector vfecmds_always_c(fecmds_always_c);

        /* The front end arguments that have been translated from the
         * arguments to this particular compilation.
         */
        CmdArgumentVector vcommands(commands);

        /* Using the accumulated information, generate one
         * FrontEndCommand per source file.
         */
        std::vector<FrontEndCommand> fecs;
        std::vector<CmdArgument> cao;
        for(size_t i=0;i<source_files.size();++i){
            source_file& sf = source_files[i];
            CmdArgumentVector& extras = sf.second == csl_edgcp_cxx ? vfecmds_always_cxx : vfecmds_always_c;

            /* Some CodeSonar-specific front end arguments
             * (such as the path to the configuration files in
             * compiler_confs) are independent of the compiler
             * model. Use the EDGFrontEnd class to get those arguments
             * and pass them along. These arguments must be provided
             * before any other front end arguments.  See the EDGFrontEnd
             * class documentation (cscm_compmodels.hpp) for more
             * details.
             */

            /* The values returned by EDGFrontEnd::first_options() must
             * always be the first arguments to the front end.
             */
            const std::vector<CmdArgument>& edg_fe_ffa = EDGFrontEnd::first_options(sf.first,"object_file.o");
            cao.insert(cao.end(),edg_fe_ffa.begin(),edg_fe_ffa.end());

            const CmdArgumentNativeCmdLineFile cso_native_cmd_line_file(native_flags,
                                                                        args, wall_flags, seen_wall_disable_flags,
                                                                        werror_flags, seen_werror_disable_flags );
            std::vector<CmdArgument> _vnative_cmd_line_file;
            _vnative_cmd_line_file.push_back(cso_native_cmd_line_file);
            CmdArgumentVector vnative_cmd_line_file(_vnative_cmd_line_file);

            /* The values returned by edg_fe_cmds_c() or
             * edg_fe_cmds_cxx() (depending on the source file
             * language) must come after the values from
             * first_options() and before any other arguments.
             */
            CmdArgumentVector& edg_fe_commands = sf.second == csl_edgcp_cxx ? edg_fe_cmds_cxx : edg_fe_cmds_c;
            cao.push_back(edg_fe_commands);

           /* Now add in the arguments generated by this compiler model.*/
            cao.push_back(vfecmds_always);
            cao.push_back(extras);
            cao.push_back(vcommands);
            cao.push_back(vnative_cmd_line_file);

           /* Generate the front end command. */
            FrontEndCommand fec(cao,sf.first,sf.second,FET_EDGCP);
            fecs.push_back(fec);
            cao.clear();
        }

        /* Create and return a CompilerModelResult based on the
         * accumulated set of front end commands. */
        CompilerModelResult ret(fecs);

        return ret;
    }

    virtual ~SampleModel(){

    }

private:

    /* Like a number of the compiler models shipped with
     * CodeSonar (for example, the gcc model), this model
     * invokes the native compiler to obtain some of the information
     * it needs.
     *
     * In this case, the model is invoking the native compiler with
     * --version to obtain its version string, then using that version
     * string to decide whether or not to instruct the front end to
     * define version-dependent macro __INVISIBLE_TREE__.
    */
    void add_version_predefined_macros(strings& commands, 
                                       const ParseConfiguration& config){
        cu::SpawnProcess sp(config);
        sp.args.push_back(config.get_compiler_path().str());
        sp.args.push_back("--version");
        sp.redirect_output = true;
        sp.output_file_name = cu::unique_file_name(config.get_csurf_tmp_dir());
        
        sp.spawn_blocking();
        const strings& lines = sp.get_output_text();
        if(sp.get_return_code() == 0 && lines.size() > 0){
            if(lines[0].find("Codename Invisible Tree")){
                commands.push_back("-D__INVISIBLE_TREE__");
            }
        }
    }


}; /* SampleModel */

}  /* cscm namespace */


/* Invoke register_cm_plugin() on the class name. This serves as the
 * entry point from CodeSonar to this model, and must occur after the
 * class definition.
 */
register_cm_plugin(SampleModel)

Build and Install the Example Model

We have provided scripts for building the example model.

  1. Make sure you have the following prerequisites installed.
  2. Set the variables in the build script you will be using. Comments in the scripts explain how to set each variable.
    Windows
    (with Visual Studio 2010 or later)
    Edit your working copy of samplecm_build.bat to set values for BOOST_INST, BOOST_REGEX_SRC_FILES, and BOOST_REGEX_LIB_FLAG.
    Other Systems
    (with make)
    Edit your working copy of makefile to set values for BOOST_INST, BOOST_REGEX_SRC_FILES, BOOST_LIBRARY_PATH, and BOOST_REGEX_LIB_FLAG. Check to make sure that the values of CXX and SOSUFFIX are suitable, and modify them if necessary.

    Note: the makefile is not suitable for use on Windows, including Cygwin: use the provided batch file instead.).

    Otherwise If neither of the above are suitable, construct a build script or build command to build samplecm.cpp into a shared library libsamplecm.ext (with system-appropriate ext), making sure your build incorporates all the requirements described above. You may find it useful to base your command on the ones used by samplecm_build.bat and makefile.
  3. Execute the build script.
    Windows
    (with Visual Studio 2010 or later)
    Open a Visual Studio command prompt, then execute the following command.
    Make sure that the Visual Studio command prompt target has the same pointer width as your CodeSonar installation. (If you are not sure whether you are using 32-bit or 64-bit CodeSonar, look at the "Host Platform" line in $CSONAR/SIGNATURE.txt.)
    samplecm_build.bat
    This will build the compiler model and then copy the resulting libsamplecm.dll to the correct location, so you do not need to perform a separate copying step.
    Other Systems
    (with make)
    make install
    This will build the compiler model and then copy the resulting library file to the correct location, so you do not need to perform a separate copying step.
    Otherwise Execute the command you constructed in the previous step to build samplecm.cpp into a shared library libsamplecm.ext (with system-appropriate ext).
    Once it is built, copy libsamplecm.ext into $CSONAR/csurf/lib/.
    cp libsamplecm.ext $CSONAR/csurf/lib/
  4. The modeled compiler has been specified as handling both C and C++ and targeting a 32-bit architecture, so we need two compiler template configuration files. We can just copy the project-compiler defaults:
    cp $CSONAR/csurf/compiler_confs/unknown.c.32.conf $CSONAR/csurf/compiler_confs/samplecm.c.32.conf
    cp $CSONAR/csurf/compiler_confs/unknown.c++.32.conf $CSONAR/csurf/compiler_confs/samplecm.c++.32.conf
  5. Add the following rule to your general template configuration file.
    Windows
    COMPILER_MODELS += sc2000.exe -> samplecm
    
    Otherwise
    COMPILER_MODELS += sc2000 -> samplecm
    

Test the Example Model

If you want to practise compiler model testing on the samplecm model, you can do something along the following lines.

  1. Since you (presumably) don't have a native sc2000 compiler, make a copy of the null-cc executable to serve this role. Put it in your working directory, or another suitable location that is on your PATH.
    Windows
    cp $CSONAR/codesonar/bin/null-cc.exe sc2000.exe
    Otherwise
    cp $CSONAR/codesonar/bin/null-cc sc2000
  2. Create source files testC.c, testCPP.cxx, testRTTI.cxx to exercise the properties and behaviors noted for the native sc2000 compiler. The test source files should use an assortment of the preprocessor macros defined by the native compiler.
  3. For your first check, see what happens when you test the model on testRTTI.cxx with and without the --ENABLE-RTTI flag.
  4. Keep testing with other build flag/source file combinations until you are satisfied that you have tested the model thoroughly.

Cleaning Up

If you don't want to keep the sample compiler model:

  1. Edit the general template configuration file to remove the COMPILER_MODELS rule you added for samplecm.
  2. Remove the shared libsamplecm library from $CSONAR/csurf/lib/.
  3. Remove your model 'sc2000 executable' from your working directory (or other location where you created it).
  4. Remove $CSONAR/csurf/compiler_confs/samplecm.c.32.conf and $CSONAR/csurf/compiler_confs/samplecm.c++.32.conf.
 

To report problems with this documentation, please visit https://support.codesecure.com/.