SARIF Importer for CodeSonar

• Overview
• Using the SARIF Importer
• codesonar import_sarif.py
• Typical Use Case
• Example
• SARIF File Requirements

Overview

Static Analysis Results Interchange Format (SARIF) is a format for describing the output of static analysis tools. CodeSonar is capable of importing SARIF files, allowing users to incorporate static analysis results produced by other tools.

The CodeSonar SARIF Importer will load warning information from zero or more SARIF files and add those warnings to the set of analysis results recorded on the hub. The warnings can then be browsed alongside any warnings reported by CodeSonar.

Section SARIF File Requirements (below) describes requirements for file naming, format version, and contents.

For more information about SARIF, including full file format specifications, see Static Analysis Results Interchange Format.

Note

Using the SARIF Importer

The following diagram provides an overview of a CodeSonar build/analysis process that includes imported SARIF results.

As described in Command Line Build/Analysis: Command, the general form of the usual CodeSonar build/analysis command is

To import one or more SARIF files, command must incorporate one or more invocations of codesonar import_sarif.py.

• It will usually be most convenient to accomplish this through your regular build system (see the example below).
  1. Extend your regular build system to invoke codesonar import_sarif.py once the necessary SARIF files have been produced.
  2. Invoke codesonar analyze with the command that executes your regular build.
• However, you can also specify codesonar import_sarif.py directly in your codesonar analyze (or codesonar build) command line.

codesonar import_sarif.py

To add warning information from SARIF files to a project, CodeSonar observes the execution of a command that incorporates one or more invocations of

where:

-include-sources/-exclude-sources Examples

We assume for the sake of these examples that other codesonar import_sarif.py options are not required.

• Import results.sarif with no accompanying source files.
  codesonar import_sarif.py results.sarif
• Import results.sarif along with srcdir/mysource.kt only.
  codesonar import_sarif.py results.sarif \
  -include-sources srcdir/mysource.kt
  We have omitted quoting on the -include-sources argument, since there is nothing for the shell to expand. The command would have the same effect if srcdir/mysource.kt was single-quoted (double-quoted, for Windows cmd).
• Import results.sarif along with all files that have names ending in .kt and are located in a direct subdirectory of srcdir/.
  codesonar import_sarif.py results.sarif \
  -include-sources 'srcdir/*/*.kt'
  Note that the srcdir/*/*.kt pattern is single-quoted ('srcdir/*/*.kt'). This is so that the shell passes it through to codesonar import_sarif.py rather than attempting to expand it. For Windows cmd, use double quotes instead ("srcdir/*/*.kt").
• Import results.sarif along with all files that have names ending in .kt and are located anywhere in the directory tree rooted at srcdir/.
  codesonar import_sarif.py results.sarif \
  -include-sources 'srcdir/**/*.kt'
• Import results.sarif along with all files that have names ending in .kt and are located anywhere in the directory tree rooted at srcdir/, unless they are in the directory tree under srcdir/noinclude/.
  codesonar import_sarif.py results.sarif \
  -include-sources 'srcdir/**/*.kt' -exclude-sources 'srcdir/noinclude/**/*'
  The relative order of -include-sources and -exclude-sources is important: if -exclude-sources were first it would be superseded by the subsequent -include-sources and all .kt files under srcdir/ would be imported.
• Import results.sarif along with all files that are located anywhere in the directory tree rooted at srcdir/, unless they are in the tree under srcdir/noinclude/ and not under a directory named actuallyyes/ in that tree.
  codesonar import_sarif.py results.sarif \
  -include-sources 'srcdir/**/*.kt' -exclude-sources 'srcdir/noinclude/**' \
  -include-sources 'srcdir/noinclude/**/actuallyyes/**/*.kt'

Typical Use Case

In the typical use case, codesonar import_sarif.py is used to incorporate static analysis results produced by other tools into a CodeSonar project.

1. Start with your normal software build.
   In most cases, this will be managed through some build system (such as make) so that you can perform the build by executing a single command instead of performing all compilations and other operations manually.
2. Make sure your normal software build accounts for running the third-party analysis tool and outputting its results in SARIF format.
   1. Run the third-party analysis tool to perform your required analysis.
   2. If the third-party tool output requires any additional processing, perform that processing.
      • If the tool does not produce SARIF output, convert its output to SARIF format.
      • Make sure all SARIF files conform to the SARIF file requirements for the SARIF importer.
      • If any SARIF file contains relative paths, make sure those paths are correct relative to the location of the SARIF file.
3. Define an extended variant of your normal software build that includes commands to accumulate CodeSonar project components. This is your CodeSonar-facing build.
   All entities (for example, procedures and files) named in the SARIF file must be present in the CodeSonar project.

   The required additions will depend on the languages involved in your project.

   Language	Required Additions
   C and C++	none: CodeSonar will recognize C and C++ compiler calls directly and incorporate the source files
   Java	One or more invocations of codesonar cs-java-scan, identifying the Java source files and artifacts to be incorporated.
   C#	Add one or more invocations of codesonar cs-dotnet-scan, identifying the C# source files and artifacts to be incorporated.
   any other language	Identify the files to be incorporated as part of your codesonar import_sarif.py invocation, using the -include-sources and -exclude-sources options.

   For example:

   Build Tool	Regular build command	Example CodeSonar-facing build
   make	make all	Add a new phony target to the Makefile for the CodeSonar-facing build. The name can be anything you choose: you may find it convenient to use csonar_facing or similar. Specify the target recipe to include the required additions as described above. Set dependencies for the target to include all files mentioned in the commands in the recipe. The dependencies should also include any existing Makefile targets for C and C++ components so that the CodeSonar-facing build accounts for the respective compilations. The command to invoke the CodeSonar-facing build will be something like: make csonar_facing
   Windows batch file	cmd /c myProj.bat	Extend the batch file by adding an optional argument along with a block of operations such that the block is executed only if the new argument is present and and has an appropriate value (for example, you might choose csonar_facing). The new instruction block should: include the required additions as described above, and be executed after the existing operations in the batch file. The command to invoke the CodeSonar-facing build will be something like: cmd /c myProj.bat csonar_facing
   (and so forth) For more examples demonstrating how to construct a CodeSonar build/analysis command for various build tools, see Command Line Build/Analysis: Language-Specific Examples and the Basic Tutorial.

4. Update your CodeSonar-facing build to include one or more invocations of codesonar import_sarif.py to add the file or files produced by the third-party analysis tool to the CodeSonar project.
   Make sure this is performed after the commands that CodeSonar will observe to accumulate project components.
5. Perform the CodeSonar build/analysis, observing the entire execution of your extended software build. For example:

   Build Tool	Example CodeSonar-facing build command	Corresponding CodeSonar build/analysis command
   make	make csonar_facing	codesonar analyze myProject make csonar_facing
   Windows batch file	cmd /c myProj.bat csonar_facing	codesonar analyze myProject cmd /c myProj.bat csonar_facing
   in general	<csonar_facing_build_cmd>	codesonar analyze <csonar_facing_build_cmd>

Example

Suppose your software project is written entirely in C, and you have an in-house static analyzer that can produce warnings about issued in C code.

If you use make to manage your regular software build, your Makefile might look something like the following.
(If you use another tool to manage your regular build then the mechanism for expressing the steps and dependencies will be different, but analogous.)

CC = gcc
.PHONY: all clean houseAnalysis

all: Red Blue houseAnalysis

Red: Red.c
        $(CC) -o Red Red.c

Blue: Blue.c
        $(CC) -o Blue Blue.c

houseAnalysis: Red-analysis.sarif Blue-analysis.sarif

# house_analyzer is the in-house analyzer
# In this example the analysis tool looks at both the source code and the compiled binary, 
# but does not observe the compilation process.
Red-analysis.sarif : Red Red.c
        house_analyzer Red Red.c -o Red-analysis.sarif

Blue-analysis.sarif: Blue Blue.c
        house_analyzer Blue Blue.c -o Blue-analysis.sarif

clean:
        rm -f *.o *.exe *~ core

If you build a CodeSonar project based on observing the execution of this Makefile in its original state, CodeSonar will recognize the compiler invocations and incorporate the associated source files into the project. Internal representation (IR) will be produced and the analysis will include checks for all enabled C/C++ warning classes, as for any other C project.

To import the additional analysis results from your in-house static analyzer into the CodeSonar project, do the following.

1. Build and analyze CodeSonar project based on observing the execution of the original Makefile. Assuming your hub is at the default location, this will look something like the following.
   make clean
   codesonar analyze ProjectX make all
   Once the CodeSonar build/analysis is complete, open its hub GUI Analysis page.
   • The Warnings tab will include any warnings issued by CodeSonar for Red.c and Blue.c as compiled for the all target.
   • The Files tab will include entries for for Blue.c and Red.c.
2. Edit the Makefile to add a new csonar_scan target. This target should:
   • depend on the all target, so that all steps of the regular software build are accounted for.
   • have a recipe that invokes codesonar import_sarif.py to import the SARIF files produced by the in-house analyzer.
   The updated example Makefile is shown below, with the new contents highlighted like this.

   CC = gcc
   .PHONY: all clean houseAnalysis csonar_scan
   
   all: Red Blue houseAnalysis
   
   Red: Red.c
           $(CC) -o Red Red.c
   
   Blue: Blue.c
           $(CC) -o Blue Blue.c
   
   houseAnalysis: Red-analysis.sarif Blue-analysis.sarif
   
   Red-analysis.sarif : Red Red.c
           house_analyzer Red Red.c -o Red-analysis.sarif
   
   Blue-analysis.sarif: Blue Blue.c
           house_analyzer Blue Blue.c -o Blue-analysis.sarif
   
   clean:
           rm -f *.o *.exe *~ core
   
   csonar_scan: all
           codesonar import_sarif.py Red-analysis.sarif Blue-analysis.sarif
   
   

   If you are not using make, add these invocations in the appropriate analogous location in your build infrastructure.

   For this example, we are assuming that the house_analyzer tool outputs SARIF that conforms to the SARIF file requirements. If it did not, we would need to modify the csonar_scan recipe to include a processing step and then import the processed SARIF files.

3. Build and analyze a CodeSonar project, using a codesonar analyze command based on building the new csonar_scan target.
   make clean
   codesonar analyze ProjectX make csonar_scan
4. Once the CodeSonar build/analysis is complete, open its hub GUI Analysis page.
   • The Warnings tab will include:
     • All the warnings that were issued for the analysis you did in step 1 before extending the Makefile.
       That is, all issued by the usual CodeSonar analysis Red.c and Blue.c as compiled for the all target.
     • One warning for each result object in Blue-analysis.sarif.
     • One warning for each result object in Red-analysis.sarif.
   • The Files tab will include entries for for Blue.c. and Red.c.

A concrete example for this feature is provided in SARIF Importer for CodeSonar: Example.
For more examples, see Including Tier 3 Components in a CodeSonar Project, and the individual language examples linked from that page.

SARIF File Requirements

SARIF files must meet the following requirements in order to be used with the SARIF importer.

• The file name(s) must have the .sarif extension.
• The format must conform to SARIF version 2.1.0. (While SARIF v2.0.0-csd.2.beta.2018-11-28 is currently also supported, it will be deprecated in a future release, so we strongly recommend using SARIF v2.1.0).
• Each .sarif file may contain more than one run, and each run may contain more than one result.
• The rules defined in each .sarif file describe the warning classes for the warnings that may be reported.
  For each rule object in rules, the importer will determine an associated warning class and create this warning class if it does not already exist.
  • The name property of the rule object is used to construct the warning class name.
    • For SARIF files exported by CodeSonar, warning class names exactly match the name values.
    • For SARIF files produced by certain third party analyzers, CodeSonar provides analyzer-specific SARIF import handling, including customized schemes for producing warning class names from name values. For more information, see the following:
      • detekt (Kotlin)
      • Pylint (Python)
      • Clippy (Rust)
      • Staticcheck (Go)
      • ESLint (JavaScript and TypeScript)
    • For SARIF files from any other source, warning class names exactly match the name values.
  • The id property of the rule object is associated with the warning class and can be used in results.
  Some of the other rule object properties are also used.
• The results defined in each .sarif file describe individual warning instances.
  For each result object in results, the importer will create a warning instance such that:
  • ruleID corresponds to the id property of a rule object, and identifies the warning class.
  • message is reproduced in the warning's warning description box.
  • The last location in either codeFlows or locations is used as the warning location.
    • If both codeFlows and locations are present, the SARIF importer will use codeFlows (and ignore locations).
    • If codeFlows is not present and locations is empty or not present, the warning is imported without location information and is not associated with any source file.