Building and Analyzing a CodeSonar Project

JavaScript is not currently enabled, but is required for full CodeSonar manual search and browse functionality.

If you are viewing this file in your hub's Web GUI, enable JavaScript in your browser: you will also need it for GUI functionality.

If you opened this file directly from disk, your browser may be directly suppressing JavaScript functionality: certain browsers perform this suppression on local files (but not files delivered by web servers) for security reasons.

If you access the manual through the hub's Web GUI, the functionality will not be suppressed because the hub is a web server.
Alternatively, your browser may allow you to explicitly disable the security setting that suppresses functionality. See the CodeSonar FAQ for more information.

CodeSonar^® 9.2p0

CONFIDENTIAL

CodeSecure Inc

General

Building and Analyzing a CodeSonar Project

Generating CodeSonar results from your software project involves two steps:

building a CodeSonar project that models the contents and structure of your software project, and
analyzing the CodeSonar project and putting the analysis results into a hub database for later investigation in a web browser.

Overview
The CodeSonar-Facing Build: Using Your Normal Software Build
Language Tiers
How to Build and Analyze a Project
Launch Daemon Interaction
Permissions and Analysis Authentication

Overview

The following diagram provides an overview of the build and analysis process. The inputs and mechanisms used in the building step depend on the language or combination of languages used in the underlying project.

diagram: building and analyzing a CodeSonar project

The steps of the build process are as follows.

1. Observe

The user instructs CodeSonar to observe the execution of a specified command. The remaining steps will refer to this command as <observed_command>.

2. Execute

<observed_command> is executed.

3. Recognize

CodeSonar recognizes relevant calls within <observed_command> as it executes.

"Relevant calls" are invocations of any of the following, in any combination.

recognized C and C++ compilers	(on C or C++ source code)
cs-java-scan	on Java bytecode with Java source or Kotlin source
cs-dotnet-scan	on Microsoft C# assembly files
codesonar go_scan.py	on Go source code
codesonar kotlin_scan.py	on Kotlin source code
codesonar python_scan.py	on Python source code
codesonar rust_scan.py	on Rust source code
codesonar import_sarif.py	on SARIF files (and their associated source files, if they are not already present in the project)
codesonar add_source_files.py	on any other files that you want to include in the project.

Note that all of the information collected for a single CodeSonar build is incorporated into a single project, even if the observed operations create multiple executables.

4. Update

Each time CodeSonar recognizes one of these invocations, the appropriate language-specific front end or importer updates the project definition to incorporate the relevant information.

Once <observed_command> terminates, CodeSonar finishes building the project.

The CodeSonar-Facing Build: Using Your Normal Software Build

While source code is rich in information, it does not provide a complete picture of the software as it will be executed: computers don't execute source code directly, so the effects of the build process must also be taken into account. Basing the CodeSonar analysis on your normal software build means that the analyzed project will reflect the software resulting from that build.

In general, we recommend accomplishing this as follows.

Define an extended variant of your normal software build that performs all the steps of your normal build, then performs any additional steps required to inform CodeSonar of components to be included in the CodeSonar project.
We will refer to this as the CodeSonar-facing build.
Instruct CodeSonar to observe the execution of the CodeSonar-facing build in order to build the CodeSonar project.

CodeSonar accounts for the influence of the build system in different ways for different languages, as shown in the following table. The table is organized by language tier: see Language Tiers, below, for more information.

Language

The CodeSonar-facing build must include...

CodeSonar will...

Tier 1: CodeSonar has a language-specific front end and produces highly detailed IR (C, C++)

C and C++

... all C and C++ compilations that take place in the normal software build.

Note: for projects that contain only C and C++ code, it is sufficient to have CodeSonar observe your normal software build directly. You don't need a separate CodeSonar-facing build.

...recognize C and C++ compiler calls and update the CodeSonar project to incorporate the C/C++ source files compiled in those calls, with compiler-specific behavior and the effects of compiler flags taken into account.

Tier 2: CodeSonar has a language-specific front end and produces less-detailed IR (Java, C#)

Java

...one or more cs-java-scan invocations, identifying the Java bytecode files produced by your normal build, and the Java source files they were built from.

These invocations must necessarily take place after the corresponding Java compilations have been performed.
CodeSonar does not need to directly observe the Java compilations performed by your normal build, but it is not harmful for it to do so.

...update the CodeSonar project to incorporate the Java source and bytecode files identified in those invocations.

Incorporating the Java bytecode means that the CodeSonar project directly reflects what will be executed by the JVM.

...one or more codesonar cs-dotnet-scan invocations, identifying the assembly (.exe, .dll) files produced by your normal build, and the C# source (.cs) files they were built from.

These invocations must necessarily take place after the corresponding C# compilations have been performed.
CodeSonar does not need to directly observe the C# compilations performed by your normal build, but it is not harmful for it to do so.

The CodeSonar C# build/analysis can be applied to managed code only.

...update the CodeSonar project to incorporate the C# source and assembly files identified in those invocations.

Incorporating the Microsoft C# assembly means that the CodeSonar project directly reflects what will be executed.

Note that CodeSonar C# projects can be built and analyzed on Windows machines only, although results can be submitted to a hub on any platform.

Kotlin
when targeting the JVM

..one or more cs-java-scan invocations, identifying the Java bytecode files produced by your normal build, and the Kotlin source files they were built from.

Note that Kotlin code only receives tier 2 treatment when it is analyzed with cs-java-scan.
Kotlin code analyzed with codesonar kotlin_scan.py receives tier 3 treatment only.

...update the CodeSonar project to incorporate the Kotlin source and Java bytecode files identified in those invocations.

Tier 3: CodeSonar has no language-specific front end, and produces file-level IR only (all other languages)

Go, Kotlin, Python, Rust, JavaScript, TypeScript

...one or more invocations of the corresponding tier 3 subcommand, identifying the source files of interest.

The tier 3 subcommands are wrappers that handle both invoking a third party analysis tool and importing the resulting SARIF into the CodeSonar project along with the specified source files. The following table summarizes the available tier 3 subcommands.

Source language	Tier 3 subcommand	Third party analyzer
Go	codesonar go_scan.py	Staticcheck
JavaScript/TypeScript	codesonar es_scan.py	ESLint
Kotlin^note	codesonar kotlin_scan.py	detekt
Python	codesonar python_scan.py	Pylint
Rust	codesonar rust_scan.py	Rust Clippy

If you have additional third party analysis results: invoke codesonar import_sarif.py to identify the additional analysis results (as SARIF files) after you have invoked the tier 3 subcommand. You do not need to identify the source files a second time in this case. This approach is useful if you want to run multiple third party analysis tools on the same source code.
If you do not wish to run the third party analysis tool associated with a tier 3 subcommand: use one of the general techniques for tier 3 languages.

Important: tier 3 subcommands are not language-specific front ends. The source and SARIF files are imported using the same functionality that is used when you invoke codesonar import_sarif.py directly.

Kotlin note. Kotlin code receives tier 3 treatment when analyzed with codesonar kotlin_scan.py, but receives tier 2 treatment when analyzed with cs-java-scan.

...update the CodeSonar project to incorporate the files identified in those invocations.

Any language not in tier 1 or tier 2
(including Go, Kotlin, Python, Rust, JavaScript, TypeScript)

If there is an associated tier 3 subcommand that is appropriate for your needs: one or more invocations of that subcommand. If you have additional third party analysis results (for example, because you have previously run a different third party analysis tool), invoke codesonar import_sarif.py to identify the additional analysis results (as SARIF files) after you have invoked the tier 3 subcommand. You do not need to identify the source files a second time in this case.
Otherwise, if you already have third-party analysis results for the component: one or more invocations of codesonar import_sarif.py to identify both the analysis results (as SARIF files) and the associated tier 3 source files.
Otherwise: one or more invocations of codesonar add_source_files.py to identify tier 3 source files.

...update the CodeSonar project to incorporate the files identified in those invocations.

Once your CodeSonar-facing build has accounted for the various components in the regular software project, it can optionally invoke codesonar import_sarif.py one or more times to add results from third-party analysis tools invoked on any of those components.

Language Tiers

In the context of building and analyzing a CodeSonar project, programming languages can be divided into three tiers.

Tier 1	C, C++ CodeSonar has a language-specific front end, and produces highly detailed internal representation (IR).
Tier 2	Java, C# CodeSonar has a language-specific front end, and produces less-detailed IR.
Tier 3	all other languages, including Go, Kotlin, Python, Rust, JavaScript, TypeScript CodeSonar has no language-specific front end, and produces file-level IR only. (Note: tier 3 subcommands such as codesonar python_scan.py are not language-specific front ends.)

Functionality consequences for the different tiers are described in the following table.

Language Tier

Tier 1
(C, C++)

Tier 2
(Java, C#)

Tier 3
(all other languages)

Include component via...

...language-specific mechanism (see table above).

Go	codesonar go_scan.py
JavaScript/TypeScript	codesonar es_scan.py
Kotlin^note	codesonar kotlin_scan.py
Python	codesonar python_scan.py
Rust	codesonar rust_scan.py
otherwise	codesonar import_sarif.py or codesonar add_source_files.py

CodeSonar analysis checks

Applied.

Applied only if they rely solely on file-level information (such as file name).

IR produced

Highly detailed IR, including symbols, CFGs, and ASTs along with all IR for tier 2.

Procedures, cross-references, and file-level IR.

File-level IR only.

Plug-In API

IR can be traversed and inspected by plug-ins.

Plug-ins that make use of IR information other than file-level IR will not be applied to tier 3 components.

Code Coloring and Interaction

Applied.

Not applied.

Extension API

Available (C and C++ only). Extension API directives in the component will be incorporated into the CodeSonar project.

Not available. CodeSonar will not recognize or interpret Extension API directives that are present in tier 2 components.

Not available. CodeSonar will not recognize or interpret Extension API directives that are present in tier 3 components.

Note on using codesonar import_sarif.py / codesonar add_source_files.py

When you use codesonar import_sarif.py or codesonar add_source_files.py to add source files (or files of other types) to a CodeSonar project, those files are treated as tier 3 components. This is the case even if a given file is actually in a language that is in tier 1 or tier 2.

If a project component uses a tier 1 or tier 2 language, we therefore strongly recommend using the standard CodeSonar mechanism for adding information about that component to the CodeSonar project. This allows you to take advantage of all the functionality that depends on the resulting internal representation (IR).

How to Build and Analyze a Project

The following sections explain how to build and analyze a project.

General	Building and Analyzing a Project (Command Line) Full documentation: Build/Analysis Commands. Troubleshooting the Build
Language-Specific	C and C++: Build and Analysis for C/C++ Projects Building and Analyzing a Project (Windows Build Wizard) Full documentation: The Windows Build Wizard Java: Build and Analysis for Java Projects and Kotlin Projects Targeting the JVM C# [Windows only] : Build and Analysis for C# Projects Go: Including Go Components in a CodeSonar Project JavaScript/TypeScript: Including JavaScript and TypeScript Components in a CodeSonar Project Kotlin: cs-java-scan : Build and Analysis for Java Projects and Kotlin Projects Targeting the JVM codesonar kotlin_scan.py : Including Kotlin Components in a CodeSonar Project Python: Including Python Components in a CodeSonar Project Rust: Including Rust Components in a CodeSonar Project
Results from other tools	Importing Other File Types to CodeSonar

Launch Daemon Interaction

The CodeSonar analysis interacts with the hub through a launch daemon that

is connected to the same hub, and
has the same Machine, System User, Hub User, and Installation values as the analysis.
(The launch daemon's identifying tuple includes these values along with a key.)

If you have specified an analysis launch daemon with -remote analysis-launchd, it must already be running.

Otherwise, the analysis will use a launch daemon whose Machine, System User, Hub User, and Installation match those that were used to invoke the analysis command. If such a launch daemon is not already running, CodeSonar will start one when you execute the analysis, failing if the Hub User does not have sufficient permissions.

In certain special cases, the analysis does not require a launch daemon. CodeSonar will not start a launch daemon in these cases.

If your analysis command includes the -foreground option, the analysis process does not run as a child of a launch daemon.
If your analysis command includes the -offline option, the analysis process does not interact with the hub at all.

Permissions and Analysis Authentication

CodeSonar permissions apply to the CodeSonar build/analysis as follows.

Note also that a new analysis cannot be started if the hub is in an unlicensed state, unless it is a dry run (DRY_RUN=Yes).

Permissions Required

The permissions required to perform a CodeSonar build/analysis depend on whether or not the project already exists.

To perform a build/analysis on an existing project P:
- PROJECT_ADD_CHILD permission for P.
To perform a CodeSonar build/analysis on a project that does not already exist (thus implicitly creating that project):
- PTREE_ADD_CHILD permission for the project tree under which you are creating the project.

Analysis tokens.
Once the build/analysis is authorized, the authorization persists until the analysis finishes running. To manage this persistent authorization, CodeSonar stores a special analysis token in the analysis directory.

Don't manually delete or modify the contents of the analysis directory while the analysis is running: you may disrupt the analysis token (among other possible ill-effects).
An incremental analysis will only be authorized if the analysis token for its parent analysis is present in the analysis directory.
Once the analysis is authorized, it will remain authorized even if the password or permissions of the authorizing user are changed.

Permissions Assigned

The permissions assigned when you perform a CodeSonar build/analysis depend on whether or not the project already exists.

If a user performs a new analysis A on existing project P, then:
- All ANALYSIS_* role-permissions that (implicitly or explicitly) apply to P will implicitly apply to A.
If a user performs a new analysis A on a new project P that is directly contained in project tree T, then:
- All PROJECT_* and ANALYSIS_* role-permissions that (implicitly or explicitly) apply to T will implicitly apply to P.
- All ANALYSIS_* role-permissions that (implicitly or explicitly) apply to T will implicitly apply to A.

There are initially no other role-permissions applied to the new elements. To adjust role-permissions for an element, use the corresponding Resource Role-Permissions page.

To report problems with this documentation, please visit https://support.codesecure.com/.