Importing Other File Types to CodeSonar

JavaScript is not currently enabled, but is required for full CodeSonar manual search and browse functionality.

If you are viewing this file in your hub's Web GUI, enable JavaScript in your browser: you will also need it for GUI functionality.

If you opened this file directly from disk, your browser may be directly suppressing JavaScript functionality: certain browsers perform this suppression on local files (but not files delivered by web servers) for security reasons.

If you access the manual through the hub's Web GUI, the functionality will not be suppressed because the hub is a web server.
Alternatively, your browser may allow you to explicitly disable the security setting that suppresses functionality. See the CodeSonar FAQ for more information.

CodeSonar^® 9.2p0

CONFIDENTIAL

CodeSecure Inc

General

Importing Other File Types to CodeSonar

There are two main use cases for direct file import.

To include a file or set of files in your project for completeness or documentation purposes (a ReadMe file, for example).
To import source files that are in tier 3 languages (all languages except C, C++, Java, C#) when you do not have third-party analysis results.
- If you do have third-party analysis results, see Including Tier 3 Components in a CodeSonar Project.
- To include tier 1 (C, C++) and tier 2 (Java, C#) source files in your CodeSonar project, use the appropriate language-specific mechanism.

Overview
Importing Files
codesonar add_source_files.py
Typical Use Case
Example

Overview

There are two main use cases for direct file import.

To include a file or set of files in your project for completeness or documentation purposes (a ReadMe file, for example).
Note that imported files are counted against your licensed line limit: we recommend that you do not import media files or other large files.
To import source files that are in tier 3 languages (all languages except C, C++, Java, C#) when you do not have third-party analysis results.
If you do have third party analysis results for your tier 3 source files, see Including Tier 3 Components in a CodeSonar Project.

Note

In this section (and throughout this manual), $CSONAR indicates the CodeSonar installation directory.

If you have defined environment variable CSONAR to the location of the CodeSonar installation directory, you can use $CSONAR directly in your command lines. On Windows systems, use %CSONAR% in place of $CSONAR.
If you don't want to use environment variables, replace $CSONAR with the path to your CodeSonar installation directory before using the command lines.

Importing Files

The following diagram provides an overview of a CodeSonar build process that brings files into the CodeSonar project.

diagram: building and analyzing a CodeSonar project with an add_source_files.py component

As described in Command Line Build/Analysis: Command, the general form of the usual CodeSonar build/analysis command is

codesonar analyze /path/to/pfiles-name [[protocol://]host:port] [other_options] [command]

To import one or more files, command must incorporate one or more invocations of codesonar add_source_files.py.

It will usually be most convenient to accomplish this through your regular build system (see the example below).
1. Extend your regular build system to invoke codesonar add_source_files.py.
2. Invoke codesonar analyze with the command that executes your regular build.
However, you can also specify codesonar add_source_files.py directly in your codesonar analyze (or codesonar build) command line.

codesonar add_source_files.py

To import files or directory contents to a project, CodeSonar observes the execution of a command that incorporates one or more invocations of

codesonar add_source_files.py [file_or_dir ...] [-include file_pat] [-exclude file_pat] \
[-language lang] [-max-bytes num]

where

[file_or_dir ...]	is a space-separated list of source file or directory paths. For each file path, that file is imported. For each directory path, all files in the directory tree rooted at that directory are imported. Note: These files are imported regardless of any -exclude options: an -exclude option can only modify the effects of preceding -include options. To avoid confusion, we recommend specifying the files to import through direct arguments or through a series of -include/-exclude options, but not both. The file_or_dir arguments are not glob patterns (as used for -include and -exclude). However, your shell may expand patterns into file and directory names. If no file or directory paths are specified and the set of additional files specified with -include and -exclude is empty, the codesonar add_source_files.py subcommand has no effect. The importer will ignore files with certain extensions, including those that indicate images (.jpg, .gif,...), archives (.tar, .gz,...) and library/object files (.o, .lib, .dll,...).
[-include file_pat] [-exclude file_pat]	Specify an additional set of source files to be imported. file_pat is a a recursive glob pattern representing a source file path. It matches all files whose file path matches file_pat. When a single codesonar add_source_files.py invocation contains a combination of -include and -exclude options, the set of additional files is determined by applying the corresponding inclusions and exclusions in the order in which they appear on the command line. Note: Your shell may be configured to expand patterns into file and directory names. If so, make sure you quote the file_pat appropriately to indicate that the shell should pass it through to codesonar add_source_files.py. In most cases, this means using single quotes ('); for Windows cmd, use double quotes ("). The -exclude option can only modify the effects of preceding -include options: it does not affect any files or directories specified as direct arguments to codesonar add_source_files.py. To avoid confusion, we recommend specifying the files to import through direct arguments or through a series of -include/-exclude options, but not both.
[-language lang]	Specifies that the imported files should be recorded as having language lang. If -language is not specified, CodeSonar will attempt to determine the language for each file based on its file extension. If it cannot determine a language, the file language will be recorded as "text". The available lang values are: actionscript, ada, awk, c, cfml, clojure, cobol, cpp, csharp, d, dart, eiffel, erlang, forth, fortran, fsharp, go, haskell, java, javascript, julia, kotlin, lisp, lua, objective_c, pascal, perl, php, prolog, python, r, ruby, rust, sas, scala, scheme, smalltalk, sql, swift, text, typescript, vbscript, verilog, vhdl, visualbasic The file language will be displayed in the UI and can be accessed programmatically through the CodeSonar Plug-in API. Specifying a language with this option does not cause the importer to apply language-specific parsing and other handling to the imported files. If you have source files in tier 1 (C, C++) or tier 2 (Java, C#) languages, use the appropriate language-specific mechanism to include them in the CodeSonar project.
[-max-bytes num]	Specifies a maximum size of num bytes for imported files: files larger than this maximum size will be ignored by the importer. If -max-bytes is not specified, the importer will ignore files larger than 500KB. You will need to specify this option with a suitably high value if you wish to import source files that are larger than the default limit. You may wish to specify this option with a lower value in cases where you know that all your source files are relatively small and wish to protect against accidentally importing larger, irrelevant files.

Command Examples

Import srcdir/mysource.kt only.
codesonar add_source_files.py srcdir/mysource.kt
Import all files with names ending in .kt that are located in a direct subdirectory of srcdir/.
codesonar add_source_files.py -include 'srcdir/*/*.kt'
Note that the srcdir/*/*.kt pattern is single-quoted ('srcdir/*/*.kt'). This is so that the shell passes it through to codesonar add_source_files.py rather than attempting to expand it. For Windows cmd, use double quotes instead ("srcdir/*/*.kt").
Import all files with names ending in .kt that are located anywhere in the directory tree rooted at srcdir/.
codesonar add_source_files.py -include 'srcdir/**/*.kt'
For Windows cmd, use double quotes instead.
Import all files with names ending in .kt that are located anywhere in the directory tree rooted at srcdir/, unless they are in the directory tree under srcdir/noinclude/.
codesonar add_source_files.py -include 'srcdir/**/*.kt' -exclude 'srcdir/noinclude/**/*'
The relative order of -include and -exclude is important: if -exclude were first it would be superseded by the subsequent -include and all .kt files under srcdir/ would be imported.
Import all files that are located anywhere in the directory tree rooted at srcdir/, unless they are in the tree under srcdir/noinclude/ and not under a directory named actuallyyes/ in that tree.
codesonar add_source_files.py -include 'srcdir/**/*.kt' -exclude 'srcdir/noinclude/**/*.kt' \
-include 'srcdir/noinclude/**/actuallyyes/**/*.kt'

Typical Use Case

Suppose we want to import a ReadMe file into the CodeSonar project.

Start with your normal software build.
In most cases, this will be managed through some build system (such as make) so that you can perform the build by executing a single command instead of performing all compilations and other operations manually.
Define an extended variant of your normal software build that performs all the steps of the regular build, then invokes codesonar add_source_files.py to add the ReadMe file to the CodeSonar project.
This is your CodeSonar-facing build.
- It is important to use a variant rather than augmenting the regular build directly, because the codesonar add_source_files.py should only be invoked in order to be observed by CodeSonar and not when you are building your software in other contexts.

Perform the CodeSonar build/analysis, observing the entire execution of your CodeSonar-facing build. For example:

Build Tool	Example regular build command	Example CodeSonar-facing build
make	make all	make csonar_scan Where the Makefile has been extended to include a csonar_scan target that depends on all and has a recipe that includes invocations of codesonar add_source_files.py. Then the CodeSonar build/analysis command will be: codesonar analyze myProject make csonar_scan
Windows batch file	cmd /c myProj.bat	cmd /c myProj.bat CSonarScan Where myProj.bat has been extended to invoke codesonar add_source_files.py after the normal build steps have been performed, but only if argument CSonarScan is passed. Then the CodeSonar build/analysis command will be: codesonar analyze myProject cmd /c myProj.bat CSonarScan
(and so forth) For more examples demonstrating how to construct a CodeSonar build/analysis command for various build tools, see Command Line Build/Analysis: Language-Specific Examples and the Basic Tutorial.

Example

Suppose part of your software project involves a rare programming language: so rare it does not have an analysis ecosystem.

For the sake of this example, we will call this language rarelang.
If rarelang did have an analysis ecosystem, you would follow the procedure for including tier 3 components in a CodeSonar project rather than using codesonar add_source_files.py as described here.

If you use make to manage your regular software build, your Makefile might look something like the following.
(If you use another tool to manage your regular build then the mechanism for expressing the steps and dependencies will be different, but analogous.)

.PHONY: all clean

# RareCompSrc.rare: a rarelang source file
# rarecompile: the compiler for rarelang
RareCompExe: RareCompSrc.rare
       rarecompile RareCompSrc.rare -o RareCompExe

all: RareCompExe othercomponent1 [... other project components]

clean:
        rm -f RareCompExe

[... recipes for remaining project components]

If you built a CodeSonar project based on observing the execution of this Makefile in its original state, the CodeSonar project would not contain any information about the rarelang component.

To import RareCompSrc.rare into your CodeSonar project, do the following.

Build and analyze a CodeSonar project based on observing the execution of the original Makefile. Assuming your hub is at the default location, the CodeSonar build/analysis command will be something like:
make clean
codesonar analyze ProjectX make all
Once the CodeSonar build/analysis is complete, open its hub GUI Analysis page.
- The Files tab will not include an entry for RareCompSrc.rare.
- Depending on the operations used to build other project components (othercomponent1 ...), the Warnings and Files tabs may have other contents. In particular, if the recipes for other project components incorporate C/C++ compiler invocations or other commands recognized by CodeSonar, those components will be correspondingly represented in the CodeSonar project.
Edit the Makefile to add a new csonar_scan target. This target should:
- depend on the all target, so that all steps of the regular software build are accounted for.
- have a recipe that invokes codesonar add_source_files.py to instruct CodeSonar to import the rarelang source file.
The updated example Makefile is shown below, with the new contents highlighted like this.
```
.PHONY: all clean csonar_scan

RareCompExe: RareCompSrc.rare
       rarecompile RareCompSrc.rare -o RareCompExe

all: RareCompExe othercomponent1 [... other project components]

clean:
        rm -f RareCompExe

csonar_scan: RareCompSrc.rare
        codesonar add_source_files.py RareCompSrc.rare

[... recipes for remaining project components]
```
If you are not using make, add these invocations in the appropriate analogous location in your build infrastructure.
Build and analyze a CodeSonar project, using a codesonar analyze command based on building the new csonar_scan target.
make clean
codesonar analyze ProjectX make csonar_scan
Once the CodeSonar build/analysis is complete, open its hub GUI Analysis page.
- The Files tab will include an entry for RareCompSrc.rare. Click this entry to view the full file.
  Note that code coloring and interaction will are not available for RareCompSrc.rare, since the file was imported with codesonar add_source_files.py.

To report problems with this documentation, please visit https://support.codesecure.com/.