Build and Analysis for Java Projects
and Kotlin Projects Targeting the JVM

• Introduction
• How the Build Works
• Command Line Build
• cs-java-scan Build Options
• Example Command Lines
• Windows Build Wizard Note
• Troubleshooting

Introduction

The cs-java-scan command can be used to analyze both Java and Kotlin components.

Note that cs-java-scan does not invoke the Kotlin detekt analyzer. If you also want detekt results for your Kotlin code, include the corresponding codesonar kotlin_scan.py invocations in your CodeSonar-facing build.

How The Build Works

The following diagram shows the CodeSonar build and analysis process for Java projects (and Kotlin projects targeting the JVM)

The steps of the build and analysis process are described in Building: Overview. For projects based on Java code, the steps manifest as follows.

Once it has finished building the project, CodeSonar can run the analysis. If you use the analyze command, the analysis will run (and send its results to the hub) after the build has finished.

The degree of parallelism in the analysis will depend on the setting of ANALYSIS_SLAVES. Additional parallelism can be achieved by executing multiple cs-java-scan commands in parallel for various smaller pieces of a project.

Command Line Build

As described in Command Line Build/Analysis: Command, the general form of the usual CodeSonar build/analysis command is

The command element is described below; the other parts of the command line are discussed in Command Line Build/Analysis: Elements.

command

To build a Java project, CodeSonar observes the execution of a command of one of the following forms.
For Kotlin projects, we recommend using the -include-artifacts,-include-sources form only (not the -include-sourceless-artifacts form).

There are therefore two general forms of the CodeSonar build/analysis command when utilizing cs-java-scan:

where remaining-options is any combination of include options, exclude options, and other options.

AOSP note: If you are analyzing the Android Open Source Project, you can use the cs_android.py subcommand. This provides functionality for partitioning your sources and artifacts into groups and producing an appropriate cs-java-scan invocation for each group. See codesonar cs_android.py : Prepare and Analyze the Android Open Source Project for full details.

Patterns

Wildcard patterns follow the rules of the Java Spring-Framework AntPathMatcher class. In particular:

• ? matches one character. It does not match directory separator characters.
• * matches zero or more characters. It does not match directory separator characters.
• ** matches zero or more directories in a path.
• Patterns including any wildcards (?, *, or **) must be quoted.

A pattern that matches a directory is considered to match all files in that directory with the appropriate suffixes:

• .java or .kt for -include-sources and -exclude-sources,
• .class, .jar, .xml, .sarl, or .properties for all other include options and exclude options.

cs-java-scan Build Options

This section describes the available options for cs-java-scan, all of which can be added using configuration file settings.

• Specifying cs-java-scan Build Options
• Options
• Categories

As described in section Build and Analysis for Java Projects, there are two general forms of the CodeSonar build/analysis command with a cs-java-scan command element:

Where remaining-options is any combination of include options, exclude options, and other options.

The relative ordering of include and exclude options is important:

• Files specified for inclusion in the project with an include option will be removed if they match a subsequent occurrence of the corresponding exclude option.
• Files removed by an exclude option will be included again if they match a subsequent occurrence of the corresponding include option.

Specifying cs-java-scan Options

You can specify cs-java-scan options using the following configuration file parameters.

Options

The following table lists all options understood by cs-java-scan.

Artifact Categories

Artifacts are files with the following extensions: .class, .jar, .xml, .sarl, and .properties.

Each artifact included in a particular CodeSonar analysis belongs to one of four categories: ordinary artifacts, sourceless artifacts, libraries, and signatures. The artifact's category determines how the analysis will handle the artifact, as described in the table below.

• The category of an artifact is determined by the command line option you use to include it in the analysis.
• Inclusion and exclusion are handled separately for each artifact category. For example, an occurrence of -exclude-sourceless-artifacts will have no effect on the inclusion of artifacts specified with earlier occurrences of -include-artifacts, -include-libraries, or -include-signatures.
• If an artifact is specified as a member of multiple categories (for example, because it matches both an -include-artifacts patern and an -include-libraries pattern specified in the project build command), it will be treated as a member of the most important of those categories.

The following table describes the artifact categories in order of importance.

Example Command Lines

Some example CodeSonar command lines for building and analyzing Java projects.

Note: Depending on the hub configuration, you may be prompted for hub user account credentials to authenticate and authorize codesonar build and codesonar analyze commands. See Hub Authentication: Authenticated codesonar Subcommands for more information.

The first five examples below are for projects with simple structure.

Example 1

Build a CodeSonar project based on the classes contained in Example.jar and the source file Example.java, both located in the current working directory.

• Analyze all classes contained in Example.jar
• Output warnings relevant to Example.java.

Example 2

Build and analyze a CodeSonar project based on the source file /path/to/sources/D.java and class file /path/to/classes/D.class.

Example 3

Build and analyze a CodeSonar project based on the source files in /path/to/sources and the artifacts in /path/to/classes.

• The sources will be all .java (Java) and .kt (Kotlin) files in the tree below /path/to/sources.
• The artifacts will be all .class, .jar, .xml, .sarl, and .properties files in the tree below /path/to/classes.
  For a simple project, this will simply be a set of .class files.

Example 4

Build and analyze a CodeSonar project based on the Java source files in /path/to/sources and the artifacts in /path/to/classes.

• The sources will be all .java files in the tree below /path/to/sources.
• The artifacts will be all .class files in the tree below /path/to/classes.
• /path/to/classes/**/*.class is called a wildcard pattern, where ** corresponds to zero or more directories, and *.class corresponds to any file name with suffix .class. All files with the suffix .class will be found, an arbitrary number of directories deep.
• /path/to/sources/**/*.java is similar, with ** corresponding to zero or more directories, and *.java corresponding to any file name with suffix .java. All files with the suffix .java will be found, an arbitrary number of directories deep.
• Note options including wildcard patterns must be quoted.

Example 5

Build and analyze a CodeSonar project based on the source files in /path/to/sources and the .jar file /path/to/Project.jar.

• The sources will be all .java (Java) and .kt (Kotlin) files in the tree below /path/to/sources.
• The contents of /path/to/Project.jar will be analyzed.

The following examples illustrate more fine-grained control of the analysis.

Example 6

Build and analyze a CodeSonar project based on the Java source files in /path/to/sources and the artifacts in /path/to/classes, except for the artifacts in /path/to/classes/A.

• The sources will be all .java files in the tree below /path/to/sources.
• The artifacts will be all .class files in the tree below /path/to/classes, except for the ones below /path/to/classes/A .
• The order of -include-artifacts and -exclude-artifacts is critical. In this case, if they were reversed, no class files would be excluded.
• Note options including wildcard patterns must be quoted.

Example 7

Build and analyze a CodeSonar project based on the Java source files in /path/to/sources and the artifacts in /path/to/classes, except for the artifacts in /path/to/classes/A that are not in /path/to/classes/A/B.

• The sources will be all .java files in the tree below /path/to/sources.
• The artifacts will be all .class files in the tree below /path/to/classes, except for the ones below /path/to/classes/A that are not below /path/to/classes/A/B.
• Once again, the order of -include-artifacts and -exclude-artifacts is critical to ensure expected results.
• Note options including wildcard patterns must be quoted.
• Note also -exclude-sources, which interacts with -include-sources in the same way that -exclude-artifacts interacts with -include-artifacts.

For the analysis, there is a hierarchy of artifacts: artifacts, sourceless artifacts, libraries, and signatures. For each of these, there is an -include option and an -exclude option.

Example 8

Build and analyze a CodeSonar project based on the source files in /path/to/sources and the .jar file /path/to/Project.jar, using the type signatures of the classes found inside /path/to/External.jar.

• The sources will be all .java (Java) and .kt (Kotlin) files in the tree below /path/to/sources.
• The contents of /path/to/Project.jar will be analyzed.
• The contents of /path/to/External.jar will be used to provide type signatures to the analysis engine.

Example 9

Build and analyze a CodeSonar project based on the full contents of the current directory.

Example 10

Build and analyze a CodeSonar project based on the full contents of the current directory, excluding classes and directories whose name contains "Test".

Example 11

cat <<EOF > buildscan.sh
#!/bin/sh
ant
cs-java-scan -include-artifacts foo -include-sources foo &
cs-java-scan -include-artifacts bar -include-sources bar &
wait
EOF
chmod +x buildscan.sh
codesonar analyze foo ./buildscan.sh

Build the project with ant, then run two instances of cs-java-scan in parallel on different parts of the project.

Windows Build Wizard Note

In general, we recommend using the command line CodeSonar build/analysis for Java projects.

However, it is also possible to use the Windows build wizard to observe cs-java-scan invocations.

• Make sure the wizard is set up to watch the process (for example, the command prompt) in which you will invoke cs-java-scan.
• Once the wizard is recording, run your cs-java-scan commands (details and examples are given above).

Troubleshooting

• Resolving Alerts
• "Too many potential candidates" parse errors

Resolving Alerts

The most typical alerts related to the Java build/analysis are listed below. For descriptions of all CodeSonar alerts, see Alerts.

"Too many potential candidates" parse errors

In some cases, you may see a parse error that looks something like the following.

For each analyzed ordinary artifact and sourceless artifact, CodeSonar attempts to identify the corresponding source file. To do this, CodeSonar iterates over those files specified with -include-sources and -exclude-sources whose basenames correspond to that of the artifact, checking each in turn to see if it is the correct match. Iteration finishes either when a match is found, or when the number of source files checked for a particular artifact exceeds the limit specified in environment variable JAVA_JFE_INCLUDESOURCE_LIMIT. The default value for this limit is 100.

If CodeSonar does not find the correct matching source file before reaching the iteration limit, then:

• The artifact is treated as having a missing source file.
  • No analysis results will be produced for this artifact.
  • A Missing Source File alert will be produced.
  • If it is an ordinary artifacts, the analysis may fail: see the discussion of ordinary artifacts above for details.
• You will see a parse error message like the one above.

There are two options for resolving this problem.

• Option 1: Use -include-sources and -exclude-sources to specify a more precise set of source files.

  This option is generally preferable if some of the source files with duplicate basenames are not relevant to your analysis. A more precise set of candidate source files will generally reduce the time spent on matching, and so reduce overall analysis time.

• Option 2 : Increase the setting of environment variable JAVA_JFE_INCLUDESOURCE_LIMIT to your preferred value.

  If you have a large number of artifacts with the same basename, there will necessarily be a large number of corresponding source files to iterate through for matching. In this case, you will need to increase the JAVA_JFE_INCLUDESOURCE_LIMIT to a value that allows all your source files to be considered.

  This approach imposes additional time costs. Firstly, a higher limit means that more time can be spent on matching and so the overall analysis time is generally increased. Secondly, you will need to spend time investigating or experimenting to determine a suitable JAVA_JFE_INCLUDESOURCE_LIMIT setting. We recommend the following approach.

  1. Identify the highly-duplicated basenames by reading them from parse error mesages like the example above.
  2. For each of these basenames, count the number of file instances in your source directory or directories.
  3. Set JAVA_JFE_INCLUDESOURCE_LIMIT to a value slightly higher than the highest of these counts.

In some cases, the most convenient solution might be to split your build/analysis across two or more cs-java-scan invocations, each of which covers a smaller set of artifacts and specifies a more precise set of source files. This approach may still require you to increase the JAVA_JFE_INCLUDESOURCE_LIMIT, but typically not to the same extent.

Related Sections

• Command Line Build Instructions
• The Build/Analyze Commands (detailed documentation)