Advanced Settings for GitLab

version 2.3 or later

1. Use Project Access Token instead of CI/CD Job Token

Sider Scan accesses to the artifact with “CI/CD Job Token”. However, if your GitLab is Free Tier version, CI/CD Job Token is not available. In that case, please use “Project Access Token” instead of CI/CD Job Token. This section explains how to generate and use a “Project Access Token”. As shown in the figure below, select Settings -> Access Tokens from the side menu of your repository, and enter the Project Access Token management view.

Next, as shown in the figure below, enter an arbitrary string for Token name, check “read_api” for “Select scopes,” and click “Create Project Access Token” to generate the token string.

On the next screen, copy the generated token string (as shown below, there is an icon to copy to the clipboard at the right to the text box where the token sequence is displayed).

It is not recommended to write the token string directly in the YAML script for security reasons. Instead, use GitLab’s CI/CD variable to hide the token string. In the figure below, the PAT_SIDERSCAN is the CI/CD variable to hide the token string. The “Protected” flag of that environment variable should be set to x (disabled). Please refer to the GitLab manual for more details.

When you use the Project Access Token instead of CI/CD Job Token, you need to replace "JOB-TOKEN: ${CI_JOB_TOKEN}" to "PRIVATE-TOKEN: ${PAT_SIDERSCAN}" in the following wget line in .gitlab-ci.yml :

'wget --header "JOB-TOKEN: ${CI_JOB_TOKEN}" "${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/jobs/artifacts/${CI_COMMIT_REF_NAME}/raw/output.radump?job=scan-job" -O output.radump || true'

where PAT_SIDERSCAN is the CI/CD variable that hides the token string generated as the Project Access Token.

2. Change the retention period of analysis results (artifacts)

The GitLab’s default retention period for artifacts is 30 days. If you want to extend it, enter the desired number of days in the expire_in field of the .gitlab-ci.yml configuration file. The following is an example of extending the expire_in to 90 days.

  artifacts:
    paths:
      - output.radump
      - scan_result/*
    expire_in: 90 days # Retention period

 

3. Extend CI/CD timeout period

Sider Scan analyzes the entire source code of your repository for the first time. The initial analysis time depends on the size of the repository and the number of duplicate codes, but may take more than one hour. In this case, please extend the CI/CD timeout time. Please refer to this GitLab manual on how to set this up. For the second and subsequent analyses, Sider Scan will read the previous analysis result and analyze only those files that have been modified since the last analysis, so the analysis will be completed in a shorter time.

4. Suppress the number of concurrent runs of Sider Scan on GitLab CI

Sider Scan detects duplicates in the source code and uses a proprietary algorithm to detect omissions of modifications. We are constantly working to shorten the processing time of analysis, but, especially the first analysis of a repository, which can take more than one hour depending on the size of the repository, since all source code in the repository is analyzed.

Although GitLab recommends the processes used by CI/CD tasks separately from the server running the GitLab’s main task to manage source code, you may run GitLab CI/CD process on the same server as the GitLab itself. Even if CI/CD processes is run on a separate server, you may want to control the number of concurrent runs of Sider Scan on GitLab CI.

This section describes how to suppress the number of Sider Scans running simultaneously in the GitLab.

4-1. Register Shard-Runner

Only Admin user can create a Shard-Runner. Click “Admin” in the top menu, then select “Runners” to enter the Runner management view.

Click “Register an instance runner” to register a new Shared-Runner.

4-2. Shared-Runner Settings

Uncheck “Run untagged Jobs” and enter an arbitrary string in “Tags”. In this example, we use siderscan-tag as a tag name.

4-3. Set the number of concurrent executions

The configuration file for the maximum number of concurrent executions of each runner is located in /etc/gitlab-runner/config.toml. Open this file with an editor, and input the number of concurrent executions. For example, if you want to set it to one, please input the line like this:

concurrent = 1

See also this GitLab document for more information about .toml file.

Now you have a single Sider Scan running across the entire GitLab. If multiple jobs are using Sider Scanat the same time, the job that is issued later will simply wait.