Polaris - 静态代码分析

这可能是中文网里介绍Polaris最详细的文章了

什么是 Polaris?

Polaris - 托管静态应用程序软件测试(SAST)工具的 SaaS 平台,它是用于分类和修复漏洞并运行报告的 Web 站点。
SAST - 一种对源代码分析或构建过程中去寻找安全漏洞的工具,是一种在软件开发的生命周期(SDLC)中确保安全的重要步骤。
Coverity - Coverity 是 Synopsys 公司提供的原始静态应用软件测试 (SAST) 工具。Polaris 是 Coverity 的 SaaS 版本。
Synopsys - 是开发 Polaris 和其他软件扫描工具的公司,比如 BlackDuck 也是他们的产品。

Polaris 支持哪些语言?

C/C++
C#
Java
JavaScript
TypeScript
PHP
Python
Fortran
Swift
...and more

Polaris SaaS 平台

通常如果你的组织引入了 Polaris 的 SaaS 服务,你将会有如下网址可供访问 URL: https://organization.polaris.synopsys.com

然后登录,你就可以给自己的 Git Repository 创建对应的项目了。

建议:创建的项目名称与 Git Repository 的名称一致。

Polaris 如何进行漏洞扫描?

Polaris 安装

在进行 Polaris 扫描之前,你需要先下载并安装 polaris。

如果你的 Polaris server URL 为:POLARIS_SERVER_URL=https://organization.polaris.synopsys.com

下载连接为:$POLARIS_SERVER_URL/api/tools/polaris_cli-linux64.zip

然后将下载到本地的 polaris_cli-linux64.zip 进行解压,将其 bin 目录添加到 PATH 中。

Polaris YAML 文件配置

在进行扫描之前,你需要为你的项目创建 YAML 文件。默认配置文件名为 polaris.yml,位于项目根目录。如果你希望指定不同的配置文件名,你可以在 polaris 命令中使用 -c 选项。

在项目根目录运行 polaris setup 以生成通用的 polaris.yml 文件。

运行 polaris configure 以确认你的文件在语法上是正确的并且 polaris 没有任何问题。

Capture - 捕获

YAML 配置文件可以包含三种类型的 Capture:

  • Build(构建) - 运行构建命令,然后分析结果
  • Filesystem(文件系统) - 对于解释型语言,提供项目类型和要分析的扩展列表
  • Buildless - 对于一些可以使用依赖管理器的语言,比如 maven
Languages Build Options
C, C++, ObjectiveC, Objective C++,Go, Scala, Swift 使用 Build 捕获
PHP, Python, Ruby 使用 Buildless 或 Filesystem 捕获
C#, Visual Basic. 如果想获得更准确的结果使用 Build 捕获;如果寻求简单使用 Buildless 捕获
Java 如果想获得更准确的结果使用 Build 捕获;如果寻求简单使用 Buildless 捕获
JavaScript,TypeScript 使用 Filesystem 捕获;如果寻求简单使用 Buildless 捕获

Analyze - 分析

如果你正在扫描 C/C++ 代码,则应包括此分析部分以充分利用 Polaris 的扫描功能:

analyze:
mode: central
coverity:
cov-analyze: ["--security","--concurrency"]

Polaris YAML 示例文件

Read More

Git 常用命令备忘录

不管是对于 Git 的初学者还是经常使用 Git 的码农们,在日常工作中难免会有遇到有的命令一时想不起来。不妨将下面总结的一些 Git 常用命令及技巧收藏或打印出来,以备需要的时候可以很快找到。

git config

Read More

How does SonarQube Community Edition integrate with the project

After you have set up the SonarQube instance, you will need to integrate SonarQube with project.

Because I used the community edition version, it doesn’t support the C/C++ project, so I only demo how to integrate with Maven, Gradle, and Others.

For example, the demo project name and ID in SonarQube are both test-demo, and I build with Jenkins.

Read More

Run lcov failed "Can't locate JSON/PP.pm in @INC ..."

When execute command: lcov --capture --directory . --no-external --output-file coverage.info to generate code coverage report, I encountered the following error:

$ lcov --capture --directory . --no-external --output-file coverage.info
Capturing coverage data from .
Can't locate JSON/PP.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/bin/geninfo line 63.
BEGIN failed--compilation aborted at /usr/local/bin/geninfo line 63.
sh-4.2$ perl -MCPAN -e 'install JSON'
Can't locate CPAN.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .).
BEGIN failed--compilation aborted.

Read More

使用 Gcov 和 LCOV 做 C/C++ 项目的代码覆盖率

本篇分享如何使用 Gcov 和 LCOV 对 C/C++ 项目进行代码覆盖率的度量。

如果你想了解代码覆盖率工具 Gcov 是如何工作的,或是以后需要做 C/C++ 项目的代码覆盖率,希望本篇对你有所帮助。

Read More

SonarQube installation and troubleshootings

Backgroud

In my opinion, SonarQube is not a very easy setup DevOps tool to compare with Jenkins, Artifactory. You can’t just run some script under the bin folder to let the server boot up.

You must have an installed database, configuration LDAP in the config file, etc.

So I’d like to document some important steps for myself, like setup LDAP or PostgreSQL when I install SonarQube of v9.0.1. It would be better if it can help others.

Prerequisite and Download

  1. Need to be installed JRE/JDK 11 on the running machine.

    Here is the prerequisites overview: https://docs.sonarqube.org/latest/requirements/requirements/

  2. Download SonarQube: https://www.sonarqube.org/downloads/

    cd sonarqube/
    ls
    wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-9.0.1.46107.zip

    unzip sonarqube-9.0.1.46107.zip
    cd sonarqube-9.0.1.46107/bin/linux-x86-64
    sh sonar.sh console

Change Java version

I installed SonarQube on CentOS 7 machine, the Java version is OpenJDK 1.8.0_242 by default, but the prerequisite shows at least need JDK 11. There is also JDK 11 available on my machine, so I just need to change the Java version.

I recommend using the alternatives command change Java version,refer as following:

$ java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

$ alternatives --config java

There are 3 programs which provide 'java'.

Selection Command
-----------------------------------------------
1 java-1.7.0-openjdk.x86_64 (/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.251-2.6.21.1.el7.x86_64/jre/bin/java)
*+ 2 java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-1.el7.x86_64/jre/bin/java)
3 java-11-openjdk.x86_64 (/usr/lib/jvm/java-11-openjdk-11.0.12.0.7-0.el7_9.x86_64/bin/java)

Enter to keep the current selection[+], or type selection number: 3
$ java -version
openjdk version "11.0.12" 2021-07-20 LTS
OpenJDK Runtime Environment 18.9 (build 11.0.12+7-LTS)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.12+7-LTS, mixed mode, sharing)

Install Database

SonarQube needs you to have installed a database. It supports several database engines, like Microsoft SQL Server, Oracle, and PostgreSQL. Since PostgreSQL is open source, light, and easy to install, so I choose PostgreSQL as its database.

How to download and install PostgreSQL please see this page: https://www.postgresql.org/download/linux/redhat/

Troubleshooting

1. How to establish a connection with SonarQube and PostgreSQL

Please refer to the sonar.properties file at the end of this post.

2. How to setup LDAP for users to log in

sonar.security.realm=LDAP
ldap.url=ldap://den.exmaple-org:389
ldap.bindDn=user@exmaple-org.com
ldap.bindPassword=mypassword
ldap.authentication=simple
ldap.user.baseDn=DC=exmaple-org,DC=com
ldap.user.request=(&(objectClass=user)(sAMAccountName={login}))
ldap.user.realNameAttribute=cn
ldap.user.emailAttribute=email

3. How to fix LDAP login SonarQube is very slowly

Comment out ldap.followReferrals=false in sonar.properties file would be help.

Related post: https://community.sonarsource.com/t/ldap-login-takes-2-minutes-the-first-time/1573/7

4. How to fix ‘Could not resolve 11 file paths in lcov.info’

I want to display Javascript code coverage result in SonarQube, so I added sonar.javascript.lcov.reportPaths=coverage/lcov.info to the sonar-project.properties

But when I run sonar-scanner.bat in the command line, the code coverage result can not show in sonar. I noticed the following error from the output:

INFO: Analysing [C:\workspace\xvm-ide\client\coverage\lcov.info]
WARN: Could not resolve 11 file paths in [C:\workspace\xvm-ide\client\coverage\lcov.info]

There are some posts related to this problem, for example, https://github.com/kulshekhar/ts-jest/issues/542, but no one works in my case.

# here is an example error path in lcov.info
..\src\auto-group\groupView.ts

Finally, I have to use the sed command to remove ..\ in front of the paths before running sonar-scanner.bat, then the problem was solved.

sed -i 's/\..\\//g' lcov.info

Please comment if you can solve the problem with changing options in the tsconfig.json file.

4. How to output to more logs

To output more logs, change sonar.log.level=INFO to sonar.log.level=DEBUG in below.

Note: all above changes of sonar.properties need to restart the SonarQube instance to take effect.

Final sonar.properties

For the sonar.properties file, please see below or link

Read More

How to fix "hidden symbol `__gcov_init' in ../libgcov.a(_gcov.o) is referenced by DSO"

Problem

When we introduced Gocv to build my project for code coverage, I encountered the following error message:

Read More

Add or update Bitbucket build status with REST API

Backgorud

  1. When you want to add build status to your Bitbucket the specific commit of a branch when you start a build from the branch

  2. When the build status is wrong, you want to update it manually. for example, update build status from FAILED to SUCCESSFUL

You can call Bitbucket REST API to do these.

Code snippet

Below is the code snippet to update Bitbucket build status with REST API in the shell script.

The code on GitHub Gist: https://gist.github.com/shenxianpeng/bd5eddc5fb39e54110afb8e2e7a6c4fb

Click Read More to view the code here.

Read More

关于代码覆盖率 (About Code Coverage)

本篇简要介绍:什么是代码覆盖率?为什么要做代码覆盖率?代码覆盖率的指标、工作原理,主流的代码覆盖率工具以及不要高估代码覆盖率指标。

什么是代码覆盖率?

代码覆盖率是对整个测试过程中被执行的代码的衡量,它能测量源代码中的哪些语句在测试中被执行,哪些语句尚未被执行。

Read More

Code coverage testing of C/C++ projects using Gcov and LCOV

This article shares how to use Gcov and LCOV to metrics code coverage for C/C++ projects.
If you want to know how Gcov works, or you need to metrics code coverage for C/C++ projects later,
I hope this article is useful to you.

Read More

How to make Jenkins job fail after timeout? (Resolved)

I’ve run into some situations when the build fails, perhaps because some processes don’t finish, and even setting a timeout doesn’t make the Jenkins job fail.

So, to fix this problem, I used try .. catch and error to make my Jenkins job failed, hopes this also helps you.

Read More

解决在 AIX 上 Git Clone 失败的两个问题

前言

本篇记录两个在做 Jenkins 与 AIX 做持续集成得时候遇到的 Git clone 代码失败的问题,并已解决,分享出来或许能有所帮助。

  1. Dependent module /usr/lib/libldap.a(libldap-2.4.so.2) could not be loaded.
  2. 通过 SSH 进行 git clone 出现 Authentication failed

Read More

通过解除文件资源限制:解决在 AIX 使用 Git 下载大容量仓库失败问题

最近使用 AIX 7.1 从 Bitbucket 下载代码的时候遇到了这个错误:

Read More

关于 Artifactory 上传制品变得非常缓慢,偶尔失败的问题分享

最近在我使用 Artifactory Enterprise 遇到了上传制品非常缓慢的问题,在经过与 IT,Artifactory 管理员一起合作终于解决这个问题,在此分享一下这个问题的解决过程。

如果你也遇到类似或许有所帮助。

Read More

Resolved problem that ESlint HTML report is not displayed correctly in Jenkins job

I’m just documenting to myself that it was solved by following.

When I want to integrate the ESlint report with Jenkins. I encourage a problem

That is eslint-report.html display different with it on my local machine, and I also log to Jenkins server and grab the eslint-report.html to local, it works well.

I used HTML Publisher plugin to display the HTML report, but only the ESlint HTML report has problems other report work well, so I guess this problem may be caused by Jenkins.

Finally, I find it. (Stackoverflow URL)

Read More