SonarQube数据库迁移小记

背景

2020年由于团队需要,部署了一套SonarQube的代码质量管理系统,使用的是社区版本,数据库放在了公司的测试服务器上

21年发现数据库查询过于频繁导致内存占用经常居高不下,遂准备将数据库迁出测试环境

由于对SonarQube的依赖不太强(就是用的不多),所以对于可用性之类的并不是特别在意,于是乎考虑使用PostgreSql数据库并且跟SonarQube部署在同一台服务器上

埋坑

最开始的时候SonarQube使用了docker的部署方式,毕竟用起来比较方便嘛

这里直接使用了docker-compose的部署文件类部署的(相当方便了),刚开始用没有配置相关的Volume(给自己挖坑)

version: "3"

services:
  sonarqube:
    image: sonarqube:8.6-community
    restart: always
    environment:
      SONAR_JDBC_URL: jdbc:postgresql://***/sonarqube-dev
      SONAR_JDBC_USERNAME: postgres
      SONAR_JDBC_PASSWORD: ********
    ports:
      - "9000:9000"

前端使用了一个NGINX服务器作了转发

跳坑

迁移后使用容器化的数据库,配置文件如下

version: "3"

networks:
      default:
            external:
                 name: sonarqube_default

services:
  sonarqube:
    image: sonarqube:8.7-community
    restart: always
    environment:
      SONAR_JDBC_URL: jdbc:postgresql://sonarqube_db/sonarqube-dev
      SONAR_JDBC_USERNAME: ****
      SONAR_JDBC_PASSWORD: ****
    volumes:
      - sonarqube_data:/opt/sonarqube/data
      - sonarqube_extensions:/opt/sonarqube/extensions
      - sonarqube_logs:/opt/sonarqube/logs
      - sonarqube_temp:/opt/sonarqube/temp
    ports:
      - "9000:9000"

  sonarqube_db:
    image: postgres:9.6.19
    restart: always
    environment:
      POSTGRES_PASSWORD: eya.psql.123456
      PGDATA: /var/lib/postgresql/data/pgdata
    ports:
      - "5432:5432"
    volumes:
      - postgresql_data:/var/lib/postgresql/data/pgdata
    networks:
        default:
            ipv4_address: 172.18.0.10

volumes:
  sonarqube_data:
  sonarqube_extensions:
  sonarqube_logs:
  sonarqube_temp:
  postgresql_data:

完成容器的加载之后,毛病就出来了

目前遇到了两个问题

问题列表加载报错

迁移过后照例分析代码然后查看代码问题,发现在问题列表页面会弹出错误提示

遂查看SonarQube的日志 发现如下报错信息

2021.03.11 02:30:07 ERROR web[AXgfHRLRR9QeKP02AAA1][o.s.s.w.WebServiceEngine] Fail to process request http://****/api/issues/search?s=FILE_LINE&resolved=false&ps=100&facets=owaspTop10%2CsansTop25%2Cseverities%2CsonarsourceSecurity%2Ctypes&additionalFields=_all&timeZone=Asia%2FShanghai
java.lang.IllegalStateException: Fail to read ISSUES.LOCATIONS [KEE=AXbQl3SfQ0OC-M1crhBO]
        at org.sonar.db.issue.IssueDto.parseLocations(IssueDto.java:652)
        at org.sonar.server.issue.ws.SearchResponseLoader$Collector.collectComponentsFromIssueLocations(SearchResponseLoader.java:268)
        at org.sonar.server.issue.ws.SearchResponseLoader$Collector.collect(SearchResponseLoader.java:263)
        at org.sonar.server.issue.ws.SearchResponseLoader.load(SearchResponseLoader.java:93)
        at org.sonar.server.issue.ws.SearchAction.doHandle(SearchAction.java:427)
        at org.sonar.server.issue.ws.SearchAction.handle(SearchAction.java:389)
        at org.sonar.server.ws.WebServiceEngine.execute(WebServiceEngine.java:110)
        at org.sonar.server.platform.web.WebServiceFilter.doFilter(WebServiceFilter.java:85)
        at org.sonar.server.platform.web.MasterServletFilter$GodFilterChain.doFilter(MasterServletFilter.java:139)
        at org.sonar.server.platform.web.MasterServletFilter.doFilter(MasterServletFilter.java:108)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.sonar.server.platform.web.UserSessionFilter.doFilter(UserSessionFilter.java:81)
        at org.sonar.server.platform.web.UserSessionFilter.doFilter(UserSessionFilter.java:68)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.sonar.server.platform.web.CacheControlFilter.doFilter(CacheControlFilter.java:76)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.sonar.server.platform.web.SecurityServletFilter.doHttpFilter(SecurityServletFilter.java:76)
        at org.sonar.server.platform.web.SecurityServletFilter.doFilter(SecurityServletFilter.java:48)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.sonar.server.platform.web.RedirectFilter.doFilter(RedirectFilter.java:58)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.sonar.server.platform.web.RequestIdFilter.doFilter(RequestIdFilter.java:66)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.sonar.server.platform.web.RootFilter.doFilter(RootFilter.java:62)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:109)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:544)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
        at ch.qos.logback.access.tomcat.LogbackValve.invoke(LogbackValve.java:256)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:616)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1634)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.
        at com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:106)
        at com.google.protobuf.CodedInputStream$ArrayDecoder.checkLastTagWas(CodedInputStream.java:635)
        at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:160)
        at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:191)
        at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:203)
        at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:208)
        at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:48)
        at org.sonar.db.protobuf.DbIssues$Locations.parseFrom(DbIssues.java:330)
        at org.sonar.db.issue.IssueDto.parseLocations(IssueDto.java:650)
        ... 51 common frames omitted

解决

这里不再对问题的排查过程进行敷述,仅列出解决方式

  1. 清空issueissue_changes

    issue_changes 可能不需要清空,这里保险起见我也清空了

  2. 清理elasticsearch缓存数据

    data/es7/目录下 清空该目录下所有文件即可

  3. 在SonarQube的配置里点击重启服务器

  4. 进行代码分析即可正常看到代码的问题

版权声明:
作者:暴打小盆友
链接:https://wandhi.com/post-1104.html
来源:玩的嗨
文章版权归作者所有,未经允许请勿转载。

THE END
分享
二维码
< <上一篇
下一篇>>