redash docker deploy

2018/11/12 posted in  python

redash官方提供了基于docker下面的deploy文档,在目录setup下面。https://github.com/yourwilliam/redash/tree/master/setup

这份脚本是基于ubuntu18.04的,建议在ubuntu18下面进行安装部署

具体使用步骤如下:

1. 切换apt源

ubuntu18 版本安装完成之后发现里面并没有包含所有的官方源,最好是切换到国内的源,这样速度也更快。

修改/etc/apt/sources.list文件,记得最好把这个文件备份一个,方便后面修复。

中科大源

deb https://mirrors.ustc.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse

然后执行命令:

sudo apt-get update
sudo apt-get upgrade

也可以使用其他的源,如下

阿里源

deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

163源

deb http://mirrors.163.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.163.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.163.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.163.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.163.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic-backports main restricted universe multiverse

清华源

deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse

git clone 下载

注意脚本里面很多东西都写了绝对目录,默认认为redash在目录 /opt/redash下,所以也是在opt目录下面进行git clone可以指定到对应的路径下。尽量不要自己修改目录,否则要修改的地方会很多。

执行setup/setup.sh文件

执行setup文件就可以安装

docker配置

在docker中检查当前网络状况要使用ping命令的时候,有时候由于没有安装无法使用,这时候可以自己安装

#1. 首先一定要更新源

apt-get update

#2. 安装工具包

apt-get install vim      # 安装vim

 apt-get install  telnet  # 安装telnet 

 apt-get install  net-tools  # 安装ifconfig 

apt install iputils-ping  # 安装ping

通过root进入docker容器

docker exec -i -t -u 0 e575dad3beae /bin/bash
# 其中e575dad3beae 通过docker ps获取

问题

问题1:

直接安装后,会出现一个权限不足的问题,页面无法访问

是由于在docker中执行npm build 和 npm start的时候是使用的redash用户,而不是root用户,所有数据库无法连接,同时很多文件没有权限访问

当前解决办法:

通过docker ps查询所在使用的容器实例,找到实例id。然后通过root进入到docker中执行命令docker exec -i -t -u 0 e575dad3beae /bin/bash,进去之后重新执行upm build 和 npm start即可。

问题2:

使用Nginx解析到5000端口之后无法登陆。

状态是访问 api目录的时候返回502.
返回结果,报错为:
It seems like we encountered an error. Try refreshing this page or contact your administrator.

使用docker-compose up看日志,没有看到系统的错误日志,返回都是200。那么应该是前端的错误。找到Nginx错误日志,返回以下结果:

2018/11/12 18:57:10 [error] 2002#0: *26219243 upstream sent too big header while reading response header from upstream, client: 61.183.143.20, server: bi2.hansap.com, request: "GET /api/organization/status HTTP/1.1", upstream: "http://*.*.*.*:5000/api/organization/status", host: "*******", referrer: "http://*******/"

从这里看那么就可以说明应该是Proxy配置问题,需要添加配置:

server {

        listen       80;
        server_name  **********;

        gzip on;
        gzip_types *;
        gzip_proxied any;

        location / {
            proxy_pass http://***.***.***.***:5000/;
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
            #proxy_pass_request_body on;
            client_max_body_size 100m;
            proxy_redirect off;
            proxy_buffer_size 64k;
            proxy_buffers   32 32k;
            proxy_busy_buffers_size 128k;
        }
    }

完成之后重新load就可以访问了。

问题3 配置邮件

开发环境的配置文件在 /opt/redash/.env 下,正式环境的配置文件在/opt/redash/env下

但是如果是docker安装,docker会默认读取docker的环境变量,所以在/opt/redash/env下的环境变量不会生效(不知道官网怎么这么写,害人啊) 具体的可以参看

官网 https://redash.io/help/open-source/setup 官网的作为参考,按这个配置不能生效

解决问题的帖子 https://discuss.redash.io/t/where-is-opt-redash-env/999

原因是docker的环境变量需要写在docker compose的配置文件内,内部才能读取。所以这里的env 仅仅只是一个参考,修改了也是没有效果的

具体配置如下:

# /opt/redash/docker-compose.yml文件

version: '2'
services:
  server:
    build: .
    command: dev_server
    depends_on:
      - postgres
      - redis
    ports:
      - "5000:5000"
    volumes:
      - ".:/app"
    environment:
      PYTHONUNBUFFERED: 0
      REDASH_LOG_LEVEL: "INFO"
      REDASH_REDIS_URL: "redis://redis:6379/0"
      REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
      REDASH_MAIL_SERVER: "smtp.hansap.com"
      REDASH_MAIL_PORT: 25
      REDASH_MAIL_USE_TLS: "false"
      REDASH_MAIL_USE_SSL: "false"
      REDASH_MAIL_USERNAME: "*****"
      REDASH_MAIL_PASSWORD: "*****"
      REDASH_MAIL_DEFAULT_SENDER: "hansap@hansap.com"
      REDASH_HOST: "http://bi2.hansap.com"
  worker:
    build: .
    command: scheduler
    volumes_from:
      - server
    depends_on:
      - server
    environment:
      PYTHONUNBUFFERED: 0
      REDASH_LOG_LEVEL: "INFO"
      REDASH_REDIS_URL: "redis://redis:6379/0"
      REDASH_DATABASE_URL: "postgresql://postgres@postgres/postgres"
      QUEUES: "queries,scheduled_queries,celery"
      WORKERS_COUNT: 2
      REDASH_MAIL_SERVER: "smtp.hansap.com"
      REDASH_MAIL_PORT: 25
      REDASH_MAIL_USE_TLS: "false"
      REDASH_MAIL_USE_SSL: "false"
      REDASH_MAIL_USERNAME: "******"
      REDASH_MAIL_PASSWORD: "******"
      REDASH_MAIL_DEFAULT_SENDER: "hansap@hansap.com"
      REDASH_HOST: "http://bi2.hansap.com"
      
      ...