CentOS 7 用 nginx 搭建 WebDav 服务器

标签: none

WebDAV (Web-based Distributed Authoring and Versioning) 一种基于 HTTP 1.1协议的通信协议。它扩展了HTTP 1.1,在GET、POST、HEAD等几个HTTP标准方法以外添加了一些新的方法,使应用程序可直接对Web Server直接读写,并支持写文件锁定(Locking)及解锁(Unlock),还可以支持文件的版本控制。

WebDAV 就是通过 Restful API ,实现对服务端文件的 创建 / 删除 / 读取 / 修改,比起其他文件传输协议,它基于 HTTP,不容易被当作不明流量被砍掉。同时能够利用 HTTP 的各种扩展,比如 HTTPS 提供数据加密功能、HTTP 2.0 提供数据流传输、HTTP 范围请求(RFC7233)等。

正是因为这些好处,很多系统和软件都提供了对 WebDAV 的支持。比如说 OS X 的 finder 支持远程连接到 WebDAV 服务器。IOS 的播放器 nPlayer 能够播放 WebDAV 上的视频文件,且传输速度高于 FTP / SMB 等协议。

旧笔记于 2019年05月01日 整理
本笔记于 2022年01月30日 更新

准备工作

  1. CentOS Linux release 7.9.2009 (Core)
  2. nginx 1.20.2
  3. nginx-dav-ext-module git.r112.f5e3088 (release-v3.0.0) : 提供了额外的 PROPFIND,OPTIONS,LOCK,UNLOCK 方法支持
  4. headers-more-nginx-module git.r259.a4a0686 (v0.33) : IOS、windows等默认客户端使用COPY、MOVE等方法时,$http_destination 中的URI没有带上 /, 导致出现无法删除、重命名文件或文件夹等错误,该模块可以修复该错误,兼容更多的客户端
  5. openssl-1.1.1m # TLS 1.3
  6. zlib 1.2.11 # gzip
  7. pcre 8.45 # 正则表达式
  8. gcc 9.3.1 20200408 (Red Hat 9.3.1-2) 可选

安装编译环境

yum install epel-release expat-devel httpd-tools unzip wget centos-release-scl git libxslt-devel libxml2-devel -y
yum install devtoolset-9-gcc* -y
yum groupinstall "Development tools" -y

创建非特权服务帐户

创建一个与 EPEL 仓库中相同的 nginx 服务帐户:

[root@centos7 ~]# groupadd -g 994 nginx
[root@centos7 ~]# useradd -g 994 -u 996 -c "nginx user" -d /var/cache/nginx -s /sbin/nologin nginx

如果你得到下面的输出,你可以安全地继续前进。这只是意味着 nginx 用户已经或已经从仓库安装时创建。

[root@centos7 ~]#  groupadd -g 994 nginx
groupadd: group 'nginx' already exists
[root@centos7 ~]# useradd -g 994 -u 996 -c "Nginx web server" -d /var/lib/nginx -s /sbin/nologin nginx
useradd: user 'nginx' already exists

准备源码包

# Create Directory
mkdir nginx-webdav
cd nginx-webdav

# download nginx 1.20.2 source
wget wget https://nginx.org/download/nginx-1.20.2.tar.gz


# download pcre 8.45 / zlib 1.2.11 / openssl 1.1.1m dependency
wget https://sourceforge.net/projects/pcre/files/pcre/8.45/pcre-8.45.tar.gz
wget http://zlib.net/zlib-1.2.11.tar.gz
wget http://www.openssl.org/source/openssl-1.1.1m.tar.gz


# download nginx-dav-ext-module git.r112.f5e3088
git clone https://github.com/arut/nginx-dav-ext-module.git

# download headers-more-nginx-module git.r259.a4a0686
git clone https://github.com/openresty/headers-more-nginx-module.git


# Extract source file
tar -zxf pcre-8.45.tar.gz
tar -zxf zlib-1.2.11.tar.gz
tar -zxf openssl-1.1.1m.tar.gz
tar -zxf nginx-1.20.2.tar.gz

准备好的文件列表

[root@centos7 nginx-webdav]# tree -L 1
├── headers-more-nginx-module
├── nginx-1.20.2
├── nginx-1.20.2.tar.gz
├── nginx-dav-ext-module
├── openssl-1.1.1m
├── openssl-1.1.1m.tar.gz
├── pcre-8.45
├── pcre-8.45.tar.gz
├── zlib-1.2.11
└── zlib-1.2.11.tar.gz

6 directories, 4 files

编译安装 Nginx

详细参数配置参数

./configure \
--prefix=/etc/nginx \
--sbin-path=/usr/sbin/nginx \
--modules-path=/usr/lib64/nginx/modules \
--conf-path=/etc/nginx/nginx.conf \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--pid-path=/var/run/nginx.pid \
--lock-path=/var/run/nginx.lock \
--http-client-body-temp-path=/var/cache/nginx/client_temp \
--http-proxy-temp-path=/var/cache/nginx/proxy_temp \
--http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \
--http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \
--http-scgi-temp-path=/var/cache/nginx/scgi_temp \
--user=nginx \
--group=nginx \
--with-zlib=../zlib-1.2.11 \
--with-zlib-opt='-g -Ofast -fPIC -m64 -march=native -fstack-protector-strong -D_FORTIFY_SOURCE=2' \
--with-pcre=../pcre-8.45 \
--with-pcre-opt='-g -Ofast -fPIC -m64 -march=native -fstack-protector-strong -D_FORTIFY_SOURCE=2' \
--with-pcre-jit \
--with-compat \
--with-file-aio \
--with-threads \
--with-http_addition_module \
--with-http_auth_request_module \
--with-http_dav_module \
--add-module=../nginx-dav-ext-module \
--add-module=../headers-more-nginx-module \
--with-openssl=../openssl-1.1.1m \
--with-http_xslt_module \
--with-http_flv_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--with-http_mp4_module \
--with-http_random_index_module \
--with-http_realip_module \
--with-http_secure_link_module \
--with-http_slice_module \
--with-http_ssl_module \
--with-http_stub_status_module \
--with-http_sub_module \
--with-http_v2_module \
--with-mail \
--with-mail_ssl_module \
--with-stream \
--with-stream_realip_module \
--with-stream_ssl_module \
--with-stream_ssl_preread_module \
--with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC' \
--with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'

可选操作步骤, 实现网页目录不裁剪文件名

Snipaste_2022-01-30_17-38-15.png

  1. 修改 nginx-1.20.2/src/http/modules/ngx_http_autoindex_module.c 源码
#define NGX_HTTP_AUTOINDEX_PREALLOCATE  50

#define NGX_HTTP_AUTOINDEX_NAME_LEN     50

修改为

#define NGX_HTTP_AUTOINDEX_PREALLOCATE  110

#define NGX_HTTP_AUTOINDEX_NAME_LEN     110

nginx: Long filenames in directory listing
Stop nginx from trimming file name in directory list?

综上所述整理好的执行代码

[root@centos7 nginx-webdav]# cd nginx-1.20.2

[root@centos7 nginx-1.20.2]# scl enable devtoolset-9 "./configure --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-zlib=../zlib-1.2.11 --with-zlib-opt='-g -Ofast -fPIC -m64 -march=native -fstack-protector-strong -D_FORTIFY_SOURCE=2' --with-pcre=../pcre-8.45 --with-pcre-opt='-g -Ofast -fPIC -m64 -march=native -fstack-protector-strong -D_FORTIFY_SOURCE=2' --with-pcre-jit --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --add-module=../nginx-dav-ext-module --add-module=../headers-more-nginx-module --with-openssl=../openssl-1.1.1m --with-http_xslt_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'"

[root@centos7 nginx-1.20.2]# scl enable devtoolset-9 "make -j"

[root@centos7 nginx-1.20.2]# ./objs/nginx -V
nginx version: nginx/1.20.2
built by gcc 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC) 
built with OpenSSL 1.1.1m  14 Dec 2021
TLS SNI support enabled
configure arguments: ......

[root@centos7 nginx-1.20.2]# make install

安装完检查 nginx 的版本 nginx -V

[root@centos7 ~]# nginx -V
nginx version: nginx/1.20.2
built by gcc 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC) 
built with OpenSSL 1.1.1m  14 Dec 2021
TLS SNI support enabled
configure arguments: ......

如果遇到了依赖库报错可以这样处理

objs/ngx_modules.o \
-Wl,-z,relro -Wl,-z,now -pie -ldl -lpthread -lpthread -lcrypt ../pcre-8.45/.libs/libpcre.a ../openssl-1.1.1m/.openssl/lib/libssl.a ../openssl-1.1.1m/.openssl/lib/libcrypto.a -ldl -lpthread ../zlib-1.2.11/libz.a -lxml2 -lxslt -lexslt \
-Wl,-E
/usr/bin/ld: ../pcre-8.45/.libs/libpcre.a(libpcre_la-pcre_compile.o): relocation R_X86_64_32S against hidden symbol `_pcre_OP_lengths' can not be used when making a shared object
/usr/bin/ld: ../pcre-8.45/.libs/libpcre.a(libpcre_la-pcre_config.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: ../pcre-8.45/.libs/libpcre.a(libpcre_la-pcre_exec.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: ../pcre-8.45/.libs/libpcre.a(libpcre_la-pcre_fullinfo.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: ../pcre-8.45/.libs/libpcre.a(libpcre_la-pcre_jit_compile.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: ../pcre-8.45/.libs/libpcre.a(libpcre_la-pcre_study.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: ../zlib-1.2.11/libz.a(deflate.o): relocation R_X86_64_32S against hidden symbol `_length_code' can not be used when making a shared object
/usr/bin/ld: ../zlib-1.2.11/libz.a(inflate.o): relocation R_X86_64_32S against hidden symbol `zcfree' can not be used when making a shared object
/usr/bin/ld: ../zlib-1.2.11/libz.a(inftrees.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: ../zlib-1.2.11/libz.a(trees.o): relocation R_X86_64_32S against hidden symbol `_length_code' can not be used when making a shared object
/usr/bin/ld: ../zlib-1.2.11/libz.a(zutil.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: ../zlib-1.2.11/libz.a(crc32.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: ../zlib-1.2.11/libz.a(inffast.o): relocation R_X86_64_32S against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
make[1]: *** [objs/nginx] Error 1
make[1]: Leaving directory `../nginx-1.20.2'
make: *** [build] Error 2

如果出现这个错误解决方法是在 ./configure 中加入两行参数

--with-pcre-opt='-g -Ofast -fPIC -m64 -march=native -fstack-protector-strong -D_FORTIFY_SOURCE=2' \
--with-zlib-opt='-g -Ofast -fPIC -m64 -march=native -fstack-protector-strong -D_FORTIFY_SOURCE=2' \

libpcre throwing a ".rodata can not be used when making a shared object" error while compiling NGINX

创建系统服务

[root@centos7 ~]# vim /usr/lib/systemd/system/nginx.service
[Unit]
Description=nginx - high performance web server
Documentation=http://nginx.org/en/docs/
After=network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/var/run/nginx.pid
ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf
ExecReload=/bin/sh -c "/bin/kill -s HUP $(/bin/cat /var/run/nginx.pid)"
ExecStop=/bin/sh -c "/bin/kill -s TERM $(/bin/cat /var/run/nginx.pid)"

[Install]
WantedBy=multi-user.target



[root@centos7 ~]# systemctl enable nginx
[root@centos7 ~]# systemctl start nginx

配置Nginx

站点目录

[root@centos7 ~]# mkdir /etc/nginx/conf.d

修改文件 vim /etc/nginx/nginx.conf

user nginx;

worker_processes auto;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

# Specifies the value for maximum file descriptors that can be opened by this process.
worker_rlimit_nofile 51200;
# PCRE JIT can speed up processing of regular expressions significantly.
pcre_jit on;

events {
    use epoll;
    worker_connections 51200;
    multi_accept on;
}

http {
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
    '$status $body_bytes_sent "$http_referer" '
    '"$http_user_agent" "$http_x_forwarded_for"';

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    server_names_hash_bucket_size 128;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;
    client_max_body_size 50m;

    charset utf-8;
    sendfile on;
    server_tokens off;
    tcp_nodelay on;
    tcp_nopush on;
    real_ip_header X-Forwarded-For;
    types_hash_max_size 2048;
    keepalive_timeout 60;
    access_log /var/log/nginx/access.log main;


    fastcgi_connect_timeout 300;
    fastcgi_send_timeout 300;
    fastcgi_read_timeout 300;
    fastcgi_buffer_size 64k;
    fastcgi_buffers 4 64k;
    fastcgi_busy_buffers_size 128k;
    fastcgi_temp_file_write_size 256k;

    gzip on;
    gzip_min_length 1k;
    gzip_buffers 4 16k;
    gzip_http_version 1.1;
    gzip_comp_level 2;
    gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript;
    gzip_vary on;
    gzip_proxied expired no-cache no-store private auth;
    gzip_disable "MSIE [1-6]\.";

    server {
        listen 80 default_server;
        listen [::]:80 default_server;

        if ($host ~ "\d+\.\d+\.\d+\.\d+") {
            return 404;
        }
        return 301 https://$host$request_uri;
    }

    include /etc/nginx/conf.d/*.conf;
}

修改文件 vim /etc/nginx/conf.d/webdav.conf

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name example.com;

    ssl_certificate "/home/SSL/example.com.crt";
    ssl_certificate_key "/home/SSL/example.com.key";
    ssl_session_cache shared:SSL:20m;
    ssl_session_timeout 30m;
    ssl_session_tickets off;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-256-GCM-SHA384:TLS13-AES-128-GCM-SHA256:EECDH+CHACHA20:EECDH+AESGCM:EECDH+AES;
    ssl_prefer_server_ciphers on;

    access_log /var/log/nginx/access-example.com.log main;
    error_log /var/log/nginx/error-example.com.log error;

    location / {
        # 设置webdav目录,注意Nginx worker用户对该目录需有读/写/执行权限
        root /home/qaq;

        auth_basic "closed site";
        auth_basic_user_file /home/SSL/webdav.htpasswd;

        dav_methods PUT DELETE MKCOL COPY MOVE;
        dav_ext_methods PROPFIND OPTIONS;

        # 启用完整的创建目录支持
        create_full_put_path on;
        dav_access user:rw group:rw;

        autoindex on;
        autoindex_localtime on;
        autoindex_exact_size off;
        # 不限制文件大小
        client_max_body_size 0;

        # 为各种方法的URI后加上斜杠,解决各平台webdav客户端的兼容性问题
        set $dest $http_destination;
        if (-d $request_filename) {
            rewrite ^(.*[^/])$ $1/;
            set $dest $dest/;
        }

        if ($request_method ~ (MOVE|COPY)) {
            more_set_input_headers 'Destination: $dest';
        }

        if ($request_method ~ MKCOL) {
            rewrite ^(.*[^/])$ $1/ break;
        }
    }

    # Mac挂载webdav后会自动写入很多文件,可以通过nginx配置屏蔽掉,保持webdav目录的干净
    location ~ \.(_.*|DS_Store|Spotlight-V100|TemporaryItems|Trashes|hidden|localized)$ {
        access_log off;
        error_log off;

        if ($request_method = PUT) {
            return 403;
        }
        return 404;
    }

    location ~ \.metadata_never_index$ {
        return 200 "Don't index this drive, Finder!";
    }
}

本套配置经过4篇文章整理而成

  1. 开启 TLSv1.2 TLSv1.3, 并且平衡了兼容性和安全性. SSL Labs 评分为 A
  2. 开启 http 基本验证
  3. dav 权限设置为 770 dav_access user:rw group:rw;
  4. 启用 nginx 自带的网页目录列表
  5. 修复某些客户端路径结尾没有斜杠
  6. 屏蔽 Mac 挂载 webdav 后会自动写入很多文件

Snipaste_2022-01-31_06-54-20.png

此段配置作用

set $dest $http_destination;
if (-d $request_filename) {
    rewrite ^(.*[^/])$ $1/;
    set $dest $dest/;
}
if ($request_method ~ (MOVE|COPY)) {
    more_set_input_headers 'Destination: $dest';
}

if ($request_method ~ MKCOL) {
    rewrite ^(.*[^/])$ $1/ break;
}

处理的是某些 WebDAV 客户端(如 OS X 下的 ForkLift) 在 创建文件夹 / 复制文件夹 / 移动文件夹 失败的问题:

2018/06/03 08:03:58 [error] 7#7: *1 MKCOL can create a collection only, client: 8.8.8.8, server: example.com, request: "MKCOL /sp HTTP/2.0", host: "example.com"
2018/06/03 08:17:20 [error] 7#7: *18 "/sp" is collection, client: 8.8.8.8, server: example.com, request: "MOVE /sp HTTP/2.0", host: "example.com"

这是因为 nginx 创建的 WebDAV server 要求所有对文件夹的操作,路径结尾都要有斜杠 / 。而某些客户端在对文件夹进行操作时路径结尾没有斜杠 /,于是报错。 解决方法是 url rewrite,如果操作的是文件夹,则补上 /。

但对于 MOVE / COPY 操作,文件路径还会设置到 header 的 Destination 中,也需要 rewrite ,这需要借助 headers-more-nginx-module 模块。

webdav支持以下方法:

  • OPTIONS : 检索服务
  • GET :获取文件
  • PUT、POST : 上传文件
  • DELETE : 删除文件或集合
  • COPY : 复制文件
  • MOVE : 移动文件
  • MKCOL : 创建由一个或多个文件 URI 组成的新集合
  • PROPFIND : 获取属性(创建日期、文件作者等),实现文件的查找与管理
  • LOCK、UNLOCK : 添加、删除文件锁,实现写操作保护

使用 curl 测试时,URI后面一定要带 / , 不然就会报错

创建目录

curl -X MKCOL -u USER:PASSWORD https://dav.example.com/test/ 

上传文件

curl -T FILE -u USER:PASSWORD  https://dav.example.com/test/

重命名

curl -X MOVE -u USER:PASSWORD --header 'Destination:https://dav.example.com/test/newname' https://dav.example.com/test/File

删除

curl -X DELETE -u USER:PASSWORD  https://dav.example.com/test/File

此时,安装了Nginx并配置了基本的WebDav服务器。如果您在Web浏览器中浏览到新的WebDav实例,您应该能够看到该目录中任何现有文件的目录列表,但您将无法通过Web浏览器上载任何新文件。需要使用WebDav客户端(移动应用程序,Windws / Mac / Linux应用程序等),这是管理这些文件的理想方式。

创建 htpasswd 用户

[root@centos7 ~]# yum install httpd-tools -y
[root@centos7 ~]# htpasswd -c /home/SSL/webdav.htpasswd 'rbq'

都配置完之后重启 nginx

[root@centos7 ~]# nginx -s reload

如果目录在 /home 目录 需要相互添加用户组 把 nginx 用户添加到 qaq 用户组里面, qaq 添加到 nginx 组不然无权访问.

[root@centos7 ~]# id qaq
uid=1000(qaq) gid=1000(qaq) groups=1000(qaq)

[root@centos7 ~]# id nginx
uid=996(nginx) gid=994(nginx) groups=994(nginx)

[root@centos7 ~]# usermod -G qaq nginx
[root@centos7 ~]# usermod -G nginx qaq

[root@centos7 ~]# id nginx
uid=996(nginx) gid=994(nginx) groups=994(nginx),1000(qaq)

#并且需要把文件夹权限修改为 774
[root@centos7 ~]# chmod -R 774 /home/qaq

文章安装贡献价值排序
编译贡献原文
在 CentOS 7 上从源代码安装 Nginx
How to Build Nginx from source on CentOS 7
Centos7.x 编译安装全功能的Nginx
配置贡献原文 - 仅配置部分可供参考 其余部分没啥用
nginx配置功能完整的webdav服务器
使用 nginx 搭建 WebDAV 服务器
Install Nginx as a WebDav File Server on CentOS 7
如何在 Nginx 上启用 TLS 1.3
报错解决原文
libpcre throwing a ".rodata can not be used when making a shared object" error while compiling NGINX
nginx: Long filenames in directory listing
Stop nginx from trimming file name in directory list?


扫描二维码,在手机上阅读!

添加新评论