index.json

[{"content":"Background I have a freebsd laptop at home but it doesn\u0026rsquo;t have a public ip so I can\u0026rsquo;t access it remotely, I also have a server so I\u0026rsquo;m trying to achieve access through tailscale networking.\nInstall Tailscale Here I used pkg to install tailscale\nUpdate pkg 1 pkg update Install Tailscale 1 pkg install tailscale Enable Tailscale ⚠️ It\u0026rsquo;s tailscaled not tailscale\n1 sysrc tailscaled_enable=\u0026#34;YES\u0026#34; Start Tailscale ⚠️ It\u0026rsquo;s tailscaled not tailscale\n1 service tailscaled start Check Tailscale 1 service tailscaled status you should see tailscale is running like this:\n1 tailscaled is running as pid 67663. Connect to server I used headscale as my server, and I tried to connect to it with authkey. For more information on how to install and use headscale, please see the official documentation.\nGenerate authkey We need to generate the authkey on the headscale server first.\n1 headscale --namespace default preauthkeys create --reusable --expiration 24h Then the authkey is like this:\n1 c9bdab8d415f53940f920ee9e729b79ee4e7298cb5acd27d Connect to server 1 tailscale up --login-server=http://\u0026lt;your-server-ip\u0026gt;:41641 --accept-routes=true --accept-dns=false --authkey=c9bdab8d415f53940f920ee9e729b79ee4e7298cb5acd27d Now you have successfully connected to the server. This exposes only that node to other nodes.\nEnable subnet routes Setting up the configuration file /etc/sysctl.conf\n1 2 echo \u0026#39;net.ipv4.ip_forward = 1\u0026#39; \u0026gt;\u0026gt; /etc/sysctl.conf echo \u0026#39;net.ipv6.conf.all.forwarding = 1\u0026#39; \u0026gt;\u0026gt; /etc/sysctl.conf Connect to the server, the subnet range for this node is 192.168.0.1/24.\n1 tailscale up --login-server=http://\u0026lt;your-server-ip\u0026gt;:41641 --accept-routes=true --accept-dns=false --advertise-routes=192.168.0.1/24 --authkey=c9bdab8d415f53940f920ee9e729b79ee4e7298cb5acd27d Get the node list on the server.\n1 headscale node list --namespace default You can see this node on the server. The id of this node is 3.\nEnable subnet routes in the server.\n1 2 # the id is 3, the subnet range is 192.168.0.1/24 headscale routes enable -i 3 -r \u0026#34;192.168.0.1/24\u0026#34; Check if subnet is enabled successfully.\n1 headscale routes list -i 3 we can see that the subnet is enabled like this:\n1 2 Route | Enabled 192.168.0.1/24 | true Viewing nodes on the server 1 headscale node list --namespace default You can see this node on the server.\nUse ping to check connectivity between nodes You can use ping to check connectivity between nodes. My current node\u0026rsquo;s ip is 100.64.0.2 and another node\u0026rsquo;s ip is 100.64.0.1. So I use ping 100.64.0.1 to check connectivity.\n1 ping 100.64.0.1 You can see that the ping is successful like this:\n1 2 3 PING 100.64.0.1 (100.64.0.1): 56 data bytes 64 bytes from 100.64.0.1: icmp_seq=0 ttl=64 time=965.130 ms 64 bytes from 100.64.0.1: icmp_seq=1 ttl=64 time=12.775 ms You can also use tailscale netcheck to view the report of the connectivity.\n1 tailscale netcheck ","permalink":"https://puffinjiang.github.io/posts/solution/networkingwithtailscaleatfreebsd/","summary":"Networking With Tailscale At FreeBSD","title":"Networking With Tailscale At FreeBSD"},{"content":"Background I want to use wget to download the file, so I tried to install wget on ubuntu, but it failed.\nSymptom I use apt to install wget.\n1 apt install wget but received the following：\n1 2 3 4 5 Command \u0026#39;wget\u0026#39; not found, but can be installed with: apt install wget gpg: no valid OpenPGP data found. Reason There is no valid OpenPGP data found\nSoultion Install the digital certificate\n1 apt install ca-certificates ","permalink":"https://puffinjiang.github.io/posts/solution/wgetinstallfailedwithnovalidopenpgponubuntu/","summary":"apt install failed without valid OpenPGP data found","title":"Wget Install Failed With No Valid Openpgp On Ubuntu"},{"content":"Background I previously ran cloudflare tunnel using docker, then updated the latest docker image today, but it won\u0026rsquo;t start after using the new image.\nSymptom I installed and run a connector of cloudflare tunnel with docker, and I ran the docker command provided of the tunnel like this:\n1 docker run cloudflare/cloudflared:latest tunnel --no-autoupdate run --token ${token} But it fails to run and returns the following result：\n1 2 3 4 5 6 7 8 9 10 11 Incorrect Usage: flag provided but not defined: -token NAME: cloudflared tunnel run - Proxy a local web server by running the given tunnel USAGE: cloudflared tunnel run [command options] TUNNEL-ID OPTIONS: --force, -f By default, if a tunnel is currently being run from a cloudflared, you can\u0026#39;t simultaneously rerun it again from a second cloudflared. The --force flag lets you overwrite the previous tunnel. If you want to use a single hostname with multiple tunnels, you can do so with Cloudflare\u0026#39;s Load Balancer product. (default: false) --help, -h show help (default: false) Reason 由于 cloudflared 升级之后，cloudflared tunnel run 命令出现了兼容性问题。旧命令与新版本不兼容，或者是文档未及时更新，回退 cloudflared 镜像版本即可。\nSoultion 修改 docker 镜像的版本为旧版即可，由 cloudflare/cloudflared:latest -\u0026gt; cloudflare/cloudflared:2023.7.0\n启动命令如下：\n1 docker run cloudflare/cloudflared:2023.7.0 tunnel --no-autoupdate run --token ${token} ","permalink":"https://puffinjiang.github.io/posts/solution/cloudflareddockercannotconnect/","summary":"Can not connect to server with cloudflared docker","title":"Cloudflared Docker Can Not Connect"},{"content":"Background I have a windows system hard disk and want to backup its old data to my freebsd service\nInstallation Load fusefs kernel module Load the module\n1 kldload fusefs Setup to load the module at startup\n1 sysrc kld_list+=fusefs Install the NTFS file system 1 pkg install fusefs-ntfs Mount the NTFS disk Get the device info Use dmesg\n1 dmesg | tail Get the device info\n1 2 3 4 5 6 7 8 9 10 umass0 on uhub0 umass0: \u0026lt;Ugreen Ugreen Storage Device, class 0/0, rev 2.10/1.00, addr 1\u0026gt; on usbus0 umass0: SCSI over Bulk-Only; quirks = 0x0100 umass0:4:0: Attached to scbus4 da0 at umass-sim0 bus 0 scbus4 target 0 lun 0 da0: \u0026lt;CT1000MX 500SSD1 0\u0026gt; Fixed Direct Access SPC-4 SCSI device da0: Serial Number 0000000247EA da0: 40.000MB/s transfers da0: 953869MB (1953525168 512 byte sectors) da0: quirks=0x2\u0026lt;NO_6_BYTE\u0026gt; Use camcontrol devlist\n1 camcontrol devlist The device info\n1 2 3 4 5 \u0026lt;TSSTcorp CDDVDW SN-208FB TM00\u0026gt; at scbus0 target 0 lun 0 (cd0,pass0) \u0026lt;HGST HTS721010A9E630 JB0OA3J0\u0026gt; at scbus1 target 0 lun 0 (pass1,ada0) \u0026lt;HFS256G3AMNB-2200A 10108L00\u0026gt; at scbus2 target 0 lun 0 (pass2,ada1) \u0026lt;AHCI SGPIO Enclosure 2.00 0001\u0026gt; at scbus3 target 0 lun 0 (pass3,ses0) \u0026lt;CT1000MX 500SSD1 0\u0026gt; at scbus4 target 0 lun 0 (da0,pass4) As we can see, the device is da0\nShow the part of the disk 1 gpart show da0 We can see the device has 6 partitions.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 =\u0026gt; 34 1953525101 da0 GPT (932G) 34 2014 - free - (1.0M) 2048 1083392 1 ms-recovery (529M) 1085440 204800 2 efi (100M) 1290240 32768 3 ms-reserved (16M) 1323008 207159698 4 ms-basic-data (99G) 208482706 1646 - free - (823K) 208484352 1230848 5 ms-recovery (601M) 209715200 2048 - free - (1.0M) 209717248 1334204416 6 ms-basic-data (636G) 1543921664 409603471 - free - (195G) Mount the part Mount the partition 4 /dev/da0p4 to /mnt/ntfs/part4 1 ntfs-3g /dev/da0p4 /mnt/ntfs/part4 Mount the partition 6 /dev/da0p6 to /mnt/ntfs/part6 1 ntfs-3g /dev/da0p6 /mnt/ntfs/part6 Troubleshooting Metadata kept in Windows cache 1 2 3 4 5 6 Windows is hibernated, refused to mount. The disk contains an unclean file system (0, 0). Metadata kept in Windows cache, refused to mount. Falling back to read-only mount because the NTFS partition is in an unsafe state. Please resume and shutdown Windows fully (no hibernation or fast restarting.) Use ntfsfix to fix the part\n1 ntfsfix /dev/da0 ","permalink":"https://puffinjiang.github.io/posts/learning/mountingntfsonfreebsd/","summary":"Backing up data on my windows system disk to freebsd","title":"Mounting NTFS On FreeBSD"},{"content":"Background I want to deploy the NFS service on FreeBSD and make it available to my mac.\nDevice info macOS OS version: 10.15.7 ip: 192.168.1.88\nusername: puffin FreeBSD OS version: 13.2-RELEASE ip: 192.168.1.99 Configure NFS We configure NFS on the FreeBSD with following steps:\nsetup the /etc/rc.conf setup the file and add configuration of NFS.\nchange the file /etc/rc.conf as follows:\n1 vim /etc/rc.conf add following lines:\n1 2 3 4 5 nfs_server_enable=\u0026#34;YES\u0026#34; nfs_server_flags=\u0026#34;-u -t -n 4\u0026#34; rpcbind_enable=\u0026#34;YES\u0026#34; mountd_flags=\u0026#34;-r\u0026#34; mountd_enable=\u0026#34;YES\u0026#34; setup the /etc/exports change the file /etc/exports as follows:\n1 vim /etc/exports add following lines:\n⚠️ the parameter -mapall=puffin should be the username of my mac, and the ip 192.168.1.88 should be the same as the one in my mac.\n1 /mnt/data/nfs -mapall=puffin -alldirs 192.168.1.88 Start NFS Start Service 1 service nfsd start Reload mountd after change the configuration 1 service mountd reload ","permalink":"https://puffinjiang.github.io/posts/learning/deployingnfsserviceonfreebsd/","summary":"Deploying NFS Service On FreeBSD And Make It Available To My Mac","title":"Deploying NFS Service On FreeBSD"},{"content":"Dockerfile 1 2 3 4 5 6 7 8 9 10 FROM python:3.7.9 RUN sed -i \u0026#34;s@http://deb.debian.org@http://mirrors.aliyun.com@g\u0026#34; /etc/apt/sources.list \u0026amp;\u0026amp; rm -Rf /var/lib/apt/lists/* \u0026amp;\u0026amp; apt-get update \u0026amp;\u0026amp; apt-get install -y supervisor --no-install-recommends ADD . /var/web_backend WORKDIR /var/web_backend RUN pip install -i https://mirrors.aliyun.com/pypi/simple -r requirements.txt COPY daphne.conf /etc/supervisor/conf.d/ RUN chmod u+x ./start.sh EXPOSE 80 ENTRYPOINT [\u0026#34;sh\u0026#34;] CMD [\u0026#34;./start.sh\u0026#34;] Supervisor ⚠️ 注意，这里的 socket=tcp://0.0.0.0:80 不能使用 localhost，否则只能在容器内访问服务。\n下面是我的 supervisor 配置文件 daphne.conf 的内容。配置文件的内容参考自文档\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 [fcgi-program:daphne] # TCP socket used by Nginx backend upstream socket=tcp://0.0.0.0:80 # Directory where your site\u0026#39;s project files are located directory=/var/web_backend user=root # Each process needs to have a separate socket file, so we use process_num # Make sure to update \u0026#34;mysite.asgi\u0026#34; to match your project name command=daphne --fd 0 --access-log - --proxy-headers web_backend.asgi:application # Number of processes to startup, roughly the number of CPUs you have numprocs=4 # Give each process a unique name so they can be told apart process_name=asgi%(process_num)d # Automatically start and recover processes autostart=true autorestart=true # Choose where you want your log to go redirect_stderr=true stdout_logfile=/var/log/web_backend/asgi.log stderr_logfile=/var/log/web_backend/asgierror.log 启动脚本 start.sh ⚠️ 注意，这里的 -n 参数一定要加上，让服务以非后台服务的方式启动，否则 docker 服务会一直不停重启。\n启动脚本 start.sh 内容如下：\n1 2 #!bin/bash supervisord -n -c /etc/supervisor/supervisord.conf Use docker-compose docker-compose 配置文件 使用 docker-compose 启动 docker 服务。docker-compose.yml 文件内容如下：\n1 2 3 4 5 6 7 8 9 10 11 12 version: \u0026#39;3\u0026#39; services: asgi_backend: build: . container_name: web_backend ports: - \u0026#34;8080:80\u0026#34; volumes: - /var/log/web_backend:/var/log/web_backend environment: - DJANGO_SETTINGS_MODULE=web_backend.settings.dev restart: always 启动服务 1 docker-compose -f docker-compose.yml up -d ","permalink":"https://puffinjiang.github.io/posts/learning/deploydjangoasgiserverwithsupervisoranddaphneindocker/","summary":"Deploy django asgi server with supervisor and daphne in docker and managed with docker-compose","title":"Deploy Django Asgi Server With Supervisor And Daphne In Docker"},{"content":"Background I have a remote server and have reinstalled the system via a browser.\nSymptom After I added the ssh public key to the new remote server from my browser, I tried to login from my computer using the terminal, but it failed and showed the following error:\n1 2 3 4 5 6 7 8 9 10 11 12 13 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the ED25519 key sent by the remote host is SHA256:7ODakED63JRWjxUNhSEYZb5K7evWP4eeYwh. Please contact your system administrator. Add correct host key in /Users/username/.ssh/known_hosts to get rid of this message. Offending ECDSA key in /Users/username/.ssh/known_hosts:41 Host key for 192.168.0.100 has changed and you have requested strict checking. Host key verification failed. Reason When we connect to the server via SSH, ssh will get a fingerprint of the ECDSA key and save it to the home directory under ~/.ssh/known_hosts at the first time. SSH will query the file every time you connect to that server after that. After I have changed the server, the fingerprint has been changed, so it\u0026rsquo;s not the same with the local in the file known_hosts.\nSoultion Use ssh-keygen Both hostname and ip will work after ssh-keygen -R\n1 ssh-keygen -R 192.168.0.100 Remove the record in the file ~/.ssh/known_hosts Open the file ~/.ssh/known_hosts and remove the record about the ip 192.168.0.100.\n","permalink":"https://puffinjiang.github.io/posts/solution/sshfailedwithserverchanged/","summary":"ssh login failed after server have been changed","title":"Ssh Failed With Server Changed"},{"content":"Get Mimetypes Of File In Python with module mimetypes The mimetypes module converts between a filename or URL and the MIME type associated with the filename extension.\n1 2 3 4 5 6 7 8 import mimetypes def get_mimetype(path): \u0026#34;\u0026#34;\u0026#34; get mime type of file with file path \u0026#34;\u0026#34;\u0026#34; mime_type, _ = mimetypes.guess_type(path) return mime_type ","permalink":"https://puffinjiang.github.io/posts/summary/getmimetypesoffileinpython/","summary":"get mimetypes of file in python","title":"Get Mimetypes Of File In Python"},{"content":"有时候会有一些使用国外手机号的需求，记录一下。\n英国的 GiffGaff 资费以及激活流程\n","permalink":"https://puffinjiang.github.io/posts/summary/foreigncellphonecards/","summary":"Usage of Foreign Cell Phone Cards","title":"ForeignCellPhoneCards"},{"content":"Background We\u0026rsquo;re Installing Seafile Using Docker Compose and Employing Nginx for Reverse Proxy.\nOS: Ubuntu 22.04.2 LTS Seafile Version: Docker Community Edition 8.0.7 Nginx Version: nginx/1.18.0 (Ubuntu)\nSymptom I am downloading large files in seafile with file size of 2.3GB and after some time it says download failed. But I can successfully download small files.\nAnalysis Process Determine the direction of troubleshooting based on the phenomenon I\u0026rsquo;ve successfully downloaded many small files of a few tens of MB in the past, so I think the reason for the error has to do with the file size! First I tried to download another 1.5G file and it failed. Then I made several more attempts and found that the browser\u0026rsquo;s file download progress bar went close to 1GB with an error. View nginx logs I started looking at the nginx error logs and saw the error logs. 1 upstream prematurely closed connection while reading upstream, client: 192.168.5.65... I\u0026rsquo;m downloading the file on the server without going through nginx, it worked fine. So I\u0026rsquo;ve decided the problem is with nginx. I tried to search nginx download large file 1G and found nginx-does-not-serve-large-files I found proxy_max_temp_file_size and proxy_buffering in ngx_http_proxy_module. then I confirmed the default size of proxy_max_temp_file_size is 1GB. I tried to set the proxy_max_temp_file_size 0; and proxy_buffering off; in the conf of nginx and download the file again. The file was successfully downloaded. Solution Change proxy_buffering off, add proxy_buffering off; to the conf of nginx.\n1 2 3 4 5 6 7 8 9 10 11 server { ... proxy_buffering off; proxy_max_temp_file_size 0; access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ... location / { proxy_pass http://seafile; } } ","permalink":"https://puffinjiang.github.io/posts/solution/seafilelargefiledownloadfailure/","summary":"Seafile Large File Download Failure with Nginx Reverse Proxy Configuration","title":"Seafile Large File Download Failure"},{"content":"背景 准备把所有的 http 服务全部升级到 https。如果使用服务商提供的 ssl 免费证书服务，都有一定的限制，如腾讯云证书只有 1 年，阿里云证书有效期 3 个月等等。想找一个配置完成后就不需要再关注的方式，于是找到了 acme。\n安装 使用如下命令安装：\n1 curl https://get.acme.sh | sh -s email=my@example.com 此命令会将 acme.sh 安装到你的 home 目录下，通过 cd 查看安装的相关文件。\n1 cd ~/.acme.sh/ 此命令会自动创建一个定时任务，每天 00:00:00 自动检测所有证书，如果快过期了，会自动更新相关证书。\n证书签发 acme 支持签发单域名证书和多域名证书，我这里签发的是单域名证书（以 www.mydomain.com 为例）。签发证书过程正需要进行域名所有权的验证，有两种验证方式：http 和 dns 验证。\nhttp 验证 http 方式验证需要在网站根目录下放一个文件，来验证域名所有权，完成验证就可以生成证书了。如果是 apache 或者 nginx， acme 还可以智能的从 apache 或 nginx 的配置中自动完成验证，不需要你指定网站根目录。我这里使用的是 nginx 模式签发，故略过其他方式，感兴趣的可以看相关文档\n使用 apache 模式签发 略过\n使用 Nginx 模式签发 ⚠️ 此处需保持 nginx 的 80 端口已被监听。\n1 acme.sh --issue -d www.mydomain.com --nginx dns 验证 我这里没有使用 dns 的方式验证，故略过，如果想使用此方式的，可查看相关文档\n将证书安装到 Nginx 默认生成的证书都放在安装目录下： ~/.acme.sh/，不要直接使用此目录下的文件，可能会发生变化，我们应该使用 --install-cert 将证书文件复制到目标文件。我这里将文件放到路径 /etc/nginx/cert/ 下 命令如下：\n1 2 3 4 acme.sh --install-cert -d www.mydomain.com \\ --key-file /etc/nginx/cert/www.mydomain.com.key \\ --fullchain-file /etc/nginx/cert/www.mydomain.com.pem \\ --reloadcmd \u0026#34;service nginx force-reload\u0026#34; 问题 网络问题导致下载 acme.sh 超时 使用如下方式下载：\n1 2 3 git clone https://gitee.com/neilpang/acme.sh.git cd acme.sh ./acme.sh --install -m my@example.com Nginx 配置 80 rewrite 导致证书签发失败 原配置如下：\n1 2 3 4 5 server { listen 80; server_name www.mydomain.com; rewrite ^(.*)$ ${server_name}$1 permanent; } 修改后的配置：\n1 2 3 4 5 6 7 8 server { listen 80; server_name www.mydomain.com; #rewrite ^(.*)$ ${server_name}$1 permanent; location / { return 301 https://$host$request_uri; } } ","permalink":"https://puffinjiang.github.io/posts/solution/applyingforansslcertificatewithacme/","summary":"使用 acme 配置 免费的 ssl 证书并自动更新","title":"Applying For An Ssl Certificate With Acme"},{"content":"Install base environment OS: Ubuntu 18.04.6 LTS\nCurrent Nginx: 1.18.0\nAdd Nginx repo 1 2 3 4 echo \u0026#34;deb http://nginx.org/packages/mainline/ubuntu `lsb_release -cs` nginx\u0026#34; \\ | sudo tee /etc/apt/sources.list.d/nginx.list curl -o /tmp/nginx_signing.key https://nginx.org/keys/nginx_signing.key sudo mv /tmp/nginx_signing.key /etc/apt/trusted.gpg.d/nginx_signing.asc Install Nginx 1 2 sudo apt update sudo apt install nginx Restart Nginx check the status of nginx\n1 sudo systemctl status nginx.service restart nginx\n1 sudo systemctl restart nginx.service ","permalink":"https://puffinjiang.github.io/posts/solution/upgradingnginxtothelatestonubuntu/","summary":"Install base environment OS: Ubuntu 18.04.6 LTS Current Nginx: 1.18.0 Add Nginx repo 1 2 3 4 echo \u0026#34;deb http://nginx.org/packages/mainline/ubuntu `lsb_release -cs` nginx\u0026#34; \\ | sudo tee /etc/apt/sources.list.d/nginx.list curl -o /tmp/nginx_signing.key https://nginx.org/keys/nginx_signing.key sudo mv /tmp/nginx_signing.key /etc/apt/trusted.gpg.d/nginx_signing.asc Install Nginx 1 2 sudo apt update sudo apt install nginx Restart Nginx check the status of nginx 1 sudo systemctl status nginx.service restart nginx 1 sudo systemctl restart nginx.service","title":"Upgrading Nginx To The Latest On Ubuntu"},{"content":"Background OS: Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-211-generic x86_64)\nI want use tar to compressing many large files.\nAppearance The compressed file content is close to 1TB, I use the tar -czvf target.tar.gz target/ to compress the target files, it was very slow.\nReason The tar command uses a single thread.\nSolution Compression using the multi-threaded tool pigz\nUse pigz Install pigz 1 apt -y install pigz Compression with pigz 1 tar --use-compress-program=pigz -cvf target.tar.gz target/ Result Using the top command, we can see that the cpu usage is significantly higher, and the tar execution compression process is singificantly accelerated.\n","permalink":"https://puffinjiang.github.io/posts/solution/speeduptarfilecompressionwithpigz/","summary":"Use pigz to accelerate compression of tar files","title":"Speed Up Tar File Compression With Pigz"},{"content":"Background We used the Seafile to store files and folders on the cloud and upload file and folders with the web client.\nAppearance I created a library of other people\u0026rsquo;s data and upload my file and floders. After 10G of data have been uploaded, the remaining data cannot be uploaded, showing out of quota.\nReason The storage quota is 10GB for the user who own the repository.\nSolution Use the admin account to login the seafile and hange the storage quota for the user.\n","permalink":"https://puffinjiang.github.io/posts/solution/exceededstoragequotawhenuploadingtoseafile/","summary":"Reslove Exceeded Storage Quota When Uploading To Seafile","title":"Exceeded Storage Quota When Uploading To Seafile"},{"content":"编写规范清晰的 Commit Message，是使用 Git 过程中很重要的一环。合理的 Commit Message 可以让团队成员快速了解代码变更历史，方便代码审查与追踪Bug。\nCommit Message 的格式 目前有多种 Commit Message 的写法，使用较多的是 Angular 团队的规范，继而演化出了 常规提交 。\ngit Commit Message 的每次提交都包含三个部分，Header、Body 和 Footer。\n1 2 3 4 5 \u0026lt;Header\u0026gt; \u0026lt;Body\u0026gt; \u0026lt;Footer\u0026gt; 其中的 Header 是必须的，Body 和 Footer 可以省略。\nHeader Header 是必填的内容，在 Header 中包含三部分内容，如下：\n1 \u0026lt;type\u0026gt;(\u0026lt;scope\u0026gt;): \u0026lt;description\u0026gt; 其中 和 内容是必填的， 是可选的。\nType 主要为以下几种类型的其中之一。\nfeat: 新功能，表示向代码库中引入了一个新的功能。 fix: 修复，表示修复了代码库中的错误。 docs: 文档更新。 style: 格式改进，不影响代码含义的更新，如空格、缩进、分号等。 refactor: 重构，既不修正错误，也不增加功能的代码更改。 perf: 性能优化，代码的更改可提高性能。 test: 测试，添加新的或改正现有的测试。 build: 影响构建系统或外部依赖项的更改（如 gulp，npm等）。 ci: CI 配置文件的更改。 chore: 其他非 src 和 test 目录的更改（如 .gitingore等）。 Scope 可选内容，主要用来描述改动影响的范围，一般为 项目名/模块名，标识此次改动主要涉及到代码中的哪些模块。\nDescription 必填，对此次改动的简洁描述。\n使用祈使句和现在时态，如 \u0026ldquo;add\u0026rdquo;、\u0026ldquo;change\u0026rdquo;。 首字母不要大写。 结尾不加句号。 Body 可选，针对本次 commit 的详细描述，使用祁使语句和现在时态，如 \u0026ldquo;add\u0026rdquo;、\u0026ldquo;change\u0026rdquo;。内容较多时需要注意合理换行。需要清晰的表达此次变动的范围以及可能影响的内容。\nFooter 可选，Footer 只用于两种情况\n不兼容变动 如果当前代码与上一个版本不兼容，则 Footer 部分已 BREAKING CHANGE 开头，后面是对变动的描述、以及变动理由和迁移方法。\n关闭 Issue 如果当前 commit 针对某个 issue， 可以在 Footer 中关闭这个 issue。\n1 CLoses #123, #456 提交工具 具体可以参考 Conventional Commits 的工具列表\n","permalink":"https://puffinjiang.github.io/posts/solution/gitcommitguidelines/","summary":"Git Commit Message 指南","title":"Git Commit Message Guidelines"},{"content":"背景 使用 gitlab-runner 做 CI/CD 使用 docker 的方式启动 gitlab-runner gitlab-ci.yml 中使用 docker dind 的方式打包和构建镜像 我们使用了自建的 Maven 仓库以及 Docker 镜像仓库。 现象 每次在推送代码到指定分支时，一次完整的 CI/CD 流水线耗时在 32～36 分钟。流程中包含两个作业，分别是打包 SpringBoot 项目和构建以及部署 Docker 镜像。\n作业 1：打包 SpringBoot 的过程每次耗时在 17 分钟左右。 作业 2：构建和部署 Docker 镜像耗时约在 16 分钟左右。 作业 1 的问题 通过查看作业 1 的日志发现，主要耗时有两点，分别是：\n下载 maven 依赖包。 上传打包好的 SprintBoot jar 包。 作业 2 的问题 通过查看作业 2 的日志发现，主要耗时也是两点，分别为：\n下载生成的 SpringBoot jar 包。 上传打包好的 Docker 镜像。 原因 Maven依赖包， SpringBoot jar 包以及 docker 镜像的上传或下载的瓶颈都在网络上。\n解决方案 在 gitlab-runner 中配置 maven 仓库、gitlab 以及 docker 仓库走内网访问，加速上传和下载的过程。在 /gitlab-runner/config.toml 配置文件中添加如下内容：\n1 2 [runners.docker] extra_hosts = [\u0026#34;gitlab.example.com:192.168.0.123\u0026#34;, \u0026#34;maven.example.com:192.168.0.124\u0026#34;, \u0026#34;docker.example.com:192.168.0.125\u0026#34;] ","permalink":"https://puffinjiang.github.io/posts/solution/acceleratorofmavenprojectwithgitlabcicd/","summary":"加速 Gitlab CI/CD 中的 Maven 项目","title":"Accelerator Of Maven Project With Gitlab CI CD"},{"content":"解决方案 可查询自己的 IP 地址的网址如下:\n1 2 3 4 5 https://checkip.amazonaws.com https://api.ipify.org https://ident.me 使用 curl 命令查询自己的 IP 地址 1 curl https://checkip.amazonaws.com ","permalink":"https://puffinjiang.github.io/posts/solution/%E5%A6%82%E4%BD%95%E6%9F%A5%E8%AF%A2%E8%87%AA%E5%B7%B1%E7%9A%84-ip-%E5%9C%B0%E5%9D%80/","summary":"如何查询自己的 IP 地址","title":"Check Your Own IP"},{"content":"前置条件 Mysql 数据库版本为 8.0.20 现象 有一个 Web 服务，服务里有一个分页查询的页面，在查询的过程中同一条数据分别出现在了第 1 页和第 2 页。\n原因 排查思路 对请求进行抓包，发现查询第一页和第二页的参数，除了当前页数不同，无任何其他异常\n排查服务器日志，查看相关的 SQL 语句，SQL 语句大致如下（隐去了部分隐私内容）：\n1 2 SELECT * FROM sys_table ORDER BY sort LIMIT 0,10; SELECT * FROM sys_table ORDER BY sort LIMIT 10,10; 将 sql 语句拿到数据库上运行，查看返回的数据结果。发现返回的数据有问题，两个 sql 查询出的结果出现了重复。\n具体原因 在 sys_table 中的 sort 字段字段，有大量的值都为 0，为同一个值。并且此字段未添加 index，因此在查询的过程中使用了外部排序。可查看官方案例。\n解决方案 在字段 sort 上添加索引 在字段后面再添加一个排序字段，如 id，保证唯一性。 ","permalink":"https://puffinjiang.github.io/posts/solution/dataduplicationinmysqlusingorderbyandlimitpagination/","summary":"在 Mysql 使用 ORDER BY 和 LIMIT 分页时数据重复","title":"Data Duplication In Mysql Using Order By And Limit Pagination"},{"content":"配置 GPG 安装 我这里使用 scoop 来管理常用的软件。\n1 scoop install gpg 查看安装的版本\n1 gpg --version 结果如下：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 gpg (GnuPG) 2.4.3 libgcrypt 1.10.2 Copyright (C) 2023 g10 Code GmbH License GNU GPL-3.0-or-later \u0026lt;https://gnu.org/licenses/gpl.html\u0026gt; This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Home: D:\\Development\\Scoop\\apps\\gpg\\current\\home Supported algorithms: Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH, CAMELLIA128, CAMELLIA192, CAMELLIA256 Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224 Compression: Uncompressed, ZIP, ZLIB, BZIP2 生成新的 GPG 密钥 输入如下命令：\n1 gpg --full-generate-key 上面的命令执行之后，会要求选用对应的加密算法。我们在这里直接回车选择默认的 9 使用 ECC 椭圆曲线加密算法用于加密和解密。提示内容如下:\n1 2 3 4 5 6 7 8 9 10 11 12 13 gpg (GnuPG) 2.4.3; Copyright (C) 2023 g10 Code GmbH This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Please select what kind of key you want: (1) RSA and RSA (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) (9) ECC (sign and encrypt) *default* (10) ECC (sign only) (14) Existing key from card Your selection? 选择完成之后，会提示选用对应的椭圆曲线。我们选择默认的 Curve 25519 椭圆曲线。提示内容如下：\n1 2 3 4 Please select which elliptic curve you want: (1) Curve 25519 *default* (4) NIST P-384 (6) Brainpool P-256 设置过期时间，根据需要设置，我这里选的是 0，设置不过期。\n1 2 3 4 5 6 7 Please specify how long the key should be valid. 0 = key does not expire \u0026lt;n\u0026gt; = key expires in n days \u0026lt;n\u0026gt;w = key expires in n weeks \u0026lt;n\u0026gt;m = key expires in n months \u0026lt;n\u0026gt;y = key expires in n years Key is valid for? (0) 过期时间选择之后，会提示确认上述的操作，直接输入 y 确认。\n设置相关的用户标识。上一步确认之后，会提示输入用户名，邮箱和注释这三个内容，按自己的情况输入即可。输入并确认后 GPG 密钥则生成。\n1 2 3 4 5 6 7 GnuPG needs to construct a user ID to identify your key. Real name: name Email address: name@example.com Comment: You selected this USER-ID: \u0026#34;name \u0026lt;name@example.com\u0026gt;\u0026#34; 查看已经生成的密钥 1 gpg --list-secret-keys --keyid-format=long 显示如下(相关内容已经过处理)：\n1 2 3 4 sec ed25519/5D4C3FXXXXXX2034EF xxxx-xx-xx [SC] D27B94066FFDEDXXXXXXXD4885D4C3F97ASD034EF uid [ultimate] name \u0026lt;name@example.com\u0026gt; ssb cv25519/901FXXXAXX204X8C xxxx-xx-xx [E] 导出公钥 这里公钥的导出以上面的 5D4C3FXXXXXX2034EF 为例，命令如下：\n1 gpg --armor --export 5D4C3FXXXXXX2034EF 显示的公钥格式如下（中间的具体内容以 ··· 代替）：\n1 2 3 4 -----BEGIN PGP PUBLIC KEY BLOCK----- ... -----END PGP PUBLIC KEY BLOCK----- 配置 Git 提交时签名 Github 配置密钥 我们先导出当前要使用的公钥\n点击 Github 右上角的头像，点击 Settings 选项。\n在左侧的页面单击 SSH and GPG keys 选项。\n在 GPG keys 旁边，点击 New GPG key 按钮。\n在新页中的 Title 输入自己的备注信息，在 Key 中填入第一步中导出的公钥。\n点击 Add key 按钮，公钥添加成功。\n本地配置 配置本地 Git 签名使用的密钥 ID，我们这里使用的密钥 ID 为：5D4C3FXXXXXX2034EF (与配置在 Github 里面的保持一致)。\n1 2 # 这里添加 --global 表示设置为全局 git config --global user.signingkey 5D4C3FXXXXXX2034EF 配置默认对所有提交进行签名，命令如下：\n1 git config --global commit.gpgsign true 可以使用 --unset 取消指定的签名 ID。如下：\n1 git config --global --unset user.signingkey ","permalink":"https://puffinjiang.github.io/posts/solution/signingcommitsofgitonwindows/","summary":"在 Windows 上使用 GPG 对 Git 提交进行签名","title":"Signing Commits Of Git On Windows"},{"content":"前置依赖 添加MongoDB GPG Key\n1 2 3 curl -fsSL https://pgp.mongodb.com/server-4.4.asc | \\ sudo gpg -o /usr/share/keyrings/mongodb-server-4.4.gpg \\ --dearmor 添加mongodb列表，我们使用的系统是Ubuntu 20.04 (Focal)\n1 echo \u0026#34;deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-4.4.gpg ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/4.4 multiverse\u0026#34; | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list 更新系统软件包\n1 sudo apt-get update MongoDB Server 安装 安装服务\n1 sudo apt-get install -y mongodb-org 配置 ⚠️ 在配置的过程中尽量使用 DNS 主机名的方式而不是IP地址，防止 IP 改变导致需要更改配置\n我们这里的副本集是一个 3 节点的副本架构，3 个节点的内网 IP 分别为 192.168.1.6 、 192.168.1.7、192.168.1.8 ，对应的 hostname 分别为 mongo1、mongo2、mongo3\n新增主机 IP 地址映射 分别在三台主机下的 /etc/hosts 添加映射\n打开 /etc/hosts 文件\n1 sudo vim /etc/hosts 在文件中添加如下内容\n1 2 3 192.168.1.6 mongo1 192.168.1.7 mongo2 192.168.1.8 mongo3 在三台机器上分别使用 ping 命令测试添加的映射是否生效\n1 2 3 ping mongo1 ping mongo2 ping mongo3 配置 mongod.conf 配置文件 MongoDB 的配置文件路径为 /etc/mongod.conf，这里我们主要配置3个参数。\n配置 bindIp 来控制那些人可以访问 mongodb 服务，在 /etc/mongod.conf 添加如下内容\n1 2 3 net: port: 27017 # 修改了 port 内容，不使用默认端口加强安全性 bindIp: 0.0.0.0 # 配置允许远程连接访问 配置 replSetName 参数，注意三个服务的配置文件里，这个值是一摸一样的！\n1 2 replication: replSetName: rs0 配置 keyFile 参数让副本成员质检通过 keyFile 进行身份校验\n生成 keyfile 文件到 /data/mongo/mongodb.key\n1 openssl rand -base64 756 \u0026gt; /data/mongo/mongodb.key 在 /etc/mongod.conf 中配置 keyFile\n1 2 security: keyFile: /data/mongo/mongodb.key 到此，配置完成。\n启动 我们这里使用的是 systemd(systemctl) 方式，service 方式可参考官方文档。分别在三个节点中执行如下命令启动 mongo\n1 sudo service mongod start 初始化集群 使用 mongo 连接到服务\n1 mongo 127.0.0.1:27017 初始化集群\n1 2 3 4 5 6 7 8 rs.initiate( { _id : \u0026#34;rs0\u0026#34;, members: [ { _id: 0, host: \u0026#34;mongo1:27017\u0026#34; }, { _id: 1, host: \u0026#34;mongo2:27017\u0026#34; }, { _id: 2, host: \u0026#34;mongo3:27017\u0026#34; } ] }) 集群连接用户创建，这里我们创建用户名为 root 的用户\n1 2 use admin # 切换到 admin 用户 db.createUser({user:\u0026#39;root\u0026#39;, pwd:\u0026#39;xxxx\u0026#39;, roles: [\u0026#39;root\u0026#39;]}) # 创建用户名为 root， 密码为 xxxx， 角色为 root 的用户 测试连接 ⚠️ 阿里云等云服务主机需要修改安全组策略开通 27017 服务端口\n使用如下命令进行连通性测试：\n1 mongo mongodb://root:xxxx@mongo1:27017,mongo2:27017,mongo3:27017 ","permalink":"https://puffinjiang.github.io/posts/solution/mongodbreplicasetservicesdeployment/","summary":"部署一个 3 节点的 MongoDB4.4 版本副本集服务","title":"MongoDB4.4 副本集服务部署"},{"content":"安装 基础环境 我使用的系统环境如下：\n1 FreeBSD 13.2-RELEASE releng/13.2-n254617-525ecfdad597 GENERIC 安装 在pkg中搜索 Apache Bench，由于 Apache Bench 是 apache 自带的性能测试软件，所有我们需要安装 apache\n1 pkg search apache 搜索结果如下：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 apache-ant-1.10.13 Java- and XML-based build tool, conceptually similar to make apache-bcel-6.4.1 Apache Commons Byte Code Engineering Library apache-commons-beanutils-1.9.4 JavaBeans utility library apache-commons-cli-1.4 Java library for command line arguments and options apache-commons-codec-1.16.0 Implementations of common encoders and decoders apache-commons-collections-3.2.2 Classes that extend/augment the Java Collections Framework apache-commons-collections4-4.4 Classes that extend/augment the Java Collections Framework apache-commons-compress-1.23.0 Classes for working with various archivers apache-commons-configuration-1.10_1 Java package for the reading of configuration/preferences files apache-commons-csv-1.10.0 Java support for Comma Separated Value format apache-commons-daemon-1.3.2 Wrapper code to start/stop a Java application as a daemon apache-commons-dbcp-2.9.0 Apache Commons Database Connection Pool apache-commons-dbutils-1.7 JDBC helper library apache-commons-digester-2.1_2 XML-to-Java-object mapping utility package apache-commons-httpclient-3.1_2 Package implementing the client side of the HTTP standard apache-commons-io-2.11.0 Collection of I/O utilities for Java apache-commons-jelly-1.0_1 XML based scripting engine apache-commons-jxpath-1.3 Utilities for manipulating Java Beans using the XPath syntax apache-commons-lang-2.6 Apache library with helper utilities for the java.lang API apache-commons-lang3-3.12.0 Apache library with helper utilities for the java.lang API apache-commons-logging-1.2 Generic logging wrapper library apache-commons-math-3.6.1_1 Java library of self-contained mathematics and statistics components apache-commons-net-3.9.0 Collection of network protocol implementations for Java apache-commons-pool-2.11.1 Apache Commons Object-pooling API apache-jmeter-2.11_3 Functional behaviour load and performance test application apache-log4j2-2.20.0 Fast and flexible logging library for Java apache-mode.el-2.0_1 Emacs major mode for editing Apache configuration files apache-openoffice-4.1.14_2 Integrated wordprocessor/dbase/spreadsheet/drawing/chart/browser apache-openoffice-devel-4.2.1678061694_2,4 Integrated wordprocessor/dbase/spreadsheet/drawing/chart/browser (developer version) apache-poi-5.2.3 Java API To Access Microsoft Format Files apache-rat-0.13 Release audit tool apache-solr-9.2.1_1,1 High performance search server built using Lucene Java apache-solr8-8.11.2,1 High performance search server built using Lucene Java apache-spark-3.3.0 Fast big data processing engine apache-xml-security-c-2.0.4 Apache XML security libraries - C++ version apache-xmlbeans-5.1.1_1 Java classes for XML apache24-2.4.57_1 Version 2.4.x of Apache web server apachetop-0.19.7 Apache realtime log stats p5-Apache-ASP-2.63 Active Server Pages for Apache p5-Apache-Admin-Config-0.95_1 Perl module to manipulate Apache configuration files p5-Apache-AuthCookie-3.31 Perl module to provide custom forms for reauthentication p5-Apache-AuthTicket-0.94 Perl modules that implement a cookie-based authentication system p5-Apache-Clean-2.00_4 Mod_perl interface into HTML::Clean p5-Apache-Config-Preproc-1.07 Preprocess Apache server configuration files p5-Apache-ConfigFile-1.23_1 Parse Apache style httpd.conf configuration files p5-Apache-ConfigParser-1.02 Load Apache configuration files p5-Apache-DB-0.18_1 Debugging and profiling tools for mod_perl p5-Apache-DBI-1.12_2 DBI persistent connection, authentication and authorization p5-Apache-Defaults-1.03 Get default settings for Apache httpd daemon p5-Apache-Htgroup-1.23_1 Manage Apache authentication group files p5-Apache-Htpasswd-1.9_2 Manage Unix crypt-style password file p5-Apache-LogFormat-Compiler-0.36 Compile a log format string to perl-code p5-Apache-MP3-4.00_4 MP3 browsing and streaming under mod_perl and Apache p5-Apache-ParseFormData-0.09_3 Module allows you to easily decode/parse form and query data p5-Apache-ParseLog-1.02_1 Perl5 module to parse Apache log files p5-Apache-Profiler-0.10_2 Profiles time seconds needed for every request p5-Apache-Session-1.94 Persistence framework for session data p5-Apache-Session-PHP-0.05_1 Glue Apache::Session with PHP::Session p5-Apache-Session-SQLite3-0.03_1 Use DBD::SQLite 1.x for Apache::Session storage p5-Apache-Session-SharedMem-0.6_1 Apache::Session extension that store session in shared memory p5-Apache-Session-Wrapper-0.34_1 Simple wrapper around Apache::Session p5-Apache-SessionX-2.01_1 Extented persistence framework for session data p5-Apache-Singleton-0.17 Singleton class for mod_perl p5-Apache-Solr-1.09 High level interface to the Solr server p5-Apache2-SOAP-0.73_4 Apache2 mod_perl2 SOAP Server p5-Apache2-SiteControl-1.05_3 Perl web site authentication/authorization system p5-ApacheBench-0.73_1 Perl module for HTTP benchmarking p5-CGI-Application-Plugin-Apache-1.02_2 Provides easy access to Apache::* modules for CGI::Application p5-Catalyst-Engine-Apache-1.16_1 Catalyst Apache Engines p5-Config-ApacheFormat-1.2_2 Parse a configuration file in the same syntax as the Apache web server p5-MasonX-Request-WithApacheSession-0.31_2 Interface to Apache::Session for HTML::Mason p5-Text-MultiMarkdown-ApacheHandler-0.01_2 Module for processing files with MultiMarkdown syntax for Apache p5-XML-ApacheFOP-0.03_3 Access Apache FOP from Perl to create PDF files using XSL-FO py39-apache-arrow-8.0.0_3 Columnar in-memory analytics layer for big data py39-apache-beam-2.43.0_1 Apache Beam SDK for Python py39-apache_conf_parser-1.0.1_1 Parse and manipulate apache conf files py39-certbot-apache-2.6.0 Apache plugin for Certbot rubygem-passenger-apache-6.0.17 Modules for running Ruby on Rails and Rack applications 安装\n从上面搜索的结果来看，pgk 中的 apache 是 apache24-2.4.57_1，安装命令如下：\n1 pkg install apache24 使用 参数说明 我们可以使用 help 查看参数说明：\n1 ab --help 对接口进行压力测试\n1 ab -n 5000 -c 500 http://192.168.0.3:8080/api/v1/hello 返回结果如下：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 This is ApacheBench, Version 2.3 \u0026lt;$Revision: 1903618 $\u0026gt; Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.0.3 (be patient) Completed 500 requests Completed 1000 requests Completed 1500 requests Completed 2000 requests Completed 2500 requests Completed 3000 requests Completed 3500 requests Completed 4000 requests Completed 4500 requests Completed 5000 requests Finished 5000 requests Server Software: # 被测软件信息 Server Hostname: 192.168.0.3 # 被测主机名 Server Port: 8080 # 被测服务端口 Document Path: /api/v1/hello # 测试的文档页面 Document Length: 11 bytes # 测试的文档大小 Concurrency Level: 500 # 并发数 Time taken for tests: 0.795 seconds # 整个测试持续的时间 Complete requests: 5000 # 完成的请求数量 Failed requests: 0 # 失败的请求数量 Total transferred: 640000 bytes # 整个测试中网络传输量 HTML transferred: 55000 bytes # 整个测试中HTML传输量 Requests per second: 6288.87 [#/sec] (mean) # 每秒处理的请求数（请求数）， mean 表示这是一个平均数 Time per request: 79.506 [ms] (mean) # 用户平均请求等待时间 Time per request: 0.159 [ms] (mean, across all concurrent requests) # 服务器平均请求等待时间 Transfer rate: 786.11 [Kbytes/sec] received # 带宽传输速度 Connection Times (ms) min mean[+/-sd] median max Connect: 0 1 2.8 0 16 Processing: 6 74 36.7 63 251 Waiting: 0 74 36.7 63 251 Total: 11 75 37.2 63 258 Percentage of the requests served within a certain time (ms) 50% 63 66% 82 75% 96 80% 105 90% 127 95% 145 98% 169 99% 207 100% 258 (longest request) ","permalink":"https://puffinjiang.github.io/posts/learning/usingapachebenchonfreebsd/","summary":"在 FreeBSD 上使用 Apache Bench 进行性能压力测试","title":"在 FreeBSD 上使用Apache Bench 进行性能压力测试"},{"content":"在Windows下使用Ghostscript压缩pdf文件大小 Ghostscript 下载安装 下载和安装\n官方网站下载地址\n配置环境变量\n在高级系统设置中添加环境变量\n使用\n在 cmd 中使用如下命令进行 pdf 压缩。\n1 gswin64c.exe -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/ebook -dNOPAUSE -dBATCH -sOutputFile=output.pdf input.pdf 问题解决 在powershell中压缩报错 现象 在 powershell 使用上述的压缩命令报错如下：\n1 2 3 4 5 6 GPL Ghostscript 10.01.1 (2023-03-27) Copyright (C) 2023 Artifex Software, Inc. All rights reserved. This software is supplied under the GNU AGPLv3 and comes with NO WARRANTY: see the file COPYING for details. GPL Ghostscript 10.01.1: Device \u0026#39;pdfwrite\u0026#39; requires an output file but no file was specified. **** Unable to open the initial device, quitting. 解决方法 在 cmd 中运行压缩命令而不是在 powershell 中运行\n","permalink":"https://puffinjiang.github.io/posts/solution/pdfcompress/","summary":"压缩PDF文件大小","title":"PDF 压缩"},{"content":"安装的时候，我们使用的是 pkg 包安装的方式。\ninfluxdb 的安装和配置 查找 influxdb 包\n1 pkg search influxdb 搜索结果如下：\n1 2 3 4 influxdb-1.8.10_14 Open-source distributed time series database influxdb2-cli-2.6.1_6 CLI for managing resources in InfluxDB v2 py39-influxdb-5.0.0_1 Python Client for InfluxDB rubygem-influxdb-0.8.1 Ruby library for InfluxDB 安装\n这里选用的是 influxdb-1.8.10_14\n1 pkg install influxdb-1.8.10_14 设置为随系统启动\n1 sysrc influxd_enable=\u0026#34;YES\u0026#34; 启动服务\n1 service influxd start 修改 influxd 配置\n配置文件的路径为 /usr/local/etc/influxd.conf，配置文件详细内容请看文档\n1 vim /usr/local/etc/influxd.conf 修改完成后重启服务\n1 service influxd restart 设置要使用的databse以及用户名和密码\n使用如下命令连接数据库\n1 influx 添加数据库：\n1 CREATE DATABASE mydb 查看数据库：\n1 SHOW DATABASES 创建用户名和密码：\n1 CREATE USER username WITH PASSWORD \u0026#39;password\u0026#39; Grafana 的安装和配置 查找Grafana包\n1 pkg search grafana 结果如下：\n1 2 3 4 5 6 7 8 grafana-loki-2.7.4_2 Scalable, highly-available, multi-tenant log aggregation system grafana8-8.5.26 Dashboard and graph editor for multiple data stores grafana9-9.5.3 Dashboard and graph editor for multiple data stores icingaweb2-module-grafana-php80-1.4.2_1 Add Grafana graphs into Icinga Web 2 to display performance metrics icingaweb2-module-grafana-php81-1.4.2_1 Add Grafana graphs into Icinga Web 2 to display performance metrics icingaweb2-module-grafana-php82-1.4.2_1 Add Grafana graphs into Icinga Web 2 to display performance metrics py39-grafana-dashboard-manager-0.1.3 CLI for saving and restoring Grafana dashboards via HTTP API rubygem-aws-sdk-managedgrafana-1.13.0 Official AWS Ruby gem for Amazon Managed Grafana 安装Grafana\n我这里选用的安装版本为 grafana9-9.5.3 1 pkg install grafana9-9.5.3 设置为系统启动\n1 sysrc grafana_enable=\u0026#34;YES\u0026#34; 启动服务，服务默认端口为 3000\n1 service grafana start 登录服务 http://localhost:3000\n默认的账号密码为：\n1 2 admin admin 且登录后会要求修改密码，请务必使用强密码。\n配置数据源\n登录后点击左上角的 Toggle Menu -\u0026gt; 选择 Connections -\u0026gt; 选择 Connect data\n在右边的输入框中输入 InfluxDB -\u0026gt; 选择搜索出来的 InfluxDB\n点击右上角的 Create a InfluxDB data source 按钮 -\u0026gt; 配置InfluxDB相关的内容。\n在数据源配置页面填写相关的InfluxDB连接信息，需配置内容如下：\n点击Save \u0026amp; Test，显示如下弹窗则表示配置正确。\n选择展示模版，这里通过导入已有模版的方式。我的模版效果如下，更多的可在官方模版库查找\n选择合适的模版，并点击 Load 加载模版 模版加载之后选择对应的数据源，最终确认后的效果如下：\ntelegraf的安装和配置 查找 telegraf 包\n1 pkg search telegraf 结果如下：\n1 telegraf-1.26.0_1 Time-series data collection 安装\n1 pkg install telegraf-1.26.0_1 设置为随系统启动\n1 sysrc telegraf_enable=\u0026#34;YES\u0026#34; 修改配置文件\n配置文件地址: /usr/local/etc/telegraf.conf，在其中添加如下内容：\n1 2 3 4 5 6 # 配置 influxdb连接内容，此处的内容和上面配置的 InfluxDB 数据库连接账号密码 [[outputs.influxdb]] urls = [\u0026#34;http://127.0.0.1:8086\u0026#34;] database = \u0026#34;my_db\u0026#34; username = \u0026#34;username\u0026#34; password = \u0026#34;password\u0026#34; 启动服务\n1 service telegraf start ","permalink":"https://puffinjiang.github.io/posts/learning/telegrafandinfluxdbandgrafanamonitoronfreebsd/","summary":"在FreeBSD上使用Telegraf和InfluxDB以及Grafana监控系统运行情况","title":"FreeBSD上的 Telegraf、InfluxDB 和 Grafana 监控 "},{"content":"安装 InfluxDB 和 Grafana 首先是 Influxdb 和 Grafana 服务的安装。\n使用docker-compose 安装和启动服务 创建 docker-compose.yml 文件 使用 vim 创建文件\n1 vim docker-compose.yml 在文件中添加如下内容：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 version: \u0026#39;3.3\u0026#39; services: influxdb: image: influxdb:latest container_name: influxdb ports: - \u0026#39;8086:8086\u0026#39; volumes: - ./influxdb/storage:/var/lib/influxdb - ./influxdb/conf:/etc/influxdb environment: DOCKER_INFLUXDB_INIT_MODE: setup DOCKER_INFLUXDB_INIT_USERNAME: username DOCKER_INFLUXDB_INIT_PASSWORD: password DOCKER_INFLUXDB_INIT_ORG: telegraf DOCKER_INFLUXDB_INIT_BUCKET: telegraf INFLUXDB_ADMIN_TOKEN: token networks: - monitor grafana: image: grafana/grafana:latest container_name: grafana ports: - \u0026#39;3000:3000\u0026#39; volumes: - ./grafana/storage:/var/lib/grafana - ./grafana/provisioning/:/etc/grafana/provisioning/ depends_on: - influxdb environment: GF_SECURITY_ADMIN_USER: gf_security_admin_username GF_SECURITY_ADMIN_PASSWORD: gf_security_admin_password DOCKER_INFLUXDB_INIT_ORG: telegraf DOCKER_INFLUXDB_INIT_BUCKET: telegraf DOCKER_INFLUXDB_INIT_ADMIN_TOKEN: token networks: - monitor networks: monitor: external: true 启动服务 创建docker网络\n从上面的 docker-compose.yml 文件中我们指定了网络容器网络名为 monitor，在启动服务之前，需手动创建网络，命令如下：\n1 docker network create monitor 在配置文件中可以看到，我们将 grafana 文件挂载到宿主机磁盘上了，可能会产生权限问题，故对 grafana 文件夹授予读写权限，命令如下：\n1 chown 472:472 grafana -R 启动服务，命令如下：\n1 docker-compose -f docker-compose.yml up -d 数据采集 数据的采集是通过 telegraf 来实现，这里给出在 FreeBSD 上安装的案例\n在FreeBSD上安装telegraf 安装命令如下：\n1 2 3 pkg update pkg upgrade pkg install telegraf ","permalink":"https://puffinjiang.github.io/posts/solution/telegrafandinfluxdbmonitor/","summary":"使用Telegraf Influxdb 和 Grafana 进行服务监控","title":"Telegraf And Influxdb Monitor"},{"content":"国内可选源 阿里源\n1 https://{阿里云分配的id}.mirror.aliyuncs.com 网易源\n1 http://hub-mirror.c.163.com 中科大源\n1 https://docker.mirrors.ustc.edu.cn 修改docker配置 docker配置文件位置 /etc/docker/daemon.json， 使用 vim 修改\n1 vim /etc/docker/daemon.json 添加如下内容（这里选择的是中科大源）：\n1 2 3 4 5 { \u0026#34;registry-mirrors\u0026#34;:[ \u0026#34;https://docker.mirrors.ustc.edu.cn\u0026#34; ] } 更新并重启docker服务 更新服务\n1 systemctl daemon-reload 重启服务\n1 systemctl restart docker.service ","permalink":"https://puffinjiang.github.io/posts/learning/modifythesourceofdockeronubuntu/","summary":"在Ubuntu中更新docker源","title":"Modify The Source Of Docker On Ubuntu"},{"content":"安装 我们直接通过包管理工具 pkg 来安装，命令如下：\n1 pkg install nginx 配置 1 2 sysrc nginx_enable=YES service nginx start Nginx配置文件 nginx的配置文件位置： /usr/local/etc/nginx/nginx.conf 默认根目录位置： /usr/local/www/nginx ","permalink":"https://puffinjiang.github.io/posts/learning/usenginxonfreebsd/","summary":"在FreeBSD上使用Nginx","title":"Use Nginx On FreeBSD"},{"content":"问题以及解决方案 使用gitlab-runner运行docker build命令时报错 前置条件 此gitlab-runner由docker启动并运行。\n现象 报错内容如下：\n1 ERROR: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running? 解决方案: 开启docker的 Remote API 访问 2375端口\n⚠️ 此端口无安全认证措施，所以千万不要暴露在公网。\n我这里采用的是修改 /etc/docker/daemon.json 配置文件\n1 vim /etc/docker/daemon.json 在配置文件中添加如下内容\n1 2 3 { \u0026#34;hosts\u0026#34;: [\u0026#34;tcp://0.0.0.0:2375\u0026#34;, \u0026#34;unix:///var/run/docker.sock\u0026#34;] } 在 .gitlab-ci.yml 文件中添加如下内容\n1 2 variables: DOCKER_HOST: tcp://172.17.0.1:2375 更新 Nginx SSL 证书后，CI build 时报错 前置条件 gitlab 以及 gitlab-runner 使用 docker 安装。\n使用 nginx 反向代理 gitlab\nnginx 原有的 ssl 证书即将到期，需要更换新的证书。\n现象 使用gitlab-runner运行docker build命令时报错：\n1 fatal: unable to access \u0026#39;https://gitlab.example.com/example/example.git/\u0026#39;: SSL certificate problem: unable to get local issuer certificate 解决方案 使用新的 nginx 证书替换原来的 gitlab.example.com.crt gitlab.example.com.key 证书.\n进入 gitlab-ce 镜像，执行 gitlab-ctl hup nginx 重新加载证书。\n","permalink":"https://puffinjiang.github.io/posts/solution/gitlabcierror/","summary":"Gitlab使用过程中遇到的问题记录","title":"Gitlab使用过程中遇到的问题记录"},{"content":"远程连接 远程桌面中发送 CTRL+ALT+DEL 命令 直接使用　Ctrl+Alt+Del 案件总是转到本地的调用，无法输入到远程计算机，如果要输入到远程则使用　Ctrl+Alt+End　按键。\n","permalink":"https://puffinjiang.github.io/posts/learning/windowsusagetips/","summary":"记录在Windows 使用过程中的一些小技巧","title":"Windows 使用小技巧"},{"content":"解决方案 1 2 3 4 git checkout upstreambranch git branch master upstreambranch -f git checkout master git push origin master -f 参考了 stackoverflow 上的答案\n","permalink":"https://puffinjiang.github.io/posts/solution/githubbroswercannotaddpullrequests/","summary":"解决方案 1 2 3 4 git checkout upstreambranch git branch master upstreambranch -f git checkout master git push origin master -f 参考了 stackoverflow 上的答案","title":"Github Broswer Can Not Add Pull Requests"},{"content":"安装 下载相应的 podman 安装包并安装， 当前版本是 podman-4.5.0-setup.exe ，Github下载地址\n设置 现在我们的 podman 已经成功的安装到电脑上面了，接下来开始使用。\n初始化 使用下面的命令初始化 podman\n1 podman machine init 启动 podman 通常情况下，我们只需要执行如下的命令就可以正常启动和使用 podman 了\n1 podman machine start 但是某些时刻，可能需要root权限才能支持，这个时候可以在 podman 启动之前先执行如下命令启用 podman 的 root 支持。\n1 podman machine set --rootful 使用 podman 与 docker 命令完全兼容，只需要将 docker 替换成 podman 即可。\n登录 有时我们使用自建或者其他的一些镜像存储平台，需要先登录，使用如下命令：\n1 2 3 4 5 # 登录到 docker.io podman login docker.io # 登录到 example.com podman login example.com 镜像构建 使用 podman 打包镜像。\n1 podman build -f .\\Dockerfile -t example.com/image-name:tag . 推送镜像 我们将镜像打包完成后，可能会推送到 harbor 之类的平台。\n1 podman push example.com/image-name:tag ","permalink":"https://puffinjiang.github.io/posts/learning/usepodmanonwindows/","summary":"在 Windows10 下使用 podman 代替 docker","title":"Use Podman On Windows"},{"content":"最近把限制的旧笔记本电脑折腾出来了，刚好又看到了关于FreeBSD相关的文章，就想着安装一个玩玩，也把旧电脑当作简单的服务器用起来，记录一下我的踩坑历程。\n前期的一些准备以及说明 由于我只想安装FreeBSD单系统，准备使用SSD硬盘作为系统盘，使用HDD硬盘做存储。\n设备的一些硬件相关的参数 电脑型号： GE60 2PE-449\nCPU型号： Intel 酷睿i5 4210H\nGPU型号： NVIDIA GeForce GTX 860M＋Intel GMA HD 4600\n内存容量： 16GB（DDR3 1600MHz 8GB x 2）\n硬盘容量： 256GB ssd + 1TB hdd（7200转）\nUSB启动盘的制作 镜像下载（注意：不同的版本选择的镜像不同），在FreeBSD官方下载站选择对应的版本下载，由于我是安装在物理机上面的，故选择后缀为 memstick.img 版本的镜像，具体可看相关文档\n在Windows10系统上使用 rufus 制作USB启动盘\n笔记本Bios U盘启动设置 开机出现MSI的logo时，按DELETE键进入Bios设置 设置Boot Option #1 选项为 USB Hard Disk, 配置路径： Boot -\u0026gt; Boot Option #1 设置Boot mode select 选项为 UEFI and CSM, 配置路径： Boot -\u0026gt; Boot mode select FreeBSD的安装 安装过程中遇到的问题以及解决方案 使用 startx 启动失败 https://forums.freebsd.org/threads/xauthority-why.52903/\n硬盘挂载 系统重启后发现其中一块硬盘没有挂载，使用挂载 mount -w /dev/ada0 /opt/data 报错如下：\n1 mount: /dev/ada0: R/W mount of /opt/data denied. Filesystem is not clean - run fsck.: Operation not permitted 修复命令：\n1 fsck -y /dev/ada0 重新挂载：\n1 mount -w /dev/ada0 /opt/data 设置为系统启动挂载，在 /etc/fstab 文件中添加如下内容，参考文档\n1 /dev/ada0 /opt/data ufs rw 2 2 ","permalink":"https://puffinjiang.github.io/posts/summary/installingfreebsdonlaptop/","summary":"记录一下使用旧笔记本电脑安装FreeBSD的过程，以及其中遇到的一些坑","title":"在笔记本电脑上安装 FreeBSD"},{"content":"定时任务 crontab修改默认编辑器为vim 设置\n1 export VISUAL=vim 使用命令select-editor\n输入 select-editor 后会显示下面的内容，输入2选择vim编辑器\n1 2 3 4 5 6 7 Select an editor. To change later, run \u0026#39;select-editor\u0026#39;. 1. /bin/nano \u0026lt;---- easiest 2. /usr/bin/vim.basic 3. /usr/bin/vim.tiny 4. /bin/ed Choose 1-4 [1]: 添加随系统启动时的任务 1 2 # 系统重启时启动 /prod/backup/backup.sh 脚本 @reboot /bin/bash /prod/backup/backup.sh \u0026gt;\u0026gt; /dev/null 2\u0026gt;\u0026amp;1 设置定时任务 打开定时任务脚本\n1 crontab -e 添加定时任务\n1 2 # 每天3:30执行/prod/backup/backup.sh脚本 30 3 * * * /bin/bash /prod/backup/backup.sh \u0026gt;\u0026gt; /dev/null 2\u0026gt;\u0026amp;1 后台执行命令 使用 nohup 将需要执行的命令放到后台运行，退出终端页不会影响程序的运行。例如后台运行一个start.sh启动脚本。\n1 2 # \u0026amp; 符号表示让命令在后台运行，终端退出后命令依旧执行。 nohup ./start.sh \u0026amp; shell脚本 使用sh时，定义方法写法：\n1 2 3 4 # 不能写成 function hello() hello() { } Ubuntu 服务器 root 密码修改 使用命令 sudo passwd\n1 sudo passwd 按照终端的提示输入新密码\n查看服务pid 使用 ps 命令\n1 2 # server_name 为对应的服务名称 ps aux|grep server_name 使用 pgrep 命令\n1 2 # server_name 为对应的服务名称 pgrep server_name 查看本机端口 查看端口占用情况，可以使用 lsof 和 netstat 命令。lsof是一个查看当前系统打开文件的工具，netstat 是一个查看网络状态的工具。\nnetstat 查看当前所有 tcp 端口\n1 netstat -at 查看当前所有 udp 端口\n1 netstat -au 详细参数请通过 netstat --help 查看。\nlsof 查看所有 tcp 相关端口\n1 lsof -i tcp 更多参数请通过 lsof --help 查看。\n使用 scp 传输文件 从本地往远程传输文件 1 scp local_file remote_username@remote_ip:remote_path 指定端口号传输 使用 -P 参数\n1 scp -P remote_port local_file remote_username@remote_ip:remote_path 从远程往本地传输文件 1 scp remote_username@remote_ip:remote_path local_file ssh key generate ssh key generate ssh key with ed25519 1 ssh-keygen -t ed25519 -f example_ed25519 -C \u0026#34;example@email.com\u0026#34; copy key to remote server use ssh-copy-id 1 ssh-copy-id -i ~/.ssh/example_ed25519.pub remote_username@remote_ip copy pub key and add to the remote server authorized_keys file ","permalink":"https://puffinjiang.github.io/posts/learning/linux%E5%B8%B8%E7%94%A8%E5%B0%8F%E6%8A%80%E5%B7%A7/","summary":"定时任务 crontab修改默认编辑器为vim 设置 1 export VISUAL=vim 使用命令select-editor 输入 select-editor 后会显示下面的内容，输入2选择vim编辑器 1 2","title":"Linux 常用小技巧"},{"content":"现象 开发了一个下载文件的接口，使用 Content-Disposition 指定默认下载附件文件名，但是每次下载默认文件名与指定的不一致\n基础环境 JDK 8 SpringBoot 2.2.5 接口代码 出于某些原因，这里只展示下载相关的代码。\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 import io.swagger.annotations.ApiOperation; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.core.io.FileSystemResource; import org.springframework.http.CacheControl; import org.springframework.http.MediaType; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import java.io.File; import java.net.URLEncoder; import java.util.concurrent.TimeUnit; @RestController @RequestMapping({\u0026#34;/api/v1\u0026#34;}) public class DownloadController { private Logger logger = LoggerFactory.getLogger(getClass()); @GetMapping(\u0026#34;/download/{id}\u0026#34;) @ApiOperation(\u0026#34;Download file by id\u0026#34;) public ResponseEntity\u0026lt;FileSystemResource\u0026gt; downloadFile(@PathVariable(\u0026#34;id\u0026#34;) Long id) throws Exception { String url = String.format(\u0026#34;data/%s.pdf\u0026#34;, id); File file = new File(url); if (!file.isFile()) { logger.error(\u0026#34;ERROR download file, can not find the file, ID: {}, url: {}\u0026#34;, id, url); return ResponseEntity.notFound().build(); } return ResponseEntity .ok() .header(\u0026#34;Content-Disposition\u0026#34;, \u0026#34;attachment; filename=\u0026#34; + URLEncoder.encode(file.getName(), \u0026#34;utf-8\u0026#34;)) .contentType(MediaType.APPLICATION_OCTET_STREAM) .contentLength(file.length()) .cacheControl(CacheControl.maxAge(60, TimeUnit.SECONDS)) .body(new FileSystemResource(file)); } } 具体现象 在点击下载的过程中，如果下载的文件ID为 1 时，默认下载的文件名则为 1, 而非Response header 中设置的filename 1.pdf\n原因排查 确定排查范围 接口已经经过测试，除默认设置的文件名不生效外，暂未发现其他问题。而默认设置的文件名由 .header(\u0026quot;Content-Disposition\u0026quot;, \u0026quot;attachment; filename=\u0026quot; + URLEncoder.encode(file.getName(), \u0026quot;utf-8\u0026quot;)) 这一行控制，则将范围设置为 Response header Content-Disposition 相关的问题\n查询相关信息 在确定了排查范围之后，根据相关的关键词查询信息。\n根据关键词 http response attachment filename 搜索相关的信息，在MDN Web Docs上找到了对于Content-Disposition的描述 MDN Web Docs中描述的格式如下 Content-Disposition: attachment; filename=\u0026quot;filename.jpg\u0026quot; 将接口返回的结果 Content-Disposition: attachment; filename=filename.pdf 与文档描述进行对比，发现 filename= 后面缺失了 \u0026quot; 符号。 修改相关的代码，添加 \u0026quot; 符号，将相关的代码修改如下后进行测试 1 2 3 4 5 6 7 return ResponseEntity .ok() .header(\u0026#34;Content-Disposition\u0026#34;, \u0026#34;attachment; filename=\\\u0026#34;\u0026#34; + URLEncoder.encode(file.getName(), \u0026#34;utf-8\u0026#34;) + \u0026#34;\\\u0026#34;\u0026#34;) .contentType(MediaType.APPLICATION_OCTET_STREAM) .contentLength(file.length()) .cacheControl(CacheControl.maxAge(60, TimeUnit.SECONDS)) .body(new FileSystemResource(file)); 解决方案 修改代码，在 filename= 后面添加 \u0026quot;\n","permalink":"https://puffinjiang.github.io/posts/summary/downloadattachmentfilenameinvalid/","summary":"解决文件下载中附件设置filename不生效问题","title":"Download Attachment Filename Invalid"},{"content":"挂载步骤 将硬盘插入到电脑上\n查看新磁盘是否存在 1 fdisk -l 根据显示结果查找对应的硬盘，这里是 /dev/sde1：\n1 2 3 4 5 6 7 8 9 10 Disk /dev/sde: 1.9 TiB, 2048408248320 bytes, 4000797360 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x636b9949 Device Boot Start End Sectors Size Id Type /dev/sde1 2048 4000794623 4000792576 1.9T 7 HPFS/NTFS/exFAT 挂载硬盘 这里我将硬盘挂载到 /prod/backup 路径下，并授权读写(-rw)\n1 mount -o rw /dev/sde1 /prod/backup 查看磁盘情况 1 df -lh 使用 df -lh 命令查看磁盘是否成功挂载，如下有显示 /dev/sde 则表示正常\n1 2 tmpfs 9.5G 0 9.5G 0% /run/user/0 /dev/sde1 1.9T 951G 957G 50% /prod/backup 卸载挂载点 1 umount /dev/sde1 ","permalink":"https://puffinjiang.github.io/posts/learning/mountingaharddriveinubuntu/","summary":"在Ubuntu下挂载新硬盘","title":"Mounting A Hard Drive In Ubuntu"},{"content":"背景\u0026amp;现象 环境 操作系统：MacOS 工具：Idea、Maven\n现象 Java下的常规父子工程项目，project1依赖 project2。\n对project2单独使用mvn install时，成功 install 对project1单独使用mvn package时 报错如下： 1 Failed to execute goal on project/Could not resolve dependencies for project 原因 解决办法 步骤如下：\n使用父工程的pom文件，使用mvn clean清空相关文件 使用mvn install命令，将相关的jar安装到本地的maven仓库 对具体的project1 使用mvn package命令打包 ","permalink":"https://puffinjiang.github.io/posts/solution/mavenfailtoexecutegoalonproject/","summary":"在父子工程项目中、有依赖关系的子项目打包时找不到依赖","title":"Maven父子工程，子项目相互依赖，打包时找不到依赖"},{"content":"安装virtualbox ❗注意，一定要设置拓展包特性，后续才能使用windows远程桌面连接。Ubuntu默认是VNC的连接方式，通过设置拓展包特性，修改为vrdp的连接方式\n安装virtualbox软件 1 apt-get install virtualbox 安装拓展包 1 apt install virtualbox-ext-pack 检查拓展包是否安装成功：\n1 VBoxManage list extpacks 设置拓展包特性，为后续的远程桌面连接做准备\n1 VBoxManage setproperty vrdeextpack \u0026#34;Oracle VM VirtualBox Extension Pack\u0026#34; 创建虚拟机，并安装操作系统 创建一个虚拟机，名称为：guest_os_1，操作系统为windows10\n创建虚拟机使用的虚拟硬盘，文件路径为 /prod/vmos/guest_os_1/guest_os_1.vdi，大小为100G，即102400m\n1 VBoxManage createmedium --filename /prod/vmos/guest_os_1/guest_os_1.vdi --size 102400 创建虚拟机，名称为：guest_os_1，操作系统为：Windows10_64\n1 VBoxManage createvm --name \u0026#34;guest_os_1\u0026#34; --ostype Windows10_64 --register 如果需要安装其他类型的操作系统，可以通过如下命令查看：\n1 VBoxManage list ostypes 创建stat磁盘控制器，并和步骤一中的虚拟硬盘绑定\n1 VBoxManage storagectl guest_os_1 --name \u0026#34;SATA Controller\u0026#34; --add sata --controller IntelAHCI 绑定步骤一中的虚拟硬盘\n1 VBoxManage storageattach guest_os_1 --storagectl \u0026#34;SATA Controller\u0026#34; --port 0 --device 0 --type hdd --medium /prod/vmos/guest_os_1/guest_os_1.vdi 创建IDE控制器，设置为DVD，并和下载好的Windows10 ISO文件绑定\n1 VBoxManage storagectl guest_os_1 --name \u0026#34;IDE Controller\u0026#34; --add ide 绑定windows iso镜像文件，镜像文件路径为：/prod/iso/zh-cn_windows_10_business_editions_version_22h2_updated_nov_2022_x64_dvd_e310fb02.iso\n1 VBoxManage storageattach guest_os_1 --storagectl \u0026#34;IDE Controller\u0026#34; --port 1 --device 0 --type dvddrive --medium \u0026#34;/prod/iso/zh-cn_windows_10_business_editions_version_22h2_updated_nov_2022_x64_dvd_e310fb02.iso\u0026#34; 给虚拟机分配cpu和内存，以分配4个cpu和4g内存为例：\n1 VBoxManage modifyvm guest_os_1 --cpus 4 --memory 4096 设置虚拟机的启动顺序，首先是硬盘，其次是dvd\n1 VBoxManage modifyvm guest_os_1 --boot1 disk --boot2 dvd 打开和关闭VRDE远程连接功能，默认端口为3389\n开启VRDE远程连接：\n1 VBoxManage modifyvm guest_os_1 --vrde on 关闭VRDE远程连接：\n1 VBoxManage modifyvm guest_os_1 --vrde off 设置为自定义端口，以设置为2233为例：\n1 VBoxManage modifyvm guest_os_1 --vrdeport 2233 # 默认端口为3389 开启虚拟机\n1 VBoxManage startvm guest_os_1 --type=headless 查看虚拟机\n1 2 3 4 5 # 查看所有的虚拟机 VBoxManage list vms # 查看运行中的虚拟机 VBoxManage list runningvms 通过windows远程桌面连接虚拟机，并开始安装windows系统。以宿主机Ubuntu的ip是 192.168.0.66 为例，远程桌面连接的ip和端口应设置为 192.168.0.66:2233 ，此处的端口与步骤7 中设置的 vrdeport 保持一致。\n挂载和安装 VBoxGuestAdditions.iso 镜像\n挂载镜像：\n1 VBoxManage storageattach guest_os_1 --storagectl \u0026#34;IDE Controller\u0026#34; --port 1 --device 0 --type dvddrive --medium \u0026#34;/usr/share/virtualbox/VBoxGuestAdditions.iso\u0026#34; 开启虚拟机，进入驱动器文件下，点击对应的安装文件\nwindows安装文件路径：\n1 D:\\VBoxWindowsAdditions.exe 挂载和删除宿主机的文件夹到虚拟机\n❗注意，此挂载命令在6.1版本才生效\n挂载宿主机文件夹 \u0026lsquo;/prod/data\u0026rsquo; 为虚拟机Windows系统的 \u0026lsquo;Y:/\u0026rsquo;，相关参数作用如下：\n\u0026ndash;name 挂载的文件夹名称 \u0026ndash;hostpath 指定宿主机文件夹 \u0026ndash;readonly 设置虚拟机系统只能读数据，不允许写 1 VBoxManage sharedfolder add guest_os_1 --name \u0026#39;data\u0026#39; --hostpath \u0026#39;/prod/data\u0026#39; --readonly --automount --auto-mount-point \u0026#39;Y:/\u0026#39; 删除挂载\n1 VBoxManage sharedfolder remove guest_os_1 --name \u0026#39;data\u0026#39; 删除虚拟机 关闭虚拟机\n1 VBoxManage controlvm guest_os_1 poweroff 删除虚拟机，命令如下、此命令会删除对应虚拟机的文件及配置信息。\n1 VBoxManage unregistervm --delete guest_os_1 ","permalink":"https://puffinjiang.github.io/posts/learning/installwindowsoswithvirtualboxheadlessunderubuntu/","summary":"Ubuntu下使用无Gui的Virtualbox虚拟机安装Windows操作系统","title":"Ubuntu 下使用 Virtualbox Headless 安装 Windows 系统"},{"content":"记录一些工作过程中经常会使用的Mysql命令\nDDL（Database Definition Language） Create Database\n1 CREATE DATABASE dbname; Create user with password\n1 2 -- 设置 host为 % 时，username可用于远程登录 CREATE user \u0026#39;username\u0026#39;@\u0026#39;host\u0026#39; identified by \u0026#39;password\u0026#39;; Show Database\n1 SHOW DATABASES; Use database\n1 USE dbname; Drop Database\n1 DROP DATABASE dbname; Create Table\n1 2 3 4 5 6 7 8 CREATE TABLE IF NOT EXISTS `demo`( `id` INT UNSIGNED AUTO_INCREMENT, `name` VARCHAR(100) NOT NULL, `password` VARCHAR(40) NOT NULL, `create_time` TIMESTAMP NOT NULL DEFAULT NOW(), `update_time` TIMESTAMP NOT NUll DEFAULT CURRENT_TIMESTAMP(), PRIMARY KEY ( `runoob_id` ) )ENGINE=InnoDB DEFAULT CHARSET=utf8; Change Table\nChange name of column password to passwd\n1 ALTER TABLE demo CHANGE password passwd; Drop Table\n1 DROP TABLE demo; Change Remote root password\nchange password to 123456 (mysql8.0)\n1 ALTER user \u0026#39;root\u0026#39;@\u0026#39;%\u0026#39; identified with mysql_native_password by \u0026#39;123456\u0026#39;; DML add data into table demo\n1 INSERT INTO demo(name,password) VALUES(\u0026#39;name1\u0026#39;,\u0026#39;password1\u0026#39;); Change data\n1 UPDATE demo SET name=\u0026#39;name2\u0026#39; where id = 1; delete data\n1 DELETE FROM demo WHERE id = 1; select data\nselect data by time, accurate to the second\n1 SELECT * FROM demo WHERE timestamp(create_date) BETWEEN \u0026#39;2023-01-01 00:00:00\u0026#39; AND \u0026#39;2023-01-31 00:00:00\u0026#39;; select data by date, accurate to the day\n1 SELECT * FROM demo WHERE date(create_date) BETWEEN \u0026#39;2023-01-01\u0026#39; AND \u0026#39;2023-01-31\u0026#39;; use case when to conversion some select column\n1 SELECT CASE gender WHEN 0 THEN \u0026#39;男\u0026#39; WHEN 1 THEN \u0026#39;女\u0026#39; END AS \u0026#39;性别\u0026#39;, CASE WHEN status IS NULL THEN \u0026#39;未提交\u0026#39; END AS \u0026#39;提交状态\u0026#39; FROM demo; DCL Authorization to users\n1 2 GRANT all privileges on dbname.* to username@\u0026#39;%\u0026#39; identified by password; flush privileges; Others backup backup database with mysqldump\n1 mysqldump -u root -h host -p dbname \u0026gt; backup_file.sql; restore restore database with mysqldump\n1 mysqldump -u root -h host -m dbname \u0026lt; backup_file.sql; ","permalink":"https://puffinjiang.github.io/posts/learning/mysqlcommoncommands/","summary":"记录一些常用的Mysql数据库命令","title":"Mysql Common Commands"},{"content":"记录一下开启Ubuntu服务器的ssh远程连接过程\nSSH服务安装 1. 检查服务是否开启 通过下面命令检查服务是否开启\n1 ps -e | grep sshd 2. 安装服务 1 apt install openssh-server 3. 启用服务 1 systemctl enable ssh 4. 启动服务 1 systemctl start sshd.service 服务配置 配置文件路径 注意： 使用的是sshd而不是ssh，ssh是ssh-client相关的，sshd是ssh-server相关信息，要注意区分！\n1 vim /etc/ssh/sshd_config 配置文件内容及作用 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 # Package generated configuration file # See the sshd_config(5) manpage for details # What ports, IPs and protocols we listen for Port 22 # Use these options to restrict which interfaces/protocols sshd will bind to #ListenAddress :: #ListenAddress 0.0.0.0 Protocol 2 # HostKeys for protocol version 2 HostKey /etc/ssh/ssh_host_rsa_key HostKey /etc/ssh/ssh_host_dsa_key HostKey /etc/ssh/ssh_host_ecdsa_key HostKey /etc/ssh/ssh_host_ed25519_key #Privilege Separation is turned on for security UsePrivilegeSeparation yes # Lifetime and size of ephemeral version 1 server key KeyRegenerationInterval 3600 ServerKeyBits 1024 # Logging SyslogFacility AUTH LogLevel INFO # Authentication: LoginGraceTime 120 #PermitRootLogin prohibit-password PermitRootLogin yes # 允许root账号使用ssh登录 StrictModes yes RSAAuthentication yes PubkeyAuthentication yes #AuthorizedKeysFile %h/.ssh/authorized_keys # Don\u0026#39;t read the user\u0026#39;s ~/.rhosts and ~/.shosts files IgnoreRhosts yes # For this to work you will also need host keys in /etc/ssh_k nown_hosts RhostsRSAAuthentication no # similar for protocol version 2 HostbasedAuthentication no # Uncomment if you don\u0026#39;t trust ~/.ssh/known_hosts for RhostsRSAAuthentication #IgnoreUserKnownHosts yes # To enable empty passwords, change to yes (NOT RECOMMENDED) PermitEmptyPasswords no # Change to yes to enable challenge-response passwords (beware issues with # some PAM modules and threads) ChallengeResponseAuthentication no # Change to no to disable tunnelled clear text passwords #PasswordAuthentication yes # Kerberos options #KerberosAuthentication no #KerberosGetAFSToken no #KerberosOrLocalPasswd yes #KerberosTicketCleanup yes # GSSAPI options #GSSAPIAuthentication no #GSSAPICleanupCredentials yes X11Forwarding yes X11DisplayOffset 10 PrintMotd no PrintLastLog yes TCPKeepAlive yes #UseLogin no #MaxStartups 10:30:60 #Banner /etc/issue.net # Allow client to pass locale environment variables AcceptEnv LANG LC_* Subsystem sftp /usr/lib/openssh/sftp-server # Set this to \u0026#39;yes\u0026#39; to enable PAM authentication, account processing, # and session processing. If this is enabled, PAM authentication will # be allowed through the ChallengeResponseAuthentication and # PasswordAuthentication. Depending on your PAM configuration, # PAM authentication via ChallengeResponseAuthentication may bypass # the setting of \u0026#34;PermitRootLogin without-password\u0026#34;. # If you just want the PAM account and session checks to run without # PAM authentication, then enable this but set PasswordAuthentication # and ChallengeResponseAuthentication to \u0026#39;no\u0026#39;. UsePAM yes ","permalink":"https://puffinjiang.github.io/posts/learning/remoteloginofubuntuserversetting/","summary":"Ubuntu 开启ssh远程连接服务","title":"Remote Login Of Ubuntu Server Setting"},{"content":"记录一下使用docker-compose部署常用的服务\nNginx 1 2 3 4 5 6 7 8 9 10 11 12 13 version: \u0026#39;3.4\u0026#39; services: nginx: image: nginx:alpine #镜像地址 container_name: nginx #容器名 restart: always ports: - \u0026#34;443:443\u0026#34; - \u0026#34;80:80\u0026#34; volumes: - ./conf/cert/:/etc/nginx/cert/ # ssl证书路径 - ./conf/conf.d/:/etc/nginx/conf.d/ # conf配置文件 - ./logs:/var/log/nginx # 日志 Mysql 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 version: \u0026#39;3.4\u0026#39; services: db: image: mysql container_name: mysql8 restart: always environment: MYSQL_ROOT_PASSWORD: password command: --default-authentication-plugin=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci --explicit_defaults_for_timestamp=true --lower_case_table_names=1 # 重要，设置mysql不区分大小写 --max_allowed_packet=128M; ports: - \u0026#34;3306:3306\u0026#34; volumes: - ./data:/var/lib/mysql # 数据文件 - ./conf:/etc/mysql/conf.d # 配置文件 gitlab 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 version: \u0026#39;3.6\u0026#39; services: gitlab: image: \u0026#34;yrzr/gitlab-ce-arm64v8:latest\u0026#34; # 此处使用的是arm64位的镜像，系统是amd架构的直接使用 gitlab/gitlab-ce 即可 container_name: \u0026#34;gitlab\u0026#34; hostname: \u0026#39;gitlab.example.cn\u0026#39; # 域名配置 restart: always environment: GITLAB_OMNIBUS_CONFIG: | external_url \u0026#39;http://gitlab.example.cn:8181\u0026#39; #要与下面ports中的端口对应 gitlab_rails[\u0026#39;gitlab_shell_ssh_port\u0026#39;] = 2222 # gitlab_rails[\u0026#39;smtp_enable\u0026#39;] = true # gitlab_rails[\u0026#39;smtp_address\u0026#39;] = \u0026#34;smtp.aliyun.com\u0026#34; # gitlab_rails[\u0026#39;smtp_port\u0026#39;] = 465 # gitlab_rails[\u0026#39;smtp_user_name\u0026#39;] = \u0026#34;example@example.com\u0026#34; #用自己的aliyun邮箱 # gitlab_rails[\u0026#39;smtp_password\u0026#39;] = \u0026#34;password\u0026#34; # gitlab_rails[\u0026#39;smtp_domain\u0026#39;] = \u0026#34;aliyun.com\u0026#34; # gitlab_rails[\u0026#39;smtp_authentication\u0026#39;] = \u0026#34;login\u0026#34; # gitlab_rails[\u0026#39;smtp_enable_starttls_auto\u0026#39;] = true # gitlab_rails[\u0026#39;smtp_tls\u0026#39;] = true # gitlab_rails[\u0026#39;gitlab_email_from\u0026#39;] = \u0026#39;admin@example.com\u0026#39; ports: - \u0026#39;8181:8181\u0026#39; - \u0026#39;443:443\u0026#39; - \u0026#39;2222:22\u0026#39; volumes: - \u0026#39;/prod/gitlab/config:/etc/gitlab\u0026#39; - \u0026#39;/prod/gitlab/logs:/var/log/gitlab\u0026#39; - \u0026#39;/prod/gitlab/data:/var/opt/gitlab\u0026#39; shm_size: \u0026#39;256m\u0026#39; nexus个人仓库 此处需要注意的是，对于映射的data，需要授予777权限，保证正常的文件夹的创建和文件的写入\n1 2 3 4 5 6 7 8 9 10 version: \u0026#39;3.4\u0026#39; services: nexus: container_name: nexus3 image: sonatype/nexus3 restart: always ports: - \u0026#39;8889:8081\u0026#39; volumes: - /prod/nexus/data:/nexus-data ","permalink":"https://puffinjiang.github.io/posts/learning/deployservicewithdockercompose/","summary":"使用docker-compose部署常用的服务","title":"Deploy Service With Docker Compose"},{"content":"额外开销 每个索引占据一定的存储空间，在进行插入，更新和删除操作时也需要对索引进行操作。所以，如果你很少对集合进行读取操作，建议不使用索引。\n内存(RAM)使用 由于索引是存储在内存(RAM)中,你应该确保该索引的大小不超过内存的限制。\n如果索引的大小大于内存的限制，MongoDB会删除一些索引，这将导致性能下降。\n查询限制 索引不能被以下的查询使用：\n正则表达式及非操作符，如 $nin, $not, 等。 算术运算符，如 $mod, 等。 $where 子句 所以，检测你的语句是否使用索引是一个好的习惯，可以用explain来查看。\n索引键限制 从2.6版本开始，如果现有的索引字段的值超过索引键的限制，MongoDB中不会创建索引。\n插入文档超过索引键限制 如果文档的索引字段值超过了索引键的限制，MongoDB不会将任何文档转换成索引的集合。与mongorestore和mongoimport工具类似。\n最大范围 集合中索引不能超过64个 索引名的长度不能超过128个字符 一个复合索引最多可以有31个字段 ","permalink":"https://puffinjiang.github.io/posts/learning/indexlimitofmongodb/","summary":"使用MongoDB时，索引相关的限制和优化","title":"Limit Of MongoDB Index"},{"content":"Utools未实时更新环境变量配置导致命令行工具使用错误 背景 今天在使用 utools工具打开 powershell 安装 scoop ，scoop 安装成功后，可以使用 scoop 相关命令，重新打开 powershell 后使用 scoop 命令无法识别。\n问题复现 打开utools\n使用utools打开powershell\n使用powershell安装scoop\n在powershell使用scoop help命令\n关闭powershell，使用utools重新打开powershell\n在powershell中使用scoop help命令\n此时出现如下的错误信息：\n1 2 3 4 5 6 7 D : 无法将“D”项识别为 cmdlet、函数、脚本文件或可运行程序的名称。请检查名称的拼写，如果包括路径，请确保路径正确，然后 再试一次。 所在位置 行:1 字符: 1 + D + ~ + CategoryInfo : ObjectNotFound: (D:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException 解决方法 重新启动 utools\n","permalink":"https://puffinjiang.github.io/posts/solution/utoolscannotgetnewenvfromsystem/","summary":"utools无法及时更新最新的系统环境变量，导致在使用powershell等命令行工具时无法读取相应的环境变量","title":"Utools Can not Get New Env From System"},{"content":"背景 今天在我更新完苹果推送的系统更新之后，打开vscode时，弹窗提示 Git无法使用 之类的信息。让我以为更新之后Git被删除了， 于是我使用 brew upgrade git 命令想看一下之前安装过的Git是否还在\n问题 使用 brew install git 命令时，报如下错误：\n1 2 3 4 5 6 7 8 9 10 11 Warning: You are using macOS 13. We do not provide support for this pre-release version. You will encounter build failures with some formulae. Please create pull requests instead of asking for help on Homebrew\u0026#39;s GitHub, Twitter or any other official channels. You are responsible for resolving any issues you experience while you are running this pre-release version. Error: No developer tools installed. Install the Command Line Tools: xcode-select --install 解决方法 运行 xcode-select --install ，此命令会安装会帮助我们安装 Command Line Tools，安装完成后即可正常使用 brew 命令了。\n","permalink":"https://puffinjiang.github.io/posts/solution/upgradebrewerrorafterupdatingmacos/","summary":"Git can not used after I updated the MacOS","title":"Upgrade brew error after updating MacOS"},{"content":"Flask简介 Flask是一个“微” python web 框架，只实现了最基本的功能，简单、灵活、可拓展性强。\n简单：使用简单，通过 @app.route() 装饰器即可配置路由 灵活：配置灵活，支持多种不同类型的配置方式。可自定义 使用dict对象 使用环境变量 app.config.from_envvar() 使用配置文件 app.config.from_file() 包括json、toml 可拓展性强：可自定义其他部分的组件，如 flask-SQLAlchemy、flask-session 等 依赖 flask依赖的核心组件有2个： werkzeug 和 jinja。 jinja 是一个模版引擎，负责模版的渲染，werkzeug 是一个 WSGI 工具集的库\n启动流程 在Flask的官方文档中给了一个 \u0026lsquo;Hello World\u0026rsquo; 启动案例，我们本地启动一下\n1 2 3 4 5 6 7 8 9 10 from flask import Flask app = Flask(__name__) @app.route(\u0026#34;/\u0026#34;) def hello_world(): return \u0026#34;\u0026lt;p\u0026gt;Hello, World!\u0026lt;/p\u0026gt;\u0026#34; if __name__ == \u0026#39;__main__\u0026#39;: app.run(\u0026#34;127.0.0.1\u0026#34;, 5000) 我们可以看出来，启动的时候首先调用了 app.run() 方法，我们来看一下这个方法\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 class Flask(Scaffold): def run( self, host: t.Optional[str] = None, port: t.Optional[int] = None, debug: t.Optional[bool] = None, load_dotenv: bool = True, **options: t.Any, ) -\u0026gt; None: # Change this into a no-op if the server is invoked from the # command line. Have a look at cli.py for more information. if os.environ.get(\u0026#34;FLASK_RUN_FROM_CLI\u0026#34;) == \u0026#34;true\u0026#34;: from .debughelpers import explain_ignored_app_run explain_ignored_app_run() return if get_load_dotenv(load_dotenv): cli.load_dotenv() # if set, let env vars override previous values if \u0026#34;FLASK_ENV\u0026#34; in os.environ: self.env = get_env() self.debug = get_debug_flag() elif \u0026#34;FLASK_DEBUG\u0026#34; in os.environ: self.debug = get_debug_flag() # debug passed to method overrides all other sources if debug is not None: self.debug = bool(debug) server_name = self.config.get(\u0026#34;SERVER_NAME\u0026#34;) sn_host = sn_port = None if server_name: sn_host, _, sn_port = server_name.partition(\u0026#34;:\u0026#34;) if not host: if sn_host: host = sn_host else: host = \u0026#34;127.0.0.1\u0026#34; if port or port == 0: port = int(port) elif sn_port: port = int(sn_port) else: port = 5000 options.setdefault(\u0026#34;use_reloader\u0026#34;, self.debug) options.setdefault(\u0026#34;use_debugger\u0026#34;, self.debug) options.setdefault(\u0026#34;threaded\u0026#34;, True) cli.show_server_banner(self.env, self.debug, self.name, False) from werkzeug.serving import run_simple try: run_simple(t.cast(str, host), port, self, **options) finally: # reset the first request information if the development server # reset normally. This makes it possible to restart the server # without reloader and that stuff from an interactive shell. self._got_first_request = False 我们可以看到，前面一部分主要是处理一些启动参数。在参数处理完成之后，调用了 werkzeug.serving 中的 run_simple 函数，能看到这个函数传入了一个 self 参数，即我们的 flask application，我们再进入这个 run_simple 当中。\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 # werkzeug相关代码 def run_simple( hostname: str, port: int, application: \u0026#34;WSGIApplication\u0026#34;, use_reloader: bool = False, use_debugger: bool = False, use_evalex: bool = True, extra_files: t.Optional[t.Iterable[str]] = None, exclude_patterns: t.Optional[t.Iterable[str]] = None, reloader_interval: int = 1, reloader_type: str = \u0026#34;auto\u0026#34;, threaded: bool = False, processes: int = 1, request_handler: t.Optional[t.Type[WSGIRequestHandler]] = None, static_files: t.Optional[t.Dict[str, t.Union[str, t.Tuple[str, str]]]] = None, passthrough_errors: bool = False, ssl_context: t.Optional[_TSSLContextArg] = None, ) -\u0026gt; None: if not isinstance(port, int): raise TypeError(\u0026#34;port must be an integer\u0026#34;) if static_files: from .middleware.shared_data import SharedDataMiddleware # 静态文件的加载 application = SharedDataMiddleware(application, static_files) if use_debugger: from .debug import DebuggedApplication # Debug调试 application = DebuggedApplication(application, evalex=use_evalex) if not is_running_from_reloader(): # 启动前 socket 的准备和检查 ，使用IPV4还是IPV6、端口是否占用 s = prepare_socket(hostname, port) fd = s.fileno() os.environ[\u0026#34;WERKZEUG_SERVER_FD\u0026#34;] = str(fd) else: fd = int(os.environ[\u0026#34;WERKZEUG_SERVER_FD\u0026#34;]) srv = make_server( hostname, port, application, threaded, processes, request_handler, passthrough_errors, ssl_context, fd=fd, ) if not is_running_from_reloader(): srv.log_startup() if use_reloader: from ._reloader import run_with_reloader run_with_reloader( srv.serve_forever, extra_files=extra_files, exclude_patterns=exclude_patterns, interval=reloader_interval, reloader_type=reloader_type, ) else: srv.serve_forever() 从上面的代码我们可以看到有两处关键的函数调用，一处是 make_server ，另外一处是 serve_forever 。我们先看 make_server 做了什么：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 # werkzeug相关代码 def make_server( host: str, port: int, app: \u0026#34;WSGIApplication\u0026#34;, threaded: bool = False, processes: int = 1, request_handler: t.Optional[t.Type[WSGIRequestHandler]] = None, passthrough_errors: bool = False, ssl_context: t.Optional[_TSSLContextArg] = None, fd: t.Optional[int] = None, ) -\u0026gt; BaseWSGIServer: if threaded and processes \u0026gt; 1: raise ValueError(\u0026#34;Cannot have a multi-thread and multi-process server.\u0026#34;) if threaded: # ThreadedWSGIServer 继承了 BaseWSGIServer return ThreadedWSGIServer( host, port, app, request_handler, passthrough_errors, ssl_context, fd=fd ) if processes \u0026gt; 1: # ForkingWSGIServer 继承了 BaseWSGIServer return ForkingWSGIServer( host, port, app, processes, request_handler, passthrough_errors, ssl_context, fd=fd, ) return BaseWSGIServer( host, port, app, request_handler, passthrough_errors, ssl_context, fd=fd ) 能看到这里是返回了一个 BaseWSGIServer 实例化对象。BaseWSGIServer 在初始化的时候主要做了 两件事情，一是 绑定IP和端口以及监听；二是添加 WSGIRequestHandler 这个请求处理器。\n我们再来看看 serve_forever 做了什么：\n1 2 3 4 5 6 7 8 9 10 # werkzeug相关代码 class BaseWSGIServer(HTTPServer): def serve_forever(self, poll_interval: float = 0.5) -\u0026gt; None: try: super().serve_forever(poll_interval=poll_interval) except KeyboardInterrupt: pass finally: self.server_close() 再来进一步看看父类的 serve_forever 做了什么：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 # werkzeug相关代码 class BaseServer: def serve_forever(self, poll_interval=0.5): \u0026#34;\u0026#34;\u0026#34;Handle one request at a time until shutdown. Polls for shutdown every poll_interval seconds. Ignores self.timeout. If you need to do periodic tasks, do them in another thread. \u0026#34;\u0026#34;\u0026#34; self.__is_shut_down.clear() try: # XXX: Consider using another file descriptor or connecting to the # socket to wake this up instead of polling. Polling reduces our # responsiveness to a shutdown request and wastes cpu at all other # times. # I/O多路复用选择器，有poll时选poll with _ServerSelector() as selector: selector.register(self, selectors.EVENT_READ) while not self.__shutdown_request: # 轮询 ready = selector.select(poll_interval) # bpo-35017: shutdown() called during select(), exit immediately. if self.__shutdown_request: break if ready: self._handle_request_noblock() self.service_actions() finally: self.__shutdown_request = False self.__is_shut_down.set() 我们可以看到的是直接跳到了 BaseServer 这个类，这因为 BaseWSGIServer \u0026ndash;\u0026gt; HTTPServer \u0026ndash;\u0026gt; TCPServer \u0026ndash;\u0026gt; BaseServer 他们之间有继承关系。此处选用了 PollSelector 作为 I/O多路复用选择器，关于I/O多路复用就不详说了。当有活跃的文件描述符时，执行 _handle_request_noblock()方法，到了这里，我们的应用启动并开始监听相关的端口了。\n请求处理过程 在上面我们已经说到了应用的启动过程，接下来看一看当接收到请求之后是怎么处理的。 上面说到，当有活跃的文件描述符会执行 _handle_request_noblock() 方法，来看一下这个方法：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 # werkzeug相关代码 class BaseServer: def _handle_request_noblock(self): \u0026#34;\u0026#34;\u0026#34;Handle one request, without blocking. I assume that selector.select() has returned that the socket is readable before this function was called, so there should be no risk of blocking in get_request(). \u0026#34;\u0026#34;\u0026#34; try: request, client_address = self.get_request() except OSError: return if self.verify_request(request, client_address): try: self.process_request(request, client_address) except Exception: self.handle_error(request, client_address) self.shutdown_request(request) except: self.shutdown_request(request) raise else: self.shutdown_request(request) 在这段代码里我们可以看出来，这里是调用了 process_request 方法去处理接收到的request，这个时候就到了我们前面提到的 WSGIRequestHandler 这个请求处理器了。 WSGIRequestHandler 是 BaseRequestHandler 的子类，看看初始化相关的代码：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 # werkzeug相关代码 class BaseRequestHandler: def __init__(self, request, client_address, server): self.request = request self.client_address = client_address self.server = server self.setup() try: self.handle() finally: self.finish() def setup(self): pass def handle(self): pass def finish(self): pass 从上面的代码中可以看到，初始化时调用了 handle 方法，在 handle 方法里是调用了 handle_one_request 方法，我们来看看这个方法：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 # werkzeug相关代码 class BaseHTTPRequestHandler(socketserver.StreamRequestHandler): def handle_one_request(self): \u0026#34;\u0026#34;\u0026#34;Handle a single HTTP request. You normally don\u0026#39;t need to override this method; see the class __doc__ string for information on how to handle specific HTTP commands such as GET and POST. \u0026#34;\u0026#34;\u0026#34; try: self.raw_requestline = self.rfile.readline(65537) if len(self.raw_requestline) \u0026gt; 65536: self.requestline = \u0026#39;\u0026#39; self.request_version = \u0026#39;\u0026#39; self.command = \u0026#39;\u0026#39; self.send_error(HTTPStatus.REQUEST_URI_TOO_LONG) return if not self.raw_requestline: self.close_connection = True return if not self.parse_request(): # An error code has been sent, just exit return mname = \u0026#39;do_\u0026#39; + self.command if not hasattr(self, mname): self.send_error( HTTPStatus.NOT_IMPLEMENTED, \u0026#34;Unsupported method (%r)\u0026#34; % self.command) return method = getattr(self, mname) method() self.wfile.flush() #actually send the response if not already done. except socket.timeout as e: #a read or a write timed out. Discard this connection self.log_error(\u0026#34;Request timed out: %r\u0026#34;, e) self.close_connection = True return 可以看到关键的地方是这个 method 方法的调用，而这个 method 其实就是 do_GET、do_POST 等方法。通过 debug 看到，这个方法和 WSGIRequestHandler.run_wsgi 绑定了，即调用了 WSGIRequestHandler.run_wsgi 方法，这是很关键的一个方法：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 # werkzeug相关代码 class WSGIRequestHandler(BaseHTTPRequestHandler): def run_wsgi(self) -\u0026gt; None: if self.headers.get(\u0026#34;Expect\u0026#34;, \u0026#34;\u0026#34;).lower().strip() == \u0026#34;100-continue\u0026#34;: self.wfile.write(b\u0026#34;HTTP/1.1 100 Continue\\r\\n\\r\\n\u0026#34;) self.environ = environ = self.make_environ() status_set: t.Optional[str] = None headers_set: t.Optional[t.List[t.Tuple[str, str]]] = None status_sent: t.Optional[str] = None headers_sent: t.Optional[t.List[t.Tuple[str, str]]] = None chunk_response: bool = False def write(data: bytes) -\u0026gt; None: nonlocal status_sent, headers_sent, chunk_response assert status_set is not None, \u0026#34;write() before start_response\u0026#34; assert headers_set is not None, \u0026#34;write() before start_response\u0026#34; if status_sent is None: status_sent = status_set headers_sent = headers_set try: code_str, msg = status_sent.split(None, 1) except ValueError: code_str, msg = status_sent, \u0026#34;\u0026#34; code = int(code_str) self.send_response(code, msg) header_keys = set() for key, value in headers_sent: self.send_header(key, value) header_keys.add(key.lower()) # Use chunked transfer encoding if there is no content # length. Do not use for 1xx and 204 responses. 304 # responses and HEAD requests are also excluded, which # is the more conservative behavior and matches other # parts of the code. # https://httpwg.org/specs/rfc7230.html#rfc.section.3.3.1 if ( not ( \u0026#34;content-length\u0026#34; in header_keys or environ[\u0026#34;REQUEST_METHOD\u0026#34;] == \u0026#34;HEAD\u0026#34; or (100 \u0026lt;= code \u0026lt; 200) or code in {204, 304} ) and self.protocol_version \u0026gt;= \u0026#34;HTTP/1.1\u0026#34; ): chunk_response = True self.send_header(\u0026#34;Transfer-Encoding\u0026#34;, \u0026#34;chunked\u0026#34;) # Always close the connection. This disables HTTP/1.1 # keep-alive connections. They aren\u0026#39;t handled well by # Python\u0026#39;s http.server because it doesn\u0026#39;t know how to # drain the stream before the next request line. self.send_header(\u0026#34;Connection\u0026#34;, \u0026#34;close\u0026#34;) self.end_headers() assert isinstance(data, bytes), \u0026#34;applications must write bytes\u0026#34; if data: if chunk_response: self.wfile.write(hex(len(data))[2:].encode()) self.wfile.write(b\u0026#34;\\r\\n\u0026#34;) self.wfile.write(data) if chunk_response: self.wfile.write(b\u0026#34;\\r\\n\u0026#34;) self.wfile.flush() def start_response(status, headers, exc_info=None): # type: ignore nonlocal status_set, headers_set if exc_info: try: if headers_sent: raise exc_info[1].with_traceback(exc_info[2]) finally: exc_info = None elif headers_set: raise AssertionError(\u0026#34;Headers already set\u0026#34;) status_set = status headers_set = headers return write def execute(app: \u0026#34;WSGIApplication\u0026#34;) -\u0026gt; None: application_iter = app(environ, start_response) try: for data in application_iter: write(data) if not headers_sent: write(b\u0026#34;\u0026#34;) if chunk_response: self.wfile.write(b\u0026#34;0\\r\\n\\r\\n\u0026#34;) finally: if hasattr(application_iter, \u0026#34;close\u0026#34;): application_iter.close() # type: ignore try: execute(self.server.app) except (ConnectionError, socket.timeout) as e: self.connection_dropped(e, environ) except Exception as e: if self.server.passthrough_errors: raise if status_sent is not None and chunk_response: self.close_connection = True try: # if we haven\u0026#39;t yet sent the headers but they are set # we roll back to be able to set them again. if status_sent is None: status_set = None headers_set = None execute(InternalServerError()) except Exception: pass from .debug.tbtools import DebugTraceback msg = DebugTraceback(e).render_traceback_text() self.server.log(\u0026#34;error\u0026#34;, f\u0026#34;Error on request:\\n{msg}\u0026#34;) 这段代码里是 execute(self.server.app) 这一段，而这个 server.app 其实就是我们传进来的 flask application了，通过 application_iter = app(environ, start_response) 这一行调用了 Flask 的 __call__ 方法。上面涉及到了很多 werkzeug 相关的代码，然后在这里又回到了 flask。 来看一看这个方法：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 # Flask相关代码 class Flask(Scaffold): def __call__(self, environ: dict, start_response: t.Callable) -\u0026gt; t.Any: \u0026#34;\u0026#34;\u0026#34;The WSGI server calls the Flask application object as the WSGI application. This calls :meth:`wsgi_app`, which can be wrapped to apply middleware. \u0026#34;\u0026#34;\u0026#34; return self.wsgi_app(environ, start_response) def wsgi_app(self, environ: dict, start_response: t.Callable) -\u0026gt; t.Any: # 创建请求的上下文 ctx = self.request_context(environ) error: t.Optional[BaseException] = None try: try: ctx.push() response = self.full_dispatch_request() except Exception as e: error = e response = self.handle_exception(e) except: # noqa: B001 error = sys.exc_info()[1] raise return response(environ, start_response) finally: if self.should_ignore_error(error): error = None ctx.auto_pop(error) 这段代码首先是创建请求的上下文，然后通过 self.full_dispatch_request() 方法分发请求，我们再来看看 full_dispatch_request 的具体代码：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 class Flask(Scaffold): def full_dispatch_request(self) -\u0026gt; Response: # Run before_first_request functions if this is the thread\u0026#39;s first request. # Inlined to avoid a method call on subsequent requests. # This is deprecated, will be removed in Flask 2.3. if not self._got_first_request: with self._before_request_lock: if not self._got_first_request: for func in self.before_first_request_funcs: self.ensure_sync(func)() self._got_first_request = True try: request_started.send(self) rv = self.preprocess_request() if rv is None: rv = self.dispatch_request() except Exception as e: rv = self.handle_user_exception(e) return self.finalize_request(rv) def dispatch_request(self) -\u0026gt; ft.ResponseReturnValue: req = request_ctx.request if req.routing_exception is not None: self.raise_routing_exception(req) rule: Rule = req.url_rule # type: ignore[assignment] # if we provide automatic options for this URL and the # request came with the OPTIONS method, reply automatically if ( getattr(rule, \u0026#34;provide_automatic_options\u0026#34;, False) and req.method == \u0026#34;OPTIONS\u0026#34; ): return self.make_default_options_response() # otherwise dispatch to the handler for that endpoint view_args: t.Dict[str, t.Any] = req.view_args # type: ignore[assignment] return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) 能看的到这里 dispatch_request 分发请求，使用 handle_user_exception 处理分发过程中的异常处理。在 dispatch_request 当中，根据url路径，找到相应的方法，并调用，代码为 self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)。我们再来看一看 self.finalize_request(rv) 做了什么：\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # Flask相关代码 class Flask(Scaffold): def finalize_request( self, rv: t.Union[ResponseReturnValue, HTTPException], from_error_handler: bool = False, ) -\u0026gt; Response: response = self.make_response(rv) try: response = self.process_response(response) request_finished.send(self, response=response) except Exception: if not from_error_handler: raise self.logger.exception( \u0026#34;Request finalizing failed with an error while handling an error\u0026#34; ) return response 这里的 finalize_request() 就是把 view_functions 返回的结果组装成 Response 并返回，在其中处理了 status、headers、session 等，到此我们的请求处理就结束了。\n","permalink":"https://puffinjiang.github.io/posts/learning/flask%E6%BA%90%E7%A0%81%E9%98%85%E8%AF%BB/","summary":"记录一下阅读Flask源码的过程","title":"Flask源码阅读"},{"content":"最近在使用celery这个异步框架，在使用的过程中，发现worker运行一段时间后，内存持续上涨，然后告警 在网上查询之后发现了Celery有内存泄漏的问题，requests包也会有内存泄漏的问题 在网上查到的解决方法是\n在worker执行指定次数任务后重建新进程\n1 2 # 执行32次后被干掉 worker_max_tasks_per_child = 32 在worker内存到达指定限制后重建新进程\n1 2 # 内存使用量达到 12m 后被干掉 worker_max_memory_per_child = 12000 # 12m ","permalink":"https://puffinjiang.github.io/posts/summary/celery%E5%86%85%E5%AD%98%E6%B3%84%E6%BC%8F%E8%A7%A3%E5%86%B3/","summary":"记一次Celery内存泄漏的问题解决","title":"Celery内存泄漏解决"},{"content":"","permalink":"https://puffinjiang.github.io/links/","summary":"","title":"🤝友链"},{"content":"配置 允许远程连接 ⚠️ 这两个文件在不同操作系统下的路径不同。\n我们需要修改两个文件，分别是 postgresql.conf 和 pg_hba.conf\n在 postgresql.conf 添加如下内容\n1 listen_addresses = \u0026#39;*\u0026#39; 在 pg_hba.conf 中添加如下内容\n1 2 3 4 5 # ipv4 host all all 0.0.0.0/0 md5 # ipv6 host all all ::0/0 md5 FreeBSD 我这里是 postgresql15 的版本\n1 vi /var/db/postgres/data15/postgresql.conf 1 vi /var/db/postgres/data15/pg_hba.conf Ubuntu 1 vi /etc/postgresql/15/main/postgresql.conf 1 vi /etc/postgresql/15/main/pg_hba.conf 连接 使用 psql 远程连接 使用uri方式 1 psql postgres://username:password@host:port/dbname 使用普通的参数方式 1 psql -U username -h hostname -p port -d dbname 查询 列出所有的数据库名 1 SELECT datname FROM pg_database; 数据库创建操作 创建用户名和密码 1 create user username with encrypted password \u0026#39;password\u0026#39;; 创建新的数据库 1 create database dbname ; 数据库授权操作 1 2 -- 将数据库 dbname 授权给用户 username grant all privileges on database dbname to username; 备份和恢复 数据库备份和恢复 具体可参考教程\n表备份 1 CREATE TABLE new_table AS SELECT * FROM public.old_table; 表数据删除 TRUNCATE 命令（全表数据删除）\n1 TRUNCATE TABLE public.new_table; DELETE 命令\n1 DELETE FROM TABLE WHERE id = 1; ","permalink":"https://puffinjiang.github.io/posts/learning/postgresql%E6%95%B0%E6%8D%AE%E5%BA%93%E5%B8%B8%E7%94%A8%E6%93%8D%E4%BD%9C/","summary":"记录一些常用的PostgreSQL数据库操作命令","title":"PostgreSQL数据库常用操作"},{"content":"前置条件 windows系统、github设置隐私邮箱\n使用Github给的邮箱替换了原有的邮箱，命令如下：\n1 2 git config --global user.name \u0026#34;xxx\u0026#34; git config --global user.email \u0026#34;xxx\u0026#34; 用旧的邮箱进行了commit\n过程 在替换为新的邮箱之后使用 git push 命令提交代码被拒绝，返回以下提示：\n1 2 3 4 Can\u0026#39;t finish GitHub sharing process Successfully created project \u0026#39;xxx\u0026#39; on GitHub, but initial push failed: remote: error: GH007: Your push would publish a private email address. failed to push some refs to \u0026#39;xxx\u0026#39; 使用 git config --list 查看邮箱是否正确\n1 2 user.name=xxx user.email=xxx 发现没有问题，和修改后的邮箱一直\n经过回溯之后，发现在修改邮箱之前，使用原来的邮箱进行了 commit，但是没有 push。因此commit的内容用的还是原来的邮箱，导致push时报错\n解决方法 使用 git reset 命令回退版本到 commit 之前 重新 commit 和 push，push 成功 ","permalink":"https://puffinjiang.github.io/posts/solution/git%E6%8E%A8%E9%80%81%E4%BB%A3%E7%A0%81%E5%A4%B1%E8%B4%A5/","summary":"在使用 git push时报 remote: error: GH007: Your push would publish a private email address.","title":"Git推送代码失败"},{"content":"👨‍💻 职业 程序员\n开发语言 Python Java Rust 数据库 Mysql PostgreSQL Redis MongoDB DevOps Jenkins Docker Docker-compose ⛹ 兴趣爱好 运动：足球、羽毛球、篮球等 阅读：技术书籍、小说等 动漫：《凡人修仙传》、《某科学的超电磁炮》等 游戏：CSGO 📬 联系我呀","permalink":"https://puffinjiang.github.io/about/","summary":"","title":"🙋🏻‍♂️关于我"}]