# lagou-mongodb-sharding **Repository Path**: hackerodm/lagou-mongodb-sharding ## Basic Information - **Project Name**: lagou-mongodb-sharding - **Description**: 业务场景：用户在拉勾网投递简历时，我们会为每次投递的简历生成一份快照，将快照信息存储到 MongoDB中。 功能需求：搭建MongoDB分片集群，模拟简历快照数据进行操作，具体要求如下： (1) 如图搭建一个分片集群，要求每个分片节点中的复制集含有一个仲裁节点 (2) 使用权限控制，建立要访问的数据库lg_resume，这个账号名字是lagou_gx、密码是abc321 这个账号对数据库有读写权限 (3) 使用SpringBoot 进行访问分片集群，对lg_resume 库中的lg_resume_datas 进行增加和查询操作 - **Primary Language**: Java - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 2 - **Created**: 2021-04-08 - **Last Updated**: 2023-01-16 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # lagou-mongodb-sharding ### 一、介绍 #### 1.1、视频讲解地址 链接：https://pan.baidu.com/s/1gnTlxOCcM6VLVqry6lY4MQ 提取码：fmm7 #### 1.2、作业内容 业务场景：用户在拉勾网投递简历时，我们会为每次投递的简历生成一份快照，将快照信息存储到 MongoDB中。 功能需求：搭建MongoDB分片集群，模拟简历快照数据进行操作，具体要求如下： (1) 如图搭建一个分片集群，要求每个分片节点中的复制集含有一个仲裁节点 (2) 使用权限控制，建立要访问的数据库lg_resume，这个账号名字是lagou_gx、密码是abc321 这个账号对数据库有读写权限 (3) 使用SpringBoot 进行访问分片集群，对lg_resume 库中的lg_resume_datas 进行增加和查询操作 ### 二、环境及软件版本 | 环境 | 版本 | | ------------------------- | ----- | | 操作系统：Centos 7 x86_64 | 7 | | 虚拟机：VMware | 15.5 | | 远程连接：XShell | 6.0 | | 数据库：mongo | 4.1.3 | | 文件传输：XFtp | 6.0 | | JDK | 1.8 | ### 三、整体架构 ![image-20200925111247142](images/image-20200925111247142.png) 本集群采用伪集群方式搭建，所有节点均部署在一台虚拟机。虚拟机IP：192.168.0.234 #### 3.1、路径规划 | 端口 | 目录名称 | 用途 | | ----- | ------------------------------------------ | ------------------------- | | | /usr/local/mongodb/config | Config Server配置信息目录 | | | /usr/local/mongodb/config/logs | Config Server日志目录 | | 17011 | /usr/local/mongodb/config/cnf17011 | Config Server1数据目录 | | 17013 | /usr/local/mongodb/config/cnf17013 | Config Server2数据目录 | | 17015 | /usr/local/mongodb/config/cnf17015 | Config Server3数据目录 | | | /usr/local/mongodb/shard | Shard Server分片配置目录 | | | /usr/local/mongodb/shard/logs | shard Server日志文件 | | | /usr/local/mongodb/shard/shard1 | Shard1数据目录 | | 37011 | /usr/local/mongodb/shard/shard1/shard37011 | Shard1集群节点37011 | | 37013 | /usr/local/mongodb/shard/shard1/shard37013 | Shard1集群节点37013 | | 37015 | /usr/local/mongodb/shard/shard1/shard37015 | Shard1集群节点37015 | | 37017 | /usr/local/mongodb/shard/shard1/shard37017 | Shard1仲裁节点37017 | | | /usr/local/mongodb/shard/shard2 | Shard2数据目录 | | 47011 | /usr/local/mongodb/shard/shard2/shard47011 | Shard2集群节点47011 | | 47013 | /usr/local/mongodb/shard/shard2/shard47013 | Shard2集群节点47013 | | 47015 | /usr/local/mongodb/shard/shard2/shard47015 | Shard2集群节点47015 | | 47017 | /usr/local/mongodb/shard/shard2/shard47017 | shard2仲裁节点47017 | | | /usr/local/mongodb/shard/shard3 | Shard3数据目录 | | 57011 | /usr/local/mongodb/shard/shard3/shard57011 | Shard3集群节点57011 | | 57013 | /usr/local/mongodb/shard/shard3/shard57013 | Shard3集群节点57013 | | 57015 | /usr/local/mongodb/shard/shard3/shard57015 | Shard3集群节点57015 | | 57017 | /usr/local/mongodb/shard/shard3/shard57017 | Shard3集群节点57017 | | | /usr/local/mongodb/shard/shard4 | Shard4数据目录 | | 58011 | /usr/local/mongodb/shard/shard4/shard58011 | Shard4集群节点58011 | | 58013 | /usr/local/mongodb/shard/shard4/shard58013 | Shard4集群节点58011 | | 58015 | /usr/local/mongodb/shard/shard4/shard58015 | Shard4集群节点58011 | | 58017 | /usr/local/mongodb/shard/shard4/shard58017 | Shard4集群节点58011 | | 27017 | /usr/local/mongodb/route | 路由节点 | | | /usr/local/mongodb/route/logs | 路由节点日志 | #### 3.2、创建目录 ```shell [root@192 mongodb]# mkdir config shard route [root@192 mongodb]# mkdir route/logs config/logs shard/logs shard/shard1 shard/shard2 shard/shard3 shard/shard4 [root@192 mongodb]# mkdir shard/shard1/shard37011 shard/shard1/shard37013 shard/shard1/shard37015 shard/shard1/shard37017 [root@192 mongodb]# mkdir shard/shard2/shard47011 shard/shard2/shard47013 shard/shard2/shard47015 shard/shard2/shard47017 [root@192 mongodb]# mkdir shard/shard3/shard57011 shard/shard3/shard57013 shard/shard3/shard57015 shard/shard3/shard57017 [root@192 mongodb]# mkdir shard/shard4/shard58011 shard/shard4/shard58013 shard/shard4/shard58015 shard/shard4/shard58017 [root@192 mongodb]# mkdir config/cnf17011 config/cnf17013 config/cnf17015 ``` #### 3.3、安装mongodb 下载mongodb-linux-x86_64-4.1.3.tgz，上传到服务器的mongodb目录并解压，解压之后将文件拷贝到mongodb目录： ```shell [root@192 mongodb]# tar -xvf mongodb-linux-x86_64-4.1.3.tgz [root@192 mongodb]# mv mongodb-linux-x86_64-4.1.3/* ./ ``` #### 3.4、安装psmisc ```shell [root@192 mongodb]# yum install psmisc -y ``` ### 四、MongoDB集群搭建 #### 4.1、配置Config节点 ##### 4.1.1、节点1配置config17011.conf ```properties # 数据库文件位置 dbpath=config/cnf17011 # 日志文件位置 logpath=config/logs/config17011.log # 追加方式写入日志 logappend=true # fork守护进程 fork=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=17011 # 标志为配置服务器 configsvr=true # 配置服务器副本集名称 replSet=configsvr ``` ##### 4.1.2、节点2配置config17013.conf ```properties # 数据库文件位置 dbpath=config/cnf17013 # 日志文件位置 logpath=config/logs/config17013.log # 追加方式写入日志 logappend=true # fork守护进程 fork=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=17013 # 标志为配置服务器 configsvr=true # 配置服务器副本集名称 replSet=configsvr ``` ##### 4.1.3、节点3配置config17015.conf ```properties # 数据库文件位置 dbpath=config/cnf17015 # 日志文件位置 logpath=config/logs/config17015.log # 追加方式写入日志 logappend=true # fork守护进程 fork=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=17015 # 标志为配置服务器 configsvr=true # 配置服务器副本集名称 replSet=configsvr ``` ##### 4.1.4、启动配置节点 ```shell [root@192 mongodb]# ./bin/mongod -f config/config17011.conf [root@192 mongodb]# ./bin/mongod -f config/config17013.conf [root@192 mongodb]# ./bin/mongod -f config/config17015.conf ``` ##### 4.1.5、配置节点集群 进入任意节点的mongo shell 添加配置节点集群，在添加配置节点集群的时候，注意use admin数据库。 配置集群成员并初始化，然后使用rs.status()查看集群状态。 ```shell ./bin/mongo --port 17011 use admin var cfg = {"_id":"configsvr", "members":[ {"_id":1,"host":"192.168.0.234:17011"}, {"_id":2,"host":"192.168.0.234:17013"}, {"_id":3,"host":"192.168.0.234:17015"}, ] }; rs.initiate(cfg) rs.status(); ``` #### 4.2、分片Sharding节点 分片节点包含四组集群节点配置，并且每组集群 ##### 4.2.1、shard1节点配置 shard1集群节点包含37011、37013、37015、37017，将37017作为仲裁节点。 ###### 4.2.1.1、37011节点配置 ```properties # 数据库目录 dbpath=shard/shard1/shard37011 # 日志文件存放目录 logpath=shard/logs/shard1-37011.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=37011 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard1 # 指定为shardsvr shardsvr=true ``` ###### 4.2.1.2、37013节点配置 ```properties # 数据库目录 dbpath=shard/shard1/shard37013 # 日志文件存放目录 logpath=shard/logs/shard1-37013.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=37013 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard1 # 指定为shardsvr shardsvr=true ``` ###### 4.2.1.3、37015节点配置 ```properties # 数据库目录 dbpath=shard/shard1/shard37015 # 日志文件存放目录 logpath=shard/logs/shard1-37015.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=37015 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard1 # 指定为shardsvr shardsvr=true ``` ###### 4.2.1.4、37017节点配置 ```properties # 数据库目录 dbpath=shard/shard1/shard37017 # 日志文件存放目录 logpath=shard/logs/shard1-37017.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=37017 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard1 # 指定为shardsvr shardsvr=true ``` ###### 4.2.1.5、启动shard1集群 ```shell [root@192 mongodb]# ./bin/mongod -f shard/shard1/shard37011.conf [root@192 mongodb]# ./bin/mongod -f shard/shard1/shard37013.conf [root@192 mongodb]# ./bin/mongod -f shard/shard1/shard37015.conf [root@192 mongodb]# ./bin/mongod -f shard/shard1/shard37017.conf ``` ###### 4.2.1.6、配置shard1集群 ```shell [root@192 mongodb]# ./bin/mongo -port 37011 use admin var cfg = {"_id":"shard1", "protocolVersion":1, "members":[ {"_id":1,"host":"192.168.0.234:37011","priority":10}, {"_id":2,"host":"192.168.0.234:37013","priority":5}, {"_id":3,"host":"192.168.0.234:37015","priority":0}, {"_id":4,"host":"192.168.0.234:37017","arbiterOnly":true} ] }; rs.initiate(cfg) rs.status() ``` ##### 4.2.2、shard2节点配置 ###### 4.2.2.1、47011节点配置 ```properties # 数据库目录 dbpath=shard/shard2/shard47011 # 日志文件存放目录 logpath=shard/logs/shard2-47011.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=47011 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard2 # 指定为shardsvr shardsvr=true ``` ###### 4.2.2.2、47013节点配置 ```properties # 数据库目录 dbpath=shard/shard2/shard47013 # 日志文件存放目录 logpath=shard/logs/shard2-47013.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=47013 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard2 # 指定为shardsvr shardsvr=true ``` ###### 4.2.2.3、47015节点配置 ```properties # 数据库目录 dbpath=shard/shard2/shard47015 # 日志文件存放目录 logpath=shard/logs/shard2-47015.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=47015 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard2 # 指定为shardsvr shardsvr=true ``` ###### 4.2.2.4、47017节点配置 ```properties # 数据库目录 dbpath=shard/shard2/shard47017 # 日志文件存放目录 logpath=shard/logs/shard2-47017.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=47017 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard2 # 指定为shardsvr shardsvr=true ``` ###### 4.2.2.5、启动shard2集群 ```shell [root@192 mongodb]# ./bin/mongod -f shard/shard2/shard47011.conf [root@192 mongodb]# ./bin/mongod -f shard/shard2/shard47013.conf [root@192 mongodb]# ./bin/mongod -f shard/shard2/shard47015.conf [root@192 mongodb]# ./bin/mongod -f shard/shard2/shard47017.conf ``` ###### 4.2.2.6、配置shard2集群 进入任意节点，进行初始化操作。 ```shell ./bin/mongo -port 47011 use admin var cfg = {"_id":"shard2", "protocolVersion":1, "members":[ {"_id":1,"host":"192.168.0.234:47011","priority":10}, {"_id":2,"host":"192.168.0.234:47013","priority":5}, {"_id":3,"host":"192.168.0.234:47015","priority":0}, {"_id":4,"host":"192.168.0.234:47017","arbiterOnly":true} ] }; rs.initiate(cfg); rs.status(); ``` ##### 4.2.3、shard3节点配置 ###### 4.2.3.1、57011节点配置 ```properties # 数据库目录 dbpath=shard/shard3/shard57011 # 日志文件存放目录 logpath=shard/logs/shard3-57011.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=57011 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard3 # 指定为shardsvr shardsvr=true ``` ###### 4.2.3.2、57013节点配置 ```properties # 数据库目录 dbpath=shard/shard3/shard57013 # 日志文件存放目录 logpath=shard/logs/shard3-57013.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=57013 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard3 # 指定为shardsvr shardsvr=true ``` ###### 4.2.3.3、57015节点配置 ```properties # 数据库目录 dbpath=shard/shard3/shard57015 # 日志文件存放目录 logpath=shard/logs/shard3-57015.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=57015 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard3 # 指定为shardsvr shardsvr=true ``` ###### 4.2.3.4、57017节点配置 ```properties # 数据库目录 dbpath=shard/shard3/shard57017 # 日志文件存放目录 logpath=shard/logs/shard3-57017.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=57017 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard3 # 指定为shardsvr shardsvr=true ``` ###### 4.2.3.5、启动shard3集群 ```shell [root@192 mongodb]# ./bin/mongod -f shard/shard3/shard57011.conf [root@192 mongodb]# ./bin/mongod -f shard/shard3/shard57013.conf [root@192 mongodb]# ./bin/mongod -f shard/shard3/shard57015.conf [root@192 mongodb]# ./bin/mongod -f shard/shard3/shard57017.conf ``` ###### 4.2.3.6、配置shard3集群 ```shell [root@192 mongodb]# ./bin/mongo -port 57011 use admin var cfg = {"_id":"shard3", "protocolVersion":1, "members":[ {"_id":1,"host":"192.168.0.234:57011","priority":10}, {"_id":2,"host":"192.168.0.234:57013","priority":5}, {"_id":3,"host":"192.168.0.234:57015","priority":0}, {"_id":4,"host":"192.168.0.234:57017","arbiterOnly":true} ] }; rs.initiate(cfg); rs.status(); ``` ##### 4.2.4、shard4节点配置 ###### 4.2.4.1、58011节点配置 ```properties # 数据库目录 dbpath=shard/shard4/shard58011 # 日志文件存放目录 logpath=shard/logs/shard4-58011.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=58011 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard4 # 指定为shardsvr shardsvr=true ``` ###### 4.2.4.2、58013节点配置 ```properties # 数据库目录 dbpath=shard/shard4/shard58013 # 日志文件存放目录 logpath=shard/logs/shard4-58013.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=58013 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard4 # 指定为shardsvr shardsvr=true ``` ###### 4.2.4.3、58015节点配置 ```properties # 数据库目录 dbpath=shard/shard4/shard58015 # 日志文件存放目录 logpath=shard/logs/shard4-58015.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=58015 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard4 # 指定为shardsvr shardsvr=true ``` ###### 4.2.4.4、58017节点配置 ```properties # 数据库目录 dbpath=shard/shard4/shard58017 # 日志文件存放目录 logpath=shard/logs/shard4-58017.log # 追加方式写入日志 logappend=true # 绑定IP bind_ip=0.0.0.0 # 端口 port=58017 # 后台守护进程启动 fork=true # 复制集名称 replSet=shard4 # 指定为shardsvr shardsvr=true ``` ###### 4.2.4.5、启动shard4集群 ```shell [root@192 mongodb]# ./bin/mongod -f shard/shard4/shard58011.conf [root@192 mongodb]# ./bin/mongod -f shard/shard4/shard58013.conf [root@192 mongodb]# ./bin/mongod -f shard/shard4/shard58015.conf [root@192 mongodb]# ./bin/mongod -f shard/shard4/shard58017.conf ``` ###### 4.2.4.6、配置shard4集群 ```shell [root@192 mongodb]# ./bin/mongo -port 58011 use admin var cfg = {"_id":"shard4", "protocolVersion":1, "members":[ {"_id":1,"host":"192.168.0.234:58011","priority":10}, {"_id":2,"host":"192.168.0.234:58013","priority":5}, {"_id":3,"host":"192.168.0.234:58015","priority":0}, {"_id":4,"host":"192.168.0.234:58017","arbiterOnly":true} ] }; rs.initiate(cfg); rs.status(); ``` #### 4.3、路由节点 ##### 4.3.1、路由节点27017配置 route27017.conf ```properties # 端口 port=27017 # 绑定IP bind_ip=0.0.0.0 # fork fork=true # 日志文件 logpath=route/logs/route.log # 追加方式记录日志 logappend=true # 配置集群 configdb=configsvr/192.168.0.234:17011,192.168.0.234:17013,192.168.0.234:17015 ``` ##### 4.3.2、路由节点启动 采用mongs启动路由节点。 ```shell [root@192 mongodb]# ./bin/mongos -f route/route-27017.conf ``` ##### 4.3.3、mongos添加分片节点 ```shell [root@192 mongodb]# ./bin/mongos -f route/route-27017.conf sh.status() sh.addShard("shard1/192.168.0.234:37011,192.168.0.234:37013,192.168.0.234:37015,192.168.0.234:37017"); sh.addShard("shard2/192.168.0.234:47011,192.168.0.234:47013,192.168.0.234:47015,192.168.0.234:47017"); sh.addShard("shard3/192.168.0.234:57011,192.168.0.234:57013,192.168.0.234:57015,192.168.0.234:57017"); sh.addShard("shard4/192.168.0.234:58011,192.168.0.234:58013,192.168.0.234:58015,192.168.0.234:58017"); sh.status() ``` ##### 4.3.4、开启数据库和集合分片（指定片键） 继续使用mongos完成分片开启和分片大小设置。 本例存储简历快照数据库，分片键采用hash算法，对resumeid字段进行分片。 ```shell #开启数据库分片功能 mongos> sh.enableSharding("lagou_resume"); #指定集合开启分片功能 sh.shardCollection("lagou_resume.lagou_resume_datas",{"resumeid":"hashed"}) ``` 最终的集群状态信息： ```json mongos> sh.status(); --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5f6eed16d073eb1450241ff9") } shards: { "_id" : "shard1", "host" : "shard1/192.168.0.234:37011,192.168.0.234:37013,192.168.0.234:37015", "state" : 1 } { "_id" : "shard2", "host" : "shard2/192.168.0.234:47011,192.168.0.234:47013,192.168.0.234:47015", "state" : 1 } { "_id" : "shard3", "host" : "shard3/192.168.0.234:57011,192.168.0.234:57013,192.168.0.234:57015", "state" : 1 } { "_id" : "shard4", "host" : "shard4/192.168.0.234:58011,192.168.0.234:58013,192.168.0.234:58015", "state" : 1 } active mongoses: "4.1.3" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shard1 1 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) { "_id" : "lagou_resume", "primary" : "shard4", "partitioned" : true, "version" : { "uuid" : UUID("e2a6bd19-a6be-46ea-97c9-95ac9a3b99d1"), "lastMod" : 1 } } lagou_resume.lagou_resume_datas shard key: { "resumeid" : "hashed" } unique: false balancing: true chunks: shard1 2 shard2 2 shard3 2 shard4 2 { "resumeid" : { "$minKey" : 1 } } -->> { "resumeid" : NumberLong("-6917529027641081850") } on : shard1 Timestamp(1, 0) { "resumeid" : NumberLong("-6917529027641081850") } -->> { "resumeid" : NumberLong("-4611686018427387900") } on : shard1 Timestamp(1, 1) { "resumeid" : NumberLong("-4611686018427387900") } -->> { "resumeid" : NumberLong("-2305843009213693950") } on : shard2 Timestamp(1, 2) { "resumeid" : NumberLong("-2305843009213693950") } -->> { "resumeid" : NumberLong(0) } on : shard2 Timestamp(1, 3) { "resumeid" : NumberLong(0) } -->> { "resumeid" : NumberLong("2305843009213693950") } on : shard3 Timestamp(1, 4) { "resumeid" : NumberLong("2305843009213693950") } -->> { "resumeid" : NumberLong("4611686018427387900") } on : shard3 Timestamp(1, 5) { "resumeid" : NumberLong("4611686018427387900") } -->> { "resumeid" : NumberLong("6917529027641081850") } on : shard4 Timestamp(1, 6) { "resumeid" : NumberLong("6917529027641081850") } -->> { "resumeid" : { "$maxKey" : 1 } } on : shard4 Timestamp(1, 7) ``` #### 4.4、写入测试数据 ##### 4.4.1、写入测试数据 ```js db.lagou_resume_datas.insert({resumeid:1,name:'zhangsan',city:'cs',birth_day:new ISODate('2001-08-01'),expectSalary:18000}); db.lagou_resume_datas.insert({resumeid:2,name:'lisi',city:'gz',birth_day:new ISODate('2001-08-01'),expectSalary:18000}); db.lagou_resume_datas.insert({resumeid:3,name:'wangwu',city:'sh',birth_day:new ISODate('2001-08-01'),expectSalary:15000}); db.lagou_resume_datas.insert({resumeid:4,name:'zhaoliu',city:'bj',birth_day:new ISODate('2001-08-01'),expectSalary:12000}); db.lagou_resume_datas.insert({resumeid:5,name:'刘超杰',city:'cs',birth_day:new ISODate('1985-08-03'),expectSalary:8000}); db.lagou_resume_datas.insert({resumeid:6,name:'谢霆锋',city:'hk',birth_day:new ISODate('1980-08-01'),expectSalary:98000}); db.lagou_resume_datas.insert({resumeid:7,name:'汪涵',city:'cs',birth_day:new ISODate('1976-08-01'),expectSalary:58000}); db.lagou_resume_datas.insert({resumeid:8,name:'zhaoliying',city:'cs',birth_day:new ISODate('1987-08-01'),expectSalary:38000}); db.lagou_resume_datas.insert({resumeid:9,name:'nono',city:'cs',birth_day:new ISODate('2001-08-01'),expectSalary:10000}); ``` ##### 4.4.2、分片效果验证 shard1存储的数据: ```shell [root@192 mongodb]# ./bin/mongo -port 37011 shard1:PRIMARY> use lagou_resume switched to db lagou_resume shard1:PRIMARY> show collections lagou_resume_datas shard1:PRIMARY> db.lagou_resume_datas.find(); { "_id" : ObjectId("5f701419ecdfe62f7c8cb679"), "resumeid" : 6, "name" : "谢霆锋", "city" : "hk", "birth_day" : ISODate("1980-08-01T00:00:00Z"), "expectSalary" : 98000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb67b"), "resumeid" : 8, "name" : "zhaoliying", "city" : "cs", "birth_day" : ISODate("1987-08-01T00:00:00Z"), "expectSalary" : 38000 } ``` shard2存储的数据： ```shell [root@192 mongodb]# ./bin/mongo -port 47011 shard2:PRIMARY> use lagou_resume switched to db lagou_resume shard2:PRIMARY> show tables; lagou_resume_datas shard2:PRIMARY> db.lagou_resume_datas.find(); { "_id" : ObjectId("5f701419ecdfe62f7c8cb676"), "resumeid" : 3, "name" : "wangwu", "city" : "sh", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 15000 } ``` shard3存储的数据： ```shell [root@192 mongodb]# ./bin/mongo -port 57011 shard3:PRIMARY> use lagou_resume switched to db lagou_resume shard3:PRIMARY> show tables; lagou_resume_datas shard3:PRIMARY> db.lagou_resume_datas.find(); { "_id" : ObjectId("5f701419ecdfe62f7c8cb675"), "resumeid" : 2, "name" : "lisi", "city" : "gz", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 18000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb677"), "resumeid" : 4, "name" : "zhaoliu", "city" : "bj", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 12000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb678"), "resumeid" : 5, "name" : "刘超杰", "city" : "cs", "birth_day" : ISODate("1985-08-03T00:00:00Z"), "expectSalary" : 8000 } ``` shard4存储的数据： ```shell [root@192 mongodb]# ./bin/mongo -port 58011 shard4:PRIMARY> use lagou_resume switched to db lagou_resume shard4:PRIMARY> show tables; lagou_resume_datas shard4:PRIMARY> db.lagou_resume_datas.find(); { "_id" : ObjectId("5f701419ecdfe62f7c8cb674"), "resumeid" : 1, "name" : "zhangsan", "city" : "cs", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 18000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb67a"), "resumeid" : 7, "name" : "汪涵", "city" : "cs", "birth_day" : ISODate("1976-08-01T00:00:00Z"), "expectSalary" : 58000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb67c"), "resumeid" : 9, "name" : "nono", "city" : "cs", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 10000 } ``` ### 五、安全认证配置以及集群启动shell脚本 #### 5.1、创建管理员账号和普通账号 ##### 5.1.1、进入路由节点创建管理员账号 MongoDB服务端开启安全检查之前，至少需要一个管理员账号，admin数据库中的用户都被视为管理员。否则其它账号不会进入安全审查。 ```shell mongos> use admin switched to db admin mongos> db admin mongos> db.createUser({ ... user:"root", ... pwd:"123456", ... roles:[{role:"root",db:"admin"}] ... }); ``` ##### 5.1.2、创建普通账号 ```shell mongos> use lagou_resume switched to db lagou_resume mongos> db lagou_resume mongos> db.createUser({ ... user:"lagou_gx", ... pwd:"abc321", ... roles:[{role:"readWrite",db:"lagou_resume"}] ... }); Successfully added user: { "user" : "lagou_gx", "roles" : [ { "role" : "readWrite", "db" : "lagou_resume" } ] } ``` #### 5.2、集群启动shell脚本 ##### 5.2.1、编写集群启动shell脚本 再bin的同级目录编写startup.sh文件，并依次启动配置集群、分片集群、路由节点。 ```sh ./bin/mongod -f config/config17011.conf ./bin/mongod -f config/config17013.conf ./bin/mongod -f config/config17015.conf ./bin/mongod -f shard/shard1/shard37011.conf ./bin/mongod -f shard/shard1/shard37013.conf ./bin/mongod -f shard/shard1/shard37015.conf ./bin/mongod -f shard/shard1/shard37017.conf ./bin/mongod -f shard/shard2/shard47011.conf ./bin/mongod -f shard/shard2/shard47013.conf ./bin/mongod -f shard/shard2/shard47015.conf ./bin/mongod -f shard/shard2/shard47017.conf ./bin/mongod -f shard/shard3/shard57011.conf ./bin/mongod -f shard/shard3/shard57013.conf ./bin/mongod -f shard/shard3/shard57015.conf ./bin/mongod -f shard/shard3/shard57017.conf ./bin/mongod -f shard/shard4/shard58011.conf ./bin/mongod -f shard/shard4/shard58013.conf ./bin/mongod -f shard/shard4/shard58015.conf ./bin/mongod -f shard/shard4/shard58017.conf ./bin/mongos -f route/route-27017.conf ``` ##### 5.2.2、startup.sh执行权限 ```shell [root@192 mongodb]# chmod +x startup.sh [root@192 mongodb]# ll -rwx--x--x. 1 root root 774 Sep 27 13:32 startup.sh ``` #### 5.3、集群安全认证配置 ##### 5.3.1、生成数字安全密匙 ```shell [root@192 mongodb]# mkdir keyfile/mongodb -p [root@192 mongodb]# openssl rand -base64 756 > keyfile/mongodb/lagouKeyfile.file [root@192 mongodb]# chmod 600 keyfile/mongodb/lagouKeyfile.file ``` ##### 5.3.2、关闭集群所有节点 采用psmisc关闭所有mongo进程 ```shell [root@192 mongodb]# killall mongod [root@192 mongodb]# killall mongos ``` ##### 5.3.3、配置节点和分片节点开启安全认证和指定密匙文件 修改所有的配置节点和分片节点的配置。 ```properties auth=true keyFile=keyfile/mongodb/lagouKeyfile.file ``` ##### 5.3.4、配置路由节点密匙文件 修改路由节点配置。 ```properties keyFile=keyfile/mongodb/lagouKeyfile.file ``` ##### 5.3.5、重启集群 ```shell ./start.sh ``` ##### 5.3.6、安全验证 ```shell [root@192 mongodb]# ./bin/mongo --port 27017 MongoDB shell version v4.1.3 connecting to: mongodb://127.0.0.1:27017/ Implicit session: session { "id" : UUID("9ef748fe-69f4-4f41-8ecd-708c300855c5") } MongoDB server version: 4.1.3 mongos> use lagou_resume switched to db lagou_resume mongos> db lagou_resume mongos> show tables; Warning: unable to run listCollections, attempting to approximate collection names by parsing connectionStatus mongos> db.auth("lagou_gx","abc321") 1 mongos> show tables; lagou_resume_datas mongos> db.lagou_resume_datas.find(); { "_id" : ObjectId("5f701419ecdfe62f7c8cb674"), "resumeid" : 1, "name" : "zhangsan", "city" : "cs", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 18000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb67a"), "resumeid" : 7, "name" : "汪涵", "city" : "cs", "birth_day" : ISODate("1976-08-01T00:00:00Z"), "expectSalary" : 58000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb67c"), "resumeid" : 9, "name" : "nono", "city" : "cs", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 10000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb675"), "resumeid" : 2, "name" : "lisi", "city" : "gz", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 18000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb677"), "resumeid" : 4, "name" : "zhaoliu", "city" : "bj", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 12000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb678"), "resumeid" : 5, "name" : "刘超杰", "city" : "cs", "birth_day" : ISODate("1985-08-03T00:00:00Z"), "expectSalary" : 8000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb676"), "resumeid" : 3, "name" : "wangwu", "city" : "sh", "birth_day" : ISODate("2001-08-01T00:00:00Z"), "expectSalary" : 15000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb679"), "resumeid" : 6, "name" : "谢霆锋", "city" : "hk", "birth_day" : ISODate("1980-08-01T00:00:00Z"), "expectSalary" : 98000 } { "_id" : ObjectId("5f701419ecdfe62f7c8cb67b"), "resumeid" : 8, "name" : "zhaoliying", "city" : "cs", "birth_day" : ISODate("1987-08-01T00:00:00Z"), "expectSalary" : 38000 } ``` ### 六、SpringBoot访问分片集群 #### 6.1、maven创建springboot工程lagou-resume-snapshot ```xml 4.0.0 com.lagou.edu lagou-resume-snapshot 1.0-SNAPSHOT org.springframework.boot spring-boot-starter-data-mongodb 2.2.2.RELEASE org.springframework.boot spring-boot-starter-test 2.2.2.RELEASE test org.apache.maven.plugins maven-compiler-plugin 3.8.0 1.8 1.8 ``` #### 6.2、mongodb数据库连接配置 ```yaml spring: data: mongodb: host: 192.168.0.234 port: 27017 username: lagou_gx password: abc321 database: lagou_resume ``` #### 6.3、Entity类ResumeSnapshot ```java package com.lagou.edu.resume.snapshot.entity; import org.springframework.data.annotation.Id; import org.springframework.data.mongodb.core.mapping.Document; import org.springframework.data.mongodb.core.mapping.Field; import java.util.Date; @Document("lagou_resume_datas") public class ResumeSnapshot { @Id private String id; private Long resumeid; private String name; private String city; @Field("birth_day") private Date birthday; private double expectSalary; @Override public String toString() { return "ResumeSnapshot{" + "id='" + id + '\'' + ", resumeid='" + resumeid + '\'' + ", name='" + name + '\'' + ", city='" + city + '\'' + ", birthday=" + birthday + ", expectSalary=" + expectSalary + '}'; } .....getter setter..... } ``` #### 6.4、Repository接口LagouResumeSnapshotRespository ```java package com.lagou.edu.resume.snapshot.repository; import com.lagou.edu.resume.snapshot.entity.ResumeSnapshot; import org.springframework.data.mongodb.repository.MongoRepository; public interface LagouResumeSnapshotRepository extends MongoRepository { /** * 根据ResumeId查找resume * @param resumeid * @return */ ResumeSnapshot findByResumeidEquals(Long resumeid); /** * 删除简历 * @param resumeid */ void deleteByResumeidEquals(Long resumeid); } ``` #### 6.5、SpringBoot应用入口 ```java package com.lagou.edu.resume.snapshot; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.context.ApplicationContext; @SpringBootApplication public class LagouResumeSnapshotApplication { public static void main(String[] args) { ApplicationContext context = SpringApplication.run(LagouResumeSnapshotApplication.class,args); } } ``` #### 6.6、测试ResumeSnopshotTest ##### 6.6.1、保存简历快照 ```java /** * 保存简历快照测试 */ @Test public void saveResume(){ ResumeSnapshot resume = new ResumeSnapshot(); resume.setResumeid(12L); resume.setCity("tw"); resume.setName("周杰伦"); resume.setBirthday(new Date(System.currentTimeMillis()-(365*20*24*60*60*1000L))); resume.setExpectSalary(100000); ResumeSnapshot resumeSnapshot = resumeSnapshotRepository.save(resume); System.out.println("After save to mongo :"+resumeSnapshot); } ``` 测试结果： ```shell After save to mongo :ResumeSnapshot{id='5f71829cca497f59a23bc210', resumeid='12', name='周杰伦', city='tw', birthday=Tue Oct 03 14:28:44 CST 2000, expectSalary=100000.0} ``` ##### 6.6.2、查询所有简历 ```java /** * 查询所有简历快照测试 */ @Test public void findAll(){ List snapshotList = resumeSnapshotRepository.findAll(); snapshotList.forEach((snapshot)->{ System.out.println(snapshot); }); } ``` 测试结果： ```shell ResumeSnapshot{id='5f701419ecdfe62f7c8cb676', resumeid='3', name='wangwu', city='sh', birthday=Wed Aug 01 08:00:00 CST 2001, expectSalary=15000.0} ResumeSnapshot{id='5f701419ecdfe62f7c8cb679', resumeid='6', name='谢霆锋', city='hk', birthday=Fri Aug 01 08:00:00 CST 1980, expectSalary=98000.0} ResumeSnapshot{id='5f701419ecdfe62f7c8cb67b', resumeid='8', name='zhaoliying', city='cs', birthday=Sat Aug 01 09:00:00 CDT 1987, expectSalary=38000.0} ResumeSnapshot{id='5f71829cca497f59a23bc210', resumeid='12', name='周杰伦', city='tw', birthday=Tue Oct 03 14:28:44 CST 2000, expectSalary=100000.0} ResumeSnapshot{id='5f701419ecdfe62f7c8cb675', resumeid='2', name='lisi', city='gz', birthday=Wed Aug 01 08:00:00 CST 2001, expectSalary=18000.0} ResumeSnapshot{id='5f701419ecdfe62f7c8cb677', resumeid='4', name='zhaoliu', city='bj', birthday=Wed Aug 01 08:00:00 CST 2001, expectSalary=12000.0} ResumeSnapshot{id='5f701419ecdfe62f7c8cb678', resumeid='5', name='刘超杰', city='cs', birthday=Sat Aug 03 08:00:00 CST 1985, expectSalary=8000.0} ResumeSnapshot{id='5f701419ecdfe62f7c8cb674', resumeid='1', name='zhangsan', city='cs', birthday=Wed Aug 01 08:00:00 CST 2001, expectSalary=18000.0} ResumeSnapshot{id='5f701419ecdfe62f7c8cb67a', resumeid='7', name='汪涵', city='cs', birthday=Sun Aug 01 08:00:00 CST 1976, expectSalary=58000.0} ResumeSnapshot{id='5f701419ecdfe62f7c8cb67c', resumeid='9', name='nono', city='cs', birthday=Wed Aug 01 08:00:00 CST 2001, expectSalary=10000.0} ``` ##### 6.6.3、根据Resumeid查询简历快照 ```java /** * 根据resumeid获取简历快照测试 */ @Test public void getResumeByResumeid(){ Long resumeid = 12L; ResumeSnapshot resumeSnapshot = resumeSnapshotRepository .findByResumeidEquals(resumeid); System.out.println(resumeSnapshot); } ``` 测试结果 ```shell ResumeSnapshot{id='5f71829cca497f59a23bc210', resumeid='12', name='周杰伦', city='tw', birthday=Tue Oct 03 14:28:44 CST 2000, expectSalary=100000.0} ``` ##### 6.6.4、根据Resumeid删除简历 ```java /** * 根据resumeid删除简历测试 */ @Test public void removeResumeById(){ Long resumeid = 12L; resumeSnapshotRepository.deleteByResumeidEquals(12L); } ```