## 什么是openstack

https://blog.csdn.net/weixin_69897928/article/details/124779724



[openstack官网](https://www.openstack.org/)

![image-20221124101208510](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211241012623.png)

![image-20221124101232734](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211241012823.png)

![image-20221124101252355](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211241012449.png)

官网的翻译只到M版，可以进到Q版的文档修改URL，添加zh_CN，可以读取未公布的翻译版，但只有部份翻译



![image-20221124101325336](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211241013418.png)





学习架构

![image-20221124101509007](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211241015079.png)



### openstack架构图

![image-20221124102333982](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211241023069.png)





### OpenStack项目间的逻辑关系

以下列举OpenStack的6个最重要的核心项目。

<font color='red'>Keystone</font>：Keystone是OpenStack的认证服务，Keystone为所有的OpenStack组件提供认证和访问策略服务，它依赖自身REST系统进行工作，主要对Swift、Glance、Nova等进行认证与授权，它对动作消息中的来源者进行合法性鉴定。 Keystone采用两种授权方式，一种基于用户名/密码，另一种基于令牌（Token）。除此之外，Keystone提供以下三种服务：

令牌服务：令牌中含有授权用户或群组的授权信息，授权给合法用户或群组。
目录服务：目录中含有合法用户或群组的可用服务列表。
策略服务：利用Keystone具体指定用户或群组的某些访问权限。

<font color='red'>Nova</font>：Nova是一套控制器，用于为单个用户或使用群组管理虚拟机实例的整个生命周期，根据用户需求来提供虚拟服务，Nova负责管理整个云的计算资源、网络资源、授权及测度。虽然Nova本身并不提供任何虚拟能力，但是它将使用libvirt API与虚拟机的宿主机进行交互。Nova通过Web服务API来对外提供处理接口。

<font color='red'>Glance</font>：Glance负责OpenStack的镜像服务，Glance是一套虚拟机镜像发现、注册、检索系统，它提供虚拟机镜像的存储，查询和检索功能，为nova进行服务，依赖于存储服务和数据库服务。

<font color='red'>Swift</font>：Swift为OpenStack提供了一种分布式、持续虚拟对象存储。Swift具有跨节点的存储能力。Swift组件有冗余和失效备援管理功能，也能够处理归档和媒体流，特别是对大数据和大容量的测度非常高效。

<font color='red'>Cinder</font>：Cinder是OpenStack的块存储服务组件，它管理所有块存储设备，为VM服务。

<font color='red'>Neutorn</font>：Neutorn为OpenStack提供虚拟的网络功能，为每个不同的租户建立独立的网络环境。



[**openstack工作原理**](https://blog.csdn.net/lv74134/article/details/119814696)



## 实验环境

|   主机名    |     IP地址      | 备注 |
| :---------: | :-------------: | :--: |
| controller  | 192.168.245.221 | 2U5G |
| computer-01 | 192.168.245.222 | 1U2G |
| computer-02 | 192.168.245.223 | 1U2G |
|     nfs     | 192.168.188.17  |      |



<font color='red'>**注意：**</font>

**1，这是openstack q版安装教程笔记，其中密码默认使用官方文档中的密码，生产中建议替换掉，另外数据库没用设定root密码，**
**如果使用请按照生产标准配置密码。**
**2，配置需求：**
**控制节点默认为controller，计算节点为compute**
**controller节点建议2核4g以上**
**compute节点1核2g即可**

**3.所有组件都是yum安装，日志全在/var/log目录下**



**<font color='red'>若无特别说明都是在controller操作</font>**





## 基础服务

### 关闭NetworkManager

```shell
# 所有节点都操作
systemctl stop NetworkManager
systemctl disable NetworkManager
systemctl status NetworkManager
```



### 上传安装包

**<font color='red'>controller</font>**

```shell
[root@controller ~]# mkdir /data
[root@controller ~]# cd /data/   # 上传包
[root@controller data]# ll -h
总用量 1.9G
-rw-r--r-- 1 root root 1.9G 5月  12 09:41 openstack-queens.tar.gz
```



**<font color='red'>两个compute节点都要创建一个/data/目录</font>**

```shell
mkdir /data
```

**<font color='red'>controller</font>**

```shell
# 分发包
[root@controller data]# scp openstack-queens.tar.gz 192.168.245.222:`pwd`
[root@controller data]# scp openstack-queens.tar.gz 192.168.245.223:`pwd`
```



### 所有机子都要准备好阿里源的base和OpenStack源

```shell

wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

yum makecache
```

 [阿里源openstack源地址](https://mirrors.aliyun.com/centos/7/cloud/x86_64/openstack-queens/?spm=a2c6h.25603864.0.0.29255ec81WOduZ)

### acii码的问题解决

```shell
# acii码的问题解决方法：
# 在/etc/environment文件中添加定义 

LANG=en_US.utf-8
LC_ALL=en_US.utf-8
```

```shell
# 写好配置分发
[root@controller ~]# vim /etc/environment
[root@controller ~]# scp /etc/environment 192.168.245.222:/etc/environment
[root@controller ~]# scp /etc/environment 192.168.245.223:/etc/environment
```



### 映射

```shell
[root@controller ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.245.221 controller
192.168.245.222 computer-01
192.168.245.223 computer-02

[root@controller ~]# scp /etc/hosts 192.168.245.222:/etc/hosts
[root@controller ~]# scp /etc/hosts 192.168.245.223:/etc/hosts
```



### 时间同步

```shell
# 所有节点安装：
yum install chrony -y

# 控制节点
#（选择性更改ntp服务器上游，此处不做更改）
vim /etc/chrony.conf
# 只保留第四行的 server 1.centos.pool.ntp.org iburst剩下server注释掉
26行 allow 192.168.0.0/16

systemctl restart chronyd

# 计算节点：
vim /etc/chrony.conf
# 修改3行
server 192.168.245.221 iburst
# 剩下server注释掉

# 重启chronyd
systemctl restart chronyd
date
```



### 解压包

```shell
# 所有节点都操作
tar xf /data/openstack-queens.tar.gz -C /opt/
```



### 建立OpenStack源

```shell
[root@controller ~]# ll /opt/
[root@controller ~]# cd /etc/yum.repos.d/
[root@controller yum.repos.d]# vim openstack.repo 
[openstack]
name=openstack
baseurl=file:///opt/openstack
enabled=1
gpgcheck=0


[root@controller yum.repos.d]# scp openstack.repo 192.168.245.222:`pwd`
[root@controller yum.repos.d]# scp openstack.repo 192.168.245.223:`pwd`


# 所有节点执行
yum makecache
```



### 安装mariadb

**<font color='red'>controller</font>**

```shell
[root@controller ~]# yum install mariadb mariadb-server python2-PyMySQL -y

# 增加mariadb的openstack配置
[root@controller ~]# echo '[mysqld]
bind-address = 192.168.245.221
default-storage-engine = innodb
innodb_file_per_table = on #设置每个表的独立表空间文件
max_connections = 4096
collation-server = utf8_general_ci
character-set-server = utf8' >> /etc/my.cnf.d/openstack.cnf

systemctl start mariadb
systemctl enable mariadb

[root@controller ~]# mysql_secure_installation   # 数据库安全初始化，不做会同步有问题


[root@controller ~]# mysql -uroot -p
```



### 直接创建核心组件所需数据库以及使用用户及密码

<font color='red'>**kestone相关**</font>

```mysql
create database keystone;
grant all on keystone.* to 'keystone'@'localhost' identified by 'KEYSTONE_DBPASS';
grant all on keystone.* to 'keystone'@'%' identified by 'KEYSTONE_DBPASS';
```



**<font color='red'>glance相关</font>**

```mysql
create database glance;
grant all on glance.* to 'glance'@'localhost' identified by 'GLANCE_DBPASS';
grant all on glance.* to 'glance'@'%' identified by 'GLANCE_DBPASS';
```



**<font color='red'>nova相关</font>**

```mysql
create database nova;
grant all on nova.* to 'nova'@'localhost' identified by 'NOVA_DBPASS';
grant all on nova.* to 'nova'@'%' identified by 'NOVA_DBPASS';
```



<font color='red'>**nova api相关**</font>

```mysql
create database nova_api;
grant all on nova_api.* to 'nova'@'localhost' identified by 'NOVA_DBPASS';
grant all on nova_api.* to 'nova'@'%' identified by 'NOVA_DBPASS';
```



<font color='red'>**nova_cell0相关（q版新增）**</font>

```mysql
CREATE DATABASE nova_cell0;
GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'localhost' \
  IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'%' \
  IDENTIFIED BY 'NOVA_DBPASS';
```



**<font color='red'>neutron相关</font>**

```mysql
create database neutron;
grant all on neutron.* to 'neutron'@'localhost' identified by 'NEUTRON_DBPASS';
grant all on neutron.* to 'neutron'@'%' identified by 'NEUTRON_DBPASS';
```



**<font color='red'>跳过nosql（涉及计费服务）</font>**
<font color='red'>**私有云没必要计费**</font>



```mysql
FLUSH PRIVILEGES;
```



### 安装消息队列

```shell
[root@controller ~]# yum install rabbitmq-server -y
[root@controller ~]# systemctl start rabbitmq-server.service
[root@controller ~]# systemctl status rabbitmq-server.service
[root@controller ~]# systemctl enable rabbitmq-server.service

[root@controller ~]# rabbitmqctl add_user openstack RABBIT_PASS  （增加OpenStack用户设置密码为RABBIT_PASS）
[root@controller ~]# rabbitmqctl set_permissions openstack ".*" ".*" ".*"  （可读可写可配置）
[root@controller ~]# rabbitmq-plugins enable rabbitmq_management (启用rabbitmq管理插件）默认登录账号guest 密码guest

[root@controller ~]# ss -lp | grep 15672
```



```shell
# 如果更改host没用断开再重进，导致节点名字命bash变化引起rabbit不可用的情况，请按照以下步骤修复：
# 将mq的进程杀掉
ps -ef | grep rabbitmq | grep -v grep | awk '{print $2}' | xargs kill -9
# 启动mq
rabbitmq-server -detached
# 查询mq的状态
rabbitmqctl status
# 验证rabbitmq：
http://192.168.245.221:15672/   guest guest
```



### 安装memcached

<font color='red'>**注意更改memcached的监听ip默认是127.0.0.1，此处更改为0.0.0.0生产慎重更改**</font>

```shell
[root@controller ~]# yum install memcached python-memcached -y
[root@controller ~]# sed -i "s#127.0.0.1#0.0.0.0#g" /etc/sysconfig/memcached  （或者改成本机ip）在生产上需要改成本机IP
[root@controller ~]# systemctl start memcached
[root@controller ~]# systemctl status memcached
[root@controller ~]# systemctl enable memcached
```



### 安装(etcd)

```shell
[root@controller ~]# yum install etcd -y

# 编辑 
[root@controller ~]# cp /etc/etcd/etcd.conf /etc/etcd/etcd.conf.bak
[root@controller ~]# vim /etc/etcd/etcd.conf   # 清空添加,ip地址请根据业务更改
#[Member]
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.245.221:2380"   
ETCD_LISTEN_CLIENT_URLS="http://192.168.245.221:2379"  
ETCD_NAME="controller"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.245.221:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.245.221:2379"
ETCD_INITIAL_CLUSTER="controller=http://192.168.245.221:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-01"
ETCD_INITIAL_CLUSTER_STATE="new"


# 启动并开机自启etcd：
[root@controller ~]# systemctl start etcd
[root@controller ~]# systemctl status etcd
[root@controller ~]# systemctl enable etcd
```



**以上为基础服务**



## keystone认证服务

**通过阿帕奇启动：**
**认证管理，授权管理，服务目录**
**认证：账号密码**
**授权：授权管理**
**服务目录： 记录作用记录每个服务的相关信息**

```shell
yum install openstack-utils -y  #（自动改配置文件工具建议所有节点安装此包后面配置生产都是用的他）
yum install openstack-keystone httpd mod_wsgi -y   # keystone基于httpd启动,httpd需要mod_wsgi模块才能运行python开发的程序
```



<font color='red'>**遇到错误**</font>

**在安装 openstack keystone 时出现问题**

```shell
错误：软件包：python2-qpid-proton-0.22.0-1.el7.x86_64 (centos-openstack-queens)
          需要：qpid-proton-c(x86-64) = 0.22.0-1.el7
          可用: qpid-proton-c-0.14.0-2.el7.x86_64 (extras)
              qpid-proton-c(x86-64) = 0.14.0-2.el7
          可用: qpid-proton-c-0.17.0-4.el7.x86_64 (centos-openstack-queens)
              qpid-proton-c(x86-64) = 0.17.0-4.el7
          可用: qpid-proton-c-0.22.0-1.el7.x86_64 (centos-openstack-queens)
              qpid-proton-c(x86-64) = 0.22.0-1.el7
          正在安装: qpid-proton-c-0.35.0-1.el7.x86_64 (epel)
              qpid-proton-c(x86-64) = 0.35.0-1.el7
 您可以尝试添加 --skip-broken 选项来解决该问题
 您可以尝试执行：rpm -Va --nofiles --nodigest
```



**解决办法**

1. 去阿里开源社区下载一个https://developer.aliyun.com/packageSearch?word=mitaka
2. 搜索：python2-qpid-proton-0.22.0-1.el7.x86_64

![在这里插入图片描述](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121158495.png)

3. 下好后放到用户目录下

   ```shell
   # 由于包有依赖关系：不能直接rpm直接安装
   [root@controller ~]# rpm -ivh python2-qpid-proton-0.22.0-1.el7.x86_64.rpm 
   错误：依赖检测失败：
   	qpid-proton-c(x86-64) = 0.22.0-1.el7 被 python2-qpid-proton-0.22.0-1.el7.x86_64 需要
   	libqpid-proton.so.11()(64bit) 被 python2-qpid-proton-0.22.0-1.el7.x86_64 需要
   ```

   ```shell
   # 使用 yum 会自动将包和依赖包安装好
   [root@controller ~]# yum install -y python2-qpid-proton-0.22.0-1.el7.x86_64.rpm
   [root@controller ~]# yum install openstack-keystone httpd mod_wsgi -y   #重新安装
   ```
   
   

**更改keystone配置**

```shell
openstack-config --set /etc/keystone/keystone.conf database connection  mysql+pymysql://keystone:KEYSTONE_DBPASS@controller/keystone

openstack-config --set /etc/keystone/keystone.conf token provider  fernet
```

```shell
[root@controller data]# grep -v "#\|^$" /etc/keystone/keystone.conf

[database]
connection = mysql+pymysql://keystone:KEYSTONE_DBPASS@controller/keystone   # 数据库类型+pymysql包://用户名:密码@用户地址/库

[token]
provider = fernet   # 令牌的发放方式fernet为令牌的提供者(也就是令牌的一种方式,fernet方式小巧且加密)；另外两种是uuid、pki
```



**同步数据库**

```shell
# su -s表示给bash环境,因为keystone默认不是/bin/bash (su -c keystone表示以keystone用户身份执行命令)
# 应该有44张表
su -s /bin/sh -c "keystone-manage db_sync" keystone
```



**初始化fernet**

```shell
[root@controller ~]# cp /etc/keystone/keystone.conf /etc/keystone/keystone.conf.bak
[root@controller ~]# grep -Ev "#|^$" /etc/keystone/keystone.conf.bak > /etc/keystone/keystone.conf
# 自己初始化，给自己一个身份
keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone
keystone-manage credential_setup --keystone-user keystone --keystone-group keystone
```



```shell
# 在Queens发布之前，keystone需要在两个独立的端口上运行，以容纳Identity v2 API
# 该API通常在端口35357上运行单独的仅管理服务。
# 通过删除v2 API, keystone可以在同一端口上运行所有端口
# q版的keystone初始化认证，去掉35357认证，通过5000可以全部认证：
keystone-manage bootstrap --bootstrap-password ADMIN_PASS \
  --bootstrap-admin-url http://controller:5000/v3/ \
  --bootstrap-internal-url http://controller:5000/v3/ \
  --bootstrap-public-url http://controller:5000/v3/ \
  --bootstrap-region-id RegionOne
```



**配置httpd**

```shell
# 编辑/etc/httpd/conf/httpd.conf文件，配置ServerName选项为控制节点

echo "ServerName controller"  >> /etc/httpd/conf/httpd.conf   # 优化apache
```



**q版需要创建软连接**

```shell
ln -s /usr/share/keystone/wsgi-keystone.conf /etc/httpd/conf.d/

# 进行优化，启动apache
systemctl start httpd.service
systemctl enable httpd.service
```



**创建注册账户，注册keystone自己api**

```shell
# 声明安装注册参数

export OS_USERNAME=admin
export OS_PASSWORD=ADMIN_PASS
export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_AUTH_URL=http://controller:5000/v3
export OS_IDENTITY_API_VERSION=3
```



**q版创建域（地区）、项目（租户）、用户、角色 因为q版开始在初始化验证部分生成了default域，此处无需创建：**
**只需创建service服务项即可：**

```shell
openstack project create --domain default --description "Service Project" service
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121213469.png)



**<font color='red'>报错原因</font>**
openstack未找到命令
**解决方法**

```shell
yum install -y python2-openstackclient
```



**测试keystone服务：**

```shell
openstack token issue
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121214604.png)



**<font color='red'>环境变量退出终端变量会消失。</font>**
**家目录下创建环境变量脚本：**

```shell
echo 'export OS_PROJECT_DOMAIN_NAME=Default
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=ADMIN_PASS
export OS_AUTH_URL=http://controller:5000/v3
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2' >/root/admin-openrc

source ~/admin-openrc
```

```shell
# 直接写到，.bashrc中
echo 'source ~/admin-openrc' >> ~/.bashrc
# 再次验证：
openstack user list
openstack token issue
# 无报错则keystone正常
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121215951.png)





## glance相关：（q版没有过多改动）

### 在keystone上创建glance相关关联角色

```shell
openstack user create --domain default --password GLANCE_PASS glance   # 明文创建(生产不允许明文)
或
openstack user create --domain default --password-prompt glance   # 交互式创建
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121219978.png)



### 添加admin角色到glance用户和service项目上（这个命令执行后没有输出）

```shell
openstack role add --project service --user glance admin
```

```shell
# 验证：
openstack role assignment list
openstack user list  #（可以与上面的表对应）
openstack project list  #（与上面一样）
```



### 在keystone上创建服务和api

```shell
openstack service create --name glance   --description "OpenStack Image" image   # 创建glance服务实体
openstack endpoint create --region RegionOne   image public http://controller:9292   # 公共端口
openstack endpoint create --region RegionOne   image internal http://controller:9292  # 网络端口
openstack endpoint create --region RegionOne   image admin http://controller:9292  # vip端口，最高级
```



### 安装glance服务

```shell
yum install openstack-glance -y
```



### 修改glance配置文件

```shell
# 用openstack配置命令生成（glanceapi的文件）

cp /etc/glance/glance-api.conf /etc/glance/glance-api.conf.bak
cp /etc/glance/glance-registry.conf /etc/glance/glance-registry.conf.bak
```



### 用openstack配置命令生成（glanceapi的文件）

```shell
openstack-config --set /etc/glance/glance-api.conf  database  connection  mysql+pymysql://glance:GLANCE_DBPASS@controller/glance
openstack-config --set /etc/glance/glance-api.conf  glance_store stores  file,http
openstack-config --set /etc/glance/glance-api.conf  glance_store default_store  file
openstack-config --set /etc/glance/glance-api.conf  glance_store filesystem_store_datadir  /var/lib/glance/images/
openstack-config --set /etc/glance/glance-api.conf  keystone_authtoken auth_uri  http://controller:5000
openstack-config --set /etc/glance/glance-api.conf  keystone_authtoken auth_url  http://controller:5000 # 因为q版keystone验证改为全部5000端口所以此处有更改

openstack-config --set /etc/glance/glance-api.conf  keystone_authtoken memcached_servers  controller:11211
openstack-config --set /etc/glance/glance-api.conf  keystone_authtoken auth_type  password
openstack-config --set /etc/glance/glance-api.conf  keystone_authtoken project_domain_name  default
openstack-config --set /etc/glance/glance-api.conf  keystone_authtoken user_domain_name  default
openstack-config --set /etc/glance/glance-api.conf  keystone_authtoken project_name  service
openstack-config --set /etc/glance/glance-api.conf  keystone_authtoken username  glance
openstack-config --set /etc/glance/glance-api.conf  keystone_authtoken password  GLANCE_PASS
openstack-config --set /etc/glance/glance-api.conf  paste_deploy flavor  keystone
```



### 用openstack配置命令生成（glanceregistry文件）

```shell
openstack-config --set /etc/glance/glance-registry.conf  database  connection  mysql+pymysql://glance:GLANCE_DBPASS@controller/glance
openstack-config --set /etc/glance/glance-registry.conf  keystone_authtoken auth_uri  http://controller:5000
openstack-config --set /etc/glance/glance-registry.conf  keystone_authtoken auth_url  http://controller:5000 #同上
openstack-config --set /etc/glance/glance-registry.conf  keystone_authtoken memcached_servers  controller:11211
openstack-config --set /etc/glance/glance-registry.conf  keystone_authtoken auth_type  password
openstack-config --set /etc/glance/glance-registry.conf  keystone_authtoken project_domain_name  default
openstack-config --set /etc/glance/glance-registry.conf  keystone_authtoken user_domain_name  default
openstack-config --set /etc/glance/glance-registry.conf  keystone_authtoken project_name  service
openstack-config --set /etc/glance/glance-registry.conf  keystone_authtoken username  glance
openstack-config --set /etc/glance/glance-registry.conf  keystone_authtoken password  GLANCE_PASS
openstack-config --set /etc/glance/glance-registry.conf  paste_deploy flavor  keystone
```





### 两个文件部分配置解释

```shell
[database]
# ...
connection = mysql+pymysql://glance:GLANCE_DBPASS@controller/glance

[keystone_authtoken]   # keystone验证部分，告诉glance服务在哪
# ...
auth_uri = http://controller:5000
auth_url = http://controller:5000
memcached_servers = controller:11211   # memcached(/etc/sysconfig/memcached)监听端口，IP地址/主机名映射:11211
auth_type = password   # 验证的方式，以账号密码去验证
project_domain_name = Default   # 验证域的项目名
user_domain_name = Default   # 验证域的用户名
project_name = service   # 选择的服务项，前面创建的
username = glance   # glance的账号
password = GLANCE_PASS   # glance的密码

[paste_deploy]
# ...
flavor = keystone   # 令牌谁应用

[glance_store]
# ...
stores = file,http   # 镜像存储方式:文件、http
default_store = file   # 默认存储方式是文件
filesystem_store_datadir = /var/lib/glance/images/   # 在磁盘存储的镜像的文件夹在哪
```



### 同步数据库

```shell
su -s /bin/sh -c "glance-manage db_sync" glance   # 有输出，应该有15张表
```

 ![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121226845.png)



### 启动服务

```shell
systemctl start openstack-glance-api.service openstack-glance-registry.service
systemctl enable openstack-glance-api.service openstack-glance-registry.service
```



### 上传镜像测试：（镜像请自行下载）

**使用Cirros对镜像服务进行验证，Cirros是一个小型的Linux镜像可以用来帮助进行OpenStack部署测试**

```shell
[root@controller ~]# cd /data/
[root@controller data]# ll
总用量 1959092
-rw-r--r-- 1 root root   13287936 6月  12 12:27 cirros-0.3.4-x86_64-disk.img
-rw-r--r-- 1 root root 1992816640 6月  12 11:04 openstack-queens.tar.gz
```



```shell
上传命令：openstack image create "cirros"（名字） --file cirros-0.3.4-x86_64-disk.img --disk-format qcow2 （格式）\
--container-format bare（说明是openstack的镜像） --public （指定为公共镜像）
```

```shell
[root@controller ~]# cd /data/
# 使用QCOW2磁盘格式，bare容器格式上传镜像到镜像服务并设置公共可见，这样所有的项目都可以访问
# OpenStack是动态生成ID

[root@controller data]# openstack image create "cirros"  --file cirros-0.3.4-x86_64-disk.img --disk-format qcow2 \
--container-format bare --public
```

```shell
# 查看相关镜像：
openstack image list
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121237982.png)

 ![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121237105.png)





## nova的计算服务（核心服务）

**<font color='red'>在controller节点操作</font>**

### keystone创建系统用户

```shell
openstack user create --domain default --password-prompt nova   # 交互式
# 或者
openstack user create --domain default --password NOVA_PASS nova   # 明文


openstack role add --project service --user nova admin
```



### 在keystone上注册服务和api

```shell
openstack service create --name nova   --description "OpenStack Compute" compute   # 是计算功能不是计算节点
openstack endpoint create --region RegionOne   compute public http://controller:8774/v2.1
openstack endpoint create --region RegionOne   compute internal http://controller:8774/v2.1
openstack endpoint create --region RegionOne   compute admin http://controller:8774/v2.1
```



### 创建placement api相关

#### 在keystone创建系统用户

```shell
openstack user create --domain default --password-prompt placement   # 交互式；placement分布式锁需要的一个平台
# 或者
openstack user create --domain default --password PLACEMENT_PASS placement   # 明文


openstack role add --project service --user placement admin
```



#### 在keystone上注册服务和api

**在服务目录中创建Placement API条目**

```shell
openstack service create --name placement --description "Placement API" placement
openstack endpoint create --region RegionOne placement public http://controller:8778
openstack endpoint create --region RegionOne placement internal http://controller:8778
openstack endpoint create --region RegionOne placement admin http://controller:8778
```





### 在controller节点安装nova服务

```shell
yum install openstack-nova-api openstack-nova-conductor \
  openstack-nova-console openstack-nova-novncproxy \
  openstack-nova-scheduler openstack-nova-placement-api -y
```

```shell
openstack-nova-api: 接收数据，接收元数据
openstack-nova-conductor: 中间件，连接数据库的中间件，nova不能直接连接数据库
openstack-nova-console: 提供console连接方式
openstack-nova-novncproxy: 提供vnc界面，其实就是UI界面
openstack-nova-scheduler: 调度器，负责计算节点资源、计算节点资源落在哪里、告诉生成器如何生成
openstack-nova-placement-api: 分布式锁接收的入口
```



### 生成配置文件

```shell
cp /etc/nova/nova.conf /etc/nova/nova.conf.bak
```

**<font color='red'>需要注意IP地址，请根据业务修改</font>**

```shell
openstack-config --set /etc/nova/nova.conf  DEFAULT enabled_apis  osapi_compute,metadata
openstack-config --set /etc/nova/nova.conf  DEFAULT transport_url rabbit://openstack:RABBIT_PASS@controller
openstack-config --set /etc/nova/nova.conf  DEFAULT auth_strategy  keystone
openstack-config --set /etc/nova/nova.conf  DEFAULT my_ip  192.168.245.221
openstack-config --set /etc/nova/nova.conf  DEFAULT use_neutron  True
openstack-config --set /etc/nova/nova.conf  DEFAULT firewall_driver  nova.virt.firewall.NoopFirewallDriver
openstack-config --set /etc/nova/nova.conf  api_database connection  mysql+pymysql://nova:NOVA_DBPASS@controller/nova_api
openstack-config --set /etc/nova/nova.conf  database  connection  mysql+pymysql://nova:NOVA_DBPASS@controller/nova
openstack-config --set /etc/nova/nova.conf  glance api_servers  http://controller:9292
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  auth_url  http://controller:5000/v3
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  memcached_servers  controller:11211
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  auth_type  password
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  project_domain_name  default
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  user_domain_name  default
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  project_name  service
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  username  nova
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  password  NOVA_PASS
openstack-config --set /etc/nova/nova.conf  oslo_concurrency lock_path  /var/lib/nova/tmp
openstack-config --set /etc/nova/nova.conf  libvirt  virt_type qemu
openstack-config --set /etc/nova/nova.conf  vnc vncserver_listen  '$my_ip'
openstack-config --set /etc/nova/nova.conf  vnc vncserver_proxyclient_address  '$my_ip'
openstack-config --set /etc/nova/nova.conf  placement os_region_name RegionOne
openstack-config --set /etc/nova/nova.conf  placement project_domain_name Default
openstack-config --set /etc/nova/nova.conf  placement user_domain_name  Default
openstack-config --set /etc/nova/nova.conf  placement project_name  service
openstack-config --set /etc/nova/nova.conf  placement auth_type  password
openstack-config --set /etc/nova/nova.conf  placement auth_url  http://controller:5000/v3
openstack-config --set /etc/nova/nova.conf  placement username  placement
openstack-config --set /etc/nova/nova.conf  placement password  PLACEMENT_PASS
```





#### 配置文件配置项解析

```shell
# 在[DEFAULT]部分，只启用计算数据和元数据API
[DEFAULT]
# ...
enabled_apis = osapi_compute,metadata

# Compute数据库选择的密码来代替NOVA_DBPASS
[api_database]
# ...
connection = mysql+pymysql://nova:NOVA_DBPASS@controller/nova_api

[database]
# ...
connection = mysql+pymysql://nova:NOVA_DBPASS@controller/nova

# 在[DEFAULT]部分，配置RabbitMQ消息队列访问：
# 在RabbitMQ中为openstack选择的密码替换RABBIT_PASS
[DEFAULT]
# ...
transport_url = rabbit://openstack:RABBIT_PASS@controller


# 在[api]和[keystone_authtoken]部分，配置认证服务访问：
# 使用在身份认证服务中设置的nova用户的密码替换NOVA_PASS
[api]
# ...
auth_strategy = keystone

[keystone_authtoken]
# ...
auth_url = http://controller:5000/v3
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = NOVA_PASS

# 配置my_ip来使用控制节点的管理接口的IP地址，进行vnc连接的
[DEFAULT]
# ...
my_ip = 10.0.0.11

#在 [DEFAULT] 部分，使能 Networking 服务：
# 默认情况下，计算服务使用内置的防火墙服务。
# 由于网络服务包含了防火墙服务，
# 必须使用nova.virt.firewall.NoopFirewallDriver防火墙服务来禁用掉计算服务内置的防火墙服务
# 相当于云服务器的安全组
[DEFAULT]
# ...
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver


# 在[vnc]部分，配置VNC代理使用控制节点的管理接口IP地址 ：
[vnc]
enabled = true
# ...
server_listen = $my_ip
server_proxyclient_address = $my_ip   # 代理，没有就遇上一个配置项一样


# 在 [glance] 区域，配置镜像服务 API 的位置
[glance]
# ...
api_servers = http://controller:9292


# 在 [oslo_concurrency] 部分，配置锁路径
[oslo_concurrency]
# ...
lock_path = /var/lib/nova/tmp


# 在该[placement]部分中，配置Placement API
[placement]
# ...
os_region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = http://controller:5000/v3
username = placement
password = PLACEMENT_PASS
```



### 更改placement httpd设置

```shell
# 跨域设置
# 由于打包错误，必须通过将以下配置添加到以下内容来启用对Placement API的访问，解决bug

vim /etc/httpd/conf.d/00-nova-placement-api.conf
#添加：
<Directory /usr/bin>
   <IfVersion >= 2.4>
      Require all granted
   </IfVersion>
   <IfVersion < 2.4>
      Order allow,deny
      Allow from all
   </IfVersion>
</Directory>


# 重启httpd
systemctl restart httpd
```

 

### 同步数据库

**<font color='red'>nova-api</font>**

```shell
su -s /bin/sh -c "nova-manage api_db sync" nova
```

**<font color='red'>cell0</font>**

```shell
su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova
```

**<font color='red'>生成cell1 cell</font>**

```shell
[root@controller data]# su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova
911d85c8-e044-48c4-aafe-afb50b22f08b

# 会生成一个id用于验证：911d85c8-e044-48c4-aafe-afb50b22f08b
```

<font color='red'>**nova**</font>

```shell
su -s /bin/sh -c "nova-manage db sync" nova   # 会有输出提示，不是报错可忽略;生产上需要解决
```



```shell
# 验证nova cell0 and cell1
# 会返回id值
nova-manage cell_v2 list_cells
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121247791.png)



### 启动开机自启并服务

```shell
systemctl enable openstack-nova-api.service \
  openstack-nova-consoleauth.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service openstack-nova-novncproxy.service openstack-nova-console
```

```shell
systemctl start openstack-nova-api.service \
  openstack-nova-consoleauth.service openstack-nova-scheduler.service \
  openstack-nova-conductor.service openstack-nova-novncproxy.service
```



### 验证

```shell
openstack compute service list  # 看state
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121250793.png)





## 计算（compute）节点安装nova服务

**<font color='red'>computer-01操作</font>**

```shell
yum install openstack-nova-compute -y
yum install openstack-utils -y
```

```shell
#报错解决：
如果出现Requires: qemu-kvm-rhev >= 2.10.0
在base源里增加：
[Virt]
name=CentOS-$releasever - Base
baseurl=http://mirrors.aliyun.com/centos/7/virt/x86_64/kvm-common/
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

yum makecache
```



### 配置

```shell
cp /etc/nova/nova.conf /etc/nova/nova.conf.bak
```



**<font color='red'>需要注意IP地址，请根据业务修改</font>**

```shell
openstack-config --set /etc/nova/nova.conf  DEFAULT enabled_apis  osapi_compute,metadata
openstack-config --set /etc/nova/nova.conf  DEFAULT auth_strategy  keystone
openstack-config --set /etc/nova/nova.conf  DEFAULT my_ip  192.168.245.222
openstack-config --set /etc/nova/nova.conf  DEFAULT use_neutron  True
openstack-config --set /etc/nova/nova.conf  DEFAULT firewall_driver  nova.virt.firewall.NoopFirewallDriver
openstack-config --set /etc/nova/nova.conf  glance api_servers  http://controller:9292
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  auth_url  http://controller:5000/v3
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  memcached_servers  controller:11211
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  auth_type  password
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  project_domain_name  default
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  user_domain_name  default
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  project_name  service
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  username  nova
openstack-config --set /etc/nova/nova.conf  keystone_authtoken  password  NOVA_PASS
openstack-config --set /etc/nova/nova.conf  oslo_concurrency lock_path  /var/lib/nova/tmp
openstack-config --set /etc/nova/nova.conf  DEFAULT transport_url rabbit://openstack:RABBIT_PASS@controller
openstack-config --set /etc/nova/nova.conf  vnc enabled  True
openstack-config --set /etc/nova/nova.conf  vnc vncserver_listen  0.0.0.0
openstack-config --set /etc/nova/nova.conf  vnc vncserver_proxyclient_address  '$my_ip'
openstack-config --set /etc/nova/nova.conf  vnc novncproxy_base_url  http://controller:6080/vnc_auto.html
openstack-config --set /etc/nova/nova.conf  placement os_region_name RegionOne
openstack-config --set /etc/nova/nova.conf  placement project_domain_name Default
openstack-config --set /etc/nova/nova.conf  placement user_domain_name  Default
openstack-config --set /etc/nova/nova.conf  placement project_name  service
openstack-config --set /etc/nova/nova.conf  placement auth_type  password
openstack-config --set /etc/nova/nova.conf  placement auth_url  http://controller:5000/v3
openstack-config --set /etc/nova/nova.conf  placement username  placement
openstack-config --set /etc/nova/nova.conf  placement password  PLACEMENT_PASS
openstack-config --set /etc/nova/nova.conf  libvirt  virt_type  qemu
#openstack-config --set /etc/nova/nova.conf  scheduler discover_hosts_in_cells_interval 300 #不明确是在compute节点还算在controller加，不加不影响使用
```



### 启动并开机自启服务

```
systemctl enable libvirtd.service openstack-nova-compute.service
systemctl start libvirtd.service openstack-nova-compute.service
```



**<font color='red'>注意：</font>**
**<font color='red'>回到controller节点</font>**

```shell
# 验证：
openstack compute service list --service nova-compute
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121317375.png)

```shell
# 创建compute节点网格
su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
```

```shell
# 如果增加新节点，可在/etc/nova/nova.conf增加配置（此处不是必备步骤）
# discover_hosts_in_cells_interval = 300
# 验证：

openstack compute service list
openstack catalog list
openstack image list
nova-status upgrade check
```



## neutron网络服务安装

**<font color='red'>此处为平面网络未涉及vxlan</font>**
**在controller节点：**

### 创建用户关联角色

```shell
openstack user create --domain default --password NEUTRON_PASS neutron
openstack role add --project service --user neutron admin
```



### 创建服务注册api

```shell
openstack service create --name neutron   --description "OpenStack Networking" network
openstack endpoint create --region RegionOne   network public http://controller:9696
openstack endpoint create --region RegionOne   network internal http://controller:9696
openstack endpoint create --region RegionOne   network admin http://controller:9696
```



### 安装相关包

```shell
yum install openstack-neutron openstack-neutron-ml2 \
  openstack-neutron-linuxbridge ebtables ipset -y
```



### 开启br_netfilter模块

```shell
需要内核开启br_netfilter模块，方法如下：
# 在/etc/sysctl.conf中添加：
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1 

/sbin/sysctl -p时会报错没用这个文件或目录，解决方法：
modprobe br_netfilter
/sbin/sysctl -p

# 开机模块会消失，解决方法：
#在/etc/新建rc.sysinit 文件
cat /etc/rc.sysinit
#!/bin/bash
for file in /etc/sysconfig/modules/*.modules ; do
[ -x $file ] && $file
done

#在/etc/sysconfig/modules/目录下新建文件如下
vim /etc/sysconfig/modules/br_netfilter.modules
#增加内容：
modprobe br_netfilter
#增加权限
chmod 755 /etc/sysconfig/modules/br_netfilter.modules
#如果出现PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory错误修复方法：

#解决方法：
yum install -y libibverbs
```



### 修改neutron相应配置

```shell
cp /etc/neutron/neutron.conf /etc/neutron/neutron.conf.bak
```



```shell
openstack-config --set /etc/neutron/neutron.conf  DEFAULT core_plugin  ml2
openstack-config --set /etc/neutron/neutron.conf  DEFAULT service_plugins
openstack-config --set /etc/neutron/neutron.conf  DEFAULT transport_url rabbit://openstack:RABBIT_PASS@controller
openstack-config --set /etc/neutron/neutron.conf  DEFAULT auth_strategy  keystone
openstack-config --set /etc/neutron/neutron.conf  DEFAULT notify_nova_on_port_status_changes  True
openstack-config --set /etc/neutron/neutron.conf  DEFAULT notify_nova_on_port_data_changes  True
openstack-config --set /etc/neutron/neutron.conf  database connection  mysql+pymysql://neutron:NEUTRON_DBPASS@controller/neutron
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken auth_uri  http://controller:5000
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken auth_url  http://controller:35357
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken memcached_servers  controller:11211
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken auth_type  password
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken project_domain_name  default
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken user_domain_name  default
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken project_name  service
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken username  neutron
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken password  NEUTRON_PASS
openstack-config --set /etc/neutron/neutron.conf  nova auth_url  http://controller:35357
openstack-config --set /etc/neutron/neutron.conf  nova auth_type  password 
openstack-config --set /etc/neutron/neutron.conf  nova project_domain_name  default
openstack-config --set /etc/neutron/neutron.conf  nova user_domain_name  default
openstack-config --set /etc/neutron/neutron.conf  nova region_name  RegionOne
openstack-config --set /etc/neutron/neutron.conf  nova project_name  service
openstack-config --set /etc/neutron/neutron.conf  nova username  nova
openstack-config --set /etc/neutron/neutron.conf  nova password  NOVA_PASS
openstack-config --set /etc/neutron/neutron.conf  oslo_concurrency lock_path  /var/lib/neutron/tmp
```



```shell
#cat ml2_conf.ini >/etc/neutron/plugins/ml2/ml2_conf.ini 
cp /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugins/ml2/ml2_conf.ini.bak
```



```shell
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini  ml2 type_drivers  flat,vlan
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini  ml2 tenant_network_types 
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini  ml2 mechanism_drivers  linuxbridge
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini  ml2 extension_drivers  port_security
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini  ml2_type_flat flat_networks  provider
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini  securitygroup enable_ipset  True
```



```shell
#cat linuxbridge_agent.ini >/etc/neutron/plugins/ml2/linuxbridge_agent.ini 
cp /etc/neutron/plugins/ml2/linuxbridge_agent.ini /etc/neutron/plugins/ml2/linuxbridge_agent.ini.bak
```

```shell
openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini  linux_bridge physical_interface_mappings  provider:ens33
openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini  securitygroup enable_security_group  True
openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini  securitygroup firewall_driver  neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini  vxlan enable_vxlan  False
```



```shell
#cat dhcp_agent.ini >/etc/neutron/dhcp_agent.ini 
cp /etc/neutron/dhcp_agent.ini /etc/neutron/dhcp_agent.ini.bak

openstack-config --set /etc/neutron/dhcp_agent.ini  DEFAULT interface_driver linuxbridge
openstack-config --set /etc/neutron/dhcp_agent.ini  DEFAULT dhcp_driver neutron.agent.linux.dhcp.Dnsmasq
openstack-config --set /etc/neutron/dhcp_agent.ini  DEFAULT enable_isolated_metadata true
```



```shell
#cat metadata_agent.ini >/etc/neutron/metadata_agent.ini 
cp /etc/neutron/metadata_agent.ini /etc/neutron/metadata_agent.ini.bak


openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT nova_metadata_host  controller
openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT metadata_proxy_shared_secret  METADATA_SECRET
```



### 在nova中增加neutron的相关配置

```shell
openstack-config --set /etc/nova/nova.conf  neutron url  http://controller:9696
openstack-config --set /etc/nova/nova.conf  neutron auth_url  http://controller:35357
openstack-config --set /etc/nova/nova.conf  neutron auth_type  password
openstack-config --set /etc/nova/nova.conf  neutron project_domain_name  default
openstack-config --set /etc/nova/nova.conf  neutron user_domain_name  default
openstack-config --set /etc/nova/nova.conf  neutron region_name  RegionOne
openstack-config --set /etc/nova/nova.conf  neutron project_name  service
openstack-config --set /etc/nova/nova.conf  neutron username  neutron
openstack-config --set /etc/nova/nova.conf  neutron password  NEUTRON_PASS
openstack-config --set /etc/nova/nova.conf  neutron service_metadata_proxy  True
openstack-config --set /etc/nova/nova.conf  neutron metadata_proxy_shared_secret  METADATA_SECRET
```



### 做软连接找到linux_bridge插件

```shell
ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini
```



### 同步数据库

```shell
su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf \
  --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron
```



### 重启nova-api

```shell
systemctl restart openstack-nova-api.service
```



### 启动并开机自启neutron服务

```shell
systemctl enable neutron-server.service \
  neutron-linuxbridge-agent.service neutron-dhcp-agent.service \
  neutron-metadata-agent.service
  
systemctl start neutron-server.service \
  neutron-linuxbridge-agent.service neutron-dhcp-agent.service \
  neutron-metadata-agent.service
```



### 验证

```shell
openstack network agent list
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121332501.png)



### 在计算节点（compute）

**<font color='red'>computer-01</font>**

```shell
yum install openstack-neutron-linuxbridge ebtables ipset -y
```



#### 内核开启br_netfilter模块

```shell
需要内核开启br_netfilter模块，方法如下：
# 在/etc/sysctl.conf中添加：
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1 

/sbin/sysctl -p时会报错没用这个文件或目录，解决方法：
modprobe br_netfilter
/sbin/sysctl -p

# 开机模块会消失，解决方法：
#在/etc/新建rc.sysinit 文件
vim /etc/rc.sysinit
#!/bin/bash
for file in /etc/sysconfig/modules/*.modules ; do
[ -x $file ] && $file
done

#在/etc/sysconfig/modules/目录下新建文件如下

vim /etc/sysconfig/modules/br_netfilter.modules
#增加内容：
modprobe br_netfilter
#增加权限
chmod 755 /etc/sysconfig/modules/br_netfilter.modules
#如果出现PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory错误修复方法：

#解决方法：
yum install -y libibverbs
```



#### 更改neutron配置

```shell
cp /etc/neutron/neutron.conf /etc/neutron/neutron.conf.bak
 
openstack-config --set /etc/neutron/neutron.conf  DEFAULT transport_url rabbit://openstack:RABBIT_PASS@controller
openstack-config --set /etc/neutron/neutron.conf  DEFAULT auth_strategy  keystone
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken auth_uri  http://controller:5000
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken auth_url  http://controller:35357
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken memcached_servers  controller:11211
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken auth_type  password
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken project_domain_name  default
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken user_domain_name  default
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken project_name  service
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken username  neutron
openstack-config --set /etc/neutron/neutron.conf  keystone_authtoken password  NEUTRON_PASS
openstack-config --set /etc/neutron/neutron.conf  oslo_concurrency lock_path  /var/lib/neutron/tmp
```



```shell
#cat linuxbridge_agent.ini >/etc/neutron/plugins/ml2/linuxbridge_agent.ini 
cp /etc/neutron/plugins/ml2/linuxbridge_agent.ini /etc/neutron/plugins/ml2/linuxbridge_agent.ini.bak
 
openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini  linux_bridge physical_interface_mappings  provider:ens33
openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini  securitygroup enable_security_group  True
openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini  securitygroup firewall_driver  neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
openstack-config --set /etc/neutron/plugins/ml2/linuxbridge_agent.ini  vxlan enable_vxlan  False
```



#### 增加nova配置

```shell
openstack-config --set /etc/nova/nova.conf  neutron url  http://controller:9696
openstack-config --set /etc/nova/nova.conf  neutron auth_url  http://controller:35357
openstack-config --set /etc/nova/nova.conf  neutron auth_type  password
openstack-config --set /etc/nova/nova.conf  neutron project_domain_name  default
openstack-config --set /etc/nova/nova.conf  neutron user_domain_name  default
openstack-config --set /etc/nova/nova.conf  neutron region_name  RegionOne
openstack-config --set /etc/nova/nova.conf  neutron project_name  service
openstack-config --set /etc/nova/nova.conf  neutron username  neutron
openstack-config --set /etc/nova/nova.conf  neutron password  NEUTRON_PASS
```



#### 重启compute的nova服务

```shell
systemctl restart openstack-nova-compute.service
```



#### 启动并开机自启neutron服务

```shell
systemctl enable neutron-linuxbridge-agent.service
systemctl start neutron-linuxbridge-agent.service
```



#### 验证网络服务（在controller节点验证）

```shell
openstack network agent list
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121338617.png)





## 安装web页面

**<font color='red'>安装在controller节点</font>：**
**此处有更改好的模板，直接将模板导入安装后的配置文件中。**

```shell
yum install openstack-dashboard -y
```



### 上传模板进行替换

```shell
cat local_settings >/etc/openstack-dashboard/local_settings #记录好模板
sed -i '3a WSGIApplicationGroup %{GLOBAL}' /etc/httpd/conf.d/openstack-dashboard.conf #修改不能访问的bug
```



### 重启apache和memcached

```shell
systemctl restart httpd.service memcached
```



### 访问

```shell
http://ip/dashboard
```

```shell
域 default 账户：admin  密码：ADMIN_PASS   # env | grep OS可以使用这条命令查看
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121342308.png)



![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121343808.png)

```shell
# web界面里面的可用域和主机的域不是一样的，可用域是指可用的空间(比如说可以在哪里创建这个东西) ; 主机的域(比如:default域)指的是区域
```

![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121520151.png)



![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121524787.png)



![](https://figure-bed-1304788733.cos.ap-guangzhou.myqcloud.com/typora/202206121524482.png)



## 命令行创建模板

### 创建网络

**<font color='red'>注意，此处会提示neutron的cli会被移除，这个命令还可以用</font>**

```shell
# provider与前面配置文件写的对应
neutron net-create --shared --provider:physical_network provider --provider:network_type flat class2 

# 7435a72c-d398-4522-926e-b0b3458d6aa1
```

![image-20221129142326855](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291423955.png)



![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415441.png)

**<font color='red'>注意：下面ip替换成宿主机ip网段，用于平面网络的ip分配</font>**

```shell
neutron subnet-create --name subnet-class2 --allocation-pool \
start=192.168.188.100,end=192.168.188.200 --dns-nameserver 223.5.5.5 \
--gateway 192.168.188.2 class2 192.168.188.0/24
```

![image-20221129143845193](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291438285.png)



### 生成硬件配置模板

**也可以在web页面配置生产**

```shell
openstack flavor create --id 0 --vcpus 1 --ram 64 --disk 1 m1.nano
```

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415493.png)

### 生成sshkey，并且作为controller节点链接虚拟机的默认注入密钥

```
ssh-keygen -q -N "" -f ~/.ssh/id_rsa
openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey
```



### 生成默认安全组（安全组名字default）

```shell
openstack security group rule create --proto icmp default
openstack security group rule create --proto tcp --dst-port 22 default
```

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415448.png)

查看网络ID

![image-20221129150400536](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291504594.png)

```
[root@controller data]# openstack server create --flavor m1.nano --image cirros \
> --nic net-id=153a9ad0-953e-4274-bac7-0203927efbdf --security-group default \
> --key-name mykey provider-instance
```

![image-20221129150502268](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291505343.png)



### web页面使用VNC连接使用

![image-20221129151235892](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291512973.png)



![image-20221129152958760](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291529870.png)



需要在宿主机的本地DNS文件中添加controller

![image-20221129154042224](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291540282.png)





## 增加一个计算节点

### 1、配置yum源

### 2、时间同步

### 3、安装openstack客户端和openstack-selinux

```shell
yum install openstack-utils -y
yum install python-openstackclient.noarch openstack-selinux.noarch -y
```

### 4、修改base源

```shell
[root@computer-02 ~]# vim /etc/yum.repos.d/CentOS-Base.repo   # 添加以下内容
[Virt]
name=CentOS-$releasever - Base
baseurl=http://mirrors.aliyun.com/centos/7/virt/x86_64/kvm-common/
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

[root@computer-02 ~]# yum makecache
```

### 5、安装nova-compute

```shell
yum install openstack-nova-compute openstack-utils.noarch -y
yum install openstack-utils.noarch -y
```

```shell
[root@computer-02 ~]# cp /etc/nova/nova.conf /etc/nova/nova.conf.bak
```

```shell
# 回到computer-01上操作
[root@computer-01 ~]# cd /etc/nova/
[root@computer-01 nova]# scp -r nova.conf computer-02:`pwd`
```

```shell
# 回到compute-27上操作
[root@computer-02 ~]# cd /etc/nova/
[root@computer-02 nova]# vim nova.conf
改: my_ip = 192.168.245.222
为: my_ip = 192.168.245.223
```

```shell
# 开机自启
systemctl enable libvirtd.service openstack-nova-compute.service
systemctl start libvirtd.service openstack-nova-compute.service
```

**<font color='red'>注意：</font>**
**<font color='red'>回到controller节点</font>**

```shell
# 验证：
openstack compute service list --service nova-compute
```

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415393.png)

```shell
# 创建compute节点网格
su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
```

```shell
openstack compute service list
```

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415008.png)



### 6、安装neutron

```shell
[root@computer-02 nova]# yum install openstack-neutron-linuxbridge ebtables ipset -y
```

```shell
需要内核开启br_netfilter模块，方法如下：
# 在/etc/sysctl.conf中添加：
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1 

/sbin/sysctl -p时会报错没用这个文件或目录，解决方法：
modprobe br_netfilter
/sbin/sysctl -p

# 开机模块会消失，解决方法：
#在/etc/新建rc.sysinit 文件
cat /etc/rc.sysinit
#!/bin/bash
for file in /etc/sysconfig/modules/*.modules ; do
[ -x $file ] && $file
done

#在/etc/sysconfig/modules/目录下新建文件如下
cat /etc/sysconfig/modules/br_netfilter.modules
modprobe br_netfilter
vim /etc/sysconfig/modules/br_netfilter.modules
#增加内容：
modprobe br_netfilter
#增加权限
chmod 755 /etc/sysconfig/modules/br_netfilter.modules
#如果出现PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory错误修复方法：

#解决方法：
yum install -y libibverbs
```

```shell
[root@computer-02 nova]# cp /etc/neutron/neutron.conf /etc/neutron/neutron.conf.bak
```

```shell
# 回到computer-01上操作
[root@computer-01 nova]# cd ../neutron/
[root@computer-01 neutron]# scp -r neutron.conf computer-02:`pwd`
```

```shell
# 回到compute-27上操作
[root@computer-02 nova]# cp /etc/neutron/plugins/ml2/linuxbridge_agent.ini /etc/neutron/plugins/ml2/linuxbridge_agent.ini.bak
```

```shell
# 回到computer-01上操作
[root@computer-01 neutron]# cd plugins/ml2/
[root@computer-01 ml2]# scp linuxbridge_agent.ini computer-02:`pwd`
```

```shell
# 回到compute-27上操作
# 重启compute的nova服务
systemctl restart openstack-nova-compute.service

# 启动并开机自启neutron服务
systemctl enable neutron-linuxbridge-agent.service
systemctl start neutron-linuxbridge-agent.service
```

```shell
# 验证网络服务（在controller节点验证）
[root@controller ~]# openstack network agent list
```

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415003.png)





## glance镜像服务迁移

```shell
1，停掉控制节点的glance服务
停掉glance-api glance-registry

2，在先节点上安装数据库，python2-PyMySQL
启动数据库并且安全初始化
创建glance用户以及密码

3，恢复glance数据库数据，mysqldump -B glance > glance.sql
把生成的sql文件放到新的节点
在新的计算节点：
mysql导入：mysql < "glance.sql“


4，安装配置glance服务：
yum install openstack-glance -y 
配置glance
拉取旧配置然后更改：
拉取glance两个文件，glance-api，glance-registry配置
更改链接数据库的信息
更改数据库controller为本机地址
启动服务

5，迁移glance镜像
scp -r /var/lib/glance/images/
注意镜像权限

6，更改keystone的注册信息（更改endpoint的信息） 注意备份（在控制节点操作）
mysqldump keystone endpoint > endpoint.sql
cp endpoint.sql /data/bak
vim endpoint.sql
%s#http://controller:9292#http://ip:9292#gc（c参数是有交互的检查）

use keystone
source endpoint.sql

检测：
openstack endpoint list | grep image
openstack image list

7,此时启动实例报错，更改nova配置(包括控制节点，计算节点)
sed -i 's#http://controller:9292#http://192.168.245.223:9292#g' /etc/nova/nova.conf (可用ansilbe批量更改)
重启控制节点：
systemctl restart openstack-nova-api (控制节点)
systemctl restart openstack-nova-compute (计算节点)
systemctl restart httpd memcached
```



## 增加一个flat网段

```shell
准备工作：
1，增加一块网卡。生成配置文件，更改名字和ip   不能重启网卡，
用命令启动： ifup 网卡名字 启动网卡           （同样在计算节点也做）

2，vim /etc/neutron/plugins/ml2/ml2_conf.ini  (controller-node)

flat_networks = provider,net172_16

3，vim /etc/neutron/plugins/ml2/linuxbridge_agent.ini

physical_interface_mappings = provider:ens33,net172_16:新网卡名字

4，重启服务：
systemctl restart neutron-server.service neutron-linuxbridge-agent.service

5，计算节点：
仅仅更改linuxbridge_agent.ini配置
vim /etc/neutron/plugins/ml2/linuxbridge_agent.ini

physical_interface_mappings = provider:ens33,net172_16:新网卡名字

6，重启服务：
systemctl restart neutron-linuxbridge-agent.service


7，创建网络：
neutron net-create --shared --provider:physical_network net172_16 --provider:network_type flat class2172

neutron subnet-create --name subnet-172class2 --allocation-pool \
start=172.16.0.50,end=172.16.0.200 --dns-nameserver 223.5.5.5 \
--gateway 172.16.0.2 class2172 172.16.0.0/24
```



## cinder卷存储

**此处选择扩容后的computer-02为存储节点，且增加了两块硬盘作为存储测试**

**computer-02兼职存储卷节点ip为192.168.245.223（生产建议存储卷节点单独部署）**

**命名sata的硬盘20G，ssd的30G**



### 增加两块硬盘

**<font color='red'>在computer-02节点操作</font>**

```shell
[root@computer-02 ~]# echo '- - -' >/sys/class/scsi_host/host0/scan   # 扫描识别硬盘
```



### 数据库创库授权

**<font color='red'>在controller节点操作</font>**

```shell
CREATE DATABASE cinder;
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' \
  IDENTIFIED BY 'CINDER_DBPASS';
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%' \
  IDENTIFIED BY 'CINDER_DBPASS';
```



### 在keystone创建系统用户(glance,nova,neutron,cinder)关联角色

**<font color='red'>在controller节点操作</font>**

```shell
openstack user create --domain default --password CINDER_PASS cinder
openstack role add --project service --user cinder admin
```



### 在keystone上创建服务和注册api

**<font color='red'>在controller节点操作</font>**

**为了兼容性使用了两种协议**

```shell
openstack service create --name cinderv2 \
  --description "OpenStack Block Storage" volumev2
openstack service create --name cinderv3 \
  --description "OpenStack Block Storage" volumev3
openstack endpoint create --region RegionOne \
  volumev2 public http://controller:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne \
  volumev2 internal http://controller:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne \
  volumev2 admin http://controller:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne \
  volumev3 public http://controller:8776/v3/%\(project_id\)s
openstack endpoint create --region RegionOne \
  volumev3 internal http://controller:8776/v3/%\(project_id\)s
openstack endpoint create --region RegionOne \
  volumev3 admin http://controller:8776/v3/%\(project_id\)s
```



### 装包配置

**<font color='red'>在controller节点操作</font>**

```shell
[root@controller ~]# yum install openstack-cinder -y
```

```shell
[root@controller ~]# cd /etc/cinder/
[root@controller cinder]# cp cinder.conf cinder.conf.bak
[root@controller cinder]# grep -Ev "^$|#" cinder.conf.bak > cinder.conf
```

**<font color='red'>请根据业务修改对应的IP地址</font>**

```shell
openstack-config --set /etc/cinder/cinder.conf   DEFAULT  transport_url rabbit://openstack:RABBIT_PASS@controller
openstack-config --set /etc/cinder/cinder.conf   DEFAULT  auth_strategy  keystone
openstack-config --set /etc/cinder/cinder.conf   DEFAULT  my_ip  192.168.245.221   # 注意IP地址
openstack-config --set /etc/cinder/cinder.conf   database connection mysql+pymysql://cinder:CINDER_DBPASS@controller/cinder
openstack-config --set /etc/cinder/cinder.conf   keystone_authtoken   auth_uri  http://controller:5000
openstack-config --set /etc/cinder/cinder.conf   keystone_authtoken   auth_url  http://controller:35357
openstack-config --set /etc/cinder/cinder.conf   keystone_authtoken   memcached_servers  controller:11211
openstack-config --set /etc/cinder/cinder.conf   keystone_authtoken   auth_type  password
openstack-config --set /etc/cinder/cinder.conf   keystone_authtoken   project_domain_name  default
openstack-config --set /etc/cinder/cinder.conf   keystone_authtoken   user_domain_name  default
openstack-config --set /etc/cinder/cinder.conf   keystone_authtoken   project_name  service
openstack-config --set /etc/cinder/cinder.conf   keystone_authtoken   username  cinder
openstack-config --set /etc/cinder/cinder.conf   keystone_authtoken   password  CINDER_PASS
openstack-config --set /etc/cinder/cinder.conf   oslo_concurrency  lock_path  /var/lib/cinder/tmp
openstack-config --set /etc/cinder/cinder.conf   DEFAULT  glance_api_servers  http://controller:9292
```



### 同步数据库

```shell
su -s /bin/sh -c "cinder-manage db sync" cinder
```



### 启动服务

```shell
openstack-config --set /etc/nova/nova.conf cinder os_region_name RegionOne
systemctl restart openstack-nova-api.service
systemctl enable openstack-cinder-api.service openstack-cinder-scheduler.service
systemctl start openstack-cinder-api.service openstack-cinder-scheduler.service
```



### 验证

```shell
openstack volume service list
```

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415967.png)



#### 使用computer-02节点兼职cinder存储节点（此处为块存储演示）

**安装lvm**

```shell
[root@computer-02 ~]# yum install lvm2 device-mapper-persistent-data -y

[root@computer-02 ~]# systemctl enable lvm2-lvmetad.service
[root@computer-02 ~]# systemctl start lvm2-lvmetad.service
```

**创建pv、vg**

```shell
pvcreate /dev/sdb  # 创建pv
pvcreate /dev/sdc
vgcreate cinder-sats /dev/sdb  # 创建vg
vgcreate cinder-ssd /dev/sdc
```

**lvm的优化**

```shell
# 修改/etc/lvm/lvm.conf
# 在130下面插入一行：
filter = [ "a/sdb/", "a/sdc/","r/.*/"]
```



```shell
yum install openstack-cinder targetcli python-keystone -y
```



```shell
[root@computer-02 ~]# cd /etc/cinder/
[root@computer-02 cinder]# cp cinder.conf cinder.conf.bak
[root@computer-02 cinder]# vim cinder.conf
```

**配置**


```
[DEFAULT]
transport_url = rabbit://openstack:RABBIT_PASS@controller
auth_strategy = keystone
my_ip = 192.168.245.223
glance_api_servers = http://controller:9292
enabled_backends = ssd,sata
[BACKEND]
[BRCD_FABRIC_EXAMPLE]
[CISCO_FABRIC_EXAMPLE]
[COORDINATION]
[FC-ZONE-MANAGER]
[KEYMGR]
[cors]
[cors.subdomain]
[database]
connection = mysql+pymysql://cinder:CINDER_DBPASS@controller/cinder
[keystone_authtoken]
auth_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = cinder
password = CINDER_PASS
[matchmaker_redis]
[oslo_concurrency]
lock_path = /var/lib/cinder/tmp
[oslo_messaging_amqp]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_middleware]
[oslo_policy]
[oslo_reports]
[oslo_versionedobjects]
[ssl]
[ssd]
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group = cinder-ssd
iscsi_protocol = iscsi
iscsi_helper = lioadm
volume_backend_name = ssd
[sata]
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_group = cinder-sata
iscsi_protocol = iscsi
iscsi_helper = lioadm
volume_backend_name = sata
```

**启动**

```shell
systemctl enable openstack-cinder-volume.service target.service
systemctl start openstack-cinder-volume.service target.service
```

**验证**

```shell
# 在controller节点上操作
openstack volume service list
```

此时重新登录web界面会多出一个卷的选项即cinder卷服务

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415977.png)



![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415147.png)



#### 创建卷类型

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415224.png)



![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415590.png)



![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415677.png)

**最终创建好**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415642.png)



**此时选择创建卷会有卷类型可选**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415740.png)



#### 块存储对接nfs(在存储节点)

**<font color='red'>nfs不能与cinder一个节点</font>**

**安装nfs**

这里我选择自写的脚本安装配置

```shell
[root@nfs ~]# sh nfs.sh

[root@nfs ~]# cat /etc/exports
/data 192.168.188.0/24(rw,no_root_squash)
```



**修改cinder配置文件**

```shell
vim /etc/cinder/cinder.conf
```

```shell
[DEFAULT]
transport_url = rabbit://openstack:RABBIT_PASS@controller
auth_strategy = keystone
my_ip = 192.168.245.223
glance_api_servers = http://controller:9292
enabled_backends = ssd,sata,nfs                   # 这里添加nfs驱动
……
[nfs]                                             # 添加关于nfs驱动的配置
volume_driver = cinder.volume.drivers.nfs.NfsDriver
nfs_shares_config = /etc/cinder/nfs_shares
volume_backend_name = nfs
```



```shell
vim /etc/cinder/nfs_shares
# 添加
192.168.188.17:/data
```

**重启**

```shell
[root@computer-02 cinder]# systemctl restart openstack-cinder-volume.service
```

**验证**

```shell
# controller节点操作
[root@controller cinder]# openstack volume service list
```

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415687.png)

**在web页面创建卷类型，把nfs添加进去**





## 实例冷迁移

```shell
# 要处于同一主机聚合(管理员 ==> 计算 ==> 主机聚合)
# nova节点之间免密钥互信
usermod -s /bin/bash nova
su 进nova用户
# 生成密钥 ssh-keygen -t rsa -q -N ''
# 免密登录自己：
# 在.ssh目录下
cp -fa id_rsa.pub authorized_keys
# 然后用nova用户ssh登录下本机
# 然后两台compute可以用nava账户互相登录
scp -rp .ssh root@ip:`pwd`
# 将公钥发送到其他计算节点的/var/lib/nova/.ssh 注意权限和权限组(都是nova)
# 修改控制节点nova.conf
vi /etc/nova/nova.conf   # 添加内容进DEFAULT

[DEFAULT]
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter

# 重启openstack-nova-scheduler
systemctl restart openstack-nova-scheduler.service
# 修改所有计算节点

vi /etc/nova/nova.conf

[DEFAULT]
allow_resize_to_same_host = True

# 重启openstack-nova-compute

systemctl restart openstack-nova-compute.service
# ui界面迁移，另外如果配置后加，之前生成的实例不能被迁移
```



## openstack vxlan三层网络

**可用IP数量**

- **vlan:1-4094**
- **vxlan:4096*4096-2**



**配置说明：所有节点都开一个新网卡(LAN网卡)，设置网段为172.16.1.0/24的网段。**
**配置静态ip注意设置netmask为255.255.255.0**
**对应controller节点为：172.16.1.25**
**computer-01节点为：172.16.1.26**
**computer-02节点为：172.16.1.27**

**如下图**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415844.png)



### 识别新添加的网卡

```shell
ip addres
```

**在controller节点操作**

```shell
# 修改网卡配置文件
# 从上面得出网卡名称是ens37

[root@controller ~]# cp /etc/sysconfig/network-scripts/ifcfg-ens33 /etc/sysconfig/network-scripts/ifcfg-ens37
[root@controller ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens37
```

```shell
TYPE="Ethernet"
BOOTPROTO="static"
DEFROUTE="yes"
NAME="ens37"
DEVICE="ens37"
ONBOOT="yes"
IPADDR=172.16.1.25
```

```shell
# 分发进行修改对应IP
[root@controller ~]# scp -r /etc/sysconfig/network-scripts/ifcfg-ens37 computer-01:/etc/sysconfig/network-scripts/

[root@controller ~]# scp -r /etc/sysconfig/network-scripts/ifcfg-ens37 computer-02:/etc/sysconfig/network-scripts/


# 26的机子
vim /etc/sysconfig/network-scripts/ifcfg-ens37
改:IPADDR=172.16.1.25
为:IPADDR=172.16.1.26

# 27的机子
vim /etc/sysconfig/network-scripts/ifcfg-ens37
改:IPADDR=172.16.1.25
为:IPADDR=172.16.1.27
```

```shell
# 挂在网卡验证网络
ifup ens37

# 互ping
```



### 在控制节点（controller）

1，删除所有平面网络的实例
2，修改/etc/neutron/neutron.conf的[DEFAULT]区域

```shell
core_plugin = ml2
service_plugins = router
allow_overlapping_ips = True
```



```shell
[root@controller ~]# vim /etc/neutron/neutron.conf
# 改: service_plugins = 
# 为: service_plugins = router

# 添加一项在service_plugins下面
allow_overlapping_ips = True
```



3，修改/etc/neutron/plugins/ml2/ml2_conf.ini文件

```shell
[root@controller ~]# vim /etc/neutron/plugins/ml2/ml2_conf.ini   
[ml2]区域修改如下
type_drivers = flat,vlan,vxlan
tenant_network_types = vxlan
mechanism_drivers = linuxbridge,l2population
在[ml2_type_vxlan]区域增加一行
vni_ranges = 1:1000
最终的配置文件如下
[root@controller ~]# grep -Ev "^$|#" /etc/neutron/plugins/ml2/ml2_conf.ini
[DEFAULT]
[l2pop]
[ml2]
type_drivers = flat,vlan,vxlan
tenant_network_types = vxlan
mechanism_drivers = linuxbridge,l2population
extension_drivers = port_security
[ml2_type_flat]
flat_networks = provider
[ml2_type_geneve]
[ml2_type_gre]
[ml2_type_vlan]
[ml2_type_vxlan]
vni_ranges = 1:1000
[securitygroup]
enable_ipset = True
```



4，修改/etc/neutron/plugins/ml2/linuxbridge_agent.ini文件

```shell
[root@controller ~]# vim /etc/neutron/plugins/ml2/linuxbridge_agent.ini
在[vxlan]区域下
修改为
enable_vxlan = True
local_ip = 172.16.1.25
l2_population = True

#172.16.1.15这个IP还没有，马上就配，主机规划的第二块网卡，就是现在用的
最终的配置文件如下
[root@controller ~]# grep -Ev "^$|#" /etc/neutron/plugins/ml2/linuxbridge_agent.ini
[DEFAULT]
[agent]
[linux_bridge]
physical_interface_mappings = provider:ens33
[network_log]
[securitygroup]
enable_security_group = True
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[vxlan]
enable_vxlan = True
local_ip = 172.16.1.25
l2_population = True
```

```shell
# 此时使用ifconfig命令查看私网网卡发现子网掩码是255.255.0.0
# 私网的子网掩码有严格要求一定是255.255.255.0

# 修改网卡配置文件
[root@controller ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens38
添加内容如下
NETMASK=255.255.255.0

# 千万不要重启网卡!!!
# 使用以下方法来
[root@controller ~]# ifdown ens37
[root@controller ~]# ifup ens37
[root@controller ~]# ifconfig   # 验证

# 其余节点的机子皆是如此操作
```



5，修改/etc/neutron/l3_agent.ini文件

```shell
[root@controller ~]# cp /etc/neutron/l3_agent.ini /etc/neutron/l3_agent.ini.bak
[root@controller ~]# grep -Ev "^$|#" /etc/neutron/l3_agent.ini.bak > /etc/neutron/l3_agent.ini
[root@controller ~]# vim /etc/neutron/l3_agent.ini
#在[DEFAULT]区域下，增加
interface_driver = linuxbridge


#重启neutron服务
systemctl restart neutron-server.service  neutron-linuxbridge-agent.service neutron-dhcp-agent.service neutron-metadata-agent.service

#启动L3服务:
systemctl start neutron-l3-agent.service

#开机启动L3服务:
systemctl enable neutron-l3-agent.service
```



6，验证

```shell
openstack network agent list
```

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415174.png)





### 在计算节点(compute)

```shell
修改/etc/neutron/plugins/ml2/linuxbridge_agent.ini文件
[root@computer-01 ~]# vim /etc/neutron/plugins/ml2/linuxbridge_agent.ini
在[vxlan]区域下
enable_vxlan = True
local_ip = 172.16.1.26
l2_population = True

最终的配置文件如下：
[root@computer-01 ml2]# grep -Ev "^$|#" /etc/neutron/plugins/ml2/linuxbridge_agent.ini
[DEFAULT]
[agent]
[linux_bridge]
physical_interface_mappings = provider:ens33
[network_log]
[securitygroup]
enable_security_group = True
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[vxlan]
enable_vxlan = True
local_ip = 172.16.1.26
l2_population = True

# 启动
# 重启agent服务
systemctl restart neutron-linuxbridge-agent.service

# 备注：computer-02节点（即192.168.245.223）重复计算节点computer-01的步骤即可。
```



### 在web界面

**创建vpc网络类型**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415621.png)





```
回到控制节点打开web界面的路由配置：
vim /etc/openstack-dashboard/local_settings
将263行的
'enable_router': False,
修改为
'enable_router': True,
重启httpd与memcache
systemctl restart httpd.service memcached.service
在dashboard上开启三层路由
只有修改了/etc/openstack-dashboard/local_settings，才能开启三层路由器
图形化操作步骤：
1，在管理员网络处增加vpc网络将原来的class2网设置成外部网络，vpc自建ip与子网
2，在web界面路由处创建路由，将路由与class2网段接起来。
3，到网络拓扑界面，在路由的图标上设置接口与vpc链接。
4，在实例界面生成实例，网络选择vpc
5，此时实例可以出外网，但是外网链接不进去。需要配置浮动ip
6，如果在实例界面分配不了浮动ip可以到管理员的网络界面先生成浮动ip再关联。
```



**连续创建两个vxlan网络类型的实例**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415331.png)

**查看网络拓扑图**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415371.png)



#### **开启外网IP**

**把class2变成外部网络**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415366.png)

**此时再回到网络拓扑图，发现发生了变化**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415403.png)

**开启路由**

```shell
[root@controller ~]# vim /etc/openstack-dashboard/local_settings
将263行的
'enable_router': False,
修改为
'enable_router': True,
```

```shell
[root@controller ~]# systemctl restart httpd.service memcached.service
```

**回到web页面，发现多了几个选项**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415695.png)

**新建路由**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415885.png)

**添加接口**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415948.png)



![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415936.png)



![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415058.png)



**现在是能出去，但是进不来**

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415231.png)



#### 创建弹性IP

![](https://note-1308251438.cos.ap-guangzhou.myqcloud.com/typora/202211291415406.png)