如何使用 Mgmt Driver 部署 Kubernetes 集群 (Kubespray)¶
概述¶
1. Mgmt Driver 介绍¶
Mgmt Driver 允许用户在其 VNF 生命周期管理操作之前和/或之后配置其 VNF。用户可以通过实现自己的 Mgmt Driver 来自定义 Mgmt Driver 的逻辑，这些自定义由 NFV-SOL001 v2.6.1 中的“接口”定义指定。本用户指南旨在通过用户自定义的 Mgmt Driver 部署 Kubernetes 集群。
2. 用例¶
在本用户指南中，Kubernetes 集群由 Kubespray（使用 ansible 部署 Kubernetes 集群的工具）部署，因此您需要提前设置一台服务器来运行 Kubespray。负载均衡器 VM 将与 Kubernetes 集群的节点一起实例化。您可以通过负载均衡器访问 Kubernetes 集群。Mgmt Driver 可以使用 Kubespray 部署 Kubernetes 集群，安装和配置负载均衡器。它仅支持一个用例，即示例 Mgmt Driver。
简单：部署一个 master 节点和 worker 节点。在这种情况下，它支持扩展 worker 节点和修复 worker 节点。
简单：单个 Master 节点¶
简单的 Kubernetes 集群包含一个 master 节点作为控制器节点和一个负载均衡器 VM 用于外部访问。您可以使用我们提供的示例脚本部署它。下图显示了简单的 Kubernetes 集群架构
+--------------+ +--------------+
| user traffic | | Kubespray |
| | | (Pre-install)|
+------+-------+ +------+-------+
| |
|To access pod |Install k8s cluster
|Load Balancer SSH IP:NodePort |
| |
| |
+-------------------+ |
| Load Balancer | |
| | |
| kubectl | |
+------+------------+ |
| |
+---|----------------------+-------------------------+
| | | |
| |----------------------|---+---------------------|---+
| |MasterNodeIP | |WorkerNodeIP1 | |WorkerNodeIP2
| |:6443 | |:NodePort1 | |:NodePort2
v v v v v v
+-------------------+ +----------------------+ +----------------------+
| MasterNode | | WorkerNode1 | | WorkerNode2 |
| | | | | | | | |
| | | | | | | | |
| v | | | | | | |
| +---------+ | | | | | | |
| | k8s-api | | | | | | | |
| | | | | | | | | |
| +---------+ | | | | | | |
| | | v | | v |
| +-------------------------------------------------------------+ |
| | Service | |
| | | |
| +----------------------------+-------------------------+------+ |
| | | | | | | |
| | | |CLUSTER-IP1| | |CLUSTER-IP2|
| | | |:Port1 | | |:Port2 |
| | | v | | v |
| +---------+ | | +---------+ | | +---------+ |
| | etcd | | | | pod | | | | pod | |
| | | | | | | | | | | |
| +---------+ | | +---------+ | | +---------+ |
+-------------------+ +----------------------+ +----------------------+
准备工作¶
部署 Kubernetes 集群与 Kubespray 一起需要 OpenStack、ubuntu 镜像和 Mgmt Driver 的准备。您可以参考 准备 在 如何使用 Mgmt Driver 部署 Kubernetes 集群 中，了解 OpenStack 的路由器配置、ubuntu 镜像以及 Mgmt Driver 文件的使用方法。
1. 配置 Kubespray¶
对于本用户指南，您应该首先安装 Kubespray。此 Mgmt Driver 支持的 Kubespray 版本是 2.16.0。您可以参考 Kubespray 的官方文档 安装 Kubespray。
安装 Kubespray 后，您应该修改 /etc/ansible/ansible.cfg 文件，以避免在使用 Mgmt Driver 部署 Kubernetes 集群时出现 host_key_checking 问题。
$ sudo vi /etc/ansible/ansible.cfg
...
# uncomment this to disable SSH key host checking
host_key_checking = False
...
2. 上传 ubuntu 镜像¶
由于 master 节点、worker 节点和负载均衡器使用的镜像都是 ubuntu 20.04，为了节省资源，我们建议您提前在 openstack 上创建一个镜像，并将其名称设置为 VNFD 文件中指定的名称。
以下是创建镜像的示例 CLI 命令。
$ openstack image create --file ubuntu-20.04-server-cloudimg-amd64.img \
--container-format bare --disk-format qcow2 ubuntu-20.04-server-cloudimg-amd64
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| container_format | bare |
| created_at | 2021-09-03T02:26:46Z |
| disk_format | qcow2 |
| file | /v2/images/a99e9430-af8f-408d-acb2-9b0140a75974/file |
| id | a99e9430-af8f-408d-acb2-9b0140a75974 |
| min_disk | 0 |
| min_ram | 0 |
| name | ubuntu-20.04-server-cloudimg-amd64 |
| owner | 7e757a0cfea940dab100216036212a65 |
| properties | os_hidden='False', owner_specified.openstack.md5='', owner_specified.openstack.object='images/ubuntu-20.04-server-cloudimg-amd64-2', owner_specified.openstack.sha256='' |
| protected | False |
| schema | /v2/schemas/image |
| status | queued |
| tags | |
| updated_at | 2021-09-03T02:26:46Z |
| visibility | shared |
+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
创建并上传 VNF 包¶
您可以参考章节 创建和上传 VNF 包 在 如何使用 Mgmt Driver 部署 Kubernetes 集群 中，了解 VNF 包的介绍和使用方法。
1. 目录结构¶
VNF 包的示例结构如下所示。
注意
您也可以在 tacker 的 samples/mgmt_driver/kubernetes/kubespray/kubespray_vnf_package/ 目录中找到它们。
目录结构
TOSCA-Metadata/TOSCA.meta
Definitions/
Scripts/
BaseHOT/
UserData/
!----TOSCA-Metadata
!---- TOSCA.meta
!----Definitions
!---- etsi_nfv_sol001_common_types.yaml
!---- etsi_nfv_sol001_vnfd_types.yaml
!---- sample_kubernetes_top.vnfd.yaml
!---- sample_kubernetes_types.yaml
!---- sample_kubernetes_df_simple.yaml
!----Scripts
!---- install_external_lb.sh
!---- kubespray_mgmt.py
!----BaseHOT
!---- simple
!---- nested
!---- base_hot_nested_master.yaml
!---- base_hot_nested_worker.yaml
!---- base_hot_top.yaml
!----UserData
!---- __init__.py
!---- lcm_user_data.py
部署 Kubernetes 集群¶
单个 Master 节点¶
可以使用 Kubespray 安装单个 master Kubernetes 集群，并在“instantiate_end”操作中进行设置，允许您在实例化后执行任何脚本，并启用 Mgmt Driver 支持。使用 Kubespray 部署 Kubernetes 集群仅支持一个 master 节点和多个 worker 节点。
1. 创建参数文件¶
创建一个名为 simple_kubernetes_param_file.json 的文件，格式如下。此文件定义了将在实例化请求的主体中设置的参数。
其他参数与 如何使用 Mgmt Driver 部署 Kubernetes 集群 中的配置相同，您可以参考 创建参数文件 获取详细配置。本用户指南主要修改了 k8s_cluster_installation_param 以适应 Kubespray。
部署 Kubernetes 集群的参数说明
要部署 Kubernetes 集群，您必须在 additionalParams 中设置 k8s_cluster_installation_param 键。KeyValuePairs 如下表所示
参数 |
数据类型 |
description |
必需/可选 |
|---|---|---|---|
vim_name |
字符串 |
由 tacker 注册的已部署 Kubernetes 集群的 vim 名称 |
可选 |
master_node |
dict |
master 节点组的 VM 信息 |
必需 |
worker_node |
dict |
master 节点组的 VM 信息 |
必需 |
proxy |
dict |
VM 上代理设置的信息 |
可选 |
ansible |
dict |
指定 Ansible 相关配置，例如 IP 地址 ( |
必需 |
external_lb_param |
dict |
安装外部负载均衡器的属性 |
必需 |
参数 |
数据类型 |
description |
必需/可选 |
|---|---|---|---|
aspect_id |
字符串 |
master 节点组的资源名称，与 vnfd 中的 aspect 相同。如果使用用户数据，则必须设置 |
可选 |
ssh_cp_name |
字符串 |
与 master 节点的 ssh ip 对应的端口的资源名称 |
必需 |
nic_cp_name |
字符串 |
与 master 节点的 nic ip 对应的端口的资源名称（用于部署 Kubernetes 集群）。如果 ssh ip 是浮动 ip，则需要此项 |
可选 |
username |
字符串 |
用于访问 VM 的用户名 |
必需 |
password |
字符串 |
用于访问 VM 的密码 |
必需 |
pod_cidr |
字符串 |
pod 的 CIDR |
可选 |
cluster_cidr |
字符串 |
服务的 CIDR |
可选 |
参数 |
数据类型 |
description |
必需/可选 |
|---|---|---|---|
aspect_id |
字符串 |
worker 节点组的资源名称，与 vnfd 中的 aspect 相同。如果使用用户数据，则必须设置 |
可选 |
ssh_cp_name |
字符串 |
与 worker 节点的 ssh ip 对应的端口的资源名称 |
必需 |
nic_cp_name |
字符串 |
与 worker 节点的 nic ip 对应的端口的资源名称（用于部署 Kubernetes 集群）。如果 ssh ip 是浮动 ip，则需要此项 |
可选 |
username |
字符串 |
用于访问 VM 的用户名 |
必需 |
password |
字符串 |
用于访问 VM 的密码 |
必需 |
参数 |
数据类型 |
description |
必需/可选 |
|---|---|---|---|
http_proxy |
字符串 |
Http 代理服务器地址 |
可选 |
https_proxy |
字符串 |
Https 代理服务器地址 |
可选 |
参数 |
数据类型 |
description |
必需/可选 |
|---|---|---|---|
ip_address |
字符串 |
Ansible 服务器的 IP 地址 |
必需 |
username |
字符串 |
Ansible 服务器的用户名 |
必需 |
password |
字符串 |
Ansible 服务器的密码 |
必需 |
kubespray_root_path |
字符串 |
kubespray 的根目录 |
必需 |
transferring_inventory_path |
字符串 |
传输生成的 inventory 文件的目标路径 |
必需 |
参数 |
数据类型 |
description |
必需/可选 |
|---|---|---|---|
ssh_cp_name |
字符串 |
通过 SSH 访问已部署 VM 的 CP 资源名称 |
必需 |
ssh_username |
字符串 |
通过 SSH 访问已部署 VM 的用户名 |
必需 |
ssh_password |
字符串 |
通过 SSH 访问已部署 VM 的密码 |
必需 |
script_path |
字符串 |
外部负载均衡器安装 shell 脚本的路径 |
必需 |
simple_kubernetes_param_file.json
{
"flavourId": "simple",
"vimConnectionInfo": [{
"id": "daa80cf2-9b73-4806-8176-56cd6fab8cea",
"vimId": "c3369b54-e376-4423-bb61-afd255900fea", #Set the uuid of the VIM to use
"vimType": "openstack"
}],
"additionalParams": {
"k8s_cluster_installation_param": {
"vim_name": "kubernetes_vim",
"master_node": {
"aspect_id": "master_instance",
"ssh_cp_name": "masterNode_FloatingIP",
"nic_cp_name": "masterNode_CP1",
"username": "ubuntu",
"password": "ubuntu",
"pod_cidr": "192.168.0.0/16",
"cluster_cidr": "10.199.187.0/24"
},
"worker_node": {
"aspect_id": "worker_instance",
"ssh_cp_name": "workerNode_FloatingIP",
"nic_cp_name": "workerNode_CP2",
"username": "ubuntu",
"password": "ubuntu"
},
"proxy": {
"http_proxy": "http://user1:password1@host1:port1",
"https_proxy": "https://user2:password2@host2:port2"
},
"ansible": {
"ip_address": "10.10.0.50", # set your own Kubespray server's ssh_ip
"username": "ubuntu",
"password": "ubuntu",
"kubespray_root_path": "/home/ubuntu/kubespray-2.16.0",
"transferring_inventory_path": "/home/ubuntu/kubespray-2.16.0/inventory/mycluster"
},
"external_lb_param": {
"ssh_cp_name": "externalLB_FloatingIP",
"ssh_username": "ubuntu",
"ssh_password": "ubuntu",
"script_path": "Scripts/install_external_lb.sh"
}
},
"lcm-operation-user-data": "./UserData/lcm_user_data.py",
"lcm-operation-user-data-class": "SampleUserData"
},
"extVirtualLinks": [{
"id": "35d69de0-bf3b-4690-8724-16e9edb68b19",
"resourceId": "1642ac54-642c-407c-9c7d-e94c55ba5d33", #Set the uuid of the network to use
"extCps": [{
"cpdId": "masterNode_CP1",
"cpConfig": [{
"linkPortId": "35d69de0-bf3b-4690-8724-16e9edb68b19"
}]
}]
}, {
"id": "fb60ddb1-8787-49cd-8439-d379a449b6fa",
"resourceId": "1642ac54-642c-407c-9c7d-e94c55ba5d33", #Set the uuid of the network to use
"extCps": [{
"cpdId": "workerNode_CP2",
"cpConfig": [{
"linkPortId": "fb60ddb1-8787-49cd-8439-d379a449b6fa"
}]
}]
}, {
"id": "dd462031-64d6-4fed-be90-b9c366223a12",
"resourceId": "1642ac54-642c-407c-9c7d-e94c55ba5d33", #Set the uuid of the network to use
"extCps": [{
"cpdId": "externalLB_CP3",
"cpConfig": [{
"linkPortId": "dd462031-64d6-4fed-be90-b9c366223a12"
}]
}]
}]
}
注意
如果您想在 OpenStack 中部署多个 Kubernetes 集群，则必须在 ansible 参数中的 transferring_inventory_path 中设置不同的值。
2. 执行实例化操作¶
执行以下 CLI 命令来实例化 VNF 实例。
$ openstack vnflcm create b1db0ce7-ebca-2fb7-95ed-4840d70a1163
+--------------------------+---------------------------------------------------------------------------------------------+
| Field | Value |
+--------------------------+---------------------------------------------------------------------------------------------+
| ID | 4cdc110f-b21e-4b79-b3f5-252ee5937a36 |
| Instantiation State | NOT_INSTANTIATED |
| Links | { |
| | "self": { |
| | "href": "/vnflcm/v1/vnf_instances/4cdc110f-b21e-4b79-b3f5-252ee5937a36" |
| | }, |
| | "instantiate": { |
| | "href": "/vnflcm/v1/vnf_instances/4cdc110f-b21e-4b79-b3f5-252ee5937a36/instantiate" |
| | } |
| | } |
| VNF Instance Description | None |
| VNF Instance Name | vnf-4cdc110f-b21e-4b79-b3f5-252ee5937a36 |
| VNF Package ID | 2ed0d70f-cb42-4535-91e9-2a87362180ef |
| VNF Product Name | Sample VNF |
| VNF Provider | Company |
| VNF Software Version | 1.0 |
| VNFD ID | b1db0ce7-ebca-2fb7-95ed-4840d70a1163 |
| VNFD Version | 1.0 |
+--------------------------+---------------------------------------------------------------------------------------------+
$ openstack vnflcm instantiate 4cdc110f-b21e-4b79-b3f5-252ee5937a36 ./simple_kubernetes_param_file.json
Instantiate request for VNF Instance 4cdc110f-b21e-4b79-b3f5-252ee5937a36 has been accepted.
$ openstack vnflcm show 4cdc110f-b21e-4b79-b3f5-252ee5937a36
+--------------------------+--------------------------------------------------------------------------------------------------+
| Field | Value |
+--------------------------+--------------------------------------------------------------------------------------------------+
| ID | 4cdc110f-b21e-4b79-b3f5-252ee5937a36 |
| Instantiated Vnf Info | { |
| | "flavourId": "simple", |
| | "vnfState": "STARTED", |
| | "scaleStatus": [ |
| | { |
| | "aspectId": "master_instance", |
| | "scaleLevel": 0 |
| | }, |
| | { |
| | "aspectId": "worker_instance", |
| | "scaleLevel": 0 |
| | } |
| | ], |
| | "extCpInfo": [ |
| | { |
| | "id": "ebe12a4c-5cbf-4b19-848b-d68592f27dd7", |
| | "cpdId": "masterNode_CP1", |
| | "extLinkPortId": null, |
| | "associatedVnfcCpId": "dc0e4e27-91e1-4ff1-8aba-48e402e18092", |
| | "cpProtocolInfo": [] |
| | }, |
| | { |
| | "id": "4a6db6a4-1048-4f6f-83cf-0fccda25e92c", |
| | "cpdId": "workerNode_CP2", |
| | "extLinkPortId": null, |
| | "associatedVnfcCpId": "f8829938-d57e-4b96-b383-cf3783651822", |
| | "cpProtocolInfo": [] |
| | }, |
| | { |
| | "id": "538c068e-c2c4-449d-80f6-383242c602f2", |
| | "cpdId": "externalLB_CP3", |
| | "extLinkPortId": null, |
| | "associatedVnfcCpId": "d5c51363-1acd-4f72-b98b-cd2f8da810f9", |
| | "cpProtocolInfo": [] |
| | } |
| | ], |
| | "extVirtualLinkInfo": [ |
| | { |
| | "id": "35d69de0-bf3b-4690-8724-16e9edb68b19", |
| | "resourceHandle": { |
| | "vimConnectionId": null, |
| | "resourceId": "1642ac54-642c-407c-9c7d-e94c55ba5d33", |
| | "vimLevelResourceType": null |
| | } |
| | }, |
| | { |
| | "id": "fb60ddb1-8787-49cd-8439-d379a449b6fa", |
| | "resourceHandle": { |
| | "vimConnectionId": null, |
| | "resourceId": "1642ac54-642c-407c-9c7d-e94c55ba5d33", |
| | "vimLevelResourceType": null |
| | } |
| | }, |
| | { |
| | "id": "dd462031-64d6-4fed-be90-b9c366223a12", |
| | "resourceHandle": { |
| | "vimConnectionId": null, |
| | "resourceId": "1642ac54-642c-407c-9c7d-e94c55ba5d33", |
| | "vimLevelResourceType": null |
| | } |
| | } |
| | ], |
| | "vnfcResourceInfo": [ |
| | { |
| | "id": "d5c51363-1acd-4f72-b98b-cd2f8da810f9", |
| | "vduId": "externalLB", |
| | "computeResource": { |
| | "vimConnectionId": "c3369b54-e376-4423-bb61-afd255900fea", |
| | "resourceId": "6361ae44-7d34-4afc-b89b-00b2445425de", |
| | "vimLevelResourceType": "OS::Nova::Server" |
| | }, |
| | "storageResourceIds": [], |
| | "vnfcCpInfo": [ |
| | { |
| | "id": "ec72958c-16ff-4947-94c8-e5e32c3c453b", |
| | "cpdId": "externalLB_CP3", |
| | "vnfExtCpId": "dd462031-64d6-4fed-be90-b9c366223a12", |
| | "vnfLinkPortId": "7db09403-cc2c-42c9-9661-438b9345a549" |
| | } |
| | ] |
| | }, |
| | { |
| | "id": "dc0e4e27-91e1-4ff1-8aba-48e402e18092", |
| | "vduId": "masterNode", |
| | "computeResource": { |
| | "vimConnectionId": "c3369b54-e376-4423-bb61-afd255900fea", |
| | "resourceId": "5877e1ba-dfe0-4a9e-9ddf-08a6590942c4", |
| | "vimLevelResourceType": "OS::Nova::Server" |
| | }, |
| | "storageResourceIds": [], |
| | "vnfcCpInfo": [ |
| | { |
| | "id": "ae5d8222-ac43-4a89-a1b0-1d414c84f5c9", |
| | "cpdId": "masterNode_CP1", |
| | "vnfExtCpId": "35d69de0-bf3b-4690-8724-16e9edb68b19", |
| | "vnfLinkPortId": "23d92486-3584-49ed-afd1-0e5a71750269" |
| | } |
| | ] |
| | }, |
| | { |
| | "id": "f8829938-d57e-4b96-b383-cf3783651822", |
| | "vduId": "workerNode", |
| | "computeResource": { |
| | "vimConnectionId": "c3369b54-e376-4423-bb61-afd255900fea", |
| | "resourceId": "b8cc0d5b-600a-47eb-a67f-45d4d6051c44", |
| | "vimLevelResourceType": "OS::Nova::Server" |
| | }, |
| | "storageResourceIds": [], |
| | "vnfcCpInfo": [ |
| | { |
| | "id": "140332c5-a9b1-4663-b3e3-8006ed1b26ea", |
| | "cpdId": "workerNode_CP2", |
| | "vnfExtCpId": "fb60ddb1-8787-49cd-8439-d379a449b6fa", |
| | "vnfLinkPortId": "02c54919-290c-42a4-a0f0-dbff2b7a4725" |
| | } |
| | ] |
| | }, |
| | { |
| | "id": "cebda558-e1bd-4b98-9e11-c60a1e9fcdd7", |
| | "vduId": "workerNode", |
| | "computeResource": { |
| | "vimConnectionId": "c3369b54-e376-4423-bb61-afd255900fea", |
| | "resourceId": "89b58460-df0a-41c8-82ce-8386491d65d8", |
| | "vimLevelResourceType": "OS::Nova::Server" |
| | }, |
| | "storageResourceIds": [], |
| | "vnfcCpInfo": [ |
| | { |
| | "id": "63124ada-576a-44d4-a9c8-eec5f53fc776", |
| | "cpdId": "workerNode_CP2", |
| | "vnfExtCpId": "fb60ddb1-8787-49cd-8439-d379a449b6fa", |
| | "vnfLinkPortId": "c43a748a-4fd4-4d2f-9545-c0faec985001" |
| | } |
| | ] |
| | } |
| | ], |
| | "vnfVirtualLinkResourceInfo": [ |
| | { |
| | "id": "e3167cae-64e4-471d-83f8-93f1ed0701f0", |
| | "vnfVirtualLinkDescId": "35d69de0-bf3b-4690-8724-16e9edb68b19", |
| | "networkResource": { |
| | "vimConnectionId": null, |
| | "resourceId": "1642ac54-642c-407c-9c7d-e94c55ba5d33", |
| | "vimLevelResourceType": "OS::Neutron::Net" |
| | }, |
| | "vnfLinkPorts": [ |
| | { |
| | "id": "23d92486-3584-49ed-afd1-0e5a71750269", |
| | "resourceHandle": { |
| | "vimConnectionId": "c3369b54-e376-4423-bb61-afd255900fea", |
| | "resourceId": "2fecdfa7-bb48-4a59-81a7-aa369a3b8ab8", |
| | "vimLevelResourceType": "OS::Neutron::Port" |
| | }, |
| | "cpInstanceId": "ae5d8222-ac43-4a89-a1b0-1d414c84f5c9" |
| | } |
| | ] |
| | }, |
| | { |
| | "id": "28ee72d2-eacd-4d97-ab3b-aae552db4ca1", |
| | "vnfVirtualLinkDescId": "fb60ddb1-8787-49cd-8439-d379a449b6fa", |
| | "networkResource": { |
| | "vimConnectionId": null, |
| | "resourceId": "1642ac54-642c-407c-9c7d-e94c55ba5d33", |
| | "vimLevelResourceType": "OS::Neutron::Net" |
| | }, |
| | "vnfLinkPorts": [ |
| | { |
| | "id": "02c54919-290c-42a4-a0f0-dbff2b7a4725", |
| | "resourceHandle": { |
| | "vimConnectionId": "c3369b54-e376-4423-bb61-afd255900fea", |
| | "resourceId": "82f78566-7746-4b58-a03d-c40602279e6c", |
| | "vimLevelResourceType": "OS::Neutron::Port" |
| | }, |
| | "cpInstanceId": "140332c5-a9b1-4663-b3e3-8006ed1b26ea" |
| | }, |
| | { |
| | "id": "c43a748a-4fd4-4d2f-9545-c0faec985001", |
| | "resourceHandle": { |
| | "vimConnectionId": "c3369b54-e376-4423-bb61-afd255900fea", |
| | "resourceId": "2cfdbf95-a8b1-45da-bb18-34390bb08f46", |
| | "vimLevelResourceType": "OS::Neutron::Port" |
| | }, |
| | "cpInstanceId": "63124ada-576a-44d4-a9c8-eec5f53fc776" |
| | } |
| | ] |
| | }, |
| | { |
| | "id": "7bbe2404-9532-4eea-8872-d585aa9ceb49", |
| | "vnfVirtualLinkDescId": "dd462031-64d6-4fed-be90-b9c366223a12", |
| | "networkResource": { |
| | "vimConnectionId": null, |
| | "resourceId": "1642ac54-642c-407c-9c7d-e94c55ba5d33", |
| | "vimLevelResourceType": "OS::Neutron::Net" |
| | }, |
| | "vnfLinkPorts": [ |
| | { |
| | "id": "7db09403-cc2c-42c9-9661-438b9345a549", |
| | "resourceHandle": { |
| | "vimConnectionId": "c3369b54-e376-4423-bb61-afd255900fea", |
| | "resourceId": "9c5211ee-e008-47c6-a984-a00291f01624", |
| | "vimLevelResourceType": "OS::Neutron::Port" |
| | }, |
| | "cpInstanceId": "ec72958c-16ff-4947-94c8-e5e32c3c453b" |
| | } |
| | ] |
| | } |
| | ], |
| | "vnfcInfo": [ |
| | { |
| | "id": "ef18f7d2-75ed-4f1c-ba4f-47f3d8d8af87", |
| | "vduId": "externalLB", |
| | "vnfcState": "STARTED" |
| | }, |
| | { |
| | "id": "c4388ce1-31ec-4b07-a263-7fccba839a68", |
| | "vduId": "masterNode", |
| | "vnfcState": "STARTED" |
| | }, |
| | { |
| | "id": "b829f679-861c-4c86-8f21-384dee974b12", |
| | "vduId": "workerNode", |
| | "vnfcState": "STARTED" |
| | }, |
| | { |
| | "id": "18f6eed0-5fd4-4983-a3f4-e05f41ba2c56", |
| | "vduId": "workerNode", |
| | "vnfcState": "STARTED" |
| | } |
| | ], |
| | "additionalParams": { |
| | "lcm-operation-user-data": "./UserData/lcm_user_data.py", |
| | "lcm-operation-user-data-class": "SampleUserData", |
| | "k8sClusterInstallationParam": { |
| | "proxy": { |
| | "httpProxy": "http://voltserver:7926612078@10.85.45.88:8080", |
| | "httpsProxy": "http://voltserver:7926612078@10.85.45.88:8080" |
| | }, |
| | "ansible": { |
| | "password": "ubuntu", |
| | "username": "ubuntu", |
| | "ipAddress": "10.10.0.50", |
| | "kubesprayRootPath": "/home/ubuntu/kubespray-2.16.0", |
| | "transferringInventoryPath": "/home/ubuntu/kubespray-2.16.0/inventory/mycluster" |
| | }, |
| | "masterNode": { |
| | "password": "ubuntu", |
| | "podCidr": "192.168.0.0/16", |
| | "username": "ubuntu", |
| | "aspectId": "master_instance", |
| | "nicCpName": "masterNode_CP1", |
| | "sshCpName": "masterNode_FloatingIP", |
| | "clusterCidr": "10.199.187.0/24" |
| | }, |
| | "workerNode": { |
| | "password": "ubuntu", |
| | "username": "ubuntu", |
| | "aspectId": "worker_instance", |
| | "nicCpName": "workerNode_CP2", |
| | "sshCpName": "workerNode_FloatingIP" |
| | }, |
| | "externalLbParam": { |
| | "scriptPath": "Scripts/install_external_lb.sh", |
| | "sshCpName": "externalLB_FloatingIP", |
| | "sshPassword": "ubuntu", |
| | "sshUsername": "ubuntu" |
| | } |
| | } |
| | } |
| | } |
| Instantiation State | INSTANTIATED |
| Links | { |
| | "self": { |
| | "href": "/vnflcm/v1/vnf_instances/4cdc110f-b21e-4b79-b3f5-252ee5937a36" |
| | }, |
| | "terminate": { |
| | "href": "/vnflcm/v1/vnf_instances/4cdc110f-b21e-4b79-b3f5-252ee5937a36/terminate" |
| | }, |
| | "scale": { |
| | "href": "/vnflcm/v1/vnf_instances/4cdc110f-b21e-4b79-b3f5-252ee5937a36/scale" |
| | }, |
| | "heal": { |
| | "href": "/vnflcm/v1/vnf_instances/4cdc110f-b21e-4b79-b3f5-252ee5937a36/heal" |
| | }, |
| | "changeExtConn": { |
| | "href": "/vnflcm/v1/vnf_instances/4cdc110f-b21e-4b79-b3f5-252ee5937a36/change_ext_conn" |
| | } |
| | } |
| VIM Connection Info | [ |
| | { |
| | "id": "daa80cf2-9b73-4806-8176-56cd6fab8cea", |
| | "vimId": "c3369b54-e376-4423-bb61-afd255900fea", |
| | "vimType": "openstack", |
| | "interfaceInfo": {}, |
| | "accessInfo": {} |
| | }, |
| | { |
| | "id": "b41d21ae-fd65-4c4e-8a8e-2faa73dd1805", |
| | "vimId": "9b95449d-cac9-4e23-8e35-749c917ed181", |
| | "vimType": "kubernetes", |
| | "interfaceInfo": null, |
| | "accessInfo": { |
| | "authUrl": "https://192.168.10.182:8383" |
| | } |
| | } |
| | ] |
| VNF Instance Description | None |
| VNF Instance Name | vnf-4cdc110f-b21e-4b79-b3f5-252ee5937a36 |
| VNF Package ID | 2ed0d70f-cb42-4535-91e9-2a87362180ef |
| VNF Product Name | Sample VNF |
| VNF Provider | Company |
| VNF Software Version | 1.0 |
| VNFD ID | b1db0ce7-ebca-2fb7-95ed-4840d70a1163 |
| VNFD Version | 1.0 |
+--------------------------+--------------------------------------------------------------------------------------------------+
扩展 Kubernetes Worker 节点¶
您可以参考 扩展 Kubernetes Worker 节点 在 如何使用 Mgmt Driver 部署 Kubernetes 集群 中，了解扩展操作。
1. 创建参数文件¶
以下是发送到“POST /vnf_instances/{id}/scale”的缩放参数，数据类型为 ScaleVnfRequest，如 ETSI NFV-SOL003 v2.6.1 中所述
+------------------+---------------------------------------------------------+
| Attribute name | Parameter description |
+------------------+---------------------------------------------------------+
| type | User specify scaling operation type: |
| | "SCALE_IN" or "SCALE_OUT" |
+------------------+---------------------------------------------------------+
| aspectId | User specify target aspectId, aspectId is defined in |
| | above VNFD and user can know by |
| | ``InstantiatedVnfInfo.ScaleStatus`` that contained in |
| | the response of "GET /vnf_instances/{id}" |
+------------------+---------------------------------------------------------+
| numberOfSteps | Number of scaling steps |
+------------------+---------------------------------------------------------+
| additionalParams | If your want to change info of worker node, Kubespray |
| | Server, or Load Balancer, you can set the parameters in |
| | additionalParams. The format is the same as the one in |
| | `simple_kubernetes_param_file.json`. |
+------------------+---------------------------------------------------------+
注意
如果在 ScaleVnfRequest 的 additionalParams 中定义了 Kubernetes 信息，则需要在相应的字典中定义所有参数。这三个字典是‘worker_node’、‘ansible’和‘external_lb_param’。即使您只想更改一个值，例如密码，也需要为所有键设置值。
以下是缩放请求主体的四个示例
SCALE_OUT_sample (没有 additionalParams)
{
"type": "SCALE_OUT",
"aspectId": "worker_instance",
"numberOfSteps": "1"
}
SCALE_OUT_sample (有 additionalParams)
{
"type": "SCALE_OUT",
"aspectId": "worker_instance",
"numberOfSteps": "1"
"additionalParams": {
"k8s_cluster_installation_param": {
"worker_node": {
"aspect_id": "worker_instance",
"ssh_cp_name": "workerNode_FloatingIP",
"nic_cp_name": "workerNode_CP2",
"username": "ubuntu",
"password": "workernode1"
},
"ansible": {
"ip_address": "10.10.0.23",
"username": "ansible",
"password": "ansible",
"kubespray_root_path": "/home/ubuntu/kubespray-2.16.0",
"transferring_inventory_path":
"/home/ubuntu/kubespray-2.16.0/inventory/mycluster"
},
"external_lb_param": {
"ssh_cp_name": "externalLB_FloatingIP",
"ssh_username": "external_lb_user",
"ssh_password": "externallb",
"script_path": "Scripts/install_external_lb.sh"
}
}
}
}
SCALE_IN_sample (没有 additionalParams)
{
"type": "SCALE_IN",
"aspectId": "worker_instance",
"numberOfSteps": "1"
}
SCALE_IN_sample (有 additionalParams)
{
"type": "SCALE_IN",
"aspectId": "worker_instance",
"numberOfSteps": "1"
"additionalParams": {
"k8s_cluster_installation_param": {
"worker_node": {
"aspect_id": "worker_instance",
"ssh_cp_name": "workerNode_FloatingIP",
"nic_cp_name": "workerNode_CP2",
"username": "ubuntu",
"password": "workernode1"
},
"ansible": {
"ip_address": "10.10.0.23",
"username": "ansible",
"password": "ansible",
"kubespray_root_path": "/home/ubuntu/kubespray-2.16.0",
"transferring_inventory_path":
"/home/ubuntu/kubespray-2.16.0/inventory/mycluster"
},
"external_lb_param": {
"ssh_cp_name": "externalLB_FloatingIP",
"ssh_username": "external_lb_user",
"ssh_password": "externallb",
"script_path": "Scripts/install_external_lb.sh"
}
}
}
}
注意
本用户指南支持 worker 节点的缩放操作，但不支持 master 节点的缩放操作，因为 Kubespray 部署的 Kubernetes 集群只有一个 master 节点。
2. 执行缩放操作¶
在执行 scale 命令之前，必须确保您的 VNF 实例已实例化。VNF 包应在 创建 和 上传 VNF 包 中上传，并且 Kubernetes 集群应使用 部署 Kubernetes 集群 中的过程进行部署。
执行 worker 节点的缩放操作时，Tacker 从 Heat 调用 Heat API。
堆栈资源信号
堆栈更新
确认缩放是否成功的步骤如下
1. 执行 Heat CLI 命令，并在缩放前后检查堆栈的 ‘worker_instance’ 资源列表的数量。
2. 登录到负载均衡器，并在缩放前后检查 worker 节点数量。
要确认缩放后 worker 节点的数量，您可以使用 Heat CLI 找到 ‘worker_instance 资源列表’ 中资源 ID 的变化。此外，Kubernetes 集群中注册的 worker 节点数量应更新。有关 Heat CLI 命令的详细信息，请参阅 Heat CLI 参考。
缩放前的堆栈信息
$ openstack stack resource list vnflcm_4cdc110f-b21e-4b79-b3f5-252ee5937a36 -n 2 \
--filter type=base_hot_nested_worker.yaml -c resource_name -c physical_resource_id \
-c resource_type -c resource_status
+---------------+--------------------------------------+-----------------------------+-----------------+
| resource_name | physical_resource_id | resource_type | resource_status |
+---------------+--------------------------------------+-----------------------------+-----------------+
| hq3esjpgdtp6 | d3405e32-f049-45e5-8dd1-71791b369235 | base_hot_nested_worker.yaml | CREATE_COMPLETE |
| k8bhx9mta1vu | a6994377-e488-aeca-1083-9b4c538b2b6d | base_hot_nested_worker.yaml | CREATE_COMPLETE |
+---------------+--------------------------------------+-----------------------------+-----------------+
缩放前的 Kubernetes 集群中的 worker 节点
$ ssh ubuntu@192.168.10.182
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
master228 Ready control-plane,master 22m v1.20.7
worker55 Ready <none> 19m v1.20.7
worker75 Ready <none> 19m v1.20.7
vnf_instance 的缩放执行
$ openstack vnflcm scale 4cdc110f-b21e-4b79-b3f5-252ee5937a36 --type "SCALE_OUT" --aspect-id worker_instance --number-of-steps 1
Scale request for VNF Instance 4cdc110f-b21e-4b79-b3f5-252ee5937a36 has been accepted.
缩放后的堆栈信息
$ openstack stack resource list vnflcm_4cdc110f-b21e-4b79-b3f5-252ee5937a36 -n 2 \
--filter type=base_hot_nested_worker.yaml -c resource_name -c physical_resource_id \
-c resource_type -c resource_status
+---------------+--------------------------------------+-----------------------------+-----------------+
| resource_name | physical_resource_id | resource_type | resource_status |
+---------------+--------------------------------------+-----------------------------+-----------------+
| hq3esjpgdtp6 | d3405e32-f049-45e5-8dd1-71791b369235 | base_hot_nested_worker.yaml | UPDATE_COMPLETE |
| k8bhx9mta1vu | 56c9ec6f-5e52-44db-9d0d-57e3484e763f | base_hot_nested_worker.yaml | UPDATE_COMPLETE |
| ls8ecxdtkg4m | a6994377-e488-aeca-1083-9b4c538b2b6d | base_hot_nested_worker.yaml | CREATE_COMPLETE |
+---------------+--------------------------------------+-----------------------------+-----------------+
缩放后的 Kubernetes 集群中的 worker 节点
$ ssh ubuntu@192.168.10.182
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
master228 Ready control-plane,master 32m v1.20.7
worker55 Ready <none> 29m v1.20.7
worker75 Ready <none> 29m v1.20.7
worker43 Ready <none> 5m48s v1.20.7
缩放执行 vnf_instance
$ openstack vnflcm scale 4cdc110f-b21e-4b79-b3f5-252ee5937a36 --type "SCALE_IN" --aspect-id worker_instance --number-of-steps 1
Scale request for VNF Instance 4cdc110f-b21e-4b79-b3f5-252ee5937a36 has been accepted.
注意
此示例显示了在 “SCALE_OUT” 操作后的 “SCALE_IN” 操作的输出。
缩放后的堆栈信息
$ openstack stack resource list vnflcm_4cdc110f-b21e-4b79-b3f5-252ee5937a36 -n 2 \
--filter type=base_hot_nested_worker.yaml -c resource_name -c physical_resource_id \
-c resource_type -c resource_status
+---------------+--------------------------------------+-----------------------------+-----------------+
| resource_name | physical_resource_id | resource_type | resource_status |
+---------------+--------------------------------------+-----------------------------+-----------------+
| k8bhx9mta1vu | 56c9ec6f-5e52-44db-9d0d-57e3484e763f | base_hot_nested_worker.yaml | UPDATE_COMPLETE |
| ls8ecxdtkg4m | a6994377-e488-aeca-1083-9b4c538b2b6d | base_hot_nested_worker.yaml | UPDATE_COMPLETE |
+---------------+--------------------------------------+-----------------------------+-----------------+
缩放后的 Kubernetes 集群中的 worker 节点
$ ssh ubuntu@192.168.10.182
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
master228 Ready control-plane,master 40m v1.20.7
worker75 Ready <none> 37m v1.20.7
worker43 Ready <none> 13m v1.20.7
修复 Kubernetes Worker 节点¶
您可以参考 修复 Kubernetes Master/Worker 节点 在 如何使用 Mgmt Driver 部署 Kubernetes 集群 中，了解修复操作。
注意
本用户指南仅支持 修复 worker 节点，因为 Kubespray 部署的 Kubernetes 集群只有一个 master 节点。
1. 创建参数文件¶
以下是发送到“POST /vnf_instances/{id}/heal”的修复参数，数据类型为 HealVnfRequest。它与 SOL002 和 SOL003 不同。
+------------------+---------------------------------------------------------+
| Attribute name | Parameter description |
+------------------+---------------------------------------------------------+
| vnfcInstanceId | User specify heal target, user can know "vnfcInstanceId"|
| | by ``InstantiatedVnfInfo.vnfcResourceInfo`` that |
| | contained in the response of "GET /vnf_instances/{id}". |
+------------------+---------------------------------------------------------+
| cause | Not needed |
+------------------+---------------------------------------------------------+
| additionalParams | If your want to change info of worker node, Kubespray |
| | Server, or Load Balancer, you can set the parameters in |
| | additionalParams. So format is the same as the one in |
| | `simple_kubernetes_param_file.json`. |
+------------------+---------------------------------------------------------+
+------------------+---------------------------------------------------------+
| Attribute name | Parameter description |
+------------------+---------------------------------------------------------+
| cause | Not needed |
+------------------+---------------------------------------------------------+
| additionalParams | If your want to change info of worker node, Kubespray |
| | Server, or Load Balancer, you can set the parameters in |
| | additionalParams. So format is the same as the one in |
| | `simple_kubernetes_param_file.json`. |
+------------------+---------------------------------------------------------+
cause 和 additionalParams 在 SOL002 和 SOL003 中都受支持。
如果 vnfcInstanceId 参数为 null，则表示需要对整个 Kubernetes 集群进行修复操作，这在 SOL003 中是这种情况。
以下是 SOL002 的修复请求主体的示例
HEAL_sample (没有 additionalParams)
{
"vnfcInstanceId": "f8829938-d57e-4b96-b383-cf3783651822"
}
HEAL_sample (有 additionalParams)
{
"vnfcInstanceId": "f8829938-d57e-4b96-b383-cf3783651822",
"additionalParams": {
"k8s_cluster_installation_param": {
"worker_node": {
"aspect_id": "worker_instance",
"ssh_cp_name": "workerNode_FloatingIP",
"nic_cp_name": "workerNode_CP2",
"username": "ubuntu",
"password": "ubuntu"
},
"ansible": {
"ip_address": "10.10.0.50",
"username": "ubuntu",
"password": "ubuntu",
"kubespray_root_path": "/home/ubuntu/kubespray-2.16.0",
"transferring_inventory_path":
"/home/ubuntu/kubespray-2.16.0/inventory/mycluster"
},
"external_lb_param": {
"ssh_cp_name": "externalLB_FloatingIP",
"ssh_username": "ubuntu",
"ssh_password": "ubuntu",
"script_path": "Scripts/install_external_lb.sh"
}
}
}
}
注意
在章节 部署 Kubernetes 集群 中，VNF 实例实例化结果已在 CLI 命令 openstack vnflcm show VNF INSTANCE ID 中显示。
您可以从上述结果的 Instantiated Vnf Info 中获取 vnfcInstanceId。 vnfcResourceInfo.id 是 vnfcInstanceId。
下面提到的 physical_resource_id 与 vnfcResourceInfo.computeResource.resourceId 相同。
2. 执行修复操作¶
1. 修复 Worker 节点¶
修复指定 VNFC 实例时，Heat API 将从 Tacker 调用。
堆栈资源标记为不健康
堆栈更新
确认修复是否成功的步骤如下
1. 执行 Heat CLI 命令，并在修复前后检查 worker 节点的 physical_resource_id 和 resource_status。
2. 登录到负载均衡器，并在修复前后检查 worker 节点的年龄。
要确认 worker 节点的修复成功，您可以使用 Heat CLI 找到 ‘worker_instance 资源列表’ 中此资源的 physical_resource_id 已更改。此外，Kubernetes 集群中修复的 worker 节点的年龄应更新。有关 Heat CLI 命令的详细信息，请参阅 Heat CLI 参考。
注意
请注意，Tacker 管理的“vnfc-instance-id”和 Heat 管理的“physical-resource-id”是不同的。
修复前的 worker 节点信息
$ openstack stack resource list vnflcm_4cdc110f-b21e-4b79-b3f5-252ee5937a36 -n 2 \
--filter type=OS::Nova::Server -c resource_name -c physical_resource_id -c \
resource_type -c resource_status
+---------------+--------------------------------------+------------------+-----------------+
| resource_name | physical_resource_id | resource_type | resource_status |
+---------------+--------------------------------------+------------------+-----------------+
| workerNode | b8cc0d5b-600a-47eb-a67f-45d4d6051c44 | OS::Nova::Server | CREATE_COMPLETE |
| workerNode | 89b58460-df0a-41c8-82ce-8386491d65d8 | OS::Nova::Server | CREATE_COMPLETE |
| masterNode | 5877e1ba-dfe0-4a9e-9ddf-08a6590942c4 | OS::Nova::Server | CREATE_COMPLETE |
+---------------+--------------------------------------+------------------+-----------------+
修复前的 Kubernetes 集群中的 worker 节点
$ ssh ubuntu@192.168.10.182
$ kubectl get node
NAME STATUS ROLES AGE VERSION
master228 Ready control-plane,master 82m v1.20.7
worker55 Ready <none> 79m v1.20.7
worker75 Ready <none> 79m v1.20.7
使用 physical_resource_id b8cc0d5b-600a-47eb-a67f-45d4d6051c44 和 vnfc_instance_id f8829938-d57e-4b96-b383-cf3783651822 修复 worker 节点。
修复 worker 节点的 vnf_instance 执行
$ openstack vnflcm heal 4cdc110f-b21e-4b79-b3f5-252ee5937a36 --vnfc-instance f8829938-d57e-4b96-b383-cf3783651822
Heal request for VNF Instance 4cdc110f-b21e-4b79-b3f5-252ee5937a36 has been accepted.
修复后的 worker 节点信息
$ openstack stack resource list vnflcm_4cdc110f-b21e-4b79-b3f5-252ee5937a36 -n 2 \
--filter type=OS::Nova::Server -c resource_name -c physical_resource_id -c \
resource_type -c resource_status
+---------------+--------------------------------------+------------------+-----------------+
| resource_name | physical_resource_id | resource_type | resource_status |
+---------------+--------------------------------------+------------------+-----------------+
| workerNode | e046adef-937b-4b39-b96e-0c56cedb318c | OS::Nova::Server | CREATE_COMPLETE |
| workerNode | 89b58460-df0a-41c8-82ce-8386491d65d8 | OS::Nova::Server | CREATE_COMPLETE |
| masterNode | 5877e1ba-dfe0-4a9e-9ddf-08a6590942c4 | OS::Nova::Server | CREATE_COMPLETE |
+---------------+--------------------------------------+------------------+-----------------+
修复后的 Kubernetes 集群中的 worker 节点
$ ssh ubuntu@192.168.10.182
$ kubectl get node
NAME STATUS ROLES AGE VERSION
master228 Ready control-plane,master 102m v1.20.7
worker55 Ready <none> 79m v1.20.7
worker75 Ready <none> 5m48s v1.20.7
2. 修复整个 Kubernetes 集群¶
修复整个 VNF 时，以下 API 将从 Tacker 调用到 Heat。
堆栈删除
堆栈创建
1. 执行 Heat CLI 命令，并在修复前后检查堆栈的 ‘ID’ 和 ‘Stack Status’。
2. Kubernetes 集群的所有信息都将更改。
这用于确认堆栈 ‘ID’ 在修复前后已更改。
修复前堆栈信息
$ openstack stack list -c 'ID' -c 'Stack Name' -c 'Stack Status'
+--------------------------------------+---------------------------------------------+-----------------+
| ID | Stack Name | Stack Status |
+--------------------------------------+---------------------------------------------+-----------------+
| 62477704-94a9-4ab4-26f8-685c146a9129 | vnflcm_4cdc110f-b21e-4b79-b3f5-252ee5937a36 | CREATE_COMPLETE |
+--------------------------------------+---------------------------------------------+-----------------+
修复前的 Kubernetes 集群信息
$ ssh ubuntu@192.168.10.182
$ kubectl get node
NAME STATUS ROLES AGE VERSION
master228 Ready control-plane,master 102m v1.20.7
worker55 Ready <none> 79m v1.20.7
worker75 Ready <none> 5m48s v1.20.7
执行整个 VNF 的修复
$ openstack vnflcm heal 4cdc110f-b21e-4b79-b3f5-252ee5937a36
Heal request for VNF Instance 4cdc110f-b21e-4b79-b3f5-252ee5937a36 has been accepted.
修复后堆栈信息
$ openstack stack list -c 'ID' -c 'Stack Name' -c 'Stack Status'
+--------------------------------------+---------------------------------------------+-----------------+
| ID | Stack Name | Stack Status |
+--------------------------------------+---------------------------------------------+-----------------+
| f0e87a99-0b90-aad8-4af0-151c1303ed22 | vnflcm_4cdc110f-b21e-4b79-b3f5-252ee5937a36 | CREATE_COMPLETE |
+--------------------------------------+---------------------------------------------+-----------------+
修复后的 Kubernetes 集群信息
$ ssh ubuntu@192.168.10.232
$ kubectl get node
NAME STATUS ROLES AGE VERSION
master26 Ready control-plane,master 23m v1.20.7
worker78 Ready <none> 20m v1.20.7
worker119 Ready <none> 20m v1.20.7
负载均衡器的配置¶
当你使用这种类型的 Kubernetes vim（由 Kubespray 部署）实例化 CNF 时，如果部署的资源包含 ‘NodePort’ 类型的服务，你应该将 NodePort 设置为 Load Balancer。我们提供了一个 CNF 的示例 VNF 包和 Mgmt Driver 来配置 Load Balancer。
Mgmt Driver 仅支持实例化和heal整个操作，因为只有这两个操作才能在 Kubernetes 集群中创建 service 资源。
注意
本用户指南中提到的负载均衡器不是 Kubernetes 中类型为 ExternalLB 的 service，而是 简单：单 Master 节点 图中的 Load Balancer 服务器。
1. CNF 的 VNF 包结构¶
CNF 的 VNF 包使用方法与 创建和上传 VNF 包 中的相同。
此包的示例结构如下所示。
注意
你也可以在 tacker 的 samples/mgmt_driver/kubernetes/kubespray/cnf_nodeport_setting/cnf_nodeport_setting_vnf_package 目录中找到它们。
目录结构
TOSCA-Metadata/TOSCA.meta
Definitions/
Scripts/
文件/
!----TOSCA-Metadata
!---- TOSCA.meta
!----Definitions
!---- etsi_nfv_sol001_common_types.yaml
!---- etsi_nfv_sol001_vnfd_types.yaml
!---- helloworld3_df_simple.yaml
!---- helloworld3_top.vnfd.yaml
!---- helloworld3_types.yaml
!----Scripts
!---- configure_lb.sh
!---- cnf_nodeport_mgmt.py
!----Files
!---- kubernetes
!---- deployment.yaml
!---- service_with_nodeport.yaml
!---- service_without_nodeport.yaml
2. 部署 CNF¶
1. 创建参数文件¶
你可以参考 ETSI NFV-SOL 容器化 VNF 部署 中的 设置请求参数文件的值，了解 CNF 的参数介绍。
以下是在部署 CNF 时的一个示例请求体。
simple_cnf_param_file.json
{
"flavourId": "simple",
"vimConnectionInfo": [{
"id": "f5d17ce5-1e48-4971-8946-5c0126c0425e",
"vimId": "9b95449d-cac9-4e23-8e35-749c917ed181",
"vimType": "kubernetes"
}],
"additionalParams": {
"lcm-kubernetes-def-files": ["Files/kubernetes/deployment.yaml", "Files/kubernetes/service_with_nodeport.yaml", "Files/kubernetes/service_without_nodeport.yaml"],
"lcm-kubernetes-external-lb": {
"script_path": "Scripts/configure_lb.sh",
"external_lb_param": {
"ssh_ip": "192.168.10.182",
"ssh_username": "ubuntu",
"ssh_password": "ubuntu"
}
}
}
}
2. 执行实例化操作¶
执行以下 CLI 命令来实例化 CNF。
使用 VNFD ID 创建 VNF
$ openstack vnflcm create babb0ce7-ebca-4fa7-95ed-4840d70a1177
+--------------------------+---------------------------------------------------------------------------------------------+
| Field | Value |
+--------------------------+---------------------------------------------------------------------------------------------+
| ID | 342a083d-caec-4b44-8881-733fa7cd1754 |
| Instantiation State | NOT_INSTANTIATED |
| Links | { |
| | "self": { |
| | "href": "/vnflcm/v1/vnf_instances/342a083d-caec-4b44-8881-733fa7cd1754" |
| | }, |
| | "instantiate": { |
| | "href": "/vnflcm/v1/vnf_instances/342a083d-caec-4b44-8881-733fa7cd1754/instantiate" |
| | } |
| | } |
| VNF Instance Description | None |
| VNF Instance Name | vnf-342a083d-caec-4b44-8881-733fa7cd1754 |
| VNF Package ID | 39b5942e-41b5-4cf5-8619-ade0cd86d755 |
| VNF Product Name | Sample VNF |
| VNF Provider | Company |
| VNF Software Version | 1.0 |
| VNFD ID | babb0ce7-ebca-4fa7-95ed-4840d70a1177 |
| VNFD Version | 1.0 |
+--------------------------+---------------------------------------------------------------------------------------------+
使用 VNF ID 实例化 VNF
$ openstack vnflcm instantiate 342a083d-caec-4b44-8881-733fa7cd1754 ./simple_cnf_param_file.json
Instantiate request for VNF Instance 342a083d-caec-4b44-8881-733fa7cd1754 has been accepted.
检查实例化状态
$ openstack vnflcm show 342a083d-caec-4b44-8881-733fa7cd1754
+--------------------------+-------------------------------------------------------------------------------------------------+
| Field | Value |
+--------------------------+-------------------------------------------------------------------------------------------------+
| ID | 342a083d-caec-4b44-8881-733fa7cd1754 |
| Instantiated Vnf Info | { |
| | "flavourId": "simple", |
| | "vnfState": "STARTED", |
| | "scaleStatus": [ |
| | { |
| | "aspectId": "vdu1_aspect", |
| | "scaleLevel": 0 |
| | } |
| | ], |
| | "extCpInfo": [], |
| | "vnfcResourceInfo": [ |
| | { |
| | "id": "29db4f9d-6e50-4d5a-bd84-9e0b747e09d9", |
| | "vduId": "VDU1", |
| | "computeResource": { |
| | "vimConnectionId": null, |
| | "resourceId": "vdu1-simple-5b84bf645f-rnw8j", |
| | "vimLevelResourceType": "Deployment" |
| | }, |
| | "storageResourceIds": [] |
| | } |
| | ], |
| | "additionalParams": { |
| | "lcm-kubernetes-def-files": [ |
| | "Files/kubernetes/deployment.yaml", |
| | "Files/kubernetes/service_with_nodeport.yaml", |
| | "Files/kubernetes/service_without_nodeport.yaml" |
| | ], |
| | "lcm-kubernetes-external-lb": { |
| | "scriptPath": "Scripts/configure_lb.sh", |
| | "externalLbParam": { |
| | "sshIp": "192.168.10.182", |
| | "sshPassword": "ubuntu", |
| | "sshUsername": "ubuntu" |
| | } |
| | } |
| | } |
| | } |
| Instantiation State | INSTANTIATED |
| Links | { |
| | "self": { |
| | "href": "/vnflcm/v1/vnf_instances/342a083d-caec-4b44-8881-733fa7cd1754" |
| | }, |
| | "terminate": { |
| | "href": "/vnflcm/v1/vnf_instances/342a083d-caec-4b44-8881-733fa7cd1754/terminate" |
| | }, |
| | "scale": { |
| | "href": "/vnflcm/v1/vnf_instances/342a083d-caec-4b44-8881-733fa7cd1754/scale" |
| | }, |
| | "heal": { |
| | "href": "/vnflcm/v1/vnf_instances/342a083d-caec-4b44-8881-733fa7cd1754/heal" |
| | }, |
| | "changeExtConn": { |
| | "href": "/vnflcm/v1/vnf_instances/342a083d-caec-4b44-8881-733fa7cd1754/change_ext_conn" |
| | } |
| | } |
| VIM Connection Info | [ |
| | { |
| | "id": "f5d17ce5-1e48-4971-8946-5c0126c0425e", |
| | "vimId": "9b95449d-cac9-4e23-8e35-749c917ed181", |
| | "vimType": "kubernetes", |
| | "interfaceInfo": {}, |
| | "accessInfo": {} |
| | } |
| | ] |
| VNF Instance Description | None |
| VNF Instance Name | vnf-342a083d-caec-4b44-8881-733fa7cd1754 |
| VNF Package ID | 39b5942e-41b5-4cf5-8619-ade0cd86d755 |
| VNF Product Name | Sample VNF |
| VNF Provider | Company |
| VNF Software Version | 1.0 |
| VNFD ID | babb0ce7-ebca-4fa7-95ed-4840d70a1177 |
| VNFD Version | 1.0 |
+--------------------------+-------------------------------------------------------------------------------------------------+
要确认 NodePort 已设置为 Load Balancer，你应该通过 ssh 登录 Load Balancer。Load Balancer 将监控 service 创建的 NodePort。你可以执行以下命令来确认它们。
$ ssh ubuntu@192.168.10.182
$ kubectl get svc --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.199.187.1 <none> 443/TCP 2d18h
default nginx-service NodePort 10.199.187.127 <none> 80:30422/TCP,8080:32019/TCP 7m42s
kube-system coredns ClusterIP 10.199.187.3 <none> 53/UDP,53/TCP,9153/TCP 2d18h
kube-system nginx-service NodePort 10.199.187.43 <none> 80:30058/TCP 7m42s
$ ss -lnt
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 490 0.0.0.0:30058 0.0.0.0:*
LISTEN 0 490 0.0.0.0:32019 0.0.0.0:*
LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:*
LISTEN 0 490 0.0.0.0:30422 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 490 0.0.0.0:8383 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
你可以发现 NodePort 为 ‘30422’、‘32019’ 和 ‘30058’，并且所有 NodePort 都在 Load Balancer 上监听。
3. Heal 整个 CNF¶
当 service 类型为 NodePort 且未指定 NodePort 时，Kubernetes 将随机生成一个端口。因此，当你执行 heal CNF 整个命令并且资源包含一个未指定 NodePort 的 service 时，随机生成的端口可能会更改。此时，CNF 的 Mgmt Driver 将删除原始端口并重置新的端口到 Load Balancer。
1. 创建参数文件¶
heal CNF 整个的参数文件与 Heal Kubernetes Worker Nodes 中的 创建参数文件 章节相同。它也遵循 NFV-SOL003 v2.6.1 的规则。
2. 执行 Heal 操作¶
执行以下 CLI 命令来 heal CNF 整个。
$ openstack vnflcm heal 342a083d-caec-4b44-8881-733fa7cd1754
Heal request for VNF Instance 342a083d-caec-4b44-8881-733fa7cd1754 has been accepted.
要确认新的 NodePort 已设置为 Load Balancer，你应该通过 ssh 登录 Load Balancer。Load Balancer 将监控 service 创建的新 NodePort。你可以执行以下命令来确认它们。
$ ssh ubuntu@192.168.10.182
$ kubectl get svc --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.199.187.1 <none> 443/TCP 2d19h
default nginx-service NodePort 10.199.187.137 <none> 80:30422/TCP,8080:32019/TCP 73s
kube-system coredns ClusterIP 10.199.187.3 <none> 53/UDP,53/TCP,9153/TCP 2d19h
kube-system nginx-service NodePort 10.199.187.132 <none> 80:30702/TCP 73s
$ ss -lnt
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 490 0.0.0.0:30702 0.0.0.0:*
LISTEN 0 490 0.0.0.0:32019 0.0.0.0:*
LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:*
LISTEN 0 490 0.0.0.0:30422 0.0.0.0:*
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 490 0.0.0.0:8383 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
在本用户指南中，‘kube-system’ 命名空间中的名为 ‘nginx-service’ 的 service 没有指定 NodePort。你可以发现 NodePort 暴露的端口号已从 ‘30058’ 更改为 ‘30702’，并且所有 NodePort 都在 Load Balancer 上监听。
限制¶
本用户指南提供了一个 UserData 格式的 VNF 包。你也可以使用基于 TOSCA 的 VNF 包，方式与 SOL001 v2.6.1 相同，但它不支持缩放操作。
由于 Tacker 当前仅支持安装单 Master Kubernetes 集群，因此你无法 heal Master 节点。
