OpenStack Blazar 架构解析与功能实践

                                                                                  版权声明:转载请注明出处 JmilkFan_范桂飓:http://www.thescrewshack.com/jmilk http://www.thescrewshack.com/Jmilk/article/details/88866735

                                                                                  目录

                                                                                  Blazar

                                                                                  Blazar is a resource reservation service for OpenStack.

                                                                                  Blazar 是 OpenStack 的资源预留项目,通过 Blazar 用户可以让 OpenStack 在 “租约(leased)” 期内预留出特定的资源以供使用。

                                                                                  预留的资源类型

                                                                                  • 虚拟预留资源:Nova Instances、Cinder Volumes、Neutron Networks
                                                                                  • 物理预留资源:Compute Host(full hosts with specific characteristics of RAM, CPU, etc)

                                                                                  应用场景:

                                                                                  • 为尖峰负载准备资源
                                                                                  • 将租约作为计量结算单元
                                                                                  • 优化能源消耗
                                                                                  • 申请专用资源

                                                                                  Blazar 的安装部署

                                                                                  通过 Devstack 部署,只需要 enable blazar devstack plugin 即可:

                                                                                  enable_plugin blazar https://git.openstack.org/openstack/blazar
                                                                                  

                                                                                  Blazar 服务进程

                                                                                  • devstack@blazar-a.service:API 服务
                                                                                  • devstack@blazar-m.service:Worker 服务

                                                                                  Blazar 的软件仓库

                                                                                  NOTE 1:需要手动添加 openstack/blazar-nova 中的 blazar_filter.py 模块手动拷贝到路径 /opt/stack/nova/nova/scheduler/filters/blazar_filter.py
                                                                                  NOTE 2:nova.conf 的 enabled_filters 配置项必须启用 ServerGroupAntiAffinityFilter、AggregateInstanceExtraSpecsFilter、AggregateMultiTenancyIsolation 和 BlazarFilter 过滤器。

                                                                                  Blazar 的软件架构

                                                                                  在这里插入图片描述

                                                                                  • blazar-dashboard:Horizon Dashboard 的 Blazar 面板插件
                                                                                  • blazar-api: 提供北向 RESTful API
                                                                                  • blazar-manager:负责完成与租户签订的 Leases(租约)
                                                                                  • resource-plugin:可预留的资源类型插件,通过插件的方式开发者可以定义希望的资源类型。原生支持 physical.host 和 virtual.instance 两种 Plugins 实现。可以通过 blazar.conf 配置启用:
                                                                                  [manager]
                                                                                  plugins = physical.host.plugin,virtual.instance.plugin
                                                                                  
                                                                                  • blazar-client:Linux 上的 CLI 工具
                                                                                  [root@localhost ~]# blazar help
                                                                                  ...
                                                                                    host-create    Create a host.
                                                                                    host-delete    Delete a host.
                                                                                    host-list      Print a list of hosts.
                                                                                    host-show      Show host details.
                                                                                    host-update    Update attributes of a host.
                                                                                    lease-create   Create a lease.
                                                                                    lease-delete   Delete a lease.
                                                                                    lease-list     Print a list of leases.
                                                                                    lease-show     Show details about the given lease.
                                                                                    lease-update   Update a lease.
                                                                                  

                                                                                  Blazar 的资源模型与状态机

                                                                                  • Reservation:预留,表示将某些特定的云资源分配给指定租户。
                                                                                  • Lease:租约,Blazar 与租户之间的约定资源预留期限,Blazar 保证在约定的时间范围内为租户要求的资源进行 Reservation,所以 Lease 往往具有一个开发时间(start time)和结束时间(end time)。
                                                                                    • Immediate reservation
                                                                                    • Reservation with retries.
                                                                                    • Best-effort reservation.
                                                                                    • Delayed resource acquiring or scheduled reservation.
                                                                                  • Event:事件,指 Blazar 执行 Lease 期间所发生的事件,比如: 开发执行租约(start lease)和结束租约(end lease)。

                                                                                  Reservation status
                                                                                  在这里插入图片描述
                                                                                  Lease status
                                                                                  在这里插入图片描述
                                                                                  Event status
                                                                                  在这里插入图片描述

                                                                                  Blazar 的主机资源预留功能(Host Reservation)

                                                                                  NOTE:Host Reservation 只有管理员有操作权限。

                                                                                  在 blazar-manager 启动时会调用 Nova API 创建一个 Host Aggregate freepool:

                                                                                  [root@localhost ~]# openstack aggregate show freepool
                                                                                  +-------------------+----------------------------+
                                                                                  | Field             | Value                      |
                                                                                  +-------------------+----------------------------+
                                                                                  | availability_zone | None                       |
                                                                                  | created_at        | 2019-03-27T07:43:28.000000 |
                                                                                  | deleted           | False                      |
                                                                                  | deleted_at        | None                       |
                                                                                  | hosts             | []                         |
                                                                                  | id                | 1                          |
                                                                                  | name              | freepool                   |
                                                                                  | properties        |                            |
                                                                                  | updated_at        | None                       |
                                                                                  +-------------------+----------------------------+
                                                                                  

                                                                                  “freepool” 的名称可以通过 blazar.conf 配置:

                                                                                  [physical:host]
                                                                                  aggregate_freepool_name = freepool
                                                                                  

                                                                                  Step1. Add hosts into the freepool

                                                                                  [root@localhost ~]# openstack host list
                                                                                  +-----------------------+-------------+----------+
                                                                                  | Host Name             | Service     | Zone     |
                                                                                  +-----------------------+-------------+----------+
                                                                                  | localhost.localdomain | scheduler   | internal |
                                                                                  | localhost.localdomain | consoleauth | internal |
                                                                                  | localhost.localdomain | conductor   | internal |
                                                                                  | localhost.localdomain | conductor   | internal |
                                                                                  | localhost.localdomain | compute     | nova     |
                                                                                  +-----------------------+-------------+----------+
                                                                                  
                                                                                  [root@localhost ~]# blazar host-create localhost.localdomain
                                                                                  Created a new host:
                                                                                  +---------------------+----------------------------------+
                                                                                  | Field               | Value                            |
                                                                                  +---------------------+----------------------------------+
                                                                                  | availability_zone   | nova                             |
                                                                                  | cpu_info            | {                                |
                                                                                  |                     |     "arch": "x86_64",            |
                                                                                  |                     |     "model": "Haswell-noTSX",    |
                                                                                  |                     |     "vendor": "Intel",           |
                                                                                  |                     |     "features": [                |
                                                                                  |                     |         "pge",                   |
                                                                                  |                     |         "avx",                   |
                                                                                  |                     |         "xsaveopt",              |
                                                                                  |                     |         "clflush",               |
                                                                                  |                     |         "sep",                   |
                                                                                  |                     |         "syscall",               |
                                                                                  |                     |         "tsc_adjust",            |
                                                                                  |                     |         "tsc-deadline",          |
                                                                                  |                     |         "invpcid",               |
                                                                                  |                     |         "tsc",                   |
                                                                                  |                     |         "fsgsbase",              |
                                                                                  |                     |         "xsave",                 |
                                                                                  |                     |         "vmx",                   |
                                                                                  |                     |         "erms",                  |
                                                                                  |                     |         "cmov",                  |
                                                                                  |                     |         "smep",                  |
                                                                                  |                     |         "fpu",                   |
                                                                                  |                     |         "pat",                   |
                                                                                  |                     |         "arat",                  |
                                                                                  |                     |         "lm",                    |
                                                                                  |                     |         "msr",                   |
                                                                                  |                     |         "nx",                    |
                                                                                  |                     |         "fxsr",                  |
                                                                                  |                     |         "sse4.1",                |
                                                                                  |                     |         "pae",                   |
                                                                                  |                     |         "sse4.2",                |
                                                                                  |                     |         "pclmuldq",              |
                                                                                  |                     |         "pcid",                  |
                                                                                  |                     |         "fma",                   |
                                                                                  |                     |         "vme",                   |
                                                                                  |                     |         "mmx",                   |
                                                                                  |                     |         "osxsave",               |
                                                                                  |                     |         "cx8",                   |
                                                                                  |                     |         "mce",                   |
                                                                                  |                     |         "de",                    |
                                                                                  |                     |         "aes",                   |
                                                                                  |                     |         "mca",                   |
                                                                                  |                     |         "pse",                   |
                                                                                  |                     |         "lahf_lm",               |
                                                                                  |                     |         "abm",                   |
                                                                                  |                     |         "popcnt",                |
                                                                                  |                     |         "pdpe1gb",               |
                                                                                  |                     |         "apic",                  |
                                                                                  |                     |         "sse",                   |
                                                                                  |                     |         "f16c",                  |
                                                                                  |                     |         "ds",                    |
                                                                                  |                     |         "invtsc",                |
                                                                                  |                     |         "pni",                   |
                                                                                  |                     |         "rdtscp",                |
                                                                                  |                     |         "avx2",                  |
                                                                                  |                     |         "sse2",                  |
                                                                                  |                     |         "ss",                    |
                                                                                  |                     |         "hypervisor",            |
                                                                                  |                     |         "bmi1",                  |
                                                                                  |                     |         "bmi2",                  |
                                                                                  |                     |         "ssse3",                 |
                                                                                  |                     |         "cx16",                  |
                                                                                  |                     |         "pse36",                 |
                                                                                  |                     |         "mtrr",                  |
                                                                                  |                     |         "movbe",                 |
                                                                                  |                     |         "rdrand",                |
                                                                                  |                     |         "x2apic"                 |
                                                                                  |                     |     ],                           |
                                                                                  |                     |     "topology": {                |
                                                                                  |                     |         "cores": 1,              |
                                                                                  |                     |         "cells": 1,              |
                                                                                  |                     |         "threads": 1,            |
                                                                                  |                     |         "sockets": 16            |
                                                                                  |                     |     }                            |
                                                                                  |                     | }                                |
                                                                                  | created_at          | 2019-03-27 07:52:53              |
                                                                                  | hypervisor_hostname | localhost.localdomain            |
                                                                                  | hypervisor_type     | QEMU                             |
                                                                                  | hypervisor_version  | 2012000                          |
                                                                                  | id                  | 1                                |
                                                                                  | local_gb            | 189                              |
                                                                                  | memory_mb           | 40959                            |
                                                                                  | reservable          | True                             |
                                                                                  | service_name        | localhost.localdomain            |
                                                                                  | status              |                                  |
                                                                                  | trust_id            | 266b2c90dad24959a98bb0eea6defede |
                                                                                  | updated_at          |                                  |
                                                                                  | vcpus               | 16                               |
                                                                                  +---------------------+----------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# blazar host-list
                                                                                  +----+-----------------------+-------+-----------+----------+
                                                                                  | id | hypervisor_hostname   | vcpus | memory_mb | local_gb |
                                                                                  +----+-----------------------+-------+-----------+----------+
                                                                                  | 1  | localhost.localdomain |    16 |     40959 |      189 |
                                                                                  +----+-----------------------+-------+-----------+----------+
                                                                                  
                                                                                  [root@localhost ~]# openstack aggregate show freepool
                                                                                  +-------------------+----------------------------+
                                                                                  | Field             | Value                      |
                                                                                  +-------------------+----------------------------+
                                                                                  | availability_zone | None                       |
                                                                                  | created_at        | 2019-03-27T07:43:28.000000 |
                                                                                  | deleted           | False                      |
                                                                                  | deleted_at        | None                       |
                                                                                  | hosts             | [u'localhost.localdomain'] |
                                                                                  | id                | 1                          |
                                                                                  | name              | freepool                   |
                                                                                  | properties        |                            |
                                                                                  | updated_at        | None                       |
                                                                                  +-------------------+----------------------------+
                                                                                  

                                                                                  NOTE:freepool 中的 hosts 是 Blazar 从 Nova 主机资源中通过 Host Aggregate 划分出来的用于进行资源预留的主机资源。

                                                                                  选择主机加入 freepool
                                                                                  在这里插入图片描述
                                                                                  支持输入主机的扩展属性:只起到了自定义标签属性的作用(PS:Any key-value pair can be stored as one of extra capabilities of the host.)。
                                                                                  在这里插入图片描述

                                                                                  Step 2. Create a Host Reservation Lease

                                                                                  [root@localhost ~]# blazar lease-create --physical-reservation min=1,max=1,hypervisor_properties='[">=", "$vcpus", "2"]' --start-date "2019-03-28 08:10" --end-date "2019-03-28 08:25" host-lease-1
                                                                                  Matches: (u'min=1,max=1', u'hypervisor_properties', u'[">=", "$vcpus", "2"]')
                                                                                  Matches: (u'min=1', u'max', u'1')
                                                                                  Matches: (None, u'min', u'1')
                                                                                  Created a new lease:
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  | Field        | Value                                                        |
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  | created_at   | 2019-03-28 08:06:40                                          |
                                                                                  | degraded     | False                                                        |
                                                                                  | end_date     | 2019-03-28T08:25:00.000000                                   |
                                                                                  | events       | {                                                            |
                                                                                  |              |     "status": "UNDONE",                                      |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "start_lease",                             |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": null,                                      |
                                                                                  |              |     "time": "2019-03-28T08:10:00.000000",                    |
                                                                                  |              |     "id": "ac7c0963-5de6-4bf9-a60e-f74416a057dc"             |
                                                                                  |              | }                                                            |
                                                                                  |              | {                                                            |
                                                                                  |              |     "status": "UNDONE",                                      |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "end_lease",                               |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": null,                                      |
                                                                                  |              |     "time": "2019-03-28T08:25:00.000000",                    |
                                                                                  |              |     "id": "bdb2e272-a1a1-4b27-9464-a631cf999371"             |
                                                                                  |              | }                                                            |
                                                                                  |              | {                                                            |
                                                                                  |              |     "status": "UNDONE",                                      |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "before_end_lease",                        |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": null,                                      |
                                                                                  |              |     "time": "2019-03-28T08:10:00.000000",                    |
                                                                                  |              |     "id": "feaeb0bb-b28d-449e-a93d-a10613c33139"             |
                                                                                  |              | }                                                            |
                                                                                  | id           | a8b973f5-c61c-472b-a11e-68620a3f7c80                         |
                                                                                  | name         | host-lease-1                                                 |
                                                                                  | project_id   | f47d5bee7acd48d286224f2c411a17fb                             |
                                                                                  | reservations | {                                                            |
                                                                                  |              |     "status": "pending",                                     |
                                                                                  |              |     "before_end": "default",                                 |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "resource_id": "08f7f776-11bd-44b7-bc31-89faa68f55a1",   |
                                                                                  |              |     "max": 1,                                                |
                                                                                  |              |     "created_at": "2019-03-28 08:06:40",                     |
                                                                                  |              |     "min": 1,                                                |
                                                                                  |              |     "updated_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "missing_resources": false,                              |
                                                                                  |              |     "hypervisor_properties": "[\">=\", \"$vcpus\", \"2\"]",  |
                                                                                  |              |     "resource_properties": "",                               |
                                                                                  |              |     "id": "383371a7-652f-495d-b05e-df5ab5d67aa0",            |
                                                                                  |              |     "resource_type": "physical:host",                        |
                                                                                  |              |     "resources_changed": false                               |
                                                                                  |              | }                                                            |
                                                                                  | start_date   | 2019-03-28T08:10:00.000000                                   |
                                                                                  | status       | PENDING                                                      |
                                                                                  | trust_id     | abb316d377ee4f139379b509b3f4da87                             |
                                                                                  | updated_at   | 2019-03-28 08:06:45                                          |
                                                                                  | user_id      | e289dac11233451badb1409104a6505b                             |
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# blazar lease-list
                                                                                  +--------------------------------------+--------------+----------------------------+----------------------------+
                                                                                  | id                                   | name         | start_date                 | end_date                   |
                                                                                  +--------------------------------------+--------------+----------------------------+----------------------------+
                                                                                  | a8b973f5-c61c-472b-a11e-68620a3f7c80 | host-lease-1 | 2019-03-28T08:10:00.000000 | 2019-03-28T08:25:00.000000 |
                                                                                  +--------------------------------------+--------------+----------------------------+----------------------------+
                                                                                  
                                                                                  • --physical-reservation:表示创建 Host Reservation 类型的 Lease。
                                                                                  • min=1,max=1:表示要预留的主机数量(区间)
                                                                                  • hypervisor_properties='[">=", "$vcpus", "2"]':描述 freepool 中的主机(Hypervisor)的特征信息
                                                                                    在这里插入图片描述
                                                                                  • Resource Properties 描述主机资源属性。

                                                                                  NOTE:初始状态下(当前时刻没有到达 Lease 时间),Lease host-lease-1 的 status 为 PENDING,对应的 Aggregate(Blazar 通过 Nova API 创建)e1c5add2-e3a8-433f-9b2d-ca19c8d329a8 中不包含任何主机。

                                                                                  [root@localhost ~]# blazar lease-show a8b973f5-c61c-472b-a11e-68620a3f7c80
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  | Field        | Value                                                        |
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  | created_at   | 2019-03-28 08:06:40                                          |
                                                                                  | degraded     | False                                                        |
                                                                                  | end_date     | 2019-03-28T08:25:00.000000                                   |
                                                                                  | events       | {                                                            |
                                                                                  |              |     "status": "UNDONE",                                      |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "start_lease",                             |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": null,                                      |
                                                                                  |              |     "time": "2019-03-28T08:10:00.000000",                    |
                                                                                  |              |     "id": "ac7c0963-5de6-4bf9-a60e-f74416a057dc"             |
                                                                                  |              | }                                                            |
                                                                                  |              | {                                                            |
                                                                                  |              |     "status": "UNDONE",                                      |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "end_lease",                               |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": null,                                      |
                                                                                  |              |     "time": "2019-03-28T08:25:00.000000",                    |
                                                                                  |              |     "id": "bdb2e272-a1a1-4b27-9464-a631cf999371"             |
                                                                                  |              | }                                                            |
                                                                                  |              | {                                                            |
                                                                                  |              |     "status": "UNDONE",                                      |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "before_end_lease",                        |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": null,                                      |
                                                                                  |              |     "time": "2019-03-28T08:10:00.000000",                    |
                                                                                  |              |     "id": "feaeb0bb-b28d-449e-a93d-a10613c33139"             |
                                                                                  |              | }                                                            |
                                                                                  | id           | a8b973f5-c61c-472b-a11e-68620a3f7c80                         |
                                                                                  | name         | host-lease-1                                                 |
                                                                                  | project_id   | f47d5bee7acd48d286224f2c411a17fb                             |
                                                                                  | reservations | {                                                            |
                                                                                  |              |     "status": "pending",                                     |
                                                                                  |              |     "before_end": "default",                                 |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "resource_id": "08f7f776-11bd-44b7-bc31-89faa68f55a1",   |
                                                                                  |              |     "max": 1,                                                |
                                                                                  |              |     "created_at": "2019-03-28 08:06:40",                     |
                                                                                  |              |     "min": 1,                                                |
                                                                                  |              |     "updated_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "missing_resources": false,                              |
                                                                                  |              |     "hypervisor_properties": "[\">=\", \"$vcpus\", \"2\"]",  |
                                                                                  |              |     "resource_properties": "",                               |
                                                                                  |              |     "id": "383371a7-652f-495d-b05e-df5ab5d67aa0",            |
                                                                                  |              |     "resource_type": "physical:host",                        |
                                                                                  |              |     "resources_changed": false                               |
                                                                                  |              | }                                                            |
                                                                                  | start_date   | 2019-03-28T08:10:00.000000                                   |
                                                                                  | status       | PENDING                                                      |
                                                                                  | trust_id     | abb316d377ee4f139379b509b3f4da87                             |
                                                                                  | updated_at   | 2019-03-28 08:06:45                                          |
                                                                                  | user_id      | e289dac11233451badb1409104a6505b                             |
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack aggregate show 383371a7-652f-495d-b05e-df5ab5d67aa0
                                                                                  +-------------------+-------------------------------------------------+
                                                                                  | Field             | Value                                           |
                                                                                  +-------------------+-------------------------------------------------+
                                                                                  | availability_zone | blazar_383371a7-652f-495d-b05e-df5ab5d67aa0     |
                                                                                  | created_at        | 2019-03-28T08:06:42.000000                      |
                                                                                  | deleted           | False                                           |
                                                                                  | deleted_at        | None                                            |
                                                                                  | hosts             | []                      |
                                                                                  | id                | 7                                               |
                                                                                  | name              | 383371a7-652f-495d-b05e-df5ab5d67aa0            |
                                                                                  | properties        | blazar:owner='f47d5bee7acd48d286224f2c411a17fb' |
                                                                                  | updated_at        | None                                            |
                                                                                  +-------------------+-------------------------------------------------+
                                                                                  

                                                                                  Step 3. Use the leased resources

                                                                                  [root@localhost ~]# openstack server create --flavor m1.tiny --image cirros-0.3.4-x86_64-disk --network private --hint reservation=383371a7-652f-495d-b05e-df5ab5d67aa0 VM1
                                                                                  +-------------------------------------+-----------------------------------------------------------------+
                                                                                  | Field                               | Value                                                           |
                                                                                  +-------------------------------------+-----------------------------------------------------------------+
                                                                                  | OS-DCF:diskConfig                   | MANUAL                                                          |
                                                                                  | OS-EXT-AZ:availability_zone         |                                                                 |
                                                                                  | OS-EXT-SRV-ATTR:host                | None                                                            |
                                                                                  | OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                            |
                                                                                  | OS-EXT-SRV-ATTR:instance_name       |                                                                 |
                                                                                  | OS-EXT-STS:power_state              | NOSTATE                                                         |
                                                                                  | OS-EXT-STS:task_state               | scheduling                                                      |
                                                                                  | OS-EXT-STS:vm_state                 | building                                                        |
                                                                                  | OS-SRV-USG:launched_at              | None                                                            |
                                                                                  | OS-SRV-USG:terminated_at            | None                                                            |
                                                                                  | accessIPv4                          |                                                                 |
                                                                                  | accessIPv6                          |                                                                 |
                                                                                  | addresses                           |                                                                 |
                                                                                  | adminPass                           | Pm4XEBViavaZ                                                    |
                                                                                  | config_drive                        |                                                                 |
                                                                                  | created                             | 2019-03-28T08:08:34Z                                            |
                                                                                  | flavor                              | m1.tiny (1)                                                     |
                                                                                  | hostId                              |                                                                 |
                                                                                  | id                                  | afb7db27-ebe1-4313-b5d0-a42253f14a08                            |
                                                                                  | image                               | cirros-0.3.4-x86_64-disk (0bdcd04f-2f77-42c3-ae61-5f10e2f1955c) |
                                                                                  | key_name                            | None                                                            |
                                                                                  | name                                | VM1                                                             |
                                                                                  | progress                            | 0                                                               |
                                                                                  | project_id                          | f47d5bee7acd48d286224f2c411a17fb                                |
                                                                                  | properties                          |                                                                 |
                                                                                  | security_groups                     | name='default'                                                  |
                                                                                  | status                              | BUILD                                                           |
                                                                                  | updated                             | 2019-03-28T08:08:35Z                                            |
                                                                                  | user_id                             | e289dac11233451badb1409104a6505b                                |
                                                                                  | volumes_attached                    |                                                                 |
                                                                                  +-------------------------------------+-----------------------------------------------------------------+
                                                                                  [root@localhost ~]# openstack server list
                                                                                  +--------------------------------------+------+--------+----------+--------------------------+---------+
                                                                                  | ID                                   | Name | Status | Networks | Image                    | Flavor  |
                                                                                  +--------------------------------------+------+--------+----------+--------------------------+---------+
                                                                                  | afb7db27-ebe1-4313-b5d0-a42253f14a08 | VM1  | ERROR  |          | cirros-0.3.4-x86_64-disk | m1.tiny |
                                                                                  +--------------------------------------+------+--------+----------+--------------------------+---------+
                                                                                  
                                                                                  • --hint reservation=383371a7-652f-495d-b05e-df5ab5d67aa0:应用我(Tenant)的 Host Reservation Lease 预留资源来创建虚拟机。

                                                                                  虚拟机创建失败!nova-scheduler.service 日志报错,没有通过 BlazarFilter 过滤器。该结果在预期之内,因为我现在只有一台计算节点,而且被加入到 Host Reservation Lease,也就是说在 Lease 租期时间到来之前,我(Tenant)已经没有可以使用的主机资源了。

                                                                                  Filter BlazarFilter returned 0 hosts
                                                                                  

                                                                                  NOTE:Blazar 履行 Lease 时,lease 的 status 为 ACTIVE,对应的 Aggregate e1c5add2-e3a8-433f-9b2d-ca19c8d329a8 中包含从 freepool 中 “划分” 过来的主机。e.g.

                                                                                  [root@localhost ~]# blazar lease-show a8b973f5-c61c-472b-a11e-68620a3f7c80
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  | Field        | Value                                                        |
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  | created_at   | 2019-03-28 08:06:40                                          |
                                                                                  | degraded     | False                                                        |
                                                                                  | end_date     | 2019-03-28T08:25:00.000000                                   |
                                                                                  | events       | {                                                            |
                                                                                  |              |     "status": "DONE",                                        |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "start_lease",                             |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": "2019-03-28 08:10:16",                     |
                                                                                  |              |     "time": "2019-03-28T08:10:00.000000",                    |
                                                                                  |              |     "id": "ac7c0963-5de6-4bf9-a60e-f74416a057dc"             |
                                                                                  |              | }                                                            |
                                                                                  |              | {                                                            |
                                                                                  |              |     "status": "UNDONE",                                      |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "end_lease",                               |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": null,                                      |
                                                                                  |              |     "time": "2019-03-28T08:25:00.000000",                    |
                                                                                  |              |     "id": "bdb2e272-a1a1-4b27-9464-a631cf999371"             |
                                                                                  |              | }                                                            |
                                                                                  |              | {                                                            |
                                                                                  |              |     "status": "DONE",                                        |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "before_end_lease",                        |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": "2019-03-28 08:10:08",                     |
                                                                                  |              |     "time": "2019-03-28T08:10:00.000000",                    |
                                                                                  |              |     "id": "feaeb0bb-b28d-449e-a93d-a10613c33139"             |
                                                                                  |              | }                                                            |
                                                                                  | id           | a8b973f5-c61c-472b-a11e-68620a3f7c80                         |
                                                                                  | name         | host-lease-1                                                 |
                                                                                  | project_id   | f47d5bee7acd48d286224f2c411a17fb                             |
                                                                                  | reservations | {                                                            |
                                                                                  |              |     "status": "active",                                      |
                                                                                  |              |     "before_end": "default",                                 |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "resource_id": "08f7f776-11bd-44b7-bc31-89faa68f55a1",   |
                                                                                  |              |     "max": 1,                                                |
                                                                                  |              |     "created_at": "2019-03-28 08:06:40",                     |
                                                                                  |              |     "min": 1,                                                |
                                                                                  |              |     "updated_at": "2019-03-28 08:10:16",                     |
                                                                                  |              |     "missing_resources": false,                              |
                                                                                  |              |     "hypervisor_properties": "[\">=\", \"$vcpus\", \"2\"]",  |
                                                                                  |              |     "resource_properties": "",                               |
                                                                                  |              |     "id": "383371a7-652f-495d-b05e-df5ab5d67aa0",            |
                                                                                  |              |     "resource_type": "physical:host",                        |
                                                                                  |              |     "resources_changed": false                               |
                                                                                  |              | }                                                            |
                                                                                  | start_date   | 2019-03-28T08:10:00.000000                                   |
                                                                                  | status       | ACTIVE                                                       |
                                                                                  | trust_id     | abb316d377ee4f139379b509b3f4da87                             |
                                                                                  | updated_at   | 2019-03-28 08:10:16                                          |
                                                                                  | user_id      | e289dac11233451badb1409104a6505b                             |
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack aggregate show 383371a7-652f-495d-b05e-df5ab5d67aa0
                                                                                  +-------------------+-------------------------------------------------+
                                                                                  | Field             | Value                                           |
                                                                                  +-------------------+-------------------------------------------------+
                                                                                  | availability_zone | blazar_383371a7-652f-495d-b05e-df5ab5d67aa0     |
                                                                                  | created_at        | 2019-03-28T08:06:42.000000                      |
                                                                                  | deleted           | False                                           |
                                                                                  | deleted_at        | None                                            |
                                                                                  | hosts             | [u'localhost.localdomain']                      |
                                                                                  | id                | 7                                               |
                                                                                  | name              | 383371a7-652f-495d-b05e-df5ab5d67aa0            |
                                                                                  | properties        | blazar:owner='f47d5bee7acd48d286224f2c411a17fb' |
                                                                                  | updated_at        | None                                            |
                                                                                  +-------------------+-------------------------------------------------+
                                                                                  

                                                                                  此时再尝试 Use the leased resources:

                                                                                  [root@localhost ~]# openstack server create --flavor m1.tiny --image cirros-0.3.4-x86_64-disk --network private --hint reservation=383371a7-652f-495d-b05e-df5ab5d67aa0 VM2
                                                                                  +-------------------------------------+-----------------------------------------------------------------+
                                                                                  | Field                               | Value                                                           |
                                                                                  +-------------------------------------+-----------------------------------------------------------------+
                                                                                  | OS-DCF:diskConfig                   | MANUAL                                                          |
                                                                                  | OS-EXT-AZ:availability_zone         |                                                                 |
                                                                                  | OS-EXT-SRV-ATTR:host                | None                                                            |
                                                                                  | OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                            |
                                                                                  | OS-EXT-SRV-ATTR:instance_name       |                                                                 |
                                                                                  | OS-EXT-STS:power_state              | NOSTATE                                                         |
                                                                                  | OS-EXT-STS:task_state               | scheduling                                                      |
                                                                                  | OS-EXT-STS:vm_state                 | building                                                        |
                                                                                  | OS-SRV-USG:launched_at              | None                                                            |
                                                                                  | OS-SRV-USG:terminated_at            | None                                                            |
                                                                                  | accessIPv4                          |                                                                 |
                                                                                  | accessIPv6                          |                                                                 |
                                                                                  | addresses                           |                                                                 |
                                                                                  | adminPass                           | 7Sgn5itS2xgb                                                    |
                                                                                  | config_drive                        |                                                                 |
                                                                                  | created                             | 2019-03-28T08:15:38Z                                            |
                                                                                  | flavor                              | m1.tiny (1)                                                     |
                                                                                  | hostId                              |                                                                 |
                                                                                  | id                                  | ec0a8419-c957-425d-bc32-deca2b193a14                            |
                                                                                  | image                               | cirros-0.3.4-x86_64-disk (0bdcd04f-2f77-42c3-ae61-5f10e2f1955c) |
                                                                                  | key_name                            | None                                                            |
                                                                                  | name                                | VM2                                                             |
                                                                                  | progress                            | 0                                                               |
                                                                                  | project_id                          | f47d5bee7acd48d286224f2c411a17fb                                |
                                                                                  | properties                          |                                                                 |
                                                                                  | security_groups                     | name='default'                                                  |
                                                                                  | status                              | BUILD                                                           |
                                                                                  | updated                             | 2019-03-28T08:15:39Z                                            |
                                                                                  | user_id                             | e289dac11233451badb1409104a6505b                                |
                                                                                  | volumes_attached                    |                                                                 |
                                                                                  +-------------------------------------+-----------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack server list
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+---------+
                                                                                  | ID                                   | Name | Status | Networks             | Image                    | Flavor  |
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+---------+
                                                                                  | ec0a8419-c957-425d-bc32-deca2b193a14 | VM2  | ACTIVE | private=192.168.1.19 | cirros-0.3.4-x86_64-disk | m1.tiny |
                                                                                  | afb7db27-ebe1-4313-b5d0-a42253f14a08 | VM1  | ERROR  |                      | cirros-0.3.4-x86_64-disk | m1.tiny |
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+---------+
                                                                                  

                                                                                  虚拟机创建成功!因为在 Lease 租期范围内,这台 Compute Host 就是专属于我(Tenant)的预留资源。

                                                                                  NOTE:Lease 执行完之后,lease 的 status 为 TERMINATED,对应的 Aggregate e1c5add2-e3a8-433f-9b2d-ca19c8d329a8 被删除,对应的虚拟机被删除。Lease 租约到期,blazar-manager 守护进程自动执行资源回收。e.g.

                                                                                  [root@localhost ~]# blazar lease-show a8b973f5-c61c-472b-a11e-68620a3f7c80
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  | Field        | Value                                                        |
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  | created_at   | 2019-03-28 08:06:40                                          |
                                                                                  | degraded     | False                                                        |
                                                                                  | end_date     | 2019-03-28T08:25:00.000000                                   |
                                                                                  | events       | {                                                            |
                                                                                  |              |     "status": "DONE",                                        |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "start_lease",                             |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": "2019-03-28 08:10:16",                     |
                                                                                  |              |     "time": "2019-03-28T08:10:00.000000",                    |
                                                                                  |              |     "id": "ac7c0963-5de6-4bf9-a60e-f74416a057dc"             |
                                                                                  |              | }                                                            |
                                                                                  |              | {                                                            |
                                                                                  |              |     "status": "DONE",                                        |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "end_lease",                               |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": "2019-03-28 08:25:21",                     |
                                                                                  |              |     "time": "2019-03-28T08:25:00.000000",                    |
                                                                                  |              |     "id": "bdb2e272-a1a1-4b27-9464-a631cf999371"             |
                                                                                  |              | }                                                            |
                                                                                  |              | {                                                            |
                                                                                  |              |     "status": "DONE",                                        |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "event_type": "before_end_lease",                        |
                                                                                  |              |     "created_at": "2019-03-28 08:06:45",                     |
                                                                                  |              |     "updated_at": "2019-03-28 08:10:08",                     |
                                                                                  |              |     "time": "2019-03-28T08:10:00.000000",                    |
                                                                                  |              |     "id": "feaeb0bb-b28d-449e-a93d-a10613c33139"             |
                                                                                  |              | }                                                            |
                                                                                  | id           | a8b973f5-c61c-472b-a11e-68620a3f7c80                         |
                                                                                  | name         | host-lease-1                                                 |
                                                                                  | project_id   | f47d5bee7acd48d286224f2c411a17fb                             |
                                                                                  | reservations | {                                                            |
                                                                                  |              |     "status": "deleted",                                     |
                                                                                  |              |     "before_end": "default",                                 |
                                                                                  |              |     "lease_id": "a8b973f5-c61c-472b-a11e-68620a3f7c80",      |
                                                                                  |              |     "resource_id": "08f7f776-11bd-44b7-bc31-89faa68f55a1",   |
                                                                                  |              |     "max": 1,                                                |
                                                                                  |              |     "created_at": "2019-03-28 08:06:40",                     |
                                                                                  |              |     "min": 1,                                                |
                                                                                  |              |     "updated_at": "2019-03-28 08:25:21",                     |
                                                                                  |              |     "missing_resources": false,                              |
                                                                                  |              |     "hypervisor_properties": "[\">=\", \"$vcpus\", \"2\"]",  |
                                                                                  |              |     "resource_properties": "",                               |
                                                                                  |              |     "id": "383371a7-652f-495d-b05e-df5ab5d67aa0",            |
                                                                                  |              |     "resource_type": "physical:host",                        |
                                                                                  |              |     "resources_changed": false                               |
                                                                                  |              | }                                                            |
                                                                                  | start_date   | 2019-03-28T08:10:00.000000                                   |
                                                                                  | status       | TERMINATED                                                   |
                                                                                  | trust_id     | abb316d377ee4f139379b509b3f4da87                             |
                                                                                  | updated_at   | 2019-03-28 08:25:21                                          |
                                                                                  | user_id      | e289dac11233451badb1409104a6505b                             |
                                                                                  +--------------+--------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack aggregate list
                                                                                  +----+----------+-------------------+
                                                                                  | ID | Name     | Availability Zone |
                                                                                  +----+----------+-------------------+
                                                                                  |  1 | freepool | None              |
                                                                                  +----+----------+-------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack server list
                                                                                  +--------------------------------------+------+--------+----------+--------------------------+---------+
                                                                                  | ID                                   | Name | Status | Networks | Image                    | Flavor  |
                                                                                  +--------------------------------------+------+--------+----------+--------------------------+---------+
                                                                                  | afb7db27-ebe1-4313-b5d0-a42253f14a08 | VM1  | ERROR  |          | cirros-0.3.4-x86_64-disk | m1.tiny |
                                                                                  +--------------------------------------+------+--------+----------+--------------------------+---------+
                                                                                  

                                                                                  代码实现

                                                                                  我们主要关注实现 “预留” 的 BlazarFilter。

                                                                                  ...
                                                                                  opts = [
                                                                                      cfg.StrOpt('aggregate_freepool_name',
                                                                                                 default='freepool',
                                                                                                 help='Name of the special aggregate where all hosts '
                                                                                                      'are candidate for physical host reservation'),
                                                                                      cfg.StrOpt('project_id_key',
                                                                                                 default='blazar:tenant',
                                                                                                 help='Aggregate metadata value for key matching project_id'),
                                                                                      cfg.StrOpt('blazar_owner',
                                                                                                 default='blazar:owner',
                                                                                                 help='Aggregate metadata key for knowing owner project_id'),
                                                                                      cfg.StrOpt('blazar_az_prefix',
                                                                                                 default='blazar_',
                                                                                                 help='Prefix for Availability Zones created by Blazar')
                                                                                  ]
                                                                                  ...
                                                                                  
                                                                                  class BlazarFilter(filters.BaseHostFilter):
                                                                                      """Blazar Filter for nova-scheduler."""
                                                                                      ...
                                                                                      def host_passes(self, host_state, spec_obj):
                                                                                          """Check if a host in a pool can be used for a request
                                                                                  
                                                                                          A host is in a pool if it is a member of an aggregate that has
                                                                                          a metadata item with a key value of "blazar:owner"
                                                                                  
                                                                                          If the user does not pass "reservation=<id>" as a hint then only
                                                                                          hosts which are not in a pool (including freepool) pass.
                                                                                  
                                                                                          If the user does pass "reservation=<id>" as a hint then the host only
                                                                                          passes if it is a member of the specified pool and that pool
                                                                                          has a metadata key of either :
                                                                                              - "blazar:owner=tenant_id (which grants automatically all the
                                                                                                  users from the tenant from which the request came from)
                                                                                              - or, "tenant_id=blazar:tenant" (which grants extra tenants for
                                                                                                  the reservation)
                                                                                          """
                                                                                  
                                                                                          # Find which Pools the user wants to use (if any)
                                                                                          # e.g. --hint reservation={reservation_id}
                                                                                          requested_pools = spec_obj.get_scheduler_hint('reservation')
                                                                                          if isinstance(requested_pools, six.text_type):
                                                                                              requested_pools = [requested_pools]
                                                                                  
                                                                                          # the request is host reservation
                                                                                          if requested_pools:
                                                                                              # 检查该 Host 是否处于 --hint reservation={reservation_id} 指定的 freepool 中
                                                                                              # 如果处于则检测 Created Lease 和 Created Instance 的 Tenant 身份是否一致
                                                                                              # 如果一致,则 return True
                                                                                              return self.host_reservation_request(host_state, spec_obj,
                                                                                                                                   requested_pools)
                                                                                          ...
                                                                                          return True
                                                                                  
                                                                                      def host_reservation_request(self, host_state, spec_obj, requested_pools):
                                                                                  
                                                                                          # 获取 Host 所属的 freepools 对应的 Host Aggregates
                                                                                          pools = self.fetch_blazar_pools(host_state)
                                                                                  
                                                                                          # 如果 pool(Host Aggregates) 的 name 等于 {reservation_id}
                                                                                          # 表示希望虚拟机在这个 pool 上启动
                                                                                          for pool in [p for p in pools if p.name in requested_pools]:
                                                                                              # Check tenant is allowed to use this Pool
                                                                                  
                                                                                              # NOTE(sbauza): Currently, the key is only the project_id,
                                                                                              #  but later will possibly be blazar:tenant:{project_id}
                                                                                              key = spec_obj.project_id
                                                                                              access = pool.metadata.get(key)
                                                                                              if access:
                                                                                                  return True
                                                                                              # NOTE(sbauza): We also need to check the blazar:owner key
                                                                                              #  until we modify the reservation pool for including the
                                                                                              #  project_id key as for any other extra project
                                                                                              
                                                                                              # pool(Host Aggregates)的元数据包含有 Created Lease 的 Tenant ID
                                                                                              # e.g. blazar:owner='f47d5bee7acd48d286224f2c411a17fb'
                                                                                              owner = cfg.CONF['blazar:physical:host'].blazar_owner
                                                                                              owner_project_id = pool.metadata.get(owner)
                                                                                              if owner_project_id == spec_obj.project_id:
                                                                                                  return True
                                                                                              LOG.info(_("Unauthorized request to use Pool "
                                                                                                         "%(pool_id)s by tenant %(tenant_id)s"),
                                                                                                       {'pool_id': pool.name,
                                                                                                        'tenant_id': spec_obj.project_id})
                                                                                              return False
                                                                                          return False
                                                                                  
                                                                                      def fetch_blazar_pools(self, host_state):
                                                                                          # Get any reservation pools this host is part of
                                                                                          # Note this include possibly the freepool
                                                                                          # 获取 Host 处于的所有 Host Aggregate
                                                                                          aggregates = host_state.aggregates
                                                                                          pools = []
                                                                                          for agg in aggregates:
                                                                                              # 如果 Aggregate 的 AZ 以 blazar_ 或 blazar_ 开头,表示 Host 属于 Blazar freepool
                                                                                              if (str(agg.availability_zone).startswith(
                                                                                                      cfg.CONF['blazar:physical:host'].blazar_az_prefix)
                                                                                                      # NOTE(hiro-kobayashi): following 2 lines are for keeping
                                                                                                      # backward compatibility
                                                                                                      or str(agg.availability_zone).startswith('blazar:')):
                                                                                                  pools.append(agg)
                                                                                              # 如果 Aggregate 的 name 为 freepool,表示 Host 属于 Blazar freepool
                                                                                              if agg.name == (
                                                                                                      cfg.CONF['blazar:physical:host'].aggregate_freepool_name):
                                                                                                  pools.append(agg)
                                                                                  
                                                                                          return pools
                                                                                  

                                                                                  Blazar 的虚拟机资源预留(Instance Reservation)

                                                                                  Instance Reservation 的实现依赖于 Nova Host Aggregate 和 Server Group 的功能,所以需要先对两者有一个了解。

                                                                                  Host Aggregate

                                                                                  Host Aggregates 是在 Availability Zones 的基础上更进一步地进行逻辑的分组和隔离。例如我们可以根据不同的 Compute Nodes 的物理硬件配置(e.g. NUMA、SR-IOV、SSD Disk)将具有相同共性的物理资源规划在同一个 Host Aggregate 之下,或者根据用户的具体需求将几个 Compute Nodes 规划在具有相同用途(e.g. 高性能主机聚合)的同一个 Host Aggregate 之下,通过这样的划分有利于提高 Nova 主机资源调度的灵活性。

                                                                                  管理员可以给 Aggregate 设置 metadata,一个 Compute Host 可以属于多个 Aggregates,基于 Aggregate 的 metadata 可以实现高级调度功能。下面通过一个案例来感受。

                                                                                  Step 1. 在指定的 AZ 下创建 Host Aggregate,如果指定的 AZ 不存在,则自动创建一个。

                                                                                  # nova aggregate-create <ag_name> <az_name>
                                                                                  [root@control01 ~]# nova aggregate-create vmware-agg nova
                                                                                  +----+------------+-------------------+-------+--------------------------+--------------------------------------+
                                                                                  | Id | Name       | Availability Zone | Hosts | Metadata                 | UUID                                 |
                                                                                  +----+------------+-------------------+-------+--------------------------+--------------------------------------+
                                                                                  | 1  | vmware-agg | nova              |       | 'availability_zone=nova' | 154bb7b6-217e-45ef-8759-34d418cd9dbe |
                                                                                  +----+------------+-------------------+-------+--------------------------+--------------------------------------+
                                                                                  

                                                                                  Step 2. 设置指定的 Host Aggregate 的 metadata。

                                                                                  # nova aggregate-set-metadata <ag_name> <metadatas (key=value)>
                                                                                  [root@control01 ~]# nova aggregate-set-metadata vmware-agg DRIVE=esxi
                                                                                  Metadata has been successfully updated for aggregate 1.
                                                                                  +----+------------+-------------------+-------+----------------------------------------+--------------------------------------+
                                                                                  | Id | Name       | Availability Zone | Hosts | Metadata                               | UUID                                 |
                                                                                  +----+------------+-------------------+-------+----------------------------------------+--------------------------------------+
                                                                                  | 1  | vmware-agg | nova              |       | 'DRIVE=esxi', 'availability_zone=nova' | 154bb7b6-217e-45ef-8759-34d418cd9dbe |
                                                                                  +----+------------+-------------------+-------+----------------------------------------+--------------------------------------+
                                                                                  

                                                                                  NOTE:Nova 还专门预留了一些特殊字段的 Aggregate metadata 用于 Nova Scheduler,e.g. cpu_allocation_ratio、ram_allocation_ratio、disk_allocation_ratio 表示超配比,在创建虚拟机,nova-scheduler.service 会根据这些特殊的 metadata 作为过滤条件。前提是启用了 AggregateCoreFilter,AggregateRamFilter 和 AggregateDiskFilter 过滤器。否则使用全局 ratios,全局的 ratios 参数配置在 nova.conf 文件里。

                                                                                  Step 3. 设置 Flavor 的 extra_specs 参数,保证这些参数与 Host aggregate 的 metadata 参数能够匹配上(一致)。这样就能够 达到通过选定 Flavor 来选定虚拟机启动的 Host Aggregate 了。

                                                                                  [root@control01 ~]# nova flavor-key 9cb153ea-da73-47bc-86ab-dd1ae41bfed7 set "DRIVE=esxi"
                                                                                  
                                                                                  [root@control01 ~]# nova flavor-show 9cb153ea-da73-47bc-86ab-dd1ae41bfed7
                                                                                  +----------------------------+--------------------------------------+
                                                                                  | Property                   | Value                                |
                                                                                  +----------------------------+--------------------------------------+
                                                                                  | OS-FLV-DISABLED:disabled   | False                                |
                                                                                  | OS-FLV-EXT-DATA:ephemeral  | 0                                    |
                                                                                  | disk                       | 5                                    |
                                                                                  | extra_specs                | {"DRIVE": "esxi"}                    |
                                                                                  | id                         | 9cb153ea-da73-47bc-86ab-dd1ae41bfed7 |
                                                                                  | name                       | m1.xsmall                            |
                                                                                  | os-flavor-access:is_public | True                                 |
                                                                                  | ram                        | 1024                                 |
                                                                                  | rxtx_factor                | 1.0                                  |
                                                                                  | swap                       |                                      |
                                                                                  | vcpus                      | 1                                    |
                                                                                  +----------------------------+———————————————————+
                                                                                  

                                                                                  Step 4. 将 AggregateInstanceExtraSpecsFilter(自定义主机聚合过滤器)添加到所有 Nova Scheduler 的配置文件中。

                                                                                  enabled_filters =...,AggregateInstanceExtraSpecsFilter
                                                                                  

                                                                                  Step 5. 将 Compute Host 添加到 Host Aggregates 里。

                                                                                  [root@control01 ~]# nova aggregate-add-host vmware-agg aju-dev-vmware
                                                                                  Host aju-dev-vmware has been successfully added for aggregate 1
                                                                                  +----+------------+-------------------+------------------+----------------------------------------+--------------------------------------+
                                                                                  | Id | Name       | Availability Zone | Hosts            | Metadata                               | UUID                                 |
                                                                                  +----+------------+-------------------+------------------+----------------------------------------+--------------------------------------+
                                                                                  | 1  | vmware-agg | nova              | 'aju-dev-vmware' | 'DRIVE=esxi', 'availability_zone=nova' | 154bb7b6-217e-45ef-8759-34d418cd9dbe |
                                                                                  +----+------------+-------------------+------------------+----------------------------------------+———————————————————+
                                                                                  

                                                                                  Server Group

                                                                                  Server Group: From Icehouse version, OpenStack allows user to explicitly specify if he wants or not a group of virtual machines (VMs) to share the same hosts. These policies are called “Affinity” and “Anti-Affinity”.

                                                                                  • Affinity: the policy that forces Nova to hosts the concerned VMs in a same hypervisor.
                                                                                  • Anti-Affinity: the policy that forces Nova to hosts the concerned VMs each in a different hypervisor.

                                                                                  在这里插入图片描述

                                                                                  Step 1. Creating ServerGroup

                                                                                  nova server-group-create <group_name> [< policy >]
                                                                                  
                                                                                  • <group_name>: Name of the ServerGroup
                                                                                  • <policy>: Policy of the ServerGroup (“affinity” or “anti-affinity”)

                                                                                  Step 2. Create a virtual machine in a ServerGroup

                                                                                  nova boot  --image ubuntu --flavor m1.small --hint group=aa8a69f4-2567-47c6-a28c-2934a8c7959c vm1
                                                                                  
                                                                                  • --hint group=<group_id>:Group ID of the ServerGroup

                                                                                  Instance Reservation

                                                                                  Step 1. Add hosts into the freepool(这里不再重复)
                                                                                  Step 2. Create a lease

                                                                                  [root@localhost ~]# blazar lease-create --reservation resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1,affinity=False --start-date "2019-03-28 09:28" --end-date "2020-03-28 09:45" instance-lease-1
                                                                                  Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1', u'affinity', u'False')
                                                                                  Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20', u'amount', u'1')
                                                                                  Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024', u'disk_gb', u'20')
                                                                                  Matches: (u'resource_type=virtual:instance,vcpus=1', u'memory_mb', u'1024')
                                                                                  Matches: (u'resource_type=virtual:instance', u'vcpus', u'1')
                                                                                  Matches: (None, u'resource_type', u'virtual:instance')
                                                                                  Created a new lease:
                                                                                  +--------------+-----------------------------------------------------------------+
                                                                                  | Field        | Value                                                           |
                                                                                  +--------------+-----------------------------------------------------------------+
                                                                                  | created_at   | 2019-03-28 09:19:20                                             |
                                                                                  | degraded     | False                                                           |
                                                                                  | end_date     | 2020-03-28T09:45:00.000000                                      |
                                                                                  | events       | {                                                               |
                                                                                  |              |     "status": "UNDONE",                                         |
                                                                                  |              |     "lease_id": "5e6bced2-4031-4688-bdc4-df7e05ec24e6",         |
                                                                                  |              |     "event_type": "before_end_lease",                           |
                                                                                  |              |     "created_at": "2019-03-28 09:19:21",                        |
                                                                                  |              |     "updated_at": null,                                         |
                                                                                  |              |     "time": "2020-03-28T08:45:00.000000",                       |
                                                                                  |              |     "id": "6be9533e-c504-4d00-83c8-dc76b1d511c4"                |
                                                                                  |              | }                                                               |
                                                                                  |              | {                                                               |
                                                                                  |              |     "status": "UNDONE",                                         |
                                                                                  |              |     "lease_id": "5e6bced2-4031-4688-bdc4-df7e05ec24e6",         |
                                                                                  |              |     "event_type": "end_lease",                                  |
                                                                                  |              |     "created_at": "2019-03-28 09:19:21",                        |
                                                                                  |              |     "updated_at": null,                                         |
                                                                                  |              |     "time": "2020-03-28T09:45:00.000000",                       |
                                                                                  |              |     "id": "745b2d1b-c30f-4012-b781-4da3a51c8f79"                |
                                                                                  |              | }                                                               |
                                                                                  |              | {                                                               |
                                                                                  |              |     "status": "UNDONE",                                         |
                                                                                  |              |     "lease_id": "5e6bced2-4031-4688-bdc4-df7e05ec24e6",         |
                                                                                  |              |     "event_type": "start_lease",                                |
                                                                                  |              |     "created_at": "2019-03-28 09:19:21",                        |
                                                                                  |              |     "updated_at": null,                                         |
                                                                                  |              |     "time": "2019-03-28T09:28:00.000000",                       |
                                                                                  |              |     "id": "bd445bde-d8ce-4e0e-81dc-46c1e1ec29c7"                |
                                                                                  |              | }                                                               |
                                                                                  | id           | 5e6bced2-4031-4688-bdc4-df7e05ec24e6                            |
                                                                                  | name         | instance-lease-1                                                |
                                                                                  | project_id   | f47d5bee7acd48d286224f2c411a17fb                                |
                                                                                  | reservations | {                                                               |
                                                                                  |              |     "status": "pending",                                        |
                                                                                  |              |     "memory_mb": 1024,                                          |
                                                                                  |              |     "lease_id": "5e6bced2-4031-4688-bdc4-df7e05ec24e6",         |
                                                                                  |              |     "resource_properties": "",                                  |
                                                                                  |              |     "disk_gb": 20,                                              |
                                                                                  |              |     "resource_id": "122b6fe9-fc75-41df-bbaa-25890ed1896e",      |
                                                                                  |              |     "created_at": "2019-03-28 09:19:20",                        |
                                                                                  |              |     "updated_at": "2019-03-28 09:19:21",                        |
                                                                                  |              |     "missing_resources": false,                                 |
                                                                                  |              |     "server_group_id": "484c4291-6a7d-4ba0-8e22-422374f64e50",  |
                                                                                  |              |     "amount": 1,                                                |
                                                                                  |              |     "affinity": false,                                          |
                                                                                  |              |     "flavor_id": "67479531-7c37-4c12-97fa-ed56f9f45a55",        |
                                                                                  |              |     "id": "67479531-7c37-4c12-97fa-ed56f9f45a55",               |
                                                                                  |              |     "aggregate_id": 9,                                          |
                                                                                  |              |     "vcpus": 1,                                                 |
                                                                                  |              |     "resource_type": "virtual:instance",                        |
                                                                                  |              |     "resources_changed": false                                  |
                                                                                  |              | }                                                               |
                                                                                  | start_date   | 2019-03-28T09:28:00.000000                                      |
                                                                                  | status       | PENDING                                                         |
                                                                                  | trust_id     | fddf072c97ba46c88a76183f95a05dd1                                |
                                                                                  | updated_at   | 2019-03-28 09:19:21                                             |
                                                                                  | user_id      | e289dac11233451badb1409104a6505b                                |
                                                                                  +--------------+-----------------------------------------------------------------+
                                                                                  
                                                                                  • resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1,affinity=False:指定为申请 Instance Reservation 类型资源预留,Instance 的规格和数量。

                                                                                  NOTE:可以通过 affinity=False 来指定 Instances 之间是否启动 Server Group 亲和/反亲和。但现阶段支持 anti-affinity,否则触发错误 ERROR: Malformed parameter affinity (only affinity = False is supported)。所以如果你希望同时预留多个 instances,那么 freepool 就需要具有相应数量 Hosts。
                                                                                  在这里插入图片描述

                                                                                  创建 Instance Reservation Lease 之后,Blazar 会调用 Nova API 分别创建相应的 Host Aggregate、Server Group、Flavor 记录。e.g.

                                                                                  [root@localhost ~]# openstack aggregate show 67479531-7c37-4c12-97fa-ed56f9f45a55
                                                                                  +-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | Field             | Value                                                                                                                                                                                                        |
                                                                                  +-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | availability_zone | None                                                                                                                                                                                                         |
                                                                                  | created_at        | 2019-03-28T09:19:21.000000                                                                                                                                                                                   |
                                                                                  | deleted           | False                                                                                                                                                                                                        |
                                                                                  | deleted_at        | None                                                                                                                                                                                                         |
                                                                                  | hosts             | []                                                                                                                                                                                                           |
                                                                                  | id                | 9                                                                                                                                                                                                            |
                                                                                  | name              | 67479531-7c37-4c12-97fa-ed56f9f45a55                                                                                                                                                                         |
                                                                                  | properties        | affinity_id='484c4291-6a7d-4ba0-8e22-422374f64e50', blazar:owner='f47d5bee7acd48d286224f2c411a17fb', filter_tenant_id='f47d5bee7acd48d286224f2c411a17fb', reservation='67479531-7c37-4c12-97fa-ed56f9f45a55' |
                                                                                  | updated_at        | None                                                                                                                                                                                                         |
                                                                                  +-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack server group show 484c4291-6a7d-4ba0-8e22-422374f64e50
                                                                                  +----------+--------------------------------------------------+
                                                                                  | Field    | Value                                            |
                                                                                  +----------+--------------------------------------------------+
                                                                                  | id       | 484c4291-6a7d-4ba0-8e22-422374f64e50             |
                                                                                  | members  |                                                  |
                                                                                  | name     | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 |
                                                                                  | policies | anti-affinity                                    |
                                                                                  +----------+--------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack flavor show 67479531-7c37-4c12-97fa-ed56f9f45a55
                                                                                  +----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | Field                      | Value                                                                                                                                 |
                                                                                  +----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | OS-FLV-DISABLED:disabled   | False                                                                                                                                 |
                                                                                  | OS-FLV-EXT-DATA:ephemeral  | 0                                                                                                                                     |
                                                                                  | access_project_ids         |                                                                                                                                       |
                                                                                  | disk                       | 20                                                                                                                                    |
                                                                                  | id                         | 67479531-7c37-4c12-97fa-ed56f9f45a55                                                                                                  |
                                                                                  | name                       | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55                                                                                      |
                                                                                  | os-flavor-access:is_public | False                                                                                                                                 |
                                                                                  | properties                 | affinity_id='484c4291-6a7d-4ba0-8e22-422374f64e50', aggregate_instance_extra_specs:reservation='67479531-7c37-4c12-97fa-ed56f9f45a55' |
                                                                                  | ram                        | 1024                                                                                                                                  |
                                                                                  | rxtx_factor                | 1.0                                                                                                                                   |
                                                                                  | swap                       |                                                                                                                                       |
                                                                                  | vcpus                      | 1                                                                                                                                     |
                                                                                  +----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  

                                                                                  同样的,如果没有到 Lease instance-lease-1 租约时间之前使用 instance-lease-1 创建虚拟机会创建失败:

                                                                                  [root@localhost ~]# openstack server create --flavor 67479531-7c37-4c12-97fa-ed56f9f45a55 --image cirros-0.3.4-x86_64-disk --network private VM1
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  | Field                               | Value                                                                                   |
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  | OS-DCF:diskConfig                   | MANUAL                                                                                  |
                                                                                  | OS-EXT-AZ:availability_zone         |                                                                                         |
                                                                                  | OS-EXT-SRV-ATTR:host                | None                                                                                    |
                                                                                  | OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                                                    |
                                                                                  | OS-EXT-SRV-ATTR:instance_name       |                                                                                         |
                                                                                  | OS-EXT-STS:power_state              | NOSTATE                                                                                 |
                                                                                  | OS-EXT-STS:task_state               | scheduling                                                                              |
                                                                                  | OS-EXT-STS:vm_state                 | building                                                                                |
                                                                                  | OS-SRV-USG:launched_at              | None                                                                                    |
                                                                                  | OS-SRV-USG:terminated_at            | None                                                                                    |
                                                                                  | accessIPv4                          |                                                                                         |
                                                                                  | accessIPv6                          |                                                                                         |
                                                                                  | addresses                           |                                                                                         |
                                                                                  | adminPass                           | suypHUv8F92v                                                                            |
                                                                                  | config_drive                        |                                                                                         |
                                                                                  | created                             | 2019-03-28T09:26:11Z                                                                    |
                                                                                  | flavor                              | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 (67479531-7c37-4c12-97fa-ed56f9f45a55) |
                                                                                  | hostId                              |                                                                                         |
                                                                                  | id                                  | 4f324ba9-39b0-4890-b09c-6d1fcae25c61                                                    |
                                                                                  | image                               | cirros-0.3.4-x86_64-disk (0bdcd04f-2f77-42c3-ae61-5f10e2f1955c)                         |
                                                                                  | key_name                            | None                                                                                    |
                                                                                  | name                                | VM1                                                                                     |
                                                                                  | progress                            | 0                                                                                       |
                                                                                  | project_id                          | f47d5bee7acd48d286224f2c411a17fb                                                        |
                                                                                  | properties                          |                                                                                         |
                                                                                  | security_groups                     | name='default'                                                                          |
                                                                                  | status                              | BUILD                                                                                   |
                                                                                  | updated                             | 2019-03-28T09:26:12Z                                                                    |
                                                                                  | user_id                             | e289dac11233451badb1409104a6505b                                                        |
                                                                                  | volumes_attached                    |                                                                                         |
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  [root@localhost ~]# openstack server list
                                                                                  +--------------------------------------+------+--------+----------+--------------------------+--------------------------------------------------+
                                                                                  | ID                                   | Name | Status | Networks | Image                    | Flavor                                           |
                                                                                  +--------------------------------------+------+--------+----------+--------------------------+--------------------------------------------------+
                                                                                  | 4f324ba9-39b0-4890-b09c-6d1fcae25c61 | VM1  | ERROR  |          | cirros-0.3.4-x86_64-disk | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 |
                                                                                  +--------------------------------------+------+--------+----------+--------------------------+--------------------------------------------------+
                                                                                  
                                                                                  • --flavor 67479531-7c37-4c12-97fa-ed56f9f45a55 通过 Lease 自动生成的 Flavor 来创建虚拟机。

                                                                                  nova-scheduler.service 错误日志:

                                                                                  (localhost.localdomain, localhost.localdomain) ram: 40447MB disk: 177152MB io_ops: 0 instances: 0 fails instance_type extra_specs requirements. Extra_spec reservation is not in aggregate.
                                                                                  Filter AggregateInstanceExtraSpecsFilter returned 0 hosts
                                                                                  

                                                                                  根本的原因还是因为 Lease Flavor 匹配的 Aggregate 67479531-7c37-4c12-97fa-ed56f9f45a55 中并没有 hosts。等到了 Lease instance-lease-1 的租约时间后,再执行一次创建:

                                                                                  [root@localhost ~]# openstack server create --flavor 67479531-7c37-4c12-97fa-ed56f9f45a55 --image cirros-0.3.4-x86_64-disk --network private VM2
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  | Field                               | Value                                                                                   |
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  | OS-DCF:diskConfig                   | MANUAL                                                                                  |
                                                                                  | OS-EXT-AZ:availability_zone         |                                                                                         |
                                                                                  | OS-EXT-SRV-ATTR:host                | None                                                                                    |
                                                                                  | OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                                                    |
                                                                                  | OS-EXT-SRV-ATTR:instance_name       |                                                                                         |
                                                                                  | OS-EXT-STS:power_state              | NOSTATE                                                                                 |
                                                                                  | OS-EXT-STS:task_state               | scheduling                                                                              |
                                                                                  | OS-EXT-STS:vm_state                 | building                                                                                |
                                                                                  | OS-SRV-USG:launched_at              | None                                                                                    |
                                                                                  | OS-SRV-USG:terminated_at            | None                                                                                    |
                                                                                  | accessIPv4                          |                                                                                         |
                                                                                  | accessIPv6                          |                                                                                         |
                                                                                  | addresses                           |                                                                                         |
                                                                                  | adminPass                           | NdBfrWUp47wx                                                                            |
                                                                                  | config_drive                        |                                                                                         |
                                                                                  | created                             | 2019-03-28T09:36:34Z                                                                    |
                                                                                  | flavor                              | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 (67479531-7c37-4c12-97fa-ed56f9f45a55) |
                                                                                  | hostId                              |                                                                                         |
                                                                                  | id                                  | 267e6621-b085-4d58-81e6-138c190a45cb                                                    |
                                                                                  | image                               | cirros-0.3.4-x86_64-disk (0bdcd04f-2f77-42c3-ae61-5f10e2f1955c)                         |
                                                                                  | key_name                            | None                                                                                    |
                                                                                  | name                                | VM2                                                                                     |
                                                                                  | progress                            | 0                                                                                       |
                                                                                  | project_id                          | f47d5bee7acd48d286224f2c411a17fb                                                        |
                                                                                  | properties                          |                                                                                         |
                                                                                  | security_groups                     | name='default'                                                                          |
                                                                                  | status                              | BUILD                                                                                   |
                                                                                  | updated                             | 2019-03-28T09:36:35Z                                                                    |
                                                                                  | user_id                             | e289dac11233451badb1409104a6505b                                                        |
                                                                                  | volumes_attached                    |                                                                                         |
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack server list
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
                                                                                  | ID                                   | Name | Status | Networks             | Image                    | Flavor                                           |
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
                                                                                  | 267e6621-b085-4d58-81e6-138c190a45cb | VM2  | ACTIVE | private=192.168.1.55 | cirros-0.3.4-x86_64-disk | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 |
                                                                                  | 4f324ba9-39b0-4890-b09c-6d1fcae25c61 | VM1  | ERROR  |                      | cirros-0.3.4-x86_64-disk | reservation:67479531-7c37-4c12-97fa-ed56f9f45a55 |
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
                                                                                  

                                                                                  创建成功!因为此时的 Aggregate 67479531-7c37-4c12-97fa-ed56f9f45a55 已经有 hosts 了:

                                                                                  [root@localhost ~]# openstack aggregate show 67479531-7c37-4c12-97fa-ed56f9f45a55
                                                                                  +-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | Field             | Value                                                                                                                                                                                                        |
                                                                                  +-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | availability_zone | None                                                                                                                                                                                                         |
                                                                                  | created_at        | 2019-03-28T09:19:21.000000                                                                                                                                                                                   |
                                                                                  | deleted           | False                                                                                                                                                                                                        |
                                                                                  | deleted_at        | None                                                                                                                                                                                                         |
                                                                                  | hosts             | [u'localhost.localdomain']                                                                                                                                                                                   |
                                                                                  | id                | 9                                                                                                                                                                                                            |
                                                                                  | name              | 67479531-7c37-4c12-97fa-ed56f9f45a55                                                                                                                                                                         |
                                                                                  | properties        | affinity_id='484c4291-6a7d-4ba0-8e22-422374f64e50', blazar:owner='f47d5bee7acd48d286224f2c411a17fb', filter_tenant_id='f47d5bee7acd48d286224f2c411a17fb', reservation='67479531-7c37-4c12-97fa-ed56f9f45a55' |
                                                                                  | updated_at        | None                                                                                                                                                                                                         |
                                                                                  +-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  

                                                                                  这是因为 blazar-manager 监控 Lease instance-lease-1 的租期之间到了,所以调用 Nova API 从 Aggregate freepool 中选择一个资源满足的 Host 加入到 Lease instance-lease-1 对应的 Aggregate 中。

                                                                                  同样的,当 Lease 租约到期之后,blazar-manager 就会自动进行资源回收,将于 Lease 相关的(成功或失败)Instances、Server Group、Host Aggregate、Flavor 全部删除。

                                                                                  代码实现:

                                                                                  # /opt/stack/nova/nova/scheduler/filters/blazar_filter.py
                                                                                  ...
                                                                                  FLAVOR_EXTRA_SPEC = "aggregate_instance_extra_specs:reservation"
                                                                                  ...
                                                                                  
                                                                                  class BlazarFilter(filters.BaseHostFilter):
                                                                                       """Blazar Filter for nova-scheduler."""
                                                                                       ...
                                                                                      def host_passes(self, host_state, spec_obj):
                                                                                          ...
                                                                                          
                                                                                          # the request is instance reservation
                                                                                          if FLAVOR_EXTRA_SPEC in spec_obj.flavor.extra_specs.keys():
                                                                                              # Scheduling requests for instance reservation are processed by
                                                                                              # other Nova filters: AggregateInstanceExtraSpecsFilter,
                                                                                              # AggregateMultiTenancyIsolation, and
                                                                                              # ServerGroupAntiAffinityFilter. What BlazarFilter needs to
                                                                                              # do is just pass the host if the request has an instance
                                                                                              # reservation key.
                                                                                              return True
                                                                                  

                                                                                  对于 Instance Reservation Lease 而言 BlazarFilter 只是单纯的判断一下 Instance Flavor 是否具有 Blazar 标签 aggregate_instance_extra_specs:reservation 而已,当然在 BlazarFilter 之前,还经过了以下 Filters:

                                                                                  • AggregateInstanceExtraSpecsFilter:第一确保 Flavor 具有 aggregate_instance_extra_specs:reservation='16f3c0be-f069-4f1a-ab76-cab94833ad7a' 扩展属性;第二确保 Flavor 对应的 Host Aggregate 内含有 hosts。因为如果 Host Aggregate 没有 hosts 的话 metadata = utils.aggregate_metadata_get_by_host(host_state) 返回的 metadata 是一个空字典,无法匹配到 key:reservation
                                                                                  • AggregateMultiTenancyIsolation:确保创建预留的租户和使用预留的租户一致。
                                                                                  • ServerGroupAntiAffinityFilter:确保在 freepool 中不同的 Hosts 上安排 Instances。

                                                                                  Blazar 的设计思路与实现原理

                                                                                  整体来看,Blazar 的设计思路是基于资源隔离来实现的资源预留。Blazar 通过应用 Nova Host Aggregate 和 Nova Scheduler Filters 机制将 OpenStack 计算资源(以 Compute Node 为单位)划分成 “Nova 原生计算资源” 和 “Blazar 预留计算资源”。

                                                                                  • Nova 原生计算资源:原生的资源管理系统,由 Nova Compute 维护。
                                                                                  • Blazar 预留计算资源:Blazar 自身实现了一套基于 “资源租赁机制” 的资源管理系统。

                                                                                  Scheduling mechanism in Nova

                                                                                  Blazar 按照下列算法来保证 “预留资源” 调度的可靠性

                                                                                  1. non-reserved instances: scheduled to hypervisors which are outside of both the freepool aggregate and reservation-related aggregates.
                                                                                  2. instances related to host reservation: scheduled to hypervisors which are inside the reservation-related aggregate. The hypervisors are not included in the freepool aggregate.
                                                                                  3. instances related to instance reservation: scheduled to hypervisors which are inside the reservation-related aggregate. The hypervisors are included in the freepool aggregate.

                                                                                  Nova filters used by Blazar choose hypervisors with the following rules:

                                                                                  • AggregateInstanceExtraSpecsFilter picks up hypervisors from the aggregate related to an instance reservation based on extra_specs of the flavor, if the request is related to instance reservation. If not, the filter picks up hypervisors from neither reservation-related aggregates nor the freepool.
                                                                                  • BlazarFilter picks up hypervisors from the aggregate related to a host reservation based on the ‘reservation’ scheduler hint, if the request is related to host reservation. If not, the filter picks up hypervisors from neither host reservation-related aggregates nor the freepool.
                                                                                  • AggregateMultiTenancyIsolationFilter blocks requests to be scheduled to the freepool by users who do not have active reservation.
                                                                                  • Combination of AggregateInstanceExtraSpecsFilter and AggregateMultiTenancyIsolationFilter enables requests using instance reservation to be scheduled in the corresponding aggregate.
                                                                                  • ServerGroupAntiAffinityFilter ensures instance reservation related instances are spread on different hypervisors.

                                                                                  Blazar Pike 的局限

                                                                                  NOTE:以上分析适用于 Pike, Queens,Rocky 版本

                                                                                  1. Instance Reservation 要依赖 Host Reservation,并没有实现真正 意义上纯粹的虚拟机预留。
                                                                                  2. Instance Reservation 不支持 Affinity Server Group。
                                                                                  3. Instance Reservation 只支持 CPU、RAM、Disk 资源类型,不支持 NUMA、SR-IOV 等高级资源类型。

                                                                                  Blazar 的未来

                                                                                  现在 Blazar 的实现是基于 Nova Scheduler Filters 机制来实现的,正如 Nova Scheduler 会被 Placement 替代一样,Blazar 最新的实现方式也正过渡到依赖 Placement 的 Resource Class。Blazar Instance reservation supports both affinity rule and anti-affinity rule. 的工作正在进行中。

                                                                                  The affinity rule reservation allows other instances or reservation to use unused instance slots in reserved hypervisors. The Nova team is developing placement API. The API already has custom resource classes and is now implementing a scheduler function[3] that uses custom resources classes. It enables operator to more efficiently manage hypervisors in the freepool.

                                                                                  当 Blazar 迁移至 Placement 之后,其调度过程将会变成

                                                                                  1. A tenant user creates their reservation, in term of Blazar called “lease”, with a time frame, the instance size, and how many instances.
                                                                                  2. Blazar checks whether the reservation is acceptable during the time frame or not. If acceptable, Blazar records the reservation request in its database and updates the usage of hypervisor in freepool. Then Blazar returns the reservation id. If not, Blazar responds the reservation is not acceptable.
                                                                                  3. At the start time of the reservation, Blazar creates a custom resource class, a flavor, and a resource provider of the custom resource class.
                                                                                  4. The user creates reserved instances with the newly created flavor.

                                                                                  详细请看:https://docs.openstack.org/blazar/latest/specs/pike/new-instance-reservation.html

                                                                                  Blazar Stein: Placement API support for instance reservation

                                                                                  这个功能虽然是 Rocky 版本被提出(Introduced),但仅限于单元测试调用,直到 Stein 才被引入到主干程序逻辑。所以如果希望使用该功能,还是要部署 Stein 版本的 Blazar 代码库。

                                                                                  使用 Placement API 的 Instance Reservation 可以实现

                                                                                  • A user wants to reserve instance resources with arbitrary affinity policy.
                                                                                  • A user wants to reserve more instances than the number of hosts.

                                                                                  使用 Placement API 的 Instance Reservation 带来以下改变

                                                                                  • A host aggregate is not created for each instance reservation anymore because reserved hosts can be distinguished by the reservation provider inventory.
                                                                                  • A server group is not created anymore because the proposed approach does not depend on the ServerGroup(Anti)AffinityFilter.

                                                                                  详细的设计思路可浏览:https://docs.openstack.org/blazar/latest/specs/rocky/placement-api.html

                                                                                  Creating a host

                                                                                  创建一条定义为 nested resource provider 的 Resource Provider,又称为 reservation provider。

                                                                                  API:POST /placement/resource_providers
                                                                                  Body:

                                                                                  {
                                                                                      "name": "blazar_compute-1",
                                                                                      "parent_provider_uuid": "{compute node resource provider uuid}"
                                                                                  }
                                                                                  

                                                                                  CLI:

                                                                                  [root@localhost ~]# openstack resource provider list
                                                                                  +--------------------------------------+------------------------------+------------+
                                                                                  | uuid                                 | name                         | generation |
                                                                                  +--------------------------------------+------------------------------+------------+
                                                                                  | 88c5326f-68a9-490b-8298-f69dbe03b453 | localhost.localdomain        |          3 |
                                                                                  | 3aa43b09-a7b9-4f21-ab01-a15cb29d1a8c | blazar_localhost.localdomain |          0 |
                                                                                  +--------------------------------------+------------------------------+------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack resource provider inventory list 3aa43b09-a7b9-4f21-ab01-a15cb29d1a8c
                                                                                  
                                                                                  

                                                                                  Creating a lease

                                                                                  API:POST /placement/resource_classes
                                                                                  Body:

                                                                                  {
                                                                                      "name": "CUSTOM_RESERVATION_4D17D41A_830D_47B2_91C7_4F9FC0AE611E"
                                                                                  }
                                                                                  

                                                                                  CLI:

                                                                                  [root@localhost ~]# blazar lease-create --reservation resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1,affinity=True --start-date "2019-03-29 03:55" --end-date "2020-03-29 04:10" instance-lease-1
                                                                                  Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20,amount=1', u'affinity', u'True')
                                                                                  Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024,disk_gb=20', u'amount', u'1')
                                                                                  Matches: (u'resource_type=virtual:instance,vcpus=1,memory_mb=1024', u'disk_gb', u'20')
                                                                                  Matches: (u'resource_type=virtual:instance,vcpus=1', u'memory_mb', u'1024')
                                                                                  Matches: (u'resource_type=virtual:instance', u'vcpus', u'1')
                                                                                  Matches: (None, u'resource_type', u'virtual:instance')
                                                                                  Created a new lease:
                                                                                  +--------------+-----------------------------------------------------------------+
                                                                                  | Field        | Value                                                           |
                                                                                  +--------------+-----------------------------------------------------------------+
                                                                                  | created_at   | 2019-03-29 03:54:08                                             |
                                                                                  | degraded     | False                                                           |
                                                                                  | end_date     | 2020-03-29T04:10:00.000000                                      |
                                                                                  | events       | {                                                               |
                                                                                  |              |     "status": "UNDONE",                                         |
                                                                                  |              |     "lease_id": "6a178427-ae5c-4ac5-a1dc-b69e6207166a",         |
                                                                                  |              |     "event_type": "before_end_lease",                           |
                                                                                  |              |     "created_at": "2019-03-29 03:54:15",                        |
                                                                                  |              |     "updated_at": null,                                         |
                                                                                  |              |     "time": "2020-03-29T03:10:00.000000",                       |
                                                                                  |              |     "id": "994c8a58-25b6-4e0f-982e-d92ad08e59cc"                |
                                                                                  |              | }                                                               |
                                                                                  |              | {                                                               |
                                                                                  |              |     "status": "UNDONE",                                         |
                                                                                  |              |     "lease_id": "6a178427-ae5c-4ac5-a1dc-b69e6207166a",         |
                                                                                  |              |     "event_type": "end_lease",                                  |
                                                                                  |              |     "created_at": "2019-03-29 03:54:15",                        |
                                                                                  |              |     "updated_at": null,                                         |
                                                                                  |              |     "time": "2020-03-29T04:10:00.000000",                       |
                                                                                  |              |     "id": "d33fd540-e0bd-44bc-903b-1e04f63b3fe4"                |
                                                                                  |              | }                                                               |
                                                                                  |              | {                                                               |
                                                                                  |              |     "status": "UNDONE",                                         |
                                                                                  |              |     "lease_id": "6a178427-ae5c-4ac5-a1dc-b69e6207166a",         |
                                                                                  |              |     "event_type": "start_lease",                                |
                                                                                  |              |     "created_at": "2019-03-29 03:54:15",                        |
                                                                                  |              |     "updated_at": null,                                         |
                                                                                  |              |     "time": "2019-03-29T03:55:00.000000",                       |
                                                                                  |              |     "id": "d4936d4e-d797-4654-ba30-610c82ab5683"                |
                                                                                  |              | }                                                               |
                                                                                  | id           | 6a178427-ae5c-4ac5-a1dc-b69e6207166a                            |
                                                                                  | name         | instance-lease-1                                                |
                                                                                  | project_id   | 9e782daaa46c4a7bb5ffda88160993c8                                |
                                                                                  | reservations | {                                                               |
                                                                                  |              |     "status": "pending",                                        |
                                                                                  |              |     "memory_mb": 1024,                                          |
                                                                                  |              |     "lease_id": "6a178427-ae5c-4ac5-a1dc-b69e6207166a",         |
                                                                                  |              |     "resource_properties": "",                                  |
                                                                                  |              |     "disk_gb": 20,                                              |
                                                                                  |              |     "resource_id": "b9bbb153-e661-49cb-8305-3f4ae748a1fb",      |
                                                                                  |              |     "created_at": "2019-03-29 03:54:08",                        |
                                                                                  |              |     "updated_at": "2019-03-29 03:54:15",                        |
                                                                                  |              |     "missing_resources": false,                                 |
                                                                                  |              |     "server_group_id": "96a89cb4-5a7e-4546-9b0f-90dc3f599fe3",  |
                                                                                  |              |     "amount": 1,                                                |
                                                                                  |              |     "affinity": true,                                           |
                                                                                  |              |     "flavor_id": "abeaea5c-b14f-48df-ae46-aa56b55f61fc",        |
                                                                                  |              |     "id": "abeaea5c-b14f-48df-ae46-aa56b55f61fc",               |
                                                                                  |              |     "aggregate_id": 2,                                          |
                                                                                  |              |     "vcpus": 1,                                                 |
                                                                                  |              |     "resource_type": "virtual:instance",                        |
                                                                                  |              |     "resources_changed": false                                  |
                                                                                  |              | }                                                               |
                                                                                  | start_date   | 2019-03-29T03:55:00.000000                                      |
                                                                                  | status       | PENDING                                                         |
                                                                                  | trust_id     | 00e5c6c14e794af49f25489d8136c872                                |
                                                                                  | updated_at   | 2019-03-29 03:54:15                                             |
                                                                                  | user_id      | 8a4b574706964edf85d2cc46b3bbef26                                |
                                                                                  +--------------+-----------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack resource class list
                                                                                  +---------------------------------------------------------+
                                                                                  | name                                                    |
                                                                                  +---------------------------------------------------------+
                                                                                  | VCPU                                                    |
                                                                                  | MEMORY_MB                                               |
                                                                                  | DISK_GB                                                 |
                                                                                  | PCI_DEVICE                                              |
                                                                                  | SRIOV_NET_VF                                            |
                                                                                  | NUMA_SOCKET                                             |
                                                                                  | NUMA_CORE                                               |
                                                                                  | NUMA_THREAD                                             |
                                                                                  | NUMA_MEMORY_MB                                          |
                                                                                  | IPV4_ADDRESS                                            |
                                                                                  | VGPU                                                    |
                                                                                  | VGPU_DISPLAY_HEAD                                       |
                                                                                  | NET_BW_EGR_KILOBIT_PER_SEC                              |
                                                                                  | NET_BW_IGR_KILOBIT_PER_SEC                              |
                                                                                  | PCPU                                                    |
                                                                                  | CUSTOM_RESERVATION_ABEAEA5C_B14F_48DF_AE46_AA56B55F61FC |
                                                                                  +---------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack flavor show abeaea5c-b14f-48df-ae46-aa56b55f61fc
                                                                                  +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | Field                      | Value                                                                                                                                                                                                        |
                                                                                  +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | OS-FLV-DISABLED:disabled   | False                                                                                                                                                                                                        |
                                                                                  | OS-FLV-EXT-DATA:ephemeral  | 0                                                                                                                                                                                                            |
                                                                                  | access_project_ids         | 9e782daaa46c4a7bb5ffda88160993c8                                                                                                                                                                             |
                                                                                  | disk                       | 20                                                                                                                                                                                                           |
                                                                                  | id                         | abeaea5c-b14f-48df-ae46-aa56b55f61fc                                                                                                                                                                         |
                                                                                  | name                       | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc                                                                                                                                                             |
                                                                                  | os-flavor-access:is_public | False                                                                                                                                                                                                        |
                                                                                  | properties                 | affinity_id='96a89cb4-5a7e-4546-9b0f-90dc3f599fe3', aggregate_instance_extra_specs:reservation='abeaea5c-b14f-48df-ae46-aa56b55f61fc', resources:CUSTOM_RESERVATION_ABEAEA5C_B14F_48DF_AE46_AA56B55F61FC='1' |
                                                                                  | ram                        | 1024                                                                                                                                                                                                         |
                                                                                  | rxtx_factor                | 1.0                                                                                                                                                                                                          |
                                                                                  | swap                       |                                                                                                                                                                                                              |
                                                                                  | vcpus                      | 1                                                                                                                                                                                                            |
                                                                                  +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack server group show 96a89cb4-5a7e-4546-9b0f-90dc3f599fe3
                                                                                  +----------+--------------------------------------------------+
                                                                                  | Field    | Value                                            |
                                                                                  +----------+--------------------------------------------------+
                                                                                  | id       | 96a89cb4-5a7e-4546-9b0f-90dc3f599fe3             |
                                                                                  | members  |                                                  |
                                                                                  | name     | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc |
                                                                                  | policies | affinity                                         |
                                                                                  +----------+--------------------------------------------------+
                                                                                  

                                                                                  Starting a lease

                                                                                  API:PUT /placement/resource_providers/{reservation_provider_uuid}/inventories
                                                                                  Body:

                                                                                  {
                                                                                      "inventories": {
                                                                                          "CUSTOM_RESERVATION_4D17D41A_830D_47B2_91C7_4F9FC0AE611E": {
                                                                                              "total": 3,
                                                                                              "allocation_ratio": 1.0,
                                                                                              "min_unit": 1,
                                                                                              "max_unit": 1,
                                                                                              "step_size": 1
                                                                                          },
                                                                                          "snip"
                                                                                      },
                                                                                      "resource_provider_generation": 5
                                                                                  }
                                                                                  

                                                                                  CLI:

                                                                                  [root@localhost ~]# openstack resource provider inventory list 3aa43b09-a7b9-4f21-ab01-a15cb29d1a8c
                                                                                  +---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
                                                                                  | resource_class                                          | allocation_ratio | max_unit | reserved | step_size | min_unit | total |
                                                                                  +---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
                                                                                  | CUSTOM_RESERVATION_ABEAEA5C_B14F_48DF_AE46_AA56B55F61FC |              1.0 |        1 |        0 |         1 |        1 |     1 |
                                                                                  +---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
                                                                                  
                                                                                  [root@localhost ~]# openstack aggregate show 2
                                                                                  +-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | Field             | Value                                                                                                                                                                                                        |
                                                                                  +-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | availability_zone | None                                                                                                                                                                                                         |
                                                                                  | created_at        | 2019-03-29T03:54:13.000000                                                                                                                                                                                   |
                                                                                  | deleted           | False                                                                                                                                                                                                        |
                                                                                  | deleted_at        | None                                                                                                                                                                                                         |
                                                                                  | hosts             | [u'localhost.localdomain']                                                                                                                                                                                   |
                                                                                  | id                | 2                                                                                                                                                                                                            |
                                                                                  | name              | abeaea5c-b14f-48df-ae46-aa56b55f61fc                                                                                                                                                                         |
                                                                                  | properties        | affinity_id='96a89cb4-5a7e-4546-9b0f-90dc3f599fe3', blazar:owner='9e782daaa46c4a7bb5ffda88160993c8', filter_tenant_id='9e782daaa46c4a7bb5ffda88160993c8', reservation='abeaea5c-b14f-48df-ae46-aa56b55f61fc' |
                                                                                  | updated_at        | None                                                                                                                                                                                                         |
                                                                                  +-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  

                                                                                  Launching instances

                                                                                  [root@localhost ~]# openstack flavor show abeaea5c-b14f-48df-ae46-aa56b55f61fc
                                                                                  +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | Field                      | Value                                                                                                                                                                                                        |
                                                                                  +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  | OS-FLV-DISABLED:disabled   | False                                                                                                                                                                                                        |
                                                                                  | OS-FLV-EXT-DATA:ephemeral  | 0                                                                                                                                                                                                            |
                                                                                  | access_project_ids         | 9e782daaa46c4a7bb5ffda88160993c8                                                                                                                                                                             |
                                                                                  | disk                       | 20                                                                                                                                                                                                           |
                                                                                  | id                         | abeaea5c-b14f-48df-ae46-aa56b55f61fc                                                                                                                                                                         |
                                                                                  | name                       | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc                                                                                                                                                             |
                                                                                  | os-flavor-access:is_public | False                                                                                                                                                                                                        |
                                                                                  | properties                 | affinity_id='96a89cb4-5a7e-4546-9b0f-90dc3f599fe3', aggregate_instance_extra_specs:reservation='abeaea5c-b14f-48df-ae46-aa56b55f61fc', resources:CUSTOM_RESERVATION_ABEAEA5C_B14F_48DF_AE46_AA56B55F61FC='1' |
                                                                                  | ram                        | 1024                                                                                                                                                                                                         |
                                                                                  | rxtx_factor                | 1.0                                                                                                                                                                                                          |
                                                                                  | swap                       |                                                                                                                                                                                                              |
                                                                                  | vcpus                      | 1                                                                                                                                                                                                            |
                                                                                  +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack server create --flavor abeaea5c-b14f-48df-ae46-aa56b55f61fc --image f2029152-4af3-460d-acf0-4798808753c0 --network 54726ee2-38c2-448f-ad88-06e4058f573a VM1
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  | Field                               | Value                                                                                   |
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  | OS-DCF:diskConfig                   | MANUAL                                                                                  |
                                                                                  | OS-EXT-AZ:availability_zone         |                                                                                         |
                                                                                  | OS-EXT-SRV-ATTR:host                | None                                                                                    |
                                                                                  | OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                                                    |
                                                                                  | OS-EXT-SRV-ATTR:instance_name       |                                                                                         |
                                                                                  | OS-EXT-STS:power_state              | NOSTATE                                                                                 |
                                                                                  | OS-EXT-STS:task_state               | scheduling                                                                              |
                                                                                  | OS-EXT-STS:vm_state                 | building                                                                                |
                                                                                  | OS-SRV-USG:launched_at              | None                                                                                    |
                                                                                  | OS-SRV-USG:terminated_at            | None                                                                                    |
                                                                                  | accessIPv4                          |                                                                                         |
                                                                                  | accessIPv6                          |                                                                                         |
                                                                                  | addresses                           |                                                                                         |
                                                                                  | adminPass                           | zNKCQVoSe3W2                                                                            |
                                                                                  | config_drive                        |                                                                                         |
                                                                                  | created                             | 2019-03-29T04:55:26Z                                                                    |
                                                                                  | flavor                              | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc (abeaea5c-b14f-48df-ae46-aa56b55f61fc) |
                                                                                  | hostId                              |                                                                                         |
                                                                                  | id                                  | 65203ec7-ba30-45af-b2db-8648f9e10f2a                                                    |
                                                                                  | image                               | cirros-0.3.4-x86_64-disk (f2029152-4af3-460d-acf0-4798808753c0)                         |
                                                                                  | key_name                            | None                                                                                    |
                                                                                  | name                                | VM1                                                                                     |
                                                                                  | progress                            | 0                                                                                       |
                                                                                  | project_id                          | 9e782daaa46c4a7bb5ffda88160993c8                                                        |
                                                                                  | properties                          |                                                                                         |
                                                                                  | security_groups                     | name='default'                                                                          |
                                                                                  | status                              | BUILD                                                                                   |
                                                                                  | updated                             | 2019-03-29T04:55:26Z                                                                    |
                                                                                  | user_id                             | 8a4b574706964edf85d2cc46b3bbef26                                                        |
                                                                                  | volumes_attached                    |                                                                                         |
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack server list
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
                                                                                  | ID                                   | Name | Status | Networks             | Image                    | Flavor                                           |
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
                                                                                  | 65203ec7-ba30-45af-b2db-8648f9e10f2a | VM1  | ACTIVE | private=192.168.1.27 | cirros-0.3.4-x86_64-disk | reservation:abeaea5c-b14f-48df-ae46-aa56b55f61fc |
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
                                                                                  

                                                                                  Launching instances 的实现逻辑

                                                                                  1. 存在 Nested Resource Provider:blazar_{hypervisor}
                                                                                  [root@localhost ~]# openstack resource provider list
                                                                                  +--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
                                                                                  | uuid                                 | name                         | generation | root_provider_uuid                   | parent_provider_uuid                 |
                                                                                  +--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
                                                                                  | 7b155336-faaa-4e39-a51b-851ae7495ac5 | localhost.localdomain        |          4 | 7b155336-faaa-4e39-a51b-851ae7495ac5 | None                                 |
                                                                                  | 88f75d49-7512-4969-981f-fc893d8d552f | blazar_localhost.localdomain |          1 | 7b155336-faaa-4e39-a51b-851ae7495ac5 | 7b155336-faaa-4e39-a51b-851ae7495ac5 |
                                                                                  +--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
                                                                                  
                                                                                  1. Nested Resource Provider 存在有 CUSTOM_RESERVATION_{reservation_flavor_uuid},到达 lease start time,Blazar 会自动将 CUSTOM_RESERVATION_{reservation_flavor_uuid} 添加到 Nested Resource Provider。
                                                                                  [root@localhost ~]# openstack resource provider inventory list 88f75d49-7512-4969-981f-fc893d8d552f
                                                                                  +---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
                                                                                  | resource_class                                          | allocation_ratio | max_unit | reserved | step_size | min_unit | total |
                                                                                  +---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
                                                                                  | CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB |              1.0 |        1 |        0 |         1 |        1 |     1 |
                                                                                  +---------------------------------------------------------+------------------+----------+----------+-----------+----------+-------+
                                                                                  
                                                                                  1. Nova Scheduler 调用 Placement API 获取 allocation candidate。
                                                                                  [root@localhost ~]# openstack allocation candidate list --resource CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB=1
                                                                                  +---+-----------------------------------------------------------+--------------------------------------+-------------------------------------------------------------+--------+
                                                                                  | # | allocation                                                | resource provider                    | inventory used/capacity                                     | traits |
                                                                                  +---+-----------------------------------------------------------+--------------------------------------+-------------------------------------------------------------+--------+
                                                                                  | 1 | CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB=1 | 88f75d49-7512-4969-981f-fc893d8d552f | CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB=0/1 |        |
                                                                                  +---+-----------------------------------------------------------+--------------------------------------+-------------------------------------------------------------+--------+
                                                                                  
                                                                                  [root@localhost ~]# openstack resource provider usage show 88f75d49-7512-4969-981f-fc893d8d552f
                                                                                  +---------------------------------------------------------+-------+
                                                                                  | resource_class                                          | usage |
                                                                                  +---------------------------------------------------------+-------+
                                                                                  | CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB |     0 |
                                                                                  +---------------------------------------------------------+-------+
                                                                                  

                                                                                  NOTE:对于普通的虚拟机而言,BlazarFilter 限制不能创建在 Host Aggregate freepool 包含的 Hosts 上。

                                                                                  1. 实际启动一个虚拟机后,Nested Resource Provider 的资源(CUSTOM_RESERVATION_{reservation_flavor_uuid})被扣除,Nested Resource Provider 的 Parent Provider 即实际的 Hypervisor Resource Provider 的资源(CPU、RAM、DISK)被扣除。
                                                                                  [root@localhost ~]# openstack server create --flavor 9259448d-88bc-45d3-a106-e56f7016f2ab --image c4b1cc17-33f6-4964-933a-810fb81f581b --network 0b97b93b-3c55-4ef2-b8d8-ac4065a66c01 VM1
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  | Field                               | Value                                                                                   |
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  | OS-DCF:diskConfig                   | MANUAL                                                                                  |
                                                                                  | OS-EXT-AZ:availability_zone         |                                                                                         |
                                                                                  | OS-EXT-SRV-ATTR:host                | None                                                                                    |
                                                                                  | OS-EXT-SRV-ATTR:hypervisor_hostname | None                                                                                    |
                                                                                  | OS-EXT-SRV-ATTR:instance_name       |                                                                                         |
                                                                                  | OS-EXT-STS:power_state              | NOSTATE                                                                                 |
                                                                                  | OS-EXT-STS:task_state               | scheduling                                                                              |
                                                                                  | OS-EXT-STS:vm_state                 | building                                                                                |
                                                                                  | OS-SRV-USG:launched_at              | None                                                                                    |
                                                                                  | OS-SRV-USG:terminated_at            | None                                                                                    |
                                                                                  | accessIPv4                          |                                                                                         |
                                                                                  | accessIPv6                          |                                                                                         |
                                                                                  | addresses                           |                                                                                         |
                                                                                  | adminPass                           | XYpiXKSYELm6                                                                            |
                                                                                  | config_drive                        |                                                                                         |
                                                                                  | created                             | 2019-04-03T08:54:25Z                                                                    |
                                                                                  | flavor                              | reservation:9259448d-88bc-45d3-a106-e56f7016f2ab (9259448d-88bc-45d3-a106-e56f7016f2ab) |
                                                                                  | hostId                              |                                                                                         |
                                                                                  | id                                  | 9c1af3e8-4985-46c1-ba25-e2b5f0fc0f52                                                    |
                                                                                  | image                               | cirros-0.3.4-x86_64-disk (c4b1cc17-33f6-4964-933a-810fb81f581b)                         |
                                                                                  | key_name                            | None                                                                                    |
                                                                                  | name                                | VM1                                                                                     |
                                                                                  | progress                            | 0                                                                                       |
                                                                                  | project_id                          | e12cff1acee9459b8836cb2bad4ce0e4                                                        |
                                                                                  | properties                          |                                                                                         |
                                                                                  | security_groups                     | name='default'                                                                          |
                                                                                  | status                              | BUILD                                                                                   |
                                                                                  | updated                             | 2019-04-03T08:54:26Z                                                                    |
                                                                                  | user_id                             | 3501ca5ee41d430ca628be73d63d1d4c                                                        |
                                                                                  | volumes_attached                    |                                                                                         |
                                                                                  +-------------------------------------+-----------------------------------------------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack server list
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
                                                                                  | ID                                   | Name | Status | Networks             | Image                    | Flavor                                           |
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
                                                                                  | 9c1af3e8-4985-46c1-ba25-e2b5f0fc0f52 | VM1  | ACTIVE | private=192.168.1.33 | cirros-0.3.4-x86_64-disk | reservation:9259448d-88bc-45d3-a106-e56f7016f2ab |
                                                                                  +--------------------------------------+------+--------+----------------------+--------------------------+--------------------------------------------------+
                                                                                  
                                                                                  [root@localhost ~]# openstack resource provider usage show 88f75d49-7512-4969-981f-fc893d8d552f
                                                                                  +---------------------------------------------------------+-------+
                                                                                  | resource_class                                          | usage |
                                                                                  +---------------------------------------------------------+-------+
                                                                                  | CUSTOM_RESERVATION_9259448D_88BC_45D3_A106_E56F7016F2AB |     1 |
                                                                                  +---------------------------------------------------------+-------+
                                                                                  
                                                                                  [root@localhost ~]# openstack resource provider usage show 7b155336-faaa-4e39-a51b-851ae7495ac5
                                                                                  +----------------+-------+
                                                                                  | resource_class | usage |
                                                                                  +----------------+-------+
                                                                                  | VCPU           |     1 |
                                                                                  | MEMORY_MB      |  1024 |
                                                                                  | DISK_GB        |    20 |
                                                                                  +----------------+-------+
                                                                                  
                                                                                  [root@localhost ~]# openstack resource provider list
                                                                                  +--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
                                                                                  | uuid                                 | name                         | generation | root_provider_uuid                   | parent_provider_uuid                 |
                                                                                  +--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
                                                                                  | 7b155336-faaa-4e39-a51b-851ae7495ac5 | localhost.localdomain        |          5 | 7b155336-faaa-4e39-a51b-851ae7495ac5 | None                                 |
                                                                                  | 88f75d49-7512-4969-981f-fc893d8d552f | blazar_localhost.localdomain |          2 | 7b155336-faaa-4e39-a51b-851ae7495ac5 | 7b155336-faaa-4e39-a51b-851ae7495ac5 |
                                                                                  +--------------------------------------+------------------------------+------------+--------------------------------------+--------------------------------------+
                                                                                  

                                                                                  Blazar Stein 依旧存在的局限

                                                                                  1. Instance Reservation 要依赖 Host Reservation,并没有实现真正 意义上纯粹的虚拟机预留。
                                                                                  2. Instance Reservation 只支持 CPU、RAM、Disk 资源类型,不支持 NUMA、SR-IOV 等高级资源类型。

                                                                                  对于依赖 Host Reservation 问题的思索

                                                                                  我个人认为 “Instance Reservation 依赖 Host Reservation” 的局限性实属无奈之举,这完全是为了实现 Lease(租约)功能所作出的妥协。道理很简单,准守协约的文明人(租约预留资源的虚拟机)和随时来随时走的流氓(原生虚拟机)显然是没法共存于同一个资源池的。如果 Blazar 不存在 Lease 的概念,自然也不会有这样的局限,但没有 Lease 的资源预留很显然也是没有意义的。

                                                                                  所以,回过头再看 Blazar 采用 “基于资源隔离实现的资源预留” 思路或许才是一个好招。

                                                                                  展开阅读全文

                                                                                  没有更多推荐了,返回首页