提供商网络

提供商(外部)网络将实例桥接到提供第 3 层服务的物理网络基础设施。在大多数情况下,提供商网络使用 VLAN ID 实现第 2 层分段。提供商网络映射到每个计算节点上的提供商桥接,该桥接支持启动提供商网络上的实例。您可以创建多个提供商桥接,每个桥接都需要一个唯一的名称和底层的物理网络接口,以防止交换环路。提供商网络和桥接可以使用任意名称,但每个映射必须引用有效的提供商网络和桥接名称。每个提供商桥接可以包含一个 flat(未标记)网络,以及物理网络基础设施支持的最多数量的 vlan(标记)网络,通常约为 4000 个。

创建提供商网络涉及主机、OVS 和网络服务级别上的多个命令,这些命令会在 OVN 级别产生一系列操作,以创建虚拟网络组件。以下示例创建了一个使用提供商桥接 br-providerflat 提供商网络 provider,并将其绑定到一个子网。

创建提供商网络

  1. 在每个计算节点上,创建提供商桥接,将提供商网络映射到它,并将底层的物理或逻辑(通常是绑定)网络接口添加到它。

    # ovs-vsctl --may-exist add-br br-provider -- set bridge br-provider \
      protocols=OpenFlow13
    # ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-mappings=provider:br-provider
    # ovs-vsctl --may-exist add-port br-provider INTERFACE_NAME
    

    INTERFACE_NAME 替换为底层网络接口的名称。

    注意

    如果成功,这些命令不会产生任何输出。

  2. 在控制器节点上,激活管理项目凭据。

  3. 在控制器节点上,要启用此底盘来托管外部连接的网关路由器,请将 ovn-cms-options 设置为 enable-chassis-as-gw。

    # ovs-vsctl set Open_vSwitch . external-ids:ovn-cms-options="enable-chassis-as-gw"
    

    注意

    如果成功,此命令不会产生任何输出。

  4. 在控制器节点上,在网络服务中创建提供商网络。在这种情况下,其他项目中的实例和路由器可以使用该网络。

    $ openstack network create --external --share \
      --provider-physical-network provider --provider-network-type flat \
      provider
    +---------------------------+--------------------------------------+
    | Field                     | Value                                |
    +---------------------------+--------------------------------------+
    | admin_state_up            | UP                                   |
    | availability_zone_hints   |                                      |
    | availability_zones        | nova                                 |
    | created_at                | 2016-06-15 15:50:37+00:00            |
    | description               |                                      |
    | id                        | 0243277b-4aa8-46d8-9e10-5c9ad5e01521 |
    | ipv4_address_scope        | None                                 |
    | ipv6_address_scope        | None                                 |
    | is_default                | False                                |
    | mtu                       | 1500                                 |
    | name                      | provider                             |
    | project_id                | b1ebf33664df402693f729090cfab861     |
    | provider:network_type     | flat                                 |
    | provider:physical_network | provider                             |
    | provider:segmentation_id  | None                                 |
    | qos_policy_id             | None                                 |
    | router:external           | External                             |
    | shared                    | True                                 |
    | status                    | ACTIVE                               |
    | subnets                   | 32a61337-c5a3-448a-a1e7-c11d6f062c21 |
    | tags                      | []                                   |
    | updated_at                | 2016-06-15 15:50:37+00:00            |
    +---------------------------+--------------------------------------+
    

    注意

    --provider-physical-network 的值必须引用映射中的提供商网络名称。

OVN 操作

OVN 机制驱动程序和 OVN 在创建提供商网络期间执行以下操作。

  1. 机制驱动程序将网络转换为 OVN 北向数据库中的逻辑交换机。

     _uuid               : 98edf19f-2dbc-4182-af9b-79cafa4794b6
     acls                : []
     external_ids        : {"neutron:network_name"=provider}
     load_balancer       : []
     name                : "neutron-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"
     ports               : [92ee7c2f-cd22-4cac-a9d9-68a374dc7b17]
    
    .. note::
    
       The ``neutron:network_name`` field in ``external_ids`` contains
       the network name and ``name`` contains the network UUID.
    
  2. 此外,由于提供商网络由单独的桥接处理,因此在 OVN 北向数据库中创建以下逻辑端口。

    _uuid               : 92ee7c2f-cd22-4cac-a9d9-68a374dc7b17
    addresses           : [unknown]
    enabled             : []
    external_ids        : {}
    name                : "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"
    options             : {network_name=provider}
    parent_name         : []
    port_security       : []
    tag                 : []
    type                : localnet
    up                  : false
    
  3. OVN 北向服务将这些对象转换为 OVN 南向数据库中的数据平面绑定、端口绑定和适当的多播组。

    • 数据路径绑定

      _uuid               : f1f0981f-a206-4fac-b3a1-dc2030c9909f
      external_ids        : {logical-switch="98edf19f-2dbc-4182-af9b-79cafa4794b6"}
      tunnel_key          : 109
      
    • 端口绑定

      _uuid               : 8427506e-46b5-41e5-a71b-a94a6859e773
      chassis             : []
      datapath            : f1f0981f-a206-4fac-b3a1-dc2030c9909f
      logical_port        : "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"
      mac                 : [unknown]
      options             : {network_name=provider}
      parent_port         : []
      tag                 : []
      tunnel_key          : 1
      type                : localnet
      
    • 逻辑流

      Datapath: f1f0981f-a206-4fac-b3a1-dc2030c9909f  Pipeline: ingress
        table= 0(  ls_in_port_sec_l2), priority=  100, match=(eth.src[40]),
          action=(drop;)
        table= 0(  ls_in_port_sec_l2), priority=  100, match=(vlan.present),
          action=(drop;)
        table= 0(  ls_in_port_sec_l2), priority=   50,
          match=(inport == "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"),
          action=(next;)
        table= 1(  ls_in_port_sec_ip), priority=    0, match=(1),
          action=(next;)
        table= 2(  ls_in_port_sec_nd), priority=    0, match=(1),
          action=(next;)
        table= 3(      ls_in_pre_acl), priority=    0, match=(1),
          action=(next;)
        table= 4(       ls_in_pre_lb), priority=    0, match=(1),
          action=(next;)
        table= 5( ls_in_pre_stateful), priority=  100, match=(reg0[0] == 1),
          action=(ct_next;)
        table= 5( ls_in_pre_stateful), priority=    0, match=(1),
          action=(next;)
        table= 6(          ls_in_acl), priority=    0, match=(1),
          action=(next;)
        table= 7(           ls_in_lb), priority=    0, match=(1),
          action=(next;)
        table= 8(     ls_in_stateful), priority=  100, match=(reg0[1] == 1),
          action=(ct_commit; next;)
        table= 8(     ls_in_stateful), priority=  100, match=(reg0[2] == 1),
          action=(ct_lb;)
        table= 8(     ls_in_stateful), priority=    0, match=(1),
          action=(next;)
        table= 9(      ls_in_arp_rsp), priority=  100,
          match=(inport == "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"),
          action=(next;)
        table= 9(      ls_in_arp_rsp), priority=    0, match=(1),
          action=(next;)
        table=10(      ls_in_l2_lkup), priority=  100, match=(eth.mcast),
          action=(outport = "_MC_flood"; output;)
        table=10(      ls_in_l2_lkup), priority=    0, match=(1),
          action=(outport = "_MC_unknown"; output;)
      Datapath: f1f0981f-a206-4fac-b3a1-dc2030c9909f  Pipeline: egress
        table= 0(      ls_out_pre_lb), priority=    0, match=(1),
          action=(next;)
        table= 1(     ls_out_pre_acl), priority=    0, match=(1),
          action=(next;)
        table= 2(ls_out_pre_stateful), priority=  100, match=(reg0[0] == 1),
          action=(ct_next;)
        table= 2(ls_out_pre_stateful), priority=    0, match=(1),
          action=(next;)
        table= 3(          ls_out_lb), priority=    0, match=(1),
          action=(next;)
        table= 4(         ls_out_acl), priority=    0, match=(1),
          action=(next;)
        table= 5(    ls_out_stateful), priority=  100, match=(reg0[1] == 1),
          action=(ct_commit; next;)
        table= 5(    ls_out_stateful), priority=  100, match=(reg0[2] == 1),
          action=(ct_lb;)
        table= 5(    ls_out_stateful), priority=    0, match=(1),
          action=(next;)
        table= 6( ls_out_port_sec_ip), priority=    0, match=(1),
          action=(next;)
        table= 7( ls_out_port_sec_l2), priority=  100, match=(eth.mcast),
          action=(output;)
        table= 7( ls_out_port_sec_l2), priority=   50,
          match=(outport == "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"),
          action=(output;)
      
    • 多播组

      _uuid               : 0102f08d-c658-4d0a-a18a-ec8adcaddf4f
      datapath            : f1f0981f-a206-4fac-b3a1-dc2030c9909f
      name                : _MC_unknown
      ports               : [8427506e-46b5-41e5-a71b-a94a6859e773]
      tunnel_key          : 65534
      
      _uuid               : fbc38e51-ac71-4c57-a405-e6066e4c101e
      datapath            : f1f0981f-a206-4fac-b3a1-dc2030c9909f
      name                : _MC_flood
      ports               : [8427506e-46b5-41e5-a71b-a94a6859e773]
      tunnel_key          : 65535
      

在提供商网络上创建子网

提供商网络至少需要一个包含实例可用的 IP 地址分配、默认网关 IP 地址以及名称解析等元数据的子网。

  1. 在控制器节点上,创建一个绑定到提供商网络 provider 的子网。

    $ openstack subnet create --network provider --subnet-range \
      203.0.113.0/24 --allocation-pool start=203.0.113.101,end=203.0.113.250 \
      --dns-nameserver 8.8.8.8,8.8.4.4 --gateway 203.0.113.1 provider-v4
      +-------------------+--------------------------------------+
      | Field             | Value                                |
      +-------------------+--------------------------------------+
      | allocation_pools  | 203.0.113.101-203.0.113.250          |
      | cidr              | 203.0.113.0/24                       |
      | created_at        | 2016-06-15 15:50:45+00:00            |
      | description       |                                      |
      | dns_nameservers   | 8.8.8.8, 8.8.4.4                     |
      | enable_dhcp       | True                                 |
      | gateway_ip        | 203.0.113.1                          |
      | host_routes       |                                      |
      | id                | 32a61337-c5a3-448a-a1e7-c11d6f062c21 |
      | ip_version        | 4                                    |
      | ipv6_address_mode | None                                 |
      | ipv6_ra_mode      | None                                 |
      | name              | provider-v4                          |
      | network_id        | 0243277b-4aa8-46d8-9e10-5c9ad5e01521 |
      | project_id        | b1ebf33664df402693f729090cfab861     |
      | subnetpool_id     | None                                 |
      | updated_at        | 2016-06-15 15:50:45+00:00            |
      +-------------------+--------------------------------------+
    

如果使用 DHCP 来管理实例 IP 地址,添加子网会在网络服务和 OVN 中产生一系列操作。

  • 网络服务会在适当数量的 DHCP 代理上调度网络。示例环境包含三个 DHCP 代理。

  • 每个 DHCP 代理都会生成一个使用来自子网分配的 IP 地址的网络命名空间,并运行一个 dnsmasq 进程。

  • OVN 机制驱动程序为每个 dnsmasq 进程在 OVN 北向数据库中创建一个逻辑交换机端口对象。

OVN 操作

OVN 机制驱动程序和 OVN 在提供商网络上创建子网期间执行以下操作。

  1. 如果子网使用 DHCP 进行 IP 地址管理,则为服务于该子网的每个 DHCP 代理创建逻辑端口,并将它们绑定到逻辑交换机。在此示例中,该子网包含两个 DHCP 代理。

    _uuid               : 5e144ab9-3e08-4910-b936-869bbbf254c8
    addresses           : ["fa:16:3e:57:f9:ca 203.0.113.101"]
    enabled             : true
    external_ids        : {"neutron:port_name"=""}
    name                : "6ab052c2-7b75-4463-b34f-fd3426f61787"
    options             : {}
    parent_name         : []
    port_security       : []
    tag                 : []
    type                : ""
    up                  : true
    
    _uuid               : 38cf8b52-47c4-4e93-be8d-06bf71f6a7c9
    addresses           : ["fa:16:3e:e0:eb:6d 203.0.113.102"]
    enabled             : true
    external_ids        : {"neutron:port_name"=""}
    name                : "94aee636-2394-48bc-b407-8224ab6bb1ab"
    options             : {}
    parent_name         : []
    port_security       : []
    tag                 : []
    type                : ""
    up                  : true
    
    _uuid               : 924500c4-8580-4d5f-a7ad-8769f6e58ff5
    acls                : []
    external_ids        : {"neutron:network_name"=provider}
    load_balancer       : []
    name                : "neutron-670efade-7cd0-4d87-8a04-27f366eb8941"
    ports               : [38cf8b52-47c4-4e93-be8d-06bf71f6a7c9,
                           5e144ab9-3e08-4910-b936-869bbbf254c8,
                           a576b812-9c3e-4cfb-9752-5d8500b3adf9]
    
  2. OVN 北向服务为这些逻辑端口创建端口绑定,并将其添加到适当的多播组。

    • 端口绑定

      _uuid               : 030024f4-61c3-4807-859b-07727447c427
      chassis             : fc5ab9e7-bc28-40e8-ad52-2949358cc088
      datapath            : bd0ab2b3-4cf4-4289-9529-ef430f6a89e6
      logical_port        : "6ab052c2-7b75-4463-b34f-fd3426f61787"
      mac                 : ["fa:16:3e:57:f9:ca 203.0.113.101"]
      options             : {}
      parent_port         : []
      tag                 : []
      tunnel_key          : 2
      type                : ""
      
      _uuid               : cc5bcd19-bcae-4e29-8cee-3ec8a8a75d46
      chassis             : 6a9d0619-8818-41e6-abef-2f3d9a597c03
      datapath            : bd0ab2b3-4cf4-4289-9529-ef430f6a89e6
      logical_port        : "94aee636-2394-48bc-b407-8224ab6bb1ab"
      mac                 : ["fa:16:3e:e0:eb:6d 203.0.113.102"]
      options             : {}
      parent_port         : []
      tag                 : []
      tunnel_key          : 3
      type                : ""
      
    • 多播组

      _uuid               : 39b32ccd-fa49-4046-9527-13318842461e
      datapath            : bd0ab2b3-4cf4-4289-9529-ef430f6a89e6
      name                : _MC_flood
      ports               : [030024f4-61c3-4807-859b-07727447c427,
                             904c3108-234d-41c0-b93c-116b7e352a75,
                             cc5bcd19-bcae-4e29-8cee-3ec8a8a75d46]
      tunnel_key          : 65535
      
  3. OVN 北向服务将逻辑端口转换为 OVN 南向数据库中的其他逻辑流。

    Datapath: bd0ab2b3-4cf4-4289-9529-ef430f6a89e6  Pipeline: ingress
      table= 0(  ls_in_port_sec_l2), priority=   50,
        match=(inport == "94aee636-2394-48bc-b407-8224ab6bb1ab"),
        action=(next;)
      table= 0(  ls_in_port_sec_l2), priority=   50,
        match=(inport == "6ab052c2-7b75-4463-b34f-fd3426f61787"),
        action=(next;)
      table= 9(      ls_in_arp_rsp), priority=   50,
        match=(arp.tpa == 203.0.113.101 && arp.op == 1),
        action=(eth.dst = eth.src; eth.src = fa:16:3e:57:f9:ca;
                arp.op = 2; /* ARP reply */ arp.tha = arp.sha;
                arp.sha = fa:16:3e:57:f9:ca; arp.tpa = arp.spa;
                arp.spa = 203.0.113.101; outport = inport; inport = "";
                /* Allow sending out inport. */ output;)
      table= 9(      ls_in_arp_rsp), priority=   50,
        match=(arp.tpa == 203.0.113.102 && arp.op == 1),
        action=(eth.dst = eth.src; eth.src = fa:16:3e:e0:eb:6d;
                arp.op = 2; /* ARP reply */ arp.tha = arp.sha;
                arp.sha = fa:16:3e:e0:eb:6d; arp.tpa = arp.spa;
                arp.spa = 203.0.113.102; outport = inport;
                inport = ""; /* Allow sending out inport. */ output;)
      table=10(      ls_in_l2_lkup), priority=   50,
        match=(eth.dst == fa:16:3e:57:f9:ca),
        action=(outport = "6ab052c2-7b75-4463-b34f-fd3426f61787"; output;)
      table=10(      ls_in_l2_lkup), priority=   50,
        match=(eth.dst == fa:16:3e:e0:eb:6d),
        action=(outport = "94aee636-2394-48bc-b407-8224ab6bb1ab"; output;)
    Datapath: bd0ab2b3-4cf4-4289-9529-ef430f6a89e6  Pipeline: egress
      table= 7( ls_out_port_sec_l2), priority=   50,
        match=(outport == "6ab052c2-7b75-4463-b34f-fd3426f61787"),
        action=(output;)
      table= 7( ls_out_port_sec_l2), priority=   50,
        match=(outport == "94aee636-2394-48bc-b407-8224ab6bb1ab"),
        action=(output;)
    
  4. 对于没有 DHCP 代理的每个计算节点

    • OVN 控制器服务将逻辑流转换为集成桥接 br-int 上的流。

      cookie=0x0, duration=22.303s, table=32, n_packets=0, n_bytes=0,
          idle_age=22, priority=100,reg7=0xffff,metadata=0x4
          actions=load:0x4->NXM_NX_TUN_ID[0..23],
              set_field:0xffff/0xffffffff->tun_metadata0,
              move:NXM_NX_REG6[0..14]->NXM_NX_TUN_METADATA0[16..30],
              output:5,output:4,resubmit(,33)
      
  5. 对于具有 DHCP 代理的每个计算节点

    • 创建 DHCP 网络命名空间会添加两个虚拟交换机端口。第一个端口将 DHCP 代理与 dnsmasq 进程连接到集成桥接,第二个端口将集成桥接桥接到提供商桥接 br-provider

      # ovs-ofctl show br-int
      OFPT_FEATURES_REPLY (xid=0x2): dpid:000022024a1dc045
      n_tables:254, n_buffers:256
      capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
      actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
       7(tap6ab052c2-7b): addr:00:00:00:00:10:7f
           config:     PORT_DOWN
           state:      LINK_DOWN
           speed: 0 Mbps now, 0 Mbps max
       8(patch-br-int-to): addr:6a:8c:30:3f:d7:dd
          config:     0
          state:      0
          speed: 0 Mbps now, 0 Mbps max
      
      # ovs-ofctl -O OpenFlow13 show br-provider
      OFPT_FEATURES_REPLY (OF1.3) (xid=0x2): dpid:0000080027137c4a
      n_tables:254, n_buffers:256
      capabilities: FLOW_STATS TABLE_STATS PORT_STATS GROUP_STATS QUEUE_STATS
      OFPST_PORT_DESC reply (OF1.3) (xid=0x3):
       1(patch-provnet-0): addr:fa:42:c5:3f:d7:6f
           config:     0
           state:      0
           speed: 0 Mbps now, 0 Mbps max
      
    • OVN 控制器服务将这些逻辑流转换为集成桥接上的流。

      cookie=0x0, duration=17.731s, table=0, n_packets=3, n_bytes=258,
          idle_age=16, priority=100,in_port=7
          actions=load:0x2->NXM_NX_REG5[],load:0x4->OXM_OF_METADATA[],
              load:0x2->NXM_NX_REG6[],resubmit(,16)
      cookie=0x0, duration=17.730s, table=0, n_packets=15, n_bytes=954,
          idle_age=2, priority=100,in_port=8,vlan_tci=0x0000/0x1000
          actions=load:0x1->NXM_NX_REG5[],load:0x4->OXM_OF_METADATA[],
              load:0x1->NXM_NX_REG6[],resubmit(,16)
      cookie=0x0, duration=17.730s, table=0, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,in_port=8,dl_vlan=0
          actions=strip_vlan,load:0x1->NXM_NX_REG5[],
              load:0x4->OXM_OF_METADATA[],load:0x1->NXM_NX_REG6[],
              resubmit(,16)
      cookie=0x0, duration=17.732s, table=16, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,metadata=0x4,
              dl_src=01:00:00:00:00:00/01:00:00:00:00:00
          actions=drop
      cookie=0x0, duration=17.732s, table=16, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,metadata=0x4,vlan_tci=0x1000/0x1000
          actions=drop
      cookie=0x0, duration=17.732s, table=16, n_packets=3, n_bytes=258,
          idle_age=16, priority=50,reg6=0x2,metadata=0x4 actions=resubmit(,17)
      cookie=0x0, duration=17.732s, table=16, n_packets=0, n_bytes=0,
          idle_age=17, priority=50,reg6=0x3,metadata=0x4 actions=resubmit(,17)
      cookie=0x0, duration=17.732s, table=16, n_packets=15, n_bytes=954,
          idle_age=2, priority=50,reg6=0x1,metadata=0x4 actions=resubmit(,17)
      cookie=0x0, duration=21.714s, table=17, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,18)
      cookie=0x0, duration=21.714s, table=18, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,19)
      cookie=0x0, duration=21.714s, table=19, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,20)
      cookie=0x0, duration=21.714s, table=20, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,21)
      cookie=0x0, duration=21.714s, table=21, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x1/0x1,metadata=0x4
          actions=ct(table=22,zone=NXM_NX_REG5[0..15])
      cookie=0x0, duration=21.714s, table=21, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x1/0x1,metadata=0x4
          actions=ct(table=22,zone=NXM_NX_REG5[0..15])
      cookie=0x0, duration=21.714s, table=21, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,22)
      cookie=0x0, duration=21.714s, table=22, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,23)
      cookie=0x0, duration=21.714s, table=23, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,24)
      cookie=0x0, duration=21.714s, table=24, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x4/0x4,metadata=0x4
          actions=ct(table=25,zone=NXM_NX_REG5[0..15],nat)
      cookie=0x0, duration=21.714s, table=24, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x4/0x4,metadata=0x4
          actions=ct(table=25,zone=NXM_NX_REG5[0..15],nat)
      cookie=0x0, duration=21.714s, table=24, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x2/0x2,metadata=0x4
          actions=ct(commit,zone=NXM_NX_REG5[0..15]),resubmit(,25)
      cookie=0x0, duration=21.714s, table=24, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x2/0x2,metadata=0x4
          actions=ct(commit,zone=NXM_NX_REG5[0..15]),resubmit(,25)
      cookie=0x0, duration=21.714s, table=24, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,25)
      cookie=0x0, duration=21.714s, table=25, n_packets=15, n_bytes=954,
          idle_age=6, priority=100,reg6=0x1,metadata=0x4 actions=resubmit(,26)
      cookie=0x0, duration=21.714s, table=25, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,arp,metadata=0x4,
              arp_tpa=203.0.113.101,arp_op=1
          actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],
              mod_dl_src:fa:16:3e:f9:5d:f3,load:0x2->NXM_OF_ARP_OP[],
              move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],
              load:0xfa163ef95df3->NXM_NX_ARP_SHA[],
              move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],
              load:0xc0a81264->NXM_OF_ARP_SPA[],
              move:NXM_NX_REG6[]->NXM_NX_REG7[],
              load:0->NXM_NX_REG6[],load:0->NXM_OF_IN_PORT[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=25, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,arp,metadata=0x4,
              arp_tpa=203.0.113.102,arp_op=1
          actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],
              mod_dl_src:fa:16:3e:f0:a5:9f,
              load:0x2->NXM_OF_ARP_OP[],
              move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],
              load:0xfa163ef0a59f->NXM_NX_ARP_SHA[],
              move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],
              load:0xc0a81265->NXM_OF_ARP_SPA[],
              move:NXM_NX_REG6[]->NXM_NX_REG7[],
              load:0->NXM_NX_REG6[],load:0->NXM_OF_IN_PORT[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=25, n_packets=3, n_bytes=258,
          idle_age=20, priority=0,metadata=0x4 actions=resubmit(,26)
      cookie=0x0, duration=21.714s, table=26, n_packets=18, n_bytes=1212,
          idle_age=6, priority=100,metadata=0x4,
              dl_dst=01:00:00:00:00:00/01:00:00:00:00:00
          actions=load:0xffff->NXM_NX_REG7[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=26, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,metadata=0x4,dl_dst=fa:16:3e:f0:a5:9f
          actions=load:0x3->NXM_NX_REG7[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=26, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,metadata=0x4,dl_dst=fa:16:3e:f9:5d:f3
          actions=load:0x2->NXM_NX_REG7[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=26, n_packets=0, n_bytes=0,
          idle_age=21, priority=0,metadata=0x4
          actions=load:0xfffe->NXM_NX_REG7[],resubmit(,32)
      cookie=0x0, duration=17.731s, table=33, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,reg7=0x2,metadata=0x4
          actions=load:0x2->NXM_NX_REG5[],resubmit(,34)
      cookie=0x0, duration=118.126s, table=33, n_packets=0, n_bytes=0,
          idle_age=118, hard_age=17, priority=100,reg7=0xfffe,metadata=0x4
          actions=load:0x1->NXM_NX_REG5[],load:0x1->NXM_NX_REG7[],
              resubmit(,34),load:0xfffe->NXM_NX_REG7[]
      cookie=0x0, duration=118.126s, table=33, n_packets=18, n_bytes=1212,
          idle_age=2, hard_age=17, priority=100,reg7=0xffff,metadata=0x4
          actions=load:0x2->NXM_NX_REG5[],load:0x2->NXM_NX_REG7[],
              resubmit(,34),load:0x1->NXM_NX_REG5[],load:0x1->NXM_NX_REG7[],
              resubmit(,34),load:0xffff->NXM_NX_REG7[]
      cookie=0x0, duration=17.730s, table=33, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,reg7=0x1,metadata=0x4
          actions=load:0x1->NXM_NX_REG5[],resubmit(,34)
      cookie=0x0, duration=17.697s, table=33, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,reg7=0x3,metadata=0x4
          actions=load:0x1->NXM_NX_REG7[],resubmit(,33)
      cookie=0x0, duration=17.731s, table=34, n_packets=3, n_bytes=258,
          idle_age=16, priority=100,reg6=0x2,reg7=0x2,metadata=0x4
          actions=drop
      cookie=0x0, duration=17.730s, table=34, n_packets=15, n_bytes=954,
          idle_age=2, priority=100,reg6=0x1,reg7=0x1,metadata=0x4
          actions=drop
      cookie=0x0, duration=21.714s, table=48, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,49)
      cookie=0x0, duration=21.714s, table=49, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,50)
      cookie=0x0, duration=21.714s, table=50, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x1/0x1,metadata=0x4
          actions=ct(table=51,zone=NXM_NX_REG5[0..15])
      cookie=0x0, duration=21.714s, table=50, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x1/0x1,metadata=0x4
          actions=ct(table=51,zone=NXM_NX_REG5[0..15])
      cookie=0x0, duration=21.714s, table=50, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,51)
      cookie=0x0, duration=21.714s, table=51, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,52)
      cookie=0x0, duration=21.714s, table=52, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,53)
      cookie=0x0, duration=21.714s, table=53, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x4/0x4,metadata=0x4
          actions=ct(table=54,zone=NXM_NX_REG5[0..15],nat)
      cookie=0x0, duration=21.714s, table=53, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x4/0x4,metadata=0x4
          actions=ct(table=54,zone=NXM_NX_REG5[0..15],nat)
      cookie=0x0, duration=21.714s, table=53, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x2/0x2,metadata=0x4
          actions=ct(commit,zone=NXM_NX_REG5[0..15]),resubmit(,54)
      cookie=0x0, duration=21.714s, table=53, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x2/0x2,metadata=0x4
          actions=ct(commit,zone=NXM_NX_REG5[0..15]),resubmit(,54)
      cookie=0x0, duration=21.714s, table=53, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,54)
      cookie=0x0, duration=21.714s, table=54, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,55)
      cookie=0x0, duration=21.714s, table=55, n_packets=18, n_bytes=1212,
          idle_age=6, priority=100,metadata=0x4,
              dl_dst=01:00:00:00:00:00/01:00:00:00:00:00
          actions=resubmit(,64)
      cookie=0x0, duration=21.714s, table=55, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,reg7=0x3,metadata=0x4
          actions=resubmit(,64)
      cookie=0x0, duration=21.714s, table=55, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,reg7=0x2,metadata=0x4
          actions=resubmit(,64)
      cookie=0x0, duration=21.714s, table=55, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,reg7=0x1,metadata=0x4
          actions=resubmit(,64)
      cookie=0x0, duration=21.712s, table=64, n_packets=15, n_bytes=954,
          idle_age=6, priority=100,reg7=0x3,metadata=0x4 actions=output:7
      cookie=0x0, duration=21.711s, table=64, n_packets=3, n_bytes=258,
          idle_age=20, priority=100,reg7=0x1,metadata=0x4 actions=output:8