The Flow Networking architecture uses a three-plane approach to simplify network virtualization.

Prism Central provides the management plane, the network controller itself acts as the control plane while the AHV nodes provide the data plane. This architecture provides a strong foundation for Flow Networking. This architecture is depicted in the following chart.

Figure. Flow Networking Architecture Click to enlarge

Deployment Scale

Flow Networking supports the following scale:

VPC

A Virtual Private Cloud (VPC) is an independent and isolated IP address space that functions as a logically isolated virtual network. A VPC could be made up of one or more subnets that are connected through a logical or virtual router. The IP addresses within a VPC must be unique. However, IP addresses may overlap across VPCs. As VPCs are provisioned on top of another IP-based infrastructure (connecting AHV nodes), they are often referred to as the overlay networks. Tenants may spin up VMs and connect them to one or more subnets within a VPC. Virtual Private Cloud (VPC) is a virtualized network of resources that are specifically isolated from the rest of the resource pool. VPC allows you to manage the isolated and secure virtual network with enhanced automation and scaling. The isolation is done using network namespace techniques like IP-based subnets or VLAN based networking.

VPC Subnets

You can use IP address-based subnets to network virtual machines within a VPC. A VPC may use multiple subnets. VPC subnets use private IP address ranges. IP addresses within a single VPC must be unique, in other words, IP addresses inside the same VPC cannot be repeated. However, IP addresses can overlap across multiple VPCs. The following figure shows two VPCs named Blue and Green. Each VPC has two subnets, 192.168.1.0/24 and 192.168.2.0/24, that are connected by a logical router. Each subnet has a VM with an IP address assigned. The subnets and VM IP addresses overlap between the two VPCs.

Figure. VPC Subnet Click to enlarge

The communication between VMs in the same subnets or different subnets in the same VPC (also called East-West communication) is enabled using GEneric NEtwork Virtualization Encapsulation (GENEVE). If a Prism Central manages multiple clusters, then the VMs that belong to the same VPC could be deployed across different clusters. The virtual switch on the AHV nodes provide distributed virtual switching and distributed virtual routing for all VPCs.

Figure. External Communication Click to enlarge

External Subnets

Subnets outside a VPC are external subnets. External subnets may be subnets within the deployment but not included in a specific VPC. External subnets may also be subnets that connect to the endpoints outside the deployment such as another deployment or site.

External subnets can be deployed with NAT or without NAT. You can add a maximum of two external subnets - one external subnet with NAT and one external subnet without NAT to a VPC. Both external subnets cannot be of the same type. For example, you cannot add two external subnets, both with NAT. You can update an existing VPC similarly.

Primary and Secondary IP Addresses for VMs

See VM IP Address Management.

SNAT and Floating IP Address

SNAT and Floating IP addresses are used only when you use NAT for an external subnet.

In Source Network Address Translation (SNAT), the NAT router modifies the IP address of the sender in IP packets. SNAT is commonly used to enable hosts with private addresses to communicate with servers on the public Internet.

For VMs within the VPC to communicate with the rest of the deployment, the VPC must be associated with an external network. In such a case, the VPC is assigned a unique IP address, called the SNAT IP, from the subnet prefix of the external network. When the traffic from a VM needs to be transmitted outside the VPC, the source IP address of the VM, which is a private IP address, is translated to the SNAT IP address. The reverse translation from SNAT IP to private IP address occurs for the return traffic. Since the SNAT IP is shared by multiple VMs within a VPC, only the VMs within the VPC can initiate connections to endpoints outside the VPC. The NAT gateway allows the return traffic for these connections only. Endpoints outside the VPC cannot initiate connections to VMs within a VPC.

In addition to the SNAT IP address, you can also request a Floating IP address — an IP from the external subnet prefix that is assigned to a VM via the VPC that manages the network of the VM. Unless the floating IP address is assigned to the private IP address (primary or secondary IP address) of the VM, the floating IP address is not reachable. When the VM transmits packets outside the VPC, the private IP of the VM is modified to the Floating IP. The reverse translation occurs on the return traffic. As the VM uses the Floating IP address, an endpoint outside the VPC can also initiate a connection to the VM with the floating IP address.

The translation of the private IP addresses to Floating IP or SNAT IP address, and vice versa, is performed in the hypervisor virtual switch. Therefore, the VM is not aware of this translation. Floating IP translation may be performed on the hypervisor that hosts the VM to which the floating IP is assigned to. However, SNAT translation is typically performed in a centralized manner on a specific host.

NAT Gateway

NAT Gateways are used only when you use NAT for an external subnet.

Network Address Translation (NAT) is a process for modifying the source or destination addresses in the headers of an IP packet while the packet is in transit. In general, the sender and receiver applications are not aware that the IP packets are being manipulated.

A NAT Gateway provides the entities inside an internal network with connectivity to the Internet without exposing the internal network and its entities.

A NAT Gateway is:

• A node or a AHV host. You need a host or a node to implement a NAT Gateway because NAT gateways require operations like load balancing and routing that are automatically performed by Flow Networking.
• Connected to the internal network with an internal subnet based IP address and to the external network with an externally-routable IP address.

  The externally-routable IP address may be an IP address from a private IP address space or an RFC1918 address that is used as a NAT gateway. The NAT Gateway IP address could be a static IP address or a DHCP assigned IP address.

Static IP Address

A static IP address is a fixed IP address that is manually assigned to an interface in a network. Static IP addresses provide stable routes that do not have to be updated frequently in the routing table since the static routes generated using static IP addresses do not need to be updated.

Usually in a large IP-based network (a network that uses IP addresses), a Dynamic Host Configuration Protocol or DHCP server assigns IP addresses to interfaces of an entity (using DHCP client service on the entity). However, some entities may require a static IP address that can be reached (manual remote access or via VPN) quickly. A static IP address can be reached quickly because the IP address is fixed, assigned manually and is stored in the routing table for a long duration. For example, a printer in an internal network would need a static IP address so that it can be connected reliably. Static IP addresses can be used to generate static routes which remain unchanged in routing tables, thus providing stable long-term connectivity to the entity that has the static IP address assigned.

Static Route

Static routes are fixed routes that are created manually by the network administrator. Static routes are more suited for small networks or subnets. Irrespective of the size of a network, static routes may be required in a variety of cases. For example, in VPCs where you use virtual private networks (VPNs) or Virtual Tunnel End Point (VTEP) over VxLAN transport connections to manage secure connections, you could use static routes for specific connections such as site-to-site connections for disaster recovery. In such a case it is necessary to have a known reliable route over which the disaster recovery operations can be performed smoothly. Static routes are primarily used for:

• Facilitating the easy maintenance of the routing table in small networks that are not expected to grow.
• Routing to and from other internal route or stub networks. A stub network or an internal route network is a network accessed using a single route and the router has only one neighbor.
• Use as a default or backup route. Such a route is not expected to specifically match any other route in the routing table.

In a network that is not constantly changing, static routes can provide faster and more reliable services by avoiding the network overheads like route advertisement and routing table updates for specific routes.

Overlay networks

You can create an IP-based Overlay subnet for a VPC. An Overlay network is a virtualized network that is configured on top of an underlying virtual or physical network. A special purpose multicast network can be created as an Overlay network within an existing network. A peer-to-peer network or a VPN are also examples of Overlay networks. An important assumption for an Overlay network is that the underlying network is fully connected. Nutanix provides the capability to create Overlay network-based VPCs.

Comparing Overlay with VLAN

See how overlay networks compare with VLAN networks. A virtual local area network or VLAN network is a Layer 2 network that provides virtualized network segmentation solution. VLANs route and balance traffic in a network based on MAC addresses, Protocols such as Ethernet, ports or specific subnets. A VLAN creates a virtual Layer 3 network using Layer 2 addressing by separating broadcast domains virtually or logically. A VLAN configured network behaves as if the network is segmented using a physical layer 2 switch without implementing a layer 3 IP based subnet for the segmentation. VLAN traffic usually cannot traverse outside the VLAN.

The main advantage that VLAN networks provide is that VLAN networks require only layer 2 (L2) connectivity. VLANs do not require any of the layer 3 (L3) Flow Networking features.

Overlay networks can be laid on underlying physical network connections including VLAN networks. Overlay networks provide the following advantages and constraints:

• IP address namespace is decoupled from the physical network.
• You can create, update or delete overlay networks without requiring any configurations on the physical network and powering down the systems.
• You can create overlay networks that can span across multiple clusters.
• VLAN networks are necessary for Bootstrapping of Flow Networking.
  Note: Nutanix recommends using VLAN0 as the default untagged (also called native) VLAN for a CVM and AHV host. You can create VLANs for user VMs using the Network Configuration page. You can use the Create Virtual Switch dialog box from the Network Configuration page to create virtual switches for the user VM VLANs.
• AHV Networking VLAN and Flow Networking VLAN: VLAN backed subnets for external connectivity are managed by the Flow Networking control plane. Traditional AHV VLAN IPAM networks are managed by Acropolis. Do not configure the same VLAN as both a Flow Networking external network and an AHV IPAM network, as this can lead to IP address conflicts.

Traffic Behavior

Broadcast Traffic

When all the guest VMs belonging to a subnet are in the same AHV: Flow Networking broadcasts the traffic to all guest VMs in the same subnet.

When some VMs belonging to a subnet are in other AHVs: Flow Networking tunnels the traffic to only those AHVs which have endpoints in the same subnet.

In other words, Flow Networking broadcasts traffic to all the guest VMs in the same subnet.

Unicast Traffic

Unicast traffic is traffic transmitted on a one-to-one basis between IP addresses and ports. There is only one sender and one receiver for the traffic. Unicast traffic is usually the most used form of traffic in any LAN network using Ethernet or IP networking. Flow Networking transmits unicast traffic based on the networking policies set.

Unknown Unicast Traffic

Flow Networking always drops unknown unicast traffic. It is not transmitted to any guest VM within or outside the source AHV.

Multicast Traffic

Flow Networking transmits the traffic to the VMs in the multicast group within the same subnet. If the VM is on another AHV, the destination AHV must have an endpoint in the subnet.

Multicast Group

A multicast group is defined by an IP address (called a multicast IP address, usually a Class D IP address) and a port number. Once a host has group membership, the host will receive any data packets that are sent to that group defined by an IP address/port number.

Audit Logs

Prism Central generates audit logs for all the flow networking activities like it does for other activities on Prism Central. See Audit Summary View in the Prism Central Guide , for more information about Audit log.

Support Bundle Collection

To support troubleshooting for Flow Networking, you can collect logs.

To collect the logs, run the following commands on the Prism Central VM console:

nutanix@cvm$ logbay collect -t msp,anc

An example of the command is as follows:

nutanix@cvm$ logbay collect -t msp,anc -O msp_pod=true,msp_systemd=true,kubectl_cmds=true,persistent=true --duration=-48h0m0s

Where:

• -t flag indicates the tags to collect

  • msp tag will collect logs from the services running on MSP pods and persistent log volumes (application-level logs)

  • anc tag will collect the support bundle, which includes database dumps and OVN state

• -O flag adds tag-level options

  • msp_pod=true collects logs from MSP service pods

    On the PC, these logs can be found under /var/log/containers .

  • persistent=true collects persistent log volumes (application-level logs for ANC)

    On the PC, these can be found under /var/log/ctrlog

  • kubectl_cmds=true runs kubectl commands to get the Kubernetes resource state

• --duration sets the duration from the present to collect

The command run generates a zip file at a location, for example: /home/nutanix/data/logbay/bundles/<filename>.zip

Unzip the bundle and you'll find the anc logs under a directory specific to your MSP cluster, the worker VM where the pod is running, and the logging persistent volume of that pod. For example:

./msp/f9684be8-b4e8-4524-74b4-076ed53ca1fd/10.48.128.185__worker_master_etcd/persistent/default/ovn/anc-ovn_StatefulSet/

For more information about the task run, see the text file that the command generates at a location, for example: /home/nutanix/data/logbay/taskdata/<taskID>/collection_result.txt

For more information about the logbay collect command, see the Logbay Log Collection (Command Line) topic in the Nutanix Cluster Check Guide (NCC Guide).

Layer 2 Virtual Subnet Extension Alert

The L2StretchLocalIfConflict alert (Alert with Check ID - 801109) may occur while performing Layer 2 virtual subnet extensions. See KB-10395 for more information about its resolution.

The Network and Security category in the Entities Menu expands on-click to display the following networking and security entities that are configured for the registered clusters:

1. Subnets : This dashboard displays the subnets and the operations that you can perform on subnets.

2. Virtual Private Clouds : This dashboard displays the VPCs and the operations that you can perform on VPCs.

3. Floating IPs : This dashboard displays a list of floating IP addresses that you are using in the network. It allows you to request for floating IP addresses from the free pool of I addresses available to the clusters managed by the Prism Central instance.

4. Connectivity : This dashboard allows you to manage the following networking capabilities:

   • Gateways : This tab provides a list of network gateways that you have created and configured, and the operations you can perform on the network gateways. You can check and upgrade the Gateway bundle in Administration > LCM > Inventory .

   • VPN Connections : This tab provides a list of VPN connections that you have created and configured, and the operations you can perform on VPN connections.

   • Subnet Extensions : This tab provides a list of subnets that you have extended at the Layer 2 level using VPN (point-to-point over Nutanix VPN) or VTEP (point-to-multi-point including third party).

5. Security Policies : This dashboard provides a list of security policies you configured using Flow Segmentation. For more information about Security Policies, see the Flow Microsegmentation Guide.

See "Network Connections" section for information on how to configure network connections.

Subnets (Overlay IP subnets), Virtual private clouds, floating IPs, and Connectivity are Flow Networking features. These features support flexible app-driven networking that focuses on VMs and applications instead of virtual LANs and network addresses. Flow Networking powers network virtualization to offer a seamless network experience with enhanced security. It is disabled by default. It is a software-defined network virtualization solution providing overlay capabilities for the on-premises AHV clusters.

Security policies drives the Flow Segmentation features for secure communications. See Flow Microsegmentation Guide.

For information about floating IP addresses and their role in Flow Networking, see SNAT and Floating IP Address.

Go to the Floating IPs dashboard by clicking Network and Security > Floating IPs on the left side-bar.

Figure. Floating IPs dashboard Click to enlarge

To view or select actions you can perform on a floating IP address assigned, select the floating IP address and click the Actions drop down. The following actions are available for a selected floating IP address:

• Update—Assign or change the assignment of the floating IP address. You can assign the floating IP address to a IP address such as a private IP address in a VPC or the primary IP address of a VM or a secondary IP address created on a VM.
• Delete—Delete the floating IP address. The deleted IP address returns to the IP address pool as unused. Before you delete, ensure that it is not assigned to a private IP address or a VM. Change the assignment to None if it is already assigned, using the Update action.

To filter the list of floating IP address assignments, click the Filters option and select the appropriate filtering parameters.

To request floating IP addresses, see Requesting Floating IPs.

A virtual private cloud (VPC) can be deployed on Nutanix cluster infrastructure to manage the internal and external networking requirements using Flow Networking. The workflow to create a complete network based on VPC is described below.

1. Create a VPC—See Creating Virtual Private Cloud. See Updating Virtual Private Cloud to update a VPC you created.
2. Add Subnets to the VPC—See Creating a Subnet to create a Subnet. See Updating a Subnet to update a subnet.
3. Attach the Subnet to VMs—See Attaching a Subnet to a Virtual Machine.