Failover traffic is not managed by L4-7 Service Graphs

See the document ACI Service Graph Design Guide for ACI 5.2 and Later.pdf, page=50.

With the ‘out-of-band’ method

Use the dedicated firewall failover link ports and firewall stateful failover link ports, or designate some ports for that role; Failover traffic and stateful failover traffic are said to be managed ‘out-of-band’

With the ‘in-band’ method

i.e. use the physical network of the data traffic for failover and stateful failover traffic too. When the firewall is attached to ACI, the physical network path of firewall traffic is the ACI fabric.

EPG-to-EPG PBR contract in ACI multi-site with independent HA-firewalls: I must define the BD subnet also under the consumer EPG and set the no Gateway SVI flag.

Contract Enforcement vs PBR Contract Enforcement vs PBR Policy Enforcement Contract enforcement == zoning rules are deployed to a leaf X. So leaf X takes action on the contract-matched traffic based on the deployed zoning rules. Leaf X is determined according to the Contract Policy Enforcement Locations table in the document ACI Contract Guide White Paper, page=20.

There are three ways to manage L4-7 services when we use service graphs. So if no service graph is inserted, we do not talk about these management modes!

[!Abstract] Unless otherwise specified, when Service Graphs are mentioned, then I am referring to the Unmanaged Mode.

Service Graph in Managed Mode (deprecated concept)

aka Service Policy mode: horizontal integration; requires a Device Package.

Device Package

adding a Device Package

[!TLDR] Personal Observation I find few sources discussing standard SG and more literature discussing SGR. It seems to me that Cisco is pushing toward SG with PBR whenever a design involves L4-7 Service Devices.

aka Service Graph with Policy-Based Redirect, Service Graph Redirect. I call it SGR. One-Node SGR: Service Graph with one PBR Node.

Concept of a multi-node Service Graph

the concept of node in the terms ‘single-node SG’ or ‘multi-node SG’ refers to the L4-7 Devices in the SG. So, the term ‘multi-node Service Graph’ refers to a Service Graph with multiple L4-7 Devices, where each L4-7 Device could be:

Stretched Active/Standby Firewall

I must define an ACI L4-7 Device with two Concrete Devices; One for the Active node and one for the Standby node. See ACI Multi-Pod and Service Node Integration, page=5. The active firewall is in pod A, the standby firewall is in pod B. Support east-west and north-south traffic flows. No risk of asymmetric traffic flows because there is only one active firewall at a time.

Impact of traffic hair-pinning

Hair-pinning is a sub-optimal yet not avoidable traffic pattern. Beware of the latency impact of any hair-pinning traffic between pods due to traffic traversing the active firewall. The worst-case occurs when:

In VMware, a good practice is to configure ESXi hosts in a cluster, then present the LUNs to the ESXi cluster. A VMware datastore is a LUN that has been formatted with a filesystem like VMFS. A VMware VM can access the datastore but has no awareness of the underlying LUN or its constituing storage disks. Any protocol that carries the SCSI Command descriptor Block (like FC, FCoE, iSCSI, SAS) uses the LUN concept. A successor to SCSI is NVMe. When I configure a new datastore in a vCenter datacenter, I am presented by a menu to choose a ESXi from. (sc) This is because vCenter simply does not have direct access to the storage devices; it asks that selected ESXi server (can be seen as a ’temporary agent’) to access the storage devices ot its behalf, format the access storage as VMFS disks and make them available in the datastore to many ESXi hosts. This action is only required once during the configuration of the datastore. This interaction takes place on the management vmkernel of the ESXi host. Later when I attach the datastore to the ESXI hosts, they will be able to access the VMFS disks directly, and they will access the storage devices.

#QA SSG with L4-7 Device in Go-To mode: in which situations do we need only layer-2 BDs attached to the Device? Layer-3 BDs? one Layer-2 and one Layer-3 BD? what is the recommended design? Go-To mode == Routed mode When NAT needed on the L4-7 Device: use Service Graph in Go-To mode with NAT. another ‘sub-mode’ is an L3Out between the L4-7 Device and ACI. With both NAT sub-mode and L3Out sub-mode:

VMware NSX-T does not support VXLAN!

NSX-T Segments

a VM is connected to a segment.

• is a layer-2 domain.
• invokes a transport zone.
• once the transport zone configured, you can no longer edit it later.
• VLAN-based segments are equivalent to VLAN IDs
• VLAN-based segments are not permitted to connect to Tier-0 or Tier-1 gateways.

Edge Uplink Segments

require VLAN transport zones. A single Edge Uplink segment can be used more than once.

PBR in two-arm design, the PBR Node Interfaces are in the same VRF

#QA In the SGR in Go-To in Two-arm design, how does IP routing on the firewall scale when we have dozens of (Provider, Consumer) pairs? #QA How cumbersome is the management of the static IP routes related to the SGR on the service device? #QA Is BD2 the IP default gateway for EPG Web endpoints? #QA Which component takes the role of IP default gateway of the firewall: BD1 or FW-Out BD?

Bridge Domains and Subnets

A bridge domain can be configured with an IP subnet; it is then called a layer-3 bridge domain. The BD subnet is aka ACI internal subnet.

The IP address that we configure under the bridge domain is the IP default gateway of the IP subnet. The IP address is configured using a CIDR notation.

Subnets configured under bridge domains are added to the leaf’s routing table as static routes.