Animation - Cisco Communities

Report
Architecture & Solutions Group
US Public Sector Advanced Services
Mark Stinnette, CCIE Data Center #39151
Date 28 August 2013
Version 1.6.2
© 2013 Cisco and/or its affiliates. All rights reserved.
1
This Quick Start Guide (QSG) is a Cookbook style guide to Deploying Data Center
technologies with end-to-end configurations for several commonly deployed architectures.
This presentation will provide end-to-end configurations mapped directly to commonly
deployed data center architecture topologies. In this cookbook style; quick start guide;
configurations are broken down in an animated step by step process to a complete end-toend good clean configuration based on Cisco best practices and strong recommendations.
Each QSG will contain set the stage content, technology component definitions,
recommended best practices, and more importantly different scenario data center
topologies mapped directly to complete end-to-end configurations. This QSG is geared for
network engineers, network operators, and data center architects to allow them to quickly
and effectively deploy these technologies in their data center infrastructure based on
proven commonly deployed designs.
© 2013 Cisco and/or its affiliates. All rights reserved.
2
Commonly Deployed Firewall Designs :: Standalone with Failover
•
•
•
•
•
•
Cisco recommended
Commonly deployed & Typical firewall attachment model
ASA configured for port channels connected via vPC or vPC+
External and Internal traffic traverse same port channel to firewall
Insertion point at the Aggregation layer (Nexus 7000)
10GE interfaces
Altered ASA design topology
ASA configured for port channels connected via vPC or vPC+
Physical interface isolation for external and internal traffic
• External traffic traverse dedicated port channel to firewall
• Internal traffic traverse dedicated port channel to firewall
• Insertion point at the Aggregation layer (Nexus 7000)
• 10GE interfaces
•
•
•
Altered ASA design topology
ASA VDC (Virtual Device Context) sandwich
ASA physically inline
ASA configured for port channels connected via vPC or vPC+
Physical interface isolation for external and internal traffic
• External traffic traverse dedicated port channel to firewall
• Internal traffic traverse dedicated port channel to firewall
• Insertion point at the Aggregation layer (Nexus 7000)
• External firewall port channel connected to Aggregation (VDC)
• Internal firewall port channel connected to Sub-Aggregation (VDC)
• Uses more 10GE interfaces; less effective firewall bandwidth usage
•
•
•
•
•
© 2013 Cisco and/or its affiliates. All rights reserved.
3
Commonly Deployed Firewall Designs :: Cluster Mode
•
•
•
•
•
•
•
•
•
•
Same firewall
Illustrated
Cisco recommended :: ASA Cluster design
Scaling ASA appliances into one logical firewall within the DC architecture
Typical firewall cluster attachment model
ASA configured for port channels connected via vPC or vPC+
External and Internal traffic traverse same cluster data port channel to firewall
Insertion point at the Aggregation layer (Nexus 7000)
10GE interfaces
Cluster two or more (up to 8) ASA firewalls
Greatly increase the throughput of traffic (up to 100Gbps)
True active-active model; in multi-context mode every member interface for all
contexts are capable of forwarding every traffic flow
Alternative View
Cluster up to 8 ASA firewalls
 ASA 5580
 ASA 5585-X
© 2013 Cisco and/or its affiliates. All rights reserved.
4
Firewall Logical Deployment Modes
© 2013 Cisco and/or its affiliates. All rights reserved.
5
Firewall Routing Considerations
Static Routing
Dynamic Routing
No dynamic routing
supported over vPC or vPC+
© 2013 Cisco and/or its affiliates. All rights reserved.
6
Firewall Logical Security Models :: Multi-Tenancy Infrastructure
 Simple Tenant Container
 Single Tier model
 FW Context  VRF  VLAN mapping
© 2013 Cisco and/or its affiliates. All rights reserved.




High Security Use Cases
N-Tier Application Segmentation
Single FW Context instance
Multiple VRFs to VLAN mappings






Enterprise-Class Data Center
Service Provider / Cloud
Zone Based
Shared Multi-Tenant Context
Single FW Context and VRF instance
Multiple VLANs per Zone
7
Firewall Logical Security Models :: Multi-Tenancy Infrastructure
 Tenant Containers
 Private
 Public
 Shared Services DMZ
 N-Tier Application Segmentation
 Rigorous Separation
 High Security Use Cases
 DoD / Federal Government
 Dedicated VRF per Tier
 Tenants mapped to unique firewall context
Unique Tenant Based Containers
Zone Based Containers
 Service Provider / Cloud
 Enterprise-Class Data Center
 Zone Containers
 Organization
 Departments
 Prod, Stage, Dev, Test
 Classification Types
 Application Type (Ent Apps, DB, BigData, VDI)
 Zones mapped to firewall context
 Share the same Security Zone Container
 Optionally, virtual firewalls can be applied if
additional zoning is required within the
containers (ie. VSG & ASA 1000v)
© 2013 Cisco and/or its affiliates. All rights reserved.
8
Benefits Overview
The adaptation of an enterprise-wide security framework is a crucial part of the overall enterprise network architecture. Within
the data center new application rollouts, virtualization, the adaptation of various cloud services and an increasingly
transparent perimeter are creating radical shifts in the data center security requirements. The need for stackable scalable
high capacity firewalls at the data center perimeter is becoming essential. Adaptive Security Appliance (ASA) clustering
feature on the ASA family of firewalls satisfies such a requirement. The clustering feature allows for an efficient way to scale
up the throughput of a group of ASAs, by having them all work in concert to pass connections as one logical ASA device.
Using up to 8 ASA appliances, the clustering feature allows the scaling of up to 100Gbps of aggregate throughput within the
data center perimeter.
ASA Clustering provides the following benefits:
• The ability to aggregate traffic to achieve higher throughput
• Scaling the number of ASA appliances into one logical firewall within the Data Center architecture
• True Active / Active model; when in multi-context mode every member for all contexts of the cluster are capable of
forwarding every traffic flow
• Can force state-full flows to take more symmetrical path which improves predictability and session consistency
• Can operate in either Layer 2 and Layer 3 modes
• Supports single and multiple contexts (firewall virtualization)
• (In Theory) Clustering can be implemented across different data centers over dark fibre as the means of transport.
This use case should be validated and supported in future releases
• Cluster-wide statistics are provided to track resource usage
• A single configuration is maintained across all units in the cluster using automatic configuration sync
© 2013 Cisco and/or its affiliates. All rights reserved.
9
Terminology & Components
CL Master
CL Slave
CL Slave
CL Slave
Po100
Po100
Po100
Po100
ASA Cluster
(n-node)
cLACP Spanned Port Channel
Nexus vPC
Same single vPC ID for all ASA
units in the Cluster
vPC 100
vPC Domain
(vPC or vPC+ supported)
Cluster Data Plane
Cluster Control Plane
Peer-Link
Unique vPC IDs used on the Nexus
Aggregation layer towards each
ASA unit for the CCL
© 2013 Cisco and/or its affiliates. All rights reserved.
Same Port Channel ID used across all
ASA units in the Cluster for the Data
Links towards the Nexus Aggregation
vPC 10
vPC 20
vPC 30
vPC 40
Po50
Po50
Po50
Po50
CL Master
CL Slave
CL Slave
CL Slave
Same Port Channel ID used across all
ASA units in the Cluster for CCL
towards the Nexus Aggregation layer
10
Additional Features, Terminology, & Components
Feature
Overview
Cluster Control Link (CCL)
The CCL provides control plane information between the different cluster members. Also the flows are
redirected within the CCL. To configure the CCL, one configures local port channels with the same
channel identifier on each firewall and connect them to separate vPCs on the corresponding
Nexus7000s. All CCL links are part of same access VLAN.
Cluster Data Link
The most important difference in implementing the cluster data plane is the configuration of a "spanned
port channel (cLACP)" on the firewall. This is necessary because only one Port-Channel/vPC pair is
used in the data plane. To provide channel consistency and seamless operation between both sides, it
is necessary to configure a logical port-channel construct across all the members of the ASA cluster
members. Data Link is a trunk port for all the inside and outside VLANs.
Spanned port channel
(cLACP)
ASA uses a logical link aggregation construct called the Cluster Link Aggregation Control Protocol
(cLACP). It is designed to extend standard LACP to multiple devices so that it can support spancluster. EtherChannels need to be span across the cluster. cLACP allows link aggregation between
one switch, or pair of switches, to multiple (more than two) ASAs in a cluster.
Local port channel
(LACP)
Each ASA uses only two interfaces in a local port channel; meaning its not spanned or shared across
the cluster. The local port-channel (vPC on the Nexus side) gives us local redundancy should we lose
a single cluster control link.
LACP
LACP (Link Aggregation Control Protocol) :: This is the protocol that the ASA runs to negotiate the
ether channel to the adjacent switch. For clustering, the ASAs all share one instance of LACP, such
that the adjacent switch considers the cluster of ASAs as one logical device.
Master
The ASA Cluster elects a master unit that designates which unit responds to the cluster management
address and which unit is used for configuration replication. All configuration is performed on the
master unit. Hard set the master via the priority command.
Slave
All other members in the cluster are slave units. Hard set the slaves accordingly via the priority
command.
© 2013 Cisco and/or its affiliates. All rights reserved.
11
Additional Features, Terminology, & Components
Feature
Overview
Owner Role
Data path Packet Flow Through the Cluster
The unit that initially receives the connection. The owner maintains the TCP state and processes packets. A
connection has only one owner.
The first ASA to receive traffic for a connection is designated as the owner
Director Role
Data path Packet Flow Through the Cluster
The unit that handles owner lookup requests from forwarders and also maintains the connection state to
serve as a backup if the owner fails. When the owner receives a new connection, it chooses a director
based on a hash of the source/destination IP address and TCP ports, and sends a message to the director
to register the new connection. If packets arrive at any unit other than the owner, the unit queries the
director about which unit is the owner so it can forward the packets. A connection has only one director.
Forwarder Role
Data path Packet Flow Through the Cluster
A unit that forwards packets to the owner. If a forwarder receives a packet for a connection it does not own,
it queries the director for the owner, and then establishes a flow to the owner for any other packets it
receives for this connection. The director can also be a forwarder. Note that if a forwarder receives the SYNACK packet, it can derive the owner directly from a SYN cookie in the packet, so it does not need to query
the director (if you disable TCP sequence randomization, the SYN cookie is not used; a query to the director
is required). For short-lived flows such as DNS and ICMP, instead of querying, the forwarder immediately
sends the packet to the director, which then sends them to the owner. A connection can have multiple
forwarders; the most efficient throughput is achieved by a good load-balancing method where there are no
forwarders and all packets of a connection are received by the owner.
© 2013 Cisco and/or its affiliates. All rights reserved.
12
Additional Features, Terminology, & Components
Feature
Overview
Cluster Connection
(Owner Flow)
The actual connection flow that is passing the traffic. We can't know for sure which unit in the cluster
will "own" the flow since whichever ASA receives the first packet in the flow will become the owner.
Only TCP and UDP flows send logical flow updates to the stub flow (and possibly the director stub
flow).
Cluster Connection
(Forwarding Stub Flow)
If a unit receives a packet for a flow that it does not own, it will contact the director of that flow to learn
which unit owns the flow. Once it knows this, it will create and maintain a forwarder flow, which it will
then be used to forward any packets it receives on that connection directly to the owner, bypassing the
director. Forwarder flows do not receive Link Updates (LUs) (since they're just forwarding the packets
and don't care about state). Short lived flows such as DNS and ICMP will not have forwarder flows; the
unit receiving the packets for those conns will simply forward them to the director, which will forward
them to the owner, and the director will not reply back to the forwarder unit asking it to create a
forwarder flow.
Cluster Connection
(Backup Stub Flow)
Based on the flow's characteristics, all units can derive the Director unit for the flow. The director unit
typically maintains the stub (or backup) flow, which can become the full flow in the case the flow's
owner unit fails, and also be used to redirect units towards the flow's owner unit if they receive packets
for the flow. Backup flows receive conn updates to keep them up-to-date in case the owner fails and
the stub flow needs to become the full flow.
Cluster Connection
(Stub or Backup Director
Flow)
If the director chosen for the flow is also the owner (meaning the director received the first packet in
the flow) then it can't be its own backup. Therefore a 'director backup' flow will be created, and a
second hash table will be used to track this. Obviously this director backup flow will receive LUs, since
it needs to be ready to take over if the director/owner fails.
© 2013 Cisco and/or its affiliates. All rights reserved.
13
Additional Features, Terminology, & Components
Feature
Overview
Cluster Group
Names the cluster and enters cluster configuration mode. The name must be an ASCII string from 1 to 38
characters. You can only configure one cluster group per unit. All members of the cluster must use the same
name.
Local Unit
Names this member of the cluster with a unique ASCII string from 1 to 38 characters. Each unit must have a
unique name. A unit with a duplicated name will be not be allowed in the cluster.
Cluster Interface
Specifies the cluster control link interface, preferably an Ether Channel. Specify an IP address; This
interface cannot have a nameif configured. For each unit, specify a different IP address on the same
network.
Console Replicate
Enables console replication from slave units to the master unit. This feature is disabled by default. The ASA
prints out some messages directly to the console for certain critical events. If you enable console replication,
slave units send the console messages to the master unit so you only need to monitor one console port for
the cluster.
Health Check
ASA unit health monitoring and interface health monitoring. When you are adding new units to the cluster,
and making topology changes on the ASA or the switch, you should disable this feature temporarily until the
cluster is complete. You can re-enable this feature after cluster and topology changes are complete.
cLACP System Mac
When using spanned Ether Channels, the ASA uses cLACP to negotiate the Ether Channel with the
neighbor switch. ASAs in a cluster collaborate in cLACP negotiation so that they appear as a single (virtual)
device to the switch. By default, the ASA uses priority 1, which is the highest priority.
Authentication Key
Sets an authentication key for control traffic on the cluster control link. The shared secret is an ASCII string
from 1 to 63 characters. The shared secret is used to generate the key. This command does not affect
datapath traffic, including connection state update and forwarded packets, which are always sent in the
clear.
Cluster Priority
Sets the priority of this unit for master unit elections, between 1 and 100, where 1 is the highest priority.
© 2013 Cisco and/or its affiliates. All rights reserved.
14
Quick Start Guide Assumptions
Physical View – Connectivity Map
ASA Characteristics
 2-wide ASA cluster
 routed mode w/ static routing
 multi-context
 cluster spanned etherchannel mode
Nexus Characteristics
 2-wide 7k Aggregation
 FabricPath vPC+
 Static Routing & VRFs
Each ASA has two 10GE interfaces connected to each respective Nexus 7K representing the data plane for the
cluster. This is a spanned port-channel (recommended) across the ASA cluster in a single vPC. This is called the
Cluster Data Link.
Each ASA has two 10GE interfaces in a local port channel (not spanned or shared across the cluster) called the
Cluster Control Link (CCL). The CCL is the same on each ASA and will connect to the Nexus 7k via a unique vPC; since
these are individual port channels and specific to each ASA.
© 2013 Cisco and/or its affiliates. All rights reserved.
15
ASA Characteristics
 2-wide ASA cluster
 routed mode w/ static routing
 multi-context
 cluster spanned etherchannel mode
Prep for ASA Attachment :: vPC (Option)
feature lacp
feature vpc
feature lacp
feature vpc
vlan 10-20, 2000 – 2999
vlan 10-20, 2000 – 2999
spanning-tree pathcost method long
spanning-tree port type edge bpduguard default
spanning-tree port type edge bpdufilter default
no spanning-tree loopguard default
spanning-tree pathcost method long
spanning-tree port type edge bpduguard default
spanning-tree port type edge bpdufilter default
no spanning-tree loopguard default
spanning-tree vlan 10-20,2000-2999 priority 0
spanning-tree pseudo-information
vlan 10-20,2000-2999 root priority 4096
vlan 10-15,2000-2499 designated priority 8192
vlan 16-20,2500-2999 designated priority 16384
spanning-tree vlan 10-20, 2000-2999 priority 0
spanning-tree pseudo-information
vlan 10-20,2000-2999 root priority 4096
vlan 10-15,2000-2499 designated priority 16384
vlan 16-20,2500-2999 designated priority 8192
vpc domain 1
role priority 1
system-priority 4096
peer-keepalive destination [….] source [….] vrf
management
peer-switch
peer-gateway
auto-recovery
auto-recovery reload-delay
delay restore 30
ip arp synchronize
vpc domain 1
role priority 2
system-priority 4096
peer-keepalive destination [….] source [….] vrf
management
peer-switch
peer-gateway
auto-recovery
auto-recovery reload-delay
delay restore 30
ip arp synchronize
interface port-channel 2
switchport
switchport mode trunk
switchport trunk allowed vlan 10-20,2000-2999
spanning-tree port type network
vpc peer-link
interface port-channel 2
switchport
switchport mode trunk
switchport trunk allowed vlan 10-20,2000-2999
spanning-tree port type network
vpc peer-link
interface e3/1 , e4/1
channel-group 2 force mode active
© 2013 Cisco and/or its affiliates. All rights reserved.
See QSG :: vPC for more details …
interface e3/1 , e4/1
channel-group 2 force mode active
16
ASA Characteristics
 2-wide ASA cluster
 routed mode w/ static routing
 multi-context
 cluster spanned etherchannel mode
Prep for ASA Attachment :: FabricPath vPC+ (Option)
feature lacp
feature vpc
install feature-set fabricpath
feature-set fabricpath
feature lacp
feature vpc
install feature-set fabricpath
feature-set fabricpath
vlan 10-20, 2000 – 2999
mode fabricpath
vlan 10-20, 2000 – 2999
mode fabricpath
fabricpath switch-id 10
fabricpath switch-id 11
fabricpath domain default
root-priority 255
fabricpath domain default
root-priority 254
spanning-tree pseudo-information
vlan 10-20,2000-2999 root priority 0
spanning-tree pseudo-information
vlan 10-20,2000-2999 root priority 0
vpc domain 1
role priority 1
system-priority 4096
peer-keepalive destination [….] source [….] vrf
management
peer-gateway
auto-recovery
auto-recovery reload-delay
delay restore 30
ip arp synchronize
fabricpath switch-id 1000
vpc domain 1
role priority 2
system-priority 4096
peer-keepalive destination [….] source [….] vrf
management
peer-gateway
auto-recovery
auto-recovery reload-delay
delay restore 30
ip arp synchronize
fabricpath switch-id 1000
interface port-channel 2
switchport mode fabricpath
vpc peer-link
interface port-channel 2
switchport mode fabricpath
vpc peer-link
interface e3/1 , e4/1
channel-group 2 force mode active
© 2013 Cisco and/or its affiliates. All rights reserved.
See QSG :: FabricPath for more details …
interface e3/1 , e4/1
channel-group 2 force mode active
17
ASA Characteristics
 2-wide ASA cluster
 routed mode w/ static routing
 multi-context
 cluster spanned etherchannel mode
Initial Firewall Configuration & Verification Checks
mode multiple
mode multiple
no firewall transparent
no firewall transparent
------------------------------------------------------
------------------------------------------------------
show activation-key
show activation-key
Serial Number: JMX1232L11M
...
Security Contexts : 10 perpetual
Cluster : Disabled perpetual
…
Serial Number: JMX1232L11M
...
Security Contexts : 10 perpetual
Cluster : Disabled perpetual
…
activation-key ab42d738 a03b23fc 1bd3c87e d4d4c6d4
4e99ecbb
activation-key ab42d738 a03b23fc 1bd3c87e d4d4c6d4
4e99ecbb
show activation-key
show activation-key
Serial Number: JMX1232L11M
...
Security Contexts : 10 perpetual
Cluster : Enabled perpetual
…
Serial Number: JMX1232L11M
...
Security Contexts : 10 perpetual
Cluster : Enabled perpetual
port-channel load-balance src-dst ip-l4port
Step 1 :: enable multi-context mode
Step 2 :: validate firewall status is routed
Step 3 :: install | validate Cluster license
Step 4 :: configure ECLB
…
port-channel load-balance src-dst ip-l4port
Perform the configuration steps on
the console port of each ASA.
Verify the firewall status as routed. If not routed,
execute the no firewall transparent command.
ciscoasa (config)# show firewall
Firewall mode: Router
Enabling multi-context mode will force a reload;
perform this on all the ASAs.
© 2013 Cisco and/or its affiliates. All rights reserved.
Traffic being load-balanced through ECLB :: it is important
to choose a hash algorithm that is "symmetric," meaning
that packets from both directions will have the same hash,
and will be sent to the same ASA in the spanned Ether
Channel. The hashing value selected should match
between the aggregation switches and ASA, if possible.
The clustering feature requires a specific license
and code version 9.0.1 or greater. If you don’t have
the proper license installed, refer to the “Managing
Feature Licenses for Cisco ASA version 9.0” guide.
http://www.cisco.com/en/US/docs/security/asa/asa9
0/license/license_management/license.html
18
ASA Characteristics
 2-wide ASA cluster
 routed mode w/ static routing
 multi-context
 cluster spanned etherchannel mode
Cluster Control Link
[system context]
cluster interface-mode spanned
Perform the configuration steps on
the console port of each ASA.
master
[system context]
cluster interface-mode spanned
interface Port-channel 40
description Clustering Interface
port-channel load-balance src-dst ip-l4port
interface Port-channel 40
description Clustering Interface
port-channel load-balance src-dst ip-l4port
interface TenGigabitEthernet 0/8, 0/9
channel-group 40 mode active
no nameif
no security-level
interface TenGigabitEthernet 0/8, 0/9
channel-group 40 mode active
no nameif
no security-level
cluster group ASA-CLUSTER
key Cisc0!
local-unit ASA-1
cluster-interface Port-channel40 ip 192.168.1.1
255.255.255.0
priority 1
console-replicate
health-check holdtime 3
clacp system-mac auto system-priority 1
enable
interface port-channel 41
switchport
switchport access vlan 10
spanning-tree port type edge
no lacp graceful-convergence
vpc 41
interface port-channel 42
switchport
switchport access vlan 10
spanning-tree port type edge
no lacp graceful-convergence
vpc 42
© 2013 Cisco and/or its affiliates. All rights reserved.
vPC 41
vPC 42
cluster group ASA-CLUSTER
key Cisc0!
local-unit ASA-2
cluster-interface Port-channel40 ip 192.168.1.2
255.255.255.0
priority 2
Step 1 :: configure cluster interface type
Step 2 :: configure CCL local port channels
Step 3 :: enable clustering
enable
interface e1/1
channel-group 41 force mode active
interface e1/1
channel-group 41 force mode active
interface e1/2
channel-group 42 force mode active
interface e1/2
channel-group 42 force mode active
vlan 10
mode fabricpath
name CLUSTER-CLL
vlan 10
mode fabricpath
name CLUSTER-CLL
interface port-channel 41
switchport
switchport access vlan 10
spanning-tree port type edge
no lacp graceful-convergence
vpc 41
interface port-channel 42
switchport
switchport access vlan 10
spanning-tree port type edge
no lacp graceful-convergence
vpc 42
19
NOTES
Cluster Control Link
[system context]
Recommend you use a Ten Gigabit Ethernet interface for the cluster control link.
cluster interface-mode spanned
interface Port-channel 40
description Clustering Interface
port-channel load-balance src-dst ip-l4port
interface TE 0/8, 0/9
channel-group 40 mode active
no nameif
no security-level
interface port-channel 42
switchport
switchport access vlan 10
spanning-tree port type edge
no lacp graceful-convergence
vpc 42
Each ASA communicates with each other across this common Vlan to form the cluster, update state information
and pass data (when necessary).
The port channel configurations for 41, 42 on aggregation switch N7k-1 map to port-channel 40 on each ASA. The
aggregation switch N7k-2 is configured the same with the only difference is that it physically connects to a different
port (0/8) on each ASA. It is recommended to configure spanning-tree port type edge for the port-channels.
cluster group ASA-CLUSTER
key Cisc0!
local-unit ASA-1
cluster-interface Port-channel40 ip 192.168.1.1
255.255.255.0
priority 1
console-replicate
health-check holdtime 3
clacp system-mac auto system-priority 1
enable
interface port-channel 41
switchport
switchport access vlan 10
spanning-tree port type edge
no lacp graceful-convergence
vpc 41
The recommended method is to use a spanned Ether Channel. When configured, if it detects any incompatibilities,
it will clear them from the configuration and force a reload. This needs to be executed on each unit.
Port channel 40 is configured on each ASA and maps to 41, 42 on each N7k. The CCL interface configuration is
not replicated from the master unit to slave units; however, you must use the same configuration on each unit.
Ports te0/8 and te0/9 will be used for the CCL port-channel on each unit.
The ASA is actively negotiating LACP on the channel. This is another best practice; make sure all interfaces
participating in channeling are actively using LACP. Also note there is no nameif or security-level configuration on
the physical interfaces or the logical interface since this is being used for clustering control plane only.
All members of the cluster must share the same cluster group name and key if configured. The local-unit name,
cluster-interface IP address and priority value needs to be unique for each unit in the cluster. The cluster master
unit is determined by the priority setting, between 1 and 100, where 1 is the highest priority.
‘Enable’ command at the end of cluster configuration will start the cluster mode.
interface e1/1
channel-group 41 force mode active
interface e1/2
channel-group 42 force mode active
vlan 10
mode fabricpath
name CLUSTER-CLL
© 2013 Cisco and/or its affiliates. All rights reserved.
Console-replicate is an optional command that allows slave units to replicate console messages
to the master. Since we spend most of our time on the master for configuration and
troubleshooting purposes.
Step 1 :: configure cluster interface type
Step 2 :: configure CCL local port channels
Step 3 :: enable clustering
20
ASA Characteristics
 2-wide ASA cluster
 routed mode w/ static routing
 multi-context
 cluster spanned etherchannel mode
Cluster Control Link & MTU
[system context]
Perform the configuration steps on
the console port of each ASA.
[system context]
mtu cluster 9216
mtu cluster 9216
jumbo-frame reservation
jumbo-frame reservation
vlan 10
mode fabricpath
name CLUSTER-CLL
vlan 10
mode fabricpath
name CLUSTER-CLL
interface port-channel 41
switchport
switchport access vlan 10
spanning-tree port type edge
mtu 9216
no lacp graceful-convergence
vpc 41
interface port-channel 41
switchport
switchport access vlan 10
spanning-tree port type edge
mtu 9216
no lacp graceful-convergence
vpc 41
interface port-channel 42
switchport
switchport access vlan 10
spanning-tree port type edge
mtu 9216
no lacp graceful-convergence
vpc 42
interface port-channel 42
switchport
switchport access vlan 10
spanning-tree port type edge
mtu 9216
no lacp graceful-convergence
vpc 42
interface e1/1
channel-group 41 force mode active
mtu 9216
interface e1/2
channel-group 42 force mode active
mtu 9216
© 2013 Cisco and/or its affiliates. All rights reserved.
Step 1 :: enable mtu cluster [system context]
Step 2 :: enable jumbo frame reservation [system context]
Step 2 :: enable jumbo frame on the Nexus aggregation
It is recommended to enable jumbo frame reservation and mtu cluster
at least to1600 for the use with the cluster control link. When a packet
is forwarded over cluster control link an additional trailer will be added,
which could cause fragmentation. Set this to 9216 to match the system
jumbo frame size configured on the N7k. Configure this on the master
system context, save the config and then reboot the cluster.
A reboot is required to enable jumbo frames on the ASA.
interface e1/1
channel-group 41 force mode active
mtu 9216
interface e1/2
channel-group 42 force mode active
mtu 9216
21
ASA Characteristics
 2-wide ASA cluster
 routed mode w/ static routing
 multi-context
 cluster spanned etherchannel mode
Cluster Control Link & Management Access
[system context]
master
Perform the configuration steps on
the console port of each ASA.
interface Management0/0
admin-context admin
context admin
allocate-interface Management0/0
config-url disk0:/admin.cfg
--------------------------------------------------------------[admin context]
ip local pool mgmt 10.0.0.201-10.0.0.207 mask 255.255.255.0
interface Management0/0
management-only
nameif mgmt
security-level 100
ip address 10.0.0.200 255.255.255.0 cluster-pool mgmt
route mgmt 0.0.0.0 0.0.0.0 10.0.0.1 1
--------------------------------------------------------------[system context]
Step 1 :: allocate management interface [system context]
Step 2 :: configure cluster management [admin context]
Step 3 :: configure cluster host name prompt (optional) [system context]
prompt hostname context cluster-unit
In the system context allocate the management interface(0/0) to the admin context.
The management interface is configured with a primary IP address, along with a pool of addresses.
Display the pool IP addresses :: show ip local pool mgmt
The primary management IP address always belongs to the current master unit, while the pool addresses
are used to connect to each unit individually. Each unit, including the master gets a pool address assigned.
You can connect to the master through either address, but if a failover should occur, the primary address
will move to the new master. In the admin context configure the management IP addresses.
© 2013 Cisco and/or its affiliates. All rights reserved.
22
ASA Characteristics
 2-wide ASA cluster
 routed mode w/ static routing
 multi-context
 cluster spanned etherchannel mode
Cluster Data Link
[system context]
interface Port-channel26
description Data Spanned Port-channel
port-channel load-balance src-dst ip-l4port
port-channel span-cluster vss-load-balance
master
The N7k aggregation pair data port-channel is
configured as a single vPC for all ASA units in the
cluster. The vPC is configured as a trunk on the
N7ks and as sub-interfaces on the ASA units.
interface TenGigabitEthernet 0/6
description Data Link to N7k-2
channel-group 26 mode active vss-id 1
interface TenGigabitEthernet 0/7
description Data Link to N7k-1
channel-group 26 mode active vss-id 2
vPC 26
The spanned data port-channel is configured in the
‘system context’. These port channels are shared
across all ASA units and act as a single bundle. The
N7k aggregation switches see this as a single portchannel, each having 4 interfaces configured.
The vss-id x command is used to identify the
specific switch in the aggregation pair it connects to
The port-channel span-cluster vss-load-balance
enables spanning.
Together these commands form the spanned Ether
Channel. A spanned Ether Channel requires active
LACP negotiation to be configured.
feature lacp
feature vpc
interface port-channel 26
switchport
switchport mode trunk
switchport trunk allowed vlan 51, 2011-2012
spanning-tree port type edge trunk
no lacp graceful-convergence
vpc 26
interface e1/4, e1/5
lacp rate fast
channel-group 26 force mode active
© 2013 Cisco and/or its affiliates. All rights reserved.
feature lacp
feature vpc
It is recommended to configure the following for
the best link aggregation and convergence ::
 lacp rate fast
 no lacp graceful-convergence
 spanning-tree port type edge trunk
Step 1 :: configure Nexus aggregation port channels
Step 2 :: configure spanned data port channel
interface port-channel 26
switchport
switchport mode trunk
switchport trunk allowed vlan 51, 2011-2012
spanning-tree port type edge trunk
no lacp graceful-convergence
vpc 26
interface e1/4, e1/5
lacp rate fast
channel-group 26 force mode active
23
Simple Tenant Container
Logical Firewall Security Model
Now we have the network infrastructure built; lets configure a simple but yet flexible tenant
container. Route summarization and static redistribution is used to advertise tenancy subnets
into the Core or WAN Edge layer using OSPF. This will allow flexibility when adding additional
server VLANs in any tenant without making any changes to static routes and routing at the
aggregation layer. Since gateways for all VLANs within the VRF are at the aggregation layer,
all interfaces are directly connected. No routing protocol is required to distribute routes
within a given VRF.
Security Container
ASA Context Characteristics
 Single Tiered Private Zone
 1 outside VLAN
 1 inside VLAN
© 2013 Cisco and/or its affiliates. All rights reserved.
Nexus Characteristics
 1 VRF [internal private zone]
 3 VLANs
 3 HSRP Groups
[Outside, Inside, Server]
24
Simple Tenant Container
master
Logical Firewall
Security Model
[system context]
[Tenant_Zone_1 context]
interface Port-channel26
description Data Spanned Port-channel
port-channel load-balance src-dst ip-l4port
port-channel span-cluster vss-load-balance
Hostname Tenant_Zone_1
interface TenGigabitEthernet 0/6
channel-group 26 mode active vss-id 1
interface TenGigabitEthernet 0/7
channel-group 26 mode active vss-id 2
interface Port-channel26.51
vlan 51
interface Port-channel26.2011
vlan 2011
interface Port-channel26.2012
vlan 2012
context Tenant_Zone_1
description Tenant Zone 1 FW Context
allocate-interface Port-channel26.51
allocate-interface Port-channel26.2011
allocate-interface Port-channel26.2012
config-url disk0:/Tenant_Zone_1.cfg
Step 1 :: create sub-interfaces
Step 2 :: create virtual firewall context
Step 3 :: allocate sub-interfaces to context
Step 4 :: configure context interfaces
Step 5 :: configure context default route
Step 6 :: configure context static route(s) to servers vlans
The data port-channel is configured as sub-interfaces and allocated to
the proper Tenant Zone context as required.
The context has a default route to the outside interface (N7k
aggregation), while more specific routes are used to reach servers
through the inside interface; those routes use the HSRP address
as the gateway IP (N7k aggregation).
Followed by the security information which is configured for each
context (sub-set shown here).
Port-channel26.51 is used for inband management (in this example)
© 2013 Cisco and/or its affiliates. All rights reserved.
interface Port-channel26.51
description Mgmt Vlan
management-only
nameif mgmt
security-level 0
ip address 200.1.51.2 255.255.255.0
interface Port-channel26.2011
description Tenant Zone 1 OUTSIDE Vlan
nameif outside
security-level 10
ip address 200.1.1.11 255.255.255.0
interface Port-channel26.2012
description Tenant Zone 1 INSIDE Vlan
nameif inside
security-level 100
ip address 200.1.2.11 255.255.255.0
route outside 0.0.0.0 0.0.0.0 200.1.1.253 1
route inside 200.1.3.0 255.255.255.0 200.1.2.253 1
access-list inside-in extended permit ip any any
access-list outside-in extended permit ip any any
access-group outside-in in interface outside
access-group inside-in in interface inside
25
Simple Tenant Container
Logical Firewall
Security Model
[N7k-1]
[N7k-2]
ip route 200.1.3.0/24 200.1.1.11
ip route 200.1.3.0/24 200.1.1.11
interface Vlan2011
description Tenant Zone 1 OUTSIDE Vlan
mtu 9216
no ip redirects
ip address 200.1.1.251/24
hsrp 1
ip 200.1.1.253
interface Vlan2011
description Tenant Zone 1 OUTSIDE Vlan
mtu 9216
no ip redirects
ip address 200.1.1.252/24
hsrp 1
ip 200.1.1.253
ip prefix-list static2ospfPfx seq 10 permit 200.0.0.0/10 le 24
ip prefix-list static2ospfPfx seq 10 permit 200.0.0.0/10 le 24
route-map direct2ospf permit 10
match ip address prefix-list static2ospfPfx
route-map direct2ospf permit 10
match ip address prefix-list static2ospfPfx
router ospf 1
router-id [x.x.x.x]
redistribute static route-map direct2ospf
router ospf 1
router-id [x.x.x.x]
redistribute static route-map direct2ospf
Note, the outside SVIs belong to the default global
VRF. Nexus is already VRF aware and by default
everything belongs to the default VRF.
Route summarization is used to advertise tenancy
subnets into the Core / WAN Edge layer using
OSPF. This allows adding of server VLANs in any
tenancy without making any changes to static
routes and routing at the aggregation layer.
Step 1 :: create firewall outside vlan SVI & HSRP
Step 2 :: add static route for server vlan towards firewall context outside IP
Step 3 :: redistribute server vlan into OSPF
© 2013 Cisco and/or its affiliates. All rights reserved.
26
Simple Tenant Container
Logical Firewall
Security Model
The AGG pair uses a default route in the VRF to route through the ASA
cluster for outbound traffic.
The SVIs are configured to use HSRP. VLANs 2011 and 2012
represent the outside and inside interfaces of the ASA units for context
Tenant_Zone_1. VLAN 2013 is used as a server VLAN. The inside
VLANs are contained in a VRF to isolate the traffic and routing.
Step 1 :: create tenant zone VRF
Step 2 :: add default route to firewall context inside IP
Step 3 :: create firewall inside vlan SVI & HSRP
Step 4 :: create server vlan SVI & HSRP
© 2013 Cisco and/or its affiliates. All rights reserved.
[N7k-1]
[N7k-2]
vrf context Tenant_Zone_1
ip route 0.0.0.0/0 200.1.2.11
vrf context Tenant_Zone_1
ip route 0.0.0.0/0 200.1.2.11
interface Vlan2012
description Tenant Zone 1 INSIDE Vlan
mtu 9216
vrf member Tenant_Zone_1
no ip redirects
ip address 200.1.2.251/24
hsrp 1
ip 200.1.2.253
interface Vlan2012
description Tenant Zone 1 INSIDE Vlan
mtu 9216
vrf member Tenant_Zone_1
no ip redirects
ip address 200.1.2.252/24
hsrp 1
ip 200.1.2.253
interface Vlan2013
description Tenant Zone 1 SERVER Vlan
mtu 9216
vrf member Tenant_Zone_1
no ip redirects
ip address 200.1.3.251/24
hsrp 1
ip 200.1.3.253
interface Vlan2013
description Tenant Zone 1 SERVER Vlan
mtu 9216
vrf member Tenant_Zone_1
no ip redirects
ip address 200.1.3.252/24
hsrp 1
ip 200.1.3.253
27
Simple Tenant Container
Logical Firewall
Security Model
[Tenant_Zone_1 context]
route outside 0.0.0.0 0.0.0.0 200.1.1.253 1
route inside 200.1.3.0 255.255.0.0 200.1.2.253 1
route inside 200.1.111.0 255.255.255.0 200.1.2.253 1
[Load Balancer virtual context]
interface [floating]
ip address 200.1.2.50 /24
[N7k-1]
[N7k-2]
ip route 0.0.0.0/0 200.1.2.11
Ip route 200.1.3.0/24 200.1.2.253
vrf context Tenant_Zone_1
ip route 0.0.0.0/0 200.1.2.11
ip route 200.1.112.0/24 200.1.2.50
vrf context Tenant_Zone_1
ip route 0.0.0.0/0 200.1.2.11
ip route 200.1.112.0/24 200.1.2.50
Step 1 :: add firewall route to load balancer VIP [firewall context]
Step 2 :: add route to load balancer SNAT address pool [Nexus aggregation]
Step 3 :: add routes on load balancer
On the firewall context, add a specific route to reach the load-balancer
through the inside interface; towards Nexus aggregation HSRP
address. The route will use the alias IP address or floating IP address
(similar to HSRP) on the load balancer.
On the Nexus aggregation, add a specific route to reach the loadbalancer SNAT pool in the one-arm configuration; LB is the next hop.
Load Balancer vendor selection or configuration is outside scope of this document
© 2013 Cisco and/or its affiliates. All rights reserved.
On the load balancer add the default route towards the firewall’s inside
interface and add a more specific route to the servers, towards the
Nexus aggregation HSRP address.
28
Show Commands
Here are some helpful commands executed in the ‘system context’ on the master unit:
• Shows the cluster status :: show cluster info
• Shows cluster wide connection distribution :: show cluster info conn-distribution
• Shows cluster wide packet distribution :: show cluster info packet-distribution
•
•
•
•
Clear asp counters :: cluster exec clear asp drop
Show asp counters. Helpful to isolate drops :: cluster exec show asp drop
Shows the port channel summary on all units in the cluster :: cluster exec show port-channel summary
Shows all connections across the cluster. This command can show how traffic for a single flow arrives at different ASAs in
the cluster :: cluster exec show conn
• Shows connection detail for a particular flow across all units in the cluster. Note, this needs to be executed in a context that
is handling the flow :: cluster exec show conn detail address [x.x.x.x]
• Show the unique MAC for the entire cluster that will be used for the LACP partner :: show lacp cluster system-id
• Show the cluster system MAC (automatically generated) :: show lacp cluster system-mac
Commands executed in the ‘admin context’ on the master unit:
• Display the pool IP addresses :: show ip local pool mgmt
© 2013 Cisco and/or its affiliates. All rights reserved.
29
Strong Recommendations and Key Notes
• Clustering is best enabled in a specific, phased manner. To reduce the potential for errors, enable the CCL first and bring
up the cluster before adding the remaining configuration. At a minimum, an active cluster control link network is required
before you configure the units to join the cluster; this includes the upstream and downstream equipment port channels.
• When configuring clustering you need to select the cluster interface-mode first, as it will clear the existing configuration and
force a reboot. It is recommended to use spanned Ether Channel.
• A console connection is always required to enable or disable clustering.
• Cluster control link bandwidth should match or exceed the highest available bandwidth of data interfaces on a single
cluster unit.
• Recommend that you use Ten Gigabit Ethernet interfaces for the cluster control link, especially if there is high amount of
centralized traffic or asymmetric traffic. If most traffic is centralized or asymmetric (undesirable) the cluster control link
should have a higher bandwidth than data interface on each unit, because this traffic will have to be forwarded over cluster
control link.
• Recommend that you use a port-channel for the CCL for additional resiliency. The port-channel configuration should use
LACP mode active.
• The cluster control link should be in an isolated network and must not be a spanned Ether Channel. It needs to be
configured on the aggregation switches as a unique port-channel for each unit in the cluster.
‘switchport access vlan [x]’
© 2013 Cisco and/or its affiliates. All rights reserved.
30
Strong Recommendations and Key Notes
• It is recommended that spanning-tree port type edge or edge trunk is configured on the aggregation switch interfaces
connecting to the cluster control and data interfaces. If this is not enabled, initial synchronization communication between
ASA units in the cluster could fail and connections might be dropped.
• Using the same port channel load balancing hash algorithm between the ASA and Nexus 7000 (src-dst ip-l4port). Do not
use the vlan keyword in the load-balance algorithm because it can cause unevenly distributed traffic to the ASAs in a
cluster.
• Recommend that you do not specify the maximum and minimum links for a port-channel (The lacp max-bundle and portchannel min-bundle commands) on either the ASA or the switch.
• It is recommended that the spanned data port-channel is configured on the switch with no lacp graceful-convergence and
lacp rate fast to achieve fast link aggregation and convergence.
• Recommend to use spanned Ether Channels (cluster interface-mode spanned) instead of individual interfaces because
individual interfaces rely on routing protocols to load-balance traffic, and routing protocols often have slow convergence
during a link failure.
• An IGP routing protocol peered with the ASA cluster does not provide the best convergence at the moment, static routes
and Ether Channel Load Balancing (ECLB) is recommended to route and hash traffic to and from the ASA cluster. Note:
dynamic routing is not supported over vPC or vPC+
• It is recommended to enable jumbo frame reservation and mtu cluster 1600 for use with the cluster control link (CCL).
When a packet is forwarded over cluster control link an additional trailer will be added, which could cause fragmentation.
© 2013 Cisco and/or its affiliates. All rights reserved.
31
Strong Recommendations and Key Notes
• For the management interface, we recommend using one of the dedicated management interfaces (m0/0 or m0/1). This
should be configured to use an isolated network apart from the CCL or data interface configuration.
• In spanned Ether Channel mode, if you configure the management interface as an individual interface, you cannot enable
dynamic routing for the management interface. You must use a static route.
• Recommend that you manually force an ASA unit to be the designated master and the other units as slaves via the priority
command under the cluster group configuration.
• In single context mode, it is strongly recommended to configure static MAC addresses for a spanned Ether Channel, so
that the MAC address does not change when the current master unit leaves the cluster. Manually configured MAC
addresses will always stay with the master unit.
• In multiple context mode, if you share an interface between contexts, auto-generation of MAC addresses is enabled by
default. You should verify this to avoid any potential issues. The following command ‘mac-address auto prefix 1’ in the
configuration is used to auto-generate MAC addresses
• Note :: In spanned Ether Channel mode, if you configure the management interface as an individual interface, you cannot
enable dynamic routing for the management interface. You must use a static route.
• Note :: you enable clustering when you enter the ’enable’ command under the cluster group configuration. If you disable
clustering, all data interfaces are shut down, and only the management interface is active.
• A Cluster license is required on each unit. For other feature licenses, cluster units do not require the same license on each
unit. If you have feature licenses on multiple units, they combine into a single running ASA cluster license. Note, each unit
must have the same encryption license when in cluster mode.
© 2013 Cisco and/or its affiliates. All rights reserved.
32
Strong Recommendations and Key Notes
• Recommended in principle to first maximize the number of active ports in the channel, and secondly keep the number of
active primary ports and the number of active secondary ports in balance. Having an even number of ASA units in the
clusters will allow traffic to balance evenly.
Note that when an odd number unit joins the cluster, traffic is not balanced evenly between all units. Link or device failure
is handled with the same principle; you may end up with a less-than-perfect load balancing situation.
• Recommend to use the health check feature; which is configured under the cluster group configuration and the default
holdtime is 3 seconds. After you add all the slave units, and the cluster topology is stable, re-enable the cluster health
check feature, which includes unit health monitoring and interface health monitoring. Keepalive messages between
members determine member health. If a unit does not receive any keepalive messages from a peer unit within the
holdtime period, the peer unit is considered unresponsive or dead.
• When any topology changes occur (such as adding or removing a data interface, enabling or disabling an interface on the
ASA or the switch, or adding an additional switch to form a vPC) you should disable the health check feature. When the
topology change is complete, and the configuration change is synced to all units, you can re-enable the health check
feature.
• When the firewall is deployed in transparent mode (vlan translation between inside and outside vlans that belong to same
bridge-group with associated BVI interface) all cluster configuration recommendations remain the same; but an additional
strong recommendation is to filter STP BPDU forwarding using an access-list on the inside and outside interfaces when
the ASA Cluster is connected to a vPC or vPC+ domain on the Nexus platform.
access-list 1 ethertype deny bpdu
access-group 1 in interface inside
access-group 1 in interface outside
© 2013 Cisco and/or its affiliates. All rights reserved.
33
Additional Resources & Further Reading
External (public)
Great External
Resources
ASA Clustering within VMDC Architecture
http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/VMDC/ASA_Cluster/ASA_Cluster.html
VMDC (Virtual Multi-Service Data Center) 3.0.1 Implementation Guide
http://www.cisco.com/en/US/partner/docs/solutions/Enterprise/Data_Center/VMDC/3.0.1/IG/VMDC301_IG1.html
ASA 5500 Configuration Guides
http://www.cisco.com/en/US/partner/products/ps6120/products_installation_and_configuration_guides_list.html
Configure a Cluster of ASAs (version 9.1 code)
http://www.cisco.com/en/US/partner/docs/security/asa/asa91/configuration/general/ha_cluster.html
Nexus 7000 Configuration Guides
http://www.cisco.com/en/US/products/ps9402/products_installation_and_configuration_guides_list.html
© 2013 Cisco and/or its affiliates. All rights reserved.
34
Additional Resources & Further Reading
Quick Start Guide :: Virtual Port Channel (vPC)
https://communities.cisco.com/docs/DOC-35728
Quick Start Guide :: FabricPath
https://communities.cisco.com/docs/DOC-35725l
© 2013 Cisco and/or its affiliates. All rights reserved.
35
© 2013 Cisco and/or its affiliates. All rights reserved.
36

similar documents