Introduction

This document presents an ASN GW implementation using industry standard ATCA/microTCA system. The proposed solution is: Modular, scalable, highly available and cost effective. Proposed implementation takes a phased approach and maximizes the use of COTS hardware and software components. This will help in early availability of the system and reduces the risks. Three implementation phases (Basic system, redundancy, Macro Mobility) are proposed. Approximate delivery timelines for the three phases are 8 months, 12 months and 16 months from the start of work.
This is a suggestive solution and will require modifications/improvements for changes in the Wimax specifications, system requirements and other needs. Reader of this document is encouraged to get the basic understanding of WiMAX standards and ATCA/microTCA specifications.
Rest of the document describes the software and hardware architecture of the proposed ASN GW, Scalability and Availability, Manageability, Performance Analysis of the proposed solution. In the end, an implementation approach is also suggested.

ASN GATEWAY: ATCA Based implementation

This section describes the software architecture of the proposed ASN GW and implementation using microTCA/ATCA based system.

ATCA Based ASN GW : Hardware Architecture

ATCA provides open architecture framework for building scalable, high density and high-availability system for telecommunication application. ATCA backplane provides necessary infrastructure for inter card communication (base interface, fabric interface), providing high availability (Dual start for base interface; dual star, dual dual start, full mesh for fabric interface; redundant hub slot; update channel for data replication and failure detection) and other features which makes it natural choice for the ASN GW.
Slot 1 and slot 2 are the HUB slots. Other slots are the node slots and accommodate any ATCA backplane complied card. At least one of the node slots is populated with shelf management controller for managing the shelf through IPMB 0.
The hardware architecture for the proposed ASN GW is shown in Figure 1. For ASN GW, an eight slots shelf providing Base Interface and Update ports will be good enough. Base interface, will permit inter card communication up to 1GB, which is good enough for ASN GW. However, if larger data rate is required then fabric interface is a must. This will not require any change in the user data path or control plane software. Changes will be limited to the infrastructure and shelf management part.

Figure 1: Hardware Architecture

Update ports will be used for exchanging redundancy related data updates and control information.

All the boards and shelf will be ATCA based COTS hardware components. CSN and BS termination boards will be network processor (single/dual) based board providing at least 2x1Gig Ethernet port. Control processor board shall be G-CPU (Pentium/PPC603/XEON, 1GIG Memory) based board. Shelf manager board shall be standard ShMC board designed for ATCA shelf management controller.
microTCA is a small form factor alternative for ATCA and provides most of the features of ATCA that are required for ASN GW. microTCA is cheaper alternative in terms of cost and power budget. The only disadvantage is that there are not many vendors providing commercial microTCA platform. microTCA uses AMC boards and provides interconnectivity among the slots for data exchange through Ethernet (SERDES Lines at least 20). Otherwise, from software point of view, microTCA and ATCA backplanes more less compatible. microTCA shall be an ideal choice for ANS GW.

ATCA Based ASN GW: Software Architecture

ASN GW functionality can be broadly divided in three components on the basis of nature of functionality, scalability, performance and availability:

1. Control Plane
2. Data Plane
3. Management Plane
4. Basic Software Platform

Control Plane

Control plane of the proposed solution is shown Figure 2.

Figure 2:ASN GW Control Plane

Logical Interface Management

ASN GW shall have several types of physical and logical interfaces. This module will be responsible for creating interface, destroying interface and modifying the interface properties (MTU, Cost, IP address etc). Following interfaces will be supported.

• Network Interfaces : For example, Ethernet over copper, Ethernet over fiber
• VPLS: This will be a logical interface for transporting Ethernet traffic in CSN, for example, to a corporate. This can also be used as a transport for R3 interface. 9
• Aggregation: This will be an aggregation of several Ethernet interfaces for enhancing the reliability and capacity. Functionality of this interface will be as defined in IEEE 802.3ad. This is proposed to be used for R4/R6 interface in this solution.
• R6 Interface: This will be a logical interface for exchanging user data traffic between BS and ASN GW. This interface will be a tunnel interface, for example GRE, VLAN (MS aggregate, or per MS flow).
• R4 Interface: This will be a logical interface for exchanging user data traffic between relay data path (DP) entity and anchor data path (DP) entity. This interface will be a tunnel interface, for example GRE.
• R3 Interface: This will be a logical interface for interconnection of NAP with CSN. This interface will be a tunnel interface, for example MIP tunnel, VPLS.

L2 Bridging/Forwarding

User data path will implement Forwarding Data (FWD) Table which will be used to learn source mac address and forward packet on the basis of learned mac entries (destination mac lookup) in this table. This table will exist per VLAN for security reasons. Control plane function associated with this will include, entry aging, addition/deletion of static entries and controlling few specific forwarding behaviors (user to user switching, broadcast, multicast etc).
This module will also implement standard LACP functionality as defined in 802.3ad.

L3 Routing Core

L3 routing core will consist of routing protocols for establishing the L3 routing table. L3 routing will be actually performed by the user data path function (in Network Processor) on the basis of the routing table. Routing table will be generated by the Route Manager on the basis of routes learned from several sources (RIP, OSPF, and Static) and route attributes (cost, learning preferences, age etc). Route table will be used for routing packets into NAP and CSN if L3 transport is used for R3, R4 and R6 interfaces.
ASN GW will not distribute any routing prefix from NAP to CSN and vice versa. It will also not distribute MS address prefixes. It will support both IPv4 and IPv6 functionality.
MPLS LSP will enable setup of LSPs for providing VPLS interfaces and QoS enabled paths. Both static configuration and OSPF extension for sharing MPLS labels can be used.
L3 Routing Core is an optional function and in most of the cases can be performed by adjunct router. However, for deployment and costing reason, this functionality should be integrated in ASN GW.

IP Address Management

IP Address Management will manage the address (IPv4/IPv6) allocation for MS. MS discovers its IP address during initial registrations either through DHCP or MIP. MIP registration can be initiated by the MIP located on the MS (MIP enabled MS) or PMIP (Proxy MIP) client located on ASN GW. ASN GW determines the MS capability by sending FA advertisements as soon as R6 data path is established (ranging is done).
MS IPv4 address/IPv6 address prefix (HoA) can be provided by one of the following methods:

• AAA server during authentication
• Fetched from DHCP server
• Provided by HA during MIP registration

DHCP relay is used when MS initiates DHCP and address need to be retrieved from DHCP server. DHCP Proxy is used when MS initiates the DHCP and address is available during authentication from AAA server.
A filter for allowing traffic on R3 interface for MS IP address (UL/DL) will also be deployed in used data path.
IPv6 router function will implement IPv6 router functionality as defined in IPv6 RFCs (1) (sending unsolicited RA, replying to RS, providing link and site local prefixes).
Proposed ASN GW will implement standard functionality of all these modules being coordinated by MIP control and other external modules, for example ASN GW session Management.

DoS/Security

In order to hide MAC address of the MS from being known in the external network and avoiding broadcasts in the WiMAX network, ASN GW will implement the Proxy ARP function (2) and ARP Relay. Proxy ARP will reply to the external ARP requests for MS. ARP relay removes the MS MAC from the ARP requests generated by MS.
Option 82 (3) will be used for hiding MS identity in DHCP discover or RADIUS access request. In such a case MS identity specific field (e.g. source mac in DHCP DISCOVER) will be replace by ASN GW identity and an additional information element will be added to message (commonly known as option 82). Option 82 may be filled as defined in 7 .
Proposed implementation shall also provide user defined filters for allow/deny/monitor MS specific UL/DL traffic. Monitor function will copy the qualified traffic onto a monitor interface. This method can also be used for LI (Legal Intercept), that is, monitor interface will work as an IAP (Intercept Access Point).

Multicast/Broadcast Control

It will deploy multicast/broadcast specific filter rules in user data path. Following rules will be supported:

• Allow/Disallow downlink broadcast : Per ASN GW(Anchor/Relay), Per MS, Per BS, Per group of MS
• Allow/Disallow downlink multicast : Per ASN GW(Anchor/Relay)

Proposed implementation will provide IGMP Relay/Proxy (4) for assisting MS in joining the multicast group and using the resources in optimal way, i.e. media splitting near to MS.
A future extension of this could be to implement per channel access restrictions, channel priority and bandwidth distribution.

IP Applications

These applications will be used by control plane entities for DNS and AAA. It will also hide protocol specific information from the invoking applications, for example use of DIAMETER in place of RADIUS.
Accounting Control
Both offline and online charging will be implemented. ASN GW specific offline charging information will be collected in user data path for volume based charging and retrieved by the offline charging control plane module. Duration based charging information in UDR (usage data record) will be provided by the offline charging module.
In case of on line charging, online charging control plane module will get the MS credit from AAA server and configure in user data path. Volume based rating measurements will be done by the data path. Time based rating will be performed by the control plane entity. Once the credit expires, data flow will be stopped.
Authenticator and Authorization
Authenticator will be responsible for interacting with AAA server and get the MS authenticated at the time of registration. Whole process will be coordinated by MS Ranging and Registration module.
Admission control will allocate the resources for MS at ASN-GW and BS as per the local policies or MS specific policy retrieved from policy server. Basically, it will implement the SFA anchor and relay functionality. On the basis of requested resources, available resources and subscribed resources, admission control function will admit MS flow. MS subscription will define resource requirements in terms of bandwidth parameters (max, peak, sustained etc), latencies, delays and jitter. These parameters will be translated into the scheduling and queuing parameters for resource allocation. This module will also implement interface towards Authorization Module (AM) on BS.
Key manager will generate secondary keys from the MSK. (Master Session Key) and functions like key distributor. In future this module will also be responsible for managing keys for user data session keys, for example IPSec session keys. For key generation, security accelerator coprocessor may be used.

Radio Resource Management

It will be an optional function and will implement one or more of the RRM profile (Radio Resource Controller function) as defined in standard. However, in profile C case this functionality will not be implemented on ASN GW but on BS.

ASN GW Session Management

It will implement high level control functionality which will coordinate underlying control modules for end-to-end control flows. Following will be the major functions performed by this module:

• MS Ranging and Registration: It will control learning MS capabilities, setup R6 bearers, MS authentication and authorization, IP address assignment, R3 bearer and configuring the user session in user data path through data path control. To optimize performance, data path control may allocate different ASN GW IP (that is card) for terminating the R6/R3 bearer. It will also manage the MS session state and configure the user data path and update the MS context manager.
• MS context manager: It will provide interfaces for create, delete, modify and read MS context. MS context is a critical data and will be kept in failure safe memory areas, for example replicated to the backup card. It will also provide interface to BS for retrieving MS contexts.
• HO Control: It will control both micro (R6) and macro (R3) handoff. During R6 handoff, it will select the target BSs from the list of BSs in the HO messages (MOB_MSHO-REQ); complete the HO with target BS (authentication, new R4 or R4/R6 bearer setup, buffering/bicasting etc). During HO, it will configure the user data path for bicasting/buffering to minimize the data lose during HO.
• R3 handoff will be initiated by this module on the basis of certain configurable policies, e.g. resource utilization on CSN interface. Since proposed implementation will has an option of multiple IP addresses (CoA addresses), one per network card in multi card scenario, R3 handoff may or may not involve FA re-anchoring.
• Data Path Control: It will be responsible for setting up user data bearer in user data path. It will also keep track the resource utilization in user data path and will choose the most optimal instance of user data path for setting up the bearer. It will also responsible for choosing the right attributes for the bearer (transport, type of bearer Ethernet or IP etc) and will setup the proper switching of intra NAP bearer(R4, R6 bearers) to CSN bearer (R3). It will also setup the relay paths on non anchor ASN GW for control traffic, e.g. HO.
• Paging Controller: Paging function will triggered when user data path receives a packet for idle R6/R4 session. It will implement standard paging functionality and location register (LR). Location Register will be implemented in failed safe memory.

User Data Plane

User data path for the proposed solution will be designed assuming Network Processor based implementation. User Data Path architecture is shown in Figure 3.

Following are the basic characteristics of the proposed architecture:

• Inter NP communication blocks will allow splitting the processing among multiple NPs located on the same card or interconnected through fast switching backplane.
• It will support both type 1 and type 2 data bearer for both Ethernet CS and IP CS.
• It will support control plane independent relay of control traffic in a non Anchor case.
• It will support IP, VLAN, Aggregation, MPLS, VPLS options for transport of Wimax data bearer.
• It will support per MS, per MS flow traffic scheduling.
• It will support both bicasting and buffering configurable options for data integrity during HO.
• It will support volume based both online and off line charging models.
• It will provide policing and filtering to control malicious use of the network.

Management Plane

Management plane of the proposed solution is shown in Figure 4.

• It will provide CLI, Web and SNMP interfaces for managing the ASN GW.
• It will provide configuration management operations like: save, apply, apply-default, upload, download.
• It will provide user defined filters for controlling alarm generation and reporting.
• It will provide RMON type performance management. Only current and history group will be implemented.
• It will provide shelf management functions for:
• Managing software: Application software, Operating system, Booters. Recovery from un-complete or incorrect upgrades.
• Managing insertion/removal of cards and other FRUs.
• Managing the HUB and physical interfaces.
• Controls the system failover, that is, when to initiate switch over.

Basic Software Platform

Basic software platform (Figure 5) will provide underlying hardware and OS services to the higher level applications (control plane and management plane application). Basic software platform will provide portable API interface for the higher level application for hardware and OS specific features.

Operating System

RT Linux is being widely used in telecom products these days. This is primarily because of cost and availability of software components that can be re-used. For the proposed solution RT Linux is recommended as the OS for all the boards/Processors.
Other choice can be any commercially available OS, such as VxWork, LinuxWorks etc.

Infrastructure Software

It will provide following services:
Inter Process/Processor Communication: It will enable application to application messaging and hide the physical location of the communicating applications. For inter process communication message queues will be used. Inter processor communication will use UDP for the message exchange.
Debug/Trace: It will provide user configurable and multilevel tracing frame work. Trace level can be set through management interface. For trace output, syslog is used.
Drivers: Most of the drivers will be available either part of Linux distribution or board vendors.

NPF Compliant API

Network Processor Forum (NPF) has published few specifications for defining API to interface network processor based user data path, for example IPv6, IPSec. In addition to this NPF has also published API framework for developing APIs for other network processor based user data path. This will help in development of portable control and user data path application.
NPF compliant API shall be developed using the NPF API framework and other published APIs. In addition to this, it will also hide multiple instances of the user data path, for example R4/R6 and R3 user data processing located on two different network processors on same or different cards.

High Availability

It will provide frame work for building 1:1 redundancy for all the hardware boards. There are SA Forum (Service Availability Forum) Compliant HA frameworks available from Motorola (NetPlane), GoAhead (SelfReliant) etc. These platforms provide most of the functionality required and can be extended as well.

Software Components Mapping to Hardware

Shelf Manager Board

It will be a standard ATCA board implementing ShMC (Shelf Management Controller) function. There are two possibilities.

1. ShMC is mounted on Control Processor board itself. In such a case separate Shelf Manager Board will not be required. Whole of Management plane along with Control Plane software (exception to this are highlighted in sections 2.3.2 and 2.3.2) will be located on this board itself and separate control plane processor board is not required.
2. Shelf Management is done through dedicated board (proposed solution). In such a case Management Plane software can be ported to Shelf Management Board. This will be more scalable option.

BS Termination Board/CSN Termination Board

These will be identical and network processor based ATCA boards. Network Processor should be capable of processing 2Gb (line rate) traffic for small PDU size (typically 64 bytes). Agere (APP5xx), Broadcom (BCM 12xx), and Intel (IXP 28xx) are the possibilities. For network termination 2x1Gig Ethernet interfaces will be provided. For BS termination n x 100M Ethernet interfaces may work well.
There are COTS NP ATCA boards available in following configurations:

• Dual NP: In such a configuration both UL and DL user data path will be implemented on the same board. This means that two cards will be sufficient for both CSN and BS termination.
• Single NP + General Purpose CPU: In such a configuration Ingress of UL and Egress of DL will be implemented on BS Termination Card. Ingress of DL and Egress of UL will be implemented on CSN Termination card.

If General Purpose CPU or the CPU core is available on termination board then it will be used to perform part of the control plane processing.

Control Processor Board

It will be a standard COTS ATCA CPU board. CPU can be 603e, Intel Pentium 4, or XEON and 1Gbyte of memory. Following control plane functions will be located on the control processor board.

• ASN GW Session Management
• Accounting Control
• Authentication and Authorization
• For Key generation, if hardware accelerator present on the network termination board then it can be used for key generation.
• IP Applications
• Radio Resource Management
• L3 Routing Core
• DoS/Security
• Option 82 and User Defined Filters

Network termination board may also have CPU core. This core will be used for implementing other functionality.

• DoS/Security
• ARP Relay on BS Termination Board
• ARP Proxy on CSN Termination Board
• IP Address Management
• MIP Control, DHCP Relay/Proxy, and DHCPv6 on CSN Termination Board
• IPv6 Router Function on BS Termination Board
• L2 Bridging/Forwarding
• On both CSN and BS Termination Boards
• Logical Interface Management
• Network Interfaces on both CSN and BS Termination board
• Aggregation on BS Termination Board
• VPLS on CSN Termination Board.

System Startup
It will consist of the following major steps:

1. Slot power on: ShMC power on all the configured slots through IPMB. To conserve the power, un-configured slots will be kept in power down state.
2. Board boot up: After power is applied to the slot, hardware board (assuming it was al ready present in the slot) will in to power on cycle. First booter will come up, which will download the active application image. Since the application image for a card will not going to be very big, it will be kept in onboard flash.
3. Application start: Application Image start will be three phase start. In first phase Basic software platform will be activated and user data path software will be downloaded and initialized. In second phase role arbitration will happen among the members of the redundancy group. In third phase complete software as per the role (Active/backup) of the card will be activated.
4. Configuration: On recognizing successful start of card in a slot, shelf manager will configure the board as per the available configure.

Scalability and Availability

In ASN GW following will impact the system scalability:

• Concurrent MS registrations
• HO, Paging and buffering
• User data on R3, R4 and R6 interfaces

In the proposed solution ASN GW functionality will be distributed among the processing engines in way that there will be minimal cross impact of the processing. Solution can further be scaled by having more then one instance of control plane processor boards and network termination boards. From hardware points of view, this will be very easy to achieve in ATCA system since all the slots are identical. From software point of view, every instance will have a unique IP and MS context manager will be located at central place. BS will initiate communication with configured ASN GW IP. This IP will be changed to the allocated control plane and data path instance in reply by context manager.
System architecture will provide 1:1 redundancy for all the components.

Performance

Performance, of the solution, will be evaluated after firm selection of hardware boards. However, initial estimation indicates that user data path of the proposed solution with Intel IXP2800 can process full duplex 1Gig traffic for 64bytes PDUs.

Manageability

This section provides the description of few aspects of manageability. Other aspects are addressed in Section 2.2.3.

Configuration

A hierarchical scheme for system configuration will be used. It will be possible to configure through CLI/SNMP/Web. Following describes the major configuration steps to be followed.

• Configure the Shelf: For example MIB II information.
• Configure Slot: slot attributes: e.g. kind of card it can receives, Type (BS/CSN), startup HA role etc.
• Configure Cards: Network Interfaces, serial port, software version, temperature thresholds, back plane interfaces etc.
• Configure Application :
• System Wide
• CSN parameters: DNS server, DHCP server, AAA server etc.
• NAP parameters: BSs, Neighboring ASN GW
• Local Authorization Policies
• Security/DoS configuration
• L3 Routing
• MS specific configuration (?)
• Card/Slot Wide
• setup logical interfaces: Aggregation, VPLS, VLAN
• setup VLAN and bridging attributes

Software upgrade

Software release for ASN GW will consists of few or all of the following components.

• Booter image
• OS Image
• Application Image for Shelf Management board
• Application Image for Control Plane Processor Board
• Application Image for Network Termination Board
• Single self configurable image on the basis of slot configuration. This will enable a pizza box version where CSN and BS functionalities are performed on the same card.
• This image will also have user data plane software.

All cards will have an application flash sufficient enough to store two application images (Active, Backup). Software upgrade process will be initiated through management action and will be controlled by the Management Plane (software).

1. Download release from the ftp/tftp server. If release is preloaded then this step will be skipped.
2. Analyze (correctness, compatibility etc) the release components, system inventory and prepare a list of the cards/slots need to be upgraded.
3. Start upgrading the cards/slots. It will use the redundancy to minimize the service downtime.
1. Download all the changed software images
2. Set the Backup area of the flash to Active
3. Reset the card

There will be a recovery/field repair mechanism provided for aborted or failed upgrade.

Implementation Approach

Use of software and hardware COTS component is a must for quick realization of the proposed solution. Following COTS components are proposed to be used. However, a detail feasibility analysis needs to be done.

• All the hardware components
• Software components
• LACP
• Route Manager and Routing Protocol : Zebra (Freeware)
• FA, DHCP Relay/Proxy, DHCPv6, IPv6 Router Function : Linux/NetBSD/KAME (Freeware)
• IGMP Relay/Proxy: Linux (Freeware)
• IP Application : Linux (Freeware)
• SNMP Agent : UCD Tool kit (Freeware)
• CLI : CLI Tool Kit (Freeware)
• Web : Web Tool Kit (Freeware)
• HA : NetPlane/SelfReliant
• Board Support Package (BSP) : Hardware vendors

Implementation of the entire system will be done in following phases for early availability of the product, optimized cost/effort and reducing risk.

1. Basic System: System shall support L3 transport for Wimax bearer, authentication and admission control, micro mobility, L3 routing, support of CMIP and PMIP, ARP Proxy/Relay, CLI/Web based management
2. Redundancy: System shall additionally support 1:1 redundancy, MPLS support, accounting, User defined filter, Multicast/Broadcast
3. Macro Mobility: System shall support all the features.

References

[1] RFC 2460 - Internet Protocol, Version 6 (IPv6) Specification, S.Deering and R.Hinden, 	     December 1998
[2] RFC 1027 - Using ARP to Implement Transparent Subnet Gateways, Smoot Carl-Mitchell and  John S. Quarterman, October 1987
[3] RFC 3046 - DHCP Relay Agent Information Option, M. Patrick, January 2001
[4] RFC 1112 - Host Extensions for IP Multicasting, S. Deering, August 1989
[5] WiMAX End-to-End Network Systems Architecture (Stage 2: Architecture Tenets, Reference Model and Reference Points)
[6] WiMAX End-to-End Network Systems Architecture (Stage 3:  Detailed Protocols and Procedures)
[7] TR-101 – DSL Forum, “Migration to Ethernet-Based DSL Aggregation”
[8] PICMG 3.0 Short Form Specification

This page has been viewed times.

 Create your own educational wiki!Help Add to Del.ici.ous Add to Digg Add to Reddit Add to fUrl Add to SpurlPublic, Platinum wiki Privacy Policy RSS