HSC welcomes all external visitors to this site, especially students and members of the academic community. Please use the comments box at the bottom of each page to record any comments or suggestions for improvement. ATCA

Introduction

Recent trend for telecom networking has been driven by speed. Due to this trend, the architecture is moving away from shared resource of a bus towards point to point connectivity between processors and data resources. There are several reasons for this

• Multiplexing between data sources is much easier and faster to do within a silicon bridge chip, switch or a processor then it is across a backplane or a cable.
• Data rates achievable across a bus are limited by difficulties of maintaining equal round trip time times for each line of the bus and compensating for the signal integrity issues of driving a partially loaded bus equally as well as fully loaded one
• Wider the bus becomes more the pins required on silicon chipthis makes the system expensive
• Bussed system do not scale as extra processors are added since the available I/O bw is both limited and shared.

The migration of point to point switch is based system is now taking place because improvements in signal processing have overcome many of the difficulties of digital transmission through copper interconnects.

Overview

ATCA is an Advanced telecommunication Computing Architecture. ATCA is also known as the PICMG3 family of standards. ATCA standard defines mechanical form factor , power and cooling parameters, backplane interconnects, system architecture necessary to construct a compliant back plane, chassis and plugin boards. It also defines base fabric for system control and management. ATCA is valuable as the basis for a standards-based modular platform, on which many applications can be built. Primary focus of ATCA is Telco Carrier Grade applications based on standard fabric solutions, while other application can be as a modular servers.

Why ATCA ?

The ATCA 3.0 based specification defines physical and electrical characteristics of an off the shelf modular chassis based on witch fabric connection between hot swappable blades. The ATCA based base specification. supports multiple fabric connection . Despite the advent of the highly successful compact PCI standard, the fact remains that the majority of the telecommunications market has no standard form factor, back plane or fabric interconnect that can the 10 Gbps bandwidth requirement today. There continues to be tens of different chassis designs in the telecommunication industry that drives the cost of this equipment higher prevents multi sourcing and interoperability across a common back-plane of fabric. ATCA is attractive because it helps in leveraging off the shelf subsystem, can actually outstrip the competition in the performance and breadth of product offering.

PICMG 3.x Standards

PICMG 3.1 :Ethernet and fiber channel switch fabric over the general fabric interconnect. It provides data rates upto 10 Gbps per user PICMG 3.2 :Defines how InfiniBand? systems are built within the architecture and will specify link physical layers, protocols and protocol mappings. PICMG 3.3 ;Defines a StarFabric? implementation over the backplane providing TDM, cell, control and packet connectivity over the same fabric. PICMG 3.4 :Defines how PCI express and PCI express switching Advanced switching transport is mapped onto the generic backplane fabric interconnect PICMG 3.5 :RapidIO

Basic Block

Figure below shows the very basic ATCA block

Four of the popular PMC daughter board footprint (not a part of the spec) B: Rear transitional module (allows mounting of external connectivity from rear of chassis) D: Connector for access to the transitional module E: Connector that connects main board to the backplane data and control transport connection F: Backplane G: Power Connector Backplane carries following interconnects Shelf management : Management of chassis contents(bcoz equipment could be from various vendor whose I/O might not be compatible therefore needsto be verified) Base Interface : Logical slots 1 and 2 dedicated to being redundant hubs for dual star interface using a 10/100/100 baseT Ethernet interconnect to every other slot. The base interface offers a medium speed control path and parallels the high speed interface Synchronization clock interface : three clock across each slot two of them are sonet SDH clock (8 Kz, 19.44 Mhz) third user defined. Update channel interface : Each board has 10 differential pair connecting it to neighbors. These are to be used for proprietary use with proprietary protocol. Fabric Interface: two differential transport architecture(Dual Star and Full Mesh)

Salient Features

• Switch fabric technology : The shared bus architecture is not capable of meeting these requirement. In order to overcome these limitation switch fabric architectures have emerged where each I/O input can make temporary connections to any of the I/O outputs. These connections are made through sophisticated switches which can offer advanced feature including QoS, flow control, integrated management, fault recovery and scalability.
• High speed fiber, ATM, IP and DSl broadband connection to data centers and central offices have increased the bandwidth requirement beyond a few gigabits/sec that can be supported by shared bus architecture such as compactPCI.
• Aggregate BW : Scalable capacity of 2.5Tbps per chassis
• Individual BW : 40gbps
• System availability :99.99
• Broad range of interfaces and services are required
• Multi vendor support
• Converged voice data server storage video and wireless function
• Platform architecture that would be able to meet the increasing demands for scalability, bandwidth, availability and performance
• Delivering availability and fault tolerance requires redundancy in every subsystem.

ATCA Components

• Chassis
• Power supplies
• Chassis management module
• Cooling
• Backplane
• Rear transition module

The ATCA backplane and management modules are standardized and open thus any vendor can produce ATCA compatible boards. The boards may be tailored to

 -Compute
 -Network processing and I/O
 -Fabric switch storage
 -Digital signal processing
 -Radio/Wireless/Controller/Receiver

Architecture

ATCA Boards : Individual I/O or computing blades that are hot insertable and removable in a shelf Shelf: The 12U tall chassis that provides power, cooling, backplane connectivity, and the slots to accept up to 16 boards Frame: A rack (typically 46U high) provides a rigid frame work accepting upto 3 shelves and commonly deployed within enterprise and internet data centers and telco central offices. Passive Backplane and switch fiber connectivity : ATCA defines passive backplane and details the location of high-speed fabric connectors, management connection, power connectors, alignment key structures for mechanical integrity and electrical keying to support different types of boards. The ATCA specification also defines the backplane connectivity, which can support either a full mesh architecture or a dual star connection. The 8U board height allows upto 4 standard PMC card to be accessed via the front panelBoard area is approximately 140 square inch of area allowing multiprocessor design accommodating large amount of memory.The back side of board has power connectors, alignment keys, rear I/O access, and high speed connections to the passive backplane

An ATCA backplane is divided into three zones The Zone 1 connectors are for power and system management. This zone provides for system management of chassis is based on dual I2C buses and the IPMI protocol. ATCA has defined a number of standard sensors and commands for platform environment, and blades developed for the environment wil be readilty manageable for these. The zone 1 connector is optimized for power functions and heavy duty pins and a nalignemnt post to form the lower edge board alignment The Zone 2 connectors are for data transport. Zone 2 can be subdivided into

  -  a Base Interface
  -  Fabric Interface
  -  Update Channel Interface
  -  Synchronization Clock Interface.

The zones base frame is based on dual star 10/100/1000 Base-T Ethernet networks while the extended fabric consists of full mesh(for 16 boards). Each channel has four links, alloeing for rates of 10 Gbps pr higher per channel. Using the full mesh the extended favric has capability of 2.4Tbps. ATCA allows the extended fabric to be implemented on full mesh or a star depending on the application. The Base Interface carries Ethernet traffic in dual star configuration only. The Fabric Interface is more flexible. It can support a number of standards, including Ethernet, Fibre Channel, Infiniband, StarFabric?, PCI-Express, Advanced Switching, and RapidIO. It can also be configured in a dual star or mesh topology. In a dual star topology, logical slots number 1 and 2 host the hub boards, while the remaining slots host the node boards

The Zone 3 connectors are undefined. They provide a flexible method to implement user-defined proprietary I/O connections. This zone provides 95 mm of open space for rear I/O.

ATCA Highlights

• Backplane Flexibility
• Fabric Flexibility
• Maximizing Platform density
• Scalability
• Processing performance for converged application
• Mezzanines
• Manageability
• Reliability
• Serviceability
• Middleware Flexibility
• TDM support

Backplane Flexibility Backplane topology determines how blades are connected within backplane. ATCA provides more fabric interface options, provides the flexibility to interface variety of network element with ATCA.. ATCA also provides flexibility by supporting both base and fabric interface into the backplane. . This enables TEM to scale their design to new user and usage requirement by providing higher bandwidth per port using industry standard interface. Fabric Flexibility Supports multiple network ATCA supports industry standard full duplex interface including ATCA provides chassis level management based on industry standard Intelligent Platform Management Interface. (IPMI) which can be implemented in either radial or bussed topology using IPMB Update port interconnects provides special purpose interface consisting of 10X differential pair. This channel provides inter blade communication for applications including fail over and clustering ATCA provides very high bandwidth between blades Base interface BW of 1Gbps Wide range of configuration options, including system that supports bw of 2Gbps, 4, 8, 10 Gbps per port to accommodate the bandwidth requirement Maximizing Platform density Density determines the redundancy that can be economically provided for a given unit of floor space. Density is detrmined by No of blades per chassis No of chassis per frame Including thermal, power and cooling considerations. ATCA can support 14 blades and two switches per full size chassis, with a relatively large form factor that allows superior heat dissipation, supports a large number of plug in card and allows a rear and front I/O without cables that can block air flow. Scalability TEMs can easily add blades to a chassis to accommodate more traffic and to support new services, and the multi-protocol flexibility of AdvancedTCA enables it to be used across multiple networks. Processing Performance for Converged Applications ATCA supports both advanced I/O and compute capabilities, which enables it to support converged computing and communications applications based on high-performance fully programmable Intel network processors based on Intel. Internet Exchange Architecture (Intel IXA). The architecture is ideal for a range of converged applications including IP soft-switches, VoIP routers, and second-generation and third-generation base station controllers. By supporting a broad spectrum of network elements, ATCA can help equipment manufacturers minimize the time and cost of development. Mezzanines ATCA blades are designed for easy upgradeability with new processor, memory, I/O and management processor modules based on industry-standard mezzanines. Form factors include PCI Mezzanine Card (PMC) and emerging Advanced Mezzanine Card (AMC) modules. Advantages of AMC include IPMI command support for manageability, a larger thermal envelope compared to PMC and hot-swap capability. ATCA carriers support both mezzanine types with up to four PMC slots and eight AMC hot-swappable slots. Manageability The IPMI provides the basis for management on ATCA. This detailed definition in an open standard provides interoperability that enables chassis and blowers, compute blades, switches and management solutions from multiple vendors to work together seamlessly. The ATCA specification has been hardened for telecom use through the inclusion of capabilities including redundancy, failover and synchronization of critical management information between ATCA chassis management modules. Reliability ATCA ensures very low Mean Time to Repair (MTTR) and high Mean Time Between Failures (MTBF). Some proprietary architectures require active components in the backplane, which can lead to expensive repairs in the field. In ATCA architecture, there are no active components on the backplane. In addition all components can be hot-swapped in the field, and the architecture provides redundancy at all levels. To enable power management, the ATCA specification requires descriptions of chassis input power, voltages feeds and other variables to be stored in a standard format in a hot-swappable shelf field replaceable unit (FRU) device to reduce the likelihood of data loss due to backplane failure. The ATCA specification allows for redundant Shelf FRU, so that failure of a single device will not propagate to the entire chassis. Serviceability ATCA has been designed to allow all critical components such as fans, switch fabric modules, management modules and FRUs to be front-serviceable. ATCA chassis are designed with modular, redundant/hot-swappable fan trays that eliminate fans as a single point of failure within the chassis. ATCA provides two mechanisms that help prevent blades, boards, and other chassis-level components from being accidentally plugged into the wrong slots or misaligned with their connectors. The first protection is known as electronic keying (E-keying). E-Keying compares chassis configuration data which is stored in a Chassis Data Module (CDM) shelf FRU device, together with data from a boards IPMI controller. As the board is inserted into a shelf, its IPMI controller powers-up, identifies the board and determines whether or not to power-up the board. In addition, ATCA specifies four mechanical alignment pins to ensure that boards are plugged in consistently eliminating the possibility of pin misalignment. Middleware Flexibility Management middleware provides services that aid in monitoring and managing the system components for maximum service availability. ATCA supports industry standard interfaces, including the SA Forum Hardware Platform Interface (HPI) which enables integration with a variety of COTS middleware solutions. TEMs can use HPI with their own high-availability middleware to reduce development and test time for hardware modifications and upgrades. TDM Support A large proportion of the installed base of telecom infrastructure equipment continues to depend on TDM inter-chassis synchronization clocking, and this requirement needs to be addressed by next-generation communications architectures. ATCA provides the flexibility to support both legacy and next-generation network elements. TDM synchronization clocks are fully supported within the ATCA specification and vendors can implement them in a modular fashion. ATCA requires three synchronization clock signal pairs on all backplanes with redundancy, but their use is optional for the vendor.

Comparsion of ATCA with Bus based standard

Synchronization in ATCA

ATCA forms its Synchronization Clock Interface architecture using a set of six differential clock buses on the Zone 2 connector of the ATCA backplane. These buses are divided into three groups of two differential pairs (CLK1A/B, CLK2A/B, CLK3A/B), and are used by the system to exchange redundant network synchronization information to multiple boards in the shelf. Depending on its individual needs and functions within the system, a board may interface to any, all, or none of the available synchronization buses. Boards that interface to the buses should support the following requirements:

 Hot-swap
 Maintain synchronization integrity during short duration
 glitches
 Present a high impedance to the bus, unless authorized by
 the shelf manager to drive the bus;
 At least one bus out of a redundant pair group must remain
 active and uninterrupted during a single board failure
 Provide and/or accept MLVDS differential clock pairs to/from
 the backplane.

CLK1 Buses The CLK1A and CLK1B buses carry a pair of redundant 8 kHz system clocks/frame pulses. CLK1 frame pulse must have a minimum pulse width (high or low) of 122 ns, which is equal to one period of a 8.192 MHz clock. The rising edges of the A and B clocks must be phase-aligned within the ATCA skew requirements which allows a nominal 10 ns skew on the backplane between CLK1A and CLK1B. To account for PLL phase corrections, additional allowances are made to bring the maximum skew in any one clock cycle to 30 ns. CLK2 Buses The second ATCA clock synchronization bus is dedicated to a redundant pair of 19.44 MHz system clocks that are commonly used for SONET/SDH networks. The 19.44 Mhz MLVDS clocks on CLK2A and CLK2B should have a duty-cycle of 50% +/-10%. The skew requirements for CLK2A and CLK2B are identical to the CLK1 requirements shown in Figure 1. CLK3 Buses CLK3A and CLK3B are user-defined buses that enable centralized master system clock sources to receive external network reference clocks from a slave board located in the ATCA shelf. The CLK3A/B buses can be either a user-defined clock of frequency ARCHITECTURE ATCA is designed to support two basic architectures

• a centralized system clock source synchronization model
• a distributed synchronization system model
• centralized System Clock Architecture

The centralized system clock architecture employs a pair of dedicated master timing cards to deploy the redundant system clocks to the rest of the system. These master timing cards are equipped to receive external network references from any of the other boards in the system via the CLK3A and CLK3B sync buses or from their own external network reference interface. The timing cards synchronize to the appropriate network reference (as selected by the shelf manager) and distribute the system clocks via the CLK1 and/or CLK buses for use by the slave cards located in the shelf. Which of the A or B buses are chosen as primary or secondary is arbitrary and will change during normal operation

Fig : Centralized System Clock Architecture Timing Functions Redundancy Protection The ATCA architecture provides redundancy protection. The A and B buses should be driven from separate boards to ensure robust redundancy protection. System clock failures can be detected in two ways. Hard failures of the active system clock should be detected independently by each slave card. Soft failures or out-of-range failures should be detected by the master clock sources. If the slave card detects a failure (the absence of their primary/active reference) it must seamlessly and automatically switch to the secondary clock source. If the primary clock source is deemed out-of-specification by the redundant or secondary master card, then a procedure must be initiated to convert the secondary master timing card to the primary master timing card and stop the current out-of-specification primary clock from driving the sync bus. This then forces the slave cards to stop using an invalid clock source for their synchronization. Steps for conversion of secondary to primary card : 1. The secondary card should be placed in holdover immediately upon detection of a failure on the primary clock. Since the secondary clock is locked to the primary clock source, this ensures at least one set of the system clocks being driven onto the sync buses remains valid; 2. The PLL on the secondary card should be switched from a high bandwidth mode, which was necessary to track the movements of the primary clock and maintain the skew requirements between the A-B buses, to a low bandwidth mode that can provide the necessary standards compliant filtering. 3. The secondary PLL switches its active reference from the primary system clocks (the CLK1 or CLK2 buses) to a valid network reference from the CLK3 bus; and 4. The secondary card is taken out of holdover and now assumes the responsibility of the primary master timing card. This procedure should take place without causing problems on the slave cards due to transients caused by mode and reference switching. Master Timing Card Function The primary master card locks to one of the external network references (as selected by the shelf manager based on quality and availability) received from the CLK3A and CLK3B synchronization buses, and then provides standards compliant system clocks onto the active system clock buses (CLK1A/B, and/or CLK2A/B). The secondary master card locks to one of the primary system clocks from the active system clock bus, and provides clocks to the redundant system clock buses that are frequency and phase aligned. The design of a master timing solution for ATCA is complicated by the fact that it must be able to perform as both primary and secondary masters. When configured as a primary master, it may have to meet the strict Telecordia GR-1244-CORE Stratum 3 standards for network synchronization. When configured as a secondary master, it must continuously track the reference and maintain acceptable phase alignment in the presence of jitter and wander. The master timing solution must be able to lock to and monitor the incoming network references on the CLK3 buses and perform hitless reference switches between them in the case of a detected failure. It must provide standard compliant jitter and wander filtering,. Line Interface Function Boards that have access to an external reference source, can be configured to distribute the network reference via the CLK3A or CLK3B buses. The shelf manager decides which boards actively drive these buses. However, the boards that supply the external network references to the synchronization buses are responsible for monitoring its external network references and driving the bus only with a valid reference. Slave Card Function These boards can lock to either of the system clock buses and provide rate conversion and jitter filtering in order to supply the necessary local clocks. In the event of a failure on the active references, these cards must switch to the alternate system clock bus for synchronization. Therefore, these cards must monitor the CLK1 and CLK 2 bus clocks, automatically determine which ones are valid and lock to them. No long-term holdover is required on the slave cards, since the redundancy protection of the master timing cards ensures the presence of a valid reference at all times. This allows the use of a simple, non-compensated oscillator in the slave timing solution.

Distributed Timing Architecture In a fully distributed architecture, there are no system clocks provided on the CLK1 and CLK2 sync buses. Instead, any board in the system that requires network synchronization uses the network references from the CLK3 buses to generate all of their local clocks. In this model, boards are frequency locked, but do not have phase synchronization. It is also possible to use a combination architecture, where two boards in the system provide the system clocks on the CLK1 and CLK2 buses. In this architecture, only some of the slave cards use these references to generate their local clocks, while other boards use the external network references to generate their local clocks.

Fig : Distributed System Clock Architecture Timing Functions Timing Solutions in Distributed ATCA systems As there are no master timing cards distributing system clocks that meet the appropriate network synchronization specifications, the individual boards in the system must implement a timing solution that meets its own network synchronization requirements. Because of this, a system that has a greater number of boards requiring a high level of network synchronization, such as Telecordia Stratum 3, may be better suited to take advantage of the centralized architecture in order to minimize costs and complexity. However, a system that has a limited number of cards requiring any network synchronization, or a system with relaxed synchronization requirements, may find a distributed architecture to be simpler and cost-effective.

References

{1}AdvancedTCA PICMG 3.0 Short Form Specification January 2003 {2}http://www.picmg.org/pdf/Supercomm_Tutorial.pdf {3}http://www.hep.ucl.ac.uk/lc/calice/DAQ/Meetings/All_060109/Hill_ATCAOvrvw.pdf {4}Distributing Network Synchronization in ATCA Systems: Part 1 by Marvin Ng {5}Distributing Network Synchronization in ATCA Systems: Part 2 by Tyler Bailey {6}AdvancedTCA and Sun Microsystems: A Technical Overview of Suns AdvancedTCA

Maintainer: rajenish.jain@hsc.com Views: ###

Comments

Add Comment
Email address(will be kept hidden)
Enter code: