Ethernet Compliance Testing at Toradex
Toradex offers robust and reliable embedded systems, which are required to work continuously in harsh environments. Ethernet is one of the most important interfaces for the Internet of Things (IoT). We will review some Ethernet standards and show you how Toradex tests for compliance with them.
After looking at the standards, we’ll describe our test configuration, test procedures, and the test results. A Colibri iMX6ULL SoM and Iris Carrier Board were used in this example, but you can use this as a model for testing custom carrier boards if that testing will be part of your verification process.
Why We Use Standards and Do Compliance Testing
Ethernet designs adhere to the IEEE 802.3 standard, which defines the Physical and Data Link layer of the seven-layer Open Systems Interconnection (OSI) model. Waveform characteristics are specified in the standard. Designing to this standard allows compatibility and interoperability with other devices, in all kinds of environments all over the world. Otherwise, transmission issues and data losses are likely to occur. Compliance testing ensures that the implementation meets the standard.
In addition to the waveform characteristics specified in the IEEE 802.3 standard, the University of New Hampshire InterOperability Laboratory (UNH-IOL) has provided standard conformance test procedures for those signals.
The documents can be found at these links:
https://ieeexplore.ieee.org/browse/standards/get-program/page/series?id=68
Ethernet Physical Layer Basics
The Ethernet standard is several thousand pages, so we’ll just cover the most important concepts and some key terminology.
Figure 1 OSI Reference Model from IEEE Standard for Ethernet
Let’s start with the physical medium. Signals typically arrive through a twisted pair copper cable to an Ethernet jack with magnetics on our Carrier Board, then continue through impedance matched differential traces on the PCB to the Ethernet PHY IC. This device converts analog signals from the medium to digital signals for the processor and vice versa.
The electrical signals first encounter the Medium Dependent Interface (MDI) of the PHY, a part of the Physical layer. Different physical media have different characteristics. In accordance with the specific kind of the media, the signals are transformed and sent to the next layer of the OSI-model, the Data Link Layer. We provide 10Base-T and 100Base-TX (Fast Ethernet) on our Colibri Modules and 1000Base-T (Gigabit) on the Apalis modules. The standardized interface between the first two OSI-layers is called Media Independent Interface (MII) and is independent of the physical layer.
Meanwhile, we are talking about the advanced backward-compatible Reduced Gigabit Media Independent Interface (RGMII) and the next interface for the 10 Gigabit is already named as XGMII. Reduced means that there are fewer signals needed for the same standard. The xMII’s are parallel data buses. There is a supplementary serial bus for management purpose called Management Data Input/Output (MDIO). The xMII interface ends at the Media Access Control (MAC) layer. Here the well-known MAC address is used as a unique identifier. The MAC layer can be integrated with the System on Chip (SoC), like on NXP® processors. But it could be already embedded in the same IC as the PHY, which is better known as an Ethernet Controller. The Ethernet Controller IC, in turn, is connected with the SoC through a separate interface like USB or PCIe. Note that we are not looking at higher OSI layers and protocols like ARP, NDP, IP, TCP, UDP, etc., which are organized in frames and packages, because for all these protocols the electrical characteristics on the first physical levels are the same!
For now, let’s go back to the physical layer. There are 2 main characteristics of the physical link I’d like to expand on, namely, speed and duplex mode. Our modules support speeds up to 1Gbit on Apalis Modules and 100Mbit on Colibri Modules, and both half and full duplex modes. In full duplex mode of operation, PHYs on both ends of the link can communicate with each other simultaneously. For the half duplex mode, where the PHY can’t receive and transmit data at the same time, there need to use the Carrier Sense Multiple Access with Collision Detection (CSMA/CD) to avoid collisions and control the data flow.
As already described, our Apalis Modules are capable of Gigabit Ethernet, but how do the communication partners know with which speed they can send the data? An-auto negotiation procedure exists, where the linking partners set the best link trough 16ms link pulses. Please be careful with auto-negotiation settings, as there is a well-known problem of the duplex mismatch, when the linking partners are configured in a fixed way. On the electrical side of the physical layer 10Base-T and 100Base-TX use two twisted pairs while 1000Base-TX uses 4. 100Base-TX is faster than 10Base-T based on the much faster frequency of 62.5 MHz instead of 10 MHz and denser signal modulation scheme (PAM-3). 1000Base-TX uses the same frequency as 100Base-TX, but transmits across 4 twisted pairs and with a higher level of modulation (PAM-5). Finally, there is an additional feature called EEE, Energy Efficient Ethernet. The aim of this standard is to save energy.
https://www.analog.com/media/en/technical-documentation/application-notes/EE-269.pdf
https://en.wikipedia.org/wiki/Duplex_mismatch
https://en.wikipedia.org/wiki/Media-independent_interface
https://www.asix.com.tw/new_alias.php?alias=93&full=http://www.embedded.com/design/202804534
The picture below summarizes the Ethernet possibilities on the Toradex SoM approach:
Before we continue to the Compliance Testing, I want to quickly let you know that I receive a lot of questions about 100Base-T1, better known as Automotive Ethernet. The customers want to know, if it is possible to connect Automotive Ethernet to a fast Ethernet PHY. The 100Base-T1 has a different physical layer specification to fulfill the requirements in a harsher automotive environment. It is not possible to connect it, but the MII is still the same! The solution is to connect the 100Base-T1 PHY to the Multimedia Independent Interface of the SoC directly. Consequently, a Module with an xMII on the Module Edge Connector must be selected! Of course, you have to design your Custom Carrier Board with a 100Base-T1 PHY. Here you can find the list of Toradex Modules which provide an xMII on the edge connector. Please note that this is not a Standard Toradex Interface and the pin assignment varies with each module.
Ethernet Compliance Testing for Toradex Systems
Figure 2 10Base-T Test: DOV Internal MAU Normal
After this very short overview, I want to continue with the compliance testing, where we test the electrical signals in time and voltage. The electrical signals look totally different for the 10/100/1000 Mbps and have different requirements as you can see in the oscillograms.
The tests evaluate the voltage amplitudes, jitter values, rise/fall times and other signal characteristics. For each test a defined test signal must be generated by the DUT, e.g. a continuous pseudo-random signal has to be emitted. The easiest way to test the signal requirements is to define a test mask. The signals must not intersect with the mask in order to fulfill the specification. The 10Base-T tests are very often defined through a test mask, as you can see in the first figure. I want to mention some values: The Peak Differential Output Voltage must be between 2.2 V and 2.8 V. The Differential Output Voltage Harmonics must be greater than 27 dB and all Jitter values must be smaller than 22 ns. Very interesting is the twisted-pair model for 10Base-T, which must be used for some compliance tests. With the equivalent circuit based on lumped elements, it is possible to model multiple different transmission elements with just passive components. Depending on the PHY a linking partner is needed for 10Base-T compliance tests. You can download the Test Report of Colibri iMX6ULL as an example, where you can find all tests.
Figure 3 Measured differential random 10Base-T signal without load
The interface characteristics for 100Base-TX are defined in table 2 based on the MLT-3 voltage signals with three levels. The data is encoded before with 4B5B algorithm, so a clock recovery out of the data stream is possible because a level transition is forced. For our test equipment, a pseudo-random test pattern (PRBS7) is enough for all the tests. However, some tests only trigger on defined patterns and measures only at that moment the specific values.
Table 2 Interface Characteristics 100Base-TX Characteristics Min Max Unit UTP DOV Base to Upper/Lower 950 1050 mV Signal Amplitude Symmetry 98 102 % Rise/Fall Time 3 5 ns Rise/Fall Time Symmetry 0 500 ps Duty Cycle Distortion -250 250 ps Transmit Jitter 0 1.4 ns Overshoot 0 5 %
Figure 4 Measured differential random signal 100Base-TX
The test criteria are defined in a similar manner for the 1000Base-T. I am not going to list them. For these tests an additional disturber in the form of an Arbitrary Waveform Generator (AWG) is needed to create the required disturbing signals with the frequency of 31.25 MHz and 20.833 MHz. In figure 5 you can see the test pattern of test mode 1 produced by the PHY. There are four different test patterns, which can be generated through PHY’s MDIO register settings. Please don’t forget that we have to perform the compliance test four times, because of the four twisted pairs.
Figure 5 Definition of Test Mode 1 Waveform 1000Base-T from IEEE Standard for Ethernet
Figure 6 Measured differential Test Mode 4 Distortion Test 1000Base-T without disturber
We have now seen some of the electrical requirements that must be fulfilled to be compliant with the interface definition. Before we have a closer look at the test equipment, I want to try and solve the most important question of this Blog: How do you generate those test signals?
Each PHY vendor has a custom method to modify the necessary register settings to enter the test modes. That is often not publicly available, and you must ask your PHY vendor. A very good example is Microchip, who has provided all information about Ethernet Compliance in one document since last year. These are the Ethernet PHYs, which we use in our modules, KSZ8041 and KSZ9031. I also want to share a document from TI with you, just as a further example. If you are looking for a new Ethernet PHY and want to do Ethernet Compliance Testing, please ask your vendor for detailed information about the register settings in advance!
http://ww1.microchip.com/downloads/en/AppNotes/AN2686-Ethernet-Compliance-Test-10BASET-100BASETX-1000BASET.pdf
http://www.ti.com/lit/an/snla239a/snla239a.pdf
As depicted above, we must measure some picoseconds precisely. For that we need very good tooling. You should use an oscilloscope with a bandwidth of 1 GHz and memory of 4MS or greater. Usually you need a test fixture, which transforms the Ethernet signals from an Ethernet Jack to the oscilloscope input channels. That is why we work closely with Teledyne LeCroy. We have a great collaboration on a technical level. We use a high-end oscilloscope from the WaveMaster series with appropriate hardware and software tools. Of course, there is equipment from other vendors available like Tektronix, Rhode&Schwarz, Keysight and others.
Quotation from Mr. Hofferbert, specialist at Teledyne LeCroy:
“Teledyne LeCroy is a leading manufacturer of digital storage oscilloscopes (DSO). With modern DSOs it is possible to perform qualification measurements. Toradex uses appropriate equipment from Teledyne LeCroy to test the design of Ethernet PHYs. With the combination of our QualiPHY Software and latest oscilloscopes, a semi-automatic test has been implemented to test the physical level of the Ethernet PHYs according to the IEEE 802.3 specification. This measurement solution allows the development engineers at Toradex to test and resolve issues with signal integrity in an early development stage of their embedded systems. Toradex is very interested in using the measurement equipment as efficiently as possible. If there are any uncertainties or measurement deviation, we work together to quickly solve these issues and provide our expertise in measurement application.”
Gregor Hofferbert, Teledyne LeCroy
http://cdn.teledynelecroy.com/files/manuals/qualiphyenetmanual.pdf
After performing a compliance test, we can create a compliance report to verify our PCB design. As you can see in the test report from Teledyne LeCroy, the Colibri iMX6ULL with Iris Carrier Board is compliant with 10Base-T and 100Base-TX standard. So, we are sure that our implementation will work with other compliant systems. We can share with confidence our PCB implementation of the Ethernet Interface with our customers. You can find the Carrier Board Design Guides here:
https://developer.toradex.com/carrier-board-design
We also share our Carrier Boards Designs as Altium Designer projects for free with our customers. We provide a lot of help through our community channel as well:
https://www.toradex.com/community
We started to implement and provide our testing SW in our latest BSP. We adopt the existing drivers to be easily run with our modules to perform Compliance Tests. But again be careful, for each PHY you need different SW techniques to get the test pattern. There is no simple handbook. For example:
https://git.toradex.com/cgit/linux-toradex.git/commit/?h=tegra-next&id=13bd0f089ac6babeb7248fe3db4b9c19233cce3c
But issues can occur with wrong routing, bad ground layout, or inaccurate crystal circuit design. It also depends on the testing environment. Ground loops and noisy or underpowered power supplies can cause measuring errors. It is important to follow the design guides of the PHY vendors. I like the troubleshooting document provided by Intel, that you can use for the first debug consultation. There is a basic overview of failures and possible sources. Designs with too long traces, low quality magnetics or improper use of the measurement equipment can always be the source of failing the compliance test. But there are also specific errors which can be put in a strong correlation. E.g. wrong amplitude values are often caused by wrongly assembled bias resistors or issues with the centre tap circuits. Whereas too high Jitter values are due to crystal issues, impedance mismatch or bad power supply. In general, your Ethernet PHY vendor should be able to help you, probably even with a schematic or layout review.
https://www.intel.com/content/dam/www/public/us/en/documents/application-notes/ieee-conformance-phy-app-note.pdf
In this blog, I gave you some insights to one of the many verifications Toradex does to achieve such high-quality products. Internal testing by us and your adherence to our design guides reduces your risk to a minimum. For the highest quality, you can do your verification with your own customized carrier board, and that you follow the system engineering approach (as documented by NASA, for instance) which recommends testing in early stages to reduce risk and cost. I hope I gave you plenty of information to get started. If you need more information feel free to reach out.
