Institute of Parallel and Distributed Systems University of Stuttgart Universitätsstraße 38 D–70569 Stuttgart Bachelorarbeit A Data Plane Interface for Resource-Constrained Microcontrollers in Time-Senstitive Networking Bastian Kupka Course of Study: B.Sc. Informatik Examiner: Prof. Dr. Christian Becker Supervisor: Simon Egger, M.Sc. Jona Herrmannm, M.Sc. Commenced: July 16, 2025 Completed: January 16, 2026 Abstract Time-Sensitive Networking (TSN) enables deterministic Ethernet communication through coordi- nated transmission control. While there are dedicated TSN switches and network interface cards, they are rarely available on resource-constrained microcontrollers. In particular, transmit-time-based scheduling approaches such as the Earliest TxTime First (ETF) queuing discipline are typically not supported on embedded platforms. This thesis investigates the feasibility of implementing ETF-like transmit-time scheduling on a microcontroller using Zephyr RTOS. The implementation targets the NXP i.MX RT1062, which provides a precise PTP hardware clock and compare interrupt mechanism but lacks dedicated hardware support for traffic shaping. The proposed solution integrates transmit-time scheduling into the Zephyr networking stack by combining hardware-triggered interrupts with software-based buffer management inside the network driver. Scheduled frames are prepared in advance and transmitted using a PTP compare event to trigger the transmission routine close to the target time, followed by a short busy-wait phase to improve precision. Best-effort traffic is handled through a driver-level guard band to avoid interference with scheduled transmissions. Experimental evaluation shows that, for periodic traffic with fixed inter-packet gaps, the implementa- tion achieves a bounded transmission window of approximately 3 µs. For variable inter-packet gaps, a larger timing spread is observed. The results demonstrate that ETF-like transmit-time scheduling can be realized on a low-cost microcontroller for certain traffic patterns by leveraging existing PTP hardware features. 3 Kurzfassung Time-Sensitive Networking (TSN) ermöglicht deterministische Ethernet-Kommunikation durch koordinierte Übertragungssteuerung. Während dafür extra TSN switche und Netzwerkkarten existieren, sind sie auf ressourcen beschränkten Mikrocontrollern selten verfügbar. Insbesondere zeitbasierte Scheduling-Ansätze wie die Earliest TxTime First (ETF)-Warteschlangendisziplin werden auf eingebetteten Plattformen in der Regel nicht unterstützt. Diese Arbeit untersucht die Machbarkeit der Implementierung einer ETF-ähnlichen Sendezeit- planung auf einem Mikrocontroller unter Verwendung von Zephyr RTOS. Die Implementierung zielt auf den NXP i.MX RT1062 ab, der eine präzise PTP-Hardwareuhr und einen Vergleichsin- terrupt mechanismus bietet, aber keine dedizierte Hardwareunterstützung für Traffic Shaping hat. Die vorgeschlagene Lösung integriert die Sendezeitplanung in den Zephyr-Netzwerkstack, indem sie hardwaregesteuerte Interrupts mit softwarebasiertem Puffermanagement innerhalb des Netzwerktreibers kombiniert. Geplante Rahmen werden im Voraus vorbereitet und unter Verwendung eines PTP- Vergleichsereignisses übertragen, um die Übertragungsroutine nahe der Zielzeit auszulösen, gefolgt von einer kurzen Busy-Wait-Phase zur Verbesserung der Präzision. Beste-Güte-Verkehr wird über eine Sicherheitsband auf Treiber-Ebene behandelt, um Interferenzen mit geplanten Übertragungen zu vermeiden. Experimentelle Auswertungen unter Verwendung zeigen, dass die Implementierung für peri- odischen Verkehr mit festen Inter-Paket-Abständen ein begrenztes Sendezeitfenster von etwa 3 µs erreicht. Bei variablen Lücken zwischen den Paketen wird eine größere zeitliche Streuung beobachtet. Die Ergebnisse zeigen, dass eine ETF-ähnliche Übertragungszeitplanung für bes- timmte Verkehrsmuster auf einem kostengünstigen Mikrocontroller realisiert werden kann, indem vorhandene PTP-Hardwarefunktionen genutzt werden. 4 Contents 1 Introduction 11 2 Background 13 2.1 Basics and Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2 Configuration of the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.3 Queuing Disciplines (Qdisc) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.4 Categorization of Implementations for TSN Support . . . . . . . . . . . . . . . 23 3 Related Work 25 3.1 Linux Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.2 FPGA Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.3 Microcontroller Implementations . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4 Positioning of this Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4 System Model and Problem Statement 29 4.1 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.3 Key Performance Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 5 Design 39 5.1 Design Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 5.2 Transmission Path Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.3 Best-Effort Traffic Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.4 Unsuccessful Transmission Path Design Approaches . . . . . . . . . . . . . . . 45 6 Implementation 49 6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 6.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6.3 Interface Changes and Data Structures . . . . . . . . . . . . . . . . . . . . . . 52 6.4 Pseudocode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 7 Evaluation 57 7.1 Test Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 7.2 Clock Synchronization Precision . . . . . . . . . . . . . . . . . . . . . . . . . 62 7.3 Traffic Generation and Scheduling Characteristics . . . . . . . . . . . . . . . . 63 7.4 Packet Loss Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 7.5 Scheduled Transmission Accuracy . . . . . . . . . . . . . . . . . . . . . . . . 64 7.6 Internal Timing Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 8 Conclusion and Outlook 71 5 Bibliography 73 6 List of Figures 2.1 TSN architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2 IEEE 802.1Q Ethernet frame extension . . . . . . . . . . . . . . . . . . . . . . . 16 2.3 IEEE 802.1Qbv design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.4 Time-Aware scheduling based on transmission time . . . . . . . . . . . . . . . . 18 2.5 Example of a fully distributed configuration model using IEEE 802.1Qcc . . . . 19 2.6 Example of a centralized configuration model using IEEE 802.1Qcc (SRP) . . . . 20 4.1 Deployments of TSN applications . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.2 Complete transmission path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.3 Transmission path after Zephyr networking stack . . . . . . . . . . . . . . . . . 33 5.1 TXTIME frame admission from the network stack and scheduling preparation in the network driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.2 Triggering of the transmit routine using an interrupt from the PTP clock . . . . . 42 5.3 Design to multiplex TSN and BE traffic . . . . . . . . . . . . . . . . . . . . . . 43 5.4 Transmission completion and scheduling state update . . . . . . . . . . . . . . . 44 5.5 Unsuccessful design to multiplex TSN and BE traffic using ring buffer partitioning 46 5.6 Unsuccessful design to multiplex TSN and BE traffic using dynamic ring switching 47 7.1 Test environment’s hardware setup . . . . . . . . . . . . . . . . . . . . . . . . . 57 7.2 PPS signal from the NIC observed with an oscilloscope . . . . . . . . . . . . . . 58 7.3 PPS signal with impedance mismatch and reflections on an unterminated line . . 59 7.4 PPS signal on a terminated line . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 7.5 Histogram of clock drift. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 7.6 Distribution of inter-packet gap distance . . . . . . . . . . . . . . . . . . . . . . 63 7.7 Distribution of receiver-observed timing error relative to the scheduled time . . . 64 7.8 Distribution of MAC-observed timing error relative to the scheduled time . . . . 65 7.9 Distribution of transmit routine start time (before busy-waiting) relative to PTP compare value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 7.10 Distribution of transmit routine start time (before busy-waiting) relative to MAC timestamp delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 7 Acronyms ASIC Application Specific Integrated Circuit BE Best Effort (Traffic) CRC Cyclic Redundancy Check DMA Direct Memory Access ETF Earliest Transmit Time First FPGA Field Programmable Gate Array GPIO General Purpose Input Output (Pin) HAL Hardware Abstraction Layer ISR Interrupt Service Routine MAC Media Access Control NIC Network Interface Controller PPS Pulse Per Second (Signal) PTP Precision Time Protocol RTOS Real Time Operating System TDAR Transmit Descriptor Active Register TDSR Transmit Buffer Descriptor Ring Start Register TSN Time Sensitive Networking TxBdRing Transmit Buffer Descriptor Ring UDP User Datagram Protocol ZLI Zero Latency Interrupt 9 1 Introduction Time-Sensitive Networking (TSN) extends the standard Ethernet (IEEE 802.3 [3]) to support deterministic communication with bounded latency, low packet delay variation, and low packet loss [1]. These properties are essential for industrial automation, control systems, and cyber-physical systems, where correct system behavior depends not only on the transmitted data but also on precise timing guarantees. To address these requirements, the IEEE 802.1 standards define mechanisms for time synchronization, traffic classification, and scheduled transmission. On general-purpose systems, TSN functionality is commonly realized using operating system abstractions. In Linux, scheduled transmission is implemented through queueing disciplines such as Earliest Transmit Time First (ETF), in combination with the SO_TXTIME socket option[15]. This allows applications to attach an absolute transmission time to each packet using standard BSD socket APIs. Then combined with special TSN network interface cards (NICs) that provide hardware scheduling support based on a PTP hardware clock, such systems can achieve high transmission accuracy with low jitter [2]. In contrast, microcontrollers are widely used in industrial environments as sensors, actuators, and embedded controllers due to their low cost, low power consumption, and tight integration into real-world systems. Despite their widespread use, microcontroller platforms generally do not provide explicit support for transmit-time scheduling mechanisms such as ETF. Existing work demonstrates that microcontrollers such as the NXP i.MX RT1062 can achieve precise clock synchronization using hardware PTP support [18]. This establishes an important prerequisite for scheduled transmission but does not address how packets can be released at precisely defined times in the presence of limited hardware resources. In particular, the stated microcontroller Ethernet controller only provides a single transmit buffer descriptor ring and lacks hardware mechanisms for transmit-time scheduling. This thesis addresses this gap by investigating whether ETF-like transmit-time scheduling semantics can be realized on a resource-constrained microcontroller without specialized TSN hardware. The focus is on the NXP i.MX RT1062 microcontroller running the Zephyr Real-Time Operating System (RTOS), which provides a BSD socket API and some basic support to implement transmit-time scheduling. The goal is to design, implement, and evaluate a scheduling approach that preserves standard socket semantics but provides bounded transmission accuracy. To achieve this behavior, the existing NXP Ethernet driver got extended to allow transmit-time scheduling support using the high-resolution hardware PTP clock and carefully designed interrupt and DMA mechanisms. The remainder of this thesis is structured as follows. Chapter 2 introduced the background and fundamental concepts of TSN and queuing disciplines. Chapter 3 reviews related work, categorized by hardware platforms. Chapter 4 defines the system model and problem statement. Chapters 5 and 6 describe the design and the implementation of the proposed solution. Chapter 7 evaluates the implementation with respect to clock synchronization precision and transmission accuracy. Finally, chapter 8 concludes this thesis and outlines directions for future work. 11 2 Background Time-Sensitive Networking (TSN) has been introduced to expand the capabilities of IEEE 802.3[3] (Ethernet) to a more robust and deterministic behavior. The core challenge of TSN lies in delivering messages over Ethernet deterministically as well as guaranteeing their delivery in a specified time window. To achieve this, TSN introduces constraints in latency and jitter for the network as well as its participants. Thereby slicing the network traffic in different time frames to avoid collision (Time Division Multiple Access). In this chapter, we will first give an introduction to the basics and terminology of TSN, followed by configuration approaches for TSN. After that, we will discuss different queuing disciplines needed for TSN and how Linux implements these. At the end of this chapter, different implementations for TSN support, like hardware offloading, are categorized. Figure 2.1: TSN architecture 13 2 Background 2.1 Basics and Terminology 2.1.1 Endstation In a TSN network, an end station is a device that is connected to the network edge and participates in communication but does not directly forward traffic between network segments. Essentially, end stations either take the role of a talker, which sends application data; a listener, which receives application data; or both in the network. An example TSN network consisting of three end stations with one talker, one listener, and one end device taking both roles is illustrated in 2.1. An end station implements the required networking stack and timing mechanism to interact with the TSN network. This typically includes support for time synchronization, traffic classification, and configuration for TSN streams. However, their concrete behavior is always subject to its assigned role and application context. Talkers and listeners are described in more detail later. End stations can be implemented on a wide range of platforms, from industrial PCs to embedded controllers and microcontrollers. While their hardware capabilities differ, they all follow the same fundamental principle of interacting with the TSN network. 2.1.2 Bridge In TNS, a bridge functions similarly to a conventional Ethernet switch but provides additional capabilities required for deterministic communication. A TSN-capable bridge interconnects multiple Ethernet segments and forwards frames according to rules defined in IEEE 802.1Q, while extending this behavior with TSN-specific mechanisms. In addition to basic forwarding, a TSN bridge supports multiple egress queues, traffic classification, and TSN-specific scheduling and shaping mechanisms. These mechanisms control when frames are transmitted and how bandwidth is allocated. Common examples include: • Strict Priority Scheduling: Frames are transmitted based on their assigned priority, where higher-priority traffic is always served before lower-priority traffic. While this approach is simple and ensures low latency for critical traffic, it may lead to starvation of lower-priority classes if high-priority traffic is continuously present. • Credit-Based Shaping (CBS): CBS approach is to control the bandwidth usage of selected traffic classes by accumulating and consuming credits over time. This mechanism ensures that time-sensitive streams receive a guaranteed share of the link bandwidth while still allowing other classes to transmit, thereby providing bounded latency without completely blocking lower-priority traffic. • Asynchronous Traffic Shaping (ATS): ATS smooths traffic flows by regulating packet release times independently of a global time schedule. Therefore, it does not rely on synchronized time slots but instead shapes traffic based on per-flow state, reducing burstiness and improving predictability across the network hops. 14 2.1 Basics and Terminology • Time-Aware Shaping (TAS): TAS uses a globally synchronized time schedule to control when different traffic classes are allowed to transmit. This is typically realized by opening and closing transmission gates according to a predefined schedule or by explicitly scheduling the transmission of individual frames. TAS enables strict timing guarantees and is commonly used for highly time-critical communication. This will be discussed in more detail in subsection 2.1.7. In this work, the focus is on the Time-Aware Shaper and, in particular, on transmit-time-based scheduling as realized in ETF queueing disciplines in Linux. These bridges are typically implemented using dedicated switching hardware, integrating TSN behavior directly into hardware. This hardware-based implementation allows scheduling and shaping decisions to be executed with high precision and low jitter. As a result, bridges can enforce strict latency and timing guarantees for time-sensitive streams. A practical example of a TSN bridge is the Kontron KSwitch D10 MMT Series [9], which supports time synchronization, CBS, TAS, and per-stream filtering and policing. 2.1.3 Talker A talker implements an endpoint that generates network traffic in the form of specific TSN streams. It is responsible for complying with the timing and resource constraints defined by the TSN configuration of the network, including a reserved bandwidth, traffic specifications, and transmission intervals. To do this, a talker does not directly allow an application to send data over a network but instead uses queues for the outgoing frames (egress queues) for further processing. Those egress queues are processed using TSN-specific scheduling and shaping mechanisms such as those described in the subsection 2.1.2. During the initialization phase, a talker may participate in the fully distributed configuration model of the Stream Reservation Protocol (SRP, IEEE 802.1Qcc [7]), announcing its stream requirements like VLAN ID, Priority Code Point (PCP), traffic class, and other QoS parameters of the network. It can also notify a central network controller (CNC) of its capabilities like buffer size, supported traffic shapers, latencies, and jitter, as well as running applications to allow the CNC to create a suitable configuration for the talker. A typical example of a talker is a lightweight sensor node (as illustrated in figure 2.1), which periodically transmits time-critical measurement data to a processing system. 2.1.4 Listener The listener implements an endpoint that receives specific TSN streams. By participating in the fully distributed configuration model of SRP, the listener expresses its interest in a particular stream and relies on the talker to generate the data and on the network to deliver it within the agreed latency, jitter, and interval bounds. Like the talker, a listener can also rely on the CNC for configuration. Listeners may be resource-rich devices that summarize information from multiple talkers and compute control actions for the time-critical system or lightweight actuators performing the calculated control action. 15 2 Background Figure 2.2: IEEE 802.1Q Ethernet frame extension In figure 2.1, a listener is shown as a resource-constrained actuator that receives commands, which must be executed in a time-critical manner (e.g., controlling a motor). 2.1.5 Traffic Class Traffic classes group frames with similar quality-of-service or real-time requirements. It uses IEEE 802.1Q [6] which extends the standard Ethernet header from IEEE 802.3 with extra control fields, allowing the creation of new virtual local area networks (VLANs), thereby creating an overlay network on top of the physical Layer-2 network. For traffic classes, the 3-bit Priority Code Point (PCP) contained in the VLAN tag of an Ethernet frame (see figure 2.2) is used for identification, allowing the existence of up to eight traffic classes. Frames belonging to different traffic classes are typically mapped to distinct egress queues and may be processed by a different traffic shaper. While streams describe the specific timing and resource requirements of a flow, traffic classes determine how frames are prioritized and dequeued at endpoints and bridges, directly influencing transmission order and latency. 2.1.6 Stream A stream is a unidirectional flow of Ethernet frames transmitted from a talker to one or more listeners. Streams are identified by flow descriptors such as source MAC address, VLAN ID (VID), or PCP. VLAN ID and PCP are defined in IEEE 802.1Q and shown in figure 2.2. Each stream is associated with a traffic specification that defines parameters such as maximum frame size, transmission interval, latency requirements, and priority level. Through the SRP or a CNC, the required resources for maintaining these guarantees are reserved along the entire network path from talker to listener. Thereby, a stream symbolizes a contract between the talker, the listener(s), and all intermediate network components, ensuring that the predefined QoS parameters are upheld. 16 2.1 Basics and Terminology Figure 2.3: IEEE 802.1Qbv design 2.1.7 Time-Aware Traffic Scheduling in IEEE 802.1 This subsection further introduces the Time Aware Shaper, which was already briefly mentioned in subsection 2.1.2 and establishes a transition from classical traffic shaping mechanisms to the transmit-time-based scheduling approach used in this thesis. The goal of TAS is to control network traffic by introducing a globally synchronized time schedule that defines when different traffic classes are allowed to transmit. This approach inherently requires precise time synchronization across all participating network devices, typically achieved using IEEE 802.1AS based on Precision Time Protocol. To enable time-based scheduling, traffic must first be correctly classified. For this purpose, IEEE 802.1Q extends the classical Ethernet header defined in IEEE 802.3 by introducing VLAN tagging and the PCP field. The extension is illustrated in figure 2.2. The three-bit PCP field allows a frame to be assigned to one of eight traffic classes, each typically mapped to a dedicated egress queue. This mechanism enables network devices to distinguish between traffic with different quality-of-service and timing requirements. One widely used realization of time-based traffic control is the Time-Aware Shaper (TAS), standardized in IEEE 802.1Qbv[4]. It introduces gates for egress queues, which can be either in the state open or closed, according to a predefined time schedule. By controlling the gate state of each queue, TAS precisely determines when frames of a specific traffic class are allowed to be dequeued and transmitted. If gate schedules are coordinated across the network, time-triggered communication with strict temporal guarantees can be achieved. 17 2 Background Figure 2.4: Time-Aware scheduling based on transmission time The TAS schedule is typically configured by a centralized entity, such as a Contralized Network Controller (CNC, see subsection 2.2.2), using a Gate Control List (GCL). The GCL specifies exact time intervals during which individual egress queues are permitted to transmit. During these intervals, other traffic classes are blocked, preventing interference and ensuring deterministic transmission behavior. With this scheduling discipline, traffic scheduling is explicitly based on time instead of solely relying on priority or rate. Time-critical streams are assigned to higher-prioritized traffic classes with higher gate opening frequency or longer gate opening time, while less critical traffic is deferred to lower-prioritized traffic classes. This strict separation makes TAS particularly suitable for industrial control systems and applications requiring hard real-time guarantees. A visual example of the described behavior can be seen in figure 2.3. Outgoing traffic either originates from an application at a talker or from an ingress port in a bridge. This traffic is first classified and enqueued into the appropriate egress queue. The GCL then controls the transmission of frames by opening or closing the corresponding gates, indicated by the switching elements in the figure. While TAS implicitly defines transmission times through gate schedules, an alternative approach is to explicitly assign a transmission time to each frame. Instead of relying on gated egress queues, frames are classified and then associated with a calculated transmission time derived from the global schedule. This concept is illustrated in figure 2.4. Such an approach closely resembles the transmit-time-based scheduling semantics forming the conceptual basis for the implementation presented in this thesis. 2.2 Configuration of the Network For TSN to function correctly, all involved components (talker, listener, and bridges) in the system have to be coordinated with each other. Each device needs to know which resources it has to reserve for TSN streams, for example, buffer space, bandwidth, or queue capacity. However, in a network, devices usually only have knowledge about their directly connected neighbors. By that, a bridge or end device does not necessarily know where a stream originates, which path it takes through the network, or where it is consumed. This local view of the network makes it difficult to ensure end-to-end QoS guarantees such as guaranteed delivery, bounded latency, or low jitter without coordinating all involved participants. Also, to support TSN streams, the participants in the network on a path of a stream must be configured for each stream with respect to traffic shapers, 18 2.2 Configuration of the Network Figure 2.5: Example of a fully distributed configuration model using IEEE 802.1Qcc scheduling parameters, and the intervals at which data is expected to be sent or delivered. To address this problem, IEEE 802.1Qcc (SRP) exists with two general approaches for TSN networks: a centralized [7, Section 46.1.3.2] and a fully distributed configuration model [7, Section 46.1.3.1]. Both approaches are designed for allocating resources along the streams’ paths, but they differ significantly in their prerequisites for the participants and the topology of the network, as well as how configuration decisions are distributed. 2.2.1 Fully Distributed Configuration For the decentralized configuration approach, the participants of the network must configure themselves by only exchanging messages with their direct neighbor without a central configuration manager. The participants make decisions based on locally available information and generally do not know the parameters of the network, like the number of participants or the topology, as well as the global resource usage. As seen in figure 2.5, talkers need to advertise their requirements of their stream, such as bandwidth, interval, size, and priority, to the network. Bridges then forward these control messages, allowing listeners to signal their interest in receiving the stream. This results in bridges along the path independently deciding whether they can support the requested parameters of the stream based on their local resource usage. If enough resources are available, the stream is admitted, and all bridges along the path configure themselves to forward the stream based on its requirements and reserve necessary resources. In the case that at least one participant on the stream’s path does not have sufficient resources in the form of buffer space, bandwidth, or other requirements, the stream is rejected. Since configuration decisions are made by the participants themselves, no additional configuration manager is required, which simplifies deployment and reduces architectural complexity. 19 2 Background Figure 2.6: Example of a centralized configuration model using IEEE 802.1Qcc (SRP) However, not all TSN features can be configured by bridges using this model. Besides that, the computation of some TSN use cases is computationally complex, which makes it not suitable for resource-lightweight computation modules used in talkers or listeners. Because each participant only has a local view of the network, it is not guaranteed that a resulting configuration is not the best achievable configuration for the network, as global interactions between streams are not taken into account. As a result, decentralized models can ensure bounded latency and bandwidth reservation, but they cannot compute globally optimized schedules. Decentralized configuration is therefore more suitable in small TSN deployments where soft real-time guarantees are acceptable, and the computation can be done on lightweight devices. 2.2.2 Centralized Configuration To overcome the limitations of the fully distributed configuration approach, a dedicated central configuration manager with a global view of the network is introduced. More specifically, the IEEE 802.1Qcc (SRP) amendment further distinguishes between a central network, a distributed user model, and a fully centralized model. The first two models mentioned share the same main concept of introducing a Central Network Controller (CNC) as a central configuration manager. Its purpose is to compute and provision a network-wide configuration that ensures deterministic end-to-end behavior. Like the fully distributed model, the participants of the TSN exchange control messages 20 2.3 Queuing Disciplines (Qdisc) of their capabilities and requirements, but instead of exchanging it with all their direct neighbors, the participants only send this information to the CNC. An example topology using a CNC is seen in figure 2.6. One way of determining the topology of the network is by using the IEEE 802.1AB (Link Layer Discovery Protocol, LLDP) [5], which allows devices to advertise their chassis ID, port ID, system capabilities, VLAN configuration, and other properties. Additional user or network configuration info can either be directly specified in the CNC or outsourced into a separate Central User Configuration (CUC) component. The result is the CNC knowing the capabilities and requirements of all participants, as well as the network topology, and the resulting global resource allocation for streams. After accumulating all that necessary information, the CNC is able to compute an optimized network configuration. It can thereby take the influence of multiple concurrent streams into account, as well as find a better global path for a stream from talker to listener instead of only using the shortest path. As already explained in the fully distributed model, the computation of a configuration can be com- putationally expensive; it involves selecting forwarding paths, assigning traffic classes, configuring shaper parameters, and computing transmission schedules for TSN. After the configuration has been computed, the CNC provisions the participants using, e.g., YANG and Netconf stated in IEEE 802.1Qcc. Thereby, each participant of the TSN only receives their local configuration necessary for its part of the network. The CNC ensures that end-to-end QoS parameters like latency or jitter are met for every hop on a stream’s path. The CNC introduces additional deployment and architectural complexity. But it also supports more TSN features and enables strict determinism because the configuration of the participants is not dependent on the arrival of configuration messages. Therefore, this method is more suitable for applications with hard real-time requirements when precise time-triggered communication and tight latency bounds are required. 2.3 Queuing Disciplines (Qdisc) To allow applications to communicate with each other, a standardized way of interprocess com- munication is needed. A common standard in Linux is BSD / POSIX sockets. They allow two processes to communicate with each other, regardless of if they run on the same machine, in the same network, in the same physical area, or share the same type of operating system. Therefore, the operating system provides an interface with the help of syscalls (system calls) for applications to create, modify, or delete sockets. While these sockets allow a variety of different socket families, we focus on the AF_INET family, which corresponds to the IPv4 Internet Protocol [13]. Sockets can have different kinds of types; the most prominent are SOCK_DGRAM, which uses UDP, and SOCK_STREAM, which uses TCP for communication. Applications can create these sockets, then either connect to a remote already-created socket or accept incoming connection requests. After a successful connection between two sockets is made, the application using the socket can either write or read data to or from the socket, resulting in the exchange of information over a network [14]. Each of the described operations is a syscall, telling the operating system to perform tasks like packet processing, queueing, and scheduling, allowing developers to focus on communication logic without needing to manage low-level networking details like different network drivers or hardware. 21 2 Background This standard allows the multiplexing of multiple sockets to the same network interface, thereby enabling multiple applications to use the same network to communicate to different applications on different machines. To allow fairness or prioritization, operating systems usually use queueing disciplines. Queuing disciplines operate at the egress queues of a system and provide a set of rules on how packets are buffered, ordered, delayed, or dropped before transmission. After the app writes to a socket, the OS enqueues the frames into an egress queue. Qdiscs then decide when and in which order those egress queues are dequeued and sent to the network device. The simplest behavior of a qdisc is a first-in-first-out queue, sending frames in the order, and the time they are queued. An extension to this is the usage of a priority queue, allowing prioritization of frames, yet still those strategies are insufficient for real-time deterministic communication in TSN. For receiving frames, commonly no disciplines are required, as packets get processed as soon as they arrive. 2.3.1 Queueing Disciplines in Linux Linux provides a flexible and extensible framework for implementing and using qdiscs through its Traffic Control (tc) subsystem[17]. Traffic control allows different scheduling and shaping disciplines to be attached to network interfaces, thereby allowing fine-grained control over packet transmission behavior. It is possible to attach predefined qdiscs to a network interface as well as create a tree structure of qdiscs, allowing a complex operation of shaping, scheduling, and filtering of traffic. To support TSN features, Linux implements several specialized qdiscs, some of which directly map to IEEE 802.1 standards. Thereby, two particular qdiscs are especially relevant for time-triggered communication: TAPRIO and ETF. TAPRIO The TAPRIO (Time-Aware Priority) qdisc is the implementation of the IEEE 802.1Qbv Time-Aware Shaper already described in subsection 2.1.7 in Linux. TAPRIO allows the configuration of gate control lists, which define the time window a particular traffic class is allowed to transmit. This timing is aligned to a global time base, requiring the need for time synchronization, usually provided by IEEE 802.1AS (gPTP). After specifying a mapping of traffic classes to egress queues, data from the sockets will automatically be sorted in the corresponding queue with its defined transmit window. TAPRIO can operate purely in software, but this results in some inaccuracy because of, for example, the scheduling of different processes of the machine. To achieve better accuracy, some network hardware supports offloading of the gate control list to the device, using the hardware directly to allow more precise control of the gates’ operation [16]. ETF While TAPRIO focuses on traffic class-based, gate-controlled scheduling, it does not operate on individual frame transmission deadlines. For applications that require the setting of a deadline per frame, Linux provides the Earliest Transmit First (ETF) qdisc. This qdisc does not directly map to an IEEE 802.1 standard like the TAPRIO qdisc. ETF allows applications using BSD sockets to explicitly set a transmit timestamp when writing data to the socket using a special S0_TXTIME 22 2.4 Categorization of Implementations for TSN Support socket option. The qdisc then holds packets until their specified transmit time is reached and then transmits them in deadline order. This differs from the TAPTIO qdisc as ETF guarantees the transmit on a predefined point, unlike TAPRIO where traffic classes are sent in time windows, which could result in our frame being stuck in the queue until earlier frames in the queue are sent. Packets in the past, or if packets expire while waiting for being dequeued, will be dropped, ensuring that disturbances do not violate timing guarantees. It is also possible to improve the accuracy by hardware offloading. With this approach, ETF bridges the gap between application-level timing requirements and low-level packet transmission [15]. 2.4 Categorization of Implementations for TSN Support TSN functionality can be realized at different layers of the system, ranging from pure software-based to fully hardware-based solutions. Each approach to implementing TSN features has its advantages as well as disadvantages. Factors like determinism, precision, flexibility, complexity, and cost play a critical factor in designing solutions. Therefore, this section gives a high-level overview and comparison of different implementation approaches. A quick summary can be seen in the table 2.1. Table 2.1: Overview of TSN Implementation Approaches Approach Scheduling Precision Flexibility Cost Software OS (kernel) Low High Low NIC offloading NIC High Medium Medium Microcontroller App/Driver Medium Medium Low FPGA Hardware High High Medium ASIC Hardware High Low High 2.4.1 Software Approaches solely based on software relay entirely rely on the operating system to enforce timing and scheduling constraints. An example is using the Linux kernel with the ETF or TAPRIO qdisc without hardware offloading. Every scheduling decision as well as time-sensitive events like transmitting a frame happens in the kernel. The main advantage of this approach is high flexibility, as new functionality or modification can be done in software, Besides that, no special hardware is needed, which reduces the cost for implementing TSN. However, the lack of hardware assistance limits the achievable timing precision, as software is sensitive to scheduling decisions of the OS. These scheduling decisions are directly influenced by CPU load, interrupt latency, and system jitter, resulting in slight changes in behavior under different circumstances. 23 2 Background 2.4.2 NIC Offloading A better approach than purely relying on software is using special network interface cards (NICs) that support hardware offloading. With this setup, the OS still performs high-level scheduling and classification, but time-critical operations like transmitting a frame in the ETF qdisc are delegated to the NIC. An example of this setup is the usage of the ETF or TAPRIO qdisc in Linux together with the Intel I210 Ethernet controller [8]. This approach significantly improves time accuracy, as the NIC makes time-sensitive decisions close to hardware instead of relying on the kernel. But for this approach, special hardware is needed; currently, only a handful of Intel Ethernet controllers support this hardware offloading [8]. Besides that, the hardware must also support the particular qdisc, as hardware offloading is custom for every qdisc. Therefore, it is not guaranteed that all TSN features are supported by the NIC. 2.4.3 Microcontrollers Microcontrollers are resource-constrained devices, often used in industrial applications as lightweight and inexpensive sensors or actors. To allow real-time critical systems, special operating systems like Zephyr or FreeRTOS are used. In order to support TSN capability, an Ethernet MAC with timestamping capabilities is needed. This approach allows using a real-time operating system in combination with a specially designed network driver to imitate a classical system with NIC offloading. Thereby making the time-critical decisions in the network driver close to the hardware. The main advantage of this approach compared to NIC offloading is its low cost and low power consumption, but it still allows the flexibility to expand or modify the driver to implement new TSN features. 2.4.4 FPGA Field-Programmable Gate Arrays (FPGAs) are reconfigurable digital hardware devices that allow custom logic to be implemented at the level of digital circuits, enabling hardware for specific applications without manufacturing a dedicated circuit. Therefore, FPGAs can implement different TSN features like the TAPRIO or ETF qdisc directly in the data path (for a sensor: measuring right before transmitting its reading). This enables very low latency and jitter but still a high level of determinism. However, FPGAs require deep technical knowledge for development and therefore are not as flexible as software-based solutions. 2.4.5 ASIC Application-Specific Integrated Circuits (ASICs) are fixed-function digital circuits designed to implement a specific function. Bridges supporting TSN and high-end NICs implement scheduling, shaping, and forwarding entirely in ASIC, resulting in very low jitter and a high level of determinism. The primary advantage of ASICs is their performance and power efficiency. But on the contrary, the function implemented by an ASIC cannot be changed after design, allowing no flexibility. Therefore, development costs are high, which limits ASICs only to mass-produced devices like TSN bridges. 24 3 Related Work Time-Sensitive Networking has been an active research area over the last years, with a focus on achieving deterministic communication over IEEE 802.3 (Ethernet). Existing work mainly differs in the hardware platform used, the degree to support hardware offloading of scheduling decisions, and the achievable accuracy. This section discusses related work with a particular focus on the implementations that evaluate the accuracy of scheduled transmission, as this aspect is central for ETF-like mechanisms. The discussion is structured by hardware class, ranging from Linux-based systems with NICs supporting TSN to FPGA and ASIC solutions, and finally to solutions based on microcontrollers. 3.1 Linux Implementations Linux is often used as a reference platform for TSN research, as it provides mature networking abstractions and implements several queuing disciplines (qdiscs). In particular, the ETF qdisc allows applications to specify an absolute transmit time per packet using the SO_TXTIME socket option[15]. This mechanism is frequently used as a baseline for transmit-time-based scheduling solutions. Fejes et al. [2] provide a detailed overview of TSN building blocks in Linux, including ETF and TAPRIO, and discuss their interaction with the kernel networking stack. Their work shows that software-only implementations suffer from limited accuracy due to process scheduling effects and interrupt latency. To address this, special NICs supporting hardware offloading of the scheduling of a transmission can be used with Linux. Some studies evaluate Linux in combination with NICs supporting hardware offloading, such as the Intel i210[8]. In these systems, Linux mainly performs classification and enqueueing, while the NIC ensures the precise transmission timing in hardware. The resulting accuracy is therefore high (in the tens of nanoseconds), with a short guaranteed transmission window, but it strongly depends on specialized hardware [2]. Therefore, while Linux provides the ETF semantics at the API level, the actual precision is largely a property of the hardware used rather than the operating system itself. In contrast to this work, the thesis does not assume the availability of special NICs. Instead, it targets commonly used microcontrollers where such hardware offloading is not available but where similar socket API semantics are desirable. 25 3 Related Work 3.2 FPGA Implementations Approaches based on FPGAs are frequently used when very high timing accuracy is required but no specific circuitry is manufactured. Lui et al. [10] present an NIC architecture based on FPGA that implements TSN features directly in hardware and evaluates its scheduling accuracy under different traffic patterns. By offloading scheduling decisions onto the FPGA, a precisely guaranteed transmission window comparable to that of dedicated TSN-capable NICs can be achieved. To realize this functionality, the FPGA design integrates several dedicated modules, including a PTP clock synchronization module, a MAC module, and a transmit engine responsible for hardware-level egress scheduling. Communication with the host system is handled via a PCIe-based I/O engine, allowing the FPGA to operate as a TSN-capable NIC. From the perspective of the operating system, such a device can be used for hardware offloading. However, FPGA systems come with increased development complexity and cost. Besides that, they normally do not run the application code directly, as this would be too time-consuming and thereby expensive to be implemented in FPGA. Therefore, they would require an extra microcontroller to run the application and allow non-experts to modify it. 3.3 Microcontroller Implementations Compared to Linux, FPGA, and solutions built in circuitry, implementations based on micro- controllers are less frequently studied, despite their widespread use in industrial systems. Mi- crocontrollers are often used as sensors or actuators and are typically inexpensive but resource constrained. The TSN-specific capabilities of resource-constrained microcontrollers are highlighted by previous work on the NXP i.MX Rt1062 platform. Riep [18] presents the implementation and evaluation of a gPTP time receiver on the NXP i.MX RT1063 microcontroller using Zephyr. This work mainly focuses on clock synchronization, but it shows that the platform provides a sufficiently accurate PTP hardware clock and interrupt mechanisms needed for TSN features. Siebert [20] investigates latency and jitter on embedded platforms and uses the NXP i.MX RT1062 to analyze timing behavior in wireless communication systems. The results show that, on microcontrollers, factors such as interrupt latency, memory access patterns, and software structure have a significant impact on timing accuracy. This underlines the importance of careful system design when implementing time-sensitive mechanisms in software. In this work, we aim to further extend the above work to investigate the achievable precision of ETF-like transmission scheduling on the NXP i.MX RT 1062 platform. 26 3.4 Positioning of this Thesis 3.4 Positioning of this Thesis The reviewed work shows a gap between high-accuracy TSN implementations relying on specialized hardware and the need for low-cost but resource-constrained devices. While Linux provides a well-defined transmit-time scheduling API, its accuracy largely depends on hardware offloading. FPGA and ASIC implementations achieve excellent precision but are often too costly or complex for embedded endpoints. This thesis positions itself between these extremes. It investigates whether ETF-like semantics can be implemented on a widely available and inexpensive microcontroller, without relying on specialized TSN hardware. By extending the existing network driver of the NXP i.MX RT1062 under Zephyr RTOS, this work explores how far software-based scheduling can be pushed with limited hardware support like a high-resolution PTP clock. The focus is not on outperforming other solutions but on understanding the achievable accuracy, iden- tifying sources of jitter, and demonstrating that deterministic, transmit-time-based communication is available on resource-constrained microcontrollers. 27 4 System Model and Problem Statement This chapter defines the system model and formalizes the problem addressed in this work. It describes the hardware and software environment in which the proposed solution operates, outlines the assumptions and limitations imposed by the platform, and justifies the need for time-based scheduling on devices with limited resources. Based on this system model, the problem statement is derived, followed by a series of performance indicators that will later be adapted to evaluate the proposed approach. 4.1 System Model In this section, the system model assumed in this thesis is described. It first starts by introducing relevant hardware components and their constraints, followed up by the used real-time operating system (RTOS). The focus is on the NXP i.MX RT1062 microcontroller and its Ethernet subsystem and Zephyr as its RTOS. 4.1.1 Hardware In this work the NXP i.MX RT1062 is used; it is a microcontroller based on an ARM Cortex-M7 core operating at frequencies of up to 600 MHz and is optimized for real-time applications requiring low interrupt latency and deterministic execution behavior. The microcontroller is intended for use in domains like industrial human-machine interfaces, Internet of Things devices, motor control systems, and home appliances [19, Section 2.3]. Because of its flexible architecture and rich peripheral set, it is also usable for a wide range of general-purpose embedded applications, providing the necessary interfaces to connect a variety of external peripherals. The processor features on-chip SRAM interfaces for external memory as well as a heterogeneous memory architecture consisting of tightly coupled memory (TCM) for instruction and data access on a single CPU cycle. This allows time-sensitive code paths like interrupt handlers or network driver routines to be executed with predictably low-latency timing when correctly placed in memory. For clock generation and distribution, a dedicated clock controller module exists; it can be used to control the input clock for various modules. Particularly relevant for TSN is the ENET PTP clock, which runs independently of the CPU execution flow with nanosecond resolution and can be synchronized using IEEE 1588 (PTP). For this, the Ethernet connectivity of the integrated ENET controller supporting 10/100 Mbit/s is used. The PTP clock supports comparing events with the help of the ENET peripheral, which triggers dedicated interrupt lines to the CPU when a specified timepoint (only specified in nanoseconds) is reached. This enables precise, time-based scheduling of software routines, which are essential for implementing TSN features. Furthermore, this clock can also be used as a common reference to other timers in the system. 29 4 System Model and Problem Statement (a) System architecture with the tsnd (b) System architecture without the tsnd Figure 4.1: Deployments of TSN applications 4.1.2 Software The microcontroller runs the Zephyr Real-Time Operating System (RTOS) [23]. Zephyr is a modular and configurable environment suitable for resource-constrained embedded systems in industrial settings, e.g., used in the Voiager e-scooters [22]. The RTOS offers a full TCP/IP networking stack, POSIX APIs, and support for multiple concurrent applications, making it a good choice for implementing TSN features on microcontrollers. OS and Networking Zephyr implements a layered networking architecture, in which applications interact with the help of, e.g., BSD socket interfaces. Internally, the network stack handles protocol processing, packet buffering, and interaction with the underlying network driver for the specific hardware. The Ethernet subsystem in Zephyr relies on hardware abstraction layers (HALs) provided by board manufacturers. For the NXP i.MX RT1062 board, Ethernet support is implemented through hal_nxp[11], which contains the ENET driver responsible for configuring the MAC, managing DMA descriptors, and handling interrupts. An example of the described transmission path can be seen in the right component of figure 4.1. The current implementation of the ENET driver exposes a conventional best-effort transmission model to the networking stack. Therefore, packets passed down from the networking stack in Zephyr to the driver are immediately enqueued into the transmit ring buffer and handed over to the hardware for transmission (further explained in 4.1.6). While this transmission model is enough for standard Ethernet communication, it does not provide mechanisms for delayed transmission, time of transmission scheduling, or other TSN-relevant features besides PTP. These limitations motivate the driver extensions of this thesis. TSN Abstraction Layer (tsnd) On top of the Zephyr networking stack, the system model assumes the existence of a dedicated TSN management daemon, referred to as tsnd. Its purpose is to act as an intermediary between applications and the network stack. Rather than requiring applications to compute the precise 30 4.1 System Model transmission timestamp themselves, tsnd is responsible for deriving appropriate transmit times based on the current schedule and configuration obtained by the CNC. These timestamps are then applied to outgoing packets using the TXTIME socket option. An example architecture of tsnd acting as an intermediary between apps and the networking stack is illustrated in figure 4.1a. This design has many advantages; the tsnd can abstract specific TSN concepts away from the application, keeping code simple and short. Besides that, tsnd provides important safety guarantees for the applications by enforcing applications to use the CNC’s specified schedule and to comply with the defined configuration, such as interval, frame size, or other TSN properties. This prevents a single misbehaving app from disturbing the complete host or network. Even though the tsnd provides advantages, it is also possible for applications to directly run on the Zepyhr network stack, benefitting from the extensions to the network driver by this thesis. But this architecture (illustrated in figure 4.1b) requires that the app be TSN-capable and that the TXTIME option be set by the app for the according use case. 4.1.3 Coexistence of TSN and Best-Effort (BE) Traffic The system model assumes that time-sensitive and best-effort traffic coexist on the same network interface. While TSN streams require bounded latency and precise transmission timing, best-effort traffic must still be transmitted without violating the guarantees of time-critical flows, for instance, to allow ARP to function. The ENET hardware only provides a single transmit ring and offers no possibility for multiple transmit queues; therefore, the responsibility for multiplexing traffic lies in software. This implies that the network driver must be able to distinguish between TSN traffic and best-effort traffic, which may be transmitted whenever no TSN traffic is pending. To prevent best-effort frames from interfering with scheduled TSN frames, the driver must enforce a guard band before each scheduled transmit time, blocking all best-effort traffic. The length of the guard band must account for the maximum transmission time of BE traffic, ensuring that no frame transmission overlaps with a scheduled TSN frame. As a result, BE traffic may be temporarily blocked even if bandwidth is available, preserving the deterministic behavior of TSN streams. 4.1.4 Time Synchronization The system model assumes the availability of precise time synchronization across all TSN devices in the network. In particular, a common global time must be established by, e.g., using PTP as defined in IEEE 802.1AS. Time synchronization is required to ensure the time-based transmission decisions are consistent across devices and that scheduled transmission occurs at the intended global time. This requirement implies that the i.MX RT1062 board using Zephyr has its time, in more detail the ENET PTP clock, synchronized to the common global time. Besides using PTP for time synchronization, the board supports connecting a Pulse-Per-Second signal to its General Purpose Input Output pins, allowing it to trigger an interrupt of the ENET PTP clock for better synchronization. The model assumes that the driver as well as higher software layers have access to the synchronized ENET PTP time. Accurate and stable time synchronization is therefore a fundamental prerequisite for extending the network driver with TSN features. 31 4 System Model and Problem Statement Figure 4.2: Complete transmission path 4.1.5 Interfaces and Network Stack This subsection aims to give an insight into what interfaces and their behavior will need change for the proposed design along the transmission path seen in figure 4.2. Application The application sends data through Zephyr’s socket API. This API uses the BSD sockets standard and already allows the specification of socket options. One of these socket options is the TXTIME option used to attach a transmission timestamp. This API as well as the socket option already exists in Zephyr, so no changes are required. Ethernet API in Zephyr Networking Stack The Zephyr networking stack uses a layered abstraction to support different Layer-2 technologies like Ethernet, CAN, and WiFi in a uniform way. For this purpose, Zephyr defines a generic L2 API that is shared across all technologies. This abstraction as well as those before the Ethernet subsystem remain unchanged in the context of this work. For communication using the Ethernet subsystem, Zephyr provides a dedicated interface to interact with hardware-specific network drivers. For the i.MX RT1062, the eth_nxp_enet driver is used. Changes to the API are listed in 4.1.5. The Ethernet component is extended to distinguish between TSN and BE traffic and selects the corresponding endpoint of the driver API. This decision is made based on the existence of the TXTIME field in the network packet. If this field is set, the packet is forwarded to the TSN endpoint of the driver; otherwise, the BE packet continues using the existing endpoint. Network Driver API The network driver API is extended by introducing a dedicated transmission endpoint for TSN traffic. This extension allows other network drivers to add TSN functionality in a consistent and transparent manner, without requiring changes to higher layers of the networking stack. Depending on which transmission endpoint is selected, the network driver applied the guard band logic for BE traffic. For TSN traffic, the network driver enqueues the frame into a priority queue and passes the 32 4.1 System Model Figure 4.3: Transmission path after Zephyr networking stack earliest frame due for transmission first to the hardware abstraction layer. Therefore, in both cases, the prepared frame is passed to the underlying hardware abstraction layer for transmission. This fork of the transmission path is indicated by two arrows going out of the ethernet component to the network driver in the figure 4.2. HAL API The hardware abstraction layer (HAL) API is not standardized across different board manufacturers. For NXP platforms, Ethernet support is provided through the hal_nxp HAL. Ethernet communication in particular for the NXP i.MX RT1062 is processed by the fsl_enet component of the HAL. Like the network driver layer, a dedicated transmission endpoint for TSN traffic is introduced. This endpoint prepares a frame for transmission by preparing the buffer descriptor ring. Despite this separation at the API level, all outgoing frames ultimately share the same transmit buffer descriptor ring and are processed sequentially by the ENET MAC and the connected PHY. 4.1.6 TX Transmission Path We will now focus on the transmission path after the Zephyr networking stack. For a full view of the path from the app to the hardware, visit 4.1.5. The ENET peripheral handles frame transmission through a descriptor-based direct memory access (DMA) engine. The complete transmission path spans from software-managed buffers over to hardware buffer descriptors to the Ethernet MAC, as illustrated in figure 4.3. 33 4 System Model and Problem Statement In the first step, the software places outgoing frames in memory buffers, for example, tx_frame_buf in the figure. This buffer contains the fully formatted Ethernet frame, including headers and payload. After calling the network driver, the payload of the frame gets copied into the txBuffer. Alongside that, the driver prepares the corresponding txBdRing buffer slot. The transmit buffer descriptor ring (txBdRing) is a ring buffer containing metadata required by the ENET DMA engine, most relevantly including control flags and a pointer to a buffer containing the payload of the frame. The following control flags are essential for sending frames: • READY: Indicates whether the current buffer slot is ready to transmit. If set, the ownership of the buffer slot is owned by the hardware and shall not be modified by the driver. • LAST: Indicates the final descriptor of a frame. This is required to send payloads longer than the available buffer space in txBuffer (e.g., for Jumbo frames), so multiple buffer slots are used. • TX_CRC: Tells the MAC to compute and append the CRC. • WRAP: Indicates the end of the ring buffer in memory. This is required for the hardware to form the circular structure of the ring. For operation, two ENET registers are needed: the TDAR (Transmit Descriptor Active Register) and the TDSR (Transmit Buffer Descriptor Ring Start Register). The TDSR register points to the base address of the txBdRing, while writing to the TDAR signals the ENET hardware that a new buffer descriptor is available for transmitting. After signaling the ENET hardware, the DMA engine scans the next buffer descriptor in the ring and checks if the READY bit is set. The location is calculated by the base address provided by the TDSR register alongside an index. This operation is strictly sequential, so the ring buffer ensembles a FIFO queue. After the ENET DMA engine encounters a ready descriptor, it uses the reference frame buffer to send the data. If this descriptor did not set the LAST bit, it will scan the next descriptor until the LAST bit is set. The ENET controller then transmits the frame on the Ethernet interface and, if specified, appends the CRC. After transmission, the ENET hardware resets the READY bit of the involved descriptors, and the ownership is transferred back to software. The network driver thereby can reuse the descriptor and txBuffer for future traffic. Besides that, the ENET hardware also triggers a transmit interrupt, allowing the network driver to signal to upper software layers that the frame has been sent successfully and, if enabled, to retrieve the exact timestamp of the transmission (useful, for example, for gPTP). This process continues cyclically as long as frames are sent. The ENET controller also has some restrictions, e.g., it only supports a single buffer descriptor ring and processes descriptors strictly in ring order. The skipping of descriptor slots, or the adjusting of the TDSR register, is prohibited, as later explained in more detail in the design chapter. This means there is no native support for multiple hardware transmit queues, priority-based, or delayed transmission based on a target timestamp. Therefore, implementing TSN features for the NXP i.MX RT1062 microcontroller must solely rely on software with hardware support as well as serialize the frames into the same buffer descriptor. In particular, software must carefully control when descriptors are enqueued into the txBdRind, marked as ready, and when the ENET hardware gets triggered. This constraint of the commonly used microcontroller motivates the extensions in the driver proposed in this thesis to enable similar behavior as Linux’s ETF qdisc. 34 4.2 Problem Statement 4.2 Problem Statement Time-Sensitive Networking focuses on deterministic communication over Ethernet by providing precise control over frame transmission times. On general-purpose systems such as Linux, these capabilities are implemented by queueing disciplines (qdiscs) like the Earliest Transmit Time First (ETF), which allows frames to be scheduled for transmitting at a specific point in time. On the contrary, a commonly used embedded microcontroller typically lacks the support for TSN, despite being widely used in industrial applications. For example, the NXP i.MX RT1062 provides fundamental features required for TSN, like a high-resolution PTP clock with IEEE 1580 capabilities and its ability to trigger interrupts at a specified time. However, its Ethernet subsystem is limited to a single transmit ring buffer and does not natively support delayed or time-triggered frame transmission. Furthermore, the Zephyr RTOS only implements basic (extension of the network packet struct with a timestamp), and the NXP network driver has no TSN support. This gap between available TSN concepts and the capabilities of the embedded networking stack poses a challenge for deterministic Ethernet communication on systems built with microcontrollers. In particular, it is unclear how transmit time-based scheduling mechanisms such as ETF can be adapted to resource-constrained environments with limited hardware support for TSN while still maintaining compatibility with existing applications and best-effort traffic. The problem addressed in this thesis is therefore the design, implementation, and evaluation of a qdisc similar to ETF for the NXP i.MX RT1062 using the Zephyr RTOS. Its purpose is the demonstration of deterministic, transmit time-triggered Ethernet communication on a microcontroller using a real-time operating system, without requiring specialized TSN hardware. From this, the following objectives can be identified: 1. Design an ETF-like transmit-time scheduling mechanism for Zephyr RTOS and the hal_nxp network driver 2. Implement the designed solution 3. Support coexistence of time-sensitive and best-effort traffic 4. Evaluate the timing accuracy of the implementation 5. Analyze sources of timing jitter 4.2.1 Design Objectives The primary objective of this work is to provide ETF-like transmit-time scheduling semantics for Ethernet communication on the NXP i.MX RT1062 running the Zephyr RTOS. With this extension, the application is able to use the socket API to specify an absolute transmit time for individual packets. This behavior is the same as in Linux-based systems, allowing applications to specify the TXTIME socket option. Besides supporting the scheduling of TSN packets, the proposed solutions must also support the coexistence of Best Effort traffic on the same network interface without violating timing guarantees for TSN. Furthermore, the solution is designed to integrate into the existing Zephyr networking stack, using standard socket interfaces and minimizing the 35 4 System Model and Problem Statement need for application changes. In line with this, another objective is to keep the impact on the existing networking stack as small as possible, limiting modifications to the Ethernet driver and the underlying HAL where necessary. 4.2.2 Scope The scope of this design is intentionally narrow and specialized. The solution is specifically developed for the NXP i.MX RT1062 microcontroller and its Ethernet subsystem. Therefore, the design does not aim to provide generic TSN scheduling for all devices supported by Zephyr but rather to a targeted group of platforms that exploit the capabilities and limitations of this particular hardware. Within this scope, the design focuses on the packet transmit path of the network stack. Only a single network interface is considered, and all scheduling decisions are performed by the network driver in software as no hardware-assisted scheduling or offloading is available. Explicitly out of scope is: • Support for different microcontroller platforms • Support for multiple interfaces • Dynamic configuration (e.g., via YANG with the help of a CNC) 4.3 Key Performance Indicators The performance of the ETF-like qdisc is primarily determined by how accurately and consistently frames are transmitted about their targeted transmission time. For the scope of this thesis, the relevant key performance indicators are Transmission time error, Worst-case transmission time error, and Minimum guaranteed time window. Unlike classical networking metrics, such as end-to-end latency or throughput, the central concern in this work is the temporal correctness of the transmission event itself. Therefore, the focus is not on how fast traffic can be sent but rather on how deterministic, making e.g., bandwidth a secondary goal, which is not in the scope of this thesis. 4.3.1 Transmission Time Error The transmission time error is the primary metric and measures the deviation of the actual transmission compared to the targeted time. For a frame 𝑖 that is scheduled at time 𝑡𝑠𝑐ℎ𝑒𝑑,𝑖 but sent at 𝑡𝑡 𝑥,𝑖, the transmission time error calculates the following: 𝛿𝑖 = 𝑡𝑡 𝑥,𝑖 − 𝑡𝑠𝑐ℎ𝑒𝑑,𝑖(4.1) 36 4.3 Key Performance Indicators A positive value of 𝛿𝑖 indicated the frame was sent later than targeted, while a negative value indicated that it was transmitted earlier. Both cases are relevant as they represent deviations from the intended transmission point, interfering with the desired deterministic behavior. Therefore, we want 𝛿𝑖 to be as closely distributed around 0 as possible. 4.3.2 Wort-Case Transmission Time Error To measure the worst-case behavior of the system, the maximum observed lateness or earliness can be given by: 𝛿𝑚𝑖𝑛 = min 𝑖 𝛿𝑖(4.2) 𝛿𝑚𝑎𝑥 = max 𝑖 𝛿𝑖(4.3) Here 𝛿𝑚𝑖𝑛 represents the earliest observed transmission and 𝛿𝑚𝑎𝑥 the latest transmission. Worst-case bounds are emphasized in TSN, as deterministic systems must be designed to tolerate the worst possible deviation rather than average behavior. 4.3.3 Minimum Guaranteed Transmission Window A key metric to measure the precision of the transmit-time-based system is the minimum guaranteed transmission window. This window represents the smallest time interval around the scheduled transmitted time within all packets that must be transmitted. Using the worst-case transmission time error, the minimum guaranteed window with 𝑊𝑚𝑖𝑛 is defined as: 𝑊𝑚𝑖𝑛 = |𝛿𝑚𝑖𝑛 | + |𝛿𝑚𝑎𝑥 |(4.4) Following in this thesis, we will call 𝑊𝑚𝑖𝑛 the accuracy of the ETF-like implementation. The closer-to-zero 𝑊𝑚𝑖𝑛 is, the better the accuracy of the system. This accuracy is a practically relevant metric, as it determines how closely scheduled transmission could be placed without overlap or violation of timing guarantees. 37 5 Design This chapter describes the design of the approach developed in this work for time-based transmission scheduling. The goal is to enable ETF-like transmission semantics on the NXP i.MX RT1062 with Zephyr, despite the limited hardware support for time-sensitive networking. The design must take into account the limitation that the ENET controller provides only a single ring of transmit buffer descriptors. The chapter begins with an explanation of the design principles and requirements arising from the problem statement, and the characteristics of the target platform. Based on these requirements, the internal transmission path for scheduled traffic is presented, including buffer management, time-based triggering of the transmission routine, and the coexistence of scheduled and best-effort traffic. Finally, design approaches that were implemented but did not work due to hardware limitations are examined, as they provide important information on the behavior of the ENET DMA engine and motivate the final design. 5.1 Design Rationale This section explains the reasoning behind the design decisions taken in this thesis. The goal of the design rationale is to motivate why specific mechanisms, interfaces, and constraints were chosen, based on the problem statement and the characteristics of the target platform. Instead of describing how the solution is designed, this section focuses on what requirements the design must fulfill and why these requirements are necessary. The design rationale is structured into functional and non-functional requirements. Functional requirements describe the concrete capabilities the system must provide to support transmit-time- based scheduling, while non-functional requirements capture constraints related to timing accuracy or resource usage. Together, these requirements form the basis for the design choices presented in the following sections. 5.1.1 Functional Requirements In order to fulfill its function, the design must satisfy several functional requirements. The network driver must be able to accept TXTIME metadata from the Zephyr network stack and use this information to distinguish between TSN packets and Best Effort (BE) traffic. Thereby packets that have the TXTIME set are treated as TSN traffic, and packets that don’t have the TXTIME set are classified as BE traffic. The TXTIME needs to be interpreted as absolute global network time, based on the PTP clock provided by the ENET hardware, which has IEEE 1588 capabilities. After the network driver receives a TSN packet from the Zephyr networking stack, the proposed design needs to prepare as much as possible for the upcoming transmission beforehand. This includes, 39 5 Design e.g., copying the payload of the frame to the dedicated DMA memory section used by the ENET hardware. After this, the network driver must make sure that the transmission routine sending the packet gets triggered as close to the specified TXTIME as possible. To ensure that BE traffic does not interfere with TSN traffic, the design must implement a guard band that prevents BE frames from being sent too close before TSN traffic. Due to hardware limitations, the design must operate on a single transmit buffer descriptor ring (txBdRing). Therefore, the design must guarantee that BE transmission cannot occupy buffer slots reserved for TSN traffic. 5.1.2 Non-Functional Requirements In addition to functional correctness, the design must meet several non-functional requirements. Most importantly, the transmission time error must be bounded, ensuring deterministic behavior suitable for TSN applications. The design aims to keep driver complexity to a minimum, trying to avoid changes to the already existing part of the Zephyr networking stack. While operating, the design should keep memory and CPU consumption within reasonable limits for microcontrollers. Finally, the design should be maintainable and extensible, allowing future work to build upon the implemented mechanisms. 5.2 Transmission Path Design This section describes the internal transmission path of scheduled traffic, starting from the network stack and ending at the hardware abstraction layer (HAL). The focus is on how transmit-time-based scheduling is implemented despite the hardware limitation of a single transmit buffer descriptor ring. This section explains how buffers are allocated and managed, how the transmit operation is triggered on a time basis, how buffer descriptors are prepared, and how the driver updates its scheduling state after a transmission is complete. 5.2.1 Buffer Allocation / Usage All buffers used for transmission are statically allocated during the initialization phase of the Ethernet driver. This includes both payload buffers and auxiliary data structures required for scheduling. By avoiding allocating buffers dynamically at runtime, the design ensures predictable timing behavior and prevents possible bugs. For BE traffic, buffer usage is bounded by a counting semaphore that limits the number of frames that may be enqueued concurrently. This mechanism was already in place before the design was implemented and now ensures that BE traffic is restricted to a specific number of slots in the ring buffer. Remaining slots are used for future TSN traffic. TSN traffic uses a separate transmit buffer to store its frame payload, with one buffer allocated per TSN stream. When a TSN packet is received for transmission by the network driver, its payload as well as necessary metadata for transmission are copied into a priority queue. The network driver ensures that every time a change in the priority queue is made, the current top frame is copied into the DMA buffer, and the necessary metadata for transmission is visible for the HAL. 40 5.2 Transmission Path Design Figure 5.1: TXTIME frame admission from the network stack and scheduling preparation in the network driver This priority queue is sorted based on the transmission timestamp and thereby stores information related to the scheduling of the TSN frame, such as the absolute transmission time and transmit descriptor fields. A dedicated pointer in the driver tracks the next TSN frame that is due for transmission and is used by the scheduled transmit routine to select the appropriate metadata as well as the payload entry. 5.2.2 Frame Admission and Scheduling Preparation When the network driver receives a frame annotated with a target transmission timestamp via the dedicated scheduled-transmission endpoint from the Zephyr networking stack, it is admitted into the scheduling system rather than being transmitted immediately. As illustrated in figure 5.1, the driver treats such frames as scheduled traffic and inserts them into a priority queue that is ordered by the target transmission time. This queue represents all pending transmissions and enforces strict temporal ordering. During frame admission, the driver must ensure that the frame currently prepared for transmission in the DMA-accessible buffer always corresponds to the earliest scheduled transmission. For this reason, after inserting a new frame into the priority queue, the driver checks whether the head of the queue has changed. If changes are detected, the previously prepared frame in the DMA buffer is no longer the next frame marked for transmission. 41 5 Design Figure 5.2: Triggering of the transmit routine using an interrupt from the PTP clock In this case, the driver updated its scheduling state by replacing the prepared DMA buffer contents with the payload of the new queue head. The transmission time of this frame is then used to update the PTP compare value that triggers the transmit routine via an interrupt. If the head of the queue remains unchanged, no update is required, and the existing DMA buffer contents and interrupt configuration remain valid. By immediately updating the DMA buffer on changes of the scheduling state, the design avoids buffer manipulation close to the transmission deadline and minimizes the amount of work required in the time-critical transmit routine. Therefore, the DMA buffer, along with the PTP compare value, always reflects the earliest scheduled transmission. 5.2.3 Triggering the Transmit Routine The design relies on the PTP compare event offered by the PTP clock of the ENET controller to achieve time-based triggering of the transmit routine. The transmit routine is scheduled to run a small time interval before the actual transmission time to provide enough time for computation. As illustrated by figure 5.2, the PTP clock operates independently of the CPU and is therefore not affected by scheduling delays or software timers. When the preprogrammed compare value is reached, the hardware generates an interrupt event. The clock uses a timer counter with nanosecond resolution and provides compare registers that can be programmed with an absolute nanosecond time value. It should be noted that this clock does not keep track of the seconds but only about the current nanoseconds within a second. Therefore, this counter periodically resets itself every second. 42 5.2 Transmission Path Design Figure 5.3: Design to multiplex TSN and BE traffic To minimize interrupt latency, the compare event interrupt is handled using Zephyr’s zero-latency interrupts (ZLI). This special interrupt bypasses the regular kernel interrupt handling path of Zephyr and is executed immediately. This prevents delay by kernel scheduling, interrupt masking, or context switching and is therefore well suited for TSN, where even small variable interrupt latencies can lead to timing errors. But this design choice also prevents the interrupt routine from reliably using the kernel API for features like semaphores, mutexes, or data structures and prevents the kernel from applying an IRQ lock to disable the interrupt temporarily [21]. 5.2.4 Transmit Routine and TX Buffer Descriptor Ring Interaction Figure 5.3 illustrated the central design concept used to multiplex TSN and BE traffic within a single transmit buffer descriptor ring (txBdRing). The txBdRing remains unchanged in its structure and continues to be managed as a circular ring by the ENET DMA engine. One generator index is used to point to an empty buffer descriptor, and one consumer index is used to identify the last buffer descriptor used to send a frame by the DMA. Both indexes must strictly follow the sequential ordering requirement imposed by the hardware. Each descriptor contains transmission metadata, which is further described in subsection 4.1.6 and a pointer to a DMA region holding the Ethernet frame. A key challenge of the design is to support scheduled transmissions alongside best-effort traffic while respecting the given hardware constraints. This is addressed by introducing a dedicated transmit buffer (txBuffer) in DMA-accessible memory for scheduled traffic while leaving the descriptor ring itself untouched. Best-effort traffic continues to use the default transmit buffers and follows the standard transmission path. In contrast, TSN frames are copied into a separate TXTIME transmit buffer. At transmission time, descriptors in the ring dynamically reference either a BE txBuffer or the TXTIME txBuffer, depending on the traffic type. To guarantee that scheduled traffic can always 43 5 Design Figure 5.4: Transmission completion and scheduling state update be transmitted, the design restricts the number of consecutive occupied transmit buffer descriptor slots for BE traffic. Assigning fixed or dedicated slots in the txBdRing to individual traffic classes or even duplicating the txBdRing is not possible due to constraints discussed in section 5.4. For scheduled traffic, frames are placed into a priority queue ordered by their target transmission time by the network driver (explained in subsection 5.1). Whenever the head of this queue changes, the network driver updates the TXTIME txBuffer accordingly. As a result, the next scheduled frame is always already present in the txBuffer well before its transmission time. This allows memory copies and buffer preparation to be completed in advance, reducing the amount of work required in the time-critical execution path. When the scheduled transmission target time approaches, the transmit routine is triggered by the PTP’s clock compare event (described in subsection 5.2.3). At this point in time, only a minimal set of actions is performed: the next descriptor indicated by the generator index is populated, the buffer pointer is set to reference the TXTIME txBuffer, the corresponding control flags are written, and then the descriptor is marked ready for transmission by setting the READY flag. The current time within the second is then read from the PTP clock, and a short busy-wait phase based on a calibrated number of no-operation (NOP) instructions is executed to compensate for residual jitter. Finally, after the busy-waiting, the DMA engine is triggered to transmit the frame. 5.2.5 Transmission Completion and Scheduling Update As illustrated in figure 5.4, once the MAC completes the transmission of a frame, an interrupt is delivered to the NXP network driver. The driver reclaims the transmit buffer descriptor at the consumer index and uses this event to update its scheduling state. 44 5.3 Best-Effort Traffic Integration If the reclaimed descriptor references the TXTIME txBuffer, the driver removes the frame from the priority queue and reevaluates the scheduling state. This results in computing a new head in the queue, requiring the TXTIME txBuffer as well as the PTP compare event to be updated. The driver performs the same steps already described in subsection 5.1 to perform the state update. 5.3 Best-Effort Traffic Integration Since the hardware does not support frame preemption or multiple transmission queues, BE traffic must be controlled to prevent interference with TSN traffic. To achieve this, the proposed design introduces a guard band at the network driver level that temporarily blocks BE transmission shortly before a scheduled TSN frame is due to be sent. This is done by the network driver by first checking whether any scheduled frames are currently pending after receiving a BE frame from the Zephyr networking stack. If that’s not the case, the BE traffic is forwarded immediately for transmission. But if one or more frames are pending, the driver obtains the currently programmed PTP compare value used to schedule the transmit routine. A fixed but configurable time window before the scheduled transmission time defines the width of the guard band. If the current time lies outside this guard band window, the BE frame proceeds normally. However, if the current time is within the guard band’s time window, the BE frame is temporarily blocked. This prevents the BE traffic from writing to the same buffer descriptor the TSN transmission routine would use, resulting in misbehavior, but also prevents the MAC from being occupied at transmission time. To avoid busy waiting and therefore unnecessary CPU usage, blocked BE traffic is suspended using a condition variable provided by the Zephyr kernel. A condition variable allows a thread to sleep until a specified condition becomes true, rather than permanently checking the condition. In this design the condition variable is signaled from the transmission callback interrupt after the TSN frame has been sent. Once signaled, the blocked BE thread reevaluates the current timing state and resumes transmission if no further guard band restrictions apply. 5.4 Unsuccessful Transmission Path Design Approaches Before arriving at the final design presented in 5.3, several alternative approaches were explored. These approaches appeared promising at first, as they would have simplified the implementation and potentially further reduced transmission time error. However, hardware constraints discovered with those approaches revealed that they were incompatible with the behavior of the ENET DMA engine. This section aims to describe these approaches and summarize them with a lesson learned. 5.4.1 Ring Buffer Partitioning The first design attempt seen in figure 5.5 tried to statically partition the transmit buffer descriptor ring into separate regions for best-effort and time-sensitive networking traffic. Under this design, each traffic class would have two virtual buffers, each with their own respective generator and consumer indexes in the transmit ring buffer. 45 5 Design Figure 5.5: Unsuccessful design to multiplex TSN and BE traffic using ring buffer partitioning This approach was attractive for several reasons. It would have simplified buffer management, and eliminated the need to dynamically assign txBuffers holding the payload to the buffer descriptor slots at runtime. As a result, TSN transmission would have been completely prepared by the time the transmission routine gets triggered, and only the routine would have to trigger the ENET DMA for transmission. However, this approach assumes that either the DMA engine scans the whole transmit buffer descriptor ring for frames, which have the READY flag set, or when using pointers indicating the next buffer descriptor for transmission requiring the ability to skip certain buffer slots. During implementation and testing, it became clear that the ENET DMA engine does not support this behavior. Further, the DMA strictly processes buffer descriptors sequentially and assumes that the next descriptor in the ring is valid. Attempting to skip or bypass descriptors, or the assumption that the DMA would scan the entire ring buffer, causes the transmit engine to stop functioning completely. This was later confirmed by the NXP support, stating “The answer is no, and it is indeed related to hardware constraints; the transmit buffer descriptor ring (txBdRing) of the Ethernet controller (ENET) typically has strict sequential access requirements” [12]. As this aspect of the DMA engine’s behavior is not available in more detail publicly, it could not be found out from the reference manual or other official sources and had to be identified through implementation and experimental evaluation. 5.4.2 Dynamic Ring Switching The second attempt at multiplexing the traffic was by having two independent transmit buffer descriptor rings, one dedicated for best-effort traffic and one dedicated to time-sensitive networking traffic, as seen in figure 5.6. Each ring would have its own transmit buffers and generator / consumer indexes. Depending on the type of traffic to be sent, the driver would dynamically update the Transmit Buffer Descriptor Ring Start Register (TDSR) to point the ENET DMA engine to the appropriate ring. Similar to the partitioning approach at 5.4.1, this design would have reduced complexity and allowed descriptors as well as its payload to be fully prepared in advance before the actual scheduled transmit routine. 46 5.4 Unsuccessful Transmission Path Design Approaches Figure 5.6: Unsuccessful design to multiplex TSN and BE traffic using dynamic ring switching In practice, this approach also proved infeasible. While it is possible to configure the TDSR register during initialization, modifying it at runtime after the DMA engine has been started caused the transmit engine to stop functioning completely too. Therefore, it implies that the DMA engine of the MAC internally maintains some kind of hardware state, not exposed to the HAL or network driver, which assumes a fixed descriptor ring address throughout the operation. As a result, switching the rig pointer dynamically is not supported by the hardware. 5.4.3 Summary and Lessons Learned Although the investigated design approaches were not possible in practice, they led to important insight into the behavior of the ENET DMA engine. Experimental evaluation showed that the DMA strictly processes a single transmit buffer descriptor ring in sequential order and does not support skipping descriptors or changing the ring base address after initialization. These constraints are not documented publicly and therefore could not be found out beforehand. Identifying them required implementation and testing that directly influenced the final design choices presented. 47 6 Implementation This chapter describes the concrete implementation of the transmit-time-based scheduling design introduced in chapter 5. It focuses on how the conceptual design decisions are realized within the Zephyr networking stack and the NXP driver, while respecting the constraints of the underlying hard- ware and operating system. To do so, this chapter highlights the relevant components, configuration operations, interface extensions, and core algorithms that enable scheduled transmission. 6.1 Overview This section provides an overview of the main software components involved in the implementation of the transmit-time-based scheduling. This solution is integrated into the Zephyr networking stack and spans multiple abstraction layers, ranging from the generic Ethernet subsystem down to the hardware abstraction layer of the ENET controller. The following components will be described further in this section: • zephyr/subsys/net/l2/ethernet/ethernet.c • zephyr/drivers/ethernet/eth_nxp_enet.c • zephyr/drivers/ptp_clock/ptp_clock_nxp_enet.c • hal_nxp/mcux/mcux-sdk-ng/drivers/enet/fsl_enet.c 6.1.1 ethernet.c The ethernet.c component is part of the generic Zephyr networking stack and implements the Ethernet Layer-2 subsystem. Its main responsibility is to abstract Ethernet functionality from higher network layers and to forward outgoing L2 traffic to the appropriate network driver specific to the hardware. To achieve this, the Ethernet subsystem defines a common API that is used to communicate with the Ethernet drivers. This abstraction allows different Ethernet implementations to coexist while presenting a uniform interface to the rest of the networking stack. In the context of this work, ethernet.c was extended to classify outgoing traffic into TSN and BE traffic. Based on the classification, packets are forwarded to different transmission endpoints provided by the network driver. This extension enables specific handling for TSN without modifying the generic Ethernet L2 abstraction or affecting other network technologies supported by Zephyr. 49 6 Implementation 6.1.2 eth_nxp_enet.c The eth_nxp_enet.c component implements the Ethernet network driver specific for the NXP i.MX RT1062 board. It is invoked by the Ethernet subsystem and is responsible for configuring the ENET MAC handling packet transmission by interacting with the underlying hardware abstraction layer. This driver was extended to support scheduled transmission for TSN traffic in addition to the existing BE transmission path and managing a priority queue of TSN frames. A dedicated transmission routine was introduced for TSN traffic to enqueue them in the min-heap data structure used as a priority queue, along with an additional callback handling distinguishing between TSN and BE traffic. The driver also implements the guard band that prevents BE transmissions from interfering with scheduled TSN frames. Furthermore, the driver is responsible for allocating and managing the buffers passed to the HAL. This also includes updating the current txBuffer in the DMA of the next due TSN frame when the min heap gets updated as well as sharing the new transmission metadata with the HAL. 6.1.3 ptp_clock_nxp_enet.c The ptp_clock_nxp_enet.c component provides the PTP clock driver for the specified board. It exposes access to the hardware PTP clock, which serves as the global time reference for tasks like frame timestamping and TSN scheduling. Within this implementation, the PTP clock driver is used to call the transmission routine after the interrupt of the used PTP compare channel got triggered. 6.1.4 fsl_enet.c The fsl_enet component is the hardware abstraction layer (HAL) of the enet controller on the NXP platform. It provides low-level access to the ENET MAC, DMA engine, PTP timer, and various registers for configuration and is called by both the network driver and the PTP clock driver. 6.2 Configuration This section describes how the implementation is configured and integrated into the Zephyr build system. 6.2.1 Zephyr Kconfig System Kconfig is a configuration system originally developed for the Linux kernel to manage the large number of build-time options required by a modular operating system. It provides a structured way to describe configuration parameters, their dependencies, and valid value ranges. Over time, Kconfig has been adopted by other platforms, including Zephyr, as a standardized way for built-time configuration. 50 6.2 Configuration In general, Kconfig is used to control which components are included in building the system, how they are parameterized, and how configuration options depend on each other. Configuration options are defined in dedicated Kconfig files and can represent boolean features, numerical values, or string parameters. During configuration, options are selected either through configuration files such as prj.conf or interactive configuration tools. By building the system, these options are resolved and translated into preprocessor definitions that are accessible to the source code. Zephyr makes extensive use of Kconfig to configure the operating system kernel, device drivers, networking stack, hardware-specific features, and many more. For example, Kconfig is used to enable or disable subsystems, configure memory sizes, and control driver behavior. This approach allows a single code base to be adapted to a wide range of hardware platforms and application requirements without needing to change source code. 6.2.2 Design-related Configuration Parameters The implementation relies on a set of configuration parameters that control timing behavior, resource allocation, and hardware interaction. These parameters are defined using Zephyr’s Kconfig system and are resolved at build time. The config parameters can be divided into two groups: newly introduced options specific to this work and already existing Zephyr options that must be configured to support the implementation. Introduced Configuration Parameters The following Kconfig options were introduced as part of this implementation to parameterize the ETF implementation: • CONFIG_ETH_NXP_ENET_TXTIME_PTP_CHANNEL This option is used to specify the PTP channel that should be used for the PTP compare event to schedule the transmit routine. • CONFIG_NET_CONTEXT_TXTIME_BUFFERS This option specifies how many buffers are reserved exclusively for TSN traffic. • CONFIG_NET_CONTEXT_TXTIME_SCHED_INTERRUPT_MARGIN This option specifies how far in advance the PTP compare interrupt is programmed (in nanoseconds) before the targeted transmission time. • CONFIG_NET_CONTEXT_TXTIME_SCHED_WINDOW_BEFORE Defines a tolerance window where the TSN frame is allowed to be sent before the target time (in nanoseconds). • CONFIG_NET_CONTEXT_TXTIME_SCHED_WINDOW_AFTER Defines a tolerance window where the TSN frame is allowed to be sent after the target time (in nanoseconds). • CONFIG_NET_CONTEXT_TXTIME_GUARD_BAND_WIDTH Defines a fixed guard band’s width in nanoseconds in which BE traffic is blocked prior to a scheduled TSN frame’s transmission time. 51 6 Implementation Existing Configuration Parameters In addition to the newly introduced parameters, several existing Zephyr configuration options must be enabled: • CONFIG_MIN_HEAP This option enables the support of min heaps. The min heap is used as a priority queue of pending TSN frames in the network driver. • CONFIG_ZERO_LATENCY_IRQS This option enables zero-latency interrupts used for the transmission routine. The interrupt can be set to the highest priority, which will not be blocked by interrupt locking. • CONFIG_NETWORKING This option enables generic link layer and networking support. •