UART Serial Communication Guide: Principles, Parsing & Visualization

As one of the most fundamental communication methods in embedded systems and the Internet of Things (IoT), serial communication (UART) is a core technology that every hardware/software engineer must master. It serves not only as a “window” for device debugging but also as a “link” for data exchange between sensors, microcontrollers, and industrial equipment. This article will start with the underlying principles of serial communication, explain protocol parsing logic and data processing methods in detail, and demonstrate how to efficiently implement visual analysis of serial data using practical tools—helping you truly grasp the key essentials of serial communication technology.

I. The Underlying Logic of Serial Communication: Why Can Data Be Transmitted with Just Three Wires?

Many engineers use serial communication regularly, yet they may not fully understand its underlying communication logic. At its core, serial communication is asynchronous serial communication, which enables bidirectional data transmission via two data lines (TX for transmission, RX for reception) and a GND line for reference voltage. Unlike SPI or I2C, it does not require a clock line for synchronization; instead, it relies on “pre-agreed timing” to ensure accurate data reception.

1.1 Core Parameters of Serial Communication: The “Language Rules” Determining Data Transmission

To establish stable serial communication, both parties must first align on “language rules”—specifically, the following 5 core parameters:

• Baud Rate: The number of binary bits transmitted per unit time. Common values include 9600, 115200, and 38400 bps. For example, 9600 bps means 9600 bits are transmitted per second (including start bits, data bits, parity bits, and stop bits); the actual effective data rate excludes control bits.
• Data Bits: The number of valid data bits in each data frame, typically 8 bits (corresponding to one byte) or 7 bits (compatible with ASCII encoding).
• Stop Bits: A flag indicating the end of a data frame, configurable as 1, 1.5, or 2 bits. It gives the receiver time to prepare for the next frame.
• Parity Bit: Used to check for transmission errors. Options include odd parity (the total number of 1s in data bits + parity bit is odd), even parity (the total number of 1s is even), and no parity (the most common choice, relying on upper-layer protocols for error tolerance).
• Flow Control: An optional parameter to prevent data overflow (e.g., RTS/CTS hardware flow control, XON/XOFF software flow control). It is unnecessary in most simple scenarios (e.g., sensor data transmission).

Key Principle: The transmitter encapsulates data into frames (start bit + data bits + parity bit + stop bit) based on the parameters. The receiver parses the frame structure using the same parameters—mismatched parameters will result in garbled data (e.g., “####” or random characters when baud rates differ).

1.2 Serial Data Frame Structure: How a Frame Is “Split” and “Identified”

Serial communication transmits data in “frames.” The standard frame structure (using 8 data bits, 1 stop bit, and no parity as an example) is as follows:

• Start Bit (1 bit): A low level (logic 0) that signals the start of a frame, breaking the previous high-level idle state.
• Data Bits (8 bits): Transmitted from the least significant bit (LSB) to the most significant bit (MSB). For example, to send the byte 0x5A (binary 01011010), the actual transmission order is 0→1→0→1→1→0→1→0.
• Stop Bit (1 bit): A high level (logic 1) that marks the end of a frame. Configurable as 1, 1.5, or 2 bits, it ensures the receiver has sufficient time to prepare for the next frame.

Example: To send the character “A” (ASCII code 0x41, binary 01000001), the complete data frame is:
Start bit (0) → Data bits (1→0→0→0→0→0→1→0) → Stop bit (1)

The receiver triggers data reception by detecting the “low-level start bit,” then samples subsequent bits synchronously based on the baud rate, and finally reconstructs the complete byte.

II. Serial Protocol Parsing: How to Extract Valid Data from a “Binary Stream”?

In practical projects, serial communication rarely transmits single bytes—it typically sends “data packets” encapsulated in custom protocols (e.g., sensor data, device control commands). Parsing byte-by-byte directly leads to data confusion, so mastering core protocol parsing methods is essential.

2.1 Common Custom Serial Protocol Formats

Most projects adopt a protocol structure of “Header + Length + Data + Checksum” to ensure data integrity and identifiability. A typical format is shown below:

Why This Structure?
Suppose a sensor sends temperature and humidity data once per second. Without a header/trailer, the receiver might misidentify “residual data from the previous frame” or “interference noise” as valid data. By using a fixed header (e.g., 0xAA), the receiver first filters out non-header data, extracts the complete data segment based on “data length,” and finally verifies data correctness via the checksum.

2.2 Core Logic of Protocol Parsing: Implementation with a State Machine

The optimal way to parse custom serial protocols is to use a state machine, which processes each field of the data frame through different states to avoid data sticking or loss. Taking the protocol “0xAA (header) + 1-byte length + N-byte data + 1-byte XOR checksum” as an example, the state machine design is as follows:

Step 1: Define Parsing States

// Example in C; logic applies to other languages
typedef enum {STATE_WAIT_SOF = 0,  // Wait for header (0xAA)
    STATE_GET_LEN,       // Receive data length
    STATE_GET_DATA,      // Receive data segment
    STATE_GET_CRC,       // Receive checksum
    STATE_CHECK_CRC      // Verify and process data
} ParseState;

Step 2: Process Each Byte of Data by State

ParseState state = STATE_WAIT_SOF;
uint8_t data_buf[64] = {0};  // Data buffer
uint8_t data_len = 0;        // Length of data segment
uint8_t data_idx = 0;        // Index for data segment reception
uint8_t crc_calc = 0;        // Calculated checksum value

void parse_serial_data(uint8_t byte) {switch(state) {
        case STATE_WAIT_SOF:
            if (byte == 0xAA) {  // Header detected
                state = STATE_GET_LEN;
                crc_calc = byte;  // Initialize checksum with header
            }
            break;
        
        case STATE_GET_LEN:
            data_len = byte;
            crc_calc ^= byte;  // Accumulate checksum
            if (data_len > 0 && data_len <= 60) {  // Limit length to prevent buffer overflow
                state = STATE_GET_DATA;
                data_idx = 0;
            } else {state = STATE_WAIT_SOF;  // Reset on invalid length}
            break;
        
        case STATE_GET_DATA:
            data_buf[data_idx++] = byte;
            crc_calc ^= byte;
            if (data_idx == data_len) {  // Data segment reception complete
                state = STATE_GET_CRC;
            }
            break;
        
        case STATE_GET_CRC:
            if (byte == crc_calc) {  // Checksum verified
                state = STATE_CHECK_CRC;
            } else {state = STATE_WAIT_SOF;  // Reset on checksum failure}
            break;
        
        case STATE_CHECK_CRC:
            // Process data after verification (e.g., extract temperature and humidity)
            uint16_t temp = (data_buf[0] << 8) | data_buf[1];  // Temperature (16-bit)
            uint16_t humi = (data_buf[2] << 8) | data_buf[3];  // Humidity (16-bit)
            printf("Temperature: %.1f°C, Humidity: %.1f%%\n", temp/10.0, humi/10.0);
            
            // Reset state to wait for next frame after processing
            state = STATE_WAIT_SOF;
            break;
    }
}

Key Considerations:

• Limit data length to prevent buffer overflow from malicious data.
• The checksum must include the header, length, and data segment to ensure the integrity of the entire frame.
• If data reception is incomplete for an extended period (e.g., timeout), reset the state machine to avoid system halts.

III. Serial Data Processing and Visualization: From “Byte Stream” to “Intuitive Charts”

After extracting valid data, how can you quickly analyze data trends (e.g., sensor data changes over time)? The traditional method—exporting data to Excel and plotting manually—is highly inefficient. Tools enable “real-time parsing + visualization” integration, significantly improving debugging efficiency.

3.1 Core Requirements for Data Visualization: Real-Time Performance and Flexibility

Serial data visualization must meet two core scenarios:

• Real-Time Monitoring: For example, when debugging environmental monitoring devices, you need to view real-time curves of temperature, humidity, or air pressure.
• Historical Review: For example, when testing device stability, you need to record data over hours to analyze abnormal fluctuations.

The key to fulfilling these requirements is “linkage between data parsing and chart rendering”—after the parsing module extracts valid data, it transmits it to the chart module in real time, which updates the view based on the timeline.

3.2 Visualization Implementation with JavaScript: From Parsing to Plotting

First, we introduce an online serial tool (https://serial.it-res.com) that supports online serial connection, data transmission/reception, and a powerful plugin system for chart plotting using JavaScript (with the built-in ECharts library for enhanced functionality).

Step 1: Serial Connection and Data Reception (Based on Web Serial API)

let port;
let reader;
let chart;  // Chart instance

// Connect to serial port
async function connectSerial() {port = await navigator.serial.requestPort();
    await port.open({baudRate: 9600});  // Match device baud rate
    
    // Receive serial data (read byte by byte)
    reader = port.readable.getReader();
    const decoder = new TextDecoder('utf-8');  // Use Uint8Array for binary data
    while (true) {const { value, done} = await reader.read();
        if (done) break;
        const rawData = decoder.decode(value);  // Raw data (e.g., "AA 04 00 1E 00 3C 58")
        parseSerialData(rawData);  // Parse data
    }
}

Step 2: Protocol Parsing (Corresponding to the Custom Protocol in Section 2.2)

let parseState = "WAIT_SOF";
let dataBuf = [];
let dataLen = 0;
let dataIdx = 0;
let crcCalc = 0;

function parseSerialData(rawData) {// Convert raw string (e.g., "AA 04 00 1E 00 3C 58") to byte array
    const bytes = rawData.split(" ")
                         .map(hex => parseInt(hex, 16))
                         .filter(byte => !isNaN(byte));
    
    bytes.forEach(byte => {switch(parseState) {
            case "WAIT_SOF":
                if (byte === 0xAA) {
                    parseState = "GET_LEN";
                    crcCalc = byte;
                    dataBuf = [];
                }
                break;
            case "GET_LEN":
                dataLen = byte;
                crcCalc ^= byte;
                if (dataLen > 0 && dataLen <= 60) {
                    parseState = "GET_DATA";
                    dataIdx = 0;
                } else {parseState = "WAIT_SOF";}
                break;
            case "GET_DATA":
                dataBuf.push(byte);
                crcCalc ^= byte;
                if (dataIdx++ === dataLen - 1) {parseState = "GET_CRC";}
                break;
            case "GET_CRC":
                if (byte === crcCalc) {// Extract temperature and humidity (assume data segment: [temp_high, temp_low, humi_high, humi_low])
                    const temp = (dataBuf[0] << 8 | dataBuf[1]) / 10.0;
                    const humi = (dataBuf[2] << 8 | dataBuf[3]) / 10.0;
                    updateChart(temp, humi);  // Update chart
                }
                parseState = "WAIT_SOF";
                break;
        }
    });
}

Step 3: Real-Time Chart Rendering with ECharts

// Initialize chart
function initChart() {const chartDom = document.getElementById('serial-chart');
    chart = echarts.init(chartDom);
    
    const option = {title: { text: 'Real-Time Temperature & Humidity Monitoring'},
        tooltip: {trigger: 'axis'},
        legend: {data: ['Temperature (°C)', 'Humidity (%)'] },
        xAxis: {
            type: 'time',
            splitLine: {show: false},
            axisLabel: {formatter: '{hh}:{mm}:{ss}' }  // Time format
        },
        yAxis: [
            {name: 'Temperature (°C)', type: 'value', min: 0, max: 50 },
            {name: 'Humidity (%)', type: 'value', min: 0, max: 100, position: 'right' }
        ],
        series: [
            {name: 'Temperature (°C)',
                type: 'line',
                data: [],
                smooth: true,  // Smooth curve
                yAxisIndex: 0
            },
            {name: 'Humidity (%)',
                type: 'line',
                data: [],
                smooth: true,
                yAxisIndex: 1,
                lineStyle: {color: '#ff4500'}
            }
        ]
    };
    chart.setOption(option);
}

// Update chart data
function updateChart(temp, humi) {const now = new Date();
    const timeStr = now.toISOString();  // Timestamp
    
    const option = chart.getOption();
    // Limit number of data points (retain only last 10 minutes of data)
    if (option.series[0].data.length > 600) {option.series[0].data.shift();
        option.series[1].data.shift();}
    // Add new data
    option.series[0].data.push([timeStr, temp]);
    option.series[1].data.push([timeStr, humi]);
    chart.setOption(option);
}

Practical Application: The above logic can be integrated into online serial tools (e.g., browsers supporting the Web Serial API). No local development environment is required—simply open the browser to complete the entire workflow of “serial connection → protocol parsing → real-time plotting,” eliminating the need to rewrite basic code repeatedly.

IV. Common Serial Debugging Issues and Solutions

Even with a solid grasp of technical principles, you may encounter various issues during debugging. Below is a troubleshooting guide for high-frequency problems:

4.1 Garbled Received Data: Parameter Mismatch or Voltage Issues

Troubleshooting Step 1: Confirm that the baud rate, data bits, stop bits, and parity bit exactly match the device (mismatched baud rates are the most common cause).
Troubleshooting Step 2: Check if TX/RX pins are reversed (connect device TX to USB module RX, and device RX to USB module TX).
Troubleshooting Step 3: For 3.3V devices, verify that the USB module’s output voltage is compatible (avoid damaging 3.3V devices with 5V levels).

4.2 Data Loss or Sticking: Protocol Design or Hardware Issues

• Software Layer: Optimize protocol parsing logic (e.g., use a state machine) and add a timeout reset mechanism.
• Hardware Layer: Check for loose wiring (shielded cables are recommended to reduce interference) and lower the baud rate (high baud rates are prone to interference-induced data loss).
• Protocol Layer: If data is sent at a high frequency, add a frame trailer identifier or adopt “fixed frame intervals” (e.g., 10ms between frames) for transmission.

V. Advanced Application Scenarios of Serial Communication: Beyond “Debugging”

Serial communication is not limited to “viewing device logs”—in IoT and industrial control, it also plays a core role in data collection and device control. Below are typical advanced scenarios:

5.1 Multi-Device Serial Networking: Application of RS485 Bus

When multiple serial devices (e.g., multiple sensors, multiple controllers) need to be connected, simplex UART (TX/RX) cannot meet the demand. Instead, the RS485 bus is commonly used for multi-device communication:

• Hardware Principle: RS485 uses differential signal transmission (two signal lines: A and B), offering far stronger anti-interference capabilities than UART. It supports transmission distances up to 1200 meters and can connect up to 32 devices on the same bus.
• Communication Mode: Adopts a “master-slave mode,” where the master distinguishes between different slaves via device addresses (e.g., the master sends “0x01 + command,” and only the slave with address 0x01 responds).
• Protocol Adaptation: Most RS485 devices still communicate based on serial protocols (e.g., Modbus-RTU protocol). Only an “equipment address” field needs to be added to the serial parameters to enable multi-device interaction.

Tool Assistance: When debugging RS485 networked devices, use the “multi-device address configuration” plugin of online serial assistants to quickly switch target device addresses, send commands, and receive responses—no need to manually modify address parameters in the code.

5.2 Serial-to-Network Bridging: Enabling Remote Device Monitoring

In IoT scenarios, remote monitoring of serial devices (e.g., sensors in factory workshops, monitoring devices in remote areas) is often required. This is typically achieved using “serial-to-network” modules (e.g., ESP8266, ESP32):

• Implementation Logic: The module connects to the serial device via UART, converts serial data into TCP/UDP network data, and uploads it to the cloud via WiFi or Ethernet.
• Data Interaction: A remote terminal (e.g., computer, mobile APP) sends commands to the module over the network. The module converts the commands into serial data and sends them to the device, while transmitting the device’s response data back to the remote terminal.
• Protocol Selection: For low power consumption, the MQTT protocol is suitable for data transmission (adapted to IoT scenarios); for real-time performance, direct TCP connection is preferred.

Debugging Tip: Use the “network serial bridging” function of online serial assistants to connect directly to a remote TCP server via a browser, receive network data from serial devices, and visualize it in real time—no need to deploy complex network debugging tools locally.

5.3 Serial Data Storage and Playback: Retracing Debugging Processes

In scenarios such as device stability testing or fault diagnosis, serial data needs to be recorded and played back for post-analysis. The traditional method—saving data as a TXT file via serial tools and analyzing it manually—is inefficient:

• Efficient Solution: Online serial assistants support saving serial data as JSON files in the format of “timestamp + data content,” including raw data, parsed data, and chart data.
• Data Playback: During playback, the tool plays back data frame by frame at the original time interval, synchronously updating charts and parsing results to simulate real communication processes and quickly locate data anomalies when faults occur.
• Data Filtering: Supports filtering data by time range and data type (e.g., commands, responses, abnormal data) to focus on key debugging nodes.

VI. Optimization of Serial Protocols: Improving Communication Efficiency and Reliability

As device complexity increases, basic “header + length + data + checksum” protocols may no longer meet requirements. Protocol design needs to be optimized in the following dimensions:

6.1 Data Compression: Reducing Bandwidth Usage

When serial devices transmit large volumes of data (e.g., continuous waveform data collected by sensors), uncompressed data occupies significant bandwidth and causes transmission delays:

• Compression Algorithm Selection: For data with large repetitions (e.g., consecutive 0x00), Run-Length Encoding (RLE) is suitable; for numerical data (e.g., temperature, voltage), differential encoding (storing the difference between adjacent data points) is preferred.
• Protocol Adaptation: Add a “compression flag” field (1 bit) to the protocol to indicate whether data is compressed. The receiver decides whether to decompress based on the flag.
• Example: For continuous temperature data “25.1, 25.2, 25.1, 25.3,” differential encoding converts it to “25.1, +0.1, -0.1, +0.2,” reducing data length.

Tool Support: The “data compression/decompression” plugin of online serial assistants can automatically identify compression flags, decompress data in real time, and visualize it—no manual compression handling is required.

6.2 Batch Transmission of Multiple Commands: Improving Control Efficiency

When controlling multiple devices or performing complex operations, multiple serial commands need to be sent. Manual one-by-one transmission is time-consuming and error-prone:

• Batch Transmission Solution: Design a “batch command packet” protocol, where multiple commands are concatenated in the format of “command length + command content.” A “batch flag” is added to the header, and the receiver parses and executes commands one by one.
• Command Priority: Add a “priority” field (e.g., levels 0–3) to batch commands. The receiver executes commands by priority to ensure critical commands (e.g., emergency stop commands) are executed first.
• Execution Feedback: After executing each command, the receiver returns “execution results” (success/failure + error code). The master determines whether to proceed to the next command based on the feedback.

Operational Convenience: Online serial assistants support importing batch command files (TXT/JSON format), setting command transmission intervals (e.g., 100ms per command), automatically sending commands, recording feedback results for each command, and generating execution reports.

6.3 Fault Tolerance Mechanisms: Addressing Communication Anomalies

Serial communication may experience anomalies due to interference or disconnection. Fault tolerance mechanisms need to be added to the protocol:

• Retransmission Mechanism: If the master does not receive a response within the timeout period (e.g., 100ms) after sending a command, it automatically retransmits the command (retransmission times can be set, e.g., 3 times) to avoid communication failure caused by single interference.
• Data Backup: Before executing a command, the receiver backs up critical data (e.g., device parameters). If command execution fails (e.g., checksum error), it restores the backup data to prevent abnormal device status.
• Heartbeat Packets: The master and device send heartbeat packets (e.g., “0xAA 0x01 0x00 0xAA”) at regular intervals (e.g., 1 second). If the master does not receive a heartbeat packet for 3 consecutive times, it determines the device is offline and triggers an alarm mechanism.

Debugging Assistance: The “communication fault tolerance test” function of online serial assistants can simulate abnormal scenarios such as data loss, interference, and disconnection, test the device’s fault tolerance, and record communication data when anomalies occur to help optimize the protocol’s fault tolerance logic.

VII. Conclusion: Technological Evolution of Serial Communication and Tool Empowerment

Serial communication has evolved from the original RS232 protocol (suitable for short-distance, single-device communication) to the RS485 protocol (suitable for long-distance, multi-device communication), and further to integration with networks and the cloud. It remains a core communication method in embedded systems and IoT. Its technical core lies in “reasonable protocol design” and “efficient data processing,” while tools add value by lowering technical barriers and improving debugging efficiency.

As a web-based tool, online serial assistants not only solve the pain points of traditional tools (e.g., “installation dependencies,” “limited functionality”) but also adapt to full-scenario needs from basic debugging to advanced applications through plugin systems, data visualization, and network bridging:

• For Novice Engineers: No in-depth mastery of low-level code is required—serial connection, data parsing, and chart analysis can be completed via a visual interface.
• For Senior Engineers: Custom plugins and protocol optimization are supported, enabling rapid verification of complex communication logic and fault tolerance mechanisms.
• For Team Collaboration: Data files, plugins, and debugging reports can be shared online, reducing debugging costs across devices and environments.

In the future, with the development of IoT technology, serial communication will be further integrated with edge computing and AI analysis (e.g., real-time analysis of abnormal patterns in serial data via AI algorithms). Online serial assistants will also continue to iterate, adapting to more complex scenarios and providing more efficient technical support for engineers.

If you encounter specific problems in serial communication practice or need to optimize protocol design, feel free to share your scenarios in the comment section—we can discuss solutions together. You can also use the “technical community” function of online serial assistants to exchange experiences with other engineers and jointly promote the application and innovation of serial communication technology.