WebSocket Dynamic Subscription: Build Stable US Stock Quote & Order Book Data Pipeline
When developing quantitative trading strategies, backtesting systems and automated trading tools, the real-time performance, continuity and integrity of market data are the cornerstone that determines model outputs and trading execution quality.
Traditional data interfaces bring a host of common troubles. REST polling suffers from high latency and is easily blocked by access limits under high-frequency requests. Standard WebSocket connections have to disconnect and reconnect every time you add or remove trading symbols, which frequently causes reconnection storms and duplicate data calculations. Even minor network fluctuations can lead to silent socket disconnections and interrupted data streams, severely disrupting high-frequency strategies and order book factor models.
To resolve these issues, I adopted a WebSocket dynamic subscription architecture. By reusing a single persistent connection, we achieve full integration of real-time US stock quotes and multi-level order book depth data. This article walks through core concepts, interface comparisons, parameter rules, operational pitfalls and practical solutions. I will also discuss its practical value for quantitative backtesting and trading systems.
1. Core Concept: What is Dynamic Subscription
Dynamic subscription refers to maintaining one permanent WebSocket connection, and adding or removing symbol codes via standard commands without any disconnection or reconnection.
We can distinguish it from traditional data fetching methods clearly:
REST Polling: Pull data through periodic requests. It is inherently poor for real-time scenarios.
Standard WebSocket: Requires reconnection whenever you modify the subscription list, making status consistency hard to guarantee.
Dynamic Subscription WebSocket: Updates subscriptions on the existing connection. It ensures continuous data flow and works perfectly for multi-symbol monitoring and dynamic position adjustment in quantitative trading.
2. Comparison of Mainstream Market Interfaces
I evaluated three common interface solutions for US stock tick data and multi-level order book data, focusing on latency, connection stability, data sequence and maintenance cost.
REST is easy to implement, yet its data refresh rate is fully restricted by polling intervals. Frequent requests will trigger rate limits and cause data lag and distortion. Delayed and inaccurate data will directly corrupt backtesting results, so it is not a viable choice for quantitative development.
As a long-push solution, WebSocket delivers lower latency than REST. However, it has a fatal flaw: you must terminate and rebuild the connection to change subscribed symbols. Frequent reconnections result in connection storms, mismatched subscription status and distorted order book levels. These anomalies will contaminate factor calculations and create huge deviations between backtesting results and live market performance.
2.3 WebSocket with Dynamic Subscription
This is the production-ready solution we finally adopted. Compliant with standard WSS protocols, it divides endpoints by asset types and natively supports dynamic subscription, as well as order book snapshot + incremental updates. One single connection can manage all subscribed symbols efficiently, balancing transmission speed and operational stability.
WSS endpoint for US stock market data:
wss://shturl.cc/XPbXuHTj3bkDvPhwidN7h7YtMtYObV1XoD4T0vNssQcO
Dedicated endpoints are available for forex, crypto assets and other financial products. The layered structure simplifies development and long-term maintenance for multi-asset trading systems.
3. Parameter Rules for Common Usage Scenarios
All subscription operations use the fixed command ID cmd_id=22004. Below I break down typical scenarios, common errors, standard request formats and verification methods for daily development and backtesting integration.
Initialize Connection & Bulk Subscribe Symbols
It is quite common to encounter missing market data or malformed request errors during service startup. Always send subscription commands inside the on_open callback after the connection is established.
Standard request structure:
{"cmd_id": 22004, "action": "subscribe", "code": ["NASDAQ:AAPL","NASDAQ:TSLA"]}
You can verify the whole request packet via system logs to confirm symbol formats and ensure normal data streaming after initialization.
Add New Symbols While Running
Many developers rebuild connections to add new symbols, which creates redundant links and disturbs data sequence. Keep the original connection alive and send a separate subscribe request for new symbols.
Standard request structure:
{"cmd_id": 22004, "action": "subscribe", "code": ["NASDAQ:MSFT"]}
Check that the original data stream remains uninterrupted, new quotes arrive normally, and update your local symbol list synchronously.
Unsubscribe Symbols During Operation
"Ghost subscription" — receiving data from unsubscribed symbols — is a frequent issue that interferes with factor computation. Switch the action field to unsubscribe to stop data delivery for target symbols.
Standard request structure:
{"cmd_id": 22004, "action": "unsubscribe", "code": ["NASDAQ:TSLA"]}
Confirm data from selected symbols stops immediately, and remove related codes from your local subscription list.
Handle Duplicate Subscription Requests
Sending repeated requests will lead to redundant calculations on the client side. The server ignores duplicate subscribe commands silently without returning errors.
Standard request structure:
{"cmd_id": 22004, "action": "subscribe", "code": ["NASDAQ:AAPL"]}
It is recommended to add client-side deduplication logic to avoid invalid requests and reduce system load.
Block Empty Request Lists
Passing an empty array to the code field counts as an invalid parameter and will force the connection to drop. Never send requests with empty symbol lists.
{"cmd_id": 22004, "action": "subscribe", "code": []}
Add pre-validation on the client to block such requests and prevent unexpected service interruptions.
4. Core Advantages of Dynamic Subscription Architecture
After long-term online operation, I have summarized four major strengths of this solution for quantitative data collection.
4.1 Eliminate Risks Brought by Reconnections
We use cmd_id=22004 for all subscription operations. There is no need to create or destroy sockets when adjusting symbol lists. This fundamentally prevents reconnection storms and data gaps. Continuous tick data and order book records are the foundation of reliable backtesting and reproducible live trading results for time-series and high-frequency models.
4.2 Optimize Transmission & Calculation Efficiency
Market data is split into two independent modules: real-time quotes and order book depth. Real-time quotes deliver latest prices and trading volumes, while order book data adopts snapshot initialization plus incremental updates.
This design greatly reduces network traffic and prevents disordered or jumping order book levels. It guarantees the calculation accuracy of order density, buying and selling pressure and other critical market factors.
4.3 Adapt to Unstable Network Conditions
The WebSocket service is equipped with native heartbeat detection. Combined with custom heartbeat settings in business logic, it can quickly identify silent disconnections caused by network jitter. The interface supports Python, Java, Go and other mainstream languages, so it can be seamlessly integrated into various quantitative frameworks and data collection services.
4.4 High Reusability Across Asset Classes
Besides US stocks, this interface supports Hong Kong stocks, forex, precious metals and cryptocurrencies. One set of connection logic can be reused across multi-asset strategies and data tools, cutting down repeated development and maintenance work.
5. Common Operational Issues & Solutions
Based on real-world maintenance experience, here are four typical problems, diagnostic methods and optimization strategies to avoid abnormal data affecting backtesting and live trading.
5.1 Message Backlog from High-Frequency Tick Data
During active trading sessions, massive tick data streams in rapidly. The callback function may fail to process messages in time and cause program lag. You can identify this issue by checking timestamps in logs.
To fix it, separate raw data forwarding from complex business logic. Keep callback functions lightweight, and move factor calculation and advanced analysis to asynchronous queues.
5.2 Silent Disconnection Caused by Network Fluctuation
Temporary network outages may not trigger the on_close callback, while data delivery stops completely. Add data receiving timeout rules for anomaly detection.
On top of native heartbeat, implement application-layer timeout judgment. Actively disconnect and perform orderly reconnection once timeout is detected.
5.3 Subscription Status Mismatch
Rapid addition and removal of symbols often cause inconsistency between local records and server status, resulting in ghost subscription. Compare received data with your local symbol list for troubleshooting.
Add serial locks for all subscription operations to avoid concurrent execution. Always sync the local list after every subscribe or unsubscribe action.
5.4 Failed Subscription Due to Wrong Symbol Format
Using short codes like AAPL will not trigger errors, yet no market data will be received. The standard format is Exchange:Symbol.
Standardize all symbol codes in your program and add format validation rules to reject malformed requests in advance.
Before deployment, please clarify the boundaries of this architecture:
✅ Supported: Dynamically add and remove symbols within a single WebSocket connection.
❌ Not supported: Synchronize subscription status across multiple connections, retrieve historical tick data, or use private commands other than cmd_id=22004.
This dynamic Web subscribe solution has been running steadily for US stock data collection, order book factor mining, quantitative backtesting and live trading for a long time. It largely eliminates failures related to reconnection and data distortion.
For quantitative developers and traders, stable and traceable underlying data pipelines are essential to trustworthy backtesting and steady live performance. This lightweight, scalable architecture can be deployed as an independent data service or embedded into existing quantitative frameworks. The same logic can also be migrated to other global financial markets.
Feel free to share your ideas or ask questions about market data integration and WebSocket optimization.
