Harnessing the power of agent-based modeling for equity market simulation and strategy testing

This post was contributed by Ilan Gleiser from AWS, Dr. Namid Stillman, Dr. Perukrishnen Vytelingum , Dr Rory Baggott from Simudyne, Ross Pivovar from AWS, and Richard Harmon from Red Hat

In the fast-paced world of finance, understanding and forecasting market behavior is crucial for making informed decisions. Conventional backtesting of trading strategies often falls short, failing to capture the complexity and unpredictability inherent in real-world markets.

This is where Simudyne’s agent-based models come into play. These models simulate the interactions and decision-making of individual agents within a market, offering invaluable insights for asset managers, equity trading desks, exchanges, regulators, and risk officers.

In this post, we’ll delve into how Simudyne’s agent-based models, powered by AWS and Red Hat OpenShift, to equip financial institutions and stock exchanges like the Hong Kong Exchanges and Clearing Limited (HKEX) to effectively analyze and navigate both market and liquidity risks.

Understanding agent-based modeling

At its core, agent-based modeling (ABM) involves creating a virtual environment populated by software agents representing the different participants in a system. These agents are programmed with specific behaviors, objectives, and decision rules, allowing them to interact with each other and their environment in parallel, just as participants would in the real world.

Fig 1. Simulated orderbook, where virtual traders meet to buy and sell stocks.

Key benefits of ABM for financial markets

When applied to financial markets and trading strategies, ABMs offer several compelling advantages over traditional tools and approaches:

1. Unparalleled Experimental Control: Unlike historical data, ABM provides researchers with full control over the environment, decision rules, information flows, and other parameters. This allows for rigorous testing of variables and the isolation of causative factors.
2. Transparency of Market Participants: In an ABM, the motivations and decision processes of virtual agents representing traders are transparent and controllable, providing valuable insights into the impact of different trading algorithms and strategies on market dynamics.
3. Flexible Customization: ABMs can be customized to represent a wide range of scenarios and domains, incorporating real-world details such as latencies, communication protocols, multi-asset landscapes, and historical data streams.
4. Exploration of Adjacent Areas: Beyond market simulation, ABM is valuable for testing new exchange mechanisms, policies, and regulations, developing technologies like federated learning, studying adoption dynamics, and optimizing automated trading strategies.
5. Efficient Parallelization: Simudyne’s software is designed with parallel execution in mind, allowing researchers to leverage advances in distributed computing and scale to large-scale scenarios.

Simudyne’s market simulators: capturing market dynamics with precision

Simudyne’s market simulators deliver a robust and highly realistic model of market behavior that’s both grounded in exchange data and designed for straightforward operation. Unlike traditional methods, Simudyne harnesses an agent-based approach for simulating and backtesting execution strategies, offering a suite of advantages.

Agent-based models stand apart in capturing the intricacies of market behavior, offering a nuanced and precise understanding of the effectiveness of execution strategies. This approach fosters a highly realistic, dynamic, and responsive simulation environment that mirrors the complexities of real-world markets with remarkable accuracy, including aspects like price dynamics, liquidity fluctuations, and participant behaviors.

This high degree of realism empowers traders and firms to rigorously test their execution strategies across a spectrum of market conditions, enabling a more accurate evaluation of their impact, performance, and resilience under stress.

Fig 2: Transient and permanent impact emerge from executing a metaorder – see (a) and (b). In (c), we observe the relevance of illustrative execution strategies on price impact. ‘VWAP000A’ is a front-loaded strategy, ‘VWAP000B’ a balanced strategy and ‘VWAP000C’ a back-loaded strategy.

The Simudyne simulator also incorporates advances in simulation techniques which allow for a more granular representation of market participants and their decision-making processes. Complex interactions cause realistic market dynamics to emerge, including realistic order-book dynamics, market impact, and price formation.

“Like an ‘algo-gym’, agent-based simulations can help ensure execution algorithms are ready to be deployed in as wide a range of potential futures as possible.” — Justin Lyon, Simudyne CEO

Calibrating market simulators for accuracy

To use market simulators effectively, it’s important to calibrate them to accurately reflect real market data. This process is known as closing the simulation-to-reality gap and it’s essential for generating useful data for training AI systems and testing trading strategies.

With new advancements in calibration methods, such as surrogate modeling and simulation-based inference, market simulators are becoming even more powerful tools for understanding market dynamics. Additionally, calibration provides valuable meta-information about the market, such as key drivers of market behavior and relationships between different variables.

Leveraging AWS HPC for efficient and scalable simulations

Simudyne’s ABMs are built on AWS HPC computing infrastructure that allows for efficient and scalable simulations. This enables users to run complex agent-based simulations and explore large-scale market scenarios. By reducing the time to results by scaling up the resources required for simulations, Simudyne’s ABMs offer a cost-effective solution for banks, hedge funds, regulators, asset managers and broker dealers.

Fig 3: Example of running many simulations on the ROSA cluster across different indices: Nikkei225, Hang Seng Index, and S&P 500.

Case study: implementing an agent-based liquidity risk model in equity markets

“Should we produce a simulation on a daily basis for each of the member portfolios, the likelihood of their next day’s total margin, in dollar amount, including both variation margin and initial margin?” — Tao Chen, Head of Quantitative Risk, HKEX, at WFE Conference

In a project conducted by Simudyne with Hong Kong Exchanges and Clearing Limited (HKEX), an advanced agent-based model was deployed to provide daily updates on the likely margin impact as the result of stress events, greatly improving transparency and predictability in liquidity management.

The model created a simulated Heng Seng Index Futures market which replicates the continuous double auction system prevalent in global financial exchanges – facilitating nuanced interactions among diverse trading agents employing strategies reflective of realistic market behaviors.

Key components of the ABM include:

• Trading Agents programmed with distinct strategies mirroring real market participants.
• Market Environment emulating a realistic setting with a continuous double auction mechanism.
• Stochastic Elements integrating Monte-Carlo simulations to estimate transaction costs and evaluate liquidity risks.

The simulations offer vital insights into market behavior under stress, like rapid large-scale liquidations or unexpected market shifts. Key findings highlight the non-linear relationship between order size, execution strategy, and market impact, as well as the ability to test and refine trading strategies within a realistic yet controlled environment.

The derived insights prove invaluable for liquidity risk management, providing a risk management framework and equipping traders with tools to optimize strategies and mitigate risks associated with substantial trades.

Figure 4: HKEX is using the simulator to design optimal close-out strategies. Illustrative example.

Technology: implementing Simudyne’s equity market simulator

The deployment of Simudyne’s Equity Market Simulator at HKEX uses Red Hat OpenShift Service on AWS, reflecting the simulator’s advanced operational needs and the requirement for a scalable, robust infrastructure. Red Hat enhances the scalable infrastructure from AWS with a system for streamlining operations while accelerating the delivery of applications and artificial intelligence.

Figure 5: Simudyne has partnered with Red Hat and AWS to enable the Market Simulator as a Red Hat OpenShift Service on AWS.

Advantages of using Red Hat OpenShift service on AWS include:

1. Scalability: Facilitates resource scaling based on simulation demands.
2. Reliability: Ensures high availability and effective disaster recovery.
3. Security: Incorporates essential security measures, crucial for sensitive financial data.
4. Operational Efficiency: Automates management tasks, reducing operational overhead.
5. Developer Productivity: Streamlines application deployment and management.

Deploying the simulator on this platform allows HKEX to harness necessary computational resources efficiently, enhancing the simulator’s effectiveness in liquidity risk assessments and market simulations without extensive in-house infrastructure.

Conclusion

Agent-based modeling is essential for understanding and predicting complex systems, particularly in the realm of financial markets. As organizations explore how to use ABMs to improve strategic decision-making and drive innovation, those who gain expertise in developing and deploying these simulations will be well-positioned to uncover valuable insights and gain a competitive edge.

HKEX’s deployment of Simudyne’s simulator on Red Hat OpenShift Service demonstrates how to leverage novel technology to deliver innovative solutions to real-world problems, ensuring HKEX remains at the forefront of market simulation technology. This deployment highlights the increasing relevance of cloud solutions in managing complex data and simulation needs in finance.

With its ability to illuminate the emergent behavior of interconnected systems, Simudyne’s agent-based simulators are set to become an indispensable tool in the arsenal of forward-thinking researchers and decision-makers. As computational power advances and our ability to incorporate real-world data improves, the fidelity and utility of agent-based simulations will increase rapidly, revolutionizing the way we study and navigate the complex world of financial markets.