Apache Synapse Enterprise Service Bus (ESB) Performance Testing - Round 3
We have just concluded the third round of Enterprise Service Bus (ESB) performance testing here at WSO2! The WSO2 Enterprise Service Bus (ESB) v1.7 embeds the Apache Synapse Enterprise Service Bus (ESB) v.1.2 at its core, and thus the performance numbers between these are identical.
Its great to hear that even BEA / Oracle was very much interested about the performance numbers we demonstrated last year, and wanted to beat our numbers with their latest release of AquaLogic Service Bus 3.0 on Weblogic 10. I think this demonstrates a level of trust by the Enterprise Service Bus (ESB) vendors and users alike, on the tests we conducted. The fact that we made the configurations/tuning and the tools used openly available, and asked for help from the vendors to optimally configure their ESB's for the scenarios, and reporting back any problems we encountered adds to the level of openness we demonstrated. Additionally this round of testing shows that even the WSO2 Enterprise Service Bus (ESB) / Apache Synapse Enterprise Service Bus (ESB) has room for improvement against a proprietary ESB that we benchmarked against, but also shows a very clear lead against the other open source alternatives.
It should be noted that the advantages of the WSO2 Enterprise Service Bus (ESB) / Apache Synapse Enterprise Service Bus (ESB) does not lie only on its low resource usage foot print, excellent performance or scalability or its free and open source Apache License v2.0 alone; but in its ease of use, the ability to define a configuration graphically, and the fact that it ships over 55 working samples and documentation that demonstrates various features to help users become effective from day 1.
Let me just highlight some of the observations and conclusions here
- Mule CE 2.0.1 couldn't handle the cases where we used a concurrency level of 80; while other ESB's scaled to support to over 2500 concurrent connections. This was after tuning the maximum active thread count to 100 from its default value, which limited Mule to a very few concurrent connections.
- A proprietary version of an open source ESB had the same problem described above.
- Mule CE 2.0.1 also dropped 1% of ALL requests it received
- Apache ServiceMix 3.2.1 failed to forward the incoming SOAPAction for proxy services, and this was now a known issue (I would consider this a blocker, and would suggest that ServiceMix folks follow up a 3.2.2 release just to fix this critical issue)
- A proprietary ESB we benchmarked and beat last year, did some major improvements to their performance, and did 1.6~1.9 times better than us for some of the scenarios
- The WSO2 Enterprise Service Bus (ESB) / Apache Synapse Enterprise Service Bus (ESB) shows a clear lead and dominates the open source ESB space
Read about it all here: http://wso2.org/library/3740 and run the benchmarks yourself!
Its great to hear that even BEA / Oracle was very much interested about the performance numbers we demonstrated last year, and wanted to beat our numbers with their latest release of AquaLogic Service Bus 3.0 on Weblogic 10. I think this demonstrates a level of trust by the Enterprise Service Bus (ESB) vendors and users alike, on the tests we conducted. The fact that we made the configurations/tuning and the tools used openly available, and asked for help from the vendors to optimally configure their ESB's for the scenarios, and reporting back any problems we encountered adds to the level of openness we demonstrated. Additionally this round of testing shows that even the WSO2 Enterprise Service Bus (ESB) / Apache Synapse Enterprise Service Bus (ESB) has room for improvement against a proprietary ESB that we benchmarked against, but also shows a very clear lead against the other open source alternatives.
It should be noted that the advantages of the WSO2 Enterprise Service Bus (ESB) / Apache Synapse Enterprise Service Bus (ESB) does not lie only on its low resource usage foot print, excellent performance or scalability or its free and open source Apache License v2.0 alone; but in its ease of use, the ability to define a configuration graphically, and the fact that it ships over 55 working samples and documentation that demonstrates various features to help users become effective from day 1.
Let me just highlight some of the observations and conclusions here
- Mule CE 2.0.1 couldn't handle the cases where we used a concurrency level of 80; while other ESB's scaled to support to over 2500 concurrent connections. This was after tuning the maximum active thread count to 100 from its default value, which limited Mule to a very few concurrent connections.
- A proprietary version of an open source ESB had the same problem described above.
- Mule CE 2.0.1 also dropped 1% of ALL requests it received
- Apache ServiceMix 3.2.1 failed to forward the incoming SOAPAction for proxy services, and this was now a known issue (I would consider this a blocker, and would suggest that ServiceMix folks follow up a 3.2.2 release just to fix this critical issue)
- A proprietary ESB we benchmarked and beat last year, did some major improvements to their performance, and did 1.6~1.9 times better than us for some of the scenarios
- The WSO2 Enterprise Service Bus (ESB) / Apache Synapse Enterprise Service Bus (ESB) shows a clear lead and dominates the open source ESB space
Read about it all here: http://wso2.org/library/3740 and run the benchmarks yourself!
Labels: Apache Synapse Enterprise Service Bus (ESB), AquaLogic, benchmark results, Enterprise Service Bus, ESB, Mule, performance, ServiceMix
posted by Asankha at 10:12 PM
2 comments
