Benchmark
Looking at the forums concerning the UML tools, I often see people complaining about the problems of performances, so I compared BOUML with some other tools at the beginning of its development :
The easiest way for me to have a project containing a large number of classes having the same definition with several tools was to reverse a part of the Java JDK 1.4.2 sources. Obviously during the reverses made by BOUML I did not specify a catalog. All the tests were made with firewall and virus scanning off.
The first observation is that only BOUML and Enterprise Architect allows to reverse all sources of Java (packages com, java, javax, launcher, org and sunw), for the others there was not enough memory, or in the case of Visual Paradigm and UModel it is not possible to save the project and starUML can't reload the project !
On my PC under Windows XP, Pentium 4, CPU clock 3Ghz and 512 Mb of memory, Bouml 2.25 reverses all the sources of the Java version 6 (update 1) in 6 minutes 40 seconds, the process size when all is reversed is 172 Mo. This creates 9425 classes, 48506 properties (attributes and relations) and 85939 operations.
BOUML 2.2 versus Enterprise Architect 5.0
Test made with my PC under Windows XP, Pentium 4, CPU clock 3Ghz, 512 Mb of memory.
Enterprise Architect saves all during the reverse (this is very strange for me, I consider that all saving must be explicit) so I put the reverse and saving phases together also for BOUML. Obviously I don't ask Enterprise Architect to automatically make diagrams not to slow down it.
The benchmark sequence is :
- time needed to reverse the package java and all its sub packages and save the project
- exit, restart the tool
- time needed to load the previous project
- look at the size of the process
- the needed time to generate all the sources
| BOUML | Enterprise Arch. | Enterprise Arch. / BOUML | |
| Reverse+Save | 51 sec | 11 min 25 | 13 |
| Load | 3 sec | 4 sec | 1 |
| Process size | 38 Mb | 31 Mb | 1 |
| Generation | 16 sec | 4 min 31 | 17 |
BOUML 2.2 versus Poseidon Professional Edition 3.1
Test made with my PC under Windows XP, Pentium 4, CPU clock 3Ghz, 512 Mb of memory.
Poseidon
is very memory (and time) consuming, it is just possible to reverse the
package java.awt and its sub packages. I ask Poseidon to
not automatically make the diagrams. The benchmark sequence is :
- time needed to reverse the package java.awt and all its sub packages
- time needed to save the project
- exit, restart the tool (the starting of Poseidon takes much time, this is not included in the time below)
- time needed to load the previous project
- look at the size of the process
| BOUML | Poseidon | Poseidon / BOUML | |
| Reverse | 17 sec | 2 min 30 | 9 |
| Save | 2 sec | 51 sec | 25 |
| Load | 0.7 sec | 49 sec | 70 |
| Process size | 17 Mb | 314 Mb | 18 |
Note : It is not possible to generate the code, Poseidon propose to generate the source code for only a few number of classes.
BOUML 2.2 versus Rhapsody 6.0
Test made with my PC under Windows XP, Pentium 4, CPU clock 3Ghz, 512 Mb of memory.
Rhapsody
allows to reverse the package java and its sub
packages. The benchmark sequence is :
- time needed to reverse the package java and all its sub packages
- time needed to save the project
- exit, restart the tool
- time needed to load the previous project
- look at the size of the process
- the needed time to generate all the sources
| BOUML | Rhapsody | Rhapsody / BOUML | |
| Reverse | 47 sec | 6 min 17 | 8 |
| Save | 4 sec | 28 sec | 7 |
| Load | 3 sec | 23 sec | 8 |
| Process size | 38 Mb | 165 Mb | 4 |
| Generation | 16 sec | 3 min 18 | 12 |
BOUML 2.2 versus Rose Enterprise 2002
Test made under Windows XP prof 2002, Pentium 4, CPU clock 2.8Ghz, 1 Gb of memory.
Rose
allows to reverse the package java and its sub
packages. The benchmark sequence is :
- time needed to reverse the package java and all its sub packages
- time needed to save the project
- exit, restart the tool
- time needed to load the previous project
- look at the size of the process
| BOUML | Rose | Rose / BOUML | |
| Reverse | 39 sec | 12 min 55 | 20 |
| Save | 4 sec | 11 sec | 3 |
| Load | 2 sec | 17 sec | 7 |
| Process size | 38 Mb | 157 Mb | 4 |
Note : I don't measure the needed time to produce the sources because under Rose it is necessary to ask for the generation component by component !
BOUML 2.2 versus Visual Paradigm for UML 4.1
Test made with my PC under Windows XP, Pentium 4, CPU clock 3Ghz, 512 Mb of memory.
With 
Visual Paradigm
it is possible to reverse the package java and its sub packages,
but you can't save the project : on my PC I prefer to kill the process
after 11 minutes. On a PC having 1 Gb of memory the saving need 44
seconds but in fact nothing is saved : when you try to exit VP
ask you to save the project, if you say yes : 44 seconds ... and nothing
is saved, if you say no and reload the project this one is empty !
It is possible to reverse the package java.awt and its sub packages (and to save the project). The benchmark sequence is :
- time needed to reverse the package java.awt and all its sub packages
- time needed to save the project
- exit, restart the tool
- time needed to load the previous project
- look at the size of the process
- the needed time to generate all the sources
| BOUML | Visual P | Visual P / BOUML | |
| Reverse | 17 sec | 25 sec | 1.5 |
| Save | 2 sec | 49 sec | 25 |
| Load | 0.7 sec | 21 sec | 30 |
| Process size | 17 Mb | 279 Mb | 16 |
| Generation | 6 sec | 12 sec | 2 |
Note : as you can see, Visual Paradigm takes twice more time to save the project than to make the reverse !
BOUML 2.15.4 versus starUML 5.0
Test made with my PC under Windows XP, Pentium 4, CPU clock 3Ghz, 512 Mb of memory.
With 
starUML
it is possible to reverse all the sources of java and its sub packages
(in almost 40 min !), to save the result but you can't read the project :
on my PC I prefer to kill the process after 10 minutes.
I don't ask starUML to automatically make diagrams not to slow down it.
The benchmark sequence is :
- time needed to reverse the package java and all its sub packages
- time needed to save the project
- exit, restart the tool
- time needed to load the previous project
- look at the size of the process
- the needed time to generate all the sources
| BOUML | starUML | starUML / BOUML | |
| Reverse | 57 sec | 5 min 5 | 5 |
| Save | 4 sec | 8 sec | 2 |
| Load | 3 sec | 40 sec | 13 |
| Process size | 38 Mb | 165 Mb | 4 |
| Generation | 16 sec | 11 | 0.7 |
BOUML 2.15.4 versus UModel 2006
Test made with my PC under Windows XP, Pentium 4, CPU clock 3Ghz, 512 Mb of memory.
With 
UModel
it is possible to reverse all the sources of java and its sub packages
(in 5 minutes rather than 3 minutes 35 for Bouml), but you can't save the project :
on my PC I prefer to kill the process after 10 minutes because the process size was
near 1 Go and nothing append except disk swap.
I don't ask UModel to automatically make diagrams not to slow down it.
The benchmark sequence is :
- time needed to reverse the package java and all its sub packages
- time needed to save the project
- exit, restart the tool
- time needed to load the previous project
- look at the size of the process
- rename the directory containing the reversed sources
- the needed time to generate all the sources, note : the bodies of the operations are not generated because their are not part of the model.
| BOUML | UModel | UModel / BOUML | |
| Reverse | 57 sec | 50 sec | 0.9 |
| Save | 4 sec | 34 sec | 9 |
| Load | 3 sec | 14 sec | 5 |
| Process size | 38 Mb | 115 Mb | 3 |
| Generation | 16 sec | 21 sec | 1.3 |
BOUML 2.2 versus Jude Professional version 5.1
Test made with my PC under Windows XP, Pentium 4, CPU clock 3Ghz, 512 Mb of memory.
Jude
allows to reverse the package java and its sub
packages. The benchmark sequence is :
- time needed to reverse the package java and all its sub packages
- time needed to save the project
- exit, restart the tool
- time needed to load the previous project
- look at the size of the process
- the needed time to generate all the sources
| BOUML | Jude | Jude / BOUML | |
| Reverse | 47 sec | 8 min 20 | 11 |
| Save | 4 sec | 25 sec | 6 |
| Load | 3 sec | 33 sec | 11 |
| Process size | 38 Mb | 228 Mb | 6 |
| Generation | 16 sec | 6 min | 23 |
BOUML 4.12 versus Omondo studio edition 3.4.1
Test made with my PC under Windows XP, Pentium 4, CPU clock 3Ghz, 2.5 Gb of memory.
Omondo
is one of the tools using Eclipse, because it is impossible to work with Eclipse
with only 512 Mo of memory I largely extended it adding 2 Go.
I applied the reverse on sources of Xerces 2.8.1 because Omondo fail to reverse Java package : after 3 minutes the tool was dead and frozen without consuming CPU, the only way was to kill it. The reverse is asked through Reverse UML > Deep Reverse > XMI Backup
I adapted the test to the specificities of the tool, the sequence is :
- time needed to reverse the sources of Xerces 2.8.1, this time doesn't take into account the time to import the Java project nor to build it even this is mandatory because Omondo works on object files rather than source files
- size of the process after the reverse (1)
- save, exit, restart the tool reading the project
- size of the process (2)
- needed time to create a class diagram containing only the 10 classes of the package xerces.impl.dtd.models asking for all the relations. In the case of Bouml it ie needed to first create the diagram then to ask for insert the classes of the corresponding class view, all these execution times are too short to be measured.
- size of the process (3)
- exit
- time needed to restart the tool reading the project with the 10 classes diagram open
- close the diagram
- needed time to open again the 10 classes diagram. In the case of Bouml the time is too short to be measured.
- needed time to do basic operations like to move a class having relations or to change diagram settings to hide/show class members. In the case of Bouml all is done immediately.
| BOUML | Omondo | Omondo / BOUML | |
| Reverse | 33 sec | 13 min 48 | 25 |
| Process size (1) | 24 Mo | 420 Mo | 17 |
| Process size (2) | 24 Mo | 162 Mo | 7 |
| Class Diagram | < 0.1 sec | 1 min 50 | > 1000 |
| Process size (3) | 25 Mo | 330 Mo | 13 |
| Load | 1.5 sec | 1 min | 40 |
| Open diagram | < 0.1 sec | 10 sec | > 100 |
| Basic operations | ~ 0 sec | several sec | > 1000 |
