IBExpert Benchmark

(redirected from Doc.IBExpertBenchmark)

Tutorial

The current version was released on 23rd January 2022, and includes some bugfixes, preventing the benchmark to crash on some servers.

We also have a new result, but it is just to measure the network speed between the client and server (we know that it does not show useful values when using the free benchmark as this version always using localhost as the server network address, but especially for the full version incorporated in the IBExpert IDE, it is very helpful to detect slow network configs to understand why Firebird-based applications run slower than expected. Outdated Router technology, network bridge components etc. are often responsible for poor software speed.

The Net Test executes 2000 inserts to a table with 100 columns and sends all params from client to the server in individual params, so this function will create a few hundred thousand very small TCP IP packages over the network and the total time is measured by how many operations were done within each second.

A good value on fast networks should have around >=400 IOPs. If it is much slower, you should ask your network support to check the speed of components used.

It is not part of the calculated percentage result values, and also the time it takes is not used there, since these result values use minimal network traffic to measure the server speed and not the network speed.

How do you measure the performance of your own server or your clients'? Have you ever tried to compare the database speed of your Firebird server with new server hardware? Is a XEON Machine better than an Opteron? What improvements can you expect from fast SAS hard disks, RAID controllers or Enterprise SSDs?

This simple but effective tool gives you a reference benchmark, based on our recommended IFS Servers. Using IBExpert Benchmark you can generate and compare a reproducible speed index for your Firebird Server hardware. On average, our IFS servers with Firebird are 5 to 10 times faster than the server or virtual machines that our customers have previously used.

Benchmark (extended) results using Firebird 4.0 on an IFSML Server

To perform the IBExpert Benchmark at your customers and also take advantage of the extended and XXL benchmarks, you can use either the IBExpert Developer Studio customer version or the IBExpert Day Edition. To perform standard benchmarks locally, you may use our Free IBExpert Benchmark Tool.

back to top of page

You can use the IBExpert Benchmark Tool free of charge on any Windows computer. This limited special version allows standard benchmarks to be performed locally and includes Firebird 2.5, 3.0 and 4.0 versions, executed in application mode, so you do not need to install Firebird on the server to be tested.

The Benchmark Tool is quick and easy to use: download it here: https://ibexpert.net/benchmark/setup_benchmark.exe, install it on the drive that you want to test.

Since September 2021 the benchmark now uses the latest Firebird versions 2.5.9 and 3.0.7, and can also use Firebird 4.0.0 with the 32K page size. IBExpert Benchmark 2.0 was released on 17th November 2021.

The Benchmark Tool tests 10 parallel running threads.

Running the benchmark will not overwrite any programs already in use, nor does it install anything, so you can safely run the benchmark on your server. The IBExpert Benchmark Tool does not use an existing Firebird installation and will only test the drive on which it is installed.

Select the Firebird version you wish to use for your benchmark in the left-hand panel: Firebird 2.5 Superserver, Firebird 3.0 Superserver or Firebird 4.0 Superserver. The selection in this free version has been limited so that even users with no Firebird knowledge or experience can easily perform this quick benchmark test.

The right-hand window in this free version is read-only. (In the paid version you can enter any and all of this information, e.g. define your own connection string and client library file).

Start the benchmark by clicking Start Benchmark in the bottom left-hand corner.

Important: you will need to install and run this as Admin.

Note: Should you receive an error message, please download and install the current version.

The benchmark process and results are explained below under How does it work?

If you would like a more flexible version of our Benchmark, which allows you to change various test options, test multiple drives, various Firebird instances or even perform a benchmark on a Linux operating system or remote servers, you should use the following Benchmark, included in all IBExpert customer version IDEs.

back to top of page

This version offers standard, extended and XXL benchmarks, which can be performed on the Firebird 2.5 Superserver, Classic server, Superclassic server or Firebird 3.0 and Firebird 4.0 32- or 64-bit, Windows, Linux etc. either locally or remote. The Benchmark software is implemented as a 32-bit Windows software, but it can connect to any remote Firebird installation.

If you wish to collect the results for comparison with different hard and software combinations, enter a brief description of the configuration to be tested, along with the recipient email address for the statistic report.

Benchmark (extended) results using Firebird 4.0 on an IFSML Server

Then add the required database connection information - you need to set a new path for the newly generated database; if you specify the path of an existing database, you will receive a warning message:

Specify the Firebird client, as well as the user name and password (the standard SYSDBA and masterkey). Finally click Start benchmark to run the benchmark test.

IBExpert now creates the demo database, which always follows the same guidelines. It then executes the INITALL procedure in this database. Following this parallel threads are created, which attempt to input further orders into the database. Following the Drive Test, the CPU Test test then is executed.

The results are displayed in the lower window: the drive and the CPU index are percentage index values comparing to our our IFS Server v. 6.0 series, which we rated at the time at 100%.

If you have specified an email address, you will receive the results by mail, with the heading as specified in the Please add a short description of the Firebird server hard- and software field.

back to top of page

Based on a demo database, which is created automatically, the standard benchmark, which is also included in the Free IBExpert Benchmark Tool, runs approximately 8 million operations (individual insert/update/delete/select statements) using 10 simultaneous processes in 2 runs on databases created with different parameters to demonstrate the impact of the physical drive speed.

The extended benchmark uses 50 parallel processes (simulating 50 concurrent users) and the XXL benchmark simulates 250 concurrent users with its 250 simultaneous processes.

The Drive Test is simply creating a benchmark database from a script. Within the Drive Test we changed the value of the so-called cache buffers (DefaultDBCachePages in firebird.conf) to 50, the smallest value possible. Then a stored procedure is executed inside the database to generate a lot of test data. In the Drive Test approximately 850,000 operations (i.e. INSERT, UPDATE, DELETE, SELECT statements) are carried out. This really measures the speed of the drive, because of the low-level of cache that is being used, is responsible for making a lot of I/O operations on the external drive.

• Create DB - shows the time in seconds that it takes to create the Demo database.
• TMP - Temporary file management that is used when for example, Firebird does not have any indices, which it can use, or it needs to perform some GROUP BY, HAVING or similar instructions.
• Thread test - this is the multi-threaded test. If you have for example a test on this machine, and you have 1 thread or 10 threads running, you will also see this running in the Task Manager. This can result in huge differences on different machines.
• Result - the basic value that is calculated here displays the combined results of the drive test, and we compare this value to a server that was distributed by ourselves around 2010 which had 100% speed. In reality, when you test on typical customer machines, especially in virtual environments, you will often see values of as little as 30%. Typical values are often in the range of 100%-150%.

This category calculates the percentage value for multi-threaded CPU tests with 5000 pages cache result to better see the advantages when using Firebird 3. This means that it no longer needs any operations on the disk. You will see a much faster result. The Thread Test, the multi-threaded line - with this one 10 parallel threads are always executed, which means it generates several thousand new orders inside the Demo database simultaneously. The Threads value here is the most important value, when you later have to check whether the server is really fast. Only this Threads value is recalculated to the Thread Test.

The Thread Test simply takes the CPU Test / Threads value and recalculate it. Thread Test simply means that we take the CPU Test / Threads value and compare it to the original machine. This machine is compared to our original machine as an index value.

back to top of page

Compare your server's performance to our IFS servers! This simple but effective tool gives you a reference benchmark, based on our recommended IFS Servers.

Our servers will show you a typical Firebird 2.5 Superserver performance of at least 200% for the CPU and at least 200% for the HDD Test with our IFS server series), und using Firebird 3.0 at least 230% for the CPU and at least 500% for the HDD Test. Results under 100% indicate an outdated server performance.

Important: This benchmark gives you a value to compare several hardware or software configurations running the Firebird server.

You will notice a huge performance increase, when comparing a Windows environment mit a Linux environment on the same hardware. This is partially due to the Windows NTFS file system, which is standard on Windows.

If, for example, you want to restart the Benchmark, especially on virtual machines, we always recommend executing this Benchmark several times, because sometimes when you execute it again, you can achieve better or worse values. This can happen because other jobs are being executed by the virtual machine host.

If the results are poor, click in the results window. You will be automatically redirected to our homepage. Here you can view our current offers for our really fast IFS Firebird servers.

Changing the server to a more powerful hardware might improve the speed by 100%. Additionally optimizing your software and database operations can often improve the speed by more than 1000%. Ask sales@ibexpert.biz for further support.