Dima Computation times...

deals with computation of distance matrices

Dima Computation times...

Postby Bernd Welter » Tue Nov 14, 2017 3:53 pm

Hi there,

every once in a while I'm getting asked for "performance of distance matrix calculation". Here are some simple results I produced on my local machine (I7- details below - 16GB memory, xTour 1.24.0.3, HIGH PERFORMANCE ROUTING enabled). I calculated the distance matrices from scratch.
dima.PNG
Overview

500 x 500 : 2.0 seconds
1.000 x 1.000 : 3.0 seconds
2.000 x 2.000 : 5.5 seconds
3.000 x 3.000 : 9.5 seconds
4.000 x 4.000 : 14.5 seconds
5.000 x 5.000 : 21.0 seconds
7.500 x 7.500 : 39.0 seconds
10.000 x 10.000 : 64.0 seconds
15.000 x 15.000 : 136 seconds
20.000 x 20.000 : 240 seconds
Feedback is welcome.

Best regards,
Bernd

dima 0500.png
500 x 500
2.0 seconds
dima 1000.png
1000 x 1000
3.0 seconds
dima 2000.png
2000 x 2000
5.5 seconds
dima 3000.png
3000 x 3000
9.5 seconds
dima 4000.png
4000 x 4000
14.5 seconds
dima 5000.png
5000 x 5000
21.0 seconds
dima 7500.png
7500 x 7500
39.0 seconds
dima 10000.png
10.000 x 10.000
64 seconds
dima 15000.png
15.000 x 15.000
136 seconds


Betriebssystemname: Microsoft Windows 10 Enterprise
Betriebssystemversion: 10.0.10586 Nicht zutreffend Build 10586
Betriebssystemhersteller: Microsoft Corporation
Systemhersteller: LENOVO
Systemtyp: x64-based PC
Prozessor(en): 1 Prozessor(en) installiert.
[01]: Intel64 Family 6 Model 60 Stepping 3 GenuineIntel ~2494 MHz
BIOS-Version: LENOVO GNET83WW (2.31 ), 03.05.2017
Gesamter physischer Speicher: 16.009 MB
Verfügbarer physischer Speicher: 3.209 MB
Virtueller Arbeitsspeicher: Maximale Größe: 22.338 MB
Virtueller Arbeitsspeicher: Verfügbar: 2.902 MB
Virtueller Arbeitsspeicher: Zurzeit verwendet: 19.436 MB
Bernd Welter
Manager Technical Consulting & Requirement Engineering
Senior Technical Consultant Developer Components
PTV GROUP - Germany

Bernd at Youtube
User avatar
Bernd Welter
Site Admin
 
Posts: 1131
Joined: Mon Apr 14, 2014 10:28 am

Re: Dima Computation times...

Postby Joost » Wed Nov 15, 2017 9:20 am

For users who want to test this themselves: When testing performance of xServers and especially xDima, keep in mind that the first request after an xServer has started always take much longer then subsequent request since all data has to be read in from the hard drive and there is nothing yet cached by either the xDima or the OS.

Before benchmarking the performance of your local system, please do a few request as " warm up phase" before actually measuring.
Joost Claessen
Senior Technical Consultant
PTV Benelux
Joost
 
Posts: 208
Joined: Fri Apr 25, 2014 1:46 pm

Re: Dima Computation times... ( XSERS-944 )

Postby Bernd Welter » Wed Mar 07, 2018 4:20 pm

Hello Joost,

imagine we have about 4-5GB per searchgraph, about N graphs (profiles) all together (required filespace = 5GB x N).
How does a server react (with more than 5GB x N memory) if he changes from one profile/SG to another? (after the warm up)

Would be benefit from "5GB x N" memory? Or does the process clean the memory with a changing graph and the required amount of memory does only depend on the target number of backend modules? (e.g. 8 backend modules, so only 40GB plus core memory)

Best regards,
Bernd
Bernd Welter
Manager Technical Consulting & Requirement Engineering
Senior Technical Consultant Developer Components
PTV GROUP - Germany

Bernd at Youtube
User avatar
Bernd Welter
Site Admin
 
Posts: 1131
Joined: Mon Apr 14, 2014 10:28 am

Re: Dima Computation times...

Postby Joost » Thu Mar 08, 2018 9:37 am

There has been a lot of optimization is the use of search graphs in the few latest versions in the engine. Currently I'm not completely up to date what the xDima (or other xServer that uses high performance routing) actually needs to read in when switching graph. Let's see what development has to say about this.
Joost Claessen
Senior Technical Consultant
PTV Benelux
Joost
 
Posts: 208
Joined: Fri Apr 25, 2014 1:46 pm

Re: Dima Computation times...

Postby Bernd Welter » Tue Jun 12, 2018 10:00 am

Here is some more infro from DEV (Maximilian):

xServer 2 answer:
SearchGraphs/HPR networks are loaded and unloaded for each xroute/xdima request, i.e. the xServer does not cache the files explicitly but uses memory mapped files. For sure, the file system or the operating system can and hopefully will cache the HPR networks when these files are loaded frequently. I'm no expert on FS/OS caching, but I assume that more RAM is beneficial. Falling back on OS/FS caching has another benefit: It doesn't matter which backend module loads the HPR network.
The required memory does not depend directly on the number of backend modules (because of the memory mapped files). It is more important to have N times M amount of RAM where N is the number of profiles and M the size of the searchgraph/HPR network.

xServer 1 answer:
Frank thinks there should be no fundamental difference.

Regarding your dima performance experiments:
Not only the amount of locations impact the performance, but also their distribution. A 2000x2000 dima in Bavaria is calculated faster than a 2000x2000 dima across Europe. The size of the area of all the locations has a much stronger influence on the performance of conventional routing than on the performance of high-performance routing.
Bernd Welter
Manager Technical Consulting & Requirement Engineering
Senior Technical Consultant Developer Components
PTV GROUP - Germany

Bernd at Youtube
User avatar
Bernd Welter
Site Admin
 
Posts: 1131
Joined: Mon Apr 14, 2014 10:28 am


Return to PTV xDimaServer

cron