| T O P I C R E V I E W |
| quinncurtis |
Posted - 14 Dec 2007 : 15:08:55
We need to be able to render scatter plots with large volumes of points (at least 100K. 1000K would be nice.) I ran a few brief experiments to test the performance at 100K and the performance was not what we would like.
Do you have any advice on how to get that kind of performance out of a scatter plot?
On a related note, we would also like to be able to dynamically update the plot in response to changes in the data. From what I can tell from your examples, this would require a redraw after every data change, which is not desirable give the performance issues mentioned above.
Do you have any suggestions on how to manage updates more efficiently?
Thanks
Andy |
| 5 L A T E S T R E P L I E S (Newest First) |
| quinncurtis |
Posted - 18 Dec 2007 : 11:39:24
We see no way to provide an alternative scatter plot update mechanism. Based on your description, you will always need to redraw the entire scatter plot in order to ensure that the graph is properly updated with all current data. Any sort of incremental update will result in partial erasures and artifacts, because a graph with that many scatter plot points is bound to have thousands of overlapping symbols. Incrementally adding new points will ruin any implicit z-ordering of overlapping symbols, and removing old points will partially erase overlappling symbols. Your best choice is to make the redraw operation as fast as possible by using QCChart2D and a fast computer.
|
| aclune |
Posted - 18 Dec 2007 : 11:10:44
The size of our data set is not set. Point can come and go in batches. The size of the batches can vary anywhere from 1 to n, where n is the number of points.
When and how fast these updates come in is highly variable. They can be the result of user interactions or automatic processes.
There are three kinds of updates.
1) Appends and deletions - points getting added and deleted basically. FYI sometimes this is more a result of points being hidden than being deleted. I am just assuming that it is easier to just delete them from the plot.
2) State changes - We can color points based on certain criteria. The color of a given point can change at any time and we need that to be reflected in the plot. Also, we will need to have some concept of points being selected as well. Thats another issue though.
3) Data changes - This is by far the least likely, but it could happen. The data used to position a given point could change and we would need that point to move accordingly. |
| quinncurtis |
Posted - 17 Dec 2007 : 14:50:11
Our own tests show that you can expect about a 4-5 times speed improvement if you use QCChart2D instead of QCChart3D, which gets you to about 12 seconds per update on whatever computer you ran your benchmark on. A faster computer may be an option, since we ran our tests on a 5 year old 1.8GHz Celeron and yours seem to be about 2X times as fast.
Describe the nature of your updates. Are you always displaying the same number of data points, and the underlying values of the data points change? Or are you appending new data, in small batches, and you would like to see those points added to the graph?
|
| aclune |
Posted - 17 Dec 2007 : 14:03:14
I did my bench marks using ChartObj.SQUARE. As far as I know I used a 2 1/2 D plot with no transformations save scaling. The only complication I threw in was using 5 different data sets that I colored differently. Under these condition I am seeing update times of a little over a minute for 100K points.
Sorry, I was a little vague on the performance statement. We have (possibly) very large (100K +), very dynamic data set that we would like to display in close to real time. I need to be able to plot 100K points very quickly (< 5 secs). I also will need to be able to update these plots at least that quickly as changes are made to the backing data.
My first concern is the drawing the initial data set quickly. I think we can deal with a little lag on the initial plotting, but a minute or so will probably not work for our customers.
My second concern is that it appears that in order to update the scatter plot I have to redraw the whole thing ever time any data changes. Updates to our data set can occur quite frequently, and we need to handle these updates quickly. once again, lags of a minutes or so are not good.
What I am looking for are ways to address these performance concerns. Chart 2D is certainly a distinct possibility since I don't think we have a requirement for 3D. I guess the only issue there is licensing. |
| quinncurtis |
Posted - 14 Dec 2007 : 15:24:21
You do not say what scatter plot symbol you are using. Certainly you do not want to be using any of the 3D scatter plot symbols (they all end in 3D). The simple 2D ChartObj.SQUARE symbol would be the fastest to draw.
Are the scatter plot symbols positioned in 3D space ? Or, are you just drawing a 2 1/2 D graph, with an implicit z-value. If this is just a 2 1/2 graph, try displaying it without any rotation, which can result in much, much faster rendering. It may be that you should be using QCChart2D, which renders upwards of a 100 times faster that QCChart3D.
"Do you have any advice on how to get that kind of performance out of a scatter plot? " What kind of performance; you don't say ?
We may be able to offer more advise after you answer the questions above.
|
|
|
