Quinn-Curtis Forums
Quinn-Curtis Forums
Home | Profile | Register | Active Topics | Members | Search | FAQ
Username:
Password:
Save Password
Forgot your Password?

 All Forums
 Tools for Microsoft .Net & .Net Compact Framework
 QCChart3D 3D Charting Tools for .Net (VB and C#)
 Displaying large volumes of data in a scatter plot
 New Topic  Reply to Topic
 Printer Friendly
Author Previous Topic Topic Next Topic  

quinncurtis

1534 Posts

Posted - 14 Dec 2007 :  15:08:55  Show Profile  Reply with Quote
We need to be able to render scatter plots with large volumes of points (at least 100K. 1000K would be nice.) I ran a few brief experiments to test the performance at 100K and the performance was not what we would like.

Do you have any advice on how to get that kind of performance out of a scatter plot?

On a related note, we would also like to be able to dynamically update the plot in response to changes in the data. From what I can tell from your examples, this would require a redraw after every data change, which is not desirable give the performance issues mentioned above.

Do you have any suggestions on how to manage updates more efficiently?

Thanks

Andy

quinncurtis

1534 Posts

Posted - 14 Dec 2007 :  15:24:21  Show Profile  Reply with Quote
You do not say what scatter plot symbol you are using. Certainly you do not want to be using any of the 3D scatter plot symbols (they all end in 3D). The simple 2D ChartObj.SQUARE symbol would be the fastest to draw.

Are the scatter plot symbols positioned in 3D space ? Or, are you just drawing a 2 1/2 D graph, with an implicit z-value. If this is just a 2 1/2 graph, try displaying it without any rotation, which can result in much, much faster rendering. It may be that you should be using QCChart2D, which renders upwards of a 100 times faster that QCChart3D.

"Do you have any advice on how to get that kind of performance out of a scatter plot? " What kind of performance; you don't say ?

We may be able to offer more advise after you answer the questions above.
Go to Top of Page

aclune

4 Posts

Posted - 17 Dec 2007 :  14:03:14  Show Profile  Reply with Quote
I did my bench marks using ChartObj.SQUARE. As far as I know I used a 2 1/2 D plot with no transformations save scaling. The only complication I threw in was using 5 different data sets that I colored differently. Under these condition I am seeing update times of a little over a minute for 100K points.

Sorry, I was a little vague on the performance statement. We have (possibly) very large (100K +), very dynamic data set that we would like to display in close to real time. I need to be able to plot 100K points very quickly (< 5 secs). I also will need to be able to update these plots at least that quickly as changes are made to the backing data.

My first concern is the drawing the initial data set quickly. I think we can deal with a little lag on the initial plotting, but a minute or so will probably not work for our customers.

My second concern is that it appears that in order to update the scatter plot I have to redraw the whole thing ever time any data changes. Updates to our data set can occur quite frequently, and we need to handle these updates quickly. once again, lags of a minutes or so are not good.

What I am looking for are ways to address these performance concerns. Chart 2D is certainly a distinct possibility since I don't think we have a requirement for 3D. I guess the only issue there is licensing.
Go to Top of Page

quinncurtis

1534 Posts

Posted - 17 Dec 2007 :  14:50:11  Show Profile  Reply with Quote
Our own tests show that you can expect about a 4-5 times speed improvement if you use QCChart2D instead of QCChart3D, which gets you to about 12 seconds per update on whatever computer you ran your benchmark on. A faster computer may be an option, since we ran our tests on a 5 year old 1.8GHz Celeron and yours seem to be about 2X times as fast.

Describe the nature of your updates. Are you always displaying the same number of data points, and the underlying values of the data points change? Or are you appending new data, in small batches, and you would like to see those points added to the graph?
Go to Top of Page

aclune

4 Posts

Posted - 18 Dec 2007 :  11:10:44  Show Profile  Reply with Quote
The size of our data set is not set. Point can come and go in batches. The size of the batches can vary anywhere from 1 to n, where n is the number of points.

When and how fast these updates come in is highly variable. They can be the result of user interactions or automatic processes.

There are three kinds of updates.

1) Appends and deletions - points getting added and deleted basically. FYI sometimes this is more a result of points being hidden than being deleted. I am just assuming that it is easier to just delete them from the plot.

2) State changes - We can color points based on certain criteria. The color of a given point can change at any time and we need that to be reflected in the plot. Also, we will need to have some concept of points being selected as well. Thats another issue though.

3) Data changes - This is by far the least likely, but it could happen. The data used to position a given point could change and we would need that point to move accordingly.

Edited by - aclune on 18 Dec 2007 11:11:47
Go to Top of Page

quinncurtis

1534 Posts

Posted - 18 Dec 2007 :  11:39:24  Show Profile  Reply with Quote
We see no way to provide an alternative scatter plot update mechanism. Based on your description, you will always need to redraw the entire scatter plot in order to ensure that the graph is properly updated with all current data. Any sort of incremental update will result in partial erasures and artifacts, because a graph with that many scatter plot points is bound to have thousands of overlapping symbols. Incrementally adding new points will ruin any implicit z-ordering of overlapping symbols, and removing old points will partially erase overlappling symbols. Your best choice is to make the redraw operation as fast as possible by using QCChart2D and a fast computer.

Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
 Printer Friendly
Jump To:
Quinn-Curtis Forums © 2000-2018 Quinn-Curtis, Inc. Go To Top Of Page
Powered By: Snitz Forums 2000 Version 3.4.07