Handling really large numbers, dynamic ranges....

From: Gene Silvernail <genes_at_belhaven.com>
Date: Tue, 3 Sep 2002 10:57:02 -0700

I have some data that extends into the 18 to 24 bit range and of course
it changes over temperature. I'm measuring temp also.
Our measured noise coming off the A/D is +/- 10, very very quiet

Non-linear regression is getting to be a bugger as the numbers can
become very large even at the 3rd order. The max number that I can
Effectively cube is something like 1290 which gets you to the top of
double precision signed numbers. The 3rd degree poly below is a good
example.

I have data that tops somewhere around 12,000,000 and at the low end
its around 100,000. If I start pre-scaling then I'm into keeping track
of how much pre-scaling to apply verses loss of resolution... LOGS?

John Harts book which is referenced in the SwiftX refs is out of print.
I've tried to locate a used one without luck.

Any ideas

Gene Silvernail

y = -7.0970E-11x3 + 2.0936E-06x2 - 2.4255E-02x + 1.6214E+02

----------------------------------------------------------------------
swiftx_at_forth.com The SwiftX programming discussion email list
To unsubscribe, send subject "unsubscribe swiftx" to listar_at_forth.com
For help with listar commands, send subject "help" to listar_at_forth.com
Archives are located at http://www.forth.com/swiftx -- check them out!
----------------------------------------------------------------------
THIS LIST IS NOT FOR BUG REPORTS! Send bug reports to support_at_forth.com.
Received on Tue Sep 03 2002 - 10:57:04 PDT


Subscribe to our e-mail list service. It's free for all SwiftForth and SwiftX users!

This archive was generated 09-Feb-2012. Archive updated nightly.