Test for numeric overflow

Posted by: Bob Cozzi

Rogue Programmer

Cozzi Productions, Inc.

www.cozTools.com/qryfile

Chicagoland

Test for numeric overflow
has no ratings.

Published: 01 Nov 2012
Revised: 23 Jan 2013 - 4109 days ago
Last viewed on: 24 Apr 2024 (4951 views)

Main URL:

Test for numeric overflow Published by: Bob Cozzi on 01 Nov 2012 view comments(3)

Wow! I can't believe the number of numeric overflow operations found in legacy code that is converted to /Free syntax.

This is when you have a target variable that is (for example) defined as 7p2 and the code moves a value to that field that is larger than 5-digits, such as: 99999.00 + 1.00

So much legacy code.... So I'm using the ChkDecOvr() (check for decimal overflow) subprocedure to first validate if it will fit and then if it won't take an alternate course of action; rather than just blow up. Or I could wrap the code in MONITOR/on-error groups.

/free
    if chkDecOvr( sales * qty : %len(ordTot) : %decPos(ordTot));
       // Value too big to fit in target variable!
    else;
       ordTot = sales * qty; // We're good.
    endif;
/end-free

Anyway, we've added ChkDecOvr() to the COZTOOLS runtime.

Sort Ascend | Descend

COMMENTS

(Sign in to Post a Comment)

Posted by: neilrh

Premium member *

Jackson, MI

Comment on: Test for numeric overflow
Posted: 11 years 5 months 21 days 15 hours 18 minutes ago
Edited: Fri, 02 Nov, 2012 at 09:04:18 (4191 days ago)

Yup, just this last week, 2 programs, 2 calcs in each...

// Calc percentage
monitor;                                
  eval(h) Result = (Value1 * 100) / Value2;
on-error;                               
  if Value1 < 0 and Value2 < 0 or       
     Value1 > 0 and Value2 > 0;         
    Result = 100;                          
  else;                                 
    Result = -100;                         
  endif;                                
endmon;

But they're breeding like orange barrels on a highway.

Posted by: TFisher

Premium member *

Comment on: Test for numeric overflow
Posted: 11 years 5 months 21 days 14 hours 55 minutes ago

I use the MONITOR approach for the best possible performance.

Posted by: DaleB

Premium member *

Reading, PA

Comment on: Test for numeric overflow
Posted: 11 years 5 months 21 days 14 hours 42 minutes ago

I've also used the Monitor variant, but the chkDecOvr() looks interesting.

When converting existing code, I needed to emulate what the fixed format would have done so the program behaved the same way. Early on I think I did something with %REM(x:SomePowerOf10), but that gets messy when you have non-zero decimal places. A much simpler alternative is to use a DS, OVERLAYing so the decimal points align. In some cases you can do this with Packed, but Zoned is a no-brainer.