patch 9.0.2114: overflow detection not accurate when adding digits

Problem:  overflow detection not accurate when adding digits
Solution: Use a helper function

Use a helper function to better detect overflows before adding integer
digits to a long or an integer variable respectively. Signal the
overflow to the caller function.

closes: #13539

Signed-off-by: Christian Brabandt <cb@256bit.org>
Signed-off-by: Michael Henry <vim@drmikehenry.com>
Signed-off-by: Ernie Rael <errael@raelity.com>
diff --git a/src/misc1.c b/src/misc1.c
index 5f9828e..dc0deae 100644
--- a/src/misc1.c
+++ b/src/misc1.c
@@ -975,9 +975,8 @@
 	c = safe_vgetc();
 	if (VIM_ISDIGIT(c))
 	{
-	    if (n > INT_MAX / 10)
+	    if (vim_append_digit_int(&n, c - '0') == FAIL)
 		return 0;
-	    n = n * 10 + c - '0';
 	    msg_putchar(c);
 	    ++typed;
 	}
@@ -2817,3 +2816,25 @@
     restore_v_event(v_event, &save_v_event);
 #endif
 }
+
+// For overflow detection, add a digit safely to an int value.
+    int
+vim_append_digit_int(int *value, int digit)
+{
+    int x = *value;
+    if (x > ((INT_MAX - digit) / 10))
+	return FAIL;
+    *value = x * 10 + digit;
+    return OK;
+}
+
+// For overflow detection, add a digit safely to a long value.
+    int
+vim_append_digit_long(long *value, int digit)
+{
+    long x = *value;
+    if (x > ((LONG_MAX - (long)digit) / 10))
+	return FAIL;
+    *value = x * 10 + (long)digit;
+    return OK;
+}