Fixing the “Postfix: Value Too Large for Defined Data Type” Error

For lack of better introduction, this post is about diagnosing and fixing the “Value Too Large for Defined Data Type” error in Postfix on Linux. A few weeks ago, shortly after deploying a new email server, I got a bounce notification with the following error:

Diagnostic-Code: X-Postfix; cannot update mailbox /var/spool/mail/username for user username. cannot open file: Value too large for defined data type

Checking the mail log revealed the following, more informative lines:

postfix/local[8354]: warning: this program was built for 32-bit file handles, but some number does not fit in 32 bits

postfix/local[8354]: warning: possible solution: recompile in 64-bit mode, or recompile in 32-bit mode with 'large file' support

postfix/local[8354]: E8FAA86251F: to=username, relay=local, delay=0.01, delays=0/0/0/0.01, dsn=5.2.0, status=bounced (cannot update mailbox /var/spool/mail/username for user username. cannot open file: Value too large for defined data type)

Diagnosing the Problem

This is not a pretty error, and it stems from the fact that 32-bit binary numbers have a fixed maximum value that they can hold. If you overflow this maximum value, bad things will happen (such as mail bouncing from your email server). In modern 64-bit programs, we don’t run into these constraints as much, because 64-bit binary numbers can hold much larger values than 32-bit ones. However, it turned out that the affected email server was running a 32-bit version of Postfix, which meant it was stuck with the limitations of 32-bit binary numbers.

When a program (e.g. Postfix) opens a file (e.g. a user’s inbox) in Linux, some information about the file is passed to the program. The program must store the information in some sort of data structure. In this case, one of the pieces of information is the size of the file, which the 32-bit version of Postfix stores in a 32-bit signed integer. The maximum value that can be stored in a 32-bit signed integer happens to be 2,147,483,647, which is 2 GB expressed in bytes. (Check out the last section of this post for how that number is derived). This means that the largest file size that can be “understood” by the 32-bit version of Postfix is 2 GB.

This works fine until Postfix encounters a file larger than 2 GB. For example, if you have a 3 GB file, the file size expressed in bytes is 3,221,225,472. It is not physically possible to express that value in the 32-bit signed integer that Postfix uses, so the file cannot be opened because its size cannot be understood. This is the root cause of the this program was built for 32-bit file handles error in the mail log.

The GNU Core Utilities FAQ has a paragraph that confirms what the “value too large for defined data type” means:

The message “Value too large for defined data type” is a system error message reported when an operation on a large file is attempted using a non-large file data type. Large files are defined as anything larger than a signed 32-bit integer, or stated differently, larger than 2GB.

Confirming the Issue

I verified that the issue was caused by a 32-bit version of Postfix with the following two steps:

1. By running the file command on the main Postfix binary, which yielded the following output: ELF 32-bit LSB executable, Intel 80386.
2. By verifying that the size of the mail spool file that caused the error was larger than 2 GB.

Additionally, the problem affected a few other users, all of whom had mail spool files larger than 2 GB.

Fixing Postfix

Unfortunately, there’s no configuration parameter that you can change or set that will allow a 32-bit installation of Postfix to overcome the innate 32-bit size limit for files. There is an option — mailbox_size_limit — that controls the maximum mailbox size that Postfix will allow, but that is unrelated to the file size limitation imposed by running a 32-bit application.

Given that the issue is caused by running a 32-bit version of Postfix, fixing the problem means recompiling a 64-bit version of Postfix or installing a 64-bit version directly. In my case, the 32-bit version had been installed by a piece of proprietary software, and I had to switch to a different installation of Postfix entirely.

Running file on the fresh installation of Postfix returned ELF 64-bit LSB shared object, x86-64.

Bonus Section: What Determines the Max Size of a 32-bit Signed Integer?

Traditional 32-bit programs are unable to open files larger than 2 GB due to a limitation imposed by the nature of 32-bit signed integers. For reference, a signed integer is a binary number that is stored using a method that indicates whether the number is positive or negative. The opposite of a signed integer is an unsigned integer. Unsigned integers contain no information about the sign of the number and therefore can only be positive.

There are a few different methods that can be used to store signed integers, but the most common method is two’s complement because of the various benefits it provides. Using two’s complement limits the maximum possible value of an integer, as we can see in the following example:

A 32-bit integer gives us 32 binary bits to use for storing data. If you store the number “zero” in a 32-bit unsigned integer, you would get something like this:

00000000 00000000 00000000 00000000

If we set all the binary digits to 1, we get the following number:

11111111 11111111 11111111 11111111

Converted to decimal, this number is 2³²-1, or 4,294,967,295. The value is 2³²-1 instead of 2³² because binary numbers start at zero. The maximum number of values is 2³², but the maximum value is 2³²-1. This happens to be the maximum size of an unsigned 32-bit integer.

However, we are concerned with the maximum value of a signed integer. If we’re using two’s complement to store the integer, the first bit is reserved to indicate the sign:

10000000 00000000 00000000 00000000

Because we lose the farthest bit to the left, the maximum value is reduced to 2³¹-1, or 2,147,483,647.