More information requested on LOAD_MEMORY_MB [CR 426783]

The amount of virtual memory used by load command is a function of the total number of bytes (as defined by the table schema) for all columns being loaded. For example, consider the following schema:

CREATE table xx(c1 integer, c2 varchar(300), c2 double)

The total number of bytes is 312:

integer       4 bytes
varchar(300)  300 bytes
double        8 bytes
-----------------------
total         312 bytes

Using this example, the load would require approximately 140MB:

312 * 45 * 10000 => 140MB

The amount of virtual memory used can become quite large if many columns (such as in a very wide table) are loaded at once. The wider the table, the more the load memory. The more users doing loads, the more heap/load memory is allocated outside IQ.

There are several courses of action you can take if you encounter the following error:

"All available virtual memory has been used ..."

You can set an upper limit on the amount of virtual memory a LOAD command can use by setting LOAD_MEMORY_MB to a non-zero value, with 2000MB the maximum allowed value.

You can also adjust BLOCK FACTOR or BLOCK SIZE LOAD command options. These command options default to 10000 and 500000, respectively, but you can set them to any number. Setting them lower forces the load to use less virtual memory.

You can also resort to loading a subset of the columns at a time, which is referred to as a partial-width load.