Follow these steps for Windows NT to get up to
3GB
Make a backup copy of sqlsrvr.exe in the sybase bin directory
Verify the current settings of sqlsrvr.exe using imagecfg.exe (a utility that is available with the NT resource kit, or advanced server)
imagecfg sqlsrvr.exe
sqlsrvr.exe contains the following configuration information:
Subsystem Version of 4.0
Stack Reserve Size: 0x20000
Stack Commit Size: 0x4
Use imagecfg to switch on large addressing using the -l (lowercase L) switch: imagecfg -l sqlsrvr.exe sqlsrvr.exe contains the following configuration information:
Subsystem Version of 4.0
Stack Reserve Size: 0x20000
Stack Commit Size: 0x4
sqlsrvr.exe updated with the following configuration information:
Subsystem Version of 4.0
Image can handle large (>2GB) addresses
Stack Reserve Size: 0x20000
Stack Commit Size: 0x4
verify ASE is able to start
The NT machine must be booted with the /3GB flag in the boot.ini file and must have sufficient paging file space (e.g., if you want ASE to access 3G of memory then the paging file must be at least that size)
Increase total memory to anything > 2gb.
Increase starting virtual memory address to 23662592 decimal (which is 1691000 hex) as shown:
sp_configure 'shared memory starting address', 23662592
Restart server.