Sybase recommends that you use the RSA algorithm to protect passwords on the network.
To use the RSA algorithm, you must have Adaptive Server version 15.0.2 and new Connectivity SDK clients (version 15.0 ESD#7 and later.) Sybase provides the net password encryption reqd configuration parameter and the net password encryption server option to allow settings equivalent to pre-15.0.2 release and maintain backward compatibility with older clients and older servers.
Older clients that do not support RSA algorithm can set the property to encrypt passwords using the Sybase proprietary algorithm, available since pre-12.0 releases. Adaptive Server then uses the Sybase proprietary algorithm.
New clients that support both RSA and Sybase proprietary algorithms can set properties for both algorithms. When communicating with such clients, Adaptive Server 15.0.2 uses RSA encryption. A pre-15.0.2 Adaptive Server uses Sybase’s proprietary algorithm.
See client documentation for details for RSA encryption algorithm, referred to as Extended Password Encryption feature in the sections “What’s new in ESD #7” and “Extended Password Encryption” in New Features Open Server 15.0 and SDK 15.0 for Microsoft Windows, Linux, and UNIX. These documents describe how to enable password encryption (Sybase proprietary algorithm) or extended password encryption (RSA algorithm) in the client.