Appendix A: Troubleshooting Monitor Server


Error messages

This section describes the error messages that might appear in the Monitor Server error log. The error log file name is specified in the -l parameter to the Monitor Server start-up command. The default path name is ms.log in the current working directory.

The messages are listed alphabetically. Possible causes and resolutions follow each message. Sometimes several messages have the same root cause and the same solution. If you see messages grouped together in the error log, consider whether the same solution might address all of them.

Allocation failure for <n> bytes: all configured summary buffers in use.

A client connection has exceeded the configured maximum per-connection buffer allocations. To increase the maximum size of the buffers or the maximum number of buffers per connection, increase the value of the bufsize or the max_mem_blks Monitor Server configuration parameters. See “Configuring Monitor Server heap space usage” for more information.

Event buffer wrap: <n> events lost.

Low-level monitoring information is being written by Adaptive Server into its event buffers more rapidly than Monitor Server can extract it.

The recommended procedure is to first reset the event buffers per engine Adaptive Server configuration parameter to a larger number. See “Configuring event buffers” for procedures. If event loss still occurs, consider overriding the default value of the scan_interval Monitor Server configuration parameter.

Upon detecting event loss, Monitor Server dynamically reduces its scanning interval from its initially computed value to a value at which event loss no longer occurs. You can determine this recomputed scanning interval by using the sms_status scan_interval command. Enter the value returned by this command in the Monitor Server configuration file as the value of the scan_interval parameter. This causes Monitor Server to scan at or below this interval, rather than starting at its calculated interval whenever it is booted. See “Initial scan interval” for more information.

Failed to allocate a chunk of <n> bytes for summarized event data.

Monitor Server tried to allocate space for summary data but no more space was available. This condition occurs most often when data for many Historical Server views is being collected.

To resolve this condition, you can increase the ceiling of the address space available to Monitor Server by changing the shared memory starting address Adaptive Server configuration parameter. Specifying a higher shared memory starting address provides more heap space to Monitor Server.

The problem is independent of the amount of memory Adaptive Server starts with as dictated by the sp_configure memory command. It is dependent on the amount of memory available between the size of the Monitor Server binary and the starting address of shared memory.

To change the shared memory starting address:

  1. Make sure that Adaptive Server is running and Monitor Server is not running.

  2. Change the shared memory starting address parameter using either of the following methods:

    Using Sybase Central

    Using isql

    1. Select the appropriate Adaptive Server folder.

    2. From the File menu, choose Configure.

    3. In the scroll box, find the entry for shared memory starting address. In the Value column, change the number to a properly aligned starting address that is higher than the current value. For example, to start shared memory at the 16MB address, use 16777216.

    4. Select OK. The new value appears in the Pending column until you restart Adaptive Server.

    1. Open an isql connection to the appropriate Adaptive Server.

    2. Issue the following commands:

      1> sp_configure "shared memory starting address", number 
      2> go 
      

      where number is a properly aligned starting address that is higher than the current value. For example, to start shared memory at the 16MB address, use 16777216.

  3. Stop Adaptive Server, restart it, and verify that it starts correctly. See step 5 if Adaptive Server fails to start.

  4. Start Monitor Server and verify that it starts correctly. See Chapter 3, “Starting, Verifying, and Stopping Monitor Server” for more information.

  5. If Adaptive Server fails to start with the new shared memory starting address, edit the Adaptive Server configuration file. The file resides in the SYBASE root directory. Its name is serverName.cfg, where serverName is the Adaptive Server name. Search for the phrase “shared memory starting address” and change the associated value to the previously used value. To find the most recently used configuration values for your Adaptive Server, look in the file whose name is serverName.nnn. If there are multiple files with .nnn extensions for serverName, the file with the highest value for .nnn is the latest one.

    After editing the configuration file and successfully restarting Adaptive Server, try changing the starting address again, making sure to use a shared memory address valid for your platform.

    ftok(<$SYBASE/install/sqlServerName>.krg) failed: No such file or directory.
    Fatal error -1 detected
    

Monitor Server is unable to locate the Adaptive Server .krg file, which it requires in order to attach to shared memory. In the message text, sqlServerName is the Adaptive Server being monitored. Check the following:

Monitor Server has encountered a fatal error and is quitting.

Monitor Server detected an unrecoverable error condition and is terminating. A message describing the error condition is written to the Monitor Server log file, if possible. The exit status of Monitor Server may be of use to Sybase Technical Support in diagnosing the problem if the log file does not provide sufficient information.

Stored procedure ’master..mon_rpc_attach’ not found. 

Monitor Server attempted to call the mon_rpc_attach stored procedure, but the procedure is not in Adaptive Server. Either the stored procedure was not created during Monitor Server installation, or it was deleted from the Adaptive Server.

Create the stored procedure in the Adaptive Server you are monitoring by executing the installmon batch file in isql. The installmon batch file contains the SQL text to create two stored procedures required by Monitor Server. Follow these steps:

  1. Make sure Adaptive Server is configured and running. Monitor Server can be running or not running.

  2. Execute isql, attaching to the Adaptive Server you are monitoring, and using the installmon batch file as input. Example isql commands are:

    UNIX:

    isql -Uuser -Ppasswd -SsqlServer $SYBASE/scripts/installmon

    NT:

    isql -Uuser -Ppasswd -SsqlServer %SYBASE%/scripts/installmon

msgid:20073 -- os_attach_region: shmget(0xc70e0703): No such file or directory
msgid:20111 -- kbattach: couldn’t attach to Kernel region
Unable to attach with shared memory. 
Fatal error -1 detected

For Monitor Server to attach to Adaptive Server shared memory, the user account that starts Monitor Server must have:

Log in using the same account that started Adaptive Server and attempt to start Monitor Server again.

msgid:20112 -- kbattach: attached to wrong Kernel region
Unable to attach with shared memory.
Fatal error -1 detected

A version mismatch between Monitor Server and Adaptive Server exists. Make sure that Monitor Server is attached to a supported Adaptive Server.

<n> failures to allocate space for summarized data - no more will be logged.

Check earlier messages in the error log describing failed attempts to allocate memory for summarizing event-based monitoring information. Take the actions prescribed for those error messages.

No server log file open; Using stderr for log.
Open Server:  Error: 16012/10/0: Can’t open log file ’$SYBASE/install/ms.log’
Openserver srv_init call failed.
Monitor Server has encountered a fatal error and quitting

Start Monitor Server as a user who has permission to read and write to the Monitor Server log file.

Open Server: Fatal Error: 16029/20/0: Failed to start any network listeners

Monitor Server is already running or you specified a port number in the interfaces file that is already in use by another application.

See Chapter 3, “Starting, Verifying, and Stopping Monitor Server” for ways to verify whether Monitor Server is running. If a Monitor Server is not running for Adaptive Server, change the Monitor Server port number used in the interfaces file. Then restart Monitor Server.

Open Server:  Error: 16104/10/1: Unable to allocate stack, size 34816, for new thread

The starting address of the shared memory segment that Monitor Server is sharing with Adaptive Server is too low. It is not a fatal error for Monitor Server, but no new connections can be made.

To change the shared memory starting address to a higher number, follow the procedures described earlier in this chapter for the error message:

Failed to allocate a chunk of <n> bytes for summarized event data.
Adaptive Server seems to be down. Shutting down the monitor server.

In its default configuration, Monitor Server shuts itself down gracefully if Adaptive Server has been down for approximately two minutes. After restarting Adaptive Server, restart Monitor Server.

The Monitor Server heartbeat feature checks periodically to see if Adaptive Server is still running. When Monitor Server detects that Adaptive Server is down, it shuts itself down. The heartbeat_interval Monitor Server configuration parameter controls the frequency that Monitor Server checks on Adaptive Server. The default value is 120 seconds.

Unable to attach with shared memory.

Monitor Server cannot find the correct .krg file. Adaptive Server creates a .krg file during start-up. Monitor Server reads the .krg file during start-up, and uses the information to attach to the Adaptive Server shared memory segment.

Check for these situations:

To confirm that a .krg file is old, compare the date of the .krg file to the last Adaptive Server start-up date. You can find the last Adaptive Server start-up date by checking the Adaptive Server error log. If the dates do not match, Monitor Server was trying to access a bad version of the .krg file.

Also check the usage of start-up parameters for both Monitor Server and Adaptive Server. The value specified for the -M parameter in the Adaptive Server start-up command must match the value for the -m parameter in the Monitor Server start-up command. The -M parameter tells Adaptive Server where to put the .krg file, and the -m parameter tells Monitor Server where to look for it.

Do not move the .krg file after Adaptive Server starts.