- Click Start, and then click Run.
- In the Open box, type Regedt32.exe, and then press ENTER.
- Locate the following subkey in the registry:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon
- Double-click the DefaultUserName entry, type your user name, and then click OK.
- Double-click the DefaultPassword entry, type your password, and then click OK.NOTE: If the DefaultPassword value does not exist, it must be added. To add the value, follow these steps:
- On the Edit menu, click New, and then point to String Value.
- Type DefaultPassword, and then press ENTER.
- Double-click DefaultPassword.
- In the Edit String dialog, type your password and then click OK.
- On the Edit menu, click New, and then point to String Value.
- Type AutoAdminLogon, and then press ENTER.
- Double-click AutoAdminLogon.
- In the Edit String dialog box, type 1 and then click OK.
- Quit Registry Editor.
- Click Start, click Shutdown, and then type a reason in the Comment text box.
- Click OK to turn off your computer.
- Restart your computer. You can now log on automatically.
Wednesday, December 31, 2008
Auto Login
To use Registry Editor (Regedt32.exe) to turn on automatic logon, follow these steps:
Shutdown Event Tracker
Windows Server 2003 and Windows XP 64-Bit Edition Version 2003
By default, Shutdown Event Tracker is enabled for all Windows Server 2003 operating systems and for Windows XP 64-Bit Edition Version 2003.To disable Shutdown Event Tracker on all Windows Server 2003 operating systems and in Windows XP 64-Bit Edition Version 2003, disable the Display Shutdown Event Tracker policy by using Group Policy. To do this by using Local Group Policy, follow these steps:
- Click Start, and then click Run.
- Type gpedit.msc, and then click OK.
- Expand Computer Configuration, expand Administrative Templates, and then expand System.
- Double-click Display Shutdown Event Tracker.
- Click Disabled, and then click OK.
Thursday, December 4, 2008
Running Multiple AppServers
Introduction
The core of BladeLogic’s three-tier architecture is the Application Server. Controlling communication between clients and servers as well as access to database, file, and mail servers, the Application Server can be adjusted to scale a system and to fine tune its performance. The purpose of this document is to complement the Bladelogic Administrator Guide with additional information that can be useful for tuning and monitoring the BladeLogic Application Server and Database. Familiarity with the Bladelogic Application Server and knowledge of the Administration Guide is assumed.
Application Server System Requirements
This section describes various Application Server settings which can be configured to maximize performance and increase scalability.
JVM parameters
The JVM settings used by Bladelogic Application Server on Windows are stored in the following registry key:
HKEY_LOCAL_MACHINE\SOFTWARE\BladeLogic\Operations Manager\Application Server
On a Linux or Solaris Application Server, these JVM settings are located in the $NSHDIR/br/blappserv script.
These settings are used to specify:
* The JVM Max Heap Size
* The JVM Stack Size
* The Application Server’s Classpath
* The location of various Application Server directories
* The Temp directory used by the Application Server
Most of these settings will never require modification. The JVM settings which are most likely to require modification are described below.
JVM Max Heap size
The JVM setting most likely to require modification is the Maximum Heap Size. This setting controls the maximum amount of memory which can be made available to the Application Server’s JVM. Its initial value is specified during Application Server installation.
On a Windows Application Server, this value is set in the following registry entry:
HKEY_LOCAL_MACHINE\SOFTWARE\BladeLogic\Operations Manager\Application Server\option1
On a Linux or Solaris Application Server, this value is set in the following line of the $NSHDIR/br/blappserv script:
$JAVA_HOME/bin/java -Xss2m -Xmx512M -Djava.io.tmpdir=$BLADELOGIC_HOME/tmp…….
-Xmx512M specifies a max heap size of 512Mb. -Xmx1024M or -Xmx1G specified a max heap size of 1GB.
There are several factors to consider when setting the maximum heap size:
* The Operating System on which the Application Server is running.
o Windows can only manage a 1400MB JVM. 1024MB is considered a safe figure as an upper boundary.
o Linux can only manage a 2000MB JVM. 1536MB is considered as a safe figure as an upper boundary.
o Solaris is said to be able to manage a 3+Gb JVM.
* The impact of Garbage Collection. The larger the JVM is, the later the Garbage Collection will happen and the longer it will run.
* The amount of physical memory and other applications running on the server.
Assuming the Application Server has the recommended 4GB or more of physical memory, the recommended Max Heap Size value for each platform is:
Windows: 1024MB Linux: 1536 MB Solaris: 2048MB
[edit] Application Server Performance/Scalability settings
The BladeLogic Application Server Administration console is a command line utility(blasadmin) that lets you quickly set parameters needed for the Application Server. These parameters define the location and behavior of the application, database, file, mail, and SNMP servers. For more information on the Application Server Administration, see the Bladelogic Administration Guide. This sections describes some of the most important performance and scalability settings which can be configured using the Application Server Administration console
MaxWorkItemThreads
A single Application Server processes Configuration Manager jobs using one job execution thread and a configurable number of work item threads. The job execution thread constantly watches for jobs that require execution. When a job comes due, the job execution thread loads the job and allocates work item threads, which perform all work required for that job. After initiating a job in this way, the job execution thread continues to watch for other scheduled jobs. Bladelogic recommends a value of 50 for MaxWorkItemThreads. Internal testing and experience in the field have shown that the additional overhead required both in the base OS and in the Application Server JVM to support more than 50 WorkItemThreads (additional locks, forks, etc.) rarely results in a performance gain.
Default Value: 50
Recommended Value: 50
blasadmin command to display current value: show appserver MaxWorkItemThreads
blasadmin command to set new value: set appserver MaxWorkItemThreads
MaxJobs
This setting specifies the maximum number of jobs which the Application Server will execute in parallel. By controlling the number of jobs that are processed simultaneously, you can avoid overtaxing Application Server resources.
Default Value: 20
Recommended Value: 20
blasadmin command to display current value: show appserver MaxJobs
blasadmin command to set new value: set appserver MaxJobs
MaxWorkerThreads
The BladeLogic Application Server is designed to process connections from many clients simultaneously. Rather than dedicating a thread to each client connection, the Application Server maintains a pool of threads that can be used for processing client activity. BladeLogic calls these worker threads. When a client requests any type of activity, the Application Server picks a worker thread from the pool to execute that task. When the request is complete, the worker thread is returned to the pool. Using this approach, the Application Server can handle many more client connections than it has worker threads.
This MaxWorkItemThreads setting controls the maximum number of threads which can process requests from clients. For example, you might set this to 10, which means that only 10 client connections can be serviced at a time even though many more users might actually be connected to the Application Server.
Default Value: 10
Recommended Value: If number of concurrent users is significantly larger than the default value of 10, set this to the likely number of concurrent users.
blasadmin command to display current value: show appserver MaxJobs
blasadmin command to set new value: set appserver MaxJobs
MaxContexts
The maximum number of client connections the appserver can manage. This includes GUI connections and CLI connections.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver MaxContexts
blasadmin command to set new value: set appserver MaxContexts
ComponentCacheMaxSize
The maximum number of Components to cache.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver ComponentCacheMaxSize
blasadmin command to set new value: set appserver ComponentCacheMaxSize
TemplateCacheMaxSize
The maximum number of Templates to cache.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver TemplateCacheMaxSize
blasadmin command to set new value: set appserver TemplateCacheMaxSize
SnapshotCacheMaxSize
The maximum number of Snapshots to cache.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver SnapshotCacheMaxSize
blasadmin command to set new value: set appserver SnapshotCacheMaxSize
AssetPathCacheMaxSize
The maximum number of Asset Paths to cache.
Default Value: 10000
Recommended Value: 10000
blasadmin command to display current value: show appserver AssetPathCacheMaxSize
blasadmin command to set new value: set appserver AssetPathCacheMaxSize
FileSystemObjectCacheMaxSize
The maximum number of File Assets to cache.
Default Value: 50000
Recommended Value: 50000
blasadmin command to display current value: show appserver FileSystemObjectCacheMaxSize
blasadmin command to set new value: set appserver FileSystemObjectCacheMaxSize
PropagateWorkItemTimeout
The Application Server lets you specify time-out behavior for work item threads. By default, if you assign job part time-outs, a work item thread is canceled when it exceeds the time period you have defined in the JOB_PART_TIMEOUT property. In addition, all other work item threads acting on the same server are also canceled. This prevents situations where multiple work item threads time out serially on the same unresponsive server. If necessary, you can use this procedure to override the default behavior so that only one work item thread times out automatically.
Default Value: true
Recommended Value: true
blasadmin command to display current value: show appserver PropagateWorkItemTimeout
blasadmin command to set new value: set appserver PropagateWorkItemTimeout
MaxTimeForCancelToFinish
You can specify a maximum period of time (in minutes) that can elapse for a job part to be canceled. If cancellation of a job part exceeds this maximum, the job part is classified as stuck and the job part is aborted. This prevents situations where cancellation of a job is not performing as expected and the act of canceling the job can potentially hang the job.
Default Value: 20
Recommended Value: 10
blasadmin command to display current value: show appserver MaxTimeForCancelToFinish
blasadmin command to set new value: set appserver MaxTimeForCancelToFinish
EnableSessionBasedCache
Enables/disables caching for each session between the client and app-server
Default Value: true
Recommended Value: true
blasadmin command to display current value: show appserver EnableSessionBasedCaching
blasadmin command to set new value: set appserver EnableSessionBasedCaching
ComplianceResultMaxNumberOfAssets
The Application Server lets you specify the maximum number of compliance results that are displayed for any failed condition in a compliance rule. A Compliance Job examines a component and compares its parts to conditions defined in compliance rules for a component template. Parts that do not comply are shown in Compliance Job results. If you are running a Compliance Job that examines many server objects that fail a compliance condition, you may tax your system resources, particularly disk space. If results for a Compliance Job exceed the limits you set in this procedure, the job log includes a message saying that job results have been truncated.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver ComplianceResultMaxNumberOfAssets
blasadmin command to set new value: set appserver ComplianceResultMaxNumberOfAssets
AssetThresholds MaxConfigRecords
This setting limits the number of records that an Application Server can process for a single configuration object or extended object. Processing large numbers of records for a configuration object or extended object can consume large quantities of memory, particularly when the Application Server processes multiple configuration objects or extended objects on multiple servers simultaneously. By default this value is set to 50,000. Setting this value to 0 means no records are processed. Setting this value to a negative number (for example, -1) means there is no limit to the number of records that can be processed.
Default Value: 50000
Recommended Value: 50000
blasadmin command to display current value: show AssetThresholds MaxConfigRecords
blasadmin command to set new value: set AssetThresholds MaxConfigRecords
Application Server Database-Connectivity settings
The BladeLogic Application Server maintains three different pools of database connections. One is used for processing Configuration Manager jobs, one is use for non-job Application Server activity and the third is used for processing all other activity, such as client connections. When a thread needs a database connection, it acquires one from the appropriate pool of database connections. When the database activity is complete, the database connection is returned to its pool. A thread watches the pool of database connections, ensuring that the number of connections stays within high and low boundaries. If the number of database connections reaches the high boundary, the thread attempts to trim the number of database connections back to the low boundary. These high and low boundaries are controlled by the following blasadmin settings:
Note: The sum of MaxJobExecutionConnections, MaxGeneralConnections and MaxClient Connections should never exceed the connection limit specified by the database server.
MinJobExecutionConnections
The minimum number of job-related database connections that are open at any given time.
Default Value: 0
Recommended Value: 0
blasadmin command to display current value: show database MinJobExecutionConnections
blasadmin command to set new value: set database MinJobExecutionConnections
MaxJobExecutionConnections
The maximum number of job related database connections that are open at any given time.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show database MaxJobExecutionConnections
blasadmin command to set new value: set database MaxJobExecutionConnections
MinGeneralConnections
The minimum number of general, non-job related database connections that are open at any given time. An example of a general connection would be the Application Server polling the DB for jobs which require execution.
Default Value: 0
Recommended Value: 0
blasadmin command to display current value: show database MinGeneralConnections
blasadmin command to set new value: set database MinGeneralConnections
MaxGeneralConnections
The maximum number of general, non-job related database connections that are open at any given time. An example of a general connection would be the Application Server polling the DB for jobs which require execution.
Default Value:20
Recommended Value: 20
blasadmin command to display current value: show database MaxGeneralConnections
blasadmin command to set new value: set database MaxGeneralConnections
MinClientConnections
The minimum number of client related database connections that are open at any given time. An example of a client connection would be viewing job results in the CM GUI.
Default Value: 0
Recommended Value: 0
blasadmin command to display current value: show database MinClientConnections
blasadmin command to set new value: set database MinClientConnections
MaxClientConnections
The maximum number of client related database connections that are open at any given time. An example of a client connection would be viewing job results in the CM GUI.
Default Value: 20
Recommended Value: 20
blasadmin command to display current value: show database MaxClientConnections
blasadmin command to set new value: set database MaxClientConnections
TransactionAttempts
The maximum retry attempts for deadlocked transactions.
Default Value: 10
Recommended Value: 100
blasadmin command to display current value: show database TransactionAttempts
blasadmin command to set new value: set database TransactionAttempts
[edit] FetchSize
The JDBC fetch size gives the JDBC driver a hint as to the number of rows that should be fetched from the database when more rows are needed. For large queries that return a large number of objects this can improve performance by decreasing the number database hits required to satisfy the selection criteria.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show database FetchSize
blasadmin command to set new value: set database FetchSize
Defining Multiple Application Servers on the Same Host
As of version 7.4, it is possible to define multiple application servers on a single host. This is best done when an application server is under-utilized from a memory perspective.
By default, only one application server will run on a single host. However, by configuring a AppServerProfiles.xml file in the/br directory, you can set up an unlimited number of application servers on a single host. Also, you have the ability to specify if each individual application server should function as a Configuration Server (only for client connections), a Job Server (only for executing jobs), a NSH Proxy Server (only serving as a NSH Proxy), or all of the above.
Please refer to the BladeLogic Administration guide for details on how to configure multiple application servers on a single host.
Deciding to Add Multiple Application Servers
Typically you would want to configure multiple application servers on a single host if the application server has 4 GB or greater of memory. The primary reason why is to use more than 4GB of memory and for better thread management. For instance, defining a config-only app server allows you to make the GUIs much more responsive even under heavy load on the appserver, because the JVM doesn’t share threads with the job threads.
Recommended AppServerProfiles.xml
The following AppServerProfiles.xml file will allow for one configuration server and two job servers. Depending on the amount of memory available on your application server, experiment with adding or removing Job servers to this file. You should be able to create this file in the/br directory without making any additional modifications.
\
\
ConfigAppServer1
ConfigAppServer1
CONFIGURATION
JobAppServer1
JobAppServer1
JOB
JobAppServer2
JobAppServer2
JOB
Application Server Monitoring and Maintenance
This section will cover the options and recommended processes for monitoring the application server\’s health and responsiveness.
Checking Application Server Availability
Confirm Open SRP Port
The first step in confirming Application Server availability is to confirm that the SRP port can be reached. Telnet can be used to probe the SRP port on the Application Server. By default, this port is 9829 but is configurable by modifying the SRPPort setting using the Application Server Administration console.
C:\>telnet supwin2k3serv1 9829
If telnet can successfully establish a connection to the Application Server\’s SRP port the following output will be displayed:
BladeLogic A/S v1.0 (BLManager Version: 7.x.x.xxx)
If telnet fails to establish a connection to the Application Server\’s SRP port the following output will be displayed:
Connecting To supwin2k3serv1...Could not open connection to the host, on port 9829: Connect failed
Confirm Application Server Connectivity
The next step in monitoring the availability and responsiveness of the Application Server is to confirm that it is capable of accepting and processing client connections. A simple technique to confirm this is to invoke a basic BLCLI (Bladelogic Command Line Interface) command and confirm that the Application Server returns the expected output. The "RBACUser isUserExists" BLCLI command is a good candidate for this task. This command checks to see if a user exists in the system. It returns a simple boolean value which can be checked programmatically. This test can be automated and run on a scheduled basis using OS specific tools such as crontab on a Unix OS.
Examples: C:\Program Files\BladeLogic\OM\br>blcli RBACUser isUserExists "RBACAdmin" true C:\Program Files\BladeLogic\OM\br>blcli RBACUser isUserExists "UserA" false
In both of these examples, the Application Server successfully received and processed the client request and returned the expected value.
The following is an example of expected output from this command if the Application Server was not running or was unreachable:
C:\Program Files\BladeLogic\OM\br>blcli RBACUser isUserExists "RBACAdmin" Failed to connect to server. com.bladelogic.mfw.util.BlException: Failed to create socket for host \’supwin2k3serv1\’ and port \’9829\’
If the Application Server was alive but had no Worker Threads available, this command would hang until one became available to process the request.
If the Application Server was alive, had available Worker Threads but had exceeded the value of MaxContexts (defined earlier) the expected output would be:
C:\Program Files\BladeLogic\OM\br>blcli RBACUser isUserExists "RBACAdmin" Failed to connect to server. com.bladelogic.mfw.util.BlException: Unexpected welcome message : connection limit exceeded
For more information on running BLCLI commands please see the BLCLI help.
Bladelogic Best Practice
Create a script which probes the Application Server SRP port and runs a simple CLI command. The script should be capable of detecting unexpected output and alerting the Bladelogic Administrator. A suggested interval for running this script would be hourly.
Obtaining a detailed Application Server health report
The previous section used the "RBACUser isUserExists" BLCLI command to determine if the Application Server was alive, reachable and capable of processing a simple client request. This is useful for performing automated checks on Application Server availability but does not provide us with any detailed information regarding the current state of the Application Server e.g. Number of jobs currently running, number of DB connections currently in use, amount of memory currently being consumed, number of open client connections etc. The BLCLI contains a set of commands which are designed to provide a detailed report on the current state of the Application Server. The output of these commands provide crucial information for solving and debugging errors.
AppServerProfiler : getAppServerDetails
This command returns a general status report for the Application Server. The status report contains information such as database connections, memory usage, application server version, etc.
AppServerProfiler : getThreadDump
This command returns the Java thread dump of the Application Server. This shows the live status of the Application Server thread dump and exactly what each individual thread is currently doing. This information can be critical to Bladelogic support in debugging certain error conditions. This command should be run twice - five minutes apart – so that thread progress can be gauged.
DBManager : getDBManagerStatus
This command shows the status of different database-related systems on the Application Server, including the number of available database connections, the number of used connections, the number of threads waiting to get a connection, and so forth.
JobManagement : getJobManagerFullStatus
This command provides detailed status information about all the job-related threads running on the Application Server. It also provides information about all queued and running jobs. This type of information is often used to troubleshoot jobs that are not completing successfully
JobManagement : getRunningJobsStatus
This command shows what jobs are running on the Application Server. It provides each job\’s current status and other detailed information about each job.
New in 7.3:
Profiler : getCompleteStatusReport
This command is only available as an unreleased command. It gathers all the above information in one shot for all the available application servers.
It is the recommended choice for gathering the application server health status.
On Windows, use:
blcli2 -Dcom.bladelogic.cli.debug.release-only="false" Profiler getCompleteStatusReport
On Unix/Linux, use:
blcli -Dcom.bladelogic.cli.debug.release-only="false" Profiler getCompleteStatusReport
Bladelogic Best Practice
Create a script which runs these Application Server diagnostic commands on a nightly basis. These results should be saved so that a recent Application Server history is available for Bladelogic support if required.
[edit] Monitoring the Bladelogic File Server
The Application Server uses the file server to store the contents of files included in snapshots, Network Shell scripts, BLPackages, software packages, and other types of information that is not easily stored in the database. In addition, the file server stores registry, COM+, and metabase values longer than 255 characters.A file server should meet the following requirements:
* An RSCD agent must be installed and licensed.
* A file server must have, as a minimum, 72 GB of available, non-redundant, disk space. BladeLogic recommends that the file server have 200 GB or more of available RAID 5 disk space.
* A user name must be defined on the file server, and all Configuration Manager users must be mapped to that user. Without this mapping a user may not be able to access a file that another user has stored on the file server. One way to accomplish the necessary mapping is to create an entry like the following in the exports file on the file server: rw,user= where is a comma-separated list of Application Server names or IP addresses and is the name to which all users are mapped.
* The internal System:System role/user must be mapped to the user name defined on the file server. To accomplish the mapping, create an entry like the following in the users.local file on the file server: System:System rw,user= where is the name to which all users are mapped, typically bladmin or administrator
You can determine the current file server hostname and storage location using the Application Server Administration console (blasadmin):
C:\Program Files\BladeLogic\OM\br>blasadmin show fileserver all \[FileServer\] name:fs01.sup.bladelogic.com location:/support/storage/
The Bladelogic Administrator should frequently monitor this fileserver storage location (using a command such as "df –h") to ensure it is not running low on available disk space.
Bladelogic Best Practice
Create a script which uses the df command to check the amount of available file server space on a weekly basis and notifies the Application Server Administrator of the results.
Application Server Logging
Overview
The Bladelogic application uses the industry standard log4j package for logging. Log4j is a flexible and efficient logging system.
The log4j system is configured through the log4j configuration file called "appserver.cf". The "appserver.cf" is located under the application server installation directory and follows the format defined by the log4j specifications.
The output of the application server logging goes by default to a file called "appserver.log" which is located under the application server installation directory.
In addition to the "appserver.log" file, an Application Server running on Linux or Solaris can be configured to write all the standard output and standard error information into a file called "console.log". The "console.log" file will contain the same information as the "appserver.log" file but in addition it will also capture any output that does not go through the log4j logging system. This may include information like the java thread dump and any messages generated by third party code used by the application server that logs messages to standard out/err.
Note: More information about log4j can be found here: [1]
Configuring
The "appserver.cf" file can be used to control the amount of logging information available in the log files. This section will explain how to manipulate the configuration file in order to control what logging information gets generated by the application server.
In addition to controlling the logging information, the configuration file can also control where the logging information is stored and how the log files are managed. This information will be covered in the next section.
The log4j logging system used by the application server is a hierarchical tree of loggers. A logger is an entity that can receive and log messages generated by the application. Each application module uses one or more loggers for logging messages. Each logger has a logging level which is used for filtering messages. A logger\’s logging level can be set directly on a logger or implicitly by the setting the level on the parent logger.
For detailed understanding of the logging system it is recommended to read the log4j documentation.
Log Levels
Each message generated by the application has a level associated with it. The level can be "DEBUG", INFO", "WARN", "ERROR". A message will be written to the log file if the message log level is equal or higher then the logger\’s log level.
For example: Application module 1 uses the logger called "com.bladelogic.module1". The logger level is set to INFO. This means that when the application logs a new message with the level DEBUG, the message will be filtered out and will not show up in the log file.
If the application generates a message with the level INFO, WARN or ERROR, the message will show up in the log file.
Adding Debug Information
The "appserver.cf" file contains a large list of loggers that can be configured to add useful debug information. These loggers are initially configured with the log level INFO to prevent the log files from containing too much information. When debugging specific issues with the system, the user can open the file and modify one or more of the specified loggers and set the log level to DEBUG. Debugging issues with the application server will often be assisted by the Bladelogic support team.
The exact list of loggers that can be modified is quite large. Here are a few examples for loggers that are useful for debugging:
* log4j.logger.com.bladelogic.app.db.DBServiceImpl – This logger controls messages generated by the database service.
* log4j.logger.com.bladelogic.mfw.demux.Demux – This logger controls messages generated by the networking layer.
Enabling More Detailed Logging
The console.log file is useful for debugging when certain output is not captured by the regular log files.
The main use case for the console.log file is for getting the java thread dump in rare cases where the getThreadDump BLCLI command cannot be run.
The problem with the console.log file is that the file size is not limited. As opposed to the log4j log files, the file will continue to grow indefinitely. In general, it is recommended that the usage of the console.log file be limited to periods in which specific errors are being debugged.
Log Rotation
Log rotation is full supported by log4j by configuring the configuration for the Application Server in the log4crc.txt file. For more information on configuring the log4crc.txt file, see the Setting Up Configuration Files section in the BladeLogic Administration Guide.
Managing Log Files
Log4j Log Files
The following log attributes can be controlled by modifying options in the "appserver.cf" file; log files location, log files max size, number of roll-over log files.
Log Files Location The log file location can be controlled by modifying the property "log4j.appender.R.File" with the relative or full path of the log file. This will instruct the log4j system to use the specified path for logging.
Max File Size The maximum file size for the log file can be controlled by setting the value of the property "log4j.appender.R.MaxFileSize". By default, the max log file size is 5000KB.
Number of Roll-Over Files The number of roll-over files can be controlled by setting the value of the property "log4j.appender.R.MaxBackupIndex". When a log file reaches its maximum size, a backup file will be made and a new log file will be created. The property value controls how many backup files to keep. By default, the number of backups retained is 5.
Console.log File
The "console.log" file is a file that must be maintained manually by the user. By default, the file does not exist on the application server. The file has to be created by the user when debugging a problem and then removed by the user. If the file is left in place it can grow and consume an unlimited amount of disk space.
Bladelogic Best Practice
Unless instructed otherwise by Bladelogic Support, the default logging configuration is recommended for normal operation. Bladelogic Support may ask the Application Server Administrator to enable DEBUG logging for a single logger when debugging a particular issue. This change will typically be backed out once the requested DEBUG information has been gathered. Likewise, a console.log should only be created on Solaris or Linux application servers when requested by Bladelogic Support.
Database Configuration
The following sections lists some Bladelogic recommendations around configuring the Oracle or SQL Server database instances used by the Bladelogic Application server.
Oracle DB Configuration
When configuring init.ora in Oracle, you must set the following initialization parameters:
* db_block_size should be set to at least 8192.
* The processes parameter should be set to at least 200. Ideally, the number of processes should be 30 to 40 higher than the total number of job-related and non-job-related database connections configured for the Application Server. If you are running BladeLogic Reports, the number of processes should ideally be 110 to 120 higher than the total number of job-related and non-job-related database connections.
* session_cached_cursors 2048
* open_cusors 2096
* The following recommendations are for a 10 GB database. Scale the configuration recommendations up or down based on the relative size of your database.
o db_cache_size = 1 GB
o log_buffer = 512 KB
o pga_aggregate_target = 1GB
o sga_target = 2GB
o undo tablespace size = 2 GB
SQL Server DB Configuration
For advice on SQL Server DB Configuration, please see the following Microsoft KB article: http://support.microsoft.com/kb/319942
Database Monitoring and Maintenance
Bladelogic recommends the following DBA tasks be performed regularly on any database instance which contains a Bladelogic CM or Bladelogic Reports database.
Recommended Oracle DB Maintenance
Gathering schema statistics
BladeLogic recommends you gather statistics for BladeLogic Reports and the core BladeLogic database on at least a weekly basis. Use the following command to gather statistics: exec dbms_stats.gather_schema_stats( ownname => USER, cascade=>TRUE, options => \’GATHER\’, estimate_percent => dbms_stats.auto_sample_size);
Replace USER with the correct user name e.g. bladelogic, blreports.
SQL Server DB Maintenance
Rebuilding SQL Server DB Indexes.
Bladelogic recommends that you rebuild the indexes in both the core Bladelogic and Reports warehouse DBs on a weekly basis using the \’DBCC DBREINDEX" command. (TODO: confirm with dbas that weekly is sufficient) This command only accepts one table name as an argument so the following script can be used in SQL Server Enterprise Manager to loop through all the tables in each database and run the DBREINDEX command against them:
USE DatabaseName --Enter the name of the database you want to reindex
DECLARE @TableName varchar(255)
DECLARE TableCursor CURSOR FOR SELECT table_name FROM information_schema.tables WHERE table_type = \’base table\’
OPEN TableCursor
FETCH NEXT FROM TableCursor INTO @TableName WHILE @@FETCH_STATUS = 0 BEGIN DBCC DBREINDEX(@TableName) FETCH NEXT FROM TableCursor INTO @TableName END
CLOSE TableCursor
DEALLOCATE TableCursor
Bladelogic Best Practice
Run the recommend gather_schema_stats (Oracle) or DBREINDEX (SQL Server) commands against the Bladelogic CM and Reports databases on a weekly basis. These commands should immediately be run against the Reports DB after the initial run of the populate reports job.
Bladelogic Housekeeping Jobs
The following CM jobs should be run at the suggested frequency to keep all CM data up to date.
Update Server Properties
The Update Server Properties Job lets you automatically update intrinsic server property values for a list of targeted servers. Running this job ensures that Configuration Manager reflects the current values for all the intrinsic properties on your servers. Intrinsic properties are properties that are derived from the nature and configuration of a server, such as the server\’s name, root directory, operating system, and so forth. For more information on creating and running Update Server Properties jobs, see the Bladelogic Users Guide.
Bladelogic Best Practice
An Update Server Properties job should be run against all servers in the environment on at least a nightly basis. Several daily runs is recommend since the population of Smart Server Groups depends on these properties values.
ACL Push Jobs
An ACL Push Job uses the authorizations specified for all roles granted access to a server and generates entries in an access control list for that server. The ACL Push Job then copies that ACL to the server\’s agent. On the agent this ACL is called the users configuration file. For more information on creating and running ACL Push jobs, see the Bladelogic Users Guide.
[edit] Bladelogic Best Practice
An ACL Push job should be run against all servers in the environment on at least a nightly basis. An ACL push job should also be run when a user or role is deleted from CM or when a new user or role is added.
Component Discovery Job
A Component Discovery Job associates one or more components with a server. Discovering a component merely determines that a server satisfies the signature of a component template. It does not ensure that all component template parts actually exist on a server.
Bladelogic Best Practice
A Component Discovery Job should be created which discovers all components on all servers in the environment and should be configured to run nightly. This helps to avoid errors during Compliance Job runs due to the version of the component associated with the target server being out of synch with the latest version of the component template. It is also a good idea to run a Component Discovery job immediately before a Compliance Job as part of a single Batch Job. For more information on creating and running ACL Push jobs, see the Bladelogic Users Guide.
OS Config
The OS Config NSH Script job collects data for inventory reports. When you create and run a Network Shell Script Job based on os_config.nsh, the script creates one XML file (known as a server data file) for every server you specify in the job. The server data file contains configuration information about that server. The script copies the server data file to a directory on the reports server. This data is then processed by Populate Reports so it should be run before the Populate Reports job runs.
Bladelogic Best Practice
BladeLogic recommends running this job no more than once a week per server. Ideally, this job should run during off peak hours.
Collect Agent Logs
The Collect Agent Logs NSH Script Job collects data for Server Usage reports. It copies data from agent logs on the target servers you specify to files on the reports server. This data is then processed by Populate Reports so it should be run before the Populate Reports job runs.
Bladelogic Best Practice
The Collect Agent Logs NSH Script Job should only be run if you are interested in Server Usage Reports. If you are not interested in these particular reports there is no need to run the Collect Agent Logs job. Not running this job will result in Populate Reports running to completion quicker and also less space
Populate Reports
The Populate Reports NSH Script Job performs all data aggregation and populates the reporting data warehouse. For more information on setting up this job, see the Bladelogic Installation Guide.
[edit] Bladelogic Best Practice
To ensure that the latest CM data is available in Bladelogic Reports, the Populate Reports job should be run against the Reports Server on a nightly basis.
The core of BladeLogic’s three-tier architecture is the Application Server. Controlling communication between clients and servers as well as access to database, file, and mail servers, the Application Server can be adjusted to scale a system and to fine tune its performance. The purpose of this document is to complement the Bladelogic Administrator Guide with additional information that can be useful for tuning and monitoring the BladeLogic Application Server and Database. Familiarity with the Bladelogic Application Server and knowledge of the Administration Guide is assumed.
Application Server System Requirements
This section describes various Application Server settings which can be configured to maximize performance and increase scalability.
JVM parameters
The JVM settings used by Bladelogic Application Server on Windows are stored in the following registry key:
HKEY_LOCAL_MACHINE\SOFTWARE\BladeLogic\Operations Manager\Application Server
On a Linux or Solaris Application Server, these JVM settings are located in the $NSHDIR/br/blappserv script.
These settings are used to specify:
* The JVM Max Heap Size
* The JVM Stack Size
* The Application Server’s Classpath
* The location of various Application Server directories
* The Temp directory used by the Application Server
Most of these settings will never require modification. The JVM settings which are most likely to require modification are described below.
JVM Max Heap size
The JVM setting most likely to require modification is the Maximum Heap Size. This setting controls the maximum amount of memory which can be made available to the Application Server’s JVM. Its initial value is specified during Application Server installation.
On a Windows Application Server, this value is set in the following registry entry:
HKEY_LOCAL_MACHINE\SOFTWARE\BladeLogic\Operations Manager\Application Server\option1
On a Linux or Solaris Application Server, this value is set in the following line of the $NSHDIR/br/blappserv script:
$JAVA_HOME/bin/java -Xss2m -Xmx512M -Djava.io.tmpdir=$BLADELOGIC_HOME/tmp…….
-Xmx512M specifies a max heap size of 512Mb. -Xmx1024M or -Xmx1G specified a max heap size of 1GB.
There are several factors to consider when setting the maximum heap size:
* The Operating System on which the Application Server is running.
o Windows can only manage a 1400MB JVM. 1024MB is considered a safe figure as an upper boundary.
o Linux can only manage a 2000MB JVM. 1536MB is considered as a safe figure as an upper boundary.
o Solaris is said to be able to manage a 3+Gb JVM.
* The impact of Garbage Collection. The larger the JVM is, the later the Garbage Collection will happen and the longer it will run.
* The amount of physical memory and other applications running on the server.
Assuming the Application Server has the recommended 4GB or more of physical memory, the recommended Max Heap Size value for each platform is:
Windows: 1024MB Linux: 1536 MB Solaris: 2048MB
[edit] Application Server Performance/Scalability settings
The BladeLogic Application Server Administration console is a command line utility(blasadmin) that lets you quickly set parameters needed for the Application Server. These parameters define the location and behavior of the application, database, file, mail, and SNMP servers. For more information on the Application Server Administration, see the Bladelogic Administration Guide. This sections describes some of the most important performance and scalability settings which can be configured using the Application Server Administration console
MaxWorkItemThreads
A single Application Server processes Configuration Manager jobs using one job execution thread and a configurable number of work item threads. The job execution thread constantly watches for jobs that require execution. When a job comes due, the job execution thread loads the job and allocates work item threads, which perform all work required for that job. After initiating a job in this way, the job execution thread continues to watch for other scheduled jobs. Bladelogic recommends a value of 50 for MaxWorkItemThreads. Internal testing and experience in the field have shown that the additional overhead required both in the base OS and in the Application Server JVM to support more than 50 WorkItemThreads (additional locks, forks, etc.) rarely results in a performance gain.
Default Value: 50
Recommended Value: 50
blasadmin command to display current value: show appserver MaxWorkItemThreads
blasadmin command to set new value: set appserver MaxWorkItemThreads
MaxJobs
This setting specifies the maximum number of jobs which the Application Server will execute in parallel. By controlling the number of jobs that are processed simultaneously, you can avoid overtaxing Application Server resources.
Default Value: 20
Recommended Value: 20
blasadmin command to display current value: show appserver MaxJobs
blasadmin command to set new value: set appserver MaxJobs
MaxWorkerThreads
The BladeLogic Application Server is designed to process connections from many clients simultaneously. Rather than dedicating a thread to each client connection, the Application Server maintains a pool of threads that can be used for processing client activity. BladeLogic calls these worker threads. When a client requests any type of activity, the Application Server picks a worker thread from the pool to execute that task. When the request is complete, the worker thread is returned to the pool. Using this approach, the Application Server can handle many more client connections than it has worker threads.
This MaxWorkItemThreads setting controls the maximum number of threads which can process requests from clients. For example, you might set this to 10, which means that only 10 client connections can be serviced at a time even though many more users might actually be connected to the Application Server.
Default Value: 10
Recommended Value: If number of concurrent users is significantly larger than the default value of 10, set this to the likely number of concurrent users.
blasadmin command to display current value: show appserver MaxJobs
blasadmin command to set new value: set appserver MaxJobs
MaxContexts
The maximum number of client connections the appserver can manage. This includes GUI connections and CLI connections.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver MaxContexts
blasadmin command to set new value: set appserver MaxContexts
ComponentCacheMaxSize
The maximum number of Components to cache.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver ComponentCacheMaxSize
blasadmin command to set new value: set appserver ComponentCacheMaxSize
TemplateCacheMaxSize
The maximum number of Templates to cache.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver TemplateCacheMaxSize
blasadmin command to set new value: set appserver TemplateCacheMaxSize
SnapshotCacheMaxSize
The maximum number of Snapshots to cache.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver SnapshotCacheMaxSize
blasadmin command to set new value: set appserver SnapshotCacheMaxSize
AssetPathCacheMaxSize
The maximum number of Asset Paths to cache.
Default Value: 10000
Recommended Value: 10000
blasadmin command to display current value: show appserver AssetPathCacheMaxSize
blasadmin command to set new value: set appserver AssetPathCacheMaxSize
FileSystemObjectCacheMaxSize
The maximum number of File Assets to cache.
Default Value: 50000
Recommended Value: 50000
blasadmin command to display current value: show appserver FileSystemObjectCacheMaxSize
blasadmin command to set new value: set appserver FileSystemObjectCacheMaxSize
PropagateWorkItemTimeout
The Application Server lets you specify time-out behavior for work item threads. By default, if you assign job part time-outs, a work item thread is canceled when it exceeds the time period you have defined in the JOB_PART_TIMEOUT property. In addition, all other work item threads acting on the same server are also canceled. This prevents situations where multiple work item threads time out serially on the same unresponsive server. If necessary, you can use this procedure to override the default behavior so that only one work item thread times out automatically.
Default Value: true
Recommended Value: true
blasadmin command to display current value: show appserver PropagateWorkItemTimeout
blasadmin command to set new value: set appserver PropagateWorkItemTimeout
MaxTimeForCancelToFinish
You can specify a maximum period of time (in minutes) that can elapse for a job part to be canceled. If cancellation of a job part exceeds this maximum, the job part is classified as stuck and the job part is aborted. This prevents situations where cancellation of a job is not performing as expected and the act of canceling the job can potentially hang the job.
Default Value: 20
Recommended Value: 10
blasadmin command to display current value: show appserver MaxTimeForCancelToFinish
blasadmin command to set new value: set appserver MaxTimeForCancelToFinish
EnableSessionBasedCache
Enables/disables caching for each session between the client and app-server
Default Value: true
Recommended Value: true
blasadmin command to display current value: show appserver EnableSessionBasedCaching
blasadmin command to set new value: set appserver EnableSessionBasedCaching
ComplianceResultMaxNumberOfAssets
The Application Server lets you specify the maximum number of compliance results that are displayed for any failed condition in a compliance rule. A Compliance Job examines a component and compares its parts to conditions defined in compliance rules for a component template. Parts that do not comply are shown in Compliance Job results. If you are running a Compliance Job that examines many server objects that fail a compliance condition, you may tax your system resources, particularly disk space. If results for a Compliance Job exceed the limits you set in this procedure, the job log includes a message saying that job results have been truncated.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show appserver ComplianceResultMaxNumberOfAssets
blasadmin command to set new value: set appserver ComplianceResultMaxNumberOfAssets
AssetThresholds MaxConfigRecords
This setting limits the number of records that an Application Server can process for a single configuration object or extended object. Processing large numbers of records for a configuration object or extended object can consume large quantities of memory, particularly when the Application Server processes multiple configuration objects or extended objects on multiple servers simultaneously. By default this value is set to 50,000. Setting this value to 0 means no records are processed. Setting this value to a negative number (for example, -1) means there is no limit to the number of records that can be processed.
Default Value: 50000
Recommended Value: 50000
blasadmin command to display current value: show AssetThresholds MaxConfigRecords
blasadmin command to set new value: set AssetThresholds MaxConfigRecords
Application Server Database-Connectivity settings
The BladeLogic Application Server maintains three different pools of database connections. One is used for processing Configuration Manager jobs, one is use for non-job Application Server activity and the third is used for processing all other activity, such as client connections. When a thread needs a database connection, it acquires one from the appropriate pool of database connections. When the database activity is complete, the database connection is returned to its pool. A thread watches the pool of database connections, ensuring that the number of connections stays within high and low boundaries. If the number of database connections reaches the high boundary, the thread attempts to trim the number of database connections back to the low boundary. These high and low boundaries are controlled by the following blasadmin settings:
Note: The sum of MaxJobExecutionConnections, MaxGeneralConnections and MaxClient Connections should never exceed the connection limit specified by the database server.
MinJobExecutionConnections
The minimum number of job-related database connections that are open at any given time.
Default Value: 0
Recommended Value: 0
blasadmin command to display current value: show database MinJobExecutionConnections
blasadmin command to set new value: set database MinJobExecutionConnections
MaxJobExecutionConnections
The maximum number of job related database connections that are open at any given time.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show database MaxJobExecutionConnections
blasadmin command to set new value: set database MaxJobExecutionConnections
MinGeneralConnections
The minimum number of general, non-job related database connections that are open at any given time. An example of a general connection would be the Application Server polling the DB for jobs which require execution.
Default Value: 0
Recommended Value: 0
blasadmin command to display current value: show database MinGeneralConnections
blasadmin command to set new value: set database MinGeneralConnections
MaxGeneralConnections
The maximum number of general, non-job related database connections that are open at any given time. An example of a general connection would be the Application Server polling the DB for jobs which require execution.
Default Value:20
Recommended Value: 20
blasadmin command to display current value: show database MaxGeneralConnections
blasadmin command to set new value: set database MaxGeneralConnections
MinClientConnections
The minimum number of client related database connections that are open at any given time. An example of a client connection would be viewing job results in the CM GUI.
Default Value: 0
Recommended Value: 0
blasadmin command to display current value: show database MinClientConnections
blasadmin command to set new value: set database MinClientConnections
MaxClientConnections
The maximum number of client related database connections that are open at any given time. An example of a client connection would be viewing job results in the CM GUI.
Default Value: 20
Recommended Value: 20
blasadmin command to display current value: show database MaxClientConnections
blasadmin command to set new value: set database MaxClientConnections
TransactionAttempts
The maximum retry attempts for deadlocked transactions.
Default Value: 10
Recommended Value: 100
blasadmin command to display current value: show database TransactionAttempts
blasadmin command to set new value: set database TransactionAttempts
[edit] FetchSize
The JDBC fetch size gives the JDBC driver a hint as to the number of rows that should be fetched from the database when more rows are needed. For large queries that return a large number of objects this can improve performance by decreasing the number database hits required to satisfy the selection criteria.
Default Value: 100
Recommended Value: 100
blasadmin command to display current value: show database FetchSize
blasadmin command to set new value: set database FetchSize
Defining Multiple Application Servers on the Same Host
As of version 7.4, it is possible to define multiple application servers on a single host. This is best done when an application server is under-utilized from a memory perspective.
By default, only one application server will run on a single host. However, by configuring a AppServerProfiles.xml file in the
Please refer to the BladeLogic Administration guide for details on how to configure multiple application servers on a single host.
Deciding to Add Multiple Application Servers
Typically you would want to configure multiple application servers on a single host if the application server has 4 GB or greater of memory. The primary reason why is to use more than 4GB of memory and for better thread management. For instance, defining a config-only app server allows you to make the GUIs much more responsive even under heavy load on the appserver, because the JVM doesn’t share threads with the job threads.
Recommended AppServerProfiles.xml
The following AppServerProfiles.xml file will allow for one configuration server and two job servers. Depending on the amount of memory available on your application server, experiment with adding or removing Job servers to this file. You should be able to create this file in the
\
\
Application Server Monitoring and Maintenance
This section will cover the options and recommended processes for monitoring the application server\’s health and responsiveness.
Checking Application Server Availability
Confirm Open SRP Port
The first step in confirming Application Server availability is to confirm that the SRP port can be reached. Telnet can be used to probe the SRP port on the Application Server. By default, this port is 9829 but is configurable by modifying the SRPPort setting using the Application Server Administration console.
C:\>telnet supwin2k3serv1 9829
If telnet can successfully establish a connection to the Application Server\’s SRP port the following output will be displayed:
BladeLogic A/S v1.0 (BLManager Version: 7.x.x.xxx)
If telnet fails to establish a connection to the Application Server\’s SRP port the following output will be displayed:
Connecting To supwin2k3serv1...Could not open connection to the host, on port 9829: Connect failed
Confirm Application Server Connectivity
The next step in monitoring the availability and responsiveness of the Application Server is to confirm that it is capable of accepting and processing client connections. A simple technique to confirm this is to invoke a basic BLCLI (Bladelogic Command Line Interface) command and confirm that the Application Server returns the expected output. The "RBACUser isUserExists" BLCLI command is a good candidate for this task. This command checks to see if a user exists in the system. It returns a simple boolean value which can be checked programmatically. This test can be automated and run on a scheduled basis using OS specific tools such as crontab on a Unix OS.
Examples: C:\Program Files\BladeLogic\OM\br>blcli RBACUser isUserExists "RBACAdmin" true C:\Program Files\BladeLogic\OM\br>blcli RBACUser isUserExists "UserA" false
In both of these examples, the Application Server successfully received and processed the client request and returned the expected value.
The following is an example of expected output from this command if the Application Server was not running or was unreachable:
C:\Program Files\BladeLogic\OM\br>blcli RBACUser isUserExists "RBACAdmin" Failed to connect to server. com.bladelogic.mfw.util.BlException: Failed to create socket for host \’supwin2k3serv1\’ and port \’9829\’
If the Application Server was alive but had no Worker Threads available, this command would hang until one became available to process the request.
If the Application Server was alive, had available Worker Threads but had exceeded the value of MaxContexts (defined earlier) the expected output would be:
C:\Program Files\BladeLogic\OM\br>blcli RBACUser isUserExists "RBACAdmin" Failed to connect to server. com.bladelogic.mfw.util.BlException: Unexpected welcome message : connection limit exceeded
For more information on running BLCLI commands please see the BLCLI help.
Bladelogic Best Practice
Create a script which probes the Application Server SRP port and runs a simple CLI command. The script should be capable of detecting unexpected output and alerting the Bladelogic Administrator. A suggested interval for running this script would be hourly.
Obtaining a detailed Application Server health report
The previous section used the "RBACUser isUserExists" BLCLI command to determine if the Application Server was alive, reachable and capable of processing a simple client request. This is useful for performing automated checks on Application Server availability but does not provide us with any detailed information regarding the current state of the Application Server e.g. Number of jobs currently running, number of DB connections currently in use, amount of memory currently being consumed, number of open client connections etc. The BLCLI contains a set of commands which are designed to provide a detailed report on the current state of the Application Server. The output of these commands provide crucial information for solving and debugging errors.
AppServerProfiler : getAppServerDetails
This command returns a general status report for the Application Server. The status report contains information such as database connections, memory usage, application server version, etc.
AppServerProfiler : getThreadDump
This command returns the Java thread dump of the Application Server. This shows the live status of the Application Server thread dump and exactly what each individual thread is currently doing. This information can be critical to Bladelogic support in debugging certain error conditions. This command should be run twice - five minutes apart – so that thread progress can be gauged.
DBManager : getDBManagerStatus
This command shows the status of different database-related systems on the Application Server, including the number of available database connections, the number of used connections, the number of threads waiting to get a connection, and so forth.
JobManagement : getJobManagerFullStatus
This command provides detailed status information about all the job-related threads running on the Application Server. It also provides information about all queued and running jobs. This type of information is often used to troubleshoot jobs that are not completing successfully
JobManagement : getRunningJobsStatus
This command shows what jobs are running on the Application Server. It provides each job\’s current status and other detailed information about each job.
New in 7.3:
Profiler : getCompleteStatusReport
This command is only available as an unreleased command. It gathers all the above information in one shot for all the available application servers.
It is the recommended choice for gathering the application server health status.
On Windows, use:
blcli2 -Dcom.bladelogic.cli.debug.release-only="false" Profiler getCompleteStatusReport
On Unix/Linux, use:
blcli -Dcom.bladelogic.cli.debug.release-only="false" Profiler getCompleteStatusReport
Bladelogic Best Practice
Create a script which runs these Application Server diagnostic commands on a nightly basis. These results should be saved so that a recent Application Server history is available for Bladelogic support if required.
[edit] Monitoring the Bladelogic File Server
The Application Server uses the file server to store the contents of files included in snapshots, Network Shell scripts, BLPackages, software packages, and other types of information that is not easily stored in the database. In addition, the file server stores registry, COM+, and metabase values longer than 255 characters.A file server should meet the following requirements:
* An RSCD agent must be installed and licensed.
* A file server must have, as a minimum, 72 GB of available, non-redundant, disk space. BladeLogic recommends that the file server have 200 GB or more of available RAID 5 disk space.
* A user name must be defined on the file server, and all Configuration Manager users must be mapped to that user. Without this mapping a user may not be able to access a file that another user has stored on the file server. One way to accomplish the necessary mapping is to create an entry like the following in the exports file on the file server:
* The internal System:System role/user must be mapped to the user name defined on the file server. To accomplish the mapping, create an entry like the following in the users.local file on the file server: System:System rw,user=
You can determine the current file server hostname and storage location using the Application Server Administration console (blasadmin):
C:\Program Files\BladeLogic\OM\br>blasadmin show fileserver all \[FileServer\] name:fs01.sup.bladelogic.com location:/support/storage/
The Bladelogic Administrator should frequently monitor this fileserver storage location (using a command such as "df –h") to ensure it is not running low on available disk space.
Bladelogic Best Practice
Create a script which uses the df command to check the amount of available file server space on a weekly basis and notifies the Application Server Administrator of the results.
Application Server Logging
Overview
The Bladelogic application uses the industry standard log4j package for logging. Log4j is a flexible and efficient logging system.
The log4j system is configured through the log4j configuration file called "appserver.cf". The "appserver.cf" is located under the application server installation directory and follows the format defined by the log4j specifications.
The output of the application server logging goes by default to a file called "appserver.log" which is located under the application server installation directory.
In addition to the "appserver.log" file, an Application Server running on Linux or Solaris can be configured to write all the standard output and standard error information into a file called "console.log". The "console.log" file will contain the same information as the "appserver.log" file but in addition it will also capture any output that does not go through the log4j logging system. This may include information like the java thread dump and any messages generated by third party code used by the application server that logs messages to standard out/err.
Note: More information about log4j can be found here: [1]
Configuring
The "appserver.cf" file can be used to control the amount of logging information available in the log files. This section will explain how to manipulate the configuration file in order to control what logging information gets generated by the application server.
In addition to controlling the logging information, the configuration file can also control where the logging information is stored and how the log files are managed. This information will be covered in the next section.
The log4j logging system used by the application server is a hierarchical tree of loggers. A logger is an entity that can receive and log messages generated by the application. Each application module uses one or more loggers for logging messages. Each logger has a logging level which is used for filtering messages. A logger\’s logging level can be set directly on a logger or implicitly by the setting the level on the parent logger.
For detailed understanding of the logging system it is recommended to read the log4j documentation.
Log Levels
Each message generated by the application has a level associated with it. The level can be "DEBUG", INFO", "WARN", "ERROR". A message will be written to the log file if the message log level is equal or higher then the logger\’s log level.
For example: Application module 1 uses the logger called "com.bladelogic.module1". The logger level is set to INFO. This means that when the application logs a new message with the level DEBUG, the message will be filtered out and will not show up in the log file.
If the application generates a message with the level INFO, WARN or ERROR, the message will show up in the log file.
Adding Debug Information
The "appserver.cf" file contains a large list of loggers that can be configured to add useful debug information. These loggers are initially configured with the log level INFO to prevent the log files from containing too much information. When debugging specific issues with the system, the user can open the file and modify one or more of the specified loggers and set the log level to DEBUG. Debugging issues with the application server will often be assisted by the Bladelogic support team.
The exact list of loggers that can be modified is quite large. Here are a few examples for loggers that are useful for debugging:
* log4j.logger.com.bladelogic.app.db.DBServiceImpl – This logger controls messages generated by the database service.
* log4j.logger.com.bladelogic.mfw.demux.Demux – This logger controls messages generated by the networking layer.
Enabling More Detailed Logging
The console.log file is useful for debugging when certain output is not captured by the regular log files.
The main use case for the console.log file is for getting the java thread dump in rare cases where the getThreadDump BLCLI command cannot be run.
The problem with the console.log file is that the file size is not limited. As opposed to the log4j log files, the file will continue to grow indefinitely. In general, it is recommended that the usage of the console.log file be limited to periods in which specific errors are being debugged.
Log Rotation
Log rotation is full supported by log4j by configuring the
Managing Log Files
Log4j Log Files
The following log attributes can be controlled by modifying options in the "appserver.cf" file; log files location, log files max size, number of roll-over log files.
Log Files Location The log file location can be controlled by modifying the property "log4j.appender.R.File" with the relative or full path of the log file. This will instruct the log4j system to use the specified path for logging.
Max File Size The maximum file size for the log file can be controlled by setting the value of the property "log4j.appender.R.MaxFileSize". By default, the max log file size is 5000KB.
Number of Roll-Over Files The number of roll-over files can be controlled by setting the value of the property "log4j.appender.R.MaxBackupIndex". When a log file reaches its maximum size, a backup file will be made and a new log file will be created. The property value controls how many backup files to keep. By default, the number of backups retained is 5.
Console.log File
The "console.log" file is a file that must be maintained manually by the user. By default, the file does not exist on the application server. The file has to be created by the user when debugging a problem and then removed by the user. If the file is left in place it can grow and consume an unlimited amount of disk space.
Bladelogic Best Practice
Unless instructed otherwise by Bladelogic Support, the default logging configuration is recommended for normal operation. Bladelogic Support may ask the Application Server Administrator to enable DEBUG logging for a single logger when debugging a particular issue. This change will typically be backed out once the requested DEBUG information has been gathered. Likewise, a console.log should only be created on Solaris or Linux application servers when requested by Bladelogic Support.
Database Configuration
The following sections lists some Bladelogic recommendations around configuring the Oracle or SQL Server database instances used by the Bladelogic Application server.
Oracle DB Configuration
When configuring init.ora in Oracle, you must set the following initialization parameters:
* db_block_size should be set to at least 8192.
* The processes parameter should be set to at least 200. Ideally, the number of processes should be 30 to 40 higher than the total number of job-related and non-job-related database connections configured for the Application Server. If you are running BladeLogic Reports, the number of processes should ideally be 110 to 120 higher than the total number of job-related and non-job-related database connections.
* session_cached_cursors 2048
* open_cusors 2096
* The following recommendations are for a 10 GB database. Scale the configuration recommendations up or down based on the relative size of your database.
o db_cache_size = 1 GB
o log_buffer = 512 KB
o pga_aggregate_target = 1GB
o sga_target = 2GB
o undo tablespace size = 2 GB
SQL Server DB Configuration
For advice on SQL Server DB Configuration, please see the following Microsoft KB article: http://support.microsoft.com/kb/319942
Database Monitoring and Maintenance
Bladelogic recommends the following DBA tasks be performed regularly on any database instance which contains a Bladelogic CM or Bladelogic Reports database.
Recommended Oracle DB Maintenance
Gathering schema statistics
BladeLogic recommends you gather statistics for BladeLogic Reports and the core BladeLogic database on at least a weekly basis. Use the following command to gather statistics: exec dbms_stats.gather_schema_stats( ownname => USER, cascade=>TRUE, options => \’GATHER\’, estimate_percent => dbms_stats.auto_sample_size);
Replace USER with the correct user name e.g. bladelogic, blreports.
SQL Server DB Maintenance
Rebuilding SQL Server DB Indexes.
Bladelogic recommends that you rebuild the indexes in both the core Bladelogic and Reports warehouse DBs on a weekly basis using the \’DBCC DBREINDEX" command. (TODO: confirm with dbas that weekly is sufficient) This command only accepts one table name as an argument so the following script can be used in SQL Server Enterprise Manager to loop through all the tables in each database and run the DBREINDEX command against them:
USE DatabaseName --Enter the name of the database you want to reindex
DECLARE @TableName varchar(255)
DECLARE TableCursor CURSOR FOR SELECT table_name FROM information_schema.tables WHERE table_type = \’base table\’
OPEN TableCursor
FETCH NEXT FROM TableCursor INTO @TableName WHILE @@FETCH_STATUS = 0 BEGIN DBCC DBREINDEX(@TableName) FETCH NEXT FROM TableCursor INTO @TableName END
CLOSE TableCursor
DEALLOCATE TableCursor
Bladelogic Best Practice
Run the recommend gather_schema_stats (Oracle) or DBREINDEX (SQL Server) commands against the Bladelogic CM and Reports databases on a weekly basis. These commands should immediately be run against the Reports DB after the initial run of the populate reports job.
Bladelogic Housekeeping Jobs
The following CM jobs should be run at the suggested frequency to keep all CM data up to date.
Update Server Properties
The Update Server Properties Job lets you automatically update intrinsic server property values for a list of targeted servers. Running this job ensures that Configuration Manager reflects the current values for all the intrinsic properties on your servers. Intrinsic properties are properties that are derived from the nature and configuration of a server, such as the server\’s name, root directory, operating system, and so forth. For more information on creating and running Update Server Properties jobs, see the Bladelogic Users Guide.
Bladelogic Best Practice
An Update Server Properties job should be run against all servers in the environment on at least a nightly basis. Several daily runs is recommend since the population of Smart Server Groups depends on these properties values.
ACL Push Jobs
An ACL Push Job uses the authorizations specified for all roles granted access to a server and generates entries in an access control list for that server. The ACL Push Job then copies that ACL to the server\’s agent. On the agent this ACL is called the users configuration file. For more information on creating and running ACL Push jobs, see the Bladelogic Users Guide.
[edit] Bladelogic Best Practice
An ACL Push job should be run against all servers in the environment on at least a nightly basis. An ACL push job should also be run when a user or role is deleted from CM or when a new user or role is added.
Component Discovery Job
A Component Discovery Job associates one or more components with a server. Discovering a component merely determines that a server satisfies the signature of a component template. It does not ensure that all component template parts actually exist on a server.
Bladelogic Best Practice
A Component Discovery Job should be created which discovers all components on all servers in the environment and should be configured to run nightly. This helps to avoid errors during Compliance Job runs due to the version of the component associated with the target server being out of synch with the latest version of the component template. It is also a good idea to run a Component Discovery job immediately before a Compliance Job as part of a single Batch Job. For more information on creating and running ACL Push jobs, see the Bladelogic Users Guide.
OS Config
The OS Config NSH Script job collects data for inventory reports. When you create and run a Network Shell Script Job based on os_config.nsh, the script creates one XML file (known as a server data file) for every server you specify in the job. The server data file contains configuration information about that server. The script copies the server data file to a directory on the reports server. This data is then processed by Populate Reports so it should be run before the Populate Reports job runs.
Bladelogic Best Practice
BladeLogic recommends running this job no more than once a week per server. Ideally, this job should run during off peak hours.
Collect Agent Logs
The Collect Agent Logs NSH Script Job collects data for Server Usage reports. It copies data from agent logs on the target servers you specify to files on the reports server. This data is then processed by Populate Reports so it should be run before the Populate Reports job runs.
Bladelogic Best Practice
The Collect Agent Logs NSH Script Job should only be run if you are interested in Server Usage Reports. If you are not interested in these particular reports there is no need to run the Collect Agent Logs job. Not running this job will result in Populate Reports running to completion quicker and also less space
Populate Reports
The Populate Reports NSH Script Job performs all data aggregation and populates the reporting data warehouse. For more information on setting up this job, see the Bladelogic Installation Guide.
[edit] Bladelogic Best Practice
To ensure that the latest CM data is available in Bladelogic Reports, the Populate Reports job should be run against the Reports Server on a nightly basis.
Tuesday, June 10, 2008
Delete from database
6.3.3
select device_id from device where name='00-11-43-E7-35-4A'
select * from device_physical_attributes where device_id=165
delete from device_physical_attributes where device_id=165
or
select * from device_physical_attributes where mac_address='00-14-22-11-1C-28'
delete from device_physical_attributes where device_id=165
7.3.0
select * from pm_device where mac_address = '00-11-43-E7-8B-A3'
or
select * from device where name like '<server name>'
delete from device where device_id = 189 (or whatever the number is)
delete from pm_device where pm_device_id = 189
delete from pm_device_audit_trail where pm_device_id = 189
Monday, February 4, 2008
Autolic configuration with proxy server
When a proxy server is being used to access the internet you must configure the autolic.conf found here:
/OM/shared/autolic.conf
Subscribe to:
Comments (Atom)