Understand Performance Objectives:
Gather information about the level of activity expected on your server, the anticipated number of users, the number of requests, acceptable response time, and an optimal hardware configuration (e.g., fast CPU, disk size vs. speed, sufficient memory, and so on.).
There is no single formula for determining your hardware requirements. The process of determining what type of hardware and software configuration is required to meet application needs adequately is called capacity planning. Capacity planning requires assessment of your system performance goals and an understanding of your application. Capacity planning for server hardware should focus on maximum performance requirements
Identify the Best JVM Settings
Tune your JVM's heap garbage collection and heap size parameters to get the best performance out of your JVM. The Sun HotSpot and WebLogic JRockit JVM parameters that most significantly affect performance are listed below.
Sun JDK
When using the HotSpot VM option (-server or -client), experiment with the following garbage collection parameters:
When using JRockit's JVM, experiment with the following garbage collection parameters:
The JVM heap size determines how often and how long the VM spends collecting garbage. An acceptable rate for garbage collection is application-specific and should be adjusted after analyzing the actual time and frequency of garbage collections. If you set a large heap size, full garbage collection is slower, but it occurs less frequently. If you set your heap size in accordance with your memory needs, full garbage collection is faster, but occurs more frequently.
The goal of tuning your heap size is to minimize the time that your JVM spends doing garbage collection while maximizing the number of clients that WebLogic Server can handle at a given time. To ensure maximum performance during benchmarking, you might set high heap size values to ensure that garbage collection does not occur during the entire run of the benchmark.
Choosing a Garbage Collection Scheme
Depending on which JVM you are using, you can choose from several garbage collection schemes to manage your system memory. For example, some garbage collection schemes are more appropriate for a given type of application. Once you have an understanding of the workload of the application and the different garbage collection algorithms utilized by the JVM, you can optimize the configuration of the garbage collection.
Specifying Heap Size Values
You must specify Java heap size values each time you start an instance of WebLogic Server. This can be done either from the java command line or by modifying the default values in the sample startup scripts that are provided with the WebLogic distribution for starting WebLogic Server.
For example, when you start a WebLogic Server instance from a java command line, you could specify the HotSpot VM heap size values as follows:
color:black;mso-ansi-language:EN">
$ java -XX:NewSize=128m -XX:MaxNewSize=128m -XX:SurvivorRatio=8 -Xms512m -Xmx512m
The default size for these values is measured in bytes. Append the letter `k' or `K' to the value to indicate kilobytes, `m' or `M' to indicate megabytes, and `g' or `G' to indicate gigabytes. The example above allocates 128 megabytes of memory to the New generation and maximum New generation heap sizes, and 512 megabytes of memory to the minimum and maximum heap sizes for the WebLogic Server instance running in the JVM.
You achieve best performance by individually tuning each application. However, configuring the Java HotSpot VM heap size options listed in table below when starting WebLogic Server increases performance for most applications.
These options may differ depending on your architecture and operating system. See your vendor's documentation for platform-specific JVM tuning options.
Task
Option
Recommended Value
Setting the New generation heap size
-XX:NewSize
Set this value to a multiple of 1024 that is greater than 1MB. As a general rule, set -XX:NewSize to be one-fourth the size of the maximum heap size. Increase the value of this option for larger numbers of short-lived objects.
Be sure to increase the New generation as you increase the number of processors. Memory allocation can be parallel, but garbage collection is not parallel.
Setting the maximum New generation heap size
-XX:MaxNewSize
Set this value to a multiple of 1024 that is greater than 1MB.
Setting New heap size ratios
-XX:SurvivorRatio
The New generation area is divided into three sub-areas: Eden, and two survivor spaces that are equal in size.
Configure the ratio of the Eden/survivor space size. Try setting this value to 8, and then monitor your garbage collection.
Setting minimum heap size
-Xms
Set the minimum size of the memory allocation pool. Set this value to a multiple of 1024 that is greater than 1MB. As a general rule, set minimum heap size (-Xms) equal to the maximum heap size (-Xmx) to minimize garbage collections.
Setting maximum heap size
-Xmx
Set the maximum size of the memory allocation pool. Set this value to a multiple of 1024 that is greater than 1MB.
Setting Java Parameters for Starting WebLogic Server
Java parameters must be specified whenever you start WebLogic Server. For simple invocations, this can be done from the command line with the weblogic.Server command. However, because the arguments needed to start WebLogic Server from the command line can be lengthy and prone to error, BEA recommends that you incorporate the command into a script. To simply this process, you can modify the default values in the sample scripts that are provided with the WebLogic distribution to start WebLogic Server as follows:
Specifying Java Options for a WebLogic Server Startup Script
If you use a WebLogic Server script to start servers, do the following:
If you use the Node Manager to start Managed Servers, do the following for each server:
Weblogic Parameters:
Setting Performance-Related Configuration Parameters
The WebLogic Server configuration file (config.xml) contains a number of performance-related parameters that can be fine-tuned depending on your environment and applications. Tuning these parameters based on your system requirements (rather than running with default settings) can greatly improve both single-node performance and the scalability characteristics of an application.
Within a WebLogic Server domain, the configuration file is located on the machine that hosts the Administration Server, and provides persistent storage of WebLogic MBean attribute values. The Administration Server serves as a central point of contact for server instances and system administration tools. A domain may also include additional WebLogic Server instances called Managed Servers, which are used mainly for servicing applications.
When the Administration Server starts, it reads the domain configuration file and overrides the default attribute values of the administration MBeans with any attribute values found in the configuration file. Every time you change an attribute using the system administration tools (using either the command-line interface or the Administration Console), its value is stored in the appropriate administration MBean and written to the configuration file.
Performance-Related config.xml Elements
Element
Attributes
Console Field
For information
Server
NativeIOEnabled
Native IO Enabled
ExecuteQueue
ThreadCount
Thread Count
ExecuteQueue
QueueLength
QueueLengthThresholdPercent
ThreadsIncrease
ThreadsMaximum
Thread Priority
Queue Length
Queue Length Threshold Percent
Threads Increase
Threads Maximum
Thread Priority
Server
StuckThreadMaxTime
StuckThreadTimerInterval
Stuck Thread Max Time
Stuck Thread Timer Interval
Server
ThreadPoolPercentSocketReaders
Socket Readers
Server
AcceptBacklog
Accept Backlog
JDBCConnectionPool
InitialCapacity
MaxCapacity
Initial Capacity
Max Capacity
JDBCConnectionPool
StatementCacheSize
Statement Cache Size
The defaults values for Development and Production environments are as follows:
Tuning Parameter
Development Mode Default
Production Mode Default
Execute Queue: ThreadCount
15 threads
25 threads
JDBC Connection Pool: MaxCapacity
15 connections
25 connections
The above mentioned parameters are described in more detail below:
Using WebLogic Server "Native IO" Performance Packs
Benchmarks show major performance improvements when you use native performance packs on machines that host WebLogic Server instances. Performance packs use a platform-optimized, native socket multiplexor to improve server performance. For example, the native socket reader multiplexor threads have their own execute queue and do not borrow threads from the default execute queue, which frees up default execute threads to do application work.
However, if you must use the pure-Java socket reader implementation for host machines, you can still improve the performance of socket communication by configuring the proper number of socket reader threads for each server instance and client machine.
Enabling Performance Packs
The use of native performance packs are enabled by default in the config.xml shipped with your distribution. To verify this setting in your configuration file, check that the NativeIOEnabled attribute of the Server element is set to "true" (NativeIOEnabled=true).
You can also use the Administration Console to verify that performance packs are enabled:
The WebLogic Server configuration file (config.xml) contains a number of OOTB (out-of-the-box) performance-related parameters that can be fine-tuned depending on your environment and applications. Tuning these parameters based on your system requirements (rather than running with default settings) can greatly improve both single-node performance and the scalability characteristics of an application.
Setting Thread Count:
The value of the ThreadCount attribute of an ExecuteQueue element in the config.xml file equals the number of simultaneous operations that can be performed by applications that use the execute queue. As work enters an instance of WebLogic Server, it is placed in an execute queue. This work is then assigned to a thread that does the work on it. Threads consume resources, so handle this attribute with care—you can degrade performance by increasing the value unnecessarily.
By default, a new WebLogic Server instance is configured with a development mode execute queue, weblogic.kernel.default, that contains 15 threads. In addition, WebLogic Server provides two other pre-configured queues:
Note: If native performance packs are not being used for your platform, you may need to tune the default number of execute queue threads and the percentage of threads that act as socket readers to achieve optimal performance.
Scenarios for Modifying the Default Thread Count
To determine the ideal thread count for an execute queue, monitor the queue's throughput while all applications in the queue are operating at maximum load. Increase the number of threads in the queue and repeat the load test until you reach the optimal throughput for the queue. (At some point, increasing the number of threads will lead to enough context switching that the throughput for the queue begins to decrease.).
When...
Results
Do This:
Thread Count < number of CPUs
Your thread count is too low if:
§ CPU is waiting to do work, but there is work that could be done.
§ Cannot get 100 percent CPU utilization rate.
Increase the thread count.
Thread Count = number of CPUs
Theoretically ideal, but the CPUs are still under-utilized.
Increase the thread count.
Thread Count > number of CPUs (by a moderate number of threads)
Practically ideal, with a moderate amount of context switching and a high CPU utilization rate.
Tune the moderate number of threads and compare performance results.
Thread Count > number of CPUs (by a large number of threads)
Too much context switching, which can lead to significant performance degradation.
Your performance may increase as you decrease the number of threads.
Reduce the number of threads so that it equals the number of CPUs, and then add only the number of "stuck" threads that you have determined.
For example, if you have four processors, then four threads can be running concurrently with the number of stuck threads. So, you want the execute threads to be 4 + the number of stuck threads.
Note: This recommendation is highly application-dependent. For instance, the length of time the application might block threads can invalidate the formula.
Note:
Performance will be degraded by increasing the value unnecessarily. By default, a new WebLogic Server instance is configured with a default execute queue (named “default”) that contains 15 threads.
How to modify the Default Thread Count
To modify the default execute queue thread count using the Administration Console:
Assigning Applications to Execute Queues
Although the default execute queue can be configured to supply the optimal number of threads for all WebLogic Server applications, configuring multiple execute queues can provide additional control for key applications. By using multiple execute queues, we can guarantee that selected applications have access to a fixed number of execute threads, regardless of the load on WebLogic Server.
Default WebLogic Server installations are configured with a default execute queue which is used by all applications running on the server instance. You may want to configure additional queues to:
* Optimize the performance of critical applications.
* Throttle the performance of nonessential applications
* Remedy deadlocked thread usage
To use user-defined execute queues in a WebLogic Server 9.0 domain, you need to include the use81-style-execute-queues sub-element of the server element in the config.xml file and reboot the server.
Allocating threads to act as Socket Reader
To set the maximum percentage of execute threads that read messages from a socket, the attribute ThreadPoolPercentSocketReaders should be used in the config.xml file. The optimal value for this attribute is application-specific. The default is value 33, and the valid range is 1-99. Allocating execute threads to act as socket reader threads increases the speed and the ability of the server to accept client requests.
Allocating execute threads to act as socket reader threads increases the speed and the ability of the server to accept client requests. It is essential to balance the number of execute threads that are devoted to reading messages from a socket and those threads that perform the actual execution of tasks in the server. . When the native performance packs are not being used, execute threads must be allocated to act as socket reader threads.
How to set Number of Socket Reader Threads for a Server Instance
To use the Administration Console to set the maximum percentage of execute threads that read messages from a socket:
On client machines, you can configure the number of available socket reader threads in the JVM that runs the client. Specify the socket readers by defining the following parameters in the java command line for the client:
-Dweblogic.ThreadPoolSize=value
-Dweblogic.ThreadPoolPercentSocketReaders=value
Tuning Execute Queues for Overflow Conditions
You can configure WebLogic Server to detect and optionally address potential overflow conditions in the default execute queue or any user-defined execute queue. WebLogic Server considers a queue to have a possible overflow condition when its current size reaches a user-defined percentage of its maximum size. When this threshold is reached, the server changes its health state to "warning" and can optionally allocate additional threads to perform the outstanding work in the queue, thereby reducing its size.
To automatically detect and address overflow conditions in a queue, you configure the following items:
Tuning the Execute Thread Detection Behavior
WebLogic Server automatically detects when a thread in an execute queue becomes "stuck." Because a stuck thread cannot complete its current work or accept new work, the server logs a message each time it diagnoses a stuck thread. If all threads in an execute queue become stuck, the server changes its health state to either "warning" or "critical" depending on the execute queue:
To configure WebLogic Server stuck thread detection behavior:
Use the AcceptBacklog attribute of the Server element in the config.xml file to set the number of connection requests the WebLogic Server instance will accept before refusing additional requests. The AcceptBacklog attribute specifies how many Transmission Control Protocol (TCP) connections can be buffered in a wait queue. This fixed-size queue is populated with requests for connections that the TCP stack has received, but the application has not accepted yet. The default value is 50 and the maximum value is operating system dependent.
To tune the Accept Backlog value from the Administration Console:
How JDBC Connection Pools Enhance Performance
Establishing a JDBC connection with a DBMS can be very slow. If your application requires database connections that are repeatedly opened and closed, this can become a significant performance issue. WebLogic connection pools offer an efficient solution to the problem.
When WebLogic Server starts, connections from the connection pools are opened and are available to all clients. When a client closes a connection from a connection pool, the connection is returned to the pool and becomes available for other clients; the connection itself is not closed. There is little cost to opening and closing pool connections.
How many connections should you create in the pool? A connection pool can grow and shrink according to configured parameters, between a minimum and a maximum number of connections. The best performance occurs when the connection pool has as many connections as there are concurrent client sessions.
Tuning JDBC Connection Pool Initial Capacity
The InitialCapacity attribute of the JDBCConnectionPool element enables you to set the number of physical database connections to create when configuring the pool. If the server cannot create this number of connections, the creation of this connection pool will fail.
During development, it may be convenient to set the value of the InitialCapacity attribute to a low number to help the server start up faster. In production systems, consider setting the InitialCapacity value equal to the MaxCapacity attribute's default production mode setting of 25. This way, all database connections are acquired during server start-up. And if you need to tune the MaxCapacity value, make sure to set the InitialCapacity so that it equals the MaxCapacity value.
If InitialCapacity is less than MaxCapacity, the server needs to create additional database connections when its load is increased. When the server is under load, all resources should be working to complete requests as fast as possible, rather than creating new database connections.
Tuning JDBC Connection Pool Maximum Capacity
The MaxCapacity attribute of the JDBCConnectionPool element allows you to set the maximum number of physical database connections that a connection pool can contain. Different JDBC drivers and database servers might limit the number of possible physical connections.
The default settings for development and production mode are equal to the default number of execute threads: 15 for development mode; 25 for production mode. However, in production, it is advisable that the number of connections in the pool equal the number of concurrent client sessions that require JDBC connections. The pool capacity is independent of the number of execute threads in the server. There may be many more ongoing user sessions than there are execute threads.
Caching Prepared and Callable Statements
Increasing Performance with the Statement Cache
When you use a prepared statement or callable statement in an application or EJB, there is considerable processing overhead for the communication between the application server and the database server and on the database server itself. To minimize the processing costs, WebLogic Server can cache prepared and callable statements used in your applications. When an application or EJB calls any of the statements stored in the cache, WebLogic Server reuses the statement stored in the cache. Reusing prepared and callable statements reduces CPU usage on the database server, improving performance for the current statement and leaving CPU cycles for other tasks.
Each connection in a connection pool has its own individual cache of prepared and callable statements used on the connection. However, you configure statement cache options per connection pool. That is, the statement cache for each connection in a connection pool uses the statement cache options specified for the connection pool. Statement cache configuration options include:
Usage Restrictions for the Statement Cache
Using the statement cache can dramatically increase performance, but you must consider its limitations before you decide to use it. Please note the following restrictions when using the statement cache.
There may be other issues related to caching statements that are not listed here. If you see errors in your system related to prepared or callable statements, you should set the statement cache size to 0, which turns off statement caching, to test if the problem is caused by caching prepared statements.
Setting Your Java Compiler
The standard Java compiler for compiling JSP servlets is javac. You can improve performance significantly by setting your server's java compiler to sj or jikes instead of javac. The following sections discuss this procedure and other compiler considerations.
Changing Compilers in the Administration Console
To change your compiler in the Administration Console:
In the weblogic.xml file, the jsp-descriptor element defines parameter names and values for servlet JSPs.
Use the weblogic.appc utility to compile EJB 2.0 and 1.1 container classes. If you compile Jar files for deployment into the EJB container, you must use weblogic.appc to generate the container classes. By default, ejbc uses the javac compiler. For faster performance, specify a different compiler (such as Symantec sj) using the -compiler flag.
Compiling on UNIX
If you receive the following error message received when compiling JSP files on a UNIX machine:
failed: java.io.IOException: Not enough space
Try any or all of the following solutions:
set rlim_fd_cur = 1024
Several factors beyond the WebLogic server can impact performance, including hardware and OS configuration, other processes running on the system, performance of back-end database resources, network latency, and so on. You must factor in these items when conducting your own performance evaluations.
Tuning the Operating System:
Each operating system sets default tuning parameters differently. For Windows platforms, the default settings are usually sufficient. However, the UNIX and Linux operating systems usually need to be tuned appropriately.
Linux:
For better packet transfer performance, set the /sbin/ifconfig lo mtu parameter as follows:
# /sbin/ifconfig lo mtu 1500
The mtu (maximum transfer unit) parameter refers to largest number of bytes that a packet can carry over the network. If the packet size is set too low, then your network performance will decrease due to fragmented data. Tuning parameters listed below when adjusted, can enhance application performance
/sbin/ifconfig lo mtu
kernel.msgmni
kernel.sem
fs.file-max
kernel.shmmax
net.ipv4.tcp_max_syn_backlog
Network:
Network performance is affected when the supply of resources is unable to keep up with the demand for resources. Today's enterprise-level networks are very fast and are now rarely the direct cause of performance in well-designed applications.
You should also verify that you have an appropriate amount of network bandwidth available for WebLogic Server and the connections it makes to other tiers in your architecture, such as client and database connections. Therefore, it is important to continually monitor your network performance to troubleshoot potential performance bottlenecks.
Network Bandwidth:
A common definition of bandwidth is "the rate of the data communications transmission, usually measured in bits-per-second, which is the capacity of the link to send and receive communications." A machine running WebLogic Server requires enough network bandwidth to handle all WebLogic Server client connections. In the case of programmatic clients, each client JVM has a single socket to the server, and each socket requires dedicated bandwidth. A WebLogic Server instance handling programmatic clients should have 125-150 percent of the bandwidth that a similar Web server would handle. If you are handling only HTTP clients, expect a bandwidth requirement similar to a Web server serving static pages.
To find out network activity use commands such as:
# sar –n DEV 1 3
LAN Infrastructure:
Your local area network must be fast enough to handle your application's peak capacity. If your network is fully utilized, in that the amount of traffic consistently exceeds its bandwidth capacity, yet your WebLogic Server machine is not fully utilized, do one of the following:
Optimizing the Database
Your database can be a major enterprise-level bottleneck. Configure your database for optimal performance by following the tuning guidelines in this section and in the product documentation for the database you are using.
General Suggestions
Here are some general database tuning suggestions:
Conclusion
The above are only some of the various ways that the server can be tuned. Bear in mind, however, that a poorly designed, poorly written application will usually have poor performance, regardless of tuning. Performance must always be a key consideration throughout the stages of the application development life cycle - from design to deployment. It happens too often that performance takes a back seat to functionality, and problems are found later that are difficult to fix.
References:
http://docs.oracle.com/cd/E13222_01/wls/docs81/perform/index.html
http://wladocsbypani.blogspot.in/2010/01/performance-tuning-on-weblogic-server.html
Gather information about the level of activity expected on your server, the anticipated number of users, the number of requests, acceptable response time, and an optimal hardware configuration (e.g., fast CPU, disk size vs. speed, sufficient memory, and so on.).
There is no single formula for determining your hardware requirements. The process of determining what type of hardware and software configuration is required to meet application needs adequately is called capacity planning. Capacity planning requires assessment of your system performance goals and an understanding of your application. Capacity planning for server hardware should focus on maximum performance requirements
Identify the Best JVM Settings
Tune your JVM's heap garbage collection and heap size parameters to get the best performance out of your JVM. The Sun HotSpot and WebLogic JRockit JVM parameters that most significantly affect performance are listed below.
Sun JDK
When using the HotSpot VM option (-server or -client), experiment with the following garbage collection parameters:
- -Xms and -Xmx (use equal settings at start up)
- -XX:NewSize and -XX:MaxNewSize
- -XX:SurvivorRatio
- -XX:+UseISM-XX:+AggressiveHeap
When using JRockit's JVM, experiment with the following garbage collection parameters:
- -Xms and -Xmx (use equal settings at startup)
- -Xns
- -Xgc: parallel
- -XXenablefatspin
The JVM heap size determines how often and how long the VM spends collecting garbage. An acceptable rate for garbage collection is application-specific and should be adjusted after analyzing the actual time and frequency of garbage collections. If you set a large heap size, full garbage collection is slower, but it occurs less frequently. If you set your heap size in accordance with your memory needs, full garbage collection is faster, but occurs more frequently.
The goal of tuning your heap size is to minimize the time that your JVM spends doing garbage collection while maximizing the number of clients that WebLogic Server can handle at a given time. To ensure maximum performance during benchmarking, you might set high heap size values to ensure that garbage collection does not occur during the entire run of the benchmark.
Choosing a Garbage Collection Scheme
Depending on which JVM you are using, you can choose from several garbage collection schemes to manage your system memory. For example, some garbage collection schemes are more appropriate for a given type of application. Once you have an understanding of the workload of the application and the different garbage collection algorithms utilized by the JVM, you can optimize the configuration of the garbage collection.
Specifying Heap Size Values
You must specify Java heap size values each time you start an instance of WebLogic Server. This can be done either from the java command line or by modifying the default values in the sample startup scripts that are provided with the WebLogic distribution for starting WebLogic Server.
For example, when you start a WebLogic Server instance from a java command line, you could specify the HotSpot VM heap size values as follows:
color:black;mso-ansi-language:EN">
$ java -XX:NewSize=128m -XX:MaxNewSize=128m -XX:SurvivorRatio=8 -Xms512m -Xmx512m
The default size for these values is measured in bytes. Append the letter `k' or `K' to the value to indicate kilobytes, `m' or `M' to indicate megabytes, and `g' or `G' to indicate gigabytes. The example above allocates 128 megabytes of memory to the New generation and maximum New generation heap sizes, and 512 megabytes of memory to the minimum and maximum heap sizes for the WebLogic Server instance running in the JVM.
You achieve best performance by individually tuning each application. However, configuring the Java HotSpot VM heap size options listed in table below when starting WebLogic Server increases performance for most applications.
These options may differ depending on your architecture and operating system. See your vendor's documentation for platform-specific JVM tuning options.
Task
Option
Recommended Value
Setting the New generation heap size
-XX:NewSize
Set this value to a multiple of 1024 that is greater than 1MB. As a general rule, set -XX:NewSize to be one-fourth the size of the maximum heap size. Increase the value of this option for larger numbers of short-lived objects.
Be sure to increase the New generation as you increase the number of processors. Memory allocation can be parallel, but garbage collection is not parallel.
Setting the maximum New generation heap size
-XX:MaxNewSize
Set this value to a multiple of 1024 that is greater than 1MB.
Setting New heap size ratios
-XX:SurvivorRatio
The New generation area is divided into three sub-areas: Eden, and two survivor spaces that are equal in size.
Configure the ratio of the Eden/survivor space size. Try setting this value to 8, and then monitor your garbage collection.
Setting minimum heap size
-Xms
Set the minimum size of the memory allocation pool. Set this value to a multiple of 1024 that is greater than 1MB. As a general rule, set minimum heap size (-Xms) equal to the maximum heap size (-Xmx) to minimize garbage collections.
Setting maximum heap size
-Xmx
Set the maximum size of the memory allocation pool. Set this value to a multiple of 1024 that is greater than 1MB.
Setting Java Parameters for Starting WebLogic Server
Java parameters must be specified whenever you start WebLogic Server. For simple invocations, this can be done from the command line with the weblogic.Server command. However, because the arguments needed to start WebLogic Server from the command line can be lengthy and prone to error, BEA recommends that you incorporate the command into a script. To simply this process, you can modify the default values in the sample scripts that are provided with the WebLogic distribution to start WebLogic Server as follows:
Specifying Java Options for a WebLogic Server Startup Script
If you use a WebLogic Server script to start servers, do the following:
- Create a backup copy of the WebLogic Server start scripts:
- For scripts that start an Administration Server, back up domain-name\startWebLogic.cmd (startWebLogic.sh on UNIX)
- For scripts that start a Managed Server, back up domain-name\startManagedWebLogic.cmd (startManagedWebLogic.sh on UNIX)
- Open the start scripts in a text editor.
- Edit the set JAVA_OPTIONS command to specify the Java options. If you specify multiple options, separate each option by a space, and place quotes around the entire set of options. For example:
set JAVA_OPTIONS="-Xgc:gencopy -Xns:30" - Save the start script.
- Restart the server.
If you use the Node Manager to start Managed Servers, do the following for each server:
- In left pane of the Administration Console, select the server.
- In the right pane, select Configuration —> Remote Start.
- In the Arguments field, specify the Java options. If you specify multiple options, separate each option by a space.
- Click Apply.
- Restart the Managed Server.
Weblogic Parameters:
Setting Performance-Related Configuration Parameters
The WebLogic Server configuration file (config.xml) contains a number of performance-related parameters that can be fine-tuned depending on your environment and applications. Tuning these parameters based on your system requirements (rather than running with default settings) can greatly improve both single-node performance and the scalability characteristics of an application.
Within a WebLogic Server domain, the configuration file is located on the machine that hosts the Administration Server, and provides persistent storage of WebLogic MBean attribute values. The Administration Server serves as a central point of contact for server instances and system administration tools. A domain may also include additional WebLogic Server instances called Managed Servers, which are used mainly for servicing applications.
When the Administration Server starts, it reads the domain configuration file and overrides the default attribute values of the administration MBeans with any attribute values found in the configuration file. Every time you change an attribute using the system administration tools (using either the command-line interface or the Administration Console), its value is stored in the appropriate administration MBean and written to the configuration file.
Performance-Related config.xml Elements
Element
Attributes
Console Field
For information
Server
NativeIOEnabled
Native IO Enabled
ExecuteQueue
ThreadCount
Thread Count
ExecuteQueue
QueueLength
QueueLengthThresholdPercent
ThreadsIncrease
ThreadsMaximum
Thread Priority
Queue Length
Queue Length Threshold Percent
Threads Increase
Threads Maximum
Thread Priority
Server
StuckThreadMaxTime
StuckThreadTimerInterval
Stuck Thread Max Time
Stuck Thread Timer Interval
Server
ThreadPoolPercentSocketReaders
Socket Readers
Server
AcceptBacklog
Accept Backlog
JDBCConnectionPool
InitialCapacity
MaxCapacity
Initial Capacity
Max Capacity
JDBCConnectionPool
StatementCacheSize
Statement Cache Size
The defaults values for Development and Production environments are as follows:
Tuning Parameter
Development Mode Default
Production Mode Default
Execute Queue: ThreadCount
15 threads
25 threads
JDBC Connection Pool: MaxCapacity
15 connections
25 connections
The above mentioned parameters are described in more detail below:
Using WebLogic Server "Native IO" Performance Packs
Benchmarks show major performance improvements when you use native performance packs on machines that host WebLogic Server instances. Performance packs use a platform-optimized, native socket multiplexor to improve server performance. For example, the native socket reader multiplexor threads have their own execute queue and do not borrow threads from the default execute queue, which frees up default execute threads to do application work.
However, if you must use the pure-Java socket reader implementation for host machines, you can still improve the performance of socket communication by configuring the proper number of socket reader threads for each server instance and client machine.
Enabling Performance Packs
The use of native performance packs are enabled by default in the config.xml shipped with your distribution. To verify this setting in your configuration file, check that the NativeIOEnabled attribute of the Server element is set to "true" (NativeIOEnabled=true).
You can also use the Administration Console to verify that performance packs are enabled:
- Start the Administration Server if it is not already running.
- Access the Administration Console for the domain.
- Expand the Servers node in the left pane to display the servers configured in your domain.
- Click the name of the server instance that you want to configure.
- Select the Configuration —> Tuning tab.
- If the Enable Native IO check box is not selected, select the check box.
- Click Apply.
- Restart the server.
The WebLogic Server configuration file (config.xml) contains a number of OOTB (out-of-the-box) performance-related parameters that can be fine-tuned depending on your environment and applications. Tuning these parameters based on your system requirements (rather than running with default settings) can greatly improve both single-node performance and the scalability characteristics of an application.
Setting Thread Count:
The value of the ThreadCount attribute of an ExecuteQueue element in the config.xml file equals the number of simultaneous operations that can be performed by applications that use the execute queue. As work enters an instance of WebLogic Server, it is placed in an execute queue. This work is then assigned to a thread that does the work on it. Threads consume resources, so handle this attribute with care—you can degrade performance by increasing the value unnecessarily.
By default, a new WebLogic Server instance is configured with a development mode execute queue, weblogic.kernel.default, that contains 15 threads. In addition, WebLogic Server provides two other pre-configured queues:
- weblogic.admin.HTTP—Available only on Administration Servers, this queue is reserved for communicating with the Administration Console; you cannot reconfigure it.
- weblogic.admin.RMI—Both Administration Servers and Managed Servers have this queue; it is reserved for administrative traffic; you cannot reconfigure it.
Note: If native performance packs are not being used for your platform, you may need to tune the default number of execute queue threads and the percentage of threads that act as socket readers to achieve optimal performance.
Scenarios for Modifying the Default Thread Count
To determine the ideal thread count for an execute queue, monitor the queue's throughput while all applications in the queue are operating at maximum load. Increase the number of threads in the queue and repeat the load test until you reach the optimal throughput for the queue. (At some point, increasing the number of threads will lead to enough context switching that the throughput for the queue begins to decrease.).
When...
Results
Do This:
Thread Count < number of CPUs
Your thread count is too low if:
§ CPU is waiting to do work, but there is work that could be done.
§ Cannot get 100 percent CPU utilization rate.
Increase the thread count.
Thread Count = number of CPUs
Theoretically ideal, but the CPUs are still under-utilized.
Increase the thread count.
Thread Count > number of CPUs (by a moderate number of threads)
Practically ideal, with a moderate amount of context switching and a high CPU utilization rate.
Tune the moderate number of threads and compare performance results.
Thread Count > number of CPUs (by a large number of threads)
Too much context switching, which can lead to significant performance degradation.
Your performance may increase as you decrease the number of threads.
Reduce the number of threads so that it equals the number of CPUs, and then add only the number of "stuck" threads that you have determined.
For example, if you have four processors, then four threads can be running concurrently with the number of stuck threads. So, you want the execute threads to be 4 + the number of stuck threads.
Note: This recommendation is highly application-dependent. For instance, the length of time the application might block threads can invalidate the formula.
Note:
Performance will be degraded by increasing the value unnecessarily. By default, a new WebLogic Server instance is configured with a default execute queue (named “default”) that contains 15 threads.
How to modify the Default Thread Count
To modify the default execute queue thread count using the Administration Console:
- Start the Administration Server if it is not already running.
- Access the Administration Console for the domain.
- Expand the Servers node in the left pane to display the servers configured in your domain.
- Right-click the name of the server instance that contains the execute queue you want to configure, and then select View Execute Queues on the pop-up menu to display a table of execute queues that can be modified.
- In the Name column, click directly on the default execute queue name to display the Configuration tab for modifying execute queues.
- Locate the Thread Count value and increase or decrease it, as appropriate.
- Click Apply to save your changes.
- Reboot the selected server to enable the new execute queue settings.
Assigning Applications to Execute Queues
Although the default execute queue can be configured to supply the optimal number of threads for all WebLogic Server applications, configuring multiple execute queues can provide additional control for key applications. By using multiple execute queues, we can guarantee that selected applications have access to a fixed number of execute threads, regardless of the load on WebLogic Server.
Default WebLogic Server installations are configured with a default execute queue which is used by all applications running on the server instance. You may want to configure additional queues to:
* Optimize the performance of critical applications.
* Throttle the performance of nonessential applications
* Remedy deadlocked thread usage
To use user-defined execute queues in a WebLogic Server 9.0 domain, you need to include the use81-style-execute-queues sub-element of the server element in the config.xml file and reboot the server.
Allocating threads to act as Socket Reader
To set the maximum percentage of execute threads that read messages from a socket, the attribute ThreadPoolPercentSocketReaders should be used in the config.xml file. The optimal value for this attribute is application-specific. The default is value 33, and the valid range is 1-99. Allocating execute threads to act as socket reader threads increases the speed and the ability of the server to accept client requests.
Allocating execute threads to act as socket reader threads increases the speed and the ability of the server to accept client requests. It is essential to balance the number of execute threads that are devoted to reading messages from a socket and those threads that perform the actual execution of tasks in the server. . When the native performance packs are not being used, execute threads must be allocated to act as socket reader threads.
How to set Number of Socket Reader Threads for a Server Instance
To use the Administration Console to set the maximum percentage of execute threads that read messages from a socket:
- Start the Administration Server if it is not already running.
- Access the Administration Console for the domain.
- Expand the Servers node in the left pane to display the servers configured in your domain.
- Click the name of the server that you want to configure.
- Select the Configuration —> Tuning tab.
- Edit the percentage of Java reader threads in the Socket Readers attribute field. The number of Java socket readers is computed as a percentage of the number of total execute threads (as shown in the Thread Count field for the Execute Queue).
- Apply the changes.
On client machines, you can configure the number of available socket reader threads in the JVM that runs the client. Specify the socket readers by defining the following parameters in the java command line for the client:
-Dweblogic.ThreadPoolSize=value
-Dweblogic.ThreadPoolPercentSocketReaders=value
Tuning Execute Queues for Overflow Conditions
You can configure WebLogic Server to detect and optionally address potential overflow conditions in the default execute queue or any user-defined execute queue. WebLogic Server considers a queue to have a possible overflow condition when its current size reaches a user-defined percentage of its maximum size. When this threshold is reached, the server changes its health state to "warning" and can optionally allocate additional threads to perform the outstanding work in the queue, thereby reducing its size.
To automatically detect and address overflow conditions in a queue, you configure the following items:
- The threshold at which the server indicates an overflow condition. This value is set as a percentage of the configured size of the execute queue (the QueueLength value).
- The number of threads to add to the execute queue when an overflow condition is detected. These additional threads work to reduce the size of the queue to its normal operating size.
- The maximum number of threads available to the queue. In particular, setting the maximum number of threads prevents the server from assigning an overly high thread count in response to overload conditions.
- Start the Administration Server if it is not already running.
- Access the Administration Console for the domain.
- Expand the Servers node in the left pane to display the servers configured in your domain.
- Right-click the name of the server instance that contains the execute queue you want to configure, and then select View Execute Queues from the pop-up menu to display a table of execute queues that can be modified.
- In the Name column, directly click the default execute queue name (or the user-defined execute queue) that you want to configure.
- On the execute queue Configuration tab, specify how the server instance should detect an overflow condition for the selected queue by modifying the following attributes:
- Queue Length: Specifies the maximum number of simultaneous requests that the server can hold in the queue. The default of 65536 requests represents a very large number of requests; outstanding requests in the queue should rarely, if ever reach this maximum value. Always leave the Queue Length at the default value of 65536 entries.
- Queue Length Threshold Percent: The percentage (from 1-99) of the Queue Length size that can be reached before the server indicates an overflow condition for the queue. All actual queue length sizes below the threshold percentage are considered normal; sizes above the threshold percentage indicate an overflow. By default, the Queue Length Threshold Percent is set to 90 percent.
- Thread Priority: The priority of the threads associated with the queue. By default, the Thread Priority is set to 5.
- To specify how this server should address an overflow condition for the selected queue, modify the following attribute:
- Threads Increase: The number of threads WebLogic Server should add to this execute queue when it detects an overflow condition. If you specify zero threads (the default), the server changes its health state to "warning" in response to an overflow condition in the execute queue, but it does not allocate additional threads to reduce the workload.
- To limit the maximum number of threads that can be added to the selected queue, modify the following attribute:
- Threads Maximum: The maximum number of threads that this execute queue can have; this value prevents WebLogic Server from creating an overly high thread count in the queue in response to continual overflow conditions. By default, Threads Maximum is set to 400.
- Click Apply to save your changes.
- Reboot the selected server to enable the new execute queue settings.
Tuning the Execute Thread Detection Behavior
WebLogic Server automatically detects when a thread in an execute queue becomes "stuck." Because a stuck thread cannot complete its current work or accept new work, the server logs a message each time it diagnoses a stuck thread. If all threads in an execute queue become stuck, the server changes its health state to either "warning" or "critical" depending on the execute queue:
- If all threads in the default queue become stuck, the server changes its health state to "critical."
- If all threads in weblogic.admin.HTTP,weblogic.admin.RMI, or a user-defined execute queue become stuck, the server changes its health state to "warning."
To configure WebLogic Server stuck thread detection behavior:
- Start the Administration Server if it is not already running.
- Access the Administration Console for the domain.
- Expand the Servers node in the left pane to display the servers configured in your domain.
- Click the name of the server instance that you want to modify for improved stuck thread detection.
- Select the Configuration —> Tuning tab in the right pane.
- Modify the following attributes as necessary to tune thread detection behavior for the server:
- Stuck Thread Max Time: Enter the number of seconds, that a thread must be continually working before this server diagnoses the thread as being stuck. By default, WebLogic Server considers a thread to be "stuck" after 600 seconds of continuous use.
- Stuck Thread Timer Interval: Enter the number of seconds, after which WebLogic Server periodically scans threads to see if they have been continually working for the length of time specified by Stuck Thread Max Time. By default, WebLogic Server sets this interval to 600 seconds.
- Click Apply to save your changes.
- Reboot the server to use the new settings.
Use the AcceptBacklog attribute of the Server element in the config.xml file to set the number of connection requests the WebLogic Server instance will accept before refusing additional requests. The AcceptBacklog attribute specifies how many Transmission Control Protocol (TCP) connections can be buffered in a wait queue. This fixed-size queue is populated with requests for connections that the TCP stack has received, but the application has not accepted yet. The default value is 50 and the maximum value is operating system dependent.
To tune the Accept Backlog value from the Administration Console:
- Start the Administration Server if it is not already running.
- Access the Administration Console for the domain.
- Expand the Servers node in the left pane to display the servers configured in your domain.
- Click the name of the server instance that you want to configure.
- Select the Configuration —> Tuning tab.
- Modify the default Accept Backlog value as necessary to tune how many TCP connections can be buffered in a wait queue:
- During operations, if many connections are dropped or refused at the client, and no other error messages are on the server, the Accept Backlog value might be set too low.
- If you are getting "connection refused" messages when you try to access WebLogic Server, raise the Accept Backlog value from the default by 25 percent. Continue increasing the value by 25 percent until the messages cease to appear.
- Click Apply to save your changes.
How JDBC Connection Pools Enhance Performance
Establishing a JDBC connection with a DBMS can be very slow. If your application requires database connections that are repeatedly opened and closed, this can become a significant performance issue. WebLogic connection pools offer an efficient solution to the problem.
When WebLogic Server starts, connections from the connection pools are opened and are available to all clients. When a client closes a connection from a connection pool, the connection is returned to the pool and becomes available for other clients; the connection itself is not closed. There is little cost to opening and closing pool connections.
How many connections should you create in the pool? A connection pool can grow and shrink according to configured parameters, between a minimum and a maximum number of connections. The best performance occurs when the connection pool has as many connections as there are concurrent client sessions.
Tuning JDBC Connection Pool Initial Capacity
The InitialCapacity attribute of the JDBCConnectionPool element enables you to set the number of physical database connections to create when configuring the pool. If the server cannot create this number of connections, the creation of this connection pool will fail.
During development, it may be convenient to set the value of the InitialCapacity attribute to a low number to help the server start up faster. In production systems, consider setting the InitialCapacity value equal to the MaxCapacity attribute's default production mode setting of 25. This way, all database connections are acquired during server start-up. And if you need to tune the MaxCapacity value, make sure to set the InitialCapacity so that it equals the MaxCapacity value.
If InitialCapacity is less than MaxCapacity, the server needs to create additional database connections when its load is increased. When the server is under load, all resources should be working to complete requests as fast as possible, rather than creating new database connections.
Tuning JDBC Connection Pool Maximum Capacity
The MaxCapacity attribute of the JDBCConnectionPool element allows you to set the maximum number of physical database connections that a connection pool can contain. Different JDBC drivers and database servers might limit the number of possible physical connections.
The default settings for development and production mode are equal to the default number of execute threads: 15 for development mode; 25 for production mode. However, in production, it is advisable that the number of connections in the pool equal the number of concurrent client sessions that require JDBC connections. The pool capacity is independent of the number of execute threads in the server. There may be many more ongoing user sessions than there are execute threads.
Caching Prepared and Callable Statements
Increasing Performance with the Statement Cache
When you use a prepared statement or callable statement in an application or EJB, there is considerable processing overhead for the communication between the application server and the database server and on the database server itself. To minimize the processing costs, WebLogic Server can cache prepared and callable statements used in your applications. When an application or EJB calls any of the statements stored in the cache, WebLogic Server reuses the statement stored in the cache. Reusing prepared and callable statements reduces CPU usage on the database server, improving performance for the current statement and leaving CPU cycles for other tasks.
Each connection in a connection pool has its own individual cache of prepared and callable statements used on the connection. However, you configure statement cache options per connection pool. That is, the statement cache for each connection in a connection pool uses the statement cache options specified for the connection pool. Statement cache configuration options include:
- Statement Cache Type—The algorithm that determines which statements to store in the statement cache.
- Statement Cache Size—The number of statements to store in the cache for each connection. The default value is 10.
- Using the Administration Console (preferred).
- Using the WebLogic management API.
- getStatementCacheType()
- setStatementCacheType(string type)
- getStatementCacheSize()
- setStatementCacheSize(int cacheSize)
Usage Restrictions for the Statement Cache
Using the statement cache can dramatically increase performance, but you must consider its limitations before you decide to use it. Please note the following restrictions when using the statement cache.
There may be other issues related to caching statements that are not listed here. If you see errors in your system related to prepared or callable statements, you should set the statement cache size to 0, which turns off statement caching, to test if the problem is caused by caching prepared statements.
Setting Your Java Compiler
The standard Java compiler for compiling JSP servlets is javac. You can improve performance significantly by setting your server's java compiler to sj or jikes instead of javac. The following sections discuss this procedure and other compiler considerations.
Changing Compilers in the Administration Console
To change your compiler in the Administration Console:
- Start the Administration Server if it is not already running.
- Access the Administration Console for the domain.
- Expand the Servers node in the left pane to display the servers configured in your domain.
- Click the name of the server instance that you want to configure.
- Select the Configuration —> General tab and enter the full path of the compiler in the Java Compiler field. For example:
- Click Show on the Advanced Options bar to display additional attributes.
- Enter the full path to the JRE rt.jar library in the Append to the Classpath field. For example:
- Click Apply.
- Restart your server for the new Java Compiler and Append to Classpath values to take effect.
In the weblogic.xml file, the jsp-descriptor element defines parameter names and values for servlet JSPs.
- Use the compileCommand parameter to specify the Java compiler for compiling the generated JSP servlets.
- Use the precompile parameter to configure WebLogic Server to precompile your JSPs when WebLogic Server starts up.
Use the weblogic.appc utility to compile EJB 2.0 and 1.1 container classes. If you compile Jar files for deployment into the EJB container, you must use weblogic.appc to generate the container classes. By default, ejbc uses the javac compiler. For faster performance, specify a different compiler (such as Symantec sj) using the -compiler flag.
Compiling on UNIX
If you receive the following error message received when compiling JSP files on a UNIX machine:
failed: java.io.IOException: Not enough space
Try any or all of the following solutions:
- Add more RAM if you have only 256 MB.
- Raise the file descriptor limit, for example:
set rlim_fd_cur = 1024
- Use the -native flag to use native threads when starting the JVM.
Several factors beyond the WebLogic server can impact performance, including hardware and OS configuration, other processes running on the system, performance of back-end database resources, network latency, and so on. You must factor in these items when conducting your own performance evaluations.
Tuning the Operating System:
Each operating system sets default tuning parameters differently. For Windows platforms, the default settings are usually sufficient. However, the UNIX and Linux operating systems usually need to be tuned appropriately.
Linux:
For better packet transfer performance, set the /sbin/ifconfig lo mtu parameter as follows:
# /sbin/ifconfig lo mtu 1500
The mtu (maximum transfer unit) parameter refers to largest number of bytes that a packet can carry over the network. If the packet size is set too low, then your network performance will decrease due to fragmented data. Tuning parameters listed below when adjusted, can enhance application performance
/sbin/ifconfig lo mtu
kernel.msgmni
kernel.sem
fs.file-max
kernel.shmmax
net.ipv4.tcp_max_syn_backlog
Network:
Network performance is affected when the supply of resources is unable to keep up with the demand for resources. Today's enterprise-level networks are very fast and are now rarely the direct cause of performance in well-designed applications.
You should also verify that you have an appropriate amount of network bandwidth available for WebLogic Server and the connections it makes to other tiers in your architecture, such as client and database connections. Therefore, it is important to continually monitor your network performance to troubleshoot potential performance bottlenecks.
Network Bandwidth:
A common definition of bandwidth is "the rate of the data communications transmission, usually measured in bits-per-second, which is the capacity of the link to send and receive communications." A machine running WebLogic Server requires enough network bandwidth to handle all WebLogic Server client connections. In the case of programmatic clients, each client JVM has a single socket to the server, and each socket requires dedicated bandwidth. A WebLogic Server instance handling programmatic clients should have 125-150 percent of the bandwidth that a similar Web server would handle. If you are handling only HTTP clients, expect a bandwidth requirement similar to a Web server serving static pages.
To find out network activity use commands such as:
# sar –n DEV 1 3
LAN Infrastructure:
Your local area network must be fast enough to handle your application's peak capacity. If your network is fully utilized, in that the amount of traffic consistently exceeds its bandwidth capacity, yet your WebLogic Server machine is not fully utilized, do one of the following:
- Redesign the network and redistribute the load.
- Reduce the number of network clients.
- Increase the number of systems handling the network load.
Optimizing the Database
Your database can be a major enterprise-level bottleneck. Configure your database for optimal performance by following the tuning guidelines in this section and in the product documentation for the database you are using.
General Suggestions
Here are some general database tuning suggestions:
- Good database design — Distribute the database workload across multiple disks to avoid or reduce disk overloading. Good design also includes proper sizing and organization of tables, indexes, logs, and so on.
- Disk I/O optimization — Disk I/O optimization is related directly to throughput and scalability. Access to even the fastest disk is orders of magnitude slower than memory access. Whenever possible, optimize the number of disk accesses. In general, selecting a larger block/buffer size for I/O reduces the number of disk accesses and might substantially increase throughput in a heavily loaded production environment.
- Checkpointing — This mechanism periodically flushes all dirty cache data to disk, which increases the I/O activity and system resource usage for the duration of the checkpoint. Although frequent checkpointing can increase the consistency of on-disk data, it can also slow database performance. Most database systems have checkpointing capability, but not all database systems provide user-level controls. Oracle, for example, allows administrators to set the frequency of checkpoints while users have no control over SQLServer 7.x checkpoints. For recommended settings, see the product documentation for the database you are using.
Conclusion
The above are only some of the various ways that the server can be tuned. Bear in mind, however, that a poorly designed, poorly written application will usually have poor performance, regardless of tuning. Performance must always be a key consideration throughout the stages of the application development life cycle - from design to deployment. It happens too often that performance takes a back seat to functionality, and problems are found later that are difficult to fix.
References:
http://docs.oracle.com/cd/E13222_01/wls/docs81/perform/index.html
http://wladocsbypani.blogspot.in/2010/01/performance-tuning-on-weblogic-server.html