|
By Joachim Wolf, Srinivas Nagaraju, Sridhar Chava, and others, February 2002
|
|
|
Introduction
System availability and performance play a key role in achieving customer satisfaction. Reliable system sizing helps to ensure these two important criteria are met.
The goal of this template is to increase the efficiency and quality of a system sizing by providing sizing guidelines. The template includes questions, suggestions, and tables that are easily adaptable to meet your particular requirements.
This document includes the following sections:
-
Section 1 - Process Overview
Describes the phases of the sizing process and provides examples of how to:
- Perform a sizing study
- Prepare the results
- Complete sizing for workload requirements
-
Section 2 - Sizing Study Templates
Provides system sizing study guidelines beginning with the project plan and ending with the results. This section includes an explanation about mapping results to actual workloads.
-
Section 3 - Templates for Sizing a Real-World System
Provides templates for sizing a customer system
Contents
1. Process Overview
There are several steps to sizing a system:
- Definition phase
- Baseline tests
- Scalability tests
- Generalization of rules for other hardware products
- Determination of the required hardware for a given workload
The following overview provides a short description of each step with examples based on a sizing study for the modules "Sales and Distribution" and "Finance" of an ERP system.
| Definition Phase |
| Description |
Example |
Determine what is typically done with the application and how the application is typically used. For example,
- Identify typical ways the application is used (e.g., typical business processes). If there are several types of application usage, it might be possible to define a standard usage and map the various alternate methods to one standard method.
- Determine the CPU consumption-per-dialog steps needed to complete a transaction, such as "creating a sales order"
- Define key figures and restrictions for the sizing, such as:
- Maximum response time allowed
- Maximum CPU utilization allowed
- Throughput (operations per time)
- Number of users
|
Typical "Sales and Distribution" transactions include:
- Creating a sales order
- Creating a delivery note
- Creating an invoice
Typical "Finance" transactions are:
- Posting a document
- Displaying a customer line item
- Posting incoming payments
Note: Two seconds is defined as the maximum average response time for dialog steps allowed in the ERP system.
|
|
| Baseline Tests |
| Description |
Example |
|
Perform tests using the typical application usage defined in the Definition Phase.
Perform tests for one user per one task.
Obtain test scripts from the ERP vendor.
|
- Become familiar with the ERP system.
- Build the test environment
- Perform functional tests.
- Simulate a user who runs the "Sales and Distribution" and "Finance" steps separately.
Scripts provided by the ERP vendor are used as a load generator. These scripts:
- Read test input from a file
- Send test output to the ERP system using specific interfaces provided by the ERP system
- Obtain the response
- Compare the response with the expected response (to verify the system operates properly).
The overall CPU consumption-per-dialog step (from the Definition Phase) and the CPU consumption of the components of the system (e.g., database and ERP system) are the main results of the Baseline Tests. These results will provide a baseline for the following Scalability Tests.
|
|
| Scalability Tests |
| Description |
Example |
Scalability Tests assess how the application should scale:
- Run parallel tests with more users and tasks.
- Check how well the application scales.
- Determine if there are bottlenecks.
Be sure to identify and measure any resource consumption change as the load increases.
|
By increasing the number of simulated users, the system load will increase. Review the system behavior:
- Can the application run on more than one CPU using more than one Solaris Operating Environment process? Is there a CPU limit that can be determined (for instance, the application scales well only up to 4, 8 or 16 CPUs)?
- How does the CPU consumption-per-dialog (Definition Phase) step change under a high system load?
- Is the response time under higher load still acceptable (i.e., below 2.0 seconds)?
The key results of the Scalability Tests include:
- The overall CPU consumption and the CPU consumption of each component (database, ERP system) per dialog step
- The maximum number of users who can run on one CPU having an average response time below 2.0 seconds (in this example: 40 "Sales and Distribution" users)
- The maximum number of dialog steps per second ("throughput") which can be processed on one CPU
- The maximum number of line items per second that can be processed on one CPU
The results show that the average CPU consumption-per-dialog step (Definition Phase) for "Sales and Distribution" is twice as high as for "Finance."
|
|
| Generalization of Rules for Other Hardware Products |
| Description |
Example |
| The Baseline and Scalability Tests have been performed on specific hardware. If there is a need for sizing numbers for other machines and network components, the results of the tests must be either mapped to the new environment, or they need to be rerun on this other hardware. |
The tests above have been performed on 300 MHz CPUs.
To provide sizing numbers for 400 MHz CPUs, we again run the "Sales and Distribution" scalability tests on faster CPU machines. We obtained an increase of 25% in both the number of users and the throughput.
Due to the similarity of both the "Finance" and "Sales and Distribution" modules from previous investigations, we assume a 25% increase for "Finance" as well.
|
|
| Determination of the Required Hardware for a Given Workload
Description |
| Description |
Example |
|
Map customer requirements to the results of the previous tests.
Determine the specific system resources that are needed to handle the load.
|
1000 users of a sales and distribution department and 1000 users of a finance department plan to use the ERP system on 400 MHz systems. For performance and security reasons (for instance, head room for batch jobs), a maximum CPU utilization of 67% is requested by the customer.
First we map the "Finance" users to "Sales and Distribution" users, so we get 500 "Sales and Distribution" users instead of 1000 "Finance" users.
The scalability tests have shown that a 300 MHz CPU can handle the load of 40 "Sales and Distribution" users. Therefore, a 400 MHz CPU handles 50 users, and 1500 users need 30 CPUs.
Using the customer requirement for 67% CPU utilization, we configure a system containing 45 CPUs.
(The software configuration consists of the database and the ERP system. Both components could also run on separated machines. To configure them, we use the ratio of the CPU consumption of both the database and the ERP system (33% to 66%) determined through the Scalability Tests, and add one CPU on each machine to handle the network traffic. As a result, we configure 16 CPUs for the database and 31 CPUs for the ERP system.)
|
|
2. Sizing Study Templates
Complete the templates provided in this section.
2.1 Sizing Study Goal
The goal of a system sizing study is to determine the system configuration baseline for a given workload.
2.2 Roles and Responsibilities
Define who is responsible for each of the following roles and provide deliverables, where applicable.
| Roles |
Responsibilities and Deliverables |
| Sizing definition |
|
| Definition of workload and sizing baseline |
|
| Definition of hardware and architecture used for the test |
|
| Define the backup and restore concept for the test |
|
| Hardware installation and configuration |
|
| Hardware support |
|
| Software installation and configuration |
|
| Software support |
|
| Baseline and scalability runs |
|
| Performance analysis and tuning |
|
| Interpretation of results |
|
| Reporting (for instance, prepare sizing guide) |
|
| Additional steps: |
|
|
2.3 Test Schedule
Define the test schedule based on experience and any information available from similar tests. The following schedule is provided as a sample only. It shows some of the typical milestones in a sizing study project. Actual steps and duration will vary greatly depending on the sizing goals, size of the tests, and the test engineer's expertise.
| Activity |
Duration
(Example) |
Duration
(Complete duration estimated for each activity) |
Definitions
Application, components, modules
Transactions, master data, transactional data
Sizing baseline
Hardware and architecture
|
4 days |
|
Hardware setup
Servers
Storage
Network
|
2 days |
|
Software installation and configuration
OS and patches
Volume Manager
Database
Middleware
Application
Benchmark driver
Monitoring tools
|
3 days |
|
| Database creation and population |
2 days |
|
| Baseline runs |
3 days |
|
| Scalability runs |
10 days |
|
| Backup |
1 day |
|
| Reporting |
2 days |
|
| Additional steps: |
|
|
|
2.4 Sizing Definition
2.4.1 Define:
- The application (for example, mySAP.com or Baan IV)
The application component (for example, database, application logic, web, presentation layer, or network)
The application module (for example, Finance or Warehousing) to be sized
2.4.2 Define the sizing audience. Determine who will consume the results. For example, Field Sales (either Sun's or ISV's).
2.5 Workload Definition
2.5.1 Describe application-level transactions to test (ensure these transactions comprise typical application usage)
2.5.2 Define master data and transactional data for the transactions to test
2.5.3 Define the sizing rules to apply (for example, think time and response time rules)
2.6 Sizing Baseline
Describe the sizing baseline (i.e., the sizing metrics) to use. Examples include:
- Number and type of concurrent users per CPU
- Transaction throughput per CPU
2.7 Product Stack Checklist
If applicable, answer the following questions when completing the Product Stack Checklist table:
- List all the software products, including version numbers required: OS, Volume Manager, database, middleware, application, client, etc.
- Are all these products certified for the application? Do the products support the application?
- Are all these products licensed? If not, who can provide the license?
- What load simulation tool will be used? Is it available for the Solaris Operating Environment?
- Have the simulation tools or scripts been fully developed and tested?
- Are the products or tools instrumented to collect the required metrics, according to the objective?
- Does all the software have documentation, and is it available?
- Have the software support teams been identified and contacted?
| Product Stack Checklist Table |
| Component |
Software Name |
Certification |
License |
| Operating System |
|
|
|
| JVM |
|
|
|
| Volume Manager |
|
|
|
| Database Server |
|
|
|
| Web Server |
|
|
|
| Application Server |
|
|
|
| EJB Technology-Based Server |
|
|
|
| Servlet Engine |
|
|
|
| Proxy Server |
|
|
|
| Simulation Tool |
|
|
|
| Additional entries: |
|
|
|
|
2.8 Hardware Test Environment
There might be several layers of your application, such as database, application, web, and presentation layers. Describe the logical architecture of your application. Figure 1 illustrates a sample architecture.
Figure 1: Logical architecture of a sample application
(Click image to enlarge.)
|
Map this logical architecture to the test environment. Figure 2 provides a sample configuration.
Figure 2: Test configuration for a sizing study (example)
(Click image to enlarge.)
|
Answer the following questions and complete the table.
- What configurations and topologies are being tested?
- What systems are required? What is their availability?
- Where will the test take place?
| Hardware Test Environment Table |
| Type |
Model |
# of CPUs |
Memory Size |
Storage Requirements |
Redundancy, HA-Requirements |
| App Server |
|
|
|
|
|
| Web Server |
|
|
|
|
|
| DB Server |
|
|
|
|
|
| Load Generator |
|
|
|
|
|
| Network Components |
|
- / -
- / -
- / -
- / -
|
- / - |
- / - |
- / - |
| Additional entries: |
|
|
|
|
|
|
2.9 Evidence of Scalability
2.9.1 Describe configurations of previous tests (Sun hardware or other platforms), if any
2.9.2 Describe any unresolved scalability limits from previous tests
2.9.3 Describe the scalability tests
2.10 Configuration Instructions
Provide hints as to how to make sure the application runs stable, scales, and performs well. For example: adjust buffer sizes and/or other parameters.
| Configuration Instruction Suggestions |
| Component/Hardware/Software |
Configuration hints, monitoring tools, essential tuning methods and parameters |
| Operating System |
|
| Network |
|
| JVM |
|
| Volume Manager |
|
| Database Server |
|
| Web Server |
|
| Application Server |
|
| EJB Server |
|
| Servlet Engine |
|
| Proxy Server |
|
| Simulation Tool |
|
| Additional entries: |
|
|
2.11 Results
Before completing this section, copy the table for each sizing-relevant component (such as database and application servers) according to your needs.
- Most sizing studies will determine a CPU-user ratio or a CPU-load ratio similar to the data in Figure 3.
Figure 3: CPU-user ratio of an application (example)
(Click image to enlarge.)
|
For instance, you can use the CPU Consumption Table to document the CPU consumption of the application.
| CPU Consumption Table |
| # of CPUs |
# of Users |
# of CPUs |
# of Users |
# of CPUs |
# of Users |
| 2 |
|
10 |
|
18 |
|
| 4 |
|
12 |
|
20 |
|
| 6 |
|
14 |
|
22 |
|
| 8 |
|
16 |
|
24 |
|
|
2.11.1 Describe the conditions for which the values in the table are valid (e.g., response time or CPU utilization)
2.11.2 If available, add sizing information regarding main memory, storage, network, and the like here. For example, you could create a formula to calculate memory usage of an application, such as: 100MB+#Users*15MB.
2.11.3 Explain how to map these results to actual, real-world loads.
3. Templates for Sizing a Real-World System
3.1 Customer Information
Complete the customer data for the system configuration.
| Company |
| Company Name |
|
| Street |
|
| City/Postal Code |
|
| Contact |
| Last Name |
|
| First Name |
|
| Department |
|
| Phone |
|
| Fax |
|
| E-mail |
|
|
3.2 Load Definition for the Customer System
Describe the load for the particular system configuration.
| Load |
Value |
| # of users |
|
| Throughput |
|
|
3.3 Configuration of the Customer System
Map the results from Section 2.11 to the described load.
| Type |
Model |
# of CPUs |
Memory Size |
Storage Requirements |
Redundancy, HA-Requirements |
| App Server |
|
|
|
|
|
| Web Server |
|
|
|
|
|
| DB Server |
|
|
|
|
|
| Network Components |
|
- / - |
- / - |
- / - |
- / - |
| Additional entries: |
|
|
|
|
|
|
3.4 Additional Requirements
In addition to the system configuration, other sizing topics might be relevant to your particular need:
- Is there only a production system or also development and/or test systems to be configured?
- How does the application being sized collaborate with the existing system landscape?
- What LAN/WAN connections already exist? What are the requirements?
- Are migration/integration services required?
- What peak load is expected when? e.g.: Which are the hours when the maximum number of users are logged in? When is the backup and/or the batch time window planned? How often? Is an online backup necessary?
- How many working days per year are planned?
- Do users reside in more than one time zone? What languages are necessary?
- Is there system downtime planned? How would unplanned downtime be damaging to the business?
- Is high-availability required? What kind of failover is required (manual, automatic)?
- How long can a restore take to be completed?
What kind of disk topology is used (e.g., mirroring, striping, RAID 5)?
- What is the expected database growth?
- What are the printing requirements?
- For which CPU utilization should the configuration be sized? How much headroom for unexpected load should remain?
Additional Resources
Capacity Planning for Internet Services Online Discussion
February 2002
|
|