Planning and Measureing Performance of a SharePoint

Ivan Neganov
Ivan Neganov
Founder and CEO of SoftForte, Inc. 11 years of experience in
developing WCM solutions based on ASP.NET and SharePoint
platforms. Focusing on SharePoint since 2007.
the Science of Quality
Email: [email protected]
Part I – Planning for Performance
Part II – Planning for Throughput
Part I – Planning for Performance
How Fast is “Fast”?
Human Psysiology Factor
2006 Akamai/Jupiter Research
Under 0.1 sec – virtually unnoticeable.
Under 1 sec – perceived as interactive
Under 10 sec – willing to focus on a task
33% of broadband consumers will wait no longer than 4 sec for a page to load.
2009 Akamai/Forrester Research
2 sec. – average expectation of online shopper
3 sec. – max time 40% shoppers are willing to wait for a page to load
KB40 – Keynote Business 40. Keynote Systems, Inc. maintains index of fastest
business internet sites:
WM100 – Webmetrics maintains index of top 100 sites by performance
SharePoint Response Time Guidance
Type of
Rare operation
Acceptable user
response time
· Browsing to the home
<3 seconds
· Browsing to a document
· Creating a subsite
<5 seconds
Creating a list
· Uploading a document to
a document library
· Backing up a site
<7 seconds
· Creating a site collection
How Fast is “Fast” in my Company?
 Study
publicly available metrics
 Study organization’s historical metrics
 Estimate average and peak traffic
 Define a matrix of PLT1 and PLT2:
 For various pages
 For various authentication groups
 For peak and average usage
Response Time
Page Load Time (PLT) or User
Response Time (URT) – time until a
page fully renders.
Microsoft uses PLT1 and PLT2 – the
very first access to the page, and
subsequent access to the same page.
Part I – Planning for Performance
 Client
 Network
 Server
URT Formula
URT Formula (Netforecast)
R – response time
Payload – total size of page and all its resources
AppTurns – round trips made at application level (excluding TCP
handshake/congestion control round trips & authentication)
RTT – round trip time
Cs – constant server time component
Cc – constant client time component
Need for Testing
Simply applying the formula will lead to
significant errors.
 You need to calibrate every part of it.
 Testing produces data for calibration.
Part I – Planning for Performance
Client Scripting Performance
J-Query profiler from John Resig allows to measure
performance by method and calculate Big-O
Profiling script from within script is very imprecise,
partly due to platform implementation. For example,
on Windows XP timer would show intervals shorter
than 15ms as 0.
 YSlow for Firebug -
 JScript Profiler
 DynaTrace profiler (can profile script parsing time!) Article:
Part I – Planning for Performance
Network Performance – the Bottleneck
Bandwidth limitations – can be
addressed via technology
 Latency limitations – Speed of Light
RTT/2 = (36,000 *2)/300,000
RTT ~ 0.5 sec.
TCP limitations
 Signal strength/QoS
Latency and Bandwidth
Overall link bandwidth = 3 Mbit/s
What is my actual bandwidth & latency? detects your local
bandwidth and latency.
TCP Communication
A max. packet size on Ethernet is 1500 bytes, aka
MTU or max. transfer unit.
On IPv4 networks IP overhead takes 40 bytes,
hence max payload equals 1460 bytes, aka MSS or
max. segment size.
TCP requires acknowledgement (ACK) of all packets
sent but allows sending a number of packets without
waiting for ACK to improve speed. Eventually ACK
must arrive.
If some packets are lost, i.e. there is no ACK within a
timeout, then packets are re-transmitted.
TCP Communication: Naïve Model
TCP Communication: Realistic Model
TCP Communication: TCP Window
TCP Window
TCP Window is a number of bytes a
receiver can accept without sending ACK
Too large window means network
congestion >> lost packets >> retransmission >> performance degradation
Too small window means low bandwidth
utilization >> performance degradation
TCP Slow Start
Optimal window size is twice the amount of data that can be “in flight” on
the wire from sender to receiver at any given time:
RWIN = 2 * (Bandwidth * RTT/2), or
RWIN = 2 * BDP
BDP – bandwidth-delay product.
RWIN – TCP receive window buffer.
TCP detects bandwidth and latency and dynamically sets window size.
Usually initial RWIN = 64KB. Once connection is established, TCP
increases RWIN, process aka “Slow Start”. ”. On a slower WAN it can take
up to 12 round trips to optimize the receive window.
Initial RWIN size on W2K3:
TCP Congestion Control
Sender maintains congestion window, CWND and
constantly tweaks it according to bandwidth and delay
to avoid congestion:
Effective bandwidth = CWND/RTT
Various congestion control algorithms are known, ex.
Tahoe, Reno. Windows Vista, 7 and 2008 use CTCP. It
is advantageous over WAN, enabled by default on
2008, but not on Vista and Windows 7.
TCP Congestion Window Scaling
TCP Packet Loss
Packet loss may occur for many reasons, ex. when network
is congested or equipment is misconfigured, or there is a
signal loss, etc. Packet loss severely impacts throughput:
Throughput <= 0.7 * MSS/(RTT * Sqrt(Ploss))
MSS – Max. segment size, 1460 bytes for IPv4, 1440 bytes for IPv6 on Ethernet.
Ploss – probability of a packet loss.
Example: At 100ms round trip time and 10-4 probability of a
packet loss you would get no more than 8Mbit/s throughput.
Contemporary networks have very low packet loss
probability, yet some packet loss occurs on long links. WAN
testing is sometimes done assuming 1 – 3% of packet loss.
Addressing TCP Limitations
Using UDP instead of TCP
 Minimizing number of round trips
 Using few large files vs. many small files
 Using multiple browser connections
 Using HTTP persistent connections
 Using client-side caching
 Using Content Delivery Networks (CDN)
 Using WAN accelerators & offloading
Multiple Browser Connections
Contemporary browsers use multiple
TCP connections per hostname:
 IE6, IE7 – 2 connections max;
 IE8, FireFox 3.5 – 6 connections max.
Open multiple (source) ports for multiple
TCP connections.
 Despite having multiple connections a
lot of sequential loading still takes place.
IE8 is the first browser to download
multiple script files in parallel.
HTTP Persistent Connections
HTTP 1.1 supports persistent connections through
Keep-Alive header.
The goal is to re-use underlying TCP connection with
its current CWND avoiding having to go through
Slow Start again.
Enabled by default on most browsers and on IIS 6, 7.
Keep-alive timeout is 1 min for IE and 15 sec. for
FireFox, and is adjustable. For changing timeout on
IE6, 7 see:
Enabling Keep-Alive in IIS7:
Content Delivery Networks
CDNs distribute cached content on multiple servers, which are close to
end users. Internet traffic is redirected to the closest CDN server
instead of the origin server.
Very expensive, typically affordable to large enterprises only. Ex. $0.5/GB on 50
TB monthly ~25,000$/month
Less efficient for highly volatile content.
It can be technically difficult to invalidate CDN cache explicitly.
Free CDNs, primarily AJAX support:
Low latency & high bandwidth when accessing a CDN server result in much better
performance for the end users.
As a result of many users hitting CDN cache the load on original server is reduced.
Excellent for media streaming.
Google AJAX Libraries API -
Microsoft AJAX CDN
More Info about CDNs:
WAN Accelerators & Offloading Devices
Use packet compression, differencing,
caching, optimal route calculation algorithms,
reducing packet loss.
Solutions include Cisco, Citrix, Packeteer,
Riverbed, F5, Brocade.
Microsoft’s ISA and IAG, and their successor
Unified Access Gateway (UAG 2010) provide
caching, offloaded compression, differencing
and authentication delegation.
Determining Network Performance
Nature of network transmission complicates its
mathematical modeling and projection of results between
different networks. This increases amount of calibration
testing needed.
Create a reference set of web pages and test them on
various networks. Calibrate earlier discussed CNS
formulas using these test results.
Tools are available:
Part I – Planning for Performance
Server Performance
Create baseline measurement for various
load profiles and PLT1/PLT2
Use Performance counters:
 ASP.NET Request Execution Time
 ASP.NET Request Wait Time
 Server Response Time (SRT) = the sum of the
Essential performance counters:
SharePoint 2010 Performance Improvements
More load on WFE, SQL & Client
PLT performance improvements and
optimization for WAN, early page rendering
“Cobalt” protocol – asynchronous
uploading of an office file from client cache
to server.
Developer Dashboard – improves
bottleneck diagnostics
Performance Measurement Tools
SharePoint Developer Dashboard
Measuring PLT1 & PLT2
Part II – Planning for Throughput
 Objectives
 Models
 Rules
of Thumb
 Selecting Hardware
 SharePoint 2010 & Capacity
About Capacity Planning
 Know expected load levels for the application
 Ensure acceptable performance at expected load
 Determine how to scale application for the future
In the CNS model above, focus is primarily on
Server part.
Networking part matters however:
 CDNs do reduce server load for Internet scenarios.
 In geographically distributed farms WAN bandwidth
and latency affect capacity planning.
Theoretical Web Server Model
Server Under Load: Theoretical Model
M/M/1 queue for single web server and MM/c queue
for load-balanced servers
Poisson Distribution – Memorylessness:
 knowledge of last occurred event does not have an impact
on successive events
Little’s Law:
Nqueue = SRT * Ratearrival
 Understanding of physical capacity limits
 Approximate but practical server load function
 Importance of RPS as a measure of capacity
Theoretical Server Response Time
Server performance is analyzed together
with the server load.
From queuing analysis for M/M/1 queue:
SRT = SRT(0)/(1 – U)
SRT – server response time
SRT(0) – server response time at 0 utilization
U – utilization, or average percentage of time the server is busy.
SRT is a Function of Utilization
Load-Balanced Servers
SharePoint Farm Capacity Planning
Theory explains guidance parameters & helps
with rough estimates
Rules-of-Thumb, best practices & reference
performance tests are used to determine
components of the farm
Requests per Second (RPS) are used to
measure farm capacity
Additional tools: SPCP:
Throughput Targets: Classic Usage Model
All SharePoint site users can be classified into 4 groups:
Light users – generate 20 RPH or 2 User Ops/Hour
Typical users – generate 36 RPH or 3.6 User Ops/Hour
Heavy users – generate 60 RPH or 6 User Ops/Hour
Extreme users – generate 120 RPH or 12 User Ops/Hour
RPH are calculated based on daily average non-401 requests made
by distinct users.
Given total number of users in each class set percentage of them that
is active, i.e. actively using the SharePoint site. This is also known as
concurrency. Even at peak usage 10% is a high concurrency, 5% is
Weighted sum yields total demand in RPS.
Classic Usage Model - Example
There are total of 30,000 users of the portal.
 25,000 of them are typical users.
 4,500 of them are heavy users.
 500 of them are extreme users.
During the peak hour on average 10% of
typical users and 5% of heavy and extreme
users are accessing the site.
What is the required farm capacity?
Capacity = (0.1 * 25,000 * 36 + 0.05 * 4,500 *
60 + 0.05 * 500 * 120)/3600 = 29.6 RPS
SharePoint Activities Affect Capacity
A farm is serving a number of activities:
User operations (web page & file requests)
Search indexing
Profile import/sync
Variations, workflows, scheduled jobs
Office clients requests
AJAX calls
User activity and number of concurrent users are the
primary factors used in capacity planning.
The picture is different when backend activities
cannot be confined into 12-hour window.
Plan for Peak Concurrency!
Rules-of-Thumb: Web Front End
Portal Collaboration Scenario
WSS Collaboration Scenario
Rules-of-Thumb: Web Front End
HA prevail over capacity requirements for small and
medium installations.
Max RPS achieved at 5 WFEs per DB server. More
WFEs overload ConfigDB.
1 DC per 3-4 WFEs, if NTLM authentication is used.
Set 1 WFE as crawl target, remove it from load
Average WFE CPU utilization should be 30%.
Rules-of-Thumb: Storage Sizing
Important for performance planning
because storage estimates contribute to
IOPS requirement for the disk subsystem.
 100 GB per content database
Use reference installations, or Microsoft
estimation guidance:
Two common measures of disk throughput:
 IOPS – used for random access to disk, typical for
SharePoint workloads.
 MB/s – used for mostly sequential access, common to
serving large files, running large reports on cubes.
Use performance counter: Disk Transefers/sec to
determine peak IOPS based on RPS.
10K RPM drives give 100-130 IOPS; 15K RPM
drives give 150-180 IOPS.
Use sqlio.exe utility to determine actual IOPS of a
Rules-of-Thumb: SQL Server
Resources on SQL for SharePoint
Resources on SQL Mirroring:
Rules-of-Thumb: SQL Server
Disk Latency: Disk sec/transfer
 Data files < 10ms
 T-log files < 5ms
Disk Capacity:
*RAID-5 can be used for static web content.
Rules-of-Thumb: SQL Server
Typical Deployment Sizes:
Content db size
< 50GB
> 50GB
# of Content dbs
< 20
> 20
# of concurrent requests to SQL
< 200
> 200
# of Users
< 1000
> 1000
# of items in regularly accessed
< 2000
> 2000
< 20
> 20
# of columns in regularly
accessed list
Rules-of-Thumb: SQL Server
Recommended Capacities:
Recommended DB server
8 GB +
16 GB +
32 GB +
2 MB
> 2 MB
> 2MB
< 20
< 10
< 10 (data)
< 5 (T-log)
Processor L2 cache
Bus bandwidth
Disks latencies (msec)
Network latency (msec)
Capacity Planning Summary
SharePoint 2010 Capacity Improvements
Large list throttling
WFE will return 503 when overloaded
Office clients are aware of this, and will in turn
throttle server requests
 Co-authoring of documents;
 PPT broadcasting.
HTTP throttling
 Blocks robots, search indexing
 Gives first priority to client traffic
Bit rate throttling – used by assets library,
implemented in IIS Media Services extension
SQL Server 2008 Throttling – Resource Governor
can limit use of resources by specific processes
Software boundaries improvement
SharePoint 2010 Capacity Planning
SharePoint 2010 Capacity Management
 Logging
 Developer Dashboard
 Load Testing Toolkit (a part of
SharePoint Administration Toolkit)
 There is more to come…
Capacity Validation Load Test
Performance Counters
Performance counters are central in
determining all aspects of performance.
 One example for capacity planning:
ASP.NET Applications\Request/sec
A comprehensive list of relevant
counters is available here:
Load Testing Tools
SharePoint 2010:Load Testing Kit, part of
SharePoint Administration Toolkit –
reference Web & Load tests.
Useful blog post by Bill Baer lists tools
used for stress testing of SharePoint
Part III – Best Practices
 Information Architecture
 Web
Front End (WFE) Servers
 SQL Server
Information Architecture: Best Practices
Account for software boundaries:
For large lists, follow performance guidance:
Separate content with different usage profiles into
different site collections
Account for authentication performance impact:
Anonymous - fastest
Forms - slowest
WFE Best Practices: Caching
Output caching & cache profiles
BLOB caching
Native to ASP.NET 2.0, individual page level
Turned off by default in 2007.
Need Publishing Infrastructure Feature on.
Enable for read-only users.
Never cache search results for authenticated users, alternatively disable search results page.
Uses RAM on WFE, adjust ASP.NET private byte limit
Used on document libraries only
Minimizes round-trips to database for HTML, CSS, image or media files, etc. by creating disk-based cache on WFE
Not enabled by default
Important to use max-age attribute to instruct clients to cache resources
Affects disk I/O of the WFE servers
Object caching
Benefit for certain page items: navigation data, cross-list query data
Uses RAM: default 100MB. Monitor cache hit ratio counters and adjust RAM to have over 90% hits.
The only caching turned on by default
Office Web Applications Caching (SharePoint 2010)
Branch Caching (Windows 2008)
More Info:
WFE Best Practices: IIS Compression
Static IIS compression is on by default in IIS 6, 7. Used for
*.html, *.htm, *.css, *.txt files by default.
Dynamic compression is off by default on both IIS 6 & 7. Used
for *.asp, *exe files by default.
Using IIS compression increases load on WFE CPU, but it
reduces disk I/O, which is much slower, so it can dramatically
boost performance.
You need to configure compression levels, and add extensions
for *.js, *.aspx etc.
IIS 7 can be configured to compress items before adding them
to cache. This needs to be turned on to reduce load on the
WFE Best Practices: Custom Code
Releasing resources for SPSite, SPWeb
Avoid thread synchronization issues when caching objects
 Accessing folders and lists
Do not use SPList.Items
Use SPList.GetItems(SPQuery)
Do not iterate over SPList.Items
Use PortalSiteMapProvider to enumerate lists
Scalability: avoid code, enumerating OM objects for large
# of concurrent users
SPQuery objects
 Do not use unbounded SPQuery objects
 Use indexed fields in queries
Timer jobs
 Break long-running operations into small pieces to minimize re-
do work when restarting a job.
WFE Best Practices: Other
Load scripts outside of script engine using
document.write(<script src=…); SharePoint 2010
does this!
Make sure HTTP 1.1 keep-alive header is on. It is
used by persistent connections, turned on by default
in IIS 6,7
Minimize number of small file downloads. Reason:
many small files do not use link capacity fully. Single
large file downloads are more efficient.
Load scripts on demand where possible. Ex. Core.js
script on Internet sites
WFE Best Practices: More Info
SharePoint Dispose Checker Tool
12 Steps for Faster Web Pages – Jim Pierson:
Tuning web server performance:
Andrew Connell on Performance:
Common Coding Issues with SharePoint OM:
Optimizing Custom WP for the WAN:
Configuring Caching & Performance – James Petrosky:
Search Best Practices (MOSS 2007)
Use dedicated server for Indexing when
 Do not combine search and query roles
on the same server.
 Set one of WFEs as a crawl target, and
remove it from load balancer
 Search query performance is improved
when using multiple load-balanced
query servers:
SQL Server Best Practices
Ensure correct Host Bus Adapter drivers and firmware
Configure correct NTFS allocation unit size (done during
formatting the drive, as a format.exe option)
 64K – best. Equals to SQL Server extent size. Default is 4K, can
result in 30% performance hit.
Ensure correct Windows sector alignment. Windows 2008
aligns sectors by default (done during partitioning of the
 Incorrect alignment can result in up to 50% performance hit.
More Info:
SQL Server Best Practices
Database file placement priority among faster disks:
tempdb data and T-log files
Db T-log files
Search database data files
Content Database data files
Place tempdb, Content db and T-logs on separate LUNs.
Use multiple data files for Content and Search db
 distribute them across disks.
 # of files should be <= # of processor cores
 Multiple data files are not supported for other dbs.
Place SharePoint Search crawl and query tables on
separate spindles.
SQL Server Best Practices
100 GB content databases (soft) limit.
Break content into content databases by IO profile. Example: store
collaboration sites content and publishing portal content in different
Use dedicated database for large site collections (> 50 GB)
Configure tempdb files = # of processors
Configure tempdb to be 25% of content db size. Alternatively either at
least 10% or the size of the largest table, whichever is greater.
More Info:
Please complete your evaluations
to enter in the prize drawing!

similar documents