XenDesktop im Unternehmenseinsatz - Erfahrungen und Best Practices von Citrix Consulting
Thomas Berger, Architect Michael Palesch, Architect Citrix Consulting Juni 2011
• Part 1 – XenDesktop in the Enterprise
• Part 2 – Lessons Learned in Desktop Transformation Projects
Agenda
XenDesktop Architecture Review
User
XD Controller Web Interface
Active Directory
License Server
Provisioning Server
Storage
Virtual Desktop
Hypervisor
HT
TP
(s)
XM
L
LD
AP
H
TT
P(s
)
UDP
HT
TP
(s)
/ W
CF
NFS, iSCSI, FC
CIFS
NFS iSCSI
FC
ICA
SQL Server
XenDesktop Architecture Review
User
XD Controller Web Interface
Active Directory
License Server
Provisioning Server
Storage
Virtual Desktop
Hypervisor
HT
TP
(s)
XM
L
LD
AP
H
TT
P(s
)
UDP
HT
TP
(s)
/ W
CF
NFS, iSCSI, FC
CIFS
NFS iSCSI
FC
ICA
SQL Server
• WI 5.x itself scales to the IIS specification • The real bottleneck in “Web Interface Scalability” is the XML Service
• Placing WI on either 2003 or 2008 has no real impact on scalability • However, using 2008-based XML Brokers reduces scalability by almost 50%!
• We can no longer “exploit” the Log on Locally user right assignment in 2008
• A 2008 R2 WI box with 2 CPU and 2 GB RAM has been proven to scale to ~31k users/hour or ~9 users/sec • Almost 60k users/hour with 2003-based XML Brokers
Web Interface Scalability
• Always deploy 2 Web Interface servers for redundancy
• Use a hardware load balancer if possible (i.e. NetScaler) • Intelligent monitoring of WI availability / XML Service
• WI is a good candidate for virtualization • 2 vCPUs and 4 GB RAM is a good starting spec
• Check if encryption is required (User WI XML) • Otherwise user credentials are transferred as clear text
• Disable Socket Pooling if not using SSL
Enterprise Considerations
XenDesktop Architecture Review
User
XD Controller Web Interface
Active Directory
License Server
Provisioning Server
Storage
Virtual Desktop
Hypervisor
HT
TP
(s)
XM
L
LD
AP
H
TT
P(s
)
UDP
HT
TP
(s)
/ W
CF
NFS, iSCSI, FC
CIFS
NFS iSCSI
FC
ICA
SQL Server
• Important to understand the new XenDesktop architecture • No more IMA service or Data Store
• New XD5 Controllers are stateless and the database (SQL) is relational
• The SQL database must be made highly available
• Many of the services (XML, etc.) have been re-written in .NET
• Registry-based discovery & registration is now used by default
• New architecture allows for greater scalability
The All New XenDesktop 5 Controller
• In XenDesktop 4 it was a best practice to dedicate servers to certain roles • 2 DDCs for IMA and PMS
• 2 DDCs for XML and Controller
• In XenDesktop 5 it is recommended to not dedicate servers • 2 DDCs per XD5 “site” (instead of 4 DDCs per XD4 farm)
• Site services are load balanced automatically
• Run PowerShell Cmdlet “Get-BrokerController” and look for property “ActiveSiteServices” to verify
• Specify all XD5 Controllers as XML Brokers in Web Interface
DDC Bottlenecks (XD4 vs. XD5)
• Scalability tests using two (physical) 2x4’s and 16 GB RAM: • Boot Storm (20,000 desktops in 10 minutes): 40-50% CPU utilization during the
virtual desktop registration process
• Logon Storm (20,000 logons in 13 minutes): 50-60% CPU utilization during user connection process
• User Perception: 99.9% of the brokered connections responded to launch requests in less than 2.5 seconds
XD5 Controller Scalability
• Scalability tests using two virtual servers with 4 vCPUs and 4 GB RAM: • Boot Storm (2,500 desktops in 4 hours): 10% CPU utilization during the virtual
desktop registration process
• Logon Storm (2,500 logons in 30 minutes): 12-15% CPU utilization during user connection process
• User Perception: 100% of the brokered connections responded to launch requests in less than 2.5 seconds
XD5 Controller Scalability
• Rough estimate based on scalability tests: • A single XD5 site can scale to 10,000 desktops with 2 Controllers under normal
conditions
• Need to get more granular? • 125-180 virtual desktop registrations per minute per dedicated core
• 100-120 user logons per minute per dedicated core
• Assumes the desktops are delivered via PVS and the Controller’s CPUs are not shared with other components
XD5 Controller Scalability
• XenDesktop 5 uses SQL Database: • To store configuration information
• A message bus between the Controllers
This causes a massively higher performance impact on SQL
XD SQL Database Scalability
• Scalability tests using three (physical) 2x4’s with 16 GB RAM: • Boot Storm (20,000 desktops in 10 minutes):
• 15-25% CPU (SQL principal database)
• 5-10% CPU (SQL mirror)
• SQL witness was essentially idle
• Logon Storm (20,000 logons in 13 minutes):
• 32% CPU (SQL principal database)
• 10% CPU (SQL mirror)
• SQL witness was essentially idle
XD SQL Database Scalability
Pay Attention to the Transaction Log!
• 20,000 desktop scenario
• High number of SQL transactions
• 666 transactions / sec equals 20k
desktops sending heart beat
(every 30s)
• Can cause transaction log to grow
excessively (gigabytes)
– Check CTX126916
• Ensure SQL database is made highly available • Check Citrix XenDesktop Design Handbook (bit.ly/xdhandbook) for SQL
Database Sizing
• The database itself will be small (MBs), but the transaction log will be big (GBs)
• Leverage database mirroring, failover clustering or “HA” built into the hypervisor
Enterprise Considerations – XD SQL Database
XenDesktop Architecture Review
User
XD Controller Web Interface
Active Directory
License Server
Provisioning Server
Storage
Virtual Desktop
Hypervisor
HT
TP
(s)
XM
L
LD
AP
H
TT
P(s
)
UDP
HT
TP
(s)
/ W
CF
NFS, iSCSI, FC
CIFS
NFS iSCSI
FC
ICA
SQL Server
• Most frequently asked question is:
How many VMs / box?
• The most definitive answer is:
It depends!
Hypervisor Scalability
…because all users and apps are different!
• Real world ratios range from…
Hypervisor Scalability – Why Does it Depend?
16 VMs per Core
(Light Task Worker)
4 Cores per VM
(Heavy Trader / CAD)
• You will need to test! • Formal P&S Testing with tools such as ESLT or LoadRunner are preferred
• If P&S Testing cannot be performed, conduct an extended pilot within your environment • This will at least allow you to gather some baseline data
• Don’t forget to include a “buffer” in case you’re off
Hypervisor Scalability – Where to Begin?
• Gather some performance statistics from existing workstations • Only works if the workload will not change when going virtual
• 3rd party software may help such as: • Liquidware Labs Stratusphere
• Novell PlateSpin Recon
• Microsoft Assessment and Planning (MAP) Toolkit for Hyper-V
• Make sure to include an even bigger “buffer”
What if a Pilot is Not Possible?
• XD on XenServer, Hyper-V and vSphere is all about the same in terms of user density • Architecture and features can be slightly different
• Processors that support nested paging are highly recommended • Extended Page Tables (Intel)
• Rapid Virtualization Indexing (AMD)
• Remember to “save” 1 core* and ~1-3 GB of memory* for the hypervisor itself • However, XS 5.6 FP1 now uses 4 CPUs by default instead of 1
Hypervisor Scalability
• Certain memory over-commitment features can be helpful in VDI deployments • XenServer, vSphere and Hyper-V now all support dynamic memory
management (essentially “ballooning”)
• Transparent Page Sharing (TPS) doesn’t help much with “new” operating systems (legacy pages of 4KB vs. new large pages of 2MB)
• Don’t turn anything on/off unless you know what you’re doing
Hypervisor Scalability
• Dedicate NICs for certain functions • Ensures highest scalability
• Eases monitoring / trend analysis
• Avoid single points of failure • Create Bonds/Teams whenever possible
• NIC virtualization (i.e. HP Virtual Connect Flex-10) can help here greatly
Hypervisors – Network Recommendations
eth0 + eth 4
User and
Infrastructure traffic
Embedded (dual port) Add on (quad port)
Hypervisor NIC Bonding
eth2 + eth5
Host Management
and HA traffic
eth1 + eth3
Storage traffic
• Create separate resource pools or clusters for servers and desktops
• Be aware of the limitations of your hypervisor i.e.: • XenServer Pool Size
• Maximum supported VMs per vCenter / SCVMM instance
• Be aware of the specific requirements of you hypervisor i.e.: • Hyper-V save state file
• Resource requirements for vCenter / SCVMM
• XenServer dom0 vCPU and RAM assignment
Hypervisors – General Recommendations
XenDesktop Architecture Review
User
XD Controller Web Interface
Active Directory
License Server
Provisioning Server
Storage
Virtual Desktop
Hypervisor
HT
TP
(s)
XM
L
LD
AP
H
TT
P(s
)
UDP
HT
TP
(s)
/ W
CF
NFS, iSCSI, FC
CIFS
NFS iSCSI
FC
ICA
SQL Server
• CPU and Memory are not typically the bottlenecks
• Disk and Network I/O are!
• With careful design and planning, PVS can be virtualized
• The virtual vs. physical decision depends on several factors: • Number and type of target devices (50 target devices vs. 5000, XA vs. XD, etc.)
• Networking infrastructure and NICs (1 Gb vs. 10 Gb, SR-IOV compatibility, etc.)
• Hypervisor and LACP support (vSphere vs. XenServer)
Provisioning Server Scalability
• PVS cannot cache, but Windows can!
• Block-level storage is recommended • FC, iSCSI and even local disk storage
• No caching by default with CIFS or NFS
• Need to tweak!
• 64-bit is the key to success • It’s all about the File/System Cache!
• Read IOPS from the vDisk Store in the Steady State approach ZERO!
PVS Scalability – Eliminating the Disk Bottleneck
• The number of unique vDisks streamed simultaneously greatly affects scalability • Total RAM Required = 2 GB + (# vDisks x 2 GB)
• Shared storage optimized for write performance is ideal • 80-90% writes is common in VDI deployments
• RAID 1, 10 are good options (RAID 5 is bad)
• More streams = longer failover process • 1,000 streams ≈ 5 minutes
• 1,500 streams ≈ 8 minutes
PVS Scalability – Enterprise Considerations
• Implement at least two PVS servers • Use PVS HA Mode to distribute the load
• In case of different hardware, leverage PVS “Power Rating” feature
• Teaming NICs for throughput is also highly recommended to achieve maximum scalability
• As a rule of thumb, for every 1 Gb NIC, expect ~500 target devices able to be streamed by PVS
• Citrix has scaled a single PVS box up to 3300 target devices • Consulting typically scales each PVS box to 1000-1500 target devices
PVS Scalability – Enterprise Considerations
XenDesktop Architecture Review
User
XD Controller Web Interface
Active Directory
License Server
Provisioning Server
Storage
Virtual Desktop
Hypervisor
HT
TP
(s)
XM
L
LD
AP
H
TT
P(s
)
UDP
HT
TP
(s)
/ W
CF
NFS, iSCSI, FC
CIFS
NFS iSCSI
FC
ICA
SQL Server
• It’s all about the Write IOPS
• 90/10 Write to Read Ratio is common in the steady-state
• RAID 1 or 10 is best, RAID 5 or 6 *not* recommended (unless a
huge amount of spindles or write cache on RAID controller)
• Spread the write cache drives over several LUNs and ensure the
LUNs are sized properly • For example, with 100 desktops and 5 GB write cache drives, consider using
4 or 5 100-150 GB RAID10 LUNs
Storage Usage: Write Cache
• Check registry setting (CTX123570 -
FILE_NO_INTERMEDIATE_BUFFERING)
• Lost of write cache will cause a BSOD
• Things that causes Write Cache activity to be high • Boot / Shutdown / User logging in or off
• User starting application (streamed or local, hosted should have minimal effect)
• Application behavior
• Windows Perfmon <Physical Disk \ Disk Writes/sec>
(\ Disk Transfers / sec gives you the whole picture)
Storage usage: Write Cache (cont‟)
Example (very simplistic):
• 1,000 XD environment (Windows 7, vDisks 20 GB)
Why is “How much disk space do I need for the write cache” a dangerous question?
• Write Cache max. size est. to 5 GB
= Write Cache space 5 TB
11 x 500GB disks (RAID 5)
20 x 500GB disks (RAID 10)
• If we want to be cheap, we opt for SATA
• IOPS est. to peak at 20 IOPS / user
• Estimated 100 users simultaneous (R/W ratio 30/70)
• IO load - Logical
2.000 IOPS (600 reads / 1400 writes)
• IO load - Physical
RAID 5 = 600 + 4*1400 = 6.200 IOPS
over 120 SATA disks required (50 IOPS / disk)
over 40 SCSI disks (150 IOPS / disk)
• HBA must support disk throughput • Especially important for shared HBA scenarios (i.e. Blades)
• Storage controllers must cope with the load • SAN may not be exclusively for VDI
• Prevent concurrent hard disk intensive tasks • Active Scan after anti virus pattern update / Scheduled defrag
• Use only fixed-size VHDs for write-cache drives and Provisioning Services vDisk • Disk can become fragmented on physical media
Storage Considerations
XenDesktop Architecture Review
User
XD Controller Web Interface
Active Directory
License Server
Provisioning Server
Storage
Virtual Desktop
Hypervisor
HT
TP
(s)
XM
L
LD
AP
H
TT
P(s
)
UDP
HT
TP
(s)
/ W
CF
NFS, iSCSI, FC
CIFS
NFS iSCSI
FC
ICA
SQL Server
Virtual Desktop Operating System
• Windows XP • Requires around 5000 IOPS each for startup
• Performed better on XenServer and vSphere than Hyper-V
• Windows 7 • Generates more IOPS than XP for startup / logon (VRC shows +83% for boot)
• However, generates considerably less IOPS than XP when working / idle
• Optimized for virtualized environment (Host Integration Services)
• Windows 7 performed better on Hyper-V
User Group Operating
System
vCPU
Allocation
Memory
Allocation
Avg IOPS
(Steady State)
Estimate
Users/Core
Light Windows XP 1 768MB-1 GB 4-8 10-12
Windows 7 1 1-1.5 GB 5-10 8-10
Normal Windows XP 1 1-1.5 GB 8-14 8-10
Windows 7 1 1.5-2 GB 10-15 6-8
Power Windows XP 1 1.5-2 GB 14-25 6-8
Windows 7 1-2 2-3 GB 15-30 4-6
Heavy Windows XP 1 2 GB 25-50 4-6
Windows 7 2 4 GB 30-60 2-4
Average Resource Allocation
• Windows XP base image
• Uniprocessor HAL: give 2 vCPUs and hypervisor won’t utilize
• Multiprocessor HAL: use 1 vCPU and waste resources while system tries to align processors
Do NOT give virtual desktops more resources than needed
• There is a myriad of desktop optimization guides available
• Good guides are: • Citrix XenDesktop Design Handbook (bit.ly/xdhandbook)
• Windows XP / 7 – Optimization Guides
• CTX124239
• http://www.virtualrealitycheck.net
• http://www.virtualfeller.com
• http://www.citrixtools.net
Desktop Optimizations
XenDesktop Architecture Review
User
XD Controller Web Interface
Active Directory
License Server
Provisioning Server
Storage
Virtual Desktop
Hypervisor
HT
TP
(s)
XM
L
LD
AP
H
TT
P(s
)
UDP
HT
TP
(s)
/ W
CF
NFS, iSCSI, FC
CIFS
NFS iSCSI
FC
ICA
SQL Server
• License check-outs • A standalone Intel® Xeon® 2.83 GHz quad-core processor with 4GB of RAM is
able to handle 248 license check-outs per second (446,400 user / 30 minutes)
• Dell PowerEdge 2650 with a 2.2 GHz processor can handle 170 license check-outs per second (306,000 user / 30 minutes)
• Virtually no CPU resources for check-ins
• Good candidate for virtualization!
License Server Scalability
• The Good – Keeping it Easy
• The Bad – Keeping it Crazy
• The Ugly – Keeping it Real
What„s on Tap?
• Spend extra time gathering and flushing out requirements
The Good – Focus on Requirements (1 of 5)
• Spend extra time gathering and flushing out requirements
• User segmentation / app assessment is key and 3rd party tools can help
The Good – Focus on Requirements (1 of 5)
Minimize Risk in Projects
Pro
of o
f C
on
ce
pt
Asse
ssm
en
t
De
sig
n R
eq
t’s
De
tail
De
sig
n
De
sig
n V
erifica
tio
n
Inte
gra
tion
Te
stin
g
Pilo
t
Bu
ild to
Sca
le
Op
era
tio
n
10-15%
85-90% Spend Risk
Bu
sin
ess D
ev.
Start
Ro
ll O
ut
Analysis Design Build & Test Rollout
• Focus on early phases to minimize the risk and spending
• “Desktop Virtualization” spans many functional areas
• It’s really tough to pull these projects off without the right sponsorship
• Define a clear escalation path, change control process and use it
The Good – Obtain Sponsorship (2 of 5)
• Spend some time developing an actual Architecture Design
• There is a difference between a Conceptual Design and a Detailed Architecture Design
• Engage someone smarter than you (preferably) for a formal Design Review
• Even our smartest, biggest customers call in the “big guns” sometimes
The Good – Architecture Design (3 of 5)
• The success of a desktop virtualization project has a direct correlation to the amount of time and energy spent in the Testing Phase
• User Acceptance Testing (UAT) is our last chance to flush out remaining requirements before an actual Pilot
The Good – Conduct Extensive UAT (4 of 5)
• Separating the operating system from the applications from the user data is absolutely key
• Increases manageability and predictability of each component
• The customers who have been able to do this (or get close) have the happiest users
The Good – Abide by the Layers of Cake (5 of 5)
• The customers who try to roll out things quickly are usually the ones who suffer the most
• This stuff is complicated – take your time!
The Bad – Don„t Rush (1 of 5)
• The “Big Bang” rollout approach usually ends in disaster
• A carefully planned, phased rollout approach is highly recommended to mitigate risk
• Can be structured by department, application, geographic location, etc. – whatever works for your business!
The Bad – Avoid the Big Bang Approach (2 of 5)
• XenDesktop Platinum now includes almost every Citrix product known to mankind
• It’s OK to take baby steps and break the overall project into multiple phases • Ex: “Phase 2” involves external components (NS),
monitoring (ES), etc.
• There is leveraging your investment and then there’s just being insane
The Bad – Ease into Platinum (3 of 5)
• Contrary to popular belief, there is actually a Design and Build & Test Phase in between the Analysis Phase (POC) and Rollout Phase (Pilot)
• Moving directly from POC to Pilot/Production always ends in disaster
• Spend some time designing a scalable architecture and conducting testing the proper way
The Bad – POC to Pilot (4 of 5)
• XenDesktop is designed for specific use cases and applications
• Most Desktop Virtualization projects in the enterprise utilize a hybrid approach (XA+XD)
• XenApp can typically handle about 80% of the use cases and apps, while XD is best suited for the other 20%
• Make sure you are deploying XD for the right reasons
The Bad – Don„t Deploy XD Because it„s Cool (5 of 5)
• Spend some time up-front understanding how much this little project is actually going to cost you
• Don’t just focus on CAPEX (hardware costs, etc.) – OPEX costs are no slouch
• Numbers can be “fudged” in many ways, but the process of doing a ROI/TCO Analysis will help uncover hidden costs you might not have ever thought of if you didn’t do it
The Ugly – Due Diligence (1 of 5)
• Conduct Performance & Scalability Testing if at all possible
• Tools such as ESLT and LoadRunner are here to help
• Uncover potential resource bottlenecks in a lab before they ever occur in production
• Extend UAT and the Pilot if formal P&S Testing cannot be done
The Ugly – P&S Testing (2 of 5)
• The lines are blurred with desktop virtualization and several teams need to work together and understand their specific roles and responsibilities
• PVS is the trickiest component to determine who does what
• The customers who have conducted some type of “Operations & Support Design” are typically the most successful
The Ugly – Operations & Support (3 of 5)
• A little end user education and help desk training can go a very long way
• Take a checkpoint after each phase with all stakeholders
• Making sure everyone is on the same page throughout the project is critical
The Ugly – Communicate (4 of 5)
• DIY is pretty popular in this day and age, but implementing Desktop Virtualization can be tricky without professional assistance or expertise
• A 1 week Design Review or Production Readiness Health Check can be invaluable and does not cost a lot
The Ugly – Do it Yourself (5 of 5)
• Take your time
• You can either have happy users or users that cringe when they see/hear the word “Citrix” – the choice is yours
• Follow the methodology (as easy as it sounds) • Analysis -> Design -> Build & Test -> Rollout
Summary
• Ihre Meinung ist uns wichtig. Bitte nehmen Sie sich einige Minuten Zeit, unseren Online Feedbackbogen auszufüllen. Den Link dazu erhalten Sie einige Tage nach der Veranstaltung.
• Im Anschluss an den Fragebogen haben Sie Zugriff auf die Downloadseite der Präsentationen.
Feedback und Präsentationen