end-to-end performance tuning brian davies gridpp28 manchester 2012
TRANSCRIPT
![Page 1: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/1.jpg)
End-to-End performance tuning
Brian DaviesGridpp28
Manchester 2012
![Page 2: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/2.jpg)
18/04/2012 End-End performance tuning 2
Areas of Improvements• FTS
– # Streams– # Files– Tx Timeout– Preparing File Ratio– Copy Mechanism
• G-U-C vs srmcp• Asynchronous Ptp
• Access Method– Synchronous vs Asynchronous calls– Scheduled vs unscheduled connections.– Srm-less – Federated xrootd
• Costs as well as benefits• TCP/ Window Tuning• Jumbo Frames
– Some changes only see benefit in end site also make changes.• WN-WAN tuning
– Tcp Window size– ‘Nat’ing– Xrootd configuration/optimisation.
![Page 3: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/3.jpg)
18/04/2012 End-End performance tuning 3
Other areas of optimisation??
• Homogenous network– Mixing WN and Disk Servers can easily reduce network
bandwidth
• VM components– Splitting services
• Databases– Type– Separate node– Internal operation
• Cleansing– EG DPM requests DB
• DNS Aliasing of multiple machines• Knowledge transfer
![Page 4: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/4.jpg)
18/04/2012 End-End performance tuning 4
Who does optimisation and when is the benefit seen.
• Some optimisations can be unilateral.• FTS tuning, database improvements etc.
– Others have to be co-ordinated• Jumbo Frames
• Some optimisation benefits seen straight away.– Others require end host to make changes before benefit is
seen.
• An “improvement” at one site may degrade another site.– TCP tuning at QMUL/RHUL
![Page 5: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/5.jpg)
18/04/2012 End-End performance tuning 5
Differences between T1 and T2s?
• (Some) Current T1 issues will/are problems that a “large” T2 will face in X* years time.
• Problems a T2 are facing now may have been solved by (a) T1 Y* years ago.
• A site should/does not need try to re-solve a problem(s) that another site(s)/tier has solved in the past.
• *(Where X and Y are to be determined).
![Page 6: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/6.jpg)
18/04/2012 End-End performance tuning 6
Manchester Network improvements
•Transfers from Manc’ to T1s
![Page 7: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/7.jpg)
18/04/2012 End-End performance tuning 7
Manchester network improvements
•Reverse direction
![Page 8: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/8.jpg)
18/04/2012 End-End performance tuning 8
Manchester Network improvements
•Transfers from Manc’ to UK T2s
![Page 9: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/9.jpg)
18/04/2012 End-End performance tuning 9
Manchester Network improvements
•Transfers from Manc’ to Non UK T2s
![Page 10: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/10.jpg)
18/04/2012 End-End performance tuning 10
Manchester Network improvements
![Page 11: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/11.jpg)
18/04/2012 End-End performance tuning 11
Effect of streams on RALPPD-T1s
• Failing transfers required number of threads per transfer to have to be reduced to 1 per file transfer.`– Deemed more important to receive data from CERN than
getting good rate for other sites
• Mitigated overall throughput by increasing number of concurrent transfers.– Individual file transfer speed reduced and latency increased.
• Once underlining issue re-solved returned to previous settings.
![Page 12: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/12.jpg)
18/04/2012 End-End performance tuning 12
Effect of streams on T1s-RALPPD
•#Streams reduced/Increased•Effects transfers TO the site.
Effects Some T1s more than others
![Page 13: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/13.jpg)
18/04/2012 End-End performance tuning 13
Effect of streams on RALPPD-T1s
No (known) changes in streams in reverse direction
![Page 14: End-to-End performance tuning Brian Davies Gridpp28 Manchester 2012](https://reader035.vdocument.in/reader035/viewer/2022081603/56649f335503460f94c50ae4/html5/thumbnails/14.jpg)
18/04/2012 End-End performance tuning 14
Other areas of optimisation??
• “In the wiki”– But where?
• Which wiki?
• ????