• Standard performance/scalability test suite/buildout

last modified January 8, 2009 by rossp

http://dev.plone.org/collective/browser/collective.loadtesting


  • Applications
    • Baseline performance benchmarks for core plone development, buildbot
    • Developers/Integrators evaluating add-on performance impacts
    • Deployment configuration performance evaluation 
  • Well-documented buildout
    • Reproducible buildout
    • Good documentation for all goals
  • Load testing tool
    • Must be able to aggregate statistics on each request type (view vs add vs edit vs workflow change)
    • Ease of use and ease of extending trumps clustering
    • JMeter
      • Pros
        • Wide acceptance, everyone uses it
        • supports clustered load testing
      • Cons
        • Java
        • Can only do with responses what JMeter supports
        • ugly regexp stuff
    • Funkload
      • Pros
        • Python
        • Test plans are Python classes, easier complex test plans
        • Can use Python to handle/extract from responses
      • Cons
        • doesn't support clustered load testing
  • Test scenarios and content profiles
    • Core buildout will have 2 test scenarios and 2 content profiles for a 2x2 grid of 4 load tests
    • Test Scenarios
      • Read-only public facing CMS
      • Write-intensive intranet
    • Content profiles
      • Small: few users and groups, few folders, little bit of content
      • Large: many users and groups, many folders, some large folders, lots of content, lots of binary content
      • Have the buildout generate GenericSetup profiles so that they can be used for purposes other than just load testing
  • Blogs