From last 6-7 months, I was working closely on load testing of a couple of application. While working on it I realized few capabilities a responsible person should have.
1) One should have a good understanding of the application domain, even more than a functional tester. If one needs to craft the load test plan then he should be aware of how the user does use it. He should clarify his assumptions of how the system will be used. Recommended way is to observe few users, to understand better about application usage.
2) one should have a better understanding of the application architecture, how components interact? What is the difference between non-prod and prod system from an infrastructure perspective? What will be the network bandwidth? What will be the average think time? etc.
3) One should be technically strong and creative enough to gather the load test data or way to generate the load test data. This one is very critical to building independent, low maintenance, automated load test. One can have a utility which will run prior to the Load test to generate test data. A process of test data management is the key to having successful continues delivery with zero human interaction.
4) One should be able to identify the peak load details based on historical data. If the historical data is not available then one needs to come up with the approximate numbers. As load test may run for 1-2 hour, one needs to have a good combination of different transactions. NOTE: once you baseline the performance of each transaction through the full test (1-2 hours), in CD you can have tested for lesser numbers just to verify if the build is not impacting the baseline numbers. For major builds, one can run the full test.
5) And the key and important one, expert knowledge of the load testing tool, protocols, command-line or configuration options, how the tool works etc.