Test phases

Security (SEC) and Penetration (PEN) testing are generally part of a programme's non-functional requirements (NFR) testing (NFT), along with performance and volume (P&V) and failover and recovery (F&R).

Security Testing

Security Testing, an important practice within a modern test campaign, includes testing for source code vulnerabilities, network and systems security, and information security such as protection of personal identification information. This technical testing that needs to be completed professionally and efficiently.

Security and Penetration Comparison

"SEC" - Security testing is generally white or grey box testing, validating the security architecture and design;
In comparison,
"PEN" - Penetration testing is late stage, black box testing, validating the fully assembled delivery.

Security Test Phases

MTP – Master Test Planning - Decide on test phase timing, scope & approach
SEC1 – Early Phase Security Testing - Early warnings, secure platform, process
SEC2 – Later Phase Security Testing - Complete re-tests; ISO acceptance
SAT – Systems Acceptance Testing - Systems accepted and locked down
PEN – Penetration Testing - Final external analysis with penetration approach

Essential to plan and execute security testing professionally

Security Testing is a program level test campaign integrated with UNIT, SYS, SIT, UAT, SAT etc. It’s essential that it’s planning is integrated with the others phases and scope and coverage agreed early (Master Test Planning)
Significant stakeholders are the Program Test Office (Program Test Manager) and the Information Security Office (Information Security Manager and Information Risk Manager)
Multiple test phases are likely to secure the test platforms early and attain final ISO acceptance
Penetration testing in comparison is often executed at the very end so that there is no requirement to re-test.

Security and Penetration Testing Comparison

Owner

SEC often is managed as part of a programme's larger test campaign; whereas PEN testing often still is considered too specialised and managed instead by the Information Security Office (ISO), though this need not necessarily be the case. The ISO needs to be confident in the security profile to allow change to migrate to, or stay in, production.

Focus

SEC testing's focus is verifying and validating the security characteristics of the architecture and design, and specifically that controls required by the ISO have been successfully commissioned.

Approach

SEC testing is inherently white-box. The architecture and design generally, and particularly the security architecture and design should be available, and necessarily inspected as part of the verification process. Specifically security controls are the obvious target of security test design and must be included; PEN testing in it's purest "penetration" approach is black box - i.e. the testing challenge is for an external agency, without the benefit of the programme's architecture and design, to penetrate or circumvent the controls in place. It's similar to the classic "double-blind" testing that has been held in such high regard in test theory. Of course the more information the would-be penetrator can access the better their chances of finding a vulnerability. That's why there is necessary secrecy around (i) security architecture and design and (ii) security defects discovered. Security defects really do need to be kept to a limited audience, even when they have been remedied because historical vulnerabilities may point to current inadequacies.

Timing

PEN testing is generally the last phase of testing, just before release. This way it doesn't need to be repeated.
SEC testing is more flexible as to its timing in the overall test campaign, but having both early and late phase is good practice to give sufficient time for remediation and retesting.

Target Environment

PEN testing is ideally delivered on the final production system, otherwise too much is being relied on the environment change management. SEC testing can be, and should be, carried out on a number of environments - e.g. DEV, TST and Pre-Production systems. This way development environments can be validated and early notification of security defects gained. Not all tests are valid on all environments though and differing levels of system access have to be accommodated in the different environments.

SEC Testing Methodologies

Security Testing Bodies, Frameworks and Concepts

General Concepts

Information Security Management describes activities that relate to the protection of information and information infrastructure assets against the risks of loss, misuse, disclosure or damage. Information security management (ISM) describes controls that an organization needs to implement to ensure that it is sensibly managing these risks.

Australian Frameworks

PSPF Protective Security Policy Framework from the Australian Government provides the appropriate controls for the Australian Government to protect its people, information and assets, at home and overseas.

Information Security Manual from Australia's Defense Signals Directorate

Australian Information Security Association. SANS in Australia.

American Frameworks

NIST Standard from USA. It's computer security division has details of frameworks to use as a basis for security testing. In particular specific standards exist for logging, mobile devices, BIOSs, wifi, key management, and much more. A good technical resource on security.

SANS is "the most trusted source for computer security training, certification and research". It's is a private US company that specializes in internet security training. Originally for - SysAdmin, Audit, Networking, and Security, but now much broader. See wikipedia entry.