Security testing types

 Security Testing Type

From LinkedInLearning

What is back-end Testing?

 

https://www.guru99.com/frontend-testing-vs-backend-testing.html

Backend testing is a type of testing that checks the Application and Database layer of a 3 Tier Architecture.

API Testing -> Business Layer

Database Testing -> Database Layer

Database Testing

 It means that data entered in the front end will be checked in the back-end database. 

Databases are checked for ACID properties, CRUD operations, their Schema, Business rule conformance. Databases is also checked for Security and Performance.

In back-end testing, there is no need to use the GUI. You can directly pass the data using a browser with the parameters required for the function to get the response in some default format. For Example, XML or JSON. You also connect to the database directly and verify the data using SQL queries.

• Structural testing deals with the database inner structure (i.e. metadata). It involves validation of Tables and all the other database objects that aren’t directly accessible by users, like Columns, Keys, Indexes, Schema, Triggers, Functions, etc. It can also involve testing your DB servers.
• Functional testing focuses on the way data is mapped from the frontend to the database. (e.g. whether clicking a button correctly affects the related tables/columns).
• Non-functional testing checks how the database performs under the expected loads and extreme levels of stress. Security tests also belong here 

• 
  

What is Context Driven Testing

https://context-driven-testing.com/

Context Driven Testing is a mindset shift (or School of testing) developed by Cem Kaner, James Bach & Bret Pettichord. Details about it can be found in their famous book: Lessons Learned in Software Testing.

There are 7 basic principles to it. The following are directly picked from their book:

#1) The value of any practice depends on its context.

#2) There are good practices in context, but there are no best practices.

#3) People, working together, are the most important part of any project’s context.

#4) Projects unfold over time in ways that are often not predictable.

#5) The product is a solution. If the problem isn’t solved, the product doesn’t work.

#6) Good software testing is a challenging intellectual process.

#7) Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

Context-driven testers choose their testing objectives, techniques, and deliverables (including test documentation) by looking first to the details of the specific situation, including the desires of the stakeholders who commissioned the testing. The essence of context-driven testing is project-appropriate application of skill and judgment. The Context-Driven School of testing places this approach to testing within a humanistic social and ethical framework.

Ultimately, context-driven testing is about doing the best we can with what we get. Rather than trying to apply “best practices,” we accept that very different practices (even different definitions of common testing terms) will work best under different circumstances.

What is solid principle

https://medium.com/@mari_azevedo/s-o-l-i-d-principles-what-are-they-and-why-projects-should-use-them-50b85e4aa8b6
https://medium.com/better-programming/solid-principles-simple-and-easy-explanation-f57d86c47a7f

S.O.L.I.D is an acronym that represents five principles of object-oriented programming and code design theorized by our beloved Uncle Bob (Robert C. Martin) by the year 2000. The author Michael Feathers was responsible for creating the acronym:

[S]ingle Responsibility Principle
[O]pen/Closed Principle
[L]iskov Substitution Principle
[I]nterface Segregation Principle
[D]ependency Inversion Principle

Single Responsibility Principle:

A class should have one, and only one, reason to change.

One class should serve only one purpose. This does not imply that each class should have only one method, but they should all relate directly to the responsibility of the class. All the methods and properties should work towards the same goal. When a class serves multiple purposes or responsibilities, it should be made into a new class.

Open-Closed Principle:

Entities should be open for extension, but closed for modification.

Software entities (classes, modules, functions, etc.) should be extendable without actually changing the contents of the class you’re extending. If we could follow this principle strongly enough, it is possible to then modify the behavior of our code without ever touching a piece of the original code.

Liskov Substitution Principle:

The Liskov Substitution principle was introduced by Barbara Liskov in her conference keynote “Data abstraction” in 1987. 

Robert Martin made the definition smoother and more concise in 1996:

Functions that use pointers of references to base classes must be able to use objects of derived classes without knowing it.

Or simply: Subclass/derived classes should be substitutable for their base/parent class.

It states that any implementation of an abstraction (interface) should be substitutable in any place that the abstraction is accepted. Basically, it takes care that while coding using interfaces in our code, we not only have a contract of input that the interface receives, but also the output returned by different classes implementing that interface; they should be of the same type.

Interface Segregation Principle:

A client should not be forced to implement an interface that it doesn’t use.

This rule means that we should break our interfaces into many smaller ones, so they better satisfy the exact needs of our clients.

Similar to the Single Responsibility Principle, the goal of the Interface Segregation Principle is to minimize side consequences and repetition by dividing the software into multiple, independent parts.

Make fine grained interfaces that are client specific.

Dependency Inversion Principle:

High-level modules should not depend on low-level modules. Both should depend on abstractions.

Abstractions should not depend on details. Details should depend on abstractions.

Or simply: Depend on abstractions, not on concretions.

By applying the Dependency Inversion Principle, the modules can be easily changed by other modules just changing the dependency module. Any changes to the low-level module won’t affect the high-level module.

What is Embedded Software Testing?

From guru99

In this tutorial, you will learn

• What are Embedded systems?
• What is Embedded Testing?
• Embedded Software Testing Types
• Difference: Embedded testing and Software Testing
• Challenges: Embedded Software Testing

What are Embedded systems?

Embedded systems are the electronically controlled devices where software and hardware are tightly coupled. Embedded systems may contain a variety of computing devices. These are PCs incorporated in other devices to operate application-specific functions. The end user usually is not even aware of their existence.

What is Embedded Testing?

EMBEDDED TESTING is checking the functional and non-functional attributes of both software and hardware in an embedded system. The purpose of Embedded test is to verify and validate the Embedded software as well as hardware against client requirement.

Embedded Software testing checks and ensure the concerned software is of good quality and complies with all the requirements it should meet. Embedded software testing is an excellent approach to guarantee security in critical applications like medical equipment, railways, aviation, vehicle industry, etc. Strict and careful testing is crucial to grant software certification.

How to perform Embedded Software Testing

In general, you test for four reasons:

• To find bugs in software
• Helps to reduce risk to both users and the company
• Cut down development and maintenance costs
• To improve performance

In Embedded Testing, the following activities are performed:

1. The software is provided with some inputs.

2. A Piece of the software is executed.

3. The software state is observed, and the outputs are checked for expected properties like whether the output matches the expected outcome, conformance to the requirements and absence of system crashes.

Embedded Software Testing Types

Fundamentally, there are five levels of testing that can be applied to embedded software

Software Unit Testing

The unit module is either a function or class. Unit Testing is performed by the development team, primarily the developer and is usually carried out in a peer-review model. Based on the specification of the module test cases are developed.

Integration Testing

Integration testing can be classified into two segments:

1. Software integration testing
2. Software/hardware integration testing.

In the end, the interaction of the hardware domain and software components are tested. This can incorporate examining the interaction between built-in peripheral devices and software.

Embedded software development has a unique characteristic which focuses on the actual environment, in which the software is run, is generally created in parallel with the software. This causes inconvenience for testing since comprehensive testing cannot be performed in a simulated condition.

System Unit Testing

Now the module to be tested is a full framework that consists of complete software code additionally all real-time operating system (RTOS) and platform-related pieces such as interrupts, tasking mechanisms, communications and so on. The Point of Control protocol is not anymore a call to a function or a method invocation, but rather a message sent/got utilizing the RTOS message queues.

System resources are observed to evaluate the system's ability to support embedded system execution. For this aspect, gray-box testing is the favored testing method. Depending on the organization, system unit testing is either the duty of the developer or a dedicated system integration team.

System Integration Testing

The module to be tested begins from a set of components within a single node. The Points of Control and Observations (PCOs) are a mix of network related communication protocols and RTOS, such as network messages and RTOS events. Additionally to a component, a Virtual Tester can likewise play the role of a node.

System Validation Testing

The module to be tested is a subsystem with a complete implementation or the complete embedded system. The objective of this final test is to meet external entity functional requirements. Note that an external entity either be a person, or a device in a telecom network, or both.

Difference: Embedded testing and Software Testing

Software Testing	Embedded Testing
Software testing is related to software only.	Embedded testing is related to both software as well as hardware.
On average 90% testing done in the world is purely manual black box testing.	Embedded testing is done on embedded systems or chips it can be a black box or white box testing.
Primary areas of testing are GUI checks, functionality, validation and some level of database testing.	Primary areas of testing are the behavior of the hardware for the no. of inputs given to it.
Software testing is majorly performed on client-server, web and mobile based applications.	Embedded testing generally performed on the Hardware.
e.g., Google Mail, Yahoo Mail, Android applications.	e.g., Machines of healthcare domain, Microcontrollers used in computers.

Challenges: Embedded Software Testing

Some of the challenges that one can face during Embedded software testing:

Hardware Dependency

Hardware dependency is among the main difficulties faced during embedded software testing because of limited access to hardware. However, Emulators and Simulators may not precisely represent the behavior of the actual device and could give a wrong sense of system performance and application's usability.

Open Source Software

The majority of the embedded software components are open source in nature, not created in-house and absence of complete test available for it. There is a wide range of test combinations and resulting scenarios.

Software vs. Hardware Defects

Another aspect is when software is being developed for a freshly created hardware, during this process high ratio of hardware defects can be identified. The found defect is just not limited to software. It may be related to hardware also.

Reproducible Defects

Defects are harder to reproduce/recreate in the case of the embedded system. That enforces the embedded testing procedure to value every defect occurrence substantially higher than in a standard case, other than to gather as much data as could sensibly be required to alter the system to find the foundation of the defect.

Continuous Software Updates

Embedded systems require regular software updates like the kernel upgrade, security fixes, different device drivers, etc. Constraints identified with the software updates influence makes bug identification difficult. Additionally, it increases the significance of build and deployment procedure.

Summary

There are some difficulties in testing embedded software testing that makes it more difficult than regular software testing. The most fundamental issue is the tight reliance on the hardware environment that is prepared simultaneously with the software, and that is regularly required to perform reliable software testing. Sometimes it is even difficult to test the software without custom tools, which effortlessly makes concentrating on testing in late stages exceptionally enticing.

One of the most important things is that you should think about is the fact that you should often opt for automated software testing. The embedded automated testing is a quicker process which would take some hours to complete, and in this way, the issue of your software is settled.

Security testing

• What is Security Testing

a type of software testing that intends to uncover vulnerabilities of the system and determine that its data and resources are protected from possible intruders.

• Types of Security Testing

• How to do Security Testing

corresponding Security processes to be adopted for every phase in SDLC

 

Security Test should start as early as possible

The test plan should include
Security-related test cases or scenarios
Test Data related to security testing
Test Tools required for security testing
Analysis of various tests outputs from different security tools

• OWASP Open Web Application Security Project (OWASP) 

Testing Guide: https://www.owasp.org/index.php/Category:OWASP_Testing_Project

https://www.owasp.org/index.php/OWASP_Testing_Guide_v4_Table_of_Contents

• Tools

netsparker: Vulnerability Scanning tool, scan a system against known vulnerability signatures.

Acunetix:  penetration testing,  simulates an attack from a malicious hacker,  check for potential vulnerabilities to an external hacking attempt

.....

Performance Testing

• What is Performance testing

Non Functional Testing
To ensure software applications will perform well under their expected workload. like its response time, reliability, resource usage and scalability
demonstrate that your software system meets certain pre-defined performance criteria. 

Speed - Determines whether the application responds quickly
Scalability - Determines maximum user load the software application can handle.
Stability - Determines if the application is stable under varying loads

• Types of Performance testing

Load testing - checks the application's ability to perform under anticipated user loads. The objective is to identify performance bottlenecks before the software application goes live.Some common performance bottlenecks are CPU utilization,
Memory utilization,Network utilization,Operating System limitations,Disk usage.
Stress testing - involves testing an application under extreme workloads to see how it handles high traffic or data processing. The objective is to identify the breaking point of an application.
Endurance testing - is done to make sure the software can handle the expected load over a long period of time.
Spike testing - tests the software's reaction to sudden large spikes in the load generated by users.
Volume testing - Under Volume Testing large no. of. Data is populated in a database and the overall software system's behavior is monitored. The objective is to check software application's performance under varying database volumes.
Scalability testing - The objective of scalability testing is to determine the software application's effectiveness in "scaling up" to support an increase in user load. It helps plan capacity addition to your software system.

• When to do performance testing

Performance Testing is always done for client-server based systems only.

• How to do Performance Testing

• Tools

LoadNinja – is revolutionizing the way we load test. This cloud-based load testing tool empowers teams to record & instantly playback comprehensive load tests, without complex dynamic correlation & run these load tests in real browsers at scale. Teams are able to increase test coverage. & cut load testing time by over 60%.

NeoLoad - is the performance testing platform designed for DevOps that seamlessly integrates into your existing Continuous Delivery pipeline. With NeoLoad, teams test 10x faster than with traditional tools to meet the new level of requirements across the full Agile software development lifecycle - from component to full system-wide load tests.

HP LoadRunner - is the most popular performance testing tools on the market today. This tool is capable of simulating hundreds of thousands of users, putting applications under real-life loads to determine their behavior under expected loads. Loadrunner features a virtual user generator which simulates the actions of live human users.

Jmeter - one of the leading tools used for load testing of web and application servers.

What is TDD, BDD

Learning from guru99

TDD, test driven development, test first development, running test before development

Suitable for "detailed specification"

Two types

• TDD test driven development, normally means Developer TDD
• BDD Behavior driven development, also called ATDD acceptance test driven development

BDD vs TDD

From the above diagram, we can see that 

When the team receives a new function requirement

1. BDD testers

testers design acceptance test cases according to requirement. Only If the test failed, developers would write new code.

2. TDD developer

developers need to write new code to fulfill the new requirement

design automated unit test first, then run the test and if the test failed, do refactoring

add test->run test->fail then refactoring

The primary goal of TDD is to make the code clearer, simple and bug-free.

More quality code

Defects Management

When logging a defect, which information should be included?

• Defect_ID
• Defect Description/title - Detailed description of the defect including information about the module in which defect was found.
• Version - Version of the application in which defect was found.
• Reproduce Steps - Detailed steps along with screenshots with which the developer can reproduce the defects.
Expected Result, Actual Result
• Date Raised - Date when the defect is raised
• Reference- where in you Provide reference to the documents like . requirements, design, architecture or maybe even screenshots of the error to help understand the defect
• Detected By
• Status - Status of the defect
• Fixed by - Name/ID of the developer who fixed it
• Date Closed - Date when the defect is closed
• Severity
• Priority

• Severity VS Priority

Severity describes the impact of the defect on the application/system,  for example, the system is still working or broke down.
Critical: Nothing can proceed further
Major: Certain parts of the system remain functional
Medium: The system is still functional
Low: It won't cause any major break-down of the system

Priority is defined as the order in which a defect should be fixed. Higher the priority the sooner the defect should be resolved.
Low: The Defect is an irritant but repair can be done once the more serious Defect has been fixed
Medium: During the normal course of the development activities defect should be resolved. It can wait until a new version is created
High: The defect must be resolved as soon as possible as it affects the system severely and cannot be used until it is fixed

Some defects may be low severity, but high priority. For example, wrong company log doesn't affect the major function, but it definitely need be corrected before releasing.

Some defects may be high severity, but low priority. For example, typing random symbols in number text box may cause system crash, but it is a rare case. Then it's a low priority defect.

QA can't decide the severity and priority. Normally product manger will be responsible for that.

• Defect Status

New, Assigned
Open, Fixed, Failed, Confirmed/Verified, Closed, Reopened
Duplicate, Not a Defect
Deferred, Rejected


QA->QA Lead (approved)->Dev Lead, PM(reviewed)->Dev (Assigned, Fixed, Rejected)->QA(retest,pass,failed)->Dev

Types of testing

Get more understanding on some terminology which I was confused

• Differences between Alpha Testing, Beta Testing, UAT user acceptance testing

Alpha Testing: At the end of development, in House testing, Visual or simulation user environment, Done by QA or other team members

Beta Testing: Last step before releasing, Done by wildly end users, in user's environment

UAT user acceptance testing: Done by client or user, Verify if the system meets customer requirement. Alpha testing and Beta testing are both user acceptance testing.

In Agile development, UAT can be done in any phase.

• Sanity testing vs Smoke testing

diagram from guru99

Sanity Testing: 

testing new functionalities or minor changes, new fixes, relevant areas which may be affected by the changes
a subset of regression testing that focuses on one or a few areas of functionality
narrow and deep
unscripted, rough set of test cases
like specialized health check up

Smoke Testing:

first thing to be done on a new build, decide if the build will be accepted or rejected for deep testing
test the most critical basic functionality
wide and shallow testing
scripted either written or automated
a subset of acceptance testing
like General Health Check Up

holds high regards in SCRUM and Agile
individual QA's perform this testing for the stories that they own, the first 2-3 criteria of each user story 
automation



As per the needs of testing, you may have to execute both Sanity and Smoke Tests in the software build. In such cases, you will first execute Smoke tests and then go ahead with Sanity Testing. In industry, test cases for Sanity Testing are commonly combined with that for smoke tests, to speed up test execution. Hence, it's a common that the terms are often confused and used interchangeably.

• ad hoc testing

random testing. Error Guessing. Error guessing can be done by the people having enough experience on the system to "guess" the most likely source of errors.

• usability testing

who: a small set of target end-users

what: focuses on the user's ease to use the application, if user friendly

when: during the initial design phase of SDLC

• exploratory test

who: testers

what: test design and test execution at the same time, testing as "thinking" activity

when: test cases are not created, with or without requirement

• unit test

unit test:
white box testing
usually done in development
to isolate a section of code and verify its correctness

tools: Junit, Nunit

Test Plan

STLC (software testing life cycle)
Requirement Analysis
Test Planning
Test Case Development
Environment Setup
Test Execution
Test Cycle Closure/ Evaluating Exit Criteria and Reporting


What does a test plan include? 
(From ISTQB)

• test plan: A document describing the scope, approach, resources and schedule of intended test activities. It identifies amongst others test items, the features to be tested, the testing tasks, who will do each task, degree of tester independence, the test environment, the test design techniques and entry and exit criteria to be used, and the rationale for their choice,and any risks requiring contingency planning. It is a record of the test planning process.

What is the benefits of a test plan?
(From ReQtest)
It is the guide book for the testing process. It directs your testing approach and describes the testing practices to be followed.
It contains details of the testing scope which prevents team from putting any efforts in testing ‘Out of scope’ functionalities.
It helps to determine the time and effort required for testing the product.
It clearly defines roles and responsibilities of every team member so every individual in the testing team knows what is required of him.
It provides schedule for testing activities. Hence, it provides you a baseline schedule to control and track your team’s testing progress.
It outlines the resource requirements and equipment needs which are essential to carry out the testing process.
It can be shared with your client to give them insight about your testing process and gain their confidence.

What is a good test plan?
Simple, Clear, Easy to Read and Follow.

How to write a test plan?
First make sure what contents need to be included in the the test plan.
For example

• Introduction – what is the project all about?
• In-Scope – what will we be testing?
• Out-Of-Scope – what won’t we be testing?
• Entry Conditions – when can we start?
• Exit Conditions – when can say we’re done?
• Timescales/Schedule – how long do we think it’ll take?
• People – who’ll be working on the project?
• Test Environment – where will we best testing on?
• Risks – what could go wrong?

Then Choose a Template. There are many templates online. Compared to the old lengthy test plan, One Page Test Plan means to put or fit the most important test plan contents into one page. Therefore the readers can get a good and clear view.
Talking about "one page test plan", I found this dashboard style was really clever.

(From https://testsheepnz.blogspot.com/) 

A good test plan is very necessary and helpful.

SDLC software development life cycle

SDLC software development life cycle

Planning and Requirement Analysis
Defining Requirement
Designing Product Architecture/ System design
Building or developing the product
Testing the product
Deploying in the market and Maintenance