In the space of startup, you will always find yourself with resource constaints. You do not have much but you need to still build, test, and deploy quickly for sales. If you want to survive, you need to build; and you need to build fast. There is no time for 10 steps feature evaluation and delay for another month. You have to build on day 1, learn on day 2, and deploy by day 4. Soon, you will reach a point when you introduce in a new feature, you introduce more bugs into existing workflows. When do you start considering QA?

What is QA in the software sense?

Checking whether a product meets the required specifications, customer expectations, and legal compliance.

The QA Principles

1. The ability to prevent defects rather than fidning defects

2. The ability to identify risks and challenges (functional and nonfunctional

3. The ability to ensure that processes are supporting/favourable to achieve quality both at code & application level

4. The ability to keep continuous improvement in the process/techniques

5. The ability to work towards reducing development cycle time

6. The ability to evaluate and monitor the development process quality

7. The ability to maintain high standards in test selection and execution

What is the cost of preventing defects in the Software Development Cycle

When is automated QA required in a startup?

When your sprint iterations slows down, and more testers are involved for each release, automation will then create a leverage for you.

Typically start with a manual tester who can execute tests and update test-cases on the fly before every new build is good to create a high standard of quality. Good manual testers would cost relatively lower with more coverage. They can test things automation typically cannot (human eye performance, layout issues, edge case testing, exploratory testing).

The outcome of manual testing is documentation on the product, laying down the foundations of your automated testing.

QA to Engineer ratios do not make sense in startup. Not enough money to hire people to perform single-hatted roles. Do consider QA cost of testing in sprint planning as well. During sprint planning, whatever tasks committed should also be followed with a test to write automated test. With the added costs on whatever is required for testing, we know the size of the sprint + ensure coverage.

We schedule test case reviews for major features. We also run a QA triage every time we do full regression after any features go into an integration phase with everything else going into a release. Startups may be small, but communicating issues and visibility cannot be taken for granted. This is especially true when scaling.

ATDD => Acceptance Test Driven Development. QA, Design, and Engineering collaborates very early in the development process to establish Acceptance Criteria. This way, it cuts risk, scope creep, and manages expectation. Automations can request for its requirements up front

What is the role of a QA in agile?

1. Attend sprint planning sessions

Within the sprint, they need to plan out how much time they need to test each of the feature. The estimation process software engineers go through to plan for their features, QA does the same in the space of testing.

2. Attend daily stand-ups

A member of the QA team should always attend daily standup. Being part of the sprint helps them to stay up to date with the sprint progress, allowing them to plan their workload. If a tester has a blocker, they can bring it up during stand-up. A QA’s update in daily standup helps engineers have a sense of hows the testing progress and plan their workload in the sprint as well.

3. Test through out the sprint

Find out the bugs in the features as early as possible. This will leave space and time within the sprint for the features’ bugs to be fixed and completed within the sprint. Integrating test and development allows the 2 teams to work closely and resolve issues faster, resulting in higher quality results.

4. Meet with developers for short, hand-off demonstrations

once a feature is done, you can consider scheduling a handoff demonstration so the QA can find out exactly how the feature works. This gives room for the QA to ask the developer any question. If you want to work in a more asynchronous manner, you can consider getting your engineers to take screen recording of the feature and ask questions from there.

5. Attend sprint retrospective

Sprint retrospective’s objectives article reference:

Find out what went well during the previous sprint cycle, and talk about what can be improved for the next sprint.

Whatever that went well, you should examine a few aspects including: people, relationships, processes, tools. Push a layer deeper into what allowed the sprint to be successful? Example: was the tasks planned in sprint well scoped and well spec-ed with clear QA acceptance criteria with good communication between the testing team and development team?

Typically done before sprint planning so you can curate lessons learnt from the previous sprint and apply new solutions for the next sprint. Example, you may have over estimated one of the tasks that you have added into the sprint. What was the cause of over-estimation. Complexity or Scope of feature was under estimated. Now you can get a better sense of the effort moving forward for similar tasks as that.

6. Document test cases

Your documentations will be live. They are bound to change, so write it lean with the expectation of change.

The importance of documentation

1. A single source of truth saves time and energy

Your team is starting a project that’s new to all of you. You heard through the grapevine that your company successfully tackled something similar just a couple of years ago. The only problem? Nobody on your current team was actually around during that time.

Documentation encourages knowledge sharing, which empowers your team to understand how processes work and what finished projects typically look like.

1. Start with what does your reader need to get out of your documentation.

An example of a need: User needs to set up CLI.
It can be narrow tasks or broad ones such as the current architecture setup to facilitate understanding.

A documentation is useful when it actually helps the user get “that one thing” done.
So this question is extremely important

2. Write less
   Good documentation should be lean and short. Think of what is the amount of documentation you are able/willing to commit to. Commit to update 6months from now it will still be updated.

Once you have out-dated documentations, you are training users to expect nothing from the docs. so focus on weeding out whatever that is useless from the odcumentation.

3. Write the outline first

As with all articles, plan out the flow. Remember point 1. How can your documentation help your reader achieve the 1 objective they set out to do.

Once you have the steps down, and these steps helps the user get something done. Put in bullet points then the details thereafter. Just covering the points you outlined can help you to write less.

4. Rubber ducking documentations works
   It is useful to unblock mental blocks

5. Write readably
   Write with structure. Use meaningful headers. Short paragraphs. One idea per paragraph. Keep under 25 words per sentence. Front load a sentence by putting the important part first.

Example:

“You need to do X if Y is the case” => “If Y is the case, do X”

“There is a Z that can be used to do X” || “Using Z, it is possible to do X” => “to do X, use Z”

• Tell them exactly what you need to do.

Heading: “Custom code generation” => Heading: “Writing a code generator”

• Connect your headings to whatever your user is trying to get done.

6. Its not just about documentation

Don’t write for the sake of it. Its written to help people get stuff done.

Source: testlodge.com

What are some QA Metrics we need to look out for?

Reference: QA Metrics

Absolute QA Metrics

1. Total number of test cases
2. Number of passed test cases
3. Number of failed test cases
4. Number of blocked test cases
5. Number of identified bugs
6. Number of accepted bugs
7. Number of rejected bugs
8. Number of deferred bugs
9. Number of critical bugs
10. Number of determined test hours
11. Number of actual test hours
12. Number of bugs detected after release

#TLDR

In short, you want your QA process to be based on clear, concise documentation on the test-cases. You want your QA to be part of your development process from day 0 (Sprint planning). You want them to plan for a feature’s test cases just as how you would want a developer to plan for a feature development.

Your QA’s metrics should go

Reference

Should early stage startup be bothered with QA?