Thoughts and Practice on Quality Standardization

a definition

1 Quality standardization

What is quality standardization? To use an inappropriate analogy, everyone knows the assembly line operation of the factory. What the people or machines in each position have to do is regulated and defined in advance. The products on the assembly line go through the same process from raw materials to finished products. A set of processes, so the quality of the finished products will not vary too far. There are certain similarities between R&D process and assembly line operation. We hope that by standardizing the R&D process of each business line, we will use a better business line as a standard model room to standardize a set of standardized processes and apply it to other business lines, so that each business line can carry out high-level research and development. Quality delivered.

2 quality built-in

Many students in the R&D team think that the quality is protected by the test students, and they will think that the quality problems of the product are the fault of the test. This is a narrow understanding of the test work and quality assurance. Test students are the patron saints of quality, but quality cannot be determined only by test students at the last stage. If you want to deliver high-quality products and experiences to users, you must abide by the quality standards in all aspects of the R&D process. Students in product, development, and testing in the stages of demand review, program design, development, and self-test must have quality awareness and Strictly implement the quality norms, this is the built-in quality.

3 Delivery quality

Delivery quality includes two parts: system quality and product experience. System quality refers to the functional completeness, stability and robustness of the system, and product experience refers to whether customers have a silky experience when using this product.

Two consensus

To make each role in the R&D process willing to practice a unified set of standards and norms, the indispensable condition is the ideological unity of each role. Standards and norms can only be promoted and implemented if the team's thinking is consistent. In the process of promoting the standardized construction of ICBU's cross-border funds-international settlement team, our team first reached a high degree of unity under the following viewpoints from top to bottom.

1 Quality is not just measured

On Zhihu, I saw a discussion on the question of "Is quality designed or tested?" I think it is well written. I analyzed this proposition from different angles. I extracted it and aimed at "quality" The debate on the origin of "The Way" was very enthusiastic. I have excerpted a few representative ones:

Student A: I think quality is designed, and all kinds of non-functional quality data considered in the design will be implemented in the code. The optimization of the design will continuously drive the optimization of the system quality.

Student B: I think quality comes from testing. Designing things can avoid known problems, but in the process of actual testing, you will still find other problems that have not been considered, such as compatibility problems. You can prevent them through design in advance ? Therefore, the test finds problems, and the problems drive quality improvement.

Student C: After listening to student B's speech, I firmly believe that quality is created by design. Driven by constant bugs, will the quality of our patched system be better? Patching is an urgent solution, and subsequent systematic design, reconstruction, and upgrade are the key points to improve quality. so...

Student D: If we stand at the product level, how will we define the product? In our quality model that defines whether a product is good or bad, it is likely to include non-functional quality attributes (ISO9126) related to software development, and may include things unearthed from product public opinion and competing product comparisons. For example, when we define the quality of a content recommendation product, in addition to dimensions such as "non-repetitive content" and "diversity", "whether it supports sharing" and "whether it supports likes" will also become the criteria for judging the quality. , When new functions are launched and the needs are met, users will think that the product is good. Our cognition will continue to upgrade, and the standard of "good" will also have higher requirements. Users are using, testing, and giving feedback all the time, so that the quality continues to improve.

2 both, and, and, and

The views of the above four students represent different quality concepts and pursuits:

The point of view before taking action: Whether it is the ambiguity analysis of requirements, the process analysis of UML diagrams in design, timing analysis, and state analysis, it is hoped that it can sharpen the knife without cutting firewood by mistake and reduce costs. Student A is right.

The point of view of exploratory testing: Whether it is to ensure that the translation is 100% completed in the process of converting design into code, or to be inspired to write more logic codes during the testing process, we hope that what we see is what we get. Think about what people didn't think about. Student B is right.

Viewpoint on technical debt: Whether it is refactoring the patch code some time ago, upgrading the system architecture, or optimizing infrastructure capabilities, we all hope to lay a good foundation and go further and more steadily. Classmate C is reliable.

The point of view of continuous improvement: Whether it is doing competitive product analysis, public opinion analysis, online active detection, monitoring, product quality models, etc., we all hope to find problems and deficiencies in existing cognition and uncognition. D students think broadly.

Since everyone is right and represents an excellent quality concept, then there is no need to make a trade-off, it is necessary, necessary, necessary, and necessary.

So our final conclusion becomes:

"Quality is not only designed, but also tested, or forced", but quality must not only be measured, quality assurance is not only the responsibility of testing a role, but also the common responsibility of all roles throughout the R&D process.

3 The earlier the defect is found, the lower the cost
There is no doubt that the earlier a defect is found, the less expensive it is to fix it. Unreasonable requirements are found in the requirements review stage, problems in technical solutions are found in the technical design stage, system bugs and product bugs are found in the test verification stage, and the cost of repairing the problems found in these links gradually increases, but if If the problem is discovered after the customer has been contacted, the cost of repair will be huge, which may affect customer experience and satisfaction, or cause capital loss, and even cause damage to the company's reputation.

4 Testing strategy is essentially a method to strike a balance between quality cost and quality risk
We know that the execution scenarios of the program and the data input involved in the scenarios cannot be exhausted, so the tests cannot be exhausted. Therefore, we need to design the test plan. By using the equivalence class and boundary value method of the black box test case design, the conditional coverage method and the path coverage method of the white box test case design, we can select effective test data from unlimited test data. Test data for effective testing within limited test time.

5 Developing self-assessment is the basic requirement and basic literacy

We have been emphasizing the development of self-test, no matter whether the requirements are tested or not, the development must consciously complete the self-test. As a qualified developer, you must have sufficient quality assurance for the code you deliver. This should be the idea and consensus that need to be implemented in the team from top to bottom.

three practice

ICBU's cross-border funds-international settlement team is a model room for ICBU's quality standardization practice. Our team has strict quality specifications in all aspects of the R&D process. The international settlement team undertakes many and complicated businesses, especially when it comes to the delivery of funds, we are very cautious. In the international settlement team, from the proposal to the launch of a requirement, the important requirements for the test takeover category need to go through the following steps:

Requirements Review->Resource Input Evaluation->Technical Solution Design Review->Development & Joint Debugging & Self-Test->Test Solution Review->Function Preview->Test->Release Plan Review->Grayscale->Full Volume

For self-test requirements, you need to go through the following links:

Demand review->resource investment evaluation->technical solution design review->development & joint debugging>self-test-> grayscale->full quantity
The following will describe the norms of these links and the indicators to measure whether each link is doing well or not.

1 stage of demand

The problem we often encounter in the requirements stage is that during the project development process, we discover that there are unassessed demand points. If these unevaluated demand points are to be realized, the project may be delayed, and it will undoubtedly increase the burden on the project team. member pressure. To address this, we propose the following coping mechanisms and norms:

1. During the project development process, it is found that there are unassessed demand points. If it does not affect the main process, it can be solved by changing.

Responsible:

Identify changes: module owner or module development
Change of Operation: PM & PD

2. Before a large-scale project enters development, it is necessary to review the detailed PRD again. If there is a UED change, it is necessary to prepare the design draft before organizing the PRD review;

Person in charge: PD (organization), PM (supervision & coordination)

2 Evaluation of resource input
After the demand review is over, the development and testing students need to evaluate the development resources and testing resources invested. The demand PM will coordinate all parties to set a schedule, mainly including joint debugging time, testing time and release time. Once the time of each node Make sure that PM and TPM will advance strictly according to this time point, and generally no delay in delivery is allowed, unless there are special reasons, such as temporary insertion of other urgent needs, etc.

In the evaluation of resource input, in addition to setting a schedule, the test will also evaluate whether the demand is for development self-test or test intervention, and there will be a set of evaluation mechanisms. In the international settlement team, the development-test ratio is 9:1, which means that it is impossible for the test to take over every requirement. For some page requirements that are estimated to have no financial risk and are not suitable for customers, the development and self-test will be required.

In general, the most important specifications in this link are:

1. Pull together resources from all parties and coordinate scheduling
Person in charge: PM

2. Determine the joint debugging time, testing time, release and launch time, and then PM and TPM will strictly follow these three nodes to promote the project
Responsible: PM and TPM

3. The test confirms whether it is necessary to take over the test
Person in charge: QA

The indicators to measure whether this link is good or not are:

Improve on-time rate
Test pass rate
Release punctuality

3 series in stages
Before entering development, the essential link is the design and review of technical solutions. We require team masters and architects to participate in the technical solution review, and there is a set of technical solution design templates. If this link is done well, it can make the following links get twice the result with half the effort.

In general, technical solution design will require development to take into account the following aspects:

* Goals and context
*Function points and developer days
* Interaction sequence diagrams between systems and within systems
*Database Design
*Interface design (including the interface provided to the front end and the interface provided to the upstream and downstream of the business)
* Timing task design
*Interface performance evaluation
*Compatibility evaluation (whether it is compatible with old functions)
*Grayscale scheme
*System monitoring
* Check plan
* Fund security checkList

If the design of the above aspects is not clearly described in the technical plan, it will be rejected during the technical review, and the next technical plan review will be organized again after the modification is completed. Only after passing, will it enter the development. This method can block those implementations that have problems in technical solution design, and avoid the uncontrollable situation where system design problems are not discovered until after testing.

4 testing stages
Undoubtedly, the detailed test plan and test case review can firstly make the test students understand the changes and risks of this requirement, secondly, it can help the development to sort out the function points and scope of influence, and thirdly, it can formally connect the three parties (Testing, development, product) Consensus on the function points of this delivery. In order to achieve these three goals, we sorted out the templates and specifications of the test plan design, and asked the test students in the team to:

1. When taking over a test project for more than 2 days, there must be a review link of test plan and test case

2. The review of the test plan should be completed within five days after the review of the development technical plan

In general, test scenario design will require testing to take into account the following:

*Smoking use cases that need to be provided to development
*The function points of this requirement and the corresponding coverage use cases
*Old functions and regression use cases that may be affected by this requirement change point
*Upstream and downstream joint testing scheme and time point
*Analysis of possible capital loss points and scenarios that need to be built and checked
*Interface performance analysis
*Gray release plan analysis
*Maintain and update the main use case library

5 Development & joint debugging & self-test stage
In relatively large projects, such as the xx project, as many as 13 development students participated in the research and development, and the research and development time lasted as long as one month. And the quality is likely to be uncontrolled. The xx project stepped on this pit, and did not synchronize the daily R&D progress in the project team at key nodes, which caused problems (someone was inserted by other requirements in the middle of the project, which caused a module to fail to be debugged on time, and the quality of the test was poor; the development was a newcomer , failed to launch the test on time, etc.) concentrated on the outbreak of the testing node. Based on this lesson, we conducted a review and set the follow-up norms in this link, namely:

1. After entering the joint debugging link of the project, it is necessary to issue a joint debugging daily report (summarized by PM), and to synchronize the joint debugging progress in the morning meeting every day.

Person in charge: PM

2. Principle: Before joint debugging, you need to complete the self-test in the domain;

Responsible Person: Development

3. When encountering a stuck point problem, the interface relying party will actively push the dependent party to solve it. If the dependent party fails to solve it in time and affects the progress, the risk needs to be synchronized

Responsible Person: Upstream Development

4. In the joint debugging stage, the upstream Mock joint debugging or real link joint debugging sends a message, confirms whether the message is correct with the other party development, and increases the confirmation process; the real link joint debugging and mock joint debugging interfaces are not allowed to have differences.

Responsible Person: Downstream Development

5. The test students provide the smoke use case to the development self-test. The development must go through all the scenes of the smoke use case before it can be tested

Responsible Person: Development Execution, Test Supervision

6. Before the release, the single-test incremental coverage rate needs to reach 60%, and the single-test pass rate is 100%

Responsible Person: Development Execution, Test Supervision

7. CR within the group needs to be completed two days in advance

Responsible Person: Development

In the self-test link, the test will provide a smoking use case in advance for the development of self-test.

We will require developers to go through all the scenarios of the smoke use case before they can be tested. Currently, smoke use cases are provided to development in the form of Yuque documents. This is an offline method, which is not conducive to retrospective and standardized promotion. Therefore, the ICBU quality team provides online use case management on the Fields platform combined with aone In the future, it will provide online smoke use cases and smoke status management based on the Fields platform.

The indicators to measure whether this link is good or not are:

Improve on-time rate
Test pass rate
Self-test detectable bug rate
6 Function preview stage
Before the function preview is proposed, there will often be low-quality problems such as the functions proposed by the developers cannot even open the page or the interface cannot be adjusted. Low-quality testing will make the test very passive. In order to avoid this kind of problem, we proposed a function preview in the team, that is, before the test launch, the PM needs to pull the test and develop a round of function preview for the test version to ensure that the main process can go through. Quickly solve very low-level problems before testing, and also allow test students to have more time to test the abnormal flow of the system to ensure the delivery quality of the project.

If the function preview fails, the test proposal will be called back. After the problem is solved, the development needs to organize the function preview again and postpone the release time. We will record the reasons for the callback and the delayed release on the Fields platform The reason, and regularly review the data, review the phenomenon of multiple delays, and discuss the improvement plan.

In the past, many teams may have mentioned that the test was not passed, so they would call back, and then submit the test after solving the problem, but they never mentioned that the release time needs to be postponed, which caused the pressure of the test to be very high, and the pot of previous development was thrown to In order to test overtime to cover the bottom line. What's more serious is that many teams recognize the logic of testing the bottom line. In order to correct this kind of thinking, we have made a lot of efforts. For the postponed projects, we specially organized project review meetings and set the norms:

1. Inter-domain joint debugging needs to be completed before the rehearsal

2. The function rehearsal is dominated by the test, and the development needs to be present. If the main link is stuck, the development needs to give the time of the next rehearsal, and postpone the release time, and let the project team know;
Responsible person: rehearsal-test; investigation-development;

3. For the problems in the functional rehearsal, if it is a bug problem, it needs to be resolved by the developer himself; if it is a data problem, it should be promoted by the test

4. Environmental issues need to be dealt with
Follow up: Architect

7 testing phase
At present, for projects with more than 3 days of testing time, the test students will send a daily report to synchronize the test progress and risks in a timely manner through the daily report, and at the same time require bug development to be closed on a daily basis. Daily template:

1. Project Milestones
2. Progress and problems
3. Risk
4. Tomorrow's plan

For example, a daily report on the invoice platform:

1. Project Milestones

11.9-12.28: design & development & joint debugging, completed
12.21: Invoice configuration background function preview, completed
12.30: The rehearsal of the invoicing process optimization function has been completed (delayed by 6.5 working days)
12.22-1.6: Offline testing, completed
1.8-1.12: Pre-release regression test, in progress (pre-release postponed for 2 working days, originally planned for 1.5 days)
1.13: Release (Delayed by 5 working days, reason: Developers are occupied by other projects and enter this project late)
2. Progress and problems

Offline testing progress: 100%
Pre-release test progress:

Background test progress of invoice configuration: 100%
Invoicing process optimization test progress: 90%
3. Risk

Project extension: The project will be postponed to 2022.1.13 for 5 working days. Reasons: ①Developers were occupied by other projects and entered late; ②Developers estimated the workload in the early stage slightly less than the actual; ③The xxx project was urgently inserted, and the front-end students needed to spend a day. ④ The completion time of defect repair was not as expected, and the pre-release and testing time was postponed by 2 days.
bugfix
4. Tomorrow's plan

Regression verification The remaining defects of the new billing function;
Return to the embedded invoicing process;
Return to the original invoicing process.
Through daily contact with all the students in the project team (including development, product, business, and testing), you can have a clear understanding of the current testing progress and problems. At the same time, if the project has risks and needs to be delayed, you can also inform the product in time And business, convenient Lacy everyone expected. At present, everyone still agrees with the daily, and will adhere to this standard in the future.

8 Release Plan Review
I believe that many students will have encountered such a situation. Many scenarios have no problems during the pre-release acceptance, but once they are sent online, these scenarios will not work. After some investigation, it is found that the reason is: the interface is not registered more, The metaq topic is not configured, the switch or diamond switch is not configured properly, and the dts scheduled task is not enabled and other configuration problems. In order to prevent this type of problem from happening again, we require that:

Before the release of a large project, a review of the release plan is required, and the preparation work is synchronized within the project team before the release, and then released.

Responsible Person: Project PM

The template is as follows:

1. Release scope
a.Application range
b. Release order

2. Resource sorting
a.HSF interface
b. Database resources
c. Message resource (metaq)
d. Scheduling resources (dts)
e. Configure resources (diamond, switch, antx)

3. Grayscale plan

4. Release process
a. Pre-release release process
b. Online publishing process

5. Rollback plan

6. Tracking plan
a. Check
b. System monitoring
c. Service monitoring

7. Risk control and precautions
a. Historical data compatibility
b. Idempotent logical confirmation
c. Grayscale scheme
d. Emergency switch

The indicators to measure whether this link is good or not are:

Online bug rate

9 Publish & After Publish
Publishing follows the group's safety redlines

beta for at least one hour, rollback possible
In terms of business, it can be grayed out. After the grayscale is no problem, it will be released to all users
For the system tracking after the function is released, it is mainly covered by system monitoring, service monitoring, and checking. When an exception occurs, development and testing will pay attention to the cause and solve the problem as soon as possible.

For the user experience tracking after the release of the function, it is mainly through the feedback of Grayscale customers and the analysis of the work order after the full release.

You can see which business modules have more work orders recently, so as to carry out targeted optimization. At present, this part is mainly developed and followed up.

10 Project Review
After a large-scale project is released, if there are some problems in the project process, we generally ask to organize a project review meeting to review the problems in the project, and discuss the follow-up mechanism and specifications, so as to avoid follow-up projects Step on the same pit again.

In this way, we can continuously improve the specifications of our team's R&D process and solve current problems in a timely manner.

11 Combination with Fields Publishing Platform
The Fields platform is positioned as a platform for release control and quality management, as well as more input and help for self-test development.

Self-test requirements and Fields platform practice

At present, the Fields platform is connected with the aone card point. For self-test requirements, developers are required to complete the development self-test before release, and provide self-test feedback on the platform before releasing it online.

By making the self-test process online, we hope to provide more help for the development of self-test.

Construct the main case library, so that each scene has detailed operation steps, and one-click trigger interface automation
For self-test requirements, the test circles the backbone use cases that need to be returned on the Fields platform and gives them to development and execution
We found that many developers only understand part of the business, or only know part of the back-end business logic. Sometimes it is difficult to assess the impact of their own changes, or sometimes they want to return to a certain business but do not know how to operate it. In order to solve these problems , we are currently building a backbone use case library for international settlement business, so that even if developers are not familiar with the relevant business, they can return to our use cases "fool-like operation", thereby solving the pain point of self-test development.

Test Takeover Requirements and Fields Platform Practice

Generally, the demand for the test takeover class is relatively large, the development cycle is relatively long, and there will be more participants. At present, the capabilities provided by the Fields platform for such needs are:

The smoking process is online, the test can add smoking use cases on the platform, and the development will mark the smoking status, so we can count the smoking pass rate
The testing process is online, and the test can be returned on the platform, so we can count the passing rate of the testing, so as to measure the quality of the current development and testing, and carry out targeted review
But for the follow-up process, such as the test report in the test link, risk synchronization, release plan, etc., the capabilities that the platform can provide are still under discussion, and we look forward to the follow-up.

Four summary

Earlier, I explained to you some consensuses of the international settlement team on quality standardization, as well as the norms and standards summarized after reviewing many previous projects. After these norms and standards are finalized, you still have to rely on testing and development students to operate, implement, improve and continue to optimize in subsequent projects. We are also thinking about whether the entire R&D process can be operated online. At present, there is no good solution, and we still rely more on people to analyze and drive.