wiki:RtgDirDocQa

Working Group Draft Quality Assurance

The WG Draft Quality Assurance process exists to provide cross-WG and expert review early in the IETF process after a WG has adopted a WG draft or while the WG is deciding to adopt a draft. Since a WG adopts a draft as a good starting point for the work, providing early excellent review of such drafts allows for good technical discussion and the ability to enhance the WG draft to solve identified issues. The earlier in the process that substantial issues (technical or editorial) are resolved, the more quickly and smoothly a WG draft is likely to proceed.

This page describes a process for WG Draft QA. The reviews are done by a member of the Routing Directorate, chosen by round-robin assignment across the set of all Routing Directorate members. Separate from the WG Draft QA process, the Routing ADs may request additional Routing Directorate reviews be done on any draft.

The WG Draft QA process for an internet-draft is started by a WG Chair of the relevant WG. The WG Chairs are encouraged to use their judgement as to whether a QA review is likely to be useful. Having a QA review is not a required process step.

The Problem

Drafts get little review outside the working group and sometimes inside the working group until after working group last call. At that point it is often painful and costly to make meaningful improvements to the drafts.

The Goals

  1. Excellent and consistent review at WG adoption and before WG last call.
  2. Ability for prolonged discussion and problem-solving when the draft can still be easily enhanced.
  3. Reduced time to delivery of RFCs.
  4. Quality documents

The Solution

As with many processes in the IETF, this process is intended to be used for the spirit of the goal, subject to common-sense and technical perspective. In particular, none of this process is intended to gate or otherwise impede the normal progression of documents through IETF working groups; rather, it is intended to provide additional input, review, and comments that the working groups can use to help improve the quality of their documents, and to help chairs determine quality and consensus for documents. We do not expect IETF process to wait for reviews to happen.

Upon request from the WG Chair, the Routing Directorate Administrators will assign a QA Reviewer to review a draft.

Ideally, the QA Reviewer should:

  1. Be a known neutral party, who is not expected to be an expert in the area covered by the draft.
  2. Review the draft when the request is received - when a WG call for adoption occurs or when a draft has been newly adopted by the WG.
  3. Work with the authors to resolve issues that arise from the review.
  4. Review the draft again when WG Last Call happens - and help verify that issues expressed earlier have been resolved.
  5. Be willing to provide opinions about the draft to the WG Chairs and ADs.

Initial Review Guidance

When reviewing a draft at WG Adoption, the QA Reviewer should determine whether the draft is readable, understandable, makes sense and is a good start for a WG draft. Any issues the QA Reviewer finds are written down, sent to the mailing list and discussed for future versions, but should not gate the progress of the draft in the WG – this is at the WG chairs’ discretion.

Note that the reviewer may not be an expert in the area covered by the draft, and that is OK. The aim is to get fresh perspectives on drafts from experienced reviewers who are nonetheless not expected to be experts in the thing they are reviewing. The minimum goal of the review is to check for understandability of the document, as this is the most common sticking point when a document gets to the IESG.

The Expected Workload

The Routing Directorate has approximately 40 people. If we assume that approximately 50 drafts pass WG adoption and a similar number pass WGLC each year, then the load is 2.5 documents per person per year. If we assume that there are about 200 (214 when counted on March 31, 2014) WG drafts in the Routing Area and that they all have QA reviewers, then each Routing Directorate member will have 5 drafts that she or he is the QA Reviewer for.

Although the process, described below, is for every draft that is asking for WG adoption to be discussed as to whether it needs a QA reviewer, it is not expected that all WG drafts will need a QA review.

The Basic Process

When the WG chairs start the process for adoption, if the WG Chairs think that the draft would benefit from a QA review, then the WG chairs SHOULD initiate the request to have a QA reviewer assigned by emailing the Routing Directorate coordinators. The Routing Directorate Coordinators assign a QA reviewer using a round-robin algorithm. Then the Routing Directorate Coordinators confirm with the selected Routing Directorate member about their availability and willingness to be the QA Reviewer for this particular draft. Once one is selected, the Routing Directorate Coordinators update the wiki and notify the WG chairs and draft authors. If none is selected, this should be noted too.

The WG Chairs may choose to track the QA Reviewer assigned to a draft in the Datatracker as a comment when the state is changed to either Candidate for WG Adoption or Call for Adoption by WG Issued.

  1. With the agreement of the ADs, a QA Reviewer who is not a member of the Routing Directorate may be selected.
  2. The WG chairs can, after consulting with the draft authors, decline to have a QA Reviewer for the draft.
  3. The QA Reviewer should communicate all concerns to the Document Shepherd for hand-off after WGLC.

The Detailed Process for Adding a Draft for QA

  1. The WG Chair or Secretary finds the draft's page in the datatracker and clicks on the "request review" button. The review type should be set to "Early". Give a deadline for the review (2 to 3 weeks is normal) and add any additional context for the review in the notes field (such as why the review is being requested, what stage the draft is at, relevant history of the draft, anything you want the reviewer to look for particularly, whether you need a reviewer with particular background knowledge, and so on).
  1. The Routing Directorate Coordinators find a QA Reviewer. The review can be done during the WG Call for Adoption. The WG Chair or Secretary may add a comment in the draft's datatracker history page indicating whom is the QA Reviewer.
  1. The QA review does not block or gate a call for adoption. The QA Reviewer should send their comments to the WG and authors. If the document is or becomes a WG draft, the QA Reviewer will work with the document editors to resolve comments. Comments may be tracked via the WG's trac tools page.
  1. The WG Chair should notify the QA reviewer and Routing Directorate Coordinators when the draft goes into WGLC. If the QA Reviewer cannot do the review, then the WG Chair and Coordinators should assign another QA Reviewer.
  1. The QA review does not block or gate the WGLC, but is treated as an individual's review. The QA Reviewer should send their review to the WG and authors during the WGLC. The QA Reviewer will work with the document editors to resolve comments.

Wiki Pages to Support WG Draft QA

  1. The members of the Routing Directorate are listed here.
  2. The areas of knowledge for the Routing Directorate are given here.
  3. The QA Reviews are documented with other reviews done by the Routing Directorate here.
  4. Template for QA review here.

Back to Routing Area wiki

Back to Routing Area Directorate wiki

Last modified 2 years ago Last modified on Jul 5, 2017, 1:10:24 AM