If you’ve been following the saga of the National Institutes of Health Information Technology Acquisition and Assessment Center’s (NITAAC) Chief Information Officer-Solutions and Partners (CIO-SP4) procurement, you likely know the Government Accountability Office (GAO) recently issued two decisions sustaining post-award protests by 91 unsuccessful CIO-SP4 offerors. These decisions are here and here. In this article, we’ll walk through what the GAO found and discuss what it means for both successful and unsuccessful CIO-SP4 offerors.
Background
CIO-SP4 will replace NITAAC’s existing CIO-SP3 contracts as a suite of government-wide acquisition vehicles under which federal agencies can issue task orders for various information technology solutions and services. The CIO-SP4 solicitation contemplated award of 305 to 510 separate indefinite delivery, indefinite quantity (IDIQ) contracts, spread across various socioeconomic categories. Each contract will have a maximum value of $50 billion and a ten-year maximum period of performance (inclusive of options). The agency received more than 1,000 proposals in response to the solicitation.
Like several other large IDIQs of this nature, CIO-SP4’s primary evaluation methodology involves tallying “points” a proposal earns by citing various categories of experience, certifications, and other qualifications that the solicitation defines as worth specific numbers of points. For the first phase of the competition, each offeror was required to populate a self-scoring sheet with the particular point-generating items its team had assembled and submit documentation to allow the agency to validate the offeror’s claimed points. The solicitation stated the agency would “validate the offerors’ completed self-scoring” as part of a phase 1 evaluation, and “[o]nly the highest rated offerors will advance to phase 2 of the evaluation.” Whether a proposal advanced to phase 2 depended on whether the proposal’s validated points were above a cut-off line (called a “cutline”). The cutline would eliminate all but the most highly rated proposals, while attempting to approximate the solicitation’s estimated number of awards for each socioeconomic category. The solicitation did not state how exactly the agency would do that.
Evaluations and First Rounds of Protests
After much frustration from offerors and a rocky start in early 2022, including a sustained pre-award protest of the solicitation terms and any number of solicitation amendments, the CIO-SP4 competition proceeded. In September 2022, the agency notified offerors with scores below the undisclosed cutline for each socio-economic category that their proposals would not advance to phase 2. Approximately 120 excluded offerors filed bid protests. In November 2022, the Agency announced voluntary corrective action, in the form of reassessing the cutlines and making a new decision of which proposals would advance to phase 2, thus mooting the protests.
The agency implemented its corrective action and, in February 2023, again provided notifications to offerors that did not make the cut. Another slew of protests followed. A month later, the agency again announced voluntary corrective action, in the form of reassessing the source-selection methodology and making new advancement determinations. The agency also stated it would “address inconsistencies that arose during implementation of the prior corrective action.” In mid-March, the GAO again dismissed the protests as moot.
Third Corrective Action and Protests
A few days after the GAO dismissed the second wave of exclusion protests, the agency completed its corrective action and again provided exclusion notices. In pre-award debriefings, the agency disclosed to each excluded offeror its self-score, its “validated” score, and the phase 1 cut-off (called “cutline”) scores for each socioeconomic category under which the offeror had submitted a proposal. By the end of March, another wave of protests hit the GAO, with 91 offerors eventually filing challenges to their exclusion. Around the same time, the agency posted a “preliminary” public notice of the “apparent successful offerors.”
Unlike the first two rounds of protests, the agency fought this third round all the way to a written decision of the GAO. The GAO sustained the protests in two merits decisions—one decision consolidating the protests of represented protesters, whose attorneys were admitted to a protective order and reviewed the internal agency record, and a separate decision consolidating pro se protests by unrepresented offerors, which did not have access to the full record.
The 91 protesters raised numerous challenges, timely and untimely, meritorious and not. Sifting the wheat from the chaff, the GAO sustained the protests for overarching errors in the way the agency validated (or did not validate) self-scored points and (as a result of that) the agency’s method for drawing the cutlines for the various socioeconomic categories. In addition to these broad, methodological problems (which are the focus of this article), the GAO also sustained one protester’s challenge to the particular “validated” score it received.
Validating Self-Scores and Establishing Cutlines
The GAO decisions go into excruciating detail about documentation problems, confessed errors, internally inconsistent assertions by agency personnel in response to the protests, and the GAO’s own inability to figure out exactly what the agency had done, even though the agency had three shots at correcting and perfecting its record. The heart of the problem, though, was the agency’s apparent approach to validating the self-scores submitted by offerors. Although the solicitation did not define how agency validation would occur, it stated it would happen, and the offerors’ validated scores would determine which proposals to advance to phase 2.
Although the agency claimed it validated all self-scores before establishing cutlines, the GAO found the agency’s multiple and inconsistent responses to supplemental protest arguments “show that the agency’s initial responses to the protesters and the descriptions of the validation process in the final cutline methodology memorandum were incomplete and ultimately misleading.” As best the GAO could determine, the agency conducted an “initial validation” that consisted of nothing but a check to see whether an offeror had submitted documentation to support its score, without any further analysis, review or score adjustment. Based on the self-scores (which in many instances appear to have been inflated), the agency established initial cutlines for each socioeconomic category and eliminated proposals with self-scores below the cutline. The agency appears to have conducted a second “hard validation” to determine whether the submitted documentation actually supported the points claimed, with corresponding adjustments to self-scores and further eliminations, if needed.
The GAO provided one example where an offeror gave itself 10,000 points, which apparently figured into the initial setting of that socioeconomic group’s cutline and culling of the proposals. Only after the cutline was established, however, did the agency apparently perform a “hard validation” on this proposal, which showed the offeror had substantiated only 500 of the 10,000 points it claimed. The GAO also noted that the agency adjusted a third of the self-scores of proposals advanced to phase 2, while adjusting none of the self-scores that fell below the initial cutlines, which further undercut the claim that the agency validated all self-scores. The GAO expressed doubt as to whether the cutlines “were based on hard validated scores at all.”
Based on these indications, the contradictory agency explanations, and a dearth of contemporaneous documentation, the GAO was unable to conclude the agency performed a full “hard validation” on all self-scores, as the solicitation contemplated, and found the agency relied on only partially-validated scores to establish cutlines.
Because the agency apparently used unadjusted, unvalidated self-scores to establish cutlines (and to reject proposals that fell below that line), many of those self-scores were presumably overstated. As a result, it is possible the agency set the cutlines higher than it otherwise would have, and may have excluded some offerors that would have had scores higher than a reasonably-established cutline.
The agency argued that any errors or lack of clarity had no effect on the outcome of the phase 1 decisions. The GAO conceded that a reasonable evaluation “might have yielded the same or substantially similar cutlines even if the agency performed a hard validation on all proposals,” but found the record was insufficiently documented to establish that. The GAO also observed that the agency’s responses to all but two of the protests were incomplete and inaccurate, thus depriving these 89 protesters of the opportunity to address an accurate version of the evaluation process. For these reasons, the GAO found a sufficient likelihood of prejudice to sustain the protest.
Takeaways
- Assuming the agency really did “hard validate” the self-scores of all proposals that advanced to phase 2, it appears unlikely that corrective action will displace any of the current “apparent successful offerors.” The stay of award/performance, however, presumably will remain in place and prevent any task order competitions from proceeding under CIO-SP4 until the agency completes this latest round of corrective action.
- Winning a protest is not the same as winning an award. It appears unlikely that most of the 91 protesters will receive awards as a result of full validation and new cutlines, and some may see their scores adjusted downward if they were not previously subjected to a “hard validation.” Given the agency’s track record in implementing corrective action, it is likely we will see yet another wave of protests once this latest round of corrective action is complete.
- The agency’s recent extension of the legacy CIO-SP3 contracts through next April suggests it is accounting for another round of protests. And, even if the agency finally prevails in the next GAO protests, some disappointed offerors will likely contemplate second-bite protests at the Court of Federal Claims.