FAR from accurate

FAR from accurate

This post is not specifically about FAR (Federal Acquisition Regulations).  It’s about using relative grading or estimates (evaluations), before absolute grading or estimates (evaluations).  As an advisor to a source selection committee, I’m not allowed to write or talk about what is happening behind closed doors.  I can, however, write about the FAR .  Don’t click away just yet!
What exactly does the FAR say?

Subpart 15.303  Source Selection Responsibilities

Paragraph (b4) The source selection authority shall ensure that proposals are evaluated based solely on the factors and subfactors contained in the solicitation (10 U.S.C. 2305(b)(1) and 41 U.S.C. 253b(d)(3)).

That means is if you have 10 proposals, you are not allowed to compare them.  Based on content contained within them, you can not say one proposal is better or worse than another.  You may only grade the proposal against the solicitation, as if it was the sole submission.  So, this is where I see the problem with the FAR.  After reading Predictably Irrational, by Dan Ariely, I know humans don’t think like this.  We compare things and then react (usually irrationally).  We’re also pretty bad at estimating if we don’t have something to use an a baseline or anchor.

The only proposal submission that will truly comply with the FAR is the first one read.  The rest of the proposals will be compared to all those read before it.  This isn’t intentional.  It will happen subconsciously.  Though the FAR is attempting to be fair, I don’t think that it’s realistic.  If this was a scientific experiment, you would re-baseline after each test.  But the human memory is a tricky thing. You can’t just reboot and clear the cache.

I’ve seen the same thing happen with providing task estimates.  You can spend a lot of time and money trying to get an absolute estimate.  For some projects, they require that up front. But, where there is a lot of uncertainty, relative estimating is commonly going to get you closer to where you want to be.  You many not be able to accurately predict how long it will take to complete a task but you can say if the task being estimated will be easier or harder than a similar task recently completed.  Use that as the anchor.

What I am proposing with the FAR and with estimating is provide a relative evaluation (bigger/smaller, better/worse, sm/med/lg…) based on all available information.  As more information becomes available, refine your evaluation or estimate to absolute terms (SLA1=4, Task1=Nhr(s), Task2=NDay(s)…).

If I were to change the FAR, I would require all of the proposals be reviewed once (establishing an overall baseline) and then focus my attention on the best proposals.  But, since I’ll only be advising the source selection committee for two more days, I’ll leave it up to the FAR.

Which do you prefer, relative or absolute estimates?

Yes, the book link is an affiliate link.

2 Replies to “FAR from accurate”

  1. I’ve never been on a source selection committee (but I do write proposals), I don’t see how that piece of FAR means you can’t compare proposals. To me it reads like you can only SCORE them against the solicitation criteria, but then you can readily compare the scored proposals against each other.

    In my head, you’d be assigned to review the Technical Approach, perhaps even just a section. You’d rate the section on the appropriate Section M criteria, so you have an overall section score and all the factor scores, and you’d have a matrix for proposals 1-10 and you’d see that proposal 3 had the highest score.

    1. Chris, you nailed it. You are correct. But, we can’t do any comparisons until after all proposals are scored individually against the solicitation. Only then are we allowed to compare the scores. I’m of the belief that people will begin the comparisons immediately. They can’t help it. Ideally, the proposals have quantifiable data. But reality is rarely ideal.

Leave a Reply

Your email address will not be published. Required fields are marked *