Evaluating tenders the wiki way

M’colleague Iza is investigating use of wikis for collaborative team work.

At the University of Melbourne we used a wiki in late 2004 to enable about 30 staff from across the University to collaboratively evaluate tenders submitted for our new CMS.

Tenders were received as PDFs, which the evaluation team could download from the secure wiki site. The wiki’s structure and navigation were based on the list of requirements we had released as the Request For Tender (RFT) document.

Each team member reviewed every tender against 2-3 individual RFT requirements. Each RFT requirement was evaluated by at least two people.

We used a numerical scoring system, along these lines:

0 = doesn’t meet requirements

1 = partially meets requirements

2 = meets requirements

3 = exceeds requirements

Team members were encouraged to make comments in the wiki alongside their score for each RFT requirement. The comments were intended generate discussion among team members, and to give us a list of specific questions to ask later in the evaluation process.

Team members were also encouraged to communicate via email, phone or meetings during the 10-day evaluation period, especially with the people who were evaluating the same RFT requirements.

At the end of the evaluation period, we added up the scores for each requirement and collated the comments. This resulted in a shortlist of tenders to be further considered.

The scores for the shortlisted tenders were reviewed in a meeting that all the evaluation team members were required to attend. Where a particular RFT requirement received widely varying scores, eg a 3 and a 1, the evaluators were asked to speak for and against.

A further shortlist was made by consensus at this meeting, and 3-4 vendors were invited to demonstrate their systems. (We started with about 16 tenders.)

If we were doing it again, we’d probably use different wiki software. However, the collaborative process itself was definitely worthwhile:

• stakeholders in the project were actively involved in making informed decisions about which system we would choose
• experts in different technical fields (system admin, web development, accessibility, information management, programming, helpdesk and all the rest) were able to focus on their particular field — but were also encouraged to contribute to evaluating other RFT requirements
• a numerical scoring system takes some of the subjectivity out of decision-making
• we were able to involve more people in this process than is possible in other tender-evaluation processes (fewer meetings, more flexibility in where and when you did the work)
• it was an enjoyable learning process for everyone — some were new to the technology, others were new to the evaluation process
• the careful, extensive documentation and broad consultation satisfied the University’s quite rigorous expectations for due diligence and probity of selection process.

So far, the chosen CMS is working well. We completed a ‘proof of concept’ installation and tested it rigorously before starting the ‘pilot implementation’ with some real web sites. The first sites in the new CMS are due to go live in early August 2006.

Want more? Try these:

• Nope, sorry, nothing relevant springs to mind.