How to Choose a Software Development Partner: A Checklist and Scorecard for CIOs, CISOs and Procurement

Choosing a software development partner means comparing vendors on the same terms and picking the one you can trust with a system you depend on. The safest approach is to rule out vendors that fail your must-haves, score the rest on criteria set in advance, and test the leaders on a small piece of your work.

Getting this choice right is important because the partner will work inside your code and data for years. In a large company, several people answer for that risk.

The chief information officer (CIO) focuses on delivery, the chief information security officer (CISO) on data protection, and procurement and legal on price and contract terms. Any of them can stop the deal, but most guides picture one person making the call.

That is why we wrote this guide. It draws on our engineering projects, including a migration of millions of user accounts that Zoolatech engineers delivered with MasterControl’s team.

We show how to divide the evaluation among your team and which basic requirements a vendor must meet to stay on your list. Then we explain how to compare the remaining vendors with a scorecard and how to test the strongest ones on a small paid task.

By the end, you will have a process your team can run on every vendor and a choice you can rely on.

How to Choose a Software Development Partner in Six Steps

We suggest choosing a software development partner in six steps:

1. Define the work and name the evaluation team.
2. Agree on the pass/fail gates.
3. Fix the criteria, weights, and scoring scale.
4. Send every vendor the same questions.
5. Verify the finalists with work.
6. Make the decision and agree on exit terms before you sign.

The order matters because you agree on the rules first and only then read the proposals. This way, you judge every vendor by the same standard and check each claim before you trust it.

US government agencies follow the same logic. Federal rules require them to tell vendors in advance what they will evaluate and how much each part counts. For a private company, this is optional, and it makes your decision much easier to explain later.

The table below adds detail to each step. It shows who leads the step, what you have at the end, and how you know it is done, so you can copy it straight into your project plan.

Steps 2 and 3 use two terms that come up throughout this guide. A pass/fail gate is a requirement a vendor must meet to stay in the evaluation, and a weighted scorecard compares the vendors that pass on criteria with different weights. We explain both in detail below.

Before you start step 1, make sure these points are settled:

• You need outside help. It makes sense to outsource when you need skills or capacity you cannot hire in time, or when you want one company to own a whole workstream. If your team can deliver on schedule, you may not need a partner.
• You can describe the job. You should be able to say what the partner will own and how you will measure success. If you cannot, run a short discovery phase or an internal workshop before you build a shortlist.
• You know how you want to work together. A partner can add engineers to teams under your own leads, own a whole workstream, or deliver a fixed project. Our guide to software development outsourcing models explains how these options differ.
• You know where this choice fits. Sometimes the new partner will take over work from several current vendors. In that case, read our guide to consolidating engineering vendors under one partner and decide which systems the partner will own before you compare candidates.

Once these points are settled, start with step 1 and decide who on your team will evaluate the vendors and what each will check.

Who Scores What: The CIO, CISO, Procurement and Legal

This section is about your evaluation team, the people on your side who will judge the vendors. We start with it because step 1 of the process is to name this team and agree on each member’s responsibilities.

Each person checks something different, and any of them can stop the deal. If roles are agreed only after proposals arrive, each person compares vendors in their own way, and the team loses time arguing about how to compare them.

The basic rule is that each person scores only the criteria they can judge from evidence. If everyone scores everything, the average blurs the opinions that matter most. For example, a security lead’s view on code quality would count as much as the architect’s, even though the architect knows far more about it.

Bring the CISO and legal into the evaluation before you build the shortlist. If they first see a vendor after the team has picked a favorite, a failed security check or an unacceptable contract can stop the deal at the last moment. The US National Institute of Standards and Technology (NIST) makes the same point in its voluntary Cybersecurity Framework 2.0, which calls for planning and due diligence before a company enters a supplier relationship.

The table below shows how to split the vendor selection criteria by role. Each row describes one person’s job in the evaluation, from the decision they make to the evidence they ask vendors for. We explain the security documents from the CISO row later in this guide.

In a smaller company, one person may hold two of these roles. Keep the split anyway, so each score has one owner and you can trace every score to the person who gave it.

With owners named, the CISO, legal, and procurement can run their pass/fail checks before anyone starts scoring.

The Pass/Fail Checklist: What Rules a Vendor Out Before Scoring

The pass/fail checklist is step 2 of the process. It holds the few requirements a vendor must meet to stay in the evaluation, and each one is answered yes or no. A vendor that fails any of them leaves the evaluation, whatever its other strengths.

These checks stay outside the weighted scorecard. In a weighted total, a high score in one area can cover a failure in another, so a vendor with a strong technical plan and no security evidence could still win. A peer-reviewed study of Dutch public tenders also found that without a minimum quality bar, the winner was more likely to change when extra bids were added to the comparison.

Keep the list short. A US Digital Service (USDS) playbook on technical evaluations advises scoring on a scale wherever possible and using strict yes-or-no criteria sparingly. Use gates only for true must-haves and score everything else.

Here are the gates we recommend checking for every vendor:

1. Security evidence
   The vendor shares a current SOC 2 Type 2 report or ISO/IEC 27001 certificate that covers the people and locations that would work on your systems. A completed security questionnaire can also work if your CISO accepts it. The gate is passed once your CISO has read the document and signed off.
2. Data and access
   The vendor agrees to work through accounts you control, in your repositories, with only the access each engineer needs. Copies of production data stay inside your systems. You need this confirmed in writing.
3. Contract terms
   The vendor accepts your terms on ownership of the work, confidentiality, disclosure of subcontractors, and help with the handover if you part ways. Legal reviews the vendor’s edits to your draft contract and approves them.
4. Budget and engagement model
   The offer fits your budget range and the way you want to work with a partner. Procurement confirms both before scoring starts.
5. Named team
   The people proposed for the first months are named and available for interviews. This check is complete once you have their CVs and the interviews are booked.
6. Conflict of interest
   The vendor does not build a competing product for your direct competitor in the same scope. Decide whether this gate applies to your program, and if it does, get the vendor’s written confirmation.

Run the gates on every vendor before scoring, and note which gate each vendor failed. That record helps you explain the decision later.

Vendors that pass every gate move on to the weighted scorecard, where you compare them on everything else.

How to Evaluate a Software Development Company: The Weighted Scorecard

This section covers step 3, the weighted scorecard. The gates have already removed vendors that cannot work with you, and the remaining ones now need a fair comparison on everything else.

To evaluate a software development company fairly, score every vendor on the same seven criteria. Each criterion gets a weight and a scale from 1 to 5, and you describe what each score means before proposals arrive. Then you multiply each score by its weight and compare the totals.

The table below lists the software vendor evaluation criteria we recommend. For each criterion, it shows who scores it, what a 5 and a 1 look like, and an example weight out of 100, so you can adapt it for your own evaluation.

These weights are only an example. When you build your own software vendor evaluation scorecard, set the weights before proposals arrive and keep price from outweighing everything else.

Set weights and scoring levels before proposals arrive

Before any proposal arrives, describe at least what a 5 and a 1 look like for each criterion. Add a description for a 3 where evaluators are likely to disagree. A request for proposal (RFP) guide from the Harvard Kennedy School Government Performance Lab gives the same advice, so the team knows what a strong proposal looks like before reading one.

The archived guide from the US government technology team 18F treated technical approach, staffing, and similar experience together as significantly more important than price.

We recommend that each evaluator score alone first. The team then discusses any criterion where scores differ by more than one point.

To turn scores into points, use one formula: points = weight × score ÷ 5. With this formula, a vendor’s total falls between 20 and 100.

A worked example

The example below uses three hypothetical vendors and the weights from the table above. Each cell shows the score from 1 to 5 and the points it earns.

Without the gates, Vendor C would win on points, even though it could not show security evidence that covers its delivery team. Vendors A and B are only 2 points apart, which in our view is too close to call on scores alone, so a paid pilot decides between them.

When a scorecard misleads

A scorecard can point to the wrong vendor in four cases:

1. Scoring price against the other bids
   The Dutch study we mentioned earlier covered 303 public tenders. When tenders scored price against other bids, adding fictional losing bids changed the winner in about 1 out of 5 tenders with no quality threshold. The same math applies to a private RFP that scores price against the cheapest bid, so we recommend scoring price against a scale you set in advance, such as your budget range.
2. Setting levels or weights while reading proposals
   If the team decides what a good answer looks like after reading the proposals, the scale drifts toward the vendor it already prefers. Fix levels and weights before the first proposal arrives.
3. Letting one criterion outweigh the rest
   When one criterion carries too much weight, it decides the outcome on its own. The Harvard guide warns against this and names price as the usual example, which is why no criterion in our example carries more than 20 points.
4. Hiding a failed must-have in the total
   A high total can hide a vendor that should have left the evaluation. That is why the gates from the previous section sit outside the total.

Putting the scorecard into your RFP

Tell vendors in the RFP what you will judge and how much each part counts. US federal rules require this, as we showed in the six steps, and for a private company it is the simplest way to get answers you can compare side by side.

Include these items in your software development RFP:

• The pass/fail gates.
• The criteria with their weights.
• A short description of a strong answer for each criterion.
• The evidence vendors must attach.
• The questions from the next section.
• The plan for the paid pilot.
• The timeline of the evaluation.

Those questions come next, and we group them by the person on your team who asks them.

12 Questions to Ask a Software Development Company by Role

This section covers step 4, the questions you send to every vendor. They come after the scorecard because each question feeds one criterion, so the answers turn into scores without extra work.

The most useful questions to ask a software development company are the same for every vendor. Behind each answer, ask for something you can check, such as a name, a document, a report, or a demonstration. An answer you cannot verify tells you little about how the vendor will work.

The table below lists 12 questions grouped by the person who asks them. For each one, it shows what a strong answer includes and which answers should worry you, so you can use it in both the RFP and live interviews.

Send the questions with the RFP, then ask the same ones in live interviews and compare the two sets of answers. A gap between what a vendor wrote and what its team says is worth a follow-up question.

Question 8 often exposes a common misunderstanding. SOC 2 is an examination performed by certified public accountants (CPAs) that ends in a report, so there is no such thing as a SOC 2 certificate. A vendor that calls itself “SOC 2 certified” and cannot share the report has not answered the question.

The security review goes further than this one question, and your CISO needs a reliable way to read the documents a vendor sends.

Software Development Vendor Security Due Diligence: What the CISO Should Request

This section explains how your CISO checks the security evidence that vendors send with their answers. It sits here because the security gate decides who stays in the evaluation, and every later step depends on it.

Vendor security is a growing source of risk. Verizon’s 2026 Data Breach Investigations Report found that breaches involving a third party rose 60% from the previous year and now make up 48% of all breaches. The figure covers confirmed breaches worldwide, and “involving a third party” covers any third-party role, which is broader than fault.

Software development vendor security due diligence comes down to two tasks. You ask every finalist for standard security evidence, read it carefully, and then write your security requirements into the contract.

In every document, check these three things:

• What kind of evidence it is.
• Which period it covers and how recent it is.
• Whether its scope includes the people and locations that will work on your systems.

The table below describes the four most common types of security evidence. For each one, it shows what the document is and what your CISO should check before accepting it.

A few details in these documents are easy to miss:

• The SOC 2 period. A Type 1 report shows the vendor’s controls on a single day. A Type 2 report shows how those controls worked over several months, so it tells you much more. According to a CPA firm writing for the Cloud Security Alliance, there is no fixed length, and most reports cover 3 to 12 months.
• The ISO edition. Make sure the certificate is for the 2022 version of the standard. The certification body DQS explains that this version has a separate group of rules for working with suppliers and cloud services.
• The SIG edition. The SIG comes from Shared Assessments, and the organization updates it regularly. Check which version the vendor filled in and read the answers that matter most to you, such as who gets access and how the vendor handles security incidents.

Beyond the documents, we recommend asking each finalist how it handles these points:

• How engineers get and lose access to your systems.
• Where code and data live during the work.
• How the team reviews code and third-party libraries.
• How fast the vendor reports a security incident to you.
• Which subcontractors touch your work.

Once the evidence passes, write your requirements into the contract. As we mentioned earlier, the NIST Cybersecurity Framework 2.0 includes security requirements in supplier contracts as an expected outcome. If you need a full supply chain program, NIST Special Publication 800-161 Rev. 1 Update 1 covers it in depth.

Smaller vendors often do not have a SOC 2 report or an ISO certificate. In that case, ask them to fill in a SIG or CAIQ questionnaire and have your CISO review the answers. Your CISO then decides whether that is enough to pass the security gate.

Security clauses are only one part of the contract, and procurement and legal check the rest.

Contract, IP and Exit: What Procurement and Legal Check

This section covers the rest of the contract. Procurement and legal review it while the CISO checks security, so the terms are settled before you pick a winner.

Exit terms matter because companies change course more often than they expect. In Deloitte’s 2024 Global Outsourcing Survey of more than 500 executives, 70% said they had brought some previously outsourced work back in-house over the last five years. The survey doesn’t say why, and Deloitte sells outsourcing advice itself, so read the figure as a sign of how common these changes are.

Before you sign, make sure the contract covers these points:

1. Ownership of the work
   The contract states who owns the code and everything else the vendor produces for you. Ask your lawyers which of the vendor’s own tools stay with the vendor, under what license, and how open-source components are handled.
2. Subcontractors
   The vendor names every subcontractor in advance. Adding a new one needs your approval.
3. Security clauses
   The requirements from your CISO’s review become part of the agreement. That way, you can enforce them later.
4. Exit and handover
   The contract lists what you receive if you part ways, such as code, documentation, and access credentials kept in your own systems. It also states how long the vendor helps with the transition to a new team. The NIST Cybersecurity Framework 2.0 lists planning for the period after a supplier relationship ends as an expected outcome.
5. A phased commitment
   Start with a defined first phase that ends with a natural exit point. US federal IT rules follow the same idea and call it modular contracting, where the government buys a system in small, connected parts.

On price, compare the total cost and look beyond the hourly rate. Total cost includes your own management time and the cost of rework when quality slips.

Each pricing model also rewards something different. Time and materials pays for the hours worked, a fixed price pays for a defined result, and a managed service pays for an outcome the vendor owns.

With the terms on paper, the next step is to check whether the finalists can do the work they describe.

Verify with Work: Team Interviews, Reference Calls and a Paid Pilot

This section covers step 5. Proposals and documents show how a vendor writes, and the work shows how it delivers, so this step often decides between close finalists like Vendors A and B in our example.

Before you decide, we recommend three checks:

• Interview the people who will do the work.
• Call references from similar programs.
• Give up to three finalists a small paid task with acceptance criteria agreed in advance.

Team interviews

Interview the named engineers and the delivery lead who would run your program. The sales team can join the call, and the people doing the work should answer the technical questions themselves.

The Harvard guide we cited earlier makes the same point. It suggests interviews or product demos to go beyond written answers, especially in a second-round evaluation.

Reference calls

Ask each finalist for a client with a similar system or scale. Then ask that client a few direct questions:

• What went wrong, and how did the vendor handle it?
• Was the proposed team the team that showed up?
• Would you hire this vendor again?

Review platforms such as Clutch show patterns across many clients. Read the reviews from companies closest to your size, and still make the calls.

A paid pilot

A paid pilot is a small piece of your own work that each finalist completes for a fee. The US Digital Service (USDS) playbook on technical challenges explains that buyers benefit when vendors demonstrate their skills during the selection, and it suggests focusing on whether a team can solve technical problems.

Pilots take time and money on both sides. The same playbook calls them resource-intensive and recommends running them only after a first round has narrowed the list.

Agree on these points before the pilot starts:

• The task, taken from your own work and limited in size.
• The fee you pay for it.
• The acceptance criteria, including what gets delivered, the quality bar, and how the team communicates.
• The people on your side who review the result.
• What you keep afterward, such as code and documentation.

Add the pilot results to the scorecard. The same people should judge every finalist against the same criteria.

When a pilot is not worth it

A pilot rarely pays off for small, short, and well-specified work, or when nobody on your side has time to review it. In those cases, rely on interviews, references, and a first phase with an exit point, which we covered in the contract section.

Starting small also works without a pilot. For example, our team at MasterControl grew from 2 engineers to a 60-engineer delivery organization within approximately 18 months. This is one engagement, and team size alone says nothing about quality, but it shows how a small first step can grow into a long partnership.

For a modernization program, the evaluation needs a few extra checks beyond these.

Choosing a Partner for a Modernization Program

This section is for readers whose program changes a system that people already use every day, such as a customer portal or an internal platform. Modernization adds risks that a new build does not have, so it needs a few extra checks on top of the general process.

When you choose a software development partner for legacy modernization, judge it by its plan to move users and data in stages. The partner should also explain how the old and new systems will work together during the move and how it will roll back a stage that fails. Build speed matters less here than a safe path from the old system to the new one.

Ask each finalist these five questions:

1. Which users or data move first, and why?
2. How will the old and new systems run side by side during the move?
3. What is the rollback plan for each stage?
4. How will you check the migrated data?
5. Who prepares our support teams for the switch?

Case study: how MasterControl moved millions of users in stages

A phased migration is easier to picture on a concrete project. Here is how it looked in one of our engagements.

Challenge

MasterControl decided to move from a third-party identity service to its own identity provider. The move covered millions of user accounts, so the company needed a plan that let people keep signing in while the migration ran.

What we did

3 Zoolatech software engineers worked inside a joint team with MasterControl’s developers.

Together, the team:

• Moved lower-risk user groups first.
• Ran phased rollouts across development, test, and production environments.
• Built a hybrid integration with the existing multi-factor authentication (MFA) service to keep sign-ins working during early migrations.
• Created an automated migration tool with validation steps to check the migrated data.
• Gave support teams documentation, training, and live assistance.

Result

About 20% of users moved within the first 90 days, and more than 60% moved by the six-month mark. The team migrated millions of user accounts with zero disruption.

These figures are shares of users, and they come from one project. You can read more about how MasterControl moved millions of users to its own identity provider in the full case study.

If you still need candidates for your shortlist, our review of legacy software modernization companies is a good place to start.

Red Flags When Choosing a Software Development Partner

This section collects the warning signs from the whole process in one place. You can run every vendor against this list before the final decision.

The most serious warning sign is a vendor that refuses to show you something. For example, the vendor will not name the team, share its security report, give you a client to call, or agree on exit terms. Treat each refusal as a failed check, and don’t try to negotiate it away.

Watch for these red flags:

• The team is “to be assigned after signing”, or the CVs look copied from the internet.
• The vendor calls itself “SOC 2 certified” and has no report to share.
• No client from a similar program agrees to a reference call.
• The price is far below the others and comes with no written assumptions.
• The vendor offers a fixed price and timeline for a complex system before anyone has looked at it.
• Subcontractors appear only after the contract is signed.
• The contract has no exit or transition terms.

Buyers make their own mistakes too. Teams often choose the vendor with the best presentation, let price outweigh every other criterion, or set the weights only after reading the proposals. Another common mistake is bringing in security and legal after the team has already picked a winner.

Final Word

Figuring out how to choose a software development partner takes time. Your team has to agree on the rules, read every proposal, check security documents, call references, and often run a paid pilot before it is ready to decide.

This effort pays off. A wrong choice shows up later as rework, missed deadlines, and a contract that is hard to leave, and each of those costs far more than a careful evaluation.

Start with the rules. Agree on the gates and weights before the first proposal arrives, verify the finalists with work, and settle the exit terms before you sign. This is how to choose the right software development partner and still be able to explain the decision a year later.

If you are adding Zoolatech to your shortlist, here is what to expect from us.

Every Zoolatech engagement includes a dedicated Delivery Manager, a live risk register, and a formal quality assurance (QA) checkpoint, and our teams work in the client’s time zones. We are not the right fit for one-off engagements under six months or budgets under $250K.

If your program fits, our team is ready to discuss your systems and answer your evaluation team’s questions. You can also learn more about our legacy modernization services.