Choosing the right AI platform V2

For those not interested in the full context (or you just don’t have time), there is a TLDR at the end of this article. However, it’s helpful to have a fuller context around your decisions, so I would suggest reading the whole thing if possible.

Choosing an AI platform used to sound like a cloud infrastructure question. Which provider already has approval? Where does the data live? Can we build within the client’s existing environment?

Those questions still matter. But a team building AI into a product now has several connected decisions to make: which models to use, how to give them business context, what actions to let them take, and how to keep the whole system useful as traffic, costs, and expectations grow.

A convincing demo can answer a question. A production system has to answer the right question, respect permissions, survive failures, and give someone a clear way to fix it when it gets something wrong.

Here is how we would approach that decision today.

What changed since our original guide

In January, we published our comparison of Amazon Bedrock, Azure OpenAI, and Google Vertex AI. The article, subsequently updated in August, focused on choosing a cloud platform for client work.

Its strongest advice still applies: existing infrastructure and vendor approval can make one route much easier to deliver. But the three-cloud comparison is too narrow to serve as a complete buying framework. Direct APIs, model gateways such as OpenRouter, and inference platforms such as Fireworks AI deserve consideration. Enterprise cloud platforms also extend well beyond model access into agent execution, tools, evaluation, and governance.

Microsoft’s current documentation describes Microsoft Foundry as a platform for models, agents, and tools. Google describes Gemini Enterprise Agent Platform as an evolution of Vertex AI. AWS positions Bedrock AgentCore around operating production agents. These are overlapping capabilities, rather than three distinct personalities for three kinds of team.

Microsoft Foundry · Google Agent Platform · AWS AgentCore

The useful shift is to evaluate the system you need to operate, then choose the services that make it manageable.

Understand the layers before comparing vendors

“AI platform” can describe a model API, a development framework, a managed runtime, or a business application. Comparing them as though they replace one another leads to confusing decisions.

A product might combine a direct model API, its existing application backend, a search index, and a queue. Another might use a cloud provider’s managed agent runtime. Both can be appropriate enterprise architectures.

Employee tools such as chat assistants and coding assistants answer a related but different question. Buying seats can improve how a team works. Building AI into a customer product also requires application permissions, deployment, support, and a commercial model for ongoing usage.

Think about an AI assistant that helps customers check on an order. The model understands the question and writes the reply. Your business systems supply the actual order details. Tools let the assistant look up a shipment or request a change. The runtime keeps that work moving, even when several customers ask at once or a system takes too long to respond. Evaluation and operations help your team catch wrong answers and fix problems.

You can get several of those pieces from one company or combine services from different providers. The important thing is knowing which job each piece handles. That makes it easier to compare what vendors are offering, see what your team still needs to build, and change one part later without rebuilding the whole experience.

The platform options worth evaluating

Fireworks AI is our preferred starting point

Fireworks AI focuses on serving and training open models. Its platform offers serverless inference, dedicated GPU deployments, and fine-tuning, with OpenAI-compatible interfaces for supported operations. This gives teams a managed route from testing an available model to operating a more specialized deployment. Fireworks AI overview

Our opinion is that Fireworks AI is the way to go for US-based teams that want managed open-model inference, model flexibility, and a clear path to keeping inference in the US. Its combination of hosting options, data controls, and customization makes it our preferred starting point. The deployment still needs to meet the project’s model, performance, and contractual requirements.

Fireworks documents a US-only serverless option using a US endpoint and supported model IDs. That is a specific hosting choice, rather than a promise that every Fireworks request runs in the US. Its open-model inference policy also provides zero persistent retention of prompts and outputs unless you opt in, while retaining service metadata. Its Response API has an important exception: conversation storage defaults to 30 days unless store=False is set.

US-only hosting · Retention policy

In everyday terms, where a model was developed and where your customer’s data goes are separate questions. Running an open model on Fireworks means Fireworks serves that deployment; each request does not need to go to the company that originally developed the model. For a US customer, combining an appropriate US deployment with the right retention settings can make the data handling easier to understand and approve.

That is why we favor Fireworks: a direct relationship with the inference host, a documented US-only option, and access to open models without having to operate the serving infrastructure ourselves. Those are concrete reasons for confidence. A company’s US address alone would not establish the same protections.

For example, a manufacturer processing thousands of similar service documents could test whether a smaller model meets its extraction requirements. If it does, dedicated capacity or a tuned deployment may offer better economics or performance than sending every document to a general-purpose frontier model. Measure that advantage using accepted outputs, actual throughput, and the full operating bill.

The distinction from OpenRouter matters. OpenRouter’s role here is access and routing across providers. Fireworks’ role is inference and model customization. The shortlist can include both, with each evaluated for the responsibility it would hold.

Open model weights can also improve future hosting options when their license and availability permit. Preserve training data, evaluation sets, and access to any custom model artifacts you would need to move. Fireworks-specific training or deployment features can still create migration work.

OpenRouter for model access across providers

OpenRouter gives developers a unified API for models from multiple providers. Its routing controls let teams select eligible providers and express preferences such as price or performance. Provider routing and model fallback are separate choices: one changes who serves a model, while the other can change the model itself.

OpenRouter quickstart · Provider routing · Model fallbacks

That makes it a strong starting point when a product benefits from comparing and using several model families. A team can test alternatives through a common integration and assign different models to different workloads without maintaining a separate integration for every provider.

We would consider OpenRouter a leading candidate for a product whose model choices will evolve. The benefit is negotiating room: a change in quality, availability, or economics can become a routing decision rather than a major integration project.

Production routing still needs boundaries. OpenRouter documents provider-specific training and retention policies, with controls for restricting eligible providers. Apply the approved data policy to every possible destination, including fallbacks. Review OpenRouter’s own handling separately. Provider data policies

OpenRouter is also US-based; its company profile lists New York as its headquarters. It offers US In-Region Routing on Business and Enterprise plans. Through that configuration, it says requests are decrypted and processed within the US and fail rather than route outside the region when no eligible endpoint is available. Its global endpoint does not provide that same geographic guarantee.

Company profile · US routing

OpenRouter says prompt and response storage is opt-in, and it provides controls to require zero-retention downstream endpoints. These are meaningful protections, but the review covers both OpenRouter and the provider running inference. We prefer Fireworks when a simpler relationship with the inference host is the priority; OpenRouter remains a strong option when access across a broader range of providers is more important.

OpenRouter data collection · Zero-retention routing

A unified API reduces integration work; it does not make models interchangeable. Evaluate tool calls, structured output, context limits, and task quality for each approved route. Include the gateway’s fees, availability, and support arrangements in the business case.

Why an independent model layer may be the best fit

For a custom product that needs model flexibility, we would start with Fireworks AI and compare OpenRouter, direct APIs, and cloud platforms against the workload. These options address a practical problem: the best model for one part of a product may come from a different developer than the best model for another part.

Consider a warranty workflow. A smaller open model might extract claim fields. A stronger model might interpret an ambiguous policy. Application code would validate the account and enforce approval rules. The team could change the model used for extraction without rebuilding the claim process.

That architecture creates several potential advantages:

• Better task fit. Choose models using measured results for each job.
• More control over cost. Reserve expensive reasoning for the work that needs it.
• A simpler path to new models. Evaluate a candidate behind a stable application interface.
• More options during disruption. Maintain tested alternatives for eligible workloads.
• Greater commercial flexibility. Compare providers when pricing or service terms change.

These advantages are reasons to evaluate the approach, rather than guarantees that a gateway or inference platform will always win. Bedrock and other cloud catalogs also offer multiple model families. Cloud providers can remain the strongest fit when their identity, networking, procurement, and managed operations materially simplify delivery.

Build portability into the application as well as the API connection. Keep business rules, permissions, retrieval, and durable workflow state in clearly defined components. Version prompts and tool schemas, retain your evaluation set, and isolate provider-specific capabilities behind adapters.

Start with one proven model path and add alternatives where they solve a measured need. Independence means preserving the ability to choose, with enough testing to make that choice safely.

OpenAI directly

OpenAI’s documentation distinguishes direct model integration through the Responses API, application-controlled orchestration through the Agents SDK, and managed execution through the Agents API. That distinction gives teams a useful starting point: decide how much execution and state management they want to own. OpenAI agent options

We would shortlist a direct integration when the product needs OpenAI capabilities and a direct vendor relationship fits the organization. Test the exact combination of model, tools, output format, and response time the application needs.

The engineering question is where the application boundary should sit. A focused extraction feature may need a model call and validation. A long-running task may benefit from managed execution. Either way, your product still owns its business rules and the decision to accept an outcome.

Anthropic directly

Claude is available through Anthropic’s direct API and cloud access routes. Anthropic explicitly notes differences in feature availability, billing, and platform operation across those routes. Choosing Claude and choosing how to buy and run Claude are separate decisions. Claude API and cloud access

Shortlist the direct API when its capabilities fit your workload and direct access fits procurement. Compare it with cloud access using the same test cases, including tool behavior, document handling, throughput, and error recovery.

Avoid assigning permanent jobs to brands, such as one model for creativity and another for reasoning. Test the work your users actually need to complete. Those results are more useful than a reputation that may belong to an earlier release.

Amazon Bedrock and AgentCore

For an organization operating in AWS, evaluate model access through Bedrock alongside the infrastructure already supporting the application. AgentCore adds services for running agents and connecting them to tools, with identity and operational capabilities. AWS documents support for multiple frameworks and models. Bedrock AgentCore

We would shortlist this route when existing AWS operations, procurement, and access management reduce the amount of new infrastructure the team must introduce.

Keep model access and agent hosting distinct in the architecture. You may need one without the other. Verify the exact model, region, capacity, and service capabilities required by the workload before committing.

Microsoft Foundry and Azure OpenAI

Azure OpenAI remains relevant to teams evaluating OpenAI access within Azure, but the broader Microsoft comparison should include Foundry. Its documented scope includes a model catalog, managed agents, evaluation, tracing, identity, and network controls. Some capabilities remain in preview. Microsoft Foundry overview

We would shortlist this route for organizations whose Azure infrastructure and Microsoft identity environment are already central to delivery.

Using Microsoft 365 does not automatically make an application integration straightforward. Access to documents, enterprise search, and business systems still needs deliberate configuration. Test the supported deployment type and network setup, rather than assuming every capability in the catalog is available under the same conditions.

Google Gemini and Agent Platform

Google’s current enterprise documentation describes Gemini Enterprise Agent Platform as the evolution of Vertex AI. It brings model-based solutions and agents into a broader enterprise platform, with managed runtime, sessions, monitoring, and access controls documented separately.

Platform overview · Runtime and operations

For teams already evaluating Vertex AI, expand the shortlist to this current offering. Compare it with the Gemini Developer API when deciding how to access Google models.

We would pay particular attention to this route when the application’s existing data and operations live in Google Cloud. Choose the services the feature requires. A model integration does not automatically require a custom training pipeline or a complete machine learning environment.

Platforms close to enterprise data

If the organization already relies on Snowflake or Databricks, include their AI offerings in the evaluation. Snowflake documents Cortex AI functions and managed Cortex Agents. Databricks positions Agent Bricks around building and governing enterprise agents.

Snowflake AI functions · Cortex Agents · Databricks Agent Bricks

These options are worth testing when an application depends heavily on governed business data. Keeping execution close to an existing data environment may simplify the architecture, but that is a workload-specific benefit to verify.

Check how business definitions, row access, and tool execution permissions carry through to the final answer. A technically valid query can still answer the wrong business question.

Open models and infrastructure you operate

Some workloads justify serving model weights in infrastructure the organization controls. vLLM is one available serving engine, with documentation covering online inference, deployment, scaling, and monitoring. vLLM documentation

Consider this route when requirements for deployment control, customization, or sustained workload economics justify the operational work.

Include hardware capacity, utilization, upgrades, security, redundancy, and engineering time in the comparison. Available weights do not make inference free. Review the license for each model and measure the effect of quantization or other deployment choices on your own tasks.

Frameworks and automation tools fill different roles

A framework can help assemble an application without supplying the model or operating all of its infrastructure.

Vercel’s AI SDK provides TypeScript interfaces for model access, streaming, structured output, and tools. It is worth evaluating for teams building AI features into web applications. LangGraph focuses on stateful orchestration, including persistence and human intervention. LangSmith covers tracing and evaluation in that ecosystem.

AI SDK · LangGraph

Choose an application SDK when the main challenge is integrating model behavior into your product. Consider explicit workflow orchestration when work must pause, branch, resume, or wait for approval. Keep the framework optional until a real requirement makes it useful.

Visual workflow platforms such as n8n also deserve a place in the comparison for business process automation. They can connect AI steps with existing systems and make a workflow easier for an operations team to inspect. n8n

Before using any workflow tool for a critical process, verify versioning, credential ownership, retries, approval handling, and support responsibility. Ease of assembly is one part of the operating model.

Choose the workflow before choosing an agent

This is a central theme in the AI for Leaders seminar series we’re leading at Triptych. The first session, Audit, focused on how leaders can begin using AI personally. The second, Rewire, focused on workflows: looking closely at how work moves through a team and where AI can help. That starts with understanding the steps, handoffs, and decisions in a process before choosing the technology to support it.

For example, “we need an AI agent” gives a team very little direction. “We need to review incoming invoices, identify mismatches, and get exceptions to the right person” gives everyone a workflow they can examine and improve. From there, it becomes much easier to decide which steps belong in software, which could benefit from AI, and which need a person’s judgment.

A fixed workflow tells the system which steps to perform. An agent has room to choose its next action based on the task and results so far.

That flexibility is useful when the path genuinely varies. It also makes testing and cost control harder. Anthropic’s engineering guidance recommends starting with simple approaches and adding complexity when it improves outcomes. Building effective agents

Consider invoice exception routing. Software can validate required fields, check purchase order records, and calculate totals. A model can interpret an unusual description or suggest a reason for a mismatch. The workflow can then send the exception to the appropriate person.

An open-ended agent may add little value there. A research task that must investigate several sources and adapt to missing information may benefit much more.

Use multiple agents only when distinct responsibilities or parallel work produce a measurable benefit. Each additional handoff introduces another place for context to be lost and another unit of work to pay for.

If you’d like help identifying where AI belongs in your organization and developing a practical plan for moving forward - join us Thursday, October 22, from noon to 1:30 p.m. at Triptych’s office in downtown Chattanooga. Lunch is included. Sign up for Rebuild.

Give AI the right business context

Most useful enterprise applications need information the model cannot reliably supply from training alone.

Use retrieval augmented generation, or RAG, when the system needs relevant passages from approved documents. Use application tools when it needs current inventory, account status, prices, or other live records. Keep calculations and business rules in software where they can be checked directly.

Retrieval needs an ingestion process as well as a search query. Decide how documents are parsed, split, indexed, updated, and removed. Preserve their source and effective date. Test whether the system retrieves the right revision when several versions look similar.

Permissions belong in retrieval itself. An assistant should only receive information the current user is allowed to access. The same boundary must hold for caches, conversation history, generated files, and citations.

A larger context window can help with some tasks, but sending more material does not establish that it is current, relevant, or authorized.

Fine-tuning addresses a different need: changing a model’s learned behavior for a task. Consider it after measuring a persistent gap in behavior or efficiency. It is a poor substitute for a reliable connection to changing business facts.

Treat tools as application capabilities

Tool calling lets a model request an action, such as looking up an order or preparing a draft. Your software should decide whether the request is valid and permitted.

The Model Context Protocol, or MCP, standardizes connections between AI applications and services exposing tools and other context. That can make integrations easier to reuse. It does not remove the need to authorize each action. MCP architecture

Design tools around narrow business capabilities. A tool that checks one customer’s order is easier to govern than unrestricted database access. Validate arguments, enforce tenant boundaries, limit returned data, and record the outcome.

For actions with consequences, define exactly when review is required. Preparing a refund recommendation and issuing a refund are different permissions. A reviewer should see the proposed action, its supporting evidence, and what will happen when it is approved.

Also treat retrieved documents, websites, and tool responses as untrusted input. Text inside them can try to redirect the system. Access controls and execution limits must continue to work even when the model follows a misleading instruction.

Design for production traffic and failures

A production architecture needs to account for work that takes longer than a normal web request.

For short interactions, streaming lets users see an answer as it is generated instead of waiting for the entire response to finish. In a customer support assistant, for example, the first few sentences can appear while the rest of the explanation is still being written. This can make the experience feel much more responsive, even when the total processing time stays the same.

Design that feedback around what the system is actually doing. If it needs to look up an order before answering, show “Checking your shipment” while that lookup runs. Clearly distinguish work in progress from a completed result, and only confirm an action once the connected system reports success. For structured outputs or information that requires validation, show progress first and display the result once it has passed those checks.

Retries need particular care when a tool changes something. If a request times out after creating an order, repeating it blindly can create a second order. Use idempotency keys or equivalent business safeguards, then reconcile uncertain outcomes.

Define limits for model calls, tool calls, elapsed time, and spend per job. Add bounded retries with backoff and queue limits so an upstream slowdown does not become an uncontrolled backlog.

Capacity planning should include burst traffic, token volume, tool latency, and the slowest acceptable user experience. “Serverless” still has quotas and capacity constraints.

For fallback models, test the complete workflow. Another model may accept the request while producing different tool calls or weaker results. In some situations, delaying the task or handing it to a person is the better fallback.

Evaluate the outcome you intend to sell

The evaluation set should resemble the work the application will receive: ordinary requests, incomplete inputs, conflicting documents, unauthorized requests, and failures in connected systems.

Define success before comparing platforms. For a support assistant, that might include factual accuracy, appropriate escalation, and resolution. For document extraction, it might include field accuracy and the ability to flag missing information. For an agent, inspect the actions it took as well as the final response.

Use deterministic checks for schemas, totals, required fields, and permission boundaries. Use human review and calibrated model-based grading for qualities that require judgment. A second model’s approval is useful evidence, but it is not proof of correctness.

Run the same tests when changing prompts, retrieval logic, tools, or models. Record the versions used so a production regression can be investigated.

Start with read-only access or draft output. Expand permissions after the team has evidence that the workflow performs reliably. The release gate should reflect the consequences of a mistake, with separate thresholds for critical failures.

Measure cost per completed task

Token pricing matters, but the unit the business cares about is a useful result.

Track model calls, tool fees, retrieval, infrastructure, retries, and human correction. Include the engineering and support needed to operate the feature.

As a hypothetical example, a workflow costing $0.08 per attempt and producing an accepted result on 80% of attempts has a model-and-tool cost of $0.10 per accepted result, assuming the unsuccessful attempts produce no accepted output. That calculation still excludes review and other operating costs.

Cheap attempts can become expensive outcomes when they require frequent correction.

Use smaller models where tests show they are sufficient. Reduce irrelevant context, reuse stable work where safe, and process nonurgent work in batches where the chosen service supports it. Introduce model routing only when its savings exceed the complexity of evaluating and maintaining several execution paths.

Make governance specific to the deployment

An enterprise product label is the beginning of a review. The review needs to cover the exact service and configuration you will deploy.

Confirm where prompts, files, tool results, and traces are processed and retained. Review model training use separately from retention and abuse monitoring. Include any external tool providers and observability services in the data flow.

Verify region and networking requirements for every selected component. Check whether a feature is generally available, in preview, or subject to separate access approval.

For applications serving several customers, isolate credentials, data, retrieval results, and job state by tenant. Decide who can inspect traces, how long they remain available, and how deletion requests propagate through derived data.

Give the system an operational owner. Someone needs to review failures, manage access, approve upgrades, and turn off a problematic capability. An AI feature without ownership becomes a support problem waiting for traffic.

A practical selection process

Start with a short description of one workflow: the trigger, inputs, desired outcome, allowed actions, and exceptions. Record how the work performs today so improvement can be measured.

Then apply hard constraints. Remove options that cannot meet requirements for data handling, region, licensing, network access, or procurement.

Compare the remaining options on the same representative workload. Include response quality, completion rate, latency, throughput, and operating cost. Score the actual deployment route, since the same model family can have different capabilities and limits across services.

Treat this as a shortlist, then let the tests decide.

Before committing, run a limited pilot with real users, an assigned owner, and clear stop conditions. Prove the failure path as well as the successful demonstration. Make sure the team can export the data and configuration it would need to change course.

What we would build first

Imagine a manufacturer wants a warranty assistant.

The first version could retrieve current warranty terms, collect the required claim information, and prepare a recommendation for a service representative. Application code would verify the product and customer record. The representative would approve the next step.

That version creates a useful evaluation boundary. The team can measure whether the assistant finds the correct policy, gathers the necessary evidence, and prepares a sound recommendation.

Once that works, the system might create a claim automatically under defined conditions. More consequential actions would retain their own approval rules. The platform choice would follow the manufacturer’s infrastructure, data requirements, and measured workload.

The same approach works for a member services portal, a quote intake tool, or an internal knowledge assistant. Pick a useful job. Make the evidence and permissions explicit. Expand from observed results.

Build around the work

For US-based teams building with open models, Fireworks AI is our preferred starting point. Its managed inference, documented US-only hosting option, retention controls, and customization capabilities make a strong combination. OpenRouter is a valuable alternative for broader provider access, while Bedrock, Microsoft Foundry, and Google’s Agent Platform can be the strongest fit where existing enterprise infrastructure drives the decision.

The strongest architecture gives each part a clear responsibility. Models interpret and generate. Business systems provide authoritative records. Software enforces rules. People handle decisions that need judgment or accountability.

At Triptych, that is where we would begin: understand the workflow, identify where AI can improve it, and build a system the team can operate with confidence.

If you are deciding how AI fits into a product or business process, let’s talk through what it needs to do.

TLDR

For US-based teams building with open models, Fireworks AI is our preferred starting point. Its managed inference, US-only hosting option, retention controls, and customization capabilities offer a strong combination of flexibility and control when configured for the project’s requirements. OpenRouter is a strong alternative for accessing models across more providers, while an existing AWS, Microsoft, or Google environment may make its own AI services the best fit. Choose a platform that lets you use the right model for each job and change course as your needs evolve.

Start with the work you want to improve. Map the steps, identify the information each step needs, and decide where AI can interpret, recommend, or generate something useful. Keep calculations, permissions, and firm business rules in software, and make human review part of the process where the consequences call for it. A focused workflow can deliver substantial value without needing an open-ended agent.

Production AI also needs a plan for what happens after the demo. Test it against real tasks, measure cost per useful result, and design for slow responses, service failures, and changing models. Give users clear feedback, protect customer data throughout the process, and assign someone to monitor and improve the system. The goal is an experience your customers can trust and your team can operate reliably.