OCI Enterprise AI Agents

From Agent Logic to Production Runtime

Framework = how the agent thinks |  Runtime = how the agent operates in production 

 

1. Agent Logic vs Runtime Architecture

The car analogy separates the intelligence of an agent from the infrastructure required to run it reliably.

Car component	AI-agent equivalent	Purpose
Engine	Agent logic	Reasoning, decisions, tool calling and prompt orchestration.
Fuel system	Execution environment	Runs the application code.
Cooling system	Scaling and reliability	Keeps the service stable under load.
Roads	Request handling and APIs	Allows users and applications to reach the agent.
Traffic control	Observability and logs	Shows what happened during execution.

• Agent logic: LangChain, OpenAI Agents SDK or Responses API.

• Runtime architecture: hosting, APIs, authentication, state, tools, scaling, logging and monitoring.

Designing the engine is not the same as driving safely and reliably on the road. 

2. The Beginner Problem: It Works Locally, Now What?

from langchain import create_agent 
from openai import OpenAI 
 
agent = create_agent( 
    model="gpt-4.1", 
    tools=[search, calculator] 
) 
 
result = agent.invoke("What is 15 x 8 / 3?") 
print(result) 
 
# Works on your laptop. 
# Now what? 

• Where does the agent run when the laptop is closed?

• What happens when 1,000 users call it at once?

• How do users call the agent?

• Where is conversation history stored?

• What happens when a tool call fails?

• How can prior actions be investigated?

3. The Three-Layer Agent Stack

Layer	What it contains
User / App	UI, API call or chatbot interface.
Runtime architecture	Request handling, execution, orchestration, state, sandboxing, integrations, scaling and observability.
Agent logic	Reasoning, tool calling and frameworks such as LangChain or the OpenAI Agents SDK.

User / App  ->  Runtime Architecture  ->  Agent Logic 

4. Inside the Runtime

Component	Brief purpose
Request handling	Authentication, API gateway and routing.
Execution environment	Server, container or serverless compute.
Orchestration & state	Memory, multi-step workflows and retries.
Tool sandbox	Isolation and resource limits for code, browsing and API calls.
Integration layer	Connectors and credentials for databases, APIs and enterprise systems.
Scaling & reliability	Auto-scaling, load balancing and failover.
Observability	Logs, traces and metrics.

5. OCI Enterprise AI Platform

Platform area	Value
OCI Enterprise AI Models	Adds intelligence through model access and managed inference.
OCI Enterprise AI Agents	Adds action through management, orchestration and runtime.
OCI Enterprise AI Governance	Adds trust through guardrails, security and visibility.

6. OCI Enterprise AI Agents

Area	Capabilities shown
Agent Management	Authoring, testing and evaluation, observability, hosted deployment.
Agent Orchestration	Human-in-the-loop, MCP, A2A and open-source frameworks.
Agent Runtime	Responses API, tools, vector stores and memory.

What you focus on

• Agent instructions and persona.

• Tools the agent can call.

• Knowledge bases used for grounding.

• Business outcome.

What OCI handles

• Hosted endpoints and scaling.

• Memory and sessions.

• Sandboxed tools.

• Logging, monitoring and audit.

• Integrations with OCI data and services.

7. Two Approaches to Building

OCI Responses API	Hosted Agentic Application
Application calls a managed API.	Package the framework-based agent as a container.
OCI handles the orchestration loop.	Upload the image to OCI Registry and deploy it.
No agent infrastructure to deploy.	OCI provides endpoint, scaling, networking, storage and authentication.

8. Two Paths to Production

Do it yourself	Managed OCI runtime
Provision servers, containers or serverless.	Fully hosted execution environment.
Build authentication, request handling and gateway.	Built-in API endpoints and request handling.
Implement state, session storage and retries.	State and sessions managed.
Secure tool execution and credentials.	Tools sandboxed by default.
Build scaling, failover and observability.	Scaling plus OCI logging, monitoring and audit.

9. Deployment: Local Development to Production

Step	Action
1. Build	Develop locally with LangGraph, LangChain, OpenAI Agents SDK, AutoGen or another framework.
2. Package	Create a container image with the code and dependencies.
3. Upload	Push the image to OCI Container Registry (OCIR).
4. Configure	Set scaling, storage, networking and authentication.
5. Deploy	Create the deployment using the container image.
6. Invoke	Use the OCI-provisioned HTTP endpoint.

10. Agent Tools

Tool	Purpose
File Search	Semantic retrieval from uploaded documents.
Code Interpreter	Execute Python in a sandboxed environment.
Function Calling	Invoke custom business functions.
MCP Calling	Connect to remote MCP servers.

11. Agent Memory

Memory type	Purpose	Analogy
Short-term	Responses API + Conversations API maintain state across turns.	A notepad in a meeting.
Long-term	Durable memory across conversations using a unique subject_id in a project.	A personal journal.
Context optimization	Condenses chat history to reduce latency and token usage.	Summarizing a long meeting.

12. OCI Responses API

• OpenAI-compatible APIs with minimal SDK workflow changes.

• Built-in reasoning across multiple tool calls.

• Multi-model routing across supported model providers.

• State management for conversation history, memory and context compaction.

13. OCI Enterprise AI Models

Model type	Purpose	Examples shown in screenshots
Chat	Question answering, summarization, content, code, reasoning and agent workflows.	Cohere Command A, Meta Llama 4, xAI Grok 4, Google Gemini 2.5, OpenAI gpt-oss.
Embed	Convert text or images into semantic vectors.	Cohere Embed 4, Embed English v3, Embed Multilingual v3.
Rerank	Sort candidate documents by relevance.	Cohere Rerank 3.5.

Typical retrieval flow: Query -> Embedding search -> Candidate documents -> Reranker -> Best results 

14. Model Serving Modes

On-Demand	Dedicated Cluster
Shared GPU infrastructure across tenants.	GPUs dedicated to one tenancy.
Pay per inference call.	Commit to cluster hours.
Suitable for experiments, PoCs and variable load.	Predictable latency and production SLA use cases.
Dynamic throttling may require exponential backoff.	Required for fine-tuned and imported custom models.
No cluster provisioning required.	GPU isolation and sustained high-volume workloads.

15. Authentication

Method 1: API key

from openai import OpenAI 
import os 
 
client = OpenAI( 
    api_key=os.getenv("OCI_API_KEY"), 
    base_url=( 
        "https://inference." 
        "generativeai.us-chicago-1" 
        ".oci.oraclecloud.com" 
        "/20231130/actions/v1" 
    ) 
) 

Method 2: OCI IAM

import httpx 
from openai import OpenAI 
from oci_genai_auth import OciSessionAuth 
 
client = OpenAI( 
    base_url="https://inference.generativeai...com/openai/v1", 
    api_key="not-used", 
    http_client=httpx.Client( 
        auth=OciSessionAuth() 
    ) 
) 

IAM policy shown

allow any-user to use generative-ai-family in compartment <your-compartment> where ALL { request.principal.type='generativeaiapikey' } 

16. First OCI Agent: Simple Chat

from openai import OpenAI 
import os 
 
OCI_REGION = "us-chicago-1"   # CHANGE THIS to your region 
OCI_BASE_URL = ( 
    f"https://inference.generativeai.{OCI_REGION}" 
    f".oci.oraclecloud.com/openai/v1" 
) 
MODEL = "openai.gpt-oss-120b" 
OCI_PROJECT_ID = "ocid1.generativeaiproject.oc1.us-chicago-1.xxxxxxxx" 
 
# Create client pointing to OCI 
client = OpenAI( 
    api_key=os.getenv("OCI_GENAI_API_Key"), 
    base_url=OCI_BASE_URL, 
    default_headers={ 
        "OpenAI-Project": OCI_PROJECT_ID, 
    }, 
) 
 
# Call the Responses API 
response = client.responses.create( 
    model=MODEL, 
    input="Explain what an AI agent is in one paragraph." 
) 
 
print(f"Response:\n{response.output_text}\n") 

17. Native OpenAI vs OCI-Compatible Responses API

Native OpenAI Responses API

from openai import OpenAI 
 
# Reads OPENAI_API_KEY from the environment 
client = OpenAI() 
 
response = client.responses.create( 
    model="gpt-4.1", 
    input="Explain what an AI agent is in one paragraph." 
) 
 
print(response.output_text) 

OCI-compatible configuration

from openai import OpenAI 
import os 
 
OCI_REGION = "us-chicago-1"   # CHANGE to your region 
OCI_BASE_URL = ( 
    f"https://inference.generativeai.{OCI_REGION}" 
    f".oci.oraclecloud.com/openai/v1" 
) 
MODEL = "openai.gpt-oss-120b" 
OCI_PROJECT_ID = "ocid1.generativeaiproject.oc1.us-chicago-1.xxxx" 
 
client = OpenAI( 
    api_key=os.getenv("OCI_GENAI_API_Key"), 
    base_url=OCI_BASE_URL, 
    default_headers={ 
        "OpenAI-Project": OCI_PROJECT_ID, 
    }, 
) 
 
response = client.responses.create( 
    model=MODEL, 
    input="Explain what an AI agent is in one paragraph." 
) 

OCI_GENAI_API_Key=sk-xx 

18. OCI Enterprise AI Governance

Governance area	Controls shown
Network security	Private endpoints, secure boundaries, ZPR identity-based communication and private API access.
Identity control	IAM policies, resource-level controls, compartment/tenancy policies and least privilege.
AI behavior	Guardrails, moderation, prompt-injection defense and PII detection/masking.

19. End-to-End Mental Model

User / Application

↓

Request Handling + Authentication

↓

Agent Runtime

↓

Agent Logic / Orchestration

↓

Model + Memory + Knowledge

↓

Tools / Functions / MCP / Code Interpreter

↓

Enterprise Systems and Data

↓

Response

Security + IAM + Guardrails + Scaling + Reliability + Logging + Monitoring + Audit 

20. Key Takeaway

Concept	Meaning
Agent logic	The intelligence written with LangChain, OpenAI Agents SDK or the Responses API.
Runtime architecture	The production system for execution, APIs, state, tools, scaling, reliability and observability.
OCI Enterprise AI Agents	A managed environment for the agent lifecycle and production runtime.

Framework = how the agent thinks.  Runtime = how the agent survives production.  OCI Enterprise AI Agents = managed lifecycle + runtime.