Azure OpenAI Service Primer: Model Selection, PTU, Prompt Engineering, Function Calling & Embeddings

Azure OpenAI Service is a managed service that delivers OpenAI's GPT models (gpt-4o, gpt-4o-mini, o1, o3 series), DALL-E (image generation), Whisper (speech recognition), and Embeddings models on Microsoft Azure. While functionally and performance-wise nearly identical to the native OpenAI API, Azure integration provides: 1) Microsoft Entra ID auth + Managed Identity (secretless), 2) fully private access via Private Endpoint (no public internet), 3) Customer-Managed Key encryption, 4) Microsoft Defender for AI integration, 5) enterprise compliance (SOC 2 / HIPAA / ISO 27001 / PCI DSS / FedRAMP), 6) Azure SLA 99.9% (99.99% with Provisioned Throughput Units), 7) Microsoft 365 / Power Platform integration. It is the standard choice for production enterprise generative AI and a core topic on the AI-103 exam.

Key Azure OpenAI models: 1) gpt-4o (multimodal, image input, high performance, low cost, default pick), 2) gpt-4o-mini (fast, cheapest, ~1/10 the price of gpt-4o, for high-volume workloads), 3) gpt-4 Turbo (legacy, long-term support), 4) o1-preview / o1-mini (reasoning models for complex inference, expensive), 5) o3-mini (latest reasoning model, cost-efficient), 6) text-embedding-3-large / small (embedding generation, required for RAG), 7) DALL-E 3 (image generation), 8) Whisper (speech recognition), 9) GPT-4o Realtime (real-time voice dialog). Selection: 1) general chat / summarization → gpt-4o-mini (cost-optimal), 2) high accuracy → gpt-4o, 3) complex reasoning, math, coding → o1 / o3, 4) RAG embeddings → text-embedding-3-large. Apply for quota in Model Catalog, then deploy; note that available models vary by region.

Standard Deployment (Pay-as-you-go): per-token billing (input + output), shared resources, scales with demand, throughput limits (tokens per minute / TPM), suitable for dev, PoC, and small-scale production. Provisioned Throughput Units (PTU): reserved capacity with fixed monthly cost, dedicated resources, latency guarantees, for large-scale production and 99.99% SLA requirements; minimum 50 PTU at several hundred thousand yen/month. Selection: under a few million tokens/month → Standard, tens of millions of tokens/month + latency requirements → PTU, enterprise production apps → PTU recommended. There is also a Global Standard vs Data Zone Standard choice: Global Standard processes in any Microsoft region (data region agnostic), Data Zone Standard processes within specific data-residency regions (GDPR / Japan data residency). Reserved Capacity (1-year contract) provides substantial PTU discounts.

Prompt Engineering is the discipline of designing inputs (prompts) to LLMs. Basic structure: 1) System Message (defines the assistant's role and constraints; the most important), 2) User Message (the user's question or instruction), 3) Assistant Message (response, retained as ChatHistory). Key techniques: 1) Zero-shot (no examples, simple questions), 2) Few-shot (3-5 examples to teach behavior patterns), 3) Chain of Thought (CoT — instruct "think step by step" to improve reasoning), 4) Role Playing (assign a persona like "you are a medical specialist"), 5) Structured Output (specify a JSON Schema for structured responses), 6) Temperature tuning (0 = deterministic, 1 = creative), 7) Top-p tuning (cumulative probability threshold), 8) Frequency / Presence Penalty (suppress repetition). In production, centrally manage a Prompt Library, iterate with A/B tests, and design defenses against Prompt Injection.

Function Calling (Tool Calling) lets an LLM autonomously invoke external functions / APIs. Common use cases: 1) weather API calls ("What's the weather in Tokyo?" → call get_weather('Tokyo') → summarize result), 2) database queries ("What were 2024 sales?" → call query_sales('2024') → format result), 3) calculations ("complex tax calculation" → call calculate_tax), 4) email / messaging ("notify Slack" → call send_slack_message), 5) calling other LLM APIs (multi-agent pattern). Implementation flow: 1) define Functions Schema (name, args, types, descriptions), 2) pass the function list via the Tools argument on ChatCompletion, 3) if the LLM decides a call is needed, it returns a tool_calls array, 4) the client executes the corresponding function, 5) the result is returned to the LLM as a Tool Message, 6) the LLM produces the final response based on the result. Azure AI Foundry Agent Service abstracts complex multi-step Function Calling and enables more advanced agent patterns.

Embeddings convert "text → vector (high-dimensional numeric array)" so semantic similarity can be computed. text-embedding-3-large is 3,072-dimensional (high accuracy); small is 1,536-dimensional (fast, low cost). Flow: 1) split documents into chunks (200-1,000 tokens), 2) vectorize each chunk via the Embedding API, 3) store vectors in a vector database (Azure AI Search Vector, Cosmos DB Vector, PostgreSQL pgvector), 4) vectorize the user's query via the Embedding API, 5) retrieve the top-K nearest chunks using Cosine Similarity / Dot Product, 6) include retrieved chunks in the GPT-4o prompt to generate the response (RAG pattern). Azure AI Search's Vector Search is built-in, and Hybrid Search (Vector + Full-text + BM25) further improves accuracy. See the RAG Pattern Implementation Guide for the detailed implementation.

Azure OpenAI security features: 1) Microsoft Entra ID auth + Managed Identity (secretless, no API keys), 2) Private Endpoint for fully private access (privatelink.openai.azure.com), 3) Customer-Managed Key encryption (Key Vault integration), 4) Microsoft Defender for AI for anomalous-usage detection, 5) Content Filter auto-blocks Hate / Self-harm / Sexual / Violence (4 severity tiers), 6) Prompt Shield detects Jailbreak / Prompt Injection (direct + indirect attacks), 7) Groundedness Detection catches Hallucinations, 8) Protected Material Detection flags copyright-infringing content, 9) Usage Logs (Diagnostic Logs) ship all Prompt / Response audit logs, 10) Cost Management integration for token spend. In production, Content Filter + Prompt Shield + Microsoft Sentinel integration for anomaly detection plus Logic App Playbook auto-response is the standard pattern, and Responsible AI 6-principle compliance is mandatory.

AI-103 (Developing AI Apps and Agents on Azure, GA 2026-06) covers Azure OpenAI deeply in Domain 2 (Generative AI 30-35%) and is the flagship certification for this area. AI-901 (AI Fundamentals, GA 2026-06, AI-900 successor) covers generative AI fundamentals; AZ-204 (Developer Associate, retiring 2026-07) covers Azure OpenAI SDK from a developer view; SC-100 (Cybersecurity Architect Expert) covers AI security and Responsible AI; DP-700 (Fabric Data Engineer) covers data prep pipelines for AI; AZ-305 (Solutions Architect Expert) covers AI architecture decisions. For Azure AI engineers, understanding Azure OpenAI is mandatory and is a core focus area for AI-103 study.