Snowflake Document AI: Extract Data from Unstructured Documents

Document AI is a managed Snowflake capability that extracts structured data — text, numbers, dates, and more — from unstructured documents like PDFs and images. It combines pre-trained OCR (Optical Character Recognition) and NER (Named Entity Recognition) models so you can pull required fields from invoices, receipts, contracts, and medical records directly with SQL.

Document AI Architecture

ドキュメント（PDF/画像）
  → Stage（Internal or External）に格納
    → BUILD_SCOPED_FILE_URL() でファイル参照
      → Document AI モデル（PREDICT関数で推論）
        → VARIANT型の抽出結果
          → FLATTEN + SELECTで構造化テーブルに格納

Document AI runs on Snowflake's Serverless Compute, so you don't have to provision a warehouse yourself. Extraction results come back as a VARIANT value, and you can access individual fields using JSON path notation.

Creating a Document AI Model

-- Document AIモデルの作成（Snowsight UIで作成するのが推奨）
-- SQLでの作成例
CREATE OR REPLACE SNOWFLAKE.ML.DOCUMENT_INTELLIGENCE
  invoice_extractor
  FROM @doc_stage
  USING (
    SELECT
      BUILD_SCOPED_FILE_URL(@doc_stage, relative_path) AS file_url
    FROM DIRECTORY(@doc_stage)
    WHERE relative_path LIKE '%.pdf'
    LIMIT 10
  );

In the Snowsight UI you can visually annotate the fields (Values) you want to extract on the uploaded sample documents, then fine-tune the model from there.

Example Field Definitions

File Reference Functions

-- BUILD_SCOPED_FILE_URL: セッションスコープの一時URL
-- Document AIのPREDICT関数への入力として最も一般的
SELECT BUILD_SCOPED_FILE_URL(
  @invoice_stage,
  'invoices/2026/INV-001.pdf'
) AS scoped_url;

-- BUILD_STAGE_FILE_URL: ステージ参照URL
SELECT BUILD_STAGE_FILE_URL(
  @invoice_stage,
  'invoices/2026/INV-001.pdf'
) AS stage_url;

-- GET_PRESIGNED_URL: 署名付きURL（外部共有用）
SELECT GET_PRESIGNED_URL(
  @invoice_stage,
  'invoices/2026/INV-001.pdf',
  3600  -- 有効期限: 秒
) AS presigned_url;

Comparing File Reference Functions

Inference with the PREDICT Function

-- 単一ドキュメントの推論
SELECT invoice_extractor!PREDICT(
  BUILD_SCOPED_FILE_URL(@invoice_stage, 'invoices/INV-001.pdf')
) AS result;

-- 結果から個別フィールドを抽出
SELECT
  result:invoice_number::VARCHAR AS invoice_number,
  result:invoice_date::DATE AS invoice_date,
  result:total_amount::NUMBER(12,2) AS total_amount,
  result:vendor_name::VARCHAR AS vendor_name,
  result:__confidence::FLOAT AS overall_confidence
FROM (
  SELECT invoice_extractor!PREDICT(
    BUILD_SCOPED_FILE_URL(@invoice_stage, 'invoices/INV-001.pdf')
  ) AS result
);

Batch Processing Pipeline

-- Directory Tablesと組み合わせてバッチ推論
INSERT INTO extracted_invoices (file_path, invoice_number, invoice_date, total_amount, vendor_name, confidence, extracted_at)
SELECT
  d.relative_path,
  r.result:invoice_number::VARCHAR,
  r.result:invoice_date::DATE,
  r.result:total_amount::NUMBER(12,2),
  r.result:vendor_name::VARCHAR,
  r.result:__confidence::FLOAT,
  CURRENT_TIMESTAMP()
FROM DIRECTORY(@invoice_stage) d,
LATERAL (
  SELECT invoice_extractor!PREDICT(
    BUILD_SCOPED_FILE_URL(@invoice_stage, d.relative_path)
  ) AS result
) r
WHERE d.relative_path LIKE '%.pdf'
  AND d.relative_path NOT IN (
    SELECT file_path FROM extracted_invoices
  );

Extracting Line Items (Array Data)

-- FLATTEN で明細行を展開
SELECT
  result:invoice_number::VARCHAR AS invoice_number,
  item.value:description::VARCHAR AS item_description,
  item.value:quantity::INT AS quantity,
  item.value:unit_price::NUMBER(10,2) AS unit_price,
  item.value:amount::NUMBER(12,2) AS line_amount
FROM (
  SELECT invoice_extractor!PREDICT(
    BUILD_SCOPED_FILE_URL(@invoice_stage, 'invoices/INV-001.pdf')
  ) AS result
),
LATERAL FLATTEN(input => result:line_items) AS item;

Using Confidence Scores

Each extracted field comes with a confidence score from 0.0 to 1.0. Low-confidence fields need human review, so the production best practice is to set a threshold and route anything below it to a review queue.

-- 信頼度の低い抽出結果をレビューキューに投入
INSERT INTO review_queue (file_path, field_name, extracted_value, confidence)
SELECT
  relative_path,
  'total_amount',
  result:total_amount::VARCHAR,
  result:total_amount_confidence::FLOAT
FROM extracted_results
WHERE result:total_amount_confidence::FLOAT < 0.85;

Required Privileges

Best Practices

• Validate accuracy on sample documents before batch processing: run 5-10 test inferences to verify accuracy and cost, then move on to the production batch
• Set a confidence threshold and build a human-review workflow: for example, auto-approve anything at 0.85 or above and route the rest to a review queue
• Use Directory Tables and Streams for arrival-driven processing: build a pipeline that auto-detects new documents, runs PREDICT, and stores the results in your extracted-data table
• Keep the raw VARIANT result and structure it later: model updates can change the set of extracted fields, so retain the raw VARIANT in addition to the parsed columns

Check Your Understanding

Frequently Asked Questions