How Does Apple Intelligence Protect Personal Data? On-Device AI, Private Cloud Compute, APIs, and JSON Privacy

The short version: Apple Intelligence does not protect personal data by “never using the cloud.” It keeps on the device what the on-device model can finish; what must go to the cloud goes through Private Cloud Compute — stateless, not retained, and not readable by Apple staff; only an explicit user handoff enters a third-party API such as ChatGPT. For developers, leaks usually start in the JSON fields you put on an App Action or tool call, not on the PCC node.

This is written on 10 September 2026, the day after Surprise and Shine. Yesterday’s keynote moved Siri AI into a user beta this month and set iOS 27 for 14 September; the foldable’s shipping name is iPhone Duo. The privacy architecture was not rewritten on stage. It is still the three layers locked at WWDC 2026 on 8 June: on-device Foundation Models, Private Cloud Compute (including the Google Cloud expansion), and optional third-party handoff. The 9 September piece Why iPhone 18 became an AI phone covers hardware and the agent. The 3 September Ternus and Apple’s AI strategy covers who pays and what the user sees. This article only answers how personal data moves, where JSON shows up, and which hop PCC does not cover.

Split the three data paths first

When people say “Apple says it never leaves the device,” they usually mash three paths into one. The official privacy page is blunt: an on-device model first decides whether the task can finish locally; if not, Apple Intelligence sends only data relevant to that request to Private Cloud Compute. The third path is outside PCC: the user turns on the ChatGPT extension, or your app POSTs JSON to your own backend.

Writing “PCC that runs on Google Cloud” as “the data was given to Google” is the most common 2026 misread. Whose rack holds the GPU is not the same question as whether the prompt entered the consumer Gemini API. The rest of this piece keeps them apart.

On-device AI: if it can stay, it stays

Since 2024 Apple has treated on-device processing as the first brick of the privacy story: data that exists only on user devices is disaggregated, with no single pot to steal. Preview summaries of Mail, Messages, and notifications are the official example of on-device generation. “Which film did my brother tell me to watch last week?” is answered from a rebuilt Spotlight semantic index on the phone, not by pasting the whole thread into a cloud box.

WWDC 2026 thickened the on-device Foundation Models again: image input, tighter instruction following, better tool calling. The developer table is blunt:

One gap PCC cannot fill: Siri voice with adjustable pace and expressiveness, and the higher-precision system dictation, need at least 12GB of unified memory. Short of memory, the features do not appear. The cloud does not “helpfully run them for you.” In yesterday’s AI-phone piece that was a hardware gate. In privacy language: when the advanced on-device tier is missing, the system does not pretend the data never left by bouncing the same feature through the cloud.

Siri AI’s App Toolbox and Spotlight index are described as working entirely on device. Users can turn on Apple Intelligence Report (Settings → Privacy & Security), export a file, and see which requests left the device, which hit PCC, and — if the ChatGPT extension is on — which were handed to OpenAI. That is the cheapest way to check the marketing copy.

System Orchestrator: which fields leave

Which path, and which data, is not the model grabbing whatever it likes. Public write-ups after WWDC name this role the System Orchestrator: on the device it picks the model tier and which apps may touch which data. Ars Technica’s example is concrete — a recipe arrived in Messages; the device can answer “how do I cook this” with the recipe body and leave out the sender, the timestamp, and why it was sent.

For JSON developers, the orchestrator is OS-level field minimization. It is the same discipline as additionalProperties: false and requiring messageId instead of the full email. The difference: you cannot edit Apple’s layer. The field list on an App Action you register is yours.

So “personal context” sounds scarier than it is — Mail, Messages, Photos, Calendar are indexed — but the index stays on device. What leaves, on the official line, is only what is needed to fulfill that request. If the orchestrator picks poorly, or your Schema is wide, the leak stops being “Apple used the cloud” and becomes “you put the whole contact graph in parameters.”

Private Cloud Compute: a stateless cloud, not a normal LLM API

PCC is the 2024 Apple Security Research design for foundation models larger than the phone. It is not “POST the prompt to someone’s Chat Completions.” The device builds a request (prompt plus parameters) and encrypts it to the public keys of attested PCC nodes; load balancers and ordinary sidecars do not hold the decrypt keys. Apple listed five requirements. The 2026 expansion to third-party halls left the sentences intact:

1. Stateless computation — personal data is used only to fulfill the request; nothing is retained after the response, including logs and debug copies.
2. Enforceable guarantees — not a policy PDF; cryptography and isolation have to make the behavior hard to violate.
3. No privileged runtime access — staff, operators, and cloud admins should not see plaintext.
4. Non-targetability — an attacker should not easily aim at “the node that has this one user.”
5. Verifiable transparency — binaries published, researchers can inspect; the bounty covers research-mode nodes.

The privacy page also says what Apple still collects: approximate request and response size, which features ran, how long it took. That metadata does not include content and is not linked to the Apple Account or other service data. The content itself is “not stored and not made accessible to Apple,” then returned securely to the device.

For developers, Foundation Models folds PCC into a one-line swap: default LanguageModelSession() is on-device; pass PrivateCloudComputeLanguageModel() to go to the cloud. @Generable structured output and tool-call signatures stay the same. No API key, no per-token bill to the developer; users share a daily quota on the iCloud account, with higher caps on most iCloud+ plans. WWDC 2026 Session 319 also states the eligibility gate: apps under 2 million downloads can use PCC at no token cost, with a managed entitlement. Hitting the cap throws quotaLimitReached — check quotaUsage.isLimitReached and show persistent, upgradeable UI, not a dead alert.

Calling PCC “a free GPT API” fails in two places. The quota is per user, not per app — other apps drink from the same well. And the privacy promise is bound to the PCC path; the moment you POST the same JSON to your own server, the story ends.

2026: PCC on Google Cloud — did the promise change?

On 8 June, Apple Security Research published “Expanding Private Cloud Compute”: PCC left Apple’s own data centers for the first time, working with Google and NVIDIA to run the heaviest Apple Intelligence workloads on Google Cloud — including agentic tool use and complex reasoning. This is an expansion, not a migration. Apple silicon PCC nodes keep running; the summer was a preview ramp toward the full protection set.

The five requirements did not change. The implementation did: NVIDIA Confidential Computing on GPUs, Intel TDX on CPUs, Google Titan as a root of trust. Apple also added layers a stock confidential-computing deploy does not have:

• The trusted computing base runs from firmware through host and guest OS to application code, not just a confidential VM wrapper.
• A cryptographically verifiable, append-only ledger of Google Cloud hardware in the PCC fleet; software attestation for sensitive components is rooted in at least two independent vendors.
• Inbound parsing in a dedicated namespaced process; inference software recycled on a short TTL; attested keys held in a separate confidential VM isolated from external input.
• Devices trust only Apple-signed PCC software; binaries stay public; the bounty covers research nodes on Google Cloud.

Apple working with Google, and using technology behind Gemini to build the next Foundation Models, is not the same as “the user’s prompt hit the consumer Gemini API.” The official sentence is: cloud capability runs inside PCC; the heaviest requests use the PCC fleet that now includes Google Cloud. Press figures for a custom Gemini (about 1.2 trillion MoE parameters, about $1 billion a year) are not a price list — mark them as reported numbers in product docs. Even if the weights share Gemini technology, the public commitment is still inference inside PCC’s stateless boundary, not “POST the email body to ai.google.dev.”

Some legal-page paragraphs still say “Apple silicon servers.” The engineering blog already covers third-party halls. Use the 8 June Expanding PCC note and the WWDC line as of September 2026; do not treat “only our racks” as current fact.

The hop that is not PCC: ChatGPT, the web, and your API

PCC’s promise stops here. One hop further is someone else’s privacy policy.

• ChatGPT extension — the user turns it on in Settings before Siri, Writing Tools, or Visual Intelligence can hand off a session. Apple Intelligence Report lists those requests separately. A signed-in OpenAI account follows OpenAI’s policy. This is not PCC, and it is not “Apple cannot see it, therefore Google cannot either.”
• World knowledge from the web — Siri AI can fetch live information. The search itself leaves the device; whether the query carries a slab of personal context is the orchestrator’s job, not a slogan’s.
• Your own backend — after your app receives an App Action or a Foundation Models tool result, if you JSON.stringify it to your API, your logs, or an analytics pile, PCC does not cover that hop. Apple’s guarantee ends at the PCC node. Past your process, it is your contract.

Copy that says “nothing Apple Intelligence touches ever leaves the device” repeats the iPhone 16 promise accident the moment ChatGPT is on, PCC is used, or you uploaded the JSON yourself. The accurate sentence is layered: on-device does not leave; PCC is stateless and inspectable; third parties need a user tap and their own policy; your servers follow your privacy notice.

JSON privacy: App Actions, structured output, field minimization

Siri AI reaches third-party apps through App Actions (the next baton after App Intents). Foundation Models tool calling and @Generable structured output still look like “name + description + JSON Schema.” On-device and PCC share the same signatures — swapping the model is one line; swapping the field list swaps the outbound data.

For “politely reply to the message already open,” the orchestrator fills parameters from your Schema. The left-hand contract works. The right-hand one ships the body and the sender:

{
  "name": "replyToOpenMessage",
  "description": "Reply to the message the user already has open; do not pick another thread",
  "parameters": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
      "messageId": {
        "type": "string",
        "description": "Stable mailbox ID, not the subject or the full body"
      },
      "tone": {
        "type": "string",
        "enum": ["brief", "polite", "formal"]
      }
    },
    "required": ["messageId"]
  }
}

{
  "name": "replyToOpenMessage",
  "parameters": {
    "type": "object",
    "properties": {
      "emailBody": { "type": "string" },
      "senderAddress": { "type": "string" },
      "thread": { "type": "array" },
      "contactGraph": { "type": "array" }
    }
  }
}

The first lets the model carry an ID and a tone; the body stays in the on-device Mail process for the app to read. The second asks the orchestrator to serialize personal data into a payload that may hit PCC. Even if PCC does not retain it, you widened the attack surface of that request; log the same JSON and you now have a plaintext copy.

Drop the Schema into the JSON toolbox for a local check. It is the same discipline as tools.parameters on Gemini or OpenAI: stable IDs, enums, integer ranges, no extra properties. For the layered version see Tool Calling and JSON Schema validation, What structured output is, and What MCP is. Apple changed the host, not the contract language. Loose fields do not make the small on-device model “understand you”; they make it fill the wrong keys — a wrong action, or a key/value that should never have left.

What PCC cannot save

• A wide Schema — PCC promises “unread, unstored,” not “you did not put extra fields in the request.”
• ChatGPT handoff and your API — OpenAI’s policy or yours.
• Metadata — size, feature name, duration are still collected. Content is not metadata, but “the user is in Writing Tools” still is a signal.
• Quota and eligibility — daily caps, iCloud tier, the 2 million-download gate, the entitlement. Hitting the limit is not “silently switch to a less private cloud”; the feature fails or falls back on-device. Do not quietly POST to your own key when fallback fails.
• Regional switches — EU iPhone / iPad have no Siri AI at the start; China is still in regulatory review. A complete privacy architecture does not create a data path for a feature that is off — and a China-sold phone is not “PCC out of the box.”
• The 12GB on-device advanced tier — dictation and timbre do not appear because PCC got bigger. Using the cloud to impersonate on-device is a product lie and a privacy lie.

What to do now

1. Write the privacy notice as three paths — on-device / PCC / third-party. Do not cover them with one “we never upload.”
2. Minimize App Action fields — IDs, enums, ranges; additionalProperties: false. Run the same fixture in the browser through JSON validation and JSON Diff.
3. Default to on-device; treat PCC as an upgrade — see if 4K context is enough. Switch to PrivateCloudComputeLanguageModel only when you must, and handle isLimitReached.
4. Do not log raw tool arguments — keep a request id and an error code. Bodies, addresses, and contact graphs do not belong on your servers.
5. Open Apple Intelligence Report on a device and check — requests you thought were local will say whether they left.
6. Treat EU iPhone and China as if this cloud path does not exist — the core flow must still work with taps. There is no public timeline.

FAQ

Wrap-up

Apple Intelligence protects personal data by layering paths, not by promising “no cloud”: what the device can finish stays; what must leave uses inspectable, stateless PCC; third parties apply only after a user tap and under their policy. The 2026 move onto Google Cloud did not rewrite the five requirements — it changed the racks and the confidential-computing stack. For readers of a JSON toolbox, the homework is narrower: write App Action fields as a minimal contract, and do not let the Schema talk more than the orchestrator.

The 9 September keynote filled in dates and product names. It did not replace the privacy skeleton published in June. Agent design, tool calling, and structured output are already on this site; this piece only adds how the data walks and which JSON hop leaves the phone. The local check fits in the browser. You do not have to upload the fixture to anyone’s cloud first.