AI in Childcare: Why Human Review, Privacy and Australian Data Hosting Matter

ECEC providers hold information about children, families, staff, attendance, health, behaviour, payments and regulatory matters. A poorly governed AI tool can expose that information, invent an answer, obscure the source of a recommendation or influence a high-impact decision without adequate human oversight.

The sector does not need AI that acts confident. It needs AI that is accountable.

Community expectations are already clear

The Office of the Australian Information Commissioner’s 2026 privacy research found strong community concern about AI and personal information.

The research reported that:

• 69 per cent of Australians recognised AI as a privacy risk
• trust in AI companies was very low
• 81 per cent prioritised a right to human review
• 80 per cent prioritised limits on third-party retention of personal information
• 79 per cent wanted to be told when AI was being used.

These expectations are especially relevant in services responsible for young children.

Providers should assume that families and staff will increasingly ask:

• Is AI being used?
• What information does it receive?
• Where does the information go?
• Does it make decisions?
• Can a person review or correct the outcome?
• Is the activity recorded?
• How long is the data retained?

A provider that cannot answer those questions does not yet have adequate AI governance.

The first rule: AI should not silently decide high-impact outcomes

AI should not independently determine matters such as:

• a child’s eligibility or entitlement
• a family’s subsidy or payment outcome
• whether an incident is reportable
• whether a staff member is suitable to work
• whether a child is at risk
• whether a regulatory obligation has been met
• disciplinary action
• enrolment exclusion
• a final legal or compliance interpretation.

AI may help organise information, identify a possible issue or prepare a draft. A suitably authorised person must make and record the final decision.

Human review should be genuine. It is not enough to place an “approve” button after an automated conclusion if the reviewer cannot see the source, reasoning limits and uncertainty.

A responsible-AI framework for ECEC

1. Start with a permitted-use register

List every AI use case and classify it by risk.

For each use, record:

• business purpose
• information involved
• user roles
• AI provider or model
• where processing occurs
• retention settings
• required human review
• prohibited actions
• testing performed
• owner
• approval date
• next review date.

Unapproved AI tools should not be used with service information.

2. Minimise personal information

Only provide the minimum information needed for the approved purpose.

Before sending content to an AI system, consider removing or replacing:

• names
• dates of birth
• addresses
• contact details
• child identifiers
• health information
• incident details
• family circumstances
• payment information
• staff identifiers.

Data minimisation is not only a privacy principle. It reduces the consequence of error, misuse or breach.

3. Keep sources visible

An AI-generated compliance response should point back to the official material used.

The reviewer should be able to distinguish between:

• a direct requirement
• official guidance
• organisational policy
• an AI-generated interpretation
• a recommended action.

Without source traceability, plausible wording can be mistaken for law.

4. Design meaningful human review

The reviewer needs:

• the original source
• the relevant extracted text
• the proposed interpretation
• uncertainty or conflicts
• affected services and roles
• recommended actions
• authority to reject or change the draft.

The system should record who reviewed the output, what changed and when it was approved.

5. Control access at the action level

Access should not be all or nothing.

A centre director may need to review an operational action but should not necessarily access another service’s family or employee information. A finance user may need billing functions but not child-safety investigations.

AI tools should operate within the same role, tenant and service boundaries as the user. They should not become a shortcut around permissions.

6. Test before release

AI features should be tested for:

• accuracy
• unsupported claims
• source mismatch
• privacy leakage
• cross-service data exposure
• biased or inappropriate output
• unsafe action recommendations
• failure when information is incomplete
• ability to refuse prohibited tasks.

A feature should not move into production simply because a demonstration looks impressive.

7. Monitor after release

Models, prompts, source material and user behaviour change.

Providers and software vendors should monitor:

• corrections
• rejected outputs
• user complaints
• privacy incidents
• recurring hallucinations
• source failures
• inappropriate access attempts
• high-risk actions
• model or vendor changes.

Responsible AI is an operating discipline, not a one-off policy.

Australian data hosting matters, but it is not the whole answer

Keeping data in Australia can support sovereignty, procurement, risk management and customer confidence. Oiva stores data in AWS Sydney.

However, the physical region is only one control. Providers should also ask:

• Does a subcontractor process data overseas?
• Is customer data used to train a model?
• How long are prompts and outputs retained?
• Who can access support logs?
• Are backups kept in the same approved region?
• Can data be deleted?
• Are encryption and access controls independently tested?
• What happens when the vendor changes an AI provider?

“Hosted in Australia” should be supported by architecture, contracts, monitoring and evidence.

What providers should ask software vendors

1. Where is our data stored and processed?
2. Which AI providers or subprocessors are used?
3. Is our information used to train shared models?
4. Can AI make decisions or only prepare drafts?
5. What human approval is required?
6. Can every output be traced to its source?
7. How is access restricted by provider, service and role?
8. What testing is completed before release?
9. How are incidents and model changes communicated?
10. Can we export our records and evidence?

Vague answers should be treated as a warning.

How Oiva approaches AI

Oiva is built around three principles:

• Source-grounded: actions trace back to the official update that triggered them.
• Human-reviewed: AI drafts, but authorised people approve.
• Privacy-aware: personal information is minimised before AI processing and Oiva data is stored in AWS Sydney.

This is not designed to make humans a ceremonial final step. It is designed to help providers act faster without giving away responsibility.

Final message for the sector

AI can improve ECEC administration and compliance, but speed is not the only measure of value.

The standard should be trustworthy assistance: limited data, visible sources, controlled access, tested outputs, human decisions and an evidence trail.

In a regulated sector responsible for child safety, responsible AI is not a premium feature. It is the minimum acceptable design.

Frequently asked questions

Official sources

• OAIC, Australian Community Attitudes to Privacy Survey 2026 (oaic.gov.au)
• OAIC, Guidance on privacy and the use of commercially available AI products (oaic.gov.au)
• OAIC, Australian Privacy Principles (oaic.gov.au)
• OAIC, APP 3 Collection of solicited personal information (oaic.gov.au)
• OAIC, APP 11 Security of personal information (oaic.gov.au)

This article provides general information for Australian ECEC services and approved providers. It does not constitute legal, regulatory or professional advice. Requirements vary by jurisdiction, service type and individual circumstances. Providers should confirm their obligations with ACECQA, their state or territory regulatory authority and qualified professional advisers.