# oarepo\_oaipmh\_harvester.writers.oai\_service

## `oarepo_oaipmh_harvester.writers.oai_service`

`oai-service` writer - synchronizes local records with harvested ones.

Datastream configuration: `oai-service{model:"<model>", ...}`, see OAIServiceWriter for the options.

The writer expects entries `{"record": <data for the service or None>, "oai_record": <OAI record>}`. How to produce them, what is stored on the harvest record and how failures are reported is described in [Readers, transformers and writers](<https://oarepo.github.io/oarepo-oaipmh-harvester/readers-transformers-writers.html>).

For each harvested entry the writer

1. loads the harvest record (OAIHarvestedRecord) of the OAI identifier; the local pid is taken from it, or - for a record harvested for the first time - from the transformed data via the pid provider's `get_pid_value_from_record` (records deposited locally with the same pid are taken over);
2. classifies the local record - LocalRecord.resolve returns a LocalState (absent, live, draft only, tombstoned, hard deleted) together with the record and a local draft, if any;
3. picks the Operation that brings the local record to the harvested state - LocalRecord.operation. The harvest always wins: locally deleted records are restored (rdm tombstones, `restore_record`) or revived (hard deleted records are reverted to their last revision, keeping the pid), locally restored records are deleted again, local edit drafts are overwritten or removed;
4. skips the entry if the OAI datestamp and deleted flag are unchanged, the last write had no errors and the local record is already in the harvested state;
5. executes the operation inside a unit of work (multi-step operations such as restore + update roll back together) and publishes drafts where needed;
6. stores the harvest record with the datestamp, transformed data, local pid and errors.

Draft services do not allow updating published records, so updates go through edit - update draft - publish unless `direct_update` is set. Records of rdm services are deleted with `delete_record` (tombstone with a note about the deletion at the source), other services with `delete`.

Records whose data have no `files` / `media_files` section are written with `{"enabled": False}` there (for the sections the model supports) - harvested metadata do not bring files.

Entries marked by a transformer with `stream_entry.filtered = True` (e.g. by skip-if-owned) are not written - the local record is left as it is. The harvest record is stored with `has_warnings` and a `skipped` entry (the reason from `stream_entry.entry["skip_reason"]`) in `errors`. Such a harvest record is never up to date, so the entry is evaluated again by every harvest.

Written records are added to the `community` writer option (slug or id) and to the communities listed by the transformer in `parent.communities.ids` of the transformed data. They are added directly (rdm `bulk_add`), without a community inclusion request; communities the record is already in are skipped, and the record is never removed from a community.

### `SKIP_REASON = 'skip_reason'`

Key of `stream_entry.entry` with the reason why a filtered entry was not written.

### `LocalState`

Bases: `StrEnum`

State of the local record that corresponds to a harvested OAI record.

#### `ABSENT = 'absent'`

No local record nor draft.

#### `LIVE = 'live'`

Published, not deleted record (or a plain record for models without drafts).

#### `DRAFT_ONLY = 'draft_only'`

No published record (or a hard deleted one), but a draft exists.

#### `TOMBSTONED = 'tombstoned'`

RDM record soft-deleted via `delete_record` (deletion status `D` or `X`).

#### `HARD_DELETED = 'hard_deleted'`

Record deleted via `service.delete` - `model.is_deleted` set, pid status `D`.

### `Operation`

Bases: `StrEnum`

Operation that brings the local record to the state reported by the OAI-PMH source.

#### `UPDATE = 'update'`

Update a live record (for draft services: edit, update draft, publish - unless `direct_update`).

#### `RESTORE = 'restore'`

Restore a tombstoned rdm record (unmarking it from purge if needed), then update it.

#### `REVIVE = 'revive'`

Revert a hard deleted record to its last revision, re-register its pid, then update it.

#### `DELETE = 'delete'`

Delete via `service.delete` - for services without rdm deletion workflow.

#### `DELETE_RECORD = 'delete_record'`

Tombstone via rdm `delete_record`.

### `LocalRecord`

Local record state together with the resolved record and draft (if any).

#### `is_rdm: bool = False`

Service supports rdm deletion workflow (tombstones via `delete_record`).

#### `deleted_pid: PersistentIdentifier | None = None`

Pid of a hard deleted record (status `D`).

#### `resolve(service: RecordService, pid_value: str | None, *, with_drafts: bool) -> LocalRecord`

Find the local record (and draft) for the pid value and classify its state.

#### `operation(oai_deleted: bool) -> Operation`

Determine the operation that brings the local record to the harvested state.

The harvest always wins - whatever happened to the record locally, the state reported by the OAI-PMH source becomes the local state.

### `OAIServiceWriter`

Bases: `BaseWriter`

Writer for OAI-PMH harvested records.

This writer uses a service to write records harvested via OAI-PMH. It also stores an OAIRecord model instance to keep track of the harvesting status.

#### `service: RecordService`

Get the service for the specified model.

#### `__init__(model: str, *args: Any, identity: Identity | None = None, update_all: bool = False, harvester_id: str | None = None, pid_field: str = 'id', publish: bool = True, direct_update: bool = False, community: str | None = None, **kwargs: Any)`

Initialize the OAI service writer.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `model` | `str` | Name of the target model (`current_runtime.models`). | *required* |
| `identity` | `Identity \| None` | Identity used for the service calls, defaults to the system identity. | `None` |
| `update_all` | `bool` | Write every record, even if it is up to date. | `False` |
| `harvester_id` | `str \| None` | Harvester the harvest records belong to (set by the harvest task). | `None` |
| `pid_field` | `str` | Path of the pid value in the service result. | `'id'` |
| `publish` | `bool` | Publish created / updated drafts. Required (`ValueError` otherwise) for rdm models; `False` only for models without drafts. | `True` |
| `direct_update` | `bool` | Update published records via `service.update` instead of edit - update draft - publish. Only for services that allow updating published records (drafts services do not by default). | `False` |
| `community` | `str \| None` | Slug (or id) of a community every written record is added to. Only for rdm models. | `None` |

#### `write(stream_entry: StreamEntry, *args: Any, **kwargs: Any) -> StreamEntry`

Write the input entry using a given service.

#### `write_many(stream_entries: list[StreamEntry], *args: Any, **kwargs: Any) -> list[StreamEntry]`

Write the input entries using a given service.

### `get_pid_value_from_record(service: RecordService, record_data: dict) -> str | None`

Read the pid value from (not yet written) record data via the pid provider, if it supports it.
