oaebu_workflows.oaebu_partners

Module Contents

Classes

DataPartnerFiles

Generates the file names of the data partner. These files may or may not exist depending on whether the

OaebuPartner

Class for storing information about data sources we are using to produce oaebu intermediate tables for.

DataPartner

Represents a data partner.

Functions

partner_from_str(partner[, metadata_partner])

Get the partner from a string.

Attributes

OAEBU_METADATA_PARTNERS

OAEBU_DATA_PARTNERS

class oaebu_workflows.oaebu_partners.DataPartnerFiles(*, partner_name)[source]

Generates the file names of the data partner. These files may or may not exist depending on whether the partner uses them.

Parameters:

partner_name (str) – The name of the partner. The type_id from the OaebuPartner class.

Generates the following file names: - book_product_metrics_schema: The schema for the metrics field in the book_products table. - book_product_metadata_schema: The schema for the metadata field in the book_products table. - book_metrics_schema: The schema for the book metrics export table. - book_metrics_author_schema: The schema for the author metrics table. - book_metrics_country_schema: The schema for the country metrics table. - book_metrics_subject_schema: The schema for the subject metrics export tables. - book_product_body_sql: The SQL template for the body of the book_product table. - book_product_functions_sql: The SQL for the functions in the book_product table. - book_metrics_sql: The SQL for the book metrics export table. - book_metrics_country_body_sql: The SQL template for metrics extion of the country export table. - book_metrics_country_join_sql: The SQL for the join of the country export table. - book_metrics_country_null_sql: The SQL for the null value assertion in the country export table. - book_metrics_country_struct_sql: The SQL for the struct in the country export table. - month_metrics_sum_sql: The SQL for the sum of the data partner’s month metrics. - month_null_sql: The SQL for the null value assertion in the data partner’s month metrics.

class oaebu_workflows.oaebu_partners.OaebuPartner[source]

Class for storing information about data sources we are using to produce oaebu intermediate tables for.

Parameters:

  • type_id – The dataset type id. Should be the same as its dictionary key

  • bq_dataset_id – The BigQuery dataset ID Bigquery Dataset ID.

  • bq_table_name – The BigQuery table name Bigquery Table name

  • isbn_field_name – Name of the field containing the ISBN.

  • title_field_name – Name of the field containing the Title.

  • sharded – whether the table is sharded or not.

  • schema_path – The path of the partner’s schema folder.

type_id: str[source]
bq_dataset_id: str[source]
bq_table_name: str[source]
isbn_field_name: str[source]
title_field_name: str[source]
sharded: bool[source]
schema_path: str[source]
__str__()[source]

Return str(self).

class oaebu_workflows.oaebu_partners.DataPartner(*, type_id, bq_dataset_id, bq_table_name, isbn_field_name, title_field_name, sharded, schema_path, schema_directory, sql_directory, book_product_functions, export_author, export_book_metrics, export_country, export_subject, has_metdata=True)[source]

Bases: OaebuPartner

Represents a data partner.

Parameters:

  • OaebuPartner – The parent class

  • type_id (str) –

  • bq_dataset_id (str) –

  • bq_table_name (str) –

  • isbn_field_name (str) –

  • title_field_name (str) –

  • sharded (bool) –

  • schema_path (str) –

  • schema_directory (str) –

  • sql_directory (str) –

  • book_product_functions (str) –

  • export_author (bool) –

  • export_book_metrics (bool) –

  • export_country (bool) –

  • export_subject (bool) –

  • has_metdata (bool) –

Initialises the class. Also uses the DataPartnerFiles class to set up the file names.

Parameters:

  • schema_directory (str) – The directory containing the schema files.

  • sql_directory (str) – The directory containing the SQL files/templates.

  • book_product_functions (str) – Whether the partner uses functions in the book product table creation.

  • export_author (bool) – Indicates if the partner will use the author export table.

  • export_book_metrics (bool) – Indicates if the partner will use the book metrics export table.

  • export_country (bool) – Indicates if the partner will use the country export table.

  • export_subject (bool) – Indicates if the partner will use the subject export tables (bic, bisac, thema).

  • has_metdata (bool) – Whether the partner has book metadata records

  • type_id (str) –

  • bq_dataset_id (str) –

  • bq_table_name (str) –

  • isbn_field_name (str) –

  • title_field_name (str) –

  • sharded (bool) –

  • schema_path (str) –

oaebu_workflows.oaebu_partners.OAEBU_METADATA_PARTNERS[source]
oaebu_workflows.oaebu_partners.OAEBU_DATA_PARTNERS[source]
oaebu_workflows.oaebu_partners.partner_from_str(partner, metadata_partner=False)[source]

Get the partner from a string.

Parameters:

  • partner (Union[str, OaebuPartner]) – The partner name.

  • metadata_partner (bool) – If True, use the metadata partner dictionary; otherwise, use the data partners dictionary

Raises:

Exception – Raised if the partner name is not found

Returns:

The OaebuPartner

Return type:

OaebuPartner