UNIHAN - cihai.data.unihan

Bootstrapping

Fetch + extract + transform + load UNIHAN dataset to Cihai.

cihai.data.unihan.bootstrap.bootstrap_unihan(engine, metadata, options=None)

function

UNIHAN bootstrap script (download from web, import to database).

Parameters:

• engine (Engine)

• metadata (MetaData)

• options (dict[str, object] | Options | None)

Return type:

None

cihai.data.unihan.bootstrap.is_bootstrapped(metadata)

function

Return True if cihai is correctly bootstrapped.

Parameters:

metadata (MetaData)

Return type:

bool

cihai.data.unihan.bootstrap.create_unihan_table(columns, metadata)

function

Create table and return sqlalchemy.sql.schema.Table.

Parameters:

• columns (list) – columns for table, e.g. ['kDefinition', 'kCantonese']

• metadata (sqlalchemy.schema.MetaData) – Instance of sqlalchemy metadata

Returns:

Newly created table with columns and index.

Return type:

sqlalchemy.schema.Table

class cihai.data.unihan.dataset.Unihan

class

Bases: Dataset, SQLAlchemyMixin

UNIHAN Dataset for cihai.

char: str

attribute

kDefinition: str

attribute

kTraditionhalVariant: str

attribute

kSimplifiedVariant: str

attribute

tagged_vars: Callable[[str], ParsedVars]

attribute

untagged_vars: Callable[[str], UntaggedVars]

attribute

bootstrap(options=None)

method

Fetch, extract, import UNIHAN to DB, and initialize DB mapping.

Parameters:

options (dict[str, object] | None)

Return type:

None

lookup_char(char)

method

Return character information from datasets.

Parameters:

char (str) – character / string to lookup

Returns:

list of matches

Return type:

sqlalchemy.orm.query.Query

reverse_char(hints)

method

Return QuerySet of objects from SQLAlchemy of results.

Parameters:

hints (list of str) – strings to lookup

Returns:

reverse matches

Return type:

sqlalchemy.orm.query.Query

with_fields(fields)

method

Return list of characters with information for certain fields.

Parameters:

*fields (list of str) – fields for which information should be available

Returns:

list of matches

Return type:

sqlalchemy.orm.query.Query

property is_bootstrapped: bool

property

Return True if UNIHAN and database is set up.

Returns:

True if Unihan application fixture data installed.

Return type:

bool

add_plugin(_cls, namespace, bootstrap=True)

method

Add plugin for Cihai dataset.

Parameters:

• _cls (DSP | str)

• namespace (str)

• bootstrap (bool)

Return type:

None

sql: Database

attribute

engine: Engine

attribute

sqlalchemy.engine.Engine instance.

metadata: MetaData

attribute

sqlalchemy.schema.MetaData instance.

session: Session

attribute

sqlalchemy.orm.session.Session instance.

base: AutomapBase

attribute

sqlalchemy.ext.automap.AutomapBase instance.

Constants for UNIHAN cihai dataset.

cihai.data.unihan.constants.UNIHAN_FIELDS: list[str] = ['kAccountingNumeric', 'kCangjie', 'kCantonese', 'kCheungBauer', 'kCihaiT', 'kCompatibilityVariant', 'kDefinition', 'kFenn', 'kFourCornerCode', 'kGradeLevel', 'kHDZRadBreak', 'kHKGlyph', 'kHangul', 'kHanyuPinlu', 'kHanyuPinyin', 'kJapaneseKun', 'kJapaneseOn', 'kKorean', 'kMandarin', 'kOtherNumeric', 'kPhonetic', 'kPrimaryNumeric', 'kRSAdobe_Japan1_6', 'kRSUnicode', 'kSemanticVariant', 'kSimplifiedVariant', 'kSpecializedSemanticVariant', 'kTang', 'kTotalStrokes', 'kTraditionalVariant', 'kVietnamese', 'kXHC1983', 'kZVariant']

data

Mapping of field names from unihan-etl (UNIHAN database)

cihai.data.unihan.constants.UNIHAN_ETL_DEFAULT_OPTIONS = {'expand': False, 'fields': ['kAccountingNumeric', 'kCangjie', 'kCantonese', 'kCheungBauer', 'kCihaiT', 'kCompatibilityVariant', 'kDefinition', 'kFenn', 'kFourCornerCode', 'kGradeLevel', 'kHDZRadBreak', 'kHKGlyph', 'kHangul', 'kHanyuPinlu', 'kHanyuPinyin', 'kJapaneseKun', 'kJapaneseOn', 'kKorean', 'kMandarin', 'kOtherNumeric', 'kPhonetic', 'kPrimaryNumeric', 'kRSAdobe_Japan1_6', 'kRSUnicode', 'kSemanticVariant', 'kSimplifiedVariant', 'kSpecializedSemanticVariant', 'kTang', 'kTotalStrokes', 'kTraditionalVariant', 'kVietnamese', 'kXHC1983', 'kZVariant'], 'format': 'python', 'input_files': ['Unihan_DictionaryLikeData.txt', 'Unihan_IRGSources.txt', 'Unihan_NumericValues.txt', 'Unihan_RadicalStrokeCounts.txt', 'Unihan_Readings.txt', 'Unihan_Variants.txt']}

data

Default settings passed to unihan-etl

Variants plugin

class cihai.data.unihan.dataset.UnihanVariants

class

Bases: DatasetPlugin, SQLAlchemyMixin

Support for CJK Variant lookups through UNIHAN dataset.

bootstrap()

method

Map custom lookup for UNIHAN variants to Unihan SQLAlchemy table.

Return type:

None

sql: Database

attribute

engine: Engine

attribute

sqlalchemy.engine.Engine instance.

metadata: MetaData

attribute

sqlalchemy.schema.MetaData instance.

session: Session

attribute

sqlalchemy.orm.session.Session instance.

base: AutomapBase

attribute

sqlalchemy.ext.automap.AutomapBase instance.