Database

db_connection

class aser.database.db_connection.BaseDBConnection(db_path, chunksize)[source]

Bases: object

Base KG connection for database

Create an connection to database

Parameters:

db_path (str) – database path
chunksize (int) – the chunksize to load/write database

close()[source]: Close the connection safely

create_table(table_name, columns, column_types)[source]

Create a table with given columns and types

Parameters:

table_name (str) – the table name to create
columns (List[str]) – the columns to create
column_types (List[str]) – the corresponding column types

get_columns(table_name, columns)[source]

Get column information from a table

Parameters:

table_name (str) – the table name to retrieve
columns (List[str]) – the columns to retrieve

Returns:

a list of retrieved rows

Return type:

List[Dict[str, object]]

get_rows_by_keys(table_name, bys, keys, columns, order_bys=None, reverse=False, top_n=None)[source]

Retrieve rows by specific keys in some order

Parameters:

table_name (str) – the table name to retrieve
bys (List[str]) – the given columns to match
keys (List[str]) – the given values to match
columns (List[str]) – the given columns to retrieve
order_bys (List[str]) – the columns whose value are used to sort rows
reverse (bool) – whether to sort in a reversed order
top_n (int) – how many rows to return, default None for all rows

Returns:

retrieved rows

Return type:

List[Dict[str, object]]

get_update_op(update_columns, operator)[source]

Get an update operator based on columns and a operator

Parameters:

update_columns (List[str]) – a list of columns to update
operator (str) – an operator that applies to the columns, including “+”, “-”, “*”, “/”, “=”

Returns:

an operator that suits the backend database

Return type:

object

insert_row(table_name, row)[source]

Insert a row into a table

Parameters:

table_name (str) – the table name to insert
row (Dict[str, object]) – the row to insert

insert_rows(table_name, rows)[source]

Insert several rows into a table

Parameters:

table_name (str) – the table name to insert
rows (List[Dict[str, object]]) – the rows to insert

select_row(table_name, _id, columns)[source]

Select a row from a table

Parameters:

table_name (str) – the table name to retrieve
_id (str) – the row id
columns (List[str]) – the columns to retrieve

Returns:

a retrieved row

Return type:

Dict[str, object]

select_rows(table_name, _ids, columns)[source]

Select rows from a table

Parameters:

table_name (str) – the table name to retrieve
_ids (List[str]) – the row ids
columns (List[str]) – the columns to retrieve

Returns:

retrieved rows

Return type:

List[Dict[str, object]]

update_row(table_name, row, update_op, update_columns)[source]

Update a row that exists in a table

Parameters:

table_name (str) – the table name to update
row (Dict[str, object]) – a new row
update_op (object) – an operator that returned by get_update_op
update_columns (List[str]) – the columns to update

update_rows(table_name, rows, update_ops, update_columns)[source]

Update rows that exist in a table

Parameters:

table_name (str) – the table name to update
rows (List[Dict[str, object]]) – new rows
update_ops (Union[List[object], object]) – operator(s) that returned by get_update_op
update_columns (List[str]) – the columns to update

class aser.database.db_connection.MongoDBConnection(db_path, chunksize)[source]

Bases: BaseDBConnection

KG connection for MongoDB

Create an connection to SQLite database

Parameters:

db_path (str) – database path, e.g., mongodb://localhost:27017/ASER
chunksize (int) – the chunksize to load/write database

close()[source]: Close the connection safely

create_table(table_name)[source]

Create a table without the necessary to provide column information

Parameters:: table_name (str) – the table name to create

get_columns(table_name, columns)[source]

Get column information from a table

Parameters:

table_name (str) – the table name to retrieve
columns (List[str]) – the columns to retrieve

Returns:

a list of retrieved rows

Return type:

List[Dict[str, object]]

get_rows_by_keys(table_name, bys, keys, columns, order_bys=None, reverse=False, top_n=None)[source]

Retrieve rows by specific keys in some order

Parameters:

table_name (str) – the table name to retrieve
bys (List[str]) – the given columns to match
keys (List[str]) – the given values to match
columns (List[str]) – the given columns to retrieve
order_bys (List[str]) – the columns whose value are used to sort rows
reverse (bool) – whether to sort in a reversed order
top_n (int) – how many rows to return, default None for all rows

Returns:

retrieved rows

Return type:

List[Dict[str, object]]

get_update_op(update_columns, operator)[source]

Get an update operator based on columns and a operator

Parameters:

update_columns (List[str]) – a list of columns to update
operator (str) – an operator that applies to the columns, including “+”, “-”, “*”, “/”, “=”

Returns:

an operator that suits the backend database

Return type:

Dict[str, Dict[str, float]]

insert_row(table_name, row)[source]

Insert a row into a table (suggestion: consider to use insert_rows if you want to insert multiple rows)

Parameters:

table_name (str) – the table name to insert
row (Dict[str, object]) – the row to insert

insert_rows(table_name, rows)[source]

Insert several rows into a table

Parameters:

table_name (str) – the table name to insert
rows (List[Dict[str, object]]) – the rows to insert

select_row(table_name, _id, columns)[source]

Select a row from a table (suggestion: consider to use select_rows if you want to retrieve multiple rows)

Parameters:

table_name (str) – the table name to retrieve
_id (str) – the row id
columns (List[str]) – the columns to retrieve

Returns:

a retrieved row

Return type:

Dict[str, object]

select_rows(table_name, _ids, columns)[source]

Select rows from a table

Parameters:

table_name (str) – the table name to retrieve
_ids (List[str]) – the row ids
columns (List[str]) – the columns to retrieve

Returns:

retrieved rows

Return type:

List[Dict[str, object]]

update_row(table_name, row, update_op, update_columns)[source]

Update a row that exists in a table (suggestion: consider to use update_rows if you want to update multiple rows)

Parameters:

table_name (str) – the table name to update
row (Dict[str, object]) – a new row
update_op (Dict[str, Dict[str, float]]) – an operator that returned by get_update_op
update_columns (List[str]) – the columns to update

update_rows(table_name, rows, update_ops, update_columns)[source]

Update rows that exist in a table

Parameters:

table_name (str) – the table name to update
rows (List[Dict[str, object]]) – new rows
update_ops (Union[List[Dict[str, Dict[str, float]]], Dict[str, Dict[str, float]]]) – operator(s) that returned by get_update_op
update_columns (List[str]) – the columns to update

class aser.database.db_connection.SqliteDBConnection(db_path, chunksize)[source]

Bases: BaseDBConnection

KG connection for SQLite database

Create an connection to SQLite database

Parameters:

db_path (str) – database path, e.g., /home/xliucr/ASER/KG.db
chunksize (int) – the chunksize to load/write database

close()[source]: Close the connection safely

create_table(table_name, columns, column_types)[source]

Create a table with given columns and types

Parameters:

table_name (str) – the table name to create
columns (List[str]) – the columns to create
column_types (List[str]) – the corresponding column types, please refer to https://www.sqlite.org/datatype3.html

get_columns(table_name, columns)[source]

Get column information from a table

Parameters:

table_name (str) – the table name to retrieve
columns (List[str]) – the columns to retrieve

Returns:

a list of retrieved rows

Return type:

List[Dict[str, object]]

get_rows_by_keys(table_name, bys, keys, columns, order_bys=None, reverse=False, top_n=None)[source]

Retrieve rows by specific keys in some order

Parameters:

table_name (str) – the table name to retrieve
bys (List[str]) – the given columns to match
keys (List[str]) – the given values to match
columns (List[str]) – the given columns to retrieve
order_bys (List[str]) – the columns whose value are used to sort rows
reverse (bool) – whether to sort in a reversed order
top_n (int) – how many rows to return, default None for all rows

Returns:

retrieved rows

Return type:

List[Dict[str, object]]

get_update_op(update_columns, operator)[source]

Get an update operator based on columns and a operator

Parameters:

update_columns (List[str]) – a list of columns to update
operator (str) – an operator that applies to the columns, including “+”, “-”, “*”, “/”, “=”

Returns:

an operator that suits the backend database

Return type:

str

insert_row(table_name, row)[source]

Insert a row into a table (suggestion: consider to use insert_rows if you want to insert multiple rows)

Parameters:

table_name (str) – the table name to insert
row (Dict[str, object]) – the row to insert

insert_rows(table_name, rows)[source]

Insert several rows into a table

Parameters:

table_name (str) – the table name to insert
rows (List[Dict[str, object]]) – the rows to insert

select_row(table_name, _id, columns)[source]

Select a row from a table (suggestion: consider to use select_rows if you want to retrieve multiple rows)

Parameters:

table_name (str) – the table name to retrieve
_id (str) – the row id
columns (List[str]) – the columns to retrieve

Returns:

a retrieved row

Return type:

Dict[str, object]

select_rows(table_name, _ids, columns)[source]

Select rows from a table

Parameters:

table_name (str) – the table name to retrieve
_ids (List[str]) – the row ids
columns (List[str]) – the columns to retrieve

Returns:

retrieved rows

Return type:

List[Dict[str, object]]

update_row(table_name, row, update_op, update_columns)[source]

Update a row that exists in a table (suggestion: consider to use update_rows if you want to update multiple rows)

Parameters:

table_name (str) – the table name to update
row (Dict[str, object]) – a new row
update_op (str) – an operator that returned by get_update_op
update_columns (List[str]) – the columns to update

update_rows(table_name, rows, update_ops, update_columns)[source]

Update rows that exist in a table

Parameters:

table_name (str) – the table name to update
rows (List[Dict[str, object]]) – new rows
update_ops (Union[List[str], str]) – operator(s) that returned by get_update_op
update_columns (List[str]) – the columns to update

kg_connection

class aser.database.kg_connection.ASERConceptConnection(db_path, db='sqlite', mode='cache', chunksize=32768)[source]

Bases: object

Concept connection for ASER (including concepts, concept_instance_pairs, and relations)

Parameters:

db_path (str) – database path
db (str (default = sqlite)) – the backend database, e.g., “sqlite” or “mongodb”
mode (str (default = "cache")) – the mode to use the connection. “insert”: this connection is only used to insert/update rows; “cache”: this connection caches some contents that have been retrieved; “memory”: this connection loads all contents in memory;
chunksize (int (default = 32768)) – the chunksize to load/write database

close()[source]: Close the ASERConceptConnection safely

get_concept_columns(columns)[source]

Get column information from concepts

Parameters:: columns (List[str]) – the columns to retrieve
Returns:: a list of retrieved rows
Return type:: List[Dict[str, object]]

get_concept_given_str(concept_str)[source]

Retrieve the exact matched concept given a string from ASER

Parameters:: concept_str (str) – a string representation of a concept
Returns:: the exact matched concept
Return type:: aser.concept.ASERConcept

get_concept_instance_pair_columns(columns)[source]

Get column information from concepts

Parameters:: columns (List[str]) – the columns to retrieve
Returns:: a list of retrieved rows
Return type:: List[Dict[str, object]]

get_concepts_by_keys(bys, keys, order_bys=None, reverse=False, top_n=None)[source]

Retrieve multiple partial matched concepts by keys and values from ASER

Parameters:

bys (List[str]) – the given columns to match
keys (List[str]) – the given values to match
order_bys (List[str]) – the columns whose value are used to sort rows
reverse (bool) – whether to sort in a reversed order
top_n (int) – how many concepts to return, default None for all concepts

Returns:

the partial matched concepts

Return type:

List[aser.concept.Concepts]

get_concepts_given_eventuality(eventuality)[source]

Retrieve concepts given an eventuality from ASER

Parameters:: eventuality (Union[aser.eventuality.Eventuality, Dict[str, object], str]) – eventuality that conceptualizes to the given concept
Returns:: the linked concepts
Return type:: List[aser.concept.ASERConcepts]

get_concepts_given_strs(concept_strs)[source]

Retrieve the exact matched concepts given strings from ASER

Parameters:: concept_str (List[str]) – string representations of concepts
Returns:: the exact matched concepts
Return type:: List[aser.concept.ASERConcept]

get_eventualities_given_concept(concept)[source]

Retrieve original eventualities given a concept from ASER

Parameters:: concept (Union[aser.concept.ASERConcpet, Dict[str, object], str]) – concept that corresponds to some eventualities
Returns:: the linked eventualities
Return type:: List[aser.eventuality.Eventuality]

get_exact_match_concept(concept)[source]

Retrieve a exact matched concept from ASER (suggestion: consider to use get_exact_match_concepts if you want to retrieve multiple concepts)

Parameters:: concept (Union[aser.concept.ASERConcept, Dict[str, object], str]) – a concept that contains the cid
Returns:: the exact matched concept
Return type:: aser.concept.ASERConcept

get_exact_match_concepts(concepts)[source]

Retrieve multiple exact matched concepts from ASER

Parameters:: concepts (Union[List[aser.concept.ASERConcept], List[Dict[str, object]], List[str]]) – concepts
Returns:: the exact matched concepts
Return type:: List[aser.concept.ASERConcept]

get_exact_match_relation(relation)[source]

Retrieve an exact matched relation from ASER (suggestion: consider to use get_exact_match_relations if you want to retrieve multiple relations)

Parameters:: relation (Union[aser.relation.Relation, Dict[str, object], str, Tuple[aser.concept.ASERConcept, aser.concept.ASERConcept], Tuple[str, str]]) – a relation that contains the rid or a concept pair that contains two cids
Returns:: the exact matched relation
Return type:: aser.relation.Relation

get_exact_match_relations(relations)[source]

Retrieve exact matched relations from ASER

Parameters:: relations (Union[List[aser.relation.Relation], List[Dict[str, object]], List[str], List[Tuple[aser.concept.ASERConcept, aser.concept.ASERConcept]], List[Tuple[str, str]]]) – a relations that contain the rids or concept pairs each of which contains two cids
Returns:: the exact matched relations
Return type:: List[aser.relation.Relation]

get_related_concepts(concept)[source]

Retrieve related (connected) concepts from ASER

Parameters:: eventuality – a concept that contains the eid
Returns:: the related concepts
Return type:: List[Tuple[aser.concept.ASERConcept, aser.relation.Relation]]

get_relation_columns(columns)[source]

Get column information from relations

Parameters:: columns (List[str]) – the columns to retrieve
Returns:: a list of retrieved rows
Return type:: List[Dict[str, object]]

get_relations_by_keys(bys, keys, order_bys=None, reverse=False, top_n=None)[source]

Retrieve multiple partial matched relations by keys and values from ASER

Parameters:

bys (List[str]) – the given columns to match
keys (List[str]) – the given values to match
order_bys (Union[List[str], None] (default = None)) – the columns whose value are used to sort rows
reverse (bool (default = False)) – whether to sort in a reversed order
top_n (Union[int, None] (default = None)) – how many relations to return, default None for all relations

Returns:

the partial matched relations

Return type:

List[aser.relation.Relation]

init()[source]: Initialize the ASERConceptConnection, including creating tables, loading cids, eids, rids, and building cache

insert_concept(concept)[source]

Insert/Update a concept into ASER (suggestion: consider to use insert_concepts if you want to insert multiple concepts)

Parameters:: concept (aser.concept.ASERConcept) – a concept to insert/update
Returns:: the inserted/updated concept
Return type:: aser.concept.ASERConcept

insert_concept_instance_pair(concept_instance_pair)[source]

Insert/Update a concept_instance_pair into ASER (suggestion: consider to use insert_concept_instance_pairs if you want to insert multiple pairs)

Parameters:: concept_instance_pair (Union[aser.concept.ASERConceptInstancePair, Tuple[aser.concept.ASERConcpet, aser.event.Eventuality, float]]) – a concept-instance pair to insert/update
Returns:: the inserted/updated concept-instance pair
Return type:: aser.concept.ASERConceptInstancePair

insert_concept_instance_pairs(concept_instance_pairs)[source]

Insert/Update concept_instance_pairs into ASER

Parameters:: concept_instance_pairs (Union[List[aser.concept.ASERConceptInstancePair], List[Tuple[aser.concept.ASERConcpet, aser.event.Eventuality, float]]]) – concept-instance pairs to insert/update
Returns:: the inserted/updated concept-instance pairs
Return type:: List[aser.concept.ASERConceptInstancePair]

insert_concepts(concepts)[source]

Insert/Update concepts into ASER

Parameters:: concepts (List[aser.concept.ASERConcept]) – concepts to insert/update
Returns:: the inserted/updated concepts
Return type:: List[aser.concept.ASERConcept]

insert_relation(relation)[source]

Insert/Update a relation into ASER (suggestion: consider to use insert_relations if you want to insert multiple relations)

Parameters:: relation (aser.relation.Relation) – a relation to insert/update
Returns:: the inserted/updated relation
Return type:: aser.relation.Relation

insert_relations(relations)[source]

Insert/Update relations into ASER

Parameters:: relations (List[aser.relation.Relation]) – relations to insert/update
Returns:: the inserted/updated relations
Return type:: List[aser.relation.Relation]

class aser.database.kg_connection.ASERKGConnection(db_path, db='sqlite', mode='cache', grain=None, chunksize=32768)[source]

Bases: object

KG connection for ASER (including eventualities and relations)

Parameters:

db_path (str) – database path
db (str (default = "sqlite")) – the backend database, e.g., “sqlite” or “mongodb”
mode (str (default = "cache")) – the mode to use the connection. “insert”: this connection is only used to insert/update rows; “cache”: this connection caches some contents that have been retrieved; “memory”: this connection loads all contents in memory;
grain (Union[str, None] (default = None)) – the grain to build cache “words”: cache is built on “verbs”, “skeleton_words”, and “words” “skeleton_words”: cache is built on “verbs”, and “skeleton_words” “verbs”: cache is built on “verbs” None: no cache
chunksize (int (default = 32768)) – the chunksize to load/write database

close()[source]: Close the ASERKGConnection safely

get_eventualities_by_keys(bys, keys, order_bys=None, reverse=False, top_n=None)[source]

Retrieve multiple partial matched eventualities by keys and values from ASER

Parameters:

bys (List[str]) – the given columns to match
keys (List[str]) – the given values to match
order_bys (List[str]) – the columns whose value are used to sort rows
reverse (bool) – whether to sort in a reversed order
top_n (int) – how many eventualities to return, default None for all eventualities

Returns:

the partial matched eventualities

Return type:

List[aser.eventuality.Eventuality]

get_eventuality_columns(columns)[source]

Get column information from eventualities

Parameters:: columns (List[str]) – the columns to retrieve
Returns:: a list of retrieved rows
Return type:: List[Dict[str, object]]

get_exact_match_eventualities(eventualities)[source]

Retrieve multiple exact matched eventualities from ASER

Parameters:: eventualities (Union[List[aser.eventuality.Eventuality], List[Dict[str, object]], List[str]]) – eventualities
Returns:: the exact matched eventualities
Return type:: List[aser.eventuality.Eventuality]

get_exact_match_eventuality(eventuality)[source]

Retrieve an exact matched eventuality from ASER (suggestion: consider to use get_exact_match_eventualities if you want to retrieve multiple eventualities)

Parameters:: eventuality (Union[aser.eventuality.Eventuality, Dict[str, object], str]) – an eventuality that contains the eid
Returns:: the exact matched eventuality
Return type:: aser.eventuality.Eventuality

get_exact_match_relation(relation)[source]

Retrieve an exact matched relation from ASER (suggestion: consider to use get_exact_match_relations if you want to retrieve multiple relations)

Parameters:: relation (Union[aser.relation.Relation, Dict[str, object], str, Tuple[aser.eventuality.Eventuality, aser.eventuality.Eventuality], Tuple[str, str]]) – a relation that contains the rid or an eventuality pair that contains two eids
Returns:: the exact matched relation
Return type:: aser.relation.Relation

get_exact_match_relations(relations)[source]

Retrieve exact matched relations from ASER

Parameters:: relations (Union[List[aser.relation.Relation], List[Dict[str, object]], List[str], List[Tuple[aser.eventuality.Eventuality, aser.eventuality.Eventuality]], List[Tuple[str, str]]]) – a relations that contain the rids or eventuality pairs each of which contains two eids
Returns:: the exact matched relations
Return type:: List[aser.relation.Relation]

get_partial_match_eventualities(eventuality, bys, top_n=None, threshold=0.8, sort=True)[source]

Retrieve multiple partial matched eventualities by a given eventuality and properties from ASER

Parameters:

eventuality (aser.eventuality.Eventuality) – the given eventuality to match
bys (List[str]) – the given properties to match
top_n (int) – how many rows to return, default None for all rows
threshold (float (default = 0.8)) – the minimum similarity
sort (bool (default = True)) – whether to sort

Returns:

the partial matched eventualities

Return type:

List[aser.eventuality.Eventuality]

get_related_eventualities(eventuality)[source]

Retrieve related (connected) eventualities from ASER

Parameters:: eventuality (Union[aser.eventuality.Eventuality, Dict[str, object], str]) – an eventuality that contains the eid
Returns:: the related eventualities
Return type:: List[Tuple[aser.eventuality.Eventuality, aser.relation.Relation]]

get_relation_columns(columns)[source]

Get column information from relations

Parameters:: columns (List[str]) – the columns to retrieve
Returns:: a list of retrieved rows
Return type:: List[Dict[str, object]]

get_relations_by_keys(bys, keys, order_bys=None, reverse=False, top_n=None)[source]

Retrieve multiple partial matched relations by keys and values from ASER

Parameters:

bys (List[str]) – the given columns to match
keys (List[str]) – the given values to match
order_bys (Union[List[str], None] (default = None)) – the columns whose value are used to sort rows
reverse (bool (default = False)) – whether to sort in a reversed order
top_n (Union[int, None] (default = None)) – how many relations to return, default None for all relations

Returns:

the partial matched relations

Return type:

List[aser.relation.Relation]

init()[source]: Initialize the ASERKGConnection, including creating tables, loading eids and rids, and building cache

insert_eventualities(eventualities)[source]

Insert/Update eventualities into ASER

Parameters:: eventualities (List[aser.eventuality.Eventuality]) – eventualities to insert/update
Returns:: the inserted/updated eventualities
Return type:: List[aser.eventuality.Eventuality]

insert_eventuality(eventuality)[source]

Insert/Update an eventuality into ASER (suggestion: consider to use insert_eventualities if you want to insert multiple eventualities)

Parameters:: eventuality (aser.eventuality.Eventuality) – an eventuality to insert/update
Returns:: the inserted/updated eventuality
Return type:: aser.eventuality.Eventuality

insert_relation(relation)[source]

Insert/Update a relation into ASER (suggestion: consider to use insert_relations if you want to insert multiple relations)

Parameters:: relation (aser.relation.Relation) – a relation to insert/update
Returns:: the inserted/updated relation
Return type:: aser.relation.Relation

insert_relations(relations)[source]

Insert/Update relations into ASER

Parameters:: relations (List[aser.relation.Relation]) – relations to insert/update
Returns:: the inserted/updated relations
Return type:: List[aser.relation.Relation]