MaxCompute:MaxFrame-specific APIs

API name: new_session. For more information about the source code, see new_session.

new_session(
  session_id: str = None,
  default: bool = True,
  new: bool = True,
  odps_entry: Optional[ODPS] = None
)

Description: starts a MaxFrame task session.

Input parameters

Return value

The session object.

Sample code

from maxframe import new_session
from odps import ODPS

# Use the MaxFrame account to initialize MaxCompute.
o = ODPS(
    # Set the environment variable ALIBABA_CLOUD_ACCESS_KEY_ID to the AccessKey ID of your Alibaba Cloud account. 
    # Set the environment variable ALIBABA_CLOUD_ACCESS_KEY_SECRET to the AccessKey secret of your Alibaba Cloud account. 
    # We recommend that you do not directly use the actual AccessKey ID and AccessKey secret. 
    os.getenv('ALIBABA_CLOUD_ACCESS_KEY_ID'),
    os.getenv('ALIBABA_CLOUD_ACCESS_KEY_SECRET'),
    project='your-default-project',
    endpoint='your-end-point',
)

# Initialize the MaxFrame session.
session = new_session(odps_entry=o)

API name: read_odps_table. For more information about the source code, see read_odps_table.

read_odps_table(
  table_name: Union[str, Table],
  partitions: Union[None, str, List[str]] = None,
  columns: Optional[List[str]] = None,
  index_col: Union[None, str, List[str]] = None,
  odps_entry: ODPS = None,
  string_as_binary: bool = None,
  append_partitions: bool = False
)

Description: reads data from a MaxCompute table and builds a DataFrame object. You can specify specific columns as indexes. If you do not specify indexes, a RangeIndex is generated.

Input parameters

Return value

The DataFrame object.

Sample code

import maxframe.dataframe as md

df = md.read_odps_table('BIGDATA_PUBLIC_DATASET.data_science.maxframe_ml_100k_users', index_col='user_id', columns=['age', 'sex'])
print(df.execute().fetch())

# Return value		
user_id	age   sex
1	24	M
2	53	F
3	23	M
4	24	M
5	33	F
...	...	...
939	26	F
940	32	M
941	20	M
942	48	F
943	22	M

API name: read_odps_query. For more information about the source code, see read_odps_query.

read_odps_query(
  query: str,
  odps_entry: ODPS = None,
  index_col: Union[None, str, List[str]] = None,
  string_as_binary: bool = None
  )

Description: reads data from a MaxCompute SQL query and creates a DataFrame object. You can specify specific columns as indexes. If you do not specify indexes, a RangeIndex is generated.

Input parameters

Return value

The DataFrame object.

Sample code

import maxframe.dataframe as md

df = md.read_odps_query('select user_id, age, sex FROM `BIGDATA_PUBLIC_DATASET.data_science.maxframe_ml_100k_users`')

API name: to_odps_table. For more information about the source code, see to_odps_table.

to_odps_table(
  table: Union[Table, str],
  partition: Optional[str] = None,
  partition_col: Union[None, str, List[str]] = None,
  overwrite: bool = False,
  unknown_as_string: Optional[bool] = None,
  index: bool = True,
  index_label: Union[None, str, List[str]] = None,
  lifecycle: Optional[int] = None
)

Description: writes a DataFrame object to a MaxCompute table. If the table does not exist in MaxCompute, MaxFrame automatically creates the table.

Input parameters

Return value

The DataFrame object.

Example

import maxframe.dataframe as md

df = md.read_odps_query('select user_id, age, sex FROM `BIGDATA_PUBLIC_DATASET.data_science.maxframe_ml_100k_users`', index_col='user_id'))
ouput_df = df.to_odps_table('output_table', lifecycle = 7)

API name: to_odps_model.

to_odps_model(
    model_name: str,
    model_version: str = None,
    schema: str = None,
    project: str = None,
    description: Optional[str] = None,
    version_description: Optional[str] = None,
    create_model: bool = True,
    set_default_version: bool = False
)

Description: Saves an XGBoost model that is trained by a MaxFrame job as a MaxCompute model object.

Input parameters

Return value

A Scalar object. You can call .execute() to trigger the model saving operation.

Example

# Train an XGBoost model
from maxframe.learn.contrib.xgboost import XGBClassifier
X_df = md.DataFrame(X, columns=cols)
clf = XGBClassifier(n_estimators=10)
clf.fit(X_df, y)

# Save the model to MaxCompute
clf.to_odps_model(
    model_name="my_model",
    # If you specify a project and schema, the format of model_name is as follows:
    # model_name="project.schema.my_model"
    model_version="version1"
).execute()

API name: execute. For more information about the source code, see execute.

execute(
  session: SessionType = None
)

Description: calls the execute method to start a data processing task.

Input parameters

Return value

N/A.

Example

import maxframe.dataframe as md

df = md.read_odps_query('select user_id, age, sex FROM BIGDATA_PUBLIC_DATASET.data_science.maxframe_ml_100k_users', index_col='user_id'))
df.execute()

API name: fetch. For more information about the source code, see fetch.

fetch(
  session: SessionType = None
)

Description: returns the result data to the on-premises environment.

Input parameters

Return value

The DataFrame or Series of Pandas.

Sample code

import maxframe.dataframe as md

df = md.read_odps_query('select user_id, age, sex FROM `BIGDATA_PUBLIC_DATASET.data_science.maxframe_ml_100k_users`', index_col='user_id')
res = df.execute().fetch()
print(res)

# Obtain the returned result.      
user_id   age  sex
1         24   M
2         53   F
3         23   M
4         24   M
5         33   F
...      ...  ..
939       26   F
940       32   M
941       20   M
942       48   F
943       22   M

[943 rows x 2 columns]