Pipelines API Reference

Reference documentation for the Kubeflow Pipelines API Version 2
Kubeflow Pipelines API

Kubeflow Pipelines API

Version: 2.0.0

This file contains REST API specification for Kubeflow Pipelines. The file is autogenerated from the swagger definition.

Default request content-types: application/json
Default response content-types: application/json
Schemes: http, https

Summary

Tag: AuthService

OperationDescription
GET /apis/v2beta1/auth

Tag: ExperimentService

OperationDescription
GET /apis/v2beta1/experiments

Finds all experiments. Supports pagination, and sorting on certain fields.

POST /apis/v2beta1/experiments

Creates a new experiment.

GET /apis/v2beta1/experiments/{experiment_id}

Finds a specific experiment by ID.

DELETE /apis/v2beta1/experiments/{experiment_id}

Deletes an experiment without deleting the experiment's runs and recurring runs. To avoid unexpected behaviors, delete an experiment's runs and recurring runs before deleting the experiment.

POST /apis/v2beta1/experiments/{experiment_id}:archive

Archives an experiment and the experiment's runs and recurring runs.

POST /apis/v2beta1/experiments/{experiment_id}:unarchive

Restores an archived experiment. The experiment's archived runs and recurring runs will stay archived.

Tag: HealthzService

OperationDescription
GET /apis/v2beta1/healthz

Get healthz data.

Tag: PipelineService

OperationDescription
GET /apis/v2beta1/pipelines

Finds all pipelines within a namespace.

POST /apis/v2beta1/pipelines

Creates a pipeline.

POST /apis/v2beta1/pipelines/create

Creates a new pipeline and a new pipeline version in a single transaction.

GET /apis/v2beta1/pipelines/names/{name}

Finds a specific pipeline by name and namespace.

GET /apis/v2beta1/pipelines/{pipeline_id}

Finds a specific pipeline by ID.

DELETE /apis/v2beta1/pipelines/{pipeline_id}

Deletes an empty pipeline by ID. Returns error if the pipeline has pipeline versions.

GET /apis/v2beta1/pipelines/{pipeline_id}/versions

Lists all pipeline versions of a given pipeline ID.

POST /apis/v2beta1/pipelines/{pipeline_id}/versions

Adds a pipeline version to the specified pipeline ID.

GET /apis/v2beta1/pipelines/{pipeline_id}/versions/{pipeline_version_id}

Gets a pipeline version by pipeline version ID and pipeline ID.

DELETE /apis/v2beta1/pipelines/{pipeline_id}/versions/{pipeline_version_id}

Deletes a specific pipeline version by pipeline version ID and pipeline ID.

Tag: PipelineUploadService

OperationDescription
POST /apis/v2beta1/pipelines/upload
POST /apis/v2beta1/pipelines/upload_version

Tag: RecurringRunService

OperationDescription
GET /apis/v2beta1/recurringruns

Finds all recurring runs given experiment and namespace. If experiment ID is not specified, find all recurring runs across all experiments.

POST /apis/v2beta1/recurringruns

Creates a new recurring run in an experiment, given the experiment ID.

GET /apis/v2beta1/recurringruns/{recurring_run_id}

Finds a specific recurring run by ID.

DELETE /apis/v2beta1/recurringruns/{recurring_run_id}

Deletes a recurring run.

POST /apis/v2beta1/recurringruns/{recurring_run_id}:disable

Stops a recurring run and all its associated runs. The recurring run is not deleted.

POST /apis/v2beta1/recurringruns/{recurring_run_id}:enable

Restarts a recurring run that was previously stopped. All runs associated with the recurring run will continue.

Tag: ReportService

OperationDescription
POST /apis/v2beta1/scheduledworkflows
POST /apis/v2beta1/workflows

Tag: RunService

OperationDescription
GET /apis/v2beta1/runs

Finds all runs in an experiment given by experiment ID. If experiment id is not specified, finds all runs across all experiments.

POST /apis/v2beta1/runs

Creates a new run in an experiment specified by experiment ID. If experiment ID is not specified, the run is created in the default experiment.

GET /apis/v2beta1/runs/{run_id}

Finds a specific run by ID.

DELETE /apis/v2beta1/runs/{run_id}

Deletes a run in an experiment given by run ID and experiment ID.

GET /apis/v2beta1/runs/{run_id}/nodes/{node_id}/artifacts/{artifact_name}:read

Finds artifact data in a run.

POST /apis/v2beta1/runs/{run_id}:archive

Archives a run in an experiment given by run ID and experiment ID.

POST /apis/v2beta1/runs/{run_id}:retry

Re-initiates a failed or terminated run.

POST /apis/v2beta1/runs/{run_id}:terminate

Terminates an active run.

POST /apis/v2beta1/runs/{run_id}:unarchive

Restores an archived run in an experiment given by run ID and experiment ID.

Tag: VisualizationService

OperationDescription
POST /apis/v2beta1/visualizations/{namespace}

Security

Bearer

Type: apiKey
Name:

authorization

In:

header

Paths

GET /apis/v2beta1/auth

Tags: AuthService
namespacequerystring
resourcesquerystring , x ∈ { UNASSIGNED_RESOURCES (default) , VIEWERS }
verbquerystring , x ∈ { UNASSIGNED_VERB (default) , CREATE , GET , DELETE }

Uses default content-types: application/json

200 OK

A successful response.

default
Finds all experiments. Supports pagination, and sorting on certain fields.

GET /apis/v2beta1/experiments

Tags: ExperimentService
page_token

A page token to request the next page of results. The token is acquried from the nextPageToken field of the response from the previous ListExperiments call or can be omitted when fetching the first page.

querystring
page_size

The number of experiments to be listed per page. If there are more experiments than this number, the response message will contain a nextPageToken field you can use to fetch the next page.

queryinteger (int32)
sort_by

Can be format of "field_name", "field_name asc" or "field_name desc" Ascending by default.

querystring
filter

A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto).

querystring
namespace

Which namespace to filter the experiments on.

querystring

Uses default content-types: application/json

200 OK

A successful response.

Creates a new experiment.

POST /apis/v2beta1/experiments

Tags: ExperimentService

Uses default content-types: application/json

The experiment to be created.

Uses default content-types: application/json

200 OK

A successful response.

Deletes an experiment without deleting the experiment's runs and recurring runs. To avoid unexpected behaviors, delete an experiment's runs and recurring runs before deleting the experiment.

DELETE /apis/v2beta1/experiments/{experiment_id}

Tags: ExperimentService
experiment_id

The ID of the experiment to be deleted.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

Finds a specific experiment by ID.

GET /apis/v2beta1/experiments/{experiment_id}

Tags: ExperimentService
experiment_id

The ID of the experiment to be retrieved.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

Archives an experiment and the experiment's runs and recurring runs.

POST /apis/v2beta1/experiments/{experiment_id}:archive

Tags: ExperimentService
experiment_id

The ID of the experiment to be archived.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

Restores an archived experiment. The experiment's archived runs and recurring runs will stay archived.

POST /apis/v2beta1/experiments/{experiment_id}:unarchive

Tags: ExperimentService
experiment_id

The ID of the experiment to be restored.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

Get healthz data.

GET /apis/v2beta1/healthz

Tags: HealthzService

Uses default content-types: application/json

200 OK

A successful response.

default
Finds all pipelines within a namespace.

GET /apis/v2beta1/pipelines

Tags: PipelineService
namespace

Optional input. Namespace for the pipelines.

querystring
page_token

A page token to request the results page.

querystring
page_size

The number of pipelines to be listed per page. If there are more pipelines than this number, the response message will contain a valid value in the nextPageToken field.

queryinteger (int32)
sort_by

Sorting order in form of "field_name", "field_name asc" or "field_name desc". Ascending by default.

querystring
filter

A url-encoded, JSON-serialized filter protocol buffer (see filter.proto).

querystring

Uses default content-types: application/json

200 OK

A successful response.

default
Creates a pipeline.

POST /apis/v2beta1/pipelines

Tags: PipelineService

Uses default content-types: application/json

Required input. Pipeline that needs to be created.

Uses default content-types: application/json

200 OK

A successful response.

default
Creates a new pipeline and a new pipeline version in a single transaction.

POST /apis/v2beta1/pipelines/create

Tags: PipelineService
Finds a specific pipeline by name and namespace.

GET /apis/v2beta1/pipelines/names/{name}

Tags: PipelineService
name

Required input. Name of the pipeline to be retrieved.

pathstring
namespace

Optional input. Namespace of the pipeline. It could be empty if default namespaces needs to be used or if multi-user support is turned off.

querystring

Uses default content-types: application/json

200 OK

A successful response.

default

POST /apis/v2beta1/pipelines/upload

Tags: PipelineUploadService

multipart/form-data

uploadfile

The pipeline to upload. Maximum size of 32MB is supported.

formDatafile
namequerystring
descriptionquerystring
namespacequerystring

application/json

200 OK
default

POST /apis/v2beta1/pipelines/upload_version

Tags: PipelineUploadService

multipart/form-data

uploadfile

The pipeline to upload. Maximum size of 32MB is supported.

formDatafile
namequerystring
pipelineidquerystring
descriptionquerystring

application/json

200 OK
default
Deletes an empty pipeline by ID. Returns error if the pipeline has pipeline versions.

DELETE /apis/v2beta1/pipelines/{pipeline_id}

Tags: PipelineService
pipeline_id

Required input. ID of the pipeline to be deleted.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

default
Finds a specific pipeline by ID.

GET /apis/v2beta1/pipelines/{pipeline_id}

Tags: PipelineService
pipeline_id

Required input. The ID of the pipeline to be retrieved.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

default
Lists all pipeline versions of a given pipeline ID.

GET /apis/v2beta1/pipelines/{pipeline_id}/versions

Tags: PipelineService
pipeline_id

Required input. ID of the parent pipeline.

pathstring
page_token

A page token to request the results page.

querystring
page_size

The number of pipeline versions to be listed per page. If there are more pipeline versions than this number, the response message will contain a valid value in the nextPageToken field.

queryinteger (int32)
sort_by

Sorting order in form of "field_name", "field_name asc" or "field_name desc". Ascending by default.

querystring
filter

A url-encoded, JSON-serialized filter protocol buffer (see filter.proto).

querystring

Uses default content-types: application/json

200 OK

A successful response.

default
Adds a pipeline version to the specified pipeline ID.

POST /apis/v2beta1/pipelines/{pipeline_id}/versions

Tags: PipelineService

Uses default content-types: application/json

Required input. Pipeline version ID to be created.

pipeline_id

Required input. ID of the parent pipeline.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

default
Deletes a specific pipeline version by pipeline version ID and pipeline ID.

DELETE /apis/v2beta1/pipelines/{pipeline_id}/versions/{pipeline_version_id}

Tags: PipelineService
pipeline_id

Required input. ID of the parent pipeline.

pathstring
pipeline_version_id

Required input. The ID of the pipeline version to be deleted.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

default
Gets a pipeline version by pipeline version ID and pipeline ID.

GET /apis/v2beta1/pipelines/{pipeline_id}/versions/{pipeline_version_id}

Tags: PipelineService
pipeline_id

Required input. ID of the parent pipeline.

pathstring
pipeline_version_id

Required input. ID of the pipeline version to be retrieved.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

default
Finds all recurring runs given experiment and namespace. If experiment ID is not specified, find all recurring runs across all experiments.

GET /apis/v2beta1/recurringruns

Tags: RecurringRunService
page_token

A page token to request the next page of results. The token is acquired from the nextPageToken field of the response from the previous ListRecurringRuns call or can be omitted when fetching the first page.

querystring
page_size

The number of recurring runs to be listed per page. If there are more recurring runs than this number, the response message will contain a nextPageToken field you can use to fetch the next page.

queryinteger (int32)
sort_by

Can be formatted as "field_name", "field_name asc" or "field_name desc". Ascending by default.

querystring
namespace

Optional input. The namespace the recurring runs belong to.

querystring
filter

A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto).

querystring
experiment_id

The ID of the experiment to be retrieved. If empty, list recurring runs across all experiments.

querystring

Uses default content-types: application/json

200 OK

A successful response.

Creates a new recurring run in an experiment, given the experiment ID.

POST /apis/v2beta1/recurringruns

Tags: RecurringRunService

Uses default content-types: application/json

The recurring run to be created.

Uses default content-types: application/json

200 OK

A successful response.

Deletes a recurring run.

DELETE /apis/v2beta1/recurringruns/{recurring_run_id}

Tags: RecurringRunService
recurring_run_id

The ID of the recurring run to be deleted.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

Finds a specific recurring run by ID.

GET /apis/v2beta1/recurringruns/{recurring_run_id}

Tags: RecurringRunService
recurring_run_id

The ID of the recurring run to be retrieved.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

Stops a recurring run and all its associated runs. The recurring run is not deleted.

POST /apis/v2beta1/recurringruns/{recurring_run_id}:disable

Tags: RecurringRunService
recurring_run_id

The ID of the recurring runs to be disabled.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

Restarts a recurring run that was previously stopped. All runs associated with the recurring run will continue.

POST /apis/v2beta1/recurringruns/{recurring_run_id}:enable

Tags: RecurringRunService
recurring_run_id

The ID of the recurring runs to be enabled.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

Finds all runs in an experiment given by experiment ID. If experiment id is not specified, finds all runs across all experiments.

GET /apis/v2beta1/runs

Tags: RunService
namespace

Optional input field. Filters based on the namespace.

querystring
experiment_id

The ID of the parent experiment. If empty, response includes runs across all experiments.

querystring
page_token

A page token to request the next page of results. The token is acquired from the nextPageToken field of the response from the previous ListRuns call or can be omitted when fetching the first page.

querystring
page_size

The number of runs to be listed per page. If there are more runs than this number, the response message will contain a nextPageToken field you can use to fetch the next page.

queryinteger (int32)
sort_by

Can be format of "field_name", "field_name asc" or "field_name desc" (Example, "name asc" or "id desc"). Ascending by default.

querystring
filter

A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto).

querystring

Uses default content-types: application/json

200 OK

A successful response.

default
Creates a new run in an experiment specified by experiment ID. If experiment ID is not specified, the run is created in the default experiment.

POST /apis/v2beta1/runs

Tags: RunService

Uses default content-types: application/json

Run to be created.

Uses default content-types: application/json

200 OK

A successful response.

default
Deletes a run in an experiment given by run ID and experiment ID.

DELETE /apis/v2beta1/runs/{run_id}

Tags: RunService
run_id

The ID of the run to be deleted.

pathstring
experiment_id

The ID of the parent experiment.

querystring

Uses default content-types: application/json

200 OK

A successful response.

default
Finds a specific run by ID.

GET /apis/v2beta1/runs/{run_id}

Tags: RunService
run_id

The ID of the run to be retrieved.

pathstring
experiment_id

The ID of the parent experiment.

querystring

Uses default content-types: application/json

200 OK

A successful response.

default
Finds artifact data in a run.

GET /apis/v2beta1/runs/{run_id}/nodes/{node_id}/artifacts/{artifact_name}:read

Tags: RunService
run_id

ID of the run.

pathstring
node_id

ID of the running node.

pathstring
artifact_name

Name of the artifact.

pathstring
experiment_id

The ID of the parent experiment.

querystring

Uses default content-types: application/json

200 OK

A successful response.

default
Archives a run in an experiment given by run ID and experiment ID.

POST /apis/v2beta1/runs/{run_id}:archive

Tags: RunService
run_id

The ID of the run to be archived.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

default
Re-initiates a failed or terminated run.

POST /apis/v2beta1/runs/{run_id}:retry

Tags: RunService
run_id

The ID of the run to be retried.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

default
Terminates an active run.

POST /apis/v2beta1/runs/{run_id}:terminate

Tags: RunService
run_id

The ID of the run to be terminated.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

default
Restores an archived run in an experiment given by run ID and experiment ID.

POST /apis/v2beta1/runs/{run_id}:unarchive

Tags: RunService
run_id

The ID of the run to be restored.

pathstring

Uses default content-types: application/json

200 OK

A successful response.

default

POST /apis/v2beta1/scheduledworkflows

Tags: ReportService

Uses default content-types: application/json

ScheduledWorkflow a ScheduledWorkflow resource marshalled into a json string.

Uses default content-types: application/json

200 OK

A successful response.

POST /apis/v2beta1/visualizations/{namespace}

Tags: VisualizationService

Uses default content-types: application/json

namespacepathstring

Uses default content-types: application/json

200 OK

A successful response.

default

POST /apis/v2beta1/workflows

Tags: ReportService

Uses default content-types: application/json

Workflow is a workflow custom resource marshalled into a json string.

Uses default content-types: application/json

200 OK

A successful response.

Schema definitions

AuthorizeRequestResources: string , x ∈ { UNASSIGNED_RESOURCES (default) , VIEWERS }

Type of resources in pipelines system.

AuthorizeRequestVerb: string , x ∈ { UNASSIGNED_VERB (default) , CREATE , GET , DELETE }

Type of verbs that act on the resources.

googlerpcStatus: object

The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC. Each Status message contains three pieces of data: error code, error message, and error details.

You can find out more about this error model and how to work with it in the API Design Guide.

code: integer (int32)

The status code, which should be an enum value of [google.rpc.Code][google.rpc.Code].

message: string

A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the [google.rpc.Status.details][google.rpc.Status.details] field, or localized by the client.

details: object[]

A list of messages that carry the error details. There is a common set of message types for APIs to use.

PipelineTaskDetailChildTask: object

A dependent task that requires this one to succeed. Represented by either task_id or pod_name.

task_id: string

System-generated ID of a task.

pod_name: string

Name of the corresponding pod assigned by the orchestration engine. Also known as node_id.

PredicateIntValues: object

List of integers.

values: integer[]
integer (int32)

PredicateLongValues: object

List of long integers.

values: string[]
string (int64)

PredicateStringValues: object

List of strings.

values: string[]
string

protobufAny: object

Any contains an arbitrary serialized protocol buffer message along with a URL that describes the type of the serialized message.

Protobuf library provides support to pack/unpack Any values in the form of utility functions or additional generated methods of the Any type.

Example 1: Pack and unpack a message in C++.

Foo foo = ...;
Any any;
any.PackFrom(foo);
...
if (any.UnpackTo(&foo)) {
...
}

Example 2: Pack and unpack a message in Java.

Foo foo = ...;
Any any = Any.pack(foo);
...
if (any.is(Foo.class)) {
foo = any.unpack(Foo.class);
}

Example 3: Pack and unpack a message in Python.

foo = Foo(...)
any = Any()
any.Pack(foo)
...
if any.Is(Foo.DESCRIPTOR):
any.Unpack(foo)
...

Example 4: Pack and unpack a message in Go

 foo := &pb.Foo{...}
any, err := anypb.New(foo)
if err != nil {
...
}
...
foo := &pb.Foo{}
if err := any.UnmarshalTo(foo); err != nil {
...
}

The pack methods provided by protobuf library will by default use 'type.googleapis.com/full.type.name' as the type URL and the unpack methods only use the fully qualified type name after the last '/' in the type URL, for example "foo.bar.com/x/y.z" will yield type name "y.z".

JSON

The JSON representation of an Any value uses the regular representation of the deserialized, embedded message, with an additional field @type which contains the type URL. Example:

package google.profile;
message Person {
string first_name = 1;
string last_name = 2;
}
{
"@type": "type.googleapis.com/google.profile.Person",
"firstName": <string>,
"lastName": <string>
}

If the embedded message type is well-known and has a custom JSON representation, that representation will be embedded adding a field value which holds the custom JSON in addition to the @type field. Example (for message [google.protobuf.Duration][]):

{
"@type": "type.googleapis.com/google.protobuf.Duration",
"value": "1.212s"
}
type_url: string

A URL/resource name that uniquely identifies the type of the serialized protocol buffer message. This string must contain at least one "/" character. The last segment of the URL's path must represent the fully qualified name of the type (as in path/google.protobuf.Duration). The name should be in a canonical form (e.g., leading "." is not accepted).

In practice, teams usually precompile into the binary all types that they expect it to use in the context of Any. However, for URLs which use the scheme http, https, or no scheme, one can optionally set up a type server that maps type URLs to message definitions as follows:

  • If no scheme is provided, https is assumed.
  • An HTTP GET on the URL must yield a [google.protobuf.Type][] value in binary format, or produce an error.
  • Applications are allowed to cache lookup results based on the URL, or have them precompiled into a binary to avoid any lookup. Therefore, binary compatibility needs to be preserved on changes to types. (Use versioned type names to manage breaking changes.)

Note: this functionality is not currently available in the official protobuf release, and it is not used for type URLs beginning with type.googleapis.com.

Schemes other than http, https (or the empty scheme) might be used with implementation specific semantics.

value: string (byte)

Must be a valid serialized protocol buffer of the above specified type.

protobufNullValue: string , x ∈ { NULL_VALUE (default) }

NullValue is a singleton enumeration to represent the null value for the Value type union.

The JSON representation for NullValue is JSON null.

  • NULL_VALUE: Null value.

RecurringRunMode: string , x ∈ { MODE_UNSPECIFIED (default) , ENABLE , DISABLE }

Required input. User setting to enable or disable the recurring run. Only used for creation of recurring runs. Later updates use enable/disable API.

  • DISABLE: The recurring run won't schedule any run if disabled.

v2beta1ArtifactList: object

A list of artifact metadata.

artifact_ids: string[]

A list of artifact metadata ids.

string (int64)

v2beta1CreatePipelineAndVersionRequest: object

pipeline: v2beta1Pipeline

Required input. Pipeline (parent) to be created.

pipeline_version: v2beta1PipelineVersion

Required input. Pipeline version (child) to be created. Pipeline spec will be downloaded from pipeline_version.package_url.

v2beta1CronSchedule: object

CronSchedule allow scheduling the recurring run with unix-like cron.

start_time: string (date-time)

The start time of the cron job.

end_time: string (date-time)

The end time of the cron job.

cron: string

v2beta1Experiment: object

experiment_id: string

Output. Unique experiment ID. Generated by API server.

display_name: string

Required input field. Unique experiment name provided by user.

description: string

Optional input field. Describes the purpose of the experiment.

created_at: string (date-time)

Output. The time that the experiment was created.

namespace: string

Optional input field. Specify the namespace this experiment belongs to.

storage_state: v2beta1ExperimentStorageState

Output. Specifies whether this experiment is in archived or available state.

v2beta1ExperimentStorageState: string , x ∈ { STORAGE_STATE_UNSPECIFIED (default) , AVAILABLE , ARCHIVED }

Describes whether an entity is available or archived.

  • STORAGE_STATE_UNSPECIFIED: Default state. This state in not used
  • AVAILABLE: Entity is available.
  • ARCHIVED: Entity is archived.

v2beta1Filter: object

Filter is used to filter resources returned from a ListXXX request.

Example filters:

1) Filter runs with status = 'Running' filter { predicate { key: "status" operation: EQUALS string_value: "Running" } }

2) Filter runs that succeeded since Dec 1, 2018 filter { predicate { key: "status" operation: EQUALS string_value: "Succeeded" } predicate { key: "created_at" operation: GREATER_THAN timestamp_value { seconds: 1543651200 } } }

3) Filter runs with one of labels 'label_1' or 'label_2'

filter { predicate { key: "label" operation: IN string_values { value: 'label_1' value: 'label_2' } } }

predicates: object[]

All predicates are AND-ed when this filter is applied.

v2beta1GetHealthzResponse: object

multi_user: boolean (boolean)

Returns if KFP in multi-user mode

v2beta1ListExperimentsResponse: object

experiments: object[]

A list of experiments returned.

total_size: integer (int32)

The number of experiments for the given query.

next_page_token: string

The token to list the next page of experiments.

v2beta1ListPipelinesResponse: object

pipelines: object[]

Returned pipelines.

total_size: integer (int32)

The total number of pipelines for the given query.

next_page_token: string

The token to list the next page of pipelines. This token can be used on the next ListPipelinesRequest.

v2beta1ListPipelineVersionsResponse: object

pipeline_versions: object[]

Returned pipeline versions.

next_page_token: string

The token to list the next page of pipeline versions.

total_size: integer (int32)

The total number of pipeline versions for the given query.

v2beta1ListRecurringRunsResponse: object

recurringRuns: object[]

A list of recurring runs returned.

total_size: integer (int32)

The total number of recurring runs for the given query.

next_page_token: string

The token to list the next page of recurring runs.

v2beta1ListRunsResponse: object

runs: object[]

List of retrieved runs.

total_size: integer (int32)

The total number of runs for the given query.

next_page_token: string

The token to list the next page of runs.

v2beta1PeriodicSchedule: object

PeriodicSchedule allow scheduling the recurring run periodically with certain interval.

start_time: string (date-time)

The start time of the periodic recurring run.

end_time: string (date-time)

The end time of the periodic recurring run.

interval_second: string (int64)

The time interval between the starting time of consecutive recurring runs.

v2beta1Pipeline: object

pipeline_id: string

Output. Unique pipeline ID. Generated by API server.

display_name: string

Required input field. Pipeline name provided by user.

description: string

Optional input field. A short description of the pipeline.

created_at: string (date-time)

Output. Creation time of the pipeline.

namespace: string

Input. A namespace this pipeline belongs to. Causes error if user is not authorized to access the specified namespace. If not specified in CreatePipeline, default namespace is used.

error: googlerpcStatus

In case any error happens retrieving a pipeline field, only pipeline ID, and the error message is returned. Client has the flexibility of choosing how to handle the error. This is especially useful during listing call.

v2beta1PipelineTaskDetail: object

Runtime information of a task execution.

run_id: string

ID of the parent run.

task_id: string

System-generated ID of a task.

display_name: string

User specified name of a task that is defined in [Pipeline.spec][].

create_time: string (date-time)

Creation time of a task.

start_time: string (date-time)

Starting time of a task.

end_time: string (date-time)

Completion time of a task.

executor_detail: v2beta1PipelineTaskExecutorDetail

Execution information of a task.

state: v2beta1RuntimeState

Runtime state of a task.

execution_id: string (int64)

Execution id of the corresponding entry in ML metadata store.

error: googlerpcStatus

The error that occurred during task execution. Only populated when the task is in FAILED or CANCELED state.

inputs: object

Input artifacts of the task.

outputs: object

Output artifacts of the task.

parent_task_id: string

ID of the parent task if the task is within a component scope. Empty if the task is at the root level.

state_history: object[]

A sequence of task statuses. This field keeps a record of state transitions.

pod_name: string

Name of the corresponding pod assigned by the orchestration engine. Also known as node_id.

child_tasks: object[]

Sequence of dependen tasks.

v2beta1PipelineTaskExecutorDetail: object

Runtime information of a pipeline task executor.

main_job: string

The name of the job for the main container execution.

pre_caching_check_job: string

The name of the job for the pre-caching-check container execution. This job will be available if the Run.pipeline_spec specifies the pre_caching_check hook in the lifecycle events.

failed_main_jobs: string[]

The names of the previously failed job for the main container executions. The list includes the all attempts in chronological order.

string
failed_pre_caching_check_jobs: string[]

The names of the previously failed job for the pre-caching-check container executions. This job will be available if the Run.pipeline_spec specifies the pre_caching_check hook in the lifecycle events. The list includes the all attempts in chronological order.

string

v2beta1PipelineVersion: object

pipeline_id: string

Required input field. Unique ID of the parent pipeline.

pipeline_version_id: string

Output. Unique pipeline version ID. Generated by API server.

display_name: string

Required input field. Pipeline version name provided by user.

description: string

Optional input field. Short description of the pipeline version.

created_at: string (date-time)

Output. Creation time of the pipeline version.

package_url: v2beta1Url

Input. Required. The URL to the source of the pipeline version. This is required when creating the pipeine version through CreatePipelineVersion API.

code_source_url: string

Input. Optional. The URL to the code source of the pipeline version. The code is usually the Python definition of the pipeline and potentially related the component definitions. This allows users to trace back to how the pipeline YAML was created.

pipeline_spec: object

Output. The pipeline spec for the pipeline version.

error: googlerpcStatus

In case any error happens retrieving a pipeline version field, only pipeline ID, pipeline version ID, and the error message are returned. Client has the flexibility of choosing how to handle the error. This is especially useful during List() calls.

v2beta1PipelineVersionReference: object

Reference to an existing pipeline version.

pipeline_id: string

Input. Required. Unique ID of the parent pipeline.

pipeline_version_id: string

Input. Required. Unique ID of an existing pipeline version.

v2beta1Predicate: object

Predicate captures individual conditions that must be true for a resource being filtered.

operation: v2beta1PredicateOperation
key: string

Key for the operation (first argument).

int_value: integer (int32)

Integer.

long_value: string (int64)

Long integer.

string_value: string

String.

timestamp_value: string (date-time)

Timestamp values will be converted to Unix time (seconds since the epoch) prior to being used in a filtering operation.

int_values: PredicateIntValues

Array values below are only meant to be used by the IN operator.

long_values: PredicateLongValues

List of long integers.

string_values: PredicateStringValues

List of strings.

v2beta1PredicateOperation: string , x ∈ { OPERATION_UNSPECIFIED (default) , EQUALS , NOT_EQUALS , GREATER_THAN , GREATER_THAN_EQUALS , LESS_THAN , LESS_THAN_EQUALS , IN , IS_SUBSTRING }

Operation is the operation to apply.

  • OPERATION_UNSPECIFIED: Default operation. This operation is not used.
  • EQUALS: Operation on scalar values. Only applies to one of |int_value|, |long_value|, |string_value| or |timestamp_value|.
  • NOT_EQUALS: Negated EQUALS.
  • GREATER_THAN: Greater than operation.
  • GREATER_THAN_EQUALS: Greater than or equals operation.
  • LESS_THAN: Less than operation.
  • LESS_THAN_EQUALS: Less than or equals operation
  • IN: Checks if the value is a member of a given array, which should be one of |int_values|, |long_values| or |string_values|.
  • IS_SUBSTRING: Checks if the value contains |string_value| as a substring match. Only applies to |string_value|.

v2beta1ReadArtifactResponse: object

data: string (byte)

Byte array of the artifact content.

v2beta1RecurringRun: object

recurring_run_id: string

Output. Unique run ID generated by API server.

display_name: string

Required input field. Recurring run name provided by user. Not unique.

description: string

Optional input field. Describes the purpose of the recurring run.

pipeline_version_id: string

The ID of the pipeline version used for creating runs.

pipeline_spec: object

The pipeline spec.

pipeline_version_reference: v2beta1PipelineVersionReference

Reference to a pipeline version containing pipeline_id and pipeline_version_id.

runtime_config: v2beta1RuntimeConfig

Runtime config of the pipeline.

service_account: string

Optional input field. Specifies which Kubernetes service account this recurring run uses.

max_concurrency: string (int64)

Required input field. Specifies how many runs can be executed concurrently. Range [1-10].

trigger: v2beta1Trigger

Required input field. Specifies how a run is triggered. Support cron mode or periodic mode.

mode: RecurringRunMode
created_at: string (date-time)

Output. The time this recurring run was created.

updated_at: string (date-time)

Output. The last time this recurring run was updated.

status: v2beta1RecurringRunStatus
error: googlerpcStatus

In case any error happens retrieving a recurring run field, only recurring run ID and the error message is returned. Client has the flexibility of choosing how to handle the error. This is especially useful during listing call.

no_catchup: boolean (boolean)

Optional input field. Whether the recurring run should catch up if behind schedule. If true, the recurring run will only schedule the latest interval if behind schedule. If false, the recurring run will catch up on each past interval.

namespace: string

TODO (gkclat): consider removing this field if it can be obtained from the parent experiment. Output only. Namespace this recurring run belongs to. Derived from the parent experiment.

experiment_id: string

ID of the parent experiment this recurring run belongs to.

v2beta1RecurringRunStatus: string , x ∈ { STATUS_UNSPECIFIED (default) , ENABLED , DISABLED }

Output. The status of the recurring run.

v2beta1Run: object

experiment_id: string

Input. ID of the parent experiment. The default experiment ID will be used if this is not specified.

run_id: string

Output. Unique run ID. Generated by API server.

display_name: string

Required input. Name provided by user, or auto generated if run is created by a recurring run.

storage_state: v2beta1RunStorageState

Output. Specifies whether this run is in archived or available mode.

description: string

Optional input. Short description of the run.

pipeline_version_id: string

ID of an existing pipeline version.

pipeline_spec: object

Pipeline spec.

pipeline_version_reference: v2beta1PipelineVersionReference

Reference to a pipeline version containing pipeline_id and pipeline_version_id.

runtime_config: v2beta1RuntimeConfig

Required input. Runtime config of the run.

service_account: string

Optional input. Specifies which kubernetes service account is used.

created_at: string (date-time)

Output. Creation time of the run.

scheduled_at: string (date-time)

Output. When this run is scheduled to start. This could be different from created_at. For example, if a run is from a backfilling job that was supposed to run 2 month ago, the created_at will be 2 month behind scheduled_at.

finished_at: string (date-time)

Output. Completion of the run.

state: v2beta1RuntimeState

Output. Runtime state of a run.

error: googlerpcStatus

In case any error happens retrieving a run field, only run ID and the error message is returned. Client has the flexibility of choosing how to handle the error. This is especially useful during listing call.

run_details: v2beta1RunDetails

Output. Runtime details of a run.

recurring_run_id: string

ID of the recurring run that triggered this run.

state_history: object[]

Output. A sequence of run statuses. This field keeps a record of state transitions.

v2beta1RunDetails: object

Runtime details of a run.

pipeline_context_id: string (int64)

Pipeline context ID of a run.

pipeline_run_context_id: string (int64)

Pipeline run context ID of a run.

task_details: object[]

Runtime details of the tasks that belong to the run.

v2beta1RunStorageState: string , x ∈ { STORAGE_STATE_UNSPECIFIED (default) , AVAILABLE , ARCHIVED }

Describes whether an entity is available or archived.

  • STORAGE_STATE_UNSPECIFIED: Default state. This state in not used
  • AVAILABLE: Entity is available.
  • ARCHIVED: Entity is archived.

v2beta1RuntimeConfig: object

The runtime config.

parameters: object

The runtime parameters of the Pipeline. The parameters will be used to replace the placeholders at runtime.

object
pipeline_root: string

v2beta1RuntimeState: string , x ∈ { RUNTIME_STATE_UNSPECIFIED (default) , PENDING , RUNNING , SUCCEEDED , SKIPPED , FAILED , CANCELING , CANCELED , PAUSED }

Describes the runtime state of an entity.

  • RUNTIME_STATE_UNSPECIFIED: Default value. This value is not used.
  • PENDING: Service is preparing to execute an entity.
  • RUNNING: Entity execution is in progress.
  • SUCCEEDED: Entity completed successfully.
  • SKIPPED: Entity has been skipped. For example, due to caching.
  • FAILED: Entity execution has failed.
  • CANCELING: Entity is being canceled. From this state, an entity may only change its state to SUCCEEDED, FAILED or CANCELED.
  • CANCELED: Entity has been canceled.
  • PAUSED: Entity has been paused. It can be resumed.

v2beta1RuntimeStatus: object

Timestamped representation of a runtime state with an optional error.

update_time: string (date-time)

Update time of this state.

state: v2beta1RuntimeState

The state of a runtime instance.

error: googlerpcStatus

The error that occurred during the state. May be set when the state is any of the non-final states (PENDING/RUNNING/CANCELING) or FAILED state. If the state is FAILED, the error here is final and not going to be retried. If the state is a non-final state, the error indicates that a system-error being retried.

v2beta1Trigger: object

Trigger defines what starts a pipeline run.

cron_schedule: v2beta1CronSchedule
periodic_schedule: v2beta1PeriodicSchedule

v2beta1Url: object

pipeline_url: string

URL of the pipeline version definition.

v2beta1Visualization: object

type: v2beta1VisualizationType
source: string

Path pattern of input data to be used during generation of visualizations. This is required when creating the pipeline through CreateVisualization API.

arguments: string

Variables to be used during generation of a visualization. This should be provided as a JSON string. This is required when creating the pipeline through CreateVisualization API.

html: string

Output. Generated visualization html.

error: string

In case any error happens when generating visualizations, only visualization ID and the error message are returned. Client has the flexibility of choosing how to handle the error.

v2beta1VisualizationType: string , x ∈ { ROC_CURVE (default) , TFDV , TFMA , TABLE , CUSTOM }

Type of visualization to be generated. This is required when creating the pipeline through CreateVisualization API.

Feedback

Was this page helpful?