Pipelines API Reference

Reference documentation for the Kubeflow Pipelines API
Kubeflow Pipelines API

Kubeflow Pipelines API

Version: 0.1.38

This file contains REST API specification for Kubeflow Pipelines. The file is autogenerated from the swagger definition.

Default request content-types: application/json
Default response content-types: application/json
Schemes: http, https

Summary

Tag: RunService

Operation Description
GET /apis/v1beta1/runs

Find all runs.

POST /apis/v1beta1/runs

Create a new run.

DELETE /apis/v1beta1/runs/{id}

Delete a run.

POST /apis/v1beta1/runs/{id}:archive

Archive a run.

POST /apis/v1beta1/runs/{id}:unarchive

Restore an archived run.

GET /apis/v1beta1/runs/{run_id}

Find a specific run by ID.

GET /apis/v1beta1/runs/{run_id}/nodes/{node_id}/artifacts/{artifact_name}:read

Find a run's artifact data.

POST /apis/v1beta1/runs/{run_id}/retry

Re-initiate a failed or terminated run.

POST /apis/v1beta1/runs/{run_id}/terminate

Terminate an active run.

POST /apis/v1beta1/runs/{run_id}:reportMetrics

ReportRunMetrics reports metrics of a run. Each metric is reported in its own transaction, so this API accepts partial failures. Metric can be uniquely identified by (run_id, node_id, name). Duplicate reporting will be ignored by the API. First reporting wins.

Tag: JobService

Operation Description
GET /apis/v1beta1/jobs

Find all jobs.

POST /apis/v1beta1/jobs

Create a new job.

GET /apis/v1beta1/jobs/{id}

Find a specific job by ID.

DELETE /apis/v1beta1/jobs/{id}

Delete a job.

POST /apis/v1beta1/jobs/{id}/disable

Stops a job and all its associated runs. The job is not deleted.

POST /apis/v1beta1/jobs/{id}/enable

Restarts a job that was previously stopped. All runs associated with the job will continue.

Tag: PipelineService

Operation Description
GET /apis/v1beta1/pipeline_versions
POST /apis/v1beta1/pipeline_versions
GET /apis/v1beta1/pipeline_versions/{version_id}
DELETE /apis/v1beta1/pipeline_versions/{version_id}
GET /apis/v1beta1/pipeline_versions/{version_id}/templates
GET /apis/v1beta1/pipelines

Find all pipelines.

POST /apis/v1beta1/pipelines

Add a pipeline.

GET /apis/v1beta1/pipelines/{id}

Find a specific pipeline by ID.

DELETE /apis/v1beta1/pipelines/{id}

Delete a pipeline.

GET /apis/v1beta1/pipelines/{id}/templates

Returns a single YAML template that contains the description, parameters, and metadata associated with the pipeline provided.

Tag: ExperimentService

Operation Description
GET /apis/v1beta1/experiments

Find all experiments.

POST /apis/v1beta1/experiments

Create a new experiment.

GET /apis/v1beta1/experiments/{id}

Find a specific experiment by ID.

DELETE /apis/v1beta1/experiments/{id}

Delete an experiment.

Tag: PipelineUploadService

Operation Description
POST /apis/v1beta1/pipelines/upload

Security

Bearer

Type: apiKey
Name:

authorization

In:

header

Paths

Find all experiments.

GET /apis/v1beta1/experiments

Tags: ExperimentService
page_token query string
page_size query integer (int32)
sort_by

Can be format of "field_name", "field_name asc" or "field_name des" Ascending by default.

query string
filter

A url-encoded, JSON-serialized Filter protocol buffer (see [filter.proto](https://github.com/kubeflow/pipelines/ blob/master/backend/api/filter.proto)).

query string

Uses default content-types: application/json

200 OK

A successful response.

default
Create a new experiment.

POST /apis/v1beta1/experiments

Tags: ExperimentService

Uses default content-types: application/json

The experiment to be created.

Uses default content-types: application/json

200 OK

A successful response.

default
Delete an experiment.

DELETE /apis/v1beta1/experiments/{id}

Tags: ExperimentService
id

The ID of the experiment to be deleted.

path string

Uses default content-types: application/json

200 OK

A successful response.

default
Find a specific experiment by ID.

GET /apis/v1beta1/experiments/{id}

Tags: ExperimentService
id

The ID of the experiment to be retrieved.

path string

Uses default content-types: application/json

200 OK

A successful response.

default
Find all jobs.

GET /apis/v1beta1/jobs

Tags: JobService
page_token query string
page_size query integer (int32)
sort_by

Can be format of "field_name", "field_name asc" or "field_name des". Ascending by default.

query string
resource_reference_key.type

The type of the resource that referred to.

query string , x ∈ { UNKNOWN_RESOURCE_TYPE (default) , EXPERIMENT , JOB , PIPELINE , PIPELINE_VERSION , NAMESPACE }
resource_reference_key.id

The ID of the resource that referred to.

query string
filter

A url-encoded, JSON-serialized Filter protocol buffer (see [filter.proto](https://github.com/kubeflow/pipelines/ blob/master/backend/api/filter.proto)).

query string

Uses default content-types: application/json

200 OK

A successful response.

default
Create a new job.

POST /apis/v1beta1/jobs

Tags: JobService

Uses default content-types: application/json

The job to be created

Uses default content-types: application/json

200 OK

A successful response.

default
Delete a job.

DELETE /apis/v1beta1/jobs/{id}

Tags: JobService
id

The ID of the job to be deleted

path string

Uses default content-types: application/json

200 OK

A successful response.

default
Find a specific job by ID.

GET /apis/v1beta1/jobs/{id}

Tags: JobService
id

The ID of the job to be retrieved

path string

Uses default content-types: application/json

200 OK

A successful response.

default
Stops a job and all its associated runs. The job is not deleted.

POST /apis/v1beta1/jobs/{id}/disable

Tags: JobService
id

The ID of the job to be disabled

path string

Uses default content-types: application/json

200 OK

A successful response.

default
Restarts a job that was previously stopped. All runs associated with the job will continue.

POST /apis/v1beta1/jobs/{id}/enable

Tags: JobService
id

The ID of the job to be enabled

path string

Uses default content-types: application/json

200 OK

A successful response.

default

GET /apis/v1beta1/pipeline_versions

Tags: PipelineService
resource_key.type

The type of the resource that referred to.

query string , x ∈ { UNKNOWN_RESOURCE_TYPE (default) , EXPERIMENT , JOB , PIPELINE , PIPELINE_VERSION , NAMESPACE }
resource_key.id

The ID of the resource that referred to.

query string
page_size query integer (int32)
page_token query string
sort_by

Can be format of "field_name", "field_name asc" or "field_name des" Ascending by default.

query string
filter

A base-64 encoded, JSON-serialized Filter protocol buffer (see filter.proto).

query string

Uses default content-types: application/json

200 OK

A successful response.

default

POST /apis/v1beta1/pipeline_versions

Tags: PipelineService

Uses default content-types: application/json

ResourceReference inside PipelineVersion specifies the pipeline that this version belongs to.

Uses default content-types: application/json

200 OK

A successful response.

default

DELETE /apis/v1beta1/pipeline_versions/{version_id}

Tags: PipelineService
version_id path string

Uses default content-types: application/json

200 OK

A successful response.

default

GET /apis/v1beta1/pipeline_versions/{version_id}

Tags: PipelineService
version_id path string

Uses default content-types: application/json

200 OK

A successful response.

default

GET /apis/v1beta1/pipeline_versions/{version_id}/templates

Tags: PipelineService
version_id path string

Uses default content-types: application/json

200 OK

A successful response.

default
Find all pipelines.

GET /apis/v1beta1/pipelines

Tags: PipelineService
page_token query string
page_size query integer (int32)
sort_by

Can be format of "field_name", "field_name asc" or "field_name des" Ascending by default.

query string
filter

A url-encoded, JSON-serialized Filter protocol buffer (see [filter.proto](https://github.com/kubeflow/pipelines/ blob/master/backend/api/filter.proto)).

query string

Uses default content-types: application/json

200 OK

A successful response.

default
Add a pipeline.

POST /apis/v1beta1/pipelines

Tags: PipelineService

Uses default content-types: application/json

Uses default content-types: application/json

200 OK

A successful response.

default

POST /apis/v1beta1/pipelines/upload

Tags: PipelineUploadService

multipart/form-data

uploadfile

The pipeline to upload. Maximum size of 32MB is supported.

formData file
name query string

application/json

200 OK
default
Delete a pipeline.

DELETE /apis/v1beta1/pipelines/{id}

Tags: PipelineService
id path string

Uses default content-types: application/json

200 OK

A successful response.

default
Find a specific pipeline by ID.

GET /apis/v1beta1/pipelines/{id}

Tags: PipelineService
id path string

Uses default content-types: application/json

200 OK

A successful response.

default
Returns a single YAML template that contains the description, parameters, and metadata associated with the pipeline provided.

GET /apis/v1beta1/pipelines/{id}/templates

Tags: PipelineService
id path string

Uses default content-types: application/json

200 OK

A successful response.

default
Find all runs.

GET /apis/v1beta1/runs

Tags: RunService
page_token query string
page_size query integer (int32)
sort_by

Can be format of "field_name", "field_name asc" or "field_name des" (Example, "name asc" or "id des"). Ascending by default.

query string
resource_reference_key.type

The type of the resource that referred to.

query string , x ∈ { UNKNOWN_RESOURCE_TYPE (default) , EXPERIMENT , JOB , PIPELINE , PIPELINE_VERSION , NAMESPACE }
resource_reference_key.id

The ID of the resource that referred to.

query string
filter

A url-encoded, JSON-serialized Filter protocol buffer (see [filter.proto](https://github.com/kubeflow/pipelines/ blob/master/backend/api/filter.proto)).

query string

Uses default content-types: application/json

200 OK

A successful response.

default
Create a new run.

POST /apis/v1beta1/runs

Tags: RunService

Uses default content-types: application/json

Uses default content-types: application/json

200 OK

A successful response.

default
Delete a run.

DELETE /apis/v1beta1/runs/{id}

Tags: RunService
id path string

Uses default content-types: application/json

200 OK

A successful response.

default
Archive a run.

POST /apis/v1beta1/runs/{id}:archive

Tags: RunService
id path string

Uses default content-types: application/json

200 OK

A successful response.

default
Restore an archived run.

POST /apis/v1beta1/runs/{id}:unarchive

Tags: RunService
id path string

Uses default content-types: application/json

200 OK

A successful response.

default
Find a specific run by ID.

GET /apis/v1beta1/runs/{run_id}

Tags: RunService
run_id path string

Uses default content-types: application/json

200 OK

A successful response.

default
Find a run's artifact data.

GET /apis/v1beta1/runs/{run_id}/nodes/{node_id}/artifacts/{artifact_name}:read

Tags: RunService
run_id

The ID of the run.

path string
node_id

The ID of the running node.

path string
artifact_name

The name of the artifact.

path string

Uses default content-types: application/json

200 OK

A successful response.

default
Re-initiate a failed or terminated run.

POST /apis/v1beta1/runs/{run_id}/retry

Tags: RunService
run_id path string

Uses default content-types: application/json

200 OK

A successful response.

default
Terminate an active run.

POST /apis/v1beta1/runs/{run_id}/terminate

Tags: RunService
run_id path string

Uses default content-types: application/json

200 OK

A successful response.

default
ReportRunMetrics reports metrics of a run. Each metric is reported in its own transaction, so this API accepts partial failures. Metric can be uniquely identified by (run_id, node_id, name). Duplicate reporting will be ignored by the API. First reporting wins.

POST /apis/v1beta1/runs/{run_id}:reportMetrics

Tags: RunService

Uses default content-types: application/json

run_id

Required. The parent run ID of the metric.

path string

Uses default content-types: application/json

200 OK

A successful response.

default

Schema definitions

apiCronSchedule: object

start_time: string (date-time)
end_time: string (date-time)
cron: string

apiExperiment: object

id: string

Output. Unique experiment ID. Generated by API server.

name: string

Required input field. Unique experiment name provided by user.

description: string
created_at: string (date-time)

Output. The time that the experiment created.

apiGetTemplateResponse: object

template: string

apiJob: object

id: string

Output. Unique run ID. Generated by API server.

name: string

Required input field. Job name provided by user. Not unique.

description: string
pipeline_spec: apiPipelineSpec

Required input field. Describing what the pipeline manifest and parameters to use for the scheduled job.

resource_references: object[]

Optional input field. Specify which resource this run belongs to.

max_concurrency: string (int64)
trigger: apiTrigger

Required input field. Specify how a run is triggered. Support cron mode or periodic mode.

mode: JobMode
created_at: string (date-time)

Output. The time this job is created.

updated_at: string (date-time)

Output. The last time this job is updated.

status: string
error: string

In case any error happens retrieving a job field, only job ID and the error message is returned. Client has the flexibility of choosing how to handle error. This is especially useful during listing call.

enabled: boolean (boolean)

Input. Whether the job is enabled or not.

apiListExperimentsResponse: object

experiments: object[]

A list of experiments returned.

total_size: integer (int32)

The total number of experiments for the given query.

next_page_token: string

The token to list the next page of experiments.

apiListJobsResponse: object

jobs: object[]

A list of jobs returned.

total_size: integer (int32)
next_page_token: string

apiListPipelinesResponse: object

pipelines: object[]
total_size: integer (int32)
next_page_token: string

apiListPipelineVersionsResponse: object

versions: object[]
next_page_token: string
total_size: integer (int32)

apiListRunsResponse: object

runs: object[]
total_size: integer (int32)
next_page_token: string

apiParameter: object

name: string
value: string

apiPeriodicSchedule: object

start_time: string (date-time)
end_time: string (date-time)
interval_second: string (int64)

apiPipeline: object

id: string

Output. Unique pipeline ID. Generated by API server.

created_at: string (date-time)

Output. The time this pipeline is created.

name: string

Optional input field. Pipeline name provided by user. If not specified, file name is used as pipeline name.

description: string

Optional input field. Describing the purpose of the job.

parameters: object[]

Output. The input parameters for this pipeline. TODO(jingzhang36): replace this parameters field with the parameters field inside PipelineVersion when all usage of the former has been changed to use the latter.

url: apiUrl

The URL to the source of the pipeline. This is required when creating the pipeine through CreatePipeline API. TODO(jingzhang36): replace this url field with the code_source_urls field inside PipelineVersion when all usage of the former has been changed to use the latter.

error: string

In case any error happens retrieving a pipeline field, only pipeline ID and the error message is returned. Client has the flexibility of choosing how to handle error. This is especially useful during listing call.

default_version: apiPipelineVersion

apiPipelineRuntime: object

pipeline_manifest: string

Output. The runtime JSON manifest of the pipeline, including the status of pipeline steps and fields need for UI visualization etc.

workflow_manifest: string

Output. The runtime JSON manifest of the argo workflow. This is deprecated after pipeline_runtime_manifest is in use.

apiPipelineSpec: object

pipeline_id: string

Optional input field. The ID of the pipeline user uploaded before.

pipeline_name: string

Optional output field. The name of the pipeline. Present if the pipeline id is present.

workflow_manifest: string

Optional input field. The marshalled raw argo JSON workflow. This will be deprecated when pipeline_manifest is in use.

pipeline_manifest: string

Optional input field. The raw pipeline JSON spec.

parameters: object[]

The parameter user provide to inject to the pipeline JSON. If a default value of a parameter exist in the JSON, the value user provided here will replace.

apiPipelineVersion: object

id: string

Output. Unique version ID. Generated by API server.

name: string

Optional input field. Version name provided by user.

created_at: string (date-time)

Output. The time this pipeline version is created.

parameters: object[]

Output. The input parameters for this pipeline.

code_source_url: string

Input. Optional. Pipeline version code source.

package_url: apiUrl

Input. Required. Pipeline version package url. Whe calling CreatePipelineVersion API method, need to provide one package file location.

resource_references: object[]

Input. Required. E.g., specify which pipeline this pipeline version belongs to.

apiReadArtifactResponse: object

data: string (byte)

The bytes of the artifact content.

apiRelationship: string , x ∈ { UNKNOWN_RELATIONSHIP (default) , OWNER , CREATOR }

apiReportRunMetricsRequest: object

run_id: string

Required. The parent run ID of the metric.

metrics: object[]

List of metrics to report.

apiReportRunMetricsResponse: object

apiResourceKey: object

type: apiResourceType

The type of the resource that referred to.

id: string

The ID of the resource that referred to.

apiResourceReference: object

key: apiResourceKey
name: string

The name of the resource that referred to.

relationship: apiRelationship

Required field. The relationship from referred resource to the object.

apiResourceType: string , x ∈ { UNKNOWN_RESOURCE_TYPE (default) , EXPERIMENT , JOB , PIPELINE , PIPELINE_VERSION , NAMESPACE }

apiRun: object

id: string

Output. Unique run ID. Generated by API server.

name: string

Required input field. Name provided by user, or auto generated if run is created by scheduled job. Not unique.

storage_state: RunStorageState
description: string
pipeline_spec: apiPipelineSpec

Required input field. Describing what the pipeline manifest and parameters to use for the run.

resource_references: object[]

Optional input field. Specify which resource this run belongs to.

created_at: string (date-time)

Output. The time that the run created.

scheduled_at: string (date-time)

Output. When this run is scheduled to run. This could be different from created_at. For example, if a run is from a backfilling job that was supposed to run 2 month ago, the scheduled_at is 2 month ago, v.s. created_at is the current time.

finished_at: string (date-time)

Output. The time this run is finished.

status: string
error: string

In case any error happens retrieving a run field, only run ID and the error message is returned. Client has the flexibility of choosing how to handle error. This is especially useful during listing call.

metrics: object[]

Output. The metrics of the run. The metrics are reported by ReportMetrics API.

apiRunDetail: object

run: apiRun
pipeline_runtime: apiPipelineRuntime

apiRunMetric: object

name: string

Required. The user defined name of the metric. It must between 1 and 63 characters long and must conform to the following regular expression: [a-z]([-a-z0-9]*[a-z0-9])?.

node_id: string

Required. The runtime node ID which reports the metric. The node ID can be found in the RunDetail.workflow.Status. Metric with same (node_id, name) are considerd as duplicate. Only the first reporting will be recorded. Max length is 128.

number_value: number (double)

The number value of the metric.

format: RunMetricFormat

The display format of metric.

apiStatus: object

error: string
code: integer (int32)
details: object[]

apiTrigger: object

Trigger defines what starts a pipeline run.

cron_schedule: apiCronSchedule
periodic_schedule: apiPeriodicSchedule

apiUrl: object

pipeline_url: string

JobMode: string , x ∈ { UNKNOWN_MODE (default) , ENABLED , DISABLED }

Required input.

  • DISABLED: The job won't schedule any run if disabled.

protobufAny: object

Any contains an arbitrary serialized protocol buffer message along with a URL that describes the type of the serialized message.

Protobuf library provides support to pack/unpack Any values in the form of utility functions or additional generated methods of the Any type.

Example 1: Pack and unpack a message in C++.

Foo foo = ...;
Any any;
any.PackFrom(foo);
...
if (any.UnpackTo(&foo)) {
...
}

Example 2: Pack and unpack a message in Java.

Foo foo = ...;
Any any = Any.pack(foo);
...
if (any.is(Foo.class)) {
foo = any.unpack(Foo.class);
}

Example 3: Pack and unpack a message in Python.

foo = Foo(...)
any = Any()
any.Pack(foo)
...
if any.Is(Foo.DESCRIPTOR):
any.Unpack(foo)
...

Example 4: Pack and unpack a message in Go

 foo := &pb.Foo{...}
any, err := ptypes.MarshalAny(foo)
...
foo := &pb.Foo{}
if err := ptypes.UnmarshalAny(any, foo); err != nil {
...
}

The pack methods provided by protobuf library will by default use 'type.googleapis.com/full.type.name' as the type URL and the unpack methods only use the fully qualified type name after the last '/' in the type URL, for example "foo.bar.com/x/y.z" will yield type name "y.z".

JSON

The JSON representation of an Any value uses the regular representation of the deserialized, embedded message, with an additional field @type which contains the type URL. Example:

package google.profile;
message Person {
string first_name = 1;
string last_name = 2;
}
{
"@type": "type.googleapis.com/google.profile.Person",
"firstName": <string>,
"lastName": <string>
}

If the embedded message type is well-known and has a custom JSON representation, that representation will be embedded adding a field value which holds the custom JSON in addition to the @type field. Example (for message [google.protobuf.Duration][]):

{
"@type": "type.googleapis.com/google.protobuf.Duration",
"value": "1.212s"
}
type_url: string

A URL/resource name that uniquely identifies the type of the serialized protocol buffer message. The last segment of the URL's path must represent the fully qualified name of the type (as in path/google.protobuf.Duration). The name should be in a canonical form (e.g., leading "." is not accepted).

In practice, teams usually precompile into the binary all types that they expect it to use in the context of Any. However, for URLs which use the scheme http, https, or no scheme, one can optionally set up a type server that maps type URLs to message definitions as follows:

  • If no scheme is provided, https is assumed.
  • An HTTP GET on the URL must yield a [google.protobuf.Type][] value in binary format, or produce an error.
  • Applications are allowed to cache lookup results based on the URL, or have them precompiled into a binary to avoid any lookup. Therefore, binary compatibility needs to be preserved on changes to types. (Use versioned type names to manage breaking changes.)

Note: this functionality is not currently available in the official protobuf release, and it is not used for type URLs beginning with type.googleapis.com.

Schemes other than http, https (or the empty scheme) might be used with implementation specific semantics.

value: string (byte)

Must be a valid serialized protocol buffer of the above specified type.

ReportRunMetricsResponseReportRunMetricResult: object

metric_name: string

Output. The name of the metric.

metric_node_id: string

Output. The ID of the node which reports the metric.

status: ReportRunMetricsResponseReportRunMetricResultStatus

Output. The status of the metric reporting.

message: string

Output. The detailed message of the error of the reporting.

ReportRunMetricsResponseReportRunMetricResultStatus: string , x ∈ { UNSPECIFIED (default) , OK , INVALID_ARGUMENT , DUPLICATE_REPORTING , INTERNAL_ERROR }

  • UNSPECIFIED: Default value if not present.
  • OK: Indicates successful reporting.
  • INVALID_ARGUMENT: Indicates that the payload of the metric is invalid.
  • DUPLICATE_REPORTING: Indicates that the metric has been reported before.
  • INTERNAL_ERROR: Indicates that something went wrong in the server.

RunMetricFormat: string , x ∈ { UNSPECIFIED (default) , RAW , PERCENTAGE }

  • UNSPECIFIED: Default value if not present.
  • RAW: Display value as its raw format.
  • PERCENTAGE: Display value in percentage format.

RunStorageState: string , x ∈ { STORAGESTATE_AVAILABLE (default) , STORAGESTATE_ARCHIVED }