Project Management

Reminders:

1. Please read Access and Authentication REST API and understand how authentication works.
2. On Curl command line, don't forget to quote the URL if it contains the special char &.

Create a Project

• POST http://host:port/kylin/api/projects

• HTTP Header

  • Accept: application/vnd.apache.kylin-v4-public+json
  • Accept-Language: en
  • Content-Type: application/json;charset=utf-8

• HTTP Body: JSON Object

  • name - required string, project name.
  • description - optional string, project description.
  • maintain_model_type - required string, project type, MANUAL_MAINTAIN for default.

• Curl Request Example

  curl -X POST \
  'http://host:port/kylin/api/projects' \
  -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Basic QURNSU46S1lMSU4=' \
  -H 'Content-Type: application/json;charset=utf-8' \
  -d '{
    "name": "a",
    "description": "",
    "maintain_model_type": "MANUAL_MAINTAIN"
  }'

• Response Example

  {
      "code":"000",
      "data":{
          "uuid":"c7713bea-7df5-49d5-9713-f0094addbafe",
          "last_modified":1574389912687,
          "create_time":1574389912687,
          "version":"4.0.0.0",
          "mvcc":0,
          "name":"a",
          "owner":"ADMIN",
          "status":"ENABLED",
          "create_time_utc":1574389912687,
          "default_database":"DEFAULT",
          "description":"",
          "ext_filters":[
  
          ],
          "maintain_model_type":"AUTO_MAINTAIN",
          "override_kylin_properties":{
  
          },
          "segment_config":{
              "auto_merge_enabled":true,
              "auto_merge_time_ranges":[
                  "WEEK",
                  "MONTH",
                  "QUARTER",
                  "YEAR"
              ],
              "volatile_range":{
                  "volatile_range_number":0,
                  "volatile_range_enabled":false,
                  "volatile_range_type":"DAY"
              },
              "retention_range":{
                  "retention_range_number":1,
                  "retention_range_enabled":false,
                  "retention_range_type":"MONTH"
              }
          }
      },
      "msg":""
  }

Delete a Project

• DELETE http://host:port/kylin/api/projects/{project}

• URL Parameters

  • project - required string, project name.

• HTTP Header

  • Accept: application/vnd.apache.kylin-v4-public+json
  • Accept-Language: en
  • Content-Type: application/json;charset=utf-8

• Curl Request Example

  curl -X DELETE \
  'http://host:port/kylin/api/projects/b' \
  -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Basic QURNSU46S1lMSU4=' \
  -H 'Content-Type: application/json;charset=utf-8'

• Response Example

  {
      "code":"000",
      "data":null,
      "msg":""
  }

Pushdown Setting

• PUT http://host:port/kylin/api/projects/{project}/push_down_config

• URL Parameters

  • project - required string, project name.

• HTTP Header

  • Content-Type: application/json;charset=utf-8
  • Accept: application/vnd.apache.kylin-v4-public+json
  • Accept-Language: en

• HTTP Body: JSON Object

  • push_down_enabled - required boolean, whether to turn on query pushdown, true for turning on, false for turning off.

• Curl Request Example

  curl -X PUT \
  'http://host:port/kylin/api/projects/b/push_down_config' \
  -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Basic QURNSU46S1lMSU4=' \
  -H 'Content-Type: application/json;charset=utf-8' \
  -d '{
   "push_down_enabled":true
  }'

• Response Example

  {
      "code":"000",
      "data":"",
      "msg":""
  }

Pushdown Configure Setting

• PUT http://host:port/kylin/api/projects/{project}/push_down_project_config

• URL Parameters

  • project - required string，project name。

• HTTP Header

  • Content-Type: application/json;charset=utf-8
  • Accept: application/vnd.apache.kylin-v4-public+json
  • Accept-Language: en

• HTTP Body: JSON Object

• runner_class_name - required string，project config property kylin.query.pushdown.runner-class-name. To specify the query engine when query pushdown. For default, when pushdown to the native Spark, the value is org.apache.kylin.query.pushdown.PushDownRunnerSparkImpl.

• converter_class_names - required string，project config SQL conversion property kylin.query.pushdown.converter-class-names.

• Curl Request Example

  curl -X PUT \
  'http://host:port/kylin/api/projects/b/push_down_project_config' \
  -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Basic QURNSU46S1lMSU4=' \
  -H 'Content-Type: application/json;charset=utf-8' \
  -d '{
   "project": "project_name",
   "runner_class_name":"org.apache.kylin.query.pushdown.PushDownRunnerSparkImpl",
   "converter_class_names":"org.apache.kylin.query.security.HackSelectStarWithColumnACL,org.apache.kylin.query.util.SparkSQLFunctionConverter"
  }' 

• Response Example

  {
      "code":"000",
      "data":"",
      "msg":""
  }

Segment Settings

• PUT http://host:port/kylin/api/projects/{project}/segment_config

• URL Parameters

  • project - required string, project name.

• HTTP Header

  • Accept: application/vnd.apache.kylin-v4-public+json
  • Accept-Language: en
  • Content-Type: application/json;charset=utf-8

• HTTP Body: JSON Object

  • auto_merge_enabled - required boolean, whether to turn on segment auto-merge, true for turning on, false for turning off.
  • auto_merge_time_ranges - required array, the period of segment auto-merge and the unit of retentio threshold. Optional values: DAY, WEEK, MONTH, QUARTER, YEAR. For example, you can set auto-merge the segments in 1 week.
  • volatile_range - optional json, the volatile range of auto-merge, which means 'Auto-Merge' will not merge latest [Volatile Range] days segment. For example, If you set 10 days, it means that segments within 10 days will not be merged automatically.
    • volatile_range_number - optional int，The time of volatile range. The default value is 0. Optional values: positive integer and 0.
    • volatile_range_type - optional string，The time unit of the volatile range. The default value is Day. Optional values: DAY, WEEK, MONTH.
  • retention_range - optional json, retention threshold, which means to retain the segments in the retention threshold. If you set 1 year, that means the segments exceed the 1 year will be removed by the system automatically.
    • retention_range_enabled - optional boolean, whether to turn on the retention threshold, true for turning on, false for turning off, and the default value is false.
    • retention_range_number - optional int，The time to retention threshold. The default value is 1. Optional values: positive integer and 0.
    • create_empty_segment_enabled - optional boolean, whether to allow add new segment. true for turning on, false for turning off, and the default value is false.

• Curl Request Example

  curl -X PUT \
  'http://host:port/kylin/api/projects/b/segment_config' \
  -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Basic QURNSU46S1lMSU4=' \
  -H 'Content-Type: application/json;charset=utf-8' \
  -d '{
          "auto_merge_time_ranges":[
              "WEEK",
              "MONTH",
              "QUARTER",
              "DAY",
              "YEAR"
          ],
          "auto_merge_enabled":true,
          "volatile_range":{
              "volatile_range_number":0,
              "volatile_range_type":"DAY"
          },
          "retention_range":{
              "retention_range_number":2,
              "retention_range_enabled":false
          }
      }'

Notes：

• auto_merge_time_ranges : array contains at least one value.
• The time unit of retention_range is equal to the maximum time unit in the auto_merge_time_ranges array. If "auto_merge_time_ranges":["DAY",MONTH"], then the time unit of retention_range is MONTH. If "auto_merge_time_ranges":["YEAR",WEEK"], then the time unit of retention_range is YEAR.

• Response Example

    {
        "code":"000",
        "data":"",
        "msg":""
    }

Default Database

• PUT http://host:port/kylin/api/projects/{project}/default_database

• URL Parameters

  • project - required string, project name.

• HTTP Header

  • Accept: application/vnd.apache.kylin-v4-public+json
  • Accept-Language: en
  • Content-Type: application/json;charset=utf-8

• HTTP Body: JSON Object

• default_database - required string, default database name.

• Curl Request Example

  curl -X PUT \
  'http://host:port/kylin/api/projects/b/default_database' \
  -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Basic QURNSU46S1lMSU4=' \
  -H 'Content-Type: application/json;charset=utf-8' \
  -d '{
          "default_database":"EDW"
      }'

• Response Example

    {
        "code":"000",
        "data":"",
        "msg":""
    }

Job Notification

• PUT http://host:port/kylin/api/projects/{project}/job_notification_config

• URL Parameters

  • project - required string, project name.

• HTTP Header

  • Accept: application/vnd.apache.kylin-v4-public+json
  • Accept-Language: en
  • Content-Type: application/json;charset=utf-8

• HTTP Body: JSON Object

  • data_load_empty_notification_enabled - required boolean, whether to turn on empty data load notification. True means to notify user if there is an empty data load job.
  • job_error_notification_enabled - required boolean, whether to turn on error job notification. True means to notify user if there is an error job.
  • job_notification_emails - required array，email address for the job notification. Email format: xx@xx.xx.

• Curl Request Example

  curl -X PUT \
  'http://host:port/kylin/api/projects/b/job_notification_config' \
  -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Basic QURNSU46S1lMSU4=' \
  -H 'Content-Type: application/json;charset=utf-8' \
  -d '{
          "data_load_empty_notification_enabled":true,
          "job_error_notification_enabled":false,
          "job_notification_emails":[
              "nnnn@yourmail.io","tttt@yourmail.io"
          ]
      }'

• Response Example

    {
        "code":"000",
        "data":"",
        "msg":""
    }

YARN Application Queue

• PUT http://host:port/kylin/api/projects/{project}/yarn_queue

• URL Parameters

  • project - required string, project name.

• HTTP Header

  • Accept: application/vnd.apache.kylin-v4-public+json
  • Accept-Language: en
  • Content-Type: application/json;charset=utf-8

• HTTP Body: JSON Object

  • queue_name - required string, the name of YARN application queue.

• Curl Request Example

  curl -X PUT \
  'http://host:port/kylin/api/projects/b/yarn_queue' \
  -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Basic QURNSU46S1lMSU4=' \
  -H 'Content-Type: application/json;charset=utf-8' \
  -d '{
          "queue_name":"yarnqueue"
      }'

• Response Example

  {
      "code": "000",
      "data": "",
      "msg": ""
  }

Project Reset Setting

The Low Usage Storage, Segment Settings, Storage Quota and Job Notification can be reset to the default values. Only one of the settings can be reset each time.

• PUT http://host:port/kylin/api/projects/{project}/project_config

• URL Parameters

  • project - required string, project name.

• HTTP Header

  • Accept: application/vnd.apache.kylin-v4-public+json
  • Accept-Language: en
  • Content-Type: application/json;charset=utf-8

• HTTP Body: JSON Object

  • reset_item - required string, reset project settings item. Optional values: job_notification_config, query_accelerate_threshold, garbage_cleanup_config, segment_config, storage_quota_config.

• Curl Request Example

  curl -X PUT \
  'http://host:port/kylin/api/projects/a/project_config' \
  -H 'Accept: application/vnd.apache.kylin-v4-public+json' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Basic QURNSU46S1lMSU4=' \
  -H 'Content-Type: application/json;charset=utf-8' \
  -d '{
          "reset_item":"job_notification_config"
      }'

• Response Example

   {
       "code":"000",
       "data":{
           "project":"a",
           "description":"",
           "maintain_model_type":"MANUAL_MAINTAIN",
           "default_database":"EDW",
           "semi_automatic_mode":false,
           "storage_quota_size":10995116277760,
           "push_down_enabled":true,
           "auto_merge_enabled":true,
           "auto_merge_time_ranges":[
               "WEEK",
               "MONTH",
               "QUARTER",
               "YEAR"
           ],
           "volatile_range":{
               "volatile_range_number":0,
               "volatile_range_enabled":true,
               "volatile_range_type":"DAY"
           },
           "retention_range":{
               "retention_range_number":1,
               "retention_range_enabled":false,
               "retention_range_type":"YEAR"
           },
           "job_error_notification_enabled":false,
           "data_load_empty_notification_enabled":false,
           "job_notification_emails":[
   
           ],
           "threshold":20,
           "tips_enabled":true,
           "frequency_time_window":"MONTH",
           "low_frequency_threshold":5
       },
       "msg":""
   }

• The default values for Low Usage Strage:

  • "frequency_time_window":"MONTH"
  • "low_frequency_threshold":0

• The default values for Segment Settings:

  • "auto_merge_enabled":true
    • "auto_merge_time_ranges":["DAY", "MONTH", "QUARTER", "YEAR"]
  • "volatile_range"
    • "volatile_range_number":0
    • "volatile_range_type":"DAY"
  • retention_range
    • "retention_range_enabled":false
    • "retention_range_number":1

• "create_empty_segment_enabled":false

• The default values for Job Notification:

  • "data_load_empty_notification_enabled":false
  • "job_error_notification_enabled":false
  • "job_notification_emails" is null