Set Up System Cube
What is System Cube
For better supporting self-monitoring, a set of system Cubes are created under the system project, called "KYLIN_SYSTEM". Currently, there are five Cubes. Three are for query metrics, "METRICS_QUERY_EXECUTION", "METRICS_QUERY_SPARK_JOB", "METRICS_QUERY_SPARK_STAGE". And the other two are for job metrics, "METRICS_JOB", "METRICS_JOB_EXCEPTION".
How to Set Up System Cube
Please check document: How to use System Cube in Kylin4.0
Details of System Cube
Common Dimension
For all of these Cube, admins can query at four time granularities. From higher level to lower, it's as follows:
| KYEAR_BEGIN_DATE | year |
| KMONTH_BEGIN_DATE | month |
| KWEEK_BEGIN_DATE | week |
| KDAY_DATE | date |
METRICS_QUERY
This Cube is for collecting query metrics at the Spark Execution level.
METRICS_QUERY_SPARK_JOB
This Cube is for collecting Spark Job level query metrics.
METRICS_QUERY_SPARK_STAGE
This Cube is for collecting Spark Stage level query metrics.
METRICS_JOB
In Kylin, there are mainly three types of job:
- "BUILD", for building Cube segments from HIVE.
- "MERGE", for merging Cube segments in storage engine.
- "OPTIMIZE", for dynamically adjusting the precalculated cuboid tree base on the base cuboid in storage engine.
This Cube is for collecting job metrics. The details are as follows:
| Dimension | |
|---|---|
| PROJECT | |
| CUBE_NAME | |
| JOB_TYPE | |
| CUBING_TYPE | in kylin, there are two cubing algorithms, Layered & Fast(InMemory) |
| Measure | |
|---|---|
| COUNT | |
| MIN, MAX, SUM of DURATION | the duration from a job start to finish |
| MIN, MAX, SUM of TABLE_SIZE | the size of data source in bytes |
| MIN, MAX, SUM of CUBE_SIZE | the size of created Cube segment in bytes |
| MIN, MAX, SUM of PER_BYTES_TIME_COST | = DURATION / TABLE_SIZE |
| MIN, MAX, SUM of WAIT_RESOURCE_TIME | a job may includes several MR(map reduce) jobs. Those MR jobs may wait because of lack of Hadoop resources. |
METRICS_JOB_EXCEPTION
This Cube is for collecting job exception metrics. The details are as follows:
| Dimension | |
|---|---|
| PROJECT | |
| CUBE_NAME | |
| JOB_TYPE | |
| CUBING_TYPE | |
| EXCEPTION | when running a job, exceptions may happen. It's for classifying different exception types |
| Measure |
|---|
| COUNT |