Kylin detects Hadoop/Hive/HBase configurations from the environments automatically, for example the “core-site.xml”, the “hbase-site.xml” and others. Besides, Kylin has its own configurations, managed in the “conf” folder.

-bash-4.1# ls -l $KYLIN_HOME/conf

kylin_hive_conf.xml
kylin_job_conf_inmem.xml
kylin_job_conf.xml
kylin-kafka-consumer.xml
kylin.properties
kylin-server-log4j.properties
kylin-tools-log4j.properties
setenv.sh

kylin_hive_conf.xml

The Hive configurations that Kylin applied when fetching data from Hive.

kylin_job_conf.xml and kylin_job_conf_inmem.xml

Hadoop MR configurations when Kylin run MapReduce jobs. The “kylin_job_conf_inmem.xml” one requests more memory for mapper, used for Kylin’s “In-mem cubing” job.

kylin-kafka-consumer.xml

Kafka configurations when Kylin fetching data from Kafka brokers.

kylin-server-log4j.properties

Kylin server’s log configurations.

kylin-tools-log4j.properties

Kylin command line’s log configurations.

setenv.sh

Shell script to set environment variables. It will be invoked in “kylin.sh” and other scripts in “bin” folder. Typically, you can adjust the Kylin JVM heap size here, and set “KAFKA_HOME” and other environment variables.

kylin.properties

The main configuration file of Kylin.

Key Default value Description Overwritten at Cube
kylin.env Dev Whether this env is a Dev, QA, or Prod environment No
kylin.env.hdfs-working-dir /kylin Working directory on HDFS No
kylin.env.zookeeper-base-path /kylin Path on ZK No
kylin.env.zookeeper-connect-string   ZK connection string; If blank, use HBase’s ZK No
kylin.env.zookeeper-acl-enabled false   No
kylin.env.zookeeper.zk-auth digest:ADMIN:KYLIN   No
kylin.env.zookeeper.zk-acl world:anyone:rwcda   No
kylin.metadata.url kylin_metadata@hbase Kylin metadata storage No
kylin.metadata.sync-retries 3   No
kylin.metadata.sync-error-handler     No
kylin.metadata.check-copy-on-write false   No
kylin.metadata.hbase-client-scanner-timeout-period 10000   No
kylin.metadata.hbase-rpc-timeout 5000   No
kylin.metadata.hbase-client-retries-number 1   No
kylin.dictionary.use-forest-trie true   No
kylin.dictionary.forest-trie-max-mb 500   No
kylin.dictionary.max-cache-entry 3000   No
kylin.dictionary.growing-enabled false   No
kylin.dictionary.append-entry-size 10000000   No
kylin.dictionary.append-max-versions 3   No
kylin.dictionary.append-version-ttl 259200000   No
kylin.snapshot.max-cache-entry 500   No
kylin.snapshot.max-mb 300   No
kylin.snapshot.ext.shard-mb 500   No
kylin.snapshot.ext.local.cache.path lookup_cache   No
kylin.snapshot.ext.local.cache.max-size-gb 200   No
kylin.cube.size-estimate-ratio 0.25   Yes
kylin.cube.size-estimate-memhungry-ratio 0.05 Deprecated Yes
kylin.cube.size-estimate-countdistinct-ratio 0.05   Yes
kylin.cube.algorithm auto Cubing algorithm for MR engine, other options: layer, inmem Yes
kylin.cube.algorithm.layer-or-inmem-threshold 7   Yes
kylin.cube.algorithm.inmem-split-limit 500   Yes
kylin.cube.algorithm.inmem-concurrent-threads 1   Yes
kylin.cube.ignore-signature-inconsistency false    
kylin.cube.aggrgroup.max-combination 4096 Max cuboid numbers in a Cube Yes
kylin.cube.aggrgroup.is-mandatory-only-valid false Whether allow a Cube only has the base cuboid. Yes
kylin.cube.rowkey.max-size 63 Max columns in Rowkey No
kylin.metadata.dimension-encoding-max-length 256 Max length for one dimension’s encoding Yes
kylin.cube.max-building-segments 10 Max building segments in one Cube Yes
kylin.cube.allow-appear-in-multiple-projects false Whether allow a Cueb appeared in multiple projects No
kylin.cube.gtscanrequest-serialization-level 1    
kylin.cube.is-automerge-enabled true Whether enable auto merge. Yes
kylin.job.log-dir /tmp/kylin/logs    
kylin.job.allow-empty-segment true Whether tolerant data source is emtpy. Yes
kylin.job.max-concurrent-jobs 10 Max concurrent running jobs No
kylin.job.sampling-percentage 100 Data sampling percentage, to calculate Cube statistics; Default be all. Yes
kylin.job.notification-enabled false Whether send email notification on job error/succeed. No
kylin.job.notification-mail-enable-starttls false   No
kylin.job.notification-mail-port 25   No
kylin.job.notification-mail-host     No
kylin.job.notification-mail-username     No
kylin.job.notification-mail-password     No
kylin.job.notification-mail-sender     No
kylin.job.notification-admin-emails     No
kylin.job.retry 0   No
       
kylin.job.scheduler.priority-considered false   No
kylin.job.scheduler.priority-bar-fetch-from-queue 20   No
kylin.job.scheduler.poll-interval-second 30   No
kylin.job.error-record-threshold 0   No
kylin.source.hive.keep-flat-table false Whether keep the intermediate Hive table after job finished. No
kylin.source.hive.database-for-flat-table default Hive database to create the intermediate table. No
kylin.source.hive.flat-table-storage-format SEQUENCEFILE   No
kylin.source.hive.flat-table-field-delimiter \u001F   No
kylin.source.hive.redistribute-flat-table true Whether or not to redistribute the flat table. Yes
kylin.source.hive.client cli   No
kylin.source.hive.beeline-shell beeline   No
kylin.source.hive.beeline-params     No
kylin.source.hive.enable-sparksql-for-table-ops false   No
kylin.source.hive.sparksql-beeline-shell     No
kylin.source.hive.sparksql-beeline-params     No
kylin.source.hive.table-dir-create-first false   No
kylin.source.hive.flat-table-cluster-by-dict-column      
kylin.source.hive.default-varchar-precision 256   No
kylin.source.hive.default-char-precision 255   No
kylin.source.hive.default-decimal-precision 19   No
kylin.source.hive.default-decimal-scale 4   No
kylin.source.jdbc.connection-url      
kylin.source.jdbc.driver      
kylin.source.jdbc.dialect default    
kylin.source.jdbc.user      
kylin.source.jdbc.pass      
kylin.source.jdbc.sqoop-home      
kylin.source.jdbc.sqoop-mapper-num 4    
kylin.source.jdbc.field-delimiter |    
kylin.storage.default 2   No
kylin.storage.hbase.table-name-prefix KYLIN_   No
kylin.storage.hbase.namespace default   No
kylin.storage.hbase.cluster-fs      
kylin.storage.hbase.cluster-hdfs-config-file      
kylin.storage.hbase.coprocessor-local-jar      
kylin.storage.hbase.min-region-count 1    
kylin.storage.hbase.max-region-count 500    
kylin.storage.hbase.hfile-size-gb 2.0    
kylin.storage.hbase.run-local-coprocessor false    
kylin.storage.hbase.coprocessor-mem-gb 3.0    
kylin.storage.partition.aggr-spill-enabled true    
kylin.storage.partition.max-scan-bytes 3221225472    
kylin.storage.hbase.coprocessor-timeout-seconds 0    
kylin.storage.hbase.max-fuzzykey-scan 200    
kylin.storage.hbase.max-fuzzykey-scan-split 1    
kylin.storage.hbase.max-visit-scanrange 1000000    
kylin.storage.hbase.scan-cache-rows 1024    
kylin.storage.hbase.region-cut-gb 5.0    
kylin.storage.hbase.max-scan-result-bytes 5242880    
kylin.storage.hbase.compression-codec none    
kylin.storage.hbase.rowkey-encoding FAST_DIFF    
kylin.storage.hbase.block-size-bytes 1048576    
kylin.storage.hbase.small-family-block-size-bytes 65536    
kylin.storage.hbase.owner-tag      
kylin.storage.hbase.endpoint-compress-result true    
kylin.storage.hbase.max-hconnection-threads 2048    
kylin.storage.hbase.core-hconnection-threads 2048    
kylin.storage.hbase.hconnection-threads-alive-seconds 60    
kylin.engine.mr.lib-dir      
kylin.engine.mr.reduce-input-mb 500    
kylin.engine.mr.reduce-count-ratio 1.0    
kylin.engine.mr.min-reducer-number 1    
kylin.engine.mr.max-reducer-number 500    
kylin.engine.mr.mapper-input-rows 1000000    
kylin.engine.mr.max-cuboid-stats-calculator-number 1    
kylin.engine.mr.uhc-reducer-count 1    
kylin.engine.mr.build-uhc-dict-in-additional-step false    
kylin.engine.mr.build-dict-in-reducer true    
kylin.engine.mr.yarn-check-interval-seconds 10    
kylin.env.hadoop-conf-dir   Hadoop conf directory; If not specified, parse from environment. No
kylin.engine.spark.rdd-partition-cut-mb 10.0 Spark Cubing RDD partition split size. Yes
kylin.engine.spark.min-partition 1 Spark Cubing RDD min partition number Yes
kylin.engine.spark.max-partition 5000 RDD max partition number Yes
kylin.engine.spark.storage-level MEMORY_AND_DISK_SER RDD persistent level. Yes
kylin.query.skip-empty-segments true Whether directly skip empty segment (metadata shows size be 0) when run SQL query. Yes
kylin.query.force-limit -1    
kylin.query.max-scan-bytes 0    
kylin.query.max-return-rows 5000000    
kylin.query.large-query-threshold 1000000    
kylin.query.cache-threshold-duration 2000    
kylin.query.cache-threshold-scan-count 10240    
kylin.query.cache-threshold-scan-bytes 1048576    
kylin.query.security-enabled true    
kylin.query.cache-enabled true    
kylin.query.timeout-seconds 0    
kylin.query.pushdown.runner-class-name      
kylin.query.pushdown.update-enabled false    
kylin.query.pushdown.cache-enabled false    
kylin.query.pushdown.jdbc.url      
kylin.query.pushdown.jdbc.driver      
kylin.query.pushdown.jdbc.username      
kylin.query.pushdown.jdbc.password      
kylin.query.pushdown.jdbc.pool-max-total 8    
kylin.query.pushdown.jdbc.pool-max-idle 8    
kylin.query.pushdown.jdbc.pool-min-idle 0    
kylin.query.security.table-acl-enabled true   No
kylin.server.mode all Kylin node mode: all|job|query. No
kylin.server.cluster-servers localhost:7070   No
kylin.server.cluster-name     No
kylin.server.query-metrics-enabled false   No
kylin.server.query-metrics2-enabled false   No
kylin.server.auth-user-cache.expire-seconds 300   No
kylin.server.auth-user-cache.max-entries 100   No
kylin.server.external-acl-provider     No
kylin.security.ldap.user-search-base     No
kylin.security.ldap.user-group-search-base     No
kylin.security.acl.admin-role     No
kylin.web.timezone PST   No
kylin.web.cross-domain-enabled true   No
kylin.web.export-allow-admin true   No
kylin.web.export-allow-other true   No
kylin.web.dashboard-enabled false   No