Command-Line
awk/sed/perl 一條線來編輯 json 文件
我們要添加以下行：
export KAFKA_HEAP_OPTS="-Xmx8g -Xms8g"和：
export KAFKA_JVM_PERFORMANCE_OPTS=" -XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80"在“內容”行
我們能否得到建議如何使用 awk、sed、perl 單線等編輯文件
更新前的json範例
{ "href" : "http://master02:8080/api/v1/clusters/HDP/configurations?type=kafka-env&tag=version1527250007610", "items" : [ { "href" : "http://master02:8080/api/v1/clusters/HDP/configurations?type=kafka-env&tag=version1527250007610", "tag" : "version1527250007610", "type" : "kafka-env", "version" : 8, "Config" : { "cluster_name" : "HDP", "stack_id" : "HDP-2.6" }, "properties" : { "content" : "\n#!/bin/bash\n\n# Set KAFKA specific environment variables here.\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}\nexport PATH=$PATH:$JAVA_HOME/bin\nexport PID_DIR={{kafka_pid_dir}}\nexport LOG_DIR={{kafka_log_dir}}\nexport KAFKA_KERBEROS_PARAMS={{kafka_kerberos_params}}\nexport JMX_PORT=9997\n# Add kafka sink to classpath and related depenencies\nif [ -e \"/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\" ]; then\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/lib/*\nfi\n\nif [ -f /etc/kafka/conf/kafka-ranger-env.sh ]; then\n. /etc/kafka/conf/kafka-ranger-env.sh\nfi", "is_supported_kafka_ranger" : "true", "kafka_log_dir" : "/var/log/kafka", "kafka_pid_dir" : "/var/run/kafka", "kafka_user" : "kafka", "kafka_user_nofile_limit" : "128000", "kafka_user_nproc_limit" : "65536" } } ]預期輸出（更新後的 json 範例）
{ "href" : "http://master02:8080/api/v1/clusters/HDP/configurations?type=kafka-env&tag=version1527250007610", "items" : [ { "href" : "http://master02:8080/api/v1/clusters/HDP/configurations?type=kafka-env&tag=version1527250007610", "tag" : "version1527250007610", "type" : "kafka-env", "version" : 8, "Config" : { "cluster_name" : "HDP", "stack_id" : "HDP-2.6" }, "properties" : { "content" : "\n#!/bin/bash\n\n# Set KAFKA specific environment variables here.\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}\nexport PATH=$PATH:$JAVA_HOME/bin\nexport PID_DIR={{kafka_pid_dir}}\nexport LOG_DIR={{kafka_log_dir}}\nexport KAFKA_KERBEROS_PARAMS={{kafka_kerberos_params}}\nexport JMX_PORT=9997\n# Add kafka sink to classpath and related depenencies\nif [ -e \"/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\" ]; then\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/lib/*\nfi\n\nif [ -f /etc/kafka/conf/kafka-ranger-env.sh ]; then\n. /etc/kafka/conf/kafka-ranger-env.sh\nfi\nexport KAFKA_HEAP_OPTS=\"-Xmx8g -Xms8g\"\nKAFKA_JVM_PERFORMANCE_OPTS=\"-XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80\n"", "is_supported_kafka_ranger" : "true", "kafka_log_dir" : "/var/log/kafka", "kafka_pid_dir" : "/var/run/kafka", "kafka_user" : "kafka", "kafka_user_nofile_limit" : "128000", "kafka_user_nproc_limit" : "65536" } } ]更新後內容行的其他範例
"content" : "\n#!/bin/bash\n\n# Set KAFKA specific environment variables here.\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}\nexport PATH=$PATH:$JAVA_HOME/bin\nexport PID_DIR={{kafka_pid_dir}}\nexport LOG_DIR={{kafka_log_dir}}\nexport KAFKA_KERBEROS_PARAMS={{kafka_kerberos_params}}\nexport JMX_PORT=9997\n# Add kafka sink to classpath and related depenencies\nif [ -e \"/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\" ]; then\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/lib/*\nfi\n\nif [ -f /etc/kafka/conf/kafka-ranger-env.sh ]; then\n. /etc/kafka/conf/kafka-ranger-env.sh\nfi\nexport KAFKA_HEAP_OPTS=\"-Xmx8g -Xms8g\"\nKAFKA_JVM_PERFORMANCE_OPTS=\"-XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80\n"",
不是單線，而是…
$ new_lines='\\nexport KAFKA_HEAP_OPTS=\\"-Xmx8g -Xms8g\\"\\nexport KAFKA_JVM_PERFORMANCE_OPTS=\\" -XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80\\"' $ new_content=$( jq '.items[0].properties.content' file.json | sed 's/"$/'"$new_lines"'"/') $ jq '.items[0].properties.content = '"$new_content" file.json { "href": "http://master02:8080/api/v1/clusters/HDP/configurations?type=kafka-env&tag=version1527250007610", "items": [ { "href": "http://master02:8080/api/v1/clusters/HDP/configurations?type=kafka-env&tag=version1527250007610", "tag": "version1527250007610", "type": "kafka-env", "version": 8, "Config": { "cluster_name": "HDP", "stack_id": "HDP-2.6" }, "properties": { "content": "\n#!/bin/bash\n\n# Set KAFKA specific environment variables here.\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}\nexport PATH=$PATH:$JAVA_HOME/bin\nexport PID_DIR={{kafka_pid_dir}}\nexport LOG_DIR={{kafka_log_dir}}\nexport KAFKA_KERBEROS_PARAMS={{kafka_kerberos_params}}\nexport JMX_PORT=9997\n# Add kafka sink to classpath and related depenencies\nif [ -e \"/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\" ]; then\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar\n export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/lib/*\nfi\n\nif [ -f /etc/kafka/conf/kafka-ranger-env.sh ]; then\n. /etc/kafka/conf/kafka-ranger-env.sh\nfi\nexport KAFKA_HEAP_OPTS=\"-Xmx8g -Xms8g\"\nexport KAFKA_JVM_PERFORMANCE_OPTS=\" -XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80\"", "is_supported_kafka_ranger": "true", "kafka_log_dir": "/var/log/kafka", "kafka_pid_dir": "/var/run/kafka", "kafka_user": "kafka", "kafka_user_nofile_limit": "128000", "kafka_user_nproc_limit": "65536" } } ] }以可讀方式驗證新內容：
$ printf "$new_content\n" " #!/bin/bash # Set KAFKA specific environment variables here. # The java implementation to use. export JAVA_HOME={{java64_home}} export PATH=$PATH:$JAVA_HOME/bin export PID_DIR={{kafka_pid_dir}} export LOG_DIR={{kafka_log_dir}} export KAFKA_KERBEROS_PARAMS={{kafka_kerberos_params}} export JMX_PORT=9997 # Add kafka sink to classpath and related depenencies if [ -e "/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar" ]; then export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/lib/* fi if [ -f /etc/kafka/conf/kafka-ranger-env.sh ]; then . /etc/kafka/conf/kafka-ranger-env.sh fi export KAFKA_HEAP_OPTS="-Xmx8g -Xms8g" export KAFKA_JVM_PERFORMANCE_OPTS=" -XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 - XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M-XX:MinMetaspaceFreeRatio=50 - XX:MaxMetaspaceFreeRatio=80""
使用JSON模組的 Perl 解決方案：
perl -MJSON -we ' $json = JSON->new->pretty->canonical; open $fh, "<", shift or die $!; $j = $json->decode(do { undef $/; <$fh> }); $j->{items}[0]{properties}{content} .= qq(\nexport KAFKA_HEAP_OPTS="-Xmx8g -Xms8g"\nexport KAFKA_JVM_PERFORMANCE_OPTS=" -XX:MetaspaceSize=96m -XX:+UseG1GC-XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M -XX:MinMetaspaceFreeRatio=50 -XX:MaxMetaspaceFreeRatio=80"); print $json->encode($j); ' -- file.json > new.json我必須修復您預期的文件（
export第二個變數失去，加上一些空白問題）並將其規範化以獲得完全相同的輸出。