Bash
xargs 中用於並行處理的萬用字元變數
問題
當此函式在bash 呼叫中時，我在帶有 wilcard
VAR的函式（即）中使用輸入變數時遇到問題。ls``*``xargs此
xargs函式對文件名前綴列表的每個成員重複InfilePrefix_List.csv。輸入變數
VAR是格式為的字元串：VAR="dir/sub_dir/SharedPrefix_InfilePrefix"。
ls ${VAR}*給出它無法找到目錄的錯誤，因為它VAR由於存在*.解決方案
最後，輸入列表被保存為 .csv 文件，並且 xargs 在每個列表條目之後讀取一個額外的“\r”。修改 .csv 文件後，功能執行順利。在這裡看到同樣的問題：https ://www.unix.com/unix-for-dummies-questions-and-answers/124289-xargs-appending-r.html
乾杯
簡化範例和並行計算
實際上，我使用的函式不是
ls，而是：cdo ensmean infiles*.nc outfile.nc，這使得來自多個複制氣候數據文件的集合均值。但是，對於這篇文章，使用的範例
ls infiles*.nc很好。請注意，我使用xargs的是替代 for 循環，因為我需要利用xargs --max-procs 44選項提供的並行計算能力。範例 bash 腳本
#!/bin/bash more /dir/subdir/File_Prefix_List.csv | xargs -I {} bash -c " echo 'arg is '{}; VAR=\$(echo /dir/subdir/SharedPrefix_{}); echo 'var is '\$VAR; ls \${VAR}*.nc; exit 1;"筆記
第 2 行 - {} 表示特定
InfilePrefix字元串第 3,5 行 - 回顯檢查 VAR 是否正確 - 它是…
第 6 行 - 範例函式是
ls，列出SharedPrefix_InfilePrefix名稱中包含的所有文件調試編輯
根據要求，這是帶有此更改的範例 bash 腳本的輸出：
head -n 1 ...<my specific directories>... xargs -I {} bash -vxc " ...請注意，對於我的具體情況，範例和我執行的程式碼之間的區別是： (1)
/dir/subdir = /nobackup/b9035080/NCdata_RealsPalau。(2)/SharedPrefix_ = /Reals_Palau_tos_Oday_。(3){} = ACCESS-CM2_historical。輸出：第 1 行和第 2 行：xargs 參數 (
{} = ACCESS-CM2_historical) 和目錄變數 (VAR = /nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical) 是正確的。第3行：讀取萬用字元有錯誤。第 1 節輸出的結尾顯示了我認為相關的錯誤消息。
;cho 'arg is 'ACCESS-CM2_historical ' echo 'arg is ACCESS-CM2_historical arg is ACCESS-CM2_historical );R=$(echo /nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical ++ echo $'/nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical\r' + VAR=$'/nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical\r' echo 'var is '$VAR; ' echo 'var is /nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical var is /nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical ls ${VAR}*.nc; *.nc''/nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical *.nc: No such file or directory35080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical exit 1; + exit 1第 2 部分 - 完整輸出，其中一些我不明白 - 不確定它是否全部相關 - 抱歉 - 我是 Unix 新手！
module () { eval $($LMOD_CMD bash "$@"); [ $? = 0 ] && eval $(${LMOD_SETTARG_CMD:-:} -s sh) } ml () { eval $($LMOD_DIR/ml_cmd "$@") } # -*- shell-script -*- if [[ ${-/x} != $- ]]; then echo "Start of Lmod init/bash script to define the module command" fi + [[ hvBc != hvxBc ]] + echo 'Start of Lmod init/bash script to define the module command' Start of Lmod init/bash script to define the module command LMOD_PKG=/usr/share/lmod/lmod + LMOD_PKG=/usr/share/lmod/lmod LMOD_DIR=/usr/share/lmod/lmod/libexec + LMOD_DIR=/usr/share/lmod/lmod/libexec LMOD_CMD=/usr/share/lmod/lmod/libexec/lmod + LMOD_CMD=/usr/share/lmod/lmod/libexec/lmod MODULESHOME=/usr/share/lmod/lmod + MODULESHOME=/usr/share/lmod/lmod export LMOD_PKG + export LMOD_PKG export LMOD_CMD + export LMOD_CMD export LMOD_DIR + export LMOD_DIR export MODULESHOME + export MODULESHOME ######################################################################## # Define the module command: The first line runs the "lmod" command # to generate text: # export PATH="..." # then the "eval" converts the text into changes in the current shell. # # The second command is the settarg command. Normally LMOD_SETTARG_CMD # is undefined or is ":". Either way the eval does nothing. When the # settarg module is loaded, it defines LMOD_SETTARG_CMD. The settarg # command knows how to read the ModuleTable that Lmod maintains and # generates a series of env. vars that describe the current state of # loaded modules. So if one is on a x86_64 linux computer with gcc/4.7.2 # and openmpi/1.6.3 loaded, then settarg will assign: # # TARG=_x86_64_gcc-4.7.2_openmpi-1.6.3 # TARG_COMPILER=gcc-4.7.2 # TARG_COMPILER_FAMILY=gcc # TARG_MACH=x86_64 # TARG_MPI=openmpi-1.6.3 # TARG_MPI_FAMILY=openmpi # TARG_SUMMARY=x86_64_gcc-4.7.2_openmpi-1.6.3 # TARG_TITLE_BAR=gcc-4.7.2 O-1.6.3 # TARG_TITLE_BAR_PAREN=(gcc-4.7.2 O-1.6.3) # # unloading openmpi/1.6.3 automatically changes these vars to be: # # TARG=_x86_64_gcc-4.6.3 # TARG_COMPILER=gcc-4.6.3 # TARG_COMPILER_FAMILY=gcc # TARG_MACH=x86_64 # TARG_SUMMARY=x86_64_gcc-4.6.3 # TARG_TITLE_BAR=gcc-4.6.3 # TARG_TITLE_BAR_PAREN=(gcc-4.6.3) # # See Lmod web site for more details. module() { eval $($LMOD_CMD bash "$@") [ $? = 0 ] && eval $(${LMOD_SETTARG_CMD:-:} -s sh) } LMOD_VERSION="6.5.1" + LMOD_VERSION=6.5.1 export LMOD_VERSION + export LMOD_VERSION if [ "${LMOD_SETTARG_CMD:-:}" != ":" ]; then settarg () { eval $(${LMOD_SETTARG_CMD:-:} -s sh "$@" ) } fi + '[' : '!=' : ']' ######################################################################## # ml is a shorthand tool for people who can't type moduel, err, module # It is also a combination command: # ml -> module list # ml gcc -> module load gcc # ml -gcc intel -> module unload gcc; module load intel # It does much more do: "ml --help" for more information. unalias ml 2> /dev/null || true + unalias ml + true ml() { eval $($LMOD_DIR/ml_cmd "$@") } export_module=$(echo "YES" | /usr/bin/tr '[:upper:]' '[:lower:]') ++ echo YES ++ /usr/bin/tr '[:upper:]' '[:lower:]' + export_module=yes if [ -n "${BASH_VERSION:-}" -a "$export_module" != no ]; then export -f module export -f ml fi + '[' -n '4.2.46(2)-release' -a yes '!=' no ']' + export -f module + export -f ml unset export_module + unset export_module ######################################################################## # clearMT removes the ModuleTable from your environment. It is rarely # needed but it useful sometimes. clearMT() { eval $($LMOD_DIR/clearMT_cmd bash) } ######################################################################## # The following make the action of the settarg available to the titlebar # for both xterm's and screen but only for interactive shells. if [ -n "${PS1:-}" ]; then if [ -n "${LMOD_FULL_SETTARG_SUPPORT:-}" -a "$LMOD_FULL_SETTARG_SUPPORT" != no ]; then xSetTitleLmod() { builtin echo -n -e "\033]2;$1\007"; } SET_TITLE_BAR=: case "$TERM" in xterm*) SET_TITLE_BAR=xSetTitleLmod ;; esac SHOST=${SHOST-${HOSTNAME%%.*}} precmd() { eval $(${LMOD_SETTARG_CMD:-:} -s bash) ${SET_TITLE_BAR:-:} "${TARG_TITLE_BAR_PAREN}${USER}@${SHOST}:${PWD/#$HOME/~}" ${USER_PROMPT_CMD:-:} } # define the PROMPT_COMMAND to be precmd iff it isn't defined already. : ${PROMPT_COMMAND:=precmd} fi fi + '[' -n '' ']' ######################################################################## # Make tab completions available to bash users. if [ ${BASH_VERSINFO:-0} -ge 3 ] && [ -r /usr/share/lmod/lmod/init/lmod_bash_completions ] && [ -n "${PS1:-}" ]; then . /usr/share/lmod/lmod/init/lmod_bash_completions fi + '[' 4 -ge 3 ']' + '[' -r /usr/share/lmod/lmod/init/lmod_bash_completions ']' + '[' -n '' ']' if [[ ${-/x} != $- ]]; then echo "End of Lmod init/bash script to define the module command" fi + [[ hvBc != hvxBc ]] + echo 'End of Lmod init/bash script to define the module command' End of Lmod init/bash script to define the module command ;cho 'arg is 'ACCESS-CM2_historical ' echo 'arg is ACCESS-CM2_historical arg is ACCESS-CM2_historical );R=$(echo /nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical ++ echo $'/nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical\r' + VAR=$'/nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical\r' echo 'var is '$VAR; ' echo 'var is /nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical var is /nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical ls ${VAR}*.nc; *.nc''/nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical *.nc: No such file or directory35080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical exit 1; + exit 1
正如您在
bash -vx輸出行中看到的+ VAR=$'/nobackup/b9035080/NCdata_RealsPalau/Reals_Palau_tos_Oday_ACCESS-CM2_historical\r'
\r輸入行的末尾有一個輸入 (CR, ) 字元。除非 Unix 換行符 (LF,\n) 未被剝離並成為字元串的一部分，然後當然不再匹配文件系統路徑。