すでに実行されているnohupプロセスに新しいコマンドを送信する方法、またはnohupで2つのコマンドを一緒に/同時に実行する方法は？

Question

理由はわかりませんが、ここでは代替案が機能しないようです。tty / stty（ターミナル）ではなく、Expect、sshpassがない場合、どのようにコマンドにパスワードを送信しますか？

3つの試みがすべて成功したようです。

参照されたshスクリプトでreauthを実行する
Pythonで認証を実行する
上記の両方を実行してください。

ジョブの実行に使用するサンプルスクリプト：

# https://unix.stackexchange.com/questions/724902/how-does-one-send-new-commands-to-run-to-an-already-running-nohup-process-e-g-r
# sh ~/diversity-for-predictive-success-of-meta-learning/main_nohup_snap.sh
# - set up this main sh script
export RUN_PWD=$(pwd)
source ~/.bashrc
source ~/.bash_profile
source ~/.bashrc.user
echo HOME = $HOME
# since snap .bash.user cd's me into HOME at dfs
cd $RUN_PWD
echo RUN_PWD = $RUN_PWD
realpath .

source cuda11.1

conda init bash
conda activate metalearning_gpu

# - get a job id for this tmux session
export SLURM_JOBID=$(python -c "import random;print(random.randint(0, 1_000_000))")
echo SLURM_JOBID = $SLURM_JOBID
export OUT_FILE=$PWD/main.sh.o$SLURM_JOBID
export ERR_FILE=$PWD/main.sh.e$SLURM_JOBID
export WANDB_DIR=$HOME/wandb_dir
echo $OUT_FILE
echo $ERR_FILE

export CUDA_VISIBLE_DEVICES=5
echo CUDA_VISIBLE_DEVICES = $CUDA_VISIBLE_DEVICES
python -c "import torch; print(torch.cuda.get_device_name(0));"

# - CAREFUL, if a job is already running it could do damage to it, rm reauth process, qian doesn't do it so skip it
# top -u brando9
# pkill -9 reauth -u brando9

# - expt python script then inside that python pid attach a reauth process
# should I run rauth within python with subprocess or package both the nohup command and the rauth together in badsh somehow
#python -u ~/diversity-for-predictive-success-of-meta-learning/div_src/diversity_src/experiment_mains/main_sl_with_ddp.py --manual_loads_name sl_hdb1_5cnn_adam_cl_filter_size --filter_size 4 > $OUT_FILE 2> $ERR_FILE &
nohup sh -c 'echo $SU_PASSWORD | /afs/cs/software/bin/reauth; python -u ~/diversity-for-predictive-success-of-meta-learning/div_src/diversity_src/experiment_mains/main_sl_with_ddp.py --manual_loads_name sl_hdb1_5cnn_adam_cl_filter_size --filter_size 4 > $OUT_FILE 2> $ERR_FILE' > $PWD/nohup.out$SLURM_JOBID &
#nohup python -u ~/diversity-for-predictive-success-of-meta-learning/div_src/diversity_src/experiment_mains/main_sl_with_ddp.py --manual_loads_name sl_hdb1_5cnn_adam_cl_filter_size --filter_size 4 > $OUT_FILE 2> $ERR_FILE &

# other option is to run `echo $SU_PASSWORD | /afs/cs/software/bin/reauth` inside of python, right?
export JOB_PID=$!
echo JOB_PID = $JOB_PID
echo SLURM_JOBID = $SLURM_JOBID

# - Done
echo "Done with the dispatching (daemon) sh script"

主なコマンド：

nohup sh -c 'echo $SU_PASSWORD | /afs/cs/software/bin/reauth; python -u ~/diversity-for-predictive-success-of-meta-learning/div_src/diversity_src/experiment_mains/main_sl_with_ddp.py --manual_loads_name sl_hdb1_5cnn_adam_cl_filter_size --filter_size 4 > $OUT_FILE 2> $ERR_FILE' > $PWD/nohup.out$SLURM_JOBID &

私はオプション3を使い続けると思いますので、両方を実行してください。なぜなら、分散ジョブを実行しても、そのプロセスが私の許可なしにランダムに終了したくないからです。これを行うには、Pythonスクリプトは次のことを行います。

def run_bash_command(cmd: str) -> Any:
    import subprocess

    process = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
    output, error = process.communicate()
    if error:
        raise Exception(error)
    else:
        return output

def stanford_reauth():
    """"
    re-authenticates the python process in the kerberos system so that the
    python process is not killed randomly.

    ref: https://unix.stackexchange.com/questions/724902/how-does-one-send-new-commands-to-run-to-an-already-running-nohup-process-or-run
    """
    reauth_cmd: str = f'echo $SU_PASSWORD | /afs/cs/software/bin/reauth'
    out = run_bash_command(reauth_cmd)
    print('Output of reauth (/afs/cs/software/bin/reauth with password): ')
    print(f'{out=}')

再認証作業を頻繁に確認することを忘れないでください。

# - CAREFUL, if a job is already running it could do damage to it, rm reauth process
# pkill -9 reauth -u brando9

Answer 1

理由はわかりませんが、ここでは代替案が機能しないようです。tty / stty（ターミナル）ではなく、Expect、sshpassがない場合、どのようにコマンドにパスワードを送信しますか？

3つの試みがすべて成功したようです。

参照されたshスクリプトでreauthを実行する
Pythonで認証を実行する
上記の両方を実行してください。

ジョブの実行に使用するサンプルスクリプト：

# https://unix.stackexchange.com/questions/724902/how-does-one-send-new-commands-to-run-to-an-already-running-nohup-process-e-g-r
# sh ~/diversity-for-predictive-success-of-meta-learning/main_nohup_snap.sh
# - set up this main sh script
export RUN_PWD=$(pwd)
source ~/.bashrc
source ~/.bash_profile
source ~/.bashrc.user
echo HOME = $HOME
# since snap .bash.user cd's me into HOME at dfs
cd $RUN_PWD
echo RUN_PWD = $RUN_PWD
realpath .

source cuda11.1

conda init bash
conda activate metalearning_gpu

# - get a job id for this tmux session
export SLURM_JOBID=$(python -c "import random;print(random.randint(0, 1_000_000))")
echo SLURM_JOBID = $SLURM_JOBID
export OUT_FILE=$PWD/main.sh.o$SLURM_JOBID
export ERR_FILE=$PWD/main.sh.e$SLURM_JOBID
export WANDB_DIR=$HOME/wandb_dir
echo $OUT_FILE
echo $ERR_FILE

export CUDA_VISIBLE_DEVICES=5
echo CUDA_VISIBLE_DEVICES = $CUDA_VISIBLE_DEVICES
python -c "import torch; print(torch.cuda.get_device_name(0));"

# - CAREFUL, if a job is already running it could do damage to it, rm reauth process, qian doesn't do it so skip it
# top -u brando9
# pkill -9 reauth -u brando9

# - expt python script then inside that python pid attach a reauth process
# should I run rauth within python with subprocess or package both the nohup command and the rauth together in badsh somehow
#python -u ~/diversity-for-predictive-success-of-meta-learning/div_src/diversity_src/experiment_mains/main_sl_with_ddp.py --manual_loads_name sl_hdb1_5cnn_adam_cl_filter_size --filter_size 4 > $OUT_FILE 2> $ERR_FILE &
nohup sh -c 'echo $SU_PASSWORD | /afs/cs/software/bin/reauth; python -u ~/diversity-for-predictive-success-of-meta-learning/div_src/diversity_src/experiment_mains/main_sl_with_ddp.py --manual_loads_name sl_hdb1_5cnn_adam_cl_filter_size --filter_size 4 > $OUT_FILE 2> $ERR_FILE' > $PWD/nohup.out$SLURM_JOBID &
#nohup python -u ~/diversity-for-predictive-success-of-meta-learning/div_src/diversity_src/experiment_mains/main_sl_with_ddp.py --manual_loads_name sl_hdb1_5cnn_adam_cl_filter_size --filter_size 4 > $OUT_FILE 2> $ERR_FILE &

# other option is to run `echo $SU_PASSWORD | /afs/cs/software/bin/reauth` inside of python, right?
export JOB_PID=$!
echo JOB_PID = $JOB_PID
echo SLURM_JOBID = $SLURM_JOBID

# - Done
echo "Done with the dispatching (daemon) sh script"

主なコマンド：

nohup sh -c 'echo $SU_PASSWORD | /afs/cs/software/bin/reauth; python -u ~/diversity-for-predictive-success-of-meta-learning/div_src/diversity_src/experiment_mains/main_sl_with_ddp.py --manual_loads_name sl_hdb1_5cnn_adam_cl_filter_size --filter_size 4 > $OUT_FILE 2> $ERR_FILE' > $PWD/nohup.out$SLURM_JOBID &

私はオプション3を使い続けると思いますので、両方を実行してください。なぜなら、分散ジョブを実行しても、そのプロセスが私の許可なしにランダムに終了したくないからです。これを行うには、Pythonスクリプトは次のことを行います。

def run_bash_command(cmd: str) -> Any:
    import subprocess

    process = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
    output, error = process.communicate()
    if error:
        raise Exception(error)
    else:
        return output

def stanford_reauth():
    """"
    re-authenticates the python process in the kerberos system so that the
    python process is not killed randomly.

    ref: https://unix.stackexchange.com/questions/724902/how-does-one-send-new-commands-to-run-to-an-already-running-nohup-process-or-run
    """
    reauth_cmd: str = f'echo $SU_PASSWORD | /afs/cs/software/bin/reauth'
    out = run_bash_command(reauth_cmd)
    print('Output of reauth (/afs/cs/software/bin/reauth with password): ')
    print(f'{out=}')

再認証作業を頻繁に確認することを忘れないでください。

# - CAREFUL, if a job is already running it could do damage to it, rm reauth process
# pkill -9 reauth -u brando9

すでに実行されているnohupプロセスに新しいコマンドを送信する方法、またはnohupで2つのコマンドを一緒に/同時に実行する方法は？

ベストアンサー1

おすすめ記事