プロセス出力をキャッシュするためのシェルコマンドまたはユーティリティはありますか?

プロセス出力をキャッシュするためのシェルコマンドまたはユーティリティはありますか?

ここが正しい質問であることを願っています。

これに似たUNIXツールはありますか?

# invokes echo, saves to cache (using command with arguments as key), returns "hi"
cache 10 echo hi 

# wait 2 seconds
sleep 2

# doesn't invoke echo, from cache returns "hi"
cache 10 echo hi 

# wait 10 seconds
sleep 10

# with cache expired, invokes echo, returns "hi"
cache 10 echo hi 

明らかに、echoは実際のユースケースではありません。

デフォルトでは、指定されたコマンド+パラメーターのSTDOUT、STDERR、および状態をキャッシュするため、次に同じプロセスが呼び出されたときに再実行する必要はありません。

これを行うためのスクリプトを書くことができますが、Unixツールセットに私が知らないスクリプトがあるかどうか疑問に思います。

ベストアンサー1

私はこれについてかなり完全なスクリプトを書いた。最新バージョンは次のとおりです。https://gist.github.com/akorn/51ee2fe7d36fa139723c851d87e56096

#!/bin/zsh
#
# Purpose: run speficied command with specified arguments and cache result. If cache is fresh enough, don't run command again but return cached output.
# Also cache exit status and stderr.
# License: GPLv3

# Use silly long variable names to avoid clashing with whatever the invoked program might use
RUNCACHED_MAX_AGE=${RUNCACHED_MAX_AGE:-300}
RUNCACHED_IGNORE_ENV=${RUNCACHED_IGNORE_ENV:-0}
RUNCACHED_IGNORE_PWD=${RUNCACHED_IGNORE_PWD:-0}
[[ -n "$HOME" ]] && RUNCACHED_CACHE_DIR=${RUNCACHED_CACHE_DIR:-$HOME/.runcached}
RUNCACHED_CACHE_DIR=${RUNCACHED_CACHE_DIR:-/var/cache/runcached}

function usage() {
    echo "Usage: runcached [--ttl <max cache age>] [--cache-dir <cache directory>]"
    echo "       [--ignore-env] [--ignore-pwd] [--help] [--prune-cache]"
    echo "       [--] command [arg1 [arg2 ...]]"
    echo
    echo "Run 'command' with the specified args and cache stdout, stderr and exit"
    echo "status. If you run the same command again and the cache is fresh, cached"
    echo "data is returned and the command is not actually run."
    echo
    echo "Normally, all exported environment variables as well as the current working"
    echo "directory are included in the cache key. The --ignore options disable this."
    echo "The OLDPWD variable is always ignored."
    echo
    echo "--prune-cache deletes all cache entries older than the maximum age. There is"
    echo "no other mechanism to prevent the cache growing without bounds."
    echo
    echo "The default cache directory is ${RUNCACHED_CACHE_DIR}."
    echo "Maximum cache age defaults to ${RUNCACHED_MAX_AGE}."
    echo
    echo "CAVEATS:"
    echo
    echo "Side effects of 'command' are obviously not cached."
    echo
    echo "There is no cache invalidation logic except cache age (specified in seconds)."
    echo
    echo "If the cache can't be created, the command is run uncached."
    echo
    echo "This script is always silent; any output comes from the invoked command. You"
    echo "may thus not notice errors creating the cache and such."
    echo
    echo "stdout and stderr streams are saved separately. When both are written to a"
    echo "terminal from cache, they will almost certainly be interleaved differently"
    echo "than originally. Ordering of messages within the two streams is preserved."
    exit 0
}

while [[ -n "$1" ]]; do
    case "$1" in
        --ttl)      RUNCACHED_MAX_AGE="$2"; shift 2;;
        --cache-dir)    RUNCACHED_CACHE_DIR="$2"; shift 2;;
        --ignore-env)   RUNCACHED_IGNORE_ENV=1; shift;;
        --ignore-pwd)   RUNCACHED_IGNORE_PWD=1; shift;;
        --prune-cache)  RUNCACHED_PRUNE=1; shift;;
        --help)     usage;;
        --)     shift; break;;
        *)      break;;
    esac
done

zmodload zsh/datetime
zmodload zsh/stat

# This is racy, but the race is harmless; at worst, the program is run uncached 
# because the cache is unusable. Testing for directory existence saves an
# mkdir(1) execution in the common case, improving performance infinitesimally;
# it could matter if runcached is run from inside a tight loop.
# Hide errors so that runcached itself is transparent (doesn't mix new messages 
# into whatever the called program outputs).
[[ -d "$RUNCACHED_CACHE_DIR/." ]] || mkdir -p "$RUNCACHED_CACHE_DIR" >/dev/null 2>/dev/null

((RUNCACHED_PRUNE)) && find "$RUNCACHED_CACHE_DIR/." -maxdepth 1 -type f \! -newermt @$[EPOCHSECONDS-RUNCACHED_MAX_AGE] -delete 2>/dev/null

[[ -n "$@" ]] || exit 0 # if no command specified, exit silently

(
    # Almost(?) nothing uses OLDPWD, but taking it into account potentially reduces cache efficency.
    # Thus, we ignore it for the purpose of coming up with a cache key.
    unset OLDPWD
    ((RUNCACHED_IGNORE_PWD)) && unset PWD
    ((RUNCACHED_IGNORE_ENV)) || env
    echo -E "$@"
) | md5sum | read RUNCACHED_CACHE_KEY RUNCACHED__crap__

# Unfortunately, I couldn't find a less convoluted way of getting rid of an error message when trying to open a logfile in a nonexistent cache directory.
exec {RUNCACHED_temp_stderr} >&2
exec 2>/dev/null
exec {RUNCACHED_LOCK_FD}>>$RUNCACHED_CACHE_DIR/$RUNCACHED_CACHE_KEY.lock
exec 2>&$RUNCACHED_temp_stderr
exec {RUNCACHED_temp_stderr}>&-

# If we can't obtain a lock, we want to run uncached; otherwise
# 'runcached' wouldn't be transparent because it would prevent
# parallel execution of several instances of the same command.
# Locking is necessary to avoid races between the mv(1) command
# below replacing stderr with a newer version and another instance
# of runcached using a newer stdout with the older stderr.
if flock -n $RUNCACHED_LOCK_FD 2>/dev/null; then
    if [[ -f $RUNCACHED_CACHE_DIR/$RUNCACHED_CACHE_KEY.stdout ]]; then
        if [[ $[EPOCHSECONDS-$(zstat +mtime $RUNCACHED_CACHE_DIR/$RUNCACHED_CACHE_KEY.stdout)] -le $RUNCACHED_MAX_AGE ]]; then
            cat $RUNCACHED_CACHE_DIR/$RUNCACHED_CACHE_KEY.stdout &
            cat $RUNCACHED_CACHE_DIR/$RUNCACHED_CACHE_KEY.stderr >&2 &
            wait
            exit $(<$RUNCACHED_CACHE_DIR/$RUNCACHED_CACHE_KEY.exitstatus)
        else
            rm -f $RUNCACHED_CACHE_DIR/$RUNCACHED_CACHE_KEY.{stdout,stderr,exitstatus} 2>/dev/null
        fi
    fi

    # only reached if cache didn't exist or was too old
    if [[ -d $RUNCACHED_CACHE_DIR/. ]]; then
        RUNCACHED_tempdir=$(mktemp -d 2>/dev/null)
        if [[ -d $RUNCACHED_tempdir/. ]]; then
            $@ >&1 >$RUNCACHED_tempdir/$RUNCACHED_CACHE_KEY.stdout 2>&2 2>$RUNCACHED_tempdir/$RUNCACHED_CACHE_KEY.stderr
            RUNCACHED_ret=$?
            echo $RUNCACHED_ret >$RUNCACHED_tempdir/$RUNCACHED_CACHE_KEY.exitstatus 2>/dev/null
            mv $RUNCACHED_tempdir/$RUNCACHED_CACHE_KEY.{stdout,stderr,exitstatus} $RUNCACHED_CACHE_DIR/ 2>/dev/null
            rmdir $RUNCACHED_tempdir 2>/dev/null
            exit $RUNCACHED_ret
        fi
    fi
fi

# only reached if cache not created successfully or lock couldn't be obtained
exec $@

おすすめ記事