bashで複数の区切り文字を使用してテキストファイルをCSVに分割しますか?

bashで複数の区切り文字を使用してテキストファイルをCSVに分割しますか?

テキストファイルをCSVに解析してみてください。問題は、現在理想的には列ヘッダーとして使用したいが、csv結果から削除できるいくつかの区切り文字があることです。理想的にはbashを使用することをお勧めしますが、動作するものは何でもMac OSシステムで実行してください。

Sample text (DISA STIG)


 ----------
Group ID (Vulid): V-81749
Group Title: SRG-OS-000067-GPOS-00035
Rule ID: SV-96463r1_rule
Severity: CAT II
Rule Version (STIG-ID): AOSX-13-067035
Rule Title: The macOS system must enable certificate for smartcards.
_
_

 Vulnerability Discussion: To prevent untrusted certificates the certificates on a smartcard card must be valid in these ways: its issuer is system-trusted, the certificate is not expired, its "valid-after" date is in the past, and it passes CRL and OCSP checking.
Check Content:
To view the setting for the smartcard certification configuration, run the following command:
sudo /usr/sbin/system_profiler SPConfigurationProfileDataType | /usr/bin/grep checkCertificateTrust
If the output is null or not "checkCertificateTrust = 1;" this is a finding.
Fix Text: This setting is enforced using the "Smartcard" configuration profile.
CCI: CCI-000186 ___________________________________________________
<Break>

----------

基本的に、次の列のCSVを分割したいと思います。

Group ID (Vulid)
Group Title:
Rule ID:
Severity:
Rule Version (STIG-ID):
Rule Title:
Vulnerability Discussion:
Check Content:
Fix Text:
CCI:

区切り文字は<Break>次の行に移動します。

私の列が次のように表示されることを願っています。

    Group ID (Vulid)    Group Title:    Rule ID:    Severity:   Rule Version (STIG-ID)  Rule Title: Vulnerability Discussion    Check Content   Fix Text:   CCI:    CCI:

各ヘッダーを削除し、任意の区切り文字に置き換えてから、awkを使用して分割する最善の方法は何ですか?このような複数の基準に分割しようとしたことがないため、これを処理する最善の方法を理解するのは少し難しいです。

ベストアンサー1

回答

まず、ファイルをクリーンアップして、PostScriptのように統一されているように見えるようにする必要があります。505942.txt。元のファイルとその複雑さだけを知り、簡単なsedコマンドだけをGoogleで検索すると簡単に知ることができるので、この作業はあなたに任せます。標準外の一部の行に対して特定のコマンドを作成する必要がある場合があり、sed大きな問題ではない場合は手動で変更する必要があります(たとえば、単純な文字を削除するために5〜6行のスクリプトを作成しません)。

文字列で作業するときは、作業を単純な作業に分割する必要があります。与えられたファイルをカンマ区切り値ファイル(CSV)に変換する方法の例を挙げました。最終ファイルは505942.csvこれは私のPostScriptでもあります。

sed -i 's/^.*\(: \)/\1/g' 505942.txt # Use '-i' for editing files in place (in the file itself). replace everything until the first colon ':' excluding, in other words, remove the headers from each line.
sed -i 's/^\(: \)//g' 505942.txt # Remove the first colon and the subsequent white space.
sed -i 's/^/"/' 505942.txt # Add double quotes in the beginning of each line. Quotes whill help you to parse the final comma seperated value file, since some of the fields seem to already contain commas.
sed -i 's/$/",/' 505942.txt # Add double quotes in the end of each line.
cat 505942.txt | xargs -n10 -d'\n' > 505942-after-xargs.txt # Join every 10 lines of the file.
sed -i 's/,$//' 505942-after-xargs.txt # Remove the last comma from each line.

sed -n 1,10p 505942.txt > 505942-headers.txt # Keep the first 10 lines from which you will extract the headers.
sed -i 's/:.*//' 505942-headers.txt # Remove everything after (including) the first colon.
sed -i 's/^/"/' 505942-headers.txt # Similar to above command.
sed -i 's/$/",/' 505942-headers.txt # Similar to above command.
cat 505942-headers.txt | xargs -n10 -d'\n' > 505942-headers-after-xargs.txt # Similar to above command.
sed -i 's/,$//' 505942-headers-after-xargs.txt # Similar to above command.

cat 505942-after-xargs.txt >> 505942-headers-after-xargs.txt # Join the files; append to the header file.

cat 505942-headers-after-xargs.txt # Check everything seems fine.
cp 505942-headers-after-xargs.txt 505942.csv # Copy to the final .csv file.

PS

コンテンツ505942.txt:

Group Title: SRG-OS-000067-GPOS-00035
Rule ID: SV-96463r1_rule
Severity: CAT II
Rule Version (STIG-ID): AOSX-13-067035
Rule Title: The macOS system must enable certificate for smartcards.
Vulnerability Discussion: To prevent untrusted certificates the certificates on a smartcard card must be valid in these ways: its issuer is system-trusted, the certificate is not expired, its "valid-after" date is in the past, and it passes CRL and OCSP checking.
Check Content: To view the setting for the smartcard certification configuration, run the following command: sudo /usr/sbin/system_profiler SPConfigurationProfileDataType | /usr/bin/grep checkCertificateTrust If the output is null or not "checkCertificateTrust = 1;" this is a finding.
Fix Text: This setting is enforced using the "Smartcard" configuration profile.
CCI: CCI-000186
Group ID (Vulid): V-81749
Group Title: SRG-OS-000067-GPOS-00035
Rule ID: SV-96463r1_rule
Severity: CAT II
Rule Version (STIG-ID): AOSX-13-067035
Rule Title: The macOS system must enable certificate for smartcards.
Vulnerability Discussion: To prevent untrusted certificates the certificates on a smartcard card must be valid in these ways: its issuer is system-trusted, the certificate is not expired, its "valid-after" date is in the past, and it passes CRL and OCSP checking.
Check Content: To view the setting for the smartcard certification configuration, run the following command: sudo /usr/sbin/system_profiler SPConfigurationProfileDataType | /usr/bin/grep checkCertificateTrust If the output is null or not "checkCertificateTrust = 1;" this is a finding.
Fix Text: This setting is enforced using the "Smartcard" configuration profile.
CCI: CCI-000186

コンテンツ505942.csv:

"Group ID (Vulid)", "Group Title", "Rule ID", "Severity", "Rule Version (STIG-ID)", "Rule Title", "Vulnerability Discussion", "Check Content", "Fix Text", "CCI"
"V-81749", "SRG-OS-000067-GPOS-00035", "SV-96463r1_rule", "CAT II", "AOSX-13-067035", "The macOS system must enable certificate for smartcards.", "its issuer is system-trusted, the certificate is not expired, its "valid-after" date is in the past, and it passes CRL and OCSP checking.", "sudo /usr/sbin/system_profiler SPConfigurationProfileDataType | /usr/bin/grep checkCertificateTrust If the output is null or not "checkCertificateTrust = 1;" this is a finding.", "This setting is enforced using the "Smartcard" configuration profile.", "CCI-000186"
"V-81749", "SRG-OS-000067-GPOS-00035", "SV-96463r1_rule", "CAT II", "AOSX-13-067035", "The macOS system must enable certificate for smartcards.", "its issuer is system-trusted, the certificate is not expired, its "valid-after" date is in the past, and it passes CRL and OCSP checking.", "sudo /usr/sbin/system_profiler SPConfigurationProfileDataType | /usr/bin/grep checkCertificateTrust If the output is null or not "checkCertificateTrust = 1;" this is a finding.", "This setting is enforced using the "Smartcard" configuration profile.", "CCI-000186"

おすすめ記事