bashで複数のjsonファイルを1つにマージする方法

bashで複数のjsonファイルを1つにマージする方法

次のような何千ものjsonファイルがあります。

ファイル1(key1:value_list1

{"2mac:acg":["1-248","3-245","3-246","4-245","4-246","5-245","5-246","6-243","6-245","6-246","6-247","6-296","7-245","7-295","7-296","8-236","8-239","8-240","8-294","8-295","8-296","9-235","9-236","9-239","9-294","10-293","10-294","10-295","11-15","11-16","11-293","11-294","12-16","12-290","12-291","12-292","12-293","12-294","13-25","13-26","13-27","13-28","13-290","13-292","13-293","14-24","14-25","14-26","14-27","14-290","15-24","15-25","16-24","16-25","16-233","16-234","16-235","17-22","17-23","17-24","17-25","17-59","17-233","17-234","17-235","18-22","18-23","18-24","18-25","18-43","18-213","18-214","18-215","18-229","18-230","18-232","18-233","18-234","19-42","19-43"]}

ファイル2(key2:value_list2

{"4qld:aaa":["3-245","3-246","4-245","4-246","5-245","5-246","6-243","6-245","6-246","6-247","6-296","7-245","7-295","7-296","8-236","8-239","8-240","8-294","8-295","8-296","9-235","9-236","9-239","9-294","10-293","10-294","10-295","11-15","11-16","11-293","11-294","12-16","12-290","12-291","12-292","12-293","12-294","13-25","13-26","13-27","13-28","13-290","13-292","13-293","14-24","14-25","14-26","14-27","14-290","15-24","15-25","16-24","16-25","16-233","16-234","16-235","17-22","17-23","17-24","17-25","17-59","17-233","17-234","17-235","18-22","18-23","18-24","18-25","18-43","18-213","18-214","18-215","18-229","18-230","18-232","18-233","18-234","19-42","19-43","19-55"]}

ファイル3(key3:value_list3

{"6k8h:c":["1-248","2-134","3-245","3-246","4-245","4-246","5-245","5-246","6-243","6-245","6-246","6-247","6-296","7-245","7-295","7-296","8-236","8-239","8-240","8-294","8-295","8-296","9-235","9-236","9-239","9-294","10-293","10-294","10-295","11-15","11-16","11-293","11-294","12-16","12-290","12-291","12-292","12-293","12-294","13-25","13-26","13-27","13-28","13-290","13-292","13-293","14-24","14-25","14-26","14-27","14-290","15-24","15-25","16-24","16-25","16-233","16-234","16-235","17-22","17-23","17-24","17-25","17-59","17-233","17-234","17-235","18-22","18-23","18-24","18-25","18-43","18-213","18-214","18-215","18-229","18-230","18-232","18-233","18-234","19-42","19-43"]}

これらのファイルを1つにマージしたいです。次のようにする必要があります。

{"2mac:acg":["1-248","3-245","3-246","4-245","4-246","5-245","5-246","6-243","6-245","6-246","6-247","6-296","7-245","7-295","7-296","8-236","8-239","8-240","8-294","8-295","8-296","9-235","9-236","9-239","9-294","10-293","10-294","10-295","11-15","11-16","11-293","11-294","12-16","12-290","12-291","12-292","12-293","12-294","13-25","13-26","13-27","13-28","13-290","13-292","13-293","14-24","14-25","14-26","14-27","14-290","15-24","15-25","16-24","16-25","16-233","16-234","16-235","17-22","17-23","17-24","17-25","17-59","17-233","17-234","17-235","18-22","18-23","18-24","18-25","18-43","18-213","18-214","18-215","18-229","18-230","18-232","18-233","18-234","19-42","19-43"], "4qld:aaa":["3-245","3-246","4-245","4-246","5-245","5-246","6-243","6-245","6-246","6-247","6-296","7-245","7-295","7-296","8-236","8-239","8-240","8-294","8-295","8-296","9-235","9-236","9-239","9-294","10-293","10-294","10-295","11-15","11-16","11-293","11-294","12-16","12-290","12-291","12-292","12-293","12-294","13-25","13-26","13-27","13-28","13-290","13-292","13-293","14-24","14-25","14-26","14-27","14-290","15-24","15-25","16-24","16-25","16-233","16-234","16-235","17-22","17-23","17-24","17-25","17-59","17-233","17-234","17-235","18-22","18-23","18-24","18-25","18-43","18-213","18-214","18-215","18-229","18-230","18-232","18-233","18-234","19-42","19-43","19-55"], "6k8h:c":["1-248","2-134","3-245","3-246","4-245","4-246","5-245","5-246","6-243","6-245","6-246","6-247","6-296","7-245","7-295","7-296","8-236","8-239","8-240","8-294","8-295","8-296","9-235","9-236","9-239","9-294","10-293","10-294","10-295","11-15","11-16","11-293","11-294","12-16","12-290","12-291","12-292","12-293","12-294","13-25","13-26","13-27","13-28","13-290","13-292","13-293","14-24","14-25","14-26","14-27","14-290","15-24","15-25","16-24","16-25","16-233","16-234","16-235","17-22","17-23","17-24","17-25","17-59","17-233","17-234","17-235","18-22","18-23","18-24","18-25","18-43","18-213","18-214","18-215","18-229","18-230","18-232","18-233","18-234","19-42","19-43"]}

接続モデルは次のようになります。{key1:value_list_1, key2:value_list2, key3:value_list3}

私はjqを使って接続しますjq -s 'reduce .[] as $item ({}; . * $item)' file1 file2 file3。結果が得られますが、ファイルが何百もの場合、一方では時間がかかります。一方、(グループに属していない)サーバーにはjqがなく、ダウンロードする権限がありません。もしそうなら、他の処理方法はないかと思います。ありがとうございます!

ベストアンサー1

シンボルを使用すると、jq次のようにできます。

jq -s 'add' file1 file2 file3

上記の方法が十分に高速でない場合、またはJSONプロセッサを使用できない場合、およびJSONが非常に単純な場合は、ファイルの最後のファイルを最後のファイル以外のすべてのファイルに置き換え、最初のファイル },空の{文字列に置き換えることができます。たとえば、最初のファイルを除くすべてのファイル。 JSONは単純なテキストではないので、これは安全ではないことがわかります。

おすすめ記事