クラスタヘッドノードでlinpackを実行できません。

Question

ついに問題を見つけて解決しました。

私はrpislave1 2つのIPアドレス（192.168.50.1（rpislave1に設定したDHCPサーバーから来たもの）と別の169.254.131.47）にeth0を提供していることを発見しました。これにより、Open MPIがmpirunアドレスの間に混乱しました。問題は - プロセスとは何ですか？ rpislave1は、すべてのRaspberry Piの内部ネットワーク（範囲192.168.50.0/24）と外部世界への接続用のDHCPサーバーなので、net.ipv4.ip_forward = 1でiptablesだと思いました。いいえ、いくつかのテストを行いました。まったく効果がありません。

次に、ラズベリーパイが同じネットワークインターフェイスに2つのIPアドレスを持つ理由を調べ、dhcpcd.serviceを見つけました。

user@rpislave1:/etc $ sudo systemctl status  dhcpcd.service
● dhcpcd.service - DHCP Client Daemon
     Loaded: loaded (/lib/systemd/system/dhcpcd.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/dhcpcd.service.d
             └─wait.conf
     Active: active (running) since Fri 2023-07-28 16:22:45 +08; 16min ago
       Docs: man:dhcpcd(8)
    Process: 473 ExecStart=/usr/sbin/dhcpcd -w -q (code=exited, status=0/SUCCESS)
      Tasks: 2 (limit: 8755)
        CPU: 685ms
     CGroup: /system.slice/dhcpcd.service
             ├─564 wpa_supplicant -B -c/etc/wpa_supplicant/wpa_supplicant.conf -iwlan0
             └─765 dhcpcd: [master] [ip4] [ip6]

Jul 28 16:22:42 rpislave1 dhcpcd[473]: eth0: probing for an IPv4LL address
Jul 28 16:22:45 rpislave1 dhcpcd[473]: forked to background, child pid 765
Jul 28 16:22:45 rpislave1 systemd[1]: Started DHCP Client Daemon.
Jul 28 16:22:45 rpislave1 dhcpcd[765]: wlan0: leased 192.168.101.15 for 259200 seconds
Jul 28 16:22:45 rpislave1 dhcpcd[765]: wlan0: adding route to 192.168.101.0/24
Jul 28 16:22:45 rpislave1 dhcpcd[765]: wlan0: adding default route via 192.168.101.1
Jul 28 16:22:47 rpislave1 dhcpcd[765]: eth0: using IPv4LL address 169.254.131.47
Jul 28 16:22:47 rpislave1 dhcpcd[765]: eth0: adding route to 169.254.0.0/16
Jul 28 16:22:50 rpislave1 dhcpcd[765]: eth0: offered 192.168.50.1 from 192.168.50.1
Jul 28 16:22:51 rpislave1 dhcpcd[765]: eth0: no IPv6 Routers available

DHCP要求を送信する前にIPv4LLを割り当てたサービスを見つけます（ただし、同じデバイスにあります）。このIPv4LLは、同じサーバー上で稼働しているDHCPサーバーからIPアドレスを取得する前に、IPアドレスとパスを設定します。しかし、169.254.0.0/16へのパスは削除されないようです（rpislave.localで再利用されると疑われますが、これは私の目的には関係ありません）。

dhcpcd.serviceを見て、このオプションが見つかりました。

Local Link configuration
If dhcpcd failed to obtain a lease, it probes for a valid IPv4LL address (aka ZeroConf, aka APIPA). Once obtained it restarts the process of looking for a DHCP server to get a proper address.

When using IPv4LL, dhcpcd nearly always succeeds and returns an exit code of 0. In the rare case it fails, it normally means that there is a reverse ARP proxy installed which always defeats IPv4LL probing. To disable this behaviour, you can use the -L, --noipv4ll option.

したがって、/etc/systemd/system/dhcpcd.service.d の wait.conf ファイルが編集されました。これらすべての詳細は、上記のステータスコードにあります。

[Service]
ExecStart=
ExecStart=/usr/sbin/dhcpcd -w -q --noipv4ll

再起動してdhcpcdをもう一度確認してください。

user@rpislave1:/etc/systemd/system/dhcpcd.service.d $ sudo systemctl status  dhcpcd.service
● dhcpcd.service - DHCP Client Daemon
     Loaded: loaded (/lib/systemd/system/dhcpcd.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/dhcpcd.service.d
             └─wait.conf
     Active: active (running) since Fri 2023-07-28 16:44:15 +08; 3min 17s ago
       Docs: man:dhcpcd(8)
    Process: 472 ExecStart=/usr/sbin/dhcpcd -w -q --noipv4ll (code=exited, status=0/SUCCESS)
      Tasks: 2 (limit: 8755)
        CPU: 619ms
     CGroup: /system.slice/dhcpcd.service
             ├─568 wpa_supplicant -B -c/etc/wpa_supplicant/wpa_supplicant.conf -iwlan0
             └─775 dhcpcd: [master] [ip4] [ip6]

Jul 28 16:44:10 rpislave1 dhcpcd[472]: wlan0: adding route to 2001:f40:950:b164::/64
Jul 28 16:44:10 rpislave1 dhcpcd[472]: wlan0: requesting DHCPv6 information
Jul 28 16:44:10 rpislave1 dhcpcd[472]: wlan0: adding default route via fe80::101
Jul 28 16:44:15 rpislave1 dhcpcd[472]: wlan0: leased 192.168.101.15 for 259200 seconds
Jul 28 16:44:15 rpislave1 dhcpcd[472]: wlan0: adding route to 192.168.101.0/24
Jul 28 16:44:15 rpislave1 dhcpcd[472]: wlan0: adding default route via 192.168.101.1
Jul 28 16:44:15 rpislave1 dhcpcd[472]: forked to background, child pid 775
Jul 28 16:44:15 rpislave1 systemd[1]: Started DHCP Client Daemon.
Jul 28 16:44:19 rpislave1 dhcpcd[775]: eth0: offered 192.168.50.1 from 192.168.50.1
Jul 28 16:44:21 rpislave1 dhcpcd[775]: eth0: no IPv6 Routers available
user@rpislave1:/etc/systemd/system/dhcpcd.service.d $

そして、「eth0：IPv4LLアドレス169.254.131.47を使用」のパスはまったく設定されていません。

それで、クラスタ全体をテストし始めました。

user@rpislave1:/sharezone/ClusterProcessing/HPL $  time mpirun -hostfile ../host_file -np 16 /sharezone/xhpl/bin/xhpl |tee HPL.log
================================================================================
HPLinpack 2.3  --  High-Performance Linpack benchmark  --   December 2, 2018
Written by A. Petitet and R. Clint Whaley,  Innovative Computing Laboratory, UTK
Modified by Piotr Luszczek, Innovative Computing Laboratory, UTK
Modified by Julien Langou, University of Colorado Denver
================================================================================

An explanation of the input/output parameters follows:
T/V    : Wall time / encoded variant.
N      : The order of the coefficient matrix A.
NB     : The partitioning blocking factor.
P      : The number of process rows.
Q      : The number of process columns.
Time   : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.

The following parameter values will be used:

N      :   57600
NB     :     192
PMAP   : Row-major process mapping
P      :       4
Q      :       4
PFACT  :   Right
NBMIN  :       4
NDIV   :       2
RFACT  :   Crout
BCAST  :  1ringM
DEPTH  :       1
SWAP   : Mix (threshold = 64)
L1     : transposed form
U      : transposed form
EQUIL  : yes
ALIGN  : 8 double precision words

--------------------------------------------------------------------------------

- The matrix A is randomly generated for each test.
- The following scaled residual check will be computed:
      ||Ax-b||_oo / ( eps * ( || x ||_oo * || A ||_oo + || b ||_oo ) * N )
- The relative machine precision (eps) is taken to be               1.110223e-16
- Computational tests pass if scaled residuals are less than                16.0

================================================================================
T/V                N    NB     P     Q               Time                 Gflops
--------------------------------------------------------------------------------
WR11C2R4       57600   192     4     4            2558.14             4.9804e+01
HPL_pdgesv() start time Fri Jul 28 16:55:04 2023

HPL_pdgesv() end time   Fri Jul 28 17:37:42 2023

--------------------------------------------------------------------------------
||Ax-b||_oo/(eps*(||A||_oo*||x||_oo+||b||_oo)*N)=   3.39874554e-03 ...... PASSED
================================================================================

Finished      1 tests with the following results:
              1 tests completed and passed residual checks,
              0 tests completed and failed residual checks,
              0 tests skipped because of illegal input values.
--------------------------------------------------------------------------------

End of Tests.
================================================================================
[rpislave4:00682] PMIX ERROR: NO-PERMISSIONS in file ../../../../../../src/mca/common/dstore/dstore_base.c at line 237
[rpislave3:00688] PMIX ERROR: NO-PERMISSIONS in file ../../../../../../src/mca/common/dstore/dstore_base.c at line 237
[rpislave2:00696] PMIX ERROR: NO-PERMISSIONS in file ../../../../../../src/mca/common/dstore/dstore_base.c at line 237

real    43m33.445s
user    141m36.616s
sys     28m21.561s
user@rpislave1:/sharezone/ClusterProcessing/HPL $

そして効果がありますよ〜！これで、ヘッドを含む4つのユニットすべてに対して全体のベンチマークができるようになりました〜！

Answer 1