Khala-node和kusama-node的服务状态都是stop

中文（简） 节点部署支持
#1

【问题】solo部署正常运行了大半年的worker最近khala-node和kusama-node的服务状态都是stop
【操作复盘】我当时进行了重启phala操作,更新phala操作（phala update），重启服务器操作
然后尝试从其它正常机器上复制了节点数据到问题机器上，复制进程中两台主机的phala都是停止状态，复制完后问题依旧：
不正常主机执行：sudo rm -rf /var/khala/node-data/chains
正常主机执行：sudo scp -r /var/khala/node-data/chains [email protected]:/var/khala/node-data/
【报错信息】
phala-node日志：

image
image1920×387 67 KB

Starting Khala node as role ‘MINER’ with extra parachain args ‘’ extra relaychain args ‘’
2022-06-09 00:29:47 Khala Node
2022-06-09 00:29:47 :v: version 0.1.14-12d76b5
2022-06-09 00:29:47 :heart: by Phala Network, 2018-2022
2022-06-09 00:29:47 :clipboard: Chain specification: Khala
2022-06-09 00:29:47 :label: Node name: node-0001
2022-06-09 00:29:47 :bust_in_silhouette: Role: FULL
2022-06-09 00:29:47 :floppy_disk: Database: RocksDb at /root/data/chains/khala/db/full
2022-06-09 00:29:47 :chains: Native runtime: khala-1140 (khala-0.tx4.au1)
2022-06-09 00:29:47 It isn’t safe to expose RPC publicly without a proxy server that filters available set of RPC methods.
2022-06-09 00:29:47 It isn’t safe to expose RPC publicly without a proxy server that filters available set of RPC methods.
2022-06-09 00:29:47 Parachain id: Id(2004)
2022-06-09 00:29:47 Parachain Account: 5Ec4AhPVjsshXjh8ynp6MwaJTJBnen3pkHiiyDhHfie5VWkN
2022-06-09 00:29:47 Parachain genesis state: 0x000000000000000000000000000000000000000000000000000000000000000000fd2e3e07ed2d610c6c0c6c3cd6858fff733fe53c03bd75d46f25e7c69dec490b03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c11131400
2022-06-09 00:29:47 Is collating: no
2022-06-09 00:29:48 [Relaychain] DB has been previously marked as corrupted, attempting repair

phala-pherry日志：

image
image1789×650 115 KB

0: Rpc error: Networking or low-level protocol error: Error when opening the TCP socket: Connection refused (os error 111)
1: Networking or low-level protocol error: Error when opening the TCP socket: Connection refused (os error 111)
2: Error when opening the TCP socket: Connection refused (os error 111)
[2022-06-09T00:52:55Z INFO pherry] Restarting…
[2022-06-09T00:52:55Z INFO pherry] bridge() exited with error: Connect to substrate

Caused by:
    0: Rpc error: Networking or low-level protocol error: Error when opening the TCP socket: Connection refused (os error 111)
    1: Networking or low-level protocol error: Error when opening the TCP socket: Connection refused (os error 111)
    2: Error when opening the TCP socket: Connection refused (os error 111)

[2022-06-09T00:52:57Z INFO pherry] Restarting…
[2022-06-09T00:52:57Z INFO pherry] bridge() exited with error: Connect to substrate

Caused by:
    0: Rpc error: Networking or low-level protocol error: Error when opening the TCP socket: Connection refused (os error 111)
    1: Networking or low-level protocol error: Error when opening the TCP socket: Connection refused (os error 111)
    2: Error when opening the TCP socket: Connection refused (os error 111)

[2022-06-09T00:52:58Z INFO pherry] bridge() exited with error: Connect to substrate

phala-pruntime日志：

image
image1920×1054 169 KB

Starting pRuntime with extra opts ‘–cores=10’
PRuntime will running in hardware mode
pRuntime starting… It may take up to 2 minutes.
Waiting for device. Sleep 6s
aesm_service: warning: Turn to daemon. Use “–no-daemon” option to execute in foreground.
[2022-06-09T00:30:00.511088Z INFO app] [+] Init Enclave Successful, pid=2!
[2022-06-09T00:30:00.511135Z INFO app] init_args: InitArgs {
sealing_path: “/root/data”,
log_filter: “info”,
init_bench: false,
version: “0.2.4”,
git_revision: “848b954d581c2fa60b1024cc7f3a1322b4ca027f”,
geoip_city_db: “./GeoLite2-City.mmdb”,
enable_checkpoint: true,
checkpoint_interval: 300,
skip_corrupted_checkpoint: true,
gc_interval: 100,
}
WARN: dlsym unable to load symbol “statx”
[2022-06-09T00:30:10.422903Z INFO enclaveapp] Loaded checkpoint
[2022-06-09T00:30:10.423467Z INFO app] Bench cores: 10
[2022-06-09T00:30:10.424553Z INFO enclaveapp] [0] Benchmark thread started
[2022-06-09T00:30:10.424544Z INFO enclaveapp::pal_sgx] Async executor started
[2022-06-09T00:30:10.424654Z INFO enclaveapp::pal_sgx] Async reactor started
[2022-06-09T00:30:10.424795Z INFO enclaveapp] [1] Benchmark thread started
[2022-06-09T00:30:10.424897Z INFO enclaveapp] [3] Benchmark thread started
[2022-06-09T00:30:10.425050Z INFO enclaveapp] [2] Benchmark thread started
[2022-06-09T00:30:10.425100Z INFO enclaveapp] [4] Benchmark thread started
[2022-06-09T00:30:10.425401Z INFO app] Methods under /prpc:
[2022-06-09T00:30:10.425409Z INFO app] /prpc/PhactoryAPI.GetInfo
[2022-06-09T00:30:10.425412Z INFO app] /prpc/PhactoryAPI.SyncHeader
[2022-06-09T00:30:10.425413Z INFO app] /prpc/PhactoryAPI.SyncParaHeader
[2022-06-09T00:30:10.425414Z INFO app] /prpc/PhactoryAPI.SyncCombinedHeaders
[2022-06-09T00:30:10.425415Z INFO app] /prpc/PhactoryAPI.DispatchBlocks
[2022-06-09T00:30:10.425416Z INFO app] /prpc/PhactoryAPI.InitRuntime
[2022-06-09T00:30:10.425417Z INFO app] /prpc/PhactoryAPI.GetRuntimeInfo
[2022-06-09T00:30:10.425418Z INFO app] /prpc/PhactoryAPI.GetEgressMessages
[2022-06-09T00:30:10.425419Z INFO app] /prpc/PhactoryAPI.ContractQuery
[2022-06-09T00:30:10.425420Z INFO app] /prpc/PhactoryAPI.GetWorkerState
[2022-06-09T00:30:10.425421Z INFO app] /prpc/PhactoryAPI.Echo
[2022-06-09T00:30:10.425459Z WARN rocket::config::config] :wrench: Configured for debug.
[2022-06-09T00:30:10.425464Z WARN rocket::config::config] address: 0.0.0.0
[2022-06-09T00:30:10.425467Z WARN rocket::config::config] port: 8000
[2022-06-09T00:30:10.425469Z WARN rocket::config::config] workers: 1
[2022-06-09T00:30:10.425471Z WARN rocket::config::config] ident: Rocket
[2022-06-09T00:30:10.425473Z WARN rocket::config::config] keep-alive: 5s
[2022-06-09T00:30:10.425475Z WARN rocket::config::config] limits: bytes = 100MiB, data-form = 2MiB, file = 1MiB, form = 32KiB, json = 100MiB, msgpack = 1MiB, string = 8KiB
[2022-06-09T00:30:10.425481Z WARN rocket::config::config] tls: disabled
[2022-06-09T00:30:10.425483Z WARN rocket::config::config] temp dir: /tmp
[2022-06-09T00:30:10.425485Z WARN rocket::config::config] log level: critical
[2022-06-09T00:30:10.425487Z WARN rocket::config::config] cli colors: true
[2022-06-09T00:30:10.425488Z WARN rocket::config::config] shutdown: ctrlc = true, force = true, signals = [SIGTERM], grace = 2s, mercy = 3s
[2022-06-09T00:30:10.425507Z WARN rocket::rocket] :artificial_satellite: Routes:
[2022-06-09T00:30:10.425516Z WARN rocket::rocket] (get_info) GET /get_info
[2022-06-09T00:30:10.425520Z WARN rocket::rocket] (get_info_post) POST /get_info application/json
[2022-06-09T00:30:10.425523Z WARN rocket::rocket] (prpc_proxy) POST /prpc/
[2022-06-09T00:30:10.425527Z WARN rocket::rocket] (sync_header) POST /bin_api/sync_header
[2022-06-09T00:30:10.425529Z WARN rocket::rocket] (dispatch_block) POST /bin_api/dispatch_block
[2022-06-09T00:30:10.425531Z WARN rocket::rocket] (sync_para_header) POST /bin_api/sync_para_header
[2022-06-09T00:30:10.425535Z WARN rocket::rocket] (sync_combined_headers) POST /bin_api/sync_combined_headers
[2022-06-09T00:30:10.425539Z WARN rocket::fairing::fairings] :satellite: Fairings:
[2022-06-09T00:30:10.425541Z WARN rocket::fairing::fairings] Shield (liftoff, response, singleton)
[2022-06-09T00:30:10.425576Z INFO rocket::shield::shield] :shield: Shield:
[2022-06-09T00:30:10.425579Z INFO rocket::shield::shield] X-Frame-Options: SAMEORIGIN
[2022-06-09T00:30:10.425581Z INFO rocket::shield::shield] Permissions-Policy: interest-cohort=()
[2022-06-09T00:30:10.425582Z INFO rocket::shield::shield] X-Content-Type-Options: nosniff
[2022-06-09T00:30:10.425585Z WARN rocket::rocket] :rocket: Rocket has launched from http://0.0.0.0:8000
[2022-06-09T00:30:10.425731Z INFO enclaveapp] [6] Benchmark thread started
[2022-06-09T00:30:10.425739Z INFO enclaveapp] [8] Benchmark thread started
[2022-06-09T00:30:10.425763Z INFO enclaveapp] [5] Benchmark thread started
[2022-06-09T00:30:10.425767Z INFO enclaveapp] [7] Benchmark thread started
[2022-06-09T00:30:10.425770Z INFO enclaveapp] [9] Benchmark thread started

【补充说明】无

#2
#3
#4
#5
#6
#7
#8
#9
#10
#11
#12
#13
#14
#15
#16
#17
#18
#19