Panic invalid memory address after reconnect a node

[davidroman0O]

Describe the bug
Using the example genstage, change it to have two nodes, connect one to the other, disconnect, reconnect and panic

It kills the producer process directly, the entire node does NOT crash

• Open two terminals
• First one: go run ./producer
• Second one: go run ./consumer
• They should communicate and trigger events
• CTRL + C on consumer
• Restart it again
• The error should show up

2023/01/06 17:06:25 WARNING! Server terminated <336B493D.0.1011>["producer"]. Panic reason: "invalid memory address or nil 

To Reproduce
Steps to reproduce the behavior:

I've made a gist with all the code and a readme there https://gist.github.com/davidroman0O/db04fa224c710a4ae33d53d28304e88b

Expected behavior
Maybe a better error why it has a nil pointer? why? where? stacktrace?

Or maybe it should just not panic

Environment (please complete the following information):

• OS: Tested on Windows 10 and QubeOS 4.11
• Framework Version v1.999.220
• Number of CPU 12

Additional context

I was just experimenting with the examples, trying to plug a bit of my codebase of some side projects and end up with this issue
I came back on the examples to test it again with very little modification and the error appear!

[halturin]

Hello, thanks for the report. May I ask you to run with -ergo.norecover?

[davidroman0O]

Of course

I will take a look soon

Either it's missing a condition or there is a desync on some state (since disconnection of a node, trying to interact with previous demand)

https://github.com/ergo-services/ergo/blob/master/gen/stage_dispatcher.go#L479

> go run .\producer --ergo.norecover --ergo.trace --ergo.warning

to stop press Ctrl-C

Starting nodes 'producer@localhost'
2023/01/06 17:27:44 Start node with name "producer@localhost" and cookie "cookies"
2023/01/06 17:27:44 Node listener[0] port range: 15000...65000
2023/01/06 17:27:44 Started embedded EMPD service and listen port: 4369
2023/01/06 17:27:44 EPMD accepted new connection from [::1]:64323
2023/01/06 17:27:44 Request from EPMD client: [0 26 120 58 152 77 0 0 6 0 5 0 8 112 114 111 100 117 99 101 114 0 4 17 59 1 0 0]
2023/01/06 17:27:44 [producer@localhost] EPMD client: node registered
2023/01/06 17:27:44 SYSTEM: Application load
2023/01/06 17:27:44 [producer@localhost] CORE registering behavior "system_app" in group "ergo:applications"
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1001>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1001> (registered name: "")
2023/01/06 17:27:44 [producer@localhost] CORE registering name (<336B493D.0.1002>): system_app_sup
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1002>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1002> (registered name: "system_app_sup")
2023/01/06 17:27:44 [producer@localhost] SUPERVISOR "system_app_sup" with restart strategy: one_for_one[permanent]
2023/01/06 17:27:44 [producer@localhost] CORE registering name (<336B493D.0.1003>): system_metrics
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1003>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1003> (registered name: "system_metrics")
2023/01/06 17:27:44 [producer@localhost] SYSTEM_METRICS: Init: []etf.Term{node.System{DisableAnonMetrics:false}}
2023/01/06 17:27:44 [producer@localhost] LINK process: <336B493D.0.1002> => <336B493D.0.1003>
2023/01/06 17:27:44 [producer@localhost] LINK process: <336B493D.0.1001> => <336B493D.0.1002>
2023/01/06 17:27:44 [producer@localhost] SYSTEM: Application started
2023/01/06 17:27:44 [producer@localhost] CORE registering behavior "erlang" in group "ergo:applications"
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1004>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1004> (registered name: "")
2023/01/06 17:27:44 [producer@localhost] CORE registering name (<336B493D.0.1005>): net_kernel_sup
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1005>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1005> (registered name: "net_kernel_sup")
2023/01/06 17:27:44 [producer@localhost] SUPERVISOR "net_kernel_sup" with restart strategy: one_for_one[permanent]
2023/01/06 17:27:44 [producer@localhost] CORE registering name (<336B493D.0.1006>): net_kernel
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1006>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1006> (registered name: "net_kernel")
2023/01/06 17:27:44 NET_KERNEL: Init: []etf.Term(nil)
2023/01/06 17:27:44 [producer@localhost] LINK process: <336B493D.0.1005> => <336B493D.0.1006>
2023/01/06 17:27:44 [producer@localhost] CORE registering name (<336B493D.0.1007>): global_name_server
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1007>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1007> (registered name: "global_name_server")
2023/01/06 17:27:44 [producer@localhost] LINK process: <336B493D.0.1005> => <336B493D.0.1007>
2023/01/06 17:27:44 [producer@localhost] CORE registering name (<336B493D.0.1008>): rex
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1008>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1008> (registered name: "rex")
2023/01/06 17:27:44 REX: Init: []etf.Term(nil)
2023/01/06 17:27:44 [producer@localhost] CORE registering behavior "erpc" in group "ergo:remote"
2023/01/06 17:27:44 [producer@localhost] LINK process: <336B493D.0.1005> => <336B493D.0.1008>
2023/01/06 17:27:44 [producer@localhost] CORE registering name (<336B493D.0.1009>): observer_backend
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1009>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1009> (registered name: "observer_backend")
2023/01/06 17:27:44 OBSERVER: Init: []etf.Term(nil)
2023/01/06 17:27:44 [producer@localhost] RPC provide: proc_lib:translate_initial_call (gen.RPC)(0x64be20)
2023/01/06 17:27:44 [producer@localhost] RPC provide: appmon_info:start_link2 (gen.RPC)(0x64c060)
2023/01/06 17:27:44 [producer@localhost] LINK process: <336B493D.0.1005> => <336B493D.0.1009>
2023/01/06 17:27:44 [producer@localhost] CORE registering name (<336B493D.0.1010>): erlang
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1010>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1010> (registered name: "erlang")
2023/01/06 17:27:44 ERLANG: Init: []etf.Term(nil)
2023/01/06 17:27:44 [producer@localhost] LINK process: <336B493D.0.1005> => <336B493D.0.1010>
2023/01/06 17:27:44 [producer@localhost] LINK process: <336B493D.0.1004> => <336B493D.0.1005>
Spawn producer on 'producer@localhost'
2023/01/06 17:27:44 [producer@localhost] CORE registering name (<336B493D.0.1011>): producer
2023/01/06 17:27:44 [producer@localhost] CORE registering process: <336B493D.0.1011>
2023/01/06 17:27:44 [producer@localhost] CORE spawn a new process <336B493D.0.1011> (registered name: "producer")
2023/01/06 17:27:52 EPMD accepted new connection from [::1]:64336
2023/01/06 17:27:52 Request from EPMD client: [0 26 120 58 153 77 0 0 6 0 5 0 8 99 111 110 115 117 109 101 114 0 4 17 59 1 0 0]
2023/01/06 17:27:52 EPMD accepted new connection from [::1]:64337
2023/01/06 17:27:52 Request from EPMD client: [0 9 122 112 114 111 100 117 99 101 114]
2023/01/06 17:27:52 [producer@localhost] NETWORK accepted new connection from 127.0.0.1:64339
2023/01/06 17:27:52 [producer@localhost] NETWORK registering peer "consumer@localhost"
2023/01/06 17:27:52 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, "", "producer"}
2023/01/06 17:27:52 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:27:52 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, "", "producer"}
2023/01/06 17:27:52 [producer@localhost] CONTROL MONITOR [from consumer@localhost]: etf.Tuple{19, etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, "producer", etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23d, 0x5cdf, 0x0, 0x0, 0x0}}}
2023/01/06 17:27:52 [producer@localhost] CONTROL MONITOR [from consumer@localhost]: etf.Tuple{19, etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, "producer", etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23b, 0x5cdf, 0x0, 0x0, 0x0}}}
2023/01/06 17:27:52 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:52 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:27:52 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:52 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1011>
2023/01/06 17:27:52 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23b, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"subscribe", "nil", etf.List{etf.Tuple{"min_demand", 1}, etf.Tuple{"max_demand", 2}, etf.Tuple{"cancel", 0}, etf.Tuple{"manual", false}, etf.Tuple{"partition", 0}}}}
New subscription from: <A8F228F6.0.1011> with min: 1 and max: 2
2023/01/06 17:27:52 [producer@localhost] MONITOR process: <336B493D.0.1011> => <A8F228F6.0.1011>
2023/01/06 17:27:52 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:52 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1011>
2023/01/06 17:27:52 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23b, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"ask", 2}}
Producer: just got demand for 2 event(s) from <A8F228F6.0.1011>
2023/01/06 17:27:52 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, "", "producer"}
Producer. Generate random numbers and send them to consumers... [41 96 58 14 77]
2023/01/06 17:27:52 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:27:52 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:52 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, "", "producer"}
2023/01/06 17:27:52 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:27:52 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:53 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:53 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1011>
2023/01/06 17:27:53 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:53 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1011>
2023/01/06 17:27:53 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1012>
2023/01/06 17:27:53 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23d, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"ask", 4}}
2023/01/06 17:27:53 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1012>
2023/01/06 17:27:53 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23d, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"subscribe", "nil", etf.List{etf.Tuple{"min_demand", 2}, etf.Tuple{"max_demand", 4}, etf.Tuple{"cancel", 0}, etf.Tuple{"manual", false}, etf.Tuple{"partition", 1}}}}
New subscription from: <A8F228F6.0.1012> with min: 2 and max: 4
2023/01/06 17:27:53 [producer@localhost] MONITOR process: <336B493D.0.1011> => <A8F228F6.0.1012>
2023/01/06 17:27:53 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:53 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, "", "producer"}
2023/01/06 17:27:53 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:27:53 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:53 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1011>
2023/01/06 17:27:53 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23b, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"ask", 2}}
Producer: just got demand for 2 event(s) from <A8F228F6.0.1011>
Producer. Generate random numbers and send them to consumers... [10 31 10 20 88]
2023/01/06 17:27:53 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:53 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1011>
2023/01/06 17:27:53 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:53 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1011>
2023/01/06 17:27:53 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:53 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1011>
2023/01/06 17:27:53 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, "", "producer"}
2023/01/06 17:27:53 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:27:53 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:53 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1011>
2023/01/06 17:27:53 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23b, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"ask", 2}}
Producer: just got demand for 2 event(s) from <A8F228F6.0.1011>
Producer. Generate random numbers and send them to consumers... [28 60 49 37 48]
2023/01/06 17:27:53 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:53 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1011>
2023/01/06 17:27:53 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:53 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1011>
2023/01/06 17:27:54 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:54 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <336B493D.0.1011>
2023/01/06 17:27:54 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23d, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"retry-ask", 0x4}}
Producer: just got demand for 4 event(s) from <A8F228F6.0.1012>
Producer. Generate random numbers and send them to consumers... [9 61 38 49 40 9 74]
2023/01/06 17:27:54 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:54 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1012>
2023/01/06 17:27:54 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:54 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1012>
2023/01/06 17:27:54 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, "", "producer"}
2023/01/06 17:27:54 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:27:54 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:54 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1012>
2023/01/06 17:27:54 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23d, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"ask", 4}}
2023/01/06 17:27:54 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, "", "producer"}
Producer: just got demand for 4 event(s) from <A8F228F6.0.1012>
2023/01/06 17:27:54 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
Producer. Generate random numbers and send them to consumers... [82 57 0 58 60 97 59]
2023/01/06 17:27:54 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:54 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:54 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1012>
2023/01/06 17:27:54 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, "", "producer"}
2023/01/06 17:27:54 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:27:54 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:55 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1012>
2023/01/06 17:27:55 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23d, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"ask", 4}}
Producer: just got demand for 4 event(s) from <A8F228F6.0.1012>
Producer. Generate random numbers and send them to consumers... [95 61 88 36 60 98 26]
2023/01/06 17:27:55 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:55 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1012>
2023/01/06 17:27:55 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1012>
2023/01/06 17:27:55 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a068, ID:[5]uint32{0x3a23d, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"ask", 4}}
Producer: just got demand for 4 event(s) from <A8F228F6.0.1012>
Producer. Generate random numbers and send them to consumers... [47 96 46 5 55 66 39]
2023/01/06 17:27:55 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:27:55 [producer@localhost] CORE route message by pid (remote) <A8F228F6.0.1012>
2023/01/06 17:27:57 Request from EPMD client: []
2023/01/06 17:27:57 EPMD unregistering node: 'consumer'
2023/01/06 17:27:57 [producer@localhost] NETWORK unregistering peer consumer@localhost
2023/01/06 17:27:57 [producer@localhost] MONITOR NODE  down: consumer@localhost
2023/01/06 17:27:57 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:57 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:27:57 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <336B493D.0.1011>
2023/01/06 17:27:57 m: gen.MessageDown{Ref:etf.Ref{Node:"producer@localhost", Creation:0x63b8a060, ID:[5]uint32{0x33227, 0x5cdf, 0x0, 0x0, 0x0}}, ProcessID:gen.ProcessID{Name:"", Node:""}, Pid:etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a068}, Reason:"noconnection"}
2023/01/06 17:27:57 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <336B493D.0.1011>
2023/01/06 17:27:57 m: gen.MessageDown{Ref:etf.Ref{Node:"producer@localhost", Creation:0x63b8a060, ID:[5]uint32{0x33228, 0x5cdf, 0x0, 0x0, 0x0}}, ProcessID:gen.ProcessID{Name:"", Node:""}, Pid:etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a068}, Reason:"noconnection"}
2023/01/06 17:27:59 EPMD accepted new connection from [::1]:64344
2023/01/06 17:27:59 Request from EPMD client: [0 26 120 58 153 77 0 0 6 0 5 0 8 99 111 110 115 117 109 101 114 0 4 17 59 1 0 0]
2023/01/06 17:27:59 EPMD accepted new connection from [::1]:64345
2023/01/06 17:27:59 Request from EPMD client: [0 9 122 112 114 111 100 117 99 101 114]
2023/01/06 17:28:00 [producer@localhost] NETWORK accepted new connection from 127.0.0.1:64347
2023/01/06 17:28:00 [producer@localhost] NETWORK registering peer "consumer@localhost"
2023/01/06 17:28:00 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a06f}, "", "producer"}
2023/01/06 17:28:00 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:28:00 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:28:00 [producer@localhost] CONTROL MONITOR [from consumer@localhost]: etf.Tuple{19, etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a06f}, "producer", etf.Ref{Node:"consumer@localhost", Creation:0x63b8a06f, ID:[5]uint32{0x5283, 0x5cdf, 0x0, 0x0, 0x0}}}
2023/01/06 17:28:00 [producer@localhost] CONTROL MONITOR [from consumer@localhost]: etf.Tuple{19, etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a06f}, "producer", etf.Ref{Node:"consumer@localhost", Creation:0x63b8a06f, ID:[5]uint32{0x5285, 0x5cdf, 0x0, 0x0, 0x0}}}
2023/01/06 17:28:00 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a06f}, "", "producer"}
2023/01/06 17:28:00 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:28:00 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:28:00 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a06f}, "", "producer"}
2023/01/06 17:28:00 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:28:00 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:28:00 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1011>
2023/01/06 17:28:00 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a06f}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a06f, ID:[5]uint32{0x5283, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"ask", 2}}
2023/01/06 17:28:00 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1011>
2023/01/06 17:28:00 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f3, Creation:0x63b8a06f}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a06f, ID:[5]uint32{0x5283, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"subscribe", "nil", etf.List{etf.Tuple{"min_demand", 1}, etf.Tuple{"max_demand", 2}, etf.Tuple{"cancel", 0}, etf.Tuple{"manual", false}, etf.Tuple{"partition", 0}}}}
New subscription from: <A8F228F6.0.1011> with min: 1 and max: 2
2023/01/06 17:28:00 [producer@localhost] MONITOR process: <336B493D.0.1011> => <A8F228F6.0.1011>
2023/01/06 17:28:00 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:28:00 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1012>
2023/01/06 17:28:00 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a06f}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a06f, ID:[5]uint32{0x5285, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"subscribe", "nil", etf.List{etf.Tuple{"min_demand", 2}, etf.Tuple{"max_demand", 4}, etf.Tuple{"cancel", 0}, etf.Tuple{"manual", false}, etf.Tuple{"partition", 1}}}}
New subscription from: <A8F228F6.0.1012> with min: 2 and max: 4
2023/01/06 17:28:00 [producer@localhost] MONITOR process: <336B493D.0.1011> => <A8F228F6.0.1012>
2023/01/06 17:28:00 [producer@localhost] NETWORK found active connection with consumer@localhost
2023/01/06 17:28:00 [producer@localhost] CONTROL REG_SEND [from consumer@localhost]: etf.Tuple{6, etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a06f}, "", "producer"}
2023/01/06 17:28:00 [producer@localhost] CORE route message by gen.ProcessID (local) <producer:producer@localhost>
2023/01/06 17:28:00 [producer@localhost] CORE route message by pid (local) <336B493D.0.1011>
2023/01/06 17:28:00 [producer@localhost] GEN_SERVER <336B493D.0.1011> got message from <A8F228F6.0.1012>
2023/01/06 17:28:00 [producer@localhost] GEN_SERVER etf.Pid{Node:"producer@localhost", ID:0x3f3, Creation:0x63b8a060} got simple message etf.Tuple{"$gen_producer", etf.Tuple{etf.Pid{Node:"consumer@localhost", ID:0x3f4, Creation:0x63b8a06f}, etf.Ref{Node:"consumer@localhost", Creation:0x63b8a06f, ID:[5]uint32{0x5285, 0x5cdf, 0x0, 0x0, 0x0}}}, etf.Tuple{"ask", 4}}
Producer: just got demand for 4 event(s) from <A8F228F6.0.1012>
Producer. Generate random numbers and send them to consumers... [95 69 87 59 44 97 27]
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x58 pc=0x5e46bb]

goroutine 122 [running]:
github.com/ergo-services/ergo/gen.(*dispatcherPartition).Dispatch(0xc0000982f0, {0x689ae0?, 0xc00008d140}, {0xc000164280, 0x7, 0x46b8ce?})
        desktop/go/pkg/mod/github.com/ergo-services/ergo@v1.999.220/gen/stage_dispatcher.go:479 +0x4db
github.com/ergo-services/ergo/gen.(*StageProcess).SendEvents(0xc0000ea000, {0xc000164280?, 0xc0000f5888?, 0x2?})
        desktop/go/pkg/mod/github.com/ergo-services/ergo@v1.999.220/gen/stage.go:352 +0x74
main.(*Producer).HandleDemand(0x0?, 0x0?, {{{0xc00047a060, 0x12}, 0x3f4, 0x63b8a06f}, {{0xc00047a078, 0x12}, 0x63b8a06f, {0x5285, ...}}}, ...)
        D:/ergoex/producer/producer.go:40 +0x185
github.com/ergo-services/ergo/gen.(*StageProcess).handleProducer(0xc0000ea000, {{{0xc00047a060, 0x12}, 0x3f4, 0x63b8a06f}, {{0xc00047a078, 0x12}, 0x63b8a06f, {0x5285, 0x5cdf, ...}}}, ...)
        desktop/go/pkg/mod/github.com/ergo-services/ergo@v1.999.220/gen/stage.go:914 +0x604
github.com/ergo-services/ergo/gen.(*StageProcess).handleStageRequest(0x100000001?, {{0xc000212090, 0xd}, {{{0xc00047a060, 0x12}, 0x3f4, 0x63b8a06f}, {{0xc00047a078, 0x12}, 0x63b8a06f, ...}}, ...})
        desktop/go/pkg/mod/github.com/ergo-services/ergo@v1.999.220/gen/stage.go:612 +0x3bb
github.com/ergo-services/ergo/gen.(*Stage).HandleInfo(0x0?, 0xc0000ac300, {0x6a2460?, 0xc000238288})
        desktop/go/pkg/mod/github.com/ergo-services/ergo@v1.999.220/gen/stage.go:497 +0x2c5
github.com/ergo-services/ergo/gen.(*ServerProcess).handleInfo(0xc0000ac300, {{0x6a2460?, 0xc000238288?}})
        desktop/go/pkg/mod/github.com/ergo-services/ergo@v1.999.220/gen/server.go:571 +0xd5
github.com/ergo-services/ergo/gen.(*ServerProcess).waitCallbackOrDeferr.func3()
        desktop/go/pkg/mod/github.com/ergo-services/ergo@v1.999.220/gen/server.go:463 +0x2e
created by github.com/ergo-services/ergo/gen.(*ServerProcess).waitCallbackOrDeferr
        desktop/go/pkg/mod/github.com/ergo-services/ergo@v1.999.220/gen/server.go:461 +0x32a
exit status 2

[halturin]

Thanks. It helped. Nice catch.

PS: I wouldn't recommend to use this approach https://gist.github.com/davidroman0O/db04fa224c710a4ae33d53d28304e88b#file-consumer-go-L88

for ergoNode.IsAlive() {
		if len(previousNodes) != len(ergoNode.Nodes()) {
			isConnected = false
		}
		if !isConnected {
			if err := ergoNode.Connect("producer@localhost"); err != nil {
				isConnected = false
			}
			if !strings.Contains(strings.Join(ergoNode.Nodes(), " "), "producer@localhost") {
				isConnected = false
			} else {
				isConnected = true
			}
		}
		previousNodes = ergoNode.Nodes()
	}

This code burns the CPU.

The better way is using ergoNode.Wait() to lock the main goroutine and use one of the approaches to monitor connections

• gen.Process.MonitorNode(...) for the existing connection
• gen.Process.MonitorEvent(node.EventNetwork) - you will receive node.MessageEventNetwork{...} message on connection/disconnection node.

[davidroman0O]

This code burns the CPU.

Yeah I know, it was just to play around quickly 😄

[halturin]

Fixed. In the v221 branch.

[davidroman0O]

Damn I didn't even had the time to look at it
Bravo!