DNS resolve failed on remote server with smartDNS or dnsmsq since 5.1.0+

[shell12345]

Please read contributing guidelines. Thanks.

Describe the bug
unable to resolve hostname with smartDNS on remote server since 5.10+.
but 5.06 is fine.
smartDNS listen to udp DNS request on 53.
To Reproduce
Steps to reproduce the behavior:

1. 5.1.0+
2. remote server uses smartDNS or dnsmsq to unlock stream, like Netflix
3. set remote DNS 127.0.0.1 or 0.0.0.0
4. VPN mode
5. unable to resolve hostname

Expected behavior
5.06 works fine with the mentioned process.
And 5.10+ seems works fine with proxy only and transparent proxy mode (just past the connection test, not root yet.).
If set remote DNS 1.1.1.1 or 8.8.8.8. It works fine but can not unlock stream.
pls, fix it.
Smartphone (please complete the following information):

• Android 10
• Device: Google Pixel 3 XL
• Last version

Configuration
Put an x inside the [ ] that applies.

• [ x] IPv4 server address
• IPv6 server address
• [x ] Client IPv4 availability
• Client IPv6 availability
• Encrypt method:
• Route
  • All
  • Bypass LAN
  • Bypass China
  • [x ] Bypass LAN & China
  • GFW List
  • China List
  • Custom rules
• IPv6 route
• Apps VPN mode
  • Bypass mode
• Remote DNS: 127.0.0.1 or 0.0.0.0
• [ x] DNS over UDP
• Plugin configuration (if applicable):
• Auto Connect
• TCP Fast Open

[madeye]

Make sure the TCP port is opened in you smartDNS config.

[madeye]

@Mygod do we still support DNS over UDP now? I thought we removed this feature from 5.1.0.

[shell12345]

@madeye ,Thanks.
It turns out bind-tcp not open at 53.

1. 5.1.0+ use tcp as dns request, it seems slower than 5.06 in the test. Any particular reason to use tcp instead of udp? Even DNS forward by upd option is on , it can not use udp now?
2. Android 10 with private DNS, it should force all the DNS request go through the private DNS server like dns.google. But it seems can still unlock stream through the server. How does the SS DNS request process work when private DNS is set?
3. Shadowsocks-libev deployed on a Merlin router, it works with smart DNS on local, which means DNS request through something like 8.8.8.8, not from the remote server. But it still can unlock stream on the remote server. I wonder why?
   These things bother me for a long time.
   Thanks again

[Mygod]

@madeye #2518 is still unresolved. Currently the UDP setting is still there but the switch does not do anything. Feel free to fix/remove it.

@shell12345 Currently on Android 10 private DNS does not work well with VPN. If you enable private DNS, all DNS queries will be sent to Shadowsocks behind the DoT server you configured and we cannot change this behavior.

This app does not currently respect DNS setting configured on server side. See shadowsocks/shadowsocks-rust#256