-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathsearch.xml
More file actions
1117 lines (998 loc) · 327 KB
/
search.xml
File metadata and controls
1117 lines (998 loc) · 327 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>不同linux发行版软件安装工具</title>
<url>/2020/02/09/%E4%B8%8D%E5%90%8Clinux%E5%8F%91%E8%A1%8C%E7%89%88%E8%BD%AF%E4%BB%B6%E5%AE%89%E8%A3%85%E5%B7%A5%E5%85%B7/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h3 id="RPM"><a href="#RPM" class="headerlink" title="RPM"></a>RPM</h3><p>RPM是Red-Hat Package Manager（RPM软件包管理器）的缩写，这一文件格式名称虽然打上了RedHat的标志<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="Redhat guide: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html/deployment_guide/
">[1]</span></a></sup>，但是其原始设计理念是开放式的，现在包括OpenLinux、S.u.S.E.以及Turbo Linux等Linux的分发版本都有采用，可以算是公认的行业标准了。</p>
<h3 id="apt-get"><a href="#apt-get" class="headerlink" title="apt-get"></a>apt-get</h3><p>apt-get<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="RPM使用参考：https://www.cnblogs.com/xiaochaohuashengmi/archive/2011/10/08/2203153.html
">[2]</span></a></sup>，是一条linux命令，适用于deb包管理式的操作系统，主要用于自动从互联网的软件仓库中搜索、安装、升级、卸载软件或操作系统。主要适用于Debian、Ubuntu的系统。 </p>
<a id="more"></a>
<p>Ubuntu 16.04 发布时，一个引人注目的新特性便是 apt 命令的引入。其实早在 2014 年，apt 命令就已经发布了第一个稳定版，只是直到 2016 年的 Ubuntu 16.04 系统发布时才开始引人关注。</p>
<h3 id="yum"><a href="#yum" class="headerlink" title="yum"></a>yum</h3><p>yum（全称为 Yellow dog Updater, Modified）是一个在Fedora和RedHat以及SUSE中的Shell前端软件包管理器。基於RPM包管理，能够从指定的服务器自动下载RPM包并且安装，可以自动处理依赖性关系，并且一次安装所有依赖的软体包，无须繁琐地一次次下载、安装。yum提供了查找、安装、删除某一个、一组甚至全部软件包的命令，而且命令简洁而又好记。</p>
<h3 id="zyppe"><a href="#zyppe" class="headerlink" title="zyppe"></a>zyppe</h3><p>zypper<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="opensuse使用zypper安装软件：https://www.cnblogs.com/littleatp/p/9313563.html
">[3]</span></a></sup>， opensuse中使用的包软件软件，类似CentOS中Yum.</p>
<h3 id="yast"><a href="#yast" class="headerlink" title="yast"></a>yast</h3><p>OpenSUSE中可以使用YAST图形界面中来安装。</p>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">Redhat guide: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html/deployment_guide/<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">RPM使用参考：https://www.cnblogs.com/xiaochaohuashengmi/archive/2011/10/08/2203153.html<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">opensuse使用zypper安装软件：https://www.cnblogs.com/littleatp/p/9313563.html<a href="#fnref:3" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Linux</category>
</categories>
<tags>
<tag>Linux</tag>
</tags>
</entry>
<entry>
<title>maven构建的一些小技巧</title>
<url>/2020/02/10/maven%E6%9E%84%E5%BB%BA%E7%9A%84%E4%B8%80%E4%BA%9B%E5%B0%8F%E6%8A%80%E5%B7%A7/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><p>maven从设计上来说非常的简单，但是却是一个非常强大的工具。很多人将maven只是当做一个构建工具（从源代码生成可运行软件），但是实际上可以将maven当做一个项目管理工具，比如工程管理、插件管理、Jar包依赖管理、软件发布仓库管理等。</p>
<p>下面讲解一些在使用maven当中所遇到的一些小的技巧。</p>
<a id="more"></a>
<h3 id="maven是如何判断操作系统的？"><a href="#maven是如何判断操作系统的？" class="headerlink" title="maven是如何判断操作系统的？"></a>maven是如何判断操作系统的？</h3><p>操作系统家族是通过Maven Enforcer Plugin来实现的，具体来说，和如下代码的效果是完全一致的：</p>
<blockquote><p>Family is calculated based on testing against the name string retreived from the JDK. The name, arch and version values are retreived from the JDK using the following code：</p>
<footer><strong>引用自</strong><cite><a href="http://maven.apache.org/enforcer/enforcer-rules/requireOS.html" target="_blank" rel="noopener">官方文档</a></cite></footer></blockquote>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String OS_NAME = System.getProperty( <span class="string">"os.name"</span> ).toLowerCase( Locale.US );</span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String OS_ARCH = System.getProperty( <span class="string">"os.arch"</span> ).toLowerCase( Locale.US );</span><br><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String OS_VERSION = System.getProperty( <span class="string">"os.version"</span> ).toLowerCase( Locale.US );</span><br></pre></td></tr></table></figure>
<h3 id="profiles可以配置哪些POM节点？"><a href="#profiles可以配置哪些POM节点？" class="headerlink" title="profiles可以配置哪些POM节点？"></a>profiles可以配置哪些POM节点？</h3><p><profiles> 内置在POM文件中，这样是比较推荐的方式，这样不会造成工程在迁移的时候造成信息丢失。<br>可以修改如下的节点:</p>
<figure class="highlight plain"><table><tr><td class="code"><pre><span class="line"><repositories></span><br><span class="line"><pluginRepositories></span><br><span class="line"><dependencies></span><br><span class="line"><plugins></span><br><span class="line"><properties> (not actually available in the main POM, but used behind the scenes)</span><br><span class="line"><modules></span><br><span class="line"><reporting></span><br><span class="line"><dependencyManagement></span><br><span class="line"><distributionManagement></span><br><span class="line">a subset of the <build> element, which consists of:</span><br><span class="line"><defaultGoal></span><br><span class="line"><resources></span><br><span class="line"><testResources></span><br><span class="line"><finalName></span><br></pre></td></tr></table></figure>
<h4 id="POM-elements-outside"><a href="#POM-elements-outside" class="headerlink" title="POM elements outside "></a>POM elements outside <profiles></h4><p>这样做的方式是不允许的，对工程迁移、编译都非常的不友好。可以通过外部文件比如settings.xml profiles.xml来一些参数，但是相对来说，危害要稍微小一些。</p>
<h3 id="如何配置仓库镜像（MirrorOf）？"><a href="#如何配置仓库镜像（MirrorOf）？" class="headerlink" title="如何配置仓库镜像（MirrorOf）？"></a>如何配置仓库镜像（MirrorOf）？</h3><p>在POM文件中使用Repositories标签就可以从你希望的地址上去下载构件，比如依赖库或者插件。这样对于工程的迁移性比较好。在任何地方都可以构建起来。</p>
<p>但是有的时候，你希望在不修改工程POM文件的情况下使用另外不同的仓库，这个时候就可以使用mirror。关于mirrorOf的规则可以参考官方文档<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="Apache maven 配置仓库镜像：http://maven.apache.org/guides/mini/guide-mirror-settings.html">[1]</span></a></sup>。</p>
<p>几个需要注意的地方：<br>1、最快的镜像应该放在最前面<br>2、如果第一个镜像的规则是*，那么后面的仓库可能不会使用。<br>3、<id>的默认值是central</p>
<figure class="highlight plain"><table><tr><td class="code"><pre><span class="line"><mirrors></span><br><span class="line"> <mirror> </span><br><span class="line"> <id>aliyun</id> </span><br><span class="line"> <mirrorOf>central</mirrorOf> </span><br><span class="line"> <name>阿里云公共仓库</name> </span><br><span class="line"> <url>http://maven.aliyun.com/nexus/content/groups/public/</url> </span><br><span class="line"> </mirror></span><br><span class="line"> </span><br><span class="line"> <mirror> </span><br><span class="line"> <id>repo2</id> </span><br><span class="line"> <mirrorOf>!central,*</mirrorOf> </span><br><span class="line"> <name>Human Readable Name for this Mirror.</name> </span><br><span class="line"> <url>http://repo2.maven.org/maven2/</url> </span><br><span class="line"> </mirror></span><br><span class="line"></mirrors></span><br></pre></td></tr></table></figure>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">Apache maven 配置仓库镜像：http://maven.apache.org/guides/mini/guide-mirror-settings.html<a href="#fnref:1" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Building tools - maven</category>
</categories>
<tags>
<tag>maven</tag>
<tag>profiles</tag>
<tag>settings.xml</tag>
<tag>mirrorOf</tag>
</tags>
</entry>
<entry>
<title>Kafka调研（1）：跨数据中心的多集群镜像</title>
<url>/2020/02/14/Kafka%E5%88%9D%E6%AD%A5%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A-%E8%B7%A8%E6%95%B0%E6%8D%AE%E4%B8%AD%E5%BF%83%E9%9B%86%E7%BE%A4/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h2 id="调研结果和建议"><a href="#调研结果和建议" class="headerlink" title="调研结果和建议"></a>调研结果和建议</h2><p>针对是否可以跨数据中心部署Kafka集群的问题，答案是可行，但是不建议。建议先支持单集群部署，以简化运维成本，支持多地多应用业务上线。</p>
<h3 id="可行性与跨网络部署产生的问题"><a href="#可行性与跨网络部署产生的问题" class="headerlink" title="可行性与跨网络部署产生的问题"></a>可行性与跨网络部署产生的问题</h3><p>Kafka实际上并不禁止将Broker部署到不同的网络地域。但是，Kafka的设计是按照单个数据中心进行设计、开发、测试和调优的。从Kafka的官方文档<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="Kafka Documentation#Datacenter。http://kafka.apache.org/documentation/#datacenters
">[1]</span></a></sup>来看，并不推荐将集群跨网络部署。 由于Broker分布在不同的跨地域的网络（WAN）, 除了数据同步会极大地增加延迟之外， 并且如果由于网络中断无法保证zk始终可用，带来其他问题，比如：</p>
<ol>
<li>Partition Leader无法选举。或者由于网络问题，某个Broker频繁掉线导致集群不稳定。</li>
<li>在Producer推送数据时候，产生延迟（推送到remote Broker）。或者由于网络问题，无法获取ACK。</li>
</ol>
<a id="more"></a>
<h3 id="跨广域网部署Stretch-Cluster"><a href="#跨广域网部署Stretch-Cluster" class="headerlink" title="跨广域网部署Stretch Cluster"></a>跨广域网部署Stretch Cluster</h3><p><strong><em>部署条件：</em></strong></p>
<ol>
<li>至少具有3个高带宽和低延迟的DC（考虑zookeeper）</li>
<li>有需要异地容灾和100%恢复的需求</li>
<li>配置参数min.isr和acks=all。这样就可以要求异地所有的Broker都可以收到，并且确认消息。</li>
</ol>
<p><strong><em>带来的复杂度：</em></strong></p>
<ol>
<li>运维的复杂度，以及带来的成本上升</li>
<li>无法对应Kafka或者应用的故障</li>
</ol>
<h3 id="多集群架构"><a href="#多集群架构" class="headerlink" title="多集群架构"></a>多集群架构</h3><p>有时候需要有场景在不同的地方将数据进行共享或者汇聚。那么可以选择多集群架构。在选择集群的时候需要考虑的一些问题：</p>
<ol>
<li>高延迟</li>
<li>有限的带宽</li>
<li>高成本 主要来自于多个集群部署的硬件成本、人力成本以及带宽的占用</li>
</ol>
<p><strong><em>常见的几种架构<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="Kafa权威指南,第8章。
">[2]</span></a></sup></em></strong></p>
<h5 id="Hub和Spoke架构"><a href="#Hub和Spoke架构" class="headerlink" title="Hub和Spoke架构"></a>Hub和Spoke架构</h5><img src="/2020/02/14/Kafka%E5%88%9D%E6%AD%A5%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A-%E8%B7%A8%E6%95%B0%E6%8D%AE%E4%B8%AD%E5%BF%83%E9%9B%86%E7%BE%A4/hub&spoke.png" class="" title="Hub And Spoke">
<h5 id="双活架构"><a href="#双活架构" class="headerlink" title="双活架构"></a>双活架构</h5><img src="/2020/02/14/Kafka%E5%88%9D%E6%AD%A5%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A-%E8%B7%A8%E6%95%B0%E6%8D%AE%E4%B8%AD%E5%BF%83%E9%9B%86%E7%BE%A4/twolives.png" class="" title="双活">
<h5 id="主备架构"><a href="#主备架构" class="headerlink" title="主备架构"></a>主备架构</h5><img src="/2020/02/14/Kafka%E5%88%9D%E6%AD%A5%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A-%E8%B7%A8%E6%95%B0%E6%8D%AE%E4%B8%AD%E5%BF%83%E9%9B%86%E7%BE%A4/master&slave.png" class="" title="主备">
<h3 id="Mirror-Maker官方镜像工具"><a href="#Mirror-Maker官方镜像工具" class="headerlink" title="Mirror Maker官方镜像工具"></a>Mirror Maker官方镜像工具</h3><p>推荐的部署模式<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="Mirror Maker。https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330
">[3]</span></a></sup> 是各地独立部署Kafka cluster, 通过Mirror Maker连接。这种部署的好处在于每个集群是互相独立的。并不互相影响。通过mm连接集群AB之后，可以确保A的数据是B的一个子集，但是两者并不能完全将彼此服务替代。因为Consumer offset两边并不同步。</p>
<p>通过MM连接示意如下：</p>
<img src="/2020/02/14/Kafka%E5%88%9D%E6%AD%A5%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A-%E8%B7%A8%E6%95%B0%E6%8D%AE%E4%B8%AD%E5%BF%83%E9%9B%86%E7%BE%A4/mm.png" class="" title="MM">
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">Kafka Documentation#Datacenter。http://kafka.apache.org/documentation/#datacenters<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">Kafa权威指南,第8章。<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">Mirror Maker。https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330<a href="#fnref:3" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Message Queue - Kafka</category>
</categories>
<tags>
<tag>Message Queue</tag>
<tag>Kafka</tag>
<tag>Data center</tag>
<tag>Cluster</tag>
<tag>Mirror Maker</tag>
<tag>Architecture</tag>
</tags>
</entry>
<entry>
<title>Kafka调研（2）：消息序列化框架建议与对比</title>
<url>/2020/02/14/Kafka%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A2-%E6%B6%88%E6%81%AF%E5%BA%8F%E5%88%97%E5%8C%96%E6%A1%86%E6%9E%B6%E6%A8%AA%E5%90%91%E5%AF%B9%E6%AF%94/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h2 id="调研结果和建议"><a href="#调研结果和建议" class="headerlink" title="调研结果和建议"></a>调研结果和建议</h2><p>综合来看，Avro或者Protocol Buffer二选一即可， 但是出于灵活性和标准JSON的支持力度上来看，更推荐Protocol Buffers。同时部分数据依然可以通过JSON方式传输。</p>
<h2 id="主流序列化框架对比"><a href="#主流序列化框架对比" class="headerlink" title="主流序列化框架对比"></a>主流序列化框架对比</h2><h3 id="Kafka的序列化"><a href="#Kafka的序列化" class="headerlink" title="Kafka的序列化"></a>Kafka的序列化</h3><p>Kafka并不限定的消息格式。Kafka自身提供了一系列的序列化器，比如ByteArraySerializer，ByteBufferSerializer，DoubleSerializer，FloatSerializer，StringSerializer，UUIDSerializer， IntegerDeserializer, LongSerializer, ShortSerializer, 总结起来就是基本类型、字符串类型、二进制消息。</p>
<p>但是在通常情况下不会使用比如Float这种来做序列化，原因在于系统间交互或者放到Kafka的数据一般来说都比较复杂，无法用一个字段来表示。于是，这种情况下，就需要在Kafka之上建立一个消息格式的契约，以便于Kafka的生产者和消费者之间都能理解消息中的语义。最好地方就是在Kafka基础之上增加一层应用层的消息协议，比如JSON格式、比如二进制编解码协议（序列化），出于性能考虑，一般也不直接使用Java POJO对象的原生序列化（虽然可行）。在Apache Spark中使用的Kryo Register，由于场景限制（不提供IDL定义），不适用Kafka。</p>
<p>下面对常见的几个序列化框架做一个横向对比：</p>
<a id="more"></a>
<h3 id="几种主流序列化对比"><a href="#几种主流序列化对比" class="headerlink" title="几种主流序列化对比"></a>几种主流序列化对比</h3><p>当前比较流行的有JSON<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf
">[1]</span></a></sup>, Apache Avro<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://avro.apache.org/
">[2]</span></a></sup>, Google开源的Protocol Buffer<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://developers.google.com/protocol-buffers
">[3]</span></a></sup>，Apache Thrift<sup id="fnref:4"><a href="#fn:4" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://thrift.apache.org/">[4]</span></a></sup>。</p>
<table>
<thead>
<tr>
<th>消息格式</th>
<th>JSON</th>
<th>Apache Avro</th>
<th>Protocol Buffer</th>
</tr>
</thead>
<tbody><tr>
<td>License</td>
<td>ECMA-404标准</td>
<td>Apache License 2.0</td>
<td>BSD License</td>
</tr>
<tr>
<td>IDL</td>
<td>无IDL定义，也就意味着系统间无契约</td>
<td>通过JSON定义，灵活度不够。语言无关。</td>
<td>类似Thrift格式，灵活定义，支持Any类型</td>
</tr>
<tr>
<td>Code Generator</td>
<td>无，但是应该可以找到对应的开源工具</td>
<td>有</td>
<td>有</td>
</tr>
<tr>
<td>性能</td>
<td>低</td>
<td>高</td>
<td>高</td>
</tr>
<tr>
<td>兼容性</td>
<td>高</td>
<td>前向兼容</td>
<td>前向兼容</td>
</tr>
<tr>
<td>JSON<->二进制</td>
<td>支持</td>
<td>支持，但是非标准JSON</td>
<td>支持</td>
</tr>
<tr>
<td>支持语言</td>
<td>几乎所有</td>
<td>Java/C++/C/C#</td>
<td>Java/Python/C++。 其他语言有对应开源库，比如Nodejs可以使用暴雪开源库。</td>
</tr>
</tbody></table>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://avro.apache.org/<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://developers.google.com/protocol-buffers<a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://thrift.apache.org/<a href="#fnref:4" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Message Queue - Kafka</category>
</categories>
<tags>
<tag>Message Queue</tag>
<tag>Kafka</tag>
<tag>Architecture</tag>
<tag>Serialization</tag>
<tag>JSON</tag>
<tag>Apahce Avro</tag>
<tag>Apahce Thrift</tag>
<tag>Protocol Buffers</tag>
</tags>
</entry>
<entry>
<title>什么是Kafka</title>
<url>/2020/02/14/%E4%BB%80%E4%B9%88%E6%98%AFKafka/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><p><strong><em>占坑，后面有时间对Kafka进行详细的更新</em></strong></p>
<h1 id="A-bit-of-History"><a href="#A-bit-of-History" class="headerlink" title="A bit of History"></a>A bit of History</h1><p>Kafka是高性能的、分布式的、数据可持久化的消息丢列。最初由LinkedIn开发，用来跟踪Web上的用户的一系列事件，譬如页面浏览(Page view)，搜索框的输入，展示的广告等等。这些事件对于监控和调查用于对于网站交互的参与度非常的重要。每天都有大量的数据产生，因为，需要一个可以轻松扩展，同时又不会增加额外负担的一个解决方案。</p>
<p>总体来说，Kafka从设计之初有3个目标：</p>
<ol>
<li>简单的生产者/消费者API设计</li>
<li>尽可能减少网络传输和磁盘存储的额外成本</li>
<li>从一开始就为可扩展性而设计</li>
</ol>
<a id="more"></a>
<h1 id="什么是Kafka"><a href="#什么是Kafka" class="headerlink" title="什么是Kafka"></a>什么是Kafka</h1><p>Kafka<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://kafka.apache.org/
">[1]</span></a></sup>是最初由Linkedin公司开发，是一个分布式、分区的、多副本的、多订阅者，基于zookeeper协调的分布式日志系统（也可以当做MQ系统），常见可以用于web/nginx日志、访问日志，消息服务等等，Linkedin于2010年贡献给了Apache基金会并成为顶级开源项目。</p>
<p>主要应用场景是：日志收集系统和消息系统。</p>
<p>Kafka主要设计目标如下：</p>
<ol>
<li>以时间复杂度为O(1)的方式提供消息持久化能力，即使对TB级以上数据也能保证常数时间的访问性能。</li>
<li>高吞吐率。即使在非常廉价的商用机器上也能做到单机支持每秒100K条消息的传输。</li>
<li>支持Kafka Server间的消息分区，及分布式消费，同时保证每个partition内的消息顺序传输。</li>
<li>同时支持离线数据处理和实时数据处理。</li>
<li>Scale out:支持在线水平扩展</li>
</ol>
<h1 id="Kafka中的术语说明"><a href="#Kafka中的术语说明" class="headerlink" title="Kafka中的术语说明"></a>Kafka中的术语说明</h1><p>在深入理解Kafka之前，先介绍一下Kafka中的术语。下图展示了Kafka的相关术语以及之间的关系<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://www.cnblogs.com/qingyunzong/p/9004509.html">[2]</span></a></sup>：</p>
<img src="/2020/02/14/%E4%BB%80%E4%B9%88%E6%98%AFKafka/1228818-20180507190731172-1317551019.png" class="" title="Kafka-structure">
<p>上图中一个topic配置了3个partition。Partition1有两个offset：0和1。Partition2有4个offset。Partition3有1个offset。副本的id和副本所在的机器的id恰好相同。</p>
<p>如果一个topic的副本数为3，那么Kafka将在集群中为每个partition创建3个相同的副本。集群中的每个broker存储一个或多个partition。多个producer和consumer可同时生产和消费数据。</p>
<h2 id="broker"><a href="#broker" class="headerlink" title="broker"></a>broker</h2><p>Kafka 集群包含一个或多个服务器，服务器节点称为broker。</p>
<p>broker存储topic的数据。如果某topic有N个partition，集群有N个broker，那么每个broker存储该topic的一个partition。</p>
<p>如果某topic有N个partition，集群有(N+M)个broker，那么其中有N个broker存储该topic的一个partition，剩下的M个broker不存储该topic的partition数据。</p>
<p>如果某topic有N个partition，集群中broker数目少于N个，那么一个broker存储该topic的一个或多个partition。在实际生产环境中，尽量避免这种情况的发生，这种情况容易导致Kafka集群数据不均衡。</p>
<h2 id="Topic"><a href="#Topic" class="headerlink" title="Topic"></a>Topic</h2><p>每条发布到Kafka集群的消息都有一个类别，这个类别被称为Topic。（物理上不同Topic的消息分开存储，逻辑上一个Topic的消息虽然保存于一个或多个broker上但用户只需指定消息的Topic即可生产或消费数据而不必关心数据存于何处）</p>
<p>类似于数据库的表名</p>
<h2 id="Partition"><a href="#Partition" class="headerlink" title="Partition"></a>Partition</h2><p>topic中的数据分割为一个或多个partition。每个topic至少有一个partition。每个partition中的数据使用多个segment文件存储。partition中的数据是有序的，不同partition间的数据丢失了数据的顺序。如果topic有多个partition，消费数据时就不能保证数据的顺序。在需要严格保证消息的消费顺序的场景下，需要将partition数目设为1。</p>
<h2 id="Producer"><a href="#Producer" class="headerlink" title="Producer"></a>Producer</h2><p>生产者即数据的发布者，该角色将消息发布到Kafka的topic中。broker接收到生产者发送的消息后，broker将该消息追加到当前用于追加数据的segment文件中。生产者发送的消息，存储到一个partition中，生产者也可以指定数据存储的partition。</p>
<h2 id="Consumer"><a href="#Consumer" class="headerlink" title="Consumer"></a>Consumer</h2><p>消费者可以从broker中读取数据。消费者可以消费多个topic中的数据。</p>
<h2 id="Consumer-Group"><a href="#Consumer-Group" class="headerlink" title="Consumer Group"></a>Consumer Group</h2><p>每个Consumer属于一个特定的Consumer Group（可为每个Consumer指定group name，若不指定group name则属于默认的group）。</p>
<h2 id="Leader"><a href="#Leader" class="headerlink" title="Leader"></a>Leader</h2><p>每个partition有多个副本，其中有且仅有一个作为Leader，Leader是当前负责数据的读写的partition。</p>
<h2 id="Follower"><a href="#Follower" class="headerlink" title="Follower"></a>Follower</h2><p>Follower跟随Leader，所有写请求都通过Leader路由，数据变更会广播给所有Follower，Follower与Leader保持数据同步。如果Leader失效，则从Follower中选举出一个新的Leader。当Follower与Leader挂掉、卡住或者同步太慢，leader会把这个follower从“in sync replicas”（ISR）列表中删除，重新创建一个Follower。</p>
<h1 id="扩展阅读"><a href="#扩展阅读" class="headerlink" title="扩展阅读"></a>扩展阅读</h1><ol>
<li>Kafka<a href="https://kafka.apache.org/documentation/" target="_blank" rel="noopener">官网文档</a></li>
<li>郭俊的<a href="http://www.jasongj.com/" target="_blank" rel="noopener">博客</a>，以及他在Infoq.cn上发布的<a href="https://www.infoq.cn/profile/1278686/publish" target="_blank" rel="noopener">文章</a>。部分内容存在一些时效性的问题。当做入门的阅读来看，是很不错的。</li>
</ol>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://kafka.apache.org/<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.cnblogs.com/qingyunzong/p/9004509.html<a href="#fnref:2" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Message Queue - Kafka</category>
<category>Architecture</category>
</categories>
<tags>
<tag>Architecture</tag>
<tag>Message Queue - Kafka</tag>
<tag>JMS</tag>
</tags>
</entry>
<entry>
<title>什么是MQ</title>
<url>/2020/02/14/%E4%BB%80%E4%B9%88%E6%98%AFMQ/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><p><strong><em>这篇文章是一篇未完成稿，有时间再写</em></strong></p>
<h1 id="什么是消息队列（Message-Queue）"><a href="#什么是消息队列（Message-Queue）" class="headerlink" title="什么是消息队列（Message Queue）?"></a>什么是消息队列（Message Queue）?</h1><p>消息队列使得应用之间可以通过互发消息的方式来通信，这和传统的RPC方式有很大的区别。通过消息队列提供的服务，解除了点对点，星形拓扑通信耦合的问题。同时，当目标服务不可用的时候，消息队列充当了临时的消息缓存。</p>
<p>下面先解释一下什么是消息队列<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://www.cloudamqp.com/blog/2014-12-03-what-is-message-queuing.html
">[1]</span></a></sup>，怎么使用，在架构层面又会带来什么样的好处<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="扩展阅读：https://www.ibm.com/cloud/learn/message-queues
">[2]</span></a></sup>。</p>
<p><strong><em>队列</em></strong>就是将事物排成一列，依次处理，并且依照先进先出(First In, First Out)的顺序。那么消息队列（MQ），就是用于不同系统或者应用之间通信的的列表， 将需要通信的消息保存在队列中，等待应用依次处理。<a href="http://queues.io/" target="_blank" rel="noopener">queues.io</a>是一个比较好的网站可以查找各种不同功能的队列。</p>
<a id="more"></a>
<img src="/2020/02/14/%E4%BB%80%E4%B9%88%E6%98%AFMQ/message-queue-small.png" class="" title="MQ">
<p><strong><em>消息</em></strong>就是在发送者和接受者之间传递的数据。这里的数据可以是任何东西，数据、指令、文件等等，比如一个消息可能是系统A告诉系统B可以开始处理某个任务。</p>
<h1 id="订阅-发布模式"><a href="#订阅-发布模式" class="headerlink" title="订阅/发布模式"></a>订阅/发布模式</h1><p>这种模式也叫生产者和消费者模式。生产者连接到MQ，生成消息然后放入MQ；消费者连接到MQ，获取消息然后进行处理。</p>
<h1 id="架构上的解耦和扩展性"><a href="#架构上的解耦和扩展性" class="headerlink" title="架构上的解耦和扩展性"></a>架构上的解耦和扩展性</h1><p>在现代云架构中，应用程序被分解为多个规模较小且更易于开发、部署和维护的独立构建块。消息队列可为这些分布式应用程序提供通信和协调。消息队列可以显著简化分离应用程序的编码，同时提高性能、可靠性和可扩展性。</p>
<p>借助消息队列，系统的不同部分可相互通信并异步执行处理操作。消息队列提供一个临时存储消息的轻量级缓冲区，以及允许软件组件连接到队列以发送和接收消息的终端节点。这些消息通常较小，可以是请求、恢复、错误消息或明文信息等。要发送消息时，一个名为“创建器”的组件会将消息添加到队列。消息将存储在队列中，直至名为“处理器”的另一组件检索该消息并执行相关操作。</p>
<p>MQ 提供了一个异步通信的机制，当另外一个系统没有响应的时候，发送者并不会因此而阻塞。</p>
<h2 id="解耦和扩展性"><a href="#解耦和扩展性" class="headerlink" title="解耦和扩展性"></a>解耦和扩展性</h2><img src="/2020/02/14/%E4%BB%80%E4%B9%88%E6%98%AFMQ/thumb-mq.jpg" class="" title="Decouple">
<h1 id="消息协议"><a href="#消息协议" class="headerlink" title="消息协议"></a>消息协议</h1><p>现代的MQ支持很多不同的协议<sup id="fnref:5"><a href="#fn:5" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="WIKI百科：https://en.wikipedia.org/wiki/Message_queue">[5]</span></a></sup>。有了这些协议之后，很多不同类型的MQ都可以在一起协同使用。<br>常见的有AMQP<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="AMQP: https://dl.acm.org/doi/10.1145/1255421.1255424
">[3]</span></a></sup>, MQTT<sup id="fnref:4"><a href="#fn:4" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="物联网通信息协议: http://mqtt.org/
">[4]</span></a></sup>等等。</p>
<h1 id="待完成"><a href="#待完成" class="headerlink" title="待完成"></a>待完成</h1><div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.cloudamqp.com/blog/2014-12-03-what-is-message-queuing.html<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">扩展阅读：https://www.ibm.com/cloud/learn/message-queues<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">AMQP: https://dl.acm.org/doi/10.1145/1255421.1255424<a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">物联网通信息协议: http://mqtt.org/<a href="#fnref:4" rev="footnote"> ↩</a></span></li><li id="fn:5"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">5.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">WIKI百科：https://en.wikipedia.org/wiki/Message_queue<a href="#fnref:5" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Message Queue - Kafka</category>
</categories>
<tags>
<tag>Message Queue</tag>
<tag>Architecture</tag>
<tag>JMS</tag>
<tag>AMQP</tag>
<tag>MQTT</tag>
</tags>
</entry>
<entry>
<title>使用Socket.io搭建WS服务并通过Nginx反向代理</title>
<url>/2020/02/17/%E7%94%A8Nodejs%E5%92%8CNginx%E6%90%AD%E5%BB%BAWebsocket%E6%9C%8D%E5%8A%A1%E4%BB%A5%E5%8F%8A%E7%9B%B8%E5%85%B3%E9%85%8D%E7%BD%AE/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h1 id="Nodejs、Express、Socket-io、Nginx"><a href="#Nodejs、Express、Socket-io、Nginx" class="headerlink" title="Nodejs、Express、Socket.io、Nginx"></a>Nodejs、Express、Socket.io、Nginx</h1><p>使用Express框架和Socket.io库将websocket搭建起来，并监听同一个端口，参考官方例子<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://socket.io/docs/#Using-with-Express
">[1]</span></a></sup>就可以，非常简单。</p>
<figure class="highlight javascript"><table><tr><td class="code"><pre><span class="line"><span class="keyword">var</span> app = <span class="built_in">require</span>(<span class="string">'express'</span>)();</span><br><span class="line"><span class="keyword">var</span> server = <span class="built_in">require</span>(<span class="string">'http'</span>).Server(app);</span><br><span class="line"><span class="keyword">var</span> io = <span class="built_in">require</span>(<span class="string">'socket.io'</span>)(server);</span><br><span class="line"></span><br><span class="line">server.listen(<span class="number">80</span>);</span><br><span class="line"><span class="comment">// WARNING: app.listen(80) will NOT work here!</span></span><br><span class="line"></span><br><span class="line">app.get(<span class="string">'/'</span>, <span class="function"><span class="keyword">function</span> (<span class="params">req, res</span>) </span>{</span><br><span class="line">res.sendFile(__dirname + <span class="string">'/index.html'</span>);</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line">io.on(<span class="string">'connection'</span>, <span class="function"><span class="keyword">function</span> (<span class="params">socket</span>) </span>{</span><br><span class="line">socket.emit(<span class="string">'news'</span>, { <span class="attr">hello</span>: <span class="string">'world'</span> });</span><br><span class="line">socket.on(<span class="string">'my other event'</span>, <span class="function"><span class="keyword">function</span> (<span class="params">data</span>) </span>{</span><br><span class="line"> <span class="built_in">console</span>.log(data);</span><br><span class="line">});</span><br><span class="line">});</span><br></pre></td></tr></table></figure>
<h1 id="Nginx同时作为Web-Server和WebSocket-Reverse-Proxy"><a href="#Nginx同时作为Web-Server和WebSocket-Reverse-Proxy" class="headerlink" title="Nginx同时作为Web Server和WebSocket Reverse Proxy"></a>Nginx同时作为Web Server和WebSocket Reverse Proxy</h1><p>在配置Nginx的时候，遇到了很多的问题，并且在网上现在找不到一篇对这个问题讲解的非常清楚的文档。<br>比如：</p>
<a id="more"></a>
<ol>
<li>Nginx是否支持http/websocket在同一个端口。答案是可以的。</li>
<li>Nginx支持支持代理socket.io。 结论：支持。</li>
<li>var socket = io(); 初始化的时候，io()要不要传入地址参数。 结论：不需要传入参数。</li>
<li>Nginx配置的URL匹配规则是什么。</li>
</ol>
<p>全部代码请跳转<a href="https://github.com/semonw/ws-proxy" target="_blank" rel="noopener">Github</a>.</p>
<p>经过试验和参考Nginx的文档<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://www.nginx.com/blog/nginx-nodejs-websockets-socketio/
">[2]</span></a></sup>，Nginx配置如下：</p>
<figure class="highlight nginx"><table><tr><td class="code"><pre><span class="line"><span class="section">server</span> {</span><br><span class="line"> <span class="attribute">listen</span> <span class="number">8080</span>;</span><br><span class="line"> <span class="attribute">server_name</span> localhost;</span><br><span class="line"></span><br><span class="line"> <span class="attribute">location</span> / {</span><br><span class="line"> <span class="attribute">root</span> D:\\Projects\\ws-proxy\\docroot;</span><br><span class="line"> <span class="attribute">index</span> index.html index.htm;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="attribute">location</span><span class="regexp"> ^~/socket.io</span> {</span><br><span class="line"> <span class="attribute">proxy_set_header</span> Upgrade <span class="variable">$http_upgrade</span>;</span><br><span class="line"> <span class="attribute">proxy_set_header</span> Connection <span class="string">"upgrade"</span>;</span><br><span class="line"> <span class="attribute">proxy_http_version</span> <span class="number">1</span>.<span class="number">1</span>;</span><br><span class="line"> <span class="attribute">proxy_set_header</span> X-Forwarded-For <span class="variable">$proxy_add_x_forwarded_for</span>;</span><br><span class="line"> <span class="attribute">proxy_set_header</span> Host <span class="variable">$host</span>;</span><br><span class="line"> <span class="attribute">proxy_pass</span> http://127.0.0.1:8885;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="attribute">error_page</span> <span class="number">500</span> <span class="number">502</span> <span class="number">503</span> <span class="number">504</span> /50x.html;</span><br><span class="line"> <span class="attribute">location</span> = /50x.html {</span><br><span class="line"> <span class="attribute">root</span> html;</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
<p>为什么nginx上配置的Location必须是固定的，通过调试socket.io的机制就清楚了，见下。如果将nginx location中的规则修改成其他的，前端将无法连接socket.io服务。</p>
<img src="/2020/02/17/%E7%94%A8Nodejs%E5%92%8CNginx%E6%90%AD%E5%BB%BAWebsocket%E6%9C%8D%E5%8A%A1%E4%BB%A5%E5%8F%8A%E7%9B%B8%E5%85%B3%E9%85%8D%E7%BD%AE/20200217100716.png" class="" title="snapshot_socket.io">
<h1 id="除了Socket-io之外，是否还要其他选择？"><a href="#除了Socket-io之外，是否还要其他选择？" class="headerlink" title="除了Socket.io之外，是否还要其他选择？"></a>除了Socket.io之外，是否还要其他选择？</h1><p>可以使用ws库来搭建一个websocket服务。这个和socket.io的区别在于socket.io做了额外的很多工作，比如某些浏览器上不支持websocket，那么socket.io会通过轮询的方式来模拟websocket.</p>
<p>如果使用nodejs, 可以参考Nginx官方的文档<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://www.nginx.com/blog/websocket-nginx/
">[3]</span></a></sup></p>
<h1 id="配置Nginx"><a href="#配置Nginx" class="headerlink" title="配置Nginx"></a>配置Nginx</h1><h2 id="申请SSL证书"><a href="#申请SSL证书" class="headerlink" title="申请SSL证书"></a>申请SSL证书</h2><p>先申请SSL证书，有付费的，有免费的，一般来说，如果购买了腾讯云/阿里云的域名都可以申请到免费证书。也可以到<a href="https://letsencrypt.org/" target="_blank" rel="noopener">Let’s Encrypt</a>网站上申请。这个网站的服务是由互联网安全研究工作组<a href="https://www.abetterinternet.org/" target="_blank" rel="noopener">ISRG</a>提供的。</p>
<p>不过我使用Caddy软件一键自动部署（代理）和自动申请证书。不过从使用体验上来看，似乎Caddy要稍微性能要慢一些。</p>
<p>Caddy自动申请的证书存放的位置<sup id="fnref:5"><a href="#fn:5" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://www.cmsky.com/caddy-proxy-shell/
">[5]</span></a></sup>在：<br>~/.caddy/acme/acme-v01.api.letsencrypt.org/sites/xxx.xxx(域名)/</p>
<p>所以，如果有Caddy已经自动申请了证书，那么nginx可以直接使用该证书。</p>
<h2 id="配置nginx-开启HTTPS"><a href="#配置nginx-开启HTTPS" class="headerlink" title="配置nginx, 开启HTTPS"></a>配置nginx, 开启HTTPS</h2><p>默认nginx的配置目录在/etc/nginx下面。</p>
<p>可以通过DigitalOcean提供的免费Nginx的配置工具<sup id="fnref:6"><a href="#fn:6" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://www.digitalocean.com/community/tools/nginx">[6]</span></a></sup>来生成配置文件，这个网站非常棒，提供了很丰富的配置。生成出来的配置按照Site进行了分组。比如我的域名node-nj.typedefine.cn, 我只需要打开sites-available下面的node-nj.typedefine.cn.conf配置进行查看就行了。</p>
<p>nginx依然通过nginx.conf来加载配置，但是nginx.conf->sites-enabled->sites-available。基本上只需要关注sites-available中每个站点的独立的配置就可以了。</p>
<p>如果需要将SSL证书共用，也可以配置到Global Config中。具体使用参考说明。</p>
<img src="/2020/02/17/%E7%94%A8Nodejs%E5%92%8CNginx%E6%90%AD%E5%BB%BAWebsocket%E6%9C%8D%E5%8A%A1%E4%BB%A5%E5%8F%8A%E7%9B%B8%E5%85%B3%E9%85%8D%E7%BD%AE/20200313111923.png" class="" title="config_nginx">
<h2 id="配置nginx到Systemd中"><a href="#配置nginx到Systemd中" class="headerlink" title="配置nginx到Systemd中"></a>配置nginx到Systemd中</h2><p>新增nginx.service到/etc/systemd/system目录下</p>
<figure class="highlight plain"><table><tr><td class="code"><pre><span class="line">[Unit]</span><br><span class="line">Description=The NGINX HTTP and reverse proxy server</span><br><span class="line">After=syslog.target network.target remote-fs.target nss-lookup.target</span><br><span class="line"></span><br><span class="line">[Service]</span><br><span class="line">Type=forking</span><br><span class="line">PIDFile=/run/nginx.pid</span><br><span class="line">ExecStartPre=/usr/sbin/nginx -t -c /etc/nginx/nginx.conf</span><br><span class="line">ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf</span><br><span class="line">ExecReload=/usr/sbin/nginx -s reload</span><br><span class="line">ExecStop=/bin/kill -s QUIT $MAINPID</span><br><span class="line">PrivateTmp=true</span><br><span class="line"></span><br><span class="line">[Install]</span><br><span class="line">WantedBy=multi-user.target</span><br></pre></td></tr></table></figure>
<p><strong>重新加载systemd</strong><br>systemctl daemon-reload<br><strong>启动服务</strong><br>systemctl start nginx<br><strong>查看服务状态</strong><br>systemctl status nginx<br><strong>查看日志</strong><br>sudo journalctl -u nginx</p>
<h2 id="配置反向代理："><a href="#配置反向代理：" class="headerlink" title="配置反向代理："></a>配置反向代理：</h2><p>按照如下配置即可<sup id="fnref:4"><a href="#fn:4" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://nginx.org/en/docs/http/websocket.html
">[4]</span></a></sup>：</p>
<figure class="highlight nginx"><table><tr><td class="code"><pre><span class="line"><span class="attribute">location</span> /wsapp/ {</span><br><span class="line"> <span class="attribute">proxy_pass</span> http://wsbackend;</span><br><span class="line"> <span class="attribute">proxy_http_version</span> <span class="number">1</span>.<span class="number">1</span>;</span><br><span class="line"> <span class="attribute">proxy_set_header</span> Upgrade <span class="variable">$http_upgrade</span>;</span><br><span class="line"> <span class="attribute">proxy_set_header</span> Connection <span class="string">"Upgrade"</span>;</span><br><span class="line"> <span class="attribute">proxy_set_header</span> Host <span class="variable">$host</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h2 id="完整的站点配置"><a href="#完整的站点配置" class="headerlink" title="完整的站点配置:"></a>完整的站点配置:</h2><figure class="highlight nginx"><table><tr><td class="code"><pre><span class="line"><span class="section">server</span> {</span><br><span class="line"> <span class="attribute">listen</span> <span class="number">443</span> ssl http2;</span><br><span class="line"> <span class="attribute">listen</span> [::]:<span class="number">443</span> ssl http2;</span><br><span class="line"></span><br><span class="line"> <span class="attribute">server_name</span> node-nj.typedefine.cn;</span><br><span class="line"> </span><br><span class="line"> <span class="comment"># SSL</span></span><br><span class="line"> <span class="attribute">ssl_certificate</span> /root/.caddy/acme/acme-v02.api.letsencrypt.org/sites/node-nj.typedefine.cn/node-nj.typedefine.cn.crt;</span><br><span class="line"> <span class="attribute">ssl_certificate_key</span> /root/.caddy/acme/acme-v02.api.letsencrypt.org/sites/node-nj.typedefine.cn/node-nj.typedefine.cn.key;</span><br><span class="line"></span><br><span class="line"> <span class="comment"># security</span></span><br><span class="line"> <span class="attribute">include</span> nginxconfig.io/security.conf;</span><br><span class="line"> </span><br><span class="line"> <span class="attribute">location</span> / {</span><br><span class="line"> <span class="attribute">root</span> /root/docroot; </span><br><span class="line"> <span class="attribute">index</span> index.html index.htm index.php;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="attribute">location</span><span class="regexp"> ^~/ray</span> {</span><br><span class="line"> <span class="attribute">proxy_set_header</span> Upgrade <span class="variable">$http_upgrade</span>;</span><br><span class="line"> <span class="attribute">proxy_set_header</span> Connection <span class="string">"upgrade"</span>;</span><br><span class="line"> <span class="attribute">proxy_http_version</span> <span class="number">1</span>.<span class="number">1</span>;</span><br><span class="line"> <span class="attribute">proxy_set_header</span> X-Forwarded-For <span class="variable">$proxy_add_x_forwarded_for</span>;</span><br><span class="line"> <span class="attribute">proxy_set_header</span> Host <span class="variable">$host</span>;</span><br><span class="line"> <span class="attribute">proxy_pass</span> http://127.0.0.1:8442;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment"># handle .php</span></span><br><span class="line"> <span class="attribute">location</span> <span class="regexp">~ \.php$</span> {</span><br><span class="line"> <span class="attribute">include</span> nginxconfig.io/php_fastcgi.conf;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment"># additional config</span></span><br><span class="line"> <span class="attribute">include</span> nginxconfig.io/general.conf;</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://socket.io/docs/#Using-with-Express<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.nginx.com/blog/nginx-nodejs-websockets-socketio/<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.nginx.com/blog/websocket-nginx/<a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://nginx.org/en/docs/http/websocket.html<a href="#fnref:4" rev="footnote"> ↩</a></span></li><li id="fn:5"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">5.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.cmsky.com/caddy-proxy-shell/<a href="#fnref:5" rev="footnote"> ↩</a></span></li><li id="fn:6"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">6.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.digitalocean.com/community/tools/nginx<a href="#fnref:6" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>websokcet</category>
<category>nodejs</category>
</categories>
<tags>
<tag>websocket</tag>
<tag>nginx</tag>
<tag>Socket.io</tag>
<tag>Nodejs</tag>
<tag>express</tag>
</tags>
</entry>
<entry>
<title>快车道系列：理解文档对象模型DOM</title>
<url>/2020/03/30/Web%E5%BF%AB%E8%BD%A6%E9%81%93%E7%B3%BB%E5%88%97%EF%BC%9A%E7%90%86%E8%A7%A3%E6%96%87%E6%A1%A3%E5%AF%B9%E8%B1%A1%E6%A8%A1%E5%9E%8BDOM/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h3 id="DOM是什么？"><a href="#DOM是什么？" class="headerlink" title="DOM是什么？"></a>DOM是什么？</h3><p>Document Object Model<sup id="fnref:0"><a href="#fn:0" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://www.w3.org/TR/DOM-Level-2-Core/introduction.html
">[0]</span></a></sup>，简称DOM, 文档对象模型， 是一套抽象的、单一的、具有一致性的API。这一套API是与具体实现和具体语言无关的。在WHATAG CORE 3 Living Standard<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://dom.spec.whatwg.org/
">[1]</span></a></sup>上通过Web IDL定义给出。</p>
<p>HTML和定义良好的XML都是结构化，里面的内容、表现、属性都可以通过节点和对象的方式来表示，形成一棵树形结构（或者是森林）。而DOM所定义的API就是用来访问、操作、修改里面的数据。</p>
<a id="more"></a>
<p>举个例子：</p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">TABLE</span>></span></span><br><span class="line"><span class="tag"><<span class="name">ROWS</span>></span> </span><br><span class="line"><span class="tag"><<span class="name">TR</span>></span> </span><br><span class="line"><span class="tag"><<span class="name">TD</span>></span>Shady Grove<span class="tag"></<span class="name">TD</span>></span></span><br><span class="line"><span class="tag"><<span class="name">TD</span>></span>Aeolian<span class="tag"></<span class="name">TD</span>></span> </span><br><span class="line"><span class="tag"></<span class="name">TR</span>></span> </span><br><span class="line"><span class="tag"><<span class="name">TR</span>></span></span><br><span class="line"><span class="tag"><<span class="name">TD</span>></span>Over the River, Charlie<span class="tag"></<span class="name">TD</span>></span></span><br><span class="line"><span class="tag"><<span class="name">TD</span>></span>Dorian<span class="tag"></<span class="name">TD</span>></span> </span><br><span class="line"><span class="tag"></<span class="name">TR</span>></span> </span><br><span class="line"><span class="tag"></<span class="name">ROWS</span>></span></span><br><span class="line"><span class="tag"></<span class="name">TABLE</span>></span></span><br></pre></td></tr></table></figure>
<p>那么，如果用DOM模型来表示这个表格，就应该下面这个样子的：</p>
<img src="/2020/03/30/Web%E5%BF%AB%E8%BD%A6%E9%81%93%E7%B3%BB%E5%88%97%EF%BC%9A%E7%90%86%E8%A7%A3%E6%96%87%E6%A1%A3%E5%AF%B9%E8%B1%A1%E6%A8%A1%E5%9E%8BDOM/table.gif" class="" title="DOM表示的结构">
<p>DOM中的对象就是传统意义上面向对象设计中的意思，在DOM语义中，节点不仅仅是树状的结构，还表示了这个节点所代表的各种操作和组成它的各种数据（比如它有子节点，有属性，有各种Function, 有自己的标识）。 DOM定义了如下3个方面：</p>
<ul>
<li>表示和操作文档的接口和对象</li>
<li>这些接口和对象的语义（包括行为和属性）</li>
<li>这些接口和对象之间的关系和协作</li>
</ul>
<h3 id="DOM标准的版本"><a href="#DOM标准的版本" class="headerlink" title="DOM标准的版本"></a>DOM标准的版本</h3><h4 id="DOM-1-级"><a href="#DOM-1-级" class="headerlink" title="DOM 1 级"></a>DOM 1 级</h4><p>1998 年 10 月，W3C 推出 DOM 1.0 版本规范，作为推荐标准进行正式发布，主要包括两个子规范。</p>
<ul>
<li>DOM Core（核心部分）：把 XML 文档设计为树形节点结构，并为这种结构的运行机制制订了一套规范化标准，同时定义了创建、编辑、操纵这些文档结构的基本属性和方法。</li>
<li>DOM HTML：针对 HTML 文档、标签集合，以及与个别 HTML 标签相关的元素定义了对象、属性和方法。</li>
</ul>
<h4 id="DOM-2-级"><a href="#DOM-2-级" class="headerlink" title="DOM 2 级"></a>DOM 2 级</h4><p>2000 年 11 月，W3C 正式发布了更新后的 DOM 核心部分，并在这次发布中添加了一些新规范，于是人们就把这次发布的 DOM 称为 2 级规范。</p>
<p>2003 年 1 月，W3C 又正式发布了对 DOM HTML 子规范的修订，添加了针对 HTML 4.01 和 XHTML 1.0 版本文档中很多对象、属性和方法。W3C 把新修订的 DOM 规范同义称为 DOM 2.0 推荐版本，该版本主要包括 6 个推荐子规范。</p>
<ul>
<li>DOM2 Core：继承于 DOM Core 子规范，规定了 DOM 文档结构模型，添加了更多的特性，如针对命名空间的方法等。</li>
<li>DOM2 HTML：继承于 DOM HTML，规定了针对 HTML 的 DOM 文档结构模型，并添加了一些属性。</li>
<li>DOM2 Events：规定了与鼠标相关的事件（包括目标、捕获冒泡和取消）的控制机制，但不包含与键盘相关事件的处理部分。</li>
<li>DOM2 Style（或 DOM2 CSS）：提供了访问和操纵所有与 CSS 相关的样式及规则的能力。</li>
<li>DOM2 Traversal 和 DOM2 Range：DOM2 Traversal 规范允许开发人员通过迭代方式访问 DOM，DOM2 Range 规范允许对指定范围的内容进行操作。</li>
<li>DOM2 Views：提供了访问和更新文档表现（视图）的能力。</li>
</ul>
<h4 id="DOM-3-级"><a href="#DOM-3-级" class="headerlink" title="DOM 3 级"></a>DOM 3 级</h4><p>2004 年 4 月，W3C 发布了 DOM3 版本。DOM3 版本主要包括以下 3 个推荐子规范。</p>
<ul>
<li>DOM3 Core：继承于 DOM2 Core，并添加了更多的新方法和属性，同时修改了已有的一些方法。</li>
<li>DOM3 Load and Save：提供将 XML 文档的内容加载到 DOM 文档中，以及将 DOM 文档序列化为 XML 文档的能力。</li>
<li>DOM3 Validation：提供了确保动态生成的文档的有效性的能力，即如何符合文档类型声明。</li>
</ul>
<h4 id="DOM-CORE之外的扩展"><a href="#DOM-CORE之外的扩展" class="headerlink" title="DOM CORE之外的扩展"></a>DOM CORE之外的扩展</h4><p>DOM标准定义了核心DOM的模型和接口。但是不同的厂商和语言特定的库都对DOM做了扩展。有的专门用来解析XML DOM。有的用来解析HTML DOM。比如在Java中就有JDOM、DOM4J等库来解析DOM。</p>
<h3 id="DOM的基本类型"><a href="#DOM的基本类型" class="headerlink" title="DOM的基本类型"></a>DOM的基本类型</h3><p>以下表格参考<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Introduction">[2]</span></a></sup>：</p>
<!-- >
| 类型 | 描述 |
| ------ | ------ |
| **Document** | 文档本身。 |
| **Node** | 文档中的所有对象都是一个节点。要么是元素节点、要么是文本节点，要么就是属性节点。 |
| **Element** | 元素本身实际上也是一个节点，但是同时也实现了其他的接口，比如HTMLElement，或者其他接口，比如HTMLTableElement 用来表示 <table> 表格。 |
| **NodeList** | nodeList 是一个元素的数组，如从 document.getElementsByTagName() 方法返回的就是这种类型。 |
| **Attribute** | 当 attribute 通过成员函数 (例如，通过 createAttribute()方法) 返回时，它是一个为属性暴露出专门接口的对象引用。DOM中的属性也是节点，就像元素一样，只不过您可能会很少使用它。 |
| **NamedNodeMap** | namedNodeMap 和数组类似，但是条目是由name或index访问的，虽然后一种方式仅仅是为了枚举方便，因为在 list 中本来就没有特定的顺序。 出于这个目的， namedNodeMap 有一个 item() 方法，你也可以从 namedNodeMap 添加或移除条目。 |
<-->
<h3 id="如何用Javascript操作HTML"><a href="#如何用Javascript操作HTML" class="headerlink" title="如何用Javascript操作HTML"></a>如何用Javascript操作HTML</h3><p>下面给出一段代码，在浏览器中直接操作HTML DOM。其中整个DOM对象直接通过document全局变量获得。</p>
<figure class="highlight javascript"><table><tr><td class="code"><pre><span class="line"><span class="keyword">const</span> p1 = <span class="built_in">document</span>.createElement(<span class="string">"p"</span>);</span><br><span class="line"><span class="keyword">const</span> p1_text = <span class="built_in">document</span>.createTextNode(xmlDoc.getElementsByTagName(<span class="string">"to"</span>)[<span class="number">0</span>].childNodes[<span class="number">0</span>].nodeValue);</span><br><span class="line">p1.appendChild(p1_text);</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> p2 = <span class="built_in">document</span>.createElement(<span class="string">"p"</span>);</span><br><span class="line"><span class="keyword">const</span> p2_text = <span class="built_in">document</span>.createTextNode(xmlDoc.getElementsByTagName(<span class="string">"from"</span>)[<span class="number">0</span>].childNodes[<span class="number">0</span>].nodeValue);</span><br><span class="line">p2.appendChild(p2_text);</span><br><span class="line"></span><br><span class="line"><span class="keyword">const</span> p3 = <span class="built_in">document</span>.createElement(<span class="string">"p"</span>);</span><br><span class="line"><span class="keyword">const</span> p3_text = <span class="built_in">document</span>.createTextNode(xmlDoc.getElementsByTagName(<span class="string">"body"</span>)[<span class="number">0</span>].childNodes[<span class="number">0</span>].nodeValue);</span><br><span class="line">p3.appendChild(p3_text);</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> nodeEle = <span class="built_in">document</span>.getElementsByClassName(<span class="string">'note'</span>);</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> body = <span class="built_in">document</span>.body;</span><br><span class="line">body.appendChild(p1);</span><br><span class="line">body.appendChild(p2);</span><br><span class="line">body.appendChild(p3);</span><br></pre></td></tr></table></figure>
<p><strong><em>TO BE CONTINUED</em></strong></p>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:0"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">0.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.w3.org/TR/DOM-Level-2-Core/introduction.html<a href="#fnref:0" rev="footnote"> ↩</a></span></li><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://dom.spec.whatwg.org/<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Introduction<a href="#fnref:2" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>web</category>
</categories>
<tags>
<tag>web</tag>
<tag>DOM</tag>
<tag>Fast Track</tag>
<tag>快车道</tag>
</tags>
</entry>
<entry>
<title>Nacos不支持MYSQL 8的问题</title>
<url>/2020/04/29/Nacos%E4%B8%8D%E6%94%AF%E6%8C%81MYSQL-8%E7%9A%84%E9%97%AE%E9%A2%98/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><p>Nacos遇到MySQL 8版本连接会报错，并且MySQL多次连接报错之后会拒绝连接。遇到这种情况之后登陆到mysql，然后flush hosts。</p>
<p>在Nacos 1.2.0版本中解决了连接MySQL 8报错的问题<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://github.com/alibaba/nacos/issues/2042
">[1]</span></a></sup>.</p>
<h2 id="1-下载对应版本的connector"><a href="#1-下载对应版本的connector" class="headerlink" title="1. 下载对应版本的connector"></a>1. 下载对应版本的connector</h2><p>比如mysql-connector-java-8.0.17.jar<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://downloads.mysql.com/archives/c-j/">[2]</span></a></sup>。注意这里加载的jar包对应的版本必须要和MySQL版本完全一致。否则可能出现报错。</p>
<img src="/2020/04/29/Nacos%E4%B8%8D%E6%94%AF%E6%8C%81MYSQL-8%E7%9A%84%E9%97%AE%E9%A2%98/image2020-4-29_22-12-27.png" class="" title="download_mysql_connector">
<h2 id="2-配置nacos的mysql插件"><a href="#2-配置nacos的mysql插件" class="headerlink" title="2. 配置nacos的mysql插件"></a>2. 配置nacos的mysql插件</h2><p>在nacos目录下新建 plugins/mysql/mysql-connector-java-8.0.17.jar</p>
<img src="/2020/04/29/Nacos%E4%B8%8D%E6%94%AF%E6%8C%81MYSQL-8%E7%9A%84%E9%97%AE%E9%A2%98/image2020-4-29_19-34-11.png" class="" title="console_nacos">
<h2 id="3-配置连接数据库，然后启动nacos"><a href="#3-配置连接数据库，然后启动nacos" class="headerlink" title="3. 配置连接数据库，然后启动nacos"></a>3. 配置连接数据库，然后启动nacos</h2><div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://github.com/alibaba/nacos/issues/2042<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://downloads.mysql.com/archives/c-j/<a href="#fnref:2" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>框架</category>
<category>微服务</category>
</categories>
<tags>
<tag>微服务</tag>
<tag>Nacos</tag>
<tag>MySQL</tag>
</tags>
</entry>
<entry>
<title>MongoDB建模的简单原则</title>
<url>/2020/12/07/MongoDB%E5%BB%BA%E6%A8%A1%E7%9A%84%E4%B8%80%E4%BA%9B%E7%AE%80%E5%8D%95%E5%8E%9F%E5%88%99/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h1 id="建模原则："><a href="#建模原则：" class="headerlink" title="建模原则："></a>建模原则：</h1><ol>
<li><p>数据库名字、Collection名字、属性字段均小写，并使用下划线连接。<br>但是对应的Java DAO对象的属性字段需要按照驼峰命名法书写，并且每个字段都需要加上@Field注解，即使数据库字段和名字是完全一样的。</p>
</li>
<li><p>对于大规模数据，字段存储的时候使用缩写。</p>
</li>
<li><p>如果嵌套的Document数量会不断的增长或者频繁发生变化，那么可以考虑单独存Collection</p>
<p> The maximum BSON document size is 16 megabytes.</p>
</li>
<li><p>如果从逻辑上对象是从属并且是1:1的关系，直接使用嵌套对象。</p>
<p> 但是如果多个地方使用了同一个对象，比如A包含了B， C也包含了B，那么可以考虑将B单独建集合，然后通过DBRefs（对于Spring Data Mongo来说就是注解@DBRef）来引用。</p>
</li>
</ol>
<h1 id="建索引原则："><a href="#建索引原则：" class="headerlink" title="建索引原则："></a>建索引原则：</h1><ol>
<li><p>复合索引不在Java POJO对象上加注解完成。而是手工创建。</p>
</li>
<li><p>对于复合索引，要将能够快速缩小数据范围（命中率更高）的字段放在复合索引前面并排序。</p>
</li>
<li><p>稀疏索引，如果某字段对于大部分数据都可能不存在，或者为null， 要指定该索引为稀疏索引</p>
</li>
</ol>
<h1 id="参考文档："><a href="#参考文档：" class="headerlink" title="参考文档："></a>参考文档：</h1><p><a href="https://docs.mongodb.com/manual/core/document/" target="_blank" rel="noopener">https://docs.mongodb.com/manual/core/document/</a></p>
]]></content>
<categories>
<category>架构</category>
</categories>
<tags>
<tag>MongoDB</tag>
<tag>建模</tag>
<tag>NoSQL</tag>
</tags>
</entry>
<entry>
<title>Maven配置部署指北</title>
<url>/2021/03/29/Maven%E9%85%8D%E7%BD%AE%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8C%97/</url>
<content><
">[2]</span></a></sup>，快照，是一种特殊的版本，指定了某个当前的开发进度的副本。不同于常规的版本，Maven 每次构建都会在远程仓库中检查新的快照。比如发布了一个app-1.0-SNAPSHOT版本，那么maven构件的时候，每次都会去仓库检查是否有最新的版本已经被发布。如果有，则替换。如果是release仓库，那么除非版本升级，否则maven构件的时候不会去检查最新的jar包，即便是在nexus仓库中已经更新了。</p>
<p>maven中的仓库分为两种<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="[maven快照版本和发布版本](https://www.cnblogs.com/wuchanming/p/5484091.html)
">[3]</span></a></sup>，snapshot快照仓库和release发布仓库。snapshot快照仓库用于保存开发过程中的不稳定版本，release正式仓库则是用来保存稳定的发行版本。</p>
<p>定义一个组件/模块为快照版本，只需要在pom文件中在该模块的版本号后加上-SNAPSHOT即可(注意这里必须是大写)，如下：</p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">groupId</span>></span>com.schdri<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"><span class="tag"><<span class="name">artifactId</span>></span>bimgis<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"><span class="tag"><<span class="name">version</span>></span>3.10.0-SNAPSHOT<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"><span class="tag"><<span class="name">name</span>></span>${project.artifactId}<span class="tag"></<span class="name">name</span>></span></span><br><span class="line"><span class="tag"><<span class="name">packaging</span>></span>pom<span class="tag"></<span class="name">packaging</span>></span></span><br><span class="line"><span class="tag"><<span class="name">organization</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">name</span>></span>pig4cloud<span class="tag"></<span class="name">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">url</span>></span>https://www.pig4cloud.com<span class="tag"></<span class="name">url</span>></span></span><br><span class="line"><span class="tag"></<span class="name">organization</span>></span></span><br></pre></td></tr></table></figure>
<a id="more"></a>
<h2 id="Snapshot版本更新策略"><a href="#Snapshot版本更新策略" class="headerlink" title="Snapshot版本更新策略"></a>Snapshot版本更新策略</h2><p>maven会在什么时候检查更新jar包呢？可以在<repository>节点下面配置更新策略：</p>
<img src="/2021/03/29/Maven%E9%85%8D%E7%BD%AE%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8C%97/image2021-3-22_9-17-19.png" class="" title>
<p>也可以在使用maven的时候，通过命令行参数强制更新：</p>
<img src="/2021/03/29/Maven%E9%85%8D%E7%BD%AE%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8C%97/image2021-3-22_9-47-50.png" class="" title>
<h1 id="如何在POM中使用多个仓库"><a href="#如何在POM中使用多个仓库" class="headerlink" title="如何在POM中使用多个仓库"></a>如何在POM中使用多个仓库</h1><h2 id="在POM中声明"><a href="#在POM中声明" class="headerlink" title="在POM中声明"></a>在POM中声明</h2><p>只对当前工程生效<sup id="fnref:5"><a href="#fn:5" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="[Setting up Multiple Repositories](http://maven.apache.org/guides/mini/guide-multiple-repositories.html)
">[5]</span></a></sup></p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">project</span>></span></span><br><span class="line">...</span><br><span class="line"> <span class="tag"><<span class="name">repositories</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>my-repo1<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">name</span>></span>your custom repo<span class="tag"></<span class="name">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">url</span>></span>http://jarsm2.dyndns.dk<span class="tag"></<span class="name">url</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>my-repo2<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">name</span>></span>your custom repo<span class="tag"></<span class="name">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">url</span>></span>http://jarsm2.dyndns.dk<span class="tag"></<span class="name">url</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repositories</span>></span></span><br><span class="line">...</span><br><span class="line"><span class="tag"></<span class="name">project</span>></span></span><br></pre></td></tr></table></figure>
<p>每一个工程的POM都会继承自Super POM， 除非显示的声明不继承。</p>
<p>Super POM中的仓库包含了默认的maven 中央仓库如下<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="[超级POM](http://maven.apache.org/ref/3.6.3/maven-model-builder/super-pom.html)
">[1]</span></a></sup>：</p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">repositories</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>central<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">name</span>></span>Central Repository<span class="tag"></<span class="name">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">url</span>></span>https://repo.maven.apache.org/maven2<span class="tag"></<span class="name">url</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">layout</span>></span>default<span class="tag"></<span class="name">layout</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">snapshots</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">enabled</span>></span>false<span class="tag"></<span class="name">enabled</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">updatePolicy</span>></span>always<span class="tag"></<span class="name">updatePolicy</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">snapshots</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repository</span>></span></span><br><span class="line"><span class="tag"></<span class="name">repositories</span>></span></span><br></pre></td></tr></table></figure>
<h2 id="在settings-xml中配置"><a href="#在settings-xml中配置" class="headerlink" title="在settings.xml中配置"></a>在settings.xml中配置</h2><p>配置如下<sup id="fnref:6"><a href="#fn:6" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="[maven Settings](http://maven.apache.org/ref/3.2.2/maven-settings/settings.html)
">[6]</span></a></sup><br>${user.home}/.m2/settings.xml or ${maven.home}/conf/settings.xml中声明仓库：</p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">settings</span>></span></span><br><span class="line"> ...</span><br><span class="line"> <span class="tag"><<span class="name">profiles</span>></span></span><br><span class="line"> ...</span><br><span class="line"> <span class="tag"><<span class="name">profile</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>myprofile<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">repositories</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>my-repo2<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">name</span>></span>your custom repo<span class="tag"></<span class="name">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">url</span>></span>http://jarsm2.dyndns.dk<span class="tag"></<span class="name">url</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repositories</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">profile</span>></span></span><br><span class="line"> ...</span><br><span class="line"> <span class="tag"></<span class="name">profiles</span>></span></span><br><span class="line"> </span><br><span class="line"> <span class="tag"><<span class="name">activeProfiles</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">activeProfile</span>></span>myprofile<span class="tag"></<span class="name">activeProfile</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">activeProfiles</span>></span></span><br><span class="line"> ...</span><br><span class="line"><span class="tag"></<span class="name">settings</span>></span></span><br></pre></td></tr></table></figure>
<h2 id="仓库解析的顺序"><a href="#仓库解析的顺序" class="headerlink" title="仓库解析的顺序"></a>仓库解析的顺序</h2><p>Remote repository URLs are queried in the following order for artifacts until one returns a valid result:</p>
<p>effective settings:<br>Global settings.xml<br>User settings.xml<br>local effective build POM:<br>Local pom.xml<br>Parent POMs, recursively<br>Super POM<br>effective POMs from dependency path to the artifact.<br>For each of these locations, the repositories within the profiles are queried first in the order outlined at Introduction to build profiles.</p>
<p>Before downloading from a repository, mirrors configuration is applied.</p>
<p>Effective settings and local build POM, with profile taken into account, can easily be reviewed to see their repositories order with mvn help:effective-settings and mvn help:effective-pom -Dverbose.</p>
<h1 id="仓库镜像mirror"><a href="#仓库镜像mirror" class="headerlink" title="仓库镜像mirror"></a>仓库镜像mirror</h1><p>mirror<sup id="fnref:4"><a href="#fn:4" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="[maven官方文档 Using Mirrors for Repositories](http://maven.apache.org/guides/mini/guide-mirror-settings.html)
">[4]</span></a></sup>配置只能存在于settings.xml中，用来指定对某个repository的镜像，一个repository只能指定一个mirror，如果配置多个mirror，那么maven不会做聚合，并且选择匹配的第一个。</p>
<p>如果强制要将所有的仓库都镜像的话，可以将<mirrorOf>标签设置为*</p>
<p>一些更多的匹配规则，从maven 2.0.9+版本以后开始支持：</p>
<ul>
<li>*星号，匹配所有</li>
<li>external:*， 除了使用localhost和file://开始的仓库</li>
<li>repo1,repo2 多个</li>
<li>*,!repo1 非</li>
</ul>
<h1 id="Nexus中Maven仓库类型"><a href="#Nexus中Maven仓库类型" class="headerlink" title="Nexus中Maven仓库类型"></a>Nexus中Maven仓库类型</h1><p>hosted 本地存储。像官方仓库一样提供本地私库功能<br>proxy 提供代理其它仓库的类型<br>group 组类型，能够组合多个仓库为一个地址提供服务</p>
<h1 id="如何将artifact发布到nexus仓库"><a href="#如何将artifact发布到nexus仓库" class="headerlink" title="如何将artifact发布到nexus仓库"></a>如何将artifact发布到nexus仓库</h1><h2 id="增加Maven-POM中的发布配置"><a href="#增加Maven-POM中的发布配置" class="headerlink" title="增加Maven POM中的发布配置"></a>增加Maven POM中的发布配置</h2><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">distributionManagement</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">repository</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>nexus<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">name</span>></span>maven-releases<span class="tag"></<span class="name">name</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">url</span>></span>http://192.168.201.155:8081/repository/bimgis/<span class="tag"></<span class="name">url</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">repository</span>></span></span><br><span class="line"><span class="tag"></<span class="name">distributionManagement</span>></span></span><br></pre></td></tr></table></figure>
<h2 id="在settings-xml中增加鉴权"><a href="#在settings-xml中增加鉴权" class="headerlink" title="在settings.xml中增加鉴权"></a>在settings.xml中增加鉴权</h2><p>如果使用IDEA，那么请查看</p>
<img src="/2021/03/29/Maven%E9%85%8D%E7%BD%AE%E9%83%A8%E7%BD%B2%E6%8C%87%E5%8C%97/image2021-3-16_9-13-24.png" class="" title>
<p>找到对应的settings.xml中，增加如下配置：</p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="comment"><!--配置服务端的一些设置。一些设置如安全证书不应该和pom.xml一起分发。这种类型的信息应该存在于构建服务器上的settings.xml文件中。--></span></span><br><span class="line"><span class="tag"><<span class="name">servers</span>></span></span><br><span class="line"> <span class="comment"><!--服务器元素包含配置服务器时需要的信息 --></span></span><br><span class="line"> <span class="tag"><<span class="name">server</span>></span></span><br><span class="line"> <span class="comment"><!--这是server的id（注意不是用户登陆的id），该id与distributionManagement中repository元素的id相匹配。--></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>nexus<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="comment"><!--鉴权用户名。鉴权用户名和鉴权密码表示服务器认证所需要的登录名和密码。 --></span></span><br><span class="line"> <span class="tag"><<span class="name">username</span>></span>admin<span class="tag"></<span class="name">username</span>></span></span><br><span class="line"> <span class="comment"><!--鉴权密码 。鉴权用户名和鉴权密码表示服务器认证所需要的登录名和密码。密码加密功能已被添加到2.1.0 +。详情请访问密码加密页面--></span></span><br><span class="line"> <span class="tag"><<span class="name">password</span>></span>Admin@123<span class="tag"></<span class="name">password</span>></span></span><br><span class="line"> <span class="comment"><!--鉴权时使用的私钥位置。和前两个元素类似，私钥位置和私钥密码指定了一个私钥的路径（默认是${user.home}/.ssh/id_dsa）以及如果需要的话，一个密语。将来passphrase和password元素可能会被提取到外部，但目前它们必须在settings.xml文件以纯文本的形式声明。 --></span></span><br><span class="line"> <span class="comment"><!-- privateKey>${usr.home}/.ssh/id_dsa</privateKey --></span></span><br><span class="line"> <span class="comment"><!--鉴权时使用的私钥密码。--></span></span><br><span class="line"> <span class="comment"><!--passphrase>some_passphrase</passphrase --></span></span><br><span class="line"> <span class="comment"><!--文件被创建时的权限。如果在部署的时候会创建一个仓库文件或者目录，这时候就可以使用权限（permission）。这两个元素合法的值是一个三位数字，其对应了unix文件系统的权限，如664，或者775。 --></span></span><br><span class="line"> <span class="tag"><<span class="name">filePermissions</span>></span>664<span class="tag"></<span class="name">filePermissions</span>></span></span><br><span class="line"> <span class="comment"><!--目录被创建时的权限。 --></span></span><br><span class="line"> <span class="tag"><<span class="name">directoryPermissions</span>></span>775<span class="tag"></<span class="name">directoryPermissions</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">server</span>></span></span><br><span class="line"><span class="tag"></<span class="name">servers</span>></span></span><br></pre></td></tr></table></figure>
<p>如果使用mvn命令行，请参考MVN_HOME环境变量配置。</p>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="http://maven.apache.org/ref/3.6.3/maven-model-builder/super-pom.html" target="_blank" rel="noopener">超级POM</a><a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="https://www.runoob.com/maven/maven-snapshots.html" target="_blank" rel="noopener">Maven快照</a><a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="https://www.cnblogs.com/wuchanming/p/5484091.html" target="_blank" rel="noopener">maven快照版本和发布版本</a><a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="http://maven.apache.org/guides/mini/guide-mirror-settings.html" target="_blank" rel="noopener">maven官方文档 Using Mirrors for Repositories</a><a href="#fnref:4" rev="footnote"> ↩</a></span></li><li id="fn:5"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">5.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="http://maven.apache.org/guides/mini/guide-multiple-repositories.html" target="_blank" rel="noopener">Setting up Multiple Repositories</a><a href="#fnref:5" rev="footnote"> ↩</a></span></li><li id="fn:6"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">6.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="http://maven.apache.org/ref/3.2.2/maven-settings/settings.html" target="_blank" rel="noopener">maven Settings</a><a href="#fnref:6" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Maven</category>
<category>DevOps</category>
</categories>
<tags>
<tag>Maven</tag>
<tag>DevOps</tag>
</tags>
</entry>
<entry>
<title>Kafka调研（3）：多点通信方案的设计-1</title>
<url>/2020/02/17/Kafka%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A3%EF%BC%9A%E5%9F%BA%E4%BA%8EKafka%E5%A4%9A%E7%82%B9%E9%80%9A%E4%BF%A1%E6%96%B9%E6%A1%88/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h1 id="1-引言"><a href="#1-引言" class="headerlink" title="1. 引言"></a>1. 引言</h1><p>在华为核心网和无线领域有几个重量级的产品比如U2000/M2000等网管系统<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://ishare.iask.sina.com.cn/f/LmFNcBVsuz.html
">[1]</span></a></sup>。里面的底层通信就是一套基于MDP(Message Dispatching Process)的星形拓扑通信系统。不同的业务进程之间通过消息(REQ/RSP)来通信，而不是通信RPC或者库的依赖。</p>
<p>由于年代久远，和现在主流的MQ技术相比有一些不同的地方，随便说几点:</p>
<ol>
<li><p><strong><em>消息序列化只支持ASN.1</em></strong>. 当然好处在于提供一套抽象的语义定义，并且大家只需要传递ASN.1文件即可。或者由一个公共方将ASN.1文件编程对应的模型和支撑库提供给各个端对进行开发。但是同时呢，不好的地方也很明显，一方面是技术过时，再一个就是需要一个专门的团队来维护ASN.1定义和相应的工具；另一方面在于管理上没有形成一个好的ASN.1的分发机制。导致版本之间定义很混乱。对于前后向兼容性问题很难做到。当然，在网上也可以找到一些开源的ASN.1 Compiler<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://github.com/vlm/asn1c
">[3]</span></a></sup>。能不能用不清楚。</p>
</li>
<li><p><strong><em>MDP并不支持消息的缓存和队列机制，本身也不提供可靠性保证。</em></strong> 那么如何来提供不丢失的消息呢？ 在华为，答案是提供一个框架层，业务团队并不直接使用MDP提供的库和接口，而是使用封装之后的一套框架，这套框架中提供了可靠性事件的发送的。实现的机制就是在发送之前存库。在接收业务处理完成之后，再从数据库中移除。当然这套机制也出现过不少问题，比如消息积压，数据库的内容不断上涨，甚至在局点可能出现过数据库表过大，导致其他业务访问很慢，甚至于极端情况下，整个数据库崩掉的情况都有。</p>
</li>
<li><p><strong><em>MDP无法水平扩展，当消息出现峰值的时候，MDP有可能出现丢包的问题。</em></strong> 这个问题也比较头疼。这是从一开始设计的时候就没有考虑的问题。考虑到U2000/M2000对应的数据量实际上是很大的。假如MDP的处理峰值在于2000/s，每秒钟2W条消息尽力保证不丢包。 这个阈值总会在未来的某一天会被打破，那么这个时候问题就出来了，系统会出现频繁丢包，系统业务频繁故障，但是由于底层数据通道，在应用层面几乎无法做到一个比较好的解决方案。</p>
</li>
<li><p><strong><em>MDP采用ACE<sup id="fnref:4"><a href="#fn:4" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://www.dre.vanderbilt.edu/~schmidt/TAO-overview.html
">[4]</span></a></sup>，这一套框架比较复杂。</em></strong>当然整体的U2000/M2000都是基于这个来开发，导致比较重型。<br>另外就是在U2000/M2000北向领域，采用了大量的ACE/TAO/Event Channel机制。灵活性不够。<br><strong><em>为什么说CORBA是一个很不好的技术，最终会被时代所抛弃? 可以看这篇论文</em></strong>。<a href="/2020/02/17/Kafka%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A3%EF%BC%9A%E5%9F%BA%E4%BA%8EKafka%E5%A4%9A%E7%82%B9%E9%80%9A%E4%BF%A1%E6%96%B9%E6%A1%88/1142031.1142044.pdf" title="下载链接">下载链接</a></p>
</li>
<li><p><strong><em>当前通信架构转异构通信架构的难度。</em></strong>对于U2000/M2000来说，所有的开发支撑库都需要自己来完成。当然，对于华为来说，这个不是问题。问题这是人力预算的问题。</p>
</li>
</ol>
<p>上面说的一些问题在华为内部看来，有一些并不是问题，有一些问题可以绕过。有几个点：</p>
<ol>
<li><strong><em>客户的OSS/BSS系统都是基于CORBA调用。</em></strong></li>
<li><strong><em>ITU的标准定义就是基于ASN.1模型发布。</em></strong></li>
<li><strong><em>整个产品技术切换的成本太高, 所以要保护既有投资，CORBA技术带来的复杂度也可以成为一种护城河。</em></strong></li>
<li><strong><em>整个业务开发团队转型所面临的学习成本、人才招聘、人力缺口等等。</em></strong></li>
</ol>
<p>那么，从这几点来考虑，保持当前技术稳定，保持客户服务的稳定变是重点。所以，对于大企业来说，真的不是某个技术好就要上的。上面这个并非是吐槽，而是纯粹从技术上来分析我所经历过的产品在架构上的一些问题。</p>
<p>那么回过头来，那么如何通过messge queue来解耦不同系统（同构的还是异构的)来交换和通信来实现多异构同构系统之间的 星形通信呢？ 我们来做一个简单的设计。</p>
<a id="more"></a>
<h1 id="2-使用Kafka作为消息存储和转发中心"><a href="#2-使用Kafka作为消息存储和转发中心" class="headerlink" title="2. 使用Kafka作为消息存储和转发中心"></a>2. 使用Kafka作为消息存储和转发中心</h1><p>Kafka的设计思想相对来说比较简单，Broker本身也不存储客户端关于如何读取数据以及读取位置的信息。这些信息都是保存在ZooKeeper集群当中。</p>
<p><strong><em>几个关键点</em></strong>：</p>
<ol>
<li>每个应用都至少有一个Producer和Consumer连接到Kafka集群</li>
<li>每个应用都有唯一的独立的ID（SystemId），此ID通过公共的协议来约定（Protocol Buffer）。</li>
<li>每个应用收到消息之后，检查SystemId是否匹配，如果匹配则处理消息</li>
<li>应用在处理消息的时候，检查消息类型，并在分派给应用层逻辑来进一步处理。</li>
<li>处理完成之后commit消息<img src="/2020/02/17/Kafka%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A3%EF%BC%9A%E5%9F%BA%E4%BA%8EKafka%E5%A4%9A%E7%82%B9%E9%80%9A%E4%BF%A1%E6%96%B9%E6%A1%88/20200217114045.png" class="" title="multi-point-comm">
</li>
</ol>
<h1 id="3-接入的应用在业务层自己分派消息"><a href="#3-接入的应用在业务层自己分派消息" class="headerlink" title="3. 接入的应用在业务层自己分派消息"></a>3. 接入的应用在业务层自己分派消息</h1><p>每个接入的应用之后接收到消息之后：</p>
<ol>
<li>二进制消息解包成Message</li>
<li>根据消息类型（或者业务ID）并分派任务</li>
<li>可以支持更复杂的应用层消息路由<br> 通过<br> （system_id, app_id, domain_id, module_id, function_id）<br> 可以支持超级复杂的消息路由，用来找到对应的消息处理函数。</li>
</ol>
<img src="/2020/02/17/Kafka%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A3%EF%BC%9A%E5%9F%BA%E4%BA%8EKafka%E5%A4%9A%E7%82%B9%E9%80%9A%E4%BF%A1%E6%96%B9%E6%A1%88/20200217114417.png" class="" title="multi-point-dispatch">
<p>TBD.</p>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://ishare.iask.sina.com.cn/f/LmFNcBVsuz.html<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.itu.int/en/ITU-T/asn1/Pages/introduction.aspx<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://github.com/vlm/asn1c<a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://www.dre.vanderbilt.edu/~schmidt/TAO-overview.html<a href="#fnref:4" rev="footnote"> ↩</a></span></li><li id="fn:5"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">5.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://dl.acm.org/doi/pdf/10.1145/1142031.1142044?download=true<a href="#fnref:5" rev="footnote"> ↩</a></span></li><li id="fn:6"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">6.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://developers.google.com/protocol-buffers<a href="#fnref:6" rev="footnote"> ↩</a></span></li><li id="fn:7"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">7.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://www.typedefine.cn/2020/02/14/Kafka%E8%B0%83%E7%A0%94%E6%8A%A5%E5%91%8A2-%E6%B6%88%E6%81%AF%E5%BA%8F%E5%88%97%E5%8C%96%E6%A1%86%E6%9E%B6%E6%A8%AA%E5%90%91%E5%AF%B9%E6%AF%94/<a href="#fnref:7" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Kafka</category>
<category>Architecture</category>
</categories>
<tags>
<tag>Message Queue</tag>
<tag>Kafka</tag>
<tag>Topo Comm model</tag>
</tags>
</entry>
<entry>
<title>通过maven自动化构建protobuf class代码</title>
<url>/2020/02/10/%E9%80%9A%E8%BF%87maven%E8%87%AA%E5%8A%A8%E6%9E%84%E5%BB%BAprotobuf/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h2 id="通过maven自动化构建protocal-buffer"><a href="#通过maven自动化构建protocal-buffer" class="headerlink" title="通过maven自动化构建protocal buffer"></a>通过maven自动化构建protocal buffer</h2><p>当对代码进行工程构建的时候，如果能够尽可能的自动化，这样对于提升整个软件质量、节省人力成本、减少因为人工干预导致的各种不可预知的错误是很好。<br>从protobuf的官方文档来看，通过cmd调用protoc当然可以生成相应的代码，但是对构建不够友好。于是，可以考虑通过maven来进行自动化构建，在启动maven构建的时候，自动调用protoc，生成代码，并打包成相应的jar包，集成到工程发布包中。<br>这里有2种方式，如下。</p>
<a id="more"></a>
<h2 id="第1种方式：使用protobuf-maven-plugin"><a href="#第1种方式：使用protobuf-maven-plugin" class="headerlink" title="第1种方式：使用protobuf-maven-plugin"></a>第1种方式：使用protobuf-maven-plugin</h2><h3 id="配置插件protobuf-maven-plugin"><a href="#配置插件protobuf-maven-plugin" class="headerlink" title="配置插件protobuf-maven-plugin"></a>配置插件protobuf-maven-plugin</h3><p>先从官网<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="下载protoc: https://developers.google.com/protocol-buffers/docs/downloads
">[2]</span></a></sup>上将protobuf的工具包下载下来，然后拷贝到工程目录下，配置POM文件如下, 通过相对路径引用protoc.exe文件。<br>.protow文件默认的路径在src/main/proto文件中。下面所有的子目录都会被当成包结构。</p>
<p>protobuf-maven-plugin<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="插件使用：https://www.xolstice.org/protobuf-maven-plugin/plugin-info.html
">[1]</span></a></sup>的maven配置如下：</p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">build</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugins</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-compiler-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.xolstice.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>protobuf-maven-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">checkStaleness</span>></span>false<span class="tag"></<span class="name">checkStaleness</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">staleMillis</span>></span>10000<span class="tag"></<span class="name">staleMillis</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">protocExecutable</span>></span>${basedir}\..\protobuf\bin\protoc.exe<span class="tag"></<span class="name">protocExecutable</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">useArgumentFile</span>></span>true<span class="tag"></<span class="name">useArgumentFile</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>compile<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>test-compile<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugins</span>></span></span><br><span class="line"><span class="tag"></<span class="name">build</span>></span></span><br></pre></td></tr></table></figure>
<h3 id="进一步改进maven配置"><a href="#进一步改进maven配置" class="headerlink" title="进一步改进maven配置"></a>进一步改进maven配置</h3><p>如果我们的POM构建需要同时支持不同的平台，比如windows,MAC,Linux，但是这些平台上的protoc路径和文件都不一样，怎么办呢？ 可以考虑如下方式。</p>
<p>通过profiles<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="maven profiles: http://maven.apache.org/guides/introduction/introduction-to-profiles.html
">[3]</span></a></sup>的<code><activation></code>如果通过profiles来激活不同的脚本调用：</p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">profiles</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">profile</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>windows<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">activation</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">os</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">family</span>></span>Windows<span class="tag"></<span class="name">family</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">arch</span>></span>x86<span class="tag"></<span class="name">arch</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">os</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">activation</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">build</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugins</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-compiler-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.xolstice.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>protobuf-maven-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">checkStaleness</span>></span>false<span class="tag"></<span class="name">checkStaleness</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">staleMillis</span>></span>10000<span class="tag"></<span class="name">staleMillis</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">protocExecutable</span>></span>${basedir}\..\protobuf\bin\protoc.exe<span class="tag"></<span class="name">protocExecutable</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">useArgumentFile</span>></span>true<span class="tag"></<span class="name">useArgumentFile</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>compile<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>test-compile<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugins</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">build</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">profile</span>></span></span><br><span class="line"><span class="tag"></<span class="name">profiles</span>></span></span><br></pre></td></tr></table></figure>
<h2 id="第2种方式通过antrun插件来完成"><a href="#第2种方式通过antrun插件来完成" class="headerlink" title="第2种方式通过antrun插件来完成"></a>第2种方式通过antrun插件来完成</h2><p>通过组合os-maven-plugin, maven-antrun-plugin,build-helper-maven-plugin等多个插件来完成，效果是完全一样的。<br>这个例子<sup id="fnref:4"><a href="#fn:4" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="maven-protobuf-compile: https://vlkan.com/blog/post/2015/11/27/maven-protobuf/
">[4]</span></a></sup>的价值在于maven的灵活性和可定制型，对于理解如何使用maven来达到比较复杂的目的很有帮助。特别是antrun插件，很多遗留的工程在满足条件的情况下可以往maven进行迁移。<br>参考文档：</p>
<h3 id="前置条件："><a href="#前置条件：" class="headerlink" title="前置条件："></a>前置条件：</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">properties</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="comment"><!-- protobuf paths --></span></span><br><span class="line"> <span class="tag"><<span class="name">protobuf.input.directory</span>></span>${project.basedir}/src/main/proto<span class="tag"></<span class="name">protobuf.input.directory</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">protobuf.output.directory</span>></span>${project.build.directory}/generated-sources<span class="tag"></<span class="name">protobuf.output.directory</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="comment"><!-- library versions --></span></span><br><span class="line"> <span class="tag"><<span class="name">build-helper-maven-plugin.version</span>></span>1.9.1<span class="tag"></<span class="name">build-helper-maven-plugin.version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">maven-antrun-plugin.version</span>></span>1.8<span class="tag"></<span class="name">maven-antrun-plugin.version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">maven-dependency-plugin.version</span>></span>2.10<span class="tag"></<span class="name">maven-dependency-plugin.version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">maven-shade-plugin.version</span>></span>2.4.2<span class="tag"></<span class="name">maven-shade-plugin.version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">os-maven-plugin.version</span>></span>1.4.1.Final<span class="tag"></<span class="name">os-maven-plugin.version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">protobuf.version</span>></span>3.0.0-beta-1<span class="tag"></<span class="name">protobuf.version</span>></span></span><br><span class="line"></span><br><span class="line"><span class="tag"></<span class="name">properties</span>></span></span><br></pre></td></tr></table></figure>
<h3 id="配置依赖的Protocol-Buffer-API"><a href="#配置依赖的Protocol-Buffer-API" class="headerlink" title="配置依赖的Protocol Buffer API"></a>配置依赖的Protocol Buffer API</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>com.google.protobuf<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>protobuf-java<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${protobuf.version}<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"><span class="tag"></<span class="name">dependency</span>></span></span><br></pre></td></tr></table></figure>
<h3 id="检查操作系统类型，并生成相应的工程属性Properties"><a href="#检查操作系统类型，并生成相应的工程属性Properties" class="headerlink" title="检查操作系统类型，并生成相应的工程属性Properties"></a>检查操作系统类型，并生成相应的工程属性Properties</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">build</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">extensions</span>></span></span><br><span class="line"> <span class="comment"><!-- provides os.detected.classifier (i.e. linux-x86_64, osx-x86_64) property --></span></span><br><span class="line"> <span class="tag"><<span class="name">extension</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>kr.motd.maven<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>os-maven-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${os-maven-plugin.version}<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">extension</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">extensions</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="comment"><!-- ... --></span></span><br><span class="line"><span class="tag"></<span class="name">build</span>></span></span><br></pre></td></tr></table></figure>
<h3 id="自动下载平台相关的编译器"><a href="#自动下载平台相关的编译器" class="headerlink" title="自动下载平台相关的编译器"></a>自动下载平台相关的编译器</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="comment"><!-- copy protoc binary into build directory --></span></span><br><span class="line"><span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-dependency-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${maven-dependency-plugin.version}<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>copy-protoc<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">phase</span>></span>generate-sources<span class="tag"></<span class="name">phase</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>copy<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactItems</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactItem</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>com.google.protobuf<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>protoc<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${protobuf.version}<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">classifier</span>></span>${os.detected.classifier}<span class="tag"></<span class="name">classifier</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">type</span>></span>exe<span class="tag"></<span class="name">type</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">overWrite</span>></span>true<span class="tag"></<span class="name">overWrite</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">outputDirectory</span>></span>${project.build.directory}<span class="tag"></<span class="name">outputDirectory</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">artifactItem</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">artifactItems</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">executions</span>></span></span><br><span class="line"><span class="tag"></<span class="name">plugin</span>></span></span><br></pre></td></tr></table></figure>
<h3 id="调用protoc生成相关的Java代码。"><a href="#调用protoc生成相关的Java代码。" class="headerlink" title="调用protoc生成相关的Java代码。"></a>调用protoc生成相关的Java代码。</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="comment"><!-- compile proto buffer files using copied protoc binary --></span></span><br><span class="line"><span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-antrun-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${maven-antrun-plugin.version}<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>exec-protoc<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">phase</span>></span>generate-sources<span class="tag"></<span class="name">phase</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">target</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"protoc.filename"</span> <span class="attr">value</span>=<span class="string">"protoc-${protobuf.version}-${os.detected.classifier}.exe"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"protoc.filepath"</span> <span class="attr">value</span>=<span class="string">"${project.build.directory}/${protoc.filename}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">chmod</span> <span class="attr">file</span>=<span class="string">"${protoc.filepath}"</span> <span class="attr">perm</span>=<span class="string">"ugo+rx"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">mkdir</span> <span class="attr">dir</span>=<span class="string">"${protobuf.output.directory}"</span> /></span></span><br><span class="line"> <span class="tag"><<span class="name">path</span> <span class="attr">id</span>=<span class="string">"protobuf.input.filepaths.path"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">fileset</span> <span class="attr">dir</span>=<span class="string">"${protobuf.input.directory}"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">include</span> <span class="attr">name</span>=<span class="string">"**/*.proto"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">fileset</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">path</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">pathconvert</span> <span class="attr">pathsep</span>=<span class="string">" "</span> <span class="attr">property</span>=<span class="string">"protobuf.input.filepaths"</span> <span class="attr">refid</span>=<span class="string">"protobuf.input.filepaths.path"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">exec</span> <span class="attr">executable</span>=<span class="string">"${protoc.filepath}"</span> <span class="attr">failonerror</span>=<span class="string">"true"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span> <span class="attr">value</span>=<span class="string">"-I"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span> <span class="attr">value</span>=<span class="string">"${protobuf.input.directory}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span> <span class="attr">value</span>=<span class="string">"--java_out"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span> <span class="attr">value</span>=<span class="string">"${protobuf.output.directory}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span> <span class="attr">line</span>=<span class="string">"${protobuf.input.filepaths}"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">exec</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">target</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>run<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">executions</span>></span></span><br><span class="line"><span class="tag"></<span class="name">plugin</span>></span></span><br></pre></td></tr></table></figure>
<h3 id="将Java代码添加到包中"><a href="#将Java代码添加到包中" class="headerlink" title="将Java代码添加到包中"></a>将Java代码添加到包中</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="comment"><!-- add generated proto buffer classes into the package --></span></span><br><span class="line"><span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.codehaus.mojo<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>build-helper-maven-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${build-helper-maven-plugin.version}<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>add-classes<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">phase</span>></span>generate-sources<span class="tag"></<span class="name">phase</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>add-source<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">sources</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">source</span>></span>${protobuf.output.directory}<span class="tag"></<span class="name">source</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">sources</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">executions</span>></span></span><br><span class="line"><span class="tag"></<span class="name">plugin</span>></span></span><br></pre></td></tr></table></figure>
<h3 id="对Protocol-Buffer包进行shading"><a href="#对Protocol-Buffer包进行shading" class="headerlink" title="对Protocol Buffer包进行shading"></a>对Protocol Buffer包进行shading</h3><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="comment"><!-- shade protobuf to avoid version conflicts --></span></span><br><span class="line"><span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-shade-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${maven-shade-plugin.version}<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">phase</span>></span>package<span class="tag"></<span class="name">phase</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>shade<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">relocations</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">relocation</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">pattern</span>></span>com.google.protobuf<span class="tag"></<span class="name">pattern</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">shadedPattern</span>></span>${project.groupId}.${project.artifactId}.shaded.protobuf<span class="tag"></<span class="name">shadedPattern</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">relocation</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">relocations</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">executions</span>></span></span><br><span class="line"><span class="tag"></<span class="name">plugin</span>></span></span><br></pre></td></tr></table></figure>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">插件使用：https://www.xolstice.org/protobuf-maven-plugin/plugin-info.html<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">下载protoc: https://developers.google.com/protocol-buffers/docs/downloads<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">maven profiles: http://maven.apache.org/guides/introduction/introduction-to-profiles.html<a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">maven-protobuf-compile: https://vlkan.com/blog/post/2015/11/27/maven-protobuf/<a href="#fnref:4" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Building tools - maven</category>
<category>protocol buffer</category>
</categories>
<tags>
<tag>maven</tag>
<tag>protocol</tag>
<tag>buffer</tag>
</tags>
</entry>
<entry>
<title>SQL建模规范</title>
<url>/2021/03/29/SQL%E5%BB%BA%E6%A8%A1%E8%A7%84%E8%8C%83/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h1 id="建模原则"><a href="#建模原则" class="headerlink" title="建模原则"></a>建模原则</h1><ol>
<li><p>数据库字段小写，下划线连接；每个字段都需要由COMMENT，说明其业务含义。</p>
</li>
<li><p>使用id作为主键，建议类型为bigint(20)， 其对应的java类型为Long。</p>
<ul>
<li>Mybatis Plus框架自动生成主键插入（雪花算法）</li>
<li>也可以使用数据库自增，当只有1个服务，数据量较小的时候建议选择，提升插入性能。</li>
</ul>
</li>
<li><p>update_time、create_time<br>使用timestamp或者datetime作为时间类型的首选。Mybatis-Plus框架生成的相应Java类型都是LocalDateTime。大部分场景下2种类型可以互相替换，不过在时间范围和时区支持上有差异。<br>使用MySQL的自动初始化和自动更新功能。 字段变化之后，update_time数据库自动更新，如下：</p>
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line">create_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP <span class="keyword">COMMENT</span> <span class="string">'创建时间'</span>,</span><br><span class="line">create_by <span class="built_in">varchar</span>(<span class="number">36</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span> <span class="keyword">COMMENT</span> <span class="string">'创建人'</span>,</span><br><span class="line">update_at <span class="built_in">timestamp</span> <span class="keyword">NOT</span> <span class="literal">NULL</span> <span class="keyword">DEFAULT</span> <span class="keyword">CURRENT_TIMESTAMP</span> <span class="keyword">ON</span> <span class="keyword">UPDATE</span> <span class="keyword">CURRENT_TIMESTAMP</span> <span class="keyword">COMMENT</span> <span class="string">'更新时间'</span>,</span><br><span class="line">update_by <span class="built_in">varchar</span>(<span class="number">36</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span> <span class="keyword">COMMENT</span> <span class="string">'更新人'</span>,</span><br></pre></td></tr></table></figure>
</li>
<li><p>布尔型字段使用tinyint(1)即可。</p>
</li>
<li><p>参考《阿里巴巴Java开发手册(华山版).pdf》。</p>
</li>
</ol>
<h1 id="雪花算法实现"><a href="#雪花算法实现" class="headerlink" title="雪花算法实现"></a>雪花算法实现</h1><a id="more"></a>
<ol>
<li><p>IDGenerator接口声明，这里值得注意的是，雪花算法中提供了一个datacenterId用来避免在不同的集群（数据中心）产生碰撞。如果有多个集群，建议进行配置。</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Licensed to the Apache Software Foundation (ASF) under one or more</span></span><br><span class="line"><span class="comment"> * contributor license agreements. See the NOTICE file distributed with</span></span><br><span class="line"><span class="comment"> * this work for additional information regarding copyright ownership.</span></span><br><span class="line"><span class="comment"> * The ASF licenses this file to You under the Apache License, Version 2.0</span></span><br><span class="line"><span class="comment"> * (the "License"); you may not use this file except in compliance with</span></span><br><span class="line"><span class="comment"> * the License. You may obtain a copy of the License at</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * http://www.apache.org/licenses/LICENSE-2.0</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * Unless required by applicable law or agreed to in writing, software</span></span><br><span class="line"><span class="comment"> * distributed under the License is distributed on an "AS IS" BASIS,</span></span><br><span class="line"><span class="comment"> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span></span><br><span class="line"><span class="comment"> * See the License for the specific language governing permissions and</span></span><br><span class="line"><span class="comment"> * limitations under the License.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * An interface for database ID generators.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@author</span> Sebastian Schaffert (sschaffert@apache.org)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">IDGenerator</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Return the next unique id for the type with the given name using the generator's id generation strategy.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function"><span class="keyword">long</span> <span class="title">nextId</span><span class="params">()</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Shut down this id generator, performing any cleanups that might be necessary.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function"><span class="keyword">void</span> <span class="title">shutdown</span><span class="params">()</span></span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
</li>
<li><p>SnowflakeIDGenerator实现：</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Licensed to the Apache Software Foundation (ASF) under one or more</span></span><br><span class="line"><span class="comment"> * contributor license agreements. See the NOTICE file distributed with</span></span><br><span class="line"><span class="comment"> * this work for additional information regarding copyright ownership.</span></span><br><span class="line"><span class="comment"> * The ASF licenses this file to You under the Apache License, Version 2.0</span></span><br><span class="line"><span class="comment"> * (the "License"); you may not use this file except in compliance with</span></span><br><span class="line"><span class="comment"> * the License. You may obtain a copy of the License at</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * http://www.apache.org/licenses/LICENSE-2.0</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * Unless required by applicable law or agreed to in writing, software</span></span><br><span class="line"><span class="comment"> * distributed under the License is distributed on an "AS IS" BASIS,</span></span><br><span class="line"><span class="comment"> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span></span><br><span class="line"><span class="comment"> * See the License for the specific language governing permissions and</span></span><br><span class="line"><span class="comment"> * limitations under the License.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> org.slf4j.Logger;</span><br><span class="line"><span class="keyword">import</span> org.slf4j.LoggerFactory;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.net.NetworkInterface;</span><br><span class="line"><span class="keyword">import</span> java.net.SocketException;</span><br><span class="line"><span class="keyword">import</span> java.net.UnknownHostException;</span><br><span class="line"><span class="keyword">import</span> java.util.Enumeration;</span><br><span class="line"><span class="keyword">import</span> java.util.Random;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Generate unique IDs using the Twitter Snowflake algorithm (see https://github.com/twitter/snowflake). Snowflake IDs</span></span><br><span class="line"><span class="comment"> * are 64 bit positive longs composed of:</span></span><br><span class="line"><span class="comment"> * - 41 bits time stamp</span></span><br><span class="line"><span class="comment"> * - 10 bits machine id</span></span><br><span class="line"><span class="comment"> * - 12 bits sequence number</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@author</span> Sebastian Schaffert (sschaffert@apache.org)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">SnowflakeIDGenerator</span> <span class="keyword">implements</span> <span class="title">IDGenerator</span> </span>{</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> Logger log = LoggerFactory.getLogger(SnowflakeIDGenerator<span class="class">.<span class="keyword">class</span>)</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">long</span> datacenterIdBits = <span class="number">10L</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">long</span> maxDatacenterId = -<span class="number">1L</span> ^ (-<span class="number">1L</span> << datacenterIdBits);</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">long</span> sequenceBits = <span class="number">12L</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">long</span> datacenterIdShift = sequenceBits;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">long</span> timestampLeftShift = sequenceBits + datacenterIdBits;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">long</span> sequenceMask = -<span class="number">1L</span> ^ (-<span class="number">1L</span> << sequenceBits);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">long</span> twepoch = <span class="number">1288834974657L</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">long</span> datacenterId;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">volatile</span> <span class="keyword">long</span> lastTimestamp = -<span class="number">1L</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">volatile</span> <span class="keyword">long</span> sequence = <span class="number">0L</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="title">SnowflakeIDGenerator</span><span class="params">(<span class="keyword">long</span> datacenterId)</span> </span>{</span><br><span class="line"> <span class="keyword">if</span>(datacenterId == <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">this</span>.datacenterId = getDatacenterId();</span><br><span class="line"> } <span class="keyword">catch</span> (SocketException | UnknownHostException | NullPointerException e) {</span><br><span class="line"> log.warn(<span class="string">"SNOWFLAKE: could not determine machine address; using random datacenter ID"</span>);</span><br><span class="line"> Random rnd = <span class="keyword">new</span> Random();</span><br><span class="line"> <span class="keyword">this</span>.datacenterId = rnd.nextInt((<span class="keyword">int</span>)maxDatacenterId) + <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">this</span>.datacenterId = datacenterId;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (<span class="keyword">this</span>.datacenterId > maxDatacenterId || datacenterId < <span class="number">0</span>){</span><br><span class="line"> log.warn(<span class="string">"SNOWFLAKE: datacenterId > maxDatacenterId; using random datacenter ID"</span>);</span><br><span class="line"> Random rnd = <span class="keyword">new</span> Random();</span><br><span class="line"> <span class="keyword">this</span>.datacenterId = rnd.nextInt((<span class="keyword">int</span>)maxDatacenterId) + <span class="number">1</span>;</span><br><span class="line"> }</span><br><span class="line"> log.info(<span class="string">"SNOWFLAKE: initialised with datacenter ID {}"</span>, <span class="keyword">this</span>.datacenterId);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">protected</span> <span class="keyword">long</span> <span class="title">tilNextMillis</span><span class="params">(<span class="keyword">long</span> lastTimestamp)</span></span>{</span><br><span class="line"> <span class="keyword">long</span> timestamp = System.currentTimeMillis();</span><br><span class="line"> <span class="keyword">while</span> (timestamp <= lastTimestamp) {</span><br><span class="line"> timestamp = System.currentTimeMillis();</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> timestamp;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">protected</span> <span class="keyword">long</span> <span class="title">getDatacenterId</span><span class="params">()</span> <span class="keyword">throws</span> SocketException, UnknownHostException </span>{</span><br><span class="line"> NetworkInterface network = <span class="keyword">null</span>;</span><br><span class="line"></span><br><span class="line"> Enumeration<NetworkInterface> en = NetworkInterface.getNetworkInterfaces();</span><br><span class="line"> <span class="keyword">while</span> (en.hasMoreElements()) {</span><br><span class="line"> NetworkInterface nint = en.nextElement();</span><br><span class="line"> <span class="keyword">if</span> (!nint.isLoopback() && nint.getHardwareAddress() != <span class="keyword">null</span>) {</span><br><span class="line"> network = nint;</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">byte</span>[] mac = network.getHardwareAddress();</span><br><span class="line"></span><br><span class="line"> Random rnd = <span class="keyword">new</span> Random();</span><br><span class="line"> <span class="keyword">byte</span> rndByte = (<span class="keyword">byte</span>)(rnd.nextInt() & <span class="number">0x000000FF</span>);</span><br><span class="line"></span><br><span class="line"> <span class="comment">// take the last byte of the MAC address and a random byte as datacenter ID</span></span><br><span class="line"> <span class="keyword">return</span> ((<span class="number">0x000000FF</span> & (<span class="keyword">long</span>)mac[mac.length-<span class="number">1</span>]) | (<span class="number">0x0000FF00</span> & (((<span class="keyword">long</span>)rndByte)<<<span class="number">8</span>)))>><span class="number">6</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Return the next unique id for the type with the given name using the generator's id generation strategy.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">synchronized</span> <span class="keyword">long</span> <span class="title">nextId</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">long</span> timestamp = System.currentTimeMillis();</span><br><span class="line"> <span class="keyword">if</span>(timestamp<lastTimestamp) {</span><br><span class="line"> log.warn(<span class="string">"Clock moved backwards. Refusing to generate id for {} milliseconds."</span>,(lastTimestamp - timestamp));</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Thread.sleep((lastTimestamp - timestamp));</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException e) {</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (lastTimestamp == timestamp) {</span><br><span class="line"> sequence = (sequence + <span class="number">1</span>) & sequenceMask;</span><br><span class="line"> <span class="keyword">if</span> (sequence == <span class="number">0</span>) {</span><br><span class="line"> timestamp = tilNextMillis(lastTimestamp);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> sequence = <span class="number">0</span>;</span><br><span class="line"> }</span><br><span class="line"> lastTimestamp = timestamp;</span><br><span class="line"> <span class="keyword">long</span> id = ((timestamp - twepoch) << timestampLeftShift) | (datacenterId << datacenterIdShift) | sequence;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(id < <span class="number">0</span>) {</span><br><span class="line"> log.warn(<span class="string">"ID is smaller than 0: {}"</span>,id);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> id;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Shut down this id generator, performing any cleanups that might be necessary.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">shutdown</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="comment">//To change body of implemented methods use File | Settings | File Templates.</span></span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure> </li>
</ol>
]]></content>
<categories>
<category>SQL</category>
<category>建模</category>
<category>数据库</category>
</categories>
<tags>
<tag>建模</tag>
<tag>SQL</tag>
<tag>数据库</tag>
</tags>
</entry>
<entry>
<title>Kafka调研（3）：多点通信方案的设计-2</title>
<url>/2020/03/13/%E5%88%A9%E7%94%A8Kafka%E8%AE%BE%E8%AE%A1%E5%BA%94%E7%94%A8%E5%B1%82%E5%A4%9A%E7%82%B9%E9%80%9A%E4%BF%A1%E6%96%B9%E6%A1%88/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h1 id="1-星形拓扑的通信模型"><a href="#1-星形拓扑的通信模型" class="headerlink" title="1. 星形拓扑的通信模型"></a>1. 星形拓扑的通信模型</h1><p>星形的意思是说，有一个消息接收和转发中心，应用两两之间并不直接通信。所有的应用层消息都通过Kafka集群进行中转。但是对Kafka来说，消息的机制是推拉机制。如果需要接收消息，需要应用层连接到集群，订阅感兴趣的topic，然后poll消息。</p>
<p>为了实现这种模型，<br><strong>1. 我们利用Kafka本身的ACL控制机制来对topic权限进行控制</strong>。</p>
<ul>
<li>对每一个应用分配唯一的Consumer Group。</li>
<li>对每一个接入的应用分配唯一的账户和密码。</li>
<li>对每一个接入的应用分配SSL证书</li>
<li>对每一个topic按照应用进行配置读写权限。</li>
<li>对每一个接入的应用分配一个唯一的秘钥（该秘钥通过安全方式传递）。</li>
</ul>
<p><strong><em>2. 同时我们在应用包装一层API进行消息分派机制</em></strong></p>
<ul>
<li>应用层的消息需要进行加密。</li>
<li>应用层需要校验消息的来源和完整性。</li>
<li>判断该消息是否应该被处理，比如消息的接收方是否包含了自己。</li>
<li>应用层需要按照消息的业务ID（或者其他机制）分派合适的处理函数。</li>
</ul>
<a id="more"></a>
<p><strong>3. 星形拓扑通信模型</strong>：</p>
<img src="/2020/03/13/%E5%88%A9%E7%94%A8Kafka%E8%AE%BE%E8%AE%A1%E5%BA%94%E7%94%A8%E5%B1%82%E5%A4%9A%E7%82%B9%E9%80%9A%E4%BF%A1%E6%96%B9%E6%A1%88/20200313152842.jpg" class="" title="topo-comm-model-design">
<h1 id="2-应用层框架能力分析"><a href="#2-应用层框架能力分析" class="headerlink" title="2. 应用层框架能力分析"></a>2. 应用层框架能力分析</h1><p>Kafka Client API<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://kafka.apache.org/10/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html">[1]</span></a></sup>提供了功能丰富的接口, 同时通过Properties进行不同的配置。 Consumer API不是线程安全的。但是实际在应用过程中，一般来说会固定一套配置，同时希望应用层只需要配置topic行，其他的细节不需要关心。</p>
<p>经过考虑，该框架需要支持的能力如下：<br><strong>应用层：</strong></p>
<ul>
<li>提供一套简化的API支持Kafka消息的发送和接收。</li>
<li>和Spring进行集成，服务通过@Service的API进行暴露给应用层。</li>
<li>提供Request-Reply语义的接口（其中请求消息和响应的消息的匹配规则可以自定义）。</li>
<li>自动编译protobuf IDL定义并生成相应的JAR包。</li>
<li>支持消息接收注册和自定义分发机制</li>
</ul>
<p><strong>包装层：</strong></p>
<ul>
<li>支持默认的数据序列化和反序列化（POJO对象、protobuf、FastJSON）。</li>
<li>底层框架API支持自定义序列化框架注入。</li>
<li>提供了底层多线程的能力。</li>
<li>提供Offset commit的高级控制机制</li>
</ul>
<!-- more -->
<h1 id="2-应用层消息框架设计"><a href="#2-应用层消息框架设计" class="headerlink" title="2. 应用层消息框架设计"></a>2. 应用层消息框架设计</h1><h2 id="总体设计如下："><a href="#总体设计如下：" class="headerlink" title="总体设计如下："></a>总体设计如下：</h2><h2 id="模块划分："><a href="#模块划分：" class="headerlink" title="模块划分："></a>模块划分：</h2><p>于是，经过设计可以初步划分为5个模块：</p>
<p>kafka-message-service-api<br>kafka-message-service-impl<br>kafka-protobuf-model-generator<br>kafka-api-wrapper-api<br>kafka-api-wrapper-impl</p>
<p>具体模块设计图如下：</p>
<img src="/2020/03/13/%E5%88%A9%E7%94%A8Kafka%E8%AE%BE%E8%AE%A1%E5%BA%94%E7%94%A8%E5%B1%82%E5%A4%9A%E7%82%B9%E9%80%9A%E4%BF%A1%E6%96%B9%E6%A1%88/20200313145921.jpg" class="" title="message_dispatch-design">
<h2 id="详细类图设计："><a href="#详细类图设计：" class="headerlink" title="详细类图设计："></a>详细类图设计：</h2><p>后面再补充</p>
<h2 id="API设计："><a href="#API设计：" class="headerlink" title="API设计："></a>API设计：</h2><p>对应用层来说，只需要熟悉和理解这几个API就可以完全的收发消息。至于底层的多线层、偏移量的控制、异常的处理、Kafka的配置等都不需要直接处理。</p>
<p><strong><em>Message Sender Service API</em></strong>:</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">IMessageReceiverService</span> </span>{</span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 监听Topic application.yml: ${bimgis.kafka.topics.read.froom}，</span></span><br><span class="line"><span class="comment"> * 并按照KafkaFrameMsg格式解析</span></span><br><span class="line"><span class="comment"> * 业务层需要自定义实现是否处理该消息，覆写接口：{IMessageReceiver.canHandle}</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <T extends Message> <span class="function">String <span class="title">subscribe</span><span class="params">(IMessageReceiver<T> receiver)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 监听某个topic</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> topic 主题</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> receiver 消息处理函数</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> 唯一的ID = receiver.id</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function">String <span class="title">subscribe</span><span class="params">(String topic, IMessageReceiver<KafkaFrameMsg> receiver)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 监听某个主题，按照protobuf定义的格式进行预解析</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> topic</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> receiver</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> tClass</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <T></span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> 唯一的ID = receiver.id</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <T extends Message> <span class="function">String <span class="title">subscribe</span><span class="params">(String topic, IMessageReceiver<T> receiver, Class<T> tClass)</span></span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 去订阅某个主题，同时移除掉监听的consumer</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> recvId 订阅时返回的recvId</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function"><span class="keyword">void</span> <span class="title">unsubscribe</span><span class="params">(String recvId)</span></span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 去注册对应的消息接收器</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> receiver</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <T></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <T extends Message> <span class="function"><span class="keyword">void</span> <span class="title">unsubscribe</span><span class="params">(IMessageReceiver<T> receiver)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 用于主题消费计数和统计</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function">List<MessageTopicStat> <span class="title">stat</span><span class="params">()</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Data</span></span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">MessageTopicStat</span> </span>{</span><br><span class="line"> String consumerId;</span><br><span class="line"> List<MessageReceiverStatItem> handlers = <span class="keyword">new</span> ArrayList<>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Data</span></span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">MessageReceiverStatItem</span> </span>{</span><br><span class="line"> String id;</span><br><span class="line"> String topic;</span><br><span class="line"> String className;</span><br><span class="line"> <span class="keyword">boolean</span> isDelete = <span class="keyword">false</span>;</span><br><span class="line"> String createdTime;</span><br><span class="line"> String updatedTime;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p><strong><em>Message Receiver Service API</em></strong>:</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">IMessageSenderService</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 给消息添加时间戳、生成auth_token、添加唯一msgId</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> req</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> KafkaFrameMsg.<span class="function">Builder <span class="title">buildHeader</span><span class="params">(KafkaFrameMsg.Builder req)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 发送消息（指定topic,按照protobuf序列化进行发送）</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> topic</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> req</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function">Future<Integer> <span class="title">send</span><span class="params">(String topic,</span></span></span><br><span class="line"><span class="function"><span class="params"> KafkaFrameMsg.Builder req)</span></span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 发送固定Topic消息，不需要任何响应也不要回调</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> req</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function">Future<Integer> <span class="title">send</span><span class="params">(KafkaFrameMsg.Builder req)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 发送固定Topic消息，处理响应消息，默认2s超时</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> req</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> receiver</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function">Future<Integer> <span class="title">send</span><span class="params">(KafkaFrameMsg.Builder req,</span></span></span><br><span class="line"><span class="function"><span class="params"> IKafkaFrameMessageReplyHandler receiver)</span></span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 发送固定Topic消息，处理响应消息，自定义超时时间</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> req</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> receiver</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function">Future<Integer> <span class="title">send</span><span class="params">(KafkaFrameMsg.Builder req,</span></span></span><br><span class="line"><span class="function"><span class="params"> IKafkaFrameMessageReplyHandler receiver,</span></span></span><br><span class="line"><span class="function"><span class="params"> Duration timeout)</span></span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="class"><span class="keyword">interface</span> <span class="title">IMessageReplyHandler</span><<span class="title">T</span>, <span class="title">R</span>> </span>{</span><br><span class="line"> <span class="function"><span class="keyword">void</span> <span class="title">onReply</span><span class="params">(T req, R rsp)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">void</span> <span class="title">onTimeout</span><span class="params">(T req)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">void</span> <span class="title">onFailed</span><span class="params">(T req, <span class="keyword">int</span> errorCode)</span></span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * Request-Response 回复消息处理回调接口</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">abstract</span> <span class="class"><span class="keyword">class</span> <span class="title">IKafkaFrameMessageReplyHandler</span><<span class="title">T</span> <span class="keyword">extends</span> <span class="title">Message</span>></span></span><br><span class="line"><span class="class"> <span class="keyword">implements</span> <span class="title">IMessageReplyHandler</span><<span class="title">KafkaFrameMsg</span>, <span class="title">KafkaFrameMsg</span>> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">static</span> Logger logger = LoggerFactory.getLogger(IKafkaFrameMessageReplyHandler<span class="class">.<span class="keyword">class</span>)</span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 自动将消息的Body解析成对应的对象</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> req</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> rsp</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> busiBody</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function"><span class="keyword">protected</span> <span class="keyword">abstract</span> <span class="keyword">void</span> <span class="title">onReply</span><span class="params">(KafkaFrameMsg req, KafkaFrameMsg rsp, T busiBody)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title">onReply</span><span class="params">(KafkaFrameMsg req, KafkaFrameMsg rsp)</span> </span>{</span><br><span class="line"> <span class="comment">//解码先</span></span><br><span class="line"> logger.debug(String.format(<span class="string">"[request-and-reply] Receiving response, req id:%s, req auth:%s"</span> +</span><br><span class="line"> <span class="string">"rsp id:%s, rsp auth:%s, rsp sender:%s, rsp dest:%s"</span>,</span><br><span class="line"> req.getMsgId(), req.getAuthToken(),</span><br><span class="line"> rsp.getMsgId(), rsp.getAuthToken(),</span><br><span class="line"> rsp.getMsgSource().toString(),</span><br><span class="line"> rsp.getMsgDestList().toString()));</span><br><span class="line"></span><br><span class="line"> <span class="comment">//先对消息进行解码</span></span><br><span class="line"> T body = (T) AnyMessageUnpacker.getInstance().unpack(rsp.getBusiData()).get();</span><br><span class="line"> <span class="comment">//调用接口</span></span><br><span class="line"> onReply(req, rsp, body);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">onTimeout</span><span class="params">(KafkaFrameMsg req)</span> </span>{</span><br><span class="line"> logger.error(String.format(<span class="string">"message timeout for msg id:%s, busiCode:%s.\r\n"</span>,</span><br><span class="line"> req.getMsgId(),</span><br><span class="line"> req.getBusiCode().toString()));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">onFailed</span><span class="params">(KafkaFrameMsg req, <span class="keyword">int</span> errorCode)</span> </span>{</span><br><span class="line"> logger.error(String.format(<span class="string">"message failed, error code ->%d,msg id:%s, busiCode:%s\r\n"</span>,</span><br><span class="line"> errorCode,</span><br><span class="line"> req.getMsgId(),</span><br><span class="line"> req.getBusiCode().toString()));</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
<p><strong><em>MessageReceiver</em></strong></p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 消息接收处理回调接口定义</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <T></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">IMessageReceiver</span><<span class="title">T</span> <span class="keyword">extends</span> <span class="title">Message</span>> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">boolean</span> <span class="title">onReceive</span><span class="params">(T msg)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">default</span> <span class="keyword">boolean</span> <span class="title">canHandle</span><span class="params">(T msg)</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">true</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h1 id="3-消息协议制定"><a href="#3-消息协议制定" class="headerlink" title="3. 消息协议制定"></a>3. 消息协议制定</h1><figure class="highlight plain"><table><tr><td class="code"><pre><span class="line">message MessageEnvelope {</span><br><span class="line"> MessageId message_id = 1;</span><br><span class="line"> MessageType message_type = 2;</span><br><span class="line"></span><br><span class="line"> //用来做消息路由</span><br><span class="line"> SystemId receiver_system_id = 16;</span><br><span class="line"> int32 receiver_app_id = 17;</span><br><span class="line"> int32 receiver_domain_id = 18;</span><br><span class="line"> int32 receiver_module_id = 19;</span><br><span class="line"> int32 receiver_function_id = 20;</span><br><span class="line"></span><br><span class="line"> SystemId sender_system_id = 21;</span><br><span class="line"> int32 sender_app_id = 22;</span><br><span class="line"> int32 sender_domain_id = 23;</span><br><span class="line"> int32 sender_module_id = 24;</span><br><span class="line"> int32 sender_function_id = 25;</span><br><span class="line"></span><br><span class="line"> //消息体</span><br><span class="line"> MessageBody body = 30;</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://kafka.apache.org/10/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html<a href="#fnref:1" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>Kafka</category>
<category>Architecture</category>
</categories>
<tags>
<tag>Message Queue</tag>
<tag>Kafka</tag>
<tag>Topo Comm model</tag>
<tag>Frameword</tag>
<tag>星形拓扑通信</tag>
<tag>应用层设计</tag>
</tags>
</entry>
<entry>
<title>Thrift介绍、编译参数以及如何和maven集成</title>
<url>/2020/02/13/thrift%E7%BC%96%E8%AF%91%E5%B7%A5%E5%85%B7%E6%94%AF%E6%8C%81%E5%8F%82%E6%95%B0/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><h1 id="什么是Thrift"><a href="#什么是Thrift" class="headerlink" title="什么是Thrift?"></a>什么是Thrift?</h1><p>Thrift是一种接口描述语言和二进制通讯协议，它被用来定义和创建跨语言的服务。它被当作一个远程过程调用（RPC）框架来使用，是由Facebook为“大规模跨语言服务开发”而开发的。它通过一个代码生成引擎联合了一个软件栈，来创建不同程度的、无缝的跨平台高效服务，可以使用C#、C++（基于POSIX兼容系统）、Cappuccino、Cocoa、Delphi、Erlang、Go、Haskell、Java、Node.js、OCaml、Perl、PHP、Python、Ruby和Smalltalk。虽然它以前是由Facebook开发的，但它现在是Apache软件基金会的开源项目了。该实现被描述在2007年4月的一篇由Facebook发表的技术论文中，该论文现由Apache掌管。</p>
<p>目前流行的服务调用方式有很多种，例如基于 SOAP 消息格式的 Web Service，基于 JSON 消息格式的 RESTful 服务等。其中所用到的数据传输方式包括 XML，JSON 等，然而 XML 相对体积太大，传输效率低，JSON 体积较小，新颖，但还不够完善。本文将介绍由 Facebook 开发的远程服务调用框架 Apache Thrift，它采用接口描述语言定义并创建服务，支持可扩展的跨语言服务开发，所包含的代码生成引擎可以在多种语言中，如 C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk 等创建高效的、无缝的服务，其传输数据采用二进制格式，相对 XML 和 JSON 体积更小，对于高并发、大数据量和多语言的环境更有优势。本文将详细介绍 Thrift 的使用，并且提供丰富的实例代码加以解释说明，帮助使用者快速构建服务。Thrift的下载地址是：<a href="http://thrift.apache.org/download" target="_blank" rel="noopener">http://thrift.apache.org/download</a></p>
<h1 id="thrift-exe支持的参数有哪些？"><a href="#thrift-exe支持的参数有哪些？" class="headerlink" title="thrift.exe支持的参数有哪些？"></a>thrift.exe支持的参数有哪些？</h1><a id="more"></a>
<figure class="highlight bash"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line">Microsoft Windows [版本 10.0.18362.657]</span><br><span class="line">(c) 2019 Microsoft Corporation。保留所有权利。</span><br><span class="line"></span><br><span class="line">~>thrift-0.13.0.exe -<span class="built_in">help</span></span><br><span class="line">Usage: thrift [options] file</span><br><span class="line">Options:</span><br><span class="line"> -version Print the compiler version</span><br><span class="line"> -o dir Set the output directory <span class="keyword">for</span> gen-* packages</span><br><span class="line"> (default: current directory)</span><br><span class="line"> -out dir Set the ouput location <span class="keyword">for</span> generated files.</span><br><span class="line"> (no gen-* folder will be created)</span><br><span class="line"> -I dir Add a directory to the list of directories</span><br><span class="line"> searched <span class="keyword">for</span> include directives</span><br><span class="line"> -nowarn Suppress all compiler warnings (BAD!)</span><br><span class="line"> -strict Strict compiler warnings on</span><br><span class="line"> -v[erbose] Verbose mode</span><br><span class="line"> -r[ecurse] Also generate included files</span><br><span class="line"> -debug Parse debug trace to stdout</span><br><span class="line"> --allow-neg-keys Allow negative field keys (Used to preserve protocol</span><br><span class="line"> compatibility with older .thrift files)</span><br><span class="line"> --allow-64bit-consts Do not <span class="built_in">print</span> warnings about using 64-bit constants</span><br><span class="line"> --gen STR Generate code with a dynamically-registered generator.</span><br><span class="line"> STR has the form language[:key1=val1[,key2[,key3=val3]]].</span><br><span class="line"> Keys and values are options passed to the generator.</span><br><span class="line"> Many options will not require values.</span><br><span class="line"></span><br><span class="line">Options related to audit operation</span><br><span class="line"> --audit OldFile Old Thrift file to be audited with <span class="string">'file'</span></span><br><span class="line"> -Iold dir Add a directory to the list of directories</span><br><span class="line"> searched <span class="keyword">for</span> include directives <span class="keyword">for</span> old thrift file</span><br><span class="line"> -Inew dir Add a directory to the list of directories</span><br><span class="line"> searched <span class="keyword">for</span> include directives <span class="keyword">for</span> new thrift file</span><br><span class="line"></span><br><span class="line">Available generators (and options):</span><br><span class="line"> as3 (AS3):</span><br><span class="line"> bindable: Add [bindable] metadata to all the struct classes.</span><br><span class="line"> c_glib (C, using GLib):</span><br><span class="line"> cl (Common Lisp):</span><br><span class="line"> no_asd: Do not define ASDF systems <span class="keyword">for</span> each generated Thrift program.</span><br><span class="line"> sys_pref= The prefix to give ASDF system names. Default: thrift-gen-</span><br><span class="line"> cpp (C++):</span><br><span class="line"> cob_style: Generate <span class="string">"Continuation OBject"</span>-style classes.</span><br><span class="line"> no_client_completion:</span><br><span class="line"> Omit calls to completion__() <span class="keyword">in</span> CobClient class.</span><br><span class="line"> no_default_operators:</span><br><span class="line"> Omits generation of default operators ==, != and <</span><br><span class="line"> templates: Generate templatized reader/writer methods.</span><br><span class="line"> pure_enums: Generate pure enums instead of wrapper classes.</span><br><span class="line"> include_prefix: Use full include paths <span class="keyword">in</span> generated files.</span><br><span class="line"> moveable_types: Generate move constructors and assignment operators.</span><br><span class="line"> no_ostream_operators:</span><br><span class="line"> Omit generation of ostream definitions.</span><br><span class="line"> no_skeleton: Omits generation of skeleton.</span><br><span class="line"> csharp (C<span class="comment">#):</span></span><br><span class="line"> async: Adds Async support using Task.Run.</span><br><span class="line"> wcf: Adds bindings <span class="keyword">for</span> WCF to generated classes.</span><br><span class="line"> serial: Add serialization support to generated classes.</span><br><span class="line"> nullable: Use nullable types <span class="keyword">for</span> properties.</span><br><span class="line"> hashcode: Generate a hashcode and equals implementation <span class="keyword">for</span> classes.</span><br><span class="line"> union: Use new union typing, <span class="built_in">which</span> includes a static <span class="built_in">read</span> <span class="keyword">function</span> <span class="keyword">for</span> union types.</span><br><span class="line"> d (D):</span><br><span class="line"> dart (Dart):</span><br><span class="line"> library_name: Optional override <span class="keyword">for</span> library name.</span><br><span class="line"> library_prefix: Generate code that can be used within an existing library.</span><br><span class="line"> Use a dot-separated string, e.g. <span class="string">"my_parent_lib.src.gen"</span></span><br><span class="line"> pubspec_lib: Optional override <span class="keyword">for</span> thrift lib dependency <span class="keyword">in</span> pubspec.yaml,</span><br><span class="line"> e.g. <span class="string">"thrift: 0.x.x"</span>. Use a pipe delimiter to separate lines,</span><br><span class="line"> e.g. <span class="string">"thrift:| git:| url: git@foo.com"</span></span><br><span class="line"> delphi (delphi):</span><br><span class="line"> ansistr_binary: Use AnsiString <span class="keyword">for</span> binary datatype (default is TBytes).</span><br><span class="line"> register_types: Enable TypeRegistry, allows <span class="keyword">for</span> creation of struct, union</span><br><span class="line"> and container instances by interface or TypeInfo()</span><br><span class="line"> constprefix: Name TConstants classes after IDL to reduce ambiguities</span><br><span class="line"> events: Enable and use processing events <span class="keyword">in</span> the generated code.</span><br><span class="line"> xmldoc: Enable XMLDoc comments <span class="keyword">for</span> Help Insight etc.</span><br><span class="line"> async: Generate IAsync interface to use Parallel Programming Library (XE7+ only).</span><br><span class="line"> erl (Erlang):</span><br><span class="line"> legacynames: Output files retain naming conventions of Thrift 0.9.1 and earlier.</span><br><span class="line"> maps: Generate maps instead of dicts.</span><br><span class="line"> otp16: Generate non-namespaced dict and <span class="built_in">set</span> instead of dict:dict and sets:<span class="built_in">set</span>.</span><br><span class="line"> go (Go):</span><br><span class="line"> package_prefix= Package prefix <span class="keyword">for</span> generated files.</span><br><span class="line"> thrift_import= Override thrift package import path (default:github.com/apache/thrift/lib/go/thrift)</span><br><span class="line"> package= Package name (default: inferred from thrift file name)</span><br><span class="line"> ignore_initialisms</span><br><span class="line"> Disable automatic spelling correction of initialisms (e.g. <span class="string">"URL"</span>)</span><br><span class="line"> read_write_private</span><br><span class="line"> Make <span class="built_in">read</span>/write methods private, default is public Read/Write</span><br><span class="line"> gv (Graphviz):</span><br><span class="line"> exceptions: Whether to draw arrows from <span class="built_in">functions</span> to exception.</span><br><span class="line"> haxe (Haxe):</span><br><span class="line"> callbacks Use onError()/onSuccess() callbacks <span class="keyword">for</span> service methods (like AS3)</span><br><span class="line"> rtti Enable @:rtti <span class="keyword">for</span> generated classes and interfaces</span><br><span class="line"> buildmacro=my.macros.Class.method(args)</span><br><span class="line"> Add @:build macro calls to generated classes and interfaces</span><br><span class="line"> hs (Haskell):</span><br><span class="line"> html (HTML):</span><br><span class="line"> standalone: Self-contained mode, includes all CSS <span class="keyword">in</span> the HTML files.</span><br><span class="line"> Generates no style.css file, but HTML files will be larger.</span><br><span class="line"> noescape: Do not escape html <span class="keyword">in</span> doc text.</span><br><span class="line"> java (Java):</span><br><span class="line"> beans: Members will be private, and setter methods will <span class="built_in">return</span> void.</span><br><span class="line"> private-members: Members will be private, but setter methods will <span class="built_in">return</span> <span class="string">'this'</span> like usual.</span><br><span class="line"> nocamel: Do not use CamelCase field accessors with beans.</span><br><span class="line"> fullcamel: Convert underscored_accessor_or_service_names to camelCase.</span><br><span class="line"> android: Generated structures are Parcelable.</span><br><span class="line"> android_legacy: Do not use java.io.IOException(throwable) (available <span class="keyword">for</span> Android 2.3 and above).</span><br><span class="line"> option_type: Wrap optional fields <span class="keyword">in</span> an Option <span class="built_in">type</span>.</span><br><span class="line"> rethrow_unhandled_exceptions:</span><br><span class="line"> Enable rethrow of unhandled exceptions and <span class="built_in">let</span> them propagate futher. (Default behavior is to catch and <span class="built_in">log</span> it.)</span><br><span class="line"> java5: Generate Java 1.5 compliant code (includes android_legacy flag).</span><br><span class="line"> reuse-objects: Data objects will not be allocated, but existing instances will be used (<span class="built_in">read</span> and write).</span><br><span class="line"> sorted_containers:</span><br><span class="line"> Use TreeSet/TreeMap instead of HashSet/HashMap as a implementation of <span class="built_in">set</span>/map.</span><br><span class="line"> generated_annotations=[undated|suppress]:</span><br><span class="line"> undated: suppress the date at @Generated annotations</span><br><span class="line"> suppress: suppress @Generated annotations entirely</span><br><span class="line"> unsafe_binaries: Do not copy ByteBuffers <span class="keyword">in</span> constructors, getters, and setters.</span><br><span class="line"> javame (Java ME):</span><br><span class="line"> js (Javascript):</span><br><span class="line"> jquery: Generate jQuery compatible code.</span><br><span class="line"> node: Generate node.js compatible code.</span><br><span class="line"> ts: Generate TypeScript definition files.</span><br><span class="line"> with_ns: Create global namespace objects when using node.js</span><br><span class="line"> es6: Create ES6 code with Promises</span><br><span class="line"> thrift_package_output_directory=<path>:</span><br><span class="line"> Generate episode file and use the <path> as prefix</span><br><span class="line"> imports=<paths_to_modules>:</span><br><span class="line"> <span class="string">':'</span> separated list of paths of modules that has episode files <span class="keyword">in</span> their root</span><br><span class="line"> json (JSON):</span><br><span class="line"> merge: Generate output with included files merged</span><br><span class="line"> lua (Lua):</span><br><span class="line"> omit_requires: Suppress generation of require <span class="string">'somefile'</span>.</span><br><span class="line"> netcore (C<span class="comment">#):</span></span><br><span class="line"> wcf: Adds bindings <span class="keyword">for</span> WCF to generated classes.</span><br><span class="line"> serial: Add serialization support to generated classes.</span><br><span class="line"> nullable: Use nullable types <span class="keyword">for</span> properties.</span><br><span class="line"> hashcode: Generate a hashcode and equals implementation <span class="keyword">for</span> classes.</span><br><span class="line"> union: Use new union typing, <span class="built_in">which</span> includes a static <span class="built_in">read</span> <span class="keyword">function</span> <span class="keyword">for</span> union types.</span><br><span class="line"> netstd (C<span class="comment">#):</span></span><br><span class="line"> wcf: Adds bindings <span class="keyword">for</span> WCF to generated classes.</span><br><span class="line"> serial: Add serialization support to generated classes.</span><br><span class="line"> union: Use new union typing, <span class="built_in">which</span> includes a static <span class="built_in">read</span> <span class="keyword">function</span> <span class="keyword">for</span> union types.</span><br><span class="line"> ocaml (OCaml):</span><br><span class="line"> perl (Perl):</span><br><span class="line"> php (PHP):</span><br><span class="line"> inlined: Generate PHP inlined files</span><br><span class="line"> server: Generate PHP server stubs</span><br><span class="line"> oop: Generate PHP with object oriented subclasses</span><br><span class="line"> classmap: Generate old-style PHP files (use classmap autoloading)</span><br><span class="line"> rest: Generate PHP REST processors</span><br><span class="line"> nsglobal=NAME: Set global namespace</span><br><span class="line"> validate: Generate PHP validator methods</span><br><span class="line"> json: Generate JsonSerializable classes (requires PHP >= 5.4)</span><br><span class="line"> py (Python):</span><br><span class="line"> zope.interface: Generate code <span class="keyword">for</span> use with zope.interface.</span><br><span class="line"> twisted: Generate Twisted-friendly RPC services.</span><br><span class="line"> tornado: Generate code <span class="keyword">for</span> use with Tornado.</span><br><span class="line"> no_utf8strings: Do not Encode/decode strings using utf8 <span class="keyword">in</span> the generated code. Basically no effect <span class="keyword">for</span> Python 3.</span><br><span class="line"> coding=CODING: Add file encoding <span class="built_in">declare</span> <span class="keyword">in</span> generated file.</span><br><span class="line"> slots: Generate code using slots <span class="keyword">for</span> instance members.</span><br><span class="line"> dynamic: Generate dynamic code, less code generated but slower.</span><br><span class="line"> dynbase=CLS Derive generated classes from class CLS instead of TBase.</span><br><span class="line"> dynfrozen=CLS Derive generated immutable classes from class CLS instead of TFrozenBase.</span><br><span class="line"> dynexc=CLS Derive generated exceptions from CLS instead of TExceptionBase.</span><br><span class="line"> dynimport=<span class="string">'from foo.bar import CLS'</span></span><br><span class="line"> Add an import line to generated code to find the dynbase class.</span><br><span class="line"> package_prefix=<span class="string">'top.package.'</span></span><br><span class="line"> Package prefix <span class="keyword">for</span> generated files.</span><br><span class="line"> old_style: Deprecated. Generate old-style classes.</span><br><span class="line"> rb (Ruby):</span><br><span class="line"> rubygems: Add a <span class="string">"require 'rubygems'"</span> line to the top of each generated file.</span><br><span class="line"> namespaced: Generate files <span class="keyword">in</span> idiomatic namespaced directories.</span><br><span class="line"> rs (Rust):</span><br><span class="line"></span><br><span class="line"> st (Smalltalk):</span><br><span class="line"> swift (Swift 3.0):</span><br><span class="line"> log_unexpected: Log every time an unexpected field ID or <span class="built_in">type</span> is encountered.</span><br><span class="line"> debug_descriptions:</span><br><span class="line"> Allow use of debugDescription so the app can add description via a cateogory/extension</span><br><span class="line"> async_clients: Generate clients <span class="built_in">which</span> invoke asynchronously via block syntax.</span><br><span class="line"> namespaced: Generate <span class="built_in">source</span> <span class="keyword">in</span> Module scoped output directories <span class="keyword">for</span> Swift Namespacing.</span><br><span class="line"> cocoa: Generate Swift 2.x code compatible with the Thrift/Cocoa library</span><br><span class="line"> promise_kit: Generate clients <span class="built_in">which</span> invoke asynchronously via promises (only use with cocoa flag)</span><br><span class="line"> safe_enums: Generate enum types with an unknown <span class="keyword">case</span> to handle unspecified values rather than throw a serialization error</span><br><span class="line"> xml (XML):</span><br><span class="line"> merge: Generate output with included files merged</span><br><span class="line"> no_default_ns: Omit default xmlns and add idl: prefix to all elements</span><br><span class="line"> no_namespaces: Do not add namespace definitions to the XML model</span><br><span class="line"> xsd (XSD):</span><br><span class="line"> </span><br></pre></td></tr></table></figure>
<h1 id="基本用法"><a href="#基本用法" class="headerlink" title="基本用法:"></a>基本用法:</h1><p>thrift –gen <language> <Thrift filename> 把一个thrift文件编译成对应语言的代码<br>thrift -r –gen <language> <Thrift filename> 将所有<Thrift filename> thrift所有引用的thrift全部编译成代码</p>
<p>常用做法是定义一个顶级的include.thrift，将所有其他的thrift文件全部包含进来，加上-r参数之后，这样所有的模型都能编译了。</p>
<p>常用参数解释：<br>-o dir gen-* 包的输出目录, 比如gen-java, 下面就是java的包<br>-out dir 生成文件的输出地址，gen-* 目录不会生成，和上面的命令互斥，用一个就行了。<br>-I dir 将目录加到引用文件的搜索路径列表中<br>–gen STR <strong><em>用动态注册的生成器来产生代码。</em></strong><br> <strong><em>这里的参数格式是这样的: language:[:key1=val1[,key2[,key3=val3]]]</em></strong><br> <strong><em>冒号后的键和值是传递给生成器的参数。有的参数不需要值。</em></strong></p>
<pre><code>以java为例：
thrift.exe --gen java:beans,nocamel xxx.thrift
的意思就是，生产java代码，传递给生成器的参数是beans和nocamel
beans: 所有的成员变量私有，setter返回void
nocamel:beans代码的accessor不用驼峰命名法。</code></pre><h1 id="如何支持maven集成"><a href="#如何支持maven集成" class="headerlink" title="如何支持maven集成"></a>如何支持maven集成</h1><h2 id="插件maven-thrift-plugin"><a href="#插件maven-thrift-plugin" class="headerlink" title="插件maven-thrift-plugin"></a>插件maven-thrift-plugin</h2><p>插件的Github目录<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://github.com/dtrott/maven-thrift-plugin
">[2]</span></a></sup>上表示最小配置是这样的：</p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"> <span class="tag"><<span class="name">build</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugins</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.thrift.tools<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-thrift-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>0.1.11<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="comment"><!--<thriftExecutable>/usr/local/bin/thrift</thriftExecutable>--></span></span><br><span class="line"> <span class="comment"><!--<thriftSourceRoot>src/main/thrift</thriftSourceRoot>--></span></span><br><span class="line"> <span class="comment"><!--<outputDirectory>src/main/java</outputDirectory>--></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">id</span>></span>thrift-sources<span class="tag"></<span class="name">id</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">phase</span>></span>generate-sources<span class="tag"></<span class="name">phase</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>compile<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">execution</span>></span></span><br><span class="line"> <span class="comment"><!--<execution>--></span></span><br><span class="line"> <span class="comment"><!--<id>thrift-test-sources</id>--></span></span><br><span class="line"> <span class="comment"><!--<phase>generate-test-sources</phase>--></span></span><br><span class="line"> <span class="comment"><!--<goals>--></span></span><br><span class="line"> <span class="comment"><!--<goal>testCompile</goal>--></span></span><br><span class="line"> <span class="comment"><!--</goals>--></span></span><br><span class="line"> <span class="comment"><!--</execution>--></span></span><br><span class="line"> <span class="tag"></<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">plugins</span>></span></span><br><span class="line"><span class="tag"></<span class="name">build</span>></span></span><br></pre></td></tr></table></figure>
<p>但是从我的使用来看，这个插件没有人维护，最后一次release的时间是2013年11月。插件使用起来有很多的问题。所以我建议是第二种方式。</p>
<img src="/2020/02/13/thrift%E7%BC%96%E8%AF%91%E5%B7%A5%E5%85%B7%E6%94%AF%E6%8C%81%E5%8F%82%E6%95%B0/20200213221502.png" class="" title="github截图">
<h2 id="maven-antrun-plugin"><a href="#maven-antrun-plugin" class="headerlink" title="maven-antrun-plugin"></a>maven-antrun-plugin</h2><p>用这种方式虽然可能比较ugly，但是非常的灵活。几乎可以做你想用命令行或者脚本做到的任何事情。<br>我的配置是这样的：</p>
<figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">plugin</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.maven.plugins<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>maven-antrun-plugin<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>${maven-antrun-plugin.version}<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">executions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">phase</span>></span>generate-sources<span class="tag"></<span class="name">phase</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">target</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"thrift.filename"</span> <span class="attr">value</span>=<span class="string">"thrift-${thrift.versoin}.exe"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"thrift.filepath"</span> <span class="attr">value</span>=<span class="string">"${basedir}/../thrift/${thrift.filename}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">property</span> <span class="attr">name</span>=<span class="string">"thrift.compile.result"</span>></span>0<span class="tag"></<span class="name">property</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">mkdir</span> <span class="attr">dir</span>=<span class="string">"${thrift.output.directory}"</span>/></span></span><br><span class="line"> <span class="comment"><!-- Define fileset of thrift files --></span></span><br><span class="line"> <span class="tag"><<span class="name">fileset</span> <span class="attr">id</span>=<span class="string">"thrift.src.files"</span> <span class="attr">dir</span>=<span class="string">"${thrift.input.directory}"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">include</span> <span class="attr">name</span>=<span class="string">"**/*.thrift"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">fileset</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="comment"><!-- Invoke thrift binary for each of these files --></span></span><br><span class="line"> <span class="tag"><<span class="name">apply</span> <span class="attr">executable</span>=<span class="string">"${thrift.filepath}"</span> <span class="attr">resultproperty</span>=<span class="string">"thrift.compile.result"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">failifexecutionfails</span>=<span class="string">"true"</span> <span class="attr">failonerror</span>=<span class="string">"true"</span></span></span><br><span class="line"><span class="tag"> <span class="attr">searchpath</span>=<span class="string">"true"</span> <span class="attr">dir</span>=<span class="string">"${basedir}"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span> <span class="attr">value</span>=<span class="string">"-o"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span> <span class="attr">value</span>=<span class="string">"${thrift.output.directory}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span> <span class="attr">value</span>=<span class="string">"--gen"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">arg</span> <span class="attr">value</span>=<span class="string">"java"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">srcfile</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">fileset</span> <span class="attr">refid</span>=<span class="string">"thrift.src.files"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">apply</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">echo</span> <span class="attr">message</span>=<span class="string">"execution result is: ${thrift.compile.result}"</span>></span><span class="tag"></<span class="name">echo</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">target</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">configuration</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">goal</span>></span>run<span class="tag"></<span class="name">goal</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">goals</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">execution</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">executions</span>></span></span><br><span class="line"><span class="tag"></<span class="name">plugin</span>></span></span><br></pre></td></tr></table></figure>
<p>这里重点介绍一下apply<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://ant.apache.org/manual/Tasks/apply.html
">[3]</span></a></sup>的用法。 Apply是Ant的一个内置Task，用来执行一个系统命令。在windows上就是run, 在linux命令上类似bash。<br>解释一下这几个参数的用法:<br><em>executable</em> 可执行文件的路径。<br><em>resultproperty</em> 将命令的返回值保存到属性中。 <strong><em>实际上我们只有在命令执行报错的时候才会关注这个值。比如当我们将failonerror设置为true, 这个时候命令执行报错，但是构建不会停止，这个时候我们才需要将返回值保存下来留待后面检查。</em></strong><br><em>failonerror</em> 当命令返回值不是0的时候停止构建。默认false。<br><em>failifexecutionfails</em> 如果无法启动程序的时候停止构建。默认true。</p>
<p><fileset><sup id="fnref:4"><a href="#fn:4" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://ant.apache.org/manual/Types/fileset.html
">[4]</span></a></sup>就是一组文件，通过模式匹配或者其他参数来搜索到一组文件。比较简单。</p>
<p>也有兴趣也可以翻阅一下，其他一些ant任务的介绍<sup id="fnref:5"><a href="#fn:5" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://blog.csdn.net/fanxiaobin577328725/article/details/53699735">[5]</span></a></sup>。</p>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.ibm.com/developerworks/cn/java/j-lo-apachethrift/index.html<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://github.com/dtrott/maven-thrift-plugin<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://ant.apache.org/manual/Tasks/apply.html<a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://ant.apache.org/manual/Types/fileset.html<a href="#fnref:4" rev="footnote"> ↩</a></span></li><li id="fn:5"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">5.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://blog.csdn.net/fanxiaobin577328725/article/details/53699735<a href="#fnref:5" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>thrift</category>
</categories>
<tags>
<tag>thrift</tag>
<tag>compiler</tag>
<tag>command line options</tag>
</tags>
</entry>
<entry>
<title>Spring Cloud支持MongoDB</title>
<url>/2020/12/07/%E5%A6%82%E4%BD%95%E5%9C%A8Spring-Cloud%E6%9C%8D%E5%8A%A1%E4%B8%AD%E9%85%8D%E7%BD%AEMongoDB/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><p>MongoDB无疑是当前在各个领域中NoSQL优选的文档数据库之一。原因在于部署简单，可扩展性强，原生支持javascript脚本编程。对于使用Spring全家桶的团队来说，是一个很好的选择，可以快速的搭建一个服务出来。如果是用JS或者Typescript语言的话，使用mongoose的ORM框架结合到express框架，生产力会提高一大截。</p>
<p>稍显不足的是，MongoDB对于中文索引支持的力度不够，对于某些要求强事务的领域应用（比如金融、财务等）略显不足，至少还没有看到有成熟应用的案例。如果有，请告诉我。另外一个就是，在搭建MongoDB集群的时候，相对来说是比较复杂的。水平扩展虽然很简单，但是做Sharding的时候也需要仔细的考量。</p>
<p>本文简单介绍一下如何在Spring Boot/Spring Cloud程序中使用MongoDB数据库。</p>
<h1 id="Java-Config"><a href="#Java-Config" class="headerlink" title="Java Config"></a>Java Config</h1><p>第一种方式就是采用Java Config的方式，直接上代码，如下：</p>
<a id="more"></a>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">import</span> com.mongodb.ConnectionString;</span><br><span class="line"><span class="keyword">import</span> com.mongodb.MongoClientSettings;</span><br><span class="line"><span class="keyword">import</span> com.mongodb.MongoCredential;</span><br><span class="line"><span class="keyword">import</span> com.mongodb.client.MongoClient;</span><br><span class="line"><span class="keyword">import</span> com.mongodb.client.MongoClients;</span><br><span class="line"><span class="keyword">import</span> org.springframework.beans.factory.annotation.Value;</span><br><span class="line"><span class="keyword">import</span> org.springframework.context.annotation.Configuration;</span><br><span class="line"><span class="keyword">import</span> org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;</span><br><span class="line"></span><br><span class="line"><span class="meta">@Configuration</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">MongoClientConfiguration</span> <span class="keyword">extends</span> <span class="title">AbstractMongoClientConfiguration</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Value</span>(<span class="string">"${bimgis.mongodb.connection:mongodb://192.168.201.156:27017}"</span>)</span><br><span class="line"> <span class="keyword">private</span> String mongoConnectionStr = <span class="string">"mongodb://192.168.201.156:27017"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Value</span>(<span class="string">"${bimgis.mongodb.auth.user:admin}"</span>)</span><br><span class="line"> <span class="keyword">private</span> String user = <span class="string">"admin"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Value</span>(<span class="string">"${bimgis.mongodb.auth.password:}"</span>)</span><br><span class="line"> <span class="keyword">private</span> String password = <span class="string">""</span>;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Value</span>(<span class="string">"${bimgis.mongodb.auth.source:admin}"</span>)</span><br><span class="line"> <span class="keyword">private</span> String adminDatabase = <span class="string">"admin"</span>;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Value</span>(<span class="string">"${bimgis.mongodb.database:}"</span>)</span><br><span class="line"> <span class="keyword">private</span> String database;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> MongoClient <span class="title">mongoClient</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="comment">//Refer http://mongodb.github.io/mongo-java-driver/3.12/driver/tutorials/connect-to-mongodb/</span></span><br><span class="line"> MongoCredential credential = MongoCredential.createCredential(user, adminDatabase, password.toCharArray());</span><br><span class="line"> MongoClientSettings settings = MongoClientSettings.builder()</span><br><span class="line"> .credential(credential)</span><br><span class="line"> .applyConnectionString(<span class="keyword">new</span> ConnectionString(mongoConnectionStr))</span><br><span class="line"> .build();</span><br><span class="line"> <span class="keyword">return</span> MongoClients.create(settings);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">protected</span> String <span class="title">getDatabaseName</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> database;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>同时在Spring Cloud Config服务（或者Nacos服务）中做如下配置：</p>
<figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="attr">bimgis:</span></span><br><span class="line"> <span class="attr">mongodb:</span></span><br><span class="line"> <span class="attr">connection:</span> <span class="string">mongodb://192.168.201.155:27011,192.168.201.128:27011,192.168.201.128:27012/?replicaSet=share</span></span><br><span class="line"> <span class="attr">auth:</span></span><br><span class="line"> <span class="attr">user:</span> <span class="string">admin</span></span><br><span class="line"> <span class="attr">password:</span> <span class="string">Admin@123</span></span><br><span class="line"> <span class="comment">#授权的数据库</span></span><br><span class="line"> <span class="attr">source:</span> <span class="string">admin</span></span><br><span class="line"> <span class="attr">database:</span> <span class="string">bimgis_project</span></span><br></pre></td></tr></table></figure>
<h1 id="直接采用注解的方式"><a href="#直接采用注解的方式" class="headerlink" title="直接采用注解的方式"></a>直接采用注解的方式</h1><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"></span><br><span class="line"><span class="attr">spring:</span></span><br><span class="line"> <span class="attr">data:</span></span><br><span class="line"> <span class="attr">mongodb:</span></span><br><span class="line"> <span class="attr">uri:</span> <span class="string">mongodb://admin:Admin%40123@192.168.201.155:27011,192.168.201.128:27011,192.168.201.128:27012/bimgis_sync?replicaSet=share&authSource=admin</span> </span><br></pre></td></tr></table></figure>
<p>其他的参数，请参考链接<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="Spring Boot程序配置项：https://docs.spring.io/spring-boot/docs/current/reference/html/appendix-application-properties.html
">[2]</span></a></sup>， 如图：</p>
<img src="/2020/12/07/%E5%A6%82%E4%BD%95%E5%9C%A8Spring-Cloud%E6%9C%8D%E5%8A%A1%E4%B8%AD%E9%85%8D%E7%BD%AEMongoDB/mongo_config.png" class="" title="mongodb配置">
<h1 id="定义DAO对象"><a href="#定义DAO对象" class="headerlink" title="定义DAO对象"></a>定义DAO对象</h1><p>在选择定义DAO对象的时候，我使用的是Kotin语言。</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Document</span>(collection = <span class="string">"section"</span>)</span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Section</span> </span>{</span><br><span class="line"> <span class="meta">@Id</span></span><br><span class="line"> <span class="keyword">var</span> _id: String? = <span class="keyword">null</span></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Field</span>(<span class="string">"name"</span>)</span><br><span class="line"> <span class="keyword">var</span> sectionName: String? = <span class="keyword">null</span></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Field</span>(<span class="string">"project_code"</span>)</span><br><span class="line"> <span class="keyword">var</span> projectCode: String? = <span class="keyword">null</span></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Field</span>(<span class="string">"del"</span>)</span><br><span class="line"> <span class="meta">@JsonProperty</span>(<span class="string">"isDelete"</span>)</span><br><span class="line"> <span class="keyword">var</span> isDelete = <span class="number">0</span></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Field</span>(<span class="string">"ct"</span>)</span><br><span class="line"> <span class="keyword">var</span> createTime: Date? = <span class="keyword">null</span></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Field</span>(<span class="string">"cb"</span>)</span><br><span class="line"> <span class="keyword">var</span> createBy: String? = <span class="keyword">null</span></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Field</span>(<span class="string">"ut"</span>)</span><br><span class="line"> <span class="keyword">var</span> updateTime: Date? = <span class="keyword">null</span></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Field</span>(<span class="string">"ub"</span>)</span><br><span class="line"> <span class="keyword">var</span> updateBy: String? = <span class="keyword">null</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>对代码做一些简单的说明，@Id来声明主键，在每个字段上都增加@Field注解,这样可以增加代码的清晰度。值得注意的时候，@Field注解有很几个参数，比较关键的一个是targetType, 这个参数定义了应用和数据库之间在做序列化的时候DAO对象和MongoDB BSON存储的映射关系。如果不显示声明的话，将使用默认的映射关系。比如BigDecimal的Java类型并不会存储为Decimal128，而是转换成String进行存储。如果想要存储为Decimal128，最简单的手段就是在定义字段的时候做如下声明：</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Field</span>(name = <span class="string">"balance"</span>, targetType = FieldType.DECIMAL128)</span><br><span class="line"> <span class="keyword">var</span> balance: BigDecimal = BigDecimal(<span class="number">0.0</span>)</span><br></pre></td></tr></table></figure>
<p>在Spring Data Mongo的MongoConverters中声明了对象的转化实现，有兴趣可以阅读相关代码，这里贴出一部分代码：</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">abstract</span> <span class="class"><span class="keyword">class</span> <span class="title">MongoConverters</span> </span>{</span><br><span class="line"> <span class="function"><span class="keyword">private</span> <span class="title">MongoConverters</span><span class="params">()</span> </span>{</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">static</span> Collection<Object> <span class="title">getConvertersToRegister</span><span class="params">()</span> </span>{</span><br><span class="line"> List<Object> converters = <span class="keyword">new</span> ArrayList();</span><br><span class="line"> converters.add(MongoConverters.BigDecimalToStringConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.BigDecimalToDecimal128Converter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.StringToBigDecimalConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.Decimal128ToBigDecimalConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.BigIntegerToStringConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.StringToBigIntegerConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.URLToStringConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.StringToURLConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.DocumentToStringConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.TermToStringConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.NamedMongoScriptToDocumentConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.DocumentToNamedMongoScriptConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.CurrencyToStringConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.StringToCurrencyConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.AtomicIntegerToIntegerConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.AtomicLongToLongConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.LongToAtomicLongConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.IntegerToAtomicIntegerConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.BinaryToByteArrayConverter.INSTANCE);</span><br><span class="line"> converters.add(MongoConverters.BsonTimestampToInstantConverter.INSTANCE);</span><br><span class="line"> converters.add(ConverterBuilder.reading(String.class, URI.class, URI::create).andWriting(URI::toString));</span><br><span class="line"> <span class="keyword">return</span> converters;</span><br><span class="line"> }</span><br><span class="line"> ...</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h1 id="使用MongoTemplate查询DB并支持分页"><a href="#使用MongoTemplate查询DB并支持分页" class="headerlink" title="使用MongoTemplate查询DB并支持分页"></a>使用MongoTemplate查询DB并支持分页</h1><p>MongoTemplete的使用相当的直观<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="MongoTempate: https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongo-template
">[3]</span></a></sup>，这里不做示范，直接参考官方文档即可。需要说明的时候，MongoDB的分页支持，需要自己实现。这里贴出MongoDB分页查询的工具类，以供参考<sup id="fnref:4"><a href="#fn:4" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="MongoPageHelepr原始博客: https://www.cnblogs.com/woshimrf/p/mongodb-pagenation-performance.html
">[4]</span></a></sup>。</p>
<figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">package</span> demo;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> com.schdri.bimgis.data.manage.common.entity.PageResult;</span><br><span class="line"><span class="keyword">import</span> org.springframework.beans.factory.annotation.Autowired;</span><br><span class="line"><span class="keyword">import</span> org.springframework.data.domain.Sort;</span><br><span class="line"><span class="keyword">import</span> org.springframework.data.domain.Sort.Direction;</span><br><span class="line"><span class="keyword">import</span> org.springframework.data.domain.Sort.Order;</span><br><span class="line"><span class="keyword">import</span> org.springframework.data.mongodb.core.MongoTemplate;</span><br><span class="line"><span class="keyword">import</span> org.springframework.data.mongodb.core.query.Criteria;</span><br><span class="line"><span class="keyword">import</span> org.springframework.data.mongodb.core.query.Query;</span><br><span class="line"><span class="keyword">import</span> org.springframework.stereotype.Component;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.util.ArrayList;</span><br><span class="line"><span class="keyword">import</span> java.util.List;</span><br><span class="line"><span class="keyword">import</span> java.util.function.Function;</span><br><span class="line"><span class="keyword">import</span> java.util.stream.Collectors;</span><br><span class="line"><span class="keyword">import</span> java.util.stream.Stream;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * MongoDB分页查询工具类.</span></span><br><span class="line"><span class="comment"> **/</span></span><br><span class="line"><span class="meta">@Component</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">MongoPageHelper</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="keyword">int</span> FIRST_PAGE_NUM = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> <span class="keyword">int</span> DEFAULT_PAGE_SIZE = <span class="number">10</span>;</span><br><span class="line"> <span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">final</span> String ID = <span class="string">"_id"</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> MongoTemplate mongoTemplate;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Autowired</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="title">MongoPageHelper</span><span class="params">(MongoTemplate mongoTemplate)</span> </span>{</span><br><span class="line"> <span class="keyword">this</span>.mongoTemplate = mongoTemplate;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 分页查询，直接返回集合类型的结果.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> MongoPageHelper#pageQuery(Query,</span></span><br><span class="line"><span class="comment"> * Class, Function, Integer, Integer,</span></span><br><span class="line"><span class="comment"> * String)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <T> <span class="function">PageResult<T> <span class="title">pageQuery</span><span class="params">(Query query, Class<T> entityClass, Integer pageSize,</span></span></span><br><span class="line"><span class="function"><span class="params"> Integer pageNum, String collectionName)</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> pageQuery(query, entityClass, Function.identity(), pageSize, pageNum, <span class="keyword">null</span>, collectionName);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 分页查询，默认集合、可对集合元素进行转换</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> MongoPageHelper#pageQuery(Query,</span></span><br><span class="line"><span class="comment"> * Class, Function, Integer, Integer,</span></span><br><span class="line"><span class="comment"> * String)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <T, R> <span class="function">PageResult<R> <span class="title">pageQuery</span><span class="params">(Query query, Class<T> entityClass, Function<T, R> mapper,</span></span></span><br><span class="line"><span class="function"><span class="params"> Integer pageSize, Integer pageNum)</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> pageQuery(query, entityClass, mapper, pageSize, pageNum, <span class="keyword">null</span>, <span class="keyword">null</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 分页查询，使用lastId分页，直接返回集合类型的结果.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> MongoPageHelper#pageQuery(Query,</span></span><br><span class="line"><span class="comment"> * Class, Function, Integer, Integer,</span></span><br><span class="line"><span class="comment"> * String)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <T> <span class="function">PageResult<T> <span class="title">pageQuery</span><span class="params">(Query query, Class<T> entityClass, Integer pageSize,</span></span></span><br><span class="line"><span class="function"><span class="params"> Integer pageNum, Long lastId, String collectionName)</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> pageQuery(query, entityClass, Function.identity(), pageSize, pageNum, lastId, collectionName);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 分页查询，不考虑条件分页，直接使用skip-limit来分页.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@see</span> MongoPageHelper#pageQuery(Query,</span></span><br><span class="line"><span class="comment"> * Class, Function, Integer, Integer,</span></span><br><span class="line"><span class="comment"> * String)</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <T, R> <span class="function">PageResult<R> <span class="title">pageQuery</span><span class="params">(Query query, Class<T> entityClass, Function<T, R> mapper,</span></span></span><br><span class="line"><span class="function"><span class="params"> Integer pageSize, Integer pageNum, String collectionName)</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> pageQuery(query, entityClass, mapper, pageSize, pageNum, <span class="keyword">null</span>, collectionName);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 分页查询.</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> query Mongo Query对象，构造你自己的查询条件.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> entityClass Mongo collection定义的entity class，用来确定查询哪个集合.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> mapper 映射器，你从db查出来的list的元素类型是entityClass, 如果你想要转换成另一个对象，比如去掉敏感字段等，可以使用mapper来决定如何转换.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> pageSize 分页的大小.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> pageNum 当前页.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> lastId 条件分页参数, 区别于skip-limit，采用find(_id>lastId).limit分页.</span></span><br><span class="line"><span class="comment"> * 如果不跳页，像朋友圈，微博这样下拉刷新的分页需求，需要传递上一页的最后一条记录的ObjectId。 如果是null，则返回pageNum那一页.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <T> collection定义的class类型.</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <R> 最终返回时，展现给页面时的一条记录的类型。</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> collectionName 集合名称</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> PageResult，一个封装page信息的对象.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <T, R> <span class="function">PageResult<R> <span class="title">pageQuery</span><span class="params">(Query query, Class<T> entityClass, Function<T, R> mapper,</span></span></span><br><span class="line"><span class="function"><span class="params"> Integer pageSize, Integer pageNum, Long lastId, String collectionName)</span> </span>{</span><br><span class="line"> <span class="comment">//总条数</span></span><br><span class="line"> <span class="keyword">long</span> total = StringUtils.isEmpty(collectionName) ? mongoTemplate.count(query, entityClass) : mongoTemplate.count(query, entityClass, collectionName);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (pageNum == <span class="keyword">null</span> || pageNum < <span class="number">0</span>) {</span><br><span class="line"> pageNum = FIRST_PAGE_NUM;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (pageSize == <span class="keyword">null</span> || pageSize <= <span class="number">0</span>) {</span><br><span class="line"> pageSize = DEFAULT_PAGE_SIZE;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">//计算取记录的范围</span></span><br><span class="line"> <span class="keyword">final</span> Integer pages = (<span class="keyword">int</span>) Math.ceil(total / (<span class="keyword">double</span>) pageSize);</span><br><span class="line"> <span class="keyword">if</span> (pageNum > (pages - <span class="number">1</span>)) {</span><br><span class="line"> pageNum = FIRST_PAGE_NUM;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">//确定分页方式</span></span><br><span class="line"> <span class="keyword">if</span> (lastId != <span class="keyword">null</span> && lastId > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">if</span> (pageNum != FIRST_PAGE_NUM) {</span><br><span class="line"> query.addCriteria(Criteria.where(ID).gt(lastId));</span><br><span class="line"> }</span><br><span class="line"> query.limit(pageSize);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">int</span> skip = pageSize * pageNum;</span><br><span class="line"> query.skip(skip).limit(pageSize);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">//查询</span></span><br><span class="line"> <span class="keyword">final</span> List<T> entityList = StringUtils.isEmpty(collectionName) ? mongoTemplate</span><br><span class="line"> .find(query</span><br><span class="line"> .with(Sort.by(<span class="keyword">new</span> Order(Direction.ASC, ID))),</span><br><span class="line"> entityClass) :</span><br><span class="line"> mongoTemplate</span><br><span class="line"> .find(query</span><br><span class="line"> .with(Sort.by(<span class="keyword">new</span> Order(Direction.ASC, ID))),</span><br><span class="line"> entityClass, collectionName);</span><br><span class="line"></span><br><span class="line"> <span class="comment">//组织查询结果</span></span><br><span class="line"> <span class="keyword">final</span> PageResult<R> pageResult = <span class="keyword">new</span> PageResult<>();</span><br><span class="line"> pageResult.setTotal(total);</span><br><span class="line"> pageResult.setPages(pages);</span><br><span class="line"> pageResult.setPageSize(pageSize);</span><br><span class="line"> pageResult.setPageNum(pageNum);</span><br><span class="line"> pageResult.setData(entityList.stream().map(mapper).collect(Collectors.toList()));</span><br><span class="line"> <span class="keyword">return</span> pageResult;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 内嵌列表分页查询：将内嵌的列表展开关联外层集合的数据形成新列表进行分页查询</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> query 查询筛选参数</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> entityClass 外层集合映射的实体类</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> flatMapper 展开内嵌列表操作的Lambda表达式</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> pageSize 每页记录数</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> pageNum 第几页，从0开始</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> collectionName 外层集合名称，为null则使用entityClass关联的默认集合</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <T> 外层集合实体类</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <R> 经flatMapper转换且分页操作后返回的实体类</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <T, R> <span class="function">PageResult<R> <span class="title">pageQueryByInner</span><span class="params">(Query query, Class<T> entityClass, Function<T, Stream<R>> flatMapper,</span></span></span><br><span class="line"><span class="function"><span class="params"> Integer pageSize, Integer pageNum, String collectionName)</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="comment">//查询数据库返回外层集合列表</span></span><br><span class="line"> <span class="keyword">final</span> List<T> entityList = StringUtils.isEmpty(collectionName) ? mongoTemplate</span><br><span class="line"> .find(query.with(Sort.by(<span class="keyword">new</span> Order(Direction.ASC, ID))),</span><br><span class="line"> entityClass) :</span><br><span class="line"> mongoTemplate</span><br><span class="line"> .find(query.with(Sort.by(<span class="keyword">new</span> Order(Direction.ASC, ID))),</span><br><span class="line"> entityClass, collectionName);</span><br><span class="line"></span><br><span class="line"> <span class="comment">//转换为内嵌集合列表</span></span><br><span class="line"> <span class="keyword">final</span> List<R> innerEntityList = entityList == <span class="keyword">null</span> ? <span class="keyword">null</span> : entityList.stream().flatMap(flatMapper).collect(Collectors.toList());</span><br><span class="line"></span><br><span class="line"> <span class="comment">//生成分页参数</span></span><br><span class="line"> <span class="keyword">int</span> total = innerEntityList == <span class="keyword">null</span> ? <span class="number">0</span> : innerEntityList.size();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (pageNum == <span class="keyword">null</span> || pageNum < <span class="number">0</span>) {</span><br><span class="line"> pageNum = FIRST_PAGE_NUM;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (pageSize == <span class="keyword">null</span> || pageSize <= <span class="number">0</span>) {</span><br><span class="line"> pageSize = DEFAULT_PAGE_SIZE;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> Integer pages = (<span class="keyword">int</span>) Math.ceil(total / (<span class="keyword">double</span>) pageSize);</span><br><span class="line"> <span class="keyword">if</span> (pageNum > (pages - <span class="number">1</span>)) {</span><br><span class="line"> pageNum = FIRST_PAGE_NUM;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">//分页返回</span></span><br><span class="line"> <span class="keyword">final</span> PageResult<R> pageResult = <span class="keyword">new</span> PageResult<>();</span><br><span class="line"> <span class="keyword">if</span> (total > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">int</span> fromIndex = pageSize * pageNum;</span><br><span class="line"> <span class="keyword">int</span> toIndexTmp = pageSize * (pageNum + <span class="number">1</span>);</span><br><span class="line"> <span class="keyword">int</span> toIndex = toIndexTmp > total ? total : toIndexTmp;</span><br><span class="line"> pageResult.setData(innerEntityList.subList(fromIndex, toIndex));</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> pageResult.setData(<span class="keyword">new</span> ArrayList<>());</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> pageResult.setTotal(Long.valueOf(total));</span><br><span class="line"> pageResult.setPages(pages);</span><br><span class="line"> pageResult.setPageSize(pageSize);</span><br><span class="line"> pageResult.setPageNum(pageNum);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> pageResult;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 不分页列表查询，指定集合的数据</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> query 筛选条件</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> entityClass 列表元素类</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> collectionName mongoDB集合名称</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <T></span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <T> <span class="function">PageResult<T> <span class="title">noPageQuery</span><span class="params">(Query query, Class<T> entityClass, String collectionName)</span> </span>{</span><br><span class="line"></span><br><span class="line"> List<T> entityList = <span class="keyword">null</span>; <span class="comment">//查询结果实体</span></span><br><span class="line"> Long total = <span class="number">0L</span>;<span class="comment">//总条数</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">//查询</span></span><br><span class="line"> <span class="keyword">if</span> (!StringUtils.isEmpty(collectionName)) {</span><br><span class="line"> total = mongoTemplate.count(query, entityClass, collectionName);</span><br><span class="line"> entityList = mongoTemplate.find(query, entityClass, collectionName);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> total = mongoTemplate.count(query, entityClass);</span><br><span class="line"> entityList = mongoTemplate.find(query, entityClass);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> PageResult<T> pageResult = <span class="keyword">new</span> PageResult<>();</span><br><span class="line"> pageResult.setTotal(total);</span><br><span class="line"> pageResult.setPages(<span class="number">1</span>);</span><br><span class="line"> pageResult.setPageSize(total.intValue());</span><br><span class="line"> pageResult.setPageNum(<span class="number">0</span>);</span><br><span class="line"> pageResult.setData(entityList);</span><br><span class="line"> <span class="keyword">return</span> pageResult;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 不分页列表查询，指定集合的数据,支持数据转换</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> query 筛选条件</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> entityClass 列表元素类</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> collectionName mongoDB集合名称</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <T></span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <T, R> <span class="function">PageResult<R> <span class="title">noPageQuery</span><span class="params">(Query query, Class<T> entityClass, String collectionName, Function<T, R> mapper)</span> </span>{</span><br><span class="line"></span><br><span class="line"> List<T> entityList = <span class="keyword">null</span>; <span class="comment">//查询结果实体</span></span><br><span class="line"> Long total = <span class="number">0L</span>;<span class="comment">//总条数</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">//查询</span></span><br><span class="line"> <span class="keyword">if</span> (!StringUtils.isEmpty(collectionName)) {</span><br><span class="line"> total = mongoTemplate.count(query, entityClass, collectionName);</span><br><span class="line"> entityList = mongoTemplate.find(query, entityClass, collectionName);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> total = mongoTemplate.count(query, entityClass);</span><br><span class="line"> entityList = mongoTemplate.find(query, entityClass);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">final</span> PageResult<R> pageResult = <span class="keyword">new</span> PageResult<>();</span><br><span class="line"> pageResult.setTotal(total);</span><br><span class="line"> pageResult.setPages(<span class="number">1</span>);</span><br><span class="line"> pageResult.setPageSize(total.intValue());</span><br><span class="line"> pageResult.setPageNum(<span class="number">0</span>);</span><br><span class="line"> pageResult.setData(entityList.stream().map(mapper).collect(Collectors.toList()));</span><br><span class="line"> <span class="keyword">return</span> pageResult;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 不分页列表查询,默认集合的数据</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> query</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> entityClass</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> <T></span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span></span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="keyword">public</span> <T> <span class="function">PageResult<T> <span class="title">noPageQuery</span><span class="params">(Query query, Class<T> entityClass)</span> </span>{</span><br><span class="line"> <span class="keyword">return</span> noPageQuery(query, entityClass, <span class="keyword">null</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h1 id="使用Spring-Data-Repository接口"><a href="#使用Spring-Data-Repository接口" class="headerlink" title="使用Spring Data Repository接口"></a>使用Spring Data Repository接口</h1><p>Spring Data中的Repository是一个比较核心的抽象概念，它标识将数据访问层的代码尽量的声明式使用，而非写一大堆的模板代码。在Spring Data背后，可以使结合具体的底层存储系统，比如SQL, MongoDB, ElaticSearch等等，都是通过引入额外的支持库。由于Spring是自动配置的，所以只要引入相应的starter, 就可以了。</p>
<h2 id="先引入Spring-Data-MongoDB的starter依赖"><a href="#先引入Spring-Data-MongoDB的starter依赖" class="headerlink" title="先引入Spring Data MongoDB的starter依赖"></a>先引入Spring Data MongoDB的starter依赖</h2><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="string">libraries.springDataRest</span> <span class="string">=</span> <span class="string">"org.springframework.boot:spring-boot-starter-data-rest:${versions.springBootVersion}"</span></span><br><span class="line"><span class="string">libraries.springDataMongoDB</span> <span class="string">=</span> <span class="string">"org.springframework.boot:spring-boot-starter-data-mongodb:${versions.springBootVersion}"</span></span><br></pre></td></tr></table></figure>
<h2 id="再声明DAO对象对应的Repository"><a href="#再声明DAO对象对应的Repository" class="headerlink" title="再声明DAO对象对应的Repository"></a>再声明DAO对象对应的Repository</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RepositoryRestResource</span>(exported = <span class="keyword">true</span>)</span><br><span class="line">interface SectionRepository : MongoRepository<Section?, String?>{</span><br><span class="line"> <span class="meta">@Query</span>(<span class="string">"{ 'project_code' : ?0,'section_name': ?1}"</span>)</span><br><span class="line"> <span class="function">fun <span class="title">findByProjectCodeAndSectionName</span><span class="params">(projectCode: String?, sectionName: String?)</span>: MutableIterable<Section?>?</span></span><br><span class="line"><span class="function">}</span></span><br></pre></td></tr></table></figure>
<h2 id="自动配置Repository"><a href="#自动配置Repository" class="headerlink" title="自动配置Repository"></a>自动配置Repository</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Configuration</span></span><br><span class="line"><span class="meta">@EnableMongoRepositories</span>(basePackages = [<span class="string">"your_own_package_name"</span>])</span><br><span class="line">class ResourceExposureConfig : RepositoryRestConfigurerAdapter() {</span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 之所以配置这个，是因为想在接口查询的时候返回主键</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function">override fun <span class="title">configureRepositoryRestConfiguration</span><span class="params">(config: RepositoryRestConfiguration)</span> </span>{</span><br><span class="line"> config.exposeIdsFor(Section::<span class="class"><span class="keyword">class</span>.<span class="title">java</span></span></span><br><span class="line"><span class="class"> )</span></span><br><span class="line"><span class="class"> }</span></span><br><span class="line"><span class="class">}</span></span><br></pre></td></tr></table></figure>
<h2 id="在业务层注入Repository，直接使用即可"><a href="#在业务层注入Repository，直接使用即可" class="headerlink" title="在业务层注入Repository，直接使用即可"></a>在业务层注入Repository，直接使用即可</h2><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@RestController</span></span><br><span class="line"><span class="meta">@RequestMapping</span>(<span class="string">"/"</span>)</span><br><span class="line"><span class="meta">@Api</span>(value = <span class="string">"接口描述"</span>, tags = [<span class="string">"接口描述"</span>])</span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ProjectController</span> </span>{</span><br><span class="line"> <span class="keyword">private</span> val logger: Logger = LoggerFactory.getLogger(<span class="keyword">this</span>.javaClass)</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Autowired</span></span><br><span class="line"> <span class="keyword">private</span> lateinit <span class="keyword">var</span> sectionRepo: SectionRepository</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Autowired</span></span><br><span class="line"> <span class="keyword">private</span> lateinit <span class="keyword">var</span> mongoPageHelper: MongoPageHelper</span><br><span class="line"></span><br><span class="line"> <span class="meta">@PostMapping</span>(<span class="string">"demo"</span>)</span><br><span class="line"> <span class="meta">@ApiOperation</span>(value = <span class="string">""</span>)</span><br><span class="line"> <span class="function">fun <span class="title">someInterface</span><span class="params">(@RequestBody x: X)</span>: ResponseMessage<X> </span>{</span><br><span class="line"> <span class="comment">//在可以就可以开心的调用Repository的各种接口了</span></span><br><span class="line"> TODO()</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>如果官方提供的CrudRepository和QueryByExampleExecutor还不满足的话，直接按照规则声明对应的接口就行了。参考官方文档<sup id="fnref:5"><a href="#fn:5" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="Spring Reposiroty 自定义查询：https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongodb.repositories.queries">[5]</span></a></sup>。当然更多的支持的是查询。如果想要去修改数据，或者用复杂的规则去更新数据的话，可能mongoTemplate更适合你。</p>
<p>结束。后面随时补充。</p>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">Spring Data MongoDB https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#introduction<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">Spring Boot程序配置项：https://docs.spring.io/spring-boot/docs/current/reference/html/appendix-application-properties.html<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">MongoTempate: https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongo-template<a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">MongoPageHelepr原始博客: https://www.cnblogs.com/woshimrf/p/mongodb-pagenation-performance.html<a href="#fnref:4" rev="footnote"> ↩</a></span></li><li id="fn:5"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">5.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">Spring Reposiroty 自定义查询：https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongodb.repositories.queries<a href="#fnref:5" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>架构</category>
</categories>
<tags>
<tag>MongoDB</tag>
<tag>建模</tag>
<tag>NoSQL</tag>
<tag>Spring</tag>
<tag>Spring Cloud</tag>
</tags>
</entry>
<entry>
<title>为什么会害怕人工智能</title>
<url>/2020/02/12/%E4%B8%BA%E4%BB%80%E4%B9%88%E4%BC%9A%E5%AE%B3%E6%80%95%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/</url>
<content><![CDATA[<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/hint.css/2.4.1/hint.min.css"><p>原作者： 保罗▪福特 2015年2月11日<br>译者： 山野@成都 2017</p>
<h1 id="在阅读之前"><a href="#在阅读之前" class="headerlink" title="在阅读之前"></a>在阅读之前</h1><p>这篇文章<sup id="fnref:1"><a href="#fn:1" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://www.technologyreview.com/s/534871/our-fear-of-artificial-intelligence/
">[1]</span></a></sup>是我在2017年在麻省理工评论（MIT Technology Review）上看到的翻译的。 作者是保罗▪福特<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="https://www.technologyreview.com/profile/paul-ford/
">[2]</span></a></sup>。距离当初的翻译已经过去了2年。现在人工智能的概念已经深入到各行各业，从大家经常接触的SNS社交数据挖掘、智能客服、B2C商品推荐系统、新闻推荐系统、智能汽车，以及到更广泛的更偏向于传统的领域，比如医疗、城市规划、交通领域、建筑领域、社区管理、行政服务等等的一切都在提到一个词，那就是“智慧”，“智能”。</p>
<p>这一切的背后都和机器学习技术（深度学习、加强学习）等等的兴起，以及更加容易使用的框架和API，比如谷歌的tensorflow；更加容易处理大规模数据的框架，比如Apache Spark、Apache Hadoop套件等，这些都助力于并提高了各种业务对于基于大量数据的建模和预测能力。而这些能力就是大家所说的人工智能。</p>
<p>下面这篇文章并没有讲解具体的某种应用于人工智能的技术，而是从更高更抽象的层面，从社会和伦理的层面探讨了人工智能所带来的一些问题。</p>
<p>下面是正文：</p>
<a id="more"></a>
<hr>
<p><strong>* 真正的人工智能可能毁灭这个世界——但前提是人工智能可以被实现。*</strong></p>
<p>信任计算机去控制复杂的系统。</p>
<p>几年前我和一个创业的朋友喝咖啡。他刚满40，生病的父亲和背部疼痛让他感觉自己快要被生活压垮。 “别笑话我”, 他说， “我可是指望着奇点呐。”</p>
<p>我朋友在科技领域工作；他看到了更快的芯片处理器和网络带来的巨大变化。他相信，在他中年时，机器的智能会超过人类——一个被未来学家称之为奇点的时刻。一个友善的超级智能以更快的速度分析人类的基因，解开青春不老的秘密。至少，它可能知道怎么治好背部的疼痛。</p>
<p>可是，如果它不这么友善呢？领导牛津大学人类未来研究所的哲学家Nick Bostrom，在《Superintelligence》描述了这样一种场景，引发了对人工智能的未来的大量讨论。假设有有一个名字叫做“回形针最大化”的机器，它的目的就是尽可能的制造更多的回形针。现在，让我们再想象一下，这个机器变得无比的智能。为了达到目的，他可能制造出一种新的，更有效率的回形针生产设备——直到，它将所有的东西都变成回形针，如同米达斯把一切都变成黄金。</p>
<p>你也许会说，不用担心，只需要编程让它只制造一百万个回形针就停止了啊。但是，假如它完成之后决定检查一下工作呢？计数是否准确？无疑它需要变得更加聪明。这个超级智能的机器制造某种还未发明的、原始计算材料（姑且叫它“计算单元”），然后用它来检查每一个疑问。每一个疑问引发另外的疑问，直到整个地球都被转化成这种“计算单元”，而不是那一百万个回形针。</p>
<p>其实，Bostrom并不认为回形针制造器会真的出现。这是一个思想实验：用来表明无论多么精心设计的系统都无法约束极端的机器智能。不过他确信，超级智能肯定会产生，有可能会很棒，Bostrom认为也有另外一种可能：它不再需要人类。或者做了一些事情，毁灭了整个地球。第八章的标题是：默认的结局是毁灭吗？</p>
<p>如果你觉得很可笑，这很正常。批评者诸如机器人先锋Rodney Brooks评论道，当我们说计算机思考或者正在变得聪明时，这些担心AI会偏离正常轨道的人对计算机在做什么有所误解。从这个角度来看，Bostrom所描述的超级智能会在遥远的未来，也许永远都不会出现。</p>
<p>但依然有很多聪明而有见地的人通知Bostrom的观点，并表示忧虑，这又是为何呢？</p>
<h1 id="自由意志"><a href="#自由意志" class="headerlink" title="自由意志"></a>自由意志</h1><p>机器可以思考吗？ 这个问题从一开始就给计算机科学蒙上了一层阴影。阿兰图灵在1950年提出了一个观点：可以像教小孩一样教机器。1955年，LISP语言的发明者，约翰 · 麦卡锡，发明了”人工智能“这个词。60年代和70年代，AI研究人员用计算机来识别图像、在不同语言之间进行翻译，识别自然语言中的指令，而不仅仅是代码，与此同时，计算机最终发展出可以交谈和思考的能力——进而变得邪恶——的看法开始进入主流文化中。不提老生常谈的《2001，空间奥德赛》的Hal电脑，在1970年的电影《巨人：福宾计划》，展示了一个巨大的闪闪微光的主机计算机将整个世界带入核战的边缘。13年以后，星际大战游戏也探索了同样的主题。1973年的电影《西部世界》中，机器人发疯了，随后开始了屠杀。</p>
<p>当AI的研究和高远的目标遥不可及的时候，AI领域的投资大幅缩水，开始了“漫长的冬天”。即便如此，智能机器的火炬在80年代和90年代被诸如科幻作者Vernor Vinge，研究学者诸如机器人专家、计算机视觉专家Hans Moravec，以及企业家兼工程师Ray Kurzweil等接力而向前发展。Vinge让奇点这个观点变得流行。Kurzweil1999年出版了《精神机器的时代》。图灵提出类人智能的假设，而Vinge、Moravec、Kurzweil想的更加深远：当一个计算机能够独立找到方法达到目标，很有可能它具备了内省的能力——修改自己的软件让自己变得更加智能。说白了，这样的计算机能够自己设计硬件。</p>
<p>像Kurzweil所描绘的那样，这将是一个美好新时代。这样的计算有足够的insight和耐心（用皮秒来计量）来解决纳米科技和空间飞行的难题；他们可以提高人类的生活水平，可以让我们把意识上传成为一种永生的电子形式。智能可以在整个宇宙中扩散。</p>
<p>对于这种阳光乐观派，亦存在完全相反的观点。霍金警告说，由于人类无法和高级AI竞争，这 “会是人类种族终结的魔咒<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://www.bbc.com/news/technology-30290540
">[3]</span></a></sup>”。在阅读了《Superintelligence》之后，埃隆马斯克发推文说，“希望我们不是数字超级智能的生物启动加载器。不幸的是，越来越有这种可能性。” 马斯克接着给未来生活研究所捐助了一百万美金。和Bostrom的研究中心不同的是，这是一个“致力于降低全人类生存风险”，其中包括来自于 “开发人类水平的人工智能”。</p>
<p>现在还没有任何证据表明超级智能的存在。事实上，对如何达到通用目的的人工智能，几乎没有什么进展，甚至连如何达到目标，也没有清晰的路径。近期AI方面的发展，包括Apple的Siri助手和谷歌的无人驾驶汽车， 都揭示出当前技术存在严重的局限。在遇到之前从来没有遇到过的问题的时候，两者都束手无策。人工神经网络通过学习可以识别照片中的猫<sup id="fnref:4"><a href="#fn:4" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://www.technologyreview.com/news/523181/an-ai-chip-to-help-computers-understand-images/
">[4]</span></a></sup>，但是在此之前，需要经过上万个例子的学习，并且和小孩相比，依然不够准确。</p>
<p>这也是为什么有许多人，比如Brooks（iRobot和Rethink Robotics的创始人)<sup id="fnref:5"><a href="#fn:5" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://www.rethinkrobotics.com/artificial-intelligence-tool-threat/)
">[5]</span></a></sup>，表示出怀疑。和早期计算机的能力相比，现在的计算机可谓非常令人感到惊讶，即便如此，计算机可以识别出图片中的猫，但是因为机器没有意志，它不知道猫到底是什么，也不知道图片中正在发生着什么，也不具备许多其他人类所独有的洞察力。按照这个观点来看，也许可以达到机器具有智能的地步，但是还需要做的工作远超Bostrom的想象了。假设即便发生了，也不意味着这就是末日审判。从当前的AI发展推导出未来的超级AI，“如同看到热效率提升的内燃机就推出曲速引擎唾手可得的结论一样荒谬”, Brooks在近期在Edge.org<sup id="fnref:6"><a href="#fn:6" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://edge.org/response-detail/26057
">[6]</span></a></sup>上写到。 至少数百年之内，没有必要担心“存在恶意的超级智能”。</p>
<h1 id="保险策略"><a href="#保险策略" class="headerlink" title="保险策略"></a>保险策略</h1><p>超级智能的崛起可能在遥远的未来，但因此冒风险就有点不负责任了。加州大学伯克利计算机科学教授，斯图尔特 罗素(Stuart J.Russell)表达了和Bostrom同样的担心。罗素和Peter Novig(Kurzweil在谷歌的同事)一起合著了20年来的AI领域的标准教科书《人工智能：一种现代方法》。</p>
<p>“有很多本应非常聪明的公众学者对此毫无头绪”，罗素告诉我，并指出，在过去的十年中, AI的发展极为巨大。公众也许会从摩尔定律的角度来理解AI的发展（更快的计算机可以做更多的事情），实际上，近期AI的工作更多的是更底层的基础研究，使用类似深度学习这样的技术给计算机增加对周遭世界的自动感知。</p>
<p>鉴于谷歌，Facebook和其他公司正在积极创造一种聪明的，“会学习的”的机器，他说，“我想说的是，我们不应该做的事情之一就是全力以赴研发超级智能，而没有去考虑其潜在的风险。这显得有点狂热了。” 罗素做了一个比喻：“这就是像是核聚变试验。如果你问一个核聚变研究员他们在做什么，他们会说他们的工作就是控制。如果你需要无限制的能量，那么你需要控制核聚变反应。” 同样的，如果你需要无限制的智能，那你最好把计算机和人类的需求利益保持一致。</p>
<p>Bostrom的书中建议现在开始研究超级智能和人类利益关系。一个超级智能如同上帝一样，可是他是充满愤怒的还是充满爱意？ 这取决于我们（也就是，工程师们）。如同父母一样，我们必须给孩子设定价值观。并且不是任意价值观，而是哪些符合人类利益的价值观。基本上，我们是在告诉一个神我们应当怎样被对待。那么，如何做到呢？</p>
<p>Bostrom的设想主要建立在一个叫Eliezer Yudkowsky的思想家提出的“统一推理意志”之上——也即是我们所有人心中更好的自己。我们希望，AI会带给我们富有、欢乐、充实的人生：解决背部的疼痛，告诉我们如何去火星旅行。并且，由于我们无法真正就某一件事情达成一致，有时候我们需要AI帮我们做出决定——对人类全体利益最有利的决定。 那么，我们应该怎样把这些价值观编程到超级智能中呢？用什么样的数学公式来定义它？这些都是研究者们应该着手去解决的问题，并且是我们这个时代最核心的任务。</p>
<p>对普罗大众来说，没有必要因为机器人的可怕而辗转失眠。我们当前还没有任何技术能够达到人工智能。也即是为什么一些超级大公司深度投资研发智能计算机；一个真正的AI可以使得其中任何一家公司获得难以想象的优势。不过，他们也应当了解其中的劣势并且找到规避的办法。</p>
<p>这点闹人的建议是基于一封在未来生活研究所的官方网站的公开信<sup id="fnref:7"><a href="#fn:7" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://futureoflife.org/misc/open_letter
">[7]</span></a></sup>（没有宣称AI战争的来临）。信中号召在如何收割AI的收益并避免潜在瑕疵方面投入更多的研究，而不是警告人类的生存灾难。这封信的签名有不仅来自AI外行如霍金、马斯克和Bostrom等人的签名，还有知名计算机科学家（包括顶级AI研究员Demis Hassabis<sup id="fnref:8"><a href="#fn:8" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="http://www.technologyreview.com/news/532876/googles-intelligence-designer/">[8]</span></a></sup>）。你可以清楚看到他们从哪里来。毕竟，如何他们创造了一个不认同人类价值观的AI，那就是说，他们还不够聪明，无法控制他们自己的创造物。</p>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.technologyreview.com/s/534871/our-fear-of-artificial-intelligence/<a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">https://www.technologyreview.com/profile/paul-ford/<a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://www.bbc.com/news/technology-30290540<a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://www.technologyreview.com/news/523181/an-ai-chip-to-help-computers-understand-images/<a href="#fnref:4" rev="footnote"> ↩</a></span></li><li id="fn:5"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">5.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://www.rethinkrobotics.com/artificial-intelligence-tool-threat/)<a href="#fnref:5" rev="footnote"> ↩</a></span></li><li id="fn:6"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">6.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://edge.org/response-detail/26057<a href="#fnref:6" rev="footnote"> ↩</a></span></li><li id="fn:7"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">7.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://futureoflife.org/misc/open_letter<a href="#fnref:7" rev="footnote"> ↩</a></span></li><li id="fn:8"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">8.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;">http://www.technologyreview.com/news/532876/googles-intelligence-designer/<a href="#fnref:8" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>科技 - 人工智能</category>
<category>翻译</category>
</categories>
<tags>
<tag>人工智能</tag>
<tag>未来科技</tag>
<tag>麻省理工评论</tag>
<tag>自由意志</tag>
</tags>
</entry>
<entry>
<title>如何设计一个标签系统？</title>
<url>/2021/07/27/%E5%A6%82%E4%BD%95%E8%AE%BE%E8%AE%A1%E4%B8%80%E4%B8%AA%E6%A0%87%E7%AD%BE%E7%B3%BB%E7%BB%9F%EF%BC%9F/</url>
<content><
">[1]</span></a></sup>一文中已经列举出了三种设计方式，并对三种方式做了一些性能测试<sup id="fnref:2"><a href="#fn:2" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="[Tagsystems: performance tests](http://howto.philippkeller.com/2005/06/19/Tagsystems-performance-tests/)
">[2]</span></a></sup>。</p>
<ol>
<li>“MySQLicious” solution</li>
<li>“Scuttle” solution</li>
<li>“Toxi” solution<br>在这里不做赘述。</li>
</ol>
<h1 id="Toxi设计"><a href="#Toxi设计" class="headerlink" title="Toxi设计"></a>Toxi设计</h1><p>数据库表结构如下，这种表可以随便修改成你想要的任意对象和扩展。</p>
<figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> objects (</span><br><span class="line"> <span class="keyword">id</span> <span class="built_in">bigint</span> <span class="keyword">NOT</span> <span class="literal">NULL</span> AUTO_INCREMENT,</span><br><span class="line"> <span class="keyword">name</span> <span class="built_in">varchar</span>(<span class="number">128</span>) <span class="keyword">NOT</span> <span class="literal">NULL</span>,</span><br><span class="line"> <span class="keyword">url</span> <span class="built_in">varchar</span>(<span class="number">128</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</span><br><span class="line"> description <span class="built_in">text</span> <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</span><br><span class="line"> is_delete <span class="built_in">tinyint</span> <span class="keyword">DEFAULT</span> <span class="number">0</span>,</span><br><span class="line"> create_user <span class="built_in">varchar</span>(<span class="number">64</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</span><br><span class="line"> create_date datetime <span class="keyword">DEFAULT</span> <span class="keyword">CURRENT_TIMESTAMP</span>,</span><br><span class="line"> update_user <span class="built_in">varchar</span>(<span class="number">64</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</span><br><span class="line"> update_date datetime <span class="keyword">DEFAULT</span> <span class="keyword">CURRENT_TIMESTAMP</span> <span class="keyword">ON</span> <span class="keyword">UPDATE</span> <span class="keyword">CURRENT_TIMESTAMP</span>,</span><br><span class="line"> PRIMARY <span class="keyword">KEY</span> (<span class="keyword">id</span>)</span><br><span class="line">)</span><br><span class="line"><span class="keyword">ENGINE</span> = <span class="keyword">INNODB</span>,</span><br><span class="line"><span class="built_in">CHARACTER</span> <span class="keyword">SET</span> utf8,</span><br><span class="line"><span class="keyword">COLLATE</span> utf8_general_ci,</span><br><span class="line"><span class="keyword">COMMENT</span> = <span class="string">'对象表'</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> tags (</span><br><span class="line"> <span class="keyword">id</span> <span class="built_in">bigint</span> <span class="keyword">NOT</span> <span class="literal">NULL</span> AUTO_INCREMENT,</span><br><span class="line"> tag <span class="built_in">varchar</span>(<span class="number">256</span>) <span class="keyword">NOT</span> <span class="literal">NULL</span>,</span><br><span class="line"> is_delete <span class="built_in">tinyint</span> <span class="keyword">DEFAULT</span> <span class="number">0</span>,</span><br><span class="line"> create_user <span class="built_in">varchar</span>(<span class="number">64</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</span><br><span class="line"> create_date datetime <span class="keyword">DEFAULT</span> <span class="keyword">CURRENT_TIMESTAMP</span>,</span><br><span class="line"> update_user <span class="built_in">varchar</span>(<span class="number">64</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</span><br><span class="line"> update_date datetime <span class="keyword">DEFAULT</span> <span class="keyword">CURRENT_TIMESTAMP</span> <span class="keyword">ON</span> <span class="keyword">UPDATE</span> <span class="keyword">CURRENT_TIMESTAMP</span>,</span><br><span class="line"> PRIMARY <span class="keyword">KEY</span> (<span class="keyword">id</span>)</span><br><span class="line">)</span><br><span class="line"><span class="keyword">ENGINE</span> = <span class="keyword">INNODB</span>,</span><br><span class="line"><span class="built_in">CHARACTER</span> <span class="keyword">SET</span> utf8,</span><br><span class="line"><span class="keyword">COLLATE</span> utf8_general_ci,</span><br><span class="line"><span class="keyword">COMMENT</span> = <span class="string">'标签元数据表'</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> object_tag_relation (</span><br><span class="line"> <span class="keyword">id</span> <span class="built_in">bigint</span> <span class="keyword">NOT</span> <span class="literal">NULL</span> AUTO_INCREMENT,</span><br><span class="line"> object_id <span class="built_in">bigint</span> <span class="keyword">NOT</span> <span class="literal">NULL</span>,</span><br><span class="line"> tag_id <span class="built_in">bigint</span> <span class="keyword">NOT</span> <span class="literal">NULL</span>,</span><br><span class="line"> is_delete <span class="built_in">tinyint</span> <span class="keyword">DEFAULT</span> <span class="number">0</span>,</span><br><span class="line"> create_user <span class="built_in">varchar</span>(<span class="number">64</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</span><br><span class="line"> create_date datetime <span class="keyword">DEFAULT</span> <span class="keyword">CURRENT_TIMESTAMP</span>,</span><br><span class="line"> update_user <span class="built_in">varchar</span>(<span class="number">64</span>) <span class="keyword">DEFAULT</span> <span class="literal">NULL</span>,</span><br><span class="line"> update_date datetime <span class="keyword">DEFAULT</span> <span class="keyword">CURRENT_TIMESTAMP</span> <span class="keyword">ON</span> <span class="keyword">UPDATE</span> <span class="keyword">CURRENT_TIMESTAMP</span>,</span><br><span class="line"> PRIMARY <span class="keyword">KEY</span> (<span class="keyword">id</span>),</span><br><span class="line"> <span class="keyword">INDEX</span> tag_id(tag_id)</span><br><span class="line">)</span><br><span class="line"><span class="keyword">ENGINE</span> = <span class="keyword">INNODB</span>,</span><br><span class="line"><span class="built_in">CHARACTER</span> <span class="keyword">SET</span> utf8,</span><br><span class="line"><span class="keyword">COLLATE</span> utf8_general_ci,</span><br><span class="line"><span class="keyword">COMMENT</span> = <span class="string">'对象标签关系表'</span>;</span><br></pre></td></tr></table></figure>
<div id="footnotes"><hr><div id="footnotelist"><ol style="list-style: none; padding-left: 0; margin-left: 40px"><li id="fn:1"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">1.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="http://howto.philippkeller.com/2005/04/24/Tags-Database-schemas/" target="_blank" rel="noopener">Tags: Database schemas</a><a href="#fnref:1" rev="footnote"> ↩</a></span></li><li id="fn:2"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">2.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="http://howto.philippkeller.com/2005/06/19/Tagsystems-performance-tests/" target="_blank" rel="noopener">Tagsystems: performance tests</a><a href="#fnref:2" rev="footnote"> ↩</a></span></li><li id="fn:3"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">3.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="http://web.mit.edu/6.033/2007/wwwdocs/writing-samples/Tag-ChuanZhang.pdf" target="_blank" rel="noopener">Simple, Effective and Reliable Tag Storage System Design</a><a href="#fnref:3" rev="footnote"> ↩</a></span></li><li id="fn:4"><span style="display: inline-block; vertical-align: top; padding-right: 10px; margin-left: -40px">4.</span><span style="display: inline-block; vertical-align: top; margin-left: 10px;"><a href="https://jgefroh.medium.com/architecture-tagging-stuff-dcda3218880a" target="_blank" rel="noopener">How to Design Software — Tags and Groups</a><a href="#fnref:4" rev="footnote"> ↩</a></span></li></ol></div></div>]]></content>
<categories>
<category>系统设计</category>
</categories>
<tags>
<tag>标签</tag>
</tags>
</entry>
<entry>
<title>如何降低MongoDB的使用内存</title>
<url>/2021/07/28/%E5%A6%82%E4%BD%95%E9%99%8D%E4%BD%8EMongoDB%E7%9A%84%E4%BD%BF%E7%94%A8%E5%86%85%E5%AD%98/</url>
<content><
">[2]</span></a></sup>。<a id="more"></a>
</li>
</ol>
<h1 id="配置MongoDB的内存占用"><a href="#配置MongoDB的内存占用" class="headerlink" title="配置MongoDB的内存占用"></a>配置MongoDB的内存占用</h1><p>在MongoDB的配置文件中配置允许缓存占用的最大内存值，这里的单位是GByte</p>
<figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="attr">storage:</span></span><br><span class="line"> <span class="attr">dbPath:</span> <span class="string"><string></span></span><br><span class="line"> <span class="attr">journal:</span></span><br><span class="line"> <span class="attr">enabled:</span> <span class="string"><boolean></span></span><br><span class="line"> <span class="attr">commitIntervalMs:</span> <span class="string"><num></span></span><br><span class="line"> <span class="attr">directoryPerDB:</span> <span class="string"><boolean></span></span><br><span class="line"> <span class="attr">syncPeriodSecs:</span> <span class="string"><int></span></span><br><span class="line"> <span class="attr">engine:</span> <span class="string"><string></span></span><br><span class="line"> <span class="attr">wiredTiger:</span></span><br><span class="line"> <span class="attr">engineConfig:</span></span><br><span class="line"> <span class="attr">cacheSizeGB:</span> <span class="string"><number></span></span><br><span class="line"> <span class="attr">journalCompressor:</span> <span class="string"><string></span></span><br><span class="line"> <span class="attr">directoryForIndexes:</span> <span class="string"><boolean></span></span><br><span class="line"> <span class="attr">maxCacheOverflowFileSizeGB:</span> <span class="string"><number></span> <span class="string">//</span> <span class="string">deprecated</span> <span class="string">in</span> <span class="string">MongoDB</span> <span class="number">4.4</span></span><br><span class="line"> <span class="attr">collectionConfig:</span></span><br><span class="line"> <span class="attr">blockCompressor:</span> <span class="string"><string></span></span><br><span class="line"> <span class="attr">indexConfig:</span></span><br><span class="line"> <span class="attr">prefixCompression:</span> <span class="string"><boolean></span></span><br><span class="line"> <span class="attr">inMemory:</span></span><br><span class="line"> <span class="attr">engineConfig:</span></span><br><span class="line"> <span class="attr">inMemorySizeGB:</span> <span class="string"><number></span></span><br><span class="line"> <span class="attr">oplogMinRetentionHours:</span> <span class="string"><double></span></span><br></pre></td></tr></table></figure>
<p>对cacheSizeGB这个配置，官方的解释<sup id="fnref:3"><a href="#fn:3" rel="footnote"><span class="hint--top hint--error hint--medium hint--rounded hint--bounce" aria-label="[MongoDB Configuration Options](https://docs.mongodb.com/manual/reference/configuration-options/)
">[3]</span></a></sup>如下：</p>
<blockquote>
<p>Starting in MongoDB 3.4, the default WiredTiger internal cache size is the larger of either:</p>
<ul>
<li>50% of (RAM - 1 GB), or</li>
<li>256 MB.<br>For example, on a system with a total of 4GB of RAM the WiredTiger cache will use 1.5GB of RAM (0.5 * (4 GB - 1 GB) = 1.5 GB). Conversely, a system with a total of 1.25 GB of RAM will allocate 256 MB to the WiredTiger cache because that is more than half of the total RAM minus one gigabyte (0.5 * (1.25 GB - 1 GB) = 128 MB < 256 MB).<br>The storage.wiredTiger.engineConfig.cacheSizeGB limits the size of the WiredTiger internal cache. The operating system will use the available free memory for filesystem cache, which allows the compressed MongoDB data files to stay in memory. In addition, the operating system will use any free RAM to buffer file system blocks and file system cache.</li>
</ul>
<p>To accommodate the additional consumers of RAM, you may have to decrease WiredTiger internal cache size.</p>
</blockquote>
<p>意思是什么呢？ WiredTiger引擎缓存大小几乎没有上限。而在缓存占用内存快速上升的时候触发了OOM Killer，导致MongoDB实例进程直接被杀掉。所以要限制缓存的最大值。</p>
<p>在我们的系统中配置的值是4, 重新启动MongoDB服务之后，使用top命令观察如下：</p>
<img src="/2021/07/28/%E5%A6%82%E4%BD%95%E9%99%8D%E4%BD%8EMongoDB%E7%9A%84%E4%BD%BF%E7%94%A8%E5%86%85%E5%AD%98/D326D899-4744-4cf3-9D32-C6F5A2567BCF.png" class="" title>
<p>自此，问题已经解决了。后来观察集群，也没有出现过进程被杀掉的情况。</p>
<h1 id="关于systemd的一些补充"><a href="#关于systemd的一些补充" class="headerlink" title="关于systemd的一些补充"></a>关于systemd的一些补充</h1><p>MongoDB的服务启停建议都通过systemd来管理。</p>
<p>一般来讲，systemd的service文件可以放在这些目录下面：</p>
<ul>
<li>/usr/lib/systemd/system/</li>