7.7 服务远程暴露 - 订阅与通知（TODO）

为了安全：服务启动的ip全部使用10.10.10.10

远程服务的暴露总体步骤：

将ref封装为invoker
将invoker转换为exporter
启动netty
注册服务到zookeeper
订阅与通知
返回新的exporter实例

在7.4 服务远程暴露 - 创建Exporter与启动netty服务端中，实现了前三步，在7.6 服务远程暴露 - 注册服务到zookeeper实现了第四步。本节实现第五步：订阅。总体代码如下：RegistryProtocol.export(final Invoker<T> originInvoker)

1         // 订阅override数据
2         // FIXME 提供者订阅时，会影响同一JVM即暴露服务，又引用同一服务的的场景，因为subscribed以服务名为缓存的key，导致订阅信息覆盖。
3         final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registedProviderUrl);
4         final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);
5         overrideListeners.put(overrideSubscribeUrl, overrideSubscribeListener);
6         registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);

说明：

第一句代码根据registedProviderUrl来获取overrideSubscribeUrl。
第二句代码创建overrideSubscribeListener
第三句代码将{ overrideSubscribeUrl : overrideSubscribeListener放入缓存 }
第四句代码实现真正的订阅与通知

一获取overrideSubscribeUrl

final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registedProviderUrl);

1     /**
2      * 1 将协议改为provider；
3      * 2 添加参数：category=configurators和check=false；
4      */
5     private URL getSubscribedOverrideUrl(URL registedProviderUrl) {
6         return registedProviderUrl.setProtocol(Constants.PROVIDER_PROTOCOL)
7                                   .addParameters(Constants.CATEGORY_KEY, Constants.CONFIGURATORS_CATEGORY, Constants.CHECK_KEY, String.valueOf(false));
8     }

开始时的registedProviderUrl如下：

dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5259&side=provider&timestamp=1507294508053

最终的overrideSubscribeUrl如下：

provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5259&side=provider&timestamp=1507294508053

二创建overrideSubscribeListener

final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);

overrideSubscribeListener是RegistryProtocol的内部类，来看一下声明和属性：

1     private class OverrideListener implements NotifyListener {
2         private final URL subscribeUrl;
3         private final Invoker originInvoker;
4 
5         public OverrideListener(URL subscribeUrl, Invoker originalInvoker) {
6             this.subscribeUrl = subscribeUrl;
7             this.originInvoker = originalInvoker;
8         }

这里创建出来的OverrideListener实例属性如下：

subscribeUrl：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=1818&side=provider&timestamp=1507366969962
originInvoker：该实例还是在ServiceConfig.doExportUrlsFor1Protocol(ProtocolConfig protocolConfig, List<URL> registryURLs)创建出来的AbstractProxyInvoker实例（具体见7.4 服务远程暴露 - 创建Exporter与启动netty服务端）
- proxy：DemoServiceImpl实例
- type：Class<com.alibaba.dubbo.demo.DemoService>
- url：registry://10.211.55.5:2181/com.alibaba.dubbo.registry.RegistryService?application=demo-provider&client=curator&dubbo=2.0.0&export=dubbo%3A%2F%2F10.10.10.10%3A20880%2Fcom.alibaba.dubbo.demo.DemoService%3Fanyhost%3Dtrue%26application%3Ddemo-provider%26dubbo%3D2.0.0%26generic%3Dfalse%26interface%3Dcom.alibaba.dubbo.demo.DemoService%26methods%3DsayHello%26pid%3D993%26side%3Dprovider%26timestamp%3D1507100322516&pid=993&registry=zookeeper&timestamp=1507100319830

最后，将创建出来的OverrideListener实例存储在RegistryProtocol的属性Map<URL, NotifyListener> overrideListeners中：

key: （overrideSubscribeUrl，也就是subscribeUrl） provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=1818&side=provider&timestamp=1507366969962
value: 上述的OverrideListener实例

三真正的订阅

registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);

这里的registry是ZookeeperRegistry实例，subscribe(URL url, NotifyListener listener)方法在其父类FailbackRegistry中，如下：

 1     @Override
 2     public void subscribe(URL url, NotifyListener listener) {
 3         if (destroyed.get()){
 4             return;
 5         }
 6         super.subscribe(url, listener);
 7         removeFailedSubscribed(url, listener);
 8         try {
 9             // 向服务器端发送订阅请求
10             doSubscribe(url, listener);
11         } catch (Exception e) {
12             Throwable t = e;
13 
14             List<URL> urls = getCacheUrls(url);
15             if (urls != null && urls.size() > 0) {
16                 notify(url, listener, urls);
17                 logger.error("Failed to subscribe " + url + ", Using cached list: " + urls + " from cache file: " + getUrl().getParameter(Constants.FILE_KEY, System.getProperty("user.home") + "/dubbo-registry-" + url.getHost() + ".cache") + ", cause: " + t.getMessage(), t);
18             } else {
19                 // 如果开启了启动时检测check=true，则直接抛出异常
20                 boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
21                         && url.getParameter(Constants.CHECK_KEY, true);
22                 boolean skipFailback = t instanceof SkipFailbackWrapperException;
23                 if (check || skipFailback) {
24                     if (skipFailback) {
25                         t = t.getCause();
26                     }
27                     throw new IllegalStateException("Failed to subscribe " + url + ", cause: " + t.getMessage(), t);
28                 } else {
29                     logger.error("Failed to subscribe " + url + ", waiting for retry, cause: " + t.getMessage(), t);
30                 }
31             }
32             // 将失败的订阅请求记录到失败列表，定时重试
33             addFailedSubscribed(url, listener);
34         }
35     }

步骤：

首先调用其父类AbstractRegistry的方法，将之前创建出来的overrideSubscribeListener实例加入到overrideSubscribeUrl所对应的监听器集合中；
然后从failedSubscribed／failedUnsubscribed中overrideSubscribeUrl所对应的监听器集合中删除overrideSubscribeListener实例；从failedNotified获取当前url的通知失败map Map<NotifyListener, List<URL>>，之后从中删除掉该NotifyListener实例以及其需要通知的所有的url。
之后使用具体的子类（这里是ZookeeperRegistry）向服务器端发送订阅请求
如果在订阅的过程中抛出了异常，那么尝试获取缓存url，如果有缓存url，则进行失败通知，之后“将失败的订阅请求记录到失败列表，定时重试”，如果没有缓存url，如果开启了启动时检测或者直接抛出的异常是SkipFailbackWrapperException，则直接抛出异常，不会“将失败的订阅请求记录到失败列表，定时重试”

将之前创建出来的overrideSubscribeListener实例加入到overrideSubscribeUrl所对应的监听器集合中

 1    private final ConcurrentMap<URL, Set<NotifyListener>> subscribed = new ConcurrentHashMap<URL, Set<NotifyListener>>();//已经订阅的<URL, Set<NotifyListener>>
 2     
 3      /**
 4      * 首先从ConcurrentMap<URL, Set<NotifyListener>> subscribed中获取key为url的集合Set<NotifyListener>，
 5      * 如果该集合存在，直接将当前的NotifyListener实例存入该集合，
 6      * 如果集合不存在，先创建，之后放入subscribed中，并将当前的NotifyListener实例存入刚刚创建的集合
 7      *
 8      * @param url      订阅条件，不允许为空，如：consumer://10.10.10.10/com.alibaba.foo.BarService?version=1.0.0&application=kylin
 9      * @param listener 变更事件监听器，不允许为空
10      */
11     public void subscribe(URL url, NotifyListener listener) {
12         if (url == null) {
13             throw new IllegalArgumentException("subscribe url == null");
14         }
15         if (listener == null) {
16             throw new IllegalArgumentException("subscribe listener == null");
17         }
18         if (logger.isInfoEnabled()) {
19             logger.info("Subscribe: " + url);
20         }
21         Set<NotifyListener> listeners = subscribed.get(url);
22         if (listeners == null) {
23             subscribed.putIfAbsent(url, new ConcurrentHashSet<NotifyListener>());
24             listeners = subscribed.get(url);
25         }
26         listeners.add(listener);
27     }

从失败集合移除overrideSubscribeListener实例

 1     /**
 2      * 1 从ConcurrentMap<URL, Set<NotifyListener>> failedSubscribed 中获取当前url的订阅失败列表Set<NotifyListener>，之后从中删除掉该NotifyListener实例；
 3      * 2 从ConcurrentMap<URL, Set<NotifyListener>> failedUnsubscribed 中获取当前url的反订阅失败列表Set<NotifyListener>，之后从中删除掉该NotifyListener实例；
 4      * 3 从ConcurrentMap<URL, Map<NotifyListener, List<URL>>> failedNotified 中获取当前url的通知失败map Map<NotifyListener, List<URL>>，之后从中删除掉该NotifyListener实例以及其需要通知的所有的url。
 5      *
 6      * @param url
 7      * @param listener
 8      */
 9     private void removeFailedSubscribed(URL url, NotifyListener listener) {
10         Set<NotifyListener> listeners = failedSubscribed.get(url);
11         if (listeners != null) {
12             listeners.remove(listener);
13         }
14         listeners = failedUnsubscribed.get(url);
15         if (listeners != null) {
16             listeners.remove(listener);
17         }
18         Map<NotifyListener, List<URL>> notified = failedNotified.get(url);
19         if (notified != null) {
20             notified.remove(listener);
21         }
22     }

ZookeeperRegistry.doSubscribe(final URL url, final NotifyListener listener)

 1 protected void doSubscribe(final URL url, final NotifyListener listener) {
 2         try {
 3             if (Constants.ANY_VALUE.equals(url.getServiceInterface())) {//这条分支先不说
 4                 String root = toRootPath();
 5                 ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
 6                 if (listeners == null) {
 7                     zkListeners.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, ChildListener>());
 8                     listeners = zkListeners.get(url);
 9                 }
10                 ChildListener zkListener = listeners.get(listener);
11                 if (zkListener == null) {
12                     listeners.putIfAbsent(listener, new ChildListener() {
13                         public void childChanged(String parentPath, List<String> currentChilds) {
14                             for (String child : currentChilds) {
15                                 child = URL.decode(child);
16                                 if (!anyServices.contains(child)) {
17                                     anyServices.add(child);
18                                     subscribe(url.setPath(child).addParameters(Constants.INTERFACE_KEY, child,
19                                             Constants.CHECK_KEY, String.valueOf(false)), listener);
20                                 }
21                             }
22                         }
23                     });
24                     zkListener = listeners.get(listener);
25                 }
26                 zkClient.create(root, false);
27                 List<String> services = zkClient.addChildListener(root, zkListener);
28                 if (services != null && services.size() > 0) {
29                     for (String service : services) {
30                         service = URL.decode(service);
31                         anyServices.add(service);
32                         subscribe(url.setPath(service).addParameters(Constants.INTERFACE_KEY, service,
33                                 Constants.CHECK_KEY, String.valueOf(false)), listener);
34                     }
35                 }
36             } else {
37                 /**
38                  * ConcurrentMap<URL, ConcurrentMap<NotifyListener, ChildListener>> zkListeners
39                  * 1 根据url获取ConcurrentMap<NotifyListener, ChildListener>，没有就创建
40                  * 2 根据listener从ConcurrentMap<NotifyListener, ChildListener>获取ChildListener，没有就创建（创建的ChildListener用来监听子节点的变化）
41                  * 3 创建path持久化节点
42                  * 4 创建path子节点监听器
43                  */
44                 List<URL> urls = new ArrayList<URL>();
45                 for (String path : toCategoriesPath(url)) {
46                     ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
47                     if (listeners == null) {
48                         zkListeners.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, ChildListener>());
49                         listeners = zkListeners.get(url);
50                     }
51                     ChildListener zkListener = listeners.get(listener);
52                     if (zkListener == null) {
53                         listeners.putIfAbsent(listener, new ChildListener() {
54                             //监听子节点列表的变化
55                             public void childChanged(String parentPath, List<String> currentChilds) {
56                                 ZookeeperRegistry.this.notify(url, listener, toUrlsWithEmpty(url, parentPath, currentChilds));
57                             }
58                         });
59                         zkListener = listeners.get(listener);
60                     }
61                     zkClient.create(path, false);//创建持久化节点/dubbo/com.alibaba.dubbo.demo.DemoService/configurators
62                     List<String> children = zkClient.addChildListener(path, zkListener);
63                     if (children != null) {
64                         urls.addAll(toUrlsWithEmpty(url, path, children));
65                     }
66                 }
67                 notify(url, listener, urls);
68             }
69         } catch (Throwable e) {
70             throw new RpcException("Failed to subscribe " + url + " to zookeeper " + getUrl() + ", cause: " + e.getMessage(), e);
71         }
72     }

说明：

url（overrideSubscribeUrl）：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076
listener：之前创建出来的overrideSubscribeListener实例

步骤：

首先获取categorypath：实际上就是获取/dubbo/{servicename}/{url中的category参数，默认是providers，这里是final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registedProviderUrl);这句代码中添加到overrideSubscribeUrl上的category=configurators}

 1     private String[] toCategoriesPath(URL url) {
 2         String[] categroies;
 3         if (Constants.ANY_VALUE.equals(url.getParameter(Constants.CATEGORY_KEY))) {
 4             categroies = new String[]{Constants.PROVIDERS_CATEGORY, Constants.CONSUMERS_CATEGORY,
 5                     Constants.ROUTERS_CATEGORY, Constants.CONFIGURATORS_CATEGORY};
 6         } else {
 7             categroies = url.getParameter(Constants.CATEGORY_KEY, new String[]{Constants.DEFAULT_CATEGORY});
 8         }
 9         String[] paths = new String[categroies.length];
10         for (int i = 0; i < categroies.length; i++) {
11             paths[i] = toServicePath(url) + Constants.PATH_SEPARATOR + categroies[i];
12         }
13         return paths; // /dubbo/com.alibaba.dubbo.demo.DemoService/configurators
14     }

然后就是获取并创建：ConcurrentMap<overrideSubscribeUrl, ConcurrentMap<overrideSubscribeListener实例, ChildListener>> zkListeners，这里创建出来的ChildListener实例中的childChanged(String parentPath, List<String> currentChilds)方法实际上就是最终当parentPath（实际上就是上边的categorypath）下的currentChilds发生变化时，执行的逻辑。
之后创建持久化节点：/dubbo/com.alibaba.dubbo.demo.DemoService/configurators
然后使用AbstractZookeeperClient<TargetChildListener>的addChildListener(String path, final ChildListener listener)方法为path下的子节点添加上边创建出来的内部类ChildListener实例
最后进行通知

AbstractZookeeperClient<TargetChildListener>.addChildListener(String path, final ChildListener listener)

 1     /**
 2      *  1 根据path从ConcurrentMap<String, ConcurrentMap<ChildListener, TargetChildListener>> childListeners获取ConcurrentMap<ChildListener, TargetChildListener>，没有就创建
 3      *  2 根据ChildListener获取TargetChildListener，没有就创建，TargetChildListener是真正的监听path的子节点变化的监听器
 4      *  createTargetChildListener(String path, final ChildListener listener)：创建一个真正的用来执行当path节点的子节点发生变化时的逻辑
 5      *  3 addTargetChildListener(path, targetListener)：将刚刚创建出来的子节点监听器订阅path的变化，这样之后，path的子节点发生了变化时，TargetChildListener才会执行相应的逻辑。
 6      *  而实际上TargetChildListener又会调用ChildListener的实现类的childChanged(String parentPath, List<String> currentChilds)方法，而该实现类，正好是ZookeeperRegistry中实现的匿名内部类，
 7      *  在该匿名内部类的childChanged(String parentPath, List<String> currentChilds)方法中，调用了ZookeeperRegistry.notify(URL url, NotifyListener listener, List<URL> urls)
 8      */
 9     public List<String> addChildListener(String path, final ChildListener listener) {
10         ConcurrentMap<ChildListener, TargetChildListener> listeners = childListeners.get(path);
11         if (listeners == null) {
12             childListeners.putIfAbsent(path, new ConcurrentHashMap<ChildListener, TargetChildListener>());
13             listeners = childListeners.get(path);
14         }
15         TargetChildListener targetListener = listeners.get(listener);
16         if (targetListener == null) {
17             listeners.putIfAbsent(listener, createTargetChildListener(path, listener));
18             targetListener = listeners.get(listener);
19         }
20         return addTargetChildListener(path, targetListener);
21     }

步骤：

首先是一顿获取和创建：ConcurrentMap<categorypath, ConcurrentMap<ZookeeperRegistry的内部类ChildListener实例, TargetChildListener>> childListeners，这里主要是创建TargetChildListener；
之后是真正的为path添加TargetChildListener实例。

CuratorZookeeperClient.createTargetChildListener(path, listener)

 1     public CuratorWatcher createTargetChildListener(String path, ChildListener listener) {
 2         return new CuratorWatcherImpl(listener);
 3     }
 4 
 5     private class CuratorWatcherImpl implements CuratorWatcher {
 6 
 7         private volatile ChildListener listener;
 8 
 9         public CuratorWatcherImpl(ChildListener listener) {
10             this.listener = listener;
11         }
12 
13         public void unwatch() {
14             this.listener = null;
15         }
16 
17         public void process(WatchedEvent event) throws Exception {
18             if (listener != null) {
19                 listener.childChanged(event.getPath(), client.getChildren().usingWatcher(this).forPath(event.getPath()));
20             }
21         }
22     }

很简单，就是创建一个监听path子节点的watcher，当path下有子节点变化时，调用listener（即传入的ZookeeperRegistry的内部类ChildListener实例的childChanged(String parentPath, List<String> currentChilds)方法）。

CuratorZookeeperClient.addTargetChildListener(String path, CuratorWatcher targetChildListener)

1     public List<String> addTargetChildListener(String path, CuratorWatcher listener) {
2         try {
3             return client.getChildren().usingWatcher(listener).forPath(path);
4         } catch (NoNodeException e) {
5             return null;
6         } catch (Exception e) {
7             throw new IllegalStateException(e.getMessage(), e);
8         }
9     }

从上边的分析我们可以看出，当path节点下的子节点发生变化的时候，会首先调用TargetChildListener的process(WatchedEvent event)方法，在该方法中又会调用ChildListener实例的childChanged(String parentPath, List<String> currentChilds)方法，那么我们来分析一下该方法：

1                             //监听子节点列表的变化
2                             public void childChanged(String parentPath, List<String> currentChilds) {
3                                 ZookeeperRegistry.this.notify(url, listener, toUrlsWithEmpty(url, parentPath, currentChilds));
4                             }

步骤：

首先获取子节点urls或者是一个consumer的empty协议的url

 1     /**
 2      * 过滤出providers中与consumer匹配的url集合
 3      */
 4     private List<URL> toUrlsWithoutEmpty(URL consumer, List<String> providers) {
 5         List<URL> urls = new ArrayList<URL>();
 6         if (providers != null && providers.size() > 0) {
 7             for (String provider : providers) {
 8                 provider = URL.decode(provider);
 9                 if (provider.contains("://")) {
10                     URL url = URL.valueOf(provider);
11                     if (UrlUtils.isMatch(consumer, url)) {
12                         urls.add(url);
13                     }
14                 }
15             }
16         }
17         return urls;
18     }
19 
20     /**
21      * 1 首先过滤出providers中与consumer匹配的providerUrl集合
22      * 2 如果providerUrl集合不为空，直接返回这个集合
23      * 3 如果为空，首先从path中获取category，然后将consumer的协议换成empty，添加参数category=configurators
24      * @param consumer provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076
25      * @param path /dubbo/com.alibaba.dubbo.demo.DemoService/configurators
26      * @param providers 
27      */
28     private List<URL> toUrlsWithEmpty(URL consumer, String path, List<String> providers) {
29         List<URL> urls = toUrlsWithoutEmpty(consumer, providers);
30         if (urls == null || urls.isEmpty()) {
31             int i = path.lastIndexOf('/');
32             String category = i < 0 ? path : path.substring(i + 1);//configurators
33             URL empty = consumer.setProtocol(Constants.EMPTY_PROTOCOL).addParameter(Constants.CATEGORY_KEY, category);
34             urls.add(empty);
35         }
36         return urls; // empty://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=1237&side=provider&timestamp=1507352638483
37     }

之后调用ZookeeperRegistry的父类FailbackRegistry.notify(URL url, NotifyListener listener, List<URL> urls)

 1     @Override
 2     protected void notify(URL url, NotifyListener listener, List<URL> urls) {
 3         if (url == null) {
 4             throw new IllegalArgumentException("notify url == null");
 5         }
 6         if (listener == null) {
 7             throw new IllegalArgumentException("notify listener == null");
 8         }
 9         try {
10             doNotify(url, listener, urls);
11         } catch (Exception t) {
12             // 将失败的通知请求记录到失败列表，定时重试
13             Map<NotifyListener, List<URL>> listeners = failedNotified.get(url);
14             if (listeners == null) {
15                 failedNotified.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, List<URL>>());
16                 listeners = failedNotified.get(url);
17             }
18             listeners.put(listener, urls);
19             logger.error("Failed to notify for subscribe " + url + ", waiting for retry, cause: " + t.getMessage(), t);
20         }
21     }
22 
23     protected void doNotify(URL url, NotifyListener listener, List<URL> urls) {
24         super.notify(url, listener, urls);
25     }

说明：这里传入的

url（overrideSubscribeUrl）：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076
listener：之前创建出来的overrideSubscribeListener实例
urls：[ empty://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076 ]
这里首先执行父类的AbstractRegistry.notify(URL url, NotifyListener listener, List<URL> urls)，如果失败，则获取或创建ConcurrentMap<overrideSubscribeUrl, Map<overrideSubscribeListener实例, urls>> failedNotified，后续做重试

来看一下通知的最核心部分：

AbstractRegistry.notify(URL url, NotifyListener listener, List<URL> urls)

 1     /**
 2      * 1 首先遍历List<URL> urls，将urls按照category进行分类，存储在Map<"categoryName", List<URL>> result中；
 3      * 2 之后遍历result：（每遍历一次，都是一个新的category）
 4      * （1）将Map<"categoryName", List<URL>>存储在ConcurrentMap<URL, Map<String, List<URL>>> notified的Map<String, List<URL>>中
 5      * （2）进行properties设置和文件保存
 6      * （3）调用传入放入listener的notify()方法。
 7      * @param url
 8      * @param listener
 9      * @param urls
10      */
11     protected void notify(URL url, NotifyListener listener, List<URL> urls) {
12         if (url == null) {
13             throw new IllegalArgumentException("notify url == null");
14         }
15         if (listener == null) {
16             throw new IllegalArgumentException("notify listener == null");
17         }
18         if ((urls == null || urls.size() == 0)
19                 && !Constants.ANY_VALUE.equals(url.getServiceInterface())) {
20             logger.warn("Ignore empty notify urls for subscribe url " + url);
21             return;
22         }
23         if (logger.isInfoEnabled()) {
24             logger.info("Notify urls for subscribe url " + url + ", urls: " + urls);
25         }
26         /**
27          * 遍历List<URL> urls，将urls按照category进行分类
28          */
29         Map<String, List<URL>> result = new HashMap<String, List<URL>>(); //{ "categoryName" : List<URL> }
30         for (URL u : urls) {
31             if (UrlUtils.isMatch(url, u)) {
32                 String category = u.getParameter(Constants.CATEGORY_KEY, Constants.DEFAULT_CATEGORY);
33                 List<URL> categoryList = result.get(category);
34                 if (categoryList == null) {
35                     categoryList = new ArrayList<URL>();
36                     result.put(category, categoryList);
37                 }
38                 categoryList.add(u);
39             }
40         }
41         if (result.size() == 0) {
42             return;
43         }
44         Map<String, List<URL>> categoryNotified = notified.get(url);
45         if (categoryNotified == null) {
46             notified.putIfAbsent(url, new ConcurrentHashMap<String, List<URL>>());
47             categoryNotified = notified.get(url);
48         }
49         for (Map.Entry<String, List<URL>> entry : result.entrySet()) {
50             String category = entry.getKey();
51             List<URL> categoryList = entry.getValue();
52             categoryNotified.put(category, categoryList);//填充notified集合
53             saveProperties(url);//该行代码为什么不写在循环体外边
54             listener.notify(categoryList);
55         }
56     }

说明：这里传入的

url（overrideSubscribeUrl）：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076
listener：之前创建出来的overrideSubscribeListener实例
urls：[ empty://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076 ]

步骤：

首先遍历List<URL> urls，将urls按照category进行分类，存储在Map<"categoryName", List<URL>> result中；
然后获取或创建ConcurrentMap<overrideSubscribeUrl, Map<"categoryName", subList(urls)>> notified
最后遍历Map<"categoryName", List<URL>> result
- 去填充notified集合
- 保存传入的url到Properties properties（本地磁盘缓存中）
- 调用传入的listener的notify方法（注意：这里调用的正是文章开头创建的overrideSubscribeListener实例的notify方法）

AbstractRegistry.saveProperties(URL url)

 1     /**
 2      * 1 按照url从将ConcurrentMap<URL, Map<String, List<URL>>> notified中将Map<String, List<URL>>拿出来，之后将所有category的list组成一串buf（以空格分隔）
 3      * 2 将< serviceKey<->buf >写入本地磁盘缓存中：Properties properties
 4      * 3 将AtomicLong lastCacheChanged加1
 5      * 4 之后根据syncSaveFile判断时同步保存properties到文件，还是异步保存properties到文件
 6      * @param url
 7      */
 8     private void saveProperties(URL url) {
 9         if (file == null) {
10             return;
11         }
12 
13         try {
14             StringBuilder buf = new StringBuilder();
15             Map<String, List<URL>> categoryNotified = notified.get(url);
16             if (categoryNotified != null) {
17                 for (List<URL> us : categoryNotified.values()) {
18                     for (URL u : us) {
19                         if (buf.length() > 0) {
20                             buf.append(URL_SEPARATOR);
21                         }
22                         buf.append(u.toFullString());
23                     }
24                 }
25             }
26             properties.setProperty(url.getServiceKey(), buf.toString());
27             long version = lastCacheChanged.incrementAndGet();
28             if (syncSaveFile) {
29                 doSaveProperties(version);
30             } else {
31                 registryCacheExecutor.execute(new SaveProperties(version));
32             }
33         } catch (Throwable t) {
34             logger.warn(t.getMessage(), t);
35         }
36     }

说明：

入参：url：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5033&side=provider&timestamp=1507720343596
properties：{ "com.alibaba.dubbo.demo.DemoService" -> "empty://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5033&side=provider&timestamp=1507720343596" }
最后采用异步线程将properties中的内容写入到文件中

AbstractRegistry$SaveProperties

 1     private class SaveProperties implements Runnable {
 2         private long version;
 3 
 4         private SaveProperties(long version) {
 5             this.version = version;
 6         }
 7 
 8         public void run() {
 9             doSaveProperties(version);
10         }
11     }

AbstractRegistry.doSaveProperties(long version)

 1     /**
 2      * 1 先将文件中的内容读取到一个新的Properties newProperties中；
 3      * 2 之后将properties中的信息写入这个newProperties中；
 4      * 3 之后创建dubbo-registry-10.211.55.5.cache.lock文件；
 5      * 4 最后将这个newProperties中的内容写入到文件中
 6      */
 7     public void doSaveProperties(long version) {
 8         if (version < lastCacheChanged.get()) {
 9             return;
10         }
11         if (file == null) {
12             return;
13         }
14         Properties newProperties = new Properties();
15         // 保存之前先读取一遍，防止多个注册中心之间冲突
16         InputStream in = null;
17         try {
18             if (file.exists()) {
19                 in = new FileInputStream(file);
20                 newProperties.load(in);
21             }
22         } catch (Throwable e) {
23             logger.warn("Failed to load registry store file, cause: " + e.getMessage(), e);
24         } finally {
25             if (in != null) {
26                 try {
27                     in.close();
28                 } catch (IOException e) {
29                     logger.warn(e.getMessage(), e);
30                 }
31             }
32         }
33         // 保存
34         try {
35             newProperties.putAll(properties);
36             File lockfile = new File(file.getAbsolutePath() + ".lock");
37             if (!lockfile.exists()) {
38                 lockfile.createNewFile();//创建lock文件
39             }
40             RandomAccessFile raf = new RandomAccessFile(lockfile, "rw");
41             try {
42                 FileChannel channel = raf.getChannel();
43                 try {
44                     FileLock lock = channel.tryLock();
45                     if (lock == null) {
46                         throw new IOException("Can not lock the registry cache file " + file.getAbsolutePath() + ", ignore and retry later, maybe multi java process use the file, please config: dubbo.registry.file=xxx.properties");
47                     }
48                     // 保存
49                     try {
50                         if (!file.exists()) {
51                             file.createNewFile();
52                         }
53                         FileOutputStream outputFile = new FileOutputStream(file);
54                         try {
55                             newProperties.store(outputFile, "Dubbo Registry Cache");
56                         } finally {
57                             outputFile.close();
58                         }
59                     } finally {
60                         lock.release();
61                     }
62                 } finally {
63                     channel.close();
64                 }
65             } finally {
66                 raf.close();
67             }
68         } catch (Throwable e) {
69             if (version < lastCacheChanged.get()) {
70                 return;
71             } else {
72                 registryCacheExecutor.execute(new SaveProperties(lastCacheChanged.incrementAndGet()));
73             }
74             logger.warn("Failed to save registry store file, cause: " + e.getMessage(), e);
75         }
76     }

步骤见注释。这里有一个version，实际上是一个CAS判断，我们在saveProperties(URL url)方法中执行了long version = lastCacheChanged.incrementAndGet();之后在doSaveProperties(long version)进行if (version < lastCacheChanged.get())判断，如果满足这个条件，说明当前线程在进行doSaveProperties(long version)时，已经有其他线程执行了saveProperties(URL url)，马上就要执行doSaveProperties(long version)，所以当前线程放弃操作，让后边的这个线程来做保存操作。

保存操作执行之后，会在文件夹/Users/jigangzhao/.dubbo下生成两个文件：

dubbo-registry-10.211.55.5.cache
dubbo-registry-10.211.55.5.cache.lock

前者的内容：

#Wed Oct 11 19:42:29 CST 2017
com.alibaba.dubbo.demo.DemoService=empty\://10.10.10.10\:20880/com.alibaba.dubbo.demo.DemoService?anyhost\=true&application\=demo-provider&category\=configurators&check\=false&dubbo\=2.0.0&generic\=false&interface\=com.alibaba.dubbo.demo.DemoService&methods\=sayHello&pid\=5165&side\=provider&timestamp\=1507722024953

最后就是OverrideListener.notify(List<URL> urls)

 1     /**
 2      * 重新export
 3      * 1.protocol中的exporter destroy问题
 4      * 1.要求registryprotocol返回的exporter可以正常destroy
 5      * 2.notify后不需要重新向注册中心注册
 6      * 3.export 方法传入的invoker最好能一直作为exporter的invoker.
 7      */
 8     private class OverrideListener implements NotifyListener {
 9         private final URL subscribeUrl;
10         private final Invoker originInvoker;
11 
12         public OverrideListener(URL subscribeUrl, Invoker originalInvoker) {
13             this.subscribeUrl = subscribeUrl;
14             this.originInvoker = originalInvoker;
15         }
16 
17         /**
18          * 目的：
19          * 对原本注册了的providerUrl进行校验，如果url发生了变化，那么要重新export
20          *
21          * @param urls 已注册信息列表，总不为空，含义同{@link com.alibaba.dubbo.registry.RegistryService#lookup(URL)}的返回值。
22          */
23         public synchronized void notify(List<URL> urls) {
24             logger.debug("original override urls: " + urls);
25             List<URL> matchedUrls = getMatchedUrls(urls, subscribeUrl);
26             logger.debug("subscribe url: " + subscribeUrl + ", override urls: " + matchedUrls);
27             //没有匹配的
28             if (matchedUrls.isEmpty()) {
29                 return;
30             }
31 
32             List<Configurator> configurators = RegistryDirectory.toConfigurators(matchedUrls);//这里是一个空列表
33 
34             final Invoker<?> invoker;
35             if (originInvoker instanceof InvokerDelegete) {
36                 invoker = ((InvokerDelegete<?>) originInvoker).getInvoker();
37             } else {
38                 invoker = originInvoker;
39             }
40             //最原始的invoker
41             URL originUrl = RegistryProtocol.this.getProviderUrl(invoker);//dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5279&side=provider&timestamp=1507723571451
42             String key = getCacheKey(originInvoker);//dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5279&side=provider&timestamp=1507723571451
43             ExporterChangeableWrapper<?> exporter = bounds.get(key);//在doLocalExport方法中已经存放在这里了
44             if (exporter == null) {
45                 logger.warn(new IllegalStateException("error state, exporter should not be null"));
46                 return;
47             }
48             //当前的，可能经过了多次merge
49             URL currentUrl = exporter.getInvoker().getUrl();//dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5279&side=provider&timestamp=1507723571451
50             //与本次配置merge的
51             URL newUrl = getConfigedInvokerUrl(configurators, originUrl);
52             if (!currentUrl.equals(newUrl)) {
53                 RegistryProtocol.this.doChangeLocalExport(originInvoker, newUrl);//重新将invoker暴露为exporter
54                 logger.info("exported provider url changed, origin url: " + originUrl + ", old export url: " + currentUrl + ", new export url: " + newUrl);
55             }
56         }
57 
58         private List<URL> getMatchedUrls(List<URL> configuratorUrls, URL currentSubscribe) {
59             List<URL> result = new ArrayList<URL>();
60             for (URL url : configuratorUrls) {
61                 URL overrideUrl = url;
62                 // 兼容旧版本
63                 if (url.getParameter(Constants.CATEGORY_KEY) == null
64                         && Constants.OVERRIDE_PROTOCOL.equals(url.getProtocol())) {
65                     overrideUrl = url.addParameter(Constants.CATEGORY_KEY, Constants.CONFIGURATORS_CATEGORY);
66                 }
67 
68                 //检查是不是要应用到当前服务上
69                 if (UrlUtils.isMatch(currentSubscribe, overrideUrl)) {
70                     result.add(url);
71                 }
72             }
73             return result;
74         }
75 
76         //合并配置的url
77         private URL getConfigedInvokerUrl(List<Configurator> configurators, URL url) {
78             for (Configurator configurator : configurators) {
79                 url = configurator.configure(url);
80             }
81             return url;
82         }
83     }

最后：总结一下：

当前的provider订阅了/dubbo/com.alibaba.dubbo.demo.DemoService/configurators，当其下的子节点发生变化时，如果其下的子节点的url或者当前的providerUrl发生了变化，需要重新暴露。

重新暴露：

 1     /**
 2      * 对修改了url的invoker重新export
 3      *
 4      * @param originInvoker
 5      * @param newInvokerUrl
 6      */
 7     @SuppressWarnings("unchecked")
 8     private <T> void doChangeLocalExport(final Invoker<T> originInvoker, URL newInvokerUrl) {
 9         String key = getCacheKey(originInvoker);
10         final ExporterChangeableWrapper<T> exporter = (ExporterChangeableWrapper<T>) bounds.get(key);
11         if (exporter == null) {
12             logger.warn(new IllegalStateException("error state, exporter should not be null"));
13         } else {
14             final Invoker<T> invokerDelegete = new InvokerDelegete<T>(originInvoker, newInvokerUrl);
15             exporter.setExporter(protocol.export(invokerDelegete));
16         }
17     }

posted on 2017-10-11 20:38 赵计刚阅读(2685) 评论(0) 编辑收藏举报