反向路由检测 问题记录

　　 查看代码可知，查找路由后校验src dst ip 不过！

同时通过ip route查看ip 命中的路由

 

 可知 会发往lo接口

查看路由也能看到结果：

ip rule add fwmark 1 lookup 100 
ip route add local default dev lo table 100 
//内核中涉及到如下hokk

ip_fib_init in fib_frontend.c (net\ipv4) :
rtnl_register(PF_INET, RTM_NEWROUTE, inet_rtm_newroute, NULL, NULL);
rtnl_register(PF_INET, RTM_DELROUTE, inet_rtm_delroute, NULL, NULL);
rtnl_register(PF_INET, RTM_GETROUTE, NULL, inet_dump_fib, NULL);
fib_rules_init in fib_rules.c (net\core) :
rtnl_register(PF_UNSPEC, RTM_NEWRULE, fib_nl_newrule, NULL, NULL);
rtnl_register(PF_UNSPEC, RTM_DELRULE, fib_nl_delrule, NULL, NULL);
rtnl_register(PF_UNSPEC, RTM_GETRULE, NULL, fib_nl_dumprule, NULL);

 mark 为0x的命中table 100的路由 也就是local dev lo

　　既然命中了路由为啥还要去执行fib_validate_source  做校验呢？ 

因为收到包后就”必须回包“，所以要反向查找路由，免得后续回报找不到出口或者怎样

• 查看rpf 即为 rp_filter，sysctl -a | grep rp_filter 能查到对应的值。
• 当其为0时，不校验source_ip是否可达，
• 当前为1时，强校验source_ip 是否input_interface = output_interface
• 当前为2时，校验source_ip是否可达

/* Given (packet source, input interface) and optional (dst, oif, tos):
 * - (main) check, that source is valid i.e. not broadcast or our local
 *   address.
 * - figure out what "logical" interface this packet arrived
 *   and calculate "specific destination" address.
 * - check, that packet arrived from expected physical interface.
 * called with rcu_read_lock()
 */
static int __fib_validate_source(struct sk_buff *skb, __be32 src, __be32 dst,
                 u8 tos, int oif, struct net_device *dev,
                 int rpf, struct in_device *idev, u32 *itag)
{
    int ret, no_addr;
    struct fib_result res;
    struct flowi4 fl4;
    struct net *net;
    bool dev_match;
//组装查找key
//注意源地址和目的地址互换，然后查路由之所以何种方式查路由，是有一定意图的。
//首先valid_source顾名思义，就是看源地址是否有效，怎样说明才是有效，看我有没有到你的路由
//如果我有到你的路由，好吧，暂且认为你有效。如果没有到你的路由，而你却偏偏给我发了一个报文
//那么我认为你是伪造原地址了。(该逻辑的前提是“对称路由”，即连在相同网络中的设备，都有相同的路由)

    fl4.flowi4_oif = 0;
    fl4.flowi4_iif = l3mdev_master_ifindex_rcu(dev);
    if (!fl4.flowi4_iif)
        fl4.flowi4_iif = oif ? : LOOPBACK_IFINDEX;
    fl4.daddr = src;
    fl4.saddr = dst;
    fl4.flowi4_tos = tos;
    fl4.flowi4_scope = RT_SCOPE_UNIVERSE;
    fl4.flowi4_tun_key.tun_id = 0;
    fl4.flowi4_flags = 0;
//入接口没有ip地址 no_addr =1
    no_addr = idev->ifa_list == NULL;

    fl4.flowi4_mark = IN_DEV_SRC_VMARK(idev) ? skb->mark : 0;

    trace_fib_validate_source(dev, &fl4);

    net = dev_net(dev);
    //........又查了一次路由
//如果没有查到，即我没有到你的路由goto last_resort;在 last_resort中判断if (rpf)，即判断是否开启了防护
    if (fib_lookup(net, &fl4, &res, 0))// 查询失败是允许的，只要rp_filter未启用
        goto last_resort;
        //如果查询的结果不是RTN_UNICAST,不行。你这个报文到这是RTN_UNICAST的，而我到你却不是RTN_UNICAST的，扔掉
        //排除同时 LOCAL路由 接口开启local 情况  可以参考这个patch http://patchwork.ozlabs.org/project/netdev/patch/20091203112557.15100.85827.sendpatchset@x2.localnet/
        /*Change fib_validate_source() to accept packets with a local source address when
    the "accept_local" sysctl is set for the incoming inet device. Combined with the
    previous patches, this allows to communicate between multiple local interfaces
    over the wire.*/
    if (res.type != RTN_UNICAST &&
        (res.type != RTN_LOCAL || !IN_DEV_ACCEPT_LOCAL(idev)))
        goto e_inval;
    if (!rpf && !fib_num_tclassid_users(dev_net(dev)) &&
        (dev->ifindex != oif || !IN_DEV_TX_REDIRECTS(idev)))
        goto last_resort;
    fib_combine_itag(itag, &res);
    dev_match = false;

#ifdef CONFIG_IP_ROUTE_MULTIPATH
    for (ret = 0; ret < res.fi->fib_nhs; ret++) {
        struct fib_nh *nh = &res.fi->fib_nh[ret];

        if (nh->nh_dev == dev) {
            dev_match = true;
            break;
        } else if (l3mdev_master_ifindex_rcu(nh->nh_dev) == dev->ifindex) {
            dev_match = true;
            break;
        }
    }
#else
    if (FIB_RES_DEV(res) == dev)
        dev_match = true;//如果我查询的出接口，和你报文的入接口时一个，那么ok，我认为你是合法的
#endif
    if (dev_match) {
        ret = FIB_RES_NH(res).nh_scope >= RT_SCOPE_HOST;
        return ret;
    }
    
    //到这步了，表示查询到了我到你的路由，但是出接口和你入接口不匹配
    if (no_addr)
        goto last_resort;
    if (rpf == 1)
        goto e_rpf;
        
        //到这表示，我没有开启安全防护，入接口有ip，而你给我的报文不是从这我查询的出接口出去的
//勉强通过，不过我得看看是否可以从我查询的出接口，有到你的路由，即指定出接口查询路由。如果查询成功，就表示，
//从你这个入口出去，其实也可以到达你，所以算了，放过你吧，不扔掉你了。

    fl4.flowi4_oif = dev->ifindex;

    ret = 0;
    if (fib_lookup(net, &fl4, &res, FIB_LOOKUP_IGNORE_LINKSTATE) == 0) {
        if (res.type == RTN_UNICAST)
            ret = FIB_RES_NH(res).nh_scope >= RT_SCOPE_HOST;
    }
    return ret;

last_resort:
    if (rpf)
        goto e_rpf;
    *itag = 0;
    return 0;

e_inval:
    return -EINVAL;
e_rpf:
    return -EXDEV;
}
/* Ignore rp_filter for packets protected by IPsec. */
int fib_validate_source(struct sk_buff *skb, __be32 src, __be32 dst,
            u8 tos, int oif, struct net_device *dev,
            struct in_device *idev, u32 *itag)
{
    
    //该接口开启防护，rpf = 1
    int r = secpath_exists(skb) ? 0 : IN_DEV_RPFILTER(idev);

    if (!r && !fib_num_tclassid_users(dev_net(dev)) &&
        IN_DEV_ACCEPT_LOCAL(idev) &&
        (dev->ifindex != oif || !IN_DEV_TX_REDIRECTS(idev))) {
        *itag = 0;
        return 0;
    }
    return __fib_validate_source(skb, src, dst, tos, oif, dev, r, idev, itag);
}

 

根据fib_validate_source 的log ：

 可知只需要关闭rpf 就行

 ip_rcv

　--> ip_rcv_finish

　　--> ip_route_input_noref  ##如果skb还没有目的条目（路由相关），初始化虚拟路径cache

　　　　##目的地址是组播地址 ，这就是我们要分析的一支路径

　　 　　-->ip_route_input_mc -->fib_validate_source --> __fib_validate_source -->fib_lookup

　　　　##else 目的地址 非组播地址

　　　　-->ip_route_input_slow

　　　　　　-->fib_validate_source ## 通过fib_lookup查找到RTN_LOCAL类型路由，做反向检查，最终走local_input流程

　　　　　　　　　　　　　　　　##查找到RTN_BROADCAST类型路由且源地址非全0，也做反向检查

　　　　　　##不满足RTN_LOCAL和RTN_BROADCAST类型路由，则调用ip_mkroute_intput, 创建route cache entry

　　　　　　-->ip_mkroute_input-->__mkroute_input-->fib_validate_source

/*
 *    NOTE. We drop all the packets that has local source
 *    addresses, because every properly looped back packet
 *    must have correct destination already attached by output routine.
 *
 *    Such approach solves two big problems:
 *    1. Not simplex devices are handled properly.
 *    2. IP spoofing attempts are filtered with 100% of guarantee.
 *    called with rcu_read_lock()
 */

static int ip_route_input_slow(struct sk_buff *skb, __be32 daddr, __be32 saddr,
                   u8 tos, struct net_device *dev)
{
    struct fib_result res;
    struct in_device *in_dev = __in_dev_get_rcu(dev);
    struct ip_tunnel_info *tun_info;
    struct flowi4    fl4;
    unsigned int    flags = 0;
    u32        itag = 0;
    struct rtable    *rth;
    int        err = -EINVAL;
    struct net    *net = dev_net(dev);
    bool do_cache;
------------------------------------

    /*
     *    Now we are ready to route packet.
     */
    fl4.flowi4_oif = 0;
    fl4.flowi4_iif = l3mdev_fib_oif_rcu(dev);
    fl4.flowi4_mark = skb->mark;
    fl4.flowi4_tos = tos;
    fl4.flowi4_scope = RT_SCOPE_UNIVERSE;
    fl4.flowi4_flags = 0;
    fl4.daddr = daddr;
    fl4.saddr = saddr;
    err = fib_lookup(net, &fl4, &res, 0);
    if (err != 0) {
        if (!IN_DEV_FORWARD(in_dev))
            err = -EHOSTUNREACH;
        goto no_route;
    }

    if (res.type == RTN_BROADCAST)
        goto brd_input;

    if (res.type == RTN_LOCAL) {
        err = fib_validate_source(skb, saddr, daddr, tos,
                      0, dev, in_dev, &itag);
        if (err < 0)
            goto martian_source;
        goto local_input;
    }

    if (!IN_DEV_FORWARD(in_dev)) {
        err = -EHOSTUNREACH;
        goto no_route;
    }
    if (res.type != RTN_UNICAST)
        goto martian_destination;

    err = ip_mkroute_input(skb, &res, &fl4, in_dev, daddr, saddr, tos);
out:    return err;

brd_input:
    if (skb->protocol != htons(ETH_P_IP))
        goto e_inval;

    if (!ipv4_is_zeronet(saddr)) {
        err = fib_validate_source(skb, saddr, 0, tos, 0, dev,
                      in_dev, &itag);
        if (err < 0)
            goto martian_source;
    }
    flags |= RTCF_BROADCAST;
    res.type = RTN_BROADCAST;
    RT_CACHE_STAT_INC(in_brd);

local_input:
    do_cache = false;
    if (res.fi) {
        if (!itag) {
            rth = rcu_dereference(FIB_RES_NH(res).nh_rth_input);
            if (rt_cache_valid(rth)) {
                skb_dst_set_noref(skb, &rth->dst);
                err = 0;
                goto out;
            }
            do_cache = true;
        }
    }

    rth = rt_dst_alloc(net->loopback_dev, flags | RTCF_LOCAL, res.type,
               IN_DEV_CONF_GET(in_dev, NOPOLICY), false, do_cache);
    if (!rth)
        goto e_nobufs;

    rth->dst.output= ip_rt_bug;
    rth->rt_is_input = 1;
    if (res.table)
        rth->rt_table_id = res.table->tb_id;

    RT_CACHE_STAT_INC(in_slow_tot);
    if (res.type == RTN_UNREACHABLE) {
        rth->dst.input= ip_error;
        rth->dst.error= -err;
        rth->rt_flags     &= ~RTCF_LOCAL;
    }
    if (do_cache) {
        if (unlikely(!rt_cache_route(&FIB_RES_NH(res), rth))) {
            rth->dst.flags |= DST_NOCACHE;
            rt_add_uncached_list(rth);
        }
    }
    skb_dst_set(skb, &rth->dst);
    err = 0;
    goto out;

no_route:
    RT_CACHE_STAT_INC(in_no_route);
    res.type = RTN_UNREACHABLE;
    res.fi = NULL;
    res.table = NULL;
    goto local_input;

    /*
     *    Do not cache martian addresses: they should be logged (RFC1812)
     */
martian_destination:
    RT_CACHE_STAT_INC(in_martian_dst);

e_inval:
    err = -EINVAL;
    goto out;

e_nobufs:
    err = -ENOBUFS;
    goto out;

martian_source:
    ip_handle_martian_source(dev, in_dev, skb, daddr, saddr);
    goto out;
}

 

static inline int fib_lookup(struct net *net, struct flowi4 *flp,
                 struct fib_result *res, unsigned int flags)
{
    struct fib_table *tb;
    int err = -ENETUNREACH;

    flags |= FIB_LOOKUP_NOREF;
     /* 系统配置了路由规则，因此走这个路径； 只要添加了rule，此值为1，即使删除添加的rule，仍为1 */
    if (net->ipv4.fib_has_custom_rules)
        return __fib_lookup(net, flp, res, flags);

    rcu_read_lock();
    /* 无策略路由规则时，直接查询local/main/default三张路由表 */
    res->tclassid = 0;

    tb = rcu_dereference_rtnl(net->ipv4.fib_main);
    if (tb)
        err = fib_table_lookup(tb, flp, res, flags);

    if (!err)
        goto out;

    tb = rcu_dereference_rtnl(net->ipv4.fib_default);
    if (tb)
        err = fib_table_lookup(tb, flp, res, flags);

out:
    if (err == -EAGAIN)
        err = -ENETUNREACH;

    rcu_read_unlock();

    return err;
}

 

int __fib_lookup(struct net *net, struct flowi4 *flp,
         struct fib_result *res, unsigned int flags)
{
    struct fib_lookup_arg arg = {
        .result = res,
        .flags = flags,
    };
    int err;
    /* 通过路由规则查询对应的路由表 
    分析代码我们可以导到，路由匹配函数为fib4_rule_match，一般action函数为fib4_rule_action，在下面会继续分析这两个函数。*/
    err = fib_rules_lookup(net->ipv4.rules_ops, flowi4_to_flowi(flp), 0, &arg);
#ifdef CONFIG_IP_ROUTE_CLASSID
    if (arg.rule)
        res->tclassid = ((struct fib4_rule *)arg.rule)->tclassid;
    else
        res->tclassid = 0;
#endif

    if (err == -ESRCH)
        err = -ENETUNREACH;

    return err;
}

当 rp_filter=1  src_valid_mark=0
022] IPv4: dev_name:eth10  src:192.168.42.27-->dst:192.168.42.11 mark:1 fib_loopup[err0 res.type:2]
[Tue Mar  1 12:47:32 2022] dev:eth10 idev:eth10  src:192.168.42.27-->dst:192.168.42.11 mark:1 111111__fib_validate_source lookup restype:1  match:0
[Tue Mar  1 12:47:32 2022] IPv4: dev_name:eth10  src:192.168.42.30-->dst:192.168.42.11 mark:1 fib_loopup[err0 res.type:2]
[Tue Mar  1 12:47:32 2022] dev:eth10 idev:eth10  src:192.168.42.30-->dst:192.168.42.11 mark:1 111111__fib_validate_source lookup restype:1  match:0
[Tue Mar  1 12:47:32 2022] dev:eth10 idev:eth10  src:192.168.42.30-->dst:192.168.42.11 mark:1 2222222__fib_validate_source lookup restype:1  match:0
[Tue Mar  1 12:47:32 2022] IPv4: dev_name:eth10  src:192.168.42.30-->dst:192.168.42.11 mark:1 fib_validate_source[err:-18 ]
[Tue Mar  1 12:47:32 2022] IPv4: dev_name:eth0  src:10.67.8.226-->dst:10.67.11.138 mark:0 fib_loopup[err0 res.type:1]
[Tue Mar  1 12:47:32 2022] IPv4: dev_name:eth0  src:10.67.10.179-->dst:10.67.10.178 mark:0 fib_loopup[err0 res.type:1]
[Tue Mar  1 12:47:32 2022] IPv4: dev_name:eth0  src:10.67.10.3-->dst:10.67.8.214 mark:0 fib_loopup[err0 res.type:1]
[Tue Mar  1 12:47:32 2022] dev:eth10 idev:eth10  src:192.168.42.27-->dst:192.168.42.11 mark:1 2222222__fib_validate_source lookup restype:1  match:0
[Tue Mar  1 12:47:32 2022] IPv4: dev_name:eth10  src:192.168.42.27-->dst:192.168.42.11 mark:1 fib_validate_source[err:-18 ]
[Tue Mar  1 12:47:32 2022] net_ratelimit: 2 callbacks suppressed
[Tue Mar  1 12:47:32 2022] IPv4: martian source 192.168.42.11 from 192.168.42.27, on dev eth10


当 rp_filter=1  src_valid_mark=1


 IPv4: dev_name:eth10  src:192.168.42.20-->dst:192.168.42.11 mark:1 fib_loopup[err0 res.type:2]
[Tue Mar  1 12:57:55 2022] dev:eth10 idev:eth10  src:192.168.42.20-->dst:192.168.42.11 mark:1 111111__fib_validate_source lookup restype:2  match:0
[Tue Mar  1 12:57:55 2022] IPv4: dev_name:eth10  src:192.168.42.20-->dst:192.168.42.11 mark:1 fib_validate_source[err:-22 ]
[Tue Mar  1 12:57:55 2022] IPv4: martian source 192.168.42.11 from 192.168.42.20, on dev eth10
[Tue Mar  1 12:57:55 2022] ll header: 00000000: 02 25 57 75 0e d4 02 25 57 75 0e cd 08 00        .%Wu...%Wu....

 

ip route get 192168.42.20 执行时， 也是调用fib_lookup 函数去查找 其结果也就是上述的结果

所以会看到反向查找的时候 入口和出口不一样 如果开启rp_filter 就会导致drop

 类似的可以去看 arp_filter  arp_ignore arp_accept 等选项

ps： 处理arp 请求报文的时候也会涉及到路由查找以及ip 校验

if (arp->ar_op == htons(ARPOP_REQUEST) &&
        ip_route_input_noref(skb, tip, sip, 0, dev) == 0) {