信息收集框架——recon-ng
背景：在渗透测试前期做攻击面发现（信息收集）时候往往需要用到很多工具，最后再将搜集到的信息汇总到一块。
现在有这样一个现成的框架，里面集成了许多信息收集模块、信息存储数据库、以及报告生成模块，为工程化信息收集提供了可能。
它就是recon-ng。recon-ng使用python编写，其使用方式和metasploit十分相似
使用方法介绍：
1、新建工作区（建议一个渗透目标一个工作区，这样能确保搜集到的信息都是针对一个目标的）
命令：Recon-ng -w 工作区名字
例：
recon-ng -w cctv
# 通过上面的命令创建‘cctv’工作区后可以通过如下命令查看工作区情况
[recon-ng][cctv] > show workspaces +------------+ | Workspaces | +------------+ | cctv | | default | +------------+
2、设置搜索引擎api
Keys list ===>查看现有搜索引擎api
keys add shodan fdkasjkfljklasjkldffjalks ===>设置shodan搜索api
[recon-ng][cctv] > keys list
+--------------------------+
| Name | Value |
+--------------------------+
| bing_api | |
| builtwith_api | |
| censysio_id | |
| censysio_secret | |
| flickr_api | |
| fullcontact_api | |
| github_api | |
| google_api | |
| hashes_api | |
| ipinfodb_api | |
| ipstack_api | |
| jigsaw_api | |
| jigsaw_password | |
| jigsaw_username | |
| pwnedlist_api | |
| pwnedlist_iv | |
| pwnedlist_secret | |
| shodan_api | |
| twitter_api | |
| twitter_secret | |
| virustotal_api | |
+--------------------------+
[recon-ng][cctv] > keys add shodan_api fdkasjkfljklasjkldffjalks
3、show options(查看全局设置)
[recon-ng][cctv] > show options Name Current Value Required Description ---------- ------------- -------- ----------- NAMESERVER 8.8.8.8 yes nameserver for DNS interrogation PROXY no proxy server (address:port) THREADS 10 yes number of threads (where applicable) TIMEOUT 10 yes socket timeout (seconds) USER-AGENT Recon-ng/v4 yes user-agent string VERBOSITY 1 yes verbosity level (0 = minimal, 1 = verbose, 2 = debug)
建议设置代理，让可以访问google(不得不佩服google的搜索能力)
set PROXY 127.0.0.1:1087
4、查询包含哪些可用模块
通过use加tab键可以查看有哪些可用模块
[recon-ng][cctv] > use discovery/info_disclosure/cache_snoop recon/domains-companies/pen recon/domains-hosts/threatcrowd recon/netblocks-hosts/shodan_net discovery/info_disclosure/interesting_files recon/domains-contacts/metacrawler recon/domains-hosts/threatminer recon/netblocks-hosts/virustotal exploitation/injection/command_injector recon/domains-contacts/pen recon/domains-vulnerabilities/ghdb recon/netblocks-ports/census_2012 exploitation/injection/xpath_bruter recon/domains-contacts/pgp_search recon/domains-vulnerabilities/punkspider recon/netblocks-ports/censysio import/csv_file recon/domains-contacts/whois_pocs recon/domains-vulnerabilities/xssed recon/ports-hosts/migrate_ports import/list recon/domains-credentials/pwnedlist/account_creds recon/domains-vulnerabilities/xssposed recon/profiles-contacts/dev_diver recon/companies-contacts/bing_linkedin_cache recon/domains-credentials/pwnedlist/api_usage recon/hosts-domains/migrate_hosts recon/profiles-contacts/github_users recon/companies-contacts/jigsaw/point_usage recon/domains-credentials/pwnedlist/domain_creds recon/hosts-hosts/bing_ip recon/profiles-profiles/namechk recon/companies-contacts/jigsaw/purchase_contact recon/domains-credentials/pwnedlist/domain_ispwned recon/hosts-hosts/ipinfodb recon/profiles-profiles/profiler recon/companies-contacts/jigsaw/search_contacts recon/domains-credentials/pwnedlist/leak_lookup recon/hosts-hosts/ipstack recon/profiles-profiles/twitter_mentioned recon/companies-contacts/pen recon/domains-credentials/pwnedlist/leaks_dump recon/hosts-hosts/resolve recon/profiles-profiles/twitter_mentions recon/companies-domains/pen recon/domains-domains/brute_suffix recon/hosts-hosts/reverse_resolve recon/profiles-repositories/github_repos recon/companies-multi/github_miner recon/domains-hosts/bing_domain_api recon/hosts-hosts/ssltools recon/repositories-profiles/github_commits recon/companies-multi/whois_miner recon/domains-hosts/bing_domain_web recon/hosts-hosts/virustotal recon/repositories-vulnerabilities/gists_search recon/contacts-contacts/mailtester recon/domains-hosts/brute_hosts recon/hosts-locations/migrate_hosts recon/repositories-vulnerabilities/github_dorks recon/contacts-contacts/mangle recon/domains-hosts/builtwith recon/hosts-ports/shodan_ip reporting/csv recon/contacts-contacts/unmangle recon/domains-hosts/certificate_transparency recon/locations-locations/geocode reporting/html recon/contacts-credentials/hibp_breach recon/domains-hosts/findsubdomains recon/locations-locations/reverse_geocode reporting/json recon/contacts-credentials/hibp_paste recon/domains-hosts/google_site_web recon/locations-pushpins/flickr reporting/list recon/contacts-domains/migrate_contacts recon/domains-hosts/hackertarget recon/locations-pushpins/shodan reporting/proxifier recon/contacts-profiles/fullcontact recon/domains-hosts/mx_spf_ip recon/locations-pushpins/twitter reporting/pushpin recon/credentials-credentials/adobe recon/domains-hosts/netcraft recon/locations-pushpins/youtube reporting/xlsx recon/credentials-credentials/bozocrack recon/domains-hosts/shodan_hostname recon/netblocks-companies/whois_orgs reporting/xml recon/credentials-credentials/hashes_org recon/domains-hosts/ssl_san recon/netblocks-hosts/reverse_resolve
也可以通过search命令来查找相关模块
[recon-ng][cctv] > search google
[*] Searching for 'google'...
Recon
-----
recon/domains-hosts/google_site_web
此时大家可能会有疑问，这么多模块我怎么知道哪个模块是干什么使的呢？ 这个时候我们可以use相应模块后用show info看到关于该模块的详细解释
[recon-ng][cctv] > use recon/domains-hosts/google_site_web
[recon-ng][cctv][google_site_web] > show info
Name: Google Hostname Enumerator
Path: modules/recon/domains-hosts/google_site_web.py
Author: Tim Tomes (@LaNMaSteR53)
Description:
Harvests hosts from Google.com by using the 'site' search operator. Updates the 'hosts' table with
the results.
Options:
Name Current Value Required Description
------ ------------- -------- -----------
SOURCE cctv.com yes source of input (see 'show info' for details)
Source Options:
default SELECT DISTINCT domain FROM domains WHERE domain IS NOT NULL
<string> string representing a single input
<path> path to a file containing a list of inputs
query <sql> database query returning one column of inputs
此外recon-ng会将收集到的信息自动存入数据库，后面咱们可以将这些数据掏出来进行二次查询。可以通过下面这个命令查看数据库有哪些表：
[recon-ng][cctv] > show schema +---------------+ | domains | +---------------+ | domain | TEXT | | module | TEXT | +---------------+ +--------------------+ | companies | +--------------------+ | company | TEXT | | description | TEXT | | module | TEXT | +--------------------+ +-----------------+ | netblocks | +-----------------+ | netblock | TEXT | | module | TEXT | +-----------------+ +-----------------------+ | locations | +-----------------------+ | latitude | TEXT | | longitude | TEXT | | street_address | TEXT | | module | TEXT | +-----------------------+ +---------------------+ | vulnerabilities | +---------------------+ | host | TEXT | | reference | TEXT | | example | TEXT | | publish_date | TEXT | | category | TEXT | | status | TEXT | | module | TEXT | +---------------------+ +-------------------+ | ports | +-------------------+ | ip_address | TEXT | | host | TEXT | | port | TEXT | | protocol | TEXT | | module | TEXT | +-------------------+ +-------------------+ | hosts | +-------------------+ | host | TEXT | | ip_address | TEXT | | region | TEXT | | country | TEXT | | latitude | TEXT | | longitude | TEXT | | module | TEXT | +-------------------+ +--------------------+ | contacts | +--------------------+ | first_name | TEXT | | middle_name | TEXT | | last_name | TEXT | | email | TEXT | | title | TEXT | | region | TEXT | | country | TEXT | | module | TEXT | +--------------------+ +-----------------+ | credentials | +-----------------+ | username | TEXT | | password | TEXT | | hash | TEXT | | type | TEXT | | leak | TEXT | | module | TEXT | +-----------------+ +-----------------------------+ | leaks | +-----------------------------+ | leak_id | TEXT | | description | TEXT | | source_refs | TEXT | | leak_type | TEXT | | title | TEXT | | import_date | TEXT | | leak_date | TEXT | | attackers | TEXT | | num_entries | TEXT | | score | TEXT | | num_domains_affected | TEXT | | attack_method | TEXT | | target_industries | TEXT | | password_hash | TEXT | | password_type | TEXT | | targets | TEXT | | media_refs | TEXT | | module | TEXT | +-----------------------------+ +---------------------+ | pushpins | +---------------------+ | source | TEXT | | screen_name | TEXT | | profile_name | TEXT | | profile_url | TEXT | | media_url | TEXT | | thumb_url | TEXT | | message | TEXT | | latitude | TEXT | | longitude | TEXT | | time | TEXT | | module | TEXT | +---------------------+ +-----------------+ | profiles | +-----------------+ | username | TEXT | | resource | TEXT | | url | TEXT | | category | TEXT | | notes | TEXT | | module | TEXT | +-----------------+ +--------------------+ | repositories | +--------------------+ | name | TEXT | | owner | TEXT | | description | TEXT | | resource | TEXT | | category | TEXT | | url | TEXT | | module | TEXT | +--------------------+
5、使用方法举例（拿搜索子域名与对应ip的场景来举例）
使用google搜索来查询目标有哪些子域名
[recon-ng][cctv] > use recon/domains-hosts/google_site_web [recon-ng][cctv][google_site_web] > show options # 查看需要填哪些数据 Name Current Value Required Description ------ ------------- -------- ----------- SOURCE default yes source of input (see 'show info' for details) [recon-ng][cctv][google_site_web] > set SOURCE cctv.com # 设置目标域名 SOURCE => cctv.com [recon-ng][cctv][google_site_web] > run #开始运行
也可以使用暴力猜解的方式来获取目标子域名：
[recon-ng][cctv] > use recon/domains-hosts/brute_hosts [recon-ng][cctv][brute_hosts] > show options Name Current Value Required Description -------- ------------- -------- ----------- SOURCE default yes source of input (see 'show info' for details) WORDLIST /usr/local/Cellar/recon-ng/4.9.6/libexec/data/hostnames.txt yes path to hostname wordlist # 字典路径 [recon-ng][cctv][brute_hosts] > set SOURCE cctv.com # 设置目标域名 SOURCE => cctv.com [recon-ng][cctv][brute_hosts] > run #开始运行
运行完毕后查询到的数据将自动存入数据库，我们可以通过'show hosts'或'query+sql语句'的方式来查询，例：
[recon-ng][cctv] > show hosts +-----------------------------------------------------------------------------------------------------------+ | rowid | host | ip_address | region | country | latitude | longitude | module | +-----------------------------------------------------------------------------------------------------------+ | 1 | tv.cctv.com | | | | | | google_site_web | | 2 | www.cctv.com | | | | | | google_site_web | | 3 | news.cctv.com | | | | | | google_site_web | +-----------------------------------------------------------------------------------------------------------+
[recon-ng][cctv] >query select * from hosts;
+-----------------------------------------------------------------------------------------------------------+ | rowid | host | ip_address | region | country | latitude | longitude | module | +-----------------------------------------------------------------------------------------------------------+ | 1 | tv.cctv.com | | | | | | google_site_web | | 2 | www.cctv.com | | | | | | google_site_web | | 3 | news.cctv.com | | | | | | google_site_web | +-----------------------------------------------------------------------------------------------------------+
# 为了保证隐私删掉了大部分数据，只给3个做为举例
数据库里已经有目标的子域名信息，现在想基于数据库里信息做进一步查询可以吗？ 当然可以，我们以查询域名对应的ip为例：
[recon-ng][cctv] > use recon/hosts-hosts/resolve [recon-ng][cctv][resolve] > show options Name Current Value Required Description ------ ------------- -------- ----------- SOURCE default yes source of input (see 'show info' for details) # 正常来说SOURCE后应该是跟一个域名信息，比如'www.cctv.com' [recon-ng][cctv][resolve] > set SOURCE query select host from hosts # 这里厉害了哦！我们要查的是一个表的内容，如果一个域名设置一次那还不累死了？ recon-ng竟然支持将值设为一个sql语句！ 这样就可以批量查询表内的数据了！ SOURCE => query select host from hosts [recon-ng][cctv][resolve] > run
执行完成后我们可以看下现在数据库里的内容有什么变化：
[recon-ng][cctv][resolve] > show hosts +----------------------------------------------------------------------------------------------------------------+ | rowid | host | ip_address | region | country | latitude | longitude | module | +----------------------------------------------------------------------------------------------------------------+ | 1 | tv.cctv.com | 123.125.195.125 | | | | | google_site_web | | 2 | www.cctv.com | 114.112.172.231 | | | | | google_site_web | | 3 | news.cctv.com | 111.206.186.245 | | | | | google_site_web | | 4 | tv.cctv.com | 123.125.195.125 | | | | | resolve | | 5 | www.cctv.com | 114.112.172.231 | | | | | resolve | | 6 | news.cctv.com | 111.206.186.245 | | | | | resolve | +----------------------------------------------------------------------------------------------------------------+
# 可以看到已经把查询到的ip地址填入表内了
就拿我们现在查询到的数据来举例说明一下该怎么导出报表
[recon-ng][cctv] > search report # 查下看有哪些报表相关模块
[*] Searching for 'report'...
Reporting
---------
reporting/csv
reporting/html
reporting/json
reporting/list
reporting/proxifier
reporting/pushpin
reporting/xlsx
reporting/xml
[recon-ng][cctv] > use reporting/html # 导出成html文件
[recon-ng][cctv][html] > show options
Name Current Value Required Description
-------- ------------- -------- -----------
CREATOR yes creator name for the report footer
CUSTOMER yes customer name for the report header
FILENAME /Users/liwei/.recon-ng/workspaces/cctv/results.html yes path and filename for report output # 报表导出路径
SANITIZE True yes mask sensitive data in the report
[recon-ng][cctv][html] > set CREATOR liwei # 填写报告作者
CREATOR => liwei
[recon-ng][cctv][html] > set CUSTOMER cctv # 填写用户单位名称
CUSTOMER => cctv
[recon-ng][cctv][html] > run
[*] Report generated at '/Users/liwei/.recon-ng/workspaces/cctv/results.html'. # 导出成功
[recon-ng][cctv][html] >
最终报表长这样：
注：以下是引自网友对各个模块的简要说明：
cache_snoop – DNS缓存录制 interesting_files – 敏感文件探测 command_injector – 远程命令注入shell接口 xpath_bruter – Xpath注入爆破 csv_file – 高级csv文件导入 list – List文件导入 point_usage – Jigsaw – 统计信息提取用法 purchase_contact – Jigsaw – 简单的联系查询 search_contacts – Jigsaw联系枚举 jigsaw_auth – Jigsaw认证联系枚举 linkedin_auth – LinkedIn认证联系枚举 github_miner – Github资源挖掘 whois_miner – Whois数据挖掘 bing_linkedin – Bing Linkedin信息采集 email_validator – SalesMaple邮箱验证 mailtester – MailTester邮箱验证 mangle – 联系分离 unmangle –联系反分离 hibp_breach –Breach搜索 hibp_paste – Paste搜索 pwnedlist – PwnedList验证 migrate_contacts – 域名数据迁移联系 facebook_directory – Facebook目录爬行 fullcontact – FullContact联系枚举 adobe – Adobe Hash破解 bozocrack – PyBozoCrack Hash 查询 hashes_org – Hashes.org Hash查询 leakdb – leakdb Hash查询 metacrawler – 元数据提取 pgp_search – PGP Key Owner查询 salesmaple – SalesMaple联系获取 whois_pocs – Whois POC获取 account_creds – PwnedList – 账户认证信息获取 api_usage – PwnedList – API使用信息 domain_creds – PwnedList – Pwned域名认证获取 domain_ispwned – PwnedList – Pwned域名统计获取 leak_lookup – PwnedList – 泄露信息查询 leaks_dump – PwnedList –泄露信息获取 brute_suffix – DNS公共后缀爆破 baidu_site – Baidu主机名枚举 bing_domain_api – Bing API主机名枚举 bing_domain_web – Bing主机名枚举 brute_hosts – DNS主机名爆破 builtwith – BuiltWith枚举 google_site_api – Google CSE主机名枚举 google_site_web – Google主机名枚举 netcraft – Netcraft主机名枚举 shodan_hostname – Shodan主机名枚举 ssl_san – SSL SAN查询 vpnhunter – VPNHunter查询 yahoo_domain – Yahoo主机名枚举 zone_transfer – DNS域文件收集 ghdb – Google Hacking数据库 punkspider – PunkSPIDER漏洞探测 xssed – XSSed域名查询 xssposed – XSSposed域名查询 migrate_hosts – 域名数据迁移host bing_ip – Bing API旁站查询 freegeoip –FreeGeoIP ip定位查询 ip_neighbor – My-IP-Neighbors.com查询 ipinfodb – IPInfoDB GeoIP查询 resolve – 主机名解析器 reverse_resolve – 反解析 ssltools – SSLTools.com主机名查询 geocode – 地理位置编码 reverse_geocode – 反地理位置编码 flickr – Flickr地理位置查询 instagram – Instagram地理位置查询 picasa – Picasa地理位置查询 shodan – Shodan地理位置查询 twitter – Twitter地理位置查询 whois_orgs – Whois公司信息收集 reverse_resolve – 反解析 shodan_net – Shodan网络枚举 census_2012 – Internet Census 2012 查询 sonar_cio – Project Sonar查询 migrate_ports – 主机端口数据迁移 dev_diver – Dev Diver Repository检查 linkedin – Linkedin联系获取 linkedin_crawl – Linkedin信息抓取 namechk – NameChk.com用户名验证 profiler – OSINT HUMINT信息收集 twitter – Twitter操作 github_repos – Github代码枚举 gists_search – Github Gist搜索 github_dorks – Github Dork分析 csv – CSV文件生成 html – HTML报告生成 json – JSON报告生成 list – List生成 pushpin – PushPin报告生成 xlsx – XLSX文件创建 xml – XML报告生成
