Amass, go deep in the sea with free APIs
There are several types of Subdomains scanning tools. Amass, Subfinder, findomain, etc… In my opinion, the tool at its peak is Amass, and many Bugbounty hunters have automated systems through Amass. Today I’m going to talk about ways to expand Amass’ datasources and get more results.
Subdomains 스캐닝 도구에는 여러가지가 있습니다. 대표적으로 Amass, Subfinder, findomain, etc… 개인적인 생각으론 정점에 있는 도구는 Amass 이며, 많은 버그바운티 헌터들이 Amass를 통한 자동화 시스템을 가지고 있습니다.
오늘은 Amass의 datasources를 확장을 통해 좀 더 많은 결과를 얻어오기 위한 방법에 대해 이야기하려고 합니다.
Amass datasource
All of the tools mentioned above call multiple APIs to get a list of subdomains quickly. Of course, it has its own testing function, but as a result, it’s a necessary process to get a lot of domains in a short time. Amass also has this logic and supports a number of services.
위에서 이야기한 도구들 모두 여러 API를 호출하여 서브도메인 리스트를 빠르게 얻어옵니다. 물론 자체적인 테스팅 기능도 갖추고 있지만, 결과적으로 단시간에 많은 도메인을 확보하기 위해선 꼭 필요한 과정이죠. Amass 또한 이러한 로직을 가지고 있고 여러가지 서비스들을 지원하고 있습니다.
Know yourself. default available datasources
Know yourself
from Socrates
First, we need to know our Amass status. The currently installed Amass version is v3.10.3, and the currently active datasources can be obtained through the amass enum -list command.
먼저 우리의 Amass 상태를 알아야합니다. 현재 설치된 Amass 버전은 v3.10.3이며 amass enum -list 명령을 통해 현재 활성화된 datasources 를 얻어올 수 있습니다.
amass -version
v3.10.3
default available apis
amass enum -list
Data Source | Type | Available
--------------------------------------------------------------------------------
AlienVault api *
Alterations alt *
ArchiveIt archive *
Ask scrape *
Baidu scrape *
BinaryEdge api
Bing scrape *
Brute Forcing brute *
BufferOver api *
BuiltWith scrape *
C99 api
CIRCL api
Censys cert *
CertSpotter cert *
Chaos api
Cloudflare api *
CommonCrawl api *
Crtsh cert *
DNSDB api
DNSDumpster scrape *
FacebookCT cert
GitHub api
GoogleCT cert *
HackerOne scrape *
HackerTarget api *
IPToASN api *
LoCArchive archive *
Mnemonic api *
NetworksDB scrape *
PassiveTotal api
Pastebin api *
RADb api *
RapidDNS scrape *
ReconDev api
Riddler scrape *
Robtex api *
SecurityTrails api
ShadowServer api *
Shodan api
SiteDossier scrape *
Spyse api
Sublist3rAPI api *
TeamCymru api *
ThreatCrowd api *
ThreatMiner api *
Twitter api
UKGovArchive archive *
URLScan api *
Umbrella api
ViewDNS scrape *
VirusTotal api *
Wayback archive *
WhoisXML none
Yahoo scrape *
ZETAlytics api
ZoomEye api
find unabled apis..
이제 활성화되지 않은 api 들만 뽑아봅시다.
amass enum -list | grep -v "\*"
Data Source | Type | Available
--------------------------------------------------------------------------------
BinaryEdge api
C99 api
CIRCL api
Chaos api
DNSDB api
FacebookCT cert
GitHub api
PassiveTotal api
ReconDev api
SecurityTrails api
Shodan api
Spyse api
Twitter api
Umbrella api
WhoisXML none
ZETAlytics api
ZoomEye api
17 datasource is disabled, and let’s go enable it. 17개의 datasource가 비활성화 상태이고, 이를 활성화 시켜봅시다.
Amass -config flag and automatically tries to discover config file
It’s a good idea to know an option before looking at how to apply it. This is the -config flag, and you can define and use Amass settings in advance. You can also specify API Token here, so you can eliminate the hassle of giving an option every time you scan.
적용 방법을 알아보기 전에 옵션을 하나 알고가면 좋습니다. -config flag 이고 미리 Amass의 설정을 정의하고 사용할 수 있는데, 여기에 API Token 또한 지정할 수 있어서, 매번 스캐닝 시 옵션을 주어야하는 번거로움을 없앨 수 있습니다.
amass enum -d target.com -config ~/.conf/your_amass_conf.ini
https://github.com/OWASP/Amass/blob/master/examples/config.ini
The path below is the default config that references without the -config option. I think recommend using config separately for services with api limit.
그리고 아래 경로에 config 생성 시 -config 옵션이 없어도 해당 config를 참조하게 됩니다. 저는 개인적으로 api limit이 있는 서비스를 위해서 config를 분리해서 사용하는걸 추천합니다.
| Operating System | Path |
|---|---|
| Linux / Unix | $XDG_CONFIG_HOME/amass/config.ini or $HOME/.config/amass/config.ini |
| Windows | %AppData%\amass\config.ini |
| OSX | $HOME/Library/Application Support/amass/config.ini |
Price and limit?
Summarize the price and limit of the remaining inactive APIs, it looks like this. 우선 나머지 비활성화 되어있던 APIs들을 가격과 Limit을 정리해보면 이렇습니다.
| Name | Price | Memo |
|---|---|---|
| BinaryEdge | - Free (Free) - $10/Month (Starter) - $500/Month (Business) - X (Enterprise) |
https://www.binaryedge.io/pricing.html |
| C99 | - $5/Month - $25/Year |
http://api.c99.nl/dashboard/shop |
| CIRCL | ||
| Chaos | - Free | https://chaos.projectdiscovery.io/#/ but contributed nuclei-template or public-bugbounty-programs |
| DNSDB | - Free (500q/Month) | https://www.farsightsecurity.com/get-started/ |
| FacebookCT | - Free | https://developers.facebook.com/docs/certificate-transparency-api/ |
| Github | - Free | |
| PassiveTotal | - Free | https://community.riskiq.com/ |
| ReconDev | - Free (30q/Month) - 1$/10q - 20$/Month (200q/Month) |
https://recon.dev/pricing.html |
| SecurityTrails | - Free (50q/Month) - $50/Month (1500q/m) - $500/Month (20000q/m) - $1500/Month (65000q/m) |
https://securitytrails.com/corp/pricing#api |
| Shodan | - 7$ (Event!) - $59/Month (Freelancer) - $299/Month (business) - $899/Month (corp) |
https://developer.shodan.io/pricing |
| Spyse | - Free - $50/Month (Standard) - $250/Month (Pro) |
https://spyse.com/user/subscription |
| - Free | ||
| Umbrella | https://umbrella.cisco.com/ | |
| WhoisXML | - Free (500q/Month) - $15/Month (2000q/Month) ~ - $1800/Month (2000000/m) |
https://whois.whoisxmlapi.com/pricing |
| ZETAlytics | - $199/Month (4000q/m) | https://zetalytics.com/#pricetable |
| ZoomEye | - Free (10000/Month) | https://www.zoomeye.org/business |
All Free? and how to get key
Chaos , FacebookCT, Github, PassiveTotal, Twitter, ZoomEye
Chaos
contribute nuclei-template or public-bugbounty-programs and get key. I am an early Chaos beta user, so I already have a api key
Chaos는 현재 nuclei 템플릿과 pbp에 컨트리뷰트 하는 사용자에게만 키를 발급해주고 있습니다. 이는 어려운게 아니기 때문에 컨트리뷰트하고 키를 받아서 사용하시면 됩니다. 물론 Amass API가 아니여도 Chaos는 정말 좋은 도구에요 :D (아 저는 베타 초기 사용자라 따로 키를 받았습니다.)
- https://github.com/projectdiscovery/nuclei-templates * https://github.com/projectdiscovery/public-bugbounty-programs
[data_sources.Chaos]
ttl = 4320
[data_sources.Chaos.Credentials]
apikey = 89
FacebookCT
Open https://developers.facebook.com and Sign in facebook account(developer) Go apps
First, create app
Create app > Your app page
Get apikey
setting > advance setting > security > client token
Get Secret
setting > basic setting > app secret code
[data_sources.FacebookCT]
ttl = 4320
[data_sources.FacebookCT.app1]
apikey = your_api_key
secret = yuur_secret_key
Github
Open your github setting tokens
Generate Token > if you added token, get token string
[data_sources.GitHub]
ttl = 4320
[data_sources.GitHub.accountname]
apikey = your_api_key
Amass simply use search queries, so we don’t need to add additional permissions.
단순하게 검색 쿼리만 사용하는거라 추가 권한은 안넣어줘도 되는 것 같습니다.
PassiveTotal
https://community.riskiq.com/settings
Profile > User > Secret
[data_sources.PassiveTotal]
ttl = 10080
[data_sources.PassiveTotal.Credentials]
username = your_name
apikey = your_api_key
https://developer.twitter.com/en/apps
Create an app > Choose any type > If confirmed app, get key!
[data_sources.Twitter]
[data_sources.Twitter.account1]
apikey = your_api_key
secret = your_api_secret
ZoomEye
I don’t think I can tell you in detail because most of this datasource is in Chinese. I can’t speak Chinese.and I don’t use ZoomEye datasource either.
Shodan
Shodan sometimes put a big discount. Aim for that moment!
[data_sources.Shodan]
ttl = 10080
[data_sources.Shodan.Credentials]
apikey = your_api_key
Finally, let’s apply it!
Add your config data / 설정 넣어주면 잘 나옵니다. 이후 스캔부턴 해당 datasource를 사용합니다.
vim ~/.config/amass/config.ini
amass enum -list -config ~/.config/amass/config.ini
Other source? and conclusion
In fact, there are many other free datasource. However, API limit is short, so it is not included, but if you make and use various configs depending on the conditions, you can get good scan results depending on the situation.
사실 다른 무료 소스들도 많이 있습니다. 다만 API limit이 짧아서 포함하진 않았는데요, 조건에 따라서 여러가지 config를 만들고 정해서 사용하면 상황에 따라 좋은 스캔 결과를 얻을 수 있습니다.
e.g
for automation scanning
amass enum -d [TARGET] -config ~/.config/amass/auto.ini
for manual scanning
amass enum -d [TARGET] -config ~/.config/amass/all.ini