Search the Community

Inventus Inventus is a spider designed to find subdomains of a specific domain by crawling it and any subdomains it discovers. It's a Scrapy spider, meaning it's easily modified and extendable to your needs. Demo https://asciinema.org/a/PGIeEpEwZTUdgxrolBpCjljHL# Requirements Linux -- I haven't tested this on Windows. Python 2.7 or Python 3.3+ Scrapy 1.4.0 or above. Installation Inventus requires Scrapy to be installed before it can be run. Firstly, clone the repo and enter it. $ git clone https://github.com/nmalcolm/Inventus $ cd Inventus Now install the required dependencies using pip. $ pip install -r requirements.txt Assuming the installation succeeded, Inventus should be ready to use. Usage The most basic usage of Inventus is as follows: $ cd Inventus $ scrapy crawl inventus -a domain=facebook.com This tells Scrapy which spider to use ("inventus" in this case), and passes the domain to the spider. Any subdomains found will be sent to STDOUT. The other custom parameter is subdomain_limit. This sets a max limit of subdomains to discover before quitting. The default value is 10000, but isn't a hard limit. $ scrapy crawl inventus -a domain=facebook.com -a subdomain_limit=100 Exporting Exporting data can be done in multiple ways. The easiest way is redirecting STDOUT to a file. $ scrapy crawl inventus -a domain=facebook.com > facebook.txt Scrapy has a built-in feature which allows you to export items into various formats, including CSV, JSON, and XML. Currently only subdomains will be exported, however this may change in the future. $ scrapy crawl inventus -a domain=facebook.com -t csv -o Facebook.csv Configuration Configurations can be made to how Inventus behaves. By default Inventus will ignore robots.txt, has a 30 second timeout, caches crawl data for 24 hours, has a crawl depth of 5, and uses Scrapy's AutoThrottle extension. These and more can all be changed by editing the inventus_spider/settings.py file. Scrapy's settings are well documented too. Bugs/Suggestions/Feedback Feel free to open a new issue for any of the above. Inventus was built in only a few hours and will likely contain bugs. You can also connect with me on Twitter. License Released under the MIT License. See LICENSE. Download: Inventus-master.zip or git clone https://github.com/nmalcolm/Inventus.git Source

The Angler Exploit Kit continues to evolve at an alarming rate, seamlessly adding not only zero-day exploits as they become available, but also a host of evasion techniques that have elevated it to the ranks of the more formidable hacker toolkits available. Researchers at Cisco’s Talos intelligence team today reported on a technique used in a recent Angler campaign in which attackers are using stolen domain registrant credentials to create massive lists of subdomains that are used in rapid-fire fashion to either redirect victims to attack sites, or serve as hosts for malicious payloads. The technique has been called domain shadowing, and it is considered the next evolution of fast flux; so far it has enabled attackers to have thousands of subdomains at their disposal. In this case, the attackers are taking advantage of the fact that domain owners rarely monitor their domain registration credentials, which are being stolen in phishing attacks.They’re then able to create a seemingly endless supply of subdomains to be used in additional compromises. “It’s one thing that people just don’t do,” said Craig Williams, security outreach manager for Cisco Talos. “No one logs back into their registrant account unless they are going to change something, or renew it.” Researchers Nick Biasani and Joel Esler wrote that Cisco has found hundreds of compromised accounts—most of them GoDaddy accounts—and control up to 10,000 unique domains. “This behavior has shown to be an effective way to avoid typical detection techniques like blacklisting of sites or IP addresses,” Biasini and Esler said. “Additionally, these subdomains are being rotated quickly minimizing the time the exploits are active, further hindering analysis. This is all done with the users already registered domains. No additional domain registration was found.” Cisco said the campaign began in earnest in December, though some early samples date back to September 2011; more than 75 percent of subdomain activity, however, has occurred since December. There are multiple tiers to the attack, with different subdomains being created for different stages. The attacks start with a malicious ad redirecting users to the first tier of subdomains which send the user to a page serving an Adobe Flash or Microsoft Silverlight exploit. The final page is rotated heavily and sometimes, those pages are live only for a few minutes, Cisco said. “The same IP is utilized across multiple subdomains for a single domain and multiple domains from a single domain account,” Biasini and Esler wrote. “There are also multiple accounts with subdomains pointed to the same IP. The addresses are being rotated periodically with new addresses being used regularly. Currently more than 75 unique IPs have been seen utilizing malicious subdomains.” Domain shadowing may soon supercede fast flux, a technique that allow hackers to stay one step ahead of detection and blocking technology. Unlike fast flux, which is the rapid rotation of a large list of IP addresses to which a single domain or DNS entry points, domain shadowing rotates in new subdomains and points those at a single domain or small group of IP addresses. “When you think about it, this is likely the next evolution of fast flux. It allows attackers an easy way to come up with domains they can use in a short amount of time and move on,” Williams said. “It doesn’t cost them anything and it’s tough to detect because it’s difficult to use blocklisting technology to defend against it. It’s not something we’ve observed before.” The attackers have zeroed in almost exclusively on GoDaddy accounts since the registrar is by far the biggest on the Internet; for now, that is the only commonality to the attacks carried out in this Angler campaign, Cisco said. “The accounts are largely random so there is no way to track which domains will be used next. Additionally, the subdomains are very high volume, short lived, and random, with no discernible patterns,” Biasini and Esler wrote. “This makes blocking increasingly difficult. Finally, it has also hindered research. It has become progressively more difficult to get active samples from an exploit kit landing page that is active for less than an hour. This helps increase the attack window for threat actors since researchers have to increase the level of effort to gather and analyze the samples.” Williams, meanwhile, warns that as security technologies catch up to domain shadowing, there is a risk that mitigations could impact legitimate traffic. “If the block list is made incorrectly, it could block both bad and legitimate traffic and harm an innocent victim,” Williams said. “If you know an attacker has credentials, you could make the case to block everything associated with a domain. That could also block the legitimate domain.” Source