(UTC)
Change Template:
Choose Language:
YaBB Development & Mods › YaBB Development › New Features › Post Reply ( Re: Search Engine Identifiers )
context_title
context_text
Topic Summary - Displaying all post(s). Click here to show the reduced amount |
Posted by: Red Barchetta Posted on: Dec 6th, 2014 at 2:01am |
This is related, and possibly has more applications? GeoLite Legacy Downloadable Databases are updated once a month. How interesting it would be to have a bot or crawler's location automatically identified? One step further, how about a new user's location (country) automatically being entered into the application?
Maybe its just something I am interested in as my forum is geared more toward local users, but I'll just toss this out there to see if anyone else has any thoughts or interest, like in YAMMS. |
Posted by: Red Barchetta Posted on: Nov 22nd, 2014 at 10:15pm |
You can get the SE, Bot or Crawler's name from here:
http://udger.com/resources/ua-list/crawlers-ip This is the source that I have used to expand my list and identify the items in the previous list. |
Posted by: Red Barchetta Posted on: Nov 22nd, 2014 at 10:12pm |
I have also expanded my Search Engine list:
4seohunt|4SeoHuntBot aesop|AESOP_SpiderMan abacho|AbachoBOT acoon|Acoon Robot boson027|AhrefsBot/5.0 ahrefs|AhrefsBot/5.0 ia_archiver|Alexa alexa|Alexa archiver vestris|AlkalineBOT altavista|AltaVista scooter|AltaVista sv.av|AltaVista tarantula|AltaVista alta-vista|Altavista av|Altavista apercite.fr|Apercite aport|Aport girafa|Aranha archiver-web|Archive.org ask|Ask Jeeves askjeeves|Ask Jeeves directhit|Ask Jeeves teoma|Ask Jeeves atomz|Atomz axmo|AxmoRobot baidu|Baidu baiduspider|Baidu net263|Baidu buscaplus|Buscaplus Robi ip3000|C-PBWF-ip3000-crawler canseek|CanSeek christcrawler|ChristCRAWLER clush|Clushbot crawler|Crawler pinpoint|CrawlerBoy powerinter|DIIbot daadle|DaAdLe ROBOT deepindex|DeepIndex ditto|DittoSpyder dotbot|Dotbot Research dotnetdotcom|Dotbot Research earthcom|EARTHCOM travel-finder|ESISmartSpider ezresults|EZResult eurip|EuripBot muscat|EuroFerret arachnoidea|EuroSeek euroseek|EuroSeek Arachnoidea exabot|Exava architext|Excite atext|Excite excite|Excite ArchitextSpider alltheweb|FAST-WebCrawler fastsearch|Fast Crawler yelo.no|Findexa Crawler searchhippo|Fluffy the spider fybersearch|FyberSearch galaxy|GalaxyBot gendoor|GenCrawler genieo|Genieo/1.0 geona|GeonaBot gigabot|Gigablast backrub|Google google|Google googlebot|Googlebot mirago|HenryTheMiragoRobot inktomi|HotBot inktomisearch|Hotbot hubat|Hubater istarthere|I Start here igde|Igde iltrovatore|IlTrovatore-Setaccio incywincy|IncyWincy infoseek|InfoSeek infoseeksidewinder|InfoSeek ultraseek|InfoSeek verno.ueda.info.waseda.ac.jp|Iron33 domanova|Jack joocer|JoocerBot fireball|KIT-Fireball knowledge|Knowledge linkfluence|Kraken lexis-nexis|LNSpiderguy ActiveBookmark|Link Checker, Monitor ALink|Link Checker, Monitor AMeta|Link Checker, Monitor ASPSearch|Link Checker, Monitor BlogBot|Link Checker, Monitor BMChecker|Link Checker, Monitor Bookmark|Link Checker, Monitor Check&Get|Link Checker, Monitor CheckWeb|Link Checker, Monitor CNET_Snoop|Link Checker, Monitor DRKSpider|Link Checker, Monitor DISCo Watchman|Link Checker, Monitor DoctorHTML|Link Checker, Monitor EmailSiphon|Link Checker, Monitor EmailWolf|Link Checker, Monitor FavOrg|Link Checker, Monitor FreshLinks|Link Checker, Monitor HTMLParser|Link Checker, Monitor InternetLinkAgent|Link Checker, Monitor InternetPeriscope|Link Checker, Monitor javElink|Link Checker, Monitor jdwhatsnew|Link Checker, Monitor Lambda|Link Checker, Monitor LinkAlarm|Link Checker, Monitor Linkbot|Link Checker, Monitor Linkman|Link Checker, Monitor LinkProver|Link Checker, Monitor LinkScan|Link Checker, Monitor LinkSweeper|Link Checker, Monitor LinkVerify|Link Checker, Monitor LinkWalker|Link Checker, Monitor MoveAnnouncer|Link Checker, Monitor mylinkcheck|Link Checker, Monitor NetLookout|Link Checker, Monitor NetMechanic|Link Checker, Monitor elsop|Link Checker, Monitor netmechanic|Link Checker, Monitor NetMind-Minder|Link Checker, Monitor marvin.netmind|Link Checker, Monitor gary.netmind|Link Checker, Monitor meg.netmind|Link Checker, Monitor inyanga.netmind|Link Checker, Monitor leo.netmind|Link Checker, Monitor gemini.netmind|Link Checker, Monitor NetMonitor|Link Checker, Monitor Netprospector|Link Checker, Monitor Rational|Link Checker, Monitor Robozilla|Link Checker, Monitor SiteBar|Link Checker, Monitor SpurlBot|Link Checker, Monitor SurfMaster|Link Checker, Monitor SyncIT|Link Checker, Monitor Watchfire|Link Checker, Monitor WatzNew|Link Checker, Monitor WebSite-Watcher|Link Checker, Monitor WebTrends|Link Checker, Monitor Weblink|Link Checker, Monitor Xenu's Link Sleuth|Link Checker, Monitor Z-Add Link Checker|Link Checker, Monitor ActiveBookmark|Link Checker, Monitor ALink|Link Checker, Monitor AMeta|Link Checker, Monitor ASPSearch|Link Checker, Monitor BlogBot|Link Checker, Monitor BMChecker|Link Checker, Monitor Bookmark|Link Checker, Monitor Check&Get|Link Checker, Monitor CheckWeb|Link Checker, Monitor CNET_Snoop|Link Checker, Monitor DRKSpider|Link Checker, Monitor DISCo Watchman|Link Checker, Monitor DoctorHTML|Link Checker, Monitor EmailSiphon|Link Checker, Monitor EmailWolf|Link Checker, Monitor FavOrg|Link Checker, Monitor FreshLinks|Link Checker, Monitor HTMLParser|Link Checker, Monitor InternetLinkAgent|Link Checker, Monitor InternetPeriscope|Link Checker, Monitor javElink|Link Checker, Monitor jdwhatsnew|Link Checker, Monitor Lambda|Link Checker, Monitor LinkAlarm|Link Checker, Monitor Linkbot|Link Checker, Monitor Linkman|Link Checker, Monitor LinkProver|Link Checker, Monitor LinkScan|Link Checker, Monitor LinkSweeper|Link Checker, Monitor LinkVerify|Link Checker, Monitor LinkWalker|Link Checker, Monitor MoveAnnouncer|Link Checker, Monitor mylinkcheck|Link Checker, Monitor NetLookout|Link Checker, Monitor NetMechanic|Link Checker, Monitor elsop|Link Checker, Monitor netmechanic|Link Checker, Monitor NetMind-Minder|Link Checker, Monitor marvin.netmind|Link Checker, Monitor gary.netmind|Link Checker, Monitor meg.netmind|Link Checker, Monitor inyanga.netmind|Link Checker, Monitor leo.netmind|Link Checker, Monitor gemini.netmind|Link Checker, Monitor NetMonitor|Link Checker, Monitor Netprospector|Link Checker, Monitor Rational|Link Checker, Monitor Robozilla|Link Checker, Monitor SiteBar|Link Checker, Monitor SpurlBot|Link Checker, Monitor SurfMaster|Link Checker, Monitor SyncIT|Link Checker, Monitor Watchfire|Link Checker, Monitor WatzNew|Link Checker, Monitor WebSite-Watcher|Link Checker, Monitor WebTrends|Link Checker, Monitor Weblink|Link Checker, Monitor Xenu's|Link Checker, Monitor Z-Add|Link Checker, Monitor LinkLint-checkonly|Link Checker, Monitor/ LinkLint-checkonly|Link Checker, Monitor/ linklint-checkonly|LinkLint.org linklint|LinkLint.org linknz|Linknzbot magma|LookBot fuzine.mt.cs.cmu.edu|Lycos lycos|Lycos_Spider_(T-Rex) majestic12|MJ12bot/v1.3.3 mp3bot|MP3Bot msnbot-media|MSN Search msnbot|MSN Search search.msn|MSN Search looksmart|MantraAgent search.live|Microsoft Live Search mojeek|MojeekBot intags|Mole webtop|MuscatFerret nationaldirectory|NationalDirectory-SuperSpider navadoo|Navadoo Crawler websmostlinked|Nazilla loopimprovements|NetResearchServer northernlight|Northern Light Gulliver objectssearch|ObjectsSearch omgilibot|Omgili szukaj|OnetSzukaj openfind|Openfind piranha,Shark portaljuice|PJspider picsearch|PicSearchBot picosearch|PicoSearch plonebot|Plone Spambot qweery|QweeryBot daum|RaBot supersnooper|Robot@SuperSnooper scoutjet|ScoutJet scrubtheweb|Scrubby search4free|Search 4 Free search-10|Search-10 searchbyusa|SearchByUsa charlotte|SearchMe Visual Search searchme|SearchMe Visual Search searchspider|Searchspider seznam|SeznamBot sightquest|SightQuestBot similarpages|Similar Pages Slurp|Slurp sogou|Sogou entireweb|Speedy Spider sphere|Sphere Scout traficdublu|Spider TraficDublu maxbot|Spider/maxbot spidermonkey|Spider_Monkey rambler|StackRambler surfnomore|Surfnomore Spider mapper.teradex|Teradex_Mapper hoppa|Toutatis tutorgig|Tutorial Crawler cuill|Twiceler twiceler|Twiceler uksearcher|UK Searcher Spider vivante|Vivante Link Checker orange-ftpgroup|Voila Spambot voila|Voila Spambot voilabot|Voila Spambot 80legs|Voltron wasalive|WASALive-Bot wire.co.uk|WIRE WebRefiner worldsearchcenter|WSCbot webalta|WebAlta Crawler webcrawler|WebCrawler webwombat|WebWombat whizbanglabs|WhizBang! Lab wisewire|WiseWire yourbettersearch|YBSbot search engine indexer yahoo-mmcrawler|Yahoo! yahoo|Yahoo! yandex|Yandex yanga|Yanga WorldSearch nhn|Yeti yeti|Yeti yetiBot|Yeti naver|Yeti(NaverRobot) youdao|Youdao youdaobot|Youdao wisenut|ZyBorg abcdatos|abcdatos_botlink ah-ha|ah-ha crawler almaden|almaden crawler antisearch|antibot walhello|appie singingfish|asteris singingfish crawl.baidu|baiduspider france.misesajour|france.misesajour geckobot|geckobot getrax|getRAX petersnews|ip3000 kuloko|kuloko-bot look|lookbot webseek|marvin/infoseek mozdex|mozDex(ComCast) noxtrum|noxtrumbot navi.ocn.ne.jp|nttdirectory_robot speedfind|speedfind ramBot xtreme whatuseek|whatUseek winona|whatUseek yacy.net|yacybot |
Posted by: Red Barchetta Posted on: Nov 22nd, 2014 at 10:10pm |
I have started adding IP blocks to my firewall when ever I find the "The User ID you specified does not exist or you entered a wrong password." error right after the "Slider Captcha: You have failed the safety function Slider Captcha!" error. Its amazing how many "Guests" have disappeared from my forum. Sofar I have blocked these, mostly from Russia and China.
5.15.15.1-5.15.254.254,5.79.153.11,5.248.85.159,27.159.196.64,37.57.231.1-37.57. 231.254,37.59.20.217,46.246.39.31,60.173.11.1-60.173.11.254,62.210.88.107,87.155 .243.58,89.169.123.246,91.207.4.1-91.207.9.254,91.236.75.1-91.236.75.254,95.42.2 9.1-95.42.29.254,95.133.238.145,107.150.52.242,107.183.140.1-107.183.140.254,110 .85.106.15,110.89.1.1-110.89.254.254,162.244.10.189,175.42.40.122,178.33.203.159 ,178.49.154.84,178.158.80.61,180.76.1.1-180.76.254.254,193.201.224.176,195.154.1 81.15,195.211.155.1-195.211.155.254,198.245.51.90,199.192.207.146,213.252.170.1- 213.252.170.254,222.186.21.1-222.186.21.254 |
Posted by: Red Barchetta Posted on: Nov 2nd, 2014 at 5:07pm |
My research has shown me, that most of my visitors are actually search engines.
"AhrefsBot/5.0 requests a new password." huh? |
Posted by: Dandello Posted on: Oct 26th, 2014 at 4:31am |
That looks like a promising addition to the honeypot. Thanks
|
Posted by: Red Barchetta Posted on: Oct 26th, 2014 at 4:23am |
Posted by: Red Barchetta Posted on: Oct 26th, 2014 at 4:23am |
Here is a thought. Would it be possible to have bots automatically added to the "Search Engine Identifiers" list using the bot trap in YABB? This way unknown or unlisted bots can be identified and added to the Search Engine list instead of the Guest List. I got this idea from a previous message about the bot trap here on YABB, and on Elxsy. (Link will be posted later when I have access).
|
YaBB Development & Mods » Powered by YaBB 2.7.00!
YaBB Forum Software © 2000-2024. All Rights Reserved.
HTML 5
Page completed in 0.8639 seconds.