Download Indexed Cache

Download Indexed Cache

• 1.
  OWASP “Google Hacking”Project Download Indexed Cache Christian Heinrich christian.heinrich@owasp.org OWASP “Google Hacking” Project Lead Last Updated 2 July 2009 Copyright © The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the OWASP License. The OWASP Foundation http://www.owasp.org
• 2.
  Copyright Notice Slides andNotes Licensed as:  AU Creative Commons 2.5  Attribution-Non Commercial-No Derivative Works OWASP “Google Hacking” Project 2
• 3.
  Updates to Slides Incorporatesall previous slides from:  OWASP USA Conference 2008  ToorCon X (USA)  SecTor 2K8 (Canada)  RUXCON 2K8 (Australia)  OWASP Australian Conference 2009  OWASP European Conference 2009  5th CONFidence 2009 (Poland)  OWASP London Chapter Meeting May 2009  SyScan’09 Singapore Lasted Updated 2 July 2009 OWASP “Google Hacking” Project 3
• 4.
  Latest (SFW) Slides Publishedon http://www.slideshare.net/cmlh OWASP “Google Hacking” Project 4
• 5.
  Published as SeparatePPT Presentations Recommended Delivery: 1. OWASP “Google Hacking” Project 1.1 “Search Engine Recon/Discovery” 1.2. “Download Indexed Cache” 2. “TCP Input Text” 3. OWASP “Google Hacking” Project 3.1 “Spiders/Robots/Crawlers” 3.2 “Continuous Improvement” OWASP “Google Hacking” Project 5
• 6.
  Slide References andFurther Info Refer to the Notes Page of each Slide Some slides are hidden due to time limit OWASP “Google Hacking” Project 6
• 7.
  Christian Heinrich aka“cmlh” Experience Since 1996: Penetration Tester Web Application Security Reverse Engineer Crypto Analyst Governance (i.e. PCI, ISO, etc) OWASP “Google Hacking” Project 7
• 8.
  Christian Heinrich aka“cmlh” .gov.au Procurement Panels: Federal Attorney General’s CNVA Program NSW Government 2319/2020 OWASP “Google Hacking” Project 8
• 9.
  Christian Heinrich aka“cmlh”  Wireless Network  https://twitter.com/ruxcon OWASP “Google Hacking” Project 9
• 10.
  Christian Heinrich aka“cmlh” Presented at: OWASP Conferences Australia, Europe and USA. ToorCon (San Diego, USA) SecTor (Toronto, Canada) CONFidence (Poland, Europe) SyScan (Singapore) RUXCON (Sydney, Australia) OWASP “Google Hacking” Project 10
• 11.
  Christian Heinrich aka“cmlh” “End User” Experience Since 1996: Security Thought Leader within AU Media: Former CSO of FOXTEL Former CSO of News Limited (AU part of News Corp) OWASP “Google Hacking” Project 11
• 12.
  Christian Heinrich aka“cmlh” “End User” Experience Since 1996: Federal .gov.au DSD Certified Gateway Service Provider  ASIO Web Hosting Government Endorsed Business (GEB) State .nsw.gov.au Critical Infrastructure OWASP “Google Hacking” Project 12
• 13.
  Christian Heinrich aka“cmlh” Contributions to OWASP: OWASP Testing Guide v3  4.2.1 “Spiders/Robots/Crawlers”  4.2.2 “Search Engine Reconnaissance” OWASP “Google Hacking” Project  “Download Indexed Cache” PoC Presentations at OWASP Conferences:  Australia, Europe and USA OWASP “Google Hacking” Project 13
• 14.
  OWASP “Google Hacking”Project Download Indexed Cache Christian Heinrich christian.heinrich@owasp.org OWASP “Google Hacking” Project Lead Last Updated 2 July 2009 Copyright © The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the OWASP License. The OWASP Foundation http://www.owasp.org
• 15.
  Download Indexed Cache SupportsOWASP Testing Guide v3 4.2.2 “Search Engine Reconnaissance” Provides Evidence of Cached Page during Fieldwork Repository at: http://code.google.com/p/dic OWASP “Google Hacking” Project 15
• 16.
  Command Line Arguments GoogleSOAP Search API related:  -key API Key demo is embedded API Key  -query Google Search Query  -start Starting Google Search Result (Zero Based Index i.e. 1=0) OWASP “Google Hacking” Project 16
• 17.
  Results 1 to10 cmlh$ /usr/bin/perl dic.pl –key “demo” -query “site:owasp.org" -start 1 "Download Indexed Cache" Proof of Concept (PoC) 0.1 (Released at RUXCON 2K8) Copyright 2009 Christian Heinrich Licensed under the Apache License, Version 2.0 Creating ./siteowasp.org 1. Downloading https://www.owasp.org/ from Google Cache [46k] as 1.html 2. Downloading http://www.owasp.org/ from Google Cache [46k] as 2.html [SNIP] 8. Downloading http://www.owasp.org/index.php/Session_Management from Google Cache [88k] as 8.html 9. Downloading http://www.owasp.org/index.php/Testing_for_file_extensions handling from Google Cache [24k] as 9.html 10. Downloading http://www.owasp.org/index.php/OWASP_SoC_2008_ASDR_Reviewers from Google Cache [20k] as 10.html OWASP “Google Hacking” Project 17
• 18.
  Results 11 to… cmlh$ /usr/bin/perl dic.pl –key demo -query “site:owasp.org" -start 11 "Download Indexed Cache" Proof of Concept (PoC) 0.1 [SNIP] Copyright 2008 Christian Heinrich Licensed under the Apache License, Version 2.0 Appending ./siteowasp.org 11. Downloading https://www.owasp.org/index.php/System_Information_Leak from Google Cache [26k] as 11.html 12. Downloading http://www.owasp.org/index.php/Buffer_overflows from Google Cache [34k] as 12.html [SNIP] 18. Downloading http://www.owasp.org/index.php/Testing_Guide_Introduction from Google Cache [111k] as 18.html 19. Downloading http://www.owasp.org/index.php/OWASP_Java_Project from Google Cache [28k] as 19.html 20. Downloading https://www.owasp.org/index.php/Insecure_Temporary_File from Google Cache [26k] as 20.html OWASP “Google Hacking” Project 18
• 19.
  Google Search Results- 1 to 1000 #!/usr/bin/perl –w for (my $result=0; $result < 990; $result = $result + 10) { system (“./dic.pl -key “[key]" -query “[query]" -start $resultn"); } OWASP “Google Hacking” Project 19
• 20.
  Exploiting Page Rank PageRank Orders “Less Public” Results Last Descending $start of doGoogleSearch:  e.g. –start:990, -start:980, etc  Remember $start – 1 i.e. 0 OWASP “Google Hacking” Project 20
• 21.
  Google Search Results- 1000 to 1 #!/usr/bin/perl –w for (my $result=990; $result >= 1; $result = $result - 10) { system (“./dic.pl -key “[key]" -query “[query]" -start $resultn"); } OWASP “Google Hacking” Project 21
• 22.
  Generated Output cmlh$ /usr/bin/perldic.pl –key “demo” -query “site:owasp.org" -start 1 "Download Indexed Cache" Proof of Concept (PoC) 0.1 (Released at RUXCON 2K8) Copyright 2009 Christian Heinrich Licensed under the Apache License, Version 2.0 Creating ./siteowasp.org 1. Downloading https://www.owasp.org/ from Google Cache [46k] as 1.html 2. Downloading http://www.owasp.org/ from Google Cache [46k] as 2.html [SNIP] 8. Downloading http://www.owasp.org/index.php/Session_Management from Google Cache [88k] as 8.html 9. Downloading http://www.owasp.org/index.php/Testing_for_file_extensions handling from Google Cache [24k] as 9.html 10. Downloading http://www.owasp.org/index.php/OWASP_SoC_2008_ASDR_Reviewers from Google Cache [20k] as 10.html OWASP “Google Hacking” Project 22
• 23.
  Generated Output Directory: Name Strippedof “:” from Google Operator /dic sub-directory Files in Directory: x.html x is Search Result Number [SearchQuery].csv SearchResultNumber, URL OWASP “Google Hacking” Project 23
• 24.
  1.html Example cmlh$ cdsiteowasp.org/dic/ cmlh$ head –n 25 1.html <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><base href="https://www.owasp.org/index.php /Main_Page"><div style="margin:-1px - 1px 0;padding: 0;border:1px solid #999;background:#fff"><div style= "margin:12px;p adding:8px;border:1px solid #999;background:#ddd;font:13px arial,sans- serif;color:#000;font-weight:normal;text-align:left">This is Google&#39;s cache of <a href="https://www.owasp.org/" style="text decoration:underline;color:#00c">https:// www.owasp.org/</a>. It is a snapshot of the page as it appeared on 17 Feb 2009 17:00:03 [snip] OWASP “Google Hacking” Project 24
• 25.
  [SearchQuery].csv Example cmlh$ catsiteowasp.org.csv 1,http://www.owasp.org/ 2,http://www.owasp.org/download/ 3,http://www.owasp.org:443/ 4,https://www.owasp.org/images/b/b1/OWASP_gr_newsle [snip] 5,http://www.owasp.org/images/0/06/Dublin_Sponsorsh [snip] 6,https://www.owasp.org/images/2/21/OWASP_gr_newsle [snip] 7,http://www.owasp.org/index.php/Cincinnati 8,http://www.owasp.org/index.php/Testing_for_file_e [snip] 9,http://www.owasp.org/index.php/OWASP_SoC_2008_ASD [snip] 10,http://www.owasp.org/index.php/OWASP_Taiwan_Tran [snip] OWASP “Google Hacking” Project 25
• 26.
  DataDumper.txt Example $VAR1 =bless( { 'searchTime' => '0.136083‘ 'endIndex' => '10', 'searchComments' => '', 'documentFiltering' => 0, 'searchTips' => '', 'estimatedTotalResultsCount' => '41100', 'searchQuery' => 'site:owasp.org', 'startIndex' => '1', 'resultElements' => [ bless( { [SNIP] OWASP “Google Hacking” Project 26
• 27.
  Google SOAP SearchAPI in Perl doGoogleSearch  $key  $q  $start -1 subtracted for Zero Index doGoogleSearchResponse  URL  cachedSize OWASP “Google Hacking” Project 27
• 28.
  Google SOAP SearchAPI in Perl doGetCachedPage  $key  $URL doGetCachedPageResponse  … xsi:type="ns2:base64"> OWASP “Google Hacking” Project 28
• 29.
  Google SOAP SearchAPI Limitations Search Query limited to: 10 Words 2048 Bytes 1K Search Queries Per Day Limited to Search Results within 0…999 10K Possible Results from 10 Different Queries OWASP “Google Hacking” Project 29
• 30.
  “10K Possible Resultsfrom 10 Different Queries” Specific each FQDN over 10 site: -queries For example:  … -query “site:www.google.com” …  … -query “site:video.google.com” … 3. … 9. [snip]  … -query “code.google.com” … OWASP “Google Hacking” Project 30
• 31.
  Google SOAP SearchAPI Limitations Issuing of API Keys Discontinued 5 Dec 2006 OWASP “Google Hacking” Project 31
• 32.
  Google SOAP SearchAPI Limitations Will be Deprecated on 31 August 2009 OWASP “Google Hacking” Project 32
• 33.
  dic Roadmap PoC v0.1 Previewedat OWASP USA, ToorCon and SecTor (CA) Released at RUXCON 2K8 in Sydney, AU, Nov 2008 PoC v0.2 Moving repository to code.google.com/p/dic Records the Timestamp from Google Cache Previewed at OWASP AU/EU 2009, SyScan09SG OWASP “Google Hacking” Project 33
• 34.
  dic Roadmap PoC v0.3 SpecifyRange of Google Search Results to 1000 Code Sync with “TCP Input Text” Consider Net::Google CPAN Perl Module PoC v0.4 Maintenance Release Released approx 31 August 2009 Once Google deprecates SOAP Search API OWASP “Google Hacking” Project 34
• 35.
  Call for ProjectReviewers Perl – CPAN Modules SOAP::Lite Net::Google Interested? christian.heinrich@owasp.org OWASP “Google Hacking” Project 35
• 36.
  Call for ProjectReviewers Perl – Quality Assurance: Perl::Critic CPAN Module perltidy Code Contribution Licensed as: Apache License, Version 2.0 Interested? christian.heinrich@owasp.org OWASP “Google Hacking” Project 36
• 37.
  Call for ProjectReviewers Development Eclipse EPIC Plug-in Subclipse Plug-in Subversion Repository code.google.com Interested? christian.heinrich@owasp.org OWASP “Google Hacking” Project 37
• 38.
  Call for ProjectReviewers OWASP Alpha Project Reviewers: pdp @ GNUCITIZEN Chris Gates @ Carnal0wnage Glenn Roberts @ Solutionary Interested? christian.heinrich@owasp.org OWASP “Google Hacking” Project 38
• 39.
  OWASP Project Project Endorsers Justin Derry (OWASP AU Conference Chair)  Dinis Cruz (OWASP Board) OWASP Project Manager  Paulo Coimbra OWASP “Google Hacking” Project 39
• 40.
  Project Controversy - OWASP “Google Hacking” Role: 2. Someone in an Engineering Function at Google 3. Complaint Received by Tom Brennan (OWASP) Facts:  Not an Google or OWASP Summer of Code  Does not violate Google’s Terms of Service  Contacted for Sec. Role at Google Sydney AU  Google SOAP API perl code related to tit  Separation with OWASP Project due to new scope OWASP “Google Hacking” Project 40
• 41.
  Project Controversy code.google.com denies“Google Hacking” labels But permits project names of “Google Hacking” http://code.google.com/p/googlehacking OWASP “Google Hacking” Project 41
• 42.
  Closing Remarks Mitigation strategiesare in the following slides: “Spiders/Robots/Crawlers” “Continuous Improvement” OWASP “Google Hacking” Project 42
• 43.
  Closing Remarks Upcoming Presentations: http://snipurl.com/cmlh_speaking_schedule E-mail: christian.heinrich@owasp.org Slidesavailable from: http://www.slideshare.net/cmlh OWASP “Google Hacking” Project 43