Simple Python Scraper

Using this code

from BeautifulSoup import BeautifulSoup
import urllib2
import re

html_page = urllib2.urlopen("http://some_website.com")
soup = BeautifulSoup(html_page)

links = []

for link in soup.findAll('a', attrs={
	'href': re.compile("^http://"),
	'class': 'mestergull'
	}):
	links.append(link.get('href'))

for link in links:
	print link

Still, you need to install the BeautifulSoup using

pip install BeautifulSoup

Clickjacking vulnerability check

Having the following code written in Python

import optparse
import requests
import re

parser = optparse.OptionParser()
parser.add_option('-t', '--target', action="store", dest="hostname", help="Host where you want to check for common files.", default="spam")
parser.add_option('-p', '--port', action="store", dest="port", help="Port number to be used while hitting the host", default="80")
options, args = parser.parse_args()

hostregex = re.compile("^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$")
ipregex = re.compile("^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$")

host = options.hostname
port = options.port

if hostregex.match(host):
	print "Checking clickjacking on %s:%s" % (host,port)
	if (port == 443):
		req = requests.get("https://" + host)
	else:
		req = requests.get("http://" + host + ":" + port)

	try:
		print "[-] Not vulnerable to ClickJ\nX-Frame-Options response header present, Contains value %s\n" % (req.headers['X-Frame-Options'])
	except:
		print "[+] Vulnerable to ClickJacking, but check framebusting.\n"

elif ipregex.match(host):
	print "Checking clickjacking on %s:%s" % (host,port)

	if (port == 443):
		req = requests.get("https://" + host)
	else:
		req = requests.get("http://" + host + ":" + port)

	try:
		print "[-] Not vulnerable to ClickJ\nX-Frame-Options response header present, Contains value %s\n" % (req.headers['X-Frame-Options'])
	except:
		print "[+] Vulnerable to ClickJacking, but check framebusting.\n"

else:
	print "Please enter valid Hostname / IP Address"

I am testing it using python h1.py -t google.com

Result is nice:

Checking clickjacking on google.com:80
[-] Not vulnerable to ClickJ
X-Frame-Options response header present, Contains value SAMEORIGIN

Using the same command, I’m testing kinnell core

python h1.py -t internal.kinnell.co.uk

And there’s a surprise:

Checking clickjacking on internal.kinnell.co.uk:80
[+] Vulnerable to ClickJacking, but check framebusting.

This needs to be fixed!

In nginx/conf folder add following parameter in nginx.conf under server section:

add_header X-Frame-Options "SAMEORIGIN";

And then restart nginx server.

Check for software updates on OS X

Use this command line

softwareupdate --list

To install a particular one use

softwareupdate --install the_particular_one

Otherwise you can use

softwareupdate --install --all

Unzip password protected archive

1. Create a file on CLI

touch hide.txt

2. Now zip it with a password using

zip -e cool.zip hide.txt

3. Let’s create the python decrypting script…

import time
import zipfile
import itertools

start_time = time.time()

zFile = zipfile.ZipFile("cool.zip")

chrs = 'abcdefghijklmnopqrstuvwxyz'
s = ''

for x in range(3, 5):
	for xs in itertools.product(chrs, repeat=x):
		word = s.join(xs)

		try:
			zFile.extractall(pwd=word)			

		except Exception, e:
			#print e
			break

		else:
			print word
			break

print("--- %s seconds ---" % (time.time() - start_time))

In the code above I assume the password contains 3 to 5 lowercase letters, so this can be extended.
Also, I’d like to see elapsed time.

4. Oh, and you should execute it like

python wow.py

… assuming you saved it as ‘wow.py’

NFS is reporting that your exports file is invalid

Just follow this procedure and you’ll get out of the woods

sudo rm /etc/exports
sudo touch /etc/exports

vagrant halt
vagrant up --provision