Posted by & filed under Tools & Methodology.

I’m currently experimenting with Python 3, and made myself a simple HTTP/HTTPS python banner grabber. The code is included below, I hope someone finds it useful.

The code can take input URLs from file or rom the command line, and defaults to just printing the HTTP ‘Server’ header.

Update: Thanks to Jeff who pointed out an obvious flaw in my first version of this script. I’ve thrown in some error handling when servers returns unexpected headers as well. See below for an updated version of the script.


#!/usr/bin/env python3.1
'''
Copyright (c) 2011 Carsten Maartmann-Moe

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

Created on Jan 22, 2011

@author: Carsten Maartmann-Moe <[email protected]>
'''
import getopt
import sys
import urllib.request
import locale

ALL_HEADERS = False
VERBOSE = False

def main(argv):
    encoding = locale.getpreferredencoding()
    input_list = None

    try:
        opts, args = getopt.getopt(argv, 'hi:va', ['help', 'input-list=',
                                                   'verbose', 'all-headers'])
    except getopt.GetoptError as err:
        print(err)
        usage()
        sys.exit(2)
    for opt, arg in opts:
        if opt in ('-h', '--help'):
            usage()
            sys.exit()
        elif opt in ('-i', '--input-list'):
            input_list = arg
        elif opt in ('-v', '--verbose'):
            global VERBOSE
            VERBOSE = True
        elif opt in ('-a', '--all-headers'):
            global ALL_HEADERS
            ALL_HEADERS = True
        else:
            assert False, 'Unhandled option: ' + opt

    if len(args) < 1: # Print usage if no arguments are given
        usage()

    urls = args # The remainder of arguments are treated like URLs

    if input_list: # If the user supplied a input list, add those URLs
        read_file(input_list, encoding, urls)

    headers = {} # Dictionary for storing headers with URLs as keys
    for url in urls:
        headers[url] = grab_headers(url)

    print_results(headers)

def read_file(filename, preferred_encoding, urls):
    ''' Reads a file and appends the lines to the 'urls' parameter '''
    line_num = 0
    with open(filename, encoding=preferred_encoding) as file:
        for line in file:
            line_num += 1
            urls.append(line.rstrip())

def grab_headers(url):
    ''' Grabs the headers of a from the response of a given URL '''
    response = None
    try:
        response = urllib.request.urlopen(url)
    except Exception as e:
        print("Could not complete HTTP request: " + str(e))
        sys.exit(1)
    return response.info()

def print_results(headers):
    for url in headers:
        print('[+] ' + url + ':')
        if ALL_HEADERS:
            print(headers[url])
        else:
            try:
                server_header = dict(headers[url])['Server']
                if server_header:
                    print(server_header)
            except KeyError as e:
                print('[!] No \'Server\' header received, the response ' +
                      'contained the following headers:')
                print(headers[url])

def usage():
    print('''Usage: ./pybgrab.py [OPTIONS] URL

Supply an URL to grab the web server's 'Server' HTTP Header.

    -a, --all-headers:    Returns all HTTP Headers instead of just the
                          'Server' header
    -h, --help:           Displays this message
    -i FILE, --input-list=FILE:
                          Specify a list of URLs in an input file
    -v/--verbose:         Verbose mode''')

if __name__ == '__main__':
    main(sys.argv[1:])

One Response to “A HTTP banner grabber in Python 3”

  1. Miranda

    I read a lot of interesting posts here. Probably you spend a
    lot of time writing, i know how to save you a lot of work, there
    is an online tool that creates readable, google friendly posts
    in minutes, just type in google – laranitas free content source

    Reply

Trackbacks/Pingbacks

  1.  Norwegian newspaper VG sports rick rolls and riddles in HTTP headers | Break & Enter
  2.  Python script to grab HTTP responses | -= Cujef.com =-

Leave a Reply

  • (will not be published)