Tag: beautifulsoup

  • Looking a Scrapy

    As much as I’ve found the basic webscraping to be really simple with urllib and BeautifulSoup. It leaves somethings to be desired. The BeautifulSoup project has languished and recent versions have switched the HTML parser for one that is less able to manage with the poorly encoded pages on real websites. Scrapy is a full […]

  • Scraping the ClickBank Marketplace

    I got an email the other day from Frank Kern who was pimping another make money online product from his cousin Trey. The Number Effect is a DVD containing the results of an experiment where he created an affiliate link to every one of the 12,000 products for sale on ClickBank and sent paid (PPV) […]

  • Scrape Digg Search Results Python Script

    Digg is by far the most popular social news site on the internet. With it’s simple “thumbs up” system the users of the site promote the most interesting and high quality stores and the best of those make it to the front page. What you end up with is a filtered view of the most […]

  • SEOCheck: Track Your Google Position Over Time

    Have you ever wanted to track and assess your SEO efforts by seeing how they change your position in Google’s organic SERP? With this script you can now track and chart your position for any number of search queries and find the position of the site/page you are trying to rank. This will allow you […]

  • Scrape Yahoo Search Results Page

    Ok, even though Yahoo search is on the way out and will be replace by the search engine behind Bing. That transition won’t happen until sometime in 2010. Until then Yahoo still has 20% of the search engine market share and it’s important to consider it as an important source of traffic for your websites. […]

  • Scrape Google Search Results Page

    Here’s a short script that will scrape the first 100 listings in the Google Organic results. You might want to use this to find the position of your sites and track their position for certain target keyword phrases over time. That could be a very good way to determine, for example, if your SEO efforts […]

  • Find Links on Del.icio.us with a Python Script

    In yet another of my series of web scrapers this time I’m posting some code that will scrape links from delicious.com. This is a pretty cool way of finding links that other people have found relevant. And this could be used to generate useful content for visitors. You could easily add this to a WordPress […]

  • Scrape Technorati Search Results in Python

    Today’s script will perform a search on Technorati and then scrape out the search results. It is useful because Technorati is up to date about things that are happening in the blogosphere. And that gives you a way to tune into everything going on there. The scope of the blogosphere matched with Technoratis ability sort […]

  • Getting links to a domain using Alexa and Python

    Sometimes it’s useful to know where all the back-links to a website are coming from. As a competitor it can give you information about how your competition is promoting their site. You can shortcut the process of finding the good places to get links from, and who might be a client or a good contact […]

  • Getting Ezine Article Content Automatically with Python

    If you’re not familiar with Ezine articles they are basically niche content about 200 to 2000 words long that some ‘expert’ writes and shares for re-publishing the content under the stipulation that it includes the signature (and usually a link) for the author. Articles are great from both the advertiser and publisher perspective since the […]