Sunday, June 24, 2007

Web Crawling

    It's Sunday morning, so let me tell my crawler story. Maybe this will help people get a perspective.
    I've had a web site (not blog/blogger) for several years. It's not a high volume site, and I don't seek crawlers. Typically, I have a story to tell or shoot some interesting (to me) photos and I post them and tell friends the URL.
    One page has a story with 700+ pictures of info many people may be interested in and I'm sure the subject is searched daily. It has never been crawled.
    One page has a story with a handful of pictures and has received hundreds of world-wide hits. It has never been crawled.
    Recently, I was asked to help with some problems viewing material on a blog - my first blog experience. While troubleshooting, I was reading the comments on one blog that frequently was seeing 50+ comments per day. I noticed that none of the commenters were inserting links; they were leaving URLs, which sometimes went out of frame and meant typing or copying/pasting. I found this very curious since some people had been leaving comments for a couple of years.
    On investigation, I found that no one knew how to leave links-in-comments. Some were using <b> and there were even some <a> useages for highlighting, but no actual links. So, I went ahead and wrote up some instructions that anyone could follow and put them on my web site. Of course, I left a blog-comment of where to find them and they became a little popular and still gets some hits.
    So, I left one comment with a link to my web site on one or two blogs and wrote a short post with link on my then-new blog.
    Since then, that, AND ONLY that, page has been crawled by Google a couple of times a week. When, I look at the log tommorrow, I bet I find some crawls for the week, and still, only that page will be crawled.
    So, a few people hitting that link in my comment had an immediate and continuing effect


No comments: