My Website & Google Webmaster Tools

[HorseOptions]

I've read previous posts on this subject and I'm not sure if my situation quite fits in the same square, so I thought I'd ask again in my own words. Sorry to be repetative......

I've created a google webmaster tools account. I've verified my site, added a sitemap successfully, the home page has been crawled successfully and the index status is good, google analytics is good, but google still says that it's having trouble crawling the following page - http://www.horseoptions.com/userlogin.php - why?

I'm also not too keen on the 80+ (out of 87 total) useless keywords GoogleBot has drummed up. The keywords we've entered into the admin area of 68Classifieds - do we need to have those keywords in the text somewhere on the home page or do we embed them in code somewhere? If so, where? And doesn't the sitemap provide enough info for Google to crawl my site accurately?

Just curious.

Thanks for your time

[Lhotch +]

Originally Posted by HorseOptions

but google still says that it's having trouble crawling the following page - http://www.horseoptions.com/userlogin.php - why?

have you tried going to the link you posted? I get a 404 page not found error. The file you see when you login ins login.php

[HorseOptions]

Originally Posted by Lhotch

have you tried going to the link you posted? I get a 404 page not found error. The file you see when you login ins login.php

Larry,

Exactly. Do you know why? Any thoughts on the keywords?

Rob

[HotAir +]

Hi Rob,
You have a link somewhere pointing to that non-existant page. Googlebot is following the links throughout your site and is being sent to that page even though it isn't really there.

When you are loggingged in to your Google Webmaster Tools account, go to Dashboard -> Diagnostics -> Web crawl. Click the "Not found" link. On that page you'll see a column titled "Linked from". Click the entry under that column heading to find out where the errant link is and fix it if it is on your site. It's possible the link is from an outside site you can't control (I have a bunch of these) - in which case you might want to create a useful 404 error page that helps people find what they are looking for on your site.

Regards,
Jim

[HorseOptions]

I clicked on each of the 4 links and could still get to the login page. So not sure what to do there.

Any thoughts on my question about keywords? How to get my site to showup with the keywords and increase it's rank in google?

[nashford +]

I see it crawls but produces errors like this?

1089 Invalid URL
This is not a valid URL. Please correct it and resubmit. URL: http://listing/27/.htmlParent tag: url
Tag: loc
Problem detected on: Sep 16, 2009

Also what should be in the robots.txt file on my site?

sitemap: 404 - Not Found

What would cause this???

[seymourjames +]

There is no page called userlogin.php . Search your site files to where this link exists and delete it. Google will take care of the rest over time. You can also exclude it from being indexed using a robots.txt file. In fact you should have this file to stop google indexing several pointless pages where there is zero or useless content .

[tidy +]

Hi HorseOptions,

A great tool for identifying where broken links on your site are is Xenu's Link Sleuth.
You can find out more and download it at Find broken links on your site with Xenu's Link Sleuth (TM)

As James has said, once you know where the broken links are, you can fix them and then Google will stop trying to crawl them. Also check that you don't have any broken links in your Google sitemap file (sitemap.xml) if you are using one.

[nashford +]

This is what confuses me....

The sitemap generates this: (i replace my domain name in this posting before someone says that cant be accessed)

- <url>
<loc>http://www.mysite.com/listing/27/APPLE_IPOD_30_GB.html</loc>
<lastmod>2009-09-17</lastmod>
<changefreq>monthly</changefreq>
<priority>0.2</priority>
</url>

and google gives me this error and accordingly wont submit the sitemap. The issue is not with categories, only with listings. The listings are the issues.

1089 Invalid URL
This is not a valid URL. Please correct it and resubmit. URL: http://listing/27/.htmlParent tag: url
Tag: loc
Problem detected on: Sep 17, 2009

How did google truncuate this or otherwise. Is the module at fault or is google and either way how do i fix this?!? I need my listings to appear obviously.

[seymourjames +]

Don't know. Our site map module does not do this. Interesting if you find out.