google sitemap

[Lhotch]

Originally Posted by CB

Larry looks good, urls like: /ads/category.php?type=10 google won't accept, i've been trying and it rejects any url with a session id or dynamic ?type= I installed the SEO module, does this work with the SEO and does it pull the friendly url. Thanks Larry.

There is code in the sitemap to account for the SEO mod, however I dont have a live v3.1.x site using the seo mod so cant be 100% certain that its working as intended. Ill take a little closer look at the v3.1.x SEO and see whats going on.

Ive read that google doesnt generally have a problem with get variables in the url. At one time this was a big hold up for many of the search engines but with dynamic script based sites the norm these days I beleive most wont have an issue.

I do beleive that you have a point about the session ID though. By default session data is written to a cookie but on browsers where people have cookies disabled, or a spider/bot that doesnt accept cookies the SID is appended to the url. Ill do some more checking on this as well.

If your site is not being indexed because of session ID's thats not the scripts fault but how your php is setup. Make sure to disable session.use_trans_sid in your php.ini. If you dont have access to the php.ini try adding the following to .htaccess.

php_flag session.use_trans_sid off

[juven14]

Here is my sitemap with SEO (slightly modified due to not being in the classifieds directory)

http://www.boergoatcentral.com/sitemap.php

[juven14]

I had to add the .html to each url.

Removed the require for classifieds.php since that is now done at init.php. Then I just added a few main directory pages at the bottom.

[CB]

Hi Larry, is there anyway to use the SEF url in the sitemap?

[Lhotch]

Originally Posted by CB

Hi Larry, is there anyway to use the SEF url in the sitemap?

Ive been rather busy and havent had time to look over the code. I do know that there are now 2 versions of the SEO module, one thats compatable with the old version and a new style one, which are you using?

On a side note after taking a closer look at my google sitemap page I only had 3 pages that were indexed by google. Keep in mind I am not using seo on my older 3.0.15 site but I was using stemap.

After doing a little more homework I deemed it was not the get variables in the links but it was that my web host had trans.sessionid enabled so when a cookie could not be written the session ID was added to the URL and this was preventing google from crawling my site so I disabled it.

I just checked again last night and after googles last crawl I now have approx 900 pages off my site indexed, including all my custom RSS feeds.

[CB]

Wow! I'm using the latest of everything 3.1.3 and SEO 3.1. My urls look good on the site but the sitemap generator is generating the raw URL so Google is not accepting most of the links. If you could help me out that would much appreciated. Thanks Larry...cb

[Lhotch]

I still dont think it has anything to do with th raw link. Google accepted all my raw links just fine.

I dont know all the aspects of your sites layout but based on the error you PM'd me, he is what I found on googles site regarding URL not allowed errors. My guess is its related to an issues outlined in this error and not the raw links.

URL not allowed

Google has detected some URLs that appear to be at a higher level or different domain than the Sitemap file location. If you receive this error, try moving your Sitemap file to a higher location in your site and submitting again. For instance, if your Sitemap is listed under http://www.example.com/mysite/sitemap.xml, the following URLs are not valid for that sitemap

* http://www.example.com/ - it's at a higher level than the Sitemap
* http://www.example.com/yoursite/ - it's in a directory parallel to the Sitemap

You should also check that the urls all begin with the same domain as your Sitemap location. For instance, if your sitemap is listed under http://www.example.com/sitemap.xml, the following URLs are not valid for that sitemap:

* http://www.google.com - it's in the google.com domain rather than the example.com domain
* http://example.com/ - it's missing the initial www
* www.example.com/ - it's missing the protocol (http)
* https://www.example.com/ - it's using a different protocol (https rather than http)

Any URLs in the Sitemap that are not denied are processed normally.

[CB]

Well everything else in the sitmap has been indexed in google except the listings with the raw links, session id's or ? characters in the URL. There's another topic at google that covers this but you only get it when you submit a sitemap with the raw links....I just deleted my sitemap from google but i'll post it when I see it again. Basically it says google does not accept dynamic urls with ? session id's ect. Anyway I guess the right question is how to pull the friendly urls.

Looking at http://www.boergoatcentral.com/sitemap.php those URL's are Google friendly. I think he's using the older SEF.

Any ideas how to pull the new URLs. The new URLs are much more discriptive then the old ones so it's worth looking into PLEASE

[Lhotch]

Im working on getting the new SEO to work with sitemap and making progress, but im not sure how soon it will be done. I will post the file when I have it complete.

Where on the google sitemap page are the errors outlined for you? Im not seeing any errors when I look at my sites stats and it isnt even using SEO. To see what I mean check this link and you can see all my pages that have been indexed.

http://www.google.com/search?q=site:...ords.com&hl=en

[Lhotch]

Sitemap.php for V3.1.X posted here....

http://www.68classifieds.com/forums/...=4977#post4977