Net Doctors
Gold Membership
Nuke Wiz
Posts:127
 |
| 4/18/2007 8:25 AM |
|
I just figured out they put in Google Site map creation in DNN 4.5 which is great to see. It appears that it makes a map via the menu system only. My question is this... sites that have news articles you normally don't have links to articles in the menu system. So all your articles are left out of the site map. Should the Google Map creation go beyond what is in the main menu and crawl the whole site? Or do you think Google can take it from your main Articles page and crawl the remaining pages?
If you don't know what I am talking about if you have a DNN 4.5 site go to www.url.com/sitemap.aspx
It will create a map based on the role you are logged in as. |
|
Net Doctors, Inc.
www.netmds.com |
|
|
DNN Professor
Gold Membership
Nuke Master
Posts:752
|
| 4/18/2007 9:28 AM |
|
Hi Netmds,
I just converted 5 sites, yesterday, to DNN 4.5.1
Starting to take a look under the hood.
Is there no configuration option on the built-in sitemap?
I have been using both the sitemap from donein.net
http://www.donein.net/Default.aspx?tabid=55
and the sitemap from dnnmasters.com
I like them both.
The module from DNNMasters allows you to also index DNN blogs as an option. |
|
Buck Anderson
DNNprofessor.com
dnnprofessor.com
JustNorth Outdoors
justnorth.com
Jive Media Group LLC
jivemg.com |
|
|
y01nk
Registered Users
Nuke Master
Posts:201
|
|
Net Doctors
Gold Membership
Nuke Wiz
Posts:127
|
| 4/18/2007 9:54 AM |
|
I don't see any other config options or documentation.
To be effective should the sitemap go below the menu links? |
|
Net Doctors, Inc.
www.netmds.com |
|
|
Dax Davis
Gold Membership
Nuke Master
Posts:344
|
| 4/18/2007 1:44 PM |
|
There are no other options. I think it would be great if the sitemap would do all pages on the site and not just navigation. I'm sure google will do it on its own, but not as well as if it was in the sitemap. I think, though, that the core should allow modules to insert their pages into the sitemap. Not make DNN figure it out. That would be a lot of trouble. It would be cool if there was some sort of ISitemap interface.
Dax |
|
|
|
|
Tom Kraak
Gold Membership
Nuke Wiz
Posts:135
|
| 4/18/2007 3:13 PM |
|
Yup, it's just building a an xml sitemap based on the tabs table excluding admin and host pages. And there are no config options (I confirmed in the benefactor forum earlier today.) So for a News Articles based blog like mine where I want my posts indexed it's totally useless.
y01nk, so I guess there is not much to review here 
Buck, don't confuse Google sitemaps with html sitemaps for your visitors.
|
|
Tom Kraak
Seablick Consulting LLC
DNN SEO Quickstart Guide |
|
|
y01nk
Registered Users
Nuke Master
Posts:201
|
| 4/18/2007 3:31 PM |
|
What the... ?!?
I guess, like so many other things DNN, this core offering has to be relegated to the "also ran" bucket.
"what eveerrrrrr" |
|
 |
|
|
DNN Professor
Gold Membership
Nuke Master
Posts:752
|
| 4/18/2007 3:31 PM |
|
Posted By tkraak on 4/18/2007 3:13 PM
Yup, it's just building a an xml sitemap based on the tabs table excluding admin and host pages. And there are no config options (I confirmed in the benefactor forum earlier today.) So for a News Articles based blog like mine where I want my posts indexed it's totally useless.
y01nk, so I guess there is not much to review here
Buck, don't confuse Google sitemaps with html sitemaps for your visitors.
I have been using Google sitemaps for over a year on 20 sites. What part do you think I am confused on? The Donein.net and the DNNMasters sitemap modules have been reliable and have placed our sites on page one for numerous keywords.
The Donein product is a straight Google sitemap generator and the DNNMasters product is a combination of a page presentation sitemap and a configurable Google sitemap generator.
Your up |
|
Buck Anderson
DNNprofessor.com
dnnprofessor.com
JustNorth Outdoors
justnorth.com
Jive Media Group LLC
jivemg.com |
|
|
Tom Kraak
Gold Membership
Nuke Wiz
Posts:135
|
| 4/18/2007 3:34 PM |
|
Sorry Buck, my fault ... I did not realize that the DNNMasters module also includes a configurable Google sitemap generator.
|
|
Tom Kraak
Seablick Consulting LLC
DNN SEO Quickstart Guide |
|
|
y01nk
Registered Users
Nuke Master
Posts:201
|
| 4/18/2007 3:40 PM |
|
I was hoping to learn a bit more about the whole sitemap business- sort of in general...
It seems to me that there are really two purposes for the sitemap:
1) for humans to learn about the content and organization of the site
and
2) for "bots" to crawl efficiently (or something)
In either case, it seems that the new core piece falls short of what has already been offered as a 3rd party tool.
Is this about right? |
|
|
|
|
DNN Professor
Gold Membership
Nuke Master
Posts:752
|
| 4/18/2007 3:48 PM |
|
Posted By tkraak on 4/18/2007 3:34 PM
Sorry Buck, my fault ... I did not realize that the DNNMasters module also includes a configurable Google sitemap generator.
No Worries |
|
Buck Anderson
DNNprofessor.com
dnnprofessor.com
JustNorth Outdoors
justnorth.com
Jive Media Group LLC
jivemg.com |
|
|
DNN Professor
Gold Membership
Nuke Master
Posts:752
|
| 4/18/2007 4:08 PM |
|
Posted By y01nk on 4/18/2007 3:40 PM
I was hoping to learn a bit more about the whole sitemap business- sort of in general...
It seems to me that there are really two purposes for the sitemap:
1) for humans to learn about the content and organization of the site
and
2) for "bots" to crawl efficiently (or something)
In either case, it seems that the new core piece falls short of what has already been offered as a 3rd party tool.
Is this about right?
Y01nk,
You are correct.
When Google sends a crawler to your site, it begins by looking at your homepage. If all your down links are presented in navigation from your homepage, it will crawl those as well. All you can do is hope that Google finds all your pages. With the SolPart menu being javascript based, it is even questionable that the Google crawler can find your down links.
Google recommends that you create a sitemap and submit it to them at
https://www.google.com/webmasters/tools/siteoverview?hl=en
This gives Google an exact list of the pages that they should crawl on your site.
Google Webmaster tools also gives you a look at relevant information about your site through Goggle's eyes. Check out the link above.
You don't want all your pages being indexed by Google. Maybe you have some page creation in progress that shouldn't be shown yet, or maybe you have a section of your site that only content editors should see. That is why most good sitmap generators will only show the pages that you have set for all users to see.
Yahoo, Google, and MSN have all reached an agreement on a common shared Sitemap format. This means you can also go to Yahoo Site Explorer
https://siteexplorer.search.yahoo.com/mysites
and submit the same Sitemap for their crawlers to use.
If you don't want to purchase one of the sitemaps I had mentioned in a previous post, at least use the one offered in DNN. It is better than not submitting any. You will be amazed, if you are utilizing good keywords and search engine optimization, how many more pages will be indexed by the search engines when submitting a sitemap.
Here is a look at the user interface on the DNNMasters module:
http://www.jivemg.com/AboutUs/SiteMap.aspx
This module is a combination of a user view navigational sitmap that will be updated whenever your pages change. In the module settings, it has a Google sitemap generator as well. If you are using the DNN blog module, it will also index those blogs on the Google sitemap. This is an option you can turn on or off.
Here are some articles:
http://www.jivemg.com/AboutUs/SampleArticles.aspx
Buck |
|
Buck Anderson
DNNprofessor.com
dnnprofessor.com
JustNorth Outdoors
justnorth.com
Jive Media Group LLC
jivemg.com |
|
|
Tom Kraak
Gold Membership
Nuke Wiz
Posts:135
|
| 4/18/2007 4:26 PM |
|
One comment though. Google and other engines will still index pages (once found) even if they are not included in your sitemap.xml. To explicitly exclude pages or directories use robots.txt.
I use xml-sitemaps.com as opposed to a module. Works great. |
|
Tom Kraak
Seablick Consulting LLC
DNN SEO Quickstart Guide |
|
|
DNN Professor
Gold Membership
Nuke Master
Posts:752
|
| 4/18/2007 4:39 PM |
|
Posted By tkraak on 4/18/2007 4:26 PM
One comment though. Google and other engines will still index pages (once found) even if they are not included in your sitemap.xml. To explicitly exclude pages or directories use robots.txt.
I use xml-sitemaps.com as opposed to a module. Works great.
Tom,
I agree with you on robots.txt.
We use them and one humble word of advice to those that do. If you look at the robots.txt of dotnetnuke.com (www.dotnetnuke.com/robots.txt), you will notice that they disallow the desktopmodules folder. Seemed like a good idea until you realize that all many modules call into images and files from this folder structure.
Found this one out the hard way. When I went to Google webmaster tools and saw all the URLs that Google had blocked via the robots.txt. There are a lot of modules that call into their folders under the desktopmodules folder. In NewsArticles module, for instance, the print calls all coming from /DesktopModules/DnnForge%20-%20NewsArticles/.
If anybody wants to learn more about robots.txt, here is a link:
robotstxt.org
|
|
Buck Anderson
DNNprofessor.com
dnnprofessor.com
JustNorth Outdoors
justnorth.com
Jive Media Group LLC
jivemg.com |
|
|
ROBAX
Gold Membership
Nuke Master
Posts:882
|
| 4/18/2007 4:49 PM |
|
The value of a Google sitemap to the end-user (the person using Google Search to find information) dimishes when it simply lists every single page on a site and rates them equally. That makes it no different from conventional indexing.
Site operators are supposed to use Google sitemaps to create value hierarchy of the content in their site... to state which pages are relevant and which are less so, and help everyone (and Google) spend less time indexing and sifting through dead and pointless web pages. None of the DNN modules do it right... they all basically tell Google to come along every day and look at every page on the menu... plus some also tell it to look for a few generated pages, such as Blog articles.
Although personally I feel that the above is more than enough for most sites (how many DNN privacy statements does Google really need to index?), I also think site owners should know what it's doing.
If you really want your site fully indexed - including your Ventrian News Articles, kill the sitemap.
Compare "http://yoursite.com/sitemap.aspx"
with a Google search of "site:yoursite.com"
Rob
|
|
|
|
|
Tom Kraak
Gold Membership
Nuke Wiz
Posts:135
|
| 4/18/2007 5:24 PM |
|
| I think sitemaps are still a valuable tool especially to get new sites indexed quickly. But I do agree with some of your comments Rob. That's why I "massage" my sitemaps by hand before submitting them to the engines. |
|
Tom Kraak
Seablick Consulting LLC
DNN SEO Quickstart Guide |
|
|
y01nk
Registered Users
Nuke Master
Posts:201
|
| 4/18/2007 5:35 PM |
|
Great stuff, thanks for the expert opinions.
I hope netmds will forgive the minor thread hi-jack! |
|
|
|
|
ROBAX
Gold Membership
Nuke Master
Posts:882
|
| 4/18/2007 5:54 PM |
|
yes definitely, and I always use a sitemap myself too. They have to be created manually by the person who is publishing the information in the first place. It's the automatic creation of sitemaps that goes against their intended purpose.
What DNN needs to add is some sort of interface to the generated map that lets one edit the variables and remove or add pages and modules. Then it'd actually provide additional value for the person using Google to find our information.
For DNN I've been using the BTB Sitemap module: http://www.bitethebullet.co.uk/GoogleSiteMapDNNModule/tabid/59/Default.aspx (all BTB's stuff is nice btw). It indexes the core Blog. I'll switch to the core sitemap soon as it has developed a little further.
For non-DNN sites I've been using an online sitemap tool. Google's page on it here:
https://www.google.com/webmasters/tools/docs/en/about.html
has links to many such tools listed at the bottom of this page here:
http://code.google.com/sm_thirdparty.html
It's hard to recommend an automatic sitemap creator for DNN at this point when they are all likely to remove more content than add value.
On a side note, I'm also betting that Google will get wise to it pretty quick and possibly ignore maps that are created like this... they defeat the value of the whole scheme.
Rob
..and now it's time to go read Tom's weekly DNN blog - there's always something good in there.
|
|
|
|
|
Net Doctors
Gold Membership
Nuke Wiz
Posts:127
|
| 4/18/2007 9:47 PM |
|
I am glad this sparked this discussion... #1 for others to learn about what sitemaps do and #2 to help get ideas out there to maybe improved on the DNN Core sitemap functionaility.
Since there is not a project for this I am not sure who fedback should be directed too?? |
|
Net Doctors, Inc.
www.netmds.com |
|
|
y01nk
Registered Users
Nuke Master
Posts:201
|
| 4/18/2007 10:15 PM |
|
Posted By tkraak on 4/18/2007 4:26 PM
I use xml-sitemaps.com as opposed to a module. Works great.
That is WAY COOL |
|
|
|
|