Google is dropping my pages big time

[judyo]

I realize sometimes Google does things across the board with updates, but I went from thousands of pages (once my custom IDX was up - I believe I was up to 8,000+ pages that were indexed) and am down to less than ever (200 & something) when I do a "site" search on Google. I checked a competitor that also has a custom IDX and his site still shows over 10,000 pages indexed.

My serps haven't changed and I'll keep checking it out but I'm not sure what's happening. It seems my custom IDX pages are no longer being indexed and I'm not sure why.

I did notice in Google Analytics that some warnings were given for duplicate title tags and meta descriptions which seemed to come from the IDX created pages. Could that be an issue?

[webmaster]

They aren't actually "gone" they have just been moved into the supplemental index from what I can see - do this query in Google for me if you would site:http://www.cook-county-real-estate.net/idx/

Chances are you are correct - probably need to get a bit more creative with your title / meta data and maybe pull some extra info from the feed into your details pages.

[judyo]

Morgan, thanks for answering. In the past it was easy to see what pages were supplemental results since they were labeled that way. Since these pages didn't come up with a site search on the main url is that the only proof that these are now supplemental?

Just to make sure, I cannot change the title tags or descriptions for idx listings, correct? I've seen other custom IDX's where the address is part of the url but we are showing the MLS #. What would it take to show addresses instead?

The Google descriptions seem to be coming from the listing descriptions which are fine. I think the dupe stuff (title & description) was coming from my idx snippet pages, not individual listings.

Will getting some links into these listings help?

[webmaster]

Always - but other items might need to be considered. Can you PM me your GA and Google webmasters tools logins and I will take a look and make some recomendations for you.
thanks

Will getting some links into these listings help?

[RonnieG]

I have also seen this happening on my site and several of my PPC client's REW sites with integrated / custom IDX. Google only shows one IDX details page from my site still in the site: command results. However, I know the IDX pages are still in the Google indexes, possibly supplemental, as Morgan says, because I still get hits and traffic from MLS# and address searches. And when I replicate the same searches that Analytics says they used, I do see my pages on page 1 of the SERPs, often along with IDX details pages from other REW sites.

So, it looks to me that Google may have changed the site: command results sets, but has not (at least not yet) totally deleted alll of my IDX details pages from its indexes.

That said, there are other sites in my market that do not seem to have seen the same drop in site: results. One used to be REW, but no longer, and a couple of others are non-REW sites with integrated IDXs that have on-site IDX details page urls just as REW does, but different page layouts of course.

[OCTeam]

Most of mine seemed to have disappeared or even be "gone" as well.

[RonnieG]

So, Drew, according to GA, do you still get as much traffic from MLS# and address specific searches as you used to? Mine is about the same as before, which was puzzling based on the site: command results, and only makes sense if Google changed what it shows in the site: command results.

[webmaster]

Without getting to technical regarding how Google defines and uses their supplemental index let me try to explain how this works (and why it isn't really the end of the world)

The IDX pages are sufficiently similar in Google's eyes (because they are very similar to each in terms of what is on them) that Google does not feel they are unique enough to make the main index thus they get the boot down to supplemental status.

Now Google only uses supplemental results (generally) as back fill when a particular query does not provide sufficient responses - They "supplement" their results.

The "purpose" of spiderable IDX pages is to capture longtail search traffic for very "specific" queries (which will have very few results because there are very few instances (comparatively) of spiderable IDX - thus for the queries that IDX pages are meant to rank for, even though they are supplement pages, they still should be served and generate the traffic they were meant to (this explains your situation Ronnie)

Does that make sense?

[judyo]

Ronnie, I don't have time to check my stats and I just recently redid my GA code since it got removed during my site upgrade and I didn't realize it.

Again, it seems like REW sites are being affected but not others that have custom idx set-ups.

Check this out: I did a Google search for this (try it yourself):
MLS #: 07126447

I'm getting 3 results. Look at the titles for the 1st two results - good titles with the MLS # and address. The urls go to the actual listing page on those results.

My site is listed last (cook-county-real-estate.net), but does not go to the actual listing description page but to the town page that the listing appears on via my snippets. So that tells me that Google is not bringing up supplemental results for my site, making the search results less relevant on my site which, if people actually clicked on my link, would lead to a higher bounce rate since they'd have to search for that listing on the page. Not good.

Is this because we require registration to see detail pages? I know the 1st site doesn't, but I'm not sure about the 2nd site although it appears it doesn't, either.

I guess I'd rather get the leads than get indexed, but if this is how G treats the registration issue (and I'm not sure that is the problem) then having all of these pages doesn't give me the serps benefit I expected.

[RonnieG]

Morgan, I generally agree, at least for now, that most IDX details pages are, and probably have been for a long time, relegated to what Google used to call the supplemental index. I think the recent change, starting at least since early November, is that Google has also started to drop "near dup" pages from the Site: command results, even though they may still be in the supplemental index. There are still other pages in the site: results that are also supplemental, but are unique enough that Google continues to list them.

Google does still list site: command results somewhat differently, even though the "supplemental" tag is no longer there as it used to be. If you add /* to the end of the site: command, the results appear to be a small fraction of the total site: results, and at least on my site, appear to be those pages that I have really worked on, as far as SEO factors, and that have at least some appreciable PR.

So, I guess what I am saying, based on my own observations and recent experience, is that there now seems to be a 4-tiered system to Google site: command results:
1) site:www.myurl.com/* returns those pages that Google has assigned at least some fractional PR to. From the time Google first changed the site: results to eliminate the "supplemental" tag, I have been tracking those closely, and they appear to be the very same pages Google used to consider "primary" index candidates.
2) Site:www.myurl.com (without /*) returns a larger set of pages that also includes what were previously tagged as supplemental pages, those that have very low, if any, PR, but are still unique enough to show in the list.
3) Recently: Site: results no longer list pages that may be in the supplemental index, but that also appear to Google algos be "near dups", or otherwise have PR too low to even list in any site: results. These pages will sometimes trigger SERPs for their unique qualities, like MLS# or a specific property address. I'll call these "hidden" supplementals. They used to be in the site: results, but no longer.
4) Finally, there are the pages that are so low in PR and/or have not been discovered/crawled/ranked, that are not even in the "hidden" supplemental index and never have been. These are not in site: results and do not trigger SERPs, regardless of how unique and applicable to the page the user's search terms may be.