Growing Pains

We're wrapping up another month of double digit growth (percentage terms) - that's five straight months of solid growth.

Unfortunately, our Web server is feeling the strain now. You may have noticed that during mid day hours, the site gets really sluggish.

We've been scrambling for the last few weeks to keep things moving - blocking over-aggressive bots, slowing down Googlebot, and doing little optimizations. These are all short term solutions, however, and are not going to cut it for much longer.

To make a long story short, I've decided to bite the bullet and just put in a quote for a new high end, quad Web server to complement our monster database server. With luck, we'll have a new box in on Friday or early next week.

Best Ads List - another YouTube powered video list

Let's add the Best Commercials List to the group of video enabled rating lists. RIA'er extraordinaire (and the original creator of this list) Kamylienne has gone through and added videos to most of them.

Thanks Kamy!

YouTube to Emulate RateItAll

At a recent conference in Davos, YouTube founder Chad Hurley announced that soon, YouTube would begin allowing contributors of video to share in the advertising revenue associated with their contributions. This sounds a lot like the RateItAll Economy Program, right?

And I think it's great news. Back in August of last year I wrote a long rant on this blog about how I felt that too many sites were exploiting the contributions of their members. Here's an excerpt:

Now big companies and venture capital firms are realizing that people are happy to work for these Web sites for free. Thousands of new "user generated content" sites are launching. Thousands hoping to be the next YouTube, the next MySpace - thousands hoping they can slap together a site, and have volunteers make it worth millions.

It ain't right, and it ain't sustainable. It took me a while to realize that revenue sharing is the only possible direction for RateItAll, but I'm happy to say that we are still one of the only sites on the Web to step up to the plate on this.

I'm thrilled that YouTube has come around, and I hope that others will follow.

I hope that the primary reason to visit RateItAll never becomes money. If that happens, something about our community will have been lost. But that doesn't mean that we shouldn't do things the right way and reward contributors.

RIA Review Widget

Check this widget out:



I built this on MuseStorm, using the RSS feed available from Magellan's profile page. Anybody can do it - it's easy. Let me know if you need help.

We're working on something similar that you'll be able to grab on your own profile page... it should be ready in a few weeks.

Very Cool Search Feature Just Launched

OK, we now have the ability to filter search results by "just items" or "just lists." Among other things, this makes it much easier to discover if a Weblist of a certain theme has already been published.

Here are some examples:

Here are the search results, site wide, for Beer as an item.

Here are the search resuls, site wide, for lists about beer.

To use this feature, just do a normal search to trigger the advanced search filter options. Use the "Show Me" option in the drop down to indicate whether you would like to see everything, just items, or just lists.

Site Search Tweak

Guys, the site search engine may be a little dicey for a bit - we're rolling in a new feature that will let us search for just lists, or just items. This should be very helpful in figuring out if a Weblist has been done before or not. Stay tuned.

Video Lists

Man, these video lists are addictive. When somebody takes the time to pull together the best video clips within a particular genre, the result can be pretty impressive. And I think we're only at the tip of the iceberg - I'm getting ready to fire up RateItAll's annual Super Bowl Commercial list... this year we'll have the videos embedded.

If you haven't checked out Planetary Gear's list of Chapelle Show Skits, do so now... it's excellent.

Also, if you make a list with videos, try and remember to tag it with "riavideos" - this way we can keep them all together.

Sharing Revenue on RateItAll

Has anyone checked in on their Google AdSense earnings via the RateItAll Economy program recently? Mine are showing a bump. RateItAll has posted five straight months of growth, and it makes sense that all of us should be seeing more traffic to our lists, listing suggestions, and profile pages.

Here's one glitch that started for me yesterday, however - because I never verified my AdSense account with Google (put in my SS# and address so I can be paid) - they started running Public Service Annoucements on my pages (not ads). I went in yesterday to verify my account, and they asked me for a pin number that they had supposedly mailed me. I never got it, and had to request a new one (which takes 2-4 weeks). I'm trying to get in touch with Google to see why they are showing PSA's now while I wait for the new pin... it doesn't seem fair. I'll keep you posted.

But in the meantime, if you are participating in the RIA Economy program and have not yet verified your Google AdSense account, I recommend that you do so by going to the AdSense site and signing in.

Daily Show List

Check out Loerke's list of Daily Show Correspondents. He's dropped in videos for a number of the pages... I wasted about 45 minutes this morning watching Daily Show videos. This is a fantastic Weblist that makes use of all of the media tools that RIA has to offer.

Best. Bug Fixing. Day. Ever.

Your feedback was almost unanimous about how we governed the reposting of a review. You wanted the ability to edit and repost reviews without losing helpful votes, and also pointed out a couple of bugs in the process. Here is what we've done:

In the last fix of the day, we've switched up the rules regarding reposting of a review. Before, this is what happened when a review was reposted:

- all of the helpful and not helpful votes were cleared

- anyone who has voted a review helpful or not helpful was not able to vote again

- all comments associated with that review got erased


Here is what now happens when a review is reposted:


- the review retains its helpful / not helpful votes

- people are free to change their helpful vote to not helpful and vice versa

- comments associated with the review are retained.


Thanks for all the feedback.

New Item Suggestion Tally Faxed

The tally on everybody's profile page that displays how many new items that people have contributed to ratings lists is now fixed and displaying the correct number.








The site leader in this category? MA Duron. For those keeping score, he's contributed 657 new pages to existing ratings lists. We're working on flipping this contribution page around so that the most recent items are listed first.

A Wake Up Call

Yesterday a pretty good reviewer left the site. He hadn't been around long, but he was prolific. He wrote very solid reviews, and made some good weblists.

He left the site because he was growing increasingly frustrated with some of the buggier aspects of the site. I think the straw that broke the camel's back for him was when he spent some time on a weblist, only to see 6 duplicates of each item show up.

Anyway, he said some pretty nasty stuff and deleted his account.

This is bad news, any way you look at it. If there is a silver lining, it's that this loss lit a fire under us to get around to fixing some of these nagging bugs so that it doesn't happen again.

The challenge from an administrator point of view is finding time to keep the "little stuff" working, while continuing to try and move the site forward with the "big stuff." Sometimes you need to just stop what you're doing and sharpen the knife a little bit.

I'm please to say that we've knocked out the most nagging bugs facing the site (dupe items, search, out of control alerts, and a few others - and it feels good.

Email Alert Bug Fixed

We've fixed that bug that was automatically signing people up for an email alert for every item that they rated.

You now have to opt in to this alert, and we've added a check box to be low the alert field that makes it perfectly clear what you're signing up for.

Search Bug and Dupe Item Bug Squashed

OK, the search bug - which causes an error if you try and search for something before the page is fully loaded - has been solved.

So has the dupe item bug. You can now disable items and hit back on the browser without duplicating your items. However, if you hit back on the browser and resubmit the same items twice, they will of course be duplicated.

Let me know if anybody sees any more weird weblist publishing bugs.

This is a good day - those two bugs had been, uh, bugging me for a while.

Inching Towards Consolidating Weblists and RIA Lists

I think I've mentioned in a previous post that the plan is to combine the weblist channel, and the RIA list channel. To a new user, there's no difference - everything is rating lists. Some are created by members, some are created by RIA staff (who are also members). An artificial division between the two doesn't make a lot of sense. Also, this consolidation will allow us to simplify and standardize some of the more confusing aspects of the site.

You may have noticed some formatting tweaks to the main Weblist pages as we inch towards this standardization:

- Specifically, we've gotten rid of that big yellow bar that says "This list was created by....." and instead by the list owner and publishing date in parentheses after the list name. (soon, there will be a similar indicator for RIA lists that displays the data and "RateItAll" as the list owner)
- We've gotten rid of a duplicate display of the weblist title
- We've (finally!) added the weblist scale at the end of every weblist description (e.g. 5=great!)

I'm happy that the important content on each page seems to be moving up.... as always, please keep your feedback coming as to how we can make this site simpler, cleaner, easier, and more fun.

Bug Eradication

We are working today on eradicating the following bugs:

- Weblist Dupe Item Bug
- Search Bug (if you launch a search before a page is fully loaded, you get an error)
- Alert Bug (everybody is getting default signed up for new review alerts on anything they review - this is annoying, and unintentional)

Duplicate Weblist Item Bug

Guys, I'm really looking to nail down exactly when the bug that causes new weblist items to be duplicated occurs. If any of you serious weblisters can give me examples or theories as to when this bug happens, I'd really appreciate it.

I'm going to try and stamp this particular bug out tomorrow - it's been lingering too long.

Barrack Obama Video

Democrat Barrack Obama announced his presidential candidacy recently with an embeddable video. You can see it on RateItAll's Barrack Obama page on the 2008 Democratic Candidate list.

This sort of list will be a lot better once we can add videos for all the candidates, don't you think?

Did You Know: Adding Videos to Item Pages

Did you know that any logged in RateItAll member can add a YouTube video to any item page - in real time?

Here's how:

Click the link on the item page that says: "Edit Description"
Copy and paste the YouTube embed code into the item description field.

That's it.

I added Weird Al's "White and Nerdy" here. It's easy.

Content is Moving Up, Ads are Moving Down

Some of you may have noticed that we got rid of the big Google Ads at the top of every list and item page, and instead placed an additional Google Ad block at the top of the right column.

This means that the main content of RIA - lists and reviews - is moving up on the page. In other words, you don't have to scroll down through ads to see some good stuff.

This is an experiment to see if we can convert some more new folks to regular users by showing them more content, without damaging ad revenue. If ad revenue plummets too much, we may have to undo this (RIA actually bounced its hosting check today, gulp).

RateItAll Growth

We've put up four straight months of pretty solid growth. Here are the numbers of visits per month since September, 2006:

September - 699K
October - 794K
November - 802K
December - 851K

January is off to a pretty good start - I'd like to see us push the streak to five in a row.

Ambitious Spammer

Wow. An RIA user recently set up more than 100 different accounts in order to spam Eddie Van Halen to the top of the best guitarist list.

I have to give this guy high marks for persistence - but low marks for ethics and brains.

In addition to the scores of regular users who keep in eye out for these sorts of spam attacks, we also have automated detection tools to pick up unusual behavior.

The end result? This guy has been permanently blocked at our firewall - no warnings or explanations required.

As always, your feedback in helping us to be aware of suspicious activity is appreciated.

Weblist Suggestion (Thanks GenghisTheHun!)

Guys, the very polite tradition that many RIA'ers have about giving additional attribution to new item suggester's using parentheses is actually a bad idea.

For example, let's say GTH suggests adding "Dingoes" to a list of "Dangerous Animals to Babies." The list owner may be tempted to add the new item to his/her list like this, in order to thank GTH for the suggestion:

Animals Dangerous to Babies

1) Dingoes (Thanks GenghisTheHun!)
2) Snakes
3) Alligators
etc.

Here's the problem with this - RateItAll relies on Google for a lot of its traffic. The newly added page about dingoes has excellent potential to attract Google traffic for searches like:

"dingoes baby danger"
"are dingoes dangerous to babies"
"dingoes baby animal danger"
etc.

However, I can guarantee you that NOBODY, with the possible exception of GTH himself, is searching for:

"Dingoes Babies Thanks GenghisTheHun
"Thanks GenghisTheHun Baby Danger Dingoes"
etc.

So unwittingly, by giving credit to GTH via parentheses in the new item suggestion, the list owner is actually putting obstacles up to people who might be looking for the dingo page.

This hurts the list owner by hiding his list from folks who might want to rate things, it hurts GTH by preventing new users from visiting the page he added (for which he gets ad sharing credit), and it hurts RIA.

Fortunately, RateItAll already tracks who submits what items to lists, and gives public attribution to the submitter on the item page in a way that won't mess up search traffic.

So please... let's change this polite, but damaging custom of adding a (Thanks!) after the item name on a weblist suggestion.

Batch Delete for Alerts

Guys, we really need a way to delete alerts in bulk. I'm aware of it, and it's on the list of pending enhancements.

Site Search

OK, we have the site fully indexed again, meaning that any page that was created before today should be findable via the RateItAll search engine.

I'm working on a tweak that will allow searchers to indicate whether they are looking for a list, or an item. This should be very helpful in figuring out if a certain Weblist is a duplicate or not.

Rough Week

RateItAll has had a rough week. We've had a power supply fail, our mail server crash, and a DDOS attack. The site's most important page, the recent comments page, has broken and is taking 10-15 seconds to load. In our efforts to protect ourselves from the bot attack, we accidentally banned our search partner from the site, which broke the search engine. We've had to adjust our firewall to give us more control over who we let into the site, and who we keep out.

And of course, all of these things pale in comparison to what Ridge and Stark are going through.

I feel like we're finally getting things under control. Search is working again, though things that have recently been added to the site will most likely not start showing up in the search engine again. The power supply has been fixed. Our lead developer is working on recoding the recent comments page to make it faster and more stable. I've put in an order for a new mail server which should be up and running early next week.

Hopefully next week will be a better week than this one.

Scheduled Downtime for 3PM EST TODAY

Hi all, RateItAll will be going offline between 3PM and about 4PM today. We're making some changes to our firewall that will allow better protection against the rogue bots that have been hammering us recently.

Ridgewalker Update

I know a lot of us have been worried about Ridge. In a lot of ways, he's the heart and soul of the RIA community - a true gentleman who gives everyone the benefit of the doubt, helps out newbies, and is always quick with an encouraging word.

Here is the full update from Stark:

"You can tell folks (post it on the blog if you want) that Ridge thought that he had been struck with the worst flu bug of his life. He thought he might feel better in a couple of days if he drank lots of fluids and got plenty of sleep. Only it turned out not to be the flu. The pain seemed to be getting worse until it was difficult to breath. At that point I took him to the emergency room. He had been through a full blown heart attack and the local ER put him in an ambulance and transported him to the cardiac hospital in the next town.
They did an angiogram and put a balloon in the artery to open it up, then inserted a little spring like tube to keep it open.
He's not out of the woods yet, but he's doing much much better.
His humor is returning, and that's a good sign.
Ridge is one tough bird. He's survived cancer, a stroke and a heart attack. And he's still grinning."

Well wishers can chime in here.

Site is stabilizing.... slowly

We almost have things under control. We were hit by a couple of malicious bots that slowed the site to a crawl, which started a chain reaction of certain process breaking.

One of the processes that got screwed up is the tabulation of helpful votes on the recent comments page. Folks trying to visit the recent comments page while logged in have been reporting long load times.

In an effort to speed things up while we figure out the root cause of the issue, I've limited the most recent comments page to the most recent 100 - I'm now able to pull up the page while logged in (but it's taking 10-15 seconds).

And in the category of "when it rains, it pours," the search engine is currently not returning any results.

We will spend the day tomorrow trying to work through these remaining issues.

t-shirts went out today

Guys, look for your t-shirts in the mail over the next few days... all but two went out. For those of you who ordered black baby t's, I'm going to have to resend them. The ones that were sent to me came out terribly... i need to tweak the design a little bit in order to make them legible.

Power Supply

Hi Guys, we've had a power supply fail on one of our primary servers. We're working on replacing it... stay tuned.

Recent Comments Page

Some members are reporting problems getting to the recent comments page while logged in. We are aware of the issue and are working on it.

The Site is Very Sluggish

We're aware of the issue and are working on it.

Updated Senators and Governors

Thanks to a lot of help from GTH, most of the newest Senators and Governors have been added to our lists.

Backlogged New Item Suggestions

Well, I spent most of my New Year's Day going through about 2,000 backlogged new item suggestions on both RIA topics and Weblists. I'm happy to say that the new batch delete feature made it infinitely easier to manage this project, and that I am almost completely caught up on RIA and Weblist new suggestions.

I still have about another 200 local suggestions to go through, but I've tried to approve any local suggestions from RIA regulars.

Happy New Year! (and t-shirt update)

I hope everyone had a fun and safe New Year's Eve and New Year's Day.

The t-shirts have finally arrived, and the plan is to spend a few hours on Wednesday packing them up and shipping them out. Everyone who got their names in should have their shirt by early next week...