9/25/07

We're Working on a Couple of Things

Here are the projects that we are currently working on:

Helfpul Vote Changes

Based on a lot of excellent feedback, we are making some important, structural changes to how we manage helpful votes.

One of the issues that I currently see with the helpful vote system is that there is only one tool to acknowledge a review. A review is posted by your friend, and your only option to indicate that you appreciate the review is to mark it "helpful."

In the world of RateItAll, "helpful" has come to mean all sorts of things. It has come to mean "I agree," it has come to mean "lol," it has come to mean "I love you man," and it has come to be a simple acknowledgement of "I read your review."

In other words, "helpful" has lost its meaning.

So, one of the things that are we are working on is to make it so there are three acknowledgement options for any review:

Helpful
Funny
Agree

This will give vote givers a way to be more precise in what their review acknowledgement means, and will give vote receivers a way to get more specific feedback. It should also start to put meaning back in the word "helpful."

Of course, the lack of tools has not been the only issue involved in the abuse of the helpful button. Some folks have flat out abused it - giving helpfuls to their friends, regardless of the content of the review, and not giving helpfuls to folks that aren't their friends, regardless of the content of the review.

So, unfortunately, we are also working on adding a tool that will let us remove vote giving priviliges for those that abuse the tool. This tool will be used sparingly, and only with the goal of ensuring that folks act in good faith in awarding helpful votes.

Finally, we are getting rid of the not helpful votes. I have long resisted getting rid of the not helpful button, as I felt that it was necessary to retain a why to indicate disapproval of a review. I know that there are some of you that still feel this way now.

However, there IS a tool to express disapproval - the commenting tool. If a review rubs you the wrong way enough to make you want to take action, you are more than welcome to comment on the review and indicate exactly why. No longer will there be anonymous "trolls" dropping unhelfpuls with no explanation or reason given. We're going to try it this way and see how it goes.

Home Page

The second major project that we're working on is a revamping of the home page. I don't know about you guys, but the big yellow block of real estate on the home page sort of bugs me. We're making too much of an effort to explain what the site is about, as opposed to SHOW what the site is about.

In the coming weeks, you're going to see some changes to the home page with an emphasis on providing all sorts of fresh content that shows what's hot on the site at any given time - most active raters, hottest lists by category, hottest items, etc. I think it's going to be pretty cool.

Stay tuned.

9/23/07

Debrief

It’s been an awful week for our site.

In the past week alone, RateItAll has lost four excellent reviewers: IJR, StarkTruth, Ridgewalker, and Wiseguy. Each has chosen to leave the site, and exercised their rights as RateItAll members to permanently delete their accounts.

As I’ve seen a lot of rumors floating around as to the circumstances regarding their departures, I thought it was important that I clear the air and present the facts as I understand them.

As a rule, I try not to discuss site disciplinary matters publicly – these have historically been left between the admin and the member. But as these are extraordinary circumstances, I’m going to be as transparent as possible in order to present the most accurate picture of what transpired.

Those involved are welcome to comment on this blog if they feel that I got anything wrong.


IJR


As we all know, IJR was a longtime reviewer who was very popular. He was prolific, he was funny, and he was pretty unique. At the time of his departure, he was third ranked in the list of Most Helpful Reviewers.

About four weeks ago, I noticed a series of very odd sounding reviews from him. They were not written in his style, and had an oddly formal tone to them. I ran one of the reviews through Google, and found it be copied directly from another site. There was no citation given to the author of the review.

I messaged IJR, pointed him to the site rule about plagiarism, and asked him not do it again.

The following day it happened again, so I warned him again.

The following day it happened again, so I warned him again, and said that if this continued to happen, I’d have no choice but to suspend his account for two weeks.

Note: it’s almost unheard of for me to give three separate warnings, in rapid succession, for the same, fairly serious, violation. IJR was given this courtesy only because of his standing with the site.

A day or two later, it predictably happened again, so I suspended his account.

When the two week suspension was up, I reactivated his account. Within 24 hours or so, IJR had sent messages to a few folks (myself included), complaining of an old boys’ network, and deleted his account.

I feel that my hands were tied in this particular situation. The violations were so blatant and so defiant, that my choices were either to say, “screw it, the rules don’t apply to IJR.” Or to take action. I guess I could have asked him if there was something else that was bothering him.

Looking back, I wish I had.

My personal opinion is that IJR was intent, bordering on obsessed, with becoming the reviewer with the most helpful votes. When he started to feel like it wasn’t going to happen, he prepared himself to self destruct by flaunting the rules. I wouldn’t be surprised if he felt that the odds were unfairly stacked against him, hence the “old boys network” (which I take to mean cronyism) accusations.


StarkTruth and Ridgewalker

As most of you probably know, StarkTruth and Ridgewalker are a couple. Whip smart, Stark is an excellent writer who made a pretty significant impact on the site with both her reviews and her lists. From an admin perspective, I considered Stark to be very much a “tough cookie.” While she didn’t start fights, she had a history of coming in pretty hard when she felt that she was slighted. I had a bit of an admin run in with her shortly after she joined, when she began messaging a number of regular reviewers with the false information that I was planning on selling the site. A heads up was given to me by several regulars. I’m not sure why she thought this (I’ve never actively tried to sell the site), or why she felt the need to try and start a panic.

Despite her impressive contributions, I never got the feeling she particularly liked (or trusted) the site (or me).

And then there’s Ridge. In the history of RateItAll, I can’t remember anybody who has been more universally liked and admired than Ridgewalker. Besides writing the best reviews and making the best lists (and helping others with their lists), he was also the first to welcome new users and the first to try and talk overheated reviewers down. He was a giant on RateItAll, and somebody who I personally admire a great deal. When he fell ill, I sent him a care package – in the site’s nearly 8 years online, this is the only time I’ve done this.

About two months ago, Ridgewalker brought it to my attention that a certain RateItAll reviewer was accumulating helpfuls at an unnatural rate – something like 2,000 in a 24 hour period. I investigated the matter and found this to indeed be the case.

What was strange is that the helpfuls were coming from a user that was not logged in, who was at the same IP address of reviewer in question. In other words, somebody seemed to be logging out, and giving helpfuls to themself.

As far as I knew, this was an impossible scenario. You can’t give helpful votes if you’re not logged in, and you certainly can’t give helpful votes to yourself while logged in.

This was happening at around the same time that RateItAll was going to a multi server configuration. A known bug at this time is that folks were getting logged out in mid browse, as they bounced back and forth between servers. In other words, some members found themselves in a weird limbo status where they were logged in at one moment, and logged out the next. Then logged in again.

In this weird limbo status, it was apparently possible to give your own reviews helpful votes.

This is important, because it gets to intent. This was not a situation in which the member had gone out of their way to set up a duplicate account with the sole purpose of giving themselves helpful votes. My understanding is that this was a situation when a member was browsing the site, and suddenly realized that they had the ability to give themselves helpful votes.

So, I suspended the user’s account for 3-4 days while I sorted out the technical glitch that enabled this, and deleted the bogus helpful votes.

Amazingly, we discovered that at that moment, it was not possible to subtract helpful votes in our system, as explained here. I felt that it was important enough to get this particular user’s bogus helpfuls off the books, that I commissioned the development work to fix this. When this was fixed, not only did the offending user lose their bogus helpful votes, but everybody’s tallies were recalibrated to reflect subtracted helpful votes (primarily from folks who had left the site).

I also formally warned the user, and expressed my disappointment at the bad judgment.

So why didn’t I ban this person? For a couple of reasons. First, the person is a regular, and regulars nearly always get the benefit of the doubt. Secondly, there have been at least 7 other cases of members in the top 100 who have been discovered showing bad judgment in regards to manipulating the system (specifically, setting up dupe accounts) in order to pad their own tallies. All of these reviewers were warned, and given a second chance. None of them have broken the rules again in regards to dupe accounts.

Finally, given that the vote getting was enabled by a technical glitch, and not premeditated, I don’t see how it would have been appropriate to jump the warning and suspension stages of discipline, and go right to ban. Just as there’s a difference between “stealing” a bag of money that falls out of the back of a truck and robbing a bank, there’s a difference between creating a false ID to give yourself helpfuls, and suddenly realizing that you have the power to give yourself helpfuls.

There was bad judgment displayed here, but it simply was not a ban worthy violation. If anything, it was less severe than stuff that I had let go with a warning previously. Banning this user would have been utterly inconsistent with how I’ve managed similar situations in the past.

So here’s where things get murky.

At exactly the same time as this helpful vote situation, Ridge told me that he and the member described above, had been emailing with each other. Allegedly, there was some inappropriate stuff emailed (not messaged through RIA) to Ridge. This allegedly inappropriate stuff came to Stark’s attention, who understandably, was not happy about the situation.

Stark posted an angry personal attack against the reviewer who had been emailing with Ridge on RateItAll (though without actually naming the person), and the reviewer responded with a threatening message. Unfortunately, both of these posts were not immediately noticed, and stayed on the site for longer than I would have liked.

Eventually, the two posts were deleted, and both reviewers were warned. No disciplinary action was taken against Stark (as it was an extenuating circumstance) or the other reviewer (as it was a response to an attack.)

And of course, it is not within RateItAll’s domain to discipline RIA members for what (allegedly) happens on their own time, via their own personal email servers. From the perspective of site discipline, any email or real world correspondence that happens between RIA members is simply not relevant.

I did make at least one critical mistake in my handling of this situation – I discussed some of the details of the bogus vote getting with Ridge. Over the years, I discussed A LOT of site issues with Ridge. Both strategic, and tactical. In a lot of ways, Ridge was one of my most trusted advisors on things related to RIA. He was probably the most respected member of the RIA community, he had the site’s best interest in mind, and he is an incredibly smart, mature, and insightful guy. Of course I’d want to tap his wisdom.

So why was it a mistake to discuss the bogus helpfuls with Ridge? Because of the timing, and his relationship with Stark.

I can imagine Stark’s outrage when she found out that not only was a RateItAll member emailing inappropriate stuff to her man, but that same member was accumulating bogus helpful votes.

Flash forward to last Friday. Stark again posted a personal attack on the reviewer (without specifically naming the person). I removed it. Stark immediately posted it again. I removed it and suspended her account to try and talk things with Stark via Ridge.

There was no way that I was going to allow a public, nasty, personal shootout on the site.

In messaging with Ridge, it became pretty clear that Stark (and probably Ridge) expected the reviewer to be banned, and saw my decision not to ban the user as a slap in the face. I responded that it was not within my domain to ban somebody for something that happened away from RateItAll. Ridge responded that he understood that, but what about the bogus helpfuls? I responded that the user had already gone through a suspension, and based on how I’ve moderated the site in the past, I could not justify going right to a ban for that particular violation.

So we were stuck. Stark was furious that an RIA member could get away with inflating their helpfuls and crossing the line with her boyfriend, and she wasn’t even given the opportunity to respond on the site. Ridge, already concerned about what he perceived as a growing problem with RIA’s culture (more on this to follow), was also in the awkward situation in which his continued participation on RIA might be seen as disrespectful to Stark (my speculation).

What a mess, what a mess, what a mess.

Soon after, I received a note from Ridge requesting that I reactivate Stark’s account so that she could self delete. I complied. Stark deleted her account. Soon after, Ridge deleted his account. This was a devastating surprise, though looking back, he probably felt himself with no other options.

At a micro level, I feel that my only mistake was discussing the details of the bogus votes with Ridge – you don’t tell something to one half of a couple, and expect it not to get to the other half. That’s not fair. This information, which was incomplete and should have been confidential, only inflamed the situation even more.

I stand by my decision not to ban the other reviewer, and not to let my respect and admiration for Ridge make me moderate the site in an unfair manner.

But at a macro level, I failed miserably. The site lost its best reviewer and list maker and friend in Ridge. Worse, I don’t really think he wanted to leave (I could be wrong about that).

My gut tells me that Stark was going to take off regardless, but perhaps more skilled diplomacy by me could have salvaged her RIA career as well.

Wiseguy

This brings us to Saturday.

Wiseguy was a respected and passionate member of RateItAll. His main interest was politics, which meant that he got into it from time to time with folks (including me), but I think that there was a lot of respect for him on both sides of the political aisle. He’s a guy that I know cared deeply about the site, and when he said something regarding site management issues, I listened.

Saturday morning, Wiseguy posted the following video to YouTube expressing concern about the recent loss of talent from RateItAll. He also suggested that “if it had been him, he’d be gone” (I’m assuming that he meant the bogus helpfuls) and that I needed to get a handle on some of the shenanigans that certain reviewers were getting away with. Finally, he suggested that I work harder to keep the talent from leaving.



His departure came as a shock to me. I had sensed some growing frustration from him over recent months – specifically, that he felt that liberals were treated better on RIA than conservatives. But still, I didn’t expect him to leave, and am pretty bummed out about it.

Two things in particular make me sad about Guy’s departure; 1) that it wasn’t his fight; and 2) he was given bad / incomplete information. The rumor that was floating around was that a reviewer had hacked into RateItAll and given themselves thousands of helpful votes, and was allowed to stay. This is very different scenario than what really happened – a technical glitch on RateItAll allowed a reviewer to give themself helpful votes.

So no Guy. You would not have been banned if it were you. Nobody would have, except perhaps somebody on final warning status. And you’re more than welcome to come back – we’d be thrilled to have you.


Moving Forward

So where are we at? Well, we’re bending, but hopefully not breaking. I’m heartened by some of the level headed things that folks are saying here. There’s a lot of talent, and humor, and class left on RateItAll, and although we’ve taken a terrible hit this week and lost some great, great people, I’m hopeful that we can get things moving in the right direction again.

One of the things that I’d like to address in the next few days is this issue of “culture.” I’m prioritizing community things over all else for now – step one is going to be getting rid of that damn not helpful button. No need for the negativity. I also have some other ideas for some tools with which we can recognize our friends, but without over dependence on the helpful button. Finally, given that helpful votes have become the de facto currency of the site, I believe that we are in need of better regulation around abusing the helpful button, so that we can make excessive cronyism unprofitable so to speak. Stay tuned.

9/7/07

RateItAll Has An Intern!

I'd like everyone to extend a warm welcome to Jose. Jose is a grad student at UC Hayward focused on e-business. He's an exchange student from Granada, Spain.

His primary role with RIA is going to be making sure that we have coverage on the site for things that are hot in the news. He's also going to be helping with responding to user feedback, and rolling out our new feedback widget. He may also blog occasionally.

I met Jose on the basketball court - he's the best European player that I've ever played with / against, and was at one point, getting paid to play hoops.

So please be nice to our new intern, and if you get a chance, shoot him a note to say hi.

9/6/07

New Mail Server Online

Wow. So far, it seems like the new mail server is having a substantial impact on overall site performance, specifically in terms of:

- Leaving a review (page is no longer hanging as email alerts are sent out)
- Sending a personal message
- Inviting others to rate / join

I knew our old mail server was crappy, but I didn't know it was THAT crappy.

9/5/07

Search Engine Just Got A Lot Better

We just made a small but important tweak to our search engine.

As you know, the site is partitioned into three primary channels: RIA, Weblists, and Local. Until just a few minutes ago, our general search results were prioritized in that order. So if you did a search for George W. Bush, you would see results from RIA before you saw results from Weblists - we did this because we could vouch for the quality of the listing on RIA, whereas the quality of the listings on Weblists run the gamut from excellent to weak.

The problem with the way things were set up, is that we didn't have control over the weighting involved in driving the results. So the search engine would display ALL results from RIA regarding George W. Bush, even if it was just a casual mention of the word "Bush" in passing in a review, ahead of any listing from Weblists - even ahead of a listing that was specifically about George W. Bush. This was really annoying, and it made it really hard to find, say, where Barry Bonds ranked in the Biggest Jerks in Sports list.

This is no longer the case. We now have control over the weighting for the three partitions for the site. Currently, a search for George W. Bush will still show results from RIA first, but a Weblist listing for George W. Bush will now outrank a passing reference of "Bush" in a review in the RIA channel.

This is a small tweak, but I think you'll find it improves the quality of the search engine dramatically.

9/4/07

I Love Rate-O-Matic

Rate-O-Matic is one of my favorite RIA features to come down the pipe in a long time. I especially like the "speed rate" option for specific lists.

There's a few things I really like about this feature:

- It helps you find stuff that you wouldn't have otherwise rated
- Your rating is not biased by what other people have said
- It's a quick and easy way to blast through a list
- It's a quick and easy way to blast through your own new weblist, to announce its presence to the site

According to our stats package, in just a few short days, the Rate-O-Matic page has become the second most popular page on the site (after Recent Comments).

It seems that I'm not the only person enjoying this feature, and over the next few days you'll see more integration points where you can hook into this service.

Email Alerts Turned Off

RateItAll's mail server is a piece of garbage. It's always crapping out on us, and worse, it slows down the reviewing process on the site - here's why:

When somebody sets up an email alert on an item or for a reviewer, when a review is posted, the site waits for the email alert to be generated before loading the page. Because the mail server is so crappy, this sometimes takes a few seconds (sometimes more than a few), meaning the page hangs until that connection is made with the mail server.

Well the server has been getting progressively worse. Today, it was taking so long to post a review, that I've decided to just turn email alerts off until we can get a new mail server in place.

So, we're shelling out the $600 for a new server, which should be up sometime tomorrow. Until then, there will be no email alerts (new messages, new ratings, etc.) sent from RateItAll.

The good news is that the reviewing process is super fast now.

Apture