Puzzle Description Suppressed:Click below to view spoilers

#1: Tom King (sgusa) on Dec 2, 2012

Kind of Aldege length in the intro, but I wish to comment further. If you are an anonymous puzzle rater who likes this forum and the puzzles, would it not make sense to at least rate accurately and comment? By commenting, you cement your place in this forum and supply constructive criticism for those that are doing you a pleasure, creating puzzles. Put yourself out there and see how it feels from the other side.

Out of the last 15 puzzles that I have done, I have had two rated average or slightly better in difficulty and two average or slightly better in quality. The rest are BELOW AVERAGE. That means I am an abject failure at something I give my time for free. I can waste my time better elsewhere...

If you rate puzzles poorly, it is time to start making them better. Comment or don't bother rating. Better yet, start creating some 20x20 black and white that I will enjoy..

This is one of the reasons why I don't create anymore puzzles on WPBN.

I understand your frustration Tom and Kai however,is it really fair that those of us who enjoy your puzzles should suffer and be deprived of them,not to mention that you will be suffering too,because you know you enjoy and have fun creating.I get discouraged at times and feel like quitting,especially when the ideas for creating puzzles are getting harder to come by.There are always going to be people that mess with the system,that's just something we have to live with.Your puzzles are great Tom,and very entertaining,please hang in there,and Kai,you have always been one of the premier creators,please make more for those of us who really appreciate them.:l

Tom, this issue has been addressed before and the consensus of opinions is to simply ignore the ratings as there are idiots out there who take great pleasure in messing with the ratings, just as there have always been bullies and always will be apparently. These are people who do not have the slightest inkling on how to make puzzles and, therefore, cannot appreciate the effort that goes into making them. So ignore the ratings and pay attention to the comments of those of us who really enjoy your puzzles. Your puzzles are always good and you have a really great sense of humor that you convey in your puzzles. So please don't stop!

Kai: your puzzles have always been wonderful!
Tom: If you stop then the rest of us will still have to put up with Mr/Ms 1 pointer. Quiting will only give them power. Please re-consider.

I couldn't agree more with Norma and Aldège.

I have always felt that the 3 quality rating should read "good" rather than "average," which would make the 2 and 4 "fair" and "very good" respectively, so that's what I'm thinking when I rate them. I save the 1s for fill-ins. As for difficulty, I find that as I have gotten better at solving I tend to rate them as easier than when I first started. You are right, Tom, we should rate difficulty as we find 'em.

As someone who loves webPBN but doesn't create puzzles, I thank all of you who do.

I agree with Lollipop. The quality rating is a bit restrictive and makes it difficult to rate puzzles equally and fairly. Perhaps a rating similar to grades in school would work using 1 through 5. Then a puzzle could be rated, say, a 3+ or a 3- when you don't think a straight 3 really fits the puzzle.

naw, I agree with Tom. 3 means "average". anything below is below average, and above is above average. an average puzzle with average difficulty should be given a 3 for both. it makes it seem ridiculous to see 90% of all puzzles created rated as "below-average". how is that even possible? defies logic....

but I hear ya, Tom...some idiot gets off on giving ones....just ignore it. the more you let em know it irks you, the more he/she will enjoy giving them to ya. hell, my charlie brown xmas tree got a 1 right off the get-go just yest.... more than likely cuz they know it annoys me. I understand your frustration, esp when hours are spent creating and tweaking for someone else's enjoyment. makes you think "why bother?", when a few of the people you do it for don't appreciate in the least

I reiterate. Ignore the ratings, pay attention to the comments. Bite the bullet and keep creating. Thank you. :)

I do alot of these puzzles and totally enjoy them. I know that I am not capable of making a puzzle so I depend on you guys to do so. So please don't get discouraged and stop making them!!! I rarely pay attention to the ratings, I look at the size and number of colors and usually the name of the puzzle. I prefer 20x20 and only black.

I didn't say I wasn't going to make puzzles. I wanted to get my point across in a puzzle specifically designed to make this point.

I hear all of you who comment. I am just making a point. The system says 3 is average, not good, not very good.

I've always considered average to be a good rating. What I object to is people who don't solve the puzzles but rate them high because they want to be "good" guys.

Average=average If you think about them as letter grades 5=A, 4=B, 3=C, 2=D, 1=F. As a puzzle creator, that is how I look at it.

In schools that use numbers instead of letters, 1=A so anyone who went to a school that used numbers would be more comfortable with this.

ditto #16 completely! that's how I rate em, that's how they should be

Then don't be surprised when I give you and "F" by mistake.:)

hahahaha

I'm always thrilled when I get a 3 for quality,but realistically if a 3 is just average and I know how much I put in that puzzle,I do feel a little disappointed.Lately I've been concentrating on trying to make my puzzles more of a challenging solve.:)

My schools used numbers but it was on a 4.0 scale. 4.0 was the best. A 5.0 scale is what we have, then it directly corresponds to my scale. Point being, 3 IS AVERAGE

def been getting more challenging, Al... don't go too much one way though. variety is good, and there are so many diff solvers and skill levels of solvers. toss out some big, some small, some easy, some hard.... WE know how much time is spent creating, but I doubt the majority do. for ex, I got home from work and started making one today a lil before 9am. after tweaking and solving several versions, I looked up and the clock read 1 30.... in short, keep up the good work y'all! forget the raters and do it cuz you luv it

Hear, hear!

Food for thought: plenty of people solve puzzles without reading comments. I didn't even notice the comments box for a long time, because I was solving at work. I did notice the comments before I became a member; at that time, I could read the comments but not leave one of my own. Then you look at someone like BlackCat, who apparently knows how to leave a comment, but hasn't discovered the Forums tab to read responses to his/her comments.

I can't remember whether I was able to rate a puzzle without being a member, but I'd have to guess that there are dozens if not hundreds of people who come here just to solve and rate the puzzles and couldn't give a rip about the comments.

When you sort the users as to people currently logged on, the names go on for page after page and many of those with a large number of solves have names that are totally unfamiliar. And when you look at the number of people who have rated a puzzle, the percentage is very small compared the the number solved. I would imagine that there are some who would comment but don't speak English.

I enjoy solving almost everybody's puzzles as long as they are not too big. I am only occasionally inspired to create a puzzle, and at this point, I'm more interested in the comments than the ratings. Sometimes, I have nothing to say about a puzzle I've solved, but I always rate them, unless it's more than a few months old. I wish that people wouldn't let a few jerks that rate everything 1 discourage them from creating puzzles.

JL, I am not discouraged. I understand all this. Here is a perfect example:

Puzzle #20516: And the winner of the Big10 er, ah, Big 12, I don't get it?




Version:

1



Status:

Published on Dec 1, 2012 by sgusa.
Solved by sgusa.



Size:

20 rows, 30 columns, 3 colors



Players:

78 (63 completed)



Raters:

12



Quality:

(3)



Difficulty:

(1.75)



Unique:

Definitely has a unique solution



Logical:

Definitely line/color solvable

12 out of 63 solvers rated the puzzle. I think the quality and difficulty should be half a point to a point higher, but OK...

Here is this puzzle:

Puzzle #20518: What is an average puzzle in Quality and Difficulty?




Version:

1



Status:

Published on Dec 2, 2012 by sgusa.
Solved by sgusa.



Size:

20 rows, 20 columns, 3 colors



Players:

129 (111 completed)



Raters:

10



Quality:

(2.25)



Difficulty:

(1.5)



Unique:

Definitely has a unique solution



Logical:

Definitely line/color solvable

The quality of the former puzzle is 1/2 point higher and the difficulty 1/4? That is BS. The former is much higher in both quality and difficulty.

Here's another and this one is a really tough solve:

Puzzle #20496: Movie title #110?




Version:

1



Status:

Published on Nov 30, 2012 by sgusa.
Solved by sgusa.



Size:

20 rows, 20 columns, 2 colors



Players:

173 (59 completed)



Raters:

10



Quality:

(2)



Difficulty:

(3)



Unique:

Definitely has a unique solution



Logical:

Believed to require moderate lookahead

These ratings are wrong and incongruent... Low on quality on this puzzle I accept (but it does look like Max from How the Grinch Stole Christmas). Really, really tough solve. Solve should have been north of 4.

Look at the # of solvers of my 3rd example. Roughly a third. The number of ratings are basically the same. It is exactly average in difficulty. In this puzzle, the first rating came in as 1,1, and I decided I would rate it. I was angry and rated it a 5 (A) in difficulty and a 2 (D) in quality. My ratings skew this as I don't rate my puzzles (except this time). I would be below 3 (C) in difficulty. What a load of BS...

And JL, why don't you rate puzzles more than a few months old? I do. If you comment, it is always a nugget in the creators inbox. David rated Route 66 today and left a comment. That is really cool.

Thoughts on difficulty. Recognizing that a fair number of puzzle solvers are going to pull up the solution to solve puzzles that they can't, here is my idea. 60% unsolved, automatic A, 50% unsolved automatic B. 25% unsolved automatic C. 10% unsolved automatic D. Less than, automatic F. Takes it out of raters hands and puts it where it should be. Solvability.

To clarify, an attempted solve but unsolved with a moderate lookahead should default to a 5. The solver couldn't solve it because it was TOO HARD for their skills. TOO HARD means a MOST DIFFICULT puzzle.

I've probably beaten this to death, but it really needs to be entertained. This puzzle is now above 2 in quality and almost 2 in difficulty. The difficulty rating I will agree with. Quality, it is terrible. It is rated better in quality than 8 of my last 14 puzzles. Get a clue, folks! I shamed you into rating it in quality. It doesn't deserve it, but you are screwing quality puzzles including many of mine.

What do the ratings refer to? The quality of the puzzle? The difficulty of the puzzle? The originality of the puzzle? The quality of the puzzle viewed as artwork? All those things and more? Pulling all these disparate aspects of a puzzle together to arrive at a rating means that a low rating is not necessarily a criticism of the puzzle maker.It is as possible to rate a really crap picture quite highly because it makes a good puzzle as it is to rate a great picture lowly because it took seconds to solve, so easy was it. Perhaps there needs to be more than one rating, say, <artistic worth, originality, degree of difficulty>.

Excellent point Leigh. I'd much rather solve a puzzle that is fun to solve but maybe not the greatest artistically. Sometimes puzzle makers are so intent on making the next Mona Lisa that they forget they are also making a puzzle, one that is not really fun to solve. Fortunately we have puzzle makers here that do both and those puzzles are really satisfying to solve. But as you have so aptly noted, there are just too many abstracts to consider.

Yep, artistic worth is entirely subjective. Some give points for humor, some don't. Some people might give a higher score to their favorite puzzle maker, just because it's one of their puzzles.

Also, if you're lucky enough to find that one pixel that completes a solve, then the solve seems way easier. Some folks are better at finding edge logic than others. One man's 3 is another man's 5.

I think the two rating categories are adequate. When I rate for quality, I include the art & humor. When I rate for difficulty, I attempt to give credit to hard puzzles. I am not loathe giving out 5s in either. A 5 is an A. The puzzle doesn't have to be perfect. Rarely do I go below a 3 (C) in quality. Difficulty, I give out a fair share of 1s & 2s.

as a judge for awhile... we give quality, difficulty a score. Title and humor added. Sticking with the required subject is important. what other peeps score has some importance. Mostly it is a good feeling about the puzzle that rules. Art is in the eye of the beholder! Its not an easy job. Tom your cool!

Tom, I don't give a rats arse about other people, nor do I care who the creator is. Leigh's point seems to be combining the two. Quality and difficulty are two entirely distinct and different issues.

I did a puzzle for WCP on Angry Birds. Really simple solve, but fairly high quality. The difficulty bumped up because it was high quality. I don't want props from artwork increasing difficulty. It wasn't a fill-in, but deserved a 2.

Conversely, I don't believe a really tough solve should be bumped down by the image. If we stick to 3=Average, we are fine. Average does not mean good. It means a "C." At least in quality. In difficulty, I would say it "tests you a little." 2 would be fairly simple solve. 1 would be fill-in. 4 would be pretty tough. Advanced logic necessary in several areas. 5 would be really tough (or couldn't solve it without peeking).

We'll see if any of my rant on this subject improves ratings...

i think sensei was saying that what other people rate it has some importance to the WCP judges, not that he rates a puzzle based on what it is already being rated.

i also think that it is funny that you say you "don't give a rats arse about other people"...because i thought the point of this puzzle was that you do.

...and for the record, i rarely rate puzzles and that's primarily because i rarely pay them any attention in any capacity.

One of your stated "idiots" who likes to give low ratings is someone who participates in the forums. Just because people choose to remain silent doesn't mean they are any less important. And just because they participate in the forums doesn't mean they judge honestly. No one should have to say a word to "cement" their place on this website. That's ridiculous.

And as a former WCP judge (albeit quite awhile ago), I never took into account what the ratings were. It takes time for a proper rating to be established, and that's not going to happen in a week.

Several weeks ago when I thought someone was messing with the ratings of a certain rising puzzle creator, I asked Jan to check to see if I was right. It's not an easy task for him to do, but he reluctantly did it anyway, and I was surprised to find out the results were the opposite of what I had suspected. It's not who you probably think it is, Deana. That person promised Jan he wouldn't do that again, and he hasn't, according to Jan.

One of the persons who has been messing with the ratings is indeed someone who posts comments. So when everyone calls them "idiots" you might be name calling your friends. :-D

Note to those who use multiple id's to mess with the ratings: Last time Jan did an investigation, he outed the individuals on the public forum. So if you don't want to be really embarrassed, you might want to rate fairly and singularly.

thx David :)

oh, so it's a "he", is it? you're in the clear, Kristen! hahahahaha! jk, ofc....

Yeah, well, having had dealings with this person, I would trust his word as far as i can throw him and his many personalities. *shrugs*

going back into my hole.

Whew!
(to comment 45)

Many things are imperfect in this world and the ratings could be leading the bunch. I personally solve every puzzle i the WC and most of those from creators I like (unless is pattern, I'm bored or things like that). To me a puzzle is average in quality when it is what I expect from good creators, not one in particular but all of you out there. Same with difficulty. But at the end is subjective so go figure. I personally don't pay attention to the ratings when judging or else. People tend to give excellent ratings to huge puzzles that to me are lame and very rarely solve.

Maybe it's time to agree to disagree.

glad you peeked out from your hole for even just a little while, Deana. nice to meet you(even though we never really met)

I agree that ratings in general will always be somewhat skewed because it's entirely subjective. no use worrying about it. just have fun doing what you do....

however... "average" is what you expect from good creators?? I'd assume they are good creators cuz they put out good puzzles.... so what you're saying is "average" is good. that doesn't make much sense to me. oh well.... the whole ratings thing will never change cuz everyone has a diff take on it. it's def not worth getting worked up over, though johnny 1-rater can be irksome at times. right about now, I'll just be happy when the site is up and running smoothly again. Have a great weekend, y'all!

David, what I meant to say was that I don't give a rat's arse about what other people have rated a puzzle.

Deana, thanks for climbing out of your hole and participating. I was leaning toward constructive criticism from those who rate would help puzzle creators. Rather than leaving a 1 and hiding. I didn't call the 1.1 rater an "idiot," rather a loser.

Jota, I can see how that bias may enter a bit. However, I try to not rate a creator against their creations.

Kurt, agree whole-heartedly.

well, Tom....you gotta remember that the title of your puzzle asks a very subjective question. there really is no right or wrong answer. it is good to hear what everyone thinks though, and it'd be even better if everyone was on the same page with what qualifies as average. I just comment with what I think, and by no means am I criticizing anyone. everyone has a right to their own opinion....

I do wanna say this though... I don't really know about the past so much, cuz I rarely bothered to read comments and never commented myself for a long time. But the 1-rater is NOT Adam. If there's anything that he is, I think we can all agree that he's honest with what he thinks, even brutally so at times. It's not him. And if it was, he'd have no problem saying it was. But it's not. He doesn't always seem to get a fair shake. Deanna implied it was and Teresa(who's in the know) specifically implied it wasn't. I'm not taking anyone's side with anything(as that seems to go on), but I hate to see someone get accused of something he didn't do, nor does anyone have any real reason to think he did other than they don't like/trust him for whatever reason or for whatever happened before.

Let's be careful about implications and assumptions. I never named Adam. And I was referring to people who use multiple accounts to mess with the ratings. It is my understanding that Adam has only two accounts and he only uses one of them for rating. How he rates puzzles, I have no idea.

On the other hand, there are many "1 raters" out there (according to Jan), and there are more than a few who have been using multiple accounts to mess with ratings. They are the ones who need to be worried, because Jan in the past has published identities of such persons on the forum.

As for the puzzle ratings, it really doesn't matter that much in the long run. If someone makes a good puzzle, it will eventually attract lots of solvers. The more solvers, the more ratings. The more it gets rated, the more accurate the rating will be.

What makes a good (better than just average) puzzle? Well, that is a whole nuther Forum Topic. :-)

The discussion about rating, the "truth" about some raters and the the development of the Weekly Contest in the last month is frustrating me. I don't want to go on this way. Shaking my head I will look here from time to time, but not more. Sorry, Guys and Girls, but I have better things to do.

Well said (most of you). :-)

Thank you for investigating, Teresa.
As Norma said, this issue has been discussed to death in countless other forums and in other puzzle comments.
Forget it and move on.
Thanks, Kurt. I admittedly have given puzzles ratings of every category: 1s, 2s, 3s, 4s, and 5s, when appropriate.
De-anna can't you ever let anything go??? :-)

It all comes down to this (to paraphrase Kristen's comment # 35): one person's rating of "x" is another person's rating of "y." For as much as people complain about the rating system, it is what it is.

Definition of average: A guy stands with a foot in a bucket of very hot water and the other in a bucket of very cold water and says that on the "average" he is quite comfortable.

If you look a room full of people who are all geniuses, the "average" IQ might be 170. This does not mean that they are "average" in the general population. There are a multitude of excellent puzzles on this site, so "average" means a pretty darn good puzzle.

by inverse, one might say: "there are a multitude of crappy puzzles as well, so average means crappy" ;) you're talking about 20,000 puzzles. sure there are lots of good ones. lots of bad ones too.

by def, 50% of the puzzles should be in the mean average. if you ask me, that should be around "3". Most puzzles here are rated to fall into a range of something more like 2-2.5(in the D to C- range). if the puzzles and creators are so good, then why is the mean average so low?

You've got to accentuate the positive,
And eliminate the negative,

http://www.youtube.com/watch?v=f3jdbFOidds

Totally disagree, Norma. I don't know how this devolved into what it did with personal stuff, again. Average=average. Kurt is correct that an average score would equal the rating for average on quality which is a 2 according to most raters. The scale states that it is a 3. All I am asking for is to treat the scale as it is. If Jan wishes to change average to a 2, then I am fine with a 2 on an "average quality" puzzle.

Gateway Time Out

Gateway Time Out

Gateway Time Out

For pity sakes. I was just trying to insert an encouraging word. Period. It's time to stop beating a dead horse with a sleeping dog.

Never trust sleeping dogs; they lie.

Tom.... let go of it. it's a damn rating. does it matter? ready for a story? it's that time of year again, and I'll not deny it. I have my own demons to work out. almost exactly one year ago, he came to me. you know what I did? I gave him a gun. my best friend in the whole world shot himself in the head. you're worried about some silly rating. I'm worried if I'm going to hell for the rest of my life. relax.

Tom.... let go of it. it's a damn rating. does it matter? ready for a story? it's that time of year again, and I'll not deny it. I have my own demons to work out. almost exactly one year ago, he came to me. you know what I did? I gave him a gun. my best friend in the whole world shot himself in the head. you're worried about some silly rating. I'm worried if I'm going to hell for the rest of my life. relax.

(((((((((kurt)))))))

Oh, Kurt. What an awful heart breaking pain that must be. Just a year ago. {{{HUGS}}}

That's heavy Kurt! I feel for you!

Why did you give him a gun?

damn Adam... do you ever know when to SHUT UP?

oh hell...got drunker than hell at darts a couple nights ago. shoulda never said any of that..... it was his own gun, Adam. I had an inkling of what was to come, so I took it from him. gave it back a few hours later though, after he insisted he was all right..... figured I couldn't watch him 24/7, and you don't need a gun to do the deed. still, I feel guilty as hell. I thought I came to peace with the whole thing months ago.... guess not. I suppose the timing has something to do with bringing it all back fresh. still have no idea why I'd say anything here. maybe cuz it's easier to tell a stranger cuz there's no repercussions whereas friends/fam will continue to bother you if they know you're not ok

Sometimes we speak in here thinking we are among friends ... in my case we are Kurt!

Still feeling for you!

Me too, Kurt

Ditto!

I'm glad....I still am not sure what possessed me to say, but in retrospect, lay off. I thought I got over, but anyone can see with my puzzles that I never did. a gun. send me an angel. not too hard to put 2 and 2 together. don't give Adam shit. I screamed aloud. didn't mean to, but I did. anyone with half a brain should assume I kept it hidden long enough and wanted to talk. this thing actually consumes me. Adam asked the logical question. I'm sure more than one of you thought it, but never had the guts. out of respect. and out of respect, I answered. I have no problem answering any questions. I'll do this for one more day, only cuz I brought it on. but I'd really rather try to let it go. I puzzle and keep busy to keep my mind occupied. I don't need reminders. even if it is my own fault that they come. I appreciate all the empathy. I do. but I gotta do this on my own

I can totally empathize with you because of a tragic occurrence in my life. I was helping a friend when I was distracted by an approaching neighbor I knew my friend didn't like and was going to tell the neighbor to come back when my friend was through with his project. When I turned back it was too late. I felt for years that if I hadn't been distracted I could have saved him. But now I'm not so sure. Perhaps it was meant to be. But time has gently eased the pain. And so it should be for you.

Quality Ratings:

	0
	14
	111
	252
	492
	1067
	1523
	1884
	2093
	2614
	2823
	2258
	1378
	802
	479
	219
	71

Those are the current statistics on webpbn quality ratings. Looks like a bell curve. Average is 2.76, so it tends a little lower than the "average is 3" concept, but it isn't wildly far away.

I'm not saying that the statistics aren't skewed, but in aggregate they don't look awful. Some of the correlations are a bit odd though:

Correlations:

Quality to Difficulty:	0.525
Quality to Cells:	0.479
Difficulty to Cells:	0.474
Quality to Lines:	0.540
Difficulty to Lines:	0.529
Quality to Colors:	-0.020
Difficulty Colors:	-0.028

A correlation of 1 would mean that the two things always go up and down together. -1 would mean that they always go in opposite directions from each other. 0 means there is no connection at all.

So difficulty, quality, and puzzle size all seem to go up together. Which is weird, because you can obviously make wonderful puzzles at all sizes and difficulty levels. On the other hand, the correlation is only around .5, so there is a tendency for them to go together, but it isn't any kind of iron-clad rule.

So if you are hungry for high ratings, do huge difficult puzzles with pretty pictures that only a few people will solve, and your chances improve.

Personally, I think it's cooler to do puzzles that lots of people play, which are going to be smaller, easier puzzles. I like getting lots of comments. This puzzle one of the site's champion comment collectors.

Jan, I've missed your comments and am so happy to have you here with us again. Please come back often. And thank you for everything you do for us!

Interesting stats, Jan. The dearth of 5 ratings in quality probably skews it down that 1/4%. How about the stats on difficulty? Is it an identical bell curve? To reiterate, thanks for this site ;)

It is now 2016. I got to this puzzle through the random puzzle generator. I agree that this puzzle is a 1 in quality. It was also easy to solve. The solvability was fine, but I did not understand such a low-quality puzzle from one of the excellent makers, so I read the comments.

I try to be fair when I rate puzzles. Difficulty is easy for me to rate now, as I go by how much I had to think to do the puzzle. If I had to clear parts (or all) of the board to try again, or if it was a fill-in, I am able to understand the rating system. I had stopped rating difficulty for a while, since I realized that I had learned to recognize patterns in the numbers, which made it easier for me to solve puzzles that others might find very difficult.

Quality, for me, is a conglomeration of parts. I look at the overall puzzle, the title, and the story. An average puzzle can get an above average rating from me if it has a good title and story. However, bad title and story will not make an excellent puzzle less than excellent.

Show: Spoilers

Goto next topic

You must register and log in to be able to participate in this discussion.