Web Paint-by-Number Forum
Topic #215: Thinking About Revamping Solvability Tags
By Jan Wolter (jan)

#1: Jan Wolter (jan) on Mar 25, 2010

I'm thinking about doing some fine tuning on the way puzzles are tagged for solvability on this site.

Currently we tag whether the puzzle has a unique solution or multiple solutions. This is almost always automatically done by my solving program, and is uncontroversial.

We also tag whether the puzzle is "logically solvable", "requires some guessing" or "requires much guessing." This is more of a problem, partly because the meaning of those phrases is not obvious, though fairly clear definitions have evolved over time, and partly because as the collective puzzle solving acumen of the webpbn users has evolved, the old terms don't really fit our needs any more.

So I'm thinking of changing the categories, maybe something like this:

multiple-solutions, some guessing
All puzzles that have multiple solutions require at least some guessing. This category covers puzzles that have multiple solutions, but for which it is not particularly difficult to come up with the right solution, either because there are just a few possibilities, or because the picture is recognizable and you can easily see how things have to go. Puzzles like this can still be enjoyable to many users, but purists tend to hate them.
multiple-solutions, much guessing
For puzzles with multiple solutions, and for which it is absurdly difficult to find the correct solution. Puzzles in this category are strong candidates for deletion from the site, not being fun for anybody.
unique-solution, line/color solvable
These are puzzles solvable entirely by working one line at a time, if the puzzle is multicolor, with color logic. No edge-logic, either-or logic or anything else fancy is needed. These are the puzzles that my solving program already automatically marks as "logically solvable" under the current scheme.
unique-solution, some lookahead required
These are the puzzles that require edge logic or other advanced logic that is nevertheless pretty doable by ordinary mortals without a lot of undoing. Currently these puzzles are marked logically solvable by me or Gator ruling them to be so. The distinction between this and the previous class currently exists on the system, as you can tell by looking at the comments where a "logically solvable" rating came from, but I think it would be good to make it a bit more explicit. However, as will be discussed below, I don't want to be really in-your-face about this distinction. I think solvers should be allowed to discover whether they need edge logic or not, and not always be told up front. Note that I'm not being really explicit about how much lookahead is "some". I think allowing some flexibility in that is best, as what is reasonable depends a lot on the situation.
unique solution, much lookahead required
These are puzzles for which people are able to discover logical solving techniques, but they are techniques that require either some trial and error, or a really extraordinary capability to visualize many moves ahead, beyond what we would ordinary expect from a casual solver. Brain burners that most people would solve by guessing. Currently these fall in the "some guessing required" category, though that doesn't really do them proper justice.
unique solution, some guessing required
These are puzzles that nobody has found a logical solution path for, but which can be solved by resorting to symmetry or "picture logic". For most casual solvers, especially those who don't mind guessing when it is obvious how things are going to go, these are pretty much the same as the previous class.
unique solution, much guessing required
These are puzzles which are known to have a unique solution, but for which no logical solution path has been found and most people can't get to the solution even if they make guesses. These puzzles are candidates for deletion.

On the puzzle list pages, we currently flag some classes of puzzles with question marks. This would be modified slightly as follows:

No Mark unique solutionline/color solvable
No Mark unique solutionsome lookahead
! unique solutionmuch lookahead
? unique solutionsome guessing
? unique solutionmuch guessing
? multiple solutionssome guessing
? multiple solutionsmuch guessing
Note that we don't distinguish "line solvable" from "some lookahead" on the puzzle list page. If you want one or the other, you'll be able to do a search or use the random puzzle page to get just puzzles of that type, but normally you won't be told up front if you need to be looking beyond line solving or not. So the puzzle tags under the title on the solving page won't make this distinction either, but the tags under the title on the comment page will. So if you really want to know which category a puzzle you are working on has been placed in, you can just pop up the comment page. I don't think this will be terribly hard to implement. Not sure I have time to implement it, but I wanted to see how people felt about the idea.
#2: Gator (Gator) on Mar 25, 2010
Could an option be added to the user settings to always show a "some lookahead" tag in the puzzle list? I would imagine some people would want to know before they start a puzzle if they are going to be getting into advanced logic territory, while others (like me) would not want to know. If it would be hard to implement, I would say don't worry about it through.

One other thing I would like added to the search capability would be to find puzzles that do not have a definite rating on solvability. This way it would be easier to find puzzles that need to be ruled on.

I think better tagging of puzzles is a great idea of course. :)
#3: Jan Wolter (jan) on Mar 25, 2010
I think having an option for that wouldn't work well. If I had that option, then I think it would be natural to have it automatically turned ON for new users, because that would be the set of people most likely to want it. But if I did that, then it would probably never get turned off, and I feel that it really is better to have it turned off.

If I left it turned OFF by default for new users, then it would probably never get turned on, since the user probably wouldn't find the option until they had solved enough puzzles that they had gotten the idea of edge logic.

I think the way people learn about edge logic on the site now is probably OK. They get stuck, read the comments. Learn something new. Someday I'd like to do a better hinting tool that would do this better, but that's a separate project.

Hmm...I actually have a page that gives me a list of puzzles that need rulings. I haven't looked at it for a long time. I should try to figure some way to give you access to that or some equivalent.
#4: Jan Wolter (jan) on Mar 25, 2010
One thing that makes this change a bit complex to do is that the voting would have to change. Currently, if no ruling has been made on a puzzle, the value displayed is an average of people's ratings of (1=logical, 2=some guessing, 3=much guessing). So if 3 people say logical and 2 say much guessing, then the rating is probably going to be "some guessing".

That's a little stupid now, and would be a lot stupid if we averaged "logical" with "much guessing" to get "some lookahead". So instead of averaging we need to make it a vote. Whichever category gets the most votes is the winner. So 3 people saying "logical" and 2 saying "much guessing" means "logical" not something in between. But that's a significant change to the algorithm and requires changes to the database so the values can be computed incrementally. Not hugely difficult, but a pain.
#5: Teresa K (fasstar) on Mar 25, 2010
My opinion: It would be nice to see "some look ahead."

Give Gator access to the puzzles that need rulings.

Everything else doesn't matter enough to me to put you to the trouble of reprogramming.

Except for one thing: I would love to be able to search the description field.
#6: Wombat (wombatilim) on Mar 28, 2010
I like the idea of this new system, assuming that implementation isn't too time-consuming.
#7: Josh Greifer (joshgreifer) on Apr 4, 2010
After reading these posts I made up puzzle 8274, which I think should be classed as logically solvable, but the logic used to solve it is a kind of inductive reasoning: Unlike a computer program, we can notice patterns, which means we don't have to try all possibilities.

I'd like it so that puzzles like 8274 were classed as logically solvable -- because they are! That's something only a human can decide in general, I think. Which is why I like the fact that we have humans on here judging and commenting on the puzzles.
#8: Jan Wolter (jan) on Apr 8, 2010
I agree that these judgments cannot be made 100% rigid, nor can they be fully automated.

I'd play your puzzle, and others that require "whole puzzle" type reasoning, like some of the "summing" examples in the advanced puzzles solving notes, in the "logically solvable with some lookahead" category. It isn't really a good fit, since the logic used isn't really "lookahead" but the point is that though the puzzle is not line solvable, it is solvable with a modest amount of in-head visualization.

But the distinction between "some" and "much" is always going to be a judgment call, and a lot of discussion among solvers, so that it is based on a community consensus.
#9: Jan Wolter (jan) on Apr 8, 2010
I'm thinking of adding one more category to the list, for "trivial" puzzles. The ones with little or no white space that are so easy that they are hardly puzzles at all.

I've actually started some of the coding needed to implement this. A bit of a pain, since it requires small changes in many dozens of places.
#10: Kadou (Kadou) on Nov 18, 2010
I think we can distinguish between line/color solvable,
some lookahead, much lookahead by looking at the difficulty rating.

What I don't get is guessing on a puzzle with a unique solution. If it has a unique solution then it is solvable using logic. The guessing on this type of puzzle is more likely the difficulty of the advanced level.

I think the "?" should only be for puzzles with multiple solutions.
While the "!" could be for puzzles with a unique solution that haven't been tested to be logically solvable yet. Maybe the other (red or black) "!" could be for tagging the very advanced puzzles which have been tested.

Goto next topic

You must register and log in to be able to participate in this discussion.