The XY Problem and the Five Whys

Have you ever heard of the XY problem?

What is it?
The XY problem is asking about your attempted solution rather than your actual problem.
That is, you are trying to solve problem X, and you think solution Y would work, but instead of asking about X when you run into trouble, you ask about Y. 

The Problem
This can lead to frustration by people who are trying to help you solve the problem because by the time you ask about it, the solution that you need help with might not have any obvious connections to the problem that you are trying to solve.

It comes up a lot, and it's something you should try to recognize both in yourself and in others. I recently encountered it with a coworker of mine:

Coworker: Have you used the HtmlAgilityPack?
Ed: a little
Coworker: I can't get it to work.
Ed: what part of it
Coworker: Sitecore apparently has it included?
Ed: it does
Coworker: But I can't reference it.

I was perplexed at this point. HTMLAgilityPack is an assembly included by default in Sitecore installations. In fact, Sitecore itself relies upon it and will not work without it. So I probed a little deeper:

Ed: the project should already include a reference to it
Ed: ok
Ed: so, when you reference it with the using statement, its just not available?
Coworker: Oh.
Coworker: It looks like the sample code I got needs a later version?

Now we're getting somewhere. The problem wasn't that my coworker couldn't get the HTMLAgilityPack to work. Instead it was that he had the wrong version of the assembly. We then thought about upgrading the version of the assembly, but before we did, something struck my mind:

Ed: do you need the latest HTML agility pack? 
Coworker: Probably.
Coworker: At least, the one included doesn't have methods I need.
Ed: what are you trying to do
Coworker: So, I'm trying to take a substring of content, to display in a "Featured Pages" section.
Coworker: But if there are any tags that open in the substring, but close after, the formatting breaks.

Now we're really getting somewhere! We now were both on the same page about what he was trying to do (already a far cry from "I can't get the HTMLAgilityPack to work"). From here it was just a hop skip and jump away from monkey-patching in the method he needed from the newer assembly (for those with a desperate need of closure, the method was "Descendants()"). 

This whole exchange reminded me of the Five Whys:

To reach this sweet spot, we borrowed an idea from Sakichi Toyoda, the founder of Toyota. He calls it Five Whys. When something goes wrong, you ask why, again and again, until you ferret out the root cause. Then you fix the root cause, not the symptoms.

This is basically what the XY problem boils down to, a lack of finding the root cause/problem. However, simply asking "Why" won't really get you to the proper solution in the XY problem case. Employing pure "Five Whys" in this situation would not have gone so well:

• I can't get the HTMLAgilityPack to work. 
• Why? It won't compile.
• Why? The compiler says the methods I need from sample code I found aren't there.
• Why? The sample code was using a different version of the HTMLAgilityPack. 
• Why? I don't know, man, ask the author of the sample code!
• Why? I can't read the man's mind! I don't even know him!

I find it helps to modify the question from "Why?" to "Why is that important?" or "How so?" or "What do you really mean?" when just asking why wouldn't work. . 

To summarize my interaction with my coworker in my modified "Five Whys" method (call it the XY5Y method) would look like this:

• I can't get the HTMLAgilityPack to work. 
• Why? It won't compile.
• Why? The compiler says the methods I need from sample code I found aren't there.
• Why? The sample code was using a different version of the HTMLAgilityPack. 
• How do we fix this? Let's upgrade the version of the HTMLAgilityPack.
• How would that solve the problem? It would give me the methods I need to solve my problem.
• Are the methods all you need? Well, yes...
• Is there another way to get the methods? We could disassemble the newer version of the HTMLAgilityPack and monkey-patch in the methods we need
• Profit!

If you find yourself working with a new programming language or framework (or really a new anything), make sure to question yourself (or your comrade) to find what the real problem is.