The Slonimer Rebbe explains in Nesivos Shalom the meaning of the blessing when Yosef's father Yaakov blesses him with the words "Ben Poras Yosef, Ben poras alei ayin, bnos tzoados alei shur," ("A charming son is Yosef, charming on to look upon; young women climb walls to gaze at him).
He says that according to the Midrash, the young women would even fight for a place to get a good view of Yosef when he would pass by. And that they would throw jewelry at him to get his attention so that he would look at them. And he never did. He explains that Yaakov was praising Yosef for this midah, that he always guarded his eyes from seeing that which they shouldn't see. And that this is in accordance with his general trait, which is as the merkava for the mida of Yesod.
But he points out that Yosef withstood much greater trials with the wife of Potifar, and passed. Why is his father praising him for this aspect of that trait of purity?
He answers by pointing out the well known expression that "The eyes see, the heart covets, and the body sins." The eyes are the gateway to the whole person. When a person allows his eyes to serve merely as tools for his own enjoyment, then he allows the Sitra Achra, the other side, to rule over him. But one who guards over his eyes, only to look at things for purposes of Kedusha, then holiness rules over his life, rather than the opposite. This is why the pasuk says about Yosef, "Hu hashalit, hu hamashbir." He was the ruler and the sustainer (of the people of Egypt). He was the ruler over himself and that is why he merited to be the ruler over the nation of Egypt.
The fact that the way one watches his eyes is the key to his entire success in the middah of Yesod, in shmiras habris, is why Yosef's conduct with his eyes is his main reason for praise by his father Yaakov. And that is why Yaakov focuses more on praising Yosef in that area, rather than for his success in the test with the wife of Potifar.