(To prevent any 'mar'is ayin,' this post was posted in America, and while it is already Shabbos in Israel, it is not yet Shabbos in the US.)
The gemara introduces us to the concept of hezek re'iyah. At the simplest level, this means that it is possible to cause damage to another with one's vision. The question is, What is the damage that is being discussed here?
I understood that there are at least two different types of damage spoken of here. One is that there is a damage that can be caused by simply looking with an 'ayin hara' - evil eye - at someone elses property. Another understanding is that there is a damage that is caused when a person can not go about his private affairs because I have visual access to his domain.
It is significant that the Gemara, as it continues onto daf gimel, gives two different assumptions as to whether or not the responsibility for preventing this type of damage is upon the neighbor who 'sees' or it is upon the one who is being looked at. The first assumption of the Gemara is that 'hezek re'iyah lav shmei hezek' - damaging sight is not considered a damage. This would mean that the 'offender' who is seeing is not the problem, but rather it is the problem of the one being seen. The question is, Why? The second section of the gemara assumes the opposite, that in fact this type of damaging sight is indeed a significant damage and is the responsibility of the one who is looking. Again, the question is, Why?
If we try to understand more deeply the two different explanations we gave earlier, we will see that the difference between them will indeed be upon whom the responsibility will be placed.
If we understand that the damage here is based on the concept of ayin hara - evil eye - then we need to understand what is an evil eye? The concept of the evil eye is that if someone (we will call him Reuven) looks jealously at another's property (we will call him Shimon), it brings about a question in the heavenly court as to whether or not the one who owns it really deserves it. Thus they look at the records for Shimon and if he is indeed unworthy, he may lose it because of Reuven's negative look.
Of course, this begs for explanation. Why should Reuven lose something because of Shimon's bad thoughts? If anything, we should judge Reuven the negative thinker, not Shimon! The answer is that there is a responsibility on Shimon to be tzanuah - to be modest and not show off the good things that he has. The negative thought that Reuven has is a result of Shimon's showy way of life, and thus, Shimon is judged, for his lack of modesty has brought about a negative thought on the part of Reuven.
Once we understand this, we can understand why hezek re'iyah is not considered the responsibility of the one who is looking. It is the problem of the one who is being looked at! Therefore, he is the one who must build a wall to protect himself.
In the other understanding, however, the damage that is being caused is because the one who is being looked at (again Shimon) can not go about his private affairs because there is someone (again Reuven) 'intruding' with his gaze. This limitation is solely the responsibility of the gazer, and thus we would say the concept that hezek re'iyah is indeed considered a damage, and the responsibility is upon the gazer to prevent himself from damaging and to therefore build a wall.