You tell me that there are trees out there, but can you tell me if they’re real? They could be very good wax imitations and you’d never know, would you?
If I ask you to tell me your image of Japan you’ll probably mention concrete, grey, lots of buildings, etc. If you even mention the trees you’d probably mention the lack thereof or lament the lack of yards in Japanese society. Given the locales of the general populace and the average population density, if you never visit Japan I can’t really blame you for your image of Japan (as long as you don’t blame other people in the same situation). Nevertheless, this image of Japan is incorrect. To fix this I’ll push your nose into a picture of some trees and yell at yourself until you put your tail in between your legs and hide under the bed.
Every country has some percentage of forest cover. What do you think Japan’s is? Most people would find it surprising that Japan’s forest cover is 68.2%; this puts them as the 12th most forested country in the world. So where is this forest cover as you can’t find it in Kanto or much of Kansai. Well, for the most part it’s everywhere else.

Neo neo neo neo neo neo Tokyo, the setting for another anime with big robots.
As you should know, Japan is on the edge of a couple tectonic plates. Volcanoes and land shifts tend to happen at the edges of these plates, which means that landmasses in these areas are going to be moire mountainous; Japan is one of these regions. How mountainous is Japan? 73%. But mountainous doesn’t mean you can’t build on it (there are many mountain temples in Japan) and it doesn’t mean tree-covered. There’s more to it than a country that concentrates its population on coastal plains.
Aokigahara (青木ヶ原) is the forest closest to world famous Mt. Fuji. Besides being the place where most Japanese go to kill themselves this forest, like the mountain, is sacred and legends say that the forest is inhabited by demons and other spirits. What this means is that the forest is not logged and there probably won’t be any housing plans there any time soon. And just like this forest, there are many hilly/mountainous places where one could have built a house, castle, fortress, something, and yet, in most cases there’s nothing but forest. These areas were (and some still are) sacred and building on or near them could cause trouble with a deity, spirit, or demon. They might not be sacred anymore but since everyone is already concentrated in certain areas there’s not much of a drive to build on these lands.

Yay trees! Where’s the nearest convenience store, I’m thirsty for Calpis.
So what do you get when you mostly build on coastal plains, stray away from building on mountains/large hills, and then get a population of 120 million? Well, you know. If you only go to Japan on vacation that means that you’ll probably go to Kyoto, Tokyo, and maybe Nara. So when you travel from Kyoto to Tokyo (or vice-versa) you’re probably going to take the Shinkansen. If you’ve ever ridden this part of the Shinkansen in the daytime you’ll know that it mostly follows close to the coastline, which means two things: it will be close to towns/cities and (consequently) you won’t see much forest or many trees. When you combine what people consider to be Japan and you give them a little experience that seems to validate their point you get people saying that Japan is one giant city (i.e. no forests).
