One must have the belief that Allah Most High is pure from space and time. It is wrong to say that He is in the Kaaba and it is also wrong to believe that He is on something, as all these are limited whereas Allah Almighty is limitless. However, we must believe that His knowledge encompasses everything, and he knows, sees and listens to everything.
Mosques are also regarded as houses of Allah. In a similar way as Muslims worship in the direction of the Kaaba we say the Kaaba is also the house of Allah.
Allah is free from space and time therefore he is not in those places physically. Allah does not have a physical form but Allah has blessed those place where his worship is taking place hence those places are called the "houses of Allah".
Why is Kaaba so special in Islam?
Muslims revere the Kaaba because God tells us of its special status in the Qur'an.
The Kaaba was the first house where the worship of Allah took place.
"Verily, the first house appointed for men is the one at Bakka, blessed and a guidance for the nations. In it are clear signs, the standing place of Ibrahim, and whoever enters it shall be secure, and pilgrimage to the House is incumbent upon men for the sake of Allah, (upon) every one who is able to undertake the journey to it..." (Qur'an, 3:96-97)
Abu Dhar asked,"O Allah's Messenger! Which mosque was first built on the surface of the earth?" He said, "The Sacred Mosque (in Makkah)." I said, "Which was built next?" He replied "The mosque of Al-Aqsa (in Jerusalem) ." I said, "What was the period of construction between the two?" He said, "Forty years." [Narrated by Bukhari]
The building of the Ka`bah by Prophet Abraham (peace and blessings be upon him) is referred to in Qur'an, 2:125-129 and 22:26-33, along with some related points, and you are encouraged to read these passages.
The Kaaba also symbolizes the unity of Muslims, as they are all commanded to face towards it in prayer (Qur'an, 2:144, 149, 150). It should be noted that Muslims do not worship the Ka`bah; it is merely a direction in which we are ordered to face during the ritual prayer. Muslims worship only God. Some Muslim jurists have discussed that if (hypothetically) the structure of the Kaaba were to be moved to somewhere else, Muslims would continue to pray to the vacant site of the original Ka`bah, regardless of whether there is a building there or not. Indeed, the building itself has been rebuilt several times over history.