I know that there is more to this island than most people know, or care to tell me. People are tight-lipped about the past. The tower and all of the house pods sprouting around it were grown quite recently. I suspect that no one alive is actually old enough to remember or even have heard about the things I need to know.
There are ancient tales of how the Aldmeri explorers charted the lands of Tamriel, including Zafirbel Bay. After them Aldmeri settlers came as well and they built towers. These towers were nothing like our own, grown from organic materials, but stone solid enough to remain to this day in some places. One was built here in Tel Aruhn, also known as the Old Tower Home. It seems to have vanished, but I am nearly convinced that the old tower is not entirely gone.
The steep wall looming high above the docks could be mistaken for a natural formation, but I believe that it is in fact a construction, but so covered with moss and worn down by the elements that it is near impossible to tell. Maybe it was a protective wall around the tower, or even the actual tower wall?
The Dwemer were here before the Aldmeri wizards came from the sea of course, but as far as I can tell there was no incentive for conflict. One people favored the inland while the other settled along the coast. They were experts of their own realms and kept mostly to themselves. There must have been respect or even appreciation between the races rather than rivalry.
Neither was related to the primitive tribesmen who came much later. They needed Azura's help to teach them civilization. The Aldmer and the Dwemer did just fine without divine intervention.
We are Telvanni, the tower dwelling wizards. How related are we to the Chimer? Perhaps the outcast Mabrigash and necromancers who shunned the ways of their tribes were absorbed by the wizard communities along the coast. Over generations of contact, the lines blurred and we came to be seen as another Great House of the Dunmer.
When Azura cast her curse, she probably did not check very carefully who was who. Gold turned to ash and eyes that were once any color under the sky turned red as the blood that was spilled on the slopes of Red Mountain.