During school holidays, Jack, his sister Lucy-Ann, and their parrot Kiki go to stay with their new friends, Philip and Dinah, in their isolated old house, "Craggy-Tops", set on a steep cliff on the coast. The house is owned by Phillip and Dinah's uncle Jocelyn (a focused historian) and overworked aunt Polly, who are helped by a handyman named Joe.

Joe warns the boys that, from their tower room in Craggy-Tops, they can occasionally see the dangerous Isle of Gloom, though it is usually shrouded in fog and mist. Soon the children meet Bill Smugs who shares Jack's fascination with birds. The children sail out to the Isle of Gloom and eventually learn that Joe is working with men who are counterfeiting money in the old mines on the island.