In the 6pm incident, the seven men were on their way from Jerejak Island to North Beach, transporting rocks to be used as wave barriers.
As of 10.30pm, six men had been rescued by authorities, while another man, a 60-year-old Malaysian, is still missing.
Five men, aged between 26 and 30, are from Indonesia, while two other locals, aged 60 and 72, were on the boat.
The 72-year-old man from Ipoh, believed to be their boss, survived, while the 60-year-old is still missing.
Quarry worker Hamdan Osman, 35, said he and three others held on to a sea marker while three others swam helplessly for nearly two hours.
He said they then waved at a vessel on its way to the port, which in turn alerted maritime authorities for help.
“The waves were so strong. The boat flipped and we were hanging on to its perut (belly).
“Then the waves got stronger and we lost our grip. Luckily, we spotted a giant red ball (maritime sea marker). Three of us hung on to the ball, while another three of us hung on to a drum.
“Thankfully, the men from the Bomba arrived and fished us out. We did not know how long more we could have hung on if we had not been rescued,” said Hamdan, who is from Aceh, Indonesia.
Meanwhile, Beach Street Fire and Rescue Department Operations Chief Samsudin Ahmad said the men were involved in the Gurney Wharf reclamation works.
He said a distress call was received at 9.05pm.
“The seven men had gone to Jerejak Island to survey for stones. They told us they were on their way back to Gurney Drive when their boat flipped some 500m from North Beach.
“Only one of them was wearing a life jacket.
“As of now, we have rescued 4 Indonesian workers and two locals. Another man, in his 60s, a local, is believed to have drowned.”
Samsudin said the operations would carry on to find the man, with the assistance of marine police and the boat unit from the Bagan Jermal Fire and Rescue Department.