It followed a report to police that he walked into the emergency department with a gun on Tuesday night.
According to sources, patients watched in the reception area watched as hospital security and the police spoke to the man and persuaded him to hand over the firearm.
It is understood the 26-year-old asked the receptionist for help before making his way into the public toilets.
The Belfast Health Trust said the man did not threaten staff.
A spokesperson said: “Security staff on duty were on the scene within minutes.”
Armed PSNI response teams were alerted to the incident which is normal procedure.
Area police chief Inspector Paul Noble said: “A man has been arrested on suspicion of possession of a firearm in suspicious circumstances and currently remains in custody.”
He is being questioned by detectives at Musgrave PSNI station under the Firearms (NI) Order.