On Thursday, Taos County Sheriff Jerry Hogrefe issued a search warrant for the compound, described as an earthen berm in a local subdivision was allegedly occupied by Morten and Siraj Wahhaj.
A group of 11 children aged between one year old and 15 years have been rescued from a dilapidated compound where they were being held by armed men. Three women, believed to be the children's mothers, were detained and later released.
The discovery was prompted by a months-long search for a three-year-old child, Abdul-Ghani Wahhaj, who went missing in Atlanta, Georgia, in November.
After investigating, multiple agencies raided the compound Friday morning.
Taos County Sheriff Jerry Hogrefe said the search of the Amalia dwelling, which is just a few minutes south of the Colorado border, was the result of a two-month investigation conducted with the FBI and Clayton County Police Department into the whereabouts of the boy, who turns 4 on August 6. Deputies reported one of the men being heavily armed with an AR-15, five 30-round magazines and four loaded pistols on his person when he was taken into custody.
"We all gave the kids our water and what snacks we had", Hogrefe said.
Police found many more ammo rounds in the ramshackle hideout, they said, which they described as "a small travel trailer buried in the ground covered by plastic with no water, plumbing, or electricity".
The search for the boy, AG Wahhaj, is ongoing, the sheriff said. The warrant could not be obtained until it was eventually sparked by a note sent from inside the compound, which ultimately reached a Georgia detective reading "We are starving and need food and water".
Police got to them after intercepting a message from either them or one of their mothers which read: 'We are starving and we need food and water'.
Both Wahhaj and Morten were taken into custody, but Abdul-GhaniWahhaj has still not been located.
Officials said they believed little Abdul-Ghani Wahhaj had been there several weeks before, and are continuing to search for the boy's whereabouts.
He says 35-year-old Jany Leveille, 38-year-old Hujrah Wahhaj and 35-year-old Subhannah Wahha were arrested without incident in the town of Taos and booked into jail. On Monday, the boy's fourth birthday, investigators were scrambling to find him - and to understand what happened at the New Mexico compound.
The boy's mother told police her child, who she said suffered from seizures along with development and cognitive delays, went to the park with his father Mr Wahhaj last December and never returned. The station said the vehicle was registered to Morten and that according to policed five additional children and two adults were traveling with Wahhaj and his son.
"While Hogrefe said in his press release the Federal Bureau of Investigation "didn't feel there was enough probable cause" to enter the property", the sheriff said "that all changed" for him when a Clayton County Police Detective relayed a distress message believed to have come from within the compound.
Police reports show that a Georgia boy missing after authorities raided a New Mexico compound over the weekend was last seen in Alabama in December.