The National Farmers Federation and other groups have been urging the federal government to introduce a special agricultural visa, which would allow a greater flow of foreign workers for jobs such as fruit picking and packing.

Rural industries have found local workers are not attracted to short-term and seasonal work and they instead heavily rely on foreign labour.

It's been estimated as many as 30,000 extra workers are needed each year.

However, Agriculture Minister David Littleproud has been forced to go back to the drawing board after cabinet colleagues shot down the idea of a special agricultural visa early last week.

It is understood Mr Littleproud is now working on a plan which would tweak the existing Pacific island labour program while also encouraging more Australians to take up farm jobs.

Foreign Minister Marise Payne told cabinet the Pacific program was central to Australia's efforts to deal with Chinese influence in the region, and would be put at risk by the plan.

Home Affairs Minister Peter Dutton voiced concerns a new visa could open up a new way for illegal immigrants to enter the country.

Asked about the discussion on Wednesday, Finance Minister Mathias Cormann said he didn't want to "get into cabinet deliberations".

But he emphasised the government was keen to maintain strong international relationships and current arrangements were popular with Australia's Pacific neighbours.

"But by the same token, we are also very conscious of the fact that there is a need across the agricultural sector for more workers," he told Sky News on Wednesday.

Labor says a new plan would have to be assessed in line with the seasonal program for Pacific countries.

Opposition treasury spokesman Chris Bowen told reporters in Townsville the region had high levels of unemployment, and locals would want to be considered for farm work over foreigners on 457 visas.

"457 visas play a role, but that's got to be where there are genuine shortages, where businesses have tried and failed to get their jobs filled, he said on Wednesday.

"But until and when that is the case, it's just not what the system is designed for."

Australian Associated Press