The Dons boss and assistant Archie Knox are out of contract in the summer and Aberdeen vice-chairman George Yule has insisted talks are on hold until their top-half fate is decided.
The Dons have won just twice in their last 10 SPL games - both against relegation certainties Dundee - and feebly limped out of the cup to Hibs.
But Brown insists he can still deliver success at the club.
He told the Daily Record: “It is up to the board if I am given the time to get success.
“But it would be unfair if I wasn’t because it takes time and this is a work in progress – and it’s very nearly there.
“I am very optimistic we can still bring success to Aberdeen. It’s not all doom and gloom.
“But it’s an over-simplification to say change the manager. The fans and directors may say that but that is up to them.
“Changing a manager is not a magic wand. I love this football club and if they don’t think this is for me I’d be disappointed.”