With Arsene Wenger serving the first game of a three-match touchline ban, Bould took over for the first time and saw the side battle back from conceding an early penalty.
Thomas Vermaelen gave away the spot kick with just nine minutes on the clock after a clumsy challenge on Younes Belhanda, although replays suggested the foul was probably just outside the box.
The player chipped home his penalty to take the lead, but the English Premier League club promptly replied with a fine finish by German summer signing Lukas Podolski.
Before the French champions, making their debut in the competition, could recover, Arsenal swept ahead on 18 minutes after Gervinho tipped home a neat Carl Jenkinson cross.
Although the hosts rallied after the break, with lively winger Remi Cabella hitting the crossbar and Belhanda missing a great chance late on when he fired straight at stand-in goalkeeper Vito Mannone, the Gunners closed out a battling victory.
Bould said afterwards: "I am not sure I enjoyed it. I have got a headache and a half.
"It is tough, a different pressure than I have been used to taking the Under-18s - but we won and we had a great result."
"It is quite an intimidating place to come and play so we are glad to get the result.
"We have had a decent start to the season and we have kept the run going so it's important all round, for the whole season.
"We have got great team spirit and I think that was evident tonight. Everybody is digging in and things are looking OK."
Podolski netted his third goal in as many games tonight, with a neat finish after quick passing on the edge of the penalty area.
Bould said: "Podolski has made a real impact. I have to say I've not seen many finishers as good as him - ever, right foot, left foot, they always go in at 100 miles per hour in training.
"He is a good trainer, a good boy and has really added something this year."
It was, though, another barren evening for Olivier Giroud, the £12m (Dh71.4m) summer signing from Montpellier.
However, Bould insisted there was no cause for concern.
"Olivier did a really good team job for us tonight. He works really, really hard for the team - at set pieces he is fabulous," he said.
"He had one chance at the near post that was difficult. Olivier is a really hard-working boy and I am sure the goals will come."
Montpellier coach Rene Girard felt his team could take some positives out of their Champions League debut.
"I have some regrets but also some satisfaction," he said.
"Maybe we were a bit too shy in the first half, then looked more determined.
"We had a few chances to equalise, but at this level when you make some mistakes, you will pay for it.
"We can be proud, because apart from these two minutes we managed to play well against this top European team like Arsenal, so there
Follow us @SprtNationalUAE