They bounced back from the 24-19 opening-day home defeat by Northampton to come from behind three times.
England number eight Ben Morgan, with his first try for his new club, winger James Simpson-Daniel and replacement flanker Akapusi Qera, touched down for Gloucester with Burns landing seven penalties and two conversions.
Right-wing Topsy Ojo scored the Exiles' opening try with Ireland scrum-half Tomas O'Leary and replacement prop Halani Aulika marking their home debuts with tries. Fly-half Steven Shingler kicked four penalties and two conversions for a 16-point haul.
Irish took the lead with a fourth-minute penalty from Shingler - handed the kicking duties ahead of last season's top Premiership points scorer Tom Homer who began with his left knee heavily strapped.
But the visitors equalised through Burns five minutes later when Irish strayed offside on their 22 line.
Shingler kicked the Exiles ahead with his second penalty in the 19th minute and two minutes later the home side grabbed the opening try.
Shingler figured twice in the build-up before releasing Homer, who put his grubber kick behind the Gloucester defence before Ojo won the race for the loose ball.
Burns cut the gap to two points with two penalties before Shingler responded from two metres inside the Gloucester half after the visitors were penalised for pulling down line-out jumper Bryn Evans.
Irish withdrew the struggling Homer in the 34th minute, replacing him with England Under-20 star Anthony Watson.
Burns pulled his fourth penalty attempt wide four minutes later but landed a conversion to send Gloucester in at half-time with a 16-14 lead after Morgan exploited some appalling defending by Irish to score his side's first try.
Taking the ball from the base of a ruck, he went between two hesitant would-be tacklers, was missed by hooker Scott Lawson and brushed off O'Leary before crossing.
Burns landed his fourth penalty five minutes into the second half after the Irish scrum was yet again penalised by referee Greg Garner.
Irish were level again two minutes later when Gloucester's defence cracked, allowing number eight John Fisher to run through before offloading to Aulika, who forced his way over for his first Exiles try since arriving from Championship side Leeds during the summer. Shingler converted to put the Exiles 21-19 ahead.
But Gloucester edged a point in front with Burns' fifth penalty from just inside the Irish half as the home scrum, forced into rapid reverse, was penalised once more.
Irish finally got their scrum right and it led to O'Leary shooting in for their second try after the diving Charlie Sharples conceded the five-metre set-piece as he slid over the line desperately defending a through kick from Jonathan Joseph. Shingler again converted.
Burns kicked Gloucester level again after Irish offended at the breakdown.
Shingler booted Irish back in front before Burns squeezed down the left touchline for Gloucester before passing inside to present Simpson-Daniel with what proved to be the decisive 72nd-minute try, converted by the fly-half.
Gloucester struck again two minutes later when Qera forced his way in from close range.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The entire All Blacks apparel line has been updated for 2013/14. Check out the New Zealand All Blacks polo.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Gilbert Blitz 8S rugby cleat is a great cleat at a great price of $69.99. Get a new pair of cleats today.
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
$29.99
The All Blacks Performance t-shirt is black with hints of blue from the training jersey. Very Cool.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.