They bounced back from the 24-19 opening-day home defeat by Northampton to come from behind three times.
England number eight Ben Morgan, with his first try for his new club, winger James Simpson-Daniel and replacement flanker Akapusi Qera, touched down for Gloucester with Burns landing seven penalties and two conversions.
Right-wing Topsy Ojo scored the Exiles' opening try with Ireland scrum-half Tomas O'Leary and replacement prop Halani Aulika marking their home debuts with tries. Fly-half Steven Shingler kicked four penalties and two conversions for a 16-point haul.
Irish took the lead with a fourth-minute penalty from Shingler - handed the kicking duties ahead of last season's top Premiership points scorer Tom Homer who began with his left knee heavily strapped.
But the visitors equalised through Burns five minutes later when Irish strayed offside on their 22 line.
Shingler kicked the Exiles ahead with his second penalty in the 19th minute and two minutes later the home side grabbed the opening try.
Shingler figured twice in the build-up before releasing Homer, who put his grubber kick behind the Gloucester defence before Ojo won the race for the loose ball.
Burns cut the gap to two points with two penalties before Shingler responded from two metres inside the Gloucester half after the visitors were penalised for pulling down line-out jumper Bryn Evans.
Irish withdrew the struggling Homer in the 34th minute, replacing him with England Under-20 star Anthony Watson.
Burns pulled his fourth penalty attempt wide four minutes later but landed a conversion to send Gloucester in at half-time with a 16-14 lead after Morgan exploited some appalling defending by Irish to score his side's first try.
Taking the ball from the base of a ruck, he went between two hesitant would-be tacklers, was missed by hooker Scott Lawson and brushed off O'Leary before crossing.
Burns landed his fourth penalty five minutes into the second half after the Irish scrum was yet again penalised by referee Greg Garner.
Irish were level again two minutes later when Gloucester's defence cracked, allowing number eight John Fisher to run through before offloading to Aulika, who forced his way over for his first Exiles try since arriving from Championship side Leeds during the summer. Shingler converted to put the Exiles 21-19 ahead.
But Gloucester edged a point in front with Burns' fifth penalty from just inside the Irish half as the home scrum, forced into rapid reverse, was penalised once more.
Irish finally got their scrum right and it led to O'Leary shooting in for their second try after the diving Charlie Sharples conceded the five-metre set-piece as he slid over the line desperately defending a through kick from Jonathan Joseph. Shingler again converted.
Burns kicked Gloucester level again after Irish offended at the breakdown.
Shingler booted Irish back in front before Burns squeezed down the left touchline for Gloucester before passing inside to present Simpson-Daniel with what proved to be the decisive 72nd-minute try, converted by the fly-half.
Gloucester struck again two minutes later when Qera forced his way in from close range.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The Barbarians are one of the top invitational rugby sides with a long history and classic rugby jersey.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Lions get ready for their matches with this green training jersey. It's what the players wear. Get in the Gear!
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
The Lions are ready to get their Australia tour underway. They arrive in Perth on Monday.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.