In a frenetic, error-strewn encounter in Toulouse, tries from hooker Brice Mach and replacement winger Pierre Gilles Lakafia, either side of a Doug Howlett try for the visitors, gave Castres an 18-10 half-time lead.
Peter O'Mahony's early try in the second half gave Munster hope, however, and Will Chambers put them in front 14 minutes from time after O'Gara charged down Remi Tales' attempted clearance kick.
Pierre Bernard levelled things with a penalty only for O'Gara, just as he did against Northampton, to snatch a narrow victory with the last kick of the game.
Castres had moved the game to the Stade Ernest Wallon 70 kilometres away in Toulouse in an effort to capitalise on the increased capacity it affords.
It was the home side who started brightest, with Bernard's strike from the left touchline after four minutes putting them 3-0 ahead. The Castres number 10 then landed another wind-assisted penalty from inside his own half just two minutes later.
The French outfit, much-changed from their opening loss at the Scarlets, suffered a blow when pacy international winger Marc Andreu was carried off and replaced by Lakafia but it had little effect on Castres' momentum.
Their impressive captain Chris Masoe burst his way into the Munster 22 and when the ball was quickly moved from left to right, Mach held off two Munster defenders to cross in the right corner.
Mach was injured in scoring the try and replaced by Mathieu Bonello and while Bernard's conversion attempt came off both posts, Castres held a commanding 11-0 lead after just 13 minutes.
Munster responded strongly but O'Gara will have been disappointed to hook a drop goal attempt wide from well inside the 22.
Munster's forwards were starting to have a telling influence however and they were finally rewarded after 23 minutes. When the strong-running Wian du Preez was held up in front of the posts, some slick handling by scrum half Conor Murray and number eight James Coughlan allowed Howlett to score on the right.
O'Gara nailed the conversion from the right corner of the 22 to bring his side to within four points. The Heineken Cup's record points scorer missed a penalty before the visitors were dealt a hammer blow three minutes before the interval.
Hesitation and an unfavourable bounce left Murray in trouble in the left corner and then Paul O'Connell coughed up possession as Munster tried to run their way out of difficulty.
Castres took full advantage, sweeping the ball swiftly across the 22 and Lakafia sidestepped Howlett's despairing challenge to touch down in the right corner. Bernard added the extras and Castres led 18-7, before O'Gara scored a penalty with the last kick of the half after a succession of five-metre scrums.
That penalty gave Munster a timely jolt and they drew to within one point five minutes into the second half. Munster had troubled Castres during the opening 40 minutes when they moved the ball at pace.
A wraparound move between Murray and O'Connell saw the pair combine beautifully to find Niall Ronan and he fed the influential O'Mahony who showed a considerable burst of pace himself to slalom through a huge gap in the home defence.
Bernard responded with a penalty to stretch Castres' advantage to 21-17 but the hosts handed Munster a try just moments later. As Tales tried to clear on his own line, O'Gara charged his kick down and Denis Leamy fed fellow replacement Chambers for an easy score.
O'Gara converted as Munster found themselves in front for the first time but Bernard levelled things with 10 minutes left, setting up a thrilling climax.
Unsurprisingly, it was two-time champions Munster who held their nerve when it mattered most.
O'Connell and company battered Castres for the final few minutes, and Tomas O'Leary's pinpoint pass gave O'Gara just enough time to rifle over the match-winning drop goal from the 10-metre line.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The Barbarians are one of the top invitational rugby sides with a long history and classic rugby jersey.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Lions get ready for their matches with this green training jersey. It's what the players wear. Get in the Gear!
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
The Lions are ready to get their Australia tour underway. They arrive in Perth on Monday.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.