PITTSBURGH (AP) — The Cincinnati Reds were hurt by balks early and late in their loss to the Pittsburgh Pirates.

Starting pitcher Homer Bailey balked to bring in the Pirates’ first run, and Tony Cingrani’s infraction led to the winning run in Pittsburgh’s 4-3 victory in 12 innings on Thursday.

Bailey was perfect through three innings before giving up a hit to Gregory Polanco in the fourth. Bailey then allowed three runs in the fifth.

With runners at the corners, and Polanco batting, Bailey went through his motion but didn’t release the ball. He was called for the balk that cut the Reds’ lead to 2-1.

“It was one of those deals where you’re thinking more about not falling than throwing the ball,” Bailey said. “My back foot slipped out from me, then I land on the side of my other foot, so it was a circus all the way around. I was just trying not to fall backward.”

Polanco, Starling Marte and Andrew McCutchen followed with singles that scored two runs for a 3-2 Pittsburgh lead.

In the 12th inning, Cingrani was called for a balk that moved Gaby Sanchez to second base after his one-out single. Cingrani then picked off Sanchez, but the out was negated by the balk.

Cingrani disagreed with the call.

“I’ve done that move a million times, and they just pick and choose when they call it,” he said.

Cingrani (2-8) then intentionally walked Josh Harrison and hit Clint Barmes. With two outs and the bases loaded, Cingrani walked Russell Martin to force in the winning run with a pitch that was over the plate but ruled low.

“I’m not saying we would have won this game, but it certainly didn’t give us an opportunity to go out there and compete in the 13th,” Reds manager Bryan Price said.

Martin admitted that the pitch might have been missed.

“I think we got a fortunate call on that 3-1 count, but it’s nice to have a call go our way for once,” he said.

Billy Hamilton led off the game with a double against Jeff Locke, stole third, and scored on a sacrifice fly by Joey Votto to put the Reds ahead 1-0. Votto doubled and scored on a sacrifice fly by Jay Bruce to make it 2-0 in the fourth.

Locke settled down though and worked six innings. He gave up two runs, six hits and a walk.

Price pulled Bailey in the sixth, and as reliever Sam LeCure took his warm-up pitches, heavy rain began to fall. That caused a 35-minute delay.

After Pittsburgh took the lead, Devin Mesoraco sent the game to extra innings with his 10th home run of the season off Jason Grilli.

Grilli gave up a home run for the second time in three days, and has allowed four in 20 games this year. He gave up four homers in 54 appearances last season.

The Pirates twice put runners in scoring position after Mesoraco’s tying homer.

McCutchen struck out looking at an offspeed pitch from Aroldis Chapman in the ninth with runners at first and second. Marte struck out swinging in the 11th in the same situation.

Justin Wilson (2-0) tossed a perfect 12th inning.

The Reds won the series from the Pirates, but Price didn’t head home happily.

“I’m really, really angry with what happened here,” Price said. “I’m not at all happy with this — 4-2, 5-1 would have been a lot better.”

NOTES: Polanco extended his hitting streak to nine games, the longest by a Pirates player to begin a major league career. … Barmes had his first four-hit game since May 30, 2009. … RHP Charlie Morton (4-7, 3.09) opposes Cubs RHP Edwin Jackson (4-7, 5.11) on Friday as the Pirates open a three-game series at Chicago. … RHP Mat Latos (0-0, 0.00) is slated to make his second start of the season when the Reds return home to face RHP R.A. Dickey (6-5, 4.08) and the Toronto Blue Jays.