While he was carrying mail (from 1952-55, as
Purple Stickpin said), he wrote very little. While he worked as a mail sorter (from 1958-70), he wrote very much. But mainly poems. In 1969 he published a book of short stories,
Notes of a Dirty Old Man, and started writing a weekly alternative newspaper column, also called
Notes of a Dirty Old Man.
He didn't finish his first novel until he quit the post office. But that doesn't mean he wasn't writing, he was writing a lot. And a lot of it was
notable. But again, mainly poems.
So to answer to your question of whether working in the post office slowed down his writing, no, it did not. He wrote much more after leaving the post office, but you'd expect that since he had 9 or 10 more hours a day to write.