Each coin is justified only as long as there is something more or less frequently bought for the price of that nominal. If cheapest thing in your average grocery store cost a quarter, everything less is simply redundant. Price of producing that penny is somewhat irrelevant as long as reprocessing that penny is not worth it. Gold had that problem (You can smelt it anywhere and it's somewhat hard to spoil it unless you are really determined to do that). All in all, it's not used only once.
The one-cent coin, known colloquially as a "penny", is a unit of currency that is an equivalent to 1⁄100 of the United States dollar.
It is simple math. The penny costs more to make than it is worth in currency exchange. This makes no sense economically. To keep minting them for sentimental reasons is illogical. Other countries have discontinued minting them without serious consequence. It is simply time for us to move on.
If the issue is the cost of the penny, then make it out of something cheaper. As long as we have sales taxes, we will need the penny. In my state, the sales tax is 6%. If I buy something that costs a dollar, How am I to come up with $1.06 without the penny? And I for one do not want to see the sales tax go up in 5% increments each time simply because we don't have the pennies that would allow us to do otherwise. It's also foolish to use a debit or credit card for small transactions because of the transaction fees involved. Those fees are absorbed either by the store or by the financial institution that issued the card, and it drives prices up indirectly. So make pennies out of beer can aluminum or plastic if you have to, but don't get rid of them.